Deep Dive Into Containers

Content

Containers are increasingly used in HPC as they allow users to make their codes and programs they need to use more independent of the exact Linux distro, configuration, software, and software versions residing on a cluster, and not have competing software environments in their HOME directory fighting each other. Containers, their place in HPC, and their creation and use in a HPC context, specifically on the GWDG SCC and GWDG NHR clusters, will be described in detail. Furthermore, the GWDG container database and how to use it will be described, followed by the integration of containers with GWDG GitLab (making containers, tests, storage, etc.).

Requirements

• Using the GWDG SCC and/or GWDG NHR clusters, including Slurm job submission
• GWDG account for GWDG Gitlab
• Either a GWDG account activated for HPC on the SCC, a GWDG NHR account, or a course account for GWDG SCC or GWDG NHR
• Practical experience with the Linux command line, Linux shell scripting, and git.

Learning goal

• Understand when to use containers for HPC
• Be able to convert a local workflow on workstation/laptop into a container suitable for HPC systems
• Be able to run containers on the GWDG HPC systems including getting data in and out of containers, getting MPI and CUDA to work, etc.
• Use the GWDG container database
• Use the GWDG GitLab for building containers, using containers for testing, and storing containers

Skills

Trainer

• Dr. Freja Nordsiek

Next appointment