GPU Partitions

Nodes in these partitons provide GPUs for parallelizing calculations. See GPU Usage for more details on how to use GPU partitions, particularly those where GPUs are split into MiG slices.

Partitions

The partitions are listed in the table below by which users can use them, without hardware details. Note that some users are members of multiple classifications (e.g. all CIDBN users are also SCC users).

The hardware for the different nodes in each partition are listed in the table below. Note that some partitions are heterogeneous, having nodes with different hardware. Additionally, many nodes are in more than one partition.

*) The actually available memory per node is always less than installed in hardware. Some is reserved by the BIOS, and on top of that, Slurm reserves around 20 GiB for the operating system and background services. To be on the safe side, if you don’t reserve a full node, always deduct ~30 GiB, divide by the number of GPUs and round down to get a number per GPU you can safely request with --mem when submitting jobs.

How to pick the right partition for your job

If you have access to multiple partitions, it can be important to choose one that fits your use case. As a rule of thumb, if you need (or can scale your job to run on) mutliple GPUs, and you are not using a shared partition, make sure to always use a multiple of 4 nodes, as you will be billed for the whole node regardless of a lower number of GPUs you requested via -G. For jobs that need less than 4 GPUs, use a shared partition and make sure to not request more than your fair share of RAM (see the note above). If you need to get your to start quickly, i.e. for testing if your scripts work or interactive tweaking of hyperparameters, use an interactive partition (most users have access to grete:interactive).

The CPUs and GPUs

For partitions that have heterogeneous hardware, you can give Slurm options to request the particular hardware you want. For CPUs, you can specify the kind of CPU you want by passing a -C/--constraint option to slurm to get the CPUs you want. For GPUs, you can specify the name of the GPU when you pass the -G/--gpus option (or --gpus-per-task) and larger VRAM using a -C/--constraint option. See Slurm and GPU Usage for more information.

The GPUs, the options to request them, and some of their properties are given in the table below.

The CPUs, the options to request them, and some of their properties are give in the table below.

Hardware Totals

The total nodes, cores, GPUs, RAM, and VRAM for each cluster and sub-cluster are given in the table below.