Workflow OpenMPI

Code execution

When using OpenMPI, binding is controlled using the –-bind-to parameter. To bind processes to cores, use –bind-to core. Possible other values can be found in the man page.

mpirun --bind-to core ./yourprogram

Our hardware supports hyperthreading, allowing you start 192 processes on Cascade Lake machines (*96 partitions) and 80 on Skylake machines.

If no specific requests regarding the number of tasks has been done, mpirun defaults to hyperthreading and starts cores*2 processes. If a number of tasks has been specified (with -N and/or –tasks-per-node), an (Open) mpirun honors this via the flag -map-by. For example:

#!/bin/bash
#SBATCH -N 4
 
module load gcc/9.3.0 openmpi/gcc.9/4.1.4
 
cpus_per_node=${SLURM_JOB_CPUS_PER_NODE%\(*}
 
mpirun -map-by ppr:$cpus_per_node:node ./yourexe

#!/bin/bash
#SBATCH -N 4
#SBATCH --tasks-per-node 96
 
module load gcc/9.3.0 openmpi/gcc.9/4.1.4
 
tasks_per_node=${SLURM_TASKS_PER_NODE%\(*}
 
mpirun -map-by ppr:$tasks_per_node:node ./yourexe