Module Basics

To make the provided software more manageable, the software is provided as modules in a module system. In many ways, it works like a package version control system letting multiple packages with different versions co-exist in non-default places letting the user and scripts select the exact ones they want to use at any given time. “loading a module” can be thought as synonymous for “making software accessable”.

The software stacks use two different module systems that are very compatible from the user side: Lmod and Environment Modules (Tmod). Both provide a module command that works nearly identically, with the main exceptions being the formatting they use to print information (e.g. with module avail) and how deep module trees work. Additionall, Lmod has some compatibility support for using modules written for Environment Modules (Tmod) without modification, making it easier to write your own modules (see Using Your Own Module Files for more information).

Getting Started

The most important commands to know are

module avail
module avail SEARCHTEXT
module help MODULE
module load MODULE/VERSION
module unload MODULE
module list

All modules have a version, specified as /VERSION. If the /VERSION part is omitted, the default version is used. So, one could do

module load gcc

to get the default GCC version, or

module load gcc/9.3.0

to get GCC 9.3.0.

Loading a module manipulates the shell environment to make the software visible for shell commands. Note, these changes of environmental variables are fully reversible and are undone if a module is unloaded. Loading a module may prepend the path of the directory containing the package’s programs to environmental variable PATH. This makes the package’s executables visible for the shell or script. The modules do similar things for other environmental variables like LIBRARY_PATH, MANPATH, etc. For example, if you want to work with GCC version 9, load the appropriate module and check its version.

module load gcc/9.3.0
gcc --version

The base operating system’s GCC is invisible to your shell as long as the module gcc is loaded, but becomes available again, if the module gcc is unloaded.

A list of all available modules arranged by compiler/MPI/cuda (SCC Modules (scc-lmod) and NHR Modules (nhr-lmod) software stacks) or category (HLRN Modules (hlrn-tmod) software stack) can be printed with the command module avail. A list of loaded modules is shown with the command module list. To learn the available commands and how to use them, you can run either of the following

module help
man module

When you get a user account, your environment is provided by the system profile and should be set up correctly to use modules. If you change your shell or modify your environment in such a way to make modules fail, contact the your consultant or start a support ticket for help. Additional documentation can be found at Lmod (SCC Modules (scc-lmod) and NHR Modules (nhr-lmod) software stacks) and Environment Modules (Tmod) (HLRN Modules (hlrn-tmod) software stack) for the respective module systems.

Module commands

Module commands can be issued

• On the command line. This modifies the current shell environment, which makes the package’s binaries visible and defines environment variables to support compiling and linking. These modification are reversible and undone when unloading a module.
• In Slurm batch scripts. This can be used to prepare the environment for jobs.

A complete list of all module commands can be seen by running module help. Below, a few examples are introduced.

Available Modules Overview

module avail

This command shows all modules available to be loaded at the present time. Notice the version numbers and that some are marked as default. The modules are arranged by compiler/MPI/cuda (SCC Modules (scc-lmod) and NHR Modules (nhr-lmod) software stacks) or category (HLRN Modules (hlrn-tmod) software stack).

You can also search for modules whose names contain STRING (Lmod) or begin with STRING (Environment Modules (Tmod)) by the command

module avail STRING

You can also search for modules containing STRING in their so-called whatis information by

module spider STRING

module apropos STRING

module apropos -i STRING

Load and Unload Modules

module load MODULE
module load MODULE/VERSION
module unload MODULE
module unload MODULE/VERSION
module switch MODULE_1/VERSION_1 MODULE_2/VERSION_2
module purge

The load and unload commands load and unload the specified module exactly as expected, while the module switch command switches one module for another (unload the first and load the second). The purge command unloads ALL modules. If you don’t include the /VERSION part, the commands operate on the default version. Loading a module manipulates the shell environment to make software visible for shell commands. Note, these changes of environmental variables are fully reversible and are undone if a module is unloaded. For example, loading a module could prepend the path to the directory containing its executable programs to the environmental variable PATH. To see all currently loaded modules, use

module list

Getting module information

To investigate and learn more about a particular module, use the commands

module help MODULE
module help MODULE/VERSION
module whatis MODULE
module whatis MODULE/VERSION
module show MODULE
module show MODULE/VERSION

If you don’t include the /VERSION part, the commands operate on the default version. The help command shows the help string for the module. The whatis command shows the whatis information for the module. The show command shows all environment modifications the module would make if loaded.

Compiling and linking installed libraries using modules

Many software packages depend on installed libraries like the MPI implementation, FFTW 3, BLAS and LAPACK, HDF5, NetCDF, etc. On a normal machine, those libraries and headers would reside at well defined places on the filesystemsystem, where they are found by the compiler and linker and also by the loader during runtime. As the module system can provide several versions of those libraries, so they must be installed at non-default places. So the compiler, linker and loader need help to find them, especially the correct version. The dream of software users is that loading a module is all to be done to deliver this help. Unfortunately, this is not the case for several reasons.

Compilers and linkers use some well defined environmental variables to serve or get information on libraries. Typical examples are PATH, LD_RUNPATH, LD_LIBRARY_PATH, LIBRARY_PATH or PKG_CONFIG_PATH. Many modules modify them so that their libraries and programs can be found. To see which environmental variables are set by a module, run

module show MODULE/VERSION

However, for building complex software, build-tools like autotools or cmake are used, which come with own rules for how programs and libraries are found. In turn, the netcdf library uses nc-config and nf-config to deliver the information on the paths to the netcdf headers and libraries. All serious build tools use this option and a netcdf module just needs to deliver the path (PATH) to these tools. Very specific is cmake that searches for libraries instead of requesting information in a well defined way. This does not fit well in the philosophy of modules. Hence, there may be cases where you must consult the documentation of the software you are trying to build and/or the build system to get it to use software from the modules.

Another issue may be that successfully linked libraries are not found when running the executable. One may be tempted to load the related module again. However, in most cases this does not help to overcome the problem, as many modules do not have any effect during runtime. Here the solution is to avoid overlinking and to burn the path to the libraries into the executable. Read more on linking HDF5 and NetCDF libraries with the help of the related modules.

Using Your Own Module Files

It is possible to write your own module files (say to include your own software or manage environmental variables in an easily reversed way) and include them in the module tool’s search path so they can be used just like the provided modules. The module command looks through the colon separated directories in the MODULEPATH environmental variable in order. The order of directories in MODULEPATH determines the order with which they are printed by module avail and can determine which version of a module is the default if nothing else chooses the default (tie breaker is the first version found), so appending or prepending a directory to MODULEPATH can lead to different results. If you have your modules in directory DIRECTORY, you can preppend it to the search path by

module use DIRECTORY

or append it with the -a option like

module use -a DIRECTORY

You can remove the directory with the unuse command like

module unuse DIRECTORY

module load use.own

For actually making the modules, you must write the modulefiles themselves in one of the languages supported by the module system used by the software stack and use its API. The module systems and their supported languages for each software stack are given in the table below, along with links to the documentation for the module systems. It is generally easier to use existing module files as template.