Support &
Documentation
To set up your environment for using a particular (set of) software package(s), you can use the modules that are provided centrally.
Interacting with the modules is done via Lmod, using the module command or the handy shortcut command ml.
The modules are installed in a hierarchial layout. This means that some modules are only available after loading a specific compiler and/or MPI version. See our support page on "Accessing software with Lmod" for information on layout.
A very quick introduction to Lmod. Below you will find more details and examples.
ml
lists the currently loaded modules, and is equivalent with module list
ml spider gromacs
searches (case-insensitive) for gromacs in the whole module hierarchyml spider GCC/6.2.0-2.27
show information about the module GCC/6.2.0-2.27for a
ml GCC/6.2.0-2.27
loads the GCC/6.2.0-2.27
module, and is equivalent with module load GCC/6.2.0-2.27
ml -GCC
unloads the currently loaded GCC
module, and is equivalent with module unload GCC
ml av gcc
prints the currently available modules that match gcc (case-insensitively), and is equivalent with module avail GCC
ml show GCC/6.2.0-2.27
prints more information about the GCC/6.2.0-2.27
module, and is equivalent with module show GCC/6.2.0-2.27
ml save mycollection
stores the currently loaded modules to a collectionml restore mycollection
restores a previously stored collection of modulesmodule list
(or ml
)To get an overview of the currently loaded modules, use module list
or ml
(without specifying extra arguments).
In a default environment, you should see the snicenvironment and systemdefault modules loaded:
$ ml Currently Loaded Modules: 1) systemdefault (S) 2) snicenvironment (S) Where: S: Module is Sticky, requires --force to unload or purge
(for more details on sticky modules, see the section on ml purge
)
The (Lmod-specific) spider
subcommand lets you search for modules through the whole module hierarchy.
If you just provide a software name, for example gromacs
, it prints an overview of all available modules for GROMACS.
$ ml spider gromacs ---------------------------------------------------------------------------------- GROMACS: ---------------------------------------------------------------------------------- Description: GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. - Homepage: http://www.gromacs.org Versions: GROMACS/2016-hybrid GROMACS/2016-mt ---------------------------------------------------------------------------------- For detailed information about a specific "GROMACS" module (including how to load the modules) use the module's full name. For example: $ module spider GROMACS/2016-mt ----------------------------------------------------------------------------------
Note that spider search is case-insensitive.
If you use spider
on a full module name (in this case spider is case sensitive) like GROMACS/2016-mt it will tell you what you need to load to access it:
$ ml spider GROMACS/2016-mt ---------------------------------------------------------------------------------- GROMACS: GROMACS/2016-mt ---------------------------------------------------------------------------------- Description: GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. - Homepage: http://www.gromacs.org You will need to load all module(s) on any one of the lines below before the "GROMACS/2016-mt" module is available to load. GCC/6.2.0-2.27 OpenMPI/2.0.1 Help: GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. - Homepage: http://www.gromacs.org
This tells you that the module named GROMACS/2016-mt is available after first loading the GCC/6.2.0-2.27 and OpenMPI/2.0.1 modules (in that order). It also tells you what the module contains and a URL to the homepage of the software.
Using just ml spider without arguments, gives you a complete list of modules in the hierarchy.
To get an overview of all directly accessible modules, you can use module avail or simply ml av:
--------------------------- /hpc2n/eb/modules/all/Core --------------------------- Autoconf/2.69 foss/2016b Automake/1.15 foss/2016.09 (D) Autotools/20150215 gettext/0.19.8 EasyBuild/2.9.0 gompi/2016b GCC/4.9.3-binutils-2.25 gompi/2016.09 (D) GCC/5.4.0-2.26 icc/2017.0.098-GCC-5.4.0-2.26 GCC/6.2.0-2.27 (D) iccifort/2017.0.098-GCC-5.4.0-2.26 ... ...
In the current module naming scheme, each module name consists of two parts:
/
, corresponding to the software name; andFor example, the module name iccifort/2017.0.098-GCC-5.4.0-2.26
would set up the environment for using icc and ifort version 2017.0.098, with the GCC/5.4.0-2.26
compiler toolchain.
The (D)
indicates that this particular version of the module is the default, but we strongly recommend to not rely on this as the default can change at any point. Usuall, the default will point to the latest version available.
To check which versions of a particular software package that are available, you can provide the software name to ml av
.
For example, to check which versions of gompi are available:
$ ml av gompi --------------------------- /hpc2n/eb/modules/all/Core --------------------------- gompi/2016b gompi/2016.09 (D)
Note that the specified software name is treated case-insensitively.
Lmod does a partial match on the module name, so sometimes you need to use /
to indicate the end of the software name you are interested in:
$ ml av gcc/ --------------------------- /hpc2n/eb/modules/all/Core --------------------------- GCC/4.9.3-binutils-2.25 GCC/5.4.0-2.26 GCC/6.2.0-2.27 (D)
To see how a module would change the environment, use module show
or ml show
:
$ ml show foss/2016b ---------------------------------------------------------------------------------- /hpc2n/eb/modules/all/Core/foss/2016b.lua: ---------------------------------------------------------------------------------- help([[GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK. - Homepage: (none)]]) whatis("Description: GNU Compiler Collection (GCC) based compiler toolchain, including OpenMPI for MPI support, OpenBLAS (BLAS and LAPACK support), FFTW and ScaLAPACK. - Homepage: (none)") conflict("foss") load("GCC/5.4.0-2.26") load("OpenMPI/1.10.3") load("OpenBLAS/0.2.18-LAPACK-3.6.1") load("FFTW/3.3.4") load("ScaLAPACK/2.0.2-OpenBLAS-0.2.18-LAPACK-3.6.1") setenv("EBROOTFOSS","/hpc2n/eb/software/Core/foss/2016b") setenv("EBVERSIONFOSS","2016b") setenv("EBDEVELFOSS","/hpc2n/eb/software/Core/foss/2016b/easybuild/Core-foss-2016b-easybuild-devel")
Note that both the direct changes to the environment as well as other modules that will be loaded are shown.
If you're not sure what all of this means: don't worry, you don't have to know; just try loading the module and try using the software.
To effectively apply the changes to the environment that are specified by a module, use module load
or ml
and specify the name of the module.
For example, to set up your environment to use foss:
$ ml foss/2016b $ ml Currently Loaded Modules: 1) snicenvironment 7) OpenMPI/1.10.3 2) systemdefault 8) OpenBLAS/0.2.18-LAPACK-3.6.1 3) GCCcore/5.4.0 9) FFTW/3.3.4 4) GCC/5.4.0-2.26 10) ScaLAPACK/2.0.2-OpenBLAS-0.2.18-LAPACK-3.6.1 5) numactl/2.0.11 11) foss/2016b 6) hwloc/1.11.3
Note that even though we only loaded a single module, the output of ml
shows that a whole bunch of modules were loaded, which are required dependencies for foss.
It is important to note that only modules that are compatible with each other can be loaded together. In particular, modules must be installed either with the same toolchain as the modules that are already loaded, or with a compatible (sub)toolchain.
For example, once you have loaded one or more modules that were installed with the intel/2017.01
toolchain, all other modules that you load should have been installed with the same toolchain. Due to the nature of the hierarchial module naming system, only modules built with that same toolchain are actually visible, thus reducing the probablility of making a mistake.
In addition, only one single version of each software package can be loaded at a particular time. For example, once you have loaded the module Python/2.7.12
, you can not load a different version of Python in the same session/job script; neither directly, nor indirectly as a dependency of another module you want to load.
See also the topic "module conflicts" in the list of key differences with the previously used module system.
To revert the changes to the environment that were made by a particular module, you can use module unload
or ml -<modname>
.
For example:
To revert the changes to the environment that were made by a particular module, you can use module unload
or ml -<modname>
.
For example:
$ ml Currently Loaded Modules: 1) snicenvironment (S) 2) systemdefault (S) Where: S: Module is Sticky, requires --force to unload or purge $ which gcc /usr/bin/gcc $ ml GCC/6.2.0-2.27 $ ml Currently Loaded Modules: 1) snicenvironment (S) 2) systemdefault (S) 3) GCCcore/6.2.0 4) GCC/6.2.0-2.27 Where: S: Module is Sticky, requires --force to unload or purge $ which gcc /hpc2n/eb/software/Core/GCCcore/6.2.0/bin/gcc $ ml -GCC/6.2.0-2.27 $ ml Currently Loaded Modules: 1) snicenvironment (S) 2) systemdefault (S) Where: S: Module is Sticky, requires --force to unload or purge $ which gcc /usr/bin/gcc
To reset your environment back to a clean state, you can use module purge
or ml purge
:
$ ml Currently Loaded Modules: 1) snicenvironment (S) 7) OpenMPI/2.0.1 2) systemdefault (S) 8) OpenBLAS/0.2.19-LAPACK-3.6.1 3) GCCcore/6.2.0 9) FFTW/3.3.5 4) GCC/6.2.0-2.27 10) ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1 5) numactl/2.0.11 11) GROMACS/2016-hybrid 6) hwloc/1.11.4 Where: S: Module is Sticky, requires --force to unload or purge $ ml purge The following modules were not unloaded: (Use "module --force purge" to unload all): 1) systemdefault 2) snicenvironment $ ml Currently Loaded Modules: 1) systemdefault (S) 2) snicenvironment (S) Where: S: Module is Sticky, requires --force to unload or purge
Note that the snicenvironment
and systemdefault
modules will always remain loaded, since it defines some important environment variables.
As such, you should not (re)load the snicenvironment
or systemdefault module anymore after running ml purge
. See also the topic on the purge command in the list of key differences with the previously used module implementation.
If you have a set of modules that you need to load often, you can save these in a collection (only works with Lmod).
First, load all the modules you need, for example:
$ ml GCC/6.2.0-2.27 OpenMPI/2.0.1 GROMACS/2016-hybrid
Now store them in a collection using ml save
:
$ ml save my-collection Saved current collection of modules to: my-collection
Later, for example in a job script, you can reload all these modules with ml restore
:
$ ml restore my-collection Restoring modules to user's my-collection
With ml savelist
you can get a list of all saved collections:
$ ml savelist Named collection list : 1) my-collection
To inspect a collection, use ml describe
.
To remove a module collection, remove the corresponding entry in $HOME/.lmod.d
.
In October 2016, we switched to Lmod as a modules tool, a modern alternative to the outdated & no longer actively maintained Tcl-based environment modules tool.
Consult the Lmod documentation web site for more information.
module avail
avail
, the ml
command, etc.)The switch to Lmod (or more specifically the use of EasyBuild for building and installing software) means that we have a completely different module naming system. You need to make changes to you submit files to take this into account.
There are a couple of minor differences between Lmod & the old modules tool you should be aware of:
module purge
does not unload the snicenvironment and systemdefault
modulesSee below for more detailed information.
Conflicting modules can no longer be loaded together.
Lmod has been configured to unload the conflicting module and replace it with the one being loaded.
Although it seemed like the old modules system did allow for conflicting modules to be loaded together, this was highly discouraged already since it usually resulted in a broken environment. Lmod will ensure that the environment stays consistent.
If you do try to load conflicting modules, you will see something similar to this:
$ ml foss/2016.09 $ ml list Currently Loaded Modules: 1) systemdefault (S) 7) OpenMPI/2.0.1 2) snicenvironment (S) 8) OpenBLAS/0.2.19-LAPACK-3.6.1 3) GCCcore/6.2.0 9) FFTW/3.3.5 4) GCC/6.2.0-2.27 10) ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1 5) numactl/2.0.11 11) foss/2016.09 6) hwloc/1.11.4 Where: S: Module is Sticky, requires --force to unload or purge $ ml GCC/5.4.0-2.26 Inactive Modules: 1) FFTW/3.3.5 4) ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1 2) OpenBLAS/0.2.19-LAPACK-3.6.1 5) hwloc/1.11.4 3) OpenMPI/2.0.1 Due to MODULEPATH changes the following have been reloaded: 1) numactl/2.0.11 The following have been reloaded with a version change: 1) GCC/6.2.0-2.27 => GCC/5.4.0-2.26 2) GCCcore/6.2.0 => GCCcore/5.4.0 $ ml Currently Loaded Modules: 1) systemdefault (S) 3) foss/2016.09 5) GCC/5.4.0-2.26 2) snicenvironment (S) 4) GCCcore/5.4.0 6) numactl/2.0.11 Where: S: Module is Sticky, requires --force to unload or purge Inactive Modules: 1) FFTW/3.3.5 4) ScaLAPACK/2.0.2-OpenBLAS-0.2.19-LAPACK-3.6.1 2) OpenBLAS/0.2.19-LAPACK-3.6.1 5) hwloc/1.11.4 3) OpenMPI/2.0.1
Using module purge
effectively resets your environment to a pristine working state, i.e. the snicenvironment module stays loaded after the purge
.
As such, it is no longer required to run module load snicenvironment
to restore your environment to a working state.
In case of questions or problems, please do not hesitate to contact the HPC2N support team via support@hpc2n.umu.se.
(This text is in large parts copied from https://www.vscentrum.be/cluster-doc/software/modules-lmod with their consent)