Modules (Lmod)

Environment Modules are designed to provide a flexible development environment. Some of the CAC's Private Clusters are managed with OpenHPC, which includes the use of modules via Lmod. Lmod is an environment module system developed by TACC which provides you with a very easy way to configure your environment for compiling, linking, and running applications using a variety of different compilers and a variety of external libraries.

Some of the information on this page has been adapted from the Cornell Virtual Workshop topic on Frontera, specifically the section on Modules.

Basic Usage

Generally, the module utility allows you to configure your environment to prioritize the compiler you want, the libraries you need to link against, and any runtime dependencies you may have. It also helps you to run selected pre-built applications in the correct, fully-configured environment.

Some Commands

  • module with no options set will get you a listing of the available options for module.
  • module avail to find out what's available via the module system. This displays all available modules that you can load.
  • module list to list all of the current modules that you have loaded.
    • It's good to know what you already have loaded in your environment, as some of these modules may conflict with each other.
    • The caveat here is that this doesn't tell you anything about environment settings that have been changed outside of the module command. So if you have attempted to manually configure or add a different compiler, it won't show up in this list. The moral of this is to be aware of potential conflicts when you manually add paths or introduce environment settings in other shell configuration files.
  • module spider - list all modules, even those not available to load
  • module load <module> add/load a module
  • module unload <module> unload a module

Finding Modules

In many cases, finding the module you need might be easily accomplished by using module avail or module spider, but let's consider a more complicated example. Say you'd like to run an R code that requires snow, an R package. Naturally, you would try module avail snow and/or module spider snow first, but both would fail to find it. However, it could still be there, included in another module.

If you try looking at R with module avail R, you will see the output arranged in groups, and that each group is ordered alphabetically so that uppercase letters precede lowercase ones. For example:

Rstats/3.4.0 RstatsPackages/3.4.0

Since snow is an R package, you can learn more about the which R packages are available with:

$ module spider rstatspackages

RstatsPackages:

 Versions:
    RstatsPackages/3.4.0
    RstatsPackages/3.5.1

For detailed information about a specific "RstatsPackages" module (including how to load the modules) use the module's full name. For example:

$ module spider RstatsPackages/3.4.0

  ----------------------------------------------------------------------------
    RstatsPackages: RstatsPackages/3.4.0
  ----------------------------------------------------------------------------

  You will need to load all module(s) on any one of the lines below before the "RstatsPackages/3.4.0" module is available to load.

    intel/17.0.4  impi/17.0.3

  Help:
    This is the R statistics (RstatsPackages) package built on September 28, 2018.

    It includes the following accessory packages including but not limited to:
    Rmpi, snow, snowfall
    pdbMPI, pbdSLAP, pbdBASE,

And you can see in the above list that in this example we've found at least one module to load that includes snow, along with additional modules that are required. The additional modules turn out to be the defaults for the Intel compiler and Intel MPI.

Running the command module spider RstatsPackages/3.5.1 shows that it too contains snow. The reason we didn't see the 3.5.1 version initially, with module avail R, is because it depends on later versions of the Intel compiler and MPI. Once the appropriate modules are loaded, the 3.5.1 version of RstatsPackages becomes available. Therefore, prior to loading RstatsPackages/3.5.1, we would want to use module spider to find a valid pair of modules for the compiler and MPI, then make sure those modules are loaded.

Module Information

Before proceeding to load a module, you want to be sure it's the right module and understand what it would do to the shell environment. The module tool itself provides a lot of information about each module, so that you know what the module does when you load it.

    module help <modulefile> shows module-specific information such as the environment variables defined, packages installed, and software version. Depending on the complexity of the module the information in this text will vary.
    module whatis <modulefile> displays just a few standard lines of information about the module, as provided by the "whatis" text inside the specified modulefile.
    module display <modulefile> dumps the full contents of modulefile, including the "help" and "whatis" text, and all of the environment changes modulefile will make if loaded.

Loading Modules

As mentioned before, when you load any new modules, it's best to see what you already have in your module list and in our path (via echo $PATH. Checking with these two commands before and after module load can reveal what dependent modules were loaded and if/how your path was modified. In many situations, Lmod is able to detect and adjust dependencies among the modules automatically. One really nice part about the module system is that if you decide to switch to a different set of software packages, module unload is able to undo the changes that the corresponding load did to your environment. This allows you to shift cleanly from one set of packages to another.

A couple of caveats apply here. First, module supports multiple versions of the same software, so you'll have to be specific if you want a particular version of software. Thus, if you had a special desire to load PETSc version 3.7 when 3.9 was the default, you'd need to issue module load petsc/3.7. Second, you should recognize that in some cases, module allows you to load two or more versions of the same module at the same time. In such cases, the version that you actually use is the last module loaded (due to the prepend-path directive), which could lead to some confusion.

The best advice is to issue the list subcommand once you think you're all set. This will let you check that you have the appropriate modules loaded. If you do have mistakes, then use the unload subcommand to remove the offending package(s). Be aware that unload will fail silently if you mistype a package name, so you should type cautiously or check your results after making changes. If things go totally wrong, you can start over like this: 

$ module purge
$ module reset

You have now reached the point where you should be able to use the module command to configure your environment using any of the available modules.

One Final Note: the load and unload subcommands are case-sensitive, so type carefully!

Personal Modulefiles

If you are a user of one of the CAC's Private Clusters, there are several reasons you may find it beneficial to use your own modulefiles so you can define your development environment. For example, if you would like to use a newer version of software than is currently available on the cluster, you can install the software to your home directory and create a modulefile for it. If you decide you would like to do this, we recommend reviewing the Lmod Documentation on the following topics:

References