Creating private software modules

How can I create private software modules?

Software modules are used to manage the software environment on Linux systems, by allowing users to load and unload particular software versions. Loading a software module sets environmental variables in order to make a particular piece of software accessible. Many pieces of software have been installed and made available on MSI systems. To view the software modules available on a Linux system, use the command: module avail

Users can create private software modules, which allow them to load and unload software which has been installed within their home or group directories. The process for creating private software modules is described below.

Supercomputing FAQs
  1. First, choose a directory that will hold the private software module files. If the modules will be shared with other group members, place them in a directory for which the group members have read-access. If the modules will be used by a single user, the module files may be placed in a subdirectory of the user's home directory. For example, the directory ~/modulefiles may be used to hold a user's private software module files.
  2. Prepend the module files directory to the MODULEPATH environmental variable using the command:
    export MODULEPATH=~/modulefiles:$MODULEPATH
    If you wish the private modules to be automatically available after every login, this command can be placed in the ~/.bashrc file.
  3. In the directory that holds the software module files, create a subdirectory with the name of the software package. For example, create the directory ~/modulefiles/MyProgram .
  4. Within the subdirectory named after the software, create a text file named after the software version. For example, create the text file ~/modulefiles/MyProgram/1.0 . This file will be the software module file for MyProgram/1.0.
  5. Within the newly created text file, place the appropriate environmental alteration commands for making the software accessible. The language used in specifying these commands is the Tool Command Language (TCL). An example of a software module file is shown below. 

    #%Module
    module load intel/2016/update3
    set basedir ~/software/MyProgram
    prepend-path PATH $basedir/bin
    prepend-path LD_LIBRARY_PATH $basedir/lib
    prepend-path CPATH $basedir/include
    setenv OMP_NUM_THREADS 1


    Module files should begin with #%Module in order for the system to recognize them as module files. Modules may load other modules, and in this example the module file loads the intel/2016/update3 module. The example module sets an internal variable named basedir to store the location of the directory containing the MyProgram installation. The prepend-path commands make the module add the referenced directories to the beginning of environmental path variables. For example, this module will prepend ~/software/MyProgram/bin to the PATH environmental variable, which will cause the system to search that directory for program files after the module is loaded. Similarly, the module adds library and include directories to appropriate environmental variables, to allow the system to locate library and header files. Finally, the module sets the environmental variable OMP_NUM_THREADS to the value of 1.

 

After these steps are taken, and a new module file has been created, the new private software module can be viewed and loaded. The module should be visible using the command module avail MyProgram , and can be loaded using the command module load MyProgram/1.0 . The module can now be loaded and unloaded just like any other module. Unloading a module reverses the actions that the module performs when loaded. To view the actions the module takes when loaded, use the command: module show MyProgram/1.0 .

The environmental alterations that a module needs to make are different for every piece of software. Some common alterations are prepending program directories to the PATH environmental variable, and library directories to the LD_LIBRARY_PATH environmental variable. Modules intended to be used when compiling often also need to have header include directories prepended to variables such as CPATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, FPATH, or INCLUDE, depending on the compiler and language. Other variables such as PYTHONPATH or R_LIBS can be altered to point to local Python packages or R libraries, respectively. The variable OMP_NUM_THREADS is commonly used to control how many threads an OpenMP program will spawn


Warnings: 

It is recommended to avoid creating modules with the same names as already-existing modules, as this can cause system confusion.

Some alterations to environmental variables can cause instability. In particular, altering LD_LIBRARY_PATH to point to directories containing dynamic libraries which conflict with operating system dynamic libraries can cause instability. In general, different versions of core libraries such as the C library (GLIBC), cannot be used without system instability.


 

Discover Advanced Computing and Data Solutions at MSI

Our Services