Modules

From HPC

(Difference between revisions)
Jump to: navigation, search
(New page: =Modules= Modules are a tool to allow you to change the working enviroment to support various application and version of those applications. It allows use for example to support three ve...)
(→Show available modules)
 
(8 intermediate revisions not shown)
Line 1: Line 1:
-
=Modules=
+
=Environment Modules (module)=
-
Modules are a tool to allow you to change the working enviroment to support various application and version of those applications.
+
Modules are a tool to allow you to change the working environment to support various application and version of those applications.
It allows use for example to support three versions of java and 32bit and 64 version of each. This equates to 6 different versions, using modules you can load the one you want to use or unload the ones you don't.
It allows use for example to support three versions of java and 32bit and 64 version of each. This equates to 6 different versions, using modules you can load the one you want to use or unload the ones you don't.
 +
 +
 +
http://modules.sourceforge.net/
 +
 +
https://en.wikipedia.org/wiki/Environment_Modules_(software)
=Commands=
=Commands=
Line 17: Line 22:
Example
Example
-
  > module list
+
  # module list
  Currently Loaded Modulefiles:
  Currently Loaded Modulefiles:
-
   1) openmpi/1.2.6-1/intel      3) mkl/10/em64t
+
   1) R/3.6.0   2) samtools/1.9   3) mrbayes/3.2.7a
-
   2) intel/compiler101_x86_64   4) local_libs
+
-
>
+
==Show available modules==
==Show available modules==
Line 27: Line 30:
  module avail
  module avail
-
This would display something similar to this (Note: not all these are licensed, please see [[Available Software]] for a list)
+
This would display something similar to this  
-
  ------------------------------ /usr/share/modules ------------------------------
+
 
-
3.1.6                                modulefiles/mkl/10.0.3.020/64
+
# module avail
-
modulefiles/atlas                    modulefiles/mkl/10.0.3.020/em64t
+
  -------------------------------  /usr/local/modulefiles --------------------------------------------------------------------------------------------------  
-
modulefiles/dot                      modulefiles/module-cvs
+
  bcftools/1.9 beast/1.10.4 dot  gcc/6.5.0 gcc/7.4.0 gdal/2.3.3  module-git  module-info  modules  mrbayes/3.2.7a null R/3.4.1  R/3.4.4 R/3.5.2 R/3.5.3 R/3.6.0 samtools/1.9 use.own
-
modulefiles/intel/compiler101_ia32  modulefiles/module-info
+
-
modulefiles/intel/compiler101_x86_64 modulefiles/modules
+
-
modulefiles/intel/compiler91_ia32    modulefiles/mpich2
+
-
modulefiles/intel/compiler91_x86_64  modulefiles/null
+
-
modulefiles/java/i586/j2sdk1.4.2_17  modulefiles/openmpi/1.2.6-1/gcc
+
-
modulefiles/java/i586/jdk1.5.0_15    modulefiles/openmpi/1.2.6-1/intel
+
-
modulefiles/java/i586/jdk1.6.0_06    modulefiles/openmpi/1.2.6-1/path
+
-
modulefiles/java/x86_64/jdk1.5.0_15  modulefiles/openmpi/1.2.6-1/pgi
+
-
modulefiles/java/x86_64/jdk1.6.0_06  modulefiles/pgi/7.1-4_linux86
+
-
modulefiles/local_libs              modulefiles/pgi/7.1-4_linux86-64
+
-
modulefiles/mkl/10/32                modulefiles/pgi/7.1_linux86
+
-
modulefiles/mkl/10/64                modulefiles/pgi/7.1_linux86-64
+
-
modulefiles/mkl/10/em64t            modulefiles/score
+
-
modulefiles/mkl/10.0.3.020/32        modulefiles/use.own
+
-
+
-
------------------------ /usr/share/modules/modulefiles ------------------------
+
-
  atlas                    local_libs              null
+
-
dot                      mkl/10/32                openmpi/1.2.6-1/gcc
+
-
  intel/compiler101_ia32  mkl/10/64                openmpi/1.2.6-1/intel
+
-
  intel/compiler101_x86_64 mkl/10/em64t            openmpi/1.2.6-1/path
+
-
  intel/compiler91_ia32    mkl/10.0.3.020/32        openmpi/1.2.6-1/pgi
+
-
  intel/compiler91_x86_64 mkl/10.0.3.020/64        pgi/7.1-4_linux86
+
-
  java/i586/j2sdk1.4.2_17 mkl/10.0.3.020/em64t    pgi/7.1-4_linux86-64
+
-
  java/i586/jdk1.5.0_15    module-cvs              pgi/7.1_linux86
+
-
  java/i586/jdk1.6.0_06    module-info              pgi/7.1_linux86-64
+
-
  java/x86_64/jdk1.5.0_15  modules                  score
+
-
java/x86_64/jdk1.6.0_06 mpich2                  use.own
+
==Load a module==
==Load a module==
-
  module load modulename
+
  module load <module_name>
For example
For example
-
  module load java/x86_64/jdk1.5.0_15
+
  module load gcc/7.4.0
==Unload a module==
==Unload a module==
-
  module unload modulename
+
  module unload <module_name>
For example
For example
-
  module unload java/x86_64/jdk1.5.0_15
+
  module unload gcc/7.4.0
 +
 
 +
==Using module command in your bash scripts==
 +
 
 +
To use the module command in you bash scripts you must first intialise it within you script using the following line
 +
 
 +
source /etc/profile.d/modules.sh
 +
 
 +
For example
 +
 
 +
#!/bin/bash
 +
 
 +
source /etc/profile.d/modules.sh
 +
 
 +
module list
 +
module load gcc/7.4.0
 +
module list
 +
 
 +
==Using module command in your qsub scripts==
 +
 
 +
'''Option One'''
 +
 
 +
Set you qsub script to include all environment variables
 +
 
 +
#$ -V
 +
 
 +
For example
 +
 
 +
#!/bin/bash
 +
#$ -N R_JOB
 +
#$ -M me@lshtm.ac.uk -m be
 +
#$ -q short.q
 +
#$ -l mem_free=1G,h_vmem=1.2G
 +
#$ -V -cwd
 +
 +
module load R/3.4.1
 +
 +
R CMD BATCH myrscript myrscript.out
 +
 
 +
'''Option two'''
 +
 
 +
Load the module environment directly to the qsub script with
 +
 
 +
source /etc/profile.d/modules.sh
 +
 
 +
For example
 +
 
 +
#!/bin/bash
 +
#$ -q short.q
 +
#$ -l mem_free=1G,h_vmem=1.2G
 +
 
 +
source /etc/profile.d/modules.sh
 +
 +
module load R/3.4.1
 +
 +
R CMD BATCH myrscript myrscript.out
[[Category:Getting Started]]
[[Category:Getting Started]]

Current revision as of 09:41, 4 July 2019

Contents

Environment Modules (module)

Modules are a tool to allow you to change the working environment to support various application and version of those applications.

It allows use for example to support three versions of java and 32bit and 64 version of each. This equates to 6 different versions, using modules you can load the one you want to use or unload the ones you don't.


http://modules.sourceforge.net/

https://en.wikipedia.org/wiki/Environment_Modules_(software)

Commands

Help

module help

Show currently loaded modules

module list

Example 

# module list
Currently Loaded Modulefiles:
 1) R/3.6.0   2) samtools/1.9   3) mrbayes/3.2.7a

Show available modules

module avail

This would display something similar to this 


# module avail
-------------------------------  /usr/local/modulefiles -------------------------------------------------------------------------------------------------- 
bcftools/1.9  beast/1.10.4  dot  gcc/6.5.0  gcc/7.4.0  gdal/2.3.3  module-git  module-info  modules  mrbayes/3.2.7a  null  R/3.4.1  R/3.4.4  R/3.5.2  R/3.5.3  R/3.6.0  samtools/1.9  use.own

Load a module

module load <module_name>

For example 

module load gcc/7.4.0

Unload a module

module unload <module_name>

For example 

module unload gcc/7.4.0

Using module command in your bash scripts

To use the module command in you bash scripts you must first intialise it within you script using the following line 

source /etc/profile.d/modules.sh

For example 

#!/bin/bash

source /etc/profile.d/modules.sh

module list
module load gcc/7.4.0
module list

Using module command in your qsub scripts

Option One

Set you qsub script to include all environment variables 

#$ -V

For example 

#!/bin/bash
#$ -N R_JOB
#$ -M me@lshtm.ac.uk -m be
#$ -q short.q
#$ -l mem_free=1G,h_vmem=1.2G
#$ -V -cwd 

module load R/3.4.1

R CMD BATCH myrscript myrscript.out

Option two

Load the module environment directly to the qsub script with 

source /etc/profile.d/modules.sh

For example 

#!/bin/bash
#$ -q short.q
#$ -l mem_free=1G,h_vmem=1.2G

source /etc/profile.d/modules.sh

module load R/3.4.1

R CMD BATCH myrscript myrscript.out
Retrieved from "http://wiki.lshtm.ac.uk/hpc/index.php5/Modules"
Personal tools