Conda (R/Python package management)
From HPC
Contents |
Conda
Package, dependency and environment management for any language---Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, FORTRAN
Conda is an open source package management system and environment management system that runs on Windows, macOS, and Linux. Conda quickly installs, runs and updates packages and their dependencies. Conda easily creates, saves, loads, and switches between environments on your local computer. It was created for Python programs but it can package and distribute software for any language.
Conda as a package manager helps you find and install packages. If you need a package that requires a different version of Python, you do not need to switch to a different environment manager because conda is also an environment manager. With just a few commands, you can set up a totally separate environment to run that different version of Python, while continuing to run your usual version of Python in your normal environment.
https://docs.conda.io/projects/conda/en/latest/
Installing Conda (MiniConda)
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh bash ./Miniconda3-latest-Linux-x86_64.sh
miniconda will be installed to your home directory (~/miniconda3) and can be used with the command conda
(base) [username@comp25 ~]$ conda info
active environment : base
active env location : /home/ussername/miniconda3
shell level : 1
user config file : /home/username/.condarc
populated config files :
conda version : 4.6.14
conda-build version : not installed
python version : 3.7.3.final.0
base environment : /home/aitsswhi/miniconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/linux-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/aitsswhi/miniconda3/pkgs
/home/aitsswhi/.conda/pkgs
envs directories : /home/aitsswhi/miniconda3/envs
/home/aitsswhi/.conda/envs
platform : linux-64
user-agent : conda/4.6.14 requests/2.21.0 CPython/3.7.3 Linux/3.10.0-957.12.1.el7.x86_64 scientific/7.6 glibc/2.17
UID:GID : 10033:10000
netrc file : None
offline mode : False
Guides and useful resources
https://docs.conda.io/projects/conda/en/latest/user-guide/index.html
https://www.anaconda.com/distribution/
R Specific
https://docs.anaconda.com/anaconda/user-guide/tasks/using-r-language/
Installing packages
Install gdal
Reference https://gdal.org/download.html#binaries (GDAL is most up to date on Conda Forge)
conda install -c conda-forge gdal
Alternatively enable Conda Forge repository and then install GDAL
conda config --add channels conda-forge conda config --set channel_priority strict conda install gdal
Troubleshooting
If you're getting messages like
/bin/sh: module: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `BASH_FUNC_module'
/bin/sh: switchml: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `BASH_FUNC_switchml'
/bin/sh: _module_raw: line 1: syntax error: unexpected end of file
/bin/sh: error importing function definition for `BASH_FUNC__module_raw'
in the error output after submitting your job using conda, ty including the following statement at the start of your script:
source ~/.bashrc
When activating an environment in your job submission script use
source activate [ENVIROMENT]
