From HPC

Revision as of 12:09, 8 November 2016 (view source) Aitsswhi (Talk | contribs) (→Running Task array jobs) ← Older edit		Revision as of 12:37, 17 March 2017 (view source) Aitsswhi (Talk | contribs) (→Python) Newer edit →
Line 7:		Line 7:
	python2		python2
	pip2		pip2
		+
		+	For example
		+	pip2 install --user package_name
	python3.5		python3.5
	pip3.5		pip3.5
		+
		+	For example
		+	pip3.5 install --user package_name
	==Running Task array jobs==		==Running Task array jobs==

---

Revision as of 12:37, 17 March 2017

Python

Version 2.7 and 3.5 are available on the HPC

Use the following executable binaries to start the correct version

python2
pip2

For example
pip2 install --user package_name

python3.5
pip3.5

For example
pip3.5 install --user package_name

Running Task array jobs

Example SGE job script (One job with 10 tasks)

#!/bin/bash
#$ -N ARRAY_TEST_JOB
#$ -cwd -V
#$ -q short.q
#$ -l mem_free=1G,h_vmem=1.2G
#$ -t 1-10

python2 mycode.py

To use the current SGE job task number your python code you need to access the variable and assign it.

#!/bin/python2
import os
os.environ.keys() 
taskid = os.environ['JOB_ID']

Managing packages with virtual environments

It is recommend to use Python Virtual Environments to create isolated Python environments where you can install any package combinations you need without worrying about dependencies and versions, and indirectly permissions.

Please read | Creating Virtual Environments for more information

The basic usage is like so:

Using virtualenv: (Python 2.6)

virtualenv <DIR>
source <DIR>/bin/activate

Using venv: (Pyhton 3.5)

python3.5 -m venv <DIR>
source <DIR>/bin/activate