PythonVirtualenv < Main

---+ Python Virtual Environments

This document is meant to be a quick introduction to virtualenv and Python virtual environments. For more information, see the official [[https://virtualenv.pypa.io/en/latest/userguide.html][virtualenv user guide.]].
---++ What is virtualenv?

virtualenv simplifies the process of creating your own tree of Python modules, so you can install modules that are not installed system-wide, or that conflict with system-wide versions. It's not a virtual machine, just a convenient way of setting up your environment to use a different Python tree. While this can be done by hand, virtualenv automates the process and makes it significantly easier.
---++ Walkthrough

For this example, we're going to set up a python 3.4 virtual environment on patas.ling.washington.edu, and install the Python port of Hunt the Wumpus*.

First, create the virtual environment. This is as simple as =virtualenv3.4 NAME=, where NAME is the directory you want the virtual environment to be in. There are several options; for this example we'll use =--system-site-packages=, which tells it to make the system-wide module directory accessible in our virtual environment. This will reduce the number of dependencies we have to install. You may want to avoid this if, for example, you're installing a conflicting version of a system module.
<verbatim>brodbd@patas:~[0]$ virtualenv3.4 --system-site-packages wumpus
Using base prefix '/opt/python-3.4.1'
New python executable in /home2/brodbd/wumpus/bin/python3.4
Also creating executable in /home2/brodbd/wumpus/bin/python
Installing setuptools, pip, wheel...done.</verbatim>

Next, we can source the newly-created activation script into our shell, to enter our new environment.
<verbatim>brodbd@patas:~[0]$ source wumpus/bin/activate
(wumpus) brodbd@patas:~[0]$</verbatim>

Notice the shell prompt has changed to remind us we're now in a different python environment. This will persist until we either log out or use the
<literal><span class='WYSIWYG_TT'>deactivate</span>&nbsp;<span style='color: #000000; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: small; background-color: transparent;'>command. Now that we have a new Python environment exclusive to us, we can use pip to install the module we want and any dependencies.</span></literal> <verbatim>(wumpus) brodbd@patas:~[0]$ pip install pywumpus
Collecting pywumpus
Installing collected packages: pywumpus
Successfully installed pywumpus-1.2</verbatim>

Pywumpus installs an executable script in the bin directory, which the activate script conveniently added to our path.
<verbatim>(wumpus) brodbd@patas:~[0]$ pywumpus.py
Instructions (Y-N)? n

You are in room 9.
    I smell a Wumpus.
    Bats nearby.
Tunnels lead to 8, 10, 18.
Shoot or Move (S-M)?</verbatim>

When you're done enjoying classic computer gaming circa 1972, you can exit back to the system python environment with
<literal><span class='WYSIWYG_TT'>deactivate</span><span style='color: #000000; font-family: Verdana, Arial, Helvetica, sans-serif; font-size: small; background-color: transparent;'>:</span></literal> <verbatim>(wumpus) brodbd@patas:~[0]$ deactivate
brodbd@patas:~[0]$</verbatim>

---++ Condor usage

Since the activation script just changes environment variables, the easiest way to use a virtualenv environment from Condor is to wrap your job in a shell script. Something like this should work:
<verbatim>#!/bin/bash
source $HOME/my_environment/bin/activate
python3.4 my_job.py</verbatim>

Put this in a script file (say, =my_job.sh=) and use =chown +x my_job.sh= to make it executable. You would then use =Executable=my_job.sh= in your condor submit script. In this case it's unnecessary to call deactivate, since the environment changes will disappear at the end of the script.

---

* [[https://en.wikipedia.org/wiki/Hunt_the_Wumpus][Hunt the Wumpus]] has a long history, having been written in 1972 at University of Massachusetts - Dartmouth. It was originally written for a mainframe system, in BASIC (a language that was itself invented at Dartmouth.) It was endlessly ported and re-published and many older geeks (myself included) have fond memories of typing it into our microcomputers from a listing in a book or magazine. - [[Main.brodbd][brodbd]] - 2016-05-06
This topic: Main > WebHome > PythonVirtualenv
Topic revision: r2 - 2016-05-06 - brodbd