Installing Python Packages

If you try to install a Python package using pip install <name> on the Maths systems, you will get an error such as:

error: externally-managed-environment

× This environment is externally managed
[...]

This is because the systemwide packages are installed and managed by Ubuntu, not by pip.

There are two ways around this:

Per-User Installation

The simplest option is to use the --break-system-packages to bypass this check:

pip install PACKAGENAME --break-system-packages

Then pip will continue to install the package in your home directory (~/.local/lib/pythonVERSION/site-packages), which will take priority over the system packages.

The downside is you can only install one version of each package per user.

Virtual Environments

The recommended option is to set up a virtual environment for each project:

python3 -m venv ~/my-venv

Activate it:

source ~/my-venv/bin/activate

Then install the required packages in it:

pip install PACKAGENAME

Now when you run your Python script (e.g. python test.py) it will use the packages installed into the current virtual environment. This allows you to have a separate set of packages for each project - simply give each venv a different name, and activate/deactivate each virtual environment as needed.

To stop using a virtual environment, run:

deactivate

If you want to delete the virtual environment completely, deactivate it and then run:

# WARNING: Make sure there is nothing you need in the ~/my-venv/ directory before doing this!
rm -rf ~/my-venv

Tip: If you need to install a large number/size of dependencies, you may prefer to install them in the local scratch partition (e.g. /scratch/$USER/my-venv) instead of your networked home directory, as it will be faster and generally has more storage space available - but make sure you don't store anything important there as it is backed up less often.