Virtual environment

ℹ️ Objectives of this section

• List tools for managing Python environments
• Declare dependencies
• Using pipenv to install dependencies

• Definition
• Tools
  • Install Pipenv today!
• Declare your dependencies
  • requirements.txt example
    • 🛠️ Exercise
  • environment.yml example
• Create and activate an environment
• Install dependencies
  • 🛠️ Exercise
• Deactivate the active environment

Definition

A python virtual environment is a small, easily reproducible and erasable space that takes into account all the dependencies of our Python project. It allows you to work in isolation. It also allows other contributors to work in the same configuration.

Tools

There are a number of tools available, and it's by using them that you can make up your own mind. In this tutorial we will use pipenv, based on venv, because it's simple, lightweight and easy to use.

Install Pipenv today!

pip install --user pipenv

Here's a list of tools you can use to create your own environments:

• venv

  This tool is available since version 3.3 of Python and is installed on your system at the same time as Python. It is a subset of virtualenv.

• virtualenv

  This tool offers more flexibility than venv. Especially it let you choose the version of python.

• virtualenvwrapper

  This tool is a wrapper of virtualenv that let you create, delete, copy, switch environment easily.

• pipenv

  This tool combines pip and virtualenv into one. You no longer need to use them separately. It manages environment per project, it creates a Pipfile(.lock) inside your project, maintaining your dependances.

• mamba

  This tool allows you to manage multiple environments using the packages on PyPI and the various conda channels.

• poetry

  This tool manages environments and dependencies in a similar way to pipenv, but it can also build distributions, and it can upload it to PyPI. It uses the pyproject.toml standard, but it does not follow the standard specifying how metadata should be represented in a pyproject.toml file.

• hatch

  Hatch can also manage environments, allowing multiple environments per project. By default it has a central location for all environments but it can be configured to put a project's environment(s) in the project root directory. It can manage packages but without lockfile support. It can also be used to package a project and upload it to PyPI. It uses the pyproject.toml standard, and follow the standard specifying how metadata should be represented in a pyproject.toml file.

• pixi

  This tool can be used to create nodejs-style environments where the configuration is included in the development directory, using packages found on conda. It is therefore not dedicated solely to Python development and is much more general-purpose.

Declare your dependencies

While we have already seen that some tools can write their own files indicating the dependencies of an environment, we will now look at two slightly more universal formats

• requirements.txt is a file for describing dependencies, and can be used with all the tools listed above.
• environment.yml is used by tools based on conda packages. It also allows to choose the python version and the channels you want to fetch your dependencies.

requirements.txt example

Without version specifier, it refers the latest version

numpy
matplotlib
jupyterlab
requests[security]

With version specifier and environment marker, you can be more precise.

numpy>1.20
matplotlib==3.7.1
jupyterlab>4.0,<4.1; platform_system == "Linux"
requests [security] >= 2.8.1, == 2.8.* ; python_version < "2.7"

🛠️ Exercise

Create your requirements.txt and start your pipenvenvironment:

nano requirements.txt
pipenv install -r requirements.txt

Look at Pipfileand Pifile.lock.

To reset your environment: pipenv --rm, it doesn't destroy the Pipfile(.lock), so you can recreate the environment with: pip install.

To check in which folder are the dependencies placed: pipenv --venv.

environment.yml example

See more information on requirements specifiers on https://pip.pypa.io/en/stable/reference/requirement-specifiers/

Without version specifier

name: my_env
channels:
    - conda-forge
dependencies:
    - numpy
    - matplotlib
    - jupyterlab

With version specifier

name: my_env
channels:
    - conda-forge
dependencies:
    - numpy>1.20
    - matplotlib=3.7.1
    - jupyterlab>4.0,<4.1

Create and activate an environment

Before installing your dependencies, you need to create and activate your environment.

• With pipenv

  pipenv install
  pipenv shell

And with some other environment management tools:

• With venv (Linux ou Mac)

  python3 -m venv my_env
  source my_env/bin/activate

• With venv (Windows)

  py -m venv my_env
  my_env\Scripts\activate

• With virtualenv

  virtualenv my_env
  source my_env/bin/activate

• With mamba

  • empty environment:

    mamba create -n my_env
    mamba activate my_env

  • environment with dependencies downloaded:

    mamba create -n my_env --file environment.yml
    mamba activate my_env

Note

After activating an environment, its name should appear in the prompt of your terminal. The python executable path should be the one from your environment and not from your system.

(my_env) foo@far:~/path/to/somewhere $ which python
path/to/my_env/bin/python

Install dependencies

• With pip

  pip install -r requirements.txt

• With pipenv

  pipenv install -r requirements.txt

• With mamba

  mamba install --file requirements.txt

🛠️ Exercise

Install the dependencies for this tutorial using pipenv.

The requirements.txt file is located in the practical_session/configuration directory.

If you did the previous exercices with pipenv, you need to clean up your environment:

pipenv --rm
rm Pipfile*
cp practical_session/configuration/requirements.txt requirements.txt
pipenv install -r requirements.txt

Deactivate the active environment

To deactivate it you need to run deactivate in your terminal:

deactivate