Deployment

This repository contains Python and bash scripts to create a deployment. It is expected to be used as a git-subrepo or a git-subtree. The reservation of resources and the deployment leverages the EnOSlib library which then runs Ansible recipes.

Warning: this repository will not behave as intended if used as a git-submodule. This is mainly due to the way the git rev-parse --show-toplevel command works: inside a subtree or subrepo, it will report the root of the parent repo, whilst for a submodule it will report the submodule directory itself.

Note: in the following, git-root refers to the working tree directory root.

Usage

To deploy and run, simply execute run. Please note that you need to provide at least the git-root/ansible/main.yml playbook.

Dependencies

To install the required dependencies, only Guix with support for Inferiors. is needed. The provided environment file will be loaded by run and will download all needed dependencies. Some python-specific dependencies are then managed by Pipenv. Depending on the Ansible modules you plan to use, several additional dependencies may be needed.

Guix-managed dependencies

If Guix is unavailable or if you do not want to use it, you may instead install the following:

• Python 3.6+ (actually, the minimum version required by the version of EnOSlib you plan to use)
• Pip (for use with pipenv)
• Pipenv (for locking and syncing dependencies in a python venv)
• ssh (for connecting to remote hosts with EnOSlib and Ansible)
• git (for gitpython)

Ansible dependencies

Some modules have additional dependencies, which are not always easy to find out. In this repository, they are also managed inside the Guix environment:

• bash
• coreutils (at least for mkdir)
• rsync (if you plan to use the synchronize module)

Python dependencies

Likewise, if you do not plan to use Pipenv, you may instead install the EnOSlib library and all its dependencies. Please note that Pipenv is used as too many python libraries are missing from the Guix repository and adding them all incurs a really high development cost. gitpython is also needed for this to work.

Configuration

Internally, EnOSlib uses python-grid5000 to connect to Grid5000. This library needs your username and password to connect to the Grid5000 frontends. This information needs to be stored in the $HOME/.python-grid5000.yaml file:

username: LOGIN
password: PASSWORD

Additionally, EnOSlib requires the file $HOME/.ssh/id_rsa to connect to the reserved nodes.

SSH must also be configured properly to connect to the reserved nodes, see https://discovery.gitlabpages.inria.fr/enoslib/tutorials/grid5000.html#ssh-external-access

Host *.grid5000.fr
ProxyCommand ssh -A <login>@194.254.60.33 -W "$(basename %h):%p"
User <login>
ForwardAgent yes

This step is not needed if you are connected through the Grid5000 VPN or on the Grid5000 network. Note: if you use the exact configuration from above, you need to add basename to the guix environment. Alternatively, you may use:

Host *.grid5000.fr
ProxyCommand ssh -A <login>@access.grid5000.fr -W "%h:%p"
User <login>
ForwardAgent yes

Architecture

First, run calls guix environment --pure to create a clean environment with only what is strictly needed for the execution. It then calls the run-in-guix-env script through the ssh-agent wrapper. This script first adds the default ssh key used by EnOSlib ($HOME/.ssh/id_rsa). This way, the execution will not hang of fail if ansible needs the password to the key but it was not provided during the execution. This also makes it easier to use: user input is only needed at the start of the execution ; afterwards everything is automated. It then syncs dependencies before running the main.py python script. The main python file starts with reserving a node on Grid5000, deploying a standard environment, and then launches the main.yml Ansible playbook.

Customization

To use another provider such as a virtual machine, another cluster on Grid5000, or on another infrastructure altogether, please refer to the EnOSlib documentation and then modify directly the main.py file. You may pass additional variables to Ansible from the python script by adding information to the extra_vars dictionary.