A GitLab upgrade is scheduled for Monday, June 2, 2025. The service will be unavailable for a few minutes in the morning. We'll keep you posted on the progress of the upgrade on the Mattermost channel: https://mattermost.inria.fr/devel/channels/gitlab. We recommend that you do not work on the platform until an announcement indicates that maintenance is complete.
<p><spanclass="figure-number">Figure 1: </span>Overview of <code>overleaf</code> editor: on the left users can collaboratively edit the document. On the right the document is rendered.</p>
</div>
<ulclass="org-ul">
<li>Write bug reports / ask questions</li>
<li>Fix bugs / add your features</li>
</ul>
<p>
<b>After experimenting</b>
Here is the plan:
</p>
<ulclass="org-ul">
<li>Give your feedback</li>
<li>Add yourself to the list: <ahref="https://discovery.gitlabpages.inria.fr/enoslib/theyuseit.html">https://discovery.gitlabpages.inria.fr/enoslib/theyuseit.html</a></li>
<li><b><b>Deployment</b></b> You'll first deploy our own <code>overleaf</code> instance (we don't want to use
the official/commercial instance).</li>
<li><b><b>Load generation</b></b> You'll generate compilation of different projects programatically.</li>
<li><b><b>Observation</b></b> You'll observe the effect of the load in the running system through various metrics.</li>
<li><b><b>Feedback</b></b> You'll formulate some hypothesis on the load characteristics and the observed effects on the system.</li>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Note 1: that you don't need to be on rennes frontend to use nodes</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Note 2: Adapt the site/cluster according to the availibility</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">see the Gantt in https://www.grid5000.fr/w/Status</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Configuration object describes the resource we want</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">here: 2 machines on the same cluster using the production network</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Reserve the ressources corresponding to the configuration</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">you'll get two **physical machine** (not virtual)</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">the roles object is a dictionnary of the concrete compute resources</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Below is the experimentation logic</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- It installs the bare minimum to run iperf3</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'server' is used to run a iperf3 server</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">started in the background (using tmux)</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'client' connects to that server and initiate a</span>
Sorry, your browser does not support SVG.</object></a>
</p>
<p><spanclass="figure-number">Figure 2: </span>Simplified architecture of the system under study (blue) and the generated users (black). Overleaf is composed of a dozen services ony four are represented here.</p>
</div>
<divclass="note">
<p>
Before moving to the next questions, you'll need to clean the reservation.
You can either uncomment the line <code>provider.destroy()</code> at the end of the script.
You can also do it manually using the low-level <code>oarstat</code> / <code>oardel</code> tools.
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Note 1: that you don't need to be on rennes frontend to use nodes</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Note 2: Adapt the site/cluster according to the availibility</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">see the Gantt in https://www.grid5000.fr/w/Status</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Configuration object describes the resource we want</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">here: 2 machines on the same cluster using the production network</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Reserve the ressources corresponding to the configuration</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">you'll get two **physical machine** (not virtual)</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">the roles object is a dictionnary of the concrete compute resources</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">This deploys a monitoring stack. It is composed of</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- some agents on each monitored nodes</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- one collector that collects the metrics from the agents</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- one UI to visualize</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Below is the experimentation logic</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- It installs the bare minimum to run iperf3</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'server' is used to run a iperf3 server</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">started in the background (using tmux)</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;">## </span><spanstyle="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'client' connects to that server and initiate a</span>
Today you can just create a tunnel like this (from your local machine).
</p>
<divclass="org-src-container">
<preclass="src src-bash"><spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Adapt the node names with the node where grafana (the UI) has been installed</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Replace <login> by your Grid'5000 login</span>
So, far this seems (at least for me) very handy. But there might be some problems in our setup:
</p>
<ulclass="org-ul">
<li>we aren't isolated from the other users</li>
<li>we aren't isolated from ourself in the sense that the monitoring stack generates its own
network traffic (yes, this is negligible in our case)</li>
<li>You can change the cluster name with any cluster on Grid'5000: see <ahref="https://www.grid5000.fr/w/Hardware">https://www.grid5000.fr/w/Hardware</a></li>
<li>This can take several minutes…</li>
</ul>
<p>
Sometimes it's desirable to have the following setup (see Fig. <ahref="#orgf90cb97">2</a>).
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Below is the experimentation logic</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">It installs the bare minimum to run iperf3</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">The machine with the role 'server' is used to run a iperf3 server</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">started in the background in a tmux</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">The machine with the role 'client' connects to that server</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Report is printed in stdout</span>
<preclass="src src-bash"><spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Adapt the node names with the node where the portal has been installed</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Replace <login> by your Grid'5000 login</span>
<code>enoslib.api.run</code> is a specialisation of <code>enoslib.api.run_command</code>.
The latter let's you use <ahref="https://docs.ansible.com/ansible/latest/user_guide/intro_patterns.html">some fancy patterns</a> to determine the list of hosts to run the command on.
</p>
<p>
And yes, it uses Ansible behind the scene.
You can access the web portal of your friends by replacing the address of the web portal.
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Passing a variable to the Ansible World using a global level variable</span>
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Passing a variable to the Ansible World using a host level variable</span>
<code>enoslib.api.play_on</code> is the entry point to the module system.
</p>
<p>
You can run this code as many times as you want without any error. You'll
eventually find one user <code>foo</code> and one directory <code>plop</code> in your target
systems.
</p>
<p>
They are more than 2500 modules: <ahref="https://docs.ansible.com/ansible/latest/modules/list_of_all_modules.html">https://docs.ansible.com/ansible/latest/modules/list_of_all_modules.html</a>
</p>
<p>
If you can't find what you want you must know that:
Observations can be made on two portals:
</p>
<ulclass="org-ul">
<li>Writing your own module is possible</li>
<li>Falling back to the idempotency trick is reasonable</li>
<li>the benchmarking portal: you'll get some information on the requests sent
to the server</li>
<li>the monitorig portal: you'll get some information about the resource
<h2id="orgb39fe8f"><spanclass="section-number-2">8</span> Tasks: to organize your experiment</h2>
<divclass="outline-text-2"id="text-8">
<p>
To discover the Task API, head to <ahref="https://discovery.gitlabpages.inria.fr/enoslib/tutorials/using-tasks.html">https://discovery.gitlabpages.inria.fr/enoslib/tutorials/using-tasks.html</a>.
</p>
<p>
The examples are written for Vagrant but may be changed to whatever provider you like/have.
</p>
<divclass="question">
<p>
Adapt the <code>iperf3</code> example to provide a command line
</p>
<ulclass="org-ul">
<li><p>
Either using G5k physical machines:
</p>
<divclass="org-src-container">
<preclass="src src-bash"><spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">deploy the dependencies of the experimentation using the G5k provider</span>
<preclass="src src-bash"><spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">deploy the dependencies of the experimentation using the G5k provider</span>
myperf vm5k
<spanstyle="color: #2aa1ae; background-color: #292e34;"># </span><spanstyle="color: #2aa1ae; background-color: #292e34;">Subsequent command line should be the same as above</span>
~enoslib.api.run~ is a specialisation of ~enoslib.api.run_command~.
The latter let's you use [[https://docs.ansible.com/ansible/latest/user_guide/intro_patterns.html][some fancy patterns]] to determine the list of hosts to run the command on.
And yes, it uses Ansible behind the scene.
#+END_note
** Advanced usages
#+BEGIN_note
For all the remote interactions, {{{enoslib}}} relies on [[https://docs.ansible.com/ansible/latest/index.html][Ansible]]. Ansible
has it own variables management system.
For instance the task ~Gather Facts~ at the beginning of the previous tasks
gathers informations about all/some remote hosts and store them in the
Ansible management system.
#+END_note
Let's see what Ansible is gathering about the hosts:
