diff --git a/public/tuto1/index.html b/public/tuto1/index.html
index f69b6efaeed60d17525568a37ce78897011a2d49..16a5ee05bdc41e50dd1820a6a7ae5451c9f86936 100644
--- a/public/tuto1/index.html
+++ b/public/tuto1/index.html
@@ -3,7 +3,7 @@
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
-<!-- 2019-11-15 ven. 14:14 -->
+<!-- 2019-11-29 ven. 01:18 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Distributed experiments on Grid'5000 … and beyond !</title>
@@ -235,130 +235,77 @@ for the JavaScript code in this tag.
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
-<li><a href="#orgb1aa660">1. Foreword</a>
+<li><a href="#org5a4c054">1. Benchmarking a real application</a></li>
+<li><a href="#orgcbc76d2">2. Before you start</a>
<ul>
-<li><a href="#org33af631">1.1. Existing tools (Grid'5000)</a></li>
-<li><a href="#org769cfcb">1.2. EnOSlib quicktour</a></li>
-<li><a href="#org4a53df5">1.3. Contributing</a></li>
+<li><a href="#orgfc26e5c">2.1. Grid'5000 stuffs</a></li>
+<li><a href="#org3b0eb38">2.2. Setup on Grid'5000</a></li>
</ul>
</li>
-<li><a href="#orgc7e91b9">2. Before you start</a></li>
-<li><a href="#org782c78a">3. Setup on Grid'5000</a></li>
-<li><a href="#org441c9dd">4. Your first experiment on Grid'5000</a>
+<li><a href="#org1f349ae">3. Deployment time !</a>
<ul>
-<li><a href="#orgf5c5450">4.1. First iteration</a></li>
-<li><a href="#org993bcaf">4.2. Let's observe in real-time what is happening</a></li>
-<li><a href="#org830e31f">4.3. Discussion</a></li>
-<li><a href="#org5f35d8c">4.4. A bit better approach</a></li>
-<li><a href="#orgdddf696">4.5. Ninja level</a></li>
-<li><a href="#org1193557">4.6. Some references</a></li>
+<li><a href="#orgfd70908">3.1. Deploy it</a></li>
+<li><a href="#orgafb102d">3.2. Access it</a></li>
</ul>
</li>
-<li><a href="#orgf965b96">5. Providers: to replicate your experiment</a>
+<li><a href="#org9752580">4. Deploy the monitoring stack</a></li>
+<li><a href="#orgd7c542a">5. Benchmark the system</a>
<ul>
-<li><a href="#orgad471c7">5.1. iperf3 on virtual machines on Grid'5000</a></li>
-<li><a href="#org08faf88">5.2. References</a></li>
+<li><a href="#org9c6c9c3">5.1. Deploy the benchmarking nodes</a></li>
+<li><a href="#orge88815f">5.2. Observations</a></li>
</ul>
</li>
-<li><a href="#org56f9c08">6. Variables in EnOSlib</a>
-<ul>
-<li><a href="#org2281689">6.1. Discover the <code>run</code> command and its variants</a></li>
-<li><a href="#orgd2c7291">6.2. Advanced usages</a></li>
-<li><a href="#orgd78a6d8">6.3. Ninja level</a></li>
-<li><a href="#orge739be4">6.4. Putting all together</a></li>
-<li><a href="#org5dafa08">6.5. Some references</a></li>
-</ul>
-</li>
-<li><a href="#orga167566">7. Modules: for safer remote actions</a>
-<ul>
-<li><a href="#orgc30170a">7.1. Idempotency</a></li>
-<li><a href="#org334a244">7.2. One reason why idempotency is important</a></li>
-<li><a href="#orgcf77118">7.3. Idempotency trick</a></li>
-<li><a href="#orgcaa5aa0">7.4. General idempotency</a></li>
-</ul>
-</li>
-<li><a href="#orgb39fe8f">8. Tasks: to organize your experiment</a></li>
</ul>
</div>
</div>
-<div id="outline-container-orgb1aa660" class="outline-2">
-<h2 id="orgb1aa660"><span class="section-number-2">1</span> Foreword</h2>
-<div class="outline-text-2" id="text-1">
-</div>
-<div id="outline-container-org33af631" class="outline-3">
-<h3 id="org33af631"><span class="section-number-3">1.1</span> Existing tools (Grid'5000)</h3>
-<div class="outline-text-3" id="text-1-1">
-<ul class="org-ul">
-<li>EnOSlib falls under the <b><b>Experiment management tools</b></b> of the following
-list:
-<a href="https://www.grid5000.fr/w/Grid5000:Software">https://www.grid5000.fr/w/Grid5000:Software</a></li>
-<li>EnOSlib can target Grid'5000 but also other testbeds (Chameleon, local machines…)</li>
-
-<li>EnOSlib provides high level constructs to help you with your experiments</li>
-</ul>
-</div>
-</div>
-
-<div id="outline-container-org769cfcb" class="outline-3">
-<h3 id="org769cfcb"><span class="section-number-3">1.2</span> EnOSlib quicktour</h3>
-<div class="outline-text-3" id="text-1-2">
-<ul class="org-ul">
-<li>Documentation: <a href="https://discovery.gitlabpages.inria.fr/enoslib/index.html">https://discovery.gitlabpages.inria.fr/enoslib/index.html</a></li>
-<li>Source: <a href="https://gitlab.inria.fr/discovery/enoslib">https://gitlab.inria.fr/discovery/enoslib</a></li>
-<li>Reach us on:
-<ul class="org-ul">
-<li><a href="https://framateam.org/enoslib">https://framateam.org/enoslib</a></li>
-<li><a href="https://gitlab.inria.fr/discovery/enoslib/issues">https://gitlab.inria.fr/discovery/enoslib/issues</a></li>
-</ul></li>
-</ul>
-</div>
-</div>
-
-<div id="outline-container-org4a53df5" class="outline-3">
-<h3 id="org4a53df5"><span class="section-number-3">1.3</span> Contributing</h3>
-<div class="outline-text-3" id="text-1-3">
+<div id="outline-container-org5a4c054" class="outline-2">
+<h2 id="org5a4c054"><span class="section-number-2">1</span> Benchmarking a real application</h2>
+<div class="outline-text-2" id="text-1">
<p>
-<b>Before experimenting</b>
+In this tutorial we'll cover some aspects of evaluating the performance of a
+real application. We'll work with <code>overleaf</code>. <code>overleaf</code> is a collaborative
+text editor that uses Latex to produce pdf files. Figure <a href="#orgbe25ee3">1</a>
+is an overview of the editing part of the software.
</p>
-<ul class="org-ul">
-<li>Tell us what your plans are:
-<ul class="org-ul">
-<li>There might be already users doing similar thing</li>
-<li>There might be some missing/hidden pieces in the library you might need</li>
-</ul></li>
-</ul>
-<p>
-<b>While experimenting</b>
+<div id="orgbe25ee3" class="figure">
+<p><a href="./figs/overleaf-v2-editor.png"><img src="./figs/overleaf-v2-editor.png" alt="overleaf-v2-editor.png" /></a>
</p>
+<p><span class="figure-number">Figure 1: </span>Overview of <code>overleaf</code> editor: on the left users can collaboratively edit the document. On the right the document is rendered.</p>
+</div>
-<ul class="org-ul">
-<li>Write bug reports / ask questions</li>
-<li>Fix bugs / add your features</li>
-</ul>
<p>
-<b>After experimenting</b>
+Here is the plan:
</p>
<ul class="org-ul">
-<li>Give your feedback</li>
-<li>Add yourself to the list: <a href="https://discovery.gitlabpages.inria.fr/enoslib/theyuseit.html">https://discovery.gitlabpages.inria.fr/enoslib/theyuseit.html</a></li>
+<li><b><b>Deployment</b></b> You'll first deploy our own <code>overleaf</code> instance (we don't want to use
+the official/commercial instance).</li>
+<li><b><b>Load generation</b></b> You'll generate compilation of different projects programatically.</li>
+<li><b><b>Observation</b></b> You'll observe the effect of the load in the running system through various metrics.</li>
+<li><b><b>Feedback</b></b> You'll formulate some hypothesis on the load characteristics and the observed effects on the system.</li>
</ul>
</div>
</div>
-</div>
-
-<div id="outline-container-orgc7e91b9" class="outline-2">
-<h2 id="orgc7e91b9"><span class="section-number-2">2</span> Before you start</h2>
+<div id="outline-container-orgcbc76d2" class="outline-2">
+<h2 id="orgcbc76d2"><span class="section-number-2">2</span> Before you start</h2>
<div class="outline-text-2" id="text-2">
+<p>
+Make sure you are ok with the following.
+</p>
+</div>
+
+<div id="outline-container-orgfc26e5c" class="outline-3">
+<h3 id="orgfc26e5c"><span class="section-number-3">2.1</span> Grid'5000 stuffs</h3>
+<div class="outline-text-3" id="text-2-1">
<div class="note">
<p>
-make sure you are familiar with the grid'5000 architecture. see section 1 & 2 of
+Make sure you are familiar with the Grid'5000 architecture. see section 1 & 2 of
<a href="https://www.grid5000.fr/w/Getting_Started">https://www.grid5000.fr/w/Getting_Started</a>. note that we won't do this tutorial
we'll prefer to use higher level tools for now.
</p>
@@ -367,9 +314,10 @@ we'll prefer to use higher level tools for now.
</div>
</div>
-<div id="outline-container-org782c78a" class="outline-2">
-<h2 id="org782c78a"><span class="section-number-2">3</span> Setup on Grid'5000</h2>
-<div class="outline-text-2" id="text-3">
+
+<div id="outline-container-org3b0eb38" class="outline-3">
+<h3 id="org3b0eb38"><span class="section-number-3">2.2</span> Setup on Grid'5000</h3>
+<div class="outline-text-3" id="text-2-2">
<p>
Connect to a Grid'5000 frontend of your choice (e.g rennes, nancy …)
</p>
@@ -381,11 +329,11 @@ Connect to a Grid'5000 frontend of your choice (e.g rennes, nancy …)
</ul>
<div class="org-src-container">
-<pre class="src src-bash">$<span style="color: #7590db;">frontend</span>: mkdir enoslib_seminar
-$<span style="color: #7590db;">frontend</span>: cd enoslib_seminar
+<pre class="src src-bash">$<span style="color: #7590db;">frontend</span>: cp -r ~msimonin/public/ccs-g5k-tuto2 .
+$<span style="color: #7590db;">frontend</span>: cd ccs-g5k-tuto2
$<span style="color: #7590db;">frontend</span>: virtualenv --python=python3 venv
$<span style="color: #7590db;">frontend</span>: source venv/bin/activate
-$<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</span>venv<span style="color: #4f97d7;">)</span>: pip install enoslib
+$<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</span>venv<span style="color: #4f97d7;">)</span>: pip install -r requirements.txt
$<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</span>venv<span style="color: #4f97d7;">)</span>: echo <span style="color: #2d9574;">'</span>
<span style="color: #2d9574;">verify_ssl: False</span>
<span style="color: #2d9574;">'</span> > ~/.python-grid5000.yaml
@@ -393,894 +341,201 @@ $<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</s
</div>
</div>
</div>
-
-<div id="outline-container-org441c9dd" class="outline-2">
-<h2 id="org441c9dd"><span class="section-number-2">4</span> Your first experiment on Grid'5000</h2>
-<div class="outline-text-2" id="text-4">
-<p>
-Let's experiment with <a href="https://iperf.fr/">iperf3</a>: a network bandwidth measuring tool. The goal is
-to deploy a simple benchmark between two hosts.
-</p>
-
-<p>
-We'll also instrument the deployment in order to visualize in real-time the
-network traffic between the hosts. Since this is super common, EnOSlib
-exposes a <i>monitoring service</i> that lets you deploy very quickly what is
-needed.
-</p>
</div>
-<div id="outline-container-orgf5c5450" class="outline-3">
-<h3 id="orgf5c5450"><span class="section-number-3">4.1</span> First iteration</h3>
-<div class="outline-text-3" id="text-4-1">
+<div id="outline-container-org1f349ae" class="outline-2">
+<h2 id="org1f349ae"><span class="section-number-2">3</span> Deployment time !</h2>
+<div class="outline-text-2" id="text-3">
<p>
-We consider the following script
+Figure <a href="#org35d463f">2</a> represents a simplified view of what we'll deploy. In blue
+some services of <code>overleaf</code> are represented. First the Web portal is the entry
+point to all the user requests. The three other services in the picture are
+involved when compiling a document. The compilation service interacts with the
+filestore (where the files of the image of the projects are stored) and the
+docstore (where the text of the project is stored). There are other service
+involved to provide the chat feature, history feature, real-time interaction
+… but we'll focus ony on the compilation process.
</p>
-<div class="org-src-container">
-<pre class="src src-python"><span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.api <span style="color: #4f97d7; font-weight: bold;">import</span> run_command, wait_ssh
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.infra.enos_g5k.provider <span style="color: #4f97d7; font-weight: bold;">import</span> G5k
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.infra.enos_g5k.configuration <span style="color: #4f97d7; font-weight: bold;">import</span> Configuration, NetworkConfiguration
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.service <span style="color: #4f97d7; font-weight: bold;">import</span> Monitoring
-
-<span style="color: #4f97d7; font-weight: bold;">import</span> logging
-
-
-<span style="color: #4f97d7; font-weight: bold;">def</span> <span style="color: #bc6ec5; font-weight: bold;">pprint</span><span style="color: #4f97d7;">(</span>d<span style="color: #4f97d7;">)</span>:
- <span style="color: #2aa1ae;">"""Utils fonction to pretty print the results"""</span>
- <span style="color: #4f97d7; font-weight: bold;">for</span> k, v <span style="color: #4f97d7; font-weight: bold;">in</span> d<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"ok"</span><span style="color: #4f97d7;">]</span>.items<span style="color: #4f97d7;">()</span>:
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"Result for {k}"</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"-"</span> * <span style="color: #a45bad;">70</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"STDOUT:"</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span>v.get<span style="color: #bc6ec5;">(</span><span style="color: #2d9574;">"stdout"</span>, <span style="color: #2d9574;">""</span><span style="color: #bc6ec5;">)</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"STDERR:"</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span>v.get<span style="color: #bc6ec5;">(</span><span style="color: #2d9574;">"stderr"</span>, <span style="color: #2d9574;">""</span><span style="color: #bc6ec5;">)</span><span style="color: #4f97d7;">)</span>
-
-
-logging.basicConfig<span style="color: #4f97d7;">(</span>level=logging.INFO<span style="color: #4f97d7;">)</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Some parameters.</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Note 1: that you don't need to be on rennes frontend to use nodes</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">from rennes</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Note 2: Adapt the site/cluster according to the availibility</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">see the Gantt in https://www.grid5000.fr/w/Status</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #7590db;">SITE</span> = <span style="color: #2d9574;">"rennes"</span>
-<span style="color: #7590db;">CLUSTER</span> = <span style="color: #2d9574;">"paravance"</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Configuration object describes the resource we want</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">here: 2 machines on the same cluster using the production network</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #7590db;">network</span> = NetworkConfiguration<span style="color: #4f97d7;">(</span><span style="color: #4f97d7;">id</span>=<span style="color: #2d9574;">"n1"</span>,
- <span style="color: #4f97d7;">type</span>=<span style="color: #2d9574;">"prod"</span>,
- roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"my_network"</span><span style="color: #bc6ec5;">]</span>,
- site=SITE<span style="color: #4f97d7;">)</span>
-
-<span style="color: #7590db;">conf</span> = Configuration.from_settings<span style="color: #4f97d7;">(</span>job_name=<span style="color: #2d9574;">"enoslib_tutorial"</span>,
- job_type=<span style="color: #2d9574;">"allow_classic_ssh"</span><span style="color: #4f97d7;">)</span>\
- .add_network_conf<span style="color: #4f97d7;">(</span>network<span style="color: #4f97d7;">)</span>\
- .add_machine<span style="color: #4f97d7;">(</span>roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #bc6ec5;">]</span>,
- cluster=CLUSTER,
- nodes=<span style="color: #a45bad;">1</span>,
- primary_network=network<span style="color: #4f97d7;">)</span>\
- .add_machine<span style="color: #4f97d7;">(</span>roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #bc6ec5;">]</span>,
- cluster=CLUSTER,
- nodes=<span style="color: #a45bad;">1</span>,
- primary_network=network<span style="color: #4f97d7;">)</span>\
- .finalize<span style="color: #4f97d7;">()</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Reserve the ressources corresponding to the configuration</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">you'll get two **physical machine** (not virtual)</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">the roles object is a dictionnary of the concrete compute resources</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">roles = {"server": [host1], "client": [host2] }</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #7590db;">provider</span> = G5k<span style="color: #4f97d7;">(</span>conf<span style="color: #4f97d7;">)</span>
-<span style="color: #7590db;">roles</span>, <span style="color: #7590db;">networks</span> = provider.init<span style="color: #4f97d7;">()</span>
-wait_ssh<span style="color: #4f97d7;">(</span>roles<span style="color: #4f97d7;">)</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Below is the experimentation logic</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- It installs the bare minimum to run iperf3</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'server' is used to run a iperf3 server</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">started in the background (using tmux)</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'client' connects to that server and initiate a</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">transfer for 30s (duration variable)</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- Report is printed in stdout</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #7590db;">server</span> = roles<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #4f97d7;">][</span><span style="color: #a45bad;">0</span><span style="color: #4f97d7;">]</span>
-<span style="color: #7590db;">duration</span> = <span style="color: #a45bad;">30</span>
-run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"apt update && apt install -y iperf3 tmux"</span>, roles=roles<span style="color: #4f97d7;">)</span>
-run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"tmux new-session -d 'exec iperf3 -s'"</span>, pattern_hosts=<span style="color: #2d9574;">"server"</span>, roles=roles<span style="color: #4f97d7;">)</span>
-<span style="color: #7590db;">result</span> = run_command<span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"iperf3 -c {server.address} -t {duration}"</span>, pattern_hosts=<span style="color: #2d9574;">"client"</span>, roles=roles<span style="color: #4f97d7;">)</span>
-pprint<span style="color: #4f97d7;">(</span>result<span style="color: #4f97d7;">)</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Destroy the reservation, uncomment when needed</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">provider.destroy()</span>
-</pre>
-</div>
-
-<div class="question">
-<p>
-How fast is the network between the nodes you have chosen ?
+<div id="org35d463f" class="figure">
+<p><a href="./figs/simple_compilation.png"><object type="image/svg+xml" data="./figs/simple_compilation.svg" class="org-svg">
+Sorry, your browser does not support SVG.</object></a>
</p>
-
+<p><span class="figure-number">Figure 2: </span>Simplified architecture of the system under study (blue) and the generated users (black). Overleaf is composed of a dozen services ony four are represented here.</p>
</div>
<div class="note">
<p>
-Before moving to the next questions, you'll need to clean the reservation.
-You can either uncomment the line <code>provider.destroy()</code> at the end of the script.
-You can also do it manually using the low-level <code>oarstat</code> / <code>oardel</code> tools.
-</p>
-
-<div class="org-src-container">
-<pre class="src src-bash"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">get you reservation id</span>
-$<span style="color: #7590db;">frontend</span>: oarstat -u
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">release the resources / kill the reservation</span>
-$<span style="color: #7590db;">frontend</span>: oardel <the id of the reservation goes here>
-</pre>
-</div>
-
-</div>
-
-<div class="question">
-<p>
-Can you adapt the script so that:
+After the deployment you'll be able to:
</p>
-<ol class="org-ol">
-<li>The two nodes are in two different cluster in the same site ?</li>
-<li>The two nodes are in two different sites ?</li>
-</ol>
+<ul class="org-ul">
+<li>access the web portal and play with your own overleaf instance</li>
+<li>access the web portal of your friends and collaborate on a document
+(that's not the main objective of the tutorial but that's fun)</li>
+</ul>
</div>
</div>
-</div>
-
-
-<div id="outline-container-org993bcaf" class="outline-3">
-<h3 id="org993bcaf"><span class="section-number-3">4.2</span> Let's observe in real-time what is happening</h3>
-<div class="outline-text-3" id="text-4-2">
-<div class="note">
-<p>
-Make sure you have cleaned your previous reservations.
-</p>
-
-</div>
-
-<p>
-The following script installs a monitoring stack on your nodes. This is almost
-the same script as before except the lines corresponding to the configuration
-of the monitoring stack.
-</p>
+<div id="outline-container-orgfd70908" class="outline-3">
+<h3 id="orgfd70908"><span class="section-number-3">3.1</span> Deploy it</h3>
+<div class="outline-text-3" id="text-3-1">
<div class="org-src-container">
-<pre class="src src-python"><span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.api <span style="color: #4f97d7; font-weight: bold;">import</span> run_command, wait_ssh
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.infra.enos_g5k.provider <span style="color: #4f97d7; font-weight: bold;">import</span> G5k
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.infra.enos_g5k.configuration <span style="color: #4f97d7; font-weight: bold;">import</span> Configuration, NetworkConfiguration
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.service <span style="color: #4f97d7; font-weight: bold;">import</span> Monitoring
-
-<span style="color: #4f97d7; font-weight: bold;">import</span> logging
-
-
-<span style="color: #4f97d7; font-weight: bold;">def</span> <span style="color: #bc6ec5; font-weight: bold;">pprint</span><span style="color: #4f97d7;">(</span>d<span style="color: #4f97d7;">)</span>:
- <span style="color: #2aa1ae;">"""Utils fonction to pretty print the results"""</span>
- <span style="color: #4f97d7; font-weight: bold;">for</span> k, v <span style="color: #4f97d7; font-weight: bold;">in</span> d<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"ok"</span><span style="color: #4f97d7;">]</span>.items<span style="color: #4f97d7;">()</span>:
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"Result for {k}"</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"-"</span> * <span style="color: #a45bad;">70</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"STDOUT:"</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span>v.get<span style="color: #bc6ec5;">(</span><span style="color: #2d9574;">"stdout"</span>, <span style="color: #2d9574;">""</span><span style="color: #bc6ec5;">)</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"STDERR:"</span><span style="color: #4f97d7;">)</span>
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span>v.get<span style="color: #bc6ec5;">(</span><span style="color: #2d9574;">"stderr"</span>, <span style="color: #2d9574;">""</span><span style="color: #bc6ec5;">)</span><span style="color: #4f97d7;">)</span>
-
-
-logging.basicConfig<span style="color: #4f97d7;">(</span>level=logging.INFO<span style="color: #4f97d7;">)</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Some parameters.</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Note 1: that you don't need to be on rennes frontend to use nodes</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">from rennes</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Note 2: Adapt the site/cluster according to the availibility</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">see the Gantt in https://www.grid5000.fr/w/Status</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #7590db;">SITE</span> = <span style="color: #2d9574;">"rennes"</span>
-<span style="color: #7590db;">CLUSTER</span> = <span style="color: #2d9574;">"paravance"</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Configuration object describes the resource we want</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">here: 2 machines on the same cluster using the production network</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #7590db;">network</span> = NetworkConfiguration<span style="color: #4f97d7;">(</span><span style="color: #4f97d7;">id</span>=<span style="color: #2d9574;">"n1"</span>,
- <span style="color: #4f97d7;">type</span>=<span style="color: #2d9574;">"prod"</span>,
- roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"my_network"</span><span style="color: #bc6ec5;">]</span>,
- site=SITE<span style="color: #4f97d7;">)</span>
-
-<span style="color: #7590db;">conf</span> = Configuration.from_settings<span style="color: #4f97d7;">(</span>job_name=<span style="color: #2d9574;">"enoslib_tutorial"</span>,
- job_type=<span style="color: #2d9574;">"allow_classic_ssh"</span><span style="color: #4f97d7;">)</span>\
- .add_network_conf<span style="color: #4f97d7;">(</span>network<span style="color: #4f97d7;">)</span>\
- .add_machine<span style="color: #4f97d7;">(</span>roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #bc6ec5;">]</span>,
- cluster=CLUSTER,
- nodes=<span style="color: #a45bad;">1</span>,
- primary_network=network<span style="color: #4f97d7;">)</span>\
- .add_machine<span style="color: #4f97d7;">(</span>roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #bc6ec5;">]</span>,
- cluster=CLUSTER,
- nodes=<span style="color: #a45bad;">1</span>,
- primary_network=network<span style="color: #4f97d7;">)</span>\
- .finalize<span style="color: #4f97d7;">()</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Reserve the ressources corresponding to the configuration</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">you'll get two **physical machine** (not virtual)</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">the roles object is a dictionnary of the concrete compute resources</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">roles = {"server": [host1], "client": [host2] }</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #7590db;">provider</span> = G5k<span style="color: #4f97d7;">(</span>conf<span style="color: #4f97d7;">)</span>
-<span style="color: #7590db;">roles</span>, <span style="color: #7590db;">networks</span> = provider.init<span style="color: #4f97d7;">()</span>
-wait_ssh<span style="color: #4f97d7;">(</span>roles<span style="color: #4f97d7;">)</span>
-
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">This deploys a monitoring stack. It is composed of</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- some agents on each monitored nodes</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- one collector that collects the metrics from the agents</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- one UI to visualize</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #7590db;">m</span> = Monitoring<span style="color: #4f97d7;">(</span>collector=roles<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #bc6ec5;">]</span>,
- agent=roles<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #bc6ec5;">]</span> + roles<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #bc6ec5;">]</span>,
- ui=roles<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #bc6ec5;">]</span><span style="color: #4f97d7;">)</span>
-m.deploy<span style="color: #4f97d7;">()</span>
-
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Below is the experimentation logic</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- It installs the bare minimum to run iperf3</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'server' is used to run a iperf3 server</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">started in the background (using tmux)</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- The machine with the role 'client' connects to that server and initiate a</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">transfer for 600s (duration variable)</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">- Report is printed in stdout</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #7590db;">server</span> = roles<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #4f97d7;">][</span><span style="color: #a45bad;">0</span><span style="color: #4f97d7;">]</span>
-<span style="color: #7590db;">duration</span> = <span style="color: #a45bad;">600</span>
-run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"apt update && apt install -y iperf3 tmux"</span>, roles=roles<span style="color: #4f97d7;">)</span>
-run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"tmux new-session -d 'exec iperf3 -s'"</span>, pattern_hosts=<span style="color: #2d9574;">"server"</span>, roles=roles<span style="color: #4f97d7;">)</span>
-<span style="color: #7590db;">result</span> = run_command<span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"iperf3 -c {server.address} -t {duration}"</span>, pattern_hosts=<span style="color: #2d9574;">"client"</span>, roles=roles<span style="color: #4f97d7;">)</span>
-pprint<span style="color: #4f97d7;">(</span>result<span style="color: #4f97d7;">)</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">## </span><span style="color: #2aa1ae; background-color: #292e34;">Destroy the reservation, uncomment when needed</span>
-<span style="color: #2aa1ae; background-color: #292e34;">##</span>
-<span style="color: #2aa1ae; background-color: #292e34;">######################################################################</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">provider.destroy()</span>
+<pre class="src src-bash">$<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</span>venv<span style="color: #4f97d7;">)</span>: python overleaf.py deploy --cluster=paravance
</pre>
</div>
-<p>
-Now, let's visualize the network traffic in real-time !
-</p>
<div class="note">
-<p>
-Usually I follow this to access services running inside Grid'5000:
-<a href="https://discovery.gitlabpages.inria.fr/enoslib/tutorials/grid5000.html#accessing-http-services-inside-grid-5000">https://discovery.gitlabpages.inria.fr/enoslib/tutorials/grid5000.html#accessing-http-services-inside-grid-5000</a>.
-</p>
-
-
-<p>
-Today you can just create a tunnel like this (from your local machine).
-</p>
-
-<div class="org-src-container">
-<pre class="src src-bash"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Adapt the node names with the node where grafana (the UI) has been installed</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Replace <login> by your Grid'5000 login</span>
-$<span style="color: #7590db;">yourmachine</span>: ssh -NL <span style="color: #a45bad;">3000:paravance-16.rennes.grid5000.fr:3000</span> <login>@access.grid5000.fr
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">point your browser to localhost:3000</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">username/mdp: admin/admin</span>
-</pre>
-</div>
-
-</div>
-
-<p>
-Part of the experimenter work also consists in analysing the data. Here it
-corresponds in writing the right request to monitor the traffic (check the
-Fig. <a href="#orga4a34ec">1</a>). You should be able to visualize such a thing (after a bit
-of point and clicks).
-</p>
-
-
-<div id="orga4a34ec" class="figure">
-<p><a href="figs/iperf3.png" width="100%" style="border:1px solid black;"><img src="figs/iperf3.png" alt="iperf3.png" width="100%" style="border:1px solid black;" /></a>
-</p>
-<p><span class="figure-number">Figure 1: </span>iperf3 / monitoring</p>
-</div>
-</div>
-</div>
-
-
-<div id="outline-container-org830e31f" class="outline-3">
-<h3 id="org830e31f"><span class="section-number-3">4.3</span> Discussion</h3>
-<div class="outline-text-3" id="text-4-3">
-<p>
-So, far this seems (at least for me) very handy. But there might be some problems in our setup:
-</p>
<ul class="org-ul">
-<li>we aren't isolated from the other users</li>
-<li>we aren't isolated from ourself in the sense that the monitoring stack generates its own
-network traffic (yes, this is negligible in our case)</li>
+<li>You can change the cluster name with any cluster on Grid'5000: see <a href="https://www.grid5000.fr/w/Hardware">https://www.grid5000.fr/w/Hardware</a></li>
+<li>This can take several minutes…</li>
</ul>
-<p>
-Sometimes it's desirable to have the following setup (see Fig. <a href="#orgf90cb97">2</a>).
-</p>
-
-
-<div id="orgf90cb97" class="figure">
-<p><a href="figs/skydive_enoslib.png"><img src="figs/skydive_enoslib.png" alt="skydive_enoslib.png" /></a>
-</p>
-<p><span class="figure-number">Figure 2: </span>nodes are using two network interfaces. Monitoring traffic and benchmark traffic are separated.</p>
</div>
</div>
</div>
-<div id="outline-container-org5f35d8c" class="outline-3">
-<h3 id="org5f35d8c"><span class="section-number-3">4.4</span> A bit better approach</h3>
-<div class="outline-text-3" id="text-4-4">
-<p>
-Analyse/Understand the following script <a href="exercices/iperf3_better.py">exercices/iperf3_better.py</a>
-Launch it.
-</p>
-
-<div class="note">
-<p>
-On Grid'5000, using the secondary interfaces requires to <b>deploy</b> the nodes:
-an new OS will be installed on your nodes. This will give you full control on
-the physical machine (root access). This might be longer to run the
-experiment due to this deployment phase.
-</p>
-
-</div>
-</div>
-</div>
-
-<div id="outline-container-orgdddf696" class="outline-3">
-<h3 id="orgdddf696"><span class="section-number-3">4.5</span> Ninja level</h3>
-<div class="outline-text-3" id="text-4-5">
-<p>
-Add the <a href="https://discovery.gitlabpages.inria.fr/enoslib/apidoc/service.html#skydive">Skydive</a> service to your deployment.
-It should be accessible on the port <code>8082</code> of the analyzer node. You should
-get something like Fig. <a href="#orgf90cb97">2</a>.
-</p>
-</div>
-</div>
-
-<div id="outline-container-org1193557" class="outline-3">
-<h3 id="org1193557"><span class="section-number-3">4.6</span> Some references</h3>
-<div class="outline-text-3" id="text-4-6">
-<ul class="org-ul">
-<li>Services: <a href="https://discovery.gitlabpages.inria.fr/enoslib/apidoc/service.html">https://discovery.gitlabpages.inria.fr/enoslib/apidoc/service.html</a></li>
-</ul>
-</div>
-</div>
-</div>
-
-<div id="outline-container-orgf965b96" class="outline-2">
-<h2 id="orgf965b96"><span class="section-number-2">5</span> Providers: to replicate your experiment</h2>
-<div class="outline-text-2" id="text-5">
-<div class="note">
-<p>
-The resources that are used for your experiment are acquired through a
-provider. Providers are a mean to decouple the infrastructure code (the code
-that gets the resources) from the code that runs the experiment. Changing the
-provider allows to replicate the experiment on another testbed.
-</p>
-
-</div>
-
-<p>
-Originally it was used to iterate on the code locally (using the Vagrant
-provider) and to only test on Grid'5000 when necessary.
-</p>
-
-<p>
-We now have couple of providers that you may picked or mixed.
-</p>
-</div>
-
-<div id="outline-container-orgad471c7" class="outline-3">
-<h3 id="orgad471c7"><span class="section-number-3">5.1</span> iperf3 on virtual machines on Grid'5000</h3>
-<div class="outline-text-3" id="text-5-1">
-<p>
-We'll adapt the initial iperf3 example to use virtual machines instead of
-bare-metal machine.
-</p>
-
+<div id="outline-container-orgafb102d" class="outline-3">
+<h3 id="orgafb102d"><span class="section-number-3">3.2</span> Access it</h3>
+<div class="outline-text-3" id="text-3-2">
<p>
-Note that:
+To know where your services is installed you can run:
</p>
-<ul class="org-ul">
-<li>The configuration object is different</li>
-<li>The experimentation logic is the same</li>
-<li>Some part have been rewritten using modules (see later in the dedicated section).</li>
-</ul>
-
<div class="org-src-container">
-<pre class="src src-python"><span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.api <span style="color: #4f97d7; font-weight: bold;">import</span> play_on, wait_ssh
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.infra.enos_vmong5k.provider <span style="color: #4f97d7; font-weight: bold;">import</span> VMonG5k
-<span style="color: #4f97d7; font-weight: bold;">from</span> enoslib.infra.enos_vmong5k.configuration <span style="color: #4f97d7; font-weight: bold;">import</span> Configuration
-
-<span style="color: #4f97d7; font-weight: bold;">import</span> logging
-<span style="color: #4f97d7; font-weight: bold;">import</span> os
-
-logging.basicConfig<span style="color: #4f97d7;">(</span>level=logging.DEBUG<span style="color: #4f97d7;">)</span>
-
-<span style="color: #7590db;">CLUSTER</span> = <span style="color: #2d9574;">"paravance"</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">path to the inventory</span>
-<span style="color: #7590db;">inventory</span> = os.path.join<span style="color: #4f97d7;">(</span>os.getcwd<span style="color: #bc6ec5;">()</span>, <span style="color: #2d9574;">"hosts"</span><span style="color: #4f97d7;">)</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">claim the resources</span>
-<span style="color: #7590db;">conf</span> = Configuration.from_settings<span style="color: #4f97d7;">(</span>job_name=<span style="color: #2d9574;">"enoslib_tutorial"</span>, gateway=<span style="color: #a45bad;">True</span><span style="color: #4f97d7;">)</span>\
- .add_machine<span style="color: #4f97d7;">(</span>roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #bc6ec5;">]</span>,
- cluster=CLUSTER,
- number=<span style="color: #a45bad;">1</span>,
- flavour=<span style="color: #2d9574;">"large"</span><span style="color: #4f97d7;">)</span>\
- .add_machine<span style="color: #4f97d7;">(</span>roles=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #bc6ec5;">]</span>,
- cluster=CLUSTER,
- number=<span style="color: #a45bad;">1</span>,
- flavour=<span style="color: #2d9574;">"medium"</span><span style="color: #4f97d7;">)</span>\
- .finalize<span style="color: #4f97d7;">()</span>
-
-<span style="color: #7590db;">provider</span> = VMonG5k<span style="color: #4f97d7;">(</span>conf<span style="color: #4f97d7;">)</span>
-
-<span style="color: #7590db;">roles</span>, <span style="color: #7590db;">networks</span> = provider.init<span style="color: #4f97d7;">()</span>
-wait_ssh<span style="color: #4f97d7;">(</span>roles<span style="color: #4f97d7;">)</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Below is the experimentation logic</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">It installs the bare minimum to run iperf3</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">The machine with the role 'server' is used to run a iperf3 server</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">started in the background in a tmux</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">The machine with the role 'client' connects to that server</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Report is printed in stdout</span>
-<span style="color: #7590db;">server</span> = roles<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #4f97d7;">][</span><span style="color: #a45bad;">0</span><span style="color: #4f97d7;">]</span>
-
-<span style="color: #4f97d7; font-weight: bold;">with</span> play_on<span style="color: #4f97d7;">(</span>roles=roles<span style="color: #4f97d7;">)</span> <span style="color: #4f97d7; font-weight: bold;">as</span> p:
- p.apt<span style="color: #4f97d7;">(</span>name=<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"iperf3"</span>, <span style="color: #2d9574;">"tmux"</span><span style="color: #bc6ec5;">]</span>, state=<span style="color: #2d9574;">"present"</span><span style="color: #4f97d7;">)</span>
-
-<span style="color: #4f97d7; font-weight: bold;">with</span> play_on<span style="color: #4f97d7;">(</span>pattern_hosts=<span style="color: #2d9574;">"server"</span>, roles=roles<span style="color: #4f97d7;">)</span> <span style="color: #4f97d7; font-weight: bold;">as</span> p:
- p.shell<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"tmux new-session -d 'exec iperf3 -s'"</span><span style="color: #4f97d7;">)</span>
-
-<span style="color: #4f97d7; font-weight: bold;">with</span> play_on<span style="color: #4f97d7;">(</span>pattern_hosts=<span style="color: #2d9574;">"client"</span>, roles=roles<span style="color: #4f97d7;">)</span> <span style="color: #4f97d7; font-weight: bold;">as</span> p:
- p.shell<span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"iperf3 -c {server.address} -t 30"</span><span style="color: #4f97d7;">)</span>
-
-<span style="color: #4f97d7; font-weight: bold;">with</span> play_on<span style="color: #4f97d7;">(</span>pattern_hosts=<span style="color: #2d9574;">"client"</span>, roles=roles<span style="color: #4f97d7;">)</span> <span style="color: #4f97d7; font-weight: bold;">as</span> p:
- p.shell<span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"iperf3 -c {server.address} -t 30 --logfile iperf3.out"</span><span style="color: #4f97d7;">)</span>
- p.fetch<span style="color: #4f97d7;">(</span>src=<span style="color: #2d9574;">"iperf3.out"</span>, dest=<span style="color: #2d9574;">"iperf3.out"</span><span style="color: #4f97d7;">)</span>
+<pre class="src src-bash"> $<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</span>venv<span style="color: #4f97d7;">)</span>: python overleaf.py describe
+
+ <span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Possible outputp</span>
+╒══════════════════════╤════════════╤════════╕
+│ Name │ Address │ Port │
+╞══════════════════════╪════════════╪════════╡
+│ Web portal │ <span style="color: #a45bad;">10.144.0.2</span> │ <span style="color: #a45bad;">3000</span> │
+├──────────────────────┼────────────┼────────┤
+│ Monitoring portal │ <span style="color: #a45bad;">10.144.0.2</span> │ <span style="color: #a45bad;">2000</span> │
+├──────────────────────┼────────────┼────────┤
+│ Benchmark portal │ <span style="color: #a45bad;">10.144.0.3</span> │ <span style="color: #a45bad;">8089</span> │
+├──────────────────────┼────────────┼────────┤
+│ Compilation machines │ <span style="color: #a45bad;">10.144.0.4</span> │ - │
+╘══════════════════════╧════════════╧════════╛
</pre>
</div>
+<div class="note">
<p>
-Using module using the <code>play_on</code> context manager does not bring back the
-results of the commands. Iperf3 let's you write the result of the command on
-a file. We just need to scp the file back to our local machine using the
-<code>fetch</code> module.
-</p>
-</div>
-</div>
-
-
-<div id="outline-container-org08faf88" class="outline-3">
-<h3 id="org08faf88"><span class="section-number-3">5.2</span> References</h3>
-<div class="outline-text-3" id="text-5-2">
-<ul class="org-ul">
-<li>Doc: <a href="https://discovery.gitlabpages.inria.fr/enoslib/apidoc/infra.html">https://discovery.gitlabpages.inria.fr/enoslib/apidoc/infra.html</a></li>
-<li>Sources: <a href="https://gitlab.inria.fr/discovery/enoslib/tree/v4.8.1/enoslib/infra">https://gitlab.inria.fr/discovery/enoslib/tree/v4.8.1/enoslib/infra</a></li>
-</ul>
-</div>
-</div>
-</div>
-
-
-<div id="outline-container-org56f9c08" class="outline-2">
-<h2 id="org56f9c08"><span class="section-number-2">6</span> Variables in EnOSlib</h2>
-<div class="outline-text-2" id="text-6">
-<p>
-Learn how to get 2 nodes from Grid'5000 and start launching remote commands.
+To access the web portal, you can create a tunnel from your local machine to
+the machine running the web portal as follows
</p>
-</div>
-<div id="outline-container-org2281689" class="outline-3">
-<h3 id="org2281689"><span class="section-number-3">6.1</span> Discover the <code>run</code> command and its variants</h3>
-<div class="outline-text-3" id="text-6-1">
-<p>
-Before proceeding you can add this util function to your code. It is only
-used to pretty print a python dictionnary.
-</p>
<div class="org-src-container">
-<pre class="src src-python"><span style="color: #4f97d7; font-weight: bold;">def</span> <span style="color: #bc6ec5; font-weight: bold;">pprint</span><span style="color: #4f97d7;">(</span>d<span style="color: #4f97d7;">)</span>:
- <span style="color: #4f97d7; font-weight: bold;">import</span> json
- <span style="color: #4f97d7; font-weight: bold;">print</span><span style="color: #4f97d7;">(</span>json.dumps<span style="color: #bc6ec5;">(</span>d, indent=<span style="color: #a45bad;">4</span><span style="color: #bc6ec5;">)</span><span style="color: #4f97d7;">)</span>
-</pre>
-</div>
+<pre class="src src-bash"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Adapt the node names with the node where the portal has been installed</span>
+<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Replace <login> by your Grid'5000 login</span>
+$<span style="color: #7590db;">yourmachine</span>: ssh -NL <span style="color: #a45bad;">3000:10.144.0.2:3000</span> <login>@access.grid5000.fr
-<p>
-And use the <code>enoslib.api.run</code> function
-</p>
-<div class="org-src-container">
-<pre class="src src-python"><span style="color: #7590db;">server</span> = roles<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #4f97d7;">][</span><span style="color: #a45bad;">0</span><span style="color: #4f97d7;">]</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">---</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Using run</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">--------------------------------------------------------------------</span>
-<span style="color: #7590db;">result</span> = run<span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"ping -c 5 {server.address}"</span>, roles<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #bc6ec5;">]</span><span style="color: #4f97d7;">)</span>
-pprint<span style="color: #4f97d7;">(</span>result<span style="color: #4f97d7;">)</span>
+<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">point your browser to localhost:3000</span>
+<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">username/mdp: toto@toto.com / toto4242</span>
</pre>
</div>
-<p>
-Or the <code>enoslib.api.run_command</code> function
-</p>
-<div class="org-src-container">
-<pre class="src src-python"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">---</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Using run_command 1/2</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">--------------------------------------------------------------------</span>
-<span style="color: #7590db;">result</span> = run_command<span style="color: #4f97d7;">(</span>f<span style="color: #2d9574;">"ping -c 5 {server.address}"</span>,
- pattern_hosts=<span style="color: #2d9574;">"client"</span>,
- roles=roles<span style="color: #4f97d7;">)</span>
-pprint<span style="color: #4f97d7;">(</span>result<span style="color: #4f97d7;">)</span>
-</pre>
</div>
<div class="note">
<p>
-<code>enoslib.api.run</code> is a specialisation of <code>enoslib.api.run_command</code>.
-The latter let's you use <a href="https://docs.ansible.com/ansible/latest/user_guide/intro_patterns.html">some fancy patterns</a> to determine the list of hosts to run the command on.
-</p>
-
-<p>
-And yes, it uses Ansible behind the scene.
+You can access the web portal of your friends by replacing the address of the web portal.
</p>
</div>
</div>
</div>
-
-<div id="outline-container-orgd2c7291" class="outline-3">
-<h3 id="orgd2c7291"><span class="section-number-3">6.2</span> Advanced usages</h3>
-<div class="outline-text-3" id="text-6-2">
-<div class="note">
-<p>
-For all the remote interactions, EnOSlib relies on <a href="https://docs.ansible.com/ansible/latest/index.html">Ansible</a>. Ansible
-has it own variables management system.
-For instance the task <code>Gather Facts</code> at the beginning of the previous tasks
-gathers informations about all/some remote hosts and store them in the
-Ansible management system.
-</p>
-
</div>
-<p>
-Let's see what Ansible is gathering about the hosts:
-</p>
-
+<div id="outline-container-org9752580" class="outline-2">
+<h2 id="org9752580"><span class="section-number-2">4</span> Deploy the monitoring stack</h2>
+<div class="outline-text-2" id="text-4">
<div class="org-src-container">
-<pre class="src src-python"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">---</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Gather facts</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">--------------------------------------------------------------------</span>
-<span style="color: #7590db;">result</span> = gather_facts<span style="color: #4f97d7;">(</span>roles=roles<span style="color: #4f97d7;">)</span>
-pprint<span style="color: #4f97d7;">(</span>result<span style="color: #4f97d7;">)</span>
+<pre class="src src-bash">$<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</span>venv<span style="color: #4f97d7;">)</span>: python overleaf.py monitoring
</pre>
</div>
-<div class="note">
-<p>
-EnOSlib sits in between two worlds: the Python world and the Ansible
-world. One common need is to pass a variables from one world to another.
-</p>
+<div class="question">
<ul class="org-ul">
-<li><code>enoslib.api.gather_facts</code> is a way to get, in Python, the variables known
-by Ansible about each host.</li>
-<li><code>extra_vars</code> keyword argument of <code>enoslib.api.run</code> or <code>enoslib.api.run_command</code> will
-pass variables from Python world to Ansible world (global variable)</li>
-<li>Injecting a key/value in a <code>Host.extra</code> attribute will make the variable <code>key</code> available to Ansible.
-This makes the variables Host specific.</li>
+<li>Create another tunnel and access the monitoring portal</li>
+<li>Import the dashboard [TODO]</li>
</ul>
</div>
-
-<p>
-The following inject a global variable in the Ansible world
-</p>
-<div class="org-src-container">
-<pre class="src src-python"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">---</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Passing a variable to the Ansible World using a global level variable</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">--------------------------------------------------------------------</span>
-<span style="color: #7590db;">server</span> = roles<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #4f97d7;">][</span><span style="color: #a45bad;">0</span><span style="color: #4f97d7;">]</span>
-<span style="color: #7590db;">extra_vars</span>=<span style="color: #4f97d7;">{</span><span style="color: #2d9574;">"server_ip"</span>: server.address<span style="color: #4f97d7;">}</span>
-<span style="color: #7590db;">result</span> = run<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"ping -c 5 {{ server_ip }}"</span>, roles<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #bc6ec5;">]</span>, extra_vars=extra_vars<span style="color: #4f97d7;">)</span>
-</pre>
-</div>
-</div>
-</div>
-
-<div id="outline-container-orgd78a6d8" class="outline-3">
-<h3 id="orgd78a6d8"><span class="section-number-3">6.3</span> Ninja level</h3>
-<div class="outline-text-3" id="text-6-3">
-<p>
-The following is valid and inject in the <code>client</code> host a specific variable to
-keep track of the server IP.
-</p>
-
-<div class="org-src-container">
-<pre class="src src-python"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">---</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Passing a variable to the Ansible World using a host level variable</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">--------------------------------------------------------------------</span>
-<span style="color: #7590db;">server</span> = roles<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"server"</span><span style="color: #4f97d7;">][</span><span style="color: #a45bad;">0</span><span style="color: #4f97d7;">]</span>
-<span style="color: #7590db;">client</span> = roles<span style="color: #4f97d7;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #4f97d7;">][</span><span style="color: #a45bad;">0</span><span style="color: #4f97d7;">]</span>
-client.extra.update<span style="color: #4f97d7;">(</span>server_ip=server.address<span style="color: #4f97d7;">)</span>
-<span style="color: #7590db;">result</span> = run<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"ping -c 5 {{ server_ip }}"</span>, roles<span style="color: #bc6ec5;">[</span><span style="color: #2d9574;">"client"</span><span style="color: #bc6ec5;">]</span><span style="color: #4f97d7;">)</span>
-</pre>
-</div>
-
-<div class="note">
-<p>
-Host level variables are interesting to introduce some dissymetry between
-hosts while still using one single command to reach all of them.
-</p>
-
</div>
-
-<div class="question">
-<p>
-How to perform simultaneously the ping to the other machine in calling only
-once <code>run</code> or <code>run_command</code> and using host level variables?
-</p>
-
</div>
-<div class="question">
-<p>
-We'd like to create 5 <code>server</code> machines and 5 <code>client</code> machines and start 5
-<b>parallel</b> streams of data using <code>iperf3</code>. To answer this we'll need to learn
-a bit more on how variables are handled in EnOSlib.
-</p>
-
-</div>
-</div>
-</div>
-
-<div id="outline-container-orge739be4" class="outline-3">
-<h3 id="orge739be4"><span class="section-number-3">6.4</span> Putting all together</h3>
-<div class="outline-text-3" id="text-6-4">
+<div id="outline-container-orgd7c542a" class="outline-2">
+<h2 id="orgd7c542a"><span class="section-number-2">5</span> Benchmark the system</h2>
+<div class="outline-text-2" id="text-5">
<p>
-Access the full file: <a href="exercices/run.py">exercices/run.py</a>
+Benchmarking will consist in generating some load on the system.
+This will be made in two steps:
</p>
-</div>
-</div>
-
-<div id="outline-container-org5dafa08" class="outline-3">
-<h3 id="org5dafa08"><span class="section-number-3">6.5</span> Some references</h3>
-<div class="outline-text-3" id="text-6-5">
<ul class="org-ul">
-<li>G5k configuration schema: <a href="https://discovery.gitlabpages.inria.fr/enoslib/apidoc/infra.html#g5k-schema">https://discovery.gitlabpages.inria.fr/enoslib/apidoc/infra.html#g5k-schema</a></li>
-<li>API Reference: <a href="https://discovery.gitlabpages.inria.fr/enoslib/apidoc/api.html">https://discovery.gitlabpages.inria.fr/enoslib/apidoc/api.html</a></li>
+<li>First, we'll install a benchmarking portal and some agent responsible
+for generating the load</li>
+<li>Second, from the portal we'll generate some load</li>
</ul>
-</div>
-</div>
-</div>
-
-<div id="outline-container-orga167566" class="outline-2">
-<h2 id="orga167566"><span class="section-number-2">7</span> Modules: for safer remote actions</h2>
-<div class="outline-text-2" id="text-7">
-<p>
-In this section we'll discover the idiomatic way of managing resources on the
-remote hosts. A resource can be anything: a user, a file, a line in a file, a
-repo on Gitlab, a firewall rule …
-</p>
-</div>
-
-
-<div id="outline-container-orgc30170a" class="outline-3">
-<h3 id="orgc30170a"><span class="section-number-3">7.1</span> Idempotency</h3>
-<div class="outline-text-3" id="text-7-1">
-<p>
-Let's assume you want to create a user (<code>foo</code>). With the <code>run_command</code> this would look like:
-</p>
-
-<div class="org-src-container">
-<pre class="src src-python">run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"useradd -m foo"</span>, roles=role<span style="color: #4f97d7;">)</span>
-</pre>
-</div>
-
-<p>
-The main issue with this code is that it is not <b>idempotent</b>. Running it once
-will applied the effect (create the user). But, as soon as the user exist in
-the system, this will raise an error.
-</p>
-</div>
-</div>
-
-<div id="outline-container-org334a244" class="outline-3">
-<h3 id="org334a244"><span class="section-number-3">7.2</span> One reason why idempotency is important</h3>
-<div class="outline-text-3" id="text-7-2">
-<p>
-Let's consider the following snippet (mispelling the second command is intentional)
-</p>
-<div class="org-src-container">
-<pre class="src src-python">run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"useradd -m foo"</span>, roles=role<span style="color: #4f97d7;">)</span>
-run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"mkdirz plop"</span><span style="color: #4f97d7;">)</span>
-</pre>
-</div>
-<p>
-Executing the above leads the system with the user <code>foo</code> created but the the
-directory <code>plop</code> not created since the second command fails.
-</p>
<p>
-So what you want to do is to fix the second command and re-run the snippet again.
-But, you can't do that because <code>useradd</code> isn't idempotent.
+The load will consist in (many) users creating a project and compiling it once
+before destroying it
</p>
</div>
-</div>
-
-<div id="outline-container-orgcf77118" class="outline-3">
-<h3 id="orgcf77118"><span class="section-number-3">7.3</span> Idempotency trick</h3>
-<div class="outline-text-3" id="text-7-3">
-<p>
-One easy solution is to protect your call to non idempotent commands with
-some ad'hoc tricks
-</p>
-
-<p>
-Here it can look like this:
-</p>
+<div id="outline-container-org9c6c9c3" class="outline-3">
+<h3 id="org9c6c9c3"><span class="section-number-3">5.1</span> Deploy the benchmarking nodes</h3>
+<div class="outline-text-3" id="text-5-1">
<div class="org-src-container">
-<pre class="src src-python">run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"id foo || useradd -m foo"</span>, roles=role<span style="color: #4f97d7;">)</span>
-run_command<span style="color: #4f97d7;">(</span><span style="color: #2d9574;">"mkdir -p plop"</span><span style="color: #4f97d7;">)</span>
+<pre class="src src-bash">$<span style="color: #7590db;">frontend</span><span style="color: #4f97d7;">(</span>venv<span style="color: #4f97d7;">)</span>: python overleaf.py bench
</pre>
</div>
-<p>
-<b>What's wrong with that</b>
-</p>
-
+<div class="question">
<ul class="org-ul">
-<li>The trick depends on the command</li>
-<li>Re-reading the code is more complex: the code focus on the <b><b>how</b></b> not the <b><b>what</b></b></li>
+<li>Create another tunnel and access the benchmarking portal</li>
+<li>Generate a small load (1 user)</li>
</ul>
+
</div>
</div>
-
-<div id="outline-container-orgcaa5aa0" class="outline-3">
-<h3 id="orgcaa5aa0"><span class="section-number-3">7.4</span> General idempotency</h3>
-<div class="outline-text-3" id="text-7-4">
-<p>
-The idiomatic solution is to use modules (inherited from the Ansible
-Modules). The modules are specified in a <b>declarative</b> way and they ensure
-<b>idempotency</b> for most of them.
-</p>
-
-<p>
-So rewriting the example with modules looks like:
-</p>
-<div class="org-src-container">
-<pre class="src src-python"><span style="color: #4f97d7; font-weight: bold;">with</span> play_on<span style="color: #4f97d7;">(</span>roles=roles<span style="color: #4f97d7;">)</span> <span style="color: #4f97d7; font-weight: bold;">as</span> p:
- p.user<span style="color: #4f97d7;">(</span>name=<span style="color: #2d9574;">"foo"</span>, state=<span style="color: #2d9574;">"present"</span>, create_home=<span style="color: #2d9574;">"yes"</span><span style="color: #4f97d7;">)</span>
- p.<span style="color: #4f97d7;">file</span><span style="color: #4f97d7;">(</span>name=<span style="color: #2d9574;">"plop"</span>, state=<span style="color: #2d9574;">"directory"</span><span style="color: #4f97d7;">)</span>
-</pre>
</div>
+<div id="outline-container-orge88815f" class="outline-3">
+<h3 id="orge88815f"><span class="section-number-3">5.2</span> Observations</h3>
+<div class="outline-text-3" id="text-5-2">
<p>
-<code>enoslib.api.play_on</code> is the entry point to the module system.
-</p>
-
-<p>
-You can run this code as many times as you want without any error. You'll
-eventually find one user <code>foo</code> and one directory <code>plop</code> in your target
-systems.
-</p>
-
-<p>
-They are more than 2500 modules: <a href="https://docs.ansible.com/ansible/latest/modules/list_of_all_modules.html">https://docs.ansible.com/ansible/latest/modules/list_of_all_modules.html</a>
-</p>
-
-<p>
-If you can't find what you want you must know that:
+Observations can be made on two portals:
</p>
<ul class="org-ul">
-<li>Writing your own module is possible</li>
-<li>Falling back to the idempotency trick is reasonable</li>
+<li>the benchmarking portal: you'll get some information on the requests sent
+to the server</li>
+<li>the monitorig portal: you'll get some information about the resource
+consumption and some application metrics</li>
</ul>
-</div>
-</div>
-</div>
-
-<div id="outline-container-orgb39fe8f" class="outline-2">
-<h2 id="orgb39fe8f"><span class="section-number-2">8</span> Tasks: to organize your experiment</h2>
-<div class="outline-text-2" id="text-8">
-<p>
-To discover the Task API, head to <a href="https://discovery.gitlabpages.inria.fr/enoslib/tutorials/using-tasks.html">https://discovery.gitlabpages.inria.fr/enoslib/tutorials/using-tasks.html</a>.
-</p>
-
-<p>
-The examples are written for Vagrant but may be changed to whatever provider you like/have.
-</p>
<div class="question">
-<p>
-Adapt the <code>iperf3</code> example to provide a command line
-</p>
<ul class="org-ul">
-<li><p>
-Either using G5k physical machines:
-</p>
-<div class="org-src-container">
-<pre class="src src-bash"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">deploy the dependencies of the experimentation using the G5k provider</span>
-myperf g5k
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">launch a performance measurement</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">ideally exposes all the iperf3 client options there ;)</span>
-myperf bench -t <span style="color: #a45bad;">120</span>
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Backup the reports / influxdb database</span>
-myperf backup
-</pre>
-</div>
-
-<p>
-myperf destroy
-</p></li>
-
-<li><p>
-Either using the virtual machines on Grid'5000:
-</p>
-<div class="org-src-container">
-<pre class="src src-bash"><span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">deploy the dependencies of the experimentation using the G5k provider</span>
-myperf vm5k
-
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">Subsequent command line should be the same as above</span>
-<span style="color: #2aa1ae; background-color: #292e34;"># </span><span style="color: #2aa1ae; background-color: #292e34;">enjoy :)</span>
-</pre>
-</div></li>
+<li>Can you see the effect of the load ?</li>
+<li>Scale the workload by starting 10 users
+<ul class="org-ul">
+<li>what kind of metrics seem impacted on the monitoring portal ?</li>
+</ul></li>
+<li>the projects that are compiled aren't all identical, can you characterize them ?
+(in term of CPU/IO … consumed)</li>
</ul>
</div>
</div>
</div>
</div>
+</div>
<div id="postamble" class="status">
<p class="author">Author: Matthieu Simonin</p>
-<p class="date">Created: 2019-11-15 ven. 14:14</p>
+<p class="date">Created: 2019-11-29 ven. 01:18</p>
<p class="validation"><a href="http://validator.w3.org/check?uri=referer">Validate</a></p>
</div>
</body>
diff --git a/public/tuto1/index.org b/public/tuto1/index.org
index 777996da50565341ddd8ed47ba611b9e44dd874c..7f360c29dc257824d078e88b6607ccc46911cf13 100644
--- a/public/tuto1/index.org
+++ b/public/tuto1/index.org
@@ -14,415 +14,179 @@
#+MACRO: doc_api https://discovery.gitlabpages.inria.fr/enoslib/apidoc/api.html
#+MACRO: doc_services https://discovery.gitlabpages.inria.fr/enoslib/apidoc/service.html
-* Foreword
-** Existing tools (Grid'5000)
-
- - {{{enoslib}}} falls under the **Experiment management tools** of the following
- list:
- https://www.grid5000.fr/w/Grid5000:Software
+* Benchmarking a real application
+
+ In this tutorial we'll cover some aspects of evaluating the performance of a
+ real application. We'll work with ~overleaf~. ~overleaf~ is a collaborative
+ text editor that uses Latex to produce pdf files. Figure [[overleaf]]
+ is an overview of the editing part of the software.
+
+ #+NAME: overleaf
+ #+CAPTION: Overview of ~overleaf~ editor: on the left users can collaboratively edit the document.
+ #+CAPTION: On the right the document is rendered.
+ [[file:./figs/overleaf-v2-editor.png][file:./figs/overleaf-v2-editor.png]]
- - {{{enoslib}}} can target Grid'5000 but also other testbeds (Chameleon, local machines...)
- - {{{enoslib}}} provides high level constructs to help you with your experiments
+ Here is the plan:
-** EnOSlib quicktour
-
- - Documentation: https://discovery.gitlabpages.inria.fr/enoslib/index.html
- - Source: https://gitlab.inria.fr/discovery/enoslib
- - Reach us on:
- + https://framateam.org/enoslib
- + https://gitlab.inria.fr/discovery/enoslib/issues
+ - **Deployment** You'll first deploy our own ~overleaf~ instance (we don't want to use
+ the official/commercial instance).
+ - **Load generation** You'll generate compilation of different projects programatically.
+ - **Observation** You'll observe the effect of the load in the running system through various metrics.
+ - **Feedback** You'll formulate some hypothesis on the load characteristics and the observed effects on the system.
-** Contributing
+* Before you start
- *Before experimenting*
+ Make sure you are ok with the following.
- - Tell us what your plans are:
- + There might be already users doing similar thing
- + There might be some missing/hidden pieces in the library you might need
+** Grid'5000 stuffs
- *While experimenting*
+ #+begin_note
+ Make sure you are familiar with the Grid'5000 architecture. see section 1 & 2 of
+ https://www.grid5000.fr/w/Getting_Started. note that we won't do this tutorial
+ we'll prefer to use higher level tools for now.
+ #+end_note
- - Write bug reports / ask questions
- - Fix bugs / add your features
- *After experimenting*
+** Setup on Grid'5000
- - Give your feedback
- - Add yourself to the list: https://discovery.gitlabpages.inria.fr/enoslib/theyuseit.html
+ Connect to a Grid'5000 frontend of your choice (e.g rennes, nancy ...)
+ - create a new directory to host all the scripts of the session
+ - bootstrap a new python3 virtualenv
+ - install {{{enoslib}}} and configure the access to the API
-* Before you start
+ #+BEGIN_SRC bash :noeval
+ $frontend: cp -r ~msimonin/public/ccs-g5k-tuto2 .
+ $frontend: cd ccs-g5k-tuto2
+ $frontend: virtualenv --python=python3 venv
+ $frontend: source venv/bin/activate
+ $frontend(venv): pip install -r requirements.txt
+ $frontend(venv): echo '
+ verify_ssl: False
+ ' > ~/.python-grid5000.yaml
+ #+END_SRC
+
+* Deployment time !
+
+ Figure [[architecture]] represents a simplified view of what we'll deploy. In blue
+ some services of ~overleaf~ are represented. First the Web portal is the entry
+ point to all the user requests. The three other services in the picture are
+ involved when compiling a document. The compilation service interacts with the
+ filestore (where the files of the image of the projects are stored) and the
+ docstore (where the text of the project is stored). There are other service
+ involved to provide the chat feature, history feature, real-time interaction
+ ... but we'll focus ony on the compilation process.
+
+ #+NAME: architecture
+ #+CAPTION: Simplified architecture of the system under study (blue) and the generated users (black).
+ #+CAPTION: Overleaf is composed of a dozen services ony four are represented here.
+ [[file:./figs/simple_compilation.png][file:./figs/simple_compilation.svg]]
#+begin_note
- make sure you are familiar with the grid'5000 architecture. see section 1 & 2 of
- https://www.grid5000.fr/w/Getting_Started. note that we won't do this tutorial
- we'll prefer to use higher level tools for now.
+ After the deployment you'll be able to:
+ - access the web portal and play with your own overleaf instance
+ - access the web portal of your friends and collaborate on a document
+ (that's not the main objective of the tutorial but that's fun)
#+end_note
-* Setup on Grid'5000
-
- Connect to a Grid'5000 frontend of your choice (e.g rennes, nancy ...)
-
- - create a new directory to host all the scripts of the session
- - bootstrap a new python3 virtualenv
- - install {{{enoslib}}} and configure the access to the API
+** Deploy it
#+BEGIN_SRC bash :noeval
- $frontend: mkdir enoslib_seminar
- $frontend: cd enoslib_seminar
- $frontend: virtualenv --python=python3 venv
- $frontend: source venv/bin/activate
- $frontend(venv): pip install enoslib
- $frontend(venv): echo '
- verify_ssl: False
- ' > ~/.python-grid5000.yaml
+ $frontend(venv): python overleaf.py deploy --cluster=paravance
#+END_SRC
-* Your first experiment on Grid'5000
-
- Let's experiment with [[https://iperf.fr/][iperf3]]: a network bandwidth measuring tool. The goal is
- to deploy a simple benchmark between two hosts.
-
- We'll also instrument the deployment in order to visualize in real-time the
- network traffic between the hosts. Since this is super common, {{{enoslib}}}
- exposes a /monitoring service/ that lets you deploy very quickly what is
- needed.
-
-** First iteration
-
- We consider the following script
- #+INCLUDE: exercices/iperf3.py src python
-
- #+BEGIN_question
- How fast is the network between the nodes you have chosen ?
- #+END_question
-
- #+BEGIN_note
- Before moving to the next questions, you'll need to clean the reservation.
- You can either uncomment the line ~provider.destroy()~ at the end of the script.
- You can also do it manually using the low-level ~oarstat~ / ~oardel~ tools.
-
- #+BEGIN_SRC bash :noeval
- # get you reservation id
- $frontend: oarstat -u
- # release the resources / kill the reservation
- $frontend: oardel <the id of the reservation goes here>
- #+END_SRC
- #+END_note
-
- #+BEGIN_question
- Can you adapt the script so that:
- 1. The two nodes are in two different cluster in the same site ?
- 2. The two nodes are in two different sites ?
- #+END_question
-
-
-** Let's observe in real-time what is happening
-
- #+BEGIN_note
- Make sure you have cleaned your previous reservations.
- #+END_note
-
- The following script installs a monitoring stack on your nodes. This is almost
- the same script as before except the lines corresponding to the configuration
- of the monitoring stack.
-
- #+INCLUDE: exercices/iperf3_monitoring.py src python
-
- Now, let's visualize the network traffic in real-time !
- #+BEGIN_note
- Usually I follow this to access services running inside Grid'5000:
- {{{doc_external_access}}}.
-
-
- Today you can just create a tunnel like this (from your local machine).
-
- #+BEGIN_SRC bash :noeval
- # Adapt the node names with the node where grafana (the UI) has been installed
- # Replace <login> by your Grid'5000 login
- $yourmachine: ssh -NL 3000:paravance-16.rennes.grid5000.fr:3000 <login>@access.grid5000.fr
-
- # point your browser to localhost:3000
- # username/mdp: admin/admin
- #+END_SRC
-
- #+END_note
-
- Part of the experimenter work also consists in analysing the data. Here it
- corresponds in writing the right request to monitor the traffic (check the
- Fig. [[fig:iperf3]]). You should be able to visualize such a thing (after a bit
- of point and clicks).
-
- #+CAPTION: iperf3 / monitoring
- #+NAME: fig:iperf3
- #+ATTR_HTML: :width 100% :style border:1px solid black;
- [[file:figs/iperf3.png][file:figs/iperf3.png]]
-
-
-** Discussion
-
- So, far this seems (at least for me) very handy. But there might be some problems in our setup:
- - we aren't isolated from the other users
- - we aren't isolated from ourself in the sense that the monitoring stack generates its own
- network traffic (yes, this is negligible in our case)
-
- Sometimes it's desirable to have the following setup (see Fig. [[fig:two_networks]]).
-
- #+CAPTION: nodes are using two network interfaces.
- #+CAPTION: Monitoring traffic and benchmark traffic are separated.
- #+NAME: fig:two_networks
- [[file:figs/skydive_enoslib.png][file:figs/skydive_enoslib.png]]
+ #+begin_note
+ - You can change the cluster name with any cluster on Grid'5000: see https://www.grid5000.fr/w/Hardware
+ - This can take several minutes...
+ #+end_note
-** A bit better approach
+** Access it
- Analyse/Understand the following script [[file:exercices/iperf3_better.py]]
- Launch it.
+ To know where your services is installed you can run:
- #+BEGIN_note
- On Grid'5000, using the secondary interfaces requires to *deploy* the nodes:
- an new OS will be installed on your nodes. This will give you full control on
- the physical machine (root access). This might be longer to run the
- experiment due to this deployment phase.
- #+END_note
+ #+BEGIN_SRC bash :noeval
+ $frontend(venv): python overleaf.py describe
+
+ # Possible outputp
+╒══════════════════════╤════════════╤════════╕
+│ Name │ Address │ Port │
+╞══════════════════════╪════════════╪════════╡
+│ Web portal │ 10.144.0.2 │ 3000 │
+├──────────────────────┼────────────┼────────┤
+│ Monitoring portal │ 10.144.0.2 │ 2000 │
+├──────────────────────┼────────────┼────────┤
+│ Benchmark portal │ 10.144.0.3 │ 8089 │
+├──────────────────────┼────────────┼────────┤
+│ Compilation machines │ 10.144.0.4 │ - │
+╘══════════════════════╧════════════╧════════╛
+ #+END_SRC
-** Ninja level
+ #+BEGIN_note
+ To access the web portal, you can create a tunnel from your local machine to
+ the machine running the web portal as follows
- Add the [[https://discovery.gitlabpages.inria.fr/enoslib/apidoc/service.html#skydive][Skydive]] service to your deployment.
- It should be accessible on the port ~8082~ of the analyzer node. You should
- get something like Fig. [[fig:two_networks]].
+ #+BEGIN_SRC bash :noeval
+ # Adapt the node names with the node where the portal has been installed
+ # Replace <login> by your Grid'5000 login
+ $yourmachine: ssh -NL 3000:10.144.0.2:3000 <login>@access.grid5000.fr
-** Some references
-
- - Services: {{{doc_services}}}
-
-* Providers: to replicate your experiment
+ # point your browser to localhost:3000
+ # username/mdp: toto@toto.com / toto4242
+ #+END_SRC
+ #+END_note
#+BEGIN_note
- The resources that are used for your experiment are acquired through a
- provider. Providers are a mean to decouple the infrastructure code (the code
- that gets the resources) from the code that runs the experiment. Changing the
- provider allows to replicate the experiment on another testbed.
+ You can access the web portal of your friends by replacing the address of the web portal.
#+END_note
- Originally it was used to iterate on the code locally (using the Vagrant
- provider) and to only test on Grid'5000 when necessary.
-
- We now have couple of providers that you may picked or mixed.
-
-** iperf3 on virtual machines on Grid'5000
-
- We'll adapt the initial iperf3 example to use virtual machines instead of
- bare-metal machine.
-
- Note that:
-
- - The configuration object is different
- - The experimentation logic is the same
- - Some part have been rewritten using modules (see later in the dedicated section).
-
- #+INCLUDE: exercices/iperf3_vms.py src python
-
- Using module using the ~play_on~ context manager does not bring back the
- results of the commands. Iperf3 let's you write the result of the command on
- a file. We just need to scp the file back to our local machine using the
- ~fetch~ module.
-
-
-** References
-
- - Doc: {{{doc_provider}}}
- - Sources: {{{src_provider}}}
-
-
-* Variables in {{{enoslib}}}
-
- Learn how to get 2 nodes from Grid'5000 and start launching remote commands.
-
-** Discover the ~run~ command and its variants
-
- Before proceeding you can add this util function to your code. It is only
- used to pretty print a python dictionnary.
- #+INCLUDE: exercices/run.py :lines "35-39" src python
-
- And use the ~enoslib.api.run~ function
- #+INCLUDE: exercices/run.py :lines "40-47" src python
-
- Or the ~enoslib.api.run_command~ function
- #+INCLUDE: exercices/run.py :lines "48-56" src python
-
- #+BEGIN_note
- ~enoslib.api.run~ is a specialisation of ~enoslib.api.run_command~.
- The latter let's you use [[https://docs.ansible.com/ansible/latest/user_guide/intro_patterns.html][some fancy patterns]] to determine the list of hosts to run the command on.
-
- And yes, it uses Ansible behind the scene.
- #+END_note
-
-** Advanced usages
-
- #+BEGIN_note
- For all the remote interactions, {{{enoslib}}} relies on [[https://docs.ansible.com/ansible/latest/index.html][Ansible]]. Ansible
- has it own variables management system.
- For instance the task ~Gather Facts~ at the beginning of the previous tasks
- gathers informations about all/some remote hosts and store them in the
- Ansible management system.
- #+END_note
-
- Let's see what Ansible is gathering about the hosts:
-
- #+INCLUDE: exercices/run.py :lines "58-65" src python
-
- #+BEGIN_note
- {{{enoslib}}} sits in between two worlds: the Python world and the Ansible
- world. One common need is to pass a variables from one world to another.
- - ~enoslib.api.gather_facts~ is a way to get, in Python, the variables known
- by Ansible about each host.
- - ~extra_vars~ keyword argument of ~enoslib.api.run~ or ~enoslib.api.run_command~ will
- pass variables from Python world to Ansible world (global variable)
- - Injecting a key/value in a ~Host.extra~ attribute will make the variable ~key~ available to Ansible.
- This makes the variables Host specific.
- #+END_note
-
- The following inject a global variable in the Ansible world
- #+INCLUDE: exercices/run.py :lines "65-71" src python
-
-** Ninja level
-
- The following is valid and inject in the ~client~ host a specific variable to
- keep track of the server IP.
-
- #+INCLUDE: exercices/run.py :lines "73-81" src python
-
- #+BEGIN_note
- Host level variables are interesting to introduce some dissymetry between
- hosts while still using one single command to reach all of them.
- #+END_note
-
- #+BEGIN_question
- How to perform simultaneously the ping to the other machine in calling only
- once ~run~ or ~run_command~ and using host level variables?
- #+END_question
-
- #+BEGIN_question
- We'd like to create 5 ~server~ machines and 5 ~client~ machines and start 5
- *parallel* streams of data using ~iperf3~. To answer this we'll need to learn
- a bit more on how variables are handled in {{{enoslib}}}.
- #+END_question
-
-** Putting all together
- Access the full file: [[file:exercices/run.py]]
-
-** Some references
-
- - G5k configuration schema: {{{doc_g5k_schema}}}
- - API Reference: {{{doc_api}}}
-
-* Modules: for safer remote actions
+* Deploy the monitoring stack
- In this section we'll discover the idiomatic way of managing resources on the
- remote hosts. A resource can be anything: a user, a file, a line in a file, a
- repo on Gitlab, a firewall rule ...
-
-
-** Idempotency
-
- Let's assume you want to create a user (~foo~). With the ~run_command~ this would look like:
-
- #+BEGIN_SRC python :noeval
- run_command("useradd -m foo", roles=role)
- #+END_SRC
-
- The main issue with this code is that it is not *idempotent*. Running it once
- will applied the effect (create the user). But, as soon as the user exist in
- the system, this will raise an error.
-
-** One reason why idempotency is important
-
- Let's consider the following snippet (mispelling the second command is intentional)
- #+BEGIN_SRC python :noeval
- run_command("useradd -m foo", roles=role)
- run_command("mkdirz plop")
+ #+BEGIN_SRC bash :noeval
+ $frontend(venv): python overleaf.py monitoring
#+END_SRC
- Executing the above leads the system with the user ~foo~ created but the the
- directory ~plop~ not created since the second command fails.
-
- So what you want to do is to fix the second command and re-run the snippet again.
- But, you can't do that because ~useradd~ isn't idempotent.
-** Idempotency trick
+ #+begin_question
+ - Create another tunnel and access the monitoring portal
+ - Import the dashboard [TODO]
+ #+end_question
- One easy solution is to protect your call to non idempotent commands with
- some ad'hoc tricks
+* Benchmark the system
- Here it can look like this:
+ Benchmarking will consist in generating some load on the system.
+ This will be made in two steps:
+ - First, we'll install a benchmarking portal and some agent responsible
+ for generating the load
+ - Second, from the portal we'll generate some load
- #+BEGIN_SRC python :noeval
- run_command("id foo || useradd -m foo", roles=role)
- run_command("mkdir -p plop")
- #+END_SRC
+ The load will consist in (many) users creating a project and compiling it once
+ before destroying it
- *What's wrong with that*
+** Deploy the benchmarking nodes
- - The trick depends on the command
- - Re-reading the code is more complex: the code focus on the **how** not the **what**
+ #+BEGIN_SRC bash :noeval
+ $frontend(venv): python overleaf.py bench
+ #+END_SRC
-** General idempotency
-
- The idiomatic solution is to use modules (inherited from the Ansible
- Modules). The modules are specified in a *declarative* way and they ensure
- *idempotency* for most of them.
+ #+begin_question
+ - Create another tunnel and access the benchmarking portal
+ - Generate a small load (1 user)
+ #+end_question
- So rewriting the example with modules looks like:
- #+BEGIN_SRC python :noeval
- with play_on(roles=roles) as p:
- p.user(name="foo", state="present", create_home="yes")
- p.file(name="plop", state="directory")
- #+END_SRC
+** Observations
- ~enoslib.api.play_on~ is the entry point to the module system.
-
- You can run this code as many times as you want without any error. You'll
- eventually find one user ~foo~ and one directory ~plop~ in your target
- systems.
-
- They are more than 2500 modules: https://docs.ansible.com/ansible/latest/modules/list_of_all_modules.html
-
- If you can't find what you want you must know that:
- - Writing your own module is possible
- - Falling back to the idempotency trick is reasonable
-
-* Tasks: to organize your experiment
-
- To discover the Task API, head to {{{doc_tasks}}}.
-
- The examples are written for Vagrant but may be changed to whatever provider you like/have.
-
- #+BEGIN_question
- Adapt the ~iperf3~ example to provide a command line
- - Either using G5k physical machines:
- #+BEGIN_SRC bash
- # deploy the dependencies of the experimentation using the G5k provider
- myperf g5k
-
- # launch a performance measurement
- # ideally exposes all the iperf3 client options there ;)
- myperf bench -t 120
-
- # Backup the reports / influxdb database
- myperf backup
- #+END_SRC
-
- # Destroy the ressources on Grid'5000
- myperf destroy
-
- - Either using the virtual machines on Grid'5000:
- #+BEGIN_SRC bash
- # deploy the dependencies of the experimentation using the G5k provider
- myperf vm5k
-
- # Subsequent command line should be the same as above
- # enjoy :)
- #+END_SRC
- #+END_question
+ Observations can be made on two portals:
+ - the benchmarking portal: you'll get some information on the requests sent
+ to the server
+ - the monitorig portal: you'll get some information about the resource
+ consumption and some application metrics
+
+ #+begin_question
+ - Can you see the effect of the load ?
+ - Scale the workload by starting 10 users
+ + what kind of metrics seem impacted on the monitoring portal ?
+ - the projects that are compiled aren't all identical, can you characterize them ?
+ (in term of CPU/IO ... consumed)
+ #+end_question