Commit 2702f759 authored by Romain Olivo's avatar Romain Olivo

doc

parent 6b5df7a4
Pipeline #88512 passed with stages
in 4 minutes and 30 seconds
API
===
.. automodule:: distem.api
.. automodule::
:members:
:undoc-members:
......@@ -11,16 +11,17 @@ This tutorial show how the use of Distem, a provider that prepare containers for
Prerequisites
-------------
First of all, you have to reserve some nodes on Grid'5000, deploy a system on the nodes and then use the distem bootstrap.
First of all, you have to reserve some nodes on Grid'5000, deploy a system on the nodes.
.. code-block:: bash
$ oarsub -t deploy -l "slash_22=1+nodes=2, walltime=02:00:00" -I"
$ oarsub -t deploy -l "slash_22=1+nodes=2, walltime=02:00:00" -I
$ kadeploy3 -f $OAR_NODE_FILE -e debian9-x64-nfs -k
$ distem-bootstrap --debian-version stretch
Get image for tutorial
----------------------
Then us the distem bootstrap:
.. code-block:: bash
$ distem-bootstrap --debian-version stretch
You have to download the image in your home directory.
......@@ -33,9 +34,10 @@ Initialisation
To process to any experiments, you need to initialize nodes as the folowing code.
.. literalinclude:: tutorials/scp_vsync/initialisation.py
:language: python
:linenos:
.. literalinclude:: scp_rsync/initialisation.py
:language: python
:linenos:
Experiment
----------
......@@ -43,7 +45,7 @@ Experiment
The experiment can be launched from the coordinator node as root.
Below, the python code of the experiment.
.. literalinclude:: tutorials/scp_vsync/experience.py
:language: python
:lineos:
.. literalinclude:: scp_rsync/experiment.py
:language: python
:linenos:
import logging
import time
import os
from statistics import mean, stdev
from distem import Distem
if __name__ == "__main__":
# The path to the compressed filesystem image
# We can point to local file since our homedir is available from NFS
FSIMG = "file:///home/msimonin/public/distem-fs-jessie.tar.gz"
# Put the physical machines that have been assigned to you
# You can get that by executing: cat $OAR_NODE_FILE | uniq
PNODES = ["parapide-10.rennes.grid5000.fr", "parapide-11.rennes.grid5000.fr"]
# The first argument of the script is the address (in CIDR format)
# of the virtual network to set-up in our platform
# This ruby hash table describes our virtual network
vnet = {
'name': 'mynetworkk',
'address': "10.158.0.0/22"
}
nodelist = ['node-1', 'node-2']
latencies = ['0ms', '20ms', '40ms', '60ms']
results = {
'scp': {},
'rsync': {}
}
node1 = {
'name': nodelist[0],
'address': []
}
node2 = {
'name': nodelist[1],
'address': []
}
ifname = 'if0'
iteration = 5
# Connect to the Distem server (on http://localhost:4567 by default)
distem = Distem()
# Getting the -automatically affected- address of each virtual nodes
# virtual network interfaces
node1['address'] = distem.viface_info(node1['name'], ifname)['address'].split("/")[0]
node2['address'] = distem.viface_info(node2['name'], ifname)['address'].split("/")[0]
# Creating the files we will use in our experimentation
distem.vnode_execute(node1['name'],
'mkdir -p /tmp/src ; cd /tmp/src ; \
for i in `seq 1 100`; do \
dd if=/dev/zero of=$i bs=1K count=50; \
done'
)
# Printing the current latency
start_time = time.time()
distem.vnode_execute(node1['name'], 'hsjvdjq')
print("Latency without any limitations # %s" %(str(time.time() - start_time)))
desc = {
'output': {
'latency': {
'delay': []
}
}
}
# Starting our experiment for each specified latencies
print('Starting tests')
for latency in latencies:
results['scp'][latency] = []
results['rsync'][latency] = []
print("Latency #%s" %str(latency))
# Update the latency description on virtual nodes
desc['output']['latency']['delay'] = latency
distem.viface_update(node1['name'], ifname, desc)
distem.viface_update(node2['name'], ifname, desc)
for i in range(iteration):
print("\tIteration ## %s" %i)
# Launch SCP test
# Cleaning target directory on node2
distem.vnode_execute(node2['name'], 'rm -rf /tmp/dst')
# Starting the copy from node1 to node2
start_time = time.time()
cmd = "scp -rq /tmp/src %s:/tmp/dst" % node2["address"]
distem.vnode_execute(node1['name'],
cmd,
)
end_time = time.time()
import ipdb; ipdb.set_trace()
results['scp'][latency].append(time.time() - start_time)
# Launch RSYNC test
# Cleaning target directory on node2
distem.vnode_execute(node2['name'], 'rm -rf /tmp/dst')
# Starting the copy from node1 to node2
start_time = time.time()
distem.vnode_execute(node1['name'],
"rsync -r /tmp/src #{node2['address']}:/tmp/dst"
)
results['rsync'][latency].append(time.time() - start_time)
values = []
print("Rsync results :")
for latency in latencies:
print(results['rsync'][latency])
print("%s: [average=%s,standard_deviation=%s]"
%(latency, mean(results['rsync'][latency]), stdev(results['rsync'][latency])))
values = []
print("Scp results :")
for latency in latencies:
print(results['scp'][latency])
print("%s: [average=%s,standard_deviation=%s]"
%(latency, mean(results['scp'][latency]), stdev(results['scp'][latency])))
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment