Commit c775b8ff authored by MAUPETIT Charly's avatar MAUPETIT Charly
Browse files

add docs

parent 29f55eed
#Configuration to run Allgo
##Woody (your computer)
Needs: Ruby 2.2.2-dev, Nginx, mariadb, ruby + rails via rvm, rails app (via capistrano), redis, docker, git, zip, curl,sudo
Use RVM to install the correct version of Ruby and make it the default one.
```zsh
#What is Allgo (A||Go)
A||Go est une infrastructure web pour mettre en ligne le plus rapidement et simplement
possible des logiciels ou prototypes scientifiques. L'objectif est de réduire le coût du transfert de
l'ordinateur du chercheur vers les utilisateurs finaux (membre de l’équipe, communauté
scientifique,Startup PME, grand public).
Le déploiement des applications, et surtout leurs utilisations, s'en retrouvent simplifiés. On peut les exécuter via l'interface web ou une API HTTP.
Techniquement, A||Go est basé sur le framework Ruby on Rails pour offrir une interface web
et des services HTTP REST, ainsi que l’utilisation de Docker pour la gestion des images applicatives.
Docker fournit des conteneurs Linux pour gérer / stocker / utiliser les applications que nous
hébergeons.
Un job est l'action d'exécuter une application, avc un ou plusieurs fichier(s) qui retournera un résultat.
#Architecture
![](doc/allgo.jpg)
## Woody (web server) - Debian 8
### What
Allgo frontend via the web application in RoR. We also have the datastore on Woody, to store files from jobs, and the SQL database.
### Need
Ruby 2.2.2-dev, Nginx, mariadb, ruby + rails via rvm, rails app (via capistrano), redis, docker, git, zip, curl, sudo
### How
su -c "adduser allgo"
su -c "usermod -aG sudo allgo"
su allgo
Use RVM to install the correct version of Ruby
```zsh
\curl -sSL https://get.rvm.io | bash
rvm install ruby-2.2.2-dev
rvm --default use ruby-2.2.2-dev
```
Install the gem bundler to download the dependancies
```zsh
gem install bundler
```
The Gemfile will install the gem mysql2 that needs MariaDB and Sidekiq that needs Redis:
```zsh
sudo yum install mariadb-devel
sudo yum install redis
sudo apt-get install mariadb-devel redis -y
sudo service redis enable (to launch at boot)
sudo service redis start
sudo service mariadb enable
sudo service mariadb start
```
To run the project you need to migrate the project's database:
To run the project you need to migrate the project's database, there are backup in db/backup
```zsh
rake db:migrate
rake db:create ( RAILS_ENV=production )
rake db:migrate ( RAILS_ENV=production )
```
Then launch the project and check if there is no error message in the correct port:
......@@ -25,21 +53,31 @@ Then launch the project and check if there is no error message in the correct po
rails s
```
## Infrastructure
There is three types of machines in the infrastructure (other than Woody):
- The workers, not connected to Internet. There are just here to do the jobs asked.
- Sid, a worker with an internet connection.
- Cargo, the Docker registry used by workers to do ```docker pull``` commands.
The best way to create the infrastructure is to build your first worker then you can clone your machines to create more workers, Sid and Cargo.
We also need to run sidekiq separately :
```zsh
bundle exec sidekiq (-e production)
```
###Installation of Debian and Docker
We're using VM VirtualBox to simulate the workers. Download a .iso of Debian, launch your VM and configure it (we're creating a new user called 'allgo' in this example). Once it boots, you need to configure it to be ready for Docker:
##WorkerX
### What
The worker are server that will execute job created by users in Allgo.
we sent the command through ssh from Woody and send a docker pull request to cargo (see below)
the datastore from woody is mount through ssh so Worker can write in.
### Need
Docker 1.6, openssh-server
### How
```zsh
sudo
su -c "adduser allgo sudo"
curl + curl -sSL https://test.docker.com/ | sh #to get last docker version
su -c "apt-get update && apt-get install sudo openssh-server -y"
su -c "adduser allgo"
su -c "usermod -aG sudo allgo"
su allgo
```
Install the last docker version docker :
```zsh
curl + curl -sSL https://test.docker.com/ | sh
sudo usermod -aG docker allgo
/etc/init.d/docker start
```
Try a Docker command to see if it works:
......@@ -53,16 +91,16 @@ sudo chown allgo /home/allgo/.docker/
sudo chown allgo /home/allgo/.docker/key.json
```
###Allowing insecuring registries
As we do not have an HTTPS certification yet, you need to allow insecure registries:
*Allowing insecure registry*
As we do not have an HTTPS certification yet, you need to allow an insecure registry in cargo server (see below) :
####Debian < v8
```zsh
sudo nano /etc/default/docker
```
Then modify the line concerning DOCKER_OPTS to have: ```DOCKER_OPTS="--insecure-registry cargo.irisa.fr:5000"```.
####Debian >= v8
(or myregistry:5000 if development or other server)
#####Debian >= v8
```zsh
sudo nano /lib/systemd/system/docker.service
```
......@@ -74,8 +112,28 @@ sudo systemctl daemon-reload
sudo /etc/init.d/docker restart
```
###A different IP address for every machine
All the workers should have a different IP address configured. Add in VirtualBox second host-only adapter in the Network tab. This adapter sould be in promiscuous mode allowing VMs witha connected cable.
##Sid - Sandbox
It's the same as worker, but we use it when an user install an application/algorithm.
We create the docker image and push it to the registry
##Cargo
Cargo run the docker image registry, which allow us to store our own docker images.
We use a NAS to replicate the created images, it's safer.
Install docker (same as above)
*To run the registry, with external storage*
```zsh
docker pull registry
docker run -d -p 5000:5000 -e SETTING_FLAVOR=dev -e STORAGE_PATH=/the_folder_of/your_choice_on_container/ -v /the_folder_of/your_choice/on_host:/the_folder_of/your_choice_on_container/ registry:latest
```
It's unsecure for the moment.
#The development environment :
You can use the Vagrantfile for Virtualbox (no provisioning yet)
Or follow those instructions :
Create 4 or 5 VM (depends of if you want to use your host as web server)
Add in VirtualBox second host-only adapter in the Network tab, with DHCP.
Relaunch your VM and modify the network interfaces:
......@@ -115,15 +173,4 @@ sudo nano /etc/hostname
Change the name and then:
```zsh
sudo /etc/init.d/hostname.sh start
```
##Cargo
You need to install docker-registry on it:
```zsh
docker pull registry
```
When running the registry, you need to mount a datastore:
```zsh
docker run -d -p 5000:5000 -e SETTING_FLAVOR=dev -e STORAGE_PATH=/the_folder_of/your_choice_on_container/ -v /the_folder_of/your_choice/on_host:/the_folder_of/your_choice_on_container/ registry:latest
```
```
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment