GitLab

- add the Token model to store the authentication tokens
- add the WebappTokenCreate & WebappTokenDelete views to manage
  the tokens
- enable the docker panel in the WebappSandboxPanel
- rename the jwt token class 'Token' as 'JwtToken'
  (to avoid any confusion)
- enable access to the registry (only for pushing for the moment)

close #44
close allgo.inria.fr#5
mitigates #227

(error messages for the sysadmin should be sent to the logs,
 not to the user)

django-nginx conf file has been reordered, and API location has been slightly rewritten with inner locations.

Here we conserved the CORS principles and make them used to all API, datastore included.
Note that trying to get a file with the api (api/v1/datastore/...) will return a 404.
-> but the end user has no reason to known this path.

fix #226

authorization is done by django route /auth if user is logged or by token

move the db seeding outside the django container_init script

seeding is now performed by the bootstrap script directly (except called
with --noseed). It consists of two parts:
 - seeding the db: with 3 users (admin@localhost, devel@localhots and
   user@localhost) and one webapp ('sleep' that has 1 version 'latest')
 - seeding the registry with one image: "localhost/sleep:latest" for the
   sleep webapp

Initially the authentication with the registry was performed with a TLS
client certificate installed on the controller.

The registry is now configured to use token-based authentication (to
give access to the users), but unfortunately it cannot be configured to
support multiple auth methods. So we have to provide a token for the
controller.

This is a 'God' token, it gives total access (pull and push) on all
images.

There are two purposes:
- implement server push (using long-lived HTTP requests) for:
    - sending status updates for the jobs and sandboxes
    - live-streaming of the job logs
- have a really async implementation for pushing image manifests into
  the registry (the preliminary implementation in
  5451a6df was blocking)

It is implemented with aiohttp.web (a lighweight HTTP framework,
similar to what we can do with flask but asynchronously).

The alternative would have been to use the django channels plugin, but:
- it went through a major refactoring (v2) recently
- it requires replacing unicorn with an ASGI server (daphne)
- django-channels and daphne are not yet debian, and for the HTTP server
  i would prefer an implementation for which we have stable security
  updates

(anyway this can be ported to django-channels later)

The nginx config redirects the /aio/ path to this server (and the image
manifests pushes too).

The allgo.aio server interacts only with the nginx, reddis and django
containers. It relies totally on the django server for authenticating
the user and for accessing the mysql db (so there is no ORM).

NOTE: in this design the django server has to trust blindly the requests
coming from the allgo.aio server (for some endpoints). To prevent
security issues, the nginx configuration is modified to set the header
'X-Origin: nginx'. Thus django knowns who he can trust.

This commits implements only the image pushs. Job updated and logs
streaming will come in a later pull request.

This commit will have to be reverted when we go to production

This commit removes the calls to the 'migration.sh' script that
did a full a import of the rails db (rails apps will be imported
on-demand)

The django db is now initialised from scratch, using the db_seed command

(and have it  generated by the container_init script)

- images manifest push/pull are routed through the django server
- all other endpoints are routed directly to the registry


On client-side the registry url is the is the same as the allgo server.
The backend registry location is configured by the
ALLGO_REGISTRY_PRIVATE_URL which is set to "http://{ENV}-registry"
by default (i.e. 'http://dev-registry' in dev)


Because the registry URL is configured at runtime (by the env
var), the nginx config can no longer be static.
/etc/nginc/conf.d/allgo.conf is generated at runtime
 (after loading the django config)

This makes the config simpler, now that the reverse-proxy is required
(because of the image push/pull endpoint routed through django)

(to store the secret_key)

(thus we'll have security updates)

- install django statics into /var/www at image build time
- django now on port 8000 (django's default) instead of 4000
- rename @allgo as @Django in nginx's config to decrease confusion
- publish port 8080 in docker-compose

- django is now launch in the foregroud, and its stdout/stderr is now
  the container stdout/stderr

- nginx is now run as a background process, without supervision. A
  HEALTHCHECK CMD is added to monitor the status of nginx (so as to
  a nagios alert in case an hypothetical nginx crash)

- add ALLGO_HTTP_SERVER to select the http server. Default is
 'gunicorn', docker-compose.yml sets it to 'django' for development.

+ensure that ENV is defined

(rather than everytime the container is run)

(more reliable)

(like in the other images)