switch all containers to debian stretch

TL;DR before rebuilding and launching your containers:

• run make base-debian to rebuild the base image
• edit data/nginx/ro/config, to change the ALLGO line into: ALLGO(dev, localhost, 443)

Note: I did some house cleaning tasks in addition to the migration

Summary of changes (django container)

• now built from the same base image (as the other containers)

• backports are configured with scores (apt pinning) to ensure that security upgrades are correctly applied. Note: all install backported packages need to be listed explicitely.

• pip package are installed before COPY . /opt/allgo to cache the intermediate docker image efficiently (this stage is rebuilt only if if the requirements.txt is updated)

• derive the name of other containers from the ENV variable. In development we have ENV=dev, thus in run-allgo we use "$ENV-mysql" to refer to the mysql container.

• run shell scripts with -e to prevent ignoring error silently

• move the import of the rails database into the /dk/container_init script so that this task is run only at bootstrap.

Summary of changes (all containers)

• simplified the Dockerfiles and apply mostly the same layout for all images:

  • files/ contains the files to be installed in the image (with their full path, thus the Dockerfile just contains COPY files/. /)
  • patches/ contains the patches to be applied (if any), they are applied at the root of the image
  • setup.sh to contain arbitrary shell commands. There are two motivations to do this way (rather that writing the commands in the docker file):
    • reduce the build time: more RUN statements means longer builds (note: it is ok to use a separate RUN statement commands that take a long time, eg: apt-get install)
    • be easier to read and to comment and be less error prone (because putting multiple shell commands in the same RUN statement is less readable and does not allow interleaving comments).

• added the wait-socket script into the base image, to be used to avoid race condition when an app depends on another app running in another container that is starting in the same time (eg: django needs mysql to be up before importing the db). This script polls the target (tcp or unix socket) repeatedly, and gives up after a timeout (60s by default), exiting with non zero status (i think this it is preferable to have a script that aborts the container because it triggers a nagios alert, which is usually better than ignoring the error silently)

• the mysql container now runs a mariadb server

• the ssh and toolbox container now have a multi-stage Dockerfiles for building their binaries (they no longer need to be built by the bootstrap script).

• reimplemented the smtpsink container with aiosmtpd (because smtpfixture is no longer maintained)

• added a init-container script (but i think i will merge it again with the bootstrap script, following the discussion we had today)

• removed the debian:7 factory, because it is no longer available ;-)

• rewrote the nginx entrypoint (which is no longer an entripoint). The configuration no longer contains a container name (like dev-rails) but just the environment name (aka dev). The actual container names are automatically from the environment name. Thus we have:

    ALLGO(dev, allgo.inria.fr, 443)

  instead of:

    ALLGO(dev-rails, 8080, allgo.inria.fr, 443)