MAJ terminée. Nous sommes passés en version 14.6.2 . Pour consulter les "releases notes" associées c'est ici :

  1. 27 Jun, 2018 7 commits
    • BAIRE Anthony's avatar
      Use the redis db to trigger controller actions · 01dd48e6
      BAIRE Anthony authored
      This commit removes the old notification channel (socket listening
      on port 4567), and uses the redis channel 'notify:controller' instead.
      The django job creation views are updated accordingly.
    • BAIRE Anthony's avatar
      Stream job logs and job state updates to the user · 1bb4acf4
      BAIRE Anthony authored
      This commit makes several changes.
      In the controller:
      - duplicates the logs produced by the jobs. Initially they were only
        stored into allgo.log, now they are also forwarded to the container
        output (using the 'tee' command) so that the controller can read
      - add a log_task that reads the logs from docker and feeds them into
        the redis db key "log:job:<ID>" (this is implemented with aiohttp
        in order to be fully asynchronous)
      - store the job state in a new redis key "state:job:<ID>"
      - send notification to the redis pubsub 'notify:aio' channel when
        the job state has changed or when new logs are available
      In the allgo.aio frontend:
      - implement the /aio/jobs/<ID>/events endpoints which streams all
        job events & logs to the client (using json formatted messages)
      In django:
      - refactor the JobDetail view and template to update the page
        dynamically for job updates (state/logs)
          - allgo.log is read only when the job is already terminated.
            Otherwise the page uses the /aio/jobs/<ID>/events channel
            to stream the logs
          - the state icon is patched on the page when the state changes,
            except for the DONE state which triggers a full page reload
            (because there are other parts to be updated)
    • BAIRE Anthony's avatar
      update the location of the job files · c5f93183
      BAIRE Anthony authored
      now located in the 'django' container and full path is
    • BAIRE Anthony's avatar
      add a redis client in the controller · 553fee62
      BAIRE Anthony authored
    • BAIRE Anthony's avatar
      remove the factories · 52a9e3ec
      BAIRE Anthony authored
      Fix #185
      - webapps are now located directly at the root of the registry
        (not in the /webapp subdir)
      - factories are no longer stored in our registry, we directly reference
        images on the official docker registry
    • BAIRE Anthony's avatar
      support registry authentication in the controller · 5a5a06d2
      BAIRE Anthony authored
      because it is now based on tokens (instead of using TLS client certificates)
    • BAIRE Anthony's avatar
      rename tables ad dj_* · a0ecda20
      BAIRE Anthony authored
  2. 12 Apr, 2018 1 commit
  3. 21 Nov, 2017 1 commit
    • BAIRE Anthony's avatar
      fix out of memory message · 06420260
      BAIRE Anthony authored
      this variant is closer to the actual meaning
      (the fact that the limit was reached does not automatically imply
       that the process is starving, we cannot decide how much memory
       a process needs without doing some profiling)
  4. 20 Nov, 2017 5 commits
  5. 16 Nov, 2017 1 commit
  6. 14 Nov, 2017 4 commits
    • BAIRE Anthony's avatar
      update the job container command · 2cfd30b8
      BAIRE Anthony authored
      - to have SIGTERM forwarded to the process
      - to propagate the exit code of the process
    • BAIRE Anthony's avatar
      add the 'rescheduled' future · 7d5df09e
      BAIRE Anthony authored
      to let the task implementations detect that they are being rescheduled
    • BAIRE Anthony's avatar
      factorisation · 5173f5e1
      BAIRE Anthony authored
    • BAIRE Anthony's avatar
      refactor the management of swarm/sandbox resources · 0e301e74
      BAIRE Anthony authored
      - add SwarmAbstractionClient: a class that extends docker.Client and
        hides the API differences between the docker remote API and the
        swarm API. Thus a single docker engine can be used like a swarm
      - add SharedSwarmClient: a class that extends SwarmAbstractionClient
        and monitors the swarm health and its resource (cpu/mem) and manages
        the resource allocation.
        - resources are partitioned in groups (to allow reserving resources
          for higher priority jobs)
        - two SharedSwarmClient can work together over TCP in a master/slave
          configuration (to allow the production and qualification platforms
          to use the same swarm without any interference)
      - the controller is modified to:
        - use SharedSwarmClient to:
          - wait for the end of a job (in place of DockerWatcher)
          - manage resource reservation (LONG_APPS vs. BIGMEM_APPS vs normal
            apps) and monitor swarm health (fix #124)
          - NOTE: resources of the swarm and sandbox are now managed
            separately (2 instances of SharedSwarmClient), whereas it was
            global before (this was suboptimal)
        - rely on SwarmAbstractionClient to compute the cpu quotas
        - store the container_id of jobs into the DB (fix #128), this is a
          prerequisite to permit renaming apps in the future
        - store the class of the job (normal vs. long app) in the container
          name (for the resource management with SharedSwarmClient)
        - read the configuration from a yaml file (/vol/ro/config.yml) for:
          - cpu/mem quotas
          - swarm resources allocation policy
          - master/slave configuration
  7. 29 May, 2017 1 commit
  8. 25 Apr, 2017 2 commits
  9. 24 Apr, 2017 1 commit
    • BAIRE Anthony's avatar
      improve robustness in matching docker events · 4bbb6b8d
      BAIRE Anthony authored
      with swarm, we had:
      2017-Apr-24 18:09:13 DEBUG    docker event b'{"from":"
      2017-Apr-24 18:09:13 ERROR    docker watcher exception
      Traceback (most recent call last):
        File "/opt/allgo-docker/", line 950, in _thread_func
            if event["status"] == "die":
            KeyError: 'status'
  10. 20 Apr, 2017 1 commit
  11. 11 Apr, 2017 1 commit
  12. 23 Mar, 2017 1 commit
  13. 21 Mar, 2017 1 commit
  14. 15 Mar, 2017 1 commit
    • BAIRE Anthony's avatar
      replace pipecmd with a real ssh connection · e789f843
      BAIRE Anthony authored
      - sshd server installed in the toolbox
      - ssh keys & config stored in ssh:/vol/cache and mounted as
        /.sandbox inside the sandbox
      - toolbox mounted as /.toolbox inside the sandbox
      - ssh agent & X11 forwarding are now working
      - the toolbox commands available by default in every sandboxes
        (vim, less, nc, scp, ...)
      - sandboxes now attached to a separate network (named
        'allgo_sandboxes' by default)
      fix #88
  15. 01 Mar, 2017 1 commit
  16. 28 Feb, 2017 1 commit
  17. 21 Feb, 2017 3 commits
  18. 09 Feb, 2017 4 commits
  19. 08 Feb, 2017 3 commits