1. 23 Apr, 2020 1 commit
    • BAIRE Anthony's avatar
      prevent potential option injections in job input files · 26aac5ab
      BAIRE Anthony authored
      A malicious user may submit an input file starting with '-'.
      A loosely-implemented webapp entrypoint could misinterpret
      it as a command-line option a let the user inject arbitrary
      options to the commands executed inside the job.
      
      To prevent this the leading '-' in input filenames are silently
      changed into '_'.
      26aac5ab
  2. 09 Apr, 2020 2 commits
    • BAIRE Anthony's avatar
    • BAIRE Anthony's avatar
      Squashed commit of the following: · c6e8fab9
      BAIRE Anthony authored
      commit 5311d3a7
      Author: LETORT Sebastien <sebastien.letort@irisa.fr>
      Date:   Tue Sep 17 09:29:45 2019 +0200
      
          WebappVersion.webapp related_name changed from 'webapp' to 'version'.
      
      commit 0f68b0e0
      Merge: 7aa839c4 8a251945
      Author: LETORT Sebastien <sebastien.letort@irisa.fr>
      Date:   Mon Sep 16 12:08:26 2019 +0200
      
          Merge branch 'django' into 266-allow-unpublished-webapp-version
      
      commit 7aa839c4
      Author: LETORT Sebastien <sebastien.letort@irisa.fr>
      Date:   Mon Sep 16 12:07:00 2019 +0200
      
          Several fixes after code review.
          query_webapps_for_user doesn't have the only_published_version param anymore.
          comment explains the filtering with 'webapp__published'.
          debug log comment are removed.
          WebappVersion in error state are show to the user.
          WebappVersionList.get_queryset return query_set and not values.
          no more mention of atomic transaction.
          WebappVersionList.post: var name correction.
      
      commit ab7107e3
      Author: LETORT Sebastien <sebastien.letort@irisa.fr>
      Date:   Mon Jul 22 12:19:55 2019 +0200
      
          sugar +/- PEP8 syntax correction.
          remove useless else in helpers.query_webapps_for_user
      
      commit e32ec378
      Author: LETORT Sebastien <sebastien.letort@irisa.fr>
      Date:   Mon Jul 22 11:13:54 2019 +0200
      
          update of query_webapps_for_user, specially when used by views.WebappList.
          Now the user cannot view public app if no version is published.
      
      commit 55061286
      Merge: 5a2b0b7e 979954ef
      Author: LETORT Sebastien <sebastien.letort@irisa.fr>
      Date:   Mon Jul 1 14:12:17 2019 +0200
      
          Merge branch 'django' into 266-allow-unpublished-webapp-version
      
      commit 5a2b0b7e
      Merge: b7f1b67c ffa355b0
      Author: sebastien letort <sletort@capri.irisa.fr>
      Date:   Fri Apr 19 10:16:16 2019 +0200
      
          Merge branch 'django' into 266-allow-unpublished-webapp-version
      
      commit b7f1b67c
      Author: sebastien letort <sletort@capri.irisa.fr>
      Date:   Fri Apr 19 10:15:27 2019 +0200
      
          Published status of version can now be updated.
          I used @transaction atomic to make all the update in one access to the DB (so speed up). But there is no constraint for that.
      
      commit ddf2cf17
      Author: sebastien letort <sletort@capri.irisa.fr>
      Date:   Wed Apr 17 14:05:42 2019 +0200
      
          list all versions available for an app.
          The app owner will be able to update published status, not yet implemented.
      
      commit d2d6176a
      Author: sebastien letort <sletort@capri.irisa.fr>
      Date:   Tue Apr 16 11:11:21 2019 +0200
      
          The owner of an app can run job on unpublished webapp (it appears in the list of available version).
      
      commit 0c053a2d
      Merge: a8c1d1d4 610f9c90
      Author: sebastien letort <sletort@capri.irisa.fr>
      Date:   Mon Apr 15 15:21:42 2019 +0200
      
          Merge branch 'django' into 266-allow-unpublished-webapp-version
      
      commit a8c1d1d4
      Author: sebastien letort <sletort@capri.irisa.fr>
      Date:   Mon Apr 15 13:54:04 2019 +0200
      
          Webapp version can now be commited 'unpublished'.
      
      commit 22799031
      Author: sebastien letort <sletort@capri.irisa.fr>
      Date:   Tue Apr 2 16:12:50 2019 +0200
      
          Only the published webapp versions are shown in the jobCreate view.
      c6e8fab9
  3. 02 Apr, 2020 1 commit
  4. 23 May, 2019 1 commit
  5. 23 Oct, 2018 1 commit
    • BAIRE Anthony's avatar
      fix privacy issues in TagList and TagWebappList · aedd3283
      BAIRE Anthony authored
      webapp lists should never display apps not visible by the request.user
      (obviously!)
      
      TagWebappList did not implement such a filter. I added the
      query_webapps_for_user() helper and use it for TagWebappList, TagList
      and WebappList (the list returned by this function is the superset of
      webapps that these views are allowed to display).
      aedd3283
  6. 27 Sep, 2018 1 commit
  7. 21 Sep, 2018 1 commit
  8. 18 Sep, 2018 2 commits
  9. 08 Aug, 2018 1 commit
  10. 31 Jul, 2018 2 commits
  11. 05 Jul, 2018 1 commit
  12. 03 Jul, 2018 5 commits
    • BAIRE Anthony's avatar
      rename 'file_obj' as 'uploaded_file' · 2ef76595
      BAIRE Anthony authored
      
      'file_obj' is misleading, because it has a meaning in python
      https://docs.python.org/3/glossary.html#term-file-object
      
      also i fixed the doc, because the actual param is not a dict
      2ef76595
    • BAIRE Anthony's avatar
      ensure the job dir is always created, even if no files are uploaded · 494f3bc9
      BAIRE Anthony authored
      (because the controller requires it to store the results)
      
      (note: i used the name 'filepath' instead of 'endpoint' because it
       is less misleading: the variable containes the path of the file
       in the local filesystem whereas 'endpoint' would rather refer to
       the path in the http request)
      494f3bc9
    • BAIRE Anthony's avatar
      sanitise filenames submitted by the user · 5cc6b01d
      BAIRE Anthony authored
      This is to fix a vulnerability. The user submitting the job
      **must not** be able to write a file outside the job dir
      (for example by submitting a file named "../../something")
      5cc6b01d
    • BAIRE Anthony's avatar
      do not create job dirs in mode 0777 · deaae0f7
      BAIRE Anthony authored
      Two points
      - it is not guaranteed to work properly in production because the
        gunicorn server is multithreaded (the umask may not be correctly
        restored if two jobs are created in the same time)
      - it is actually not needed. In production, the datastore is a nfs
        volume exported with the 'all_squash' option (which makes all file
        accesses from the jobs made with the same uid as django).
      deaae0f7
    • BAIRE Anthony's avatar
      allow using network prefixes in ALLGO_ALLOWED_IP_ADMIN · 40c2945a
      BAIRE Anthony authored
      Rationale: in development the ip address of the local machine is not
      easily predictible because when docker creates virtual networks it
      assigns the ip prefixes/addres dynamically by default (and i do not want
      to configure static addresses because of it may interefere and cause
      nasty side-effects if using docker for other projects on the same
      machine)
      
      Now in development we allow admin actions from 0.0.0.0/0 (which means
      all ip addresses)
      
      Note: I used the IPy package (whose purpose is to handle ranges of
      IP addresses)
      40c2945a
  13. 27 Jun, 2018 2 commits
    • BAIRE Anthony's avatar
      Use the redis db to trigger controller actions · 01dd48e6
      BAIRE Anthony authored
      This commit removes the old notification channel (socket listening
      on port 4567), and uses the redis channel 'notify:controller' instead.
      
      The django job creation views are updated accordingly.
      01dd48e6
    • BAIRE Anthony's avatar
      Stream job logs and job state updates to the user · 1bb4acf4
      BAIRE Anthony authored
      This commit makes several changes.
      
      In the controller:
      
      - duplicates the logs produced by the jobs. Initially they were only
        stored into allgo.log, now they are also forwarded to the container
        output (using the 'tee' command) so that the controller can read
        them
      
      - add a log_task that reads the logs from docker and feeds them into
        the redis db key "log:job:<ID>" (this is implemented with aiohttp
        in order to be fully asynchronous)
      
      - store the job state in a new redis key "state:job:<ID>"
      
      - send notification to the redis pubsub 'notify:aio' channel when
        the job state has changed or when new logs are available
      
      In the allgo.aio frontend:
      
      - implement the /aio/jobs/<ID>/events endpoints which streams all
        job events & logs to the client (using json formatted messages)
      
      In django:
      
      - refactor the JobDetail view and template to update the page
        dynamically for job updates (state/logs)
          - allgo.log is read only when the job is already terminated.
            Otherwise the page uses the /aio/jobs/<ID>/events channel
            to stream the logs
          - the state icon is patched on the page when the state changes,
            except for the DONE state which triggers a full page reload
            (because there are other parts to be updated)
      1bb4acf4
  14. 19 Jun, 2018 1 commit
    • BERJON Matthieu's avatar
      Reading of Redis logs for a specific job · 96741268
      BERJON Matthieu authored
      I added a helper function to extract the job logs for a specific job in the Redis database.
      The function is a copy of another part of the code not merged yet in the master branch.
      
      I updated the view accordingly, added a proper docstring and moved the class in the module to keep the code ordered.
      I updated as well the template to display the data.
      96741268
  15. 05 Jun, 2018 1 commit
  16. 15 May, 2018 1 commit
    • BERJON Matthieu's avatar
      Update of the Job management system · 85ad5ac8
      BERJON Matthieu authored
      
      
      I fully updated the job management by updating or creating the following
      views:
      
      - job submission
      - job list
      - job detail
      - job deletion
      
      I updated the job form in order to:
      
      - add a field to manage the upload of multiple files
      - get better error messages
      
      I wrote a small function to upload any type of files into the data
      store. This function requires a job ID in order to work properly and
      create a folder according to this number and store data inside it.
      
      I added two specific routes to see a job details and delete one.
      
      I added a detail view that display information related to a given job.
      This view list as well any files stored for this job if any have been
      uploaded in the datastore.
      
      The job created had been updated in order to display a success message
      one created. Save the job data into the database and upload any given
      file in the data store.
      
      I added links that lead to both the webapp page and job details. I added
      as well the display of any messages that could be given for specific
      actions (delete a job, job creation, ...)
      I added as well some javascript to display tooltips on specific links
      (deletion in this case)
      
      I added few more things in the job submission template such as the
      input to upload multiple files at one time. I added as well the
      management of error messages.
      
      I added two templates to display job details and to delete a specific
      job.
      Signed-off-by: BERJON Matthieu's avatarMatthieu Berjon <matthieu.berjon@inria.fr>
      85ad5ac8
  17. 02 May, 2018 1 commit
  18. 25 Apr, 2018 1 commit