Commit a32de3a4 authored by Mathieu Giraud's avatar Mathieu Giraud

Merge branch 'feature-s/docker-emails' into 'dev'

Docker emails

Closes #2760

See merge request !666
parents 5f355606 90ca3875
Pipeline #139559 failed with stages
in 10 minutes and 3 seconds
......@@ -332,6 +332,7 @@ test_server_unit:
- sed -i 's/\:latest/\:test/g' ./docker/docker-compose.yml
- cd docker/vidjil-server/conf/ && mv defs.py defs_https.py && mv defs_http.py defs.py && cd ../../..
- make germline && cp browser/js/germline.js docker/vidjil-client/conf
- mkdir -p docker/postfix/ssl
- cd docker && docker-compose up -d && cd ..
- sed -i "s/^python\ \.\.\/\.\.\/\.\./docker\ exec\ docker_uwsgi_1\ python\ \/usr\/share\/vidjil\/server\/web2py/" server/web2py/applications/vidjil/tests/init_tests.sh
- docker exec docker_uwsgi_1 sed -i "s/^\(FILE_SOURCE .*\)/FILE_SOURCE = '\/usr\/share\/vidjil\/demo'/" /usr/share/vidjil/server/web2py/applications/vidjil/modules/defs.py
......@@ -405,6 +406,7 @@ deploy_review_server:
- cp ./docker/vidjil-client/conf/conf.js ./browser/js/conf.js
- sed -i -r "s/localhost/${CI_BUILD_REF_SLUG}.server.ci.vidjil.org/g" ./browser/js/conf.js
- sed -r 's/^.*FILE_SOURCE.*$/FILE_SOURCE="\/usr\/share\/vidjil\/demo"/' ./docker/vidjil-server/conf/defs.py > ./server/web2py/applications/vidjil/modules/defs.py
- sed -i -e "/EMAIL_SUBJECT_START/c EMAIL_SUBJECT_START='[Vidjil/CI][$CI_COMMIT_REF_NAME]'" -e '/FROM_EMAIL/c FROM_EMAIL="server.ci@vidjil.org"' ./server/web2py/applications/vidjil/modules/defs.py
- mv docker docker_${CI_BUILD_REF_SLUG}
- cd docker_${CI_BUILD_REF_SLUG}
- TMP_DOCKER=$(tempfile)
......@@ -414,7 +416,7 @@ deploy_review_server:
- sed -i -r "s/WEB2PY_ADMIN_PASSWORD=/WEB2PY_ADMIN_PASSWORD=$REVIEW_SERVER_ADMIN_PASSWORD/" docker-compose.yml
- cd ..
- rsync -av --exclude .git . $REVIEW_SERVER:$REVIEW_SERVER_PATH_SERVER$CI_BUILD_REF_SLUG || $(case "$?" in 0|23) exit 0 ;; *) exit $?; esac)
- ssh $REVIEW_SERVER "cd $REVIEW_SERVER_PATH_SERVER; ./install_certs.sh ${CI_BUILD_REF_SLUG}; cd ${CI_BUILD_REF_SLUG}/docker_${CI_BUILD_REF_SLUG}; docker-compose up -d mysql fuse uwsgi nginx workers"
- ssh $REVIEW_SERVER "cd $REVIEW_SERVER_PATH_SERVER; ./install_certs.sh ${CI_BUILD_REF_SLUG}; cd ${CI_BUILD_REF_SLUG}/docker_${CI_BUILD_REF_SLUG}; mkdir -p postfix/ssl; docker-compose up -d mysql fuse uwsgi nginx workers"
environment:
name: review_server/$CI_BUILD_REF_NAME
url: http://$CI_BUILD_REF_SLUG.server.ci.vidjil.org/browser/
......
......@@ -137,6 +137,7 @@ From image `vidjil/server`
- `fuse` The XmlRPCServer that handles queries for comparing samples
- `backup` Starts a cron job to schedule regular backups
- `reporter` A monitoring utility that can be configured to send monitoring information to a remote server
- `postfix` A mail relay to allow `uwsgi` to send error notifications
......@@ -165,8 +166,12 @@ openssl x509 -noout -fingerprint -text < web2py.crt
mkdir -p vidjil-client/ssl
mv web2py.* vidjil-client/ssl/
```
+ If you are using the `postfix` container you may want to generate certificates (using the same process) and place them in `postfix/ssl`.
The certificates must bear the name of your mail domain (<maildomain>.crt and <maildomain>.key)
- A better option is to use other certificates, for example by configuring free [Let's Encrypt](https://letsencrypt.org/) certificates;
In `docker-compose.yml`, update `nginx.volumes`, line `./vidjil-client/ssl:/etc/nginx/ssl`, to set the directory with the certificates.
The same can be done for the `postfix` container.
If you would prefer to use the vidjil over HTTP (not recommended outside of testing purposes), you can
......@@ -179,6 +184,8 @@ forget to make a backup of any file you replace.)
- Set the SSL certificates (see above)
- Change the mysql root password and the web2py admin password in `docker-compose.yml`
- Change the mysql vidjil password in `mysql/create_db.sql` and sets it also in `DB_ADDRESS` in `vidjil-server/conf/defs.py`
- Set the desired mail domain and credentials for the `postfix` container and update `vidjil-server/conf/defs.py`
`SMTP_CREDENTIALS` and `FROM_EMAIL` to match
- Comment backup/reporter services in `docker-compose.yml`
......
......@@ -3,7 +3,21 @@ This CHANGELOG concerns the Docker images of vidjil.
We publish here notes to help to update these images.
See <http://www.vidjil.org/doc/server>
vidjil/server:2019-11-27-XXXXXXX
vidjil/server:XXXX
- New variable in defs.py: `EMAIL_SUBJECT_START`
- Docker containers should now be able to send emails even if you didn't have a mail server.
Please customize the ADMIN_EMAILS in the defs.py file also.
vidjil/server:2020-04-21-8190bd6c
- Adding a new field in the database
- Changes in docker-compose to wait for MySQL
- Fix issue with upload not working with a single file
- Fix the possibility to relaunch some processes (#4242)
vidjil/server:2020-03-16-657ac608
- Fix issues with jobs run while the second file is still uploading (#3907)
- Warning this image contains bugs that have been corrected in the following versions
vidjil/server:2019-12-12-2ef3187e
- Fix issues with CloneDB with several sample sets
vidjil/server:2019-11-27-b19f850b
- The web2py password is now provided through an environment
variable in the docker-compos.yml file.
vidjil/server:2018-10-19-3a690203
......
......@@ -82,7 +82,7 @@ after_workers {
/workers:$/ {
after_workers=1
}
/^\s{3,6}(nginx|fuse|uwsgi|workers|mysql):$/{
/^\s{3,6}(nginx|fuse|uwsgi|workers|mysql|postfix):$/{
after_service=1
}
/\/opt\/vidjil\/mysql/ {
......
......@@ -23,9 +23,12 @@ services:
condition: service_healthy
fuse:
condition: service_started
postfix:
condition: service_started
links:
- mysql:mysql
- fuse:fuse
- postfix:postfix
fuse:
image: vidjil/server:latest
command: bash /entrypoints/fuse-entrypoint.sh
......@@ -69,6 +72,7 @@ services:
condition: service_healthy
links:
- mysql
- postfix
backup:
image: vidjil/server:latest
command: bash -c "echo '0 5 * * * cd /usr/share/vidjil/server; sh backup.sh /mnt/backup 2>1&' > /etc/cron.d/backup-cron && cron && tail -f /var/log/cron.log"
......@@ -88,3 +92,12 @@ services:
condition: service_started
links:
- uwsgi:uwsgi
postfix:
image: catatnight/postfix
expose:
- 587
volumes:
- ./postfix/ssl:/etc/postfix/certs
environment:
- maildomain=vidjil.org
- smtp_user=vidjil:smtp_pass
from ubuntu:16.04
label version="1.0"
label description="An Ubuntu based docker image which postfix"
# Install Postfix. Shamelessly stolen from https://github.com/noteed/docker-postfix/blob/master/Dockerfile
run echo "postfix postfix/main_mailer_type string Internet site" > preseed.txt
run echo "postfix postfix/mailname string mail.example.com" >> preseed.txt
# Use Mailbox format.
run debconf-set-selections preseed.txt
run apt-get update
run DEBIAN_FRONTEND=noninteractive apt-get install -q -y postfix
run postconf -e myhostname=mail.example.com
run postconf -e mydestination="mail.example.com, example.com, localhost.localdomain, localhost"
run postconf -e mail_spool_directory="/var/spool/mail/"
run postconf -e mailbox_command=""
......@@ -4,10 +4,12 @@
###
### Email notifications for server errors
SMTP_SERVER = 'localhost'
SMTP_SERVER = 'postfix'
SMTP_CREDENTIALS = 'vidjil:smtp_pass' # set to None if no auth required
# SMTP_SERVER = 'logging' # no mail
FROM_EMAIL = 'notifications@vidjil.org'
ADMIN_EMAILS = ['notifications@vidjil.org']
EMAIL_SUBJECT_START = '[Vidjil]' # Customize the start of the email subject
### address for the sql database
### sqlite (not recommended)
......
......@@ -4,7 +4,8 @@
###
### Email notifications for server errors
SMTP_SERVER = 'localhost'
SMTP_SERVER = 'postfix'
SMTP_CREDENTIALS = 'vidjil:smtp_pass' # set to None if no auth required
# SMTP_SERVER = 'logging' # no mail
FROM_EMAIL = 'notifications@vidjil.org'
ADMIN_EMAILS = ['notifications@vidjil.org']
......
......@@ -10,7 +10,7 @@ install_web2py: download_web2py
unzip web2py_src.zip
mv web2py/handlers/wsgihandler.py web2py/wsgihandler.py
rm web2py_src.zip
cp -i web2py/applications/vidjil/modules/defs.py.sample web2py/applications/vidjil/modules/defs.py
cp web2py/applications/vidjil/modules/defs.py.sample web2py/applications/vidjil/modules/defs.py
unit: clean_unit_tests
python web2py/web2py.py -S vidjil -M -R testRunner.py
......
......@@ -108,7 +108,7 @@ def run_request():
enough_space = vidjil_utils.check_enough_space(defs.DIR_RESULTS)
if not enough_space:
mail.send(to=defs.ADMIN_EMAILS,
subject="[Vidjil] Server space",
subject=defs.EMAIL_SUBJECT_START+" Server space",
message="The space in directory %s has passed below %d%%." % (defs.DIR_RESULTS, defs.FS_LOCK_THRESHHOLD))
return error_message("Runs are temporarily disabled. System admins have been made aware of the situation.")
......@@ -586,7 +586,7 @@ def error():
user_str = user_str.replace('<','').replace('>','').strip()
mail.send(to=defs.ADMIN_EMAILS,
subject="[Vidjil] Server error - %s" % user_str,
subject=defs.EMAIL_SUBJECT_START+" Server error - %s" % user_str,
message="<html>Ticket: %s<br/>At: %s<br />User: %s</html>" % (ticket_url, requested_uri, user_str))
return "Server error"
......
......@@ -144,7 +144,7 @@ def form():
enough_space = vidjil_utils.check_enough_space(defs.DIR_SEQUENCES)
if not enough_space:
mail.send(to=defs.ADMIN_EMAILS,
subject="[Vidjil] Server space",
subject=defs.EMAIL_SUBJECT_START+" Server space",
message="The space in directory %s has passed below %d%%." % (defs.DIR_SEQUENCES, defs.FS_LOCK_THRESHHOLD))
return error_message("Uploads are temporarily disabled. System admins have been made aware of the situation.")
......
......@@ -534,7 +534,7 @@ def submit():
if (p['id'] % 100) == 0:
mail.send(to=defs.ADMIN_EMAILS,
subject="[Vidjil] %d" % p['id'],
subject=defs.EMAIL_SUBJECT_START+" %d" % p['id'],
message="The %dth %s has just been created." % (p['id'], set_type))
else :
......
......@@ -88,7 +88,7 @@ auth.define_tables(username=False, signature=False)
mail = auth.settings.mailer
mail.settings.server = defs.SMTP_SERVER
mail.settings.sender = defs.FROM_EMAIL
mail.settings.login = None
mail.settings.login = defs.SMTP_CREDENTIALS
## configure auth policy
auth.settings.registration_requires_verification = False
......
......@@ -5,9 +5,11 @@
### Email notifications for server errors
SMTP_SERVER = 'localhost'
SMTP_CREDENTIALS = 'vidjil:XXXXXX' # set to None if no auth required
# SMTP_SERVER = 'logging' # no mail
FROM_EMAIL = 'root@vidjil.org'
ADMIN_EMAILS = ['notifications@vidjil.org']
EMAIL_SUBJECT_START = '[Vidjil]' # Customize the start of the email subject
### Specific accounts
LIMITED_ACCOUNTS = [ ]
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment