See `server/web2py/applications/vidjil/scripts/migrator.py --help`
An important element to note here is the –no-create-info we add this
parameter because web2py needs to be allowed to create tables itself
because it keeps track of database migrations and errors will occur if
tables exist which it considers it needs to create.
## Exporting an archive
In order to import the data into an installation you first need to ensure
the tables have been created by Web2py this can be achieved by simply
accessing a non-static page.
(to be detailed)
/\!\\ If the database has been initialised from the interface you will
likely encounter primary key collisions or duplicated data, so it is best
to skip the initialisation altogether.
## Importing an archive
Once the tables have been created, the data can be imported as follows:
### Step 1 : extract the archive on your server
``` bash
mysql -u <user> -p <db> < <file>
```
The export directory must be on your server and accessible from your vidjil-server docker container.
You can define a new shared volume; or simply put the export directory on an already accessible location such as `[DOCKER DIRECTORY]/vidjil-server/conf/export/`
Please note that with this method you should have at least one admin user
that is accessible in the imported data. Since the initialisation is being
skipped, you will not have the usual admin account present.
It is also possible to create a user directly from the database although
this is not the recommended course of action.
### Step 2 : prepare the group that will own the data
## Files
Files can simply be copied over to the new installation, their filenames
are stored in the database and should therefore be accessible as long as
they are in the correct directories.
The permissions on a vidjil server are *group* based. Users and groups may be different from one server to another one. Before importing data on a server, one must have a group ready to receive the permissions to manage the imported files.
## Filtering data (soon deprecated)
From the admin web interface has, you can create a new group ("groups" -> "+new group" -> "add group"). The group ID is displayed between parenthesis next to its name on the group page, you will need it later. If you create such a group on a blank vidjil server, the ID is *4*.
When extracting data for a given user, the whole database should not be
copied over.
There are two courses of action:
### Step 3 : prepare your server analysis configs
- create a copy of the existing database and remove the users that are
irrelevant. The cascading delete should remove any unwanted data
barring a few exceptions (notably fused<sub>file</sub>, groups and sample<sub>setmembership</sub>)
*This step may require bioinformatics support depending on your data, the config previously used, and the ones you intend to use on your new installation. We can offer support via the [VidjilNet consortium](http://www.vidjil.net) for help on setting that.*
- export the relevant data directly from the database. This method
requires multiple queries which will not be detailed here.
Vidjil analysis configs should not be directly transferred between servers. Indeed, they depend on the setup of each server setup (software, paths...) and can collide with existing configs on your installation. Before importing, you thus need to create the missing analysis configs on your server and edit the `config.json` file provided in the export folder.
Once the database has been correctly extracted, a list of files can be
obtained from sequence<sub>file</sub>, fused<sub>file</sub>, results<sub>file</sub> and analysis<sub>file</sub>
with the following query:
This `config.json` file initially contains a list of the analysis configs from the original public server, such as:
- [RESULTS DIRECTORY] the results directory path inside the container, it should be defined in your docker-compose.yml, by default it is /mnt/result/results/
- [EXPORT DIRECTORY] the export directory you installed in step 1, if you set it up in docker/vidjil-server/conf/export/ is location inside the container should be /etc/vidjil/export/
- [CONFIG.JSON FILE] this file is located in the export folder and you should have edited it during step 3
- [PPROCESS.JSON FILE] this file is located in the export folder and you should have edited it during step 4
- [GROUP ID] ID of the group you should have created/selected during step 2