contributing

The goal of the document is to explain the architecture of the code to help new contributors.
It will assume that you are already familiar with the OCR workflow and the different bricks of the web stack (especially Django).

Running eScriptorium locally:

There are two ways to run eScriptorium, the 'easy' way with Docker,
and the hard way by installing and configuring everything yourself.
Unless you are familiar with coding inside a docker environment, the second solution is preferred for development because docker tends to get in your way during debugging.

Coding workflow and guidelines

• We use gitflow (no extension), so:
  new feature/fix branch -> Pull Request -> Peer Review -> merge into develop -> merge develop into master -> tag master -> deploy
• cia python guidelines
  So basically pep8 with some additions and a few lifted restrictions, notably

Each line of text in your code should be at most 100 characters long.

General organization

eScriptorium follows Docker & Django good practices as much as possible.
The root directory is the docker-compose root. The app/ directory is the Django root.
All templates can be found in the project folder app/escriptorium/templates/.
The JS code is located in front/, it's built by npm + webpack.

Back-end apps

They lie in app/apps/, they can be imported directly from anywhere because apps/ is added to the python PATH in settings.py.

1) core
It is the main application, it contains most models and business logic. The main models are:

• Document, which represents a working directory (for the user it could be a single doc, a related ensemble, an entire corpus..).
• DocumentPart represents an image which is usually the scan of a page or a double page.
• Block and Line contains segmentation data.
• A Transcription is one representation of the text. It is unbound from the segmentation. Which means one DocumentPart can only have one segmentation but many transcriptions.
• LineTranscription makes the link between the segmentation and the content. It is also versioned to keep an history of modifications.
• OcrModel stores segmentation & recognition models.

2) api
Doesn't contain any models, it only defines django-rest-framework endpoints for internal and external communication.

3) imports
Historicaly Missleading name, it deals with both importing and exporting data.

4) helper apps: versioning, users, bootstrap, reporting

• versioning defines an abstract model to keep the history of a model instance
• users stores everything related to users but unrelated to the business logic;
• bootstrap is just a display layer to automate generating bootstrap css frontend forms.
• reporting helps getting feedback on long/asynchronous processes, and gather usage statistics.

Warning: It is very important to import one way only, from the more general app to the more specific app (to the business model). Not only does it protects against circular imports but it's also good design to have your data flow one way only.

Front-end apps

The front-end code is mostly vanilla JS and some jQuery, except the edition UI which is done in vue.js. We try to be "javascript lazy", meaning we try to use as little ajax as possible (because it's a lot harder to maintain). But sometimes the UI cost is too great, a good example of this is the edition UI which is almost entirely asynchronous JS leveraging the rest api with vue.js.
More info on the largest blocks:

1) edition panels
The edition UI relies mostly on JS because it is complex and interconnected.
There are for now 4 panels:

• source, that only displays the image (later on it will be the location of images annotations)
• segmentation
• visual transcription
• text transcription

2) wheelzoom
Somewhat independent library to deal with parallel zooming in standard DOM, SVG and CANVAS.
This breaks often..

3) baseline editor
Also an independent library whose goal is to allow the creation and correction of baselines and polygons over an image.
It is pretty much still under development and hopefully at some point will have it's own repository.

4) external libraries They are listed in front/src/vendor.js and front/package.json as npm dependencies:

• jQuery (mostly for the modal we are trying to avoid using it as much as possible)
• bootstrap + fontawesome(free)
• dropzone
• moment
• paper.js
• vue.js

Working with an asynchronous task queue: celery

Preview: https://docs.celeryproject.org/en/latest/django/first-steps-with-django.html

The channel server (daphne) and working with websockets

Preview: https://channels.readthedocs.io/en/latest/

statics and medias

[...]

Unit testing and coverage

Simply run

$ python manage.py test

To run the tests for a single app

$ python manage.py test api

Or a single test (example)

$ python manage.py test api.tests.DocumentViewSetTestCase.test_list

Coverage

$ coverage run --omit=/env/ manage.py test  # env dir may vary
$ coverage report -m