The goal of the document is to explain the architecture of the code to help new contributors.
It will assume that you are already familiar with the OCR workflow and the different bricks of the web stack (especially Django).
Running eScriptorium locally:
There are two ways to run eScriptorium, the 'easy' way with Docker,
and the hard way by installing and configuring everything yourself.
Unless you are familiar with coding inside a docker environment, the second solution is preferred for development because docker tends to get in your way during debugging.
Coding workflow and guidelines
- We use gitflow (no extension), so:
new feature/fix branch -> Pull Request -> Peer Review -> merge into develop -> merge develop into master -> tag master -> deploy
cia python guidelines
So basically pep8 with some additions and a few lifted restrictions, notably
Each line of text in your code should be at most 100 characters long.
eScriptorium follows Docker & Django good practices as much as possible.
The root directory is the docker-compose root. The
app/ directory is the Django root.
All templates can be found in the project folder
The JS code is located in
front/, it's built by npm + webpack.
They lie in app/apps/, they can be imported directly from anywhere because apps/ is added to the python PATH in settings.py.
It is the main application, it contains most models and business logic. The main models are:
Document, which represents a working directory (for the user it could be a single doc, a related ensemble, an entire corpus..).
DocumentPartrepresents an image which is usually the scan of a page or a double page.
Linecontains segmentation data.
Transcriptionis one representation of the text. It is unbound from the segmentation. Which means one DocumentPart can only have one segmentation but many transcriptions.
LineTranscriptionmakes the link between the segmentation and the content. It is also versioned to keep an history of modifications.
OcrModelstores segmentation & recognition models.
Doesn't contain any models, it only defines
django-rest-framework endpoints for internal and external communication.
Historicaly Missleading name, it deals with both importing and exporting data.
4) helper apps: versioning, users, bootstrap, reporting
- versioning defines an abstract model to keep the history of a model instance
- users stores everything related to users but unrelated to the business logic;
- bootstrap is just a display layer to automate generating bootstrap css frontend forms.
- reporting helps getting feedback on long/asynchronous processes, and gather usage statistics.
Warning: It is very important to import one way only, from the more general app to the more specific app (to the business model). Not only does it protects against circular imports but it's also good design to have your data flow one way only.
The front-end code is mostly vanilla JS and some jQuery, except the edition UI which is done in vue.js.
More info on the largest blocks:
1) edition panels
The edition UI relies mostly on JS because it is complex and interconnected.
There are for now 4 panels:
- source, that only displays the image (later on it will be the location of images annotations)
- visual transcription
- text transcription
Somewhat independent library to deal with parallel zooming in standard DOM, SVG and CANVAS.
This breaks often..
3) baseline editor
Also an independent library whose goal is to allow the creation and correction of baselines and polygons over an image.
It is pretty much still under development and hopefully at some point will have it's own repository.
4) external libraries
They are listed in
front/package.json as npm dependencies:
- jQuery (mostly for the modal we are trying to avoid using it as much as possible)
- bootstrap + fontawesome(free)
Working with an asynchronous task queue: celery
The channel server (daphne) and working with websockets
statics and medias
Unit testing and coverage
$ python manage.py test
To run the tests for a single app
$ python manage.py test api
Or a single test (example)
$ python manage.py test api.tests.DocumentViewSetTestCase.test_list
$ coverage run --omit=/env/ manage.py test # env dir may vary $ coverage report -m