Integration tests + CI : CMake -> pytest
This branch reimplements Batsim integration tests with pytest instead of CMake.
This should permit to finally use Meson instead of CMake!
Modification overview.
- Reimplement tests with pytest.
- Provide a
release.nix
to build Batsim, its doxygen doc, its integration tests. - Use
release.nix
in the CI script. CI can now be blazing fast: https://framagit.org/batsim/batsim/-/jobs/401010
Pros of using pytest over CMake.
- Maintainability++. @cmommess will no longer have "CMake is terrible" excuse to not write tests!
- Fixtures are amazing.
- Mature, convenient and powerful runtime engine.
- HTML reports are convenient to inspect an instance that failed.
- Integration tests can be fully separated from the Batsim binary. This permits a very clean test workflow.
- Tests can be skipped or marked as
expected to fail
.- Permits to keep instances known to fail in the test suite and to follow their evolution.
Things lost in the process.
- Tests cannot be executed in parallel for now. @mmercier suggested to use unix sockets in the output directory of each test to fix this.
- Unittests are no longer executed. Current unittests are very insufficient, ugly and should be rewritten — this will be for another issue/MR.
Edited by Millian Poquet