Proper testing of PyBatsim
Here are some notes for a proper testing strategy of PyBatsim.
I suggest we start by regression testing as we have almost no tests.
class of tests
1. unit testing
aim
testing the Python code of PyBatsim, ensuring functions behave as documented/expected, classes hierarchies, …
2. integration testing
aim
test we correctly wrap Batsim, and communication is correct
tools
3.a. functional testing
aim
test the simulation is correctly done
3.b. regression testing
aim
test the output of the simulation did not change
tools
- https://gitlab.com/uweschmitt/pytest-regtest / https://pypi.org/project/pytest-regtest/
- https://pytest-regressions.readthedocs.io/en/latest/
Resources
- (linting) https://pypi.org/project/pytest-pylint/
- (property-based testing) https://hypothesis.works/
- (mutation testing) https://github.com/sixty-north/cosmic-ray
- (coverage) https://pytest-cov.readthedocs.io/en/latest/
Note: we could use pytest markers to distinguish tests suites/classes (https://docs.pytest.org/en/latest/how-to/mark.html#registering-marks)