* Add `test_set_state_results` to verify that resetting a modules' state
  enables running the same computation twice. This should enable detecting
  cases when information is missing from the returned state dict.

* Note that at the moment noise-addition modules are ignored. We could look
  into a way to access and restore RNG states (when CSPRNG is not used).

Core changes to the global test suite:
* Add some device-placement verifications to the generic Model test
  suite. This should be refactored into more unitary tests as part
  of a distinct effort to revise and improve this test suite.
* For TensorFlow and Torch, parametrize the whole test suite to run
  either on CPU and GPU, and run it once per available device type.
* Ensure Vector and OptiModule unit tests run on CPU.

Changes to some 'Model' tests to run on GPU:
* `test_compute_batch_gradients_np`: allow for small numerical
  discrepancies that may result from running the test on GPU.
* `test_apply_updates`: correct the test, that had not been
  updated since `Model.get_weights` stopped to systematically
  use `NumpyVector` as return type.

Changes to 'TorchModel' tests to run on GPU:
* Override `test_serialization` due to torch-serialization relying
  on pickle. Replace pickles' comparison with a more shallow (but
  less susceptible to fail for unknown reasons) test that ensures
  a reloaded model shares the same structure of modules.

BIGAUD Nathan authored 2 years ago and ANDREY Paul committed 2 years ago

* Move up the listing of frameworks and the mock-gradient-vectors-generating
  util class to a new private submodule of `declearn.test_utils`.
* Add a shared framework-providing fixture to the optimizer tests' module.
* Harmonize relative imports' code (preserving the `sys.path` trick).

Co-authored-by: Paul Andrey <paul.andrey@inria.fr>

BIGAUD Nathan authored 2 years ago and ANDREY Paul committed 2 years ago

* Implement `GaussianNoiseModule`, to enable adding gaussian noise
  to step-wise batch-averaged gradients as part of local DP-SGD or
  to clients-averaged updates as part of central DP-SGD.
* Implement an abstract `NoiseModule` layer, paving the way for
  other noise-addition mechanisms.

This commit squashes some incremental work by multiple authors:
* Nathan Bigaud implemented `GaussianNoiseModule` after doing the
  scientific and technical review of requirements, notably as to
  the use of a cryptographically-safe RNG.
* Paul Andrey abstracted part of the code into the `NoiseModule`
  class, fixed some pending todos, and adjusted unit tests.

Co-authored-by: Paul Andrey <paul.andrey@inria.fr>

* Run [black](https://github.com/psf/black/) to reformat the existing codebase.
* Add a limited amount of `# fmt: off` comments to protect some code blocks that
  are believed to achieve better readability in their previous form.
* Add (limited) black configuration parameters to `pyproject.toml` for their use
  by any contributor to declearn.