Draft: Feature/torch plan dp
Torch training plan has been modified to enable/disable DP by simply providing a 'DP_args'
dictionary in training_args
containing instructions on the type of DP training to be performed (local or central) and privacy parameters (sigma and clipping). The notebook monai-2d-image-registration-LDP-CDP.ipynb
shows an example of not-DP vs DP training on image registration task with monai.
Few points to discuss:
- effects of different combinations of DP parameters can be further explored.
- the method for model validation and correction to make models compatible with DP using
ModuleValidator
fromopacus
should be integrated in the training plan. - can we define a method to provide a custom optimizer inside the training routine? This will allow to pass directly the learning rate for instance, provided among training arguments.
- basic example using
opacus
with MNIST should be modified according to the new training plan.