Draft: Feature/torch plan dp
Torch training plan has been modified to enable/disable DP by simply providing a 'DP_args' dictionary in training_args containing instructions on the type of DP training to be performed (local or central) and privacy parameters (sigma and clipping). The notebook monai-2d-image-registration-LDP-CDP.ipynb shows an example of not-DP vs DP training on image registration task with monai.
Few points to discuss:
- effects of different combinations of DP parameters can be further explored.
- the method for model validation and correction to make models compatible with DP using
ModuleValidatorfromopacusshould be integrated in the training plan. - can we define a method to provide a custom optimizer inside the training routine? This will allow to pass directly the learning rate for instance, provided among training arguments.
- basic example using
opacuswith MNIST should be modified according to the new training plan.
Edited by CANSIZ Sergen