Enable Fairness-Aware Federated Learning
This Merge Request extends the capabilities of DecLearn, bringing Fairness to the party.
It is the result of a year-long collaboration with Michaël Perrot and Brahim Erraji to design and evaluate algorithms to learn models under fairness constraints in a federated learning setting, using either newly-introduced algorithms or existing ones from the litterature. This collaboration first produced a legacy fairgrad branch that was used to conduct preliminary experiments resulting in a publication at the CAp 2024 conference. The current MR is the result of additional efforts to clean up, improve and robustify the resulting code, introducing a shared and extensible API abstracted from the algorithms at hand, adding support for secure aggregation, and submitting the code to extensive unit and functional tests.
Overall, this MR:
- Adds a new
declearn.fairness
submodule, that defines an API and concrete algorithms for fairness-aware federated learning.- The API is dedicated to group-fairness, enforced by altering the model training process.
- Currently, 4 distinct fairness definitions are supported (Accuracy Parity; Demographic Parity; Equality of Opportunity; Equalized Odds).
- Currently, 3 distinct algorithms are implemented (Fed-FairGrad; Fed-FairBatch/FedFB; FairFed), plus a monitoring mode to measure fairness of baseline federated learning.
- The API and all current algorithms support using SecAgg to protect exchanged fairness-related quantities.
- Integrates fairness algorithms as part of the
declearn.main
orchestration and configuration tools.-
declearn.main.config.FLOptimConfig
now exposes a "fairness" field for using a fairness algorithm as part of a federated optimization strategy. -
declearn.main.config.FLRunConfig
now exposes a "fairness" field to specify some hyper-parameters, which are otherwise automatically and dynamically filled. - Both
FederatedServer
andFederatedClient
have new methods that implement routines related to fairness evaluation and enforcement.
-
- Adds a new
declearn.training
submodule, that relocates the formerdeclearn.main.training
. - Adds extensive documentation and tests about the newly-introduced fairness-related features.
Overall to-do list:
-
Design and implement a shared API for fairness-aware federated learning. -
Integrate fairness-aware federated learning to declearn.main
. -
(Re-)implement Fed-FairGrad, Fed-FairBatch and FairFed, adding SecAgg support. -
Add unit tests for the introduced API and concrete algorithms. -
Add functional tests for the algorithms over a toy dataset. -
Write an extensive guide towards end-users. -
Finalize the implementation up to the last hanging details.
Remaining tasks:
-
Change the wayFairnessControllerServer
redefines the usedAggregator
? -
Add parsing options for server-side fairness controller choice. -
Add parsing options for client-side fairness dataset specification.
Future tasks:
- Modularize fairness rounds' frequency (and evaluation ones').
- Enable delaying fairness setup (run some rounds without fairness constraints).