Potential ordering issue in FedAvg implementation
in our FedAvg aggregate
function we do the following:
model_params_processed = [list(model_param.values())[0] for model_param in model_params] # model params are contained in a dictionary with node_id as key, we just retrieve the params
weights_processed = [weight if isinstance(weight, float) else list(weight.values())[0] for weight in weights]
weights_processed = self.normalize_weights(weights_processed)
That is to say, we build a list of model parameters based on an input list model_params
and a list of weights based on an input list weights
.
Unfortunately, these lists are populated by default_strategy
in a different way:
-
model_params
is populated based on the order of replies -
weights
is populated based on the order of iteration ofFederatedDataset.data().items()
I have not taken the time to confirm this yet, but I'm writing it here for future investigation.