Gradient Masked Averaging
This Merge Request implements a new Aggregator
subclass, GradientMaskedAveraging
, based on this paper, that provides with convergence analysis as well as empirical results that show its overall superiority to the default averaging.
Note that I did not provide with a shortname (e.g. "GMA") nor associated Strategy
preset, because I expect the latter API to be thoroughly revised soon. For the same reason, I did not yet implement unit tests (but ran the Heart-UCI example using it, and examined manually that the maths were good).
On the side, I found an error in Vector.__rsub__
that needs integrating into the main branch ASAP: basically, any x - vector
where x
is not a Vector but a scalar value would return vector + x
instead of the proper result.