Mentions légales du service

Skip to content

Fix Scaffold corrections' application.

ANDREY Paul requested to merge hotfix/scaffold into develop

The current Scaffold implementation is wrong: the correction states are properly computed (as per the secondary version of the algortihm published as an annex to the TCT paper), but they are wrongfully applied as part of the TorchTrainingPlan computations. Instead of \nabla_{i,t,k} - h_i^t, the code performs \nabla_{i,t,k} + h_i^t.

This hotfix MR changes that to restore proper computations. Running the 101 notebook with evaluation on both local and global updates shows how the model used to diverge and is now properly converging.

Merge request reports