MP-SPDZ configuration file potential conflicts
To avoid any file conflict with MP-SPDZ,
create and use a dedicated copy of ${FEDBIOMED_DIR}/modules/MP-SPDZ
for each MP-SPDZ computation and for each component.
Clean out this copy when the computation is finished.
-
adapt fedbiomed_configure_secagg
-
adapt fedbiomed_mpc
-
adapt common MPCController
to use a unique temp dir with a dedicated copy of MP-SPDZ -
adapt node side SecaggSetup
to use a dedicated MPCController -
adapt researcher side SecaggContext
to use a dedicated MPCController [ ] clean out temporary copy / files at the end of the computation-
adapt unit tests
Detailed explanation
MP-SPDZ computation need to use the Player-Data
directory of all computation configuration, input, output files.
In current version, we use a single ${FEDBIOMED_DIR}/modules/MP-SPDZ
directory in a clone for all MP-SPDZ computations,
thus a single Player-Data
.
This is fine in simple scenarios but this may lead to conflicts in cases such as:
- multiple components running in the same clone but not setting up the same computation (eg 1 researcher setting up 2 experiments with 2 sets of nodes ; 2 researchers each one setting up 1 experiment)
- one component setting up multiple experiments in parallel (this case currently does not exist)
- robustification (eg: clean
Player-Data
directory, retry after error)
We should ensure that the Player-Data
directory is protected from conflicts with any other MP-SPDZ computation that may be overlapping in time.
The secure way is :
- to use a dedicated copy of
${FEDBIOMED_DIR}/modules/MP-SPDZ
for each MP-SPDZ computation and for each component. Before starting an MP-SPDZ computation on (eg)node_12345
we create a dedicatedMPCController
with its own copy of the${FEDBIOMED_DIR}/modules/MP-SPDZ
directory tree in a unique directory (egnode_12345/unique_seed
). - clean out this copy at the end of the computation
Note: ${FEDBIOMED_DIR}/modules/MP-SPDZ
is currently ~7MB on Linux x86_64, so this sounds reasonable regarding disk usage.
Note: some intermediate scenario (eg: one clone per component) may be investigated. But they still won't support future advanced scenario where we have several MP-SPDZ computations in parallel on one component.