refactor the Experiment class to be more user friendly
This is a proposal. The use of Experiment is not very natural to me.
Rewrite Experiment() with goals of:
- provide step by step interface (constructor, set model, set strategy, set nodes and/or datasets, run one round, etc.) + getters/setters for object oriented design
- split functions to increase code quality (eg too much stuff in
__init__
) - keep backward compatibility with current interface and behaviour (may still use
e = Experiment(all params)
+e.run()
as much as possible
Background: experiment is a container, it does not make decisions, it asks job/strategy/aggregator.
Some proposed improvments:
-
the save_breakpoint parameter should be declared/used at runtime ( run() ) and not at init() time -
the Experiment.save_breakpoint() should be callable by the user at any time (and not only been part of the run() algorithm) -
provide a run_once() call to run a new round at any time (of course run() must call run_once() ) -
breakpoint selection must be more user friendly -
the available breakpoints detection must be more user_friendly/robust (ex: what about a date based research ? ) -
actual algorithm to create/detect breakpoint dir/files is too complicated (it should be understandable by a human in less than a minute :) ) -
of course, the Experiment class must "store/vehiculate" all necessary data to provide these features -
CHECK WITH TEAM - prototype: type, default values, etc. -
methods behaviour : return codes, raises -
[ ] we may use a persistent storage (sqlite ?) to ease all these features -
adapt unit tests -
adapt documentation and tutorials -
adapt sample notebooks
Basically, we have to refactor completly the Experiment and think "out of the box" on how to use the Experiment class....