Fix output handling when the preprocess command is run with the progress bar enabled
If errors/warnings are output when running dnadna preprocess
they get interleaved with the progress bar in such a way that the progress bar gets duplicated. E.g.
29%|█████████▋ | 5883/20000 [46:47<1:18:21, 3.00scenario/s]01/05/2020 15:50:01; WARNING; could not load scenario 5886 replicate 0 from "scenarios/scenario_05886/one_event_05886_00.npz": FileNotFoundError(2, 'No such file or directory'); entire scenario will be skipped
29%|██████████▎ | 5885/20000 [46:47<57:11, 4.11scenario/s]01/05/2020 15:50:02; WARNING; could not load scenario 5888 replicate 0 from "scenarios/scenario_05888/one_event_05888_00.npz": FileNotFoundError(2, 'No such file or directory'); entire scenario will be skipped
29%|█████████▋ | 5887/20000 [46:48<1:37:11, 2.42scenario/s]01/05/2020 15:50:03; WARNING; could not load scenario 5890 replicate 0 from "scenarios/scenario_05890/one_event_05890_00.npz": FileNotFoundError(2, 'No such file or directory'); entire scenario will be skipped
29%|█████████▋ | 5888/20000 [46:49<1:17:50, 3.02scenario/s]01/05/2020 15:50:03; WARNING; could not load scenario 5891 replicate 0 from "scenarios/scenario_05891/one_event_05891_00.npz": FileNotFoundError(2, 'No such file or directory'); entire scenario will be skipped
91%|█████████████████████████████▏ | 18235/20000 [2:22:32<15:18, 1.92scenario/s]
This happens because the progress bar manager needs to have total control over the terminal in order for output to be managed properly. If some code writes to the terminal in a way the progress bar is unaware of, then it has to re-draw the progress bar.
This is already fixed in the dnadna train
using utils.stdio_redirect_tqdm when setting up the logging for the command. This ensures that the "stdout" used by the logger is in a wrapper that makes it pass through the progress bar manager when writing to stdout--this way the progress bar can clear and redraw itself properly when additional output is written to stdout.
So this already works, I just forgot to do it for the preprocess command.