Handling multiple dataset name format
Issue
the actual dataset format is too rigid because consistent
means "always the same s_padding,r_padding scheme for scenario, replicates".
The actual code doesn't take into account not padded dataset format, which I add as a functionality (I needed it for cattle
dataset)
Actual dataset format
_ simulations/ # root directory of all simulation datasets; arbitrary name
\_ model-A/
\_ model-A_params.csv # the scenario parameters table
|_ model-A_simulation_config.yml # the simulation config file
|_ scenario_000/
\_ model-A_000_00.npz # scenario 0 replicate 0
...
|_ model-A_000_NN.npz
|_ scenario_001/
\_ model-A_001_00.npz # scenario 1 replicate 0
...
|_ model-A_001_NN.npz
...
|_ scenario_NNN/
\_ model-A_NNN_00.npz
...
|_ model-A_NNN_NN.npz
which stands for:
scenario_{scenario:0{s_padding}d}/{dataset_name}_{scenario:0{s_padding}d}_{replicate:0{r_padding}d}.npz
Now it can handles:
_ simulations/ # root directory of all simulation datasets; arbitrary name
\_ model-A/
\_ model-A_params.csv # the scenario parameters table
|_ model-A_simulation_config.yml # the simulation config file
|_ scenario_000/
\_ model-A_000_00.npz # scenario 0 replicate 0
...
|_ model-A_000_NN.npz
|_ scenario_001/
\_ model-A_001_00.npz # scenario 1 replicate 0
...
|_ model-A_001_NN.npz
...
|_ scenario_N/
\_ model-A_N_0.npz
...
|_ model-A_N_N.npz
|_ scenario_NNN/
\_ model-A_NNN_00.npz
...
|_ model-A_NNN_NN.npz
which stands for:
scenario_{scenario:0{s_padding}d}/{dataset_name}_{scenario:0{s_padding}d}_{replicate:0{r_padding}d}.npz
or, in case this file doesn't exist, look for:
scenario_{scenario}/{dataset_name}_{scenario}_{replicate}.npz