Commit 24507b66 authored by HE Kun's avatar HE Kun
Browse files

doc: add source code and output data of benchmark tests

parent a0dd7b1b
Pipeline #302355 failed with stages
in 99 minutes and 44 seconds
......@@ -3543,6 +3543,7 @@ AC_CONFIG_COMMANDS([executable-scripts], [
chmod +x doc/doxygen/doxygen_filter.sh
chmod +x doc/doxygen_dev/doxygen_filter.sh
chmod +x starpupy/examples/execute.sh
chmod +x starpupy/benchmark/execute.sh
chmod +x julia/examples/execute.sh
for x in \
tests/microbenchs/tasks_data_overhead.sh \
......@@ -3592,12 +3593,15 @@ AC_CONFIG_COMMANDS([executable-scripts], [
julia/examples/axpy/axpy.sh \
julia/examples/gemm/gemm.sh \
julia/examples/cholesky/cholesky.sh \
starpupy/benchmark/tasks_size_overhead.sh \
starpupy/benchmark/tasks_size_overhead.gp \
starpupy/benchmark/test_handle_perf.sh \
starpupy/benchmark/test_handle_perf_pickle.sh \
starpupy/examples/starpu_py.sh \
starpupy/examples/starpu_py_handle.sh \
starpupy/examples/starpu_py_np.sh \
starpupy/examples/starpu_py_parallel.sh \
starpupy/examples/starpu_py_partition.sh \
starpupy/examples/tasks_size_overhead.sh \
; do
test -e $x || ( mkdir -p $(dirname $x) && ln -sf $ac_abs_top_srcdir/$x $(dirname $x) )
done
......@@ -3673,6 +3677,8 @@ AC_OUTPUT([
starpupy/src/Makefile
starpupy/examples/Makefile
starpupy/examples/execute.sh
starpupy/benchmark/Makefile
starpupy/benchmark/execute.sh
examples/Makefile
examples/stencil/Makefile
tests/Makefile
......
......@@ -225,7 +225,7 @@ In Python, there are some special Python objects which wrap access to an underly
StarPU uses virtually shared memory manager to process Python objects supporting buffer protocol by default, that allows to minimize data transfers between accelerators, and avoids to copy the object each time. Therefore, the program can be optimized especially for a large amount of data. If you want to deactivate using virtually shared memory manager, you only need to set the option "arg_handle" to "False", and its default value is "True".
We will show the operation performances below. The operation that we test is Numpy addition (<c>numpy.add</c>), and the array size is 10, 20, ..., 100, 200, ..., 1000, 2000, ..., 10000, 2000, ..., 100000,200000, ..., 1000000, 2000000, ..., 10000000, ..., 50000000. We compared three cases: first, using virtually shared memory manager and returning future object, second, using virtually shared memory manager and returning handle object, third, without using StarPU task submitting, but directly calling <c>numpy.add</c> function.
We will show the operation performances below (Running <c>test_handle_perf.sh</c>). The operation that we test is Numpy addition (<c>numpy.add</c>), and the array size is 10, 20, ..., 100, 200, ..., 1000, 2000, ..., 10000, 2000, ..., 100000,200000, ..., 1000000, 2000000, ..., 10000000, ..., 50000000. We compared three cases: first, using virtually shared memory manager and returning future object, second, using virtually shared memory manager and returning handle object, third, without using StarPU task submitting, but directly calling <c>numpy.add</c> function.
In first figure, we compare the task submission time when using StarPU virtually shared memory manager and the program execution time without using StarPU. We can see that there is an obvious optimization using StarPU when the test array size is large. Even then the program execution still needs to be finished as shown in second figure, which takes the same time trend as the program execution time without using StarPU, but the advantage is that during this time we can perform other operations.
......@@ -921,7 +921,7 @@ In order to reflect this influence more intuitively, we make a performance compa
By default, StarPU uses virtually shared memory manager for Python objects supporting buffer protocol that allows to minimize data transfers. But in the case multi-interpreter, if we do not use virtually shared memory manager, data transfer can be realized only with the help of cloudpickle.
We will show the operation performances below. The operation that we test is Numpy addition (<c>numpy.add</c>), and the array size is 10, 20, ..., 100, 200, ..., 1000, 2000, ..., 10000, 2000, ..., 100000,200000, ..., 1000000, 2000000, ..., 10000000, ..., 50000000. We compared three cases: first, using virtually shared memory manager, second, without using virtually shared memory manager, third, without using StarPU task submitting, but directly calling <c>numpy.add</c> function.
We will show the operation performances below (Running <c>test_handle_perf_pickle.sh</c>). The operation that we test is Numpy addition (<c>numpy.add</c>), and the array size is 10, 20, ..., 100, 200, ..., 1000, 2000, ..., 10000, 2000, ..., 100000,200000, ..., 1000000, 2000000, ..., 10000000, ..., 50000000. We compared three cases: first, using virtually shared memory manager, second, without using virtually shared memory manager, third, without using StarPU task submitting, but directly calling <c>numpy.add</c> function.
In first figure, we compare the submission time when using StarPU and the execution time without using StarPU. We can see that there is still an obvious optimization using StarPU virtually shared memory manager when the test array size is large. However, if only using cloudpickle, StarPU Python interface cannot provide an effective optimization. And in second figure, we can see that the same operation will take more time to finish the program execution when only using cloudpickle.
......
......@@ -18,4 +18,5 @@ include $(top_srcdir)/starpu-subdirtests.mk
SUBDIRS = src
SUBDIRS += examples
SUBDIRS += benchmark
# StarPU --- Runtime system for heterogeneous multicore architectures.
#
# Copyright (C) 2021 Université de Bordeaux, CNRS (LaBRI UMR 5800), Inria
#
# StarPU is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or (at
# your option) any later version.
#
# StarPU is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the GNU Lesser General Public License in COPYING.LGPL for more details.
#
include $(top_srcdir)/starpu.mk
SUBDIRS =
CLEANFILES = *.gcno *.gcda *.linkinfo
if STARPU_USE_MPI_MASTER_SLAVE
TESTS_ENVIRONMENT = $(MPI_RUN_ARGS) MPI_LAUNCHER="$(MPI_LAUNCHER)" LOADER_ARGS="--mpirun"
endif
EXTRA_DIST = \
tasks_size_overhead.py \
tasks_size_overhead.sh \
test_handle_perf.py \
test_handle_perf.sh \
test_handle_perf_pickle.py \
test_handle_perf_pickle.sh
python_sourcesdir = $(libdir)/starpu/python
dist_python_sources_DATA = \
tasks_size_overhead.py \
test_handle_perf.py \
test_handle_perf_pickle.py
#!@REALBASH@
# StarPU --- Runtime system for heterogeneous multicore architectures.
#
# Copyright (C) 2021 Université de Bordeaux, CNRS (LaBRI UMR 5800), Inria
#
# StarPU is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation; either version 2.1 of the License, or (at
# your option) any later version.
#
# StarPU is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the GNU Lesser General Public License in COPYING.LGPL for more details.
#
benchmarkdir=@STARPU_SRC_DIR@/starpupy/benchmark
modpath=@STARPU_BUILD_DIR@/src/.libs${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}
pypath=@STARPU_BUILD_DIR@/starpupy/src/build:$PYTHONPATH
LOADER=@PYTHON@
if test -z "$MPI_LAUNCHER"
then
MPI_LAUNCHER="mpiexec -np 2"
fi
mpi=""
gdb=""
read_arg()
{
do_shift=0
if test "$1" == "--valgrind"
then
export PYTHONMALLOC=malloc
LOADER="valgrind --track-origins=yes @PYTHON@"
do_shift=1
elif test "$1" == "--gdb"
then
gdb="gdb"
if test "$mpi" == "mpi"
then
LOADER="$MPI_LAUNCHER xterm -sl 10000 -hold -e gdb --args @PYTHON@"
else
LOADER="gdb --args @PYTHON@"
fi
do_shift=1
elif test "$1" == "--mpirun"
then
mpi="mpi"
if test "$gdb" == "gdb"
then
LOADER="$MPI_LAUNCHER xterm -sl 10000 -hold -e gdb --args @PYTHON@"
else
LOADER="$MPI_LAUNCHER @PYTHON@"
fi
do_shift=1
fi
}
for x in $*
do
read_arg $x
if test $do_shift == 1
then
shift
fi
done
for x in $LOADER_ARGS
do
read_arg $x
done
benchmarkfile=$1
if test -f $benchmarkfile
then
pythonscript=$benchmarkfile
elif test -f $benchmarkdir/$benchmarkfile
then
pythonscript=$benchmarkdir/$benchmarkfile
else
echo "Error. Python script $benchmarkfile not found in current directory or in $benchmarkdir"
exit 1
fi
shift
set -x
PYTHONPATH=$pypath LD_LIBRARY_PATH=$modpath exec $LOADER $pythonscript $*
{"program_submit": [8.262395858764649e-05, 7.87496566772461e-05, 8.388757705688477e-05, 8.502006530761719e-05, 8.219480514526367e-05, 8.084774017333985e-05, 8.44120979309082e-05, 8.258819580078124e-05, 8.057355880737304e-05, 0.00012764930725097656, 0.00010379552841186524, 8.761882781982422e-05, 0.00010143518447875976, 9.701251983642578e-05, 0.0001071929931640625, 9.931325912475586e-05, 9.506940841674805e-05, 0.0001064300537109375, 0.00011214017868041993, 0.00011497735977172852, 0.00012351274490356444, 0.00012520551681518554, 0.00012085437774658203, 0.00013319253921508789, 0.00013458728790283203, 0.00013402700424194335, 0.00013554096221923828, 0.00013949871063232422, 0.00015128850936889647, 0.00015000104904174804, 0.00014837980270385743, 0.00014371871948242186, 0.00014574527740478515, 0.00014716386795043945, 0.00014929771423339845, 0.00015578269958496093, 0.0001841902732849121, 0.00018571615219116212, 0.0001846432685852051, 0.0001864790916442871, 0.00018395185470581054, 0.000183868408203125, 0.00018781423568725586, 0.00018777847290039063, 0.00018808841705322265, 0.00018830299377441405, 0.00019060373306274415, 0.00018875598907470704, 0.00018696784973144532, 0.00018854141235351561, 0.00018606185913085937, 0.00018852949142456055, 0.00018764734268188475, 0.00018815994262695314, 0.00018815994262695314, 0.00019018650054931642, 0.00018769502639770508, 0.00019502639770507812, 0.00018912553787231445], "program_await": [0.00017031431198120118, 0.0001456141471862793, 0.00015496015548706056, 0.00015997886657714844, 0.00017446279525756836, 0.0001892685890197754, 0.00019006729125976564, 0.00020149946212768554, 0.0002164602279663086, 0.00025705099105834963, 0.0003251194953918457, 0.000427556037902832, 0.0005640864372253418, 0.0006380081176757812, 0.0007571578025817871, 0.0008730530738830566, 0.0009511828422546387, 0.001069343090057373, 0.001205575466156006, 0.002228569984436035, 0.003335237503051758, 0.004403102397918701, 0.005414402484893799, 0.006395328044891358, 0.007526791095733643, 0.00861794948577881, 0.009656333923339843, 0.010649263858795166, 0.021246862411499024, 0.03184399604797363, 0.04234309196472168, 0.05369188785552979, 0.06324629783630371, 0.07432821989059449, 0.08611912727355957, 0.10168068408966065, 0.149496328830719, 0.29356046915054324, 0.4376141309738159, 0.572070300579071, 0.7170849323272706, 0.859172773361206, 0.9969146847724915, 1.127133071422577, 1.2918678283691407, 1.432572591304779, 2.866915929317474, 4.375425326824188, 5.660778772830963, 7.068734109401703, 8.48449261188507, 9.803147995471955, 11.349043202400207, 12.741027545928954, 14.163181364536285, 28.02475572824478, 42.50216084718704, 56.65349878072739, 70.73518785238267]}
\ No newline at end of file
{"program_submit": [0.00011574029922485351, 0.00012149810791015626, 0.0001286625862121582, 0.0001118302345275879, 0.00011289119720458984, 0.00012046098709106445, 0.00011543035507202148, 0.00011731386184692383, 0.00011837482452392578, 0.00011627674102783203, 0.00011805295944213867, 0.00011998414993286133, 0.00011827945709228516, 0.00011769533157348632, 0.00011546611785888672, 0.00011690855026245118, 0.00011740922927856445, 0.0001178741455078125, 0.00011690855026245118, 0.00012013912200927734, 0.0001299738883972168, 0.00012849569320678712, 0.00012792348861694335, 0.00013262033462524414, 0.0001320958137512207, 0.00013346672058105468, 0.00013196468353271484, 0.00013463497161865235, 0.00014846324920654298, 0.0001431107521057129, 0.00014771223068237304, 0.00016404390335083007, 0.00015833377838134766, 0.00016291141510009765, 0.00017646551132202148, 0.00017708539962768555, 0.00017155408859252929, 0.00020309686660766602, 0.00021284818649291992, 0.0002152562141418457, 0.0002176642417907715, 0.00021895170211791993, 0.00022068023681640626, 0.00022004842758178712, 0.00022089481353759766, 0.00022139549255371094, 0.0002246379852294922, 0.0002233266830444336, 0.00022370815277099608, 0.00022314786911010743, 0.00022472143173217772, 0.00022431612014770508, 0.00022656917572021483, 0.00022689104080200195, 0.00022411346435546875, 0.00022494792938232422, 0.00022774934768676758, 0.0002269744873046875, 0.00022503137588500977], "program_await": [6.961822509765625e-05, 0.00010547637939453124, 9.162425994873047e-05, 0.00010483264923095703, 0.0001168370246887207, 0.00013866424560546876, 0.0001443028450012207, 0.0001631498336791992, 0.00018345117568969728, 0.00018388032913208008, 0.0003322005271911621, 0.0004639744758605957, 0.0006118297576904296, 0.000729823112487793, 0.000883793830871582, 0.0010669589042663573, 0.0012346625328063965, 0.0013690829277038575, 0.0014939188957214355, 0.003033351898193359, 0.004578971862792968, 0.006168460845947266, 0.007474327087402343, 0.00906531810760498, 0.01057603359222412, 0.012390708923339844, 0.013441109657287597, 0.015353572368621827, 0.031767284870147704, 0.04568679332733154, 0.060425209999084475, 0.07672661542892456, 0.09176487922668457, 0.10764046907424926, 0.12285939455032349, 0.1370265245437622, 0.15305562019348146, 0.2996717929840088, 0.45927752256393434, 0.6155787467956543, 0.7734602332115174, 0.9111344814300537, 1.044310462474823, 1.210589587688446, 1.3467282056808472, 1.5133751511573792, 2.788231897354126, 4.4973290920257565, 5.9483099460601805, 7.477483689785004, 8.950010931491851, 10.806987166404724, 11.9709943652153, 14.092496085166932, 15.045314407348632, 30.01750999689102, 45.94067543745041, 60.19484902620316, 76.43260563611985]}
\ No newline at end of file
{"program_submit": [2.536773681640625e-05, 4.4453144073486325e-05, 6.508827209472656e-05, 8.26716423034668e-05, 0.00010216236114501953, 0.00012161731719970704, 0.00014106035232543945, 0.00015976428985595704, 0.00017976760864257812, 0.00019911527633666992, 0.00039446353912353516, 0.0005987048149108887, 0.000825953483581543, 0.0010475993156433105, 0.0013073444366455077, 0.0015305519104003907, 0.0017645835876464843, 0.001983630657196045, 0.0021353602409362794, 0.004328668117523193, 0.006612396240234375, 0.008958721160888672, 0.011139476299285888, 0.013313674926757812, 0.01587390899658203, 0.017885589599609376, 0.020295143127441406, 0.022349679470062257, 0.045082378387451175, 0.0701027512550354, 0.09108394384384155, 0.11200623512268067, 0.13735190629959107, 0.15988270044326783, 0.18046345710754394, 0.20119695663452147, 0.22502063512802123, 0.450369131565094, 0.6755733489990234, 0.900255560874939, 1.125024712085724, 1.3448821663856507, 1.5746421575546266, 1.7948955059051515, 2.0229355096817017, 2.26053991317749, 4.554677307605743, 6.828912627696991, 9.118583214282989, 11.22865629196167, 13.646826136112214, 15.671090149879456, 17.856031489372253, 20.36303472518921, 22.730864822864532, 45.55766191482544, 67.96549625396729, 89.24352589845657, 111.89673389196396]}
\ No newline at end of file
{"list_std11": [1.9151670899682628e-05, 5.788118379318557e-06, 1.0446771087870387e-05, 2.068636348199092e-05, 6.5913257106032876e-06, 9.558333251774497e-06, 1.7440955687391782e-05, 1.5430829522043604e-05, 5.966973515202939e-06, 0.0001656195521347478, 4.923203285114979e-05, 1.0922562389535084e-05, 1.981668775886889e-05, 2.0552368609092223e-05, 2.6784253900741408e-05, 2.6936299069208564e-05, 2.489245673427011e-05, 2.665481777224686e-05, 3.986484686922501e-05, 2.7789063605648754e-05, 3.0932236186082846e-05, 2.8842194284546936e-05, 3.142024771308521e-05, 2.877968195528069e-05, 2.547992616477946e-05, 2.6471670945184114e-05, 2.3311757615069734e-05, 2.3857651047325055e-05, 1.4578457051925693e-05, 8.845796030247025e-06, 8.93530766884032e-06, 3.86735064739995e-06, 4.8585569846869e-06, 4.675151600932587e-06, 6.9607566782209664e-06, 1.318580560262094e-05, 4.54246755173099e-06, 3.359725687352635e-06, 6.806850982751788e-06, 7.735039734728724e-06, 4.490135137645415e-06, 5.377355111818848e-06, 5.985695990936365e-06, 7.74050107463345e-06, 8.842624719482053e-06, 1.006531318718769e-05, 1.2435227863841713e-05, 1.1018175460807425e-05, 9.765393637125e-06, 1.2965744174334385e-05, 1.0202532461007494e-05, 1.114366097648936e-05, 9.926702014602333e-06, 9.558552349305853e-06, 9.989230827425845e-06, 1.2417803144144e-05, 9.436733921381509e-06, 3.5589786838343424e-05, 1.0483535021556742e-05], "list_std12": [2.7673022682030845e-05, 1.7580440244798432e-05, 1.3110278038726028e-05, 1.3584904811180377e-05, 1.2684480479122468e-05, 2.8821337305455008e-05, 1.4192699971544145e-05, 2.064334714599848e-05, 2.7473664087425258e-05, 7.837800588417222e-05, 2.1851187538955995e-05, 3.879955930619274e-05, 7.772870905899187e-05, 2.455315389856526e-05, 4.181681982940989e-05, 4.73802897912971e-05, 5.132998254999404e-05, 4.479473571454726e-05, 8.234772292205595e-05, 9.575139413121605e-05, 0.00011281319361303846, 0.00017424667518253426, 0.0001672889786652232, 0.00018400356882745136, 0.0001504442666593392, 0.0002778304219898809, 0.00021549818105126974, 0.00046259884159839367, 0.0003127772142910602, 0.0004729424040623762, 0.001009885907578641, 0.000731414851497155, 0.0018245962408026729, 0.001955963465583656, 0.0029580743220630487, 0.010841519179011513, 0.00298505954537821, 0.002896428740254863, 0.007815710088135802, 0.004301368775002543, 0.007739232652029031, 0.012430279720447369, 0.00889885754384029, 0.009029745554146268, 0.0166429282685462, 0.015606690908949295, 0.03817697845337797, 0.09876549178930746, 0.05645560195168419, 0.07071736667165543, 0.08871704660486307, 0.10430014501608156, 0.11189614038402397, 0.11196088252500391, 0.10286490468138862, 0.2045925007979157, 0.32049546062102685, 0.5325382212196269, 0.5216837557478257], "list_std21": [2.060493498857569e-05, 1.4197716020730588e-05, 7.120664814263668e-05, 8.70574972686656e-06, 9.635209911322592e-06, 1.2490536198297664e-05, 9.941097778230859e-06, 9.756773362673984e-06, 1.2289060048580495e-05, 1.0633772484888027e-05, 8.340266158804084e-06, 8.481691238601049e-06, 8.508113756175209e-06, 9.642899547756387e-06, 7.79776288300161e-06, 8.377489209081329e-06, 8.496000064066595e-06, 8.282357223500334e-06, 8.904114553608247e-06, 8.678015692664476e-06, 7.857073141458311e-06, 9.102232104331718e-06, 8.238197665325407e-06, 5.796782545833328e-06, 5.90396758199777e-06, 5.2392734366296374e-06, 6.157965889637078e-06, 5.65742299366945e-06, 1.7096650012555883e-05, 1.2499156017942885e-05, 1.7050982004909986e-05, 2.723420193404868e-05, 1.9324975550979523e-05, 1.8124781135921318e-05, 2.322777345561351e-05, 2.5751748038604852e-05, 1.5282979372855143e-05, 1.5367275018241972e-05, 8.531632070769725e-06, 7.256899158979377e-06, 5.3185459190013716e-06, 5.107230115026806e-06, 6.1199045057680575e-06, 5.946884244723488e-06, 5.340452927766387e-06, 6.086571444666717e-06, 6.120979896946631e-06, 5.8590135539978655e-06, 6.842632895293947e-06, 6.430314921911861e-06, 6.45678019667134e-06, 8.975063328581055e-06, 6.358084966283024e-06, 5.713294577478176e-06, 4.6880124559263735e-06, 5.141806422654449e-06, 6.082379670984205e-06, 7.933889776845048e-06, 4.657456108340416e-06], "list_std22": [1.337092097029608e-05, 5.7590401361297505e-05, 7.3693287127160375e-06, 5.742973822558812e-06, 5.608227134791077e-06, 1.7813990529771514e-05, 5.135037964213921e-06, 1.2444517081820322e-05, 2.2776869852627933e-05, 7.759599952852653e-06, 2.9852651462333902e-05, 2.0339255408416614e-05, 1.2551638522677053e-05, 2.4926929643596853e-05, 1.9885776521635428e-05, 3.410304964896131e-05, 5.71897321212636e-05, 3.426033240937506e-05, 2.4807290496819686e-05, 4.60108910632203e-05, 0.0001147168684840272, 0.00011681310282691203, 5.960506639833404e-05, 0.00019459024768822856, 0.000376020777867341, 0.0002611853200654843, 0.0001784745207476698, 0.0003304723906428356, 0.0007672858983442368, 0.0016108407942892673, 0.0011093159838289706, 0.0018218058517233374, 0.0019487390860333208, 0.0018518065369684683, 0.002167425147594817, 0.00218865670462936, 0.002331548362561712, 0.004885067691742666, 0.0023780656457983744, 0.004993448549791993, 0.0037419742724640937, 0.01740388436093088, 0.005109974012015217, 0.011321694794552977, 0.016527283600346002, 0.015657956128360587, 0.009881688135045313, 0.0202981270589006, 0.01923718747517471, 0.03486163328423956, 0.05740349226732068, 0.13181853053576634, 0.08680692978938263, 0.13980470096483077, 0.07243328809245851, 0.2672131441593285, 0.3504133715979893, 0.5152814485709378, 0.7331237749148003], "list_std3": [2.85201578941858e-06, 1.4787921283104778e-06, 1.9492392841854753e-06, 1.6497735140988626e-06, 1.728805115831029e-06, 1.8530895420810853e-06, 1.5261827812532133e-06, 9.963260504688977e-07, 1.0174225154989386e-06, 1.4079026425339657e-06, 1.7236056901736752e-06, 2.4132153369083706e-06, 8.056661788423628e-06, 7.038918194329888e-06, 9.773294615767595e-06, 3.780348377251046e-06, 4.159560072612847e-06, 5.1777409941155936e-06, 3.977656825973482e-05, 3.3330512990423816e-05, 0.00011906005653649483, 0.0001973028535243463, 0.00019847894361661302, 0.00031636687176185693, 0.00024110144151369894, 0.000345670706828526, 0.0003752173776831255, 0.0004986234902586085, 0.0008399191940733953, 0.0011099826984275972, 0.0016575775176845798, 0.0017656166298755489, 0.002003994458813225, 0.0026046552948782603, 0.0020626208502136398, 0.0016878546788948283, 0.0017591615031473716, 0.0032830198696731697, 0.0021832363275726577, 0.004719306674601739, 0.007425027538514568, 0.004065840776947588, 0.006287835983456484, 0.02130138544560911, 0.008653573107741612, 0.01972827803892921, 0.035280766271217005, 0.05326430155810662, 0.07143766300624588, 0.03193165631138721, 0.11305308176986463, 0.036797272635546455, 0.04307922102027431, 0.1520340244419987, 0.160690913435092, 0.3768995333270277, 0.6075009166811647, 0.20601907542942396, 0.3345614845722476]}
\ No newline at end of file
{"program_submit": [0.0005864143371582031, 0.0006180882453918457, 0.0006022930145263671, 0.0006150364875793457, 0.0005805015563964844, 0.0006242513656616211, 0.0006157875061035157, 0.000590205192565918, 0.0006201028823852539, 0.0005742549896240234, 0.0006127119064331055, 0.0006307005882263183, 0.0006129026412963867, 0.0006038188934326172, 0.0006337285041809082, 0.0006203174591064454, 0.0006298184394836426, 0.0006552577018737793, 0.0006336092948913575, 0.000640261173248291, 0.0006558656692504883, 0.0006553530693054199, 0.0006669044494628906, 0.0006742477416992188, 0.0006883978843688965, 0.0006853103637695313, 0.0007182836532592773, 0.0007139921188354492, 0.0007268786430358887, 0.00074232816696167, 0.0007326364517211914, 0.0007406830787658692, 0.000748753547668457, 0.0007625460624694824, 0.0007710814476013184, 0.0008214235305786133, 0.0010093331336975097, 0.000996100902557373, 0.0009898781776428224, 0.0009746789932250976, 0.0009925484657287598, 0.0010025143623352052, 0.0009767651557922364, 0.000982213020324707, 0.0009743928909301758, 0.0009795427322387695, 0.000990152359008789, 0.0010040760040283202, 0.0009970307350158692, 0.000966334342956543, 0.0009711980819702148, 0.0009699821472167969, 0.0009940266609191895, 0.0009782910346984863, 0.0009680867195129394, 0.0009670615196228028, 0.0009890198707580566, 0.0009806632995605468, 0.0009762167930603028], "program_await": [0.0002083420753479004, 0.00023077726364135743, 0.00022351741790771484, 0.000235748291015625, 0.0002241373062133789, 0.00026024580001831054, 0.0002580404281616211, 0.0002468228340148926, 0.00030573606491088865, 0.00026367902755737306, 0.0003645896911621094, 0.0004561305046081543, 0.0005619049072265625, 0.0006647229194641114, 0.000740349292755127, 0.0008083343505859375, 0.0008953571319580078, 0.001018226146697998, 0.0010584235191345216, 0.0019822120666503906, 0.002857804298400879, 0.003724324703216553, 0.004595208168029785, 0.005466914176940918, 0.006303060054779053, 0.007189738750457764, 0.008267056941986085, 0.009017181396484376, 0.01766505241394043, 0.026852726936340332, 0.03688168525695801, 0.04429854154586792, 0.053369665145874025, 0.06351903676986695, 0.07367483377456666, 0.09007608890533447, 0.12792519330978394, 0.24712406396865844, 0.3754499554634094, 0.4878512263298035, 0.6025496602058411, 0.7199360132217407, 0.8401648163795471, 0.9600584387779236, 1.0809653997421265, 1.2047353625297545, 2.520810270309448, 3.649949276447296, 4.867997932434082, 6.014769196510315, 7.203233075141907, 8.423432326316833, 9.61386067867279, 10.819083714485169, 12.070031058788299, 24.035050463676452, 36.01291927099228, 48.14714548587799, 60.3269364118576]}
\ No newline at end of file
{"program_submit": [0.0008168339729309082, 0.0008072733879089355, 0.0008378028869628906, 0.0008386373519897461, 0.0008530259132385254, 0.0008653044700622558, 0.0008473634719848633, 0.0008452773094177246, 0.000846707820892334, 0.0008440375328063965, 0.0008415579795837402, 0.0008356332778930664, 0.0008180499076843262, 0.0008130192756652832, 0.0008362531661987305, 0.0008186697959899902, 0.0008180856704711914, 0.0008198380470275879, 0.0008553028106689453, 0.0008712172508239746, 0.0008637070655822754, 0.0008467912673950195, 0.0008581876754760742, 0.0008783698081970215, 0.0008826136589050293, 0.0009027838706970215, 0.0008979439735412598, 0.0008819341659545898, 0.0009396076202392578, 0.0009815573692321777, 0.0010265231132507325, 0.0010771036148071289, 0.0011136770248413087, 0.001148223876953125, 0.0011995077133178712, 0.0012345314025878906, 0.0012560844421386718, 0.0016282916069030762, 0.0019329667091369628, 0.002244234085083008, 0.0026000380516052247, 0.002948904037475586, 0.0033601760864257813, 0.003850698471069336, 0.004303407669067383, 0.004692184925079346, 0.008991920948028564, 0.017369043827056885, 0.03890063762664795, 0.06319750547409057, 0.07593896389007568, 0.08898782730102539, 0.10046734809875488, 0.11443088054656983, 0.12515259981155397, 0.25728042125701905, 0.37703671455383303, 0.5122438073158264, 0.6399835348129272], "program_await": [0.00026254653930664064, 0.00026510953903198243, 0.0002815127372741699, 0.00029753446578979493, 0.0003885030746459961, 0.00035544633865356443, 0.00033828020095825193, 0.000355684757232666, 0.0003590941429138184, 0.0003804206848144531, 0.000489497184753418, 0.0005999565124511719, 0.0007127881050109863, 0.0008299827575683593, 0.000968027114868164, 0.0011014819145202636, 0.0012084126472473144, 0.0013505220413208008, 0.0015236258506774903, 0.0028190374374389648, 0.0040325760841369625, 0.005167067050933838, 0.006555604934692383, 0.007889032363891602, 0.00914984941482544, 0.01051783561706543, 0.011926019191741943, 0.013154399394989014, 0.02553403377532959, 0.03819231986999512, 0.051269245147705075, 0.06243621110916138, 0.07663260698318482, 0.09034155607223511, 0.1023435115814209, 0.11627074480056762, 0.1267919659614563, 0.256557035446167, 0.3820217490196228, 0.514660918712616, 0.6644879460334778, 0.7572667121887207, 0.8842404961585999, 1.0570499777793885, 1.1396759033203125, 1.2612756729125976, 2.560833418369293, 3.7640287399291994, 5.0865274310112, 6.315130150318145, 7.642990398406982, 8.870680010318756, 10.274284160137176, 11.464474785327912, 12.666851902008057, 25.484598696231842, 36.48329095840454, 51.19497195482254, 63.16609311103821]}
\ No newline at end of file
{"program_submit": [2.1195411682128908e-05, 3.7097930908203124e-05, 5.277395248413086e-05, 7.038116455078125e-05, 8.84413719177246e-05, 0.00010830163955688477, 0.00012546777725219727, 0.00014170408248901368, 0.00015937089920043946, 0.00017592906951904296, 0.0003482460975646973, 0.0005269527435302734, 0.0006978392601013183, 0.0008800983428955078, 0.0010640263557434083, 0.0012444257736206055, 0.0014308929443359376, 0.001610863208770752, 0.001801443099975586, 0.00356600284576416, 0.005280613899230957, 0.007319116592407226, 0.009225761890411377, 0.011001455783843993, 0.012552487850189208, 0.01458442211151123, 0.01655583381652832, 0.01901898384094238, 0.03751354217529297, 0.05417633056640625, 0.07244229316711426, 0.09043128490447998, 0.11075685024261475, 0.12897059917449952, 0.14475066661834718, 0.16557918787002562, 0.18186691999435425, 0.36661859750747683, 0.5415117383003235, 0.7346216559410095, 0.914909017086029, 1.078467571735382, 1.258569061756134, 1.467386209964752, 1.649903655052185, 1.7991217851638794, 3.598971092700958, 5.406276023387909, 7.237125194072723, 9.00406997203827, 11.053654670715332, 12.895877587795258, 14.699854159355164, 16.264334213733672, 18.072578847408295, 36.155164098739625, 55.11732175350189, 73.39884235858918, 92.98915385007858]}
\ No newline at end of file
{"list_std11": [7.04422865437628e-05, 7.161367056362364e-05, 3.5306004404812815e-05, 5.080476035794145e-05, 4.871161478924981e-05, 4.557857136370821e-05, 0.00010995913624856731, 5.605447124699341e-05, 4.3466986418396706e-05, 1.6487377169133622e-05, 4.300760785942461e-05, 6.536781592514163e-05, 5.6442935494180316e-05, 7.222889719995036e-05, 7.365310398876594e-05, 7.938555221645029e-05, 6.919343425170088e-05, 8.086870366610761e-05, 7.375575939296195e-05, 7.802727800021812e-05, 7.03140984132173e-05, 7.73919022131823e-05, 8.021100305792846e-05, 8.973158081074764e-05, 6.94936303571893e-05, 8.11396885037792e-05, 8.525126828227849e-05, 6.665248024668637e-05, 3.232825958379633e-05, 6.099933761230086e-05, 2.4411388477431176e-05, 2.5751562154854786e-05, 2.8255579713316772e-05, 3.6402984912984376e-05, 3.6770955121694444e-05, 8.27045765094283e-05, 3.480468381818223e-05, 3.649476784623755e-05, 2.2913041309826395e-05, 2.8416001510463477e-05, 1.7959686838597497e-05, 8.277570659428141e-05, 2.7789537303758788e-05, 2.4380632090619274e-05, 2.449812648955851e-05, 3.370785020640729e-05, 2.639852959314556e-05, 3.4586086065940294e-05, 3.02516567905474e-05, 2.5286204293011546e-05, 2.248471426523226e-05, 2.4368763098879457e-05, 0.0001042088272559804, 3.029332252604421e-05, 2.589824865935588e-05, 2.6813823697674808e-05, 2.581301656972495e-05, 2.032765014959465e-05, 2.5822704089906425e-05], "list_std12": [3.494270022593803e-05, 3.4212725064179734e-05, 2.7930558331097534e-05, 2.7326664787500824e-05, 3.189700320603197e-05, 3.428426836574949e-05, 4.232224743279221e-05, 2.0718225073361167e-05, 0.00010696513791911466, 2.518479058589506e-05, 3.3043636258407446e-05, 4.906737860407665e-05, 7.824468847141308e-05, 0.00019588259926354531, 7.503307889593429e-05, 8.867090976512027e-05, 6.134164475526889e-05, 9.856643103386767e-05, 5.873076013951839e-05, 0.00010317386132110133, 0.00013400634550814268, 0.00018118792388619284, 0.00015468751016411557, 0.00019963415028134801, 0.00017476092076841257, 0.00019629736275597595, 0.00037077198952836435, 0.00023426334495772077, 0.0004919279835475575, 0.0014989606235791641, 0.0007668131908960733, 0.0022208884801346498, 0.0011206545891906934, 0.0027800441252212997, 0.003267940862190568, 0.010822076026914994, 0.007663767335800518, 0.005198138729798226, 0.021718691857576796, 0.007778769010503861, 0.013384938034885175, 0.011098214862984146, 0.013270491313515308, 0.013599449068345258, 0.014649093994076582, 0.02008321255099332, 0.06409184161052969, 0.05754470741710934, 0.1239056632437306, 0.08104837474775077, 0.08626153207657244, 0.11540506903229726, 0.12391925778213383, 0.13399012888152245, 0.32653842307718345, 0.3142480897548745, 0.4370715772986597, 0.5981827526099386, 0.8779446232689969], "list_std21": [3.206208320435713e-05, 9.33139822481879e-06, 7.169268978474729e-06, 7.340610963752172e-06, 2.106597314995984e-05, 3.784566350032497e-05, 1.2513036992663368e-05, 1.322021997366764e-05, 1.226585571403438e-05, 6.290044911962907e-06, 5.3221447938105745e-06, 1.35044822154357e-05, 1.6940046470717587e-05, 8.290327335669404e-06, 0.00010046910921482921, 1.242088057934243e-05, 1.2165398760181363e-05, 1.338708281669605e-05, 1.6999833378461622e-05, 2.3439969744490584e-05, 3.532774782600094e-05, 2.480483315039507e-05, 2.278265183541169e-05, 3.806785760174918e-05, 3.632146529483185e-05, 5.1416055350968726e-05, 3.107123902380571e-05, 1.9233885595795143e-05, 7.624836551489086e-05, 7.788466704090147e-05, 6.142284892984283e-05, 7.867821095658772e-05, 8.159358176757291e-05, 9.110424155704398e-05, 0.00011226465226889294, 0.00010461196173277917, 9.618059244116133e-05, 0.0003212752289012523, 0.00038292553294091405, 0.0006491452644584473, 0.0008258275436117338, 0.0008867775784891569, 0.0007878148152223721, 0.0008889083793608708, 0.0012964653741681666, 0.0011409492497726913, 0.003404698877479286, 0.00933644050216292, 0.0018784954017782372, 0.0010109311249138309, 0.001315735013596468, 0.0012516415375536068, 0.0016898640891364002, 0.0019407894318360303, 0.0016900855606179414, 0.0023320011837298007, 0.003184046530142798, 0.0041910354907632496, 0.00355576615585183], "list_std22": [1.862905358660391e-05, 9.686917358773121e-06, 1.1026257285538202e-05, 1.1079472429742052e-05, 0.0001769504233719522, 5.163563900422557e-05, 2.7050324409008168e-05, 2.534651781669901e-05, 1.0544826042989205e-05, 4.8879781177180435e-05, 7.852359665588565e-06, 1.4062633319011543e-05, 1.4850862841247587e-05, 1.2448723507282873e-05, 1.4274463944284385e-05, 1.9583727874588312e-05, 1.2485409374169639e-05, 2.237440516280514e-05, 5.048814926750998e-05, 3.482216553844662e-05, 0.00011932997554966673, 3.647028107573634e-05, 0.00017743016265675044, 0.0004527657130467929, 0.00025993099624924175, 0.0005107029302632159, 0.00022698992539759878, 0.0002955307935900211, 0.0007035624176636325, 0.0007350730884730291, 0.001078920495059788, 0.0015322286871141321, 0.0015613220214008795, 0.005014836351707254, 0.0023646421397278774, 0.004517198315560036, 0.0025888943934735927, 0.006239358544966802, 0.005596525225399302, 0.0035405179532557373, 0.01236958798161121, 0.010443853189203437, 0.008465559168584031, 0.22642290210251895, 0.016040303846244986, 0.016198935677688146, 0.024336277632388447, 0.022636220425898287, 0.03333717847538949, 0.043907699991946485, 0.025625915530052664, 0.06616847598990507, 0.19754404610246054, 0.08388456167311632, 0.059333606212947074, 0.1090000406969891, 0.5673910838226139, 0.25947654701402384, 0.38288154087027504], "list_std3": [3.4305948625999526e-06, 2.3185323739167394e-06, 1.5951929418372505e-06, 3.6647788469927114e-06, 2.050188198402286e-06, 2.2054142493437256e-06, 2.392797227259908e-06, 1.6559277183067213e-06, 1.6990873251269912e-06, 1.6570565189110987e-06, 2.3920469184607998e-06, 1.8183761773534596e-06, 2.603110503232897e-06, 4.379448352865245e-06, 3.75961626386489e-06, 4.692158741182695e-06, 4.67804637745647e-06, 6.406075793391115e-06, 1.8686298546747332e-05, 9.992317879668134e-05, 3.7740634122708385e-05, 9.534221868010748e-05, 6.022607577267132e-05, 0.00011910349706650031, 0.00021611694708544043, 0.00027144324597066117, 0.0001958542061908271, 0.0003410707599372697, 0.001332100615478519, 0.0008945226592554534, 0.0012324893229831747, 0.0013989027440885512, 0.0018557509408553864, 0.0026042810426043553, 0.0018337568314215705, 0.0029086098318247228, 0.002445749092009873, 0.003505812377391009, 0.002850770110516273, 0.007822878690231782, 0.009715852125230122, 0.003815561036986978, 0.0040165331783564686, 0.014069319823690514, 0.018856367277125412, 0.004767562483293545, 0.008759757164908136, 0.03596959920586583, 0.01981856663393207, 0.029122301908332242, 0.1163078217118256, 0.13819743423474282, 0.18002337022671747, 0.07447927913832916, 0.04290156483101988, 0.10960063311435433, 0.6028973412340314, 0.8043494397734458, 0.9711835320050966]}
\ No newline at end of file
{"program_submit": [8.33272933959961e-05, 7.779598236083985e-05, 7.983446121215821e-05, 8.218288421630859e-05, 7.866621017456054e-05, 8.128881454467773e-05, 7.985830307006836e-05, 8.257627487182618e-05, 8.040666580200195e-05, 7.808208465576172e-05, 8.710622787475586e-05, 7.789134979248047e-05, 8.007287979125977e-05, 7.960796356201171e-05, 8.159875869750977e-05, 7.827281951904297e-05, 7.785558700561524e-05, 7.725954055786133e-05, 8.120536804199219e-05, 7.96198844909668e-05, 8.202791213989258e-05, 8.025169372558594e-05, 7.784366607666016e-05, 7.97271728515625e-05, 8.25643539428711e-05, 8.286237716674805e-05, 7.761716842651367e-05, 7.764101028442382e-05, 7.748603820800781e-05, 8.578300476074219e-05, 8.605718612670898e-05, 8.294582366943359e-05, 7.880926132202149e-05, 8.23974609375e-05, 8.673667907714844e-05, 8.46266746520996e-05, 7.817745208740234e-05, 8.730888366699218e-05, 9.791851043701172e-05, 8.14199447631836e-05, 0.00011287927627563476, 8.876323699951172e-05, 0.00011199712753295898, 0.00011467933654785156, 9.639263153076172e-05, 9.933710098266601e-05, 0.00011802911758422852, 0.00013628005981445313, 0.00012404918670654296, 0.00012940168380737305, 0.00013582706451416017, 0.00012658834457397462, 0.00013259649276733398, 0.00014171600341796874, 0.00013747215270996094, 0.0001508474349975586, 0.00015909671783447265, 0.00015106201171875, 0.00014963150024414063], "program_await": [0.00015438795089721679, 0.00012760162353515626, 0.00012319087982177735, 0.00012439489364624023, 0.00012115240097045898, 0.00012133121490478515, 0.00012568235397338867, 0.00012333393096923827, 0.0001203775405883789, 0.00012327432632446288, 0.00012019872665405273, 0.00012297630310058593, 0.00012685060501098632, 0.00012558698654174805, 0.00012453794479370118, 0.0001230955123901367, 0.00012125968933105469, 0.00012291669845581054, 0.0001269698143005371, 0.00012530088424682616, 0.0001246333122253418, 0.0001266956329345703, 0.00012238025665283202, 0.00012973546981811523, 0.00013257265090942382, 0.0001296401023864746, 0.00012267827987670897, 0.00012830495834350585, 0.0001357913017272949, 0.00015619993209838867, 0.00016062259674072267, 0.000157928466796875, 0.00015414953231811522, 0.00018504858016967773, 0.00018863677978515626, 0.00018187761306762696, 0.00017772912979125978, 0.00027849674224853513, 0.00040943622589111327, 0.0003589034080505371, 0.0006673336029052734, 0.0005460381507873535, 0.0007996559143066406, 0.0009391546249389648, 0.0008469581604003906, 0.0010288119316101073, 0.0020152926445007324, 0.003324639797210693, 0.003989171981811523, 0.005023157596588135, 0.006066656112670899, 0.006645715236663819, 0.007918250560760499, 0.009000635147094727, 0.009599137306213378, 0.018986928462982177, 0.02855468988418579, 0.03814445734024048, 0.0471208930015564]}
\ No newline at end of file
{"program_submit": [8.478164672851563e-05, 7.90715217590332e-05, 9.711980819702149e-05, 8.63790512084961e-05, 8.32676887512207e-05, 8.189678192138672e-05, 7.954835891723632e-05, 7.908344268798828e-05, 8.43644142150879e-05, 8.394718170166016e-05, 8.252859115600585e-05, 8.019208908081055e-05, 7.878541946411133e-05, 8.169412612915039e-05, 8.428096771240234e-05, 8.361339569091797e-05, 8.218288421630859e-05, 7.936954498291015e-05, 7.851123809814453e-05, 7.829666137695312e-05, 7.838010787963867e-05, 9.450912475585937e-05, 8.449554443359374e-05, 8.287429809570313e-05, 8.242130279541015e-05, 7.843971252441406e-05, 7.903575897216797e-05, 7.880926132202149e-05, 7.969141006469727e-05, 8.119344711303711e-05, 8.077621459960938e-05, 8.374452590942383e-05, 8.251667022705079e-05, 7.895231246948242e-05, 7.917881011962891e-05, 8.102655410766602e-05, 8.869171142578125e-05, 8.907318115234376e-05, 8.263587951660157e-05, 0.00010046958923339843, 9.851455688476562e-05, 9.342432022094727e-05, 0.0001170039176940918, 0.00015245676040649415, 0.00011618137359619141, 9.926557540893555e-05, 0.00014294385910034179, 0.0001434922218322754, 0.00014983415603637696, 0.00013911724090576172, 0.00015734434127807618, 0.0001873016357421875, 0.0001980900764465332, 0.00022100210189819337, 0.00023071765899658204, 0.00023615360260009766, 0.00023317337036132812, 0.0002328634262084961, 0.0002288818359375], "program_await": [4.061460494995117e-05, 3.691911697387695e-05, 3.6668777465820315e-05, 3.648996353149414e-05, 3.8003921508789064e-05, 3.365278244018555e-05, 3.3867359161376956e-05, 3.923177719116211e-05, 3.5810470581054685e-05, 3.616809844970703e-05, 3.601312637329102e-05, 3.4701824188232425e-05, 3.9184093475341796e-05, 3.747940063476562e-05, 3.622770309448242e-05, 3.6883354187011716e-05, 3.938674926757812e-05, 3.495216369628906e-05, 3.467798233032226e-05, 3.567934036254883e-05, 3.987550735473633e-05, 4.224777221679687e-05, 4.222393035888672e-05, 3.982782363891602e-05, 3.9577484130859375e-05, 4.0531158447265625e-05, 3.966093063354492e-05, 4.025697708129883e-05, 5.697011947631836e-05, 5.637407302856445e-05, 6.346702575683593e-05, 6.918907165527343e-05, 7.338523864746094e-05, 7.642507553100586e-05, 8.096694946289063e-05, 9.897947311401367e-05, 0.00011539459228515625, 0.00017353296279907227, 0.00021975040435791017, 0.0004099726676940918, 0.0004041910171508789, 0.00047272443771362305, 0.0006350517272949219, 0.0009477734565734863, 0.0008053779602050781, 0.0008129358291625976, 0.002022421360015869, 0.002868211269378662, 0.0038693904876708984, 0.004733514785766601, 0.005929577350616455, 0.007387387752532959, 0.008768999576568603, 0.010199058055877685, 0.012371456623077393, 0.022912359237670897, 0.03525984287261963, 0.04820419549942016, 0.060582423210144044]}
\ No newline at end of file
{"program_submit": [5.877017974853515e-06, 5.292892456054687e-06, 5.292892456054687e-06, 5.185604095458984e-06, 5.221366882324218e-06, 5.233287811279297e-06, 5.161762237548828e-06, 5.233287811279297e-06, 5.614757537841797e-06, 5.674362182617188e-06, 5.877017974853515e-06, 5.674362182617188e-06, 6.115436553955078e-06, 6.008148193359375e-06, 6.556510925292969e-06, 6.556510925292969e-06, 6.8068504333496095e-06, 6.663799285888672e-06, 6.711483001708984e-06, 8.33272933959961e-06, 9.644031524658204e-06, 1.1157989501953124e-05, 1.2564659118652344e-05, 1.4173984527587891e-05, 1.5366077423095704e-05, 1.697540283203125e-05, 1.8262863159179686e-05, 1.888275146484375e-05, 3.2401084899902345e-05, 4.476308822631836e-05, 5.804300308227539e-05, 7.03573226928711e-05, 8.34822654724121e-05, 9.546279907226562e-05, 0.00010781288146972656, 0.00012068748474121093, 0.00013347864151000977, 0.00026468038558959963, 0.00038840770721435545, 0.0005779623985290528, 0.0006683111190795898, 0.0007733464241027832, 0.0009044289588928222, 0.0010488629341125488, 0.0011983156204223634, 0.0013417601585388184, 0.002810215950012207, 0.0042204618453979496, 0.005715203285217285, 0.007102298736572266, 0.008449256420135498, 0.009972894191741943, 0.011492395401000976, 0.01275632381439209, 0.014182054996490478, 0.0284468412399292, 0.04277374744415283, 0.05641505718231201, 0.07078934907913208]}
\ No newline at end of file
{"list_std11": [2.0565698278253843e-05, 5.737240593793565e-06, 7.1952096046708135e-06, 1.6064176074243467e-05, 6.400001667155949e-06, 8.648169218883378e-06, 6.376116524427938e-06, 1.8767881774932105e-05, 1.2709692420634926e-05, 5.271664661675693e-06, 4.085393608425274e-05, 5.610160590156102e-06, 7.758983039024585e-06, 6.598595342616598e-06, 1.358443682136143e-05, 6.540852424703836e-06, 5.647565131164381e-06, 5.243197767438763e-06, 9.029938544896522e-06, 6.4475065065985586e-06, 1.987382366252558e-05, 1.2721344863847649e-05, 6.131674643854922e-06, 6.871559660530461e-06, 1.877837190427744e-05, 1.835773761345504e-05, 6.37986912681456e-06, 5.184554687905996e-06, 5.1895870263898024e-06, 1.2542577722641681e-05, 1.906022360491226e-05, 1.8926559652337373e-05, 5.704490497005364e-06, 7.128843515513038e-06, 2.0830945709226664e-05, 2.0306132548308394e-05, 6.514636845215106e-06, 1.8281492702275216e-05, 2.5359005721848812e-05, 7.602879287721655e-06, 2.2472599446518998e-05, 1.6904656866067896e-05, 2.8553378670556404e-05, 3.210856758875756e-05, 2.3703815472638112e-05, 3.147793230093655e-05, 4.0318346321840965e-05, 3.584354050517008e-05, 3.415974654830184e-05, 3.391811090832941e-05, 3.129853342210333e-05, 3.360571311587328e-05, 3.356204453863262e-05, 2.982298486084116e-05, 3.170286776613858e-05, 1.5751607740244874e-05, 2.1768309649833236e-05, 1.0367464646490753e-05, 9.244727025980327e-06], "list_std12": [3.20933273104813e-05, 2.111439198649395e-05, 8.029880814386864e-06, 1.2101728517729146e-05, 7.803966259507106e-06, 9.321269401557455e-06, 1.242940428562506e-05, 1.3974214985138338e-05, 8.936856097450892e-06, 1.9603577601464845e-05, 7.57756438669966e-06, 9.907734312476193e-06, 9.78616595797187e-06, 9.095129757028807e-06, 1.0876386894351313e-05, 7.173899548573328e-06, 6.9572313952925446e-06, 7.470683533432993e-06, 1.1500804432011748e-05, 8.285995722859714e-06, 1.5713532274955828e-05, 1.2531314132585447e-05, 8.42656536114145e-06, 1.0756900678024963e-05, 1.6714642401055195e-05, 1.8820985304906576e-05, 7.429723658634622e-06, 1.8228832150589002e-05, 9.287048367335332e-06, 1.8406368609684493e-05, 1.6574915790583975e-05, 1.944804708631118e-05, 8.300065199290155e-06, 1.815779286960902e-05, 1.7287462766175937e-05, 2.4571579579221536e-05, 9.757999821771051e-06, 5.767317442318648e-05, 7.368841933319577e-05, 2.3430778227426563e-05, 9.746549426373349e-05, 8.147675076657531e-05, 0.00018749362926547802, 0.00026037900999539653, 0.000160709041866216, 0.00023353919582655766, 0.00041890866657438353, 0.0007463435917269717, 0.0006908985240614365, 0.0008522935814251791, 0.0013079496387746546, 0.0011254657772981672, 0.0010825683444676403, 0.0015965875299335172, 0.0014144123206697816, 0.0015090789194414458, 0.0012466636828725541, 0.0019211475473493958, 0.0020427466572132155], "list_std21": [2.2948545200955208e-05, 5.96185718126744e-06, 7.660288262283981e-05, 2.418263236444692e-05, 1.9344158980480127e-05, 9.045860784335389e-06, 6.9739284439360606e-06, 6.604577422532732e-06, 2.1290444675934337e-05, 1.9334726314736658e-05, 1.9888089512424633e-05, 7.809331509698691e-06, 6.230403034813453e-06, 1.1339112637801998e-05, 1.7687262410556943e-05, 1.690748828312158e-05, 1.3878617331747033e-05, 7.146238580982824e-06, 6.767454720961514e-06, 6.191349471045791e-06, 6.412330793779608e-06, 5.7366742569862966e-05, 2.0061698569536808e-05, 1.761402207190056e-05, 1.9838510024462214e-05, 6.435674792694016e-06, 6.5465216765975675e-06, 6.592505727678549e-06, 5.7595029823666845e-06, 8.624960008459684e-06, 8.125621529338522e-06, 2.1610756151062295e-05, 2.0714618312535356e-05, 7.184390704367473e-06, 6.543047556534584e-06, 6.675843456392725e-06, 2.829636780644809e-05, 2.7005729779234842e-05, 6.813407686626289e-06, 2.6911733525616245e-05, 3.167749129933424e-05, 1.878461220478573e-05, 4.3563787160164216e-05, 3.135176641904345e-05, 3.9662291145889794e-05, 3.777413328868567e-05, 4.647894253952687e-05, 4.706580158111701e-05, 4.918425774970318e-05, 4.3854933502754464e-05, 4.715725968260216e-05, 3.867670902016301e-05, 3.914830738003106e-05, 3.8998940040035344e-05, 3.0007967922504336e-05, 1.1467726253007745e-05, 1.1332442014014558e-05, 1.0122960227294054e-05, 9.539879734831096e-06], "list_std22": [8.69949299001588e-06, 9.454597691359077e-06, 5.7234051411760075e-06, 9.994283585432473e-06, 5.7649675739305545e-06, 5.081390032830048e-06, 4.665286306109227e-06, 7.776776730822864e-06, 6.583025583720989e-06, 6.628993354203147e-06, 5.109104295736793e-06, 5.072078970688683e-06, 1.0580514142352957e-05, 5.434908816823496e-06, 4.929696527626702e-06, 5.552268178650946e-06, 1.6931801254973775e-05, 3.7795568967287415e-06, 3.908805834275136e-06, 4.255322579311601e-06, 6.579059176154273e-06, 1.0054309493869085e-05, 9.018169173596253e-06, 5.770193676996646e-06, 5.8859636353123e-06, 4.1620048003467505e-06, 3.787365518884619e-06, 3.967906341251578e-06, 2.4416973322233186e-05, 5.436491190169297e-06, 7.388182405205642e-06, 7.943913932309225e-06, 9.57043868847524e-06, 4.9882170240129374e-06, 5.100855682337575e-06, 1.2505473442862256e-05, 2.142139066800032e-05, 3.1125247651275834e-05, 3.116135101244768e-05, 0.00015083167129315245, 0.00011785592513941686, 0.00010249795043562428, 0.0001909637708856635, 0.00013635598634407803, 0.00022482394954148545, 0.00014463694099704942, 0.0004376617839266499, 0.0005312741443130617, 0.0007211767881126313, 0.0008369832818806947, 0.0008975407579328283, 0.0008003130873531564, 0.0009278564050433592, 0.000989931642974811, 0.002162257744532402, 0.001114857253635079, 0.003358529204210696, 0.007121526422731294, 0.011741635573660758], "list_std3": [3.91094833765838e-06, 1.477020846399963e-06, 1.3163687734022031e-06, 1.3856266690548274e-06, 1.2566026980615228e-06, 1.1970392304036963e-06, 1.155841654887623e-06, 1.204513790892383e-06, 2.5070970931549562e-06, 2.4976820518291174e-06, 2.8392681997432316e-06, 1.8233053709379145e-06, 2.9569519672126677e-06, 1.9223476734789686e-06, 2.50354446118417e-06, 2.074883379610612e-06, 2.0618292342157214e-06, 1.528533317636086e-06, 1.2455426813509633e-06, 1.1995359256397016e-06, 1.2460229831836616e-06, 1.2841563373440612e-06, 1.105095460963056e-06, 1.2266642296991204e-06, 1.106110210153901e-06, 1.1305217365013187e-06, 1.1561651572743453e-06, 1.146289886263893e-06, 1.6767990345860797e-06, 1.2030225940459498e-06, 1.676040574489326e-06, 9.689241140923037e-07, 2.203242697200795e-06, 2.0088017292256317e-06, 1.2463230778348624e-06, 1.6637233311559376e-06, 2.729674906841717e-06, 7.031603859770643e-06, 1.3943523396384086e-05, 5.061815690348768e-05, 5.495866847283708e-05, 2.0680993593433978e-05, 2.7715931115882667e-05, 3.131494871687145e-05, 2.7354819822691195e-05, 2.7316511247279863e-05, 1.4872126149092265e-05, 2.0138130148360997e-05, 0.00018337714966778723, 0.00019089393507613951, 5.7806854042737764e-05, 0.00022551584342902795, 0.0005931221874523529, 0.0002799002042159219, 0.0003151653201785948, 0.0012958317398656704, 0.0017238694717453683, 0.00031805899986982606, 0.000699313158818093]}
\ No newline at end of file
{"program_submit": [0.0005888581275939941, 0.0005722641944885254, 0.0005604743957519532, 0.000582277774810791, 0.0005719780921936036, 0.0005872249603271484, 0.0005797386169433594, 0.0005602121353149414, 0.0005748867988586426, 0.0005725622177124023, 0.0005780577659606934, 0.0005708694458007813, 0.0005584001541137695, 0.000574195384979248, 0.0005750894546508789, 0.0005781650543212891, 0.0005748987197875977, 0.000560593605041504, 0.000593876838684082, 0.0005714893341064453, 0.0005774378776550293, 0.0005717992782592774, 0.0005587935447692871, 0.0005742311477661133, 0.0005723476409912109, 0.0005765795707702637, 0.0005724906921386718, 0.0005573391914367676, 0.0005741477012634278, 0.0005856513977050781, 0.0005754590034484863, 0.0005742073059082031, 0.0006002545356750488, 0.0005708932876586914, 0.0005615949630737305, 0.0005834698677062988, 0.0005803465843200684, 0.0005784988403320313, 0.000560593605041504, 0.0005989313125610351, 0.0005883336067199707, 0.0005884528160095215, 0.0006085038185119629, 0.0006111383438110352, 0.0005985975265502929, 0.0006079316139221191, 0.0005988717079162597, 0.0006116271018981933, 0.0006399750709533691, 0.0006447553634643555, 0.000679636001586914, 0.0006562590599060059, 0.0006800651550292969, 0.0006809234619140625, 0.0007025718688964844, 0.0007248520851135254, 0.0007344603538513184, 0.0007240891456604004, 0.0007078289985656738], "program_await": [0.0002025604248046875, 0.00018271207809448243, 0.00018945932388305664, 0.0002100229263305664, 0.00017855167388916015, 0.00019589662551879882, 0.0001786351203918457, 0.0001834869384765625, 0.00018627643585205077, 0.00018209218978881836, 0.00019196271896362304, 0.000182342529296875, 0.00018019676208496093, 0.00018705129623413085, 0.00019011497497558593, 0.00019028186798095704, 0.00018824338912963866, 0.00017793178558349608, 0.00020219087600708007, 0.0001822829246520996, 0.00019042491912841798, 0.00018401145935058593, 0.00017833709716796875, 0.00019652843475341796, 0.00019055604934692383, 0.0001990199089050293, 0.00018573999404907225, 0.00018758773803710936, 0.00020097494125366212, 0.00021647214889526368, 0.00023205280303955077, 0.00021768808364868163, 0.0002443432807922363, 0.0002327561378479004, 0.00023746490478515625, 0.0002579212188720703, 0.0002724409103393555, 0.00032939910888671873, 0.0003746628761291504, 0.0005151748657226563, 0.0005784511566162109, 0.0007094979286193847, 0.0007637143135070801, 0.000876462459564209, 0.0009097456932067871, 0.0010488629341125488, 0.0018991827964782715, 0.0027752280235290526, 0.0036825060844421385, 0.0045209527015686035, 0.005354559421539307, 0.006316614151000976, 0.00707014799118042, 0.007961225509643555, 0.009019851684570312, 0.01739950180053711, 0.025775718688964843, 0.034304702281951906, 0.042807137966156004]}
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment