Job Failed #936989 - Old Simulation trying to send data to server!
Job #936989 failed for ffa8b5ef:
Seems that a simulation from a previous testcase is zombieing around and sends a data chunk to the server. The Server quits with
[ FTI Information ] : Reading FTI configuration file (config.fti)...
[ FTI Information ] : The execution ID is: 2021-01-21_12-12-23_60002800ffff19009e7f9e7f00000000
[ FTI Information ] : Selected Ckpt I/O is HDF5
[ FTI Information ] : FTI has been initialized.
[ FTI Information ] : Variable ID 0 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
Current 0MQ version is 4.2.1 (/home/john/builds/melissa/melissa-da/server/server.cxx:1473)
**server rank = 0 (/home/john/builds/melissa/melissa-da/server/server.cxx:1474)
[0] Start server with 10 ensemble members
[0] Configuration socket listening on port tcp://*:4000
[0] Establishing connection to launcher on 127.0.0.1:3000-3002
Successful connected to launcher (/home/john/builds/melissa/melissa-da/server/LauncherConnection.cxx:56)
[0] Init timing at 1611231143676 ms since epoch
[0] Null time at 1611231143291 ms since epoch (timing-events csv are relative to this)
Server registering Runner ID 0 (/home/john/builds/melissa/melissa-da/server/server.cxx:408)
Server registering Field variableX, runner_comm_size = 2 (/home/john/builds/melissa/melissa-da/server/server.cxx:442)
indexmapsize: 40 (/home/john/builds/melissa/melissa-da/server/server.cxx:488)
hiddenindexmapsize: 0 (/home/john/builds/melissa/melissa-da/server/server.cxx:489)
local_vect_sizes (/home/john/builds/melissa/melissa-da/server/server.cxx:582)
[160, 160, ]
[0, 0, ]
Calculated parts (/home/john/builds/melissa/melissa-da/server/Field.cxx:64)
rank 0 index map: (/home/john/builds/melissa/melissa-da/server/server.cxx:600)
[0] Chosing Dummy Assimilator
[ FTI Information ] : Variable ID 1 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 2 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 3 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 4 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 5 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 6 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 7 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 8 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 9 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
[ FTI Information ] : Variable ID 10 to protect (Stored in CPU). Current ckpt. size per rank is 0.00MB.
Change Phase (/home/john/builds/melissa/melissa-da/server/server.cxx:1637)
[0] Cycles last second: 1
[0] Error: a different runner (id=1) that never registered connected. Probably this is some zombie runner after a server restart
[0, 0, ]
-------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------