Using existing output directory "/tmp/dsc"
Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 16384, extent_count: 1600 })
start ds: 8810
start ds: 8820
start ds: 8830
start access at:127.0.0.1:9998
Starting downstairs at port 8810
Starting downstairs at port 8820
Make output file at "/tmp/dsc/downstairs-8810.txt"
[8810][0] initial start wait reports Starting
Starting downstairs at port 8830
Make output file at "/tmp/dsc/downstairs-8820.txt"
Make output file at "/tmp/dsc/downstairs-8830.txt"
[8820][1] initial start wait reports Starting
[8830][2] initial start wait reports Starting
Downstairs /var/tmp/dsc/8820 port 8820 PID:Some(1504)
[8820][1] initial start wait reports Running
Downstairs /var/tmp/dsc/8810 port 8810 PID:Some(1505)
[8810][0] initial start wait reports Running
Downstairs /var/tmp/dsc/8830 port 8830 PID:Some(1506)
[8830][2] initial start wait reports Running
All downstairs are running
Mar 07 18:16:59.585 INFO listening, local_addr: 127.0.0.1:9998
Control access at:127.0.0.1:9998
Mar 07 18:17:04.621 INFO accepted connection, remote_addr: 127.0.0.1:61201, local_addr: 127.0.0.1:9998
Main task has work to do, go find it
got dsc DisableRestartAll
disable restart on all: 3
Mar 07 18:17:04.621 INFO request completed, latency_us: 148, response_code: 204, uri: /disablerestart/all, method: POST, req_id: 5371b2f5-beba-43fa-a513-5cd258b351bf, remote_addr: 127.0[8810] Disable keep_running
.[8820] Disable keep_running
0.1:61201, local_addr: 127.0.0.1:9998[8830] Disable keep_running
Mar 07 18:18:36.875 INFO accepted connection, remote_addr: 127.0.0.1:49196, local_addr: 127.0.0.1:9998
Main task has work to do, go find it
got dsc Shutdown
Shutdown
Shut it down
Mar 07 18:18:36.876 INFO request completed, latency_us:[8810] Disable keep_running
[8810] Got stop action so:false kr:false
51[8830] Disable keep_running
, [8820] Disable keep_running
response_code[8820] Got stop action so:false kr:false
:[8830] Got stop action so:false kr:false
204, uri: /shutdown, method: POST, req_id: c2bcc5f6-3ada-4055-969e-460d7c99d528, remote_addr: 127.0.0.1:49196, local_addr: 127.0.0.1:9998
[8820] Exited with: ExitStatus(unix_wait_status(9))
[8820][1] reports Exit
[8810] Exited with: ExitStatus(unix_wait_status(9))
[8810][0] reports Exit
[8830] Exited with: ExitStatus(unix_wait_status(9))
[8830][2] reports Exit
9}
{"msg":"Crucible Version: BuildInfo {\n version: \"0.0.1\",\n git_sha: \"6c903058314cb23845d8ec9d1b70e497f207c60e\",\n git_commit_timestamp: \"2025-03-07T17:56:28.000000000Z\",\n git_branch: \"renovate/dropshot-0.x\",\n rustc_semver: \"1.84.0\",\n rustc_channel: \"stable\",\n rustc_host_triple: \"x86_64-unknown-illumos\",\n rustc_commit_sha: \"9fc6b43126469e3858e2fe86cafb4f0fd5068869\",\n cargo_triple: \"x86_64-unknown-illumos\",\n debug: false,\n opt_level: 3,\n}","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673356267Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509}
{"msg":"Upstairs <-> Downstairs Message Version: 13","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673361566Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509}
{"msg":"Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 16384, shift: 12 }, extent_count: 1600, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673366586Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509}
{"msg":"Crucible 92588d82-d196-4017-b9fc-c09ecbbcc194 has session id: 1d05a508-10cd-4b05-89d0-0108b98236dd","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673731027Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Upstairs opts: Upstairs UUID: 92588d82-d196-4017-b9fc-c09ecbbcc194, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673750326Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Crucible stats registered with UUID: 92588d82-d196-4017-b9fc-c09ecbbcc194","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673760215Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"could not find Downstairs address for Nexus","v":0,"name":"crucible","level":40,"time":"2025-03-07T18:17:04.673765085Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Downstairs in volume = 3","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673929812Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"task":"crutest"}
{"msg":"The guest has requested activation with gen:1","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673939311Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509}
{"msg":"92588d82-d196-4017-b9fc-c09ecbbcc194 active request set","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673979288Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.673988697Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"0","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting...","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674012025Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"0","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674016735Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"1","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting...","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674020985Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"1","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674025154Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"2","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting...","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674029394Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"2","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"connecting to 127.0.0.1:8830","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674033494Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"io task","client":"2","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"connecting to 127.0.0.1:8810","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674038013Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"io task","client":"0","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"connecting to 127.0.0.1:8820","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.67408294Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"io task","client":"1","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"ds_connection connected from Ok(127.0.0.1:45272)","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674141835Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"io task","client":"0","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"ds_connection connected from Ok(127.0.0.1:42709)","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674159914Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"io task","client":"2","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"ds_connection connected from Ok(127.0.0.1:65179)","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674227288Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"io task","client":"1","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"version negotiation from state Connecting { state: WaitForPromote, mode: New }","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674420264Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"0","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"version negotiation from state Connecting { state: WaitForPromote, mode: New }","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674563002Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"1","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"version negotiation from state Connecting { state: WaitForPromote, mode: New }","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674574751Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"2","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674696802Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"0","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674781885Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"2","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.674819002Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"1","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Waiting for more clients to be ready","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681296235Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Waiting for more clients to be ready","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681383648Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681559644Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681565894Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681570364Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681574643Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681582272Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681587152Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681591422Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681595471Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"[2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681599581Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Max found gen is 1","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681603691Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Generation requested: 1 >= found:1","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681607841Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Next flush: 1","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.68161197Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"All extents match","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.68161612Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"":"downstairs","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681620169Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681623819Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"0","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681628279Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"1","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681632568Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"client":"2","":"downstairs","session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"All required reconciliation work is completed","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681636938Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Set Downstairs and Upstairs active after reconciliation","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681640638Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"92588d82-d196-4017-b9fc-c09ecbbcc194 is now active with session: 1d05a508-10cd-4b05-89d0-0108b98236dd","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681644328Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"Set Active after no reconciliation","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681649527Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509,"session_id":"1d05a508-10cd-4b05-89d0-0108b98236dd"}
{"msg":"The guest has finished waiting for activation with:1","v":0,"name":"crucible","level":30,"time":"2025-03-07T18:17:04.681653327Z","hostname":"w-01JNRX3DK4TY4ZMB0CT374NE9H","pid":1509}
Wait for a query_work_queue command to finish before sending IO
Disk: sv:1 bs:4096 ts:107374182400 tb:26214400 max_io:256 or 1048576
Fill test
CLIENT: Tests done. All submitted work has been ACK'd
----------------------------------------------------------------
Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:60
States: Active Active Active
JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
104721 Acked Write 256 Done Done Done false
104722 Acked Write 256 Done Done Done false
104723 Acked Write 256 Done Done Done false
104724 Acked Write 256 Done Done Done false
104725 Acked Write 256 Done Done Done false
104726 Acked Write 256 Done Done Done false
104727 Acked Write 256 Done Done Done false
104728 Acked Write 256 Done Done Done false
104729 Acked Write 256 Done Done Done false
104730 Acked Write 256 Done Done Done false
104731 Acked Write 256 Done Done Done false
104732 Acked Write 256 Done Done Done false
104733 Acked Write 256 Done Done Done false
104734 Acked Write 256 Done Done Done false
104735 Acked Write 256 Done Done Done false
104736 Acked Write 256 Done Done Done false
104737 Acked Write 256 Done Done Done false
104738 Acked Write 256 Done Done Done false
104739 Acked Write 256 Done Done Done false
104740 Acked Write 256 Done Done Done false
104741 Acked Write 256 Done Done Done false
104742 Acked Write 256 Done Done Done false
104743 Acked Write 256 Done Done Done false
104744 Acked Write 256 Done Done Done false
104745 Acked Write 256 Done Done Done false
104746 Acked Write 256 Done Done Done false
104747 Acked Write 256 Done Done Done false
104748 Acked Write 256 Done Done Done false
104749 Acked Write 256 Done Done Done false
104750 Acked Write 256 Done Done Done false
104751 Acked Write 256 Done Done Done false
104752 Acked Write 256 Done Done Done false
104753 Acked Write 256 Done Done Done false
104754 Acked Write 256 Done Done Done false
104755 Acked Write 256 Done Done Done false
104756 Acked Write 256 Done Done Done false
104757 Acked Write 256 Done Done Done false
104758 Acked Write 256 Done Done Done false
104759 Acked Write 256 Done Done Done false
104760 Acked Write 256 Done Done Done false
104761 Acked Write 256 Done Done Done false
104762 Acked Write 256 Done Done Done false
104763 Acked Write 256 Done Done Done false
104764 Acked Write 256 Done Done Done false
104765 Acked Write 256 Done Done Done false
104766 Acked Write 256 Done Done Done false
104767 Acked Write 256 Done Done Done false
104768 Acked Write 256 Done Done Done false
104769 Acked Write 256 Done Done Done false
104770 Acked Write 256 Done Done Done false
104771 Acked Write 256 Done Done Done false
104772 Acked Write 256 Done Done Done false
104773 Acked Write 256 Done Done Done false
104774 Acked Write 256 Done Done Done false
104775 Acked Write 256 Done Done Done false
104776 Acked Write 256 Done Done Done false
104777 Acked Write 256 Done Done Done false
104778 Acked Write 256 Done Done Done false
104779 Acked Write 256 Done Done Done false
104780 Acked Flush 0 Done Done Sent false
STATES DS:0 DS:1 DS:2 TOTAL
Sent 0 0 1 1
Done 60 60 59 179
Skipped 0 0 0 0
Error 0 0 0 0
Last Flush: Some(JobId(104780)) Some(JobId(104780)) None
Downstairs last five completed: 104720 104719 104718 104717 104716
Upstairs last five completed: 104780 104720 104779 104778 104777
CLIENT: Up:0 ds:60 act:3
----------------------------------------------------------------
Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
Downstairs last five completed: 104780 104779 104778 104777 104776
Upstairs last five completed: 104780 104720 104779 104778 104777
CLIENT: Up:0 ds:0 act:3
CLIENT: All crucible jobs finished, exiting program