Using existing output directory "/tmp/dsc" Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 16384, extent_count: 1600 }) start ds: 8810 start ds: 8820 start ds: 8830 start access at:127.0.0.1:9998 Starting downstairs at port 8810 Starting downstairs at port 8820 Make output file at "/tmp/dsc/downstairs-8810.txt" Starting downstairs at port 8830 [8810][0] initial start wait reports Starting [8820][1] initial start wait reports Starting Make output file at "/tmp/dsc/downstairs-8830.txt" [8830][2] initial start wait reports Starting Make output file at "/tmp/dsc/downstairs-8820.txt" Downstairs /var/tmp/dsc/8810 port 8810 PID:Some(1489) [8810][0] initial start wait reports Running Downstairs /var/tmp/dsc/8830 port 8830 PID:Some(1490) [8830][2] initial start wait reports Running Downstairs /var/tmp/dsc/8820 port 8820 PID:Some(1491) [8820][1] initial start wait reports Running All downstairs are running Sep 27 17:14:28.731 INFO listening, local_addr: 127.0.0.1:9998 Control access at:127.0.0.1:9998 Sep 27 17:14:33.769 INFO accepted connection, remote_addr: 127.0.0.1:33567, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc DisableRestartAll disable restart on all: 3 Sep 27 17:14:33.770 INFO request completed, latency_us: 122, response_code: [8810] Disable keep_running 204[8820] Disable keep_running , uri: /disablerestart/all, method: POST, req_id: c5abad09-6e16-4e5c-902f-1bb32eef9c0a, remote_addr: [8830] Disable keep_running 127.0.0.1:33567, local_addr: 127.0.0.1:9998 Sep 27 17:16:19.955 INFO accepted connection, remote_addr: 127.0.0.1:39516, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc Shutdown Shutdown Shut it down Sep 27 17:16:19.955 INFO request completed, latency_us: 61, response_code: [8810] Disable keep_running 204[8810] Got stop action so:false kr:false , [8820] Disable keep_running uri[8820] Got stop action so:false kr:false :[8830] Disable keep_running [8830] Got stop action so:false kr:false /shutdown, method: POST, req_id: 0ff69948-17ad-4454-9643-949be3d890e3, remote_addr: 127.0.0.1:39516, local_addr: 127.0.0.1:9998 [8830] Exited with: ExitStatus(unix_wait_status(9)) [8820] Exited with: ExitStatus(unix_wait_status(9)) [8830][2] reports Exit [8820][1] reports Exit [8810] Exited with: ExitStatus(unix_wait_status(9)) [8810][0] reports Exit 4} {"msg":"Crucible Version: BuildInfo {\n version: \"0.0.1\",\n git_sha: \"93b12f70ee7c1993c73ac4256d99ff80ccbe4bac\",\n git_commit_timestamp: \"2024-09-27T16:52:31.000000000Z\",\n git_branch: \"update-omicron-common\",\n rustc_semver: \"1.80.0\",\n rustc_channel: \"stable\",\n rustc_host_triple: \"x86_64-unknown-illumos\",\n rustc_commit_sha: \"051478957371ee0084a7c0913941d2a8c4757bb9\",\n cargo_triple: \"x86_64-unknown-illumos\",\n debug: false,\n opt_level: 3,\n}","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.82221532Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494} {"msg":"Upstairs <-> Downstairs Message Version: 11","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.822221107Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494} {"msg":"Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 16384, shift: 12 }, extent_count: 1600, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.822226185Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494} {"msg":"Crucible c948047a-9419-41b9-b6d8-e1ec6cd8872f has session id: 8469b424-892f-4edf-a93a-e4635a8f861d","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.822492371Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Upstairs opts: Upstairs UUID: c948047a-9419-41b9-b6d8-e1ec6cd8872f, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.822514154Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Crucible stats registered with UUID: c948047a-9419-41b9-b6d8-e1ec6cd8872f","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.822526529Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"The guest has requested activation with gen:1","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852771824Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494} {"msg":"c948047a-9419-41b9-b6d8-e1ec6cd8872f active request set","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852785619Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852793536Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852808151Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852817697Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852827034Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852836401Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852846427Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"connecting to 127.0.0.1:8830","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852856384Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"io task","client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"connecting to 127.0.0.1:8810","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.85286648Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"io task","client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"connecting to 127.0.0.1:8820","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852876356Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"io task","client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_connection connected from Ok(127.0.0.1:34246)","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852944702Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"io task","client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_connection connected from Ok(127.0.0.1:33566)","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852958278Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"io task","client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_connection connected from Ok(127.0.0.1:60886)","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.852966235Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"io task","client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853255672Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[0] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853266298Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853310363Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[1] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853354957Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853367423Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[2] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.85337358Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853464068Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853485531Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.853617494Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859643081Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[0] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859652998Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Waiting for more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859657716Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859910906Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[1] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859917644Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Waiting for more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859921873Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859975654Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[2] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.85998512Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859990149Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859994857Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.859999225Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860003394Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860007462Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860011641Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860015829Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860019908Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"[2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860023997Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Max found gen is 1","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860028085Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Generation requested: 1 >= found:1","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860032174Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Next flush: 1","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860036412Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"All extents match","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860040891Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"":"downstairs","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860046658Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860050377Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"0","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860054536Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"1","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860058534Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"client":"2","":"downstairs","session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"All required reconciliation work is completed","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860062623Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Set Downstairs and Upstairs active after reconciliation","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860066242Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"c948047a-9419-41b9-b6d8-e1ec6cd8872f is now active with session: 8469b424-892f-4edf-a93a-e4635a8f861d","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.86006993Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"Set Active after no reconciliation","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860073589Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494,"session_id":"8469b424-892f-4edf-a93a-e4635a8f861d"} {"msg":"The guest has finished waiting for activation with:1","v":0,"name":"crucible","level":30,"time":"2024-09-27T17:14:33.860077278Z","hostname":"w-01J8T7H3J9GMYD6D2QRBX8F5TF","pid":1494} Wait for a query_work_queue command to finish before sending IO Region: es:16384 ec:1600 bs:4096 ts:107374182400 tb:26214400 max_io:256 or 1048576 Fill test CLIENT: Tests done. All submitted work has been ACK'd ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:4 States: Active Active Active GW_ID ACK DSID TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY 102598 Acked 103597 Write 256 Done Done Done false 102599 Acked 103598 Write 256 Done Done Done false 102600 Acked 103599 Write 256 Done Done Done false 102601 Acked 103600 Flush 0 Done Done Sent false STATES DS:0 DS:1 DS:2 TOTAL Sent 0 0 1 1 Done 4 4 3 11 Skipped 0 0 0 0 Error 0 0 0 0 Last Flush: 103600 103600 103596 Downstairs last five completed: 103596 103595 103594 103593 103592 Upstairs last five completed: 102601 102597 102600 102599 102598 CLIENT: Up:0 ds:4 act:3 ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0 Downstairs last five completed: 103600 103599 103598 103597 103596 Upstairs last five completed: 102601 102597 102600 102599 102598 CLIENT: Up:0 ds:0 act:3 CLIENT: All crucible jobs finished, exiting program