Using existing output directory "/tmp/dsc" Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 16384, extent_count: 1600 }) start ds: 8810 start ds: 8820 start ds: 8830 start access at:127.0.0.1:9998 Starting downstairs at port 8810 Make output file at "/tmp/dsc/downstairs-8810.txt" Starting downstairs at port 8820 [8810][0] initial start wait reports Starting Make output file at "/tmp/dsc/downstairs-8820.txt" [8820][1] initial start wait reports Starting Starting downstairs at port 8830 Make output file at "/tmp/dsc/downstairs-8830.txt" [8830][2] initial start wait reports Starting Downstairs /var/tmp/dsc/8810 port 8810 PID:Some(1488) [8810][0] initial start wait reports Running Downstairs /var/tmp/dsc/8830 port 8830 PID:Some(1489) [8830][2] initial start wait reports Running Downstairs /var/tmp/dsc/8820 port 8820 PID:Some(1490) [8820][1] initial start wait reports Running All downstairs are running Oct 31 21:04:53.596 INFO listening, local_addr: 127.0.0.1:9998 Control access at:127.0.0.1:9998 Oct 31 21:04:58.636 INFO accepted connection, remote_addr: 127.0.0.1:57909, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc DisableRestartAll disable restart on all: 3 Oct 31 21:04:58.636 INFO request completed, latency_us: 104, response_code: 204, uri: /disablerestart/all, method: POST, [8810] Disable keep_running req_id[8820] Disable keep_running : 6ce182dc-bccc-45e1-984e-6fb2e9a2c884, remote_addr: 127.[8830] Disable keep_running 0.0.1:57909, local_addr: 127.0.0.1:9998 Oct 31 21:06:54.063 INFO accepted connection, remote_addr: 127.0.0.1:44555, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc Shutdown Shutdown Shut it down Oct 31 21:06:54.063 INFO [8810] Disable keep_running request completed[8810] Got stop action so:false kr:false , latency_us[8830] Disable keep_running :[8830] Got stop action so:false kr:false [8820] Disable keep_running 66[8820] Got stop action so:false kr:false , response_code: 204, uri: /shutdown, method: POST, req_id: 00dab264-403f-4c44-a772-35d1e2b69d17, remote_addr: 127.0.0.1:44555, local_addr: 127.0.0.1:9998 [8820] Exited with: ExitStatus(unix_wait_status(9)) [8820][1] reports Exit [8810] Exited with: ExitStatus(unix_wait_status(9)) [8810][0] reports Exit [8830] Exited with: ExitStatus(unix_wait_status(9)) [8830][2] reports Exit } {"msg":"Crucible Version: BuildInfo {\n version: \"0.0.1\",\n git_sha: \"664b05998b747a86e13a12bb6c5e4428458d8bd9\",\n git_commit_timestamp: \"2024-10-31T20:43:19.000000000Z\",\n git_branch: \"alan/crutest-volume-info\",\n rustc_semver: \"1.80.0\",\n rustc_channel: \"stable\",\n rustc_host_triple: \"x86_64-unknown-illumos\",\n rustc_commit_sha: \"051478957371ee0084a7c0913941d2a8c4757bb9\",\n cargo_triple: \"x86_64-unknown-illumos\",\n debug: false,\n opt_level: 3,\n}","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.687406189Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493} {"msg":"Upstairs <-> Downstairs Message Version: 12","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.687415069Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493} {"msg":"Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 16384, shift: 12 }, extent_count: 1600, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.687424158Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493} {"msg":"Crucible a8f04090-8128-4c40-b34f-9c1caac43e03 has session id: 08101c95-ccad-4712-9432-f6aef01e260b","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.687717958Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Upstairs opts: Upstairs UUID: a8f04090-8128-4c40-b34f-9c1caac43e03, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.687736887Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Crucible stats registered with UUID: a8f04090-8128-4c40-b34f-9c1caac43e03","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.687764246Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Downstairs in volume = 3","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717556214Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"task":"crutest"} {"msg":"The guest has requested activation with gen:1","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717567743Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493} {"msg":"a8f04090-8128-4c40-b34f-9c1caac43e03 active request set","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.71761234Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717624139Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717635658Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717641668Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717646897Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717652117Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717657237Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"connecting to 127.0.0.1:8830","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717662406Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"io task","client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"connecting to 127.0.0.1:8810","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717667946Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"io task","client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"connecting to 127.0.0.1:8820","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717724032Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"io task","client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_connection connected from Ok(127.0.0.1:64847)","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717736011Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"io task","client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_connection connected from Ok(127.0.0.1:49345)","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717825465Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"io task","client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_connection connected from Ok(127.0.0.1:39042)","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.717838295Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"io task","client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718152604Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[0] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718161953Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718209Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[1] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718215469Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718224559Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[2] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718229718Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718429805Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718477152Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.718489071Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.725010835Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[2] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.725018895Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Waiting for more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.725024264Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726369394Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[0] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726379873Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Waiting for more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726384943Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726466888Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[1] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726474277Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726478697Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726483386Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726488076Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726492816Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726497096Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726501525Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726505735Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726509855Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"[2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726514015Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Max found gen is 1","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726518164Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Generation requested: 1 >= found:1","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726522344Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Next flush: 1","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726527264Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"All extents match","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726531473Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"":"downstairs","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726538093Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726541803Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"0","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726545992Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"1","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726550112Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"client":"2","":"downstairs","session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"All required reconciliation work is completed","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726554252Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Set Downstairs and Upstairs active after reconciliation","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726557952Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"a8f04090-8128-4c40-b34f-9c1caac43e03 is now active with session: 08101c95-ccad-4712-9432-f6aef01e260b","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726561751Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"Set Active after no reconciliation","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726565401Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493,"session_id":"08101c95-ccad-4712-9432-f6aef01e260b"} {"msg":"The guest has finished waiting for activation with:1","v":0,"name":"crucible","level":30,"time":"2024-10-31T21:04:58.726569081Z","hostname":"w-01JBJ67K05501G4X605W83F733","pid":1493} Wait for a query_work_queue command to finish before sending IO Region: sv:1 bs:4096 ts:107374182400 tb:26214400 max_io:256 or 1048576 Fill test CLIENT: Tests done. All submitted work has been ACK'd ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:141 States: Active Active Active JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY 103478 Acked Write 256 Done Done Done false 103479 Acked Write 256 Done Done Done false 103480 Acked Write 256 Done Done Done false 103481 Acked Write 256 Done Done Done false 103482 Acked Write 256 Done Done Done false 103483 Acked Write 256 Done Done Done false 103484 Acked Write 256 Done Done Done false 103485 Acked Write 256 Done Done Done false 103486 Acked Write 256 Done Done Done false 103487 Acked Write 256 Done Done Done false 103488 Acked Write 256 Done Done Done false 103489 Acked Write 256 Done Done Done false 103490 Acked Write 256 Done Done Done false 103491 Acked Write 256 Done Done Done false 103492 Acked Write 256 Done Done Done false 103493 Acked Write 256 Done Done Done false 103494 Acked Write 256 Done Done Done false 103495 Acked Write 256 Done Done Done false 103496 Acked Write 256 Done Done Done false 103497 Acked Write 256 Done Done Done false 103498 Acked Write 256 Done Done Done false 103499 Acked Write 256 Done Done Done false 103500 Acked Write 256 Done Done Done false 103501 Acked Write 256 Done Done Done false 103502 Acked Write 256 Done Done Done false 103503 Acked Write 256 Done Done Done false 103504 Acked Write 256 Done Done Done false 103505 Acked Write 256 Done Done Done false 103506 Acked Write 256 Done Done Done false 103507 Acked Write 256 Done Done Done false 103508 Acked Write 256 Done Done Done false 103509 Acked Write 256 Done Done Done false 103510 Acked Write 256 Done Done Done false 103511 Acked Write 256 Done Done Done false 103512 Acked Write 256 Done Done Done false 103513 Acked Write 256 Done Done Done false 103514 Acked Write 256 Done Done Done false 103515 Acked Write 256 Done Done Done false 103516 Acked Write 256 Done Done Done false 103517 Acked Write 256 Done Done Done false 103518 Acked Write 256 Done Done Done false 103519 Acked Write 256 Done Done Done false 103520 Acked Write 256 Done Done Done false 103521 Acked Write 256 Done Done Done false 103522 Acked Write 256 Done Done Done false 103523 Acked Write 256 Done Done Done false 103524 Acked Write 256 Done Done Done false 103525 Acked Write 256 Done Done Done false 103526 Acked Write 256 Done Done Done false 103527 Acked Write 256 Done Done Done false 103528 Acked Write 256 Done Done Done false 103529 Acked Write 256 Done Done Done false 103530 Acked Write 256 Done Done Done false 103531 Acked Write 256 Done Done Done false 103532 Acked Write 256 Done Done Done false 103533 Acked Write 256 Done Done Done false 103534 Acked Write 256 Done Done Done false 103535 Acked Write 256 Done Done Done false 103536 Acked Write 256 Done Done Done false 103537 Acked Write 256 Done Done Done false 103538 Acked Write 256 Done Done Done false 103539 Acked Write 256 Done Done Done false 103540 Acked Write 256 Done Done Done false 103541 Acked Write 256 Done Done Done false 103542 Acked Write 256 Done Done Done false 103543 Acked Write 256 Done Done Done false 103544 Acked Write 256 Done Done Done false 103545 Acked Write 256 Done Done Done false 103546 Acked Write 256 Done Done Done false 103547 Acked Write 256 Done Done Done false 103548 Acked Write 256 Done Done Done false 103549 Acked Write 256 Done Done Done false 103550 Acked Write 256 Done Done Done false 103551 Acked Write 256 Done Done Done false 103552 Acked Write 256 Done Done Done false 103553 Acked Write 256 Done Done Done false 103554 Acked Write 256 Done Done Done false 103555 Acked Write 256 Done Done Done false 103556 Acked Write 256 Done Done Done false 103557 Acked Write 256 Done Done Done false 103558 Acked Write 256 Done Done Done false 103559 Acked Write 256 Done Done Done false 103560 Acked Write 256 Done Done Done false 103561 Acked Write 256 Done Done Done false 103562 Acked Write 256 Done Done Done false 103563 Acked Write 256 Done Done Done false 103564 Acked Write 256 Done Done Done false 103565 Acked Write 256 Done Done Done false 103566 Acked Write 256 Done Done Done false 103567 Acked Write 256 Done Done Done false 103568 Acked Write 256 Done Done Done false 103569 Acked Write 256 Done Done Done false 103570 Acked Write 256 Done Done Done false 103571 Acked Write 256 Done Done Done false 103572 Acked Write 256 Done Done Done false 103573 Acked Write 256 Done Done Done false 103574 Acked Write 256 Done Done Done false 103575 Acked Write 256 Done Done Done false 103576 Acked Write 256 Done Done Done false 103577 Acked Write 256 Done Done Done false 103578 Acked Write 256 Done Done Done false 103579 Acked Write 256 Done Done Done false 103580 Acked Write 256 Done Done Done false 103581 Acked Write 256 Done Done Done false 103582 Acked Write 256 Done Done Done false 103583 Acked Write 256 Done Done Done false 103584 Acked Write 256 Done Done Sent false 103585 Acked Write 256 Done Done Sent false 103586 Acked Write 256 Done Done Sent false 103587 Acked Write 256 Done Done Sent false 103588 Acked Write 256 Done Done Sent false 103589 Acked Write 256 Done Done Sent false 103590 Acked Write 256 Done Done Sent false 103591 Acked Write 256 Done Done Sent false 103592 Acked Write 256 Done Done Sent false 103593 Acked Write 256 Done Done Sent false 103594 Acked Write 256 Done Done Sent false 103595 Acked Write 256 Done Done Sent false 103596 Acked Write 256 Done Done Sent false 103597 Acked Write 256 Done Done Sent false 103598 Acked Write 256 Done Done Sent false 103599 Acked Write 256 Done Done Sent false 103600 Acked Write 256 Done Done Sent false 103601 Acked Write 256 Done Done Sent false 103602 Acked Write 256 Done Done Sent false 103603 Acked Write 256 Done Done Sent false 103604 Acked Write 256 Done Done Sent false 103605 Acked Write 256 Done Done Sent false 103606 Acked Write 256 Done Done Sent false 103607 Acked Write 256 Done Done Sent false 103608 Acked Write 256 Done Done Sent false 103609 Acked Write 256 Done Done Sent false 103610 Acked Write 256 Done Done Sent false 103611 Acked Write 256 Done Done Sent false 103612 Acked Write 256 Done Done Sent false 103613 Acked Write 256 Done Done Sent false 103614 Acked Write 256 Done Done Sent false 103615 Acked Write 256 Done Done Sent false 103616 Acked Write 256 Done Done Sent false 103617 Acked Write 256 Done Done Sent false 103618 Acked Flush 0 Done Done Sent false STATES DS:0 DS:1 DS:2 TOTAL Sent 0 0 35 35 Done 141 141 106 388 Skipped 0 0 0 0 Error 0 0 0 0 Last Flush: 103618 103618 103477 Downstairs last five completed: 103477 103476 103475 103474 103473 Upstairs last five completed: 103618 103617 103616 103615 103614 CLIENT: Up:0 ds:141 act:3 ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0 Downstairs last five completed: 103618 103617 103616 103615 103614 Upstairs last five completed: 103618 103617 103616 103615 103614 CLIENT: Up:0 ds:0 act:3 CLIENT: All crucible jobs finished, exiting program