Using existing output directory "/tmp/dsc" start ds: 8810 start ds: 8820 start ds: 8830 start access at:127.0.0.1:9998 Starting downstairs at port 8810 Make output file at "/tmp/dsc/downstairs-8810.txt" [8810][0] reports Starting Starting downstairs at port 8830 Make output file at "/tmp/dsc/downstairs-8830.txt" [8830][2] reports Starting Starting downstairs at port 8820 Make output file at "/tmp/dsc/downstairs-8820.txt" [8820][1] reports Starting Downstairs /var/tmp/dsc/8810 port 8810 PID:Some(1451) [8810][0] reports Running Downstairs /var/tmp/dsc/8820 port 8820 PID:Some(1452) [8820][1] reports Running Downstairs /var/tmp/dsc/8830 port 8830 PID:Some(1453) Feb 12 01:34:40.418 INFO listening[8830][2] reports Running , local_addr: 127.0.0.1:9998 Control access at:127.0.0.1:9998 Feb 12 01:34:45.458 INFO accepted connection, remote_addr: 127.0.0.1:50052, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc DisableRestartAll disable restart on all: 3 Feb 12 01:34:45.458 INFO request completed, latency_us: 93, response_code: [8810] Disable keep_running 204[8820] Disable keep_running , uri: /disablerestart/all, method: POST, req_id: c4958e36-51cf-43a6-b8f5-11ace7dbc753, remote_addr: 127.0.0.1:50052, local_addr: 127.0.[8830] Disable keep_running 0.1:9998 Feb 12 01:38:41.920 INFO accepted connection, remote_addr: 127.0.0.1:34446, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc Shutdown Shutdown Shut it down Feb 12 01:38:41.921 INFO request completed, latency_us: [8810] Disable keep_running 47[8810] Got stop action so:false kr:false , response_code: 204, uri: /shutdown, method: POST, req_id: 11299315-d742-43e1-845b-d348793ffd07, remote_addr: 127.0[8830] Disable keep_running .[8830] Got stop action so:false kr:false 0.1:34446, local_addr: 127.0.0.[8820] Disable keep_running 1[8820] Got stop action so:false kr:false :9998 [8830] Exited with: ExitStatus(unix_wait_status(9)) [8810] Exited with: ExitStatus(unix_wait_status(9)) [8820] Exited with: ExitStatus(unix_wait_status(9)) [8830][2] reports Exit [8810][0] reports Exit [8820][1] reports Exit ":"2024-02-12T01:34:45.469752308Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456} {"msg":"Upstairs <-> Downstairs Message Version: 5","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469759298Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456} {"msg":"Crucible 59b6c94d-0b1b-40bf-ae80-645c63acf12d has session id: 08303815-a86d-4063-89ec-5c9e4623fc6c","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469764307Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Upstairs opts: Upstairs UUID: 59b6c94d-0b1b-40bf-ae80-645c63acf12d, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469774777Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Crucible stats registered with UUID: 59b6c94d-0b1b-40bf-ae80-645c63acf12d","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469779866Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"The guest has requested activation with gen:1","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469784536Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456} {"msg":"59b6c94d-0b1b-40bf-ae80-645c63acf12d active request set","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469857162Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469864342Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469878241Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469884331Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.46988975Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.4698952Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469901519Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"connecting to 127.0.0.1:8830","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469910689Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"io task","client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"connecting to 127.0.0.1:8810","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469923288Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"io task","client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"connecting to 127.0.0.1:8820","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.469929128Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"io task","client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_connection connected","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470037092Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"io task","client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_connection connected","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470153976Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"io task","client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_connection connected","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470169615Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"io task","client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470353965Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[1] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470365204Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470371224Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[2] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470376383Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470385383Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[0] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470390802Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470586692Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.470604031Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.47061443Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477042083Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[0] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477054143Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Waiting for true more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477060293Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477480409Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[1] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477494528Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Waiting for true more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477503968Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477578304Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[2] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477588363Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477593533Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477598013Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477604622Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477608962Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477613132Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477617502Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477622081Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477626301Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"[2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477630471Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Max found gen is 1","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477634681Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Generation requested: 1 >= found:1","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.4776389Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Next flush: 1","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.47764324Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"All extents match","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.47764748Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.47765166Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477655269Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477659499Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477665639Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"All required reconciliation work is completed","v":0,"name":"crucible","level":30,"Wait for a query_work_queue command to finish before sending IO time":"2024-02-12T01:34:45.477669949Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Set Downstairs and Upstairs active after reconciliation","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477677168Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"59b6c94d-0b1b-40bf-ae80-645c63acf12d is now active with session: 08303815-a86d-4063-89ec-5c9e4623fc6c","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477680918Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"Set Active after no reconciliation","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477684558Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456,"session_id":"08303815-a86d-4063-89ec-5c9e4623fc6c"} {"msg":"The guest has finished waiting for activation with:1","v":0,"name":"crucible","level":30,"time":"2024-02-12T01:34:45.477688188Z","hostname":"ip-10-150-1-49.us-west-2.compute.internal","pid":1456} Region: es:16384 ec:1600 bs:4096 ts:107374182400 tb:26214400 max_io:256 or 1048576 Fill test CLIENT: Tests done. All submitted work has been ACK'd ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:76 States: Active Active Active GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY 262532 Acked 263531 Write 100 Done Done Done false 262533 Acked 263532 Write 100 Done Done Done false 262534 Acked 263533 Write 100 Done Done Done false 262535 Acked 263534 Write 100 Done Done Done false 262536 Acked 263535 Write 100 Done Done Done false 262537 Acked 263536 Write 100 Done Done Done false 262538 Acked 263537 Write 100 Done Done Done false 262539 Acked 263538 Write 100 Done Done Done false 262540 Acked 263539 Write 100 Done Done Done false 262541 Acked 263540 Write 100 Done Done Done false 262542 Acked 263541 Write 100 Done Done Done false 262543 Acked 263542 Write 100 Done Done Done false 262544 Acked 263543 Write 100 Done Done Done false 262545 Acked 263544 Write 100 Done Done Done false 262546 Acked 263545 Write 100 Done Done Done false 262547 Acked 263546 Write 100 Done Done Done false 262548 Acked 263547 Write 100 Done Done Done false 262549 Acked 263548 Write 100 Done Done Done false 262550 Acked 263549 Write 100 Done Done Done false 262551 Acked 263550 Write 100 Done Done Done false 262552 Acked 263551 Write 100 Done Done Done false 262553 Acked 263552 Write 100 Done Done Done false 262554 Acked 263553 Write 100 Done Done Done false 262555 Acked 263554 Write 100 Done Done Done false 262556 Acked 263555 Write 100 Done Done Done false 262557 Acked 263556 Write 100 Done Done Done false 262558 Acked 263557 Write 100 Done Done Done false 262559 Acked 263558 Write 100 Done Done Done false 262560 Acked 263559 Write 100 Done Done Done false 262561 Acked 263560 Write 100 Done Done Done false 262562 Acked 263561 Write 100 Done Done Done false 262563 Acked 263562 Write 100 Done Done Done false 262564 Acked 263563 Write 100 Done Done Done false 262565 Acked 263564 Write 100 Done Done Done false 262566 Acked 263565 Write 100 Done Done Done false 262567 Acked 263566 Write 100 Done Done Done false 262568 Acked 263567 Write 100 Done Done Done false 262569 Acked 263568 Write 100 Done Done Done false 262570 Acked 263569 Write 100 Done Done Done false 262571 Acked 263570 Write 100 Done Done Done false 262572 Acked 263571 Write 100 Done Done Done false 262573 Acked 263572 Write 100 Done Done Done false 262574 Acked 263573 Write 100 Done Done Done false 262575 Acked 263574 Write 100 Done Done Done false 262576 Acked 263575 Write 100 Done Done Done false 262577 Acked 263576 Write 100 Done Done Done false 262578 Acked 263577 Write 100 Done Done Done false 262579 Acked 263578 Write 100 Done Done Done false 262580 Acked 263579 Write 100 Done Done Done false 262581 Acked 263580 Write 100 Done Done Done false 262582 Acked 263581 Write 100 Done Done Done false 262583 Acked 263582 Write 100 Done Done Done false 262584 Acked 263583 Write 100 Done Done Done false 262585 Acked 263584 Write 100 Done Done Done false 262586 Acked 263585 Write 100 Done Done Done false 262587 Acked 263586 Write 100 Done Done Done false 262588 Acked 263587 Write 100 Done Done Done false 262589 Acked 263588 Write 100 Done Done Done false 262590 Acked 263589 Write 100 Done Done Done false 262591 Acked 263590 Write 100 Done Done Done false 262592 Acked 263591 Write 100 Done Done Done false 262593 Acked 263592 Write 100 Done Done Done false 262594 Acked 263593 Write 100 Done Done Done false 262595 Acked 263594 Write 100 Done Done Done false 262596 Acked 263595 Write 100 Done Done Done false 262597 Acked 263596 Write 100 Done Done Done false 262598 Acked 263597 Write 100 Done Done Done false 262599 Acked 263598 Write 100 Done Done Done false 262600 Acked 263599 Write 100 Done Done Done false 262601 Acked 263600 Write 100 Done Done Done false 262602 Acked 263601 Write 100 Done Done Done false 262603 Acked 263602 Write 100 Done Done Done false 262604 Acked 263603 Write 100 Done Done Done false 262605 Acked 263604 Write 100 Done Done Done false 262606 Acked 263605 Write 100 Done Done Done false 262607 Acked 263606 Flush 0 Sent Done Done false STATES DS:0 DS:1 DS:2 TOTAL New 0 0 0 0 Sent 1 0 0 1 Done 75 76 76 227 Skipped 0 0 0 0 Error 0 0 0 0 Last Flush: 263530 263606 263606 Downstairs last five completed: 263530 263529 263528 263527 263526 Upstairs last five completed: 262607 262606 262605 262604 262603 CLIENT: Up:0 ds:76 act:3 ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0 Downstairs last five completed: 263606 263605 263604 263603 263602 Upstairs last five completed: 262607 262606 262605 262604 262603 CLIENT: Up:0 ds:0 act:3 CLIENT: All crucible jobs finished, exiting program