Using existing output directory "/tmp/dsc" start ds: 8810 start ds: 8820 start ds: 8830 start access at:127.0.0.1:9998 Starting downstairs at port 8810 Make output file at "/tmp/dsc/downstairs-8810.txt" Starting downstairs at port 8820 Make output file at "/tmp/dsc/downstairs-8820.txt" [8810][0] reports Starting [8820][1] reports Starting Starting downstairs at port 8830 Make output file at "/tmp/dsc/downstairs-8830.txt" [8830][2] reports Starting Downstairs /var/tmp/dsc/8810 port 8810 PID:Some(1451) [8810][0] reports Running Downstairs /var/tmp/dsc/8820 port 8820 PID:Some(1452) [8820][1] reports Running Downstairs /var/tmp/dsc/8830 port 8830 PID:Some(1453) Feb 27 21:46:25.962 INFO listening, local_addr: 127.0.0.1:9998 [8830][2] reports Running Control access at:127.0.0.1:9998 Feb 27 21:46:30.999 INFO accepted connection, remote_addr: 127.0.0.1:35667, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc DisableRestartAll disable restart on all: 3 Feb 27 21:46:31.000 INFO request completed, latency_us: 106, response_code: 204, [8810] Disable keep_running uri[8820] Disable keep_running : /disablerestart/all, method: POST, req_id: d8a3cf22-30b5-46ca-9e7d-2f759e45b85e, remote_addr: 127.0.0.1:[8830] Disable keep_running 35667, local_addr: 127.0.0.1:9998 Feb 27 21:51:48.306 INFO accepted connection, remote_addr: 127.0.0.1:46178, local_addr: 127.0.0.1:9998 Main task has work to do, go find it got dsc Shutdown Shutdown Feb 27 21:51:48.307 INFO request completed, latency_us: 56, [8810] Disable keep_running response_codeShut it down :[8820] Disable keep_running [8830] Disable keep_running 204, uri: /shutdown, method[8830] Got stop action so:false kr:false :[8820] Got stop action so:false kr:false POST, req_id: f5433008-46eb-43c4-8d57-8acede992722, remote_addr: 127.0.0.1:46178, local_addr: 127.[8810] Got stop action so:false kr:false 0.0.1:9998 [8820] Exited with: ExitStatus(unix_wait_status(9)) [8820][1] reports Exit [8830] Exited with: ExitStatus(unix_wait_status(9)) [8830][2] reports Exit [8810] Exited with: ExitStatus(unix_wait_status(9)) [8810][0] reports Exit 62187Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456} Crucible runtime is spawned {"msg":"Upstairs <-> Downstairs Message Version: 5","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.012981044Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456} {"msg":"Crucible 5ef764a3-2832-4b98-b924-f82e59ae2ce6 has session id: e29d5344-5c96-4c7e-9f16-df9a663e6c48","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.012988423Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Upstairs opts: Upstairs UUID: 5ef764a3-2832-4b98-b924-f82e59ae2ce6, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013003212Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Crucible stats registered with UUID: 5ef764a3-2832-4b98-b924-f82e59ae2ce6","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.01301092Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"The guest has requested activation with gen:1","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.01301525Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456} {"msg":"5ef764a3-2832-4b98-b924-f82e59ae2ce6 active request set","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013070653Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013076292Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013089401Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.01309434Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.01309866Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"sending connect oneshot to client","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013102869Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"client set_active_request while in New; waiting...","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013107039Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"connecting to 127.0.0.1:8830","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013120647Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"io task","client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"connecting to 127.0.0.1:8810","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013134745Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"io task","client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"connecting to 127.0.0.1:8820","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013157892Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"io task","client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_connection connected","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013324131Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"io task","client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_connection connected","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013340239Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"io task","client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_connection connected","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013350918Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"io task","client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013621454Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[0] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013635042Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013764206Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[1] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013777115Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013787953Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[2] Transition from New to WaitActive","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.013795552Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.014022744Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.014035302Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.014046711Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.020826882Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[2] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.020838571Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Waiting for true more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.02084346Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021443145Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[0] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021453834Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Waiting for true more clients to be ready","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021458973Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"ds_transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.02179818Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[1] Transition from WaitActive to WaitQuorum","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.0218055Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021810109Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021817808Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021822917Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021827497Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021832216Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021836516Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021840715Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021844855Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"[2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021849104Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Max found gen is 1","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021853304Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Generation requested: 1 >= found:1","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021857613Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Next flush: 1","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021861872Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"All extents match","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021866102Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"":"downstairs","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021870302Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021874051Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"0","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.02187835Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"1","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.02188453Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"client":"2","":"downstairs","session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"All required reconciliation work is completed","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021888829Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"Set Downstairs and Upstairs active after reconciliation","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021892619Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"5ef764a3-2832-4b98-b924-f82e59ae2ce6 is now active with session: e29d5344-5c96-4c7e-9f16-df9a663e6c48","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021896418Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} Wait for a query_work_queue command to finish before sending IO {"msg":"Set Active after no reconciliation","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021903058Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456,"session_id":"e29d5344-5c96-4c7e-9f16-df9a663e6c48"} {"msg":"The guest has finished waiting for activation with:1","v":0,"name":"crucible","level":30,"time":"2024-02-27T21:46:31.021907137Z","hostname":"ip-10-150-1-164.us-west-2.compute.internal","pid":1456} Region: es:16384 ec:1600 bs:4096 ts:107374182400 tb:26214400 max_io:256 or 1048576 Fill test CLIENT: Tests done. All submitted work has been ACK'd ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:84 States: Active Active Active GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY 262685 Acked 263684 Write 100 Done Done Done false 262686 Acked 263685 Write 100 Done Done Done false 262687 Acked 263686 Write 100 Done Done Done false 262688 Acked 263687 Write 100 Done Done Done false 262689 Acked 263688 Write 100 Done Done Done false 262690 Acked 263689 Write 100 Done Done Done false 262691 Acked 263690 Write 100 Done Done Done false 262692 Acked 263691 Write 100 Done Done Done false 262693 Acked 263692 Write 100 Done Done Done false 262694 Acked 263693 Write 100 Done Done Done false 262695 Acked 263694 Write 100 Done Done Done false 262696 Acked 263695 Write 100 Done Done Done false 262697 Acked 263696 Write 100 Done Done Done false 262698 Acked 263697 Write 100 Done Done Done false 262699 Acked 263698 Write 100 Done Done Done false 262700 Acked 263699 Write 100 Done Done Done false 262701 Acked 263700 Write 100 Done Done Done false 262702 Acked 263701 Write 100 Done Done Done false 262703 Acked 263702 Write 100 Done Done Done false 262704 Acked 263703 Write 100 Done Done Done false 262705 Acked 263704 Write 100 Done Done Done false 262706 Acked 263705 Write 100 Done Done Done false 262707 Acked 263706 Write 100 Done Done Done false 262708 Acked 263707 Write 100 Done Done Done false 262709 Acked 263708 Write 100 Done Done Done false 262710 Acked 263709 Write 100 Done Done Done false 262711 Acked 263710 Write 100 Done Done Done false 262712 Acked 263711 Write 100 Done Done Done false 262713 Acked 263712 Write 100 Done Done Done false 262714 Acked 263713 Write 100 Done Done Done false 262715 Acked 263714 Write 100 Done Done Done false 262716 Acked 263715 Write 100 Done Done Done false 262717 Acked 263716 Write 100 Done Done Done false 262718 Acked 263717 Write 100 Done Done Done false 262719 Acked 263718 Write 100 Done Done Done false 262720 Acked 263719 Write 100 Done Done Done false 262721 Acked 263720 Write 100 Done Done Done false 262722 Acked 263721 Write 100 Done Done Done false 262723 Acked 263722 Write 100 Done Done Done false 262724 Acked 263723 Write 100 Done Done Done false 262725 Acked 263724 Write 100 Done Done Done false 262726 Acked 263725 Write 100 Done Done Done false 262727 Acked 263726 Write 100 Done Done Done false 262728 Acked 263727 Write 100 Done Done Done false 262729 Acked 263728 Write 100 Done Done Done false 262730 Acked 263729 Write 100 Done Done Done false 262731 Acked 263730 Write 100 Done Done Done false 262732 Acked 263731 Write 100 Done Done Done false 262733 Acked 263732 Write 100 Done Done Done false 262734 Acked 263733 Write 100 Done Done Done false 262735 Acked 263734 Write 100 Done Done Done false 262736 Acked 263735 Write 100 Done Done Done false 262737 Acked 263736 Write 100 Done Done Done false 262738 Acked 263737 Write 100 Done Done Done false 262739 Acked 263738 Write 100 Done Done Done false 262740 Acked 263739 Write 100 Done Done Done false 262741 Acked 263740 Write 100 Done Done Done false 262742 Acked 263741 Write 100 Done Done Done false 262743 Acked 263742 Write 100 Done Done Done false 262744 Acked 263743 Write 100 Done Done Done false 262745 Acked 263744 Write 100 Done Done Done false 262746 Acked 263745 Write 100 Done Done Done false 262747 Acked 263746 Write 100 Done Done Done false 262748 Acked 263747 Write 100 Done Done Done false 262749 Acked 263748 Write 100 Done Done Done false 262750 Acked 263749 Write 100 Done Done Done false 262751 Acked 263750 Write 100 Done Done Done false 262752 Acked 263751 Write 100 Done Done Done false 262753 Acked 263752 Write 100 Done Done Done false 262754 Acked 263753 Write 100 Done Done Done false 262755 Acked 263754 Write 100 Done Done Done false 262756 Acked 263755 Write 100 Done Done Done false 262757 Acked 263756 Write 100 Done Done Done false 262758 Acked 263757 Write 100 Done Done Done false 262759 Acked 263758 Write 100 Done Done Done false 262760 Acked 263759 Write 100 Done Done Done false 262761 Acked 263760 Write 100 Done Done Done false 262762 Acked 263761 Write 100 Done Done Done false 262763 Acked 263762 Write 100 Done Done Done false 262764 Acked 263763 Write 100 Done Done Done false 262765 Acked 263764 Write 100 Done Done Done false 262766 Acked 263765 Write 100 Done Done Done false 262767 Acked 263766 Write 100 Done Done Done false 262768 Acked 263767 Flush 0 Done Done Sent false STATES DS:0 DS:1 DS:2 TOTAL New 0 0 0 0 Sent 0 0 1 1 Done 84 84 83 251 Skipped 0 0 0 0 Error 0 0 0 0 Last Flush: 263767 263767 263683 Downstairs last five completed: 263683 263682 263681 263680 263679 Upstairs last five completed: 262768 262767 262766 262765 262764 CLIENT: Up:0 ds:84 act:3 ---------------------------------------------------------------- Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0 Downstairs last five completed: 263767 263766 263765 263764 263763 Upstairs last five completed: 262768 262767 262766 262765 262764 CLIENT: Up:0 ds:0 act:3 CLIENT: All crucible jobs finished, exiting program