Max level shown:
1 starting November 19, 2024 at 05:07:53 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.26505324
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.024323769
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.020468645
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start access at:127.0.0.1:9998
14 start ds: 8820
15 start ds: 8830
16 Starting downstairs at port 8810
17 Starting downstairs at port 8820
18 Make output file at "/tmp/dsc/downstairs-8810.txt"
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Starting downstairs at port 8830
21 [8810][0] initial start wait reports Starting
22 Make output file at "/tmp/dsc/downstairs-8830.txt"
23 [8820][1] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1192)
26 [8830][2] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1193)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1194)
30 [8810][0] initial start wait reports Running
31 All downstairs are running
32 Nov 19 17:07:53.858 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Nov 19 17:07:58.951 INFO accepted connection, remote_addr: 127.0.0.1:64022, local_addr: 127.0.0.1:9998
36 Nov 19 17:07:58.954 INFO request completed, latency_us: 511, response_code: 200, uri: /regioninfo, method: GET, req_id: 74e936cd-70c7-49d7-90fb-73e8e9546cb9, remote_addr: 127.0.0.1:64022, local_addr: 127.0.0.1:9998
37 Nov 19 17:07:58.956 INFO request completed, latency_us: 148, response_code: 200, uri: /regioncount, method: GET, req_id: 89d34289-524b-4231-b67a-a0d276d08a24, remote_addr: 127.0.0.1:64022, local_addr: 127.0.0.1:9998
382024-11-19T17:07:58.956ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392024-11-19T17:07:58.957ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Nov 19 17:07:58.957 INFO request completed, latency_us: 163, response_code: 200, uri: /port/cid/0, method: GET, req_id: d4983775-92a6-498d-8644-31d6c5e48018, remote_addr: 127.0.0.1:64022, local_addr: 127.0.0.1:9998
41 Nov 19 17:07:58.958 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/1, method: GET, req_id: 50605616-b46c-44d6-931d-88b4c6474e30, remote_addr: 127.0.0.1:64022, local_addr: 127.0.0.1:9998
42 Nov 19 17:07:58.958 INFO request completed, latency_us: 89, response_code: 200, uri: /port/cid/2, method: GET, req_id: 5c056da6-9f84-4817-96e8-04d861fea5a9, remote_addr: 127.0.0.1:64022, local_addr: 127.0.0.1:9998
432024-11-19T17:07:58.958ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442024-11-19T17:07:58.960ZINFOcrucible: Upstairs starts
452024-11-19T17:07:58.960ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "775749f049384d6df9adbcf0df5fb1eb2a3a0b7e", git_commit_timestamp: "2024-11-19T16:38:47.000000000Z", git_branch: "mkeeter/early-io-rejection", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462024-11-19T17:07:58.960ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472024-11-19T17:07:58.960ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482024-11-19T17:07:58.961ZINFOcrucible: Crucible 97651f56-2540-42e8-bc17-5c1acd5047dd has session id: 8118eba2-ff9f-493d-a1e4-2d4e42836501 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
492024-11-19T17:07:58.961ZINFOcrucible: Upstairs opts: Upstairs UUID: 97651f56-2540-42e8-bc17-5c1acd5047dd, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
502024-11-19T17:07:58.961ZINFOcrucible: Crucible stats registered with UUID: 97651f56-2540-42e8-bc17-5c1acd5047dd session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
512024-11-19T17:07:58.997ZINFOcrucible: Downstairs in volume = 3 task = crutest
522024-11-19T17:07:58.997ZINFOcrucible: The guest has requested activation with gen:1
532024-11-19T17:07:58.997ZINFOcrucible: 97651f56-2540-42e8-bc17-5c1acd5047dd active request set session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
542024-11-19T17:07:58.997ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
552024-11-19T17:07:58.998ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
562024-11-19T17:07:58.998ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
572024-11-19T17:07:58.998ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
582024-11-19T17:07:58.998ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
592024-11-19T17:07:58.998ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
602024-11-19T17:07:58.998ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
612024-11-19T17:07:58.998ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
622024-11-19T17:07:58.998ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
632024-11-19T17:07:58.998ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37282) = io task client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
642024-11-19T17:07:58.998ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57872) = io task client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
652024-11-19T17:07:58.998ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51858) = io task client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
662024-11-19T17:07:59.000ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
672024-11-19T17:07:59.000ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
682024-11-19T17:07:59.000ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
692024-11-19T17:07:59.000ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
702024-11-19T17:07:59.001ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
712024-11-19T17:07:59.001ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
722024-11-19T17:07:59.002ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
732024-11-19T17:07:59.002ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
742024-11-19T17:07:59.002ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
752024-11-19T17:07:59.003ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
762024-11-19T17:07:59.003ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
772024-11-19T17:07:59.003ZINFOcrucible: Waiting for more clients to be ready session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
782024-11-19T17:07:59.003ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
792024-11-19T17:07:59.003ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
802024-11-19T17:07:59.003ZINFOcrucible: Waiting for more clients to be ready session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
812024-11-19T17:07:59.003ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
822024-11-19T17:07:59.003ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
832024-11-19T17:07:59.003ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
842024-11-19T17:07:59.003ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
852024-11-19T17:07:59.003ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
862024-11-19T17:07:59.003ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
872024-11-19T17:07:59.003ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
882024-11-19T17:07:59.003ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
892024-11-19T17:07:59.003ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
902024-11-19T17:07:59.003ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
912024-11-19T17:07:59.003ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
922024-11-19T17:07:59.003ZINFOcrucible: Max found gen is 1 = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
932024-11-19T17:07:59.003ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
942024-11-19T17:07:59.003ZINFOcrucible: Next flush: 1 = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
952024-11-19T17:07:59.004ZINFOcrucible: All extents match = downstairs session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
962024-11-19T17:07:59.004ZINFOcrucible: No downstairs reconciliation required session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
972024-11-19T17:07:59.004ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
982024-11-19T17:07:59.004ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
992024-11-19T17:07:59.004ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1002024-11-19T17:07:59.004ZINFOcrucible: All required reconciliation work is completed session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1012024-11-19T17:07:59.004ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1022024-11-19T17:07:59.004ZINFOcrucible: 97651f56-2540-42e8-bc17-5c1acd5047dd is now active with session: 8118eba2-ff9f-493d-a1e4-2d4e42836501 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1032024-11-19T17:07:59.004ZINFOcrucible: Set Active after no reconciliation session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1042024-11-19T17:07:59.004ZINFOcrucible: The guest has finished waiting for activation with:1
105 Activate successful
106 Wait for a query_work_queue command to finish before sending IO
107 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1021 Acked Read 256 Done Done Done false
117 1022 Acked Read 256 Done Done Done false
118 1023 Acked Read 256 Done Done Done false
119 1024 Acked Read 256 Done Done Done false
120 1025 Acked Read 256 Done Done Done false
121 1026 Acked Read 256 Done Done Done false
122 1027 Acked Read 256 Done Done Done false
123 1028 Acked Read 256 Done Done Done false
124 1029 Acked Read 256 Done Done Done false
125 1030 Acked Read 256 Done Done Done false
126 1031 Acked Read 256 Done Done Done false
127 1032 Acked Read 256 Done Done Done false
128 1033 Acked Read 256 Done Done Done false
129 1034 Acked Read 256 Done Done Done false
130 1035 Acked Read 256 Done Done Done false
131 1036 Acked Read 256 Done Done Done false
132 1037 Acked Read 256 Done Done Done false
133 1038 Acked Read 256 Done Done Done false
134 1039 Acked Read 256 Done Done Done false
135 1040 Acked Read 136 Done Done Done false
136 STATES DS:0 DS:1 DS:2 TOTAL
137 Sent 0 0 0 0
138 Done 20 20 20 60
139 Skipped 0 0 0 0
140 Error 0 0 0 0
141 Last Flush: 1020 1020 1020
142 Downstairs last five completed: 1020 1019 1018 1017 1016
143 Upstairs last five completed: 1040 1039 1038 1037 1036
144 CLIENT: Up:0 ds:20 act:3
145 ----------------------------------------------------------------
146 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
147 Downstairs last five completed: 1041 1040 1039 1038 1037
148 Upstairs last five completed: 1041 1040 1039 1038 1037
149 CLIENT: Up:0 ds:0 act:3
150 CLIENT: All crucible jobs finished, exiting program
1512024-11-19T17:08:03.565ZWARNcrucible: Guest handle has been dropped
1522024-11-19T17:08:03.565ZWARNcrucible: client_stop_rx closed unexpectedly: RecvError(()) = io task client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1532024-11-19T17:08:03.565ZWARNcrucible: client_stop_rx closed unexpectedly: RecvError(()) = io task client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1542024-11-19T17:08:03.565ZWARNcrucible: client task is sending Done(QueueClosed) = io task client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1552024-11-19T17:08:03.565ZWARNcrucible: client task is sending Done(QueueClosed) = io task client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1562024-11-19T17:08:03.565ZWARNcrucible: client task could not reply to main task; shutting down? = io task client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1572024-11-19T17:08:03.565ZWARNcrucible: client task could not reply to main task; shutting down? = io task client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1582024-11-19T17:08:03.565ZINFOcrucible: client task is exiting = io task client = 0 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
1592024-11-19T17:08:03.565ZINFOcrucible: client task is exiting = io task client = 1 session_id = 8118eba2-ff9f-493d-a1e4-2d4e42836501
160 Replay loop starts now November 19, 2024 at 05:08:03 PM UTC
161 Nov 19 17:08:03.640 INFO accepted connection, remote_addr: 127.0.0.1:37260, local_addr: 127.0.0.1:9998
162 Nov 19 17:08:03.641 INFO request completed, latency_us: 176, response_code: 200, uri: /regioninfo, method: GET, req_id: 35559bc3-9555-4c3c-9541-e5b7051d646d, remote_addr: 127.0.0.1:37260, local_addr: 127.0.0.1:9998
1632024-11-19T17:08:03.641ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
164 Nov 19 17:08:03.642 INFO request completed, latency_us: 110, response_code: 200, uri: /regioncount, method: GET, req_id: ec506bbb-181c-4c56-bb2f-fca63ccd8d54, remote_addr: 127.0.0.1:37260, local_addr: 127.0.0.1:9998
1652024-11-19T17:08:03.642ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
166 Nov 19 17:08:03.642 INFO request completed, latency_us: 112, response_code: 200, uri: /port/cid/0, method: GET, req_id: 7e49f2eb-f253-4cdb-969d-48fce41e41fe, remote_addr: 127.0.0.1:37260, local_addr: 127.0.0.1:9998
167 Nov 19 17:08:03.643 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/1, method: GET, req_id: 89b0c2be-e09e-4b03-8f7a-dd8f1bea5d2a, remote_addr: 127.0.0.1:37260, local_addr: 127.0.0.1:9998
168 Nov 19 17:08:03.643 INFO request completed, latency_us: 89, response_code: 200, uri: /port/cid/2, method: GET, req_id: ae6b2ea7-9c9e-4442-b266-f5a91c325dc8, remote_addr: 127.0.0.1:37260, local_addr: 127.0.0.1:9998
1692024-11-19T17:08:03.643ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1702024-11-19T17:08:03.644ZINFOcrucible: Upstairs starts
1712024-11-19T17:08:03.644ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "775749f049384d6df9adbcf0df5fb1eb2a3a0b7e", git_commit_timestamp: "2024-11-19T16:38:47.000000000Z", git_branch: "mkeeter/early-io-rejection", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1722024-11-19T17:08:03.644ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1732024-11-19T17:08:03.644ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1742024-11-19T17:08:03.644ZINFOcrucible: Crucible 277ae1a4-cfd9-4ffb-b320-525dbe922387 has session id: 05650a15-902e-469c-91e5-cba45f07470e session_id = 05650a15-902e-469c-91e5-cba45f07470e
1752024-11-19T17:08:03.644ZINFOcrucible: Upstairs opts: Upstairs UUID: 277ae1a4-cfd9-4ffb-b320-525dbe922387, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 05650a15-902e-469c-91e5-cba45f07470e
1762024-11-19T17:08:03.644ZINFOcrucible: Crucible stats registered with UUID: 277ae1a4-cfd9-4ffb-b320-525dbe922387 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1772024-11-19T17:08:03.678ZINFOcrucible: Downstairs in volume = 3 task = crutest
1782024-11-19T17:08:03.678ZINFOcrucible: The guest has requested activation with gen:2
1792024-11-19T17:08:03.678ZINFOcrucible: 277ae1a4-cfd9-4ffb-b320-525dbe922387 active request set session_id = 05650a15-902e-469c-91e5-cba45f07470e
1802024-11-19T17:08:03.678ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1812024-11-19T17:08:03.678ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1822024-11-19T17:08:03.678ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1832024-11-19T17:08:03.678ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1842024-11-19T17:08:03.678ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1852024-11-19T17:08:03.678ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1862024-11-19T17:08:03.678ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1872024-11-19T17:08:03.678ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1882024-11-19T17:08:03.678ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1892024-11-19T17:08:03.678ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51325) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1902024-11-19T17:08:03.679ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40251) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1912024-11-19T17:08:03.679ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57140) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1922024-11-19T17:08:03.679ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1932024-11-19T17:08:03.679ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1942024-11-19T17:08:03.679ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1952024-11-19T17:08:03.679ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1962024-11-19T17:08:03.679ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1972024-11-19T17:08:03.679ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1982024-11-19T17:08:03.680ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1992024-11-19T17:08:03.680ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2002024-11-19T17:08:03.680ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2012024-11-19T17:08:03.681ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2022024-11-19T17:08:03.681ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2032024-11-19T17:08:03.681ZINFOcrucible: Waiting for more clients to be ready session_id = 05650a15-902e-469c-91e5-cba45f07470e
2042024-11-19T17:08:03.681ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2052024-11-19T17:08:03.681ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2062024-11-19T17:08:03.681ZINFOcrucible: Waiting for more clients to be ready session_id = 05650a15-902e-469c-91e5-cba45f07470e
2072024-11-19T17:08:03.681ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2082024-11-19T17:08:03.681ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2092024-11-19T17:08:03.681ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2102024-11-19T17:08:03.681ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2112024-11-19T17:08:03.681ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2122024-11-19T17:08:03.681ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2132024-11-19T17:08:03.681ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2142024-11-19T17:08:03.681ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2152024-11-19T17:08:03.681ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2162024-11-19T17:08:03.681ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
217 {"Activate successful
218 msg"Wait for a query_work_queue command to finish before sending IO
219 :"[2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:08:03.681382672Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"downstairs","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
2202024-11-19T17:08:03.681ZINFOcrucible: Max found gen is 2 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2212024-11-19T17:08:03.681ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2222024-11-19T17:08:03.681ZINFOcrucible: Next flush: 2 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2232024-11-19T17:08:03.681ZINFOcrucible: All extents match = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2242024-11-19T17:08:03.681ZINFOcrucible: No downstairs reconciliation required session_id = 05650a15-902e-469c-91e5-cba45f07470e
2252024-11-19T17:08:03.681ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2262024-11-19T17:08:03.681ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2272024-11-19T17:08:03.681ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2282024-11-19T17:08:03.681ZINFOcrucible: All required reconciliation work is completed session_id = 05650a15-902e-469c-91e5-cba45f07470e
2292024-11-19T17:08:03.681ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 05650a15-902e-469c-91e5-cba45f07470e
2302024-11-19T17:08:03.681ZINFOcrucible: 277ae1a4-cfd9-4ffb-b320-525dbe922387 is now active with session: 05650a15-902e-469c-91e5-cba45f07470e session_id = 05650a15-902e-469c-91e5-cba45f07470e
2312024-11-19T17:08:03.681ZINFOcrucible: Set Active after no reconciliation session_id = 05650a15-902e-469c-91e5-cba45f07470e
2322024-11-19T17:08:03.681ZINFOcrucible: The guest has finished waiting for activation with:2
233 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
234 Loading write count information from file "/tmp/test_replay_verify.log"
235 Nov 19 17:08:03.715 INFO accepted connection, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
236 Main task has work to do, go find it
237 got dsc Stop(1)
238 stop 1
239 Nov 19 17:08:03.715 INFO request completed, latency_us: 171, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 6f868481-cdd7-4c20-8a00-f3b3442853ec, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
240 [8820] Got stop action so:false kr:false
2412024-11-19T17:08:03.716ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2422024-11-19T17:08:03.716ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2432024-11-19T17:08:03.716ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
244 Nov 19 17:08:03.716 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/1, method: GET, req_id: d5e7d568-f0ee-4af8-a7dd-2b2e0db827e2, remote_addr: 127.0.0.1:64175{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:08:03.716623937Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
245 , local_addr: 127.0.0.1:9998
2462024-11-19T17:08:03.716ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2472024-11-19T17:08:03.716ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
248 [8820] Exited with: ExitStatus(unix_wait_status(9))
249 [8820][1] reports Exit
250 Nov 19 17:08:07.719 INFO request completed, latency_us: 316, response_code: 200, uri: /state/cid/1, method: GET, req_id: 710ab91b-6e91-4aea-b0b1-4ff276e17d57, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
251 001/300 Read block 4427 len 4096
252 002/300 Flush
253 003/300 Write block 3330 len 20480 data: 2 2 2 2 2
254 004/300 Read block 864 len 12288
255 005/300 Write block 232 len 12288 data: 2 2 2
256 006/300 Flush
257 007/300 Write block 469 len 40960 data: 2 2 2 2 2 2 2 2 2 2
258 008/300 Write block 878 len 36864 data: 2 2 2 2 2 2 2 2 2
259 009/300 Read block 4147 len 8192
260 010/300 Read block 760 len 4096
261 011/300 Write block 2904 len 12288 data: 2 2 2
262 012/300 Flush
263 013/300 Write block 2436 len 32768 data: 2 2 2 2 2 2 2 2
264 014/300 Read block 2088 len 4096
265 015/300 Read block 18 len 28672
266 016/300 Read block 3003 len 40960
267 017/300 Write block 3069 len 8192 data: 2 2
268 018/300 Write block 1351 len 32768 data: 2 2 2 2 2 2 2 2
269 019/300 Read block 946 len 28672
270 020/300 Read block 4621 len 8192
271 021/300 Write block 4799 len 8192 data: 2 2
272 022/300 Read block 1519 len 8192
273 023/300 Read block 3659 len 40960
274 024/300 Read block 1079 len 24576
275 025/300 Write block 1099 len 12288 data: 2 2 2
276 026/300 Write block 3249 len 24576 data: 2 2 2 2 2 2
277 027/300 Write block 1046 len 36864 data: 2 2 2 2 2 2 2 2 2
278 028/300 Read block 4771 len 16384
279 029/300 Write block 59 len 36864 data: 2 2 2 2 2 2 2 2 2
280 030/300 Read block 170 len 8192
281 031/300 Read block 1808 len 36864
282 032/300 Read block 4353 len 20480
283 033/300 Read block 1165 len 16384
284 034/300 Write block 1817 len 40960 data: 2 2 2 2 2 2 2 2 2 2
285 035/300 Read block 3588 len 8192
286 036/300 Write block 469 len 36864 data: 3 3 3 3 3 3 3 3 3
287 037/300 Read block 1143 len 12288
288 038/300 Write block 1441 len 4096 data: 2
289 039/300 Write block 2097 len 40960 data: 2 2 2 2 2 2 2 2 2 2
290 040/300 Write block 1878 len 16384 data: 2 2 2 2
291 041/300 Write block 1181 len 36864 data: 2 2 2 2 2 2 2 2 2
292 042/300 Write block 2680 len 32768 data: 2 2 2 2 2 2 2 2
293 043/300 Read block 3577 len 20480
294 044/300 Read block 692 len 12288
295 045/300 Read block 3301 len 20480
296 046/300 Read block 4206 len 40960
297 047/300 Write block 3409 len 20480 data: 2 2 2 2 2
298 048/300 Write block 2632 len 36864 data: 2 2 2 2 2 2 2 2 2
299 049/300 Write block 858 len 4096 data: 2
300 050/300 Read block 2396 len 24576
301 051/300 Read block 4929 len 8192
302 052/300 Read block 4148 len 20480
303 053/300 Read block 1747 len 40960
304 054/300 Read block 2742 len 40960
305 055/300 Write block 100 len 16384 data: 2 2 2 2
306 056/300 Flush
307 057/300 Write block 1741 len 8192 data: 2 2
308 058/300 Read block 3956 len 16384
309 059/300 Write block 2960 len 36864 data: 2 2 2 2 2 2 2 2 2
310 060/300 Write block 3030 len 16384 data: 2 2 2 2
311 061/300 Read block 2585 len 8192
312 062/300 Read block 3032 len 28672
313 063/300 Read block 4122 len 24576
314 064/300 Write block 1542 len 40960 data: 2 2 2 2 2 2 2 2 2 2
315 065/300 Read block 4886 len 40960
316 066/300 Write block 2313 len 32768 data: 2 2 2 2 2 2 2 2
317 067/300 Read block 2076 len 40960
318 068/300 Read block 1489 len 28672
319 069/300 Write block 3855 len 36864 data: 2 2 2 2 2 2 2 2 2
320 070/300 Flush
321 071/300 Write block 3129 len 16384 data: 2 2 2 2
322 072/300 Write block 3335 len 28672 data: 2 2 2 2 2 2 2
323 073/300 Write block 905 len 20480 data: 2 2 2 2 2
324 074/300 Write block 394 len 40960 data: 2 2 2 2 2 2 2 2 2 2
325 075/300 Write block 870 len 24576 data: 2 2 2 2 2 2
326 076/300 Read block 56 len 4096
327 077/300 Write block 4282 len 28672 data: 2 2 2 2 2 2 2
328 078/300 Read block 2903 len 20480
329 079/300 Write block 535 len 36864 data: 2 2 2 2 2 2 2 2 2
330 080/300 Write block 42 len 32768 data: 2 2 2 2 2 2 2 2
331 081/300 Write block 3799 len 12288 data: 2 2 2
332 082/300 Read block 4036 len 12288
333 083/300 Write block 3872 len 40960 data: 2 2 2 2 2 2 2 2 2 2
334 084/300 Read block 2092 len 24576
335 085/300 Read block 2853 len 24576
336 086/300 Write block 3395 len 16384 data: 2 2 2 2
337 087/300 Write block 1063 len 8192 data: 2 2
338 088/300 Read block 4878 len 28672
339 089/300 Write block 644 len 40960 data: 2 2 2 2 2 2 2 2 2 2
340 090/300 Write block 1312 len 40960 data: 2 2 2 2 2 2 2 2 2 2
341 091/300 Flush
342 092/300 Read block 1822 len 8192
343 093/300 Write block 4932 len 32768 data: 2 2 2 2 2 2 2 2
344 094/300 Write block 3867 len 32768 data: 2 2 2 2 2 3 3 3
345 095/300 Read block 2460 len 36864
346 096/300 Flush
347 097/300 Read block 3808 len 36864
348 098/300 Write block 4552 len 16384 data: 2 2 2 2
349 099/300 Read block 4651 len 20480
350 100/300 Read block 285 len 24576
351 101/300 Read block 1752 len 24576
352 102/300 Read block 4828 len 4096
353 103/300 Read block 2603 len 16384
354 104/300 Write block 4254 len 20480 data: 2 2 2 2 2
355 105/300 Write block 4654 len 40960 data: 2 2 2 2 2 2 2 2 2 2
356 106/300 Write block 2025 len 16384 data: 2 2 2 2
357 107/300 Read block 4052 len 8192
358 108/300 Read block 4781 len 4096
359 109/300 Read block 4302 len 16384
360 110/300 Read block 1605 len 28672
361 111/300 Read block 3165 len 40960
362 112/300 Read block 1402 len 40960
363 113/300 Read block 3680 len 40960
364 114/300 Read block 1043 len 20480
365 115/300 Read block 1875 len 4096
366 116/300 Write block 2939 len 4096 data: 2
367 117/300 Flush
368 118/300 Read block 4080 len 12288
369 119/300 Read block 1428 len 40960
370 120/300 Read block 3543 len 28672
371 121/300 Read block 804 len 4096
372 122/300 Read block 3425 len 12288
373 123/300 Read block 4885 len 36864
374 124/300 Flush
375 125/300 Flush
376 126/300 Write block 3479 len 36864 data: 2 2 2 2 2 2 2 2 2
377 127/300 Read block 3486 len 28672
378 128/300 Read block 4388 len 24576
379 129/300 Read block 547 len 12288
380 130/300 Read block 913 len 28672
381 131/300 Write block 669 len 32768 data: 2 2 2 2 2 2 2 2
382 132/300 Flush
383 133/300 Write block 3864 len 16384 data: 2 2 2 3
384 134/300 Write block 2022 len 36864 data: 2 2 2 3 3 3 3 2 2
385 135/300 Read block 1946 len 4096
386 136/300 Read block 1657 len 36864
387 137/300 Write block 419 len 40960 data: 2 2 2 2 2 2 2 2 2 2
388 138/300 Write block 3926 len 28672 data: 2 2 2 2 2 2 2
389 139/300 Flush
390 140/300 Write block 2893 len 40960 data: 2 2 2 2 2 2 2 2 2 2
391 141/300 Read block 3736 len 24576
392 142/300 Write block 4763 len 36864 data: 2 2 2 2 2 2 2 2 2
393 143/300 Write block 2186 len 36864 data: 2 2 2 2 2 2 2 2 2
394 144/300 Read block 1057 len 16384
395 145/300 Read block 4706 len 8192
396 146/300 Write block 150 len 36864 data: 2 2 2 2 2 2 2 2 2
397 147/300 Read block 4786 len 16384
398 148/300 Read block 998 len 28672
399 149/300 Write block 4354 len 40960 data: 2 2 2 2 2 2 2 2 2 2
400 150/300 Read block 1876 len 24576
401 151/300 Read block 4034 len 4096
402 152/300 Read block 1986 len 8192
403 153/300 Read block 1768 len 8192
404 154/300 Read block 694 len 36864
405 155/300 Write block 25 len 16384 data: 2 2 2 2
406 156/300 Read block 4937 len 28672
407 157/300 Write block 3509 len 28672 data: 2 2 2 2 2 2 2
408 158/300 Read block 3189 len 8192
409 159/300 Write block 3415 len 8192 data: 2 2
410 160/300 Read block 1598 len 36864
411 161/300 Write block 4491 len 12288 data: 2 2 2
412 162/300 Read block 1179 len 20480
413 163/300 Flush
414 164/300 Read block 878 len 32768
415 165/300 Read block 73 len 20480
416 166/300 Read block 3717 len 24576
417 167/300 Read block 3920 len 8192
418 168/300 Read block 4899 len 4096
419 169/300 Flush
420 170/300 Read block 1631 len 16384
421 171/300 Read block 2625 len 32768
422 172/300 Write block 3034 len 4096 data: 2
423 173/300 Write block 4327 len 36864 data: 2 2 2 2 2 2 2 2 2
424 174/300 Read block 628 len 16384
425 175/300 Write block 179 len 12288 data: 2 2 2
426 176/300 Read block 600 len 40960
427 177/300 Write block 429 len 32768 data: 2 2 2 2 2 2 2 2
428 178/300 Write block 552 len 8192 data: 2 2
429 179/300 Read block 3580 len 24576
430 180/300 Read block 3134 len 36864
431 181/300 Write block 3383 len 40960 data: 2 2 2 2 2 2 2 2 2 2
432 182/300 Read block 4617 len 24576
433 183/300 Read block 1163 len 16384
434 184/300 Read block 4444 len 4096
435 185/300 Write block 2218 len 20480 data: 2 2 2 2 2
436 186/300 Write block 395 len 28672 data: 3 3 3 3 3 3 3
437 187/300 Write block 4037 len 12288 data: 2 2 2
438 188/300 Write block 353 len 36864 data: 2 2 2 2 2 2 2 2 2
439 189/300 Read block 1853 len 20480
440 190/300 Write block 2305 len 8192 data: 2 2
441 191/300 Write block 697 len 8192 data: 2 2
442 192/300 Write block 2774 len 12288 data: 2 2 2
443 193/300 Read block 4489 len 12288
444 194/300 Write block 2673 len 8192 data: 2 2
445 195/300 Read block 1705 len 32768
446 196/300 Write block 3895 len 16384 data: 2 2 2 2
447 197/300 Flush
448 198/300 Write block 4011 len 12288 data: 2 2 2
449 199/300 Flush
450 200/300 Write block 1230 len 32768 data: 2 2 2 2 2 2 2 2
451 201/300 Write block 667 len 12288 data: 2 2 3
452 202/300 Read block 595 len 12288
453 203/300 Read block 2911 len 28672
454 204/300 Read block 3803 len 20480
455 205/300 Write block 3945 len 32768 data: 2 2 2 2 2 2 2 2
456 206/300 Write block 2771 len 8192 data: 2 2
457 207/300 Write block 2026 len 4096 data: 4
458 208/300 Write block 4241 len 8192 data: 2 2
459 209/300 Write block 4661 len 20480 data: 3 3 3 2 2
460 210/300 Write block 2878 len 12288 data: 2 2 2
461 211/300 Read block 1008 len 36864
462 212/300 Read block 4989 len 32768
463 213/300 Write block 2773 len 16384 data: 2 3 3 3
464 214/300 Write block 527 len 8192 data: 2 2
465 215/300 Write block 1287 len 20480 data: 2 2 2 2 2
466 216/300 Read block 2083 len 8192
467 217/300 Write block 1399 len 16384 data: 2 2 2 2
468 218/300 Read block 1263 len 12288
469 219/300 Write block 2350 len 28672 data: 2 2 2 2 2 2 2
470 220/300 Read block 1711 len 40960
471 221/300 Write block 1117 len 28672 data: 2 2 2 2 2 2 2
472 222/300 Write block 3267 len 12288 data: 2 2 2
473 223/300 Read block 4524 len 24576
474 224/300 Read block 833 len 32768
475 225/300 Read block 1028 len 20480
476 226/300 Write block 173 len 36864 data: 2 2 2 2 2 2 3 3 3
477 227/300 Read block 2856 len 12288
478 228/300 Read block 1842 len 40960
479 229/300 Read block 1673 len 40960
480 230/300 Flush
481 231/300 Flush
482 232/300 Write block 2412 len 20480 data: 2 2 2 2 2
483 233/300 Write block 3386 len 32768 data: 3 3 3 3 3 3 3 2
484 234/300 Read block 2314 len 12288
485 235/300 Read block 4476 len 16384
486 236/300 Write block 787 len 40960 data: 2 2 2 2 2 2 2 2 2 2
487 237/300 Write block 2051 len 4096 data: 2
488 238/300 Read block 71 len 40960
489 239/300 Read block 55 len 8192
490 240/300 Write block 3003 len 12288 data: 2 2 2
491 241/300 Read block 1787 len 20480
492 242/300 Read block 4532 len 4096
493 243/300 Read block 1864 len 4096
494 244/300 Read block 4194 len 32768
495 245/300 Write block 1216 len 24576 data: 2 2 2 2 2 2
496 246/300 Read block 328 len 12288
497 247/300 Flush
498 248/300 Read block 4810 len 32768
499 249/300 Read block 3872 len 20480
500 250/300 Write block 1667 len 36864 data: 2 2 2 2 2 2 2 2 2
501 251/300 Read block 3156 len 24576
502 252/300 Read block 2088 len 20480
503 253/300 Write block 3671 len 20480 data: 2 2 2 2 2
504 254/300 Read block 1091 len 24576
505 255/300 Read block 1039 len 4096
506 256/300 Read block 2875 len 20480
507 257/300 Write block 1163 len 36864 data: 2 2 2 2 2 2 2 2 2
508 258/300 Flush
509 259/300 Write block 2167 len 36864 data: 2 2 2 2 2 2 2 2 2
510 260/300 Read block 3605 len 20480
511 261/300 Read block 4414 len 32768
512 262/300 Read block 160 len 28672
513 263/300 Flush
514 264/300 Flush
515 265/300 Read block 1715 len 28672
516 266/300 Write block 4432 len 36864 data: 2 2 2 2 2 2 2 2 2
517 267/300 Flush
518 268/300 Write block 4664 len 28672 data: 3 3 2 2 2 2 2
519 269/300 Write block 6 len 20480 data: 2 2 2 2 2
520 270/300 Read block 3276 len 20480
521 271/300 Read block 3264 len 28672
522 272/300 Read block 4568 len 32768
523 273/300 Read block 801 len 4096
524 274/300 Write block 364 len 20480 data: 2 2 2 2 2
525 275/300 Write block 251 len 32768 data: 2 2 2 2 2 2 2 2
526 276/300 Write block 330 len 8192 data: 2 2
527 277/300 Read block 690 len 20480
528 278/300 Read block 397 len 20480
529 279/300 Read block 1106 len 12288
530 280/300 Write block 327 len 8192 data: 2 2
531 281/300 Read block 4103 len 32768
532 282/300 Write block 916 len 32768 data: 2 2 2 2 2 2 2 2
533 283/300 Read block 532 len 20480
534 284/300 Write block 2691 len 28672 data: 2 2 2 2 2 2 2
535 285/300 Write block 4072 len 16384 data: 2 2 2 2
536 286/300 Write block 1191 len 4096 data: 2
537 287/300 Read block 1220 len 8192
538 288/300 Read block 4981 len 16384
539 289/300 Write block 1144 len 32768 data: 2 2 2 2 2 2 2 2
540 290/300 Write block 596 len 28672 data: 2 2 2 2 2 2 2
541 291/300 Write block 359 len 40960 data: 3 3 3 2 2 3 3 3 3 3
542 292/300 Read block 3410 len 24576
543 293/300 Write block 4940 len 40960 data: 2 2 2 2 2 2 2 2 2 2
544 294/300 Write block 3372 len 4096 data: 2
545 295/300 Write block 4575 len 16384 data: 2 2 2 2
546 296/300 Write block 4355 len 12288 data: 3 3 3
547 297/300 Read block 4326 len 20480
548 298/300 Read block 3480 len 4096
549 299/300 Write block 2723 len 28672 data: 2 2 2 2 2 2 2
550 300/300 Write block 2728 len 24576 data: 3 3 2 2 2 2
551 301/300 Write block 3437 len 36864 data: 2 2 2 2 2 2 2 2 2
552 Main task has work to do, go find it
553 got dsc Start(1)
554 start 1
555 Nov 19 17:08:07.957 INFO request completed, latency_us: 246, response_code: 204, uri: /start/cid/1, method: POST, req_id: 542eac12-d867-43b1-97cd-4d63b0bd2b2d, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
556 Replay: started 1, returned:Ok(())
557 ----------------------------------------------------------------
558 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
559 States: Active Offline Active
560 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
561 1000 Acked Read 1 Done Sent Done false
562 1001 Acked Flush 0 Done Sent Done false
563 1002 Acked Write 5 Done Sent Done false
564 1003 Acked Read 3 Done Sent Done false
565 1004 Acked Write 3 Done Sent Done false
566 1005 Acked Flush 0 Done Sent Done false
567 1006 Acked Write 10 Done Sent Done false
568 1007 Acked Write 9 Done Sent Done false
569 1008 Acked Read 2 Done Sent Done false
570 1009 Acked Read 1 Done Sent Done false
571 1010 Acked Write 3 Done Sent Done false
572 1011 Acked Flush 0 Done Sent Done false
573 1012 Acked Write 8 Done Sent Done false
574 1013 Acked Read 1 Done Sent Done false
575 1014 Acked Read 7 Done Sent Done false
576 1015 Acked Read 10 Done Sent Done false
577 1016 Acked Write 2 Done Sent Done false
578 1017 Acked Write 8 Done Sent Done false
579 1018 Acked Read 7 Done Sent Done false
580 1019 Acked Read 2 Done Sent Done false
581 1020 Acked Write 2 Done Sent Done false
582 1021 Acked Read 2 Done Sent Done false
583 1022 Acked Read 10 Done Sent Done false
584 1023 Acked Read 6 Done Sent Done false
585 1024 Acked Write 3 Done Sent Done false
586 1025 Acked Write 6 Done Sent Done false
587 1026 Acked Write 9 Done Sent Done false
588 1027 Acked Read 4 Done Sent Done false
589 1028 Acked Write 9 Done Sent Done false
590 1029 Acked Read 2 Done Sent Done false
591 1030 Acked Read 9 Done Sent Done false
592 1031 Acked Read 5 Done Sent Done false
593 1032 Acked Read 4 Done Sent Done false
594 1033 Acked Write 10 Done Sent Done false
595 1034 Acked Read 2 Done Sent Done false
596 1035 Acked Write 9 Done Sent Done false
597 1036 Acked Read 3 Done Sent Done false
598 1037 Acked Write 1 Done Sent Done false
599 1038 Acked Write 10 Done Sent Done false
600 1039 Acked Write 4 Done Sent Done false
601 1040 Acked Write 9 Done Sent Done false
602 1041 Acked Write 8 Done Sent Done false
603 1042 Acked Read 5 Done Sent Done false
604 1043 Acked Read 3 Done Sent Done false
605 1044 Acked Read 5 Done Sent Done false
606 1045 Acked Read 10 Done Sent Done false
607 1046 Acked Write 5 Done Sent Done false
608 1047 Acked Write 9 Done Sent Done false
609 1048 Acked Write 1 Done Sent Done false
610 1049 Acked Read 6 Done Sent Done false
611 1050 Acked Read 2 Done Sent Done false
612 1051 Acked Read 5 Done Sent Done false
613 1052 Acked Read 10 Done Sent Done false
614 1053 Acked Read 10 Done Sent Done false
615 1054 Acked Write 4 Done Sent Done false
616 1055 Acked Flush 0 Done Sent Done false
617 1056 Acked Write 2 Done Sent Done false
618 1057 Acked Read 4 Done Sent Done false
619 1058 Acked Write 9 Done Sent Done false
620 1059 Acked Write 4 Done Sent Done false
621 1060 Acked Read 2 Done Sent Done false
622 1061 Acked Read 7 Done Sent Done false
623 1062 Acked Read 6 Done Sent Done false
624 1063 Acked Write 10 Done Sent Done false
625 1064 Acked Read 10 Done Sent Done false
626 1065 Acked Write 8 Done Sent Done false
627 1066 Acked Read 10 Done Sent Done false
628 1067 Acked Read 7 Done Sent Done false
629 1068 Acked Write 9 Done Sent Done false
630 1069 Acked Flush 0 Done Sent Done false
631 1070 Acked Write 4 Done Sent Done false
632 1071 Acked Write 7 Done Sent Done false
633 1072 Acked Write 5 Done Sent Done false
634 1073 Acked Write 10 Done Sent Done false
635 1074 Acked Write 6 Done Sent Done false
636 1075 Acked Read 1 Done Sent Done false
637 1076 Acked Write 7 Done Sent Done false
638 1077 Acked Read 5 Done Sent Done false
639 1078 Acked Write 9 Done Sent Done false
640 1079 Acked Write 8 Done Sent Done false
641 1080 Acked Write 3 Done Sent Done false
642 1081 Acked Read 3 Done Sent Done false
643 1082 Acked Write 10 Done Sent Done false
644 1083 Acked Read 6 Done Sent Done false
645 1084 Acked Read 6 Done Sent Done false
646 1085 Acked Write 4 Done Sent Done false
647 1086 Acked Write 2 Done Sent Done false
648 1087 Acked Read 7 Done Sent Done false
649 1088 Acked Write 10 Done Sent Done false
650 1089 Acked Write 10 Done Sent Done false
651 1090 Acked Flush 0 Done Sent Done false
652 1091 Acked Read 2 Done Sent Done false
653 1092 Acked Write 8 Done Sent Done false
654 1093 Acked Write 8 Done Sent Done false
655 1094 Acked Read 9 Done Sent Done false
656 1095 Acked Flush 0 Done Sent Done false
657 1096 Acked Read 9 Done Sent Done false
658 1097 Acked Write 4 Done Sent Done false
659 1098 Acked Read 5 Done Sent Done false
660 1099 Acked Read 6 Done Sent Done false
661 1100 Acked Read 6 Done Sent Done false
662 1101 Acked Read 1 Done Sent Done false
663 1102 Acked Read 4 Done Sent Done false
664 1103 Acked Write 5 Done Sent Done false
665 1104 Acked Write 10 Done Sent Done false
666 1105 Acked Write 4 Done Sent Done false
667 1106 Acked Read 2 Done Sent Done false
668 1107 Acked Read 1 Done Sent Done false
669 1108 Acked Read 4 Done Sent Done false
670 1109 Acked Read 7 Done Sent Done false
671 1110 Acked Read 10 Done Sent Done false
672 1111 Acked Read 10 Done Sent Done false
673 1112 Acked Read 10 Done Sent Done false
674 1113 Acked Read 5 Done Sent Done false
675 1114 Acked Read 1 Done Sent Done false
676 1115 Acked Write 1 Done Sent Done false
677 1116 Acked Flush 0 Done Sent Done false
678 1117 Acked Read 3 Done Sent Done false
679 1118 Acked Read 10 Done Sent Done false
680 1119 Acked Read 7 Done Sent Done false
681 1120 Acked Read 1 Done Sent Done false
682 1121 Acked Read 3 Done Sent Done false
683 1122 Acked Read 9 Done Sent Done false
684 1123 Acked Flush 0 Done Sent Done false
685 1124 Acked Flush 0 Done Sent Done false
686 1125 Acked Write 9 Done Sent Done false
687 1126 Acked Read 7 Done Sent Done false
688 1127 Acked Read 6 Done Sent Done false
689 1128 Acked Read 3 Done Sent Done false
690 1129 Acked Read 7 Done Sent Done false
691 1130 Acked Write 8 Done Sent Done false
692 1131 Acked Flush 0 Done Sent Done false
693 1132 Acked Write 4 Done Sent Done false
694 1133 Acked Write 9 Done Sent Done false
695 1134 Acked Read 1 Done Sent Done false
696 1135 Acked Read 9 Done Sent Done false
697 1136 Acked Write 10 Done Sent Done false
698 1137 Acked Write 7 Done Sent Done false
699 1138 Acked Flush 0 Done Sent Done false
700 1139 Acked Write 10 Done Sent Done false
701 1140 Acked Read 6 Done Sent Done false
702 1141 Acked Write 9 Done Sent Done false
703 1142 Acked Write 9 Done Sent Done false
704 1143 Acked Read 4 Done Sent Done false
705 1144 Acked Read 2 Done Sent Done false
706 1145 Acked Write 9 Done Sent Done false
707 1146 Acked Read 4 Done Sent Done false
708 1147 Acked Read 7 Done Sent Done false
709 1148 Acked Write 10 Done Sent Done false
710 1149 Acked Read 6 Done Sent Done false
711 1150 Acked Read 1 Done Sent Done false
712 1151 Acked Read 2 Done Sent Done false
713 1152 Acked Read 2 Done Sent Done false
714 1153 Acked Read 9 Done Sent Done false
715 1154 Acked Write 4 Done Sent Done false
716 1155 Acked Read 7 Done Sent Done false
717 1156 Acked Write 7 Done Sent Done false
718 1157 Acked Read 2 Done Sent Done false
719 1158 Acked Write 2 Done Sent Done false
720 1159 Acked Read 9 Done Sent Done false
721 1160 Acked Write 3 Done Sent Done false
722 1161 Acked Read 5 Done Sent Done false
723 1162 Acked Flush 0 Done Sent Done false
724 1163 Acked Read 8 Done Sent Done false
725 1164 Acked Read 5 Done Sent Done false
726 1165 Acked Read 6 Done Sent Done false
727 1166 Acked Read 2 Done Sent Done false
728 1167 Acked Read 1 Done Sent Done false
729 1168 Acked Flush 0 Done Sent Done false
730 1169 Acked Read 4 Done Sent Done false
731 1170 Acked Read 8 Done Sent Done false
732 1171 Acked Write 1 Done Sent Done false
733 1172 Acked Write 9 Done Sent Done false
734 1173 Acked Read 4 Done Sent Done false
735 1174 Acked Write 3 Done Sent Done false
736 1175 Acked Read 10 Done Sent Done false
737 1176 Acked Write 8 Done Sent Done false
738 1177 Acked Write 2 Done Sent Done false
739 1178 Acked Read 6 Done Sent Done false
740 1179 Acked Read 9 Done Sent Done false
741 1180 Acked Write 10 Done Sent Done false
742 1181 Acked Read 6 Done Sent Done false
743 1182 Acked Read 4 Done Sent Done false
744 1183 Acked Read 1 Done Sent Done false
745 1184 Acked Write 5 Done Sent Done false
746 1185 Acked Write 7 Done Sent Done false
747 1186 Acked Write 3 Done Sent Done false
748 1187 Acked Write 9 Done Sent Done false
749 1188 Acked Read 5 Done Sent Done false
750 1189 Acked Write 2 Done Sent Done false
751 1190 Acked Write 2 Done Sent Done false
752 1191 Acked Write 3 Done Sent Done false
753 1192 Acked Read 3 Done Sent Done false
754 1193 Acked Write 2 Done Sent Done false
755 1194 Acked Read 8 Done Sent Done false
756 1195 Acked Write 4 Done Sent Done false
757 1196 Acked Flush 0 Done Sent Done false
758 1197 Acked Write 3 Done Sent Done false
759 1198 Acked Flush 0 Done Sent Done false
760 1199 Acked Write 8 Done Sent Done false
761 1200 Acked Write 3 Done Sent Done false
762 1201 Acked Read 3 Done Sent Done false
763 1202 Acked Read 7 Done Sent Done false
764 1203 Acked Read 5 Done Sent Done false
765 1204 Acked Write 8 Done Sent Done false
766 1205 Acked Write 2 Done Sent Done false
767 1206 Acked Write 1 Done Sent Done false
768 1207 Acked Write 2 Done Sent Done false
769 1208 Acked Write 5 Done Sent Done false
770 1209 Acked Write 3 Done Sent Done false
771 1210 Acked Read 9 Done Sent Done false
772 1211 Acked Read 8 Done Sent Done false
773 1212 Acked Write 4 Done Sent Done false
774 1213 Acked Write 2 Done Sent Done false
775 1214 Acked Write 5 Done Sent Done false
776 1215 Acked Read 2 Done Sent Done false
777 1216 Acked Write 4 Done Sent Done false
778 1217 Acked Read 3 Done Sent Done false
779 1218 Acked Write 7 Done Sent Done false
780 1219 Acked Read 10 Done Sent Done false
781 1220 Acked Write 7 Done Sent Done false
782 1221 Acked Write 3 Done Sent Done false
783 1222 Acked Read 6 Done Sent Done false
784 1223 Acked Read 8 Done Sent Done false
785 1224 Acked Read 5 Done Sent Done false
786 1225 Acked Write 9 Done Sent Done false
787 1226 Acked Read 3 Done Sent Done false
788 1227 Acked Read 10 Done Sent Done false
789 1228 Acked Read 10 Done Sent Done false
790 1229 Acked Flush 0 Done Sent Done false
791 1230 Acked Flush 0 Done Sent Done false
792 1231 Acked Write 5 Done Sent Done false
793 1232 Acked Write 8 Done Sent Done false
794 1233 Acked Read 3 Done Sent Done false
795 1234 Acked Read 4 Done Sent Done false
796 1235 Acked Write 10 Done Sent Done false
797 1236 Acked Write 1 Done Sent Done false
798 1237 Acked Read 10 Done Sent Done false
799 1238 Acked Read 2 Done Sent Done false
800 1239 Acked Write 3 Done Sent Done false
801 1240 Acked Read 5 Done Sent Done false
802 1241 Acked Read 1 Done Sent Done false
803 1242 Acked Read 1 Done Sent Done false
804 1243 Acked Read 8 Done Sent Done false
805 1244 Acked Write 6 Done Sent Done false
806 1245 Acked Read 3 Done Sent Done false
807 1246 Acked Flush 0 Done Sent Done false
808 1247 Acked Read 8 Done Sent Done false
809 1248 Acked Read 5 Done Sent Done false
810 1249 Acked Write 9 Done Sent Done false
811 1250 Acked Read 6 Done Sent Done false
812 1251 Acked Read 5 Done Sent Done false
813 1252 Acked Write 5 Done Sent Done false
814 1253 Acked Read 6 Done Sent Done false
815 1254 Acked Read 1 Done Sent Done false
816 1255 Acked Read 5 Done Sent Done false
817 1256 Acked Write 9 Done Sent Done false
818 1257 Acked Flush 0 Done Sent Done false
819 1258 Acked Write 9 Done Sent Done false
820 1259 Acked Read 5 Done Sent Done false
821 1260 Acked Read 8 Done Sent Done false
822 1261 Acked Read 7 Done Sent Done false
823 1262 Acked Flush 0 Done Sent Done false
824 1263 Acked Flush 0 Done Sent Done false
825 1264 Acked Read 7 Done Sent Done false
826 1265 Acked Write 9 Done Sent Done false
827 1266 Acked Flush 0 Done Sent Done false
828 1267 Acked Write 7 Done Sent Done false
829 1268 Acked Write 5 Done Sent Done false
830 1269 Acked Read 5 Done Sent Done false
831 1270 Acked Read 7 Done Sent Done false
832 1271 Acked Read 8 Done Sent Done false
833 1272 Acked Read 1 Done Sent Done false
834 1273 Acked Write 5 Done Sent Done false
835 1274 Acked Write 8 Done Sent Done false
836 1275 Acked Write 2 Done Sent Done false
837 1276 Acked Read 5 Done Sent Done false
838 1277 Acked Read 5 Done Sent Done false
839 1278 Acked Read 3 Done Sent Done false
840 1279 Acked Write 2 Done Sent Done false
841 1280 Acked Read 8 Done Sent Done false
842 1281 Acked Write 8 Done Sent Done false
843 1282 Acked Read 5 Done Sent Done false
844 1283 Acked Write 7 Done Sent Done false
845 1284 Acked Write 4 Done Sent Done false
846 1285 Acked Write 1 Done Sent Done false
847 1286 Acked Read 2 Done Sent Done false
848 1287 Acked Read 4 Done Sent Done false
849 1288 Acked Write 8 Done Sent Done false
850 1289 Acked Write 7 Done Sent Done false
851 1290 Acked Write 10 Done Sent Done false
852 1291 Acked Read 6 Done Sent Done false
853 1292 Acked Write 10 Done Sent Done false
854 1293 Acked Write 1 Done Sent Done false
855 1294 Acked Write 4 Done Sent Done false
856 1295 Acked Write 3 Done Sent Done false
857 1296 Acked Read 5 Done Sent Done false
858 1297 Acked Read 1 Done Sent Done false
859 1298 Acked Write 7 Done Sent Done false
860 1299 Acked Write 6 Done Sent Done false
861 1300 Acked Write 9 Done Sent Done false
862 STATES DS:0 DS:1 DS:2 TOTAL
863 Sent 0 301 0 301
864 Done 301 0 301 602
865 Skipped 0 0 0 0
866 Error 0 0 0 0
867 Last Flush: 1266 0 1266
868 Downstairs last five completed:
869 Upstairs last five completed: 1300 1299 1298 1297 1296
870 CLIENT: Up:0 ds:301 act:2
871 [8820] got start action
872 [8820] I am going to restart
873 Starting downstairs at port 8820
874 Make output file at "/tmp/dsc/downstairs-8820.txt"
875 [8820][1] reports Starting
876 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1203)
877 [8820][1] reports Running
878 ----------------------------------------------------------------
879 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
880 States: Active Offline Active
881 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
882 1000 Acked Read 1 Done Sent Done false
883 1001 Acked Flush 0 Done Sent Done false
884 1002 Acked Write 5 Done Sent Done false
885 1003 Acked Read 3 Done Sent Done false
886 1004 Acked Write 3 Done Sent Done false
887 1005 Acked Flush 0 Done Sent Done false
888 1006 Acked Write 10 Done Sent Done false
889 1007 Acked Write 9 Done Sent Done false
890 1008 Acked Read 2 Done Sent Done false
891 1009 Acked Read 1 Done Sent Done false
892 1010 Acked Write 3 Done Sent Done false
893 1011 Acked Flush 0 Done Sent Done false
894 1012 Acked Write 8 Done Sent Done false
895 1013 Acked Read 1 Done Sent Done false
896 1014 Acked Read 7 Done Sent Done false
897 1015 Acked Read 10 Done Sent Done false
898 1016 Acked Write 2 Done Sent Done false
899 1017 Acked Write 8 Done Sent Done false
900 1018 Acked Read 7 Done Sent Done false
901 1019 Acked Read 2 Done Sent Done false
902 1020 Acked Write 2 Done Sent Done false
903 1021 Acked Read 2 Done Sent Done false
904 1022 Acked Read 10 Done Sent Done false
905 1023 Acked Read 6 Done Sent Done false
906 1024 Acked Write 3 Done Sent Done false
907 1025 Acked Write 6 Done Sent Done false
908 1026 Acked Write 9 Done Sent Done false
909 1027 Acked Read 4 Done Sent Done false
910 1028 Acked Write 9 Done Sent Done false
911 1029 Acked Read 2 Done Sent Done false
912 1030 Acked Read 9 Done Sent Done false
913 1031 Acked Read 5 Done Sent Done false
914 1032 Acked Read 4 Done Sent Done false
915 1033 Acked Write 10 Done Sent Done false
916 1034 Acked Read 2 Done Sent Done false
917 1035 Acked Write 9 Done Sent Done false
918 1036 Acked Read 3 Done Sent Done false
919 1037 Acked Write 1 Done Sent Done false
920 1038 Acked Write 10 Done Sent Done false
921 1039 Acked Write 4 Done Sent Done false
922 1040 Acked Write 9 Done Sent Done false
923 1041 Acked Write 8 Done Sent Done false
924 1042 Acked Read 5 Done Sent Done false
925 1043 Acked Read 3 Done Sent Done false
926 1044 Acked Read 5 Done Sent Done false
927 1045 Acked Read 10 Done Sent Done false
928 1046 Acked Write 5 Done Sent Done false
929 1047 Acked Write 9 Done Sent Done false
930 1048 Acked Write 1 Done Sent Done false
931 1049 Acked Read 6 Done Sent Done false
932 1050 Acked Read 2 Done Sent Done false
933 1051 Acked Read 5 Done Sent Done false
934 1052 Acked Read 10 Done Sent Done false
935 1053 Acked Read 10 Done Sent Done false
936 1054 Acked Write 4 Done Sent Done false
937 1055 Acked Flush 0 Done Sent Done false
938 1056 Acked Write 2 Done Sent Done false
939 1057 Acked Read 4 Done Sent Done false
940 1058 Acked Write 9 Done Sent Done false
941 1059 Acked Write 4 Done Sent Done false
942 1060 Acked Read 2 Done Sent Done false
943 1061 Acked Read 7 Done Sent Done false
944 1062 Acked Read 6 Done Sent Done false
945 1063 Acked Write 10 Done Sent Done false
946 1064 Acked Read 10 Done Sent Done false
947 1065 Acked Write 8 Done Sent Done false
948 1066 Acked Read 10 Done Sent Done false
949 1067 Acked Read 7 Done Sent Done false
950 1068 Acked Write 9 Done Sent Done false
951 1069 Acked Flush 0 Done Sent Done false
952 1070 Acked Write 4 Done Sent Done false
953 1071 Acked Write 7 Done Sent Done false
954 1072 Acked Write 5 Done Sent Done false
955 1073 Acked Write 10 Done Sent Done false
956 1074 Acked Write 6 Done Sent Done false
957 1075 Acked Read 1 Done Sent Done false
958 1076 Acked Write 7 Done Sent Done false
959 1077 Acked Read 5 Done Sent Done false
960 1078 Acked Write 9 Done Sent Done false
961 1079 Acked Write 8 Done Sent Done false
962 1080 Acked Write 3 Done Sent Done false
963 1081 Acked Read 3 Done Sent Done false
964 1082 Acked Write 10 Done Sent Done false
965 1083 Acked Read 6 Done Sent Done false
966 1084 Acked Read 6 Done Sent Done false
967 1085 Acked Write 4 Done Sent Done false
968 1086 Acked Write 2 Done Sent Done false
969 1087 Acked Read 7 Done Sent Done false
970 1088 Acked Write 10 Done Sent Done false
971 1089 Acked Write 10 Done Sent Done false
972 1090 Acked Flush 0 Done Sent Done false
973 1091 Acked Read 2 Done Sent Done false
974 1092 Acked Write 8 Done Sent Done false
975 1093 Acked Write 8 Done Sent Done false
976 1094 Acked Read 9 Done Sent Done false
977 1095 Acked Flush 0 Done Sent Done false
978 1096 Acked Read 9 Done Sent Done false
979 1097 Acked Write 4 Done Sent Done false
980 1098 Acked Read 5 Done Sent Done false
981 1099 Acked Read 6 Done Sent Done false
982 1100 Acked Read 6 Done Sent Done false
983 1101 Acked Read 1 Done Sent Done false
984 1102 Acked Read 4 Done Sent Done false
985 1103 Acked Write 5 Done Sent Done false
986 1104 Acked Write 10 Done Sent Done false
987 1105 Acked Write 4 Done Sent Done false
988 1106 Acked Read 2 Done Sent Done false
989 1107 Acked Read 1 Done Sent Done false
990 1108 Acked Read 4 Done Sent Done false
991 1109 Acked Read 7 Done Sent Done false
992 1110 Acked Read 10 Done Sent Done false
993 1111 Acked Read 10 Done Sent Done false
994 1112 Acked Read 10 Done Sent Done false
995 1113 Acked Read 5 Done Sent Done false
996 1114 Acked Read 1 Done Sent Done false
997 1115 Acked Write 1 Done Sent Done false
998 1116 Acked Flush 0 Done Sent Done false
999 1117 Acked Read 3 Done Sent Done false
1000 1118 Acked Read 10 Done Sent Done false
1001 1119 Acked Read 7 Done Sent Done false
1002 1120 Acked Read 1 Done Sent Done false
1003 1121 Acked Read 3 Done Sent Done false
1004 1122 Acked Read 9 Done Sent Done false
1005 1123 Acked Flush 0 Done Sent Done false
1006 1124 Acked Flush 0 Done Sent Done false
1007 1125 Acked Write 9 Done Sent Done false
1008 1126 Acked Read 7 Done Sent Done false
1009 1127 Acked Read 6 Done Sent Done false
1010 1128 Acked Read 3 Done Sent Done false
1011 1129 Acked Read 7 Done Sent Done false
1012 1130 Acked Write 8 Done Sent Done false
1013 1131 Acked Flush 0 Done Sent Done false
1014 1132 Acked Write 4 Done Sent Done false
1015 1133 Acked Write 9 Done Sent Done false
1016 1134 Acked Read 1 Done Sent Done false
1017 1135 Acked Read 9 Done Sent Done false
1018 1136 Acked Write 10 Done Sent Done false
1019 1137 Acked Write 7 Done Sent Done false
1020 1138 Acked Flush 0 Done Sent Done false
1021 1139 Acked Write 10 Done Sent Done false
1022 1140 Acked Read 6 Done Sent Done false
1023 1141 Acked Write 9 Done Sent Done false
1024 1142 Acked Write 9 Done Sent Done false
1025 1143 Acked Read 4 Done Sent Done false
1026 1144 Acked Read 2 Done Sent Done false
1027 1145 Acked Write 9 Done Sent Done false
1028 1146 Acked Read 4 Done Sent Done false
1029 1147 Acked Read 7 Done Sent Done false
1030 1148 Acked Write 10 Done Sent Done false
1031 1149 Acked Read 6 Done Sent Done false
1032 1150 Acked Read 1 Done Sent Done false
1033 1151 Acked Read 2 Done Sent Done false
1034 1152 Acked Read 2 Done Sent Done false
1035 1153 Acked Read 9 Done Sent Done false
1036 1154 Acked Write 4 Done Sent Done false
1037 1155 Acked Read 7 Done Sent Done false
1038 1156 Acked Write 7 Done Sent Done false
1039 1157 Acked Read 2 Done Sent Done false
1040 1158 Acked Write 2 Done Sent Done false
1041 1159 Acked Read 9 Done Sent Done false
1042 1160 Acked Write 3 Done Sent Done false
1043 1161 Acked Read 5 Done Sent Done false
1044 1162 Acked Flush 0 Done Sent Done false
1045 1163 Acked Read 8 Done Sent Done false
1046 1164 Acked Read 5 Done Sent Done false
1047 1165 Acked Read 6 Done Sent Done false
1048 1166 Acked Read 2 Done Sent Done false
1049 1167 Acked Read 1 Done Sent Done false
1050 1168 Acked Flush 0 Done Sent Done false
1051 1169 Acked Read 4 Done Sent Done false
1052 1170 Acked Read 8 Done Sent Done false
1053 1171 Acked Write 1 Done Sent Done false
1054 1172 Acked Write 9 Done Sent Done false
1055 1173 Acked Read 4 Done Sent Done false
1056 1174 Acked Write 3 Done Sent Done false
1057 1175 Acked Read 10 Done Sent Done false
1058 1176 Acked Write 8 Done Sent Done false
1059 1177 Acked Write 2 Done Sent Done false
1060 1178 Acked Read 6 Done Sent Done false
1061 1179 Acked Read 9 Done Sent Done false
1062 1180 Acked Write 10 Done Sent Done false
1063 1181 Acked Read 6 Done Sent Done false
1064 1182 Acked Read 4 Done Sent Done false
1065 1183 Acked Read 1 Done Sent Done false
1066 1184 Acked Write 5 Done Sent Done false
1067 1185 Acked Write 7 Done Sent Done false
1068 1186 Acked Write 3 Done Sent Done false
1069 1187 Acked Write 9 Done Sent Done false
1070 1188 Acked Read 5 Done Sent Done false
1071 1189 Acked Write 2 Done Sent Done false
1072 1190 Acked Write 2 Done Sent Done false
1073 1191 Acked Write 3 Done Sent Done false
1074 1192 Acked Read 3 Done Sent Done false
1075 1193 Acked Write 2 Done Sent Done false
1076 1194 Acked Read 8 Done Sent Done false
1077 1195 Acked Write 4 Done Sent Done false
1078 1196 Acked Flush 0 Done Sent Done false
1079 1197 Acked Write 3 Done Sent Done false
1080 1198 Acked Flush 0 Done Sent Done false
1081 1199 Acked Write 8 Done Sent Done false
1082 1200 Acked Write 3 Done Sent Done false
1083 1201 Acked Read 3 Done Sent Done false
1084 1202 Acked Read 7 Done Sent Done false
1085 1203 Acked Read 5 Done Sent Done false
1086 1204 Acked Write 8 Done Sent Done false
1087 1205 Acked Write 2 Done Sent Done false
1088 1206 Acked Write 1 Done Sent Done false
1089 1207 Acked Write 2 Done Sent Done false
1090 1208 Acked Write 5 Done Sent Done false
1091 1209 Acked Write 3 Done Sent Done false
1092 1210 Acked Read 9 Done Sent Done false
1093 1211 Acked Read 8 Done Sent Done false
1094 1212 Acked Write 4 Done Sent Done false
1095 1213 Acked Write 2 Done Sent Done false
1096 1214 Acked Write 5 Done Sent Done false
1097 1215 Acked Read 2 Done Sent Done false
1098 1216 Acked Write 4 Done Sent Done false
1099 1217 Acked Read 3 Done Sent Done false
1100 1218 Acked Write 7 Done Sent Done false
1101 1219 Acked Read 10 Done Sent Done false
1102 1220 Acked Write 7 Done Sent Done false
1103 1221 Acked Write 3 Done Sent Done false
1104 1222 Acked Read 6 Done Sent Done false
1105 1223 Acked Read 8 Done Sent Done false
1106 1224 Acked Read 5 Done Sent Done false
1107 1225 Acked Write 9 Done Sent Done false
1108 1226 Acked Read 3 Done Sent Done false
1109 1227 Acked Read 10 Done Sent Done false
1110 1228 Acked Read 10 Done Sent Done false
1111 1229 Acked Flush 0 Done Sent Done false
1112 1230 Acked Flush 0 Done Sent Done false
1113 1231 Acked Write 5 Done Sent Done false
1114 1232 Acked Write 8 Done Sent Done false
1115 1233 Acked Read 3 Done Sent Done false
1116 1234 Acked Read 4 Done Sent Done false
1117 1235 Acked Write 10 Done Sent Done false
1118 1236 Acked Write 1 Done Sent Done false
1119 1237 Acked Read 10 Done Sent Done false
1120 1238 Acked Read 2 Done Sent Done false
1121 1239 Acked Write 3 Done Sent Done false
1122 1240 Acked Read 5 Done Sent Done false
1123 1241 Acked Read 1 Done Sent Done false
1124 1242 Acked Read 1 Done Sent Done false
1125 1243 Acked Read 8 Done Sent Done false
1126 1244 Acked Write 6 Done Sent Done false
1127 1245 Acked Read 3 Done Sent Done false
1128 1246 Acked Flush 0 Done Sent Done false
1129 1247 Acked Read 8 Done Sent Done false
1130 1248 Acked Read 5 Done Sent Done false
1131 1249 Acked Write 9 Done Sent Done false
1132 1250 Acked Read 6 Done Sent Done false
1133 1251 Acked Read 5 Done Sent Done false
1134 1252 Acked Write 5 Done Sent Done false
1135 1253 Acked Read 6 Done Sent Done false
1136 1254 Acked Read 1 Done Sent Done false
1137 1255 Acked Read 5 Done Sent Done false
1138 1256 Acked Write 9 Done Sent Done false
1139 1257 Acked Flush 0 Done Sent Done false
1140 1258 Acked Write 9 Done Sent Done false
1141 1259 Acked Read 5 Done Sent Done false
1142 1260 Acked Read 8 Done Sent Done false
1143 1261 Acked Read 7 Done Sent Done false
1144 1262 Acked Flush 0 Done Sent Done false
1145 1263 Acked Flush 0 Done Sent Done false
1146 1264 Acked Read 7 Done Sent Done false
1147 1265 Acked Write 9 Done Sent Done false
1148 1266 Acked Flush 0 Done Sent Done false
1149 1267 Acked Write 7 Done Sent Done false
1150 1268 Acked Write 5 Done Sent Done false
1151 1269 Acked Read 5 Done Sent Done false
1152 1270 Acked Read 7 Done Sent Done false
1153 1271 Acked Read 8 Done Sent Done false
1154 1272 Acked Read 1 Done Sent Done false
1155 1273 Acked Write 5 Done Sent Done false
1156 1274 Acked Write 8 Done Sent Done false
1157 1275 Acked Write 2 Done Sent Done false
1158 1276 Acked Read 5 Done Sent Done false
1159 1277 Acked Read 5 Done Sent Done false
1160 1278 Acked Read 3 Done Sent Done false
1161 1279 Acked Write 2 Done Sent Done false
1162 1280 Acked Read 8 Done Sent Done false
1163 1281 Acked Write 8 Done Sent Done false
1164 1282 Acked Read 5 Done Sent Done false
1165 1283 Acked Write 7 Done Sent Done false
1166 1284 Acked Write 4 Done Sent Done false
1167 1285 Acked Write 1 Done Sent Done false
1168 1286 Acked Read 2 Done Sent Done false
1169 1287 Acked Read 4 Done Sent Done false
1170 1288 Acked Write 8 Done Sent Done false
1171 1289 Acked Write 7 Done Sent Done false
1172 1290 Acked Write 10 Done Sent Done false
1173 1291 Acked Read 6 Done Sent Done false
1174 1292 Acked Write 10 Done Sent Done false
1175 1293 Acked Write 1 Done Sent Done false
1176 1294 Acked Write 4 Done Sent Done false
1177 1295 Acked Write 3 Done Sent Done false
1178 1296 Acked Read 5 Done Sent Done false
1179 1297 Acked Read 1 Done Sent Done false
1180 1298 Acked Write 7 Done Sent Done false
1181 1299 Acked Write 6 Done Sent Done false
1182 1300 Acked Write 9 Done Sent Done false
1183 1301 Acked Flush 0 Done Sent Done false
1184 STATES DS:0 DS:1 DS:2 TOTAL
1185 Sent 0 302 0 302
1186 Done 302 0 302 604
1187 Skipped 0 0 0 0
1188 Error 0 0 0 0
1189 Last Flush: 1301 0 1301
1190 Downstairs last five completed:
1191 Upstairs last five completed: 1301 1300 1299 1298 1297
1192 CLIENT: Up:0 ds:302 act:2
11932024-11-19T17:08:13.718ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
11942024-11-19T17:08:13.718ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48124) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
11952024-11-19T17:08:13.719ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
11962024-11-19T17:08:13.720ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
11972024-11-19T17:08:13.720ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
11982024-11-19T17:08:13.720ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
11992024-11-19T17:08:13.720ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
12002024-11-19T17:08:13.720ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
12012024-11-19T17:08:13.720ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
12022024-11-19T17:08:13.720ZINFOcrucible: [1] client re-new 302 jobs since flush 0 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
12032024-11-19T17:08:13.720ZINFOcrucible: [1] Replayed 302 jobs since flush: 0 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
1204 ----------------------------------------------------------------
1205 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1206 Downstairs last five completed: 1301 1300 1299 1298 1297
1207 Upstairs last five completed: 1301 1300 1299 1298 1297
1208 CLIENT: Up:0 ds:0 act:3
1209 Replay: All jobs finished
1210 Main task has work to do, go find it
1211 got dsc Stop(0)
1212 stop 0
1213 Nov 19 17:08:15.962 INFO request completed, latency_us: 340, [8810] Got stop action so:false kr:false
1214 response_code: 204, uri: /stop/cid/0, method: POST, req_id: 05135d13-059f-419d-a9e3-175ec6a82106, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
12152024-11-19T17:08:15.963ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
12162024-11-19T17:08:15.963ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
12172024-11-19T17:08:15.963ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
12182024-11-19T17:08:15.963ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
12192024-11-19T17:08:15.963ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
1220 Nov 19 17:08:15.963 INFO request completed, latency_us: 195, response_code:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:08:15.963404556Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
1221 200, uri: /state/cid/0, method: GET, req_id: 24636996-ab06-4298-bf84-f605d1f6c685, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
1222 [8810] Exited with: ExitStatus(unix_wait_status(9))
1223 [8810][0] reports Exit
1224 Nov 19 17:08:19.966 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/0, method: GET, req_id: 4d39fa44-825e-4be7-882f-788106f6b97c, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
1225 001/300 Write block 1628 len 20480 data: 2 2 2 2 2
1226 002/300 Read block 3990 len 28672
1227 003/300 Read block 1050 len 28672
1228 004/300 Flush
1229 005/300 Read block 2316 len 4096
1230 006/300 Read block 2303 len 32768
1231 007/300 Write block 4769 len 40960 data: 3 3 3 2 2 2 2 2 2 2
1232 008/300 Read block 2724 len 8192
1233 009/300 Write block 133 len 12288 data: 2 2 2
1234 010/300 Write block 317 len 16384 data: 2 2 2 2
1235 011/300 Write block 2833 len 12288 data: 2 2 2
1236 012/300 Write block 1998 len 4096 data: 2
1237 013/300 Write block 3059 len 28672 data: 2 2 2 2 2 2 2
1238 014/300 Flush
1239 015/300 Write block 860 len 16384 data: 2 2 2 2
1240 016/300 Write block 4421 len 16384 data: 2 2 2 2
1241 017/300 Read block 912 len 20480
1242 018/300 Read block 593 len 32768
1243 019/300 Write block 601 len 16384 data: 3 3 2 2
1244 020/300 Read block 1787 len 12288
1245 021/300 Flush
1246 022/300 Read block 4899 len 16384
1247 023/300 Flush
1248 024/300 Read block 4019 len 36864
1249 025/300 Write block 1124 len 24576 data: 2 2 2 2 2 2
1250 026/300 Read block 1211 len 16384
1251 027/300 Read block 1680 len 36864
1252 028/300 Read block 3124 len 36864
1253 029/300 Flush
1254 030/300 Write block 3353 len 32768 data: 2 2 2 2 2 2 2 2
1255 031/300 Write block 4550 len 8192 data: 2 2
1256 032/300 Read block 4257 len 28672
1257 033/300 Read block 3156 len 20480
1258 034/300 Read block 4314 len 16384
1259 035/300 Write block 4894 len 28672 data: 2 2 2 2 2 2 2
1260 036/300 Read block 942 len 24576
1261 037/300 Read block 972 len 28672
1262 038/300 Read block 825 len 36864
1263 039/300 Flush
1264 040/300 Write block 2243 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1265 041/300 Write block 1863 len 32768 data: 2 2 2 2 2 2 2 2
1266 042/300 Write block 2632 len 8192 data: 3 3
1267 043/300 Read block 3300 len 36864
1268 044/300 Flush
1269 045/300 Read block 442 len 16384
1270 046/300 Write block 2304 len 16384 data: 2 3 3 2
1271 047/300 Write block 3475 len 20480 data: 2 2 2 2 3
1272 048/300 Write block 667 len 40960 data: 3 3 4 3 3 3 3 3 3 3
1273 049/300 Read block 1112 len 40960
1274 050/300 Flush
1275 051/300 Read block 4079 len 40960
1276 052/300 Write block 1776 len 20480 data: 2 2 2 2 2
1277 053/300 Read block 930 len 4096
1278 054/300 Read block 796 len 4096
1279 055/300 Read block 4010 len 32768
1280 056/300 Read block 4424 len 4096
1281 057/300 Read block 4420 len 24576
1282 058/300 Write block 153 len 24576 data: 3 3 3 3 3 3
1283 059/300 Write block 1034 len 20480 data: 2 2 2 2 2
1284 060/300 Read block 2898 len 36864
1285 061/300 Read block 1565 len 8192
1286 062/300 Read block 4699 len 16384
1287 063/300 Read block 3904 len 12288
1288 064/300 Read block 2970 len 32768
1289 065/300 Write block 3027 len 28672 data: 2 2 2 3 3 3 3
1290 066/300 Read block 2011 len 16384
1291 067/300 Read block 672 len 36864
1292 068/300 Read block 2642 len 28672
1293 069/300 Write block 4038 len 16384 data: 3 3 2 2
1294 070/300 Read block 4793 len 28672
1295 071/300 Read block 3347 len 32768
1296 072/300 Write block 218 len 28672 data: 2 2 2 2 2 2 2
1297 073/300 Write block 4590 len 32768 data: 2 2 2 2 2 2 2 2
1298 074/300 Read block 1938 len 16384
1299 075/300 Read block 134 len 4096
1300 076/300 Flush
1301 077/300 Read block 3142 len 24576
1302 078/300 Write block 4571 len 4096 data: 2
1303 079/300 Read block 235 len 28672
1304 080/300 Read block 577 len 16384
1305 081/300 Write block 1119 len 16384 data: 3 3 3 3
1306 082/300 Read block 4328 len 4096
1307 083/300 Flush
1308 084/300 Write block 2889 len 20480 data: 2 2 2 2 3
1309 085/300 Write block 4117 len 36864 data: 2 2 2 2 2 2 2 2 2
1310 086/300 Write block 340 len 4096 data: 2
1311 087/300 Read block 1934 len 32768
1312 088/300 Write block 3862 len 40960 data: 3 3 3 3 3 4 3 3 3 3
1313 089/300 Write block 2276 len 24576 data: 2 2 2 2 2 2
1314 090/300 Read block 1013 len 20480
1315 091/300 Write block 2612 len 8192 data: 2 2
1316 092/300 Read block 3312 len 40960
1317 093/300 Read block 3735 len 20480
1318 094/300 Read block 481 len 24576
1319 095/300 Write block 606 len 36864 data: 2 2 2 2 2 2 2 2 2
1320 096/300 Read block 2612 len 16384
1321 097/300 Read block 4229 len 24576
1322 098/300 Write block 2242 len 8192 data: 2 3
1323 099/300 Write block 1890 len 16384 data: 2 2 2 2
1324 100/300 Read block 4230 len 12288
1325 101/300 Flush
1326 102/300 Write block 2646 len 20480 data: 2 2 2 2 2
1327 103/300 Read block 654 len 8192
1328 104/300 Read block 448 len 16384
1329 105/300 Write block 2378 len 28672 data: 2 2 2 2 2 2 2
1330 106/300 Read block 557 len 28672
1331 107/300 Read block 1446 len 4096
1332 108/300 Write block 3063 len 4096 data: 3
1333 109/300 Read block 4849 len 16384
1334 110/300 Write block 3601 len 16384 data: 2 2 2 2
1335 111/300 Write block 943 len 24576 data: 2 2 2 2 2 2
1336 112/300 Write block 3564 len 16384 data: 2 2 2 2
1337 113/300 Read block 1980 len 16384
1338 114/300 Write block 1731 len 28672 data: 2 2 2 2 2 2 2
1339 115/300 Write block 2213 len 4096 data: 2
1340 116/300 Read block 1270 len 4096
1341 117/300 Write block 60 len 40960 data: 3 3 3 3 3 3 3 3 2 2
1342 118/300 Read block 4980 len 20480
1343 119/300 Flush
1344 120/300 Read block 3926 len 12288
1345 121/300 Write block 3451 len 28672 data: 2 2 2 2 2 2 2
1346 122/300 Read block 1324 len 28672
1347 123/300 Write block 3197 len 28672 data: 2 2 2 2 2 2 2
1348 124/300 Read block 864 len 24576
1349 125/300 Write block 4364 len 8192 data: 2 2
1350 126/300 Read block 80 len 36864
1351 127/300 Flush
1352 128/300 Read block 4540 len 28672
1353 129/300 Write block 4659 len 12288 data: 3 3 4
1354 130/300 Read block 3841 len 8192
1355 131/300 Read block 799 len 4096
1356 132/300 Read block 4104 len 28672
1357 133/300 Write block 3987 len 36864 data: 2 2 2 2 2 2 2 2 2
1358 134/300 Read block 10 len 8192
1359 135/300 Read block 1820 len 36864
1360 136/300 Write block 186 len 16384 data: 2 2 2 2
1361 137/300 Read block 4160 len 28672
1362 138/300 Write block 320 len 24576 data: 3 2 2 2 2 2
1363 139/300 Read block 1159 len 24576
1364 140/300 Write block 4378 len 24576 data: 2 2 2 2 2 2
1365 141/300 Read block 3162 len 4096
1366 142/300 Read block 308 len 8192
1367 143/300 Write block 1450 len 32768 data: 2 2 2 2 2 2 2 2
1368 144/300 Read block 2845 len 20480
1369 145/300 Read block 2170 len 36864
1370 146/300 Write block 4122 len 28672 data: 3 3 3 3 2 2 2
1371 147/300 Write block 941 len 12288 data: 2 2 3
1372 148/300 Read block 3997 len 36864
1373 149/300 Write block 2980 len 8192 data: 2 2
1374 150/300 Write block 1738 len 36864 data: 2 2 2 3 3 2 2 2 2
1375 151/300 Write block 4632 len 12288 data: 2 2 2
1376 152/300 Read block 4171 len 36864
1377 153/300 Write block 1566 len 28672 data: 2 2 2 2 2 2 2
1378 154/300 Read block 1138 len 40960
1379 155/300 Write block 2286 len 4096 data: 2
1380 156/300 Read block 4161 len 28672
1381 157/300 Write block 3997 len 36864 data: 2 2 2 2 2 2 2 2 2
1382 158/300 Read block 3555 len 24576
1383 159/300 Write block 2524 len 12288 data: 2 2 2
1384 160/300 Write block 1091 len 8192 data: 2 2
1385 161/300 Write block 3703 len 28672 data: 2 2 2 2 2 2 2
1386 162/300 Read block 2393 len 40960
1387 163/300 Read block 914 len 28672
1388 164/300 Flush
1389 165/300 Write block 139 len 20480 data: 2 2 2 2 2
1390 166/300 Read block 2337 len 28672
1391 167/300 Write block 4333 len 36864 data: 3 3 3 2 2 2 2 2 2
1392 168/300 Read block 1338 len 16384
1393 169/300 Write block 3223 len 24576 data: 2 2 2 2 2 2
1394 170/300 Read block 4291 len 32768
1395 171/300 Write block 1937 len 28672 data: 2 2 2 2 2 2 2
1396 172/300 Flush
1397 173/300 Write block 1974 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1398 174/300 Write block 3841 len 16384 data: 2 2 2 2
1399 175/300 Flush
1400 176/300 Write block 1877 len 12288 data: 2 3 3
1401 177/300 Write block 3619 len 20480 data: 2 2 2 2 2
1402 178/300 Read block 3087 len 40960
1403 179/300 Write block 1544 len 36864 data: 3 3 3 3 3 3 3 3 2
1404 180/300 Write block 3752 len 28672 data: 2 2 2 2 2 2 2
1405 181/300 Read block 3129 len 4096
1406 182/300 Read block 4364 len 28672
1407 183/300 Read block 4377 len 12288
1408 184/300 Write block 1946 len 36864 data: 2 2 2 2 2 2 2 2 2
1409 185/300 Read block 3430 len 4096
1410 186/300 Read block 4455 len 32768
1411 187/300 Write block 3304 len 24576 data: 2 2 2 2 2 2
1412 188/300 Read block 3333 len 40960
1413 189/300 Read block 3951 len 40960
1414 190/300 Read block 23 len 20480
1415 191/300 Flush
1416 192/300 Read block 1595 len 12288
1417 193/300 Write block 3736 len 36864 data: 2 2 2 2 2 2 2 2 2
1418 194/300 Write block 4648 len 20480 data: 2 2 2 2 2
1419 195/300 Write block 3799 len 16384 data: 3 3 3 2
1420 196/300 Read block 3882 len 4096
1421 197/300 Read block 2889 len 32768
1422 198/300 Read block 1786 len 40960
1423 199/300 Read block 3053 len 24576
1424 200/300 Write block 2095 len 4096 data: 2
1425 201/300 Read block 1920 len 12288
1426 202/300 Read block 3284 len 40960
1427 203/300 Read block 119 len 16384
1428 204/300 Read block 4611 len 20480
1429 205/300 Flush
1430 206/300 Flush
1431 207/300 Write block 3707 len 24576 data: 3 3 3 2 2 2
1432 208/300 Read block 1156 len 28672
1433 209/300 Read block 2476 len 28672
1434 210/300 Flush
1435 211/300 Write block 265 len 12288 data: 2 2 2
1436 212/300 Read block 271 len 16384
1437 213/300 Read block 413 len 8192
1438 214/300 Flush
1439 215/300 Read block 3992 len 36864
1440 216/300 Write block 3948 len 4096 data: 3
1441 217/300 Read block 2850 len 36864
1442 218/300 Read block 2858 len 32768
1443 219/300 Write block 30 len 28672 data: 2 2 2 2 2 2 2
1444 220/300 Write block 129 len 36864 data: 2 2 2 2 3 3 3 2 2
1445 221/300 Write block 2638 len 36864 data: 3 3 3 2 2 2 2 2 3
1446 222/300 Read block 347 len 16384
1447 223/300 Read block 3033 len 16384
1448 224/300 Read block 2332 len 24576
1449 225/300 Write block 3758 len 32768 data: 3 2 2 2 2 2 2 2
1450 226/300 Write block 3938 len 4096 data: 2
1451 227/300 Read block 141 len 4096
1452 228/300 Read block 4720 len 8192
1453 229/300 Write block 4776 len 28672 data: 3 3 3 2 2 2 2
1454 230/300 Write block 1006 len 20480 data: 2 2 2 2 2
1455 231/300 Read block 1842 len 40960
1456 232/300 Flush
1457 233/300 Read block 3250 len 36864
1458 234/300 Write block 2921 len 4096 data: 2
1459 235/300 Write block 1584 len 8192 data: 2 2
1460 236/300 Read block 3334 len 32768
1461 237/300 Write block 388 len 28672 data: 2 2 2 2 2 2 3
1462 238/300 Read block 2544 len 32768
1463 239/300 Read block 4627 len 12288
1464 240/300 Write block 2389 len 16384 data: 2 2 2 2
1465 241/300 Read block 3538 len 16384
1466 242/300 Read block 1403 len 12288
1467 243/300 Write block 2014 len 12288 data: 2 2 2
1468 244/300 Write block 1234 len 8192 data: 3 3
1469 245/300 Read block 415 len 16384
1470 246/300 Read block 3044 len 40960
1471 247/300 Read block 2721 len 32768
1472 248/300 Write block 2300 len 24576 data: 2 2 2 2 3 4
1473 249/300 Read block 492 len 4096
1474 250/300 Write block 1230 len 16384 data: 3 3 3 3
1475 251/300 Read block 3678 len 20480
1476 252/300 Read block 742 len 12288
1477 253/300 Read block 890 len 20480
1478 254/300 Flush
1479 255/300 Read block 4890 len 36864
1480 256/300 Read block 1633 len 20480
1481 257/300 Write block 4149 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1482 258/300 Read block 1506 len 8192
1483 259/300 Write block 1598 len 36864 data: 2 2 2 2 2 2 2 2 2
1484 260/300 Flush
1485 261/300 Read block 4314 len 12288
1486 262/300 Read block 2076 len 12288
1487 263/300 Read block 290 len 4096
1488 264/300 Read block 4740 len 12288
1489 265/300 Read block 1339 len 32768
1490 266/300 Read block 1574 len 8192
1491 267/300 Write block 275 len 28672 data: 2 2 2 2 2 2 2
1492 268/300 Read block 3951 len 12288
1493 269/300 Write block 215 len 8192 data: 2 2
1494 270/300 Write block 2602 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1495 271/300 Write block 2679 len 4096 data: 2
1496 272/300 Read block 4080 len 4096
1497 273/300 Read block 845 len 24576
1498 274/300 Write block 322 len 12288 data: 3 3 3
1499 275/300 Write block 135 len 40960 data: 4 3 3 2 3 3 3 3 3 2
1500 276/300 Read block 4688 len 20480
1501 277/300 Read block 1642 len 8192
1502 278/300 Read block 349 len 36864
1503 279/300 Read block 4033 len 12288
1504 280/300 Read block 695 len 32768
1505 281/300 Write block 723 len 24576 data: 2 2 2 2 2 2
1506 282/300 Write block 2385 len 40960 data: 2 2 2 2 3 3 3 3 2 2
1507 283/300 Read block 4608 len 36864
1508 284/300 Read block 3202 len 24576
1509 285/300 Read block 488 len 12288
1510 286/300 Write block 4909 len 28672 data: 2 2 2 2 2 2 2
1511 287/300 Read block 903 len 4096
1512 288/300 Read block 1313 len 36864
1513 289/300 Write block 385 len 40960 data: 2 2 2 3 3 3 3 3 3 4
1514 290/300 Read block 3074 len 32768
1515 291/300 Write block 3363 len 4096 data: 2
1516 292/300 Write block 2106 len 4096 data: 3
1517 293/300 Read block 1144 len 28672
1518 294/300 Write block 1173 len 4096 data: 2
1519 295/300 Flush
1520 296/300 Read block 1148 len 8192
1521 297/300 Write block 2328 len 16384 data: 2 2 2 2
1522 298/300 Write block 991 len 12288 data: 2 2 2
1523 299/300 Write block 3800 len 28672 data: 4 4 3 2 2 2 2
1524 300/300 Read block 3762 len 36864
1525 301/300 Read block 1863 len 12288
1526 Main task has work to do, go find it
1527 got dsc Start(0)
1528 start 0
1529 Nov 19 17:08:20.203 INFO request completed, latency_us: 189, response_code: 204, uri: /start/cid/0, method: POST, req_id: b3dbb27b-79fd-4d41-b49d-4d0994bc4315, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
1530 Replay: started 0, returned:Ok(())
1531 ----------------------------------------------------------------
1532 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1533 States: Offline Active Active
1534 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1535 1302 Acked Write 5 Sent Done Done false
1536 1303 Acked Read 7 Sent Done Done false
1537 1304 Acked Read 7 Sent Done Done false
1538 1305 Acked Flush 0 Sent Done Done false
1539 1306 Acked Read 1 Sent Done Done false
1540 1307 Acked Read 8 Sent Done Done false
1541 1308 Acked Write 10 Sent Done Done false
1542 1309 Acked Read 2 Sent Done Done false
1543 1310 Acked Write 3 Sent Done Done false
1544 1311 Acked Write 4 Sent Done Done false
1545 1312 Acked Write 3 Sent Done Done false
1546 1313 Acked Write 1 Sent Done Done false
1547 1314 Acked Write 7 Sent Done Done false
1548 1315 Acked Flush 0 Sent Done Done false
1549 1316 Acked Write 4 Sent Done Done false
1550 1317 Acked Write 4 Sent Done Done false
1551 1318 Acked Read 5 Sent Done Done false
1552 1319 Acked Read 8 Sent Done Done false
1553 1320 Acked Write 4 Sent Done Done false
1554 1321 Acked Read 3 Sent Done Done false
1555 1322 Acked Flush 0 Sent Done Done false
1556 1323 Acked Read 4 Sent Done Done false
1557 1324 Acked Flush 0 Sent Done Done false
1558 1325 Acked Read 9 Sent Done Done false
1559 1326 Acked Write 6 Sent Done Done false
1560 1327 Acked Read 4 Sent Done Done false
1561 1328 Acked Read 9 Sent Done Done false
1562 1329 Acked Read 9 Sent Done Done false
1563 1330 Acked Flush 0 Sent Done Done false
1564 1331 Acked Write 8 Sent Done Done false
1565 1332 Acked Write 2 Sent Done Done false
1566 1333 Acked Read 7 Sent Done Done false
1567 1334 Acked Read 5 Sent Done Done false
1568 1335 Acked Read 4 Sent Done Done false
1569 1336 Acked Write 7 Sent Done Done false
1570 1337 Acked Read 6 Sent Done Done false
1571 1338 Acked Read 7 Sent Done Done false
1572 1339 Acked Read 9 Sent Done Done false
1573 1340 Acked Flush 0 Sent Done Done false
1574 1341 Acked Write 10 Sent Done Done false
1575 1342 Acked Write 8 Sent Done Done false
1576 1343 Acked Write 2 Sent Done Done false
1577 1344 Acked Read 9 Sent Done Done false
1578 1345 Acked Flush 0 Sent Done Done false
1579 1346 Acked Read 4 Sent Done Done false
1580 1347 Acked Write 4 Sent Done Done false
1581 1348 Acked Write 5 Sent Done Done false
1582 1349 Acked Write 10 Sent Done Done false
1583 1350 Acked Read 10 Sent Done Done false
1584 1351 Acked Flush 0 Sent Done Done false
1585 1352 Acked Read 10 Sent Done Done false
1586 1353 Acked Write 5 Sent Done Done false
1587 1354 Acked Read 1 Sent Done Done false
1588 1355 Acked Read 1 Sent Done Done false
1589 1356 Acked Read 8 Sent Done Done false
1590 1357 Acked Read 1 Sent Done Done false
1591 1358 Acked Read 6 Sent Done Done false
1592 1359 Acked Write 6 Sent Done Done false
1593 1360 Acked Write 5 Sent Done Done false
1594 1361 Acked Read 9 Sent Done Done false
1595 1362 Acked Read 2 Sent Done Done false
1596 1363 Acked Read 4 Sent Done Done false
1597 1364 Acked Read 3 Sent Done Done false
1598 1365 Acked Read 8 Sent Done Done false
1599 1366 Acked Write 7 Sent Done Done false
1600 1367 Acked Read 4 Sent Done Done false
1601 1368 Acked Read 9 Sent Done Done false
1602 1369 Acked Read 7 Sent Done Done false
1603 1370 Acked Write 4 Sent Done Done false
1604 1371 Acked Read 7 Sent Done Done false
1605 1372 Acked Read 8 Sent Done Done false
1606 1373 Acked Write 7 Sent Done Done false
1607 1374 Acked Write 8 Sent Done Done false
1608 1375 Acked Read 4 Sent Done Done false
1609 1376 Acked Read 1 Sent Done Done false
1610 1377 Acked Flush 0 Sent Done Done false
1611 1378 Acked Read 6 Sent Done Done false
1612 1379 Acked Write 1 Sent Done Done false
1613 1380 Acked Read 7 Sent Done Done false
1614 1381 Acked Read 4 Sent Done Done false
1615 1382 Acked Write 4 Sent Done Done false
1616 1383 Acked Read 1 Sent Done Done false
1617 1384 Acked Flush 0 Sent Done Done false
1618 1385 Acked Write 5 Sent Done Done false
1619 1386 Acked Write 9 Sent Done Done false
1620 1387 Acked Write 1 Sent Done Done false
1621 1388 Acked Read 8 Sent Done Done false
1622 1389 Acked Write 10 Sent Done Done false
1623 1390 Acked Write 6 Sent Done Done false
1624 1391 Acked Read 5 Sent Done Done false
1625 1392 Acked Write 2 Sent Done Done false
1626 1393 Acked Read 10 Sent Done Done false
1627 1394 Acked Read 5 Sent Done Done false
1628 1395 Acked Read 6 Sent Done Done false
1629 1396 Acked Write 9 Sent Done Done false
1630 1397 Acked Read 4 Sent Done Done false
1631 1398 Acked Read 6 Sent Done Done false
1632 1399 Acked Write 2 Sent Done Done false
1633 1400 Acked Write 4 Sent Done Done false
1634 1401 Acked Read 3 Sent Done Done false
1635 1402 Acked Flush 0 Sent Done Done false
1636 1403 Acked Write 5 Sent Done Done false
1637 1404 Acked Read 2 Sent Done Done false
1638 1405 Acked Read 4 Sent Done Done false
1639 1406 Acked Write 7 Sent Done Done false
1640 1407 Acked Read 7 Sent Done Done false
1641 1408 Acked Read 1 Sent Done Done false
1642 1409 Acked Write 1 Sent Done Done false
1643 1410 Acked Read 4 Sent Done Done false
1644 1411 Acked Write 4 Sent Done Done false
1645 1412 Acked Write 6 Sent Done Done false
1646 1413 Acked Write 4 Sent Done Done false
1647 1414 Acked Read 4 Sent Done Done false
1648 1415 Acked Write 7 Sent Done Done false
1649 1416 Acked Write 1 Sent Done Done false
1650 1417 Acked Read 1 Sent Done Done false
1651 1418 Acked Write 10 Sent Done Done false
1652 1419 Acked Read 5 Sent Done Done false
1653 1420 Acked Flush 0 Sent Done Done false
1654 1421 Acked Read 3 Sent Done Done false
1655 1422 Acked Write 7 Sent Done Done false
1656 1423 Acked Read 7 Sent Done Done false
1657 1424 Acked Write 7 Sent Done Done false
1658 1425 Acked Read 6 Sent Done Done false
1659 1426 Acked Write 2 Sent Done Done false
1660 1427 Acked Read 9 Sent Done Done false
1661 1428 Acked Flush 0 Sent Done Done false
1662 1429 Acked Read 7 Sent Done Done false
1663 1430 Acked Write 3 Sent Done Done false
1664 1431 Acked Read 2 Sent Done Done false
1665 1432 Acked Read 1 Sent Done Done false
1666 1433 Acked Read 7 Sent Done Done false
1667 1434 Acked Write 9 Sent Done Done false
1668 1435 Acked Read 2 Sent Done Done false
1669 1436 Acked Read 9 Sent Done Done false
1670 1437 Acked Write 4 Sent Done Done false
1671 1438 Acked Read 7 Sent Done Done false
1672 1439 Acked Write 6 Sent Done Done false
1673 1440 Acked Read 6 Sent Done Done false
1674 1441 Acked Write 6 Sent Done Done false
1675 1442 Acked Read 1 Sent Done Done false
1676 1443 Acked Read 2 Sent Done Done false
1677 1444 Acked Write 8 Sent Done Done false
1678 1445 Acked Read 5 Sent Done Done false
1679 1446 Acked Read 9 Sent Done Done false
1680 1447 Acked Write 7 Sent Done Done false
1681 1448 Acked Write 3 Sent Done Done false
1682 1449 Acked Read 9 Sent Done Done false
1683 1450 Acked Write 2 Sent Done Done false
1684 1451 Acked Write 9 Sent Done Done false
1685 1452 Acked Write 3 Sent Done Done false
1686 1453 Acked Read 9 Sent Done Done false
1687 1454 Acked Write 7 Sent Done Done false
1688 1455 Acked Read 10 Sent Done Done false
1689 1456 Acked Write 1 Sent Done Done false
1690 1457 Acked Read 7 Sent Done Done false
1691 1458 Acked Write 9 Sent Done Done false
1692 1459 Acked Read 6 Sent Done Done false
1693 1460 Acked Write 3 Sent Done Done false
1694 1461 Acked Write 2 Sent Done Done false
1695 1462 Acked Write 7 Sent Done Done false
1696 1463 Acked Read 10 Sent Done Done false
1697 1464 Acked Read 7 Sent Done Done false
1698 1465 Acked Flush 0 Sent Done Done false
1699 1466 Acked Write 5 Sent Done Done false
1700 1467 Acked Read 7 Sent Done Done false
1701 1468 Acked Write 9 Sent Done Done false
1702 1469 Acked Read 4 Sent Done Done false
1703 1470 Acked Write 6 Sent Done Done false
1704 1471 Acked Read 8 Sent Done Done false
1705 1472 Acked Write 7 Sent Done Done false
1706 1473 Acked Flush 0 Sent Done Done false
1707 1474 Acked Write 10 Sent Done Done false
1708 1475 Acked Write 4 Sent Done Done false
1709 1476 Acked Flush 0 Sent Done Done false
1710 1477 Acked Write 3 Sent Done Done false
1711 1478 Acked Write 5 Sent Done Done false
1712 1479 Acked Read 10 Sent Done Done false
1713 1480 Acked Write 9 Sent Done Done false
1714 1481 Acked Write 7 Sent Done Done false
1715 1482 Acked Read 1 Sent Done Done false
1716 1483 Acked Read 7 Sent Done Done false
1717 1484 Acked Read 3 Sent Done Done false
1718 1485 Acked Write 9 Sent Done Done false
1719 1486 Acked Read 1 Sent Done Done false
1720 1487 Acked Read 8 Sent Done Done false
1721 1488 Acked Write 6 Sent Done Done false
1722 1489 Acked Read 10 Sent Done Done false
1723 1490 Acked Read 10 Sent Done Done false
1724 1491 Acked Read 5 Sent Done Done false
1725 1492 Acked Flush 0 Sent Done Done false
1726 1493 Acked Read 3 Sent Done Done false
1727 1494 Acked Write 9 Sent Done Done false
1728 1495 Acked Write 5 Sent Done Done false
1729 1496 Acked Write 4 Sent Done Done false
1730 1497 Acked Read 1 Sent Done Done false
1731 1498 Acked Read 8 Sent Done Done false
1732 1499 Acked Read 10 Sent Done Done false
1733 1500 Acked Read 6 Sent Done Done false
1734 1501 Acked Write 1 Sent Done Done false
1735 1502 Acked Read 3 Sent Done Done false
1736 1503 Acked Read 10 Sent Done Done false
1737 1504 Acked Read 4 Sent Done Done false
1738 1505 Acked Read 5 Sent Done Done false
1739 1506 Acked Flush 0 Sent Done Done false
1740 1507 Acked Flush 0 Sent Done Done false
1741 1508 Acked Write 6 Sent Done Done false
1742 1509 Acked Read 7 Sent Done Done false
1743 1510 Acked Read 7 Sent Done Done false
1744 1511 Acked Flush 0 Sent Done Done false
1745 1512 Acked Write 3 Sent Done Done false
1746 1513 Acked Read 4 Sent Done Done false
1747 1514 Acked Read 2 Sent Done Done false
1748 1515 Acked Flush 0 Sent Done Done false
1749 1516 Acked Read 9 Sent Done Done false
1750 1517 Acked Write 1 Sent Done Done false
1751 1518 Acked Read 9 Sent Done Done false
1752 1519 Acked Read 8 Sent Done Done false
1753 1520 Acked Write 7 Sent Done Done false
1754 1521 Acked Write 9 Sent Done Done false
1755 1522 Acked Write 9 Sent Done Done false
1756 1523 Acked Read 4 Sent Done Done false
1757 1524 Acked Read 4 Sent Done Done false
1758 1525 Acked Read 6 Sent Done Done false
1759 1526 Acked Write 8 Sent Done Done false
1760 1527 Acked Write 1 Sent Done Done false
1761 1528 Acked Read 1 Sent Done Done false
1762 1529 Acked Read 2 Sent Done Done false
1763 1530 Acked Write 7 Sent Done Done false
1764 1531 Acked Write 5 Sent Done Done false
1765 1532 Acked Read 10 Sent Done Done false
1766 1533 Acked Flush 0 Sent Done Done false
1767 1534 Acked Read 9 Sent Done Done false
1768 1535 Acked Write 1 Sent Done Done false
1769 1536 Acked Write 2 Sent Done Done false
1770 1537 Acked Read 8 Sent Done Done false
1771 1538 Acked Write 7 Sent Done Done false
1772 1539 Acked Read 8 Sent Done Done false
1773 1540 Acked Read 3 Sent Done Done false
1774 1541 Acked Write 4 Sent Done Done false
1775 1542 Acked Read 4 Sent Done Done false
1776 1543 Acked Read 3 Sent Done Done false
1777 1544 Acked Write 3 Sent Done Done false
1778 1545 Acked Write 2 Sent Done Done false
1779 1546 Acked Read 4 Sent Done Done false
1780 1547 Acked Read 10 Sent Done Done false
1781 1548 Acked Read 8 Sent Done Done false
1782 1549 Acked Write 6 Sent Done Done false
1783 1550 Acked Read 1 Sent Done Done false
1784 1551 Acked Write 4 Sent Done Done false
1785 1552 Acked Read 5 Sent Done Done false
1786 1553 Acked Read 3 Sent Done Done false
1787 1554 Acked Read 5 Sent Done Done false
1788 1555 Acked Flush 0 Sent Done Done false
1789 1556 Acked Read 9 Sent Done Done false
1790 1557 Acked Read 5 Sent Done Done false
1791 1558 Acked Write 10 Sent Done Done false
1792 1559 Acked Read 2 Sent Done Done false
1793 1560 Acked Write 9 Sent Done Done false
1794 1561 Acked Flush 0 Sent Done Done false
1795 1562 Acked Read 3 Sent Done Done false
1796 1563 Acked Read 3 Sent Done Done false
1797 1564 Acked Read 1 Sent Done Done false
1798 1565 Acked Read 3 Sent Done Done false
1799 1566 Acked Read 8 Sent Done Done false
1800 1567 Acked Read 2 Sent Done Done false
1801 1568 Acked Write 7 Sent Done Done false
1802 1569 Acked Read 3 Sent Done Done false
1803 1570 Acked Write 2 Sent Done Done false
1804 1571 Acked Write 10 Sent Done Done false
1805 1572 Acked Write 1 Sent Done Done false
1806 1573 Acked Read 1 Sent Done Done false
1807 1574 Acked Read 6 Sent Done Done false
1808 1575 Acked Write 3 Sent Done Done false
1809 1576 Acked Write 10 Sent Done Done false
1810 1577 Acked Read 5 Sent Done Done false
1811 1578 Acked Read 2 Sent Done Done false
1812 1579 Acked Read 9 Sent Done Done false
1813 1580 Acked Read 3 Sent Done Done false
1814 1581 Acked Read 8 Sent Done Done false
1815 1582 Acked Write 6 Sent Done Done false
1816 1583 Acked Write 10 Sent Done Done false
1817 1584 Acked Read 9 Sent Done Done false
1818 1585 Acked Read 6 Sent Done Done false
1819 1586 Acked Read 3 Sent Done Done false
1820 1587 Acked Write 7 Sent Done Done false
1821 1588 Acked Read 1 Sent Done Done false
1822 1589 Acked Read 9 Sent Done Done false
1823 1590 Acked Write 10 Sent Done Done false
1824 1591 Acked Read 8 Sent Done Done false
1825 1592 Acked Write 1 Sent Done Done false
1826 1593 Acked Write 1 Sent Done Done false
1827 1594 Acked Read 7 Sent Done Done false
1828 1595 Acked Write 1 Sent Done Done false
1829 1596 Acked Flush 0 Sent Done Done false
1830 1597 Acked Read 2 Sent Done Done false
1831 1598 Acked Write 4 Sent Done Done false
1832 1599 Acked Write 3 Sent Done Done false
1833 1600 Acked Write 7 Sent Done Done false
1834 1601 Acked Read 9 Sent Done Done false
1835 1602 Acked Read 3 Sent Done Done false
1836 STATES DS:0 DS:1 DS:2 TOTAL
1837 Sent 301 0 0 301
1838 Done 0 301 301 602
1839 Skipped 0 0 0 0
1840 Error 0 0 0 0
1841 Last Flush: 1301 1596 1596
1842 Downstairs last five completed: 1301 1300 1299 1298 1297
1843 Upstairs last five completed: 1602 1601 1600 1599 1598
1844 CLIENT: Up:0 ds:301 act:2
1845 [8810] got start action
1846 [8810] I am going to restart
1847 Starting downstairs at port 8810
1848 Make output file at "/tmp/dsc/downstairs-8810.txt"
1849 [8810][0] reports Starting
1850 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1204)
1851 [8810][0] reports Running
1852 ----------------------------------------------------------------
1853 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1854 States: Offline Active Active
1855 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1856 1302 Acked Write 5 Sent Done Done false
1857 1303 Acked Read 7 Sent Done Done false
1858 1304 Acked Read 7 Sent Done Done false
1859 1305 Acked Flush 0 Sent Done Done false
1860 1306 Acked Read 1 Sent Done Done false
1861 1307 Acked Read 8 Sent Done Done false
1862 1308 Acked Write 10 Sent Done Done false
1863 1309 Acked Read 2 Sent Done Done false
1864 1310 Acked Write 3 Sent Done Done false
1865 1311 Acked Write 4 Sent Done Done false
1866 1312 Acked Write 3 Sent Done Done false
1867 1313 Acked Write 1 Sent Done Done false
1868 1314 Acked Write 7 Sent Done Done false
1869 1315 Acked Flush 0 Sent Done Done false
1870 1316 Acked Write 4 Sent Done Done false
1871 1317 Acked Write 4 Sent Done Done false
1872 1318 Acked Read 5 Sent Done Done false
1873 1319 Acked Read 8 Sent Done Done false
1874 1320 Acked Write 4 Sent Done Done false
1875 1321 Acked Read 3 Sent Done Done false
1876 1322 Acked Flush 0 Sent Done Done false
1877 1323 Acked Read 4 Sent Done Done false
1878 1324 Acked Flush 0 Sent Done Done false
1879 1325 Acked Read 9 Sent Done Done false
1880 1326 Acked Write 6 Sent Done Done false
1881 1327 Acked Read 4 Sent Done Done false
1882 1328 Acked Read 9 Sent Done Done false
1883 1329 Acked Read 9 Sent Done Done false
1884 1330 Acked Flush 0 Sent Done Done false
1885 1331 Acked Write 8 Sent Done Done false
1886 1332 Acked Write 2 Sent Done Done false
1887 1333 Acked Read 7 Sent Done Done false
1888 1334 Acked Read 5 Sent Done Done false
1889 1335 Acked Read 4 Sent Done Done false
1890 1336 Acked Write 7 Sent Done Done false
1891 1337 Acked Read 6 Sent Done Done false
1892 1338 Acked Read 7 Sent Done Done false
1893 1339 Acked Read 9 Sent Done Done false
1894 1340 Acked Flush 0 Sent Done Done false
1895 1341 Acked Write 10 Sent Done Done false
1896 1342 Acked Write 8 Sent Done Done false
1897 1343 Acked Write 2 Sent Done Done false
1898 1344 Acked Read 9 Sent Done Done false
1899 1345 Acked Flush 0 Sent Done Done false
1900 1346 Acked Read 4 Sent Done Done false
1901 1347 Acked Write 4 Sent Done Done false
1902 1348 Acked Write 5 Sent Done Done false
1903 1349 Acked Write 10 Sent Done Done false
1904 1350 Acked Read 10 Sent Done Done false
1905 1351 Acked Flush 0 Sent Done Done false
1906 1352 Acked Read 10 Sent Done Done false
1907 1353 Acked Write 5 Sent Done Done false
1908 1354 Acked Read 1 Sent Done Done false
1909 1355 Acked Read 1 Sent Done Done false
1910 1356 Acked Read 8 Sent Done Done false
1911 1357 Acked Read 1 Sent Done Done false
1912 1358 Acked Read 6 Sent Done Done false
1913 1359 Acked Write 6 Sent Done Done false
1914 1360 Acked Write 5 Sent Done Done false
1915 1361 Acked Read 9 Sent Done Done false
1916 1362 Acked Read 2 Sent Done Done false
1917 1363 Acked Read 4 Sent Done Done false
1918 1364 Acked Read 3 Sent Done Done false
1919 1365 Acked Read 8 Sent Done Done false
1920 1366 Acked Write 7 Sent Done Done false
1921 1367 Acked Read 4 Sent Done Done false
1922 1368 Acked Read 9 Sent Done Done false
1923 1369 Acked Read 7 Sent Done Done false
1924 1370 Acked Write 4 Sent Done Done false
1925 1371 Acked Read 7 Sent Done Done false
1926 1372 Acked Read 8 Sent Done Done false
1927 1373 Acked Write 7 Sent Done Done false
1928 1374 Acked Write 8 Sent Done Done false
1929 1375 Acked Read 4 Sent Done Done false
1930 1376 Acked Read 1 Sent Done Done false
1931 1377 Acked Flush 0 Sent Done Done false
1932 1378 Acked Read 6 Sent Done Done false
1933 1379 Acked Write 1 Sent Done Done false
1934 1380 Acked Read 7 Sent Done Done false
1935 1381 Acked Read 4 Sent Done Done false
1936 1382 Acked Write 4 Sent Done Done false
1937 1383 Acked Read 1 Sent Done Done false
1938 1384 Acked Flush 0 Sent Done Done false
1939 1385 Acked Write 5 Sent Done Done false
1940 1386 Acked Write 9 Sent Done Done false
1941 1387 Acked Write 1 Sent Done Done false
1942 1388 Acked Read 8 Sent Done Done false
1943 1389 Acked Write 10 Sent Done Done false
1944 1390 Acked Write 6 Sent Done Done false
1945 1391 Acked Read 5 Sent Done Done false
1946 1392 Acked Write 2 Sent Done Done false
1947 1393 Acked Read 10 Sent Done Done false
1948 1394 Acked Read 5 Sent Done Done false
1949 1395 Acked Read 6 Sent Done Done false
1950 1396 Acked Write 9 Sent Done Done false
1951 1397 Acked Read 4 Sent Done Done false
1952 1398 Acked Read 6 Sent Done Done false
1953 1399 Acked Write 2 Sent Done Done false
1954 1400 Acked Write 4 Sent Done Done false
1955 1401 Acked Read 3 Sent Done Done false
1956 1402 Acked Flush 0 Sent Done Done false
1957 1403 Acked Write 5 Sent Done Done false
1958 1404 Acked Read 2 Sent Done Done false
1959 1405 Acked Read 4 Sent Done Done false
1960 1406 Acked Write 7 Sent Done Done false
1961 1407 Acked Read 7 Sent Done Done false
1962 1408 Acked Read 1 Sent Done Done false
1963 1409 Acked Write 1 Sent Done Done false
1964 1410 Acked Read 4 Sent Done Done false
1965 1411 Acked Write 4 Sent Done Done false
1966 1412 Acked Write 6 Sent Done Done false
1967 1413 Acked Write 4 Sent Done Done false
1968 1414 Acked Read 4 Sent Done Done false
1969 1415 Acked Write 7 Sent Done Done false
1970 1416 Acked Write 1 Sent Done Done false
1971 1417 Acked Read 1 Sent Done Done false
1972 1418 Acked Write 10 Sent Done Done false
1973 1419 Acked Read 5 Sent Done Done false
1974 1420 Acked Flush 0 Sent Done Done false
1975 1421 Acked Read 3 Sent Done Done false
1976 1422 Acked Write 7 Sent Done Done false
1977 1423 Acked Read 7 Sent Done Done false
1978 1424 Acked Write 7 Sent Done Done false
1979 1425 Acked Read 6 Sent Done Done false
1980 1426 Acked Write 2 Sent Done Done false
1981 1427 Acked Read 9 Sent Done Done false
1982 1428 Acked Flush 0 Sent Done Done false
1983 1429 Acked Read 7 Sent Done Done false
1984 1430 Acked Write 3 Sent Done Done false
1985 1431 Acked Read 2 Sent Done Done false
1986 1432 Acked Read 1 Sent Done Done false
1987 1433 Acked Read 7 Sent Done Done false
1988 1434 Acked Write 9 Sent Done Done false
1989 1435 Acked Read 2 Sent Done Done false
1990 1436 Acked Read 9 Sent Done Done false
1991 1437 Acked Write 4 Sent Done Done false
1992 1438 Acked Read 7 Sent Done Done false
1993 1439 Acked Write 6 Sent Done Done false
1994 1440 Acked Read 6 Sent Done Done false
1995 1441 Acked Write 6 Sent Done Done false
1996 1442 Acked Read 1 Sent Done Done false
1997 1443 Acked Read 2 Sent Done Done false
1998 1444 Acked Write 8 Sent Done Done false
1999 1445 Acked Read 5 Sent Done Done false
2000 1446 Acked Read 9 Sent Done Done false
2001 1447 Acked Write 7 Sent Done Done false
2002 1448 Acked Write 3 Sent Done Done false
2003 1449 Acked Read 9 Sent Done Done false
2004 1450 Acked Write 2 Sent Done Done false
2005 1451 Acked Write 9 Sent Done Done false
2006 1452 Acked Write 3 Sent Done Done false
2007 1453 Acked Read 9 Sent Done Done false
2008 1454 Acked Write 7 Sent Done Done false
2009 1455 Acked Read 10 Sent Done Done false
2010 1456 Acked Write 1 Sent Done Done false
2011 1457 Acked Read 7 Sent Done Done false
2012 1458 Acked Write 9 Sent Done Done false
2013 1459 Acked Read 6 Sent Done Done false
2014 1460 Acked Write 3 Sent Done Done false
2015 1461 Acked Write 2 Sent Done Done false
2016 1462 Acked Write 7 Sent Done Done false
2017 1463 Acked Read 10 Sent Done Done false
2018 1464 Acked Read 7 Sent Done Done false
2019 1465 Acked Flush 0 Sent Done Done false
2020 1466 Acked Write 5 Sent Done Done false
2021 1467 Acked Read 7 Sent Done Done false
2022 1468 Acked Write 9 Sent Done Done false
2023 1469 Acked Read 4 Sent Done Done false
2024 1470 Acked Write 6 Sent Done Done false
2025 1471 Acked Read 8 Sent Done Done false
2026 1472 Acked Write 7 Sent Done Done false
2027 1473 Acked Flush 0 Sent Done Done false
2028 1474 Acked Write 10 Sent Done Done false
2029 1475 Acked Write 4 Sent Done Done false
2030 1476 Acked Flush 0 Sent Done Done false
2031 1477 Acked Write 3 Sent Done Done false
2032 1478 Acked Write 5 Sent Done Done false
2033 1479 Acked Read 10 Sent Done Done false
2034 1480 Acked Write 9 Sent Done Done false
2035 1481 Acked Write 7 Sent Done Done false
2036 1482 Acked Read 1 Sent Done Done false
2037 1483 Acked Read 7 Sent Done Done false
2038 1484 Acked Read 3 Sent Done Done false
2039 1485 Acked Write 9 Sent Done Done false
2040 1486 Acked Read 1 Sent Done Done false
2041 1487 Acked Read 8 Sent Done Done false
2042 1488 Acked Write 6 Sent Done Done false
2043 1489 Acked Read 10 Sent Done Done false
2044 1490 Acked Read 10 Sent Done Done false
2045 1491 Acked Read 5 Sent Done Done false
2046 1492 Acked Flush 0 Sent Done Done false
2047 1493 Acked Read 3 Sent Done Done false
2048 1494 Acked Write 9 Sent Done Done false
2049 1495 Acked Write 5 Sent Done Done false
2050 1496 Acked Write 4 Sent Done Done false
2051 1497 Acked Read 1 Sent Done Done false
2052 1498 Acked Read 8 Sent Done Done false
2053 1499 Acked Read 10 Sent Done Done false
2054 1500 Acked Read 6 Sent Done Done false
2055 1501 Acked Write 1 Sent Done Done false
2056 1502 Acked Read 3 Sent Done Done false
2057 1503 Acked Read 10 Sent Done Done false
2058 1504 Acked Read 4 Sent Done Done false
2059 1505 Acked Read 5 Sent Done Done false
2060 1506 Acked Flush 0 Sent Done Done false
2061 1507 Acked Flush 0 Sent Done Done false
2062 1508 Acked Write 6 Sent Done Done false
2063 1509 Acked Read 7 Sent Done Done false
2064 1510 Acked Read 7 Sent Done Done false
2065 1511 Acked Flush 0 Sent Done Done false
2066 1512 Acked Write 3 Sent Done Done false
2067 1513 Acked Read 4 Sent Done Done false
2068 1514 Acked Read 2 Sent Done Done false
2069 1515 Acked Flush 0 Sent Done Done false
2070 1516 Acked Read 9 Sent Done Done false
2071 1517 Acked Write 1 Sent Done Done false
2072 1518 Acked Read 9 Sent Done Done false
2073 1519 Acked Read 8 Sent Done Done false
2074 1520 Acked Write 7 Sent Done Done false
2075 1521 Acked Write 9 Sent Done Done false
2076 1522 Acked Write 9 Sent Done Done false
2077 1523 Acked Read 4 Sent Done Done false
2078 1524 Acked Read 4 Sent Done Done false
2079 1525 Acked Read 6 Sent Done Done false
2080 1526 Acked Write 8 Sent Done Done false
2081 1527 Acked Write 1 Sent Done Done false
2082 1528 Acked Read 1 Sent Done Done false
2083 1529 Acked Read 2 Sent Done Done false
2084 1530 Acked Write 7 Sent Done Done false
2085 1531 Acked Write 5 Sent Done Done false
2086 1532 Acked Read 10 Sent Done Done false
2087 1533 Acked Flush 0 Sent Done Done false
2088 1534 Acked Read 9 Sent Done Done false
2089 1535 Acked Write 1 Sent Done Done false
2090 1536 Acked Write 2 Sent Done Done false
2091 1537 Acked Read 8 Sent Done Done false
2092 1538 Acked Write 7 Sent Done Done false
2093 1539 Acked Read 8 Sent Done Done false
2094 1540 Acked Read 3 Sent Done Done false
2095 1541 Acked Write 4 Sent Done Done false
2096 1542 Acked Read 4 Sent Done Done false
2097 1543 Acked Read 3 Sent Done Done false
2098 1544 Acked Write 3 Sent Done Done false
2099 1545 Acked Write 2 Sent Done Done false
2100 1546 Acked Read 4 Sent Done Done false
2101 1547 Acked Read 10 Sent Done Done false
2102 1548 Acked Read 8 Sent Done Done false
2103 1549 Acked Write 6 Sent Done Done false
2104 1550 Acked Read 1 Sent Done Done false
2105 1551 Acked Write 4 Sent Done Done false
2106 1552 Acked Read 5 Sent Done Done false
2107 1553 Acked Read 3 Sent Done Done false
2108 1554 Acked Read 5 Sent Done Done false
2109 1555 Acked Flush 0 Sent Done Done false
2110 1556 Acked Read 9 Sent Done Done false
2111 1557 Acked Read 5 Sent Done Done false
2112 1558 Acked Write 10 Sent Done Done false
2113 1559 Acked Read 2 Sent Done Done false
2114 1560 Acked Write 9 Sent Done Done false
2115 1561 Acked Flush 0 Sent Done Done false
2116 1562 Acked Read 3 Sent Done Done false
2117 1563 Acked Read 3 Sent Done Done false
2118 1564 Acked Read 1 Sent Done Done false
2119 1565 Acked Read 3 Sent Done Done false
2120 1566 Acked Read 8 Sent Done Done false
2121 1567 Acked Read 2 Sent Done Done false
2122 1568 Acked Write 7 Sent Done Done false
2123 1569 Acked Read 3 Sent Done Done false
2124 1570 Acked Write 2 Sent Done Done false
2125 1571 Acked Write 10 Sent Done Done false
2126 1572 Acked Write 1 Sent Done Done false
2127 1573 Acked Read 1 Sent Done Done false
2128 1574 Acked Read 6 Sent Done Done false
2129 1575 Acked Write 3 Sent Done Done false
2130 1576 Acked Write 10 Sent Done Done false
2131 1577 Acked Read 5 Sent Done Done false
2132 1578 Acked Read 2 Sent Done Done false
2133 1579 Acked Read 9 Sent Done Done false
2134 1580 Acked Read 3 Sent Done Done false
2135 1581 Acked Read 8 Sent Done Done false
2136 1582 Acked Write 6 Sent Done Done false
2137 1583 Acked Write 10 Sent Done Done false
2138 1584 Acked Read 9 Sent Done Done false
2139 1585 Acked Read 6 Sent Done Done false
2140 1586 Acked Read 3 Sent Done Done false
2141 1587 Acked Write 7 Sent Done Done false
2142 1588 Acked Read 1 Sent Done Done false
2143 1589 Acked Read 9 Sent Done Done false
2144 1590 Acked Write 10 Sent Done Done false
2145 1591 Acked Read 8 Sent Done Done false
2146 1592 Acked Write 1 Sent Done Done false
2147 1593 Acked Write 1 Sent Done Done false
2148 1594 Acked Read 7 Sent Done Done false
2149 1595 Acked Write 1 Sent Done Done false
2150 1596 Acked Flush 0 Sent Done Done false
2151 1597 Acked Read 2 Sent Done Done false
2152 1598 Acked Write 4 Sent Done Done false
2153 1599 Acked Write 3 Sent Done Done false
2154 1600 Acked Write 7 Sent Done Done false
2155 1601 Acked Read 9 Sent Done Done false
2156 1602 Acked Read 3 Sent Done Done false
2157 1603 Acked Flush 0 Sent Done Done false
2158 STATES DS:0 DS:1 DS:2 TOTAL
2159 Sent 302 0 0 302
2160 Done 0 302 302 604
2161 Skipped 0 0 0 0
2162 Error 0 0 0 0
2163 Last Flush: 1301 1603 1603
2164 Downstairs last five completed: 1301 1300 1299 1298 1297
2165 Upstairs last five completed: 1603 1602 1601 1600 1599
2166 CLIENT: Up:0 ds:302 act:2
21672024-11-19T17:08:25.964ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21682024-11-19T17:08:25.964ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62369) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21692024-11-19T17:08:25.965ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21702024-11-19T17:08:25.965ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21712024-11-19T17:08:25.965ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21722024-11-19T17:08:25.965ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21732024-11-19T17:08:25.966ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21742024-11-19T17:08:25.966ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21752024-11-19T17:08:25.966ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21762024-11-19T17:08:25.966ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
21772024-11-19T17:08:25.966ZINFOcrucible: [0] Replayed 302 jobs since flush: 1301 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
2178 ----------------------------------------------------------------
2179 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2180 Downstairs last five completed: 1603 1602 1601 1600 1599
2181 Upstairs last five completed: 1603 1602 1601 1600 1599
2182 CLIENT: Up:0 ds:0 act:3
2183 Replay: All jobs finished
2184 Main task has work to do, go find it
2185 got dsc Stop(0)
2186 stop 0
2187 Nov 19 17:08:28.208 INFO request completed, [8810] Got stop action so:false kr:false
2188 latency_us: 344, response_code: 204, uri: /stop/cid/0, method: POST, req_id: da646cdd-a096-4aaa-8e49-18e1d3adb82d, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
2189 Nov 19 17:08:28.209 INFO {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:08:28.209044807Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
2190 request completed, latency_us: 137, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3f2b8b5e-05c4-4fca-9b2b-e168528a205c, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
21912024-11-19T17:08:28.209ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21922024-11-19T17:08:28.209ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
21932024-11-19T17:08:28.209ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21942024-11-19T17:08:28.209ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21952024-11-19T17:08:28.209ZINFOcrucible: client task is exiting = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
2196 [8810] Exited with: ExitStatus(unix_wait_status(9))
2197 [8810][0] reports Exit
2198 Nov 19 17:08:32.211 INFO request completed, latency_us: 318, response_code: 200, uri: /state/cid/0, method: GET, req_id: d9f68e84-8c07-4952-9d7e-7630e3e7e2d5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
2199 001/300 Write block 1616 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2200 002/300 Read block 1905 len 32768
2201 003/300 Write block 2807 len 28672 data: 2 2 2 2 2 2 2
2202 004/300 Flush
2203 005/300 Read block 594 len 36864
2204 006/300 Write block 972 len 16384 data: 2 2 2 2
2205 007/300 Write block 4216 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2206 008/300 Read block 1072 len 4096
2207 009/300 Read block 3530 len 20480
2208 010/300 Read block 702 len 12288
2209 011/300 Read block 4439 len 36864
2210 012/300 Read block 2647 len 8192
2211 013/300 Write block 3024 len 40960 data: 2 2 2 3 3 3 4 4 4 4
2212 014/300 Write block 1819 len 4096 data: 3
2213 015/300 Write block 666 len 40960 data: 2 4 4 5 4 4 4 4 4 4
2214 016/300 Write block 2604 len 32768 data: 3 3 3 3 3 3 3 3
2215 017/300 Read block 1360 len 20480
2216 018/300 Read block 812 len 28672
2217 019/300 Read block 964 len 12288
2218 020/300 Write block 81 len 16384 data: 2 2 2 2
2219 021/300 Read block 4415 len 36864
2220 022/300 Write block 3243 len 12288 data: 2 2 2
2221 023/300 Flush
2222 024/300 Write block 1139 len 4096 data: 2
2223 025/300 Write block 1418 len 20480 data: 2 2 2 2 2
2224 026/300 Write block 3187 len 12288 data: 2 2 2
2225 027/300 Flush
2226 028/300 Write block 4214 len 28672 data: 2 2 3 3 3 3 3
2227 029/300 Read block 3739 len 40960
2228 030/300 Write block 1268 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2229 031/300 Write block 2582 len 12288 data: 2 2 2
2230 032/300 Read block 2090 len 4096
2231 033/300 Write block 3124 len 36864 data: 2 2 2 2 2 3 3 3 3
2232 034/300 Read block 3418 len 12288
2233 035/300 Flush
2234 036/300 Write block 1446 len 32768 data: 2 2 2 2 3 3 3 3
2235 037/300 Write block 1754 len 28672 data: 2 2 2 2 2 2 2
2236 038/300 Read block 4584 len 16384
2237 039/300 Read block 291 len 16384
2238 040/300 Read block 4394 len 20480
2239 041/300 Read block 1913 len 12288
2240 042/300 Write block 4331 len 24576 data: 3 3 4 4 4 3
2241 043/300 Read block 3137 len 36864
2242 044/300 Write block 4357 len 36864 data: 4 3 3 3 3 3 3 3 3
2243 045/300 Read block 3121 len 40960
2244 046/300 Read block 2321 len 8192
2245 047/300 Read block 3719 len 32768
2246 048/300 Read block 3619 len 32768
2247 049/300 Read block 2362 len 4096
2248 050/300 Read block 2182 len 8192
2249 051/300 Read block 2740 len 20480
2250 052/300 Write block 4286 len 32768 data: 3 3 3 2 2 2 2 2
2251 053/300 Write block 1066 len 36864 data: 2 2 2 2 2 2 2 2 2
2252 054/300 Read block 668 len 4096
2253 055/300 Write block 4171 len 4096 data: 2
2254 056/300 Read block 2017 len 16384
2255 057/300 Write block 4215 len 32768 data: 3 4 4 4 4 4 3 3
2256 058/300 Read block 2657 len 12288
2257 059/300 Read block 2466 len 12288
2258 060/300 Read block 2580 len 32768
2259 061/300 Write block 3960 len 28672 data: 2 2 2 2 2 2 2
2260 062/300 Write block 188 len 28672 data: 3 3 2 2 2 2 2
2261 063/300 Read block 2574 len 36864
2262 064/300 Flush
2263 065/300 Read block 3700 len 40960
2264 066/300 Read block 2645 len 16384
2265 067/300 Write block 4517 len 12288 data: 2 2 2
2266 068/300 Write block 2896 len 4096 data: 3
2267 069/300 Read block 2011 len 40960
2268 070/300 Read block 4277 len 20480
2269 071/300 Write block 3756 len 40960 data: 3 3 4 3 3 3 3 3 3 3
2270 072/300 Write block 1400 len 32768 data: 3 3 3 2 2 2 2 2
2271 073/300 Write block 3678 len 24576 data: 2 2 2 2 2 2
2272 074/300 Read block 694 len 12288
2273 075/300 Flush
2274 076/300 Read block 3993 len 4096
2275 077/300 Write block 2787 len 28672 data: 2 2 2 2 2 2 2
2276 078/300 Read block 1892 len 12288
2277 079/300 Read block 4646 len 12288
2278 080/300 Write block 4472 len 24576 data: 2 2 2 2 2 2
2279 081/300 Write block 1688 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2280 082/300 Read block 1367 len 32768
2281 083/300 Write block 3914 len 12288 data: 2 2 2
2282 084/300 Read block 1997 len 36864
2283 085/300 Write block 4474 len 12288 data: 3 3 3
2284 086/300 Read block 94 len 28672
2285 087/300 Flush
2286 088/300 Write block 1266 len 24576 data: 2 2 3 3 3 3
2287 089/300 Read block 4710 len 24576
2288 090/300 Read block 3362 len 8192
2289 091/300 Read block 2427 len 20480
2290 092/300 Read block 1254 len 12288
2291 093/300 Write block 3238 len 24576 data: 2 2 2 2 2 3
2292 094/300 Flush
2293 095/300 Read block 2849 len 20480
2294 096/300 Write block 4041 len 36864 data: 3 2 2 2 2 2 2 2 2
2295 097/300 Read block 2221 len 8192
2296 098/300 Read block 1231 len 28672
2297 099/300 Flush
2298 100/300 Write block 1348 len 32768 data: 2 2 2 3 3 3 3 3
2299 101/300 Write block 2907 len 4096 data: 2
2300 102/300 Flush
2301 103/300 Read block 1490 len 32768
2302 104/300 Read block 1611 len 8192
2303 105/300 Read block 1916 len 32768
2304 106/300 Write block 3146 len 32768 data: 2 2 2 2 2 2 2 2
2305 107/300 Read block 477 len 16384
2306 108/300 Write block 891 len 24576 data: 2 2 2 2 2 2
2307 109/300 Read block 1981 len 4096
2308 110/300 Read block 3341 len 24576
2309 111/300 Write block 1289 len 24576 data: 3 3 3 2 2 2
2310 112/300 Read block 117 len 28672
2311 113/300 Write block 3162 len 12288 data: 2 2 2
2312 114/300 Flush
2313 115/300 Write block 127 len 40960 data: 2 2 3 3 3 3 4 4 5 4
2314 116/300 Flush
2315 117/300 Read block 53 len 24576
2316 118/300 Read block 3937 len 20480
2317 119/300 Write block 811 len 12288 data: 2 2 2
2318 120/300 Read block 1461 len 24576
2319 121/300 Read block 4091 len 8192
2320 122/300 Read block 765 len 16384
2321 123/300 Read block 2403 len 24576
2322 124/300 Write block 3819 len 36864 data: 2 2 2 2 2 2 2 2 2
2323 125/300 Write block 927 len 36864 data: 2 2 2 2 2 2 2 2 2
2324 126/300 Read block 1474 len 8192
2325 127/300 Read block 3247 len 16384
2326 128/300 Read block 4507 len 12288
2327 129/300 Read block 4409 len 32768
2328 130/300 Read block 762 len 8192
2329 131/300 Read block 1385 len 40960
2330 132/300 Read block 996 len 36864
2331 133/300 Write block 1607 len 20480 data: 2 2 2 2 2
2332 134/300 Read block 2573 len 40960
2333 135/300 Write block 4205 len 40960 data: 2 2 2 2 2 2 2 2 2 3
2334 136/300 Write block 4089 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2335 137/300 Read block 2589 len 24576
2336 138/300 Read block 3836 len 12288
2337 139/300 Write block 3321 len 12288 data: 2 2 2
2338 140/300 Write block 885 len 12288 data: 3 3 2
2339 141/300 Write block 1038 len 36864 data: 3 2 2 2 2 2 2 2 3
2340 142/300 Write block 326 len 12288 data: 2 3 3
2341 143/300 Flush
2342 144/300 Write block 88 len 20480 data: 2 2 2 2 2
2343 145/300 Read block 4921 len 8192
2344 146/300 Read block 1095 len 20480
2345 147/300 Write block 2159 len 36864 data: 2 2 2 2 2 2 2 2 3
2346 148/300 Read block 4151 len 32768
2347 149/300 Read block 1589 len 12288
2348 150/300 Write block 4245 len 28672 data: 2 2 2 2 2 2 2
2349 151/300 Write block 1898 len 28672 data: 2 2 2 2 2 2 2
2350 152/300 Read block 1261 len 4096
2351 153/300 Write block 677 len 36864 data: 2 2 2 2 2 2 2 2 2
2352 154/300 Read block 3383 len 8192
2353 155/300 Write block 136 len 16384 data: 5 4 3 4
2354 156/300 Read block 2745 len 32768
2355 157/300 Write block 3924 len 40960 data: 2 2 3 3 3 3 3 3 3 2
2356 158/300 Write block 4801 len 24576 data: 2 2 2 2 2 2
2357 159/300 Read block 476 len 32768
2358 160/300 Read block 4221 len 28672
2359 161/300 Read block 2113 len 12288
2360 162/300 Write block 4884 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2361 163/300 Read block 2709 len 36864
2362 164/300 Write block 343 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2363 165/300 Write block 4137 len 12288 data: 2 2 2
2364 166/300 Read block 351 len 8192
2365 167/300 Read block 4574 len 20480
2366 168/300 Read block 3995 len 40960
2367 169/300 Write block 2730 len 36864 data: 3 3 3 3 2 2 2 2 2
2368 170/300 Read block 677 len 20480
2369 171/300 Write block 3909 len 40960 data: 2 2 2 2 2 3 3 3 2 2
2370 172/300 Read block 3651 len 36864
2371 173/300 Write block 3739 len 20480 data: 3 3 3 3 3
2372 174/300 Flush
2373 175/300 Read block 1666 len 12288
2374 176/300 Write block 3729 len 4096 data: 2
2375 177/300 Write block 3611 len 36864 data: 2 2 2 2 2 2 2 2 3
2376 178/300 Read block 3033 len 28672
2377 179/300 Read block 4160 len 12288
2378 180/300 Write block 2208 len 32768 data: 2 2 2 2 2 3 2 2
2379 181/300 Write block 73 len 32768 data: 2 2 2 2 2 2 2 2
2380 182/300 Flush
2381 183/300 Write block 2287 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2382 184/300 Write block 1240 len 28672 data: 2 2 2 2 2 2 2
2383 185/300 Write block 3399 len 24576 data: 2 2 2 2 2 2
2384 186/300 Flush
2385 187/300 Read block 2663 len 24576
2386 188/300 Flush
2387 189/300 Write block 4437 len 20480 data: 3 3 3 3 2
2388 190/300 Write block 4330 len 16384 data: 3 4 4 5
2389 191/300 Write block 781 len 36864 data: 2 2 2 2 2 2 3 3 3
2390 192/300 Write block 4927 len 28672 data: 2 2 2 2 2 3 3
2391 193/300 Read block 4642 len 12288
2392 194/300 Read block 1539 len 4096
2393 195/300 Write block 444 len 12288 data: 2 2 2
2394 196/300 Read block 2859 len 36864
2395 197/300 Write block 2802 len 8192 data: 2 2
2396 198/300 Write block 3552 len 4096 data: 2
2397 199/300 Write block 1613 len 28672 data: 2 2 2 3 3 3 3
2398 200/300 Read block 4701 len 4096
2399 201/300 Read block 1337 len 12288
2400 202/300 Read block 1743 len 36864
2401 203/300 Write block 2102 len 40960 data: 3 3 3 3 4 2 2 2 2 2
2402 204/300 Write block 2662 len 28672 data: 2 2 2 2 2 2 2
2403 205/300 Read block 4914 len 12288
2404 206/300 Read block 3426 len 24576
2405 207/300 Read block 2335 len 40960
2406 208/300 Write block 736 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2407 209/300 Read block 4716 len 28672
2408 210/300 Read block 4261 len 28672
2409 211/300 Write block 3215 len 8192 data: 2 2
2410 212/300 Flush
2411 213/300 Write block 4211 len 4096 data: 3
2412 214/300 Write block 1411 len 36864 data: 2 2 2 2 2 2 2 3 3
2413 215/300 Write block 588 len 8192 data: 2 2
2414 216/300 Read block 4044 len 36864
2415 217/300 Flush
2416 218/300 Flush
2417 219/300 Read block 6 len 28672
2418 220/300 Read block 4771 len 32768
2419 221/300 Write block 377 len 12288 data: 2 2 2
2420 222/300 Flush
2421 223/300 Read block 3677 len 20480
2422 224/300 Read block 1687 len 16384
2423 225/300 Read block 2222 len 8192
2424 226/300 Flush
2425 227/300 Flush
2426 228/300 Read block 398 len 28672
2427 229/300 Write block 1421 len 40960 data: 3 3 2 2 2 2 2 2 2 2
2428 230/300 Read block 3970 len 40960
2429 231/300 Read block 2114 len 4096
2430 232/300 Write block 2783 len 12288 data: 2 2 2
2431 233/300 Read block 429 len 24576
2432 234/300 Read block 4491 len 16384
2433 235/300 Read block 2861 len 24576
2434 236/300 Write block 1192 len 20480 data: 2 2 2 2 2
2435 237/300 Read block 4811 len 24576
2436 238/300 Read block 1910 len 20480
2437 239/300 Read block 2501 len 12288
2438 240/300 Read block 2487 len 36864
2439 241/300 Write block 2004 len 32768 data: 2 2 2 2 2 2 2 2
2440 242/300 Read block 4358 len 4096
2441 243/300 Write block 2529 len 16384 data: 2 2 2 2
2442 244/300 Read block 4756 len 8192
2443 245/300 Read block 1801 len 8192
2444 246/300 Write block 4907 len 32768 data: 2 2 3 3 3 3 3 3
2445 247/300 Write block 723 len 36864 data: 3 3 3 3 3 3 2 2 2
2446 248/300 Read block 1042 len 4096
2447 249/300 Flush
2448 250/300 Write block 3815 len 12288 data: 2 2 2
2449 251/300 Flush
2450 252/300 Write block 1202 len 12288 data: 2 2 2
2451 253/300 Flush
2452 254/300 Write block 3912 len 36864 data: 3 3 4 4 4 3 3 2 2
2453 255/300 Write block 2608 len 40960 data: 4 4 4 4 3 3 2 2 2 2
2454 256/300 Flush
2455 257/300 Flush
2456 258/300 Write block 3406 len 24576 data: 2 2 2 3 3 3
2457 259/300 Read block 4722 len 16384
2458 260/300 Write block 2013 len 16384 data: 2 3 3 3
2459 261/300 Read block 2427 len 16384
2460 262/300 Write block 4426 len 36864 data: 2 2 2 2 2 2 3 3 3
2461 263/300 Read block 154 len 24576
2462 264/300 Write block 4652 len 8192 data: 3 2
2463 265/300 Read block 3648 len 24576
2464 266/300 Read block 1621 len 28672
2465 267/300 Read block 777 len 36864
2466 268/300 Write block 2167 len 28672 data: 4 3 3 3 3 3 3
2467 269/300 Flush
2468 270/300 Flush
2469 271/300 Write block 1070 len 36864 data: 3 3 3 3 3 2 2 2 2
2470 272/300 Write block 1373 len 32768 data: 2 2 2 2 2 2 2 2
2471 273/300 Read block 743 len 4096
2472 274/300 Flush
2473 275/300 Write block 1512 len 8192 data: 2 2
2474 276/300 Read block 1159 len 16384
2475 277/300 Read block 44 len 20480
2476 278/300 Read block 2558 len 20480
2477 279/300 Write block 1737 len 4096 data: 3
2478 280/300 Read block 2807 len 8192
2479 281/300 Read block 4209 len 36864
2480 282/300 Read block 4531 len 36864
2481 283/300 Read block 2528 len 4096
2482 284/300 Read block 3325 len 16384
2483 285/300 Write block 471 len 36864 data: 4 4 4 4 4 4 4 3 2
2484 286/300 Read block 857 len 36864
2485 287/300 Read block 3671 len 8192
2486 288/300 Flush
2487 289/300 Write block 495 len 28672 data: 2 2 2 2 2 2 2
2488 290/300 Write block 4933 len 32768 data: 4 3 3 3 3 3 3 3
2489 291/300 Write block 1996 len 4096 data: 2
2490 292/300 Read block 2667 len 40960
2491 293/300 Write block 3953 len 28672 data: 2 2 2 2 2 2 2
2492 294/300 Read block 4546 len 28672
2493 295/300 Flush
2494 296/300 Read block 3858 len 4096
2495 297/300 Write block 1783 len 28672 data: 2 2 2 2 2 2 2
2496 298/300 Write block 2705 len 20480 data: 2 2 2 2 2
2497 299/300 Read block 497 len 12288
2498 300/300 Read block 3325 len 20480
2499 301/300 Read block 1837 len 32768
2500 Main task has work to do, go find it
2501 got dsc Start(0)
2502 start 0
2503 Nov 19 17:08:32.445 INFO request completed, latency_us: 184, response_code: 204, uri: /start/cid/0, method: POST, req_id: 635da767-35c9-4e20-80eb-2c332a0100a2, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
2504 Replay: started 0, returned:Ok(())
2505 ----------------------------------------------------------------
2506 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2507 States: Offline Active Active
2508 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2509 1604 Acked Write 10 Sent Done Done false
2510 1605 Acked Read 8 Sent Done Done false
2511 1606 Acked Write 7 Sent Done Done false
2512 1607 Acked Flush 0 Sent Done Done false
2513 1608 Acked Read 9 Sent Done Done false
2514 1609 Acked Write 4 Sent Done Done false
2515 1610 Acked Write 10 Sent Done Done false
2516 1611 Acked Read 1 Sent Done Done false
2517 1612 Acked Read 5 Sent Done Done false
2518 1613 Acked Read 3 Sent Done Done false
2519 1614 Acked Read 9 Sent Done Done false
2520 1615 Acked Read 2 Sent Done Done false
2521 1616 Acked Write 10 Sent Done Done false
2522 1617 Acked Write 1 Sent Done Done false
2523 1618 Acked Write 10 Sent Done Done false
2524 1619 Acked Write 8 Sent Done Done false
2525 1620 Acked Read 5 Sent Done Done false
2526 1621 Acked Read 7 Sent Done Done false
2527 1622 Acked Read 3 Sent Done Done false
2528 1623 Acked Write 4 Sent Done Done false
2529 1624 Acked Read 9 Sent Done Done false
2530 1625 Acked Write 3 Sent Done Done false
2531 1626 Acked Flush 0 Sent Done Done false
2532 1627 Acked Write 1 Sent Done Done false
2533 1628 Acked Write 5 Sent Done Done false
2534 1629 Acked Write 3 Sent Done Done false
2535 1630 Acked Flush 0 Sent Done Done false
2536 1631 Acked Write 7 Sent Done Done false
2537 1632 Acked Read 10 Sent Done Done false
2538 1633 Acked Write 10 Sent Done Done false
2539 1634 Acked Write 3 Sent Done Done false
2540 1635 Acked Read 1 Sent Done Done false
2541 1636 Acked Write 9 Sent Done Done false
2542 1637 Acked Read 3 Sent Done Done false
2543 1638 Acked Flush 0 Sent Done Done false
2544 1639 Acked Write 8 Sent Done Done false
2545 1640 Acked Write 7 Sent Done Done false
2546 1641 Acked Read 4 Sent Done Done false
2547 1642 Acked Read 4 Sent Done Done false
2548 1643 Acked Read 5 Sent Done Done false
2549 1644 Acked Read 3 Sent Done Done false
2550 1645 Acked Write 6 Sent Done Done false
2551 1646 Acked Read 9 Sent Done Done false
2552 1647 Acked Write 9 Sent Done Done false
2553 1648 Acked Read 10 Sent Done Done false
2554 1649 Acked Read 2 Sent Done Done false
2555 1650 Acked Read 8 Sent Done Done false
2556 1651 Acked Read 8 Sent Done Done false
2557 1652 Acked Read 1 Sent Done Done false
2558 1653 Acked Read 2 Sent Done Done false
2559 1654 Acked Read 5 Sent Done Done false
2560 1655 Acked Write 8 Sent Done Done false
2561 1656 Acked Write 9 Sent Done Done false
2562 1657 Acked Read 1 Sent Done Done false
2563 1658 Acked Write 1 Sent Done Done false
2564 1659 Acked Read 4 Sent Done Done false
2565 1660 Acked Write 8 Sent Done Done false
2566 1661 Acked Read 3 Sent Done Done false
2567 1662 Acked Read 3 Sent Done Done false
2568 1663 Acked Read 8 Sent Done Done false
2569 1664 Acked Write 7 Sent Done Done false
2570 1665 Acked Write 7 Sent Done Done false
2571 1666 Acked Read 9 Sent Done Done false
2572 1667 Acked Flush 0 Sent Done Done false
2573 1668 Acked Read 10 Sent Done Done false
2574 1669 Acked Read 4 Sent Done Done false
2575 1670 Acked Write 3 Sent Done Done false
2576 1671 Acked Write 1 Sent Done Done false
2577 1672 Acked Read 10 Sent Done Done false
2578 1673 Acked Read 5 Sent Done Done false
2579 1674 Acked Write 10 Sent Done Done false
2580 1675 Acked Write 8 Sent Done Done false
2581 1676 Acked Write 6 Sent Done Done false
2582 1677 Acked Read 3 Sent Done Done false
2583 1678 Acked Flush 0 Sent Done Done false
2584 1679 Acked Read 1 Sent Done Done false
2585 1680 Acked Write 7 Sent Done Done false
2586 1681 Acked Read 3 Sent Done Done false
2587 1682 Acked Read 3 Sent Done Done false
2588 1683 Acked Write 6 Sent Done Done false
2589 1684 Acked Write 10 Sent Done Done false
2590 1685 Acked Read 8 Sent Done Done false
2591 1686 Acked Write 3 Sent Done Done false
2592 1687 Acked Read 9 Sent Done Done false
2593 1688 Acked Write 3 Sent Done Done false
2594 1689 Acked Read 7 Sent Done Done false
2595 1690 Acked Flush 0 Sent Done Done false
2596 1691 Acked Write 6 Sent Done Done false
2597 1692 Acked Read 6 Sent Done Done false
2598 1693 Acked Read 2 Sent Done Done false
2599 1694 Acked Read 5 Sent Done Done false
2600 1695 Acked Read 3 Sent Done Done false
2601 1696 Acked Write 6 Sent Done Done false
2602 1697 Acked Flush 0 Sent Done Done false
2603 1698 Acked Read 5 Sent Done Done false
2604 1699 Acked Write 9 Sent Done Done false
2605 1700 Acked Read 2 Sent Done Done false
2606 1701 Acked Read 7 Sent Done Done false
2607 1702 Acked Flush 0 Sent Done Done false
2608 1703 Acked Write 8 Sent Done Done false
2609 1704 Acked Write 1 Sent Done Done false
2610 1705 Acked Flush 0 Sent Done Done false
2611 1706 Acked Read 8 Sent Done Done false
2612 1707 Acked Read 2 Sent Done Done false
2613 1708 Acked Read 8 Sent Done Done false
2614 1709 Acked Write 8 Sent Done Done false
2615 1710 Acked Read 4 Sent Done Done false
2616 1711 Acked Write 6 Sent Done Done false
2617 1712 Acked Read 1 Sent Done Done false
2618 1713 Acked Read 6 Sent Done Done false
2619 1714 Acked Write 6 Sent Done Done false
2620 1715 Acked Read 7 Sent Done Done false
2621 1716 Acked Write 3 Sent Done Done false
2622 1717 Acked Flush 0 Sent Done Done false
2623 1718 Acked Write 10 Sent Done Done false
2624 1719 Acked Flush 0 Sent Done Done false
2625 1720 Acked Read 6 Sent Done Done false
2626 1721 Acked Read 5 Sent Done Done false
2627 1722 Acked Write 3 Sent Done Done false
2628 1723 Acked Read 6 Sent Done Done false
2629 1724 Acked Read 2 Sent Done Done false
2630 1725 Acked Read 4 Sent Done Done false
2631 1726 Acked Read 6 Sent Done Done false
2632 1727 Acked Write 9 Sent Done Done false
2633 1728 Acked Write 9 Sent Done Done false
2634 1729 Acked Read 2 Sent Done Done false
2635 1730 Acked Read 4 Sent Done Done false
2636 1731 Acked Read 3 Sent Done Done false
2637 1732 Acked Read 8 Sent Done Done false
2638 1733 Acked Read 2 Sent Done Done false
2639 1734 Acked Read 10 Sent Done Done false
2640 1735 Acked Read 9 Sent Done Done false
2641 1736 Acked Write 5 Sent Done Done false
2642 1737 Acked Read 10 Sent Done Done false
2643 1738 Acked Write 10 Sent Done Done false
2644 1739 Acked Write 10 Sent Done Done false
2645 1740 Acked Read 6 Sent Done Done false
2646 1741 Acked Read 3 Sent Done Done false
2647 1742 Acked Write 3 Sent Done Done false
2648 1743 Acked Write 3 Sent Done Done false
2649 1744 Acked Write 9 Sent Done Done false
2650 1745 Acked Write 3 Sent Done Done false
2651 1746 Acked Flush 0 Sent Done Done false
2652 1747 Acked Write 5 Sent Done Done false
2653 1748 Acked Read 2 Sent Done Done false
2654 1749 Acked Read 5 Sent Done Done false
2655 1750 Acked Write 9 Sent Done Done false
2656 1751 Acked Read 8 Sent Done Done false
2657 1752 Acked Read 3 Sent Done Done false
2658 1753 Acked Write 7 Sent Done Done false
2659 1754 Acked Write 7 Sent Done Done false
2660 1755 Acked Read 1 Sent Done Done false
2661 1756 Acked Write 9 Sent Done Done false
2662 1757 Acked Read 2 Sent Done Done false
2663 1758 Acked Write 4 Sent Done Done false
2664 1759 Acked Read 8 Sent Done Done false
2665 1760 Acked Write 10 Sent Done Done false
2666 1761 Acked Write 6 Sent Done Done false
2667 1762 Acked Read 8 Sent Done Done false
2668 1763 Acked Read 7 Sent Done Done false
2669 1764 Acked Read 3 Sent Done Done false
2670 1765 Acked Write 10 Sent Done Done false
2671 1766 Acked Read 9 Sent Done Done false
2672 1767 Acked Write 10 Sent Done Done false
2673 1768 Acked Write 3 Sent Done Done false
2674 1769 Acked Read 2 Sent Done Done false
2675 1770 Acked Read 5 Sent Done Done false
2676 1771 Acked Read 10 Sent Done Done false
2677 1772 Acked Write 9 Sent Done Done false
2678 1773 Acked Read 5 Sent Done Done false
2679 1774 Acked Write 10 Sent Done Done false
2680 1775 Acked Read 9 Sent Done Done false
2681 1776 Acked Write 5 Sent Done Done false
2682 1777 Acked Flush 0 Sent Done Done false
2683 1778 Acked Read 3 Sent Done Done false
2684 1779 Acked Write 1 Sent Done Done false
2685 1780 Acked Write 9 Sent Done Done false
2686 1781 Acked Read 7 Sent Done Done false
2687 1782 Acked Read 3 Sent Done Done false
2688 1783 Acked Write 8 Sent Done Done false
2689 1784 Acked Write 8 Sent Done Done false
2690 1785 Acked Flush 0 Sent Done Done false
2691 1786 Acked Write 10 Sent Done Done false
2692 1787 Acked Write 7 Sent Done Done false
2693 1788 Acked Write 6 Sent Done Done false
2694 1789 Acked Flush 0 Sent Done Done false
2695 1790 Acked Read 6 Sent Done Done false
2696 1791 Acked Flush 0 Sent Done Done false
2697 1792 Acked Write 5 Sent Done Done false
2698 1793 Acked Write 4 Sent Done Done false
2699 1794 Acked Write 9 Sent Done Done false
2700 1795 Acked Write 7 Sent Done Done false
2701 1796 Acked Read 3 Sent Done Done false
2702 1797 Acked Read 1 Sent Done Done false
2703 1798 Acked Write 3 Sent Done Done false
2704 1799 Acked Read 9 Sent Done Done false
2705 1800 Acked Write 2 Sent Done Done false
2706 1801 Acked Write 1 Sent Done Done false
2707 1802 Acked Write 7 Sent Done Done false
2708 1803 Acked Read 1 Sent Done Done false
2709 1804 Acked Read 3 Sent Done Done false
2710 1805 Acked Read 9 Sent Done Done false
2711 1806 Acked Write 10 Sent Done Done false
2712 1807 Acked Write 7 Sent Done Done false
2713 1808 Acked Read 3 Sent Done Done false
2714 1809 Acked Read 6 Sent Done Done false
2715 1810 Acked Read 10 Sent Done Done false
2716 1811 Acked Write 10 Sent Done Done false
2717 1812 Acked Read 7 Sent Done Done false
2718 1813 Acked Read 7 Sent Done Done false
2719 1814 Acked Write 2 Sent Done Done false
2720 1815 Acked Flush 0 Sent Done Done false
2721 1816 Acked Write 1 Sent Done Done false
2722 1817 Acked Write 9 Sent Done Done false
2723 1818 Acked Write 2 Sent Done Done false
2724 1819 Acked Read 9 Sent Done Done false
2725 1820 Acked Flush 0 Sent Done Done false
2726 1821 Acked Flush 0 Sent Done Done false
2727 1822 Acked Read 7 Sent Done Done false
2728 1823 Acked Read 8 Sent Done Done false
2729 1824 Acked Write 3 Sent Done Done false
2730 1825 Acked Flush 0 Sent Done Done false
2731 1826 Acked Read 5 Sent Done Done false
2732 1827 Acked Read 4 Sent Done Done false
2733 1828 Acked Read 2 Sent Done Done false
2734 1829 Acked Flush 0 Sent Done Done false
2735 1830 Acked Flush 0 Sent Done Done false
2736 1831 Acked Read 7 Sent Done Done false
2737 1832 Acked Write 10 Sent Done Done false
2738 1833 Acked Read 10 Sent Done Done false
2739 1834 Acked Read 1 Sent Done Done false
2740 1835 Acked Write 3 Sent Done Done false
2741 1836 Acked Read 6 Sent Done Done false
2742 1837 Acked Read 4 Sent Done Done false
2743 1838 Acked Read 6 Sent Done Done false
2744 1839 Acked Write 5 Sent Done Done false
2745 1840 Acked Read 6 Sent Done Done false
2746 1841 Acked Read 5 Sent Done Done false
2747 1842 Acked Read 3 Sent Done Done false
2748 1843 Acked Read 9 Sent Done Done false
2749 1844 Acked Write 8 Sent Done Done false
2750 1845 Acked Read 1 Sent Done Done false
2751 1846 Acked Write 4 Sent Done Done false
2752 1847 Acked Read 2 Sent Done Done false
2753 1848 Acked Read 2 Sent Done Done false
2754 1849 Acked Write 8 Sent Done Done false
2755 1850 Acked Write 9 Sent Done Done false
2756 1851 Acked Read 1 Sent Done Done false
2757 1852 Acked Flush 0 Sent Done Done false
2758 1853 Acked Write 3 Sent Done Done false
2759 1854 Acked Flush 0 Sent Done Done false
2760 1855 Acked Write 3 Sent Done Done false
2761 1856 Acked Flush 0 Sent Done Done false
2762 1857 Acked Write 9 Sent Done Done false
2763 1858 Acked Write 10 Sent Done Done false
2764 1859 Acked Flush 0 Sent Done Done false
2765 1860 Acked Flush 0 Sent Done Done false
2766 1861 Acked Write 6 Sent Done Done false
2767 1862 Acked Read 4 Sent Done Done false
2768 1863 Acked Write 4 Sent Done Done false
2769 1864 Acked Read 4 Sent Done Done false
2770 1865 Acked Write 9 Sent Done Done false
2771 1866 Acked Read 6 Sent Done Done false
2772 1867 Acked Write 2 Sent Done Done false
2773 1868 Acked Read 6 Sent Done Done false
2774 1869 Acked Read 7 Sent Done Done false
2775 1870 Acked Read 9 Sent Done Done false
2776 1871 Acked Write 7 Sent Done Done false
2777 1872 Acked Flush 0 Sent Done Done false
2778 1873 Acked Flush 0 Sent Done Done false
2779 1874 Acked Write 9 Sent Done Done false
2780 1875 Acked Write 8 Sent Done Done false
2781 1876 Acked Read 1 Sent Done Done false
2782 1877 Acked Flush 0 Sent Done Done false
2783 1878 Acked Write 2 Sent Done Done false
2784 1879 Acked Read 4 Sent Done Done false
2785 1880 Acked Read 5 Sent Done Done false
2786 1881 Acked Read 5 Sent Done Done false
2787 1882 Acked Write 1 Sent Done Done false
2788 1883 Acked Read 2 Sent Done Done false
2789 1884 Acked Read 9 Sent Done Done false
2790 1885 Acked Read 9 Sent Done Done false
2791 1886 Acked Read 1 Sent Done Done false
2792 1887 Acked Read 4 Sent Done Done false
2793 1888 Acked Write 9 Sent Done Done false
2794 1889 Acked Read 9 Sent Done Done false
2795 1890 Acked Read 2 Sent Done Done false
2796 1891 Acked Flush 0 Sent Done Done false
2797 1892 Acked Write 7 Sent Done Done false
2798 1893 Acked Write 8 Sent Done Done false
2799 1894 Acked Write 1 Sent Done Done false
2800 1895 Acked Read 10 Sent Done Done false
2801 1896 Acked Write 7 Sent Done Done false
2802 1897 Acked Read 7 Sent Done Done false
2803 1898 Acked Flush 0 Sent Done Done false
2804 1899 Acked Read 1 Sent Done Done false
2805 1900 Acked Write 7 Sent Done Done false
2806 1901 Acked Write 5 Sent Done Done false
2807 1902 Acked Read 3 Sent Done Done false
2808 1903 Acked Read 5 Sent Done Done false
2809 1904 Acked Read 8 Sent Done Done false
2810 STATES DS:0 DS:1 DS:2 TOTAL
2811 Sent 301 0 0 301
2812 Done 0 301 301 602
2813 Skipped 0 0 0 0
2814 Error 0 0 0 0
2815 Last Flush: 1603 1898 1898
2816 Downstairs last five completed: 1603 1602 1601 1600 1599
2817 Upstairs last five completed: 1904 1903 1902 1901 1900
2818 CLIENT: Up:0 ds:301 act:2
2819 ----------------------------------------------------------------
2820 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2821 States: Offline Active Active
2822 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2823 1604 Acked Write 10 Sent Done Done false
2824 1605 Acked Read 8 Sent Done Done false
2825 1606 Acked Write 7 Sent Done Done false
2826 1607 Acked Flush 0 Sent Done Done false
2827 1608 Acked Read 9 Sent Done Done false
2828 1609 Acked Write 4 Sent Done Done false
2829 1610 Acked Write 10 Sent Done Done false
2830 1611 Acked Read 1 Sent Done Done false
2831 1612 Acked Read 5 Sent Done Done false
2832 1613 Acked Read 3 Sent Done Done false
2833 1614 Acked Read 9 Sent Done Done false
2834 1615 Acked Read 2 Sent Done Done false
2835 1616 Acked Write 10 Sent Done Done false
2836 1617 Acked Write 1 Sent Done Done false
2837 1618 Acked Write 10 Sent Done Done false
2838 1619 Acked Write 8 Sent Done Done false
2839 1620 Acked Read 5 Sent Done Done false
2840 1621 Acked Read 7 Sent Done Done false
2841 1622 Acked Read 3 Sent Done Done false
2842 1623 Acked Write 4 Sent Done Done false
2843 1624 Acked Read 9 Sent Done Done false
2844 1625 Acked Write 3 Sent Done Done false
2845 1626 Acked Flush 0 Sent Done Done false
2846 1627 Acked Write 1 Sent Done Done false
2847 1628 Acked Write 5 Sent Done Done false
2848 1629 Acked Write 3 Sent Done Done false
2849 1630 Acked Flush 0 Sent Done Done false
2850 1631 Acked Write 7 Sent Done Done false
2851 1632 Acked Read 10 Sent Done Done false
2852 1633 Acked Write 10 Sent Done Done false
2853 1634 Acked Write 3 Sent Done Done false
2854 1635 Acked Read 1 Sent Done Done false
2855 1636 Acked Write 9 Sent Done Done false
2856 1637 Acked Read 3 Sent Done Done false
2857 1638 Acked Flush 0 Sent Done Done false
2858 1639 Acked Write 8 Sent Done Done false
2859 1640 Acked Write 7 Sent Done Done false
2860 1641 Acked Read 4 Sent Done Done false
2861 1642 Acked Read 4 Sent Done Done false
2862 1643 Acked Read 5 Sent Done Done false
2863 1644 Acked Read 3 Sent Done Done false
2864 1645 Acked Write 6 Sent Done Done false
2865 1646 Acked Read 9 Sent Done Done false
2866 1647 Acked Write 9 Sent Done Done false
2867 1648 Acked Read 10 Sent Done Done false
2868 1649 Acked Read 2 Sent Done Done false
2869 1650 Acked Read 8 Sent Done Done false
2870 1651 Acked Read 8 Sent Done Done false
2871 1652 Acked Read 1 Sent Done Done false
2872 1653 Acked Read 2 Sent Done Done false
2873 1654 Acked Read 5 Sent Done Done false
2874 1655 Acked Write 8 Sent Done Done false
2875 1656 Acked Write 9 Sent Done Done false
2876 1657 Acked Read 1 Sent Done Done false
2877 1658 Acked Write 1 Sent Done Done false
2878 1659 Acked Read 4 Sent Done Done false
2879 1660 Acked Write 8 Sent Done Done false
2880 1661 Acked Read 3 Sent Done Done false
2881 1662 Acked Read 3 Sent Done Done false
2882 1663 Acked Read 8 Sent Done Done false
2883 1664 Acked Write 7 Sent Done Done false
2884 1665 Acked Write 7 Sent Done Done false
2885 1666 Acked Read 9 Sent Done Done false
2886 1667 Acked Flush 0 Sent Done Done false
2887 1668 Acked Read 10 Sent Done Done false
2888 1669 Acked Read 4 Sent Done Done false
2889 1670 Acked Write 3 Sent Done Done false
2890 1671 Acked Write 1 Sent Done Done false
2891 1672 Acked Read 10 Sent Done Done false
2892 1673 Acked Read 5 Sent Done Done false
2893 1674 Acked Write 10 Sent Done Done false
2894 1675 Acked Write 8 Sent Done Done false
2895 1676 Acked Write 6 Sent Done Done false
2896 1677 Acked Read 3 Sent Done Done false
2897 1678 Acked Flush 0 Sent Done Done false
2898 1679 Acked Read 1 Sent Done Done false
2899 1680 Acked Write 7 Sent Done Done false
2900 1681 Acked Read 3 Sent Done Done false
2901 1682 Acked Read 3 Sent Done Done false
2902 1683 Acked Write 6 Sent Done Done false
2903 1684 Acked Write 10 Sent Done Done false
2904 1685 Acked Read 8 Sent Done Done false
2905 1686 Acked Write 3 Sent Done Done false
2906 1687 Acked Read 9 Sent Done Done false
2907 1688 Acked Write 3 Sent Done Done false
2908 1689 Acked Read 7 Sent Done Done false
2909 1690 Acked Flush 0 Sent Done Done false
2910 1691 Acked Write 6 Sent Done Done false
2911 1692 Acked Read 6 Sent Done Done false
2912 1693 Acked Read 2 Sent Done Done false
2913 1694 Acked Read 5 Sent Done Done false
2914 1695 Acked Read 3 Sent Done Done false
2915 1696 Acked Write 6 Sent Done Done false
2916 1697 Acked Flush 0 Sent Done Done false
2917 1698 Acked Read 5 Sent Done Done false
2918 1699 Acked Write 9 Sent Done Done false
2919 1700 Acked Read 2 Sent Done Done false
2920 1701 Acked Read 7 Sent Done Done false
2921 1702 Acked Flush 0 Sent Done Done false
2922 1703 Acked Write 8 Sent Done Done false
2923 1704 Acked Write 1 Sent Done Done false
2924 1705 Acked Flush 0 Sent Done Done false
2925 1706 Acked Read 8 Sent Done Done false
2926 1707 Acked Read 2 Sent Done Done false
2927 1708 Acked Read 8 Sent Done Done false
2928 1709 Acked Write 8 Sent Done Done false
2929 1710 Acked Read 4 Sent Done Done false
2930 1711 Acked Write 6 Sent Done Done false
2931 1712 Acked Read 1 Sent Done Done false
2932 1713 Acked Read 6 Sent Done Done false
2933 1714 Acked Write 6 Sent Done Done false
2934 1715 Acked Read 7 Sent Done Done false
2935 1716 Acked Write 3 Sent Done Done false
2936 1717 Acked Flush 0 Sent Done Done false
2937 1718 Acked Write 10 Sent Done Done false
2938 1719 Acked Flush 0 Sent Done Done false
2939 1720 Acked Read 6 Sent Done Done false
2940 1721 Acked Read 5 Sent Done Done false
2941 1722 Acked Write 3 Sent Done Done false
2942 1723 Acked Read 6 Sent Done Done false
2943 1724 Acked Read 2 Sent Done Done false
2944 1725 Acked Read 4 Sent Done Done false
2945 1726 Acked Read 6 Sent Done Done false
2946 1727 Acked Write 9 Sent Done Done false
2947 1728 Acked Write 9 Sent Done Done false
2948 1729 Acked Read 2 Sent Done Done false
2949 1730 Acked Read 4 Sent Done Done false
2950 1731 Acked Read 3 Sent Done Done false
2951 1732 Acked Read 8 Sent Done Done false
2952 1733 Acked Read 2 Sent Done Done false
2953 1734 Acked Read 10 Sent Done Done false
2954 1735 Acked Read 9 Sent Done Done false
2955 1736 Acked Write 5 Sent Done Done false
2956 1737 Acked Read 10 Sent Done Done false
2957 1738 Acked Write 10 Sent Done Done false
2958 1739 Acked Write 10 Sent Done Done false
2959 1740 Acked Read 6 Sent Done Done false
2960 1741 Acked Read 3 Sent Done Done false
2961 1742 Acked Write 3 Sent Done Done false
2962 1743 Acked Write 3 Sent Done Done false
2963 1744 Acked Write 9 Sent Done Done false
2964 1745 Acked Write 3 Sent Done Done false
2965 1746 Acked Flush 0 Sent Done Done false
2966 1747 Acked Write 5 Sent Done Done false
2967 1748 Acked Read 2 Sent Done Done false
2968 1749 Acked Read 5 Sent Done Done false
2969 1750 Acked Write 9 Sent Done Done false
2970 1751 Acked Read 8 Sent Done Done false
2971 1752 Acked Read 3 Sent Done Done false
2972 1753 Acked Write 7 Sent Done Done false
2973 1754 Acked Write 7 Sent Done Done false
2974 1755 Acked Read 1 Sent Done Done false
2975 1756 Acked Write 9 Sent Done Done false
2976 1757 Acked Read 2 Sent Done Done false
2977 1758 Acked Write 4 Sent Done Done false
2978 1759 Acked Read 8 Sent Done Done false
2979 1760 Acked Write 10 Sent Done Done false
2980 1761 Acked Write 6 Sent Done Done false
2981 1762 Acked Read 8 Sent Done Done false
2982 1763 Acked Read 7 Sent Done Done false
2983 1764 Acked Read 3 Sent Done Done false
2984 1765 Acked Write 10 Sent Done Done false
2985 1766 Acked Read 9 Sent Done Done false
2986 1767 Acked Write 10 Sent Done Done false
2987 1768 Acked Write 3 Sent Done Done false
2988 1769 Acked Read 2 Sent Done Done false
2989 1770 Acked Read 5 Sent Done Done false
2990 1771 Acked Read 10 Sent Done Done false
2991 1772 Acked Write 9 Sent Done Done false
2992 1773 Acked Read 5 Sent Done Done false
2993 1774 Acked Write 10 Sent Done Done false
2994 1775 Acked Read 9 Sent Done Done false
2995 1776 Acked Write 5 Sent Done Done false
2996 1777 Acked Flush 0 Sent Done Done false
2997 1778 Acked Read 3 Sent Done Done false
2998 1779 Acked Write 1 Sent Done Done false
2999 1780 Acked Write 9 Sent Done Done false
3000 1781 Acked Read 7 Sent Done Done false
3001 1782 Acked Read 3 Sent Done Done false
3002 1783 Acked Write 8 Sent Done Done false
3003 1784 Acked Write 8 Sent Done Done false
3004 1785 Acked Flush 0 Sent Done Done false
3005 1786 Acked Write 10 Sent Done Done false
3006 1787 Acked Write 7 Sent Done Done false
3007 1788 Acked Write 6 Sent Done Done false
3008 1789 Acked Flush 0 Sent Done Done false
3009 1790 Acked Read 6 Sent Done Done false
3010 1791 Acked Flush 0 Sent Done Done false
3011 1792 Acked Write 5 Sent Done Done false
3012 1793 Acked Write 4 Sent Done Done false
3013 1794 Acked Write 9 Sent Done Done false
3014 1795 Acked Write 7 Sent Done Done false
3015 1796 Acked Read 3 Sent Done Done false
3016 1797 Acked Read 1 Sent Done Done false
3017 1798 Acked Write 3 Sent Done Done false
3018 1799 Acked Read 9 Sent Done Done false
3019 1800 Acked Write 2 Sent Done Done false
3020 1801 Acked Write 1 Sent Done Done false
3021 1802 Acked Write 7 Sent Done Done false
3022 1803 Acked Read 1 Sent Done Done false
3023 1804 Acked Read 3 Sent Done Done false
3024 1805 Acked Read 9 Sent Done Done false
3025 1806 Acked Write 10 Sent Done Done false
3026 1807 Acked Write 7 Sent Done Done false
3027 1808 Acked Read 3 Sent Done Done false
3028 1809 Acked Read 6 Sent Done Done false
3029 1810 Acked Read 10 Sent Done Done false
3030 1811 Acked Write 10 Sent Done Done false
3031 1812 Acked Read 7 Sent Done Done false
3032 1813 Acked Read 7 Sent Done Done false
3033 1814 Acked Write 2 Sent Done Done false
3034 1815 Acked Flush 0 Sent Done Done false
3035 1816 Acked Write 1 Sent Done Done false
3036 1817 Acked Write 9 Sent Done Done false
3037 1818 Acked Write 2 Sent Done Done false
3038 1819 Acked Read 9 Sent Done Done false
3039 1820 Acked Flush 0 Sent Done Done false
3040 1821 Acked Flush 0 Sent Done Done false
3041 1822 Acked Read 7 Sent Done Done false
3042 1823 Acked Read 8 Sent Done Done false
3043 1824 Acked Write 3 Sent Done Done false
3044 1825 Acked Flush 0 Sent Done Done false
3045 1826 Acked Read 5 Sent Done Done false
3046 1827 Acked Read 4 Sent Done Done false
3047 1828 Acked Read 2 Sent Done Done false
3048 1829 Acked Flush 0 Sent Done Done false
3049 1830 Acked Flush 0 Sent Done Done false
3050 1831 Acked Read 7 Sent Done Done false
3051 1832 Acked Write 10 Sent Done Done false
3052 1833 Acked Read 10 Sent Done Done false
3053 1834 Acked Read 1 Sent Done Done false
3054 1835 Acked Write 3 Sent Done Done false
3055 1836 Acked Read 6 Sent Done Done false
3056 1837 Acked Read 4 Sent Done Done false
3057 1838 Acked Read 6 Sent Done Done false
3058 1839 Acked Write 5 Sent Done Done false
3059 1840 Acked Read 6 Sent Done Done false
3060 1841 Acked Read 5 Sent Done Done false
3061 1842 Acked Read 3 Sent Done Done false
3062 1843 Acked Read 9 Sent Done Done false
3063 1844 Acked Write 8 Sent Done Done false
3064 1845 Acked Read 1 Sent Done Done false
3065 1846 Acked Write 4 Sent Done Done false
3066 1847 Acked Read 2 Sent Done Done false
3067 1848 Acked Read 2 Sent Done Done false
3068 1849 Acked Write 8 Sent Done Done false
3069 1850 Acked Write 9 Sent Done Done false
3070 1851 Acked Read 1 Sent Done Done false
3071 1852 Acked Flush 0 Sent Done Done false
3072 1853 Acked Write 3 Sent Done Done false
3073 1854 Acked Flush 0 Sent Done Done false
3074 1855 Acked Write 3 Sent Done Done false
3075 1856 Acked Flush 0 Sent Done Done false
3076 1857 Acked Write 9 Sent Done Done false
3077 1858 Acked Write 10 Sent Done Done false
3078 1859 Acked Flush 0 Sent Done Done false
3079 1860 Acked Flush 0 Sent Done Done false
3080 1861 Acked Write 6 Sent Done Done false
3081 1862 Acked Read 4 Sent Done Done false
3082 1863 Acked Write 4 Sent Done Done false
3083 1864 Acked Read 4 Sent Done Done false
3084 1865 Acked Write 9 Sent Done Done false
3085 1866 Acked Read 6 Sent Done Done false
3086 1867 Acked Write 2 Sent Done Done false
3087 1868 Acked Read 6 Sent Done Done false
3088 1869 Acked Read 7 Sent Done Done false
3089 1870 Acked Read 9 Sent Done Done false
3090 1871 Acked Write 7 Sent Done Done false
3091 1872 Acked Flush 0 Sent Done Done false
3092 1873 Acked Flush 0 Sent Done Done false
3093 1874 Acked Write 9 Sent Done Done false
3094 1875 Acked Write 8 Sent Done Done false
3095 1876 Acked Read 1 Sent Done Done false
3096 1877 Acked Flush 0 Sent Done Done false
3097 1878 Acked Write 2 Sent Done Done false
3098 1879 Acked Read 4 Sent Done Done false
3099 1880 Acked Read 5 Sent Done Done false
3100 1881 Acked Read 5 Sent Done Done false
3101 1882 Acked Write 1 Sent Done Done false
3102 1883 Acked Read 2 Sent Done Done false
3103 1884 Acked Read 9 Sent Done Done false
3104 1885 Acked Read 9 Sent Done Done false
3105 1886 Acked Read 1 Sent Done Done false
3106 1887 Acked Read 4 Sent Done Done false
3107 1888 Acked Write 9 Sent Done Done false
3108 1889 Acked Read 9 Sent Done Done false
3109 1890 Acked Read 2 Sent Done Done false
3110 1891 Acked Flush 0 Sent Done Done false
3111 1892 Acked Write 7 Sent Done Done false
3112 1893 Acked Write 8 Sent Done Done false
3113 1894 Acked Write 1 Sent Done Done false
3114 1895 Acked Read 10 Sent Done Done false
3115 1896 Acked Write 7 Sent Done Done false
3116 1897 Acked Read 7 Sent Done Done false
3117 1898 Acked Flush 0 Sent Done Done false
3118 1899 Acked Read 1 Sent Done Done false
3119 1900 Acked Write 7 Sent Done Done false
3120 1901 Acked Write 5 Sent Done Done false
3121 1902 Acked Read 3 Sent Done Done false
3122 1903 Acked Read 5 Sent Done Done false
3123 1904 Acked Read 8 Sent Done Done false
3124 1905 Acked Flush 0 Sent Done Done false
3125 STATES DS:0 DS:1 DS:2 TOTAL
3126 Sent 302 0 0 302
3127 Done 0 302 302 604
3128 Skipped 0 0 0 0
3129 Error 0 0 0 0
3130 Last Flush: 1603 1905 1905
3131 Downstairs last five completed: 1603 1602 1601 1600 1599
3132 Upstairs last five completed: 1905 1904 1903 1902 1901
3133 CLIENT: Up:0 ds:302 act:2
31342024-11-19T17:08:38.210ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
31352024-11-19T17:08:38.210ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
31362024-11-19T17:08:38.210ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
31372024-11-19T17:08:38.210ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 05650a15-902e-469c-91e5-cba45f07470e
31382024-11-19T17:08:38.210ZINFOcrucible: client task is exiting = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
3139 [8810] got start action
3140 [8810] I am going to restart
3141 Starting downstairs at port 8810
3142 Make output file at "/tmp/dsc/downstairs-8810.txt"
3143 [8810][0] reports Starting
3144 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1205)
3145 [8810][0] reports Running
3146 ----------------------------------------------------------------
3147 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3148 States: Offline Active Active
3149 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3150 1604 Acked Write 10 Sent Done Done false
3151 1605 Acked Read 8 Sent Done Done false
3152 1606 Acked Write 7 Sent Done Done false
3153 1607 Acked Flush 0 Sent Done Done false
3154 1608 Acked Read 9 Sent Done Done false
3155 1609 Acked Write 4 Sent Done Done false
3156 1610 Acked Write 10 Sent Done Done false
3157 1611 Acked Read 1 Sent Done Done false
3158 1612 Acked Read 5 Sent Done Done false
3159 1613 Acked Read 3 Sent Done Done false
3160 1614 Acked Read 9 Sent Done Done false
3161 1615 Acked Read 2 Sent Done Done false
3162 1616 Acked Write 10 Sent Done Done false
3163 1617 Acked Write 1 Sent Done Done false
3164 1618 Acked Write 10 Sent Done Done false
3165 1619 Acked Write 8 Sent Done Done false
3166 1620 Acked Read 5 Sent Done Done false
3167 1621 Acked Read 7 Sent Done Done false
3168 1622 Acked Read 3 Sent Done Done false
3169 1623 Acked Write 4 Sent Done Done false
3170 1624 Acked Read 9 Sent Done Done false
3171 1625 Acked Write 3 Sent Done Done false
3172 1626 Acked Flush 0 Sent Done Done false
3173 1627 Acked Write 1 Sent Done Done false
3174 1628 Acked Write 5 Sent Done Done false
3175 1629 Acked Write 3 Sent Done Done false
3176 1630 Acked Flush 0 Sent Done Done false
3177 1631 Acked Write 7 Sent Done Done false
3178 1632 Acked Read 10 Sent Done Done false
3179 1633 Acked Write 10 Sent Done Done false
3180 1634 Acked Write 3 Sent Done Done false
3181 1635 Acked Read 1 Sent Done Done false
3182 1636 Acked Write 9 Sent Done Done false
3183 1637 Acked Read 3 Sent Done Done false
3184 1638 Acked Flush 0 Sent Done Done false
3185 1639 Acked Write 8 Sent Done Done false
3186 1640 Acked Write 7 Sent Done Done false
3187 1641 Acked Read 4 Sent Done Done false
3188 1642 Acked Read 4 Sent Done Done false
3189 1643 Acked Read 5 Sent Done Done false
3190 1644 Acked Read 3 Sent Done Done false
3191 1645 Acked Write 6 Sent Done Done false
3192 1646 Acked Read 9 Sent Done Done false
3193 1647 Acked Write 9 Sent Done Done false
3194 1648 Acked Read 10 Sent Done Done false
3195 1649 Acked Read 2 Sent Done Done false
3196 1650 Acked Read 8 Sent Done Done false
3197 1651 Acked Read 8 Sent Done Done false
3198 1652 Acked Read 1 Sent Done Done false
3199 1653 Acked Read 2 Sent Done Done false
3200 1654 Acked Read 5 Sent Done Done false
3201 1655 Acked Write 8 Sent Done Done false
3202 1656 Acked Write 9 Sent Done Done false
3203 1657 Acked Read 1 Sent Done Done false
3204 1658 Acked Write 1 Sent Done Done false
3205 1659 Acked Read 4 Sent Done Done false
3206 1660 Acked Write 8 Sent Done Done false
3207 1661 Acked Read 3 Sent Done Done false
3208 1662 Acked Read 3 Sent Done Done false
3209 1663 Acked Read 8 Sent Done Done false
3210 1664 Acked Write 7 Sent Done Done false
3211 1665 Acked Write 7 Sent Done Done false
3212 1666 Acked Read 9 Sent Done Done false
3213 1667 Acked Flush 0 Sent Done Done false
3214 1668 Acked Read 10 Sent Done Done false
3215 1669 Acked Read 4 Sent Done Done false
3216 1670 Acked Write 3 Sent Done Done false
3217 1671 Acked Write 1 Sent Done Done false
3218 1672 Acked Read 10 Sent Done Done false
3219 1673 Acked Read 5 Sent Done Done false
3220 1674 Acked Write 10 Sent Done Done false
3221 1675 Acked Write 8 Sent Done Done false
3222 1676 Acked Write 6 Sent Done Done false
3223 1677 Acked Read 3 Sent Done Done false
3224 1678 Acked Flush 0 Sent Done Done false
3225 1679 Acked Read 1 Sent Done Done false
3226 1680 Acked Write 7 Sent Done Done false
3227 1681 Acked Read 3 Sent Done Done false
3228 1682 Acked Read 3 Sent Done Done false
3229 1683 Acked Write 6 Sent Done Done false
3230 1684 Acked Write 10 Sent Done Done false
3231 1685 Acked Read 8 Sent Done Done false
3232 1686 Acked Write 3 Sent Done Done false
3233 1687 Acked Read 9 Sent Done Done false
3234 1688 Acked Write 3 Sent Done Done false
3235 1689 Acked Read 7 Sent Done Done false
3236 1690 Acked Flush 0 Sent Done Done false
3237 1691 Acked Write 6 Sent Done Done false
3238 1692 Acked Read 6 Sent Done Done false
3239 1693 Acked Read 2 Sent Done Done false
3240 1694 Acked Read 5 Sent Done Done false
3241 1695 Acked Read 3 Sent Done Done false
3242 1696 Acked Write 6 Sent Done Done false
3243 1697 Acked Flush 0 Sent Done Done false
3244 1698 Acked Read 5 Sent Done Done false
3245 1699 Acked Write 9 Sent Done Done false
3246 1700 Acked Read 2 Sent Done Done false
3247 1701 Acked Read 7 Sent Done Done false
3248 1702 Acked Flush 0 Sent Done Done false
3249 1703 Acked Write 8 Sent Done Done false
3250 1704 Acked Write 1 Sent Done Done false
3251 1705 Acked Flush 0 Sent Done Done false
3252 1706 Acked Read 8 Sent Done Done false
3253 1707 Acked Read 2 Sent Done Done false
3254 1708 Acked Read 8 Sent Done Done false
3255 1709 Acked Write 8 Sent Done Done false
3256 1710 Acked Read 4 Sent Done Done false
3257 1711 Acked Write 6 Sent Done Done false
3258 1712 Acked Read 1 Sent Done Done false
3259 1713 Acked Read 6 Sent Done Done false
3260 1714 Acked Write 6 Sent Done Done false
3261 1715 Acked Read 7 Sent Done Done false
3262 1716 Acked Write 3 Sent Done Done false
3263 1717 Acked Flush 0 Sent Done Done false
3264 1718 Acked Write 10 Sent Done Done false
3265 1719 Acked Flush 0 Sent Done Done false
3266 1720 Acked Read 6 Sent Done Done false
3267 1721 Acked Read 5 Sent Done Done false
3268 1722 Acked Write 3 Sent Done Done false
3269 1723 Acked Read 6 Sent Done Done false
3270 1724 Acked Read 2 Sent Done Done false
3271 1725 Acked Read 4 Sent Done Done false
3272 1726 Acked Read 6 Sent Done Done false
3273 1727 Acked Write 9 Sent Done Done false
3274 1728 Acked Write 9 Sent Done Done false
3275 1729 Acked Read 2 Sent Done Done false
3276 1730 Acked Read 4 Sent Done Done false
3277 1731 Acked Read 3 Sent Done Done false
3278 1732 Acked Read 8 Sent Done Done false
3279 1733 Acked Read 2 Sent Done Done false
3280 1734 Acked Read 10 Sent Done Done false
3281 1735 Acked Read 9 Sent Done Done false
3282 1736 Acked Write 5 Sent Done Done false
3283 1737 Acked Read 10 Sent Done Done false
3284 1738 Acked Write 10 Sent Done Done false
3285 1739 Acked Write 10 Sent Done Done false
3286 1740 Acked Read 6 Sent Done Done false
3287 1741 Acked Read 3 Sent Done Done false
3288 1742 Acked Write 3 Sent Done Done false
3289 1743 Acked Write 3 Sent Done Done false
3290 1744 Acked Write 9 Sent Done Done false
3291 1745 Acked Write 3 Sent Done Done false
3292 1746 Acked Flush 0 Sent Done Done false
3293 1747 Acked Write 5 Sent Done Done false
3294 1748 Acked Read 2 Sent Done Done false
3295 1749 Acked Read 5 Sent Done Done false
3296 1750 Acked Write 9 Sent Done Done false
3297 1751 Acked Read 8 Sent Done Done false
3298 1752 Acked Read 3 Sent Done Done false
3299 1753 Acked Write 7 Sent Done Done false
3300 1754 Acked Write 7 Sent Done Done false
3301 1755 Acked Read 1 Sent Done Done false
3302 1756 Acked Write 9 Sent Done Done false
3303 1757 Acked Read 2 Sent Done Done false
3304 1758 Acked Write 4 Sent Done Done false
3305 1759 Acked Read 8 Sent Done Done false
3306 1760 Acked Write 10 Sent Done Done false
3307 1761 Acked Write 6 Sent Done Done false
3308 1762 Acked Read 8 Sent Done Done false
3309 1763 Acked Read 7 Sent Done Done false
3310 1764 Acked Read 3 Sent Done Done false
3311 1765 Acked Write 10 Sent Done Done false
3312 1766 Acked Read 9 Sent Done Done false
3313 1767 Acked Write 10 Sent Done Done false
3314 1768 Acked Write 3 Sent Done Done false
3315 1769 Acked Read 2 Sent Done Done false
3316 1770 Acked Read 5 Sent Done Done false
3317 1771 Acked Read 10 Sent Done Done false
3318 1772 Acked Write 9 Sent Done Done false
3319 1773 Acked Read 5 Sent Done Done false
3320 1774 Acked Write 10 Sent Done Done false
3321 1775 Acked Read 9 Sent Done Done false
3322 1776 Acked Write 5 Sent Done Done false
3323 1777 Acked Flush 0 Sent Done Done false
3324 1778 Acked Read 3 Sent Done Done false
3325 1779 Acked Write 1 Sent Done Done false
3326 1780 Acked Write 9 Sent Done Done false
3327 1781 Acked Read 7 Sent Done Done false
3328 1782 Acked Read 3 Sent Done Done false
3329 1783 Acked Write 8 Sent Done Done false
3330 1784 Acked Write 8 Sent Done Done false
3331 1785 Acked Flush 0 Sent Done Done false
3332 1786 Acked Write 10 Sent Done Done false
3333 1787 Acked Write 7 Sent Done Done false
3334 1788 Acked Write 6 Sent Done Done false
3335 1789 Acked Flush 0 Sent Done Done false
3336 1790 Acked Read 6 Sent Done Done false
3337 1791 Acked Flush 0 Sent Done Done false
3338 1792 Acked Write 5 Sent Done Done false
3339 1793 Acked Write 4 Sent Done Done false
3340 1794 Acked Write 9 Sent Done Done false
3341 1795 Acked Write 7 Sent Done Done false
3342 1796 Acked Read 3 Sent Done Done false
3343 1797 Acked Read 1 Sent Done Done false
3344 1798 Acked Write 3 Sent Done Done false
3345 1799 Acked Read 9 Sent Done Done false
3346 1800 Acked Write 2 Sent Done Done false
3347 1801 Acked Write 1 Sent Done Done false
3348 1802 Acked Write 7 Sent Done Done false
3349 1803 Acked Read 1 Sent Done Done false
3350 1804 Acked Read 3 Sent Done Done false
3351 1805 Acked Read 9 Sent Done Done false
3352 1806 Acked Write 10 Sent Done Done false
3353 1807 Acked Write 7 Sent Done Done false
3354 1808 Acked Read 3 Sent Done Done false
3355 1809 Acked Read 6 Sent Done Done false
3356 1810 Acked Read 10 Sent Done Done false
3357 1811 Acked Write 10 Sent Done Done false
3358 1812 Acked Read 7 Sent Done Done false
3359 1813 Acked Read 7 Sent Done Done false
3360 1814 Acked Write 2 Sent Done Done false
3361 1815 Acked Flush 0 Sent Done Done false
3362 1816 Acked Write 1 Sent Done Done false
3363 1817 Acked Write 9 Sent Done Done false
3364 1818 Acked Write 2 Sent Done Done false
3365 1819 Acked Read 9 Sent Done Done false
3366 1820 Acked Flush 0 Sent Done Done false
3367 1821 Acked Flush 0 Sent Done Done false
3368 1822 Acked Read 7 Sent Done Done false
3369 1823 Acked Read 8 Sent Done Done false
3370 1824 Acked Write 3 Sent Done Done false
3371 1825 Acked Flush 0 Sent Done Done false
3372 1826 Acked Read 5 Sent Done Done false
3373 1827 Acked Read 4 Sent Done Done false
3374 1828 Acked Read 2 Sent Done Done false
3375 1829 Acked Flush 0 Sent Done Done false
3376 1830 Acked Flush 0 Sent Done Done false
3377 1831 Acked Read 7 Sent Done Done false
3378 1832 Acked Write 10 Sent Done Done false
3379 1833 Acked Read 10 Sent Done Done false
3380 1834 Acked Read 1 Sent Done Done false
3381 1835 Acked Write 3 Sent Done Done false
3382 1836 Acked Read 6 Sent Done Done false
3383 1837 Acked Read 4 Sent Done Done false
3384 1838 Acked Read 6 Sent Done Done false
3385 1839 Acked Write 5 Sent Done Done false
3386 1840 Acked Read 6 Sent Done Done false
3387 1841 Acked Read 5 Sent Done Done false
3388 1842 Acked Read 3 Sent Done Done false
3389 1843 Acked Read 9 Sent Done Done false
3390 1844 Acked Write 8 Sent Done Done false
3391 1845 Acked Read 1 Sent Done Done false
3392 1846 Acked Write 4 Sent Done Done false
3393 1847 Acked Read 2 Sent Done Done false
3394 1848 Acked Read 2 Sent Done Done false
3395 1849 Acked Write 8 Sent Done Done false
3396 1850 Acked Write 9 Sent Done Done false
3397 1851 Acked Read 1 Sent Done Done false
3398 1852 Acked Flush 0 Sent Done Done false
3399 1853 Acked Write 3 Sent Done Done false
3400 1854 Acked Flush 0 Sent Done Done false
3401 1855 Acked Write 3 Sent Done Done false
3402 1856 Acked Flush 0 Sent Done Done false
3403 1857 Acked Write 9 Sent Done Done false
3404 1858 Acked Write 10 Sent Done Done false
3405 1859 Acked Flush 0 Sent Done Done false
3406 1860 Acked Flush 0 Sent Done Done false
3407 1861 Acked Write 6 Sent Done Done false
3408 1862 Acked Read 4 Sent Done Done false
3409 1863 Acked Write 4 Sent Done Done false
3410 1864 Acked Read 4 Sent Done Done false
3411 1865 Acked Write 9 Sent Done Done false
3412 1866 Acked Read 6 Sent Done Done false
3413 1867 Acked Write 2 Sent Done Done false
3414 1868 Acked Read 6 Sent Done Done false
3415 1869 Acked Read 7 Sent Done Done false
3416 1870 Acked Read 9 Sent Done Done false
3417 1871 Acked Write 7 Sent Done Done false
3418 1872 Acked Flush 0 Sent Done Done false
3419 1873 Acked Flush 0 Sent Done Done false
3420 1874 Acked Write 9 Sent Done Done false
3421 1875 Acked Write 8 Sent Done Done false
3422 1876 Acked Read 1 Sent Done Done false
3423 1877 Acked Flush 0 Sent Done Done false
3424 1878 Acked Write 2 Sent Done Done false
3425 1879 Acked Read 4 Sent Done Done false
3426 1880 Acked Read 5 Sent Done Done false
3427 1881 Acked Read 5 Sent Done Done false
3428 1882 Acked Write 1 Sent Done Done false
3429 1883 Acked Read 2 Sent Done Done false
3430 1884 Acked Read 9 Sent Done Done false
3431 1885 Acked Read 9 Sent Done Done false
3432 1886 Acked Read 1 Sent Done Done false
3433 1887 Acked Read 4 Sent Done Done false
3434 1888 Acked Write 9 Sent Done Done false
3435 1889 Acked Read 9 Sent Done Done false
3436 1890 Acked Read 2 Sent Done Done false
3437 1891 Acked Flush 0 Sent Done Done false
3438 1892 Acked Write 7 Sent Done Done false
3439 1893 Acked Write 8 Sent Done Done false
3440 1894 Acked Write 1 Sent Done Done false
3441 1895 Acked Read 10 Sent Done Done false
3442 1896 Acked Write 7 Sent Done Done false
3443 1897 Acked Read 7 Sent Done Done false
3444 1898 Acked Flush 0 Sent Done Done false
3445 1899 Acked Read 1 Sent Done Done false
3446 1900 Acked Write 7 Sent Done Done false
3447 1901 Acked Write 5 Sent Done Done false
3448 1902 Acked Read 3 Sent Done Done false
3449 1903 Acked Read 5 Sent Done Done false
3450 1904 Acked Read 8 Sent Done Done false
3451 1905 Acked Flush 0 Sent Done Done false
3452 STATES DS:0 DS:1 DS:2 TOTAL
3453 Sent 302 0 0 302
3454 Done 0 302 302 604
3455 Skipped 0 0 0 0
3456 Error 0 0 0 0
3457 Last Flush: 1603 1905 1905
3458 Downstairs last five completed: 1603 1602 1601 1600 1599
3459 Upstairs last five completed: 1905 1904 1903 1902 1901
3460 CLIENT: Up:0 ds:302 act:2
3461 ----------------------------------------------------------------
3462 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3463 States: Offline Active Active
3464 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3465 1604 Acked Write 10 Sent Done Done false
3466 1605 Acked Read 8 Sent Done Done false
3467 1606 Acked Write 7 Sent Done Done false
3468 1607 Acked Flush 0 Sent Done Done false
3469 1608 Acked Read 9 Sent Done Done false
3470 1609 Acked Write 4 Sent Done Done false
3471 1610 Acked Write 10 Sent Done Done false
3472 1611 Acked Read 1 Sent Done Done false
3473 1612 Acked Read 5 Sent Done Done false
3474 1613 Acked Read 3 Sent Done Done false
3475 1614 Acked Read 9 Sent Done Done false
3476 1615 Acked Read 2 Sent Done Done false
3477 1616 Acked Write 10 Sent Done Done false
3478 1617 Acked Write 1 Sent Done Done false
3479 1618 Acked Write 10 Sent Done Done false
3480 1619 Acked Write 8 Sent Done Done false
3481 1620 Acked Read 5 Sent Done Done false
3482 1621 Acked Read 7 Sent Done Done false
3483 1622 Acked Read 3 Sent Done Done false
3484 1623 Acked Write 4 Sent Done Done false
3485 1624 Acked Read 9 Sent Done Done false
3486 1625 Acked Write 3 Sent Done Done false
3487 1626 Acked Flush 0 Sent Done Done false
3488 1627 Acked Write 1 Sent Done Done false
3489 1628 Acked Write 5 Sent Done Done false
3490 1629 Acked Write 3 Sent Done Done false
3491 1630 Acked Flush 0 Sent Done Done false
3492 1631 Acked Write 7 Sent Done Done false
3493 1632 Acked Read 10 Sent Done Done false
3494 1633 Acked Write 10 Sent Done Done false
3495 1634 Acked Write 3 Sent Done Done false
3496 1635 Acked Read 1 Sent Done Done false
3497 1636 Acked Write 9 Sent Done Done false
3498 1637 Acked Read 3 Sent Done Done false
3499 1638 Acked Flush 0 Sent Done Done false
3500 1639 Acked Write 8 Sent Done Done false
3501 1640 Acked Write 7 Sent Done Done false
3502 1641 Acked Read 4 Sent Done Done false
3503 1642 Acked Read 4 Sent Done Done false
3504 1643 Acked Read 5 Sent Done Done false
3505 1644 Acked Read 3 Sent Done Done false
3506 1645 Acked Write 6 Sent Done Done false
3507 1646 Acked Read 9 Sent Done Done false
3508 1647 Acked Write 9 Sent Done Done false
3509 1648 Acked Read 10 Sent Done Done false
3510 1649 Acked Read 2 Sent Done Done false
3511 1650 Acked Read 8 Sent Done Done false
3512 1651 Acked Read 8 Sent Done Done false
3513 1652 Acked Read 1 Sent Done Done false
3514 1653 Acked Read 2 Sent Done Done false
3515 1654 Acked Read 5 Sent Done Done false
3516 1655 Acked Write 8 Sent Done Done false
3517 1656 Acked Write 9 Sent Done Done false
3518 1657 Acked Read 1 Sent Done Done false
3519 1658 Acked Write 1 Sent Done Done false
3520 1659 Acked Read 4 Sent Done Done false
3521 1660 Acked Write 8 Sent Done Done false
3522 1661 Acked Read 3 Sent Done Done false
3523 1662 Acked Read 3 Sent Done Done false
3524 1663 Acked Read 8 Sent Done Done false
3525 1664 Acked Write 7 Sent Done Done false
3526 1665 Acked Write 7 Sent Done Done false
3527 1666 Acked Read 9 Sent Done Done false
3528 1667 Acked Flush 0 Sent Done Done false
3529 1668 Acked Read 10 Sent Done Done false
3530 1669 Acked Read 4 Sent Done Done false
3531 1670 Acked Write 3 Sent Done Done false
3532 1671 Acked Write 1 Sent Done Done false
3533 1672 Acked Read 10 Sent Done Done false
3534 1673 Acked Read 5 Sent Done Done false
3535 1674 Acked Write 10 Sent Done Done false
3536 1675 Acked Write 8 Sent Done Done false
3537 1676 Acked Write 6 Sent Done Done false
3538 1677 Acked Read 3 Sent Done Done false
3539 1678 Acked Flush 0 Sent Done Done false
3540 1679 Acked Read 1 Sent Done Done false
3541 1680 Acked Write 7 Sent Done Done false
3542 1681 Acked Read 3 Sent Done Done false
3543 1682 Acked Read 3 Sent Done Done false
3544 1683 Acked Write 6 Sent Done Done false
3545 1684 Acked Write 10 Sent Done Done false
3546 1685 Acked Read 8 Sent Done Done false
3547 1686 Acked Write 3 Sent Done Done false
3548 1687 Acked Read 9 Sent Done Done false
3549 1688 Acked Write 3 Sent Done Done false
3550 1689 Acked Read 7 Sent Done Done false
3551 1690 Acked Flush 0 Sent Done Done false
3552 1691 Acked Write 6 Sent Done Done false
3553 1692 Acked Read 6 Sent Done Done false
3554 1693 Acked Read 2 Sent Done Done false
3555 1694 Acked Read 5 Sent Done Done false
3556 1695 Acked Read 3 Sent Done Done false
3557 1696 Acked Write 6 Sent Done Done false
3558 1697 Acked Flush 0 Sent Done Done false
3559 1698 Acked Read 5 Sent Done Done false
3560 1699 Acked Write 9 Sent Done Done false
3561 1700 Acked Read 2 Sent Done Done false
3562 1701 Acked Read 7 Sent Done Done false
3563 1702 Acked Flush 0 Sent Done Done false
3564 1703 Acked Write 8 Sent Done Done false
3565 1704 Acked Write 1 Sent Done Done false
3566 1705 Acked Flush 0 Sent Done Done false
3567 1706 Acked Read 8 Sent Done Done false
3568 1707 Acked Read 2 Sent Done Done false
3569 1708 Acked Read 8 Sent Done Done false
3570 1709 Acked Write 8 Sent Done Done false
3571 1710 Acked Read 4 Sent Done Done false
3572 1711 Acked Write 6 Sent Done Done false
3573 1712 Acked Read 1 Sent Done Done false
3574 1713 Acked Read 6 Sent Done Done false
3575 1714 Acked Write 6 Sent Done Done false
3576 1715 Acked Read 7 Sent Done Done false
3577 1716 Acked Write 3 Sent Done Done false
3578 1717 Acked Flush 0 Sent Done Done false
3579 1718 Acked Write 10 Sent Done Done false
3580 1719 Acked Flush 0 Sent Done Done false
3581 1720 Acked Read 6 Sent Done Done false
3582 1721 Acked Read 5 Sent Done Done false
3583 1722 Acked Write 3 Sent Done Done false
3584 1723 Acked Read 6 Sent Done Done false
3585 1724 Acked Read 2 Sent Done Done false
3586 1725 Acked Read 4 Sent Done Done false
3587 1726 Acked Read 6 Sent Done Done false
3588 1727 Acked Write 9 Sent Done Done false
3589 1728 Acked Write 9 Sent Done Done false
3590 1729 Acked Read 2 Sent Done Done false
3591 1730 Acked Read 4 Sent Done Done false
3592 1731 Acked Read 3 Sent Done Done false
3593 1732 Acked Read 8 Sent Done Done false
3594 1733 Acked Read 2 Sent Done Done false
3595 1734 Acked Read 10 Sent Done Done false
3596 1735 Acked Read 9 Sent Done Done false
3597 1736 Acked Write 5 Sent Done Done false
3598 1737 Acked Read 10 Sent Done Done false
3599 1738 Acked Write 10 Sent Done Done false
3600 1739 Acked Write 10 Sent Done Done false
3601 1740 Acked Read 6 Sent Done Done false
3602 1741 Acked Read 3 Sent Done Done false
3603 1742 Acked Write 3 Sent Done Done false
3604 1743 Acked Write 3 Sent Done Done false
3605 1744 Acked Write 9 Sent Done Done false
3606 1745 Acked Write 3 Sent Done Done false
3607 1746 Acked Flush 0 Sent Done Done false
3608 1747 Acked Write 5 Sent Done Done false
3609 1748 Acked Read 2 Sent Done Done false
3610 1749 Acked Read 5 Sent Done Done false
3611 1750 Acked Write 9 Sent Done Done false
3612 1751 Acked Read 8 Sent Done Done false
3613 1752 Acked Read 3 Sent Done Done false
3614 1753 Acked Write 7 Sent Done Done false
3615 1754 Acked Write 7 Sent Done Done false
3616 1755 Acked Read 1 Sent Done Done false
3617 1756 Acked Write 9 Sent Done Done false
3618 1757 Acked Read 2 Sent Done Done false
3619 1758 Acked Write 4 Sent Done Done false
3620 1759 Acked Read 8 Sent Done Done false
3621 1760 Acked Write 10 Sent Done Done false
3622 1761 Acked Write 6 Sent Done Done false
3623 1762 Acked Read 8 Sent Done Done false
3624 1763 Acked Read 7 Sent Done Done false
3625 1764 Acked Read 3 Sent Done Done false
3626 1765 Acked Write 10 Sent Done Done false
3627 1766 Acked Read 9 Sent Done Done false
3628 1767 Acked Write 10 Sent Done Done false
3629 1768 Acked Write 3 Sent Done Done false
3630 1769 Acked Read 2 Sent Done Done false
3631 1770 Acked Read 5 Sent Done Done false
3632 1771 Acked Read 10 Sent Done Done false
3633 1772 Acked Write 9 Sent Done Done false
3634 1773 Acked Read 5 Sent Done Done false
3635 1774 Acked Write 10 Sent Done Done false
3636 1775 Acked Read 9 Sent Done Done false
3637 1776 Acked Write 5 Sent Done Done false
3638 1777 Acked Flush 0 Sent Done Done false
3639 1778 Acked Read 3 Sent Done Done false
3640 1779 Acked Write 1 Sent Done Done false
3641 1780 Acked Write 9 Sent Done Done false
3642 1781 Acked Read 7 Sent Done Done false
3643 1782 Acked Read 3 Sent Done Done false
3644 1783 Acked Write 8 Sent Done Done false
3645 1784 Acked Write 8 Sent Done Done false
3646 1785 Acked Flush 0 Sent Done Done false
3647 1786 Acked Write 10 Sent Done Done false
3648 1787 Acked Write 7 Sent Done Done false
3649 1788 Acked Write 6 Sent Done Done false
3650 1789 Acked Flush 0 Sent Done Done false
3651 1790 Acked Read 6 Sent Done Done false
3652 1791 Acked Flush 0 Sent Done Done false
3653 1792 Acked Write 5 Sent Done Done false
3654 1793 Acked Write 4 Sent Done Done false
3655 1794 Acked Write 9 Sent Done Done false
3656 1795 Acked Write 7 Sent Done Done false
3657 1796 Acked Read 3 Sent Done Done false
3658 1797 Acked Read 1 Sent Done Done false
3659 1798 Acked Write 3 Sent Done Done false
3660 1799 Acked Read 9 Sent Done Done false
3661 1800 Acked Write 2 Sent Done Done false
3662 1801 Acked Write 1 Sent Done Done false
3663 1802 Acked Write 7 Sent Done Done false
3664 1803 Acked Read 1 Sent Done Done false
3665 1804 Acked Read 3 Sent Done Done false
3666 1805 Acked Read 9 Sent Done Done false
3667 1806 Acked Write 10 Sent Done Done false
3668 1807 Acked Write 7 Sent Done Done false
3669 1808 Acked Read 3 Sent Done Done false
3670 1809 Acked Read 6 Sent Done Done false
3671 1810 Acked Read 10 Sent Done Done false
3672 1811 Acked Write 10 Sent Done Done false
3673 1812 Acked Read 7 Sent Done Done false
3674 1813 Acked Read 7 Sent Done Done false
3675 1814 Acked Write 2 Sent Done Done false
3676 1815 Acked Flush 0 Sent Done Done false
3677 1816 Acked Write 1 Sent Done Done false
3678 1817 Acked Write 9 Sent Done Done false
3679 1818 Acked Write 2 Sent Done Done false
3680 1819 Acked Read 9 Sent Done Done false
3681 1820 Acked Flush 0 Sent Done Done false
3682 1821 Acked Flush 0 Sent Done Done false
3683 1822 Acked Read 7 Sent Done Done false
3684 1823 Acked Read 8 Sent Done Done false
3685 1824 Acked Write 3 Sent Done Done false
3686 1825 Acked Flush 0 Sent Done Done false
3687 1826 Acked Read 5 Sent Done Done false
3688 1827 Acked Read 4 Sent Done Done false
3689 1828 Acked Read 2 Sent Done Done false
3690 1829 Acked Flush 0 Sent Done Done false
3691 1830 Acked Flush 0 Sent Done Done false
3692 1831 Acked Read 7 Sent Done Done false
3693 1832 Acked Write 10 Sent Done Done false
3694 1833 Acked Read 10 Sent Done Done false
3695 1834 Acked Read 1 Sent Done Done false
3696 1835 Acked Write 3 Sent Done Done false
3697 1836 Acked Read 6 Sent Done Done false
3698 1837 Acked Read 4 Sent Done Done false
3699 1838 Acked Read 6 Sent Done Done false
3700 1839 Acked Write 5 Sent Done Done false
3701 1840 Acked Read 6 Sent Done Done false
3702 1841 Acked Read 5 Sent Done Done false
3703 1842 Acked Read 3 Sent Done Done false
3704 1843 Acked Read 9 Sent Done Done false
3705 1844 Acked Write 8 Sent Done Done false
3706 1845 Acked Read 1 Sent Done Done false
3707 1846 Acked Write 4 Sent Done Done false
3708 1847 Acked Read 2 Sent Done Done false
3709 1848 Acked Read 2 Sent Done Done false
3710 1849 Acked Write 8 Sent Done Done false
3711 1850 Acked Write 9 Sent Done Done false
3712 1851 Acked Read 1 Sent Done Done false
3713 1852 Acked Flush 0 Sent Done Done false
3714 1853 Acked Write 3 Sent Done Done false
3715 1854 Acked Flush 0 Sent Done Done false
3716 1855 Acked Write 3 Sent Done Done false
3717 1856 Acked Flush 0 Sent Done Done false
3718 1857 Acked Write 9 Sent Done Done false
3719 1858 Acked Write 10 Sent Done Done false
3720 1859 Acked Flush 0 Sent Done Done false
3721 1860 Acked Flush 0 Sent Done Done false
3722 1861 Acked Write 6 Sent Done Done false
3723 1862 Acked Read 4 Sent Done Done false
3724 1863 Acked Write 4 Sent Done Done false
3725 1864 Acked Read 4 Sent Done Done false
3726 1865 Acked Write 9 Sent Done Done false
3727 1866 Acked Read 6 Sent Done Done false
3728 1867 Acked Write 2 Sent Done Done false
3729 1868 Acked Read 6 Sent Done Done false
3730 1869 Acked Read 7 Sent Done Done false
3731 1870 Acked Read 9 Sent Done Done false
3732 1871 Acked Write 7 Sent Done Done false
3733 1872 Acked Flush 0 Sent Done Done false
3734 1873 Acked Flush 0 Sent Done Done false
3735 1874 Acked Write 9 Sent Done Done false
3736 1875 Acked Write 8 Sent Done Done false
3737 1876 Acked Read 1 Sent Done Done false
3738 1877 Acked Flush 0 Sent Done Done false
3739 1878 Acked Write 2 Sent Done Done false
3740 1879 Acked Read 4 Sent Done Done false
3741 1880 Acked Read 5 Sent Done Done false
3742 1881 Acked Read 5 Sent Done Done false
3743 1882 Acked Write 1 Sent Done Done false
3744 1883 Acked Read 2 Sent Done Done false
3745 1884 Acked Read 9 Sent Done Done false
3746 1885 Acked Read 9 Sent Done Done false
3747 1886 Acked Read 1 Sent Done Done false
3748 1887 Acked Read 4 Sent Done Done false
3749 1888 Acked Write 9 Sent Done Done false
3750 1889 Acked Read 9 Sent Done Done false
3751 1890 Acked Read 2 Sent Done Done false
3752 1891 Acked Flush 0 Sent Done Done false
3753 1892 Acked Write 7 Sent Done Done false
3754 1893 Acked Write 8 Sent Done Done false
3755 1894 Acked Write 1 Sent Done Done false
3756 1895 Acked Read 10 Sent Done Done false
3757 1896 Acked Write 7 Sent Done Done false
3758 1897 Acked Read 7 Sent Done Done false
3759 1898 Acked Flush 0 Sent Done Done false
3760 1899 Acked Read 1 Sent Done Done false
3761 1900 Acked Write 7 Sent Done Done false
3762 1901 Acked Write 5 Sent Done Done false
3763 1902 Acked Read 3 Sent Done Done false
3764 1903 Acked Read 5 Sent Done Done false
3765 1904 Acked Read 8 Sent Done Done false
3766 1905 Acked Flush 0 Sent Done Done false
3767 STATES DS:0 DS:1 DS:2 TOTAL
3768 Sent 302 0 0 302
3769 Done 0 302 302 604
3770 Skipped 0 0 0 0
3771 Error 0 0 0 0
3772 Last Flush: 1603 1905 1905
3773 Downstairs last five completed: 1603 1602 1601 1600 1599
3774 Upstairs last five completed: 1905 1904 1903 1902 1901
3775 CLIENT: Up:0 ds:302 act:2
37762024-11-19T17:08:48.211ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37772024-11-19T17:08:48.212ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41358) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37782024-11-19T17:08:48.213ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37792024-11-19T17:08:48.213ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37802024-11-19T17:08:48.213ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37812024-11-19T17:08:48.213ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37822024-11-19T17:08:48.214ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37832024-11-19T17:08:48.214ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37842024-11-19T17:08:48.214ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
37852024-11-19T17:08:48.214ZINFOcrucible: [0] client re-new 302 jobs since flush 1603 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
37862024-11-19T17:08:48.214ZINFOcrucible: [0] Replayed 302 jobs since flush: 1603 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
3787 ----------------------------------------------------------------
3788 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3789 Downstairs last five completed: 1905 1904 1903 1902 1901
3790 Upstairs last five completed: 1905 1904 1903 1902 1901
3791 CLIENT: Up:0 ds:0 act:3
3792 Replay: All jobs finished
3793 Main task has work to do, go find it
3794 got dsc Stop(2)
3795 stop 2
3796 Nov 19 17:08:48.454 INFO request completed, latency_us: 347, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 18295b0f-a760-4aad-92ef-66acd6320712, remote_addr: 127.0.0.[8830] Got stop action so:false kr:false
3797 1:64175, local_addr: 127.0.0.1:9998
3798 Nov 19 17:08:48.454 INFO request completed, latency_us: 111, response_code: 200, uri: /state/cid/2, method: GET, req_id: 69b6e6ce-61d6-4a9f-bb58-a6da32e12166, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
37992024-11-19T17:08:48.454ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
38002024-11-19T17:08:48.454ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
38012024-11-19T17:08:48.454ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
38022024-11-19T17:08:48.454ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
38032024-11-19T17:08:48.454ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
38042024-11-19T17:08:48.454ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
3805 [8830] Exited with: ExitStatus(unix_wait_status(9))
3806 [8830][2] reports Exit
3807 Nov 19 17:08:52.457 INFO request completed, latency_us: 294, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9b42d68e-e820-492c-801f-6d77a3764069, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
3808 001/300 Write block 473 len 20480 data: 5 5 5 5 5
3809 002/300 Read block 2526 len 24576
3810 003/300 Read block 643 len 20480
3811 004/300 Write block 2253 len 20480 data: 2 2 2 2 2
3812 005/300 Write block 648 len 40960 data: 3 3 3 3 3 3 2 2 2 2
3813 006/300 Read block 2929 len 32768
3814 007/300 Read block 104 len 20480
3815 008/300 Flush
3816 009/300 Read block 2148 len 12288
3817 010/300 Read block 2580 len 40960
3818 011/300 Write block 4707 len 24576 data: 2 2 2 2 2 2
3819 012/300 Read block 4357 len 40960
3820 013/300 Flush
3821 014/300 Write block 4718 len 12288 data: 2 2 2
3822 015/300 Flush
3823 016/300 Read block 462 len 40960
3824 017/300 Read block 2152 len 32768
3825 018/300 Read block 4325 len 20480
3826 019/300 Write block 1556 len 36864 data: 2 2 2 2 2 2 2 2 2
3827 020/300 Write block 440 len 20480 data: 2 2 2 2 3
3828 021/300 Write block 4421 len 36864 data: 3 3 3 3 2 3 3 3 3
3829 022/300 Read block 1965 len 16384
3830 023/300 Read block 1604 len 32768
3831 024/300 Read block 3348 len 20480
3832 025/300 Flush
3833 026/300 Write block 2931 len 16384 data: 2 2 2 2
3834 027/300 Write block 4742 len 24576 data: 2 2 2 2 2 2
3835 028/300 Read block 144 len 12288
3836 029/300 Read block 3432 len 12288
3837 030/300 Flush
3838 031/300 Flush
3839 032/300 Write block 618 len 12288 data: 2 2 2
3840 033/300 Write block 3829 len 36864 data: 2 2 2 2 2 2 2 2 2
3841 034/300 Flush
3842 035/300 Flush
3843 036/300 Write block 2880 len 32768 data: 3 2 2 2 2 2 2 2
3844 037/300 Write block 3706 len 20480 data: 3 4 4 4 3
3845 038/300 Read block 3496 len 4096
3846 039/300 Write block 3482 len 4096 data: 3
3847 040/300 Flush
3848 041/300 Write block 4774 len 12288 data: 3 3 4
3849 042/300 Write block 1735 len 16384 data: 3 3 4 3
3850 043/300 Write block 3014 len 20480 data: 2 2 2 2 2
3851 044/300 Flush
3852 045/300 Read block 2208 len 4096
3853 046/300 Write block 3765 len 40960 data: 4 2 2 2 2 2 2 2 2 2
3854 047/300 Read block 4560 len 32768
3855 048/300 Read block 42 len 12288
3856 049/300 Flush
3857 050/300 Flush
3858 051/300 Write block 3163 len 20480 data: 3 3 2 2 2
3859 052/300 Read block 3486 len 32768
3860 053/300 Read block 4251 len 4096
3861 054/300 Write block 1884 len 32768 data: 2 2 2 2 2 2 3 3
3862 055/300 Write block 2805 len 20480 data: 2 2 3 3 3
3863 056/300 Write block 4484 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3864 057/300 Read block 1200 len 40960
3865 058/300 Read block 3842 len 40960
3866 059/300 Read block 971 len 32768
3867 060/300 Read block 1312 len 32768
3868 061/300 Read block 4443 len 40960
3869 062/300 Flush
3870 063/300 Write block 657 len 8192 data: 3 2
3871 064/300 Read block 2549 len 20480
3872 065/300 Write block 414 len 24576 data: 2 2 2 2 2 3
3873 066/300 Read block 3683 len 36864
3874 067/300 Write block 94 len 32768 data: 2 2 2 2 2 2 3 3
3875 068/300 Flush
3876 069/300 Write block 2687 len 36864 data: 3 2 2 2 3 3 3 3 3
3877 070/300 Read block 559 len 24576
3878 071/300 Flush
3879 072/300 Read block 3087 len 16384
3880 073/300 Write block 593 len 36864 data: 2 2 2 3 3 3 3 3 4
3881 074/300 Read block 2004 len 32768
3882 075/300 Read block 372 len 40960
3883 076/300 Read block 945 len 20480
3884 077/300 Write block 1330 len 12288 data: 2 2 2
3885 078/300 Read block 21 len 12288
3886 079/300 Flush
3887 080/300 Flush
3888 081/300 Write block 3004 len 12288 data: 3 3 2
3889 082/300 Write block 2914 len 40960 data: 2 2 2 2 2 2 2 3 2 2
3890 083/300 Write block 4616 len 36864 data: 2 2 2 2 2 2 2 2 2
3891 084/300 Write block 4701 len 36864 data: 2 2 2 2 2 2 3 3 3
3892 085/300 Write block 1693 len 40960 data: 3 3 3 3 3 2 2 2 2 2
3893 086/300 Read block 4124 len 4096
3894 087/300 Read block 1719 len 40960
3895 088/300 Write block 1476 len 28672 data: 2 2 2 2 2 2 2
3896 089/300 Flush
3897 090/300 Read block 2777 len 32768
3898 091/300 Read block 2980 len 4096
3899 092/300 Write block 2616 len 8192 data: 3 3
3900 093/300 Read block 2043 len 8192
3901 094/300 Write block 4370 len 4096 data: 2
3902 095/300 Write block 3888 len 4096 data: 2
3903 096/300 Write block 2364 len 20480 data: 2 2 2 2 2
3904 097/300 Write block 3164 len 32768 data: 4 3 3 3 2 2 2 2
3905 098/300 Write block 1599 len 36864 data: 3 3 3 3 3 3 3 3 3
3906 099/300 Read block 3153 len 20480
3907 100/300 Write block 4020 len 36864 data: 2 2 2 2 2 2 2 2 2
3908 101/300 Write block 2619 len 16384 data: 2 2 2 2
3909 102/300 Read block 3835 len 32768
3910 103/300 Write block 324 len 36864 data: 4 3 3 4 4 2 3 3 2
3911 104/300 Read block 3352 len 32768
3912 105/300 Read block 1931 len 32768
3913 106/300 Read block 1928 len 8192
3914 107/300 Read block 3862 len 12288
3915 108/300 Write block 2504 len 36864 data: 2 2 2 2 2 2 2 2 2
3916 109/300 Write block 180 len 24576 data: 4 4 2 2 2 2
3917 110/300 Flush
3918 111/300 Read block 493 len 36864
3919 112/300 Write block 1403 len 40960 data: 3 3 3 3 3 2 2 2 3 3
3920 113/300 Read block 2468 len 40960
3921 114/300 Flush
3922 115/300 Flush
3923 116/300 Read block 3962 len 32768
3924 117/300 Read block 2148 len 28672
3925 118/300 Read block 1755 len 28672
3926 119/300 Read block 4094 len 40960
3927 120/300 Flush
3928 121/300 Write block 1579 len 36864 data: 2 2 2 2 2 3 3 2 2
3929 122/300 Write block 4605 len 36864 data: 2 2 2 2 2 2 2 2 2
3930 123/300 Read block 391 len 8192
3931 124/300 Write block 2022 len 40960 data: 3 3 3 4 5 4 4 3 3 2
3932 125/300 Write block 4516 len 24576 data: 2 3 3 3 2 2
3933 126/300 Read block 2186 len 32768
3934 127/300 Write block 1350 len 16384 data: 3 4 4 4
3935 128/300 Write block 458 len 20480 data: 2 2 2 2 2
3936 129/300 Read block 3255 len 40960
3937 130/300 Read block 3765 len 12288
3938 131/300 Flush
3939 132/300 Read block 311 len 12288
3940 133/300 Write block 2530 len 32768 data: 3 3 3 2 2 2 2 2
3941 134/300 Read block 1215 len 24576
3942 135/300 Flush
3943 136/300 Write block 2274 len 24576 data: 2 2 3 3 3 3
3944 137/300 Write block 1020 len 8192 data: 2 2
3945 138/300 Flush
3946 139/300 Write block 4634 len 28672 data: 3 2 2 2 2 2 2
3947 140/300 Read block 4338 len 12288
3948 141/300 Write block 3736 len 40960 data: 3 3 3 4 4 4 4 4 3 2
3949 142/300 Write block 4695 len 4096 data: 2
3950 143/300 Write block 1157 len 32768 data: 2 2 2 2 2 2 3 3
3951 144/300 Write block 1488 len 4096 data: 2
3952 145/300 Write block 2191 len 8192 data: 3 3
3953 146/300 Read block 625 len 20480
3954 147/300 Read block 2582 len 36864
3955 148/300 Read block 2964 len 36864
3956 149/300 Read block 1814 len 4096
3957 150/300 Write block 4374 len 12288 data: 2 2 2
3958 151/300 Flush
3959 152/300 Write block 3305 len 36864 data: 3 3 3 3 3 2 2 2 2
3960 153/300 Read block 3301 len 28672
3961 154/300 Flush
3962 155/300 Read block 2226 len 12288
3963 156/300 Write block 1750 len 32768 data: 2 2 2 2 3 3 3 3
3964 157/300 Write block 3740 len 32768 data: 5 5 5 5 4 3 2 2
3965 158/300 Write block 4671 len 32768 data: 2 2 2 2 2 2 2 2
3966 159/300 Read block 2046 len 8192
3967 160/300 Read block 1141 len 40960
3968 161/300 Read block 2184 len 36864
3969 162/300 Read block 2090 len 32768
3970 163/300 Read block 3977 len 4096
3971 164/300 Flush
3972 165/300 Read block 2183 len 12288
3973 166/300 Read block 1538 len 32768
3974 167/300 Read block 3945 len 16384
3975 168/300 Read block 1279 len 40960
3976 169/300 Read block 3509 len 24576
3977 170/300 Read block 1706 len 40960
3978 171/300 Read block 251 len 28672
3979 172/300 Write block 2458 len 12288 data: 2 2 2
3980 173/300 Read block 3221 len 20480
3981 174/300 Read block 3163 len 16384
3982 175/300 Flush
3983 176/300 Read block 4250 len 32768
3984 177/300 Read block 3750 len 36864
3985 178/300 Write block 4032 len 28672 data: 2 2 2 2 2 3 4
3986 179/300 Write block 2761 len 8192 data: 2 2
3987 180/300 Read block 1205 len 28672
3988 181/300 Flush
3989 182/300 Read block 888 len 12288
3990 183/300 Read block 4585 len 20480
3991 184/300 Read block 2572 len 24576
3992 185/300 Write block 3508 len 12288 data: 2 3 3
3993 186/300 Write block 586 len 32768 data: 2 2 3 3 2 2 2 3
3994 187/300 Write block 1883 len 24576 data: 2 3 3 3 3 3
3995 188/300 Read block 2402 len 24576
3996 189/300 Read block 3635 len 40960
3997 190/300 Flush
3998 191/300 Flush
3999 192/300 Write block 4961 len 8192 data: 2 2
4000 193/300 Read block 1313 len 24576
4001 194/300 Write block 569 len 12288 data: 2 2 2
4002 195/300 Write block 4793 len 32768 data: 2 2 2 2 2 2 3 3
4003 196/300 Read block 1023 len 16384
4004 197/300 Flush
4005 198/300 Read block 2426 len 16384
4006 199/300 Read block 1114 len 12288
4007 200/300 Write block 4697 len 36864 data: 2 2 2 2 3 3 3 3 3
4008 201/300 Read block 673 len 20480
4009 202/300 Write block 2281 len 40960 data: 3 2 2 2 2 3 3 3 3 3
4010 203/300 Write block 4557 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4011 204/300 Read block 2413 len 32768
4012 205/300 Read block 156 len 28672
4013 206/300 Flush
4014 207/300 Write block 3327 len 12288 data: 2 2 2
4015 208/300 Read block 3421 len 4096
4016 209/300 Write block 4758 len 40960 data: 2 2 2 2 2 3 3 3 3 3
4017 210/300 Write block 2764 len 20480 data: 2 2 2 2 2
4018 211/300 Write block 728 len 32768 data: 4 3 3 3 2 2 2 2
4019 212/300 Read block 1016 len 4096
4020 213/300 Write block 777 len 8192 data: 2 2
4021 214/300 Write block 1932 len 36864 data: 2 2 2 2 2 3 3 3 3
4022 215/300 Read block 2582 len 12288
4023 216/300 Flush
4024 217/300 Write block 3336 len 28672 data: 3 3 3 3 3 3 2
4025 218/300 Flush
4026 219/300 Write block 1851 len 8192 data: 2 2
4027 220/300 Write block 1496 len 20480 data: 2 2 2 2 2
4028 221/300 Write block 491 len 36864 data: 2 2 2 2 3 3 3 3 3
4029 222/300 Write block 2750 len 20480 data: 2 2 2 2 2
4030 223/300 Write block 4126 len 12288 data: 3 3 3
4031 224/300 Read block 3493 len 8192
4032 225/300 Write block 1342 len 28672 data: 2 2 2 2 2 2 3
4033 226/300 Write block 3879 len 24576 data: 3 3 3 2 2 2
4034 227/300 Write block 2023 len 28672 data: 4 4 5 6 5 5 4
4035 228/300 Write block 2726 len 16384 data: 3 3 4 4
4036 229/300 Read block 3875 len 8192
4037 230/300 Write block 3364 len 16384 data: 2 2 2 2
4038 231/300 Read block 1813 len 16384
4039 232/300 Write block 1190 len 32768 data: 2 3 3 3 3 3 3 2
4040 233/300 Read block 3610 len 16384
4041 234/300 Read block 3287 len 8192
4042 235/300 Read block 1081 len 36864
4043 236/300 Read block 4728 len 32768
4044 237/300 Write block 346 len 8192 data: 3 3
4045 238/300 Write block 4973 len 28672 data: 2 2 2 2 2 2 2
4046 239/300 Write block 384 len 32768 data: 2 3 3 3 4 4 4 4
4047 240/300 Read block 869 len 28672
4048 241/300 Read block 4833 len 12288
4049 242/300 Read block 1013 len 12288
4050 243/300 Read block 3931 len 28672
4051 244/300 Read block 909 len 28672
4052 245/300 Write block 1061 len 32768 data: 2 2 3 3 2 3 3 3
4053 246/300 Write block 2434 len 16384 data: 2 2 3 3
4054 247/300 Read block 3512 len 16384
4055 248/300 Read block 2181 len 20480
4056 249/300 Read block 1468 len 32768
4057 250/300 Write block 373 len 8192 data: 2 2
4058 251/300 Read block 665 len 8192
4059 252/300 Read block 242 len 40960
4060 253/300 Read block 1185 len 20480
4061 254/300 Write block 2209 len 36864 data: 3 3 3 3 4 3 3 2 2
4062 255/300 Read block 1558 len 36864
4063 256/300 Read block 3026 len 32768
4064 257/300 Write block 303 len 20480 data: 2 2 2 2 2
4065 258/300 Read block 1152 len 12288
4066 259/300 Flush
4067 260/300 Read block 4152 len 24576
4068 261/300 Write block 1927 len 12288 data: 2 2 2
4069 262/300 Read block 3429 len 28672
4070 263/300 Write block 648 len 28672 data: 4 4 4 4 4 4 3
4071 264/300 Read block 1744 len 20480
4072 265/300 Write block 2660 len 32768 data: 2 2 3 3 3 3 3 3
4073 266/300 Flush
4074 267/300 Write block 4069 len 20480 data: 2 2 2 3 3
4075 268/300 Write block 3839 len 12288 data: 2 2 3
4076 269/300 Write block 2677 len 16384 data: 2 2 3 3
4077 270/300 Read block 120 len 24576
4078 271/300 Read block 189 len 4096
4079 272/300 Read block 1594 len 36864
4080 273/300 Read block 125 len 40960
4081 274/300 Read block 1633 len 36864
4082 275/300 Flush
4083 276/300 Read block 2288 len 24576
4084 277/300 Flush
4085 278/300 Read block 2503 len 40960
4086 279/300 Flush
4087 280/300 Read block 4124 len 12288
4088 281/300 Read block 278 len 12288
4089 282/300 Read block 27 len 4096
4090 283/300 Read block 3460 len 4096
4091 284/300 Read block 2975 len 12288
4092 285/300 Read block 389 len 32768
4093 286/300 Write block 1742 len 12288 data: 4 3 3
4094 287/300 Read block 4432 len 8192
4095 288/300 Read block 3081 len 8192
4096 289/300 Read block 4196 len 16384
4097 290/300 Read block 2342 len 36864
4098 291/300 Flush
4099 292/300 Write block 2298 len 20480 data: 2 2 3 3 3
4100 293/300 Read block 2090 len 16384
4101 294/300 Write block 1702 len 28672 data: 3 2 2 2 2 2 2
4102 295/300 Read block 3005 len 20480
4103 296/300 Read block 2204 len 32768
4104 297/300 Write block 4089 len 4096 data: 3
4105 298/300 Read block 3585 len 4096
4106 299/300 Flush
4107 300/300 Read block 4557 len 40960
4108 301/300 Write block 3123 len 24576 data: 2 3 3 3 3 3
4109 Main task has work to do, go find it
4110 got dsc Start(2)
4111 start 2
4112 Nov 19 17:08:52.693 INFO request completed, latency_us: 242, response_code: 204, uri: /start/cid/2, method: POST, req_id: df511582-f5f0-40bb-86e3-d5fae43e69ca, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
4113 Replay: started 2, returned:Ok(())
4114 ----------------------------------------------------------------
4115 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4116 States: Active Active Offline
4117 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4118 1906 Acked Write 5 Done Done Sent false
4119 1907 Acked Read 6 Done Done Sent false
4120 1908 Acked Read 5 Done Done Sent false
4121 1909 Acked Write 5 Done Done Sent false
4122 1910 Acked Write 10 Done Done Sent false
4123 1911 Acked Read 8 Done Done Sent false
4124 1912 Acked Read 5 Done Done Sent false
4125 1913 Acked Flush 0 Done Done Sent false
4126 1914 Acked Read 3 Done Done Sent false
4127 1915 Acked Read 10 Done Done Sent false
4128 1916 Acked Write 6 Done Done Sent false
4129 1917 Acked Read 10 Done Done Sent false
4130 1918 Acked Flush 0 Done Done Sent false
4131 1919 Acked Write 3 Done Done Sent false
4132 1920 Acked Flush 0 Done Done Sent false
4133 1921 Acked Read 10 Done Done Sent false
4134 1922 Acked Read 8 Done Done Sent false
4135 1923 Acked Read 5 Done Done Sent false
4136 1924 Acked Write 9 Done Done Sent false
4137 1925 Acked Write 5 Done Done Sent false
4138 1926 Acked Write 9 Done Done Sent false
4139 1927 Acked Read 4 Done Done Sent false
4140 1928 Acked Read 8 Done Done Sent false
4141 1929 Acked Read 5 Done Done Sent false
4142 1930 Acked Flush 0 Done Done Sent false
4143 1931 Acked Write 4 Done Done Sent false
4144 1932 Acked Write 6 Done Done Sent false
4145 1933 Acked Read 3 Done Done Sent false
4146 1934 Acked Read 3 Done Done Sent false
4147 1935 Acked Flush 0 Done Done Sent false
4148 1936 Acked Flush 0 Done Done Sent false
4149 1937 Acked Write 3 Done Done Sent false
4150 1938 Acked Write 9 Done Done Sent false
4151 1939 Acked Flush 0 Done Done Sent false
4152 1940 Acked Flush 0 Done Done Sent false
4153 1941 Acked Write 8 Done Done Sent false
4154 1942 Acked Write 5 Done Done Sent false
4155 1943 Acked Read 1 Done Done Sent false
4156 1944 Acked Write 1 Done Done Sent false
4157 1945 Acked Flush 0 Done Done Sent false
4158 1946 Acked Write 3 Done Done Sent false
4159 1947 Acked Write 4 Done Done Sent false
4160 1948 Acked Write 5 Done Done Sent false
4161 1949 Acked Flush 0 Done Done Sent false
4162 1950 Acked Read 1 Done Done Sent false
4163 1951 Acked Write 10 Done Done Sent false
4164 1952 Acked Read 8 Done Done Sent false
4165 1953 Acked Read 3 Done Done Sent false
4166 1954 Acked Flush 0 Done Done Sent false
4167 1955 Acked Flush 0 Done Done Sent false
4168 1956 Acked Write 5 Done Done Sent false
4169 1957 Acked Read 8 Done Done Sent false
4170 1958 Acked Read 1 Done Done Sent false
4171 1959 Acked Write 8 Done Done Sent false
4172 1960 Acked Write 5 Done Done Sent false
4173 1961 Acked Write 10 Done Done Sent false
4174 1962 Acked Read 10 Done Done Sent false
4175 1963 Acked Read 10 Done Done Sent false
4176 1964 Acked Read 8 Done Done Sent false
4177 1965 Acked Read 8 Done Done Sent false
4178 1966 Acked Read 10 Done Done Sent false
4179 1967 Acked Flush 0 Done Done Sent false
4180 1968 Acked Write 2 Done Done Sent false
4181 1969 Acked Read 5 Done Done Sent false
4182 1970 Acked Write 6 Done Done Sent false
4183 1971 Acked Read 9 Done Done Sent false
4184 1972 Acked Write 8 Done Done Sent false
4185 1973 Acked Flush 0 Done Done Sent false
4186 1974 Acked Write 9 Done Done Sent false
4187 1975 Acked Read 6 Done Done Sent false
4188 1976 Acked Flush 0 Done Done Sent false
4189 1977 Acked Read 4 Done Done Sent false
4190 1978 Acked Write 9 Done Done Sent false
4191 1979 Acked Read 8 Done Done Sent false
4192 1980 Acked Read 10 Done Done Sent false
4193 1981 Acked Read 5 Done Done Sent false
4194 1982 Acked Write 3 Done Done Sent false
4195 1983 Acked Read 3 Done Done Sent false
4196 1984 Acked Flush 0 Done Done Sent false
4197 1985 Acked Flush 0 Done Done Sent false
4198 1986 Acked Write 3 Done Done Sent false
4199 1987 Acked Write 10 Done Done Sent false
4200 1988 Acked Write 9 Done Done Sent false
4201 1989 Acked Write 9 Done Done Sent false
4202 1990 Acked Write 10 Done Done Sent false
4203 1991 Acked Read 1 Done Done Sent false
4204 1992 Acked Read 10 Done Done Sent false
4205 1993 Acked Write 7 Done Done Sent false
4206 1994 Acked Flush 0 Done Done Sent false
4207 1995 Acked Read 8 Done Done Sent false
4208 1996 Acked Read 1 Done Done Sent false
4209 1997 Acked Write 2 Done Done Sent false
4210 1998 Acked Read 2 Done Done Sent false
4211 1999 Acked Write 1 Done Done Sent false
4212 2000 Acked Write 1 Done Done Sent false
4213 2001 Acked Write 5 Done Done Sent false
4214 2002 Acked Write 8 Done Done Sent false
4215 2003 Acked Write 9 Done Done Sent false
4216 2004 Acked Read 5 Done Done Sent false
4217 2005 Acked Write 9 Done Done Sent false
4218 2006 Acked Write 4 Done Done Sent false
4219 2007 Acked Read 8 Done Done Sent false
4220 2008 Acked Write 9 Done Done Sent false
4221 2009 Acked Read 8 Done Done Sent false
4222 2010 Acked Read 8 Done Done Sent false
4223 2011 Acked Read 2 Done Done Sent false
4224 2012 Acked Read 3 Done Done Sent false
4225 2013 Acked Write 9 Done Done Sent false
4226 2014 Acked Write 6 Done Done Sent false
4227 2015 Acked Flush 0 Done Done Sent false
4228 2016 Acked Read 9 Done Done Sent false
4229 2017 Acked Write 10 Done Done Sent false
4230 2018 Acked Read 10 Done Done Sent false
4231 2019 Acked Flush 0 Done Done Sent false
4232 2020 Acked Flush 0 Done Done Sent false
4233 2021 Acked Read 8 Done Done Sent false
4234 2022 Acked Read 7 Done Done Sent false
4235 2023 Acked Read 7 Done Done Sent false
4236 2024 Acked Read 10 Done Done Sent false
4237 2025 Acked Flush 0 Done Done Sent false
4238 2026 Acked Write 9 Done Done Sent false
4239 2027 Acked Write 9 Done Done Sent false
4240 2028 Acked Read 2 Done Done Sent false
4241 2029 Acked Write 10 Done Done Sent false
4242 2030 Acked Write 6 Done Done Sent false
4243 2031 Acked Read 8 Done Done Sent false
4244 2032 Acked Write 4 Done Done Sent false
4245 2033 Acked Write 5 Done Done Sent false
4246 2034 Acked Read 10 Done Done Sent false
4247 2035 Acked Read 3 Done Done Sent false
4248 2036 Acked Flush 0 Done Done Sent false
4249 2037 Acked Read 3 Done Done Sent false
4250 2038 Acked Write 8 Done Done Sent false
4251 2039 Acked Read 6 Done Done Sent false
4252 2040 Acked Flush 0 Done Done Sent false
4253 2041 Acked Write 6 Done Done Sent false
4254 2042 Acked Write 2 Done Done Sent false
4255 2043 Acked Flush 0 Done Done Sent false
4256 2044 Acked Write 7 Done Done Sent false
4257 2045 Acked Read 3 Done Done Sent false
4258 2046 Acked Write 10 Done Done Sent false
4259 2047 Acked Write 1 Done Done Sent false
4260 2048 Acked Write 8 Done Done Sent false
4261 2049 Acked Write 1 Done Done Sent false
4262 2050 Acked Write 2 Done Done Sent false
4263 2051 Acked Read 5 Done Done Sent false
4264 2052 Acked Read 9 Done Done Sent false
4265 2053 Acked Read 9 Done Done Sent false
4266 2054 Acked Read 1 Done Done Sent false
4267 2055 Acked Write 3 Done Done Sent false
4268 2056 Acked Flush 0 Done Done Sent false
4269 2057 Acked Write 9 Done Done Sent false
4270 2058 Acked Read 7 Done Done Sent false
4271 2059 Acked Flush 0 Done Done Sent false
4272 2060 Acked Read 3 Done Done Sent false
4273 2061 Acked Write 8 Done Done Sent false
4274 2062 Acked Write 8 Done Done Sent false
4275 2063 Acked Write 8 Done Done Sent false
4276 2064 Acked Read 2 Done Done Sent false
4277 2065 Acked Read 10 Done Done Sent false
4278 2066 Acked Read 9 Done Done Sent false
4279 2067 Acked Read 8 Done Done Sent false
4280 2068 Acked Read 1 Done Done Sent false
4281 2069 Acked Flush 0 Done Done Sent false
4282 2070 Acked Read 3 Done Done Sent false
4283 2071 Acked Read 8 Done Done Sent false
4284 2072 Acked Read 4 Done Done Sent false
4285 2073 Acked Read 10 Done Done Sent false
4286 2074 Acked Read 6 Done Done Sent false
4287 2075 Acked Read 10 Done Done Sent false
4288 2076 Acked Read 7 Done Done Sent false
4289 2077 Acked Write 3 Done Done Sent false
4290 2078 Acked Read 5 Done Done Sent false
4291 2079 Acked Read 4 Done Done Sent false
4292 2080 Acked Flush 0 Done Done Sent false
4293 2081 Acked Read 8 Done Done Sent false
4294 2082 Acked Read 9 Done Done Sent false
4295 2083 Acked Write 7 Done Done Sent false
4296 2084 Acked Write 2 Done Done Sent false
4297 2085 Acked Read 7 Done Done Sent false
4298 2086 Acked Flush 0 Done Done Sent false
4299 2087 Acked Read 3 Done Done Sent false
4300 2088 Acked Read 5 Done Done Sent false
4301 2089 Acked Read 6 Done Done Sent false
4302 2090 Acked Write 3 Done Done Sent false
4303 2091 Acked Write 8 Done Done Sent false
4304 2092 Acked Write 6 Done Done Sent false
4305 2093 Acked Read 6 Done Done Sent false
4306 2094 Acked Read 10 Done Done Sent false
4307 2095 Acked Flush 0 Done Done Sent false
4308 2096 Acked Flush 0 Done Done Sent false
4309 2097 Acked Write 2 Done Done Sent false
4310 2098 Acked Read 6 Done Done Sent false
4311 2099 Acked Write 3 Done Done Sent false
4312 2100 Acked Write 8 Done Done Sent false
4313 2101 Acked Read 4 Done Done Sent false
4314 2102 Acked Flush 0 Done Done Sent false
4315 2103 Acked Read 4 Done Done Sent false
4316 2104 Acked Read 3 Done Done Sent false
4317 2105 Acked Write 9 Done Done Sent false
4318 2106 Acked Read 5 Done Done Sent false
4319 2107 Acked Write 10 Done Done Sent false
4320 2108 Acked Write 10 Done Done Sent false
4321 2109 Acked Read 8 Done Done Sent false
4322 2110 Acked Read 7 Done Done Sent false
4323 2111 Acked Flush 0 Done Done Sent false
4324 2112 Acked Write 3 Done Done Sent false
4325 2113 Acked Read 1 Done Done Sent false
4326 2114 Acked Write 10 Done Done Sent false
4327 2115 Acked Write 5 Done Done Sent false
4328 2116 Acked Write 8 Done Done Sent false
4329 2117 Acked Read 1 Done Done Sent false
4330 2118 Acked Write 2 Done Done Sent false
4331 2119 Acked Write 9 Done Done Sent false
4332 2120 Acked Read 3 Done Done Sent false
4333 2121 Acked Flush 0 Done Done Sent false
4334 2122 Acked Write 7 Done Done Sent false
4335 2123 Acked Flush 0 Done Done Sent false
4336 2124 Acked Write 2 Done Done Sent false
4337 2125 Acked Write 5 Done Done Sent false
4338 2126 Acked Write 9 Done Done Sent false
4339 2127 Acked Write 5 Done Done Sent false
4340 2128 Acked Write 3 Done Done Sent false
4341 2129 Acked Read 2 Done Done Sent false
4342 2130 Acked Write 7 Done Done Sent false
4343 2131 Acked Write 6 Done Done Sent false
4344 2132 Acked Write 7 Done Done Sent false
4345 2133 Acked Write 4 Done Done Sent false
4346 2134 Acked Read 2 Done Done Sent false
4347 2135 Acked Write 4 Done Done Sent false
4348 2136 Acked Read 4 Done Done Sent false
4349 2137 Acked Write 8 Done Done Sent false
4350 2138 Acked Read 4 Done Done Sent false
4351 2139 Acked Read 2 Done Done Sent false
4352 2140 Acked Read 9 Done Done Sent false
4353 2141 Acked Read 8 Done Done Sent false
4354 2142 Acked Write 2 Done Done Sent false
4355 2143 Acked Write 7 Done Done Sent false
4356 2144 Acked Write 8 Done Done Sent false
4357 2145 Acked Read 7 Done Done Sent false
4358 2146 Acked Read 3 Done Done Sent false
4359 2147 Acked Read 3 Done Done Sent false
4360 2148 Acked Read 7 Done Done Sent false
4361 2149 Acked Read 7 Done Done Sent false
4362 2150 Acked Write 8 Done Done Sent false
4363 2151 Acked Write 4 Done Done Sent false
4364 2152 Acked Read 4 Done Done Sent false
4365 2153 Acked Read 5 Done Done Sent false
4366 2154 Acked Read 8 Done Done Sent false
4367 2155 Acked Write 2 Done Done Sent false
4368 2156 Acked Read 2 Done Done Sent false
4369 2157 Acked Read 10 Done Done Sent false
4370 2158 Acked Read 5 Done Done Sent false
4371 2159 Acked Write 9 Done Done Sent false
4372 2160 Acked Read 9 Done Done Sent false
4373 2161 Acked Read 8 Done Done Sent false
4374 2162 Acked Write 5 Done Done Sent false
4375 2163 Acked Read 3 Done Done Sent false
4376 2164 Acked Flush 0 Done Done Sent false
4377 2165 Acked Read 6 Done Done Sent false
4378 2166 Acked Write 3 Done Done Sent false
4379 2167 Acked Read 7 Done Done Sent false
4380 2168 Acked Write 7 Done Done Sent false
4381 2169 Acked Read 5 Done Done Sent false
4382 2170 Acked Write 8 Done Done Sent false
4383 2171 Acked Flush 0 Done Done Sent false
4384 2172 Acked Write 5 Done Done Sent false
4385 2173 Acked Write 3 Done Done Sent false
4386 2174 Acked Write 4 Done Done Sent false
4387 2175 Acked Read 6 Done Done Sent false
4388 2176 Acked Read 1 Done Done Sent false
4389 2177 Acked Read 9 Done Done Sent false
4390 2178 Acked Read 10 Done Done Sent false
4391 2179 Acked Read 9 Done Done Sent false
4392 2180 Acked Flush 0 Done Done Sent false
4393 2181 Acked Read 6 Done Done Sent false
4394 2182 Acked Flush 0 Done Done Sent false
4395 2183 Acked Read 10 Done Done Sent false
4396 2184 Acked Flush 0 Done Done Sent false
4397 2185 Acked Read 3 Done Done Sent false
4398 2186 Acked Read 3 Done Done Sent false
4399 2187 Acked Read 1 Done Done Sent false
4400 2188 Acked Read 1 Done Done Sent false
4401 2189 Acked Read 3 Done Done Sent false
4402 2190 Acked Read 8 Done Done Sent false
4403 2191 Acked Write 3 Done Done Sent false
4404 2192 Acked Read 2 Done Done Sent false
4405 2193 Acked Read 2 Done Done Sent false
4406 2194 Acked Read 4 Done Done Sent false
4407 2195 Acked Read 9 Done Done Sent false
4408 2196 Acked Flush 0 Done Done Sent false
4409 2197 Acked Write 5 Done Done Sent false
4410 2198 Acked Read 4 Done Done Sent false
4411 2199 Acked Write 7 Done Done Sent false
4412 2200 Acked Read 5 Done Done Sent false
4413 2201 Acked Read 8 Done Done Sent false
4414 2202 Acked Write 1 Done Done Sent false
4415 2203 Acked Read 1 Done Done Sent false
4416 2204 Acked Flush 0 Done Done Sent false
4417 2205 Acked Read 10 Done Done Sent false
4418 2206 Acked Write 6 Done Done Sent false
4419 STATES DS:0 DS:1 DS:2 TOTAL
4420 Sent 0 0 301 301
4421 Done 301 301 0 602
4422 Skipped 0 0 0 0
4423 Error 0 0 0 0
4424 Last Flush: 2204 2204 1905
4425 Downstairs last five completed: 1905 1904 1903 1902 1901
4426 Upstairs last five completed: 2206 2205 2204 2203 2202
4427 CLIENT: Up:0 ds:301 act:2
4428 [8830] got start action
4429 [8830] I am going to restart
4430 Starting downstairs at port 8830
4431 Make output file at "/tmp/dsc/downstairs-8830.txt"
4432 [8830][2] reports Starting
4433 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1206)
4434 [8830][2] reports Running
4435 ----------------------------------------------------------------
4436 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4437 States: Active Active Offline
4438 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4439 1906 Acked Write 5 Done Done Sent false
4440 1907 Acked Read 6 Done Done Sent false
4441 1908 Acked Read 5 Done Done Sent false
4442 1909 Acked Write 5 Done Done Sent false
4443 1910 Acked Write 10 Done Done Sent false
4444 1911 Acked Read 8 Done Done Sent false
4445 1912 Acked Read 5 Done Done Sent false
4446 1913 Acked Flush 0 Done Done Sent false
4447 1914 Acked Read 3 Done Done Sent false
4448 1915 Acked Read 10 Done Done Sent false
4449 1916 Acked Write 6 Done Done Sent false
4450 1917 Acked Read 10 Done Done Sent false
4451 1918 Acked Flush 0 Done Done Sent false
4452 1919 Acked Write 3 Done Done Sent false
4453 1920 Acked Flush 0 Done Done Sent false
4454 1921 Acked Read 10 Done Done Sent false
4455 1922 Acked Read 8 Done Done Sent false
4456 1923 Acked Read 5 Done Done Sent false
4457 1924 Acked Write 9 Done Done Sent false
4458 1925 Acked Write 5 Done Done Sent false
4459 1926 Acked Write 9 Done Done Sent false
4460 1927 Acked Read 4 Done Done Sent false
4461 1928 Acked Read 8 Done Done Sent false
4462 1929 Acked Read 5 Done Done Sent false
4463 1930 Acked Flush 0 Done Done Sent false
4464 1931 Acked Write 4 Done Done Sent false
4465 1932 Acked Write 6 Done Done Sent false
4466 1933 Acked Read 3 Done Done Sent false
4467 1934 Acked Read 3 Done Done Sent false
4468 1935 Acked Flush 0 Done Done Sent false
4469 1936 Acked Flush 0 Done Done Sent false
4470 1937 Acked Write 3 Done Done Sent false
4471 1938 Acked Write 9 Done Done Sent false
4472 1939 Acked Flush 0 Done Done Sent false
4473 1940 Acked Flush 0 Done Done Sent false
4474 1941 Acked Write 8 Done Done Sent false
4475 1942 Acked Write 5 Done Done Sent false
4476 1943 Acked Read 1 Done Done Sent false
4477 1944 Acked Write 1 Done Done Sent false
4478 1945 Acked Flush 0 Done Done Sent false
4479 1946 Acked Write 3 Done Done Sent false
4480 1947 Acked Write 4 Done Done Sent false
4481 1948 Acked Write 5 Done Done Sent false
4482 1949 Acked Flush 0 Done Done Sent false
4483 1950 Acked Read 1 Done Done Sent false
4484 1951 Acked Write 10 Done Done Sent false
4485 1952 Acked Read 8 Done Done Sent false
4486 1953 Acked Read 3 Done Done Sent false
4487 1954 Acked Flush 0 Done Done Sent false
4488 1955 Acked Flush 0 Done Done Sent false
4489 1956 Acked Write 5 Done Done Sent false
4490 1957 Acked Read 8 Done Done Sent false
4491 1958 Acked Read 1 Done Done Sent false
4492 1959 Acked Write 8 Done Done Sent false
4493 1960 Acked Write 5 Done Done Sent false
4494 1961 Acked Write 10 Done Done Sent false
4495 1962 Acked Read 10 Done Done Sent false
4496 1963 Acked Read 10 Done Done Sent false
4497 1964 Acked Read 8 Done Done Sent false
4498 1965 Acked Read 8 Done Done Sent false
4499 1966 Acked Read 10 Done Done Sent false
4500 1967 Acked Flush 0 Done Done Sent false
4501 1968 Acked Write 2 Done Done Sent false
4502 1969 Acked Read 5 Done Done Sent false
4503 1970 Acked Write 6 Done Done Sent false
4504 1971 Acked Read 9 Done Done Sent false
4505 1972 Acked Write 8 Done Done Sent false
4506 1973 Acked Flush 0 Done Done Sent false
4507 1974 Acked Write 9 Done Done Sent false
4508 1975 Acked Read 6 Done Done Sent false
4509 1976 Acked Flush 0 Done Done Sent false
4510 1977 Acked Read 4 Done Done Sent false
4511 1978 Acked Write 9 Done Done Sent false
4512 1979 Acked Read 8 Done Done Sent false
4513 1980 Acked Read 10 Done Done Sent false
4514 1981 Acked Read 5 Done Done Sent false
4515 1982 Acked Write 3 Done Done Sent false
4516 1983 Acked Read 3 Done Done Sent false
4517 1984 Acked Flush 0 Done Done Sent false
4518 1985 Acked Flush 0 Done Done Sent false
4519 1986 Acked Write 3 Done Done Sent false
4520 1987 Acked Write 10 Done Done Sent false
4521 1988 Acked Write 9 Done Done Sent false
4522 1989 Acked Write 9 Done Done Sent false
4523 1990 Acked Write 10 Done Done Sent false
4524 1991 Acked Read 1 Done Done Sent false
4525 1992 Acked Read 10 Done Done Sent false
4526 1993 Acked Write 7 Done Done Sent false
4527 1994 Acked Flush 0 Done Done Sent false
4528 1995 Acked Read 8 Done Done Sent false
4529 1996 Acked Read 1 Done Done Sent false
4530 1997 Acked Write 2 Done Done Sent false
4531 1998 Acked Read 2 Done Done Sent false
4532 1999 Acked Write 1 Done Done Sent false
4533 2000 Acked Write 1 Done Done Sent false
4534 2001 Acked Write 5 Done Done Sent false
4535 2002 Acked Write 8 Done Done Sent false
4536 2003 Acked Write 9 Done Done Sent false
4537 2004 Acked Read 5 Done Done Sent false
4538 2005 Acked Write 9 Done Done Sent false
4539 2006 Acked Write 4 Done Done Sent false
4540 2007 Acked Read 8 Done Done Sent false
4541 2008 Acked Write 9 Done Done Sent false
4542 2009 Acked Read 8 Done Done Sent false
4543 2010 Acked Read 8 Done Done Sent false
4544 2011 Acked Read 2 Done Done Sent false
4545 2012 Acked Read 3 Done Done Sent false
4546 2013 Acked Write 9 Done Done Sent false
4547 2014 Acked Write 6 Done Done Sent false
4548 2015 Acked Flush 0 Done Done Sent false
4549 2016 Acked Read 9 Done Done Sent false
4550 2017 Acked Write 10 Done Done Sent false
4551 2018 Acked Read 10 Done Done Sent false
4552 2019 Acked Flush 0 Done Done Sent false
4553 2020 Acked Flush 0 Done Done Sent false
4554 2021 Acked Read 8 Done Done Sent false
4555 2022 Acked Read 7 Done Done Sent false
4556 2023 Acked Read 7 Done Done Sent false
4557 2024 Acked Read 10 Done Done Sent false
4558 2025 Acked Flush 0 Done Done Sent false
4559 2026 Acked Write 9 Done Done Sent false
4560 2027 Acked Write 9 Done Done Sent false
4561 2028 Acked Read 2 Done Done Sent false
4562 2029 Acked Write 10 Done Done Sent false
4563 2030 Acked Write 6 Done Done Sent false
4564 2031 Acked Read 8 Done Done Sent false
4565 2032 Acked Write 4 Done Done Sent false
4566 2033 Acked Write 5 Done Done Sent false
4567 2034 Acked Read 10 Done Done Sent false
4568 2035 Acked Read 3 Done Done Sent false
4569 2036 Acked Flush 0 Done Done Sent false
4570 2037 Acked Read 3 Done Done Sent false
4571 2038 Acked Write 8 Done Done Sent false
4572 2039 Acked Read 6 Done Done Sent false
4573 2040 Acked Flush 0 Done Done Sent false
4574 2041 Acked Write 6 Done Done Sent false
4575 2042 Acked Write 2 Done Done Sent false
4576 2043 Acked Flush 0 Done Done Sent false
4577 2044 Acked Write 7 Done Done Sent false
4578 2045 Acked Read 3 Done Done Sent false
4579 2046 Acked Write 10 Done Done Sent false
4580 2047 Acked Write 1 Done Done Sent false
4581 2048 Acked Write 8 Done Done Sent false
4582 2049 Acked Write 1 Done Done Sent false
4583 2050 Acked Write 2 Done Done Sent false
4584 2051 Acked Read 5 Done Done Sent false
4585 2052 Acked Read 9 Done Done Sent false
4586 2053 Acked Read 9 Done Done Sent false
4587 2054 Acked Read 1 Done Done Sent false
4588 2055 Acked Write 3 Done Done Sent false
4589 2056 Acked Flush 0 Done Done Sent false
4590 2057 Acked Write 9 Done Done Sent false
4591 2058 Acked Read 7 Done Done Sent false
4592 2059 Acked Flush 0 Done Done Sent false
4593 2060 Acked Read 3 Done Done Sent false
4594 2061 Acked Write 8 Done Done Sent false
4595 2062 Acked Write 8 Done Done Sent false
4596 2063 Acked Write 8 Done Done Sent false
4597 2064 Acked Read 2 Done Done Sent false
4598 2065 Acked Read 10 Done Done Sent false
4599 2066 Acked Read 9 Done Done Sent false
4600 2067 Acked Read 8 Done Done Sent false
4601 2068 Acked Read 1 Done Done Sent false
4602 2069 Acked Flush 0 Done Done Sent false
4603 2070 Acked Read 3 Done Done Sent false
4604 2071 Acked Read 8 Done Done Sent false
4605 2072 Acked Read 4 Done Done Sent false
4606 2073 Acked Read 10 Done Done Sent false
4607 2074 Acked Read 6 Done Done Sent false
4608 2075 Acked Read 10 Done Done Sent false
4609 2076 Acked Read 7 Done Done Sent false
4610 2077 Acked Write 3 Done Done Sent false
4611 2078 Acked Read 5 Done Done Sent false
4612 2079 Acked Read 4 Done Done Sent false
4613 2080 Acked Flush 0 Done Done Sent false
4614 2081 Acked Read 8 Done Done Sent false
4615 2082 Acked Read 9 Done Done Sent false
4616 2083 Acked Write 7 Done Done Sent false
4617 2084 Acked Write 2 Done Done Sent false
4618 2085 Acked Read 7 Done Done Sent false
4619 2086 Acked Flush 0 Done Done Sent false
4620 2087 Acked Read 3 Done Done Sent false
4621 2088 Acked Read 5 Done Done Sent false
4622 2089 Acked Read 6 Done Done Sent false
4623 2090 Acked Write 3 Done Done Sent false
4624 2091 Acked Write 8 Done Done Sent false
4625 2092 Acked Write 6 Done Done Sent false
4626 2093 Acked Read 6 Done Done Sent false
4627 2094 Acked Read 10 Done Done Sent false
4628 2095 Acked Flush 0 Done Done Sent false
4629 2096 Acked Flush 0 Done Done Sent false
4630 2097 Acked Write 2 Done Done Sent false
4631 2098 Acked Read 6 Done Done Sent false
4632 2099 Acked Write 3 Done Done Sent false
4633 2100 Acked Write 8 Done Done Sent false
4634 2101 Acked Read 4 Done Done Sent false
4635 2102 Acked Flush 0 Done Done Sent false
4636 2103 Acked Read 4 Done Done Sent false
4637 2104 Acked Read 3 Done Done Sent false
4638 2105 Acked Write 9 Done Done Sent false
4639 2106 Acked Read 5 Done Done Sent false
4640 2107 Acked Write 10 Done Done Sent false
4641 2108 Acked Write 10 Done Done Sent false
4642 2109 Acked Read 8 Done Done Sent false
4643 2110 Acked Read 7 Done Done Sent false
4644 2111 Acked Flush 0 Done Done Sent false
4645 2112 Acked Write 3 Done Done Sent false
4646 2113 Acked Read 1 Done Done Sent false
4647 2114 Acked Write 10 Done Done Sent false
4648 2115 Acked Write 5 Done Done Sent false
4649 2116 Acked Write 8 Done Done Sent false
4650 2117 Acked Read 1 Done Done Sent false
4651 2118 Acked Write 2 Done Done Sent false
4652 2119 Acked Write 9 Done Done Sent false
4653 2120 Acked Read 3 Done Done Sent false
4654 2121 Acked Flush 0 Done Done Sent false
4655 2122 Acked Write 7 Done Done Sent false
4656 2123 Acked Flush 0 Done Done Sent false
4657 2124 Acked Write 2 Done Done Sent false
4658 2125 Acked Write 5 Done Done Sent false
4659 2126 Acked Write 9 Done Done Sent false
4660 2127 Acked Write 5 Done Done Sent false
4661 2128 Acked Write 3 Done Done Sent false
4662 2129 Acked Read 2 Done Done Sent false
4663 2130 Acked Write 7 Done Done Sent false
4664 2131 Acked Write 6 Done Done Sent false
4665 2132 Acked Write 7 Done Done Sent false
4666 2133 Acked Write 4 Done Done Sent false
4667 2134 Acked Read 2 Done Done Sent false
4668 2135 Acked Write 4 Done Done Sent false
4669 2136 Acked Read 4 Done Done Sent false
4670 2137 Acked Write 8 Done Done Sent false
4671 2138 Acked Read 4 Done Done Sent false
4672 2139 Acked Read 2 Done Done Sent false
4673 2140 Acked Read 9 Done Done Sent false
4674 2141 Acked Read 8 Done Done Sent false
4675 2142 Acked Write 2 Done Done Sent false
4676 2143 Acked Write 7 Done Done Sent false
4677 2144 Acked Write 8 Done Done Sent false
4678 2145 Acked Read 7 Done Done Sent false
4679 2146 Acked Read 3 Done Done Sent false
4680 2147 Acked Read 3 Done Done Sent false
4681 2148 Acked Read 7 Done Done Sent false
4682 2149 Acked Read 7 Done Done Sent false
4683 2150 Acked Write 8 Done Done Sent false
4684 2151 Acked Write 4 Done Done Sent false
4685 2152 Acked Read 4 Done Done Sent false
4686 2153 Acked Read 5 Done Done Sent false
4687 2154 Acked Read 8 Done Done Sent false
4688 2155 Acked Write 2 Done Done Sent false
4689 2156 Acked Read 2 Done Done Sent false
4690 2157 Acked Read 10 Done Done Sent false
4691 2158 Acked Read 5 Done Done Sent false
4692 2159 Acked Write 9 Done Done Sent false
4693 2160 Acked Read 9 Done Done Sent false
4694 2161 Acked Read 8 Done Done Sent false
4695 2162 Acked Write 5 Done Done Sent false
4696 2163 Acked Read 3 Done Done Sent false
4697 2164 Acked Flush 0 Done Done Sent false
4698 2165 Acked Read 6 Done Done Sent false
4699 2166 Acked Write 3 Done Done Sent false
4700 2167 Acked Read 7 Done Done Sent false
4701 2168 Acked Write 7 Done Done Sent false
4702 2169 Acked Read 5 Done Done Sent false
4703 2170 Acked Write 8 Done Done Sent false
4704 2171 Acked Flush 0 Done Done Sent false
4705 2172 Acked Write 5 Done Done Sent false
4706 2173 Acked Write 3 Done Done Sent false
4707 2174 Acked Write 4 Done Done Sent false
4708 2175 Acked Read 6 Done Done Sent false
4709 2176 Acked Read 1 Done Done Sent false
4710 2177 Acked Read 9 Done Done Sent false
4711 2178 Acked Read 10 Done Done Sent false
4712 2179 Acked Read 9 Done Done Sent false
4713 2180 Acked Flush 0 Done Done Sent false
4714 2181 Acked Read 6 Done Done Sent false
4715 2182 Acked Flush 0 Done Done Sent false
4716 2183 Acked Read 10 Done Done Sent false
4717 2184 Acked Flush 0 Done Done Sent false
4718 2185 Acked Read 3 Done Done Sent false
4719 2186 Acked Read 3 Done Done Sent false
4720 2187 Acked Read 1 Done Done Sent false
4721 2188 Acked Read 1 Done Done Sent false
4722 2189 Acked Read 3 Done Done Sent false
4723 2190 Acked Read 8 Done Done Sent false
4724 2191 Acked Write 3 Done Done Sent false
4725 2192 Acked Read 2 Done Done Sent false
4726 2193 Acked Read 2 Done Done Sent false
4727 2194 Acked Read 4 Done Done Sent false
4728 2195 Acked Read 9 Done Done Sent false
4729 2196 Acked Flush 0 Done Done Sent false
4730 2197 Acked Write 5 Done Done Sent false
4731 2198 Acked Read 4 Done Done Sent false
4732 2199 Acked Write 7 Done Done Sent false
4733 2200 Acked Read 5 Done Done Sent false
4734 2201 Acked Read 8 Done Done Sent false
4735 2202 Acked Write 1 Done Done Sent false
4736 2203 Acked Read 1 Done Done Sent false
4737 2204 Acked Flush 0 Done Done Sent false
4738 2205 Acked Read 10 Done Done Sent false
4739 2206 Acked Write 6 Done Done Sent false
4740 2207 Acked Flush 0 Done Done Sent false
4741 STATES DS:0 DS:1 DS:2 TOTAL
4742 Sent 0 0 302 302
4743 Done 302 302 0 604
4744 Skipped 0 0 0 0
4745 Error 0 0 0 0
4746 Last Flush: 2207 2207 1905
4747 Downstairs last five completed: 1905 1904 1903 1902 1901
4748 Upstairs last five completed: 2207 2206 2205 2204 2203
4749 CLIENT: Up:0 ds:302 act:2
47502024-11-19T17:08:58.456ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47512024-11-19T17:08:58.456ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37569) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47522024-11-19T17:08:58.457ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47532024-11-19T17:08:58.457ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47542024-11-19T17:08:58.457ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47552024-11-19T17:08:58.458ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47562024-11-19T17:08:58.458ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47572024-11-19T17:08:58.458ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47582024-11-19T17:08:58.458ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47592024-11-19T17:08:58.458ZINFOcrucible: [2] client re-new 302 jobs since flush 1905 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
47602024-11-19T17:08:58.458ZINFOcrucible: [2] Replayed 302 jobs since flush: 1905 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
4761 ----------------------------------------------------------------
4762 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4763 Downstairs last five completed: 2207 2206 2205 2204 2203
4764 Upstairs last five completed: 2207 2206 2205 2204 2203
4765 CLIENT: Up:0 ds:0 act:3
4766 Replay: All jobs finished
4767 Main task has work to do, go find it
4768 got dsc Stop(1)
4769 stop 1
4770 Nov 19 17:09:00.699 INFO request completed, latency_us: 375, response_code: 204, uri[8820] Got stop action so:false kr:false
4771 : /stop/cid/1, method: POST, req_id: 0227c775-4b03-431c-8f45-ef790b648261, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
47722024-11-19T17:09:00.700ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47732024-11-19T17:09:00.700ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47742024-11-19T17:09:00.700ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
47752024-11-19T17:09:00.700ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47762024-11-19T17:09:00.700ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
47772024-11-19T17:09:00.700ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
4778 Nov 19 17:09:00.700 INFO request completed, latency_us: 167, response_code: 200, uri: /state/cid/1, method: GET, req_id: efd0096a-d4f6-4266-9592-bcbdfe355e5b, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
4779 [8820] Exited with: ExitStatus(unix_wait_status(9))
4780 [8820][1] reports Exit
4781 Nov 19 17:09:04.703 INFO request completed, latency_us: 287, response_code: 200, uri: /state/cid/1, method: GET, req_id: 12318867-f478-4978-ac42-db91f4996d6b, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
4782 001/300 Write block 4560 len 36864 data: 3 3 3 3 3 3 3 2 2
4783 002/300 Read block 1167 len 40960
4784 003/300 Write block 246 len 28672 data: 2 2 2 2 2 3 3
4785 004/300 Flush
4786 005/300 Read block 3009 len 20480
4787 006/300 Write block 2118 len 4096 data: 2
4788 007/300 Read block 4789 len 28672
4789 008/300 Read block 4730 len 24576
4790 009/300 Read block 1967 len 20480
4791 010/300 Write block 3089 len 4096 data: 2
4792 011/300 Read block 4993 len 4096
4793 012/300 Read block 577 len 28672
4794 013/300 Read block 3635 len 32768
4795 014/300 Read block 884 len 28672
4796 015/300 Read block 4265 len 16384
4797 016/300 Read block 920 len 32768
4798 017/300 Flush
4799 018/300 Read block 2545 len 16384
4800 019/300 Flush
4801 020/300 Read block 3058 len 8192
4802 021/300 Write block 1186 len 24576 data: 3 3 3 3 3 4
4803 022/300 Read block 572 len 32768
4804 023/300 Read block 229 len 20480
4805 024/300 Flush
4806 025/300 Read block 4856 len 4096
4807 026/300 Read block 502 len 28672
4808 027/300 Write block 4399 len 28672 data: 2 2 2 2 2 2 2
4809 028/300 Read block 916 len 20480
4810 029/300 Write block 646 len 20480 data: 3 3 5 5 5
4811 030/300 Read block 4797 len 20480
4812 031/300 Read block 1540 len 4096
4813 032/300 Write block 1245 len 32768 data: 3 3 2 2 2 2 2 2
4814 033/300 Read block 283 len 36864
4815 034/300 Read block 751 len 8192
4816 035/300 Read block 4652 len 4096
4817 036/300 Write block 2710 len 24576 data: 2 2 2 2 2 2
4818 037/300 Read block 2503 len 36864
4819 038/300 Read block 2519 len 16384
4820 039/300 Read block 340 len 32768
4821 040/300 Write block 4186 len 16384 data: 2 2 2 2
4822 041/300 Flush
4823 042/300 Read block 1380 len 20480
4824 043/300 Write block 1169 len 20480 data: 3 3 3 2 3
4825 044/300 Read block 1087 len 12288
4826 045/300 Read block 2836 len 32768
4827 046/300 Read block 40 len 36864
4828 047/300 Read block 986 len 32768
4829 048/300 Read block 3456 len 40960
4830 049/300 Write block 545 len 16384 data: 2 2 2 2
4831 050/300 Read block 3059 len 4096
4832 051/300 Flush
4833 052/300 Write block 3118 len 20480 data: 2 2 2 2 2
4834 053/300 Read block 4210 len 20480
4835 054/300 Write block 2700 len 40960 data: 2 2 2 2 2 3 3 3 3 3
4836 055/300 Write block 1174 len 20480 data: 2 2 2 2 2
4837 056/300 Read block 732 len 28672
4838 057/300 Write block 4903 len 4096 data: 2
4839 058/300 Write block 3464 len 12288 data: 2 2 2
4840 059/300 Read block 1611 len 8192
4841 060/300 Read block 5 len 12288
4842 061/300 Write block 3580 len 28672 data: 2 2 2 2 2 2 2
4843 062/300 Read block 4232 len 40960
4844 063/300 Write block 3026 len 8192 data: 3 4
4845 064/300 Read block 3522 len 12288
4846 065/300 Write block 3464 len 36864 data: 3 3 3 2 2 2 2 2 2
4847 066/300 Write block 3785 len 16384 data: 2 2 2 2
4848 067/300 Read block 4474 len 20480
4849 068/300 Read block 825 len 36864
4850 069/300 Write block 2883 len 12288 data: 3 3 3
4851 070/300 Read block 4176 len 4096
4852 071/300 Read block 1247 len 36864
4853 072/300 Read block 4039 len 4096
4854 073/300 Write block 2284 len 24576 data: 3 3 4 4 4 4
4855 074/300 Read block 4646 len 24576
4856 075/300 Read block 4955 len 28672
4857 076/300 Read block 4134 len 32768
4858 077/300 Read block 861 len 20480
4859 078/300 Write block 3097 len 28672 data: 2 2 2 2 2 2 2
4860 079/300 Write block 4583 len 36864 data: 2 2 2 2 2 2 2 3 3
4861 080/300 Write block 2621 len 8192 data: 3 3
4862 081/300 Read block 3517 len 40960
4863 082/300 Read block 4603 len 24576
4864 083/300 Read block 2241 len 16384
4865 084/300 Write block 3525 len 32768 data: 2 2 2 2 2 2 2 2
4866 085/300 Flush
4867 086/300 Flush
4868 087/300 Write block 1729 len 8192 data: 2 2
4869 088/300 Write block 1090 len 8192 data: 2 3
4870 089/300 Write block 786 len 32768 data: 3 4 4 4 3 3 3 3
4871 090/300 Read block 148 len 20480
4872 091/300 Write block 2240 len 28672 data: 2 2 3 4 3 3 3
4873 092/300 Write block 245 len 24576 data: 2 3 3 3 3 3
4874 093/300 Read block 4119 len 16384
4875 094/300 Read block 786 len 12288
4876 095/300 Read block 3434 len 20480
4877 096/300 Write block 704 len 8192 data: 2 2
4878 097/300 Read block 1718 len 32768
4879 098/300 Read block 10 len 4096
4880 099/300 Flush
4881 100/300 Read block 2074 len 12288
4882 101/300 Read block 4348 len 36864
4883 102/300 Write block 4529 len 36864 data: 2 2 2 2 2 2 2 2 2
4884 103/300 Read block 2464 len 8192
4885 104/300 Read block 398 len 24576
4886 105/300 Read block 4198 len 36864
4887 106/300 Write block 2891 len 8192 data: 3 3
4888 107/300 Read block 4776 len 28672
4889 108/300 Flush
4890 109/300 Read block 364 len 24576
4891 110/300 Flush
4892 111/300 Flush
4893 112/300 Write block 3181 len 40960 data: 2 2 2 2 2 2 3 3 3 2
4894 113/300 Read block 2991 len 20480
4895 114/300 Read block 3489 len 8192
4896 115/300 Read block 1633 len 4096
4897 116/300 Write block 1678 len 16384 data: 2 2 2 2
4898 117/300 Read block 3406 len 24576
4899 118/300 Write block 3218 len 4096 data: 2
4900 119/300 Write block 2990 len 32768 data: 2 2 2 2 2 2 2 2
4901 120/300 Read block 3879 len 20480
4902 121/300 Write block 1893 len 8192 data: 3 2
4903 122/300 Write block 1679 len 4096 data: 3
4904 123/300 Read block 3567 len 24576
4905 124/300 Flush
4906 125/300 Read block 845 len 8192
4907 126/300 Write block 471 len 28672 data: 5 5 6 6 6 6 6
4908 127/300 Write block 1659 len 8192 data: 2 2
4909 128/300 Write block 4597 len 4096 data: 3
4910 129/300 Write block 403 len 8192 data: 3 2
4911 130/300 Write block 4452 len 32768 data: 2 2 2 2 2 2 2 2
4912 131/300 Flush
4913 132/300 Write block 1428 len 8192 data: 3 3
4914 133/300 Read block 4821 len 32768
4915 134/300 Read block 3811 len 20480
4916 135/300 Read block 195 len 28672
4917 136/300 Read block 891 len 16384
4918 137/300 Read block 4882 len 4096
4919 138/300 Read block 4816 len 40960
4920 139/300 Write block 98 len 40960 data: 3 3 4 4 3 3 2 2 2 2
4921 140/300 Read block 3684 len 4096
4922 141/300 Flush
4923 142/300 Write block 3748 len 4096 data: 2
4924 143/300 Write block 2201 len 4096 data: 2
4925 144/300 Write block 1029 len 20480 data: 2 2 2 2 2
4926 145/300 Write block 120 len 32768 data: 2 2 2 2 2 2 2 3
4927 146/300 Read block 1884 len 20480
4928 147/300 Write block 2919 len 16384 data: 3 3 4 3
4929 148/300 Write block 2450 len 4096 data: 2
4930 149/300 Read block 2420 len 36864
4931 150/300 Read block 3051 len 8192
4932 151/300 Read block 2362 len 40960
4933 152/300 Read block 100 len 36864
4934 153/300 Read block 4989 len 4096
4935 154/300 Read block 2563 len 36864
4936 155/300 Write block 2301 len 20480 data: 4 4 3 4 5
4937 156/300 Flush
4938 157/300 Read block 2115 len 32768
4939 158/300 Write block 522 len 40960 data: 2 2 2 2 2 3 3 2 2 2
4940 159/300 Write block 4463 len 4096 data: 2
4941 160/300 Read block 1869 len 8192
4942 161/300 Flush
4943 162/300 Write block 1604 len 36864 data: 4 4 4 4 3 3 3 3 2
4944 163/300 Read block 2570 len 40960
4945 164/300 Read block 3623 len 12288
4946 165/300 Read block 3412 len 36864
4947 166/300 Read block 3523 len 16384
4948 167/300 Flush
4949 168/300 Write block 3218 len 24576 data: 3 2 2 2 2 3
4950 169/300 Write block 1232 len 16384 data: 4 4 4 4
4951 170/300 Flush
4952 171/300 Write block 2221 len 4096 data: 3
4953 172/300 Write block 2518 len 8192 data: 2 2
4954 173/300 Read block 110 len 28672
4955 174/300 Flush
4956 175/300 Read block 1625 len 24576
4957 176/300 Read block 2680 len 32768
4958 177/300 Write block 2930 len 32768 data: 2 3 3 3 3 2 2 2
4959 178/300 Read block 3634 len 20480
4960 179/300 Read block 1069 len 28672
4961 180/300 Read block 864 len 32768
4962 181/300 Write block 2610 len 28672 data: 5 5 4 4 3 3 4
4963 182/300 Read block 2963 len 40960
4964 183/300 Write block 3297 len 16384 data: 2 2 2 2
4965 184/300 Read block 4973 len 8192
4966 185/300 Write block 4217 len 4096 data: 5
4967 186/300 Read block 3339 len 16384
4968 187/300 Write block 654 len 4096 data: 4
4969 188/300 Flush
4970 189/300 Flush
4971 190/300 Flush
4972 191/300 Flush
4973 192/300 Write block 1447 len 8192 data: 3 3
4974 193/300 Read block 3917 len 32768
4975 194/300 Write block 3489 len 20480 data: 2 2 2 2 2
4976 195/300 Write block 2064 len 36864 data: 2 2 2 2 2 2 2 2 2
4977 196/300 Write block 935 len 20480 data: 3 2 2 2 2
4978 197/300 Read block 4763 len 16384
4979 198/300 Write block 3721 len 40960 data: 2 2 2 2 2 2 2 2 3 2
4980 199/300 Read block 2121 len 8192
4981 200/300 Read block 328 len 12288
4982 201/300 Read block 116 len 20480
4983 202/300 Flush
4984 203/300 Write block 1366 len 32768 data: 2 2 2 2 2 2 2 3
4985 204/300 Write block 2254 len 4096 data: 3
4986 205/300 Write block 1822 len 28672 data: 3 3 3 3 3 2 2
4987 206/300 Read block 4672 len 40960
4988 207/300 Read block 373 len 36864
4989 208/300 Write block 3556 len 28672 data: 2 2 2 2 2 2 2
4990 209/300 Write block 4929 len 20480 data: 3 3 3 4 5
4991 210/300 Write block 3617 len 8192 data: 3 3
4992 211/300 Write block 1269 len 4096 data: 4
4993 212/300 Write block 4620 len 32768 data: 3 3 3 3 3 2 2 2
4994 213/300 Write block 615 len 24576 data: 2 2 2 3 3 3
4995 214/300 Read block 2989 len 24576
4996 215/300 Read block 2195 len 16384
4997 216/300 Read block 3723 len 8192
4998 217/300 Write block 4649 len 16384 data: 3 3 3 4
4999 218/300 Flush
5000 219/300 Read block 1748 len 32768
5001 220/300 Read block 179 len 4096
5002 221/300 Write block 3484 len 24576 data: 3 3 3 3 2 3
5003 222/300 Read block 2623 len 40960
5004 223/300 Read block 3654 len 16384
5005 224/300 Read block 4617 len 12288
5006 225/300 Flush
5007 226/300 Write block 3386 len 32768 data: 4 4 4 4 4 4 4 3
5008 227/300 Write block 789 len 24576 data: 5 4 4 4 4 3
5009 228/300 Write block 2703 len 36864 data: 3 3 4 4 4 4 4 3 3
5010 229/300 Write block 3212 len 4096 data: 2
5011 230/300 Write block 4027 len 12288 data: 3 3 2
5012 231/300 Read block 893 len 4096
5013 232/300 Read block 2210 len 32768
5014 233/300 Flush
5015 234/300 Write block 2149 len 32768 data: 2 2 2 2 2 2 2 2
5016 235/300 Read block 901 len 20480
5017 236/300 Read block 1 len 4096
5018 237/300 Flush
5019 238/300 Write block 3279 len 16384 data: 2 2 2 2
5020 239/300 Read block 349 len 20480
5021 240/300 Write block 4843 len 16384 data: 2 2 2 2
5022 241/300 Read block 4152 len 36864
5023 242/300 Read block 1185 len 36864
5024 243/300 Read block 4951 len 16384
5025 244/300 Write block 3822 len 40960 data: 3 3 3 3 3 3 2 3 3 3
5026 245/300 Read block 3540 len 8192
5027 246/300 Read block 3777 len 4096
5028 247/300 Read block 146 len 24576
5029 248/300 Read block 4665 len 8192
5030 249/300 Write block 1246 len 20480 data: 4 3 3 3 3
5031 250/300 Read block 3313 len 12288
5032 251/300 Write block 4858 len 28672 data: 2 2 2 2 2 2 2
5033 252/300 Read block 2048 len 8192
5034 253/300 Write block 834 len 12288 data: 2 2 2
5035 254/300 Read block 2695 len 36864
5036 255/300 Read block 1882 len 28672
5037 256/300 Read block 4739 len 24576
5038 257/300 Write block 2070 len 40960 data: 3 3 3 2 2 2 2 2 2 2
5039 258/300 Read block 1981 len 20480
5040 259/300 Read block 2148 len 36864
5041 260/300 Read block 4097 len 12288
5042 261/300 Read block 2460 len 24576
5043 262/300 Read block 3353 len 8192
5044 263/300 Read block 1454 len 8192
5045 264/300 Write block 1285 len 8192 data: 2 2
5046 265/300 Read block 1629 len 20480
5047 266/300 Write block 3704 len 4096 data: 3
5048 267/300 Read block 4557 len 12288
5049 268/300 Read block 2160 len 28672
5050 269/300 Read block 3824 len 32768
5051 270/300 Read block 4557 len 24576
5052 271/300 Write block 4926 len 8192 data: 2 3
5053 272/300 Write block 4264 len 20480 data: 2 2 2 2 2
5054 273/300 Write block 139 len 36864 data: 5 4 4 4 4 3 2 2 2
5055 274/300 Write block 1565 len 4096 data: 2
5056 275/300 Read block 2948 len 28672
5057 276/300 Write block 3144 len 24576 data: 2 2 3 3 3 3
5058 277/300 Write block 1077 len 12288 data: 3 3 2
5059 278/300 Read block 3522 len 4096
5060 279/300 Write block 3749 len 32768 data: 2 2 2 3 3 3 3 4
5061 280/300 Read block 1604 len 28672
5062 281/300 Read block 3692 len 12288
5063 282/300 Write block 874 len 24576 data: 3 3 2 2 3 3
5064 283/300 Read block 4475 len 20480
5065 284/300 Read block 3551 len 20480
5066 285/300 Read block 916 len 32768
5067 286/300 Read block 646 len 20480
5068 287/300 Write block 3547 len 24576 data: 2 2 2 2 2 3
5069 288/300 Read block 4913 len 16384
5070 289/300 Write block 3130 len 12288 data: 4 4 4
5071 290/300 Read block 3172 len 16384
5072 291/300 Write block 1396 len 20480 data: 2 2 2 3 4
5073 292/300 Read block 3474 len 24576
5074 293/300 Read block 1617 len 28672
5075 294/300 Read block 4791 len 4096
5076 295/300 Flush
5077 296/300 Write block 1792 len 32768 data: 2 2 2 2 2 2 2 2
5078 297/300 Read block 3149 len 28672
5079 298/300 Flush
5080 299/300 Flush
5081 300/300 Write block 887 len 36864 data: 3 2 2 2 3 3 3 3 3
5082 301/300 Flush
5083 Main task has work to do, go find it
5084 got dsc Start(1)
5085 start 1
5086 Nov 19 17:09:04.938 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/1, method: POST, req_id: a499b148-a1c0-4bb4-998a-2bee2b495a63, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
5087 Replay: started 1, returned:Ok(())
5088 ----------------------------------------------------------------
5089 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5090 States: Active Offline Active
5091 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5092 2208 Acked Write 9 Done Sent Done false
5093 2209 Acked Read 10 Done Sent Done false
5094 2210 Acked Write 7 Done Sent Done false
5095 2211 Acked Flush 0 Done Sent Done false
5096 2212 Acked Read 5 Done Sent Done false
5097 2213 Acked Write 1 Done Sent Done false
5098 2214 Acked Read 7 Done Sent Done false
5099 2215 Acked Read 6 Done Sent Done false
5100 2216 Acked Read 5 Done Sent Done false
5101 2217 Acked Write 1 Done Sent Done false
5102 2218 Acked Read 1 Done Sent Done false
5103 2219 Acked Read 7 Done Sent Done false
5104 2220 Acked Read 8 Done Sent Done false
5105 2221 Acked Read 7 Done Sent Done false
5106 2222 Acked Read 4 Done Sent Done false
5107 2223 Acked Read 8 Done Sent Done false
5108 2224 Acked Flush 0 Done Sent Done false
5109 2225 Acked Read 4 Done Sent Done false
5110 2226 Acked Flush 0 Done Sent Done false
5111 2227 Acked Read 2 Done Sent Done false
5112 2228 Acked Write 6 Done Sent Done false
5113 2229 Acked Read 8 Done Sent Done false
5114 2230 Acked Read 5 Done Sent Done false
5115 2231 Acked Flush 0 Done Sent Done false
5116 2232 Acked Read 1 Done Sent Done false
5117 2233 Acked Read 7 Done Sent Done false
5118 2234 Acked Write 7 Done Sent Done false
5119 2235 Acked Read 5 Done Sent Done false
5120 2236 Acked Write 5 Done Sent Done false
5121 2237 Acked Read 5 Done Sent Done false
5122 2238 Acked Read 1 Done Sent Done false
5123 2239 Acked Write 8 Done Sent Done false
5124 2240 Acked Read 9 Done Sent Done false
5125 2241 Acked Read 2 Done Sent Done false
5126 2242 Acked Read 1 Done Sent Done false
5127 2243 Acked Write 6 Done Sent Done false
5128 2244 Acked Read 9 Done Sent Done false
5129 2245 Acked Read 4 Done Sent Done false
5130 2246 Acked Read 8 Done Sent Done false
5131 2247 Acked Write 4 Done Sent Done false
5132 2248 Acked Flush 0 Done Sent Done false
5133 2249 Acked Read 5 Done Sent Done false
5134 2250 Acked Write 5 Done Sent Done false
5135 2251 Acked Read 3 Done Sent Done false
5136 2252 Acked Read 8 Done Sent Done false
5137 2253 Acked Read 9 Done Sent Done false
5138 2254 Acked Read 8 Done Sent Done false
5139 2255 Acked Read 10 Done Sent Done false
5140 2256 Acked Write 4 Done Sent Done false
5141 2257 Acked Read 1 Done Sent Done false
5142 2258 Acked Flush 0 Done Sent Done false
5143 2259 Acked Write 5 Done Sent Done false
5144 2260 Acked Read 5 Done Sent Done false
5145 2261 Acked Write 10 Done Sent Done false
5146 2262 Acked Write 5 Done Sent Done false
5147 2263 Acked Read 7 Done Sent Done false
5148 2264 Acked Write 1 Done Sent Done false
5149 2265 Acked Write 3 Done Sent Done false
5150 2266 Acked Read 2 Done Sent Done false
5151 2267 Acked Read 3 Done Sent Done false
5152 2268 Acked Write 7 Done Sent Done false
5153 2269 Acked Read 10 Done Sent Done false
5154 2270 Acked Write 2 Done Sent Done false
5155 2271 Acked Read 3 Done Sent Done false
5156 2272 Acked Write 9 Done Sent Done false
5157 2273 Acked Write 4 Done Sent Done false
5158 2274 Acked Read 5 Done Sent Done false
5159 2275 Acked Read 9 Done Sent Done false
5160 2276 Acked Write 3 Done Sent Done false
5161 2277 Acked Read 1 Done Sent Done false
5162 2278 Acked Read 9 Done Sent Done false
5163 2279 Acked Read 1 Done Sent Done false
5164 2280 Acked Write 6 Done Sent Done false
5165 2281 Acked Read 6 Done Sent Done false
5166 2282 Acked Read 7 Done Sent Done false
5167 2283 Acked Read 8 Done Sent Done false
5168 2284 Acked Read 5 Done Sent Done false
5169 2285 Acked Write 7 Done Sent Done false
5170 2286 Acked Write 9 Done Sent Done false
5171 2287 Acked Write 2 Done Sent Done false
5172 2288 Acked Read 10 Done Sent Done false
5173 2289 Acked Read 6 Done Sent Done false
5174 2290 Acked Read 4 Done Sent Done false
5175 2291 Acked Write 8 Done Sent Done false
5176 2292 Acked Flush 0 Done Sent Done false
5177 2293 Acked Flush 0 Done Sent Done false
5178 2294 Acked Write 2 Done Sent Done false
5179 2295 Acked Write 2 Done Sent Done false
5180 2296 Acked Write 8 Done Sent Done false
5181 2297 Acked Read 5 Done Sent Done false
5182 2298 Acked Write 7 Done Sent Done false
5183 2299 Acked Write 6 Done Sent Done false
5184 2300 Acked Read 4 Done Sent Done false
5185 2301 Acked Read 3 Done Sent Done false
5186 2302 Acked Read 5 Done Sent Done false
5187 2303 Acked Write 2 Done Sent Done false
5188 2304 Acked Read 8 Done Sent Done false
5189 2305 Acked Read 1 Done Sent Done false
5190 2306 Acked Flush 0 Done Sent Done false
5191 2307 Acked Read 3 Done Sent Done false
5192 2308 Acked Read 9 Done Sent Done false
5193 2309 Acked Write 9 Done Sent Done false
5194 2310 Acked Read 2 Done Sent Done false
5195 2311 Acked Read 6 Done Sent Done false
5196 2312 Acked Read 9 Done Sent Done false
5197 2313 Acked Write 2 Done Sent Done false
5198 2314 Acked Read 7 Done Sent Done false
5199 2315 Acked Flush 0 Done Sent Done false
5200 2316 Acked Read 6 Done Sent Done false
5201 2317 Acked Flush 0 Done Sent Done false
5202 2318 Acked Flush 0 Done Sent Done false
5203 2319 Acked Write 10 Done Sent Done false
5204 2320 Acked Read 5 Done Sent Done false
5205 2321 Acked Read 2 Done Sent Done false
5206 2322 Acked Read 1 Done Sent Done false
5207 2323 Acked Write 4 Done Sent Done false
5208 2324 Acked Read 6 Done Sent Done false
5209 2325 Acked Write 1 Done Sent Done false
5210 2326 Acked Write 8 Done Sent Done false
5211 2327 Acked Read 5 Done Sent Done false
5212 2328 Acked Write 2 Done Sent Done false
5213 2329 Acked Write 1 Done Sent Done false
5214 2330 Acked Read 6 Done Sent Done false
5215 2331 Acked Flush 0 Done Sent Done false
5216 2332 Acked Read 2 Done Sent Done false
5217 2333 Acked Write 7 Done Sent Done false
5218 2334 Acked Write 2 Done Sent Done false
5219 2335 Acked Write 1 Done Sent Done false
5220 2336 Acked Write 2 Done Sent Done false
5221 2337 Acked Write 8 Done Sent Done false
5222 2338 Acked Flush 0 Done Sent Done false
5223 2339 Acked Write 2 Done Sent Done false
5224 2340 Acked Read 8 Done Sent Done false
5225 2341 Acked Read 5 Done Sent Done false
5226 2342 Acked Read 7 Done Sent Done false
5227 2343 Acked Read 4 Done Sent Done false
5228 2344 Acked Read 1 Done Sent Done false
5229 2345 Acked Read 10 Done Sent Done false
5230 2346 Acked Write 10 Done Sent Done false
5231 2347 Acked Read 1 Done Sent Done false
5232 2348 Acked Flush 0 Done Sent Done false
5233 2349 Acked Write 1 Done Sent Done false
5234 2350 Acked Write 1 Done Sent Done false
5235 2351 Acked Write 5 Done Sent Done false
5236 2352 Acked Write 8 Done Sent Done false
5237 2353 Acked Read 5 Done Sent Done false
5238 2354 Acked Write 4 Done Sent Done false
5239 2355 Acked Write 1 Done Sent Done false
5240 2356 Acked Read 9 Done Sent Done false
5241 2357 Acked Read 2 Done Sent Done false
5242 2358 Acked Read 10 Done Sent Done false
5243 2359 Acked Read 9 Done Sent Done false
5244 2360 Acked Read 1 Done Sent Done false
5245 2361 Acked Read 9 Done Sent Done false
5246 2362 Acked Write 5 Done Sent Done false
5247 2363 Acked Flush 0 Done Sent Done false
5248 2364 Acked Read 8 Done Sent Done false
5249 2365 Acked Write 10 Done Sent Done false
5250 2366 Acked Write 1 Done Sent Done false
5251 2367 Acked Read 2 Done Sent Done false
5252 2368 Acked Flush 0 Done Sent Done false
5253 2369 Acked Write 9 Done Sent Done false
5254 2370 Acked Read 10 Done Sent Done false
5255 2371 Acked Read 3 Done Sent Done false
5256 2372 Acked Read 9 Done Sent Done false
5257 2373 Acked Read 4 Done Sent Done false
5258 2374 Acked Flush 0 Done Sent Done false
5259 2375 Acked Write 6 Done Sent Done false
5260 2376 Acked Write 4 Done Sent Done false
5261 2377 Acked Flush 0 Done Sent Done false
5262 2378 Acked Write 1 Done Sent Done false
5263 2379 Acked Write 2 Done Sent Done false
5264 2380 Acked Read 7 Done Sent Done false
5265 2381 Acked Flush 0 Done Sent Done false
5266 2382 Acked Read 6 Done Sent Done false
5267 2383 Acked Read 8 Done Sent Done false
5268 2384 Acked Write 8 Done Sent Done false
5269 2385 Acked Read 5 Done Sent Done false
5270 2386 Acked Read 7 Done Sent Done false
5271 2387 Acked Read 8 Done Sent Done false
5272 2388 Acked Write 7 Done Sent Done false
5273 2389 Acked Read 10 Done Sent Done false
5274 2390 Acked Write 4 Done Sent Done false
5275 2391 Acked Read 2 Done Sent Done false
5276 2392 Acked Write 1 Done Sent Done false
5277 2393 Acked Read 4 Done Sent Done false
5278 2394 Acked Write 1 Done Sent Done false
5279 2395 Acked Flush 0 Done Sent Done false
5280 2396 Acked Flush 0 Done Sent Done false
5281 2397 Acked Flush 0 Done Sent Done false
5282 2398 Acked Flush 0 Done Sent Done false
5283 2399 Acked Write 2 Done Sent Done false
5284 2400 Acked Read 8 Done Sent Done false
5285 2401 Acked Write 5 Done Sent Done false
5286 2402 Acked Write 9 Done Sent Done false
5287 2403 Acked Write 5 Done Sent Done false
5288 2404 Acked Read 4 Done Sent Done false
5289 2405 Acked Write 10 Done Sent Done false
5290 2406 Acked Read 2 Done Sent Done false
5291 2407 Acked Read 3 Done Sent Done false
5292 2408 Acked Read 5 Done Sent Done false
5293 2409 Acked Flush 0 Done Sent Done false
5294 2410 Acked Write 8 Done Sent Done false
5295 2411 Acked Write 1 Done Sent Done false
5296 2412 Acked Write 7 Done Sent Done false
5297 2413 Acked Read 10 Done Sent Done false
5298 2414 Acked Read 9 Done Sent Done false
5299 2415 Acked Write 7 Done Sent Done false
5300 2416 Acked Write 5 Done Sent Done false
5301 2417 Acked Write 2 Done Sent Done false
5302 2418 Acked Write 1 Done Sent Done false
5303 2419 Acked Write 8 Done Sent Done false
5304 2420 Acked Write 6 Done Sent Done false
5305 2421 Acked Read 6 Done Sent Done false
5306 2422 Acked Read 4 Done Sent Done false
5307 2423 Acked Read 2 Done Sent Done false
5308 2424 Acked Write 4 Done Sent Done false
5309 2425 Acked Flush 0 Done Sent Done false
5310 2426 Acked Read 8 Done Sent Done false
5311 2427 Acked Read 1 Done Sent Done false
5312 2428 Acked Write 6 Done Sent Done false
5313 2429 Acked Read 10 Done Sent Done false
5314 2430 Acked Read 4 Done Sent Done false
5315 2431 Acked Read 3 Done Sent Done false
5316 2432 Acked Flush 0 Done Sent Done false
5317 2433 Acked Write 8 Done Sent Done false
5318 2434 Acked Write 6 Done Sent Done false
5319 2435 Acked Write 9 Done Sent Done false
5320 2436 Acked Write 1 Done Sent Done false
5321 2437 Acked Write 3 Done Sent Done false
5322 2438 Acked Read 1 Done Sent Done false
5323 2439 Acked Read 8 Done Sent Done false
5324 2440 Acked Flush 0 Done Sent Done false
5325 2441 Acked Write 8 Done Sent Done false
5326 2442 Acked Read 5 Done Sent Done false
5327 2443 Acked Read 1 Done Sent Done false
5328 2444 Acked Flush 0 Done Sent Done false
5329 2445 Acked Write 4 Done Sent Done false
5330 2446 Acked Read 5 Done Sent Done false
5331 2447 Acked Write 4 Done Sent Done false
5332 2448 Acked Read 9 Done Sent Done false
5333 2449 Acked Read 9 Done Sent Done false
5334 2450 Acked Read 4 Done Sent Done false
5335 2451 Acked Write 10 Done Sent Done false
5336 2452 Acked Read 2 Done Sent Done false
5337 2453 Acked Read 1 Done Sent Done false
5338 2454 Acked Read 6 Done Sent Done false
5339 2455 Acked Read 2 Done Sent Done false
5340 2456 Acked Write 5 Done Sent Done false
5341 2457 Acked Read 3 Done Sent Done false
5342 2458 Acked Write 7 Done Sent Done false
5343 2459 Acked Read 2 Done Sent Done false
5344 2460 Acked Write 3 Done Sent Done false
5345 2461 Acked Read 9 Done Sent Done false
5346 2462 Acked Read 7 Done Sent Done false
5347 2463 Acked Read 6 Done Sent Done false
5348 2464 Acked Write 10 Done Sent Done false
5349 2465 Acked Read 5 Done Sent Done false
5350 2466 Acked Read 9 Done Sent Done false
5351 2467 Acked Read 3 Done Sent Done false
5352 2468 Acked Read 6 Done Sent Done false
5353 2469 Acked Read 2 Done Sent Done false
5354 2470 Acked Read 2 Done Sent Done false
5355 2471 Acked Write 2 Done Sent Done false
5356 2472 Acked Read 5 Done Sent Done false
5357 2473 Acked Write 1 Done Sent Done false
5358 2474 Acked Read 3 Done Sent Done false
5359 2475 Acked Read 7 Done Sent Done false
5360 2476 Acked Read 8 Done Sent Done false
5361 2477 Acked Read 6 Done Sent Done false
5362 2478 Acked Write 2 Done Sent Done false
5363 2479 Acked Write 5 Done Sent Done false
5364 2480 Acked Write 9 Done Sent Done false
5365 2481 Acked Write 1 Done Sent Done false
5366 2482 Acked Read 7 Done Sent Done false
5367 2483 Acked Write 6 Done Sent Done false
5368 2484 Acked Write 3 Done Sent Done false
5369 2485 Acked Read 1 Done Sent Done false
5370 2486 Acked Write 8 Done Sent Done false
5371 2487 Acked Read 7 Done Sent Done false
5372 2488 Acked Read 3 Done Sent Done false
5373 2489 Acked Write 6 Done Sent Done false
5374 2490 Acked Read 5 Done Sent Done false
5375 2491 Acked Read 5 Done Sent Done false
5376 2492 Acked Read 8 Done Sent Done false
5377 2493 Acked Read 5 Done Sent Done false
5378 2494 Acked Write 6 Done Sent Done false
5379 2495 Acked Read 4 Done Sent Done false
5380 2496 Acked Write 3 Done Sent Done false
5381 2497 Acked Read 4 Done Sent Done false
5382 2498 Acked Write 5 Done Sent Done false
5383 2499 Acked Read 6 Done Sent Done false
5384 2500 Acked Read 7 Done Sent Done false
5385 2501 Acked Read 1 Done Sent Done false
5386 2502 Acked Flush 0 Done Sent Done false
5387 2503 Acked Write 8 Done Sent Done false
5388 2504 Acked Read 7 Done Sent Done false
5389 2505 Acked Flush 0 Done Sent Done false
5390 2506 Acked Flush 0 Done Sent Done false
5391 2507 Acked Write 9 Done Sent Done false
5392 2508 Acked Flush 0 Done Sent Done false
5393 STATES DS:0 DS:1 DS:2 TOTAL
5394 Sent 0 301 0 301
5395 Done 301 0 301 602
5396 Skipped 0 0 0 0
5397 Error 0 0 0 0
5398 Last Flush: 2508 2207 2508
5399 Downstairs last five completed: 2207 2206 2205 2204 2203
5400 Upstairs last five completed: 2508 2507 2506 2505 2504
5401 CLIENT: Up:0 ds:301 act:2
5402 [8820] got start action
5403 [8820] I am going to restart
5404 Starting downstairs at port 8820
5405 Make output file at "/tmp/dsc/downstairs-8820.txt"
5406 [8820][1] reports Starting
5407 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1207)
5408 [8820][1] reports Running
5409 ----------------------------------------------------------------
5410 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5411 States: Active Offline Active
5412 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5413 2208 Acked Write 9 Done Sent Done false
5414 2209 Acked Read 10 Done Sent Done false
5415 2210 Acked Write 7 Done Sent Done false
5416 2211 Acked Flush 0 Done Sent Done false
5417 2212 Acked Read 5 Done Sent Done false
5418 2213 Acked Write 1 Done Sent Done false
5419 2214 Acked Read 7 Done Sent Done false
5420 2215 Acked Read 6 Done Sent Done false
5421 2216 Acked Read 5 Done Sent Done false
5422 2217 Acked Write 1 Done Sent Done false
5423 2218 Acked Read 1 Done Sent Done false
5424 2219 Acked Read 7 Done Sent Done false
5425 2220 Acked Read 8 Done Sent Done false
5426 2221 Acked Read 7 Done Sent Done false
5427 2222 Acked Read 4 Done Sent Done false
5428 2223 Acked Read 8 Done Sent Done false
5429 2224 Acked Flush 0 Done Sent Done false
5430 2225 Acked Read 4 Done Sent Done false
5431 2226 Acked Flush 0 Done Sent Done false
5432 2227 Acked Read 2 Done Sent Done false
5433 2228 Acked Write 6 Done Sent Done false
5434 2229 Acked Read 8 Done Sent Done false
5435 2230 Acked Read 5 Done Sent Done false
5436 2231 Acked Flush 0 Done Sent Done false
5437 2232 Acked Read 1 Done Sent Done false
5438 2233 Acked Read 7 Done Sent Done false
5439 2234 Acked Write 7 Done Sent Done false
5440 2235 Acked Read 5 Done Sent Done false
5441 2236 Acked Write 5 Done Sent Done false
5442 2237 Acked Read 5 Done Sent Done false
5443 2238 Acked Read 1 Done Sent Done false
5444 2239 Acked Write 8 Done Sent Done false
5445 2240 Acked Read 9 Done Sent Done false
5446 2241 Acked Read 2 Done Sent Done false
5447 2242 Acked Read 1 Done Sent Done false
5448 2243 Acked Write 6 Done Sent Done false
5449 2244 Acked Read 9 Done Sent Done false
5450 2245 Acked Read 4 Done Sent Done false
5451 2246 Acked Read 8 Done Sent Done false
5452 2247 Acked Write 4 Done Sent Done false
5453 2248 Acked Flush 0 Done Sent Done false
5454 2249 Acked Read 5 Done Sent Done false
5455 2250 Acked Write 5 Done Sent Done false
5456 2251 Acked Read 3 Done Sent Done false
5457 2252 Acked Read 8 Done Sent Done false
5458 2253 Acked Read 9 Done Sent Done false
5459 2254 Acked Read 8 Done Sent Done false
5460 2255 Acked Read 10 Done Sent Done false
5461 2256 Acked Write 4 Done Sent Done false
5462 2257 Acked Read 1 Done Sent Done false
5463 2258 Acked Flush 0 Done Sent Done false
5464 2259 Acked Write 5 Done Sent Done false
5465 2260 Acked Read 5 Done Sent Done false
5466 2261 Acked Write 10 Done Sent Done false
5467 2262 Acked Write 5 Done Sent Done false
5468 2263 Acked Read 7 Done Sent Done false
5469 2264 Acked Write 1 Done Sent Done false
5470 2265 Acked Write 3 Done Sent Done false
5471 2266 Acked Read 2 Done Sent Done false
5472 2267 Acked Read 3 Done Sent Done false
5473 2268 Acked Write 7 Done Sent Done false
5474 2269 Acked Read 10 Done Sent Done false
5475 2270 Acked Write 2 Done Sent Done false
5476 2271 Acked Read 3 Done Sent Done false
5477 2272 Acked Write 9 Done Sent Done false
5478 2273 Acked Write 4 Done Sent Done false
5479 2274 Acked Read 5 Done Sent Done false
5480 2275 Acked Read 9 Done Sent Done false
5481 2276 Acked Write 3 Done Sent Done false
5482 2277 Acked Read 1 Done Sent Done false
5483 2278 Acked Read 9 Done Sent Done false
5484 2279 Acked Read 1 Done Sent Done false
5485 2280 Acked Write 6 Done Sent Done false
5486 2281 Acked Read 6 Done Sent Done false
5487 2282 Acked Read 7 Done Sent Done false
5488 2283 Acked Read 8 Done Sent Done false
5489 2284 Acked Read 5 Done Sent Done false
5490 2285 Acked Write 7 Done Sent Done false
5491 2286 Acked Write 9 Done Sent Done false
5492 2287 Acked Write 2 Done Sent Done false
5493 2288 Acked Read 10 Done Sent Done false
5494 2289 Acked Read 6 Done Sent Done false
5495 2290 Acked Read 4 Done Sent Done false
5496 2291 Acked Write 8 Done Sent Done false
5497 2292 Acked Flush 0 Done Sent Done false
5498 2293 Acked Flush 0 Done Sent Done false
5499 2294 Acked Write 2 Done Sent Done false
5500 2295 Acked Write 2 Done Sent Done false
5501 2296 Acked Write 8 Done Sent Done false
5502 2297 Acked Read 5 Done Sent Done false
5503 2298 Acked Write 7 Done Sent Done false
5504 2299 Acked Write 6 Done Sent Done false
5505 2300 Acked Read 4 Done Sent Done false
5506 2301 Acked Read 3 Done Sent Done false
5507 2302 Acked Read 5 Done Sent Done false
5508 2303 Acked Write 2 Done Sent Done false
5509 2304 Acked Read 8 Done Sent Done false
5510 2305 Acked Read 1 Done Sent Done false
5511 2306 Acked Flush 0 Done Sent Done false
5512 2307 Acked Read 3 Done Sent Done false
5513 2308 Acked Read 9 Done Sent Done false
5514 2309 Acked Write 9 Done Sent Done false
5515 2310 Acked Read 2 Done Sent Done false
5516 2311 Acked Read 6 Done Sent Done false
5517 2312 Acked Read 9 Done Sent Done false
5518 2313 Acked Write 2 Done Sent Done false
5519 2314 Acked Read 7 Done Sent Done false
5520 2315 Acked Flush 0 Done Sent Done false
5521 2316 Acked Read 6 Done Sent Done false
5522 2317 Acked Flush 0 Done Sent Done false
5523 2318 Acked Flush 0 Done Sent Done false
5524 2319 Acked Write 10 Done Sent Done false
5525 2320 Acked Read 5 Done Sent Done false
5526 2321 Acked Read 2 Done Sent Done false
5527 2322 Acked Read 1 Done Sent Done false
5528 2323 Acked Write 4 Done Sent Done false
5529 2324 Acked Read 6 Done Sent Done false
5530 2325 Acked Write 1 Done Sent Done false
5531 2326 Acked Write 8 Done Sent Done false
5532 2327 Acked Read 5 Done Sent Done false
5533 2328 Acked Write 2 Done Sent Done false
5534 2329 Acked Write 1 Done Sent Done false
5535 2330 Acked Read 6 Done Sent Done false
5536 2331 Acked Flush 0 Done Sent Done false
5537 2332 Acked Read 2 Done Sent Done false
5538 2333 Acked Write 7 Done Sent Done false
5539 2334 Acked Write 2 Done Sent Done false
5540 2335 Acked Write 1 Done Sent Done false
5541 2336 Acked Write 2 Done Sent Done false
5542 2337 Acked Write 8 Done Sent Done false
5543 2338 Acked Flush 0 Done Sent Done false
5544 2339 Acked Write 2 Done Sent Done false
5545 2340 Acked Read 8 Done Sent Done false
5546 2341 Acked Read 5 Done Sent Done false
5547 2342 Acked Read 7 Done Sent Done false
5548 2343 Acked Read 4 Done Sent Done false
5549 2344 Acked Read 1 Done Sent Done false
5550 2345 Acked Read 10 Done Sent Done false
5551 2346 Acked Write 10 Done Sent Done false
5552 2347 Acked Read 1 Done Sent Done false
5553 2348 Acked Flush 0 Done Sent Done false
5554 2349 Acked Write 1 Done Sent Done false
5555 2350 Acked Write 1 Done Sent Done false
5556 2351 Acked Write 5 Done Sent Done false
5557 2352 Acked Write 8 Done Sent Done false
5558 2353 Acked Read 5 Done Sent Done false
5559 2354 Acked Write 4 Done Sent Done false
5560 2355 Acked Write 1 Done Sent Done false
5561 2356 Acked Read 9 Done Sent Done false
5562 2357 Acked Read 2 Done Sent Done false
5563 2358 Acked Read 10 Done Sent Done false
5564 2359 Acked Read 9 Done Sent Done false
5565 2360 Acked Read 1 Done Sent Done false
5566 2361 Acked Read 9 Done Sent Done false
5567 2362 Acked Write 5 Done Sent Done false
5568 2363 Acked Flush 0 Done Sent Done false
5569 2364 Acked Read 8 Done Sent Done false
5570 2365 Acked Write 10 Done Sent Done false
5571 2366 Acked Write 1 Done Sent Done false
5572 2367 Acked Read 2 Done Sent Done false
5573 2368 Acked Flush 0 Done Sent Done false
5574 2369 Acked Write 9 Done Sent Done false
5575 2370 Acked Read 10 Done Sent Done false
5576 2371 Acked Read 3 Done Sent Done false
5577 2372 Acked Read 9 Done Sent Done false
5578 2373 Acked Read 4 Done Sent Done false
5579 2374 Acked Flush 0 Done Sent Done false
5580 2375 Acked Write 6 Done Sent Done false
5581 2376 Acked Write 4 Done Sent Done false
5582 2377 Acked Flush 0 Done Sent Done false
5583 2378 Acked Write 1 Done Sent Done false
5584 2379 Acked Write 2 Done Sent Done false
5585 2380 Acked Read 7 Done Sent Done false
5586 2381 Acked Flush 0 Done Sent Done false
5587 2382 Acked Read 6 Done Sent Done false
5588 2383 Acked Read 8 Done Sent Done false
5589 2384 Acked Write 8 Done Sent Done false
5590 2385 Acked Read 5 Done Sent Done false
5591 2386 Acked Read 7 Done Sent Done false
5592 2387 Acked Read 8 Done Sent Done false
5593 2388 Acked Write 7 Done Sent Done false
5594 2389 Acked Read 10 Done Sent Done false
5595 2390 Acked Write 4 Done Sent Done false
5596 2391 Acked Read 2 Done Sent Done false
5597 2392 Acked Write 1 Done Sent Done false
5598 2393 Acked Read 4 Done Sent Done false
5599 2394 Acked Write 1 Done Sent Done false
5600 2395 Acked Flush 0 Done Sent Done false
5601 2396 Acked Flush 0 Done Sent Done false
5602 2397 Acked Flush 0 Done Sent Done false
5603 2398 Acked Flush 0 Done Sent Done false
5604 2399 Acked Write 2 Done Sent Done false
5605 2400 Acked Read 8 Done Sent Done false
5606 2401 Acked Write 5 Done Sent Done false
5607 2402 Acked Write 9 Done Sent Done false
5608 2403 Acked Write 5 Done Sent Done false
5609 2404 Acked Read 4 Done Sent Done false
5610 2405 Acked Write 10 Done Sent Done false
5611 2406 Acked Read 2 Done Sent Done false
5612 2407 Acked Read 3 Done Sent Done false
5613 2408 Acked Read 5 Done Sent Done false
5614 2409 Acked Flush 0 Done Sent Done false
5615 2410 Acked Write 8 Done Sent Done false
5616 2411 Acked Write 1 Done Sent Done false
5617 2412 Acked Write 7 Done Sent Done false
5618 2413 Acked Read 10 Done Sent Done false
5619 2414 Acked Read 9 Done Sent Done false
5620 2415 Acked Write 7 Done Sent Done false
5621 2416 Acked Write 5 Done Sent Done false
5622 2417 Acked Write 2 Done Sent Done false
5623 2418 Acked Write 1 Done Sent Done false
5624 2419 Acked Write 8 Done Sent Done false
5625 2420 Acked Write 6 Done Sent Done false
5626 2421 Acked Read 6 Done Sent Done false
5627 2422 Acked Read 4 Done Sent Done false
5628 2423 Acked Read 2 Done Sent Done false
5629 2424 Acked Write 4 Done Sent Done false
5630 2425 Acked Flush 0 Done Sent Done false
5631 2426 Acked Read 8 Done Sent Done false
5632 2427 Acked Read 1 Done Sent Done false
5633 2428 Acked Write 6 Done Sent Done false
5634 2429 Acked Read 10 Done Sent Done false
5635 2430 Acked Read 4 Done Sent Done false
5636 2431 Acked Read 3 Done Sent Done false
5637 2432 Acked Flush 0 Done Sent Done false
5638 2433 Acked Write 8 Done Sent Done false
5639 2434 Acked Write 6 Done Sent Done false
5640 2435 Acked Write 9 Done Sent Done false
5641 2436 Acked Write 1 Done Sent Done false
5642 2437 Acked Write 3 Done Sent Done false
5643 2438 Acked Read 1 Done Sent Done false
5644 2439 Acked Read 8 Done Sent Done false
5645 2440 Acked Flush 0 Done Sent Done false
5646 2441 Acked Write 8 Done Sent Done false
5647 2442 Acked Read 5 Done Sent Done false
5648 2443 Acked Read 1 Done Sent Done false
5649 2444 Acked Flush 0 Done Sent Done false
5650 2445 Acked Write 4 Done Sent Done false
5651 2446 Acked Read 5 Done Sent Done false
5652 2447 Acked Write 4 Done Sent Done false
5653 2448 Acked Read 9 Done Sent Done false
5654 2449 Acked Read 9 Done Sent Done false
5655 2450 Acked Read 4 Done Sent Done false
5656 2451 Acked Write 10 Done Sent Done false
5657 2452 Acked Read 2 Done Sent Done false
5658 2453 Acked Read 1 Done Sent Done false
5659 2454 Acked Read 6 Done Sent Done false
5660 2455 Acked Read 2 Done Sent Done false
5661 2456 Acked Write 5 Done Sent Done false
5662 2457 Acked Read 3 Done Sent Done false
5663 2458 Acked Write 7 Done Sent Done false
5664 2459 Acked Read 2 Done Sent Done false
5665 2460 Acked Write 3 Done Sent Done false
5666 2461 Acked Read 9 Done Sent Done false
5667 2462 Acked Read 7 Done Sent Done false
5668 2463 Acked Read 6 Done Sent Done false
5669 2464 Acked Write 10 Done Sent Done false
5670 2465 Acked Read 5 Done Sent Done false
5671 2466 Acked Read 9 Done Sent Done false
5672 2467 Acked Read 3 Done Sent Done false
5673 2468 Acked Read 6 Done Sent Done false
5674 2469 Acked Read 2 Done Sent Done false
5675 2470 Acked Read 2 Done Sent Done false
5676 2471 Acked Write 2 Done Sent Done false
5677 2472 Acked Read 5 Done Sent Done false
5678 2473 Acked Write 1 Done Sent Done false
5679 2474 Acked Read 3 Done Sent Done false
5680 2475 Acked Read 7 Done Sent Done false
5681 2476 Acked Read 8 Done Sent Done false
5682 2477 Acked Read 6 Done Sent Done false
5683 2478 Acked Write 2 Done Sent Done false
5684 2479 Acked Write 5 Done Sent Done false
5685 2480 Acked Write 9 Done Sent Done false
5686 2481 Acked Write 1 Done Sent Done false
5687 2482 Acked Read 7 Done Sent Done false
5688 2483 Acked Write 6 Done Sent Done false
5689 2484 Acked Write 3 Done Sent Done false
5690 2485 Acked Read 1 Done Sent Done false
5691 2486 Acked Write 8 Done Sent Done false
5692 2487 Acked Read 7 Done Sent Done false
5693 2488 Acked Read 3 Done Sent Done false
5694 2489 Acked Write 6 Done Sent Done false
5695 2490 Acked Read 5 Done Sent Done false
5696 2491 Acked Read 5 Done Sent Done false
5697 2492 Acked Read 8 Done Sent Done false
5698 2493 Acked Read 5 Done Sent Done false
5699 2494 Acked Write 6 Done Sent Done false
5700 2495 Acked Read 4 Done Sent Done false
5701 2496 Acked Write 3 Done Sent Done false
5702 2497 Acked Read 4 Done Sent Done false
5703 2498 Acked Write 5 Done Sent Done false
5704 2499 Acked Read 6 Done Sent Done false
5705 2500 Acked Read 7 Done Sent Done false
5706 2501 Acked Read 1 Done Sent Done false
5707 2502 Acked Flush 0 Done Sent Done false
5708 2503 Acked Write 8 Done Sent Done false
5709 2504 Acked Read 7 Done Sent Done false
5710 2505 Acked Flush 0 Done Sent Done false
5711 2506 Acked Flush 0 Done Sent Done false
5712 2507 Acked Write 9 Done Sent Done false
5713 2508 Acked Flush 0 Done Sent Done false
5714 STATES DS:0 DS:1 DS:2 TOTAL
5715 Sent 0 301 0 301
5716 Done 301 0 301 602
5717 Skipped 0 0 0 0
5718 Error 0 0 0 0
5719 Last Flush: 2508 2207 2508
5720 Downstairs last five completed: 2207 2206 2205 2204 2203
5721 Upstairs last five completed: 2508 2507 2506 2505 2504
5722 CLIENT: Up:0 ds:301 act:2
57232024-11-19T17:09:10.702ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57242024-11-19T17:09:10.702ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35515) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57252024-11-19T17:09:10.703ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57262024-11-19T17:09:10.704ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57272024-11-19T17:09:10.704ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57282024-11-19T17:09:10.704ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57292024-11-19T17:09:10.704ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57302024-11-19T17:09:10.704ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57312024-11-19T17:09:10.704ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57322024-11-19T17:09:10.704ZINFOcrucible: [1] client re-new 301 jobs since flush 2207 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
57332024-11-19T17:09:10.704ZINFOcrucible: [1] Replayed 301 jobs since flush: 2207 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
5734 ----------------------------------------------------------------
5735 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5736 Downstairs last five completed: 2508 2507 2506 2505 2504
5737 Upstairs last five completed: 2508 2507 2506 2505 2504
5738 CLIENT: Up:0 ds:0 act:3
5739 Replay: All jobs finished
5740 Main task has work to do, go find it
5741 got dsc Stop(1)
5742 stop 1
5743 Nov 19 17:09:12.943 INFO request completed, latency_us: 325, response_code:[8820] Got stop action so:false kr:false
5744 204, uri: /stop/cid/1, method: POST, req_id: 912f434e-8e16-4921-b26b-c8745ef49587, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
57452024-11-19T17:09:12.943ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57462024-11-19T17:09:12.944ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
57472024-11-19T17:09:12.944ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
5748 Nov 19 17:09:12.944 INFO request completed, latency_us: 140, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:09:12.94412905Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
5749 response_code: 200, uri: /state/cid/1, method: GET, req_id: 5a0da9c0-0f2a-42c2-b4af-cad32deb32b7, remote_addr: 127.{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:09:12.944159137Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
5750 0.0.1:64175, local_addr: 127.0.0.1:9998
57512024-11-19T17:09:12.944ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
5752 [8820] Exited with: ExitStatus(unix_wait_status(9))
5753 [8820][1] reports Exit
5754 Nov 19 17:09:16.945 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/1, method: GET, req_id: 6a854ed8-176e-4c25-90b1-7a5582659942, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
5755 001/300 Read block 2966 len 20480
5756 002/300 Read block 3502 len 32768
5757 003/300 Read block 1286 len 16384
5758 004/300 Read block 4447 len 32768
5759 005/300 Write block 1187 len 40960 data: 4 4 4 4 5 4 4 4 4 4
5760 006/300 Read block 2733 len 12288
5761 007/300 Flush
5762 008/300 Write block 4512 len 40960 data: 2 2 2 2 3 4 4 4 3 3
5763 009/300 Write block 4459 len 36864 data: 3 2 2 2 3 2 2 2 2
5764 010/300 Read block 1260 len 20480
5765 011/300 Read block 1970 len 32768
5766 012/300 Read block 4670 len 28672
5767 013/300 Write block 2032 len 32768 data: 2 2 2 2 2 2 2 2
5768 014/300 Read block 1376 len 36864
5769 015/300 Read block 3854 len 24576
5770 016/300 Write block 2466 len 12288 data: 2 2 2
5771 017/300 Read block 109 len 24576
5772 018/300 Flush
5773 019/300 Read block 2081 len 16384
5774 020/300 Read block 3355 len 32768
5775 021/300 Write block 2536 len 36864 data: 3 3 2 2 2 2 2 2 2
5776 022/300 Read block 699 len 16384
5777 023/300 Flush
5778 024/300 Read block 4895 len 28672
5779 025/300 Read block 4066 len 8192
5780 026/300 Flush
5781 027/300 Read block 381 len 20480
5782 028/300 Write block 258 len 36864 data: 3 2 2 2 2 2 2 3 3
5783 029/300 Read block 2113 len 32768
5784 030/300 Write block 4392 len 28672 data: 2 2 2 2 2 2 2
5785 031/300 Read block 2230 len 28672
5786 032/300 Read block 1750 len 20480
5787 033/300 Write block 468 len 32768 data: 2 4 4 6 6 7 7 7
5788 034/300 Write block 1883 len 40960 data: 3 4 4 4 4 4 3 4 4 3
5789 035/300 Read block 1445 len 16384
5790 036/300 Read block 4680 len 8192
5791 037/300 Read block 4626 len 16384
5792 038/300 Flush
5793 039/300 Flush
5794 040/300 Read block 3088 len 16384
5795 041/300 Read block 3164 len 32768
5796 042/300 Read block 4287 len 32768
5797 043/300 Write block 3659 len 20480 data: 2 2 2 2 2
5798 044/300 Read block 2553 len 12288
5799 045/300 Write block 4365 len 20480 data: 4 2 2 2 2
5800 046/300 Read block 1062 len 16384
5801 047/300 Read block 2473 len 4096
5802 048/300 Read block 1293 len 32768
5803 049/300 Read block 885 len 8192
5804 050/300 Write block 818 len 12288 data: 2 2 2
5805 051/300 Read block 2707 len 4096
5806 052/300 Read block 4007 len 40960
5807 053/300 Write block 3125 len 20480 data: 4 4 4 4 4
5808 054/300 Write block 2364 len 16384 data: 3 3 3 3
5809 055/300 Read block 4305 len 20480
5810 056/300 Read block 4799 len 24576
5811 057/300 Read block 1359 len 16384
5812 058/300 Read block 1983 len 28672
5813 059/300 Write block 137 len 4096 data: 5
5814 060/300 Write block 2209 len 20480 data: 4 4 4 4 5
5815 061/300 Write block 4157 len 4096 data: 3
5816 062/300 Read block 899 len 4096
5817 063/300 Write block 3271 len 8192 data: 2 2
5818 064/300 Read block 616 len 36864
5819 065/300 Write block 2572 len 12288 data: 2 2 2
5820 066/300 Flush
5821 067/300 Flush
5822 068/300 Write block 780 len 20480 data: 2 3 3 3 3
5823 069/300 Read block 207 len 24576
5824 070/300 Read block 464 len 8192
5825 071/300 Read block 3074 len 4096
5826 072/300 Read block 2178 len 28672
5827 073/300 Read block 2320 len 12288
5828 074/300 Write block 3370 len 32768 data: 2 2 3 2 2 2 2 2
5829 075/300 Flush
5830 076/300 Read block 1551 len 32768
5831 077/300 Read block 2853 len 4096
5832 078/300 Read block 1102 len 8192
5833 079/300 Read block 2470 len 32768
5834 080/300 Read block 2316 len 12288
5835 081/300 Write block 1467 len 4096 data: 2
5836 082/300 Write block 824 len 36864 data: 2 2 2 2 2 2 2 2 2
5837 083/300 Write block 1539 len 4096 data: 2
5838 084/300 Write block 2830 len 20480 data: 2 2 2 3 3
5839 085/300 Read block 1488 len 8192
5840 086/300 Write block 4424 len 16384 data: 4 3 4 4
5841 087/300 Write block 1643 len 20480 data: 2 2 2 2 2
5842 088/300 Write block 3746 len 36864 data: 3 3 3 3 3 3 4 4 4
5843 089/300 Read block 3136 len 36864
5844 090/300 Read block 4197 len 40960
5845 091/300 Read block 2643 len 36864
5846 092/300 Flush
5847 093/300 Read block 1366 len 20480
5848 094/300 Write block 2129 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5849 095/300 Read block 3648 len 36864
5850 096/300 Write block 335 len 16384 data: 2 2 2 2
5851 097/300 Read block 4238 len 16384
5852 098/300 Read block 4043 len 12288
5853 099/300 Read block 906 len 32768
5854 100/300 Write block 4871 len 20480 data: 2 2 2 2 2
5855 101/300 Write block 2727 len 16384 data: 4 5 5 4
5856 102/300 Write block 3553 len 8192 data: 2 2
5857 103/300 Read block 4174 len 4096
5858 104/300 Flush
5859 105/300 Read block 3976 len 12288
5860 106/300 Write block 3317 len 40960 data: 2 2 2 2 3 3 3 2 2 2
5861 107/300 Write block 784 len 20480 data: 4 3 4 5 5
5862 108/300 Write block 4014 len 24576 data: 2 2 2 2 2 2
5863 109/300 Read block 3228 len 4096
5864 110/300 Read block 1898 len 4096
5865 111/300 Write block 3080 len 36864 data: 2 2 2 2 2 2 2 2 2
5866 112/300 Write block 1756 len 20480 data: 4 4 3 3 3
5867 113/300 Write block 1580 len 12288 data: 3 3 3
5868 114/300 Flush
5869 115/300 Write block 4471 len 20480 data: 2 3 3 4 4
5870 116/300 Read block 4460 len 20480
5871 117/300 Read block 4660 len 24576
5872 118/300 Write block 2787 len 4096 data: 3
5873 119/300 Write block 4592 len 20480 data: 3 3 3 3 3
5874 120/300 Flush
5875 121/300 Read block 4085 len 40960
5876 122/300 Write block 2974 len 20480 data: 2 2 2 2 2
5877 123/300 Write block 129 len 24576 data: 4 4 4 4 5 5
5878 124/300 Read block 1239 len 8192
5879 125/300 Read block 1604 len 16384
5880 126/300 Write block 2095 len 20480 data: 3 2 3 3 3
5881 127/300 Flush
5882 128/300 Read block 1859 len 20480
5883 129/300 Read block 998 len 32768
5884 130/300 Write block 3229 len 8192 data: 2 2
5885 131/300 Read block 3837 len 32768
5886 132/300 Read block 2623 len 36864
5887 133/300 Write block 4434 len 36864 data: 4 3 3 4 4 4 4 3 2
5888 134/300 Read block 784 len 40960
5889 135/300 Write block 1406 len 24576 data: 4 4 3 3 3 4
5890 136/300 Flush
5891 137/300 Read block 4719 len 32768
5892 138/300 Flush
5893 139/300 Write block 2744 len 40960 data: 2 2 2 2 2 2 3 3 3 3
5894 140/300 Write block 3992 len 16384 data: 3 3 3 3
5895 141/300 Read block 1555 len 36864
5896 142/300 Write block 4732 len 36864 data: 2 2 2 2 2 2 2 2 2
5897 143/300 Read block 1647 len 40960
5898 144/300 Write block 517 len 8192 data: 2 2
5899 145/300 Read block 944 len 24576
5900 146/300 Read block 1267 len 32768
5901 147/300 Write block 3881 len 36864 data: 4 3 3 3 2 2 2 3 2
5902 148/300 Read block 2658 len 24576
5903 149/300 Read block 1991 len 36864
5904 150/300 Write block 2297 len 36864 data: 2 3 3 4 5 5 4 5 6
5905 151/300 Write block 535 len 40960 data: 3 3 3 3 3 3 3 3 3 2
5906 152/300 Write block 3140 len 16384 data: 2 2 2 2
5907 153/300 Write block 1279 len 32768 data: 2 2 2 2 2 2 3 3
5908 154/300 Write block 3240 len 12288 data: 3 3 3
5909 155/300 Read block 519 len 16384
5910 156/300 Write block 1545 len 4096 data: 4
5911 157/300 Write block 2582 len 28672 data: 3 3 3 2 2 2 2
5912 158/300 Read block 3162 len 4096
5913 159/300 Write block 4087 len 4096 data: 2
5914 160/300 Read block 478 len 36864
5915 161/300 Read block 3882 len 20480
5916 162/300 Write block 2505 len 20480 data: 3 3 3 3 3
5917 163/300 Write block 3672 len 20480 data: 3 3 3 3 2
5918 164/300 Read block 2905 len 40960
5919 165/300 Read block 3820 len 20480
5920 166/300 Write block 1691 len 28672 data: 3 3 4 4 4 4 4
5921 167/300 Write block 4487 len 28672 data: 3 3 3 3 4 4 4
5922 168/300 Write block 2044 len 16384 data: 2 2 2 2
5923 169/300 Read block 1999 len 32768
5924 170/300 Write block 3740 len 24576 data: 6 6 6 6 5 4
5925 171/300 Read block 870 len 20480
5926 172/300 Read block 91 len 28672
5927 173/300 Flush
5928 174/300 Read block 3279 len 4096
5929 175/300 Write block 4192 len 24576 data: 2 2 2 2 2 2
5930 176/300 Read block 185 len 28672
5931 177/300 Flush
5932 178/300 Read block 277 len 32768
5933 179/300 Read block 3339 len 36864
5934 180/300 Read block 3882 len 20480
5935 181/300 Write block 4634 len 36864 data: 4 3 3 3 3 3 3 2 2
5936 182/300 Write block 798 len 4096 data: 2
5937 183/300 Write block 2375 len 16384 data: 2 2 2 3
5938 184/300 Read block 3052 len 40960
5939 185/300 Read block 2194 len 28672
5940 186/300 Write block 1995 len 12288 data: 2 3 2
5941 187/300 Read block 854 len 40960
5942 188/300 Read block 4511 len 12288
5943 189/300 Read block 4556 len 16384
5944 190/300 Read block 2116 len 40960
5945 191/300 Read block 4685 len 40960
5946 192/300 Flush
5947 193/300 Read block 3476 len 24576
5948 194/300 Read block 2810 len 16384
5949 195/300 Flush
5950 196/300 Write block 2206 len 16384 data: 2 2 3 5
5951 197/300 Write block 2272 len 20480 data: 2 2 3 3 4
5952 198/300 Read block 2116 len 24576
5953 199/300 Read block 2961 len 4096
5954 200/300 Read block 2939 len 8192
5955 201/300 Write block 1647 len 32768 data: 3 2 2 2 2 2 2 2
5956 202/300 Write block 3448 len 24576 data: 2 2 2 3 3 3
5957 203/300 Read block 72 len 16384
5958 204/300 Write block 380 len 40960 data: 2 2 2 2 3 4 4 4 5 5
5959 205/300 Read block 2744 len 40960
5960 206/300 Read block 680 len 40960
5961 207/300 Read block 2523 len 4096
5962 208/300 Flush
5963 209/300 Write block 424 len 12288 data: 3 3 3
5964 210/300 Write block 1299 len 4096 data: 2
5965 211/300 Write block 3735 len 24576 data: 2 4 4 4 5 7
5966 212/300 Write block 2831 len 20480 data: 3 3 4 4 3
5967 213/300 Write block 1607 len 36864 data: 5 4 4 4 4 3 3 3 3
5968 214/300 Read block 2404 len 32768
5969 215/300 Read block 4351 len 12288
5970 216/300 Write block 988 len 24576 data: 2 2 2 3 3 3
5971 217/300 Read block 4744 len 12288
5972 218/300 Write block 4001 len 20480 data: 3 3 3 3 3
5973 219/300 Read block 3106 len 16384
5974 220/300 Read block 4196 len 32768
5975 221/300 Write block 976 len 16384 data: 2 2 2 2
5976 222/300 Write block 4689 len 24576 data: 2 2 2 2 2 2
5977 223/300 Read block 1668 len 4096
5978 224/300 Read block 134 len 16384
5979 225/300 Read block 1747 len 24576
5980 226/300 Flush
5981 227/300 Write block 2926 len 20480 data: 2 2 2 2 3
5982 228/300 Write block 1708 len 8192 data: 3 2
5983 229/300 Read block 753 len 12288
5984 230/300 Write block 4952 len 4096 data: 2
5985 231/300 Write block 2493 len 8192 data: 2 2
5986 232/300 Write block 4062 len 32768 data: 2 2 2 2 2 2 2 3
5987 233/300 Read block 3554 len 4096
5988 234/300 Write block 3275 len 20480 data: 2 2 2 2 3
5989 235/300 Read block 2048 len 40960
5990 236/300 Write block 3547 len 8192 data: 3 3
5991 237/300 Flush
5992 238/300 Read block 1746 len 8192
5993 239/300 Read block 503 len 16384
5994 240/300 Read block 3029 len 12288
5995 241/300 Read block 4412 len 20480
5996 242/300 Flush
5997 243/300 Write block 3739 len 28672 data: 6 8 7 7 7 6 5
5998 244/300 Read block 873 len 40960
5999 245/300 Read block 3898 len 4096
6000 246/300 Write block 3048 len 20480 data: 2 2 2 2 2
6001 247/300 Write block 1345 len 16384 data: 3 3 3 4
6002 248/300 Read block 3637 len 8192
6003 249/300 Read block 2324 len 36864
6004 250/300 Flush
6005 251/300 Read block 1847 len 12288
6006 252/300 Read block 279 len 24576
6007 253/300 Write block 1540 len 12288 data: 2 2 3
6008 254/300 Read block 3104 len 36864
6009 255/300 Read block 4611 len 8192
6010 256/300 Write block 1198 len 24576 data: 2 2 2 2 3 3
6011 257/300 Write block 2392 len 12288 data: 4 3 3
6012 258/300 Write block 2496 len 24576 data: 2 2 2 2 2 2
6013 259/300 Read block 4919 len 28672
6014 260/300 Read block 3011 len 40960
6015 261/300 Read block 3533 len 32768
6016 262/300 Read block 2216 len 4096
6017 263/300 Read block 2118 len 36864
6018 264/300 Write block 1162 len 32768 data: 3 4 4 3 3 3 3 4
6019 265/300 Read block 4929 len 12288
6020 266/300 Read block 1681 len 24576
6021 267/300 Write block 3563 len 40960 data: 2 3 3 3 3 2 2 2 2 2
6022 268/300 Read block 4973 len 24576
6023 269/300 Read block 4762 len 28672
6024 270/300 Flush
6025 271/300 Write block 4029 len 20480 data: 3 2 2 3 3
6026 272/300 Write block 1200 len 20480 data: 3 3 4 4 3
6027 273/300 Write block 996 len 24576 data: 2 2 2 2 2 2
6028 274/300 Write block 3214 len 20480 data: 2 3 3 2 4
6029 275/300 Write block 4552 len 28672 data: 3 3 3 3 2 3 3
6030 276/300 Write block 2930 len 28672 data: 4 4 4 4 4 3 3
6031 277/300 Read block 844 len 24576
6032 278/300 Read block 2093 len 12288
6033 279/300 Write block 197 len 32768 data: 2 2 2 2 2 2 2 2
6034 280/300 Read block 4964 len 40960
6035 281/300 Write block 1405 len 28672 data: 4 5 5 4 4 4 5
6036 282/300 Read block 3465 len 4096
6037 283/300 Read block 101 len 16384
6038 284/300 Read block 1282 len 24576
6039 285/300 Read block 741 len 28672
6040 286/300 Write block 4694 len 12288 data: 3 3 2
6041 287/300 Read block 1713 len 28672
6042 288/300 Write block 3043 len 16384 data: 2 2 2 2
6043 289/300 Read block 2173 len 4096
6044 290/300 Read block 1723 len 8192
6045 291/300 Read block 1297 len 12288
6046 292/300 Flush
6047 293/300 Read block 2055 len 12288
6048 294/300 Write block 3544 len 4096 data: 2
6049 295/300 Read block 3072 len 16384
6050 296/300 Write block 945 len 36864 data: 3 3 3 3 2 2 2 2 2
6051 297/300 Write block 1100 len 20480 data: 3 3 2 2 2
6052 298/300 Read block 1628 len 24576
6053 299/300 Flush
6054 300/300 Write block 1546 len 8192 data: 4 4
6055 301/300 Write block 4909 len 24576 data: 4 4 4 4 4 4
6056 Main task has work to do, go find it
6057 got dsc Start(1)
6058 start 1
6059 Nov 19 17:09:17.184 INFO request completed, latency_us: 263, response_code: 204, uri: /start/cid/1, method: POST, req_id: 601610a0-ba82-4834-9faa-cd91c06ba394, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
6060 Replay: started 1, returned:Ok(())
6061 ----------------------------------------------------------------
6062 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6063 States: Active Offline Active
6064 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6065 2509 Acked Read 5 Done Sent Done false
6066 2510 Acked Read 8 Done Sent Done false
6067 2511 Acked Read 4 Done Sent Done false
6068 2512 Acked Read 8 Done Sent Done false
6069 2513 Acked Write 10 Done Sent Done false
6070 2514 Acked Read 3 Done Sent Done false
6071 2515 Acked Flush 0 Done Sent Done false
6072 2516 Acked Write 10 Done Sent Done false
6073 2517 Acked Write 9 Done Sent Done false
6074 2518 Acked Read 5 Done Sent Done false
6075 2519 Acked Read 8 Done Sent Done false
6076 2520 Acked Read 7 Done Sent Done false
6077 2521 Acked Write 8 Done Sent Done false
6078 2522 Acked Read 9 Done Sent Done false
6079 2523 Acked Read 6 Done Sent Done false
6080 2524 Acked Write 3 Done Sent Done false
6081 2525 Acked Read 6 Done Sent Done false
6082 2526 Acked Flush 0 Done Sent Done false
6083 2527 Acked Read 4 Done Sent Done false
6084 2528 Acked Read 8 Done Sent Done false
6085 2529 Acked Write 9 Done Sent Done false
6086 2530 Acked Read 4 Done Sent Done false
6087 2531 Acked Flush 0 Done Sent Done false
6088 2532 Acked Read 7 Done Sent Done false
6089 2533 Acked Read 2 Done Sent Done false
6090 2534 Acked Flush 0 Done Sent Done false
6091 2535 Acked Read 5 Done Sent Done false
6092 2536 Acked Write 9 Done Sent Done false
6093 2537 Acked Read 8 Done Sent Done false
6094 2538 Acked Write 7 Done Sent Done false
6095 2539 Acked Read 7 Done Sent Done false
6096 2540 Acked Read 5 Done Sent Done false
6097 2541 Acked Write 8 Done Sent Done false
6098 2542 Acked Write 10 Done Sent Done false
6099 2543 Acked Read 4 Done Sent Done false
6100 2544 Acked Read 2 Done Sent Done false
6101 2545 Acked Read 4 Done Sent Done false
6102 2546 Acked Flush 0 Done Sent Done false
6103 2547 Acked Flush 0 Done Sent Done false
6104 2548 Acked Read 4 Done Sent Done false
6105 2549 Acked Read 8 Done Sent Done false
6106 2550 Acked Read 8 Done Sent Done false
6107 2551 Acked Write 5 Done Sent Done false
6108 2552 Acked Read 3 Done Sent Done false
6109 2553 Acked Write 5 Done Sent Done false
6110 2554 Acked Read 4 Done Sent Done false
6111 2555 Acked Read 1 Done Sent Done false
6112 2556 Acked Read 8 Done Sent Done false
6113 2557 Acked Read 2 Done Sent Done false
6114 2558 Acked Write 3 Done Sent Done false
6115 2559 Acked Read 1 Done Sent Done false
6116 2560 Acked Read 10 Done Sent Done false
6117 2561 Acked Write 5 Done Sent Done false
6118 2562 Acked Write 4 Done Sent Done false
6119 2563 Acked Read 5 Done Sent Done false
6120 2564 Acked Read 6 Done Sent Done false
6121 2565 Acked Read 4 Done Sent Done false
6122 2566 Acked Read 7 Done Sent Done false
6123 2567 Acked Write 1 Done Sent Done false
6124 2568 Acked Write 5 Done Sent Done false
6125 2569 Acked Write 1 Done Sent Done false
6126 2570 Acked Read 1 Done Sent Done false
6127 2571 Acked Write 2 Done Sent Done false
6128 2572 Acked Read 9 Done Sent Done false
6129 2573 Acked Write 3 Done Sent Done false
6130 2574 Acked Flush 0 Done Sent Done false
6131 2575 Acked Flush 0 Done Sent Done false
6132 2576 Acked Write 5 Done Sent Done false
6133 2577 Acked Read 6 Done Sent Done false
6134 2578 Acked Read 2 Done Sent Done false
6135 2579 Acked Read 1 Done Sent Done false
6136 2580 Acked Read 7 Done Sent Done false
6137 2581 Acked Read 3 Done Sent Done false
6138 2582 Acked Write 8 Done Sent Done false
6139 2583 Acked Flush 0 Done Sent Done false
6140 2584 Acked Read 8 Done Sent Done false
6141 2585 Acked Read 1 Done Sent Done false
6142 2586 Acked Read 2 Done Sent Done false
6143 2587 Acked Read 8 Done Sent Done false
6144 2588 Acked Read 3 Done Sent Done false
6145 2589 Acked Write 1 Done Sent Done false
6146 2590 Acked Write 9 Done Sent Done false
6147 2591 Acked Write 1 Done Sent Done false
6148 2592 Acked Write 5 Done Sent Done false
6149 2593 Acked Read 2 Done Sent Done false
6150 2594 Acked Write 4 Done Sent Done false
6151 2595 Acked Write 5 Done Sent Done false
6152 2596 Acked Write 9 Done Sent Done false
6153 2597 Acked Read 9 Done Sent Done false
6154 2598 Acked Read 10 Done Sent Done false
6155 2599 Acked Read 9 Done Sent Done false
6156 2600 Acked Flush 0 Done Sent Done false
6157 2601 Acked Read 5 Done Sent Done false
6158 2602 Acked Write 10 Done Sent Done false
6159 2603 Acked Read 9 Done Sent Done false
6160 2604 Acked Write 4 Done Sent Done false
6161 2605 Acked Read 4 Done Sent Done false
6162 2606 Acked Read 3 Done Sent Done false
6163 2607 Acked Read 8 Done Sent Done false
6164 2608 Acked Write 5 Done Sent Done false
6165 2609 Acked Write 4 Done Sent Done false
6166 2610 Acked Write 2 Done Sent Done false
6167 2611 Acked Read 1 Done Sent Done false
6168 2612 Acked Flush 0 Done Sent Done false
6169 2613 Acked Read 3 Done Sent Done false
6170 2614 Acked Write 10 Done Sent Done false
6171 2615 Acked Write 5 Done Sent Done false
6172 2616 Acked Write 6 Done Sent Done false
6173 2617 Acked Read 1 Done Sent Done false
6174 2618 Acked Read 1 Done Sent Done false
6175 2619 Acked Write 9 Done Sent Done false
6176 2620 Acked Write 5 Done Sent Done false
6177 2621 Acked Write 3 Done Sent Done false
6178 2622 Acked Flush 0 Done Sent Done false
6179 2623 Acked Write 5 Done Sent Done false
6180 2624 Acked Read 5 Done Sent Done false
6181 2625 Acked Read 6 Done Sent Done false
6182 2626 Acked Write 1 Done Sent Done false
6183 2627 Acked Write 5 Done Sent Done false
6184 2628 Acked Flush 0 Done Sent Done false
6185 2629 Acked Read 10 Done Sent Done false
6186 2630 Acked Write 5 Done Sent Done false
6187 2631 Acked Write 6 Done Sent Done false
6188 2632 Acked Read 2 Done Sent Done false
6189 2633 Acked Read 4 Done Sent Done false
6190 2634 Acked Write 5 Done Sent Done false
6191 2635 Acked Flush 0 Done Sent Done false
6192 2636 Acked Read 5 Done Sent Done false
6193 2637 Acked Read 8 Done Sent Done false
6194 2638 Acked Write 2 Done Sent Done false
6195 2639 Acked Read 8 Done Sent Done false
6196 2640 Acked Read 9 Done Sent Done false
6197 2641 Acked Write 9 Done Sent Done false
6198 2642 Acked Read 10 Done Sent Done false
6199 2643 Acked Write 6 Done Sent Done false
6200 2644 Acked Flush 0 Done Sent Done false
6201 2645 Acked Read 8 Done Sent Done false
6202 2646 Acked Flush 0 Done Sent Done false
6203 2647 Acked Write 10 Done Sent Done false
6204 2648 Acked Write 4 Done Sent Done false
6205 2649 Acked Read 9 Done Sent Done false
6206 2650 Acked Write 9 Done Sent Done false
6207 2651 Acked Read 10 Done Sent Done false
6208 2652 Acked Write 2 Done Sent Done false
6209 2653 Acked Read 6 Done Sent Done false
6210 2654 Acked Read 8 Done Sent Done false
6211 2655 Acked Write 9 Done Sent Done false
6212 2656 Acked Read 6 Done Sent Done false
6213 2657 Acked Read 9 Done Sent Done false
6214 2658 Acked Write 9 Done Sent Done false
6215 2659 Acked Write 10 Done Sent Done false
6216 2660 Acked Write 4 Done Sent Done false
6217 2661 Acked Write 8 Done Sent Done false
6218 2662 Acked Write 3 Done Sent Done false
6219 2663 Acked Read 4 Done Sent Done false
6220 2664 Acked Write 1 Done Sent Done false
6221 2665 Acked Write 7 Done Sent Done false
6222 2666 Acked Read 1 Done Sent Done false
6223 2667 Acked Write 1 Done Sent Done false
6224 2668 Acked Read 9 Done Sent Done false
6225 2669 Acked Read 5 Done Sent Done false
6226 2670 Acked Write 5 Done Sent Done false
6227 2671 Acked Write 5 Done Sent Done false
6228 2672 Acked Read 10 Done Sent Done false
6229 2673 Acked Read 5 Done Sent Done false
6230 2674 Acked Write 7 Done Sent Done false
6231 2675 Acked Write 7 Done Sent Done false
6232 2676 Acked Write 4 Done Sent Done false
6233 2677 Acked Read 8 Done Sent Done false
6234 2678 Acked Write 6 Done Sent Done false
6235 2679 Acked Read 5 Done Sent Done false
6236 2680 Acked Read 7 Done Sent Done false
6237 2681 Acked Flush 0 Done Sent Done false
6238 2682 Acked Read 1 Done Sent Done false
6239 2683 Acked Write 6 Done Sent Done false
6240 2684 Acked Read 7 Done Sent Done false
6241 2685 Acked Flush 0 Done Sent Done false
6242 2686 Acked Read 8 Done Sent Done false
6243 2687 Acked Read 9 Done Sent Done false
6244 2688 Acked Read 5 Done Sent Done false
6245 2689 Acked Write 9 Done Sent Done false
6246 2690 Acked Write 1 Done Sent Done false
6247 2691 Acked Write 4 Done Sent Done false
6248 2692 Acked Read 10 Done Sent Done false
6249 2693 Acked Read 7 Done Sent Done false
6250 2694 Acked Write 3 Done Sent Done false
6251 2695 Acked Read 10 Done Sent Done false
6252 2696 Acked Read 3 Done Sent Done false
6253 2697 Acked Read 4 Done Sent Done false
6254 2698 Acked Read 10 Done Sent Done false
6255 2699 Acked Read 10 Done Sent Done false
6256 2700 Acked Flush 0 Done Sent Done false
6257 2701 Acked Read 6 Done Sent Done false
6258 2702 Acked Read 4 Done Sent Done false
6259 2703 Acked Flush 0 Done Sent Done false
6260 2704 Acked Write 4 Done Sent Done false
6261 2705 Acked Write 5 Done Sent Done false
6262 2706 Acked Read 6 Done Sent Done false
6263 2707 Acked Read 1 Done Sent Done false
6264 2708 Acked Read 2 Done Sent Done false
6265 2709 Acked Write 8 Done Sent Done false
6266 2710 Acked Write 6 Done Sent Done false
6267 2711 Acked Read 4 Done Sent Done false
6268 2712 Acked Write 10 Done Sent Done false
6269 2713 Acked Read 10 Done Sent Done false
6270 2714 Acked Read 10 Done Sent Done false
6271 2715 Acked Read 1 Done Sent Done false
6272 2716 Acked Flush 0 Done Sent Done false
6273 2717 Acked Write 3 Done Sent Done false
6274 2718 Acked Write 1 Done Sent Done false
6275 2719 Acked Write 6 Done Sent Done false
6276 2720 Acked Write 5 Done Sent Done false
6277 2721 Acked Write 9 Done Sent Done false
6278 2722 Acked Read 8 Done Sent Done false
6279 2723 Acked Read 3 Done Sent Done false
6280 2724 Acked Write 6 Done Sent Done false
6281 2725 Acked Read 3 Done Sent Done false
6282 2726 Acked Write 5 Done Sent Done false
6283 2727 Acked Read 4 Done Sent Done false
6284 2728 Acked Read 8 Done Sent Done false
6285 2729 Acked Write 4 Done Sent Done false
6286 2730 Acked Write 6 Done Sent Done false
6287 2731 Acked Read 1 Done Sent Done false
6288 2732 Acked Read 4 Done Sent Done false
6289 2733 Acked Read 6 Done Sent Done false
6290 2734 Acked Flush 0 Done Sent Done false
6291 2735 Acked Write 5 Done Sent Done false
6292 2736 Acked Write 2 Done Sent Done false
6293 2737 Acked Read 3 Done Sent Done false
6294 2738 Acked Write 1 Done Sent Done false
6295 2739 Acked Write 2 Done Sent Done false
6296 2740 Acked Write 8 Done Sent Done false
6297 2741 Acked Read 1 Done Sent Done false
6298 2742 Acked Write 5 Done Sent Done false
6299 2743 Acked Read 10 Done Sent Done false
6300 2744 Acked Write 2 Done Sent Done false
6301 2745 Acked Flush 0 Done Sent Done false
6302 2746 Acked Read 2 Done Sent Done false
6303 2747 Acked Read 4 Done Sent Done false
6304 2748 Acked Read 3 Done Sent Done false
6305 2749 Acked Read 5 Done Sent Done false
6306 2750 Acked Flush 0 Done Sent Done false
6307 2751 Acked Write 7 Done Sent Done false
6308 2752 Acked Read 10 Done Sent Done false
6309 2753 Acked Read 1 Done Sent Done false
6310 2754 Acked Write 5 Done Sent Done false
6311 2755 Acked Write 4 Done Sent Done false
6312 2756 Acked Read 2 Done Sent Done false
6313 2757 Acked Read 9 Done Sent Done false
6314 2758 Acked Flush 0 Done Sent Done false
6315 2759 Acked Read 3 Done Sent Done false
6316 2760 Acked Read 6 Done Sent Done false
6317 2761 Acked Write 3 Done Sent Done false
6318 2762 Acked Read 9 Done Sent Done false
6319 2763 Acked Read 2 Done Sent Done false
6320 2764 Acked Write 6 Done Sent Done false
6321 2765 Acked Write 3 Done Sent Done false
6322 2766 Acked Write 6 Done Sent Done false
6323 2767 Acked Read 7 Done Sent Done false
6324 2768 Acked Read 10 Done Sent Done false
6325 2769 Acked Read 8 Done Sent Done false
6326 2770 Acked Read 1 Done Sent Done false
6327 2771 Acked Read 9 Done Sent Done false
6328 2772 Acked Write 8 Done Sent Done false
6329 2773 Acked Read 3 Done Sent Done false
6330 2774 Acked Read 6 Done Sent Done false
6331 2775 Acked Write 10 Done Sent Done false
6332 2776 Acked Read 6 Done Sent Done false
6333 2777 Acked Read 7 Done Sent Done false
6334 2778 Acked Flush 0 Done Sent Done false
6335 2779 Acked Write 5 Done Sent Done false
6336 2780 Acked Write 5 Done Sent Done false
6337 2781 Acked Write 6 Done Sent Done false
6338 2782 Acked Write 5 Done Sent Done false
6339 2783 Acked Write 7 Done Sent Done false
6340 2784 Acked Write 7 Done Sent Done false
6341 2785 Acked Read 6 Done Sent Done false
6342 2786 Acked Read 3 Done Sent Done false
6343 2787 Acked Write 8 Done Sent Done false
6344 2788 Acked Read 10 Done Sent Done false
6345 2789 Acked Write 7 Done Sent Done false
6346 2790 Acked Read 1 Done Sent Done false
6347 2791 Acked Read 4 Done Sent Done false
6348 2792 Acked Read 6 Done Sent Done false
6349 2793 Acked Read 7 Done Sent Done false
6350 2794 Acked Write 3 Done Sent Done false
6351 2795 Acked Read 7 Done Sent Done false
6352 2796 Acked Write 4 Done Sent Done false
6353 2797 Acked Read 1 Done Sent Done false
6354 2798 Acked Read 2 Done Sent Done false
6355 2799 Acked Read 3 Done Sent Done false
6356 2800 Acked Flush 0 Done Sent Done false
6357 2801 Acked Read 3 Done Sent Done false
6358 2802 Acked Write 1 Done Sent Done false
6359 2803 Acked Read 4 Done Sent Done false
6360 2804 Acked Write 9 Done Sent Done false
6361 2805 Acked Write 5 Done Sent Done false
6362 2806 Acked Read 6 Done Sent Done false
6363 2807 Acked Flush 0 Done Sent Done false
6364 2808 Acked Write 2 Done Sent Done false
6365 2809 Acked Write 6 Done Sent Done false
6366 STATES DS:0 DS:1 DS:2 TOTAL
6367 Sent 0 301 0 301
6368 Done 301 0 301 602
6369 Skipped 0 0 0 0
6370 Error 0 0 0 0
6371 Last Flush: 2807 2508 2807
6372 Downstairs last five completed: 2508 2507 2506 2505 2504
6373 Upstairs last five completed: 2809 2808 2807 2806 2805
6374 CLIENT: Up:0 ds:301 act:2
6375 [8820] got start action
6376 [8820] I am going to restart
6377 Starting downstairs at port 8820
6378 Make output file at "/tmp/dsc/downstairs-8820.txt"
6379 [8820][1] reports Starting
6380 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1208)
6381 [8820][1] reports Running
6382 ----------------------------------------------------------------
6383 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6384 States: Active Offline Active
6385 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6386 2509 Acked Read 5 Done Sent Done false
6387 2510 Acked Read 8 Done Sent Done false
6388 2511 Acked Read 4 Done Sent Done false
6389 2512 Acked Read 8 Done Sent Done false
6390 2513 Acked Write 10 Done Sent Done false
6391 2514 Acked Read 3 Done Sent Done false
6392 2515 Acked Flush 0 Done Sent Done false
6393 2516 Acked Write 10 Done Sent Done false
6394 2517 Acked Write 9 Done Sent Done false
6395 2518 Acked Read 5 Done Sent Done false
6396 2519 Acked Read 8 Done Sent Done false
6397 2520 Acked Read 7 Done Sent Done false
6398 2521 Acked Write 8 Done Sent Done false
6399 2522 Acked Read 9 Done Sent Done false
6400 2523 Acked Read 6 Done Sent Done false
6401 2524 Acked Write 3 Done Sent Done false
6402 2525 Acked Read 6 Done Sent Done false
6403 2526 Acked Flush 0 Done Sent Done false
6404 2527 Acked Read 4 Done Sent Done false
6405 2528 Acked Read 8 Done Sent Done false
6406 2529 Acked Write 9 Done Sent Done false
6407 2530 Acked Read 4 Done Sent Done false
6408 2531 Acked Flush 0 Done Sent Done false
6409 2532 Acked Read 7 Done Sent Done false
6410 2533 Acked Read 2 Done Sent Done false
6411 2534 Acked Flush 0 Done Sent Done false
6412 2535 Acked Read 5 Done Sent Done false
6413 2536 Acked Write 9 Done Sent Done false
6414 2537 Acked Read 8 Done Sent Done false
6415 2538 Acked Write 7 Done Sent Done false
6416 2539 Acked Read 7 Done Sent Done false
6417 2540 Acked Read 5 Done Sent Done false
6418 2541 Acked Write 8 Done Sent Done false
6419 2542 Acked Write 10 Done Sent Done false
6420 2543 Acked Read 4 Done Sent Done false
6421 2544 Acked Read 2 Done Sent Done false
6422 2545 Acked Read 4 Done Sent Done false
6423 2546 Acked Flush 0 Done Sent Done false
6424 2547 Acked Flush 0 Done Sent Done false
6425 2548 Acked Read 4 Done Sent Done false
6426 2549 Acked Read 8 Done Sent Done false
6427 2550 Acked Read 8 Done Sent Done false
6428 2551 Acked Write 5 Done Sent Done false
6429 2552 Acked Read 3 Done Sent Done false
6430 2553 Acked Write 5 Done Sent Done false
6431 2554 Acked Read 4 Done Sent Done false
6432 2555 Acked Read 1 Done Sent Done false
6433 2556 Acked Read 8 Done Sent Done false
6434 2557 Acked Read 2 Done Sent Done false
6435 2558 Acked Write 3 Done Sent Done false
6436 2559 Acked Read 1 Done Sent Done false
6437 2560 Acked Read 10 Done Sent Done false
6438 2561 Acked Write 5 Done Sent Done false
6439 2562 Acked Write 4 Done Sent Done false
6440 2563 Acked Read 5 Done Sent Done false
6441 2564 Acked Read 6 Done Sent Done false
6442 2565 Acked Read 4 Done Sent Done false
6443 2566 Acked Read 7 Done Sent Done false
6444 2567 Acked Write 1 Done Sent Done false
6445 2568 Acked Write 5 Done Sent Done false
6446 2569 Acked Write 1 Done Sent Done false
6447 2570 Acked Read 1 Done Sent Done false
6448 2571 Acked Write 2 Done Sent Done false
6449 2572 Acked Read 9 Done Sent Done false
6450 2573 Acked Write 3 Done Sent Done false
6451 2574 Acked Flush 0 Done Sent Done false
6452 2575 Acked Flush 0 Done Sent Done false
6453 2576 Acked Write 5 Done Sent Done false
6454 2577 Acked Read 6 Done Sent Done false
6455 2578 Acked Read 2 Done Sent Done false
6456 2579 Acked Read 1 Done Sent Done false
6457 2580 Acked Read 7 Done Sent Done false
6458 2581 Acked Read 3 Done Sent Done false
6459 2582 Acked Write 8 Done Sent Done false
6460 2583 Acked Flush 0 Done Sent Done false
6461 2584 Acked Read 8 Done Sent Done false
6462 2585 Acked Read 1 Done Sent Done false
6463 2586 Acked Read 2 Done Sent Done false
6464 2587 Acked Read 8 Done Sent Done false
6465 2588 Acked Read 3 Done Sent Done false
6466 2589 Acked Write 1 Done Sent Done false
6467 2590 Acked Write 9 Done Sent Done false
6468 2591 Acked Write 1 Done Sent Done false
6469 2592 Acked Write 5 Done Sent Done false
6470 2593 Acked Read 2 Done Sent Done false
6471 2594 Acked Write 4 Done Sent Done false
6472 2595 Acked Write 5 Done Sent Done false
6473 2596 Acked Write 9 Done Sent Done false
6474 2597 Acked Read 9 Done Sent Done false
6475 2598 Acked Read 10 Done Sent Done false
6476 2599 Acked Read 9 Done Sent Done false
6477 2600 Acked Flush 0 Done Sent Done false
6478 2601 Acked Read 5 Done Sent Done false
6479 2602 Acked Write 10 Done Sent Done false
6480 2603 Acked Read 9 Done Sent Done false
6481 2604 Acked Write 4 Done Sent Done false
6482 2605 Acked Read 4 Done Sent Done false
6483 2606 Acked Read 3 Done Sent Done false
6484 2607 Acked Read 8 Done Sent Done false
6485 2608 Acked Write 5 Done Sent Done false
6486 2609 Acked Write 4 Done Sent Done false
6487 2610 Acked Write 2 Done Sent Done false
6488 2611 Acked Read 1 Done Sent Done false
6489 2612 Acked Flush 0 Done Sent Done false
6490 2613 Acked Read 3 Done Sent Done false
6491 2614 Acked Write 10 Done Sent Done false
6492 2615 Acked Write 5 Done Sent Done false
6493 2616 Acked Write 6 Done Sent Done false
6494 2617 Acked Read 1 Done Sent Done false
6495 2618 Acked Read 1 Done Sent Done false
6496 2619 Acked Write 9 Done Sent Done false
6497 2620 Acked Write 5 Done Sent Done false
6498 2621 Acked Write 3 Done Sent Done false
6499 2622 Acked Flush 0 Done Sent Done false
6500 2623 Acked Write 5 Done Sent Done false
6501 2624 Acked Read 5 Done Sent Done false
6502 2625 Acked Read 6 Done Sent Done false
6503 2626 Acked Write 1 Done Sent Done false
6504 2627 Acked Write 5 Done Sent Done false
6505 2628 Acked Flush 0 Done Sent Done false
6506 2629 Acked Read 10 Done Sent Done false
6507 2630 Acked Write 5 Done Sent Done false
6508 2631 Acked Write 6 Done Sent Done false
6509 2632 Acked Read 2 Done Sent Done false
6510 2633 Acked Read 4 Done Sent Done false
6511 2634 Acked Write 5 Done Sent Done false
6512 2635 Acked Flush 0 Done Sent Done false
6513 2636 Acked Read 5 Done Sent Done false
6514 2637 Acked Read 8 Done Sent Done false
6515 2638 Acked Write 2 Done Sent Done false
6516 2639 Acked Read 8 Done Sent Done false
6517 2640 Acked Read 9 Done Sent Done false
6518 2641 Acked Write 9 Done Sent Done false
6519 2642 Acked Read 10 Done Sent Done false
6520 2643 Acked Write 6 Done Sent Done false
6521 2644 Acked Flush 0 Done Sent Done false
6522 2645 Acked Read 8 Done Sent Done false
6523 2646 Acked Flush 0 Done Sent Done false
6524 2647 Acked Write 10 Done Sent Done false
6525 2648 Acked Write 4 Done Sent Done false
6526 2649 Acked Read 9 Done Sent Done false
6527 2650 Acked Write 9 Done Sent Done false
6528 2651 Acked Read 10 Done Sent Done false
6529 2652 Acked Write 2 Done Sent Done false
6530 2653 Acked Read 6 Done Sent Done false
6531 2654 Acked Read 8 Done Sent Done false
6532 2655 Acked Write 9 Done Sent Done false
6533 2656 Acked Read 6 Done Sent Done false
6534 2657 Acked Read 9 Done Sent Done false
6535 2658 Acked Write 9 Done Sent Done false
6536 2659 Acked Write 10 Done Sent Done false
6537 2660 Acked Write 4 Done Sent Done false
6538 2661 Acked Write 8 Done Sent Done false
6539 2662 Acked Write 3 Done Sent Done false
6540 2663 Acked Read 4 Done Sent Done false
6541 2664 Acked Write 1 Done Sent Done false
6542 2665 Acked Write 7 Done Sent Done false
6543 2666 Acked Read 1 Done Sent Done false
6544 2667 Acked Write 1 Done Sent Done false
6545 2668 Acked Read 9 Done Sent Done false
6546 2669 Acked Read 5 Done Sent Done false
6547 2670 Acked Write 5 Done Sent Done false
6548 2671 Acked Write 5 Done Sent Done false
6549 2672 Acked Read 10 Done Sent Done false
6550 2673 Acked Read 5 Done Sent Done false
6551 2674 Acked Write 7 Done Sent Done false
6552 2675 Acked Write 7 Done Sent Done false
6553 2676 Acked Write 4 Done Sent Done false
6554 2677 Acked Read 8 Done Sent Done false
6555 2678 Acked Write 6 Done Sent Done false
6556 2679 Acked Read 5 Done Sent Done false
6557 2680 Acked Read 7 Done Sent Done false
6558 2681 Acked Flush 0 Done Sent Done false
6559 2682 Acked Read 1 Done Sent Done false
6560 2683 Acked Write 6 Done Sent Done false
6561 2684 Acked Read 7 Done Sent Done false
6562 2685 Acked Flush 0 Done Sent Done false
6563 2686 Acked Read 8 Done Sent Done false
6564 2687 Acked Read 9 Done Sent Done false
6565 2688 Acked Read 5 Done Sent Done false
6566 2689 Acked Write 9 Done Sent Done false
6567 2690 Acked Write 1 Done Sent Done false
6568 2691 Acked Write 4 Done Sent Done false
6569 2692 Acked Read 10 Done Sent Done false
6570 2693 Acked Read 7 Done Sent Done false
6571 2694 Acked Write 3 Done Sent Done false
6572 2695 Acked Read 10 Done Sent Done false
6573 2696 Acked Read 3 Done Sent Done false
6574 2697 Acked Read 4 Done Sent Done false
6575 2698 Acked Read 10 Done Sent Done false
6576 2699 Acked Read 10 Done Sent Done false
6577 2700 Acked Flush 0 Done Sent Done false
6578 2701 Acked Read 6 Done Sent Done false
6579 2702 Acked Read 4 Done Sent Done false
6580 2703 Acked Flush 0 Done Sent Done false
6581 2704 Acked Write 4 Done Sent Done false
6582 2705 Acked Write 5 Done Sent Done false
6583 2706 Acked Read 6 Done Sent Done false
6584 2707 Acked Read 1 Done Sent Done false
6585 2708 Acked Read 2 Done Sent Done false
6586 2709 Acked Write 8 Done Sent Done false
6587 2710 Acked Write 6 Done Sent Done false
6588 2711 Acked Read 4 Done Sent Done false
6589 2712 Acked Write 10 Done Sent Done false
6590 2713 Acked Read 10 Done Sent Done false
6591 2714 Acked Read 10 Done Sent Done false
6592 2715 Acked Read 1 Done Sent Done false
6593 2716 Acked Flush 0 Done Sent Done false
6594 2717 Acked Write 3 Done Sent Done false
6595 2718 Acked Write 1 Done Sent Done false
6596 2719 Acked Write 6 Done Sent Done false
6597 2720 Acked Write 5 Done Sent Done false
6598 2721 Acked Write 9 Done Sent Done false
6599 2722 Acked Read 8 Done Sent Done false
6600 2723 Acked Read 3 Done Sent Done false
6601 2724 Acked Write 6 Done Sent Done false
6602 2725 Acked Read 3 Done Sent Done false
6603 2726 Acked Write 5 Done Sent Done false
6604 2727 Acked Read 4 Done Sent Done false
6605 2728 Acked Read 8 Done Sent Done false
6606 2729 Acked Write 4 Done Sent Done false
6607 2730 Acked Write 6 Done Sent Done false
6608 2731 Acked Read 1 Done Sent Done false
6609 2732 Acked Read 4 Done Sent Done false
6610 2733 Acked Read 6 Done Sent Done false
6611 2734 Acked Flush 0 Done Sent Done false
6612 2735 Acked Write 5 Done Sent Done false
6613 2736 Acked Write 2 Done Sent Done false
6614 2737 Acked Read 3 Done Sent Done false
6615 2738 Acked Write 1 Done Sent Done false
6616 2739 Acked Write 2 Done Sent Done false
6617 2740 Acked Write 8 Done Sent Done false
6618 2741 Acked Read 1 Done Sent Done false
6619 2742 Acked Write 5 Done Sent Done false
6620 2743 Acked Read 10 Done Sent Done false
6621 2744 Acked Write 2 Done Sent Done false
6622 2745 Acked Flush 0 Done Sent Done false
6623 2746 Acked Read 2 Done Sent Done false
6624 2747 Acked Read 4 Done Sent Done false
6625 2748 Acked Read 3 Done Sent Done false
6626 2749 Acked Read 5 Done Sent Done false
6627 2750 Acked Flush 0 Done Sent Done false
6628 2751 Acked Write 7 Done Sent Done false
6629 2752 Acked Read 10 Done Sent Done false
6630 2753 Acked Read 1 Done Sent Done false
6631 2754 Acked Write 5 Done Sent Done false
6632 2755 Acked Write 4 Done Sent Done false
6633 2756 Acked Read 2 Done Sent Done false
6634 2757 Acked Read 9 Done Sent Done false
6635 2758 Acked Flush 0 Done Sent Done false
6636 2759 Acked Read 3 Done Sent Done false
6637 2760 Acked Read 6 Done Sent Done false
6638 2761 Acked Write 3 Done Sent Done false
6639 2762 Acked Read 9 Done Sent Done false
6640 2763 Acked Read 2 Done Sent Done false
6641 2764 Acked Write 6 Done Sent Done false
6642 2765 Acked Write 3 Done Sent Done false
6643 2766 Acked Write 6 Done Sent Done false
6644 2767 Acked Read 7 Done Sent Done false
6645 2768 Acked Read 10 Done Sent Done false
6646 2769 Acked Read 8 Done Sent Done false
6647 2770 Acked Read 1 Done Sent Done false
6648 2771 Acked Read 9 Done Sent Done false
6649 2772 Acked Write 8 Done Sent Done false
6650 2773 Acked Read 3 Done Sent Done false
6651 2774 Acked Read 6 Done Sent Done false
6652 2775 Acked Write 10 Done Sent Done false
6653 2776 Acked Read 6 Done Sent Done false
6654 2777 Acked Read 7 Done Sent Done false
6655 2778 Acked Flush 0 Done Sent Done false
6656 2779 Acked Write 5 Done Sent Done false
6657 2780 Acked Write 5 Done Sent Done false
6658 2781 Acked Write 6 Done Sent Done false
6659 2782 Acked Write 5 Done Sent Done false
6660 2783 Acked Write 7 Done Sent Done false
6661 2784 Acked Write 7 Done Sent Done false
6662 2785 Acked Read 6 Done Sent Done false
6663 2786 Acked Read 3 Done Sent Done false
6664 2787 Acked Write 8 Done Sent Done false
6665 2788 Acked Read 10 Done Sent Done false
6666 2789 Acked Write 7 Done Sent Done false
6667 2790 Acked Read 1 Done Sent Done false
6668 2791 Acked Read 4 Done Sent Done false
6669 2792 Acked Read 6 Done Sent Done false
6670 2793 Acked Read 7 Done Sent Done false
6671 2794 Acked Write 3 Done Sent Done false
6672 2795 Acked Read 7 Done Sent Done false
6673 2796 Acked Write 4 Done Sent Done false
6674 2797 Acked Read 1 Done Sent Done false
6675 2798 Acked Read 2 Done Sent Done false
6676 2799 Acked Read 3 Done Sent Done false
6677 2800 Acked Flush 0 Done Sent Done false
6678 2801 Acked Read 3 Done Sent Done false
6679 2802 Acked Write 1 Done Sent Done false
6680 2803 Acked Read 4 Done Sent Done false
6681 2804 Acked Write 9 Done Sent Done false
6682 2805 Acked Write 5 Done Sent Done false
6683 2806 Acked Read 6 Done Sent Done false
6684 2807 Acked Flush 0 Done Sent Done false
6685 2808 Acked Write 2 Done Sent Done false
6686 2809 Acked Write 6 Done Sent Done false
6687 2810 Acked Flush 0 Done Sent Done false
6688 STATES DS:0 DS:1 DS:2 TOTAL
6689 Sent 0 302 0 302
6690 Done 302 0 302 604
6691 Skipped 0 0 0 0
6692 Error 0 0 0 0
6693 Last Flush: 2810 2508 2810
6694 Downstairs last five completed: 2508 2507 2506 2505 2504
6695 Upstairs last five completed: 2810 2809 2808 2807 2806
6696 CLIENT: Up:0 ds:302 act:2
66972024-11-19T17:09:22.945ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
66982024-11-19T17:09:22.945ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38286) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
66992024-11-19T17:09:22.946ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67002024-11-19T17:09:22.947ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67012024-11-19T17:09:22.947ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67022024-11-19T17:09:22.947ZINFOcrucible: send last flush ID to this DS: 2508 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67032024-11-19T17:09:22.947ZINFOcrucible: Replied this last flush ID: 2508 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67042024-11-19T17:09:22.947ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67052024-11-19T17:09:22.947ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67062024-11-19T17:09:22.947ZINFOcrucible: [1] client re-new 302 jobs since flush 2508 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
67072024-11-19T17:09:22.947ZINFOcrucible: [1] Replayed 302 jobs since flush: 2508 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
6708 ----------------------------------------------------------------
6709 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6710 Downstairs last five completed: 2810 2809 2808 2807 2806
6711 Upstairs last five completed: 2810 2809 2808 2807 2806
6712 CLIENT: Up:0 ds:0 act:3
6713 Replay: All jobs finished
6714 Main task has work to do, go find it
6715 got dsc Stop(2)
6716 stop 2
6717 Nov 19 17:09:25.189 INFO request completed, latency_us: 356, response_code: 204, uri: /stop/cid/2, method:[8830] Got stop action so:false kr:false
6718 POST, req_id: 548d869d-206e-4f6e-b22d-3a87be3b8c65, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
67192024-11-19T17:09:25.190ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67202024-11-19T17:09:25.190ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67212024-11-19T17:09:25.190ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
67222024-11-19T17:09:25.190ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67232024-11-19T17:09:25.190ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
67242024-11-19T17:09:25.190ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
6725 Nov 19 17:09:25.190 INFO request completed, latency_us: 152, response_code: 200, uri: /state/cid/2, method: GET, req_id: 49bd0632-7ace-4fde-ab50-f1c2d75d75e9, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
6726 [8830] Exited with: ExitStatus(unix_wait_status(9))
6727 [8830][2] reports Exit
6728 Nov 19 17:09:29.192 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6508ad6c-2351-43c5-a70b-689fa25cc3de, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
6729 001/300 Read block 475 len 40960
6730 002/300 Write block 3273 len 4096 data: 2
6731 003/300 Read block 4246 len 32768
6732 004/300 Read block 2000 len 28672
6733 005/300 Read block 1201 len 8192
6734 006/300 Write block 1975 len 28672 data: 3 3 3 3 3 3 3
6735 007/300 Read block 1302 len 12288
6736 008/300 Read block 892 len 16384
6737 009/300 Read block 1546 len 8192
6738 010/300 Read block 725 len 28672
6739 011/300 Read block 2293 len 8192
6740 012/300 Read block 1964 len 12288
6741 013/300 Write block 3398 len 12288 data: 3 3 3
6742 014/300 Flush
6743 015/300 Read block 2119 len 4096
6744 016/300 Read block 4054 len 12288
6745 017/300 Read block 2028 len 40960
6746 018/300 Read block 2395 len 12288
6747 019/300 Write block 320 len 24576 data: 4 3 4 4 5 4
6748 020/300 Read block 311 len 20480
6749 021/300 Write block 4964 len 40960 data: 2 2 2 2 2 2 2 2 2 3
6750 022/300 Write block 1504 len 28672 data: 2 2 2 2 2 2 2
6751 023/300 Write block 219 len 4096 data: 3
6752 024/300 Read block 3346 len 12288
6753 025/300 Read block 3973 len 16384
6754 026/300 Write block 2591 len 8192 data: 2 2
6755 027/300 Write block 1041 len 32768 data: 3 3 3 3 3 4 3 3
6756 028/300 Write block 4801 len 36864 data: 3 3 3 3 3 3 2 2 2
6757 029/300 Read block 2574 len 28672
6758 030/300 Write block 2945 len 16384 data: 2 2 2 2
6759 031/300 Read block 24 len 28672
6760 032/300 Read block 3590 len 32768
6761 033/300 Read block 2378 len 40960
6762 034/300 Read block 3367 len 12288
6763 035/300 Write block 1066 len 4096 data: 4
6764 036/300 Read block 1445 len 4096
6765 037/300 Write block 2201 len 16384 data: 3 2 2 2
6766 038/300 Write block 239 len 4096 data: 2
6767 039/300 Write block 3957 len 4096 data: 3
6768 040/300 Write block 3999 len 24576 data: 3 3 4 4 4 4
6769 041/300 Read block 2608 len 28672
6770 042/300 Write block 4094 len 20480 data: 3 3 3 3 3
6771 043/300 Read block 1324 len 40960
6772 044/300 Read block 3582 len 12288
6773 045/300 Read block 1618 len 36864
6774 046/300 Read block 4290 len 36864
6775 047/300 Read block 4262 len 4096
6776 048/300 Write block 4178 len 28672 data: 2 2 2 2 2 2 2
6777 049/300 Flush
6778 050/300 Write block 1020 len 36864 data: 3 3 2 2 2 2 2 2 2
6779 051/300 Write block 1704 len 40960 data: 3 3 3 3 4 3 2 2 2 2
6780 052/300 Write block 3758 len 24576 data: 5 4 4 4 4 4
6781 053/300 Write block 613 len 32768 data: 3 3 3 3 3 4 4 4
6782 054/300 Flush
6783 055/300 Read block 4288 len 36864
6784 056/300 Flush
6785 057/300 Read block 4744 len 8192
6786 058/300 Read block 587 len 36864
6787 059/300 Read block 2517 len 16384
6788 060/300 Write block 208 len 12288 data: 2 2 2
6789 061/300 Read block 3344 len 12288
6790 062/300 Read block 3461 len 12288
6791 063/300 Read block 3588 len 4096
6792 064/300 Write block 4845 len 40960 data: 3 3 2 2 2 2 2 2 2 2
6793 065/300 Write block 1258 len 32768 data: 2 2 2 2 2 2 2 2
6794 066/300 Read block 4920 len 36864
6795 067/300 Flush
6796 068/300 Read block 1149 len 8192
6797 069/300 Read block 4716 len 8192
6798 070/300 Read block 4091 len 20480
6799 071/300 Read block 3144 len 12288
6800 072/300 Write block 3069 len 28672 data: 3 3 2 2 2 2 2
6801 073/300 Write block 292 len 36864 data: 2 2 2 2 2 2 2 2 2
6802 074/300 Read block 598 len 4096
6803 075/300 Read block 2243 len 40960
6804 076/300 Write block 3728 len 28672 data: 3 4 3 2 2 2 2
6805 077/300 Write block 2652 len 16384 data: 2 2 2 2
6806 078/300 Read block 1556 len 4096
6807 079/300 Read block 1102 len 28672
6808 080/300 Read block 3340 len 4096
6809 081/300 Read block 3605 len 8192
6810 082/300 Write block 710 len 16384 data: 2 2 2 2
6811 083/300 Read block 3012 len 4096
6812 084/300 Write block 3696 len 28672 data: 2 2 2 2 2 2 2
6813 085/300 Flush
6814 086/300 Write block 499 len 12288 data: 4 3 3
6815 087/300 Write block 1193 len 12288 data: 5 5 5
6816 088/300 Write block 1749 len 4096 data: 2
6817 089/300 Read block 212 len 36864
6818 090/300 Flush
6819 091/300 Write block 1369 len 8192 data: 3 3
6820 092/300 Read block 3550 len 8192
6821 093/300 Write block 3591 len 36864 data: 2 2 2 2 2 2 2 2 2
6822 094/300 Read block 3826 len 32768
6823 095/300 Read block 2439 len 28672
6824 096/300 Read block 3081 len 20480
6825 097/300 Write block 3556 len 32768 data: 3 3 3 3 3 3 3 3
6826 098/300 Write block 4015 len 40960 data: 3 3 3 3 3 3 3 3 3 3
6827 099/300 Write block 3151 len 32768 data: 3 3 3 2 2 2 2 2
6828 100/300 Flush
6829 101/300 Read block 4186 len 8192
6830 102/300 Read block 2231 len 8192
6831 103/300 Flush
6832 104/300 Read block 1096 len 24576
6833 105/300 Read block 4136 len 4096
6834 106/300 Read block 2837 len 8192
6835 107/300 Write block 312 len 40960 data: 2 2 2 2 2 3 3 3 5 4
6836 108/300 Write block 3593 len 36864 data: 3 3 3 3 3 3 3 2 3
6837 109/300 Read block 2131 len 16384
6838 110/300 Read block 4319 len 12288
6839 111/300 Read block 3142 len 28672
6840 112/300 Read block 116 len 32768
6841 113/300 Write block 2204 len 20480 data: 3 2 3 3 4
6842 114/300 Read block 554 len 4096
6843 115/300 Read block 3425 len 8192
6844 116/300 Write block 3621 len 32768 data: 3 3 3 2 2 2 2 2
6845 117/300 Write block 3794 len 24576 data: 2 2 2 2 2 4
6846 118/300 Read block 1573 len 16384
6847 119/300 Read block 2677 len 40960
6848 120/300 Write block 2993 len 32768 data: 3 3 3 3 3 2 2 2
6849 121/300 Write block 1279 len 20480 data: 3 3 3 3 3
6850 122/300 Write block 824 len 8192 data: 3 3
6851 123/300 Flush
6852 124/300 Read block 4205 len 32768
6853 125/300 Write block 2250 len 36864 data: 3 3 3 3 4 3 3 3 2
6854 126/300 Write block 743 len 4096 data: 3
6855 127/300 Read block 60 len 20480
6856 128/300 Read block 3166 len 4096
6857 129/300 Read block 2849 len 12288
6858 130/300 Write block 2339 len 12288 data: 2 2 2
6859 131/300 Read block 1737 len 36864
6860 132/300 Read block 3743 len 24576
6861 133/300 Read block 840 len 40960
6862 134/300 Read block 3592 len 16384
6863 135/300 Read block 469 len 32768
6864 136/300 Write block 1866 len 32768 data: 3 3 3 3 3 2 2 2
6865 137/300 Read block 1873 len 40960
6866 138/300 Read block 995 len 8192
6867 139/300 Write block 1009 len 8192 data: 3 3
6868 140/300 Flush
6869 141/300 Write block 2945 len 16384 data: 3 3 3 3
6870 142/300 Read block 3758 len 24576
6871 143/300 Read block 3997 len 12288
6872 144/300 Write block 2800 len 32768 data: 2 2 3 3 2 3 3 4
6873 145/300 Read block 1428 len 8192
6874 146/300 Flush
6875 147/300 Read block 4389 len 20480
6876 148/300 Write block 1826 len 20480 data: 4 3 3 2 2
6877 149/300 Write block 2344 len 32768 data: 2 2 2 2 2 2 3 3
6878 150/300 Flush
6879 151/300 Flush
6880 152/300 Write block 3929 len 20480 data: 4 4 4 4 3
6881 153/300 Read block 4701 len 16384
6882 154/300 Read block 1081 len 16384
6883 155/300 Read block 591 len 28672
6884 156/300 Read block 4766 len 24576
6885 157/300 Read block 3678 len 40960
6886 158/300 Write block 1965 len 40960 data: 2 2 2 2 2 2 2 2 2 3
6887 159/300 Read block 3915 len 32768
6888 160/300 Write block 2230 len 32768 data: 2 2 2 2 2 2 2 2
6889 161/300 Read block 3009 len 32768
6890 162/300 Write block 2411 len 24576 data: 2 3 3 3 3 3
6891 163/300 Write block 3723 len 32768 data: 3 3 3 3 3 4 5 4
6892 164/300 Read block 2148 len 40960
6893 165/300 Write block 1778 len 32768 data: 3 3 3 2 2 3 3 3
6894 166/300 Write block 1543 len 36864 data: 3 4 5 5 5 4 4 4 4
6895 167/300 Read block 3947 len 40960
6896 168/300 Read block 4507 len 32768
6897 169/300 Read block 2856 len 36864
6898 170/300 Flush
6899 171/300 Write block 519 len 24576 data: 2 2 2 3 3 3
6900 172/300 Read block 1616 len 8192
6901 173/300 Write block 4273 len 4096 data: 2
6902 174/300 Write block 4499 len 32768 data: 2 2 2 2 2 2 2 2
6903 175/300 Write block 901 len 24576 data: 2 2 2 2 3 3
6904 176/300 Flush
6905 177/300 Read block 2808 len 28672
6906 178/300 Read block 3710 len 32768
6907 179/300 Read block 1087 len 20480
6908 180/300 Write block 3904 len 4096 data: 2
6909 181/300 Read block 4152 len 4096
6910 182/300 Read block 965 len 36864
6911 183/300 Write block 1752 len 12288 data: 3 3 4
6912 184/300 Read block 2142 len 8192
6913 185/300 Read block 4547 len 24576
6914 186/300 Write block 659 len 16384 data: 2 2 2 2
6915 187/300 Write block 3272 len 28672 data: 3 3 2 3 3 3 3
6916 188/300 Write block 4023 len 12288 data: 4 4 3
6917 189/300 Write block 318 len 28672 data: 4 4 6 5 5 5 6
6918 190/300 Write block 1609 len 36864 data: 5 5 5 4 4 4 4 4 4
6919 191/300 Read block 3959 len 8192
6920 192/300 Write block 3746 len 16384 data: 4 4 4 4
6921 193/300 Read block 3716 len 4096
6922 194/300 Read block 3906 len 20480
6923 195/300 Write block 1493 len 20480 data: 2 2 2 3 3
6924 196/300 Write block 3568 len 16384 data: 3 3 3 3
6925 197/300 Read block 4981 len 20480
6926 198/300 Read block 4704 len 4096
6927 199/300 Write block 1778 len 16384 data: 4 4 4 3
6928 200/300 Write block 277 len 12288 data: 3 3 3
6929 201/300 Read block 4634 len 40960
6930 202/300 Write block 2795 len 8192 data: 2 2
6931 203/300 Read block 4021 len 40960
6932 204/300 Write block 3112 len 12288 data: 2 2 2
6933 205/300 Read block 473 len 28672
6934 206/300 Write block 4255 len 20480 data: 3 3 3 3 2
6935 207/300 Write block 401 len 12288 data: 4 3 4
6936 208/300 Read block 2039 len 32768
6937 209/300 Write block 1372 len 16384 data: 3 4 3 3
6938 210/300 Write block 2301 len 4096 data: 6
6939 211/300 Write block 4486 len 8192 data: 3 4
6940 212/300 Flush
6941 213/300 Read block 2949 len 28672
6942 214/300 Read block 2955 len 4096
6943 215/300 Read block 3534 len 4096
6944 216/300 Read block 973 len 16384
6945 217/300 Write block 1109 len 32768 data: 2 2 2 2 2 2 2 2
6946 218/300 Write block 4912 len 8192 data: 5 5
6947 219/300 Flush
6948 220/300 Read block 4765 len 28672
6949 221/300 Write block 4098 len 20480 data: 4 2 2 2 2
6950 222/300 Read block 3373 len 16384
6951 223/300 Read block 4879 len 36864
6952 224/300 Read block 763 len 40960
6953 225/300 Read block 3581 len 20480
6954 226/300 Flush
6955 227/300 Write block 2947 len 28672 data: 4 4 2 2 2 2 2
6956 228/300 Write block 4689 len 36864 data: 3 3 3 3 3 4 4 3 3
6957 229/300 Read block 4851 len 16384
6958 230/300 Read block 4062 len 12288
6959 231/300 Read block 1417 len 8192
6960 232/300 Write block 2548 len 4096 data: 2
6961 233/300 Write block 1515 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6962 234/300 Read block 1811 len 24576
6963 235/300 Read block 992 len 16384
6964 236/300 Read block 1349 len 36864
6965 237/300 Read block 4059 len 20480
6966 238/300 Read block 3231 len 4096
6967 239/300 Read block 631 len 40960
6968 240/300 Write block 3584 len 28672 data: 3 3 3 2 2 2 2
6969 241/300 Read block 3727 len 12288
6970 242/300 Read block 2400 len 4096
6971 243/300 Write block 3203 len 32768 data: 3 2 2 2 2 2 2 2
6972 244/300 Read block 4233 len 16384
6973 245/300 Write block 85 len 32768 data: 2 2 2 3 3 3 3 3
6974 246/300 Read block 485 len 8192
6975 247/300 Write block 901 len 28672 data: 3 3 3 3 4 4 3
6976 248/300 Read block 3956 len 8192
6977 249/300 Read block 2558 len 28672
6978 250/300 Write block 1882 len 16384 data: 2 4 5 5
6979 251/300 Write block 4407 len 28672 data: 2 2 2 2 2 2 2
6980 252/300 Read block 687 len 20480
6981 253/300 Read block 358 len 28672
6982 254/300 Read block 612 len 32768
6983 255/300 Write block 226 len 16384 data: 2 2 2 2
6984 256/300 Write block 3718 len 12288 data: 2 2 2
6985 257/300 Flush
6986 258/300 Read block 4698 len 32768
6987 259/300 Write block 2173 len 40960 data: 4 3 3 2 2 2 2 2 2 2
6988 260/300 Write block 3697 len 24576 data: 3 3 3 3 3 3
6989 261/300 Read block 2373 len 4096
6990 262/300 Read block 3965 len 12288
6991 263/300 Read block 55 len 8192
6992 264/300 Write block 3667 len 12288 data: 2 2 2
6993 265/300 Write block 4391 len 24576 data: 2 3 3 3 3 3
6994 266/300 Read block 171 len 40960
6995 267/300 Write block 1692 len 12288 data: 4 5 5
6996 268/300 Read block 2225 len 20480
6997 269/300 Read block 3612 len 28672
6998 270/300 Write block 1397 len 28672 data: 3 3 4 5 4 4 4
6999 271/300 Read block 5 len 24576
7000 272/300 Write block 1464 len 16384 data: 2 2 2 3
7001 273/300 Read block 2687 len 32768
7002 274/300 Flush
7003 275/300 Write block 1023 len 36864 data: 3 3 3 3 3 3 3 3 3
7004 276/300 Write block 2522 len 12288 data: 2 2 3
7005 277/300 Write block 4015 len 32768 data: 4 4 4 4 4 4 4 4
7006 278/300 Read block 3449 len 32768
7007 279/300 Write block 3345 len 4096 data: 2
7008 280/300 Read block 4502 len 12288
7009 281/300 Write block 2938 len 12288 data: 2 3 2
7010 282/300 Write block 1907 len 8192 data: 2 2
7011 283/300 Read block 1243 len 24576
7012 284/300 Write block 4795 len 28672 data: 3 3 3 3 4 4 4
7013 285/300 Write block 2486 len 12288 data: 2 2 2
7014 286/300 Read block 275 len 4096
7015 287/300 Read block 3677 len 8192
7016 288/300 Read block 44 len 4096
7017 289/300 Read block 1175 len 16384
7018 290/300 Write block 849 len 16384 data: 2 2 2 2
7019 291/300 Write block 4759 len 24576 data: 3 3 3 3 4 4
7020 292/300 Write block 2324 len 36864 data: 2 2 2 2 3 3 3 3 2
7021 293/300 Read block 4837 len 12288
7022 294/300 Flush
7023 295/300 Read block 3062 len 24576
7024 296/300 Read block 3480 len 8192
7025 297/300 Read block 967 len 32768
7026 298/300 Write block 2099 len 20480 data: 4 3 3 4 4
7027 299/300 Write block 1354 len 40960 data: 4 4 3 3 3 2 2 2 2 2
7028 300/300 Write block 2816 len 20480 data: 2 2 2 2 2
7029 301/300 Read block 596 len 36864
7030 Main task has work to do, go find it
7031 got dsc Start(2)
7032 start 2
7033 Nov 19 17:09:29.422 INFO request completed, latency_us: 184, response_code: 204, uri: /start/cid/2, method: POST, req_id: f6008f2e-464d-4cf1-9d84-d43e88a9028a, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
7034 Replay: started 2, returned:Ok(())
7035 ----------------------------------------------------------------
7036 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7037 States: Active Active Offline
7038 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7039 2811 Acked Read 10 Done Done Sent false
7040 2812 Acked Write 1 Done Done Sent false
7041 2813 Acked Read 8 Done Done Sent false
7042 2814 Acked Read 7 Done Done Sent false
7043 2815 Acked Read 2 Done Done Sent false
7044 2816 Acked Write 7 Done Done Sent false
7045 2817 Acked Read 3 Done Done Sent false
7046 2818 Acked Read 4 Done Done Sent false
7047 2819 Acked Read 2 Done Done Sent false
7048 2820 Acked Read 7 Done Done Sent false
7049 2821 Acked Read 2 Done Done Sent false
7050 2822 Acked Read 3 Done Done Sent false
7051 2823 Acked Write 3 Done Done Sent false
7052 2824 Acked Flush 0 Done Done Sent false
7053 2825 Acked Read 1 Done Done Sent false
7054 2826 Acked Read 3 Done Done Sent false
7055 2827 Acked Read 10 Done Done Sent false
7056 2828 Acked Read 3 Done Done Sent false
7057 2829 Acked Write 6 Done Done Sent false
7058 2830 Acked Read 5 Done Done Sent false
7059 2831 Acked Write 10 Done Done Sent false
7060 2832 Acked Write 7 Done Done Sent false
7061 2833 Acked Write 1 Done Done Sent false
7062 2834 Acked Read 3 Done Done Sent false
7063 2835 Acked Read 4 Done Done Sent false
7064 2836 Acked Write 2 Done Done Sent false
7065 2837 Acked Write 8 Done Done Sent false
7066 2838 Acked Write 9 Done Done Sent false
7067 2839 Acked Read 7 Done Done Sent false
7068 2840 Acked Write 4 Done Done Sent false
7069 2841 Acked Read 7 Done Done Sent false
7070 2842 Acked Read 8 Done Done Sent false
7071 2843 Acked Read 10 Done Done Sent false
7072 2844 Acked Read 3 Done Done Sent false
7073 2845 Acked Write 1 Done Done Sent false
7074 2846 Acked Read 1 Done Done Sent false
7075 2847 Acked Write 4 Done Done Sent false
7076 2848 Acked Write 1 Done Done Sent false
7077 2849 Acked Write 1 Done Done Sent false
7078 2850 Acked Write 6 Done Done Sent false
7079 2851 Acked Read 7 Done Done Sent false
7080 2852 Acked Write 5 Done Done Sent false
7081 2853 Acked Read 10 Done Done Sent false
7082 2854 Acked Read 3 Done Done Sent false
7083 2855 Acked Read 9 Done Done Sent false
7084 2856 Acked Read 9 Done Done Sent false
7085 2857 Acked Read 1 Done Done Sent false
7086 2858 Acked Write 7 Done Done Sent false
7087 2859 Acked Flush 0 Done Done Sent false
7088 2860 Acked Write 9 Done Done Sent false
7089 2861 Acked Write 10 Done Done Sent false
7090 2862 Acked Write 6 Done Done Sent false
7091 2863 Acked Write 8 Done Done Sent false
7092 2864 Acked Flush 0 Done Done Sent false
7093 2865 Acked Read 9 Done Done Sent false
7094 2866 Acked Flush 0 Done Done Sent false
7095 2867 Acked Read 2 Done Done Sent false
7096 2868 Acked Read 9 Done Done Sent false
7097 2869 Acked Read 4 Done Done Sent false
7098 2870 Acked Write 3 Done Done Sent false
7099 2871 Acked Read 3 Done Done Sent false
7100 2872 Acked Read 3 Done Done Sent false
7101 2873 Acked Read 1 Done Done Sent false
7102 2874 Acked Write 10 Done Done Sent false
7103 2875 Acked Write 8 Done Done Sent false
7104 2876 Acked Read 9 Done Done Sent false
7105 2877 Acked Flush 0 Done Done Sent false
7106 2878 Acked Read 2 Done Done Sent false
7107 2879 Acked Read 2 Done Done Sent false
7108 2880 Acked Read 5 Done Done Sent false
7109 2881 Acked Read 3 Done Done Sent false
7110 2882 Acked Write 7 Done Done Sent false
7111 2883 Acked Write 9 Done Done Sent false
7112 2884 Acked Read 1 Done Done Sent false
7113 2885 Acked Read 10 Done Done Sent false
7114 2886 Acked Write 7 Done Done Sent false
7115 2887 Acked Write 4 Done Done Sent false
7116 2888 Acked Read 1 Done Done Sent false
7117 2889 Acked Read 7 Done Done Sent false
7118 2890 Acked Read 1 Done Done Sent false
7119 2891 Acked Read 2 Done Done Sent false
7120 2892 Acked Write 4 Done Done Sent false
7121 2893 Acked Read 1 Done Done Sent false
7122 2894 Acked Write 7 Done Done Sent false
7123 2895 Acked Flush 0 Done Done Sent false
7124 2896 Acked Write 3 Done Done Sent false
7125 2897 Acked Write 3 Done Done Sent false
7126 2898 Acked Write 1 Done Done Sent false
7127 2899 Acked Read 9 Done Done Sent false
7128 2900 Acked Flush 0 Done Done Sent false
7129 2901 Acked Write 2 Done Done Sent false
7130 2902 Acked Read 2 Done Done Sent false
7131 2903 Acked Write 9 Done Done Sent false
7132 2904 Acked Read 8 Done Done Sent false
7133 2905 Acked Read 7 Done Done Sent false
7134 2906 Acked Read 5 Done Done Sent false
7135 2907 Acked Write 8 Done Done Sent false
7136 2908 Acked Write 10 Done Done Sent false
7137 2909 Acked Write 8 Done Done Sent false
7138 2910 Acked Flush 0 Done Done Sent false
7139 2911 Acked Read 2 Done Done Sent false
7140 2912 Acked Read 2 Done Done Sent false
7141 2913 Acked Flush 0 Done Done Sent false
7142 2914 Acked Read 6 Done Done Sent false
7143 2915 Acked Read 1 Done Done Sent false
7144 2916 Acked Read 2 Done Done Sent false
7145 2917 Acked Write 10 Done Done Sent false
7146 2918 Acked Write 9 Done Done Sent false
7147 2919 Acked Read 4 Done Done Sent false
7148 2920 Acked Read 3 Done Done Sent false
7149 2921 Acked Read 7 Done Done Sent false
7150 2922 Acked Read 8 Done Done Sent false
7151 2923 Acked Write 5 Done Done Sent false
7152 2924 Acked Read 1 Done Done Sent false
7153 2925 Acked Read 2 Done Done Sent false
7154 2926 Acked Write 8 Done Done Sent false
7155 2927 Acked Write 6 Done Done Sent false
7156 2928 Acked Read 4 Done Done Sent false
7157 2929 Acked Read 10 Done Done Sent false
7158 2930 Acked Write 8 Done Done Sent false
7159 2931 Acked Write 5 Done Done Sent false
7160 2932 Acked Write 2 Done Done Sent false
7161 2933 Acked Flush 0 Done Done Sent false
7162 2934 Acked Read 8 Done Done Sent false
7163 2935 Acked Write 9 Done Done Sent false
7164 2936 Acked Write 1 Done Done Sent false
7165 2937 Acked Read 5 Done Done Sent false
7166 2938 Acked Read 1 Done Done Sent false
7167 2939 Acked Read 3 Done Done Sent false
7168 2940 Acked Write 3 Done Done Sent false
7169 2941 Acked Read 9 Done Done Sent false
7170 2942 Acked Read 6 Done Done Sent false
7171 2943 Acked Read 10 Done Done Sent false
7172 2944 Acked Read 4 Done Done Sent false
7173 2945 Acked Read 8 Done Done Sent false
7174 2946 Acked Write 8 Done Done Sent false
7175 2947 Acked Read 10 Done Done Sent false
7176 2948 Acked Read 2 Done Done Sent false
7177 2949 Acked Write 2 Done Done Sent false
7178 2950 Acked Flush 0 Done Done Sent false
7179 2951 Acked Write 4 Done Done Sent false
7180 2952 Acked Read 6 Done Done Sent false
7181 2953 Acked Read 3 Done Done Sent false
7182 2954 Acked Write 8 Done Done Sent false
7183 2955 Acked Read 2 Done Done Sent false
7184 2956 Acked Flush 0 Done Done Sent false
7185 2957 Acked Read 5 Done Done Sent false
7186 2958 Acked Write 5 Done Done Sent false
7187 2959 Acked Write 8 Done Done Sent false
7188 2960 Acked Flush 0 Done Done Sent false
7189 2961 Acked Flush 0 Done Done Sent false
7190 2962 Acked Write 5 Done Done Sent false
7191 2963 Acked Read 4 Done Done Sent false
7192 2964 Acked Read 4 Done Done Sent false
7193 2965 Acked Read 7 Done Done Sent false
7194 2966 Acked Read 6 Done Done Sent false
7195 2967 Acked Read 10 Done Done Sent false
7196 2968 Acked Write 10 Done Done Sent false
7197 2969 Acked Read 8 Done Done Sent false
7198 2970 Acked Write 8 Done Done Sent false
7199 2971 Acked Read 8 Done Done Sent false
7200 2972 Acked Write 6 Done Done Sent false
7201 2973 Acked Write 8 Done Done Sent false
7202 2974 Acked Read 10 Done Done Sent false
7203 2975 Acked Write 8 Done Done Sent false
7204 2976 Acked Write 9 Done Done Sent false
7205 2977 Acked Read 10 Done Done Sent false
7206 2978 Acked Read 8 Done Done Sent false
7207 2979 Acked Read 9 Done Done Sent false
7208 2980 Acked Flush 0 Done Done Sent false
7209 2981 Acked Write 6 Done Done Sent false
7210 2982 Acked Read 2 Done Done Sent false
7211 2983 Acked Write 1 Done Done Sent false
7212 2984 Acked Write 8 Done Done Sent false
7213 2985 Acked Write 6 Done Done Sent false
7214 2986 Acked Flush 0 Done Done Sent false
7215 2987 Acked Read 7 Done Done Sent false
7216 2988 Acked Read 8 Done Done Sent false
7217 2989 Acked Read 5 Done Done Sent false
7218 2990 Acked Write 1 Done Done Sent false
7219 2991 Acked Read 1 Done Done Sent false
7220 2992 Acked Read 9 Done Done Sent false
7221 2993 Acked Write 3 Done Done Sent false
7222 2994 Acked Read 2 Done Done Sent false
7223 2995 Acked Read 6 Done Done Sent false
7224 2996 Acked Write 4 Done Done Sent false
7225 2997 Acked Write 7 Done Done Sent false
7226 2998 Acked Write 3 Done Done Sent false
7227 2999 Acked Write 7 Done Done Sent false
7228 3000 Acked Write 9 Done Done Sent false
7229 3001 Acked Read 2 Done Done Sent false
7230 3002 Acked Write 4 Done Done Sent false
7231 3003 Acked Read 1 Done Done Sent false
7232 3004 Acked Read 5 Done Done Sent false
7233 3005 Acked Write 5 Done Done Sent false
7234 3006 Acked Write 4 Done Done Sent false
7235 3007 Acked Read 5 Done Done Sent false
7236 3008 Acked Read 1 Done Done Sent false
7237 3009 Acked Write 4 Done Done Sent false
7238 3010 Acked Write 3 Done Done Sent false
7239 3011 Acked Read 10 Done Done Sent false
7240 3012 Acked Write 2 Done Done Sent false
7241 3013 Acked Read 10 Done Done Sent false
7242 3014 Acked Write 3 Done Done Sent false
7243 3015 Acked Read 7 Done Done Sent false
7244 3016 Acked Write 5 Done Done Sent false
7245 3017 Acked Write 3 Done Done Sent false
7246 3018 Acked Read 8 Done Done Sent false
7247 3019 Acked Write 4 Done Done Sent false
7248 3020 Acked Write 1 Done Done Sent false
7249 3021 Acked Write 2 Done Done Sent false
7250 3022 Acked Flush 0 Done Done Sent false
7251 3023 Acked Read 7 Done Done Sent false
7252 3024 Acked Read 1 Done Done Sent false
7253 3025 Acked Read 1 Done Done Sent false
7254 3026 Acked Read 4 Done Done Sent false
7255 3027 Acked Write 8 Done Done Sent false
7256 3028 Acked Write 2 Done Done Sent false
7257 3029 Acked Flush 0 Done Done Sent false
7258 3030 Acked Read 7 Done Done Sent false
7259 3031 Acked Write 5 Done Done Sent false
7260 3032 Acked Read 4 Done Done Sent false
7261 3033 Acked Read 9 Done Done Sent false
7262 3034 Acked Read 10 Done Done Sent false
7263 3035 Acked Read 5 Done Done Sent false
7264 3036 Acked Flush 0 Done Done Sent false
7265 3037 Acked Write 7 Done Done Sent false
7266 3038 Acked Write 9 Done Done Sent false
7267 3039 Acked Read 4 Done Done Sent false
7268 3040 Acked Read 3 Done Done Sent false
7269 3041 Acked Read 2 Done Done Sent false
7270 3042 Acked Write 1 Done Done Sent false
7271 3043 Acked Write 10 Done Done Sent false
7272 3044 Acked Read 6 Done Done Sent false
7273 3045 Acked Read 4 Done Done Sent false
7274 3046 Acked Read 9 Done Done Sent false
7275 3047 Acked Read 5 Done Done Sent false
7276 3048 Acked Read 1 Done Done Sent false
7277 3049 Acked Read 10 Done Done Sent false
7278 3050 Acked Write 7 Done Done Sent false
7279 3051 Acked Read 3 Done Done Sent false
7280 3052 Acked Read 1 Done Done Sent false
7281 3053 Acked Write 8 Done Done Sent false
7282 3054 Acked Read 4 Done Done Sent false
7283 3055 Acked Write 8 Done Done Sent false
7284 3056 Acked Read 2 Done Done Sent false
7285 3057 Acked Write 7 Done Done Sent false
7286 3058 Acked Read 2 Done Done Sent false
7287 3059 Acked Read 7 Done Done Sent false
7288 3060 Acked Write 4 Done Done Sent false
7289 3061 Acked Write 7 Done Done Sent false
7290 3062 Acked Read 5 Done Done Sent false
7291 3063 Acked Read 7 Done Done Sent false
7292 3064 Acked Read 8 Done Done Sent false
7293 3065 Acked Write 4 Done Done Sent false
7294 3066 Acked Write 3 Done Done Sent false
7295 3067 Acked Flush 0 Done Done Sent false
7296 3068 Acked Read 8 Done Done Sent false
7297 3069 Acked Write 10 Done Done Sent false
7298 3070 Acked Write 6 Done Done Sent false
7299 3071 Acked Read 1 Done Done Sent false
7300 3072 Acked Read 3 Done Done Sent false
7301 3073 Acked Read 2 Done Done Sent false
7302 3074 Acked Write 3 Done Done Sent false
7303 3075 Acked Write 6 Done Done Sent false
7304 3076 Acked Read 10 Done Done Sent false
7305 3077 Acked Write 3 Done Done Sent false
7306 3078 Acked Read 5 Done Done Sent false
7307 3079 Acked Read 7 Done Done Sent false
7308 3080 Acked Write 7 Done Done Sent false
7309 3081 Acked Read 6 Done Done Sent false
7310 3082 Acked Write 4 Done Done Sent false
7311 3083 Acked Read 8 Done Done Sent false
7312 3084 Acked Flush 0 Done Done Sent false
7313 3085 Acked Write 9 Done Done Sent false
7314 3086 Acked Write 3 Done Done Sent false
7315 3087 Acked Write 8 Done Done Sent false
7316 3088 Acked Read 8 Done Done Sent false
7317 3089 Acked Write 1 Done Done Sent false
7318 3090 Acked Read 3 Done Done Sent false
7319 3091 Acked Write 3 Done Done Sent false
7320 3092 Acked Write 2 Done Done Sent false
7321 3093 Acked Read 6 Done Done Sent false
7322 3094 Acked Write 7 Done Done Sent false
7323 3095 Acked Write 3 Done Done Sent false
7324 3096 Acked Read 1 Done Done Sent false
7325 3097 Acked Read 2 Done Done Sent false
7326 3098 Acked Read 1 Done Done Sent false
7327 3099 Acked Read 4 Done Done Sent false
7328 3100 Acked Write 4 Done Done Sent false
7329 3101 Acked Write 6 Done Done Sent false
7330 3102 Acked Write 9 Done Done Sent false
7331 3103 Acked Read 3 Done Done Sent false
7332 3104 Acked Flush 0 Done Done Sent false
7333 3105 Acked Read 6 Done Done Sent false
7334 3106 Acked Read 2 Done Done Sent false
7335 3107 Acked Read 8 Done Done Sent false
7336 3108 Acked Write 5 Done Done Sent false
7337 3109 Acked Write 10 Done Done Sent false
7338 3110 Acked Write 5 Done Done Sent false
7339 3111 Acked Read 9 Done Done Sent false
7340 STATES DS:0 DS:1 DS:2 TOTAL
7341 Sent 0 0 301 301
7342 Done 301 301 0 602
7343 Skipped 0 0 0 0
7344 Error 0 0 0 0
7345 Last Flush: 3104 3104 2810
7346 Downstairs last five completed: 2810 2809 2808 2807 2806
7347 Upstairs last five completed: 3111 3110 3109 3108 3107
7348 CLIENT: Up:0 ds:301 act:2
7349 [8830] got start action
7350 [8830] I am going to restart
7351 Starting downstairs at port 8830
7352 Make output file at "/tmp/dsc/downstairs-8830.txt"
7353 [8830][2] reports Starting
7354 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1209)
7355 [8830][2] reports Running
7356 ----------------------------------------------------------------
7357 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7358 States: Active Active Offline
7359 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7360 2811 Acked Read 10 Done Done Sent false
7361 2812 Acked Write 1 Done Done Sent false
7362 2813 Acked Read 8 Done Done Sent false
7363 2814 Acked Read 7 Done Done Sent false
7364 2815 Acked Read 2 Done Done Sent false
7365 2816 Acked Write 7 Done Done Sent false
7366 2817 Acked Read 3 Done Done Sent false
7367 2818 Acked Read 4 Done Done Sent false
7368 2819 Acked Read 2 Done Done Sent false
7369 2820 Acked Read 7 Done Done Sent false
7370 2821 Acked Read 2 Done Done Sent false
7371 2822 Acked Read 3 Done Done Sent false
7372 2823 Acked Write 3 Done Done Sent false
7373 2824 Acked Flush 0 Done Done Sent false
7374 2825 Acked Read 1 Done Done Sent false
7375 2826 Acked Read 3 Done Done Sent false
7376 2827 Acked Read 10 Done Done Sent false
7377 2828 Acked Read 3 Done Done Sent false
7378 2829 Acked Write 6 Done Done Sent false
7379 2830 Acked Read 5 Done Done Sent false
7380 2831 Acked Write 10 Done Done Sent false
7381 2832 Acked Write 7 Done Done Sent false
7382 2833 Acked Write 1 Done Done Sent false
7383 2834 Acked Read 3 Done Done Sent false
7384 2835 Acked Read 4 Done Done Sent false
7385 2836 Acked Write 2 Done Done Sent false
7386 2837 Acked Write 8 Done Done Sent false
7387 2838 Acked Write 9 Done Done Sent false
7388 2839 Acked Read 7 Done Done Sent false
7389 2840 Acked Write 4 Done Done Sent false
7390 2841 Acked Read 7 Done Done Sent false
7391 2842 Acked Read 8 Done Done Sent false
7392 2843 Acked Read 10 Done Done Sent false
7393 2844 Acked Read 3 Done Done Sent false
7394 2845 Acked Write 1 Done Done Sent false
7395 2846 Acked Read 1 Done Done Sent false
7396 2847 Acked Write 4 Done Done Sent false
7397 2848 Acked Write 1 Done Done Sent false
7398 2849 Acked Write 1 Done Done Sent false
7399 2850 Acked Write 6 Done Done Sent false
7400 2851 Acked Read 7 Done Done Sent false
7401 2852 Acked Write 5 Done Done Sent false
7402 2853 Acked Read 10 Done Done Sent false
7403 2854 Acked Read 3 Done Done Sent false
7404 2855 Acked Read 9 Done Done Sent false
7405 2856 Acked Read 9 Done Done Sent false
7406 2857 Acked Read 1 Done Done Sent false
7407 2858 Acked Write 7 Done Done Sent false
7408 2859 Acked Flush 0 Done Done Sent false
7409 2860 Acked Write 9 Done Done Sent false
7410 2861 Acked Write 10 Done Done Sent false
7411 2862 Acked Write 6 Done Done Sent false
7412 2863 Acked Write 8 Done Done Sent false
7413 2864 Acked Flush 0 Done Done Sent false
7414 2865 Acked Read 9 Done Done Sent false
7415 2866 Acked Flush 0 Done Done Sent false
7416 2867 Acked Read 2 Done Done Sent false
7417 2868 Acked Read 9 Done Done Sent false
7418 2869 Acked Read 4 Done Done Sent false
7419 2870 Acked Write 3 Done Done Sent false
7420 2871 Acked Read 3 Done Done Sent false
7421 2872 Acked Read 3 Done Done Sent false
7422 2873 Acked Read 1 Done Done Sent false
7423 2874 Acked Write 10 Done Done Sent false
7424 2875 Acked Write 8 Done Done Sent false
7425 2876 Acked Read 9 Done Done Sent false
7426 2877 Acked Flush 0 Done Done Sent false
7427 2878 Acked Read 2 Done Done Sent false
7428 2879 Acked Read 2 Done Done Sent false
7429 2880 Acked Read 5 Done Done Sent false
7430 2881 Acked Read 3 Done Done Sent false
7431 2882 Acked Write 7 Done Done Sent false
7432 2883 Acked Write 9 Done Done Sent false
7433 2884 Acked Read 1 Done Done Sent false
7434 2885 Acked Read 10 Done Done Sent false
7435 2886 Acked Write 7 Done Done Sent false
7436 2887 Acked Write 4 Done Done Sent false
7437 2888 Acked Read 1 Done Done Sent false
7438 2889 Acked Read 7 Done Done Sent false
7439 2890 Acked Read 1 Done Done Sent false
7440 2891 Acked Read 2 Done Done Sent false
7441 2892 Acked Write 4 Done Done Sent false
7442 2893 Acked Read 1 Done Done Sent false
7443 2894 Acked Write 7 Done Done Sent false
7444 2895 Acked Flush 0 Done Done Sent false
7445 2896 Acked Write 3 Done Done Sent false
7446 2897 Acked Write 3 Done Done Sent false
7447 2898 Acked Write 1 Done Done Sent false
7448 2899 Acked Read 9 Done Done Sent false
7449 2900 Acked Flush 0 Done Done Sent false
7450 2901 Acked Write 2 Done Done Sent false
7451 2902 Acked Read 2 Done Done Sent false
7452 2903 Acked Write 9 Done Done Sent false
7453 2904 Acked Read 8 Done Done Sent false
7454 2905 Acked Read 7 Done Done Sent false
7455 2906 Acked Read 5 Done Done Sent false
7456 2907 Acked Write 8 Done Done Sent false
7457 2908 Acked Write 10 Done Done Sent false
7458 2909 Acked Write 8 Done Done Sent false
7459 2910 Acked Flush 0 Done Done Sent false
7460 2911 Acked Read 2 Done Done Sent false
7461 2912 Acked Read 2 Done Done Sent false
7462 2913 Acked Flush 0 Done Done Sent false
7463 2914 Acked Read 6 Done Done Sent false
7464 2915 Acked Read 1 Done Done Sent false
7465 2916 Acked Read 2 Done Done Sent false
7466 2917 Acked Write 10 Done Done Sent false
7467 2918 Acked Write 9 Done Done Sent false
7468 2919 Acked Read 4 Done Done Sent false
7469 2920 Acked Read 3 Done Done Sent false
7470 2921 Acked Read 7 Done Done Sent false
7471 2922 Acked Read 8 Done Done Sent false
7472 2923 Acked Write 5 Done Done Sent false
7473 2924 Acked Read 1 Done Done Sent false
7474 2925 Acked Read 2 Done Done Sent false
7475 2926 Acked Write 8 Done Done Sent false
7476 2927 Acked Write 6 Done Done Sent false
7477 2928 Acked Read 4 Done Done Sent false
7478 2929 Acked Read 10 Done Done Sent false
7479 2930 Acked Write 8 Done Done Sent false
7480 2931 Acked Write 5 Done Done Sent false
7481 2932 Acked Write 2 Done Done Sent false
7482 2933 Acked Flush 0 Done Done Sent false
7483 2934 Acked Read 8 Done Done Sent false
7484 2935 Acked Write 9 Done Done Sent false
7485 2936 Acked Write 1 Done Done Sent false
7486 2937 Acked Read 5 Done Done Sent false
7487 2938 Acked Read 1 Done Done Sent false
7488 2939 Acked Read 3 Done Done Sent false
7489 2940 Acked Write 3 Done Done Sent false
7490 2941 Acked Read 9 Done Done Sent false
7491 2942 Acked Read 6 Done Done Sent false
7492 2943 Acked Read 10 Done Done Sent false
7493 2944 Acked Read 4 Done Done Sent false
7494 2945 Acked Read 8 Done Done Sent false
7495 2946 Acked Write 8 Done Done Sent false
7496 2947 Acked Read 10 Done Done Sent false
7497 2948 Acked Read 2 Done Done Sent false
7498 2949 Acked Write 2 Done Done Sent false
7499 2950 Acked Flush 0 Done Done Sent false
7500 2951 Acked Write 4 Done Done Sent false
7501 2952 Acked Read 6 Done Done Sent false
7502 2953 Acked Read 3 Done Done Sent false
7503 2954 Acked Write 8 Done Done Sent false
7504 2955 Acked Read 2 Done Done Sent false
7505 2956 Acked Flush 0 Done Done Sent false
7506 2957 Acked Read 5 Done Done Sent false
7507 2958 Acked Write 5 Done Done Sent false
7508 2959 Acked Write 8 Done Done Sent false
7509 2960 Acked Flush 0 Done Done Sent false
7510 2961 Acked Flush 0 Done Done Sent false
7511 2962 Acked Write 5 Done Done Sent false
7512 2963 Acked Read 4 Done Done Sent false
7513 2964 Acked Read 4 Done Done Sent false
7514 2965 Acked Read 7 Done Done Sent false
7515 2966 Acked Read 6 Done Done Sent false
7516 2967 Acked Read 10 Done Done Sent false
7517 2968 Acked Write 10 Done Done Sent false
7518 2969 Acked Read 8 Done Done Sent false
7519 2970 Acked Write 8 Done Done Sent false
7520 2971 Acked Read 8 Done Done Sent false
7521 2972 Acked Write 6 Done Done Sent false
7522 2973 Acked Write 8 Done Done Sent false
7523 2974 Acked Read 10 Done Done Sent false
7524 2975 Acked Write 8 Done Done Sent false
7525 2976 Acked Write 9 Done Done Sent false
7526 2977 Acked Read 10 Done Done Sent false
7527 2978 Acked Read 8 Done Done Sent false
7528 2979 Acked Read 9 Done Done Sent false
7529 2980 Acked Flush 0 Done Done Sent false
7530 2981 Acked Write 6 Done Done Sent false
7531 2982 Acked Read 2 Done Done Sent false
7532 2983 Acked Write 1 Done Done Sent false
7533 2984 Acked Write 8 Done Done Sent false
7534 2985 Acked Write 6 Done Done Sent false
7535 2986 Acked Flush 0 Done Done Sent false
7536 2987 Acked Read 7 Done Done Sent false
7537 2988 Acked Read 8 Done Done Sent false
7538 2989 Acked Read 5 Done Done Sent false
7539 2990 Acked Write 1 Done Done Sent false
7540 2991 Acked Read 1 Done Done Sent false
7541 2992 Acked Read 9 Done Done Sent false
7542 2993 Acked Write 3 Done Done Sent false
7543 2994 Acked Read 2 Done Done Sent false
7544 2995 Acked Read 6 Done Done Sent false
7545 2996 Acked Write 4 Done Done Sent false
7546 2997 Acked Write 7 Done Done Sent false
7547 2998 Acked Write 3 Done Done Sent false
7548 2999 Acked Write 7 Done Done Sent false
7549 3000 Acked Write 9 Done Done Sent false
7550 3001 Acked Read 2 Done Done Sent false
7551 3002 Acked Write 4 Done Done Sent false
7552 3003 Acked Read 1 Done Done Sent false
7553 3004 Acked Read 5 Done Done Sent false
7554 3005 Acked Write 5 Done Done Sent false
7555 3006 Acked Write 4 Done Done Sent false
7556 3007 Acked Read 5 Done Done Sent false
7557 3008 Acked Read 1 Done Done Sent false
7558 3009 Acked Write 4 Done Done Sent false
7559 3010 Acked Write 3 Done Done Sent false
7560 3011 Acked Read 10 Done Done Sent false
7561 3012 Acked Write 2 Done Done Sent false
7562 3013 Acked Read 10 Done Done Sent false
7563 3014 Acked Write 3 Done Done Sent false
7564 3015 Acked Read 7 Done Done Sent false
7565 3016 Acked Write 5 Done Done Sent false
7566 3017 Acked Write 3 Done Done Sent false
7567 3018 Acked Read 8 Done Done Sent false
7568 3019 Acked Write 4 Done Done Sent false
7569 3020 Acked Write 1 Done Done Sent false
7570 3021 Acked Write 2 Done Done Sent false
7571 3022 Acked Flush 0 Done Done Sent false
7572 3023 Acked Read 7 Done Done Sent false
7573 3024 Acked Read 1 Done Done Sent false
7574 3025 Acked Read 1 Done Done Sent false
7575 3026 Acked Read 4 Done Done Sent false
7576 3027 Acked Write 8 Done Done Sent false
7577 3028 Acked Write 2 Done Done Sent false
7578 3029 Acked Flush 0 Done Done Sent false
7579 3030 Acked Read 7 Done Done Sent false
7580 3031 Acked Write 5 Done Done Sent false
7581 3032 Acked Read 4 Done Done Sent false
7582 3033 Acked Read 9 Done Done Sent false
7583 3034 Acked Read 10 Done Done Sent false
7584 3035 Acked Read 5 Done Done Sent false
7585 3036 Acked Flush 0 Done Done Sent false
7586 3037 Acked Write 7 Done Done Sent false
7587 3038 Acked Write 9 Done Done Sent false
7588 3039 Acked Read 4 Done Done Sent false
7589 3040 Acked Read 3 Done Done Sent false
7590 3041 Acked Read 2 Done Done Sent false
7591 3042 Acked Write 1 Done Done Sent false
7592 3043 Acked Write 10 Done Done Sent false
7593 3044 Acked Read 6 Done Done Sent false
7594 3045 Acked Read 4 Done Done Sent false
7595 3046 Acked Read 9 Done Done Sent false
7596 3047 Acked Read 5 Done Done Sent false
7597 3048 Acked Read 1 Done Done Sent false
7598 3049 Acked Read 10 Done Done Sent false
7599 3050 Acked Write 7 Done Done Sent false
7600 3051 Acked Read 3 Done Done Sent false
7601 3052 Acked Read 1 Done Done Sent false
7602 3053 Acked Write 8 Done Done Sent false
7603 3054 Acked Read 4 Done Done Sent false
7604 3055 Acked Write 8 Done Done Sent false
7605 3056 Acked Read 2 Done Done Sent false
7606 3057 Acked Write 7 Done Done Sent false
7607 3058 Acked Read 2 Done Done Sent false
7608 3059 Acked Read 7 Done Done Sent false
7609 3060 Acked Write 4 Done Done Sent false
7610 3061 Acked Write 7 Done Done Sent false
7611 3062 Acked Read 5 Done Done Sent false
7612 3063 Acked Read 7 Done Done Sent false
7613 3064 Acked Read 8 Done Done Sent false
7614 3065 Acked Write 4 Done Done Sent false
7615 3066 Acked Write 3 Done Done Sent false
7616 3067 Acked Flush 0 Done Done Sent false
7617 3068 Acked Read 8 Done Done Sent false
7618 3069 Acked Write 10 Done Done Sent false
7619 3070 Acked Write 6 Done Done Sent false
7620 3071 Acked Read 1 Done Done Sent false
7621 3072 Acked Read 3 Done Done Sent false
7622 3073 Acked Read 2 Done Done Sent false
7623 3074 Acked Write 3 Done Done Sent false
7624 3075 Acked Write 6 Done Done Sent false
7625 3076 Acked Read 10 Done Done Sent false
7626 3077 Acked Write 3 Done Done Sent false
7627 3078 Acked Read 5 Done Done Sent false
7628 3079 Acked Read 7 Done Done Sent false
7629 3080 Acked Write 7 Done Done Sent false
7630 3081 Acked Read 6 Done Done Sent false
7631 3082 Acked Write 4 Done Done Sent false
7632 3083 Acked Read 8 Done Done Sent false
7633 3084 Acked Flush 0 Done Done Sent false
7634 3085 Acked Write 9 Done Done Sent false
7635 3086 Acked Write 3 Done Done Sent false
7636 3087 Acked Write 8 Done Done Sent false
7637 3088 Acked Read 8 Done Done Sent false
7638 3089 Acked Write 1 Done Done Sent false
7639 3090 Acked Read 3 Done Done Sent false
7640 3091 Acked Write 3 Done Done Sent false
7641 3092 Acked Write 2 Done Done Sent false
7642 3093 Acked Read 6 Done Done Sent false
7643 3094 Acked Write 7 Done Done Sent false
7644 3095 Acked Write 3 Done Done Sent false
7645 3096 Acked Read 1 Done Done Sent false
7646 3097 Acked Read 2 Done Done Sent false
7647 3098 Acked Read 1 Done Done Sent false
7648 3099 Acked Read 4 Done Done Sent false
7649 3100 Acked Write 4 Done Done Sent false
7650 3101 Acked Write 6 Done Done Sent false
7651 3102 Acked Write 9 Done Done Sent false
7652 3103 Acked Read 3 Done Done Sent false
7653 3104 Acked Flush 0 Done Done Sent false
7654 3105 Acked Read 6 Done Done Sent false
7655 3106 Acked Read 2 Done Done Sent false
7656 3107 Acked Read 8 Done Done Sent false
7657 3108 Acked Write 5 Done Done Sent false
7658 3109 Acked Write 10 Done Done Sent false
7659 3110 Acked Write 5 Done Done Sent false
7660 3111 Acked Read 9 Done Done Sent false
7661 3112 Acked Flush 0 Done Done Sent false
7662 STATES DS:0 DS:1 DS:2 TOTAL
7663 Sent 0 0 302 302
7664 Done 302 302 0 604
7665 Skipped 0 0 0 0
7666 Error 0 0 0 0
7667 Last Flush: 3112 3112 2810
7668 Downstairs last five completed: 2810 2809 2808 2807 2806
7669 Upstairs last five completed: 3112 3111 3110 3109 3108
7670 CLIENT: Up:0 ds:302 act:2
76712024-11-19T17:09:35.192ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76722024-11-19T17:09:35.192ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61885) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76732024-11-19T17:09:35.193ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76742024-11-19T17:09:35.194ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76752024-11-19T17:09:35.194ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76762024-11-19T17:09:35.194ZINFOcrucible: send last flush ID to this DS: 2810 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76772024-11-19T17:09:35.194ZINFOcrucible: Replied this last flush ID: 2810 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76782024-11-19T17:09:35.194ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76792024-11-19T17:09:35.194ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76802024-11-19T17:09:35.194ZINFOcrucible: [2] client re-new 302 jobs since flush 2810 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
76812024-11-19T17:09:35.194ZINFOcrucible: [2] Replayed 302 jobs since flush: 2810 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
7682 ----------------------------------------------------------------
7683 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7684 Downstairs last five completed: 3112 3111 3110 3109 3108
7685 Upstairs last five completed: 3112 3111 3110 3109 3108
7686 CLIENT: Up:0 ds:0 act:3
7687 Replay: All jobs finished
7688 Main task has work to do, go find it
7689 got dsc Stop(1)
7690 stop 1
7691 Nov 19 17:09:37.428 INFO request completed, latency_us: 334[8820] Got stop action so:false kr:false
7692 , response_code: 204, uri: /stop/cid/1, method: POST, req_id: 0f10dcaa-fb4c-48dc-8676-f2e0ad9ba6ac, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
76932024-11-19T17:09:37.429ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76942024-11-19T17:09:37.429ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76952024-11-19T17:09:37.429ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
76962024-11-19T17:09:37.429ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
76972024-11-19T17:09:37.429ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
7698 Nov 19 17:09:37.429 INFO request completed, latency_us: 149, response_code: 200, uri: /state/cid/1, method: GET, req_id: ff991f9a-5802-472e-85fd-16722f23b4d6, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
76992024-11-19T17:09:37.429ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
7700 [8820] Exited with: ExitStatus(unix_wait_status(9))
7701 [8820][1] reports Exit
7702 Nov 19 17:09:41.432 INFO request completed, latency_us: 360, response_code: 200, uri: /state/cid/1, method: GET, req_id: aadd6315-ee5a-4249-8fc3-6a8e09ad3f16, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
7703 001/300 Write block 474 len 4096 data: 8
7704 002/300 Read block 4502 len 24576
7705 003/300 Read block 7 len 4096
7706 004/300 Read block 917 len 20480
7707 005/300 Read block 2065 len 4096
7708 006/300 Flush
7709 007/300 Flush
7710 008/300 Write block 1722 len 24576 data: 2 2 2 2 2 2
7711 009/300 Read block 18 len 40960
7712 010/300 Flush
7713 011/300 Read block 4492 len 28672
7714 012/300 Write block 4896 len 24576 data: 3 3 3 3 3 2
7715 013/300 Write block 560 len 12288 data: 2 2 2
7716 014/300 Read block 2119 len 28672
7717 015/300 Write block 1123 len 36864 data: 3 3 3 3 3 3 3 2 2
7718 016/300 Flush
7719 017/300 Write block 1659 len 8192 data: 3 3
7720 018/300 Read block 630 len 16384
7721 019/300 Read block 4313 len 4096
7722 020/300 Flush
7723 021/300 Read block 765 len 16384
7724 022/300 Write block 3878 len 40960 data: 3 4 4 5 4 4 4 3 3 3
7725 023/300 Read block 2278 len 12288
7726 024/300 Read block 3436 len 32768
7727 025/300 Write block 3056 len 20480 data: 2 2 2 3 3
7728 026/300 Read block 245 len 12288
7729 027/300 Write block 3523 len 12288 data: 2 2 3
7730 028/300 Flush
7731 029/300 Read block 1777 len 16384
7732 030/300 Read block 1379 len 24576
7733 031/300 Read block 2535 len 20480
7734 032/300 Flush
7735 033/300 Read block 3123 len 8192
7736 034/300 Flush
7737 035/300 Read block 3888 len 12288
7738 036/300 Write block 1926 len 32768 data: 2 3 3 3 2 2 3 3
7739 037/300 Read block 2184 len 36864
7740 038/300 Read block 301 len 28672
7741 039/300 Read block 3866 len 8192
7742 040/300 Write block 3214 len 40960 data: 3 4 4 3 5 3 3 3 3 4
7743 041/300 Flush
7744 042/300 Read block 3730 len 24576
7745 043/300 Write block 3126 len 36864 data: 5 5 5 5 5 5 5 2 2
7746 044/300 Read block 2821 len 8192
7747 045/300 Read block 3987 len 16384
7748 046/300 Write block 3378 len 12288 data: 2 2 2
7749 047/300 Write block 1904 len 16384 data: 3 2 2 3
7750 048/300 Read block 620 len 36864
7751 049/300 Read block 1525 len 16384
7752 050/300 Read block 842 len 8192
7753 051/300 Write block 2343 len 36864 data: 2 3 3 3 3 3 3 4 4
7754 052/300 Flush
7755 053/300 Read block 4448 len 40960
7756 054/300 Write block 3293 len 20480 data: 2 2 2 2 3
7757 055/300 Read block 2615 len 28672
7758 056/300 Write block 1988 len 4096 data: 2
7759 057/300 Write block 4564 len 12288 data: 4 4 4
7760 058/300 Read block 1147 len 32768
7761 059/300 Read block 1791 len 24576
7762 060/300 Read block 4363 len 8192
7763 061/300 Write block 2184 len 8192 data: 2 2
7764 062/300 Write block 2974 len 8192 data: 3 3
7765 063/300 Read block 4454 len 4096
7766 064/300 Read block 779 len 28672
7767 065/300 Write block 4074 len 12288 data: 3 3 2
7768 066/300 Read block 3521 len 24576
7769 067/300 Flush
7770 068/300 Read block 4321 len 16384
7771 069/300 Read block 2615 len 36864
7772 070/300 Write block 2343 len 20480 data: 3 4 4 4 4
7773 071/300 Read block 2517 len 40960
7774 072/300 Write block 301 len 20480 data: 2 2 3 3 3
7775 073/300 Flush
7776 074/300 Read block 2436 len 40960
7777 075/300 Write block 474 len 36864 data: 9 8 7 7 4 3 2 2 2
7778 076/300 Read block 4112 len 28672
7779 077/300 Write block 3454 len 24576 data: 3 3 3 3 2 2
7780 078/300 Read block 4504 len 24576
7781 079/300 Write block 3019 len 16384 data: 2 2 2 2
7782 080/300 Read block 381 len 36864
7783 081/300 Write block 2614 len 28672 data: 4 4 5 4 2 3 3
7784 082/300 Write block 837 len 12288 data: 2 2 2
7785 083/300 Write block 152 len 28672 data: 3 4 4 4 4 4 4
7786 084/300 Read block 2242 len 32768
7787 085/300 Read block 3246 len 40960
7788 086/300 Flush
7789 087/300 Read block 1395 len 16384
7790 088/300 Read block 1945 len 28672
7791 089/300 Read block 307 len 32768
7792 090/300 Write block 2065 len 24576 data: 3 3 3 3 3 4
7793 091/300 Write block 3222 len 20480 data: 4 5 3 3 3
7794 092/300 Write block 1548 len 8192 data: 5 5
7795 093/300 Read block 4540 len 12288
7796 094/300 Write block 1380 len 16384 data: 3 2 2 2
7797 095/300 Write block 1333 len 36864 data: 2 2 2 2 2 2 2 2 2
7798 096/300 Write block 3272 len 12288 data: 4 4 3
7799 097/300 Write block 4932 len 12288 data: 5 6 4
7800 098/300 Read block 2678 len 16384
7801 099/300 Read block 1161 len 28672
7802 100/300 Read block 4947 len 32768
7803 101/300 Write block 1841 len 8192 data: 2 2
7804 102/300 Write block 2786 len 24576 data: 2 4 3 3 3 3
7805 103/300 Write block 2077 len 12288 data: 3 3 3
7806 104/300 Write block 4546 len 8192 data: 2 2
7807 105/300 Write block 2434 len 4096 data: 3
7808 106/300 Write block 239 len 32768 data: 3 2 2 2 2 2 3 4
7809 107/300 Flush
7810 108/300 Read block 474 len 24576
7811 109/300 Read block 3981 len 40960
7812 110/300 Read block 1519 len 24576
7813 111/300 Write block 2063 len 8192 data: 2 3
7814 112/300 Write block 4939 len 32768 data: 4 4 3 3 3 3 3 3
7815 113/300 Read block 2342 len 28672
7816 114/300 Read block 714 len 4096
7817 115/300 Read block 2207 len 40960
7818 116/300 Read block 4566 len 8192
7819 117/300 Read block 4765 len 28672
7820 118/300 Read block 2705 len 16384
7821 119/300 Read block 2821 len 20480
7822 120/300 Write block 4510 len 16384 data: 2 2 3 3
7823 121/300 Write block 1861 len 8192 data: 2 2
7824 122/300 Read block 407 len 28672
7825 123/300 Read block 2434 len 40960
7826 124/300 Read block 40 len 4096
7827 125/300 Write block 2556 len 20480 data: 2 2 2 2 2
7828 126/300 Read block 4760 len 32768
7829 127/300 Write block 624 len 28672 data: 2 2 2 2 2 2 2
7830 128/300 Read block 260 len 8192
7831 129/300 Flush
7832 130/300 Write block 2467 len 8192 data: 3 3
7833 131/300 Read block 380 len 24576
7834 132/300 Read block 593 len 16384
7835 133/300 Read block 4589 len 16384
7836 134/300 Read block 1909 len 28672
7837 135/300 Write block 4786 len 8192 data: 2 2
7838 136/300 Write block 2271 len 16384 data: 2 3 3 4
7839 137/300 Read block 4648 len 28672
7840 138/300 Read block 4097 len 8192
7841 139/300 Write block 4931 len 16384 data: 4 6 7 5
7842 140/300 Write block 3449 len 20480 data: 3 3 4 4 4
7843 141/300 Write block 4622 len 8192 data: 4 4
7844 142/300 Write block 2309 len 4096 data: 2
7845 143/300 Read block 3350 len 28672
7846 144/300 Read block 1579 len 40960
7847 145/300 Write block 3531 len 4096 data: 3
7848 146/300 Flush
7849 147/300 Read block 4296 len 4096
7850 148/300 Read block 3081 len 8192
7851 149/300 Read block 2024 len 8192
7852 150/300 Write block 4501 len 24576 data: 3 3 3 3 3 3
7853 151/300 Write block 1582 len 32768 data: 4 3 4 4 3 3 2 2
7854 152/300 Write block 3485 len 28672 data: 4 4 4 3 4 3 3
7855 153/300 Write block 2910 len 28672 data: 2 2 2 2 3 3 3
7856 154/300 Read block 4456 len 4096
7857 155/300 Read block 1918 len 36864
7858 156/300 Write block 3481 len 28672 data: 3 4 3 4 5 5 5
7859 157/300 Write block 2340 len 4096 data: 3
7860 158/300 Write block 3806 len 24576 data: 3 2 2 2 2 2
7861 159/300 Write block 3128 len 20480 data: 6 6 6 6 6
7862 160/300 Read block 159 len 24576
7863 161/300 Write block 2654 len 20480 data: 3 3 2 2 2
7864 162/300 Read block 3555 len 4096
7865 163/300 Write block 662 len 40960 data: 3 2 2 2 3 5 5 6 5 5
7866 164/300 Read block 1292 len 24576
7867 165/300 Read block 3314 len 12288
7868 166/300 Flush
7869 167/300 Read block 184 len 24576
7870 168/300 Write block 4872 len 16384 data: 3 3 3 3
7871 169/300 Read block 3001 len 16384
7872 170/300 Read block 3662 len 28672
7873 171/300 Write block 2046 len 8192 data: 3 3
7874 172/300 Read block 167 len 36864
7875 173/300 Flush
7876 174/300 Flush
7877 175/300 Write block 3821 len 32768 data: 3 4 4 4 4 4 4 3
7878 176/300 Write block 4548 len 40960 data: 2 2 3 3 4 4 4 4 3 4
7879 177/300 Read block 4338 len 36864
7880 178/300 Write block 3801 len 12288 data: 5 4 3
7881 179/300 Read block 165 len 16384
7882 180/300 Read block 1113 len 8192
7883 181/300 Read block 2499 len 36864
7884 182/300 Write block 779 len 36864 data: 2 3 4 4 4 5 4 5 6
7885 183/300 Write block 754 len 8192 data: 2 2
7886 184/300 Flush
7887 185/300 Read block 947 len 28672
7888 186/300 Read block 184 len 12288
7889 187/300 Write block 1642 len 12288 data: 2 3 3
7890 188/300 Write block 1878 len 24576 data: 4 4 3 3 3 5
7891 189/300 Read block 3988 len 32768
7892 190/300 Read block 280 len 40960
7893 191/300 Write block 1468 len 28672 data: 2 2 2 2 2 2 2
7894 192/300 Read block 2771 len 28672
7895 193/300 Write block 4367 len 16384 data: 3 3 3 3
7896 194/300 Read block 3147 len 4096
7897 195/300 Flush
7898 196/300 Write block 3889 len 24576 data: 3 2 2 2 2 2
7899 197/300 Read block 1950 len 20480
7900 198/300 Write block 4209 len 8192 data: 3 3
7901 199/300 Read block 1122 len 12288
7902 200/300 Write block 3312 len 32768 data: 3 3 2 2 2 3 3 3
7903 201/300 Write block 1283 len 32768 data: 4 3 4 4 3 3 4 4
7904 202/300 Write block 1905 len 20480 data: 3 3 4 3 2
7905 203/300 Write block 1404 len 12288 data: 4 5 6
7906 204/300 Read block 1900 len 28672
7907 205/300 Read block 3872 len 4096
7908 206/300 Read block 3903 len 8192
7909 207/300 Read block 817 len 4096
7910 208/300 Flush
7911 209/300 Read block 2173 len 12288
7912 210/300 Read block 3397 len 28672
7913 211/300 Write block 488 len 36864 data: 2 2 2 3 3 3 3 4 4
7914 212/300 Write block 1936 len 40960 data: 3 4 4 4 4 3 3 3 2 2
7915 213/300 Read block 2950 len 36864
7916 214/300 Write block 3449 len 20480 data: 4 4 5 5 5
7917 215/300 Write block 2776 len 36864 data: 4 2 2 2 2 2 2 3 3
7918 216/300 Read block 2857 len 20480
7919 217/300 Write block 52 len 8192 data: 2 2
7920 218/300 Flush
7921 219/300 Write block 2525 len 32768 data: 3 3 2 2 3 4 4 4
7922 220/300 Read block 3708 len 4096
7923 221/300 Write block 1189 len 16384 data: 5 5 6 5
7924 222/300 Flush
7925 223/300 Read block 456 len 36864
7926 224/300 Write block 3313 len 36864 data: 4 3 3 3 4 4 4 3 4
7927 225/300 Read block 1604 len 24576
7928 226/300 Read block 3823 len 20480
7929 227/300 Write block 147 len 16384 data: 3 2 2 3
7930 228/300 Read block 3204 len 20480
7931 229/300 Write block 3535 len 20480 data: 2 2 2 2 2
7932 230/300 Write block 4021 len 32768 data: 5 5 5 5 4 3 4 4
7933 231/300 Read block 654 len 40960
7934 232/300 Read block 4811 len 40960
7935 233/300 Write block 2256 len 40960 data: 4 4 3 2 2 2 2 2 2 2
7936 234/300 Read block 4437 len 28672
7937 235/300 Write block 2581 len 24576 data: 2 4 4 4 3 3
7938 236/300 Read block 2423 len 40960
7939 237/300 Write block 4053 len 36864 data: 2 2 2 2 2 2 2 2 2
7940 238/300 Read block 1783 len 20480
7941 239/300 Read block 4206 len 28672
7942 240/300 Read block 2345 len 24576
7943 241/300 Read block 85 len 28672
7944 242/300 Read block 3639 len 12288
7945 243/300 Write block 3775 len 24576 data: 2 2 2 2 2 2
7946 244/300 Read block 1631 len 24576
7947 245/300 Read block 848 len 32768
7948 246/300 Write block 344 len 36864 data: 3 3 4 4 3 3 3 3 3
7949 247/300 Write block 4731 len 32768 data: 2 3 3 3 3 3 3 3
7950 248/300 Write block 2484 len 32768 data: 2 2 3 3 3 2 2 2
7951 249/300 Read block 1482 len 24576
7952 250/300 Read block 2076 len 28672
7953 251/300 Write block 201 len 20480 data: 3 3 3 3 2
7954 252/300 Write block 1312 len 36864 data: 3 3 3 3 3 3 3 3 3
7955 253/300 Read block 991 len 16384
7956 254/300 Flush
7957 255/300 Read block 119 len 8192
7958 256/300 Read block 2069 len 4096
7959 257/300 Write block 4944 len 32768 data: 4 4 4 3 3 3 2 2
7960 258/300 Read block 1329 len 8192
7961 259/300 Write block 2841 len 32768 data: 2 2 2 2 2 2 2 2
7962 260/300 Write block 1118 len 12288 data: 3 4 4
7963 261/300 Read block 3718 len 8192
7964 262/300 Write block 4817 len 8192 data: 2 2
7965 263/300 Read block 311 len 4096
7966 264/300 Read block 2438 len 40960
7967 265/300 Read block 851 len 8192
7968 266/300 Read block 3495 len 20480
7969 267/300 Write block 4618 len 28672 data: 3 3 4 4 5 5 4
7970 268/300 Read block 1806 len 8192
7971 269/300 Read block 681 len 4096
7972 270/300 Read block 3031 len 12288
7973 271/300 Read block 610 len 4096
7974 272/300 Read block 72 len 28672
7975 273/300 Read block 806 len 20480
7976 274/300 Write block 1800 len 36864 data: 2 2 2 2 2 2 2 2 2
7977 275/300 Read block 231 len 32768
7978 276/300 Read block 324 len 32768
7979 277/300 Read block 331 len 28672
7980 278/300 Write block 4425 len 4096 data: 4
7981 279/300 Write block 4264 len 24576 data: 3 3 3 3 3 2
7982 280/300 Write block 1406 len 8192 data: 7 6
7983 281/300 Write block 2992 len 28672 data: 3 4 4 4 4 4 3
7984 282/300 Write block 2553 len 20480 data: 2 2 2 3 3
7985 283/300 Read block 3188 len 36864
7986 284/300 Write block 4355 len 4096 data: 4
7987 285/300 Write block 1611 len 32768 data: 6 5 5 5 5 5 5 4
7988 286/300 Write block 3201 len 28672 data: 3 3 4 3 3 3 3
7989 287/300 Read block 1146 len 20480
7990 288/300 Read block 3150 len 40960
7991 289/300 Read block 1942 len 24576
7992 290/300 Read block 4011 len 20480
7993 291/300 Flush
7994 292/300 Read block 4669 len 16384
7995 293/300 Write block 2280 len 12288 data: 3 4 3
7996 294/300 Read block 1872 len 28672
7997 295/300 Write block 2287 len 12288 data: 5 5 5
7998 296/300 Read block 625 len 4096
7999 297/300 Read block 4683 len 16384
8000 298/300 Read block 660 len 20480
8001 299/300 Write block 4801 len 12288 data: 5 4 4
8002 300/300 Read block 2093 len 20480
8003 301/300 Read block 333 len 16384
8004 Main task has work to do, go find it
8005 got dsc Start(1)
8006 start 1
8007 Nov 19 17:09:41.668 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6e048997-a54d-4aa6-a1bf-5678c6b4d30e, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
8008 Replay: started 1, returned:Ok(())
8009 ----------------------------------------------------------------
8010 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8011 States: Active Offline Active
8012 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8013 3113 Acked Write 1 Done Sent Done false
8014 3114 Acked Read 6 Done Sent Done false
8015 3115 Acked Read 1 Done Sent Done false
8016 3116 Acked Read 5 Done Sent Done false
8017 3117 Acked Read 1 Done Sent Done false
8018 3118 Acked Flush 0 Done Sent Done false
8019 3119 Acked Flush 0 Done Sent Done false
8020 3120 Acked Write 6 Done Sent Done false
8021 3121 Acked Read 10 Done Sent Done false
8022 3122 Acked Flush 0 Done Sent Done false
8023 3123 Acked Read 7 Done Sent Done false
8024 3124 Acked Write 6 Done Sent Done false
8025 3125 Acked Write 3 Done Sent Done false
8026 3126 Acked Read 7 Done Sent Done false
8027 3127 Acked Write 9 Done Sent Done false
8028 3128 Acked Flush 0 Done Sent Done false
8029 3129 Acked Write 2 Done Sent Done false
8030 3130 Acked Read 4 Done Sent Done false
8031 3131 Acked Read 1 Done Sent Done false
8032 3132 Acked Flush 0 Done Sent Done false
8033 3133 Acked Read 4 Done Sent Done false
8034 3134 Acked Write 10 Done Sent Done false
8035 3135 Acked Read 3 Done Sent Done false
8036 3136 Acked Read 8 Done Sent Done false
8037 3137 Acked Write 5 Done Sent Done false
8038 3138 Acked Read 3 Done Sent Done false
8039 3139 Acked Write 3 Done Sent Done false
8040 3140 Acked Flush 0 Done Sent Done false
8041 3141 Acked Read 4 Done Sent Done false
8042 3142 Acked Read 6 Done Sent Done false
8043 3143 Acked Read 5 Done Sent Done false
8044 3144 Acked Flush 0 Done Sent Done false
8045 3145 Acked Read 2 Done Sent Done false
8046 3146 Acked Flush 0 Done Sent Done false
8047 3147 Acked Read 3 Done Sent Done false
8048 3148 Acked Write 8 Done Sent Done false
8049 3149 Acked Read 9 Done Sent Done false
8050 3150 Acked Read 7 Done Sent Done false
8051 3151 Acked Read 2 Done Sent Done false
8052 3152 Acked Write 10 Done Sent Done false
8053 3153 Acked Flush 0 Done Sent Done false
8054 3154 Acked Read 6 Done Sent Done false
8055 3155 Acked Write 9 Done Sent Done false
8056 3156 Acked Read 2 Done Sent Done false
8057 3157 Acked Read 4 Done Sent Done false
8058 3158 Acked Write 3 Done Sent Done false
8059 3159 Acked Write 4 Done Sent Done false
8060 3160 Acked Read 9 Done Sent Done false
8061 3161 Acked Read 4 Done Sent Done false
8062 3162 Acked Read 2 Done Sent Done false
8063 3163 Acked Write 9 Done Sent Done false
8064 3164 Acked Flush 0 Done Sent Done false
8065 3165 Acked Read 10 Done Sent Done false
8066 3166 Acked Write 5 Done Sent Done false
8067 3167 Acked Read 7 Done Sent Done false
8068 3168 Acked Write 1 Done Sent Done false
8069 3169 Acked Write 3 Done Sent Done false
8070 3170 Acked Read 8 Done Sent Done false
8071 3171 Acked Read 6 Done Sent Done false
8072 3172 Acked Read 2 Done Sent Done false
8073 3173 Acked Write 2 Done Sent Done false
8074 3174 Acked Write 2 Done Sent Done false
8075 3175 Acked Read 1 Done Sent Done false
8076 3176 Acked Read 7 Done Sent Done false
8077 3177 Acked Write 3 Done Sent Done false
8078 3178 Acked Read 6 Done Sent Done false
8079 3179 Acked Flush 0 Done Sent Done false
8080 3180 Acked Read 4 Done Sent Done false
8081 3181 Acked Read 9 Done Sent Done false
8082 3182 Acked Write 5 Done Sent Done false
8083 3183 Acked Read 10 Done Sent Done false
8084 3184 Acked Write 5 Done Sent Done false
8085 3185 Acked Flush 0 Done Sent Done false
8086 3186 Acked Read 10 Done Sent Done false
8087 3187 Acked Write 9 Done Sent Done false
8088 3188 Acked Read 7 Done Sent Done false
8089 3189 Acked Write 6 Done Sent Done false
8090 3190 Acked Read 6 Done Sent Done false
8091 3191 Acked Write 4 Done Sent Done false
8092 3192 Acked Read 9 Done Sent Done false
8093 3193 Acked Write 7 Done Sent Done false
8094 3194 Acked Write 3 Done Sent Done false
8095 3195 Acked Write 7 Done Sent Done false
8096 3196 Acked Read 8 Done Sent Done false
8097 3197 Acked Read 10 Done Sent Done false
8098 3198 Acked Flush 0 Done Sent Done false
8099 3199 Acked Read 4 Done Sent Done false
8100 3200 Acked Read 7 Done Sent Done false
8101 3201 Acked Read 8 Done Sent Done false
8102 3202 Acked Write 6 Done Sent Done false
8103 3203 Acked Write 5 Done Sent Done false
8104 3204 Acked Write 2 Done Sent Done false
8105 3205 Acked Read 3 Done Sent Done false
8106 3206 Acked Write 4 Done Sent Done false
8107 3207 Acked Write 9 Done Sent Done false
8108 3208 Acked Write 3 Done Sent Done false
8109 3209 Acked Write 3 Done Sent Done false
8110 3210 Acked Read 4 Done Sent Done false
8111 3211 Acked Read 7 Done Sent Done false
8112 3212 Acked Read 8 Done Sent Done false
8113 3213 Acked Write 2 Done Sent Done false
8114 3214 Acked Write 6 Done Sent Done false
8115 3215 Acked Write 3 Done Sent Done false
8116 3216 Acked Write 2 Done Sent Done false
8117 3217 Acked Write 1 Done Sent Done false
8118 3218 Acked Write 8 Done Sent Done false
8119 3219 Acked Flush 0 Done Sent Done false
8120 3220 Acked Read 6 Done Sent Done false
8121 3221 Acked Read 10 Done Sent Done false
8122 3222 Acked Read 6 Done Sent Done false
8123 3223 Acked Write 2 Done Sent Done false
8124 3224 Acked Write 8 Done Sent Done false
8125 3225 Acked Read 7 Done Sent Done false
8126 3226 Acked Read 1 Done Sent Done false
8127 3227 Acked Read 10 Done Sent Done false
8128 3228 Acked Read 2 Done Sent Done false
8129 3229 Acked Read 7 Done Sent Done false
8130 3230 Acked Read 4 Done Sent Done false
8131 3231 Acked Read 5 Done Sent Done false
8132 3232 Acked Write 4 Done Sent Done false
8133 3233 Acked Write 2 Done Sent Done false
8134 3234 Acked Read 7 Done Sent Done false
8135 3235 Acked Read 10 Done Sent Done false
8136 3236 Acked Read 1 Done Sent Done false
8137 3237 Acked Write 5 Done Sent Done false
8138 3238 Acked Read 8 Done Sent Done false
8139 3239 Acked Write 7 Done Sent Done false
8140 3240 Acked Read 2 Done Sent Done false
8141 3241 Acked Flush 0 Done Sent Done false
8142 3242 Acked Write 2 Done Sent Done false
8143 3243 Acked Read 6 Done Sent Done false
8144 3244 Acked Read 4 Done Sent Done false
8145 3245 Acked Read 4 Done Sent Done false
8146 3246 Acked Read 7 Done Sent Done false
8147 3247 Acked Write 2 Done Sent Done false
8148 3248 Acked Write 4 Done Sent Done false
8149 3249 Acked Read 7 Done Sent Done false
8150 3250 Acked Read 2 Done Sent Done false
8151 3251 Acked Write 4 Done Sent Done false
8152 3252 Acked Write 5 Done Sent Done false
8153 3253 Acked Write 2 Done Sent Done false
8154 3254 Acked Write 1 Done Sent Done false
8155 3255 Acked Read 7 Done Sent Done false
8156 3256 Acked Read 10 Done Sent Done false
8157 3257 Acked Write 1 Done Sent Done false
8158 3258 Acked Flush 0 Done Sent Done false
8159 3259 Acked Read 1 Done Sent Done false
8160 3260 Acked Read 2 Done Sent Done false
8161 3261 Acked Read 2 Done Sent Done false
8162 3262 Acked Write 6 Done Sent Done false
8163 3263 Acked Write 8 Done Sent Done false
8164 3264 Acked Write 7 Done Sent Done false
8165 3265 Acked Write 7 Done Sent Done false
8166 3266 Acked Read 1 Done Sent Done false
8167 3267 Acked Read 9 Done Sent Done false
8168 3268 Acked Write 7 Done Sent Done false
8169 3269 Acked Write 1 Done Sent Done false
8170 3270 Acked Write 6 Done Sent Done false
8171 3271 Acked Write 5 Done Sent Done false
8172 3272 Acked Read 6 Done Sent Done false
8173 3273 Acked Write 5 Done Sent Done false
8174 3274 Acked Read 1 Done Sent Done false
8175 3275 Acked Write 10 Done Sent Done false
8176 3276 Acked Read 6 Done Sent Done false
8177 3277 Acked Read 3 Done Sent Done false
8178 3278 Acked Flush 0 Done Sent Done false
8179 3279 Acked Read 6 Done Sent Done false
8180 3280 Acked Write 4 Done Sent Done false
8181 3281 Acked Read 4 Done Sent Done false
8182 3282 Acked Read 7 Done Sent Done false
8183 3283 Acked Write 2 Done Sent Done false
8184 3284 Acked Read 9 Done Sent Done false
8185 3285 Acked Flush 0 Done Sent Done false
8186 3286 Acked Flush 0 Done Sent Done false
8187 3287 Acked Write 8 Done Sent Done false
8188 3288 Acked Write 10 Done Sent Done false
8189 3289 Acked Read 9 Done Sent Done false
8190 3290 Acked Write 3 Done Sent Done false
8191 3291 Acked Read 4 Done Sent Done false
8192 3292 Acked Read 2 Done Sent Done false
8193 3293 Acked Read 9 Done Sent Done false
8194 3294 Acked Write 9 Done Sent Done false
8195 3295 Acked Write 2 Done Sent Done false
8196 3296 Acked Flush 0 Done Sent Done false
8197 3297 Acked Read 7 Done Sent Done false
8198 3298 Acked Read 3 Done Sent Done false
8199 3299 Acked Write 3 Done Sent Done false
8200 3300 Acked Write 6 Done Sent Done false
8201 3301 Acked Read 8 Done Sent Done false
8202 3302 Acked Read 10 Done Sent Done false
8203 3303 Acked Write 7 Done Sent Done false
8204 3304 Acked Read 7 Done Sent Done false
8205 3305 Acked Write 4 Done Sent Done false
8206 3306 Acked Read 1 Done Sent Done false
8207 3307 Acked Flush 0 Done Sent Done false
8208 3308 Acked Write 6 Done Sent Done false
8209 3309 Acked Read 5 Done Sent Done false
8210 3310 Acked Write 2 Done Sent Done false
8211 3311 Acked Read 3 Done Sent Done false
8212 3312 Acked Write 8 Done Sent Done false
8213 3313 Acked Write 8 Done Sent Done false
8214 3314 Acked Write 5 Done Sent Done false
8215 3315 Acked Write 3 Done Sent Done false
8216 3316 Acked Read 7 Done Sent Done false
8217 3317 Acked Read 1 Done Sent Done false
8218 3318 Acked Read 2 Done Sent Done false
8219 3319 Acked Read 1 Done Sent Done false
8220 3320 Acked Flush 0 Done Sent Done false
8221 3321 Acked Read 3 Done Sent Done false
8222 3322 Acked Read 7 Done Sent Done false
8223 3323 Acked Write 9 Done Sent Done false
8224 3324 Acked Write 10 Done Sent Done false
8225 3325 Acked Read 9 Done Sent Done false
8226 3326 Acked Write 5 Done Sent Done false
8227 3327 Acked Write 9 Done Sent Done false
8228 3328 Acked Read 5 Done Sent Done false
8229 3329 Acked Write 2 Done Sent Done false
8230 3330 Acked Flush 0 Done Sent Done false
8231 3331 Acked Write 8 Done Sent Done false
8232 3332 Acked Read 1 Done Sent Done false
8233 3333 Acked Write 4 Done Sent Done false
8234 3334 Acked Flush 0 Done Sent Done false
8235 3335 Acked Read 9 Done Sent Done false
8236 3336 Acked Write 9 Done Sent Done false
8237 3337 Acked Read 6 Done Sent Done false
8238 3338 Acked Read 5 Done Sent Done false
8239 3339 Acked Write 4 Done Sent Done false
8240 3340 Acked Read 5 Done Sent Done false
8241 3341 Acked Write 5 Done Sent Done false
8242 3342 Acked Write 8 Done Sent Done false
8243 3343 Acked Read 10 Done Sent Done false
8244 3344 Acked Read 10 Done Sent Done false
8245 3345 Acked Write 10 Done Sent Done false
8246 3346 Acked Read 7 Done Sent Done false
8247 3347 Acked Write 6 Done Sent Done false
8248 3348 Acked Read 10 Done Sent Done false
8249 3349 Acked Write 9 Done Sent Done false
8250 3350 Acked Read 5 Done Sent Done false
8251 3351 Acked Read 7 Done Sent Done false
8252 3352 Acked Read 6 Done Sent Done false
8253 3353 Acked Read 7 Done Sent Done false
8254 3354 Acked Read 3 Done Sent Done false
8255 3355 Acked Write 6 Done Sent Done false
8256 3356 Acked Read 6 Done Sent Done false
8257 3357 Acked Read 8 Done Sent Done false
8258 3358 Acked Write 9 Done Sent Done false
8259 3359 Acked Write 8 Done Sent Done false
8260 3360 Acked Write 8 Done Sent Done false
8261 3361 Acked Read 6 Done Sent Done false
8262 3362 Acked Read 7 Done Sent Done false
8263 3363 Acked Write 5 Done Sent Done false
8264 3364 Acked Write 9 Done Sent Done false
8265 3365 Acked Read 4 Done Sent Done false
8266 3366 Acked Flush 0 Done Sent Done false
8267 3367 Acked Read 2 Done Sent Done false
8268 3368 Acked Read 1 Done Sent Done false
8269 3369 Acked Write 8 Done Sent Done false
8270 3370 Acked Read 2 Done Sent Done false
8271 3371 Acked Write 8 Done Sent Done false
8272 3372 Acked Write 3 Done Sent Done false
8273 3373 Acked Read 2 Done Sent Done false
8274 3374 Acked Write 2 Done Sent Done false
8275 3375 Acked Read 1 Done Sent Done false
8276 3376 Acked Read 10 Done Sent Done false
8277 3377 Acked Read 2 Done Sent Done false
8278 3378 Acked Read 5 Done Sent Done false
8279 3379 Acked Write 7 Done Sent Done false
8280 3380 Acked Read 2 Done Sent Done false
8281 3381 Acked Read 1 Done Sent Done false
8282 3382 Acked Read 3 Done Sent Done false
8283 3383 Acked Read 1 Done Sent Done false
8284 3384 Acked Read 7 Done Sent Done false
8285 3385 Acked Read 5 Done Sent Done false
8286 3386 Acked Write 9 Done Sent Done false
8287 3387 Acked Read 8 Done Sent Done false
8288 3388 Acked Read 8 Done Sent Done false
8289 3389 Acked Read 7 Done Sent Done false
8290 3390 Acked Write 1 Done Sent Done false
8291 3391 Acked Write 6 Done Sent Done false
8292 3392 Acked Write 2 Done Sent Done false
8293 3393 Acked Write 7 Done Sent Done false
8294 3394 Acked Write 5 Done Sent Done false
8295 3395 Acked Read 9 Done Sent Done false
8296 3396 Acked Write 1 Done Sent Done false
8297 3397 Acked Write 8 Done Sent Done false
8298 3398 Acked Write 7 Done Sent Done false
8299 3399 Acked Read 5 Done Sent Done false
8300 3400 Acked Read 10 Done Sent Done false
8301 3401 Acked Read 6 Done Sent Done false
8302 3402 Acked Read 5 Done Sent Done false
8303 3403 Acked Flush 0 Done Sent Done false
8304 3404 Acked Read 4 Done Sent Done false
8305 3405 Acked Write 3 Done Sent Done false
8306 3406 Acked Read 7 Done Sent Done false
8307 3407 Acked Write 3 Done Sent Done false
8308 3408 Acked Read 1 Done Sent Done false
8309 3409 Acked Read 4 Done Sent Done false
8310 3410 Acked Read 5 Done Sent Done false
8311 3411 Acked Write 3 Done Sent Done false
8312 3412 Acked Read 5 Done Sent Done false
8313 3413 Acked Read 4 Done Sent Done false
8314 STATES DS:0 DS:1 DS:2 TOTAL
8315 Sent 0 301 0 301
8316 Done 301 0 301 602
8317 Skipped 0 0 0 0
8318 Error 0 0 0 0
8319 Last Flush: 3403 3112 3403
8320 Downstairs last five completed: 3112 3111 3110 3109 3108
8321 Upstairs last five completed: 3413 3412 3411 3410 3409
8322 CLIENT: Up:0 ds:301 act:2
8323 ----------------------------------------------------------------
8324 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8325 States: Active Offline Active
8326 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8327 3113 Acked Write 1 Done Sent Done false
8328 3114 Acked Read 6 Done Sent Done false
8329 3115 Acked Read 1 Done Sent Done false
8330 3116 Acked Read 5 Done Sent Done false
8331 3117 Acked Read 1 Done Sent Done false
8332 3118 Acked Flush 0 Done Sent Done false
8333 3119 Acked Flush 0 Done Sent Done false
8334 3120 Acked Write 6 Done Sent Done false
8335 3121 Acked Read 10 Done Sent Done false
8336 3122 Acked Flush 0 Done Sent Done false
8337 3123 Acked Read 7 Done Sent Done false
8338 3124 Acked Write 6 Done Sent Done false
8339 3125 Acked Write 3 Done Sent Done false
8340 3126 Acked Read 7 Done Sent Done false
8341 3127 Acked Write 9 Done Sent Done false
8342 3128 Acked Flush 0 Done Sent Done false
8343 3129 Acked Write 2 Done Sent Done false
8344 3130 Acked Read 4 Done Sent Done false
8345 3131 Acked Read 1 Done Sent Done false
8346 3132 Acked Flush 0 Done Sent Done false
8347 3133 Acked Read 4 Done Sent Done false
8348 3134 Acked Write 10 Done Sent Done false
8349 3135 Acked Read 3 Done Sent Done false
8350 3136 Acked Read 8 Done Sent Done false
8351 3137 Acked Write 5 Done Sent Done false
8352 3138 Acked Read 3 Done Sent Done false
8353 3139 Acked Write 3 Done Sent Done false
8354 3140 Acked Flush 0 Done Sent Done false
8355 3141 Acked Read 4 Done Sent Done false
8356 3142 Acked Read 6 Done Sent Done false
8357 3143 Acked Read 5 Done Sent Done false
8358 3144 Acked Flush 0 Done Sent Done false
8359 3145 Acked Read 2 Done Sent Done false
8360 3146 Acked Flush 0 Done Sent Done false
8361 3147 Acked Read 3 Done Sent Done false
8362 3148 Acked Write 8 Done Sent Done false
8363 3149 Acked Read 9 Done Sent Done false
8364 3150 Acked Read 7 Done Sent Done false
8365 3151 Acked Read 2 Done Sent Done false
8366 3152 Acked Write 10 Done Sent Done false
8367 3153 Acked Flush 0 Done Sent Done false
8368 3154 Acked Read 6 Done Sent Done false
8369 3155 Acked Write 9 Done Sent Done false
8370 3156 Acked Read 2 Done Sent Done false
8371 3157 Acked Read 4 Done Sent Done false
8372 3158 Acked Write 3 Done Sent Done false
8373 3159 Acked Write 4 Done Sent Done false
8374 3160 Acked Read 9 Done Sent Done false
8375 3161 Acked Read 4 Done Sent Done false
8376 3162 Acked Read 2 Done Sent Done false
8377 3163 Acked Write 9 Done Sent Done false
8378 3164 Acked Flush 0 Done Sent Done false
8379 3165 Acked Read 10 Done Sent Done false
8380 3166 Acked Write 5 Done Sent Done false
8381 3167 Acked Read 7 Done Sent Done false
8382 3168 Acked Write 1 Done Sent Done false
8383 3169 Acked Write 3 Done Sent Done false
8384 3170 Acked Read 8 Done Sent Done false
8385 3171 Acked Read 6 Done Sent Done false
8386 3172 Acked Read 2 Done Sent Done false
8387 3173 Acked Write 2 Done Sent Done false
8388 3174 Acked Write 2 Done Sent Done false
8389 3175 Acked Read 1 Done Sent Done false
8390 3176 Acked Read 7 Done Sent Done false
8391 3177 Acked Write 3 Done Sent Done false
8392 3178 Acked Read 6 Done Sent Done false
8393 3179 Acked Flush 0 Done Sent Done false
8394 3180 Acked Read 4 Done Sent Done false
8395 3181 Acked Read 9 Done Sent Done false
8396 3182 Acked Write 5 Done Sent Done false
8397 3183 Acked Read 10 Done Sent Done false
8398 3184 Acked Write 5 Done Sent Done false
8399 3185 Acked Flush 0 Done Sent Done false
8400 3186 Acked Read 10 Done Sent Done false
8401 3187 Acked Write 9 Done Sent Done false
8402 3188 Acked Read 7 Done Sent Done false
8403 3189 Acked Write 6 Done Sent Done false
8404 3190 Acked Read 6 Done Sent Done false
8405 3191 Acked Write 4 Done Sent Done false
8406 3192 Acked Read 9 Done Sent Done false
8407 3193 Acked Write 7 Done Sent Done false
8408 3194 Acked Write 3 Done Sent Done false
8409 3195 Acked Write 7 Done Sent Done false
8410 3196 Acked Read 8 Done Sent Done false
8411 3197 Acked Read 10 Done Sent Done false
8412 3198 Acked Flush 0 Done Sent Done false
8413 3199 Acked Read 4 Done Sent Done false
8414 3200 Acked Read 7 Done Sent Done false
8415 3201 Acked Read 8 Done Sent Done false
8416 3202 Acked Write 6 Done Sent Done false
8417 3203 Acked Write 5 Done Sent Done false
8418 3204 Acked Write 2 Done Sent Done false
8419 3205 Acked Read 3 Done Sent Done false
8420 3206 Acked Write 4 Done Sent Done false
8421 3207 Acked Write 9 Done Sent Done false
8422 3208 Acked Write 3 Done Sent Done false
8423 3209 Acked Write 3 Done Sent Done false
8424 3210 Acked Read 4 Done Sent Done false
8425 3211 Acked Read 7 Done Sent Done false
8426 3212 Acked Read 8 Done Sent Done false
8427 3213 Acked Write 2 Done Sent Done false
8428 3214 Acked Write 6 Done Sent Done false
8429 3215 Acked Write 3 Done Sent Done false
8430 3216 Acked Write 2 Done Sent Done false
8431 3217 Acked Write 1 Done Sent Done false
8432 3218 Acked Write 8 Done Sent Done false
8433 3219 Acked Flush 0 Done Sent Done false
8434 3220 Acked Read 6 Done Sent Done false
8435 3221 Acked Read 10 Done Sent Done false
8436 3222 Acked Read 6 Done Sent Done false
8437 3223 Acked Write 2 Done Sent Done false
8438 3224 Acked Write 8 Done Sent Done false
8439 3225 Acked Read 7 Done Sent Done false
8440 3226 Acked Read 1 Done Sent Done false
8441 3227 Acked Read 10 Done Sent Done false
8442 3228 Acked Read 2 Done Sent Done false
8443 3229 Acked Read 7 Done Sent Done false
8444 3230 Acked Read 4 Done Sent Done false
8445 3231 Acked Read 5 Done Sent Done false
8446 3232 Acked Write 4 Done Sent Done false
8447 3233 Acked Write 2 Done Sent Done false
8448 3234 Acked Read 7 Done Sent Done false
8449 3235 Acked Read 10 Done Sent Done false
8450 3236 Acked Read 1 Done Sent Done false
8451 3237 Acked Write 5 Done Sent Done false
8452 3238 Acked Read 8 Done Sent Done false
8453 3239 Acked Write 7 Done Sent Done false
8454 3240 Acked Read 2 Done Sent Done false
8455 3241 Acked Flush 0 Done Sent Done false
8456 3242 Acked Write 2 Done Sent Done false
8457 3243 Acked Read 6 Done Sent Done false
8458 3244 Acked Read 4 Done Sent Done false
8459 3245 Acked Read 4 Done Sent Done false
8460 3246 Acked Read 7 Done Sent Done false
8461 3247 Acked Write 2 Done Sent Done false
8462 3248 Acked Write 4 Done Sent Done false
8463 3249 Acked Read 7 Done Sent Done false
8464 3250 Acked Read 2 Done Sent Done false
8465 3251 Acked Write 4 Done Sent Done false
8466 3252 Acked Write 5 Done Sent Done false
8467 3253 Acked Write 2 Done Sent Done false
8468 3254 Acked Write 1 Done Sent Done false
8469 3255 Acked Read 7 Done Sent Done false
8470 3256 Acked Read 10 Done Sent Done false
8471 3257 Acked Write 1 Done Sent Done false
8472 3258 Acked Flush 0 Done Sent Done false
8473 3259 Acked Read 1 Done Sent Done false
8474 3260 Acked Read 2 Done Sent Done false
8475 3261 Acked Read 2 Done Sent Done false
8476 3262 Acked Write 6 Done Sent Done false
8477 3263 Acked Write 8 Done Sent Done false
8478 3264 Acked Write 7 Done Sent Done false
8479 3265 Acked Write 7 Done Sent Done false
8480 3266 Acked Read 1 Done Sent Done false
8481 3267 Acked Read 9 Done Sent Done false
8482 3268 Acked Write 7 Done Sent Done false
8483 3269 Acked Write 1 Done Sent Done false
8484 3270 Acked Write 6 Done Sent Done false
8485 3271 Acked Write 5 Done Sent Done false
8486 3272 Acked Read 6 Done Sent Done false
8487 3273 Acked Write 5 Done Sent Done false
8488 3274 Acked Read 1 Done Sent Done false
8489 3275 Acked Write 10 Done Sent Done false
8490 3276 Acked Read 6 Done Sent Done false
8491 3277 Acked Read 3 Done Sent Done false
8492 3278 Acked Flush 0 Done Sent Done false
8493 3279 Acked Read 6 Done Sent Done false
8494 3280 Acked Write 4 Done Sent Done false
8495 3281 Acked Read 4 Done Sent Done false
8496 3282 Acked Read 7 Done Sent Done false
8497 3283 Acked Write 2 Done Sent Done false
8498 3284 Acked Read 9 Done Sent Done false
8499 3285 Acked Flush 0 Done Sent Done false
8500 3286 Acked Flush 0 Done Sent Done false
8501 3287 Acked Write 8 Done Sent Done false
8502 3288 Acked Write 10 Done Sent Done false
8503 3289 Acked Read 9 Done Sent Done false
8504 3290 Acked Write 3 Done Sent Done false
8505 3291 Acked Read 4 Done Sent Done false
8506 3292 Acked Read 2 Done Sent Done false
8507 3293 Acked Read 9 Done Sent Done false
8508 3294 Acked Write 9 Done Sent Done false
8509 3295 Acked Write 2 Done Sent Done false
8510 3296 Acked Flush 0 Done Sent Done false
8511 3297 Acked Read 7 Done Sent Done false
8512 3298 Acked Read 3 Done Sent Done false
8513 3299 Acked Write 3 Done Sent Done false
8514 3300 Acked Write 6 Done Sent Done false
8515 3301 Acked Read 8 Done Sent Done false
8516 3302 Acked Read 10 Done Sent Done false
8517 3303 Acked Write 7 Done Sent Done false
8518 3304 Acked Read 7 Done Sent Done false
8519 3305 Acked Write 4 Done Sent Done false
8520 3306 Acked Read 1 Done Sent Done false
8521 3307 Acked Flush 0 Done Sent Done false
8522 3308 Acked Write 6 Done Sent Done false
8523 3309 Acked Read 5 Done Sent Done false
8524 3310 Acked Write 2 Done Sent Done false
8525 3311 Acked Read 3 Done Sent Done false
8526 3312 Acked Write 8 Done Sent Done false
8527 3313 Acked Write 8 Done Sent Done false
8528 3314 Acked Write 5 Done Sent Done false
8529 3315 Acked Write 3 Done Sent Done false
8530 3316 Acked Read 7 Done Sent Done false
8531 3317 Acked Read 1 Done Sent Done false
8532 3318 Acked Read 2 Done Sent Done false
8533 3319 Acked Read 1 Done Sent Done false
8534 3320 Acked Flush 0 Done Sent Done false
8535 3321 Acked Read 3 Done Sent Done false
8536 3322 Acked Read 7 Done Sent Done false
8537 3323 Acked Write 9 Done Sent Done false
8538 3324 Acked Write 10 Done Sent Done false
8539 3325 Acked Read 9 Done Sent Done false
8540 3326 Acked Write 5 Done Sent Done false
8541 3327 Acked Write 9 Done Sent Done false
8542 3328 Acked Read 5 Done Sent Done false
8543 3329 Acked Write 2 Done Sent Done false
8544 3330 Acked Flush 0 Done Sent Done false
8545 3331 Acked Write 8 Done Sent Done false
8546 3332 Acked Read 1 Done Sent Done false
8547 3333 Acked Write 4 Done Sent Done false
8548 3334 Acked Flush 0 Done Sent Done false
8549 3335 Acked Read 9 Done Sent Done false
8550 3336 Acked Write 9 Done Sent Done false
8551 3337 Acked Read 6 Done Sent Done false
8552 3338 Acked Read 5 Done Sent Done false
8553 3339 Acked Write 4 Done Sent Done false
8554 3340 Acked Read 5 Done Sent Done false
8555 3341 Acked Write 5 Done Sent Done false
8556 3342 Acked Write 8 Done Sent Done false
8557 3343 Acked Read 10 Done Sent Done false
8558 3344 Acked Read 10 Done Sent Done false
8559 3345 Acked Write 10 Done Sent Done false
8560 3346 Acked Read 7 Done Sent Done false
8561 3347 Acked Write 6 Done Sent Done false
8562 3348 Acked Read 10 Done Sent Done false
8563 3349 Acked Write 9 Done Sent Done false
8564 3350 Acked Read 5 Done Sent Done false
8565 3351 Acked Read 7 Done Sent Done false
8566 3352 Acked Read 6 Done Sent Done false
8567 3353 Acked Read 7 Done Sent Done false
8568 3354 Acked Read 3 Done Sent Done false
8569 3355 Acked Write 6 Done Sent Done false
8570 3356 Acked Read 6 Done Sent Done false
8571 3357 Acked Read 8 Done Sent Done false
8572 3358 Acked Write 9 Done Sent Done false
8573 3359 Acked Write 8 Done Sent Done false
8574 3360 Acked Write 8 Done Sent Done false
8575 3361 Acked Read 6 Done Sent Done false
8576 3362 Acked Read 7 Done Sent Done false
8577 3363 Acked Write 5 Done Sent Done false
8578 3364 Acked Write 9 Done Sent Done false
8579 3365 Acked Read 4 Done Sent Done false
8580 3366 Acked Flush 0 Done Sent Done false
8581 3367 Acked Read 2 Done Sent Done false
8582 3368 Acked Read 1 Done Sent Done false
8583 3369 Acked Write 8 Done Sent Done false
8584 3370 Acked Read 2 Done Sent Done false
8585 3371 Acked Write 8 Done Sent Done false
8586 3372 Acked Write 3 Done Sent Done false
8587 3373 Acked Read 2 Done Sent Done false
8588 3374 Acked Write 2 Done Sent Done false
8589 3375 Acked Read 1 Done Sent Done false
8590 3376 Acked Read 10 Done Sent Done false
8591 3377 Acked Read 2 Done Sent Done false
8592 3378 Acked Read 5 Done Sent Done false
8593 3379 Acked Write 7 Done Sent Done false
8594 3380 Acked Read 2 Done Sent Done false
8595 3381 Acked Read 1 Done Sent Done false
8596 3382 Acked Read 3 Done Sent Done false
8597 3383 Acked Read 1 Done Sent Done false
8598 3384 Acked Read 7 Done Sent Done false
8599 3385 Acked Read 5 Done Sent Done false
8600 3386 Acked Write 9 Done Sent Done false
8601 3387 Acked Read 8 Done Sent Done false
8602 3388 Acked Read 8 Done Sent Done false
8603 3389 Acked Read 7 Done Sent Done false
8604 3390 Acked Write 1 Done Sent Done false
8605 3391 Acked Write 6 Done Sent Done false
8606 3392 Acked Write 2 Done Sent Done false
8607 3393 Acked Write 7 Done Sent Done false
8608 3394 Acked Write 5 Done Sent Done false
8609 3395 Acked Read 9 Done Sent Done false
8610 3396 Acked Write 1 Done Sent Done false
8611 3397 Acked Write 8 Done Sent Done false
8612 3398 Acked Write 7 Done Sent Done false
8613 3399 Acked Read 5 Done Sent Done false
8614 3400 Acked Read 10 Done Sent Done false
8615 3401 Acked Read 6 Done Sent Done false
8616 3402 Acked Read 5 Done Sent Done false
8617 3403 Acked Flush 0 Done Sent Done false
8618 3404 Acked Read 4 Done Sent Done false
8619 3405 Acked Write 3 Done Sent Done false
8620 3406 Acked Read 7 Done Sent Done false
8621 3407 Acked Write 3 Done Sent Done false
8622 3408 Acked Read 1 Done Sent Done false
8623 3409 Acked Read 4 Done Sent Done false
8624 3410 Acked Read 5 Done Sent Done false
8625 3411 Acked Write 3 Done Sent Done false
8626 3412 Acked Read 5 Done Sent Done false
8627 3413 Acked Read 4 Done Sent Done false
8628 3414 Acked Flush 0 Done Sent Done false
8629 STATES DS:0 DS:1 DS:2 TOTAL
8630 Sent 0 302 0 302
8631 Done 302 0 302 604
8632 Skipped 0 0 0 0
8633 Error 0 0 0 0
8634 Last Flush: 3414 3112 3414
8635 Downstairs last five completed: 3112 3111 3110 3109 3108
8636 Upstairs last five completed: 3414 3413 3412 3411 3410
8637 CLIENT: Up:0 ds:302 act:2
86382024-11-19T17:09:47.431ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
86392024-11-19T17:09:47.431ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
86402024-11-19T17:09:47.431ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
86412024-11-19T17:09:47.431ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 05650a15-902e-469c-91e5-cba45f07470e
86422024-11-19T17:09:47.431ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
8643 ----------------------------------------------------------------
8644 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8645 States: Active Offline Active
8646 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8647 3113 Acked Write 1 Done Sent Done false
8648 3114 Acked Read 6 Done Sent Done false
8649 3115 Acked Read 1 Done Sent Done false
8650 3116 Acked Read 5 Done Sent Done false
8651 3117 Acked Read 1 Done Sent Done false
8652 3118 Acked Flush 0 Done Sent Done false
8653 3119 Acked Flush 0 Done Sent Done false
8654 3120 Acked Write 6 Done Sent Done false
8655 3121 Acked Read 10 Done Sent Done false
8656 3122 Acked Flush 0 Done Sent Done false
8657 3123 Acked Read 7 Done Sent Done false
8658 3124 Acked Write 6 Done Sent Done false
8659 3125 Acked Write 3 Done Sent Done false
8660 3126 Acked Read 7 Done Sent Done false
8661 3127 Acked Write 9 Done Sent Done false
8662 3128 Acked Flush 0 Done Sent Done false
8663 3129 Acked Write 2 Done Sent Done false
8664 3130 Acked Read 4 Done Sent Done false
8665 3131 Acked Read 1 Done Sent Done false
8666 3132 Acked Flush 0 Done Sent Done false
8667 3133 Acked Read 4 Done Sent Done false
8668 3134 Acked Write 10 Done Sent Done false
8669 3135 Acked Read 3 Done Sent Done false
8670 3136 Acked Read 8 Done Sent Done false
8671 3137 Acked Write 5 Done Sent Done false
8672 3138 Acked Read 3 Done Sent Done false
8673 3139 Acked Write 3 Done Sent Done false
8674 3140 Acked Flush 0 Done Sent Done false
8675 3141 Acked Read 4 Done Sent Done false
8676 3142 Acked Read 6 Done Sent Done false
8677 3143 Acked Read 5 Done Sent Done false
8678 3144 Acked Flush 0 Done Sent Done false
8679 3145 Acked Read 2 Done Sent Done false
8680 3146 Acked Flush 0 Done Sent Done false
8681 3147 Acked Read 3 Done Sent Done false
8682 3148 Acked Write 8 Done Sent Done false
8683 3149 Acked Read 9 Done Sent Done false
8684 3150 Acked Read 7 Done Sent Done false
8685 3151 Acked Read 2 Done Sent Done false
8686 3152 Acked Write 10 Done Sent Done false
8687 3153 Acked Flush 0 Done Sent Done false
8688 3154 Acked Read 6 Done Sent Done false
8689 3155 Acked Write 9 Done Sent Done false
8690 3156 Acked Read 2 Done Sent Done false
8691 3157 Acked Read 4 Done Sent Done false
8692 3158 Acked Write 3 Done Sent Done false
8693 3159 Acked Write 4 Done Sent Done false
8694 3160 Acked Read 9 Done Sent Done false
8695 3161 Acked Read 4 Done Sent Done false
8696 3162 Acked Read 2 Done Sent Done false
8697 3163 Acked Write 9 Done Sent Done false
8698 3164 Acked Flush 0 Done Sent Done false
8699 3165 Acked Read 10 Done Sent Done false
8700 3166 Acked Write 5 Done Sent Done false
8701 3167 Acked Read 7 Done Sent Done false
8702 3168 Acked Write 1 Done Sent Done false
8703 3169 Acked Write 3 Done Sent Done false
8704 3170 Acked Read 8 Done Sent Done false
8705 3171 Acked Read 6 Done Sent Done false
8706 3172 Acked Read 2 Done Sent Done false
8707 3173 Acked Write 2 Done Sent Done false
8708 3174 Acked Write 2 Done Sent Done false
8709 3175 Acked Read 1 Done Sent Done false
8710 3176 Acked Read 7 Done Sent Done false
8711 3177 Acked Write 3 Done Sent Done false
8712 3178 Acked Read 6 Done Sent Done false
8713 3179 Acked Flush 0 Done Sent Done false
8714 3180 Acked Read 4 Done Sent Done false
8715 3181 Acked Read 9 Done Sent Done false
8716 3182 Acked Write 5 Done Sent Done false
8717 3183 Acked Read 10 Done Sent Done false
8718 3184 Acked Write 5 Done Sent Done false
8719 3185 Acked Flush 0 Done Sent Done false
8720 3186 Acked Read 10 Done Sent Done false
8721 3187 Acked Write 9 Done Sent Done false
8722 3188 Acked Read 7 Done Sent Done false
8723 3189 Acked Write 6 Done Sent Done false
8724 3190 Acked Read 6 Done Sent Done false
8725 3191 Acked Write 4 Done Sent Done false
8726 3192 Acked Read 9 Done Sent Done false
8727 3193 Acked Write 7 Done Sent Done false
8728 3194 Acked Write 3 Done Sent Done false
8729 3195 Acked Write 7 Done Sent Done false
8730 3196 Acked Read 8 Done Sent Done false
8731 3197 Acked Read 10 Done Sent Done false
8732 3198 Acked Flush 0 Done Sent Done false
8733 3199 Acked Read 4 Done Sent Done false
8734 3200 Acked Read 7 Done Sent Done false
8735 3201 Acked Read 8 Done Sent Done false
8736 3202 Acked Write 6 Done Sent Done false
8737 3203 Acked Write 5 Done Sent Done false
8738 3204 Acked Write 2 Done Sent Done false
8739 3205 Acked Read 3 Done Sent Done false
8740 3206 Acked Write 4 Done Sent Done false
8741 3207 Acked Write 9 Done Sent Done false
8742 3208 Acked Write 3 Done Sent Done false
8743 3209 Acked Write 3 Done Sent Done false
8744 3210 Acked Read 4 Done Sent Done false
8745 3211 Acked Read 7 Done Sent Done false
8746 3212 Acked Read 8 Done Sent Done false
8747 3213 Acked Write 2 Done Sent Done false
8748 3214 Acked Write 6 Done Sent Done false
8749 3215 Acked Write 3 Done Sent Done false
8750 3216 Acked Write 2 Done Sent Done false
8751 3217 Acked Write 1 Done Sent Done false
8752 3218 Acked Write 8 Done Sent Done false
8753 3219 Acked Flush 0 Done Sent Done false
8754 3220 Acked Read 6 Done Sent Done false
8755 3221 Acked Read 10 Done Sent Done false
8756 3222 Acked Read 6 Done Sent Done false
8757 3223 Acked Write 2 Done Sent Done false
8758 3224 Acked Write 8 Done Sent Done false
8759 3225 Acked Read 7 Done Sent Done false
8760 3226 Acked Read 1 Done Sent Done false
8761 3227 Acked Read 10 Done Sent Done false
8762 3228 Acked Read 2 Done Sent Done false
8763 3229 Acked Read 7 Done Sent Done false
8764 3230 Acked Read 4 Done Sent Done false
8765 3231 Acked Read 5 Done Sent Done false
8766 3232 Acked Write 4 Done Sent Done false
8767 3233 Acked Write 2 Done Sent Done false
8768 3234 Acked Read 7 Done Sent Done false
8769 3235 Acked Read 10 Done Sent Done false
8770 3236 Acked Read 1 Done Sent Done false
8771 3237 Acked Write 5 Done Sent Done false
8772 3238 Acked Read 8 Done Sent Done false
8773 3239 Acked Write 7 Done Sent Done false
8774 3240 Acked Read 2 Done Sent Done false
8775 3241 Acked Flush 0 Done Sent Done false
8776 3242 Acked Write 2 Done Sent Done false
8777 3243 Acked Read 6 Done Sent Done false
8778 3244 Acked Read 4 Done Sent Done false
8779 3245 Acked Read 4 Done Sent Done false
8780 3246 Acked Read 7 Done Sent Done false
8781 3247 Acked Write 2 Done Sent Done false
8782 3248 Acked Write 4 Done Sent Done false
8783 3249 Acked Read 7 Done Sent Done false
8784 3250 Acked Read 2 Done Sent Done false
8785 3251 Acked Write 4 Done Sent Done false
8786 3252 Acked Write 5 Done Sent Done false
8787 3253 Acked Write 2 Done Sent Done false
8788 3254 Acked Write 1 Done Sent Done false
8789 3255 Acked Read 7 Done Sent Done false
8790 3256 Acked Read 10 Done Sent Done false
8791 3257 Acked Write 1 Done Sent Done false
8792 3258 Acked Flush 0 Done Sent Done false
8793 3259 Acked Read 1 Done Sent Done false
8794 3260 Acked Read 2 Done Sent Done false
8795 3261 Acked Read 2 Done Sent Done false
8796 3262 Acked Write 6 Done Sent Done false
8797 3263 Acked Write 8 Done Sent Done false
8798 3264 Acked Write 7 Done Sent Done false
8799 3265 Acked Write 7 Done Sent Done false
8800 3266 Acked Read 1 Done Sent Done false
8801 3267 Acked Read 9 Done Sent Done false
8802 3268 Acked Write 7 Done Sent Done false
8803 3269 Acked Write 1 Done Sent Done false
8804 3270 Acked Write 6 Done Sent Done false
8805 3271 Acked Write 5 Done Sent Done false
8806 3272 Acked Read 6 Done Sent Done false
8807 3273 Acked Write 5 Done Sent Done false
8808 3274 Acked Read 1 Done Sent Done false
8809 3275 Acked Write 10 Done Sent Done false
8810 3276 Acked Read 6 Done Sent Done false
8811 3277 Acked Read 3 Done Sent Done false
8812 3278 Acked Flush 0 Done Sent Done false
8813 3279 Acked Read 6 Done Sent Done false
8814 3280 Acked Write 4 Done Sent Done false
8815 3281 Acked Read 4 Done Sent Done false
8816 3282 Acked Read 7 Done Sent Done false
8817 3283 Acked Write 2 Done Sent Done false
8818 3284 Acked Read 9 Done Sent Done false
8819 3285 Acked Flush 0 Done Sent Done false
8820 3286 Acked Flush 0 Done Sent Done false
8821 3287 Acked Write 8 Done Sent Done false
8822 3288 Acked Write 10 Done Sent Done false
8823 3289 Acked Read 9 Done Sent Done false
8824 3290 Acked Write 3 Done Sent Done false
8825 3291 Acked Read 4 Done Sent Done false
8826 3292 Acked Read 2 Done Sent Done false
8827 3293 Acked Read 9 Done Sent Done false
8828 3294 Acked Write 9 Done Sent Done false
8829 3295 Acked Write 2 Done Sent Done false
8830 3296 Acked Flush 0 Done Sent Done false
8831 3297 Acked Read 7 Done Sent Done false
8832 3298 Acked Read 3 Done Sent Done false
8833 3299 Acked Write 3 Done Sent Done false
8834 3300 Acked Write 6 Done Sent Done false
8835 3301 Acked Read 8 Done Sent Done false
8836 3302 Acked Read 10 Done Sent Done false
8837 3303 Acked Write 7 Done Sent Done false
8838 3304 Acked Read 7 Done Sent Done false
8839 3305 Acked Write 4 Done Sent Done false
8840 3306 Acked Read 1 Done Sent Done false
8841 3307 Acked Flush 0 Done Sent Done false
8842 3308 Acked Write 6 Done Sent Done false
8843 3309 Acked Read 5 Done Sent Done false
8844 3310 Acked Write 2 Done Sent Done false
8845 3311 Acked Read 3 Done Sent Done false
8846 3312 Acked Write 8 Done Sent Done false
8847 3313 Acked Write 8 Done Sent Done false
8848 3314 Acked Write 5 Done Sent Done false
8849 3315 Acked Write 3 Done Sent Done false
8850 3316 Acked Read 7 Done Sent Done false
8851 3317 Acked Read 1 Done Sent Done false
8852 3318 Acked Read 2 Done Sent Done false
8853 3319 Acked Read 1 Done Sent Done false
8854 3320 Acked Flush 0 Done Sent Done false
8855 3321 Acked Read 3 Done Sent Done false
8856 3322 Acked Read 7 Done Sent Done false
8857 3323 Acked Write 9 Done Sent Done false
8858 3324 Acked Write 10 Done Sent Done false
8859 3325 Acked Read 9 Done Sent Done false
8860 3326 Acked Write 5 Done Sent Done false
8861 3327 Acked Write 9 Done Sent Done false
8862 3328 Acked Read 5 Done Sent Done false
8863 3329 Acked Write 2 Done Sent Done false
8864 3330 Acked Flush 0 Done Sent Done false
8865 3331 Acked Write 8 Done Sent Done false
8866 3332 Acked Read 1 Done Sent Done false
8867 3333 Acked Write 4 Done Sent Done false
8868 3334 Acked Flush 0 Done Sent Done false
8869 3335 Acked Read 9 Done Sent Done false
8870 3336 Acked Write 9 Done Sent Done false
8871 3337 Acked Read 6 Done Sent Done false
8872 3338 Acked Read 5 Done Sent Done false
8873 3339 Acked Write 4 Done Sent Done false
8874 3340 Acked Read 5 Done Sent Done false
8875 3341 Acked Write 5 Done Sent Done false
8876 3342 Acked Write 8 Done Sent Done false
8877 3343 Acked Read 10 Done Sent Done false
8878 3344 Acked Read 10 Done Sent Done false
8879 3345 Acked Write 10 Done Sent Done false
8880 3346 Acked Read 7 Done Sent Done false
8881 3347 Acked Write 6 Done Sent Done false
8882 3348 Acked Read 10 Done Sent Done false
8883 3349 Acked Write 9 Done Sent Done false
8884 3350 Acked Read 5 Done Sent Done false
8885 3351 Acked Read 7 Done Sent Done false
8886 3352 Acked Read 6 Done Sent Done false
8887 3353 Acked Read 7 Done Sent Done false
8888 3354 Acked Read 3 Done Sent Done false
8889 3355 Acked Write 6 Done Sent Done false
8890 3356 Acked Read 6 Done Sent Done false
8891 3357 Acked Read 8 Done Sent Done false
8892 3358 Acked Write 9 Done Sent Done false
8893 3359 Acked Write 8 Done Sent Done false
8894 3360 Acked Write 8 Done Sent Done false
8895 3361 Acked Read 6 Done Sent Done false
8896 3362 Acked Read 7 Done Sent Done false
8897 3363 Acked Write 5 Done Sent Done false
8898 3364 Acked Write 9 Done Sent Done false
8899 3365 Acked Read 4 Done Sent Done false
8900 3366 Acked Flush 0 Done Sent Done false
8901 3367 Acked Read 2 Done Sent Done false
8902 3368 Acked Read 1 Done Sent Done false
8903 3369 Acked Write 8 Done Sent Done false
8904 3370 Acked Read 2 Done Sent Done false
8905 3371 Acked Write 8 Done Sent Done false
8906 3372 Acked Write 3 Done Sent Done false
8907 3373 Acked Read 2 Done Sent Done false
8908 3374 Acked Write 2 Done Sent Done false
8909 3375 Acked Read 1 Done Sent Done false
8910 3376 Acked Read 10 Done Sent Done false
8911 3377 Acked Read 2 Done Sent Done false
8912 3378 Acked Read 5 Done Sent Done false
8913 3379 Acked Write 7 Done Sent Done false
8914 3380 Acked Read 2 Done Sent Done false
8915 3381 Acked Read 1 Done Sent Done false
8916 3382 Acked Read 3 Done Sent Done false
8917 3383 Acked Read 1 Done Sent Done false
8918 3384 Acked Read 7 Done Sent Done false
8919 3385 Acked Read 5 Done Sent Done false
8920 3386 Acked Write 9 Done Sent Done false
8921 3387 Acked Read 8 Done Sent Done false
8922 3388 Acked Read 8 Done Sent Done false
8923 3389 Acked Read 7 Done Sent Done false
8924 3390 Acked Write 1 Done Sent Done false
8925 3391 Acked Write 6 Done Sent Done false
8926 3392 Acked Write 2 Done Sent Done false
8927 3393 Acked Write 7 Done Sent Done false
8928 3394 Acked Write 5 Done Sent Done false
8929 3395 Acked Read 9 Done Sent Done false
8930 3396 Acked Write 1 Done Sent Done false
8931 3397 Acked Write 8 Done Sent Done false
8932 3398 Acked Write 7 Done Sent Done false
8933 3399 Acked Read 5 Done Sent Done false
8934 3400 Acked Read 10 Done Sent Done false
8935 3401 Acked Read 6 Done Sent Done false
8936 3402 Acked Read 5 Done Sent Done false
8937 3403 Acked Flush 0 Done Sent Done false
8938 3404 Acked Read 4 Done Sent Done false
8939 3405 Acked Write 3 Done Sent Done false
8940 3406 Acked Read 7 Done Sent Done false
8941 3407 Acked Write 3 Done Sent Done false
8942 3408 Acked Read 1 Done Sent Done false
8943 3409 Acked Read 4 Done Sent Done false
8944 3410 Acked Read 5 Done Sent Done false
8945 3411 Acked Write 3 Done Sent Done false
8946 3412 Acked Read 5 Done Sent Done false
8947 3413 Acked Read 4 Done Sent Done false
8948 3414 Acked Flush 0 Done Sent Done false
8949 STATES DS:0 DS:1 DS:2 TOTAL
8950 Sent 0 302 0 302
8951 Done 302 0 302 604
8952 Skipped 0 0 0 0
8953 Error 0 0 0 0
8954 Last Flush: 3414 3112 3414
8955 Downstairs last five completed: 3112 3111 3110 3109 3108
8956 Upstairs last five completed: 3414 3413 3412 3411 3410
8957 CLIENT: Up:0 ds:302 act:2
8958 [8820] got start action
8959 [8820] I am going to restart
8960 Starting downstairs at port 8820
8961 Make output file at "/tmp/dsc/downstairs-8820.txt"
8962 [8820][1] reports Starting
8963 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1210)
8964 [8820][1] reports Running
8965 ----------------------------------------------------------------
8966 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8967 States: Active Offline Active
8968 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8969 3113 Acked Write 1 Done Sent Done false
8970 3114 Acked Read 6 Done Sent Done false
8971 3115 Acked Read 1 Done Sent Done false
8972 3116 Acked Read 5 Done Sent Done false
8973 3117 Acked Read 1 Done Sent Done false
8974 3118 Acked Flush 0 Done Sent Done false
8975 3119 Acked Flush 0 Done Sent Done false
8976 3120 Acked Write 6 Done Sent Done false
8977 3121 Acked Read 10 Done Sent Done false
8978 3122 Acked Flush 0 Done Sent Done false
8979 3123 Acked Read 7 Done Sent Done false
8980 3124 Acked Write 6 Done Sent Done false
8981 3125 Acked Write 3 Done Sent Done false
8982 3126 Acked Read 7 Done Sent Done false
8983 3127 Acked Write 9 Done Sent Done false
8984 3128 Acked Flush 0 Done Sent Done false
8985 3129 Acked Write 2 Done Sent Done false
8986 3130 Acked Read 4 Done Sent Done false
8987 3131 Acked Read 1 Done Sent Done false
8988 3132 Acked Flush 0 Done Sent Done false
8989 3133 Acked Read 4 Done Sent Done false
8990 3134 Acked Write 10 Done Sent Done false
8991 3135 Acked Read 3 Done Sent Done false
8992 3136 Acked Read 8 Done Sent Done false
8993 3137 Acked Write 5 Done Sent Done false
8994 3138 Acked Read 3 Done Sent Done false
8995 3139 Acked Write 3 Done Sent Done false
8996 3140 Acked Flush 0 Done Sent Done false
8997 3141 Acked Read 4 Done Sent Done false
8998 3142 Acked Read 6 Done Sent Done false
8999 3143 Acked Read 5 Done Sent Done false
9000 3144 Acked Flush 0 Done Sent Done false
9001 3145 Acked Read 2 Done Sent Done false
9002 3146 Acked Flush 0 Done Sent Done false
9003 3147 Acked Read 3 Done Sent Done false
9004 3148 Acked Write 8 Done Sent Done false
9005 3149 Acked Read 9 Done Sent Done false
9006 3150 Acked Read 7 Done Sent Done false
9007 3151 Acked Read 2 Done Sent Done false
9008 3152 Acked Write 10 Done Sent Done false
9009 3153 Acked Flush 0 Done Sent Done false
9010 3154 Acked Read 6 Done Sent Done false
9011 3155 Acked Write 9 Done Sent Done false
9012 3156 Acked Read 2 Done Sent Done false
9013 3157 Acked Read 4 Done Sent Done false
9014 3158 Acked Write 3 Done Sent Done false
9015 3159 Acked Write 4 Done Sent Done false
9016 3160 Acked Read 9 Done Sent Done false
9017 3161 Acked Read 4 Done Sent Done false
9018 3162 Acked Read 2 Done Sent Done false
9019 3163 Acked Write 9 Done Sent Done false
9020 3164 Acked Flush 0 Done Sent Done false
9021 3165 Acked Read 10 Done Sent Done false
9022 3166 Acked Write 5 Done Sent Done false
9023 3167 Acked Read 7 Done Sent Done false
9024 3168 Acked Write 1 Done Sent Done false
9025 3169 Acked Write 3 Done Sent Done false
9026 3170 Acked Read 8 Done Sent Done false
9027 3171 Acked Read 6 Done Sent Done false
9028 3172 Acked Read 2 Done Sent Done false
9029 3173 Acked Write 2 Done Sent Done false
9030 3174 Acked Write 2 Done Sent Done false
9031 3175 Acked Read 1 Done Sent Done false
9032 3176 Acked Read 7 Done Sent Done false
9033 3177 Acked Write 3 Done Sent Done false
9034 3178 Acked Read 6 Done Sent Done false
9035 3179 Acked Flush 0 Done Sent Done false
9036 3180 Acked Read 4 Done Sent Done false
9037 3181 Acked Read 9 Done Sent Done false
9038 3182 Acked Write 5 Done Sent Done false
9039 3183 Acked Read 10 Done Sent Done false
9040 3184 Acked Write 5 Done Sent Done false
9041 3185 Acked Flush 0 Done Sent Done false
9042 3186 Acked Read 10 Done Sent Done false
9043 3187 Acked Write 9 Done Sent Done false
9044 3188 Acked Read 7 Done Sent Done false
9045 3189 Acked Write 6 Done Sent Done false
9046 3190 Acked Read 6 Done Sent Done false
9047 3191 Acked Write 4 Done Sent Done false
9048 3192 Acked Read 9 Done Sent Done false
9049 3193 Acked Write 7 Done Sent Done false
9050 3194 Acked Write 3 Done Sent Done false
9051 3195 Acked Write 7 Done Sent Done false
9052 3196 Acked Read 8 Done Sent Done false
9053 3197 Acked Read 10 Done Sent Done false
9054 3198 Acked Flush 0 Done Sent Done false
9055 3199 Acked Read 4 Done Sent Done false
9056 3200 Acked Read 7 Done Sent Done false
9057 3201 Acked Read 8 Done Sent Done false
9058 3202 Acked Write 6 Done Sent Done false
9059 3203 Acked Write 5 Done Sent Done false
9060 3204 Acked Write 2 Done Sent Done false
9061 3205 Acked Read 3 Done Sent Done false
9062 3206 Acked Write 4 Done Sent Done false
9063 3207 Acked Write 9 Done Sent Done false
9064 3208 Acked Write 3 Done Sent Done false
9065 3209 Acked Write 3 Done Sent Done false
9066 3210 Acked Read 4 Done Sent Done false
9067 3211 Acked Read 7 Done Sent Done false
9068 3212 Acked Read 8 Done Sent Done false
9069 3213 Acked Write 2 Done Sent Done false
9070 3214 Acked Write 6 Done Sent Done false
9071 3215 Acked Write 3 Done Sent Done false
9072 3216 Acked Write 2 Done Sent Done false
9073 3217 Acked Write 1 Done Sent Done false
9074 3218 Acked Write 8 Done Sent Done false
9075 3219 Acked Flush 0 Done Sent Done false
9076 3220 Acked Read 6 Done Sent Done false
9077 3221 Acked Read 10 Done Sent Done false
9078 3222 Acked Read 6 Done Sent Done false
9079 3223 Acked Write 2 Done Sent Done false
9080 3224 Acked Write 8 Done Sent Done false
9081 3225 Acked Read 7 Done Sent Done false
9082 3226 Acked Read 1 Done Sent Done false
9083 3227 Acked Read 10 Done Sent Done false
9084 3228 Acked Read 2 Done Sent Done false
9085 3229 Acked Read 7 Done Sent Done false
9086 3230 Acked Read 4 Done Sent Done false
9087 3231 Acked Read 5 Done Sent Done false
9088 3232 Acked Write 4 Done Sent Done false
9089 3233 Acked Write 2 Done Sent Done false
9090 3234 Acked Read 7 Done Sent Done false
9091 3235 Acked Read 10 Done Sent Done false
9092 3236 Acked Read 1 Done Sent Done false
9093 3237 Acked Write 5 Done Sent Done false
9094 3238 Acked Read 8 Done Sent Done false
9095 3239 Acked Write 7 Done Sent Done false
9096 3240 Acked Read 2 Done Sent Done false
9097 3241 Acked Flush 0 Done Sent Done false
9098 3242 Acked Write 2 Done Sent Done false
9099 3243 Acked Read 6 Done Sent Done false
9100 3244 Acked Read 4 Done Sent Done false
9101 3245 Acked Read 4 Done Sent Done false
9102 3246 Acked Read 7 Done Sent Done false
9103 3247 Acked Write 2 Done Sent Done false
9104 3248 Acked Write 4 Done Sent Done false
9105 3249 Acked Read 7 Done Sent Done false
9106 3250 Acked Read 2 Done Sent Done false
9107 3251 Acked Write 4 Done Sent Done false
9108 3252 Acked Write 5 Done Sent Done false
9109 3253 Acked Write 2 Done Sent Done false
9110 3254 Acked Write 1 Done Sent Done false
9111 3255 Acked Read 7 Done Sent Done false
9112 3256 Acked Read 10 Done Sent Done false
9113 3257 Acked Write 1 Done Sent Done false
9114 3258 Acked Flush 0 Done Sent Done false
9115 3259 Acked Read 1 Done Sent Done false
9116 3260 Acked Read 2 Done Sent Done false
9117 3261 Acked Read 2 Done Sent Done false
9118 3262 Acked Write 6 Done Sent Done false
9119 3263 Acked Write 8 Done Sent Done false
9120 3264 Acked Write 7 Done Sent Done false
9121 3265 Acked Write 7 Done Sent Done false
9122 3266 Acked Read 1 Done Sent Done false
9123 3267 Acked Read 9 Done Sent Done false
9124 3268 Acked Write 7 Done Sent Done false
9125 3269 Acked Write 1 Done Sent Done false
9126 3270 Acked Write 6 Done Sent Done false
9127 3271 Acked Write 5 Done Sent Done false
9128 3272 Acked Read 6 Done Sent Done false
9129 3273 Acked Write 5 Done Sent Done false
9130 3274 Acked Read 1 Done Sent Done false
9131 3275 Acked Write 10 Done Sent Done false
9132 3276 Acked Read 6 Done Sent Done false
9133 3277 Acked Read 3 Done Sent Done false
9134 3278 Acked Flush 0 Done Sent Done false
9135 3279 Acked Read 6 Done Sent Done false
9136 3280 Acked Write 4 Done Sent Done false
9137 3281 Acked Read 4 Done Sent Done false
9138 3282 Acked Read 7 Done Sent Done false
9139 3283 Acked Write 2 Done Sent Done false
9140 3284 Acked Read 9 Done Sent Done false
9141 3285 Acked Flush 0 Done Sent Done false
9142 3286 Acked Flush 0 Done Sent Done false
9143 3287 Acked Write 8 Done Sent Done false
9144 3288 Acked Write 10 Done Sent Done false
9145 3289 Acked Read 9 Done Sent Done false
9146 3290 Acked Write 3 Done Sent Done false
9147 3291 Acked Read 4 Done Sent Done false
9148 3292 Acked Read 2 Done Sent Done false
9149 3293 Acked Read 9 Done Sent Done false
9150 3294 Acked Write 9 Done Sent Done false
9151 3295 Acked Write 2 Done Sent Done false
9152 3296 Acked Flush 0 Done Sent Done false
9153 3297 Acked Read 7 Done Sent Done false
9154 3298 Acked Read 3 Done Sent Done false
9155 3299 Acked Write 3 Done Sent Done false
9156 3300 Acked Write 6 Done Sent Done false
9157 3301 Acked Read 8 Done Sent Done false
9158 3302 Acked Read 10 Done Sent Done false
9159 3303 Acked Write 7 Done Sent Done false
9160 3304 Acked Read 7 Done Sent Done false
9161 3305 Acked Write 4 Done Sent Done false
9162 3306 Acked Read 1 Done Sent Done false
9163 3307 Acked Flush 0 Done Sent Done false
9164 3308 Acked Write 6 Done Sent Done false
9165 3309 Acked Read 5 Done Sent Done false
9166 3310 Acked Write 2 Done Sent Done false
9167 3311 Acked Read 3 Done Sent Done false
9168 3312 Acked Write 8 Done Sent Done false
9169 3313 Acked Write 8 Done Sent Done false
9170 3314 Acked Write 5 Done Sent Done false
9171 3315 Acked Write 3 Done Sent Done false
9172 3316 Acked Read 7 Done Sent Done false
9173 3317 Acked Read 1 Done Sent Done false
9174 3318 Acked Read 2 Done Sent Done false
9175 3319 Acked Read 1 Done Sent Done false
9176 3320 Acked Flush 0 Done Sent Done false
9177 3321 Acked Read 3 Done Sent Done false
9178 3322 Acked Read 7 Done Sent Done false
9179 3323 Acked Write 9 Done Sent Done false
9180 3324 Acked Write 10 Done Sent Done false
9181 3325 Acked Read 9 Done Sent Done false
9182 3326 Acked Write 5 Done Sent Done false
9183 3327 Acked Write 9 Done Sent Done false
9184 3328 Acked Read 5 Done Sent Done false
9185 3329 Acked Write 2 Done Sent Done false
9186 3330 Acked Flush 0 Done Sent Done false
9187 3331 Acked Write 8 Done Sent Done false
9188 3332 Acked Read 1 Done Sent Done false
9189 3333 Acked Write 4 Done Sent Done false
9190 3334 Acked Flush 0 Done Sent Done false
9191 3335 Acked Read 9 Done Sent Done false
9192 3336 Acked Write 9 Done Sent Done false
9193 3337 Acked Read 6 Done Sent Done false
9194 3338 Acked Read 5 Done Sent Done false
9195 3339 Acked Write 4 Done Sent Done false
9196 3340 Acked Read 5 Done Sent Done false
9197 3341 Acked Write 5 Done Sent Done false
9198 3342 Acked Write 8 Done Sent Done false
9199 3343 Acked Read 10 Done Sent Done false
9200 3344 Acked Read 10 Done Sent Done false
9201 3345 Acked Write 10 Done Sent Done false
9202 3346 Acked Read 7 Done Sent Done false
9203 3347 Acked Write 6 Done Sent Done false
9204 3348 Acked Read 10 Done Sent Done false
9205 3349 Acked Write 9 Done Sent Done false
9206 3350 Acked Read 5 Done Sent Done false
9207 3351 Acked Read 7 Done Sent Done false
9208 3352 Acked Read 6 Done Sent Done false
9209 3353 Acked Read 7 Done Sent Done false
9210 3354 Acked Read 3 Done Sent Done false
9211 3355 Acked Write 6 Done Sent Done false
9212 3356 Acked Read 6 Done Sent Done false
9213 3357 Acked Read 8 Done Sent Done false
9214 3358 Acked Write 9 Done Sent Done false
9215 3359 Acked Write 8 Done Sent Done false
9216 3360 Acked Write 8 Done Sent Done false
9217 3361 Acked Read 6 Done Sent Done false
9218 3362 Acked Read 7 Done Sent Done false
9219 3363 Acked Write 5 Done Sent Done false
9220 3364 Acked Write 9 Done Sent Done false
9221 3365 Acked Read 4 Done Sent Done false
9222 3366 Acked Flush 0 Done Sent Done false
9223 3367 Acked Read 2 Done Sent Done false
9224 3368 Acked Read 1 Done Sent Done false
9225 3369 Acked Write 8 Done Sent Done false
9226 3370 Acked Read 2 Done Sent Done false
9227 3371 Acked Write 8 Done Sent Done false
9228 3372 Acked Write 3 Done Sent Done false
9229 3373 Acked Read 2 Done Sent Done false
9230 3374 Acked Write 2 Done Sent Done false
9231 3375 Acked Read 1 Done Sent Done false
9232 3376 Acked Read 10 Done Sent Done false
9233 3377 Acked Read 2 Done Sent Done false
9234 3378 Acked Read 5 Done Sent Done false
9235 3379 Acked Write 7 Done Sent Done false
9236 3380 Acked Read 2 Done Sent Done false
9237 3381 Acked Read 1 Done Sent Done false
9238 3382 Acked Read 3 Done Sent Done false
9239 3383 Acked Read 1 Done Sent Done false
9240 3384 Acked Read 7 Done Sent Done false
9241 3385 Acked Read 5 Done Sent Done false
9242 3386 Acked Write 9 Done Sent Done false
9243 3387 Acked Read 8 Done Sent Done false
9244 3388 Acked Read 8 Done Sent Done false
9245 3389 Acked Read 7 Done Sent Done false
9246 3390 Acked Write 1 Done Sent Done false
9247 3391 Acked Write 6 Done Sent Done false
9248 3392 Acked Write 2 Done Sent Done false
9249 3393 Acked Write 7 Done Sent Done false
9250 3394 Acked Write 5 Done Sent Done false
9251 3395 Acked Read 9 Done Sent Done false
9252 3396 Acked Write 1 Done Sent Done false
9253 3397 Acked Write 8 Done Sent Done false
9254 3398 Acked Write 7 Done Sent Done false
9255 3399 Acked Read 5 Done Sent Done false
9256 3400 Acked Read 10 Done Sent Done false
9257 3401 Acked Read 6 Done Sent Done false
9258 3402 Acked Read 5 Done Sent Done false
9259 3403 Acked Flush 0 Done Sent Done false
9260 3404 Acked Read 4 Done Sent Done false
9261 3405 Acked Write 3 Done Sent Done false
9262 3406 Acked Read 7 Done Sent Done false
9263 3407 Acked Write 3 Done Sent Done false
9264 3408 Acked Read 1 Done Sent Done false
9265 3409 Acked Read 4 Done Sent Done false
9266 3410 Acked Read 5 Done Sent Done false
9267 3411 Acked Write 3 Done Sent Done false
9268 3412 Acked Read 5 Done Sent Done false
9269 3413 Acked Read 4 Done Sent Done false
9270 3414 Acked Flush 0 Done Sent Done false
9271 STATES DS:0 DS:1 DS:2 TOTAL
9272 Sent 0 302 0 302
9273 Done 302 0 302 604
9274 Skipped 0 0 0 0
9275 Error 0 0 0 0
9276 Last Flush: 3414 3112 3414
9277 Downstairs last five completed: 3112 3111 3110 3109 3108
9278 Upstairs last five completed: 3414 3413 3412 3411 3410
9279 CLIENT: Up:0 ds:302 act:2
92802024-11-19T17:09:57.432ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92812024-11-19T17:09:57.432ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55970) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92822024-11-19T17:09:57.433ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92832024-11-19T17:09:57.434ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92842024-11-19T17:09:57.434ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92852024-11-19T17:09:57.434ZINFOcrucible: send last flush ID to this DS: 3112 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92862024-11-19T17:09:57.434ZINFOcrucible: Replied this last flush ID: 3112 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92872024-11-19T17:09:57.434ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92882024-11-19T17:09:57.434ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
92892024-11-19T17:09:57.434ZINFOcrucible: [1] client re-new 302 jobs since flush 3112 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
92902024-11-19T17:09:57.434ZINFOcrucible: [1] Replayed 302 jobs since flush: 3112 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
9291 ----------------------------------------------------------------
9292 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9293 Downstairs last five completed: 3414 3413 3412 3411 3410
9294 Upstairs last five completed: 3414 3413 3412 3411 3410
9295 CLIENT: Up:0 ds:0 act:3
9296 Replay: All jobs finished
9297 Main task has work to do, go find it
9298 got dsc Stop(0)
9299 stop 0
9300 Nov 19 17:09:57.678 INFO request completed, latency_us: [8810] Got stop action so:false kr:false
9301 343, response_code: 204, uri: /stop/cid/0, method: POST, req_id: cf7efcaf-965c-4cc9-9125-0c4b20b3aefb, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
9302 Nov 19 17:09:57.679 INFO request completed{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:09:57.679578084Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
9303 , latency_us: 143, response_code: 200, uri: /state/cid/0, method: GET, req_id: 210c6d89-48ce-4165-ae2d-9e9a89e17e53, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
93042024-11-19T17:09:57.679ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
93052024-11-19T17:09:57.679ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
93062024-11-19T17:09:57.679ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
93072024-11-19T17:09:57.679ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
93082024-11-19T17:09:57.679ZINFOcrucible: client task is exiting = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
9309 [8810] Exited with: ExitStatus(unix_wait_status(9))
9310 [8810][0] reports Exit
9311 Nov 19 17:10:01.682 INFO request completed, latency_us: 316, response_code: 200, uri: /state/cid/0, method: GET, req_id: f0a57416-24f1-4689-8ffe-db691f351635, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
9312 001/300 Read block 563 len 28672
9313 002/300 Write block 124 len 28672 data: 3 3 3 4 3 5 5
9314 003/300 Read block 4116 len 32768
9315 004/300 Write block 508 len 20480 data: 2 2 2 2 2
9316 005/300 Read block 3692 len 40960
9317 006/300 Read block 737 len 12288
9318 007/300 Write block 3059 len 12288 data: 4 4 3
9319 008/300 Flush
9320 009/300 Flush
9321 010/300 Read block 3987 len 4096
9322 011/300 Read block 435 len 8192
9323 012/300 Read block 2700 len 36864
9324 013/300 Write block 1477 len 24576 data: 3 3 3 3 3 3
9325 014/300 Write block 934 len 40960 data: 3 4 3 3 3 3 2 3 3 4
9326 015/300 Read block 275 len 16384
9327 016/300 Flush
9328 017/300 Flush
9329 018/300 Write block 1818 len 24576 data: 3 4 3 3 4 4
9330 019/300 Write block 104 len 16384 data: 3 3 3 3
9331 020/300 Write block 2936 len 12288 data: 4 3 3
9332 021/300 Read block 3509 len 4096
9333 022/300 Read block 23 len 24576
9334 023/300 Read block 3929 len 4096
9335 024/300 Read block 3720 len 40960
9336 025/300 Flush
9337 026/300 Read block 60 len 28672
9338 027/300 Write block 2313 len 40960 data: 3 3 3 3 3 3 3 3 2 2
9339 028/300 Read block 1968 len 28672
9340 029/300 Flush
9341 030/300 Write block 1833 len 36864 data: 2 2 2 2 2 2 2 2 3
9342 031/300 Write block 3170 len 40960 data: 3 3 2 2 2 2 2 2 2 2
9343 032/300 Write block 4473 len 40960 data: 4 5 5 4 3 2 2 2 2 2
9344 033/300 Write block 2681 len 40960 data: 3 3 3 3 3 3 4 3 3 3
9345 034/300 Read block 1820 len 32768
9346 035/300 Read block 2360 len 4096
9347 036/300 Read block 4797 len 16384
9348 037/300 Read block 1171 len 16384
9349 038/300 Read block 2876 len 40960
9350 039/300 Write block 3893 len 36864 data: 3 3 3 3 3 3 2 2 2
9351 040/300 Read block 3970 len 36864
9352 041/300 Read block 3110 len 24576
9353 042/300 Write block 275 len 40960 data: 3 3 4 4 4 3 3 2 2 2
9354 043/300 Read block 1035 len 32768
9355 044/300 Read block 3835 len 12288
9356 045/300 Write block 642 len 4096 data: 2
9357 046/300 Read block 3813 len 32768
9358 047/300 Flush
9359 048/300 Write block 746 len 24576 data: 2 2 2 2 2 2
9360 049/300 Read block 3200 len 20480
9361 050/300 Read block 1954 len 40960
9362 051/300 Flush
9363 052/300 Write block 1228 len 4096 data: 2
9364 053/300 Read block 3582 len 24576
9365 054/300 Read block 1850 len 36864
9366 055/300 Flush
9367 056/300 Read block 1166 len 32768
9368 057/300 Read block 3469 len 32768
9369 058/300 Read block 418 len 24576
9370 059/300 Read block 1627 len 28672
9371 060/300 Write block 3832 len 16384 data: 3 3 3 3
9372 061/300 Write block 2393 len 36864 data: 4 4 2 2 2 2 2 2 2
9373 062/300 Write block 2208 len 36864 data: 5 6 5 5 5 6 4 4 3
9374 063/300 Read block 4471 len 20480
9375 064/300 Read block 3794 len 20480
9376 065/300 Read block 2610 len 28672
9377 066/300 Read block 2186 len 36864
9378 067/300 Read block 3524 len 28672
9379 068/300 Write block 2425 len 40960 data: 2 2 2 2 2 2 2 2 2 4
9380 069/300 Flush
9381 070/300 Write block 2343 len 16384 data: 4 5 5 5
9382 071/300 Read block 1766 len 36864
9383 072/300 Read block 151 len 32768
9384 073/300 Write block 3915 len 12288 data: 5 5 4
9385 074/300 Read block 1673 len 4096
9386 075/300 Read block 553 len 4096
9387 076/300 Read block 453 len 4096
9388 077/300 Read block 921 len 20480
9389 078/300 Write block 4923 len 28672 data: 2 2 2 3 4 3 4
9390 079/300 Read block 4103 len 20480
9391 080/300 Write block 2397 len 36864 data: 3 3 3 3 3 2 2 2 2
9392 081/300 Write block 1318 len 24576 data: 4 4 4 3 2 2
9393 082/300 Read block 2913 len 4096
9394 083/300 Write block 1959 len 8192 data: 2 2
9395 084/300 Flush
9396 085/300 Write block 1114 len 20480 data: 3 3 3 3 4
9397 086/300 Read block 2226 len 4096
9398 087/300 Flush
9399 088/300 Write block 2567 len 32768 data: 2 2 2 2 2 3 3 3
9400 089/300 Read block 2847 len 36864
9401 090/300 Write block 3974 len 16384 data: 2 2 2 2
9402 091/300 Write block 683 len 32768 data: 3 3 3 2 2 2 2 2
9403 092/300 Flush
9404 093/300 Read block 1532 len 16384
9405 094/300 Read block 7 len 4096
9406 095/300 Write block 793 len 24576 data: 5 4 3 3 2 3
9407 096/300 Read block 3386 len 36864
9408 097/300 Write block 2979 len 8192 data: 2 3
9409 098/300 Flush
9410 099/300 Write block 1796 len 8192 data: 3 3
9411 100/300 Read block 3958 len 12288
9412 101/300 Write block 4767 len 16384 data: 4 3 4 4
9413 102/300 Write block 1032 len 20480 data: 3 3 3 3 3
9414 103/300 Read block 2741 len 12288
9415 104/300 Read block 1988 len 24576
9416 105/300 Read block 3801 len 32768
9417 106/300 Read block 2809 len 36864
9418 107/300 Read block 2582 len 40960
9419 108/300 Read block 602 len 32768
9420 109/300 Read block 3979 len 40960
9421 110/300 Read block 663 len 16384
9422 111/300 Write block 212 len 16384 data: 2 2 2 3
9423 112/300 Read block 1497 len 40960
9424 113/300 Read block 4454 len 32768
9425 114/300 Read block 542 len 4096
9426 115/300 Flush
9427 116/300 Read block 490 len 4096
9428 117/300 Write block 3892 len 12288 data: 3 4 4
9429 118/300 Flush
9430 119/300 Write block 930 len 12288 data: 3 3 3
9431 120/300 Flush
9432 121/300 Write block 297 len 12288 data: 3 3 3
9433 122/300 Write block 3320 len 32768 data: 4 5 4 4 3 3 3 3
9434 123/300 Read block 2157 len 16384
9435 124/300 Read block 4016 len 32768
9436 125/300 Read block 1082 len 12288
9437 126/300 Write block 4894 len 36864 data: 3 3 4 4 4 4 4 3 2
9438 127/300 Read block 3320 len 8192
9439 128/300 Write block 3365 len 40960 data: 3 3 3 2 2 3 3 4 3 3
9440 129/300 Flush
9441 130/300 Read block 4960 len 32768
9442 131/300 Read block 4195 len 36864
9443 132/300 Read block 1545 len 12288
9444 133/300 Read block 474 len 16384
9445 134/300 Read block 643 len 8192
9446 135/300 Write block 2226 len 28672 data: 2 2 2 2 3 3 3
9447 136/300 Write block 3836 len 12288 data: 3 3 2
9448 137/300 Flush
9449 138/300 Read block 4846 len 20480
9450 139/300 Read block 4873 len 40960
9451 140/300 Read block 2190 len 32768
9452 141/300 Read block 909 len 4096
9453 142/300 Read block 2732 len 16384
9454 143/300 Write block 1991 len 12288 data: 2 2 2
9455 144/300 Read block 4940 len 8192
9456 145/300 Read block 1004 len 4096
9457 146/300 Read block 4634 len 4096
9458 147/300 Write block 3680 len 4096 data: 3
9459 148/300 Read block 4573 len 12288
9460 149/300 Write block 1526 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9461 150/300 Write block 1447 len 28672 data: 4 4 3 4 4 4 4
9462 151/300 Write block 2161 len 36864 data: 3 3 3 3 3 3 5 4 4
9463 152/300 Read block 4262 len 32768
9464 153/300 Write block 2487 len 4096 data: 4
9465 154/300 Write block 2807 len 36864 data: 5 4 4 3 3 3 3 2 2
9466 155/300 Read block 2920 len 12288
9467 156/300 Read block 2660 len 20480
9468 157/300 Write block 425 len 32768 data: 4 4 3 3 3 3 3 3
9469 158/300 Flush
9470 159/300 Write block 2353 len 36864 data: 3 3 3 3 2 2 2 2 2
9471 160/300 Read block 2326 len 32768
9472 161/300 Write block 4926 len 12288 data: 4 5 4
9473 162/300 Read block 4820 len 32768
9474 163/300 Read block 4333 len 16384
9475 164/300 Read block 4887 len 4096
9476 165/300 Read block 1835 len 36864
9477 166/300 Flush
9478 167/300 Read block 4055 len 32768
9479 168/300 Read block 3546 len 24576
9480 169/300 Write block 1062 len 20480 data: 3 4 4 3 5
9481 170/300 Read block 3449 len 36864
9482 171/300 Write block 3223 len 28672 data: 6 4 4 4 3 3 3
9483 172/300 Write block 3473 len 12288 data: 2 2 3
9484 173/300 Flush
9485 174/300 Read block 3754 len 16384
9486 175/300 Flush
9487 176/300 Read block 1124 len 32768
9488 177/300 Read block 4774 len 8192
9489 178/300 Read block 3570 len 12288
9490 179/300 Read block 2617 len 20480
9491 180/300 Write block 2916 len 40960 data: 4 3 3 4 4 5 4 3 2 2
9492 181/300 Read block 4536 len 16384
9493 182/300 Read block 3749 len 16384
9494 183/300 Write block 1158 len 12288 data: 3 3 3
9495 184/300 Write block 1284 len 28672 data: 4 5 5 4 4 5 5
9496 185/300 Read block 2371 len 40960
9497 186/300 Read block 3485 len 36864
9498 187/300 Write block 554 len 12288 data: 2 2 2
9499 188/300 Flush
9500 189/300 Write block 4098 len 28672 data: 5 3 3 3 3 2 2
9501 190/300 Read block 4341 len 20480
9502 191/300 Flush
9503 192/300 Write block 1777 len 16384 data: 3 5 5 5
9504 193/300 Read block 4835 len 4096
9505 194/300 Read block 2363 len 12288
9506 195/300 Write block 770 len 24576 data: 2 2 2 2 2 2
9507 196/300 Write block 2841 len 28672 data: 3 3 3 3 3 3 3
9508 197/300 Write block 1399 len 20480 data: 5 6 5 5 5
9509 198/300 Read block 127 len 24576
9510 199/300 Read block 692 len 12288
9511 200/300 Write block 2551 len 8192 data: 2 2
9512 201/300 Write block 3495 len 12288 data: 2 2 2
9513 202/300 Write block 3568 len 4096 data: 4
9514 203/300 Write block 3660 len 12288 data: 3 3 3
9515 204/300 Write block 1089 len 32768 data: 2 3 4 3 2 2 2 2
9516 205/300 Write block 1356 len 16384 data: 4 4 4 3
9517 206/300 Write block 1449 len 4096 data: 4
9518 207/300 Read block 1752 len 28672
9519 208/300 Write block 1159 len 32768 data: 4 4 3 4 5 5 4 4
9520 209/300 Write block 1452 len 16384 data: 5 5 3 3
9521 210/300 Write block 1389 len 40960 data: 2 2 2 2 2 2 2 3 4 4
9522 211/300 Read block 4085 len 24576
9523 212/300 Write block 4080 len 16384 data: 2 2 2 2
9524 213/300 Write block 1117 len 8192 data: 4 5
9525 214/300 Read block 4300 len 24576
9526 215/300 Read block 741 len 4096
9527 216/300 Write block 419 len 8192 data: 4 3
9528 217/300 Write block 1289 len 28672 data: 6 6 4 3 3 3 2
9529 218/300 Write block 1258 len 24576 data: 3 3 3 3 3 3
9530 219/300 Read block 1462 len 8192
9531 220/300 Read block 1235 len 28672
9532 221/300 Read block 2301 len 16384
9533 222/300 Read block 2886 len 32768
9534 223/300 Read block 952 len 16384
9535 224/300 Read block 1885 len 4096
9536 225/300 Read block 867 len 4096
9537 226/300 Write block 2154 len 32768 data: 3 3 3 2 2 3 3 4
9538 227/300 Read block 2900 len 20480
9539 228/300 Write block 4726 len 12288 data: 2 2 2
9540 229/300 Write block 2596 len 8192 data: 2 2
9541 230/300 Write block 2118 len 8192 data: 3 2
9542 231/300 Read block 4311 len 8192
9543 232/300 Write block 4571 len 8192 data: 3 2
9544 233/300 Write block 4751 len 20480 data: 2 2 2 2 2
9545 234/300 Write block 3899 len 8192 data: 3 3
9546 235/300 Flush
9547 236/300 Write block 4501 len 40960 data: 4 4 4 4 4 4 2 2 2 3
9548 237/300 Write block 3291 len 20480 data: 2 2 3 3 3
9549 238/300 Read block 4435 len 40960
9550 239/300 Write block 3315 len 36864 data: 4 4 5 5 5 5 6 5 5
9551 240/300 Read block 1943 len 24576
9552 241/300 Flush
9553 242/300 Write block 3126 len 4096 data: 6
9554 243/300 Write block 3867 len 36864 data: 5 4 4 4 4 4 4 4 3
9555 244/300 Write block 3116 len 36864 data: 2 2 3 3 3 3 3 3 4
9556 245/300 Read block 3808 len 36864
9557 246/300 Write block 4297 len 8192 data: 2 2
9558 247/300 Write block 4280 len 12288 data: 2 2 3
9559 248/300 Write block 2080 len 4096 data: 2
9560 249/300 Write block 4849 len 20480 data: 3 3 3 3 3
9561 250/300 Read block 4597 len 32768
9562 251/300 Read block 2398 len 8192
9563 252/300 Read block 3971 len 32768
9564 253/300 Read block 3993 len 12288
9565 254/300 Read block 3122 len 8192
9566 255/300 Read block 3363 len 16384
9567 256/300 Read block 2954 len 8192
9568 257/300 Read block 2729 len 20480
9569 258/300 Read block 2364 len 32768
9570 259/300 Write block 3530 len 32768 data: 3 4 3 2 2 3 3 3
9571 260/300 Flush
9572 261/300 Write block 3804 len 32768 data: 3 3 4 3 3 3 3 3
9573 262/300 Read block 709 len 28672
9574 263/300 Flush
9575 264/300 Read block 2134 len 40960
9576 265/300 Flush
9577 266/300 Write block 412 len 8192 data: 2 2
9578 267/300 Write block 616 len 4096 data: 4
9579 268/300 Read block 4928 len 12288
9580 269/300 Write block 4680 len 16384 data: 2 2 2 2
9581 270/300 Write block 2712 len 24576 data: 3 3 3 3 2 2
9582 271/300 Write block 268 len 16384 data: 2 2 2 2
9583 272/300 Read block 2693 len 24576
9584 273/300 Read block 66 len 24576
9585 274/300 Read block 2034 len 24576
9586 275/300 Write block 2019 len 32768 data: 2 2 2 4 5 5 6 7
9587 276/300 Read block 888 len 40960
9588 277/300 Read block 4202 len 40960
9589 278/300 Read block 2307 len 24576
9590 279/300 Write block 2917 len 8192 data: 4 4
9591 280/300 Read block 4191 len 16384
9592 281/300 Write block 2534 len 16384 data: 3 3 4 4
9593 282/300 Read block 3232 len 36864
9594 283/300 Read block 3780 len 40960
9595 284/300 Read block 1202 len 12288
9596 285/300 Read block 2613 len 12288
9597 286/300 Read block 1172 len 8192
9598 287/300 Write block 1606 len 40960 data: 5 6 5 6 6 7 6 6 6 6
9599 288/300 Write block 4668 len 8192 data: 3 3
9600 289/300 Read block 3789 len 24576
9601 290/300 Read block 4236 len 4096
9602 291/300 Write block 2859 len 20480 data: 2 2 2 2 2
9603 292/300 Read block 249 len 36864
9604 293/300 Write block 3540 len 12288 data: 2 2 2
9605 294/300 Write block 3462 len 40960 data: 2 2 4 4 4 3 3 3 3 3
9606 295/300 Write block 1026 len 20480 data: 4 4 4 4 4
9607 296/300 Read block 2012 len 36864
9608 297/300 Read block 1832 len 20480
9609 298/300 Write block 314 len 24576 data: 3 3 3 4 5 5
9610 299/300 Read block 2379 len 12288
9611 300/300 Flush
9612 301/300 Read block 2726 len 40960
9613 Main task has work to do, go find it
9614 got dsc Start(0)
9615 start 0
9616 Nov 19 17:10:01.920 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/0, method: POST, req_id: 7fce3328-1a0a-4f70-8a48-da6d882a542e, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
9617 Replay: started 0, returned:Ok(())
9618 ----------------------------------------------------------------
9619 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9620 States: Offline Active Active
9621 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9622 3415 Acked Read 7 Sent Done Done false
9623 3416 Acked Write 7 Sent Done Done false
9624 3417 Acked Read 8 Sent Done Done false
9625 3418 Acked Write 5 Sent Done Done false
9626 3419 Acked Read 10 Sent Done Done false
9627 3420 Acked Read 3 Sent Done Done false
9628 3421 Acked Write 3 Sent Done Done false
9629 3422 Acked Flush 0 Sent Done Done false
9630 3423 Acked Flush 0 Sent Done Done false
9631 3424 Acked Read 1 Sent Done Done false
9632 3425 Acked Read 2 Sent Done Done false
9633 3426 Acked Read 9 Sent Done Done false
9634 3427 Acked Write 6 Sent Done Done false
9635 3428 Acked Write 10 Sent Done Done false
9636 3429 Acked Read 4 Sent Done Done false
9637 3430 Acked Flush 0 Sent Done Done false
9638 3431 Acked Flush 0 Sent Done Done false
9639 3432 Acked Write 6 Sent Done Done false
9640 3433 Acked Write 4 Sent Done Done false
9641 3434 Acked Write 3 Sent Done Done false
9642 3435 Acked Read 1 Sent Done Done false
9643 3436 Acked Read 6 Sent Done Done false
9644 3437 Acked Read 1 Sent Done Done false
9645 3438 Acked Read 10 Sent Done Done false
9646 3439 Acked Flush 0 Sent Done Done false
9647 3440 Acked Read 7 Sent Done Done false
9648 3441 Acked Write 10 Sent Done Done false
9649 3442 Acked Read 7 Sent Done Done false
9650 3443 Acked Flush 0 Sent Done Done false
9651 3444 Acked Write 9 Sent Done Done false
9652 3445 Acked Write 10 Sent Done Done false
9653 3446 Acked Write 10 Sent Done Done false
9654 3447 Acked Write 10 Sent Done Done false
9655 3448 Acked Read 8 Sent Done Done false
9656 3449 Acked Read 1 Sent Done Done false
9657 3450 Acked Read 4 Sent Done Done false
9658 3451 Acked Read 4 Sent Done Done false
9659 3452 Acked Read 10 Sent Done Done false
9660 3453 Acked Write 9 Sent Done Done false
9661 3454 Acked Read 9 Sent Done Done false
9662 3455 Acked Read 6 Sent Done Done false
9663 3456 Acked Write 10 Sent Done Done false
9664 3457 Acked Read 8 Sent Done Done false
9665 3458 Acked Read 3 Sent Done Done false
9666 3459 Acked Write 1 Sent Done Done false
9667 3460 Acked Read 8 Sent Done Done false
9668 3461 Acked Flush 0 Sent Done Done false
9669 3462 Acked Write 6 Sent Done Done false
9670 3463 Acked Read 5 Sent Done Done false
9671 3464 Acked Read 10 Sent Done Done false
9672 3465 Acked Flush 0 Sent Done Done false
9673 3466 Acked Write 1 Sent Done Done false
9674 3467 Acked Read 6 Sent Done Done false
9675 3468 Acked Read 9 Sent Done Done false
9676 3469 Acked Flush 0 Sent Done Done false
9677 3470 Acked Read 8 Sent Done Done false
9678 3471 Acked Read 8 Sent Done Done false
9679 3472 Acked Read 6 Sent Done Done false
9680 3473 Acked Read 7 Sent Done Done false
9681 3474 Acked Write 4 Sent Done Done false
9682 3475 Acked Write 9 Sent Done Done false
9683 3476 Acked Write 9 Sent Done Done false
9684 3477 Acked Read 5 Sent Done Done false
9685 3478 Acked Read 5 Sent Done Done false
9686 3479 Acked Read 7 Sent Done Done false
9687 3480 Acked Read 9 Sent Done Done false
9688 3481 Acked Read 7 Sent Done Done false
9689 3482 Acked Write 10 Sent Done Done false
9690 3483 Acked Flush 0 Sent Done Done false
9691 3484 Acked Write 4 Sent Done Done false
9692 3485 Acked Read 9 Sent Done Done false
9693 3486 Acked Read 8 Sent Done Done false
9694 3487 Acked Write 3 Sent Done Done false
9695 3488 Acked Read 1 Sent Done Done false
9696 3489 Acked Read 1 Sent Done Done false
9697 3490 Acked Read 1 Sent Done Done false
9698 3491 Acked Read 5 Sent Done Done false
9699 3492 Acked Write 7 Sent Done Done false
9700 3493 Acked Read 5 Sent Done Done false
9701 3494 Acked Write 9 Sent Done Done false
9702 3495 Acked Write 6 Sent Done Done false
9703 3496 Acked Read 1 Sent Done Done false
9704 3497 Acked Write 2 Sent Done Done false
9705 3498 Acked Flush 0 Sent Done Done false
9706 3499 Acked Write 5 Sent Done Done false
9707 3500 Acked Read 1 Sent Done Done false
9708 3501 Acked Flush 0 Sent Done Done false
9709 3502 Acked Write 8 Sent Done Done false
9710 3503 Acked Read 9 Sent Done Done false
9711 3504 Acked Write 4 Sent Done Done false
9712 3505 Acked Write 8 Sent Done Done false
9713 3506 Acked Flush 0 Sent Done Done false
9714 3507 Acked Read 4 Sent Done Done false
9715 3508 Acked Read 1 Sent Done Done false
9716 3509 Acked Write 6 Sent Done Done false
9717 3510 Acked Read 9 Sent Done Done false
9718 3511 Acked Write 2 Sent Done Done false
9719 3512 Acked Flush 0 Sent Done Done false
9720 3513 Acked Write 2 Sent Done Done false
9721 3514 Acked Read 3 Sent Done Done false
9722 3515 Acked Write 4 Sent Done Done false
9723 3516 Acked Write 5 Sent Done Done false
9724 3517 Acked Read 3 Sent Done Done false
9725 3518 Acked Read 6 Sent Done Done false
9726 3519 Acked Read 8 Sent Done Done false
9727 3520 Acked Read 9 Sent Done Done false
9728 3521 Acked Read 10 Sent Done Done false
9729 3522 Acked Read 8 Sent Done Done false
9730 3523 Acked Read 10 Sent Done Done false
9731 3524 Acked Read 4 Sent Done Done false
9732 3525 Acked Write 4 Sent Done Done false
9733 3526 Acked Read 10 Sent Done Done false
9734 3527 Acked Read 8 Sent Done Done false
9735 3528 Acked Read 1 Sent Done Done false
9736 3529 Acked Flush 0 Sent Done Done false
9737 3530 Acked Read 1 Sent Done Done false
9738 3531 Acked Write 3 Sent Done Done false
9739 3532 Acked Flush 0 Sent Done Done false
9740 3533 Acked Write 3 Sent Done Done false
9741 3534 Acked Flush 0 Sent Done Done false
9742 3535 Acked Write 3 Sent Done Done false
9743 3536 Acked Write 8 Sent Done Done false
9744 3537 Acked Read 4 Sent Done Done false
9745 3538 Acked Read 8 Sent Done Done false
9746 3539 Acked Read 3 Sent Done Done false
9747 3540 Acked Write 9 Sent Done Done false
9748 3541 Acked Read 2 Sent Done Done false
9749 3542 Acked Write 10 Sent Done Done false
9750 3543 Acked Flush 0 Sent Done Done false
9751 3544 Acked Read 8 Sent Done Done false
9752 3545 Acked Read 9 Sent Done Done false
9753 3546 Acked Read 3 Sent Done Done false
9754 3547 Acked Read 4 Sent Done Done false
9755 3548 Acked Read 2 Sent Done Done false
9756 3549 Acked Write 7 Sent Done Done false
9757 3550 Acked Write 3 Sent Done Done false
9758 3551 Acked Flush 0 Sent Done Done false
9759 3552 Acked Read 5 Sent Done Done false
9760 3553 Acked Read 10 Sent Done Done false
9761 3554 Acked Read 8 Sent Done Done false
9762 3555 Acked Read 1 Sent Done Done false
9763 3556 Acked Read 4 Sent Done Done false
9764 3557 Acked Write 3 Sent Done Done false
9765 3558 Acked Read 2 Sent Done Done false
9766 3559 Acked Read 1 Sent Done Done false
9767 3560 Acked Read 1 Sent Done Done false
9768 3561 Acked Write 1 Sent Done Done false
9769 3562 Acked Read 3 Sent Done Done false
9770 3563 Acked Write 10 Sent Done Done false
9771 3564 Acked Write 7 Sent Done Done false
9772 3565 Acked Write 9 Sent Done Done false
9773 3566 Acked Read 8 Sent Done Done false
9774 3567 Acked Write 1 Sent Done Done false
9775 3568 Acked Write 9 Sent Done Done false
9776 3569 Acked Read 3 Sent Done Done false
9777 3570 Acked Read 5 Sent Done Done false
9778 3571 Acked Write 8 Sent Done Done false
9779 3572 Acked Flush 0 Sent Done Done false
9780 3573 Acked Write 9 Sent Done Done false
9781 3574 Acked Read 8 Sent Done Done false
9782 3575 Acked Write 3 Sent Done Done false
9783 3576 Acked Read 8 Sent Done Done false
9784 3577 Acked Read 4 Sent Done Done false
9785 3578 Acked Read 1 Sent Done Done false
9786 3579 Acked Read 9 Sent Done Done false
9787 3580 Acked Flush 0 Sent Done Done false
9788 3581 Acked Read 8 Sent Done Done false
9789 3582 Acked Read 6 Sent Done Done false
9790 3583 Acked Write 5 Sent Done Done false
9791 3584 Acked Read 9 Sent Done Done false
9792 3585 Acked Write 7 Sent Done Done false
9793 3586 Acked Write 3 Sent Done Done false
9794 3587 Acked Flush 0 Sent Done Done false
9795 3588 Acked Read 4 Sent Done Done false
9796 3589 Acked Flush 0 Sent Done Done false
9797 3590 Acked Read 8 Sent Done Done false
9798 3591 Acked Read 2 Sent Done Done false
9799 3592 Acked Read 3 Sent Done Done false
9800 3593 Acked Read 5 Sent Done Done false
9801 3594 Acked Write 10 Sent Done Done false
9802 3595 Acked Read 4 Sent Done Done false
9803 3596 Acked Read 4 Sent Done Done false
9804 3597 Acked Write 3 Sent Done Done false
9805 3598 Acked Write 7 Sent Done Done false
9806 3599 Acked Read 10 Sent Done Done false
9807 3600 Acked Read 9 Sent Done Done false
9808 3601 Acked Write 3 Sent Done Done false
9809 3602 Acked Flush 0 Sent Done Done false
9810 3603 Acked Write 7 Sent Done Done false
9811 3604 Acked Read 5 Sent Done Done false
9812 3605 Acked Flush 0 Sent Done Done false
9813 3606 Acked Write 4 Sent Done Done false
9814 3607 Acked Read 1 Sent Done Done false
9815 3608 Acked Read 3 Sent Done Done false
9816 3609 Acked Write 6 Sent Done Done false
9817 3610 Acked Write 7 Sent Done Done false
9818 3611 Acked Write 5 Sent Done Done false
9819 3612 Acked Read 6 Sent Done Done false
9820 3613 Acked Read 3 Sent Done Done false
9821 3614 Acked Write 2 Sent Done Done false
9822 3615 Acked Write 3 Sent Done Done false
9823 3616 Acked Write 1 Sent Done Done false
9824 3617 Acked Write 3 Sent Done Done false
9825 3618 Acked Write 8 Sent Done Done false
9826 3619 Acked Write 4 Sent Done Done false
9827 3620 Acked Write 1 Sent Done Done false
9828 3621 Acked Read 7 Sent Done Done false
9829 3622 Acked Write 8 Sent Done Done false
9830 3623 Acked Write 4 Sent Done Done false
9831 3624 Acked Write 10 Sent Done Done false
9832 3625 Acked Read 6 Sent Done Done false
9833 3626 Acked Write 4 Sent Done Done false
9834 3627 Acked Write 2 Sent Done Done false
9835 3628 Acked Read 6 Sent Done Done false
9836 3629 Acked Read 1 Sent Done Done false
9837 3630 Acked Write 2 Sent Done Done false
9838 3631 Acked Write 7 Sent Done Done false
9839 3632 Acked Write 6 Sent Done Done false
9840 3633 Acked Read 2 Sent Done Done false
9841 3634 Acked Read 7 Sent Done Done false
9842 3635 Acked Read 4 Sent Done Done false
9843 3636 Acked Read 8 Sent Done Done false
9844 3637 Acked Read 4 Sent Done Done false
9845 3638 Acked Read 1 Sent Done Done false
9846 3639 Acked Read 1 Sent Done Done false
9847 3640 Acked Write 8 Sent Done Done false
9848 3641 Acked Read 5 Sent Done Done false
9849 3642 Acked Write 3 Sent Done Done false
9850 3643 Acked Write 2 Sent Done Done false
9851 3644 Acked Write 2 Sent Done Done false
9852 3645 Acked Read 2 Sent Done Done false
9853 3646 Acked Write 2 Sent Done Done false
9854 3647 Acked Write 5 Sent Done Done false
9855 3648 Acked Write 2 Sent Done Done false
9856 3649 Acked Flush 0 Sent Done Done false
9857 3650 Acked Write 10 Sent Done Done false
9858 3651 Acked Write 5 Sent Done Done false
9859 3652 Acked Read 10 Sent Done Done false
9860 3653 Acked Write 9 Sent Done Done false
9861 3654 Acked Read 6 Sent Done Done false
9862 3655 Acked Flush 0 Sent Done Done false
9863 3656 Acked Write 1 Sent Done Done false
9864 3657 Acked Write 9 Sent Done Done false
9865 3658 Acked Write 9 Sent Done Done false
9866 3659 Acked Read 9 Sent Done Done false
9867 3660 Acked Write 2 Sent Done Done false
9868 3661 Acked Write 3 Sent Done Done false
9869 3662 Acked Write 1 Sent Done Done false
9870 3663 Acked Write 5 Sent Done Done false
9871 3664 Acked Read 8 Sent Done Done false
9872 3665 Acked Read 2 Sent Done Done false
9873 3666 Acked Read 8 Sent Done Done false
9874 3667 Acked Read 3 Sent Done Done false
9875 3668 Acked Read 2 Sent Done Done false
9876 3669 Acked Read 4 Sent Done Done false
9877 3670 Acked Read 2 Sent Done Done false
9878 3671 Acked Read 5 Sent Done Done false
9879 3672 Acked Read 8 Sent Done Done false
9880 3673 Acked Write 8 Sent Done Done false
9881 3674 Acked Flush 0 Sent Done Done false
9882 3675 Acked Write 8 Sent Done Done false
9883 3676 Acked Read 7 Sent Done Done false
9884 3677 Acked Flush 0 Sent Done Done false
9885 3678 Acked Read 10 Sent Done Done false
9886 3679 Acked Flush 0 Sent Done Done false
9887 3680 Acked Write 2 Sent Done Done false
9888 3681 Acked Write 1 Sent Done Done false
9889 3682 Acked Read 3 Sent Done Done false
9890 3683 Acked Write 4 Sent Done Done false
9891 3684 Acked Write 6 Sent Done Done false
9892 3685 Acked Write 4 Sent Done Done false
9893 3686 Acked Read 6 Sent Done Done false
9894 3687 Acked Read 6 Sent Done Done false
9895 3688 Acked Read 6 Sent Done Done false
9896 3689 Acked Write 8 Sent Done Done false
9897 3690 Acked Read 10 Sent Done Done false
9898 3691 Acked Read 10 Sent Done Done false
9899 3692 Acked Read 6 Sent Done Done false
9900 3693 Acked Write 2 Sent Done Done false
9901 3694 Acked Read 4 Sent Done Done false
9902 3695 Acked Write 4 Sent Done Done false
9903 3696 Acked Read 9 Sent Done Done false
9904 3697 Acked Read 10 Sent Done Done false
9905 3698 Acked Read 3 Sent Done Done false
9906 3699 Acked Read 3 Sent Done Done false
9907 3700 Acked Read 2 Sent Done Done false
9908 3701 Acked Write 10 Sent Done Done false
9909 3702 Acked Write 2 Sent Done Done false
9910 3703 Acked Read 6 Sent Done Done false
9911 3704 Acked Read 1 Sent Done Done false
9912 3705 Acked Write 5 Sent Done Done false
9913 3706 Acked Read 9 Sent Done Done false
9914 3707 Acked Write 3 Sent Done Done false
9915 3708 Acked Write 10 Sent Done Done false
9916 3709 Acked Write 5 Sent Done Done false
9917 3710 Acked Read 9 Sent Done Done false
9918 3711 Acked Read 5 Sent Done Done false
9919 3712 Acked Write 6 Sent Done Done false
9920 3713 Acked Read 3 Sent Done Done false
9921 3714 Acked Flush 0 Sent Done Done false
9922 3715 Acked Read 10 Sent Done Done false
9923 STATES DS:0 DS:1 DS:2 TOTAL
9924 Sent 301 0 0 301
9925 Done 0 301 301 602
9926 Skipped 0 0 0 0
9927 Error 0 0 0 0
9928 Last Flush: 3414 3714 3714
9929 Downstairs last five completed: 3414 3413 3412 3411 3410
9930 Upstairs last five completed: 3715 3714 3713 3712 3711
9931 CLIENT: Up:0 ds:301 act:2
9932 ----------------------------------------------------------------
9933 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9934 States: Offline Active Active
9935 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9936 3415 Acked Read 7 Sent Done Done false
9937 3416 Acked Write 7 Sent Done Done false
9938 3417 Acked Read 8 Sent Done Done false
9939 3418 Acked Write 5 Sent Done Done false
9940 3419 Acked Read 10 Sent Done Done false
9941 3420 Acked Read 3 Sent Done Done false
9942 3421 Acked Write 3 Sent Done Done false
9943 3422 Acked Flush 0 Sent Done Done false
9944 3423 Acked Flush 0 Sent Done Done false
9945 3424 Acked Read 1 Sent Done Done false
9946 3425 Acked Read 2 Sent Done Done false
9947 3426 Acked Read 9 Sent Done Done false
9948 3427 Acked Write 6 Sent Done Done false
9949 3428 Acked Write 10 Sent Done Done false
9950 3429 Acked Read 4 Sent Done Done false
9951 3430 Acked Flush 0 Sent Done Done false
9952 3431 Acked Flush 0 Sent Done Done false
9953 3432 Acked Write 6 Sent Done Done false
9954 3433 Acked Write 4 Sent Done Done false
9955 3434 Acked Write 3 Sent Done Done false
9956 3435 Acked Read 1 Sent Done Done false
9957 3436 Acked Read 6 Sent Done Done false
9958 3437 Acked Read 1 Sent Done Done false
9959 3438 Acked Read 10 Sent Done Done false
9960 3439 Acked Flush 0 Sent Done Done false
9961 3440 Acked Read 7 Sent Done Done false
9962 3441 Acked Write 10 Sent Done Done false
9963 3442 Acked Read 7 Sent Done Done false
9964 3443 Acked Flush 0 Sent Done Done false
9965 3444 Acked Write 9 Sent Done Done false
9966 3445 Acked Write 10 Sent Done Done false
9967 3446 Acked Write 10 Sent Done Done false
9968 3447 Acked Write 10 Sent Done Done false
9969 3448 Acked Read 8 Sent Done Done false
9970 3449 Acked Read 1 Sent Done Done false
9971 3450 Acked Read 4 Sent Done Done false
9972 3451 Acked Read 4 Sent Done Done false
9973 3452 Acked Read 10 Sent Done Done false
9974 3453 Acked Write 9 Sent Done Done false
9975 3454 Acked Read 9 Sent Done Done false
9976 3455 Acked Read 6 Sent Done Done false
9977 3456 Acked Write 10 Sent Done Done false
9978 3457 Acked Read 8 Sent Done Done false
9979 3458 Acked Read 3 Sent Done Done false
9980 3459 Acked Write 1 Sent Done Done false
9981 3460 Acked Read 8 Sent Done Done false
9982 3461 Acked Flush 0 Sent Done Done false
9983 3462 Acked Write 6 Sent Done Done false
9984 3463 Acked Read 5 Sent Done Done false
9985 3464 Acked Read 10 Sent Done Done false
9986 3465 Acked Flush 0 Sent Done Done false
9987 3466 Acked Write 1 Sent Done Done false
9988 3467 Acked Read 6 Sent Done Done false
9989 3468 Acked Read 9 Sent Done Done false
9990 3469 Acked Flush 0 Sent Done Done false
9991 3470 Acked Read 8 Sent Done Done false
9992 3471 Acked Read 8 Sent Done Done false
9993 3472 Acked Read 6 Sent Done Done false
9994 3473 Acked Read 7 Sent Done Done false
9995 3474 Acked Write 4 Sent Done Done false
9996 3475 Acked Write 9 Sent Done Done false
9997 3476 Acked Write 9 Sent Done Done false
9998 3477 Acked Read 5 Sent Done Done false
9999 3478 Acked Read 5 Sent Done Done false
10000 3479 Acked Read 7 Sent Done Done false
10001 3480 Acked Read 9 Sent Done Done false
10002 3481 Acked Read 7 Sent Done Done false
10003 3482 Acked Write 10 Sent Done Done false
10004 3483 Acked Flush 0 Sent Done Done false
10005 3484 Acked Write 4 Sent Done Done false
10006 3485 Acked Read 9 Sent Done Done false
10007 3486 Acked Read 8 Sent Done Done false
10008 3487 Acked Write 3 Sent Done Done false
10009 3488 Acked Read 1 Sent Done Done false
10010 3489 Acked Read 1 Sent Done Done false
10011 3490 Acked Read 1 Sent Done Done false
10012 3491 Acked Read 5 Sent Done Done false
10013 3492 Acked Write 7 Sent Done Done false
10014 3493 Acked Read 5 Sent Done Done false
10015 3494 Acked Write 9 Sent Done Done false
10016 3495 Acked Write 6 Sent Done Done false
10017 3496 Acked Read 1 Sent Done Done false
10018 3497 Acked Write 2 Sent Done Done false
10019 3498 Acked Flush 0 Sent Done Done false
10020 3499 Acked Write 5 Sent Done Done false
10021 3500 Acked Read 1 Sent Done Done false
10022 3501 Acked Flush 0 Sent Done Done false
10023 3502 Acked Write 8 Sent Done Done false
10024 3503 Acked Read 9 Sent Done Done false
10025 3504 Acked Write 4 Sent Done Done false
10026 3505 Acked Write 8 Sent Done Done false
10027 3506 Acked Flush 0 Sent Done Done false
10028 3507 Acked Read 4 Sent Done Done false
10029 3508 Acked Read 1 Sent Done Done false
10030 3509 Acked Write 6 Sent Done Done false
10031 3510 Acked Read 9 Sent Done Done false
10032 3511 Acked Write 2 Sent Done Done false
10033 3512 Acked Flush 0 Sent Done Done false
10034 3513 Acked Write 2 Sent Done Done false
10035 3514 Acked Read 3 Sent Done Done false
10036 3515 Acked Write 4 Sent Done Done false
10037 3516 Acked Write 5 Sent Done Done false
10038 3517 Acked Read 3 Sent Done Done false
10039 3518 Acked Read 6 Sent Done Done false
10040 3519 Acked Read 8 Sent Done Done false
10041 3520 Acked Read 9 Sent Done Done false
10042 3521 Acked Read 10 Sent Done Done false
10043 3522 Acked Read 8 Sent Done Done false
10044 3523 Acked Read 10 Sent Done Done false
10045 3524 Acked Read 4 Sent Done Done false
10046 3525 Acked Write 4 Sent Done Done false
10047 3526 Acked Read 10 Sent Done Done false
10048 3527 Acked Read 8 Sent Done Done false
10049 3528 Acked Read 1 Sent Done Done false
10050 3529 Acked Flush 0 Sent Done Done false
10051 3530 Acked Read 1 Sent Done Done false
10052 3531 Acked Write 3 Sent Done Done false
10053 3532 Acked Flush 0 Sent Done Done false
10054 3533 Acked Write 3 Sent Done Done false
10055 3534 Acked Flush 0 Sent Done Done false
10056 3535 Acked Write 3 Sent Done Done false
10057 3536 Acked Write 8 Sent Done Done false
10058 3537 Acked Read 4 Sent Done Done false
10059 3538 Acked Read 8 Sent Done Done false
10060 3539 Acked Read 3 Sent Done Done false
10061 3540 Acked Write 9 Sent Done Done false
10062 3541 Acked Read 2 Sent Done Done false
10063 3542 Acked Write 10 Sent Done Done false
10064 3543 Acked Flush 0 Sent Done Done false
10065 3544 Acked Read 8 Sent Done Done false
10066 3545 Acked Read 9 Sent Done Done false
10067 3546 Acked Read 3 Sent Done Done false
10068 3547 Acked Read 4 Sent Done Done false
10069 3548 Acked Read 2 Sent Done Done false
10070 3549 Acked Write 7 Sent Done Done false
10071 3550 Acked Write 3 Sent Done Done false
10072 3551 Acked Flush 0 Sent Done Done false
10073 3552 Acked Read 5 Sent Done Done false
10074 3553 Acked Read 10 Sent Done Done false
10075 3554 Acked Read 8 Sent Done Done false
10076 3555 Acked Read 1 Sent Done Done false
10077 3556 Acked Read 4 Sent Done Done false
10078 3557 Acked Write 3 Sent Done Done false
10079 3558 Acked Read 2 Sent Done Done false
10080 3559 Acked Read 1 Sent Done Done false
10081 3560 Acked Read 1 Sent Done Done false
10082 3561 Acked Write 1 Sent Done Done false
10083 3562 Acked Read 3 Sent Done Done false
10084 3563 Acked Write 10 Sent Done Done false
10085 3564 Acked Write 7 Sent Done Done false
10086 3565 Acked Write 9 Sent Done Done false
10087 3566 Acked Read 8 Sent Done Done false
10088 3567 Acked Write 1 Sent Done Done false
10089 3568 Acked Write 9 Sent Done Done false
10090 3569 Acked Read 3 Sent Done Done false
10091 3570 Acked Read 5 Sent Done Done false
10092 3571 Acked Write 8 Sent Done Done false
10093 3572 Acked Flush 0 Sent Done Done false
10094 3573 Acked Write 9 Sent Done Done false
10095 3574 Acked Read 8 Sent Done Done false
10096 3575 Acked Write 3 Sent Done Done false
10097 3576 Acked Read 8 Sent Done Done false
10098 3577 Acked Read 4 Sent Done Done false
10099 3578 Acked Read 1 Sent Done Done false
10100 3579 Acked Read 9 Sent Done Done false
10101 3580 Acked Flush 0 Sent Done Done false
10102 3581 Acked Read 8 Sent Done Done false
10103 3582 Acked Read 6 Sent Done Done false
10104 3583 Acked Write 5 Sent Done Done false
10105 3584 Acked Read 9 Sent Done Done false
10106 3585 Acked Write 7 Sent Done Done false
10107 3586 Acked Write 3 Sent Done Done false
10108 3587 Acked Flush 0 Sent Done Done false
10109 3588 Acked Read 4 Sent Done Done false
10110 3589 Acked Flush 0 Sent Done Done false
10111 3590 Acked Read 8 Sent Done Done false
10112 3591 Acked Read 2 Sent Done Done false
10113 3592 Acked Read 3 Sent Done Done false
10114 3593 Acked Read 5 Sent Done Done false
10115 3594 Acked Write 10 Sent Done Done false
10116 3595 Acked Read 4 Sent Done Done false
10117 3596 Acked Read 4 Sent Done Done false
10118 3597 Acked Write 3 Sent Done Done false
10119 3598 Acked Write 7 Sent Done Done false
10120 3599 Acked Read 10 Sent Done Done false
10121 3600 Acked Read 9 Sent Done Done false
10122 3601 Acked Write 3 Sent Done Done false
10123 3602 Acked Flush 0 Sent Done Done false
10124 3603 Acked Write 7 Sent Done Done false
10125 3604 Acked Read 5 Sent Done Done false
10126 3605 Acked Flush 0 Sent Done Done false
10127 3606 Acked Write 4 Sent Done Done false
10128 3607 Acked Read 1 Sent Done Done false
10129 3608 Acked Read 3 Sent Done Done false
10130 3609 Acked Write 6 Sent Done Done false
10131 3610 Acked Write 7 Sent Done Done false
10132 3611 Acked Write 5 Sent Done Done false
10133 3612 Acked Read 6 Sent Done Done false
10134 3613 Acked Read 3 Sent Done Done false
10135 3614 Acked Write 2 Sent Done Done false
10136 3615 Acked Write 3 Sent Done Done false
10137 3616 Acked Write 1 Sent Done Done false
10138 3617 Acked Write 3 Sent Done Done false
10139 3618 Acked Write 8 Sent Done Done false
10140 3619 Acked Write 4 Sent Done Done false
10141 3620 Acked Write 1 Sent Done Done false
10142 3621 Acked Read 7 Sent Done Done false
10143 3622 Acked Write 8 Sent Done Done false
10144 3623 Acked Write 4 Sent Done Done false
10145 3624 Acked Write 10 Sent Done Done false
10146 3625 Acked Read 6 Sent Done Done false
10147 3626 Acked Write 4 Sent Done Done false
10148 3627 Acked Write 2 Sent Done Done false
10149 3628 Acked Read 6 Sent Done Done false
10150 3629 Acked Read 1 Sent Done Done false
10151 3630 Acked Write 2 Sent Done Done false
10152 3631 Acked Write 7 Sent Done Done false
10153 3632 Acked Write 6 Sent Done Done false
10154 3633 Acked Read 2 Sent Done Done false
10155 3634 Acked Read 7 Sent Done Done false
10156 3635 Acked Read 4 Sent Done Done false
10157 3636 Acked Read 8 Sent Done Done false
10158 3637 Acked Read 4 Sent Done Done false
10159 3638 Acked Read 1 Sent Done Done false
10160 3639 Acked Read 1 Sent Done Done false
10161 3640 Acked Write 8 Sent Done Done false
10162 3641 Acked Read 5 Sent Done Done false
10163 3642 Acked Write 3 Sent Done Done false
10164 3643 Acked Write 2 Sent Done Done false
10165 3644 Acked Write 2 Sent Done Done false
10166 3645 Acked Read 2 Sent Done Done false
10167 3646 Acked Write 2 Sent Done Done false
10168 3647 Acked Write 5 Sent Done Done false
10169 3648 Acked Write 2 Sent Done Done false
10170 3649 Acked Flush 0 Sent Done Done false
10171 3650 Acked Write 10 Sent Done Done false
10172 3651 Acked Write 5 Sent Done Done false
10173 3652 Acked Read 10 Sent Done Done false
10174 3653 Acked Write 9 Sent Done Done false
10175 3654 Acked Read 6 Sent Done Done false
10176 3655 Acked Flush 0 Sent Done Done false
10177 3656 Acked Write 1 Sent Done Done false
10178 3657 Acked Write 9 Sent Done Done false
10179 3658 Acked Write 9 Sent Done Done false
10180 3659 Acked Read 9 Sent Done Done false
10181 3660 Acked Write 2 Sent Done Done false
10182 3661 Acked Write 3 Sent Done Done false
10183 3662 Acked Write 1 Sent Done Done false
10184 3663 Acked Write 5 Sent Done Done false
10185 3664 Acked Read 8 Sent Done Done false
10186 3665 Acked Read 2 Sent Done Done false
10187 3666 Acked Read 8 Sent Done Done false
10188 3667 Acked Read 3 Sent Done Done false
10189 3668 Acked Read 2 Sent Done Done false
10190 3669 Acked Read 4 Sent Done Done false
10191 3670 Acked Read 2 Sent Done Done false
10192 3671 Acked Read 5 Sent Done Done false
10193 3672 Acked Read 8 Sent Done Done false
10194 3673 Acked Write 8 Sent Done Done false
10195 3674 Acked Flush 0 Sent Done Done false
10196 3675 Acked Write 8 Sent Done Done false
10197 3676 Acked Read 7 Sent Done Done false
10198 3677 Acked Flush 0 Sent Done Done false
10199 3678 Acked Read 10 Sent Done Done false
10200 3679 Acked Flush 0 Sent Done Done false
10201 3680 Acked Write 2 Sent Done Done false
10202 3681 Acked Write 1 Sent Done Done false
10203 3682 Acked Read 3 Sent Done Done false
10204 3683 Acked Write 4 Sent Done Done false
10205 3684 Acked Write 6 Sent Done Done false
10206 3685 Acked Write 4 Sent Done Done false
10207 3686 Acked Read 6 Sent Done Done false
10208 3687 Acked Read 6 Sent Done Done false
10209 3688 Acked Read 6 Sent Done Done false
10210 3689 Acked Write 8 Sent Done Done false
10211 3690 Acked Read 10 Sent Done Done false
10212 3691 Acked Read 10 Sent Done Done false
10213 3692 Acked Read 6 Sent Done Done false
10214 3693 Acked Write 2 Sent Done Done false
10215 3694 Acked Read 4 Sent Done Done false
10216 3695 Acked Write 4 Sent Done Done false
10217 3696 Acked Read 9 Sent Done Done false
10218 3697 Acked Read 10 Sent Done Done false
10219 3698 Acked Read 3 Sent Done Done false
10220 3699 Acked Read 3 Sent Done Done false
10221 3700 Acked Read 2 Sent Done Done false
10222 3701 Acked Write 10 Sent Done Done false
10223 3702 Acked Write 2 Sent Done Done false
10224 3703 Acked Read 6 Sent Done Done false
10225 3704 Acked Read 1 Sent Done Done false
10226 3705 Acked Write 5 Sent Done Done false
10227 3706 Acked Read 9 Sent Done Done false
10228 3707 Acked Write 3 Sent Done Done false
10229 3708 Acked Write 10 Sent Done Done false
10230 3709 Acked Write 5 Sent Done Done false
10231 3710 Acked Read 9 Sent Done Done false
10232 3711 Acked Read 5 Sent Done Done false
10233 3712 Acked Write 6 Sent Done Done false
10234 3713 Acked Read 3 Sent Done Done false
10235 3714 Acked Flush 0 Sent Done Done false
10236 3715 Acked Read 10 Sent Done Done false
10237 3716 Acked Flush 0 Sent Done Done false
10238 STATES DS:0 DS:1 DS:2 TOTAL
10239 Sent 302 0 0 302
10240 Done 0 302 302 604
10241 Skipped 0 0 0 0
10242 Error 0 0 0 0
10243 Last Flush: 3414 3716 3716
10244 Downstairs last five completed: 3414 3413 3412 3411 3410
10245 Upstairs last five completed: 3716 3715 3714 3713 3712
10246 CLIENT: Up:0 ds:302 act:2
102472024-11-19T17:10:07.681ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
102482024-11-19T17:10:07.681ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
102492024-11-19T17:10:07.681ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
102502024-11-19T17:10:07.681ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 05650a15-902e-469c-91e5-cba45f07470e
102512024-11-19T17:10:07.681ZINFOcrucible: client task is exiting = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
10252 [8810] got start action
10253 [8810] I am going to restart
10254 Starting downstairs at port 8810
10255 Make output file at "/tmp/dsc/downstairs-8810.txt"
10256 [8810][0] reports Starting
10257 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
10258 [8810][0] reports Running
10259 ----------------------------------------------------------------
10260 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10261 States: Offline Active Active
10262 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10263 3415 Acked Read 7 Sent Done Done false
10264 3416 Acked Write 7 Sent Done Done false
10265 3417 Acked Read 8 Sent Done Done false
10266 3418 Acked Write 5 Sent Done Done false
10267 3419 Acked Read 10 Sent Done Done false
10268 3420 Acked Read 3 Sent Done Done false
10269 3421 Acked Write 3 Sent Done Done false
10270 3422 Acked Flush 0 Sent Done Done false
10271 3423 Acked Flush 0 Sent Done Done false
10272 3424 Acked Read 1 Sent Done Done false
10273 3425 Acked Read 2 Sent Done Done false
10274 3426 Acked Read 9 Sent Done Done false
10275 3427 Acked Write 6 Sent Done Done false
10276 3428 Acked Write 10 Sent Done Done false
10277 3429 Acked Read 4 Sent Done Done false
10278 3430 Acked Flush 0 Sent Done Done false
10279 3431 Acked Flush 0 Sent Done Done false
10280 3432 Acked Write 6 Sent Done Done false
10281 3433 Acked Write 4 Sent Done Done false
10282 3434 Acked Write 3 Sent Done Done false
10283 3435 Acked Read 1 Sent Done Done false
10284 3436 Acked Read 6 Sent Done Done false
10285 3437 Acked Read 1 Sent Done Done false
10286 3438 Acked Read 10 Sent Done Done false
10287 3439 Acked Flush 0 Sent Done Done false
10288 3440 Acked Read 7 Sent Done Done false
10289 3441 Acked Write 10 Sent Done Done false
10290 3442 Acked Read 7 Sent Done Done false
10291 3443 Acked Flush 0 Sent Done Done false
10292 3444 Acked Write 9 Sent Done Done false
10293 3445 Acked Write 10 Sent Done Done false
10294 3446 Acked Write 10 Sent Done Done false
10295 3447 Acked Write 10 Sent Done Done false
10296 3448 Acked Read 8 Sent Done Done false
10297 3449 Acked Read 1 Sent Done Done false
10298 3450 Acked Read 4 Sent Done Done false
10299 3451 Acked Read 4 Sent Done Done false
10300 3452 Acked Read 10 Sent Done Done false
10301 3453 Acked Write 9 Sent Done Done false
10302 3454 Acked Read 9 Sent Done Done false
10303 3455 Acked Read 6 Sent Done Done false
10304 3456 Acked Write 10 Sent Done Done false
10305 3457 Acked Read 8 Sent Done Done false
10306 3458 Acked Read 3 Sent Done Done false
10307 3459 Acked Write 1 Sent Done Done false
10308 3460 Acked Read 8 Sent Done Done false
10309 3461 Acked Flush 0 Sent Done Done false
10310 3462 Acked Write 6 Sent Done Done false
10311 3463 Acked Read 5 Sent Done Done false
10312 3464 Acked Read 10 Sent Done Done false
10313 3465 Acked Flush 0 Sent Done Done false
10314 3466 Acked Write 1 Sent Done Done false
10315 3467 Acked Read 6 Sent Done Done false
10316 3468 Acked Read 9 Sent Done Done false
10317 3469 Acked Flush 0 Sent Done Done false
10318 3470 Acked Read 8 Sent Done Done false
10319 3471 Acked Read 8 Sent Done Done false
10320 3472 Acked Read 6 Sent Done Done false
10321 3473 Acked Read 7 Sent Done Done false
10322 3474 Acked Write 4 Sent Done Done false
10323 3475 Acked Write 9 Sent Done Done false
10324 3476 Acked Write 9 Sent Done Done false
10325 3477 Acked Read 5 Sent Done Done false
10326 3478 Acked Read 5 Sent Done Done false
10327 3479 Acked Read 7 Sent Done Done false
10328 3480 Acked Read 9 Sent Done Done false
10329 3481 Acked Read 7 Sent Done Done false
10330 3482 Acked Write 10 Sent Done Done false
10331 3483 Acked Flush 0 Sent Done Done false
10332 3484 Acked Write 4 Sent Done Done false
10333 3485 Acked Read 9 Sent Done Done false
10334 3486 Acked Read 8 Sent Done Done false
10335 3487 Acked Write 3 Sent Done Done false
10336 3488 Acked Read 1 Sent Done Done false
10337 3489 Acked Read 1 Sent Done Done false
10338 3490 Acked Read 1 Sent Done Done false
10339 3491 Acked Read 5 Sent Done Done false
10340 3492 Acked Write 7 Sent Done Done false
10341 3493 Acked Read 5 Sent Done Done false
10342 3494 Acked Write 9 Sent Done Done false
10343 3495 Acked Write 6 Sent Done Done false
10344 3496 Acked Read 1 Sent Done Done false
10345 3497 Acked Write 2 Sent Done Done false
10346 3498 Acked Flush 0 Sent Done Done false
10347 3499 Acked Write 5 Sent Done Done false
10348 3500 Acked Read 1 Sent Done Done false
10349 3501 Acked Flush 0 Sent Done Done false
10350 3502 Acked Write 8 Sent Done Done false
10351 3503 Acked Read 9 Sent Done Done false
10352 3504 Acked Write 4 Sent Done Done false
10353 3505 Acked Write 8 Sent Done Done false
10354 3506 Acked Flush 0 Sent Done Done false
10355 3507 Acked Read 4 Sent Done Done false
10356 3508 Acked Read 1 Sent Done Done false
10357 3509 Acked Write 6 Sent Done Done false
10358 3510 Acked Read 9 Sent Done Done false
10359 3511 Acked Write 2 Sent Done Done false
10360 3512 Acked Flush 0 Sent Done Done false
10361 3513 Acked Write 2 Sent Done Done false
10362 3514 Acked Read 3 Sent Done Done false
10363 3515 Acked Write 4 Sent Done Done false
10364 3516 Acked Write 5 Sent Done Done false
10365 3517 Acked Read 3 Sent Done Done false
10366 3518 Acked Read 6 Sent Done Done false
10367 3519 Acked Read 8 Sent Done Done false
10368 3520 Acked Read 9 Sent Done Done false
10369 3521 Acked Read 10 Sent Done Done false
10370 3522 Acked Read 8 Sent Done Done false
10371 3523 Acked Read 10 Sent Done Done false
10372 3524 Acked Read 4 Sent Done Done false
10373 3525 Acked Write 4 Sent Done Done false
10374 3526 Acked Read 10 Sent Done Done false
10375 3527 Acked Read 8 Sent Done Done false
10376 3528 Acked Read 1 Sent Done Done false
10377 3529 Acked Flush 0 Sent Done Done false
10378 3530 Acked Read 1 Sent Done Done false
10379 3531 Acked Write 3 Sent Done Done false
10380 3532 Acked Flush 0 Sent Done Done false
10381 3533 Acked Write 3 Sent Done Done false
10382 3534 Acked Flush 0 Sent Done Done false
10383 3535 Acked Write 3 Sent Done Done false
10384 3536 Acked Write 8 Sent Done Done false
10385 3537 Acked Read 4 Sent Done Done false
10386 3538 Acked Read 8 Sent Done Done false
10387 3539 Acked Read 3 Sent Done Done false
10388 3540 Acked Write 9 Sent Done Done false
10389 3541 Acked Read 2 Sent Done Done false
10390 3542 Acked Write 10 Sent Done Done false
10391 3543 Acked Flush 0 Sent Done Done false
10392 3544 Acked Read 8 Sent Done Done false
10393 3545 Acked Read 9 Sent Done Done false
10394 3546 Acked Read 3 Sent Done Done false
10395 3547 Acked Read 4 Sent Done Done false
10396 3548 Acked Read 2 Sent Done Done false
10397 3549 Acked Write 7 Sent Done Done false
10398 3550 Acked Write 3 Sent Done Done false
10399 3551 Acked Flush 0 Sent Done Done false
10400 3552 Acked Read 5 Sent Done Done false
10401 3553 Acked Read 10 Sent Done Done false
10402 3554 Acked Read 8 Sent Done Done false
10403 3555 Acked Read 1 Sent Done Done false
10404 3556 Acked Read 4 Sent Done Done false
10405 3557 Acked Write 3 Sent Done Done false
10406 3558 Acked Read 2 Sent Done Done false
10407 3559 Acked Read 1 Sent Done Done false
10408 3560 Acked Read 1 Sent Done Done false
10409 3561 Acked Write 1 Sent Done Done false
10410 3562 Acked Read 3 Sent Done Done false
10411 3563 Acked Write 10 Sent Done Done false
10412 3564 Acked Write 7 Sent Done Done false
10413 3565 Acked Write 9 Sent Done Done false
10414 3566 Acked Read 8 Sent Done Done false
10415 3567 Acked Write 1 Sent Done Done false
10416 3568 Acked Write 9 Sent Done Done false
10417 3569 Acked Read 3 Sent Done Done false
10418 3570 Acked Read 5 Sent Done Done false
10419 3571 Acked Write 8 Sent Done Done false
10420 3572 Acked Flush 0 Sent Done Done false
10421 3573 Acked Write 9 Sent Done Done false
10422 3574 Acked Read 8 Sent Done Done false
10423 3575 Acked Write 3 Sent Done Done false
10424 3576 Acked Read 8 Sent Done Done false
10425 3577 Acked Read 4 Sent Done Done false
10426 3578 Acked Read 1 Sent Done Done false
10427 3579 Acked Read 9 Sent Done Done false
10428 3580 Acked Flush 0 Sent Done Done false
10429 3581 Acked Read 8 Sent Done Done false
10430 3582 Acked Read 6 Sent Done Done false
10431 3583 Acked Write 5 Sent Done Done false
10432 3584 Acked Read 9 Sent Done Done false
10433 3585 Acked Write 7 Sent Done Done false
10434 3586 Acked Write 3 Sent Done Done false
10435 3587 Acked Flush 0 Sent Done Done false
10436 3588 Acked Read 4 Sent Done Done false
10437 3589 Acked Flush 0 Sent Done Done false
10438 3590 Acked Read 8 Sent Done Done false
10439 3591 Acked Read 2 Sent Done Done false
10440 3592 Acked Read 3 Sent Done Done false
10441 3593 Acked Read 5 Sent Done Done false
10442 3594 Acked Write 10 Sent Done Done false
10443 3595 Acked Read 4 Sent Done Done false
10444 3596 Acked Read 4 Sent Done Done false
10445 3597 Acked Write 3 Sent Done Done false
10446 3598 Acked Write 7 Sent Done Done false
10447 3599 Acked Read 10 Sent Done Done false
10448 3600 Acked Read 9 Sent Done Done false
10449 3601 Acked Write 3 Sent Done Done false
10450 3602 Acked Flush 0 Sent Done Done false
10451 3603 Acked Write 7 Sent Done Done false
10452 3604 Acked Read 5 Sent Done Done false
10453 3605 Acked Flush 0 Sent Done Done false
10454 3606 Acked Write 4 Sent Done Done false
10455 3607 Acked Read 1 Sent Done Done false
10456 3608 Acked Read 3 Sent Done Done false
10457 3609 Acked Write 6 Sent Done Done false
10458 3610 Acked Write 7 Sent Done Done false
10459 3611 Acked Write 5 Sent Done Done false
10460 3612 Acked Read 6 Sent Done Done false
10461 3613 Acked Read 3 Sent Done Done false
10462 3614 Acked Write 2 Sent Done Done false
10463 3615 Acked Write 3 Sent Done Done false
10464 3616 Acked Write 1 Sent Done Done false
10465 3617 Acked Write 3 Sent Done Done false
10466 3618 Acked Write 8 Sent Done Done false
10467 3619 Acked Write 4 Sent Done Done false
10468 3620 Acked Write 1 Sent Done Done false
10469 3621 Acked Read 7 Sent Done Done false
10470 3622 Acked Write 8 Sent Done Done false
10471 3623 Acked Write 4 Sent Done Done false
10472 3624 Acked Write 10 Sent Done Done false
10473 3625 Acked Read 6 Sent Done Done false
10474 3626 Acked Write 4 Sent Done Done false
10475 3627 Acked Write 2 Sent Done Done false
10476 3628 Acked Read 6 Sent Done Done false
10477 3629 Acked Read 1 Sent Done Done false
10478 3630 Acked Write 2 Sent Done Done false
10479 3631 Acked Write 7 Sent Done Done false
10480 3632 Acked Write 6 Sent Done Done false
10481 3633 Acked Read 2 Sent Done Done false
10482 3634 Acked Read 7 Sent Done Done false
10483 3635 Acked Read 4 Sent Done Done false
10484 3636 Acked Read 8 Sent Done Done false
10485 3637 Acked Read 4 Sent Done Done false
10486 3638 Acked Read 1 Sent Done Done false
10487 3639 Acked Read 1 Sent Done Done false
10488 3640 Acked Write 8 Sent Done Done false
10489 3641 Acked Read 5 Sent Done Done false
10490 3642 Acked Write 3 Sent Done Done false
10491 3643 Acked Write 2 Sent Done Done false
10492 3644 Acked Write 2 Sent Done Done false
10493 3645 Acked Read 2 Sent Done Done false
10494 3646 Acked Write 2 Sent Done Done false
10495 3647 Acked Write 5 Sent Done Done false
10496 3648 Acked Write 2 Sent Done Done false
10497 3649 Acked Flush 0 Sent Done Done false
10498 3650 Acked Write 10 Sent Done Done false
10499 3651 Acked Write 5 Sent Done Done false
10500 3652 Acked Read 10 Sent Done Done false
10501 3653 Acked Write 9 Sent Done Done false
10502 3654 Acked Read 6 Sent Done Done false
10503 3655 Acked Flush 0 Sent Done Done false
10504 3656 Acked Write 1 Sent Done Done false
10505 3657 Acked Write 9 Sent Done Done false
10506 3658 Acked Write 9 Sent Done Done false
10507 3659 Acked Read 9 Sent Done Done false
10508 3660 Acked Write 2 Sent Done Done false
10509 3661 Acked Write 3 Sent Done Done false
10510 3662 Acked Write 1 Sent Done Done false
10511 3663 Acked Write 5 Sent Done Done false
10512 3664 Acked Read 8 Sent Done Done false
10513 3665 Acked Read 2 Sent Done Done false
10514 3666 Acked Read 8 Sent Done Done false
10515 3667 Acked Read 3 Sent Done Done false
10516 3668 Acked Read 2 Sent Done Done false
10517 3669 Acked Read 4 Sent Done Done false
10518 3670 Acked Read 2 Sent Done Done false
10519 3671 Acked Read 5 Sent Done Done false
10520 3672 Acked Read 8 Sent Done Done false
10521 3673 Acked Write 8 Sent Done Done false
10522 3674 Acked Flush 0 Sent Done Done false
10523 3675 Acked Write 8 Sent Done Done false
10524 3676 Acked Read 7 Sent Done Done false
10525 3677 Acked Flush 0 Sent Done Done false
10526 3678 Acked Read 10 Sent Done Done false
10527 3679 Acked Flush 0 Sent Done Done false
10528 3680 Acked Write 2 Sent Done Done false
10529 3681 Acked Write 1 Sent Done Done false
10530 3682 Acked Read 3 Sent Done Done false
10531 3683 Acked Write 4 Sent Done Done false
10532 3684 Acked Write 6 Sent Done Done false
10533 3685 Acked Write 4 Sent Done Done false
10534 3686 Acked Read 6 Sent Done Done false
10535 3687 Acked Read 6 Sent Done Done false
10536 3688 Acked Read 6 Sent Done Done false
10537 3689 Acked Write 8 Sent Done Done false
10538 3690 Acked Read 10 Sent Done Done false
10539 3691 Acked Read 10 Sent Done Done false
10540 3692 Acked Read 6 Sent Done Done false
10541 3693 Acked Write 2 Sent Done Done false
10542 3694 Acked Read 4 Sent Done Done false
10543 3695 Acked Write 4 Sent Done Done false
10544 3696 Acked Read 9 Sent Done Done false
10545 3697 Acked Read 10 Sent Done Done false
10546 3698 Acked Read 3 Sent Done Done false
10547 3699 Acked Read 3 Sent Done Done false
10548 3700 Acked Read 2 Sent Done Done false
10549 3701 Acked Write 10 Sent Done Done false
10550 3702 Acked Write 2 Sent Done Done false
10551 3703 Acked Read 6 Sent Done Done false
10552 3704 Acked Read 1 Sent Done Done false
10553 3705 Acked Write 5 Sent Done Done false
10554 3706 Acked Read 9 Sent Done Done false
10555 3707 Acked Write 3 Sent Done Done false
10556 3708 Acked Write 10 Sent Done Done false
10557 3709 Acked Write 5 Sent Done Done false
10558 3710 Acked Read 9 Sent Done Done false
10559 3711 Acked Read 5 Sent Done Done false
10560 3712 Acked Write 6 Sent Done Done false
10561 3713 Acked Read 3 Sent Done Done false
10562 3714 Acked Flush 0 Sent Done Done false
10563 3715 Acked Read 10 Sent Done Done false
10564 3716 Acked Flush 0 Sent Done Done false
10565 STATES DS:0 DS:1 DS:2 TOTAL
10566 Sent 302 0 0 302
10567 Done 0 302 302 604
10568 Skipped 0 0 0 0
10569 Error 0 0 0 0
10570 Last Flush: 3414 3716 3716
10571 Downstairs last five completed: 3414 3413 3412 3411 3410
10572 Upstairs last five completed: 3716 3715 3714 3713 3712
10573 CLIENT: Up:0 ds:302 act:2
10574 ----------------------------------------------------------------
10575 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10576 States: Offline Active Active
10577 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10578 3415 Acked Read 7 Sent Done Done false
10579 3416 Acked Write 7 Sent Done Done false
10580 3417 Acked Read 8 Sent Done Done false
10581 3418 Acked Write 5 Sent Done Done false
10582 3419 Acked Read 10 Sent Done Done false
10583 3420 Acked Read 3 Sent Done Done false
10584 3421 Acked Write 3 Sent Done Done false
10585 3422 Acked Flush 0 Sent Done Done false
10586 3423 Acked Flush 0 Sent Done Done false
10587 3424 Acked Read 1 Sent Done Done false
10588 3425 Acked Read 2 Sent Done Done false
10589 3426 Acked Read 9 Sent Done Done false
10590 3427 Acked Write 6 Sent Done Done false
10591 3428 Acked Write 10 Sent Done Done false
10592 3429 Acked Read 4 Sent Done Done false
10593 3430 Acked Flush 0 Sent Done Done false
10594 3431 Acked Flush 0 Sent Done Done false
10595 3432 Acked Write 6 Sent Done Done false
10596 3433 Acked Write 4 Sent Done Done false
10597 3434 Acked Write 3 Sent Done Done false
10598 3435 Acked Read 1 Sent Done Done false
10599 3436 Acked Read 6 Sent Done Done false
10600 3437 Acked Read 1 Sent Done Done false
10601 3438 Acked Read 10 Sent Done Done false
10602 3439 Acked Flush 0 Sent Done Done false
10603 3440 Acked Read 7 Sent Done Done false
10604 3441 Acked Write 10 Sent Done Done false
10605 3442 Acked Read 7 Sent Done Done false
10606 3443 Acked Flush 0 Sent Done Done false
10607 3444 Acked Write 9 Sent Done Done false
10608 3445 Acked Write 10 Sent Done Done false
10609 3446 Acked Write 10 Sent Done Done false
10610 3447 Acked Write 10 Sent Done Done false
10611 3448 Acked Read 8 Sent Done Done false
10612 3449 Acked Read 1 Sent Done Done false
10613 3450 Acked Read 4 Sent Done Done false
10614 3451 Acked Read 4 Sent Done Done false
10615 3452 Acked Read 10 Sent Done Done false
10616 3453 Acked Write 9 Sent Done Done false
10617 3454 Acked Read 9 Sent Done Done false
10618 3455 Acked Read 6 Sent Done Done false
10619 3456 Acked Write 10 Sent Done Done false
10620 3457 Acked Read 8 Sent Done Done false
10621 3458 Acked Read 3 Sent Done Done false
10622 3459 Acked Write 1 Sent Done Done false
10623 3460 Acked Read 8 Sent Done Done false
10624 3461 Acked Flush 0 Sent Done Done false
10625 3462 Acked Write 6 Sent Done Done false
10626 3463 Acked Read 5 Sent Done Done false
10627 3464 Acked Read 10 Sent Done Done false
10628 3465 Acked Flush 0 Sent Done Done false
10629 3466 Acked Write 1 Sent Done Done false
10630 3467 Acked Read 6 Sent Done Done false
10631 3468 Acked Read 9 Sent Done Done false
10632 3469 Acked Flush 0 Sent Done Done false
10633 3470 Acked Read 8 Sent Done Done false
10634 3471 Acked Read 8 Sent Done Done false
10635 3472 Acked Read 6 Sent Done Done false
10636 3473 Acked Read 7 Sent Done Done false
10637 3474 Acked Write 4 Sent Done Done false
10638 3475 Acked Write 9 Sent Done Done false
10639 3476 Acked Write 9 Sent Done Done false
10640 3477 Acked Read 5 Sent Done Done false
10641 3478 Acked Read 5 Sent Done Done false
10642 3479 Acked Read 7 Sent Done Done false
10643 3480 Acked Read 9 Sent Done Done false
10644 3481 Acked Read 7 Sent Done Done false
10645 3482 Acked Write 10 Sent Done Done false
10646 3483 Acked Flush 0 Sent Done Done false
10647 3484 Acked Write 4 Sent Done Done false
10648 3485 Acked Read 9 Sent Done Done false
10649 3486 Acked Read 8 Sent Done Done false
10650 3487 Acked Write 3 Sent Done Done false
10651 3488 Acked Read 1 Sent Done Done false
10652 3489 Acked Read 1 Sent Done Done false
10653 3490 Acked Read 1 Sent Done Done false
10654 3491 Acked Read 5 Sent Done Done false
10655 3492 Acked Write 7 Sent Done Done false
10656 3493 Acked Read 5 Sent Done Done false
10657 3494 Acked Write 9 Sent Done Done false
10658 3495 Acked Write 6 Sent Done Done false
10659 3496 Acked Read 1 Sent Done Done false
10660 3497 Acked Write 2 Sent Done Done false
10661 3498 Acked Flush 0 Sent Done Done false
10662 3499 Acked Write 5 Sent Done Done false
10663 3500 Acked Read 1 Sent Done Done false
10664 3501 Acked Flush 0 Sent Done Done false
10665 3502 Acked Write 8 Sent Done Done false
10666 3503 Acked Read 9 Sent Done Done false
10667 3504 Acked Write 4 Sent Done Done false
10668 3505 Acked Write 8 Sent Done Done false
10669 3506 Acked Flush 0 Sent Done Done false
10670 3507 Acked Read 4 Sent Done Done false
10671 3508 Acked Read 1 Sent Done Done false
10672 3509 Acked Write 6 Sent Done Done false
10673 3510 Acked Read 9 Sent Done Done false
10674 3511 Acked Write 2 Sent Done Done false
10675 3512 Acked Flush 0 Sent Done Done false
10676 3513 Acked Write 2 Sent Done Done false
10677 3514 Acked Read 3 Sent Done Done false
10678 3515 Acked Write 4 Sent Done Done false
10679 3516 Acked Write 5 Sent Done Done false
10680 3517 Acked Read 3 Sent Done Done false
10681 3518 Acked Read 6 Sent Done Done false
10682 3519 Acked Read 8 Sent Done Done false
10683 3520 Acked Read 9 Sent Done Done false
10684 3521 Acked Read 10 Sent Done Done false
10685 3522 Acked Read 8 Sent Done Done false
10686 3523 Acked Read 10 Sent Done Done false
10687 3524 Acked Read 4 Sent Done Done false
10688 3525 Acked Write 4 Sent Done Done false
10689 3526 Acked Read 10 Sent Done Done false
10690 3527 Acked Read 8 Sent Done Done false
10691 3528 Acked Read 1 Sent Done Done false
10692 3529 Acked Flush 0 Sent Done Done false
10693 3530 Acked Read 1 Sent Done Done false
10694 3531 Acked Write 3 Sent Done Done false
10695 3532 Acked Flush 0 Sent Done Done false
10696 3533 Acked Write 3 Sent Done Done false
10697 3534 Acked Flush 0 Sent Done Done false
10698 3535 Acked Write 3 Sent Done Done false
10699 3536 Acked Write 8 Sent Done Done false
10700 3537 Acked Read 4 Sent Done Done false
10701 3538 Acked Read 8 Sent Done Done false
10702 3539 Acked Read 3 Sent Done Done false
10703 3540 Acked Write 9 Sent Done Done false
10704 3541 Acked Read 2 Sent Done Done false
10705 3542 Acked Write 10 Sent Done Done false
10706 3543 Acked Flush 0 Sent Done Done false
10707 3544 Acked Read 8 Sent Done Done false
10708 3545 Acked Read 9 Sent Done Done false
10709 3546 Acked Read 3 Sent Done Done false
10710 3547 Acked Read 4 Sent Done Done false
10711 3548 Acked Read 2 Sent Done Done false
10712 3549 Acked Write 7 Sent Done Done false
10713 3550 Acked Write 3 Sent Done Done false
10714 3551 Acked Flush 0 Sent Done Done false
10715 3552 Acked Read 5 Sent Done Done false
10716 3553 Acked Read 10 Sent Done Done false
10717 3554 Acked Read 8 Sent Done Done false
10718 3555 Acked Read 1 Sent Done Done false
10719 3556 Acked Read 4 Sent Done Done false
10720 3557 Acked Write 3 Sent Done Done false
10721 3558 Acked Read 2 Sent Done Done false
10722 3559 Acked Read 1 Sent Done Done false
10723 3560 Acked Read 1 Sent Done Done false
10724 3561 Acked Write 1 Sent Done Done false
10725 3562 Acked Read 3 Sent Done Done false
10726 3563 Acked Write 10 Sent Done Done false
10727 3564 Acked Write 7 Sent Done Done false
10728 3565 Acked Write 9 Sent Done Done false
10729 3566 Acked Read 8 Sent Done Done false
10730 3567 Acked Write 1 Sent Done Done false
10731 3568 Acked Write 9 Sent Done Done false
10732 3569 Acked Read 3 Sent Done Done false
10733 3570 Acked Read 5 Sent Done Done false
10734 3571 Acked Write 8 Sent Done Done false
10735 3572 Acked Flush 0 Sent Done Done false
10736 3573 Acked Write 9 Sent Done Done false
10737 3574 Acked Read 8 Sent Done Done false
10738 3575 Acked Write 3 Sent Done Done false
10739 3576 Acked Read 8 Sent Done Done false
10740 3577 Acked Read 4 Sent Done Done false
10741 3578 Acked Read 1 Sent Done Done false
10742 3579 Acked Read 9 Sent Done Done false
10743 3580 Acked Flush 0 Sent Done Done false
10744 3581 Acked Read 8 Sent Done Done false
10745 3582 Acked Read 6 Sent Done Done false
10746 3583 Acked Write 5 Sent Done Done false
10747 3584 Acked Read 9 Sent Done Done false
10748 3585 Acked Write 7 Sent Done Done false
10749 3586 Acked Write 3 Sent Done Done false
10750 3587 Acked Flush 0 Sent Done Done false
10751 3588 Acked Read 4 Sent Done Done false
10752 3589 Acked Flush 0 Sent Done Done false
10753 3590 Acked Read 8 Sent Done Done false
10754 3591 Acked Read 2 Sent Done Done false
10755 3592 Acked Read 3 Sent Done Done false
10756 3593 Acked Read 5 Sent Done Done false
10757 3594 Acked Write 10 Sent Done Done false
10758 3595 Acked Read 4 Sent Done Done false
10759 3596 Acked Read 4 Sent Done Done false
10760 3597 Acked Write 3 Sent Done Done false
10761 3598 Acked Write 7 Sent Done Done false
10762 3599 Acked Read 10 Sent Done Done false
10763 3600 Acked Read 9 Sent Done Done false
10764 3601 Acked Write 3 Sent Done Done false
10765 3602 Acked Flush 0 Sent Done Done false
10766 3603 Acked Write 7 Sent Done Done false
10767 3604 Acked Read 5 Sent Done Done false
10768 3605 Acked Flush 0 Sent Done Done false
10769 3606 Acked Write 4 Sent Done Done false
10770 3607 Acked Read 1 Sent Done Done false
10771 3608 Acked Read 3 Sent Done Done false
10772 3609 Acked Write 6 Sent Done Done false
10773 3610 Acked Write 7 Sent Done Done false
10774 3611 Acked Write 5 Sent Done Done false
10775 3612 Acked Read 6 Sent Done Done false
10776 3613 Acked Read 3 Sent Done Done false
10777 3614 Acked Write 2 Sent Done Done false
10778 3615 Acked Write 3 Sent Done Done false
10779 3616 Acked Write 1 Sent Done Done false
10780 3617 Acked Write 3 Sent Done Done false
10781 3618 Acked Write 8 Sent Done Done false
10782 3619 Acked Write 4 Sent Done Done false
10783 3620 Acked Write 1 Sent Done Done false
10784 3621 Acked Read 7 Sent Done Done false
10785 3622 Acked Write 8 Sent Done Done false
10786 3623 Acked Write 4 Sent Done Done false
10787 3624 Acked Write 10 Sent Done Done false
10788 3625 Acked Read 6 Sent Done Done false
10789 3626 Acked Write 4 Sent Done Done false
10790 3627 Acked Write 2 Sent Done Done false
10791 3628 Acked Read 6 Sent Done Done false
10792 3629 Acked Read 1 Sent Done Done false
10793 3630 Acked Write 2 Sent Done Done false
10794 3631 Acked Write 7 Sent Done Done false
10795 3632 Acked Write 6 Sent Done Done false
10796 3633 Acked Read 2 Sent Done Done false
10797 3634 Acked Read 7 Sent Done Done false
10798 3635 Acked Read 4 Sent Done Done false
10799 3636 Acked Read 8 Sent Done Done false
10800 3637 Acked Read 4 Sent Done Done false
10801 3638 Acked Read 1 Sent Done Done false
10802 3639 Acked Read 1 Sent Done Done false
10803 3640 Acked Write 8 Sent Done Done false
10804 3641 Acked Read 5 Sent Done Done false
10805 3642 Acked Write 3 Sent Done Done false
10806 3643 Acked Write 2 Sent Done Done false
10807 3644 Acked Write 2 Sent Done Done false
10808 3645 Acked Read 2 Sent Done Done false
10809 3646 Acked Write 2 Sent Done Done false
10810 3647 Acked Write 5 Sent Done Done false
10811 3648 Acked Write 2 Sent Done Done false
10812 3649 Acked Flush 0 Sent Done Done false
10813 3650 Acked Write 10 Sent Done Done false
10814 3651 Acked Write 5 Sent Done Done false
10815 3652 Acked Read 10 Sent Done Done false
10816 3653 Acked Write 9 Sent Done Done false
10817 3654 Acked Read 6 Sent Done Done false
10818 3655 Acked Flush 0 Sent Done Done false
10819 3656 Acked Write 1 Sent Done Done false
10820 3657 Acked Write 9 Sent Done Done false
10821 3658 Acked Write 9 Sent Done Done false
10822 3659 Acked Read 9 Sent Done Done false
10823 3660 Acked Write 2 Sent Done Done false
10824 3661 Acked Write 3 Sent Done Done false
10825 3662 Acked Write 1 Sent Done Done false
10826 3663 Acked Write 5 Sent Done Done false
10827 3664 Acked Read 8 Sent Done Done false
10828 3665 Acked Read 2 Sent Done Done false
10829 3666 Acked Read 8 Sent Done Done false
10830 3667 Acked Read 3 Sent Done Done false
10831 3668 Acked Read 2 Sent Done Done false
10832 3669 Acked Read 4 Sent Done Done false
10833 3670 Acked Read 2 Sent Done Done false
10834 3671 Acked Read 5 Sent Done Done false
10835 3672 Acked Read 8 Sent Done Done false
10836 3673 Acked Write 8 Sent Done Done false
10837 3674 Acked Flush 0 Sent Done Done false
10838 3675 Acked Write 8 Sent Done Done false
10839 3676 Acked Read 7 Sent Done Done false
10840 3677 Acked Flush 0 Sent Done Done false
10841 3678 Acked Read 10 Sent Done Done false
10842 3679 Acked Flush 0 Sent Done Done false
10843 3680 Acked Write 2 Sent Done Done false
10844 3681 Acked Write 1 Sent Done Done false
10845 3682 Acked Read 3 Sent Done Done false
10846 3683 Acked Write 4 Sent Done Done false
10847 3684 Acked Write 6 Sent Done Done false
10848 3685 Acked Write 4 Sent Done Done false
10849 3686 Acked Read 6 Sent Done Done false
10850 3687 Acked Read 6 Sent Done Done false
10851 3688 Acked Read 6 Sent Done Done false
10852 3689 Acked Write 8 Sent Done Done false
10853 3690 Acked Read 10 Sent Done Done false
10854 3691 Acked Read 10 Sent Done Done false
10855 3692 Acked Read 6 Sent Done Done false
10856 3693 Acked Write 2 Sent Done Done false
10857 3694 Acked Read 4 Sent Done Done false
10858 3695 Acked Write 4 Sent Done Done false
10859 3696 Acked Read 9 Sent Done Done false
10860 3697 Acked Read 10 Sent Done Done false
10861 3698 Acked Read 3 Sent Done Done false
10862 3699 Acked Read 3 Sent Done Done false
10863 3700 Acked Read 2 Sent Done Done false
10864 3701 Acked Write 10 Sent Done Done false
10865 3702 Acked Write 2 Sent Done Done false
10866 3703 Acked Read 6 Sent Done Done false
10867 3704 Acked Read 1 Sent Done Done false
10868 3705 Acked Write 5 Sent Done Done false
10869 3706 Acked Read 9 Sent Done Done false
10870 3707 Acked Write 3 Sent Done Done false
10871 3708 Acked Write 10 Sent Done Done false
10872 3709 Acked Write 5 Sent Done Done false
10873 3710 Acked Read 9 Sent Done Done false
10874 3711 Acked Read 5 Sent Done Done false
10875 3712 Acked Write 6 Sent Done Done false
10876 3713 Acked Read 3 Sent Done Done false
10877 3714 Acked Flush 0 Sent Done Done false
10878 3715 Acked Read 10 Sent Done Done false
10879 3716 Acked Flush 0 Sent Done Done false
10880 STATES DS:0 DS:1 DS:2 TOTAL
10881 Sent 302 0 0 302
10882 Done 0 302 302 604
10883 Skipped 0 0 0 0
10884 Error 0 0 0 0
10885 Last Flush: 3414 3716 3716
10886 Downstairs last five completed: 3414 3413 3412 3411 3410
10887 Upstairs last five completed: 3716 3715 3714 3713 3712
10888 CLIENT: Up:0 ds:302 act:2
108892024-11-19T17:10:17.683ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108902024-11-19T17:10:17.683ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64184) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108912024-11-19T17:10:17.684ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108922024-11-19T17:10:17.684ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108932024-11-19T17:10:17.684ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108942024-11-19T17:10:17.684ZINFOcrucible: send last flush ID to this DS: 3414 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108952024-11-19T17:10:17.684ZINFOcrucible: Replied this last flush ID: 3414 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108962024-11-19T17:10:17.684ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108972024-11-19T17:10:17.684ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
108982024-11-19T17:10:17.684ZINFOcrucible: [0] client re-new 302 jobs since flush 3414 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
108992024-11-19T17:10:17.685ZINFOcrucible: [0] Replayed 302 jobs since flush: 3414 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
10900 ----------------------------------------------------------------
10901 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10902 Downstairs last five completed: 3716 3715 3714 3713 3712
10903 Upstairs last five completed: 3716 3715 3714 3713 3712
10904 CLIENT: Up:0 ds:0 act:3
10905 Replay: All jobs finished
10906 Main task has work to do, go find it
10907 got dsc Stop(2)
10908 stop 2
10909 Nov 19 17:10:17.931 INFO request completed[8830] Got stop action so:false kr:false
10910 , latency_us: 327, response_code: 204, uri: /stop/cid/2, method: POST, req_id: cebaecbf-9030-460e-a944-049fa4480980, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
109112024-11-19T17:10:17.931ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
109122024-11-19T17:10:17.931ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
109132024-11-19T17:10:17.931ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
109142024-11-19T17:10:17.931ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
109152024-11-19T17:10:17.931ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
109162024-11-19T17:10:17.931ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
10917 Nov 19 17:10:17.931 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4715318e-7259-4f3f-9288-5be15d4732c5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
10918 [8830] Exited with: ExitStatus(unix_wait_status(9))
10919 [8830][2] reports Exit
10920 Nov 19 17:10:21.934 INFO request completed, latency_us: 257, response_code: 200, uri: /state/cid/2, method: GET, req_id: af3b342b-dda6-422c-9f6e-f619df21d2c4, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
10921 001/300 Read block 4976 len 24576
10922 002/300 Read block 1481 len 28672
10923 003/300 Read block 19 len 20480
10924 004/300 Read block 4310 len 28672
10925 005/300 Read block 4755 len 28672
10926 006/300 Read block 2298 len 28672
10927 007/300 Flush
10928 008/300 Read block 937 len 4096
10929 009/300 Read block 3196 len 32768
10930 010/300 Read block 2069 len 40960
10931 011/300 Write block 462 len 4096 data: 3
10932 012/300 Read block 3903 len 4096
10933 013/300 Read block 4511 len 16384
10934 014/300 Write block 1143 len 28672 data: 2 3 3 3 3 3 3
10935 015/300 Write block 4381 len 40960 data: 3 3 3 2 2 2 2 2 2 2
10936 016/300 Write block 1477 len 24576 data: 4 4 4 4 4 4
10937 017/300 Read block 4161 len 40960
10938 018/300 Read block 4534 len 16384
10939 019/300 Read block 1670 len 40960
10940 020/300 Flush
10941 021/300 Write block 856 len 28672 data: 2 2 3 2 3 3 3
10942 022/300 Read block 2913 len 24576
10943 023/300 Write block 4663 len 4096 data: 4
10944 024/300 Read block 4427 len 28672
10945 025/300 Write block 1553 len 28672 data: 2 2 2 3 3 3 3
10946 026/300 Flush
10947 027/300 Flush
10948 028/300 Flush
10949 029/300 Write block 1559 len 32768 data: 4 3 3 3 3 3 3 3
10950 030/300 Read block 680 len 12288
10951 031/300 Read block 1921 len 40960
10952 032/300 Write block 1678 len 28672 data: 3 4 3 3 2 2 2
10953 033/300 Read block 729 len 12288
10954 034/300 Write block 4717 len 16384 data: 2 3 3 3
10955 035/300 Write block 4882 len 4096 data: 2
10956 036/300 Read block 1212 len 32768
10957 037/300 Write block 47 len 8192 data: 3 3
10958 038/300 Flush
10959 039/300 Write block 2813 len 8192 data: 4 3
10960 040/300 Write block 1286 len 20480 data: 6 5 5 7 7
10961 041/300 Read block 3248 len 32768
10962 042/300 Write block 4845 len 20480 data: 4 4 3 3 4
10963 043/300 Read block 4875 len 28672
10964 044/300 Read block 819 len 20480
10965 045/300 Read block 4127 len 8192
10966 046/300 Read block 167 len 20480
10967 047/300 Write block 1098 len 8192 data: 2 3
10968 048/300 Read block 2769 len 8192
10969 049/300 Flush
10970 050/300 Write block 3659 len 24576 data: 3 4 4 4 3 2
10971 051/300 Read block 27 len 24576
10972 052/300 Read block 1782 len 8192
10973 053/300 Write block 2867 len 8192 data: 2 2
10974 054/300 Read block 3870 len 16384
10975 055/300 Read block 4527 len 36864
10976 056/300 Write block 1371 len 24576 data: 3 4 5 4 4 3
10977 057/300 Write block 4534 len 16384 data: 3 3 3 3
10978 058/300 Read block 3093 len 20480
10979 059/300 Flush
10980 060/300 Read block 3159 len 24576
10981 061/300 Write block 4195 len 12288 data: 3 3 3
10982 062/300 Read block 2003 len 12288
10983 063/300 Read block 2721 len 20480
10984 064/300 Read block 2449 len 12288
10985 065/300 Write block 2076 len 32768 data: 3 4 4 4 3 2 2 2
10986 066/300 Write block 3906 len 24576 data: 2 2 2 3 3 3
10987 067/300 Read block 3660 len 28672
10988 068/300 Read block 716 len 16384
10989 069/300 Read block 1809 len 32768
10990 070/300 Write block 1712 len 4096 data: 3
10991 071/300 Flush
10992 072/300 Read block 1699 len 8192
10993 073/300 Write block 2617 len 24576 data: 5 3 4 4 4 4
10994 074/300 Read block 1942 len 32768
10995 075/300 Write block 2154 len 4096 data: 4
10996 076/300 Write block 1287 len 24576 data: 6 6 8 8 5 4
10997 077/300 Write block 4750 len 4096 data: 2
10998 078/300 Read block 2008 len 4096
10999 079/300 Flush
11000 080/300 Flush
11001 081/300 Read block 2702 len 12288
11002 082/300 Read block 2500 len 36864
11003 083/300 Write block 2997 len 4096 data: 5
11004 084/300 Write block 3574 len 12288 data: 2 2 2
11005 085/300 Write block 2425 len 12288 data: 3 3 3
11006 086/300 Flush
11007 087/300 Read block 172 len 28672
11008 088/300 Write block 1628 len 32768 data: 3 3 3 3 3 2 2 2
11009 089/300 Flush
11010 090/300 Read block 3758 len 16384
11011 091/300 Flush
11012 092/300 Read block 4396 len 16384
11013 093/300 Write block 3826 len 4096 data: 5
11014 094/300 Write block 3951 len 12288 data: 3 3 3
11015 095/300 Write block 426 len 28672 data: 5 4 4 4 4 4 4
11016 096/300 Write block 4266 len 32768 data: 4 4 4 3 2 2 2 3
11017 097/300 Write block 2930 len 8192 data: 5 5
11018 098/300 Write block 4927 len 16384 data: 6 5 5 4
11019 099/300 Flush
11020 100/300 Read block 2500 len 24576
11021 101/300 Read block 4576 len 24576
11022 102/300 Read block 2204 len 24576
11023 103/300 Read block 1474 len 36864
11024 104/300 Flush
11025 105/300 Read block 584 len 4096
11026 106/300 Write block 1620 len 28672 data: 3 3 3 3 3 3 2
11027 107/300 Read block 3039 len 36864
11028 108/300 Read block 2952 len 20480
11029 109/300 Write block 4260 len 28672 data: 2 2 2 2 4 4 5
11030 110/300 Read block 3232 len 4096
11031 111/300 Read block 2290 len 40960
11032 112/300 Write block 2146 len 32768 data: 2 2 2 3 3 3 3 3
11033 113/300 Read block 2817 len 20480
11034 114/300 Read block 3343 len 24576
11035 115/300 Write block 956 len 12288 data: 2 2 2
11036 116/300 Write block 198 len 40960 data: 3 3 3 4 4 4 4 3 2 2
11037 117/300 Write block 1951 len 8192 data: 3 3
11038 118/300 Read block 454 len 20480
11039 119/300 Write block 3695 len 24576 data: 2 3 4 4 4 4
11040 120/300 Read block 665 len 32768
11041 121/300 Write block 2196 len 12288 data: 2 2 2
11042 122/300 Read block 3756 len 24576
11043 123/300 Read block 1558 len 16384
11044 124/300 Read block 4240 len 8192
11045 125/300 Write block 3427 len 28672 data: 2 2 2 2 2 2 2
11046 126/300 Read block 4641 len 28672
11047 127/300 Read block 1162 len 36864
11048 128/300 Read block 189 len 12288
11049 129/300 Read block 4781 len 20480
11050 130/300 Read block 2058 len 12288
11051 131/300 Flush
11052 132/300 Flush
11053 133/300 Write block 3987 len 20480 data: 3 3 3 3 3
11054 134/300 Write block 1743 len 12288 data: 4 4 3
11055 135/300 Read block 3685 len 40960
11056 136/300 Read block 2439 len 8192
11057 137/300 Write block 3776 len 32768 data: 3 3 3 3 3 2 2 2
11058 138/300 Read block 4886 len 24576
11059 139/300 Flush
11060 140/300 Flush
11061 141/300 Read block 3489 len 28672
11062 142/300 Write block 2778 len 4096 data: 3
11063 143/300 Write block 1988 len 28672 data: 3 2 2 3 3 3 2
11064 144/300 Write block 3954 len 40960 data: 3 3 3 4 3 3 3 3 3 3
11065 145/300 Read block 2494 len 12288
11066 146/300 Read block 4513 len 24576
11067 147/300 Write block 4478 len 16384 data: 3 3 3 3
11068 148/300 Read block 2663 len 12288
11069 149/300 Write block 2307 len 20480 data: 3 2 3 2 2
11070 150/300 Write block 2281 len 4096 data: 5
11071 151/300 Flush
11072 152/300 Write block 673 len 28672 data: 5 5 5 4 3 3 3
11073 153/300 Write block 4726 len 28672 data: 3 3 3 2 2 3 4
11074 154/300 Write block 286 len 36864 data: 2 2 2 2 2 2 3 3 3
11075 155/300 Read block 1529 len 8192
11076 156/300 Read block 4099 len 28672
11077 157/300 Write block 546 len 4096 data: 3
11078 158/300 Write block 1599 len 12288 data: 4 4 4
11079 159/300 Flush
11080 160/300 Write block 1908 len 4096 data: 4
11081 161/300 Write block 2703 len 20480 data: 4 4 5 5 5
11082 162/300 Read block 3934 len 24576
11083 163/300 Write block 1910 len 40960 data: 2 2 2 2 2 2 2 2 2 2
11084 164/300 Write block 2202 len 12288 data: 3 3 4
11085 165/300 Read block 4492 len 16384
11086 166/300 Read block 1223 len 20480
11087 167/300 Read block 1455 len 40960
11088 168/300 Flush
11089 169/300 Read block 1539 len 4096
11090 170/300 Read block 4797 len 40960
11091 171/300 Read block 3869 len 20480
11092 172/300 Write block 3159 len 40960 data: 2 2 2 3 4 5 4 4 4 3
11093 173/300 Read block 4123 len 32768
11094 174/300 Read block 2739 len 32768
11095 175/300 Write block 4921 len 20480 data: 2 2 3 3 3
11096 176/300 Write block 1875 len 40960 data: 2 2 3 5 5 4 4 4 6 6
11097 177/300 Write block 4005 len 4096 data: 4
11098 178/300 Read block 655 len 4096
11099 179/300 Write block 4773 len 28672 data: 3 4 4 5 4 4 3
11100 180/300 Flush
11101 181/300 Read block 1412 len 40960
11102 182/300 Write block 2060 len 40960 data: 2 2 2 3 4 4 4 4 4 4
11103 183/300 Flush
11104 184/300 Read block 4197 len 12288
11105 185/300 Read block 257 len 28672
11106 186/300 Read block 3561 len 8192
11107 187/300 Read block 1140 len 4096
11108 188/300 Write block 801 len 16384 data: 2 2 2 2
11109 189/300 Read block 2942 len 24576
11110 190/300 Write block 4296 len 8192 data: 2 3
11111 191/300 Read block 1141 len 24576
11112 192/300 Write block 2391 len 8192 data: 4 5
11113 193/300 Write block 1624 len 24576 data: 4 4 3 2 4 4
11114 194/300 Write block 3245 len 36864 data: 3 2 2 2 3 3 3 3 3
11115 195/300 Read block 4221 len 8192
11116 196/300 Read block 903 len 36864
11117 197/300 Read block 770 len 36864
11118 198/300 Read block 4987 len 40960
11119 199/300 Read block 531 len 16384
11120 200/300 Read block 1876 len 32768
11121 201/300 Write block 1866 len 24576 data: 4 4 4 4 4 3
11122 202/300 Write block 2286 len 4096 data: 5
11123 203/300 Flush
11124 204/300 Read block 4846 len 36864
11125 205/300 Read block 4026 len 8192
11126 206/300 Read block 3400 len 12288
11127 207/300 Write block 703 len 24576 data: 2 3 3 2 2 2
11128 208/300 Write block 3635 len 16384 data: 2 2 2 2
11129 209/300 Read block 270 len 32768
11130 210/300 Read block 4018 len 8192
11131 211/300 Read block 525 len 16384
11132 212/300 Read block 3770 len 32768
11133 213/300 Flush
11134 214/300 Read block 2445 len 32768
11135 215/300 Read block 2039 len 36864
11136 216/300 Flush
11137 217/300 Write block 1935 len 32768 data: 3 4 5 5 5 5 4 4
11138 218/300 Write block 2036 len 32768 data: 3 3 3 3 2 2 2 2
11139 219/300 Write block 3747 len 28672 data: 5 5 5 4 4 5 5
11140 220/300 Flush
11141 221/300 Read block 1436 len 40960
11142 222/300 Read block 4793 len 28672
11143 223/300 Read block 1173 len 40960
11144 224/300 Write block 3134 len 16384 data: 3 2 2 2
11145 225/300 Read block 1387 len 36864
11146 226/300 Read block 4672 len 24576
11147 227/300 Write block 448 len 16384 data: 2 2 2 2
11148 228/300 Write block 4462 len 20480 data: 3 4 3 3 3
11149 229/300 Flush
11150 230/300 Read block 1864 len 8192
11151 231/300 Flush
11152 232/300 Read block 3288 len 28672
11153 233/300 Write block 3600 len 12288 data: 3 4 3
11154 234/300 Read block 959 len 40960
11155 235/300 Read block 2506 len 12288
11156 236/300 Read block 1031 len 28672
11157 237/300 Read block 1131 len 4096
11158 238/300 Write block 709 len 4096 data: 2
11159 239/300 Write block 1068 len 8192 data: 4 3
11160 240/300 Read block 2232 len 8192
11161 241/300 Flush
11162 242/300 Flush
11163 243/300 Read block 3899 len 4096
11164 244/300 Flush
11165 245/300 Write block 701 len 12288 data: 2 2 3
11166 246/300 Write block 2611 len 16384 data: 6 5 5 5
11167 247/300 Flush
11168 248/300 Read block 156 len 28672
11169 249/300 Read block 4794 len 40960
11170 250/300 Read block 2039 len 32768
11171 251/300 Flush
11172 252/300 Read block 3747 len 12288
11173 253/300 Write block 3693 len 36864 data: 2 2 3 4 5 5 5 5 4
11174 254/300 Read block 1708 len 32768
11175 255/300 Write block 1757 len 32768 data: 5 4 4 4 2 2 2 2
11176 256/300 Flush
11177 257/300 Read block 1627 len 8192
11178 258/300 Flush
11179 259/300 Read block 4596 len 16384
11180 260/300 Read block 4476 len 16384
11181 261/300 Flush
11182 262/300 Write block 1943 len 8192 data: 4 3
11183 263/300 Read block 1763 len 8192
11184 264/300 Read block 199 len 16384
11185 265/300 Write block 3830 len 4096 data: 4
11186 266/300 Read block 1481 len 20480
11187 267/300 Write block 2976 len 36864 data: 3 3 3 3 4 3 2 2 2
11188 268/300 Read block 3106 len 20480
11189 269/300 Read block 866 len 36864
11190 270/300 Write block 4526 len 32768 data: 2 2 2 3 3 3 3 3
11191 271/300 Read block 2930 len 40960
11192 272/300 Read block 1954 len 24576
11193 273/300 Read block 2678 len 36864
11194 274/300 Write block 2149 len 12288 data: 4 4 4
11195 275/300 Read block 3457 len 16384
11196 276/300 Write block 1381 len 8192 data: 3 3
11197 277/300 Write block 2388 len 24576 data: 3 4 4 5 6 5
11198 278/300 Flush
11199 279/300 Write block 2697 len 24576 data: 3 2 2 3 3 3
11200 280/300 Read block 3372 len 40960
11201 281/300 Write block 514 len 16384 data: 2 2 2 3
11202 282/300 Write block 3070 len 12288 data: 4 3 3
11203 283/300 Write block 4918 len 20480 data: 2 2 2 3 3
11204 284/300 Read block 4674 len 28672
11205 285/300 Flush
11206 286/300 Flush
11207 287/300 Read block 365 len 40960
11208 288/300 Read block 3220 len 4096
11209 289/300 Write block 3985 len 40960 data: 2 2 4 4 4 4 4 4 4 4
11210 290/300 Read block 4590 len 28672
11211 291/300 Write block 3463 len 20480 data: 3 5 5 5 4
11212 292/300 Read block 92 len 40960
11213 293/300 Read block 786 len 4096
11214 294/300 Read block 1380 len 36864
11215 295/300 Read block 4707 len 4096
11216 296/300 Write block 100 len 20480 data: 5 5 4 4 4
11217 297/300 Flush
11218 298/300 Write block 575 len 24576 data: 2 2 2 2 2 2
11219 299/300 Read block 1434 len 28672
11220 300/300 Read block 652 len 12288
11221 301/300 Read block 4737 len 32768
11222 Main task has work to do, go find it
11223 got dsc Start(2)
11224 start 2
11225 Nov 19 17:10:22.172 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/2, method: POST, req_id: 7824efda-7541-4fe5-8fdb-d5f36cc68d70, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
11226 Replay: started 2, returned:Ok(())
11227 ----------------------------------------------------------------
11228 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11229 States: Active Active Offline
11230 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11231 3717 Acked Read 6 Done Done Sent false
11232 3718 Acked Read 7 Done Done Sent false
11233 3719 Acked Read 5 Done Done Sent false
11234 3720 Acked Read 7 Done Done Sent false
11235 3721 Acked Read 7 Done Done Sent false
11236 3722 Acked Read 7 Done Done Sent false
11237 3723 Acked Flush 0 Done Done Sent false
11238 3724 Acked Read 1 Done Done Sent false
11239 3725 Acked Read 8 Done Done Sent false
11240 3726 Acked Read 10 Done Done Sent false
11241 3727 Acked Write 1 Done Done Sent false
11242 3728 Acked Read 1 Done Done Sent false
11243 3729 Acked Read 4 Done Done Sent false
11244 3730 Acked Write 7 Done Done Sent false
11245 3731 Acked Write 10 Done Done Sent false
11246 3732 Acked Write 6 Done Done Sent false
11247 3733 Acked Read 10 Done Done Sent false
11248 3734 Acked Read 4 Done Done Sent false
11249 3735 Acked Read 10 Done Done Sent false
11250 3736 Acked Flush 0 Done Done Sent false
11251 3737 Acked Write 7 Done Done Sent false
11252 3738 Acked Read 6 Done Done Sent false
11253 3739 Acked Write 1 Done Done Sent false
11254 3740 Acked Read 7 Done Done Sent false
11255 3741 Acked Write 7 Done Done Sent false
11256 3742 Acked Flush 0 Done Done Sent false
11257 3743 Acked Flush 0 Done Done Sent false
11258 3744 Acked Flush 0 Done Done Sent false
11259 3745 Acked Write 8 Done Done Sent false
11260 3746 Acked Read 3 Done Done Sent false
11261 3747 Acked Read 10 Done Done Sent false
11262 3748 Acked Write 7 Done Done Sent false
11263 3749 Acked Read 3 Done Done Sent false
11264 3750 Acked Write 4 Done Done Sent false
11265 3751 Acked Write 1 Done Done Sent false
11266 3752 Acked Read 8 Done Done Sent false
11267 3753 Acked Write 2 Done Done Sent false
11268 3754 Acked Flush 0 Done Done Sent false
11269 3755 Acked Write 2 Done Done Sent false
11270 3756 Acked Write 5 Done Done Sent false
11271 3757 Acked Read 8 Done Done Sent false
11272 3758 Acked Write 5 Done Done Sent false
11273 3759 Acked Read 7 Done Done Sent false
11274 3760 Acked Read 5 Done Done Sent false
11275 3761 Acked Read 2 Done Done Sent false
11276 3762 Acked Read 5 Done Done Sent false
11277 3763 Acked Write 2 Done Done Sent false
11278 3764 Acked Read 2 Done Done Sent false
11279 3765 Acked Flush 0 Done Done Sent false
11280 3766 Acked Write 6 Done Done Sent false
11281 3767 Acked Read 6 Done Done Sent false
11282 3768 Acked Read 2 Done Done Sent false
11283 3769 Acked Write 2 Done Done Sent false
11284 3770 Acked Read 4 Done Done Sent false
11285 3771 Acked Read 9 Done Done Sent false
11286 3772 Acked Write 6 Done Done Sent false
11287 3773 Acked Write 4 Done Done Sent false
11288 3774 Acked Read 5 Done Done Sent false
11289 3775 Acked Flush 0 Done Done Sent false
11290 3776 Acked Read 6 Done Done Sent false
11291 3777 Acked Write 3 Done Done Sent false
11292 3778 Acked Read 3 Done Done Sent false
11293 3779 Acked Read 5 Done Done Sent false
11294 3780 Acked Read 3 Done Done Sent false
11295 3781 Acked Write 8 Done Done Sent false
11296 3782 Acked Write 6 Done Done Sent false
11297 3783 Acked Read 7 Done Done Sent false
11298 3784 Acked Read 4 Done Done Sent false
11299 3785 Acked Read 8 Done Done Sent false
11300 3786 Acked Write 1 Done Done Sent false
11301 3787 Acked Flush 0 Done Done Sent false
11302 3788 Acked Read 2 Done Done Sent false
11303 3789 Acked Write 6 Done Done Sent false
11304 3790 Acked Read 8 Done Done Sent false
11305 3791 Acked Write 1 Done Done Sent false
11306 3792 Acked Write 6 Done Done Sent false
11307 3793 Acked Write 1 Done Done Sent false
11308 3794 Acked Read 1 Done Done Sent false
11309 3795 Acked Flush 0 Done Done Sent false
11310 3796 Acked Flush 0 Done Done Sent false
11311 3797 Acked Read 3 Done Done Sent false
11312 3798 Acked Read 9 Done Done Sent false
11313 3799 Acked Write 1 Done Done Sent false
11314 3800 Acked Write 3 Done Done Sent false
11315 3801 Acked Write 3 Done Done Sent false
11316 3802 Acked Flush 0 Done Done Sent false
11317 3803 Acked Read 7 Done Done Sent false
11318 3804 Acked Write 8 Done Done Sent false
11319 3805 Acked Flush 0 Done Done Sent false
11320 3806 Acked Read 4 Done Done Sent false
11321 3807 Acked Flush 0 Done Done Sent false
11322 3808 Acked Read 4 Done Done Sent false
11323 3809 Acked Write 1 Done Done Sent false
11324 3810 Acked Write 3 Done Done Sent false
11325 3811 Acked Write 7 Done Done Sent false
11326 3812 Acked Write 8 Done Done Sent false
11327 3813 Acked Write 2 Done Done Sent false
11328 3814 Acked Write 4 Done Done Sent false
11329 3815 Acked Flush 0 Done Done Sent false
11330 3816 Acked Read 6 Done Done Sent false
11331 3817 Acked Read 6 Done Done Sent false
11332 3818 Acked Read 6 Done Done Sent false
11333 3819 Acked Read 9 Done Done Sent false
11334 3820 Acked Flush 0 Done Done Sent false
11335 3821 Acked Read 1 Done Done Sent false
11336 3822 Acked Write 7 Done Done Sent false
11337 3823 Acked Read 9 Done Done Sent false
11338 3824 Acked Read 5 Done Done Sent false
11339 3825 Acked Write 7 Done Done Sent false
11340 3826 Acked Read 1 Done Done Sent false
11341 3827 Acked Read 10 Done Done Sent false
11342 3828 Acked Write 8 Done Done Sent false
11343 3829 Acked Read 5 Done Done Sent false
11344 3830 Acked Read 6 Done Done Sent false
11345 3831 Acked Write 3 Done Done Sent false
11346 3832 Acked Write 10 Done Done Sent false
11347 3833 Acked Write 2 Done Done Sent false
11348 3834 Acked Read 5 Done Done Sent false
11349 3835 Acked Write 6 Done Done Sent false
11350 3836 Acked Read 8 Done Done Sent false
11351 3837 Acked Write 3 Done Done Sent false
11352 3838 Acked Read 6 Done Done Sent false
11353 3839 Acked Read 4 Done Done Sent false
11354 3840 Acked Read 2 Done Done Sent false
11355 3841 Acked Write 7 Done Done Sent false
11356 3842 Acked Read 7 Done Done Sent false
11357 3843 Acked Read 9 Done Done Sent false
11358 3844 Acked Read 3 Done Done Sent false
11359 3845 Acked Read 5 Done Done Sent false
11360 3846 Acked Read 3 Done Done Sent false
11361 3847 Acked Flush 0 Done Done Sent false
11362 3848 Acked Flush 0 Done Done Sent false
11363 3849 Acked Write 5 Done Done Sent false
11364 3850 Acked Write 3 Done Done Sent false
11365 3851 Acked Read 10 Done Done Sent false
11366 3852 Acked Read 2 Done Done Sent false
11367 3853 Acked Write 8 Done Done Sent false
11368 3854 Acked Read 6 Done Done Sent false
11369 3855 Acked Flush 0 Done Done Sent false
11370 3856 Acked Flush 0 Done Done Sent false
11371 3857 Acked Read 7 Done Done Sent false
11372 3858 Acked Write 1 Done Done Sent false
11373 3859 Acked Write 7 Done Done Sent false
11374 3860 Acked Write 10 Done Done Sent false
11375 3861 Acked Read 3 Done Done Sent false
11376 3862 Acked Read 6 Done Done Sent false
11377 3863 Acked Write 4 Done Done Sent false
11378 3864 Acked Read 3 Done Done Sent false
11379 3865 Acked Write 5 Done Done Sent false
11380 3866 Acked Write 1 Done Done Sent false
11381 3867 Acked Flush 0 Done Done Sent false
11382 3868 Acked Write 7 Done Done Sent false
11383 3869 Acked Write 7 Done Done Sent false
11384 3870 Acked Write 9 Done Done Sent false
11385 3871 Acked Read 2 Done Done Sent false
11386 3872 Acked Read 7 Done Done Sent false
11387 3873 Acked Write 1 Done Done Sent false
11388 3874 Acked Write 3 Done Done Sent false
11389 3875 Acked Flush 0 Done Done Sent false
11390 3876 Acked Write 1 Done Done Sent false
11391 3877 Acked Write 5 Done Done Sent false
11392 3878 Acked Read 6 Done Done Sent false
11393 3879 Acked Write 10 Done Done Sent false
11394 3880 Acked Write 3 Done Done Sent false
11395 3881 Acked Read 4 Done Done Sent false
11396 3882 Acked Read 5 Done Done Sent false
11397 3883 Acked Read 10 Done Done Sent false
11398 3884 Acked Flush 0 Done Done Sent false
11399 3885 Acked Read 1 Done Done Sent false
11400 3886 Acked Read 10 Done Done Sent false
11401 3887 Acked Read 5 Done Done Sent false
11402 3888 Acked Write 10 Done Done Sent false
11403 3889 Acked Read 8 Done Done Sent false
11404 3890 Acked Read 8 Done Done Sent false
11405 3891 Acked Write 5 Done Done Sent false
11406 3892 Acked Write 10 Done Done Sent false
11407 3893 Acked Write 1 Done Done Sent false
11408 3894 Acked Read 1 Done Done Sent false
11409 3895 Acked Write 7 Done Done Sent false
11410 3896 Acked Flush 0 Done Done Sent false
11411 3897 Acked Read 10 Done Done Sent false
11412 3898 Acked Write 10 Done Done Sent false
11413 3899 Acked Flush 0 Done Done Sent false
11414 3900 Acked Read 3 Done Done Sent false
11415 3901 Acked Read 7 Done Done Sent false
11416 3902 Acked Read 2 Done Done Sent false
11417 3903 Acked Read 1 Done Done Sent false
11418 3904 Acked Write 4 Done Done Sent false
11419 3905 Acked Read 6 Done Done Sent false
11420 3906 Acked Write 2 Done Done Sent false
11421 3907 Acked Read 6 Done Done Sent false
11422 3908 Acked Write 2 Done Done Sent false
11423 3909 Acked Write 6 Done Done Sent false
11424 3910 Acked Write 9 Done Done Sent false
11425 3911 Acked Read 2 Done Done Sent false
11426 3912 Acked Read 9 Done Done Sent false
11427 3913 Acked Read 9 Done Done Sent false
11428 3914 Acked Read 10 Done Done Sent false
11429 3915 Acked Read 4 Done Done Sent false
11430 3916 Acked Read 8 Done Done Sent false
11431 3917 Acked Write 6 Done Done Sent false
11432 3918 Acked Write 1 Done Done Sent false
11433 3919 Acked Flush 0 Done Done Sent false
11434 3920 Acked Read 9 Done Done Sent false
11435 3921 Acked Read 2 Done Done Sent false
11436 3922 Acked Read 3 Done Done Sent false
11437 3923 Acked Write 6 Done Done Sent false
11438 3924 Acked Write 4 Done Done Sent false
11439 3925 Acked Read 8 Done Done Sent false
11440 3926 Acked Read 2 Done Done Sent false
11441 3927 Acked Read 4 Done Done Sent false
11442 3928 Acked Read 8 Done Done Sent false
11443 3929 Acked Flush 0 Done Done Sent false
11444 3930 Acked Read 8 Done Done Sent false
11445 3931 Acked Read 9 Done Done Sent false
11446 3932 Acked Flush 0 Done Done Sent false
11447 3933 Acked Write 8 Done Done Sent false
11448 3934 Acked Write 8 Done Done Sent false
11449 3935 Acked Write 7 Done Done Sent false
11450 3936 Acked Flush 0 Done Done Sent false
11451 3937 Acked Read 10 Done Done Sent false
11452 3938 Acked Read 7 Done Done Sent false
11453 3939 Acked Read 10 Done Done Sent false
11454 3940 Acked Write 4 Done Done Sent false
11455 3941 Acked Read 9 Done Done Sent false
11456 3942 Acked Read 6 Done Done Sent false
11457 3943 Acked Write 4 Done Done Sent false
11458 3944 Acked Write 5 Done Done Sent false
11459 3945 Acked Flush 0 Done Done Sent false
11460 3946 Acked Read 2 Done Done Sent false
11461 3947 Acked Flush 0 Done Done Sent false
11462 3948 Acked Read 7 Done Done Sent false
11463 3949 Acked Write 3 Done Done Sent false
11464 3950 Acked Read 10 Done Done Sent false
11465 3951 Acked Read 3 Done Done Sent false
11466 3952 Acked Read 7 Done Done Sent false
11467 3953 Acked Read 1 Done Done Sent false
11468 3954 Acked Write 1 Done Done Sent false
11469 3955 Acked Write 2 Done Done Sent false
11470 3956 Acked Read 2 Done Done Sent false
11471 3957 Acked Flush 0 Done Done Sent false
11472 3958 Acked Flush 0 Done Done Sent false
11473 3959 Acked Read 1 Done Done Sent false
11474 3960 Acked Flush 0 Done Done Sent false
11475 3961 Acked Write 3 Done Done Sent false
11476 3962 Acked Write 4 Done Done Sent false
11477 3963 Acked Flush 0 Done Done Sent false
11478 3964 Acked Read 7 Done Done Sent false
11479 3965 Acked Read 10 Done Done Sent false
11480 3966 Acked Read 8 Done Done Sent false
11481 3967 Acked Flush 0 Done Done Sent false
11482 3968 Acked Read 3 Done Done Sent false
11483 3969 Acked Write 9 Done Done Sent false
11484 3970 Acked Read 8 Done Done Sent false
11485 3971 Acked Write 8 Done Done Sent false
11486 3972 Acked Flush 0 Done Done Sent false
11487 3973 Acked Read 2 Done Done Sent false
11488 3974 Acked Flush 0 Done Done Sent false
11489 3975 Acked Read 4 Done Done Sent false
11490 3976 Acked Read 4 Done Done Sent false
11491 3977 Acked Flush 0 Done Done Sent false
11492 3978 Acked Write 2 Done Done Sent false
11493 3979 Acked Read 2 Done Done Sent false
11494 3980 Acked Read 4 Done Done Sent false
11495 3981 Acked Write 1 Done Done Sent false
11496 3982 Acked Read 5 Done Done Sent false
11497 3983 Acked Write 9 Done Done Sent false
11498 3984 Acked Read 5 Done Done Sent false
11499 3985 Acked Read 9 Done Done Sent false
11500 3986 Acked Write 8 Done Done Sent false
11501 3987 Acked Read 10 Done Done Sent false
11502 3988 Acked Read 6 Done Done Sent false
11503 3989 Acked Read 9 Done Done Sent false
11504 3990 Acked Write 3 Done Done Sent false
11505 3991 Acked Read 4 Done Done Sent false
11506 3992 Acked Write 2 Done Done Sent false
11507 3993 Acked Write 6 Done Done Sent false
11508 3994 Acked Flush 0 Done Done Sent false
11509 3995 Acked Write 6 Done Done Sent false
11510 3996 Acked Read 10 Done Done Sent false
11511 3997 Acked Write 4 Done Done Sent false
11512 3998 Acked Write 3 Done Done Sent false
11513 3999 Acked Write 5 Done Done Sent false
11514 4000 Acked Read 7 Done Done Sent false
11515 4001 Acked Flush 0 Done Done Sent false
11516 4002 Acked Flush 0 Done Done Sent false
11517 4003 Acked Read 10 Done Done Sent false
11518 4004 Acked Read 1 Done Done Sent false
11519 4005 Acked Write 10 Done Done Sent false
11520 4006 Acked Read 7 Done Done Sent false
11521 4007 Acked Write 5 Done Done Sent false
11522 4008 Acked Read 10 Done Done Sent false
11523 4009 Acked Read 1 Done Done Sent false
11524 4010 Acked Read 9 Done Done Sent false
11525 4011 Acked Read 1 Done Done Sent false
11526 4012 Acked Write 5 Done Done Sent false
11527 4013 Acked Flush 0 Done Done Sent false
11528 4014 Acked Write 6 Done Done Sent false
11529 4015 Acked Read 7 Done Done Sent false
11530 4016 Acked Read 3 Done Done Sent false
11531 4017 Acked Read 8 Done Done Sent false
11532 STATES DS:0 DS:1 DS:2 TOTAL
11533 Sent 0 0 301 301
11534 Done 301 301 0 602
11535 Skipped 0 0 0 0
11536 Error 0 0 0 0
11537 Last Flush: 4013 4013 3716
11538 Downstairs last five completed: 3716 3715 3714 3713 3712
11539 Upstairs last five completed: 4017 4016 4015 4014 4013
11540 CLIENT: Up:0 ds:301 act:2
11541 ----------------------------------------------------------------
11542 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11543 States: Active Active Offline
11544 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11545 3717 Acked Read 6 Done Done Sent false
11546 3718 Acked Read 7 Done Done Sent false
11547 3719 Acked Read 5 Done Done Sent false
11548 3720 Acked Read 7 Done Done Sent false
11549 3721 Acked Read 7 Done Done Sent false
11550 3722 Acked Read 7 Done Done Sent false
11551 3723 Acked Flush 0 Done Done Sent false
11552 3724 Acked Read 1 Done Done Sent false
11553 3725 Acked Read 8 Done Done Sent false
11554 3726 Acked Read 10 Done Done Sent false
11555 3727 Acked Write 1 Done Done Sent false
11556 3728 Acked Read 1 Done Done Sent false
11557 3729 Acked Read 4 Done Done Sent false
11558 3730 Acked Write 7 Done Done Sent false
11559 3731 Acked Write 10 Done Done Sent false
11560 3732 Acked Write 6 Done Done Sent false
11561 3733 Acked Read 10 Done Done Sent false
11562 3734 Acked Read 4 Done Done Sent false
11563 3735 Acked Read 10 Done Done Sent false
11564 3736 Acked Flush 0 Done Done Sent false
11565 3737 Acked Write 7 Done Done Sent false
11566 3738 Acked Read 6 Done Done Sent false
11567 3739 Acked Write 1 Done Done Sent false
11568 3740 Acked Read 7 Done Done Sent false
11569 3741 Acked Write 7 Done Done Sent false
11570 3742 Acked Flush 0 Done Done Sent false
11571 3743 Acked Flush 0 Done Done Sent false
11572 3744 Acked Flush 0 Done Done Sent false
11573 3745 Acked Write 8 Done Done Sent false
11574 3746 Acked Read 3 Done Done Sent false
11575 3747 Acked Read 10 Done Done Sent false
11576 3748 Acked Write 7 Done Done Sent false
11577 3749 Acked Read 3 Done Done Sent false
11578 3750 Acked Write 4 Done Done Sent false
11579 3751 Acked Write 1 Done Done Sent false
11580 3752 Acked Read 8 Done Done Sent false
11581 3753 Acked Write 2 Done Done Sent false
11582 3754 Acked Flush 0 Done Done Sent false
11583 3755 Acked Write 2 Done Done Sent false
11584 3756 Acked Write 5 Done Done Sent false
11585 3757 Acked Read 8 Done Done Sent false
11586 3758 Acked Write 5 Done Done Sent false
11587 3759 Acked Read 7 Done Done Sent false
11588 3760 Acked Read 5 Done Done Sent false
11589 3761 Acked Read 2 Done Done Sent false
11590 3762 Acked Read 5 Done Done Sent false
11591 3763 Acked Write 2 Done Done Sent false
11592 3764 Acked Read 2 Done Done Sent false
11593 3765 Acked Flush 0 Done Done Sent false
11594 3766 Acked Write 6 Done Done Sent false
11595 3767 Acked Read 6 Done Done Sent false
11596 3768 Acked Read 2 Done Done Sent false
11597 3769 Acked Write 2 Done Done Sent false
11598 3770 Acked Read 4 Done Done Sent false
11599 3771 Acked Read 9 Done Done Sent false
11600 3772 Acked Write 6 Done Done Sent false
11601 3773 Acked Write 4 Done Done Sent false
11602 3774 Acked Read 5 Done Done Sent false
11603 3775 Acked Flush 0 Done Done Sent false
11604 3776 Acked Read 6 Done Done Sent false
11605 3777 Acked Write 3 Done Done Sent false
11606 3778 Acked Read 3 Done Done Sent false
11607 3779 Acked Read 5 Done Done Sent false
11608 3780 Acked Read 3 Done Done Sent false
11609 3781 Acked Write 8 Done Done Sent false
11610 3782 Acked Write 6 Done Done Sent false
11611 3783 Acked Read 7 Done Done Sent false
11612 3784 Acked Read 4 Done Done Sent false
11613 3785 Acked Read 8 Done Done Sent false
11614 3786 Acked Write 1 Done Done Sent false
11615 3787 Acked Flush 0 Done Done Sent false
11616 3788 Acked Read 2 Done Done Sent false
11617 3789 Acked Write 6 Done Done Sent false
11618 3790 Acked Read 8 Done Done Sent false
11619 3791 Acked Write 1 Done Done Sent false
11620 3792 Acked Write 6 Done Done Sent false
11621 3793 Acked Write 1 Done Done Sent false
11622 3794 Acked Read 1 Done Done Sent false
11623 3795 Acked Flush 0 Done Done Sent false
11624 3796 Acked Flush 0 Done Done Sent false
11625 3797 Acked Read 3 Done Done Sent false
11626 3798 Acked Read 9 Done Done Sent false
11627 3799 Acked Write 1 Done Done Sent false
11628 3800 Acked Write 3 Done Done Sent false
11629 3801 Acked Write 3 Done Done Sent false
11630 3802 Acked Flush 0 Done Done Sent false
11631 3803 Acked Read 7 Done Done Sent false
11632 3804 Acked Write 8 Done Done Sent false
11633 3805 Acked Flush 0 Done Done Sent false
11634 3806 Acked Read 4 Done Done Sent false
11635 3807 Acked Flush 0 Done Done Sent false
11636 3808 Acked Read 4 Done Done Sent false
11637 3809 Acked Write 1 Done Done Sent false
11638 3810 Acked Write 3 Done Done Sent false
11639 3811 Acked Write 7 Done Done Sent false
11640 3812 Acked Write 8 Done Done Sent false
11641 3813 Acked Write 2 Done Done Sent false
11642 3814 Acked Write 4 Done Done Sent false
11643 3815 Acked Flush 0 Done Done Sent false
11644 3816 Acked Read 6 Done Done Sent false
11645 3817 Acked Read 6 Done Done Sent false
11646 3818 Acked Read 6 Done Done Sent false
11647 3819 Acked Read 9 Done Done Sent false
11648 3820 Acked Flush 0 Done Done Sent false
11649 3821 Acked Read 1 Done Done Sent false
11650 3822 Acked Write 7 Done Done Sent false
11651 3823 Acked Read 9 Done Done Sent false
11652 3824 Acked Read 5 Done Done Sent false
11653 3825 Acked Write 7 Done Done Sent false
11654 3826 Acked Read 1 Done Done Sent false
11655 3827 Acked Read 10 Done Done Sent false
11656 3828 Acked Write 8 Done Done Sent false
11657 3829 Acked Read 5 Done Done Sent false
11658 3830 Acked Read 6 Done Done Sent false
11659 3831 Acked Write 3 Done Done Sent false
11660 3832 Acked Write 10 Done Done Sent false
11661 3833 Acked Write 2 Done Done Sent false
11662 3834 Acked Read 5 Done Done Sent false
11663 3835 Acked Write 6 Done Done Sent false
11664 3836 Acked Read 8 Done Done Sent false
11665 3837 Acked Write 3 Done Done Sent false
11666 3838 Acked Read 6 Done Done Sent false
11667 3839 Acked Read 4 Done Done Sent false
11668 3840 Acked Read 2 Done Done Sent false
11669 3841 Acked Write 7 Done Done Sent false
11670 3842 Acked Read 7 Done Done Sent false
11671 3843 Acked Read 9 Done Done Sent false
11672 3844 Acked Read 3 Done Done Sent false
11673 3845 Acked Read 5 Done Done Sent false
11674 3846 Acked Read 3 Done Done Sent false
11675 3847 Acked Flush 0 Done Done Sent false
11676 3848 Acked Flush 0 Done Done Sent false
11677 3849 Acked Write 5 Done Done Sent false
11678 3850 Acked Write 3 Done Done Sent false
11679 3851 Acked Read 10 Done Done Sent false
11680 3852 Acked Read 2 Done Done Sent false
11681 3853 Acked Write 8 Done Done Sent false
11682 3854 Acked Read 6 Done Done Sent false
11683 3855 Acked Flush 0 Done Done Sent false
11684 3856 Acked Flush 0 Done Done Sent false
11685 3857 Acked Read 7 Done Done Sent false
11686 3858 Acked Write 1 Done Done Sent false
11687 3859 Acked Write 7 Done Done Sent false
11688 3860 Acked Write 10 Done Done Sent false
11689 3861 Acked Read 3 Done Done Sent false
11690 3862 Acked Read 6 Done Done Sent false
11691 3863 Acked Write 4 Done Done Sent false
11692 3864 Acked Read 3 Done Done Sent false
11693 3865 Acked Write 5 Done Done Sent false
11694 3866 Acked Write 1 Done Done Sent false
11695 3867 Acked Flush 0 Done Done Sent false
11696 3868 Acked Write 7 Done Done Sent false
11697 3869 Acked Write 7 Done Done Sent false
11698 3870 Acked Write 9 Done Done Sent false
11699 3871 Acked Read 2 Done Done Sent false
11700 3872 Acked Read 7 Done Done Sent false
11701 3873 Acked Write 1 Done Done Sent false
11702 3874 Acked Write 3 Done Done Sent false
11703 3875 Acked Flush 0 Done Done Sent false
11704 3876 Acked Write 1 Done Done Sent false
11705 3877 Acked Write 5 Done Done Sent false
11706 3878 Acked Read 6 Done Done Sent false
11707 3879 Acked Write 10 Done Done Sent false
11708 3880 Acked Write 3 Done Done Sent false
11709 3881 Acked Read 4 Done Done Sent false
11710 3882 Acked Read 5 Done Done Sent false
11711 3883 Acked Read 10 Done Done Sent false
11712 3884 Acked Flush 0 Done Done Sent false
11713 3885 Acked Read 1 Done Done Sent false
11714 3886 Acked Read 10 Done Done Sent false
11715 3887 Acked Read 5 Done Done Sent false
11716 3888 Acked Write 10 Done Done Sent false
11717 3889 Acked Read 8 Done Done Sent false
11718 3890 Acked Read 8 Done Done Sent false
11719 3891 Acked Write 5 Done Done Sent false
11720 3892 Acked Write 10 Done Done Sent false
11721 3893 Acked Write 1 Done Done Sent false
11722 3894 Acked Read 1 Done Done Sent false
11723 3895 Acked Write 7 Done Done Sent false
11724 3896 Acked Flush 0 Done Done Sent false
11725 3897 Acked Read 10 Done Done Sent false
11726 3898 Acked Write 10 Done Done Sent false
11727 3899 Acked Flush 0 Done Done Sent false
11728 3900 Acked Read 3 Done Done Sent false
11729 3901 Acked Read 7 Done Done Sent false
11730 3902 Acked Read 2 Done Done Sent false
11731 3903 Acked Read 1 Done Done Sent false
11732 3904 Acked Write 4 Done Done Sent false
11733 3905 Acked Read 6 Done Done Sent false
11734 3906 Acked Write 2 Done Done Sent false
11735 3907 Acked Read 6 Done Done Sent false
11736 3908 Acked Write 2 Done Done Sent false
11737 3909 Acked Write 6 Done Done Sent false
11738 3910 Acked Write 9 Done Done Sent false
11739 3911 Acked Read 2 Done Done Sent false
11740 3912 Acked Read 9 Done Done Sent false
11741 3913 Acked Read 9 Done Done Sent false
11742 3914 Acked Read 10 Done Done Sent false
11743 3915 Acked Read 4 Done Done Sent false
11744 3916 Acked Read 8 Done Done Sent false
11745 3917 Acked Write 6 Done Done Sent false
11746 3918 Acked Write 1 Done Done Sent false
11747 3919 Acked Flush 0 Done Done Sent false
11748 3920 Acked Read 9 Done Done Sent false
11749 3921 Acked Read 2 Done Done Sent false
11750 3922 Acked Read 3 Done Done Sent false
11751 3923 Acked Write 6 Done Done Sent false
11752 3924 Acked Write 4 Done Done Sent false
11753 3925 Acked Read 8 Done Done Sent false
11754 3926 Acked Read 2 Done Done Sent false
11755 3927 Acked Read 4 Done Done Sent false
11756 3928 Acked Read 8 Done Done Sent false
11757 3929 Acked Flush 0 Done Done Sent false
11758 3930 Acked Read 8 Done Done Sent false
11759 3931 Acked Read 9 Done Done Sent false
11760 3932 Acked Flush 0 Done Done Sent false
11761 3933 Acked Write 8 Done Done Sent false
11762 3934 Acked Write 8 Done Done Sent false
11763 3935 Acked Write 7 Done Done Sent false
11764 3936 Acked Flush 0 Done Done Sent false
11765 3937 Acked Read 10 Done Done Sent false
11766 3938 Acked Read 7 Done Done Sent false
11767 3939 Acked Read 10 Done Done Sent false
11768 3940 Acked Write 4 Done Done Sent false
11769 3941 Acked Read 9 Done Done Sent false
11770 3942 Acked Read 6 Done Done Sent false
11771 3943 Acked Write 4 Done Done Sent false
11772 3944 Acked Write 5 Done Done Sent false
11773 3945 Acked Flush 0 Done Done Sent false
11774 3946 Acked Read 2 Done Done Sent false
11775 3947 Acked Flush 0 Done Done Sent false
11776 3948 Acked Read 7 Done Done Sent false
11777 3949 Acked Write 3 Done Done Sent false
11778 3950 Acked Read 10 Done Done Sent false
11779 3951 Acked Read 3 Done Done Sent false
11780 3952 Acked Read 7 Done Done Sent false
11781 3953 Acked Read 1 Done Done Sent false
11782 3954 Acked Write 1 Done Done Sent false
11783 3955 Acked Write 2 Done Done Sent false
11784 3956 Acked Read 2 Done Done Sent false
11785 3957 Acked Flush 0 Done Done Sent false
11786 3958 Acked Flush 0 Done Done Sent false
11787 3959 Acked Read 1 Done Done Sent false
11788 3960 Acked Flush 0 Done Done Sent false
11789 3961 Acked Write 3 Done Done Sent false
11790 3962 Acked Write 4 Done Done Sent false
11791 3963 Acked Flush 0 Done Done Sent false
11792 3964 Acked Read 7 Done Done Sent false
11793 3965 Acked Read 10 Done Done Sent false
11794 3966 Acked Read 8 Done Done Sent false
11795 3967 Acked Flush 0 Done Done Sent false
11796 3968 Acked Read 3 Done Done Sent false
11797 3969 Acked Write 9 Done Done Sent false
11798 3970 Acked Read 8 Done Done Sent false
11799 3971 Acked Write 8 Done Done Sent false
11800 3972 Acked Flush 0 Done Done Sent false
11801 3973 Acked Read 2 Done Done Sent false
11802 3974 Acked Flush 0 Done Done Sent false
11803 3975 Acked Read 4 Done Done Sent false
11804 3976 Acked Read 4 Done Done Sent false
11805 3977 Acked Flush 0 Done Done Sent false
11806 3978 Acked Write 2 Done Done Sent false
11807 3979 Acked Read 2 Done Done Sent false
11808 3980 Acked Read 4 Done Done Sent false
11809 3981 Acked Write 1 Done Done Sent false
11810 3982 Acked Read 5 Done Done Sent false
11811 3983 Acked Write 9 Done Done Sent false
11812 3984 Acked Read 5 Done Done Sent false
11813 3985 Acked Read 9 Done Done Sent false
11814 3986 Acked Write 8 Done Done Sent false
11815 3987 Acked Read 10 Done Done Sent false
11816 3988 Acked Read 6 Done Done Sent false
11817 3989 Acked Read 9 Done Done Sent false
11818 3990 Acked Write 3 Done Done Sent false
11819 3991 Acked Read 4 Done Done Sent false
11820 3992 Acked Write 2 Done Done Sent false
11821 3993 Acked Write 6 Done Done Sent false
11822 3994 Acked Flush 0 Done Done Sent false
11823 3995 Acked Write 6 Done Done Sent false
11824 3996 Acked Read 10 Done Done Sent false
11825 3997 Acked Write 4 Done Done Sent false
11826 3998 Acked Write 3 Done Done Sent false
11827 3999 Acked Write 5 Done Done Sent false
11828 4000 Acked Read 7 Done Done Sent false
11829 4001 Acked Flush 0 Done Done Sent false
11830 4002 Acked Flush 0 Done Done Sent false
11831 4003 Acked Read 10 Done Done Sent false
11832 4004 Acked Read 1 Done Done Sent false
11833 4005 Acked Write 10 Done Done Sent false
11834 4006 Acked Read 7 Done Done Sent false
11835 4007 Acked Write 5 Done Done Sent false
11836 4008 Acked Read 10 Done Done Sent false
11837 4009 Acked Read 1 Done Done Sent false
11838 4010 Acked Read 9 Done Done Sent false
11839 4011 Acked Read 1 Done Done Sent false
11840 4012 Acked Write 5 Done Done Sent false
11841 4013 Acked Flush 0 Done Done Sent false
11842 4014 Acked Write 6 Done Done Sent false
11843 4015 Acked Read 7 Done Done Sent false
11844 4016 Acked Read 3 Done Done Sent false
11845 4017 Acked Read 8 Done Done Sent false
11846 4018 Acked Flush 0 Done Done Sent false
11847 STATES DS:0 DS:1 DS:2 TOTAL
11848 Sent 0 0 302 302
11849 Done 302 302 0 604
11850 Skipped 0 0 0 0
11851 Error 0 0 0 0
11852 Last Flush: 4018 4018 3716
11853 Downstairs last five completed: 3716 3715 3714 3713 3712
11854 Upstairs last five completed: 4018 4017 4016 4015 4014
11855 CLIENT: Up:0 ds:302 act:2
118562024-11-19T17:10:27.933ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
118572024-11-19T17:10:27.933ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
118582024-11-19T17:10:27.933ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
118592024-11-19T17:10:27.933ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 05650a15-902e-469c-91e5-cba45f07470e
118602024-11-19T17:10:27.933ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
11861 [8830] got start action
11862 [8830] I am going to restart
11863 Starting downstairs at port 8830
11864 Make output file at "/tmp/dsc/downstairs-8830.txt"
11865 [8830][2] reports Starting
11866 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1212)
11867 [8830][2] reports Running
11868 ----------------------------------------------------------------
11869 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11870 States: Active Active Offline
11871 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11872 3717 Acked Read 6 Done Done Sent false
11873 3718 Acked Read 7 Done Done Sent false
11874 3719 Acked Read 5 Done Done Sent false
11875 3720 Acked Read 7 Done Done Sent false
11876 3721 Acked Read 7 Done Done Sent false
11877 3722 Acked Read 7 Done Done Sent false
11878 3723 Acked Flush 0 Done Done Sent false
11879 3724 Acked Read 1 Done Done Sent false
11880 3725 Acked Read 8 Done Done Sent false
11881 3726 Acked Read 10 Done Done Sent false
11882 3727 Acked Write 1 Done Done Sent false
11883 3728 Acked Read 1 Done Done Sent false
11884 3729 Acked Read 4 Done Done Sent false
11885 3730 Acked Write 7 Done Done Sent false
11886 3731 Acked Write 10 Done Done Sent false
11887 3732 Acked Write 6 Done Done Sent false
11888 3733 Acked Read 10 Done Done Sent false
11889 3734 Acked Read 4 Done Done Sent false
11890 3735 Acked Read 10 Done Done Sent false
11891 3736 Acked Flush 0 Done Done Sent false
11892 3737 Acked Write 7 Done Done Sent false
11893 3738 Acked Read 6 Done Done Sent false
11894 3739 Acked Write 1 Done Done Sent false
11895 3740 Acked Read 7 Done Done Sent false
11896 3741 Acked Write 7 Done Done Sent false
11897 3742 Acked Flush 0 Done Done Sent false
11898 3743 Acked Flush 0 Done Done Sent false
11899 3744 Acked Flush 0 Done Done Sent false
11900 3745 Acked Write 8 Done Done Sent false
11901 3746 Acked Read 3 Done Done Sent false
11902 3747 Acked Read 10 Done Done Sent false
11903 3748 Acked Write 7 Done Done Sent false
11904 3749 Acked Read 3 Done Done Sent false
11905 3750 Acked Write 4 Done Done Sent false
11906 3751 Acked Write 1 Done Done Sent false
11907 3752 Acked Read 8 Done Done Sent false
11908 3753 Acked Write 2 Done Done Sent false
11909 3754 Acked Flush 0 Done Done Sent false
11910 3755 Acked Write 2 Done Done Sent false
11911 3756 Acked Write 5 Done Done Sent false
11912 3757 Acked Read 8 Done Done Sent false
11913 3758 Acked Write 5 Done Done Sent false
11914 3759 Acked Read 7 Done Done Sent false
11915 3760 Acked Read 5 Done Done Sent false
11916 3761 Acked Read 2 Done Done Sent false
11917 3762 Acked Read 5 Done Done Sent false
11918 3763 Acked Write 2 Done Done Sent false
11919 3764 Acked Read 2 Done Done Sent false
11920 3765 Acked Flush 0 Done Done Sent false
11921 3766 Acked Write 6 Done Done Sent false
11922 3767 Acked Read 6 Done Done Sent false
11923 3768 Acked Read 2 Done Done Sent false
11924 3769 Acked Write 2 Done Done Sent false
11925 3770 Acked Read 4 Done Done Sent false
11926 3771 Acked Read 9 Done Done Sent false
11927 3772 Acked Write 6 Done Done Sent false
11928 3773 Acked Write 4 Done Done Sent false
11929 3774 Acked Read 5 Done Done Sent false
11930 3775 Acked Flush 0 Done Done Sent false
11931 3776 Acked Read 6 Done Done Sent false
11932 3777 Acked Write 3 Done Done Sent false
11933 3778 Acked Read 3 Done Done Sent false
11934 3779 Acked Read 5 Done Done Sent false
11935 3780 Acked Read 3 Done Done Sent false
11936 3781 Acked Write 8 Done Done Sent false
11937 3782 Acked Write 6 Done Done Sent false
11938 3783 Acked Read 7 Done Done Sent false
11939 3784 Acked Read 4 Done Done Sent false
11940 3785 Acked Read 8 Done Done Sent false
11941 3786 Acked Write 1 Done Done Sent false
11942 3787 Acked Flush 0 Done Done Sent false
11943 3788 Acked Read 2 Done Done Sent false
11944 3789 Acked Write 6 Done Done Sent false
11945 3790 Acked Read 8 Done Done Sent false
11946 3791 Acked Write 1 Done Done Sent false
11947 3792 Acked Write 6 Done Done Sent false
11948 3793 Acked Write 1 Done Done Sent false
11949 3794 Acked Read 1 Done Done Sent false
11950 3795 Acked Flush 0 Done Done Sent false
11951 3796 Acked Flush 0 Done Done Sent false
11952 3797 Acked Read 3 Done Done Sent false
11953 3798 Acked Read 9 Done Done Sent false
11954 3799 Acked Write 1 Done Done Sent false
11955 3800 Acked Write 3 Done Done Sent false
11956 3801 Acked Write 3 Done Done Sent false
11957 3802 Acked Flush 0 Done Done Sent false
11958 3803 Acked Read 7 Done Done Sent false
11959 3804 Acked Write 8 Done Done Sent false
11960 3805 Acked Flush 0 Done Done Sent false
11961 3806 Acked Read 4 Done Done Sent false
11962 3807 Acked Flush 0 Done Done Sent false
11963 3808 Acked Read 4 Done Done Sent false
11964 3809 Acked Write 1 Done Done Sent false
11965 3810 Acked Write 3 Done Done Sent false
11966 3811 Acked Write 7 Done Done Sent false
11967 3812 Acked Write 8 Done Done Sent false
11968 3813 Acked Write 2 Done Done Sent false
11969 3814 Acked Write 4 Done Done Sent false
11970 3815 Acked Flush 0 Done Done Sent false
11971 3816 Acked Read 6 Done Done Sent false
11972 3817 Acked Read 6 Done Done Sent false
11973 3818 Acked Read 6 Done Done Sent false
11974 3819 Acked Read 9 Done Done Sent false
11975 3820 Acked Flush 0 Done Done Sent false
11976 3821 Acked Read 1 Done Done Sent false
11977 3822 Acked Write 7 Done Done Sent false
11978 3823 Acked Read 9 Done Done Sent false
11979 3824 Acked Read 5 Done Done Sent false
11980 3825 Acked Write 7 Done Done Sent false
11981 3826 Acked Read 1 Done Done Sent false
11982 3827 Acked Read 10 Done Done Sent false
11983 3828 Acked Write 8 Done Done Sent false
11984 3829 Acked Read 5 Done Done Sent false
11985 3830 Acked Read 6 Done Done Sent false
11986 3831 Acked Write 3 Done Done Sent false
11987 3832 Acked Write 10 Done Done Sent false
11988 3833 Acked Write 2 Done Done Sent false
11989 3834 Acked Read 5 Done Done Sent false
11990 3835 Acked Write 6 Done Done Sent false
11991 3836 Acked Read 8 Done Done Sent false
11992 3837 Acked Write 3 Done Done Sent false
11993 3838 Acked Read 6 Done Done Sent false
11994 3839 Acked Read 4 Done Done Sent false
11995 3840 Acked Read 2 Done Done Sent false
11996 3841 Acked Write 7 Done Done Sent false
11997 3842 Acked Read 7 Done Done Sent false
11998 3843 Acked Read 9 Done Done Sent false
11999 3844 Acked Read 3 Done Done Sent false
12000 3845 Acked Read 5 Done Done Sent false
12001 3846 Acked Read 3 Done Done Sent false
12002 3847 Acked Flush 0 Done Done Sent false
12003 3848 Acked Flush 0 Done Done Sent false
12004 3849 Acked Write 5 Done Done Sent false
12005 3850 Acked Write 3 Done Done Sent false
12006 3851 Acked Read 10 Done Done Sent false
12007 3852 Acked Read 2 Done Done Sent false
12008 3853 Acked Write 8 Done Done Sent false
12009 3854 Acked Read 6 Done Done Sent false
12010 3855 Acked Flush 0 Done Done Sent false
12011 3856 Acked Flush 0 Done Done Sent false
12012 3857 Acked Read 7 Done Done Sent false
12013 3858 Acked Write 1 Done Done Sent false
12014 3859 Acked Write 7 Done Done Sent false
12015 3860 Acked Write 10 Done Done Sent false
12016 3861 Acked Read 3 Done Done Sent false
12017 3862 Acked Read 6 Done Done Sent false
12018 3863 Acked Write 4 Done Done Sent false
12019 3864 Acked Read 3 Done Done Sent false
12020 3865 Acked Write 5 Done Done Sent false
12021 3866 Acked Write 1 Done Done Sent false
12022 3867 Acked Flush 0 Done Done Sent false
12023 3868 Acked Write 7 Done Done Sent false
12024 3869 Acked Write 7 Done Done Sent false
12025 3870 Acked Write 9 Done Done Sent false
12026 3871 Acked Read 2 Done Done Sent false
12027 3872 Acked Read 7 Done Done Sent false
12028 3873 Acked Write 1 Done Done Sent false
12029 3874 Acked Write 3 Done Done Sent false
12030 3875 Acked Flush 0 Done Done Sent false
12031 3876 Acked Write 1 Done Done Sent false
12032 3877 Acked Write 5 Done Done Sent false
12033 3878 Acked Read 6 Done Done Sent false
12034 3879 Acked Write 10 Done Done Sent false
12035 3880 Acked Write 3 Done Done Sent false
12036 3881 Acked Read 4 Done Done Sent false
12037 3882 Acked Read 5 Done Done Sent false
12038 3883 Acked Read 10 Done Done Sent false
12039 3884 Acked Flush 0 Done Done Sent false
12040 3885 Acked Read 1 Done Done Sent false
12041 3886 Acked Read 10 Done Done Sent false
12042 3887 Acked Read 5 Done Done Sent false
12043 3888 Acked Write 10 Done Done Sent false
12044 3889 Acked Read 8 Done Done Sent false
12045 3890 Acked Read 8 Done Done Sent false
12046 3891 Acked Write 5 Done Done Sent false
12047 3892 Acked Write 10 Done Done Sent false
12048 3893 Acked Write 1 Done Done Sent false
12049 3894 Acked Read 1 Done Done Sent false
12050 3895 Acked Write 7 Done Done Sent false
12051 3896 Acked Flush 0 Done Done Sent false
12052 3897 Acked Read 10 Done Done Sent false
12053 3898 Acked Write 10 Done Done Sent false
12054 3899 Acked Flush 0 Done Done Sent false
12055 3900 Acked Read 3 Done Done Sent false
12056 3901 Acked Read 7 Done Done Sent false
12057 3902 Acked Read 2 Done Done Sent false
12058 3903 Acked Read 1 Done Done Sent false
12059 3904 Acked Write 4 Done Done Sent false
12060 3905 Acked Read 6 Done Done Sent false
12061 3906 Acked Write 2 Done Done Sent false
12062 3907 Acked Read 6 Done Done Sent false
12063 3908 Acked Write 2 Done Done Sent false
12064 3909 Acked Write 6 Done Done Sent false
12065 3910 Acked Write 9 Done Done Sent false
12066 3911 Acked Read 2 Done Done Sent false
12067 3912 Acked Read 9 Done Done Sent false
12068 3913 Acked Read 9 Done Done Sent false
12069 3914 Acked Read 10 Done Done Sent false
12070 3915 Acked Read 4 Done Done Sent false
12071 3916 Acked Read 8 Done Done Sent false
12072 3917 Acked Write 6 Done Done Sent false
12073 3918 Acked Write 1 Done Done Sent false
12074 3919 Acked Flush 0 Done Done Sent false
12075 3920 Acked Read 9 Done Done Sent false
12076 3921 Acked Read 2 Done Done Sent false
12077 3922 Acked Read 3 Done Done Sent false
12078 3923 Acked Write 6 Done Done Sent false
12079 3924 Acked Write 4 Done Done Sent false
12080 3925 Acked Read 8 Done Done Sent false
12081 3926 Acked Read 2 Done Done Sent false
12082 3927 Acked Read 4 Done Done Sent false
12083 3928 Acked Read 8 Done Done Sent false
12084 3929 Acked Flush 0 Done Done Sent false
12085 3930 Acked Read 8 Done Done Sent false
12086 3931 Acked Read 9 Done Done Sent false
12087 3932 Acked Flush 0 Done Done Sent false
12088 3933 Acked Write 8 Done Done Sent false
12089 3934 Acked Write 8 Done Done Sent false
12090 3935 Acked Write 7 Done Done Sent false
12091 3936 Acked Flush 0 Done Done Sent false
12092 3937 Acked Read 10 Done Done Sent false
12093 3938 Acked Read 7 Done Done Sent false
12094 3939 Acked Read 10 Done Done Sent false
12095 3940 Acked Write 4 Done Done Sent false
12096 3941 Acked Read 9 Done Done Sent false
12097 3942 Acked Read 6 Done Done Sent false
12098 3943 Acked Write 4 Done Done Sent false
12099 3944 Acked Write 5 Done Done Sent false
12100 3945 Acked Flush 0 Done Done Sent false
12101 3946 Acked Read 2 Done Done Sent false
12102 3947 Acked Flush 0 Done Done Sent false
12103 3948 Acked Read 7 Done Done Sent false
12104 3949 Acked Write 3 Done Done Sent false
12105 3950 Acked Read 10 Done Done Sent false
12106 3951 Acked Read 3 Done Done Sent false
12107 3952 Acked Read 7 Done Done Sent false
12108 3953 Acked Read 1 Done Done Sent false
12109 3954 Acked Write 1 Done Done Sent false
12110 3955 Acked Write 2 Done Done Sent false
12111 3956 Acked Read 2 Done Done Sent false
12112 3957 Acked Flush 0 Done Done Sent false
12113 3958 Acked Flush 0 Done Done Sent false
12114 3959 Acked Read 1 Done Done Sent false
12115 3960 Acked Flush 0 Done Done Sent false
12116 3961 Acked Write 3 Done Done Sent false
12117 3962 Acked Write 4 Done Done Sent false
12118 3963 Acked Flush 0 Done Done Sent false
12119 3964 Acked Read 7 Done Done Sent false
12120 3965 Acked Read 10 Done Done Sent false
12121 3966 Acked Read 8 Done Done Sent false
12122 3967 Acked Flush 0 Done Done Sent false
12123 3968 Acked Read 3 Done Done Sent false
12124 3969 Acked Write 9 Done Done Sent false
12125 3970 Acked Read 8 Done Done Sent false
12126 3971 Acked Write 8 Done Done Sent false
12127 3972 Acked Flush 0 Done Done Sent false
12128 3973 Acked Read 2 Done Done Sent false
12129 3974 Acked Flush 0 Done Done Sent false
12130 3975 Acked Read 4 Done Done Sent false
12131 3976 Acked Read 4 Done Done Sent false
12132 3977 Acked Flush 0 Done Done Sent false
12133 3978 Acked Write 2 Done Done Sent false
12134 3979 Acked Read 2 Done Done Sent false
12135 3980 Acked Read 4 Done Done Sent false
12136 3981 Acked Write 1 Done Done Sent false
12137 3982 Acked Read 5 Done Done Sent false
12138 3983 Acked Write 9 Done Done Sent false
12139 3984 Acked Read 5 Done Done Sent false
12140 3985 Acked Read 9 Done Done Sent false
12141 3986 Acked Write 8 Done Done Sent false
12142 3987 Acked Read 10 Done Done Sent false
12143 3988 Acked Read 6 Done Done Sent false
12144 3989 Acked Read 9 Done Done Sent false
12145 3990 Acked Write 3 Done Done Sent false
12146 3991 Acked Read 4 Done Done Sent false
12147 3992 Acked Write 2 Done Done Sent false
12148 3993 Acked Write 6 Done Done Sent false
12149 3994 Acked Flush 0 Done Done Sent false
12150 3995 Acked Write 6 Done Done Sent false
12151 3996 Acked Read 10 Done Done Sent false
12152 3997 Acked Write 4 Done Done Sent false
12153 3998 Acked Write 3 Done Done Sent false
12154 3999 Acked Write 5 Done Done Sent false
12155 4000 Acked Read 7 Done Done Sent false
12156 4001 Acked Flush 0 Done Done Sent false
12157 4002 Acked Flush 0 Done Done Sent false
12158 4003 Acked Read 10 Done Done Sent false
12159 4004 Acked Read 1 Done Done Sent false
12160 4005 Acked Write 10 Done Done Sent false
12161 4006 Acked Read 7 Done Done Sent false
12162 4007 Acked Write 5 Done Done Sent false
12163 4008 Acked Read 10 Done Done Sent false
12164 4009 Acked Read 1 Done Done Sent false
12165 4010 Acked Read 9 Done Done Sent false
12166 4011 Acked Read 1 Done Done Sent false
12167 4012 Acked Write 5 Done Done Sent false
12168 4013 Acked Flush 0 Done Done Sent false
12169 4014 Acked Write 6 Done Done Sent false
12170 4015 Acked Read 7 Done Done Sent false
12171 4016 Acked Read 3 Done Done Sent false
12172 4017 Acked Read 8 Done Done Sent false
12173 4018 Acked Flush 0 Done Done Sent false
12174 STATES DS:0 DS:1 DS:2 TOTAL
12175 Sent 0 0 302 302
12176 Done 302 302 0 604
12177 Skipped 0 0 0 0
12178 Error 0 0 0 0
12179 Last Flush: 4018 4018 3716
12180 Downstairs last five completed: 3716 3715 3714 3713 3712
12181 Upstairs last five completed: 4018 4017 4016 4015 4014
12182 CLIENT: Up:0 ds:302 act:2
12183 ----------------------------------------------------------------
12184 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12185 States: Active Active Offline
12186 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12187 3717 Acked Read 6 Done Done Sent false
12188 3718 Acked Read 7 Done Done Sent false
12189 3719 Acked Read 5 Done Done Sent false
12190 3720 Acked Read 7 Done Done Sent false
12191 3721 Acked Read 7 Done Done Sent false
12192 3722 Acked Read 7 Done Done Sent false
12193 3723 Acked Flush 0 Done Done Sent false
12194 3724 Acked Read 1 Done Done Sent false
12195 3725 Acked Read 8 Done Done Sent false
12196 3726 Acked Read 10 Done Done Sent false
12197 3727 Acked Write 1 Done Done Sent false
12198 3728 Acked Read 1 Done Done Sent false
12199 3729 Acked Read 4 Done Done Sent false
12200 3730 Acked Write 7 Done Done Sent false
12201 3731 Acked Write 10 Done Done Sent false
12202 3732 Acked Write 6 Done Done Sent false
12203 3733 Acked Read 10 Done Done Sent false
12204 3734 Acked Read 4 Done Done Sent false
12205 3735 Acked Read 10 Done Done Sent false
12206 3736 Acked Flush 0 Done Done Sent false
12207 3737 Acked Write 7 Done Done Sent false
12208 3738 Acked Read 6 Done Done Sent false
12209 3739 Acked Write 1 Done Done Sent false
12210 3740 Acked Read 7 Done Done Sent false
12211 3741 Acked Write 7 Done Done Sent false
12212 3742 Acked Flush 0 Done Done Sent false
12213 3743 Acked Flush 0 Done Done Sent false
12214 3744 Acked Flush 0 Done Done Sent false
12215 3745 Acked Write 8 Done Done Sent false
12216 3746 Acked Read 3 Done Done Sent false
12217 3747 Acked Read 10 Done Done Sent false
12218 3748 Acked Write 7 Done Done Sent false
12219 3749 Acked Read 3 Done Done Sent false
12220 3750 Acked Write 4 Done Done Sent false
12221 3751 Acked Write 1 Done Done Sent false
12222 3752 Acked Read 8 Done Done Sent false
12223 3753 Acked Write 2 Done Done Sent false
12224 3754 Acked Flush 0 Done Done Sent false
12225 3755 Acked Write 2 Done Done Sent false
12226 3756 Acked Write 5 Done Done Sent false
12227 3757 Acked Read 8 Done Done Sent false
12228 3758 Acked Write 5 Done Done Sent false
12229 3759 Acked Read 7 Done Done Sent false
12230 3760 Acked Read 5 Done Done Sent false
12231 3761 Acked Read 2 Done Done Sent false
12232 3762 Acked Read 5 Done Done Sent false
12233 3763 Acked Write 2 Done Done Sent false
12234 3764 Acked Read 2 Done Done Sent false
12235 3765 Acked Flush 0 Done Done Sent false
12236 3766 Acked Write 6 Done Done Sent false
12237 3767 Acked Read 6 Done Done Sent false
12238 3768 Acked Read 2 Done Done Sent false
12239 3769 Acked Write 2 Done Done Sent false
12240 3770 Acked Read 4 Done Done Sent false
12241 3771 Acked Read 9 Done Done Sent false
12242 3772 Acked Write 6 Done Done Sent false
12243 3773 Acked Write 4 Done Done Sent false
12244 3774 Acked Read 5 Done Done Sent false
12245 3775 Acked Flush 0 Done Done Sent false
12246 3776 Acked Read 6 Done Done Sent false
12247 3777 Acked Write 3 Done Done Sent false
12248 3778 Acked Read 3 Done Done Sent false
12249 3779 Acked Read 5 Done Done Sent false
12250 3780 Acked Read 3 Done Done Sent false
12251 3781 Acked Write 8 Done Done Sent false
12252 3782 Acked Write 6 Done Done Sent false
12253 3783 Acked Read 7 Done Done Sent false
12254 3784 Acked Read 4 Done Done Sent false
12255 3785 Acked Read 8 Done Done Sent false
12256 3786 Acked Write 1 Done Done Sent false
12257 3787 Acked Flush 0 Done Done Sent false
12258 3788 Acked Read 2 Done Done Sent false
12259 3789 Acked Write 6 Done Done Sent false
12260 3790 Acked Read 8 Done Done Sent false
12261 3791 Acked Write 1 Done Done Sent false
12262 3792 Acked Write 6 Done Done Sent false
12263 3793 Acked Write 1 Done Done Sent false
12264 3794 Acked Read 1 Done Done Sent false
12265 3795 Acked Flush 0 Done Done Sent false
12266 3796 Acked Flush 0 Done Done Sent false
12267 3797 Acked Read 3 Done Done Sent false
12268 3798 Acked Read 9 Done Done Sent false
12269 3799 Acked Write 1 Done Done Sent false
12270 3800 Acked Write 3 Done Done Sent false
12271 3801 Acked Write 3 Done Done Sent false
12272 3802 Acked Flush 0 Done Done Sent false
12273 3803 Acked Read 7 Done Done Sent false
12274 3804 Acked Write 8 Done Done Sent false
12275 3805 Acked Flush 0 Done Done Sent false
12276 3806 Acked Read 4 Done Done Sent false
12277 3807 Acked Flush 0 Done Done Sent false
12278 3808 Acked Read 4 Done Done Sent false
12279 3809 Acked Write 1 Done Done Sent false
12280 3810 Acked Write 3 Done Done Sent false
12281 3811 Acked Write 7 Done Done Sent false
12282 3812 Acked Write 8 Done Done Sent false
12283 3813 Acked Write 2 Done Done Sent false
12284 3814 Acked Write 4 Done Done Sent false
12285 3815 Acked Flush 0 Done Done Sent false
12286 3816 Acked Read 6 Done Done Sent false
12287 3817 Acked Read 6 Done Done Sent false
12288 3818 Acked Read 6 Done Done Sent false
12289 3819 Acked Read 9 Done Done Sent false
12290 3820 Acked Flush 0 Done Done Sent false
12291 3821 Acked Read 1 Done Done Sent false
12292 3822 Acked Write 7 Done Done Sent false
12293 3823 Acked Read 9 Done Done Sent false
12294 3824 Acked Read 5 Done Done Sent false
12295 3825 Acked Write 7 Done Done Sent false
12296 3826 Acked Read 1 Done Done Sent false
12297 3827 Acked Read 10 Done Done Sent false
12298 3828 Acked Write 8 Done Done Sent false
12299 3829 Acked Read 5 Done Done Sent false
12300 3830 Acked Read 6 Done Done Sent false
12301 3831 Acked Write 3 Done Done Sent false
12302 3832 Acked Write 10 Done Done Sent false
12303 3833 Acked Write 2 Done Done Sent false
12304 3834 Acked Read 5 Done Done Sent false
12305 3835 Acked Write 6 Done Done Sent false
12306 3836 Acked Read 8 Done Done Sent false
12307 3837 Acked Write 3 Done Done Sent false
12308 3838 Acked Read 6 Done Done Sent false
12309 3839 Acked Read 4 Done Done Sent false
12310 3840 Acked Read 2 Done Done Sent false
12311 3841 Acked Write 7 Done Done Sent false
12312 3842 Acked Read 7 Done Done Sent false
12313 3843 Acked Read 9 Done Done Sent false
12314 3844 Acked Read 3 Done Done Sent false
12315 3845 Acked Read 5 Done Done Sent false
12316 3846 Acked Read 3 Done Done Sent false
12317 3847 Acked Flush 0 Done Done Sent false
12318 3848 Acked Flush 0 Done Done Sent false
12319 3849 Acked Write 5 Done Done Sent false
12320 3850 Acked Write 3 Done Done Sent false
12321 3851 Acked Read 10 Done Done Sent false
12322 3852 Acked Read 2 Done Done Sent false
12323 3853 Acked Write 8 Done Done Sent false
12324 3854 Acked Read 6 Done Done Sent false
12325 3855 Acked Flush 0 Done Done Sent false
12326 3856 Acked Flush 0 Done Done Sent false
12327 3857 Acked Read 7 Done Done Sent false
12328 3858 Acked Write 1 Done Done Sent false
12329 3859 Acked Write 7 Done Done Sent false
12330 3860 Acked Write 10 Done Done Sent false
12331 3861 Acked Read 3 Done Done Sent false
12332 3862 Acked Read 6 Done Done Sent false
12333 3863 Acked Write 4 Done Done Sent false
12334 3864 Acked Read 3 Done Done Sent false
12335 3865 Acked Write 5 Done Done Sent false
12336 3866 Acked Write 1 Done Done Sent false
12337 3867 Acked Flush 0 Done Done Sent false
12338 3868 Acked Write 7 Done Done Sent false
12339 3869 Acked Write 7 Done Done Sent false
12340 3870 Acked Write 9 Done Done Sent false
12341 3871 Acked Read 2 Done Done Sent false
12342 3872 Acked Read 7 Done Done Sent false
12343 3873 Acked Write 1 Done Done Sent false
12344 3874 Acked Write 3 Done Done Sent false
12345 3875 Acked Flush 0 Done Done Sent false
12346 3876 Acked Write 1 Done Done Sent false
12347 3877 Acked Write 5 Done Done Sent false
12348 3878 Acked Read 6 Done Done Sent false
12349 3879 Acked Write 10 Done Done Sent false
12350 3880 Acked Write 3 Done Done Sent false
12351 3881 Acked Read 4 Done Done Sent false
12352 3882 Acked Read 5 Done Done Sent false
12353 3883 Acked Read 10 Done Done Sent false
12354 3884 Acked Flush 0 Done Done Sent false
12355 3885 Acked Read 1 Done Done Sent false
12356 3886 Acked Read 10 Done Done Sent false
12357 3887 Acked Read 5 Done Done Sent false
12358 3888 Acked Write 10 Done Done Sent false
12359 3889 Acked Read 8 Done Done Sent false
12360 3890 Acked Read 8 Done Done Sent false
12361 3891 Acked Write 5 Done Done Sent false
12362 3892 Acked Write 10 Done Done Sent false
12363 3893 Acked Write 1 Done Done Sent false
12364 3894 Acked Read 1 Done Done Sent false
12365 3895 Acked Write 7 Done Done Sent false
12366 3896 Acked Flush 0 Done Done Sent false
12367 3897 Acked Read 10 Done Done Sent false
12368 3898 Acked Write 10 Done Done Sent false
12369 3899 Acked Flush 0 Done Done Sent false
12370 3900 Acked Read 3 Done Done Sent false
12371 3901 Acked Read 7 Done Done Sent false
12372 3902 Acked Read 2 Done Done Sent false
12373 3903 Acked Read 1 Done Done Sent false
12374 3904 Acked Write 4 Done Done Sent false
12375 3905 Acked Read 6 Done Done Sent false
12376 3906 Acked Write 2 Done Done Sent false
12377 3907 Acked Read 6 Done Done Sent false
12378 3908 Acked Write 2 Done Done Sent false
12379 3909 Acked Write 6 Done Done Sent false
12380 3910 Acked Write 9 Done Done Sent false
12381 3911 Acked Read 2 Done Done Sent false
12382 3912 Acked Read 9 Done Done Sent false
12383 3913 Acked Read 9 Done Done Sent false
12384 3914 Acked Read 10 Done Done Sent false
12385 3915 Acked Read 4 Done Done Sent false
12386 3916 Acked Read 8 Done Done Sent false
12387 3917 Acked Write 6 Done Done Sent false
12388 3918 Acked Write 1 Done Done Sent false
12389 3919 Acked Flush 0 Done Done Sent false
12390 3920 Acked Read 9 Done Done Sent false
12391 3921 Acked Read 2 Done Done Sent false
12392 3922 Acked Read 3 Done Done Sent false
12393 3923 Acked Write 6 Done Done Sent false
12394 3924 Acked Write 4 Done Done Sent false
12395 3925 Acked Read 8 Done Done Sent false
12396 3926 Acked Read 2 Done Done Sent false
12397 3927 Acked Read 4 Done Done Sent false
12398 3928 Acked Read 8 Done Done Sent false
12399 3929 Acked Flush 0 Done Done Sent false
12400 3930 Acked Read 8 Done Done Sent false
12401 3931 Acked Read 9 Done Done Sent false
12402 3932 Acked Flush 0 Done Done Sent false
12403 3933 Acked Write 8 Done Done Sent false
12404 3934 Acked Write 8 Done Done Sent false
12405 3935 Acked Write 7 Done Done Sent false
12406 3936 Acked Flush 0 Done Done Sent false
12407 3937 Acked Read 10 Done Done Sent false
12408 3938 Acked Read 7 Done Done Sent false
12409 3939 Acked Read 10 Done Done Sent false
12410 3940 Acked Write 4 Done Done Sent false
12411 3941 Acked Read 9 Done Done Sent false
12412 3942 Acked Read 6 Done Done Sent false
12413 3943 Acked Write 4 Done Done Sent false
12414 3944 Acked Write 5 Done Done Sent false
12415 3945 Acked Flush 0 Done Done Sent false
12416 3946 Acked Read 2 Done Done Sent false
12417 3947 Acked Flush 0 Done Done Sent false
12418 3948 Acked Read 7 Done Done Sent false
12419 3949 Acked Write 3 Done Done Sent false
12420 3950 Acked Read 10 Done Done Sent false
12421 3951 Acked Read 3 Done Done Sent false
12422 3952 Acked Read 7 Done Done Sent false
12423 3953 Acked Read 1 Done Done Sent false
12424 3954 Acked Write 1 Done Done Sent false
12425 3955 Acked Write 2 Done Done Sent false
12426 3956 Acked Read 2 Done Done Sent false
12427 3957 Acked Flush 0 Done Done Sent false
12428 3958 Acked Flush 0 Done Done Sent false
12429 3959 Acked Read 1 Done Done Sent false
12430 3960 Acked Flush 0 Done Done Sent false
12431 3961 Acked Write 3 Done Done Sent false
12432 3962 Acked Write 4 Done Done Sent false
12433 3963 Acked Flush 0 Done Done Sent false
12434 3964 Acked Read 7 Done Done Sent false
12435 3965 Acked Read 10 Done Done Sent false
12436 3966 Acked Read 8 Done Done Sent false
12437 3967 Acked Flush 0 Done Done Sent false
12438 3968 Acked Read 3 Done Done Sent false
12439 3969 Acked Write 9 Done Done Sent false
12440 3970 Acked Read 8 Done Done Sent false
12441 3971 Acked Write 8 Done Done Sent false
12442 3972 Acked Flush 0 Done Done Sent false
12443 3973 Acked Read 2 Done Done Sent false
12444 3974 Acked Flush 0 Done Done Sent false
12445 3975 Acked Read 4 Done Done Sent false
12446 3976 Acked Read 4 Done Done Sent false
12447 3977 Acked Flush 0 Done Done Sent false
12448 3978 Acked Write 2 Done Done Sent false
12449 3979 Acked Read 2 Done Done Sent false
12450 3980 Acked Read 4 Done Done Sent false
12451 3981 Acked Write 1 Done Done Sent false
12452 3982 Acked Read 5 Done Done Sent false
12453 3983 Acked Write 9 Done Done Sent false
12454 3984 Acked Read 5 Done Done Sent false
12455 3985 Acked Read 9 Done Done Sent false
12456 3986 Acked Write 8 Done Done Sent false
12457 3987 Acked Read 10 Done Done Sent false
12458 3988 Acked Read 6 Done Done Sent false
12459 3989 Acked Read 9 Done Done Sent false
12460 3990 Acked Write 3 Done Done Sent false
12461 3991 Acked Read 4 Done Done Sent false
12462 3992 Acked Write 2 Done Done Sent false
12463 3993 Acked Write 6 Done Done Sent false
12464 3994 Acked Flush 0 Done Done Sent false
12465 3995 Acked Write 6 Done Done Sent false
12466 3996 Acked Read 10 Done Done Sent false
12467 3997 Acked Write 4 Done Done Sent false
12468 3998 Acked Write 3 Done Done Sent false
12469 3999 Acked Write 5 Done Done Sent false
12470 4000 Acked Read 7 Done Done Sent false
12471 4001 Acked Flush 0 Done Done Sent false
12472 4002 Acked Flush 0 Done Done Sent false
12473 4003 Acked Read 10 Done Done Sent false
12474 4004 Acked Read 1 Done Done Sent false
12475 4005 Acked Write 10 Done Done Sent false
12476 4006 Acked Read 7 Done Done Sent false
12477 4007 Acked Write 5 Done Done Sent false
12478 4008 Acked Read 10 Done Done Sent false
12479 4009 Acked Read 1 Done Done Sent false
12480 4010 Acked Read 9 Done Done Sent false
12481 4011 Acked Read 1 Done Done Sent false
12482 4012 Acked Write 5 Done Done Sent false
12483 4013 Acked Flush 0 Done Done Sent false
12484 4014 Acked Write 6 Done Done Sent false
12485 4015 Acked Read 7 Done Done Sent false
12486 4016 Acked Read 3 Done Done Sent false
12487 4017 Acked Read 8 Done Done Sent false
12488 4018 Acked Flush 0 Done Done Sent false
12489 STATES DS:0 DS:1 DS:2 TOTAL
12490 Sent 0 0 302 302
12491 Done 302 302 0 604
12492 Skipped 0 0 0 0
12493 Error 0 0 0 0
12494 Last Flush: 4018 4018 3716
12495 Downstairs last five completed: 3716 3715 3714 3713 3712
12496 Upstairs last five completed: 4018 4017 4016 4015 4014
12497 CLIENT: Up:0 ds:302 act:2
124982024-11-19T17:10:37.934ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
124992024-11-19T17:10:37.934ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50048) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125002024-11-19T17:10:37.935ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125012024-11-19T17:10:37.936ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125022024-11-19T17:10:37.936ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125032024-11-19T17:10:37.936ZINFOcrucible: send last flush ID to this DS: 3716 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125042024-11-19T17:10:37.936ZINFOcrucible: Replied this last flush ID: 3716 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125052024-11-19T17:10:37.936ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125062024-11-19T17:10:37.936ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125072024-11-19T17:10:37.936ZINFOcrucible: [2] client re-new 302 jobs since flush 3716 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
125082024-11-19T17:10:37.936ZINFOcrucible: [2] Replayed 302 jobs since flush: 3716 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
12509 ----------------------------------------------------------------
12510 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12511 Downstairs last five completed: 4018 4017 4016 4015 4014
12512 Upstairs last five completed: 4018 4017 4016 4015 4014
12513 CLIENT: Up:0 ds:0 act:3
12514 Replay: All jobs finished
12515 Main task has work to do, go find it
12516 got dsc Stop(1)
12517 stop 1
12518 Nov 19 17:10:38.181 INFO [8820] Got stop action so:false kr:false
12519 request completed, latency_us: 304, response_code: 204, uri: /stop/cid/1, method: POST, req_id: eca646c8-a8b6-4eb1-92d3-c471d4731096, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
12520 Nov 19 17:10:38.182 INFO request completed, latency_us: 124, response_code: 200, uri: /state/cid/1, method: GET, req_id: 56c47ba3-28ba-4172-863c-bfa5c3947e07, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
125212024-11-19T17:10:38.182ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125222024-11-19T17:10:38.182ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125232024-11-19T17:10:38.182ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
125242024-11-19T17:10:38.182ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125252024-11-19T17:10:38.182ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
125262024-11-19T17:10:38.182ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
12527 [8820] Exited with: ExitStatus(unix_wait_status(9))
12528 [8820][1] reports Exit
12529 Nov 19 17:10:42.184 INFO request completed, latency_us: 239, response_code: 200, uri: /state/cid/1, method: GET, req_id: cced506d-5ef2-48db-ba88-3210069d6aeb, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
12530 001/300 Read block 1512 len 24576
12531 002/300 Read block 1644 len 16384
12532 003/300 Write block 1446 len 24576 data: 3 5 5 5 5 5
12533 004/300 Write block 3145 len 36864 data: 3 4 4 4 4 3 4 4 4
12534 005/300 Read block 567 len 4096
12535 006/300 Write block 4687 len 16384 data: 2 2 4 4
12536 007/300 Flush
12537 008/300 Read block 1960 len 16384
12538 009/300 Write block 2656 len 28672 data: 3 3 3 2 3 3 4
12539 010/300 Read block 4039 len 36864
12540 011/300 Read block 1347 len 24576
12541 012/300 Read block 3836 len 16384
12542 013/300 Read block 1890 len 36864
12543 014/300 Read block 2630 len 12288
12544 015/300 Read block 1085 len 8192
12545 016/300 Write block 3660 len 24576 data: 5 5 5 4 3 2
12546 017/300 Write block 2286 len 24576 data: 6 6 6 6 4 3
12547 018/300 Flush
12548 019/300 Write block 3478 len 40960 data: 3 4 3 4 5 4 5 6 6 6
12549 020/300 Read block 3378 len 24576
12550 021/300 Read block 176 len 24576
12551 022/300 Flush
12552 023/300 Write block 1329 len 28672 data: 2 3 3 3 3 3 3
12553 024/300 Read block 1082 len 24576
12554 025/300 Write block 1747 len 20480 data: 2 2 3 3 3
12555 026/300 Flush
12556 027/300 Read block 2458 len 28672
12557 028/300 Write block 188 len 32768 data: 4 4 3 3 3 3 3 2
12558 029/300 Read block 2137 len 32768
12559 030/300 Read block 586 len 12288
12560 031/300 Read block 1426 len 20480
12561 032/300 Read block 4102 len 12288
12562 033/300 Read block 1867 len 4096
12563 034/300 Read block 3962 len 20480
12564 035/300 Write block 4601 len 4096 data: 2
12565 036/300 Read block 385 len 40960
12566 037/300 Write block 670 len 36864 data: 6 6 5 6 6 6 5 4 4
12567 038/300 Write block 4129 len 4096 data: 2
12568 039/300 Read block 4956 len 32768
12569 040/300 Flush
12570 041/300 Read block 390 len 28672
12571 042/300 Read block 722 len 28672
12572 043/300 Read block 1809 len 20480
12573 044/300 Read block 1729 len 12288
12574 045/300 Flush
12575 046/300 Flush
12576 047/300 Write block 2062 len 12288 data: 3 4 5
12577 048/300 Read block 4935 len 8192
12578 049/300 Write block 4242 len 16384 data: 3 2 2 3
12579 050/300 Read block 2409 len 24576
12580 051/300 Write block 776 len 32768 data: 2 3 3 3 4 5 5 5
12581 052/300 Read block 4541 len 8192
12582 053/300 Write block 2468 len 28672 data: 4 2 2 2 2 2 2
12583 054/300 Flush
12584 055/300 Read block 4292 len 4096
12585 056/300 Read block 3536 len 12288
12586 057/300 Read block 2200 len 12288
12587 058/300 Read block 2186 len 24576
12588 059/300 Read block 1987 len 4096
12589 060/300 Read block 1749 len 28672
12590 061/300 Read block 1857 len 24576
12591 062/300 Read block 2452 len 12288
12592 063/300 Write block 2856 len 8192 data: 2 2
12593 064/300 Read block 3229 len 20480
12594 065/300 Write block 485 len 32768 data: 2 2 2 3 3 3 4 4
12595 066/300 Read block 4639 len 32768
12596 067/300 Read block 4959 len 16384
12597 068/300 Write block 2287 len 8192 data: 7 7
12598 069/300 Write block 2391 len 40960 data: 6 7 6 5 3 3 4 4 4 4
12599 070/300 Write block 989 len 28672 data: 3 3 4 4 4 2 2
12600 071/300 Read block 2080 len 24576
12601 072/300 Read block 2697 len 16384
12602 073/300 Read block 3516 len 16384
12603 074/300 Write block 3265 len 28672 data: 2 2 3 3 3 2 3
12604 075/300 Write block 2166 len 28672 data: 4 6 5 5 4 4 4
12605 076/300 Read block 2651 len 8192
12606 077/300 Write block 2325 len 8192 data: 3 3
12607 078/300 Read block 1982 len 12288
12608 079/300 Write block 455 len 12288 data: 2 2 2
12609 080/300 Read block 768 len 20480
12610 081/300 Read block 3162 len 28672
12611 082/300 Read block 3685 len 8192
12612 083/300 Read block 4619 len 16384
12613 084/300 Read block 81 len 4096
12614 085/300 Read block 3146 len 12288
12615 086/300 Read block 1093 len 8192
12616 087/300 Read block 776 len 8192
12617 088/300 Read block 1385 len 24576
12618 089/300 Read block 4962 len 32768
12619 090/300 Read block 1257 len 40960
12620 091/300 Read block 4399 len 24576
12621 092/300 Write block 830 len 36864 data: 3 3 3 2 3 3 3 3 3
12622 093/300 Read block 479 len 16384
12623 094/300 Read block 4469 len 4096
12624 095/300 Read block 346 len 12288
12625 096/300 Read block 1236 len 20480
12626 097/300 Read block 4453 len 40960
12627 098/300 Flush
12628 099/300 Read block 3817 len 8192
12629 100/300 Read block 1528 len 8192
12630 101/300 Write block 3394 len 24576 data: 2 3 3 3 4 4
12631 102/300 Read block 1423 len 36864
12632 103/300 Write block 318 len 12288 data: 6 6 7
12633 104/300 Read block 3467 len 24576
12634 105/300 Read block 4627 len 32768
12635 106/300 Write block 790 len 36864 data: 5 5 5 6 5 4 4 3 4
12636 107/300 Write block 621 len 36864 data: 2 2 2 3 3 3 3 3 3
12637 108/300 Read block 4443 len 16384
12638 109/300 Read block 551 len 20480
12639 110/300 Write block 1875 len 24576 data: 3 3 4 6 6 5
12640 111/300 Read block 3630 len 8192
12641 112/300 Write block 4252 len 12288 data: 2 2 3
12642 113/300 Flush
12643 114/300 Write block 3355 len 8192 data: 3 3
12644 115/300 Write block 3135 len 16384 data: 3 3 3 2
12645 116/300 Read block 1241 len 28672
12646 117/300 Write block 931 len 40960 data: 4 4 3 4 5 4 4 4 4 3
12647 118/300 Flush
12648 119/300 Read block 4148 len 12288
12649 120/300 Read block 3747 len 32768
12650 121/300 Flush
12651 122/300 Read block 949 len 32768
12652 123/300 Write block 150 len 12288 data: 4 3 4
12653 124/300 Read block 622 len 16384
12654 125/300 Write block 2070 len 40960 data: 5 4 4 3 3 3 4 5 5 5
12655 126/300 Write block 1331 len 16384 data: 4 4 4 4
12656 127/300 Read block 2829 len 16384
12657 128/300 Write block 4523 len 36864 data: 2 2 2 3 3 3 4 4 4
12658 129/300 Flush
12659 130/300 Read block 3273 len 8192
12660 131/300 Read block 1370 len 24576
12661 132/300 Read block 2303 len 20480
12662 133/300 Write block 1667 len 8192 data: 3 3
12663 134/300 Read block 129 len 20480
12664 135/300 Write block 366 len 28672 data: 4 4 4 2 2 2 2
12665 136/300 Write block 3456 len 4096 data: 4
12666 137/300 Write block 1632 len 20480 data: 4 3 3 3 2
12667 138/300 Read block 1149 len 32768
12668 139/300 Write block 2296 len 40960 data: 3 3 4 4 5 7 6 5 6 7
12669 140/300 Write block 2988 len 32768 data: 2 2 3 3 4 5 5 5
12670 141/300 Write block 1072 len 24576 data: 4 4 4 3 3 4
12671 142/300 Read block 2948 len 20480
12672 143/300 Write block 2232 len 16384 data: 4 3 3 3
12673 144/300 Write block 2390 len 24576 data: 5 7 8 7 6 4
12674 145/300 Write block 2089 len 20480 data: 2 2 2 2 2
12675 146/300 Read block 180 len 16384
12676 147/300 Read block 334 len 40960
12677 148/300 Read block 3567 len 28672
12678 149/300 Write block 1996 len 8192 data: 4 3
12679 150/300 Read block 4806 len 24576
12680 151/300 Write block 955 len 8192 data: 2 3
12681 152/300 Read block 3035 len 24576
12682 153/300 Read block 3668 len 40960
12683 154/300 Write block 4026 len 12288 data: 4 5 5
12684 155/300 Flush
12685 156/300 Read block 1190 len 20480
12686 157/300 Write block 4027 len 16384 data: 6 6 4 3
12687 158/300 Write block 1935 len 4096 data: 4
12688 159/300 Write block 4979 len 8192 data: 3 2
12689 160/300 Read block 261 len 28672
12690 161/300 Read block 4314 len 20480
12691 162/300 Write block 3283 len 36864 data: 2 2 2 2 2 2 2 2 3
12692 163/300 Read block 3577 len 32768
12693 164/300 Write block 3822 len 32768 data: 5 5 5 5 6 5 4 4
12694 165/300 Write block 1262 len 28672 data: 4 4 3 3 3 3 4
12695 166/300 Write block 2738 len 36864 data: 3 2 2 2 2 2 3 3 3
12696 167/300 Write block 1355 len 28672 data: 5 5 5 5 4 3 3
12697 168/300 Write block 1136 len 40960 data: 2 2 2 3 2 2 2 3 4 4
12698 169/300 Read block 2571 len 8192
12699 170/300 Read block 2467 len 16384
12700 171/300 Read block 2614 len 28672
12701 172/300 Read block 741 len 20480
12702 173/300 Read block 2748 len 36864
12703 174/300 Read block 3958 len 16384
12704 175/300 Read block 1796 len 28672
12705 176/300 Read block 3573 len 16384
12706 177/300 Write block 2154 len 16384 data: 5 4 4 3
12707 178/300 Write block 1950 len 28672 data: 3 4 4 3 3 2 2
12708 179/300 Read block 2236 len 28672
12709 180/300 Read block 33 len 32768
12710 181/300 Read block 4247 len 4096
12711 182/300 Flush
12712 183/300 Write block 62 len 36864 data: 4 4 4 4 4 4 3 3 2
12713 184/300 Read block 4400 len 16384
12714 185/300 Read block 4242 len 28672
12715 186/300 Read block 1565 len 40960
12716 187/300 Read block 4348 len 40960
12717 188/300 Flush
12718 189/300 Flush
12719 190/300 Read block 4882 len 24576
12720 191/300 Write block 4581 len 36864 data: 2 2 3 3 3 3 3 3 3
12721 192/300 Write block 4670 len 28672 data: 3 3 3 3 3 3 3
12722 193/300 Write block 1610 len 28672 data: 7 8 7 7 7 7 6
12723 194/300 Flush
12724 195/300 Read block 621 len 16384
12725 196/300 Read block 1177 len 4096
12726 197/300 Read block 3647 len 32768
12727 198/300 Write block 211 len 16384 data: 2 3 3 3
12728 199/300 Flush
12729 200/300 Write block 217 len 36864 data: 2 3 4 3 3 3 3 3 2
12730 201/300 Read block 653 len 4096
12731 202/300 Read block 2714 len 16384
12732 203/300 Read block 1438 len 24576
12733 204/300 Read block 3173 len 24576
12734 205/300 Read block 182 len 28672
12735 206/300 Read block 1969 len 36864
12736 207/300 Write block 3204 len 4096 data: 4
12737 208/300 Write block 2860 len 4096 data: 3
12738 209/300 Read block 4295 len 16384
12739 210/300 Write block 2420 len 24576 data: 2 2 2 2 2 4
12740 211/300 Write block 1777 len 32768 data: 4 6 6 6 4 3 4 4
12741 212/300 Write block 4198 len 8192 data: 2 2
12742 213/300 Read block 2813 len 16384
12743 214/300 Write block 4725 len 8192 data: 2 4
12744 215/300 Read block 4707 len 16384
12745 216/300 Read block 3037 len 40960
12746 217/300 Write block 1617 len 36864 data: 6 5 4 4 4 4 4 5 5
12747 218/300 Write block 1124 len 24576 data: 4 4 4 4 4 4
12748 219/300 Write block 902 len 4096 data: 4
12749 220/300 Read block 741 len 4096
12750 221/300 Read block 149 len 40960
12751 222/300 Read block 4119 len 4096
12752 223/300 Read block 1256 len 40960
12753 224/300 Read block 3659 len 40960
12754 225/300 Read block 1369 len 24576
12755 226/300 Read block 3943 len 12288
12756 227/300 Write block 3385 len 32768 data: 3 5 5 5 5 5 5 5
12757 228/300 Write block 4762 len 8192 data: 4 5
12758 229/300 Write block 2396 len 32768 data: 4 5 5 5 5 4 3 3
12759 230/300 Write block 2013 len 8192 data: 3 4
12760 231/300 Read block 2970 len 28672
12761 232/300 Write block 4538 len 24576 data: 2 2 2 2 2 2
12762 233/300 Read block 4981 len 8192
12763 234/300 Read block 1864 len 32768
12764 235/300 Read block 1488 len 16384
12765 236/300 Flush
12766 237/300 Flush
12767 238/300 Read block 4817 len 28672
12768 239/300 Write block 215 len 20480 data: 4 3 3 4 5
12769 240/300 Read block 4698 len 36864
12770 241/300 Write block 2176 len 4096 data: 3
12771 242/300 Read block 1301 len 40960
12772 243/300 Read block 2418 len 24576
12773 244/300 Read block 4200 len 24576
12774 245/300 Read block 2697 len 8192
12775 246/300 Read block 3734 len 36864
12776 247/300 Write block 2408 len 36864 data: 2 2 2 3 4 4 4 4 4
12777 248/300 Flush
12778 249/300 Write block 3226 len 32768 data: 5 4 4 4 3 2 2 2
12779 250/300 Read block 1750 len 32768
12780 251/300 Write block 1477 len 16384 data: 5 5 5 5
12781 252/300 Read block 137 len 12288
12782 253/300 Write block 4528 len 36864 data: 4 5 5 5 4 4 4 4 4
12783 254/300 Read block 218 len 24576
12784 255/300 Write block 4542 len 20480 data: 3 3 2 2 3
12785 256/300 Write block 3145 len 4096 data: 4
12786 257/300 Read block 3277 len 40960
12787 258/300 Write block 1841 len 40960 data: 4 3 2 2 2 2 2 2 2 2
12788 259/300 Read block 1824 len 24576
12789 260/300 Write block 3317 len 16384 data: 6 6 6 6
12790 261/300 Read block 3575 len 28672
12791 262/300 Read block 4165 len 32768
12792 263/300 Write block 4263 len 20480 data: 3 5 5 6 5
12793 264/300 Write block 1360 len 36864 data: 4 4 3 3 2 2 3 3 3
12794 265/300 Write block 829 len 24576 data: 3 4 4 4 3 4
12795 266/300 Read block 1021 len 16384
12796 267/300 Read block 943 len 8192
12797 268/300 Write block 50 len 36864 data: 2 2 3 3 2 2 2 2 2
12798 269/300 Write block 1402 len 20480 data: 6 6 5 6 8
12799 270/300 Read block 1579 len 24576
12800 271/300 Read block 3974 len 36864
12801 272/300 Write block 3374 len 4096 data: 4
12802 273/300 Write block 1185 len 28672 data: 3 4 5 5 6 6 7
12803 274/300 Flush
12804 275/300 Read block 4213 len 36864
12805 276/300 Read block 1931 len 24576
12806 277/300 Write block 4188 len 8192 data: 3 3
12807 278/300 Read block 150 len 40960
12808 279/300 Flush
12809 280/300 Read block 1908 len 16384
12810 281/300 Flush
12811 282/300 Read block 2665 len 28672
12812 283/300 Write block 1002 len 16384 data: 2 2 2 2
12813 284/300 Write block 3139 len 4096 data: 2
12814 285/300 Flush
12815 286/300 Write block 1391 len 20480 data: 3 3 3 3 3
12816 287/300 Write block 1440 len 32768 data: 2 3 2 2 2 2 4 6
12817 288/300 Read block 4175 len 8192
12818 289/300 Write block 747 len 32768 data: 3 3 3 3 3 2 2 3
12819 290/300 Read block 3145 len 32768
12820 291/300 Read block 1359 len 24576
12821 292/300 Write block 1378 len 4096 data: 3
12822 293/300 Flush
12823 294/300 Read block 4687 len 4096
12824 295/300 Read block 4889 len 28672
12825 296/300 Write block 478 len 20480 data: 5 4 3 3 3
12826 297/300 Read block 2164 len 8192
12827 298/300 Write block 4372 len 24576 data: 2 2 3 3 3 2
12828 299/300 Read block 1459 len 4096
12829 300/300 Write block 595 len 32768 data: 3 4 4 4 4 4 5 4
12830 301/300 Read block 4865 len 32768
12831 Main task has work to do, go find it
12832 got dsc Start(1)
12833 start 1
12834 Nov 19 17:10:42.426 INFO request completed, latency_us: 185, response_code: 204, uri: /start/cid/1, method: POST, req_id: 95e03450-b815-4204-9644-92be8b871a8f, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
12835 Replay: started 1, returned:Ok(())
12836 ----------------------------------------------------------------
12837 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12838 States: Active Offline Active
12839 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12840 4019 Acked Read 6 Done Sent Done false
12841 4020 Acked Read 4 Done Sent Done false
12842 4021 Acked Write 6 Done Sent Done false
12843 4022 Acked Write 9 Done Sent Done false
12844 4023 Acked Read 1 Done Sent Done false
12845 4024 Acked Write 4 Done Sent Done false
12846 4025 Acked Flush 0 Done Sent Done false
12847 4026 Acked Read 4 Done Sent Done false
12848 4027 Acked Write 7 Done Sent Done false
12849 4028 Acked Read 9 Done Sent Done false
12850 4029 Acked Read 6 Done Sent Done false
12851 4030 Acked Read 4 Done Sent Done false
12852 4031 Acked Read 9 Done Sent Done false
12853 4032 Acked Read 3 Done Sent Done false
12854 4033 Acked Read 2 Done Sent Done false
12855 4034 Acked Write 6 Done Sent Done false
12856 4035 Acked Write 6 Done Sent Done false
12857 4036 Acked Flush 0 Done Sent Done false
12858 4037 Acked Write 10 Done Sent Done false
12859 4038 Acked Read 6 Done Sent Done false
12860 4039 Acked Read 6 Done Sent Done false
12861 4040 Acked Flush 0 Done Sent Done false
12862 4041 Acked Write 7 Done Sent Done false
12863 4042 Acked Read 6 Done Sent Done false
12864 4043 Acked Write 5 Done Sent Done false
12865 4044 Acked Flush 0 Done Sent Done false
12866 4045 Acked Read 7 Done Sent Done false
12867 4046 Acked Write 8 Done Sent Done false
12868 4047 Acked Read 8 Done Sent Done false
12869 4048 Acked Read 3 Done Sent Done false
12870 4049 Acked Read 5 Done Sent Done false
12871 4050 Acked Read 3 Done Sent Done false
12872 4051 Acked Read 1 Done Sent Done false
12873 4052 Acked Read 5 Done Sent Done false
12874 4053 Acked Write 1 Done Sent Done false
12875 4054 Acked Read 10 Done Sent Done false
12876 4055 Acked Write 9 Done Sent Done false
12877 4056 Acked Write 1 Done Sent Done false
12878 4057 Acked Read 8 Done Sent Done false
12879 4058 Acked Flush 0 Done Sent Done false
12880 4059 Acked Read 7 Done Sent Done false
12881 4060 Acked Read 7 Done Sent Done false
12882 4061 Acked Read 5 Done Sent Done false
12883 4062 Acked Read 3 Done Sent Done false
12884 4063 Acked Flush 0 Done Sent Done false
12885 4064 Acked Flush 0 Done Sent Done false
12886 4065 Acked Write 3 Done Sent Done false
12887 4066 Acked Read 2 Done Sent Done false
12888 4067 Acked Write 4 Done Sent Done false
12889 4068 Acked Read 6 Done Sent Done false
12890 4069 Acked Write 8 Done Sent Done false
12891 4070 Acked Read 2 Done Sent Done false
12892 4071 Acked Write 7 Done Sent Done false
12893 4072 Acked Flush 0 Done Sent Done false
12894 4073 Acked Read 1 Done Sent Done false
12895 4074 Acked Read 3 Done Sent Done false
12896 4075 Acked Read 3 Done Sent Done false
12897 4076 Acked Read 6 Done Sent Done false
12898 4077 Acked Read 1 Done Sent Done false
12899 4078 Acked Read 7 Done Sent Done false
12900 4079 Acked Read 6 Done Sent Done false
12901 4080 Acked Read 3 Done Sent Done false
12902 4081 Acked Write 2 Done Sent Done false
12903 4082 Acked Read 5 Done Sent Done false
12904 4083 Acked Write 8 Done Sent Done false
12905 4084 Acked Read 8 Done Sent Done false
12906 4085 Acked Read 4 Done Sent Done false
12907 4086 Acked Write 2 Done Sent Done false
12908 4087 Acked Write 10 Done Sent Done false
12909 4088 Acked Write 7 Done Sent Done false
12910 4089 Acked Read 6 Done Sent Done false
12911 4090 Acked Read 4 Done Sent Done false
12912 4091 Acked Read 4 Done Sent Done false
12913 4092 Acked Write 7 Done Sent Done false
12914 4093 Acked Write 7 Done Sent Done false
12915 4094 Acked Read 2 Done Sent Done false
12916 4095 Acked Write 2 Done Sent Done false
12917 4096 Acked Read 3 Done Sent Done false
12918 4097 Acked Write 3 Done Sent Done false
12919 4098 Acked Read 5 Done Sent Done false
12920 4099 Acked Read 7 Done Sent Done false
12921 4100 Acked Read 2 Done Sent Done false
12922 4101 Acked Read 4 Done Sent Done false
12923 4102 Acked Read 1 Done Sent Done false
12924 4103 Acked Read 3 Done Sent Done false
12925 4104 Acked Read 2 Done Sent Done false
12926 4105 Acked Read 2 Done Sent Done false
12927 4106 Acked Read 6 Done Sent Done false
12928 4107 Acked Read 8 Done Sent Done false
12929 4108 Acked Read 10 Done Sent Done false
12930 4109 Acked Read 6 Done Sent Done false
12931 4110 Acked Write 9 Done Sent Done false
12932 4111 Acked Read 4 Done Sent Done false
12933 4112 Acked Read 1 Done Sent Done false
12934 4113 Acked Read 3 Done Sent Done false
12935 4114 Acked Read 5 Done Sent Done false
12936 4115 Acked Read 10 Done Sent Done false
12937 4116 Acked Flush 0 Done Sent Done false
12938 4117 Acked Read 2 Done Sent Done false
12939 4118 Acked Read 2 Done Sent Done false
12940 4119 Acked Write 6 Done Sent Done false
12941 4120 Acked Read 9 Done Sent Done false
12942 4121 Acked Write 3 Done Sent Done false
12943 4122 Acked Read 6 Done Sent Done false
12944 4123 Acked Read 8 Done Sent Done false
12945 4124 Acked Write 9 Done Sent Done false
12946 4125 Acked Write 9 Done Sent Done false
12947 4126 Acked Read 4 Done Sent Done false
12948 4127 Acked Read 5 Done Sent Done false
12949 4128 Acked Write 6 Done Sent Done false
12950 4129 Acked Read 2 Done Sent Done false
12951 4130 Acked Write 3 Done Sent Done false
12952 4131 Acked Flush 0 Done Sent Done false
12953 4132 Acked Write 2 Done Sent Done false
12954 4133 Acked Write 4 Done Sent Done false
12955 4134 Acked Read 7 Done Sent Done false
12956 4135 Acked Write 10 Done Sent Done false
12957 4136 Acked Flush 0 Done Sent Done false
12958 4137 Acked Read 3 Done Sent Done false
12959 4138 Acked Read 8 Done Sent Done false
12960 4139 Acked Flush 0 Done Sent Done false
12961 4140 Acked Read 8 Done Sent Done false
12962 4141 Acked Write 3 Done Sent Done false
12963 4142 Acked Read 4 Done Sent Done false
12964 4143 Acked Write 10 Done Sent Done false
12965 4144 Acked Write 4 Done Sent Done false
12966 4145 Acked Read 4 Done Sent Done false
12967 4146 Acked Write 9 Done Sent Done false
12968 4147 Acked Flush 0 Done Sent Done false
12969 4148 Acked Read 2 Done Sent Done false
12970 4149 Acked Read 6 Done Sent Done false
12971 4150 Acked Read 5 Done Sent Done false
12972 4151 Acked Write 2 Done Sent Done false
12973 4152 Acked Read 5 Done Sent Done false
12974 4153 Acked Write 7 Done Sent Done false
12975 4154 Acked Write 1 Done Sent Done false
12976 4155 Acked Write 5 Done Sent Done false
12977 4156 Acked Read 8 Done Sent Done false
12978 4157 Acked Write 10 Done Sent Done false
12979 4158 Acked Write 8 Done Sent Done false
12980 4159 Acked Write 6 Done Sent Done false
12981 4160 Acked Read 5 Done Sent Done false
12982 4161 Acked Write 4 Done Sent Done false
12983 4162 Acked Write 6 Done Sent Done false
12984 4163 Acked Write 5 Done Sent Done false
12985 4164 Acked Read 4 Done Sent Done false
12986 4165 Acked Read 10 Done Sent Done false
12987 4166 Acked Read 7 Done Sent Done false
12988 4167 Acked Write 2 Done Sent Done false
12989 4168 Acked Read 6 Done Sent Done false
12990 4169 Acked Write 2 Done Sent Done false
12991 4170 Acked Read 6 Done Sent Done false
12992 4171 Acked Read 10 Done Sent Done false
12993 4172 Acked Write 3 Done Sent Done false
12994 4173 Acked Flush 0 Done Sent Done false
12995 4174 Acked Read 5 Done Sent Done false
12996 4175 Acked Write 4 Done Sent Done false
12997 4176 Acked Write 1 Done Sent Done false
12998 4177 Acked Write 2 Done Sent Done false
12999 4178 Acked Read 7 Done Sent Done false
13000 4179 Acked Read 5 Done Sent Done false
13001 4180 Acked Write 9 Done Sent Done false
13002 4181 Acked Read 8 Done Sent Done false
13003 4182 Acked Write 8 Done Sent Done false
13004 4183 Acked Write 7 Done Sent Done false
13005 4184 Acked Write 9 Done Sent Done false
13006 4185 Acked Write 7 Done Sent Done false
13007 4186 Acked Write 10 Done Sent Done false
13008 4187 Acked Read 2 Done Sent Done false
13009 4188 Acked Read 4 Done Sent Done false
13010 4189 Acked Read 7 Done Sent Done false
13011 4190 Acked Read 5 Done Sent Done false
13012 4191 Acked Read 9 Done Sent Done false
13013 4192 Acked Read 4 Done Sent Done false
13014 4193 Acked Read 7 Done Sent Done false
13015 4194 Acked Read 4 Done Sent Done false
13016 4195 Acked Write 4 Done Sent Done false
13017 4196 Acked Write 7 Done Sent Done false
13018 4197 Acked Read 7 Done Sent Done false
13019 4198 Acked Read 8 Done Sent Done false
13020 4199 Acked Read 1 Done Sent Done false
13021 4200 Acked Flush 0 Done Sent Done false
13022 4201 Acked Write 9 Done Sent Done false
13023 4202 Acked Read 4 Done Sent Done false
13024 4203 Acked Read 7 Done Sent Done false
13025 4204 Acked Read 10 Done Sent Done false
13026 4205 Acked Read 10 Done Sent Done false
13027 4206 Acked Flush 0 Done Sent Done false
13028 4207 Acked Flush 0 Done Sent Done false
13029 4208 Acked Read 6 Done Sent Done false
13030 4209 Acked Write 9 Done Sent Done false
13031 4210 Acked Write 7 Done Sent Done false
13032 4211 Acked Write 7 Done Sent Done false
13033 4212 Acked Flush 0 Done Sent Done false
13034 4213 Acked Read 4 Done Sent Done false
13035 4214 Acked Read 1 Done Sent Done false
13036 4215 Acked Read 8 Done Sent Done false
13037 4216 Acked Write 4 Done Sent Done false
13038 4217 Acked Flush 0 Done Sent Done false
13039 4218 Acked Write 9 Done Sent Done false
13040 4219 Acked Read 1 Done Sent Done false
13041 4220 Acked Read 4 Done Sent Done false
13042 4221 Acked Read 6 Done Sent Done false
13043 4222 Acked Read 6 Done Sent Done false
13044 4223 Acked Read 7 Done Sent Done false
13045 4224 Acked Read 9 Done Sent Done false
13046 4225 Acked Write 1 Done Sent Done false
13047 4226 Acked Write 1 Done Sent Done false
13048 4227 Acked Read 4 Done Sent Done false
13049 4228 Acked Write 6 Done Sent Done false
13050 4229 Acked Write 8 Done Sent Done false
13051 4230 Acked Write 2 Done Sent Done false
13052 4231 Acked Read 4 Done Sent Done false
13053 4232 Acked Write 2 Done Sent Done false
13054 4233 Acked Read 4 Done Sent Done false
13055 4234 Acked Read 10 Done Sent Done false
13056 4235 Acked Write 9 Done Sent Done false
13057 4236 Acked Write 6 Done Sent Done false
13058 4237 Acked Write 1 Done Sent Done false
13059 4238 Acked Read 1 Done Sent Done false
13060 4239 Acked Read 10 Done Sent Done false
13061 4240 Acked Read 1 Done Sent Done false
13062 4241 Acked Read 10 Done Sent Done false
13063 4242 Acked Read 10 Done Sent Done false
13064 4243 Acked Read 6 Done Sent Done false
13065 4244 Acked Read 3 Done Sent Done false
13066 4245 Acked Write 8 Done Sent Done false
13067 4246 Acked Write 2 Done Sent Done false
13068 4247 Acked Write 8 Done Sent Done false
13069 4248 Acked Write 2 Done Sent Done false
13070 4249 Acked Read 7 Done Sent Done false
13071 4250 Acked Write 6 Done Sent Done false
13072 4251 Acked Read 2 Done Sent Done false
13073 4252 Acked Read 8 Done Sent Done false
13074 4253 Acked Read 4 Done Sent Done false
13075 4254 Acked Flush 0 Done Sent Done false
13076 4255 Acked Flush 0 Done Sent Done false
13077 4256 Acked Read 7 Done Sent Done false
13078 4257 Acked Write 5 Done Sent Done false
13079 4258 Acked Read 9 Done Sent Done false
13080 4259 Acked Write 1 Done Sent Done false
13081 4260 Acked Read 10 Done Sent Done false
13082 4261 Acked Read 6 Done Sent Done false
13083 4262 Acked Read 6 Done Sent Done false
13084 4263 Acked Read 2 Done Sent Done false
13085 4264 Acked Read 9 Done Sent Done false
13086 4265 Acked Write 9 Done Sent Done false
13087 4266 Acked Flush 0 Done Sent Done false
13088 4267 Acked Write 8 Done Sent Done false
13089 4268 Acked Read 8 Done Sent Done false
13090 4269 Acked Write 4 Done Sent Done false
13091 4270 Acked Read 3 Done Sent Done false
13092 4271 Acked Write 9 Done Sent Done false
13093 4272 Acked Read 6 Done Sent Done false
13094 4273 Acked Write 5 Done Sent Done false
13095 4274 Acked Write 1 Done Sent Done false
13096 4275 Acked Read 10 Done Sent Done false
13097 4276 Acked Write 10 Done Sent Done false
13098 4277 Acked Read 6 Done Sent Done false
13099 4278 Acked Write 4 Done Sent Done false
13100 4279 Acked Read 7 Done Sent Done false
13101 4280 Acked Read 8 Done Sent Done false
13102 4281 Acked Write 5 Done Sent Done false
13103 4282 Acked Write 9 Done Sent Done false
13104 4283 Acked Write 6 Done Sent Done false
13105 4284 Acked Read 4 Done Sent Done false
13106 4285 Acked Read 2 Done Sent Done false
13107 4286 Acked Write 9 Done Sent Done false
13108 4287 Acked Write 5 Done Sent Done false
13109 4288 Acked Read 6 Done Sent Done false
13110 4289 Acked Read 9 Done Sent Done false
13111 4290 Acked Write 1 Done Sent Done false
13112 4291 Acked Write 7 Done Sent Done false
13113 4292 Acked Flush 0 Done Sent Done false
13114 4293 Acked Read 9 Done Sent Done false
13115 4294 Acked Read 6 Done Sent Done false
13116 4295 Acked Write 2 Done Sent Done false
13117 4296 Acked Read 10 Done Sent Done false
13118 4297 Acked Flush 0 Done Sent Done false
13119 4298 Acked Read 4 Done Sent Done false
13120 4299 Acked Flush 0 Done Sent Done false
13121 4300 Acked Read 7 Done Sent Done false
13122 4301 Acked Write 4 Done Sent Done false
13123 4302 Acked Write 1 Done Sent Done false
13124 4303 Acked Flush 0 Done Sent Done false
13125 4304 Acked Write 5 Done Sent Done false
13126 4305 Acked Write 8 Done Sent Done false
13127 4306 Acked Read 2 Done Sent Done false
13128 4307 Acked Write 8 Done Sent Done false
13129 4308 Acked Read 8 Done Sent Done false
13130 4309 Acked Read 6 Done Sent Done false
13131 4310 Acked Write 1 Done Sent Done false
13132 4311 Acked Flush 0 Done Sent Done false
13133 4312 Acked Read 1 Done Sent Done false
13134 4313 Acked Read 7 Done Sent Done false
13135 4314 Acked Write 5 Done Sent Done false
13136 4315 Acked Read 2 Done Sent Done false
13137 4316 Acked Write 6 Done Sent Done false
13138 4317 Acked Read 1 Done Sent Done false
13139 4318 Acked Write 8 Done Sent Done false
13140 4319 Acked Read 8 Done Sent Done false
13141 STATES DS:0 DS:1 DS:2 TOTAL
13142 Sent 0 301 0 301
13143 Done 301 0 301 602
13144 Skipped 0 0 0 0
13145 Error 0 0 0 0
13146 Last Flush: 4311 4018 4311
13147 Downstairs last five completed: 4018 4017 4016 4015 4014
13148 Upstairs last five completed: 4319 4318 4317 4316 4315
13149 CLIENT: Up:0 ds:301 act:2
13150 [8820] got start action
13151 [8820] I am going to restart
13152 Starting downstairs at port 8820
13153 Make output file at "/tmp/dsc/downstairs-8820.txt"
13154 [8820][1] reports Starting
13155 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
13156 [8820][1] reports Running
13157 ----------------------------------------------------------------
13158 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13159 States: Active Offline Active
13160 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13161 4019 Acked Read 6 Done Sent Done false
13162 4020 Acked Read 4 Done Sent Done false
13163 4021 Acked Write 6 Done Sent Done false
13164 4022 Acked Write 9 Done Sent Done false
13165 4023 Acked Read 1 Done Sent Done false
13166 4024 Acked Write 4 Done Sent Done false
13167 4025 Acked Flush 0 Done Sent Done false
13168 4026 Acked Read 4 Done Sent Done false
13169 4027 Acked Write 7 Done Sent Done false
13170 4028 Acked Read 9 Done Sent Done false
13171 4029 Acked Read 6 Done Sent Done false
13172 4030 Acked Read 4 Done Sent Done false
13173 4031 Acked Read 9 Done Sent Done false
13174 4032 Acked Read 3 Done Sent Done false
13175 4033 Acked Read 2 Done Sent Done false
13176 4034 Acked Write 6 Done Sent Done false
13177 4035 Acked Write 6 Done Sent Done false
13178 4036 Acked Flush 0 Done Sent Done false
13179 4037 Acked Write 10 Done Sent Done false
13180 4038 Acked Read 6 Done Sent Done false
13181 4039 Acked Read 6 Done Sent Done false
13182 4040 Acked Flush 0 Done Sent Done false
13183 4041 Acked Write 7 Done Sent Done false
13184 4042 Acked Read 6 Done Sent Done false
13185 4043 Acked Write 5 Done Sent Done false
13186 4044 Acked Flush 0 Done Sent Done false
13187 4045 Acked Read 7 Done Sent Done false
13188 4046 Acked Write 8 Done Sent Done false
13189 4047 Acked Read 8 Done Sent Done false
13190 4048 Acked Read 3 Done Sent Done false
13191 4049 Acked Read 5 Done Sent Done false
13192 4050 Acked Read 3 Done Sent Done false
13193 4051 Acked Read 1 Done Sent Done false
13194 4052 Acked Read 5 Done Sent Done false
13195 4053 Acked Write 1 Done Sent Done false
13196 4054 Acked Read 10 Done Sent Done false
13197 4055 Acked Write 9 Done Sent Done false
13198 4056 Acked Write 1 Done Sent Done false
13199 4057 Acked Read 8 Done Sent Done false
13200 4058 Acked Flush 0 Done Sent Done false
13201 4059 Acked Read 7 Done Sent Done false
13202 4060 Acked Read 7 Done Sent Done false
13203 4061 Acked Read 5 Done Sent Done false
13204 4062 Acked Read 3 Done Sent Done false
13205 4063 Acked Flush 0 Done Sent Done false
13206 4064 Acked Flush 0 Done Sent Done false
13207 4065 Acked Write 3 Done Sent Done false
13208 4066 Acked Read 2 Done Sent Done false
13209 4067 Acked Write 4 Done Sent Done false
13210 4068 Acked Read 6 Done Sent Done false
13211 4069 Acked Write 8 Done Sent Done false
13212 4070 Acked Read 2 Done Sent Done false
13213 4071 Acked Write 7 Done Sent Done false
13214 4072 Acked Flush 0 Done Sent Done false
13215 4073 Acked Read 1 Done Sent Done false
13216 4074 Acked Read 3 Done Sent Done false
13217 4075 Acked Read 3 Done Sent Done false
13218 4076 Acked Read 6 Done Sent Done false
13219 4077 Acked Read 1 Done Sent Done false
13220 4078 Acked Read 7 Done Sent Done false
13221 4079 Acked Read 6 Done Sent Done false
13222 4080 Acked Read 3 Done Sent Done false
13223 4081 Acked Write 2 Done Sent Done false
13224 4082 Acked Read 5 Done Sent Done false
13225 4083 Acked Write 8 Done Sent Done false
13226 4084 Acked Read 8 Done Sent Done false
13227 4085 Acked Read 4 Done Sent Done false
13228 4086 Acked Write 2 Done Sent Done false
13229 4087 Acked Write 10 Done Sent Done false
13230 4088 Acked Write 7 Done Sent Done false
13231 4089 Acked Read 6 Done Sent Done false
13232 4090 Acked Read 4 Done Sent Done false
13233 4091 Acked Read 4 Done Sent Done false
13234 4092 Acked Write 7 Done Sent Done false
13235 4093 Acked Write 7 Done Sent Done false
13236 4094 Acked Read 2 Done Sent Done false
13237 4095 Acked Write 2 Done Sent Done false
13238 4096 Acked Read 3 Done Sent Done false
13239 4097 Acked Write 3 Done Sent Done false
13240 4098 Acked Read 5 Done Sent Done false
13241 4099 Acked Read 7 Done Sent Done false
13242 4100 Acked Read 2 Done Sent Done false
13243 4101 Acked Read 4 Done Sent Done false
13244 4102 Acked Read 1 Done Sent Done false
13245 4103 Acked Read 3 Done Sent Done false
13246 4104 Acked Read 2 Done Sent Done false
13247 4105 Acked Read 2 Done Sent Done false
13248 4106 Acked Read 6 Done Sent Done false
13249 4107 Acked Read 8 Done Sent Done false
13250 4108 Acked Read 10 Done Sent Done false
13251 4109 Acked Read 6 Done Sent Done false
13252 4110 Acked Write 9 Done Sent Done false
13253 4111 Acked Read 4 Done Sent Done false
13254 4112 Acked Read 1 Done Sent Done false
13255 4113 Acked Read 3 Done Sent Done false
13256 4114 Acked Read 5 Done Sent Done false
13257 4115 Acked Read 10 Done Sent Done false
13258 4116 Acked Flush 0 Done Sent Done false
13259 4117 Acked Read 2 Done Sent Done false
13260 4118 Acked Read 2 Done Sent Done false
13261 4119 Acked Write 6 Done Sent Done false
13262 4120 Acked Read 9 Done Sent Done false
13263 4121 Acked Write 3 Done Sent Done false
13264 4122 Acked Read 6 Done Sent Done false
13265 4123 Acked Read 8 Done Sent Done false
13266 4124 Acked Write 9 Done Sent Done false
13267 4125 Acked Write 9 Done Sent Done false
13268 4126 Acked Read 4 Done Sent Done false
13269 4127 Acked Read 5 Done Sent Done false
13270 4128 Acked Write 6 Done Sent Done false
13271 4129 Acked Read 2 Done Sent Done false
13272 4130 Acked Write 3 Done Sent Done false
13273 4131 Acked Flush 0 Done Sent Done false
13274 4132 Acked Write 2 Done Sent Done false
13275 4133 Acked Write 4 Done Sent Done false
13276 4134 Acked Read 7 Done Sent Done false
13277 4135 Acked Write 10 Done Sent Done false
13278 4136 Acked Flush 0 Done Sent Done false
13279 4137 Acked Read 3 Done Sent Done false
13280 4138 Acked Read 8 Done Sent Done false
13281 4139 Acked Flush 0 Done Sent Done false
13282 4140 Acked Read 8 Done Sent Done false
13283 4141 Acked Write 3 Done Sent Done false
13284 4142 Acked Read 4 Done Sent Done false
13285 4143 Acked Write 10 Done Sent Done false
13286 4144 Acked Write 4 Done Sent Done false
13287 4145 Acked Read 4 Done Sent Done false
13288 4146 Acked Write 9 Done Sent Done false
13289 4147 Acked Flush 0 Done Sent Done false
13290 4148 Acked Read 2 Done Sent Done false
13291 4149 Acked Read 6 Done Sent Done false
13292 4150 Acked Read 5 Done Sent Done false
13293 4151 Acked Write 2 Done Sent Done false
13294 4152 Acked Read 5 Done Sent Done false
13295 4153 Acked Write 7 Done Sent Done false
13296 4154 Acked Write 1 Done Sent Done false
13297 4155 Acked Write 5 Done Sent Done false
13298 4156 Acked Read 8 Done Sent Done false
13299 4157 Acked Write 10 Done Sent Done false
13300 4158 Acked Write 8 Done Sent Done false
13301 4159 Acked Write 6 Done Sent Done false
13302 4160 Acked Read 5 Done Sent Done false
13303 4161 Acked Write 4 Done Sent Done false
13304 4162 Acked Write 6 Done Sent Done false
13305 4163 Acked Write 5 Done Sent Done false
13306 4164 Acked Read 4 Done Sent Done false
13307 4165 Acked Read 10 Done Sent Done false
13308 4166 Acked Read 7 Done Sent Done false
13309 4167 Acked Write 2 Done Sent Done false
13310 4168 Acked Read 6 Done Sent Done false
13311 4169 Acked Write 2 Done Sent Done false
13312 4170 Acked Read 6 Done Sent Done false
13313 4171 Acked Read 10 Done Sent Done false
13314 4172 Acked Write 3 Done Sent Done false
13315 4173 Acked Flush 0 Done Sent Done false
13316 4174 Acked Read 5 Done Sent Done false
13317 4175 Acked Write 4 Done Sent Done false
13318 4176 Acked Write 1 Done Sent Done false
13319 4177 Acked Write 2 Done Sent Done false
13320 4178 Acked Read 7 Done Sent Done false
13321 4179 Acked Read 5 Done Sent Done false
13322 4180 Acked Write 9 Done Sent Done false
13323 4181 Acked Read 8 Done Sent Done false
13324 4182 Acked Write 8 Done Sent Done false
13325 4183 Acked Write 7 Done Sent Done false
13326 4184 Acked Write 9 Done Sent Done false
13327 4185 Acked Write 7 Done Sent Done false
13328 4186 Acked Write 10 Done Sent Done false
13329 4187 Acked Read 2 Done Sent Done false
13330 4188 Acked Read 4 Done Sent Done false
13331 4189 Acked Read 7 Done Sent Done false
13332 4190 Acked Read 5 Done Sent Done false
13333 4191 Acked Read 9 Done Sent Done false
13334 4192 Acked Read 4 Done Sent Done false
13335 4193 Acked Read 7 Done Sent Done false
13336 4194 Acked Read 4 Done Sent Done false
13337 4195 Acked Write 4 Done Sent Done false
13338 4196 Acked Write 7 Done Sent Done false
13339 4197 Acked Read 7 Done Sent Done false
13340 4198 Acked Read 8 Done Sent Done false
13341 4199 Acked Read 1 Done Sent Done false
13342 4200 Acked Flush 0 Done Sent Done false
13343 4201 Acked Write 9 Done Sent Done false
13344 4202 Acked Read 4 Done Sent Done false
13345 4203 Acked Read 7 Done Sent Done false
13346 4204 Acked Read 10 Done Sent Done false
13347 4205 Acked Read 10 Done Sent Done false
13348 4206 Acked Flush 0 Done Sent Done false
13349 4207 Acked Flush 0 Done Sent Done false
13350 4208 Acked Read 6 Done Sent Done false
13351 4209 Acked Write 9 Done Sent Done false
13352 4210 Acked Write 7 Done Sent Done false
13353 4211 Acked Write 7 Done Sent Done false
13354 4212 Acked Flush 0 Done Sent Done false
13355 4213 Acked Read 4 Done Sent Done false
13356 4214 Acked Read 1 Done Sent Done false
13357 4215 Acked Read 8 Done Sent Done false
13358 4216 Acked Write 4 Done Sent Done false
13359 4217 Acked Flush 0 Done Sent Done false
13360 4218 Acked Write 9 Done Sent Done false
13361 4219 Acked Read 1 Done Sent Done false
13362 4220 Acked Read 4 Done Sent Done false
13363 4221 Acked Read 6 Done Sent Done false
13364 4222 Acked Read 6 Done Sent Done false
13365 4223 Acked Read 7 Done Sent Done false
13366 4224 Acked Read 9 Done Sent Done false
13367 4225 Acked Write 1 Done Sent Done false
13368 4226 Acked Write 1 Done Sent Done false
13369 4227 Acked Read 4 Done Sent Done false
13370 4228 Acked Write 6 Done Sent Done false
13371 4229 Acked Write 8 Done Sent Done false
13372 4230 Acked Write 2 Done Sent Done false
13373 4231 Acked Read 4 Done Sent Done false
13374 4232 Acked Write 2 Done Sent Done false
13375 4233 Acked Read 4 Done Sent Done false
13376 4234 Acked Read 10 Done Sent Done false
13377 4235 Acked Write 9 Done Sent Done false
13378 4236 Acked Write 6 Done Sent Done false
13379 4237 Acked Write 1 Done Sent Done false
13380 4238 Acked Read 1 Done Sent Done false
13381 4239 Acked Read 10 Done Sent Done false
13382 4240 Acked Read 1 Done Sent Done false
13383 4241 Acked Read 10 Done Sent Done false
13384 4242 Acked Read 10 Done Sent Done false
13385 4243 Acked Read 6 Done Sent Done false
13386 4244 Acked Read 3 Done Sent Done false
13387 4245 Acked Write 8 Done Sent Done false
13388 4246 Acked Write 2 Done Sent Done false
13389 4247 Acked Write 8 Done Sent Done false
13390 4248 Acked Write 2 Done Sent Done false
13391 4249 Acked Read 7 Done Sent Done false
13392 4250 Acked Write 6 Done Sent Done false
13393 4251 Acked Read 2 Done Sent Done false
13394 4252 Acked Read 8 Done Sent Done false
13395 4253 Acked Read 4 Done Sent Done false
13396 4254 Acked Flush 0 Done Sent Done false
13397 4255 Acked Flush 0 Done Sent Done false
13398 4256 Acked Read 7 Done Sent Done false
13399 4257 Acked Write 5 Done Sent Done false
13400 4258 Acked Read 9 Done Sent Done false
13401 4259 Acked Write 1 Done Sent Done false
13402 4260 Acked Read 10 Done Sent Done false
13403 4261 Acked Read 6 Done Sent Done false
13404 4262 Acked Read 6 Done Sent Done false
13405 4263 Acked Read 2 Done Sent Done false
13406 4264 Acked Read 9 Done Sent Done false
13407 4265 Acked Write 9 Done Sent Done false
13408 4266 Acked Flush 0 Done Sent Done false
13409 4267 Acked Write 8 Done Sent Done false
13410 4268 Acked Read 8 Done Sent Done false
13411 4269 Acked Write 4 Done Sent Done false
13412 4270 Acked Read 3 Done Sent Done false
13413 4271 Acked Write 9 Done Sent Done false
13414 4272 Acked Read 6 Done Sent Done false
13415 4273 Acked Write 5 Done Sent Done false
13416 4274 Acked Write 1 Done Sent Done false
13417 4275 Acked Read 10 Done Sent Done false
13418 4276 Acked Write 10 Done Sent Done false
13419 4277 Acked Read 6 Done Sent Done false
13420 4278 Acked Write 4 Done Sent Done false
13421 4279 Acked Read 7 Done Sent Done false
13422 4280 Acked Read 8 Done Sent Done false
13423 4281 Acked Write 5 Done Sent Done false
13424 4282 Acked Write 9 Done Sent Done false
13425 4283 Acked Write 6 Done Sent Done false
13426 4284 Acked Read 4 Done Sent Done false
13427 4285 Acked Read 2 Done Sent Done false
13428 4286 Acked Write 9 Done Sent Done false
13429 4287 Acked Write 5 Done Sent Done false
13430 4288 Acked Read 6 Done Sent Done false
13431 4289 Acked Read 9 Done Sent Done false
13432 4290 Acked Write 1 Done Sent Done false
13433 4291 Acked Write 7 Done Sent Done false
13434 4292 Acked Flush 0 Done Sent Done false
13435 4293 Acked Read 9 Done Sent Done false
13436 4294 Acked Read 6 Done Sent Done false
13437 4295 Acked Write 2 Done Sent Done false
13438 4296 Acked Read 10 Done Sent Done false
13439 4297 Acked Flush 0 Done Sent Done false
13440 4298 Acked Read 4 Done Sent Done false
13441 4299 Acked Flush 0 Done Sent Done false
13442 4300 Acked Read 7 Done Sent Done false
13443 4301 Acked Write 4 Done Sent Done false
13444 4302 Acked Write 1 Done Sent Done false
13445 4303 Acked Flush 0 Done Sent Done false
13446 4304 Acked Write 5 Done Sent Done false
13447 4305 Acked Write 8 Done Sent Done false
13448 4306 Acked Read 2 Done Sent Done false
13449 4307 Acked Write 8 Done Sent Done false
13450 4308 Acked Read 8 Done Sent Done false
13451 4309 Acked Read 6 Done Sent Done false
13452 4310 Acked Write 1 Done Sent Done false
13453 4311 Acked Flush 0 Done Sent Done false
13454 4312 Acked Read 1 Done Sent Done false
13455 4313 Acked Read 7 Done Sent Done false
13456 4314 Acked Write 5 Done Sent Done false
13457 4315 Acked Read 2 Done Sent Done false
13458 4316 Acked Write 6 Done Sent Done false
13459 4317 Acked Read 1 Done Sent Done false
13460 4318 Acked Write 8 Done Sent Done false
13461 4319 Acked Read 8 Done Sent Done false
13462 4320 Acked Flush 0 Done Sent Done false
13463 STATES DS:0 DS:1 DS:2 TOTAL
13464 Sent 0 302 0 302
13465 Done 302 0 302 604
13466 Skipped 0 0 0 0
13467 Error 0 0 0 0
13468 Last Flush: 4320 4018 4320
13469 Downstairs last five completed: 4018 4017 4016 4015 4014
13470 Upstairs last five completed: 4320 4319 4318 4317 4316
13471 CLIENT: Up:0 ds:302 act:2
134722024-11-19T17:10:48.184ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134732024-11-19T17:10:48.184ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55970) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134742024-11-19T17:10:48.185ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134752024-11-19T17:10:48.186ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134762024-11-19T17:10:48.186ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134772024-11-19T17:10:48.186ZINFOcrucible: send last flush ID to this DS: 4018 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134782024-11-19T17:10:48.186ZINFOcrucible: Replied this last flush ID: 4018 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134792024-11-19T17:10:48.186ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134802024-11-19T17:10:48.186ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134812024-11-19T17:10:48.186ZINFOcrucible: [1] client re-new 302 jobs since flush 4018 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
134822024-11-19T17:10:48.186ZINFOcrucible: [1] Replayed 302 jobs since flush: 4018 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
13483 ----------------------------------------------------------------
13484 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13485 Downstairs last five completed: 4320 4319 4318 4317 4316
13486 Upstairs last five completed: 4320 4319 4318 4317 4316
13487 CLIENT: Up:0 ds:0 act:3
13488 Replay: All jobs finished
13489 Main task has work to do, go find it
13490 got dsc Stop(0)
13491 stop 0
13492 [8810] Got stop action so:false kr:false
13493 Nov 19 17:10:50.432 INFO request completed, latency_us: 495, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 53c8aa4b-4cdb-4a1f-b326-c0a0455f5590, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
134942024-11-19T17:10:50.432ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134952024-11-19T17:10:50.432ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134962024-11-19T17:10:50.433ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
134972024-11-19T17:10:50.433ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
134982024-11-19T17:10:50.433ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
13499 Nov 19 17:10:50.433 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/0, method: GET, req_id: 48de30bd-a16e-4217-8e95-50868eadb225, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
135002024-11-19T17:10:50.433ZINFOcrucible: client task is exiting = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
13501 [8810] Exited with: ExitStatus(unix_wait_status(9))
13502 [8810][0] reports Exit
13503 Nov 19 17:10:54.435 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/0, method: GET, req_id: 51a34447-ad49-459c-a294-13e16824fe46, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
13504 001/300 Write block 488 len 24576 data: 4 4 4 5 5 4
13505 002/300 Read block 4244 len 16384
13506 003/300 Read block 3896 len 32768
13507 004/300 Read block 1447 len 4096
13508 005/300 Write block 4491 len 28672 data: 5 5 5 2 2 2 2
13509 006/300 Flush
13510 007/300 Read block 1568 len 36864
13511 008/300 Read block 2119 len 4096
13512 009/300 Read block 1812 len 12288
13513 010/300 Write block 3926 len 16384 data: 4 4 4 5
13514 011/300 Read block 871 len 8192
13515 012/300 Write block 1910 len 20480 data: 3 3 3 3 3
13516 013/300 Write block 4281 len 16384 data: 3 4 3 3
13517 014/300 Write block 900 len 40960 data: 2 4 5 4 4 5 5 4 3 3
13518 015/300 Write block 2654 len 16384 data: 4 4 4 4
13519 016/300 Flush
13520 017/300 Read block 2586 len 36864
13521 018/300 Read block 4153 len 4096
13522 019/300 Read block 2738 len 16384
13523 020/300 Write block 3245 len 20480 data: 4 3 3 3 4
13524 021/300 Write block 4889 len 32768 data: 3 3 3 3 3 4 4 5
13525 022/300 Write block 616 len 20480 data: 5 4 5 5 5
13526 023/300 Write block 814 len 40960 data: 2 2 2 2 3 3 3 2 2 2
13527 024/300 Flush
13528 025/300 Read block 601 len 4096
13529 026/300 Write block 2937 len 12288 data: 4 4 4
13530 027/300 Write block 4526 len 24576 data: 4 4 5 6 6 6
13531 028/300 Read block 959 len 32768
13532 029/300 Read block 3091 len 28672
13533 030/300 Read block 1530 len 32768
13534 031/300 Read block 1703 len 24576
13535 032/300 Write block 868 len 36864 data: 2 2 3 3 3 3 4 4 3
13536 033/300 Read block 1189 len 24576
13537 034/300 Write block 96 len 20480 data: 3 3 4 4 6
13538 035/300 Write block 3232 len 36864 data: 3 3 2 2 2 2 3 3 4
13539 036/300 Read block 1751 len 16384
13540 037/300 Read block 1450 len 12288
13541 038/300 Read block 1497 len 8192
13542 039/300 Write block 2364 len 28672 data: 4 4 4 4 3 2 2
13543 040/300 Read block 903 len 24576
13544 041/300 Read block 748 len 16384
13545 042/300 Read block 2322 len 40960
13546 043/300 Flush
13547 044/300 Read block 2169 len 20480
13548 045/300 Read block 4351 len 4096
13549 046/300 Read block 1345 len 28672
13550 047/300 Flush
13551 048/300 Write block 1009 len 24576 data: 4 4 2 2 2 2
13552 049/300 Write block 3962 len 40960 data: 4 4 3 3 3 2 2 2 2 2
13553 050/300 Write block 4998 len 8192 data: 2 2
13554 051/300 Read block 1322 len 4096
13555 052/300 Write block 4189 len 16384 data: 4 2 2 3
13556 053/300 Write block 4987 len 32768 data: 2 2 2 2 2 2 2 2
13557 054/300 Write block 1151 len 36864 data: 3 2 2 2 2 2 3 4 5
13558 055/300 Read block 1343 len 24576
13559 056/300 Write block 1806 len 36864 data: 3 3 3 2 2 2 2 2 2
13560 057/300 Write block 4082 len 20480 data: 3 3 2 2 2
13561 058/300 Read block 147 len 32768
13562 059/300 Write block 4426 len 32768 data: 5 5 4 4 3 3 4 4
13563 060/300 Flush
13564 061/300 Write block 3013 len 40960 data: 2 3 3 3 3 3 3 3 3 3
13565 062/300 Write block 4288 len 12288 data: 4 3 3
13566 063/300 Flush
13567 064/300 Read block 1565 len 12288
13568 065/300 Flush
13569 066/300 Write block 2745 len 36864 data: 4 4 3 3 3 4 4 4 4
13570 067/300 Read block 2028 len 16384
13571 068/300 Write block 4294 len 32768 data: 2 2 3 4 3 2 2 2
13572 069/300 Write block 3605 len 4096 data: 2
13573 070/300 Write block 3811 len 4096 data: 4
13574 071/300 Read block 2413 len 16384
13575 072/300 Flush
13576 073/300 Write block 496 len 40960 data: 5 4 4 5 4 4 2 2 2 2
13577 074/300 Write block 3943 len 40960 data: 2 2 3 3 3 4 3 3 4 4
13578 075/300 Read block 1233 len 8192
13579 076/300 Flush
13580 077/300 Read block 1216 len 4096
13581 078/300 Read block 2168 len 8192
13582 079/300 Write block 3624 len 4096 data: 3
13583 080/300 Read block 192 len 8192
13584 081/300 Read block 2245 len 36864
13585 082/300 Read block 3867 len 36864
13586 083/300 Read block 3823 len 36864
13587 084/300 Read block 4720 len 4096
13588 085/300 Write block 1226 len 28672 data: 2 2 3 2 4 4 5
13589 086/300 Write block 4032 len 12288 data: 4 4 3
13590 087/300 Write block 4168 len 8192 data: 2 2
13591 088/300 Write block 4379 len 36864 data: 3 3 4 4 4 3 3 3 3
13592 089/300 Read block 2288 len 12288
13593 090/300 Write block 4181 len 20480 data: 3 3 3 3 2
13594 091/300 Flush
13595 092/300 Read block 4438 len 32768
13596 093/300 Read block 2691 len 32768
13597 094/300 Read block 1035 len 16384
13598 095/300 Read block 1219 len 4096
13599 096/300 Read block 4022 len 16384
13600 097/300 Read block 3531 len 20480
13601 098/300 Flush
13602 099/300 Read block 4551 len 36864
13603 100/300 Flush
13604 101/300 Flush
13605 102/300 Write block 3544 len 20480 data: 3 2 2 4 4
13606 103/300 Read block 4861 len 16384
13607 104/300 Read block 3214 len 24576
13608 105/300 Read block 2511 len 8192
13609 106/300 Read block 4761 len 40960
13610 107/300 Read block 4762 len 8192
13611 108/300 Flush
13612 109/300 Read block 4522 len 12288
13613 110/300 Read block 3167 len 32768
13614 111/300 Read block 2026 len 40960
13615 112/300 Write block 1585 len 8192 data: 5 4
13616 113/300 Read block 1430 len 4096
13617 114/300 Write block 483 len 40960 data: 2 2 3 3 3 5 5 5 6 6
13618 115/300 Write block 1087 len 24576 data: 2 2 3 4 5 4
13619 116/300 Write block 2814 len 20480 data: 4 3 3 3 3
13620 117/300 Read block 4015 len 20480
13621 118/300 Flush
13622 119/300 Write block 2432 len 12288 data: 3 3 5
13623 120/300 Read block 1694 len 20480
13624 121/300 Write block 1973 len 20480 data: 3 4 4 4 4
13625 122/300 Write block 4602 len 32768 data: 2 2 2 3 3 3 3 3
13626 123/300 Read block 2758 len 24576
13627 124/300 Write block 1466 len 40960 data: 3 4 3 3 3 3 3 3 3 2
13628 125/300 Write block 2255 len 20480 data: 4 5 5 4 3
13629 126/300 Write block 3714 len 8192 data: 2 2
13630 127/300 Flush
13631 128/300 Read block 3579 len 28672
13632 129/300 Flush
13633 130/300 Write block 4495 len 12288 data: 3 3 3
13634 131/300 Write block 1319 len 32768 data: 5 5 4 3 3 2 2 2
13635 132/300 Write block 3967 len 12288 data: 3 3 3
13636 133/300 Write block 4191 len 24576 data: 3 4 3 3 4 4
13637 134/300 Write block 667 len 32768 data: 6 6 7 7 7 6 7 7
13638 135/300 Flush
13639 136/300 Read block 1538 len 8192
13640 137/300 Write block 3333 len 32768 data: 3 3 3 4 4 4 4 4
13641 138/300 Read block 1107 len 40960
13642 139/300 Write block 2341 len 4096 data: 3
13643 140/300 Write block 4419 len 32768 data: 2 2 4 4 4 5 5 6
13644 141/300 Read block 1681 len 4096
13645 142/300 Flush
13646 143/300 Flush
13647 144/300 Read block 1825 len 28672
13648 145/300 Write block 3904 len 24576 data: 3 2 3 3 3 4
13649 146/300 Flush
13650 147/300 Flush
13651 148/300 Write block 2839 len 32768 data: 2 2 4 4 4 4 4 4
13652 149/300 Read block 2243 len 28672
13653 150/300 Write block 1013 len 4096 data: 3
13654 151/300 Read block 4133 len 32768
13655 152/300 Write block 1977 len 16384 data: 5 4 4 4
13656 153/300 Read block 2955 len 40960
13657 154/300 Write block 4855 len 12288 data: 2 2 2
13658 155/300 Write block 722 len 12288 data: 2 4 4
13659 156/300 Write block 1717 len 40960 data: 2 2 2 2 2 3 3 3 3 3
13660 157/300 Write block 2330 len 16384 data: 4 4 3 2
13661 158/300 Write block 4578 len 36864 data: 3 2 2 3 3 4 4 4 4
13662 159/300 Read block 3837 len 40960
13663 160/300 Read block 877 len 20480
13664 161/300 Write block 4263 len 20480 data: 4 6 6 7 6
13665 162/300 Read block 3399 len 8192
13666 163/300 Read block 2343 len 40960
13667 164/300 Write block 2047 len 40960 data: 4 2 2 2 3 2 2 2 2 2
13668 165/300 Read block 4199 len 20480
13669 166/300 Read block 1302 len 12288
13670 167/300 Write block 2660 len 32768 data: 4 4 5 4 4 4 4 4
13671 168/300 Write block 4410 len 24576 data: 3 3 3 3 2 2
13672 169/300 Read block 1029 len 32768
13673 170/300 Write block 3583 len 4096 data: 3
13674 171/300 Write block 3455 len 32768 data: 4 5 4 3 3 2 2 3
13675 172/300 Read block 1471 len 4096
13676 173/300 Write block 1884 len 8192 data: 7 6
13677 174/300 Write block 4562 len 12288 data: 4 4 5
13678 175/300 Read block 3758 len 20480
13679 176/300 Read block 3294 len 28672
13680 177/300 Flush
13681 178/300 Read block 1480 len 32768
13682 179/300 Read block 2909 len 8192
13683 180/300 Read block 1457 len 8192
13684 181/300 Read block 4694 len 16384
13685 182/300 Read block 597 len 40960
13686 183/300 Write block 719 len 28672 data: 2 2 2 3 5 5 4
13687 184/300 Read block 1717 len 36864
13688 185/300 Write block 1771 len 20480 data: 2 2 2 2 2
13689 186/300 Read block 2224 len 8192
13690 187/300 Write block 2148 len 12288 data: 3 5 5
13691 188/300 Read block 4902 len 28672
13692 189/300 Read block 663 len 4096
13693 190/300 Read block 424 len 16384
13694 191/300 Read block 965 len 40960
13695 192/300 Read block 4438 len 40960
13696 193/300 Write block 2347 len 4096 data: 5
13697 194/300 Flush
13698 195/300 Write block 1736 len 12288 data: 4 5 4
13699 196/300 Read block 2072 len 36864
13700 197/300 Write block 4846 len 8192 data: 5 4
13701 198/300 Read block 4390 len 8192
13702 199/300 Read block 2630 len 20480
13703 200/300 Write block 4773 len 32768 data: 4 5 5 6 5 5 4 3
13704 201/300 Read block 2911 len 4096
13705 202/300 Write block 1876 len 36864 data: 4 5 7 7 6 5 5 7 8
13706 203/300 Write block 1762 len 24576 data: 3 3 3 2 2 2
13707 204/300 Read block 3675 len 32768
13708 205/300 Write block 1712 len 8192 data: 4 3
13709 206/300 Write block 4514 len 24576 data: 3 3 4 5 5 5
13710 207/300 Write block 4644 len 16384 data: 2 2 2 2
13711 208/300 Read block 2795 len 40960
13712 209/300 Flush
13713 210/300 Write block 3200 len 32768 data: 3 4 4 5 5 4 4 4
13714 211/300 Read block 2720 len 8192
13715 212/300 Write block 4156 len 36864 data: 3 4 3 2 2 2 2 2 2
13716 213/300 Write block 3786 len 32768 data: 3 3 3 2 2 2 2 2
13717 214/300 Read block 3190 len 16384
13718 215/300 Write block 1333 len 36864 data: 5 5 4 3 3 3 3 3 3
13719 216/300 Flush
13720 217/300 Read block 1525 len 8192
13721 218/300 Read block 2213 len 40960
13722 219/300 Read block 1292 len 8192
13723 220/300 Write block 1287 len 20480 data: 7 7 9 9 6
13724 221/300 Read block 2896 len 16384
13725 222/300 Read block 3839 len 36864
13726 223/300 Read block 3822 len 24576
13727 224/300 Read block 1993 len 28672
13728 225/300 Write block 4252 len 12288 data: 3 3 4
13729 226/300 Write block 4588 len 20480 data: 4 4 4 4 4
13730 227/300 Write block 1332 len 8192 data: 5 6
13731 228/300 Read block 1011 len 32768
13732 229/300 Write block 657 len 20480 data: 4 3 3 3 3
13733 230/300 Read block 3739 len 16384
13734 231/300 Write block 903 len 20480 data: 5 5 6 6 5
13735 232/300 Read block 306 len 32768
13736 233/300 Read block 3542 len 20480
13737 234/300 Write block 664 len 36864 data: 3 3 4 7 7 8 8 8 7
13738 235/300 Read block 3332 len 8192
13739 236/300 Write block 359 len 28672 data: 4 4 4 3 3 4 4
13740 237/300 Write block 2008 len 24576 data: 3 3 3 3 2 4
13741 238/300 Read block 599 len 32768
13742 239/300 Write block 2201 len 24576 data: 4 4 4 5 3 4
13743 240/300 Read block 3882 len 24576
13744 241/300 Read block 4975 len 24576
13745 242/300 Read block 3190 len 36864
13746 243/300 Write block 3432 len 36864 data: 3 3 2 2 2 3 3 3 3
13747 244/300 Read block 4388 len 24576
13748 245/300 Write block 352 len 20480 data: 4 3 3 3 3
13749 246/300 Write block 684 len 12288 data: 4 4 3
13750 247/300 Read block 3082 len 28672
13751 248/300 Write block 1224 len 24576 data: 2 2 3 3 4 3
13752 249/300 Read block 4558 len 24576
13753 250/300 Read block 3589 len 12288
13754 251/300 Read block 465 len 40960
13755 252/300 Read block 1735 len 24576
13756 253/300 Read block 4036 len 20480
13757 254/300 Read block 253 len 8192
13758 255/300 Write block 1252 len 12288 data: 3 2 2
13759 256/300 Read block 4865 len 16384
13760 257/300 Write block 3133 len 16384 data: 3 4 4 4
13761 258/300 Read block 2171 len 36864
13762 259/300 Write block 16 len 12288 data: 2 2 2
13763 260/300 Flush
13764 261/300 Write block 2966 len 24576 data: 3 3 3 2 2 2
13765 262/300 Write block 2704 len 24576 data: 5 6 6 6 5 5
13766 263/300 Read block 2781 len 4096
13767 264/300 Flush
13768 265/300 Write block 3608 len 8192 data: 2 2
13769 266/300 Flush
13770 267/300 Write block 4158 len 32768 data: 4 3 3 3 3 3 3 2
13771 268/300 Write block 175 len 16384 data: 3 3 3 3
13772 269/300 Read block 4728 len 40960
13773 270/300 Read block 3230 len 16384
13774 271/300 Read block 4754 len 24576
13775 272/300 Write block 2620 len 24576 data: 5 5 5 2 2 2
13776 273/300 Read block 808 len 40960
13777 274/300 Read block 54 len 12288
13778 275/300 Write block 708 len 12288 data: 3 3 3
13779 276/300 Read block 4374 len 12288
13780 277/300 Read block 675 len 40960
13781 278/300 Read block 994 len 16384
13782 279/300 Write block 2068 len 32768 data: 5 5 6 5 5 4 4 4
13783 280/300 Write block 4145 len 36864 data: 2 2 2 2 3 3 3 3 3
13784 281/300 Flush
13785 282/300 Write block 3708 len 40960 data: 5 5 4 3 3 2 3 3 2 2
13786 283/300 Read block 2761 len 8192
13787 284/300 Read block 750 len 12288
13788 285/300 Read block 1324 len 40960
13789 286/300 Write block 3869 len 8192 data: 5 5
13790 287/300 Read block 842 len 28672
13791 288/300 Read block 351 len 36864
13792 289/300 Read block 542 len 4096
13793 290/300 Write block 3592 len 32768 data: 3 4 4 4 4 4 4 4
13794 291/300 Read block 4336 len 16384
13795 292/300 Read block 3151 len 12288
13796 293/300 Read block 3375 len 20480
13797 294/300 Read block 1209 len 32768
13798 295/300 Write block 4084 len 40960 data: 3 3 3 3 2 4 3 3 3 3
13799 296/300 Write block 3279 len 28672 data: 4 3 3 3 3 3 3
13800 297/300 Read block 4695 len 4096
13801 298/300 Read block 4265 len 32768
13802 299/300 Write block 1194 len 4096 data: 6
13803 300/300 Write block 3691 len 8192 data: 2 2
13804 301/300 Read block 4272 len 28672
13805 Main task has work to do, go find it
13806 got dsc Start(0)
13807 start 0
13808 Nov 19 17:10:54.666 INFO request completed, latency_us: 184, response_code: 204, uri: /start/cid/0, method: POST, req_id: ddecf0a3-c18a-4b42-be15-b5481d42edbf, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
13809 Replay: started 0, returned:Ok(())
13810 ----------------------------------------------------------------
13811 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13812 States: Offline Active Active
13813 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13814 4321 Acked Write 6 Sent Done Done false
13815 4322 Acked Read 4 Sent Done Done false
13816 4323 Acked Read 8 Sent Done Done false
13817 4324 Acked Read 1 Sent Done Done false
13818 4325 Acked Write 7 Sent Done Done false
13819 4326 Acked Flush 0 Sent Done Done false
13820 4327 Acked Read 9 Sent Done Done false
13821 4328 Acked Read 1 Sent Done Done false
13822 4329 Acked Read 3 Sent Done Done false
13823 4330 Acked Write 4 Sent Done Done false
13824 4331 Acked Read 2 Sent Done Done false
13825 4332 Acked Write 5 Sent Done Done false
13826 4333 Acked Write 4 Sent Done Done false
13827 4334 Acked Write 10 Sent Done Done false
13828 4335 Acked Write 4 Sent Done Done false
13829 4336 Acked Flush 0 Sent Done Done false
13830 4337 Acked Read 9 Sent Done Done false
13831 4338 Acked Read 1 Sent Done Done false
13832 4339 Acked Read 4 Sent Done Done false
13833 4340 Acked Write 5 Sent Done Done false
13834 4341 Acked Write 8 Sent Done Done false
13835 4342 Acked Write 5 Sent Done Done false
13836 4343 Acked Write 10 Sent Done Done false
13837 4344 Acked Flush 0 Sent Done Done false
13838 4345 Acked Read 1 Sent Done Done false
13839 4346 Acked Write 3 Sent Done Done false
13840 4347 Acked Write 6 Sent Done Done false
13841 4348 Acked Read 8 Sent Done Done false
13842 4349 Acked Read 7 Sent Done Done false
13843 4350 Acked Read 8 Sent Done Done false
13844 4351 Acked Read 6 Sent Done Done false
13845 4352 Acked Write 9 Sent Done Done false
13846 4353 Acked Read 6 Sent Done Done false
13847 4354 Acked Write 5 Sent Done Done false
13848 4355 Acked Write 9 Sent Done Done false
13849 4356 Acked Read 4 Sent Done Done false
13850 4357 Acked Read 3 Sent Done Done false
13851 4358 Acked Read 2 Sent Done Done false
13852 4359 Acked Write 7 Sent Done Done false
13853 4360 Acked Read 6 Sent Done Done false
13854 4361 Acked Read 4 Sent Done Done false
13855 4362 Acked Read 10 Sent Done Done false
13856 4363 Acked Flush 0 Sent Done Done false
13857 4364 Acked Read 5 Sent Done Done false
13858 4365 Acked Read 1 Sent Done Done false
13859 4366 Acked Read 7 Sent Done Done false
13860 4367 Acked Flush 0 Sent Done Done false
13861 4368 Acked Write 6 Sent Done Done false
13862 4369 Acked Write 10 Sent Done Done false
13863 4370 Acked Write 2 Sent Done Done false
13864 4371 Acked Read 1 Sent Done Done false
13865 4372 Acked Write 4 Sent Done Done false
13866 4373 Acked Write 8 Sent Done Done false
13867 4374 Acked Write 9 Sent Done Done false
13868 4375 Acked Read 6 Sent Done Done false
13869 4376 Acked Write 9 Sent Done Done false
13870 4377 Acked Write 5 Sent Done Done false
13871 4378 Acked Read 8 Sent Done Done false
13872 4379 Acked Write 8 Sent Done Done false
13873 4380 Acked Flush 0 Sent Done Done false
13874 4381 Acked Write 10 Sent Done Done false
13875 4382 Acked Write 3 Sent Done Done false
13876 4383 Acked Flush 0 Sent Done Done false
13877 4384 Acked Read 3 Sent Done Done false
13878 4385 Acked Flush 0 Sent Done Done false
13879 4386 Acked Write 9 Sent Done Done false
13880 4387 Acked Read 4 Sent Done Done false
13881 4388 Acked Write 8 Sent Done Done false
13882 4389 Acked Write 1 Sent Done Done false
13883 4390 Acked Write 1 Sent Done Done false
13884 4391 Acked Read 4 Sent Done Done false
13885 4392 Acked Flush 0 Sent Done Done false
13886 4393 Acked Write 10 Sent Done Done false
13887 4394 Acked Write 10 Sent Done Done false
13888 4395 Acked Read 2 Sent Done Done false
13889 4396 Acked Flush 0 Sent Done Done false
13890 4397 Acked Read 1 Sent Done Done false
13891 4398 Acked Read 2 Sent Done Done false
13892 4399 Acked Write 1 Sent Done Done false
13893 4400 Acked Read 2 Sent Done Done false
13894 4401 Acked Read 9 Sent Done Done false
13895 4402 Acked Read 9 Sent Done Done false
13896 4403 Acked Read 9 Sent Done Done false
13897 4404 Acked Read 1 Sent Done Done false
13898 4405 Acked Write 7 Sent Done Done false
13899 4406 Acked Write 3 Sent Done Done false
13900 4407 Acked Write 2 Sent Done Done false
13901 4408 Acked Write 9 Sent Done Done false
13902 4409 Acked Read 3 Sent Done Done false
13903 4410 Acked Write 5 Sent Done Done false
13904 4411 Acked Flush 0 Sent Done Done false
13905 4412 Acked Read 8 Sent Done Done false
13906 4413 Acked Read 8 Sent Done Done false
13907 4414 Acked Read 4 Sent Done Done false
13908 4415 Acked Read 1 Sent Done Done false
13909 4416 Acked Read 4 Sent Done Done false
13910 4417 Acked Read 5 Sent Done Done false
13911 4418 Acked Flush 0 Sent Done Done false
13912 4419 Acked Read 9 Sent Done Done false
13913 4420 Acked Flush 0 Sent Done Done false
13914 4421 Acked Flush 0 Sent Done Done false
13915 4422 Acked Write 5 Sent Done Done false
13916 4423 Acked Read 4 Sent Done Done false
13917 4424 Acked Read 6 Sent Done Done false
13918 4425 Acked Read 2 Sent Done Done false
13919 4426 Acked Read 10 Sent Done Done false
13920 4427 Acked Read 2 Sent Done Done false
13921 4428 Acked Flush 0 Sent Done Done false
13922 4429 Acked Read 3 Sent Done Done false
13923 4430 Acked Read 8 Sent Done Done false
13924 4431 Acked Read 10 Sent Done Done false
13925 4432 Acked Write 2 Sent Done Done false
13926 4433 Acked Read 1 Sent Done Done false
13927 4434 Acked Write 10 Sent Done Done false
13928 4435 Acked Write 6 Sent Done Done false
13929 4436 Acked Write 5 Sent Done Done false
13930 4437 Acked Read 5 Sent Done Done false
13931 4438 Acked Flush 0 Sent Done Done false
13932 4439 Acked Write 3 Sent Done Done false
13933 4440 Acked Read 5 Sent Done Done false
13934 4441 Acked Write 5 Sent Done Done false
13935 4442 Acked Write 8 Sent Done Done false
13936 4443 Acked Read 6 Sent Done Done false
13937 4444 Acked Write 10 Sent Done Done false
13938 4445 Acked Write 5 Sent Done Done false
13939 4446 Acked Write 2 Sent Done Done false
13940 4447 Acked Flush 0 Sent Done Done false
13941 4448 Acked Read 7 Sent Done Done false
13942 4449 Acked Flush 0 Sent Done Done false
13943 4450 Acked Write 3 Sent Done Done false
13944 4451 Acked Write 8 Sent Done Done false
13945 4452 Acked Write 3 Sent Done Done false
13946 4453 Acked Write 6 Sent Done Done false
13947 4454 Acked Write 8 Sent Done Done false
13948 4455 Acked Flush 0 Sent Done Done false
13949 4456 Acked Read 2 Sent Done Done false
13950 4457 Acked Write 8 Sent Done Done false
13951 4458 Acked Read 10 Sent Done Done false
13952 4459 Acked Write 1 Sent Done Done false
13953 4460 Acked Write 8 Sent Done Done false
13954 4461 Acked Read 1 Sent Done Done false
13955 4462 Acked Flush 0 Sent Done Done false
13956 4463 Acked Flush 0 Sent Done Done false
13957 4464 Acked Read 7 Sent Done Done false
13958 4465 Acked Write 6 Sent Done Done false
13959 4466 Acked Flush 0 Sent Done Done false
13960 4467 Acked Flush 0 Sent Done Done false
13961 4468 Acked Write 8 Sent Done Done false
13962 4469 Acked Read 7 Sent Done Done false
13963 4470 Acked Write 1 Sent Done Done false
13964 4471 Acked Read 8 Sent Done Done false
13965 4472 Acked Write 4 Sent Done Done false
13966 4473 Acked Read 10 Sent Done Done false
13967 4474 Acked Write 3 Sent Done Done false
13968 4475 Acked Write 3 Sent Done Done false
13969 4476 Acked Write 10 Sent Done Done false
13970 4477 Acked Write 4 Sent Done Done false
13971 4478 Acked Write 9 Sent Done Done false
13972 4479 Acked Read 10 Sent Done Done false
13973 4480 Acked Read 5 Sent Done Done false
13974 4481 Acked Write 5 Sent Done Done false
13975 4482 Acked Read 2 Sent Done Done false
13976 4483 Acked Read 10 Sent Done Done false
13977 4484 Acked Write 10 Sent Done Done false
13978 4485 Acked Read 5 Sent Done Done false
13979 4486 Acked Read 3 Sent Done Done false
13980 4487 Acked Write 8 Sent Done Done false
13981 4488 Acked Write 6 Sent Done Done false
13982 4489 Acked Read 8 Sent Done Done false
13983 4490 Acked Write 1 Sent Done Done false
13984 4491 Acked Write 8 Sent Done Done false
13985 4492 Acked Read 1 Sent Done Done false
13986 4493 Acked Write 2 Sent Done Done false
13987 4494 Acked Write 3 Sent Done Done false
13988 4495 Acked Read 5 Sent Done Done false
13989 4496 Acked Read 7 Sent Done Done false
13990 4497 Acked Flush 0 Sent Done Done false
13991 4498 Acked Read 8 Sent Done Done false
13992 4499 Acked Read 2 Sent Done Done false
13993 4500 Acked Read 2 Sent Done Done false
13994 4501 Acked Read 4 Sent Done Done false
13995 4502 Acked Read 10 Sent Done Done false
13996 4503 Acked Write 7 Sent Done Done false
13997 4504 Acked Read 9 Sent Done Done false
13998 4505 Acked Write 5 Sent Done Done false
13999 4506 Acked Read 2 Sent Done Done false
14000 4507 Acked Write 3 Sent Done Done false
14001 4508 Acked Read 7 Sent Done Done false
14002 4509 Acked Read 1 Sent Done Done false
14003 4510 Acked Read 4 Sent Done Done false
14004 4511 Acked Read 10 Sent Done Done false
14005 4512 Acked Read 10 Sent Done Done false
14006 4513 Acked Write 1 Sent Done Done false
14007 4514 Acked Flush 0 Sent Done Done false
14008 4515 Acked Write 3 Sent Done Done false
14009 4516 Acked Read 9 Sent Done Done false
14010 4517 Acked Write 2 Sent Done Done false
14011 4518 Acked Read 2 Sent Done Done false
14012 4519 Acked Read 5 Sent Done Done false
14013 4520 Acked Write 8 Sent Done Done false
14014 4521 Acked Read 1 Sent Done Done false
14015 4522 Acked Write 9 Sent Done Done false
14016 4523 Acked Write 6 Sent Done Done false
14017 4524 Acked Read 8 Sent Done Done false
14018 4525 Acked Write 2 Sent Done Done false
14019 4526 Acked Write 6 Sent Done Done false
14020 4527 Acked Write 4 Sent Done Done false
14021 4528 Acked Read 10 Sent Done Done false
14022 4529 Acked Flush 0 Sent Done Done false
14023 4530 Acked Write 8 Sent Done Done false
14024 4531 Acked Read 2 Sent Done Done false
14025 4532 Acked Write 9 Sent Done Done false
14026 4533 Acked Write 8 Sent Done Done false
14027 4534 Acked Read 4 Sent Done Done false
14028 4535 Acked Write 9 Sent Done Done false
14029 4536 Acked Flush 0 Sent Done Done false
14030 4537 Acked Read 2 Sent Done Done false
14031 4538 Acked Read 10 Sent Done Done false
14032 4539 Acked Read 2 Sent Done Done false
14033 4540 Acked Write 5 Sent Done Done false
14034 4541 Acked Read 4 Sent Done Done false
14035 4542 Acked Read 9 Sent Done Done false
14036 4543 Acked Read 6 Sent Done Done false
14037 4544 Acked Read 7 Sent Done Done false
14038 4545 Acked Write 3 Sent Done Done false
14039 4546 Acked Write 5 Sent Done Done false
14040 4547 Acked Write 2 Sent Done Done false
14041 4548 Acked Read 8 Sent Done Done false
14042 4549 Acked Write 5 Sent Done Done false
14043 4550 Acked Read 4 Sent Done Done false
14044 4551 Acked Write 5 Sent Done Done false
14045 4552 Acked Read 8 Sent Done Done false
14046 4553 Acked Read 5 Sent Done Done false
14047 4554 Acked Write 9 Sent Done Done false
14048 4555 Acked Read 2 Sent Done Done false
14049 4556 Acked Write 7 Sent Done Done false
14050 4557 Acked Write 6 Sent Done Done false
14051 4558 Acked Read 8 Sent Done Done false
14052 4559 Acked Write 6 Sent Done Done false
14053 4560 Acked Read 6 Sent Done Done false
14054 4561 Acked Read 6 Sent Done Done false
14055 4562 Acked Read 9 Sent Done Done false
14056 4563 Acked Write 9 Sent Done Done false
14057 4564 Acked Read 6 Sent Done Done false
14058 4565 Acked Write 5 Sent Done Done false
14059 4566 Acked Write 3 Sent Done Done false
14060 4567 Acked Read 7 Sent Done Done false
14061 4568 Acked Write 6 Sent Done Done false
14062 4569 Acked Read 6 Sent Done Done false
14063 4570 Acked Read 3 Sent Done Done false
14064 4571 Acked Read 10 Sent Done Done false
14065 4572 Acked Read 6 Sent Done Done false
14066 4573 Acked Read 5 Sent Done Done false
14067 4574 Acked Read 2 Sent Done Done false
14068 4575 Acked Write 3 Sent Done Done false
14069 4576 Acked Read 4 Sent Done Done false
14070 4577 Acked Write 4 Sent Done Done false
14071 4578 Acked Read 9 Sent Done Done false
14072 4579 Acked Write 3 Sent Done Done false
14073 4580 Acked Flush 0 Sent Done Done false
14074 4581 Acked Write 6 Sent Done Done false
14075 4582 Acked Write 6 Sent Done Done false
14076 4583 Acked Read 1 Sent Done Done false
14077 4584 Acked Flush 0 Sent Done Done false
14078 4585 Acked Write 2 Sent Done Done false
14079 4586 Acked Flush 0 Sent Done Done false
14080 4587 Acked Write 8 Sent Done Done false
14081 4588 Acked Write 4 Sent Done Done false
14082 4589 Acked Read 10 Sent Done Done false
14083 4590 Acked Read 4 Sent Done Done false
14084 4591 Acked Read 6 Sent Done Done false
14085 4592 Acked Write 6 Sent Done Done false
14086 4593 Acked Read 10 Sent Done Done false
14087 4594 Acked Read 3 Sent Done Done false
14088 4595 Acked Write 3 Sent Done Done false
14089 4596 Acked Read 3 Sent Done Done false
14090 4597 Acked Read 10 Sent Done Done false
14091 4598 Acked Read 4 Sent Done Done false
14092 4599 Acked Write 8 Sent Done Done false
14093 4600 Acked Write 9 Sent Done Done false
14094 4601 Acked Flush 0 Sent Done Done false
14095 4602 Acked Write 10 Sent Done Done false
14096 4603 Acked Read 2 Sent Done Done false
14097 4604 Acked Read 3 Sent Done Done false
14098 4605 Acked Read 10 Sent Done Done false
14099 4606 Acked Write 2 Sent Done Done false
14100 4607 Acked Read 7 Sent Done Done false
14101 4608 Acked Read 9 Sent Done Done false
14102 4609 Acked Read 1 Sent Done Done false
14103 4610 Acked Write 8 Sent Done Done false
14104 4611 Acked Read 4 Sent Done Done false
14105 4612 Acked Read 3 Sent Done Done false
14106 4613 Acked Read 5 Sent Done Done false
14107 4614 Acked Read 8 Sent Done Done false
14108 4615 Acked Write 10 Sent Done Done false
14109 4616 Acked Write 7 Sent Done Done false
14110 4617 Acked Read 1 Sent Done Done false
14111 4618 Acked Read 8 Sent Done Done false
14112 4619 Acked Write 1 Sent Done Done false
14113 4620 Acked Write 2 Sent Done Done false
14114 4621 Acked Read 7 Sent Done Done false
14115 STATES DS:0 DS:1 DS:2 TOTAL
14116 Sent 301 0 0 301
14117 Done 0 301 301 602
14118 Skipped 0 0 0 0
14119 Error 0 0 0 0
14120 Last Flush: 4320 4601 4601
14121 Downstairs last five completed: 4320 4319 4318 4317 4316
14122 Upstairs last five completed: 4621 4620 4619 4618 4617
14123 CLIENT: Up:0 ds:301 act:2
14124 [8810] got start action
14125 [8810] I am going to restart
14126 Starting downstairs at port 8810
14127 Make output file at "/tmp/dsc/downstairs-8810.txt"
14128 [8810][0] reports Starting
14129 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1214)
14130 [8810][0] reports Running
14131 ----------------------------------------------------------------
14132 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14133 States: Offline Active Active
14134 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14135 4321 Acked Write 6 Sent Done Done false
14136 4322 Acked Read 4 Sent Done Done false
14137 4323 Acked Read 8 Sent Done Done false
14138 4324 Acked Read 1 Sent Done Done false
14139 4325 Acked Write 7 Sent Done Done false
14140 4326 Acked Flush 0 Sent Done Done false
14141 4327 Acked Read 9 Sent Done Done false
14142 4328 Acked Read 1 Sent Done Done false
14143 4329 Acked Read 3 Sent Done Done false
14144 4330 Acked Write 4 Sent Done Done false
14145 4331 Acked Read 2 Sent Done Done false
14146 4332 Acked Write 5 Sent Done Done false
14147 4333 Acked Write 4 Sent Done Done false
14148 4334 Acked Write 10 Sent Done Done false
14149 4335 Acked Write 4 Sent Done Done false
14150 4336 Acked Flush 0 Sent Done Done false
14151 4337 Acked Read 9 Sent Done Done false
14152 4338 Acked Read 1 Sent Done Done false
14153 4339 Acked Read 4 Sent Done Done false
14154 4340 Acked Write 5 Sent Done Done false
14155 4341 Acked Write 8 Sent Done Done false
14156 4342 Acked Write 5 Sent Done Done false
14157 4343 Acked Write 10 Sent Done Done false
14158 4344 Acked Flush 0 Sent Done Done false
14159 4345 Acked Read 1 Sent Done Done false
14160 4346 Acked Write 3 Sent Done Done false
14161 4347 Acked Write 6 Sent Done Done false
14162 4348 Acked Read 8 Sent Done Done false
14163 4349 Acked Read 7 Sent Done Done false
14164 4350 Acked Read 8 Sent Done Done false
14165 4351 Acked Read 6 Sent Done Done false
14166 4352 Acked Write 9 Sent Done Done false
14167 4353 Acked Read 6 Sent Done Done false
14168 4354 Acked Write 5 Sent Done Done false
14169 4355 Acked Write 9 Sent Done Done false
14170 4356 Acked Read 4 Sent Done Done false
14171 4357 Acked Read 3 Sent Done Done false
14172 4358 Acked Read 2 Sent Done Done false
14173 4359 Acked Write 7 Sent Done Done false
14174 4360 Acked Read 6 Sent Done Done false
14175 4361 Acked Read 4 Sent Done Done false
14176 4362 Acked Read 10 Sent Done Done false
14177 4363 Acked Flush 0 Sent Done Done false
14178 4364 Acked Read 5 Sent Done Done false
14179 4365 Acked Read 1 Sent Done Done false
14180 4366 Acked Read 7 Sent Done Done false
14181 4367 Acked Flush 0 Sent Done Done false
14182 4368 Acked Write 6 Sent Done Done false
14183 4369 Acked Write 10 Sent Done Done false
14184 4370 Acked Write 2 Sent Done Done false
14185 4371 Acked Read 1 Sent Done Done false
14186 4372 Acked Write 4 Sent Done Done false
14187 4373 Acked Write 8 Sent Done Done false
14188 4374 Acked Write 9 Sent Done Done false
14189 4375 Acked Read 6 Sent Done Done false
14190 4376 Acked Write 9 Sent Done Done false
14191 4377 Acked Write 5 Sent Done Done false
14192 4378 Acked Read 8 Sent Done Done false
14193 4379 Acked Write 8 Sent Done Done false
14194 4380 Acked Flush 0 Sent Done Done false
14195 4381 Acked Write 10 Sent Done Done false
14196 4382 Acked Write 3 Sent Done Done false
14197 4383 Acked Flush 0 Sent Done Done false
14198 4384 Acked Read 3 Sent Done Done false
14199 4385 Acked Flush 0 Sent Done Done false
14200 4386 Acked Write 9 Sent Done Done false
14201 4387 Acked Read 4 Sent Done Done false
14202 4388 Acked Write 8 Sent Done Done false
14203 4389 Acked Write 1 Sent Done Done false
14204 4390 Acked Write 1 Sent Done Done false
14205 4391 Acked Read 4 Sent Done Done false
14206 4392 Acked Flush 0 Sent Done Done false
14207 4393 Acked Write 10 Sent Done Done false
14208 4394 Acked Write 10 Sent Done Done false
14209 4395 Acked Read 2 Sent Done Done false
14210 4396 Acked Flush 0 Sent Done Done false
14211 4397 Acked Read 1 Sent Done Done false
14212 4398 Acked Read 2 Sent Done Done false
14213 4399 Acked Write 1 Sent Done Done false
14214 4400 Acked Read 2 Sent Done Done false
14215 4401 Acked Read 9 Sent Done Done false
14216 4402 Acked Read 9 Sent Done Done false
14217 4403 Acked Read 9 Sent Done Done false
14218 4404 Acked Read 1 Sent Done Done false
14219 4405 Acked Write 7 Sent Done Done false
14220 4406 Acked Write 3 Sent Done Done false
14221 4407 Acked Write 2 Sent Done Done false
14222 4408 Acked Write 9 Sent Done Done false
14223 4409 Acked Read 3 Sent Done Done false
14224 4410 Acked Write 5 Sent Done Done false
14225 4411 Acked Flush 0 Sent Done Done false
14226 4412 Acked Read 8 Sent Done Done false
14227 4413 Acked Read 8 Sent Done Done false
14228 4414 Acked Read 4 Sent Done Done false
14229 4415 Acked Read 1 Sent Done Done false
14230 4416 Acked Read 4 Sent Done Done false
14231 4417 Acked Read 5 Sent Done Done false
14232 4418 Acked Flush 0 Sent Done Done false
14233 4419 Acked Read 9 Sent Done Done false
14234 4420 Acked Flush 0 Sent Done Done false
14235 4421 Acked Flush 0 Sent Done Done false
14236 4422 Acked Write 5 Sent Done Done false
14237 4423 Acked Read 4 Sent Done Done false
14238 4424 Acked Read 6 Sent Done Done false
14239 4425 Acked Read 2 Sent Done Done false
14240 4426 Acked Read 10 Sent Done Done false
14241 4427 Acked Read 2 Sent Done Done false
14242 4428 Acked Flush 0 Sent Done Done false
14243 4429 Acked Read 3 Sent Done Done false
14244 4430 Acked Read 8 Sent Done Done false
14245 4431 Acked Read 10 Sent Done Done false
14246 4432 Acked Write 2 Sent Done Done false
14247 4433 Acked Read 1 Sent Done Done false
14248 4434 Acked Write 10 Sent Done Done false
14249 4435 Acked Write 6 Sent Done Done false
14250 4436 Acked Write 5 Sent Done Done false
14251 4437 Acked Read 5 Sent Done Done false
14252 4438 Acked Flush 0 Sent Done Done false
14253 4439 Acked Write 3 Sent Done Done false
14254 4440 Acked Read 5 Sent Done Done false
14255 4441 Acked Write 5 Sent Done Done false
14256 4442 Acked Write 8 Sent Done Done false
14257 4443 Acked Read 6 Sent Done Done false
14258 4444 Acked Write 10 Sent Done Done false
14259 4445 Acked Write 5 Sent Done Done false
14260 4446 Acked Write 2 Sent Done Done false
14261 4447 Acked Flush 0 Sent Done Done false
14262 4448 Acked Read 7 Sent Done Done false
14263 4449 Acked Flush 0 Sent Done Done false
14264 4450 Acked Write 3 Sent Done Done false
14265 4451 Acked Write 8 Sent Done Done false
14266 4452 Acked Write 3 Sent Done Done false
14267 4453 Acked Write 6 Sent Done Done false
14268 4454 Acked Write 8 Sent Done Done false
14269 4455 Acked Flush 0 Sent Done Done false
14270 4456 Acked Read 2 Sent Done Done false
14271 4457 Acked Write 8 Sent Done Done false
14272 4458 Acked Read 10 Sent Done Done false
14273 4459 Acked Write 1 Sent Done Done false
14274 4460 Acked Write 8 Sent Done Done false
14275 4461 Acked Read 1 Sent Done Done false
14276 4462 Acked Flush 0 Sent Done Done false
14277 4463 Acked Flush 0 Sent Done Done false
14278 4464 Acked Read 7 Sent Done Done false
14279 4465 Acked Write 6 Sent Done Done false
14280 4466 Acked Flush 0 Sent Done Done false
14281 4467 Acked Flush 0 Sent Done Done false
14282 4468 Acked Write 8 Sent Done Done false
14283 4469 Acked Read 7 Sent Done Done false
14284 4470 Acked Write 1 Sent Done Done false
14285 4471 Acked Read 8 Sent Done Done false
14286 4472 Acked Write 4 Sent Done Done false
14287 4473 Acked Read 10 Sent Done Done false
14288 4474 Acked Write 3 Sent Done Done false
14289 4475 Acked Write 3 Sent Done Done false
14290 4476 Acked Write 10 Sent Done Done false
14291 4477 Acked Write 4 Sent Done Done false
14292 4478 Acked Write 9 Sent Done Done false
14293 4479 Acked Read 10 Sent Done Done false
14294 4480 Acked Read 5 Sent Done Done false
14295 4481 Acked Write 5 Sent Done Done false
14296 4482 Acked Read 2 Sent Done Done false
14297 4483 Acked Read 10 Sent Done Done false
14298 4484 Acked Write 10 Sent Done Done false
14299 4485 Acked Read 5 Sent Done Done false
14300 4486 Acked Read 3 Sent Done Done false
14301 4487 Acked Write 8 Sent Done Done false
14302 4488 Acked Write 6 Sent Done Done false
14303 4489 Acked Read 8 Sent Done Done false
14304 4490 Acked Write 1 Sent Done Done false
14305 4491 Acked Write 8 Sent Done Done false
14306 4492 Acked Read 1 Sent Done Done false
14307 4493 Acked Write 2 Sent Done Done false
14308 4494 Acked Write 3 Sent Done Done false
14309 4495 Acked Read 5 Sent Done Done false
14310 4496 Acked Read 7 Sent Done Done false
14311 4497 Acked Flush 0 Sent Done Done false
14312 4498 Acked Read 8 Sent Done Done false
14313 4499 Acked Read 2 Sent Done Done false
14314 4500 Acked Read 2 Sent Done Done false
14315 4501 Acked Read 4 Sent Done Done false
14316 4502 Acked Read 10 Sent Done Done false
14317 4503 Acked Write 7 Sent Done Done false
14318 4504 Acked Read 9 Sent Done Done false
14319 4505 Acked Write 5 Sent Done Done false
14320 4506 Acked Read 2 Sent Done Done false
14321 4507 Acked Write 3 Sent Done Done false
14322 4508 Acked Read 7 Sent Done Done false
14323 4509 Acked Read 1 Sent Done Done false
14324 4510 Acked Read 4 Sent Done Done false
14325 4511 Acked Read 10 Sent Done Done false
14326 4512 Acked Read 10 Sent Done Done false
14327 4513 Acked Write 1 Sent Done Done false
14328 4514 Acked Flush 0 Sent Done Done false
14329 4515 Acked Write 3 Sent Done Done false
14330 4516 Acked Read 9 Sent Done Done false
14331 4517 Acked Write 2 Sent Done Done false
14332 4518 Acked Read 2 Sent Done Done false
14333 4519 Acked Read 5 Sent Done Done false
14334 4520 Acked Write 8 Sent Done Done false
14335 4521 Acked Read 1 Sent Done Done false
14336 4522 Acked Write 9 Sent Done Done false
14337 4523 Acked Write 6 Sent Done Done false
14338 4524 Acked Read 8 Sent Done Done false
14339 4525 Acked Write 2 Sent Done Done false
14340 4526 Acked Write 6 Sent Done Done false
14341 4527 Acked Write 4 Sent Done Done false
14342 4528 Acked Read 10 Sent Done Done false
14343 4529 Acked Flush 0 Sent Done Done false
14344 4530 Acked Write 8 Sent Done Done false
14345 4531 Acked Read 2 Sent Done Done false
14346 4532 Acked Write 9 Sent Done Done false
14347 4533 Acked Write 8 Sent Done Done false
14348 4534 Acked Read 4 Sent Done Done false
14349 4535 Acked Write 9 Sent Done Done false
14350 4536 Acked Flush 0 Sent Done Done false
14351 4537 Acked Read 2 Sent Done Done false
14352 4538 Acked Read 10 Sent Done Done false
14353 4539 Acked Read 2 Sent Done Done false
14354 4540 Acked Write 5 Sent Done Done false
14355 4541 Acked Read 4 Sent Done Done false
14356 4542 Acked Read 9 Sent Done Done false
14357 4543 Acked Read 6 Sent Done Done false
14358 4544 Acked Read 7 Sent Done Done false
14359 4545 Acked Write 3 Sent Done Done false
14360 4546 Acked Write 5 Sent Done Done false
14361 4547 Acked Write 2 Sent Done Done false
14362 4548 Acked Read 8 Sent Done Done false
14363 4549 Acked Write 5 Sent Done Done false
14364 4550 Acked Read 4 Sent Done Done false
14365 4551 Acked Write 5 Sent Done Done false
14366 4552 Acked Read 8 Sent Done Done false
14367 4553 Acked Read 5 Sent Done Done false
14368 4554 Acked Write 9 Sent Done Done false
14369 4555 Acked Read 2 Sent Done Done false
14370 4556 Acked Write 7 Sent Done Done false
14371 4557 Acked Write 6 Sent Done Done false
14372 4558 Acked Read 8 Sent Done Done false
14373 4559 Acked Write 6 Sent Done Done false
14374 4560 Acked Read 6 Sent Done Done false
14375 4561 Acked Read 6 Sent Done Done false
14376 4562 Acked Read 9 Sent Done Done false
14377 4563 Acked Write 9 Sent Done Done false
14378 4564 Acked Read 6 Sent Done Done false
14379 4565 Acked Write 5 Sent Done Done false
14380 4566 Acked Write 3 Sent Done Done false
14381 4567 Acked Read 7 Sent Done Done false
14382 4568 Acked Write 6 Sent Done Done false
14383 4569 Acked Read 6 Sent Done Done false
14384 4570 Acked Read 3 Sent Done Done false
14385 4571 Acked Read 10 Sent Done Done false
14386 4572 Acked Read 6 Sent Done Done false
14387 4573 Acked Read 5 Sent Done Done false
14388 4574 Acked Read 2 Sent Done Done false
14389 4575 Acked Write 3 Sent Done Done false
14390 4576 Acked Read 4 Sent Done Done false
14391 4577 Acked Write 4 Sent Done Done false
14392 4578 Acked Read 9 Sent Done Done false
14393 4579 Acked Write 3 Sent Done Done false
14394 4580 Acked Flush 0 Sent Done Done false
14395 4581 Acked Write 6 Sent Done Done false
14396 4582 Acked Write 6 Sent Done Done false
14397 4583 Acked Read 1 Sent Done Done false
14398 4584 Acked Flush 0 Sent Done Done false
14399 4585 Acked Write 2 Sent Done Done false
14400 4586 Acked Flush 0 Sent Done Done false
14401 4587 Acked Write 8 Sent Done Done false
14402 4588 Acked Write 4 Sent Done Done false
14403 4589 Acked Read 10 Sent Done Done false
14404 4590 Acked Read 4 Sent Done Done false
14405 4591 Acked Read 6 Sent Done Done false
14406 4592 Acked Write 6 Sent Done Done false
14407 4593 Acked Read 10 Sent Done Done false
14408 4594 Acked Read 3 Sent Done Done false
14409 4595 Acked Write 3 Sent Done Done false
14410 4596 Acked Read 3 Sent Done Done false
14411 4597 Acked Read 10 Sent Done Done false
14412 4598 Acked Read 4 Sent Done Done false
14413 4599 Acked Write 8 Sent Done Done false
14414 4600 Acked Write 9 Sent Done Done false
14415 4601 Acked Flush 0 Sent Done Done false
14416 4602 Acked Write 10 Sent Done Done false
14417 4603 Acked Read 2 Sent Done Done false
14418 4604 Acked Read 3 Sent Done Done false
14419 4605 Acked Read 10 Sent Done Done false
14420 4606 Acked Write 2 Sent Done Done false
14421 4607 Acked Read 7 Sent Done Done false
14422 4608 Acked Read 9 Sent Done Done false
14423 4609 Acked Read 1 Sent Done Done false
14424 4610 Acked Write 8 Sent Done Done false
14425 4611 Acked Read 4 Sent Done Done false
14426 4612 Acked Read 3 Sent Done Done false
14427 4613 Acked Read 5 Sent Done Done false
14428 4614 Acked Read 8 Sent Done Done false
14429 4615 Acked Write 10 Sent Done Done false
14430 4616 Acked Write 7 Sent Done Done false
14431 4617 Acked Read 1 Sent Done Done false
14432 4618 Acked Read 8 Sent Done Done false
14433 4619 Acked Write 1 Sent Done Done false
14434 4620 Acked Write 2 Sent Done Done false
14435 4621 Acked Read 7 Sent Done Done false
14436 4622 Acked Flush 0 Sent Done Done false
14437 STATES DS:0 DS:1 DS:2 TOTAL
14438 Sent 302 0 0 302
14439 Done 0 302 302 604
14440 Skipped 0 0 0 0
14441 Error 0 0 0 0
14442 Last Flush: 4320 4622 4622
14443 Downstairs last five completed: 4320 4319 4318 4317 4316
14444 Upstairs last five completed: 4622 4621 4620 4619 4618
14445 CLIENT: Up:0 ds:302 act:2
144462024-11-19T17:11:00.434ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144472024-11-19T17:11:00.434ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53219) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144482024-11-19T17:11:00.435ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144492024-11-19T17:11:00.435ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144502024-11-19T17:11:00.435ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144512024-11-19T17:11:00.435ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144522024-11-19T17:11:00.436ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144532024-11-19T17:11:00.436ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144542024-11-19T17:11:00.436ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144552024-11-19T17:11:00.436ZINFOcrucible: [0] client re-new 302 jobs since flush 4320 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
144562024-11-19T17:11:00.436ZINFOcrucible: [0] Replayed 302 jobs since flush: 4320 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
14457 ----------------------------------------------------------------
14458 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14459 Downstairs last five completed: 4622 4621 4620 4619 4618
14460 Upstairs last five completed: 4622 4621 4620 4619 4618
14461 CLIENT: Up:0 ds:0 act:3
14462 Replay: All jobs finished
14463 Main task has work to do, go find it
14464 got dsc Stop(1)
14465 stop 1
14466 [8820] Got stop action so:false kr:false
14467 Nov 19 17:11:02.672 INFO request completed, latency_us: 345, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 2b83e881-5018-4579-92c0-7dc1383758b5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
144682024-11-19T17:11:02.673ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144692024-11-19T17:11:02.673ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
144702024-11-19T17:11:02.673ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
144712024-11-19T17:11:02.673ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
14472 Nov 19 17:11:02.673 INFO request completed, latency_us: 133, response_code: 200, uri:{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:11:02.673429851Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
14473 /state/cid/1, method: GET, req_id{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:11:02.673483336Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
14474 : 87399df9-7dc3-48b8-a717-7625cea1f1f5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
14475 [8820] Exited with: ExitStatus(unix_wait_status(9))
14476 [8820][1] reports Exit
14477 Nov 19 17:11:06.676 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4ba62571-da24-46a4-bfd0-854b27944c0f, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
14478 001/300 Write block 4006 len 12288 data: 2 2 2
14479 002/300 Read block 4426 len 40960
14480 003/300 Read block 2454 len 32768
14481 004/300 Flush
14482 005/300 Read block 4141 len 4096
14483 006/300 Write block 4483 len 40960 data: 2 3 3 4 5 4 4 4 6 6
14484 007/300 Read block 522 len 36864
14485 008/300 Write block 3890 len 16384 data: 3 3 4 5
14486 009/300 Write block 28 len 36864 data: 3 2 3 3 3 3 3 3 3
14487 010/300 Write block 3616 len 28672 data: 3 4 4 4 3 4 4
14488 011/300 Read block 56 len 40960
14489 012/300 Read block 859 len 16384
14490 013/300 Flush
14491 014/300 Flush
14492 015/300 Write block 779 len 32768 data: 4 5 6 6 6 6 5 6
14493 016/300 Read block 3125 len 40960
14494 017/300 Read block 3436 len 16384
14495 018/300 Read block 2447 len 16384
14496 019/300 Read block 3467 len 32768
14497 020/300 Read block 212 len 28672
14498 021/300 Read block 1492 len 12288
14499 022/300 Read block 4493 len 4096
14500 023/300 Flush
14501 024/300 Read block 1613 len 12288
14502 025/300 Read block 4929 len 28672
14503 026/300 Write block 2892 len 36864 data: 4 4 3 3 4 3 3 3 3
14504 027/300 Read block 1663 len 4096
14505 028/300 Flush
14506 029/300 Flush
14507 030/300 Write block 3630 len 36864 data: 2 2 2 2 2 3 3 3 3
14508 031/300 Write block 633 len 24576 data: 2 2 2 2 2 2
14509 032/300 Read block 4056 len 16384
14510 033/300 Read block 1478 len 36864
14511 034/300 Read block 1762 len 36864
14512 035/300 Flush
14513 036/300 Write block 3786 len 28672 data: 4 4 4 3 3 3 3
14514 037/300 Read block 2807 len 20480
14515 038/300 Read block 4672 len 4096
14516 039/300 Write block 2263 len 36864 data: 3 3 3 2 2 2 2 2 3
14517 040/300 Write block 870 len 8192 data: 4 4
14518 041/300 Read block 2931 len 36864
14519 042/300 Read block 4754 len 40960
14520 043/300 Read block 2252 len 36864
14521 044/300 Write block 1144 len 36864 data: 5 5 4 4 4 4 3 4 3
14522 045/300 Read block 1992 len 32768
14523 046/300 Read block 4835 len 24576
14524 047/300 Read block 3531 len 32768
14525 048/300 Write block 3969 len 16384 data: 4 3 3 2
14526 049/300 Read block 720 len 32768
14527 050/300 Write block 361 len 8192 data: 5 4
14528 051/300 Write block 4629 len 24576 data: 2 2 2 3 3 5
14529 052/300 Read block 2443 len 40960
14530 053/300 Write block 136 len 20480 data: 6 6 4 6 5
14531 054/300 Write block 4671 len 36864 data: 4 4 4 4 4 4 3 3 2
14532 055/300 Read block 4156 len 16384
14533 056/300 Read block 2618 len 40960
14534 057/300 Read block 2175 len 28672
14535 058/300 Read block 3777 len 8192
14536 059/300 Read block 3233 len 16384
14537 060/300 Read block 3948 len 28672
14538 061/300 Write block 3522 len 4096 data: 2
14539 062/300 Read block 4666 len 40960
14540 063/300 Write block 3439 len 32768 data: 4 4 3 3 3 3 3 2
14541 064/300 Write block 4778 len 8192 data: 6 5
14542 065/300 Read block 2437 len 36864
14543 066/300 Write block 1260 len 8192 data: 4 4
14544 067/300 Read block 2759 len 4096
14545 068/300 Read block 722 len 4096
14546 069/300 Read block 3793 len 20480
14547 070/300 Flush
14548 071/300 Read block 1779 len 40960
14549 072/300 Read block 3472 len 4096
14550 073/300 Read block 2200 len 16384
14551 074/300 Read block 4944 len 28672
14552 075/300 Write block 2271 len 32768 data: 4 4 4 5 4 5 4 4
14553 076/300 Write block 1713 len 28672 data: 4 2 2 2 3 3 3
14554 077/300 Write block 701 len 4096 data: 3
14555 078/300 Write block 3123 len 16384 data: 4 5 5 7
14556 079/300 Flush
14557 080/300 Write block 2633 len 20480 data: 4 3 3 3 3
14558 081/300 Write block 3562 len 4096 data: 4
14559 082/300 Read block 2551 len 32768
14560 083/300 Write block 854 len 16384 data: 2 2 3 3
14561 084/300 Read block 2178 len 20480
14562 085/300 Write block 2831 len 40960 data: 4 4 5 5 4 2 2 2 3 3
14563 086/300 Write block 216 len 8192 data: 4 4
14564 087/300 Write block 1144 len 16384 data: 6 6 5 5
14565 088/300 Read block 647 len 4096
14566 089/300 Read block 1555 len 12288
14567 090/300 Read block 2810 len 8192
14568 091/300 Read block 2094 len 32768
14569 092/300 Write block 3843 len 32768 data: 3 3 2 2 2 2 2 2
14570 093/300 Write block 1731 len 8192 data: 3 3
14571 094/300 Read block 2308 len 40960
14572 095/300 Read block 244 len 20480
14573 096/300 Write block 2502 len 16384 data: 2 2 3 4
14574 097/300 Read block 4850 len 40960
14575 098/300 Write block 4079 len 36864 data: 2 3 3 4 4 4 4 4 4
14576 099/300 Read block 3419 len 24576
14577 100/300 Read block 4233 len 24576
14578 101/300 Write block 662 len 12288 data: 4 3 4
14579 102/300 Write block 913 len 32768 data: 2 2 2 3 3 3 3 3
14580 103/300 Read block 4030 len 24576
14581 104/300 Write block 3077 len 20480 data: 2 2 2 3 3
14582 105/300 Read block 816 len 20480
14583 106/300 Write block 1320 len 28672 data: 6 5 4 4 3 3 3
14584 107/300 Flush
14585 108/300 Read block 3598 len 4096
14586 109/300 Read block 2641 len 40960
14587 110/300 Write block 641 len 4096 data: 2
14588 111/300 Read block 2459 len 12288
14589 112/300 Write block 1153 len 12288 data: 3 3 3
14590 113/300 Write block 3150 len 12288 data: 4 5 5
14591 114/300 Write block 133 len 28672 data: 6 6 6 7 7 5 7
14592 115/300 Read block 3558 len 8192
14593 116/300 Read block 2796 len 24576
14594 117/300 Write block 4020 len 40960 data: 5 6 6 6 6 5 5 7 7 5
14595 118/300 Read block 2057 len 24576
14596 119/300 Read block 4258 len 8192
14597 120/300 Read block 3726 len 36864
14598 121/300 Read block 2199 len 12288
14599 122/300 Write block 278 len 20480 data: 5 5 4 4 3
14600 123/300 Read block 213 len 36864
14601 124/300 Read block 2032 len 36864
14602 125/300 Write block 3731 len 20480 data: 3 3 3 3 3
14603 126/300 Read block 663 len 28672
14604 127/300 Read block 1343 len 24576
14605 128/300 Read block 299 len 24576
14606 129/300 Flush
14607 130/300 Write block 3413 len 8192 data: 3 2
14608 131/300 Read block 1949 len 4096
14609 132/300 Write block 3202 len 24576 data: 5 6 6 5 5 5
14610 133/300 Read block 579 len 16384
14611 134/300 Write block 2563 len 28672 data: 2 2 2 2 3 3 3
14612 135/300 Write block 823 len 20480 data: 3 4 4 3 3
14613 136/300 Flush
14614 137/300 Read block 3466 len 16384
14615 138/300 Write block 4790 len 24576 data: 2 2 2 3 3 4
14616 139/300 Read block 1891 len 4096
14617 140/300 Write block 1339 len 28672 data: 4 4 4 3 3 3 4
14618 141/300 Read block 1839 len 24576
14619 142/300 Read block 4423 len 32768
14620 143/300 Read block 4930 len 8192
14621 144/300 Flush
14622 145/300 Write block 1135 len 36864 data: 2 3 3 3 4 3 3 3 4
14623 146/300 Read block 3184 len 16384
14624 147/300 Write block 4953 len 36864 data: 2 2 2 2 2 2 2 2 3
14625 148/300 Read block 3460 len 40960
14626 149/300 Write block 3589 len 36864 data: 3 3 3 4 5 5 5 5 5
14627 150/300 Write block 4839 len 4096 data: 2
14628 151/300 Write block 2296 len 28672 data: 4 4 5 5 6 8 7
14629 152/300 Read block 3921 len 8192
14630 153/300 Write block 1201 len 8192 data: 4 5
14631 154/300 Read block 4899 len 20480
14632 155/300 Read block 330 len 28672
14633 156/300 Write block 3257 len 8192 data: 2 2
14634 157/300 Flush
14635 158/300 Flush
14636 159/300 Write block 2930 len 40960 data: 6 6 5 5 5 4 5 5 5 5
14637 160/300 Read block 1664 len 4096
14638 161/300 Flush
14639 162/300 Write block 2052 len 20480 data: 3 3 3 3 3
14640 163/300 Read block 4687 len 40960
14641 164/300 Flush
14642 165/300 Read block 3712 len 36864
14643 166/300 Read block 3153 len 20480
14644 167/300 Read block 1801 len 16384
14645 168/300 Write block 4204 len 20480 data: 2 3 3 3 3
14646 169/300 Read block 1144 len 4096
14647 170/300 Read block 3524 len 40960
14648 171/300 Write block 1303 len 28672 data: 2 2 2 2 2 2 2
14649 172/300 Read block 1193 len 32768
14650 173/300 Write block 3531 len 20480 data: 5 4 3 3 4
14651 174/300 Write block 358 len 4096 data: 3
14652 175/300 Read block 3282 len 8192
14653 176/300 Write block 1822 len 12288 data: 5 5 4
14654 177/300 Read block 3385 len 12288
14655 178/300 Read block 4090 len 12288
14656 179/300 Read block 1899 len 28672
14657 180/300 Read block 855 len 36864
14658 181/300 Read block 4766 len 28672
14659 182/300 Read block 2091 len 12288
14660 183/300 Write block 1685 len 4096 data: 2
14661 184/300 Read block 1882 len 36864
14662 185/300 Write block 3997 len 40960 data: 3 3 4 4 5 5 5 5 5 3
14663 186/300 Flush
14664 187/300 Read block 100 len 28672
14665 188/300 Flush
14666 189/300 Read block 637 len 24576
14667 190/300 Read block 2452 len 20480
14668 191/300 Write block 3810 len 8192 data: 4 5
14669 192/300 Read block 1828 len 20480
14670 193/300 Write block 846 len 20480 data: 2 2 2 3 3
14671 194/300 Read block 4172 len 32768
14672 195/300 Write block 2412 len 8192 data: 5 5
14673 196/300 Read block 1330 len 12288
14674 197/300 Write block 4692 len 12288 data: 4 4 5
14675 198/300 Write block 2764 len 40960 data: 3 3 3 3 3 2 2 3 3 3
14676 199/300 Read block 2710 len 12288
14677 200/300 Write block 1643 len 40960 data: 4 4 3 3 4 3 3 3 3 3
14678 201/300 Write block 2848 len 8192 data: 3 2
14679 202/300 Write block 4957 len 8192 data: 3 3
14680 203/300 Write block 3910 len 40960 data: 4 4 4 4 5 6 6 5 4 3
14681 204/300 Read block 795 len 28672
14682 205/300 Write block 1270 len 24576 data: 4 4 3 3 3 3
14683 206/300 Write block 2276 len 16384 data: 6 5 5 4
14684 207/300 Write block 1138 len 24576 data: 4 5 4 4 4 5
14685 208/300 Write block 1687 len 12288 data: 2 3 3
14686 209/300 Read block 33 len 20480
14687 210/300 Write block 2936 len 8192 data: 6 6
14688 211/300 Write block 3862 len 32768 data: 4 4 4 4 4 6 5 6
14689 212/300 Read block 625 len 4096
14690 213/300 Read block 1555 len 36864
14691 214/300 Read block 2807 len 4096
14692 215/300 Flush
14693 216/300 Write block 1389 len 16384 data: 3 3 4 4
14694 217/300 Read block 1938 len 16384
14695 218/300 Write block 2915 len 8192 data: 4 5
14696 219/300 Read block 2430 len 8192
14697 220/300 Read block 1756 len 20480
14698 221/300 Read block 2886 len 16384
14699 222/300 Read block 2844 len 8192
14700 223/300 Flush
14701 224/300 Read block 1078 len 20480
14702 225/300 Write block 854 len 40960 data: 3 3 4 4 4 3 4 4 4 3
14703 226/300 Write block 1193 len 4096 data: 6
14704 227/300 Flush
14705 228/300 Read block 1665 len 24576
14706 229/300 Read block 502 len 12288
14707 230/300 Read block 36 len 36864
14708 231/300 Read block 1129 len 40960
14709 232/300 Read block 1329 len 36864
14710 233/300 Read block 222 len 16384
14711 234/300 Read block 1389 len 8192
14712 235/300 Read block 4196 len 12288
14713 236/300 Write block 355 len 12288 data: 4 4 3
14714 237/300 Write block 3033 len 20480 data: 5 3 2 2 2
14715 238/300 Read block 2859 len 20480
14716 239/300 Read block 1722 len 36864
14717 240/300 Write block 2982 len 12288 data: 3 3 3
14718 241/300 Read block 3464 len 4096
14719 242/300 Write block 566 len 32768 data: 2 2 2 3 3 3 2 2
14720 243/300 Write block 602 len 4096 data: 5
14721 244/300 Read block 4290 len 24576
14722 245/300 Write block 676 len 40960 data: 6 5 5 4 3 3 3 4 5 5
14723 246/300 Flush
14724 247/300 Write block 308 len 40960 data: 2 2 2 2 3 3 4 4 4 5
14725 248/300 Write block 1137 len 28672 data: 4 5 6 5 5 5 6
14726 249/300 Read block 2351 len 36864
14727 250/300 Write block 4139 len 12288 data: 3 2 2
14728 251/300 Write block 644 len 20480 data: 3 3 4 4 6
14729 252/300 Flush
14730 253/300 Write block 3993 len 24576 data: 5 5 4 2 4 4
14731 254/300 Write block 860 len 16384 data: 5 5 5 4
14732 255/300 Read block 3321 len 40960
14733 256/300 Flush
14734 257/300 Write block 4074 len 8192 data: 4 4
14735 258/300 Read block 379 len 24576
14736 259/300 Read block 4613 len 4096
14737 260/300 Write block 3361 len 16384 data: 2 2 3 3
14738 261/300 Read block 4511 len 8192
14739 262/300 Read block 3146 len 8192
14740 263/300 Read block 2405 len 12288
14741 264/300 Read block 72 len 28672
14742 265/300 Write block 2724 len 16384 data: 3 3 4 5
14743 266/300 Read block 3749 len 16384
14744 267/300 Write block 2217 len 28672 data: 3 3 3 3 4 3 2
14745 268/300 Write block 4501 len 36864 data: 5 5 5 5 5 5 3 3 3
14746 269/300 Write block 2410 len 24576 data: 3 4 6 6 5 5
14747 270/300 Write block 796 len 8192 data: 5 4
14748 271/300 Read block 3916 len 16384
14749 272/300 Read block 1730 len 32768
14750 273/300 Read block 44 len 16384
14751 274/300 Read block 4818 len 16384
14752 275/300 Write block 2675 len 16384 data: 2 2 3 3
14753 276/300 Write block 277 len 16384 data: 5 6 6 5
14754 277/300 Write block 34 len 40960 data: 4 4 4 2 2 2 2 2 3 3
14755 278/300 Read block 4836 len 16384
14756 279/300 Flush
14757 280/300 Read block 990 len 8192
14758 281/300 Read block 2287 len 8192
14759 282/300 Write block 2141 len 32768 data: 2 2 2 2 2 3 3 4
14760 283/300 Flush
14761 284/300 Write block 4870 len 24576 data: 2 3 4 4 4 4
14762 285/300 Write block 1195 len 4096 data: 6
14763 286/300 Read block 4796 len 4096
14764 287/300 Read block 1170 len 20480
14765 288/300 Flush
14766 289/300 Write block 4195 len 24576 data: 5 5 4 3 3 2
14767 290/300 Write block 149 len 8192 data: 3 5
14768 291/300 Write block 57 len 20480 data: 3 3 3 4 4
14769 292/300 Read block 2196 len 28672
14770 293/300 Write block 2560 len 40960 data: 3 2 2 3 3 3 3 4 4 4
14771 294/300 Read block 2711 len 16384
14772 295/300 Write block 684 len 20480 data: 6 6 4 3 3
14773 296/300 Read block 1232 len 12288
14774 297/300 Read block 396 len 12288
14775 298/300 Read block 1474 len 24576
14776 299/300 Read block 2482 len 12288
14777 300/300 Read block 121 len 16384
14778 301/300 Write block 3687 len 8192 data: 2 2
14779 Main task has work to do, go find it
14780 got dsc Start(1)
14781 start 1
14782 Nov 19 17:11:06.910 INFO request completed, latency_us: 179, response_code: 204, uri: /start/cid/1, method: POST, req_id: a8eb1b97-ff25-4f63-92f7-210731b81444, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
14783 Replay: started 1, returned:Ok(())
14784 ----------------------------------------------------------------
14785 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14786 States: Active Offline Active
14787 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14788 4623 Acked Write 3 Done Sent Done false
14789 4624 Acked Read 10 Done Sent Done false
14790 4625 Acked Read 8 Done Sent Done false
14791 4626 Acked Flush 0 Done Sent Done false
14792 4627 Acked Read 1 Done Sent Done false
14793 4628 Acked Write 10 Done Sent Done false
14794 4629 Acked Read 9 Done Sent Done false
14795 4630 Acked Write 4 Done Sent Done false
14796 4631 Acked Write 9 Done Sent Done false
14797 4632 Acked Write 7 Done Sent Done false
14798 4633 Acked Read 10 Done Sent Done false
14799 4634 Acked Read 4 Done Sent Done false
14800 4635 Acked Flush 0 Done Sent Done false
14801 4636 Acked Flush 0 Done Sent Done false
14802 4637 Acked Write 8 Done Sent Done false
14803 4638 Acked Read 10 Done Sent Done false
14804 4639 Acked Read 4 Done Sent Done false
14805 4640 Acked Read 4 Done Sent Done false
14806 4641 Acked Read 8 Done Sent Done false
14807 4642 Acked Read 7 Done Sent Done false
14808 4643 Acked Read 3 Done Sent Done false
14809 4644 Acked Read 1 Done Sent Done false
14810 4645 Acked Flush 0 Done Sent Done false
14811 4646 Acked Read 3 Done Sent Done false
14812 4647 Acked Read 7 Done Sent Done false
14813 4648 Acked Write 9 Done Sent Done false
14814 4649 Acked Read 1 Done Sent Done false
14815 4650 Acked Flush 0 Done Sent Done false
14816 4651 Acked Flush 0 Done Sent Done false
14817 4652 Acked Write 9 Done Sent Done false
14818 4653 Acked Write 6 Done Sent Done false
14819 4654 Acked Read 4 Done Sent Done false
14820 4655 Acked Read 9 Done Sent Done false
14821 4656 Acked Read 9 Done Sent Done false
14822 4657 Acked Flush 0 Done Sent Done false
14823 4658 Acked Write 7 Done Sent Done false
14824 4659 Acked Read 5 Done Sent Done false
14825 4660 Acked Read 1 Done Sent Done false
14826 4661 Acked Write 9 Done Sent Done false
14827 4662 Acked Write 2 Done Sent Done false
14828 4663 Acked Read 9 Done Sent Done false
14829 4664 Acked Read 10 Done Sent Done false
14830 4665 Acked Read 9 Done Sent Done false
14831 4666 Acked Write 9 Done Sent Done false
14832 4667 Acked Read 8 Done Sent Done false
14833 4668 Acked Read 6 Done Sent Done false
14834 4669 Acked Read 8 Done Sent Done false
14835 4670 Acked Write 4 Done Sent Done false
14836 4671 Acked Read 8 Done Sent Done false
14837 4672 Acked Write 2 Done Sent Done false
14838 4673 Acked Write 6 Done Sent Done false
14839 4674 Acked Read 10 Done Sent Done false
14840 4675 Acked Write 5 Done Sent Done false
14841 4676 Acked Write 9 Done Sent Done false
14842 4677 Acked Read 4 Done Sent Done false
14843 4678 Acked Read 10 Done Sent Done false
14844 4679 Acked Read 7 Done Sent Done false
14845 4680 Acked Read 2 Done Sent Done false
14846 4681 Acked Read 4 Done Sent Done false
14847 4682 Acked Read 7 Done Sent Done false
14848 4683 Acked Write 1 Done Sent Done false
14849 4684 Acked Read 10 Done Sent Done false
14850 4685 Acked Write 8 Done Sent Done false
14851 4686 Acked Write 2 Done Sent Done false
14852 4687 Acked Read 9 Done Sent Done false
14853 4688 Acked Write 2 Done Sent Done false
14854 4689 Acked Read 1 Done Sent Done false
14855 4690 Acked Read 1 Done Sent Done false
14856 4691 Acked Read 5 Done Sent Done false
14857 4692 Acked Flush 0 Done Sent Done false
14858 4693 Acked Read 10 Done Sent Done false
14859 4694 Acked Read 1 Done Sent Done false
14860 4695 Acked Read 4 Done Sent Done false
14861 4696 Acked Read 7 Done Sent Done false
14862 4697 Acked Write 8 Done Sent Done false
14863 4698 Acked Write 7 Done Sent Done false
14864 4699 Acked Write 1 Done Sent Done false
14865 4700 Acked Write 4 Done Sent Done false
14866 4701 Acked Flush 0 Done Sent Done false
14867 4702 Acked Write 5 Done Sent Done false
14868 4703 Acked Write 1 Done Sent Done false
14869 4704 Acked Read 8 Done Sent Done false
14870 4705 Acked Write 4 Done Sent Done false
14871 4706 Acked Read 5 Done Sent Done false
14872 4707 Acked Write 10 Done Sent Done false
14873 4708 Acked Write 2 Done Sent Done false
14874 4709 Acked Write 4 Done Sent Done false
14875 4710 Acked Read 1 Done Sent Done false
14876 4711 Acked Read 3 Done Sent Done false
14877 4712 Acked Read 2 Done Sent Done false
14878 4713 Acked Read 8 Done Sent Done false
14879 4714 Acked Write 8 Done Sent Done false
14880 4715 Acked Write 2 Done Sent Done false
14881 4716 Acked Read 10 Done Sent Done false
14882 4717 Acked Read 5 Done Sent Done false
14883 4718 Acked Write 4 Done Sent Done false
14884 4719 Acked Read 10 Done Sent Done false
14885 4720 Acked Write 9 Done Sent Done false
14886 4721 Acked Read 6 Done Sent Done false
14887 4722 Acked Read 6 Done Sent Done false
14888 4723 Acked Write 3 Done Sent Done false
14889 4724 Acked Write 8 Done Sent Done false
14890 4725 Acked Read 6 Done Sent Done false
14891 4726 Acked Write 5 Done Sent Done false
14892 4727 Acked Read 5 Done Sent Done false
14893 4728 Acked Write 7 Done Sent Done false
14894 4729 Acked Flush 0 Done Sent Done false
14895 4730 Acked Read 1 Done Sent Done false
14896 4731 Acked Read 10 Done Sent Done false
14897 4732 Acked Write 1 Done Sent Done false
14898 4733 Acked Read 3 Done Sent Done false
14899 4734 Acked Write 3 Done Sent Done false
14900 4735 Acked Write 3 Done Sent Done false
14901 4736 Acked Write 7 Done Sent Done false
14902 4737 Acked Read 2 Done Sent Done false
14903 4738 Acked Read 6 Done Sent Done false
14904 4739 Acked Write 10 Done Sent Done false
14905 4740 Acked Read 6 Done Sent Done false
14906 4741 Acked Read 2 Done Sent Done false
14907 4742 Acked Read 9 Done Sent Done false
14908 4743 Acked Read 3 Done Sent Done false
14909 4744 Acked Write 5 Done Sent Done false
14910 4745 Acked Read 9 Done Sent Done false
14911 4746 Acked Read 9 Done Sent Done false
14912 4747 Acked Write 5 Done Sent Done false
14913 4748 Acked Read 7 Done Sent Done false
14914 4749 Acked Read 6 Done Sent Done false
14915 4750 Acked Read 6 Done Sent Done false
14916 4751 Acked Flush 0 Done Sent Done false
14917 4752 Acked Write 2 Done Sent Done false
14918 4753 Acked Read 1 Done Sent Done false
14919 4754 Acked Write 6 Done Sent Done false
14920 4755 Acked Read 4 Done Sent Done false
14921 4756 Acked Write 7 Done Sent Done false
14922 4757 Acked Write 5 Done Sent Done false
14923 4758 Acked Flush 0 Done Sent Done false
14924 4759 Acked Read 4 Done Sent Done false
14925 4760 Acked Write 6 Done Sent Done false
14926 4761 Acked Read 1 Done Sent Done false
14927 4762 Acked Write 7 Done Sent Done false
14928 4763 Acked Read 6 Done Sent Done false
14929 4764 Acked Read 8 Done Sent Done false
14930 4765 Acked Read 2 Done Sent Done false
14931 4766 Acked Flush 0 Done Sent Done false
14932 4767 Acked Write 9 Done Sent Done false
14933 4768 Acked Read 4 Done Sent Done false
14934 4769 Acked Write 9 Done Sent Done false
14935 4770 Acked Read 10 Done Sent Done false
14936 4771 Acked Write 9 Done Sent Done false
14937 4772 Acked Write 1 Done Sent Done false
14938 4773 Acked Write 7 Done Sent Done false
14939 4774 Acked Read 2 Done Sent Done false
14940 4775 Acked Write 2 Done Sent Done false
14941 4776 Acked Read 5 Done Sent Done false
14942 4777 Acked Read 7 Done Sent Done false
14943 4778 Acked Write 2 Done Sent Done false
14944 4779 Acked Flush 0 Done Sent Done false
14945 4780 Acked Flush 0 Done Sent Done false
14946 4781 Acked Write 10 Done Sent Done false
14947 4782 Acked Read 1 Done Sent Done false
14948 4783 Acked Flush 0 Done Sent Done false
14949 4784 Acked Write 5 Done Sent Done false
14950 4785 Acked Read 10 Done Sent Done false
14951 4786 Acked Flush 0 Done Sent Done false
14952 4787 Acked Read 9 Done Sent Done false
14953 4788 Acked Read 5 Done Sent Done false
14954 4789 Acked Read 4 Done Sent Done false
14955 4790 Acked Write 5 Done Sent Done false
14956 4791 Acked Read 1 Done Sent Done false
14957 4792 Acked Read 10 Done Sent Done false
14958 4793 Acked Write 7 Done Sent Done false
14959 4794 Acked Read 8 Done Sent Done false
14960 4795 Acked Write 5 Done Sent Done false
14961 4796 Acked Write 1 Done Sent Done false
14962 4797 Acked Read 2 Done Sent Done false
14963 4798 Acked Write 3 Done Sent Done false
14964 4799 Acked Read 3 Done Sent Done false
14965 4800 Acked Read 3 Done Sent Done false
14966 4801 Acked Read 7 Done Sent Done false
14967 4802 Acked Read 9 Done Sent Done false
14968 4803 Acked Read 7 Done Sent Done false
14969 4804 Acked Read 3 Done Sent Done false
14970 4805 Acked Write 1 Done Sent Done false
14971 4806 Acked Read 9 Done Sent Done false
14972 4807 Acked Write 10 Done Sent Done false
14973 4808 Acked Flush 0 Done Sent Done false
14974 4809 Acked Read 7 Done Sent Done false
14975 4810 Acked Flush 0 Done Sent Done false
14976 4811 Acked Read 6 Done Sent Done false
14977 4812 Acked Read 5 Done Sent Done false
14978 4813 Acked Write 2 Done Sent Done false
14979 4814 Acked Read 5 Done Sent Done false
14980 4815 Acked Write 5 Done Sent Done false
14981 4816 Acked Read 8 Done Sent Done false
14982 4817 Acked Write 2 Done Sent Done false
14983 4818 Acked Read 3 Done Sent Done false
14984 4819 Acked Write 3 Done Sent Done false
14985 4820 Acked Write 10 Done Sent Done false
14986 4821 Acked Read 3 Done Sent Done false
14987 4822 Acked Write 10 Done Sent Done false
14988 4823 Acked Write 2 Done Sent Done false
14989 4824 Acked Write 2 Done Sent Done false
14990 4825 Acked Write 10 Done Sent Done false
14991 4826 Acked Read 7 Done Sent Done false
14992 4827 Acked Write 6 Done Sent Done false
14993 4828 Acked Write 4 Done Sent Done false
14994 4829 Acked Write 6 Done Sent Done false
14995 4830 Acked Write 3 Done Sent Done false
14996 4831 Acked Read 5 Done Sent Done false
14997 4832 Acked Write 2 Done Sent Done false
14998 4833 Acked Write 8 Done Sent Done false
14999 4834 Acked Read 1 Done Sent Done false
15000 4835 Acked Read 9 Done Sent Done false
15001 4836 Acked Read 1 Done Sent Done false
15002 4837 Acked Flush 0 Done Sent Done false
15003 4838 Acked Write 4 Done Sent Done false
15004 4839 Acked Read 4 Done Sent Done false
15005 4840 Acked Write 2 Done Sent Done false
15006 4841 Acked Read 2 Done Sent Done false
15007 4842 Acked Read 5 Done Sent Done false
15008 4843 Acked Read 4 Done Sent Done false
15009 4844 Acked Read 2 Done Sent Done false
15010 4845 Acked Flush 0 Done Sent Done false
15011 4846 Acked Read 5 Done Sent Done false
15012 4847 Acked Write 10 Done Sent Done false
15013 4848 Acked Write 1 Done Sent Done false
15014 4849 Acked Flush 0 Done Sent Done false
15015 4850 Acked Read 6 Done Sent Done false
15016 4851 Acked Read 3 Done Sent Done false
15017 4852 Acked Read 9 Done Sent Done false
15018 4853 Acked Read 10 Done Sent Done false
15019 4854 Acked Read 9 Done Sent Done false
15020 4855 Acked Read 4 Done Sent Done false
15021 4856 Acked Read 2 Done Sent Done false
15022 4857 Acked Read 3 Done Sent Done false
15023 4858 Acked Write 3 Done Sent Done false
15024 4859 Acked Write 5 Done Sent Done false
15025 4860 Acked Read 5 Done Sent Done false
15026 4861 Acked Read 9 Done Sent Done false
15027 4862 Acked Write 3 Done Sent Done false
15028 4863 Acked Read 1 Done Sent Done false
15029 4864 Acked Write 8 Done Sent Done false
15030 4865 Acked Write 1 Done Sent Done false
15031 4866 Acked Read 6 Done Sent Done false
15032 4867 Acked Write 10 Done Sent Done false
15033 4868 Acked Flush 0 Done Sent Done false
15034 4869 Acked Write 10 Done Sent Done false
15035 4870 Acked Write 7 Done Sent Done false
15036 4871 Acked Read 9 Done Sent Done false
15037 4872 Acked Write 3 Done Sent Done false
15038 4873 Acked Write 5 Done Sent Done false
15039 4874 Acked Flush 0 Done Sent Done false
15040 4875 Acked Write 6 Done Sent Done false
15041 4876 Acked Write 4 Done Sent Done false
15042 4877 Acked Read 10 Done Sent Done false
15043 4878 Acked Flush 0 Done Sent Done false
15044 4879 Acked Write 2 Done Sent Done false
15045 4880 Acked Read 6 Done Sent Done false
15046 4881 Acked Read 1 Done Sent Done false
15047 4882 Acked Write 4 Done Sent Done false
15048 4883 Acked Read 2 Done Sent Done false
15049 4884 Acked Read 2 Done Sent Done false
15050 4885 Acked Read 3 Done Sent Done false
15051 4886 Acked Read 7 Done Sent Done false
15052 4887 Acked Write 4 Done Sent Done false
15053 4888 Acked Read 4 Done Sent Done false
15054 4889 Acked Write 7 Done Sent Done false
15055 4890 Acked Write 9 Done Sent Done false
15056 4891 Acked Write 6 Done Sent Done false
15057 4892 Acked Write 2 Done Sent Done false
15058 4893 Acked Read 4 Done Sent Done false
15059 4894 Acked Read 8 Done Sent Done false
15060 4895 Acked Read 4 Done Sent Done false
15061 4896 Acked Read 4 Done Sent Done false
15062 4897 Acked Write 4 Done Sent Done false
15063 4898 Acked Write 4 Done Sent Done false
15064 4899 Acked Write 10 Done Sent Done false
15065 4900 Acked Read 4 Done Sent Done false
15066 4901 Acked Flush 0 Done Sent Done false
15067 4902 Acked Read 2 Done Sent Done false
15068 4903 Acked Read 2 Done Sent Done false
15069 4904 Acked Write 8 Done Sent Done false
15070 4905 Acked Flush 0 Done Sent Done false
15071 4906 Acked Write 6 Done Sent Done false
15072 4907 Acked Write 1 Done Sent Done false
15073 4908 Acked Read 1 Done Sent Done false
15074 4909 Acked Read 5 Done Sent Done false
15075 4910 Acked Flush 0 Done Sent Done false
15076 4911 Acked Write 6 Done Sent Done false
15077 4912 Acked Write 2 Done Sent Done false
15078 4913 Acked Write 5 Done Sent Done false
15079 4914 Acked Read 7 Done Sent Done false
15080 4915 Acked Write 10 Done Sent Done false
15081 4916 Acked Read 4 Done Sent Done false
15082 4917 Acked Write 5 Done Sent Done false
15083 4918 Acked Read 3 Done Sent Done false
15084 4919 Acked Read 3 Done Sent Done false
15085 4920 Acked Read 6 Done Sent Done false
15086 4921 Acked Read 3 Done Sent Done false
15087 4922 Acked Read 4 Done Sent Done false
15088 4923 Acked Write 2 Done Sent Done false
15089 STATES DS:0 DS:1 DS:2 TOTAL
15090 Sent 0 301 0 301
15091 Done 301 0 301 602
15092 Skipped 0 0 0 0
15093 Error 0 0 0 0
15094 Last Flush: 4910 4622 4910
15095 Downstairs last five completed: 4622 4621 4620 4619 4618
15096 Upstairs last five completed: 4923 4922 4921 4920 4919
15097 CLIENT: Up:0 ds:301 act:2
15098 [8820] got start action
15099 [8820] I am going to restart
15100 Starting downstairs at port 8820
15101 Make output file at "/tmp/dsc/downstairs-8820.txt"
15102 [8820][1] reports Starting
15103 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1215)
15104 [8820][1] reports Running
15105 ----------------------------------------------------------------
15106 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15107 States: Active Offline Active
15108 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15109 4623 Acked Write 3 Done Sent Done false
15110 4624 Acked Read 10 Done Sent Done false
15111 4625 Acked Read 8 Done Sent Done false
15112 4626 Acked Flush 0 Done Sent Done false
15113 4627 Acked Read 1 Done Sent Done false
15114 4628 Acked Write 10 Done Sent Done false
15115 4629 Acked Read 9 Done Sent Done false
15116 4630 Acked Write 4 Done Sent Done false
15117 4631 Acked Write 9 Done Sent Done false
15118 4632 Acked Write 7 Done Sent Done false
15119 4633 Acked Read 10 Done Sent Done false
15120 4634 Acked Read 4 Done Sent Done false
15121 4635 Acked Flush 0 Done Sent Done false
15122 4636 Acked Flush 0 Done Sent Done false
15123 4637 Acked Write 8 Done Sent Done false
15124 4638 Acked Read 10 Done Sent Done false
15125 4639 Acked Read 4 Done Sent Done false
15126 4640 Acked Read 4 Done Sent Done false
15127 4641 Acked Read 8 Done Sent Done false
15128 4642 Acked Read 7 Done Sent Done false
15129 4643 Acked Read 3 Done Sent Done false
15130 4644 Acked Read 1 Done Sent Done false
15131 4645 Acked Flush 0 Done Sent Done false
15132 4646 Acked Read 3 Done Sent Done false
15133 4647 Acked Read 7 Done Sent Done false
15134 4648 Acked Write 9 Done Sent Done false
15135 4649 Acked Read 1 Done Sent Done false
15136 4650 Acked Flush 0 Done Sent Done false
15137 4651 Acked Flush 0 Done Sent Done false
15138 4652 Acked Write 9 Done Sent Done false
15139 4653 Acked Write 6 Done Sent Done false
15140 4654 Acked Read 4 Done Sent Done false
15141 4655 Acked Read 9 Done Sent Done false
15142 4656 Acked Read 9 Done Sent Done false
15143 4657 Acked Flush 0 Done Sent Done false
15144 4658 Acked Write 7 Done Sent Done false
15145 4659 Acked Read 5 Done Sent Done false
15146 4660 Acked Read 1 Done Sent Done false
15147 4661 Acked Write 9 Done Sent Done false
15148 4662 Acked Write 2 Done Sent Done false
15149 4663 Acked Read 9 Done Sent Done false
15150 4664 Acked Read 10 Done Sent Done false
15151 4665 Acked Read 9 Done Sent Done false
15152 4666 Acked Write 9 Done Sent Done false
15153 4667 Acked Read 8 Done Sent Done false
15154 4668 Acked Read 6 Done Sent Done false
15155 4669 Acked Read 8 Done Sent Done false
15156 4670 Acked Write 4 Done Sent Done false
15157 4671 Acked Read 8 Done Sent Done false
15158 4672 Acked Write 2 Done Sent Done false
15159 4673 Acked Write 6 Done Sent Done false
15160 4674 Acked Read 10 Done Sent Done false
15161 4675 Acked Write 5 Done Sent Done false
15162 4676 Acked Write 9 Done Sent Done false
15163 4677 Acked Read 4 Done Sent Done false
15164 4678 Acked Read 10 Done Sent Done false
15165 4679 Acked Read 7 Done Sent Done false
15166 4680 Acked Read 2 Done Sent Done false
15167 4681 Acked Read 4 Done Sent Done false
15168 4682 Acked Read 7 Done Sent Done false
15169 4683 Acked Write 1 Done Sent Done false
15170 4684 Acked Read 10 Done Sent Done false
15171 4685 Acked Write 8 Done Sent Done false
15172 4686 Acked Write 2 Done Sent Done false
15173 4687 Acked Read 9 Done Sent Done false
15174 4688 Acked Write 2 Done Sent Done false
15175 4689 Acked Read 1 Done Sent Done false
15176 4690 Acked Read 1 Done Sent Done false
15177 4691 Acked Read 5 Done Sent Done false
15178 4692 Acked Flush 0 Done Sent Done false
15179 4693 Acked Read 10 Done Sent Done false
15180 4694 Acked Read 1 Done Sent Done false
15181 4695 Acked Read 4 Done Sent Done false
15182 4696 Acked Read 7 Done Sent Done false
15183 4697 Acked Write 8 Done Sent Done false
15184 4698 Acked Write 7 Done Sent Done false
15185 4699 Acked Write 1 Done Sent Done false
15186 4700 Acked Write 4 Done Sent Done false
15187 4701 Acked Flush 0 Done Sent Done false
15188 4702 Acked Write 5 Done Sent Done false
15189 4703 Acked Write 1 Done Sent Done false
15190 4704 Acked Read 8 Done Sent Done false
15191 4705 Acked Write 4 Done Sent Done false
15192 4706 Acked Read 5 Done Sent Done false
15193 4707 Acked Write 10 Done Sent Done false
15194 4708 Acked Write 2 Done Sent Done false
15195 4709 Acked Write 4 Done Sent Done false
15196 4710 Acked Read 1 Done Sent Done false
15197 4711 Acked Read 3 Done Sent Done false
15198 4712 Acked Read 2 Done Sent Done false
15199 4713 Acked Read 8 Done Sent Done false
15200 4714 Acked Write 8 Done Sent Done false
15201 4715 Acked Write 2 Done Sent Done false
15202 4716 Acked Read 10 Done Sent Done false
15203 4717 Acked Read 5 Done Sent Done false
15204 4718 Acked Write 4 Done Sent Done false
15205 4719 Acked Read 10 Done Sent Done false
15206 4720 Acked Write 9 Done Sent Done false
15207 4721 Acked Read 6 Done Sent Done false
15208 4722 Acked Read 6 Done Sent Done false
15209 4723 Acked Write 3 Done Sent Done false
15210 4724 Acked Write 8 Done Sent Done false
15211 4725 Acked Read 6 Done Sent Done false
15212 4726 Acked Write 5 Done Sent Done false
15213 4727 Acked Read 5 Done Sent Done false
15214 4728 Acked Write 7 Done Sent Done false
15215 4729 Acked Flush 0 Done Sent Done false
15216 4730 Acked Read 1 Done Sent Done false
15217 4731 Acked Read 10 Done Sent Done false
15218 4732 Acked Write 1 Done Sent Done false
15219 4733 Acked Read 3 Done Sent Done false
15220 4734 Acked Write 3 Done Sent Done false
15221 4735 Acked Write 3 Done Sent Done false
15222 4736 Acked Write 7 Done Sent Done false
15223 4737 Acked Read 2 Done Sent Done false
15224 4738 Acked Read 6 Done Sent Done false
15225 4739 Acked Write 10 Done Sent Done false
15226 4740 Acked Read 6 Done Sent Done false
15227 4741 Acked Read 2 Done Sent Done false
15228 4742 Acked Read 9 Done Sent Done false
15229 4743 Acked Read 3 Done Sent Done false
15230 4744 Acked Write 5 Done Sent Done false
15231 4745 Acked Read 9 Done Sent Done false
15232 4746 Acked Read 9 Done Sent Done false
15233 4747 Acked Write 5 Done Sent Done false
15234 4748 Acked Read 7 Done Sent Done false
15235 4749 Acked Read 6 Done Sent Done false
15236 4750 Acked Read 6 Done Sent Done false
15237 4751 Acked Flush 0 Done Sent Done false
15238 4752 Acked Write 2 Done Sent Done false
15239 4753 Acked Read 1 Done Sent Done false
15240 4754 Acked Write 6 Done Sent Done false
15241 4755 Acked Read 4 Done Sent Done false
15242 4756 Acked Write 7 Done Sent Done false
15243 4757 Acked Write 5 Done Sent Done false
15244 4758 Acked Flush 0 Done Sent Done false
15245 4759 Acked Read 4 Done Sent Done false
15246 4760 Acked Write 6 Done Sent Done false
15247 4761 Acked Read 1 Done Sent Done false
15248 4762 Acked Write 7 Done Sent Done false
15249 4763 Acked Read 6 Done Sent Done false
15250 4764 Acked Read 8 Done Sent Done false
15251 4765 Acked Read 2 Done Sent Done false
15252 4766 Acked Flush 0 Done Sent Done false
15253 4767 Acked Write 9 Done Sent Done false
15254 4768 Acked Read 4 Done Sent Done false
15255 4769 Acked Write 9 Done Sent Done false
15256 4770 Acked Read 10 Done Sent Done false
15257 4771 Acked Write 9 Done Sent Done false
15258 4772 Acked Write 1 Done Sent Done false
15259 4773 Acked Write 7 Done Sent Done false
15260 4774 Acked Read 2 Done Sent Done false
15261 4775 Acked Write 2 Done Sent Done false
15262 4776 Acked Read 5 Done Sent Done false
15263 4777 Acked Read 7 Done Sent Done false
15264 4778 Acked Write 2 Done Sent Done false
15265 4779 Acked Flush 0 Done Sent Done false
15266 4780 Acked Flush 0 Done Sent Done false
15267 4781 Acked Write 10 Done Sent Done false
15268 4782 Acked Read 1 Done Sent Done false
15269 4783 Acked Flush 0 Done Sent Done false
15270 4784 Acked Write 5 Done Sent Done false
15271 4785 Acked Read 10 Done Sent Done false
15272 4786 Acked Flush 0 Done Sent Done false
15273 4787 Acked Read 9 Done Sent Done false
15274 4788 Acked Read 5 Done Sent Done false
15275 4789 Acked Read 4 Done Sent Done false
15276 4790 Acked Write 5 Done Sent Done false
15277 4791 Acked Read 1 Done Sent Done false
15278 4792 Acked Read 10 Done Sent Done false
15279 4793 Acked Write 7 Done Sent Done false
15280 4794 Acked Read 8 Done Sent Done false
15281 4795 Acked Write 5 Done Sent Done false
15282 4796 Acked Write 1 Done Sent Done false
15283 4797 Acked Read 2 Done Sent Done false
15284 4798 Acked Write 3 Done Sent Done false
15285 4799 Acked Read 3 Done Sent Done false
15286 4800 Acked Read 3 Done Sent Done false
15287 4801 Acked Read 7 Done Sent Done false
15288 4802 Acked Read 9 Done Sent Done false
15289 4803 Acked Read 7 Done Sent Done false
15290 4804 Acked Read 3 Done Sent Done false
15291 4805 Acked Write 1 Done Sent Done false
15292 4806 Acked Read 9 Done Sent Done false
15293 4807 Acked Write 10 Done Sent Done false
15294 4808 Acked Flush 0 Done Sent Done false
15295 4809 Acked Read 7 Done Sent Done false
15296 4810 Acked Flush 0 Done Sent Done false
15297 4811 Acked Read 6 Done Sent Done false
15298 4812 Acked Read 5 Done Sent Done false
15299 4813 Acked Write 2 Done Sent Done false
15300 4814 Acked Read 5 Done Sent Done false
15301 4815 Acked Write 5 Done Sent Done false
15302 4816 Acked Read 8 Done Sent Done false
15303 4817 Acked Write 2 Done Sent Done false
15304 4818 Acked Read 3 Done Sent Done false
15305 4819 Acked Write 3 Done Sent Done false
15306 4820 Acked Write 10 Done Sent Done false
15307 4821 Acked Read 3 Done Sent Done false
15308 4822 Acked Write 10 Done Sent Done false
15309 4823 Acked Write 2 Done Sent Done false
15310 4824 Acked Write 2 Done Sent Done false
15311 4825 Acked Write 10 Done Sent Done false
15312 4826 Acked Read 7 Done Sent Done false
15313 4827 Acked Write 6 Done Sent Done false
15314 4828 Acked Write 4 Done Sent Done false
15315 4829 Acked Write 6 Done Sent Done false
15316 4830 Acked Write 3 Done Sent Done false
15317 4831 Acked Read 5 Done Sent Done false
15318 4832 Acked Write 2 Done Sent Done false
15319 4833 Acked Write 8 Done Sent Done false
15320 4834 Acked Read 1 Done Sent Done false
15321 4835 Acked Read 9 Done Sent Done false
15322 4836 Acked Read 1 Done Sent Done false
15323 4837 Acked Flush 0 Done Sent Done false
15324 4838 Acked Write 4 Done Sent Done false
15325 4839 Acked Read 4 Done Sent Done false
15326 4840 Acked Write 2 Done Sent Done false
15327 4841 Acked Read 2 Done Sent Done false
15328 4842 Acked Read 5 Done Sent Done false
15329 4843 Acked Read 4 Done Sent Done false
15330 4844 Acked Read 2 Done Sent Done false
15331 4845 Acked Flush 0 Done Sent Done false
15332 4846 Acked Read 5 Done Sent Done false
15333 4847 Acked Write 10 Done Sent Done false
15334 4848 Acked Write 1 Done Sent Done false
15335 4849 Acked Flush 0 Done Sent Done false
15336 4850 Acked Read 6 Done Sent Done false
15337 4851 Acked Read 3 Done Sent Done false
15338 4852 Acked Read 9 Done Sent Done false
15339 4853 Acked Read 10 Done Sent Done false
15340 4854 Acked Read 9 Done Sent Done false
15341 4855 Acked Read 4 Done Sent Done false
15342 4856 Acked Read 2 Done Sent Done false
15343 4857 Acked Read 3 Done Sent Done false
15344 4858 Acked Write 3 Done Sent Done false
15345 4859 Acked Write 5 Done Sent Done false
15346 4860 Acked Read 5 Done Sent Done false
15347 4861 Acked Read 9 Done Sent Done false
15348 4862 Acked Write 3 Done Sent Done false
15349 4863 Acked Read 1 Done Sent Done false
15350 4864 Acked Write 8 Done Sent Done false
15351 4865 Acked Write 1 Done Sent Done false
15352 4866 Acked Read 6 Done Sent Done false
15353 4867 Acked Write 10 Done Sent Done false
15354 4868 Acked Flush 0 Done Sent Done false
15355 4869 Acked Write 10 Done Sent Done false
15356 4870 Acked Write 7 Done Sent Done false
15357 4871 Acked Read 9 Done Sent Done false
15358 4872 Acked Write 3 Done Sent Done false
15359 4873 Acked Write 5 Done Sent Done false
15360 4874 Acked Flush 0 Done Sent Done false
15361 4875 Acked Write 6 Done Sent Done false
15362 4876 Acked Write 4 Done Sent Done false
15363 4877 Acked Read 10 Done Sent Done false
15364 4878 Acked Flush 0 Done Sent Done false
15365 4879 Acked Write 2 Done Sent Done false
15366 4880 Acked Read 6 Done Sent Done false
15367 4881 Acked Read 1 Done Sent Done false
15368 4882 Acked Write 4 Done Sent Done false
15369 4883 Acked Read 2 Done Sent Done false
15370 4884 Acked Read 2 Done Sent Done false
15371 4885 Acked Read 3 Done Sent Done false
15372 4886 Acked Read 7 Done Sent Done false
15373 4887 Acked Write 4 Done Sent Done false
15374 4888 Acked Read 4 Done Sent Done false
15375 4889 Acked Write 7 Done Sent Done false
15376 4890 Acked Write 9 Done Sent Done false
15377 4891 Acked Write 6 Done Sent Done false
15378 4892 Acked Write 2 Done Sent Done false
15379 4893 Acked Read 4 Done Sent Done false
15380 4894 Acked Read 8 Done Sent Done false
15381 4895 Acked Read 4 Done Sent Done false
15382 4896 Acked Read 4 Done Sent Done false
15383 4897 Acked Write 4 Done Sent Done false
15384 4898 Acked Write 4 Done Sent Done false
15385 4899 Acked Write 10 Done Sent Done false
15386 4900 Acked Read 4 Done Sent Done false
15387 4901 Acked Flush 0 Done Sent Done false
15388 4902 Acked Read 2 Done Sent Done false
15389 4903 Acked Read 2 Done Sent Done false
15390 4904 Acked Write 8 Done Sent Done false
15391 4905 Acked Flush 0 Done Sent Done false
15392 4906 Acked Write 6 Done Sent Done false
15393 4907 Acked Write 1 Done Sent Done false
15394 4908 Acked Read 1 Done Sent Done false
15395 4909 Acked Read 5 Done Sent Done false
15396 4910 Acked Flush 0 Done Sent Done false
15397 4911 Acked Write 6 Done Sent Done false
15398 4912 Acked Write 2 Done Sent Done false
15399 4913 Acked Write 5 Done Sent Done false
15400 4914 Acked Read 7 Done Sent Done false
15401 4915 Acked Write 10 Done Sent Done false
15402 4916 Acked Read 4 Done Sent Done false
15403 4917 Acked Write 5 Done Sent Done false
15404 4918 Acked Read 3 Done Sent Done false
15405 4919 Acked Read 3 Done Sent Done false
15406 4920 Acked Read 6 Done Sent Done false
15407 4921 Acked Read 3 Done Sent Done false
15408 4922 Acked Read 4 Done Sent Done false
15409 4923 Acked Write 2 Done Sent Done false
15410 4924 Acked Flush 0 Done Sent Done false
15411 STATES DS:0 DS:1 DS:2 TOTAL
15412 Sent 0 302 0 302
15413 Done 302 0 302 604
15414 Skipped 0 0 0 0
15415 Error 0 0 0 0
15416 Last Flush: 4924 4622 4924
15417 Downstairs last five completed: 4622 4621 4620 4619 4618
15418 Upstairs last five completed: 4924 4923 4922 4921 4920
15419 CLIENT: Up:0 ds:302 act:2
154202024-11-19T17:11:12.674ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154212024-11-19T17:11:12.674ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52721) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154222024-11-19T17:11:12.675ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154232024-11-19T17:11:12.676ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154242024-11-19T17:11:12.676ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154252024-11-19T17:11:12.676ZINFOcrucible: send last flush ID to this DS: 4622 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154262024-11-19T17:11:12.676ZINFOcrucible: Replied this last flush ID: 4622 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154272024-11-19T17:11:12.676ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154282024-11-19T17:11:12.676ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154292024-11-19T17:11:12.676ZINFOcrucible: [1] client re-new 302 jobs since flush 4622 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
154302024-11-19T17:11:12.676ZINFOcrucible: [1] Replayed 302 jobs since flush: 4622 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
15431 ----------------------------------------------------------------
15432 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15433 Downstairs last five completed: 4924 4923 4922 4921 4920
15434 Upstairs last five completed: 4924 4923 4922 4921 4920
15435 CLIENT: Up:0 ds:0 act:3
15436 Replay: All jobs finished
15437 Main task has work to do, go find it
15438 got dsc Stop(0)
15439 stop 0
15440 [8810] Got stop action so:false kr:false
15441 Nov 19 17:11:14.914 INFO request completed, latency_us: 320, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 578b407e-9684-4e51-b962-6974b273122f, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
154422024-11-19T17:11:14.915ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154432024-11-19T17:11:14.915ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
154442024-11-19T17:11:14.915ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
154452024-11-19T17:11:14.915ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
15446 Nov 19 17:11:14.915 INFO request completed, latency_us: 136, response_code{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:11:14.9157591Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
15447 : 200, uri: /state/cid/0, method: GET, req_id:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:11:14.915789877Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
15448 6a93d35a-60fb-4974-8989-f07a68c6b728, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
15449 [8810] Exited with: ExitStatus(unix_wait_status(9))
15450 [8810][0] reports Exit
15451 Nov 19 17:11:18.918 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/0, method: GET, req_id: ff4758c9-609f-4001-930e-e053cc55b7d3, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
15452 001/300 Read block 2352 len 28672
15453 002/300 Flush
15454 003/300 Write block 3736 len 36864 data: 5 5 5 7 9 8 8 8 7
15455 004/300 Write block 3443 len 36864 data: 4 4 4 3 2 3 5 5 6
15456 005/300 Read block 495 len 28672
15457 006/300 Flush
15458 007/300 Write block 3726 len 36864 data: 4 4 5 6 5 4 4 4 4
15459 008/300 Write block 1625 len 20480 data: 6 4 3 5 5
15460 009/300 Write block 785 len 16384 data: 6 7 7 6
15461 010/300 Read block 3156 len 36864
15462 011/300 Read block 448 len 16384
15463 012/300 Read block 3962 len 40960
15464 013/300 Read block 659 len 8192
15465 014/300 Read block 2898 len 24576
15466 015/300 Read block 1246 len 24576
15467 016/300 Flush
15468 017/300 Read block 4244 len 4096
15469 018/300 Write block 997 len 12288 data: 3 3 3
15470 019/300 Write block 1848 len 8192 data: 3 3
15471 020/300 Write block 1679 len 24576 data: 5 4 4 3 3 3
15472 021/300 Read block 3036 len 40960
15473 022/300 Write block 2138 len 36864 data: 3 2 2 3 3 3 3 3 4
15474 023/300 Write block 2422 len 32768 data: 3 3 3 5 4 4 3 3
15475 024/300 Read block 1226 len 28672
15476 025/300 Flush
15477 026/300 Write block 3049 len 20480 data: 3 3 3 3 2
15478 027/300 Write block 2483 len 40960 data: 2 3 3 4 5 4 3 3 3 2
15479 028/300 Write block 1491 len 40960 data: 2 2 3 3 3 4 4 3 3 3
15480 029/300 Read block 1275 len 32768
15481 030/300 Write block 583 len 24576 data: 2 2 2 3 3 4
15482 031/300 Write block 1474 len 8192 data: 4 3
15483 032/300 Write block 706 len 8192 data: 3 3
15484 033/300 Read block 2654 len 28672
15485 034/300 Read block 4473 len 20480
15486 035/300 Write block 1245 len 24576 data: 4 5 4 4 4 4
15487 036/300 Read block 771 len 12288
15488 037/300 Read block 854 len 24576
15489 038/300 Write block 2077 len 36864 data: 6 6 6 4 3 3 3 2 2
15490 039/300 Write block 109 len 8192 data: 2 2
15491 040/300 Write block 1801 len 12288 data: 3 3 3
15492 041/300 Read block 294 len 20480
15493 042/300 Read block 1343 len 32768
15494 043/300 Write block 3419 len 4096 data: 2
15495 044/300 Read block 4224 len 8192
15496 045/300 Write block 2241 len 36864 data: 3 4 5 4 4 4 3 3 3
15497 046/300 Write block 890 len 36864 data: 3 4 4 4 4 4 3 2 2
15498 047/300 Read block 1778 len 36864
15499 048/300 Read block 4369 len 28672
15500 049/300 Flush
15501 050/300 Read block 4266 len 36864
15502 051/300 Read block 152 len 16384
15503 052/300 Read block 2710 len 28672
15504 053/300 Read block 1228 len 28672
15505 054/300 Write block 132 len 36864 data: 5 7 7 7 8 8 6 8 6
15506 055/300 Write block 824 len 40960 data: 5 5 4 4 3 4 5 5 5 4
15507 056/300 Read block 1631 len 28672
15508 057/300 Write block 2772 len 28672 data: 4 4 4 4 5 3 4
15509 058/300 Read block 3676 len 4096
15510 059/300 Write block 2254 len 16384 data: 5 5 6 6
15511 060/300 Read block 3167 len 24576
15512 061/300 Flush
15513 062/300 Read block 1045 len 4096
15514 063/300 Flush
15515 064/300 Write block 3481 len 24576 data: 5 6 5 6 7 7
15516 065/300 Write block 2789 len 16384 data: 4 4 4 3
15517 066/300 Read block 1388 len 20480
15518 067/300 Read block 559 len 40960
15519 068/300 Write block 756 len 40960 data: 2 2 2 2 2 2 2 2 2 2
15520 069/300 Flush
15521 070/300 Read block 2776 len 28672
15522 071/300 Read block 4097 len 20480
15523 072/300 Write block 277 len 20480 data: 6 7 7 6 5
15524 073/300 Write block 1629 len 32768 data: 6 4 4 5 4 4 4 3
15525 074/300 Read block 1790 len 28672
15526 075/300 Read block 1449 len 24576
15527 076/300 Read block 1752 len 16384
15528 077/300 Read block 2266 len 8192
15529 078/300 Write block 2646 len 16384 data: 4 3 3 3
15530 079/300 Write block 233 len 24576 data: 3 3 2 2 2 2
15531 080/300 Read block 3963 len 28672
15532 081/300 Write block 938 len 36864 data: 5 5 4 4 4 5 3 4 4
15533 082/300 Read block 327 len 32768
15534 083/300 Write block 3001 len 8192 data: 2 2
15535 084/300 Read block 1271 len 36864
15536 085/300 Read block 4295 len 40960
15537 086/300 Flush
15538 087/300 Read block 2488 len 4096
15539 088/300 Read block 4595 len 36864
15540 089/300 Read block 2157 len 8192
15541 090/300 Read block 1801 len 32768
15542 091/300 Write block 1679 len 24576 data: 6 5 5 4 4 4
15543 092/300 Read block 4712 len 36864
15544 093/300 Write block 1747 len 20480 data: 3 3 4 4 4
15545 094/300 Write block 106 len 4096 data: 4
15546 095/300 Write block 3283 len 16384 data: 4 4 4 3
15547 096/300 Write block 3752 len 24576 data: 6 6 5 4 5 4
15548 097/300 Read block 3946 len 24576
15549 098/300 Read block 3461 len 4096
15550 099/300 Write block 2037 len 24576 data: 4 4 4 3 3 3
15551 100/300 Read block 2826 len 24576
15552 101/300 Read block 69 len 32768
15553 102/300 Flush
15554 103/300 Write block 4739 len 24576 data: 3 3 2 3 3 3
15555 104/300 Read block 910 len 4096
15556 105/300 Write block 3482 len 40960 data: 7 6 7 8 8 7 4 5 4 4
15557 106/300 Read block 4925 len 40960
15558 107/300 Read block 1320 len 4096
15559 108/300 Read block 4845 len 8192
15560 109/300 Write block 4060 len 4096 data: 3
15561 110/300 Read block 317 len 8192
15562 111/300 Write block 1929 len 24576 data: 4 3 3 4 4 3
15563 112/300 Read block 4819 len 24576
15564 113/300 Write block 2124 len 32768 data: 2 2 2 2 2 3 3 3
15565 114/300 Read block 1130 len 32768
15566 115/300 Read block 4512 len 8192
15567 116/300 Write block 4184 len 36864 data: 4 3 3 3 4 5 3 4 5
15568 117/300 Read block 4720 len 12288
15569 118/300 Read block 4582 len 24576
15570 119/300 Write block 4544 len 24576 data: 3 3 4 3 3 3
15571 120/300 Read block 4106 len 4096
15572 121/300 Read block 905 len 28672
15573 122/300 Read block 441 len 32768
15574 123/300 Write block 3448 len 12288 data: 4 6 6
15575 124/300 Write block 174 len 40960 data: 3 4 4 4 4 4 5 5 3 3
15576 125/300 Flush
15577 126/300 Write block 4684 len 40960 data: 2 2 2 3 3 5 5 4 5 5
15578 127/300 Write block 3240 len 40960 data: 5 4 4 4 3 5 4 4 4 5
15579 128/300 Read block 3039 len 24576
15580 129/300 Read block 4924 len 32768
15581 130/300 Write block 2310 len 8192 data: 3 3
15582 131/300 Write block 2896 len 20480 data: 5 4 4 4 4
15583 132/300 Read block 1673 len 28672
15584 133/300 Flush
15585 134/300 Write block 3793 len 8192 data: 3 3
15586 135/300 Write block 2560 len 28672 data: 4 3 3 4 4 4 4
15587 136/300 Read block 1718 len 40960
15588 137/300 Read block 4181 len 20480
15589 138/300 Read block 1314 len 40960
15590 139/300 Read block 3886 len 8192
15591 140/300 Read block 3155 len 8192
15592 141/300 Flush
15593 142/300 Read block 4898 len 28672
15594 143/300 Write block 2793 len 24576 data: 3 2 3 3 2 2
15595 144/300 Read block 709 len 12288
15596 145/300 Write block 1872 len 40960 data: 3 3 2 4 5 6 8 8 7 6
15597 146/300 Read block 1047 len 24576
15598 147/300 Flush
15599 148/300 Read block 319 len 28672
15600 149/300 Write block 1446 len 36864 data: 5 7 6 6 6 6 6 6 4
15601 150/300 Write block 1850 len 16384 data: 3 3 3 2
15602 151/300 Read block 999 len 20480
15603 152/300 Flush
15604 153/300 Read block 2206 len 24576
15605 154/300 Write block 1171 len 32768 data: 4 3 4 3 3 3 3 3
15606 155/300 Flush
15607 156/300 Write block 754 len 8192 data: 4 3
15608 157/300 Read block 3095 len 8192
15609 158/300 Write block 763 len 24576 data: 3 3 3 2 2 2
15610 159/300 Read block 2668 len 16384
15611 160/300 Read block 1172 len 36864
15612 161/300 Read block 4462 len 12288
15613 162/300 Read block 1181 len 28672
15614 163/300 Flush
15615 164/300 Read block 717 len 32768
15616 165/300 Read block 1337 len 36864
15617 166/300 Read block 4846 len 28672
15618 167/300 Write block 2142 len 8192 data: 4 4
15619 168/300 Write block 3411 len 8192 data: 4 3
15620 169/300 Write block 3902 len 8192 data: 2 2
15621 170/300 Write block 1302 len 28672 data: 2 3 3 3 3 3 3
15622 171/300 Read block 2940 len 12288
15623 172/300 Read block 2481 len 20480
15624 173/300 Write block 2495 len 28672 data: 2 3 3 3 3 3 3
15625 174/300 Write block 1689 len 40960 data: 4 3 4 5 6 6 5 5 5 3
15626 175/300 Read block 4008 len 24576
15627 176/300 Read block 572 len 4096
15628 177/300 Read block 1067 len 12288
15629 178/300 Flush
15630 179/300 Write block 2469 len 24576 data: 3 3 3 3 3 3
15631 180/300 Write block 773 len 20480 data: 3 3 3 3 4
15632 181/300 Read block 1529 len 28672
15633 182/300 Write block 1322 len 40960 data: 5 5 4 4 4 2 2 3 4 5
15634 183/300 Write block 458 len 16384 data: 3 3 3 3
15635 184/300 Write block 2503 len 28672 data: 3 4 5 4 4 4 4
15636 185/300 Write block 2681 len 28672 data: 4 4 4 4 4 4 5
15637 186/300 Flush
15638 187/300 Write block 1704 len 4096 data: 4
15639 188/300 Flush
15640 189/300 Write block 1730 len 16384 data: 3 4 4 3
15641 190/300 Read block 1860 len 32768
15642 191/300 Write block 1449 len 24576 data: 7 7 7 7 7 5
15643 192/300 Write block 4483 len 16384 data: 3 4 4 5
15644 193/300 Read block 142 len 32768
15645 194/300 Read block 1477 len 20480
15646 195/300 Write block 3043 len 40960 data: 3 3 3 3 2 3 4 4 4 4
15647 196/300 Read block 2082 len 36864
15648 197/300 Write block 2374 len 12288 data: 2 3 3
15649 198/300 Read block 2859 len 16384
15650 199/300 Write block 1967 len 36864 data: 3 3 3 3 3 3 4 5 5
15651 200/300 Write block 4805 len 32768 data: 4 4 3 3 3 2 2 2
15652 201/300 Read block 585 len 4096
15653 202/300 Read block 3930 len 4096
15654 203/300 Write block 884 len 16384 data: 3 4 4 4
15655 204/300 Read block 344 len 24576
15656 205/300 Write block 1281 len 8192 data: 4 4
15657 206/300 Write block 1143 len 20480 data: 7 7 7 6 6
15658 207/300 Write block 2615 len 36864 data: 5 6 6 4 5 6 6 6 3
15659 208/300 Read block 3284 len 32768
15660 209/300 Flush
15661 210/300 Write block 4317 len 40960 data: 2 2 2 2 2 2 2 2 2 2
15662 211/300 Write block 3668 len 24576 data: 3 3 2 3 4 4
15663 212/300 Write block 3529 len 8192 data: 3 4
15664 213/300 Read block 4630 len 28672
15665 214/300 Read block 3936 len 28672
15666 215/300 Read block 155 len 20480
15667 216/300 Read block 2292 len 28672
15668 217/300 Read block 1667 len 28672
15669 218/300 Write block 4046 len 16384 data: 3 3 3 3
15670 219/300 Write block 987 len 32768 data: 2 3 4 4 5 5 5 3
15671 220/300 Write block 2055 len 20480 data: 4 4 2 2 2
15672 221/300 Write block 4043 len 20480 data: 3 3 3 4 4
15673 222/300 Write block 1579 len 40960 data: 3 4 4 5 4 5 6 5 4 3
15674 223/300 Flush
15675 224/300 Write block 2560 len 40960 data: 5 4 4 5 5 5 5 5 5 5
15676 225/300 Read block 3216 len 8192
15677 226/300 Read block 2363 len 24576
15678 227/300 Read block 4491 len 16384
15679 228/300 Read block 1182 len 40960
15680 229/300 Write block 4478 len 12288 data: 4 4 4
15681 230/300 Write block 777 len 12288 data: 5 4 5
15682 231/300 Read block 658 len 40960
15683 232/300 Read block 1986 len 8192
15684 233/300 Read block 3543 len 12288
15685 234/300 Read block 4817 len 12288
15686 235/300 Read block 1847 len 16384
15687 236/300 Read block 2096 len 8192
15688 237/300 Write block 3335 len 40960 data: 4 5 5 5 5 5 4 3 2 2
15689 238/300 Read block 1516 len 20480
15690 239/300 Write block 2081 len 4096 data: 4
15691 240/300 Write block 2857 len 24576 data: 3 2 3 4 3 3
15692 241/300 Read block 558 len 36864
15693 242/300 Write block 4114 len 8192 data: 2 2
15694 243/300 Read block 1995 len 12288
15695 244/300 Read block 3137 len 24576
15696 245/300 Read block 2994 len 12288
15697 246/300 Read block 737 len 24576
15698 247/300 Write block 4137 len 8192 data: 3 3
15699 248/300 Write block 4890 len 40960 data: 4 4 4 4 5 5 6 5 5 5
15700 249/300 Write block 3862 len 28672 data: 5 5 5 5 5 7 6
15701 250/300 Write block 3981 len 8192 data: 2 2
15702 251/300 Read block 3296 len 20480
15703 252/300 Write block 3662 len 32768 data: 6 5 4 3 2 3 4 4
15704 253/300 Read block 3545 len 36864
15705 254/300 Read block 1728 len 16384
15706 255/300 Read block 300 len 16384
15707 256/300 Read block 4734 len 36864
15708 257/300 Read block 2658 len 12288
15709 258/300 Flush
15710 259/300 Read block 2768 len 40960
15711 260/300 Read block 838 len 4096
15712 261/300 Write block 3052 len 4096 data: 5
15713 262/300 Flush
15714 263/300 Read block 1774 len 20480
15715 264/300 Write block 608 len 8192 data: 3 3
15716 265/300 Write block 3420 len 8192 data: 2 2
15717 266/300 Read block 498 len 24576
15718 267/300 Write block 876 len 28672 data: 4 3 4 4 3 3 3
15719 268/300 Read block 2329 len 32768
15720 269/300 Read block 3805 len 20480
15721 270/300 Read block 4141 len 8192
15722 271/300 Write block 1806 len 16384 data: 4 4 4 3
15723 272/300 Read block 2989 len 20480
15724 273/300 Read block 1179 len 4096
15725 274/300 Read block 3767 len 36864
15726 275/300 Write block 1526 len 32768 data: 3 3 3 3 3 3 3 3
15727 276/300 Read block 4797 len 12288
15728 277/300 Flush
15729 278/300 Write block 2779 len 16384 data: 3 3 3 3
15730 279/300 Write block 659 len 28672 data: 4 4 4 5 4 5 4
15731 280/300 Read block 4504 len 16384
15732 281/300 Write block 4495 len 24576 data: 4 4 4 2 3 3
15733 282/300 Read block 1945 len 20480
15734 283/300 Read block 3228 len 32768
15735 284/300 Flush
15736 285/300 Read block 3089 len 36864
15737 286/300 Read block 289 len 40960
15738 287/300 Write block 4865 len 16384 data: 2 2 2 2
15739 288/300 Read block 943 len 8192
15740 289/300 Write block 3280 len 32768 data: 4 4 4 5 5 5 4 3
15741 290/300 Write block 4725 len 36864 data: 3 5 4 4 3 3 4 5 4
15742 291/300 Read block 4210 len 28672
15743 292/300 Read block 2064 len 36864
15744 293/300 Write block 4148 len 16384 data: 3 4 4 4
15745 294/300 Read block 1607 len 40960
15746 295/300 Write block 1178 len 20480 data: 4 2 2 3 3
15747 296/300 Flush
15748 297/300 Write block 1395 len 24576 data: 4 4 5 5 6 7
15749 298/300 Flush
15750 299/300 Write block 3367 len 36864 data: 4 3 3 4 4 5 4 5 3
15751 300/300 Read block 1025 len 4096
15752 301/300 Flush
15753 Main task has work to do, go find it
15754 got dsc Start(0)
15755 start 0
15756 Nov 19 17:11:19.159 INFO request completed, latency_us: 182, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6cc31e23-ba78-420f-8d3e-fd68e111c238, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
15757 Replay: started 0, returned:Ok(())
15758 ----------------------------------------------------------------
15759 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15760 States: Offline Active Active
15761 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15762 4925 Acked Read 7 Sent Done Done false
15763 4926 Acked Flush 0 Sent Done Done false
15764 4927 Acked Write 9 Sent Done Done false
15765 4928 Acked Write 9 Sent Done Done false
15766 4929 Acked Read 7 Sent Done Done false
15767 4930 Acked Flush 0 Sent Done Done false
15768 4931 Acked Write 9 Sent Done Done false
15769 4932 Acked Write 5 Sent Done Done false
15770 4933 Acked Write 4 Sent Done Done false
15771 4934 Acked Read 9 Sent Done Done false
15772 4935 Acked Read 4 Sent Done Done false
15773 4936 Acked Read 10 Sent Done Done false
15774 4937 Acked Read 2 Sent Done Done false
15775 4938 Acked Read 6 Sent Done Done false
15776 4939 Acked Read 6 Sent Done Done false
15777 4940 Acked Flush 0 Sent Done Done false
15778 4941 Acked Read 1 Sent Done Done false
15779 4942 Acked Write 3 Sent Done Done false
15780 4943 Acked Write 2 Sent Done Done false
15781 4944 Acked Write 6 Sent Done Done false
15782 4945 Acked Read 10 Sent Done Done false
15783 4946 Acked Write 9 Sent Done Done false
15784 4947 Acked Write 8 Sent Done Done false
15785 4948 Acked Read 7 Sent Done Done false
15786 4949 Acked Flush 0 Sent Done Done false
15787 4950 Acked Write 5 Sent Done Done false
15788 4951 Acked Write 10 Sent Done Done false
15789 4952 Acked Write 10 Sent Done Done false
15790 4953 Acked Read 8 Sent Done Done false
15791 4954 Acked Write 6 Sent Done Done false
15792 4955 Acked Write 2 Sent Done Done false
15793 4956 Acked Write 2 Sent Done Done false
15794 4957 Acked Read 7 Sent Done Done false
15795 4958 Acked Read 5 Sent Done Done false
15796 4959 Acked Write 6 Sent Done Done false
15797 4960 Acked Read 3 Sent Done Done false
15798 4961 Acked Read 6 Sent Done Done false
15799 4962 Acked Write 9 Sent Done Done false
15800 4963 Acked Write 2 Sent Done Done false
15801 4964 Acked Write 3 Sent Done Done false
15802 4965 Acked Read 5 Sent Done Done false
15803 4966 Acked Read 8 Sent Done Done false
15804 4967 Acked Write 1 Sent Done Done false
15805 4968 Acked Read 2 Sent Done Done false
15806 4969 Acked Write 9 Sent Done Done false
15807 4970 Acked Write 9 Sent Done Done false
15808 4971 Acked Read 9 Sent Done Done false
15809 4972 Acked Read 7 Sent Done Done false
15810 4973 Acked Flush 0 Sent Done Done false
15811 4974 Acked Read 9 Sent Done Done false
15812 4975 Acked Read 4 Sent Done Done false
15813 4976 Acked Read 7 Sent Done Done false
15814 4977 Acked Read 7 Sent Done Done false
15815 4978 Acked Write 9 Sent Done Done false
15816 4979 Acked Write 10 Sent Done Done false
15817 4980 Acked Read 7 Sent Done Done false
15818 4981 Acked Write 7 Sent Done Done false
15819 4982 Acked Read 1 Sent Done Done false
15820 4983 Acked Write 4 Sent Done Done false
15821 4984 Acked Read 6 Sent Done Done false
15822 4985 Acked Flush 0 Sent Done Done false
15823 4986 Acked Read 1 Sent Done Done false
15824 4987 Acked Flush 0 Sent Done Done false
15825 4988 Acked Write 6 Sent Done Done false
15826 4989 Acked Write 4 Sent Done Done false
15827 4990 Acked Read 5 Sent Done Done false
15828 4991 Acked Read 10 Sent Done Done false
15829 4992 Acked Write 10 Sent Done Done false
15830 4993 Acked Flush 0 Sent Done Done false
15831 4994 Acked Read 7 Sent Done Done false
15832 4995 Acked Read 5 Sent Done Done false
15833 4996 Acked Write 5 Sent Done Done false
15834 4997 Acked Write 8 Sent Done Done false
15835 4998 Acked Read 7 Sent Done Done false
15836 4999 Acked Read 6 Sent Done Done false
15837 5000 Acked Read 4 Sent Done Done false
15838 5001 Acked Read 2 Sent Done Done false
15839 5002 Acked Write 4 Sent Done Done false
15840 5003 Acked Write 6 Sent Done Done false
15841 5004 Acked Read 7 Sent Done Done false
15842 5005 Acked Write 9 Sent Done Done false
15843 5006 Acked Read 8 Sent Done Done false
15844 5007 Acked Write 2 Sent Done Done false
15845 5008 Acked Read 9 Sent Done Done false
15846 5009 Acked Read 10 Sent Done Done false
15847 5010 Acked Flush 0 Sent Done Done false
15848 5011 Acked Read 1 Sent Done Done false
15849 5012 Acked Read 9 Sent Done Done false
15850 5013 Acked Read 2 Sent Done Done false
15851 5014 Acked Read 8 Sent Done Done false
15852 5015 Acked Write 6 Sent Done Done false
15853 5016 Acked Read 9 Sent Done Done false
15854 5017 Acked Write 5 Sent Done Done false
15855 5018 Acked Write 1 Sent Done Done false
15856 5019 Acked Write 4 Sent Done Done false
15857 5020 Acked Write 6 Sent Done Done false
15858 5021 Acked Read 6 Sent Done Done false
15859 5022 Acked Read 1 Sent Done Done false
15860 5023 Acked Write 6 Sent Done Done false
15861 5024 Acked Read 6 Sent Done Done false
15862 5025 Acked Read 8 Sent Done Done false
15863 5026 Acked Flush 0 Sent Done Done false
15864 5027 Acked Write 6 Sent Done Done false
15865 5028 Acked Read 1 Sent Done Done false
15866 5029 Acked Write 10 Sent Done Done false
15867 5030 Acked Read 10 Sent Done Done false
15868 5031 Acked Read 1 Sent Done Done false
15869 5032 Acked Read 2 Sent Done Done false
15870 5033 Acked Write 1 Sent Done Done false
15871 5034 Acked Read 2 Sent Done Done false
15872 5035 Acked Write 6 Sent Done Done false
15873 5036 Acked Read 6 Sent Done Done false
15874 5037 Acked Write 8 Sent Done Done false
15875 5038 Acked Read 8 Sent Done Done false
15876 5039 Acked Read 2 Sent Done Done false
15877 5040 Acked Write 9 Sent Done Done false
15878 5041 Acked Read 3 Sent Done Done false
15879 5042 Acked Read 6 Sent Done Done false
15880 5043 Acked Write 6 Sent Done Done false
15881 5044 Acked Read 1 Sent Done Done false
15882 5045 Acked Read 7 Sent Done Done false
15883 5046 Acked Read 8 Sent Done Done false
15884 5047 Acked Write 3 Sent Done Done false
15885 5048 Acked Write 10 Sent Done Done false
15886 5049 Acked Flush 0 Sent Done Done false
15887 5050 Acked Write 10 Sent Done Done false
15888 5051 Acked Write 10 Sent Done Done false
15889 5052 Acked Read 6 Sent Done Done false
15890 5053 Acked Read 8 Sent Done Done false
15891 5054 Acked Write 2 Sent Done Done false
15892 5055 Acked Write 5 Sent Done Done false
15893 5056 Acked Read 7 Sent Done Done false
15894 5057 Acked Flush 0 Sent Done Done false
15895 5058 Acked Write 2 Sent Done Done false
15896 5059 Acked Write 7 Sent Done Done false
15897 5060 Acked Read 10 Sent Done Done false
15898 5061 Acked Read 5 Sent Done Done false
15899 5062 Acked Read 10 Sent Done Done false
15900 5063 Acked Read 2 Sent Done Done false
15901 5064 Acked Read 2 Sent Done Done false
15902 5065 Acked Flush 0 Sent Done Done false
15903 5066 Acked Read 7 Sent Done Done false
15904 5067 Acked Write 6 Sent Done Done false
15905 5068 Acked Read 3 Sent Done Done false
15906 5069 Acked Write 10 Sent Done Done false
15907 5070 Acked Read 6 Sent Done Done false
15908 5071 Acked Flush 0 Sent Done Done false
15909 5072 Acked Read 7 Sent Done Done false
15910 5073 Acked Write 9 Sent Done Done false
15911 5074 Acked Write 4 Sent Done Done false
15912 5075 Acked Read 5 Sent Done Done false
15913 5076 Acked Flush 0 Sent Done Done false
15914 5077 Acked Read 6 Sent Done Done false
15915 5078 Acked Write 8 Sent Done Done false
15916 5079 Acked Flush 0 Sent Done Done false
15917 5080 Acked Write 2 Sent Done Done false
15918 5081 Acked Read 2 Sent Done Done false
15919 5082 Acked Write 6 Sent Done Done false
15920 5083 Acked Read 4 Sent Done Done false
15921 5084 Acked Read 9 Sent Done Done false
15922 5085 Acked Read 3 Sent Done Done false
15923 5086 Acked Read 7 Sent Done Done false
15924 5087 Acked Flush 0 Sent Done Done false
15925 5088 Acked Read 8 Sent Done Done false
15926 5089 Acked Read 9 Sent Done Done false
15927 5090 Acked Read 7 Sent Done Done false
15928 5091 Acked Write 2 Sent Done Done false
15929 5092 Acked Write 2 Sent Done Done false
15930 5093 Acked Write 2 Sent Done Done false
15931 5094 Acked Write 7 Sent Done Done false
15932 5095 Acked Read 3 Sent Done Done false
15933 5096 Acked Read 5 Sent Done Done false
15934 5097 Acked Write 7 Sent Done Done false
15935 5098 Acked Write 10 Sent Done Done false
15936 5099 Acked Read 6 Sent Done Done false
15937 5100 Acked Read 1 Sent Done Done false
15938 5101 Acked Read 3 Sent Done Done false
15939 5102 Acked Flush 0 Sent Done Done false
15940 5103 Acked Write 6 Sent Done Done false
15941 5104 Acked Write 5 Sent Done Done false
15942 5105 Acked Read 7 Sent Done Done false
15943 5106 Acked Write 10 Sent Done Done false
15944 5107 Acked Write 4 Sent Done Done false
15945 5108 Acked Write 7 Sent Done Done false
15946 5109 Acked Write 7 Sent Done Done false
15947 5110 Acked Flush 0 Sent Done Done false
15948 5111 Acked Write 1 Sent Done Done false
15949 5112 Acked Flush 0 Sent Done Done false
15950 5113 Acked Write 4 Sent Done Done false
15951 5114 Acked Read 8 Sent Done Done false
15952 5115 Acked Write 6 Sent Done Done false
15953 5116 Acked Write 4 Sent Done Done false
15954 5117 Acked Read 8 Sent Done Done false
15955 5118 Acked Read 5 Sent Done Done false
15956 5119 Acked Write 10 Sent Done Done false
15957 5120 Acked Read 9 Sent Done Done false
15958 5121 Acked Write 3 Sent Done Done false
15959 5122 Acked Read 4 Sent Done Done false
15960 5123 Acked Write 9 Sent Done Done false
15961 5124 Acked Write 8 Sent Done Done false
15962 5125 Acked Read 1 Sent Done Done false
15963 5126 Acked Read 1 Sent Done Done false
15964 5127 Acked Write 4 Sent Done Done false
15965 5128 Acked Read 6 Sent Done Done false
15966 5129 Acked Write 2 Sent Done Done false
15967 5130 Acked Write 5 Sent Done Done false
15968 5131 Acked Write 9 Sent Done Done false
15969 5132 Acked Read 8 Sent Done Done false
15970 5133 Acked Flush 0 Sent Done Done false
15971 5134 Acked Write 10 Sent Done Done false
15972 5135 Acked Write 6 Sent Done Done false
15973 5136 Acked Write 2 Sent Done Done false
15974 5137 Acked Read 7 Sent Done Done false
15975 5138 Acked Read 7 Sent Done Done false
15976 5139 Acked Read 5 Sent Done Done false
15977 5140 Acked Read 7 Sent Done Done false
15978 5141 Acked Read 7 Sent Done Done false
15979 5142 Acked Write 4 Sent Done Done false
15980 5143 Acked Write 8 Sent Done Done false
15981 5144 Acked Write 5 Sent Done Done false
15982 5145 Acked Write 5 Sent Done Done false
15983 5146 Acked Write 10 Sent Done Done false
15984 5147 Acked Flush 0 Sent Done Done false
15985 5148 Acked Write 10 Sent Done Done false
15986 5149 Acked Read 2 Sent Done Done false
15987 5150 Acked Read 6 Sent Done Done false
15988 5151 Acked Read 4 Sent Done Done false
15989 5152 Acked Read 10 Sent Done Done false
15990 5153 Acked Write 3 Sent Done Done false
15991 5154 Acked Write 3 Sent Done Done false
15992 5155 Acked Read 10 Sent Done Done false
15993 5156 Acked Read 2 Sent Done Done false
15994 5157 Acked Read 3 Sent Done Done false
15995 5158 Acked Read 3 Sent Done Done false
15996 5159 Acked Read 4 Sent Done Done false
15997 5160 Acked Read 2 Sent Done Done false
15998 5161 Acked Write 10 Sent Done Done false
15999 5162 Acked Read 5 Sent Done Done false
16000 5163 Acked Write 1 Sent Done Done false
16001 5164 Acked Write 6 Sent Done Done false
16002 5165 Acked Read 9 Sent Done Done false
16003 5166 Acked Write 2 Sent Done Done false
16004 5167 Acked Read 3 Sent Done Done false
16005 5168 Acked Read 6 Sent Done Done false
16006 5169 Acked Read 3 Sent Done Done false
16007 5170 Acked Read 6 Sent Done Done false
16008 5171 Acked Write 2 Sent Done Done false
16009 5172 Acked Write 10 Sent Done Done false
16010 5173 Acked Write 7 Sent Done Done false
16011 5174 Acked Write 2 Sent Done Done false
16012 5175 Acked Read 5 Sent Done Done false
16013 5176 Acked Write 8 Sent Done Done false
16014 5177 Acked Read 9 Sent Done Done false
16015 5178 Acked Read 4 Sent Done Done false
16016 5179 Acked Read 4 Sent Done Done false
16017 5180 Acked Read 9 Sent Done Done false
16018 5181 Acked Read 3 Sent Done Done false
16019 5182 Acked Flush 0 Sent Done Done false
16020 5183 Acked Read 10 Sent Done Done false
16021 5184 Acked Read 1 Sent Done Done false
16022 5185 Acked Write 1 Sent Done Done false
16023 5186 Acked Flush 0 Sent Done Done false
16024 5187 Acked Read 5 Sent Done Done false
16025 5188 Acked Write 2 Sent Done Done false
16026 5189 Acked Write 2 Sent Done Done false
16027 5190 Acked Read 6 Sent Done Done false
16028 5191 Acked Write 7 Sent Done Done false
16029 5192 Acked Read 8 Sent Done Done false
16030 5193 Acked Read 5 Sent Done Done false
16031 5194 Acked Read 2 Sent Done Done false
16032 5195 Acked Write 4 Sent Done Done false
16033 5196 Acked Read 5 Sent Done Done false
16034 5197 Acked Read 1 Sent Done Done false
16035 5198 Acked Read 9 Sent Done Done false
16036 5199 Acked Write 8 Sent Done Done false
16037 5200 Acked Read 3 Sent Done Done false
16038 5201 Acked Flush 0 Sent Done Done false
16039 5202 Acked Write 4 Sent Done Done false
16040 5203 Acked Write 7 Sent Done Done false
16041 5204 Acked Read 4 Sent Done Done false
16042 5205 Acked Write 6 Sent Done Done false
16043 5206 Acked Read 5 Sent Done Done false
16044 5207 Acked Read 8 Sent Done Done false
16045 5208 Acked Flush 0 Sent Done Done false
16046 5209 Acked Read 9 Sent Done Done false
16047 5210 Acked Read 10 Sent Done Done false
16048 5211 Acked Write 4 Sent Done Done false
16049 5212 Acked Read 2 Sent Done Done false
16050 5213 Acked Write 8 Sent Done Done false
16051 5214 Acked Write 9 Sent Done Done false
16052 5215 Acked Read 7 Sent Done Done false
16053 5216 Acked Read 9 Sent Done Done false
16054 5217 Acked Write 4 Sent Done Done false
16055 5218 Acked Read 10 Sent Done Done false
16056 5219 Acked Write 5 Sent Done Done false
16057 5220 Acked Flush 0 Sent Done Done false
16058 5221 Acked Write 6 Sent Done Done false
16059 5222 Acked Flush 0 Sent Done Done false
16060 5223 Acked Write 9 Sent Done Done false
16061 5224 Acked Read 1 Sent Done Done false
16062 5225 Acked Flush 0 Sent Done Done false
16063 STATES DS:0 DS:1 DS:2 TOTAL
16064 Sent 301 0 0 301
16065 Done 0 301 301 602
16066 Skipped 0 0 0 0
16067 Error 0 0 0 0
16068 Last Flush: 4924 5225 5225
16069 Downstairs last five completed: 4924 4923 4922 4921 4920
16070 Upstairs last five completed: 5225 5224 5223 5222 5221
16071 CLIENT: Up:0 ds:301 act:2
16072 [8810] got start action
16073 [8810] I am going to restart
16074 Starting downstairs at port 8810
16075 Make output file at "/tmp/dsc/downstairs-8810.txt"
16076 [8810][0] reports Starting
16077 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1216)
16078 [8810][0] reports Running
16079 ----------------------------------------------------------------
16080 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16081 States: Offline Active Active
16082 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16083 4925 Acked Read 7 Sent Done Done false
16084 4926 Acked Flush 0 Sent Done Done false
16085 4927 Acked Write 9 Sent Done Done false
16086 4928 Acked Write 9 Sent Done Done false
16087 4929 Acked Read 7 Sent Done Done false
16088 4930 Acked Flush 0 Sent Done Done false
16089 4931 Acked Write 9 Sent Done Done false
16090 4932 Acked Write 5 Sent Done Done false
16091 4933 Acked Write 4 Sent Done Done false
16092 4934 Acked Read 9 Sent Done Done false
16093 4935 Acked Read 4 Sent Done Done false
16094 4936 Acked Read 10 Sent Done Done false
16095 4937 Acked Read 2 Sent Done Done false
16096 4938 Acked Read 6 Sent Done Done false
16097 4939 Acked Read 6 Sent Done Done false
16098 4940 Acked Flush 0 Sent Done Done false
16099 4941 Acked Read 1 Sent Done Done false
16100 4942 Acked Write 3 Sent Done Done false
16101 4943 Acked Write 2 Sent Done Done false
16102 4944 Acked Write 6 Sent Done Done false
16103 4945 Acked Read 10 Sent Done Done false
16104 4946 Acked Write 9 Sent Done Done false
16105 4947 Acked Write 8 Sent Done Done false
16106 4948 Acked Read 7 Sent Done Done false
16107 4949 Acked Flush 0 Sent Done Done false
16108 4950 Acked Write 5 Sent Done Done false
16109 4951 Acked Write 10 Sent Done Done false
16110 4952 Acked Write 10 Sent Done Done false
16111 4953 Acked Read 8 Sent Done Done false
16112 4954 Acked Write 6 Sent Done Done false
16113 4955 Acked Write 2 Sent Done Done false
16114 4956 Acked Write 2 Sent Done Done false
16115 4957 Acked Read 7 Sent Done Done false
16116 4958 Acked Read 5 Sent Done Done false
16117 4959 Acked Write 6 Sent Done Done false
16118 4960 Acked Read 3 Sent Done Done false
16119 4961 Acked Read 6 Sent Done Done false
16120 4962 Acked Write 9 Sent Done Done false
16121 4963 Acked Write 2 Sent Done Done false
16122 4964 Acked Write 3 Sent Done Done false
16123 4965 Acked Read 5 Sent Done Done false
16124 4966 Acked Read 8 Sent Done Done false
16125 4967 Acked Write 1 Sent Done Done false
16126 4968 Acked Read 2 Sent Done Done false
16127 4969 Acked Write 9 Sent Done Done false
16128 4970 Acked Write 9 Sent Done Done false
16129 4971 Acked Read 9 Sent Done Done false
16130 4972 Acked Read 7 Sent Done Done false
16131 4973 Acked Flush 0 Sent Done Done false
16132 4974 Acked Read 9 Sent Done Done false
16133 4975 Acked Read 4 Sent Done Done false
16134 4976 Acked Read 7 Sent Done Done false
16135 4977 Acked Read 7 Sent Done Done false
16136 4978 Acked Write 9 Sent Done Done false
16137 4979 Acked Write 10 Sent Done Done false
16138 4980 Acked Read 7 Sent Done Done false
16139 4981 Acked Write 7 Sent Done Done false
16140 4982 Acked Read 1 Sent Done Done false
16141 4983 Acked Write 4 Sent Done Done false
16142 4984 Acked Read 6 Sent Done Done false
16143 4985 Acked Flush 0 Sent Done Done false
16144 4986 Acked Read 1 Sent Done Done false
16145 4987 Acked Flush 0 Sent Done Done false
16146 4988 Acked Write 6 Sent Done Done false
16147 4989 Acked Write 4 Sent Done Done false
16148 4990 Acked Read 5 Sent Done Done false
16149 4991 Acked Read 10 Sent Done Done false
16150 4992 Acked Write 10 Sent Done Done false
16151 4993 Acked Flush 0 Sent Done Done false
16152 4994 Acked Read 7 Sent Done Done false
16153 4995 Acked Read 5 Sent Done Done false
16154 4996 Acked Write 5 Sent Done Done false
16155 4997 Acked Write 8 Sent Done Done false
16156 4998 Acked Read 7 Sent Done Done false
16157 4999 Acked Read 6 Sent Done Done false
16158 5000 Acked Read 4 Sent Done Done false
16159 5001 Acked Read 2 Sent Done Done false
16160 5002 Acked Write 4 Sent Done Done false
16161 5003 Acked Write 6 Sent Done Done false
16162 5004 Acked Read 7 Sent Done Done false
16163 5005 Acked Write 9 Sent Done Done false
16164 5006 Acked Read 8 Sent Done Done false
16165 5007 Acked Write 2 Sent Done Done false
16166 5008 Acked Read 9 Sent Done Done false
16167 5009 Acked Read 10 Sent Done Done false
16168 5010 Acked Flush 0 Sent Done Done false
16169 5011 Acked Read 1 Sent Done Done false
16170 5012 Acked Read 9 Sent Done Done false
16171 5013 Acked Read 2 Sent Done Done false
16172 5014 Acked Read 8 Sent Done Done false
16173 5015 Acked Write 6 Sent Done Done false
16174 5016 Acked Read 9 Sent Done Done false
16175 5017 Acked Write 5 Sent Done Done false
16176 5018 Acked Write 1 Sent Done Done false
16177 5019 Acked Write 4 Sent Done Done false
16178 5020 Acked Write 6 Sent Done Done false
16179 5021 Acked Read 6 Sent Done Done false
16180 5022 Acked Read 1 Sent Done Done false
16181 5023 Acked Write 6 Sent Done Done false
16182 5024 Acked Read 6 Sent Done Done false
16183 5025 Acked Read 8 Sent Done Done false
16184 5026 Acked Flush 0 Sent Done Done false
16185 5027 Acked Write 6 Sent Done Done false
16186 5028 Acked Read 1 Sent Done Done false
16187 5029 Acked Write 10 Sent Done Done false
16188 5030 Acked Read 10 Sent Done Done false
16189 5031 Acked Read 1 Sent Done Done false
16190 5032 Acked Read 2 Sent Done Done false
16191 5033 Acked Write 1 Sent Done Done false
16192 5034 Acked Read 2 Sent Done Done false
16193 5035 Acked Write 6 Sent Done Done false
16194 5036 Acked Read 6 Sent Done Done false
16195 5037 Acked Write 8 Sent Done Done false
16196 5038 Acked Read 8 Sent Done Done false
16197 5039 Acked Read 2 Sent Done Done false
16198 5040 Acked Write 9 Sent Done Done false
16199 5041 Acked Read 3 Sent Done Done false
16200 5042 Acked Read 6 Sent Done Done false
16201 5043 Acked Write 6 Sent Done Done false
16202 5044 Acked Read 1 Sent Done Done false
16203 5045 Acked Read 7 Sent Done Done false
16204 5046 Acked Read 8 Sent Done Done false
16205 5047 Acked Write 3 Sent Done Done false
16206 5048 Acked Write 10 Sent Done Done false
16207 5049 Acked Flush 0 Sent Done Done false
16208 5050 Acked Write 10 Sent Done Done false
16209 5051 Acked Write 10 Sent Done Done false
16210 5052 Acked Read 6 Sent Done Done false
16211 5053 Acked Read 8 Sent Done Done false
16212 5054 Acked Write 2 Sent Done Done false
16213 5055 Acked Write 5 Sent Done Done false
16214 5056 Acked Read 7 Sent Done Done false
16215 5057 Acked Flush 0 Sent Done Done false
16216 5058 Acked Write 2 Sent Done Done false
16217 5059 Acked Write 7 Sent Done Done false
16218 5060 Acked Read 10 Sent Done Done false
16219 5061 Acked Read 5 Sent Done Done false
16220 5062 Acked Read 10 Sent Done Done false
16221 5063 Acked Read 2 Sent Done Done false
16222 5064 Acked Read 2 Sent Done Done false
16223 5065 Acked Flush 0 Sent Done Done false
16224 5066 Acked Read 7 Sent Done Done false
16225 5067 Acked Write 6 Sent Done Done false
16226 5068 Acked Read 3 Sent Done Done false
16227 5069 Acked Write 10 Sent Done Done false
16228 5070 Acked Read 6 Sent Done Done false
16229 5071 Acked Flush 0 Sent Done Done false
16230 5072 Acked Read 7 Sent Done Done false
16231 5073 Acked Write 9 Sent Done Done false
16232 5074 Acked Write 4 Sent Done Done false
16233 5075 Acked Read 5 Sent Done Done false
16234 5076 Acked Flush 0 Sent Done Done false
16235 5077 Acked Read 6 Sent Done Done false
16236 5078 Acked Write 8 Sent Done Done false
16237 5079 Acked Flush 0 Sent Done Done false
16238 5080 Acked Write 2 Sent Done Done false
16239 5081 Acked Read 2 Sent Done Done false
16240 5082 Acked Write 6 Sent Done Done false
16241 5083 Acked Read 4 Sent Done Done false
16242 5084 Acked Read 9 Sent Done Done false
16243 5085 Acked Read 3 Sent Done Done false
16244 5086 Acked Read 7 Sent Done Done false
16245 5087 Acked Flush 0 Sent Done Done false
16246 5088 Acked Read 8 Sent Done Done false
16247 5089 Acked Read 9 Sent Done Done false
16248 5090 Acked Read 7 Sent Done Done false
16249 5091 Acked Write 2 Sent Done Done false
16250 5092 Acked Write 2 Sent Done Done false
16251 5093 Acked Write 2 Sent Done Done false
16252 5094 Acked Write 7 Sent Done Done false
16253 5095 Acked Read 3 Sent Done Done false
16254 5096 Acked Read 5 Sent Done Done false
16255 5097 Acked Write 7 Sent Done Done false
16256 5098 Acked Write 10 Sent Done Done false
16257 5099 Acked Read 6 Sent Done Done false
16258 5100 Acked Read 1 Sent Done Done false
16259 5101 Acked Read 3 Sent Done Done false
16260 5102 Acked Flush 0 Sent Done Done false
16261 5103 Acked Write 6 Sent Done Done false
16262 5104 Acked Write 5 Sent Done Done false
16263 5105 Acked Read 7 Sent Done Done false
16264 5106 Acked Write 10 Sent Done Done false
16265 5107 Acked Write 4 Sent Done Done false
16266 5108 Acked Write 7 Sent Done Done false
16267 5109 Acked Write 7 Sent Done Done false
16268 5110 Acked Flush 0 Sent Done Done false
16269 5111 Acked Write 1 Sent Done Done false
16270 5112 Acked Flush 0 Sent Done Done false
16271 5113 Acked Write 4 Sent Done Done false
16272 5114 Acked Read 8 Sent Done Done false
16273 5115 Acked Write 6 Sent Done Done false
16274 5116 Acked Write 4 Sent Done Done false
16275 5117 Acked Read 8 Sent Done Done false
16276 5118 Acked Read 5 Sent Done Done false
16277 5119 Acked Write 10 Sent Done Done false
16278 5120 Acked Read 9 Sent Done Done false
16279 5121 Acked Write 3 Sent Done Done false
16280 5122 Acked Read 4 Sent Done Done false
16281 5123 Acked Write 9 Sent Done Done false
16282 5124 Acked Write 8 Sent Done Done false
16283 5125 Acked Read 1 Sent Done Done false
16284 5126 Acked Read 1 Sent Done Done false
16285 5127 Acked Write 4 Sent Done Done false
16286 5128 Acked Read 6 Sent Done Done false
16287 5129 Acked Write 2 Sent Done Done false
16288 5130 Acked Write 5 Sent Done Done false
16289 5131 Acked Write 9 Sent Done Done false
16290 5132 Acked Read 8 Sent Done Done false
16291 5133 Acked Flush 0 Sent Done Done false
16292 5134 Acked Write 10 Sent Done Done false
16293 5135 Acked Write 6 Sent Done Done false
16294 5136 Acked Write 2 Sent Done Done false
16295 5137 Acked Read 7 Sent Done Done false
16296 5138 Acked Read 7 Sent Done Done false
16297 5139 Acked Read 5 Sent Done Done false
16298 5140 Acked Read 7 Sent Done Done false
16299 5141 Acked Read 7 Sent Done Done false
16300 5142 Acked Write 4 Sent Done Done false
16301 5143 Acked Write 8 Sent Done Done false
16302 5144 Acked Write 5 Sent Done Done false
16303 5145 Acked Write 5 Sent Done Done false
16304 5146 Acked Write 10 Sent Done Done false
16305 5147 Acked Flush 0 Sent Done Done false
16306 5148 Acked Write 10 Sent Done Done false
16307 5149 Acked Read 2 Sent Done Done false
16308 5150 Acked Read 6 Sent Done Done false
16309 5151 Acked Read 4 Sent Done Done false
16310 5152 Acked Read 10 Sent Done Done false
16311 5153 Acked Write 3 Sent Done Done false
16312 5154 Acked Write 3 Sent Done Done false
16313 5155 Acked Read 10 Sent Done Done false
16314 5156 Acked Read 2 Sent Done Done false
16315 5157 Acked Read 3 Sent Done Done false
16316 5158 Acked Read 3 Sent Done Done false
16317 5159 Acked Read 4 Sent Done Done false
16318 5160 Acked Read 2 Sent Done Done false
16319 5161 Acked Write 10 Sent Done Done false
16320 5162 Acked Read 5 Sent Done Done false
16321 5163 Acked Write 1 Sent Done Done false
16322 5164 Acked Write 6 Sent Done Done false
16323 5165 Acked Read 9 Sent Done Done false
16324 5166 Acked Write 2 Sent Done Done false
16325 5167 Acked Read 3 Sent Done Done false
16326 5168 Acked Read 6 Sent Done Done false
16327 5169 Acked Read 3 Sent Done Done false
16328 5170 Acked Read 6 Sent Done Done false
16329 5171 Acked Write 2 Sent Done Done false
16330 5172 Acked Write 10 Sent Done Done false
16331 5173 Acked Write 7 Sent Done Done false
16332 5174 Acked Write 2 Sent Done Done false
16333 5175 Acked Read 5 Sent Done Done false
16334 5176 Acked Write 8 Sent Done Done false
16335 5177 Acked Read 9 Sent Done Done false
16336 5178 Acked Read 4 Sent Done Done false
16337 5179 Acked Read 4 Sent Done Done false
16338 5180 Acked Read 9 Sent Done Done false
16339 5181 Acked Read 3 Sent Done Done false
16340 5182 Acked Flush 0 Sent Done Done false
16341 5183 Acked Read 10 Sent Done Done false
16342 5184 Acked Read 1 Sent Done Done false
16343 5185 Acked Write 1 Sent Done Done false
16344 5186 Acked Flush 0 Sent Done Done false
16345 5187 Acked Read 5 Sent Done Done false
16346 5188 Acked Write 2 Sent Done Done false
16347 5189 Acked Write 2 Sent Done Done false
16348 5190 Acked Read 6 Sent Done Done false
16349 5191 Acked Write 7 Sent Done Done false
16350 5192 Acked Read 8 Sent Done Done false
16351 5193 Acked Read 5 Sent Done Done false
16352 5194 Acked Read 2 Sent Done Done false
16353 5195 Acked Write 4 Sent Done Done false
16354 5196 Acked Read 5 Sent Done Done false
16355 5197 Acked Read 1 Sent Done Done false
16356 5198 Acked Read 9 Sent Done Done false
16357 5199 Acked Write 8 Sent Done Done false
16358 5200 Acked Read 3 Sent Done Done false
16359 5201 Acked Flush 0 Sent Done Done false
16360 5202 Acked Write 4 Sent Done Done false
16361 5203 Acked Write 7 Sent Done Done false
16362 5204 Acked Read 4 Sent Done Done false
16363 5205 Acked Write 6 Sent Done Done false
16364 5206 Acked Read 5 Sent Done Done false
16365 5207 Acked Read 8 Sent Done Done false
16366 5208 Acked Flush 0 Sent Done Done false
16367 5209 Acked Read 9 Sent Done Done false
16368 5210 Acked Read 10 Sent Done Done false
16369 5211 Acked Write 4 Sent Done Done false
16370 5212 Acked Read 2 Sent Done Done false
16371 5213 Acked Write 8 Sent Done Done false
16372 5214 Acked Write 9 Sent Done Done false
16373 5215 Acked Read 7 Sent Done Done false
16374 5216 Acked Read 9 Sent Done Done false
16375 5217 Acked Write 4 Sent Done Done false
16376 5218 Acked Read 10 Sent Done Done false
16377 5219 Acked Write 5 Sent Done Done false
16378 5220 Acked Flush 0 Sent Done Done false
16379 5221 Acked Write 6 Sent Done Done false
16380 5222 Acked Flush 0 Sent Done Done false
16381 5223 Acked Write 9 Sent Done Done false
16382 5224 Acked Read 1 Sent Done Done false
16383 5225 Acked Flush 0 Sent Done Done false
16384 STATES DS:0 DS:1 DS:2 TOTAL
16385 Sent 301 0 0 301
16386 Done 0 301 301 602
16387 Skipped 0 0 0 0
16388 Error 0 0 0 0
16389 Last Flush: 4924 5225 5225
16390 Downstairs last five completed: 4924 4923 4922 4921 4920
16391 Upstairs last five completed: 5225 5224 5223 5222 5221
16392 CLIENT: Up:0 ds:301 act:2
163932024-11-19T17:11:24.917ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
163942024-11-19T17:11:24.917ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55076) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
163952024-11-19T17:11:24.918ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
163962024-11-19T17:11:24.919ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
163972024-11-19T17:11:24.919ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
163982024-11-19T17:11:24.919ZINFOcrucible: send last flush ID to this DS: 4924 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
163992024-11-19T17:11:24.919ZINFOcrucible: Replied this last flush ID: 4924 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
164002024-11-19T17:11:24.919ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
164012024-11-19T17:11:24.919ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
164022024-11-19T17:11:24.919ZINFOcrucible: [0] client re-new 301 jobs since flush 4924 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
164032024-11-19T17:11:24.919ZINFOcrucible: [0] Replayed 301 jobs since flush: 4924 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
16404 ----------------------------------------------------------------
16405 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16406 Downstairs last five completed: 5225 5224 5223 5222 5221
16407 Upstairs last five completed: 5225 5224 5223 5222 5221
16408 CLIENT: Up:0 ds:0 act:3
16409 Replay: All jobs finished
16410 Main task has work to do, go find it
16411 got dsc Stop(1)
16412 stop 1
16413 [8820] Got stop action so:false kr:false
16414 Nov 19 17:11:27.165 INFO request completed, latency_us: 329, response_code: 204, uri: /stop/cid/1, method: POST, req_id: bd634936-efb3-4709-a550-fedaf047c15d, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
16415 Nov 19 17:11:27.166 INFO request completed, latency_us: 108, response_code: 200, uri: /state/cid/1, method: GET, req_id: 07dc5e25-d89e-4e3a-acb9-60565c5004ed, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
164162024-11-19T17:11:27.166ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
164172024-11-19T17:11:27.166ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
164182024-11-19T17:11:27.166ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
164192024-11-19T17:11:27.166ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
164202024-11-19T17:11:27.166ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
164212024-11-19T17:11:27.166ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
16422 [8820] Exited with: ExitStatus(unix_wait_status(9))
16423 [8820][1] reports Exit
16424 Nov 19 17:11:31.168 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0c199b6d-ed2c-401a-ba64-bc09fd873747, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
16425 001/300 Read block 2221 len 12288
16426 002/300 Read block 4460 len 40960
16427 003/300 Write block 547 len 32768 data: 3 3 2 2 2 3 3 3
16428 004/300 Read block 4380 len 12288
16429 005/300 Read block 4415 len 28672
16430 006/300 Read block 1445 len 36864
16431 007/300 Write block 298 len 32768 data: 4 4 3 3 3 4 4 4
16432 008/300 Write block 2934 len 16384 data: 6 5 7 7
16433 009/300 Read block 844 len 24576
16434 010/300 Read block 4088 len 12288
16435 011/300 Flush
16436 012/300 Read block 4002 len 20480
16437 013/300 Write block 405 len 16384 data: 2 2 2 2
16438 014/300 Write block 3821 len 16384 data: 4 6 6 6
16439 015/300 Read block 3398 len 24576
16440 016/300 Read block 4881 len 28672
16441 017/300 Read block 3397 len 16384
16442 018/300 Flush
16443 019/300 Write block 2982 len 12288 data: 4 4 4
16444 020/300 Write block 2238 len 36864 data: 2 2 3 4 5 6 5 5 5
16445 021/300 Read block 4585 len 32768
16446 022/300 Read block 2042 len 12288
16447 023/300 Write block 1573 len 12288 data: 2 2 2
16448 024/300 Read block 799 len 24576
16449 025/300 Read block 1523 len 36864
16450 026/300 Write block 2231 len 20480 data: 4 5 4 4 4
16451 027/300 Write block 4156 len 8192 data: 4 5
16452 028/300 Read block 1617 len 16384
16453 029/300 Write block 267 len 24576 data: 3 3 3 3 3 2
16454 030/300 Write block 3619 len 36864 data: 5 4 5 5 4 4 3 3 3
16455 031/300 Read block 170 len 28672
16456 032/300 Write block 1279 len 20480 data: 4 4 5 5 5
16457 033/300 Read block 1404 len 24576
16458 034/300 Read block 4132 len 24576
16459 035/300 Read block 934 len 12288
16460 036/300 Read block 3760 len 8192
16461 037/300 Write block 3435 len 40960 data: 3 3 4 4 5 5 4 4 5 5
16462 038/300 Write block 4400 len 4096 data: 3
16463 039/300 Write block 2362 len 4096 data: 2
16464 040/300 Read block 4341 len 36864
16465 041/300 Write block 4951 len 40960 data: 3 3 3 3 3 3 4 4 3 3
16466 042/300 Read block 2274 len 4096
16467 043/300 Read block 2736 len 28672
16468 044/300 Flush
16469 045/300 Read block 3002 len 4096
16470 046/300 Write block 2926 len 4096 data: 3
16471 047/300 Read block 1604 len 12288
16472 048/300 Read block 2247 len 20480
16473 049/300 Write block 2358 len 36864 data: 3 3 3 3 3 2 5 5 5
16474 050/300 Read block 3679 len 40960
16475 051/300 Read block 851 len 28672
16476 052/300 Read block 2412 len 4096
16477 053/300 Write block 4970 len 24576 data: 3 3 3 4 3 3
16478 054/300 Write block 4236 len 20480 data: 2 2 2 2 2
16479 055/300 Write block 2131 len 28672 data: 4 3 3 3 3 3 3
16480 056/300 Read block 1417 len 20480
16481 057/300 Read block 4910 len 36864
16482 058/300 Write block 1168 len 16384 data: 4 5 4 5
16483 059/300 Write block 2293 len 32768 data: 3 3 3 5 5 6 6 7
16484 060/300 Write block 3454 len 28672 data: 4 5 6 5 4 4 3
16485 061/300 Read block 1856 len 4096
16486 062/300 Read block 4992 len 32768
16487 063/300 Flush
16488 064/300 Write block 2667 len 40960 data: 5 3 2 2 2 2 3 3 3 3
16489 065/300 Read block 4753 len 12288
16490 066/300 Read block 1049 len 40960
16491 067/300 Read block 3757 len 8192
16492 068/300 Read block 1168 len 32768
16493 069/300 Write block 3854 len 32768 data: 2 3 3 3 3 3 3 3
16494 070/300 Read block 4985 len 4096
16495 071/300 Read block 572 len 20480
16496 072/300 Flush
16497 073/300 Read block 2656 len 4096
16498 074/300 Write block 79 len 24576 data: 3 3 3 3 3 3
16499 075/300 Flush
16500 076/300 Write block 2777 len 8192 data: 4 5
16501 077/300 Read block 1909 len 4096
16502 078/300 Read block 2612 len 28672
16503 079/300 Write block 4187 len 40960 data: 4 5 6 4 5 6 4 4 6 6
16504 080/300 Read block 3434 len 36864
16505 081/300 Read block 635 len 16384
16506 082/300 Read block 3514 len 32768
16507 083/300 Write block 3896 len 40960 data: 4 4 4 4 4 3 3 3 4 3
16508 084/300 Write block 1862 len 16384 data: 3 3 3 3
16509 085/300 Read block 1354 len 40960
16510 086/300 Write block 2644 len 16384 data: 3 3 5 4
16511 087/300 Write block 149 len 40960 data: 4 6 4 5 5 5 5 5 5 5
16512 088/300 Read block 2198 len 40960
16513 089/300 Write block 1104 len 4096 data: 3
16514 090/300 Read block 2917 len 28672
16515 091/300 Read block 1910 len 40960
16516 092/300 Read block 4857 len 12288
16517 093/300 Read block 2436 len 20480
16518 094/300 Read block 2311 len 36864
16519 095/300 Read block 3201 len 36864
16520 096/300 Read block 2654 len 20480
16521 097/300 Read block 4172 len 16384
16522 098/300 Read block 1469 len 12288
16523 099/300 Read block 3066 len 40960
16524 100/300 Flush
16525 101/300 Read block 3112 len 16384
16526 102/300 Write block 531 len 36864 data: 3 2 2 2 4 4 4 4 4
16527 103/300 Read block 4922 len 4096
16528 104/300 Write block 4468 len 40960 data: 2 2 2 3 4 5 6 6 5 4
16529 105/300 Read block 4457 len 20480
16530 106/300 Read block 2443 len 8192
16531 107/300 Write block 3362 len 8192 data: 3 4
16532 108/300 Flush
16533 109/300 Read block 908 len 8192
16534 110/300 Read block 2810 len 32768
16535 111/300 Write block 2475 len 40960 data: 2 2 2 2 2 2 2 2 3 4
16536 112/300 Read block 2660 len 8192
16537 113/300 Read block 4389 len 20480
16538 114/300 Read block 922 len 32768
16539 115/300 Read block 1857 len 12288
16540 116/300 Read block 3181 len 40960
16541 117/300 Read block 1818 len 24576
16542 118/300 Read block 4014 len 24576
16543 119/300 Read block 1006 len 8192
16544 120/300 Write block 1470 len 28672 data: 4 4 4 4 5 4 3
16545 121/300 Read block 1730 len 20480
16546 122/300 Write block 40 len 32768 data: 3 3 4 4 3 3 3 4
16547 123/300 Write block 2262 len 16384 data: 3 4 4 4
16548 124/300 Read block 1825 len 20480
16549 125/300 Write block 3041 len 16384 data: 2 2 4 4
16550 126/300 Write block 1575 len 4096 data: 3
16551 127/300 Write block 4458 len 36864 data: 3 4 3 3 4 5 4 4 4
16552 128/300 Read block 2038 len 36864
16553 129/300 Read block 1756 len 36864
16554 130/300 Read block 1917 len 28672
16555 131/300 Read block 4549 len 16384
16556 132/300 Read block 3446 len 36864
16557 133/300 Read block 1624 len 20480
16558 134/300 Write block 3951 len 24576 data: 5 5 4 4 4 4
16559 135/300 Read block 2094 len 40960
16560 136/300 Read block 4535 len 12288
16561 137/300 Write block 1467 len 12288 data: 5 4 4
16562 138/300 Read block 4971 len 32768
16563 139/300 Write block 703 len 4096 data: 4
16564 140/300 Write block 1944 len 12288 data: 4 3 3
16565 141/300 Read block 2350 len 8192
16566 142/300 Read block 3168 len 32768
16567 143/300 Write block 3026 len 8192 data: 4 5
16568 144/300 Read block 4987 len 32768
16569 145/300 Read block 1807 len 20480
16570 146/300 Read block 2273 len 24576
16571 147/300 Read block 3059 len 12288
16572 148/300 Read block 28 len 20480
16573 149/300 Read block 2101 len 24576
16574 150/300 Read block 4586 len 36864
16575 151/300 Read block 1331 len 32768
16576 152/300 Read block 3975 len 8192
16577 153/300 Read block 3902 len 28672
16578 154/300 Flush
16579 155/300 Read block 4562 len 4096
16580 156/300 Read block 4555 len 4096
16581 157/300 Write block 138 len 32768 data: 7 9 7 5 5 5 4 3
16582 158/300 Write block 3855 len 36864 data: 4 4 4 4 4 4 4 6 6
16583 159/300 Write block 3858 len 20480 data: 5 5 5 5 7
16584 160/300 Read block 3947 len 4096
16585 161/300 Flush
16586 162/300 Read block 4231 len 32768
16587 163/300 Read block 1286 len 24576
16588 164/300 Write block 2148 len 36864 data: 5 6 6 5 4 4 6 5 5
16589 165/300 Read block 3122 len 40960
16590 166/300 Write block 1892 len 36864 data: 4 4 3 2 2 2 3 3 3
16591 167/300 Flush
16592 168/300 Read block 2437 len 40960
16593 169/300 Flush
16594 170/300 Read block 77 len 12288
16595 171/300 Write block 1615 len 12288 data: 8 7 7
16596 172/300 Read block 3214 len 28672
16597 173/300 Read block 2968 len 24576
16598 174/300 Read block 637 len 8192
16599 175/300 Read block 2218 len 28672
16600 176/300 Write block 3019 len 20480 data: 4 4 4 4 2
16601 177/300 Write block 2479 len 24576 data: 3 3 3 3 4 5
16602 178/300 Read block 3333 len 8192
16603 179/300 Read block 1349 len 4096
16604 180/300 Write block 2314 len 16384 data: 4 4 4 4
16605 181/300 Flush
16606 182/300 Write block 510 len 20480 data: 3 3 3 2 3
16607 183/300 Read block 1570 len 16384
16608 184/300 Read block 672 len 20480
16609 185/300 Read block 4353 len 12288
16610 186/300 Read block 2672 len 28672
16611 187/300 Write block 1597 len 36864 data: 2 3 5 5 5 4 4 5 5
16612 188/300 Write block 2501 len 16384 data: 4 3 4 5
16613 189/300 Write block 1462 len 36864 data: 2 2 3 3 4 6 5 5 5
16614 190/300 Read block 1676 len 32768
16615 191/300 Write block 2901 len 16384 data: 3 3 2 3
16616 192/300 Read block 3608 len 16384
16617 193/300 Write block 329 len 4096 data: 3
16618 194/300 Read block 2636 len 40960
16619 195/300 Read block 2864 len 40960
16620 196/300 Read block 1904 len 20480
16621 197/300 Write block 4933 len 12288 data: 8 6 4
16622 198/300 Write block 2249 len 24576 data: 4 4 4 4 4 6
16623 199/300 Write block 3594 len 16384 data: 6 6 6 6
16624 200/300 Read block 339 len 20480
16625 201/300 Write block 1979 len 24576 data: 5 5 4 3 3 2
16626 202/300 Read block 1240 len 40960
16627 203/300 Read block 530 len 28672
16628 204/300 Write block 621 len 40960 data: 3 3 3 4 4 4 4 4 4 3
16629 205/300 Read block 2113 len 20480
16630 206/300 Write block 3776 len 32768 data: 4 4 4 4 4 3 3 3
16631 207/300 Write block 4223 len 8192 data: 3 3
16632 208/300 Flush
16633 209/300 Write block 1927 len 12288 data: 4 4 5
16634 210/300 Read block 2582 len 12288
16635 211/300 Read block 3509 len 16384
16636 212/300 Write block 750 len 20480 data: 4 4 3 3 5
16637 213/300 Write block 2035 len 16384 data: 3 4 5 5
16638 214/300 Flush
16639 215/300 Read block 2008 len 16384
16640 216/300 Write block 3989 len 16384 data: 5 5 5 5
16641 217/300 Read block 2153 len 12288
16642 218/300 Flush
16643 219/300 Read block 2317 len 24576
16644 220/300 Write block 2559 len 8192 data: 3 6
16645 221/300 Write block 1111 len 12288 data: 3 3 3
16646 222/300 Read block 441 len 40960
16647 223/300 Read block 2769 len 16384
16648 224/300 Write block 2098 len 16384 data: 4 5 4 4
16649 225/300 Read block 4595 len 40960
16650 226/300 Read block 4881 len 8192
16651 227/300 Read block 2231 len 12288
16652 228/300 Write block 1609 len 36864 data: 7 8 9 8 8 8 9 8 8
16653 229/300 Read block 2715 len 32768
16654 230/300 Read block 4175 len 4096
16655 231/300 Write block 3729 len 16384 data: 7 6 5 5
16656 232/300 Read block 1193 len 8192
16657 233/300 Read block 442 len 40960
16658 234/300 Flush
16659 235/300 Read block 680 len 8192
16660 236/300 Flush
16661 237/300 Write block 959 len 20480 data: 2 2 2 2 2
16662 238/300 Read block 3676 len 8192
16663 239/300 Read block 2410 len 4096
16664 240/300 Write block 4431 len 16384 data: 4 5 5 5
16665 241/300 Write block 4317 len 4096 data: 3
16666 242/300 Read block 2186 len 4096
16667 243/300 Read block 4274 len 32768
16668 244/300 Flush
16669 245/300 Read block 185 len 32768
16670 246/300 Read block 3162 len 8192
16671 247/300 Write block 1948 len 4096 data: 3
16672 248/300 Write block 4892 len 8192 data: 5 5
16673 249/300 Flush
16674 250/300 Read block 2865 len 40960
16675 251/300 Write block 1594 len 16384 data: 2 2 2 3
16676 252/300 Write block 3840 len 32768 data: 3 4 3 4 4 3 3 3
16677 253/300 Read block 217 len 32768
16678 254/300 Read block 171 len 32768
16679 255/300 Read block 4706 len 4096
16680 256/300 Read block 851 len 16384
16681 257/300 Read block 1806 len 16384
16682 258/300 Write block 59 len 24576 data: 4 5 5 5 5 5
16683 259/300 Write block 488 len 24576 data: 6 6 6 7 7 5
16684 260/300 Read block 1618 len 40960
16685 261/300 Write block 4479 len 24576 data: 5 5 4 3 4 5
16686 262/300 Read block 3897 len 4096
16687 263/300 Read block 4266 len 16384
16688 264/300 Read block 4428 len 12288
16689 265/300 Read block 3043 len 36864
16690 266/300 Write block 3711 len 16384 data: 4 4 3 4
16691 267/300 Write block 2931 len 8192 data: 7 6
16692 268/300 Write block 4137 len 40960 data: 4 4 4 3 3 2 2 2 3 3
16693 269/300 Write block 3479 len 4096 data: 5
16694 270/300 Read block 2068 len 36864
16695 271/300 Write block 2017 len 28672 data: 2 2 3 3 3 5 6
16696 272/300 Read block 637 len 12288
16697 273/300 Write block 1415 len 16384 data: 3 3 3 4
16698 274/300 Write block 3737 len 12288 data: 6 6 8
16699 275/300 Read block 1058 len 24576
16700 276/300 Write block 1138 len 28672 data: 6 7 6 6 6 8 8
16701 277/300 Read block 4401 len 24576
16702 278/300 Read block 2440 len 4096
16703 279/300 Write block 963 len 28672 data: 3 2 2 2 2 2 2
16704 280/300 Read block 3343 len 32768
16705 281/300 Write block 3428 len 8192 data: 3 3
16706 282/300 Read block 3693 len 16384
16707 283/300 Write block 3126 len 32768 data: 8 6 7 7 7 7 7 4
16708 284/300 Read block 1010 len 16384
16709 285/300 Write block 1416 len 12288 data: 4 4 5
16710 286/300 Write block 1269 len 40960 data: 5 5 5 4 4 4 4 3 3 2
16711 287/300 Read block 4772 len 24576
16712 288/300 Read block 342 len 40960
16713 289/300 Read block 334 len 28672
16714 290/300 Write block 2674 len 28672 data: 4 4 4 4 4 4 4
16715 291/300 Write block 1417 len 32768 data: 5 6 4 3 4 4 3 3
16716 292/300 Write block 4956 len 28672 data: 4 5 5 4 4 4 3
16717 293/300 Write block 2225 len 40960 data: 2 3 3 3 3 4 5 6 5 5
16718 294/300 Write block 4432 len 4096 data: 6
16719 295/300 Read block 4837 len 28672
16720 296/300 Read block 3770 len 4096
16721 297/300 Read block 2505 len 8192
16722 298/300 Write block 2440 len 36864 data: 3 3 3 3 2 2 2 2 2
16723 299/300 Read block 618 len 8192
16724 300/300 Read block 577 len 28672
16725 301/300 Write block 3197 len 40960 data: 3 3 3 4 5 6 7 7 6 6
16726 Main task has work to do, go find it
16727 got dsc Start(1)
16728 start 1
16729 Nov 19 17:11:31.414 INFO request completed, latency_us: 230, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3fb2cd68-489f-4246-a35b-8daab567dfef, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
16730 Replay: started 1, returned:Ok(())
16731 ----------------------------------------------------------------
16732 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16733 States: Active Offline Active
16734 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16735 5226 Acked Read 3 Done Sent Done false
16736 5227 Acked Read 10 Done Sent Done false
16737 5228 Acked Write 8 Done Sent Done false
16738 5229 Acked Read 3 Done Sent Done false
16739 5230 Acked Read 7 Done Sent Done false
16740 5231 Acked Read 9 Done Sent Done false
16741 5232 Acked Write 8 Done Sent Done false
16742 5233 Acked Write 4 Done Sent Done false
16743 5234 Acked Read 6 Done Sent Done false
16744 5235 Acked Read 3 Done Sent Done false
16745 5236 Acked Flush 0 Done Sent Done false
16746 5237 Acked Read 5 Done Sent Done false
16747 5238 Acked Write 4 Done Sent Done false
16748 5239 Acked Write 4 Done Sent Done false
16749 5240 Acked Read 6 Done Sent Done false
16750 5241 Acked Read 7 Done Sent Done false
16751 5242 Acked Read 4 Done Sent Done false
16752 5243 Acked Flush 0 Done Sent Done false
16753 5244 Acked Write 3 Done Sent Done false
16754 5245 Acked Write 9 Done Sent Done false
16755 5246 Acked Read 8 Done Sent Done false
16756 5247 Acked Read 3 Done Sent Done false
16757 5248 Acked Write 3 Done Sent Done false
16758 5249 Acked Read 6 Done Sent Done false
16759 5250 Acked Read 9 Done Sent Done false
16760 5251 Acked Write 5 Done Sent Done false
16761 5252 Acked Write 2 Done Sent Done false
16762 5253 Acked Read 4 Done Sent Done false
16763 5254 Acked Write 6 Done Sent Done false
16764 5255 Acked Write 9 Done Sent Done false
16765 5256 Acked Read 7 Done Sent Done false
16766 5257 Acked Write 5 Done Sent Done false
16767 5258 Acked Read 6 Done Sent Done false
16768 5259 Acked Read 6 Done Sent Done false
16769 5260 Acked Read 3 Done Sent Done false
16770 5261 Acked Read 2 Done Sent Done false
16771 5262 Acked Write 10 Done Sent Done false
16772 5263 Acked Write 1 Done Sent Done false
16773 5264 Acked Write 1 Done Sent Done false
16774 5265 Acked Read 9 Done Sent Done false
16775 5266 Acked Write 10 Done Sent Done false
16776 5267 Acked Read 1 Done Sent Done false
16777 5268 Acked Read 7 Done Sent Done false
16778 5269 Acked Flush 0 Done Sent Done false
16779 5270 Acked Read 1 Done Sent Done false
16780 5271 Acked Write 1 Done Sent Done false
16781 5272 Acked Read 3 Done Sent Done false
16782 5273 Acked Read 5 Done Sent Done false
16783 5274 Acked Write 9 Done Sent Done false
16784 5275 Acked Read 10 Done Sent Done false
16785 5276 Acked Read 7 Done Sent Done false
16786 5277 Acked Read 1 Done Sent Done false
16787 5278 Acked Write 6 Done Sent Done false
16788 5279 Acked Write 5 Done Sent Done false
16789 5280 Acked Write 7 Done Sent Done false
16790 5281 Acked Read 5 Done Sent Done false
16791 5282 Acked Read 9 Done Sent Done false
16792 5283 Acked Write 4 Done Sent Done false
16793 5284 Acked Write 8 Done Sent Done false
16794 5285 Acked Write 7 Done Sent Done false
16795 5286 Acked Read 1 Done Sent Done false
16796 5287 Acked Read 8 Done Sent Done false
16797 5288 Acked Flush 0 Done Sent Done false
16798 5289 Acked Write 10 Done Sent Done false
16799 5290 Acked Read 3 Done Sent Done false
16800 5291 Acked Read 10 Done Sent Done false
16801 5292 Acked Read 2 Done Sent Done false
16802 5293 Acked Read 8 Done Sent Done false
16803 5294 Acked Write 8 Done Sent Done false
16804 5295 Acked Read 1 Done Sent Done false
16805 5296 Acked Read 5 Done Sent Done false
16806 5297 Acked Flush 0 Done Sent Done false
16807 5298 Acked Read 1 Done Sent Done false
16808 5299 Acked Write 6 Done Sent Done false
16809 5300 Acked Flush 0 Done Sent Done false
16810 5301 Acked Write 2 Done Sent Done false
16811 5302 Acked Read 1 Done Sent Done false
16812 5303 Acked Read 7 Done Sent Done false
16813 5304 Acked Write 10 Done Sent Done false
16814 5305 Acked Read 9 Done Sent Done false
16815 5306 Acked Read 4 Done Sent Done false
16816 5307 Acked Read 8 Done Sent Done false
16817 5308 Acked Write 10 Done Sent Done false
16818 5309 Acked Write 4 Done Sent Done false
16819 5310 Acked Read 10 Done Sent Done false
16820 5311 Acked Write 4 Done Sent Done false
16821 5312 Acked Write 10 Done Sent Done false
16822 5313 Acked Read 10 Done Sent Done false
16823 5314 Acked Write 1 Done Sent Done false
16824 5315 Acked Read 7 Done Sent Done false
16825 5316 Acked Read 10 Done Sent Done false
16826 5317 Acked Read 3 Done Sent Done false
16827 5318 Acked Read 5 Done Sent Done false
16828 5319 Acked Read 9 Done Sent Done false
16829 5320 Acked Read 9 Done Sent Done false
16830 5321 Acked Read 5 Done Sent Done false
16831 5322 Acked Read 4 Done Sent Done false
16832 5323 Acked Read 3 Done Sent Done false
16833 5324 Acked Read 10 Done Sent Done false
16834 5325 Acked Flush 0 Done Sent Done false
16835 5326 Acked Read 4 Done Sent Done false
16836 5327 Acked Write 9 Done Sent Done false
16837 5328 Acked Read 1 Done Sent Done false
16838 5329 Acked Write 10 Done Sent Done false
16839 5330 Acked Read 5 Done Sent Done false
16840 5331 Acked Read 2 Done Sent Done false
16841 5332 Acked Write 2 Done Sent Done false
16842 5333 Acked Flush 0 Done Sent Done false
16843 5334 Acked Read 2 Done Sent Done false
16844 5335 Acked Read 8 Done Sent Done false
16845 5336 Acked Write 10 Done Sent Done false
16846 5337 Acked Read 2 Done Sent Done false
16847 5338 Acked Read 5 Done Sent Done false
16848 5339 Acked Read 8 Done Sent Done false
16849 5340 Acked Read 3 Done Sent Done false
16850 5341 Acked Read 10 Done Sent Done false
16851 5342 Acked Read 6 Done Sent Done false
16852 5343 Acked Read 6 Done Sent Done false
16853 5344 Acked Read 2 Done Sent Done false
16854 5345 Acked Write 7 Done Sent Done false
16855 5346 Acked Read 5 Done Sent Done false
16856 5347 Acked Write 8 Done Sent Done false
16857 5348 Acked Write 4 Done Sent Done false
16858 5349 Acked Read 5 Done Sent Done false
16859 5350 Acked Write 4 Done Sent Done false
16860 5351 Acked Write 1 Done Sent Done false
16861 5352 Acked Write 9 Done Sent Done false
16862 5353 Acked Read 9 Done Sent Done false
16863 5354 Acked Read 9 Done Sent Done false
16864 5355 Acked Read 7 Done Sent Done false
16865 5356 Acked Read 4 Done Sent Done false
16866 5357 Acked Read 9 Done Sent Done false
16867 5358 Acked Read 5 Done Sent Done false
16868 5359 Acked Write 6 Done Sent Done false
16869 5360 Acked Read 10 Done Sent Done false
16870 5361 Acked Read 3 Done Sent Done false
16871 5362 Acked Write 3 Done Sent Done false
16872 5363 Acked Read 8 Done Sent Done false
16873 5364 Acked Write 1 Done Sent Done false
16874 5365 Acked Write 3 Done Sent Done false
16875 5366 Acked Read 2 Done Sent Done false
16876 5367 Acked Read 8 Done Sent Done false
16877 5368 Acked Write 2 Done Sent Done false
16878 5369 Acked Read 8 Done Sent Done false
16879 5370 Acked Read 5 Done Sent Done false
16880 5371 Acked Read 6 Done Sent Done false
16881 5372 Acked Read 3 Done Sent Done false
16882 5373 Acked Read 5 Done Sent Done false
16883 5374 Acked Read 6 Done Sent Done false
16884 5375 Acked Read 9 Done Sent Done false
16885 5376 Acked Read 8 Done Sent Done false
16886 5377 Acked Read 2 Done Sent Done false
16887 5378 Acked Read 7 Done Sent Done false
16888 5379 Acked Flush 0 Done Sent Done false
16889 5380 Acked Read 1 Done Sent Done false
16890 5381 Acked Read 1 Done Sent Done false
16891 5382 Acked Write 8 Done Sent Done false
16892 5383 Acked Write 9 Done Sent Done false
16893 5384 Acked Write 5 Done Sent Done false
16894 5385 Acked Read 1 Done Sent Done false
16895 5386 Acked Flush 0 Done Sent Done false
16896 5387 Acked Read 8 Done Sent Done false
16897 5388 Acked Read 6 Done Sent Done false
16898 5389 Acked Write 9 Done Sent Done false
16899 5390 Acked Read 10 Done Sent Done false
16900 5391 Acked Write 9 Done Sent Done false
16901 5392 Acked Flush 0 Done Sent Done false
16902 5393 Acked Read 10 Done Sent Done false
16903 5394 Acked Flush 0 Done Sent Done false
16904 5395 Acked Read 3 Done Sent Done false
16905 5396 Acked Write 3 Done Sent Done false
16906 5397 Acked Read 7 Done Sent Done false
16907 5398 Acked Read 6 Done Sent Done false
16908 5399 Acked Read 2 Done Sent Done false
16909 5400 Acked Read 7 Done Sent Done false
16910 5401 Acked Write 5 Done Sent Done false
16911 5402 Acked Write 6 Done Sent Done false
16912 5403 Acked Read 2 Done Sent Done false
16913 5404 Acked Read 1 Done Sent Done false
16914 5405 Acked Write 4 Done Sent Done false
16915 5406 Acked Flush 0 Done Sent Done false
16916 5407 Acked Write 5 Done Sent Done false
16917 5408 Acked Read 4 Done Sent Done false
16918 5409 Acked Read 5 Done Sent Done false
16919 5410 Acked Read 3 Done Sent Done false
16920 5411 Acked Read 7 Done Sent Done false
16921 5412 Acked Write 9 Done Sent Done false
16922 5413 Acked Write 4 Done Sent Done false
16923 5414 Acked Write 9 Done Sent Done false
16924 5415 Acked Read 8 Done Sent Done false
16925 5416 Acked Write 4 Done Sent Done false
16926 5417 Acked Read 4 Done Sent Done false
16927 5418 Acked Write 1 Done Sent Done false
16928 5419 Acked Read 10 Done Sent Done false
16929 5420 Acked Read 10 Done Sent Done false
16930 5421 Acked Read 5 Done Sent Done false
16931 5422 Acked Write 3 Done Sent Done false
16932 5423 Acked Write 6 Done Sent Done false
16933 5424 Acked Write 4 Done Sent Done false
16934 5425 Acked Read 5 Done Sent Done false
16935 5426 Acked Write 6 Done Sent Done false
16936 5427 Acked Read 10 Done Sent Done false
16937 5428 Acked Read 7 Done Sent Done false
16938 5429 Acked Write 10 Done Sent Done false
16939 5430 Acked Read 5 Done Sent Done false
16940 5431 Acked Write 8 Done Sent Done false
16941 5432 Acked Write 2 Done Sent Done false
16942 5433 Acked Flush 0 Done Sent Done false
16943 5434 Acked Write 3 Done Sent Done false
16944 5435 Acked Read 3 Done Sent Done false
16945 5436 Acked Read 4 Done Sent Done false
16946 5437 Acked Write 5 Done Sent Done false
16947 5438 Acked Write 4 Done Sent Done false
16948 5439 Acked Flush 0 Done Sent Done false
16949 5440 Acked Read 4 Done Sent Done false
16950 5441 Acked Write 4 Done Sent Done false
16951 5442 Acked Read 3 Done Sent Done false
16952 5443 Acked Flush 0 Done Sent Done false
16953 5444 Acked Read 6 Done Sent Done false
16954 5445 Acked Write 2 Done Sent Done false
16955 5446 Acked Write 3 Done Sent Done false
16956 5447 Acked Read 10 Done Sent Done false
16957 5448 Acked Read 4 Done Sent Done false
16958 5449 Acked Write 4 Done Sent Done false
16959 5450 Acked Read 10 Done Sent Done false
16960 5451 Acked Read 2 Done Sent Done false
16961 5452 Acked Read 3 Done Sent Done false
16962 5453 Acked Write 9 Done Sent Done false
16963 5454 Acked Read 8 Done Sent Done false
16964 5455 Acked Read 1 Done Sent Done false
16965 5456 Acked Write 4 Done Sent Done false
16966 5457 Acked Read 2 Done Sent Done false
16967 5458 Acked Read 10 Done Sent Done false
16968 5459 Acked Flush 0 Done Sent Done false
16969 5460 Acked Read 2 Done Sent Done false
16970 5461 Acked Flush 0 Done Sent Done false
16971 5462 Acked Write 5 Done Sent Done false
16972 5463 Acked Read 2 Done Sent Done false
16973 5464 Acked Read 1 Done Sent Done false
16974 5465 Acked Write 4 Done Sent Done false
16975 5466 Acked Write 1 Done Sent Done false
16976 5467 Acked Read 1 Done Sent Done false
16977 5468 Acked Read 8 Done Sent Done false
16978 5469 Acked Flush 0 Done Sent Done false
16979 5470 Acked Read 8 Done Sent Done false
16980 5471 Acked Read 2 Done Sent Done false
16981 5472 Acked Write 1 Done Sent Done false
16982 5473 Acked Write 2 Done Sent Done false
16983 5474 Acked Flush 0 Done Sent Done false
16984 5475 Acked Read 10 Done Sent Done false
16985 5476 Acked Write 4 Done Sent Done false
16986 5477 Acked Write 8 Done Sent Done false
16987 5478 Acked Read 8 Done Sent Done false
16988 5479 Acked Read 8 Done Sent Done false
16989 5480 Acked Read 1 Done Sent Done false
16990 5481 Acked Read 4 Done Sent Done false
16991 5482 Acked Read 4 Done Sent Done false
16992 5483 Acked Write 6 Done Sent Done false
16993 5484 Acked Write 6 Done Sent Done false
16994 5485 Acked Read 10 Done Sent Done false
16995 5486 Acked Write 6 Done Sent Done false
16996 5487 Acked Read 1 Done Sent Done false
16997 5488 Acked Read 4 Done Sent Done false
16998 5489 Acked Read 3 Done Sent Done false
16999 5490 Acked Read 9 Done Sent Done false
17000 5491 Acked Write 4 Done Sent Done false
17001 5492 Acked Write 2 Done Sent Done false
17002 5493 Acked Write 10 Done Sent Done false
17003 5494 Acked Write 1 Done Sent Done false
17004 5495 Acked Read 9 Done Sent Done false
17005 5496 Acked Write 7 Done Sent Done false
17006 5497 Acked Read 3 Done Sent Done false
17007 5498 Acked Write 4 Done Sent Done false
17008 5499 Acked Write 3 Done Sent Done false
17009 5500 Acked Read 6 Done Sent Done false
17010 5501 Acked Write 7 Done Sent Done false
17011 5502 Acked Read 6 Done Sent Done false
17012 5503 Acked Read 1 Done Sent Done false
17013 5504 Acked Write 7 Done Sent Done false
17014 5505 Acked Read 8 Done Sent Done false
17015 5506 Acked Write 2 Done Sent Done false
17016 5507 Acked Read 4 Done Sent Done false
17017 5508 Acked Write 8 Done Sent Done false
17018 5509 Acked Read 4 Done Sent Done false
17019 5510 Acked Write 3 Done Sent Done false
17020 5511 Acked Write 10 Done Sent Done false
17021 5512 Acked Read 6 Done Sent Done false
17022 5513 Acked Read 10 Done Sent Done false
17023 5514 Acked Read 7 Done Sent Done false
17024 5515 Acked Write 7 Done Sent Done false
17025 5516 Acked Write 8 Done Sent Done false
17026 5517 Acked Write 7 Done Sent Done false
17027 5518 Acked Write 10 Done Sent Done false
17028 5519 Acked Write 1 Done Sent Done false
17029 5520 Acked Read 7 Done Sent Done false
17030 5521 Acked Read 1 Done Sent Done false
17031 5522 Acked Read 2 Done Sent Done false
17032 5523 Acked Write 9 Done Sent Done false
17033 5524 Acked Read 2 Done Sent Done false
17034 5525 Acked Read 7 Done Sent Done false
17035 5526 Acked Write 10 Done Sent Done false
17036 STATES DS:0 DS:1 DS:2 TOTAL
17037 Sent 0 301 0 301
17038 Done 301 0 301 602
17039 Skipped 0 0 0 0
17040 Error 0 0 0 0
17041 Last Flush: 5474 5225 5474
17042 Downstairs last five completed: 5225 5224 5223 5222 5221
17043 Upstairs last five completed: 5526 5525 5524 5523 5522
17044 CLIENT: Up:0 ds:301 act:2
17045 ----------------------------------------------------------------
17046 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17047 States: Active Offline Active
17048 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17049 5226 Acked Read 3 Done Sent Done false
17050 5227 Acked Read 10 Done Sent Done false
17051 5228 Acked Write 8 Done Sent Done false
17052 5229 Acked Read 3 Done Sent Done false
17053 5230 Acked Read 7 Done Sent Done false
17054 5231 Acked Read 9 Done Sent Done false
17055 5232 Acked Write 8 Done Sent Done false
17056 5233 Acked Write 4 Done Sent Done false
17057 5234 Acked Read 6 Done Sent Done false
17058 5235 Acked Read 3 Done Sent Done false
17059 5236 Acked Flush 0 Done Sent Done false
17060 5237 Acked Read 5 Done Sent Done false
17061 5238 Acked Write 4 Done Sent Done false
17062 5239 Acked Write 4 Done Sent Done false
17063 5240 Acked Read 6 Done Sent Done false
17064 5241 Acked Read 7 Done Sent Done false
17065 5242 Acked Read 4 Done Sent Done false
17066 5243 Acked Flush 0 Done Sent Done false
17067 5244 Acked Write 3 Done Sent Done false
17068 5245 Acked Write 9 Done Sent Done false
17069 5246 Acked Read 8 Done Sent Done false
17070 5247 Acked Read 3 Done Sent Done false
17071 5248 Acked Write 3 Done Sent Done false
17072 5249 Acked Read 6 Done Sent Done false
17073 5250 Acked Read 9 Done Sent Done false
17074 5251 Acked Write 5 Done Sent Done false
17075 5252 Acked Write 2 Done Sent Done false
17076 5253 Acked Read 4 Done Sent Done false
17077 5254 Acked Write 6 Done Sent Done false
17078 5255 Acked Write 9 Done Sent Done false
17079 5256 Acked Read 7 Done Sent Done false
17080 5257 Acked Write 5 Done Sent Done false
17081 5258 Acked Read 6 Done Sent Done false
17082 5259 Acked Read 6 Done Sent Done false
17083 5260 Acked Read 3 Done Sent Done false
17084 5261 Acked Read 2 Done Sent Done false
17085 5262 Acked Write 10 Done Sent Done false
17086 5263 Acked Write 1 Done Sent Done false
17087 5264 Acked Write 1 Done Sent Done false
17088 5265 Acked Read 9 Done Sent Done false
17089 5266 Acked Write 10 Done Sent Done false
17090 5267 Acked Read 1 Done Sent Done false
17091 5268 Acked Read 7 Done Sent Done false
17092 5269 Acked Flush 0 Done Sent Done false
17093 5270 Acked Read 1 Done Sent Done false
17094 5271 Acked Write 1 Done Sent Done false
17095 5272 Acked Read 3 Done Sent Done false
17096 5273 Acked Read 5 Done Sent Done false
17097 5274 Acked Write 9 Done Sent Done false
17098 5275 Acked Read 10 Done Sent Done false
17099 5276 Acked Read 7 Done Sent Done false
17100 5277 Acked Read 1 Done Sent Done false
17101 5278 Acked Write 6 Done Sent Done false
17102 5279 Acked Write 5 Done Sent Done false
17103 5280 Acked Write 7 Done Sent Done false
17104 5281 Acked Read 5 Done Sent Done false
17105 5282 Acked Read 9 Done Sent Done false
17106 5283 Acked Write 4 Done Sent Done false
17107 5284 Acked Write 8 Done Sent Done false
17108 5285 Acked Write 7 Done Sent Done false
17109 5286 Acked Read 1 Done Sent Done false
17110 5287 Acked Read 8 Done Sent Done false
17111 5288 Acked Flush 0 Done Sent Done false
17112 5289 Acked Write 10 Done Sent Done false
17113 5290 Acked Read 3 Done Sent Done false
17114 5291 Acked Read 10 Done Sent Done false
17115 5292 Acked Read 2 Done Sent Done false
17116 5293 Acked Read 8 Done Sent Done false
17117 5294 Acked Write 8 Done Sent Done false
17118 5295 Acked Read 1 Done Sent Done false
17119 5296 Acked Read 5 Done Sent Done false
17120 5297 Acked Flush 0 Done Sent Done false
17121 5298 Acked Read 1 Done Sent Done false
17122 5299 Acked Write 6 Done Sent Done false
17123 5300 Acked Flush 0 Done Sent Done false
17124 5301 Acked Write 2 Done Sent Done false
17125 5302 Acked Read 1 Done Sent Done false
17126 5303 Acked Read 7 Done Sent Done false
17127 5304 Acked Write 10 Done Sent Done false
17128 5305 Acked Read 9 Done Sent Done false
17129 5306 Acked Read 4 Done Sent Done false
17130 5307 Acked Read 8 Done Sent Done false
17131 5308 Acked Write 10 Done Sent Done false
17132 5309 Acked Write 4 Done Sent Done false
17133 5310 Acked Read 10 Done Sent Done false
17134 5311 Acked Write 4 Done Sent Done false
17135 5312 Acked Write 10 Done Sent Done false
17136 5313 Acked Read 10 Done Sent Done false
17137 5314 Acked Write 1 Done Sent Done false
17138 5315 Acked Read 7 Done Sent Done false
17139 5316 Acked Read 10 Done Sent Done false
17140 5317 Acked Read 3 Done Sent Done false
17141 5318 Acked Read 5 Done Sent Done false
17142 5319 Acked Read 9 Done Sent Done false
17143 5320 Acked Read 9 Done Sent Done false
17144 5321 Acked Read 5 Done Sent Done false
17145 5322 Acked Read 4 Done Sent Done false
17146 5323 Acked Read 3 Done Sent Done false
17147 5324 Acked Read 10 Done Sent Done false
17148 5325 Acked Flush 0 Done Sent Done false
17149 5326 Acked Read 4 Done Sent Done false
17150 5327 Acked Write 9 Done Sent Done false
17151 5328 Acked Read 1 Done Sent Done false
17152 5329 Acked Write 10 Done Sent Done false
17153 5330 Acked Read 5 Done Sent Done false
17154 5331 Acked Read 2 Done Sent Done false
17155 5332 Acked Write 2 Done Sent Done false
17156 5333 Acked Flush 0 Done Sent Done false
17157 5334 Acked Read 2 Done Sent Done false
17158 5335 Acked Read 8 Done Sent Done false
17159 5336 Acked Write 10 Done Sent Done false
17160 5337 Acked Read 2 Done Sent Done false
17161 5338 Acked Read 5 Done Sent Done false
17162 5339 Acked Read 8 Done Sent Done false
17163 5340 Acked Read 3 Done Sent Done false
17164 5341 Acked Read 10 Done Sent Done false
17165 5342 Acked Read 6 Done Sent Done false
17166 5343 Acked Read 6 Done Sent Done false
17167 5344 Acked Read 2 Done Sent Done false
17168 5345 Acked Write 7 Done Sent Done false
17169 5346 Acked Read 5 Done Sent Done false
17170 5347 Acked Write 8 Done Sent Done false
17171 5348 Acked Write 4 Done Sent Done false
17172 5349 Acked Read 5 Done Sent Done false
17173 5350 Acked Write 4 Done Sent Done false
17174 5351 Acked Write 1 Done Sent Done false
17175 5352 Acked Write 9 Done Sent Done false
17176 5353 Acked Read 9 Done Sent Done false
17177 5354 Acked Read 9 Done Sent Done false
17178 5355 Acked Read 7 Done Sent Done false
17179 5356 Acked Read 4 Done Sent Done false
17180 5357 Acked Read 9 Done Sent Done false
17181 5358 Acked Read 5 Done Sent Done false
17182 5359 Acked Write 6 Done Sent Done false
17183 5360 Acked Read 10 Done Sent Done false
17184 5361 Acked Read 3 Done Sent Done false
17185 5362 Acked Write 3 Done Sent Done false
17186 5363 Acked Read 8 Done Sent Done false
17187 5364 Acked Write 1 Done Sent Done false
17188 5365 Acked Write 3 Done Sent Done false
17189 5366 Acked Read 2 Done Sent Done false
17190 5367 Acked Read 8 Done Sent Done false
17191 5368 Acked Write 2 Done Sent Done false
17192 5369 Acked Read 8 Done Sent Done false
17193 5370 Acked Read 5 Done Sent Done false
17194 5371 Acked Read 6 Done Sent Done false
17195 5372 Acked Read 3 Done Sent Done false
17196 5373 Acked Read 5 Done Sent Done false
17197 5374 Acked Read 6 Done Sent Done false
17198 5375 Acked Read 9 Done Sent Done false
17199 5376 Acked Read 8 Done Sent Done false
17200 5377 Acked Read 2 Done Sent Done false
17201 5378 Acked Read 7 Done Sent Done false
17202 5379 Acked Flush 0 Done Sent Done false
17203 5380 Acked Read 1 Done Sent Done false
17204 5381 Acked Read 1 Done Sent Done false
17205 5382 Acked Write 8 Done Sent Done false
17206 5383 Acked Write 9 Done Sent Done false
17207 5384 Acked Write 5 Done Sent Done false
17208 5385 Acked Read 1 Done Sent Done false
17209 5386 Acked Flush 0 Done Sent Done false
17210 5387 Acked Read 8 Done Sent Done false
17211 5388 Acked Read 6 Done Sent Done false
17212 5389 Acked Write 9 Done Sent Done false
17213 5390 Acked Read 10 Done Sent Done false
17214 5391 Acked Write 9 Done Sent Done false
17215 5392 Acked Flush 0 Done Sent Done false
17216 5393 Acked Read 10 Done Sent Done false
17217 5394 Acked Flush 0 Done Sent Done false
17218 5395 Acked Read 3 Done Sent Done false
17219 5396 Acked Write 3 Done Sent Done false
17220 5397 Acked Read 7 Done Sent Done false
17221 5398 Acked Read 6 Done Sent Done false
17222 5399 Acked Read 2 Done Sent Done false
17223 5400 Acked Read 7 Done Sent Done false
17224 5401 Acked Write 5 Done Sent Done false
17225 5402 Acked Write 6 Done Sent Done false
17226 5403 Acked Read 2 Done Sent Done false
17227 5404 Acked Read 1 Done Sent Done false
17228 5405 Acked Write 4 Done Sent Done false
17229 5406 Acked Flush 0 Done Sent Done false
17230 5407 Acked Write 5 Done Sent Done false
17231 5408 Acked Read 4 Done Sent Done false
17232 5409 Acked Read 5 Done Sent Done false
17233 5410 Acked Read 3 Done Sent Done false
17234 5411 Acked Read 7 Done Sent Done false
17235 5412 Acked Write 9 Done Sent Done false
17236 5413 Acked Write 4 Done Sent Done false
17237 5414 Acked Write 9 Done Sent Done false
17238 5415 Acked Read 8 Done Sent Done false
17239 5416 Acked Write 4 Done Sent Done false
17240 5417 Acked Read 4 Done Sent Done false
17241 5418 Acked Write 1 Done Sent Done false
17242 5419 Acked Read 10 Done Sent Done false
17243 5420 Acked Read 10 Done Sent Done false
17244 5421 Acked Read 5 Done Sent Done false
17245 5422 Acked Write 3 Done Sent Done false
17246 5423 Acked Write 6 Done Sent Done false
17247 5424 Acked Write 4 Done Sent Done false
17248 5425 Acked Read 5 Done Sent Done false
17249 5426 Acked Write 6 Done Sent Done false
17250 5427 Acked Read 10 Done Sent Done false
17251 5428 Acked Read 7 Done Sent Done false
17252 5429 Acked Write 10 Done Sent Done false
17253 5430 Acked Read 5 Done Sent Done false
17254 5431 Acked Write 8 Done Sent Done false
17255 5432 Acked Write 2 Done Sent Done false
17256 5433 Acked Flush 0 Done Sent Done false
17257 5434 Acked Write 3 Done Sent Done false
17258 5435 Acked Read 3 Done Sent Done false
17259 5436 Acked Read 4 Done Sent Done false
17260 5437 Acked Write 5 Done Sent Done false
17261 5438 Acked Write 4 Done Sent Done false
17262 5439 Acked Flush 0 Done Sent Done false
17263 5440 Acked Read 4 Done Sent Done false
17264 5441 Acked Write 4 Done Sent Done false
17265 5442 Acked Read 3 Done Sent Done false
17266 5443 Acked Flush 0 Done Sent Done false
17267 5444 Acked Read 6 Done Sent Done false
17268 5445 Acked Write 2 Done Sent Done false
17269 5446 Acked Write 3 Done Sent Done false
17270 5447 Acked Read 10 Done Sent Done false
17271 5448 Acked Read 4 Done Sent Done false
17272 5449 Acked Write 4 Done Sent Done false
17273 5450 Acked Read 10 Done Sent Done false
17274 5451 Acked Read 2 Done Sent Done false
17275 5452 Acked Read 3 Done Sent Done false
17276 5453 Acked Write 9 Done Sent Done false
17277 5454 Acked Read 8 Done Sent Done false
17278 5455 Acked Read 1 Done Sent Done false
17279 5456 Acked Write 4 Done Sent Done false
17280 5457 Acked Read 2 Done Sent Done false
17281 5458 Acked Read 10 Done Sent Done false
17282 5459 Acked Flush 0 Done Sent Done false
17283 5460 Acked Read 2 Done Sent Done false
17284 5461 Acked Flush 0 Done Sent Done false
17285 5462 Acked Write 5 Done Sent Done false
17286 5463 Acked Read 2 Done Sent Done false
17287 5464 Acked Read 1 Done Sent Done false
17288 5465 Acked Write 4 Done Sent Done false
17289 5466 Acked Write 1 Done Sent Done false
17290 5467 Acked Read 1 Done Sent Done false
17291 5468 Acked Read 8 Done Sent Done false
17292 5469 Acked Flush 0 Done Sent Done false
17293 5470 Acked Read 8 Done Sent Done false
17294 5471 Acked Read 2 Done Sent Done false
17295 5472 Acked Write 1 Done Sent Done false
17296 5473 Acked Write 2 Done Sent Done false
17297 5474 Acked Flush 0 Done Sent Done false
17298 5475 Acked Read 10 Done Sent Done false
17299 5476 Acked Write 4 Done Sent Done false
17300 5477 Acked Write 8 Done Sent Done false
17301 5478 Acked Read 8 Done Sent Done false
17302 5479 Acked Read 8 Done Sent Done false
17303 5480 Acked Read 1 Done Sent Done false
17304 5481 Acked Read 4 Done Sent Done false
17305 5482 Acked Read 4 Done Sent Done false
17306 5483 Acked Write 6 Done Sent Done false
17307 5484 Acked Write 6 Done Sent Done false
17308 5485 Acked Read 10 Done Sent Done false
17309 5486 Acked Write 6 Done Sent Done false
17310 5487 Acked Read 1 Done Sent Done false
17311 5488 Acked Read 4 Done Sent Done false
17312 5489 Acked Read 3 Done Sent Done false
17313 5490 Acked Read 9 Done Sent Done false
17314 5491 Acked Write 4 Done Sent Done false
17315 5492 Acked Write 2 Done Sent Done false
17316 5493 Acked Write 10 Done Sent Done false
17317 5494 Acked Write 1 Done Sent Done false
17318 5495 Acked Read 9 Done Sent Done false
17319 5496 Acked Write 7 Done Sent Done false
17320 5497 Acked Read 3 Done Sent Done false
17321 5498 Acked Write 4 Done Sent Done false
17322 5499 Acked Write 3 Done Sent Done false
17323 5500 Acked Read 6 Done Sent Done false
17324 5501 Acked Write 7 Done Sent Done false
17325 5502 Acked Read 6 Done Sent Done false
17326 5503 Acked Read 1 Done Sent Done false
17327 5504 Acked Write 7 Done Sent Done false
17328 5505 Acked Read 8 Done Sent Done false
17329 5506 Acked Write 2 Done Sent Done false
17330 5507 Acked Read 4 Done Sent Done false
17331 5508 Acked Write 8 Done Sent Done false
17332 5509 Acked Read 4 Done Sent Done false
17333 5510 Acked Write 3 Done Sent Done false
17334 5511 Acked Write 10 Done Sent Done false
17335 5512 Acked Read 6 Done Sent Done false
17336 5513 Acked Read 10 Done Sent Done false
17337 5514 Acked Read 7 Done Sent Done false
17338 5515 Acked Write 7 Done Sent Done false
17339 5516 Acked Write 8 Done Sent Done false
17340 5517 Acked Write 7 Done Sent Done false
17341 5518 Acked Write 10 Done Sent Done false
17342 5519 Acked Write 1 Done Sent Done false
17343 5520 Acked Read 7 Done Sent Done false
17344 5521 Acked Read 1 Done Sent Done false
17345 5522 Acked Read 2 Done Sent Done false
17346 5523 Acked Write 9 Done Sent Done false
17347 5524 Acked Read 2 Done Sent Done false
17348 5525 Acked Read 7 Done Sent Done false
17349 5526 Acked Write 10 Done Sent Done false
17350 5527 Acked Flush 0 Done Sent Done false
17351 STATES DS:0 DS:1 DS:2 TOTAL
17352 Sent 0 302 0 302
17353 Done 302 0 302 604
17354 Skipped 0 0 0 0
17355 Error 0 0 0 0
17356 Last Flush: 5527 5225 5527
17357 Downstairs last five completed: 5225 5224 5223 5222 5221
17358 Upstairs last five completed: 5527 5526 5525 5524 5523
17359 CLIENT: Up:0 ds:302 act:2
173602024-11-19T17:11:37.168ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
173612024-11-19T17:11:37.168ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
173622024-11-19T17:11:37.168ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
173632024-11-19T17:11:37.168ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 05650a15-902e-469c-91e5-cba45f07470e
173642024-11-19T17:11:37.168ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
17365 [8820] got start action
17366 [8820] I am going to restart
17367 Starting downstairs at port 8820
17368 Make output file at "/tmp/dsc/downstairs-8820.txt"
17369 [8820][1] reports Starting
17370 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1217)
17371 [8820][1] reports Running
17372 ----------------------------------------------------------------
17373 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17374 States: Active Offline Active
17375 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17376 5226 Acked Read 3 Done Sent Done false
17377 5227 Acked Read 10 Done Sent Done false
17378 5228 Acked Write 8 Done Sent Done false
17379 5229 Acked Read 3 Done Sent Done false
17380 5230 Acked Read 7 Done Sent Done false
17381 5231 Acked Read 9 Done Sent Done false
17382 5232 Acked Write 8 Done Sent Done false
17383 5233 Acked Write 4 Done Sent Done false
17384 5234 Acked Read 6 Done Sent Done false
17385 5235 Acked Read 3 Done Sent Done false
17386 5236 Acked Flush 0 Done Sent Done false
17387 5237 Acked Read 5 Done Sent Done false
17388 5238 Acked Write 4 Done Sent Done false
17389 5239 Acked Write 4 Done Sent Done false
17390 5240 Acked Read 6 Done Sent Done false
17391 5241 Acked Read 7 Done Sent Done false
17392 5242 Acked Read 4 Done Sent Done false
17393 5243 Acked Flush 0 Done Sent Done false
17394 5244 Acked Write 3 Done Sent Done false
17395 5245 Acked Write 9 Done Sent Done false
17396 5246 Acked Read 8 Done Sent Done false
17397 5247 Acked Read 3 Done Sent Done false
17398 5248 Acked Write 3 Done Sent Done false
17399 5249 Acked Read 6 Done Sent Done false
17400 5250 Acked Read 9 Done Sent Done false
17401 5251 Acked Write 5 Done Sent Done false
17402 5252 Acked Write 2 Done Sent Done false
17403 5253 Acked Read 4 Done Sent Done false
17404 5254 Acked Write 6 Done Sent Done false
17405 5255 Acked Write 9 Done Sent Done false
17406 5256 Acked Read 7 Done Sent Done false
17407 5257 Acked Write 5 Done Sent Done false
17408 5258 Acked Read 6 Done Sent Done false
17409 5259 Acked Read 6 Done Sent Done false
17410 5260 Acked Read 3 Done Sent Done false
17411 5261 Acked Read 2 Done Sent Done false
17412 5262 Acked Write 10 Done Sent Done false
17413 5263 Acked Write 1 Done Sent Done false
17414 5264 Acked Write 1 Done Sent Done false
17415 5265 Acked Read 9 Done Sent Done false
17416 5266 Acked Write 10 Done Sent Done false
17417 5267 Acked Read 1 Done Sent Done false
17418 5268 Acked Read 7 Done Sent Done false
17419 5269 Acked Flush 0 Done Sent Done false
17420 5270 Acked Read 1 Done Sent Done false
17421 5271 Acked Write 1 Done Sent Done false
17422 5272 Acked Read 3 Done Sent Done false
17423 5273 Acked Read 5 Done Sent Done false
17424 5274 Acked Write 9 Done Sent Done false
17425 5275 Acked Read 10 Done Sent Done false
17426 5276 Acked Read 7 Done Sent Done false
17427 5277 Acked Read 1 Done Sent Done false
17428 5278 Acked Write 6 Done Sent Done false
17429 5279 Acked Write 5 Done Sent Done false
17430 5280 Acked Write 7 Done Sent Done false
17431 5281 Acked Read 5 Done Sent Done false
17432 5282 Acked Read 9 Done Sent Done false
17433 5283 Acked Write 4 Done Sent Done false
17434 5284 Acked Write 8 Done Sent Done false
17435 5285 Acked Write 7 Done Sent Done false
17436 5286 Acked Read 1 Done Sent Done false
17437 5287 Acked Read 8 Done Sent Done false
17438 5288 Acked Flush 0 Done Sent Done false
17439 5289 Acked Write 10 Done Sent Done false
17440 5290 Acked Read 3 Done Sent Done false
17441 5291 Acked Read 10 Done Sent Done false
17442 5292 Acked Read 2 Done Sent Done false
17443 5293 Acked Read 8 Done Sent Done false
17444 5294 Acked Write 8 Done Sent Done false
17445 5295 Acked Read 1 Done Sent Done false
17446 5296 Acked Read 5 Done Sent Done false
17447 5297 Acked Flush 0 Done Sent Done false
17448 5298 Acked Read 1 Done Sent Done false
17449 5299 Acked Write 6 Done Sent Done false
17450 5300 Acked Flush 0 Done Sent Done false
17451 5301 Acked Write 2 Done Sent Done false
17452 5302 Acked Read 1 Done Sent Done false
17453 5303 Acked Read 7 Done Sent Done false
17454 5304 Acked Write 10 Done Sent Done false
17455 5305 Acked Read 9 Done Sent Done false
17456 5306 Acked Read 4 Done Sent Done false
17457 5307 Acked Read 8 Done Sent Done false
17458 5308 Acked Write 10 Done Sent Done false
17459 5309 Acked Write 4 Done Sent Done false
17460 5310 Acked Read 10 Done Sent Done false
17461 5311 Acked Write 4 Done Sent Done false
17462 5312 Acked Write 10 Done Sent Done false
17463 5313 Acked Read 10 Done Sent Done false
17464 5314 Acked Write 1 Done Sent Done false
17465 5315 Acked Read 7 Done Sent Done false
17466 5316 Acked Read 10 Done Sent Done false
17467 5317 Acked Read 3 Done Sent Done false
17468 5318 Acked Read 5 Done Sent Done false
17469 5319 Acked Read 9 Done Sent Done false
17470 5320 Acked Read 9 Done Sent Done false
17471 5321 Acked Read 5 Done Sent Done false
17472 5322 Acked Read 4 Done Sent Done false
17473 5323 Acked Read 3 Done Sent Done false
17474 5324 Acked Read 10 Done Sent Done false
17475 5325 Acked Flush 0 Done Sent Done false
17476 5326 Acked Read 4 Done Sent Done false
17477 5327 Acked Write 9 Done Sent Done false
17478 5328 Acked Read 1 Done Sent Done false
17479 5329 Acked Write 10 Done Sent Done false
17480 5330 Acked Read 5 Done Sent Done false
17481 5331 Acked Read 2 Done Sent Done false
17482 5332 Acked Write 2 Done Sent Done false
17483 5333 Acked Flush 0 Done Sent Done false
17484 5334 Acked Read 2 Done Sent Done false
17485 5335 Acked Read 8 Done Sent Done false
17486 5336 Acked Write 10 Done Sent Done false
17487 5337 Acked Read 2 Done Sent Done false
17488 5338 Acked Read 5 Done Sent Done false
17489 5339 Acked Read 8 Done Sent Done false
17490 5340 Acked Read 3 Done Sent Done false
17491 5341 Acked Read 10 Done Sent Done false
17492 5342 Acked Read 6 Done Sent Done false
17493 5343 Acked Read 6 Done Sent Done false
17494 5344 Acked Read 2 Done Sent Done false
17495 5345 Acked Write 7 Done Sent Done false
17496 5346 Acked Read 5 Done Sent Done false
17497 5347 Acked Write 8 Done Sent Done false
17498 5348 Acked Write 4 Done Sent Done false
17499 5349 Acked Read 5 Done Sent Done false
17500 5350 Acked Write 4 Done Sent Done false
17501 5351 Acked Write 1 Done Sent Done false
17502 5352 Acked Write 9 Done Sent Done false
17503 5353 Acked Read 9 Done Sent Done false
17504 5354 Acked Read 9 Done Sent Done false
17505 5355 Acked Read 7 Done Sent Done false
17506 5356 Acked Read 4 Done Sent Done false
17507 5357 Acked Read 9 Done Sent Done false
17508 5358 Acked Read 5 Done Sent Done false
17509 5359 Acked Write 6 Done Sent Done false
17510 5360 Acked Read 10 Done Sent Done false
17511 5361 Acked Read 3 Done Sent Done false
17512 5362 Acked Write 3 Done Sent Done false
17513 5363 Acked Read 8 Done Sent Done false
17514 5364 Acked Write 1 Done Sent Done false
17515 5365 Acked Write 3 Done Sent Done false
17516 5366 Acked Read 2 Done Sent Done false
17517 5367 Acked Read 8 Done Sent Done false
17518 5368 Acked Write 2 Done Sent Done false
17519 5369 Acked Read 8 Done Sent Done false
17520 5370 Acked Read 5 Done Sent Done false
17521 5371 Acked Read 6 Done Sent Done false
17522 5372 Acked Read 3 Done Sent Done false
17523 5373 Acked Read 5 Done Sent Done false
17524 5374 Acked Read 6 Done Sent Done false
17525 5375 Acked Read 9 Done Sent Done false
17526 5376 Acked Read 8 Done Sent Done false
17527 5377 Acked Read 2 Done Sent Done false
17528 5378 Acked Read 7 Done Sent Done false
17529 5379 Acked Flush 0 Done Sent Done false
17530 5380 Acked Read 1 Done Sent Done false
17531 5381 Acked Read 1 Done Sent Done false
17532 5382 Acked Write 8 Done Sent Done false
17533 5383 Acked Write 9 Done Sent Done false
17534 5384 Acked Write 5 Done Sent Done false
17535 5385 Acked Read 1 Done Sent Done false
17536 5386 Acked Flush 0 Done Sent Done false
17537 5387 Acked Read 8 Done Sent Done false
17538 5388 Acked Read 6 Done Sent Done false
17539 5389 Acked Write 9 Done Sent Done false
17540 5390 Acked Read 10 Done Sent Done false
17541 5391 Acked Write 9 Done Sent Done false
17542 5392 Acked Flush 0 Done Sent Done false
17543 5393 Acked Read 10 Done Sent Done false
17544 5394 Acked Flush 0 Done Sent Done false
17545 5395 Acked Read 3 Done Sent Done false
17546 5396 Acked Write 3 Done Sent Done false
17547 5397 Acked Read 7 Done Sent Done false
17548 5398 Acked Read 6 Done Sent Done false
17549 5399 Acked Read 2 Done Sent Done false
17550 5400 Acked Read 7 Done Sent Done false
17551 5401 Acked Write 5 Done Sent Done false
17552 5402 Acked Write 6 Done Sent Done false
17553 5403 Acked Read 2 Done Sent Done false
17554 5404 Acked Read 1 Done Sent Done false
17555 5405 Acked Write 4 Done Sent Done false
17556 5406 Acked Flush 0 Done Sent Done false
17557 5407 Acked Write 5 Done Sent Done false
17558 5408 Acked Read 4 Done Sent Done false
17559 5409 Acked Read 5 Done Sent Done false
17560 5410 Acked Read 3 Done Sent Done false
17561 5411 Acked Read 7 Done Sent Done false
17562 5412 Acked Write 9 Done Sent Done false
17563 5413 Acked Write 4 Done Sent Done false
17564 5414 Acked Write 9 Done Sent Done false
17565 5415 Acked Read 8 Done Sent Done false
17566 5416 Acked Write 4 Done Sent Done false
17567 5417 Acked Read 4 Done Sent Done false
17568 5418 Acked Write 1 Done Sent Done false
17569 5419 Acked Read 10 Done Sent Done false
17570 5420 Acked Read 10 Done Sent Done false
17571 5421 Acked Read 5 Done Sent Done false
17572 5422 Acked Write 3 Done Sent Done false
17573 5423 Acked Write 6 Done Sent Done false
17574 5424 Acked Write 4 Done Sent Done false
17575 5425 Acked Read 5 Done Sent Done false
17576 5426 Acked Write 6 Done Sent Done false
17577 5427 Acked Read 10 Done Sent Done false
17578 5428 Acked Read 7 Done Sent Done false
17579 5429 Acked Write 10 Done Sent Done false
17580 5430 Acked Read 5 Done Sent Done false
17581 5431 Acked Write 8 Done Sent Done false
17582 5432 Acked Write 2 Done Sent Done false
17583 5433 Acked Flush 0 Done Sent Done false
17584 5434 Acked Write 3 Done Sent Done false
17585 5435 Acked Read 3 Done Sent Done false
17586 5436 Acked Read 4 Done Sent Done false
17587 5437 Acked Write 5 Done Sent Done false
17588 5438 Acked Write 4 Done Sent Done false
17589 5439 Acked Flush 0 Done Sent Done false
17590 5440 Acked Read 4 Done Sent Done false
17591 5441 Acked Write 4 Done Sent Done false
17592 5442 Acked Read 3 Done Sent Done false
17593 5443 Acked Flush 0 Done Sent Done false
17594 5444 Acked Read 6 Done Sent Done false
17595 5445 Acked Write 2 Done Sent Done false
17596 5446 Acked Write 3 Done Sent Done false
17597 5447 Acked Read 10 Done Sent Done false
17598 5448 Acked Read 4 Done Sent Done false
17599 5449 Acked Write 4 Done Sent Done false
17600 5450 Acked Read 10 Done Sent Done false
17601 5451 Acked Read 2 Done Sent Done false
17602 5452 Acked Read 3 Done Sent Done false
17603 5453 Acked Write 9 Done Sent Done false
17604 5454 Acked Read 8 Done Sent Done false
17605 5455 Acked Read 1 Done Sent Done false
17606 5456 Acked Write 4 Done Sent Done false
17607 5457 Acked Read 2 Done Sent Done false
17608 5458 Acked Read 10 Done Sent Done false
17609 5459 Acked Flush 0 Done Sent Done false
17610 5460 Acked Read 2 Done Sent Done false
17611 5461 Acked Flush 0 Done Sent Done false
17612 5462 Acked Write 5 Done Sent Done false
17613 5463 Acked Read 2 Done Sent Done false
17614 5464 Acked Read 1 Done Sent Done false
17615 5465 Acked Write 4 Done Sent Done false
17616 5466 Acked Write 1 Done Sent Done false
17617 5467 Acked Read 1 Done Sent Done false
17618 5468 Acked Read 8 Done Sent Done false
17619 5469 Acked Flush 0 Done Sent Done false
17620 5470 Acked Read 8 Done Sent Done false
17621 5471 Acked Read 2 Done Sent Done false
17622 5472 Acked Write 1 Done Sent Done false
17623 5473 Acked Write 2 Done Sent Done false
17624 5474 Acked Flush 0 Done Sent Done false
17625 5475 Acked Read 10 Done Sent Done false
17626 5476 Acked Write 4 Done Sent Done false
17627 5477 Acked Write 8 Done Sent Done false
17628 5478 Acked Read 8 Done Sent Done false
17629 5479 Acked Read 8 Done Sent Done false
17630 5480 Acked Read 1 Done Sent Done false
17631 5481 Acked Read 4 Done Sent Done false
17632 5482 Acked Read 4 Done Sent Done false
17633 5483 Acked Write 6 Done Sent Done false
17634 5484 Acked Write 6 Done Sent Done false
17635 5485 Acked Read 10 Done Sent Done false
17636 5486 Acked Write 6 Done Sent Done false
17637 5487 Acked Read 1 Done Sent Done false
17638 5488 Acked Read 4 Done Sent Done false
17639 5489 Acked Read 3 Done Sent Done false
17640 5490 Acked Read 9 Done Sent Done false
17641 5491 Acked Write 4 Done Sent Done false
17642 5492 Acked Write 2 Done Sent Done false
17643 5493 Acked Write 10 Done Sent Done false
17644 5494 Acked Write 1 Done Sent Done false
17645 5495 Acked Read 9 Done Sent Done false
17646 5496 Acked Write 7 Done Sent Done false
17647 5497 Acked Read 3 Done Sent Done false
17648 5498 Acked Write 4 Done Sent Done false
17649 5499 Acked Write 3 Done Sent Done false
17650 5500 Acked Read 6 Done Sent Done false
17651 5501 Acked Write 7 Done Sent Done false
17652 5502 Acked Read 6 Done Sent Done false
17653 5503 Acked Read 1 Done Sent Done false
17654 5504 Acked Write 7 Done Sent Done false
17655 5505 Acked Read 8 Done Sent Done false
17656 5506 Acked Write 2 Done Sent Done false
17657 5507 Acked Read 4 Done Sent Done false
17658 5508 Acked Write 8 Done Sent Done false
17659 5509 Acked Read 4 Done Sent Done false
17660 5510 Acked Write 3 Done Sent Done false
17661 5511 Acked Write 10 Done Sent Done false
17662 5512 Acked Read 6 Done Sent Done false
17663 5513 Acked Read 10 Done Sent Done false
17664 5514 Acked Read 7 Done Sent Done false
17665 5515 Acked Write 7 Done Sent Done false
17666 5516 Acked Write 8 Done Sent Done false
17667 5517 Acked Write 7 Done Sent Done false
17668 5518 Acked Write 10 Done Sent Done false
17669 5519 Acked Write 1 Done Sent Done false
17670 5520 Acked Read 7 Done Sent Done false
17671 5521 Acked Read 1 Done Sent Done false
17672 5522 Acked Read 2 Done Sent Done false
17673 5523 Acked Write 9 Done Sent Done false
17674 5524 Acked Read 2 Done Sent Done false
17675 5525 Acked Read 7 Done Sent Done false
17676 5526 Acked Write 10 Done Sent Done false
17677 5527 Acked Flush 0 Done Sent Done false
17678 STATES DS:0 DS:1 DS:2 TOTAL
17679 Sent 0 302 0 302
17680 Done 302 0 302 604
17681 Skipped 0 0 0 0
17682 Error 0 0 0 0
17683 Last Flush: 5527 5225 5527
17684 Downstairs last five completed: 5225 5224 5223 5222 5221
17685 Upstairs last five completed: 5527 5526 5525 5524 5523
17686 CLIENT: Up:0 ds:302 act:2
17687 ----------------------------------------------------------------
17688 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17689 States: Active Offline Active
17690 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17691 5226 Acked Read 3 Done Sent Done false
17692 5227 Acked Read 10 Done Sent Done false
17693 5228 Acked Write 8 Done Sent Done false
17694 5229 Acked Read 3 Done Sent Done false
17695 5230 Acked Read 7 Done Sent Done false
17696 5231 Acked Read 9 Done Sent Done false
17697 5232 Acked Write 8 Done Sent Done false
17698 5233 Acked Write 4 Done Sent Done false
17699 5234 Acked Read 6 Done Sent Done false
17700 5235 Acked Read 3 Done Sent Done false
17701 5236 Acked Flush 0 Done Sent Done false
17702 5237 Acked Read 5 Done Sent Done false
17703 5238 Acked Write 4 Done Sent Done false
17704 5239 Acked Write 4 Done Sent Done false
17705 5240 Acked Read 6 Done Sent Done false
17706 5241 Acked Read 7 Done Sent Done false
17707 5242 Acked Read 4 Done Sent Done false
17708 5243 Acked Flush 0 Done Sent Done false
17709 5244 Acked Write 3 Done Sent Done false
17710 5245 Acked Write 9 Done Sent Done false
17711 5246 Acked Read 8 Done Sent Done false
17712 5247 Acked Read 3 Done Sent Done false
17713 5248 Acked Write 3 Done Sent Done false
17714 5249 Acked Read 6 Done Sent Done false
17715 5250 Acked Read 9 Done Sent Done false
17716 5251 Acked Write 5 Done Sent Done false
17717 5252 Acked Write 2 Done Sent Done false
17718 5253 Acked Read 4 Done Sent Done false
17719 5254 Acked Write 6 Done Sent Done false
17720 5255 Acked Write 9 Done Sent Done false
17721 5256 Acked Read 7 Done Sent Done false
17722 5257 Acked Write 5 Done Sent Done false
17723 5258 Acked Read 6 Done Sent Done false
17724 5259 Acked Read 6 Done Sent Done false
17725 5260 Acked Read 3 Done Sent Done false
17726 5261 Acked Read 2 Done Sent Done false
17727 5262 Acked Write 10 Done Sent Done false
17728 5263 Acked Write 1 Done Sent Done false
17729 5264 Acked Write 1 Done Sent Done false
17730 5265 Acked Read 9 Done Sent Done false
17731 5266 Acked Write 10 Done Sent Done false
17732 5267 Acked Read 1 Done Sent Done false
17733 5268 Acked Read 7 Done Sent Done false
17734 5269 Acked Flush 0 Done Sent Done false
17735 5270 Acked Read 1 Done Sent Done false
17736 5271 Acked Write 1 Done Sent Done false
17737 5272 Acked Read 3 Done Sent Done false
17738 5273 Acked Read 5 Done Sent Done false
17739 5274 Acked Write 9 Done Sent Done false
17740 5275 Acked Read 10 Done Sent Done false
17741 5276 Acked Read 7 Done Sent Done false
17742 5277 Acked Read 1 Done Sent Done false
17743 5278 Acked Write 6 Done Sent Done false
17744 5279 Acked Write 5 Done Sent Done false
17745 5280 Acked Write 7 Done Sent Done false
17746 5281 Acked Read 5 Done Sent Done false
17747 5282 Acked Read 9 Done Sent Done false
17748 5283 Acked Write 4 Done Sent Done false
17749 5284 Acked Write 8 Done Sent Done false
17750 5285 Acked Write 7 Done Sent Done false
17751 5286 Acked Read 1 Done Sent Done false
17752 5287 Acked Read 8 Done Sent Done false
17753 5288 Acked Flush 0 Done Sent Done false
17754 5289 Acked Write 10 Done Sent Done false
17755 5290 Acked Read 3 Done Sent Done false
17756 5291 Acked Read 10 Done Sent Done false
17757 5292 Acked Read 2 Done Sent Done false
17758 5293 Acked Read 8 Done Sent Done false
17759 5294 Acked Write 8 Done Sent Done false
17760 5295 Acked Read 1 Done Sent Done false
17761 5296 Acked Read 5 Done Sent Done false
17762 5297 Acked Flush 0 Done Sent Done false
17763 5298 Acked Read 1 Done Sent Done false
17764 5299 Acked Write 6 Done Sent Done false
17765 5300 Acked Flush 0 Done Sent Done false
17766 5301 Acked Write 2 Done Sent Done false
17767 5302 Acked Read 1 Done Sent Done false
17768 5303 Acked Read 7 Done Sent Done false
17769 5304 Acked Write 10 Done Sent Done false
17770 5305 Acked Read 9 Done Sent Done false
17771 5306 Acked Read 4 Done Sent Done false
17772 5307 Acked Read 8 Done Sent Done false
17773 5308 Acked Write 10 Done Sent Done false
17774 5309 Acked Write 4 Done Sent Done false
17775 5310 Acked Read 10 Done Sent Done false
17776 5311 Acked Write 4 Done Sent Done false
17777 5312 Acked Write 10 Done Sent Done false
17778 5313 Acked Read 10 Done Sent Done false
17779 5314 Acked Write 1 Done Sent Done false
17780 5315 Acked Read 7 Done Sent Done false
17781 5316 Acked Read 10 Done Sent Done false
17782 5317 Acked Read 3 Done Sent Done false
17783 5318 Acked Read 5 Done Sent Done false
17784 5319 Acked Read 9 Done Sent Done false
17785 5320 Acked Read 9 Done Sent Done false
17786 5321 Acked Read 5 Done Sent Done false
17787 5322 Acked Read 4 Done Sent Done false
17788 5323 Acked Read 3 Done Sent Done false
17789 5324 Acked Read 10 Done Sent Done false
17790 5325 Acked Flush 0 Done Sent Done false
17791 5326 Acked Read 4 Done Sent Done false
17792 5327 Acked Write 9 Done Sent Done false
17793 5328 Acked Read 1 Done Sent Done false
17794 5329 Acked Write 10 Done Sent Done false
17795 5330 Acked Read 5 Done Sent Done false
17796 5331 Acked Read 2 Done Sent Done false
17797 5332 Acked Write 2 Done Sent Done false
17798 5333 Acked Flush 0 Done Sent Done false
17799 5334 Acked Read 2 Done Sent Done false
17800 5335 Acked Read 8 Done Sent Done false
17801 5336 Acked Write 10 Done Sent Done false
17802 5337 Acked Read 2 Done Sent Done false
17803 5338 Acked Read 5 Done Sent Done false
17804 5339 Acked Read 8 Done Sent Done false
17805 5340 Acked Read 3 Done Sent Done false
17806 5341 Acked Read 10 Done Sent Done false
17807 5342 Acked Read 6 Done Sent Done false
17808 5343 Acked Read 6 Done Sent Done false
17809 5344 Acked Read 2 Done Sent Done false
17810 5345 Acked Write 7 Done Sent Done false
17811 5346 Acked Read 5 Done Sent Done false
17812 5347 Acked Write 8 Done Sent Done false
17813 5348 Acked Write 4 Done Sent Done false
17814 5349 Acked Read 5 Done Sent Done false
17815 5350 Acked Write 4 Done Sent Done false
17816 5351 Acked Write 1 Done Sent Done false
17817 5352 Acked Write 9 Done Sent Done false
17818 5353 Acked Read 9 Done Sent Done false
17819 5354 Acked Read 9 Done Sent Done false
17820 5355 Acked Read 7 Done Sent Done false
17821 5356 Acked Read 4 Done Sent Done false
17822 5357 Acked Read 9 Done Sent Done false
17823 5358 Acked Read 5 Done Sent Done false
17824 5359 Acked Write 6 Done Sent Done false
17825 5360 Acked Read 10 Done Sent Done false
17826 5361 Acked Read 3 Done Sent Done false
17827 5362 Acked Write 3 Done Sent Done false
17828 5363 Acked Read 8 Done Sent Done false
17829 5364 Acked Write 1 Done Sent Done false
17830 5365 Acked Write 3 Done Sent Done false
17831 5366 Acked Read 2 Done Sent Done false
17832 5367 Acked Read 8 Done Sent Done false
17833 5368 Acked Write 2 Done Sent Done false
17834 5369 Acked Read 8 Done Sent Done false
17835 5370 Acked Read 5 Done Sent Done false
17836 5371 Acked Read 6 Done Sent Done false
17837 5372 Acked Read 3 Done Sent Done false
17838 5373 Acked Read 5 Done Sent Done false
17839 5374 Acked Read 6 Done Sent Done false
17840 5375 Acked Read 9 Done Sent Done false
17841 5376 Acked Read 8 Done Sent Done false
17842 5377 Acked Read 2 Done Sent Done false
17843 5378 Acked Read 7 Done Sent Done false
17844 5379 Acked Flush 0 Done Sent Done false
17845 5380 Acked Read 1 Done Sent Done false
17846 5381 Acked Read 1 Done Sent Done false
17847 5382 Acked Write 8 Done Sent Done false
17848 5383 Acked Write 9 Done Sent Done false
17849 5384 Acked Write 5 Done Sent Done false
17850 5385 Acked Read 1 Done Sent Done false
17851 5386 Acked Flush 0 Done Sent Done false
17852 5387 Acked Read 8 Done Sent Done false
17853 5388 Acked Read 6 Done Sent Done false
17854 5389 Acked Write 9 Done Sent Done false
17855 5390 Acked Read 10 Done Sent Done false
17856 5391 Acked Write 9 Done Sent Done false
17857 5392 Acked Flush 0 Done Sent Done false
17858 5393 Acked Read 10 Done Sent Done false
17859 5394 Acked Flush 0 Done Sent Done false
17860 5395 Acked Read 3 Done Sent Done false
17861 5396 Acked Write 3 Done Sent Done false
17862 5397 Acked Read 7 Done Sent Done false
17863 5398 Acked Read 6 Done Sent Done false
17864 5399 Acked Read 2 Done Sent Done false
17865 5400 Acked Read 7 Done Sent Done false
17866 5401 Acked Write 5 Done Sent Done false
17867 5402 Acked Write 6 Done Sent Done false
17868 5403 Acked Read 2 Done Sent Done false
17869 5404 Acked Read 1 Done Sent Done false
17870 5405 Acked Write 4 Done Sent Done false
17871 5406 Acked Flush 0 Done Sent Done false
17872 5407 Acked Write 5 Done Sent Done false
17873 5408 Acked Read 4 Done Sent Done false
17874 5409 Acked Read 5 Done Sent Done false
17875 5410 Acked Read 3 Done Sent Done false
17876 5411 Acked Read 7 Done Sent Done false
17877 5412 Acked Write 9 Done Sent Done false
17878 5413 Acked Write 4 Done Sent Done false
17879 5414 Acked Write 9 Done Sent Done false
17880 5415 Acked Read 8 Done Sent Done false
17881 5416 Acked Write 4 Done Sent Done false
17882 5417 Acked Read 4 Done Sent Done false
17883 5418 Acked Write 1 Done Sent Done false
17884 5419 Acked Read 10 Done Sent Done false
17885 5420 Acked Read 10 Done Sent Done false
17886 5421 Acked Read 5 Done Sent Done false
17887 5422 Acked Write 3 Done Sent Done false
17888 5423 Acked Write 6 Done Sent Done false
17889 5424 Acked Write 4 Done Sent Done false
17890 5425 Acked Read 5 Done Sent Done false
17891 5426 Acked Write 6 Done Sent Done false
17892 5427 Acked Read 10 Done Sent Done false
17893 5428 Acked Read 7 Done Sent Done false
17894 5429 Acked Write 10 Done Sent Done false
17895 5430 Acked Read 5 Done Sent Done false
17896 5431 Acked Write 8 Done Sent Done false
17897 5432 Acked Write 2 Done Sent Done false
17898 5433 Acked Flush 0 Done Sent Done false
17899 5434 Acked Write 3 Done Sent Done false
17900 5435 Acked Read 3 Done Sent Done false
17901 5436 Acked Read 4 Done Sent Done false
17902 5437 Acked Write 5 Done Sent Done false
17903 5438 Acked Write 4 Done Sent Done false
17904 5439 Acked Flush 0 Done Sent Done false
17905 5440 Acked Read 4 Done Sent Done false
17906 5441 Acked Write 4 Done Sent Done false
17907 5442 Acked Read 3 Done Sent Done false
17908 5443 Acked Flush 0 Done Sent Done false
17909 5444 Acked Read 6 Done Sent Done false
17910 5445 Acked Write 2 Done Sent Done false
17911 5446 Acked Write 3 Done Sent Done false
17912 5447 Acked Read 10 Done Sent Done false
17913 5448 Acked Read 4 Done Sent Done false
17914 5449 Acked Write 4 Done Sent Done false
17915 5450 Acked Read 10 Done Sent Done false
17916 5451 Acked Read 2 Done Sent Done false
17917 5452 Acked Read 3 Done Sent Done false
17918 5453 Acked Write 9 Done Sent Done false
17919 5454 Acked Read 8 Done Sent Done false
17920 5455 Acked Read 1 Done Sent Done false
17921 5456 Acked Write 4 Done Sent Done false
17922 5457 Acked Read 2 Done Sent Done false
17923 5458 Acked Read 10 Done Sent Done false
17924 5459 Acked Flush 0 Done Sent Done false
17925 5460 Acked Read 2 Done Sent Done false
17926 5461 Acked Flush 0 Done Sent Done false
17927 5462 Acked Write 5 Done Sent Done false
17928 5463 Acked Read 2 Done Sent Done false
17929 5464 Acked Read 1 Done Sent Done false
17930 5465 Acked Write 4 Done Sent Done false
17931 5466 Acked Write 1 Done Sent Done false
17932 5467 Acked Read 1 Done Sent Done false
17933 5468 Acked Read 8 Done Sent Done false
17934 5469 Acked Flush 0 Done Sent Done false
17935 5470 Acked Read 8 Done Sent Done false
17936 5471 Acked Read 2 Done Sent Done false
17937 5472 Acked Write 1 Done Sent Done false
17938 5473 Acked Write 2 Done Sent Done false
17939 5474 Acked Flush 0 Done Sent Done false
17940 5475 Acked Read 10 Done Sent Done false
17941 5476 Acked Write 4 Done Sent Done false
17942 5477 Acked Write 8 Done Sent Done false
17943 5478 Acked Read 8 Done Sent Done false
17944 5479 Acked Read 8 Done Sent Done false
17945 5480 Acked Read 1 Done Sent Done false
17946 5481 Acked Read 4 Done Sent Done false
17947 5482 Acked Read 4 Done Sent Done false
17948 5483 Acked Write 6 Done Sent Done false
17949 5484 Acked Write 6 Done Sent Done false
17950 5485 Acked Read 10 Done Sent Done false
17951 5486 Acked Write 6 Done Sent Done false
17952 5487 Acked Read 1 Done Sent Done false
17953 5488 Acked Read 4 Done Sent Done false
17954 5489 Acked Read 3 Done Sent Done false
17955 5490 Acked Read 9 Done Sent Done false
17956 5491 Acked Write 4 Done Sent Done false
17957 5492 Acked Write 2 Done Sent Done false
17958 5493 Acked Write 10 Done Sent Done false
17959 5494 Acked Write 1 Done Sent Done false
17960 5495 Acked Read 9 Done Sent Done false
17961 5496 Acked Write 7 Done Sent Done false
17962 5497 Acked Read 3 Done Sent Done false
17963 5498 Acked Write 4 Done Sent Done false
17964 5499 Acked Write 3 Done Sent Done false
17965 5500 Acked Read 6 Done Sent Done false
17966 5501 Acked Write 7 Done Sent Done false
17967 5502 Acked Read 6 Done Sent Done false
17968 5503 Acked Read 1 Done Sent Done false
17969 5504 Acked Write 7 Done Sent Done false
17970 5505 Acked Read 8 Done Sent Done false
17971 5506 Acked Write 2 Done Sent Done false
17972 5507 Acked Read 4 Done Sent Done false
17973 5508 Acked Write 8 Done Sent Done false
17974 5509 Acked Read 4 Done Sent Done false
17975 5510 Acked Write 3 Done Sent Done false
17976 5511 Acked Write 10 Done Sent Done false
17977 5512 Acked Read 6 Done Sent Done false
17978 5513 Acked Read 10 Done Sent Done false
17979 5514 Acked Read 7 Done Sent Done false
17980 5515 Acked Write 7 Done Sent Done false
17981 5516 Acked Write 8 Done Sent Done false
17982 5517 Acked Write 7 Done Sent Done false
17983 5518 Acked Write 10 Done Sent Done false
17984 5519 Acked Write 1 Done Sent Done false
17985 5520 Acked Read 7 Done Sent Done false
17986 5521 Acked Read 1 Done Sent Done false
17987 5522 Acked Read 2 Done Sent Done false
17988 5523 Acked Write 9 Done Sent Done false
17989 5524 Acked Read 2 Done Sent Done false
17990 5525 Acked Read 7 Done Sent Done false
17991 5526 Acked Write 10 Done Sent Done false
17992 5527 Acked Flush 0 Done Sent Done false
17993 STATES DS:0 DS:1 DS:2 TOTAL
17994 Sent 0 302 0 302
17995 Done 302 0 302 604
17996 Skipped 0 0 0 0
17997 Error 0 0 0 0
17998 Last Flush: 5527 5225 5527
17999 Downstairs last five completed: 5225 5224 5223 5222 5221
18000 Upstairs last five completed: 5527 5526 5525 5524 5523
18001 CLIENT: Up:0 ds:302 act:2
180022024-11-19T17:11:47.169ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180032024-11-19T17:11:47.169ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49983) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180042024-11-19T17:11:47.171ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180052024-11-19T17:11:47.171ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180062024-11-19T17:11:47.171ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180072024-11-19T17:11:47.171ZINFOcrucible: send last flush ID to this DS: 5225 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180082024-11-19T17:11:47.172ZINFOcrucible: Replied this last flush ID: 5225 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180092024-11-19T17:11:47.172ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180102024-11-19T17:11:47.172ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180112024-11-19T17:11:47.172ZINFOcrucible: [1] client re-new 302 jobs since flush 5225 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
180122024-11-19T17:11:47.172ZINFOcrucible: [1] Replayed 302 jobs since flush: 5225 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
18013 ----------------------------------------------------------------
18014 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18015 Downstairs last five completed: 5527 5526 5525 5524 5523
18016 Upstairs last five completed: 5527 5526 5525 5524 5523
18017 CLIENT: Up:0 ds:0 act:3
18018 Replay: All jobs finished
18019 Main task has work to do, go find it
18020 got dsc Stop(1)
18021 stop 1
18022 Nov 19 17:11:47.423 INFO[8820] Got stop action so:false kr:false
18023 request completed, latency_us: 395, response_code: 204, uri: /stop/cid/1, method: POST, req_id: a6e3c032-af30-40ff-9617-b0f30428be75, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
180242024-11-19T17:11:47.424ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180252024-11-19T17:11:47.424ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180262024-11-19T17:11:47.424ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
18027 Nov 19 17:11:47.424 INFO request completed, latency_us: 134, response_code: 200, uri: /state/cid/1, method: GET, req_id: 70d7dce5-596d-4e0f-a68e-781d8cce4d76, remote_addr: 127.0.0.1:64175, local_addr{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:11:47.424807345Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
18028 : 127.0.0.1:9998
180292024-11-19T17:11:47.424ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
180302024-11-19T17:11:47.424ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
18031 [8820] Exited with: ExitStatus(unix_wait_status(9))
18032 [8820][1] reports Exit
18033 Nov 19 17:11:51.427 INFO request completed, latency_us: 272, response_code: 200, uri: /state/cid/1, method: GET, req_id: 59478f2a-fef4-40a9-883f-4fc271fa0370, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
18034 001/300 Read block 3199 len 20480
18035 002/300 Read block 4714 len 28672
18036 003/300 Write block 3799 len 4096 data: 5
18037 004/300 Write block 4780 len 28672 data: 4 3 3 2 2 2 3
18038 005/300 Read block 2618 len 32768
18039 006/300 Flush
18040 007/300 Flush
18041 008/300 Read block 2355 len 12288
18042 009/300 Read block 4443 len 4096
18043 010/300 Read block 2415 len 12288
18044 011/300 Read block 2228 len 8192
18045 012/300 Write block 897 len 12288 data: 3 3 2
18046 013/300 Read block 2870 len 16384
18047 014/300 Read block 1984 len 20480
18048 015/300 Write block 4343 len 40960 data: 2 2 2 2 2 2 2 2 2 2
18049 016/300 Read block 2435 len 32768
18050 017/300 Flush
18051 018/300 Write block 3724 len 36864 data: 4 4 5 5 6 8 7 6 6
18052 019/300 Write block 715 len 16384 data: 2 2 2 2
18053 020/300 Flush
18054 021/300 Read block 3777 len 40960
18055 022/300 Read block 296 len 40960
18056 023/300 Write block 3689 len 16384 data: 2 2 3 3
18057 024/300 Read block 1551 len 8192
18058 025/300 Read block 4328 len 12288
18059 026/300 Write block 4927 len 12288 data: 7 6 6
18060 027/300 Read block 91 len 24576
18061 028/300 Write block 8 len 20480 data: 3 3 3 2 2
18062 029/300 Read block 756 len 4096
18063 030/300 Write block 992 len 8192 data: 6 6
18064 031/300 Write block 250 len 28672 data: 4 4 4 3 3 3 3
18065 032/300 Read block 3408 len 20480
18066 033/300 Flush
18067 034/300 Read block 643 len 20480
18068 035/300 Write block 2640 len 20480 data: 4 3 3 3 4
18069 036/300 Write block 1149 len 36864 data: 5 4 5 4 4 4 4 3 4
18070 037/300 Read block 1299 len 8192
18071 038/300 Write block 4360 len 20480 data: 4 4 4 4 4
18072 039/300 Write block 4161 len 24576 data: 4 4 4 4 3 2
18073 040/300 Read block 2477 len 4096
18074 041/300 Read block 3407 len 40960
18075 042/300 Read block 2221 len 16384
18076 043/300 Write block 4991 len 12288 data: 3 3 3
18077 044/300 Read block 917 len 8192
18078 045/300 Flush
18079 046/300 Read block 2136 len 20480
18080 047/300 Write block 4861 len 40960 data: 3 3 3 3 3 3 3 3 2 3
18081 048/300 Write block 3738 len 12288 data: 7 9 10
18082 049/300 Read block 2955 len 16384
18083 050/300 Write block 1225 len 20480 data: 3 4 4 5 4
18084 051/300 Read block 3180 len 8192
18085 052/300 Write block 2025 len 12288 data: 7 8 6
18086 053/300 Read block 4955 len 36864
18087 054/300 Write block 3007 len 32768 data: 2 2 2 2 2 2 3 4
18088 055/300 Read block 2163 len 8192
18089 056/300 Read block 2466 len 20480
18090 057/300 Read block 1678 len 16384
18091 058/300 Read block 824 len 32768
18092 059/300 Write block 3503 len 16384 data: 2 2 2 2
18093 060/300 Write block 1550 len 40960 data: 5 5 3 3 3 3 4 4 4 5
18094 061/300 Read block 291 len 36864
18095 062/300 Write block 364 len 24576 data: 5 5 5 5 5 3
18096 063/300 Read block 3931 len 36864
18097 064/300 Write block 3755 len 40960 data: 5 6 5 6 5 5 5 5 5 4
18098 065/300 Write block 910 len 24576 data: 2 2 2 3 3 3
18099 066/300 Write block 3148 len 12288 data: 5 5 5
18100 067/300 Flush
18101 068/300 Read block 877 len 28672
18102 069/300 Read block 1776 len 8192
18103 070/300 Write block 580 len 12288 data: 3 2 2
18104 071/300 Write block 3854 len 28672 data: 3 5 5 5 6 6 6
18105 072/300 Flush
18106 073/300 Read block 2214 len 4096
18107 074/300 Read block 2286 len 20480
18108 075/300 Write block 3348 len 32768 data: 2 2 2 2 2 3 3 4
18109 076/300 Read block 3609 len 28672
18110 077/300 Write block 870 len 36864 data: 5 5 4 4 5 5 5 4 5
18111 078/300 Read block 3015 len 36864
18112 079/300 Write block 3174 len 36864 data: 3 3 3 3 3 3 2 3 3
18113 080/300 Flush
18114 081/300 Read block 4337 len 12288
18115 082/300 Write block 2856 len 16384 data: 3 4 3 4
18116 083/300 Write block 939 len 32768 data: 6 5 5 5 6 4 5 5
18117 084/300 Read block 1487 len 4096
18118 085/300 Write block 1383 len 36864 data: 3 2 2 2 2 2 4 4 5
18119 086/300 Read block 3633 len 16384
18120 087/300 Write block 3187 len 36864 data: 4 4 4 3 2 2 2 2 2
18121 088/300 Read block 3816 len 28672
18122 089/300 Flush
18123 090/300 Write block 4290 len 24576 data: 4 3 3 3 3 3
18124 091/300 Flush
18125 092/300 Read block 1976 len 28672
18126 093/300 Read block 4820 len 40960
18127 094/300 Read block 2816 len 32768
18128 095/300 Write block 3493 len 4096 data: 3
18129 096/300 Flush
18130 097/300 Write block 4793 len 24576 data: 4 4 5 4 4 4
18131 098/300 Read block 2576 len 4096
18132 099/300 Read block 3360 len 12288
18133 100/300 Read block 1872 len 28672
18134 101/300 Write block 220 len 36864 data: 4 4 4 4 4 3 3 3 3
18135 102/300 Read block 3495 len 40960
18136 103/300 Write block 1607 len 28672 data: 7 6 8 9 10 9 9
18137 104/300 Read block 996 len 16384
18138 105/300 Read block 1997 len 40960
18139 106/300 Write block 4550 len 28672 data: 4 4 5 5 5 5 4
18140 107/300 Write block 1101 len 8192 data: 4 3
18141 108/300 Write block 4420 len 4096 data: 3
18142 109/300 Write block 4660 len 12288 data: 4 5 4
18143 110/300 Read block 1644 len 20480
18144 111/300 Flush
18145 112/300 Read block 3988 len 16384
18146 113/300 Read block 2649 len 24576
18147 114/300 Read block 3695 len 32768
18148 115/300 Write block 4755 len 32768 data: 3 2 2 3 4 4 4 5
18149 116/300 Write block 1985 len 20480 data: 2 2 2 4 3
18150 117/300 Write block 276 len 20480 data: 4 7 8 8 7
18151 118/300 Write block 4183 len 32768 data: 4 5 4 4 5 6 7 5
18152 119/300 Flush
18153 120/300 Write block 2796 len 20480 data: 4 3 3 2 3
18154 121/300 Read block 3389 len 40960
18155 122/300 Read block 4954 len 12288
18156 123/300 Flush
18157 124/300 Write block 4587 len 24576 data: 4 5 5 5 5 5
18158 125/300 Write block 353 len 4096 data: 4
18159 126/300 Write block 2587 len 8192 data: 3 3
18160 127/300 Read block 3996 len 36864
18161 128/300 Read block 2763 len 40960
18162 129/300 Write block 3080 len 4096 data: 4
18163 130/300 Write block 710 len 12288 data: 4 3 3
18164 131/300 Read block 4631 len 20480
18165 132/300 Write block 3941 len 16384 data: 2 2 3 3
18166 133/300 Write block 394 len 4096 data: 5
18167 134/300 Read block 698 len 20480
18168 135/300 Read block 1096 len 20480
18169 136/300 Read block 1226 len 4096
18170 137/300 Read block 4986 len 12288
18171 138/300 Write block 4496 len 8192 data: 5 5
18172 139/300 Read block 2105 len 40960
18173 140/300 Read block 2981 len 8192
18174 141/300 Read block 1805 len 40960
18175 142/300 Read block 616 len 36864
18176 143/300 Read block 876 len 36864
18177 144/300 Read block 4320 len 8192
18178 145/300 Read block 2096 len 24576
18179 146/300 Read block 3337 len 12288
18180 147/300 Write block 838 len 16384 data: 4 3 2 2
18181 148/300 Write block 3512 len 40960 data: 3 3 3 3 2 2 2 2 2 2
18182 149/300 Write block 545 len 16384 data: 3 4 4 4
18183 150/300 Flush
18184 151/300 Write block 2598 len 28672 data: 2 2 2 2 3 3 4
18185 152/300 Flush
18186 153/300 Write block 255 len 12288 data: 4 4 3
18187 154/300 Read block 447 len 8192
18188 155/300 Read block 173 len 8192
18189 156/300 Write block 2977 len 4096 data: 4
18190 157/300 Read block 1653 len 12288
18191 158/300 Read block 1140 len 40960
18192 159/300 Write block 663 len 20480 data: 5 6 5 5 8
18193 160/300 Write block 3762 len 28672 data: 6 6 5 5 3 3 3
18194 161/300 Read block 4715 len 12288
18195 162/300 Read block 2009 len 24576
18196 163/300 Write block 3823 len 28672 data: 7 7 6 7 6 5 5
18197 164/300 Read block 2996 len 28672
18198 165/300 Write block 1561 len 40960 data: 4 4 4 4 4 4 3 3 3 3
18199 166/300 Read block 816 len 36864
18200 167/300 Write block 4259 len 4096 data: 3
18201 168/300 Read block 1265 len 16384
18202 169/300 Read block 3825 len 40960
18203 170/300 Read block 4841 len 24576
18204 171/300 Flush
18205 172/300 Read block 333 len 20480
18206 173/300 Write block 1020 len 24576 data: 4 4 3 4 4 4
18207 174/300 Read block 1236 len 12288
18208 175/300 Flush
18209 176/300 Read block 3315 len 12288
18210 177/300 Write block 3345 len 24576 data: 3 2 2 3 3 3
18211 178/300 Read block 1735 len 36864
18212 179/300 Read block 4654 len 32768
18213 180/300 Read block 1855 len 36864
18214 181/300 Read block 472 len 40960
18215 182/300 Write block 4084 len 28672 data: 5 5 5 5 3 5 4
18216 183/300 Flush
18217 184/300 Write block 4067 len 8192 data: 3 3
18218 185/300 Read block 789 len 32768
18219 186/300 Flush
18220 187/300 Flush
18221 188/300 Read block 535 len 40960
18222 189/300 Read block 1153 len 36864
18223 190/300 Write block 3209 len 32768 data: 3 3 2 3 2 4 5 5
18224 191/300 Read block 2664 len 4096
18225 192/300 Write block 4310 len 16384 data: 2 2 2 2
18226 193/300 Write block 4902 len 40960 data: 3 3 2 2 2 3 3 5 5 5
18227 194/300 Read block 2456 len 36864
18228 195/300 Write block 3550 len 28672 data: 3 3 4 3 3 2 4
18229 196/300 Read block 1782 len 32768
18230 197/300 Write block 3685 len 16384 data: 2 2 3 3
18231 198/300 Read block 3339 len 28672
18232 199/300 Flush
18233 200/300 Write block 2972 len 20480 data: 2 2 4 4 4
18234 201/300 Flush
18235 202/300 Read block 4036 len 8192
18236 203/300 Read block 1653 len 36864
18237 204/300 Read block 70 len 36864
18238 205/300 Read block 4629 len 8192
18239 206/300 Read block 531 len 12288
18240 207/300 Read block 2396 len 40960
18241 208/300 Read block 1565 len 40960
18242 209/300 Write block 3954 len 28672 data: 5 5 5 5 4 4 4
18243 210/300 Read block 3928 len 20480
18244 211/300 Write block 4072 len 8192 data: 4 4
18245 212/300 Flush
18246 213/300 Write block 1804 len 8192 data: 3 3
18247 214/300 Flush
18248 215/300 Read block 2597 len 12288
18249 216/300 Write block 1045 len 16384 data: 4 5 4 4
18250 217/300 Write block 4601 len 8192 data: 3 3
18251 218/300 Read block 2193 len 20480
18252 219/300 Read block 8 len 12288
18253 220/300 Flush
18254 221/300 Write block 3747 len 32768 data: 6 6 6 5 5 7 7 6
18255 222/300 Write block 2124 len 24576 data: 3 3 3 3 3 4
18256 223/300 Read block 938 len 24576
18257 224/300 Read block 3671 len 24576
18258 225/300 Read block 2895 len 16384
18259 226/300 Read block 785 len 40960
18260 227/300 Flush
18261 228/300 Read block 4868 len 32768
18262 229/300 Flush
18263 230/300 Write block 4454 len 12288 data: 3 3 3
18264 231/300 Read block 3006 len 28672
18265 232/300 Read block 4498 len 40960
18266 233/300 Read block 1358 len 20480
18267 234/300 Read block 637 len 36864
18268 235/300 Read block 2624 len 4096
18269 236/300 Write block 1486 len 32768 data: 2 2 3 2 2 3 3 4
18270 237/300 Write block 2709 len 28672 data: 6 4 4 4 4 4 4
18271 238/300 Read block 36 len 16384
18272 239/300 Read block 1254 len 32768
18273 240/300 Flush
18274 241/300 Read block 3621 len 32768
18275 242/300 Read block 4262 len 36864
18276 243/300 Read block 2396 len 16384
18277 244/300 Read block 4537 len 20480
18278 245/300 Write block 4310 len 40960 data: 3 3 3 3 2 2 2 4 3 3
18279 246/300 Write block 3273 len 8192 data: 5 4
18280 247/300 Flush
18281 248/300 Read block 4636 len 16384
18282 249/300 Write block 1040 len 24576 data: 3 4 4 4 4 5
18283 250/300 Read block 3761 len 16384
18284 251/300 Write block 3618 len 4096 data: 5
18285 252/300 Read block 1685 len 4096
18286 253/300 Read block 4056 len 20480
18287 254/300 Read block 1705 len 16384
18288 255/300 Read block 481 len 8192
18289 256/300 Flush
18290 257/300 Write block 213 len 16384 data: 4 4 5 5
18291 258/300 Write block 4247 len 40960 data: 3 3 3 3 3 4 4 5 4 4
18292 259/300 Read block 463 len 36864
18293 260/300 Write block 526 len 36864 data: 3 4 4 3 3 4 3 3 3
18294 261/300 Read block 2020 len 20480
18295 262/300 Read block 4567 len 28672
18296 263/300 Write block 585 len 36864 data: 3 4 4 5 4 3 3 3 4
18297 264/300 Write block 2138 len 36864 data: 4 3 3 4 5 5 4 4 5
18298 265/300 Read block 1579 len 32768
18299 266/300 Write block 3281 len 8192 data: 5 5
18300 267/300 Write block 672 len 24576 data: 8 8 8 7 7 6
18301 268/300 Read block 4184 len 12288
18302 269/300 Read block 1550 len 36864
18303 270/300 Write block 3281 len 32768 data: 6 6 6 6 6 5 4 3
18304 271/300 Write block 1566 len 8192 data: 5 4
18305 272/300 Read block 1195 len 36864
18306 273/300 Read block 3860 len 36864
18307 274/300 Write block 4446 len 8192 data: 2 2
18308 275/300 Read block 105 len 20480
18309 276/300 Write block 3241 len 40960 data: 5 5 5 4 6 5 5 5 6 4
18310 277/300 Write block 3303 len 32768 data: 2 3 4 4 4 4 4 3
18311 278/300 Read block 2125 len 4096
18312 279/300 Write block 1771 len 16384 data: 3 3 3 3
18313 280/300 Write block 2944 len 12288 data: 2 4 4
18314 281/300 Read block 2068 len 32768
18315 282/300 Read block 2070 len 12288
18316 283/300 Read block 3625 len 24576
18317 284/300 Read block 2243 len 16384
18318 285/300 Read block 340 len 4096
18319 286/300 Write block 94 len 28672 data: 3 3 4 4 5 5 7
18320 287/300 Read block 2007 len 24576
18321 288/300 Read block 4764 len 8192
18322 289/300 Write block 326 len 4096 data: 4
18323 290/300 Write block 2908 len 16384 data: 2 2 3 3
18324 291/300 Read block 3373 len 24576
18325 292/300 Write block 3773 len 8192 data: 3 3
18326 293/300 Read block 4658 len 32768
18327 294/300 Write block 4456 len 12288 data: 4 3 4
18328 295/300 Write block 1267 len 16384 data: 4 5 6 6
18329 296/300 Write block 4932 len 36864 data: 7 9 7 5 4 4 4 5 5
18330 297/300 Write block 2682 len 32768 data: 5 5 5 5 5 6 4 4
18331 298/300 Write block 3324 len 40960 data: 4 4 4 4 3 3 3 3 3 4
18332 299/300 Write block 4637 len 40960 data: 4 4 4 4 3 3 2 3 3 3
18333 300/300 Read block 2041 len 40960
18334 301/300 Read block 3465 len 20480
18335 Main task has work to do, go find it
18336 got dsc Start(1)
18337 start 1
18338 Nov 19 17:11:51.667 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/1, method: POST, req_id: ab59ac6c-6453-400b-b14f-b8cc23591ac4, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
18339 Replay: started 1, returned:Ok(())
18340 ----------------------------------------------------------------
18341 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18342 States: Active Offline Active
18343 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18344 5528 Acked Read 5 Done Sent Done false
18345 5529 Acked Read 7 Done Sent Done false
18346 5530 Acked Write 1 Done Sent Done false
18347 5531 Acked Write 7 Done Sent Done false
18348 5532 Acked Read 8 Done Sent Done false
18349 5533 Acked Flush 0 Done Sent Done false
18350 5534 Acked Flush 0 Done Sent Done false
18351 5535 Acked Read 3 Done Sent Done false
18352 5536 Acked Read 1 Done Sent Done false
18353 5537 Acked Read 3 Done Sent Done false
18354 5538 Acked Read 2 Done Sent Done false
18355 5539 Acked Write 3 Done Sent Done false
18356 5540 Acked Read 4 Done Sent Done false
18357 5541 Acked Read 5 Done Sent Done false
18358 5542 Acked Write 10 Done Sent Done false
18359 5543 Acked Read 8 Done Sent Done false
18360 5544 Acked Flush 0 Done Sent Done false
18361 5545 Acked Write 9 Done Sent Done false
18362 5546 Acked Write 4 Done Sent Done false
18363 5547 Acked Flush 0 Done Sent Done false
18364 5548 Acked Read 10 Done Sent Done false
18365 5549 Acked Read 10 Done Sent Done false
18366 5550 Acked Write 4 Done Sent Done false
18367 5551 Acked Read 2 Done Sent Done false
18368 5552 Acked Read 3 Done Sent Done false
18369 5553 Acked Write 3 Done Sent Done false
18370 5554 Acked Read 6 Done Sent Done false
18371 5555 Acked Write 5 Done Sent Done false
18372 5556 Acked Read 1 Done Sent Done false
18373 5557 Acked Write 2 Done Sent Done false
18374 5558 Acked Write 7 Done Sent Done false
18375 5559 Acked Read 5 Done Sent Done false
18376 5560 Acked Flush 0 Done Sent Done false
18377 5561 Acked Read 5 Done Sent Done false
18378 5562 Acked Write 5 Done Sent Done false
18379 5563 Acked Write 9 Done Sent Done false
18380 5564 Acked Read 2 Done Sent Done false
18381 5565 Acked Write 5 Done Sent Done false
18382 5566 Acked Write 6 Done Sent Done false
18383 5567 Acked Read 1 Done Sent Done false
18384 5568 Acked Read 10 Done Sent Done false
18385 5569 Acked Read 4 Done Sent Done false
18386 5570 Acked Write 3 Done Sent Done false
18387 5571 Acked Read 2 Done Sent Done false
18388 5572 Acked Flush 0 Done Sent Done false
18389 5573 Acked Read 5 Done Sent Done false
18390 5574 Acked Write 10 Done Sent Done false
18391 5575 Acked Write 3 Done Sent Done false
18392 5576 Acked Read 4 Done Sent Done false
18393 5577 Acked Write 5 Done Sent Done false
18394 5578 Acked Read 2 Done Sent Done false
18395 5579 Acked Write 3 Done Sent Done false
18396 5580 Acked Read 9 Done Sent Done false
18397 5581 Acked Write 8 Done Sent Done false
18398 5582 Acked Read 2 Done Sent Done false
18399 5583 Acked Read 5 Done Sent Done false
18400 5584 Acked Read 4 Done Sent Done false
18401 5585 Acked Read 8 Done Sent Done false
18402 5586 Acked Write 4 Done Sent Done false
18403 5587 Acked Write 10 Done Sent Done false
18404 5588 Acked Read 9 Done Sent Done false
18405 5589 Acked Write 6 Done Sent Done false
18406 5590 Acked Read 9 Done Sent Done false
18407 5591 Acked Write 10 Done Sent Done false
18408 5592 Acked Write 6 Done Sent Done false
18409 5593 Acked Write 3 Done Sent Done false
18410 5594 Acked Flush 0 Done Sent Done false
18411 5595 Acked Read 7 Done Sent Done false
18412 5596 Acked Read 2 Done Sent Done false
18413 5597 Acked Write 3 Done Sent Done false
18414 5598 Acked Write 7 Done Sent Done false
18415 5599 Acked Flush 0 Done Sent Done false
18416 5600 Acked Read 1 Done Sent Done false
18417 5601 Acked Read 5 Done Sent Done false
18418 5602 Acked Write 8 Done Sent Done false
18419 5603 Acked Read 7 Done Sent Done false
18420 5604 Acked Write 9 Done Sent Done false
18421 5605 Acked Read 9 Done Sent Done false
18422 5606 Acked Write 9 Done Sent Done false
18423 5607 Acked Flush 0 Done Sent Done false
18424 5608 Acked Read 3 Done Sent Done false
18425 5609 Acked Write 4 Done Sent Done false
18426 5610 Acked Write 8 Done Sent Done false
18427 5611 Acked Read 1 Done Sent Done false
18428 5612 Acked Write 9 Done Sent Done false
18429 5613 Acked Read 4 Done Sent Done false
18430 5614 Acked Write 9 Done Sent Done false
18431 5615 Acked Read 7 Done Sent Done false
18432 5616 Acked Flush 0 Done Sent Done false
18433 5617 Acked Write 6 Done Sent Done false
18434 5618 Acked Flush 0 Done Sent Done false
18435 5619 Acked Read 7 Done Sent Done false
18436 5620 Acked Read 10 Done Sent Done false
18437 5621 Acked Read 8 Done Sent Done false
18438 5622 Acked Write 1 Done Sent Done false
18439 5623 Acked Flush 0 Done Sent Done false
18440 5624 Acked Write 6 Done Sent Done false
18441 5625 Acked Read 1 Done Sent Done false
18442 5626 Acked Read 3 Done Sent Done false
18443 5627 Acked Read 7 Done Sent Done false
18444 5628 Acked Write 9 Done Sent Done false
18445 5629 Acked Read 10 Done Sent Done false
18446 5630 Acked Write 7 Done Sent Done false
18447 5631 Acked Read 4 Done Sent Done false
18448 5632 Acked Read 10 Done Sent Done false
18449 5633 Acked Write 7 Done Sent Done false
18450 5634 Acked Write 2 Done Sent Done false
18451 5635 Acked Write 1 Done Sent Done false
18452 5636 Acked Write 3 Done Sent Done false
18453 5637 Acked Read 5 Done Sent Done false
18454 5638 Acked Flush 0 Done Sent Done false
18455 5639 Acked Read 4 Done Sent Done false
18456 5640 Acked Read 6 Done Sent Done false
18457 5641 Acked Read 8 Done Sent Done false
18458 5642 Acked Write 8 Done Sent Done false
18459 5643 Acked Write 5 Done Sent Done false
18460 5644 Acked Write 5 Done Sent Done false
18461 5645 Acked Write 8 Done Sent Done false
18462 5646 Acked Flush 0 Done Sent Done false
18463 5647 Acked Write 5 Done Sent Done false
18464 5648 Acked Read 10 Done Sent Done false
18465 5649 Acked Read 3 Done Sent Done false
18466 5650 Acked Flush 0 Done Sent Done false
18467 5651 Acked Write 6 Done Sent Done false
18468 5652 Acked Write 1 Done Sent Done false
18469 5653 Acked Write 2 Done Sent Done false
18470 5654 Acked Read 9 Done Sent Done false
18471 5655 Acked Read 10 Done Sent Done false
18472 5656 Acked Write 1 Done Sent Done false
18473 5657 Acked Write 3 Done Sent Done false
18474 5658 Acked Read 5 Done Sent Done false
18475 5659 Acked Write 4 Done Sent Done false
18476 5660 Acked Write 1 Done Sent Done false
18477 5661 Acked Read 5 Done Sent Done false
18478 5662 Acked Read 5 Done Sent Done false
18479 5663 Acked Read 1 Done Sent Done false
18480 5664 Acked Read 3 Done Sent Done false
18481 5665 Acked Write 2 Done Sent Done false
18482 5666 Acked Read 10 Done Sent Done false
18483 5667 Acked Read 2 Done Sent Done false
18484 5668 Acked Read 10 Done Sent Done false
18485 5669 Acked Read 9 Done Sent Done false
18486 5670 Acked Read 9 Done Sent Done false
18487 5671 Acked Read 2 Done Sent Done false
18488 5672 Acked Read 6 Done Sent Done false
18489 5673 Acked Read 3 Done Sent Done false
18490 5674 Acked Write 4 Done Sent Done false
18491 5675 Acked Write 10 Done Sent Done false
18492 5676 Acked Write 4 Done Sent Done false
18493 5677 Acked Flush 0 Done Sent Done false
18494 5678 Acked Write 7 Done Sent Done false
18495 5679 Acked Flush 0 Done Sent Done false
18496 5680 Acked Write 3 Done Sent Done false
18497 5681 Acked Read 2 Done Sent Done false
18498 5682 Acked Read 2 Done Sent Done false
18499 5683 Acked Write 1 Done Sent Done false
18500 5684 Acked Read 3 Done Sent Done false
18501 5685 Acked Read 10 Done Sent Done false
18502 5686 Acked Write 5 Done Sent Done false
18503 5687 Acked Write 7 Done Sent Done false
18504 5688 Acked Read 3 Done Sent Done false
18505 5689 Acked Read 6 Done Sent Done false
18506 5690 Acked Write 7 Done Sent Done false
18507 5691 Acked Read 7 Done Sent Done false
18508 5692 Acked Write 10 Done Sent Done false
18509 5693 Acked Read 9 Done Sent Done false
18510 5694 Acked Write 1 Done Sent Done false
18511 5695 Acked Read 4 Done Sent Done false
18512 5696 Acked Read 10 Done Sent Done false
18513 5697 Acked Read 6 Done Sent Done false
18514 5698 Acked Flush 0 Done Sent Done false
18515 5699 Acked Read 5 Done Sent Done false
18516 5700 Acked Write 6 Done Sent Done false
18517 5701 Acked Read 3 Done Sent Done false
18518 5702 Acked Flush 0 Done Sent Done false
18519 5703 Acked Read 3 Done Sent Done false
18520 5704 Acked Write 6 Done Sent Done false
18521 5705 Acked Read 9 Done Sent Done false
18522 5706 Acked Read 8 Done Sent Done false
18523 5707 Acked Read 9 Done Sent Done false
18524 5708 Acked Read 10 Done Sent Done false
18525 5709 Acked Write 7 Done Sent Done false
18526 5710 Acked Flush 0 Done Sent Done false
18527 5711 Acked Write 2 Done Sent Done false
18528 5712 Acked Read 8 Done Sent Done false
18529 5713 Acked Flush 0 Done Sent Done false
18530 5714 Acked Flush 0 Done Sent Done false
18531 5715 Acked Read 10 Done Sent Done false
18532 5716 Acked Read 9 Done Sent Done false
18533 5717 Acked Write 8 Done Sent Done false
18534 5718 Acked Read 1 Done Sent Done false
18535 5719 Acked Write 4 Done Sent Done false
18536 5720 Acked Write 10 Done Sent Done false
18537 5721 Acked Read 9 Done Sent Done false
18538 5722 Acked Write 7 Done Sent Done false
18539 5723 Acked Read 8 Done Sent Done false
18540 5724 Acked Write 4 Done Sent Done false
18541 5725 Acked Read 7 Done Sent Done false
18542 5726 Acked Flush 0 Done Sent Done false
18543 5727 Acked Write 5 Done Sent Done false
18544 5728 Acked Flush 0 Done Sent Done false
18545 5729 Acked Read 2 Done Sent Done false
18546 5730 Acked Read 9 Done Sent Done false
18547 5731 Acked Read 9 Done Sent Done false
18548 5732 Acked Read 2 Done Sent Done false
18549 5733 Acked Read 3 Done Sent Done false
18550 5734 Acked Read 10 Done Sent Done false
18551 5735 Acked Read 10 Done Sent Done false
18552 5736 Acked Write 7 Done Sent Done false
18553 5737 Acked Read 5 Done Sent Done false
18554 5738 Acked Write 2 Done Sent Done false
18555 5739 Acked Flush 0 Done Sent Done false
18556 5740 Acked Write 2 Done Sent Done false
18557 5741 Acked Flush 0 Done Sent Done false
18558 5742 Acked Read 3 Done Sent Done false
18559 5743 Acked Write 4 Done Sent Done false
18560 5744 Acked Write 2 Done Sent Done false
18561 5745 Acked Read 5 Done Sent Done false
18562 5746 Acked Read 3 Done Sent Done false
18563 5747 Acked Flush 0 Done Sent Done false
18564 5748 Acked Write 8 Done Sent Done false
18565 5749 Acked Write 6 Done Sent Done false
18566 5750 Acked Read 6 Done Sent Done false
18567 5751 Acked Read 6 Done Sent Done false
18568 5752 Acked Read 4 Done Sent Done false
18569 5753 Acked Read 10 Done Sent Done false
18570 5754 Acked Flush 0 Done Sent Done false
18571 5755 Acked Read 8 Done Sent Done false
18572 5756 Acked Flush 0 Done Sent Done false
18573 5757 Acked Write 3 Done Sent Done false
18574 5758 Acked Read 7 Done Sent Done false
18575 5759 Acked Read 10 Done Sent Done false
18576 5760 Acked Read 5 Done Sent Done false
18577 5761 Acked Read 9 Done Sent Done false
18578 5762 Acked Read 1 Done Sent Done false
18579 5763 Acked Write 8 Done Sent Done false
18580 5764 Acked Write 7 Done Sent Done false
18581 5765 Acked Read 4 Done Sent Done false
18582 5766 Acked Read 8 Done Sent Done false
18583 5767 Acked Flush 0 Done Sent Done false
18584 5768 Acked Read 8 Done Sent Done false
18585 5769 Acked Read 9 Done Sent Done false
18586 5770 Acked Read 4 Done Sent Done false
18587 5771 Acked Read 5 Done Sent Done false
18588 5772 Acked Write 10 Done Sent Done false
18589 5773 Acked Write 2 Done Sent Done false
18590 5774 Acked Flush 0 Done Sent Done false
18591 5775 Acked Read 4 Done Sent Done false
18592 5776 Acked Write 6 Done Sent Done false
18593 5777 Acked Read 4 Done Sent Done false
18594 5778 Acked Write 1 Done Sent Done false
18595 5779 Acked Read 1 Done Sent Done false
18596 5780 Acked Read 5 Done Sent Done false
18597 5781 Acked Read 4 Done Sent Done false
18598 5782 Acked Read 2 Done Sent Done false
18599 5783 Acked Flush 0 Done Sent Done false
18600 5784 Acked Write 4 Done Sent Done false
18601 5785 Acked Write 10 Done Sent Done false
18602 5786 Acked Read 9 Done Sent Done false
18603 5787 Acked Write 9 Done Sent Done false
18604 5788 Acked Read 5 Done Sent Done false
18605 5789 Acked Read 7 Done Sent Done false
18606 5790 Acked Write 9 Done Sent Done false
18607 5791 Acked Write 9 Done Sent Done false
18608 5792 Acked Read 8 Done Sent Done false
18609 5793 Acked Write 2 Done Sent Done false
18610 5794 Acked Write 6 Done Sent Done false
18611 5795 Acked Read 3 Done Sent Done false
18612 5796 Acked Read 9 Done Sent Done false
18613 5797 Acked Write 8 Done Sent Done false
18614 5798 Acked Write 2 Done Sent Done false
18615 5799 Acked Read 9 Done Sent Done false
18616 5800 Acked Read 9 Done Sent Done false
18617 5801 Acked Write 2 Done Sent Done false
18618 5802 Acked Read 5 Done Sent Done false
18619 5803 Acked Write 10 Done Sent Done false
18620 5804 Acked Write 8 Done Sent Done false
18621 5805 Acked Read 1 Done Sent Done false
18622 5806 Acked Write 4 Done Sent Done false
18623 5807 Acked Write 3 Done Sent Done false
18624 5808 Acked Read 8 Done Sent Done false
18625 5809 Acked Read 3 Done Sent Done false
18626 5810 Acked Read 6 Done Sent Done false
18627 5811 Acked Read 4 Done Sent Done false
18628 5812 Acked Read 1 Done Sent Done false
18629 5813 Acked Write 7 Done Sent Done false
18630 5814 Acked Read 6 Done Sent Done false
18631 5815 Acked Read 2 Done Sent Done false
18632 5816 Acked Write 1 Done Sent Done false
18633 5817 Acked Write 4 Done Sent Done false
18634 5818 Acked Read 6 Done Sent Done false
18635 5819 Acked Write 2 Done Sent Done false
18636 5820 Acked Read 8 Done Sent Done false
18637 5821 Acked Write 3 Done Sent Done false
18638 5822 Acked Write 4 Done Sent Done false
18639 5823 Acked Write 9 Done Sent Done false
18640 5824 Acked Write 8 Done Sent Done false
18641 5825 Acked Write 10 Done Sent Done false
18642 5826 Acked Write 10 Done Sent Done false
18643 5827 Acked Read 10 Done Sent Done false
18644 5828 Acked Read 5 Done Sent Done false
18645 STATES DS:0 DS:1 DS:2 TOTAL
18646 Sent 0 301 0 301
18647 Done 301 0 301 602
18648 Skipped 0 0 0 0
18649 Error 0 0 0 0
18650 Last Flush: 5783 5527 5783
18651 Downstairs last five completed: 5527 5526 5525 5524 5523
18652 Upstairs last five completed: 5828 5827 5826 5825 5824
18653 CLIENT: Up:0 ds:301 act:2
18654 [8820] got start action
18655 [8820] I am going to restart
18656 Starting downstairs at port 8820
18657 Make output file at "/tmp/dsc/downstairs-8820.txt"
18658 [8820][1] reports Starting
18659 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
18660 [8820][1] reports Running
18661 ----------------------------------------------------------------
18662 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18663 States: Active Offline Active
18664 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18665 5528 Acked Read 5 Done Sent Done false
18666 5529 Acked Read 7 Done Sent Done false
18667 5530 Acked Write 1 Done Sent Done false
18668 5531 Acked Write 7 Done Sent Done false
18669 5532 Acked Read 8 Done Sent Done false
18670 5533 Acked Flush 0 Done Sent Done false
18671 5534 Acked Flush 0 Done Sent Done false
18672 5535 Acked Read 3 Done Sent Done false
18673 5536 Acked Read 1 Done Sent Done false
18674 5537 Acked Read 3 Done Sent Done false
18675 5538 Acked Read 2 Done Sent Done false
18676 5539 Acked Write 3 Done Sent Done false
18677 5540 Acked Read 4 Done Sent Done false
18678 5541 Acked Read 5 Done Sent Done false
18679 5542 Acked Write 10 Done Sent Done false
18680 5543 Acked Read 8 Done Sent Done false
18681 5544 Acked Flush 0 Done Sent Done false
18682 5545 Acked Write 9 Done Sent Done false
18683 5546 Acked Write 4 Done Sent Done false
18684 5547 Acked Flush 0 Done Sent Done false
18685 5548 Acked Read 10 Done Sent Done false
18686 5549 Acked Read 10 Done Sent Done false
18687 5550 Acked Write 4 Done Sent Done false
18688 5551 Acked Read 2 Done Sent Done false
18689 5552 Acked Read 3 Done Sent Done false
18690 5553 Acked Write 3 Done Sent Done false
18691 5554 Acked Read 6 Done Sent Done false
18692 5555 Acked Write 5 Done Sent Done false
18693 5556 Acked Read 1 Done Sent Done false
18694 5557 Acked Write 2 Done Sent Done false
18695 5558 Acked Write 7 Done Sent Done false
18696 5559 Acked Read 5 Done Sent Done false
18697 5560 Acked Flush 0 Done Sent Done false
18698 5561 Acked Read 5 Done Sent Done false
18699 5562 Acked Write 5 Done Sent Done false
18700 5563 Acked Write 9 Done Sent Done false
18701 5564 Acked Read 2 Done Sent Done false
18702 5565 Acked Write 5 Done Sent Done false
18703 5566 Acked Write 6 Done Sent Done false
18704 5567 Acked Read 1 Done Sent Done false
18705 5568 Acked Read 10 Done Sent Done false
18706 5569 Acked Read 4 Done Sent Done false
18707 5570 Acked Write 3 Done Sent Done false
18708 5571 Acked Read 2 Done Sent Done false
18709 5572 Acked Flush 0 Done Sent Done false
18710 5573 Acked Read 5 Done Sent Done false
18711 5574 Acked Write 10 Done Sent Done false
18712 5575 Acked Write 3 Done Sent Done false
18713 5576 Acked Read 4 Done Sent Done false
18714 5577 Acked Write 5 Done Sent Done false
18715 5578 Acked Read 2 Done Sent Done false
18716 5579 Acked Write 3 Done Sent Done false
18717 5580 Acked Read 9 Done Sent Done false
18718 5581 Acked Write 8 Done Sent Done false
18719 5582 Acked Read 2 Done Sent Done false
18720 5583 Acked Read 5 Done Sent Done false
18721 5584 Acked Read 4 Done Sent Done false
18722 5585 Acked Read 8 Done Sent Done false
18723 5586 Acked Write 4 Done Sent Done false
18724 5587 Acked Write 10 Done Sent Done false
18725 5588 Acked Read 9 Done Sent Done false
18726 5589 Acked Write 6 Done Sent Done false
18727 5590 Acked Read 9 Done Sent Done false
18728 5591 Acked Write 10 Done Sent Done false
18729 5592 Acked Write 6 Done Sent Done false
18730 5593 Acked Write 3 Done Sent Done false
18731 5594 Acked Flush 0 Done Sent Done false
18732 5595 Acked Read 7 Done Sent Done false
18733 5596 Acked Read 2 Done Sent Done false
18734 5597 Acked Write 3 Done Sent Done false
18735 5598 Acked Write 7 Done Sent Done false
18736 5599 Acked Flush 0 Done Sent Done false
18737 5600 Acked Read 1 Done Sent Done false
18738 5601 Acked Read 5 Done Sent Done false
18739 5602 Acked Write 8 Done Sent Done false
18740 5603 Acked Read 7 Done Sent Done false
18741 5604 Acked Write 9 Done Sent Done false
18742 5605 Acked Read 9 Done Sent Done false
18743 5606 Acked Write 9 Done Sent Done false
18744 5607 Acked Flush 0 Done Sent Done false
18745 5608 Acked Read 3 Done Sent Done false
18746 5609 Acked Write 4 Done Sent Done false
18747 5610 Acked Write 8 Done Sent Done false
18748 5611 Acked Read 1 Done Sent Done false
18749 5612 Acked Write 9 Done Sent Done false
18750 5613 Acked Read 4 Done Sent Done false
18751 5614 Acked Write 9 Done Sent Done false
18752 5615 Acked Read 7 Done Sent Done false
18753 5616 Acked Flush 0 Done Sent Done false
18754 5617 Acked Write 6 Done Sent Done false
18755 5618 Acked Flush 0 Done Sent Done false
18756 5619 Acked Read 7 Done Sent Done false
18757 5620 Acked Read 10 Done Sent Done false
18758 5621 Acked Read 8 Done Sent Done false
18759 5622 Acked Write 1 Done Sent Done false
18760 5623 Acked Flush 0 Done Sent Done false
18761 5624 Acked Write 6 Done Sent Done false
18762 5625 Acked Read 1 Done Sent Done false
18763 5626 Acked Read 3 Done Sent Done false
18764 5627 Acked Read 7 Done Sent Done false
18765 5628 Acked Write 9 Done Sent Done false
18766 5629 Acked Read 10 Done Sent Done false
18767 5630 Acked Write 7 Done Sent Done false
18768 5631 Acked Read 4 Done Sent Done false
18769 5632 Acked Read 10 Done Sent Done false
18770 5633 Acked Write 7 Done Sent Done false
18771 5634 Acked Write 2 Done Sent Done false
18772 5635 Acked Write 1 Done Sent Done false
18773 5636 Acked Write 3 Done Sent Done false
18774 5637 Acked Read 5 Done Sent Done false
18775 5638 Acked Flush 0 Done Sent Done false
18776 5639 Acked Read 4 Done Sent Done false
18777 5640 Acked Read 6 Done Sent Done false
18778 5641 Acked Read 8 Done Sent Done false
18779 5642 Acked Write 8 Done Sent Done false
18780 5643 Acked Write 5 Done Sent Done false
18781 5644 Acked Write 5 Done Sent Done false
18782 5645 Acked Write 8 Done Sent Done false
18783 5646 Acked Flush 0 Done Sent Done false
18784 5647 Acked Write 5 Done Sent Done false
18785 5648 Acked Read 10 Done Sent Done false
18786 5649 Acked Read 3 Done Sent Done false
18787 5650 Acked Flush 0 Done Sent Done false
18788 5651 Acked Write 6 Done Sent Done false
18789 5652 Acked Write 1 Done Sent Done false
18790 5653 Acked Write 2 Done Sent Done false
18791 5654 Acked Read 9 Done Sent Done false
18792 5655 Acked Read 10 Done Sent Done false
18793 5656 Acked Write 1 Done Sent Done false
18794 5657 Acked Write 3 Done Sent Done false
18795 5658 Acked Read 5 Done Sent Done false
18796 5659 Acked Write 4 Done Sent Done false
18797 5660 Acked Write 1 Done Sent Done false
18798 5661 Acked Read 5 Done Sent Done false
18799 5662 Acked Read 5 Done Sent Done false
18800 5663 Acked Read 1 Done Sent Done false
18801 5664 Acked Read 3 Done Sent Done false
18802 5665 Acked Write 2 Done Sent Done false
18803 5666 Acked Read 10 Done Sent Done false
18804 5667 Acked Read 2 Done Sent Done false
18805 5668 Acked Read 10 Done Sent Done false
18806 5669 Acked Read 9 Done Sent Done false
18807 5670 Acked Read 9 Done Sent Done false
18808 5671 Acked Read 2 Done Sent Done false
18809 5672 Acked Read 6 Done Sent Done false
18810 5673 Acked Read 3 Done Sent Done false
18811 5674 Acked Write 4 Done Sent Done false
18812 5675 Acked Write 10 Done Sent Done false
18813 5676 Acked Write 4 Done Sent Done false
18814 5677 Acked Flush 0 Done Sent Done false
18815 5678 Acked Write 7 Done Sent Done false
18816 5679 Acked Flush 0 Done Sent Done false
18817 5680 Acked Write 3 Done Sent Done false
18818 5681 Acked Read 2 Done Sent Done false
18819 5682 Acked Read 2 Done Sent Done false
18820 5683 Acked Write 1 Done Sent Done false
18821 5684 Acked Read 3 Done Sent Done false
18822 5685 Acked Read 10 Done Sent Done false
18823 5686 Acked Write 5 Done Sent Done false
18824 5687 Acked Write 7 Done Sent Done false
18825 5688 Acked Read 3 Done Sent Done false
18826 5689 Acked Read 6 Done Sent Done false
18827 5690 Acked Write 7 Done Sent Done false
18828 5691 Acked Read 7 Done Sent Done false
18829 5692 Acked Write 10 Done Sent Done false
18830 5693 Acked Read 9 Done Sent Done false
18831 5694 Acked Write 1 Done Sent Done false
18832 5695 Acked Read 4 Done Sent Done false
18833 5696 Acked Read 10 Done Sent Done false
18834 5697 Acked Read 6 Done Sent Done false
18835 5698 Acked Flush 0 Done Sent Done false
18836 5699 Acked Read 5 Done Sent Done false
18837 5700 Acked Write 6 Done Sent Done false
18838 5701 Acked Read 3 Done Sent Done false
18839 5702 Acked Flush 0 Done Sent Done false
18840 5703 Acked Read 3 Done Sent Done false
18841 5704 Acked Write 6 Done Sent Done false
18842 5705 Acked Read 9 Done Sent Done false
18843 5706 Acked Read 8 Done Sent Done false
18844 5707 Acked Read 9 Done Sent Done false
18845 5708 Acked Read 10 Done Sent Done false
18846 5709 Acked Write 7 Done Sent Done false
18847 5710 Acked Flush 0 Done Sent Done false
18848 5711 Acked Write 2 Done Sent Done false
18849 5712 Acked Read 8 Done Sent Done false
18850 5713 Acked Flush 0 Done Sent Done false
18851 5714 Acked Flush 0 Done Sent Done false
18852 5715 Acked Read 10 Done Sent Done false
18853 5716 Acked Read 9 Done Sent Done false
18854 5717 Acked Write 8 Done Sent Done false
18855 5718 Acked Read 1 Done Sent Done false
18856 5719 Acked Write 4 Done Sent Done false
18857 5720 Acked Write 10 Done Sent Done false
18858 5721 Acked Read 9 Done Sent Done false
18859 5722 Acked Write 7 Done Sent Done false
18860 5723 Acked Read 8 Done Sent Done false
18861 5724 Acked Write 4 Done Sent Done false
18862 5725 Acked Read 7 Done Sent Done false
18863 5726 Acked Flush 0 Done Sent Done false
18864 5727 Acked Write 5 Done Sent Done false
18865 5728 Acked Flush 0 Done Sent Done false
18866 5729 Acked Read 2 Done Sent Done false
18867 5730 Acked Read 9 Done Sent Done false
18868 5731 Acked Read 9 Done Sent Done false
18869 5732 Acked Read 2 Done Sent Done false
18870 5733 Acked Read 3 Done Sent Done false
18871 5734 Acked Read 10 Done Sent Done false
18872 5735 Acked Read 10 Done Sent Done false
18873 5736 Acked Write 7 Done Sent Done false
18874 5737 Acked Read 5 Done Sent Done false
18875 5738 Acked Write 2 Done Sent Done false
18876 5739 Acked Flush 0 Done Sent Done false
18877 5740 Acked Write 2 Done Sent Done false
18878 5741 Acked Flush 0 Done Sent Done false
18879 5742 Acked Read 3 Done Sent Done false
18880 5743 Acked Write 4 Done Sent Done false
18881 5744 Acked Write 2 Done Sent Done false
18882 5745 Acked Read 5 Done Sent Done false
18883 5746 Acked Read 3 Done Sent Done false
18884 5747 Acked Flush 0 Done Sent Done false
18885 5748 Acked Write 8 Done Sent Done false
18886 5749 Acked Write 6 Done Sent Done false
18887 5750 Acked Read 6 Done Sent Done false
18888 5751 Acked Read 6 Done Sent Done false
18889 5752 Acked Read 4 Done Sent Done false
18890 5753 Acked Read 10 Done Sent Done false
18891 5754 Acked Flush 0 Done Sent Done false
18892 5755 Acked Read 8 Done Sent Done false
18893 5756 Acked Flush 0 Done Sent Done false
18894 5757 Acked Write 3 Done Sent Done false
18895 5758 Acked Read 7 Done Sent Done false
18896 5759 Acked Read 10 Done Sent Done false
18897 5760 Acked Read 5 Done Sent Done false
18898 5761 Acked Read 9 Done Sent Done false
18899 5762 Acked Read 1 Done Sent Done false
18900 5763 Acked Write 8 Done Sent Done false
18901 5764 Acked Write 7 Done Sent Done false
18902 5765 Acked Read 4 Done Sent Done false
18903 5766 Acked Read 8 Done Sent Done false
18904 5767 Acked Flush 0 Done Sent Done false
18905 5768 Acked Read 8 Done Sent Done false
18906 5769 Acked Read 9 Done Sent Done false
18907 5770 Acked Read 4 Done Sent Done false
18908 5771 Acked Read 5 Done Sent Done false
18909 5772 Acked Write 10 Done Sent Done false
18910 5773 Acked Write 2 Done Sent Done false
18911 5774 Acked Flush 0 Done Sent Done false
18912 5775 Acked Read 4 Done Sent Done false
18913 5776 Acked Write 6 Done Sent Done false
18914 5777 Acked Read 4 Done Sent Done false
18915 5778 Acked Write 1 Done Sent Done false
18916 5779 Acked Read 1 Done Sent Done false
18917 5780 Acked Read 5 Done Sent Done false
18918 5781 Acked Read 4 Done Sent Done false
18919 5782 Acked Read 2 Done Sent Done false
18920 5783 Acked Flush 0 Done Sent Done false
18921 5784 Acked Write 4 Done Sent Done false
18922 5785 Acked Write 10 Done Sent Done false
18923 5786 Acked Read 9 Done Sent Done false
18924 5787 Acked Write 9 Done Sent Done false
18925 5788 Acked Read 5 Done Sent Done false
18926 5789 Acked Read 7 Done Sent Done false
18927 5790 Acked Write 9 Done Sent Done false
18928 5791 Acked Write 9 Done Sent Done false
18929 5792 Acked Read 8 Done Sent Done false
18930 5793 Acked Write 2 Done Sent Done false
18931 5794 Acked Write 6 Done Sent Done false
18932 5795 Acked Read 3 Done Sent Done false
18933 5796 Acked Read 9 Done Sent Done false
18934 5797 Acked Write 8 Done Sent Done false
18935 5798 Acked Write 2 Done Sent Done false
18936 5799 Acked Read 9 Done Sent Done false
18937 5800 Acked Read 9 Done Sent Done false
18938 5801 Acked Write 2 Done Sent Done false
18939 5802 Acked Read 5 Done Sent Done false
18940 5803 Acked Write 10 Done Sent Done false
18941 5804 Acked Write 8 Done Sent Done false
18942 5805 Acked Read 1 Done Sent Done false
18943 5806 Acked Write 4 Done Sent Done false
18944 5807 Acked Write 3 Done Sent Done false
18945 5808 Acked Read 8 Done Sent Done false
18946 5809 Acked Read 3 Done Sent Done false
18947 5810 Acked Read 6 Done Sent Done false
18948 5811 Acked Read 4 Done Sent Done false
18949 5812 Acked Read 1 Done Sent Done false
18950 5813 Acked Write 7 Done Sent Done false
18951 5814 Acked Read 6 Done Sent Done false
18952 5815 Acked Read 2 Done Sent Done false
18953 5816 Acked Write 1 Done Sent Done false
18954 5817 Acked Write 4 Done Sent Done false
18955 5818 Acked Read 6 Done Sent Done false
18956 5819 Acked Write 2 Done Sent Done false
18957 5820 Acked Read 8 Done Sent Done false
18958 5821 Acked Write 3 Done Sent Done false
18959 5822 Acked Write 4 Done Sent Done false
18960 5823 Acked Write 9 Done Sent Done false
18961 5824 Acked Write 8 Done Sent Done false
18962 5825 Acked Write 10 Done Sent Done false
18963 5826 Acked Write 10 Done Sent Done false
18964 5827 Acked Read 10 Done Sent Done false
18965 5828 Acked Read 5 Done Sent Done false
18966 5829 Acked Flush 0 Done Sent Done false
18967 STATES DS:0 DS:1 DS:2 TOTAL
18968 Sent 0 302 0 302
18969 Done 302 0 302 604
18970 Skipped 0 0 0 0
18971 Error 0 0 0 0
18972 Last Flush: 5829 5527 5829
18973 Downstairs last five completed: 5527 5526 5525 5524 5523
18974 Upstairs last five completed: 5829 5828 5827 5826 5825
18975 CLIENT: Up:0 ds:302 act:2
189762024-11-19T17:11:57.425ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189772024-11-19T17:11:57.425ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56405) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189782024-11-19T17:11:57.427ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189792024-11-19T17:11:57.427ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189802024-11-19T17:11:57.427ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189812024-11-19T17:11:57.427ZINFOcrucible: send last flush ID to this DS: 5527 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189822024-11-19T17:11:57.428ZINFOcrucible: Replied this last flush ID: 5527 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189832024-11-19T17:11:57.428ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189842024-11-19T17:11:57.428ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189852024-11-19T17:11:57.428ZINFOcrucible: [1] client re-new 302 jobs since flush 5527 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
189862024-11-19T17:11:57.428ZINFOcrucible: [1] Replayed 302 jobs since flush: 5527 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
18987 ----------------------------------------------------------------
18988 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18989 Downstairs last five completed: 5829 5828 5827 5826 5825
18990 Upstairs last five completed: 5829 5828 5827 5826 5825
18991 CLIENT: Up:0 ds:0 act:3
18992 Replay: All jobs finished
18993 Main task has work to do, go find it
18994 got dsc Stop(2)
18995 stop 2
18996 Nov 19 17:11:59.674 INFO request completed, latency_us: 349, [8830] Got stop action so:false kr:false
18997 response_code: 204, uri: /stop/cid/2, method: POST, req_id: 11f0f621-c693-410c-973e-fd4a0ff3853d, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
189982024-11-19T17:11:59.675ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
189992024-11-19T17:11:59.675ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
190002024-11-19T17:11:59.675ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
19001 Nov 19 17:11:59.675 INFO request completed, latency_us: 174, response_code:{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:11:59.67533247Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
19002 200, uri: /state/cid/2, method: GET, req_id: 02063872-e6b3-4d70-b15c-01d7658ea949, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:11:59.675370557Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
19003 
190042024-11-19T17:11:59.675ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
19005 [8830] Exited with: ExitStatus(unix_wait_status(9))
19006 [8830][2] reports Exit
19007 Nov 19 17:12:03.677 INFO request completed, latency_us: 234, response_code: 200, uri: /state/cid/2, method: GET, req_id: bf822b5d-2942-4979-8dc7-ee9e4a172316, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
19008 001/300 Write block 1467 len 32768 data: 7 6 6 6 5 5 5 6
19009 002/300 Flush
19010 003/300 Write block 2456 len 8192 data: 2 2
19011 004/300 Write block 2832 len 12288 data: 5 6 6
19012 005/300 Read block 4072 len 36864
19013 006/300 Write block 1802 len 4096 data: 4
19014 007/300 Read block 4159 len 24576
19015 008/300 Read block 798 len 16384
19016 009/300 Write block 1306 len 36864 data: 4 4 4 3 2 2 4 4 4
19017 010/300 Read block 205 len 16384
19018 011/300 Write block 971 len 20480 data: 2 3 3 3 3
19019 012/300 Flush
19020 013/300 Write block 3 len 40960 data: 2 2 2 3 3 4 4 4 3 3
19021 014/300 Read block 3621 len 28672
19022 015/300 Flush
19023 016/300 Read block 680 len 16384
19024 017/300 Write block 2357 len 12288 data: 3 4 4
19025 018/300 Read block 753 len 36864
19026 019/300 Read block 3200 len 4096
19027 020/300 Write block 1099 len 4096 data: 4
19028 021/300 Flush
19029 022/300 Read block 3157 len 12288
19030 023/300 Write block 3274 len 32768 data: 5 4 4 4 4 5 5 7
19031 024/300 Read block 2024 len 4096
19032 025/300 Read block 4309 len 28672
19033 026/300 Read block 2675 len 16384
19034 027/300 Read block 1551 len 32768
19035 028/300 Read block 3656 len 40960
19036 029/300 Write block 416 len 8192 data: 3 3
19037 030/300 Write block 3967 len 4096 data: 4
19038 031/300 Read block 4583 len 28672
19039 032/300 Read block 118 len 4096
19040 033/300 Write block 3581 len 40960 data: 3 3 4 4 4 4 3 3 4 4
19041 034/300 Read block 121 len 24576
19042 035/300 Read block 4146 len 16384
19043 036/300 Write block 2530 len 20480 data: 5 5 5 3 4
19044 037/300 Read block 4173 len 8192
19045 038/300 Read block 1721 len 28672
19046 039/300 Read block 3091 len 40960
19047 040/300 Read block 617 len 8192
19048 041/300 Read block 1402 len 16384
19049 042/300 Write block 3895 len 16384 data: 4 5 5 5
19050 043/300 Read block 3278 len 12288
19051 044/300 Write block 4939 len 8192 data: 6 6
19052 045/300 Read block 1852 len 20480
19053 046/300 Flush
19054 047/300 Read block 725 len 16384
19055 048/300 Read block 4353 len 20480
19056 049/300 Write block 485 len 40960 data: 4 4 4 7 7 7 8 8 6 4
19057 050/300 Write block 2759 len 12288 data: 2 2 3
19058 051/300 Write block 1165 len 24576 data: 5 5 4 5 6 5
19059 052/300 Read block 1442 len 28672
19060 053/300 Read block 2120 len 32768
19061 054/300 Read block 978 len 12288
19062 055/300 Read block 9 len 16384
19063 056/300 Write block 860 len 16384 data: 6 6 6 5
19064 057/300 Read block 1956 len 8192
19065 058/300 Write block 964 len 4096 data: 3
19066 059/300 Read block 1377 len 28672
19067 060/300 Read block 1933 len 32768
19068 061/300 Write block 2483 len 32768 data: 5 6 4 5 6 5 4 4
19069 062/300 Read block 1524 len 40960
19070 063/300 Read block 2483 len 16384
19071 064/300 Write block 3739 len 28672 data: 10 11 9 9 9 8 6
19072 065/300 Read block 3327 len 8192
19073 066/300 Write block 4710 len 16384 data: 3 3 3 2
19074 067/300 Write block 3115 len 8192 data: 2 3
19075 068/300 Flush
19076 069/300 Write block 3458 len 40960 data: 5 5 4 3 4 4 6 6 6 5
19077 070/300 Flush
19078 071/300 Read block 4831 len 12288
19079 072/300 Write block 4445 len 12288 data: 2 3 3
19080 073/300 Write block 4591 len 28672 data: 6 6 4 4 4 4 4
19081 074/300 Read block 3292 len 4096
19082 075/300 Read block 4484 len 28672
19083 076/300 Write block 4344 len 20480 data: 3 3 3 3 3
19084 077/300 Read block 3118 len 16384
19085 078/300 Read block 4876 len 20480
19086 079/300 Read block 1103 len 12288
19087 080/300 Read block 755 len 32768
19088 081/300 Read block 3250 len 24576
19089 082/300 Write block 3019 len 28672 data: 5 5 5 5 3 3 3
19090 083/300 Read block 4668 len 28672
19091 084/300 Write block 3690 len 28672 data: 3 4 4 3 3 4 5
19092 085/300 Read block 4940 len 12288
19093 086/300 Read block 3484 len 24576
19094 087/300 Write block 541 len 16384 data: 4 4 4 3
19095 088/300 Read block 591 len 28672
19096 089/300 Flush
19097 090/300 Read block 1059 len 24576
19098 091/300 Read block 4768 len 28672
19099 092/300 Read block 4283 len 28672
19100 093/300 Flush
19101 094/300 Write block 946 len 24576 data: 6 4 4 3 3 3
19102 095/300 Read block 2653 len 40960
19103 096/300 Read block 4618 len 16384
19104 097/300 Flush
19105 098/300 Read block 1073 len 12288
19106 099/300 Read block 1686 len 32768
19107 100/300 Write block 2488 len 4096 data: 6
19108 101/300 Write block 2990 len 32768 data: 4 4 5 6 6 6 5 6
19109 102/300 Write block 4603 len 4096 data: 3
19110 103/300 Read block 102 len 24576
19111 104/300 Read block 1694 len 12288
19112 105/300 Write block 1811 len 8192 data: 3 3
19113 106/300 Write block 3733 len 8192 data: 5 5
19114 107/300 Read block 359 len 40960
19115 108/300 Read block 4398 len 12288
19116 109/300 Read block 3238 len 28672
19117 110/300 Read block 1890 len 8192
19118 111/300 Flush
19119 112/300 Write block 2663 len 4096 data: 5
19120 113/300 Write block 1763 len 20480 data: 4 4 3 3 3
19121 114/300 Flush
19122 115/300 Read block 4769 len 20480
19123 116/300 Write block 1929 len 20480 data: 6 4 4 5 5
19124 117/300 Read block 594 len 4096
19125 118/300 Write block 3373 len 36864 data: 5 6 4 3 3 3 3 3 2
19126 119/300 Read block 2234 len 4096
19127 120/300 Write block 3188 len 36864 data: 5 5 4 3 3 3 3 3 2
19128 121/300 Write block 3651 len 20480 data: 2 2 2 2 2
19129 122/300 Write block 2573 len 24576 data: 4 4 2 2 2 2
19130 123/300 Read block 600 len 24576
19131 124/300 Write block 2730 len 20480 data: 5 4 4 4 3
19132 125/300 Read block 3302 len 4096
19133 126/300 Read block 4475 len 4096
19134 127/300 Read block 4522 len 12288
19135 128/300 Read block 2566 len 12288
19136 129/300 Write block 820 len 28672 data: 4 3 3 4 6 6 5
19137 130/300 Read block 1258 len 16384
19138 131/300 Read block 3685 len 28672
19139 132/300 Flush
19140 133/300 Read block 2335 len 28672
19141 134/300 Read block 4187 len 28672
19142 135/300 Write block 3821 len 12288 data: 5 7 8
19143 136/300 Read block 2811 len 24576
19144 137/300 Write block 4936 len 32768 data: 5 5 5 7 7 4 4 4
19145 138/300 Read block 4440 len 20480
19146 139/300 Read block 1834 len 20480
19147 140/300 Write block 1867 len 40960 data: 5 5 5 5 4 4 4 3 5 6
19148 141/300 Flush
19149 142/300 Read block 3790 len 20480
19150 143/300 Read block 4331 len 32768
19151 144/300 Read block 143 len 24576
19152 145/300 Read block 1481 len 12288
19153 146/300 Flush
19154 147/300 Flush
19155 148/300 Read block 4518 len 24576
19156 149/300 Write block 3503 len 4096 data: 3
19157 150/300 Read block 2410 len 36864
19158 151/300 Read block 1318 len 4096
19159 152/300 Write block 4181 len 8192 data: 4 4
19160 153/300 Write block 2027 len 8192 data: 7 6
19161 154/300 Write block 4404 len 4096 data: 3
19162 155/300 Write block 2639 len 28672 data: 4 5 4 4 4 5 4
19163 156/300 Write block 3527 len 20480 data: 3 3 4 5 6
19164 157/300 Read block 4313 len 12288
19165 158/300 Read block 4877 len 40960
19166 159/300 Write block 3290 len 28672 data: 3 4 3 4 4 4 3
19167 160/300 Write block 4566 len 16384 data: 5 3 3 2
19168 161/300 Write block 178 len 4096 data: 5
19169 162/300 Read block 671 len 28672
19170 163/300 Read block 4780 len 32768
19171 164/300 Read block 318 len 28672
19172 165/300 Read block 3659 len 20480
19173 166/300 Read block 4286 len 16384
19174 167/300 Read block 2190 len 16384
19175 168/300 Write block 241 len 20480 data: 3 3 3 3 4
19176 169/300 Write block 218 len 12288 data: 5 6 5
19177 170/300 Read block 2592 len 16384
19178 171/300 Read block 4454 len 20480
19179 172/300 Read block 3448 len 16384
19180 173/300 Flush
19181 174/300 Read block 3909 len 12288
19182 175/300 Write block 2213 len 4096 data: 7
19183 176/300 Read block 1425 len 32768
19184 177/300 Flush
19185 178/300 Read block 3601 len 20480
19186 179/300 Write block 1876 len 20480 data: 7 7 9 9 8
19187 180/300 Write block 1028 len 28672 data: 5 5 5 4 4 4 4
19188 181/300 Write block 966 len 12288 data: 3 3 3
19189 182/300 Write block 2867 len 28672 data: 3 3 2 2 2 2 2
19190 183/300 Write block 3292 len 36864 data: 4 5 5 5 4 4 3 3 3
19191 184/300 Write block 481 len 4096 data: 4
19192 185/300 Write block 4412 len 4096 data: 4
19193 186/300 Read block 4684 len 20480
19194 187/300 Flush
19195 188/300 Read block 839 len 20480
19196 189/300 Flush
19197 190/300 Read block 4687 len 28672
19198 191/300 Flush
19199 192/300 Write block 2244 len 12288 data: 6 6 6
19200 193/300 Write block 4793 len 12288 data: 5 5 6
19201 194/300 Read block 502 len 12288
19202 195/300 Flush
19203 196/300 Write block 2433 len 28672 data: 4 6 3 4 4 3 3
19204 197/300 Read block 4160 len 36864
19205 198/300 Read block 618 len 20480
19206 199/300 Write block 2690 len 40960 data: 4 4 4 4 4 4 3 4 3 3
19207 200/300 Write block 4495 len 12288 data: 5 6 6
19208 201/300 Read block 3948 len 36864
19209 202/300 Flush
19210 203/300 Write block 2369 len 20480 data: 3 3 2 2 2
19211 204/300 Flush
19212 205/300 Read block 4839 len 24576
19213 206/300 Write block 4490 len 24576 data: 5 7 7 6 3 6
19214 207/300 Read block 3698 len 36864
19215 208/300 Write block 1486 len 8192 data: 3 3
19216 209/300 Read block 2002 len 16384
19217 210/300 Write block 215 len 36864 data: 6 6 5 6 7 6 5 5 5
19218 211/300 Write block 3723 len 8192 data: 4 5
19219 212/300 Read block 4349 len 16384
19220 213/300 Flush
19221 214/300 Flush
19222 215/300 Write block 4382 len 36864 data: 5 5 4 4 4 4 3 3 3
19223 216/300 Read block 462 len 24576
19224 217/300 Flush
19225 218/300 Read block 3499 len 32768
19226 219/300 Read block 3926 len 36864
19227 220/300 Read block 4795 len 12288
19228 221/300 Flush
19229 222/300 Write block 2082 len 8192 data: 4 4
19230 223/300 Read block 420 len 32768
19231 224/300 Write block 3261 len 8192 data: 2 2
19232 225/300 Read block 2350 len 4096
19233 226/300 Read block 3356 len 28672
19234 227/300 Read block 4880 len 28672
19235 228/300 Read block 2499 len 16384
19236 229/300 Read block 1079 len 36864
19237 230/300 Read block 2368 len 36864
19238 231/300 Write block 3458 len 4096 data: 6
19239 232/300 Write block 3496 len 36864 data: 3 3 2 2 2 2 2 4 3
19240 233/300 Flush
19241 234/300 Read block 3146 len 24576
19242 235/300 Write block 4368 len 8192 data: 4 4
19243 236/300 Write block 134 len 32768 data: 8 8 9 9 8 10 8 6
19244 237/300 Write block 3225 len 24576 data: 5 6 5 5 5 4
19245 238/300 Read block 92 len 36864
19246 239/300 Write block 3263 len 12288 data: 2 2 3
19247 240/300 Write block 1203 len 40960 data: 5 4 2 2 2 2 2 2 2 2
19248 241/300 Read block 4346 len 8192
19249 242/300 Write block 1727 len 32768 data: 3 2 3 4 5 5 4 3
19250 243/300 Read block 410 len 32768
19251 244/300 Write block 2800 len 16384 data: 4 3 4 4
19252 245/300 Write block 2570 len 36864 data: 3 3 4 5 5 3 3 3 3
19253 246/300 Write block 3844 len 4096 data: 5
19254 247/300 Write block 2270 len 36864 data: 3 5 5 5 6 5 7 6 6
19255 248/300 Write block 4197 len 32768 data: 5 4 4 3 2 2 2 3
19256 249/300 Write block 2797 len 32768 data: 4 4 3 5 4 5 5 3
19257 250/300 Read block 2465 len 20480
19258 251/300 Write block 1716 len 20480 data: 3 4 4 4 3
19259 252/300 Write block 540 len 8192 data: 4 5
19260 253/300 Write block 2338 len 20480 data: 2 3 4 4 2
19261 254/300 Flush
19262 255/300 Read block 4551 len 8192
19263 256/300 Write block 2663 len 36864 data: 6 5 5 5 6 4 3 3 3
19264 257/300 Read block 4606 len 8192
19265 258/300 Read block 1024 len 28672
19266 259/300 Write block 1311 len 4096 data: 3
19267 260/300 Write block 1524 len 24576 data: 3 2 4 4 4 4
19268 261/300 Read block 4889 len 32768
19269 262/300 Read block 3171 len 28672
19270 263/300 Flush
19271 264/300 Flush
19272 265/300 Write block 2533 len 32768 data: 4 5 4 5 5 3 3 3
19273 266/300 Read block 659 len 4096
19274 267/300 Write block 784 len 4096 data: 7
19275 268/300 Read block 3364 len 40960
19276 269/300 Read block 3363 len 36864
19277 270/300 Read block 3210 len 4096
19278 271/300 Read block 2466 len 16384
19279 272/300 Write block 4194 len 16384 data: 5 7 7 6
19280 273/300 Write block 400 len 20480 data: 4 5 4 5 3
19281 274/300 Flush
19282 275/300 Read block 924 len 12288
19283 276/300 Flush
19284 277/300 Write block 471 len 16384 data: 7 7 8 10
19285 278/300 Write block 2182 len 8192 data: 3 2
19286 279/300 Write block 4206 len 8192 data: 4 4
19287 280/300 Read block 4815 len 4096
19288 281/300 Read block 4093 len 4096
19289 282/300 Write block 2589 len 24576 data: 2 2 3 3 2 2
19290 283/300 Write block 76 len 16384 data: 3 3 3 4
19291 284/300 Write block 3152 len 24576 data: 6 5 3 3 3 3
19292 285/300 Write block 4132 len 8192 data: 2 2
19293 286/300 Write block 4222 len 20480 data: 4 4 4 3 2
19294 287/300 Write block 395 len 36864 data: 4 4 4 4 4 5 6 5 6
19295 288/300 Read block 217 len 32768
19296 289/300 Write block 1712 len 32768 data: 5 5 3 3 4 5 5 5
19297 290/300 Read block 4364 len 40960
19298 291/300 Flush
19299 292/300 Write block 3033 len 36864 data: 6 4 3 3 3 2 2 2 3
19300 293/300 Write block 3308 len 32768 data: 5 5 4 3 4 5 4 5
19301 294/300 Write block 4103 len 40960 data: 3 3 2 2 2 2 2 2 2 2
19302 295/300 Write block 1718 len 32768 data: 6 6 4 3 4 4 4 4
19303 296/300 Read block 90 len 20480
19304 297/300 Flush
19305 298/300 Read block 2808 len 16384
19306 299/300 Read block 298 len 4096
19307 300/300 Write block 868 len 4096 data: 3
19308 301/300 Write block 4686 len 16384 data: 3 4 4 6
19309 Main task has work to do, go find it
19310 got dsc Start(2)
19311 start 2
19312 Nov 19 17:12:03.903 INFO request completed, latency_us: 230, response_code: 204, uri: /start/cid/2, method: POST, req_id: ef67d2d1-605b-411d-b42f-a8e06a7bc859, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
19313 Replay: started 2, returned:Ok(())
19314 ----------------------------------------------------------------
19315 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19316 States: Active Active Offline
19317 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19318 5830 Acked Write 8 Done Done Sent false
19319 5831 Acked Flush 0 Done Done Sent false
19320 5832 Acked Write 2 Done Done Sent false
19321 5833 Acked Write 3 Done Done Sent false
19322 5834 Acked Read 9 Done Done Sent false
19323 5835 Acked Write 1 Done Done Sent false
19324 5836 Acked Read 6 Done Done Sent false
19325 5837 Acked Read 4 Done Done Sent false
19326 5838 Acked Write 9 Done Done Sent false
19327 5839 Acked Read 4 Done Done Sent false
19328 5840 Acked Write 5 Done Done Sent false
19329 5841 Acked Flush 0 Done Done Sent false
19330 5842 Acked Write 10 Done Done Sent false
19331 5843 Acked Read 7 Done Done Sent false
19332 5844 Acked Flush 0 Done Done Sent false
19333 5845 Acked Read 4 Done Done Sent false
19334 5846 Acked Write 3 Done Done Sent false
19335 5847 Acked Read 9 Done Done Sent false
19336 5848 Acked Read 1 Done Done Sent false
19337 5849 Acked Write 1 Done Done Sent false
19338 5850 Acked Flush 0 Done Done Sent false
19339 5851 Acked Read 3 Done Done Sent false
19340 5852 Acked Write 8 Done Done Sent false
19341 5853 Acked Read 1 Done Done Sent false
19342 5854 Acked Read 7 Done Done Sent false
19343 5855 Acked Read 4 Done Done Sent false
19344 5856 Acked Read 8 Done Done Sent false
19345 5857 Acked Read 10 Done Done Sent false
19346 5858 Acked Write 2 Done Done Sent false
19347 5859 Acked Write 1 Done Done Sent false
19348 5860 Acked Read 7 Done Done Sent false
19349 5861 Acked Read 1 Done Done Sent false
19350 5862 Acked Write 10 Done Done Sent false
19351 5863 Acked Read 6 Done Done Sent false
19352 5864 Acked Read 4 Done Done Sent false
19353 5865 Acked Write 5 Done Done Sent false
19354 5866 Acked Read 2 Done Done Sent false
19355 5867 Acked Read 7 Done Done Sent false
19356 5868 Acked Read 10 Done Done Sent false
19357 5869 Acked Read 2 Done Done Sent false
19358 5870 Acked Read 4 Done Done Sent false
19359 5871 Acked Write 4 Done Done Sent false
19360 5872 Acked Read 3 Done Done Sent false
19361 5873 Acked Write 2 Done Done Sent false
19362 5874 Acked Read 5 Done Done Sent false
19363 5875 Acked Flush 0 Done Done Sent false
19364 5876 Acked Read 4 Done Done Sent false
19365 5877 Acked Read 5 Done Done Sent false
19366 5878 Acked Write 10 Done Done Sent false
19367 5879 Acked Write 3 Done Done Sent false
19368 5880 Acked Write 6 Done Done Sent false
19369 5881 Acked Read 7 Done Done Sent false
19370 5882 Acked Read 8 Done Done Sent false
19371 5883 Acked Read 3 Done Done Sent false
19372 5884 Acked Read 4 Done Done Sent false
19373 5885 Acked Write 4 Done Done Sent false
19374 5886 Acked Read 2 Done Done Sent false
19375 5887 Acked Write 1 Done Done Sent false
19376 5888 Acked Read 7 Done Done Sent false
19377 5889 Acked Read 8 Done Done Sent false
19378 5890 Acked Write 8 Done Done Sent false
19379 5891 Acked Read 10 Done Done Sent false
19380 5892 Acked Read 4 Done Done Sent false
19381 5893 Acked Write 7 Done Done Sent false
19382 5894 Acked Read 2 Done Done Sent false
19383 5895 Acked Write 4 Done Done Sent false
19384 5896 Acked Write 2 Done Done Sent false
19385 5897 Acked Flush 0 Done Done Sent false
19386 5898 Acked Write 10 Done Done Sent false
19387 5899 Acked Flush 0 Done Done Sent false
19388 5900 Acked Read 3 Done Done Sent false
19389 5901 Acked Write 3 Done Done Sent false
19390 5902 Acked Write 7 Done Done Sent false
19391 5903 Acked Read 1 Done Done Sent false
19392 5904 Acked Read 7 Done Done Sent false
19393 5905 Acked Write 5 Done Done Sent false
19394 5906 Acked Read 4 Done Done Sent false
19395 5907 Acked Read 5 Done Done Sent false
19396 5908 Acked Read 3 Done Done Sent false
19397 5909 Acked Read 8 Done Done Sent false
19398 5910 Acked Read 6 Done Done Sent false
19399 5911 Acked Write 7 Done Done Sent false
19400 5912 Acked Read 7 Done Done Sent false
19401 5913 Acked Write 7 Done Done Sent false
19402 5914 Acked Read 3 Done Done Sent false
19403 5915 Acked Read 6 Done Done Sent false
19404 5916 Acked Write 4 Done Done Sent false
19405 5917 Acked Read 7 Done Done Sent false
19406 5918 Acked Flush 0 Done Done Sent false
19407 5919 Acked Read 6 Done Done Sent false
19408 5920 Acked Read 7 Done Done Sent false
19409 5921 Acked Read 7 Done Done Sent false
19410 5922 Acked Flush 0 Done Done Sent false
19411 5923 Acked Write 6 Done Done Sent false
19412 5924 Acked Read 10 Done Done Sent false
19413 5925 Acked Read 4 Done Done Sent false
19414 5926 Acked Flush 0 Done Done Sent false
19415 5927 Acked Read 3 Done Done Sent false
19416 5928 Acked Read 8 Done Done Sent false
19417 5929 Acked Write 1 Done Done Sent false
19418 5930 Acked Write 8 Done Done Sent false
19419 5931 Acked Write 1 Done Done Sent false
19420 5932 Acked Read 6 Done Done Sent false
19421 5933 Acked Read 3 Done Done Sent false
19422 5934 Acked Write 2 Done Done Sent false
19423 5935 Acked Write 2 Done Done Sent false
19424 5936 Acked Read 10 Done Done Sent false
19425 5937 Acked Read 3 Done Done Sent false
19426 5938 Acked Read 7 Done Done Sent false
19427 5939 Acked Read 2 Done Done Sent false
19428 5940 Acked Flush 0 Done Done Sent false
19429 5941 Acked Write 1 Done Done Sent false
19430 5942 Acked Write 5 Done Done Sent false
19431 5943 Acked Flush 0 Done Done Sent false
19432 5944 Acked Read 5 Done Done Sent false
19433 5945 Acked Write 5 Done Done Sent false
19434 5946 Acked Read 1 Done Done Sent false
19435 5947 Acked Write 9 Done Done Sent false
19436 5948 Acked Read 1 Done Done Sent false
19437 5949 Acked Write 9 Done Done Sent false
19438 5950 Acked Write 5 Done Done Sent false
19439 5951 Acked Write 6 Done Done Sent false
19440 5952 Acked Read 6 Done Done Sent false
19441 5953 Acked Write 5 Done Done Sent false
19442 5954 Acked Read 1 Done Done Sent false
19443 5955 Acked Read 1 Done Done Sent false
19444 5956 Acked Read 3 Done Done Sent false
19445 5957 Acked Read 3 Done Done Sent false
19446 5958 Acked Write 7 Done Done Sent false
19447 5959 Acked Read 4 Done Done Sent false
19448 5960 Acked Read 7 Done Done Sent false
19449 5961 Acked Flush 0 Done Done Sent false
19450 5962 Acked Read 7 Done Done Sent false
19451 5963 Acked Read 7 Done Done Sent false
19452 5964 Acked Write 3 Done Done Sent false
19453 5965 Acked Read 6 Done Done Sent false
19454 5966 Acked Write 8 Done Done Sent false
19455 5967 Acked Read 5 Done Done Sent false
19456 5968 Acked Read 5 Done Done Sent false
19457 5969 Acked Write 10 Done Done Sent false
19458 5970 Acked Flush 0 Done Done Sent false
19459 5971 Acked Read 5 Done Done Sent false
19460 5972 Acked Read 8 Done Done Sent false
19461 5973 Acked Read 6 Done Done Sent false
19462 5974 Acked Read 3 Done Done Sent false
19463 5975 Acked Flush 0 Done Done Sent false
19464 5976 Acked Flush 0 Done Done Sent false
19465 5977 Acked Read 6 Done Done Sent false
19466 5978 Acked Write 1 Done Done Sent false
19467 5979 Acked Read 9 Done Done Sent false
19468 5980 Acked Read 1 Done Done Sent false
19469 5981 Acked Write 2 Done Done Sent false
19470 5982 Acked Write 2 Done Done Sent false
19471 5983 Acked Write 1 Done Done Sent false
19472 5984 Acked Write 7 Done Done Sent false
19473 5985 Acked Write 5 Done Done Sent false
19474 5986 Acked Read 3 Done Done Sent false
19475 5987 Acked Read 10 Done Done Sent false
19476 5988 Acked Write 7 Done Done Sent false
19477 5989 Acked Write 4 Done Done Sent false
19478 5990 Acked Write 1 Done Done Sent false
19479 5991 Acked Read 7 Done Done Sent false
19480 5992 Acked Read 8 Done Done Sent false
19481 5993 Acked Read 7 Done Done Sent false
19482 5994 Acked Read 5 Done Done Sent false
19483 5995 Acked Read 4 Done Done Sent false
19484 5996 Acked Read 4 Done Done Sent false
19485 5997 Acked Write 5 Done Done Sent false
19486 5998 Acked Write 3 Done Done Sent false
19487 5999 Acked Read 4 Done Done Sent false
19488 6000 Acked Read 5 Done Done Sent false
19489 6001 Acked Read 4 Done Done Sent false
19490 6002 Acked Flush 0 Done Done Sent false
19491 6003 Acked Read 3 Done Done Sent false
19492 6004 Acked Write 1 Done Done Sent false
19493 6005 Acked Read 8 Done Done Sent false
19494 6006 Acked Flush 0 Done Done Sent false
19495 6007 Acked Read 5 Done Done Sent false
19496 6008 Acked Write 5 Done Done Sent false
19497 6009 Acked Write 7 Done Done Sent false
19498 6010 Acked Write 3 Done Done Sent false
19499 6011 Acked Write 7 Done Done Sent false
19500 6012 Acked Write 9 Done Done Sent false
19501 6013 Acked Write 1 Done Done Sent false
19502 6014 Acked Write 1 Done Done Sent false
19503 6015 Acked Read 5 Done Done Sent false
19504 6016 Acked Flush 0 Done Done Sent false
19505 6017 Acked Read 5 Done Done Sent false
19506 6018 Acked Flush 0 Done Done Sent false
19507 6019 Acked Read 7 Done Done Sent false
19508 6020 Acked Flush 0 Done Done Sent false
19509 6021 Acked Write 3 Done Done Sent false
19510 6022 Acked Write 3 Done Done Sent false
19511 6023 Acked Read 3 Done Done Sent false
19512 6024 Acked Flush 0 Done Done Sent false
19513 6025 Acked Write 7 Done Done Sent false
19514 6026 Acked Read 9 Done Done Sent false
19515 6027 Acked Read 5 Done Done Sent false
19516 6028 Acked Write 10 Done Done Sent false
19517 6029 Acked Write 3 Done Done Sent false
19518 6030 Acked Read 9 Done Done Sent false
19519 6031 Acked Flush 0 Done Done Sent false
19520 6032 Acked Write 5 Done Done Sent false
19521 6033 Acked Flush 0 Done Done Sent false
19522 6034 Acked Read 6 Done Done Sent false
19523 6035 Acked Write 6 Done Done Sent false
19524 6036 Acked Read 9 Done Done Sent false
19525 6037 Acked Write 2 Done Done Sent false
19526 6038 Acked Read 4 Done Done Sent false
19527 6039 Acked Write 9 Done Done Sent false
19528 6040 Acked Write 2 Done Done Sent false
19529 6041 Acked Read 4 Done Done Sent false
19530 6042 Acked Flush 0 Done Done Sent false
19531 6043 Acked Flush 0 Done Done Sent false
19532 6044 Acked Write 9 Done Done Sent false
19533 6045 Acked Read 6 Done Done Sent false
19534 6046 Acked Flush 0 Done Done Sent false
19535 6047 Acked Read 8 Done Done Sent false
19536 6048 Acked Read 9 Done Done Sent false
19537 6049 Acked Read 3 Done Done Sent false
19538 6050 Acked Flush 0 Done Done Sent false
19539 6051 Acked Write 2 Done Done Sent false
19540 6052 Acked Read 8 Done Done Sent false
19541 6053 Acked Write 2 Done Done Sent false
19542 6054 Acked Read 1 Done Done Sent false
19543 6055 Acked Read 7 Done Done Sent false
19544 6056 Acked Read 7 Done Done Sent false
19545 6057 Acked Read 4 Done Done Sent false
19546 6058 Acked Read 9 Done Done Sent false
19547 6059 Acked Read 9 Done Done Sent false
19548 6060 Acked Write 1 Done Done Sent false
19549 6061 Acked Write 9 Done Done Sent false
19550 6062 Acked Flush 0 Done Done Sent false
19551 6063 Acked Read 6 Done Done Sent false
19552 6064 Acked Write 2 Done Done Sent false
19553 6065 Acked Write 8 Done Done Sent false
19554 6066 Acked Write 6 Done Done Sent false
19555 6067 Acked Read 9 Done Done Sent false
19556 6068 Acked Write 3 Done Done Sent false
19557 6069 Acked Write 10 Done Done Sent false
19558 6070 Acked Read 2 Done Done Sent false
19559 6071 Acked Write 8 Done Done Sent false
19560 6072 Acked Read 8 Done Done Sent false
19561 6073 Acked Write 4 Done Done Sent false
19562 6074 Acked Write 9 Done Done Sent false
19563 6075 Acked Write 1 Done Done Sent false
19564 6076 Acked Write 9 Done Done Sent false
19565 6077 Acked Write 8 Done Done Sent false
19566 6078 Acked Write 8 Done Done Sent false
19567 6079 Acked Read 5 Done Done Sent false
19568 6080 Acked Write 5 Done Done Sent false
19569 6081 Acked Write 2 Done Done Sent false
19570 6082 Acked Write 5 Done Done Sent false
19571 6083 Acked Flush 0 Done Done Sent false
19572 6084 Acked Read 2 Done Done Sent false
19573 6085 Acked Write 9 Done Done Sent false
19574 6086 Acked Read 2 Done Done Sent false
19575 6087 Acked Read 7 Done Done Sent false
19576 6088 Acked Write 1 Done Done Sent false
19577 6089 Acked Write 6 Done Done Sent false
19578 6090 Acked Read 8 Done Done Sent false
19579 6091 Acked Read 7 Done Done Sent false
19580 6092 Acked Flush 0 Done Done Sent false
19581 6093 Acked Flush 0 Done Done Sent false
19582 6094 Acked Write 8 Done Done Sent false
19583 6095 Acked Read 1 Done Done Sent false
19584 6096 Acked Write 1 Done Done Sent false
19585 6097 Acked Read 10 Done Done Sent false
19586 6098 Acked Read 9 Done Done Sent false
19587 6099 Acked Read 1 Done Done Sent false
19588 6100 Acked Read 4 Done Done Sent false
19589 6101 Acked Write 4 Done Done Sent false
19590 6102 Acked Write 5 Done Done Sent false
19591 6103 Acked Flush 0 Done Done Sent false
19592 6104 Acked Read 3 Done Done Sent false
19593 6105 Acked Flush 0 Done Done Sent false
19594 6106 Acked Write 4 Done Done Sent false
19595 6107 Acked Write 2 Done Done Sent false
19596 6108 Acked Write 2 Done Done Sent false
19597 6109 Acked Read 1 Done Done Sent false
19598 6110 Acked Read 1 Done Done Sent false
19599 6111 Acked Write 6 Done Done Sent false
19600 6112 Acked Write 4 Done Done Sent false
19601 6113 Acked Write 6 Done Done Sent false
19602 6114 Acked Write 2 Done Done Sent false
19603 6115 Acked Write 5 Done Done Sent false
19604 6116 Acked Write 9 Done Done Sent false
19605 6117 Acked Read 8 Done Done Sent false
19606 6118 Acked Write 8 Done Done Sent false
19607 6119 Acked Read 10 Done Done Sent false
19608 6120 Acked Flush 0 Done Done Sent false
19609 6121 Acked Write 9 Done Done Sent false
19610 6122 Acked Write 8 Done Done Sent false
19611 6123 Acked Write 10 Done Done Sent false
19612 6124 Acked Write 8 Done Done Sent false
19613 6125 Acked Read 5 Done Done Sent false
19614 6126 Acked Flush 0 Done Done Sent false
19615 6127 Acked Read 4 Done Done Sent false
19616 6128 Acked Read 1 Done Done Sent false
19617 6129 Acked Write 1 Done Done Sent false
19618 6130 Acked Write 4 Done Done Sent false
19619 STATES DS:0 DS:1 DS:2 TOTAL
19620 Sent 0 0 301 301
19621 Done 301 301 0 602
19622 Skipped 0 0 0 0
19623 Error 0 0 0 0
19624 Last Flush: 6126 6126 5829
19625 Downstairs last five completed: 5829 5828 5827 5826 5825
19626 Upstairs last five completed: 6130 6129 6128 6127 6126
19627 CLIENT: Up:0 ds:301 act:2
19628 [8830] got start action
19629 [8830] I am going to restart
19630 Starting downstairs at port 8830
19631 Make output file at "/tmp/dsc/downstairs-8830.txt"
19632 [8830][2] reports Starting
19633 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1219)
19634 [8830][2] reports Running
19635 ----------------------------------------------------------------
19636 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19637 States: Active Active Offline
19638 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19639 5830 Acked Write 8 Done Done Sent false
19640 5831 Acked Flush 0 Done Done Sent false
19641 5832 Acked Write 2 Done Done Sent false
19642 5833 Acked Write 3 Done Done Sent false
19643 5834 Acked Read 9 Done Done Sent false
19644 5835 Acked Write 1 Done Done Sent false
19645 5836 Acked Read 6 Done Done Sent false
19646 5837 Acked Read 4 Done Done Sent false
19647 5838 Acked Write 9 Done Done Sent false
19648 5839 Acked Read 4 Done Done Sent false
19649 5840 Acked Write 5 Done Done Sent false
19650 5841 Acked Flush 0 Done Done Sent false
19651 5842 Acked Write 10 Done Done Sent false
19652 5843 Acked Read 7 Done Done Sent false
19653 5844 Acked Flush 0 Done Done Sent false
19654 5845 Acked Read 4 Done Done Sent false
19655 5846 Acked Write 3 Done Done Sent false
19656 5847 Acked Read 9 Done Done Sent false
19657 5848 Acked Read 1 Done Done Sent false
19658 5849 Acked Write 1 Done Done Sent false
19659 5850 Acked Flush 0 Done Done Sent false
19660 5851 Acked Read 3 Done Done Sent false
19661 5852 Acked Write 8 Done Done Sent false
19662 5853 Acked Read 1 Done Done Sent false
19663 5854 Acked Read 7 Done Done Sent false
19664 5855 Acked Read 4 Done Done Sent false
19665 5856 Acked Read 8 Done Done Sent false
19666 5857 Acked Read 10 Done Done Sent false
19667 5858 Acked Write 2 Done Done Sent false
19668 5859 Acked Write 1 Done Done Sent false
19669 5860 Acked Read 7 Done Done Sent false
19670 5861 Acked Read 1 Done Done Sent false
19671 5862 Acked Write 10 Done Done Sent false
19672 5863 Acked Read 6 Done Done Sent false
19673 5864 Acked Read 4 Done Done Sent false
19674 5865 Acked Write 5 Done Done Sent false
19675 5866 Acked Read 2 Done Done Sent false
19676 5867 Acked Read 7 Done Done Sent false
19677 5868 Acked Read 10 Done Done Sent false
19678 5869 Acked Read 2 Done Done Sent false
19679 5870 Acked Read 4 Done Done Sent false
19680 5871 Acked Write 4 Done Done Sent false
19681 5872 Acked Read 3 Done Done Sent false
19682 5873 Acked Write 2 Done Done Sent false
19683 5874 Acked Read 5 Done Done Sent false
19684 5875 Acked Flush 0 Done Done Sent false
19685 5876 Acked Read 4 Done Done Sent false
19686 5877 Acked Read 5 Done Done Sent false
19687 5878 Acked Write 10 Done Done Sent false
19688 5879 Acked Write 3 Done Done Sent false
19689 5880 Acked Write 6 Done Done Sent false
19690 5881 Acked Read 7 Done Done Sent false
19691 5882 Acked Read 8 Done Done Sent false
19692 5883 Acked Read 3 Done Done Sent false
19693 5884 Acked Read 4 Done Done Sent false
19694 5885 Acked Write 4 Done Done Sent false
19695 5886 Acked Read 2 Done Done Sent false
19696 5887 Acked Write 1 Done Done Sent false
19697 5888 Acked Read 7 Done Done Sent false
19698 5889 Acked Read 8 Done Done Sent false
19699 5890 Acked Write 8 Done Done Sent false
19700 5891 Acked Read 10 Done Done Sent false
19701 5892 Acked Read 4 Done Done Sent false
19702 5893 Acked Write 7 Done Done Sent false
19703 5894 Acked Read 2 Done Done Sent false
19704 5895 Acked Write 4 Done Done Sent false
19705 5896 Acked Write 2 Done Done Sent false
19706 5897 Acked Flush 0 Done Done Sent false
19707 5898 Acked Write 10 Done Done Sent false
19708 5899 Acked Flush 0 Done Done Sent false
19709 5900 Acked Read 3 Done Done Sent false
19710 5901 Acked Write 3 Done Done Sent false
19711 5902 Acked Write 7 Done Done Sent false
19712 5903 Acked Read 1 Done Done Sent false
19713 5904 Acked Read 7 Done Done Sent false
19714 5905 Acked Write 5 Done Done Sent false
19715 5906 Acked Read 4 Done Done Sent false
19716 5907 Acked Read 5 Done Done Sent false
19717 5908 Acked Read 3 Done Done Sent false
19718 5909 Acked Read 8 Done Done Sent false
19719 5910 Acked Read 6 Done Done Sent false
19720 5911 Acked Write 7 Done Done Sent false
19721 5912 Acked Read 7 Done Done Sent false
19722 5913 Acked Write 7 Done Done Sent false
19723 5914 Acked Read 3 Done Done Sent false
19724 5915 Acked Read 6 Done Done Sent false
19725 5916 Acked Write 4 Done Done Sent false
19726 5917 Acked Read 7 Done Done Sent false
19727 5918 Acked Flush 0 Done Done Sent false
19728 5919 Acked Read 6 Done Done Sent false
19729 5920 Acked Read 7 Done Done Sent false
19730 5921 Acked Read 7 Done Done Sent false
19731 5922 Acked Flush 0 Done Done Sent false
19732 5923 Acked Write 6 Done Done Sent false
19733 5924 Acked Read 10 Done Done Sent false
19734 5925 Acked Read 4 Done Done Sent false
19735 5926 Acked Flush 0 Done Done Sent false
19736 5927 Acked Read 3 Done Done Sent false
19737 5928 Acked Read 8 Done Done Sent false
19738 5929 Acked Write 1 Done Done Sent false
19739 5930 Acked Write 8 Done Done Sent false
19740 5931 Acked Write 1 Done Done Sent false
19741 5932 Acked Read 6 Done Done Sent false
19742 5933 Acked Read 3 Done Done Sent false
19743 5934 Acked Write 2 Done Done Sent false
19744 5935 Acked Write 2 Done Done Sent false
19745 5936 Acked Read 10 Done Done Sent false
19746 5937 Acked Read 3 Done Done Sent false
19747 5938 Acked Read 7 Done Done Sent false
19748 5939 Acked Read 2 Done Done Sent false
19749 5940 Acked Flush 0 Done Done Sent false
19750 5941 Acked Write 1 Done Done Sent false
19751 5942 Acked Write 5 Done Done Sent false
19752 5943 Acked Flush 0 Done Done Sent false
19753 5944 Acked Read 5 Done Done Sent false
19754 5945 Acked Write 5 Done Done Sent false
19755 5946 Acked Read 1 Done Done Sent false
19756 5947 Acked Write 9 Done Done Sent false
19757 5948 Acked Read 1 Done Done Sent false
19758 5949 Acked Write 9 Done Done Sent false
19759 5950 Acked Write 5 Done Done Sent false
19760 5951 Acked Write 6 Done Done Sent false
19761 5952 Acked Read 6 Done Done Sent false
19762 5953 Acked Write 5 Done Done Sent false
19763 5954 Acked Read 1 Done Done Sent false
19764 5955 Acked Read 1 Done Done Sent false
19765 5956 Acked Read 3 Done Done Sent false
19766 5957 Acked Read 3 Done Done Sent false
19767 5958 Acked Write 7 Done Done Sent false
19768 5959 Acked Read 4 Done Done Sent false
19769 5960 Acked Read 7 Done Done Sent false
19770 5961 Acked Flush 0 Done Done Sent false
19771 5962 Acked Read 7 Done Done Sent false
19772 5963 Acked Read 7 Done Done Sent false
19773 5964 Acked Write 3 Done Done Sent false
19774 5965 Acked Read 6 Done Done Sent false
19775 5966 Acked Write 8 Done Done Sent false
19776 5967 Acked Read 5 Done Done Sent false
19777 5968 Acked Read 5 Done Done Sent false
19778 5969 Acked Write 10 Done Done Sent false
19779 5970 Acked Flush 0 Done Done Sent false
19780 5971 Acked Read 5 Done Done Sent false
19781 5972 Acked Read 8 Done Done Sent false
19782 5973 Acked Read 6 Done Done Sent false
19783 5974 Acked Read 3 Done Done Sent false
19784 5975 Acked Flush 0 Done Done Sent false
19785 5976 Acked Flush 0 Done Done Sent false
19786 5977 Acked Read 6 Done Done Sent false
19787 5978 Acked Write 1 Done Done Sent false
19788 5979 Acked Read 9 Done Done Sent false
19789 5980 Acked Read 1 Done Done Sent false
19790 5981 Acked Write 2 Done Done Sent false
19791 5982 Acked Write 2 Done Done Sent false
19792 5983 Acked Write 1 Done Done Sent false
19793 5984 Acked Write 7 Done Done Sent false
19794 5985 Acked Write 5 Done Done Sent false
19795 5986 Acked Read 3 Done Done Sent false
19796 5987 Acked Read 10 Done Done Sent false
19797 5988 Acked Write 7 Done Done Sent false
19798 5989 Acked Write 4 Done Done Sent false
19799 5990 Acked Write 1 Done Done Sent false
19800 5991 Acked Read 7 Done Done Sent false
19801 5992 Acked Read 8 Done Done Sent false
19802 5993 Acked Read 7 Done Done Sent false
19803 5994 Acked Read 5 Done Done Sent false
19804 5995 Acked Read 4 Done Done Sent false
19805 5996 Acked Read 4 Done Done Sent false
19806 5997 Acked Write 5 Done Done Sent false
19807 5998 Acked Write 3 Done Done Sent false
19808 5999 Acked Read 4 Done Done Sent false
19809 6000 Acked Read 5 Done Done Sent false
19810 6001 Acked Read 4 Done Done Sent false
19811 6002 Acked Flush 0 Done Done Sent false
19812 6003 Acked Read 3 Done Done Sent false
19813 6004 Acked Write 1 Done Done Sent false
19814 6005 Acked Read 8 Done Done Sent false
19815 6006 Acked Flush 0 Done Done Sent false
19816 6007 Acked Read 5 Done Done Sent false
19817 6008 Acked Write 5 Done Done Sent false
19818 6009 Acked Write 7 Done Done Sent false
19819 6010 Acked Write 3 Done Done Sent false
19820 6011 Acked Write 7 Done Done Sent false
19821 6012 Acked Write 9 Done Done Sent false
19822 6013 Acked Write 1 Done Done Sent false
19823 6014 Acked Write 1 Done Done Sent false
19824 6015 Acked Read 5 Done Done Sent false
19825 6016 Acked Flush 0 Done Done Sent false
19826 6017 Acked Read 5 Done Done Sent false
19827 6018 Acked Flush 0 Done Done Sent false
19828 6019 Acked Read 7 Done Done Sent false
19829 6020 Acked Flush 0 Done Done Sent false
19830 6021 Acked Write 3 Done Done Sent false
19831 6022 Acked Write 3 Done Done Sent false
19832 6023 Acked Read 3 Done Done Sent false
19833 6024 Acked Flush 0 Done Done Sent false
19834 6025 Acked Write 7 Done Done Sent false
19835 6026 Acked Read 9 Done Done Sent false
19836 6027 Acked Read 5 Done Done Sent false
19837 6028 Acked Write 10 Done Done Sent false
19838 6029 Acked Write 3 Done Done Sent false
19839 6030 Acked Read 9 Done Done Sent false
19840 6031 Acked Flush 0 Done Done Sent false
19841 6032 Acked Write 5 Done Done Sent false
19842 6033 Acked Flush 0 Done Done Sent false
19843 6034 Acked Read 6 Done Done Sent false
19844 6035 Acked Write 6 Done Done Sent false
19845 6036 Acked Read 9 Done Done Sent false
19846 6037 Acked Write 2 Done Done Sent false
19847 6038 Acked Read 4 Done Done Sent false
19848 6039 Acked Write 9 Done Done Sent false
19849 6040 Acked Write 2 Done Done Sent false
19850 6041 Acked Read 4 Done Done Sent false
19851 6042 Acked Flush 0 Done Done Sent false
19852 6043 Acked Flush 0 Done Done Sent false
19853 6044 Acked Write 9 Done Done Sent false
19854 6045 Acked Read 6 Done Done Sent false
19855 6046 Acked Flush 0 Done Done Sent false
19856 6047 Acked Read 8 Done Done Sent false
19857 6048 Acked Read 9 Done Done Sent false
19858 6049 Acked Read 3 Done Done Sent false
19859 6050 Acked Flush 0 Done Done Sent false
19860 6051 Acked Write 2 Done Done Sent false
19861 6052 Acked Read 8 Done Done Sent false
19862 6053 Acked Write 2 Done Done Sent false
19863 6054 Acked Read 1 Done Done Sent false
19864 6055 Acked Read 7 Done Done Sent false
19865 6056 Acked Read 7 Done Done Sent false
19866 6057 Acked Read 4 Done Done Sent false
19867 6058 Acked Read 9 Done Done Sent false
19868 6059 Acked Read 9 Done Done Sent false
19869 6060 Acked Write 1 Done Done Sent false
19870 6061 Acked Write 9 Done Done Sent false
19871 6062 Acked Flush 0 Done Done Sent false
19872 6063 Acked Read 6 Done Done Sent false
19873 6064 Acked Write 2 Done Done Sent false
19874 6065 Acked Write 8 Done Done Sent false
19875 6066 Acked Write 6 Done Done Sent false
19876 6067 Acked Read 9 Done Done Sent false
19877 6068 Acked Write 3 Done Done Sent false
19878 6069 Acked Write 10 Done Done Sent false
19879 6070 Acked Read 2 Done Done Sent false
19880 6071 Acked Write 8 Done Done Sent false
19881 6072 Acked Read 8 Done Done Sent false
19882 6073 Acked Write 4 Done Done Sent false
19883 6074 Acked Write 9 Done Done Sent false
19884 6075 Acked Write 1 Done Done Sent false
19885 6076 Acked Write 9 Done Done Sent false
19886 6077 Acked Write 8 Done Done Sent false
19887 6078 Acked Write 8 Done Done Sent false
19888 6079 Acked Read 5 Done Done Sent false
19889 6080 Acked Write 5 Done Done Sent false
19890 6081 Acked Write 2 Done Done Sent false
19891 6082 Acked Write 5 Done Done Sent false
19892 6083 Acked Flush 0 Done Done Sent false
19893 6084 Acked Read 2 Done Done Sent false
19894 6085 Acked Write 9 Done Done Sent false
19895 6086 Acked Read 2 Done Done Sent false
19896 6087 Acked Read 7 Done Done Sent false
19897 6088 Acked Write 1 Done Done Sent false
19898 6089 Acked Write 6 Done Done Sent false
19899 6090 Acked Read 8 Done Done Sent false
19900 6091 Acked Read 7 Done Done Sent false
19901 6092 Acked Flush 0 Done Done Sent false
19902 6093 Acked Flush 0 Done Done Sent false
19903 6094 Acked Write 8 Done Done Sent false
19904 6095 Acked Read 1 Done Done Sent false
19905 6096 Acked Write 1 Done Done Sent false
19906 6097 Acked Read 10 Done Done Sent false
19907 6098 Acked Read 9 Done Done Sent false
19908 6099 Acked Read 1 Done Done Sent false
19909 6100 Acked Read 4 Done Done Sent false
19910 6101 Acked Write 4 Done Done Sent false
19911 6102 Acked Write 5 Done Done Sent false
19912 6103 Acked Flush 0 Done Done Sent false
19913 6104 Acked Read 3 Done Done Sent false
19914 6105 Acked Flush 0 Done Done Sent false
19915 6106 Acked Write 4 Done Done Sent false
19916 6107 Acked Write 2 Done Done Sent false
19917 6108 Acked Write 2 Done Done Sent false
19918 6109 Acked Read 1 Done Done Sent false
19919 6110 Acked Read 1 Done Done Sent false
19920 6111 Acked Write 6 Done Done Sent false
19921 6112 Acked Write 4 Done Done Sent false
19922 6113 Acked Write 6 Done Done Sent false
19923 6114 Acked Write 2 Done Done Sent false
19924 6115 Acked Write 5 Done Done Sent false
19925 6116 Acked Write 9 Done Done Sent false
19926 6117 Acked Read 8 Done Done Sent false
19927 6118 Acked Write 8 Done Done Sent false
19928 6119 Acked Read 10 Done Done Sent false
19929 6120 Acked Flush 0 Done Done Sent false
19930 6121 Acked Write 9 Done Done Sent false
19931 6122 Acked Write 8 Done Done Sent false
19932 6123 Acked Write 10 Done Done Sent false
19933 6124 Acked Write 8 Done Done Sent false
19934 6125 Acked Read 5 Done Done Sent false
19935 6126 Acked Flush 0 Done Done Sent false
19936 6127 Acked Read 4 Done Done Sent false
19937 6128 Acked Read 1 Done Done Sent false
19938 6129 Acked Write 1 Done Done Sent false
19939 6130 Acked Write 4 Done Done Sent false
19940 6131 Acked Flush 0 Done Done Sent false
19941 STATES DS:0 DS:1 DS:2 TOTAL
19942 Sent 0 0 302 302
19943 Done 302 302 0 604
19944 Skipped 0 0 0 0
19945 Error 0 0 0 0
19946 Last Flush: 6131 6131 5829
19947 Downstairs last five completed: 5829 5828 5827 5826 5825
19948 Upstairs last five completed: 6131 6130 6129 6128 6127
19949 CLIENT: Up:0 ds:302 act:2
199502024-11-19T17:12:09.676ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199512024-11-19T17:12:09.676ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64472) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199522024-11-19T17:12:09.677ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199532024-11-19T17:12:09.678ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199542024-11-19T17:12:09.678ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199552024-11-19T17:12:09.678ZINFOcrucible: send last flush ID to this DS: 5829 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199562024-11-19T17:12:09.678ZINFOcrucible: Replied this last flush ID: 5829 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199572024-11-19T17:12:09.678ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199582024-11-19T17:12:09.678ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199592024-11-19T17:12:09.678ZINFOcrucible: [2] client re-new 302 jobs since flush 5829 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
199602024-11-19T17:12:09.678ZINFOcrucible: [2] Replayed 302 jobs since flush: 5829 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
19961 ----------------------------------------------------------------
19962 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19963 Downstairs last five completed: 6131 6130 6129 6128 6127
19964 Upstairs last five completed: 6131 6130 6129 6128 6127
19965 CLIENT: Up:0 ds:0 act:3
19966 Replay: All jobs finished
19967 Main task has work to do, go find it
19968 got dsc Stop(2)
19969 stop 2
19970 Nov 19 17:12:11.909[8830] Got stop action so:false kr:false
19971 INFO request completed, latency_us: 310, response_code: 204, uri: /stop/cid/2, method: POST, req_id: a82e4925-c777-4078-9ff4-ba692cf2f134, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
19972 Nov 19 17:12:11.910 INFO request completed, latency_us: 140, response_code: 200{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:11.910098268Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
19973 , uri: /state/cid/2, method: GET, req_id: c92910e9-d235-4ef5-ada4-d7979f5543a9, remote_addr: 127.0.0.1:64175, local_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:11.910165312Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
19974 .0.0.1:9998
199752024-11-19T17:12:11.910ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
199762024-11-19T17:12:11.910ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199772024-11-19T17:12:11.910ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
199782024-11-19T17:12:11.910ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
19979 [8830] Exited with: ExitStatus(unix_wait_status(9))
19980 [8830][2] reports Exit
19981 Nov 19 17:12:15.911 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/2, method: GET, req_id: 561ea0ac-5cc0-48cc-a7d9-d3244a348181, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
19982 001/300 Read block 403 len 40960
19983 002/300 Read block 1748 len 8192
19984 003/300 Write block 2659 len 40960 data: 3 5 5 6 7 6 6 6 7 5
19985 004/300 Flush
19986 005/300 Write block 2931 len 24576 data: 8 7 6 7 6 8
19987 006/300 Read block 552 len 24576
19988 007/300 Read block 4862 len 40960
19989 008/300 Flush
19990 009/300 Write block 994 len 36864 data: 4 3 3 4 4 4 3 3 3
19991 010/300 Write block 4651 len 12288 data: 4 5 3
19992 011/300 Flush
19993 012/300 Read block 3846 len 20480
19994 013/300 Write block 4892 len 28672 data: 6 6 6 6 7 6 6
19995 014/300 Read block 1060 len 32768
19996 015/300 Write block 1901 len 28672 data: 3 3 3 4 4 4 5
19997 016/300 Read block 770 len 36864
19998 017/300 Write block 614 len 24576 data: 4 4 6 5 6 6
19999 018/300 Read block 3634 len 36864
20000 019/300 Read block 2453 len 24576
20001 020/300 Read block 622 len 24576
20002 021/300 Read block 921 len 40960
20003 022/300 Write block 2508 len 20480 data: 5 5 3 3 3
20004 023/300 Write block 3849 len 40960 data: 3 3 2 2 2 4 6 6 6 7
20005 024/300 Read block 1130 len 8192
20006 025/300 Flush
20007 026/300 Write block 3850 len 20480 data: 4 3 3 3 5
20008 027/300 Read block 1257 len 32768
20009 028/300 Read block 3651 len 12288
20010 029/300 Read block 4965 len 32768
20011 030/300 Write block 1160 len 12288 data: 5 4 5
20012 031/300 Flush
20013 032/300 Write block 4745 len 4096 data: 3
20014 033/300 Flush
20015 034/300 Read block 609 len 32768
20016 035/300 Read block 3322 len 4096
20017 036/300 Flush
20018 037/300 Read block 3706 len 16384
20019 038/300 Write block 1552 len 16384 data: 4 4 4 4
20020 039/300 Write block 1979 len 28672 data: 6 6 5 4 4 3 3
20021 040/300 Read block 2401 len 4096
20022 041/300 Write block 2731 len 40960 data: 5 5 5 4 3 3 3 4 3 3
20023 042/300 Read block 2230 len 12288
20024 043/300 Write block 1179 len 4096 data: 3
20025 044/300 Read block 3119 len 28672
20026 045/300 Flush
20027 046/300 Write block 4336 len 12288 data: 4 3 3
20028 047/300 Write block 3981 len 8192 data: 3 3
20029 048/300 Write block 709 len 36864 data: 4 5 4 4 3 2 3 3 3
20030 049/300 Flush
20031 050/300 Read block 260 len 40960
20032 051/300 Flush
20033 052/300 Read block 452 len 28672
20034 053/300 Write block 2989 len 8192 data: 3 5
20035 054/300 Write block 3482 len 40960 data: 8 7 8 9 9 8 5 6 5 5
20036 055/300 Read block 2965 len 24576
20037 056/300 Flush
20038 057/300 Read block 2047 len 4096
20039 058/300 Read block 1308 len 24576
20040 059/300 Write block 819 len 16384 data: 4 5 4 4
20041 060/300 Read block 845 len 36864
20042 061/300 Read block 2861 len 28672
20043 062/300 Write block 3826 len 28672 data: 8 7 6 6 5 4 4
20044 063/300 Read block 3992 len 32768
20045 064/300 Write block 1533 len 24576 data: 4 3 3 2 2 2
20046 065/300 Read block 275 len 36864
20047 066/300 Flush
20048 067/300 Read block 2913 len 4096
20049 068/300 Read block 943 len 12288
20050 069/300 Write block 3557 len 4096 data: 4
20051 070/300 Read block 484 len 16384
20052 071/300 Write block 559 len 32768 data: 2 3 3 3 2 2 2 3
20053 072/300 Read block 453 len 36864
20054 073/300 Read block 674 len 36864
20055 074/300 Read block 867 len 8192
20056 075/300 Write block 1657 len 20480 data: 2 2 4 4 2
20057 076/300 Flush
20058 077/300 Write block 3126 len 24576 data: 9 7 8 8 8 8
20059 078/300 Read block 476 len 8192
20060 079/300 Read block 489 len 36864
20061 080/300 Read block 3198 len 36864
20062 081/300 Write block 4778 len 16384 data: 7 6 5 4
20063 082/300 Write block 3513 len 16384 data: 4 4 4 3
20064 083/300 Write block 317 len 24576 data: 6 7 7 8 6 6
20065 084/300 Write block 1175 len 12288 data: 4 4 4
20066 085/300 Write block 411 len 36864 data: 2 3 3 3 3 4 4 3 5
20067 086/300 Flush
20068 087/300 Read block 1221 len 12288
20069 088/300 Read block 1133 len 32768
20070 089/300 Write block 2800 len 12288 data: 6 5 6
20071 090/300 Read block 1588 len 28672
20072 091/300 Read block 3870 len 16384
20073 092/300 Flush
20074 093/300 Read block 2910 len 16384
20075 094/300 Write block 1075 len 12288 data: 4 4 5
20076 095/300 Read block 596 len 28672
20077 096/300 Read block 2150 len 8192
20078 097/300 Write block 3453 len 16384 data: 6 5 6 7
20079 098/300 Read block 2695 len 12288
20080 099/300 Read block 4138 len 40960
20081 100/300 Read block 2418 len 16384
20082 101/300 Read block 1715 len 20480
20083 102/300 Write block 1823 len 36864 data: 6 5 4 5 4 4 3 3 2
20084 103/300 Read block 3776 len 20480
20085 104/300 Write block 2423 len 16384 data: 4 4 6 5
20086 105/300 Read block 1381 len 36864
20087 106/300 Read block 4609 len 40960
20088 107/300 Read block 1822 len 8192
20089 108/300 Flush
20090 109/300 Read block 4499 len 28672
20091 110/300 Flush
20092 111/300 Write block 1051 len 24576 data: 3 3 3 3 2 2
20093 112/300 Write block 4370 len 4096 data: 4
20094 113/300 Write block 771 len 28672 data: 3 3 4 4 4 4 6
20095 114/300 Write block 1827 len 16384 data: 5 5 4 4
20096 115/300 Write block 815 len 8192 data: 3 3
20097 116/300 Write block 565 len 8192 data: 3 4
20098 117/300 Read block 2030 len 24576
20099 118/300 Flush
20100 119/300 Read block 4164 len 40960
20101 120/300 Read block 1852 len 20480
20102 121/300 Read block 719 len 16384
20103 122/300 Read block 4168 len 32768
20104 123/300 Write block 217 len 8192 data: 6 7
20105 124/300 Write block 3949 len 20480 data: 4 4 6 6 5
20106 125/300 Write block 3153 len 28672 data: 6 4 4 4 4 3 3
20107 126/300 Read block 317 len 16384
20108 127/300 Flush
20109 128/300 Write block 4184 len 24576 data: 6 5 5 6 7 8
20110 129/300 Read block 4703 len 8192
20111 130/300 Write block 3915 len 16384 data: 7 7 6 5
20112 131/300 Read block 414 len 4096
20113 132/300 Write block 2254 len 20480 data: 7 6 7 7 5
20114 133/300 Read block 4447 len 16384
20115 134/300 Read block 3474 len 8192
20116 135/300 Read block 3175 len 4096
20117 136/300 Read block 4040 len 16384
20118 137/300 Write block 167 len 32768 data: 2 2 2 2 2 2 3 4
20119 138/300 Write block 1049 len 24576 data: 3 3 4 4 4 4
20120 139/300 Read block 1025 len 16384
20121 140/300 Write block 4043 len 20480 data: 4 4 4 5 5
20122 141/300 Read block 3222 len 24576
20123 142/300 Write block 4145 len 16384 data: 4 4 3 4
20124 143/300 Write block 2896 len 8192 data: 6 5
20125 144/300 Write block 4069 len 4096 data: 4
20126 145/300 Write block 2186 len 16384 data: 3 3 3 3
20127 146/300 Write block 3313 len 12288 data: 6 5 6
20128 147/300 Write block 1884 len 32768 data: 9 7 5 5 5 4 5 5
20129 148/300 Flush
20130 149/300 Read block 704 len 8192
20131 150/300 Flush
20132 151/300 Write block 3403 len 20480 data: 3 3 2 3 3
20133 152/300 Flush
20134 153/300 Read block 685 len 8192
20135 154/300 Read block 2559 len 28672
20136 155/300 Write block 1191 len 20480 data: 8 6 7 7 7
20137 156/300 Read block 3929 len 24576
20138 157/300 Read block 4059 len 40960
20139 158/300 Read block 660 len 40960
20140 159/300 Read block 1783 len 20480
20141 160/300 Flush
20142 161/300 Read block 445 len 4096
20143 162/300 Read block 594 len 20480
20144 163/300 Read block 1233 len 24576
20145 164/300 Write block 2677 len 40960 data: 5 5 5 5 5 6 6 6 6 6
20146 165/300 Flush
20147 166/300 Read block 3777 len 8192
20148 167/300 Write block 2271 len 12288 data: 6 6 6
20149 168/300 Flush
20150 169/300 Write block 1763 len 8192 data: 5 5
20151 170/300 Write block 426 len 8192 data: 6 5
20152 171/300 Read block 4671 len 16384
20153 172/300 Read block 1950 len 36864
20154 173/300 Flush
20155 174/300 Read block 4012 len 8192
20156 175/300 Write block 670 len 28672 data: 9 9 9 9 9 8 8
20157 176/300 Write block 18 len 4096 data: 3
20158 177/300 Read block 4470 len 12288
20159 178/300 Read block 78 len 12288
20160 179/300 Read block 1608 len 28672
20161 180/300 Write block 2824 len 20480 data: 2 2 2 2 2
20162 181/300 Write block 1703 len 12288 data: 3 5 4
20163 182/300 Write block 1103 len 4096 data: 3
20164 183/300 Read block 4839 len 24576
20165 184/300 Flush
20166 185/300 Write block 4756 len 16384 data: 3 3 4 5
20167 186/300 Read block 4437 len 32768
20168 187/300 Read block 3226 len 20480
20169 188/300 Read block 1803 len 12288
20170 189/300 Read block 1221 len 20480
20171 190/300 Write block 4241 len 36864 data: 3 4 3 3 4 3 4 4 4
20172 191/300 Write block 4989 len 32768 data: 3 3 4 4 4 3 2 2
20173 192/300 Read block 3215 len 40960
20174 193/300 Read block 2601 len 16384
20175 194/300 Write block 3874 len 8192 data: 5 4
20176 195/300 Write block 1952 len 8192 data: 5 4
20177 196/300 Flush
20178 197/300 Read block 3812 len 20480
20179 198/300 Read block 3445 len 24576
20180 199/300 Read block 2893 len 36864
20181 200/300 Read block 4627 len 8192
20182 201/300 Flush
20183 202/300 Read block 278 len 28672
20184 203/300 Write block 571 len 28672 data: 4 3 3 2 3 3 3
20185 204/300 Write block 4322 len 36864 data: 3 3 3 3 3 3 3 3 4
20186 205/300 Read block 4948 len 12288
20187 206/300 Read block 3712 len 16384
20188 207/300 Write block 822 len 40960 data: 5 5 7 7 6 5 4 5 6 6
20189 208/300 Read block 1844 len 40960
20190 209/300 Read block 1640 len 4096
20191 210/300 Read block 774 len 28672
20192 211/300 Write block 3174 len 12288 data: 4 4 4
20193 212/300 Flush
20194 213/300 Read block 1593 len 40960
20195 214/300 Read block 1503 len 12288
20196 215/300 Write block 3873 len 24576 data: 5 6 5 3 3 4
20197 216/300 Read block 380 len 16384
20198 217/300 Flush
20199 218/300 Write block 4455 len 32768 data: 4 5 4 5 5 4 4 5
20200 219/300 Read block 1589 len 40960
20201 220/300 Read block 2903 len 40960
20202 221/300 Read block 78 len 36864
20203 222/300 Write block 2929 len 24576 data: 3 7 9 8 7 8
20204 223/300 Read block 2226 len 8192
20205 224/300 Write block 1614 len 16384 data: 9 10 9 9
20206 225/300 Read block 4089 len 20480
20207 226/300 Read block 4874 len 32768
20208 227/300 Write block 4036 len 8192 data: 3 4
20209 228/300 Write block 1824 len 24576 data: 6 5 6 6 6 5
20210 229/300 Read block 1368 len 32768
20211 230/300 Read block 2457 len 24576
20212 231/300 Write block 1817 len 8192 data: 3 4
20213 232/300 Write block 1288 len 16384 data: 8 10 10 7
20214 233/300 Read block 3358 len 24576
20215 234/300 Read block 2571 len 32768
20216 235/300 Read block 1963 len 20480
20217 236/300 Read block 3910 len 36864
20218 237/300 Write block 1943 len 36864 data: 5 5 4 4 3 4 3 4 5
20219 238/300 Read block 3128 len 20480
20220 239/300 Read block 707 len 36864
20221 240/300 Write block 373 len 36864 data: 3 3 2 2 3 3 3 3 3
20222 241/300 Write block 4080 len 28672 data: 4 4 5 5 6 6 6
20223 242/300 Write block 2795 len 12288 data: 4 5 5
20224 243/300 Read block 3041 len 36864
20225 244/300 Read block 4971 len 36864
20226 245/300 Read block 4627 len 36864
20227 246/300 Read block 2653 len 32768
20228 247/300 Write block 4757 len 8192 data: 4 5
20229 248/300 Read block 2077 len 36864
20230 249/300 Write block 3031 len 28672 data: 5 5 7 5 4 4 4
20231 250/300 Write block 1444 len 20480 data: 3 3 6 8 7
20232 251/300 Write block 3710 len 28672 data: 5 5 5 4 5 4 3
20233 252/300 Read block 595 len 28672
20234 253/300 Write block 4803 len 12288 data: 5 4 5
20235 254/300 Read block 294 len 4096
20236 255/300 Write block 3634 len 32768 data: 3 4 4 4 4 2 2 2
20237 256/300 Write block 3078 len 32768 data: 3 3 5 4 3 3 3 3
20238 257/300 Read block 752 len 32768
20239 258/300 Write block 4938 len 16384 data: 6 8 8 5
20240 259/300 Read block 2833 len 16384
20241 260/300 Read block 4457 len 12288
20242 261/300 Read block 1625 len 12288
20243 262/300 Read block 4579 len 32768
20244 263/300 Write block 2988 len 36864 data: 3 4 6 5 6 7 7 7 6
20245 264/300 Write block 1102 len 40960 data: 4 4 4 2 2 2 2 3 3 4
20246 265/300 Write block 2117 len 36864 data: 2 4 3 2 2 2 2 4 4
20247 266/300 Read block 605 len 8192
20248 267/300 Write block 3428 len 40960 data: 4 4 3 3 4 4 3 4 4 5
20249 268/300 Read block 3910 len 36864
20250 269/300 Read block 3652 len 36864
20251 270/300 Write block 3491 len 28672 data: 6 3 4 2 3 4 4
20252 271/300 Read block 4511 len 36864
20253 272/300 Write block 4322 len 12288 data: 4 4 4
20254 273/300 Write block 1322 len 4096 data: 6
20255 274/300 Read block 774 len 4096
20256 275/300 Read block 790 len 32768
20257 276/300 Read block 2982 len 24576
20258 277/300 Read block 1179 len 40960
20259 278/300 Read block 4207 len 24576
20260 279/300 Read block 4744 len 16384
20261 280/300 Write block 446 len 28672 data: 3 2 3 3 3 3 2
20262 281/300 Write block 718 len 24576 data: 3 3 3 3 4 6
20263 282/300 Read block 4918 len 24576
20264 283/300 Flush
20265 284/300 Flush
20266 285/300 Write block 4322 len 12288 data: 5 5 5
20267 286/300 Read block 3816 len 4096
20268 287/300 Read block 1866 len 16384
20269 288/300 Read block 4792 len 12288
20270 289/300 Write block 890 len 16384 data: 4 5 5 5
20271 290/300 Read block 1123 len 36864
20272 291/300 Write block 4513 len 28672 data: 4 4 4 5 6 6 6
20273 292/300 Flush
20274 293/300 Read block 35 len 40960
20275 294/300 Write block 4797 len 4096 data: 5
20276 295/300 Write block 901 len 20480 data: 5 6 6 6 7
20277 296/300 Read block 2114 len 28672
20278 297/300 Flush
20279 298/300 Write block 300 len 12288 data: 4 4 4
20280 299/300 Read block 3181 len 32768
20281 300/300 Read block 4564 len 16384
20282 301/300 Read block 3211 len 20480
20283 Main task has work to do, go find it
20284 got dsc Start(2)
20285 start 2
20286 Nov 19 17:12:16.154 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/2, method: POST, req_id: d13dc15e-70c7-463c-9a72-cfa42fa480a5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
20287 Replay: started 2, returned:Ok(())
20288 ----------------------------------------------------------------
20289 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20290 States: Active Active Offline
20291 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20292 6132 Acked Read 10 Done Done Sent false
20293 6133 Acked Read 2 Done Done Sent false
20294 6134 Acked Write 10 Done Done Sent false
20295 6135 Acked Flush 0 Done Done Sent false
20296 6136 Acked Write 6 Done Done Sent false
20297 6137 Acked Read 6 Done Done Sent false
20298 6138 Acked Read 10 Done Done Sent false
20299 6139 Acked Flush 0 Done Done Sent false
20300 6140 Acked Write 9 Done Done Sent false
20301 6141 Acked Write 3 Done Done Sent false
20302 6142 Acked Flush 0 Done Done Sent false
20303 6143 Acked Read 5 Done Done Sent false
20304 6144 Acked Write 7 Done Done Sent false
20305 6145 Acked Read 8 Done Done Sent false
20306 6146 Acked Write 7 Done Done Sent false
20307 6147 Acked Read 9 Done Done Sent false
20308 6148 Acked Write 6 Done Done Sent false
20309 6149 Acked Read 9 Done Done Sent false
20310 6150 Acked Read 6 Done Done Sent false
20311 6151 Acked Read 6 Done Done Sent false
20312 6152 Acked Read 10 Done Done Sent false
20313 6153 Acked Write 5 Done Done Sent false
20314 6154 Acked Write 10 Done Done Sent false
20315 6155 Acked Read 2 Done Done Sent false
20316 6156 Acked Flush 0 Done Done Sent false
20317 6157 Acked Write 5 Done Done Sent false
20318 6158 Acked Read 8 Done Done Sent false
20319 6159 Acked Read 3 Done Done Sent false
20320 6160 Acked Read 8 Done Done Sent false
20321 6161 Acked Write 3 Done Done Sent false
20322 6162 Acked Flush 0 Done Done Sent false
20323 6163 Acked Write 1 Done Done Sent false
20324 6164 Acked Flush 0 Done Done Sent false
20325 6165 Acked Read 8 Done Done Sent false
20326 6166 Acked Read 1 Done Done Sent false
20327 6167 Acked Flush 0 Done Done Sent false
20328 6168 Acked Read 4 Done Done Sent false
20329 6169 Acked Write 4 Done Done Sent false
20330 6170 Acked Write 7 Done Done Sent false
20331 6171 Acked Read 1 Done Done Sent false
20332 6172 Acked Write 10 Done Done Sent false
20333 6173 Acked Read 3 Done Done Sent false
20334 6174 Acked Write 1 Done Done Sent false
20335 6175 Acked Read 7 Done Done Sent false
20336 6176 Acked Flush 0 Done Done Sent false
20337 6177 Acked Write 3 Done Done Sent false
20338 6178 Acked Write 2 Done Done Sent false
20339 6179 Acked Write 9 Done Done Sent false
20340 6180 Acked Flush 0 Done Done Sent false
20341 6181 Acked Read 10 Done Done Sent false
20342 6182 Acked Flush 0 Done Done Sent false
20343 6183 Acked Read 7 Done Done Sent false
20344 6184 Acked Write 2 Done Done Sent false
20345 6185 Acked Write 10 Done Done Sent false
20346 6186 Acked Read 6 Done Done Sent false
20347 6187 Acked Flush 0 Done Done Sent false
20348 6188 Acked Read 1 Done Done Sent false
20349 6189 Acked Read 6 Done Done Sent false
20350 6190 Acked Write 4 Done Done Sent false
20351 6191 Acked Read 9 Done Done Sent false
20352 6192 Acked Read 7 Done Done Sent false
20353 6193 Acked Write 7 Done Done Sent false
20354 6194 Acked Read 8 Done Done Sent false
20355 6195 Acked Write 6 Done Done Sent false
20356 6196 Acked Read 9 Done Done Sent false
20357 6197 Acked Flush 0 Done Done Sent false
20358 6198 Acked Read 1 Done Done Sent false
20359 6199 Acked Read 3 Done Done Sent false
20360 6200 Acked Write 1 Done Done Sent false
20361 6201 Acked Read 4 Done Done Sent false
20362 6202 Acked Write 8 Done Done Sent false
20363 6203 Acked Read 9 Done Done Sent false
20364 6204 Acked Read 9 Done Done Sent false
20365 6205 Acked Read 2 Done Done Sent false
20366 6206 Acked Write 5 Done Done Sent false
20367 6207 Acked Flush 0 Done Done Sent false
20368 6208 Acked Write 6 Done Done Sent false
20369 6209 Acked Read 2 Done Done Sent false
20370 6210 Acked Read 9 Done Done Sent false
20371 6211 Acked Read 9 Done Done Sent false
20372 6212 Acked Write 4 Done Done Sent false
20373 6213 Acked Write 4 Done Done Sent false
20374 6214 Acked Write 6 Done Done Sent false
20375 6215 Acked Write 3 Done Done Sent false
20376 6216 Acked Write 9 Done Done Sent false
20377 6217 Acked Flush 0 Done Done Sent false
20378 6218 Acked Read 3 Done Done Sent false
20379 6219 Acked Read 8 Done Done Sent false
20380 6220 Acked Write 3 Done Done Sent false
20381 6221 Acked Read 7 Done Done Sent false
20382 6222 Acked Read 4 Done Done Sent false
20383 6223 Acked Flush 0 Done Done Sent false
20384 6224 Acked Read 4 Done Done Sent false
20385 6225 Acked Write 3 Done Done Sent false
20386 6226 Acked Read 7 Done Done Sent false
20387 6227 Acked Read 2 Done Done Sent false
20388 6228 Acked Write 4 Done Done Sent false
20389 6229 Acked Read 3 Done Done Sent false
20390 6230 Acked Read 10 Done Done Sent false
20391 6231 Acked Read 4 Done Done Sent false
20392 6232 Acked Read 5 Done Done Sent false
20393 6233 Acked Write 9 Done Done Sent false
20394 6234 Acked Read 5 Done Done Sent false
20395 6235 Acked Write 4 Done Done Sent false
20396 6236 Acked Read 9 Done Done Sent false
20397 6237 Acked Read 10 Done Done Sent false
20398 6238 Acked Read 2 Done Done Sent false
20399 6239 Acked Flush 0 Done Done Sent false
20400 6240 Acked Read 7 Done Done Sent false
20401 6241 Acked Flush 0 Done Done Sent false
20402 6242 Acked Write 6 Done Done Sent false
20403 6243 Acked Write 1 Done Done Sent false
20404 6244 Acked Write 7 Done Done Sent false
20405 6245 Acked Write 4 Done Done Sent false
20406 6246 Acked Write 2 Done Done Sent false
20407 6247 Acked Write 2 Done Done Sent false
20408 6248 Acked Read 6 Done Done Sent false
20409 6249 Acked Flush 0 Done Done Sent false
20410 6250 Acked Read 10 Done Done Sent false
20411 6251 Acked Read 5 Done Done Sent false
20412 6252 Acked Read 4 Done Done Sent false
20413 6253 Acked Read 8 Done Done Sent false
20414 6254 Acked Write 2 Done Done Sent false
20415 6255 Acked Write 5 Done Done Sent false
20416 6256 Acked Write 7 Done Done Sent false
20417 6257 Acked Read 4 Done Done Sent false
20418 6258 Acked Flush 0 Done Done Sent false
20419 6259 Acked Write 6 Done Done Sent false
20420 6260 Acked Read 2 Done Done Sent false
20421 6261 Acked Write 4 Done Done Sent false
20422 6262 Acked Read 1 Done Done Sent false
20423 6263 Acked Write 5 Done Done Sent false
20424 6264 Acked Read 4 Done Done Sent false
20425 6265 Acked Read 2 Done Done Sent false
20426 6266 Acked Read 1 Done Done Sent false
20427 6267 Acked Read 4 Done Done Sent false
20428 6268 Acked Write 8 Done Done Sent false
20429 6269 Acked Write 6 Done Done Sent false
20430 6270 Acked Read 4 Done Done Sent false
20431 6271 Acked Write 5 Done Done Sent false
20432 6272 Acked Read 6 Done Done Sent false
20433 6273 Acked Write 4 Done Done Sent false
20434 6274 Acked Write 2 Done Done Sent false
20435 6275 Acked Write 1 Done Done Sent false
20436 6276 Acked Write 4 Done Done Sent false
20437 6277 Acked Write 3 Done Done Sent false
20438 6278 Acked Write 8 Done Done Sent false
20439 6279 Acked Flush 0 Done Done Sent false
20440 6280 Acked Read 2 Done Done Sent false
20441 6281 Acked Flush 0 Done Done Sent false
20442 6282 Acked Write 5 Done Done Sent false
20443 6283 Acked Flush 0 Done Done Sent false
20444 6284 Acked Read 2 Done Done Sent false
20445 6285 Acked Read 7 Done Done Sent false
20446 6286 Acked Write 5 Done Done Sent false
20447 6287 Acked Read 6 Done Done Sent false
20448 6288 Acked Read 10 Done Done Sent false
20449 6289 Acked Read 10 Done Done Sent false
20450 6290 Acked Read 5 Done Done Sent false
20451 6291 Acked Flush 0 Done Done Sent false
20452 6292 Acked Read 1 Done Done Sent false
20453 6293 Acked Read 5 Done Done Sent false
20454 6294 Acked Read 6 Done Done Sent false
20455 6295 Acked Write 10 Done Done Sent false
20456 6296 Acked Flush 0 Done Done Sent false
20457 6297 Acked Read 2 Done Done Sent false
20458 6298 Acked Write 3 Done Done Sent false
20459 6299 Acked Flush 0 Done Done Sent false
20460 6300 Acked Write 2 Done Done Sent false
20461 6301 Acked Write 2 Done Done Sent false
20462 6302 Acked Read 4 Done Done Sent false
20463 6303 Acked Read 9 Done Done Sent false
20464 6304 Acked Flush 0 Done Done Sent false
20465 6305 Acked Read 2 Done Done Sent false
20466 6306 Acked Write 7 Done Done Sent false
20467 6307 Acked Write 1 Done Done Sent false
20468 6308 Acked Read 3 Done Done Sent false
20469 6309 Acked Read 3 Done Done Sent false
20470 6310 Acked Read 7 Done Done Sent false
20471 6311 Acked Write 5 Done Done Sent false
20472 6312 Acked Write 3 Done Done Sent false
20473 6313 Acked Write 1 Done Done Sent false
20474 6314 Acked Read 6 Done Done Sent false
20475 6315 Acked Flush 0 Done Done Sent false
20476 6316 Acked Write 4 Done Done Sent false
20477 6317 Acked Read 8 Done Done Sent false
20478 6318 Acked Read 5 Done Done Sent false
20479 6319 Acked Read 3 Done Done Sent false
20480 6320 Acked Read 5 Done Done Sent false
20481 6321 Acked Write 9 Done Done Sent false
20482 6322 Acked Write 8 Done Done Sent false
20483 6323 Acked Read 10 Done Done Sent false
20484 6324 Acked Read 4 Done Done Sent false
20485 6325 Acked Write 2 Done Done Sent false
20486 6326 Acked Write 2 Done Done Sent false
20487 6327 Acked Flush 0 Done Done Sent false
20488 6328 Acked Read 5 Done Done Sent false
20489 6329 Acked Read 6 Done Done Sent false
20490 6330 Acked Read 9 Done Done Sent false
20491 6331 Acked Read 2 Done Done Sent false
20492 6332 Acked Flush 0 Done Done Sent false
20493 6333 Acked Read 7 Done Done Sent false
20494 6334 Acked Write 7 Done Done Sent false
20495 6335 Acked Write 9 Done Done Sent false
20496 6336 Acked Read 3 Done Done Sent false
20497 6337 Acked Read 4 Done Done Sent false
20498 6338 Acked Write 10 Done Done Sent false
20499 6339 Acked Read 10 Done Done Sent false
20500 6340 Acked Read 1 Done Done Sent false
20501 6341 Acked Read 7 Done Done Sent false
20502 6342 Acked Write 3 Done Done Sent false
20503 6343 Acked Flush 0 Done Done Sent false
20504 6344 Acked Read 10 Done Done Sent false
20505 6345 Acked Read 3 Done Done Sent false
20506 6346 Acked Write 6 Done Done Sent false
20507 6347 Acked Read 4 Done Done Sent false
20508 6348 Acked Flush 0 Done Done Sent false
20509 6349 Acked Write 8 Done Done Sent false
20510 6350 Acked Read 10 Done Done Sent false
20511 6351 Acked Read 10 Done Done Sent false
20512 6352 Acked Read 9 Done Done Sent false
20513 6353 Acked Write 6 Done Done Sent false
20514 6354 Acked Read 2 Done Done Sent false
20515 6355 Acked Write 4 Done Done Sent false
20516 6356 Acked Read 5 Done Done Sent false
20517 6357 Acked Read 8 Done Done Sent false
20518 6358 Acked Write 2 Done Done Sent false
20519 6359 Acked Write 6 Done Done Sent false
20520 6360 Acked Read 8 Done Done Sent false
20521 6361 Acked Read 6 Done Done Sent false
20522 6362 Acked Write 2 Done Done Sent false
20523 6363 Acked Write 4 Done Done Sent false
20524 6364 Acked Read 6 Done Done Sent false
20525 6365 Acked Read 8 Done Done Sent false
20526 6366 Acked Read 5 Done Done Sent false
20527 6367 Acked Read 9 Done Done Sent false
20528 6368 Acked Write 9 Done Done Sent false
20529 6369 Acked Read 5 Done Done Sent false
20530 6370 Acked Read 9 Done Done Sent false
20531 6371 Acked Write 9 Done Done Sent false
20532 6372 Acked Write 7 Done Done Sent false
20533 6373 Acked Write 3 Done Done Sent false
20534 6374 Acked Read 9 Done Done Sent false
20535 6375 Acked Read 9 Done Done Sent false
20536 6376 Acked Read 9 Done Done Sent false
20537 6377 Acked Read 8 Done Done Sent false
20538 6378 Acked Write 2 Done Done Sent false
20539 6379 Acked Read 9 Done Done Sent false
20540 6380 Acked Write 7 Done Done Sent false
20541 6381 Acked Write 5 Done Done Sent false
20542 6382 Acked Write 7 Done Done Sent false
20543 6383 Acked Read 7 Done Done Sent false
20544 6384 Acked Write 3 Done Done Sent false
20545 6385 Acked Read 1 Done Done Sent false
20546 6386 Acked Write 8 Done Done Sent false
20547 6387 Acked Write 8 Done Done Sent false
20548 6388 Acked Read 8 Done Done Sent false
20549 6389 Acked Write 4 Done Done Sent false
20550 6390 Acked Read 4 Done Done Sent false
20551 6391 Acked Read 3 Done Done Sent false
20552 6392 Acked Read 3 Done Done Sent false
20553 6393 Acked Read 8 Done Done Sent false
20554 6394 Acked Write 9 Done Done Sent false
20555 6395 Acked Write 10 Done Done Sent false
20556 6396 Acked Write 9 Done Done Sent false
20557 6397 Acked Read 2 Done Done Sent false
20558 6398 Acked Write 10 Done Done Sent false
20559 6399 Acked Read 9 Done Done Sent false
20560 6400 Acked Read 9 Done Done Sent false
20561 6401 Acked Write 7 Done Done Sent false
20562 6402 Acked Read 9 Done Done Sent false
20563 6403 Acked Write 3 Done Done Sent false
20564 6404 Acked Write 1 Done Done Sent false
20565 6405 Acked Read 1 Done Done Sent false
20566 6406 Acked Read 8 Done Done Sent false
20567 6407 Acked Read 6 Done Done Sent false
20568 6408 Acked Read 10 Done Done Sent false
20569 6409 Acked Read 6 Done Done Sent false
20570 6410 Acked Read 4 Done Done Sent false
20571 6411 Acked Write 7 Done Done Sent false
20572 6412 Acked Write 6 Done Done Sent false
20573 6413 Acked Read 6 Done Done Sent false
20574 6414 Acked Flush 0 Done Done Sent false
20575 6415 Acked Flush 0 Done Done Sent false
20576 6416 Acked Write 3 Done Done Sent false
20577 6417 Acked Read 1 Done Done Sent false
20578 6418 Acked Read 4 Done Done Sent false
20579 6419 Acked Read 3 Done Done Sent false
20580 6420 Acked Write 4 Done Done Sent false
20581 6421 Acked Read 9 Done Done Sent false
20582 6422 Acked Write 7 Done Done Sent false
20583 6423 Acked Flush 0 Done Done Sent false
20584 6424 Acked Read 10 Done Done Sent false
20585 6425 Acked Write 1 Done Done Sent false
20586 6426 Acked Write 5 Done Done Sent false
20587 6427 Acked Read 7 Done Done Sent false
20588 6428 Acked Flush 0 Done Done Sent false
20589 6429 Acked Write 3 Done Done Sent false
20590 6430 Acked Read 8 Done Done Sent false
20591 6431 Acked Read 4 Done Done Sent false
20592 6432 Acked Read 5 Done Done Sent false
20593 STATES DS:0 DS:1 DS:2 TOTAL
20594 Sent 0 0 301 301
20595 Done 301 301 0 602
20596 Skipped 0 0 0 0
20597 Error 0 0 0 0
20598 Last Flush: 6428 6428 6131
20599 Downstairs last five completed: 6131 6130 6129 6128 6127
20600 Upstairs last five completed: 6432 6431 6430 6429 6428
20601 CLIENT: Up:0 ds:301 act:2
20602 [8830] got start action
20603 [8830] I am going to restart
20604 Starting downstairs at port 8830
20605 Make output file at "/tmp/dsc/downstairs-8830.txt"
20606 [8830][2] reports Starting
20607 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1220)
20608 [8830][2] reports Running
20609 ----------------------------------------------------------------
20610 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20611 States: Active Active Offline
20612 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20613 6132 Acked Read 10 Done Done Sent false
20614 6133 Acked Read 2 Done Done Sent false
20615 6134 Acked Write 10 Done Done Sent false
20616 6135 Acked Flush 0 Done Done Sent false
20617 6136 Acked Write 6 Done Done Sent false
20618 6137 Acked Read 6 Done Done Sent false
20619 6138 Acked Read 10 Done Done Sent false
20620 6139 Acked Flush 0 Done Done Sent false
20621 6140 Acked Write 9 Done Done Sent false
20622 6141 Acked Write 3 Done Done Sent false
20623 6142 Acked Flush 0 Done Done Sent false
20624 6143 Acked Read 5 Done Done Sent false
20625 6144 Acked Write 7 Done Done Sent false
20626 6145 Acked Read 8 Done Done Sent false
20627 6146 Acked Write 7 Done Done Sent false
20628 6147 Acked Read 9 Done Done Sent false
20629 6148 Acked Write 6 Done Done Sent false
20630 6149 Acked Read 9 Done Done Sent false
20631 6150 Acked Read 6 Done Done Sent false
20632 6151 Acked Read 6 Done Done Sent false
20633 6152 Acked Read 10 Done Done Sent false
20634 6153 Acked Write 5 Done Done Sent false
20635 6154 Acked Write 10 Done Done Sent false
20636 6155 Acked Read 2 Done Done Sent false
20637 6156 Acked Flush 0 Done Done Sent false
20638 6157 Acked Write 5 Done Done Sent false
20639 6158 Acked Read 8 Done Done Sent false
20640 6159 Acked Read 3 Done Done Sent false
20641 6160 Acked Read 8 Done Done Sent false
20642 6161 Acked Write 3 Done Done Sent false
20643 6162 Acked Flush 0 Done Done Sent false
20644 6163 Acked Write 1 Done Done Sent false
20645 6164 Acked Flush 0 Done Done Sent false
20646 6165 Acked Read 8 Done Done Sent false
20647 6166 Acked Read 1 Done Done Sent false
20648 6167 Acked Flush 0 Done Done Sent false
20649 6168 Acked Read 4 Done Done Sent false
20650 6169 Acked Write 4 Done Done Sent false
20651 6170 Acked Write 7 Done Done Sent false
20652 6171 Acked Read 1 Done Done Sent false
20653 6172 Acked Write 10 Done Done Sent false
20654 6173 Acked Read 3 Done Done Sent false
20655 6174 Acked Write 1 Done Done Sent false
20656 6175 Acked Read 7 Done Done Sent false
20657 6176 Acked Flush 0 Done Done Sent false
20658 6177 Acked Write 3 Done Done Sent false
20659 6178 Acked Write 2 Done Done Sent false
20660 6179 Acked Write 9 Done Done Sent false
20661 6180 Acked Flush 0 Done Done Sent false
20662 6181 Acked Read 10 Done Done Sent false
20663 6182 Acked Flush 0 Done Done Sent false
20664 6183 Acked Read 7 Done Done Sent false
20665 6184 Acked Write 2 Done Done Sent false
20666 6185 Acked Write 10 Done Done Sent false
20667 6186 Acked Read 6 Done Done Sent false
20668 6187 Acked Flush 0 Done Done Sent false
20669 6188 Acked Read 1 Done Done Sent false
20670 6189 Acked Read 6 Done Done Sent false
20671 6190 Acked Write 4 Done Done Sent false
20672 6191 Acked Read 9 Done Done Sent false
20673 6192 Acked Read 7 Done Done Sent false
20674 6193 Acked Write 7 Done Done Sent false
20675 6194 Acked Read 8 Done Done Sent false
20676 6195 Acked Write 6 Done Done Sent false
20677 6196 Acked Read 9 Done Done Sent false
20678 6197 Acked Flush 0 Done Done Sent false
20679 6198 Acked Read 1 Done Done Sent false
20680 6199 Acked Read 3 Done Done Sent false
20681 6200 Acked Write 1 Done Done Sent false
20682 6201 Acked Read 4 Done Done Sent false
20683 6202 Acked Write 8 Done Done Sent false
20684 6203 Acked Read 9 Done Done Sent false
20685 6204 Acked Read 9 Done Done Sent false
20686 6205 Acked Read 2 Done Done Sent false
20687 6206 Acked Write 5 Done Done Sent false
20688 6207 Acked Flush 0 Done Done Sent false
20689 6208 Acked Write 6 Done Done Sent false
20690 6209 Acked Read 2 Done Done Sent false
20691 6210 Acked Read 9 Done Done Sent false
20692 6211 Acked Read 9 Done Done Sent false
20693 6212 Acked Write 4 Done Done Sent false
20694 6213 Acked Write 4 Done Done Sent false
20695 6214 Acked Write 6 Done Done Sent false
20696 6215 Acked Write 3 Done Done Sent false
20697 6216 Acked Write 9 Done Done Sent false
20698 6217 Acked Flush 0 Done Done Sent false
20699 6218 Acked Read 3 Done Done Sent false
20700 6219 Acked Read 8 Done Done Sent false
20701 6220 Acked Write 3 Done Done Sent false
20702 6221 Acked Read 7 Done Done Sent false
20703 6222 Acked Read 4 Done Done Sent false
20704 6223 Acked Flush 0 Done Done Sent false
20705 6224 Acked Read 4 Done Done Sent false
20706 6225 Acked Write 3 Done Done Sent false
20707 6226 Acked Read 7 Done Done Sent false
20708 6227 Acked Read 2 Done Done Sent false
20709 6228 Acked Write 4 Done Done Sent false
20710 6229 Acked Read 3 Done Done Sent false
20711 6230 Acked Read 10 Done Done Sent false
20712 6231 Acked Read 4 Done Done Sent false
20713 6232 Acked Read 5 Done Done Sent false
20714 6233 Acked Write 9 Done Done Sent false
20715 6234 Acked Read 5 Done Done Sent false
20716 6235 Acked Write 4 Done Done Sent false
20717 6236 Acked Read 9 Done Done Sent false
20718 6237 Acked Read 10 Done Done Sent false
20719 6238 Acked Read 2 Done Done Sent false
20720 6239 Acked Flush 0 Done Done Sent false
20721 6240 Acked Read 7 Done Done Sent false
20722 6241 Acked Flush 0 Done Done Sent false
20723 6242 Acked Write 6 Done Done Sent false
20724 6243 Acked Write 1 Done Done Sent false
20725 6244 Acked Write 7 Done Done Sent false
20726 6245 Acked Write 4 Done Done Sent false
20727 6246 Acked Write 2 Done Done Sent false
20728 6247 Acked Write 2 Done Done Sent false
20729 6248 Acked Read 6 Done Done Sent false
20730 6249 Acked Flush 0 Done Done Sent false
20731 6250 Acked Read 10 Done Done Sent false
20732 6251 Acked Read 5 Done Done Sent false
20733 6252 Acked Read 4 Done Done Sent false
20734 6253 Acked Read 8 Done Done Sent false
20735 6254 Acked Write 2 Done Done Sent false
20736 6255 Acked Write 5 Done Done Sent false
20737 6256 Acked Write 7 Done Done Sent false
20738 6257 Acked Read 4 Done Done Sent false
20739 6258 Acked Flush 0 Done Done Sent false
20740 6259 Acked Write 6 Done Done Sent false
20741 6260 Acked Read 2 Done Done Sent false
20742 6261 Acked Write 4 Done Done Sent false
20743 6262 Acked Read 1 Done Done Sent false
20744 6263 Acked Write 5 Done Done Sent false
20745 6264 Acked Read 4 Done Done Sent false
20746 6265 Acked Read 2 Done Done Sent false
20747 6266 Acked Read 1 Done Done Sent false
20748 6267 Acked Read 4 Done Done Sent false
20749 6268 Acked Write 8 Done Done Sent false
20750 6269 Acked Write 6 Done Done Sent false
20751 6270 Acked Read 4 Done Done Sent false
20752 6271 Acked Write 5 Done Done Sent false
20753 6272 Acked Read 6 Done Done Sent false
20754 6273 Acked Write 4 Done Done Sent false
20755 6274 Acked Write 2 Done Done Sent false
20756 6275 Acked Write 1 Done Done Sent false
20757 6276 Acked Write 4 Done Done Sent false
20758 6277 Acked Write 3 Done Done Sent false
20759 6278 Acked Write 8 Done Done Sent false
20760 6279 Acked Flush 0 Done Done Sent false
20761 6280 Acked Read 2 Done Done Sent false
20762 6281 Acked Flush 0 Done Done Sent false
20763 6282 Acked Write 5 Done Done Sent false
20764 6283 Acked Flush 0 Done Done Sent false
20765 6284 Acked Read 2 Done Done Sent false
20766 6285 Acked Read 7 Done Done Sent false
20767 6286 Acked Write 5 Done Done Sent false
20768 6287 Acked Read 6 Done Done Sent false
20769 6288 Acked Read 10 Done Done Sent false
20770 6289 Acked Read 10 Done Done Sent false
20771 6290 Acked Read 5 Done Done Sent false
20772 6291 Acked Flush 0 Done Done Sent false
20773 6292 Acked Read 1 Done Done Sent false
20774 6293 Acked Read 5 Done Done Sent false
20775 6294 Acked Read 6 Done Done Sent false
20776 6295 Acked Write 10 Done Done Sent false
20777 6296 Acked Flush 0 Done Done Sent false
20778 6297 Acked Read 2 Done Done Sent false
20779 6298 Acked Write 3 Done Done Sent false
20780 6299 Acked Flush 0 Done Done Sent false
20781 6300 Acked Write 2 Done Done Sent false
20782 6301 Acked Write 2 Done Done Sent false
20783 6302 Acked Read 4 Done Done Sent false
20784 6303 Acked Read 9 Done Done Sent false
20785 6304 Acked Flush 0 Done Done Sent false
20786 6305 Acked Read 2 Done Done Sent false
20787 6306 Acked Write 7 Done Done Sent false
20788 6307 Acked Write 1 Done Done Sent false
20789 6308 Acked Read 3 Done Done Sent false
20790 6309 Acked Read 3 Done Done Sent false
20791 6310 Acked Read 7 Done Done Sent false
20792 6311 Acked Write 5 Done Done Sent false
20793 6312 Acked Write 3 Done Done Sent false
20794 6313 Acked Write 1 Done Done Sent false
20795 6314 Acked Read 6 Done Done Sent false
20796 6315 Acked Flush 0 Done Done Sent false
20797 6316 Acked Write 4 Done Done Sent false
20798 6317 Acked Read 8 Done Done Sent false
20799 6318 Acked Read 5 Done Done Sent false
20800 6319 Acked Read 3 Done Done Sent false
20801 6320 Acked Read 5 Done Done Sent false
20802 6321 Acked Write 9 Done Done Sent false
20803 6322 Acked Write 8 Done Done Sent false
20804 6323 Acked Read 10 Done Done Sent false
20805 6324 Acked Read 4 Done Done Sent false
20806 6325 Acked Write 2 Done Done Sent false
20807 6326 Acked Write 2 Done Done Sent false
20808 6327 Acked Flush 0 Done Done Sent false
20809 6328 Acked Read 5 Done Done Sent false
20810 6329 Acked Read 6 Done Done Sent false
20811 6330 Acked Read 9 Done Done Sent false
20812 6331 Acked Read 2 Done Done Sent false
20813 6332 Acked Flush 0 Done Done Sent false
20814 6333 Acked Read 7 Done Done Sent false
20815 6334 Acked Write 7 Done Done Sent false
20816 6335 Acked Write 9 Done Done Sent false
20817 6336 Acked Read 3 Done Done Sent false
20818 6337 Acked Read 4 Done Done Sent false
20819 6338 Acked Write 10 Done Done Sent false
20820 6339 Acked Read 10 Done Done Sent false
20821 6340 Acked Read 1 Done Done Sent false
20822 6341 Acked Read 7 Done Done Sent false
20823 6342 Acked Write 3 Done Done Sent false
20824 6343 Acked Flush 0 Done Done Sent false
20825 6344 Acked Read 10 Done Done Sent false
20826 6345 Acked Read 3 Done Done Sent false
20827 6346 Acked Write 6 Done Done Sent false
20828 6347 Acked Read 4 Done Done Sent false
20829 6348 Acked Flush 0 Done Done Sent false
20830 6349 Acked Write 8 Done Done Sent false
20831 6350 Acked Read 10 Done Done Sent false
20832 6351 Acked Read 10 Done Done Sent false
20833 6352 Acked Read 9 Done Done Sent false
20834 6353 Acked Write 6 Done Done Sent false
20835 6354 Acked Read 2 Done Done Sent false
20836 6355 Acked Write 4 Done Done Sent false
20837 6356 Acked Read 5 Done Done Sent false
20838 6357 Acked Read 8 Done Done Sent false
20839 6358 Acked Write 2 Done Done Sent false
20840 6359 Acked Write 6 Done Done Sent false
20841 6360 Acked Read 8 Done Done Sent false
20842 6361 Acked Read 6 Done Done Sent false
20843 6362 Acked Write 2 Done Done Sent false
20844 6363 Acked Write 4 Done Done Sent false
20845 6364 Acked Read 6 Done Done Sent false
20846 6365 Acked Read 8 Done Done Sent false
20847 6366 Acked Read 5 Done Done Sent false
20848 6367 Acked Read 9 Done Done Sent false
20849 6368 Acked Write 9 Done Done Sent false
20850 6369 Acked Read 5 Done Done Sent false
20851 6370 Acked Read 9 Done Done Sent false
20852 6371 Acked Write 9 Done Done Sent false
20853 6372 Acked Write 7 Done Done Sent false
20854 6373 Acked Write 3 Done Done Sent false
20855 6374 Acked Read 9 Done Done Sent false
20856 6375 Acked Read 9 Done Done Sent false
20857 6376 Acked Read 9 Done Done Sent false
20858 6377 Acked Read 8 Done Done Sent false
20859 6378 Acked Write 2 Done Done Sent false
20860 6379 Acked Read 9 Done Done Sent false
20861 6380 Acked Write 7 Done Done Sent false
20862 6381 Acked Write 5 Done Done Sent false
20863 6382 Acked Write 7 Done Done Sent false
20864 6383 Acked Read 7 Done Done Sent false
20865 6384 Acked Write 3 Done Done Sent false
20866 6385 Acked Read 1 Done Done Sent false
20867 6386 Acked Write 8 Done Done Sent false
20868 6387 Acked Write 8 Done Done Sent false
20869 6388 Acked Read 8 Done Done Sent false
20870 6389 Acked Write 4 Done Done Sent false
20871 6390 Acked Read 4 Done Done Sent false
20872 6391 Acked Read 3 Done Done Sent false
20873 6392 Acked Read 3 Done Done Sent false
20874 6393 Acked Read 8 Done Done Sent false
20875 6394 Acked Write 9 Done Done Sent false
20876 6395 Acked Write 10 Done Done Sent false
20877 6396 Acked Write 9 Done Done Sent false
20878 6397 Acked Read 2 Done Done Sent false
20879 6398 Acked Write 10 Done Done Sent false
20880 6399 Acked Read 9 Done Done Sent false
20881 6400 Acked Read 9 Done Done Sent false
20882 6401 Acked Write 7 Done Done Sent false
20883 6402 Acked Read 9 Done Done Sent false
20884 6403 Acked Write 3 Done Done Sent false
20885 6404 Acked Write 1 Done Done Sent false
20886 6405 Acked Read 1 Done Done Sent false
20887 6406 Acked Read 8 Done Done Sent false
20888 6407 Acked Read 6 Done Done Sent false
20889 6408 Acked Read 10 Done Done Sent false
20890 6409 Acked Read 6 Done Done Sent false
20891 6410 Acked Read 4 Done Done Sent false
20892 6411 Acked Write 7 Done Done Sent false
20893 6412 Acked Write 6 Done Done Sent false
20894 6413 Acked Read 6 Done Done Sent false
20895 6414 Acked Flush 0 Done Done Sent false
20896 6415 Acked Flush 0 Done Done Sent false
20897 6416 Acked Write 3 Done Done Sent false
20898 6417 Acked Read 1 Done Done Sent false
20899 6418 Acked Read 4 Done Done Sent false
20900 6419 Acked Read 3 Done Done Sent false
20901 6420 Acked Write 4 Done Done Sent false
20902 6421 Acked Read 9 Done Done Sent false
20903 6422 Acked Write 7 Done Done Sent false
20904 6423 Acked Flush 0 Done Done Sent false
20905 6424 Acked Read 10 Done Done Sent false
20906 6425 Acked Write 1 Done Done Sent false
20907 6426 Acked Write 5 Done Done Sent false
20908 6427 Acked Read 7 Done Done Sent false
20909 6428 Acked Flush 0 Done Done Sent false
20910 6429 Acked Write 3 Done Done Sent false
20911 6430 Acked Read 8 Done Done Sent false
20912 6431 Acked Read 4 Done Done Sent false
20913 6432 Acked Read 5 Done Done Sent false
20914 6433 Acked Flush 0 Done Done Sent false
20915 STATES DS:0 DS:1 DS:2 TOTAL
20916 Sent 0 0 302 302
20917 Done 302 302 0 604
20918 Skipped 0 0 0 0
20919 Error 0 0 0 0
20920 Last Flush: 6433 6433 6131
20921 Downstairs last five completed: 6131 6130 6129 6128 6127
20922 Upstairs last five completed: 6433 6432 6431 6430 6429
20923 CLIENT: Up:0 ds:302 act:2
209242024-11-19T17:12:21.911ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209252024-11-19T17:12:21.911ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35429) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209262024-11-19T17:12:21.912ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209272024-11-19T17:12:21.913ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209282024-11-19T17:12:21.913ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209292024-11-19T17:12:21.913ZINFOcrucible: send last flush ID to this DS: 6131 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209302024-11-19T17:12:21.913ZINFOcrucible: Replied this last flush ID: 6131 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209312024-11-19T17:12:21.913ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209322024-11-19T17:12:21.913ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209332024-11-19T17:12:21.913ZINFOcrucible: [2] client re-new 302 jobs since flush 6131 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
209342024-11-19T17:12:21.913ZINFOcrucible: [2] Replayed 302 jobs since flush: 6131 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
20935 ----------------------------------------------------------------
20936 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20937 Downstairs last five completed: 6433 6432 6431 6430 6429
20938 Upstairs last five completed: 6433 6432 6431 6430 6429
20939 CLIENT: Up:0 ds:0 act:3
20940 Replay: All jobs finished
20941 Main task has work to do, go find it
20942 got dsc Stop(0)
20943 stop 0
20944 Nov 19 17:12:24.159 INFO[8810] Got stop action so:false kr:false
20945 request completed, latency_us: 391, response_code: 204, uri: /stop/cid/0, method: POST, req_id: d352177d-29bf-4a94-a0c9-4f821ad813e9, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
209462024-11-19T17:12:24.160ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209472024-11-19T17:12:24.160ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209482024-11-19T17:12:24.160ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
209492024-11-19T17:12:24.160ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209502024-11-19T17:12:24.160ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
209512024-11-19T17:12:24.160ZINFOcrucible: client task is exiting = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
20952 Nov 19 17:12:24.160 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3188b08a-ca50-414a-b754-4ab48c8c1350, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
20953 [8810] Exited with: ExitStatus(unix_wait_status(9))
20954 [8810][0] reports Exit
20955 Nov 19 17:12:28.162 INFO request completed, latency_us: 183, response_code: 200, uri: /state/cid/0, method: GET, req_id: dc7fb231-6d63-4c54-a56f-00996aa011c5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
20956 001/300 Read block 1722 len 4096
20957 002/300 Write block 2189 len 32768 data: 4 3 4 4 3 3 2 3
20958 003/300 Write block 790 len 36864 data: 6 6 6 7 6 5 6 5 5
20959 004/300 Read block 742 len 12288
20960 005/300 Read block 1154 len 36864
20961 006/300 Write block 3219 len 28672 data: 4 4 4 5 7 5 6
20962 007/300 Write block 2387 len 40960 data: 3 4 5 6 8 9 8 7 5 5
20963 008/300 Write block 2587 len 8192 data: 4 4
20964 009/300 Write block 1065 len 28672 data: 4 6 4 5 4 4 4
20965 010/300 Read block 2970 len 28672
20966 011/300 Read block 2690 len 16384
20967 012/300 Flush
20968 013/300 Flush
20969 014/300 Read block 1589 len 24576
20970 015/300 Read block 60 len 36864
20971 016/300 Write block 3790 len 36864 data: 4 4 4 4 4 3 3 3 3
20972 017/300 Write block 1045 len 28672 data: 6 6 5 5 4 4 5
20973 018/300 Flush
20974 019/300 Read block 4173 len 24576
20975 020/300 Read block 617 len 8192
20976 021/300 Write block 1328 len 36864 data: 3 4 5 6 6 7 6 5 4
20977 022/300 Read block 2337 len 28672
20978 023/300 Read block 2847 len 40960
20979 024/300 Write block 120 len 32768 data: 3 3 3 3 4 4 4 5
20980 025/300 Write block 3326 len 8192 data: 5 5
20981 026/300 Write block 4294 len 28672 data: 4 4 4 5 4 3 3
20982 027/300 Read block 4956 len 20480
20983 028/300 Read block 3381 len 28672
20984 029/300 Read block 4847 len 4096
20985 030/300 Read block 70 len 16384
20986 031/300 Read block 1700 len 40960
20987 032/300 Write block 1088 len 12288 data: 3 4 5
20988 033/300 Read block 49 len 24576
20989 034/300 Flush
20990 035/300 Write block 4750 len 36864 data: 3 3 3 3 3 4 4 5 6
20991 036/300 Read block 224 len 4096
20992 037/300 Write block 2552 len 8192 data: 3 3
20993 038/300 Flush
20994 039/300 Read block 3904 len 40960
20995 040/300 Read block 900 len 32768
20996 041/300 Read block 2604 len 28672
20997 042/300 Read block 4287 len 8192
20998 043/300 Read block 2995 len 32768
20999 044/300 Write block 4591 len 20480 data: 7 7 5 5 5
21000 045/300 Flush
21001 046/300 Read block 4028 len 12288
21002 047/300 Read block 1852 len 24576
21003 048/300 Read block 1484 len 28672
21004 049/300 Read block 4596 len 24576
21005 050/300 Read block 689 len 24576
21006 051/300 Read block 3966 len 28672
21007 052/300 Write block 3664 len 32768 data: 5 4 3 4 5 5 3 4
21008 053/300 Read block 4369 len 4096
21009 054/300 Read block 2215 len 32768
21010 055/300 Write block 2806 len 40960 data: 4 6 5 5 4 4 4 5 5 4
21011 056/300 Read block 1225 len 40960
21012 057/300 Read block 2185 len 24576
21013 058/300 Write block 2139 len 20480 data: 4 4 5 6 6
21014 059/300 Read block 1734 len 40960
21015 060/300 Write block 4352 len 24576 data: 3 2 3 5 4 5
21016 061/300 Read block 2995 len 4096
21017 062/300 Write block 1873 len 32768 data: 5 4 6 8 8 10 10 9
21018 063/300 Read block 542 len 32768
21019 064/300 Flush
21020 065/300 Write block 1989 len 4096 data: 4
21021 066/300 Write block 2502 len 16384 data: 4 5 6 6
21022 067/300 Read block 3909 len 16384
21023 068/300 Write block 776 len 16384 data: 5 7 5 6
21024 069/300 Read block 1026 len 32768
21025 070/300 Read block 4832 len 40960
21026 071/300 Flush
21027 072/300 Write block 3879 len 28672 data: 5 5 6 5 5 5 4
21028 073/300 Write block 1819 len 36864 data: 5 4 4 6 7 7 6 7 7
21029 074/300 Write block 1112 len 20480 data: 4 4 4 4 4
21030 075/300 Write block 4876 len 12288 data: 2 2 2
21031 076/300 Read block 2089 len 40960
21032 077/300 Read block 2054 len 12288
21033 078/300 Read block 2129 len 32768
21034 079/300 Read block 602 len 40960
21035 080/300 Write block 2021 len 16384 data: 4 6 7 6
21036 081/300 Write block 2294 len 16384 data: 4 4 6 6
21037 082/300 Read block 1432 len 28672
21038 083/300 Write block 841 len 40960 data: 3 2 2 2 2 3 3 3 4 4
21039 084/300 Flush
21040 085/300 Write block 1160 len 36864 data: 6 5 6 6 6 6 6 5 6
21041 086/300 Write block 1924 len 24576 data: 2 2 3 5 5 7
21042 087/300 Read block 1007 len 28672
21043 088/300 Read block 3851 len 28672
21044 089/300 Write block 890 len 16384 data: 5 6 6 6
21045 090/300 Read block 2209 len 12288
21046 091/300 Read block 2322 len 8192
21047 092/300 Write block 3502 len 16384 data: 3 5 4 3
21048 093/300 Write block 4907 len 36864 data: 4 4 6 6 6 6 6 5 3
21049 094/300 Read block 94 len 24576
21050 095/300 Read block 1942 len 8192
21051 096/300 Flush
21052 097/300 Read block 1080 len 24576
21053 098/300 Read block 1105 len 4096
21054 099/300 Read block 3965 len 32768
21055 100/300 Write block 4175 len 8192 data: 2 2
21056 101/300 Write block 2191 len 16384 data: 5 5 4 4
21057 102/300 Write block 3547 len 20480 data: 5 5 3 4 4
21058 103/300 Write block 4469 len 8192 data: 3 3
21059 104/300 Write block 134 len 20480 data: 9 9 10 10 9
21060 105/300 Read block 4643 len 24576
21061 106/300 Read block 263 len 16384
21062 107/300 Write block 1332 len 12288 data: 7 8 7
21063 108/300 Read block 3063 len 4096
21064 109/300 Read block 525 len 36864
21065 110/300 Read block 3201 len 8192
21066 111/300 Read block 2260 len 40960
21067 112/300 Read block 2965 len 32768
21068 113/300 Read block 1363 len 24576
21069 114/300 Write block 3015 len 8192 data: 4 4
21070 115/300 Write block 1951 len 12288 data: 6 6 5
21071 116/300 Read block 3375 len 36864
21072 117/300 Write block 1377 len 20480 data: 3 4 3 4 4
21073 118/300 Read block 3343 len 36864
21074 119/300 Read block 3334 len 16384
21075 120/300 Flush
21076 121/300 Write block 1428 len 12288 data: 4 4 3
21077 122/300 Flush
21078 123/300 Read block 677 len 8192
21079 124/300 Read block 4745 len 32768
21080 125/300 Write block 3989 len 20480 data: 6 6 6 6 6
21081 126/300 Write block 4807 len 16384 data: 4 4 4 3
21082 127/300 Flush
21083 128/300 Read block 3008 len 32768
21084 129/300 Flush
21085 130/300 Read block 1056 len 8192
21086 131/300 Write block 52 len 8192 data: 4 4
21087 132/300 Read block 1942 len 40960
21088 133/300 Read block 4818 len 28672
21089 134/300 Read block 4747 len 12288
21090 135/300 Write block 4893 len 24576 data: 7 7 7 8 7 7
21091 136/300 Write block 3199 len 28672 data: 4 5 6 7 8 8 7
21092 137/300 Write block 1309 len 36864 data: 4 3 4 5 5 5 4 4 4
21093 138/300 Read block 4240 len 16384
21094 139/300 Read block 3 len 12288
21095 140/300 Read block 1867 len 16384
21096 141/300 Read block 4140 len 12288
21097 142/300 Write block 4528 len 36864 data: 6 7 7 7 5 5 5 5 5
21098 143/300 Write block 3890 len 28672 data: 4 4 5 6 5 5 6
21099 144/300 Read block 4145 len 32768
21100 145/300 Write block 1053 len 36864 data: 5 5 3 3 2 2 2 2 3
21101 146/300 Flush
21102 147/300 Read block 2412 len 28672
21103 148/300 Read block 2328 len 24576
21104 149/300 Read block 4459 len 28672
21105 150/300 Write block 7 len 36864 data: 4 5 5 5 4 4 2 2 2
21106 151/300 Flush
21107 152/300 Read block 1624 len 20480
21108 153/300 Write block 4885 len 36864 data: 3 3 3 3 4 5 5 7 8
21109 154/300 Write block 2137 len 4096 data: 4
21110 155/300 Read block 2330 len 24576
21111 156/300 Read block 3315 len 40960
21112 157/300 Read block 1575 len 20480
21113 158/300 Write block 4906 len 20480 data: 3 5 5 7 7
21114 159/300 Flush
21115 160/300 Write block 1421 len 28672 data: 5 5 4 4 3 3 3
21116 161/300 Write block 1122 len 8192 data: 4 4
21117 162/300 Read block 2331 len 4096
21118 163/300 Read block 1976 len 16384
21119 164/300 Read block 3395 len 24576
21120 165/300 Write block 3033 len 28672 data: 8 6 5 5 5 3 3
21121 166/300 Write block 1404 len 32768 data: 6 7 9 7 5 5 5 6
21122 167/300 Read block 2417 len 16384
21123 168/300 Read block 4494 len 12288
21124 169/300 Read block 3949 len 36864
21125 170/300 Read block 4400 len 28672
21126 171/300 Write block 1943 len 36864 data: 6 6 5 5 4 5 4 5 7
21127 172/300 Read block 2620 len 28672
21128 173/300 Read block 657 len 32768
21129 174/300 Read block 4211 len 16384
21130 175/300 Read block 734 len 32768
21131 176/300 Write block 524 len 20480 data: 4 3 4 5 5
21132 177/300 Read block 3799 len 28672
21133 178/300 Write block 3304 len 8192 data: 4 5
21134 179/300 Write block 4061 len 12288 data: 3 3 3
21135 180/300 Flush
21136 181/300 Read block 1475 len 24576
21137 182/300 Read block 4476 len 20480
21138 183/300 Write block 3740 len 20480 data: 12 10 10 10 9
21139 184/300 Write block 4088 len 32768 data: 4 6 5 4 4 4 4 4
21140 185/300 Read block 3928 len 40960
21141 186/300 Read block 4402 len 8192
21142 187/300 Write block 1602 len 12288 data: 5 5 6
21143 188/300 Write block 2485 len 4096 data: 5
21144 189/300 Read block 3610 len 12288
21145 190/300 Write block 4224 len 4096 data: 5
21146 191/300 Read block 2504 len 36864
21147 192/300 Flush
21148 193/300 Write block 3061 len 16384 data: 4 3 4 3
21149 194/300 Write block 1437 len 28672 data: 2 2 2 3 4 3 3
21150 195/300 Read block 3570 len 36864
21151 196/300 Read block 3830 len 28672
21152 197/300 Read block 1034 len 4096
21153 198/300 Write block 1707 len 36864 data: 4 5 4 3 3 6 6 4 4
21154 199/300 Read block 2210 len 12288
21155 200/300 Flush
21156 201/300 Write block 2974 len 24576 data: 5 5 5 5 4 4
21157 202/300 Flush
21158 203/300 Write block 4404 len 36864 data: 4 3 2 3 3 3 4 4 5
21159 204/300 Read block 4212 len 12288
21160 205/300 Read block 3153 len 36864
21161 206/300 Flush
21162 207/300 Write block 3222 len 8192 data: 6 8
21163 208/300 Read block 3764 len 12288
21164 209/300 Read block 4578 len 24576
21165 210/300 Read block 2871 len 16384
21166 211/300 Read block 3635 len 20480
21167 212/300 Read block 310 len 40960
21168 213/300 Read block 1127 len 12288
21169 214/300 Read block 4631 len 4096
21170 215/300 Read block 2440 len 4096
21171 216/300 Read block 639 len 4096
21172 217/300 Write block 378 len 32768 data: 4 4 4 4 3 3 4 5
21173 218/300 Read block 1911 len 24576
21174 219/300 Flush
21175 220/300 Read block 2187 len 8192
21176 221/300 Write block 69 len 16384 data: 4 3 2 2
21177 222/300 Write block 1963 len 36864 data: 2 2 3 3 4 4 4 4 4
21178 223/300 Write block 2518 len 32768 data: 3 3 2 2 3 3 4 4
21179 224/300 Read block 1460 len 28672
21180 225/300 Write block 1778 len 4096 data: 7
21181 226/300 Write block 2923 len 8192 data: 4 3
21182 227/300 Read block 4169 len 16384
21183 228/300 Read block 4146 len 28672
21184 229/300 Read block 4060 len 16384
21185 230/300 Read block 2644 len 24576
21186 231/300 Write block 3728 len 36864 data: 7 9 8 7 7 6 6 4 6
21187 232/300 Write block 3334 len 24576 data: 4 5 6 6 6 6
21188 233/300 Write block 1582 len 40960 data: 6 5 6 7 6 5 4 3 2 2
21189 234/300 Read block 3910 len 36864
21190 235/300 Read block 1351 len 12288
21191 236/300 Write block 4420 len 24576 data: 4 5 5 5 6 6
21192 237/300 Read block 4576 len 28672
21193 238/300 Read block 4883 len 16384
21194 239/300 Write block 2285 len 16384 data: 4 7 8 8
21195 240/300 Write block 1085 len 32768 data: 2 2 3 4 5 6 6 5
21196 241/300 Flush
21197 242/300 Write block 1812 len 28672 data: 4 3 3 2 2 4 5
21198 243/300 Write block 2588 len 36864 data: 5 3 3 4 4 3 3 2 3
21199 244/300 Read block 951 len 40960
21200 245/300 Read block 3556 len 28672
21201 246/300 Write block 3880 len 16384 data: 6 7 6 6
21202 247/300 Write block 1227 len 16384 data: 5 6 5 5
21203 248/300 Write block 2264 len 24576 data: 5 5 3 3 3 3
21204 249/300 Read block 3380 len 12288
21205 250/300 Read block 2254 len 36864
21206 251/300 Read block 4057 len 8192
21207 252/300 Write block 1649 len 12288 data: 4 4 4
21208 253/300 Write block 2313 len 4096 data: 4
21209 254/300 Read block 3897 len 24576
21210 255/300 Read block 934 len 36864
21211 256/300 Write block 667 len 12288 data: 9 8 9
21212 257/300 Read block 774 len 12288
21213 258/300 Read block 587 len 40960
21214 259/300 Read block 3059 len 12288
21215 260/300 Read block 787 len 12288
21216 261/300 Write block 835 len 28672 data: 4 4 4 5 4 3 4
21217 262/300 Write block 2640 len 12288 data: 6 5 5
21218 263/300 Write block 880 len 32768 data: 4 4 4 3 4 5 5 5
21219 264/300 Write block 4138 len 4096 data: 5
21220 265/300 Read block 1690 len 36864
21221 266/300 Read block 3725 len 20480
21222 267/300 Read block 2383 len 36864
21223 268/300 Read block 677 len 12288
21224 269/300 Write block 749 len 36864 data: 4 5 5 4 4 6 4 3 3
21225 270/300 Read block 2605 len 32768
21226 271/300 Write block 642 len 32768 data: 3 2 4 4 5 5 7 6
21227 272/300 Write block 3845 len 12288 data: 4 4 4
21228 273/300 Read block 2400 len 40960
21229 274/300 Flush
21230 275/300 Read block 2730 len 28672
21231 276/300 Write block 2709 len 20480 data: 7 5 5 5 5
21232 277/300 Read block 4115 len 16384
21233 278/300 Read block 809 len 40960
21234 279/300 Read block 3398 len 28672
21235 280/300 Read block 34 len 20480
21236 281/300 Read block 4345 len 28672
21237 282/300 Read block 503 len 32768
21238 283/300 Write block 4545 len 24576 data: 4 5 4 4 4 5
21239 284/300 Read block 920 len 8192
21240 285/300 Read block 3453 len 28672
21241 286/300 Flush
21242 287/300 Write block 1528 len 28672 data: 5 5 4 4 4 5 4
21243 288/300 Write block 2574 len 32768 data: 6 4 4 4 4 2 2 3
21244 289/300 Write block 4016 len 40960 data: 5 5 5 5 6 7 7 7 7 6
21245 290/300 Write block 446 len 32768 data: 4 3 4 4 4 4 3 2
21246 291/300 Write block 1805 len 40960 data: 4 5 5 5 4 3 4 5 4 4
21247 292/300 Read block 332 len 36864
21248 293/300 Read block 18 len 16384
21249 294/300 Write block 3144 len 8192 data: 3 5
21250 295/300 Write block 3481 len 8192 data: 6 9
21251 296/300 Write block 3637 len 12288 data: 5 5 3
21252 297/300 Read block 722 len 28672
21253 298/300 Write block 783 len 12288 data: 7 8 7
21254 299/300 Read block 3254 len 4096
21255 300/300 Write block 1574 len 20480 data: 3 4 2 2 2
21256 301/300 Read block 3563 len 4096
21257 Main task has work to do, go find it
21258 got dsc Start(0)
21259 start 0
21260 Nov 19 17:12:28.410 INFO request completed, latency_us: 188, response_code: 204, uri: /start/cid/0, method: POST, req_id: 67cccce0-3986-4b56-8eb5-753ed0769d54, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
21261 Replay: started 0, returned:Ok(())
21262 ----------------------------------------------------------------
21263 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21264 States: Offline Active Active
21265 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21266 6434 Acked Read 1 Sent Done Done false
21267 6435 Acked Write 8 Sent Done Done false
21268 6436 Acked Write 9 Sent Done Done false
21269 6437 Acked Read 3 Sent Done Done false
21270 6438 Acked Read 9 Sent Done Done false
21271 6439 Acked Write 7 Sent Done Done false
21272 6440 Acked Write 10 Sent Done Done false
21273 6441 Acked Write 2 Sent Done Done false
21274 6442 Acked Write 7 Sent Done Done false
21275 6443 Acked Read 7 Sent Done Done false
21276 6444 Acked Read 4 Sent Done Done false
21277 6445 Acked Flush 0 Sent Done Done false
21278 6446 Acked Flush 0 Sent Done Done false
21279 6447 Acked Read 6 Sent Done Done false
21280 6448 Acked Read 9 Sent Done Done false
21281 6449 Acked Write 9 Sent Done Done false
21282 6450 Acked Write 7 Sent Done Done false
21283 6451 Acked Flush 0 Sent Done Done false
21284 6452 Acked Read 6 Sent Done Done false
21285 6453 Acked Read 2 Sent Done Done false
21286 6454 Acked Write 9 Sent Done Done false
21287 6455 Acked Read 7 Sent Done Done false
21288 6456 Acked Read 10 Sent Done Done false
21289 6457 Acked Write 8 Sent Done Done false
21290 6458 Acked Write 2 Sent Done Done false
21291 6459 Acked Write 7 Sent Done Done false
21292 6460 Acked Read 5 Sent Done Done false
21293 6461 Acked Read 7 Sent Done Done false
21294 6462 Acked Read 1 Sent Done Done false
21295 6463 Acked Read 4 Sent Done Done false
21296 6464 Acked Read 10 Sent Done Done false
21297 6465 Acked Write 3 Sent Done Done false
21298 6466 Acked Read 6 Sent Done Done false
21299 6467 Acked Flush 0 Sent Done Done false
21300 6468 Acked Write 9 Sent Done Done false
21301 6469 Acked Read 1 Sent Done Done false
21302 6470 Acked Write 2 Sent Done Done false
21303 6471 Acked Flush 0 Sent Done Done false
21304 6472 Acked Read 10 Sent Done Done false
21305 6473 Acked Read 8 Sent Done Done false
21306 6474 Acked Read 7 Sent Done Done false
21307 6475 Acked Read 2 Sent Done Done false
21308 6476 Acked Read 8 Sent Done Done false
21309 6477 Acked Write 5 Sent Done Done false
21310 6478 Acked Flush 0 Sent Done Done false
21311 6479 Acked Read 3 Sent Done Done false
21312 6480 Acked Read 6 Sent Done Done false
21313 6481 Acked Read 7 Sent Done Done false
21314 6482 Acked Read 6 Sent Done Done false
21315 6483 Acked Read 6 Sent Done Done false
21316 6484 Acked Read 7 Sent Done Done false
21317 6485 Acked Write 8 Sent Done Done false
21318 6486 Acked Read 1 Sent Done Done false
21319 6487 Acked Read 8 Sent Done Done false
21320 6488 Acked Write 10 Sent Done Done false
21321 6489 Acked Read 10 Sent Done Done false
21322 6490 Acked Read 6 Sent Done Done false
21323 6491 Acked Write 5 Sent Done Done false
21324 6492 Acked Read 10 Sent Done Done false
21325 6493 Acked Write 6 Sent Done Done false
21326 6494 Acked Read 1 Sent Done Done false
21327 6495 Acked Write 8 Sent Done Done false
21328 6496 Acked Read 8 Sent Done Done false
21329 6497 Acked Flush 0 Sent Done Done false
21330 6498 Acked Write 1 Sent Done Done false
21331 6499 Acked Write 4 Sent Done Done false
21332 6500 Acked Read 4 Sent Done Done false
21333 6501 Acked Write 4 Sent Done Done false
21334 6502 Acked Read 8 Sent Done Done false
21335 6503 Acked Read 10 Sent Done Done false
21336 6504 Acked Flush 0 Sent Done Done false
21337 6505 Acked Write 7 Sent Done Done false
21338 6506 Acked Write 9 Sent Done Done false
21339 6507 Acked Write 5 Sent Done Done false
21340 6508 Acked Write 3 Sent Done Done false
21341 6509 Acked Read 10 Sent Done Done false
21342 6510 Acked Read 3 Sent Done Done false
21343 6511 Acked Read 8 Sent Done Done false
21344 6512 Acked Read 10 Sent Done Done false
21345 6513 Acked Write 4 Sent Done Done false
21346 6514 Acked Write 4 Sent Done Done false
21347 6515 Acked Read 7 Sent Done Done false
21348 6516 Acked Write 10 Sent Done Done false
21349 6517 Acked Flush 0 Sent Done Done false
21350 6518 Acked Write 9 Sent Done Done false
21351 6519 Acked Write 6 Sent Done Done false
21352 6520 Acked Read 7 Sent Done Done false
21353 6521 Acked Read 7 Sent Done Done false
21354 6522 Acked Write 4 Sent Done Done false
21355 6523 Acked Read 3 Sent Done Done false
21356 6524 Acked Read 2 Sent Done Done false
21357 6525 Acked Write 4 Sent Done Done false
21358 6526 Acked Write 9 Sent Done Done false
21359 6527 Acked Read 6 Sent Done Done false
21360 6528 Acked Read 2 Sent Done Done false
21361 6529 Acked Flush 0 Sent Done Done false
21362 6530 Acked Read 6 Sent Done Done false
21363 6531 Acked Read 1 Sent Done Done false
21364 6532 Acked Read 8 Sent Done Done false
21365 6533 Acked Write 2 Sent Done Done false
21366 6534 Acked Write 4 Sent Done Done false
21367 6535 Acked Write 5 Sent Done Done false
21368 6536 Acked Write 2 Sent Done Done false
21369 6537 Acked Write 5 Sent Done Done false
21370 6538 Acked Read 6 Sent Done Done false
21371 6539 Acked Read 4 Sent Done Done false
21372 6540 Acked Write 3 Sent Done Done false
21373 6541 Acked Read 1 Sent Done Done false
21374 6542 Acked Read 9 Sent Done Done false
21375 6543 Acked Read 2 Sent Done Done false
21376 6544 Acked Read 10 Sent Done Done false
21377 6545 Acked Read 8 Sent Done Done false
21378 6546 Acked Read 6 Sent Done Done false
21379 6547 Acked Write 2 Sent Done Done false
21380 6548 Acked Write 3 Sent Done Done false
21381 6549 Acked Read 9 Sent Done Done false
21382 6550 Acked Write 5 Sent Done Done false
21383 6551 Acked Read 9 Sent Done Done false
21384 6552 Acked Read 4 Sent Done Done false
21385 6553 Acked Flush 0 Sent Done Done false
21386 6554 Acked Write 3 Sent Done Done false
21387 6555 Acked Flush 0 Sent Done Done false
21388 6556 Acked Read 2 Sent Done Done false
21389 6557 Acked Read 8 Sent Done Done false
21390 6558 Acked Write 5 Sent Done Done false
21391 6559 Acked Write 4 Sent Done Done false
21392 6560 Acked Flush 0 Sent Done Done false
21393 6561 Acked Read 8 Sent Done Done false
21394 6562 Acked Flush 0 Sent Done Done false
21395 6563 Acked Read 2 Sent Done Done false
21396 6564 Acked Write 2 Sent Done Done false
21397 6565 Acked Read 10 Sent Done Done false
21398 6566 Acked Read 7 Sent Done Done false
21399 6567 Acked Read 3 Sent Done Done false
21400 6568 Acked Write 6 Sent Done Done false
21401 6569 Acked Write 7 Sent Done Done false
21402 6570 Acked Write 9 Sent Done Done false
21403 6571 Acked Read 4 Sent Done Done false
21404 6572 Acked Read 3 Sent Done Done false
21405 6573 Acked Read 4 Sent Done Done false
21406 6574 Acked Read 3 Sent Done Done false
21407 6575 Acked Write 9 Sent Done Done false
21408 6576 Acked Write 7 Sent Done Done false
21409 6577 Acked Read 8 Sent Done Done false
21410 6578 Acked Write 9 Sent Done Done false
21411 6579 Acked Flush 0 Sent Done Done false
21412 6580 Acked Read 7 Sent Done Done false
21413 6581 Acked Read 6 Sent Done Done false
21414 6582 Acked Read 7 Sent Done Done false
21415 6583 Acked Write 9 Sent Done Done false
21416 6584 Acked Flush 0 Sent Done Done false
21417 6585 Acked Read 5 Sent Done Done false
21418 6586 Acked Write 9 Sent Done Done false
21419 6587 Acked Write 1 Sent Done Done false
21420 6588 Acked Read 6 Sent Done Done false
21421 6589 Acked Read 10 Sent Done Done false
21422 6590 Acked Read 5 Sent Done Done false
21423 6591 Acked Write 5 Sent Done Done false
21424 6592 Acked Flush 0 Sent Done Done false
21425 6593 Acked Write 7 Sent Done Done false
21426 6594 Acked Write 2 Sent Done Done false
21427 6595 Acked Read 1 Sent Done Done false
21428 6596 Acked Read 4 Sent Done Done false
21429 6597 Acked Read 6 Sent Done Done false
21430 6598 Acked Write 7 Sent Done Done false
21431 6599 Acked Write 8 Sent Done Done false
21432 6600 Acked Read 4 Sent Done Done false
21433 6601 Acked Read 3 Sent Done Done false
21434 6602 Acked Read 9 Sent Done Done false
21435 6603 Acked Read 7 Sent Done Done false
21436 6604 Acked Write 9 Sent Done Done false
21437 6605 Acked Read 7 Sent Done Done false
21438 6606 Acked Read 8 Sent Done Done false
21439 6607 Acked Read 4 Sent Done Done false
21440 6608 Acked Read 8 Sent Done Done false
21441 6609 Acked Write 5 Sent Done Done false
21442 6610 Acked Read 7 Sent Done Done false
21443 6611 Acked Write 2 Sent Done Done false
21444 6612 Acked Write 3 Sent Done Done false
21445 6613 Acked Flush 0 Sent Done Done false
21446 6614 Acked Read 6 Sent Done Done false
21447 6615 Acked Read 5 Sent Done Done false
21448 6616 Acked Write 5 Sent Done Done false
21449 6617 Acked Write 8 Sent Done Done false
21450 6618 Acked Read 10 Sent Done Done false
21451 6619 Acked Read 2 Sent Done Done false
21452 6620 Acked Write 3 Sent Done Done false
21453 6621 Acked Write 1 Sent Done Done false
21454 6622 Acked Read 3 Sent Done Done false
21455 6623 Acked Write 1 Sent Done Done false
21456 6624 Acked Read 9 Sent Done Done false
21457 6625 Acked Flush 0 Sent Done Done false
21458 6626 Acked Write 4 Sent Done Done false
21459 6627 Acked Write 7 Sent Done Done false
21460 6628 Acked Read 9 Sent Done Done false
21461 6629 Acked Read 7 Sent Done Done false
21462 6630 Acked Read 1 Sent Done Done false
21463 6631 Acked Write 9 Sent Done Done false
21464 6632 Acked Read 3 Sent Done Done false
21465 6633 Acked Flush 0 Sent Done Done false
21466 6634 Acked Write 6 Sent Done Done false
21467 6635 Acked Flush 0 Sent Done Done false
21468 6636 Acked Write 9 Sent Done Done false
21469 6637 Acked Read 3 Sent Done Done false
21470 6638 Acked Read 9 Sent Done Done false
21471 6639 Acked Flush 0 Sent Done Done false
21472 6640 Acked Write 2 Sent Done Done false
21473 6641 Acked Read 3 Sent Done Done false
21474 6642 Acked Read 6 Sent Done Done false
21475 6643 Acked Read 4 Sent Done Done false
21476 6644 Acked Read 5 Sent Done Done false
21477 6645 Acked Read 10 Sent Done Done false
21478 6646 Acked Read 3 Sent Done Done false
21479 6647 Acked Read 1 Sent Done Done false
21480 6648 Acked Read 1 Sent Done Done false
21481 6649 Acked Read 1 Sent Done Done false
21482 6650 Acked Write 8 Sent Done Done false
21483 6651 Acked Read 6 Sent Done Done false
21484 6652 Acked Flush 0 Sent Done Done false
21485 6653 Acked Read 2 Sent Done Done false
21486 6654 Acked Write 4 Sent Done Done false
21487 6655 Acked Write 9 Sent Done Done false
21488 6656 Acked Write 8 Sent Done Done false
21489 6657 Acked Read 7 Sent Done Done false
21490 6658 Acked Write 1 Sent Done Done false
21491 6659 Acked Write 2 Sent Done Done false
21492 6660 Acked Read 4 Sent Done Done false
21493 6661 Acked Read 7 Sent Done Done false
21494 6662 Acked Read 4 Sent Done Done false
21495 6663 Acked Read 6 Sent Done Done false
21496 6664 Acked Write 9 Sent Done Done false
21497 6665 Acked Write 6 Sent Done Done false
21498 6666 Acked Write 10 Sent Done Done false
21499 6667 Acked Read 9 Sent Done Done false
21500 6668 Acked Read 3 Sent Done Done false
21501 6669 Acked Write 6 Sent Done Done false
21502 6670 Acked Read 7 Sent Done Done false
21503 6671 Acked Read 4 Sent Done Done false
21504 6672 Acked Write 4 Sent Done Done false
21505 6673 Acked Write 8 Sent Done Done false
21506 6674 Acked Flush 0 Sent Done Done false
21507 6675 Acked Write 7 Sent Done Done false
21508 6676 Acked Write 9 Sent Done Done false
21509 6677 Acked Read 10 Sent Done Done false
21510 6678 Acked Read 7 Sent Done Done false
21511 6679 Acked Write 4 Sent Done Done false
21512 6680 Acked Write 4 Sent Done Done false
21513 6681 Acked Write 6 Sent Done Done false
21514 6682 Acked Read 3 Sent Done Done false
21515 6683 Acked Read 9 Sent Done Done false
21516 6684 Acked Read 2 Sent Done Done false
21517 6685 Acked Write 3 Sent Done Done false
21518 6686 Acked Write 1 Sent Done Done false
21519 6687 Acked Read 6 Sent Done Done false
21520 6688 Acked Read 9 Sent Done Done false
21521 6689 Acked Write 3 Sent Done Done false
21522 6690 Acked Read 3 Sent Done Done false
21523 6691 Acked Read 10 Sent Done Done false
21524 6692 Acked Read 3 Sent Done Done false
21525 6693 Acked Read 3 Sent Done Done false
21526 6694 Acked Write 7 Sent Done Done false
21527 6695 Acked Write 3 Sent Done Done false
21528 6696 Acked Write 8 Sent Done Done false
21529 6697 Acked Write 1 Sent Done Done false
21530 6698 Acked Read 9 Sent Done Done false
21531 6699 Acked Read 5 Sent Done Done false
21532 6700 Acked Read 9 Sent Done Done false
21533 6701 Acked Read 3 Sent Done Done false
21534 6702 Acked Write 9 Sent Done Done false
21535 6703 Acked Read 8 Sent Done Done false
21536 6704 Acked Write 8 Sent Done Done false
21537 6705 Acked Write 3 Sent Done Done false
21538 6706 Acked Read 10 Sent Done Done false
21539 6707 Acked Flush 0 Sent Done Done false
21540 6708 Acked Read 7 Sent Done Done false
21541 6709 Acked Write 5 Sent Done Done false
21542 6710 Acked Read 4 Sent Done Done false
21543 6711 Acked Read 10 Sent Done Done false
21544 6712 Acked Read 7 Sent Done Done false
21545 6713 Acked Read 5 Sent Done Done false
21546 6714 Acked Read 7 Sent Done Done false
21547 6715 Acked Read 8 Sent Done Done false
21548 6716 Acked Write 6 Sent Done Done false
21549 6717 Acked Read 2 Sent Done Done false
21550 6718 Acked Read 7 Sent Done Done false
21551 6719 Acked Flush 0 Sent Done Done false
21552 6720 Acked Write 7 Sent Done Done false
21553 6721 Acked Write 8 Sent Done Done false
21554 6722 Acked Write 10 Sent Done Done false
21555 6723 Acked Write 8 Sent Done Done false
21556 6724 Acked Write 10 Sent Done Done false
21557 6725 Acked Read 9 Sent Done Done false
21558 6726 Acked Read 4 Sent Done Done false
21559 6727 Acked Write 2 Sent Done Done false
21560 6728 Acked Write 2 Sent Done Done false
21561 6729 Acked Write 3 Sent Done Done false
21562 6730 Acked Read 7 Sent Done Done false
21563 6731 Acked Write 3 Sent Done Done false
21564 6732 Acked Read 1 Sent Done Done false
21565 6733 Acked Write 5 Sent Done Done false
21566 6734 Acked Read 1 Sent Done Done false
21567 STATES DS:0 DS:1 DS:2 TOTAL
21568 Sent 301 0 0 301
21569 Done 0 301 301 602
21570 Skipped 0 0 0 0
21571 Error 0 0 0 0
21572 Last Flush: 6433 6719 6719
21573 Downstairs last five completed: 6433 6432 6431 6430 6429
21574 Upstairs last five completed: 6734 6733 6732 6731 6730
21575 CLIENT: Up:0 ds:301 act:2
21576 [8810] got start action
21577 [8810] I am going to restart
21578 Starting downstairs at port 8810
21579 Make output file at "/tmp/dsc/downstairs-8810.txt"
21580 [8810][0] reports Starting
21581 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1221)
21582 [8810][0] reports Running
21583 ----------------------------------------------------------------
21584 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21585 States: Offline Active Active
21586 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21587 6434 Acked Read 1 Sent Done Done false
21588 6435 Acked Write 8 Sent Done Done false
21589 6436 Acked Write 9 Sent Done Done false
21590 6437 Acked Read 3 Sent Done Done false
21591 6438 Acked Read 9 Sent Done Done false
21592 6439 Acked Write 7 Sent Done Done false
21593 6440 Acked Write 10 Sent Done Done false
21594 6441 Acked Write 2 Sent Done Done false
21595 6442 Acked Write 7 Sent Done Done false
21596 6443 Acked Read 7 Sent Done Done false
21597 6444 Acked Read 4 Sent Done Done false
21598 6445 Acked Flush 0 Sent Done Done false
21599 6446 Acked Flush 0 Sent Done Done false
21600 6447 Acked Read 6 Sent Done Done false
21601 6448 Acked Read 9 Sent Done Done false
21602 6449 Acked Write 9 Sent Done Done false
21603 6450 Acked Write 7 Sent Done Done false
21604 6451 Acked Flush 0 Sent Done Done false
21605 6452 Acked Read 6 Sent Done Done false
21606 6453 Acked Read 2 Sent Done Done false
21607 6454 Acked Write 9 Sent Done Done false
21608 6455 Acked Read 7 Sent Done Done false
21609 6456 Acked Read 10 Sent Done Done false
21610 6457 Acked Write 8 Sent Done Done false
21611 6458 Acked Write 2 Sent Done Done false
21612 6459 Acked Write 7 Sent Done Done false
21613 6460 Acked Read 5 Sent Done Done false
21614 6461 Acked Read 7 Sent Done Done false
21615 6462 Acked Read 1 Sent Done Done false
21616 6463 Acked Read 4 Sent Done Done false
21617 6464 Acked Read 10 Sent Done Done false
21618 6465 Acked Write 3 Sent Done Done false
21619 6466 Acked Read 6 Sent Done Done false
21620 6467 Acked Flush 0 Sent Done Done false
21621 6468 Acked Write 9 Sent Done Done false
21622 6469 Acked Read 1 Sent Done Done false
21623 6470 Acked Write 2 Sent Done Done false
21624 6471 Acked Flush 0 Sent Done Done false
21625 6472 Acked Read 10 Sent Done Done false
21626 6473 Acked Read 8 Sent Done Done false
21627 6474 Acked Read 7 Sent Done Done false
21628 6475 Acked Read 2 Sent Done Done false
21629 6476 Acked Read 8 Sent Done Done false
21630 6477 Acked Write 5 Sent Done Done false
21631 6478 Acked Flush 0 Sent Done Done false
21632 6479 Acked Read 3 Sent Done Done false
21633 6480 Acked Read 6 Sent Done Done false
21634 6481 Acked Read 7 Sent Done Done false
21635 6482 Acked Read 6 Sent Done Done false
21636 6483 Acked Read 6 Sent Done Done false
21637 6484 Acked Read 7 Sent Done Done false
21638 6485 Acked Write 8 Sent Done Done false
21639 6486 Acked Read 1 Sent Done Done false
21640 6487 Acked Read 8 Sent Done Done false
21641 6488 Acked Write 10 Sent Done Done false
21642 6489 Acked Read 10 Sent Done Done false
21643 6490 Acked Read 6 Sent Done Done false
21644 6491 Acked Write 5 Sent Done Done false
21645 6492 Acked Read 10 Sent Done Done false
21646 6493 Acked Write 6 Sent Done Done false
21647 6494 Acked Read 1 Sent Done Done false
21648 6495 Acked Write 8 Sent Done Done false
21649 6496 Acked Read 8 Sent Done Done false
21650 6497 Acked Flush 0 Sent Done Done false
21651 6498 Acked Write 1 Sent Done Done false
21652 6499 Acked Write 4 Sent Done Done false
21653 6500 Acked Read 4 Sent Done Done false
21654 6501 Acked Write 4 Sent Done Done false
21655 6502 Acked Read 8 Sent Done Done false
21656 6503 Acked Read 10 Sent Done Done false
21657 6504 Acked Flush 0 Sent Done Done false
21658 6505 Acked Write 7 Sent Done Done false
21659 6506 Acked Write 9 Sent Done Done false
21660 6507 Acked Write 5 Sent Done Done false
21661 6508 Acked Write 3 Sent Done Done false
21662 6509 Acked Read 10 Sent Done Done false
21663 6510 Acked Read 3 Sent Done Done false
21664 6511 Acked Read 8 Sent Done Done false
21665 6512 Acked Read 10 Sent Done Done false
21666 6513 Acked Write 4 Sent Done Done false
21667 6514 Acked Write 4 Sent Done Done false
21668 6515 Acked Read 7 Sent Done Done false
21669 6516 Acked Write 10 Sent Done Done false
21670 6517 Acked Flush 0 Sent Done Done false
21671 6518 Acked Write 9 Sent Done Done false
21672 6519 Acked Write 6 Sent Done Done false
21673 6520 Acked Read 7 Sent Done Done false
21674 6521 Acked Read 7 Sent Done Done false
21675 6522 Acked Write 4 Sent Done Done false
21676 6523 Acked Read 3 Sent Done Done false
21677 6524 Acked Read 2 Sent Done Done false
21678 6525 Acked Write 4 Sent Done Done false
21679 6526 Acked Write 9 Sent Done Done false
21680 6527 Acked Read 6 Sent Done Done false
21681 6528 Acked Read 2 Sent Done Done false
21682 6529 Acked Flush 0 Sent Done Done false
21683 6530 Acked Read 6 Sent Done Done false
21684 6531 Acked Read 1 Sent Done Done false
21685 6532 Acked Read 8 Sent Done Done false
21686 6533 Acked Write 2 Sent Done Done false
21687 6534 Acked Write 4 Sent Done Done false
21688 6535 Acked Write 5 Sent Done Done false
21689 6536 Acked Write 2 Sent Done Done false
21690 6537 Acked Write 5 Sent Done Done false
21691 6538 Acked Read 6 Sent Done Done false
21692 6539 Acked Read 4 Sent Done Done false
21693 6540 Acked Write 3 Sent Done Done false
21694 6541 Acked Read 1 Sent Done Done false
21695 6542 Acked Read 9 Sent Done Done false
21696 6543 Acked Read 2 Sent Done Done false
21697 6544 Acked Read 10 Sent Done Done false
21698 6545 Acked Read 8 Sent Done Done false
21699 6546 Acked Read 6 Sent Done Done false
21700 6547 Acked Write 2 Sent Done Done false
21701 6548 Acked Write 3 Sent Done Done false
21702 6549 Acked Read 9 Sent Done Done false
21703 6550 Acked Write 5 Sent Done Done false
21704 6551 Acked Read 9 Sent Done Done false
21705 6552 Acked Read 4 Sent Done Done false
21706 6553 Acked Flush 0 Sent Done Done false
21707 6554 Acked Write 3 Sent Done Done false
21708 6555 Acked Flush 0 Sent Done Done false
21709 6556 Acked Read 2 Sent Done Done false
21710 6557 Acked Read 8 Sent Done Done false
21711 6558 Acked Write 5 Sent Done Done false
21712 6559 Acked Write 4 Sent Done Done false
21713 6560 Acked Flush 0 Sent Done Done false
21714 6561 Acked Read 8 Sent Done Done false
21715 6562 Acked Flush 0 Sent Done Done false
21716 6563 Acked Read 2 Sent Done Done false
21717 6564 Acked Write 2 Sent Done Done false
21718 6565 Acked Read 10 Sent Done Done false
21719 6566 Acked Read 7 Sent Done Done false
21720 6567 Acked Read 3 Sent Done Done false
21721 6568 Acked Write 6 Sent Done Done false
21722 6569 Acked Write 7 Sent Done Done false
21723 6570 Acked Write 9 Sent Done Done false
21724 6571 Acked Read 4 Sent Done Done false
21725 6572 Acked Read 3 Sent Done Done false
21726 6573 Acked Read 4 Sent Done Done false
21727 6574 Acked Read 3 Sent Done Done false
21728 6575 Acked Write 9 Sent Done Done false
21729 6576 Acked Write 7 Sent Done Done false
21730 6577 Acked Read 8 Sent Done Done false
21731 6578 Acked Write 9 Sent Done Done false
21732 6579 Acked Flush 0 Sent Done Done false
21733 6580 Acked Read 7 Sent Done Done false
21734 6581 Acked Read 6 Sent Done Done false
21735 6582 Acked Read 7 Sent Done Done false
21736 6583 Acked Write 9 Sent Done Done false
21737 6584 Acked Flush 0 Sent Done Done false
21738 6585 Acked Read 5 Sent Done Done false
21739 6586 Acked Write 9 Sent Done Done false
21740 6587 Acked Write 1 Sent Done Done false
21741 6588 Acked Read 6 Sent Done Done false
21742 6589 Acked Read 10 Sent Done Done false
21743 6590 Acked Read 5 Sent Done Done false
21744 6591 Acked Write 5 Sent Done Done false
21745 6592 Acked Flush 0 Sent Done Done false
21746 6593 Acked Write 7 Sent Done Done false
21747 6594 Acked Write 2 Sent Done Done false
21748 6595 Acked Read 1 Sent Done Done false
21749 6596 Acked Read 4 Sent Done Done false
21750 6597 Acked Read 6 Sent Done Done false
21751 6598 Acked Write 7 Sent Done Done false
21752 6599 Acked Write 8 Sent Done Done false
21753 6600 Acked Read 4 Sent Done Done false
21754 6601 Acked Read 3 Sent Done Done false
21755 6602 Acked Read 9 Sent Done Done false
21756 6603 Acked Read 7 Sent Done Done false
21757 6604 Acked Write 9 Sent Done Done false
21758 6605 Acked Read 7 Sent Done Done false
21759 6606 Acked Read 8 Sent Done Done false
21760 6607 Acked Read 4 Sent Done Done false
21761 6608 Acked Read 8 Sent Done Done false
21762 6609 Acked Write 5 Sent Done Done false
21763 6610 Acked Read 7 Sent Done Done false
21764 6611 Acked Write 2 Sent Done Done false
21765 6612 Acked Write 3 Sent Done Done false
21766 6613 Acked Flush 0 Sent Done Done false
21767 6614 Acked Read 6 Sent Done Done false
21768 6615 Acked Read 5 Sent Done Done false
21769 6616 Acked Write 5 Sent Done Done false
21770 6617 Acked Write 8 Sent Done Done false
21771 6618 Acked Read 10 Sent Done Done false
21772 6619 Acked Read 2 Sent Done Done false
21773 6620 Acked Write 3 Sent Done Done false
21774 6621 Acked Write 1 Sent Done Done false
21775 6622 Acked Read 3 Sent Done Done false
21776 6623 Acked Write 1 Sent Done Done false
21777 6624 Acked Read 9 Sent Done Done false
21778 6625 Acked Flush 0 Sent Done Done false
21779 6626 Acked Write 4 Sent Done Done false
21780 6627 Acked Write 7 Sent Done Done false
21781 6628 Acked Read 9 Sent Done Done false
21782 6629 Acked Read 7 Sent Done Done false
21783 6630 Acked Read 1 Sent Done Done false
21784 6631 Acked Write 9 Sent Done Done false
21785 6632 Acked Read 3 Sent Done Done false
21786 6633 Acked Flush 0 Sent Done Done false
21787 6634 Acked Write 6 Sent Done Done false
21788 6635 Acked Flush 0 Sent Done Done false
21789 6636 Acked Write 9 Sent Done Done false
21790 6637 Acked Read 3 Sent Done Done false
21791 6638 Acked Read 9 Sent Done Done false
21792 6639 Acked Flush 0 Sent Done Done false
21793 6640 Acked Write 2 Sent Done Done false
21794 6641 Acked Read 3 Sent Done Done false
21795 6642 Acked Read 6 Sent Done Done false
21796 6643 Acked Read 4 Sent Done Done false
21797 6644 Acked Read 5 Sent Done Done false
21798 6645 Acked Read 10 Sent Done Done false
21799 6646 Acked Read 3 Sent Done Done false
21800 6647 Acked Read 1 Sent Done Done false
21801 6648 Acked Read 1 Sent Done Done false
21802 6649 Acked Read 1 Sent Done Done false
21803 6650 Acked Write 8 Sent Done Done false
21804 6651 Acked Read 6 Sent Done Done false
21805 6652 Acked Flush 0 Sent Done Done false
21806 6653 Acked Read 2 Sent Done Done false
21807 6654 Acked Write 4 Sent Done Done false
21808 6655 Acked Write 9 Sent Done Done false
21809 6656 Acked Write 8 Sent Done Done false
21810 6657 Acked Read 7 Sent Done Done false
21811 6658 Acked Write 1 Sent Done Done false
21812 6659 Acked Write 2 Sent Done Done false
21813 6660 Acked Read 4 Sent Done Done false
21814 6661 Acked Read 7 Sent Done Done false
21815 6662 Acked Read 4 Sent Done Done false
21816 6663 Acked Read 6 Sent Done Done false
21817 6664 Acked Write 9 Sent Done Done false
21818 6665 Acked Write 6 Sent Done Done false
21819 6666 Acked Write 10 Sent Done Done false
21820 6667 Acked Read 9 Sent Done Done false
21821 6668 Acked Read 3 Sent Done Done false
21822 6669 Acked Write 6 Sent Done Done false
21823 6670 Acked Read 7 Sent Done Done false
21824 6671 Acked Read 4 Sent Done Done false
21825 6672 Acked Write 4 Sent Done Done false
21826 6673 Acked Write 8 Sent Done Done false
21827 6674 Acked Flush 0 Sent Done Done false
21828 6675 Acked Write 7 Sent Done Done false
21829 6676 Acked Write 9 Sent Done Done false
21830 6677 Acked Read 10 Sent Done Done false
21831 6678 Acked Read 7 Sent Done Done false
21832 6679 Acked Write 4 Sent Done Done false
21833 6680 Acked Write 4 Sent Done Done false
21834 6681 Acked Write 6 Sent Done Done false
21835 6682 Acked Read 3 Sent Done Done false
21836 6683 Acked Read 9 Sent Done Done false
21837 6684 Acked Read 2 Sent Done Done false
21838 6685 Acked Write 3 Sent Done Done false
21839 6686 Acked Write 1 Sent Done Done false
21840 6687 Acked Read 6 Sent Done Done false
21841 6688 Acked Read 9 Sent Done Done false
21842 6689 Acked Write 3 Sent Done Done false
21843 6690 Acked Read 3 Sent Done Done false
21844 6691 Acked Read 10 Sent Done Done false
21845 6692 Acked Read 3 Sent Done Done false
21846 6693 Acked Read 3 Sent Done Done false
21847 6694 Acked Write 7 Sent Done Done false
21848 6695 Acked Write 3 Sent Done Done false
21849 6696 Acked Write 8 Sent Done Done false
21850 6697 Acked Write 1 Sent Done Done false
21851 6698 Acked Read 9 Sent Done Done false
21852 6699 Acked Read 5 Sent Done Done false
21853 6700 Acked Read 9 Sent Done Done false
21854 6701 Acked Read 3 Sent Done Done false
21855 6702 Acked Write 9 Sent Done Done false
21856 6703 Acked Read 8 Sent Done Done false
21857 6704 Acked Write 8 Sent Done Done false
21858 6705 Acked Write 3 Sent Done Done false
21859 6706 Acked Read 10 Sent Done Done false
21860 6707 Acked Flush 0 Sent Done Done false
21861 6708 Acked Read 7 Sent Done Done false
21862 6709 Acked Write 5 Sent Done Done false
21863 6710 Acked Read 4 Sent Done Done false
21864 6711 Acked Read 10 Sent Done Done false
21865 6712 Acked Read 7 Sent Done Done false
21866 6713 Acked Read 5 Sent Done Done false
21867 6714 Acked Read 7 Sent Done Done false
21868 6715 Acked Read 8 Sent Done Done false
21869 6716 Acked Write 6 Sent Done Done false
21870 6717 Acked Read 2 Sent Done Done false
21871 6718 Acked Read 7 Sent Done Done false
21872 6719 Acked Flush 0 Sent Done Done false
21873 6720 Acked Write 7 Sent Done Done false
21874 6721 Acked Write 8 Sent Done Done false
21875 6722 Acked Write 10 Sent Done Done false
21876 6723 Acked Write 8 Sent Done Done false
21877 6724 Acked Write 10 Sent Done Done false
21878 6725 Acked Read 9 Sent Done Done false
21879 6726 Acked Read 4 Sent Done Done false
21880 6727 Acked Write 2 Sent Done Done false
21881 6728 Acked Write 2 Sent Done Done false
21882 6729 Acked Write 3 Sent Done Done false
21883 6730 Acked Read 7 Sent Done Done false
21884 6731 Acked Write 3 Sent Done Done false
21885 6732 Acked Read 1 Sent Done Done false
21886 6733 Acked Write 5 Sent Done Done false
21887 6734 Acked Read 1 Sent Done Done false
21888 6735 Acked Flush 0 Sent Done Done false
21889 STATES DS:0 DS:1 DS:2 TOTAL
21890 Sent 302 0 0 302
21891 Done 0 302 302 604
21892 Skipped 0 0 0 0
21893 Error 0 0 0 0
21894 Last Flush: 6433 6735 6735
21895 Downstairs last five completed: 6433 6432 6431 6430 6429
21896 Upstairs last five completed: 6735 6734 6733 6732 6731
21897 CLIENT: Up:0 ds:302 act:2
218982024-11-19T17:12:34.160ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
218992024-11-19T17:12:34.160ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40344) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219002024-11-19T17:12:34.161ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219012024-11-19T17:12:34.162ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219022024-11-19T17:12:34.162ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219032024-11-19T17:12:34.162ZINFOcrucible: send last flush ID to this DS: 6433 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219042024-11-19T17:12:34.162ZINFOcrucible: Replied this last flush ID: 6433 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219052024-11-19T17:12:34.162ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219062024-11-19T17:12:34.162ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219072024-11-19T17:12:34.162ZINFOcrucible: [0] client re-new 302 jobs since flush 6433 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
219082024-11-19T17:12:34.162ZINFOcrucible: [0] Replayed 302 jobs since flush: 6433 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
21909 ----------------------------------------------------------------
21910 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21911 Downstairs last five completed: 6735 6734 6733 6732 6731
21912 Upstairs last five completed: 6735 6734 6733 6732 6731
21913 CLIENT: Up:0 ds:0 act:3
21914 Replay: All jobs finished
21915 Main task has work to do, go find it
21916 got dsc Stop(1)
21917 stop 1
21918 Nov 19 17:12:36.416 INFO request completed, latency_us: 346, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 95895541-0a6e-41bf-88e7-5e222877d62f, remote_addr: 127.0.0.1:64175[8820] Got stop action so:false kr:false
21919 , local_addr: 127.0.0.1:9998
21920 Nov 19 17:12:36.417 INFO request completed, latency_us: 134{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:36.417352231Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
21921 , response_code: 200, uri: /state/cid/1, method: GET, req_id: 75e87523-c095-4da1-9bd4-766f904f26d7{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:36.417432324Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
21922 , remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:36.417462362Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
21923 1:9998
219242024-11-19T17:12:36.417ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219252024-11-19T17:12:36.417ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
219262024-11-19T17:12:36.417ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
21927 [8820] Exited with: ExitStatus(unix_wait_status(9))
21928 [8820][1] reports Exit
21929 Nov 19 17:12:40.420 INFO request completed, latency_us: 247, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7c56729e-d6d1-4142-bd0c-6e74d8966ea0, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
21930 001/300 Write block 2664 len 4096 data: 7
21931 002/300 Write block 2792 len 4096 data: 4
21932 003/300 Flush
21933 004/300 Read block 2112 len 12288
21934 005/300 Read block 769 len 40960
21935 006/300 Write block 4890 len 36864 data: 6 6 8 9 8 8 9 8 8
21936 007/300 Read block 1405 len 32768
21937 008/300 Read block 605 len 8192
21938 009/300 Write block 3935 len 8192 data: 2 2
21939 010/300 Write block 2850 len 32768 data: 2 2 2 2 2 2 4 5
21940 011/300 Read block 2813 len 28672
21941 012/300 Read block 3173 len 36864
21942 013/300 Read block 2448 len 8192
21943 014/300 Read block 3171 len 36864
21944 015/300 Write block 4010 len 36864 data: 2 3 3 3 3 5 6 6 6
21945 016/300 Flush
21946 017/300 Write block 2864 len 16384 data: 2 2 2 4
21947 018/300 Write block 2597 len 20480 data: 3 3 3 3 3
21948 019/300 Read block 2346 len 36864
21949 020/300 Read block 1525 len 16384
21950 021/300 Flush
21951 022/300 Write block 4312 len 20480 data: 4 4 3 3 3
21952 023/300 Write block 1404 len 8192 data: 7 8
21953 024/300 Read block 4565 len 36864
21954 025/300 Flush
21955 026/300 Read block 1904 len 8192
21956 027/300 Write block 1057 len 20480 data: 3 3 3 3 4
21957 028/300 Write block 1561 len 24576 data: 5 5 5 5 5 6
21958 029/300 Write block 1254 len 28672 data: 3 2 2 2 4 4 5
21959 030/300 Flush
21960 031/300 Write block 4460 len 32768 data: 5 5 6 6 5 5 5 3
21961 032/300 Read block 2813 len 40960
21962 033/300 Read block 2910 len 24576
21963 034/300 Read block 3050 len 4096
21964 035/300 Write block 1799 len 20480 data: 3 3 4 5 4
21965 036/300 Read block 3232 len 28672
21966 037/300 Read block 2166 len 8192
21967 038/300 Read block 4019 len 4096
21968 039/300 Write block 3480 len 4096 data: 4
21969 040/300 Read block 4876 len 16384
21970 041/300 Read block 964 len 32768
21971 042/300 Read block 149 len 12288
21972 043/300 Write block 4742 len 36864 data: 4 4 4 4 3 3 2 2 4
21973 044/300 Flush
21974 045/300 Write block 750 len 16384 data: 6 6 5 5
21975 046/300 Read block 752 len 24576
21976 047/300 Read block 1884 len 36864
21977 048/300 Read block 104 len 20480
21978 049/300 Write block 64 len 36864 data: 6 5 5 5 4 5 4 3 3
21979 050/300 Read block 3731 len 8192
21980 051/300 Read block 3452 len 28672
21981 052/300 Write block 2411 len 4096 data: 5
21982 053/300 Read block 589 len 36864
21983 054/300 Write block 188 len 4096 data: 5
21984 055/300 Read block 4794 len 40960
21985 056/300 Read block 2242 len 32768
21986 057/300 Read block 4032 len 4096
21987 058/300 Read block 1 len 32768
21988 059/300 Write block 4332 len 8192 data: 5 6
21989 060/300 Write block 3657 len 40960 data: 2 2 4 6 6 7 6 6 5 4
21990 061/300 Flush
21991 062/300 Write block 3204 len 36864 data: 9 8 7 6 3 4 4 3 4
21992 063/300 Read block 3189 len 4096
21993 064/300 Flush
21994 065/300 Read block 1055 len 32768
21995 066/300 Read block 1594 len 4096
21996 067/300 Read block 1898 len 28672
21997 068/300 Read block 2488 len 16384
21998 069/300 Write block 4469 len 20480 data: 4 4 4 5 6
21999 070/300 Write block 3625 len 28672 data: 4 4 4 3 2 3 3
22000 071/300 Read block 19 len 12288
22001 072/300 Write block 1165 len 20480 data: 7 7 6 7 7
22002 073/300 Read block 2475 len 32768
22003 074/300 Read block 1440 len 8192
22004 075/300 Read block 2183 len 40960
22005 076/300 Read block 4725 len 32768
22006 077/300 Write block 4839 len 24576 data: 3 2 2 2 3 3
22007 078/300 Write block 4382 len 12288 data: 6 6 5
22008 079/300 Write block 2985 len 28672 data: 2 2 2 4 5 7 6
22009 080/300 Write block 2274 len 12288 data: 7 6 8
22010 081/300 Write block 2796 len 32768 data: 6 6 5 4 7 6 7 6
22011 082/300 Read block 170 len 20480
22012 083/300 Write block 445 len 28672 data: 3 5 4 5 5 5 5
22013 084/300 Write block 1510 len 36864 data: 3 2 3 3 2 3 3 3 3
22014 085/300 Read block 1181 len 24576
22015 086/300 Read block 1064 len 8192
22016 087/300 Write block 68 len 4096 data: 5
22017 088/300 Flush
22018 089/300 Read block 1883 len 8192
22019 090/300 Read block 1056 len 32768
22020 091/300 Read block 936 len 4096
22021 092/300 Write block 4051 len 24576 data: 2 2 3 3 3 3
22022 093/300 Read block 697 len 12288
22023 094/300 Write block 3318 len 32768 data: 7 7 7 7 6 6 5 5
22024 095/300 Read block 1377 len 4096
22025 096/300 Write block 792 len 12288 data: 7 8 7
22026 097/300 Write block 3844 len 20480 data: 6 5 5 5 3
22027 098/300 Read block 2955 len 20480
22028 099/300 Read block 1344 len 32768
22029 100/300 Read block 677 len 12288
22030 101/300 Flush
22031 102/300 Read block 961 len 12288
22032 103/300 Write block 4266 len 12288 data: 8 7 5
22033 104/300 Flush
22034 105/300 Read block 3256 len 8192
22035 106/300 Read block 1804 len 4096
22036 107/300 Write block 1112 len 4096 data: 5
22037 108/300 Read block 3651 len 16384
22038 109/300 Read block 4994 len 4096
22039 110/300 Flush
22040 111/300 Write block 4386 len 36864 data: 5 5 4 4 4 3 4 4 4
22041 112/300 Write block 1096 len 16384 data: 3 2 3 5
22042 113/300 Read block 3325 len 4096
22043 114/300 Read block 251 len 36864
22044 115/300 Write block 3344 len 36864 data: 3 4 3 3 4 4 4 3 3
22045 116/300 Write block 3757 len 12288 data: 6 7 6
22046 117/300 Read block 3354 len 8192
22047 118/300 Read block 918 len 8192
22048 119/300 Read block 2717 len 8192
22049 120/300 Flush
22050 121/300 Read block 327 len 32768
22051 122/300 Flush
22052 123/300 Read block 3423 len 20480
22053 124/300 Read block 3412 len 28672
22054 125/300 Flush
22055 126/300 Write block 4176 len 16384 data: 3 2 3 3
22056 127/300 Flush
22057 128/300 Read block 3652 len 16384
22058 129/300 Read block 2632 len 40960
22059 130/300 Write block 4955 len 40960 data: 4 5 6 6 5 5 5 4 2 3
22060 131/300 Flush
22061 132/300 Write block 3080 len 20480 data: 6 5 4 4 4
22062 133/300 Write block 907 len 4096 data: 6
22063 134/300 Read block 4271 len 28672
22064 135/300 Write block 3431 len 40960 data: 4 5 5 4 5 5 6 5 6 6
22065 136/300 Write block 2328 len 12288 data: 4 4 5
22066 137/300 Read block 3339 len 12288
22067 138/300 Write block 3095 len 24576 data: 2 2 3 3 3 3
22068 139/300 Read block 993 len 40960
22069 140/300 Read block 2619 len 32768
22070 141/300 Read block 4730 len 4096
22071 142/300 Write block 1392 len 36864 data: 5 4 4 5 5 6 6 7 8
22072 143/300 Read block 3834 len 28672
22073 144/300 Write block 1949 len 12288 data: 5 6 8
22074 145/300 Read block 4927 len 12288
22075 146/300 Write block 1765 len 16384 data: 4 4 4 2
22076 147/300 Read block 2977 len 20480
22077 148/300 Write block 2347 len 24576 data: 6 4 4 5 5 3
22078 149/300 Write block 66 len 40960 data: 6 6 6 6 5 4 4 3 3 3
22079 150/300 Write block 4795 len 28672 data: 7 5 6 5 5 5 6
22080 151/300 Read block 4775 len 32768
22081 152/300 Write block 1264 len 24576 data: 4 4 4 5 6 7
22082 153/300 Flush
22083 154/300 Write block 34 len 8192 data: 5 5
22084 155/300 Write block 4852 len 40960 data: 4 4 3 3 3 3 3 3 3 4
22085 156/300 Write block 3536 len 40960 data: 4 4 3 3 3 3 3 2 4 3
22086 157/300 Write block 72 len 28672 data: 5 4 4 4 4 4 4
22087 158/300 Read block 1225 len 12288
22088 159/300 Write block 4671 len 24576 data: 5 5 5 5 5 5
22089 160/300 Write block 960 len 36864 data: 3 3 3 4 4 3 4 4 4
22090 161/300 Read block 1056 len 24576
22091 162/300 Write block 4721 len 4096 data: 2
22092 163/300 Write block 3239 len 12288 data: 4 6 6
22093 164/300 Write block 1081 len 28672 data: 2 2 2 2 3 3 4
22094 165/300 Write block 2003 len 4096 data: 2
22095 166/300 Read block 3966 len 20480
22096 167/300 Write block 2736 len 8192 data: 4 4
22097 168/300 Write block 4835 len 12288 data: 2 2 2
22098 169/300 Write block 2710 len 24576 data: 6 6 6 6 5 5
22099 170/300 Read block 1411 len 8192
22100 171/300 Read block 4153 len 24576
22101 172/300 Write block 3977 len 32768 data: 3 2 2 2 4 4 2 2
22102 173/300 Write block 710 len 32768 data: 6 5 5 4 3 4 4 4
22103 174/300 Read block 597 len 36864
22104 175/300 Write block 2113 len 4096 data: 2
22105 176/300 Read block 2889 len 4096
22106 177/300 Write block 3118 len 20480 data: 4 4 4 4 4
22107 178/300 Read block 4919 len 28672
22108 179/300 Write block 562 len 40960 data: 4 3 3 4 5 3 3 4 4 5
22109 180/300 Flush
22110 181/300 Write block 2111 len 4096 data: 3
22111 182/300 Write block 4870 len 24576 data: 4 4 5 5 5 5
22112 183/300 Write block 433 len 8192 data: 3 3
22113 184/300 Read block 368 len 20480
22114 185/300 Write block 3125 len 16384 data: 6 10 8 9
22115 186/300 Read block 1257 len 40960
22116 187/300 Read block 4744 len 28672
22117 188/300 Flush
22118 189/300 Write block 4460 len 40960 data: 6 6 7 7 6 6 6 4 3 5
22119 190/300 Flush
22120 191/300 Read block 786 len 20480
22121 192/300 Read block 1486 len 20480
22122 193/300 Write block 2626 len 36864 data: 2 2 2 2 2 2 4 5 4
22123 194/300 Flush
22124 195/300 Read block 3029 len 16384
22125 196/300 Flush
22126 197/300 Write block 4959 len 4096 data: 6
22127 198/300 Write block 913 len 40960 data: 4 4 4 4 4 4 4 4 3 3
22128 199/300 Read block 97 len 20480
22129 200/300 Read block 2893 len 40960
22130 201/300 Write block 680 len 4096 data: 4
22131 202/300 Write block 1608 len 28672 data: 7 9 10 11 10 10 10
22132 203/300 Write block 618 len 8192 data: 7 7
22133 204/300 Read block 821 len 24576
22134 205/300 Flush
22135 206/300 Write block 1811 len 36864 data: 5 6 5 5 3 3 5 6 6
22136 207/300 Read block 2911 len 32768
22137 208/300 Read block 4406 len 36864
22138 209/300 Write block 618 len 24576 data: 8 8 6 4 4 4
22139 210/300 Flush
22140 211/300 Write block 4463 len 24576 data: 8 7 7 7 5 4
22141 212/300 Read block 2778 len 4096
22142 213/300 Read block 2669 len 28672
22143 214/300 Read block 3993 len 28672
22144 215/300 Write block 516 len 40960 data: 3 4 3 3 3 3 4 4 5 4
22145 216/300 Write block 1740 len 28672 data: 3 4 5 5 5 4 3
22146 217/300 Write block 3480 len 20480 data: 5 7 10 8 9
22147 218/300 Read block 4539 len 20480
22148 219/300 Read block 4301 len 32768
22149 220/300 Read block 3634 len 12288
22150 221/300 Read block 2324 len 32768
22151 222/300 Flush
22152 223/300 Write block 1038 len 8192 data: 4 3
22153 224/300 Read block 2849 len 20480
22154 225/300 Read block 2402 len 28672
22155 226/300 Read block 3384 len 8192
22156 227/300 Flush
22157 228/300 Write block 2917 len 28672 data: 5 5 5 5 6 5 5
22158 229/300 Flush
22159 230/300 Read block 637 len 8192
22160 231/300 Read block 66 len 16384
22161 232/300 Read block 781 len 4096
22162 233/300 Flush
22163 234/300 Write block 529 len 4096 data: 4
22164 235/300 Flush
22165 236/300 Read block 3104 len 28672
22166 237/300 Read block 2697 len 40960
22167 238/300 Flush
22168 239/300 Read block 4734 len 24576
22169 240/300 Write block 144 len 4096 data: 5
22170 241/300 Read block 1040 len 40960
22171 242/300 Write block 2473 len 12288 data: 4 4 3
22172 243/300 Write block 1414 len 20480 data: 3 4 5 6 7
22173 244/300 Flush
22174 245/300 Read block 979 len 16384
22175 246/300 Read block 4224 len 24576
22176 247/300 Read block 408 len 36864
22177 248/300 Read block 4988 len 16384
22178 249/300 Read block 804 len 16384
22179 250/300 Read block 4750 len 16384
22180 251/300 Write block 1931 len 16384 data: 5 6 6 4
22181 252/300 Write block 3197 len 16384 data: 4 4 5 6
22182 253/300 Write block 3303 len 8192 data: 3 5
22183 254/300 Read block 62 len 36864
22184 255/300 Write block 3571 len 24576 data: 4 3 2 3 3 3
22185 256/300 Write block 4703 len 12288 data: 4 4 4
22186 257/300 Flush
22187 258/300 Write block 682 len 8192 data: 4 5
22188 259/300 Flush
22189 260/300 Read block 1825 len 32768
22190 261/300 Write block 3993 len 8192 data: 7 6
22191 262/300 Write block 2727 len 20480 data: 6 6 6 6 6
22192 263/300 Read block 2020 len 36864
22193 264/300 Write block 1990 len 32768 data: 3 4 4 4 3 3 5 4
22194 265/300 Read block 3823 len 8192
22195 266/300 Write block 1123 len 20480 data: 5 5 5 5 5
22196 267/300 Read block 3460 len 24576
22197 268/300 Write block 4224 len 36864 data: 6 4 3 2 2 2 2 2 2
22198 269/300 Write block 2485 len 36864 data: 6 6 7 7 5 5 4 3 3
22199 270/300 Flush
22200 271/300 Write block 1445 len 16384 data: 4 7 9 8
22201 272/300 Flush
22202 273/300 Read block 1856 len 4096
22203 274/300 Read block 2784 len 8192
22204 275/300 Read block 4776 len 12288
22205 276/300 Read block 3209 len 12288
22206 277/300 Write block 3093 len 4096 data: 2
22207 278/300 Read block 3015 len 24576
22208 279/300 Read block 3550 len 40960
22209 280/300 Write block 3279 len 8192 data: 6 6
22210 281/300 Write block 4459 len 8192 data: 6 7
22211 282/300 Read block 1746 len 36864
22212 283/300 Read block 1307 len 28672
22213 284/300 Write block 694 len 32768 data: 2 2 2 3 3 2 2 4
22214 285/300 Write block 2328 len 12288 data: 5 5 6
22215 286/300 Read block 125 len 32768
22216 287/300 Read block 928 len 4096
22217 288/300 Write block 607 len 32768 data: 3 4 4 3 3 3 4 5
22218 289/300 Write block 2739 len 28672 data: 4 4 3 3 3 4 5
22219 290/300 Flush
22220 291/300 Write block 4929 len 32768 data: 7 5 5 8 10 8 6 6
22221 292/300 Write block 3262 len 28672 data: 3 3 3 4 3 4 4
22222 293/300 Write block 3702 len 4096 data: 4
22223 294/300 Write block 163 len 36864 data: 2 2 2 2 3 3 3 3 3
22224 295/300 Write block 2281 len 20480 data: 6 4 3 4 5
22225 296/300 Read block 1081 len 16384
22226 297/300 Read block 695 len 28672
22227 298/300 Read block 851 len 28672
22228 299/300 Write block 455 len 8192 data: 3 3
22229 300/300 Write block 132 len 20480 data: 6 8 10 10 11
22230 301/300 Write block 3708 len 28672 data: 6 6 6 6 6 5 6
22231 Main task has work to do, go find it
22232 got dsc Start(1)
22233 start 1
22234 Nov 19 17:12:40.644 INFO request completed, latency_us: 187, response_code: 204, uri: /start/cid/1, method: POST, req_id: 74d59674-5676-45e9-910d-74ebe2cfdf74, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
22235 Replay: started 1, returned:Ok(())
22236 ----------------------------------------------------------------
22237 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22238 States: Active Offline Active
22239 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22240 6736 Acked Write 1 Done Sent Done false
22241 6737 Acked Write 1 Done Sent Done false
22242 6738 Acked Flush 0 Done Sent Done false
22243 6739 Acked Read 3 Done Sent Done false
22244 6740 Acked Read 10 Done Sent Done false
22245 6741 Acked Write 9 Done Sent Done false
22246 6742 Acked Read 8 Done Sent Done false
22247 6743 Acked Read 2 Done Sent Done false
22248 6744 Acked Write 2 Done Sent Done false
22249 6745 Acked Write 8 Done Sent Done false
22250 6746 Acked Read 7 Done Sent Done false
22251 6747 Acked Read 9 Done Sent Done false
22252 6748 Acked Read 2 Done Sent Done false
22253 6749 Acked Read 9 Done Sent Done false
22254 6750 Acked Write 9 Done Sent Done false
22255 6751 Acked Flush 0 Done Sent Done false
22256 6752 Acked Write 4 Done Sent Done false
22257 6753 Acked Write 5 Done Sent Done false
22258 6754 Acked Read 9 Done Sent Done false
22259 6755 Acked Read 4 Done Sent Done false
22260 6756 Acked Flush 0 Done Sent Done false
22261 6757 Acked Write 5 Done Sent Done false
22262 6758 Acked Write 2 Done Sent Done false
22263 6759 Acked Read 9 Done Sent Done false
22264 6760 Acked Flush 0 Done Sent Done false
22265 6761 Acked Read 2 Done Sent Done false
22266 6762 Acked Write 5 Done Sent Done false
22267 6763 Acked Write 6 Done Sent Done false
22268 6764 Acked Write 7 Done Sent Done false
22269 6765 Acked Flush 0 Done Sent Done false
22270 6766 Acked Write 8 Done Sent Done false
22271 6767 Acked Read 10 Done Sent Done false
22272 6768 Acked Read 6 Done Sent Done false
22273 6769 Acked Read 1 Done Sent Done false
22274 6770 Acked Write 5 Done Sent Done false
22275 6771 Acked Read 7 Done Sent Done false
22276 6772 Acked Read 2 Done Sent Done false
22277 6773 Acked Read 1 Done Sent Done false
22278 6774 Acked Write 1 Done Sent Done false
22279 6775 Acked Read 4 Done Sent Done false
22280 6776 Acked Read 8 Done Sent Done false
22281 6777 Acked Read 3 Done Sent Done false
22282 6778 Acked Write 9 Done Sent Done false
22283 6779 Acked Flush 0 Done Sent Done false
22284 6780 Acked Write 4 Done Sent Done false
22285 6781 Acked Read 6 Done Sent Done false
22286 6782 Acked Read 9 Done Sent Done false
22287 6783 Acked Read 5 Done Sent Done false
22288 6784 Acked Write 9 Done Sent Done false
22289 6785 Acked Read 2 Done Sent Done false
22290 6786 Acked Read 7 Done Sent Done false
22291 6787 Acked Write 1 Done Sent Done false
22292 6788 Acked Read 9 Done Sent Done false
22293 6789 Acked Write 1 Done Sent Done false
22294 6790 Acked Read 10 Done Sent Done false
22295 6791 Acked Read 8 Done Sent Done false
22296 6792 Acked Read 1 Done Sent Done false
22297 6793 Acked Read 8 Done Sent Done false
22298 6794 Acked Write 2 Done Sent Done false
22299 6795 Acked Write 10 Done Sent Done false
22300 6796 Acked Flush 0 Done Sent Done false
22301 6797 Acked Write 9 Done Sent Done false
22302 6798 Acked Read 1 Done Sent Done false
22303 6799 Acked Flush 0 Done Sent Done false
22304 6800 Acked Read 8 Done Sent Done false
22305 6801 Acked Read 1 Done Sent Done false
22306 6802 Acked Read 7 Done Sent Done false
22307 6803 Acked Read 4 Done Sent Done false
22308 6804 Acked Write 5 Done Sent Done false
22309 6805 Acked Write 7 Done Sent Done false
22310 6806 Acked Read 3 Done Sent Done false
22311 6807 Acked Write 5 Done Sent Done false
22312 6808 Acked Read 8 Done Sent Done false
22313 6809 Acked Read 2 Done Sent Done false
22314 6810 Acked Read 10 Done Sent Done false
22315 6811 Acked Read 8 Done Sent Done false
22316 6812 Acked Write 6 Done Sent Done false
22317 6813 Acked Write 3 Done Sent Done false
22318 6814 Acked Write 7 Done Sent Done false
22319 6815 Acked Write 3 Done Sent Done false
22320 6816 Acked Write 8 Done Sent Done false
22321 6817 Acked Read 5 Done Sent Done false
22322 6818 Acked Write 7 Done Sent Done false
22323 6819 Acked Write 9 Done Sent Done false
22324 6820 Acked Read 6 Done Sent Done false
22325 6821 Acked Read 2 Done Sent Done false
22326 6822 Acked Write 1 Done Sent Done false
22327 6823 Acked Flush 0 Done Sent Done false
22328 6824 Acked Read 2 Done Sent Done false
22329 6825 Acked Read 8 Done Sent Done false
22330 6826 Acked Read 1 Done Sent Done false
22331 6827 Acked Write 6 Done Sent Done false
22332 6828 Acked Read 3 Done Sent Done false
22333 6829 Acked Write 8 Done Sent Done false
22334 6830 Acked Read 1 Done Sent Done false
22335 6831 Acked Write 3 Done Sent Done false
22336 6832 Acked Write 5 Done Sent Done false
22337 6833 Acked Read 5 Done Sent Done false
22338 6834 Acked Read 8 Done Sent Done false
22339 6835 Acked Read 3 Done Sent Done false
22340 6836 Acked Flush 0 Done Sent Done false
22341 6837 Acked Read 3 Done Sent Done false
22342 6838 Acked Write 3 Done Sent Done false
22343 6839 Acked Flush 0 Done Sent Done false
22344 6840 Acked Read 2 Done Sent Done false
22345 6841 Acked Read 1 Done Sent Done false
22346 6842 Acked Write 1 Done Sent Done false
22347 6843 Acked Read 4 Done Sent Done false
22348 6844 Acked Read 1 Done Sent Done false
22349 6845 Acked Flush 0 Done Sent Done false
22350 6846 Acked Write 9 Done Sent Done false
22351 6847 Acked Write 4 Done Sent Done false
22352 6848 Acked Read 1 Done Sent Done false
22353 6849 Acked Read 9 Done Sent Done false
22354 6850 Acked Write 9 Done Sent Done false
22355 6851 Acked Write 3 Done Sent Done false
22356 6852 Acked Read 2 Done Sent Done false
22357 6853 Acked Read 2 Done Sent Done false
22358 6854 Acked Read 2 Done Sent Done false
22359 6855 Acked Flush 0 Done Sent Done false
22360 6856 Acked Read 8 Done Sent Done false
22361 6857 Acked Flush 0 Done Sent Done false
22362 6858 Acked Read 5 Done Sent Done false
22363 6859 Acked Read 7 Done Sent Done false
22364 6860 Acked Flush 0 Done Sent Done false
22365 6861 Acked Write 4 Done Sent Done false
22366 6862 Acked Flush 0 Done Sent Done false
22367 6863 Acked Read 4 Done Sent Done false
22368 6864 Acked Read 10 Done Sent Done false
22369 6865 Acked Write 10 Done Sent Done false
22370 6866 Acked Flush 0 Done Sent Done false
22371 6867 Acked Write 5 Done Sent Done false
22372 6868 Acked Write 1 Done Sent Done false
22373 6869 Acked Read 7 Done Sent Done false
22374 6870 Acked Write 10 Done Sent Done false
22375 6871 Acked Write 3 Done Sent Done false
22376 6872 Acked Read 3 Done Sent Done false
22377 6873 Acked Write 6 Done Sent Done false
22378 6874 Acked Read 10 Done Sent Done false
22379 6875 Acked Read 8 Done Sent Done false
22380 6876 Acked Read 1 Done Sent Done false
22381 6877 Acked Write 9 Done Sent Done false
22382 6878 Acked Read 7 Done Sent Done false
22383 6879 Acked Write 3 Done Sent Done false
22384 6880 Acked Read 3 Done Sent Done false
22385 6881 Acked Write 4 Done Sent Done false
22386 6882 Acked Read 5 Done Sent Done false
22387 6883 Acked Write 6 Done Sent Done false
22388 6884 Acked Write 10 Done Sent Done false
22389 6885 Acked Write 7 Done Sent Done false
22390 6886 Acked Read 8 Done Sent Done false
22391 6887 Acked Write 6 Done Sent Done false
22392 6888 Acked Flush 0 Done Sent Done false
22393 6889 Acked Write 2 Done Sent Done false
22394 6890 Acked Write 10 Done Sent Done false
22395 6891 Acked Write 10 Done Sent Done false
22396 6892 Acked Write 7 Done Sent Done false
22397 6893 Acked Read 3 Done Sent Done false
22398 6894 Acked Write 6 Done Sent Done false
22399 6895 Acked Write 9 Done Sent Done false
22400 6896 Acked Read 6 Done Sent Done false
22401 6897 Acked Write 1 Done Sent Done false
22402 6898 Acked Write 3 Done Sent Done false
22403 6899 Acked Write 7 Done Sent Done false
22404 6900 Acked Write 1 Done Sent Done false
22405 6901 Acked Read 5 Done Sent Done false
22406 6902 Acked Write 2 Done Sent Done false
22407 6903 Acked Write 3 Done Sent Done false
22408 6904 Acked Write 6 Done Sent Done false
22409 6905 Acked Read 2 Done Sent Done false
22410 6906 Acked Read 6 Done Sent Done false
22411 6907 Acked Write 8 Done Sent Done false
22412 6908 Acked Write 8 Done Sent Done false
22413 6909 Acked Read 9 Done Sent Done false
22414 6910 Acked Write 1 Done Sent Done false
22415 6911 Acked Read 1 Done Sent Done false
22416 6912 Acked Write 5 Done Sent Done false
22417 6913 Acked Read 7 Done Sent Done false
22418 6914 Acked Write 10 Done Sent Done false
22419 6915 Acked Flush 0 Done Sent Done false
22420 6916 Acked Write 1 Done Sent Done false
22421 6917 Acked Write 6 Done Sent Done false
22422 6918 Acked Write 2 Done Sent Done false
22423 6919 Acked Read 5 Done Sent Done false
22424 6920 Acked Write 4 Done Sent Done false
22425 6921 Acked Read 10 Done Sent Done false
22426 6922 Acked Read 7 Done Sent Done false
22427 6923 Acked Flush 0 Done Sent Done false
22428 6924 Acked Write 10 Done Sent Done false
22429 6925 Acked Flush 0 Done Sent Done false
22430 6926 Acked Read 5 Done Sent Done false
22431 6927 Acked Read 5 Done Sent Done false
22432 6928 Acked Write 9 Done Sent Done false
22433 6929 Acked Flush 0 Done Sent Done false
22434 6930 Acked Read 4 Done Sent Done false
22435 6931 Acked Flush 0 Done Sent Done false
22436 6932 Acked Write 1 Done Sent Done false
22437 6933 Acked Write 10 Done Sent Done false
22438 6934 Acked Read 5 Done Sent Done false
22439 6935 Acked Read 10 Done Sent Done false
22440 6936 Acked Write 1 Done Sent Done false
22441 6937 Acked Write 7 Done Sent Done false
22442 6938 Acked Write 2 Done Sent Done false
22443 6939 Acked Read 6 Done Sent Done false
22444 6940 Acked Flush 0 Done Sent Done false
22445 6941 Acked Write 9 Done Sent Done false
22446 6942 Acked Read 8 Done Sent Done false
22447 6943 Acked Read 9 Done Sent Done false
22448 6944 Acked Write 6 Done Sent Done false
22449 6945 Acked Flush 0 Done Sent Done false
22450 6946 Acked Write 6 Done Sent Done false
22451 6947 Acked Read 1 Done Sent Done false
22452 6948 Acked Read 7 Done Sent Done false
22453 6949 Acked Read 7 Done Sent Done false
22454 6950 Acked Write 10 Done Sent Done false
22455 6951 Acked Write 7 Done Sent Done false
22456 6952 Acked Write 5 Done Sent Done false
22457 6953 Acked Read 5 Done Sent Done false
22458 6954 Acked Read 8 Done Sent Done false
22459 6955 Acked Read 3 Done Sent Done false
22460 6956 Acked Read 8 Done Sent Done false
22461 6957 Acked Flush 0 Done Sent Done false
22462 6958 Acked Write 2 Done Sent Done false
22463 6959 Acked Read 5 Done Sent Done false
22464 6960 Acked Read 7 Done Sent Done false
22465 6961 Acked Read 2 Done Sent Done false
22466 6962 Acked Flush 0 Done Sent Done false
22467 6963 Acked Write 7 Done Sent Done false
22468 6964 Acked Flush 0 Done Sent Done false
22469 6965 Acked Read 2 Done Sent Done false
22470 6966 Acked Read 4 Done Sent Done false
22471 6967 Acked Read 1 Done Sent Done false
22472 6968 Acked Flush 0 Done Sent Done false
22473 6969 Acked Write 1 Done Sent Done false
22474 6970 Acked Flush 0 Done Sent Done false
22475 6971 Acked Read 7 Done Sent Done false
22476 6972 Acked Read 10 Done Sent Done false
22477 6973 Acked Flush 0 Done Sent Done false
22478 6974 Acked Read 6 Done Sent Done false
22479 6975 Acked Write 1 Done Sent Done false
22480 6976 Acked Read 10 Done Sent Done false
22481 6977 Acked Write 3 Done Sent Done false
22482 6978 Acked Write 5 Done Sent Done false
22483 6979 Acked Flush 0 Done Sent Done false
22484 6980 Acked Read 4 Done Sent Done false
22485 6981 Acked Read 6 Done Sent Done false
22486 6982 Acked Read 9 Done Sent Done false
22487 6983 Acked Read 4 Done Sent Done false
22488 6984 Acked Read 4 Done Sent Done false
22489 6985 Acked Read 4 Done Sent Done false
22490 6986 Acked Write 4 Done Sent Done false
22491 6987 Acked Write 4 Done Sent Done false
22492 6988 Acked Write 2 Done Sent Done false
22493 6989 Acked Read 9 Done Sent Done false
22494 6990 Acked Write 6 Done Sent Done false
22495 6991 Acked Write 3 Done Sent Done false
22496 6992 Acked Flush 0 Done Sent Done false
22497 6993 Acked Write 2 Done Sent Done false
22498 6994 Acked Flush 0 Done Sent Done false
22499 6995 Acked Read 8 Done Sent Done false
22500 6996 Acked Write 2 Done Sent Done false
22501 6997 Acked Write 5 Done Sent Done false
22502 6998 Acked Read 9 Done Sent Done false
22503 6999 Acked Write 8 Done Sent Done false
22504 7000 Acked Read 2 Done Sent Done false
22505 7001 Acked Write 5 Done Sent Done false
22506 7002 Acked Read 6 Done Sent Done false
22507 7003 Acked Write 9 Done Sent Done false
22508 7004 Acked Write 9 Done Sent Done false
22509 7005 Acked Flush 0 Done Sent Done false
22510 7006 Acked Write 4 Done Sent Done false
22511 7007 Acked Flush 0 Done Sent Done false
22512 7008 Acked Read 1 Done Sent Done false
22513 7009 Acked Read 2 Done Sent Done false
22514 7010 Acked Read 3 Done Sent Done false
22515 7011 Acked Read 3 Done Sent Done false
22516 7012 Acked Write 1 Done Sent Done false
22517 7013 Acked Read 6 Done Sent Done false
22518 7014 Acked Read 10 Done Sent Done false
22519 7015 Acked Write 2 Done Sent Done false
22520 7016 Acked Write 2 Done Sent Done false
22521 7017 Acked Read 9 Done Sent Done false
22522 7018 Acked Read 7 Done Sent Done false
22523 7019 Acked Write 8 Done Sent Done false
22524 7020 Acked Write 3 Done Sent Done false
22525 7021 Acked Read 8 Done Sent Done false
22526 7022 Acked Read 1 Done Sent Done false
22527 7023 Acked Write 8 Done Sent Done false
22528 7024 Acked Write 7 Done Sent Done false
22529 7025 Acked Flush 0 Done Sent Done false
22530 7026 Acked Write 8 Done Sent Done false
22531 7027 Acked Write 7 Done Sent Done false
22532 7028 Acked Write 1 Done Sent Done false
22533 7029 Acked Write 9 Done Sent Done false
22534 7030 Acked Write 5 Done Sent Done false
22535 7031 Acked Read 4 Done Sent Done false
22536 7032 Acked Read 7 Done Sent Done false
22537 7033 Acked Read 7 Done Sent Done false
22538 7034 Acked Write 2 Done Sent Done false
22539 7035 Acked Write 5 Done Sent Done false
22540 7036 Acked Write 7 Done Sent Done false
22541 STATES DS:0 DS:1 DS:2 TOTAL
22542 Sent 0 301 0 301
22543 Done 301 0 301 602
22544 Skipped 0 0 0 0
22545 Error 0 0 0 0
22546 Last Flush: 7025 6735 7025
22547 Downstairs last five completed: 6735 6734 6733 6732 6731
22548 Upstairs last five completed: 7036 7035 7034 7033 7032
22549 CLIENT: Up:0 ds:301 act:2
22550 ----------------------------------------------------------------
22551 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22552 States: Active Offline Active
22553 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22554 6736 Acked Write 1 Done Sent Done false
22555 6737 Acked Write 1 Done Sent Done false
22556 6738 Acked Flush 0 Done Sent Done false
22557 6739 Acked Read 3 Done Sent Done false
22558 6740 Acked Read 10 Done Sent Done false
22559 6741 Acked Write 9 Done Sent Done false
22560 6742 Acked Read 8 Done Sent Done false
22561 6743 Acked Read 2 Done Sent Done false
22562 6744 Acked Write 2 Done Sent Done false
22563 6745 Acked Write 8 Done Sent Done false
22564 6746 Acked Read 7 Done Sent Done false
22565 6747 Acked Read 9 Done Sent Done false
22566 6748 Acked Read 2 Done Sent Done false
22567 6749 Acked Read 9 Done Sent Done false
22568 6750 Acked Write 9 Done Sent Done false
22569 6751 Acked Flush 0 Done Sent Done false
22570 6752 Acked Write 4 Done Sent Done false
22571 6753 Acked Write 5 Done Sent Done false
22572 6754 Acked Read 9 Done Sent Done false
22573 6755 Acked Read 4 Done Sent Done false
22574 6756 Acked Flush 0 Done Sent Done false
22575 6757 Acked Write 5 Done Sent Done false
22576 6758 Acked Write 2 Done Sent Done false
22577 6759 Acked Read 9 Done Sent Done false
22578 6760 Acked Flush 0 Done Sent Done false
22579 6761 Acked Read 2 Done Sent Done false
22580 6762 Acked Write 5 Done Sent Done false
22581 6763 Acked Write 6 Done Sent Done false
22582 6764 Acked Write 7 Done Sent Done false
22583 6765 Acked Flush 0 Done Sent Done false
22584 6766 Acked Write 8 Done Sent Done false
22585 6767 Acked Read 10 Done Sent Done false
22586 6768 Acked Read 6 Done Sent Done false
22587 6769 Acked Read 1 Done Sent Done false
22588 6770 Acked Write 5 Done Sent Done false
22589 6771 Acked Read 7 Done Sent Done false
22590 6772 Acked Read 2 Done Sent Done false
22591 6773 Acked Read 1 Done Sent Done false
22592 6774 Acked Write 1 Done Sent Done false
22593 6775 Acked Read 4 Done Sent Done false
22594 6776 Acked Read 8 Done Sent Done false
22595 6777 Acked Read 3 Done Sent Done false
22596 6778 Acked Write 9 Done Sent Done false
22597 6779 Acked Flush 0 Done Sent Done false
22598 6780 Acked Write 4 Done Sent Done false
22599 6781 Acked Read 6 Done Sent Done false
22600 6782 Acked Read 9 Done Sent Done false
22601 6783 Acked Read 5 Done Sent Done false
22602 6784 Acked Write 9 Done Sent Done false
22603 6785 Acked Read 2 Done Sent Done false
22604 6786 Acked Read 7 Done Sent Done false
22605 6787 Acked Write 1 Done Sent Done false
22606 6788 Acked Read 9 Done Sent Done false
22607 6789 Acked Write 1 Done Sent Done false
22608 6790 Acked Read 10 Done Sent Done false
22609 6791 Acked Read 8 Done Sent Done false
22610 6792 Acked Read 1 Done Sent Done false
22611 6793 Acked Read 8 Done Sent Done false
22612 6794 Acked Write 2 Done Sent Done false
22613 6795 Acked Write 10 Done Sent Done false
22614 6796 Acked Flush 0 Done Sent Done false
22615 6797 Acked Write 9 Done Sent Done false
22616 6798 Acked Read 1 Done Sent Done false
22617 6799 Acked Flush 0 Done Sent Done false
22618 6800 Acked Read 8 Done Sent Done false
22619 6801 Acked Read 1 Done Sent Done false
22620 6802 Acked Read 7 Done Sent Done false
22621 6803 Acked Read 4 Done Sent Done false
22622 6804 Acked Write 5 Done Sent Done false
22623 6805 Acked Write 7 Done Sent Done false
22624 6806 Acked Read 3 Done Sent Done false
22625 6807 Acked Write 5 Done Sent Done false
22626 6808 Acked Read 8 Done Sent Done false
22627 6809 Acked Read 2 Done Sent Done false
22628 6810 Acked Read 10 Done Sent Done false
22629 6811 Acked Read 8 Done Sent Done false
22630 6812 Acked Write 6 Done Sent Done false
22631 6813 Acked Write 3 Done Sent Done false
22632 6814 Acked Write 7 Done Sent Done false
22633 6815 Acked Write 3 Done Sent Done false
22634 6816 Acked Write 8 Done Sent Done false
22635 6817 Acked Read 5 Done Sent Done false
22636 6818 Acked Write 7 Done Sent Done false
22637 6819 Acked Write 9 Done Sent Done false
22638 6820 Acked Read 6 Done Sent Done false
22639 6821 Acked Read 2 Done Sent Done false
22640 6822 Acked Write 1 Done Sent Done false
22641 6823 Acked Flush 0 Done Sent Done false
22642 6824 Acked Read 2 Done Sent Done false
22643 6825 Acked Read 8 Done Sent Done false
22644 6826 Acked Read 1 Done Sent Done false
22645 6827 Acked Write 6 Done Sent Done false
22646 6828 Acked Read 3 Done Sent Done false
22647 6829 Acked Write 8 Done Sent Done false
22648 6830 Acked Read 1 Done Sent Done false
22649 6831 Acked Write 3 Done Sent Done false
22650 6832 Acked Write 5 Done Sent Done false
22651 6833 Acked Read 5 Done Sent Done false
22652 6834 Acked Read 8 Done Sent Done false
22653 6835 Acked Read 3 Done Sent Done false
22654 6836 Acked Flush 0 Done Sent Done false
22655 6837 Acked Read 3 Done Sent Done false
22656 6838 Acked Write 3 Done Sent Done false
22657 6839 Acked Flush 0 Done Sent Done false
22658 6840 Acked Read 2 Done Sent Done false
22659 6841 Acked Read 1 Done Sent Done false
22660 6842 Acked Write 1 Done Sent Done false
22661 6843 Acked Read 4 Done Sent Done false
22662 6844 Acked Read 1 Done Sent Done false
22663 6845 Acked Flush 0 Done Sent Done false
22664 6846 Acked Write 9 Done Sent Done false
22665 6847 Acked Write 4 Done Sent Done false
22666 6848 Acked Read 1 Done Sent Done false
22667 6849 Acked Read 9 Done Sent Done false
22668 6850 Acked Write 9 Done Sent Done false
22669 6851 Acked Write 3 Done Sent Done false
22670 6852 Acked Read 2 Done Sent Done false
22671 6853 Acked Read 2 Done Sent Done false
22672 6854 Acked Read 2 Done Sent Done false
22673 6855 Acked Flush 0 Done Sent Done false
22674 6856 Acked Read 8 Done Sent Done false
22675 6857 Acked Flush 0 Done Sent Done false
22676 6858 Acked Read 5 Done Sent Done false
22677 6859 Acked Read 7 Done Sent Done false
22678 6860 Acked Flush 0 Done Sent Done false
22679 6861 Acked Write 4 Done Sent Done false
22680 6862 Acked Flush 0 Done Sent Done false
22681 6863 Acked Read 4 Done Sent Done false
22682 6864 Acked Read 10 Done Sent Done false
22683 6865 Acked Write 10 Done Sent Done false
22684 6866 Acked Flush 0 Done Sent Done false
22685 6867 Acked Write 5 Done Sent Done false
22686 6868 Acked Write 1 Done Sent Done false
22687 6869 Acked Read 7 Done Sent Done false
22688 6870 Acked Write 10 Done Sent Done false
22689 6871 Acked Write 3 Done Sent Done false
22690 6872 Acked Read 3 Done Sent Done false
22691 6873 Acked Write 6 Done Sent Done false
22692 6874 Acked Read 10 Done Sent Done false
22693 6875 Acked Read 8 Done Sent Done false
22694 6876 Acked Read 1 Done Sent Done false
22695 6877 Acked Write 9 Done Sent Done false
22696 6878 Acked Read 7 Done Sent Done false
22697 6879 Acked Write 3 Done Sent Done false
22698 6880 Acked Read 3 Done Sent Done false
22699 6881 Acked Write 4 Done Sent Done false
22700 6882 Acked Read 5 Done Sent Done false
22701 6883 Acked Write 6 Done Sent Done false
22702 6884 Acked Write 10 Done Sent Done false
22703 6885 Acked Write 7 Done Sent Done false
22704 6886 Acked Read 8 Done Sent Done false
22705 6887 Acked Write 6 Done Sent Done false
22706 6888 Acked Flush 0 Done Sent Done false
22707 6889 Acked Write 2 Done Sent Done false
22708 6890 Acked Write 10 Done Sent Done false
22709 6891 Acked Write 10 Done Sent Done false
22710 6892 Acked Write 7 Done Sent Done false
22711 6893 Acked Read 3 Done Sent Done false
22712 6894 Acked Write 6 Done Sent Done false
22713 6895 Acked Write 9 Done Sent Done false
22714 6896 Acked Read 6 Done Sent Done false
22715 6897 Acked Write 1 Done Sent Done false
22716 6898 Acked Write 3 Done Sent Done false
22717 6899 Acked Write 7 Done Sent Done false
22718 6900 Acked Write 1 Done Sent Done false
22719 6901 Acked Read 5 Done Sent Done false
22720 6902 Acked Write 2 Done Sent Done false
22721 6903 Acked Write 3 Done Sent Done false
22722 6904 Acked Write 6 Done Sent Done false
22723 6905 Acked Read 2 Done Sent Done false
22724 6906 Acked Read 6 Done Sent Done false
22725 6907 Acked Write 8 Done Sent Done false
22726 6908 Acked Write 8 Done Sent Done false
22727 6909 Acked Read 9 Done Sent Done false
22728 6910 Acked Write 1 Done Sent Done false
22729 6911 Acked Read 1 Done Sent Done false
22730 6912 Acked Write 5 Done Sent Done false
22731 6913 Acked Read 7 Done Sent Done false
22732 6914 Acked Write 10 Done Sent Done false
22733 6915 Acked Flush 0 Done Sent Done false
22734 6916 Acked Write 1 Done Sent Done false
22735 6917 Acked Write 6 Done Sent Done false
22736 6918 Acked Write 2 Done Sent Done false
22737 6919 Acked Read 5 Done Sent Done false
22738 6920 Acked Write 4 Done Sent Done false
22739 6921 Acked Read 10 Done Sent Done false
22740 6922 Acked Read 7 Done Sent Done false
22741 6923 Acked Flush 0 Done Sent Done false
22742 6924 Acked Write 10 Done Sent Done false
22743 6925 Acked Flush 0 Done Sent Done false
22744 6926 Acked Read 5 Done Sent Done false
22745 6927 Acked Read 5 Done Sent Done false
22746 6928 Acked Write 9 Done Sent Done false
22747 6929 Acked Flush 0 Done Sent Done false
22748 6930 Acked Read 4 Done Sent Done false
22749 6931 Acked Flush 0 Done Sent Done false
22750 6932 Acked Write 1 Done Sent Done false
22751 6933 Acked Write 10 Done Sent Done false
22752 6934 Acked Read 5 Done Sent Done false
22753 6935 Acked Read 10 Done Sent Done false
22754 6936 Acked Write 1 Done Sent Done false
22755 6937 Acked Write 7 Done Sent Done false
22756 6938 Acked Write 2 Done Sent Done false
22757 6939 Acked Read 6 Done Sent Done false
22758 6940 Acked Flush 0 Done Sent Done false
22759 6941 Acked Write 9 Done Sent Done false
22760 6942 Acked Read 8 Done Sent Done false
22761 6943 Acked Read 9 Done Sent Done false
22762 6944 Acked Write 6 Done Sent Done false
22763 6945 Acked Flush 0 Done Sent Done false
22764 6946 Acked Write 6 Done Sent Done false
22765 6947 Acked Read 1 Done Sent Done false
22766 6948 Acked Read 7 Done Sent Done false
22767 6949 Acked Read 7 Done Sent Done false
22768 6950 Acked Write 10 Done Sent Done false
22769 6951 Acked Write 7 Done Sent Done false
22770 6952 Acked Write 5 Done Sent Done false
22771 6953 Acked Read 5 Done Sent Done false
22772 6954 Acked Read 8 Done Sent Done false
22773 6955 Acked Read 3 Done Sent Done false
22774 6956 Acked Read 8 Done Sent Done false
22775 6957 Acked Flush 0 Done Sent Done false
22776 6958 Acked Write 2 Done Sent Done false
22777 6959 Acked Read 5 Done Sent Done false
22778 6960 Acked Read 7 Done Sent Done false
22779 6961 Acked Read 2 Done Sent Done false
22780 6962 Acked Flush 0 Done Sent Done false
22781 6963 Acked Write 7 Done Sent Done false
22782 6964 Acked Flush 0 Done Sent Done false
22783 6965 Acked Read 2 Done Sent Done false
22784 6966 Acked Read 4 Done Sent Done false
22785 6967 Acked Read 1 Done Sent Done false
22786 6968 Acked Flush 0 Done Sent Done false
22787 6969 Acked Write 1 Done Sent Done false
22788 6970 Acked Flush 0 Done Sent Done false
22789 6971 Acked Read 7 Done Sent Done false
22790 6972 Acked Read 10 Done Sent Done false
22791 6973 Acked Flush 0 Done Sent Done false
22792 6974 Acked Read 6 Done Sent Done false
22793 6975 Acked Write 1 Done Sent Done false
22794 6976 Acked Read 10 Done Sent Done false
22795 6977 Acked Write 3 Done Sent Done false
22796 6978 Acked Write 5 Done Sent Done false
22797 6979 Acked Flush 0 Done Sent Done false
22798 6980 Acked Read 4 Done Sent Done false
22799 6981 Acked Read 6 Done Sent Done false
22800 6982 Acked Read 9 Done Sent Done false
22801 6983 Acked Read 4 Done Sent Done false
22802 6984 Acked Read 4 Done Sent Done false
22803 6985 Acked Read 4 Done Sent Done false
22804 6986 Acked Write 4 Done Sent Done false
22805 6987 Acked Write 4 Done Sent Done false
22806 6988 Acked Write 2 Done Sent Done false
22807 6989 Acked Read 9 Done Sent Done false
22808 6990 Acked Write 6 Done Sent Done false
22809 6991 Acked Write 3 Done Sent Done false
22810 6992 Acked Flush 0 Done Sent Done false
22811 6993 Acked Write 2 Done Sent Done false
22812 6994 Acked Flush 0 Done Sent Done false
22813 6995 Acked Read 8 Done Sent Done false
22814 6996 Acked Write 2 Done Sent Done false
22815 6997 Acked Write 5 Done Sent Done false
22816 6998 Acked Read 9 Done Sent Done false
22817 6999 Acked Write 8 Done Sent Done false
22818 7000 Acked Read 2 Done Sent Done false
22819 7001 Acked Write 5 Done Sent Done false
22820 7002 Acked Read 6 Done Sent Done false
22821 7003 Acked Write 9 Done Sent Done false
22822 7004 Acked Write 9 Done Sent Done false
22823 7005 Acked Flush 0 Done Sent Done false
22824 7006 Acked Write 4 Done Sent Done false
22825 7007 Acked Flush 0 Done Sent Done false
22826 7008 Acked Read 1 Done Sent Done false
22827 7009 Acked Read 2 Done Sent Done false
22828 7010 Acked Read 3 Done Sent Done false
22829 7011 Acked Read 3 Done Sent Done false
22830 7012 Acked Write 1 Done Sent Done false
22831 7013 Acked Read 6 Done Sent Done false
22832 7014 Acked Read 10 Done Sent Done false
22833 7015 Acked Write 2 Done Sent Done false
22834 7016 Acked Write 2 Done Sent Done false
22835 7017 Acked Read 9 Done Sent Done false
22836 7018 Acked Read 7 Done Sent Done false
22837 7019 Acked Write 8 Done Sent Done false
22838 7020 Acked Write 3 Done Sent Done false
22839 7021 Acked Read 8 Done Sent Done false
22840 7022 Acked Read 1 Done Sent Done false
22841 7023 Acked Write 8 Done Sent Done false
22842 7024 Acked Write 7 Done Sent Done false
22843 7025 Acked Flush 0 Done Sent Done false
22844 7026 Acked Write 8 Done Sent Done false
22845 7027 Acked Write 7 Done Sent Done false
22846 7028 Acked Write 1 Done Sent Done false
22847 7029 Acked Write 9 Done Sent Done false
22848 7030 Acked Write 5 Done Sent Done false
22849 7031 Acked Read 4 Done Sent Done false
22850 7032 Acked Read 7 Done Sent Done false
22851 7033 Acked Read 7 Done Sent Done false
22852 7034 Acked Write 2 Done Sent Done false
22853 7035 Acked Write 5 Done Sent Done false
22854 7036 Acked Write 7 Done Sent Done false
22855 7037 Acked Flush 0 Done Sent Done false
22856 STATES DS:0 DS:1 DS:2 TOTAL
22857 Sent 0 302 0 302
22858 Done 302 0 302 604
22859 Skipped 0 0 0 0
22860 Error 0 0 0 0
22861 Last Flush: 7037 6735 7037
22862 Downstairs last five completed: 6735 6734 6733 6732 6731
22863 Upstairs last five completed: 7037 7036 7035 7034 7033
22864 CLIENT: Up:0 ds:302 act:2
228652024-11-19T17:12:46.418ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
228662024-11-19T17:12:46.418ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
228672024-11-19T17:12:46.418ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
228682024-11-19T17:12:46.418ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 05650a15-902e-469c-91e5-cba45f07470e
228692024-11-19T17:12:46.419ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
22870 [8820] got start action
22871 [8820] I am going to restart
22872 Starting downstairs at port 8820
22873 Make output file at "/tmp/dsc/downstairs-8820.txt"
22874 [8820][1] reports Starting
22875 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1222)
22876 [8820][1] reports Running
22877 ----------------------------------------------------------------
22878 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22879 States: Active Offline Active
22880 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22881 6736 Acked Write 1 Done Sent Done false
22882 6737 Acked Write 1 Done Sent Done false
22883 6738 Acked Flush 0 Done Sent Done false
22884 6739 Acked Read 3 Done Sent Done false
22885 6740 Acked Read 10 Done Sent Done false
22886 6741 Acked Write 9 Done Sent Done false
22887 6742 Acked Read 8 Done Sent Done false
22888 6743 Acked Read 2 Done Sent Done false
22889 6744 Acked Write 2 Done Sent Done false
22890 6745 Acked Write 8 Done Sent Done false
22891 6746 Acked Read 7 Done Sent Done false
22892 6747 Acked Read 9 Done Sent Done false
22893 6748 Acked Read 2 Done Sent Done false
22894 6749 Acked Read 9 Done Sent Done false
22895 6750 Acked Write 9 Done Sent Done false
22896 6751 Acked Flush 0 Done Sent Done false
22897 6752 Acked Write 4 Done Sent Done false
22898 6753 Acked Write 5 Done Sent Done false
22899 6754 Acked Read 9 Done Sent Done false
22900 6755 Acked Read 4 Done Sent Done false
22901 6756 Acked Flush 0 Done Sent Done false
22902 6757 Acked Write 5 Done Sent Done false
22903 6758 Acked Write 2 Done Sent Done false
22904 6759 Acked Read 9 Done Sent Done false
22905 6760 Acked Flush 0 Done Sent Done false
22906 6761 Acked Read 2 Done Sent Done false
22907 6762 Acked Write 5 Done Sent Done false
22908 6763 Acked Write 6 Done Sent Done false
22909 6764 Acked Write 7 Done Sent Done false
22910 6765 Acked Flush 0 Done Sent Done false
22911 6766 Acked Write 8 Done Sent Done false
22912 6767 Acked Read 10 Done Sent Done false
22913 6768 Acked Read 6 Done Sent Done false
22914 6769 Acked Read 1 Done Sent Done false
22915 6770 Acked Write 5 Done Sent Done false
22916 6771 Acked Read 7 Done Sent Done false
22917 6772 Acked Read 2 Done Sent Done false
22918 6773 Acked Read 1 Done Sent Done false
22919 6774 Acked Write 1 Done Sent Done false
22920 6775 Acked Read 4 Done Sent Done false
22921 6776 Acked Read 8 Done Sent Done false
22922 6777 Acked Read 3 Done Sent Done false
22923 6778 Acked Write 9 Done Sent Done false
22924 6779 Acked Flush 0 Done Sent Done false
22925 6780 Acked Write 4 Done Sent Done false
22926 6781 Acked Read 6 Done Sent Done false
22927 6782 Acked Read 9 Done Sent Done false
22928 6783 Acked Read 5 Done Sent Done false
22929 6784 Acked Write 9 Done Sent Done false
22930 6785 Acked Read 2 Done Sent Done false
22931 6786 Acked Read 7 Done Sent Done false
22932 6787 Acked Write 1 Done Sent Done false
22933 6788 Acked Read 9 Done Sent Done false
22934 6789 Acked Write 1 Done Sent Done false
22935 6790 Acked Read 10 Done Sent Done false
22936 6791 Acked Read 8 Done Sent Done false
22937 6792 Acked Read 1 Done Sent Done false
22938 6793 Acked Read 8 Done Sent Done false
22939 6794 Acked Write 2 Done Sent Done false
22940 6795 Acked Write 10 Done Sent Done false
22941 6796 Acked Flush 0 Done Sent Done false
22942 6797 Acked Write 9 Done Sent Done false
22943 6798 Acked Read 1 Done Sent Done false
22944 6799 Acked Flush 0 Done Sent Done false
22945 6800 Acked Read 8 Done Sent Done false
22946 6801 Acked Read 1 Done Sent Done false
22947 6802 Acked Read 7 Done Sent Done false
22948 6803 Acked Read 4 Done Sent Done false
22949 6804 Acked Write 5 Done Sent Done false
22950 6805 Acked Write 7 Done Sent Done false
22951 6806 Acked Read 3 Done Sent Done false
22952 6807 Acked Write 5 Done Sent Done false
22953 6808 Acked Read 8 Done Sent Done false
22954 6809 Acked Read 2 Done Sent Done false
22955 6810 Acked Read 10 Done Sent Done false
22956 6811 Acked Read 8 Done Sent Done false
22957 6812 Acked Write 6 Done Sent Done false
22958 6813 Acked Write 3 Done Sent Done false
22959 6814 Acked Write 7 Done Sent Done false
22960 6815 Acked Write 3 Done Sent Done false
22961 6816 Acked Write 8 Done Sent Done false
22962 6817 Acked Read 5 Done Sent Done false
22963 6818 Acked Write 7 Done Sent Done false
22964 6819 Acked Write 9 Done Sent Done false
22965 6820 Acked Read 6 Done Sent Done false
22966 6821 Acked Read 2 Done Sent Done false
22967 6822 Acked Write 1 Done Sent Done false
22968 6823 Acked Flush 0 Done Sent Done false
22969 6824 Acked Read 2 Done Sent Done false
22970 6825 Acked Read 8 Done Sent Done false
22971 6826 Acked Read 1 Done Sent Done false
22972 6827 Acked Write 6 Done Sent Done false
22973 6828 Acked Read 3 Done Sent Done false
22974 6829 Acked Write 8 Done Sent Done false
22975 6830 Acked Read 1 Done Sent Done false
22976 6831 Acked Write 3 Done Sent Done false
22977 6832 Acked Write 5 Done Sent Done false
22978 6833 Acked Read 5 Done Sent Done false
22979 6834 Acked Read 8 Done Sent Done false
22980 6835 Acked Read 3 Done Sent Done false
22981 6836 Acked Flush 0 Done Sent Done false
22982 6837 Acked Read 3 Done Sent Done false
22983 6838 Acked Write 3 Done Sent Done false
22984 6839 Acked Flush 0 Done Sent Done false
22985 6840 Acked Read 2 Done Sent Done false
22986 6841 Acked Read 1 Done Sent Done false
22987 6842 Acked Write 1 Done Sent Done false
22988 6843 Acked Read 4 Done Sent Done false
22989 6844 Acked Read 1 Done Sent Done false
22990 6845 Acked Flush 0 Done Sent Done false
22991 6846 Acked Write 9 Done Sent Done false
22992 6847 Acked Write 4 Done Sent Done false
22993 6848 Acked Read 1 Done Sent Done false
22994 6849 Acked Read 9 Done Sent Done false
22995 6850 Acked Write 9 Done Sent Done false
22996 6851 Acked Write 3 Done Sent Done false
22997 6852 Acked Read 2 Done Sent Done false
22998 6853 Acked Read 2 Done Sent Done false
22999 6854 Acked Read 2 Done Sent Done false
23000 6855 Acked Flush 0 Done Sent Done false
23001 6856 Acked Read 8 Done Sent Done false
23002 6857 Acked Flush 0 Done Sent Done false
23003 6858 Acked Read 5 Done Sent Done false
23004 6859 Acked Read 7 Done Sent Done false
23005 6860 Acked Flush 0 Done Sent Done false
23006 6861 Acked Write 4 Done Sent Done false
23007 6862 Acked Flush 0 Done Sent Done false
23008 6863 Acked Read 4 Done Sent Done false
23009 6864 Acked Read 10 Done Sent Done false
23010 6865 Acked Write 10 Done Sent Done false
23011 6866 Acked Flush 0 Done Sent Done false
23012 6867 Acked Write 5 Done Sent Done false
23013 6868 Acked Write 1 Done Sent Done false
23014 6869 Acked Read 7 Done Sent Done false
23015 6870 Acked Write 10 Done Sent Done false
23016 6871 Acked Write 3 Done Sent Done false
23017 6872 Acked Read 3 Done Sent Done false
23018 6873 Acked Write 6 Done Sent Done false
23019 6874 Acked Read 10 Done Sent Done false
23020 6875 Acked Read 8 Done Sent Done false
23021 6876 Acked Read 1 Done Sent Done false
23022 6877 Acked Write 9 Done Sent Done false
23023 6878 Acked Read 7 Done Sent Done false
23024 6879 Acked Write 3 Done Sent Done false
23025 6880 Acked Read 3 Done Sent Done false
23026 6881 Acked Write 4 Done Sent Done false
23027 6882 Acked Read 5 Done Sent Done false
23028 6883 Acked Write 6 Done Sent Done false
23029 6884 Acked Write 10 Done Sent Done false
23030 6885 Acked Write 7 Done Sent Done false
23031 6886 Acked Read 8 Done Sent Done false
23032 6887 Acked Write 6 Done Sent Done false
23033 6888 Acked Flush 0 Done Sent Done false
23034 6889 Acked Write 2 Done Sent Done false
23035 6890 Acked Write 10 Done Sent Done false
23036 6891 Acked Write 10 Done Sent Done false
23037 6892 Acked Write 7 Done Sent Done false
23038 6893 Acked Read 3 Done Sent Done false
23039 6894 Acked Write 6 Done Sent Done false
23040 6895 Acked Write 9 Done Sent Done false
23041 6896 Acked Read 6 Done Sent Done false
23042 6897 Acked Write 1 Done Sent Done false
23043 6898 Acked Write 3 Done Sent Done false
23044 6899 Acked Write 7 Done Sent Done false
23045 6900 Acked Write 1 Done Sent Done false
23046 6901 Acked Read 5 Done Sent Done false
23047 6902 Acked Write 2 Done Sent Done false
23048 6903 Acked Write 3 Done Sent Done false
23049 6904 Acked Write 6 Done Sent Done false
23050 6905 Acked Read 2 Done Sent Done false
23051 6906 Acked Read 6 Done Sent Done false
23052 6907 Acked Write 8 Done Sent Done false
23053 6908 Acked Write 8 Done Sent Done false
23054 6909 Acked Read 9 Done Sent Done false
23055 6910 Acked Write 1 Done Sent Done false
23056 6911 Acked Read 1 Done Sent Done false
23057 6912 Acked Write 5 Done Sent Done false
23058 6913 Acked Read 7 Done Sent Done false
23059 6914 Acked Write 10 Done Sent Done false
23060 6915 Acked Flush 0 Done Sent Done false
23061 6916 Acked Write 1 Done Sent Done false
23062 6917 Acked Write 6 Done Sent Done false
23063 6918 Acked Write 2 Done Sent Done false
23064 6919 Acked Read 5 Done Sent Done false
23065 6920 Acked Write 4 Done Sent Done false
23066 6921 Acked Read 10 Done Sent Done false
23067 6922 Acked Read 7 Done Sent Done false
23068 6923 Acked Flush 0 Done Sent Done false
23069 6924 Acked Write 10 Done Sent Done false
23070 6925 Acked Flush 0 Done Sent Done false
23071 6926 Acked Read 5 Done Sent Done false
23072 6927 Acked Read 5 Done Sent Done false
23073 6928 Acked Write 9 Done Sent Done false
23074 6929 Acked Flush 0 Done Sent Done false
23075 6930 Acked Read 4 Done Sent Done false
23076 6931 Acked Flush 0 Done Sent Done false
23077 6932 Acked Write 1 Done Sent Done false
23078 6933 Acked Write 10 Done Sent Done false
23079 6934 Acked Read 5 Done Sent Done false
23080 6935 Acked Read 10 Done Sent Done false
23081 6936 Acked Write 1 Done Sent Done false
23082 6937 Acked Write 7 Done Sent Done false
23083 6938 Acked Write 2 Done Sent Done false
23084 6939 Acked Read 6 Done Sent Done false
23085 6940 Acked Flush 0 Done Sent Done false
23086 6941 Acked Write 9 Done Sent Done false
23087 6942 Acked Read 8 Done Sent Done false
23088 6943 Acked Read 9 Done Sent Done false
23089 6944 Acked Write 6 Done Sent Done false
23090 6945 Acked Flush 0 Done Sent Done false
23091 6946 Acked Write 6 Done Sent Done false
23092 6947 Acked Read 1 Done Sent Done false
23093 6948 Acked Read 7 Done Sent Done false
23094 6949 Acked Read 7 Done Sent Done false
23095 6950 Acked Write 10 Done Sent Done false
23096 6951 Acked Write 7 Done Sent Done false
23097 6952 Acked Write 5 Done Sent Done false
23098 6953 Acked Read 5 Done Sent Done false
23099 6954 Acked Read 8 Done Sent Done false
23100 6955 Acked Read 3 Done Sent Done false
23101 6956 Acked Read 8 Done Sent Done false
23102 6957 Acked Flush 0 Done Sent Done false
23103 6958 Acked Write 2 Done Sent Done false
23104 6959 Acked Read 5 Done Sent Done false
23105 6960 Acked Read 7 Done Sent Done false
23106 6961 Acked Read 2 Done Sent Done false
23107 6962 Acked Flush 0 Done Sent Done false
23108 6963 Acked Write 7 Done Sent Done false
23109 6964 Acked Flush 0 Done Sent Done false
23110 6965 Acked Read 2 Done Sent Done false
23111 6966 Acked Read 4 Done Sent Done false
23112 6967 Acked Read 1 Done Sent Done false
23113 6968 Acked Flush 0 Done Sent Done false
23114 6969 Acked Write 1 Done Sent Done false
23115 6970 Acked Flush 0 Done Sent Done false
23116 6971 Acked Read 7 Done Sent Done false
23117 6972 Acked Read 10 Done Sent Done false
23118 6973 Acked Flush 0 Done Sent Done false
23119 6974 Acked Read 6 Done Sent Done false
23120 6975 Acked Write 1 Done Sent Done false
23121 6976 Acked Read 10 Done Sent Done false
23122 6977 Acked Write 3 Done Sent Done false
23123 6978 Acked Write 5 Done Sent Done false
23124 6979 Acked Flush 0 Done Sent Done false
23125 6980 Acked Read 4 Done Sent Done false
23126 6981 Acked Read 6 Done Sent Done false
23127 6982 Acked Read 9 Done Sent Done false
23128 6983 Acked Read 4 Done Sent Done false
23129 6984 Acked Read 4 Done Sent Done false
23130 6985 Acked Read 4 Done Sent Done false
23131 6986 Acked Write 4 Done Sent Done false
23132 6987 Acked Write 4 Done Sent Done false
23133 6988 Acked Write 2 Done Sent Done false
23134 6989 Acked Read 9 Done Sent Done false
23135 6990 Acked Write 6 Done Sent Done false
23136 6991 Acked Write 3 Done Sent Done false
23137 6992 Acked Flush 0 Done Sent Done false
23138 6993 Acked Write 2 Done Sent Done false
23139 6994 Acked Flush 0 Done Sent Done false
23140 6995 Acked Read 8 Done Sent Done false
23141 6996 Acked Write 2 Done Sent Done false
23142 6997 Acked Write 5 Done Sent Done false
23143 6998 Acked Read 9 Done Sent Done false
23144 6999 Acked Write 8 Done Sent Done false
23145 7000 Acked Read 2 Done Sent Done false
23146 7001 Acked Write 5 Done Sent Done false
23147 7002 Acked Read 6 Done Sent Done false
23148 7003 Acked Write 9 Done Sent Done false
23149 7004 Acked Write 9 Done Sent Done false
23150 7005 Acked Flush 0 Done Sent Done false
23151 7006 Acked Write 4 Done Sent Done false
23152 7007 Acked Flush 0 Done Sent Done false
23153 7008 Acked Read 1 Done Sent Done false
23154 7009 Acked Read 2 Done Sent Done false
23155 7010 Acked Read 3 Done Sent Done false
23156 7011 Acked Read 3 Done Sent Done false
23157 7012 Acked Write 1 Done Sent Done false
23158 7013 Acked Read 6 Done Sent Done false
23159 7014 Acked Read 10 Done Sent Done false
23160 7015 Acked Write 2 Done Sent Done false
23161 7016 Acked Write 2 Done Sent Done false
23162 7017 Acked Read 9 Done Sent Done false
23163 7018 Acked Read 7 Done Sent Done false
23164 7019 Acked Write 8 Done Sent Done false
23165 7020 Acked Write 3 Done Sent Done false
23166 7021 Acked Read 8 Done Sent Done false
23167 7022 Acked Read 1 Done Sent Done false
23168 7023 Acked Write 8 Done Sent Done false
23169 7024 Acked Write 7 Done Sent Done false
23170 7025 Acked Flush 0 Done Sent Done false
23171 7026 Acked Write 8 Done Sent Done false
23172 7027 Acked Write 7 Done Sent Done false
23173 7028 Acked Write 1 Done Sent Done false
23174 7029 Acked Write 9 Done Sent Done false
23175 7030 Acked Write 5 Done Sent Done false
23176 7031 Acked Read 4 Done Sent Done false
23177 7032 Acked Read 7 Done Sent Done false
23178 7033 Acked Read 7 Done Sent Done false
23179 7034 Acked Write 2 Done Sent Done false
23180 7035 Acked Write 5 Done Sent Done false
23181 7036 Acked Write 7 Done Sent Done false
23182 7037 Acked Flush 0 Done Sent Done false
23183 STATES DS:0 DS:1 DS:2 TOTAL
23184 Sent 0 302 0 302
23185 Done 302 0 302 604
23186 Skipped 0 0 0 0
23187 Error 0 0 0 0
23188 Last Flush: 7037 6735 7037
23189 Downstairs last five completed: 6735 6734 6733 6732 6731
23190 Upstairs last five completed: 7037 7036 7035 7034 7033
23191 CLIENT: Up:0 ds:302 act:2
23192 ----------------------------------------------------------------
23193 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23194 States: Active Offline Active
23195 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23196 6736 Acked Write 1 Done Sent Done false
23197 6737 Acked Write 1 Done Sent Done false
23198 6738 Acked Flush 0 Done Sent Done false
23199 6739 Acked Read 3 Done Sent Done false
23200 6740 Acked Read 10 Done Sent Done false
23201 6741 Acked Write 9 Done Sent Done false
23202 6742 Acked Read 8 Done Sent Done false
23203 6743 Acked Read 2 Done Sent Done false
23204 6744 Acked Write 2 Done Sent Done false
23205 6745 Acked Write 8 Done Sent Done false
23206 6746 Acked Read 7 Done Sent Done false
23207 6747 Acked Read 9 Done Sent Done false
23208 6748 Acked Read 2 Done Sent Done false
23209 6749 Acked Read 9 Done Sent Done false
23210 6750 Acked Write 9 Done Sent Done false
23211 6751 Acked Flush 0 Done Sent Done false
23212 6752 Acked Write 4 Done Sent Done false
23213 6753 Acked Write 5 Done Sent Done false
23214 6754 Acked Read 9 Done Sent Done false
23215 6755 Acked Read 4 Done Sent Done false
23216 6756 Acked Flush 0 Done Sent Done false
23217 6757 Acked Write 5 Done Sent Done false
23218 6758 Acked Write 2 Done Sent Done false
23219 6759 Acked Read 9 Done Sent Done false
23220 6760 Acked Flush 0 Done Sent Done false
23221 6761 Acked Read 2 Done Sent Done false
23222 6762 Acked Write 5 Done Sent Done false
23223 6763 Acked Write 6 Done Sent Done false
23224 6764 Acked Write 7 Done Sent Done false
23225 6765 Acked Flush 0 Done Sent Done false
23226 6766 Acked Write 8 Done Sent Done false
23227 6767 Acked Read 10 Done Sent Done false
23228 6768 Acked Read 6 Done Sent Done false
23229 6769 Acked Read 1 Done Sent Done false
23230 6770 Acked Write 5 Done Sent Done false
23231 6771 Acked Read 7 Done Sent Done false
23232 6772 Acked Read 2 Done Sent Done false
23233 6773 Acked Read 1 Done Sent Done false
23234 6774 Acked Write 1 Done Sent Done false
23235 6775 Acked Read 4 Done Sent Done false
23236 6776 Acked Read 8 Done Sent Done false
23237 6777 Acked Read 3 Done Sent Done false
23238 6778 Acked Write 9 Done Sent Done false
23239 6779 Acked Flush 0 Done Sent Done false
23240 6780 Acked Write 4 Done Sent Done false
23241 6781 Acked Read 6 Done Sent Done false
23242 6782 Acked Read 9 Done Sent Done false
23243 6783 Acked Read 5 Done Sent Done false
23244 6784 Acked Write 9 Done Sent Done false
23245 6785 Acked Read 2 Done Sent Done false
23246 6786 Acked Read 7 Done Sent Done false
23247 6787 Acked Write 1 Done Sent Done false
23248 6788 Acked Read 9 Done Sent Done false
23249 6789 Acked Write 1 Done Sent Done false
23250 6790 Acked Read 10 Done Sent Done false
23251 6791 Acked Read 8 Done Sent Done false
23252 6792 Acked Read 1 Done Sent Done false
23253 6793 Acked Read 8 Done Sent Done false
23254 6794 Acked Write 2 Done Sent Done false
23255 6795 Acked Write 10 Done Sent Done false
23256 6796 Acked Flush 0 Done Sent Done false
23257 6797 Acked Write 9 Done Sent Done false
23258 6798 Acked Read 1 Done Sent Done false
23259 6799 Acked Flush 0 Done Sent Done false
23260 6800 Acked Read 8 Done Sent Done false
23261 6801 Acked Read 1 Done Sent Done false
23262 6802 Acked Read 7 Done Sent Done false
23263 6803 Acked Read 4 Done Sent Done false
23264 6804 Acked Write 5 Done Sent Done false
23265 6805 Acked Write 7 Done Sent Done false
23266 6806 Acked Read 3 Done Sent Done false
23267 6807 Acked Write 5 Done Sent Done false
23268 6808 Acked Read 8 Done Sent Done false
23269 6809 Acked Read 2 Done Sent Done false
23270 6810 Acked Read 10 Done Sent Done false
23271 6811 Acked Read 8 Done Sent Done false
23272 6812 Acked Write 6 Done Sent Done false
23273 6813 Acked Write 3 Done Sent Done false
23274 6814 Acked Write 7 Done Sent Done false
23275 6815 Acked Write 3 Done Sent Done false
23276 6816 Acked Write 8 Done Sent Done false
23277 6817 Acked Read 5 Done Sent Done false
23278 6818 Acked Write 7 Done Sent Done false
23279 6819 Acked Write 9 Done Sent Done false
23280 6820 Acked Read 6 Done Sent Done false
23281 6821 Acked Read 2 Done Sent Done false
23282 6822 Acked Write 1 Done Sent Done false
23283 6823 Acked Flush 0 Done Sent Done false
23284 6824 Acked Read 2 Done Sent Done false
23285 6825 Acked Read 8 Done Sent Done false
23286 6826 Acked Read 1 Done Sent Done false
23287 6827 Acked Write 6 Done Sent Done false
23288 6828 Acked Read 3 Done Sent Done false
23289 6829 Acked Write 8 Done Sent Done false
23290 6830 Acked Read 1 Done Sent Done false
23291 6831 Acked Write 3 Done Sent Done false
23292 6832 Acked Write 5 Done Sent Done false
23293 6833 Acked Read 5 Done Sent Done false
23294 6834 Acked Read 8 Done Sent Done false
23295 6835 Acked Read 3 Done Sent Done false
23296 6836 Acked Flush 0 Done Sent Done false
23297 6837 Acked Read 3 Done Sent Done false
23298 6838 Acked Write 3 Done Sent Done false
23299 6839 Acked Flush 0 Done Sent Done false
23300 6840 Acked Read 2 Done Sent Done false
23301 6841 Acked Read 1 Done Sent Done false
23302 6842 Acked Write 1 Done Sent Done false
23303 6843 Acked Read 4 Done Sent Done false
23304 6844 Acked Read 1 Done Sent Done false
23305 6845 Acked Flush 0 Done Sent Done false
23306 6846 Acked Write 9 Done Sent Done false
23307 6847 Acked Write 4 Done Sent Done false
23308 6848 Acked Read 1 Done Sent Done false
23309 6849 Acked Read 9 Done Sent Done false
23310 6850 Acked Write 9 Done Sent Done false
23311 6851 Acked Write 3 Done Sent Done false
23312 6852 Acked Read 2 Done Sent Done false
23313 6853 Acked Read 2 Done Sent Done false
23314 6854 Acked Read 2 Done Sent Done false
23315 6855 Acked Flush 0 Done Sent Done false
23316 6856 Acked Read 8 Done Sent Done false
23317 6857 Acked Flush 0 Done Sent Done false
23318 6858 Acked Read 5 Done Sent Done false
23319 6859 Acked Read 7 Done Sent Done false
23320 6860 Acked Flush 0 Done Sent Done false
23321 6861 Acked Write 4 Done Sent Done false
23322 6862 Acked Flush 0 Done Sent Done false
23323 6863 Acked Read 4 Done Sent Done false
23324 6864 Acked Read 10 Done Sent Done false
23325 6865 Acked Write 10 Done Sent Done false
23326 6866 Acked Flush 0 Done Sent Done false
23327 6867 Acked Write 5 Done Sent Done false
23328 6868 Acked Write 1 Done Sent Done false
23329 6869 Acked Read 7 Done Sent Done false
23330 6870 Acked Write 10 Done Sent Done false
23331 6871 Acked Write 3 Done Sent Done false
23332 6872 Acked Read 3 Done Sent Done false
23333 6873 Acked Write 6 Done Sent Done false
23334 6874 Acked Read 10 Done Sent Done false
23335 6875 Acked Read 8 Done Sent Done false
23336 6876 Acked Read 1 Done Sent Done false
23337 6877 Acked Write 9 Done Sent Done false
23338 6878 Acked Read 7 Done Sent Done false
23339 6879 Acked Write 3 Done Sent Done false
23340 6880 Acked Read 3 Done Sent Done false
23341 6881 Acked Write 4 Done Sent Done false
23342 6882 Acked Read 5 Done Sent Done false
23343 6883 Acked Write 6 Done Sent Done false
23344 6884 Acked Write 10 Done Sent Done false
23345 6885 Acked Write 7 Done Sent Done false
23346 6886 Acked Read 8 Done Sent Done false
23347 6887 Acked Write 6 Done Sent Done false
23348 6888 Acked Flush 0 Done Sent Done false
23349 6889 Acked Write 2 Done Sent Done false
23350 6890 Acked Write 10 Done Sent Done false
23351 6891 Acked Write 10 Done Sent Done false
23352 6892 Acked Write 7 Done Sent Done false
23353 6893 Acked Read 3 Done Sent Done false
23354 6894 Acked Write 6 Done Sent Done false
23355 6895 Acked Write 9 Done Sent Done false
23356 6896 Acked Read 6 Done Sent Done false
23357 6897 Acked Write 1 Done Sent Done false
23358 6898 Acked Write 3 Done Sent Done false
23359 6899 Acked Write 7 Done Sent Done false
23360 6900 Acked Write 1 Done Sent Done false
23361 6901 Acked Read 5 Done Sent Done false
23362 6902 Acked Write 2 Done Sent Done false
23363 6903 Acked Write 3 Done Sent Done false
23364 6904 Acked Write 6 Done Sent Done false
23365 6905 Acked Read 2 Done Sent Done false
23366 6906 Acked Read 6 Done Sent Done false
23367 6907 Acked Write 8 Done Sent Done false
23368 6908 Acked Write 8 Done Sent Done false
23369 6909 Acked Read 9 Done Sent Done false
23370 6910 Acked Write 1 Done Sent Done false
23371 6911 Acked Read 1 Done Sent Done false
23372 6912 Acked Write 5 Done Sent Done false
23373 6913 Acked Read 7 Done Sent Done false
23374 6914 Acked Write 10 Done Sent Done false
23375 6915 Acked Flush 0 Done Sent Done false
23376 6916 Acked Write 1 Done Sent Done false
23377 6917 Acked Write 6 Done Sent Done false
23378 6918 Acked Write 2 Done Sent Done false
23379 6919 Acked Read 5 Done Sent Done false
23380 6920 Acked Write 4 Done Sent Done false
23381 6921 Acked Read 10 Done Sent Done false
23382 6922 Acked Read 7 Done Sent Done false
23383 6923 Acked Flush 0 Done Sent Done false
23384 6924 Acked Write 10 Done Sent Done false
23385 6925 Acked Flush 0 Done Sent Done false
23386 6926 Acked Read 5 Done Sent Done false
23387 6927 Acked Read 5 Done Sent Done false
23388 6928 Acked Write 9 Done Sent Done false
23389 6929 Acked Flush 0 Done Sent Done false
23390 6930 Acked Read 4 Done Sent Done false
23391 6931 Acked Flush 0 Done Sent Done false
23392 6932 Acked Write 1 Done Sent Done false
23393 6933 Acked Write 10 Done Sent Done false
23394 6934 Acked Read 5 Done Sent Done false
23395 6935 Acked Read 10 Done Sent Done false
23396 6936 Acked Write 1 Done Sent Done false
23397 6937 Acked Write 7 Done Sent Done false
23398 6938 Acked Write 2 Done Sent Done false
23399 6939 Acked Read 6 Done Sent Done false
23400 6940 Acked Flush 0 Done Sent Done false
23401 6941 Acked Write 9 Done Sent Done false
23402 6942 Acked Read 8 Done Sent Done false
23403 6943 Acked Read 9 Done Sent Done false
23404 6944 Acked Write 6 Done Sent Done false
23405 6945 Acked Flush 0 Done Sent Done false
23406 6946 Acked Write 6 Done Sent Done false
23407 6947 Acked Read 1 Done Sent Done false
23408 6948 Acked Read 7 Done Sent Done false
23409 6949 Acked Read 7 Done Sent Done false
23410 6950 Acked Write 10 Done Sent Done false
23411 6951 Acked Write 7 Done Sent Done false
23412 6952 Acked Write 5 Done Sent Done false
23413 6953 Acked Read 5 Done Sent Done false
23414 6954 Acked Read 8 Done Sent Done false
23415 6955 Acked Read 3 Done Sent Done false
23416 6956 Acked Read 8 Done Sent Done false
23417 6957 Acked Flush 0 Done Sent Done false
23418 6958 Acked Write 2 Done Sent Done false
23419 6959 Acked Read 5 Done Sent Done false
23420 6960 Acked Read 7 Done Sent Done false
23421 6961 Acked Read 2 Done Sent Done false
23422 6962 Acked Flush 0 Done Sent Done false
23423 6963 Acked Write 7 Done Sent Done false
23424 6964 Acked Flush 0 Done Sent Done false
23425 6965 Acked Read 2 Done Sent Done false
23426 6966 Acked Read 4 Done Sent Done false
23427 6967 Acked Read 1 Done Sent Done false
23428 6968 Acked Flush 0 Done Sent Done false
23429 6969 Acked Write 1 Done Sent Done false
23430 6970 Acked Flush 0 Done Sent Done false
23431 6971 Acked Read 7 Done Sent Done false
23432 6972 Acked Read 10 Done Sent Done false
23433 6973 Acked Flush 0 Done Sent Done false
23434 6974 Acked Read 6 Done Sent Done false
23435 6975 Acked Write 1 Done Sent Done false
23436 6976 Acked Read 10 Done Sent Done false
23437 6977 Acked Write 3 Done Sent Done false
23438 6978 Acked Write 5 Done Sent Done false
23439 6979 Acked Flush 0 Done Sent Done false
23440 6980 Acked Read 4 Done Sent Done false
23441 6981 Acked Read 6 Done Sent Done false
23442 6982 Acked Read 9 Done Sent Done false
23443 6983 Acked Read 4 Done Sent Done false
23444 6984 Acked Read 4 Done Sent Done false
23445 6985 Acked Read 4 Done Sent Done false
23446 6986 Acked Write 4 Done Sent Done false
23447 6987 Acked Write 4 Done Sent Done false
23448 6988 Acked Write 2 Done Sent Done false
23449 6989 Acked Read 9 Done Sent Done false
23450 6990 Acked Write 6 Done Sent Done false
23451 6991 Acked Write 3 Done Sent Done false
23452 6992 Acked Flush 0 Done Sent Done false
23453 6993 Acked Write 2 Done Sent Done false
23454 6994 Acked Flush 0 Done Sent Done false
23455 6995 Acked Read 8 Done Sent Done false
23456 6996 Acked Write 2 Done Sent Done false
23457 6997 Acked Write 5 Done Sent Done false
23458 6998 Acked Read 9 Done Sent Done false
23459 6999 Acked Write 8 Done Sent Done false
23460 7000 Acked Read 2 Done Sent Done false
23461 7001 Acked Write 5 Done Sent Done false
23462 7002 Acked Read 6 Done Sent Done false
23463 7003 Acked Write 9 Done Sent Done false
23464 7004 Acked Write 9 Done Sent Done false
23465 7005 Acked Flush 0 Done Sent Done false
23466 7006 Acked Write 4 Done Sent Done false
23467 7007 Acked Flush 0 Done Sent Done false
23468 7008 Acked Read 1 Done Sent Done false
23469 7009 Acked Read 2 Done Sent Done false
23470 7010 Acked Read 3 Done Sent Done false
23471 7011 Acked Read 3 Done Sent Done false
23472 7012 Acked Write 1 Done Sent Done false
23473 7013 Acked Read 6 Done Sent Done false
23474 7014 Acked Read 10 Done Sent Done false
23475 7015 Acked Write 2 Done Sent Done false
23476 7016 Acked Write 2 Done Sent Done false
23477 7017 Acked Read 9 Done Sent Done false
23478 7018 Acked Read 7 Done Sent Done false
23479 7019 Acked Write 8 Done Sent Done false
23480 7020 Acked Write 3 Done Sent Done false
23481 7021 Acked Read 8 Done Sent Done false
23482 7022 Acked Read 1 Done Sent Done false
23483 7023 Acked Write 8 Done Sent Done false
23484 7024 Acked Write 7 Done Sent Done false
23485 7025 Acked Flush 0 Done Sent Done false
23486 7026 Acked Write 8 Done Sent Done false
23487 7027 Acked Write 7 Done Sent Done false
23488 7028 Acked Write 1 Done Sent Done false
23489 7029 Acked Write 9 Done Sent Done false
23490 7030 Acked Write 5 Done Sent Done false
23491 7031 Acked Read 4 Done Sent Done false
23492 7032 Acked Read 7 Done Sent Done false
23493 7033 Acked Read 7 Done Sent Done false
23494 7034 Acked Write 2 Done Sent Done false
23495 7035 Acked Write 5 Done Sent Done false
23496 7036 Acked Write 7 Done Sent Done false
23497 7037 Acked Flush 0 Done Sent Done false
23498 STATES DS:0 DS:1 DS:2 TOTAL
23499 Sent 0 302 0 302
23500 Done 302 0 302 604
23501 Skipped 0 0 0 0
23502 Error 0 0 0 0
23503 Last Flush: 7037 6735 7037
23504 Downstairs last five completed: 6735 6734 6733 6732 6731
23505 Upstairs last five completed: 7037 7036 7035 7034 7033
23506 CLIENT: Up:0 ds:302 act:2
235072024-11-19T17:12:56.419ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235082024-11-19T17:12:56.420ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60549) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235092024-11-19T17:12:56.421ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235102024-11-19T17:12:56.421ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235112024-11-19T17:12:56.421ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235122024-11-19T17:12:56.421ZINFOcrucible: send last flush ID to this DS: 6735 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235132024-11-19T17:12:56.421ZINFOcrucible: Replied this last flush ID: 6735 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235142024-11-19T17:12:56.422ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235152024-11-19T17:12:56.422ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235162024-11-19T17:12:56.422ZINFOcrucible: [1] client re-new 302 jobs since flush 6735 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
235172024-11-19T17:12:56.422ZINFOcrucible: [1] Replayed 302 jobs since flush: 6735 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
23518 ----------------------------------------------------------------
23519 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23520 Downstairs last five completed: 7037 7036 7035 7034 7033
23521 Upstairs last five completed: 7037 7036 7035 7034 7033
23522 CLIENT: Up:0 ds:0 act:3
23523 Replay: All jobs finished
23524 Main task has work to do, go find it
23525 got dsc Stop(2)
23526 stop 2
23527 Nov 19 17:12:56.653 [8830] Got stop action so:false kr:false
23528 INFO request completed, latency_us: 323, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 3ab10327-29ab-47f8-b637-2d56283a9ca7, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
23529 Nov 19 17:12:56.654{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:56.654470912Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
23530 INFO request completed, latency_us: 136, response_code: 200, uri: /state/cid/2, method: GET, req_id: 11003070-fa6c-4098-a345-6fa4c4c7e105, remote_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:56.654585902Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
23531 .0.0.1:64175, local_addr: 127{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:12:56.654630238Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
23532 .0.0.1:9998{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:12:56.654658185Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
23533 
235342024-11-19T17:12:56.654ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
235352024-11-19T17:12:56.654ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
23536 [8830] Exited with: ExitStatus(unix_wait_status(9))
23537 [8830][2] reports Exit
23538 Nov 19 17:13:00.656 INFO request completed, latency_us: 198, response_code: 200, uri: /state/cid/2, method: GET, req_id: 806a2f8d-1695-4720-817b-79f0fd438f20, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
23539 001/300 Read block 444 len 8192
23540 002/300 Read block 1610 len 24576
23541 003/300 Read block 3981 len 12288
23542 004/300 Write block 3759 len 12288 data: 7 6 6
23543 005/300 Read block 1522 len 16384
23544 006/300 Read block 4747 len 28672
23545 007/300 Flush
23546 008/300 Read block 3158 len 36864
23547 009/300 Read block 173 len 36864
23548 010/300 Read block 3326 len 36864
23549 011/300 Read block 3277 len 40960
23550 012/300 Read block 346 len 32768
23551 013/300 Read block 483 len 8192
23552 014/300 Read block 1706 len 36864
23553 015/300 Flush
23554 016/300 Read block 2561 len 36864
23555 017/300 Read block 1868 len 4096
23556 018/300 Write block 4839 len 24576 data: 4 3 3 3 4 4
23557 019/300 Read block 1058 len 28672
23558 020/300 Write block 3765 len 16384 data: 6 4 4 4
23559 021/300 Write block 231 len 32768 data: 2 3 4 4 3 3 3 3
23560 022/300 Write block 1307 len 8192 data: 5 5
23561 023/300 Write block 1649 len 24576 data: 5 5 5 4 3 3
23562 024/300 Write block 2779 len 4096 data: 4
23563 025/300 Write block 4748 len 20480 data: 3 3 5 4 4
23564 026/300 Write block 2294 len 28672 data: 5 5 7 7 7 7 8
23565 027/300 Read block 1559 len 32768
23566 028/300 Read block 1777 len 28672
23567 029/300 Read block 1106 len 28672
23568 030/300 Read block 506 len 28672
23569 031/300 Write block 2922 len 16384 data: 6 6 4 3
23570 032/300 Flush
23571 033/300 Read block 3381 len 32768
23572 034/300 Read block 3533 len 36864
23573 035/300 Write block 4785 len 8192 data: 3 4
23574 036/300 Read block 584 len 8192
23575 037/300 Read block 3832 len 16384
23576 038/300 Read block 4176 len 24576
23577 039/300 Write block 4480 len 4096 data: 6
23578 040/300 Read block 4969 len 40960
23579 041/300 Write block 1244 len 4096 data: 3
23580 042/300 Write block 2050 len 4096 data: 3
23581 043/300 Read block 4078 len 4096
23582 044/300 Flush
23583 045/300 Write block 3086 len 28672 data: 3 3 3 3 2 2 2
23584 046/300 Read block 161 len 8192
23585 047/300 Read block 3902 len 4096
23586 048/300 Read block 836 len 32768
23587 049/300 Read block 2490 len 8192
23588 050/300 Write block 433 len 32768 data: 4 4 3 3 2 2 2 3
23589 051/300 Read block 1110 len 36864
23590 052/300 Read block 2460 len 36864
23591 053/300 Write block 3259 len 40960 data: 2 2 3 4 4 4 5 4 5 5
23592 054/300 Write block 1687 len 20480 data: 3 4 5 4 5
23593 055/300 Write block 515 len 16384 data: 3 4 5 4
23594 056/300 Read block 3236 len 20480
23595 057/300 Read block 2850 len 36864
23596 058/300 Flush
23597 059/300 Read block 3731 len 24576
23598 060/300 Read block 2109 len 8192
23599 061/300 Read block 2312 len 12288
23600 062/300 Write block 4146 len 8192 data: 5 4
23601 063/300 Read block 4405 len 40960
23602 064/300 Read block 1066 len 20480
23603 065/300 Write block 2266 len 24576 data: 4 4 4 4 4 7
23604 066/300 Write block 2586 len 40960 data: 4 5 6 4 4 5 5 4 4 3
23605 067/300 Read block 3886 len 40960
23606 068/300 Read block 4013 len 8192
23607 069/300 Read block 2464 len 16384
23608 070/300 Read block 2414 len 28672
23609 071/300 Flush
23610 072/300 Flush
23611 073/300 Write block 1041 len 40960 data: 5 5 5 5 7 7 6 6 5 5
23612 074/300 Write block 4562 len 4096 data: 5
23613 075/300 Write block 4864 len 40960 data: 4 4 4 4 4 3 5 5 6 6
23614 076/300 Read block 2400 len 24576
23615 077/300 Flush
23616 078/300 Read block 3334 len 28672
23617 079/300 Read block 53 len 12288
23618 080/300 Read block 639 len 12288
23619 081/300 Read block 1488 len 32768
23620 082/300 Write block 3422 len 4096 data: 2
23621 083/300 Write block 4527 len 32768 data: 5 7 8 8 8 6 6 6
23622 084/300 Write block 2426 len 40960 data: 6 5 4 4 3 3 4 5 7 4
23623 085/300 Read block 2213 len 20480
23624 086/300 Write block 734 len 16384 data: 3 3 3 3
23625 087/300 Read block 3769 len 20480
23626 088/300 Read block 4706 len 16384
23627 089/300 Read block 3872 len 40960
23628 090/300 Write block 3770 len 36864 data: 3 3 3 4 4 3 5 5 5
23629 091/300 Write block 4972 len 16384 data: 4 5 4 4
23630 092/300 Read block 223 len 4096
23631 093/300 Read block 419 len 28672
23632 094/300 Read block 4470 len 36864
23633 095/300 Read block 1738 len 8192
23634 096/300 Write block 4652 len 12288 data: 6 4 3
23635 097/300 Read block 1840 len 28672
23636 098/300 Read block 2711 len 36864
23637 099/300 Write block 2508 len 24576 data: 6 6 4 4 4 2
23638 100/300 Read block 712 len 36864
23639 101/300 Read block 1824 len 28672
23640 102/300 Read block 79 len 24576
23641 103/300 Read block 524 len 32768
23642 104/300 Write block 1960 len 12288 data: 3 2 2
23643 105/300 Read block 2609 len 36864
23644 106/300 Read block 3591 len 8192
23645 107/300 Write block 3601 len 24576 data: 5 4 3 3 3 2
23646 108/300 Write block 4767 len 20480 data: 5 4 5 5 4
23647 109/300 Write block 3253 len 28672 data: 4 3 2 2 3 3 3
23648 110/300 Write block 3060 len 24576 data: 5 5 4 5 4 3
23649 111/300 Read block 1200 len 32768
23650 112/300 Write block 1720 len 32768 data: 5 4 5 5 5 5 4 4
23651 113/300 Write block 3217 len 8192 data: 4 6
23652 114/300 Read block 635 len 16384
23653 115/300 Read block 2714 len 16384
23654 116/300 Flush
23655 117/300 Write block 3996 len 28672 data: 3 5 5 5 5 6 6
23656 118/300 Read block 495 len 40960
23657 119/300 Write block 4176 len 28672 data: 4 3 4 4 3 5 5
23658 120/300 Read block 3300 len 24576
23659 121/300 Read block 3358 len 12288
23660 122/300 Read block 125 len 28672
23661 123/300 Read block 2189 len 12288
23662 124/300 Write block 3361 len 32768 data: 3 4 5 4 4 4 5 4
23663 125/300 Read block 2205 len 40960
23664 126/300 Read block 4863 len 8192
23665 127/300 Write block 2474 len 32768 data: 5 4 3 3 3 4 4 4
23666 128/300 Read block 3623 len 20480
23667 129/300 Write block 1890 len 4096 data: 6
23668 130/300 Flush
23669 131/300 Write block 58 len 28672 data: 4 5 6 6 6 6 7
23670 132/300 Read block 3910 len 20480
23671 133/300 Write block 2196 len 28672 data: 4 3 3 2 2 5 5
23672 134/300 Read block 362 len 28672
23673 135/300 Write block 2667 len 24576 data: 8 6 4 4 4 3
23674 136/300 Read block 2102 len 36864
23675 137/300 Flush
23676 138/300 Read block 3606 len 16384
23677 139/300 Write block 3124 len 40960 data: 6 7 11 9 10 9 9 9 8 5
23678 140/300 Flush
23679 141/300 Flush
23680 142/300 Read block 3655 len 28672
23681 143/300 Read block 180 len 4096
23682 144/300 Read block 1943 len 16384
23683 145/300 Read block 1255 len 40960
23684 146/300 Read block 91 len 4096
23685 147/300 Read block 3590 len 20480
23686 148/300 Write block 357 len 12288 data: 4 4 5
23687 149/300 Write block 2528 len 12288 data: 3 4 6
23688 150/300 Read block 1943 len 4096
23689 151/300 Write block 2094 len 16384 data: 2 4 3 4
23690 152/300 Flush
23691 153/300 Read block 2021 len 16384
23692 154/300 Flush
23693 155/300 Read block 1943 len 40960
23694 156/300 Read block 4454 len 8192
23695 157/300 Read block 1208 len 40960
23696 158/300 Flush
23697 159/300 Write block 1712 len 16384 data: 7 7 5 5
23698 160/300 Read block 3066 len 4096
23699 161/300 Write block 3616 len 28672 data: 4 5 6 6 5 6 6
23700 162/300 Read block 3385 len 16384
23701 163/300 Write block 1632 len 28672 data: 6 5 5 5 4 2 2
23702 164/300 Read block 2262 len 16384
23703 165/300 Read block 4085 len 40960
23704 166/300 Read block 4231 len 4096
23705 167/300 Read block 1002 len 40960
23706 168/300 Write block 363 len 12288 data: 4 6 6
23707 169/300 Read block 2733 len 28672
23708 170/300 Read block 1742 len 36864
23709 171/300 Write block 2657 len 4096 data: 5
23710 172/300 Write block 4853 len 12288 data: 5 4 4
23711 173/300 Write block 977 len 16384 data: 3 3 3 2
23712 174/300 Write block 1584 len 36864 data: 7 8 7 6 5 4 3 3 2
23713 175/300 Write block 1218 len 24576 data: 3 3 3 3 2 2
23714 176/300 Read block 3066 len 40960
23715 177/300 Read block 1243 len 28672
23716 178/300 Flush
23717 179/300 Write block 4672 len 24576 data: 6 6 6 6 6 4
23718 180/300 Write block 686 len 32768 data: 5 4 4 3 3 2 2 2
23719 181/300 Write block 1546 len 8192 data: 6 6
23720 182/300 Read block 2230 len 12288
23721 183/300 Read block 3173 len 16384
23722 184/300 Write block 3269 len 36864 data: 4 3 4 5 6 6 5 5 5
23723 185/300 Flush
23724 186/300 Read block 4927 len 12288
23725 187/300 Read block 535 len 40960
23726 188/300 Write block 1510 len 16384 data: 4 3 4 4
23727 189/300 Read block 1642 len 8192
23728 190/300 Write block 1669 len 16384 data: 3 3 3 3
23729 191/300 Write block 3304 len 4096 data: 6
23730 192/300 Read block 3754 len 4096
23731 193/300 Read block 2871 len 36864
23732 194/300 Write block 1870 len 36864 data: 6 5 5 6 5 7 9 9 11
23733 195/300 Read block 3738 len 40960
23734 196/300 Read block 1191 len 32768
23735 197/300 Write block 2201 len 4096 data: 6
23736 198/300 Read block 2324 len 28672
23737 199/300 Write block 987 len 40960 data: 3 4 5 5 6 7 7 5 4 4
23738 200/300 Flush
23739 201/300 Write block 1575 len 28672 data: 5 3 3 3 4 5 5
23740 202/300 Read block 186 len 40960
23741 203/300 Read block 1501 len 36864
23742 204/300 Read block 376 len 16384
23743 205/300 Write block 1692 len 8192 data: 6 7
23744 206/300 Write block 611 len 16384 data: 4 4 5 6
23745 207/300 Read block 1411 len 16384
23746 208/300 Write block 4273 len 4096 data: 4
23747 209/300 Write block 1694 len 4096 data: 7
23748 210/300 Read block 3766 len 4096
23749 211/300 Write block 140 len 4096 data: 9
23750 212/300 Flush
23751 213/300 Write block 37 len 24576 data: 3 3 3 4 4 5
23752 214/300 Flush
23753 215/300 Read block 4519 len 36864
23754 216/300 Flush
23755 217/300 Read block 1732 len 40960
23756 218/300 Write block 1206 len 20480 data: 3 3 3 3 3
23757 219/300 Write block 4292 len 24576 data: 4 4 5 5 5 6
23758 220/300 Read block 3780 len 28672
23759 221/300 Read block 3028 len 40960
23760 222/300 Flush
23761 223/300 Write block 310 len 24576 data: 3 3 4 4 5 5
23762 224/300 Read block 534 len 32768
23763 225/300 Read block 4804 len 16384
23764 226/300 Read block 56 len 40960
23765 227/300 Write block 178 len 12288 data: 6 5 6
23766 228/300 Write block 4486 len 4096 data: 6
23767 229/300 Flush
23768 230/300 Read block 1379 len 24576
23769 231/300 Read block 1611 len 28672
23770 232/300 Write block 750 len 36864 data: 7 7 6 6 7 5 4 4 3
23771 233/300 Write block 2934 len 24576 data: 9 7 9 8 6 6
23772 234/300 Write block 749 len 36864 data: 5 8 8 7 7 8 6 5 5
23773 235/300 Write block 2704 len 16384 data: 6 7 7 7
23774 236/300 Flush
23775 237/300 Read block 3098 len 20480
23776 238/300 Read block 2964 len 16384
23777 239/300 Read block 700 len 28672
23778 240/300 Write block 4833 len 8192 data: 2 2
23779 241/300 Write block 4153 len 28672 data: 4 3 3 5 6 5 4
23780 242/300 Write block 337 len 16384 data: 3 3 2 3
23781 243/300 Flush
23782 244/300 Write block 4343 len 20480 data: 3 4 4 4 4
23783 245/300 Write block 3121 len 36864 data: 5 5 5 7 8 12 10 11 10
23784 246/300 Write block 3705 len 8192 data: 3 4
23785 247/300 Read block 3007 len 32768
23786 248/300 Read block 4518 len 16384
23787 249/300 Read block 4626 len 36864
23788 250/300 Read block 3776 len 20480
23789 251/300 Write block 2469 len 24576 data: 4 4 4 4 5 6
23790 252/300 Flush
23791 253/300 Write block 2179 len 20480 data: 3 3 3 4 3
23792 254/300 Read block 1220 len 40960
23793 255/300 Read block 539 len 32768
23794 256/300 Read block 4114 len 36864
23795 257/300 Read block 369 len 4096
23796 258/300 Read block 750 len 36864
23797 259/300 Write block 2401 len 20480 data: 5 4 4 3 3
23798 260/300 Read block 4677 len 28672
23799 261/300 Write block 138 len 8192 data: 10 11
23800 262/300 Write block 3731 len 8192 data: 8 8
23801 263/300 Write block 3048 len 40960 data: 4 5 5 5 6 3 2 2 3 3
23802 264/300 Flush
23803 265/300 Read block 2548 len 24576
23804 266/300 Read block 3096 len 20480
23805 267/300 Read block 522 len 8192
23806 268/300 Write block 3915 len 8192 data: 8 8
23807 269/300 Read block 4289 len 16384
23808 270/300 Read block 1353 len 16384
23809 271/300 Write block 2210 len 36864 data: 6 6 6 8 5 5 4 4 4
23810 272/300 Write block 2139 len 32768 data: 5 5 6 7 7 5 5 6
23811 273/300 Write block 3825 len 12288 data: 7 9 8
23812 274/300 Write block 1173 len 36864 data: 5 4 5 5 5 5 4 3 4
23813 275/300 Write block 2798 len 4096 data: 6
23814 276/300 Read block 2780 len 24576
23815 277/300 Write block 1561 len 28672 data: 6 6 6 6 6 7 5
23816 278/300 Write block 4 len 28672 data: 3 3 4 5 6 6 6
23817 279/300 Read block 3814 len 40960
23818 280/300 Flush
23819 281/300 Read block 333 len 12288
23820 282/300 Read block 4140 len 4096
23821 283/300 Flush
23822 284/300 Read block 1138 len 8192
23823 285/300 Read block 2138 len 8192
23824 286/300 Write block 118 len 20480 data: 2 2 4 4 4
23825 287/300 Write block 306 len 12288 data: 3 3 3
23826 288/300 Read block 804 len 20480
23827 289/300 Read block 2601 len 40960
23828 290/300 Write block 4821 len 4096 data: 2
23829 291/300 Read block 4011 len 12288
23830 292/300 Flush
23831 293/300 Write block 3220 len 16384 data: 5 5 7 9
23832 294/300 Read block 1261 len 32768
23833 295/300 Write block 4304 len 28672 data: 2 2 2 2 2 2 4
23834 296/300 Read block 1674 len 24576
23835 297/300 Write block 4864 len 24576 data: 5 5 5 5 5 4
23836 298/300 Flush
23837 299/300 Flush
23838 300/300 Flush
23839 301/300 Read block 2316 len 8192
23840 Main task has work to do, go find it
23841 got dsc Start(2)
23842 start 2
23843 Nov 19 17:13:00.896 INFO request completed, latency_us: 187, response_code: 204, uri: /start/cid/2, method: POST, req_id: 00f0c646-9bdd-4d91-9103-39007a6f5bd5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
23844 Replay: started 2, returned:Ok(())
23845 ----------------------------------------------------------------
23846 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23847 States: Active Active Offline
23848 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23849 7038 Acked Read 2 Done Done Sent false
23850 7039 Acked Read 6 Done Done Sent false
23851 7040 Acked Read 3 Done Done Sent false
23852 7041 Acked Write 3 Done Done Sent false
23853 7042 Acked Read 4 Done Done Sent false
23854 7043 Acked Read 7 Done Done Sent false
23855 7044 Acked Flush 0 Done Done Sent false
23856 7045 Acked Read 9 Done Done Sent false
23857 7046 Acked Read 9 Done Done Sent false
23858 7047 Acked Read 9 Done Done Sent false
23859 7048 Acked Read 10 Done Done Sent false
23860 7049 Acked Read 8 Done Done Sent false
23861 7050 Acked Read 2 Done Done Sent false
23862 7051 Acked Read 9 Done Done Sent false
23863 7052 Acked Flush 0 Done Done Sent false
23864 7053 Acked Read 9 Done Done Sent false
23865 7054 Acked Read 1 Done Done Sent false
23866 7055 Acked Write 6 Done Done Sent false
23867 7056 Acked Read 7 Done Done Sent false
23868 7057 Acked Write 4 Done Done Sent false
23869 7058 Acked Write 8 Done Done Sent false
23870 7059 Acked Write 2 Done Done Sent false
23871 7060 Acked Write 6 Done Done Sent false
23872 7061 Acked Write 1 Done Done Sent false
23873 7062 Acked Write 5 Done Done Sent false
23874 7063 Acked Write 7 Done Done Sent false
23875 7064 Acked Read 8 Done Done Sent false
23876 7065 Acked Read 7 Done Done Sent false
23877 7066 Acked Read 7 Done Done Sent false
23878 7067 Acked Read 7 Done Done Sent false
23879 7068 Acked Write 4 Done Done Sent false
23880 7069 Acked Flush 0 Done Done Sent false
23881 7070 Acked Read 8 Done Done Sent false
23882 7071 Acked Read 9 Done Done Sent false
23883 7072 Acked Write 2 Done Done Sent false
23884 7073 Acked Read 2 Done Done Sent false
23885 7074 Acked Read 4 Done Done Sent false
23886 7075 Acked Read 6 Done Done Sent false
23887 7076 Acked Write 1 Done Done Sent false
23888 7077 Acked Read 10 Done Done Sent false
23889 7078 Acked Write 1 Done Done Sent false
23890 7079 Acked Write 1 Done Done Sent false
23891 7080 Acked Read 1 Done Done Sent false
23892 7081 Acked Flush 0 Done Done Sent false
23893 7082 Acked Write 7 Done Done Sent false
23894 7083 Acked Read 2 Done Done Sent false
23895 7084 Acked Read 1 Done Done Sent false
23896 7085 Acked Read 8 Done Done Sent false
23897 7086 Acked Read 2 Done Done Sent false
23898 7087 Acked Write 8 Done Done Sent false
23899 7088 Acked Read 9 Done Done Sent false
23900 7089 Acked Read 9 Done Done Sent false
23901 7090 Acked Write 10 Done Done Sent false
23902 7091 Acked Write 5 Done Done Sent false
23903 7092 Acked Write 4 Done Done Sent false
23904 7093 Acked Read 5 Done Done Sent false
23905 7094 Acked Read 9 Done Done Sent false
23906 7095 Acked Flush 0 Done Done Sent false
23907 7096 Acked Read 6 Done Done Sent false
23908 7097 Acked Read 2 Done Done Sent false
23909 7098 Acked Read 3 Done Done Sent false
23910 7099 Acked Write 2 Done Done Sent false
23911 7100 Acked Read 10 Done Done Sent false
23912 7101 Acked Read 5 Done Done Sent false
23913 7102 Acked Write 6 Done Done Sent false
23914 7103 Acked Write 10 Done Done Sent false
23915 7104 Acked Read 10 Done Done Sent false
23916 7105 Acked Read 2 Done Done Sent false
23917 7106 Acked Read 4 Done Done Sent false
23918 7107 Acked Read 7 Done Done Sent false
23919 7108 Acked Flush 0 Done Done Sent false
23920 7109 Acked Flush 0 Done Done Sent false
23921 7110 Acked Write 10 Done Done Sent false
23922 7111 Acked Write 1 Done Done Sent false
23923 7112 Acked Write 10 Done Done Sent false
23924 7113 Acked Read 6 Done Done Sent false
23925 7114 Acked Flush 0 Done Done Sent false
23926 7115 Acked Read 7 Done Done Sent false
23927 7116 Acked Read 3 Done Done Sent false
23928 7117 Acked Read 3 Done Done Sent false
23929 7118 Acked Read 8 Done Done Sent false
23930 7119 Acked Write 1 Done Done Sent false
23931 7120 Acked Write 8 Done Done Sent false
23932 7121 Acked Write 10 Done Done Sent false
23933 7122 Acked Read 5 Done Done Sent false
23934 7123 Acked Write 4 Done Done Sent false
23935 7124 Acked Read 5 Done Done Sent false
23936 7125 Acked Read 4 Done Done Sent false
23937 7126 Acked Read 10 Done Done Sent false
23938 7127 Acked Write 9 Done Done Sent false
23939 7128 Acked Write 4 Done Done Sent false
23940 7129 Acked Read 1 Done Done Sent false
23941 7130 Acked Read 7 Done Done Sent false
23942 7131 Acked Read 9 Done Done Sent false
23943 7132 Acked Read 2 Done Done Sent false
23944 7133 Acked Write 3 Done Done Sent false
23945 7134 Acked Read 7 Done Done Sent false
23946 7135 Acked Read 9 Done Done Sent false
23947 7136 Acked Write 6 Done Done Sent false
23948 7137 Acked Read 9 Done Done Sent false
23949 7138 Acked Read 7 Done Done Sent false
23950 7139 Acked Read 6 Done Done Sent false
23951 7140 Acked Read 8 Done Done Sent false
23952 7141 Acked Write 3 Done Done Sent false
23953 7142 Acked Read 9 Done Done Sent false
23954 7143 Acked Read 2 Done Done Sent false
23955 7144 Acked Write 6 Done Done Sent false
23956 7145 Acked Write 5 Done Done Sent false
23957 7146 Acked Write 7 Done Done Sent false
23958 7147 Acked Write 6 Done Done Sent false
23959 7148 Acked Read 8 Done Done Sent false
23960 7149 Acked Write 8 Done Done Sent false
23961 7150 Acked Write 2 Done Done Sent false
23962 7151 Acked Read 4 Done Done Sent false
23963 7152 Acked Read 4 Done Done Sent false
23964 7153 Acked Flush 0 Done Done Sent false
23965 7154 Acked Write 7 Done Done Sent false
23966 7155 Acked Read 10 Done Done Sent false
23967 7156 Acked Write 7 Done Done Sent false
23968 7157 Acked Read 6 Done Done Sent false
23969 7158 Acked Read 3 Done Done Sent false
23970 7159 Acked Read 7 Done Done Sent false
23971 7160 Acked Read 3 Done Done Sent false
23972 7161 Acked Write 8 Done Done Sent false
23973 7162 Acked Read 10 Done Done Sent false
23974 7163 Acked Read 2 Done Done Sent false
23975 7164 Acked Write 8 Done Done Sent false
23976 7165 Acked Read 5 Done Done Sent false
23977 7166 Acked Write 1 Done Done Sent false
23978 7167 Acked Flush 0 Done Done Sent false
23979 7168 Acked Write 7 Done Done Sent false
23980 7169 Acked Read 5 Done Done Sent false
23981 7170 Acked Write 7 Done Done Sent false
23982 7171 Acked Read 7 Done Done Sent false
23983 7172 Acked Write 6 Done Done Sent false
23984 7173 Acked Read 9 Done Done Sent false
23985 7174 Acked Flush 0 Done Done Sent false
23986 7175 Acked Read 4 Done Done Sent false
23987 7176 Acked Write 10 Done Done Sent false
23988 7177 Acked Flush 0 Done Done Sent false
23989 7178 Acked Flush 0 Done Done Sent false
23990 7179 Acked Read 7 Done Done Sent false
23991 7180 Acked Read 1 Done Done Sent false
23992 7181 Acked Read 4 Done Done Sent false
23993 7182 Acked Read 10 Done Done Sent false
23994 7183 Acked Read 1 Done Done Sent false
23995 7184 Acked Read 5 Done Done Sent false
23996 7185 Acked Write 3 Done Done Sent false
23997 7186 Acked Write 3 Done Done Sent false
23998 7187 Acked Read 1 Done Done Sent false
23999 7188 Acked Write 4 Done Done Sent false
24000 7189 Acked Flush 0 Done Done Sent false
24001 7190 Acked Read 4 Done Done Sent false
24002 7191 Acked Flush 0 Done Done Sent false
24003 7192 Acked Read 10 Done Done Sent false
24004 7193 Acked Read 2 Done Done Sent false
24005 7194 Acked Read 10 Done Done Sent false
24006 7195 Acked Flush 0 Done Done Sent false
24007 7196 Acked Write 4 Done Done Sent false
24008 7197 Acked Read 1 Done Done Sent false
24009 7198 Acked Write 7 Done Done Sent false
24010 7199 Acked Read 4 Done Done Sent false
24011 7200 Acked Write 7 Done Done Sent false
24012 7201 Acked Read 4 Done Done Sent false
24013 7202 Acked Read 10 Done Done Sent false
24014 7203 Acked Read 1 Done Done Sent false
24015 7204 Acked Read 10 Done Done Sent false
24016 7205 Acked Write 3 Done Done Sent false
24017 7206 Acked Read 7 Done Done Sent false
24018 7207 Acked Read 9 Done Done Sent false
24019 7208 Acked Write 1 Done Done Sent false
24020 7209 Acked Write 3 Done Done Sent false
24021 7210 Acked Write 4 Done Done Sent false
24022 7211 Acked Write 9 Done Done Sent false
24023 7212 Acked Write 6 Done Done Sent false
24024 7213 Acked Read 10 Done Done Sent false
24025 7214 Acked Read 7 Done Done Sent false
24026 7215 Acked Flush 0 Done Done Sent false
24027 7216 Acked Write 6 Done Done Sent false
24028 7217 Acked Write 8 Done Done Sent false
24029 7218 Acked Write 2 Done Done Sent false
24030 7219 Acked Read 3 Done Done Sent false
24031 7220 Acked Read 4 Done Done Sent false
24032 7221 Acked Write 9 Done Done Sent false
24033 7222 Acked Flush 0 Done Done Sent false
24034 7223 Acked Read 3 Done Done Sent false
24035 7224 Acked Read 10 Done Done Sent false
24036 7225 Acked Write 4 Done Done Sent false
24037 7226 Acked Read 2 Done Done Sent false
24038 7227 Acked Write 4 Done Done Sent false
24039 7228 Acked Write 1 Done Done Sent false
24040 7229 Acked Read 1 Done Done Sent false
24041 7230 Acked Read 9 Done Done Sent false
24042 7231 Acked Write 9 Done Done Sent false
24043 7232 Acked Read 10 Done Done Sent false
24044 7233 Acked Read 8 Done Done Sent false
24045 7234 Acked Write 1 Done Done Sent false
24046 7235 Acked Read 7 Done Done Sent false
24047 7236 Acked Write 10 Done Done Sent false
24048 7237 Acked Flush 0 Done Done Sent false
24049 7238 Acked Write 7 Done Done Sent false
24050 7239 Acked Read 10 Done Done Sent false
24051 7240 Acked Read 9 Done Done Sent false
24052 7241 Acked Read 4 Done Done Sent false
24053 7242 Acked Write 2 Done Done Sent false
24054 7243 Acked Write 4 Done Done Sent false
24055 7244 Acked Read 4 Done Done Sent false
24056 7245 Acked Write 1 Done Done Sent false
24057 7246 Acked Write 1 Done Done Sent false
24058 7247 Acked Read 1 Done Done Sent false
24059 7248 Acked Write 1 Done Done Sent false
24060 7249 Acked Flush 0 Done Done Sent false
24061 7250 Acked Write 6 Done Done Sent false
24062 7251 Acked Flush 0 Done Done Sent false
24063 7252 Acked Read 9 Done Done Sent false
24064 7253 Acked Flush 0 Done Done Sent false
24065 7254 Acked Read 10 Done Done Sent false
24066 7255 Acked Write 5 Done Done Sent false
24067 7256 Acked Write 6 Done Done Sent false
24068 7257 Acked Read 7 Done Done Sent false
24069 7258 Acked Read 10 Done Done Sent false
24070 7259 Acked Flush 0 Done Done Sent false
24071 7260 Acked Write 6 Done Done Sent false
24072 7261 Acked Read 8 Done Done Sent false
24073 7262 Acked Read 4 Done Done Sent false
24074 7263 Acked Read 10 Done Done Sent false
24075 7264 Acked Write 3 Done Done Sent false
24076 7265 Acked Write 1 Done Done Sent false
24077 7266 Acked Flush 0 Done Done Sent false
24078 7267 Acked Read 6 Done Done Sent false
24079 7268 Acked Read 7 Done Done Sent false
24080 7269 Acked Write 9 Done Done Sent false
24081 7270 Acked Write 6 Done Done Sent false
24082 7271 Acked Write 9 Done Done Sent false
24083 7272 Acked Write 4 Done Done Sent false
24084 7273 Acked Flush 0 Done Done Sent false
24085 7274 Acked Read 5 Done Done Sent false
24086 7275 Acked Read 4 Done Done Sent false
24087 7276 Acked Read 7 Done Done Sent false
24088 7277 Acked Write 2 Done Done Sent false
24089 7278 Acked Write 7 Done Done Sent false
24090 7279 Acked Write 4 Done Done Sent false
24091 7280 Acked Flush 0 Done Done Sent false
24092 7281 Acked Write 5 Done Done Sent false
24093 7282 Acked Write 9 Done Done Sent false
24094 7283 Acked Write 2 Done Done Sent false
24095 7284 Acked Read 8 Done Done Sent false
24096 7285 Acked Read 4 Done Done Sent false
24097 7286 Acked Read 9 Done Done Sent false
24098 7287 Acked Read 5 Done Done Sent false
24099 7288 Acked Write 6 Done Done Sent false
24100 7289 Acked Flush 0 Done Done Sent false
24101 7290 Acked Write 5 Done Done Sent false
24102 7291 Acked Read 10 Done Done Sent false
24103 7292 Acked Read 8 Done Done Sent false
24104 7293 Acked Read 9 Done Done Sent false
24105 7294 Acked Read 1 Done Done Sent false
24106 7295 Acked Read 9 Done Done Sent false
24107 7296 Acked Write 5 Done Done Sent false
24108 7297 Acked Read 7 Done Done Sent false
24109 7298 Acked Write 2 Done Done Sent false
24110 7299 Acked Write 2 Done Done Sent false
24111 7300 Acked Write 10 Done Done Sent false
24112 7301 Acked Flush 0 Done Done Sent false
24113 7302 Acked Read 6 Done Done Sent false
24114 7303 Acked Read 5 Done Done Sent false
24115 7304 Acked Read 2 Done Done Sent false
24116 7305 Acked Write 2 Done Done Sent false
24117 7306 Acked Read 4 Done Done Sent false
24118 7307 Acked Read 4 Done Done Sent false
24119 7308 Acked Write 9 Done Done Sent false
24120 7309 Acked Write 8 Done Done Sent false
24121 7310 Acked Write 3 Done Done Sent false
24122 7311 Acked Write 9 Done Done Sent false
24123 7312 Acked Write 1 Done Done Sent false
24124 7313 Acked Read 6 Done Done Sent false
24125 7314 Acked Write 7 Done Done Sent false
24126 7315 Acked Write 7 Done Done Sent false
24127 7316 Acked Read 10 Done Done Sent false
24128 7317 Acked Flush 0 Done Done Sent false
24129 7318 Acked Read 3 Done Done Sent false
24130 7319 Acked Read 1 Done Done Sent false
24131 7320 Acked Flush 0 Done Done Sent false
24132 7321 Acked Read 2 Done Done Sent false
24133 7322 Acked Read 2 Done Done Sent false
24134 7323 Acked Write 5 Done Done Sent false
24135 7324 Acked Write 3 Done Done Sent false
24136 7325 Acked Read 5 Done Done Sent false
24137 7326 Acked Read 10 Done Done Sent false
24138 7327 Acked Write 1 Done Done Sent false
24139 7328 Acked Read 3 Done Done Sent false
24140 7329 Acked Flush 0 Done Done Sent false
24141 7330 Acked Write 4 Done Done Sent false
24142 7331 Acked Read 8 Done Done Sent false
24143 7332 Acked Write 7 Done Done Sent false
24144 7333 Acked Read 6 Done Done Sent false
24145 7334 Acked Write 6 Done Done Sent false
24146 7335 Acked Flush 0 Done Done Sent false
24147 7336 Acked Flush 0 Done Done Sent false
24148 7337 Acked Flush 0 Done Done Sent false
24149 7338 Acked Read 2 Done Done Sent false
24150 STATES DS:0 DS:1 DS:2 TOTAL
24151 Sent 0 0 301 301
24152 Done 301 301 0 602
24153 Skipped 0 0 0 0
24154 Error 0 0 0 0
24155 Last Flush: 7337 7337 7037
24156 Downstairs last five completed: 7037 7036 7035 7034 7033
24157 Upstairs last five completed: 7338 7337 7336 7335 7334
24158 CLIENT: Up:0 ds:301 act:2
24159 [8830] got start action
24160 [8830] I am going to restart
24161 Starting downstairs at port 8830
24162 Make output file at "/tmp/dsc/downstairs-8830.txt"
24163 [8830][2] reports Starting
24164 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
24165 [8830][2] reports Running
24166 ----------------------------------------------------------------
24167 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24168 States: Active Active Offline
24169 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24170 7038 Acked Read 2 Done Done Sent false
24171 7039 Acked Read 6 Done Done Sent false
24172 7040 Acked Read 3 Done Done Sent false
24173 7041 Acked Write 3 Done Done Sent false
24174 7042 Acked Read 4 Done Done Sent false
24175 7043 Acked Read 7 Done Done Sent false
24176 7044 Acked Flush 0 Done Done Sent false
24177 7045 Acked Read 9 Done Done Sent false
24178 7046 Acked Read 9 Done Done Sent false
24179 7047 Acked Read 9 Done Done Sent false
24180 7048 Acked Read 10 Done Done Sent false
24181 7049 Acked Read 8 Done Done Sent false
24182 7050 Acked Read 2 Done Done Sent false
24183 7051 Acked Read 9 Done Done Sent false
24184 7052 Acked Flush 0 Done Done Sent false
24185 7053 Acked Read 9 Done Done Sent false
24186 7054 Acked Read 1 Done Done Sent false
24187 7055 Acked Write 6 Done Done Sent false
24188 7056 Acked Read 7 Done Done Sent false
24189 7057 Acked Write 4 Done Done Sent false
24190 7058 Acked Write 8 Done Done Sent false
24191 7059 Acked Write 2 Done Done Sent false
24192 7060 Acked Write 6 Done Done Sent false
24193 7061 Acked Write 1 Done Done Sent false
24194 7062 Acked Write 5 Done Done Sent false
24195 7063 Acked Write 7 Done Done Sent false
24196 7064 Acked Read 8 Done Done Sent false
24197 7065 Acked Read 7 Done Done Sent false
24198 7066 Acked Read 7 Done Done Sent false
24199 7067 Acked Read 7 Done Done Sent false
24200 7068 Acked Write 4 Done Done Sent false
24201 7069 Acked Flush 0 Done Done Sent false
24202 7070 Acked Read 8 Done Done Sent false
24203 7071 Acked Read 9 Done Done Sent false
24204 7072 Acked Write 2 Done Done Sent false
24205 7073 Acked Read 2 Done Done Sent false
24206 7074 Acked Read 4 Done Done Sent false
24207 7075 Acked Read 6 Done Done Sent false
24208 7076 Acked Write 1 Done Done Sent false
24209 7077 Acked Read 10 Done Done Sent false
24210 7078 Acked Write 1 Done Done Sent false
24211 7079 Acked Write 1 Done Done Sent false
24212 7080 Acked Read 1 Done Done Sent false
24213 7081 Acked Flush 0 Done Done Sent false
24214 7082 Acked Write 7 Done Done Sent false
24215 7083 Acked Read 2 Done Done Sent false
24216 7084 Acked Read 1 Done Done Sent false
24217 7085 Acked Read 8 Done Done Sent false
24218 7086 Acked Read 2 Done Done Sent false
24219 7087 Acked Write 8 Done Done Sent false
24220 7088 Acked Read 9 Done Done Sent false
24221 7089 Acked Read 9 Done Done Sent false
24222 7090 Acked Write 10 Done Done Sent false
24223 7091 Acked Write 5 Done Done Sent false
24224 7092 Acked Write 4 Done Done Sent false
24225 7093 Acked Read 5 Done Done Sent false
24226 7094 Acked Read 9 Done Done Sent false
24227 7095 Acked Flush 0 Done Done Sent false
24228 7096 Acked Read 6 Done Done Sent false
24229 7097 Acked Read 2 Done Done Sent false
24230 7098 Acked Read 3 Done Done Sent false
24231 7099 Acked Write 2 Done Done Sent false
24232 7100 Acked Read 10 Done Done Sent false
24233 7101 Acked Read 5 Done Done Sent false
24234 7102 Acked Write 6 Done Done Sent false
24235 7103 Acked Write 10 Done Done Sent false
24236 7104 Acked Read 10 Done Done Sent false
24237 7105 Acked Read 2 Done Done Sent false
24238 7106 Acked Read 4 Done Done Sent false
24239 7107 Acked Read 7 Done Done Sent false
24240 7108 Acked Flush 0 Done Done Sent false
24241 7109 Acked Flush 0 Done Done Sent false
24242 7110 Acked Write 10 Done Done Sent false
24243 7111 Acked Write 1 Done Done Sent false
24244 7112 Acked Write 10 Done Done Sent false
24245 7113 Acked Read 6 Done Done Sent false
24246 7114 Acked Flush 0 Done Done Sent false
24247 7115 Acked Read 7 Done Done Sent false
24248 7116 Acked Read 3 Done Done Sent false
24249 7117 Acked Read 3 Done Done Sent false
24250 7118 Acked Read 8 Done Done Sent false
24251 7119 Acked Write 1 Done Done Sent false
24252 7120 Acked Write 8 Done Done Sent false
24253 7121 Acked Write 10 Done Done Sent false
24254 7122 Acked Read 5 Done Done Sent false
24255 7123 Acked Write 4 Done Done Sent false
24256 7124 Acked Read 5 Done Done Sent false
24257 7125 Acked Read 4 Done Done Sent false
24258 7126 Acked Read 10 Done Done Sent false
24259 7127 Acked Write 9 Done Done Sent false
24260 7128 Acked Write 4 Done Done Sent false
24261 7129 Acked Read 1 Done Done Sent false
24262 7130 Acked Read 7 Done Done Sent false
24263 7131 Acked Read 9 Done Done Sent false
24264 7132 Acked Read 2 Done Done Sent false
24265 7133 Acked Write 3 Done Done Sent false
24266 7134 Acked Read 7 Done Done Sent false
24267 7135 Acked Read 9 Done Done Sent false
24268 7136 Acked Write 6 Done Done Sent false
24269 7137 Acked Read 9 Done Done Sent false
24270 7138 Acked Read 7 Done Done Sent false
24271 7139 Acked Read 6 Done Done Sent false
24272 7140 Acked Read 8 Done Done Sent false
24273 7141 Acked Write 3 Done Done Sent false
24274 7142 Acked Read 9 Done Done Sent false
24275 7143 Acked Read 2 Done Done Sent false
24276 7144 Acked Write 6 Done Done Sent false
24277 7145 Acked Write 5 Done Done Sent false
24278 7146 Acked Write 7 Done Done Sent false
24279 7147 Acked Write 6 Done Done Sent false
24280 7148 Acked Read 8 Done Done Sent false
24281 7149 Acked Write 8 Done Done Sent false
24282 7150 Acked Write 2 Done Done Sent false
24283 7151 Acked Read 4 Done Done Sent false
24284 7152 Acked Read 4 Done Done Sent false
24285 7153 Acked Flush 0 Done Done Sent false
24286 7154 Acked Write 7 Done Done Sent false
24287 7155 Acked Read 10 Done Done Sent false
24288 7156 Acked Write 7 Done Done Sent false
24289 7157 Acked Read 6 Done Done Sent false
24290 7158 Acked Read 3 Done Done Sent false
24291 7159 Acked Read 7 Done Done Sent false
24292 7160 Acked Read 3 Done Done Sent false
24293 7161 Acked Write 8 Done Done Sent false
24294 7162 Acked Read 10 Done Done Sent false
24295 7163 Acked Read 2 Done Done Sent false
24296 7164 Acked Write 8 Done Done Sent false
24297 7165 Acked Read 5 Done Done Sent false
24298 7166 Acked Write 1 Done Done Sent false
24299 7167 Acked Flush 0 Done Done Sent false
24300 7168 Acked Write 7 Done Done Sent false
24301 7169 Acked Read 5 Done Done Sent false
24302 7170 Acked Write 7 Done Done Sent false
24303 7171 Acked Read 7 Done Done Sent false
24304 7172 Acked Write 6 Done Done Sent false
24305 7173 Acked Read 9 Done Done Sent false
24306 7174 Acked Flush 0 Done Done Sent false
24307 7175 Acked Read 4 Done Done Sent false
24308 7176 Acked Write 10 Done Done Sent false
24309 7177 Acked Flush 0 Done Done Sent false
24310 7178 Acked Flush 0 Done Done Sent false
24311 7179 Acked Read 7 Done Done Sent false
24312 7180 Acked Read 1 Done Done Sent false
24313 7181 Acked Read 4 Done Done Sent false
24314 7182 Acked Read 10 Done Done Sent false
24315 7183 Acked Read 1 Done Done Sent false
24316 7184 Acked Read 5 Done Done Sent false
24317 7185 Acked Write 3 Done Done Sent false
24318 7186 Acked Write 3 Done Done Sent false
24319 7187 Acked Read 1 Done Done Sent false
24320 7188 Acked Write 4 Done Done Sent false
24321 7189 Acked Flush 0 Done Done Sent false
24322 7190 Acked Read 4 Done Done Sent false
24323 7191 Acked Flush 0 Done Done Sent false
24324 7192 Acked Read 10 Done Done Sent false
24325 7193 Acked Read 2 Done Done Sent false
24326 7194 Acked Read 10 Done Done Sent false
24327 7195 Acked Flush 0 Done Done Sent false
24328 7196 Acked Write 4 Done Done Sent false
24329 7197 Acked Read 1 Done Done Sent false
24330 7198 Acked Write 7 Done Done Sent false
24331 7199 Acked Read 4 Done Done Sent false
24332 7200 Acked Write 7 Done Done Sent false
24333 7201 Acked Read 4 Done Done Sent false
24334 7202 Acked Read 10 Done Done Sent false
24335 7203 Acked Read 1 Done Done Sent false
24336 7204 Acked Read 10 Done Done Sent false
24337 7205 Acked Write 3 Done Done Sent false
24338 7206 Acked Read 7 Done Done Sent false
24339 7207 Acked Read 9 Done Done Sent false
24340 7208 Acked Write 1 Done Done Sent false
24341 7209 Acked Write 3 Done Done Sent false
24342 7210 Acked Write 4 Done Done Sent false
24343 7211 Acked Write 9 Done Done Sent false
24344 7212 Acked Write 6 Done Done Sent false
24345 7213 Acked Read 10 Done Done Sent false
24346 7214 Acked Read 7 Done Done Sent false
24347 7215 Acked Flush 0 Done Done Sent false
24348 7216 Acked Write 6 Done Done Sent false
24349 7217 Acked Write 8 Done Done Sent false
24350 7218 Acked Write 2 Done Done Sent false
24351 7219 Acked Read 3 Done Done Sent false
24352 7220 Acked Read 4 Done Done Sent false
24353 7221 Acked Write 9 Done Done Sent false
24354 7222 Acked Flush 0 Done Done Sent false
24355 7223 Acked Read 3 Done Done Sent false
24356 7224 Acked Read 10 Done Done Sent false
24357 7225 Acked Write 4 Done Done Sent false
24358 7226 Acked Read 2 Done Done Sent false
24359 7227 Acked Write 4 Done Done Sent false
24360 7228 Acked Write 1 Done Done Sent false
24361 7229 Acked Read 1 Done Done Sent false
24362 7230 Acked Read 9 Done Done Sent false
24363 7231 Acked Write 9 Done Done Sent false
24364 7232 Acked Read 10 Done Done Sent false
24365 7233 Acked Read 8 Done Done Sent false
24366 7234 Acked Write 1 Done Done Sent false
24367 7235 Acked Read 7 Done Done Sent false
24368 7236 Acked Write 10 Done Done Sent false
24369 7237 Acked Flush 0 Done Done Sent false
24370 7238 Acked Write 7 Done Done Sent false
24371 7239 Acked Read 10 Done Done Sent false
24372 7240 Acked Read 9 Done Done Sent false
24373 7241 Acked Read 4 Done Done Sent false
24374 7242 Acked Write 2 Done Done Sent false
24375 7243 Acked Write 4 Done Done Sent false
24376 7244 Acked Read 4 Done Done Sent false
24377 7245 Acked Write 1 Done Done Sent false
24378 7246 Acked Write 1 Done Done Sent false
24379 7247 Acked Read 1 Done Done Sent false
24380 7248 Acked Write 1 Done Done Sent false
24381 7249 Acked Flush 0 Done Done Sent false
24382 7250 Acked Write 6 Done Done Sent false
24383 7251 Acked Flush 0 Done Done Sent false
24384 7252 Acked Read 9 Done Done Sent false
24385 7253 Acked Flush 0 Done Done Sent false
24386 7254 Acked Read 10 Done Done Sent false
24387 7255 Acked Write 5 Done Done Sent false
24388 7256 Acked Write 6 Done Done Sent false
24389 7257 Acked Read 7 Done Done Sent false
24390 7258 Acked Read 10 Done Done Sent false
24391 7259 Acked Flush 0 Done Done Sent false
24392 7260 Acked Write 6 Done Done Sent false
24393 7261 Acked Read 8 Done Done Sent false
24394 7262 Acked Read 4 Done Done Sent false
24395 7263 Acked Read 10 Done Done Sent false
24396 7264 Acked Write 3 Done Done Sent false
24397 7265 Acked Write 1 Done Done Sent false
24398 7266 Acked Flush 0 Done Done Sent false
24399 7267 Acked Read 6 Done Done Sent false
24400 7268 Acked Read 7 Done Done Sent false
24401 7269 Acked Write 9 Done Done Sent false
24402 7270 Acked Write 6 Done Done Sent false
24403 7271 Acked Write 9 Done Done Sent false
24404 7272 Acked Write 4 Done Done Sent false
24405 7273 Acked Flush 0 Done Done Sent false
24406 7274 Acked Read 5 Done Done Sent false
24407 7275 Acked Read 4 Done Done Sent false
24408 7276 Acked Read 7 Done Done Sent false
24409 7277 Acked Write 2 Done Done Sent false
24410 7278 Acked Write 7 Done Done Sent false
24411 7279 Acked Write 4 Done Done Sent false
24412 7280 Acked Flush 0 Done Done Sent false
24413 7281 Acked Write 5 Done Done Sent false
24414 7282 Acked Write 9 Done Done Sent false
24415 7283 Acked Write 2 Done Done Sent false
24416 7284 Acked Read 8 Done Done Sent false
24417 7285 Acked Read 4 Done Done Sent false
24418 7286 Acked Read 9 Done Done Sent false
24419 7287 Acked Read 5 Done Done Sent false
24420 7288 Acked Write 6 Done Done Sent false
24421 7289 Acked Flush 0 Done Done Sent false
24422 7290 Acked Write 5 Done Done Sent false
24423 7291 Acked Read 10 Done Done Sent false
24424 7292 Acked Read 8 Done Done Sent false
24425 7293 Acked Read 9 Done Done Sent false
24426 7294 Acked Read 1 Done Done Sent false
24427 7295 Acked Read 9 Done Done Sent false
24428 7296 Acked Write 5 Done Done Sent false
24429 7297 Acked Read 7 Done Done Sent false
24430 7298 Acked Write 2 Done Done Sent false
24431 7299 Acked Write 2 Done Done Sent false
24432 7300 Acked Write 10 Done Done Sent false
24433 7301 Acked Flush 0 Done Done Sent false
24434 7302 Acked Read 6 Done Done Sent false
24435 7303 Acked Read 5 Done Done Sent false
24436 7304 Acked Read 2 Done Done Sent false
24437 7305 Acked Write 2 Done Done Sent false
24438 7306 Acked Read 4 Done Done Sent false
24439 7307 Acked Read 4 Done Done Sent false
24440 7308 Acked Write 9 Done Done Sent false
24441 7309 Acked Write 8 Done Done Sent false
24442 7310 Acked Write 3 Done Done Sent false
24443 7311 Acked Write 9 Done Done Sent false
24444 7312 Acked Write 1 Done Done Sent false
24445 7313 Acked Read 6 Done Done Sent false
24446 7314 Acked Write 7 Done Done Sent false
24447 7315 Acked Write 7 Done Done Sent false
24448 7316 Acked Read 10 Done Done Sent false
24449 7317 Acked Flush 0 Done Done Sent false
24450 7318 Acked Read 3 Done Done Sent false
24451 7319 Acked Read 1 Done Done Sent false
24452 7320 Acked Flush 0 Done Done Sent false
24453 7321 Acked Read 2 Done Done Sent false
24454 7322 Acked Read 2 Done Done Sent false
24455 7323 Acked Write 5 Done Done Sent false
24456 7324 Acked Write 3 Done Done Sent false
24457 7325 Acked Read 5 Done Done Sent false
24458 7326 Acked Read 10 Done Done Sent false
24459 7327 Acked Write 1 Done Done Sent false
24460 7328 Acked Read 3 Done Done Sent false
24461 7329 Acked Flush 0 Done Done Sent false
24462 7330 Acked Write 4 Done Done Sent false
24463 7331 Acked Read 8 Done Done Sent false
24464 7332 Acked Write 7 Done Done Sent false
24465 7333 Acked Read 6 Done Done Sent false
24466 7334 Acked Write 6 Done Done Sent false
24467 7335 Acked Flush 0 Done Done Sent false
24468 7336 Acked Flush 0 Done Done Sent false
24469 7337 Acked Flush 0 Done Done Sent false
24470 7338 Acked Read 2 Done Done Sent false
24471 7339 Acked Flush 0 Done Done Sent false
24472 STATES DS:0 DS:1 DS:2 TOTAL
24473 Sent 0 0 302 302
24474 Done 302 302 0 604
24475 Skipped 0 0 0 0
24476 Error 0 0 0 0
24477 Last Flush: 7339 7339 7037
24478 Downstairs last five completed: 7037 7036 7035 7034 7033
24479 Upstairs last five completed: 7339 7338 7337 7336 7335
24480 CLIENT: Up:0 ds:302 act:2
244812024-11-19T17:13:06.656ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244822024-11-19T17:13:06.656ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62998) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244832024-11-19T17:13:06.657ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244842024-11-19T17:13:06.657ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244852024-11-19T17:13:06.657ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244862024-11-19T17:13:06.657ZINFOcrucible: send last flush ID to this DS: 7037 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244872024-11-19T17:13:06.657ZINFOcrucible: Replied this last flush ID: 7037 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244882024-11-19T17:13:06.658ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244892024-11-19T17:13:06.658ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
244902024-11-19T17:13:06.658ZINFOcrucible: [2] client re-new 302 jobs since flush 7037 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
244912024-11-19T17:13:06.658ZINFOcrucible: [2] Replayed 302 jobs since flush: 7037 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
24492 ----------------------------------------------------------------
24493 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24494 Downstairs last five completed: 7339 7338 7337 7336 7335
24495 Upstairs last five completed: 7339 7338 7337 7336 7335
24496 CLIENT: Up:0 ds:0 act:3
24497 Replay: All jobs finished
24498 Main task has work to do, go find it
24499 got dsc Stop(1)
24500 stop 1
24501 Nov 19 17:13:08.902 INFO request completed, [8820] Got stop action so:false kr:false
24502 latency_us: 349, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 2c0c1a00-d831-453a-be10-a3d8445832a8, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
245032024-11-19T17:13:08.903ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
245042024-11-19T17:13:08.903ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
245052024-11-19T17:13:08.903ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
24506 Nov 19 17:13:08.903 INFO request completed, latency_us: 147, response_code: 200, uri: /state/cid/1, method: GET, req_id: fdbfece0-41ab-4b07-bd71-baa5c215e67d, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
245072024-11-19T17:13:08.903ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
245082024-11-19T17:13:08.903ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
245092024-11-19T17:13:08.903ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
24510 [8820] Exited with: ExitStatus(unix_wait_status(9))
24511 [8820][1] reports Exit
24512 Nov 19 17:13:12.905 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/1, method: GET, req_id: ffa7b9b7-d2b2-484e-8447-716ac28b2219, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
24513 001/300 Read block 472 len 8192
24514 002/300 Read block 4354 len 4096
24515 003/300 Write block 2083 len 36864 data: 5 3 3 2 2 2 3 3 3
24516 004/300 Flush
24517 005/300 Write block 3029 len 32768 data: 4 5 6 6 9 7 6 6
24518 006/300 Read block 1759 len 20480
24519 007/300 Read block 844 len 28672
24520 008/300 Write block 4791 len 24576 data: 3 3 6 6 8 6
24521 009/300 Write block 835 len 28672 data: 5 5 5 6 5 4 5
24522 010/300 Write block 1186 len 4096 data: 5
24523 011/300 Read block 3084 len 28672
24524 012/300 Write block 1285 len 8192 data: 6 7
24525 013/300 Read block 3723 len 16384
24526 014/300 Write block 2167 len 20480 data: 7 6 6 5 5
24527 015/300 Flush
24528 016/300 Read block 239 len 36864
24529 017/300 Read block 2550 len 28672
24530 018/300 Write block 120 len 16384 data: 5 5 5 4
24531 019/300 Write block 3405 len 12288 data: 3 4 4
24532 020/300 Read block 4799 len 4096
24533 021/300 Write block 639 len 16384 data: 2 2 3 4
24534 022/300 Read block 3145 len 20480
24535 023/300 Read block 4678 len 28672
24536 024/300 Write block 2587 len 24576 data: 6 7 5 5 6 6
24537 025/300 Read block 4109 len 32768
24538 026/300 Read block 3350 len 8192
24539 027/300 Write block 3999 len 4096 data: 6
24540 028/300 Flush
24541 029/300 Read block 2737 len 12288
24542 030/300 Read block 324 len 36864
24543 031/300 Read block 2608 len 12288
24544 032/300 Read block 500 len 32768
24545 033/300 Read block 2072 len 32768
24546 034/300 Read block 4593 len 24576
24547 035/300 Read block 1273 len 24576
24548 036/300 Write block 3923 len 16384 data: 2 3 3 5
24549 037/300 Write block 4626 len 4096 data: 3
24550 038/300 Read block 1781 len 20480
24551 039/300 Read block 1964 len 16384
24552 040/300 Flush
24553 041/300 Read block 3119 len 36864
24554 042/300 Read block 910 len 4096
24555 043/300 Read block 1829 len 24576
24556 044/300 Write block 1695 len 36864 data: 6 6 6 4 3 3 3 4 4
24557 045/300 Write block 4649 len 16384 data: 4 4 5 7
24558 046/300 Read block 428 len 20480
24559 047/300 Write block 2809 len 4096 data: 6
24560 048/300 Read block 2125 len 32768
24561 049/300 Read block 845 len 16384
24562 050/300 Write block 3784 len 36864 data: 2 3 5 5 5 4 5 5 5
24563 051/300 Read block 4625 len 8192
24564 052/300 Read block 3077 len 12288
24565 053/300 Write block 817 len 16384 data: 3 4 5 6
24566 054/300 Flush
24567 055/300 Read block 510 len 40960
24568 056/300 Read block 4567 len 16384
24569 057/300 Read block 4151 len 24576
24570 058/300 Write block 1847 len 12288 data: 3 4 4
24571 059/300 Read block 1204 len 40960
24572 060/300 Write block 4679 len 32768 data: 3 3 3 3 3 3 3 4
24573 061/300 Read block 568 len 16384
24574 062/300 Write block 3143 len 12288 data: 3 4 6
24575 063/300 Write block 1866 len 16384 data: 5 6 6 6
24576 064/300 Write block 2036 len 8192 data: 5 6
24577 065/300 Read block 1499 len 28672
24578 066/300 Read block 2224 len 12288
24579 067/300 Read block 2096 len 28672
24580 068/300 Read block 4586 len 40960
24581 069/300 Flush
24582 070/300 Read block 195 len 28672
24583 071/300 Read block 3521 len 32768
24584 072/300 Read block 1055 len 4096
24585 073/300 Write block 1366 len 16384 data: 4 4 4 4
24586 074/300 Read block 2622 len 16384
24587 075/300 Write block 451 len 32768 data: 6 4 3 2 4 4 3 4
24588 076/300 Read block 4149 len 24576
24589 077/300 Read block 2361 len 24576
24590 078/300 Read block 3508 len 16384
24591 079/300 Write block 3688 len 12288 data: 4 3 4
24592 080/300 Read block 1284 len 16384
24593 081/300 Write block 2358 len 28672 data: 5 5 4 4 4 3 6
24594 082/300 Flush
24595 083/300 Read block 4457 len 20480
24596 084/300 Write block 2574 len 24576 data: 7 5 5 5 5 3
24597 085/300 Read block 404 len 28672
24598 086/300 Write block 4939 len 12288 data: 9 9 6
24599 087/300 Write block 3166 len 32768 data: 5 5 4 3 4 4 3 3
24600 088/300 Write block 3100 len 12288 data: 4 3 3
24601 089/300 Flush
24602 090/300 Flush
24603 091/300 Flush
24604 092/300 Read block 1912 len 12288
24605 093/300 Write block 2491 len 40960 data: 5 4 4 3 3 4 4 4 4 4
24606 094/300 Read block 3808 len 40960
24607 095/300 Read block 621 len 16384
24608 096/300 Read block 4757 len 4096
24609 097/300 Read block 1941 len 36864
24610 098/300 Read block 4805 len 32768
24611 099/300 Flush
24612 100/300 Write block 450 len 32768 data: 6 7 5 4 3 5 5 4
24613 101/300 Write block 954 len 16384 data: 2 3 4 3
24614 102/300 Write block 3868 len 20480 data: 7 7 6 5 5
24615 103/300 Write block 120 len 36864 data: 6 6 6 5 5 5 5 6 4
24616 104/300 Read block 3707 len 28672
24617 105/300 Write block 798 len 24576 data: 6 2 2 3 3 3
24618 106/300 Read block 221 len 24576
24619 107/300 Write block 3269 len 40960 data: 5 4 5 6 7 7 6 6 6 5
24620 108/300 Write block 2300 len 36864 data: 9 9 8 6 7 8 4 4 3
24621 109/300 Write block 662 len 20480 data: 6 6 7 6 6
24622 110/300 Flush
24623 111/300 Write block 3274 len 20480 data: 8 7 7 7 6
24624 112/300 Read block 4055 len 4096
24625 113/300 Write block 286 len 28672 data: 3 3 3 3 3 3 4
24626 114/300 Write block 2682 len 32768 data: 7 7 7 7 7 7 5 5
24627 115/300 Read block 2999 len 36864
24628 116/300 Read block 75 len 4096
24629 117/300 Flush
24630 118/300 Read block 4374 len 36864
24631 119/300 Read block 157 len 40960
24632 120/300 Write block 4626 len 8192 data: 4 3
24633 121/300 Write block 4670 len 16384 data: 4 6 7 7
24634 122/300 Read block 3403 len 4096
24635 123/300 Write block 2326 len 32768 data: 4 3 6 6 7 5 4 3
24636 124/300 Write block 250 len 40960 data: 5 5 5 4 4 5 5 4 4 3
24637 125/300 Read block 367 len 36864
24638 126/300 Read block 1861 len 20480
24639 127/300 Read block 2597 len 4096
24640 128/300 Read block 3312 len 36864
24641 129/300 Read block 2061 len 16384
24642 130/300 Write block 1564 len 28672 data: 7 7 8 6 4 4 4
24643 131/300 Write block 3745 len 12288 data: 7 5 7
24644 132/300 Flush
24645 133/300 Read block 672 len 20480
24646 134/300 Write block 2631 len 4096 data: 3
24647 135/300 Read block 3811 len 36864
24648 136/300 Write block 4560 len 8192 data: 4 4
24649 137/300 Write block 4847 len 4096 data: 5
24650 138/300 Read block 3543 len 36864
24651 139/300 Read block 4413 len 4096
24652 140/300 Read block 3915 len 8192
24653 141/300 Read block 1056 len 8192
24654 142/300 Write block 758 len 16384 data: 4 3 3 3
24655 143/300 Write block 127 len 16384 data: 7 5 6 6
24656 144/300 Write block 4284 len 4096 data: 4
24657 145/300 Read block 3392 len 28672
24658 146/300 Read block 2977 len 8192
24659 147/300 Write block 4018 len 24576 data: 7 6 7 8 8 8
24660 148/300 Read block 57 len 20480
24661 149/300 Read block 4420 len 16384
24662 150/300 Write block 1871 len 8192 data: 6 6
24663 151/300 Write block 4419 len 4096 data: 3
24664 152/300 Read block 947 len 36864
24665 153/300 Read block 84 len 16384
24666 154/300 Read block 716 len 8192
24667 155/300 Flush
24668 156/300 Flush
24669 157/300 Write block 496 len 40960 data: 6 5 5 6 5 5 3 3 3 3
24670 158/300 Write block 446 len 24576 data: 6 5 6 6 7 8
24671 159/300 Read block 3720 len 20480
24672 160/300 Write block 814 len 8192 data: 3 4
24673 161/300 Read block 1751 len 28672
24674 162/300 Read block 4602 len 12288
24675 163/300 Read block 4736 len 20480
24676 164/300 Write block 4427 len 8192 data: 6 5
24677 165/300 Read block 1607 len 4096
24678 166/300 Write block 2083 len 36864 data: 6 4 4 3 3 3 4 4 4
24679 167/300 Read block 3026 len 32768
24680 168/300 Write block 237 len 16384 data: 4 4 4 3
24681 169/300 Write block 4323 len 12288 data: 6 6 4
24682 170/300 Write block 2509 len 28672 data: 7 5 5 5 3 2 2
24683 171/300 Read block 4682 len 28672
24684 172/300 Flush
24685 173/300 Read block 2414 len 40960
24686 174/300 Read block 4614 len 16384
24687 175/300 Write block 491 len 28672 data: 9 9 7 5 5 7 6
24688 176/300 Write block 4076 len 4096 data: 3
24689 177/300 Read block 1152 len 40960
24690 178/300 Write block 3682 len 36864 data: 3 3 2 3 3 4 5 4 5
24691 179/300 Read block 4849 len 4096
24692 180/300 Write block 2077 len 20480 data: 7 7 7 5 5
24693 181/300 Write block 2091 len 16384 data: 5 3 3 3
24694 182/300 Read block 3517 len 28672
24695 183/300 Read block 610 len 24576
24696 184/300 Read block 1580 len 20480
24697 185/300 Read block 2667 len 36864
24698 186/300 Read block 1868 len 32768
24699 187/300 Write block 3841 len 12288 data: 5 4 5
24700 188/300 Flush
24701 189/300 Read block 3845 len 16384
24702 190/300 Write block 685 len 24576 data: 7 6 5 5 4 4
24703 191/300 Read block 504 len 40960
24704 192/300 Read block 1239 len 16384
24705 193/300 Read block 4531 len 28672
24706 194/300 Write block 698 len 32768 data: 4 3 3 5 3 5 4 4
24707 195/300 Write block 1736 len 16384 data: 5 6 5 3
24708 196/300 Read block 3056 len 40960
24709 197/300 Read block 987 len 32768
24710 198/300 Read block 410 len 40960
24711 199/300 Flush
24712 200/300 Read block 1212 len 36864
24713 201/300 Read block 263 len 40960
24714 202/300 Write block 3170 len 8192 data: 5 5
24715 203/300 Write block 3667 len 28672 data: 5 6 6 4 5 5 5
24716 204/300 Flush
24717 205/300 Flush
24718 206/300 Read block 3092 len 20480
24719 207/300 Write block 3823 len 12288 data: 9 8 8
24720 208/300 Read block 2515 len 40960
24721 209/300 Write block 1337 len 28672 data: 4 4 5 5 5 4 4
24722 210/300 Read block 2021 len 28672
24723 211/300 Write block 1332 len 20480 data: 8 9 8 6 5
24724 212/300 Write block 2190 len 36864 data: 4 6 6 5 5 3 5 4 4
24725 213/300 Read block 4566 len 32768
24726 214/300 Read block 650 len 8192
24727 215/300 Write block 4256 len 24576 data: 5 4 4 4 3 3
24728 216/300 Read block 3289 len 8192
24729 217/300 Read block 2086 len 20480
24730 218/300 Read block 4161 len 40960
24731 219/300 Read block 1403 len 4096
24732 220/300 Write block 2622 len 16384 data: 7 4 3 3
24733 221/300 Flush
24734 222/300 Read block 3396 len 8192
24735 223/300 Write block 4493 len 40960 data: 7 4 7 7 7 3 4 4 6 6
24736 224/300 Flush
24737 225/300 Flush
24738 226/300 Read block 428 len 4096
24739 227/300 Write block 4347 len 8192 data: 5 4
24740 228/300 Read block 598 len 40960
24741 229/300 Flush
24742 230/300 Write block 4385 len 20480 data: 5 6 6 5 5
24743 231/300 Write block 2920 len 24576 data: 6 7 7 7 5 4
24744 232/300 Read block 111 len 20480
24745 233/300 Write block 1419 len 20480 data: 5 4 6 6 5
24746 234/300 Write block 4222 len 8192 data: 5 5
24747 235/300 Write block 840 len 36864 data: 5 6 3 3 3 3 4 4 4
24748 236/300 Flush
24749 237/300 Read block 2959 len 28672
24750 238/300 Read block 2995 len 24576
24751 239/300 Write block 3648 len 24576 data: 2 2 2 3 3 3
24752 240/300 Write block 1289 len 12288 data: 11 11 8
24753 241/300 Write block 1443 len 12288 data: 4 4 5
24754 242/300 Read block 4451 len 28672
24755 243/300 Write block 2702 len 32768 data: 4 5 7 8 8 8 6 8
24756 244/300 Read block 3275 len 36864
24757 245/300 Read block 4104 len 36864
24758 246/300 Read block 2706 len 24576
24759 247/300 Write block 2511 len 24576 data: 6 6 4 3 3 2
24760 248/300 Read block 2752 len 24576
24761 249/300 Write block 3069 len 36864 data: 4 5 4 4 3 3 3 2 3
24762 250/300 Read block 2028 len 12288
24763 251/300 Read block 2724 len 24576
24764 252/300 Read block 1942 len 20480
24765 253/300 Write block 1889 len 32768 data: 5 7 6 5 5 4 3 3
24766 254/300 Write block 30 len 32768 data: 4 4 4 4 6 6 5 4
24767 255/300 Write block 3433 len 28672 data: 6 5 6 6 7 6 7
24768 256/300 Write block 3148 len 40960 data: 6 6 6 6 7 7 5 5 5 5
24769 257/300 Flush
24770 258/300 Write block 177 len 20480 data: 5 7 6 7 6
24771 259/300 Write block 1947 len 20480 data: 5 6 6 7 9
24772 260/300 Read block 199 len 40960
24773 261/300 Read block 916 len 36864
24774 262/300 Write block 930 len 16384 data: 4 5 5 4
24775 263/300 Write block 4849 len 16384 data: 5 4 4 5
24776 264/300 Read block 1023 len 28672
24777 265/300 Read block 4405 len 40960
24778 266/300 Write block 2135 len 40960 data: 4 4 5 5 6 6 7 8 8 6
24779 267/300 Flush
24780 268/300 Read block 3066 len 4096
24781 269/300 Read block 858 len 12288
24782 270/300 Read block 4789 len 24576
24783 271/300 Write block 2863 len 24576 data: 3 3 3 3 5 4
24784 272/300 Read block 1357 len 32768
24785 273/300 Read block 4280 len 4096
24786 274/300 Flush
24787 275/300 Read block 4551 len 4096
24788 276/300 Flush
24789 277/300 Write block 1765 len 12288 data: 5 5 5
24790 278/300 Flush
24791 279/300 Write block 125 len 24576 data: 6 6 8 6 7 7
24792 280/300 Write block 2632 len 20480 data: 5 6 5 4 4
24793 281/300 Write block 1785 len 24576 data: 4 3 3 3 3 2
24794 282/300 Read block 3891 len 4096
24795 283/300 Write block 3729 len 16384 data: 10 9 9 9
24796 284/300 Read block 2931 len 4096
24797 285/300 Read block 146 len 32768
24798 286/300 Read block 1558 len 40960
24799 287/300 Read block 4781 len 40960
24800 288/300 Write block 1392 len 36864 data: 6 5 5 6 6 7 7 8 9
24801 289/300 Read block 2434 len 8192
24802 290/300 Write block 4598 len 36864 data: 2 2 2 4 4 4 3 4 4
24803 291/300 Read block 3145 len 20480
24804 292/300 Read block 413 len 12288
24805 293/300 Write block 3241 len 36864 data: 7 6 6 5 7 6 6 6 7
24806 294/300 Read block 2045 len 8192
24807 295/300 Write block 3380 len 4096 data: 4
24808 296/300 Read block 4762 len 4096
24809 297/300 Read block 126 len 40960
24810 298/300 Read block 4861 len 4096
24811 299/300 Write block 4836 len 28672 data: 3 3 2 5 4 4 4
24812 300/300 Read block 2906 len 28672
24813 301/300 Write block 3563 len 28672 data: 4 4 4 4 4 5 4
24814 Main task has work to do, go find it
24815 got dsc Start(1)
24816 start 1
24817 Nov 19 17:13:13.143 INFO request completed, latency_us: 179, response_code: 204, uri: /start/cid/1, method: POST, req_id: 14c803dc-dde8-4421-86b6-51b75710c5b1, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
24818 Replay: started 1, returned:Ok(())
24819 ----------------------------------------------------------------
24820 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24821 States: Active Offline Active
24822 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24823 7340 Acked Read 2 Done Sent Done false
24824 7341 Acked Read 1 Done Sent Done false
24825 7342 Acked Write 9 Done Sent Done false
24826 7343 Acked Flush 0 Done Sent Done false
24827 7344 Acked Write 8 Done Sent Done false
24828 7345 Acked Read 5 Done Sent Done false
24829 7346 Acked Read 7 Done Sent Done false
24830 7347 Acked Write 6 Done Sent Done false
24831 7348 Acked Write 7 Done Sent Done false
24832 7349 Acked Write 1 Done Sent Done false
24833 7350 Acked Read 7 Done Sent Done false
24834 7351 Acked Write 2 Done Sent Done false
24835 7352 Acked Read 4 Done Sent Done false
24836 7353 Acked Write 5 Done Sent Done false
24837 7354 Acked Flush 0 Done Sent Done false
24838 7355 Acked Read 9 Done Sent Done false
24839 7356 Acked Read 7 Done Sent Done false
24840 7357 Acked Write 4 Done Sent Done false
24841 7358 Acked Write 3 Done Sent Done false
24842 7359 Acked Read 1 Done Sent Done false
24843 7360 Acked Write 4 Done Sent Done false
24844 7361 Acked Read 5 Done Sent Done false
24845 7362 Acked Read 7 Done Sent Done false
24846 7363 Acked Write 6 Done Sent Done false
24847 7364 Acked Read 8 Done Sent Done false
24848 7365 Acked Read 2 Done Sent Done false
24849 7366 Acked Write 1 Done Sent Done false
24850 7367 Acked Flush 0 Done Sent Done false
24851 7368 Acked Read 3 Done Sent Done false
24852 7369 Acked Read 9 Done Sent Done false
24853 7370 Acked Read 3 Done Sent Done false
24854 7371 Acked Read 8 Done Sent Done false
24855 7372 Acked Read 8 Done Sent Done false
24856 7373 Acked Read 6 Done Sent Done false
24857 7374 Acked Read 6 Done Sent Done false
24858 7375 Acked Write 4 Done Sent Done false
24859 7376 Acked Write 1 Done Sent Done false
24860 7377 Acked Read 5 Done Sent Done false
24861 7378 Acked Read 4 Done Sent Done false
24862 7379 Acked Flush 0 Done Sent Done false
24863 7380 Acked Read 9 Done Sent Done false
24864 7381 Acked Read 1 Done Sent Done false
24865 7382 Acked Read 6 Done Sent Done false
24866 7383 Acked Write 9 Done Sent Done false
24867 7384 Acked Write 4 Done Sent Done false
24868 7385 Acked Read 5 Done Sent Done false
24869 7386 Acked Write 1 Done Sent Done false
24870 7387 Acked Read 8 Done Sent Done false
24871 7388 Acked Read 4 Done Sent Done false
24872 7389 Acked Write 9 Done Sent Done false
24873 7390 Acked Read 2 Done Sent Done false
24874 7391 Acked Read 3 Done Sent Done false
24875 7392 Acked Write 4 Done Sent Done false
24876 7393 Acked Flush 0 Done Sent Done false
24877 7394 Acked Read 10 Done Sent Done false
24878 7395 Acked Read 4 Done Sent Done false
24879 7396 Acked Read 6 Done Sent Done false
24880 7397 Acked Write 3 Done Sent Done false
24881 7398 Acked Read 10 Done Sent Done false
24882 7399 Acked Write 8 Done Sent Done false
24883 7400 Acked Read 4 Done Sent Done false
24884 7401 Acked Write 3 Done Sent Done false
24885 7402 Acked Write 4 Done Sent Done false
24886 7403 Acked Write 2 Done Sent Done false
24887 7404 Acked Read 7 Done Sent Done false
24888 7405 Acked Read 3 Done Sent Done false
24889 7406 Acked Read 7 Done Sent Done false
24890 7407 Acked Read 10 Done Sent Done false
24891 7408 Acked Flush 0 Done Sent Done false
24892 7409 Acked Read 7 Done Sent Done false
24893 7410 Acked Read 8 Done Sent Done false
24894 7411 Acked Read 1 Done Sent Done false
24895 7412 Acked Write 4 Done Sent Done false
24896 7413 Acked Read 4 Done Sent Done false
24897 7414 Acked Write 8 Done Sent Done false
24898 7415 Acked Read 6 Done Sent Done false
24899 7416 Acked Read 6 Done Sent Done false
24900 7417 Acked Read 4 Done Sent Done false
24901 7418 Acked Write 3 Done Sent Done false
24902 7419 Acked Read 4 Done Sent Done false
24903 7420 Acked Write 7 Done Sent Done false
24904 7421 Acked Flush 0 Done Sent Done false
24905 7422 Acked Read 5 Done Sent Done false
24906 7423 Acked Write 6 Done Sent Done false
24907 7424 Acked Read 7 Done Sent Done false
24908 7425 Acked Write 3 Done Sent Done false
24909 7426 Acked Write 8 Done Sent Done false
24910 7427 Acked Write 3 Done Sent Done false
24911 7428 Acked Flush 0 Done Sent Done false
24912 7429 Acked Flush 0 Done Sent Done false
24913 7430 Acked Flush 0 Done Sent Done false
24914 7431 Acked Read 3 Done Sent Done false
24915 7432 Acked Write 10 Done Sent Done false
24916 7433 Acked Read 10 Done Sent Done false
24917 7434 Acked Read 4 Done Sent Done false
24918 7435 Acked Read 1 Done Sent Done false
24919 7436 Acked Read 9 Done Sent Done false
24920 7437 Acked Read 8 Done Sent Done false
24921 7438 Acked Flush 0 Done Sent Done false
24922 7439 Acked Write 8 Done Sent Done false
24923 7440 Acked Write 4 Done Sent Done false
24924 7441 Acked Write 5 Done Sent Done false
24925 7442 Acked Write 9 Done Sent Done false
24926 7443 Acked Read 7 Done Sent Done false
24927 7444 Acked Write 6 Done Sent Done false
24928 7445 Acked Read 6 Done Sent Done false
24929 7446 Acked Write 10 Done Sent Done false
24930 7447 Acked Write 9 Done Sent Done false
24931 7448 Acked Write 5 Done Sent Done false
24932 7449 Acked Flush 0 Done Sent Done false
24933 7450 Acked Write 5 Done Sent Done false
24934 7451 Acked Read 1 Done Sent Done false
24935 7452 Acked Write 7 Done Sent Done false
24936 7453 Acked Write 8 Done Sent Done false
24937 7454 Acked Read 9 Done Sent Done false
24938 7455 Acked Read 1 Done Sent Done false
24939 7456 Acked Flush 0 Done Sent Done false
24940 7457 Acked Read 9 Done Sent Done false
24941 7458 Acked Read 10 Done Sent Done false
24942 7459 Acked Write 2 Done Sent Done false
24943 7460 Acked Write 4 Done Sent Done false
24944 7461 Acked Read 1 Done Sent Done false
24945 7462 Acked Write 8 Done Sent Done false
24946 7463 Acked Write 10 Done Sent Done false
24947 7464 Acked Read 9 Done Sent Done false
24948 7465 Acked Read 5 Done Sent Done false
24949 7466 Acked Read 1 Done Sent Done false
24950 7467 Acked Read 9 Done Sent Done false
24951 7468 Acked Read 4 Done Sent Done false
24952 7469 Acked Write 7 Done Sent Done false
24953 7470 Acked Write 3 Done Sent Done false
24954 7471 Acked Flush 0 Done Sent Done false
24955 7472 Acked Read 5 Done Sent Done false
24956 7473 Acked Write 1 Done Sent Done false
24957 7474 Acked Read 9 Done Sent Done false
24958 7475 Acked Write 2 Done Sent Done false
24959 7476 Acked Write 1 Done Sent Done false
24960 7477 Acked Read 9 Done Sent Done false
24961 7478 Acked Read 1 Done Sent Done false
24962 7479 Acked Read 2 Done Sent Done false
24963 7480 Acked Read 2 Done Sent Done false
24964 7481 Acked Write 4 Done Sent Done false
24965 7482 Acked Write 4 Done Sent Done false
24966 7483 Acked Write 1 Done Sent Done false
24967 7484 Acked Read 7 Done Sent Done false
24968 7485 Acked Read 2 Done Sent Done false
24969 7486 Acked Write 6 Done Sent Done false
24970 7487 Acked Read 5 Done Sent Done false
24971 7488 Acked Read 4 Done Sent Done false
24972 7489 Acked Write 2 Done Sent Done false
24973 7490 Acked Write 1 Done Sent Done false
24974 7491 Acked Read 9 Done Sent Done false
24975 7492 Acked Read 4 Done Sent Done false
24976 7493 Acked Read 2 Done Sent Done false
24977 7494 Acked Flush 0 Done Sent Done false
24978 7495 Acked Flush 0 Done Sent Done false
24979 7496 Acked Write 10 Done Sent Done false
24980 7497 Acked Write 6 Done Sent Done false
24981 7498 Acked Read 5 Done Sent Done false
24982 7499 Acked Write 2 Done Sent Done false
24983 7500 Acked Read 7 Done Sent Done false
24984 7501 Acked Read 3 Done Sent Done false
24985 7502 Acked Read 5 Done Sent Done false
24986 7503 Acked Write 2 Done Sent Done false
24987 7504 Acked Read 1 Done Sent Done false
24988 7505 Acked Write 9 Done Sent Done false
24989 7506 Acked Read 8 Done Sent Done false
24990 7507 Acked Write 4 Done Sent Done false
24991 7508 Acked Write 3 Done Sent Done false
24992 7509 Acked Write 7 Done Sent Done false
24993 7510 Acked Read 7 Done Sent Done false
24994 7511 Acked Flush 0 Done Sent Done false
24995 7512 Acked Read 10 Done Sent Done false
24996 7513 Acked Read 4 Done Sent Done false
24997 7514 Acked Write 7 Done Sent Done false
24998 7515 Acked Write 1 Done Sent Done false
24999 7516 Acked Read 10 Done Sent Done false
25000 7517 Acked Write 9 Done Sent Done false
25001 7518 Acked Read 1 Done Sent Done false
25002 7519 Acked Write 5 Done Sent Done false
25003 7520 Acked Write 4 Done Sent Done false
25004 7521 Acked Read 7 Done Sent Done false
25005 7522 Acked Read 6 Done Sent Done false
25006 7523 Acked Read 5 Done Sent Done false
25007 7524 Acked Read 9 Done Sent Done false
25008 7525 Acked Read 8 Done Sent Done false
25009 7526 Acked Write 3 Done Sent Done false
25010 7527 Acked Flush 0 Done Sent Done false
25011 7528 Acked Read 4 Done Sent Done false
25012 7529 Acked Write 6 Done Sent Done false
25013 7530 Acked Read 10 Done Sent Done false
25014 7531 Acked Read 4 Done Sent Done false
25015 7532 Acked Read 7 Done Sent Done false
25016 7533 Acked Write 8 Done Sent Done false
25017 7534 Acked Write 4 Done Sent Done false
25018 7535 Acked Read 10 Done Sent Done false
25019 7536 Acked Read 8 Done Sent Done false
25020 7537 Acked Read 10 Done Sent Done false
25021 7538 Acked Flush 0 Done Sent Done false
25022 7539 Acked Read 9 Done Sent Done false
25023 7540 Acked Read 10 Done Sent Done false
25024 7541 Acked Write 2 Done Sent Done false
25025 7542 Acked Write 7 Done Sent Done false
25026 7543 Acked Flush 0 Done Sent Done false
25027 7544 Acked Flush 0 Done Sent Done false
25028 7545 Acked Read 5 Done Sent Done false
25029 7546 Acked Write 3 Done Sent Done false
25030 7547 Acked Read 10 Done Sent Done false
25031 7548 Acked Write 7 Done Sent Done false
25032 7549 Acked Read 7 Done Sent Done false
25033 7550 Acked Write 5 Done Sent Done false
25034 7551 Acked Write 9 Done Sent Done false
25035 7552 Acked Read 8 Done Sent Done false
25036 7553 Acked Read 2 Done Sent Done false
25037 7554 Acked Write 6 Done Sent Done false
25038 7555 Acked Read 2 Done Sent Done false
25039 7556 Acked Read 5 Done Sent Done false
25040 7557 Acked Read 10 Done Sent Done false
25041 7558 Acked Read 1 Done Sent Done false
25042 7559 Acked Write 4 Done Sent Done false
25043 7560 Acked Flush 0 Done Sent Done false
25044 7561 Acked Read 2 Done Sent Done false
25045 7562 Acked Write 10 Done Sent Done false
25046 7563 Acked Flush 0 Done Sent Done false
25047 7564 Acked Flush 0 Done Sent Done false
25048 7565 Acked Read 1 Done Sent Done false
25049 7566 Acked Write 2 Done Sent Done false
25050 7567 Acked Read 10 Done Sent Done false
25051 7568 Acked Flush 0 Done Sent Done false
25052 7569 Acked Write 5 Done Sent Done false
25053 7570 Acked Write 6 Done Sent Done false
25054 7571 Acked Read 5 Done Sent Done false
25055 7572 Acked Write 5 Done Sent Done false
25056 7573 Acked Write 2 Done Sent Done false
25057 7574 Acked Write 9 Done Sent Done false
25058 7575 Acked Flush 0 Done Sent Done false
25059 7576 Acked Read 7 Done Sent Done false
25060 7577 Acked Read 6 Done Sent Done false
25061 7578 Acked Write 6 Done Sent Done false
25062 7579 Acked Write 3 Done Sent Done false
25063 7580 Acked Write 3 Done Sent Done false
25064 7581 Acked Read 7 Done Sent Done false
25065 7582 Acked Write 8 Done Sent Done false
25066 7583 Acked Read 9 Done Sent Done false
25067 7584 Acked Read 9 Done Sent Done false
25068 7585 Acked Read 6 Done Sent Done false
25069 7586 Acked Write 6 Done Sent Done false
25070 7587 Acked Read 6 Done Sent Done false
25071 7588 Acked Write 9 Done Sent Done false
25072 7589 Acked Read 3 Done Sent Done false
25073 7590 Acked Read 6 Done Sent Done false
25074 7591 Acked Read 5 Done Sent Done false
25075 7592 Acked Write 8 Done Sent Done false
25076 7593 Acked Write 8 Done Sent Done false
25077 7594 Acked Write 7 Done Sent Done false
25078 7595 Acked Write 10 Done Sent Done false
25079 7596 Acked Flush 0 Done Sent Done false
25080 7597 Acked Write 5 Done Sent Done false
25081 7598 Acked Write 5 Done Sent Done false
25082 7599 Acked Read 10 Done Sent Done false
25083 7600 Acked Read 9 Done Sent Done false
25084 7601 Acked Write 4 Done Sent Done false
25085 7602 Acked Write 4 Done Sent Done false
25086 7603 Acked Read 7 Done Sent Done false
25087 7604 Acked Read 10 Done Sent Done false
25088 7605 Acked Write 10 Done Sent Done false
25089 7606 Acked Flush 0 Done Sent Done false
25090 7607 Acked Read 1 Done Sent Done false
25091 7608 Acked Read 3 Done Sent Done false
25092 7609 Acked Read 6 Done Sent Done false
25093 7610 Acked Write 6 Done Sent Done false
25094 7611 Acked Read 8 Done Sent Done false
25095 7612 Acked Read 1 Done Sent Done false
25096 7613 Acked Flush 0 Done Sent Done false
25097 7614 Acked Read 1 Done Sent Done false
25098 7615 Acked Flush 0 Done Sent Done false
25099 7616 Acked Write 3 Done Sent Done false
25100 7617 Acked Flush 0 Done Sent Done false
25101 7618 Acked Write 6 Done Sent Done false
25102 7619 Acked Write 5 Done Sent Done false
25103 7620 Acked Write 6 Done Sent Done false
25104 7621 Acked Read 1 Done Sent Done false
25105 7622 Acked Write 4 Done Sent Done false
25106 7623 Acked Read 1 Done Sent Done false
25107 7624 Acked Read 8 Done Sent Done false
25108 7625 Acked Read 10 Done Sent Done false
25109 7626 Acked Read 10 Done Sent Done false
25110 7627 Acked Write 9 Done Sent Done false
25111 7628 Acked Read 2 Done Sent Done false
25112 7629 Acked Write 9 Done Sent Done false
25113 7630 Acked Read 5 Done Sent Done false
25114 7631 Acked Read 3 Done Sent Done false
25115 7632 Acked Write 9 Done Sent Done false
25116 7633 Acked Read 2 Done Sent Done false
25117 7634 Acked Write 1 Done Sent Done false
25118 7635 Acked Read 1 Done Sent Done false
25119 7636 Acked Read 10 Done Sent Done false
25120 7637 Acked Read 1 Done Sent Done false
25121 7638 Acked Write 7 Done Sent Done false
25122 7639 Acked Read 7 Done Sent Done false
25123 7640 Acked Write 7 Done Sent Done false
25124 STATES DS:0 DS:1 DS:2 TOTAL
25125 Sent 0 301 0 301
25126 Done 301 0 301 602
25127 Skipped 0 0 0 0
25128 Error 0 0 0 0
25129 Last Flush: 7617 7339 7617
25130 Downstairs last five completed: 7339 7338 7337 7336 7335
25131 Upstairs last five completed: 7640 7639 7638 7637 7636
25132 CLIENT: Up:0 ds:301 act:2
25133 [8820] got start action
25134 [8820] I am going to restart
25135 Starting downstairs at port 8820
25136 Make output file at "/tmp/dsc/downstairs-8820.txt"
25137 [8820][1] reports Starting
25138 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1224)
25139 [8820][1] reports Running
25140 ----------------------------------------------------------------
25141 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25142 States: Active Offline Active
25143 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25144 7340 Acked Read 2 Done Sent Done false
25145 7341 Acked Read 1 Done Sent Done false
25146 7342 Acked Write 9 Done Sent Done false
25147 7343 Acked Flush 0 Done Sent Done false
25148 7344 Acked Write 8 Done Sent Done false
25149 7345 Acked Read 5 Done Sent Done false
25150 7346 Acked Read 7 Done Sent Done false
25151 7347 Acked Write 6 Done Sent Done false
25152 7348 Acked Write 7 Done Sent Done false
25153 7349 Acked Write 1 Done Sent Done false
25154 7350 Acked Read 7 Done Sent Done false
25155 7351 Acked Write 2 Done Sent Done false
25156 7352 Acked Read 4 Done Sent Done false
25157 7353 Acked Write 5 Done Sent Done false
25158 7354 Acked Flush 0 Done Sent Done false
25159 7355 Acked Read 9 Done Sent Done false
25160 7356 Acked Read 7 Done Sent Done false
25161 7357 Acked Write 4 Done Sent Done false
25162 7358 Acked Write 3 Done Sent Done false
25163 7359 Acked Read 1 Done Sent Done false
25164 7360 Acked Write 4 Done Sent Done false
25165 7361 Acked Read 5 Done Sent Done false
25166 7362 Acked Read 7 Done Sent Done false
25167 7363 Acked Write 6 Done Sent Done false
25168 7364 Acked Read 8 Done Sent Done false
25169 7365 Acked Read 2 Done Sent Done false
25170 7366 Acked Write 1 Done Sent Done false
25171 7367 Acked Flush 0 Done Sent Done false
25172 7368 Acked Read 3 Done Sent Done false
25173 7369 Acked Read 9 Done Sent Done false
25174 7370 Acked Read 3 Done Sent Done false
25175 7371 Acked Read 8 Done Sent Done false
25176 7372 Acked Read 8 Done Sent Done false
25177 7373 Acked Read 6 Done Sent Done false
25178 7374 Acked Read 6 Done Sent Done false
25179 7375 Acked Write 4 Done Sent Done false
25180 7376 Acked Write 1 Done Sent Done false
25181 7377 Acked Read 5 Done Sent Done false
25182 7378 Acked Read 4 Done Sent Done false
25183 7379 Acked Flush 0 Done Sent Done false
25184 7380 Acked Read 9 Done Sent Done false
25185 7381 Acked Read 1 Done Sent Done false
25186 7382 Acked Read 6 Done Sent Done false
25187 7383 Acked Write 9 Done Sent Done false
25188 7384 Acked Write 4 Done Sent Done false
25189 7385 Acked Read 5 Done Sent Done false
25190 7386 Acked Write 1 Done Sent Done false
25191 7387 Acked Read 8 Done Sent Done false
25192 7388 Acked Read 4 Done Sent Done false
25193 7389 Acked Write 9 Done Sent Done false
25194 7390 Acked Read 2 Done Sent Done false
25195 7391 Acked Read 3 Done Sent Done false
25196 7392 Acked Write 4 Done Sent Done false
25197 7393 Acked Flush 0 Done Sent Done false
25198 7394 Acked Read 10 Done Sent Done false
25199 7395 Acked Read 4 Done Sent Done false
25200 7396 Acked Read 6 Done Sent Done false
25201 7397 Acked Write 3 Done Sent Done false
25202 7398 Acked Read 10 Done Sent Done false
25203 7399 Acked Write 8 Done Sent Done false
25204 7400 Acked Read 4 Done Sent Done false
25205 7401 Acked Write 3 Done Sent Done false
25206 7402 Acked Write 4 Done Sent Done false
25207 7403 Acked Write 2 Done Sent Done false
25208 7404 Acked Read 7 Done Sent Done false
25209 7405 Acked Read 3 Done Sent Done false
25210 7406 Acked Read 7 Done Sent Done false
25211 7407 Acked Read 10 Done Sent Done false
25212 7408 Acked Flush 0 Done Sent Done false
25213 7409 Acked Read 7 Done Sent Done false
25214 7410 Acked Read 8 Done Sent Done false
25215 7411 Acked Read 1 Done Sent Done false
25216 7412 Acked Write 4 Done Sent Done false
25217 7413 Acked Read 4 Done Sent Done false
25218 7414 Acked Write 8 Done Sent Done false
25219 7415 Acked Read 6 Done Sent Done false
25220 7416 Acked Read 6 Done Sent Done false
25221 7417 Acked Read 4 Done Sent Done false
25222 7418 Acked Write 3 Done Sent Done false
25223 7419 Acked Read 4 Done Sent Done false
25224 7420 Acked Write 7 Done Sent Done false
25225 7421 Acked Flush 0 Done Sent Done false
25226 7422 Acked Read 5 Done Sent Done false
25227 7423 Acked Write 6 Done Sent Done false
25228 7424 Acked Read 7 Done Sent Done false
25229 7425 Acked Write 3 Done Sent Done false
25230 7426 Acked Write 8 Done Sent Done false
25231 7427 Acked Write 3 Done Sent Done false
25232 7428 Acked Flush 0 Done Sent Done false
25233 7429 Acked Flush 0 Done Sent Done false
25234 7430 Acked Flush 0 Done Sent Done false
25235 7431 Acked Read 3 Done Sent Done false
25236 7432 Acked Write 10 Done Sent Done false
25237 7433 Acked Read 10 Done Sent Done false
25238 7434 Acked Read 4 Done Sent Done false
25239 7435 Acked Read 1 Done Sent Done false
25240 7436 Acked Read 9 Done Sent Done false
25241 7437 Acked Read 8 Done Sent Done false
25242 7438 Acked Flush 0 Done Sent Done false
25243 7439 Acked Write 8 Done Sent Done false
25244 7440 Acked Write 4 Done Sent Done false
25245 7441 Acked Write 5 Done Sent Done false
25246 7442 Acked Write 9 Done Sent Done false
25247 7443 Acked Read 7 Done Sent Done false
25248 7444 Acked Write 6 Done Sent Done false
25249 7445 Acked Read 6 Done Sent Done false
25250 7446 Acked Write 10 Done Sent Done false
25251 7447 Acked Write 9 Done Sent Done false
25252 7448 Acked Write 5 Done Sent Done false
25253 7449 Acked Flush 0 Done Sent Done false
25254 7450 Acked Write 5 Done Sent Done false
25255 7451 Acked Read 1 Done Sent Done false
25256 7452 Acked Write 7 Done Sent Done false
25257 7453 Acked Write 8 Done Sent Done false
25258 7454 Acked Read 9 Done Sent Done false
25259 7455 Acked Read 1 Done Sent Done false
25260 7456 Acked Flush 0 Done Sent Done false
25261 7457 Acked Read 9 Done Sent Done false
25262 7458 Acked Read 10 Done Sent Done false
25263 7459 Acked Write 2 Done Sent Done false
25264 7460 Acked Write 4 Done Sent Done false
25265 7461 Acked Read 1 Done Sent Done false
25266 7462 Acked Write 8 Done Sent Done false
25267 7463 Acked Write 10 Done Sent Done false
25268 7464 Acked Read 9 Done Sent Done false
25269 7465 Acked Read 5 Done Sent Done false
25270 7466 Acked Read 1 Done Sent Done false
25271 7467 Acked Read 9 Done Sent Done false
25272 7468 Acked Read 4 Done Sent Done false
25273 7469 Acked Write 7 Done Sent Done false
25274 7470 Acked Write 3 Done Sent Done false
25275 7471 Acked Flush 0 Done Sent Done false
25276 7472 Acked Read 5 Done Sent Done false
25277 7473 Acked Write 1 Done Sent Done false
25278 7474 Acked Read 9 Done Sent Done false
25279 7475 Acked Write 2 Done Sent Done false
25280 7476 Acked Write 1 Done Sent Done false
25281 7477 Acked Read 9 Done Sent Done false
25282 7478 Acked Read 1 Done Sent Done false
25283 7479 Acked Read 2 Done Sent Done false
25284 7480 Acked Read 2 Done Sent Done false
25285 7481 Acked Write 4 Done Sent Done false
25286 7482 Acked Write 4 Done Sent Done false
25287 7483 Acked Write 1 Done Sent Done false
25288 7484 Acked Read 7 Done Sent Done false
25289 7485 Acked Read 2 Done Sent Done false
25290 7486 Acked Write 6 Done Sent Done false
25291 7487 Acked Read 5 Done Sent Done false
25292 7488 Acked Read 4 Done Sent Done false
25293 7489 Acked Write 2 Done Sent Done false
25294 7490 Acked Write 1 Done Sent Done false
25295 7491 Acked Read 9 Done Sent Done false
25296 7492 Acked Read 4 Done Sent Done false
25297 7493 Acked Read 2 Done Sent Done false
25298 7494 Acked Flush 0 Done Sent Done false
25299 7495 Acked Flush 0 Done Sent Done false
25300 7496 Acked Write 10 Done Sent Done false
25301 7497 Acked Write 6 Done Sent Done false
25302 7498 Acked Read 5 Done Sent Done false
25303 7499 Acked Write 2 Done Sent Done false
25304 7500 Acked Read 7 Done Sent Done false
25305 7501 Acked Read 3 Done Sent Done false
25306 7502 Acked Read 5 Done Sent Done false
25307 7503 Acked Write 2 Done Sent Done false
25308 7504 Acked Read 1 Done Sent Done false
25309 7505 Acked Write 9 Done Sent Done false
25310 7506 Acked Read 8 Done Sent Done false
25311 7507 Acked Write 4 Done Sent Done false
25312 7508 Acked Write 3 Done Sent Done false
25313 7509 Acked Write 7 Done Sent Done false
25314 7510 Acked Read 7 Done Sent Done false
25315 7511 Acked Flush 0 Done Sent Done false
25316 7512 Acked Read 10 Done Sent Done false
25317 7513 Acked Read 4 Done Sent Done false
25318 7514 Acked Write 7 Done Sent Done false
25319 7515 Acked Write 1 Done Sent Done false
25320 7516 Acked Read 10 Done Sent Done false
25321 7517 Acked Write 9 Done Sent Done false
25322 7518 Acked Read 1 Done Sent Done false
25323 7519 Acked Write 5 Done Sent Done false
25324 7520 Acked Write 4 Done Sent Done false
25325 7521 Acked Read 7 Done Sent Done false
25326 7522 Acked Read 6 Done Sent Done false
25327 7523 Acked Read 5 Done Sent Done false
25328 7524 Acked Read 9 Done Sent Done false
25329 7525 Acked Read 8 Done Sent Done false
25330 7526 Acked Write 3 Done Sent Done false
25331 7527 Acked Flush 0 Done Sent Done false
25332 7528 Acked Read 4 Done Sent Done false
25333 7529 Acked Write 6 Done Sent Done false
25334 7530 Acked Read 10 Done Sent Done false
25335 7531 Acked Read 4 Done Sent Done false
25336 7532 Acked Read 7 Done Sent Done false
25337 7533 Acked Write 8 Done Sent Done false
25338 7534 Acked Write 4 Done Sent Done false
25339 7535 Acked Read 10 Done Sent Done false
25340 7536 Acked Read 8 Done Sent Done false
25341 7537 Acked Read 10 Done Sent Done false
25342 7538 Acked Flush 0 Done Sent Done false
25343 7539 Acked Read 9 Done Sent Done false
25344 7540 Acked Read 10 Done Sent Done false
25345 7541 Acked Write 2 Done Sent Done false
25346 7542 Acked Write 7 Done Sent Done false
25347 7543 Acked Flush 0 Done Sent Done false
25348 7544 Acked Flush 0 Done Sent Done false
25349 7545 Acked Read 5 Done Sent Done false
25350 7546 Acked Write 3 Done Sent Done false
25351 7547 Acked Read 10 Done Sent Done false
25352 7548 Acked Write 7 Done Sent Done false
25353 7549 Acked Read 7 Done Sent Done false
25354 7550 Acked Write 5 Done Sent Done false
25355 7551 Acked Write 9 Done Sent Done false
25356 7552 Acked Read 8 Done Sent Done false
25357 7553 Acked Read 2 Done Sent Done false
25358 7554 Acked Write 6 Done Sent Done false
25359 7555 Acked Read 2 Done Sent Done false
25360 7556 Acked Read 5 Done Sent Done false
25361 7557 Acked Read 10 Done Sent Done false
25362 7558 Acked Read 1 Done Sent Done false
25363 7559 Acked Write 4 Done Sent Done false
25364 7560 Acked Flush 0 Done Sent Done false
25365 7561 Acked Read 2 Done Sent Done false
25366 7562 Acked Write 10 Done Sent Done false
25367 7563 Acked Flush 0 Done Sent Done false
25368 7564 Acked Flush 0 Done Sent Done false
25369 7565 Acked Read 1 Done Sent Done false
25370 7566 Acked Write 2 Done Sent Done false
25371 7567 Acked Read 10 Done Sent Done false
25372 7568 Acked Flush 0 Done Sent Done false
25373 7569 Acked Write 5 Done Sent Done false
25374 7570 Acked Write 6 Done Sent Done false
25375 7571 Acked Read 5 Done Sent Done false
25376 7572 Acked Write 5 Done Sent Done false
25377 7573 Acked Write 2 Done Sent Done false
25378 7574 Acked Write 9 Done Sent Done false
25379 7575 Acked Flush 0 Done Sent Done false
25380 7576 Acked Read 7 Done Sent Done false
25381 7577 Acked Read 6 Done Sent Done false
25382 7578 Acked Write 6 Done Sent Done false
25383 7579 Acked Write 3 Done Sent Done false
25384 7580 Acked Write 3 Done Sent Done false
25385 7581 Acked Read 7 Done Sent Done false
25386 7582 Acked Write 8 Done Sent Done false
25387 7583 Acked Read 9 Done Sent Done false
25388 7584 Acked Read 9 Done Sent Done false
25389 7585 Acked Read 6 Done Sent Done false
25390 7586 Acked Write 6 Done Sent Done false
25391 7587 Acked Read 6 Done Sent Done false
25392 7588 Acked Write 9 Done Sent Done false
25393 7589 Acked Read 3 Done Sent Done false
25394 7590 Acked Read 6 Done Sent Done false
25395 7591 Acked Read 5 Done Sent Done false
25396 7592 Acked Write 8 Done Sent Done false
25397 7593 Acked Write 8 Done Sent Done false
25398 7594 Acked Write 7 Done Sent Done false
25399 7595 Acked Write 10 Done Sent Done false
25400 7596 Acked Flush 0 Done Sent Done false
25401 7597 Acked Write 5 Done Sent Done false
25402 7598 Acked Write 5 Done Sent Done false
25403 7599 Acked Read 10 Done Sent Done false
25404 7600 Acked Read 9 Done Sent Done false
25405 7601 Acked Write 4 Done Sent Done false
25406 7602 Acked Write 4 Done Sent Done false
25407 7603 Acked Read 7 Done Sent Done false
25408 7604 Acked Read 10 Done Sent Done false
25409 7605 Acked Write 10 Done Sent Done false
25410 7606 Acked Flush 0 Done Sent Done false
25411 7607 Acked Read 1 Done Sent Done false
25412 7608 Acked Read 3 Done Sent Done false
25413 7609 Acked Read 6 Done Sent Done false
25414 7610 Acked Write 6 Done Sent Done false
25415 7611 Acked Read 8 Done Sent Done false
25416 7612 Acked Read 1 Done Sent Done false
25417 7613 Acked Flush 0 Done Sent Done false
25418 7614 Acked Read 1 Done Sent Done false
25419 7615 Acked Flush 0 Done Sent Done false
25420 7616 Acked Write 3 Done Sent Done false
25421 7617 Acked Flush 0 Done Sent Done false
25422 7618 Acked Write 6 Done Sent Done false
25423 7619 Acked Write 5 Done Sent Done false
25424 7620 Acked Write 6 Done Sent Done false
25425 7621 Acked Read 1 Done Sent Done false
25426 7622 Acked Write 4 Done Sent Done false
25427 7623 Acked Read 1 Done Sent Done false
25428 7624 Acked Read 8 Done Sent Done false
25429 7625 Acked Read 10 Done Sent Done false
25430 7626 Acked Read 10 Done Sent Done false
25431 7627 Acked Write 9 Done Sent Done false
25432 7628 Acked Read 2 Done Sent Done false
25433 7629 Acked Write 9 Done Sent Done false
25434 7630 Acked Read 5 Done Sent Done false
25435 7631 Acked Read 3 Done Sent Done false
25436 7632 Acked Write 9 Done Sent Done false
25437 7633 Acked Read 2 Done Sent Done false
25438 7634 Acked Write 1 Done Sent Done false
25439 7635 Acked Read 1 Done Sent Done false
25440 7636 Acked Read 10 Done Sent Done false
25441 7637 Acked Read 1 Done Sent Done false
25442 7638 Acked Write 7 Done Sent Done false
25443 7639 Acked Read 7 Done Sent Done false
25444 7640 Acked Write 7 Done Sent Done false
25445 7641 Acked Flush 0 Done Sent Done false
25446 STATES DS:0 DS:1 DS:2 TOTAL
25447 Sent 0 302 0 302
25448 Done 302 0 302 604
25449 Skipped 0 0 0 0
25450 Error 0 0 0 0
25451 Last Flush: 7641 7339 7641
25452 Downstairs last five completed: 7339 7338 7337 7336 7335
25453 Upstairs last five completed: 7641 7640 7639 7638 7637
25454 CLIENT: Up:0 ds:302 act:2
254552024-11-19T17:13:18.905ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254562024-11-19T17:13:18.905ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57319) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254572024-11-19T17:13:18.906ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254582024-11-19T17:13:18.907ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254592024-11-19T17:13:18.907ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254602024-11-19T17:13:18.907ZINFOcrucible: send last flush ID to this DS: 7339 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254612024-11-19T17:13:18.907ZINFOcrucible: Replied this last flush ID: 7339 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254622024-11-19T17:13:18.907ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254632024-11-19T17:13:18.907ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254642024-11-19T17:13:18.907ZINFOcrucible: [1] client re-new 302 jobs since flush 7339 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
254652024-11-19T17:13:18.907ZINFOcrucible: [1] Replayed 302 jobs since flush: 7339 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
25466 ----------------------------------------------------------------
25467 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25468 Downstairs last five completed: 7641 7640 7639 7638 7637
25469 Upstairs last five completed: 7641 7640 7639 7638 7637
25470 CLIENT: Up:0 ds:0 act:3
25471 Replay: All jobs finished
25472 Main task has work to do, go find it
25473 got dsc Stop(1)
25474 stop 1
25475 [8820] Got stop action so:false kr:false
25476 Nov 19 17:13:21.149 INFO request completed, latency_us: 318, response_code: 204, uri: /stop/cid/1, method: POST, req_id: cc6d4fe5-c53c-4e20-966c-667674022c04, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
254772024-11-19T17:13:21.150ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254782024-11-19T17:13:21.150ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254792024-11-19T17:13:21.150ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
254802024-11-19T17:13:21.150ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254812024-11-19T17:13:21.150ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
254822024-11-19T17:13:21.150ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
25483 Nov 19 17:13:21.150 INFO request completed, latency_us: 134, response_code: 200, uri: /state/cid/1, method: GET, req_id: 2fe88964-3f28-4c6f-89e4-312af74e91dd, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
25484 [8820] Exited with: ExitStatus(unix_wait_status(9))
25485 [8820][1] reports Exit
25486 Nov 19 17:13:25.153 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0e560caa-5ae8-4fe5-91b2-f2ce3d9ae9c9, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
25487 001/300 Read block 2724 len 36864
25488 002/300 Read block 1028 len 4096
25489 003/300 Read block 4531 len 12288
25490 004/300 Read block 4896 len 8192
25491 005/300 Write block 2549 len 16384 data: 2 2 3 4
25492 006/300 Write block 2925 len 24576 data: 5 4 3 3 4 8
25493 007/300 Read block 1122 len 28672
25494 008/300 Write block 3043 len 20480 data: 5 5 4 4 3
25495 009/300 Read block 2685 len 8192
25496 010/300 Read block 744 len 8192
25497 011/300 Read block 308 len 36864
25498 012/300 Read block 5 len 28672
25499 013/300 Read block 1037 len 40960
25500 014/300 Read block 2960 len 16384
25501 015/300 Write block 3986 len 16384 data: 3 5 5 7
25502 016/300 Read block 1286 len 36864
25503 017/300 Read block 2595 len 8192
25504 018/300 Read block 3642 len 36864
25505 019/300 Read block 2543 len 36864
25506 020/300 Read block 3833 len 32768
25507 021/300 Write block 3095 len 32768 data: 3 3 4 4 4 5 4 4
25508 022/300 Read block 2117 len 28672
25509 023/300 Read block 729 len 12288
25510 024/300 Write block 1985 len 8192 data: 4 3
25511 025/300 Write block 187 len 36864 data: 3 6 5 4 4 4 4 4 3
25512 026/300 Write block 26 len 32768 data: 3 3 4 3 5 5 5 5
25513 027/300 Write block 2321 len 40960 data: 3 3 2 3 4 5 4 7 7 8
25514 028/300 Read block 29 len 36864
25515 029/300 Write block 4135 len 36864 data: 2 2 5 6 5 4 4 3 3
25516 030/300 Read block 1238 len 12288
25517 031/300 Read block 2552 len 40960
25518 032/300 Write block 2325 len 36864 data: 5 6 5 8 8 9 6 5 4
25519 033/300 Read block 2781 len 40960
25520 034/300 Read block 4009 len 40960
25521 035/300 Write block 996 len 32768 data: 5 5 5 5 4 4 4 3
25522 036/300 Read block 4760 len 24576
25523 037/300 Read block 1241 len 20480
25524 038/300 Write block 2698 len 40960 data: 4 4 4 4 5 6 8 9 9 9
25525 039/300 Read block 3398 len 40960
25526 040/300 Write block 4041 len 12288 data: 4 3 5
25527 041/300 Write block 3602 len 28672 data: 5 4 4 4 3 2 3
25528 042/300 Write block 4716 len 40960 data: 2 3 4 4 4 3 2 2 2 4
25529 043/300 Flush
25530 044/300 Write block 3290 len 12288 data: 4 5 5
25531 045/300 Read block 1289 len 24576
25532 046/300 Read block 2166 len 40960
25533 047/300 Write block 1421 len 12288 data: 7 7 6
25534 048/300 Read block 2450 len 20480
25535 049/300 Read block 2157 len 40960
25536 050/300 Flush
25537 051/300 Write block 1013 len 4096 data: 4
25538 052/300 Write block 813 len 24576 data: 3 4 5 4 4 5
25539 053/300 Read block 4660 len 32768
25540 054/300 Flush
25541 055/300 Read block 1927 len 16384
25542 056/300 Read block 2848 len 4096
25543 057/300 Read block 311 len 8192
25544 058/300 Write block 2646 len 8192 data: 6 5
25545 059/300 Flush
25546 060/300 Read block 90 len 40960
25547 061/300 Read block 2106 len 20480
25548 062/300 Read block 688 len 24576
25549 063/300 Read block 1053 len 28672
25550 064/300 Read block 1677 len 28672
25551 065/300 Write block 2920 len 24576 data: 7 8 8 8 6 6
25552 066/300 Read block 4639 len 20480
25553 067/300 Write block 759 len 40960 data: 4 4 4 3 4 4 4 3 3 3
25554 068/300 Read block 1596 len 16384
25555 069/300 Flush
25556 070/300 Write block 4493 len 36864 data: 8 5 8 8 8 4 5 5 7
25557 071/300 Write block 2449 len 24576 data: 2 3 2 2 2 2
25558 072/300 Write block 2677 len 16384 data: 6 6 6 6
25559 073/300 Write block 1378 len 32768 data: 5 4 5 5 4 4 3 3
25560 074/300 Write block 1510 len 36864 data: 5 4 5 5 3 4 4 4 4
25561 075/300 Read block 1934 len 20480
25562 076/300 Flush
25563 077/300 Write block 4596 len 32768 data: 5 5 3 3 3 5 5 5
25564 078/300 Read block 1816 len 32768
25565 079/300 Flush
25566 080/300 Read block 2294 len 16384
25567 081/300 Read block 1035 len 8192
25568 082/300 Write block 1285 len 4096 data: 7
25569 083/300 Read block 807 len 12288
25570 084/300 Read block 467 len 4096
25571 085/300 Write block 325 len 4096 data: 5
25572 086/300 Write block 918 len 24576 data: 5 5 5 4 4 3
25573 087/300 Read block 631 len 40960
25574 088/300 Write block 4620 len 32768 data: 5 5 6 6 5 3 5 4
25575 089/300 Read block 3689 len 8192
25576 090/300 Write block 3617 len 12288 data: 6 7 7
25577 091/300 Write block 2361 len 20480 data: 5 5 4 7 6
25578 092/300 Write block 388 len 40960 data: 6 6 5 5 4 4 6 5 5 5
25579 093/300 Flush
25580 094/300 Read block 4244 len 20480
25581 095/300 Read block 4148 len 24576
25582 096/300 Write block 2219 len 32768 data: 4 4 5 4 3 2 3 4
25583 097/300 Write block 399 len 8192 data: 5 6
25584 098/300 Write block 1387 len 8192 data: 3 3
25585 099/300 Read block 164 len 12288
25586 100/300 Write block 3794 len 40960 data: 5 4 4 4 4 6 5 6 5 4
25587 101/300 Read block 1997 len 40960
25588 102/300 Read block 586 len 16384
25589 103/300 Read block 4093 len 16384
25590 104/300 Read block 4006 len 28672
25591 105/300 Write block 227 len 12288 data: 4 4 3
25592 106/300 Write block 1340 len 8192 data: 6 6
25593 107/300 Flush
25594 108/300 Flush
25595 109/300 Read block 1428 len 36864
25596 110/300 Write block 3594 len 32768 data: 7 7 7 7 5 5 4 6
25597 111/300 Read block 4325 len 40960
25598 112/300 Read block 2835 len 12288
25599 113/300 Write block 1242 len 20480 data: 3 3 4 5 6
25600 114/300 Read block 301 len 12288
25601 115/300 Read block 4450 len 16384
25602 116/300 Write block 3541 len 24576 data: 4 4 3 5 4 3
25603 117/300 Write block 4591 len 24576 data: 8 8 6 6 6 6
25604 118/300 Read block 1566 len 8192
25605 119/300 Read block 1915 len 36864
25606 120/300 Read block 2918 len 20480
25607 121/300 Read block 3692 len 20480
25608 122/300 Read block 2957 len 28672
25609 123/300 Read block 1047 len 12288
25610 124/300 Write block 3436 len 32768 data: 7 8 7 8 7 5 5 6
25611 125/300 Write block 3790 len 40960 data: 6 6 6 5 6 5 5 5 5 7
25612 126/300 Flush
25613 127/300 Read block 3241 len 8192
25614 128/300 Read block 197 len 28672
25615 129/300 Write block 1750 len 32768 data: 5 5 4 4 5 4 5 6
25616 130/300 Read block 1405 len 12288
25617 131/300 Write block 1930 len 32768 data: 5 6 7 7 5 5 5 6
25618 132/300 Write block 4369 len 8192 data: 5 5
25619 133/300 Read block 2054 len 36864
25620 134/300 Read block 1744 len 32768
25621 135/300 Write block 1490 len 32768 data: 3 4 4 5 4 4 5 5
25622 136/300 Write block 2339 len 16384 data: 4 5 5 3
25623 137/300 Read block 51 len 28672
25624 138/300 Read block 4661 len 4096
25625 139/300 Write block 3285 len 8192 data: 7 6
25626 140/300 Read block 2259 len 20480
25627 141/300 Write block 1363 len 40960 data: 4 3 3 5 5 5 5 4 4 5
25628 142/300 Flush
25629 143/300 Read block 4131 len 40960
25630 144/300 Write block 195 len 40960 data: 4 2 3 4 4 4 5 5 5 5
25631 145/300 Read block 4198 len 12288
25632 146/300 Read block 1698 len 8192
25633 147/300 Read block 4146 len 40960
25634 148/300 Flush
25635 149/300 Write block 254 len 36864 data: 5 6 6 5 5 4 3 3 3
25636 150/300 Write block 1566 len 12288 data: 9 7 5
25637 151/300 Read block 1696 len 4096
25638 152/300 Read block 2590 len 32768
25639 153/300 Read block 3271 len 8192
25640 154/300 Read block 2336 len 40960
25641 155/300 Read block 2167 len 20480
25642 156/300 Read block 2943 len 28672
25643 157/300 Write block 32 len 16384 data: 6 6 7 7
25644 158/300 Write block 4392 len 32768 data: 5 5 5 4 4 3 3 3
25645 159/300 Write block 4238 len 36864 data: 3 3 3 4 5 4 4 5 4
25646 160/300 Read block 3327 len 32768
25647 161/300 Read block 3401 len 12288
25648 162/300 Read block 2216 len 8192
25649 163/300 Read block 2627 len 12288
25650 164/300 Write block 3437 len 40960 data: 9 8 9 8 6 6 7 6 5 4
25651 165/300 Read block 3249 len 32768
25652 166/300 Read block 2266 len 40960
25653 167/300 Read block 1746 len 40960
25654 168/300 Write block 3724 len 16384 data: 6 5 6 6
25655 169/300 Read block 681 len 28672
25656 170/300 Read block 1696 len 36864
25657 171/300 Read block 1260 len 24576
25658 172/300 Read block 4469 len 36864
25659 173/300 Read block 1349 len 32768
25660 174/300 Read block 2133 len 12288
25661 175/300 Write block 4062 len 40960 data: 4 4 3 3 3 4 4 5 3 3
25662 176/300 Read block 3963 len 8192
25663 177/300 Write block 2827 len 20480 data: 3 3 2 3 5
25664 178/300 Write block 2283 len 12288 data: 4 5 6
25665 179/300 Flush
25666 180/300 Read block 1370 len 16384
25667 181/300 Read block 3650 len 4096
25668 182/300 Write block 389 len 16384 data: 7 6 6 5
25669 183/300 Read block 2809 len 16384
25670 184/300 Read block 4588 len 16384
25671 185/300 Write block 3528 len 40960 data: 4 5 6 7 5 4 4 5 5 5
25672 186/300 Read block 2475 len 28672
25673 187/300 Write block 3369 len 12288 data: 4 5 5
25674 188/300 Read block 4959 len 28672
25675 189/300 Read block 2801 len 8192
25676 190/300 Flush
25677 191/300 Write block 627 len 24576 data: 5 5 5 4 2 2
25678 192/300 Read block 4030 len 24576
25679 193/300 Read block 3696 len 16384
25680 194/300 Write block 943 len 32768 data: 7 5 6 7 5 5 4 4
25681 195/300 Read block 3273 len 12288
25682 196/300 Flush
25683 197/300 Read block 4422 len 32768
25684 198/300 Read block 1126 len 4096
25685 199/300 Write block 4306 len 28672 data: 3 3 3 3 5 4 5
25686 200/300 Write block 3702 len 28672 data: 5 3 4 4 5 5 7
25687 201/300 Write block 549 len 16384 data: 3 3 3 4
25688 202/300 Read block 4892 len 32768
25689 203/300 Read block 389 len 12288
25690 204/300 Write block 1071 len 20480 data: 5 5 5 5 5
25691 205/300 Read block 45 len 36864
25692 206/300 Read block 12 len 12288
25693 207/300 Read block 3001 len 20480
25694 208/300 Read block 4445 len 20480
25695 209/300 Read block 2006 len 24576
25696 210/300 Write block 1131 len 28672 data: 3 2 2 2 3 4 5
25697 211/300 Read block 895 len 12288
25698 212/300 Flush
25699 213/300 Read block 1999 len 12288
25700 214/300 Read block 1850 len 24576
25701 215/300 Read block 1894 len 28672
25702 216/300 Read block 1326 len 40960
25703 217/300 Write block 1479 len 20480 data: 6 6 5 5 2
25704 218/300 Read block 4363 len 4096
25705 219/300 Write block 2073 len 28672 data: 5 5 5 5 8 8 8
25706 220/300 Read block 4921 len 32768
25707 221/300 Write block 2706 len 24576 data: 10 10 7 9 7 7
25708 222/300 Read block 1882 len 20480
25709 223/300 Read block 3353 len 36864
25710 224/300 Read block 3188 len 20480
25711 225/300 Read block 167 len 4096
25712 226/300 Read block 1804 len 4096
25713 227/300 Write block 3632 len 8192 data: 3 3
25714 228/300 Flush
25715 229/300 Read block 3156 len 40960
25716 230/300 Read block 4509 len 16384
25717 231/300 Read block 2252 len 32768
25718 232/300 Read block 1351 len 28672
25719 233/300 Read block 4381 len 12288
25720 234/300 Read block 1925 len 40960
25721 235/300 Read block 3134 len 20480
25722 236/300 Read block 3696 len 40960
25723 237/300 Read block 2865 len 8192
25724 238/300 Read block 1229 len 40960
25725 239/300 Write block 4416 len 16384 data: 2 2 2 4
25726 240/300 Write block 4709 len 20480 data: 4 4 4 4 3
25727 241/300 Read block 217 len 32768
25728 242/300 Write block 2606 len 12288 data: 4 4 5
25729 243/300 Read block 1566 len 28672
25730 244/300 Read block 2879 len 36864
25731 245/300 Read block 3233 len 16384
25732 246/300 Read block 4243 len 8192
25733 247/300 Read block 1802 len 20480
25734 248/300 Read block 2610 len 20480
25735 249/300 Read block 3564 len 20480
25736 250/300 Read block 2708 len 12288
25737 251/300 Write block 3398 len 8192 data: 5 5
25738 252/300 Read block 3585 len 12288
25739 253/300 Read block 4821 len 36864
25740 254/300 Read block 4720 len 40960
25741 255/300 Read block 379 len 32768
25742 256/300 Read block 2915 len 40960
25743 257/300 Flush
25744 258/300 Read block 427 len 32768
25745 259/300 Read block 25 len 40960
25746 260/300 Read block 453 len 28672
25747 261/300 Flush
25748 262/300 Write block 1520 len 8192 data: 3 3
25749 263/300 Read block 4694 len 36864
25750 264/300 Read block 109 len 24576
25751 265/300 Read block 1721 len 28672
25752 266/300 Write block 2764 len 4096 data: 4
25753 267/300 Write block 1114 len 12288 data: 5 5 5
25754 268/300 Read block 4300 len 36864
25755 269/300 Read block 3322 len 32768
25756 270/300 Read block 1399 len 28672
25757 271/300 Write block 1066 len 8192 data: 7 5
25758 272/300 Write block 2225 len 12288 data: 4 5 4
25759 273/300 Read block 76 len 8192
25760 274/300 Read block 2937 len 20480
25761 275/300 Read block 1926 len 20480
25762 276/300 Write block 800 len 36864 data: 3 4 4 4 3 2 2 2 2
25763 277/300 Read block 437 len 36864
25764 278/300 Write block 880 len 4096 data: 5
25765 279/300 Read block 4305 len 40960
25766 280/300 Read block 3489 len 8192
25767 281/300 Write block 10 len 32768 data: 7 5 5 3 3 3 3 3
25768 282/300 Read block 2575 len 16384
25769 283/300 Write block 347 len 4096 data: 5
25770 284/300 Read block 1403 len 20480
25771 285/300 Write block 3615 len 28672 data: 3 5 7 8 8 6 7
25772 286/300 Read block 2147 len 28672
25773 287/300 Flush
25774 288/300 Read block 2029 len 8192
25775 289/300 Write block 580 len 32768 data: 4 3 3 3 3 4 5 5
25776 290/300 Write block 4993 len 28672 data: 5 4 3 3 2 3 3
25777 291/300 Write block 1539 len 20480 data: 3 3 3 4 4
25778 292/300 Write block 3927 len 36864 data: 5 5 6 5 5 5 4 2 3
25779 293/300 Write block 4899 len 24576 data: 6 5 4 4 4 3
25780 294/300 Read block 1204 len 24576
25781 295/300 Write block 209 len 8192 data: 3 3
25782 296/300 Read block 4345 len 36864
25783 297/300 Read block 1590 len 40960
25784 298/300 Read block 1994 len 32768
25785 299/300 Write block 483 len 16384 data: 3 3 5 5
25786 300/300 Write block 137 len 16384 data: 11 11 12 10
25787 301/300 Write block 3209 len 24576 data: 5 5 4 5 3 5
25788 Main task has work to do, go find it
25789 got dsc Start(1)
25790 start 1
25791 Nov 19 17:13:25.412 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/1, method: POST, req_id: f30ea069-78b7-44d4-89be-ae8f3379cdd5, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
25792 Replay: started 1, returned:Ok(())
25793 ----------------------------------------------------------------
25794 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25795 States: Active Offline Active
25796 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25797 7642 Acked Read 9 Done Sent Done false
25798 7643 Acked Read 1 Done Sent Done false
25799 7644 Acked Read 3 Done Sent Done false
25800 7645 Acked Read 2 Done Sent Done false
25801 7646 Acked Write 4 Done Sent Done false
25802 7647 Acked Write 6 Done Sent Done false
25803 7648 Acked Read 7 Done Sent Done false
25804 7649 Acked Write 5 Done Sent Done false
25805 7650 Acked Read 2 Done Sent Done false
25806 7651 Acked Read 2 Done Sent Done false
25807 7652 Acked Read 9 Done Sent Done false
25808 7653 Acked Read 7 Done Sent Done false
25809 7654 Acked Read 10 Done Sent Done false
25810 7655 Acked Read 4 Done Sent Done false
25811 7656 Acked Write 4 Done Sent Done false
25812 7657 Acked Read 9 Done Sent Done false
25813 7658 Acked Read 2 Done Sent Done false
25814 7659 Acked Read 9 Done Sent Done false
25815 7660 Acked Read 9 Done Sent Done false
25816 7661 Acked Read 8 Done Sent Done false
25817 7662 Acked Write 8 Done Sent Done false
25818 7663 Acked Read 7 Done Sent Done false
25819 7664 Acked Read 3 Done Sent Done false
25820 7665 Acked Write 2 Done Sent Done false
25821 7666 Acked Write 9 Done Sent Done false
25822 7667 Acked Write 8 Done Sent Done false
25823 7668 Acked Write 10 Done Sent Done false
25824 7669 Acked Read 9 Done Sent Done false
25825 7670 Acked Write 9 Done Sent Done false
25826 7671 Acked Read 3 Done Sent Done false
25827 7672 Acked Read 10 Done Sent Done false
25828 7673 Acked Write 9 Done Sent Done false
25829 7674 Acked Read 10 Done Sent Done false
25830 7675 Acked Read 10 Done Sent Done false
25831 7676 Acked Write 8 Done Sent Done false
25832 7677 Acked Read 6 Done Sent Done false
25833 7678 Acked Read 5 Done Sent Done false
25834 7679 Acked Write 10 Done Sent Done false
25835 7680 Acked Read 10 Done Sent Done false
25836 7681 Acked Write 3 Done Sent Done false
25837 7682 Acked Write 7 Done Sent Done false
25838 7683 Acked Write 10 Done Sent Done false
25839 7684 Acked Flush 0 Done Sent Done false
25840 7685 Acked Write 3 Done Sent Done false
25841 7686 Acked Read 6 Done Sent Done false
25842 7687 Acked Read 10 Done Sent Done false
25843 7688 Acked Write 3 Done Sent Done false
25844 7689 Acked Read 5 Done Sent Done false
25845 7690 Acked Read 10 Done Sent Done false
25846 7691 Acked Flush 0 Done Sent Done false
25847 7692 Acked Write 1 Done Sent Done false
25848 7693 Acked Write 6 Done Sent Done false
25849 7694 Acked Read 8 Done Sent Done false
25850 7695 Acked Flush 0 Done Sent Done false
25851 7696 Acked Read 4 Done Sent Done false
25852 7697 Acked Read 1 Done Sent Done false
25853 7698 Acked Read 2 Done Sent Done false
25854 7699 Acked Write 2 Done Sent Done false
25855 7700 Acked Flush 0 Done Sent Done false
25856 7701 Acked Read 10 Done Sent Done false
25857 7702 Acked Read 5 Done Sent Done false
25858 7703 Acked Read 6 Done Sent Done false
25859 7704 Acked Read 7 Done Sent Done false
25860 7705 Acked Read 7 Done Sent Done false
25861 7706 Acked Write 6 Done Sent Done false
25862 7707 Acked Read 5 Done Sent Done false
25863 7708 Acked Write 10 Done Sent Done false
25864 7709 Acked Read 4 Done Sent Done false
25865 7710 Acked Flush 0 Done Sent Done false
25866 7711 Acked Write 9 Done Sent Done false
25867 7712 Acked Write 6 Done Sent Done false
25868 7713 Acked Write 4 Done Sent Done false
25869 7714 Acked Write 8 Done Sent Done false
25870 7715 Acked Write 9 Done Sent Done false
25871 7716 Acked Read 5 Done Sent Done false
25872 7717 Acked Flush 0 Done Sent Done false
25873 7718 Acked Write 8 Done Sent Done false
25874 7719 Acked Read 8 Done Sent Done false
25875 7720 Acked Flush 0 Done Sent Done false
25876 7721 Acked Read 4 Done Sent Done false
25877 7722 Acked Read 2 Done Sent Done false
25878 7723 Acked Write 1 Done Sent Done false
25879 7724 Acked Read 3 Done Sent Done false
25880 7725 Acked Read 1 Done Sent Done false
25881 7726 Acked Write 1 Done Sent Done false
25882 7727 Acked Write 6 Done Sent Done false
25883 7728 Acked Read 10 Done Sent Done false
25884 7729 Acked Write 8 Done Sent Done false
25885 7730 Acked Read 2 Done Sent Done false
25886 7731 Acked Write 3 Done Sent Done false
25887 7732 Acked Write 5 Done Sent Done false
25888 7733 Acked Write 10 Done Sent Done false
25889 7734 Acked Flush 0 Done Sent Done false
25890 7735 Acked Read 5 Done Sent Done false
25891 7736 Acked Read 6 Done Sent Done false
25892 7737 Acked Write 8 Done Sent Done false
25893 7738 Acked Write 2 Done Sent Done false
25894 7739 Acked Write 2 Done Sent Done false
25895 7740 Acked Read 3 Done Sent Done false
25896 7741 Acked Write 10 Done Sent Done false
25897 7742 Acked Read 10 Done Sent Done false
25898 7743 Acked Read 4 Done Sent Done false
25899 7744 Acked Read 4 Done Sent Done false
25900 7745 Acked Read 7 Done Sent Done false
25901 7746 Acked Write 3 Done Sent Done false
25902 7747 Acked Write 2 Done Sent Done false
25903 7748 Acked Flush 0 Done Sent Done false
25904 7749 Acked Flush 0 Done Sent Done false
25905 7750 Acked Read 9 Done Sent Done false
25906 7751 Acked Write 8 Done Sent Done false
25907 7752 Acked Read 10 Done Sent Done false
25908 7753 Acked Read 3 Done Sent Done false
25909 7754 Acked Write 5 Done Sent Done false
25910 7755 Acked Read 3 Done Sent Done false
25911 7756 Acked Read 4 Done Sent Done false
25912 7757 Acked Write 6 Done Sent Done false
25913 7758 Acked Write 6 Done Sent Done false
25914 7759 Acked Read 2 Done Sent Done false
25915 7760 Acked Read 9 Done Sent Done false
25916 7761 Acked Read 5 Done Sent Done false
25917 7762 Acked Read 5 Done Sent Done false
25918 7763 Acked Read 7 Done Sent Done false
25919 7764 Acked Read 3 Done Sent Done false
25920 7765 Acked Write 8 Done Sent Done false
25921 7766 Acked Write 10 Done Sent Done false
25922 7767 Acked Flush 0 Done Sent Done false
25923 7768 Acked Read 2 Done Sent Done false
25924 7769 Acked Read 7 Done Sent Done false
25925 7770 Acked Write 8 Done Sent Done false
25926 7771 Acked Read 3 Done Sent Done false
25927 7772 Acked Write 8 Done Sent Done false
25928 7773 Acked Write 2 Done Sent Done false
25929 7774 Acked Read 9 Done Sent Done false
25930 7775 Acked Read 8 Done Sent Done false
25931 7776 Acked Write 8 Done Sent Done false
25932 7777 Acked Write 4 Done Sent Done false
25933 7778 Acked Read 7 Done Sent Done false
25934 7779 Acked Read 1 Done Sent Done false
25935 7780 Acked Write 2 Done Sent Done false
25936 7781 Acked Read 5 Done Sent Done false
25937 7782 Acked Write 10 Done Sent Done false
25938 7783 Acked Flush 0 Done Sent Done false
25939 7784 Acked Read 10 Done Sent Done false
25940 7785 Acked Write 10 Done Sent Done false
25941 7786 Acked Read 3 Done Sent Done false
25942 7787 Acked Read 2 Done Sent Done false
25943 7788 Acked Read 10 Done Sent Done false
25944 7789 Acked Flush 0 Done Sent Done false
25945 7790 Acked Write 9 Done Sent Done false
25946 7791 Acked Write 3 Done Sent Done false
25947 7792 Acked Read 1 Done Sent Done false
25948 7793 Acked Read 8 Done Sent Done false
25949 7794 Acked Read 2 Done Sent Done false
25950 7795 Acked Read 10 Done Sent Done false
25951 7796 Acked Read 5 Done Sent Done false
25952 7797 Acked Read 7 Done Sent Done false
25953 7798 Acked Write 4 Done Sent Done false
25954 7799 Acked Write 8 Done Sent Done false
25955 7800 Acked Write 9 Done Sent Done false
25956 7801 Acked Read 8 Done Sent Done false
25957 7802 Acked Read 3 Done Sent Done false
25958 7803 Acked Read 2 Done Sent Done false
25959 7804 Acked Read 3 Done Sent Done false
25960 7805 Acked Write 10 Done Sent Done false
25961 7806 Acked Read 8 Done Sent Done false
25962 7807 Acked Read 10 Done Sent Done false
25963 7808 Acked Read 10 Done Sent Done false
25964 7809 Acked Write 4 Done Sent Done false
25965 7810 Acked Read 7 Done Sent Done false
25966 7811 Acked Read 9 Done Sent Done false
25967 7812 Acked Read 6 Done Sent Done false
25968 7813 Acked Read 9 Done Sent Done false
25969 7814 Acked Read 8 Done Sent Done false
25970 7815 Acked Read 3 Done Sent Done false
25971 7816 Acked Write 10 Done Sent Done false
25972 7817 Acked Read 2 Done Sent Done false
25973 7818 Acked Write 5 Done Sent Done false
25974 7819 Acked Write 3 Done Sent Done false
25975 7820 Acked Flush 0 Done Sent Done false
25976 7821 Acked Read 4 Done Sent Done false
25977 7822 Acked Read 1 Done Sent Done false
25978 7823 Acked Write 4 Done Sent Done false
25979 7824 Acked Read 4 Done Sent Done false
25980 7825 Acked Read 4 Done Sent Done false
25981 7826 Acked Write 10 Done Sent Done false
25982 7827 Acked Read 7 Done Sent Done false
25983 7828 Acked Write 3 Done Sent Done false
25984 7829 Acked Read 7 Done Sent Done false
25985 7830 Acked Read 2 Done Sent Done false
25986 7831 Acked Flush 0 Done Sent Done false
25987 7832 Acked Write 6 Done Sent Done false
25988 7833 Acked Read 6 Done Sent Done false
25989 7834 Acked Read 4 Done Sent Done false
25990 7835 Acked Write 8 Done Sent Done false
25991 7836 Acked Read 3 Done Sent Done false
25992 7837 Acked Flush 0 Done Sent Done false
25993 7838 Acked Read 8 Done Sent Done false
25994 7839 Acked Read 1 Done Sent Done false
25995 7840 Acked Write 7 Done Sent Done false
25996 7841 Acked Write 7 Done Sent Done false
25997 7842 Acked Write 4 Done Sent Done false
25998 7843 Acked Read 8 Done Sent Done false
25999 7844 Acked Read 3 Done Sent Done false
26000 7845 Acked Write 5 Done Sent Done false
26001 7846 Acked Read 9 Done Sent Done false
26002 7847 Acked Read 3 Done Sent Done false
26003 7848 Acked Read 5 Done Sent Done false
26004 7849 Acked Read 5 Done Sent Done false
26005 7850 Acked Read 6 Done Sent Done false
26006 7851 Acked Write 7 Done Sent Done false
26007 7852 Acked Read 3 Done Sent Done false
26008 7853 Acked Flush 0 Done Sent Done false
26009 7854 Acked Read 3 Done Sent Done false
26010 7855 Acked Read 6 Done Sent Done false
26011 7856 Acked Read 7 Done Sent Done false
26012 7857 Acked Read 10 Done Sent Done false
26013 7858 Acked Write 5 Done Sent Done false
26014 7859 Acked Read 1 Done Sent Done false
26015 7860 Acked Write 7 Done Sent Done false
26016 7861 Acked Read 8 Done Sent Done false
26017 7862 Acked Write 6 Done Sent Done false
26018 7863 Acked Read 5 Done Sent Done false
26019 7864 Acked Read 9 Done Sent Done false
26020 7865 Acked Read 5 Done Sent Done false
26021 7866 Acked Read 1 Done Sent Done false
26022 7867 Acked Read 1 Done Sent Done false
26023 7868 Acked Write 2 Done Sent Done false
26024 7869 Acked Flush 0 Done Sent Done false
26025 7870 Acked Read 10 Done Sent Done false
26026 7871 Acked Read 4 Done Sent Done false
26027 7872 Acked Read 8 Done Sent Done false
26028 7873 Acked Read 7 Done Sent Done false
26029 7874 Acked Read 3 Done Sent Done false
26030 7875 Acked Read 10 Done Sent Done false
26031 7876 Acked Read 5 Done Sent Done false
26032 7877 Acked Read 10 Done Sent Done false
26033 7878 Acked Read 2 Done Sent Done false
26034 7879 Acked Read 10 Done Sent Done false
26035 7880 Acked Write 4 Done Sent Done false
26036 7881 Acked Write 5 Done Sent Done false
26037 7882 Acked Read 8 Done Sent Done false
26038 7883 Acked Write 3 Done Sent Done false
26039 7884 Acked Read 7 Done Sent Done false
26040 7885 Acked Read 9 Done Sent Done false
26041 7886 Acked Read 4 Done Sent Done false
26042 7887 Acked Read 2 Done Sent Done false
26043 7888 Acked Read 5 Done Sent Done false
26044 7889 Acked Read 5 Done Sent Done false
26045 7890 Acked Read 5 Done Sent Done false
26046 7891 Acked Read 3 Done Sent Done false
26047 7892 Acked Write 2 Done Sent Done false
26048 7893 Acked Read 3 Done Sent Done false
26049 7894 Acked Read 9 Done Sent Done false
26050 7895 Acked Read 10 Done Sent Done false
26051 7896 Acked Read 8 Done Sent Done false
26052 7897 Acked Read 10 Done Sent Done false
26053 7898 Acked Flush 0 Done Sent Done false
26054 7899 Acked Read 8 Done Sent Done false
26055 7900 Acked Read 10 Done Sent Done false
26056 7901 Acked Read 7 Done Sent Done false
26057 7902 Acked Flush 0 Done Sent Done false
26058 7903 Acked Write 2 Done Sent Done false
26059 7904 Acked Read 9 Done Sent Done false
26060 7905 Acked Read 6 Done Sent Done false
26061 7906 Acked Read 7 Done Sent Done false
26062 7907 Acked Write 1 Done Sent Done false
26063 7908 Acked Write 3 Done Sent Done false
26064 7909 Acked Read 9 Done Sent Done false
26065 7910 Acked Read 8 Done Sent Done false
26066 7911 Acked Read 7 Done Sent Done false
26067 7912 Acked Write 2 Done Sent Done false
26068 7913 Acked Write 3 Done Sent Done false
26069 7914 Acked Read 2 Done Sent Done false
26070 7915 Acked Read 5 Done Sent Done false
26071 7916 Acked Read 5 Done Sent Done false
26072 7917 Acked Write 9 Done Sent Done false
26073 7918 Acked Read 9 Done Sent Done false
26074 7919 Acked Write 1 Done Sent Done false
26075 7920 Acked Read 10 Done Sent Done false
26076 7921 Acked Read 2 Done Sent Done false
26077 7922 Acked Write 8 Done Sent Done false
26078 7923 Acked Read 4 Done Sent Done false
26079 7924 Acked Write 1 Done Sent Done false
26080 7925 Acked Read 5 Done Sent Done false
26081 7926 Acked Write 7 Done Sent Done false
26082 7927 Acked Read 7 Done Sent Done false
26083 7928 Acked Flush 0 Done Sent Done false
26084 7929 Acked Read 2 Done Sent Done false
26085 7930 Acked Write 8 Done Sent Done false
26086 7931 Acked Write 7 Done Sent Done false
26087 7932 Acked Write 5 Done Sent Done false
26088 7933 Acked Write 9 Done Sent Done false
26089 7934 Acked Write 6 Done Sent Done false
26090 7935 Acked Read 6 Done Sent Done false
26091 7936 Acked Write 2 Done Sent Done false
26092 7937 Acked Read 9 Done Sent Done false
26093 7938 Acked Read 10 Done Sent Done false
26094 7939 Acked Read 8 Done Sent Done false
26095 7940 Acked Write 4 Done Sent Done false
26096 7941 Acked Write 4 Done Sent Done false
26097 7942 Acked Write 6 Done Sent Done false
26098 STATES DS:0 DS:1 DS:2 TOTAL
26099 Sent 0 301 0 301
26100 Done 301 0 301 602
26101 Skipped 0 0 0 0
26102 Error 0 0 0 0
26103 Last Flush: 7928 7641 7928
26104 Downstairs last five completed: 7641 7640 7639 7638 7637
26105 Upstairs last five completed: 7942 7941 7940 7939 7938
26106 CLIENT: Up:0 ds:301 act:2
26107 [8820] got start action
26108 [8820] I am going to restart
26109 Starting downstairs at port 8820
26110 Make output file at "/tmp/dsc/downstairs-8820.txt"
26111 [8820][1] reports Starting
26112 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
26113 [8820][1] reports Running
26114 ----------------------------------------------------------------
26115 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26116 States: Active Offline Active
26117 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26118 7642 Acked Read 9 Done Sent Done false
26119 7643 Acked Read 1 Done Sent Done false
26120 7644 Acked Read 3 Done Sent Done false
26121 7645 Acked Read 2 Done Sent Done false
26122 7646 Acked Write 4 Done Sent Done false
26123 7647 Acked Write 6 Done Sent Done false
26124 7648 Acked Read 7 Done Sent Done false
26125 7649 Acked Write 5 Done Sent Done false
26126 7650 Acked Read 2 Done Sent Done false
26127 7651 Acked Read 2 Done Sent Done false
26128 7652 Acked Read 9 Done Sent Done false
26129 7653 Acked Read 7 Done Sent Done false
26130 7654 Acked Read 10 Done Sent Done false
26131 7655 Acked Read 4 Done Sent Done false
26132 7656 Acked Write 4 Done Sent Done false
26133 7657 Acked Read 9 Done Sent Done false
26134 7658 Acked Read 2 Done Sent Done false
26135 7659 Acked Read 9 Done Sent Done false
26136 7660 Acked Read 9 Done Sent Done false
26137 7661 Acked Read 8 Done Sent Done false
26138 7662 Acked Write 8 Done Sent Done false
26139 7663 Acked Read 7 Done Sent Done false
26140 7664 Acked Read 3 Done Sent Done false
26141 7665 Acked Write 2 Done Sent Done false
26142 7666 Acked Write 9 Done Sent Done false
26143 7667 Acked Write 8 Done Sent Done false
26144 7668 Acked Write 10 Done Sent Done false
26145 7669 Acked Read 9 Done Sent Done false
26146 7670 Acked Write 9 Done Sent Done false
26147 7671 Acked Read 3 Done Sent Done false
26148 7672 Acked Read 10 Done Sent Done false
26149 7673 Acked Write 9 Done Sent Done false
26150 7674 Acked Read 10 Done Sent Done false
26151 7675 Acked Read 10 Done Sent Done false
26152 7676 Acked Write 8 Done Sent Done false
26153 7677 Acked Read 6 Done Sent Done false
26154 7678 Acked Read 5 Done Sent Done false
26155 7679 Acked Write 10 Done Sent Done false
26156 7680 Acked Read 10 Done Sent Done false
26157 7681 Acked Write 3 Done Sent Done false
26158 7682 Acked Write 7 Done Sent Done false
26159 7683 Acked Write 10 Done Sent Done false
26160 7684 Acked Flush 0 Done Sent Done false
26161 7685 Acked Write 3 Done Sent Done false
26162 7686 Acked Read 6 Done Sent Done false
26163 7687 Acked Read 10 Done Sent Done false
26164 7688 Acked Write 3 Done Sent Done false
26165 7689 Acked Read 5 Done Sent Done false
26166 7690 Acked Read 10 Done Sent Done false
26167 7691 Acked Flush 0 Done Sent Done false
26168 7692 Acked Write 1 Done Sent Done false
26169 7693 Acked Write 6 Done Sent Done false
26170 7694 Acked Read 8 Done Sent Done false
26171 7695 Acked Flush 0 Done Sent Done false
26172 7696 Acked Read 4 Done Sent Done false
26173 7697 Acked Read 1 Done Sent Done false
26174 7698 Acked Read 2 Done Sent Done false
26175 7699 Acked Write 2 Done Sent Done false
26176 7700 Acked Flush 0 Done Sent Done false
26177 7701 Acked Read 10 Done Sent Done false
26178 7702 Acked Read 5 Done Sent Done false
26179 7703 Acked Read 6 Done Sent Done false
26180 7704 Acked Read 7 Done Sent Done false
26181 7705 Acked Read 7 Done Sent Done false
26182 7706 Acked Write 6 Done Sent Done false
26183 7707 Acked Read 5 Done Sent Done false
26184 7708 Acked Write 10 Done Sent Done false
26185 7709 Acked Read 4 Done Sent Done false
26186 7710 Acked Flush 0 Done Sent Done false
26187 7711 Acked Write 9 Done Sent Done false
26188 7712 Acked Write 6 Done Sent Done false
26189 7713 Acked Write 4 Done Sent Done false
26190 7714 Acked Write 8 Done Sent Done false
26191 7715 Acked Write 9 Done Sent Done false
26192 7716 Acked Read 5 Done Sent Done false
26193 7717 Acked Flush 0 Done Sent Done false
26194 7718 Acked Write 8 Done Sent Done false
26195 7719 Acked Read 8 Done Sent Done false
26196 7720 Acked Flush 0 Done Sent Done false
26197 7721 Acked Read 4 Done Sent Done false
26198 7722 Acked Read 2 Done Sent Done false
26199 7723 Acked Write 1 Done Sent Done false
26200 7724 Acked Read 3 Done Sent Done false
26201 7725 Acked Read 1 Done Sent Done false
26202 7726 Acked Write 1 Done Sent Done false
26203 7727 Acked Write 6 Done Sent Done false
26204 7728 Acked Read 10 Done Sent Done false
26205 7729 Acked Write 8 Done Sent Done false
26206 7730 Acked Read 2 Done Sent Done false
26207 7731 Acked Write 3 Done Sent Done false
26208 7732 Acked Write 5 Done Sent Done false
26209 7733 Acked Write 10 Done Sent Done false
26210 7734 Acked Flush 0 Done Sent Done false
26211 7735 Acked Read 5 Done Sent Done false
26212 7736 Acked Read 6 Done Sent Done false
26213 7737 Acked Write 8 Done Sent Done false
26214 7738 Acked Write 2 Done Sent Done false
26215 7739 Acked Write 2 Done Sent Done false
26216 7740 Acked Read 3 Done Sent Done false
26217 7741 Acked Write 10 Done Sent Done false
26218 7742 Acked Read 10 Done Sent Done false
26219 7743 Acked Read 4 Done Sent Done false
26220 7744 Acked Read 4 Done Sent Done false
26221 7745 Acked Read 7 Done Sent Done false
26222 7746 Acked Write 3 Done Sent Done false
26223 7747 Acked Write 2 Done Sent Done false
26224 7748 Acked Flush 0 Done Sent Done false
26225 7749 Acked Flush 0 Done Sent Done false
26226 7750 Acked Read 9 Done Sent Done false
26227 7751 Acked Write 8 Done Sent Done false
26228 7752 Acked Read 10 Done Sent Done false
26229 7753 Acked Read 3 Done Sent Done false
26230 7754 Acked Write 5 Done Sent Done false
26231 7755 Acked Read 3 Done Sent Done false
26232 7756 Acked Read 4 Done Sent Done false
26233 7757 Acked Write 6 Done Sent Done false
26234 7758 Acked Write 6 Done Sent Done false
26235 7759 Acked Read 2 Done Sent Done false
26236 7760 Acked Read 9 Done Sent Done false
26237 7761 Acked Read 5 Done Sent Done false
26238 7762 Acked Read 5 Done Sent Done false
26239 7763 Acked Read 7 Done Sent Done false
26240 7764 Acked Read 3 Done Sent Done false
26241 7765 Acked Write 8 Done Sent Done false
26242 7766 Acked Write 10 Done Sent Done false
26243 7767 Acked Flush 0 Done Sent Done false
26244 7768 Acked Read 2 Done Sent Done false
26245 7769 Acked Read 7 Done Sent Done false
26246 7770 Acked Write 8 Done Sent Done false
26247 7771 Acked Read 3 Done Sent Done false
26248 7772 Acked Write 8 Done Sent Done false
26249 7773 Acked Write 2 Done Sent Done false
26250 7774 Acked Read 9 Done Sent Done false
26251 7775 Acked Read 8 Done Sent Done false
26252 7776 Acked Write 8 Done Sent Done false
26253 7777 Acked Write 4 Done Sent Done false
26254 7778 Acked Read 7 Done Sent Done false
26255 7779 Acked Read 1 Done Sent Done false
26256 7780 Acked Write 2 Done Sent Done false
26257 7781 Acked Read 5 Done Sent Done false
26258 7782 Acked Write 10 Done Sent Done false
26259 7783 Acked Flush 0 Done Sent Done false
26260 7784 Acked Read 10 Done Sent Done false
26261 7785 Acked Write 10 Done Sent Done false
26262 7786 Acked Read 3 Done Sent Done false
26263 7787 Acked Read 2 Done Sent Done false
26264 7788 Acked Read 10 Done Sent Done false
26265 7789 Acked Flush 0 Done Sent Done false
26266 7790 Acked Write 9 Done Sent Done false
26267 7791 Acked Write 3 Done Sent Done false
26268 7792 Acked Read 1 Done Sent Done false
26269 7793 Acked Read 8 Done Sent Done false
26270 7794 Acked Read 2 Done Sent Done false
26271 7795 Acked Read 10 Done Sent Done false
26272 7796 Acked Read 5 Done Sent Done false
26273 7797 Acked Read 7 Done Sent Done false
26274 7798 Acked Write 4 Done Sent Done false
26275 7799 Acked Write 8 Done Sent Done false
26276 7800 Acked Write 9 Done Sent Done false
26277 7801 Acked Read 8 Done Sent Done false
26278 7802 Acked Read 3 Done Sent Done false
26279 7803 Acked Read 2 Done Sent Done false
26280 7804 Acked Read 3 Done Sent Done false
26281 7805 Acked Write 10 Done Sent Done false
26282 7806 Acked Read 8 Done Sent Done false
26283 7807 Acked Read 10 Done Sent Done false
26284 7808 Acked Read 10 Done Sent Done false
26285 7809 Acked Write 4 Done Sent Done false
26286 7810 Acked Read 7 Done Sent Done false
26287 7811 Acked Read 9 Done Sent Done false
26288 7812 Acked Read 6 Done Sent Done false
26289 7813 Acked Read 9 Done Sent Done false
26290 7814 Acked Read 8 Done Sent Done false
26291 7815 Acked Read 3 Done Sent Done false
26292 7816 Acked Write 10 Done Sent Done false
26293 7817 Acked Read 2 Done Sent Done false
26294 7818 Acked Write 5 Done Sent Done false
26295 7819 Acked Write 3 Done Sent Done false
26296 7820 Acked Flush 0 Done Sent Done false
26297 7821 Acked Read 4 Done Sent Done false
26298 7822 Acked Read 1 Done Sent Done false
26299 7823 Acked Write 4 Done Sent Done false
26300 7824 Acked Read 4 Done Sent Done false
26301 7825 Acked Read 4 Done Sent Done false
26302 7826 Acked Write 10 Done Sent Done false
26303 7827 Acked Read 7 Done Sent Done false
26304 7828 Acked Write 3 Done Sent Done false
26305 7829 Acked Read 7 Done Sent Done false
26306 7830 Acked Read 2 Done Sent Done false
26307 7831 Acked Flush 0 Done Sent Done false
26308 7832 Acked Write 6 Done Sent Done false
26309 7833 Acked Read 6 Done Sent Done false
26310 7834 Acked Read 4 Done Sent Done false
26311 7835 Acked Write 8 Done Sent Done false
26312 7836 Acked Read 3 Done Sent Done false
26313 7837 Acked Flush 0 Done Sent Done false
26314 7838 Acked Read 8 Done Sent Done false
26315 7839 Acked Read 1 Done Sent Done false
26316 7840 Acked Write 7 Done Sent Done false
26317 7841 Acked Write 7 Done Sent Done false
26318 7842 Acked Write 4 Done Sent Done false
26319 7843 Acked Read 8 Done Sent Done false
26320 7844 Acked Read 3 Done Sent Done false
26321 7845 Acked Write 5 Done Sent Done false
26322 7846 Acked Read 9 Done Sent Done false
26323 7847 Acked Read 3 Done Sent Done false
26324 7848 Acked Read 5 Done Sent Done false
26325 7849 Acked Read 5 Done Sent Done false
26326 7850 Acked Read 6 Done Sent Done false
26327 7851 Acked Write 7 Done Sent Done false
26328 7852 Acked Read 3 Done Sent Done false
26329 7853 Acked Flush 0 Done Sent Done false
26330 7854 Acked Read 3 Done Sent Done false
26331 7855 Acked Read 6 Done Sent Done false
26332 7856 Acked Read 7 Done Sent Done false
26333 7857 Acked Read 10 Done Sent Done false
26334 7858 Acked Write 5 Done Sent Done false
26335 7859 Acked Read 1 Done Sent Done false
26336 7860 Acked Write 7 Done Sent Done false
26337 7861 Acked Read 8 Done Sent Done false
26338 7862 Acked Write 6 Done Sent Done false
26339 7863 Acked Read 5 Done Sent Done false
26340 7864 Acked Read 9 Done Sent Done false
26341 7865 Acked Read 5 Done Sent Done false
26342 7866 Acked Read 1 Done Sent Done false
26343 7867 Acked Read 1 Done Sent Done false
26344 7868 Acked Write 2 Done Sent Done false
26345 7869 Acked Flush 0 Done Sent Done false
26346 7870 Acked Read 10 Done Sent Done false
26347 7871 Acked Read 4 Done Sent Done false
26348 7872 Acked Read 8 Done Sent Done false
26349 7873 Acked Read 7 Done Sent Done false
26350 7874 Acked Read 3 Done Sent Done false
26351 7875 Acked Read 10 Done Sent Done false
26352 7876 Acked Read 5 Done Sent Done false
26353 7877 Acked Read 10 Done Sent Done false
26354 7878 Acked Read 2 Done Sent Done false
26355 7879 Acked Read 10 Done Sent Done false
26356 7880 Acked Write 4 Done Sent Done false
26357 7881 Acked Write 5 Done Sent Done false
26358 7882 Acked Read 8 Done Sent Done false
26359 7883 Acked Write 3 Done Sent Done false
26360 7884 Acked Read 7 Done Sent Done false
26361 7885 Acked Read 9 Done Sent Done false
26362 7886 Acked Read 4 Done Sent Done false
26363 7887 Acked Read 2 Done Sent Done false
26364 7888 Acked Read 5 Done Sent Done false
26365 7889 Acked Read 5 Done Sent Done false
26366 7890 Acked Read 5 Done Sent Done false
26367 7891 Acked Read 3 Done Sent Done false
26368 7892 Acked Write 2 Done Sent Done false
26369 7893 Acked Read 3 Done Sent Done false
26370 7894 Acked Read 9 Done Sent Done false
26371 7895 Acked Read 10 Done Sent Done false
26372 7896 Acked Read 8 Done Sent Done false
26373 7897 Acked Read 10 Done Sent Done false
26374 7898 Acked Flush 0 Done Sent Done false
26375 7899 Acked Read 8 Done Sent Done false
26376 7900 Acked Read 10 Done Sent Done false
26377 7901 Acked Read 7 Done Sent Done false
26378 7902 Acked Flush 0 Done Sent Done false
26379 7903 Acked Write 2 Done Sent Done false
26380 7904 Acked Read 9 Done Sent Done false
26381 7905 Acked Read 6 Done Sent Done false
26382 7906 Acked Read 7 Done Sent Done false
26383 7907 Acked Write 1 Done Sent Done false
26384 7908 Acked Write 3 Done Sent Done false
26385 7909 Acked Read 9 Done Sent Done false
26386 7910 Acked Read 8 Done Sent Done false
26387 7911 Acked Read 7 Done Sent Done false
26388 7912 Acked Write 2 Done Sent Done false
26389 7913 Acked Write 3 Done Sent Done false
26390 7914 Acked Read 2 Done Sent Done false
26391 7915 Acked Read 5 Done Sent Done false
26392 7916 Acked Read 5 Done Sent Done false
26393 7917 Acked Write 9 Done Sent Done false
26394 7918 Acked Read 9 Done Sent Done false
26395 7919 Acked Write 1 Done Sent Done false
26396 7920 Acked Read 10 Done Sent Done false
26397 7921 Acked Read 2 Done Sent Done false
26398 7922 Acked Write 8 Done Sent Done false
26399 7923 Acked Read 4 Done Sent Done false
26400 7924 Acked Write 1 Done Sent Done false
26401 7925 Acked Read 5 Done Sent Done false
26402 7926 Acked Write 7 Done Sent Done false
26403 7927 Acked Read 7 Done Sent Done false
26404 7928 Acked Flush 0 Done Sent Done false
26405 7929 Acked Read 2 Done Sent Done false
26406 7930 Acked Write 8 Done Sent Done false
26407 7931 Acked Write 7 Done Sent Done false
26408 7932 Acked Write 5 Done Sent Done false
26409 7933 Acked Write 9 Done Sent Done false
26410 7934 Acked Write 6 Done Sent Done false
26411 7935 Acked Read 6 Done Sent Done false
26412 7936 Acked Write 2 Done Sent Done false
26413 7937 Acked Read 9 Done Sent Done false
26414 7938 Acked Read 10 Done Sent Done false
26415 7939 Acked Read 8 Done Sent Done false
26416 7940 Acked Write 4 Done Sent Done false
26417 7941 Acked Write 4 Done Sent Done false
26418 7942 Acked Write 6 Done Sent Done false
26419 7943 Acked Flush 0 Done Sent Done false
26420 STATES DS:0 DS:1 DS:2 TOTAL
26421 Sent 0 302 0 302
26422 Done 302 0 302 604
26423 Skipped 0 0 0 0
26424 Error 0 0 0 0
26425 Last Flush: 7943 7641 7943
26426 Downstairs last five completed: 7641 7640 7639 7638 7637
26427 Upstairs last five completed: 7943 7942 7941 7940 7939
26428 CLIENT: Up:0 ds:302 act:2
264292024-11-19T17:13:31.152ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264302024-11-19T17:13:31.152ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48162) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264312024-11-19T17:13:31.154ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264322024-11-19T17:13:31.154ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264332024-11-19T17:13:31.154ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264342024-11-19T17:13:31.154ZINFOcrucible: send last flush ID to this DS: 7641 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264352024-11-19T17:13:31.154ZINFOcrucible: Replied this last flush ID: 7641 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264362024-11-19T17:13:31.154ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264372024-11-19T17:13:31.155ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264382024-11-19T17:13:31.155ZINFOcrucible: [1] client re-new 302 jobs since flush 7641 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
264392024-11-19T17:13:31.155ZINFOcrucible: [1] Replayed 302 jobs since flush: 7641 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
26440 ----------------------------------------------------------------
26441 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26442 Downstairs last five completed: 7943 7942 7941 7940 7939
26443 Upstairs last five completed: 7943 7942 7941 7940 7939
26444 CLIENT: Up:0 ds:0 act:3
26445 Replay: All jobs finished
26446 Main task has work to do, go find it
26447 got dsc Stop(0)
26448 stop 0
26449 Nov 19 17:13:33.418 INFO request completed, latency_us[8810] Got stop action so:false kr:false
26450 : 318, response_code: 204, uri: /stop/cid/0, method: POST, req_id: e170535f-6224-4960-bb41-1f63a68af554, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
264512024-11-19T17:13:33.418ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264522024-11-19T17:13:33.418ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264532024-11-19T17:13:33.418ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
264542024-11-19T17:13:33.418ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
264552024-11-19T17:13:33.418ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
26456 Nov 19 17:13:33.419{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:13:33.419103491Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
26457 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/0, method: GET, req_id: da6f6cdf-9f0c-4feb-bf49-b3f815ed0d1e, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
26458 [8810] Exited with: ExitStatus(unix_wait_status(9))
26459 [8810][0] reports Exit
26460 Nov 19 17:13:37.421 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/0, method: GET, req_id: 634d2b56-667b-431c-b833-dd7a536870e9, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
26461 001/300 Write block 4907 len 36864 data: 6 6 8 8 7 7 7 6 4
26462 002/300 Write block 3384 len 24576 data: 3 4 6 6 6 6
26463 003/300 Read block 2673 len 40960
26464 004/300 Read block 2854 len 32768
26465 005/300 Read block 191 len 12288
26466 006/300 Write block 4185 len 12288 data: 6 6 7
26467 007/300 Write block 3731 len 28672 data: 10 10 7 7 5 7 7
26468 008/300 Read block 3722 len 8192
26469 009/300 Read block 1865 len 40960
26470 010/300 Write block 4612 len 12288 data: 3 3 2
26471 011/300 Read block 3768 len 40960
26472 012/300 Write block 4737 len 24576 data: 4 4 4 4 3 5
26473 013/300 Read block 2530 len 4096
26474 014/300 Read block 4061 len 16384
26475 015/300 Read block 2815 len 12288
26476 016/300 Read block 2439 len 4096
26477 017/300 Write block 2050 len 4096 data: 4
26478 018/300 Read block 4338 len 24576
26479 019/300 Read block 1761 len 8192
26480 020/300 Write block 2185 len 32768 data: 3 4 4 4 5 5 7 7
26481 021/300 Read block 4739 len 12288
26482 022/300 Flush
26483 023/300 Read block 3571 len 8192
26484 024/300 Flush
26485 025/300 Read block 3332 len 12288
26486 026/300 Flush
26487 027/300 Write block 3078 len 36864 data: 4 4 7 6 5 5 5 4 4
26488 028/300 Read block 1603 len 16384
26489 029/300 Read block 1071 len 24576
26490 030/300 Read block 4425 len 40960
26491 031/300 Read block 3616 len 20480
26492 032/300 Write block 327 len 24576 data: 5 5 4 4 4 3
26493 033/300 Write block 3475 len 16384 data: 4 3 3 4
26494 034/300 Read block 3451 len 16384
26495 035/300 Write block 4000 len 16384 data: 6 7 7 6
26496 036/300 Read block 4005 len 4096
26497 037/300 Write block 2981 len 32768 data: 4 5 5 5 3 3 3 5
26498 038/300 Read block 4858 len 4096
26499 039/300 Write block 3359 len 32768 data: 3 3 4 5 6 5 5 5
26500 040/300 Read block 3159 len 20480
26501 041/300 Write block 2789 len 20480 data: 5 5 5 5 4
26502 042/300 Read block 4829 len 16384
26503 043/300 Write block 572 len 20480 data: 4 4 3 4 4
26504 044/300 Read block 4233 len 8192
26505 045/300 Flush
26506 046/300 Write block 3563 len 12288 data: 5 5 5
26507 047/300 Read block 3010 len 40960
26508 048/300 Write block 3110 len 40960 data: 2 2 3 3 3 3 4 3 5 5
26509 049/300 Write block 2312 len 4096 data: 2
26510 050/300 Flush
26511 051/300 Read block 1655 len 12288
26512 052/300 Read block 2088 len 32768
26513 053/300 Read block 1837 len 20480
26514 054/300 Write block 118 len 32768 data: 3 3 7 7 7 6 6 7
26515 055/300 Write block 4211 len 20480 data: 4 3 3 4 4
26516 056/300 Read block 1941 len 4096
26517 057/300 Read block 738 len 20480
26518 058/300 Write block 2257 len 8192 data: 8 6
26519 059/300 Write block 2674 len 4096 data: 5
26520 060/300 Flush
26521 061/300 Write block 2048 len 4096 data: 3
26522 062/300 Write block 4409 len 28672 data: 4 5 5 6 4 3 3
26523 063/300 Read block 4729 len 40960
26524 064/300 Read block 4341 len 4096
26525 065/300 Read block 3441 len 4096
26526 066/300 Read block 3823 len 40960
26527 067/300 Read block 3304 len 4096
26528 068/300 Write block 1158 len 4096 data: 5
26529 069/300 Read block 1285 len 4096
26530 070/300 Write block 156 len 4096 data: 6
26531 071/300 Read block 329 len 28672
26532 072/300 Read block 3690 len 28672
26533 073/300 Read block 424 len 24576
26534 074/300 Write block 874 len 20480 data: 6 6 6 5 6
26535 075/300 Flush
26536 076/300 Flush
26537 077/300 Write block 1815 len 20480 data: 4 4 6 7 7
26538 078/300 Write block 3987 len 36864 data: 6 6 8 7 7 7 8 7 5
26539 079/300 Write block 3635 len 40960 data: 5 5 6 6 4 3 3 2 2 2
26540 080/300 Flush
26541 081/300 Flush
26542 082/300 Read block 731 len 24576
26543 083/300 Write block 3564 len 8192 data: 6 6
26544 084/300 Write block 2188 len 4096 data: 5
26545 085/300 Read block 630 len 4096
26546 086/300 Write block 2892 len 16384 data: 5 5 4 4
26547 087/300 Flush
26548 088/300 Read block 3583 len 16384
26549 089/300 Write block 1232 len 20480 data: 6 5 5 5 3
26550 090/300 Write block 160 len 24576 data: 2 2 2 3 3 3
26551 091/300 Write block 1060 len 4096 data: 4
26552 092/300 Write block 2333 len 32768 data: 5 2 2 2 2 3 5 6
26553 093/300 Read block 4665 len 36864
26554 094/300 Read block 1421 len 16384
26555 095/300 Write block 1194 len 12288 data: 8 8 5
26556 096/300 Read block 3590 len 20480
26557 097/300 Write block 4467 len 40960 data: 6 5 6 5 5 6 7 7 7 6
26558 098/300 Flush
26559 099/300 Write block 4187 len 40960 data: 8 8 9 6 6 7 5 6 8 8
26560 100/300 Flush
26561 101/300 Read block 779 len 16384
26562 102/300 Read block 1119 len 16384
26563 103/300 Write block 3626 len 4096 data: 5
26564 104/300 Write block 991 len 28672 data: 7 8 8 6 5 6 6
26565 105/300 Read block 221 len 32768
26566 106/300 Flush
26567 107/300 Flush
26568 108/300 Write block 2178 len 36864 data: 3 4 4 4 5 4 3 4 5
26569 109/300 Read block 1853 len 12288
26570 110/300 Read block 2506 len 20480
26571 111/300 Write block 3689 len 28672 data: 5 6 5 5 4 4 5
26572 112/300 Read block 2605 len 4096
26573 113/300 Read block 1456 len 20480
26574 114/300 Read block 1139 len 40960
26575 115/300 Write block 4436 len 36864 data: 4 5 5 5 5 4 3 2 2
26576 116/300 Read block 1620 len 8192
26577 117/300 Read block 2303 len 12288
26578 118/300 Write block 4817 len 32768 data: 3 3 2 2 3 2 2 2
26579 119/300 Write block 2090 len 28672 data: 5 6 4 4 4 5 4
26580 120/300 Write block 1036 len 32768 data: 4 3 5 4 4 6 6 6
26581 121/300 Write block 4898 len 32768 data: 9 7 6 5 5 5 4 3
26582 122/300 Read block 2447 len 24576
26583 123/300 Read block 830 len 36864
26584 124/300 Write block 3890 len 4096 data: 5
26585 125/300 Write block 2179 len 16384 data: 5 5 5 6
26586 126/300 Write block 1120 len 4096 data: 5
26587 127/300 Read block 1335 len 28672
26588 128/300 Flush
26589 129/300 Read block 1135 len 20480
26590 130/300 Write block 56 len 20480 data: 3 4 5 6 7
26591 131/300 Write block 3124 len 12288 data: 8 9 13
26592 132/300 Write block 1364 len 12288 data: 4 4 6
26593 133/300 Read block 3305 len 36864
26594 134/300 Read block 1547 len 20480
26595 135/300 Read block 3001 len 8192
26596 136/300 Read block 1405 len 40960
26597 137/300 Write block 4820 len 12288 data: 3 4 3
26598 138/300 Read block 4873 len 12288
26599 139/300 Read block 1210 len 24576
26600 140/300 Write block 3593 len 36864 data: 6 8 8 8 8 6 6 5 7
26601 141/300 Flush
26602 142/300 Write block 3253 len 28672 data: 5 4 3 3 4 4 4
26603 143/300 Flush
26604 144/300 Write block 1829 len 8192 data: 6 5
26605 145/300 Read block 814 len 24576
26606 146/300 Read block 2080 len 36864
26607 147/300 Read block 3737 len 36864
26608 148/300 Write block 1511 len 16384 data: 5 6 6 4
26609 149/300 Read block 3052 len 16384
26610 150/300 Read block 3846 len 32768
26611 151/300 Read block 359 len 12288
26612 152/300 Read block 2007 len 12288
26613 153/300 Write block 1040 len 36864 data: 5 7 7 7 6 8 8 7 7
26614 154/300 Write block 4898 len 28672 data: 10 8 7 6 6 6 5
26615 155/300 Write block 2922 len 16384 data: 9 9 7 7
26616 156/300 Write block 4279 len 20480 data: 2 3 4 5 4
26617 157/300 Read block 1259 len 20480
26618 158/300 Flush
26619 159/300 Write block 455 len 28672 data: 6 6 5 5 4 4 4
26620 160/300 Write block 1407 len 16384 data: 8 6 6 6
26621 161/300 Write block 2742 len 8192 data: 4 4
26622 162/300 Write block 4724 len 16384 data: 3 5 6 5
26623 163/300 Read block 81 len 12288
26624 164/300 Read block 2373 len 28672
26625 165/300 Write block 1515 len 4096 data: 5
26626 166/300 Read block 3067 len 40960
26627 167/300 Write block 927 len 12288 data: 3 3 3
26628 168/300 Read block 4037 len 32768
26629 169/300 Read block 2560 len 20480
26630 170/300 Write block 1850 len 4096 data: 4
26631 171/300 Read block 4055 len 32768
26632 172/300 Write block 4038 len 20480 data: 5 4 3 5 4
26633 173/300 Read block 3951 len 4096
26634 174/300 Write block 4924 len 40960 data: 4 4 5 8 7 8 6 6 9 11
26635 175/300 Read block 3209 len 20480
26636 176/300 Read block 2659 len 12288
26637 177/300 Write block 458 len 28672 data: 6 5 5 5 4 2 2
26638 178/300 Read block 1100 len 28672
26639 179/300 Read block 146 len 8192
26640 180/300 Write block 1114 len 16384 data: 6 6 6 5
26641 181/300 Read block 526 len 32768
26642 182/300 Read block 2632 len 20480
26643 183/300 Write block 3448 len 16384 data: 5 7 7 7
26644 184/300 Write block 3793 len 28672 data: 6 7 6 6 6 6 8
26645 185/300 Read block 3231 len 20480
26646 186/300 Read block 4626 len 36864
26647 187/300 Write block 3308 len 8192 data: 6 6
26648 188/300 Write block 4424 len 32768 data: 7 7 7 7 6 5 4 5
26649 189/300 Read block 4132 len 36864
26650 190/300 Write block 2575 len 4096 data: 6
26651 191/300 Read block 1554 len 12288
26652 192/300 Read block 553 len 20480
26653 193/300 Write block 4812 len 16384 data: 3 2 2 2
26654 194/300 Flush
26655 195/300 Read block 1158 len 16384
26656 196/300 Read block 3105 len 24576
26657 197/300 Write block 4502 len 8192 data: 7 6
26658 198/300 Read block 667 len 16384
26659 199/300 Read block 1861 len 28672
26660 200/300 Read block 3837 len 16384
26661 201/300 Flush
26662 202/300 Read block 4612 len 4096
26663 203/300 Read block 4097 len 16384
26664 204/300 Read block 2672 len 12288
26665 205/300 Write block 3241 len 32768 data: 8 7 7 6 8 7 7 7
26666 206/300 Read block 2639 len 16384
26667 207/300 Write block 212 len 8192 data: 4 5
26668 208/300 Write block 4916 len 28672 data: 2 2 3 3 3 4 4
26669 209/300 Write block 3778 len 36864 data: 6 5 5 4 4 4 3 4 6
26670 210/300 Read block 4713 len 8192
26671 211/300 Read block 4138 len 32768
26672 212/300 Read block 3650 len 32768
26673 213/300 Read block 254 len 40960
26674 214/300 Write block 169 len 12288 data: 4 4 4
26675 215/300 Flush
26676 216/300 Write block 1585 len 40960 data: 9 8 7 6 5 4 4 3 2 3
26677 217/300 Read block 4697 len 40960
26678 218/300 Read block 4898 len 8192
26679 219/300 Write block 4079 len 12288 data: 3 5 5
26680 220/300 Read block 3272 len 36864
26681 221/300 Write block 1295 len 8192 data: 3 2
26682 222/300 Read block 1676 len 36864
26683 223/300 Read block 183 len 12288
26684 224/300 Write block 3040 len 12288 data: 3 4 3
26685 225/300 Read block 4924 len 8192
26686 226/300 Read block 2768 len 36864
26687 227/300 Read block 3605 len 16384
26688 228/300 Flush
26689 229/300 Write block 207 len 4096 data: 3
26690 230/300 Flush
26691 231/300 Write block 2998 len 20480 data: 4 3 3 3 3
26692 232/300 Read block 4469 len 16384
26693 233/300 Read block 2741 len 4096
26694 234/300 Write block 2558 len 28672 data: 3 4 7 5 5 6 6
26695 235/300 Write block 4996 len 16384 data: 4 3 4 4
26696 236/300 Write block 1621 len 12288 data: 5 5 5
26697 237/300 Write block 853 len 24576 data: 2 4 4 5 5 5
26698 238/300 Read block 3160 len 32768
26699 239/300 Write block 416 len 4096 data: 5
26700 240/300 Write block 483 len 8192 data: 4 4
26701 241/300 Write block 2812 len 24576 data: 5 6 6 5 4 4
26702 242/300 Read block 4079 len 40960
26703 243/300 Read block 684 len 36864
26704 244/300 Read block 3056 len 36864
26705 245/300 Write block 1541 len 4096 data: 4
26706 246/300 Read block 4474 len 28672
26707 247/300 Read block 3287 len 36864
26708 248/300 Read block 3342 len 16384
26709 249/300 Read block 3188 len 32768
26710 250/300 Flush
26711 251/300 Write block 2054 len 36864 data: 4 5 5 3 3 3 3 3 4
26712 252/300 Read block 2064 len 36864
26713 253/300 Read block 4956 len 28672
26714 254/300 Read block 1721 len 24576
26715 255/300 Read block 4589 len 8192
26716 256/300 Write block 165 len 32768 data: 4 3 4 4 5 5 5 3
26717 257/300 Write block 1239 len 4096 data: 2
26718 258/300 Read block 1873 len 12288
26719 259/300 Write block 2211 len 32768 data: 7 7 9 6 6 5 5 5
26720 260/300 Flush
26721 261/300 Read block 2783 len 12288
26722 262/300 Write block 3938 len 12288 data: 3 2 2
26723 263/300 Read block 2331 len 12288
26724 264/300 Write block 3509 len 8192 data: 4 4
26725 265/300 Write block 3388 len 16384 data: 7 7 6 6
26726 266/300 Read block 788 len 24576
26727 267/300 Write block 1960 len 40960 data: 4 3 3 3 3 4 4 5 5 5
26728 268/300 Read block 3520 len 16384
26729 269/300 Write block 1813 len 8192 data: 6 6
26730 270/300 Write block 673 len 16384 data: 10 10 9 9
26731 271/300 Write block 3125 len 8192 data: 10 14
26732 272/300 Flush
26733 273/300 Write block 4197 len 4096 data: 7
26734 274/300 Read block 2153 len 24576
26735 275/300 Read block 1780 len 36864
26736 276/300 Write block 2751 len 32768 data: 5 5 5 3 2 2 2 2
26737 277/300 Write block 2935 len 16384 data: 8 10 9 7
26738 278/300 Write block 2923 len 28672 data: 10 8 8 5 4 4 5
26739 279/300 Read block 1538 len 12288
26740 280/300 Read block 95 len 36864
26741 281/300 Write block 209 len 16384 data: 4 4 3 5
26742 282/300 Read block 816 len 28672
26743 283/300 Read block 4772 len 32768
26744 284/300 Read block 3430 len 32768
26745 285/300 Write block 1329 len 28672 data: 5 6 7 9 10 9 7
26746 286/300 Write block 1232 len 16384 data: 7 6 6 6
26747 287/300 Read block 4267 len 28672
26748 288/300 Write block 3742 len 20480 data: 11 11 10 8 6
26749 289/300 Read block 2321 len 24576
26750 290/300 Write block 2513 len 24576 data: 5 4 4 3 2 4
26751 291/300 Read block 4408 len 32768
26752 292/300 Flush
26753 293/300 Write block 4097 len 28672 data: 4 6 4 4 4 4 4
26754 294/300 Write block 2561 len 36864 data: 6 6 7 7 6 6 6 6 6
26755 295/300 Read block 3115 len 12288
26756 296/300 Read block 1732 len 4096
26757 297/300 Read block 3527 len 24576
26758 298/300 Write block 1734 len 20480 data: 4 4 6 7 6
26759 299/300 Flush
26760 300/300 Read block 1711 len 20480
26761 301/300 Flush
26762 Main task has work to do, go find it
26763 got dsc Start(0)
26764 start 0
26765 Nov 19 17:13:37.651 INFO request completed, latency_us: 236, response_code: 204, uri: /start/cid/0, method: POST, req_id: c8ded38a-72ea-4745-87d4-54ee1432f330, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
26766 Replay: started 0, returned:Ok(())
26767 ----------------------------------------------------------------
26768 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26769 States: Offline Active Active
26770 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26771 7944 Acked Write 9 Sent Done Done false
26772 7945 Acked Write 6 Sent Done Done false
26773 7946 Acked Read 10 Sent Done Done false
26774 7947 Acked Read 8 Sent Done Done false
26775 7948 Acked Read 3 Sent Done Done false
26776 7949 Acked Write 3 Sent Done Done false
26777 7950 Acked Write 7 Sent Done Done false
26778 7951 Acked Read 2 Sent Done Done false
26779 7952 Acked Read 10 Sent Done Done false
26780 7953 Acked Write 3 Sent Done Done false
26781 7954 Acked Read 10 Sent Done Done false
26782 7955 Acked Write 6 Sent Done Done false
26783 7956 Acked Read 1 Sent Done Done false
26784 7957 Acked Read 4 Sent Done Done false
26785 7958 Acked Read 3 Sent Done Done false
26786 7959 Acked Read 1 Sent Done Done false
26787 7960 Acked Write 1 Sent Done Done false
26788 7961 Acked Read 6 Sent Done Done false
26789 7962 Acked Read 2 Sent Done Done false
26790 7963 Acked Write 8 Sent Done Done false
26791 7964 Acked Read 3 Sent Done Done false
26792 7965 Acked Flush 0 Sent Done Done false
26793 7966 Acked Read 2 Sent Done Done false
26794 7967 Acked Flush 0 Sent Done Done false
26795 7968 Acked Read 3 Sent Done Done false
26796 7969 Acked Flush 0 Sent Done Done false
26797 7970 Acked Write 9 Sent Done Done false
26798 7971 Acked Read 4 Sent Done Done false
26799 7972 Acked Read 6 Sent Done Done false
26800 7973 Acked Read 10 Sent Done Done false
26801 7974 Acked Read 5 Sent Done Done false
26802 7975 Acked Write 6 Sent Done Done false
26803 7976 Acked Write 4 Sent Done Done false
26804 7977 Acked Read 4 Sent Done Done false
26805 7978 Acked Write 4 Sent Done Done false
26806 7979 Acked Read 1 Sent Done Done false
26807 7980 Acked Write 8 Sent Done Done false
26808 7981 Acked Read 1 Sent Done Done false
26809 7982 Acked Write 8 Sent Done Done false
26810 7983 Acked Read 5 Sent Done Done false
26811 7984 Acked Write 5 Sent Done Done false
26812 7985 Acked Read 4 Sent Done Done false
26813 7986 Acked Write 5 Sent Done Done false
26814 7987 Acked Read 2 Sent Done Done false
26815 7988 Acked Flush 0 Sent Done Done false
26816 7989 Acked Write 3 Sent Done Done false
26817 7990 Acked Read 10 Sent Done Done false
26818 7991 Acked Write 10 Sent Done Done false
26819 7992 Acked Write 1 Sent Done Done false
26820 7993 Acked Flush 0 Sent Done Done false
26821 7994 Acked Read 3 Sent Done Done false
26822 7995 Acked Read 8 Sent Done Done false
26823 7996 Acked Read 5 Sent Done Done false
26824 7997 Acked Write 8 Sent Done Done false
26825 7998 Acked Write 5 Sent Done Done false
26826 7999 Acked Read 1 Sent Done Done false
26827 8000 Acked Read 5 Sent Done Done false
26828 8001 Acked Write 2 Sent Done Done false
26829 8002 Acked Write 1 Sent Done Done false
26830 8003 Acked Flush 0 Sent Done Done false
26831 8004 Acked Write 1 Sent Done Done false
26832 8005 Acked Write 7 Sent Done Done false
26833 8006 Acked Read 10 Sent Done Done false
26834 8007 Acked Read 1 Sent Done Done false
26835 8008 Acked Read 1 Sent Done Done false
26836 8009 Acked Read 10 Sent Done Done false
26837 8010 Acked Read 1 Sent Done Done false
26838 8011 Acked Write 1 Sent Done Done false
26839 8012 Acked Read 1 Sent Done Done false
26840 8013 Acked Write 1 Sent Done Done false
26841 8014 Acked Read 7 Sent Done Done false
26842 8015 Acked Read 7 Sent Done Done false
26843 8016 Acked Read 6 Sent Done Done false
26844 8017 Acked Write 5 Sent Done Done false
26845 8018 Acked Flush 0 Sent Done Done false
26846 8019 Acked Flush 0 Sent Done Done false
26847 8020 Acked Write 5 Sent Done Done false
26848 8021 Acked Write 9 Sent Done Done false
26849 8022 Acked Write 10 Sent Done Done false
26850 8023 Acked Flush 0 Sent Done Done false
26851 8024 Acked Flush 0 Sent Done Done false
26852 8025 Acked Read 6 Sent Done Done false
26853 8026 Acked Write 2 Sent Done Done false
26854 8027 Acked Write 1 Sent Done Done false
26855 8028 Acked Read 1 Sent Done Done false
26856 8029 Acked Write 4 Sent Done Done false
26857 8030 Acked Flush 0 Sent Done Done false
26858 8031 Acked Read 4 Sent Done Done false
26859 8032 Acked Write 5 Sent Done Done false
26860 8033 Acked Write 6 Sent Done Done false
26861 8034 Acked Write 1 Sent Done Done false
26862 8035 Acked Write 8 Sent Done Done false
26863 8036 Acked Read 9 Sent Done Done false
26864 8037 Acked Read 4 Sent Done Done false
26865 8038 Acked Write 3 Sent Done Done false
26866 8039 Acked Read 5 Sent Done Done false
26867 8040 Acked Write 10 Sent Done Done false
26868 8041 Acked Flush 0 Sent Done Done false
26869 8042 Acked Write 10 Sent Done Done false
26870 8043 Acked Flush 0 Sent Done Done false
26871 8044 Acked Read 4 Sent Done Done false
26872 8045 Acked Read 4 Sent Done Done false
26873 8046 Acked Write 1 Sent Done Done false
26874 8047 Acked Write 7 Sent Done Done false
26875 8048 Acked Read 8 Sent Done Done false
26876 8049 Acked Flush 0 Sent Done Done false
26877 8050 Acked Flush 0 Sent Done Done false
26878 8051 Acked Write 9 Sent Done Done false
26879 8052 Acked Read 3 Sent Done Done false
26880 8053 Acked Read 5 Sent Done Done false
26881 8054 Acked Write 7 Sent Done Done false
26882 8055 Acked Read 1 Sent Done Done false
26883 8056 Acked Read 5 Sent Done Done false
26884 8057 Acked Read 10 Sent Done Done false
26885 8058 Acked Write 9 Sent Done Done false
26886 8059 Acked Read 2 Sent Done Done false
26887 8060 Acked Read 3 Sent Done Done false
26888 8061 Acked Write 8 Sent Done Done false
26889 8062 Acked Write 7 Sent Done Done false
26890 8063 Acked Write 8 Sent Done Done false
26891 8064 Acked Write 8 Sent Done Done false
26892 8065 Acked Read 6 Sent Done Done false
26893 8066 Acked Read 9 Sent Done Done false
26894 8067 Acked Write 1 Sent Done Done false
26895 8068 Acked Write 4 Sent Done Done false
26896 8069 Acked Write 1 Sent Done Done false
26897 8070 Acked Read 7 Sent Done Done false
26898 8071 Acked Flush 0 Sent Done Done false
26899 8072 Acked Read 5 Sent Done Done false
26900 8073 Acked Write 5 Sent Done Done false
26901 8074 Acked Write 3 Sent Done Done false
26902 8075 Acked Write 3 Sent Done Done false
26903 8076 Acked Read 9 Sent Done Done false
26904 8077 Acked Read 5 Sent Done Done false
26905 8078 Acked Read 2 Sent Done Done false
26906 8079 Acked Read 10 Sent Done Done false
26907 8080 Acked Write 3 Sent Done Done false
26908 8081 Acked Read 3 Sent Done Done false
26909 8082 Acked Read 6 Sent Done Done false
26910 8083 Acked Write 9 Sent Done Done false
26911 8084 Acked Flush 0 Sent Done Done false
26912 8085 Acked Write 7 Sent Done Done false
26913 8086 Acked Flush 0 Sent Done Done false
26914 8087 Acked Write 2 Sent Done Done false
26915 8088 Acked Read 6 Sent Done Done false
26916 8089 Acked Read 9 Sent Done Done false
26917 8090 Acked Read 9 Sent Done Done false
26918 8091 Acked Write 4 Sent Done Done false
26919 8092 Acked Read 4 Sent Done Done false
26920 8093 Acked Read 8 Sent Done Done false
26921 8094 Acked Read 3 Sent Done Done false
26922 8095 Acked Read 3 Sent Done Done false
26923 8096 Acked Write 9 Sent Done Done false
26924 8097 Acked Write 7 Sent Done Done false
26925 8098 Acked Write 4 Sent Done Done false
26926 8099 Acked Write 5 Sent Done Done false
26927 8100 Acked Read 5 Sent Done Done false
26928 8101 Acked Flush 0 Sent Done Done false
26929 8102 Acked Write 7 Sent Done Done false
26930 8103 Acked Write 4 Sent Done Done false
26931 8104 Acked Write 2 Sent Done Done false
26932 8105 Acked Write 4 Sent Done Done false
26933 8106 Acked Read 3 Sent Done Done false
26934 8107 Acked Read 7 Sent Done Done false
26935 8108 Acked Write 1 Sent Done Done false
26936 8109 Acked Read 10 Sent Done Done false
26937 8110 Acked Write 3 Sent Done Done false
26938 8111 Acked Read 8 Sent Done Done false
26939 8112 Acked Read 5 Sent Done Done false
26940 8113 Acked Write 1 Sent Done Done false
26941 8114 Acked Read 8 Sent Done Done false
26942 8115 Acked Write 5 Sent Done Done false
26943 8116 Acked Read 1 Sent Done Done false
26944 8117 Acked Write 10 Sent Done Done false
26945 8118 Acked Read 5 Sent Done Done false
26946 8119 Acked Read 3 Sent Done Done false
26947 8120 Acked Write 7 Sent Done Done false
26948 8121 Acked Read 7 Sent Done Done false
26949 8122 Acked Read 2 Sent Done Done false
26950 8123 Acked Write 4 Sent Done Done false
26951 8124 Acked Read 8 Sent Done Done false
26952 8125 Acked Read 5 Sent Done Done false
26953 8126 Acked Write 4 Sent Done Done false
26954 8127 Acked Write 7 Sent Done Done false
26955 8128 Acked Read 5 Sent Done Done false
26956 8129 Acked Read 9 Sent Done Done false
26957 8130 Acked Write 2 Sent Done Done false
26958 8131 Acked Write 8 Sent Done Done false
26959 8132 Acked Read 9 Sent Done Done false
26960 8133 Acked Write 1 Sent Done Done false
26961 8134 Acked Read 3 Sent Done Done false
26962 8135 Acked Read 5 Sent Done Done false
26963 8136 Acked Write 4 Sent Done Done false
26964 8137 Acked Flush 0 Sent Done Done false
26965 8138 Acked Read 4 Sent Done Done false
26966 8139 Acked Read 6 Sent Done Done false
26967 8140 Acked Write 2 Sent Done Done false
26968 8141 Acked Read 4 Sent Done Done false
26969 8142 Acked Read 7 Sent Done Done false
26970 8143 Acked Read 4 Sent Done Done false
26971 8144 Acked Flush 0 Sent Done Done false
26972 8145 Acked Read 1 Sent Done Done false
26973 8146 Acked Read 4 Sent Done Done false
26974 8147 Acked Read 3 Sent Done Done false
26975 8148 Acked Write 8 Sent Done Done false
26976 8149 Acked Read 4 Sent Done Done false
26977 8150 Acked Write 2 Sent Done Done false
26978 8151 Acked Write 7 Sent Done Done false
26979 8152 Acked Write 9 Sent Done Done false
26980 8153 Acked Read 2 Sent Done Done false
26981 8154 Acked Read 8 Sent Done Done false
26982 8155 Acked Read 8 Sent Done Done false
26983 8156 Acked Read 10 Sent Done Done false
26984 8157 Acked Write 3 Sent Done Done false
26985 8158 Acked Flush 0 Sent Done Done false
26986 8159 Acked Write 10 Sent Done Done false
26987 8160 Acked Read 10 Sent Done Done false
26988 8161 Acked Read 2 Sent Done Done false
26989 8162 Acked Write 3 Sent Done Done false
26990 8163 Acked Read 9 Sent Done Done false
26991 8164 Acked Write 2 Sent Done Done false
26992 8165 Acked Read 9 Sent Done Done false
26993 8166 Acked Read 3 Sent Done Done false
26994 8167 Acked Write 3 Sent Done Done false
26995 8168 Acked Read 2 Sent Done Done false
26996 8169 Acked Read 9 Sent Done Done false
26997 8170 Acked Read 4 Sent Done Done false
26998 8171 Acked Flush 0 Sent Done Done false
26999 8172 Acked Write 1 Sent Done Done false
27000 8173 Acked Flush 0 Sent Done Done false
27001 8174 Acked Write 5 Sent Done Done false
27002 8175 Acked Read 4 Sent Done Done false
27003 8176 Acked Read 1 Sent Done Done false
27004 8177 Acked Write 7 Sent Done Done false
27005 8178 Acked Write 4 Sent Done Done false
27006 8179 Acked Write 3 Sent Done Done false
27007 8180 Acked Write 6 Sent Done Done false
27008 8181 Acked Read 8 Sent Done Done false
27009 8182 Acked Write 1 Sent Done Done false
27010 8183 Acked Write 2 Sent Done Done false
27011 8184 Acked Write 6 Sent Done Done false
27012 8185 Acked Read 10 Sent Done Done false
27013 8186 Acked Read 9 Sent Done Done false
27014 8187 Acked Read 9 Sent Done Done false
27015 8188 Acked Write 1 Sent Done Done false
27016 8189 Acked Read 7 Sent Done Done false
27017 8190 Acked Read 9 Sent Done Done false
27018 8191 Acked Read 4 Sent Done Done false
27019 8192 Acked Read 8 Sent Done Done false
27020 8193 Acked Flush 0 Sent Done Done false
27021 8194 Acked Write 9 Sent Done Done false
27022 8195 Acked Read 9 Sent Done Done false
27023 8196 Acked Read 7 Sent Done Done false
27024 8197 Acked Read 6 Sent Done Done false
27025 8198 Acked Read 2 Sent Done Done false
27026 8199 Acked Write 8 Sent Done Done false
27027 8200 Acked Write 1 Sent Done Done false
27028 8201 Acked Read 3 Sent Done Done false
27029 8202 Acked Write 8 Sent Done Done false
27030 8203 Acked Flush 0 Sent Done Done false
27031 8204 Acked Read 3 Sent Done Done false
27032 8205 Acked Write 3 Sent Done Done false
27033 8206 Acked Read 3 Sent Done Done false
27034 8207 Acked Write 2 Sent Done Done false
27035 8208 Acked Write 4 Sent Done Done false
27036 8209 Acked Read 6 Sent Done Done false
27037 8210 Acked Write 10 Sent Done Done false
27038 8211 Acked Read 4 Sent Done Done false
27039 8212 Acked Write 2 Sent Done Done false
27040 8213 Acked Write 4 Sent Done Done false
27041 8214 Acked Write 2 Sent Done Done false
27042 8215 Acked Flush 0 Sent Done Done false
27043 8216 Acked Write 1 Sent Done Done false
27044 8217 Acked Read 6 Sent Done Done false
27045 8218 Acked Read 9 Sent Done Done false
27046 8219 Acked Write 8 Sent Done Done false
27047 8220 Acked Write 4 Sent Done Done false
27048 8221 Acked Write 7 Sent Done Done false
27049 8222 Acked Read 3 Sent Done Done false
27050 8223 Acked Read 9 Sent Done Done false
27051 8224 Acked Write 4 Sent Done Done false
27052 8225 Acked Read 7 Sent Done Done false
27053 8226 Acked Read 8 Sent Done Done false
27054 8227 Acked Read 8 Sent Done Done false
27055 8228 Acked Write 7 Sent Done Done false
27056 8229 Acked Write 4 Sent Done Done false
27057 8230 Acked Read 7 Sent Done Done false
27058 8231 Acked Write 5 Sent Done Done false
27059 8232 Acked Read 6 Sent Done Done false
27060 8233 Acked Write 6 Sent Done Done false
27061 8234 Acked Read 8 Sent Done Done false
27062 8235 Acked Flush 0 Sent Done Done false
27063 8236 Acked Write 7 Sent Done Done false
27064 8237 Acked Write 9 Sent Done Done false
27065 8238 Acked Read 3 Sent Done Done false
27066 8239 Acked Read 1 Sent Done Done false
27067 8240 Acked Read 6 Sent Done Done false
27068 8241 Acked Write 5 Sent Done Done false
27069 8242 Acked Flush 0 Sent Done Done false
27070 8243 Acked Read 5 Sent Done Done false
27071 8244 Acked Flush 0 Sent Done Done false
27072 STATES DS:0 DS:1 DS:2 TOTAL
27073 Sent 301 0 0 301
27074 Done 0 301 301 602
27075 Skipped 0 0 0 0
27076 Error 0 0 0 0
27077 Last Flush: 7943 8244 8244
27078 Downstairs last five completed: 7943 7942 7941 7940 7939
27079 Upstairs last five completed: 8244 8243 8242 8241 8240
27080 CLIENT: Up:0 ds:301 act:2
27081 [8810] got start action
27082 [8810] I am going to restart
27083 Starting downstairs at port 8810
27084 Make output file at "/tmp/dsc/downstairs-8810.txt"
27085 [8810][0] reports Starting
27086 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1226)
27087 [8810][0] reports Running
27088 ----------------------------------------------------------------
27089 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27090 States: Offline Active Active
27091 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27092 7944 Acked Write 9 Sent Done Done false
27093 7945 Acked Write 6 Sent Done Done false
27094 7946 Acked Read 10 Sent Done Done false
27095 7947 Acked Read 8 Sent Done Done false
27096 7948 Acked Read 3 Sent Done Done false
27097 7949 Acked Write 3 Sent Done Done false
27098 7950 Acked Write 7 Sent Done Done false
27099 7951 Acked Read 2 Sent Done Done false
27100 7952 Acked Read 10 Sent Done Done false
27101 7953 Acked Write 3 Sent Done Done false
27102 7954 Acked Read 10 Sent Done Done false
27103 7955 Acked Write 6 Sent Done Done false
27104 7956 Acked Read 1 Sent Done Done false
27105 7957 Acked Read 4 Sent Done Done false
27106 7958 Acked Read 3 Sent Done Done false
27107 7959 Acked Read 1 Sent Done Done false
27108 7960 Acked Write 1 Sent Done Done false
27109 7961 Acked Read 6 Sent Done Done false
27110 7962 Acked Read 2 Sent Done Done false
27111 7963 Acked Write 8 Sent Done Done false
27112 7964 Acked Read 3 Sent Done Done false
27113 7965 Acked Flush 0 Sent Done Done false
27114 7966 Acked Read 2 Sent Done Done false
27115 7967 Acked Flush 0 Sent Done Done false
27116 7968 Acked Read 3 Sent Done Done false
27117 7969 Acked Flush 0 Sent Done Done false
27118 7970 Acked Write 9 Sent Done Done false
27119 7971 Acked Read 4 Sent Done Done false
27120 7972 Acked Read 6 Sent Done Done false
27121 7973 Acked Read 10 Sent Done Done false
27122 7974 Acked Read 5 Sent Done Done false
27123 7975 Acked Write 6 Sent Done Done false
27124 7976 Acked Write 4 Sent Done Done false
27125 7977 Acked Read 4 Sent Done Done false
27126 7978 Acked Write 4 Sent Done Done false
27127 7979 Acked Read 1 Sent Done Done false
27128 7980 Acked Write 8 Sent Done Done false
27129 7981 Acked Read 1 Sent Done Done false
27130 7982 Acked Write 8 Sent Done Done false
27131 7983 Acked Read 5 Sent Done Done false
27132 7984 Acked Write 5 Sent Done Done false
27133 7985 Acked Read 4 Sent Done Done false
27134 7986 Acked Write 5 Sent Done Done false
27135 7987 Acked Read 2 Sent Done Done false
27136 7988 Acked Flush 0 Sent Done Done false
27137 7989 Acked Write 3 Sent Done Done false
27138 7990 Acked Read 10 Sent Done Done false
27139 7991 Acked Write 10 Sent Done Done false
27140 7992 Acked Write 1 Sent Done Done false
27141 7993 Acked Flush 0 Sent Done Done false
27142 7994 Acked Read 3 Sent Done Done false
27143 7995 Acked Read 8 Sent Done Done false
27144 7996 Acked Read 5 Sent Done Done false
27145 7997 Acked Write 8 Sent Done Done false
27146 7998 Acked Write 5 Sent Done Done false
27147 7999 Acked Read 1 Sent Done Done false
27148 8000 Acked Read 5 Sent Done Done false
27149 8001 Acked Write 2 Sent Done Done false
27150 8002 Acked Write 1 Sent Done Done false
27151 8003 Acked Flush 0 Sent Done Done false
27152 8004 Acked Write 1 Sent Done Done false
27153 8005 Acked Write 7 Sent Done Done false
27154 8006 Acked Read 10 Sent Done Done false
27155 8007 Acked Read 1 Sent Done Done false
27156 8008 Acked Read 1 Sent Done Done false
27157 8009 Acked Read 10 Sent Done Done false
27158 8010 Acked Read 1 Sent Done Done false
27159 8011 Acked Write 1 Sent Done Done false
27160 8012 Acked Read 1 Sent Done Done false
27161 8013 Acked Write 1 Sent Done Done false
27162 8014 Acked Read 7 Sent Done Done false
27163 8015 Acked Read 7 Sent Done Done false
27164 8016 Acked Read 6 Sent Done Done false
27165 8017 Acked Write 5 Sent Done Done false
27166 8018 Acked Flush 0 Sent Done Done false
27167 8019 Acked Flush 0 Sent Done Done false
27168 8020 Acked Write 5 Sent Done Done false
27169 8021 Acked Write 9 Sent Done Done false
27170 8022 Acked Write 10 Sent Done Done false
27171 8023 Acked Flush 0 Sent Done Done false
27172 8024 Acked Flush 0 Sent Done Done false
27173 8025 Acked Read 6 Sent Done Done false
27174 8026 Acked Write 2 Sent Done Done false
27175 8027 Acked Write 1 Sent Done Done false
27176 8028 Acked Read 1 Sent Done Done false
27177 8029 Acked Write 4 Sent Done Done false
27178 8030 Acked Flush 0 Sent Done Done false
27179 8031 Acked Read 4 Sent Done Done false
27180 8032 Acked Write 5 Sent Done Done false
27181 8033 Acked Write 6 Sent Done Done false
27182 8034 Acked Write 1 Sent Done Done false
27183 8035 Acked Write 8 Sent Done Done false
27184 8036 Acked Read 9 Sent Done Done false
27185 8037 Acked Read 4 Sent Done Done false
27186 8038 Acked Write 3 Sent Done Done false
27187 8039 Acked Read 5 Sent Done Done false
27188 8040 Acked Write 10 Sent Done Done false
27189 8041 Acked Flush 0 Sent Done Done false
27190 8042 Acked Write 10 Sent Done Done false
27191 8043 Acked Flush 0 Sent Done Done false
27192 8044 Acked Read 4 Sent Done Done false
27193 8045 Acked Read 4 Sent Done Done false
27194 8046 Acked Write 1 Sent Done Done false
27195 8047 Acked Write 7 Sent Done Done false
27196 8048 Acked Read 8 Sent Done Done false
27197 8049 Acked Flush 0 Sent Done Done false
27198 8050 Acked Flush 0 Sent Done Done false
27199 8051 Acked Write 9 Sent Done Done false
27200 8052 Acked Read 3 Sent Done Done false
27201 8053 Acked Read 5 Sent Done Done false
27202 8054 Acked Write 7 Sent Done Done false
27203 8055 Acked Read 1 Sent Done Done false
27204 8056 Acked Read 5 Sent Done Done false
27205 8057 Acked Read 10 Sent Done Done false
27206 8058 Acked Write 9 Sent Done Done false
27207 8059 Acked Read 2 Sent Done Done false
27208 8060 Acked Read 3 Sent Done Done false
27209 8061 Acked Write 8 Sent Done Done false
27210 8062 Acked Write 7 Sent Done Done false
27211 8063 Acked Write 8 Sent Done Done false
27212 8064 Acked Write 8 Sent Done Done false
27213 8065 Acked Read 6 Sent Done Done false
27214 8066 Acked Read 9 Sent Done Done false
27215 8067 Acked Write 1 Sent Done Done false
27216 8068 Acked Write 4 Sent Done Done false
27217 8069 Acked Write 1 Sent Done Done false
27218 8070 Acked Read 7 Sent Done Done false
27219 8071 Acked Flush 0 Sent Done Done false
27220 8072 Acked Read 5 Sent Done Done false
27221 8073 Acked Write 5 Sent Done Done false
27222 8074 Acked Write 3 Sent Done Done false
27223 8075 Acked Write 3 Sent Done Done false
27224 8076 Acked Read 9 Sent Done Done false
27225 8077 Acked Read 5 Sent Done Done false
27226 8078 Acked Read 2 Sent Done Done false
27227 8079 Acked Read 10 Sent Done Done false
27228 8080 Acked Write 3 Sent Done Done false
27229 8081 Acked Read 3 Sent Done Done false
27230 8082 Acked Read 6 Sent Done Done false
27231 8083 Acked Write 9 Sent Done Done false
27232 8084 Acked Flush 0 Sent Done Done false
27233 8085 Acked Write 7 Sent Done Done false
27234 8086 Acked Flush 0 Sent Done Done false
27235 8087 Acked Write 2 Sent Done Done false
27236 8088 Acked Read 6 Sent Done Done false
27237 8089 Acked Read 9 Sent Done Done false
27238 8090 Acked Read 9 Sent Done Done false
27239 8091 Acked Write 4 Sent Done Done false
27240 8092 Acked Read 4 Sent Done Done false
27241 8093 Acked Read 8 Sent Done Done false
27242 8094 Acked Read 3 Sent Done Done false
27243 8095 Acked Read 3 Sent Done Done false
27244 8096 Acked Write 9 Sent Done Done false
27245 8097 Acked Write 7 Sent Done Done false
27246 8098 Acked Write 4 Sent Done Done false
27247 8099 Acked Write 5 Sent Done Done false
27248 8100 Acked Read 5 Sent Done Done false
27249 8101 Acked Flush 0 Sent Done Done false
27250 8102 Acked Write 7 Sent Done Done false
27251 8103 Acked Write 4 Sent Done Done false
27252 8104 Acked Write 2 Sent Done Done false
27253 8105 Acked Write 4 Sent Done Done false
27254 8106 Acked Read 3 Sent Done Done false
27255 8107 Acked Read 7 Sent Done Done false
27256 8108 Acked Write 1 Sent Done Done false
27257 8109 Acked Read 10 Sent Done Done false
27258 8110 Acked Write 3 Sent Done Done false
27259 8111 Acked Read 8 Sent Done Done false
27260 8112 Acked Read 5 Sent Done Done false
27261 8113 Acked Write 1 Sent Done Done false
27262 8114 Acked Read 8 Sent Done Done false
27263 8115 Acked Write 5 Sent Done Done false
27264 8116 Acked Read 1 Sent Done Done false
27265 8117 Acked Write 10 Sent Done Done false
27266 8118 Acked Read 5 Sent Done Done false
27267 8119 Acked Read 3 Sent Done Done false
27268 8120 Acked Write 7 Sent Done Done false
27269 8121 Acked Read 7 Sent Done Done false
27270 8122 Acked Read 2 Sent Done Done false
27271 8123 Acked Write 4 Sent Done Done false
27272 8124 Acked Read 8 Sent Done Done false
27273 8125 Acked Read 5 Sent Done Done false
27274 8126 Acked Write 4 Sent Done Done false
27275 8127 Acked Write 7 Sent Done Done false
27276 8128 Acked Read 5 Sent Done Done false
27277 8129 Acked Read 9 Sent Done Done false
27278 8130 Acked Write 2 Sent Done Done false
27279 8131 Acked Write 8 Sent Done Done false
27280 8132 Acked Read 9 Sent Done Done false
27281 8133 Acked Write 1 Sent Done Done false
27282 8134 Acked Read 3 Sent Done Done false
27283 8135 Acked Read 5 Sent Done Done false
27284 8136 Acked Write 4 Sent Done Done false
27285 8137 Acked Flush 0 Sent Done Done false
27286 8138 Acked Read 4 Sent Done Done false
27287 8139 Acked Read 6 Sent Done Done false
27288 8140 Acked Write 2 Sent Done Done false
27289 8141 Acked Read 4 Sent Done Done false
27290 8142 Acked Read 7 Sent Done Done false
27291 8143 Acked Read 4 Sent Done Done false
27292 8144 Acked Flush 0 Sent Done Done false
27293 8145 Acked Read 1 Sent Done Done false
27294 8146 Acked Read 4 Sent Done Done false
27295 8147 Acked Read 3 Sent Done Done false
27296 8148 Acked Write 8 Sent Done Done false
27297 8149 Acked Read 4 Sent Done Done false
27298 8150 Acked Write 2 Sent Done Done false
27299 8151 Acked Write 7 Sent Done Done false
27300 8152 Acked Write 9 Sent Done Done false
27301 8153 Acked Read 2 Sent Done Done false
27302 8154 Acked Read 8 Sent Done Done false
27303 8155 Acked Read 8 Sent Done Done false
27304 8156 Acked Read 10 Sent Done Done false
27305 8157 Acked Write 3 Sent Done Done false
27306 8158 Acked Flush 0 Sent Done Done false
27307 8159 Acked Write 10 Sent Done Done false
27308 8160 Acked Read 10 Sent Done Done false
27309 8161 Acked Read 2 Sent Done Done false
27310 8162 Acked Write 3 Sent Done Done false
27311 8163 Acked Read 9 Sent Done Done false
27312 8164 Acked Write 2 Sent Done Done false
27313 8165 Acked Read 9 Sent Done Done false
27314 8166 Acked Read 3 Sent Done Done false
27315 8167 Acked Write 3 Sent Done Done false
27316 8168 Acked Read 2 Sent Done Done false
27317 8169 Acked Read 9 Sent Done Done false
27318 8170 Acked Read 4 Sent Done Done false
27319 8171 Acked Flush 0 Sent Done Done false
27320 8172 Acked Write 1 Sent Done Done false
27321 8173 Acked Flush 0 Sent Done Done false
27322 8174 Acked Write 5 Sent Done Done false
27323 8175 Acked Read 4 Sent Done Done false
27324 8176 Acked Read 1 Sent Done Done false
27325 8177 Acked Write 7 Sent Done Done false
27326 8178 Acked Write 4 Sent Done Done false
27327 8179 Acked Write 3 Sent Done Done false
27328 8180 Acked Write 6 Sent Done Done false
27329 8181 Acked Read 8 Sent Done Done false
27330 8182 Acked Write 1 Sent Done Done false
27331 8183 Acked Write 2 Sent Done Done false
27332 8184 Acked Write 6 Sent Done Done false
27333 8185 Acked Read 10 Sent Done Done false
27334 8186 Acked Read 9 Sent Done Done false
27335 8187 Acked Read 9 Sent Done Done false
27336 8188 Acked Write 1 Sent Done Done false
27337 8189 Acked Read 7 Sent Done Done false
27338 8190 Acked Read 9 Sent Done Done false
27339 8191 Acked Read 4 Sent Done Done false
27340 8192 Acked Read 8 Sent Done Done false
27341 8193 Acked Flush 0 Sent Done Done false
27342 8194 Acked Write 9 Sent Done Done false
27343 8195 Acked Read 9 Sent Done Done false
27344 8196 Acked Read 7 Sent Done Done false
27345 8197 Acked Read 6 Sent Done Done false
27346 8198 Acked Read 2 Sent Done Done false
27347 8199 Acked Write 8 Sent Done Done false
27348 8200 Acked Write 1 Sent Done Done false
27349 8201 Acked Read 3 Sent Done Done false
27350 8202 Acked Write 8 Sent Done Done false
27351 8203 Acked Flush 0 Sent Done Done false
27352 8204 Acked Read 3 Sent Done Done false
27353 8205 Acked Write 3 Sent Done Done false
27354 8206 Acked Read 3 Sent Done Done false
27355 8207 Acked Write 2 Sent Done Done false
27356 8208 Acked Write 4 Sent Done Done false
27357 8209 Acked Read 6 Sent Done Done false
27358 8210 Acked Write 10 Sent Done Done false
27359 8211 Acked Read 4 Sent Done Done false
27360 8212 Acked Write 2 Sent Done Done false
27361 8213 Acked Write 4 Sent Done Done false
27362 8214 Acked Write 2 Sent Done Done false
27363 8215 Acked Flush 0 Sent Done Done false
27364 8216 Acked Write 1 Sent Done Done false
27365 8217 Acked Read 6 Sent Done Done false
27366 8218 Acked Read 9 Sent Done Done false
27367 8219 Acked Write 8 Sent Done Done false
27368 8220 Acked Write 4 Sent Done Done false
27369 8221 Acked Write 7 Sent Done Done false
27370 8222 Acked Read 3 Sent Done Done false
27371 8223 Acked Read 9 Sent Done Done false
27372 8224 Acked Write 4 Sent Done Done false
27373 8225 Acked Read 7 Sent Done Done false
27374 8226 Acked Read 8 Sent Done Done false
27375 8227 Acked Read 8 Sent Done Done false
27376 8228 Acked Write 7 Sent Done Done false
27377 8229 Acked Write 4 Sent Done Done false
27378 8230 Acked Read 7 Sent Done Done false
27379 8231 Acked Write 5 Sent Done Done false
27380 8232 Acked Read 6 Sent Done Done false
27381 8233 Acked Write 6 Sent Done Done false
27382 8234 Acked Read 8 Sent Done Done false
27383 8235 Acked Flush 0 Sent Done Done false
27384 8236 Acked Write 7 Sent Done Done false
27385 8237 Acked Write 9 Sent Done Done false
27386 8238 Acked Read 3 Sent Done Done false
27387 8239 Acked Read 1 Sent Done Done false
27388 8240 Acked Read 6 Sent Done Done false
27389 8241 Acked Write 5 Sent Done Done false
27390 8242 Acked Flush 0 Sent Done Done false
27391 8243 Acked Read 5 Sent Done Done false
27392 8244 Acked Flush 0 Sent Done Done false
27393 STATES DS:0 DS:1 DS:2 TOTAL
27394 Sent 301 0 0 301
27395 Done 0 301 301 602
27396 Skipped 0 0 0 0
27397 Error 0 0 0 0
27398 Last Flush: 7943 8244 8244
27399 Downstairs last five completed: 7943 7942 7941 7940 7939
27400 Upstairs last five completed: 8244 8243 8242 8241 8240
27401 CLIENT: Up:0 ds:301 act:2
274022024-11-19T17:13:43.420ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274032024-11-19T17:13:43.420ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62260) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274042024-11-19T17:13:43.422ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274052024-11-19T17:13:43.422ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274062024-11-19T17:13:43.422ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274072024-11-19T17:13:43.422ZINFOcrucible: send last flush ID to this DS: 7943 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274082024-11-19T17:13:43.422ZINFOcrucible: Replied this last flush ID: 7943 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274092024-11-19T17:13:43.422ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274102024-11-19T17:13:43.422ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274112024-11-19T17:13:43.422ZINFOcrucible: [0] client re-new 301 jobs since flush 7943 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
274122024-11-19T17:13:43.423ZINFOcrucible: [0] Replayed 301 jobs since flush: 7943 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
27413 ----------------------------------------------------------------
27414 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27415 Downstairs last five completed: 8244 8243 8242 8241 8240
27416 Upstairs last five completed: 8244 8243 8242 8241 8240
27417 CLIENT: Up:0 ds:0 act:3
27418 Replay: All jobs finished
27419 Main task has work to do, go find it
27420 got dsc Stop(0)
27421 stop 0
27422 Nov 19 17:13:45.658[8810] Got stop action so:false kr:false
27423 INFO request completed, latency_us: 347, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 4c4595d6-3ed0-4e20-8fe4-56ea498c971e, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
27424 Nov 19 17:13:45.658 INFO request completed, latency_us: 117, response_code: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:13:45.658757533Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
27425 200, uri: /state/cid/0, method: GET, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:13:45.658809998Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
27426 req_id: cfb43960-913b-4c2a-8634-5a4fd0c2ed26, remote_addr: 127.0.0{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:13:45.658848355Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
27427 .1:64175{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:13:45.658879172Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
27428 , local_addr: 127.0.0.1:9998
274292024-11-19T17:13:45.658ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
274302024-11-19T17:13:45.658ZINFOcrucible: client task is exiting = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
27431 [8810] Exited with: ExitStatus(unix_wait_status(9))
27432 [8810][0] reports Exit
27433 Nov 19 17:13:49.660 INFO request completed, latency_us: 313, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2fc070de-3b02-4811-a7f7-62dc14129851, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
27434 001/300 Read block 4671 len 4096
27435 002/300 Read block 1048 len 12288
27436 003/300 Read block 4734 len 8192
27437 004/300 Write block 763 len 28672 data: 5 5 5 4 4 4 2
27438 005/300 Read block 1849 len 16384
27439 006/300 Read block 1325 len 40960
27440 007/300 Read block 4819 len 4096
27441 008/300 Write block 4618 len 28672 data: 4 4 6 6 7 7 6
27442 009/300 Write block 2531 len 16384 data: 6 6 5 6
27443 010/300 Write block 2084 len 40960 data: 5 5 4 4 4 5 6 7 5 5
27444 011/300 Read block 394 len 36864
27445 012/300 Write block 2501 len 20480 data: 5 5 6 7 7
27446 013/300 Read block 3804 len 36864
27447 014/300 Write block 282 len 32768 data: 4 3 3 2 4 4 4 4
27448 015/300 Flush
27449 016/300 Read block 2817 len 32768
27450 017/300 Read block 4905 len 24576
27451 018/300 Write block 908 len 12288 data: 4 4 3
27452 019/300 Write block 530 len 20480 data: 4 5 4 4 4
27453 020/300 Read block 3801 len 28672
27454 021/300 Read block 1522 len 24576
27455 022/300 Read block 6 len 24576
27456 023/300 Write block 3713 len 40960 data: 6 7 5 4 3 3 3 3 3 3
27457 024/300 Read block 1727 len 4096
27458 025/300 Read block 517 len 24576
27459 026/300 Read block 4405 len 8192
27460 027/300 Read block 1636 len 8192
27461 028/300 Read block 549 len 4096
27462 029/300 Write block 3222 len 8192 data: 8 10
27463 030/300 Read block 69 len 36864
27464 031/300 Read block 1853 len 8192
27465 032/300 Read block 3080 len 20480
27466 033/300 Write block 1158 len 36864 data: 6 6 7 6 7 7 7 8 8
27467 034/300 Write block 2100 len 20480 data: 5 5 5 5 4
27468 035/300 Read block 2223 len 40960
27469 036/300 Read block 205 len 4096
27470 037/300 Write block 789 len 28672 data: 6 7 7 8 9 8 6
27471 038/300 Write block 4058 len 8192 data: 3 3
27472 039/300 Read block 364 len 4096
27473 040/300 Write block 4588 len 16384 data: 6 6 6 9
27474 041/300 Write block 3538 len 12288 data: 4 4 4
27475 042/300 Read block 3293 len 8192
27476 043/300 Write block 2259 len 8192 data: 4 3
27477 044/300 Read block 643 len 24576
27478 045/300 Write block 858 len 4096 data: 6
27479 046/300 Read block 1759 len 40960
27480 047/300 Write block 1732 len 16384 data: 6 5 5 5
27481 048/300 Write block 4576 len 36864 data: 3 3 4 3 3 4 4 5 5
27482 049/300 Write block 4195 len 24576 data: 9 9 8 5 5 4
27483 050/300 Write block 956 len 8192 data: 5 4
27484 051/300 Write block 1847 len 20480 data: 4 5 5 5 4
27485 052/300 Flush
27486 053/300 Flush
27487 054/300 Write block 3430 len 40960 data: 4 5 6 7 6 7 8 10 9 10
27488 055/300 Write block 4962 len 8192 data: 5 3
27489 056/300 Read block 589 len 16384
27490 057/300 Write block 4257 len 16384 data: 5 5 5 4
27491 058/300 Write block 4879 len 24576 data: 2 2 2 3 2 3
27492 059/300 Read block 3678 len 8192
27493 060/300 Read block 4928 len 32768
27494 061/300 Read block 272 len 24576
27495 062/300 Write block 3072 len 24576 data: 5 4 4 4 3 4
27496 063/300 Write block 3864 len 40960 data: 6 6 6 8 8 8 7 6 6 6
27497 064/300 Read block 186 len 32768
27498 065/300 Write block 4690 len 32768 data: 6 5 6 6 6 5 4 4
27499 066/300 Write block 2206 len 40960 data: 5 4 6 7 7 8 8 10 7 7
27500 067/300 Write block 4406 len 24576 data: 3 4 4 5 6 6
27501 068/300 Read block 4110 len 32768
27502 069/300 Write block 894 len 20480 data: 5 5 4 4 4
27503 070/300 Write block 4484 len 8192 data: 6 5
27504 071/300 Write block 2595 len 20480 data: 4 4 4 4 4
27505 072/300 Read block 52 len 12288
27506 073/300 Write block 1528 len 16384 data: 6 6 5 5
27507 074/300 Write block 233 len 12288 data: 5 5 4
27508 075/300 Write block 4813 len 12288 data: 3 3 3
27509 076/300 Write block 1787 len 12288 data: 4 4 4
27510 077/300 Write block 2093 len 4096 data: 6
27511 078/300 Read block 4643 len 24576
27512 079/300 Read block 4087 len 24576
27513 080/300 Read block 4028 len 16384
27514 081/300 Flush
27515 082/300 Write block 1689 len 12288 data: 6 5 6
27516 083/300 Read block 1001 len 24576
27517 084/300 Read block 4762 len 36864
27518 085/300 Write block 1930 len 40960 data: 6 7 8 8 6 6 6 7 6 6
27519 086/300 Read block 2816 len 36864
27520 087/300 Read block 3954 len 12288
27521 088/300 Read block 3361 len 36864
27522 089/300 Read block 760 len 28672
27523 090/300 Write block 3570 len 28672 data: 4 5 4 3 4 4 4
27524 091/300 Flush
27525 092/300 Read block 1592 len 20480
27526 093/300 Read block 2680 len 24576
27527 094/300 Read block 2047 len 32768
27528 095/300 Read block 2359 len 36864
27529 096/300 Flush
27530 097/300 Read block 3036 len 4096
27531 098/300 Read block 12 len 40960
27532 099/300 Read block 1428 len 28672
27533 100/300 Write block 2900 len 36864 data: 5 4 4 3 4 3 3 3 3
27534 101/300 Write block 1337 len 32768 data: 5 5 6 7 7 5 5 4
27535 102/300 Read block 2785 len 16384
27536 103/300 Read block 1441 len 12288
27537 104/300 Write block 1964 len 40960 data: 4 5 5 6 6 6 5 5 4 5
27538 105/300 Write block 1652 len 8192 data: 5 4
27539 106/300 Write block 2160 len 40960 data: 4 5 4 4 4 4 5 8 7 7
27540 107/300 Read block 1238 len 20480
27541 108/300 Write block 4302 len 16384 data: 2 2 3 3
27542 109/300 Write block 1094 len 16384 data: 3 3 4 3
27543 110/300 Read block 2082 len 12288
27544 111/300 Read block 3912 len 16384
27545 112/300 Read block 832 len 40960
27546 113/300 Read block 1265 len 28672
27547 114/300 Write block 3912 len 36864 data: 5 5 6 9 9 7 6 4 3
27548 115/300 Flush
27549 116/300 Read block 3782 len 8192
27550 117/300 Read block 2248 len 36864
27551 118/300 Write block 779 len 28672 data: 7 6 7 7 8 9 8
27552 119/300 Write block 4522 len 32768 data: 2 3 3 3 5 6 8 9
27553 120/300 Read block 1564 len 28672
27554 121/300 Read block 2824 len 20480
27555 122/300 Read block 2610 len 28672
27556 123/300 Read block 1218 len 40960
27557 124/300 Write block 1224 len 8192 data: 3 4
27558 125/300 Read block 2453 len 16384
27559 126/300 Flush
27560 127/300 Write block 1991 len 24576 data: 5 5 5 4 4 6
27561 128/300 Flush
27562 129/300 Read block 223 len 32768
27563 130/300 Flush
27564 131/300 Read block 3470 len 16384
27565 132/300 Write block 4306 len 8192 data: 4 4
27566 133/300 Write block 1579 len 20480 data: 5 6 6 7 6
27567 134/300 Write block 1960 len 28672 data: 5 4 4 4 5 6 6
27568 135/300 Read block 1218 len 12288
27569 136/300 Read block 4202 len 40960
27570 137/300 Write block 2561 len 4096 data: 7
27571 138/300 Write block 2470 len 12288 data: 5 5 5
27572 139/300 Read block 2433 len 12288
27573 140/300 Read block 2251 len 24576
27574 141/300 Read block 4118 len 20480
27575 142/300 Read block 2521 len 24576
27576 143/300 Read block 3275 len 28672
27577 144/300 Write block 519 len 12288 data: 4 4 4
27578 145/300 Read block 3525 len 16384
27579 146/300 Read block 4847 len 16384
27580 147/300 Read block 1590 len 36864
27581 148/300 Write block 4496 len 8192 data: 9 9
27582 149/300 Read block 2613 len 8192
27583 150/300 Write block 3225 len 12288 data: 7 7 6
27584 151/300 Write block 3480 len 28672 data: 6 8 11 9 10 10 10
27585 152/300 Read block 3781 len 32768
27586 153/300 Read block 2981 len 32768
27587 154/300 Write block 385 len 12288 data: 6 5 5
27588 155/300 Flush
27589 156/300 Read block 1535 len 24576
27590 157/300 Read block 2265 len 40960
27591 158/300 Flush
27592 159/300 Write block 4686 len 36864 data: 5 5 5 7 7 6 7 7 7
27593 160/300 Flush
27594 161/300 Write block 2557 len 32768 data: 4 4 5 8 8 7 8 8
27595 162/300 Read block 546 len 12288
27596 163/300 Read block 4242 len 16384
27597 164/300 Read block 3128 len 12288
27598 165/300 Read block 4657 len 24576
27599 166/300 Write block 3551 len 32768 data: 5 5 4 4 3 5 5 4
27600 167/300 Write block 811 len 12288 data: 3 3 4
27601 168/300 Read block 4685 len 16384
27602 169/300 Read block 2436 len 32768
27603 170/300 Write block 3766 len 8192 data: 5 5
27604 171/300 Write block 2337 len 28672 data: 3 4 6 7 6 4 5
27605 172/300 Write block 2808 len 24576 data: 6 7 5 5 6 7
27606 173/300 Write block 3705 len 36864 data: 5 6 6 8 7 7 7 7 7
27607 174/300 Write block 4965 len 28672 data: 3 3 3 3 3 4 4
27608 175/300 Write block 349 len 36864 data: 4 4 4 5 5 4 5 5 5
27609 176/300 Write block 3501 len 24576 data: 3 4 6 5 4 3
27610 177/300 Write block 3049 len 4096 data: 6
27611 178/300 Read block 696 len 36864
27612 179/300 Read block 680 len 4096
27613 180/300 Read block 1953 len 32768
27614 181/300 Flush
27615 182/300 Write block 494 len 28672 data: 6 6 8 7 6 7 6
27616 183/300 Write block 3180 len 12288 data: 3 4 4
27617 184/300 Write block 844 len 32768 data: 4 4 5 5 5 5 5 3
27618 185/300 Read block 2606 len 40960
27619 186/300 Read block 2009 len 36864
27620 187/300 Write block 2584 len 8192 data: 5 4
27621 188/300 Write block 50 len 16384 data: 3 3 5 5
27622 189/300 Flush
27623 190/300 Write block 3300 len 8192 data: 4 2
27624 191/300 Write block 2678 len 32768 data: 7 7 7 6 8 8 8 8
27625 192/300 Read block 1697 len 4096
27626 193/300 Read block 1530 len 40960
27627 194/300 Read block 1690 len 40960
27628 195/300 Flush
27629 196/300 Write block 690 len 36864 data: 5 3 3 3 3 3 3 4 5
27630 197/300 Flush
27631 198/300 Write block 2044 len 12288 data: 3 3 4
27632 199/300 Write block 3956 len 8192 data: 6 6
27633 200/300 Read block 2211 len 12288
27634 201/300 Read block 1669 len 12288
27635 202/300 Read block 3934 len 16384
27636 203/300 Read block 3677 len 32768
27637 204/300 Read block 4765 len 28672
27638 205/300 Read block 896 len 16384
27639 206/300 Write block 4067 len 40960 data: 5 5 6 4 4 5 5 5 5 4
27640 207/300 Read block 1070 len 8192
27641 208/300 Flush
27642 209/300 Flush
27643 210/300 Read block 3938 len 36864
27644 211/300 Read block 952 len 32768
27645 212/300 Write block 1660 len 28672 data: 5 3 2 2 2 2 2
27646 213/300 Read block 4599 len 16384
27647 214/300 Write block 3865 len 12288 data: 7 7 9
27648 215/300 Write block 597 len 4096 data: 5
27649 216/300 Write block 3801 len 12288 data: 7 6 5
27650 217/300 Write block 2436 len 8192 data: 5 5
27651 218/300 Write block 2870 len 4096 data: 3
27652 219/300 Flush
27653 220/300 Flush
27654 221/300 Read block 1328 len 4096
27655 222/300 Write block 2317 len 28672 data: 5 4 4 4 4 4 3
27656 223/300 Write block 4644 len 12288 data: 4 4 4
27657 224/300 Write block 458 len 16384 data: 7 6 6 6
27658 225/300 Write block 4300 len 24576 data: 4 3 3 3 4 4
27659 226/300 Write block 906 len 16384 data: 7 7 5 5
27660 227/300 Flush
27661 228/300 Read block 3903 len 8192
27662 229/300 Read block 4617 len 20480
27663 230/300 Read block 2892 len 8192
27664 231/300 Read block 557 len 24576
27665 232/300 Read block 3157 len 24576
27666 233/300 Flush
27667 234/300 Read block 4476 len 20480
27668 235/300 Write block 1160 len 12288 data: 8 7 8
27669 236/300 Flush
27670 237/300 Write block 3448 len 20480 data: 6 8 8 8 6
27671 238/300 Read block 3478 len 36864
27672 239/300 Read block 3658 len 40960
27673 240/300 Write block 4659 len 36864 data: 4 5 6 5 5 4 4 3 3
27674 241/300 Read block 3360 len 28672
27675 242/300 Write block 3520 len 4096 data: 3
27676 243/300 Write block 4381 len 28672 data: 5 7 7 6 6 7 7
27677 244/300 Read block 3602 len 24576
27678 245/300 Read block 1745 len 32768
27679 246/300 Flush
27680 247/300 Write block 259 len 12288 data: 5 4 4
27681 248/300 Read block 1019 len 32768
27682 249/300 Read block 489 len 20480
27683 250/300 Write block 1696 len 8192 data: 7 7
27684 251/300 Read block 3752 len 16384
27685 252/300 Read block 1684 len 40960
27686 253/300 Write block 3013 len 24576 data: 4 5 5 5 4 4
27687 254/300 Read block 1189 len 4096
27688 255/300 Read block 3505 len 28672
27689 256/300 Read block 2505 len 12288
27690 257/300 Write block 470 len 16384 data: 5 8 8 9
27691 258/300 Write block 4783 len 12288 data: 3 3 4
27692 259/300 Write block 4077 len 28672 data: 2 2 4 6 6 6 6
27693 260/300 Write block 815 len 32768 data: 6 5 5 6 6 7 5 6
27694 261/300 Read block 1331 len 20480
27695 262/300 Write block 870 len 4096 data: 6
27696 263/300 Flush
27697 264/300 Write block 1342 len 28672 data: 6 6 5 5 4 4 5
27698 265/300 Write block 303 len 12288 data: 5 5 5
27699 266/300 Read block 30 len 28672
27700 267/300 Read block 2395 len 24576
27701 268/300 Read block 3623 len 8192
27702 269/300 Flush
27703 270/300 Read block 714 len 4096
27704 271/300 Read block 2031 len 12288
27705 272/300 Read block 4296 len 24576
27706 273/300 Read block 983 len 36864
27707 274/300 Write block 1584 len 4096 data: 8
27708 275/300 Write block 1046 len 24576 data: 9 8 8 6 6 6
27709 276/300 Write block 354 len 8192 data: 5 6
27710 277/300 Read block 1723 len 20480
27711 278/300 Read block 1689 len 8192
27712 279/300 Read block 3 len 36864
27713 280/300 Read block 3151 len 20480
27714 281/300 Write block 3366 len 24576 data: 6 6 5 5 6 6
27715 282/300 Read block 635 len 24576
27716 283/300 Write block 623 len 36864 data: 5 5 5 5 6 6 6 5 3
27717 284/300 Write block 955 len 20480 data: 4 6 5 3 3
27718 285/300 Read block 4776 len 12288
27719 286/300 Read block 1510 len 20480
27720 287/300 Write block 2019 len 40960 data: 4 4 5 7 8 7 8 9 8 7
27721 288/300 Read block 1241 len 40960
27722 289/300 Write block 4490 len 8192 data: 6 8
27723 290/300 Write block 696 len 24576 data: 4 5 6 4 4 6
27724 291/300 Read block 2220 len 12288
27725 292/300 Flush
27726 293/300 Write block 1116 len 20480 data: 7 6 6 5 6
27727 294/300 Write block 1280 len 28672 data: 5 6 6 6 5 8 8
27728 295/300 Read block 1907 len 4096
27729 296/300 Read block 1172 len 28672
27730 297/300 Read block 137 len 28672
27731 298/300 Write block 2790 len 24576 data: 6 6 6 5 3 5
27732 299/300 Read block 2138 len 36864
27733 300/300 Read block 4448 len 4096
27734 301/300 Read block 3588 len 8192
27735 Main task has work to do, go find it
27736 got dsc Start(0)
27737 start 0
27738 Nov 19 17:13:49.892 INFO request completed, latency_us: 192, response_code: 204, uri: /start/cid/0, method: POST, req_id: 38c471e1-a30e-426d-ba70-a5f216628544, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
27739 Replay: started 0, returned:Ok(())
27740 ----------------------------------------------------------------
27741 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27742 States: Offline Active Active
27743 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27744 8245 Acked Read 1 Sent Done Done false
27745 8246 Acked Read 3 Sent Done Done false
27746 8247 Acked Read 2 Sent Done Done false
27747 8248 Acked Write 7 Sent Done Done false
27748 8249 Acked Read 4 Sent Done Done false
27749 8250 Acked Read 10 Sent Done Done false
27750 8251 Acked Read 1 Sent Done Done false
27751 8252 Acked Write 7 Sent Done Done false
27752 8253 Acked Write 4 Sent Done Done false
27753 8254 Acked Write 10 Sent Done Done false
27754 8255 Acked Read 9 Sent Done Done false
27755 8256 Acked Write 5 Sent Done Done false
27756 8257 Acked Read 9 Sent Done Done false
27757 8258 Acked Write 8 Sent Done Done false
27758 8259 Acked Flush 0 Sent Done Done false
27759 8260 Acked Read 8 Sent Done Done false
27760 8261 Acked Read 6 Sent Done Done false
27761 8262 Acked Write 3 Sent Done Done false
27762 8263 Acked Write 5 Sent Done Done false
27763 8264 Acked Read 7 Sent Done Done false
27764 8265 Acked Read 6 Sent Done Done false
27765 8266 Acked Read 6 Sent Done Done false
27766 8267 Acked Write 10 Sent Done Done false
27767 8268 Acked Read 1 Sent Done Done false
27768 8269 Acked Read 6 Sent Done Done false
27769 8270 Acked Read 2 Sent Done Done false
27770 8271 Acked Read 2 Sent Done Done false
27771 8272 Acked Read 1 Sent Done Done false
27772 8273 Acked Write 2 Sent Done Done false
27773 8274 Acked Read 9 Sent Done Done false
27774 8275 Acked Read 2 Sent Done Done false
27775 8276 Acked Read 5 Sent Done Done false
27776 8277 Acked Write 9 Sent Done Done false
27777 8278 Acked Write 5 Sent Done Done false
27778 8279 Acked Read 10 Sent Done Done false
27779 8280 Acked Read 1 Sent Done Done false
27780 8281 Acked Write 7 Sent Done Done false
27781 8282 Acked Write 2 Sent Done Done false
27782 8283 Acked Read 1 Sent Done Done false
27783 8284 Acked Write 4 Sent Done Done false
27784 8285 Acked Write 3 Sent Done Done false
27785 8286 Acked Read 2 Sent Done Done false
27786 8287 Acked Write 2 Sent Done Done false
27787 8288 Acked Read 6 Sent Done Done false
27788 8289 Acked Write 1 Sent Done Done false
27789 8290 Acked Read 10 Sent Done Done false
27790 8291 Acked Write 4 Sent Done Done false
27791 8292 Acked Write 9 Sent Done Done false
27792 8293 Acked Write 6 Sent Done Done false
27793 8294 Acked Write 2 Sent Done Done false
27794 8295 Acked Write 5 Sent Done Done false
27795 8296 Acked Flush 0 Sent Done Done false
27796 8297 Acked Flush 0 Sent Done Done false
27797 8298 Acked Write 10 Sent Done Done false
27798 8299 Acked Write 2 Sent Done Done false
27799 8300 Acked Read 4 Sent Done Done false
27800 8301 Acked Write 4 Sent Done Done false
27801 8302 Acked Write 6 Sent Done Done false
27802 8303 Acked Read 2 Sent Done Done false
27803 8304 Acked Read 8 Sent Done Done false
27804 8305 Acked Read 6 Sent Done Done false
27805 8306 Acked Write 6 Sent Done Done false
27806 8307 Acked Write 10 Sent Done Done false
27807 8308 Acked Read 8 Sent Done Done false
27808 8309 Acked Write 8 Sent Done Done false
27809 8310 Acked Write 10 Sent Done Done false
27810 8311 Acked Write 6 Sent Done Done false
27811 8312 Acked Read 8 Sent Done Done false
27812 8313 Acked Write 5 Sent Done Done false
27813 8314 Acked Write 2 Sent Done Done false
27814 8315 Acked Write 5 Sent Done Done false
27815 8316 Acked Read 3 Sent Done Done false
27816 8317 Acked Write 4 Sent Done Done false
27817 8318 Acked Write 3 Sent Done Done false
27818 8319 Acked Write 3 Sent Done Done false
27819 8320 Acked Write 3 Sent Done Done false
27820 8321 Acked Write 1 Sent Done Done false
27821 8322 Acked Read 6 Sent Done Done false
27822 8323 Acked Read 6 Sent Done Done false
27823 8324 Acked Read 4 Sent Done Done false
27824 8325 Acked Flush 0 Sent Done Done false
27825 8326 Acked Write 3 Sent Done Done false
27826 8327 Acked Read 6 Sent Done Done false
27827 8328 Acked Read 9 Sent Done Done false
27828 8329 Acked Write 10 Sent Done Done false
27829 8330 Acked Read 9 Sent Done Done false
27830 8331 Acked Read 3 Sent Done Done false
27831 8332 Acked Read 9 Sent Done Done false
27832 8333 Acked Read 7 Sent Done Done false
27833 8334 Acked Write 7 Sent Done Done false
27834 8335 Acked Flush 0 Sent Done Done false
27835 8336 Acked Read 5 Sent Done Done false
27836 8337 Acked Read 6 Sent Done Done false
27837 8338 Acked Read 8 Sent Done Done false
27838 8339 Acked Read 9 Sent Done Done false
27839 8340 Acked Flush 0 Sent Done Done false
27840 8341 Acked Read 1 Sent Done Done false
27841 8342 Acked Read 10 Sent Done Done false
27842 8343 Acked Read 7 Sent Done Done false
27843 8344 Acked Write 9 Sent Done Done false
27844 8345 Acked Write 8 Sent Done Done false
27845 8346 Acked Read 4 Sent Done Done false
27846 8347 Acked Read 3 Sent Done Done false
27847 8348 Acked Write 10 Sent Done Done false
27848 8349 Acked Write 2 Sent Done Done false
27849 8350 Acked Write 10 Sent Done Done false
27850 8351 Acked Read 5 Sent Done Done false
27851 8352 Acked Write 4 Sent Done Done false
27852 8353 Acked Write 4 Sent Done Done false
27853 8354 Acked Read 3 Sent Done Done false
27854 8355 Acked Read 4 Sent Done Done false
27855 8356 Acked Read 10 Sent Done Done false
27856 8357 Acked Read 7 Sent Done Done false
27857 8358 Acked Write 9 Sent Done Done false
27858 8359 Acked Flush 0 Sent Done Done false
27859 8360 Acked Read 2 Sent Done Done false
27860 8361 Acked Read 9 Sent Done Done false
27861 8362 Acked Write 7 Sent Done Done false
27862 8363 Acked Write 8 Sent Done Done false
27863 8364 Acked Read 7 Sent Done Done false
27864 8365 Acked Read 5 Sent Done Done false
27865 8366 Acked Read 7 Sent Done Done false
27866 8367 Acked Read 10 Sent Done Done false
27867 8368 Acked Write 2 Sent Done Done false
27868 8369 Acked Read 4 Sent Done Done false
27869 8370 Acked Flush 0 Sent Done Done false
27870 8371 Acked Write 6 Sent Done Done false
27871 8372 Acked Flush 0 Sent Done Done false
27872 8373 Acked Read 8 Sent Done Done false
27873 8374 Acked Flush 0 Sent Done Done false
27874 8375 Acked Read 4 Sent Done Done false
27875 8376 Acked Write 2 Sent Done Done false
27876 8377 Acked Write 5 Sent Done Done false
27877 8378 Acked Write 7 Sent Done Done false
27878 8379 Acked Read 3 Sent Done Done false
27879 8380 Acked Read 10 Sent Done Done false
27880 8381 Acked Write 1 Sent Done Done false
27881 8382 Acked Write 3 Sent Done Done false
27882 8383 Acked Read 3 Sent Done Done false
27883 8384 Acked Read 6 Sent Done Done false
27884 8385 Acked Read 5 Sent Done Done false
27885 8386 Acked Read 6 Sent Done Done false
27886 8387 Acked Read 7 Sent Done Done false
27887 8388 Acked Write 3 Sent Done Done false
27888 8389 Acked Read 4 Sent Done Done false
27889 8390 Acked Read 4 Sent Done Done false
27890 8391 Acked Read 9 Sent Done Done false
27891 8392 Acked Write 2 Sent Done Done false
27892 8393 Acked Read 2 Sent Done Done false
27893 8394 Acked Write 3 Sent Done Done false
27894 8395 Acked Write 7 Sent Done Done false
27895 8396 Acked Read 8 Sent Done Done false
27896 8397 Acked Read 8 Sent Done Done false
27897 8398 Acked Write 3 Sent Done Done false
27898 8399 Acked Flush 0 Sent Done Done false
27899 8400 Acked Read 6 Sent Done Done false
27900 8401 Acked Read 10 Sent Done Done false
27901 8402 Acked Flush 0 Sent Done Done false
27902 8403 Acked Write 9 Sent Done Done false
27903 8404 Acked Flush 0 Sent Done Done false
27904 8405 Acked Write 8 Sent Done Done false
27905 8406 Acked Read 3 Sent Done Done false
27906 8407 Acked Read 4 Sent Done Done false
27907 8408 Acked Read 3 Sent Done Done false
27908 8409 Acked Read 6 Sent Done Done false
27909 8410 Acked Write 8 Sent Done Done false
27910 8411 Acked Write 3 Sent Done Done false
27911 8412 Acked Read 4 Sent Done Done false
27912 8413 Acked Read 8 Sent Done Done false
27913 8414 Acked Write 2 Sent Done Done false
27914 8415 Acked Write 7 Sent Done Done false
27915 8416 Acked Write 6 Sent Done Done false
27916 8417 Acked Write 9 Sent Done Done false
27917 8418 Acked Write 7 Sent Done Done false
27918 8419 Acked Write 9 Sent Done Done false
27919 8420 Acked Write 6 Sent Done Done false
27920 8421 Acked Write 1 Sent Done Done false
27921 8422 Acked Read 9 Sent Done Done false
27922 8423 Acked Read 1 Sent Done Done false
27923 8424 Acked Read 8 Sent Done Done false
27924 8425 Acked Flush 0 Sent Done Done false
27925 8426 Acked Write 7 Sent Done Done false
27926 8427 Acked Write 3 Sent Done Done false
27927 8428 Acked Write 8 Sent Done Done false
27928 8429 Acked Read 10 Sent Done Done false
27929 8430 Acked Read 9 Sent Done Done false
27930 8431 Acked Write 2 Sent Done Done false
27931 8432 Acked Write 4 Sent Done Done false
27932 8433 Acked Flush 0 Sent Done Done false
27933 8434 Acked Write 2 Sent Done Done false
27934 8435 Acked Write 8 Sent Done Done false
27935 8436 Acked Read 1 Sent Done Done false
27936 8437 Acked Read 10 Sent Done Done false
27937 8438 Acked Read 10 Sent Done Done false
27938 8439 Acked Flush 0 Sent Done Done false
27939 8440 Acked Write 9 Sent Done Done false
27940 8441 Acked Flush 0 Sent Done Done false
27941 8442 Acked Write 3 Sent Done Done false
27942 8443 Acked Write 2 Sent Done Done false
27943 8444 Acked Read 3 Sent Done Done false
27944 8445 Acked Read 3 Sent Done Done false
27945 8446 Acked Read 4 Sent Done Done false
27946 8447 Acked Read 8 Sent Done Done false
27947 8448 Acked Read 7 Sent Done Done false
27948 8449 Acked Read 4 Sent Done Done false
27949 8450 Acked Write 10 Sent Done Done false
27950 8451 Acked Read 2 Sent Done Done false
27951 8452 Acked Flush 0 Sent Done Done false
27952 8453 Acked Flush 0 Sent Done Done false
27953 8454 Acked Read 9 Sent Done Done false
27954 8455 Acked Read 8 Sent Done Done false
27955 8456 Acked Write 7 Sent Done Done false
27956 8457 Acked Read 4 Sent Done Done false
27957 8458 Acked Write 3 Sent Done Done false
27958 8459 Acked Write 1 Sent Done Done false
27959 8460 Acked Write 3 Sent Done Done false
27960 8461 Acked Write 2 Sent Done Done false
27961 8462 Acked Write 1 Sent Done Done false
27962 8463 Acked Flush 0 Sent Done Done false
27963 8464 Acked Flush 0 Sent Done Done false
27964 8465 Acked Read 1 Sent Done Done false
27965 8466 Acked Write 7 Sent Done Done false
27966 8467 Acked Write 3 Sent Done Done false
27967 8468 Acked Write 4 Sent Done Done false
27968 8469 Acked Write 6 Sent Done Done false
27969 8470 Acked Write 4 Sent Done Done false
27970 8471 Acked Flush 0 Sent Done Done false
27971 8472 Acked Read 2 Sent Done Done false
27972 8473 Acked Read 5 Sent Done Done false
27973 8474 Acked Read 2 Sent Done Done false
27974 8475 Acked Read 6 Sent Done Done false
27975 8476 Acked Read 6 Sent Done Done false
27976 8477 Acked Flush 0 Sent Done Done false
27977 8478 Acked Read 5 Sent Done Done false
27978 8479 Acked Write 3 Sent Done Done false
27979 8480 Acked Flush 0 Sent Done Done false
27980 8481 Acked Write 5 Sent Done Done false
27981 8482 Acked Read 9 Sent Done Done false
27982 8483 Acked Read 10 Sent Done Done false
27983 8484 Acked Write 9 Sent Done Done false
27984 8485 Acked Read 7 Sent Done Done false
27985 8486 Acked Write 1 Sent Done Done false
27986 8487 Acked Write 7 Sent Done Done false
27987 8488 Acked Read 6 Sent Done Done false
27988 8489 Acked Read 8 Sent Done Done false
27989 8490 Acked Flush 0 Sent Done Done false
27990 8491 Acked Write 3 Sent Done Done false
27991 8492 Acked Read 8 Sent Done Done false
27992 8493 Acked Read 5 Sent Done Done false
27993 8494 Acked Write 2 Sent Done Done false
27994 8495 Acked Read 4 Sent Done Done false
27995 8496 Acked Read 10 Sent Done Done false
27996 8497 Acked Write 6 Sent Done Done false
27997 8498 Acked Read 1 Sent Done Done false
27998 8499 Acked Read 7 Sent Done Done false
27999 8500 Acked Read 3 Sent Done Done false
28000 8501 Acked Write 4 Sent Done Done false
28001 8502 Acked Write 3 Sent Done Done false
28002 8503 Acked Write 7 Sent Done Done false
28003 8504 Acked Write 8 Sent Done Done false
28004 8505 Acked Read 5 Sent Done Done false
28005 8506 Acked Write 1 Sent Done Done false
28006 8507 Acked Flush 0 Sent Done Done false
28007 8508 Acked Write 7 Sent Done Done false
28008 8509 Acked Write 3 Sent Done Done false
28009 8510 Acked Read 7 Sent Done Done false
28010 8511 Acked Read 6 Sent Done Done false
28011 8512 Acked Read 2 Sent Done Done false
28012 8513 Acked Flush 0 Sent Done Done false
28013 8514 Acked Read 1 Sent Done Done false
28014 8515 Acked Read 3 Sent Done Done false
28015 8516 Acked Read 6 Sent Done Done false
28016 8517 Acked Read 9 Sent Done Done false
28017 8518 Acked Write 1 Sent Done Done false
28018 8519 Acked Write 6 Sent Done Done false
28019 8520 Acked Write 2 Sent Done Done false
28020 8521 Acked Read 5 Sent Done Done false
28021 8522 Acked Read 2 Sent Done Done false
28022 8523 Acked Read 9 Sent Done Done false
28023 8524 Acked Read 5 Sent Done Done false
28024 8525 Acked Write 6 Sent Done Done false
28025 8526 Acked Read 6 Sent Done Done false
28026 8527 Acked Write 9 Sent Done Done false
28027 8528 Acked Write 5 Sent Done Done false
28028 8529 Acked Read 3 Sent Done Done false
28029 8530 Acked Read 5 Sent Done Done false
28030 8531 Acked Write 10 Sent Done Done false
28031 8532 Acked Read 10 Sent Done Done false
28032 8533 Acked Write 2 Sent Done Done false
28033 8534 Acked Write 6 Sent Done Done false
28034 8535 Acked Read 3 Sent Done Done false
28035 8536 Acked Flush 0 Sent Done Done false
28036 8537 Acked Write 5 Sent Done Done false
28037 8538 Acked Write 7 Sent Done Done false
28038 8539 Acked Read 1 Sent Done Done false
28039 8540 Acked Read 7 Sent Done Done false
28040 8541 Acked Read 7 Sent Done Done false
28041 8542 Acked Write 6 Sent Done Done false
28042 8543 Acked Read 9 Sent Done Done false
28043 8544 Acked Read 1 Sent Done Done false
28044 8545 Acked Read 2 Sent Done Done false
28045 STATES DS:0 DS:1 DS:2 TOTAL
28046 Sent 301 0 0 301
28047 Done 0 301 301 602
28048 Skipped 0 0 0 0
28049 Error 0 0 0 0
28050 Last Flush: 8244 8536 8536
28051 Downstairs last five completed: 8244 8243 8242 8241 8240
28052 Upstairs last five completed: 8545 8544 8543 8542 8541
28053 CLIENT: Up:0 ds:301 act:2
28054 [8810] got start action
28055 [8810] I am going to restart
28056 Starting downstairs at port 8810
28057 Make output file at "/tmp/dsc/downstairs-8810.txt"
28058 [8810][0] reports Starting
28059 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1227)
28060 [8810][0] reports Running
28061 ----------------------------------------------------------------
28062 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28063 States: Offline Active Active
28064 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28065 8245 Acked Read 1 Sent Done Done false
28066 8246 Acked Read 3 Sent Done Done false
28067 8247 Acked Read 2 Sent Done Done false
28068 8248 Acked Write 7 Sent Done Done false
28069 8249 Acked Read 4 Sent Done Done false
28070 8250 Acked Read 10 Sent Done Done false
28071 8251 Acked Read 1 Sent Done Done false
28072 8252 Acked Write 7 Sent Done Done false
28073 8253 Acked Write 4 Sent Done Done false
28074 8254 Acked Write 10 Sent Done Done false
28075 8255 Acked Read 9 Sent Done Done false
28076 8256 Acked Write 5 Sent Done Done false
28077 8257 Acked Read 9 Sent Done Done false
28078 8258 Acked Write 8 Sent Done Done false
28079 8259 Acked Flush 0 Sent Done Done false
28080 8260 Acked Read 8 Sent Done Done false
28081 8261 Acked Read 6 Sent Done Done false
28082 8262 Acked Write 3 Sent Done Done false
28083 8263 Acked Write 5 Sent Done Done false
28084 8264 Acked Read 7 Sent Done Done false
28085 8265 Acked Read 6 Sent Done Done false
28086 8266 Acked Read 6 Sent Done Done false
28087 8267 Acked Write 10 Sent Done Done false
28088 8268 Acked Read 1 Sent Done Done false
28089 8269 Acked Read 6 Sent Done Done false
28090 8270 Acked Read 2 Sent Done Done false
28091 8271 Acked Read 2 Sent Done Done false
28092 8272 Acked Read 1 Sent Done Done false
28093 8273 Acked Write 2 Sent Done Done false
28094 8274 Acked Read 9 Sent Done Done false
28095 8275 Acked Read 2 Sent Done Done false
28096 8276 Acked Read 5 Sent Done Done false
28097 8277 Acked Write 9 Sent Done Done false
28098 8278 Acked Write 5 Sent Done Done false
28099 8279 Acked Read 10 Sent Done Done false
28100 8280 Acked Read 1 Sent Done Done false
28101 8281 Acked Write 7 Sent Done Done false
28102 8282 Acked Write 2 Sent Done Done false
28103 8283 Acked Read 1 Sent Done Done false
28104 8284 Acked Write 4 Sent Done Done false
28105 8285 Acked Write 3 Sent Done Done false
28106 8286 Acked Read 2 Sent Done Done false
28107 8287 Acked Write 2 Sent Done Done false
28108 8288 Acked Read 6 Sent Done Done false
28109 8289 Acked Write 1 Sent Done Done false
28110 8290 Acked Read 10 Sent Done Done false
28111 8291 Acked Write 4 Sent Done Done false
28112 8292 Acked Write 9 Sent Done Done false
28113 8293 Acked Write 6 Sent Done Done false
28114 8294 Acked Write 2 Sent Done Done false
28115 8295 Acked Write 5 Sent Done Done false
28116 8296 Acked Flush 0 Sent Done Done false
28117 8297 Acked Flush 0 Sent Done Done false
28118 8298 Acked Write 10 Sent Done Done false
28119 8299 Acked Write 2 Sent Done Done false
28120 8300 Acked Read 4 Sent Done Done false
28121 8301 Acked Write 4 Sent Done Done false
28122 8302 Acked Write 6 Sent Done Done false
28123 8303 Acked Read 2 Sent Done Done false
28124 8304 Acked Read 8 Sent Done Done false
28125 8305 Acked Read 6 Sent Done Done false
28126 8306 Acked Write 6 Sent Done Done false
28127 8307 Acked Write 10 Sent Done Done false
28128 8308 Acked Read 8 Sent Done Done false
28129 8309 Acked Write 8 Sent Done Done false
28130 8310 Acked Write 10 Sent Done Done false
28131 8311 Acked Write 6 Sent Done Done false
28132 8312 Acked Read 8 Sent Done Done false
28133 8313 Acked Write 5 Sent Done Done false
28134 8314 Acked Write 2 Sent Done Done false
28135 8315 Acked Write 5 Sent Done Done false
28136 8316 Acked Read 3 Sent Done Done false
28137 8317 Acked Write 4 Sent Done Done false
28138 8318 Acked Write 3 Sent Done Done false
28139 8319 Acked Write 3 Sent Done Done false
28140 8320 Acked Write 3 Sent Done Done false
28141 8321 Acked Write 1 Sent Done Done false
28142 8322 Acked Read 6 Sent Done Done false
28143 8323 Acked Read 6 Sent Done Done false
28144 8324 Acked Read 4 Sent Done Done false
28145 8325 Acked Flush 0 Sent Done Done false
28146 8326 Acked Write 3 Sent Done Done false
28147 8327 Acked Read 6 Sent Done Done false
28148 8328 Acked Read 9 Sent Done Done false
28149 8329 Acked Write 10 Sent Done Done false
28150 8330 Acked Read 9 Sent Done Done false
28151 8331 Acked Read 3 Sent Done Done false
28152 8332 Acked Read 9 Sent Done Done false
28153 8333 Acked Read 7 Sent Done Done false
28154 8334 Acked Write 7 Sent Done Done false
28155 8335 Acked Flush 0 Sent Done Done false
28156 8336 Acked Read 5 Sent Done Done false
28157 8337 Acked Read 6 Sent Done Done false
28158 8338 Acked Read 8 Sent Done Done false
28159 8339 Acked Read 9 Sent Done Done false
28160 8340 Acked Flush 0 Sent Done Done false
28161 8341 Acked Read 1 Sent Done Done false
28162 8342 Acked Read 10 Sent Done Done false
28163 8343 Acked Read 7 Sent Done Done false
28164 8344 Acked Write 9 Sent Done Done false
28165 8345 Acked Write 8 Sent Done Done false
28166 8346 Acked Read 4 Sent Done Done false
28167 8347 Acked Read 3 Sent Done Done false
28168 8348 Acked Write 10 Sent Done Done false
28169 8349 Acked Write 2 Sent Done Done false
28170 8350 Acked Write 10 Sent Done Done false
28171 8351 Acked Read 5 Sent Done Done false
28172 8352 Acked Write 4 Sent Done Done false
28173 8353 Acked Write 4 Sent Done Done false
28174 8354 Acked Read 3 Sent Done Done false
28175 8355 Acked Read 4 Sent Done Done false
28176 8356 Acked Read 10 Sent Done Done false
28177 8357 Acked Read 7 Sent Done Done false
28178 8358 Acked Write 9 Sent Done Done false
28179 8359 Acked Flush 0 Sent Done Done false
28180 8360 Acked Read 2 Sent Done Done false
28181 8361 Acked Read 9 Sent Done Done false
28182 8362 Acked Write 7 Sent Done Done false
28183 8363 Acked Write 8 Sent Done Done false
28184 8364 Acked Read 7 Sent Done Done false
28185 8365 Acked Read 5 Sent Done Done false
28186 8366 Acked Read 7 Sent Done Done false
28187 8367 Acked Read 10 Sent Done Done false
28188 8368 Acked Write 2 Sent Done Done false
28189 8369 Acked Read 4 Sent Done Done false
28190 8370 Acked Flush 0 Sent Done Done false
28191 8371 Acked Write 6 Sent Done Done false
28192 8372 Acked Flush 0 Sent Done Done false
28193 8373 Acked Read 8 Sent Done Done false
28194 8374 Acked Flush 0 Sent Done Done false
28195 8375 Acked Read 4 Sent Done Done false
28196 8376 Acked Write 2 Sent Done Done false
28197 8377 Acked Write 5 Sent Done Done false
28198 8378 Acked Write 7 Sent Done Done false
28199 8379 Acked Read 3 Sent Done Done false
28200 8380 Acked Read 10 Sent Done Done false
28201 8381 Acked Write 1 Sent Done Done false
28202 8382 Acked Write 3 Sent Done Done false
28203 8383 Acked Read 3 Sent Done Done false
28204 8384 Acked Read 6 Sent Done Done false
28205 8385 Acked Read 5 Sent Done Done false
28206 8386 Acked Read 6 Sent Done Done false
28207 8387 Acked Read 7 Sent Done Done false
28208 8388 Acked Write 3 Sent Done Done false
28209 8389 Acked Read 4 Sent Done Done false
28210 8390 Acked Read 4 Sent Done Done false
28211 8391 Acked Read 9 Sent Done Done false
28212 8392 Acked Write 2 Sent Done Done false
28213 8393 Acked Read 2 Sent Done Done false
28214 8394 Acked Write 3 Sent Done Done false
28215 8395 Acked Write 7 Sent Done Done false
28216 8396 Acked Read 8 Sent Done Done false
28217 8397 Acked Read 8 Sent Done Done false
28218 8398 Acked Write 3 Sent Done Done false
28219 8399 Acked Flush 0 Sent Done Done false
28220 8400 Acked Read 6 Sent Done Done false
28221 8401 Acked Read 10 Sent Done Done false
28222 8402 Acked Flush 0 Sent Done Done false
28223 8403 Acked Write 9 Sent Done Done false
28224 8404 Acked Flush 0 Sent Done Done false
28225 8405 Acked Write 8 Sent Done Done false
28226 8406 Acked Read 3 Sent Done Done false
28227 8407 Acked Read 4 Sent Done Done false
28228 8408 Acked Read 3 Sent Done Done false
28229 8409 Acked Read 6 Sent Done Done false
28230 8410 Acked Write 8 Sent Done Done false
28231 8411 Acked Write 3 Sent Done Done false
28232 8412 Acked Read 4 Sent Done Done false
28233 8413 Acked Read 8 Sent Done Done false
28234 8414 Acked Write 2 Sent Done Done false
28235 8415 Acked Write 7 Sent Done Done false
28236 8416 Acked Write 6 Sent Done Done false
28237 8417 Acked Write 9 Sent Done Done false
28238 8418 Acked Write 7 Sent Done Done false
28239 8419 Acked Write 9 Sent Done Done false
28240 8420 Acked Write 6 Sent Done Done false
28241 8421 Acked Write 1 Sent Done Done false
28242 8422 Acked Read 9 Sent Done Done false
28243 8423 Acked Read 1 Sent Done Done false
28244 8424 Acked Read 8 Sent Done Done false
28245 8425 Acked Flush 0 Sent Done Done false
28246 8426 Acked Write 7 Sent Done Done false
28247 8427 Acked Write 3 Sent Done Done false
28248 8428 Acked Write 8 Sent Done Done false
28249 8429 Acked Read 10 Sent Done Done false
28250 8430 Acked Read 9 Sent Done Done false
28251 8431 Acked Write 2 Sent Done Done false
28252 8432 Acked Write 4 Sent Done Done false
28253 8433 Acked Flush 0 Sent Done Done false
28254 8434 Acked Write 2 Sent Done Done false
28255 8435 Acked Write 8 Sent Done Done false
28256 8436 Acked Read 1 Sent Done Done false
28257 8437 Acked Read 10 Sent Done Done false
28258 8438 Acked Read 10 Sent Done Done false
28259 8439 Acked Flush 0 Sent Done Done false
28260 8440 Acked Write 9 Sent Done Done false
28261 8441 Acked Flush 0 Sent Done Done false
28262 8442 Acked Write 3 Sent Done Done false
28263 8443 Acked Write 2 Sent Done Done false
28264 8444 Acked Read 3 Sent Done Done false
28265 8445 Acked Read 3 Sent Done Done false
28266 8446 Acked Read 4 Sent Done Done false
28267 8447 Acked Read 8 Sent Done Done false
28268 8448 Acked Read 7 Sent Done Done false
28269 8449 Acked Read 4 Sent Done Done false
28270 8450 Acked Write 10 Sent Done Done false
28271 8451 Acked Read 2 Sent Done Done false
28272 8452 Acked Flush 0 Sent Done Done false
28273 8453 Acked Flush 0 Sent Done Done false
28274 8454 Acked Read 9 Sent Done Done false
28275 8455 Acked Read 8 Sent Done Done false
28276 8456 Acked Write 7 Sent Done Done false
28277 8457 Acked Read 4 Sent Done Done false
28278 8458 Acked Write 3 Sent Done Done false
28279 8459 Acked Write 1 Sent Done Done false
28280 8460 Acked Write 3 Sent Done Done false
28281 8461 Acked Write 2 Sent Done Done false
28282 8462 Acked Write 1 Sent Done Done false
28283 8463 Acked Flush 0 Sent Done Done false
28284 8464 Acked Flush 0 Sent Done Done false
28285 8465 Acked Read 1 Sent Done Done false
28286 8466 Acked Write 7 Sent Done Done false
28287 8467 Acked Write 3 Sent Done Done false
28288 8468 Acked Write 4 Sent Done Done false
28289 8469 Acked Write 6 Sent Done Done false
28290 8470 Acked Write 4 Sent Done Done false
28291 8471 Acked Flush 0 Sent Done Done false
28292 8472 Acked Read 2 Sent Done Done false
28293 8473 Acked Read 5 Sent Done Done false
28294 8474 Acked Read 2 Sent Done Done false
28295 8475 Acked Read 6 Sent Done Done false
28296 8476 Acked Read 6 Sent Done Done false
28297 8477 Acked Flush 0 Sent Done Done false
28298 8478 Acked Read 5 Sent Done Done false
28299 8479 Acked Write 3 Sent Done Done false
28300 8480 Acked Flush 0 Sent Done Done false
28301 8481 Acked Write 5 Sent Done Done false
28302 8482 Acked Read 9 Sent Done Done false
28303 8483 Acked Read 10 Sent Done Done false
28304 8484 Acked Write 9 Sent Done Done false
28305 8485 Acked Read 7 Sent Done Done false
28306 8486 Acked Write 1 Sent Done Done false
28307 8487 Acked Write 7 Sent Done Done false
28308 8488 Acked Read 6 Sent Done Done false
28309 8489 Acked Read 8 Sent Done Done false
28310 8490 Acked Flush 0 Sent Done Done false
28311 8491 Acked Write 3 Sent Done Done false
28312 8492 Acked Read 8 Sent Done Done false
28313 8493 Acked Read 5 Sent Done Done false
28314 8494 Acked Write 2 Sent Done Done false
28315 8495 Acked Read 4 Sent Done Done false
28316 8496 Acked Read 10 Sent Done Done false
28317 8497 Acked Write 6 Sent Done Done false
28318 8498 Acked Read 1 Sent Done Done false
28319 8499 Acked Read 7 Sent Done Done false
28320 8500 Acked Read 3 Sent Done Done false
28321 8501 Acked Write 4 Sent Done Done false
28322 8502 Acked Write 3 Sent Done Done false
28323 8503 Acked Write 7 Sent Done Done false
28324 8504 Acked Write 8 Sent Done Done false
28325 8505 Acked Read 5 Sent Done Done false
28326 8506 Acked Write 1 Sent Done Done false
28327 8507 Acked Flush 0 Sent Done Done false
28328 8508 Acked Write 7 Sent Done Done false
28329 8509 Acked Write 3 Sent Done Done false
28330 8510 Acked Read 7 Sent Done Done false
28331 8511 Acked Read 6 Sent Done Done false
28332 8512 Acked Read 2 Sent Done Done false
28333 8513 Acked Flush 0 Sent Done Done false
28334 8514 Acked Read 1 Sent Done Done false
28335 8515 Acked Read 3 Sent Done Done false
28336 8516 Acked Read 6 Sent Done Done false
28337 8517 Acked Read 9 Sent Done Done false
28338 8518 Acked Write 1 Sent Done Done false
28339 8519 Acked Write 6 Sent Done Done false
28340 8520 Acked Write 2 Sent Done Done false
28341 8521 Acked Read 5 Sent Done Done false
28342 8522 Acked Read 2 Sent Done Done false
28343 8523 Acked Read 9 Sent Done Done false
28344 8524 Acked Read 5 Sent Done Done false
28345 8525 Acked Write 6 Sent Done Done false
28346 8526 Acked Read 6 Sent Done Done false
28347 8527 Acked Write 9 Sent Done Done false
28348 8528 Acked Write 5 Sent Done Done false
28349 8529 Acked Read 3 Sent Done Done false
28350 8530 Acked Read 5 Sent Done Done false
28351 8531 Acked Write 10 Sent Done Done false
28352 8532 Acked Read 10 Sent Done Done false
28353 8533 Acked Write 2 Sent Done Done false
28354 8534 Acked Write 6 Sent Done Done false
28355 8535 Acked Read 3 Sent Done Done false
28356 8536 Acked Flush 0 Sent Done Done false
28357 8537 Acked Write 5 Sent Done Done false
28358 8538 Acked Write 7 Sent Done Done false
28359 8539 Acked Read 1 Sent Done Done false
28360 8540 Acked Read 7 Sent Done Done false
28361 8541 Acked Read 7 Sent Done Done false
28362 8542 Acked Write 6 Sent Done Done false
28363 8543 Acked Read 9 Sent Done Done false
28364 8544 Acked Read 1 Sent Done Done false
28365 8545 Acked Read 2 Sent Done Done false
28366 8546 Acked Flush 0 Sent Done Done false
28367 STATES DS:0 DS:1 DS:2 TOTAL
28368 Sent 302 0 0 302
28369 Done 0 302 302 604
28370 Skipped 0 0 0 0
28371 Error 0 0 0 0
28372 Last Flush: 8244 8546 8546
28373 Downstairs last five completed: 8244 8243 8242 8241 8240
28374 Upstairs last five completed: 8546 8545 8544 8543 8542
28375 CLIENT: Up:0 ds:302 act:2
283762024-11-19T17:13:55.660ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283772024-11-19T17:13:55.660ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50822) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283782024-11-19T17:13:55.661ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283792024-11-19T17:13:55.662ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283802024-11-19T17:13:55.662ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283812024-11-19T17:13:55.662ZINFOcrucible: send last flush ID to this DS: 8244 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283822024-11-19T17:13:55.662ZINFOcrucible: Replied this last flush ID: 8244 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283832024-11-19T17:13:55.662ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283842024-11-19T17:13:55.662ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283852024-11-19T17:13:55.662ZINFOcrucible: [0] client re-new 302 jobs since flush 8244 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
283862024-11-19T17:13:55.662ZINFOcrucible: [0] Replayed 302 jobs since flush: 8244 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
28387 ----------------------------------------------------------------
28388 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28389 Downstairs last five completed: 8546 8545 8544 8543 8542
28390 Upstairs last five completed: 8546 8545 8544 8543 8542
28391 CLIENT: Up:0 ds:0 act:3
28392 Replay: All jobs finished
28393 Main task has work to do, go find it
28394 got dsc Stop(2)
28395 stop 2
28396 [8830] Got stop action so:false kr:false
28397 Nov 19 17:13:57.898 INFO request completed, latency_us: 323, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 4ea26ae5-937e-4353-a738-0a3456e60ccb, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
283982024-11-19T17:13:57.898ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
283992024-11-19T17:13:57.898ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
284002024-11-19T17:13:57.898ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
284012024-11-19T17:13:57.898ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
284022024-11-19T17:13:57.898ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
28403 Nov 19 17:13:57.899 INFO request completed, latency_us: 153, response_code: 200, uri: /state/cid/2, method: GET, {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:13:57.899086056Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
28404 req_id: 158cbd7b-95a1-4aef-8c3e-7d8d27b1e949, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
28405 [8830] Exited with: ExitStatus(unix_wait_status(9))
28406 [8830][2] reports Exit
28407 Nov 19 17:14:01.900 INFO request completed, latency_us: 192, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7ab7c865-21b9-4dce-9b22-c328f174da37, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
28408 001/300 Write block 2762 len 8192 data: 3 2
28409 002/300 Read block 4791 len 28672
28410 003/300 Write block 3378 len 24576 data: 4 4 5 3 2 3
28411 004/300 Write block 4232 len 40960 data: 3 2 2 2 3 3 4 4 4 5
28412 005/300 Write block 3937 len 28672 data: 2 4 3 3 3 3 4
28413 006/300 Read block 3830 len 20480
28414 007/300 Write block 3011 len 8192 data: 3 3
28415 008/300 Read block 2987 len 24576
28416 009/300 Read block 414 len 24576
28417 010/300 Write block 4797 len 32768 data: 7 6 6 6 7 5 6 5
28418 011/300 Write block 1806 len 24576 data: 6 6 6 5 4 6
28419 012/300 Read block 599 len 8192
28420 013/300 Write block 2514 len 16384 data: 5 5 4 3
28421 014/300 Flush
28422 015/300 Read block 4756 len 28672
28423 016/300 Write block 4643 len 24576 data: 3 5 5 5 3 3
28424 017/300 Flush
28425 018/300 Write block 3294 len 16384 data: 6 6 5 5
28426 019/300 Write block 3469 len 36864 data: 4 4 4 3 3 3 5 4 4
28427 020/300 Write block 3714 len 32768 data: 8 6 5 4 4 4 4 4
28428 021/300 Flush
28429 022/300 Flush
28430 023/300 Read block 3465 len 8192
28431 024/300 Read block 1344 len 4096
28432 025/300 Write block 4362 len 24576 data: 5 5 5 5 3 4
28433 026/300 Read block 3235 len 28672
28434 027/300 Write block 1901 len 4096 data: 4
28435 028/300 Write block 3073 len 12288 data: 5 5 5
28436 029/300 Write block 3810 len 28672 data: 5 6 2 2 2 3 3
28437 030/300 Flush
28438 031/300 Read block 3074 len 36864
28439 032/300 Read block 4646 len 32768
28440 033/300 Write block 1321 len 32768 data: 6 7 6 5 5 5 3 4
28441 034/300 Read block 955 len 36864
28442 035/300 Write block 2273 len 16384 data: 7 8 7 9
28443 036/300 Read block 4569 len 8192
28444 037/300 Read block 4685 len 16384
28445 038/300 Write block 1723 len 20480 data: 6 6 6 5 5
28446 039/300 Flush
28447 040/300 Write block 4766 len 8192 data: 4 6
28448 041/300 Read block 1678 len 16384
28449 042/300 Write block 1867 len 24576 data: 7 7 7 7 7 7
28450 043/300 Read block 4213 len 28672
28451 044/300 Read block 4092 len 36864
28452 045/300 Read block 1216 len 8192
28453 046/300 Write block 2439 len 40960 data: 4 4 4 4 4 3 3 3 3 3
28454 047/300 Read block 4700 len 8192
28455 048/300 Read block 2026 len 4096
28456 049/300 Write block 4980 len 4096 data: 3
28457 050/300 Read block 230 len 8192
28458 051/300 Write block 1262 len 12288 data: 5 5 5
28459 052/300 Read block 2049 len 20480
28460 053/300 Write block 2607 len 4096 data: 5
28461 054/300 Read block 4559 len 24576
28462 055/300 Read block 2860 len 28672
28463 056/300 Read block 2291 len 28672
28464 057/300 Read block 4608 len 24576
28465 058/300 Read block 3345 len 16384
28466 059/300 Write block 2292 len 28672 data: 3 4 6 6 8 8 8
28467 060/300 Read block 3566 len 4096
28468 061/300 Write block 3554 len 16384 data: 5 4 6 6
28469 062/300 Read block 1135 len 24576
28470 063/300 Read block 3166 len 24576
28471 064/300 Flush
28472 065/300 Write block 1770 len 32768 data: 2 4 4 4 4 3 3 5
28473 066/300 Read block 570 len 40960
28474 067/300 Read block 256 len 4096
28475 068/300 Flush
28476 069/300 Write block 4741 len 24576 data: 4 6 5 5 5 4
28477 070/300 Write block 56 len 32768 data: 4 5 6 7 8 7 7 7
28478 071/300 Read block 4167 len 20480
28479 072/300 Read block 2731 len 20480
28480 073/300 Write block 1739 len 4096 data: 4
28481 074/300 Read block 173 len 16384
28482 075/300 Read block 649 len 4096
28483 076/300 Write block 2228 len 36864 data: 4 4 5 6 7 6 6 5 3
28484 077/300 Write block 2423 len 28672 data: 5 5 7 7 6 5 5
28485 078/300 Read block 4055 len 28672
28486 079/300 Flush
28487 080/300 Write block 4109 len 32768 data: 3 3 3 3 2 3 3 2
28488 081/300 Write block 694 len 4096 data: 4
28489 082/300 Read block 3129 len 24576
28490 083/300 Read block 4931 len 12288
28491 084/300 Flush
28492 085/300 Write block 4388 len 24576 data: 6 6 5 4 6 6
28493 086/300 Read block 3497 len 12288
28494 087/300 Write block 1970 len 4096 data: 6
28495 088/300 Read block 2328 len 28672
28496 089/300 Read block 1716 len 32768
28497 090/300 Read block 1927 len 4096
28498 091/300 Read block 1912 len 40960
28499 092/300 Read block 314 len 24576
28500 093/300 Write block 620 len 40960 data: 7 5 5 6 6 6 6 7 7 7
28501 094/300 Read block 4670 len 28672
28502 095/300 Flush
28503 096/300 Read block 143 len 8192
28504 097/300 Read block 4341 len 12288
28505 098/300 Write block 604 len 16384 data: 3 2 3 4
28506 099/300 Write block 3066 len 4096 data: 2
28507 100/300 Write block 4954 len 28672 data: 4 5 6 7 7 7 6
28508 101/300 Read block 4798 len 12288
28509 102/300 Read block 457 len 24576
28510 103/300 Read block 2372 len 16384
28511 104/300 Read block 627 len 40960
28512 105/300 Write block 860 len 24576 data: 7 7 7 6 2 2
28513 106/300 Read block 1951 len 12288
28514 107/300 Write block 4548 len 32768 data: 5 5 6 5 6 6 6 6
28515 108/300 Write block 3993 len 24576 data: 9 8 6 4 6 6
28516 109/300 Write block 4511 len 36864 data: 3 4 5 5 5 6 7 7 7
28517 110/300 Flush
28518 111/300 Write block 4604 len 24576 data: 4 5 5 4 4 4
28519 112/300 Read block 2745 len 12288
28520 113/300 Read block 3497 len 40960
28521 114/300 Write block 3261 len 36864 data: 4 5 5 5 6 5 6 6 6
28522 115/300 Write block 2115 len 4096 data: 2
28523 116/300 Read block 4842 len 28672
28524 117/300 Write block 3490 len 12288 data: 6 7 4
28525 118/300 Read block 2132 len 16384
28526 119/300 Read block 2396 len 16384
28527 120/300 Read block 1936 len 16384
28528 121/300 Write block 373 len 20480 data: 4 4 3 3 4
28529 122/300 Read block 37 len 24576
28530 123/300 Write block 4445 len 36864 data: 3 4 4 2 2 2 2 3 3
28531 124/300 Read block 4500 len 36864
28532 125/300 Write block 4963 len 36864 data: 4 4 4 4 4 4 4 5 5
28533 126/300 Flush
28534 127/300 Read block 2528 len 28672
28535 128/300 Write block 386 len 28672 data: 6 6 7 8 7 7 6
28536 129/300 Write block 3256 len 20480 data: 4 5 5 5 3
28537 130/300 Read block 855 len 12288
28538 131/300 Write block 4068 len 32768 data: 6 7 5 5 6 6 6 6
28539 132/300 Read block 3647 len 16384
28540 133/300 Flush
28541 134/300 Write block 1119 len 20480 data: 6 7 4 5 6
28542 135/300 Write block 2361 len 4096 data: 6
28543 136/300 Write block 1219 len 8192 data: 4 4
28544 137/300 Write block 3565 len 28672 data: 7 5 5 6 5 5 6
28545 138/300 Read block 1550 len 12288
28546 139/300 Write block 190 len 12288 data: 5 5 5
28547 140/300 Write block 4173 len 8192 data: 2 2
28548 141/300 Read block 394 len 8192
28549 142/300 Read block 742 len 36864
28550 143/300 Write block 1413 len 8192 data: 3 4
28551 144/300 Read block 4190 len 36864
28552 145/300 Read block 3834 len 40960
28553 146/300 Write block 2353 len 32768 data: 4 4 4 4 4 6 6 5
28554 147/300 Read block 252 len 12288
28555 148/300 Read block 4860 len 36864
28556 149/300 Write block 1081 len 36864 data: 3 3 3 3 4 4 5 5 6
28557 150/300 Write block 2557 len 20480 data: 5 5 6 9 9
28558 151/300 Write block 2254 len 16384 data: 8 7 8 9
28559 152/300 Read block 1269 len 8192
28560 153/300 Write block 3884 len 4096 data: 6
28561 154/300 Read block 2725 len 36864
28562 155/300 Write block 3990 len 16384 data: 8 8 8 10
28563 156/300 Read block 1110 len 36864
28564 157/300 Write block 3337 len 32768 data: 7 7 7 6 5 4 3 4
28565 158/300 Flush
28566 159/300 Read block 1206 len 4096
28567 160/300 Read block 1721 len 28672
28568 161/300 Read block 4608 len 4096
28569 162/300 Read block 4726 len 36864
28570 163/300 Read block 1404 len 12288
28571 164/300 Write block 1504 len 24576 data: 3 3 3 3 3 3
28572 165/300 Read block 4595 len 40960
28573 166/300 Read block 4911 len 8192
28574 167/300 Write block 2144 len 32768 data: 7 6 7 4 6 7 7 6
28575 168/300 Write block 3314 len 8192 data: 6 7
28576 169/300 Read block 1571 len 40960
28577 170/300 Read block 550 len 24576
28578 171/300 Read block 1888 len 4096
28579 172/300 Write block 2259 len 8192 data: 5 4
28580 173/300 Read block 1261 len 4096
28581 174/300 Write block 455 len 16384 data: 7 7 6 8
28582 175/300 Flush
28583 176/300 Read block 4556 len 24576
28584 177/300 Read block 198 len 32768
28585 178/300 Flush
28586 179/300 Write block 1840 len 12288 data: 3 5 4
28587 180/300 Read block 2841 len 16384
28588 181/300 Write block 1207 len 12288 data: 4 4 4
28589 182/300 Write block 2735 len 28672 data: 4 5 5 5 5 5 4
28590 183/300 Read block 658 len 20480
28591 184/300 Write block 4164 len 28672 data: 5 4 3 2 3 3 2
28592 185/300 Write block 2895 len 8192 data: 5 7
28593 186/300 Write block 3139 len 4096 data: 3
28594 187/300 Write block 2958 len 4096 data: 2
28595 188/300 Read block 125 len 20480
28596 189/300 Flush
28597 190/300 Write block 1789 len 20480 data: 5 3 2 3 3
28598 191/300 Read block 3191 len 16384
28599 192/300 Write block 2011 len 12288 data: 4 3 5
28600 193/300 Read block 3493 len 28672
28601 194/300 Write block 2233 len 12288 data: 7 7 6
28602 195/300 Flush
28603 196/300 Write block 433 len 4096 data: 5
28604 197/300 Write block 3873 len 12288 data: 7 7 6
28605 198/300 Write block 751 len 8192 data: 9 8
28606 199/300 Read block 1806 len 20480
28607 200/300 Write block 3404 len 16384 data: 4 4 5 5
28608 201/300 Flush
28609 202/300 Write block 97 len 24576 data: 5 6 6 8 6 5
28610 203/300 Write block 4272 len 32768 data: 3 5 2 2 2 2 2 3
28611 204/300 Read block 4073 len 20480
28612 205/300 Flush
28613 206/300 Read block 1350 len 8192
28614 207/300 Flush
28615 208/300 Read block 886 len 20480
28616 209/300 Read block 1445 len 16384
28617 210/300 Flush
28618 211/300 Write block 446 len 16384 data: 7 6 7 7
28619 212/300 Flush
28620 213/300 Read block 1556 len 24576
28621 214/300 Read block 3982 len 36864
28622 215/300 Read block 4164 len 4096
28623 216/300 Read block 4271 len 32768
28624 217/300 Read block 3169 len 40960
28625 218/300 Flush
28626 219/300 Read block 3653 len 8192
28627 220/300 Flush
28628 221/300 Write block 4208 len 4096 data: 4
28629 222/300 Read block 2860 len 8192
28630 223/300 Write block 1786 len 28672 data: 4 5 5 6 4 3 4
28631 224/300 Write block 618 len 8192 data: 9 9
28632 225/300 Read block 4752 len 8192
28633 226/300 Write block 1658 len 40960 data: 3 5 6 4 3 3 3 3 3 4
28634 227/300 Read block 1230 len 32768
28635 228/300 Read block 3568 len 4096
28636 229/300 Write block 3180 len 16384 data: 4 5 5 3
28637 230/300 Write block 2290 len 24576 data: 5 4 4 5 7 7
28638 231/300 Read block 3419 len 4096
28639 232/300 Flush
28640 233/300 Read block 2434 len 40960
28641 234/300 Read block 858 len 32768
28642 235/300 Read block 2259 len 40960
28643 236/300 Read block 3040 len 4096
28644 237/300 Read block 3101 len 8192
28645 238/300 Write block 3150 len 24576 data: 7 7 8 8 6 6
28646 239/300 Read block 1079 len 4096
28647 240/300 Write block 1947 len 28672 data: 6 7 7 8 10 7 6
28648 241/300 Flush
28649 242/300 Read block 2894 len 12288
28650 243/300 Read block 2231 len 16384
28651 244/300 Read block 4119 len 40960
28652 245/300 Write block 2622 len 20480 data: 8 5 4 4 3
28653 246/300 Read block 339 len 8192
28654 247/300 Read block 901 len 40960
28655 248/300 Read block 3002 len 24576
28656 249/300 Read block 3241 len 8192
28657 250/300 Write block 2681 len 28672 data: 7 9 9 9 9 8 8
28658 251/300 Read block 887 len 24576
28659 252/300 Read block 1603 len 28672
28660 253/300 Write block 1787 len 40960 data: 6 6 7 5 4 5 4 3 3 4
28661 254/300 Read block 1477 len 28672
28662 255/300 Read block 3802 len 28672
28663 256/300 Write block 2587 len 16384 data: 7 8 6 6
28664 257/300 Write block 2893 len 20480 data: 6 5 6 8 6
28665 258/300 Flush
28666 259/300 Write block 4378 len 32768 data: 3 4 4 6 8 8 7 7
28667 260/300 Write block 2615 len 36864 data: 6 7 7 5 6 7 7 9 6
28668 261/300 Write block 579 len 16384 data: 3 5 4 4
28669 262/300 Write block 212 len 36864 data: 6 6 5 7 7 7 8 8 7
28670 263/300 Read block 4147 len 16384
28671 264/300 Write block 2484 len 28672 data: 7 7 7 8 8 6 6
28672 265/300 Write block 593 len 4096 data: 5
28673 266/300 Write block 1945 len 36864 data: 6 6 7 8 8 9 11 8 7
28674 267/300 Read block 1797 len 4096
28675 268/300 Write block 250 len 36864 data: 6 6 6 5 6 7 7 6 6
28676 269/300 Write block 3891 len 32768 data: 5 6 7 6 6 7 6 6
28677 270/300 Read block 1529 len 12288
28678 271/300 Write block 4460 len 8192 data: 8 7
28679 272/300 Write block 3383 len 36864 data: 4 4 5 7 7 8 8 7 7
28680 273/300 Read block 3592 len 40960
28681 274/300 Read block 4553 len 12288
28682 275/300 Read block 3333 len 32768
28683 276/300 Read block 3980 len 4096
28684 277/300 Write block 1922 len 20480 data: 2 2 3 3 4
28685 278/300 Read block 3644 len 20480
28686 279/300 Read block 126 len 4096
28687 280/300 Read block 1817 len 24576
28688 281/300 Read block 1866 len 36864
28689 282/300 Read block 1166 len 12288
28690 283/300 Write block 2439 len 12288 data: 5 5 5
28691 284/300 Read block 2327 len 16384
28692 285/300 Read block 271 len 28672
28693 286/300 Read block 3731 len 24576
28694 287/300 Write block 1661 len 4096 data: 5
28695 288/300 Write block 4587 len 16384 data: 5 7 7 7
28696 289/300 Flush
28697 290/300 Read block 2592 len 8192
28698 291/300 Flush
28699 292/300 Flush
28700 293/300 Read block 1191 len 12288
28701 294/300 Write block 3981 len 28672 data: 5 5 3 3 3 4 7
28702 295/300 Write block 428 len 28672 data: 5 5 5 5 5 6 5
28703 296/300 Read block 971 len 12288
28704 297/300 Read block 1146 len 40960
28705 298/300 Write block 3123 len 8192 data: 6 9
28706 299/300 Read block 2320 len 20480
28707 300/300 Write block 3226 len 4096 data: 8
28708 301/300 Read block 2219 len 32768
28709 Main task has work to do, go find it
28710 got dsc Start(2)
28711 start 2
28712 Nov 19 17:14:02.131 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/2, method: POST, req_id: 0b1bc875-f4ad-4649-bccc-8a1213da2bdb, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
28713 Replay: started 2, returned:Ok(())
28714 ----------------------------------------------------------------
28715 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28716 States: Active Active Offline
28717 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28718 8547 Acked Write 2 Done Done Sent false
28719 8548 Acked Read 7 Done Done Sent false
28720 8549 Acked Write 6 Done Done Sent false
28721 8550 Acked Write 10 Done Done Sent false
28722 8551 Acked Write 7 Done Done Sent false
28723 8552 Acked Read 5 Done Done Sent false
28724 8553 Acked Write 2 Done Done Sent false
28725 8554 Acked Read 6 Done Done Sent false
28726 8555 Acked Read 6 Done Done Sent false
28727 8556 Acked Write 8 Done Done Sent false
28728 8557 Acked Write 6 Done Done Sent false
28729 8558 Acked Read 2 Done Done Sent false
28730 8559 Acked Write 4 Done Done Sent false
28731 8560 Acked Flush 0 Done Done Sent false
28732 8561 Acked Read 7 Done Done Sent false
28733 8562 Acked Write 6 Done Done Sent false
28734 8563 Acked Flush 0 Done Done Sent false
28735 8564 Acked Write 4 Done Done Sent false
28736 8565 Acked Write 9 Done Done Sent false
28737 8566 Acked Write 8 Done Done Sent false
28738 8567 Acked Flush 0 Done Done Sent false
28739 8568 Acked Flush 0 Done Done Sent false
28740 8569 Acked Read 2 Done Done Sent false
28741 8570 Acked Read 1 Done Done Sent false
28742 8571 Acked Write 6 Done Done Sent false
28743 8572 Acked Read 7 Done Done Sent false
28744 8573 Acked Write 1 Done Done Sent false
28745 8574 Acked Write 3 Done Done Sent false
28746 8575 Acked Write 7 Done Done Sent false
28747 8576 Acked Flush 0 Done Done Sent false
28748 8577 Acked Read 9 Done Done Sent false
28749 8578 Acked Read 8 Done Done Sent false
28750 8579 Acked Write 8 Done Done Sent false
28751 8580 Acked Read 9 Done Done Sent false
28752 8581 Acked Write 4 Done Done Sent false
28753 8582 Acked Read 2 Done Done Sent false
28754 8583 Acked Read 4 Done Done Sent false
28755 8584 Acked Write 5 Done Done Sent false
28756 8585 Acked Flush 0 Done Done Sent false
28757 8586 Acked Write 2 Done Done Sent false
28758 8587 Acked Read 4 Done Done Sent false
28759 8588 Acked Write 6 Done Done Sent false
28760 8589 Acked Read 7 Done Done Sent false
28761 8590 Acked Read 9 Done Done Sent false
28762 8591 Acked Read 2 Done Done Sent false
28763 8592 Acked Write 10 Done Done Sent false
28764 8593 Acked Read 2 Done Done Sent false
28765 8594 Acked Read 1 Done Done Sent false
28766 8595 Acked Write 1 Done Done Sent false
28767 8596 Acked Read 2 Done Done Sent false
28768 8597 Acked Write 3 Done Done Sent false
28769 8598 Acked Read 5 Done Done Sent false
28770 8599 Acked Write 1 Done Done Sent false
28771 8600 Acked Read 6 Done Done Sent false
28772 8601 Acked Read 7 Done Done Sent false
28773 8602 Acked Read 7 Done Done Sent false
28774 8603 Acked Read 6 Done Done Sent false
28775 8604 Acked Read 4 Done Done Sent false
28776 8605 Acked Write 7 Done Done Sent false
28777 8606 Acked Read 1 Done Done Sent false
28778 8607 Acked Write 4 Done Done Sent false
28779 8608 Acked Read 6 Done Done Sent false
28780 8609 Acked Read 6 Done Done Sent false
28781 8610 Acked Flush 0 Done Done Sent false
28782 8611 Acked Write 8 Done Done Sent false
28783 8612 Acked Read 10 Done Done Sent false
28784 8613 Acked Read 1 Done Done Sent false
28785 8614 Acked Flush 0 Done Done Sent false
28786 8615 Acked Write 6 Done Done Sent false
28787 8616 Acked Write 8 Done Done Sent false
28788 8617 Acked Read 5 Done Done Sent false
28789 8618 Acked Read 5 Done Done Sent false
28790 8619 Acked Write 1 Done Done Sent false
28791 8620 Acked Read 4 Done Done Sent false
28792 8621 Acked Read 1 Done Done Sent false
28793 8622 Acked Write 9 Done Done Sent false
28794 8623 Acked Write 7 Done Done Sent false
28795 8624 Acked Read 7 Done Done Sent false
28796 8625 Acked Flush 0 Done Done Sent false
28797 8626 Acked Write 8 Done Done Sent false
28798 8627 Acked Write 1 Done Done Sent false
28799 8628 Acked Read 6 Done Done Sent false
28800 8629 Acked Read 3 Done Done Sent false
28801 8630 Acked Flush 0 Done Done Sent false
28802 8631 Acked Write 6 Done Done Sent false
28803 8632 Acked Read 3 Done Done Sent false
28804 8633 Acked Write 1 Done Done Sent false
28805 8634 Acked Read 7 Done Done Sent false
28806 8635 Acked Read 8 Done Done Sent false
28807 8636 Acked Read 1 Done Done Sent false
28808 8637 Acked Read 10 Done Done Sent false
28809 8638 Acked Read 6 Done Done Sent false
28810 8639 Acked Write 10 Done Done Sent false
28811 8640 Acked Read 7 Done Done Sent false
28812 8641 Acked Flush 0 Done Done Sent false
28813 8642 Acked Read 2 Done Done Sent false
28814 8643 Acked Read 3 Done Done Sent false
28815 8644 Acked Write 4 Done Done Sent false
28816 8645 Acked Write 1 Done Done Sent false
28817 8646 Acked Write 7 Done Done Sent false
28818 8647 Acked Read 3 Done Done Sent false
28819 8648 Acked Read 6 Done Done Sent false
28820 8649 Acked Read 4 Done Done Sent false
28821 8650 Acked Read 10 Done Done Sent false
28822 8651 Acked Write 6 Done Done Sent false
28823 8652 Acked Read 3 Done Done Sent false
28824 8653 Acked Write 8 Done Done Sent false
28825 8654 Acked Write 6 Done Done Sent false
28826 8655 Acked Write 9 Done Done Sent false
28827 8656 Acked Flush 0 Done Done Sent false
28828 8657 Acked Write 6 Done Done Sent false
28829 8658 Acked Read 3 Done Done Sent false
28830 8659 Acked Read 10 Done Done Sent false
28831 8660 Acked Write 9 Done Done Sent false
28832 8661 Acked Write 1 Done Done Sent false
28833 8662 Acked Read 7 Done Done Sent false
28834 8663 Acked Write 3 Done Done Sent false
28835 8664 Acked Read 4 Done Done Sent false
28836 8665 Acked Read 4 Done Done Sent false
28837 8666 Acked Read 4 Done Done Sent false
28838 8667 Acked Write 5 Done Done Sent false
28839 8668 Acked Read 6 Done Done Sent false
28840 8669 Acked Write 9 Done Done Sent false
28841 8670 Acked Read 9 Done Done Sent false
28842 8671 Acked Write 9 Done Done Sent false
28843 8672 Acked Flush 0 Done Done Sent false
28844 8673 Acked Read 7 Done Done Sent false
28845 8674 Acked Write 7 Done Done Sent false
28846 8675 Acked Write 5 Done Done Sent false
28847 8676 Acked Read 3 Done Done Sent false
28848 8677 Acked Write 8 Done Done Sent false
28849 8678 Acked Read 4 Done Done Sent false
28850 8679 Acked Flush 0 Done Done Sent false
28851 8680 Acked Write 5 Done Done Sent false
28852 8681 Acked Write 1 Done Done Sent false
28853 8682 Acked Write 2 Done Done Sent false
28854 8683 Acked Write 7 Done Done Sent false
28855 8684 Acked Read 3 Done Done Sent false
28856 8685 Acked Write 3 Done Done Sent false
28857 8686 Acked Write 2 Done Done Sent false
28858 8687 Acked Read 2 Done Done Sent false
28859 8688 Acked Read 9 Done Done Sent false
28860 8689 Acked Write 2 Done Done Sent false
28861 8690 Acked Read 9 Done Done Sent false
28862 8691 Acked Read 10 Done Done Sent false
28863 8692 Acked Write 8 Done Done Sent false
28864 8693 Acked Read 3 Done Done Sent false
28865 8694 Acked Read 9 Done Done Sent false
28866 8695 Acked Write 9 Done Done Sent false
28867 8696 Acked Write 5 Done Done Sent false
28868 8697 Acked Write 4 Done Done Sent false
28869 8698 Acked Read 2 Done Done Sent false
28870 8699 Acked Write 1 Done Done Sent false
28871 8700 Acked Read 9 Done Done Sent false
28872 8701 Acked Write 4 Done Done Sent false
28873 8702 Acked Read 9 Done Done Sent false
28874 8703 Acked Write 8 Done Done Sent false
28875 8704 Acked Flush 0 Done Done Sent false
28876 8705 Acked Read 1 Done Done Sent false
28877 8706 Acked Read 7 Done Done Sent false
28878 8707 Acked Read 1 Done Done Sent false
28879 8708 Acked Read 9 Done Done Sent false
28880 8709 Acked Read 3 Done Done Sent false
28881 8710 Acked Write 6 Done Done Sent false
28882 8711 Acked Read 10 Done Done Sent false
28883 8712 Acked Read 2 Done Done Sent false
28884 8713 Acked Write 8 Done Done Sent false
28885 8714 Acked Write 2 Done Done Sent false
28886 8715 Acked Read 10 Done Done Sent false
28887 8716 Acked Read 6 Done Done Sent false
28888 8717 Acked Read 1 Done Done Sent false
28889 8718 Acked Write 2 Done Done Sent false
28890 8719 Acked Read 1 Done Done Sent false
28891 8720 Acked Write 4 Done Done Sent false
28892 8721 Acked Flush 0 Done Done Sent false
28893 8722 Acked Read 6 Done Done Sent false
28894 8723 Acked Read 8 Done Done Sent false
28895 8724 Acked Flush 0 Done Done Sent false
28896 8725 Acked Write 3 Done Done Sent false
28897 8726 Acked Read 4 Done Done Sent false
28898 8727 Acked Write 3 Done Done Sent false
28899 8728 Acked Write 7 Done Done Sent false
28900 8729 Acked Read 5 Done Done Sent false
28901 8730 Acked Write 7 Done Done Sent false
28902 8731 Acked Write 2 Done Done Sent false
28903 8732 Acked Write 1 Done Done Sent false
28904 8733 Acked Write 1 Done Done Sent false
28905 8734 Acked Read 5 Done Done Sent false
28906 8735 Acked Flush 0 Done Done Sent false
28907 8736 Acked Write 5 Done Done Sent false
28908 8737 Acked Read 4 Done Done Sent false
28909 8738 Acked Write 3 Done Done Sent false
28910 8739 Acked Read 7 Done Done Sent false
28911 8740 Acked Write 3 Done Done Sent false
28912 8741 Acked Flush 0 Done Done Sent false
28913 8742 Acked Write 1 Done Done Sent false
28914 8743 Acked Write 3 Done Done Sent false
28915 8744 Acked Write 2 Done Done Sent false
28916 8745 Acked Read 5 Done Done Sent false
28917 8746 Acked Write 4 Done Done Sent false
28918 8747 Acked Flush 0 Done Done Sent false
28919 8748 Acked Write 6 Done Done Sent false
28920 8749 Acked Write 8 Done Done Sent false
28921 8750 Acked Read 5 Done Done Sent false
28922 8751 Acked Flush 0 Done Done Sent false
28923 8752 Acked Read 2 Done Done Sent false
28924 8753 Acked Flush 0 Done Done Sent false
28925 8754 Acked Read 5 Done Done Sent false
28926 8755 Acked Read 4 Done Done Sent false
28927 8756 Acked Flush 0 Done Done Sent false
28928 8757 Acked Write 4 Done Done Sent false
28929 8758 Acked Flush 0 Done Done Sent false
28930 8759 Acked Read 6 Done Done Sent false
28931 8760 Acked Read 9 Done Done Sent false
28932 8761 Acked Read 1 Done Done Sent false
28933 8762 Acked Read 8 Done Done Sent false
28934 8763 Acked Read 10 Done Done Sent false
28935 8764 Acked Flush 0 Done Done Sent false
28936 8765 Acked Read 2 Done Done Sent false
28937 8766 Acked Flush 0 Done Done Sent false
28938 8767 Acked Write 1 Done Done Sent false
28939 8768 Acked Read 2 Done Done Sent false
28940 8769 Acked Write 7 Done Done Sent false
28941 8770 Acked Write 2 Done Done Sent false
28942 8771 Acked Read 2 Done Done Sent false
28943 8772 Acked Write 10 Done Done Sent false
28944 8773 Acked Read 8 Done Done Sent false
28945 8774 Acked Read 1 Done Done Sent false
28946 8775 Acked Write 4 Done Done Sent false
28947 8776 Acked Write 6 Done Done Sent false
28948 8777 Acked Read 1 Done Done Sent false
28949 8778 Acked Flush 0 Done Done Sent false
28950 8779 Acked Read 10 Done Done Sent false
28951 8780 Acked Read 8 Done Done Sent false
28952 8781 Acked Read 10 Done Done Sent false
28953 8782 Acked Read 1 Done Done Sent false
28954 8783 Acked Read 2 Done Done Sent false
28955 8784 Acked Write 6 Done Done Sent false
28956 8785 Acked Read 1 Done Done Sent false
28957 8786 Acked Write 7 Done Done Sent false
28958 8787 Acked Flush 0 Done Done Sent false
28959 8788 Acked Read 3 Done Done Sent false
28960 8789 Acked Read 4 Done Done Sent false
28961 8790 Acked Read 10 Done Done Sent false
28962 8791 Acked Write 5 Done Done Sent false
28963 8792 Acked Read 2 Done Done Sent false
28964 8793 Acked Read 10 Done Done Sent false
28965 8794 Acked Read 6 Done Done Sent false
28966 8795 Acked Read 2 Done Done Sent false
28967 8796 Acked Write 7 Done Done Sent false
28968 8797 Acked Read 6 Done Done Sent false
28969 8798 Acked Read 7 Done Done Sent false
28970 8799 Acked Write 10 Done Done Sent false
28971 8800 Acked Read 7 Done Done Sent false
28972 8801 Acked Read 7 Done Done Sent false
28973 8802 Acked Write 4 Done Done Sent false
28974 8803 Acked Write 5 Done Done Sent false
28975 8804 Acked Flush 0 Done Done Sent false
28976 8805 Acked Write 8 Done Done Sent false
28977 8806 Acked Write 9 Done Done Sent false
28978 8807 Acked Write 4 Done Done Sent false
28979 8808 Acked Write 9 Done Done Sent false
28980 8809 Acked Read 4 Done Done Sent false
28981 8810 Acked Write 7 Done Done Sent false
28982 8811 Acked Write 1 Done Done Sent false
28983 8812 Acked Write 9 Done Done Sent false
28984 8813 Acked Read 1 Done Done Sent false
28985 8814 Acked Write 9 Done Done Sent false
28986 8815 Acked Write 8 Done Done Sent false
28987 8816 Acked Read 3 Done Done Sent false
28988 8817 Acked Write 2 Done Done Sent false
28989 8818 Acked Write 9 Done Done Sent false
28990 8819 Acked Read 10 Done Done Sent false
28991 8820 Acked Read 3 Done Done Sent false
28992 8821 Acked Read 8 Done Done Sent false
28993 8822 Acked Read 1 Done Done Sent false
28994 8823 Acked Write 5 Done Done Sent false
28995 8824 Acked Read 5 Done Done Sent false
28996 8825 Acked Read 1 Done Done Sent false
28997 8826 Acked Read 6 Done Done Sent false
28998 8827 Acked Read 9 Done Done Sent false
28999 8828 Acked Read 3 Done Done Sent false
29000 8829 Acked Write 3 Done Done Sent false
29001 8830 Acked Read 4 Done Done Sent false
29002 8831 Acked Read 7 Done Done Sent false
29003 8832 Acked Read 6 Done Done Sent false
29004 8833 Acked Write 1 Done Done Sent false
29005 8834 Acked Write 4 Done Done Sent false
29006 8835 Acked Flush 0 Done Done Sent false
29007 8836 Acked Read 2 Done Done Sent false
29008 8837 Acked Flush 0 Done Done Sent false
29009 8838 Acked Flush 0 Done Done Sent false
29010 8839 Acked Read 3 Done Done Sent false
29011 8840 Acked Write 7 Done Done Sent false
29012 8841 Acked Write 7 Done Done Sent false
29013 8842 Acked Read 3 Done Done Sent false
29014 8843 Acked Read 10 Done Done Sent false
29015 8844 Acked Write 2 Done Done Sent false
29016 8845 Acked Read 5 Done Done Sent false
29017 8846 Acked Write 1 Done Done Sent false
29018 8847 Acked Read 8 Done Done Sent false
29019 STATES DS:0 DS:1 DS:2 TOTAL
29020 Sent 0 0 301 301
29021 Done 301 301 0 602
29022 Skipped 0 0 0 0
29023 Error 0 0 0 0
29024 Last Flush: 8838 8838 8546
29025 Downstairs last five completed: 8546 8545 8544 8543 8542
29026 Upstairs last five completed: 8847 8846 8845 8844 8843
29027 CLIENT: Up:0 ds:301 act:2
29028 ----------------------------------------------------------------
29029 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29030 States: Active Active Offline
29031 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29032 8547 Acked Write 2 Done Done Sent false
29033 8548 Acked Read 7 Done Done Sent false
29034 8549 Acked Write 6 Done Done Sent false
29035 8550 Acked Write 10 Done Done Sent false
29036 8551 Acked Write 7 Done Done Sent false
29037 8552 Acked Read 5 Done Done Sent false
29038 8553 Acked Write 2 Done Done Sent false
29039 8554 Acked Read 6 Done Done Sent false
29040 8555 Acked Read 6 Done Done Sent false
29041 8556 Acked Write 8 Done Done Sent false
29042 8557 Acked Write 6 Done Done Sent false
29043 8558 Acked Read 2 Done Done Sent false
29044 8559 Acked Write 4 Done Done Sent false
29045 8560 Acked Flush 0 Done Done Sent false
29046 8561 Acked Read 7 Done Done Sent false
29047 8562 Acked Write 6 Done Done Sent false
29048 8563 Acked Flush 0 Done Done Sent false
29049 8564 Acked Write 4 Done Done Sent false
29050 8565 Acked Write 9 Done Done Sent false
29051 8566 Acked Write 8 Done Done Sent false
29052 8567 Acked Flush 0 Done Done Sent false
29053 8568 Acked Flush 0 Done Done Sent false
29054 8569 Acked Read 2 Done Done Sent false
29055 8570 Acked Read 1 Done Done Sent false
29056 8571 Acked Write 6 Done Done Sent false
29057 8572 Acked Read 7 Done Done Sent false
29058 8573 Acked Write 1 Done Done Sent false
29059 8574 Acked Write 3 Done Done Sent false
29060 8575 Acked Write 7 Done Done Sent false
29061 8576 Acked Flush 0 Done Done Sent false
29062 8577 Acked Read 9 Done Done Sent false
29063 8578 Acked Read 8 Done Done Sent false
29064 8579 Acked Write 8 Done Done Sent false
29065 8580 Acked Read 9 Done Done Sent false
29066 8581 Acked Write 4 Done Done Sent false
29067 8582 Acked Read 2 Done Done Sent false
29068 8583 Acked Read 4 Done Done Sent false
29069 8584 Acked Write 5 Done Done Sent false
29070 8585 Acked Flush 0 Done Done Sent false
29071 8586 Acked Write 2 Done Done Sent false
29072 8587 Acked Read 4 Done Done Sent false
29073 8588 Acked Write 6 Done Done Sent false
29074 8589 Acked Read 7 Done Done Sent false
29075 8590 Acked Read 9 Done Done Sent false
29076 8591 Acked Read 2 Done Done Sent false
29077 8592 Acked Write 10 Done Done Sent false
29078 8593 Acked Read 2 Done Done Sent false
29079 8594 Acked Read 1 Done Done Sent false
29080 8595 Acked Write 1 Done Done Sent false
29081 8596 Acked Read 2 Done Done Sent false
29082 8597 Acked Write 3 Done Done Sent false
29083 8598 Acked Read 5 Done Done Sent false
29084 8599 Acked Write 1 Done Done Sent false
29085 8600 Acked Read 6 Done Done Sent false
29086 8601 Acked Read 7 Done Done Sent false
29087 8602 Acked Read 7 Done Done Sent false
29088 8603 Acked Read 6 Done Done Sent false
29089 8604 Acked Read 4 Done Done Sent false
29090 8605 Acked Write 7 Done Done Sent false
29091 8606 Acked Read 1 Done Done Sent false
29092 8607 Acked Write 4 Done Done Sent false
29093 8608 Acked Read 6 Done Done Sent false
29094 8609 Acked Read 6 Done Done Sent false
29095 8610 Acked Flush 0 Done Done Sent false
29096 8611 Acked Write 8 Done Done Sent false
29097 8612 Acked Read 10 Done Done Sent false
29098 8613 Acked Read 1 Done Done Sent false
29099 8614 Acked Flush 0 Done Done Sent false
29100 8615 Acked Write 6 Done Done Sent false
29101 8616 Acked Write 8 Done Done Sent false
29102 8617 Acked Read 5 Done Done Sent false
29103 8618 Acked Read 5 Done Done Sent false
29104 8619 Acked Write 1 Done Done Sent false
29105 8620 Acked Read 4 Done Done Sent false
29106 8621 Acked Read 1 Done Done Sent false
29107 8622 Acked Write 9 Done Done Sent false
29108 8623 Acked Write 7 Done Done Sent false
29109 8624 Acked Read 7 Done Done Sent false
29110 8625 Acked Flush 0 Done Done Sent false
29111 8626 Acked Write 8 Done Done Sent false
29112 8627 Acked Write 1 Done Done Sent false
29113 8628 Acked Read 6 Done Done Sent false
29114 8629 Acked Read 3 Done Done Sent false
29115 8630 Acked Flush 0 Done Done Sent false
29116 8631 Acked Write 6 Done Done Sent false
29117 8632 Acked Read 3 Done Done Sent false
29118 8633 Acked Write 1 Done Done Sent false
29119 8634 Acked Read 7 Done Done Sent false
29120 8635 Acked Read 8 Done Done Sent false
29121 8636 Acked Read 1 Done Done Sent false
29122 8637 Acked Read 10 Done Done Sent false
29123 8638 Acked Read 6 Done Done Sent false
29124 8639 Acked Write 10 Done Done Sent false
29125 8640 Acked Read 7 Done Done Sent false
29126 8641 Acked Flush 0 Done Done Sent false
29127 8642 Acked Read 2 Done Done Sent false
29128 8643 Acked Read 3 Done Done Sent false
29129 8644 Acked Write 4 Done Done Sent false
29130 8645 Acked Write 1 Done Done Sent false
29131 8646 Acked Write 7 Done Done Sent false
29132 8647 Acked Read 3 Done Done Sent false
29133 8648 Acked Read 6 Done Done Sent false
29134 8649 Acked Read 4 Done Done Sent false
29135 8650 Acked Read 10 Done Done Sent false
29136 8651 Acked Write 6 Done Done Sent false
29137 8652 Acked Read 3 Done Done Sent false
29138 8653 Acked Write 8 Done Done Sent false
29139 8654 Acked Write 6 Done Done Sent false
29140 8655 Acked Write 9 Done Done Sent false
29141 8656 Acked Flush 0 Done Done Sent false
29142 8657 Acked Write 6 Done Done Sent false
29143 8658 Acked Read 3 Done Done Sent false
29144 8659 Acked Read 10 Done Done Sent false
29145 8660 Acked Write 9 Done Done Sent false
29146 8661 Acked Write 1 Done Done Sent false
29147 8662 Acked Read 7 Done Done Sent false
29148 8663 Acked Write 3 Done Done Sent false
29149 8664 Acked Read 4 Done Done Sent false
29150 8665 Acked Read 4 Done Done Sent false
29151 8666 Acked Read 4 Done Done Sent false
29152 8667 Acked Write 5 Done Done Sent false
29153 8668 Acked Read 6 Done Done Sent false
29154 8669 Acked Write 9 Done Done Sent false
29155 8670 Acked Read 9 Done Done Sent false
29156 8671 Acked Write 9 Done Done Sent false
29157 8672 Acked Flush 0 Done Done Sent false
29158 8673 Acked Read 7 Done Done Sent false
29159 8674 Acked Write 7 Done Done Sent false
29160 8675 Acked Write 5 Done Done Sent false
29161 8676 Acked Read 3 Done Done Sent false
29162 8677 Acked Write 8 Done Done Sent false
29163 8678 Acked Read 4 Done Done Sent false
29164 8679 Acked Flush 0 Done Done Sent false
29165 8680 Acked Write 5 Done Done Sent false
29166 8681 Acked Write 1 Done Done Sent false
29167 8682 Acked Write 2 Done Done Sent false
29168 8683 Acked Write 7 Done Done Sent false
29169 8684 Acked Read 3 Done Done Sent false
29170 8685 Acked Write 3 Done Done Sent false
29171 8686 Acked Write 2 Done Done Sent false
29172 8687 Acked Read 2 Done Done Sent false
29173 8688 Acked Read 9 Done Done Sent false
29174 8689 Acked Write 2 Done Done Sent false
29175 8690 Acked Read 9 Done Done Sent false
29176 8691 Acked Read 10 Done Done Sent false
29177 8692 Acked Write 8 Done Done Sent false
29178 8693 Acked Read 3 Done Done Sent false
29179 8694 Acked Read 9 Done Done Sent false
29180 8695 Acked Write 9 Done Done Sent false
29181 8696 Acked Write 5 Done Done Sent false
29182 8697 Acked Write 4 Done Done Sent false
29183 8698 Acked Read 2 Done Done Sent false
29184 8699 Acked Write 1 Done Done Sent false
29185 8700 Acked Read 9 Done Done Sent false
29186 8701 Acked Write 4 Done Done Sent false
29187 8702 Acked Read 9 Done Done Sent false
29188 8703 Acked Write 8 Done Done Sent false
29189 8704 Acked Flush 0 Done Done Sent false
29190 8705 Acked Read 1 Done Done Sent false
29191 8706 Acked Read 7 Done Done Sent false
29192 8707 Acked Read 1 Done Done Sent false
29193 8708 Acked Read 9 Done Done Sent false
29194 8709 Acked Read 3 Done Done Sent false
29195 8710 Acked Write 6 Done Done Sent false
29196 8711 Acked Read 10 Done Done Sent false
29197 8712 Acked Read 2 Done Done Sent false
29198 8713 Acked Write 8 Done Done Sent false
29199 8714 Acked Write 2 Done Done Sent false
29200 8715 Acked Read 10 Done Done Sent false
29201 8716 Acked Read 6 Done Done Sent false
29202 8717 Acked Read 1 Done Done Sent false
29203 8718 Acked Write 2 Done Done Sent false
29204 8719 Acked Read 1 Done Done Sent false
29205 8720 Acked Write 4 Done Done Sent false
29206 8721 Acked Flush 0 Done Done Sent false
29207 8722 Acked Read 6 Done Done Sent false
29208 8723 Acked Read 8 Done Done Sent false
29209 8724 Acked Flush 0 Done Done Sent false
29210 8725 Acked Write 3 Done Done Sent false
29211 8726 Acked Read 4 Done Done Sent false
29212 8727 Acked Write 3 Done Done Sent false
29213 8728 Acked Write 7 Done Done Sent false
29214 8729 Acked Read 5 Done Done Sent false
29215 8730 Acked Write 7 Done Done Sent false
29216 8731 Acked Write 2 Done Done Sent false
29217 8732 Acked Write 1 Done Done Sent false
29218 8733 Acked Write 1 Done Done Sent false
29219 8734 Acked Read 5 Done Done Sent false
29220 8735 Acked Flush 0 Done Done Sent false
29221 8736 Acked Write 5 Done Done Sent false
29222 8737 Acked Read 4 Done Done Sent false
29223 8738 Acked Write 3 Done Done Sent false
29224 8739 Acked Read 7 Done Done Sent false
29225 8740 Acked Write 3 Done Done Sent false
29226 8741 Acked Flush 0 Done Done Sent false
29227 8742 Acked Write 1 Done Done Sent false
29228 8743 Acked Write 3 Done Done Sent false
29229 8744 Acked Write 2 Done Done Sent false
29230 8745 Acked Read 5 Done Done Sent false
29231 8746 Acked Write 4 Done Done Sent false
29232 8747 Acked Flush 0 Done Done Sent false
29233 8748 Acked Write 6 Done Done Sent false
29234 8749 Acked Write 8 Done Done Sent false
29235 8750 Acked Read 5 Done Done Sent false
29236 8751 Acked Flush 0 Done Done Sent false
29237 8752 Acked Read 2 Done Done Sent false
29238 8753 Acked Flush 0 Done Done Sent false
29239 8754 Acked Read 5 Done Done Sent false
29240 8755 Acked Read 4 Done Done Sent false
29241 8756 Acked Flush 0 Done Done Sent false
29242 8757 Acked Write 4 Done Done Sent false
29243 8758 Acked Flush 0 Done Done Sent false
29244 8759 Acked Read 6 Done Done Sent false
29245 8760 Acked Read 9 Done Done Sent false
29246 8761 Acked Read 1 Done Done Sent false
29247 8762 Acked Read 8 Done Done Sent false
29248 8763 Acked Read 10 Done Done Sent false
29249 8764 Acked Flush 0 Done Done Sent false
29250 8765 Acked Read 2 Done Done Sent false
29251 8766 Acked Flush 0 Done Done Sent false
29252 8767 Acked Write 1 Done Done Sent false
29253 8768 Acked Read 2 Done Done Sent false
29254 8769 Acked Write 7 Done Done Sent false
29255 8770 Acked Write 2 Done Done Sent false
29256 8771 Acked Read 2 Done Done Sent false
29257 8772 Acked Write 10 Done Done Sent false
29258 8773 Acked Read 8 Done Done Sent false
29259 8774 Acked Read 1 Done Done Sent false
29260 8775 Acked Write 4 Done Done Sent false
29261 8776 Acked Write 6 Done Done Sent false
29262 8777 Acked Read 1 Done Done Sent false
29263 8778 Acked Flush 0 Done Done Sent false
29264 8779 Acked Read 10 Done Done Sent false
29265 8780 Acked Read 8 Done Done Sent false
29266 8781 Acked Read 10 Done Done Sent false
29267 8782 Acked Read 1 Done Done Sent false
29268 8783 Acked Read 2 Done Done Sent false
29269 8784 Acked Write 6 Done Done Sent false
29270 8785 Acked Read 1 Done Done Sent false
29271 8786 Acked Write 7 Done Done Sent false
29272 8787 Acked Flush 0 Done Done Sent false
29273 8788 Acked Read 3 Done Done Sent false
29274 8789 Acked Read 4 Done Done Sent false
29275 8790 Acked Read 10 Done Done Sent false
29276 8791 Acked Write 5 Done Done Sent false
29277 8792 Acked Read 2 Done Done Sent false
29278 8793 Acked Read 10 Done Done Sent false
29279 8794 Acked Read 6 Done Done Sent false
29280 8795 Acked Read 2 Done Done Sent false
29281 8796 Acked Write 7 Done Done Sent false
29282 8797 Acked Read 6 Done Done Sent false
29283 8798 Acked Read 7 Done Done Sent false
29284 8799 Acked Write 10 Done Done Sent false
29285 8800 Acked Read 7 Done Done Sent false
29286 8801 Acked Read 7 Done Done Sent false
29287 8802 Acked Write 4 Done Done Sent false
29288 8803 Acked Write 5 Done Done Sent false
29289 8804 Acked Flush 0 Done Done Sent false
29290 8805 Acked Write 8 Done Done Sent false
29291 8806 Acked Write 9 Done Done Sent false
29292 8807 Acked Write 4 Done Done Sent false
29293 8808 Acked Write 9 Done Done Sent false
29294 8809 Acked Read 4 Done Done Sent false
29295 8810 Acked Write 7 Done Done Sent false
29296 8811 Acked Write 1 Done Done Sent false
29297 8812 Acked Write 9 Done Done Sent false
29298 8813 Acked Read 1 Done Done Sent false
29299 8814 Acked Write 9 Done Done Sent false
29300 8815 Acked Write 8 Done Done Sent false
29301 8816 Acked Read 3 Done Done Sent false
29302 8817 Acked Write 2 Done Done Sent false
29303 8818 Acked Write 9 Done Done Sent false
29304 8819 Acked Read 10 Done Done Sent false
29305 8820 Acked Read 3 Done Done Sent false
29306 8821 Acked Read 8 Done Done Sent false
29307 8822 Acked Read 1 Done Done Sent false
29308 8823 Acked Write 5 Done Done Sent false
29309 8824 Acked Read 5 Done Done Sent false
29310 8825 Acked Read 1 Done Done Sent false
29311 8826 Acked Read 6 Done Done Sent false
29312 8827 Acked Read 9 Done Done Sent false
29313 8828 Acked Read 3 Done Done Sent false
29314 8829 Acked Write 3 Done Done Sent false
29315 8830 Acked Read 4 Done Done Sent false
29316 8831 Acked Read 7 Done Done Sent false
29317 8832 Acked Read 6 Done Done Sent false
29318 8833 Acked Write 1 Done Done Sent false
29319 8834 Acked Write 4 Done Done Sent false
29320 8835 Acked Flush 0 Done Done Sent false
29321 8836 Acked Read 2 Done Done Sent false
29322 8837 Acked Flush 0 Done Done Sent false
29323 8838 Acked Flush 0 Done Done Sent false
29324 8839 Acked Read 3 Done Done Sent false
29325 8840 Acked Write 7 Done Done Sent false
29326 8841 Acked Write 7 Done Done Sent false
29327 8842 Acked Read 3 Done Done Sent false
29328 8843 Acked Read 10 Done Done Sent false
29329 8844 Acked Write 2 Done Done Sent false
29330 8845 Acked Read 5 Done Done Sent false
29331 8846 Acked Write 1 Done Done Sent false
29332 8847 Acked Read 8 Done Done Sent false
29333 8848 Acked Flush 0 Done Done Sent false
29334 STATES DS:0 DS:1 DS:2 TOTAL
29335 Sent 0 0 302 302
29336 Done 302 302 0 604
29337 Skipped 0 0 0 0
29338 Error 0 0 0 0
29339 Last Flush: 8848 8848 8546
29340 Downstairs last five completed: 8546 8545 8544 8543 8542
29341 Upstairs last five completed: 8848 8847 8846 8845 8844
29342 CLIENT: Up:0 ds:302 act:2
293432024-11-19T17:14:07.899ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
293442024-11-19T17:14:07.899ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
293452024-11-19T17:14:07.899ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
293462024-11-19T17:14:07.900ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 05650a15-902e-469c-91e5-cba45f07470e
293472024-11-19T17:14:07.900ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
29348 [8830] got start action
29349 [8830] I am going to restart
29350 Starting downstairs at port 8830
29351 Make output file at "/tmp/dsc/downstairs-8830.txt"
29352 [8830][2] reports Starting
29353 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1228)
29354 [8830][2] reports Running
29355 ----------------------------------------------------------------
29356 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29357 States: Active Active Offline
29358 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29359 8547 Acked Write 2 Done Done Sent false
29360 8548 Acked Read 7 Done Done Sent false
29361 8549 Acked Write 6 Done Done Sent false
29362 8550 Acked Write 10 Done Done Sent false
29363 8551 Acked Write 7 Done Done Sent false
29364 8552 Acked Read 5 Done Done Sent false
29365 8553 Acked Write 2 Done Done Sent false
29366 8554 Acked Read 6 Done Done Sent false
29367 8555 Acked Read 6 Done Done Sent false
29368 8556 Acked Write 8 Done Done Sent false
29369 8557 Acked Write 6 Done Done Sent false
29370 8558 Acked Read 2 Done Done Sent false
29371 8559 Acked Write 4 Done Done Sent false
29372 8560 Acked Flush 0 Done Done Sent false
29373 8561 Acked Read 7 Done Done Sent false
29374 8562 Acked Write 6 Done Done Sent false
29375 8563 Acked Flush 0 Done Done Sent false
29376 8564 Acked Write 4 Done Done Sent false
29377 8565 Acked Write 9 Done Done Sent false
29378 8566 Acked Write 8 Done Done Sent false
29379 8567 Acked Flush 0 Done Done Sent false
29380 8568 Acked Flush 0 Done Done Sent false
29381 8569 Acked Read 2 Done Done Sent false
29382 8570 Acked Read 1 Done Done Sent false
29383 8571 Acked Write 6 Done Done Sent false
29384 8572 Acked Read 7 Done Done Sent false
29385 8573 Acked Write 1 Done Done Sent false
29386 8574 Acked Write 3 Done Done Sent false
29387 8575 Acked Write 7 Done Done Sent false
29388 8576 Acked Flush 0 Done Done Sent false
29389 8577 Acked Read 9 Done Done Sent false
29390 8578 Acked Read 8 Done Done Sent false
29391 8579 Acked Write 8 Done Done Sent false
29392 8580 Acked Read 9 Done Done Sent false
29393 8581 Acked Write 4 Done Done Sent false
29394 8582 Acked Read 2 Done Done Sent false
29395 8583 Acked Read 4 Done Done Sent false
29396 8584 Acked Write 5 Done Done Sent false
29397 8585 Acked Flush 0 Done Done Sent false
29398 8586 Acked Write 2 Done Done Sent false
29399 8587 Acked Read 4 Done Done Sent false
29400 8588 Acked Write 6 Done Done Sent false
29401 8589 Acked Read 7 Done Done Sent false
29402 8590 Acked Read 9 Done Done Sent false
29403 8591 Acked Read 2 Done Done Sent false
29404 8592 Acked Write 10 Done Done Sent false
29405 8593 Acked Read 2 Done Done Sent false
29406 8594 Acked Read 1 Done Done Sent false
29407 8595 Acked Write 1 Done Done Sent false
29408 8596 Acked Read 2 Done Done Sent false
29409 8597 Acked Write 3 Done Done Sent false
29410 8598 Acked Read 5 Done Done Sent false
29411 8599 Acked Write 1 Done Done Sent false
29412 8600 Acked Read 6 Done Done Sent false
29413 8601 Acked Read 7 Done Done Sent false
29414 8602 Acked Read 7 Done Done Sent false
29415 8603 Acked Read 6 Done Done Sent false
29416 8604 Acked Read 4 Done Done Sent false
29417 8605 Acked Write 7 Done Done Sent false
29418 8606 Acked Read 1 Done Done Sent false
29419 8607 Acked Write 4 Done Done Sent false
29420 8608 Acked Read 6 Done Done Sent false
29421 8609 Acked Read 6 Done Done Sent false
29422 8610 Acked Flush 0 Done Done Sent false
29423 8611 Acked Write 8 Done Done Sent false
29424 8612 Acked Read 10 Done Done Sent false
29425 8613 Acked Read 1 Done Done Sent false
29426 8614 Acked Flush 0 Done Done Sent false
29427 8615 Acked Write 6 Done Done Sent false
29428 8616 Acked Write 8 Done Done Sent false
29429 8617 Acked Read 5 Done Done Sent false
29430 8618 Acked Read 5 Done Done Sent false
29431 8619 Acked Write 1 Done Done Sent false
29432 8620 Acked Read 4 Done Done Sent false
29433 8621 Acked Read 1 Done Done Sent false
29434 8622 Acked Write 9 Done Done Sent false
29435 8623 Acked Write 7 Done Done Sent false
29436 8624 Acked Read 7 Done Done Sent false
29437 8625 Acked Flush 0 Done Done Sent false
29438 8626 Acked Write 8 Done Done Sent false
29439 8627 Acked Write 1 Done Done Sent false
29440 8628 Acked Read 6 Done Done Sent false
29441 8629 Acked Read 3 Done Done Sent false
29442 8630 Acked Flush 0 Done Done Sent false
29443 8631 Acked Write 6 Done Done Sent false
29444 8632 Acked Read 3 Done Done Sent false
29445 8633 Acked Write 1 Done Done Sent false
29446 8634 Acked Read 7 Done Done Sent false
29447 8635 Acked Read 8 Done Done Sent false
29448 8636 Acked Read 1 Done Done Sent false
29449 8637 Acked Read 10 Done Done Sent false
29450 8638 Acked Read 6 Done Done Sent false
29451 8639 Acked Write 10 Done Done Sent false
29452 8640 Acked Read 7 Done Done Sent false
29453 8641 Acked Flush 0 Done Done Sent false
29454 8642 Acked Read 2 Done Done Sent false
29455 8643 Acked Read 3 Done Done Sent false
29456 8644 Acked Write 4 Done Done Sent false
29457 8645 Acked Write 1 Done Done Sent false
29458 8646 Acked Write 7 Done Done Sent false
29459 8647 Acked Read 3 Done Done Sent false
29460 8648 Acked Read 6 Done Done Sent false
29461 8649 Acked Read 4 Done Done Sent false
29462 8650 Acked Read 10 Done Done Sent false
29463 8651 Acked Write 6 Done Done Sent false
29464 8652 Acked Read 3 Done Done Sent false
29465 8653 Acked Write 8 Done Done Sent false
29466 8654 Acked Write 6 Done Done Sent false
29467 8655 Acked Write 9 Done Done Sent false
29468 8656 Acked Flush 0 Done Done Sent false
29469 8657 Acked Write 6 Done Done Sent false
29470 8658 Acked Read 3 Done Done Sent false
29471 8659 Acked Read 10 Done Done Sent false
29472 8660 Acked Write 9 Done Done Sent false
29473 8661 Acked Write 1 Done Done Sent false
29474 8662 Acked Read 7 Done Done Sent false
29475 8663 Acked Write 3 Done Done Sent false
29476 8664 Acked Read 4 Done Done Sent false
29477 8665 Acked Read 4 Done Done Sent false
29478 8666 Acked Read 4 Done Done Sent false
29479 8667 Acked Write 5 Done Done Sent false
29480 8668 Acked Read 6 Done Done Sent false
29481 8669 Acked Write 9 Done Done Sent false
29482 8670 Acked Read 9 Done Done Sent false
29483 8671 Acked Write 9 Done Done Sent false
29484 8672 Acked Flush 0 Done Done Sent false
29485 8673 Acked Read 7 Done Done Sent false
29486 8674 Acked Write 7 Done Done Sent false
29487 8675 Acked Write 5 Done Done Sent false
29488 8676 Acked Read 3 Done Done Sent false
29489 8677 Acked Write 8 Done Done Sent false
29490 8678 Acked Read 4 Done Done Sent false
29491 8679 Acked Flush 0 Done Done Sent false
29492 8680 Acked Write 5 Done Done Sent false
29493 8681 Acked Write 1 Done Done Sent false
29494 8682 Acked Write 2 Done Done Sent false
29495 8683 Acked Write 7 Done Done Sent false
29496 8684 Acked Read 3 Done Done Sent false
29497 8685 Acked Write 3 Done Done Sent false
29498 8686 Acked Write 2 Done Done Sent false
29499 8687 Acked Read 2 Done Done Sent false
29500 8688 Acked Read 9 Done Done Sent false
29501 8689 Acked Write 2 Done Done Sent false
29502 8690 Acked Read 9 Done Done Sent false
29503 8691 Acked Read 10 Done Done Sent false
29504 8692 Acked Write 8 Done Done Sent false
29505 8693 Acked Read 3 Done Done Sent false
29506 8694 Acked Read 9 Done Done Sent false
29507 8695 Acked Write 9 Done Done Sent false
29508 8696 Acked Write 5 Done Done Sent false
29509 8697 Acked Write 4 Done Done Sent false
29510 8698 Acked Read 2 Done Done Sent false
29511 8699 Acked Write 1 Done Done Sent false
29512 8700 Acked Read 9 Done Done Sent false
29513 8701 Acked Write 4 Done Done Sent false
29514 8702 Acked Read 9 Done Done Sent false
29515 8703 Acked Write 8 Done Done Sent false
29516 8704 Acked Flush 0 Done Done Sent false
29517 8705 Acked Read 1 Done Done Sent false
29518 8706 Acked Read 7 Done Done Sent false
29519 8707 Acked Read 1 Done Done Sent false
29520 8708 Acked Read 9 Done Done Sent false
29521 8709 Acked Read 3 Done Done Sent false
29522 8710 Acked Write 6 Done Done Sent false
29523 8711 Acked Read 10 Done Done Sent false
29524 8712 Acked Read 2 Done Done Sent false
29525 8713 Acked Write 8 Done Done Sent false
29526 8714 Acked Write 2 Done Done Sent false
29527 8715 Acked Read 10 Done Done Sent false
29528 8716 Acked Read 6 Done Done Sent false
29529 8717 Acked Read 1 Done Done Sent false
29530 8718 Acked Write 2 Done Done Sent false
29531 8719 Acked Read 1 Done Done Sent false
29532 8720 Acked Write 4 Done Done Sent false
29533 8721 Acked Flush 0 Done Done Sent false
29534 8722 Acked Read 6 Done Done Sent false
29535 8723 Acked Read 8 Done Done Sent false
29536 8724 Acked Flush 0 Done Done Sent false
29537 8725 Acked Write 3 Done Done Sent false
29538 8726 Acked Read 4 Done Done Sent false
29539 8727 Acked Write 3 Done Done Sent false
29540 8728 Acked Write 7 Done Done Sent false
29541 8729 Acked Read 5 Done Done Sent false
29542 8730 Acked Write 7 Done Done Sent false
29543 8731 Acked Write 2 Done Done Sent false
29544 8732 Acked Write 1 Done Done Sent false
29545 8733 Acked Write 1 Done Done Sent false
29546 8734 Acked Read 5 Done Done Sent false
29547 8735 Acked Flush 0 Done Done Sent false
29548 8736 Acked Write 5 Done Done Sent false
29549 8737 Acked Read 4 Done Done Sent false
29550 8738 Acked Write 3 Done Done Sent false
29551 8739 Acked Read 7 Done Done Sent false
29552 8740 Acked Write 3 Done Done Sent false
29553 8741 Acked Flush 0 Done Done Sent false
29554 8742 Acked Write 1 Done Done Sent false
29555 8743 Acked Write 3 Done Done Sent false
29556 8744 Acked Write 2 Done Done Sent false
29557 8745 Acked Read 5 Done Done Sent false
29558 8746 Acked Write 4 Done Done Sent false
29559 8747 Acked Flush 0 Done Done Sent false
29560 8748 Acked Write 6 Done Done Sent false
29561 8749 Acked Write 8 Done Done Sent false
29562 8750 Acked Read 5 Done Done Sent false
29563 8751 Acked Flush 0 Done Done Sent false
29564 8752 Acked Read 2 Done Done Sent false
29565 8753 Acked Flush 0 Done Done Sent false
29566 8754 Acked Read 5 Done Done Sent false
29567 8755 Acked Read 4 Done Done Sent false
29568 8756 Acked Flush 0 Done Done Sent false
29569 8757 Acked Write 4 Done Done Sent false
29570 8758 Acked Flush 0 Done Done Sent false
29571 8759 Acked Read 6 Done Done Sent false
29572 8760 Acked Read 9 Done Done Sent false
29573 8761 Acked Read 1 Done Done Sent false
29574 8762 Acked Read 8 Done Done Sent false
29575 8763 Acked Read 10 Done Done Sent false
29576 8764 Acked Flush 0 Done Done Sent false
29577 8765 Acked Read 2 Done Done Sent false
29578 8766 Acked Flush 0 Done Done Sent false
29579 8767 Acked Write 1 Done Done Sent false
29580 8768 Acked Read 2 Done Done Sent false
29581 8769 Acked Write 7 Done Done Sent false
29582 8770 Acked Write 2 Done Done Sent false
29583 8771 Acked Read 2 Done Done Sent false
29584 8772 Acked Write 10 Done Done Sent false
29585 8773 Acked Read 8 Done Done Sent false
29586 8774 Acked Read 1 Done Done Sent false
29587 8775 Acked Write 4 Done Done Sent false
29588 8776 Acked Write 6 Done Done Sent false
29589 8777 Acked Read 1 Done Done Sent false
29590 8778 Acked Flush 0 Done Done Sent false
29591 8779 Acked Read 10 Done Done Sent false
29592 8780 Acked Read 8 Done Done Sent false
29593 8781 Acked Read 10 Done Done Sent false
29594 8782 Acked Read 1 Done Done Sent false
29595 8783 Acked Read 2 Done Done Sent false
29596 8784 Acked Write 6 Done Done Sent false
29597 8785 Acked Read 1 Done Done Sent false
29598 8786 Acked Write 7 Done Done Sent false
29599 8787 Acked Flush 0 Done Done Sent false
29600 8788 Acked Read 3 Done Done Sent false
29601 8789 Acked Read 4 Done Done Sent false
29602 8790 Acked Read 10 Done Done Sent false
29603 8791 Acked Write 5 Done Done Sent false
29604 8792 Acked Read 2 Done Done Sent false
29605 8793 Acked Read 10 Done Done Sent false
29606 8794 Acked Read 6 Done Done Sent false
29607 8795 Acked Read 2 Done Done Sent false
29608 8796 Acked Write 7 Done Done Sent false
29609 8797 Acked Read 6 Done Done Sent false
29610 8798 Acked Read 7 Done Done Sent false
29611 8799 Acked Write 10 Done Done Sent false
29612 8800 Acked Read 7 Done Done Sent false
29613 8801 Acked Read 7 Done Done Sent false
29614 8802 Acked Write 4 Done Done Sent false
29615 8803 Acked Write 5 Done Done Sent false
29616 8804 Acked Flush 0 Done Done Sent false
29617 8805 Acked Write 8 Done Done Sent false
29618 8806 Acked Write 9 Done Done Sent false
29619 8807 Acked Write 4 Done Done Sent false
29620 8808 Acked Write 9 Done Done Sent false
29621 8809 Acked Read 4 Done Done Sent false
29622 8810 Acked Write 7 Done Done Sent false
29623 8811 Acked Write 1 Done Done Sent false
29624 8812 Acked Write 9 Done Done Sent false
29625 8813 Acked Read 1 Done Done Sent false
29626 8814 Acked Write 9 Done Done Sent false
29627 8815 Acked Write 8 Done Done Sent false
29628 8816 Acked Read 3 Done Done Sent false
29629 8817 Acked Write 2 Done Done Sent false
29630 8818 Acked Write 9 Done Done Sent false
29631 8819 Acked Read 10 Done Done Sent false
29632 8820 Acked Read 3 Done Done Sent false
29633 8821 Acked Read 8 Done Done Sent false
29634 8822 Acked Read 1 Done Done Sent false
29635 8823 Acked Write 5 Done Done Sent false
29636 8824 Acked Read 5 Done Done Sent false
29637 8825 Acked Read 1 Done Done Sent false
29638 8826 Acked Read 6 Done Done Sent false
29639 8827 Acked Read 9 Done Done Sent false
29640 8828 Acked Read 3 Done Done Sent false
29641 8829 Acked Write 3 Done Done Sent false
29642 8830 Acked Read 4 Done Done Sent false
29643 8831 Acked Read 7 Done Done Sent false
29644 8832 Acked Read 6 Done Done Sent false
29645 8833 Acked Write 1 Done Done Sent false
29646 8834 Acked Write 4 Done Done Sent false
29647 8835 Acked Flush 0 Done Done Sent false
29648 8836 Acked Read 2 Done Done Sent false
29649 8837 Acked Flush 0 Done Done Sent false
29650 8838 Acked Flush 0 Done Done Sent false
29651 8839 Acked Read 3 Done Done Sent false
29652 8840 Acked Write 7 Done Done Sent false
29653 8841 Acked Write 7 Done Done Sent false
29654 8842 Acked Read 3 Done Done Sent false
29655 8843 Acked Read 10 Done Done Sent false
29656 8844 Acked Write 2 Done Done Sent false
29657 8845 Acked Read 5 Done Done Sent false
29658 8846 Acked Write 1 Done Done Sent false
29659 8847 Acked Read 8 Done Done Sent false
29660 8848 Acked Flush 0 Done Done Sent false
29661 STATES DS:0 DS:1 DS:2 TOTAL
29662 Sent 0 0 302 302
29663 Done 302 302 0 604
29664 Skipped 0 0 0 0
29665 Error 0 0 0 0
29666 Last Flush: 8848 8848 8546
29667 Downstairs last five completed: 8546 8545 8544 8543 8542
29668 Upstairs last five completed: 8848 8847 8846 8845 8844
29669 CLIENT: Up:0 ds:302 act:2
29670 ----------------------------------------------------------------
29671 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29672 States: Active Active Offline
29673 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29674 8547 Acked Write 2 Done Done Sent false
29675 8548 Acked Read 7 Done Done Sent false
29676 8549 Acked Write 6 Done Done Sent false
29677 8550 Acked Write 10 Done Done Sent false
29678 8551 Acked Write 7 Done Done Sent false
29679 8552 Acked Read 5 Done Done Sent false
29680 8553 Acked Write 2 Done Done Sent false
29681 8554 Acked Read 6 Done Done Sent false
29682 8555 Acked Read 6 Done Done Sent false
29683 8556 Acked Write 8 Done Done Sent false
29684 8557 Acked Write 6 Done Done Sent false
29685 8558 Acked Read 2 Done Done Sent false
29686 8559 Acked Write 4 Done Done Sent false
29687 8560 Acked Flush 0 Done Done Sent false
29688 8561 Acked Read 7 Done Done Sent false
29689 8562 Acked Write 6 Done Done Sent false
29690 8563 Acked Flush 0 Done Done Sent false
29691 8564 Acked Write 4 Done Done Sent false
29692 8565 Acked Write 9 Done Done Sent false
29693 8566 Acked Write 8 Done Done Sent false
29694 8567 Acked Flush 0 Done Done Sent false
29695 8568 Acked Flush 0 Done Done Sent false
29696 8569 Acked Read 2 Done Done Sent false
29697 8570 Acked Read 1 Done Done Sent false
29698 8571 Acked Write 6 Done Done Sent false
29699 8572 Acked Read 7 Done Done Sent false
29700 8573 Acked Write 1 Done Done Sent false
29701 8574 Acked Write 3 Done Done Sent false
29702 8575 Acked Write 7 Done Done Sent false
29703 8576 Acked Flush 0 Done Done Sent false
29704 8577 Acked Read 9 Done Done Sent false
29705 8578 Acked Read 8 Done Done Sent false
29706 8579 Acked Write 8 Done Done Sent false
29707 8580 Acked Read 9 Done Done Sent false
29708 8581 Acked Write 4 Done Done Sent false
29709 8582 Acked Read 2 Done Done Sent false
29710 8583 Acked Read 4 Done Done Sent false
29711 8584 Acked Write 5 Done Done Sent false
29712 8585 Acked Flush 0 Done Done Sent false
29713 8586 Acked Write 2 Done Done Sent false
29714 8587 Acked Read 4 Done Done Sent false
29715 8588 Acked Write 6 Done Done Sent false
29716 8589 Acked Read 7 Done Done Sent false
29717 8590 Acked Read 9 Done Done Sent false
29718 8591 Acked Read 2 Done Done Sent false
29719 8592 Acked Write 10 Done Done Sent false
29720 8593 Acked Read 2 Done Done Sent false
29721 8594 Acked Read 1 Done Done Sent false
29722 8595 Acked Write 1 Done Done Sent false
29723 8596 Acked Read 2 Done Done Sent false
29724 8597 Acked Write 3 Done Done Sent false
29725 8598 Acked Read 5 Done Done Sent false
29726 8599 Acked Write 1 Done Done Sent false
29727 8600 Acked Read 6 Done Done Sent false
29728 8601 Acked Read 7 Done Done Sent false
29729 8602 Acked Read 7 Done Done Sent false
29730 8603 Acked Read 6 Done Done Sent false
29731 8604 Acked Read 4 Done Done Sent false
29732 8605 Acked Write 7 Done Done Sent false
29733 8606 Acked Read 1 Done Done Sent false
29734 8607 Acked Write 4 Done Done Sent false
29735 8608 Acked Read 6 Done Done Sent false
29736 8609 Acked Read 6 Done Done Sent false
29737 8610 Acked Flush 0 Done Done Sent false
29738 8611 Acked Write 8 Done Done Sent false
29739 8612 Acked Read 10 Done Done Sent false
29740 8613 Acked Read 1 Done Done Sent false
29741 8614 Acked Flush 0 Done Done Sent false
29742 8615 Acked Write 6 Done Done Sent false
29743 8616 Acked Write 8 Done Done Sent false
29744 8617 Acked Read 5 Done Done Sent false
29745 8618 Acked Read 5 Done Done Sent false
29746 8619 Acked Write 1 Done Done Sent false
29747 8620 Acked Read 4 Done Done Sent false
29748 8621 Acked Read 1 Done Done Sent false
29749 8622 Acked Write 9 Done Done Sent false
29750 8623 Acked Write 7 Done Done Sent false
29751 8624 Acked Read 7 Done Done Sent false
29752 8625 Acked Flush 0 Done Done Sent false
29753 8626 Acked Write 8 Done Done Sent false
29754 8627 Acked Write 1 Done Done Sent false
29755 8628 Acked Read 6 Done Done Sent false
29756 8629 Acked Read 3 Done Done Sent false
29757 8630 Acked Flush 0 Done Done Sent false
29758 8631 Acked Write 6 Done Done Sent false
29759 8632 Acked Read 3 Done Done Sent false
29760 8633 Acked Write 1 Done Done Sent false
29761 8634 Acked Read 7 Done Done Sent false
29762 8635 Acked Read 8 Done Done Sent false
29763 8636 Acked Read 1 Done Done Sent false
29764 8637 Acked Read 10 Done Done Sent false
29765 8638 Acked Read 6 Done Done Sent false
29766 8639 Acked Write 10 Done Done Sent false
29767 8640 Acked Read 7 Done Done Sent false
29768 8641 Acked Flush 0 Done Done Sent false
29769 8642 Acked Read 2 Done Done Sent false
29770 8643 Acked Read 3 Done Done Sent false
29771 8644 Acked Write 4 Done Done Sent false
29772 8645 Acked Write 1 Done Done Sent false
29773 8646 Acked Write 7 Done Done Sent false
29774 8647 Acked Read 3 Done Done Sent false
29775 8648 Acked Read 6 Done Done Sent false
29776 8649 Acked Read 4 Done Done Sent false
29777 8650 Acked Read 10 Done Done Sent false
29778 8651 Acked Write 6 Done Done Sent false
29779 8652 Acked Read 3 Done Done Sent false
29780 8653 Acked Write 8 Done Done Sent false
29781 8654 Acked Write 6 Done Done Sent false
29782 8655 Acked Write 9 Done Done Sent false
29783 8656 Acked Flush 0 Done Done Sent false
29784 8657 Acked Write 6 Done Done Sent false
29785 8658 Acked Read 3 Done Done Sent false
29786 8659 Acked Read 10 Done Done Sent false
29787 8660 Acked Write 9 Done Done Sent false
29788 8661 Acked Write 1 Done Done Sent false
29789 8662 Acked Read 7 Done Done Sent false
29790 8663 Acked Write 3 Done Done Sent false
29791 8664 Acked Read 4 Done Done Sent false
29792 8665 Acked Read 4 Done Done Sent false
29793 8666 Acked Read 4 Done Done Sent false
29794 8667 Acked Write 5 Done Done Sent false
29795 8668 Acked Read 6 Done Done Sent false
29796 8669 Acked Write 9 Done Done Sent false
29797 8670 Acked Read 9 Done Done Sent false
29798 8671 Acked Write 9 Done Done Sent false
29799 8672 Acked Flush 0 Done Done Sent false
29800 8673 Acked Read 7 Done Done Sent false
29801 8674 Acked Write 7 Done Done Sent false
29802 8675 Acked Write 5 Done Done Sent false
29803 8676 Acked Read 3 Done Done Sent false
29804 8677 Acked Write 8 Done Done Sent false
29805 8678 Acked Read 4 Done Done Sent false
29806 8679 Acked Flush 0 Done Done Sent false
29807 8680 Acked Write 5 Done Done Sent false
29808 8681 Acked Write 1 Done Done Sent false
29809 8682 Acked Write 2 Done Done Sent false
29810 8683 Acked Write 7 Done Done Sent false
29811 8684 Acked Read 3 Done Done Sent false
29812 8685 Acked Write 3 Done Done Sent false
29813 8686 Acked Write 2 Done Done Sent false
29814 8687 Acked Read 2 Done Done Sent false
29815 8688 Acked Read 9 Done Done Sent false
29816 8689 Acked Write 2 Done Done Sent false
29817 8690 Acked Read 9 Done Done Sent false
29818 8691 Acked Read 10 Done Done Sent false
29819 8692 Acked Write 8 Done Done Sent false
29820 8693 Acked Read 3 Done Done Sent false
29821 8694 Acked Read 9 Done Done Sent false
29822 8695 Acked Write 9 Done Done Sent false
29823 8696 Acked Write 5 Done Done Sent false
29824 8697 Acked Write 4 Done Done Sent false
29825 8698 Acked Read 2 Done Done Sent false
29826 8699 Acked Write 1 Done Done Sent false
29827 8700 Acked Read 9 Done Done Sent false
29828 8701 Acked Write 4 Done Done Sent false
29829 8702 Acked Read 9 Done Done Sent false
29830 8703 Acked Write 8 Done Done Sent false
29831 8704 Acked Flush 0 Done Done Sent false
29832 8705 Acked Read 1 Done Done Sent false
29833 8706 Acked Read 7 Done Done Sent false
29834 8707 Acked Read 1 Done Done Sent false
29835 8708 Acked Read 9 Done Done Sent false
29836 8709 Acked Read 3 Done Done Sent false
29837 8710 Acked Write 6 Done Done Sent false
29838 8711 Acked Read 10 Done Done Sent false
29839 8712 Acked Read 2 Done Done Sent false
29840 8713 Acked Write 8 Done Done Sent false
29841 8714 Acked Write 2 Done Done Sent false
29842 8715 Acked Read 10 Done Done Sent false
29843 8716 Acked Read 6 Done Done Sent false
29844 8717 Acked Read 1 Done Done Sent false
29845 8718 Acked Write 2 Done Done Sent false
29846 8719 Acked Read 1 Done Done Sent false
29847 8720 Acked Write 4 Done Done Sent false
29848 8721 Acked Flush 0 Done Done Sent false
29849 8722 Acked Read 6 Done Done Sent false
29850 8723 Acked Read 8 Done Done Sent false
29851 8724 Acked Flush 0 Done Done Sent false
29852 8725 Acked Write 3 Done Done Sent false
29853 8726 Acked Read 4 Done Done Sent false
29854 8727 Acked Write 3 Done Done Sent false
29855 8728 Acked Write 7 Done Done Sent false
29856 8729 Acked Read 5 Done Done Sent false
29857 8730 Acked Write 7 Done Done Sent false
29858 8731 Acked Write 2 Done Done Sent false
29859 8732 Acked Write 1 Done Done Sent false
29860 8733 Acked Write 1 Done Done Sent false
29861 8734 Acked Read 5 Done Done Sent false
29862 8735 Acked Flush 0 Done Done Sent false
29863 8736 Acked Write 5 Done Done Sent false
29864 8737 Acked Read 4 Done Done Sent false
29865 8738 Acked Write 3 Done Done Sent false
29866 8739 Acked Read 7 Done Done Sent false
29867 8740 Acked Write 3 Done Done Sent false
29868 8741 Acked Flush 0 Done Done Sent false
29869 8742 Acked Write 1 Done Done Sent false
29870 8743 Acked Write 3 Done Done Sent false
29871 8744 Acked Write 2 Done Done Sent false
29872 8745 Acked Read 5 Done Done Sent false
29873 8746 Acked Write 4 Done Done Sent false
29874 8747 Acked Flush 0 Done Done Sent false
29875 8748 Acked Write 6 Done Done Sent false
29876 8749 Acked Write 8 Done Done Sent false
29877 8750 Acked Read 5 Done Done Sent false
29878 8751 Acked Flush 0 Done Done Sent false
29879 8752 Acked Read 2 Done Done Sent false
29880 8753 Acked Flush 0 Done Done Sent false
29881 8754 Acked Read 5 Done Done Sent false
29882 8755 Acked Read 4 Done Done Sent false
29883 8756 Acked Flush 0 Done Done Sent false
29884 8757 Acked Write 4 Done Done Sent false
29885 8758 Acked Flush 0 Done Done Sent false
29886 8759 Acked Read 6 Done Done Sent false
29887 8760 Acked Read 9 Done Done Sent false
29888 8761 Acked Read 1 Done Done Sent false
29889 8762 Acked Read 8 Done Done Sent false
29890 8763 Acked Read 10 Done Done Sent false
29891 8764 Acked Flush 0 Done Done Sent false
29892 8765 Acked Read 2 Done Done Sent false
29893 8766 Acked Flush 0 Done Done Sent false
29894 8767 Acked Write 1 Done Done Sent false
29895 8768 Acked Read 2 Done Done Sent false
29896 8769 Acked Write 7 Done Done Sent false
29897 8770 Acked Write 2 Done Done Sent false
29898 8771 Acked Read 2 Done Done Sent false
29899 8772 Acked Write 10 Done Done Sent false
29900 8773 Acked Read 8 Done Done Sent false
29901 8774 Acked Read 1 Done Done Sent false
29902 8775 Acked Write 4 Done Done Sent false
29903 8776 Acked Write 6 Done Done Sent false
29904 8777 Acked Read 1 Done Done Sent false
29905 8778 Acked Flush 0 Done Done Sent false
29906 8779 Acked Read 10 Done Done Sent false
29907 8780 Acked Read 8 Done Done Sent false
29908 8781 Acked Read 10 Done Done Sent false
29909 8782 Acked Read 1 Done Done Sent false
29910 8783 Acked Read 2 Done Done Sent false
29911 8784 Acked Write 6 Done Done Sent false
29912 8785 Acked Read 1 Done Done Sent false
29913 8786 Acked Write 7 Done Done Sent false
29914 8787 Acked Flush 0 Done Done Sent false
29915 8788 Acked Read 3 Done Done Sent false
29916 8789 Acked Read 4 Done Done Sent false
29917 8790 Acked Read 10 Done Done Sent false
29918 8791 Acked Write 5 Done Done Sent false
29919 8792 Acked Read 2 Done Done Sent false
29920 8793 Acked Read 10 Done Done Sent false
29921 8794 Acked Read 6 Done Done Sent false
29922 8795 Acked Read 2 Done Done Sent false
29923 8796 Acked Write 7 Done Done Sent false
29924 8797 Acked Read 6 Done Done Sent false
29925 8798 Acked Read 7 Done Done Sent false
29926 8799 Acked Write 10 Done Done Sent false
29927 8800 Acked Read 7 Done Done Sent false
29928 8801 Acked Read 7 Done Done Sent false
29929 8802 Acked Write 4 Done Done Sent false
29930 8803 Acked Write 5 Done Done Sent false
29931 8804 Acked Flush 0 Done Done Sent false
29932 8805 Acked Write 8 Done Done Sent false
29933 8806 Acked Write 9 Done Done Sent false
29934 8807 Acked Write 4 Done Done Sent false
29935 8808 Acked Write 9 Done Done Sent false
29936 8809 Acked Read 4 Done Done Sent false
29937 8810 Acked Write 7 Done Done Sent false
29938 8811 Acked Write 1 Done Done Sent false
29939 8812 Acked Write 9 Done Done Sent false
29940 8813 Acked Read 1 Done Done Sent false
29941 8814 Acked Write 9 Done Done Sent false
29942 8815 Acked Write 8 Done Done Sent false
29943 8816 Acked Read 3 Done Done Sent false
29944 8817 Acked Write 2 Done Done Sent false
29945 8818 Acked Write 9 Done Done Sent false
29946 8819 Acked Read 10 Done Done Sent false
29947 8820 Acked Read 3 Done Done Sent false
29948 8821 Acked Read 8 Done Done Sent false
29949 8822 Acked Read 1 Done Done Sent false
29950 8823 Acked Write 5 Done Done Sent false
29951 8824 Acked Read 5 Done Done Sent false
29952 8825 Acked Read 1 Done Done Sent false
29953 8826 Acked Read 6 Done Done Sent false
29954 8827 Acked Read 9 Done Done Sent false
29955 8828 Acked Read 3 Done Done Sent false
29956 8829 Acked Write 3 Done Done Sent false
29957 8830 Acked Read 4 Done Done Sent false
29958 8831 Acked Read 7 Done Done Sent false
29959 8832 Acked Read 6 Done Done Sent false
29960 8833 Acked Write 1 Done Done Sent false
29961 8834 Acked Write 4 Done Done Sent false
29962 8835 Acked Flush 0 Done Done Sent false
29963 8836 Acked Read 2 Done Done Sent false
29964 8837 Acked Flush 0 Done Done Sent false
29965 8838 Acked Flush 0 Done Done Sent false
29966 8839 Acked Read 3 Done Done Sent false
29967 8840 Acked Write 7 Done Done Sent false
29968 8841 Acked Write 7 Done Done Sent false
29969 8842 Acked Read 3 Done Done Sent false
29970 8843 Acked Read 10 Done Done Sent false
29971 8844 Acked Write 2 Done Done Sent false
29972 8845 Acked Read 5 Done Done Sent false
29973 8846 Acked Write 1 Done Done Sent false
29974 8847 Acked Read 8 Done Done Sent false
29975 8848 Acked Flush 0 Done Done Sent false
29976 STATES DS:0 DS:1 DS:2 TOTAL
29977 Sent 0 0 302 302
29978 Done 302 302 0 604
29979 Skipped 0 0 0 0
29980 Error 0 0 0 0
29981 Last Flush: 8848 8848 8546
29982 Downstairs last five completed: 8546 8545 8544 8543 8542
29983 Upstairs last five completed: 8848 8847 8846 8845 8844
29984 CLIENT: Up:0 ds:302 act:2
299852024-11-19T17:14:17.901ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299862024-11-19T17:14:17.901ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58272) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299872024-11-19T17:14:17.902ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299882024-11-19T17:14:17.903ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299892024-11-19T17:14:17.903ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299902024-11-19T17:14:17.903ZINFOcrucible: send last flush ID to this DS: 8546 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299912024-11-19T17:14:17.903ZINFOcrucible: Replied this last flush ID: 8546 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299922024-11-19T17:14:17.903ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299932024-11-19T17:14:17.903ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
299942024-11-19T17:14:17.903ZINFOcrucible: [2] client re-new 302 jobs since flush 8546 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
299952024-11-19T17:14:17.903ZINFOcrucible: [2] Replayed 302 jobs since flush: 8546 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
29996 ----------------------------------------------------------------
29997 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29998 Downstairs last five completed: 8848 8847 8846 8845 8844
29999 Upstairs last five completed: 8848 8847 8846 8845 8844
30000 CLIENT: Up:0 ds:0 act:3
30001 Replay: All jobs finished
30002 Main task has work to do, go find it
30003 got dsc Stop(1)
30004 stop 1
30005 [8820] Got stop action so:false kr:false
30006 Nov 19 17:14:18.140 INFO request completed, latency_us: 316, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 1a4e6123-3638-44f5-875b-e1887d97a6b0, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
300072024-11-19T17:14:18.141ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
300082024-11-19T17:14:18.141ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
300092024-11-19T17:14:18.141ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
300102024-11-19T17:14:18.141ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
300112024-11-19T17:14:18.141ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
300122024-11-19T17:14:18.141ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
30013 Nov 19 17:14:18.141 INFO request completed, latency_us: 146, response_code: 200, uri: /state/cid/1, method: GET, req_id: 59d6df31-e91d-41e6-a233-d789cc423644, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
30014 [8820] Exited with: ExitStatus(unix_wait_status(9))
30015 [8820][1] reports Exit
30016 Nov 19 17:14:22.144 INFO request completed, latency_us: 354, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1e37992d-a383-499b-9bac-25bc49e7a693, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
30017 001/300 Read block 1021 len 36864
30018 002/300 Write block 3308 len 8192 data: 7 7
30019 003/300 Write block 3224 len 4096 data: 6
30020 004/300 Write block 2469 len 20480 data: 5 6 6 6 6
30021 005/300 Read block 3731 len 36864
30022 006/300 Write block 11 len 36864 data: 6 6 4 4 4 4 4 4 2
30023 007/300 Read block 3058 len 40960
30024 008/300 Read block 609 len 12288
30025 009/300 Write block 191 len 36864 data: 6 6 5 5 5 3 4 5 5
30026 010/300 Read block 1350 len 20480
30027 011/300 Read block 697 len 32768
30028 012/300 Flush
30029 013/300 Read block 717 len 40960
30030 014/300 Read block 2726 len 16384
30031 015/300 Read block 414 len 40960
30032 016/300 Read block 4975 len 24576
30033 017/300 Read block 1353 len 36864
30034 018/300 Read block 1377 len 32768
30035 019/300 Read block 3643 len 20480
30036 020/300 Write block 3126 len 16384 data: 15 11 12 11
30037 021/300 Read block 1473 len 20480
30038 022/300 Read block 789 len 8192
30039 023/300 Write block 4447 len 24576 data: 5 3 3 3 3 4
30040 024/300 Read block 4991 len 20480
30041 025/300 Read block 225 len 16384
30042 026/300 Write block 687 len 16384 data: 6 6 5 6
30043 027/300 Read block 3154 len 16384
30044 028/300 Write block 2301 len 8192 data: 10 9
30045 029/300 Flush
30046 030/300 Read block 2351 len 16384
30047 031/300 Read block 4714 len 16384
30048 032/300 Read block 3400 len 12288
30049 033/300 Read block 677 len 20480
30050 034/300 Write block 1347 len 36864 data: 5 6 3 4 5 5 5 5 6
30051 035/300 Read block 4095 len 12288
30052 036/300 Write block 346 len 8192 data: 5 6
30053 037/300 Write block 4019 len 36864 data: 7 8 9 9 9 8 7 6 8
30054 038/300 Write block 272 len 8192 data: 3 2
30055 039/300 Write block 1017 len 28672 data: 2 2 2 5 5 4 5
30056 040/300 Write block 538 len 28672 data: 5 5 5 6 5 5 4
30057 041/300 Write block 31 len 32768 data: 6 7 7 8 8 6 5 4
30058 042/300 Read block 3327 len 20480
30059 043/300 Write block 85 len 24576 data: 3 3 3 4 4 4
30060 044/300 Read block 1674 len 24576
30061 045/300 Write block 561 len 8192 data: 4 5
30062 046/300 Read block 1567 len 16384
30063 047/300 Write block 2434 len 12288 data: 8 5 6
30064 048/300 Write block 304 len 24576 data: 6 6 4 4 4 3
30065 049/300 Flush
30066 050/300 Read block 1536 len 40960
30067 051/300 Write block 3047 len 28672 data: 4 5 7 6 6 7 4
30068 052/300 Write block 2400 len 12288 data: 6 6 5
30069 053/300 Read block 332 len 24576
30070 054/300 Read block 1199 len 28672
30071 055/300 Write block 2777 len 36864 data: 5 6 5 4 4 4 4 4 3
30072 056/300 Write block 906 len 24576 data: 8 8 6 6 4 3
30073 057/300 Read block 3741 len 4096
30074 058/300 Write block 2839 len 36864 data: 4 4 5 5 5 5 5 5 4
30075 059/300 Write block 1836 len 4096 data: 3
30076 060/300 Read block 2067 len 4096
30077 061/300 Write block 2082 len 40960 data: 5 7 6 6 5 5 5 6 7 8
30078 062/300 Read block 127 len 24576
30079 063/300 Read block 2010 len 32768
30080 064/300 Read block 1142 len 36864
30081 065/300 Write block 4744 len 16384 data: 6 6 5 4
30082 066/300 Write block 204 len 28672 data: 6 4 3 4 3 5 5
30083 067/300 Read block 2804 len 16384
30084 068/300 Read block 867 len 20480
30085 069/300 Write block 1814 len 4096 data: 7
30086 070/300 Read block 752 len 12288
30087 071/300 Read block 3814 len 24576
30088 072/300 Flush
30089 073/300 Write block 4812 len 24576 data: 4 4 4 4 2 4
30090 074/300 Flush
30091 075/300 Read block 4107 len 12288
30092 076/300 Write block 1859 len 20480 data: 2 2 3 4 4
30093 077/300 Read block 31 len 4096
30094 078/300 Read block 2047 len 40960
30095 079/300 Read block 4059 len 16384
30096 080/300 Write block 3238 len 24576 data: 4 5 7 9 8 8
30097 081/300 Read block 2862 len 12288
30098 082/300 Write block 2618 len 40960 data: 6 7 8 8 10 7 5 5 4 3
30099 083/300 Read block 2482 len 40960
30100 084/300 Flush
30101 085/300 Read block 1828 len 40960
30102 086/300 Write block 2207 len 28672 data: 5 7 8 8 9 9 11
30103 087/300 Read block 1823 len 8192
30104 088/300 Write block 3435 len 12288 data: 8 9 11
30105 089/300 Write block 2746 len 20480 data: 5 4 4 4 5
30106 090/300 Read block 4084 len 24576
30107 091/300 Write block 3369 len 24576 data: 6 7 7 6 6 7
30108 092/300 Read block 3625 len 40960
30109 093/300 Flush
30110 094/300 Read block 4109 len 4096
30111 095/300 Write block 816 len 24576 data: 6 6 7 7 8 6
30112 096/300 Write block 482 len 4096 data: 4
30113 097/300 Read block 4339 len 36864
30114 098/300 Write block 19 len 28672 data: 3 2 2 2 2 2 3
30115 099/300 Write block 2885 len 40960 data: 4 3 3 2 3 3 4 6 7 6
30116 100/300 Read block 4640 len 20480
30117 101/300 Read block 4784 len 40960
30118 102/300 Read block 1608 len 16384
30119 103/300 Write block 719 len 12288 data: 4 4 4
30120 104/300 Write block 1828 len 28672 data: 7 7 6 3 2 3 3
30121 105/300 Write block 2729 len 12288 data: 7 7 7
30122 106/300 Read block 527 len 36864
30123 107/300 Write block 4684 len 32768 data: 4 4 6 6 6 8 8 7
30124 108/300 Read block 586 len 40960
30125 109/300 Write block 825 len 24576 data: 8 7 6 5 6 7
30126 110/300 Read block 4250 len 32768
30127 111/300 Write block 4111 len 4096 data: 4
30128 112/300 Write block 1016 len 28672 data: 2 3 3 3 6 6 5
30129 113/300 Read block 3602 len 36864
30130 114/300 Read block 4095 len 32768
30131 115/300 Read block 1504 len 32768
30132 116/300 Write block 979 len 40960 data: 4 3 2 2 2 2 2 2 4 5
30133 117/300 Write block 3182 len 20480 data: 6 4 3 3 3
30134 118/300 Read block 69 len 4096
30135 119/300 Read block 3052 len 40960
30136 120/300 Read block 147 len 24576
30137 121/300 Write block 57 len 40960 data: 6 7 8 9 8 8 8 8 6 7
30138 122/300 Read block 4713 len 4096
30139 123/300 Read block 613 len 12288
30140 124/300 Read block 3309 len 20480
30141 125/300 Flush
30142 126/300 Read block 1493 len 40960
30143 127/300 Read block 3776 len 20480
30144 128/300 Read block 4730 len 16384
30145 129/300 Read block 3120 len 28672
30146 130/300 Read block 463 len 24576
30147 131/300 Read block 4101 len 16384
30148 132/300 Read block 4336 len 36864
30149 133/300 Read block 147 len 20480
30150 134/300 Write block 4298 len 24576 data: 5 4 5 4 4 4
30151 135/300 Write block 3496 len 4096 data: 5
30152 136/300 Read block 4859 len 20480
30153 137/300 Read block 4551 len 8192
30154 138/300 Write block 424 len 32768 data: 4 5 7 6 6 6 6 6
30155 139/300 Flush
30156 140/300 Write block 2786 len 4096 data: 3
30157 141/300 Read block 3529 len 40960
30158 142/300 Flush
30159 143/300 Write block 1869 len 12288 data: 8 8 8
30160 144/300 Write block 3902 len 16384 data: 4 4 5 4
30161 145/300 Read block 888 len 40960
30162 146/300 Write block 896 len 12288 data: 5 5 5
30163 147/300 Write block 2952 len 32768 data: 3 3 2 2 2 2 3 2
30164 148/300 Flush
30165 149/300 Flush
30166 150/300 Write block 3787 len 16384 data: 6 6 5 7
30167 151/300 Read block 3996 len 36864
30168 152/300 Write block 2865 len 36864 data: 4 4 6 5 3 4 3 3 3
30169 153/300 Read block 8 len 28672
30170 154/300 Read block 3395 len 12288
30171 155/300 Flush
30172 156/300 Read block 2425 len 8192
30173 157/300 Read block 1520 len 28672
30174 158/300 Read block 3931 len 40960
30175 159/300 Flush
30176 160/300 Read block 240 len 24576
30177 161/300 Read block 2250 len 16384
30178 162/300 Read block 4814 len 40960
30179 163/300 Read block 1912 len 24576
30180 164/300 Write block 2404 len 8192 data: 4 4
30181 165/300 Write block 1298 len 20480 data: 2 3 2 2 3
30182 166/300 Flush
30183 167/300 Write block 2588 len 20480 data: 9 7 7 7 7
30184 168/300 Flush
30185 169/300 Write block 4078 len 36864 data: 3 5 7 7 7 7 7 7 7
30186 170/300 Write block 372 len 28672 data: 3 5 5 4 4 5 5
30187 171/300 Write block 4611 len 4096 data: 3
30188 172/300 Read block 4829 len 16384
30189 173/300 Read block 683 len 8192
30190 174/300 Read block 1966 len 4096
30191 175/300 Write block 81 len 8192 data: 4 4
30192 176/300 Write block 85 len 32768 data: 4 4 4 5 5 5 4 4
30193 177/300 Read block 3652 len 20480
30194 178/300 Read block 374 len 32768
30195 179/300 Write block 1242 len 12288 data: 4 4 5
30196 180/300 Flush
30197 181/300 Read block 4549 len 4096
30198 182/300 Flush
30199 183/300 Flush
30200 184/300 Read block 4175 len 4096
30201 185/300 Read block 4610 len 16384
30202 186/300 Write block 3230 len 32768 data: 5 3 4 4 3 3 3 3
30203 187/300 Write block 733 len 24576 data: 3 4 4 4 4 3
30204 188/300 Write block 4653 len 20480 data: 5 4 3 3 3
30205 189/300 Read block 3564 len 12288
30206 190/300 Flush
30207 191/300 Read block 3592 len 8192
30208 192/300 Write block 4501 len 8192 data: 8 8
30209 193/300 Write block 368 len 36864 data: 6 4 3 3 4 6 6 5 5
30210 194/300 Write block 2477 len 16384 data: 4 4 5 5
30211 195/300 Flush
30212 196/300 Read block 4560 len 32768
30213 197/300 Write block 2272 len 4096 data: 7
30214 198/300 Read block 2323 len 40960
30215 199/300 Flush
30216 200/300 Write block 1555 len 12288 data: 5 5 5
30217 201/300 Read block 2862 len 28672
30218 202/300 Write block 3106 len 20480 data: 2 2 2 2 3
30219 203/300 Read block 4233 len 24576
30220 204/300 Read block 2711 len 8192
30221 205/300 Read block 4421 len 24576
30222 206/300 Read block 3495 len 24576
30223 207/300 Read block 3822 len 4096
30224 208/300 Write block 3781 len 32768 data: 5 5 5 4 5 7 7 7
30225 209/300 Read block 159 len 28672
30226 210/300 Write block 1667 len 20480 data: 5 4 4 4 4
30227 211/300 Read block 1475 len 32768
30228 212/300 Read block 3929 len 40960
30229 213/300 Read block 3463 len 24576
30230 214/300 Write block 563 len 4096 data: 4
30231 215/300 Write block 2886 len 28672 data: 4 4 3 4 4 5 7
30232 216/300 Write block 3599 len 20480 data: 7 6 8 6 5
30233 217/300 Read block 107 len 28672
30234 218/300 Read block 308 len 8192
30235 219/300 Read block 3404 len 8192
30236 220/300 Read block 580 len 40960
30237 221/300 Flush
30238 222/300 Read block 597 len 28672
30239 223/300 Write block 3517 len 20480 data: 3 3 3 4 3
30240 224/300 Write block 2089 len 24576 data: 7 8 9 6 7 5
30241 225/300 Flush
30242 226/300 Write block 2471 len 32768 data: 7 7 7 7 5 4 5 5
30243 227/300 Read block 2797 len 36864
30244 228/300 Write block 3709 len 16384 data: 8 8 8 8
30245 229/300 Write block 236 len 36864 data: 4 5 5 5 4 4 4 4 4
30246 230/300 Read block 2353 len 8192
30247 231/300 Read block 1472 len 36864
30248 232/300 Read block 2560 len 4096
30249 233/300 Read block 985 len 8192
30250 234/300 Read block 3843 len 4096
30251 235/300 Flush
30252 236/300 Read block 2694 len 16384
30253 237/300 Read block 4380 len 4096
30254 238/300 Read block 4138 len 28672
30255 239/300 Read block 4901 len 36864
30256 240/300 Write block 401 len 32768 data: 7 6 7 4 3 3 3 3
30257 241/300 Write block 1513 len 24576 data: 7 5 6 5 5 5
30258 242/300 Read block 769 len 16384
30259 243/300 Write block 2771 len 20480 data: 4 5 5 5 5
30260 244/300 Write block 3239 len 8192 data: 6 8
30261 245/300 Flush
30262 246/300 Write block 4647 len 4096 data: 4
30263 247/300 Flush
30264 248/300 Write block 4623 len 28672 data: 8 7 4 6 5 2 3
30265 249/300 Write block 1743 len 4096 data: 6
30266 250/300 Read block 783 len 40960
30267 251/300 Write block 1562 len 40960 data: 7 7 8 8 10 8 6 5 5 3
30268 252/300 Flush
30269 253/300 Read block 3320 len 28672
30270 254/300 Read block 107 len 16384
30271 255/300 Flush
30272 256/300 Read block 3319 len 20480
30273 257/300 Read block 3095 len 28672
30274 258/300 Write block 4348 len 4096 data: 5
30275 259/300 Write block 4979 len 12288 data: 4 4 2
30276 260/300 Read block 2682 len 28672
30277 261/300 Read block 4533 len 24576
30278 262/300 Write block 2273 len 12288 data: 8 9 8
30279 263/300 Read block 3987 len 40960
30280 264/300 Read block 1463 len 32768
30281 265/300 Write block 2562 len 24576 data: 8 9 9 7 7 7
30282 266/300 Read block 4527 len 32768
30283 267/300 Write block 282 len 12288 data: 5 4 4
30284 268/300 Write block 3355 len 12288 data: 5 4 3
30285 269/300 Write block 1123 len 36864 data: 7 6 6 6 6 5 5 3 4
30286 270/300 Write block 4037 len 40960 data: 5 6 5 4 6 5 6 5 5 6
30287 271/300 Flush
30288 272/300 Read block 3128 len 20480
30289 273/300 Read block 2241 len 28672
30290 274/300 Read block 3626 len 20480
30291 275/300 Write block 4967 len 8192 data: 5 5
30292 276/300 Read block 3449 len 12288
30293 277/300 Read block 4287 len 20480
30294 278/300 Write block 2560 len 40960 data: 10 10 9 10 10 8 8 8 7 7
30295 279/300 Read block 210 len 4096
30296 280/300 Flush
30297 281/300 Write block 2370 len 28672 data: 4 3 3 3 3 4 4
30298 282/300 Write block 3125 len 12288 data: 11 16 12
30299 283/300 Write block 3275 len 24576 data: 8 8 8 7 7 7
30300 284/300 Read block 4308 len 12288
30301 285/300 Read block 239 len 8192
30302 286/300 Read block 1546 len 40960
30303 287/300 Write block 1480 len 16384 data: 7 6 6 3
30304 288/300 Read block 1253 len 20480
30305 289/300 Read block 4478 len 40960
30306 290/300 Write block 99 len 24576 data: 7 9 7 6 5 5
30307 291/300 Write block 1210 len 8192 data: 4 3
30308 292/300 Read block 2158 len 40960
30309 293/300 Write block 3167 len 20480 data: 6 5 4 6 6
30310 294/300 Write block 33 len 8192 data: 8 9
30311 295/300 Flush
30312 296/300 Flush
30313 297/300 Write block 1486 len 8192 data: 4 4
30314 298/300 Read block 4543 len 12288
30315 299/300 Write block 3805 len 28672 data: 4 5 4 4 4 6 7
30316 300/300 Write block 130 len 28672 data: 8 5 7 9 11 11 12
30317 301/300 Write block 2676 len 24576 data: 5 7 8 8 8 8
30318 Main task has work to do, go find it
30319 got dsc Start(1)
30320 start 1
30321 Nov 19 17:14:22.382 INFO request completed, latency_us: 187, response_code: 204, uri: /start/cid/1, method: POST, req_id: 19c7d646-9f94-481b-9daf-ca5e7e017ce4, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
30322 Replay: started 1, returned:Ok(())
30323 ----------------------------------------------------------------
30324 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30325 States: Active Offline Active
30326 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30327 8849 Acked Read 9 Done Sent Done false
30328 8850 Acked Write 2 Done Sent Done false
30329 8851 Acked Write 1 Done Sent Done false
30330 8852 Acked Write 5 Done Sent Done false
30331 8853 Acked Read 9 Done Sent Done false
30332 8854 Acked Write 9 Done Sent Done false
30333 8855 Acked Read 10 Done Sent Done false
30334 8856 Acked Read 3 Done Sent Done false
30335 8857 Acked Write 9 Done Sent Done false
30336 8858 Acked Read 5 Done Sent Done false
30337 8859 Acked Read 8 Done Sent Done false
30338 8860 Acked Flush 0 Done Sent Done false
30339 8861 Acked Read 10 Done Sent Done false
30340 8862 Acked Read 4 Done Sent Done false
30341 8863 Acked Read 10 Done Sent Done false
30342 8864 Acked Read 6 Done Sent Done false
30343 8865 Acked Read 9 Done Sent Done false
30344 8866 Acked Read 8 Done Sent Done false
30345 8867 Acked Read 5 Done Sent Done false
30346 8868 Acked Write 4 Done Sent Done false
30347 8869 Acked Read 5 Done Sent Done false
30348 8870 Acked Read 2 Done Sent Done false
30349 8871 Acked Write 6 Done Sent Done false
30350 8872 Acked Read 5 Done Sent Done false
30351 8873 Acked Read 4 Done Sent Done false
30352 8874 Acked Write 4 Done Sent Done false
30353 8875 Acked Read 4 Done Sent Done false
30354 8876 Acked Write 2 Done Sent Done false
30355 8877 Acked Flush 0 Done Sent Done false
30356 8878 Acked Read 4 Done Sent Done false
30357 8879 Acked Read 4 Done Sent Done false
30358 8880 Acked Read 3 Done Sent Done false
30359 8881 Acked Read 5 Done Sent Done false
30360 8882 Acked Write 9 Done Sent Done false
30361 8883 Acked Read 3 Done Sent Done false
30362 8884 Acked Write 2 Done Sent Done false
30363 8885 Acked Write 9 Done Sent Done false
30364 8886 Acked Write 2 Done Sent Done false
30365 8887 Acked Write 7 Done Sent Done false
30366 8888 Acked Write 7 Done Sent Done false
30367 8889 Acked Write 8 Done Sent Done false
30368 8890 Acked Read 5 Done Sent Done false
30369 8891 Acked Write 6 Done Sent Done false
30370 8892 Acked Read 6 Done Sent Done false
30371 8893 Acked Write 2 Done Sent Done false
30372 8894 Acked Read 4 Done Sent Done false
30373 8895 Acked Write 3 Done Sent Done false
30374 8896 Acked Write 6 Done Sent Done false
30375 8897 Acked Flush 0 Done Sent Done false
30376 8898 Acked Read 10 Done Sent Done false
30377 8899 Acked Write 7 Done Sent Done false
30378 8900 Acked Write 3 Done Sent Done false
30379 8901 Acked Read 6 Done Sent Done false
30380 8902 Acked Read 7 Done Sent Done false
30381 8903 Acked Write 9 Done Sent Done false
30382 8904 Acked Write 6 Done Sent Done false
30383 8905 Acked Read 1 Done Sent Done false
30384 8906 Acked Write 9 Done Sent Done false
30385 8907 Acked Write 1 Done Sent Done false
30386 8908 Acked Read 1 Done Sent Done false
30387 8909 Acked Write 10 Done Sent Done false
30388 8910 Acked Read 6 Done Sent Done false
30389 8911 Acked Read 8 Done Sent Done false
30390 8912 Acked Read 9 Done Sent Done false
30391 8913 Acked Write 4 Done Sent Done false
30392 8914 Acked Write 7 Done Sent Done false
30393 8915 Acked Read 4 Done Sent Done false
30394 8916 Acked Read 5 Done Sent Done false
30395 8917 Acked Write 1 Done Sent Done false
30396 8918 Acked Read 3 Done Sent Done false
30397 8919 Acked Read 6 Done Sent Done false
30398 8920 Acked Flush 0 Done Sent Done false
30399 8921 Acked Write 6 Done Sent Done false
30400 8922 Acked Flush 0 Done Sent Done false
30401 8923 Acked Read 3 Done Sent Done false
30402 8924 Acked Write 5 Done Sent Done false
30403 8925 Acked Read 1 Done Sent Done false
30404 8926 Acked Read 10 Done Sent Done false
30405 8927 Acked Read 4 Done Sent Done false
30406 8928 Acked Write 6 Done Sent Done false
30407 8929 Acked Read 3 Done Sent Done false
30408 8930 Acked Write 10 Done Sent Done false
30409 8931 Acked Read 10 Done Sent Done false
30410 8932 Acked Flush 0 Done Sent Done false
30411 8933 Acked Read 10 Done Sent Done false
30412 8934 Acked Write 7 Done Sent Done false
30413 8935 Acked Read 2 Done Sent Done false
30414 8936 Acked Write 3 Done Sent Done false
30415 8937 Acked Write 5 Done Sent Done false
30416 8938 Acked Read 6 Done Sent Done false
30417 8939 Acked Write 6 Done Sent Done false
30418 8940 Acked Read 10 Done Sent Done false
30419 8941 Acked Flush 0 Done Sent Done false
30420 8942 Acked Read 1 Done Sent Done false
30421 8943 Acked Write 6 Done Sent Done false
30422 8944 Acked Write 1 Done Sent Done false
30423 8945 Acked Read 9 Done Sent Done false
30424 8946 Acked Write 7 Done Sent Done false
30425 8947 Acked Write 10 Done Sent Done false
30426 8948 Acked Read 5 Done Sent Done false
30427 8949 Acked Read 10 Done Sent Done false
30428 8950 Acked Read 4 Done Sent Done false
30429 8951 Acked Write 3 Done Sent Done false
30430 8952 Acked Write 7 Done Sent Done false
30431 8953 Acked Write 3 Done Sent Done false
30432 8954 Acked Read 9 Done Sent Done false
30433 8955 Acked Write 8 Done Sent Done false
30434 8956 Acked Read 10 Done Sent Done false
30435 8957 Acked Write 6 Done Sent Done false
30436 8958 Acked Read 8 Done Sent Done false
30437 8959 Acked Write 1 Done Sent Done false
30438 8960 Acked Write 7 Done Sent Done false
30439 8961 Acked Read 9 Done Sent Done false
30440 8962 Acked Read 8 Done Sent Done false
30441 8963 Acked Read 8 Done Sent Done false
30442 8964 Acked Write 10 Done Sent Done false
30443 8965 Acked Write 5 Done Sent Done false
30444 8966 Acked Read 1 Done Sent Done false
30445 8967 Acked Read 10 Done Sent Done false
30446 8968 Acked Read 6 Done Sent Done false
30447 8969 Acked Write 10 Done Sent Done false
30448 8970 Acked Read 1 Done Sent Done false
30449 8971 Acked Read 3 Done Sent Done false
30450 8972 Acked Read 5 Done Sent Done false
30451 8973 Acked Flush 0 Done Sent Done false
30452 8974 Acked Read 10 Done Sent Done false
30453 8975 Acked Read 5 Done Sent Done false
30454 8976 Acked Read 4 Done Sent Done false
30455 8977 Acked Read 7 Done Sent Done false
30456 8978 Acked Read 6 Done Sent Done false
30457 8979 Acked Read 4 Done Sent Done false
30458 8980 Acked Read 9 Done Sent Done false
30459 8981 Acked Read 5 Done Sent Done false
30460 8982 Acked Write 6 Done Sent Done false
30461 8983 Acked Write 1 Done Sent Done false
30462 8984 Acked Read 5 Done Sent Done false
30463 8985 Acked Read 2 Done Sent Done false
30464 8986 Acked Write 8 Done Sent Done false
30465 8987 Acked Flush 0 Done Sent Done false
30466 8988 Acked Write 1 Done Sent Done false
30467 8989 Acked Read 10 Done Sent Done false
30468 8990 Acked Flush 0 Done Sent Done false
30469 8991 Acked Write 3 Done Sent Done false
30470 8992 Acked Write 4 Done Sent Done false
30471 8993 Acked Read 10 Done Sent Done false
30472 8994 Acked Write 3 Done Sent Done false
30473 8995 Acked Write 8 Done Sent Done false
30474 8996 Acked Flush 0 Done Sent Done false
30475 8997 Acked Flush 0 Done Sent Done false
30476 8998 Acked Write 4 Done Sent Done false
30477 8999 Acked Read 9 Done Sent Done false
30478 9000 Acked Write 9 Done Sent Done false
30479 9001 Acked Read 7 Done Sent Done false
30480 9002 Acked Read 3 Done Sent Done false
30481 9003 Acked Flush 0 Done Sent Done false
30482 9004 Acked Read 2 Done Sent Done false
30483 9005 Acked Read 7 Done Sent Done false
30484 9006 Acked Read 10 Done Sent Done false
30485 9007 Acked Flush 0 Done Sent Done false
30486 9008 Acked Read 6 Done Sent Done false
30487 9009 Acked Read 4 Done Sent Done false
30488 9010 Acked Read 10 Done Sent Done false
30489 9011 Acked Read 6 Done Sent Done false
30490 9012 Acked Write 2 Done Sent Done false
30491 9013 Acked Write 5 Done Sent Done false
30492 9014 Acked Flush 0 Done Sent Done false
30493 9015 Acked Write 5 Done Sent Done false
30494 9016 Acked Flush 0 Done Sent Done false
30495 9017 Acked Write 9 Done Sent Done false
30496 9018 Acked Write 7 Done Sent Done false
30497 9019 Acked Write 1 Done Sent Done false
30498 9020 Acked Read 4 Done Sent Done false
30499 9021 Acked Read 2 Done Sent Done false
30500 9022 Acked Read 1 Done Sent Done false
30501 9023 Acked Write 2 Done Sent Done false
30502 9024 Acked Write 8 Done Sent Done false
30503 9025 Acked Read 5 Done Sent Done false
30504 9026 Acked Read 8 Done Sent Done false
30505 9027 Acked Write 3 Done Sent Done false
30506 9028 Acked Flush 0 Done Sent Done false
30507 9029 Acked Read 1 Done Sent Done false
30508 9030 Acked Flush 0 Done Sent Done false
30509 9031 Acked Flush 0 Done Sent Done false
30510 9032 Acked Read 1 Done Sent Done false
30511 9033 Acked Read 4 Done Sent Done false
30512 9034 Acked Write 8 Done Sent Done false
30513 9035 Acked Write 6 Done Sent Done false
30514 9036 Acked Write 5 Done Sent Done false
30515 9037 Acked Read 3 Done Sent Done false
30516 9038 Acked Flush 0 Done Sent Done false
30517 9039 Acked Read 2 Done Sent Done false
30518 9040 Acked Write 2 Done Sent Done false
30519 9041 Acked Write 9 Done Sent Done false
30520 9042 Acked Write 4 Done Sent Done false
30521 9043 Acked Flush 0 Done Sent Done false
30522 9044 Acked Read 8 Done Sent Done false
30523 9045 Acked Write 1 Done Sent Done false
30524 9046 Acked Read 10 Done Sent Done false
30525 9047 Acked Flush 0 Done Sent Done false
30526 9048 Acked Write 3 Done Sent Done false
30527 9049 Acked Read 7 Done Sent Done false
30528 9050 Acked Write 5 Done Sent Done false
30529 9051 Acked Read 6 Done Sent Done false
30530 9052 Acked Read 2 Done Sent Done false
30531 9053 Acked Read 6 Done Sent Done false
30532 9054 Acked Read 6 Done Sent Done false
30533 9055 Acked Read 1 Done Sent Done false
30534 9056 Acked Write 8 Done Sent Done false
30535 9057 Acked Read 7 Done Sent Done false
30536 9058 Acked Write 5 Done Sent Done false
30537 9059 Acked Read 8 Done Sent Done false
30538 9060 Acked Read 10 Done Sent Done false
30539 9061 Acked Read 6 Done Sent Done false
30540 9062 Acked Write 1 Done Sent Done false
30541 9063 Acked Write 7 Done Sent Done false
30542 9064 Acked Write 5 Done Sent Done false
30543 9065 Acked Read 7 Done Sent Done false
30544 9066 Acked Read 2 Done Sent Done false
30545 9067 Acked Read 2 Done Sent Done false
30546 9068 Acked Read 10 Done Sent Done false
30547 9069 Acked Flush 0 Done Sent Done false
30548 9070 Acked Read 7 Done Sent Done false
30549 9071 Acked Write 5 Done Sent Done false
30550 9072 Acked Write 6 Done Sent Done false
30551 9073 Acked Flush 0 Done Sent Done false
30552 9074 Acked Write 8 Done Sent Done false
30553 9075 Acked Read 9 Done Sent Done false
30554 9076 Acked Write 4 Done Sent Done false
30555 9077 Acked Write 9 Done Sent Done false
30556 9078 Acked Read 2 Done Sent Done false
30557 9079 Acked Read 9 Done Sent Done false
30558 9080 Acked Read 1 Done Sent Done false
30559 9081 Acked Read 2 Done Sent Done false
30560 9082 Acked Read 1 Done Sent Done false
30561 9083 Acked Flush 0 Done Sent Done false
30562 9084 Acked Read 4 Done Sent Done false
30563 9085 Acked Read 1 Done Sent Done false
30564 9086 Acked Read 7 Done Sent Done false
30565 9087 Acked Read 9 Done Sent Done false
30566 9088 Acked Write 8 Done Sent Done false
30567 9089 Acked Write 6 Done Sent Done false
30568 9090 Acked Read 4 Done Sent Done false
30569 9091 Acked Write 5 Done Sent Done false
30570 9092 Acked Write 2 Done Sent Done false
30571 9093 Acked Flush 0 Done Sent Done false
30572 9094 Acked Write 1 Done Sent Done false
30573 9095 Acked Flush 0 Done Sent Done false
30574 9096 Acked Write 7 Done Sent Done false
30575 9097 Acked Write 1 Done Sent Done false
30576 9098 Acked Read 10 Done Sent Done false
30577 9099 Acked Write 10 Done Sent Done false
30578 9100 Acked Flush 0 Done Sent Done false
30579 9101 Acked Read 7 Done Sent Done false
30580 9102 Acked Read 4 Done Sent Done false
30581 9103 Acked Flush 0 Done Sent Done false
30582 9104 Acked Read 5 Done Sent Done false
30583 9105 Acked Read 7 Done Sent Done false
30584 9106 Acked Write 1 Done Sent Done false
30585 9107 Acked Write 3 Done Sent Done false
30586 9108 Acked Read 7 Done Sent Done false
30587 9109 Acked Read 6 Done Sent Done false
30588 9110 Acked Write 3 Done Sent Done false
30589 9111 Acked Read 10 Done Sent Done false
30590 9112 Acked Read 8 Done Sent Done false
30591 9113 Acked Write 6 Done Sent Done false
30592 9114 Acked Read 8 Done Sent Done false
30593 9115 Acked Write 3 Done Sent Done false
30594 9116 Acked Write 3 Done Sent Done false
30595 9117 Acked Write 9 Done Sent Done false
30596 9118 Acked Write 10 Done Sent Done false
30597 9119 Acked Flush 0 Done Sent Done false
30598 9120 Acked Read 5 Done Sent Done false
30599 9121 Acked Read 7 Done Sent Done false
30600 9122 Acked Read 5 Done Sent Done false
30601 9123 Acked Write 2 Done Sent Done false
30602 9124 Acked Read 3 Done Sent Done false
30603 9125 Acked Read 5 Done Sent Done false
30604 9126 Acked Write 10 Done Sent Done false
30605 9127 Acked Read 1 Done Sent Done false
30606 9128 Acked Flush 0 Done Sent Done false
30607 9129 Acked Write 7 Done Sent Done false
30608 9130 Acked Write 3 Done Sent Done false
30609 9131 Acked Write 6 Done Sent Done false
30610 9132 Acked Read 3 Done Sent Done false
30611 9133 Acked Read 2 Done Sent Done false
30612 9134 Acked Read 10 Done Sent Done false
30613 9135 Acked Write 4 Done Sent Done false
30614 9136 Acked Read 5 Done Sent Done false
30615 9137 Acked Read 10 Done Sent Done false
30616 9138 Acked Write 6 Done Sent Done false
30617 9139 Acked Write 2 Done Sent Done false
30618 9140 Acked Read 10 Done Sent Done false
30619 9141 Acked Write 5 Done Sent Done false
30620 9142 Acked Write 2 Done Sent Done false
30621 9143 Acked Flush 0 Done Sent Done false
30622 9144 Acked Flush 0 Done Sent Done false
30623 9145 Acked Write 2 Done Sent Done false
30624 9146 Acked Read 3 Done Sent Done false
30625 9147 Acked Write 7 Done Sent Done false
30626 9148 Acked Write 7 Done Sent Done false
30627 9149 Acked Write 6 Done Sent Done false
30628 STATES DS:0 DS:1 DS:2 TOTAL
30629 Sent 0 301 0 301
30630 Done 301 0 301 602
30631 Skipped 0 0 0 0
30632 Error 0 0 0 0
30633 Last Flush: 9144 8848 9144
30634 Downstairs last five completed: 8848 8847 8846 8845 8844
30635 Upstairs last five completed: 9149 9148 9147 9146 9145
30636 CLIENT: Up:0 ds:301 act:2
30637 [8820] got start action
30638 [8820] I am going to restart
30639 Starting downstairs at port 8820
30640 Make output file at "/tmp/dsc/downstairs-8820.txt"
30641 [8820][1] reports Starting
30642 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1229)
30643 [8820][1] reports Running
30644 ----------------------------------------------------------------
30645 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30646 States: Active Offline Active
30647 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30648 8849 Acked Read 9 Done Sent Done false
30649 8850 Acked Write 2 Done Sent Done false
30650 8851 Acked Write 1 Done Sent Done false
30651 8852 Acked Write 5 Done Sent Done false
30652 8853 Acked Read 9 Done Sent Done false
30653 8854 Acked Write 9 Done Sent Done false
30654 8855 Acked Read 10 Done Sent Done false
30655 8856 Acked Read 3 Done Sent Done false
30656 8857 Acked Write 9 Done Sent Done false
30657 8858 Acked Read 5 Done Sent Done false
30658 8859 Acked Read 8 Done Sent Done false
30659 8860 Acked Flush 0 Done Sent Done false
30660 8861 Acked Read 10 Done Sent Done false
30661 8862 Acked Read 4 Done Sent Done false
30662 8863 Acked Read 10 Done Sent Done false
30663 8864 Acked Read 6 Done Sent Done false
30664 8865 Acked Read 9 Done Sent Done false
30665 8866 Acked Read 8 Done Sent Done false
30666 8867 Acked Read 5 Done Sent Done false
30667 8868 Acked Write 4 Done Sent Done false
30668 8869 Acked Read 5 Done Sent Done false
30669 8870 Acked Read 2 Done Sent Done false
30670 8871 Acked Write 6 Done Sent Done false
30671 8872 Acked Read 5 Done Sent Done false
30672 8873 Acked Read 4 Done Sent Done false
30673 8874 Acked Write 4 Done Sent Done false
30674 8875 Acked Read 4 Done Sent Done false
30675 8876 Acked Write 2 Done Sent Done false
30676 8877 Acked Flush 0 Done Sent Done false
30677 8878 Acked Read 4 Done Sent Done false
30678 8879 Acked Read 4 Done Sent Done false
30679 8880 Acked Read 3 Done Sent Done false
30680 8881 Acked Read 5 Done Sent Done false
30681 8882 Acked Write 9 Done Sent Done false
30682 8883 Acked Read 3 Done Sent Done false
30683 8884 Acked Write 2 Done Sent Done false
30684 8885 Acked Write 9 Done Sent Done false
30685 8886 Acked Write 2 Done Sent Done false
30686 8887 Acked Write 7 Done Sent Done false
30687 8888 Acked Write 7 Done Sent Done false
30688 8889 Acked Write 8 Done Sent Done false
30689 8890 Acked Read 5 Done Sent Done false
30690 8891 Acked Write 6 Done Sent Done false
30691 8892 Acked Read 6 Done Sent Done false
30692 8893 Acked Write 2 Done Sent Done false
30693 8894 Acked Read 4 Done Sent Done false
30694 8895 Acked Write 3 Done Sent Done false
30695 8896 Acked Write 6 Done Sent Done false
30696 8897 Acked Flush 0 Done Sent Done false
30697 8898 Acked Read 10 Done Sent Done false
30698 8899 Acked Write 7 Done Sent Done false
30699 8900 Acked Write 3 Done Sent Done false
30700 8901 Acked Read 6 Done Sent Done false
30701 8902 Acked Read 7 Done Sent Done false
30702 8903 Acked Write 9 Done Sent Done false
30703 8904 Acked Write 6 Done Sent Done false
30704 8905 Acked Read 1 Done Sent Done false
30705 8906 Acked Write 9 Done Sent Done false
30706 8907 Acked Write 1 Done Sent Done false
30707 8908 Acked Read 1 Done Sent Done false
30708 8909 Acked Write 10 Done Sent Done false
30709 8910 Acked Read 6 Done Sent Done false
30710 8911 Acked Read 8 Done Sent Done false
30711 8912 Acked Read 9 Done Sent Done false
30712 8913 Acked Write 4 Done Sent Done false
30713 8914 Acked Write 7 Done Sent Done false
30714 8915 Acked Read 4 Done Sent Done false
30715 8916 Acked Read 5 Done Sent Done false
30716 8917 Acked Write 1 Done Sent Done false
30717 8918 Acked Read 3 Done Sent Done false
30718 8919 Acked Read 6 Done Sent Done false
30719 8920 Acked Flush 0 Done Sent Done false
30720 8921 Acked Write 6 Done Sent Done false
30721 8922 Acked Flush 0 Done Sent Done false
30722 8923 Acked Read 3 Done Sent Done false
30723 8924 Acked Write 5 Done Sent Done false
30724 8925 Acked Read 1 Done Sent Done false
30725 8926 Acked Read 10 Done Sent Done false
30726 8927 Acked Read 4 Done Sent Done false
30727 8928 Acked Write 6 Done Sent Done false
30728 8929 Acked Read 3 Done Sent Done false
30729 8930 Acked Write 10 Done Sent Done false
30730 8931 Acked Read 10 Done Sent Done false
30731 8932 Acked Flush 0 Done Sent Done false
30732 8933 Acked Read 10 Done Sent Done false
30733 8934 Acked Write 7 Done Sent Done false
30734 8935 Acked Read 2 Done Sent Done false
30735 8936 Acked Write 3 Done Sent Done false
30736 8937 Acked Write 5 Done Sent Done false
30737 8938 Acked Read 6 Done Sent Done false
30738 8939 Acked Write 6 Done Sent Done false
30739 8940 Acked Read 10 Done Sent Done false
30740 8941 Acked Flush 0 Done Sent Done false
30741 8942 Acked Read 1 Done Sent Done false
30742 8943 Acked Write 6 Done Sent Done false
30743 8944 Acked Write 1 Done Sent Done false
30744 8945 Acked Read 9 Done Sent Done false
30745 8946 Acked Write 7 Done Sent Done false
30746 8947 Acked Write 10 Done Sent Done false
30747 8948 Acked Read 5 Done Sent Done false
30748 8949 Acked Read 10 Done Sent Done false
30749 8950 Acked Read 4 Done Sent Done false
30750 8951 Acked Write 3 Done Sent Done false
30751 8952 Acked Write 7 Done Sent Done false
30752 8953 Acked Write 3 Done Sent Done false
30753 8954 Acked Read 9 Done Sent Done false
30754 8955 Acked Write 8 Done Sent Done false
30755 8956 Acked Read 10 Done Sent Done false
30756 8957 Acked Write 6 Done Sent Done false
30757 8958 Acked Read 8 Done Sent Done false
30758 8959 Acked Write 1 Done Sent Done false
30759 8960 Acked Write 7 Done Sent Done false
30760 8961 Acked Read 9 Done Sent Done false
30761 8962 Acked Read 8 Done Sent Done false
30762 8963 Acked Read 8 Done Sent Done false
30763 8964 Acked Write 10 Done Sent Done false
30764 8965 Acked Write 5 Done Sent Done false
30765 8966 Acked Read 1 Done Sent Done false
30766 8967 Acked Read 10 Done Sent Done false
30767 8968 Acked Read 6 Done Sent Done false
30768 8969 Acked Write 10 Done Sent Done false
30769 8970 Acked Read 1 Done Sent Done false
30770 8971 Acked Read 3 Done Sent Done false
30771 8972 Acked Read 5 Done Sent Done false
30772 8973 Acked Flush 0 Done Sent Done false
30773 8974 Acked Read 10 Done Sent Done false
30774 8975 Acked Read 5 Done Sent Done false
30775 8976 Acked Read 4 Done Sent Done false
30776 8977 Acked Read 7 Done Sent Done false
30777 8978 Acked Read 6 Done Sent Done false
30778 8979 Acked Read 4 Done Sent Done false
30779 8980 Acked Read 9 Done Sent Done false
30780 8981 Acked Read 5 Done Sent Done false
30781 8982 Acked Write 6 Done Sent Done false
30782 8983 Acked Write 1 Done Sent Done false
30783 8984 Acked Read 5 Done Sent Done false
30784 8985 Acked Read 2 Done Sent Done false
30785 8986 Acked Write 8 Done Sent Done false
30786 8987 Acked Flush 0 Done Sent Done false
30787 8988 Acked Write 1 Done Sent Done false
30788 8989 Acked Read 10 Done Sent Done false
30789 8990 Acked Flush 0 Done Sent Done false
30790 8991 Acked Write 3 Done Sent Done false
30791 8992 Acked Write 4 Done Sent Done false
30792 8993 Acked Read 10 Done Sent Done false
30793 8994 Acked Write 3 Done Sent Done false
30794 8995 Acked Write 8 Done Sent Done false
30795 8996 Acked Flush 0 Done Sent Done false
30796 8997 Acked Flush 0 Done Sent Done false
30797 8998 Acked Write 4 Done Sent Done false
30798 8999 Acked Read 9 Done Sent Done false
30799 9000 Acked Write 9 Done Sent Done false
30800 9001 Acked Read 7 Done Sent Done false
30801 9002 Acked Read 3 Done Sent Done false
30802 9003 Acked Flush 0 Done Sent Done false
30803 9004 Acked Read 2 Done Sent Done false
30804 9005 Acked Read 7 Done Sent Done false
30805 9006 Acked Read 10 Done Sent Done false
30806 9007 Acked Flush 0 Done Sent Done false
30807 9008 Acked Read 6 Done Sent Done false
30808 9009 Acked Read 4 Done Sent Done false
30809 9010 Acked Read 10 Done Sent Done false
30810 9011 Acked Read 6 Done Sent Done false
30811 9012 Acked Write 2 Done Sent Done false
30812 9013 Acked Write 5 Done Sent Done false
30813 9014 Acked Flush 0 Done Sent Done false
30814 9015 Acked Write 5 Done Sent Done false
30815 9016 Acked Flush 0 Done Sent Done false
30816 9017 Acked Write 9 Done Sent Done false
30817 9018 Acked Write 7 Done Sent Done false
30818 9019 Acked Write 1 Done Sent Done false
30819 9020 Acked Read 4 Done Sent Done false
30820 9021 Acked Read 2 Done Sent Done false
30821 9022 Acked Read 1 Done Sent Done false
30822 9023 Acked Write 2 Done Sent Done false
30823 9024 Acked Write 8 Done Sent Done false
30824 9025 Acked Read 5 Done Sent Done false
30825 9026 Acked Read 8 Done Sent Done false
30826 9027 Acked Write 3 Done Sent Done false
30827 9028 Acked Flush 0 Done Sent Done false
30828 9029 Acked Read 1 Done Sent Done false
30829 9030 Acked Flush 0 Done Sent Done false
30830 9031 Acked Flush 0 Done Sent Done false
30831 9032 Acked Read 1 Done Sent Done false
30832 9033 Acked Read 4 Done Sent Done false
30833 9034 Acked Write 8 Done Sent Done false
30834 9035 Acked Write 6 Done Sent Done false
30835 9036 Acked Write 5 Done Sent Done false
30836 9037 Acked Read 3 Done Sent Done false
30837 9038 Acked Flush 0 Done Sent Done false
30838 9039 Acked Read 2 Done Sent Done false
30839 9040 Acked Write 2 Done Sent Done false
30840 9041 Acked Write 9 Done Sent Done false
30841 9042 Acked Write 4 Done Sent Done false
30842 9043 Acked Flush 0 Done Sent Done false
30843 9044 Acked Read 8 Done Sent Done false
30844 9045 Acked Write 1 Done Sent Done false
30845 9046 Acked Read 10 Done Sent Done false
30846 9047 Acked Flush 0 Done Sent Done false
30847 9048 Acked Write 3 Done Sent Done false
30848 9049 Acked Read 7 Done Sent Done false
30849 9050 Acked Write 5 Done Sent Done false
30850 9051 Acked Read 6 Done Sent Done false
30851 9052 Acked Read 2 Done Sent Done false
30852 9053 Acked Read 6 Done Sent Done false
30853 9054 Acked Read 6 Done Sent Done false
30854 9055 Acked Read 1 Done Sent Done false
30855 9056 Acked Write 8 Done Sent Done false
30856 9057 Acked Read 7 Done Sent Done false
30857 9058 Acked Write 5 Done Sent Done false
30858 9059 Acked Read 8 Done Sent Done false
30859 9060 Acked Read 10 Done Sent Done false
30860 9061 Acked Read 6 Done Sent Done false
30861 9062 Acked Write 1 Done Sent Done false
30862 9063 Acked Write 7 Done Sent Done false
30863 9064 Acked Write 5 Done Sent Done false
30864 9065 Acked Read 7 Done Sent Done false
30865 9066 Acked Read 2 Done Sent Done false
30866 9067 Acked Read 2 Done Sent Done false
30867 9068 Acked Read 10 Done Sent Done false
30868 9069 Acked Flush 0 Done Sent Done false
30869 9070 Acked Read 7 Done Sent Done false
30870 9071 Acked Write 5 Done Sent Done false
30871 9072 Acked Write 6 Done Sent Done false
30872 9073 Acked Flush 0 Done Sent Done false
30873 9074 Acked Write 8 Done Sent Done false
30874 9075 Acked Read 9 Done Sent Done false
30875 9076 Acked Write 4 Done Sent Done false
30876 9077 Acked Write 9 Done Sent Done false
30877 9078 Acked Read 2 Done Sent Done false
30878 9079 Acked Read 9 Done Sent Done false
30879 9080 Acked Read 1 Done Sent Done false
30880 9081 Acked Read 2 Done Sent Done false
30881 9082 Acked Read 1 Done Sent Done false
30882 9083 Acked Flush 0 Done Sent Done false
30883 9084 Acked Read 4 Done Sent Done false
30884 9085 Acked Read 1 Done Sent Done false
30885 9086 Acked Read 7 Done Sent Done false
30886 9087 Acked Read 9 Done Sent Done false
30887 9088 Acked Write 8 Done Sent Done false
30888 9089 Acked Write 6 Done Sent Done false
30889 9090 Acked Read 4 Done Sent Done false
30890 9091 Acked Write 5 Done Sent Done false
30891 9092 Acked Write 2 Done Sent Done false
30892 9093 Acked Flush 0 Done Sent Done false
30893 9094 Acked Write 1 Done Sent Done false
30894 9095 Acked Flush 0 Done Sent Done false
30895 9096 Acked Write 7 Done Sent Done false
30896 9097 Acked Write 1 Done Sent Done false
30897 9098 Acked Read 10 Done Sent Done false
30898 9099 Acked Write 10 Done Sent Done false
30899 9100 Acked Flush 0 Done Sent Done false
30900 9101 Acked Read 7 Done Sent Done false
30901 9102 Acked Read 4 Done Sent Done false
30902 9103 Acked Flush 0 Done Sent Done false
30903 9104 Acked Read 5 Done Sent Done false
30904 9105 Acked Read 7 Done Sent Done false
30905 9106 Acked Write 1 Done Sent Done false
30906 9107 Acked Write 3 Done Sent Done false
30907 9108 Acked Read 7 Done Sent Done false
30908 9109 Acked Read 6 Done Sent Done false
30909 9110 Acked Write 3 Done Sent Done false
30910 9111 Acked Read 10 Done Sent Done false
30911 9112 Acked Read 8 Done Sent Done false
30912 9113 Acked Write 6 Done Sent Done false
30913 9114 Acked Read 8 Done Sent Done false
30914 9115 Acked Write 3 Done Sent Done false
30915 9116 Acked Write 3 Done Sent Done false
30916 9117 Acked Write 9 Done Sent Done false
30917 9118 Acked Write 10 Done Sent Done false
30918 9119 Acked Flush 0 Done Sent Done false
30919 9120 Acked Read 5 Done Sent Done false
30920 9121 Acked Read 7 Done Sent Done false
30921 9122 Acked Read 5 Done Sent Done false
30922 9123 Acked Write 2 Done Sent Done false
30923 9124 Acked Read 3 Done Sent Done false
30924 9125 Acked Read 5 Done Sent Done false
30925 9126 Acked Write 10 Done Sent Done false
30926 9127 Acked Read 1 Done Sent Done false
30927 9128 Acked Flush 0 Done Sent Done false
30928 9129 Acked Write 7 Done Sent Done false
30929 9130 Acked Write 3 Done Sent Done false
30930 9131 Acked Write 6 Done Sent Done false
30931 9132 Acked Read 3 Done Sent Done false
30932 9133 Acked Read 2 Done Sent Done false
30933 9134 Acked Read 10 Done Sent Done false
30934 9135 Acked Write 4 Done Sent Done false
30935 9136 Acked Read 5 Done Sent Done false
30936 9137 Acked Read 10 Done Sent Done false
30937 9138 Acked Write 6 Done Sent Done false
30938 9139 Acked Write 2 Done Sent Done false
30939 9140 Acked Read 10 Done Sent Done false
30940 9141 Acked Write 5 Done Sent Done false
30941 9142 Acked Write 2 Done Sent Done false
30942 9143 Acked Flush 0 Done Sent Done false
30943 9144 Acked Flush 0 Done Sent Done false
30944 9145 Acked Write 2 Done Sent Done false
30945 9146 Acked Read 3 Done Sent Done false
30946 9147 Acked Write 7 Done Sent Done false
30947 9148 Acked Write 7 Done Sent Done false
30948 9149 Acked Write 6 Done Sent Done false
30949 9150 Acked Flush 0 Done Sent Done false
30950 STATES DS:0 DS:1 DS:2 TOTAL
30951 Sent 0 302 0 302
30952 Done 302 0 302 604
30953 Skipped 0 0 0 0
30954 Error 0 0 0 0
30955 Last Flush: 9150 8848 9150
30956 Downstairs last five completed: 8848 8847 8846 8845 8844
30957 Upstairs last five completed: 9150 9149 9148 9147 9146
30958 CLIENT: Up:0 ds:302 act:2
309592024-11-19T17:14:28.142ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309602024-11-19T17:14:28.142ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49920) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309612024-11-19T17:14:28.143ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309622024-11-19T17:14:28.143ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309632024-11-19T17:14:28.143ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309642024-11-19T17:14:28.143ZINFOcrucible: send last flush ID to this DS: 8848 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309652024-11-19T17:14:28.144ZINFOcrucible: Replied this last flush ID: 8848 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309662024-11-19T17:14:28.144ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309672024-11-19T17:14:28.144ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309682024-11-19T17:14:28.144ZINFOcrucible: [1] client re-new 302 jobs since flush 8848 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
309692024-11-19T17:14:28.144ZINFOcrucible: [1] Replayed 302 jobs since flush: 8848 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
30970 ----------------------------------------------------------------
30971 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30972 Downstairs last five completed: 9150 9149 9148 9147 9146
30973 Upstairs last five completed: 9150 9149 9148 9147 9146
30974 CLIENT: Up:0 ds:0 act:3
30975 Replay: All jobs finished
30976 Main task has work to do, go find it
30977 got dsc Stop(2)
30978 stop 2
30979 Nov 19 17:14:30.388 INFO request completed, latency_us: 418, response_code: 204, uri: /stop/cid/2, [8830] Got stop action so:false kr:false
30980 method: POST, req_id: c540493f-8843-4355-83f5-934f20e5c7fd, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
309812024-11-19T17:14:30.389ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
30982 Nov 19 17:14:30.389 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/2, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:14:30.389555743Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
30983 method: GET, req_id: a0c3e700-3868-4263-a4ea-b6282ee47c91, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-11-19T17:14:30.38959248Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
30984 9998
309852024-11-19T17:14:30.389ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309862024-11-19T17:14:30.389ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
309872024-11-19T17:14:30.389ZINFOcrucible: client task is exiting = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
30988 [8830] Exited with: ExitStatus(unix_wait_status(9))
30989 [8830][2] reports Exit
30990 Nov 19 17:14:34.391 INFO request completed, latency_us: 229, response_code: 200, uri: /state/cid/2, method: GET, req_id: 21d936d0-da32-4e5b-8ae2-abf0cd8cbab1, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
30991 001/300 Write block 1328 len 24576 data: 5 6 7 8 10 11
30992 002/300 Write block 3845 len 28672 data: 6 6 6 4 4 5 4
30993 003/300 Read block 1503 len 28672
30994 004/300 Flush
30995 005/300 Read block 1752 len 24576
30996 006/300 Write block 878 len 32768 data: 7 5 6 5 5 4 5 6
30997 007/300 Write block 640 len 12288 data: 3 4 5
30998 008/300 Write block 854 len 8192 data: 5 5
30999 009/300 Write block 2867 len 8192 data: 7 6
31000 010/300 Read block 446 len 8192
31001 011/300 Read block 4422 len 36864
31002 012/300 Read block 3800 len 8192
31003 013/300 Write block 4888 len 28672 data: 4 5 7 7 9 10 9
31004 014/300 Read block 2259 len 12288
31005 015/300 Write block 518 len 16384 data: 5 5 5 5
31006 016/300 Write block 4103 len 28672 data: 5 4 3 3 3 3 4
31007 017/300 Read block 1730 len 32768
31008 018/300 Flush
31009 019/300 Read block 3756 len 36864
31010 020/300 Read block 511 len 32768
31011 021/300 Read block 4728 len 40960
31012 022/300 Write block 926 len 12288 data: 2 4 4
31013 023/300 Read block 3856 len 16384
31014 024/300 Read block 273 len 8192
31015 025/300 Read block 3699 len 12288
31016 026/300 Read block 3266 len 36864
31017 027/300 Read block 4564 len 40960
31018 028/300 Write block 2438 len 12288 data: 4 6 6
31019 029/300 Read block 4491 len 36864
31020 030/300 Read block 4883 len 28672
31021 031/300 Read block 4532 len 20480
31022 032/300 Write block 3387 len 12288 data: 8 9 9
31023 033/300 Read block 907 len 32768
31024 034/300 Read block 3470 len 28672
31025 035/300 Read block 1952 len 28672
31026 036/300 Read block 4012 len 8192
31027 037/300 Write block 4058 len 36864 data: 4 4 4 4 5 5 4 4 4
31028 038/300 Read block 1996 len 16384
31029 039/300 Read block 2024 len 40960
31030 040/300 Flush
31031 041/300 Write block 1172 len 16384 data: 4 6 5 6
31032 042/300 Read block 2121 len 36864
31033 043/300 Read block 979 len 40960
31034 044/300 Write block 4493 len 4096 data: 9
31035 045/300 Read block 986 len 16384
31036 046/300 Read block 3235 len 24576
31037 047/300 Write block 1947 len 36864 data: 8 9 9 10 12 9 8 4 3
31038 048/300 Read block 3465 len 16384
31039 049/300 Flush
31040 050/300 Write block 3879 len 32768 data: 6 7 8 7 7 7 5 4
31041 051/300 Read block 908 len 40960
31042 052/300 Write block 2762 len 16384 data: 4 3 5 4
31043 053/300 Write block 4169 len 8192 data: 4 3
31044 054/300 Read block 2518 len 28672
31045 055/300 Read block 4341 len 32768
31046 056/300 Write block 4661 len 40960 data: 7 6 6 5 5 4 4 4 4 5
31047 057/300 Read block 3006 len 4096
31048 058/300 Read block 490 len 12288
31049 059/300 Write block 3588 len 36864 data: 4 5 5 4 5 7 9 9 9
31050 060/300 Read block 3060 len 32768
31051 061/300 Write block 2577 len 32768 data: 6 6 4 3 4 5 5 6
31052 062/300 Write block 924 len 20480 data: 2 2 3 5 5
31053 063/300 Write block 3460 len 8192 data: 5 4
31054 064/300 Read block 3770 len 16384
31055 065/300 Write block 1088 len 8192 data: 6 7
31056 066/300 Read block 4136 len 32768
31057 067/300 Write block 29 len 32768 data: 4 6 7 8 9 10 9 7
31058 068/300 Write block 921 len 8192 data: 5 5
31059 069/300 Read block 557 len 32768
31060 070/300 Read block 343 len 20480
31061 071/300 Read block 4681 len 16384
31062 072/300 Read block 705 len 16384
31063 073/300 Read block 2056 len 28672
31064 074/300 Read block 297 len 24576
31065 075/300 Read block 2990 len 20480
31066 076/300 Flush
31067 077/300 Flush
31068 078/300 Read block 1459 len 12288
31069 079/300 Write block 3489 len 16384 data: 7 7 8 5
31070 080/300 Read block 3534 len 28672
31071 081/300 Write block 3712 len 40960 data: 9 8 9 7 6 5 5 5 5 5
31072 082/300 Flush
31073 083/300 Read block 4679 len 32768
31074 084/300 Read block 2988 len 36864
31075 085/300 Write block 1990 len 12288 data: 4 6 6
31076 086/300 Read block 398 len 12288
31077 087/300 Read block 2986 len 32768
31078 088/300 Write block 2951 len 28672 data: 3 4 4 3 3 3 3
31079 089/300 Write block 272 len 32768 data: 4 3 2 4 5 8 9 9
31080 090/300 Read block 2011 len 16384
31081 091/300 Read block 3049 len 24576
31082 092/300 Read block 16 len 40960
31083 093/300 Write block 18 len 20480 data: 5 4 3 3 3
31084 094/300 Read block 4732 len 16384
31085 095/300 Write block 3961 len 36864 data: 4 5 5 4 4 4 5 4 5
31086 096/300 Write block 4027 len 16384 data: 9 8 6 4
31087 097/300 Read block 1108 len 8192
31088 098/300 Write block 1218 len 16384 data: 4 5 5 4
31089 099/300 Read block 43 len 12288
31090 100/300 Write block 1018 len 28672 data: 4 4 7 7 6 6 5
31091 101/300 Write block 3896 len 16384 data: 8 7 7 5
31092 102/300 Read block 448 len 12288
31093 103/300 Write block 2273 len 24576 data: 9 10 9 10 7 7
31094 104/300 Read block 1884 len 8192
31095 105/300 Read block 1811 len 4096
31096 106/300 Write block 296 len 8192 data: 3 4
31097 107/300 Write block 1212 len 24576 data: 3 2 2 2 3 3
31098 108/300 Read block 115 len 40960
31099 109/300 Write block 4335 len 24576 data: 5 5 4 4 3 3
31100 110/300 Flush
31101 111/300 Read block 2978 len 36864
31102 112/300 Read block 4772 len 40960
31103 113/300 Read block 1528 len 8192
31104 114/300 Read block 4711 len 28672
31105 115/300 Read block 3196 len 36864
31106 116/300 Write block 1105 len 20480 data: 3 3 3 3 4
31107 117/300 Write block 2706 len 36864 data: 11 11 8 10 8 8 7 7 6
31108 118/300 Read block 4349 len 40960
31109 119/300 Write block 2324 len 40960 data: 4 6 7 6 9 9 10 7 6 6
31110 120/300 Read block 4968 len 20480
31111 121/300 Read block 2435 len 20480
31112 122/300 Read block 615 len 8192
31113 123/300 Write block 4363 len 4096 data: 6
31114 124/300 Read block 2410 len 24576
31115 125/300 Write block 538 len 20480 data: 6 6 6 7 6
31116 126/300 Flush
31117 127/300 Write block 2722 len 24576 data: 2 3 4 4 5 7
31118 128/300 Read block 3048 len 24576
31119 129/300 Write block 467 len 28672 data: 2 3 5 6 9 9 10
31120 130/300 Write block 4773 len 4096 data: 5
31121 131/300 Read block 3415 len 36864
31122 132/300 Read block 2971 len 36864
31123 133/300 Read block 3036 len 32768
31124 134/300 Read block 447 len 12288
31125 135/300 Flush
31126 136/300 Read block 917 len 40960
31127 137/300 Flush
31128 138/300 Read block 2641 len 28672
31129 139/300 Read block 4136 len 4096
31130 140/300 Write block 1466 len 24576 data: 5 8 7 7 7 6
31131 141/300 Write block 1619 len 36864 data: 5 5 6 6 6 6 7 5 4
31132 142/300 Read block 2663 len 16384
31133 143/300 Read block 225 len 28672
31134 144/300 Write block 3094 len 40960 data: 2 4 4 5 5 5 6 5 5 3
31135 145/300 Read block 3711 len 32768
31136 146/300 Write block 2823 len 24576 data: 2 3 3 3 4 4
31137 147/300 Write block 3781 len 4096 data: 6
31138 148/300 Write block 1525 len 40960 data: 3 5 5 7 7 6 6 5 6 5
31139 149/300 Write block 3691 len 40960 data: 6 6 5 5 6 6 6 6 6 6
31140 150/300 Write block 4368 len 20480 data: 5 6 6 2 3
31141 151/300 Read block 1348 len 40960
31142 152/300 Write block 546 len 36864 data: 5 5 5 4 4 4 5 4 4
31143 153/300 Read block 1115 len 24576
31144 154/300 Read block 1291 len 36864
31145 155/300 Flush
31146 156/300 Flush
31147 157/300 Write block 2589 len 24576 data: 8 8 8 8 5 5
31148 158/300 Flush
31149 159/300 Read block 3354 len 16384
31150 160/300 Write block 3581 len 28672 data: 4 4 5 5 5 5 4
31151 161/300 Write block 1857 len 8192 data: 2 2
31152 162/300 Write block 2528 len 28672 data: 4 5 7 7 7 6 7
31153 163/300 Read block 330 len 8192
31154 164/300 Read block 4866 len 40960
31155 165/300 Read block 4968 len 24576
31156 166/300 Write block 3858 len 40960 data: 8 7 7 6 8 7 7 8 8 10
31157 167/300 Read block 2858 len 16384
31158 168/300 Read block 2361 len 8192
31159 169/300 Flush
31160 170/300 Write block 1291 len 40960 data: 9 5 4 4 4 3 2 3 4 3
31161 171/300 Read block 1834 len 24576
31162 172/300 Read block 4532 len 8192
31163 173/300 Read block 3369 len 16384
31164 174/300 Write block 342 len 4096 data: 2
31165 175/300 Write block 881 len 12288 data: 6 6 5
31166 176/300 Read block 3702 len 28672
31167 177/300 Read block 2579 len 32768
31168 178/300 Flush
31169 179/300 Read block 2823 len 4096
31170 180/300 Read block 3365 len 36864
31171 181/300 Read block 2825 len 12288
31172 182/300 Write block 4027 len 36864 data: 10 9 7 5 3 5 5 4 3
31173 183/300 Read block 3529 len 20480
31174 184/300 Write block 2785 len 16384 data: 4 4 5 4
31175 185/300 Write block 3467 len 32768 data: 6 4 5 5 5 4 4 4
31176 186/300 Write block 1466 len 36864 data: 6 9 8 8 8 7 6 6 7
31177 187/300 Read block 285 len 16384
31178 188/300 Write block 2230 len 28672 data: 6 7 8 8 8 7 4
31179 189/300 Write block 2579 len 24576 data: 5 4 5 6 6 7
31180 190/300 Write block 2519 len 4096 data: 4
31181 191/300 Read block 404 len 40960
31182 192/300 Read block 1342 len 20480
31183 193/300 Flush
31184 194/300 Read block 481 len 20480
31185 195/300 Write block 3390 len 20480 data: 8 8 6 4 3
31186 196/300 Write block 4708 len 32768 data: 4 5 5 5 5 4 2 2
31187 197/300 Read block 4125 len 24576
31188 198/300 Read block 4140 len 36864
31189 199/300 Read block 2007 len 20480
31190 200/300 Write block 4835 len 12288 data: 3 4 4
31191 201/300 Read block 4196 len 40960
31192 202/300 Write block 2893 len 16384 data: 8 7 7 9
31193 203/300 Read block 3604 len 16384
31194 204/300 Write block 1250 len 12288 data: 5 3 4
31195 205/300 Read block 318 len 4096
31196 206/300 Write block 4890 len 40960 data: 8 8 10 11 10 9 10 9 11 9
31197 207/300 Read block 1320 len 40960
31198 208/300 Read block 3770 len 8192
31199 209/300 Read block 3397 len 8192
31200 210/300 Write block 4173 len 8192 data: 3 3
31201 211/300 Write block 1329 len 12288 data: 7 8 9
31202 212/300 Write block 1194 len 12288 data: 9 9 6
31203 213/300 Flush
31204 214/300 Read block 3426 len 28672
31205 215/300 Write block 2802 len 32768 data: 8 7 4 4 5 7 7 8
31206 216/300 Read block 852 len 20480
31207 217/300 Write block 4620 len 4096 data: 7
31208 218/300 Write block 3847 len 20480 data: 7 5 5 6 5
31209 219/300 Write block 2625 len 12288 data: 6 5 4
31210 220/300 Write block 4887 len 4096 data: 4
31211 221/300 Write block 1762 len 24576 data: 4 6 6 6 6 6
31212 222/300 Read block 2338 len 32768
31213 223/300 Read block 1532 len 28672
31214 224/300 Read block 4885 len 8192
31215 225/300 Write block 4856 len 24576 data: 4 4 4 4 4 5
31216 226/300 Read block 1419 len 36864
31217 227/300 Read block 4808 len 36864
31218 228/300 Read block 3625 len 28672
31219 229/300 Write block 1742 len 4096 data: 6
31220 230/300 Write block 2749 len 16384 data: 5 6 6 6
31221 231/300 Write block 3645 len 24576 data: 2 2 2 3 3 3
31222 232/300 Read block 989 len 4096
31223 233/300 Write block 391 len 24576 data: 8 7 5 7 6 6
31224 234/300 Read block 148 len 20480
31225 235/300 Read block 537 len 36864
31226 236/300 Read block 3816 len 28672
31227 237/300 Read block 4234 len 40960
31228 238/300 Read block 426 len 36864
31229 239/300 Read block 614 len 4096
31230 240/300 Read block 1442 len 12288
31231 241/300 Write block 124 len 40960 data: 7 8 7 9 7 8 9 6 8 10
31232 242/300 Write block 1033 len 28672 data: 5 5 4 5 4 6 5
31233 243/300 Read block 4288 len 8192
31234 244/300 Write block 2144 len 8192 data: 8 7
31235 245/300 Read block 4226 len 8192
31236 246/300 Read block 3789 len 16384
31237 247/300 Read block 1657 len 12288
31238 248/300 Write block 4514 len 12288 data: 6 6 7
31239 249/300 Read block 2492 len 28672
31240 250/300 Write block 1416 len 36864 data: 6 7 8 6 5 8 8 7 5
31241 251/300 Read block 2128 len 28672
31242 252/300 Read block 866 len 8192
31243 253/300 Read block 1750 len 32768
31244 254/300 Read block 2021 len 24576
31245 255/300 Read block 994 len 8192
31246 256/300 Write block 483 len 20480 data: 5 5 6 6 5
31247 257/300 Write block 814 len 4096 data: 5
31248 258/300 Read block 758 len 4096
31249 259/300 Write block 4477 len 8192 data: 5 5
31250 260/300 Read block 3505 len 24576
31251 261/300 Read block 4663 len 8192
31252 262/300 Write block 3726 len 40960 data: 7 7 8 11 10 11 11 8 8 6
31253 263/300 Read block 4228 len 8192
31254 264/300 Flush
31255 265/300 Read block 1379 len 32768
31256 266/300 Write block 3003 len 28672 data: 3 4 4 3 3 3 3
31257 267/300 Write block 64 len 28672 data: 9 7 8 7 7 7 6
31258 268/300 Write block 1430 len 28672 data: 4 2 2 2 2 2 2
31259 269/300 Write block 4668 len 12288 data: 5 5 6
31260 270/300 Flush
31261 271/300 Write block 4861 len 36864 data: 6 4 4 6 6 6 6 6 5
31262 272/300 Read block 430 len 12288
31263 273/300 Read block 3694 len 16384
31264 274/300 Read block 4683 len 12288
31265 275/300 Read block 4880 len 32768
31266 276/300 Write block 3788 len 4096 data: 8
31267 277/300 Write block 2210 len 8192 data: 9 10
31268 278/300 Read block 80 len 4096
31269 279/300 Write block 1736 len 40960 data: 7 8 7 5 4 5 7 7 6 5
31270 280/300 Read block 1258 len 36864
31271 281/300 Write block 3674 len 40960 data: 4 4 3 2 3 3 4 3 4 4
31272 282/300 Write block 3575 len 20480 data: 5 5 2 2 2
31273 283/300 Read block 1806 len 28672
31274 284/300 Read block 4110 len 24576
31275 285/300 Write block 1016 len 16384 data: 3 4 5 5
31276 286/300 Write block 3376 len 40960 data: 4 4 5 5 6 4 3 5 5 6
31277 287/300 Read block 1279 len 4096
31278 288/300 Flush
31279 289/300 Flush
31280 290/300 Read block 4833 len 12288
31281 291/300 Read block 3518 len 16384
31282 292/300 Write block 1783 len 40960 data: 5 5 5 5 7 7 8 6 5 6
31283 293/300 Write block 1410 len 16384 data: 7 7 4 4
31284 294/300 Read block 487 len 24576
31285 295/300 Write block 1205 len 36864 data: 3 4 5 5 5 5 4 4 3
31286 296/300 Read block 3579 len 32768
31287 297/300 Read block 4530 len 12288
31288 298/300 Read block 1267 len 8192
31289 299/300 Read block 3786 len 4096
31290 300/300 Read block 4205 len 12288
31291 301/300 Read block 4716 len 32768
31292 Main task has work to do, go find it
31293 got dsc Start(2)
31294 start 2
31295 Nov 19 17:14:34.641 INFO request completed, latency_us: 182, response_code: 204, uri: /start/cid/2, method: POST, req_id: 38c5df32-e38a-40c7-8139-6a78e3a9ff9a, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
31296 Replay: started 2, returned:Ok(())
31297 ----------------------------------------------------------------
31298 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31299 States: Active Active Offline
31300 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31301 9151 Acked Write 6 Done Done Sent false
31302 9152 Acked Write 7 Done Done Sent false
31303 9153 Acked Read 7 Done Done Sent false
31304 9154 Acked Flush 0 Done Done Sent false
31305 9155 Acked Read 6 Done Done Sent false
31306 9156 Acked Write 8 Done Done Sent false
31307 9157 Acked Write 3 Done Done Sent false
31308 9158 Acked Write 2 Done Done Sent false
31309 9159 Acked Write 2 Done Done Sent false
31310 9160 Acked Read 2 Done Done Sent false
31311 9161 Acked Read 9 Done Done Sent false
31312 9162 Acked Read 2 Done Done Sent false
31313 9163 Acked Write 7 Done Done Sent false
31314 9164 Acked Read 3 Done Done Sent false
31315 9165 Acked Write 4 Done Done Sent false
31316 9166 Acked Write 7 Done Done Sent false
31317 9167 Acked Read 8 Done Done Sent false
31318 9168 Acked Flush 0 Done Done Sent false
31319 9169 Acked Read 9 Done Done Sent false
31320 9170 Acked Read 8 Done Done Sent false
31321 9171 Acked Read 10 Done Done Sent false
31322 9172 Acked Write 3 Done Done Sent false
31323 9173 Acked Read 4 Done Done Sent false
31324 9174 Acked Read 2 Done Done Sent false
31325 9175 Acked Read 3 Done Done Sent false
31326 9176 Acked Read 9 Done Done Sent false
31327 9177 Acked Read 10 Done Done Sent false
31328 9178 Acked Write 3 Done Done Sent false
31329 9179 Acked Read 9 Done Done Sent false
31330 9180 Acked Read 7 Done Done Sent false
31331 9181 Acked Read 5 Done Done Sent false
31332 9182 Acked Write 3 Done Done Sent false
31333 9183 Acked Read 8 Done Done Sent false
31334 9184 Acked Read 7 Done Done Sent false
31335 9185 Acked Read 7 Done Done Sent false
31336 9186 Acked Read 2 Done Done Sent false
31337 9187 Acked Write 9 Done Done Sent false
31338 9188 Acked Read 4 Done Done Sent false
31339 9189 Acked Read 10 Done Done Sent false
31340 9190 Acked Flush 0 Done Done Sent false
31341 9191 Acked Write 4 Done Done Sent false
31342 9192 Acked Read 9 Done Done Sent false
31343 9193 Acked Read 10 Done Done Sent false
31344 9194 Acked Write 1 Done Done Sent false
31345 9195 Acked Read 4 Done Done Sent false
31346 9196 Acked Read 6 Done Done Sent false
31347 9197 Acked Write 9 Done Done Sent false
31348 9198 Acked Read 4 Done Done Sent false
31349 9199 Acked Flush 0 Done Done Sent false
31350 9200 Acked Write 8 Done Done Sent false
31351 9201 Acked Read 10 Done Done Sent false
31352 9202 Acked Write 4 Done Done Sent false
31353 9203 Acked Write 2 Done Done Sent false
31354 9204 Acked Read 7 Done Done Sent false
31355 9205 Acked Read 8 Done Done Sent false
31356 9206 Acked Write 10 Done Done Sent false
31357 9207 Acked Read 1 Done Done Sent false
31358 9208 Acked Read 3 Done Done Sent false
31359 9209 Acked Write 9 Done Done Sent false
31360 9210 Acked Read 8 Done Done Sent false
31361 9211 Acked Write 8 Done Done Sent false
31362 9212 Acked Write 5 Done Done Sent false
31363 9213 Acked Write 2 Done Done Sent false
31364 9214 Acked Read 4 Done Done Sent false
31365 9215 Acked Write 2 Done Done Sent false
31366 9216 Acked Read 8 Done Done Sent false
31367 9217 Acked Write 8 Done Done Sent false
31368 9218 Acked Write 2 Done Done Sent false
31369 9219 Acked Read 8 Done Done Sent false
31370 9220 Acked Read 5 Done Done Sent false
31371 9221 Acked Read 4 Done Done Sent false
31372 9222 Acked Read 4 Done Done Sent false
31373 9223 Acked Read 7 Done Done Sent false
31374 9224 Acked Read 6 Done Done Sent false
31375 9225 Acked Read 5 Done Done Sent false
31376 9226 Acked Flush 0 Done Done Sent false
31377 9227 Acked Flush 0 Done Done Sent false
31378 9228 Acked Read 3 Done Done Sent false
31379 9229 Acked Write 4 Done Done Sent false
31380 9230 Acked Read 7 Done Done Sent false
31381 9231 Acked Write 10 Done Done Sent false
31382 9232 Acked Flush 0 Done Done Sent false
31383 9233 Acked Read 8 Done Done Sent false
31384 9234 Acked Read 9 Done Done Sent false
31385 9235 Acked Write 3 Done Done Sent false
31386 9236 Acked Read 3 Done Done Sent false
31387 9237 Acked Read 8 Done Done Sent false
31388 9238 Acked Write 7 Done Done Sent false
31389 9239 Acked Write 8 Done Done Sent false
31390 9240 Acked Read 4 Done Done Sent false
31391 9241 Acked Read 6 Done Done Sent false
31392 9242 Acked Read 10 Done Done Sent false
31393 9243 Acked Write 5 Done Done Sent false
31394 9244 Acked Read 4 Done Done Sent false
31395 9245 Acked Write 9 Done Done Sent false
31396 9246 Acked Write 4 Done Done Sent false
31397 9247 Acked Read 2 Done Done Sent false
31398 9248 Acked Write 4 Done Done Sent false
31399 9249 Acked Read 3 Done Done Sent false
31400 9250 Acked Write 7 Done Done Sent false
31401 9251 Acked Write 4 Done Done Sent false
31402 9252 Acked Read 3 Done Done Sent false
31403 9253 Acked Write 6 Done Done Sent false
31404 9254 Acked Read 2 Done Done Sent false
31405 9255 Acked Read 1 Done Done Sent false
31406 9256 Acked Write 2 Done Done Sent false
31407 9257 Acked Write 6 Done Done Sent false
31408 9258 Acked Read 10 Done Done Sent false
31409 9259 Acked Write 6 Done Done Sent false
31410 9260 Acked Flush 0 Done Done Sent false
31411 9261 Acked Read 9 Done Done Sent false
31412 9262 Acked Read 10 Done Done Sent false
31413 9263 Acked Read 2 Done Done Sent false
31414 9264 Acked Read 7 Done Done Sent false
31415 9265 Acked Read 9 Done Done Sent false
31416 9266 Acked Write 5 Done Done Sent false
31417 9267 Acked Write 9 Done Done Sent false
31418 9268 Acked Read 10 Done Done Sent false
31419 9269 Acked Write 10 Done Done Sent false
31420 9270 Acked Read 5 Done Done Sent false
31421 9271 Acked Read 5 Done Done Sent false
31422 9272 Acked Read 2 Done Done Sent false
31423 9273 Acked Write 1 Done Done Sent false
31424 9274 Acked Read 6 Done Done Sent false
31425 9275 Acked Write 5 Done Done Sent false
31426 9276 Acked Flush 0 Done Done Sent false
31427 9277 Acked Write 6 Done Done Sent false
31428 9278 Acked Read 6 Done Done Sent false
31429 9279 Acked Write 7 Done Done Sent false
31430 9280 Acked Write 1 Done Done Sent false
31431 9281 Acked Read 9 Done Done Sent false
31432 9282 Acked Read 9 Done Done Sent false
31433 9283 Acked Read 8 Done Done Sent false
31434 9284 Acked Read 3 Done Done Sent false
31435 9285 Acked Flush 0 Done Done Sent false
31436 9286 Acked Read 10 Done Done Sent false
31437 9287 Acked Flush 0 Done Done Sent false
31438 9288 Acked Read 7 Done Done Sent false
31439 9289 Acked Read 1 Done Done Sent false
31440 9290 Acked Write 6 Done Done Sent false
31441 9291 Acked Write 9 Done Done Sent false
31442 9292 Acked Read 4 Done Done Sent false
31443 9293 Acked Read 7 Done Done Sent false
31444 9294 Acked Write 10 Done Done Sent false
31445 9295 Acked Read 8 Done Done Sent false
31446 9296 Acked Write 6 Done Done Sent false
31447 9297 Acked Write 1 Done Done Sent false
31448 9298 Acked Write 10 Done Done Sent false
31449 9299 Acked Write 10 Done Done Sent false
31450 9300 Acked Write 5 Done Done Sent false
31451 9301 Acked Read 10 Done Done Sent false
31452 9302 Acked Write 9 Done Done Sent false
31453 9303 Acked Read 6 Done Done Sent false
31454 9304 Acked Read 9 Done Done Sent false
31455 9305 Acked Flush 0 Done Done Sent false
31456 9306 Acked Flush 0 Done Done Sent false
31457 9307 Acked Write 6 Done Done Sent false
31458 9308 Acked Flush 0 Done Done Sent false
31459 9309 Acked Read 4 Done Done Sent false
31460 9310 Acked Write 7 Done Done Sent false
31461 9311 Acked Write 2 Done Done Sent false
31462 9312 Acked Write 7 Done Done Sent false
31463 9313 Acked Read 2 Done Done Sent false
31464 9314 Acked Read 10 Done Done Sent false
31465 9315 Acked Read 6 Done Done Sent false
31466 9316 Acked Write 10 Done Done Sent false
31467 9317 Acked Read 4 Done Done Sent false
31468 9318 Acked Read 2 Done Done Sent false
31469 9319 Acked Flush 0 Done Done Sent false
31470 9320 Acked Write 10 Done Done Sent false
31471 9321 Acked Read 6 Done Done Sent false
31472 9322 Acked Read 2 Done Done Sent false
31473 9323 Acked Read 4 Done Done Sent false
31474 9324 Acked Write 1 Done Done Sent false
31475 9325 Acked Write 3 Done Done Sent false
31476 9326 Acked Read 7 Done Done Sent false
31477 9327 Acked Read 8 Done Done Sent false
31478 9328 Acked Flush 0 Done Done Sent false
31479 9329 Acked Read 1 Done Done Sent false
31480 9330 Acked Read 9 Done Done Sent false
31481 9331 Acked Read 3 Done Done Sent false
31482 9332 Acked Write 9 Done Done Sent false
31483 9333 Acked Read 5 Done Done Sent false
31484 9334 Acked Write 4 Done Done Sent false
31485 9335 Acked Write 8 Done Done Sent false
31486 9336 Acked Write 9 Done Done Sent false
31487 9337 Acked Read 4 Done Done Sent false
31488 9338 Acked Write 7 Done Done Sent false
31489 9339 Acked Write 6 Done Done Sent false
31490 9340 Acked Write 1 Done Done Sent false
31491 9341 Acked Read 10 Done Done Sent false
31492 9342 Acked Read 5 Done Done Sent false
31493 9343 Acked Flush 0 Done Done Sent false
31494 9344 Acked Read 5 Done Done Sent false
31495 9345 Acked Write 5 Done Done Sent false
31496 9346 Acked Write 8 Done Done Sent false
31497 9347 Acked Read 6 Done Done Sent false
31498 9348 Acked Read 9 Done Done Sent false
31499 9349 Acked Read 5 Done Done Sent false
31500 9350 Acked Write 3 Done Done Sent false
31501 9351 Acked Read 10 Done Done Sent false
31502 9352 Acked Write 4 Done Done Sent false
31503 9353 Acked Read 4 Done Done Sent false
31504 9354 Acked Write 3 Done Done Sent false
31505 9355 Acked Read 1 Done Done Sent false
31506 9356 Acked Write 10 Done Done Sent false
31507 9357 Acked Read 10 Done Done Sent false
31508 9358 Acked Read 2 Done Done Sent false
31509 9359 Acked Read 2 Done Done Sent false
31510 9360 Acked Write 2 Done Done Sent false
31511 9361 Acked Write 3 Done Done Sent false
31512 9362 Acked Write 3 Done Done Sent false
31513 9363 Acked Flush 0 Done Done Sent false
31514 9364 Acked Read 7 Done Done Sent false
31515 9365 Acked Write 8 Done Done Sent false
31516 9366 Acked Read 5 Done Done Sent false
31517 9367 Acked Write 1 Done Done Sent false
31518 9368 Acked Write 5 Done Done Sent false
31519 9369 Acked Write 3 Done Done Sent false
31520 9370 Acked Write 1 Done Done Sent false
31521 9371 Acked Write 6 Done Done Sent false
31522 9372 Acked Read 8 Done Done Sent false
31523 9373 Acked Read 7 Done Done Sent false
31524 9374 Acked Read 2 Done Done Sent false
31525 9375 Acked Write 6 Done Done Sent false
31526 9376 Acked Read 9 Done Done Sent false
31527 9377 Acked Read 9 Done Done Sent false
31528 9378 Acked Read 7 Done Done Sent false
31529 9379 Acked Write 1 Done Done Sent false
31530 9380 Acked Write 4 Done Done Sent false
31531 9381 Acked Write 6 Done Done Sent false
31532 9382 Acked Read 1 Done Done Sent false
31533 9383 Acked Write 6 Done Done Sent false
31534 9384 Acked Read 5 Done Done Sent false
31535 9385 Acked Read 9 Done Done Sent false
31536 9386 Acked Read 7 Done Done Sent false
31537 9387 Acked Read 10 Done Done Sent false
31538 9388 Acked Read 9 Done Done Sent false
31539 9389 Acked Read 1 Done Done Sent false
31540 9390 Acked Read 3 Done Done Sent false
31541 9391 Acked Write 10 Done Done Sent false
31542 9392 Acked Write 7 Done Done Sent false
31543 9393 Acked Read 2 Done Done Sent false
31544 9394 Acked Write 2 Done Done Sent false
31545 9395 Acked Read 2 Done Done Sent false
31546 9396 Acked Read 4 Done Done Sent false
31547 9397 Acked Read 3 Done Done Sent false
31548 9398 Acked Write 3 Done Done Sent false
31549 9399 Acked Read 7 Done Done Sent false
31550 9400 Acked Write 9 Done Done Sent false
31551 9401 Acked Read 7 Done Done Sent false
31552 9402 Acked Read 2 Done Done Sent false
31553 9403 Acked Read 8 Done Done Sent false
31554 9404 Acked Read 6 Done Done Sent false
31555 9405 Acked Read 2 Done Done Sent false
31556 9406 Acked Write 5 Done Done Sent false
31557 9407 Acked Write 1 Done Done Sent false
31558 9408 Acked Read 1 Done Done Sent false
31559 9409 Acked Write 2 Done Done Sent false
31560 9410 Acked Read 6 Done Done Sent false
31561 9411 Acked Read 2 Done Done Sent false
31562 9412 Acked Write 10 Done Done Sent false
31563 9413 Acked Read 2 Done Done Sent false
31564 9414 Acked Flush 0 Done Done Sent false
31565 9415 Acked Read 8 Done Done Sent false
31566 9416 Acked Write 7 Done Done Sent false
31567 9417 Acked Write 7 Done Done Sent false
31568 9418 Acked Write 7 Done Done Sent false
31569 9419 Acked Write 3 Done Done Sent false
31570 9420 Acked Flush 0 Done Done Sent false
31571 9421 Acked Write 9 Done Done Sent false
31572 9422 Acked Read 3 Done Done Sent false
31573 9423 Acked Read 4 Done Done Sent false
31574 9424 Acked Read 3 Done Done Sent false
31575 9425 Acked Read 8 Done Done Sent false
31576 9426 Acked Write 1 Done Done Sent false
31577 9427 Acked Write 2 Done Done Sent false
31578 9428 Acked Read 1 Done Done Sent false
31579 9429 Acked Write 10 Done Done Sent false
31580 9430 Acked Read 9 Done Done Sent false
31581 9431 Acked Write 10 Done Done Sent false
31582 9432 Acked Write 5 Done Done Sent false
31583 9433 Acked Read 7 Done Done Sent false
31584 9434 Acked Read 6 Done Done Sent false
31585 9435 Acked Write 4 Done Done Sent false
31586 9436 Acked Write 10 Done Done Sent false
31587 9437 Acked Read 1 Done Done Sent false
31588 9438 Acked Flush 0 Done Done Sent false
31589 9439 Acked Flush 0 Done Done Sent false
31590 9440 Acked Read 3 Done Done Sent false
31591 9441 Acked Read 4 Done Done Sent false
31592 9442 Acked Write 10 Done Done Sent false
31593 9443 Acked Write 4 Done Done Sent false
31594 9444 Acked Read 6 Done Done Sent false
31595 9445 Acked Write 9 Done Done Sent false
31596 9446 Acked Read 8 Done Done Sent false
31597 9447 Acked Read 3 Done Done Sent false
31598 9448 Acked Read 2 Done Done Sent false
31599 9449 Acked Read 1 Done Done Sent false
31600 9450 Acked Read 3 Done Done Sent false
31601 9451 Acked Read 8 Done Done Sent false
31602 STATES DS:0 DS:1 DS:2 TOTAL
31603 Sent 0 0 301 301
31604 Done 301 301 0 602
31605 Skipped 0 0 0 0
31606 Error 0 0 0 0
31607 Last Flush: 9439 9439 9150
31608 Downstairs last five completed: 9150 9149 9148 9147 9146
31609 Upstairs last five completed: 9451 9450 9449 9448 9447
31610 CLIENT: Up:0 ds:301 act:2
31611 [8830] got start action
31612 [8830] I am going to restart
31613 Starting downstairs at port 8830
31614 Make output file at "/tmp/dsc/downstairs-8830.txt"
31615 [8830][2] reports Starting
31616 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1230)
31617 [8830][2] reports Running
31618 ----------------------------------------------------------------
31619 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31620 States: Active Active Offline
31621 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31622 9151 Acked Write 6 Done Done Sent false
31623 9152 Acked Write 7 Done Done Sent false
31624 9153 Acked Read 7 Done Done Sent false
31625 9154 Acked Flush 0 Done Done Sent false
31626 9155 Acked Read 6 Done Done Sent false
31627 9156 Acked Write 8 Done Done Sent false
31628 9157 Acked Write 3 Done Done Sent false
31629 9158 Acked Write 2 Done Done Sent false
31630 9159 Acked Write 2 Done Done Sent false
31631 9160 Acked Read 2 Done Done Sent false
31632 9161 Acked Read 9 Done Done Sent false
31633 9162 Acked Read 2 Done Done Sent false
31634 9163 Acked Write 7 Done Done Sent false
31635 9164 Acked Read 3 Done Done Sent false
31636 9165 Acked Write 4 Done Done Sent false
31637 9166 Acked Write 7 Done Done Sent false
31638 9167 Acked Read 8 Done Done Sent false
31639 9168 Acked Flush 0 Done Done Sent false
31640 9169 Acked Read 9 Done Done Sent false
31641 9170 Acked Read 8 Done Done Sent false
31642 9171 Acked Read 10 Done Done Sent false
31643 9172 Acked Write 3 Done Done Sent false
31644 9173 Acked Read 4 Done Done Sent false
31645 9174 Acked Read 2 Done Done Sent false
31646 9175 Acked Read 3 Done Done Sent false
31647 9176 Acked Read 9 Done Done Sent false
31648 9177 Acked Read 10 Done Done Sent false
31649 9178 Acked Write 3 Done Done Sent false
31650 9179 Acked Read 9 Done Done Sent false
31651 9180 Acked Read 7 Done Done Sent false
31652 9181 Acked Read 5 Done Done Sent false
31653 9182 Acked Write 3 Done Done Sent false
31654 9183 Acked Read 8 Done Done Sent false
31655 9184 Acked Read 7 Done Done Sent false
31656 9185 Acked Read 7 Done Done Sent false
31657 9186 Acked Read 2 Done Done Sent false
31658 9187 Acked Write 9 Done Done Sent false
31659 9188 Acked Read 4 Done Done Sent false
31660 9189 Acked Read 10 Done Done Sent false
31661 9190 Acked Flush 0 Done Done Sent false
31662 9191 Acked Write 4 Done Done Sent false
31663 9192 Acked Read 9 Done Done Sent false
31664 9193 Acked Read 10 Done Done Sent false
31665 9194 Acked Write 1 Done Done Sent false
31666 9195 Acked Read 4 Done Done Sent false
31667 9196 Acked Read 6 Done Done Sent false
31668 9197 Acked Write 9 Done Done Sent false
31669 9198 Acked Read 4 Done Done Sent false
31670 9199 Acked Flush 0 Done Done Sent false
31671 9200 Acked Write 8 Done Done Sent false
31672 9201 Acked Read 10 Done Done Sent false
31673 9202 Acked Write 4 Done Done Sent false
31674 9203 Acked Write 2 Done Done Sent false
31675 9204 Acked Read 7 Done Done Sent false
31676 9205 Acked Read 8 Done Done Sent false
31677 9206 Acked Write 10 Done Done Sent false
31678 9207 Acked Read 1 Done Done Sent false
31679 9208 Acked Read 3 Done Done Sent false
31680 9209 Acked Write 9 Done Done Sent false
31681 9210 Acked Read 8 Done Done Sent false
31682 9211 Acked Write 8 Done Done Sent false
31683 9212 Acked Write 5 Done Done Sent false
31684 9213 Acked Write 2 Done Done Sent false
31685 9214 Acked Read 4 Done Done Sent false
31686 9215 Acked Write 2 Done Done Sent false
31687 9216 Acked Read 8 Done Done Sent false
31688 9217 Acked Write 8 Done Done Sent false
31689 9218 Acked Write 2 Done Done Sent false
31690 9219 Acked Read 8 Done Done Sent false
31691 9220 Acked Read 5 Done Done Sent false
31692 9221 Acked Read 4 Done Done Sent false
31693 9222 Acked Read 4 Done Done Sent false
31694 9223 Acked Read 7 Done Done Sent false
31695 9224 Acked Read 6 Done Done Sent false
31696 9225 Acked Read 5 Done Done Sent false
31697 9226 Acked Flush 0 Done Done Sent false
31698 9227 Acked Flush 0 Done Done Sent false
31699 9228 Acked Read 3 Done Done Sent false
31700 9229 Acked Write 4 Done Done Sent false
31701 9230 Acked Read 7 Done Done Sent false
31702 9231 Acked Write 10 Done Done Sent false
31703 9232 Acked Flush 0 Done Done Sent false
31704 9233 Acked Read 8 Done Done Sent false
31705 9234 Acked Read 9 Done Done Sent false
31706 9235 Acked Write 3 Done Done Sent false
31707 9236 Acked Read 3 Done Done Sent false
31708 9237 Acked Read 8 Done Done Sent false
31709 9238 Acked Write 7 Done Done Sent false
31710 9239 Acked Write 8 Done Done Sent false
31711 9240 Acked Read 4 Done Done Sent false
31712 9241 Acked Read 6 Done Done Sent false
31713 9242 Acked Read 10 Done Done Sent false
31714 9243 Acked Write 5 Done Done Sent false
31715 9244 Acked Read 4 Done Done Sent false
31716 9245 Acked Write 9 Done Done Sent false
31717 9246 Acked Write 4 Done Done Sent false
31718 9247 Acked Read 2 Done Done Sent false
31719 9248 Acked Write 4 Done Done Sent false
31720 9249 Acked Read 3 Done Done Sent false
31721 9250 Acked Write 7 Done Done Sent false
31722 9251 Acked Write 4 Done Done Sent false
31723 9252 Acked Read 3 Done Done Sent false
31724 9253 Acked Write 6 Done Done Sent false
31725 9254 Acked Read 2 Done Done Sent false
31726 9255 Acked Read 1 Done Done Sent false
31727 9256 Acked Write 2 Done Done Sent false
31728 9257 Acked Write 6 Done Done Sent false
31729 9258 Acked Read 10 Done Done Sent false
31730 9259 Acked Write 6 Done Done Sent false
31731 9260 Acked Flush 0 Done Done Sent false
31732 9261 Acked Read 9 Done Done Sent false
31733 9262 Acked Read 10 Done Done Sent false
31734 9263 Acked Read 2 Done Done Sent false
31735 9264 Acked Read 7 Done Done Sent false
31736 9265 Acked Read 9 Done Done Sent false
31737 9266 Acked Write 5 Done Done Sent false
31738 9267 Acked Write 9 Done Done Sent false
31739 9268 Acked Read 10 Done Done Sent false
31740 9269 Acked Write 10 Done Done Sent false
31741 9270 Acked Read 5 Done Done Sent false
31742 9271 Acked Read 5 Done Done Sent false
31743 9272 Acked Read 2 Done Done Sent false
31744 9273 Acked Write 1 Done Done Sent false
31745 9274 Acked Read 6 Done Done Sent false
31746 9275 Acked Write 5 Done Done Sent false
31747 9276 Acked Flush 0 Done Done Sent false
31748 9277 Acked Write 6 Done Done Sent false
31749 9278 Acked Read 6 Done Done Sent false
31750 9279 Acked Write 7 Done Done Sent false
31751 9280 Acked Write 1 Done Done Sent false
31752 9281 Acked Read 9 Done Done Sent false
31753 9282 Acked Read 9 Done Done Sent false
31754 9283 Acked Read 8 Done Done Sent false
31755 9284 Acked Read 3 Done Done Sent false
31756 9285 Acked Flush 0 Done Done Sent false
31757 9286 Acked Read 10 Done Done Sent false
31758 9287 Acked Flush 0 Done Done Sent false
31759 9288 Acked Read 7 Done Done Sent false
31760 9289 Acked Read 1 Done Done Sent false
31761 9290 Acked Write 6 Done Done Sent false
31762 9291 Acked Write 9 Done Done Sent false
31763 9292 Acked Read 4 Done Done Sent false
31764 9293 Acked Read 7 Done Done Sent false
31765 9294 Acked Write 10 Done Done Sent false
31766 9295 Acked Read 8 Done Done Sent false
31767 9296 Acked Write 6 Done Done Sent false
31768 9297 Acked Write 1 Done Done Sent false
31769 9298 Acked Write 10 Done Done Sent false
31770 9299 Acked Write 10 Done Done Sent false
31771 9300 Acked Write 5 Done Done Sent false
31772 9301 Acked Read 10 Done Done Sent false
31773 9302 Acked Write 9 Done Done Sent false
31774 9303 Acked Read 6 Done Done Sent false
31775 9304 Acked Read 9 Done Done Sent false
31776 9305 Acked Flush 0 Done Done Sent false
31777 9306 Acked Flush 0 Done Done Sent false
31778 9307 Acked Write 6 Done Done Sent false
31779 9308 Acked Flush 0 Done Done Sent false
31780 9309 Acked Read 4 Done Done Sent false
31781 9310 Acked Write 7 Done Done Sent false
31782 9311 Acked Write 2 Done Done Sent false
31783 9312 Acked Write 7 Done Done Sent false
31784 9313 Acked Read 2 Done Done Sent false
31785 9314 Acked Read 10 Done Done Sent false
31786 9315 Acked Read 6 Done Done Sent false
31787 9316 Acked Write 10 Done Done Sent false
31788 9317 Acked Read 4 Done Done Sent false
31789 9318 Acked Read 2 Done Done Sent false
31790 9319 Acked Flush 0 Done Done Sent false
31791 9320 Acked Write 10 Done Done Sent false
31792 9321 Acked Read 6 Done Done Sent false
31793 9322 Acked Read 2 Done Done Sent false
31794 9323 Acked Read 4 Done Done Sent false
31795 9324 Acked Write 1 Done Done Sent false
31796 9325 Acked Write 3 Done Done Sent false
31797 9326 Acked Read 7 Done Done Sent false
31798 9327 Acked Read 8 Done Done Sent false
31799 9328 Acked Flush 0 Done Done Sent false
31800 9329 Acked Read 1 Done Done Sent false
31801 9330 Acked Read 9 Done Done Sent false
31802 9331 Acked Read 3 Done Done Sent false
31803 9332 Acked Write 9 Done Done Sent false
31804 9333 Acked Read 5 Done Done Sent false
31805 9334 Acked Write 4 Done Done Sent false
31806 9335 Acked Write 8 Done Done Sent false
31807 9336 Acked Write 9 Done Done Sent false
31808 9337 Acked Read 4 Done Done Sent false
31809 9338 Acked Write 7 Done Done Sent false
31810 9339 Acked Write 6 Done Done Sent false
31811 9340 Acked Write 1 Done Done Sent false
31812 9341 Acked Read 10 Done Done Sent false
31813 9342 Acked Read 5 Done Done Sent false
31814 9343 Acked Flush 0 Done Done Sent false
31815 9344 Acked Read 5 Done Done Sent false
31816 9345 Acked Write 5 Done Done Sent false
31817 9346 Acked Write 8 Done Done Sent false
31818 9347 Acked Read 6 Done Done Sent false
31819 9348 Acked Read 9 Done Done Sent false
31820 9349 Acked Read 5 Done Done Sent false
31821 9350 Acked Write 3 Done Done Sent false
31822 9351 Acked Read 10 Done Done Sent false
31823 9352 Acked Write 4 Done Done Sent false
31824 9353 Acked Read 4 Done Done Sent false
31825 9354 Acked Write 3 Done Done Sent false
31826 9355 Acked Read 1 Done Done Sent false
31827 9356 Acked Write 10 Done Done Sent false
31828 9357 Acked Read 10 Done Done Sent false
31829 9358 Acked Read 2 Done Done Sent false
31830 9359 Acked Read 2 Done Done Sent false
31831 9360 Acked Write 2 Done Done Sent false
31832 9361 Acked Write 3 Done Done Sent false
31833 9362 Acked Write 3 Done Done Sent false
31834 9363 Acked Flush 0 Done Done Sent false
31835 9364 Acked Read 7 Done Done Sent false
31836 9365 Acked Write 8 Done Done Sent false
31837 9366 Acked Read 5 Done Done Sent false
31838 9367 Acked Write 1 Done Done Sent false
31839 9368 Acked Write 5 Done Done Sent false
31840 9369 Acked Write 3 Done Done Sent false
31841 9370 Acked Write 1 Done Done Sent false
31842 9371 Acked Write 6 Done Done Sent false
31843 9372 Acked Read 8 Done Done Sent false
31844 9373 Acked Read 7 Done Done Sent false
31845 9374 Acked Read 2 Done Done Sent false
31846 9375 Acked Write 6 Done Done Sent false
31847 9376 Acked Read 9 Done Done Sent false
31848 9377 Acked Read 9 Done Done Sent false
31849 9378 Acked Read 7 Done Done Sent false
31850 9379 Acked Write 1 Done Done Sent false
31851 9380 Acked Write 4 Done Done Sent false
31852 9381 Acked Write 6 Done Done Sent false
31853 9382 Acked Read 1 Done Done Sent false
31854 9383 Acked Write 6 Done Done Sent false
31855 9384 Acked Read 5 Done Done Sent false
31856 9385 Acked Read 9 Done Done Sent false
31857 9386 Acked Read 7 Done Done Sent false
31858 9387 Acked Read 10 Done Done Sent false
31859 9388 Acked Read 9 Done Done Sent false
31860 9389 Acked Read 1 Done Done Sent false
31861 9390 Acked Read 3 Done Done Sent false
31862 9391 Acked Write 10 Done Done Sent false
31863 9392 Acked Write 7 Done Done Sent false
31864 9393 Acked Read 2 Done Done Sent false
31865 9394 Acked Write 2 Done Done Sent false
31866 9395 Acked Read 2 Done Done Sent false
31867 9396 Acked Read 4 Done Done Sent false
31868 9397 Acked Read 3 Done Done Sent false
31869 9398 Acked Write 3 Done Done Sent false
31870 9399 Acked Read 7 Done Done Sent false
31871 9400 Acked Write 9 Done Done Sent false
31872 9401 Acked Read 7 Done Done Sent false
31873 9402 Acked Read 2 Done Done Sent false
31874 9403 Acked Read 8 Done Done Sent false
31875 9404 Acked Read 6 Done Done Sent false
31876 9405 Acked Read 2 Done Done Sent false
31877 9406 Acked Write 5 Done Done Sent false
31878 9407 Acked Write 1 Done Done Sent false
31879 9408 Acked Read 1 Done Done Sent false
31880 9409 Acked Write 2 Done Done Sent false
31881 9410 Acked Read 6 Done Done Sent false
31882 9411 Acked Read 2 Done Done Sent false
31883 9412 Acked Write 10 Done Done Sent false
31884 9413 Acked Read 2 Done Done Sent false
31885 9414 Acked Flush 0 Done Done Sent false
31886 9415 Acked Read 8 Done Done Sent false
31887 9416 Acked Write 7 Done Done Sent false
31888 9417 Acked Write 7 Done Done Sent false
31889 9418 Acked Write 7 Done Done Sent false
31890 9419 Acked Write 3 Done Done Sent false
31891 9420 Acked Flush 0 Done Done Sent false
31892 9421 Acked Write 9 Done Done Sent false
31893 9422 Acked Read 3 Done Done Sent false
31894 9423 Acked Read 4 Done Done Sent false
31895 9424 Acked Read 3 Done Done Sent false
31896 9425 Acked Read 8 Done Done Sent false
31897 9426 Acked Write 1 Done Done Sent false
31898 9427 Acked Write 2 Done Done Sent false
31899 9428 Acked Read 1 Done Done Sent false
31900 9429 Acked Write 10 Done Done Sent false
31901 9430 Acked Read 9 Done Done Sent false
31902 9431 Acked Write 10 Done Done Sent false
31903 9432 Acked Write 5 Done Done Sent false
31904 9433 Acked Read 7 Done Done Sent false
31905 9434 Acked Read 6 Done Done Sent false
31906 9435 Acked Write 4 Done Done Sent false
31907 9436 Acked Write 10 Done Done Sent false
31908 9437 Acked Read 1 Done Done Sent false
31909 9438 Acked Flush 0 Done Done Sent false
31910 9439 Acked Flush 0 Done Done Sent false
31911 9440 Acked Read 3 Done Done Sent false
31912 9441 Acked Read 4 Done Done Sent false
31913 9442 Acked Write 10 Done Done Sent false
31914 9443 Acked Write 4 Done Done Sent false
31915 9444 Acked Read 6 Done Done Sent false
31916 9445 Acked Write 9 Done Done Sent false
31917 9446 Acked Read 8 Done Done Sent false
31918 9447 Acked Read 3 Done Done Sent false
31919 9448 Acked Read 2 Done Done Sent false
31920 9449 Acked Read 1 Done Done Sent false
31921 9450 Acked Read 3 Done Done Sent false
31922 9451 Acked Read 8 Done Done Sent false
31923 9452 Acked Flush 0 Done Done Sent false
31924 STATES DS:0 DS:1 DS:2 TOTAL
31925 Sent 0 0 302 302
31926 Done 302 302 0 604
31927 Skipped 0 0 0 0
31928 Error 0 0 0 0
31929 Last Flush: 9452 9452 9150
31930 Downstairs last five completed: 9150 9149 9148 9147 9146
31931 Upstairs last five completed: 9452 9451 9450 9449 9448
31932 CLIENT: Up:0 ds:302 act:2
319332024-11-19T17:14:40.391ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319342024-11-19T17:14:40.391ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45385) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319352024-11-19T17:14:40.392ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319362024-11-19T17:14:40.392ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319372024-11-19T17:14:40.392ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319382024-11-19T17:14:40.392ZINFOcrucible: send last flush ID to this DS: 9150 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319392024-11-19T17:14:40.393ZINFOcrucible: Replied this last flush ID: 9150 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319402024-11-19T17:14:40.393ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319412024-11-19T17:14:40.393ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319422024-11-19T17:14:40.393ZINFOcrucible: [2] client re-new 302 jobs since flush 9150 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
319432024-11-19T17:14:40.393ZINFOcrucible: [2] Replayed 302 jobs since flush: 9150 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
31944 ----------------------------------------------------------------
31945 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31946 Downstairs last five completed: 9452 9451 9450 9449 9448
31947 Upstairs last five completed: 9452 9451 9450 9449 9448
31948 CLIENT: Up:0 ds:0 act:3
31949 Replay: All jobs finished
31950 Main task has work to do, go find it
31951 got dsc Stop(1)
31952 stop 1
31953 Nov 19 17:14:42.646 INFO request completed[8820] Got stop action so:false kr:false
31954 , latency_us: 381, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 48d08636-c005-4c7f-a5df-543a63d72ad8, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
319552024-11-19T17:14:42.647ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319562024-11-19T17:14:42.647ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319572024-11-19T17:14:42.647ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
319582024-11-19T17:14:42.647ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319592024-11-19T17:14:42.647ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
319602024-11-19T17:14:42.647ZINFOcrucible: client task is exiting = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
31961 Nov 19 17:14:42.647 INFO request completed, latency_us: 137, response_code: 200, uri: /state/cid/1, method: GET, req_id: c5fde4b6-9852-4eaf-8b3a-56a538cc989c, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
31962 [8820] Exited with: ExitStatus(unix_wait_status(9))
31963 [8820][1] reports Exit
31964 Nov 19 17:14:46.650 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/1, method: GET, req_id: 688f1f51-be00-4792-b7dd-e00b988b8192, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
31965 001/300 Read block 4833 len 16384
31966 002/300 Read block 837 len 32768
31967 003/300 Write block 867 len 8192 data: 2 4
31968 004/300 Read block 2610 len 12288
31969 005/300 Write block 1670 len 20480 data: 5 5 4 3 3
31970 006/300 Write block 1002 len 4096 data: 5
31971 007/300 Read block 2023 len 12288
31972 008/300 Read block 3804 len 40960
31973 009/300 Read block 4571 len 28672
31974 010/300 Write block 3516 len 32768 data: 4 4 4 4 5 4 3 3
31975 011/300 Read block 4314 len 16384
31976 012/300 Write block 1616 len 8192 data: 10 10
31977 013/300 Write block 2991 len 36864 data: 7 7 8 8 8 7 7 5 4
31978 014/300 Write block 3406 len 40960 data: 6 6 3 4 4 5 4 4 3 3
31979 015/300 Write block 4621 len 20480 data: 7 8 9 8 5
31980 016/300 Flush
31981 017/300 Flush
31982 018/300 Write block 2223 len 12288 data: 4 3 5
31983 019/300 Read block 946 len 32768
31984 020/300 Read block 2179 len 32768
31985 021/300 Write block 4456 len 40960 data: 6 5 6 7 9 8 8 9 8 8
31986 022/300 Read block 3665 len 32768
31987 023/300 Flush
31988 024/300 Flush
31989 025/300 Write block 2570 len 32768 data: 4 4 5 6 8 7 6 7
31990 026/300 Read block 2302 len 28672
31991 027/300 Read block 4198 len 32768
31992 028/300 Write block 3490 len 8192 data: 8 9
31993 029/300 Read block 2036 len 24576
31994 030/300 Read block 4471 len 8192
31995 031/300 Write block 1076 len 16384 data: 5 6 4 3
31996 032/300 Write block 327 len 20480 data: 6 6 5 5 5
31997 033/300 Read block 2699 len 36864
31998 034/300 Read block 4193 len 28672
31999 035/300 Flush
32000 036/300 Write block 395 len 16384 data: 7 7 6 5
32001 037/300 Write block 3003 len 40960 data: 4 5 5 4 4 4 4 3 4 4
32002 038/300 Read block 3954 len 12288
32003 039/300 Write block 3830 len 20480 data: 6 5 5 4 4
32004 040/300 Read block 3794 len 8192
32005 041/300 Write block 2717 len 20480 data: 3 2 2 2 2
32006 042/300 Read block 4598 len 12288
32007 043/300 Read block 2341 len 4096
32008 044/300 Read block 1903 len 16384
32009 045/300 Read block 2703 len 32768
32010 046/300 Read block 4276 len 12288
32011 047/300 Read block 2789 len 20480
32012 048/300 Write block 2491 len 28672 data: 6 5 5 4 4 5 5
32013 049/300 Write block 1734 len 20480 data: 6 6 8 9 8
32014 050/300 Write block 4033 len 12288 data: 6 5 4
32015 051/300 Write block 1467 len 20480 data: 10 9 9 9 8
32016 052/300 Read block 3732 len 20480
32017 053/300 Read block 458 len 32768
32018 054/300 Write block 3785 len 24576 data: 6 8 8 9 6 8
32019 055/300 Read block 84 len 8192
32020 056/300 Read block 3737 len 28672
32021 057/300 Flush
32022 058/300 Flush
32023 059/300 Read block 3027 len 12288
32024 060/300 Write block 3792 len 40960 data: 7 7 8 7 7 7 7 9 6 8
32025 061/300 Read block 2346 len 20480
32026 062/300 Read block 2099 len 4096
32027 063/300 Read block 2580 len 32768
32028 064/300 Write block 3233 len 4096 data: 5
32029 065/300 Flush
32030 066/300 Read block 3418 len 32768
32031 067/300 Write block 4991 len 16384 data: 5 5 6 5
32032 068/300 Read block 3837 len 36864
32033 069/300 Flush
32034 070/300 Write block 2354 len 8192 data: 5 5
32035 071/300 Read block 3206 len 32768
32036 072/300 Write block 1685 len 32768 data: 3 2 4 5 7 6 7 7
32037 073/300 Read block 102 len 36864
32038 074/300 Flush
32039 075/300 Write block 4782 len 32768 data: 4 4 4 5 5 3 2 2
32040 076/300 Write block 4154 len 8192 data: 4 4
32041 077/300 Read block 3795 len 12288
32042 078/300 Write block 4343 len 8192 data: 4 5
32043 079/300 Write block 1038 len 20480 data: 7 6 6 8 8
32044 080/300 Flush
32045 081/300 Read block 516 len 8192
32046 082/300 Write block 970 len 24576 data: 2 3 4 4 4 4
32047 083/300 Flush
32048 084/300 Flush
32049 085/300 Write block 4903 len 16384 data: 7 6 4 4
32050 086/300 Write block 1664 len 20480 data: 4 4 4 6 5
32051 087/300 Read block 4616 len 24576
32052 088/300 Write block 3803 len 20480 data: 6 4 5 6 5
32053 089/300 Write block 4481 len 32768 data: 5 4 5 7 6 7 6 5
32054 090/300 Write block 700 len 4096 data: 5
32055 091/300 Write block 4879 len 32768 data: 3 3 3 4 3 4 4 4
32056 092/300 Read block 995 len 32768
32057 093/300 Read block 3924 len 4096
32058 094/300 Write block 2330 len 32768 data: 11 8 7 7 3 3 3 4
32059 095/300 Write block 545 len 8192 data: 4 6
32060 096/300 Write block 4752 len 8192 data: 5 4
32061 097/300 Read block 3513 len 28672
32062 098/300 Read block 2861 len 16384
32063 099/300 Flush
32064 100/300 Write block 4004 len 8192 data: 6 6
32065 101/300 Flush
32066 102/300 Write block 2694 len 40960 data: 5 5 4 5 5 5 5 5 6 7
32067 103/300 Read block 779 len 28672
32068 104/300 Write block 3778 len 4096 data: 7
32069 105/300 Read block 4883 len 16384
32070 106/300 Read block 3851 len 40960
32071 107/300 Read block 3957 len 12288
32072 108/300 Read block 1713 len 16384
32073 109/300 Write block 4376 len 12288 data: 4 3 4
32074 110/300 Write block 2305 len 24576 data: 9 5 5 4 4 4
32075 111/300 Write block 4622 len 36864 data: 9 10 9 6 7 6 3 4 3
32076 112/300 Flush
32077 113/300 Read block 1162 len 32768
32078 114/300 Read block 2217 len 40960
32079 115/300 Write block 3544 len 20480 data: 6 5 4 6 6
32080 116/300 Write block 1507 len 24576 data: 4 4 4 6 6 7
32081 117/300 Write block 681 len 20480 data: 4 5 6 7 8
32082 118/300 Read block 4759 len 24576
32083 119/300 Read block 4541 len 20480
32084 120/300 Read block 1956 len 40960
32085 121/300 Read block 4558 len 24576
32086 122/300 Flush
32087 123/300 Write block 1482 len 8192 data: 7 4
32088 124/300 Read block 2624 len 40960
32089 125/300 Flush
32090 126/300 Write block 197 len 16384 data: 5 6 6 5
32091 127/300 Read block 4908 len 36864
32092 128/300 Write block 3604 len 8192 data: 5 5
32093 129/300 Write block 3377 len 32768 data: 5 6 6 7 5 4 6 6
32094 130/300 Flush
32095 131/300 Write block 1120 len 12288 data: 8 5 6
32096 132/300 Read block 3177 len 4096
32097 133/300 Write block 4853 len 8192 data: 6 5
32098 134/300 Read block 950 len 8192
32099 135/300 Write block 3797 len 8192 data: 8 8
32100 136/300 Read block 2757 len 36864
32101 137/300 Write block 2951 len 32768 data: 4 5 5 4 4 4 4 4
32102 138/300 Write block 1462 len 36864 data: 3 3 4 4 7 11 10 10 10
32103 139/300 Write block 2761 len 36864 data: 4 5 4 6 5 4 4 4 3
32104 140/300 Read block 177 len 32768
32105 141/300 Write block 4575 len 24576 data: 3 4 4 5 4 4
32106 142/300 Write block 2576 len 16384 data: 7 8 7 6
32107 143/300 Read block 4396 len 4096
32108 144/300 Read block 2297 len 24576
32109 145/300 Flush
32110 146/300 Write block 905 len 20480 data: 8 9 9 7 7
32111 147/300 Write block 4307 len 32768 data: 5 4 4 6 5 6 5 4
32112 148/300 Flush
32113 149/300 Write block 3633 len 28672 data: 4 4 6 6 7 7 5
32114 150/300 Write block 804 len 24576 data: 4 3 3 3 3 2
32115 151/300 Write block 3026 len 16384 data: 5 6 4 5
32116 152/300 Write block 3601 len 40960 data: 9 7 6 6 6 4 3 4 3 2
32117 153/300 Write block 3360 len 36864 data: 4 5 6 7 6 6 7 7 6
32118 154/300 Flush
32119 155/300 Read block 4867 len 16384
32120 156/300 Write block 1562 len 24576 data: 8 8 9 9 11 9
32121 157/300 Write block 1987 len 8192 data: 3 5
32122 158/300 Flush
32123 159/300 Read block 895 len 24576
32124 160/300 Read block 4222 len 8192
32125 161/300 Flush
32126 162/300 Read block 2531 len 36864
32127 163/300 Read block 1115 len 32768
32128 164/300 Write block 2593 len 16384 data: 6 6 5 5
32129 165/300 Read block 1020 len 16384
32130 166/300 Write block 1372 len 8192 data: 6 6
32131 167/300 Read block 3103 len 8192
32132 168/300 Write block 1079 len 24576 data: 4 2 4 4 4 4
32133 169/300 Read block 233 len 32768
32134 170/300 Write block 3084 len 20480 data: 6 5 5 4 4
32135 171/300 Read block 35 len 28672
32136 172/300 Read block 3677 len 36864
32137 173/300 Read block 4888 len 16384
32138 174/300 Read block 1988 len 32768
32139 175/300 Write block 3861 len 24576 data: 7 9 8 8 9 9
32140 176/300 Write block 2408 len 12288 data: 3 3 4
32141 177/300 Read block 169 len 32768
32142 178/300 Read block 3111 len 36864
32143 179/300 Read block 3625 len 8192
32144 180/300 Write block 4401 len 20480 data: 3 3 3 5 4
32145 181/300 Read block 2052 len 16384
32146 182/300 Read block 2142 len 20480
32147 183/300 Flush
32148 184/300 Flush
32149 185/300 Read block 2726 len 8192
32150 186/300 Write block 2540 len 28672 data: 4 3 3 3 3 2 2
32151 187/300 Write block 805 len 36864 data: 4 4 4 4 3 2 4 4 5
32152 188/300 Read block 3440 len 28672
32153 189/300 Read block 4973 len 28672
32154 190/300 Read block 1878 len 16384
32155 191/300 Flush
32156 192/300 Write block 977 len 12288 data: 4 4 5
32157 193/300 Read block 72 len 40960
32158 194/300 Write block 825 len 16384 data: 9 8 7 6
32159 195/300 Read block 2002 len 16384
32160 196/300 Read block 3007 len 24576
32161 197/300 Read block 1347 len 16384
32162 198/300 Write block 1615 len 32768 data: 11 11 11 6 6 6 7 7
32163 199/300 Read block 403 len 28672
32164 200/300 Write block 726 len 12288 data: 4 4 5
32165 201/300 Write block 1797 len 4096 data: 4
32166 202/300 Write block 1245 len 32768 data: 6 7 5 5 5 6 4 5
32167 203/300 Read block 2102 len 32768
32168 204/300 Read block 2503 len 28672
32169 205/300 Read block 3528 len 40960
32170 206/300 Read block 1966 len 32768
32171 207/300 Read block 2649 len 24576
32172 208/300 Flush
32173 209/300 Flush
32174 210/300 Flush
32175 211/300 Write block 4935 len 12288 data: 7 7 6
32176 212/300 Write block 4386 len 4096 data: 8
32177 213/300 Write block 2647 len 32768 data: 6 4 4 3 2 3 3 5
32178 214/300 Flush
32179 215/300 Write block 41 len 24576 data: 5 6 5 4 4 4
32180 216/300 Flush
32181 217/300 Read block 4114 len 12288
32182 218/300 Read block 3073 len 40960
32183 219/300 Write block 1701 len 12288 data: 4 5 5
32184 220/300 Read block 3821 len 20480
32185 221/300 Read block 2945 len 40960
32186 222/300 Read block 4487 len 12288
32187 223/300 Write block 2667 len 20480 data: 9 7 5 5 5
32188 224/300 Read block 1926 len 20480
32189 225/300 Write block 523 len 32768 data: 5 6 5 5 6 6 5 5
32190 226/300 Flush
32191 227/300 Write block 1795 len 8192 data: 4 5
32192 228/300 Read block 3436 len 36864
32193 229/300 Read block 1572 len 16384
32194 230/300 Write block 2384 len 32768 data: 3 3 3 4 5 6 7 9
32195 231/300 Write block 2504 len 36864 data: 8 8 5 5 7 8 6 7 7
32196 232/300 Read block 4669 len 16384
32197 233/300 Write block 2608 len 20480 data: 6 5 6 7 6
32198 234/300 Write block 4744 len 24576 data: 7 7 6 5 4 4
32199 235/300 Write block 3290 len 24576 data: 5 6 6 6 7 7
32200 236/300 Write block 1909 len 20480 data: 3 4 4 4 4
32201 237/300 Write block 3608 len 24576 data: 5 4 3 3 3 3
32202 238/300 Read block 4020 len 12288
32203 239/300 Read block 1718 len 16384
32204 240/300 Write block 3644 len 4096 data: 3
32205 241/300 Read block 2549 len 8192
32206 242/300 Write block 4581 len 40960 data: 5 5 6 6 5 5 6 8 8 8
32207 243/300 Write block 1924 len 4096 data: 4
32208 244/300 Read block 3307 len 8192
32209 245/300 Write block 1166 len 8192 data: 9 7
32210 246/300 Write block 1404 len 20480 data: 8 9 10 9 7
32211 247/300 Read block 3106 len 16384
32212 248/300 Write block 3443 len 32768 data: 8 7 6 5 3 7 9 9
32213 249/300 Read block 4590 len 40960
32214 250/300 Read block 3031 len 24576
32215 251/300 Write block 1475 len 8192 data: 5 4
32216 252/300 Read block 786 len 8192
32217 253/300 Read block 4236 len 20480
32218 254/300 Write block 2563 len 4096 data: 11
32219 255/300 Read block 2369 len 4096
32220 256/300 Write block 3191 len 16384 data: 4 4 4 4
32221 257/300 Read block 4944 len 20480
32222 258/300 Write block 4986 len 16384 data: 2 3 3 4
32223 259/300 Flush
32224 260/300 Read block 4476 len 36864
32225 261/300 Write block 1477 len 4096 data: 6
32226 262/300 Read block 4435 len 12288
32227 263/300 Write block 884 len 24576 data: 6 7 6 6 3 3
32228 264/300 Read block 686 len 28672
32229 265/300 Read block 3407 len 8192
32230 266/300 Write block 3224 len 28672 data: 7 8 9 7 6 6 6
32231 267/300 Read block 3322 len 8192
32232 268/300 Write block 4868 len 4096 data: 7
32233 269/300 Write block 2585 len 4096 data: 5
32234 270/300 Read block 3606 len 20480
32235 271/300 Write block 2666 len 20480 data: 7 10 8 6 6
32236 272/300 Read block 459 len 24576
32237 273/300 Read block 876 len 40960
32238 274/300 Write block 3727 len 8192 data: 8 9
32239 275/300 Write block 3814 len 32768 data: 3 4 4 3 2 3 3 6
32240 276/300 Write block 825 len 24576 data: 10 9 8 7 7 8
32241 277/300 Read block 2018 len 32768
32242 278/300 Read block 2151 len 40960
32243 279/300 Read block 4223 len 20480
32244 280/300 Read block 4500 len 16384
32245 281/300 Write block 3508 len 40960 data: 3 5 5 3 4 5 5 5 5 5
32246 282/300 Flush
32247 283/300 Read block 4707 len 20480
32248 284/300 Write block 4207 len 40960 data: 5 5 4 4 5 4 4 5 5 5
32249 285/300 Write block 2146 len 36864 data: 8 5 7 8 8 7 5 5 7
32250 286/300 Read block 3858 len 24576
32251 287/300 Write block 3861 len 4096 data: 8
32252 288/300 Read block 4838 len 12288
32253 289/300 Read block 4450 len 32768
32254 290/300 Flush
32255 291/300 Write block 1738 len 8192 data: 9 6
32256 292/300 Read block 2622 len 28672
32257 293/300 Write block 2781 len 40960 data: 5 5 5 5 5 5 6 5 6 7
32258 294/300 Write block 4794 len 16384 data: 7 9 7 8
32259 295/300 Read block 2966 len 28672
32260 296/300 Read block 2062 len 32768
32261 297/300 Read block 814 len 40960
32262 298/300 Write block 2132 len 8192 data: 4 4
32263 299/300 Write block 2042 len 36864 data: 4 3 4 4 5 5 4 3 5
32264 300/300 Read block 3036 len 8192
32265 301/300 Write block 3367 len 12288 data: 8 7 7
32266 Main task has work to do, go find it
32267 got dsc Start(1)
32268 start 1
32269 Nov 19 17:14:46.879 INFO request completed, latency_us: 250, response_code: 204, uri: /start/cid/1, method: POST, req_id: c00eb4b6-dcf2-4617-ad4e-8c0bc41b23fa, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
32270 Replay: started 1, returned:Ok(())
32271 ----------------------------------------------------------------
32272 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32273 States: Active Offline Active
32274 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32275 9453 Acked Read 4 Done Sent Done false
32276 9454 Acked Read 8 Done Sent Done false
32277 9455 Acked Write 2 Done Sent Done false
32278 9456 Acked Read 3 Done Sent Done false
32279 9457 Acked Write 5 Done Sent Done false
32280 9458 Acked Write 1 Done Sent Done false
32281 9459 Acked Read 3 Done Sent Done false
32282 9460 Acked Read 10 Done Sent Done false
32283 9461 Acked Read 7 Done Sent Done false
32284 9462 Acked Write 8 Done Sent Done false
32285 9463 Acked Read 4 Done Sent Done false
32286 9464 Acked Write 2 Done Sent Done false
32287 9465 Acked Write 9 Done Sent Done false
32288 9466 Acked Write 10 Done Sent Done false
32289 9467 Acked Write 5 Done Sent Done false
32290 9468 Acked Flush 0 Done Sent Done false
32291 9469 Acked Flush 0 Done Sent Done false
32292 9470 Acked Write 3 Done Sent Done false
32293 9471 Acked Read 8 Done Sent Done false
32294 9472 Acked Read 8 Done Sent Done false
32295 9473 Acked Write 10 Done Sent Done false
32296 9474 Acked Read 8 Done Sent Done false
32297 9475 Acked Flush 0 Done Sent Done false
32298 9476 Acked Flush 0 Done Sent Done false
32299 9477 Acked Write 8 Done Sent Done false
32300 9478 Acked Read 7 Done Sent Done false
32301 9479 Acked Read 8 Done Sent Done false
32302 9480 Acked Write 2 Done Sent Done false
32303 9481 Acked Read 6 Done Sent Done false
32304 9482 Acked Read 2 Done Sent Done false
32305 9483 Acked Write 4 Done Sent Done false
32306 9484 Acked Write 5 Done Sent Done false
32307 9485 Acked Read 9 Done Sent Done false
32308 9486 Acked Read 7 Done Sent Done false
32309 9487 Acked Flush 0 Done Sent Done false
32310 9488 Acked Write 4 Done Sent Done false
32311 9489 Acked Write 10 Done Sent Done false
32312 9490 Acked Read 3 Done Sent Done false
32313 9491 Acked Write 5 Done Sent Done false
32314 9492 Acked Read 2 Done Sent Done false
32315 9493 Acked Write 5 Done Sent Done false
32316 9494 Acked Read 3 Done Sent Done false
32317 9495 Acked Read 1 Done Sent Done false
32318 9496 Acked Read 4 Done Sent Done false
32319 9497 Acked Read 8 Done Sent Done false
32320 9498 Acked Read 3 Done Sent Done false
32321 9499 Acked Read 5 Done Sent Done false
32322 9500 Acked Write 7 Done Sent Done false
32323 9501 Acked Write 5 Done Sent Done false
32324 9502 Acked Write 3 Done Sent Done false
32325 9503 Acked Write 5 Done Sent Done false
32326 9504 Acked Read 5 Done Sent Done false
32327 9505 Acked Read 8 Done Sent Done false
32328 9506 Acked Write 6 Done Sent Done false
32329 9507 Acked Read 2 Done Sent Done false
32330 9508 Acked Read 7 Done Sent Done false
32331 9509 Acked Flush 0 Done Sent Done false
32332 9510 Acked Flush 0 Done Sent Done false
32333 9511 Acked Read 3 Done Sent Done false
32334 9512 Acked Write 10 Done Sent Done false
32335 9513 Acked Read 5 Done Sent Done false
32336 9514 Acked Read 1 Done Sent Done false
32337 9515 Acked Read 8 Done Sent Done false
32338 9516 Acked Write 1 Done Sent Done false
32339 9517 Acked Flush 0 Done Sent Done false
32340 9518 Acked Read 8 Done Sent Done false
32341 9519 Acked Write 4 Done Sent Done false
32342 9520 Acked Read 9 Done Sent Done false
32343 9521 Acked Flush 0 Done Sent Done false
32344 9522 Acked Write 2 Done Sent Done false
32345 9523 Acked Read 8 Done Sent Done false
32346 9524 Acked Write 8 Done Sent Done false
32347 9525 Acked Read 9 Done Sent Done false
32348 9526 Acked Flush 0 Done Sent Done false
32349 9527 Acked Write 8 Done Sent Done false
32350 9528 Acked Write 2 Done Sent Done false
32351 9529 Acked Read 3 Done Sent Done false
32352 9530 Acked Write 2 Done Sent Done false
32353 9531 Acked Write 5 Done Sent Done false
32354 9532 Acked Flush 0 Done Sent Done false
32355 9533 Acked Read 2 Done Sent Done false
32356 9534 Acked Write 6 Done Sent Done false
32357 9535 Acked Flush 0 Done Sent Done false
32358 9536 Acked Flush 0 Done Sent Done false
32359 9537 Acked Write 4 Done Sent Done false
32360 9538 Acked Write 5 Done Sent Done false
32361 9539 Acked Read 6 Done Sent Done false
32362 9540 Acked Write 5 Done Sent Done false
32363 9541 Acked Write 8 Done Sent Done false
32364 9542 Acked Write 1 Done Sent Done false
32365 9543 Acked Write 8 Done Sent Done false
32366 9544 Acked Read 8 Done Sent Done false
32367 9545 Acked Read 1 Done Sent Done false
32368 9546 Acked Write 8 Done Sent Done false
32369 9547 Acked Write 2 Done Sent Done false
32370 9548 Acked Write 2 Done Sent Done false
32371 9549 Acked Read 7 Done Sent Done false
32372 9550 Acked Read 4 Done Sent Done false
32373 9551 Acked Flush 0 Done Sent Done false
32374 9552 Acked Write 2 Done Sent Done false
32375 9553 Acked Flush 0 Done Sent Done false
32376 9554 Acked Write 10 Done Sent Done false
32377 9555 Acked Read 7 Done Sent Done false
32378 9556 Acked Write 1 Done Sent Done false
32379 9557 Acked Read 4 Done Sent Done false
32380 9558 Acked Read 10 Done Sent Done false
32381 9559 Acked Read 3 Done Sent Done false
32382 9560 Acked Read 4 Done Sent Done false
32383 9561 Acked Write 3 Done Sent Done false
32384 9562 Acked Write 6 Done Sent Done false
32385 9563 Acked Write 9 Done Sent Done false
32386 9564 Acked Flush 0 Done Sent Done false
32387 9565 Acked Read 8 Done Sent Done false
32388 9566 Acked Read 10 Done Sent Done false
32389 9567 Acked Write 5 Done Sent Done false
32390 9568 Acked Write 6 Done Sent Done false
32391 9569 Acked Write 5 Done Sent Done false
32392 9570 Acked Read 6 Done Sent Done false
32393 9571 Acked Read 5 Done Sent Done false
32394 9572 Acked Read 10 Done Sent Done false
32395 9573 Acked Read 6 Done Sent Done false
32396 9574 Acked Flush 0 Done Sent Done false
32397 9575 Acked Write 2 Done Sent Done false
32398 9576 Acked Read 10 Done Sent Done false
32399 9577 Acked Flush 0 Done Sent Done false
32400 9578 Acked Write 4 Done Sent Done false
32401 9579 Acked Read 9 Done Sent Done false
32402 9580 Acked Write 2 Done Sent Done false
32403 9581 Acked Write 8 Done Sent Done false
32404 9582 Acked Flush 0 Done Sent Done false
32405 9583 Acked Write 3 Done Sent Done false
32406 9584 Acked Read 1 Done Sent Done false
32407 9585 Acked Write 2 Done Sent Done false
32408 9586 Acked Read 2 Done Sent Done false
32409 9587 Acked Write 2 Done Sent Done false
32410 9588 Acked Read 9 Done Sent Done false
32411 9589 Acked Write 8 Done Sent Done false
32412 9590 Acked Write 9 Done Sent Done false
32413 9591 Acked Write 9 Done Sent Done false
32414 9592 Acked Read 8 Done Sent Done false
32415 9593 Acked Write 6 Done Sent Done false
32416 9594 Acked Write 4 Done Sent Done false
32417 9595 Acked Read 1 Done Sent Done false
32418 9596 Acked Read 6 Done Sent Done false
32419 9597 Acked Flush 0 Done Sent Done false
32420 9598 Acked Write 5 Done Sent Done false
32421 9599 Acked Write 8 Done Sent Done false
32422 9600 Acked Flush 0 Done Sent Done false
32423 9601 Acked Write 7 Done Sent Done false
32424 9602 Acked Write 6 Done Sent Done false
32425 9603 Acked Write 4 Done Sent Done false
32426 9604 Acked Write 10 Done Sent Done false
32427 9605 Acked Write 9 Done Sent Done false
32428 9606 Acked Flush 0 Done Sent Done false
32429 9607 Acked Read 4 Done Sent Done false
32430 9608 Acked Write 6 Done Sent Done false
32431 9609 Acked Write 2 Done Sent Done false
32432 9610 Acked Flush 0 Done Sent Done false
32433 9611 Acked Read 6 Done Sent Done false
32434 9612 Acked Read 2 Done Sent Done false
32435 9613 Acked Flush 0 Done Sent Done false
32436 9614 Acked Read 9 Done Sent Done false
32437 9615 Acked Read 8 Done Sent Done false
32438 9616 Acked Write 4 Done Sent Done false
32439 9617 Acked Read 4 Done Sent Done false
32440 9618 Acked Write 2 Done Sent Done false
32441 9619 Acked Read 2 Done Sent Done false
32442 9620 Acked Write 6 Done Sent Done false
32443 9621 Acked Read 8 Done Sent Done false
32444 9622 Acked Write 5 Done Sent Done false
32445 9623 Acked Read 7 Done Sent Done false
32446 9624 Acked Read 9 Done Sent Done false
32447 9625 Acked Read 4 Done Sent Done false
32448 9626 Acked Read 8 Done Sent Done false
32449 9627 Acked Write 6 Done Sent Done false
32450 9628 Acked Write 3 Done Sent Done false
32451 9629 Acked Read 8 Done Sent Done false
32452 9630 Acked Read 9 Done Sent Done false
32453 9631 Acked Read 2 Done Sent Done false
32454 9632 Acked Write 5 Done Sent Done false
32455 9633 Acked Read 4 Done Sent Done false
32456 9634 Acked Read 5 Done Sent Done false
32457 9635 Acked Flush 0 Done Sent Done false
32458 9636 Acked Flush 0 Done Sent Done false
32459 9637 Acked Read 2 Done Sent Done false
32460 9638 Acked Write 7 Done Sent Done false
32461 9639 Acked Write 9 Done Sent Done false
32462 9640 Acked Read 7 Done Sent Done false
32463 9641 Acked Read 7 Done Sent Done false
32464 9642 Acked Read 4 Done Sent Done false
32465 9643 Acked Flush 0 Done Sent Done false
32466 9644 Acked Write 3 Done Sent Done false
32467 9645 Acked Read 10 Done Sent Done false
32468 9646 Acked Write 4 Done Sent Done false
32469 9647 Acked Read 4 Done Sent Done false
32470 9648 Acked Read 6 Done Sent Done false
32471 9649 Acked Read 4 Done Sent Done false
32472 9650 Acked Write 8 Done Sent Done false
32473 9651 Acked Read 7 Done Sent Done false
32474 9652 Acked Write 3 Done Sent Done false
32475 9653 Acked Write 1 Done Sent Done false
32476 9654 Acked Write 8 Done Sent Done false
32477 9655 Acked Read 8 Done Sent Done false
32478 9656 Acked Read 7 Done Sent Done false
32479 9657 Acked Read 10 Done Sent Done false
32480 9658 Acked Read 8 Done Sent Done false
32481 9659 Acked Read 6 Done Sent Done false
32482 9660 Acked Flush 0 Done Sent Done false
32483 9661 Acked Flush 0 Done Sent Done false
32484 9662 Acked Flush 0 Done Sent Done false
32485 9663 Acked Write 3 Done Sent Done false
32486 9664 Acked Write 1 Done Sent Done false
32487 9665 Acked Write 8 Done Sent Done false
32488 9666 Acked Flush 0 Done Sent Done false
32489 9667 Acked Write 6 Done Sent Done false
32490 9668 Acked Flush 0 Done Sent Done false
32491 9669 Acked Read 3 Done Sent Done false
32492 9670 Acked Read 10 Done Sent Done false
32493 9671 Acked Write 3 Done Sent Done false
32494 9672 Acked Read 5 Done Sent Done false
32495 9673 Acked Read 10 Done Sent Done false
32496 9674 Acked Read 3 Done Sent Done false
32497 9675 Acked Write 5 Done Sent Done false
32498 9676 Acked Read 5 Done Sent Done false
32499 9677 Acked Write 8 Done Sent Done false
32500 9678 Acked Flush 0 Done Sent Done false
32501 9679 Acked Write 2 Done Sent Done false
32502 9680 Acked Read 9 Done Sent Done false
32503 9681 Acked Read 4 Done Sent Done false
32504 9682 Acked Write 8 Done Sent Done false
32505 9683 Acked Write 9 Done Sent Done false
32506 9684 Acked Read 4 Done Sent Done false
32507 9685 Acked Write 5 Done Sent Done false
32508 9686 Acked Write 6 Done Sent Done false
32509 9687 Acked Write 6 Done Sent Done false
32510 9688 Acked Write 5 Done Sent Done false
32511 9689 Acked Write 6 Done Sent Done false
32512 9690 Acked Read 3 Done Sent Done false
32513 9691 Acked Read 4 Done Sent Done false
32514 9692 Acked Write 1 Done Sent Done false
32515 9693 Acked Read 2 Done Sent Done false
32516 9694 Acked Write 10 Done Sent Done false
32517 9695 Acked Write 1 Done Sent Done false
32518 9696 Acked Read 2 Done Sent Done false
32519 9697 Acked Write 2 Done Sent Done false
32520 9698 Acked Write 5 Done Sent Done false
32521 9699 Acked Read 4 Done Sent Done false
32522 9700 Acked Write 8 Done Sent Done false
32523 9701 Acked Read 10 Done Sent Done false
32524 9702 Acked Read 6 Done Sent Done false
32525 9703 Acked Write 2 Done Sent Done false
32526 9704 Acked Read 2 Done Sent Done false
32527 9705 Acked Read 5 Done Sent Done false
32528 9706 Acked Write 1 Done Sent Done false
32529 9707 Acked Read 1 Done Sent Done false
32530 9708 Acked Write 4 Done Sent Done false
32531 9709 Acked Read 5 Done Sent Done false
32532 9710 Acked Write 4 Done Sent Done false
32533 9711 Acked Flush 0 Done Sent Done false
32534 9712 Acked Read 9 Done Sent Done false
32535 9713 Acked Write 1 Done Sent Done false
32536 9714 Acked Read 3 Done Sent Done false
32537 9715 Acked Write 6 Done Sent Done false
32538 9716 Acked Read 7 Done Sent Done false
32539 9717 Acked Read 2 Done Sent Done false
32540 9718 Acked Write 7 Done Sent Done false
32541 9719 Acked Read 2 Done Sent Done false
32542 9720 Acked Write 1 Done Sent Done false
32543 9721 Acked Write 1 Done Sent Done false
32544 9722 Acked Read 5 Done Sent Done false
32545 9723 Acked Write 5 Done Sent Done false
32546 9724 Acked Read 6 Done Sent Done false
32547 9725 Acked Read 10 Done Sent Done false
32548 9726 Acked Write 2 Done Sent Done false
32549 9727 Acked Write 8 Done Sent Done false
32550 9728 Acked Write 6 Done Sent Done false
32551 9729 Acked Read 8 Done Sent Done false
32552 9730 Acked Read 10 Done Sent Done false
32553 9731 Acked Read 5 Done Sent Done false
32554 9732 Acked Read 4 Done Sent Done false
32555 9733 Acked Write 10 Done Sent Done false
32556 9734 Acked Flush 0 Done Sent Done false
32557 9735 Acked Read 5 Done Sent Done false
32558 9736 Acked Write 10 Done Sent Done false
32559 9737 Acked Write 9 Done Sent Done false
32560 9738 Acked Read 6 Done Sent Done false
32561 9739 Acked Write 1 Done Sent Done false
32562 9740 Acked Read 3 Done Sent Done false
32563 9741 Acked Read 8 Done Sent Done false
32564 9742 Acked Flush 0 Done Sent Done false
32565 9743 Acked Write 2 Done Sent Done false
32566 9744 Acked Read 7 Done Sent Done false
32567 9745 Acked Write 10 Done Sent Done false
32568 9746 Acked Write 4 Done Sent Done false
32569 9747 Acked Read 7 Done Sent Done false
32570 9748 Acked Read 8 Done Sent Done false
32571 9749 Acked Read 10 Done Sent Done false
32572 9750 Acked Write 2 Done Sent Done false
32573 9751 Acked Write 9 Done Sent Done false
32574 9752 Acked Read 2 Done Sent Done false
32575 9753 Acked Write 3 Done Sent Done false
32576 STATES DS:0 DS:1 DS:2 TOTAL
32577 Sent 0 301 0 301
32578 Done 301 0 301 602
32579 Skipped 0 0 0 0
32580 Error 0 0 0 0
32581 Last Flush: 9742 9452 9742
32582 Downstairs last five completed: 9452 9451 9450 9449 9448
32583 Upstairs last five completed: 9753 9752 9751 9750 9749
32584 CLIENT: Up:0 ds:301 act:2
32585 [8820] got start action
32586 [8820] I am going to restart
32587 Starting downstairs at port 8820
32588 Make output file at "/tmp/dsc/downstairs-8820.txt"
32589 [8820][1] reports Starting
32590 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1231)
32591 [8820][1] reports Running
32592 ----------------------------------------------------------------
32593 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32594 States: Active Offline Active
32595 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32596 9453 Acked Read 4 Done Sent Done false
32597 9454 Acked Read 8 Done Sent Done false
32598 9455 Acked Write 2 Done Sent Done false
32599 9456 Acked Read 3 Done Sent Done false
32600 9457 Acked Write 5 Done Sent Done false
32601 9458 Acked Write 1 Done Sent Done false
32602 9459 Acked Read 3 Done Sent Done false
32603 9460 Acked Read 10 Done Sent Done false
32604 9461 Acked Read 7 Done Sent Done false
32605 9462 Acked Write 8 Done Sent Done false
32606 9463 Acked Read 4 Done Sent Done false
32607 9464 Acked Write 2 Done Sent Done false
32608 9465 Acked Write 9 Done Sent Done false
32609 9466 Acked Write 10 Done Sent Done false
32610 9467 Acked Write 5 Done Sent Done false
32611 9468 Acked Flush 0 Done Sent Done false
32612 9469 Acked Flush 0 Done Sent Done false
32613 9470 Acked Write 3 Done Sent Done false
32614 9471 Acked Read 8 Done Sent Done false
32615 9472 Acked Read 8 Done Sent Done false
32616 9473 Acked Write 10 Done Sent Done false
32617 9474 Acked Read 8 Done Sent Done false
32618 9475 Acked Flush 0 Done Sent Done false
32619 9476 Acked Flush 0 Done Sent Done false
32620 9477 Acked Write 8 Done Sent Done false
32621 9478 Acked Read 7 Done Sent Done false
32622 9479 Acked Read 8 Done Sent Done false
32623 9480 Acked Write 2 Done Sent Done false
32624 9481 Acked Read 6 Done Sent Done false
32625 9482 Acked Read 2 Done Sent Done false
32626 9483 Acked Write 4 Done Sent Done false
32627 9484 Acked Write 5 Done Sent Done false
32628 9485 Acked Read 9 Done Sent Done false
32629 9486 Acked Read 7 Done Sent Done false
32630 9487 Acked Flush 0 Done Sent Done false
32631 9488 Acked Write 4 Done Sent Done false
32632 9489 Acked Write 10 Done Sent Done false
32633 9490 Acked Read 3 Done Sent Done false
32634 9491 Acked Write 5 Done Sent Done false
32635 9492 Acked Read 2 Done Sent Done false
32636 9493 Acked Write 5 Done Sent Done false
32637 9494 Acked Read 3 Done Sent Done false
32638 9495 Acked Read 1 Done Sent Done false
32639 9496 Acked Read 4 Done Sent Done false
32640 9497 Acked Read 8 Done Sent Done false
32641 9498 Acked Read 3 Done Sent Done false
32642 9499 Acked Read 5 Done Sent Done false
32643 9500 Acked Write 7 Done Sent Done false
32644 9501 Acked Write 5 Done Sent Done false
32645 9502 Acked Write 3 Done Sent Done false
32646 9503 Acked Write 5 Done Sent Done false
32647 9504 Acked Read 5 Done Sent Done false
32648 9505 Acked Read 8 Done Sent Done false
32649 9506 Acked Write 6 Done Sent Done false
32650 9507 Acked Read 2 Done Sent Done false
32651 9508 Acked Read 7 Done Sent Done false
32652 9509 Acked Flush 0 Done Sent Done false
32653 9510 Acked Flush 0 Done Sent Done false
32654 9511 Acked Read 3 Done Sent Done false
32655 9512 Acked Write 10 Done Sent Done false
32656 9513 Acked Read 5 Done Sent Done false
32657 9514 Acked Read 1 Done Sent Done false
32658 9515 Acked Read 8 Done Sent Done false
32659 9516 Acked Write 1 Done Sent Done false
32660 9517 Acked Flush 0 Done Sent Done false
32661 9518 Acked Read 8 Done Sent Done false
32662 9519 Acked Write 4 Done Sent Done false
32663 9520 Acked Read 9 Done Sent Done false
32664 9521 Acked Flush 0 Done Sent Done false
32665 9522 Acked Write 2 Done Sent Done false
32666 9523 Acked Read 8 Done Sent Done false
32667 9524 Acked Write 8 Done Sent Done false
32668 9525 Acked Read 9 Done Sent Done false
32669 9526 Acked Flush 0 Done Sent Done false
32670 9527 Acked Write 8 Done Sent Done false
32671 9528 Acked Write 2 Done Sent Done false
32672 9529 Acked Read 3 Done Sent Done false
32673 9530 Acked Write 2 Done Sent Done false
32674 9531 Acked Write 5 Done Sent Done false
32675 9532 Acked Flush 0 Done Sent Done false
32676 9533 Acked Read 2 Done Sent Done false
32677 9534 Acked Write 6 Done Sent Done false
32678 9535 Acked Flush 0 Done Sent Done false
32679 9536 Acked Flush 0 Done Sent Done false
32680 9537 Acked Write 4 Done Sent Done false
32681 9538 Acked Write 5 Done Sent Done false
32682 9539 Acked Read 6 Done Sent Done false
32683 9540 Acked Write 5 Done Sent Done false
32684 9541 Acked Write 8 Done Sent Done false
32685 9542 Acked Write 1 Done Sent Done false
32686 9543 Acked Write 8 Done Sent Done false
32687 9544 Acked Read 8 Done Sent Done false
32688 9545 Acked Read 1 Done Sent Done false
32689 9546 Acked Write 8 Done Sent Done false
32690 9547 Acked Write 2 Done Sent Done false
32691 9548 Acked Write 2 Done Sent Done false
32692 9549 Acked Read 7 Done Sent Done false
32693 9550 Acked Read 4 Done Sent Done false
32694 9551 Acked Flush 0 Done Sent Done false
32695 9552 Acked Write 2 Done Sent Done false
32696 9553 Acked Flush 0 Done Sent Done false
32697 9554 Acked Write 10 Done Sent Done false
32698 9555 Acked Read 7 Done Sent Done false
32699 9556 Acked Write 1 Done Sent Done false
32700 9557 Acked Read 4 Done Sent Done false
32701 9558 Acked Read 10 Done Sent Done false
32702 9559 Acked Read 3 Done Sent Done false
32703 9560 Acked Read 4 Done Sent Done false
32704 9561 Acked Write 3 Done Sent Done false
32705 9562 Acked Write 6 Done Sent Done false
32706 9563 Acked Write 9 Done Sent Done false
32707 9564 Acked Flush 0 Done Sent Done false
32708 9565 Acked Read 8 Done Sent Done false
32709 9566 Acked Read 10 Done Sent Done false
32710 9567 Acked Write 5 Done Sent Done false
32711 9568 Acked Write 6 Done Sent Done false
32712 9569 Acked Write 5 Done Sent Done false
32713 9570 Acked Read 6 Done Sent Done false
32714 9571 Acked Read 5 Done Sent Done false
32715 9572 Acked Read 10 Done Sent Done false
32716 9573 Acked Read 6 Done Sent Done false
32717 9574 Acked Flush 0 Done Sent Done false
32718 9575 Acked Write 2 Done Sent Done false
32719 9576 Acked Read 10 Done Sent Done false
32720 9577 Acked Flush 0 Done Sent Done false
32721 9578 Acked Write 4 Done Sent Done false
32722 9579 Acked Read 9 Done Sent Done false
32723 9580 Acked Write 2 Done Sent Done false
32724 9581 Acked Write 8 Done Sent Done false
32725 9582 Acked Flush 0 Done Sent Done false
32726 9583 Acked Write 3 Done Sent Done false
32727 9584 Acked Read 1 Done Sent Done false
32728 9585 Acked Write 2 Done Sent Done false
32729 9586 Acked Read 2 Done Sent Done false
32730 9587 Acked Write 2 Done Sent Done false
32731 9588 Acked Read 9 Done Sent Done false
32732 9589 Acked Write 8 Done Sent Done false
32733 9590 Acked Write 9 Done Sent Done false
32734 9591 Acked Write 9 Done Sent Done false
32735 9592 Acked Read 8 Done Sent Done false
32736 9593 Acked Write 6 Done Sent Done false
32737 9594 Acked Write 4 Done Sent Done false
32738 9595 Acked Read 1 Done Sent Done false
32739 9596 Acked Read 6 Done Sent Done false
32740 9597 Acked Flush 0 Done Sent Done false
32741 9598 Acked Write 5 Done Sent Done false
32742 9599 Acked Write 8 Done Sent Done false
32743 9600 Acked Flush 0 Done Sent Done false
32744 9601 Acked Write 7 Done Sent Done false
32745 9602 Acked Write 6 Done Sent Done false
32746 9603 Acked Write 4 Done Sent Done false
32747 9604 Acked Write 10 Done Sent Done false
32748 9605 Acked Write 9 Done Sent Done false
32749 9606 Acked Flush 0 Done Sent Done false
32750 9607 Acked Read 4 Done Sent Done false
32751 9608 Acked Write 6 Done Sent Done false
32752 9609 Acked Write 2 Done Sent Done false
32753 9610 Acked Flush 0 Done Sent Done false
32754 9611 Acked Read 6 Done Sent Done false
32755 9612 Acked Read 2 Done Sent Done false
32756 9613 Acked Flush 0 Done Sent Done false
32757 9614 Acked Read 9 Done Sent Done false
32758 9615 Acked Read 8 Done Sent Done false
32759 9616 Acked Write 4 Done Sent Done false
32760 9617 Acked Read 4 Done Sent Done false
32761 9618 Acked Write 2 Done Sent Done false
32762 9619 Acked Read 2 Done Sent Done false
32763 9620 Acked Write 6 Done Sent Done false
32764 9621 Acked Read 8 Done Sent Done false
32765 9622 Acked Write 5 Done Sent Done false
32766 9623 Acked Read 7 Done Sent Done false
32767 9624 Acked Read 9 Done Sent Done false
32768 9625 Acked Read 4 Done Sent Done false
32769 9626 Acked Read 8 Done Sent Done false
32770 9627 Acked Write 6 Done Sent Done false
32771 9628 Acked Write 3 Done Sent Done false
32772 9629 Acked Read 8 Done Sent Done false
32773 9630 Acked Read 9 Done Sent Done false
32774 9631 Acked Read 2 Done Sent Done false
32775 9632 Acked Write 5 Done Sent Done false
32776 9633 Acked Read 4 Done Sent Done false
32777 9634 Acked Read 5 Done Sent Done false
32778 9635 Acked Flush 0 Done Sent Done false
32779 9636 Acked Flush 0 Done Sent Done false
32780 9637 Acked Read 2 Done Sent Done false
32781 9638 Acked Write 7 Done Sent Done false
32782 9639 Acked Write 9 Done Sent Done false
32783 9640 Acked Read 7 Done Sent Done false
32784 9641 Acked Read 7 Done Sent Done false
32785 9642 Acked Read 4 Done Sent Done false
32786 9643 Acked Flush 0 Done Sent Done false
32787 9644 Acked Write 3 Done Sent Done false
32788 9645 Acked Read 10 Done Sent Done false
32789 9646 Acked Write 4 Done Sent Done false
32790 9647 Acked Read 4 Done Sent Done false
32791 9648 Acked Read 6 Done Sent Done false
32792 9649 Acked Read 4 Done Sent Done false
32793 9650 Acked Write 8 Done Sent Done false
32794 9651 Acked Read 7 Done Sent Done false
32795 9652 Acked Write 3 Done Sent Done false
32796 9653 Acked Write 1 Done Sent Done false
32797 9654 Acked Write 8 Done Sent Done false
32798 9655 Acked Read 8 Done Sent Done false
32799 9656 Acked Read 7 Done Sent Done false
32800 9657 Acked Read 10 Done Sent Done false
32801 9658 Acked Read 8 Done Sent Done false
32802 9659 Acked Read 6 Done Sent Done false
32803 9660 Acked Flush 0 Done Sent Done false
32804 9661 Acked Flush 0 Done Sent Done false
32805 9662 Acked Flush 0 Done Sent Done false
32806 9663 Acked Write 3 Done Sent Done false
32807 9664 Acked Write 1 Done Sent Done false
32808 9665 Acked Write 8 Done Sent Done false
32809 9666 Acked Flush 0 Done Sent Done false
32810 9667 Acked Write 6 Done Sent Done false
32811 9668 Acked Flush 0 Done Sent Done false
32812 9669 Acked Read 3 Done Sent Done false
32813 9670 Acked Read 10 Done Sent Done false
32814 9671 Acked Write 3 Done Sent Done false
32815 9672 Acked Read 5 Done Sent Done false
32816 9673 Acked Read 10 Done Sent Done false
32817 9674 Acked Read 3 Done Sent Done false
32818 9675 Acked Write 5 Done Sent Done false
32819 9676 Acked Read 5 Done Sent Done false
32820 9677 Acked Write 8 Done Sent Done false
32821 9678 Acked Flush 0 Done Sent Done false
32822 9679 Acked Write 2 Done Sent Done false
32823 9680 Acked Read 9 Done Sent Done false
32824 9681 Acked Read 4 Done Sent Done false
32825 9682 Acked Write 8 Done Sent Done false
32826 9683 Acked Write 9 Done Sent Done false
32827 9684 Acked Read 4 Done Sent Done false
32828 9685 Acked Write 5 Done Sent Done false
32829 9686 Acked Write 6 Done Sent Done false
32830 9687 Acked Write 6 Done Sent Done false
32831 9688 Acked Write 5 Done Sent Done false
32832 9689 Acked Write 6 Done Sent Done false
32833 9690 Acked Read 3 Done Sent Done false
32834 9691 Acked Read 4 Done Sent Done false
32835 9692 Acked Write 1 Done Sent Done false
32836 9693 Acked Read 2 Done Sent Done false
32837 9694 Acked Write 10 Done Sent Done false
32838 9695 Acked Write 1 Done Sent Done false
32839 9696 Acked Read 2 Done Sent Done false
32840 9697 Acked Write 2 Done Sent Done false
32841 9698 Acked Write 5 Done Sent Done false
32842 9699 Acked Read 4 Done Sent Done false
32843 9700 Acked Write 8 Done Sent Done false
32844 9701 Acked Read 10 Done Sent Done false
32845 9702 Acked Read 6 Done Sent Done false
32846 9703 Acked Write 2 Done Sent Done false
32847 9704 Acked Read 2 Done Sent Done false
32848 9705 Acked Read 5 Done Sent Done false
32849 9706 Acked Write 1 Done Sent Done false
32850 9707 Acked Read 1 Done Sent Done false
32851 9708 Acked Write 4 Done Sent Done false
32852 9709 Acked Read 5 Done Sent Done false
32853 9710 Acked Write 4 Done Sent Done false
32854 9711 Acked Flush 0 Done Sent Done false
32855 9712 Acked Read 9 Done Sent Done false
32856 9713 Acked Write 1 Done Sent Done false
32857 9714 Acked Read 3 Done Sent Done false
32858 9715 Acked Write 6 Done Sent Done false
32859 9716 Acked Read 7 Done Sent Done false
32860 9717 Acked Read 2 Done Sent Done false
32861 9718 Acked Write 7 Done Sent Done false
32862 9719 Acked Read 2 Done Sent Done false
32863 9720 Acked Write 1 Done Sent Done false
32864 9721 Acked Write 1 Done Sent Done false
32865 9722 Acked Read 5 Done Sent Done false
32866 9723 Acked Write 5 Done Sent Done false
32867 9724 Acked Read 6 Done Sent Done false
32868 9725 Acked Read 10 Done Sent Done false
32869 9726 Acked Write 2 Done Sent Done false
32870 9727 Acked Write 8 Done Sent Done false
32871 9728 Acked Write 6 Done Sent Done false
32872 9729 Acked Read 8 Done Sent Done false
32873 9730 Acked Read 10 Done Sent Done false
32874 9731 Acked Read 5 Done Sent Done false
32875 9732 Acked Read 4 Done Sent Done false
32876 9733 Acked Write 10 Done Sent Done false
32877 9734 Acked Flush 0 Done Sent Done false
32878 9735 Acked Read 5 Done Sent Done false
32879 9736 Acked Write 10 Done Sent Done false
32880 9737 Acked Write 9 Done Sent Done false
32881 9738 Acked Read 6 Done Sent Done false
32882 9739 Acked Write 1 Done Sent Done false
32883 9740 Acked Read 3 Done Sent Done false
32884 9741 Acked Read 8 Done Sent Done false
32885 9742 Acked Flush 0 Done Sent Done false
32886 9743 Acked Write 2 Done Sent Done false
32887 9744 Acked Read 7 Done Sent Done false
32888 9745 Acked Write 10 Done Sent Done false
32889 9746 Acked Write 4 Done Sent Done false
32890 9747 Acked Read 7 Done Sent Done false
32891 9748 Acked Read 8 Done Sent Done false
32892 9749 Acked Read 10 Done Sent Done false
32893 9750 Acked Write 2 Done Sent Done false
32894 9751 Acked Write 9 Done Sent Done false
32895 9752 Acked Read 2 Done Sent Done false
32896 9753 Acked Write 3 Done Sent Done false
32897 9754 Acked Flush 0 Done Sent Done false
32898 STATES DS:0 DS:1 DS:2 TOTAL
32899 Sent 0 302 0 302
32900 Done 302 0 302 604
32901 Skipped 0 0 0 0
32902 Error 0 0 0 0
32903 Last Flush: 9754 9452 9754
32904 Downstairs last five completed: 9452 9451 9450 9449 9448
32905 Upstairs last five completed: 9754 9753 9752 9751 9750
32906 CLIENT: Up:0 ds:302 act:2
329072024-11-19T17:14:52.648ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329082024-11-19T17:14:52.648ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61935) = io task client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329092024-11-19T17:14:52.650ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329102024-11-19T17:14:52.650ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329112024-11-19T17:14:52.650ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329122024-11-19T17:14:52.650ZINFOcrucible: send last flush ID to this DS: 9452 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329132024-11-19T17:14:52.650ZINFOcrucible: Replied this last flush ID: 9452 = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329142024-11-19T17:14:52.650ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329152024-11-19T17:14:52.650ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329162024-11-19T17:14:52.650ZINFOcrucible: [1] client re-new 302 jobs since flush 9452 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
329172024-11-19T17:14:52.651ZINFOcrucible: [1] Replayed 302 jobs since flush: 9452 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
32918 ----------------------------------------------------------------
32919 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32920 Downstairs last five completed: 9754 9753 9752 9751 9750
32921 Upstairs last five completed: 9754 9753 9752 9751 9750
32922 CLIENT: Up:0 ds:0 act:3
32923 Replay: All jobs finished
32924 Main task has work to do, go find it
32925 got dsc Stop(0)
32926 stop 0
32927 [8810] Got stop action so:false kr:false
32928 Nov 19 17:14:54.884 INFO request completed, latency_us: 329, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 5fd49286-d07c-48ed-bf57-e0d6796b4af1, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
329292024-11-19T17:14:54.885ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329302024-11-19T17:14:54.885ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
329312024-11-19T17:14:54.885ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
32932 Nov 19 17:14:54.885{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:14:54.885493968Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
329332024-11-19T17:14:54.885ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
32934 INFO request completed, latency_us:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:14:54.88558168Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
32935 123, response_code: 200, uri: /state/cid/0, method: GET, req_id: e345388c-9afc-4dc5-b76b-abc7010795e1, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
32936 [8810] Exited with: ExitStatus(unix_wait_status(9))
32937 [8810][0] reports Exit
32938 Nov 19 17:14:58.888 INFO request completed, latency_us: 351, response_code: 200, uri: /state/cid/0, method: GET, req_id: 0a98db72-07fc-4af6-a984-9a0e547e4bc3, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
32939 001/300 Read block 4964 len 8192
32940 002/300 Flush
32941 003/300 Write block 3635 len 36864 data: 7 7 8 8 6 4 4 3 3
32942 004/300 Read block 2047 len 16384
32943 005/300 Flush
32944 006/300 Write block 1982 len 40960 data: 5 5 4 5 4 4 6 5 5 7
32945 007/300 Read block 3541 len 28672
32946 008/300 Write block 4401 len 20480 data: 4 4 4 6 5
32947 009/300 Read block 1127 len 32768
32948 010/300 Write block 3561 len 36864 data: 4 5 6 7 8 6 6 7 6
32949 011/300 Flush
32950 012/300 Write block 1793 len 32768 data: 5 4 5 6 5 3 4 4
32951 013/300 Read block 992 len 24576
32952 014/300 Read block 4260 len 36864
32953 015/300 Read block 2549 len 12288
32954 016/300 Write block 612 len 4096 data: 5
32955 017/300 Write block 3806 len 8192 data: 7 6
32956 018/300 Flush
32957 019/300 Read block 1332 len 36864
32958 020/300 Read block 1896 len 32768
32959 021/300 Read block 4930 len 20480
32960 022/300 Read block 3761 len 16384
32961 023/300 Read block 3732 len 12288
32962 024/300 Read block 3412 len 16384
32963 025/300 Flush
32964 026/300 Read block 431 len 36864
32965 027/300 Write block 3261 len 20480 data: 5 6 6 6 7
32966 028/300 Read block 650 len 8192
32967 029/300 Write block 2520 len 16384 data: 3 3 4 4
32968 030/300 Write block 2129 len 12288 data: 5 4 5
32969 031/300 Read block 4435 len 16384
32970 032/300 Read block 1933 len 4096
32971 033/300 Write block 1373 len 40960 data: 7 5 5 4 4 6 5 6 6 5
32972 034/300 Flush
32973 035/300 Read block 2667 len 8192
32974 036/300 Write block 51 len 8192 data: 4 6
32975 037/300 Read block 3598 len 40960
32976 038/300 Write block 3212 len 32768 data: 6 4 6 6 6 5 7 5
32977 039/300 Read block 1198 len 32768
32978 040/300 Read block 2738 len 16384
32979 041/300 Flush
32980 042/300 Write block 798 len 24576 data: 7 3 4 5 5 5
32981 043/300 Read block 492 len 28672
32982 044/300 Write block 1694 len 4096 data: 8
32983 045/300 Read block 4528 len 32768
32984 046/300 Read block 1512 len 20480
32985 047/300 Read block 2081 len 8192
32986 048/300 Flush
32987 049/300 Flush
32988 050/300 Read block 4661 len 20480
32989 051/300 Write block 626 len 28672 data: 7 8 8 8 6 4 3
32990 052/300 Read block 1007 len 36864
32991 053/300 Write block 5 len 36864 data: 4 5 6 7 7 8 7 7 5
32992 054/300 Read block 2171 len 16384
32993 055/300 Write block 701 len 16384 data: 7 4 6 5
32994 056/300 Write block 2877 len 16384 data: 2 3 3 4
32995 057/300 Read block 2391 len 8192
32996 058/300 Read block 1449 len 32768
32997 059/300 Read block 1557 len 4096
32998 060/300 Flush
32999 061/300 Read block 2183 len 8192
33000 062/300 Read block 4312 len 32768
33001 063/300 Read block 466 len 24576
33002 064/300 Write block 1581 len 8192 data: 7 8
33003 065/300 Read block 4845 len 20480
33004 066/300 Flush
33005 067/300 Read block 3101 len 40960
33006 068/300 Read block 1168 len 20480
33007 069/300 Write block 1010 len 24576 data: 5 3 3 5 3 2
33008 070/300 Read block 4352 len 36864
33009 071/300 Read block 2152 len 16384
33010 072/300 Read block 1209 len 16384
33011 073/300 Read block 2090 len 28672
33012 074/300 Write block 206 len 12288 data: 4 5 4
33013 075/300 Write block 4698 len 28672 data: 3 3 3 4 4 5 5
33014 076/300 Read block 1392 len 12288
33015 077/300 Read block 435 len 12288
33016 078/300 Read block 1815 len 36864
33017 079/300 Read block 4804 len 36864
33018 080/300 Write block 564 len 32768 data: 4 5 6 4 4 5 5 6
33019 081/300 Read block 3803 len 40960
33020 082/300 Read block 1250 len 24576
33021 083/300 Read block 1214 len 28672
33022 084/300 Write block 1454 len 16384 data: 6 4 3 3
33023 085/300 Write block 2510 len 4096 data: 7
33024 086/300 Flush
33025 087/300 Write block 2644 len 32768 data: 6 5 7 7 5 5 4 3
33026 088/300 Write block 2485 len 16384 data: 8 8 9 9
33027 089/300 Read block 208 len 8192
33028 090/300 Read block 4231 len 8192
33029 091/300 Read block 3510 len 40960
33030 092/300 Read block 500 len 28672
33031 093/300 Write block 530 len 20480 data: 6 6 5 5 5
33032 094/300 Write block 3556 len 20480 data: 7 7 5 4 4
33033 095/300 Flush
33034 096/300 Write block 4276 len 36864 data: 3 3 3 4 4 5 6 5 5
33035 097/300 Read block 175 len 16384
33036 098/300 Write block 2241 len 24576 data: 5 6 7 7 7 7
33037 099/300 Read block 1390 len 12288
33038 100/300 Write block 1032 len 40960 data: 5 6 6 5 6 5 8 7 7 9
33039 101/300 Flush
33040 102/300 Write block 1954 len 36864 data: 5 4 3 2 2 3 6 5 5
33041 103/300 Read block 1950 len 32768
33042 104/300 Read block 4958 len 16384
33043 105/300 Write block 3767 len 32768 data: 6 5 3 4 4 4 5 5
33044 106/300 Write block 191 len 32768 data: 7 7 6 6 6 4 6 7
33045 107/300 Read block 794 len 8192
33046 108/300 Write block 3403 len 8192 data: 4 5
33047 109/300 Read block 4958 len 24576
33048 110/300 Write block 1125 len 36864 data: 7 7 7 6 6 4 5 3 3
33049 111/300 Write block 2535 len 36864 data: 5 6 6 4 4 5 4 4 4
33050 112/300 Write block 3892 len 40960 data: 7 8 7 7 9 8 8 6 5 4
33051 113/300 Write block 4437 len 40960 data: 6 6 6 6 5 4 3 3 4 5
33052 114/300 Write block 1615 len 24576 data: 12 12 12 7 7 7
33053 115/300 Write block 2043 len 36864 data: 4 5 5 6 6 5 4 6 4
33054 116/300 Write block 937 len 12288 data: 5 6 7
33055 117/300 Read block 4181 len 36864
33056 118/300 Flush
33057 119/300 Read block 4187 len 12288
33058 120/300 Write block 1177 len 32768 data: 6 6 5 4 5 4 3 3
33059 121/300 Read block 3019 len 32768
33060 122/300 Flush
33061 123/300 Write block 2799 len 12288 data: 5 8 7
33062 124/300 Write block 3122 len 40960 data: 6 7 10 12 17 13 13 12 10 10
33063 125/300 Read block 1792 len 16384
33064 126/300 Read block 1865 len 28672
33065 127/300 Read block 704 len 32768
33066 128/300 Write block 210 len 12288 data: 6 4 7
33067 129/300 Write block 4726 len 16384 data: 7 6 5 4
33068 130/300 Read block 4766 len 32768
33069 131/300 Write block 3530 len 16384 data: 7 8 6 5
33070 132/300 Write block 1092 len 16384 data: 6 3 4 4
33071 133/300 Read block 144 len 28672
33072 134/300 Write block 4013 len 12288 data: 4 4 6
33073 135/300 Read block 3208 len 36864
33074 136/300 Read block 667 len 40960
33075 137/300 Write block 3813 len 32768 data: 3 4 5 5 4 3 4 4
33076 138/300 Write block 4967 len 20480 data: 6 6 5 6 6
33077 139/300 Write block 1012 len 24576 data: 4 6 4 3 4 5
33078 140/300 Write block 2509 len 20480 data: 9 8 8 8 6
33079 141/300 Write block 3971 len 8192 data: 4 3
33080 142/300 Write block 3769 len 16384 data: 4 5 5 5
33081 143/300 Read block 618 len 40960
33082 144/300 Read block 4537 len 28672
33083 145/300 Write block 2883 len 16384 data: 4 4 5 5
33084 146/300 Write block 3343 len 4096 data: 4
33085 147/300 Write block 4148 len 8192 data: 5 5
33086 148/300 Write block 295 len 28672 data: 3 4 5 5 5 5 5
33087 149/300 Read block 3387 len 36864
33088 150/300 Write block 1726 len 40960 data: 6 6 3 4 5 6 7 6 7 7
33089 151/300 Read block 3826 len 40960
33090 152/300 Read block 2354 len 12288
33091 153/300 Write block 1799 len 24576 data: 5 5 5 6 5 4
33092 154/300 Write block 4196 len 8192 data: 10 9
33093 155/300 Read block 2110 len 28672
33094 156/300 Write block 3142 len 24576 data: 3 4 5 7 5 5
33095 157/300 Write block 4217 len 32768 data: 6 5 5 5 4 6 6 7
33096 158/300 Read block 3570 len 16384
33097 159/300 Read block 3791 len 40960
33098 160/300 Read block 1005 len 8192
33099 161/300 Read block 4149 len 40960
33100 162/300 Flush
33101 163/300 Write block 3739 len 24576 data: 11 13 11 12 12 11
33102 164/300 Write block 1635 len 20480 data: 6 5 3 3 2
33103 165/300 Write block 3907 len 36864 data: 4 4 5 5 5 6 6 7 10
33104 166/300 Read block 1087 len 12288
33105 167/300 Read block 2481 len 40960
33106 168/300 Read block 129 len 8192
33107 169/300 Write block 229 len 40960 data: 4 2 3 4 6 6 5 5 6 6
33108 170/300 Write block 4558 len 12288 data: 4 3 5
33109 171/300 Write block 1815 len 16384 data: 5 5 7 8
33110 172/300 Read block 2574 len 16384
33111 173/300 Read block 1694 len 24576
33112 174/300 Flush
33113 175/300 Read block 2968 len 16384
33114 176/300 Read block 1876 len 12288
33115 177/300 Read block 2474 len 12288
33116 178/300 Write block 1109 len 24576 data: 5 4 5 6 5 7
33117 179/300 Read block 3187 len 12288
33118 180/300 Write block 3916 len 24576 data: 10 8 7 5 4 2
33119 181/300 Write block 3193 len 36864 data: 5 5 4 3 5 5 6 7 7
33120 182/300 Read block 3126 len 32768
33121 183/300 Write block 4272 len 40960 data: 4 6 3 3 4 4 4 5 5 6
33122 184/300 Read block 3749 len 24576
33123 185/300 Read block 3089 len 28672
33124 186/300 Read block 4750 len 24576
33125 187/300 Read block 2205 len 40960
33126 188/300 Flush
33127 189/300 Write block 4720 len 8192 data: 5 4
33128 190/300 Write block 2089 len 8192 data: 8 9
33129 191/300 Read block 3263 len 24576
33130 192/300 Write block 732 len 24576 data: 3 4 5 5 5 5
33131 193/300 Read block 3412 len 20480
33132 194/300 Read block 3839 len 16384
33133 195/300 Read block 4509 len 36864
33134 196/300 Read block 524 len 8192
33135 197/300 Read block 4316 len 20480
33136 198/300 Flush
33137 199/300 Read block 4937 len 24576
33138 200/300 Read block 3556 len 12288
33139 201/300 Write block 2948 len 32768 data: 5 3 3 5 6 6 5 5
33140 202/300 Read block 3712 len 12288
33141 203/300 Read block 3169 len 32768
33142 204/300 Write block 97 len 4096 data: 6
33143 205/300 Read block 992 len 8192
33144 206/300 Flush
33145 207/300 Write block 767 len 4096 data: 5
33146 208/300 Read block 2041 len 32768
33147 209/300 Read block 1455 len 20480
33148 210/300 Read block 3383 len 8192
33149 211/300 Read block 1284 len 32768
33150 212/300 Write block 4570 len 8192 data: 2 4
33151 213/300 Read block 4316 len 24576
33152 214/300 Read block 2740 len 8192
33153 215/300 Write block 4609 len 4096 data: 5
33154 216/300 Write block 3837 len 12288 data: 4 3 3
33155 217/300 Read block 2131 len 20480
33156 218/300 Write block 1739 len 28672 data: 7 5 6 8 8 7 6
33157 219/300 Write block 2060 len 24576 data: 4 4 5 5 6 5
33158 220/300 Read block 319 len 12288
33159 221/300 Write block 488 len 20480 data: 8 8 8 10 10
33160 222/300 Write block 3811 len 32768 data: 8 3 4 5 6 6 5 4
33161 223/300 Write block 2486 len 40960 data: 9 10 10 7 7 7 6 6 5 5
33162 224/300 Write block 1191 len 28672 data: 9 7 8 10 10 7 3
33163 225/300 Read block 486 len 32768
33164 226/300 Read block 2959 len 24576
33165 227/300 Write block 743 len 32768 data: 4 3 3 3 4 4 6 9
33166 228/300 Read block 4190 len 36864
33167 229/300 Read block 3877 len 28672
33168 230/300 Write block 3704 len 12288 data: 5 6 7
33169 231/300 Read block 100 len 40960
33170 232/300 Read block 4754 len 32768
33171 233/300 Write block 2662 len 28672 data: 7 8 8 7 8 11 9
33172 234/300 Write block 2003 len 36864 data: 3 3 3 3 3 4 4 4 5
33173 235/300 Read block 1681 len 24576
33174 236/300 Read block 4082 len 12288
33175 237/300 Read block 1281 len 24576
33176 238/300 Read block 1381 len 4096
33177 239/300 Write block 4366 len 24576 data: 4 5 6 7 7 3
33178 240/300 Read block 2587 len 32768
33179 241/300 Flush
33180 242/300 Write block 2969 len 4096 data: 3
33181 243/300 Write block 2460 len 32768 data: 3 2 2 2 2 2 3 4
33182 244/300 Write block 4035 len 24576 data: 5 4 6 7 6 5
33183 245/300 Read block 4706 len 12288
33184 246/300 Write block 1683 len 36864 data: 5 5 4 3 5 6 8 7 8
33185 247/300 Read block 4715 len 16384
33186 248/300 Read block 4505 len 12288
33187 249/300 Read block 2221 len 4096
33188 250/300 Flush
33189 251/300 Write block 853 len 8192 data: 3 6
33190 252/300 Write block 2323 len 20480 data: 4 5 7 8 7
33191 253/300 Write block 4050 len 36864 data: 2 3 3 4 4 4 4 3 5
33192 254/300 Read block 4389 len 20480
33193 255/300 Read block 3210 len 20480
33194 256/300 Write block 4084 len 4096 data: 8
33195 257/300 Write block 803 len 12288 data: 6 5 5
33196 258/300 Read block 857 len 32768
33197 259/300 Read block 3235 len 40960
33198 260/300 Write block 2339 len 4096 data: 7
33199 261/300 Write block 2878 len 12288 data: 4 4 5
33200 262/300 Write block 1626 len 12288 data: 6 5 6
33201 263/300 Write block 3166 len 28672 data: 6 7 6 5 7 7 4
33202 264/300 Write block 516 len 24576 data: 5 6 6 6 6 6
33203 265/300 Write block 800 len 12288 data: 5 6 6
33204 266/300 Read block 460 len 36864
33205 267/300 Read block 4301 len 12288
33206 268/300 Write block 10 len 8192 data: 9 8
33207 269/300 Read block 2248 len 36864
33208 270/300 Read block 29 len 16384
33209 271/300 Write block 4570 len 24576 data: 3 5 3 2 2 4
33210 272/300 Read block 4083 len 8192
33211 273/300 Write block 2122 len 12288 data: 3 3 5
33212 274/300 Write block 2155 len 32768 data: 6 6 4 3 4 5 6 5
33213 275/300 Read block 4361 len 12288
33214 276/300 Read block 4294 len 40960
33215 277/300 Read block 4178 len 20480
33216 278/300 Read block 2424 len 32768
33217 279/300 Read block 2797 len 8192
33218 280/300 Read block 3708 len 24576
33219 281/300 Write block 2999 len 12288 data: 5 4 4
33220 282/300 Write block 3095 len 12288 data: 5 5 6
33221 283/300 Read block 2859 len 40960
33222 284/300 Read block 3706 len 24576
33223 285/300 Write block 2888 len 8192 data: 4 5
33224 286/300 Write block 452 len 4096 data: 6
33225 287/300 Read block 195 len 20480
33226 288/300 Read block 1108 len 28672
33227 289/300 Write block 635 len 24576 data: 3 3 3 3 3 4
33228 290/300 Read block 1465 len 12288
33229 291/300 Write block 4298 len 12288 data: 6 5 6
33230 292/300 Write block 3589 len 28672 data: 6 6 5 6 8 10 10
33231 293/300 Read block 2161 len 28672
33232 294/300 Read block 1401 len 40960
33233 295/300 Read block 2483 len 32768
33234 296/300 Read block 1289 len 12288
33235 297/300 Read block 2829 len 16384
33236 298/300 Read block 809 len 40960
33237 299/300 Read block 3535 len 24576
33238 300/300 Write block 122 len 4096 data: 8
33239 301/300 Read block 2437 len 16384
33240 Main task has work to do, go find it
33241 got dsc Start(0)
33242 start 0
33243 Nov 19 17:14:59.130 INFO request completed, latency_us: 177, response_code: 204, uri: /start/cid/0, method: POST, req_id: f3a0753c-5e3a-41f3-b7fe-37ab822f9946, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
33244 Replay: started 0, returned:Ok(())
33245 ----------------------------------------------------------------
33246 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33247 States: Offline Active Active
33248 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33249 9755 Acked Read 2 Sent Done Done false
33250 9756 Acked Flush 0 Sent Done Done false
33251 9757 Acked Write 9 Sent Done Done false
33252 9758 Acked Read 4 Sent Done Done false
33253 9759 Acked Flush 0 Sent Done Done false
33254 9760 Acked Write 10 Sent Done Done false
33255 9761 Acked Read 7 Sent Done Done false
33256 9762 Acked Write 5 Sent Done Done false
33257 9763 Acked Read 8 Sent Done Done false
33258 9764 Acked Write 9 Sent Done Done false
33259 9765 Acked Flush 0 Sent Done Done false
33260 9766 Acked Write 8 Sent Done Done false
33261 9767 Acked Read 6 Sent Done Done false
33262 9768 Acked Read 9 Sent Done Done false
33263 9769 Acked Read 3 Sent Done Done false
33264 9770 Acked Write 1 Sent Done Done false
33265 9771 Acked Write 2 Sent Done Done false
33266 9772 Acked Flush 0 Sent Done Done false
33267 9773 Acked Read 9 Sent Done Done false
33268 9774 Acked Read 8 Sent Done Done false
33269 9775 Acked Read 5 Sent Done Done false
33270 9776 Acked Read 4 Sent Done Done false
33271 9777 Acked Read 3 Sent Done Done false
33272 9778 Acked Read 4 Sent Done Done false
33273 9779 Acked Flush 0 Sent Done Done false
33274 9780 Acked Read 9 Sent Done Done false
33275 9781 Acked Write 5 Sent Done Done false
33276 9782 Acked Read 2 Sent Done Done false
33277 9783 Acked Write 4 Sent Done Done false
33278 9784 Acked Write 3 Sent Done Done false
33279 9785 Acked Read 4 Sent Done Done false
33280 9786 Acked Read 1 Sent Done Done false
33281 9787 Acked Write 10 Sent Done Done false
33282 9788 Acked Flush 0 Sent Done Done false
33283 9789 Acked Read 2 Sent Done Done false
33284 9790 Acked Write 2 Sent Done Done false
33285 9791 Acked Read 10 Sent Done Done false
33286 9792 Acked Write 8 Sent Done Done false
33287 9793 Acked Read 8 Sent Done Done false
33288 9794 Acked Read 4 Sent Done Done false
33289 9795 Acked Flush 0 Sent Done Done false
33290 9796 Acked Write 6 Sent Done Done false
33291 9797 Acked Read 7 Sent Done Done false
33292 9798 Acked Write 1 Sent Done Done false
33293 9799 Acked Read 8 Sent Done Done false
33294 9800 Acked Read 5 Sent Done Done false
33295 9801 Acked Read 2 Sent Done Done false
33296 9802 Acked Flush 0 Sent Done Done false
33297 9803 Acked Flush 0 Sent Done Done false
33298 9804 Acked Read 5 Sent Done Done false
33299 9805 Acked Write 7 Sent Done Done false
33300 9806 Acked Read 9 Sent Done Done false
33301 9807 Acked Write 9 Sent Done Done false
33302 9808 Acked Read 4 Sent Done Done false
33303 9809 Acked Write 4 Sent Done Done false
33304 9810 Acked Write 4 Sent Done Done false
33305 9811 Acked Read 2 Sent Done Done false
33306 9812 Acked Read 8 Sent Done Done false
33307 9813 Acked Read 1 Sent Done Done false
33308 9814 Acked Flush 0 Sent Done Done false
33309 9815 Acked Read 2 Sent Done Done false
33310 9816 Acked Read 8 Sent Done Done false
33311 9817 Acked Read 6 Sent Done Done false
33312 9818 Acked Write 2 Sent Done Done false
33313 9819 Acked Read 5 Sent Done Done false
33314 9820 Acked Flush 0 Sent Done Done false
33315 9821 Acked Read 10 Sent Done Done false
33316 9822 Acked Read 5 Sent Done Done false
33317 9823 Acked Write 6 Sent Done Done false
33318 9824 Acked Read 9 Sent Done Done false
33319 9825 Acked Read 4 Sent Done Done false
33320 9826 Acked Read 4 Sent Done Done false
33321 9827 Acked Read 7 Sent Done Done false
33322 9828 Acked Write 3 Sent Done Done false
33323 9829 Acked Write 7 Sent Done Done false
33324 9830 Acked Read 3 Sent Done Done false
33325 9831 Acked Read 3 Sent Done Done false
33326 9832 Acked Read 9 Sent Done Done false
33327 9833 Acked Read 9 Sent Done Done false
33328 9834 Acked Write 8 Sent Done Done false
33329 9835 Acked Read 10 Sent Done Done false
33330 9836 Acked Read 6 Sent Done Done false
33331 9837 Acked Read 7 Sent Done Done false
33332 9838 Acked Write 4 Sent Done Done false
33333 9839 Acked Write 1 Sent Done Done false
33334 9840 Acked Flush 0 Sent Done Done false
33335 9841 Acked Write 8 Sent Done Done false
33336 9842 Acked Write 4 Sent Done Done false
33337 9843 Acked Read 2 Sent Done Done false
33338 9844 Acked Read 2 Sent Done Done false
33339 9845 Acked Read 10 Sent Done Done false
33340 9846 Acked Read 7 Sent Done Done false
33341 9847 Acked Write 5 Sent Done Done false
33342 9848 Acked Write 5 Sent Done Done false
33343 9849 Acked Flush 0 Sent Done Done false
33344 9850 Acked Write 9 Sent Done Done false
33345 9851 Acked Read 4 Sent Done Done false
33346 9852 Acked Write 6 Sent Done Done false
33347 9853 Acked Read 3 Sent Done Done false
33348 9854 Acked Write 10 Sent Done Done false
33349 9855 Acked Flush 0 Sent Done Done false
33350 9856 Acked Write 9 Sent Done Done false
33351 9857 Acked Read 8 Sent Done Done false
33352 9858 Acked Read 4 Sent Done Done false
33353 9859 Acked Write 8 Sent Done Done false
33354 9860 Acked Write 8 Sent Done Done false
33355 9861 Acked Read 2 Sent Done Done false
33356 9862 Acked Write 2 Sent Done Done false
33357 9863 Acked Read 6 Sent Done Done false
33358 9864 Acked Write 9 Sent Done Done false
33359 9865 Acked Write 9 Sent Done Done false
33360 9866 Acked Write 10 Sent Done Done false
33361 9867 Acked Write 10 Sent Done Done false
33362 9868 Acked Write 6 Sent Done Done false
33363 9869 Acked Write 9 Sent Done Done false
33364 9870 Acked Write 3 Sent Done Done false
33365 9871 Acked Read 9 Sent Done Done false
33366 9872 Acked Flush 0 Sent Done Done false
33367 9873 Acked Read 3 Sent Done Done false
33368 9874 Acked Write 8 Sent Done Done false
33369 9875 Acked Read 8 Sent Done Done false
33370 9876 Acked Flush 0 Sent Done Done false
33371 9877 Acked Write 3 Sent Done Done false
33372 9878 Acked Write 10 Sent Done Done false
33373 9879 Acked Read 4 Sent Done Done false
33374 9880 Acked Read 7 Sent Done Done false
33375 9881 Acked Read 8 Sent Done Done false
33376 9882 Acked Write 3 Sent Done Done false
33377 9883 Acked Write 4 Sent Done Done false
33378 9884 Acked Read 8 Sent Done Done false
33379 9885 Acked Write 4 Sent Done Done false
33380 9886 Acked Write 4 Sent Done Done false
33381 9887 Acked Read 7 Sent Done Done false
33382 9888 Acked Write 3 Sent Done Done false
33383 9889 Acked Read 9 Sent Done Done false
33384 9890 Acked Read 10 Sent Done Done false
33385 9891 Acked Write 8 Sent Done Done false
33386 9892 Acked Write 5 Sent Done Done false
33387 9893 Acked Write 6 Sent Done Done false
33388 9894 Acked Write 5 Sent Done Done false
33389 9895 Acked Write 2 Sent Done Done false
33390 9896 Acked Write 4 Sent Done Done false
33391 9897 Acked Read 10 Sent Done Done false
33392 9898 Acked Read 7 Sent Done Done false
33393 9899 Acked Write 4 Sent Done Done false
33394 9900 Acked Write 1 Sent Done Done false
33395 9901 Acked Write 2 Sent Done Done false
33396 9902 Acked Write 7 Sent Done Done false
33397 9903 Acked Read 9 Sent Done Done false
33398 9904 Acked Write 10 Sent Done Done false
33399 9905 Acked Read 10 Sent Done Done false
33400 9906 Acked Read 3 Sent Done Done false
33401 9907 Acked Write 6 Sent Done Done false
33402 9908 Acked Write 2 Sent Done Done false
33403 9909 Acked Read 7 Sent Done Done false
33404 9910 Acked Write 6 Sent Done Done false
33405 9911 Acked Write 8 Sent Done Done false
33406 9912 Acked Read 4 Sent Done Done false
33407 9913 Acked Read 10 Sent Done Done false
33408 9914 Acked Read 2 Sent Done Done false
33409 9915 Acked Read 10 Sent Done Done false
33410 9916 Acked Flush 0 Sent Done Done false
33411 9917 Acked Write 6 Sent Done Done false
33412 9918 Acked Write 5 Sent Done Done false
33413 9919 Acked Write 9 Sent Done Done false
33414 9920 Acked Read 3 Sent Done Done false
33415 9921 Acked Read 10 Sent Done Done false
33416 9922 Acked Read 2 Sent Done Done false
33417 9923 Acked Write 10 Sent Done Done false
33418 9924 Acked Write 3 Sent Done Done false
33419 9925 Acked Write 4 Sent Done Done false
33420 9926 Acked Read 4 Sent Done Done false
33421 9927 Acked Read 6 Sent Done Done false
33422 9928 Acked Flush 0 Sent Done Done false
33423 9929 Acked Read 4 Sent Done Done false
33424 9930 Acked Read 3 Sent Done Done false
33425 9931 Acked Read 3 Sent Done Done false
33426 9932 Acked Write 6 Sent Done Done false
33427 9933 Acked Read 3 Sent Done Done false
33428 9934 Acked Write 6 Sent Done Done false
33429 9935 Acked Write 9 Sent Done Done false
33430 9936 Acked Read 8 Sent Done Done false
33431 9937 Acked Write 10 Sent Done Done false
33432 9938 Acked Read 6 Sent Done Done false
33433 9939 Acked Read 7 Sent Done Done false
33434 9940 Acked Read 6 Sent Done Done false
33435 9941 Acked Read 10 Sent Done Done false
33436 9942 Acked Flush 0 Sent Done Done false
33437 9943 Acked Write 2 Sent Done Done false
33438 9944 Acked Write 2 Sent Done Done false
33439 9945 Acked Read 6 Sent Done Done false
33440 9946 Acked Write 6 Sent Done Done false
33441 9947 Acked Read 5 Sent Done Done false
33442 9948 Acked Read 4 Sent Done Done false
33443 9949 Acked Read 9 Sent Done Done false
33444 9950 Acked Read 2 Sent Done Done false
33445 9951 Acked Read 5 Sent Done Done false
33446 9952 Acked Flush 0 Sent Done Done false
33447 9953 Acked Read 6 Sent Done Done false
33448 9954 Acked Read 3 Sent Done Done false
33449 9955 Acked Write 8 Sent Done Done false
33450 9956 Acked Read 3 Sent Done Done false
33451 9957 Acked Read 8 Sent Done Done false
33452 9958 Acked Write 1 Sent Done Done false
33453 9959 Acked Read 2 Sent Done Done false
33454 9960 Acked Flush 0 Sent Done Done false
33455 9961 Acked Write 1 Sent Done Done false
33456 9962 Acked Read 8 Sent Done Done false
33457 9963 Acked Read 5 Sent Done Done false
33458 9964 Acked Read 2 Sent Done Done false
33459 9965 Acked Read 8 Sent Done Done false
33460 9966 Acked Write 2 Sent Done Done false
33461 9967 Acked Read 6 Sent Done Done false
33462 9968 Acked Read 2 Sent Done Done false
33463 9969 Acked Write 1 Sent Done Done false
33464 9970 Acked Write 3 Sent Done Done false
33465 9971 Acked Read 5 Sent Done Done false
33466 9972 Acked Write 7 Sent Done Done false
33467 9973 Acked Write 6 Sent Done Done false
33468 9974 Acked Read 3 Sent Done Done false
33469 9975 Acked Write 5 Sent Done Done false
33470 9976 Acked Write 8 Sent Done Done false
33471 9977 Acked Write 10 Sent Done Done false
33472 9978 Acked Write 7 Sent Done Done false
33473 9979 Acked Read 8 Sent Done Done false
33474 9980 Acked Read 6 Sent Done Done false
33475 9981 Acked Write 8 Sent Done Done false
33476 9982 Acked Read 9 Sent Done Done false
33477 9983 Acked Read 7 Sent Done Done false
33478 9984 Acked Write 3 Sent Done Done false
33479 9985 Acked Read 10 Sent Done Done false
33480 9986 Acked Read 8 Sent Done Done false
33481 9987 Acked Write 7 Sent Done Done false
33482 9988 Acked Write 9 Sent Done Done false
33483 9989 Acked Read 6 Sent Done Done false
33484 9990 Acked Read 3 Sent Done Done false
33485 9991 Acked Read 6 Sent Done Done false
33486 9992 Acked Read 1 Sent Done Done false
33487 9993 Acked Write 6 Sent Done Done false
33488 9994 Acked Read 8 Sent Done Done false
33489 9995 Acked Flush 0 Sent Done Done false
33490 9996 Acked Write 1 Sent Done Done false
33491 9997 Acked Write 8 Sent Done Done false
33492 9998 Acked Write 6 Sent Done Done false
33493 9999 Acked Read 3 Sent Done Done false
33494 10000 Acked Write 9 Sent Done Done false
33495 10001 Acked Read 4 Sent Done Done false
33496 10002 Acked Read 3 Sent Done Done false
33497 10003 Acked Read 1 Sent Done Done false
33498 10004 Acked Flush 0 Sent Done Done false
33499 10005 Acked Write 2 Sent Done Done false
33500 10006 Acked Write 5 Sent Done Done false
33501 10007 Acked Write 9 Sent Done Done false
33502 10008 Acked Read 5 Sent Done Done false
33503 10009 Acked Read 5 Sent Done Done false
33504 10010 Acked Write 1 Sent Done Done false
33505 10011 Acked Write 3 Sent Done Done false
33506 10012 Acked Read 8 Sent Done Done false
33507 10013 Acked Read 10 Sent Done Done false
33508 10014 Acked Write 1 Sent Done Done false
33509 10015 Acked Write 3 Sent Done Done false
33510 10016 Acked Write 3 Sent Done Done false
33511 10017 Acked Write 7 Sent Done Done false
33512 10018 Acked Write 6 Sent Done Done false
33513 10019 Acked Write 3 Sent Done Done false
33514 10020 Acked Read 9 Sent Done Done false
33515 10021 Acked Read 3 Sent Done Done false
33516 10022 Acked Write 2 Sent Done Done false
33517 10023 Acked Read 9 Sent Done Done false
33518 10024 Acked Read 4 Sent Done Done false
33519 10025 Acked Write 6 Sent Done Done false
33520 10026 Acked Read 2 Sent Done Done false
33521 10027 Acked Write 3 Sent Done Done false
33522 10028 Acked Write 8 Sent Done Done false
33523 10029 Acked Read 3 Sent Done Done false
33524 10030 Acked Read 10 Sent Done Done false
33525 10031 Acked Read 5 Sent Done Done false
33526 10032 Acked Read 8 Sent Done Done false
33527 10033 Acked Read 2 Sent Done Done false
33528 10034 Acked Read 6 Sent Done Done false
33529 10035 Acked Write 3 Sent Done Done false
33530 10036 Acked Write 3 Sent Done Done false
33531 10037 Acked Read 10 Sent Done Done false
33532 10038 Acked Read 6 Sent Done Done false
33533 10039 Acked Write 2 Sent Done Done false
33534 10040 Acked Write 1 Sent Done Done false
33535 10041 Acked Read 5 Sent Done Done false
33536 10042 Acked Read 7 Sent Done Done false
33537 10043 Acked Write 6 Sent Done Done false
33538 10044 Acked Read 3 Sent Done Done false
33539 10045 Acked Write 3 Sent Done Done false
33540 10046 Acked Write 7 Sent Done Done false
33541 10047 Acked Read 7 Sent Done Done false
33542 10048 Acked Read 10 Sent Done Done false
33543 10049 Acked Read 8 Sent Done Done false
33544 10050 Acked Read 3 Sent Done Done false
33545 10051 Acked Read 4 Sent Done Done false
33546 10052 Acked Read 10 Sent Done Done false
33547 10053 Acked Read 6 Sent Done Done false
33548 10054 Acked Write 1 Sent Done Done false
33549 10055 Acked Read 4 Sent Done Done false
33550 STATES DS:0 DS:1 DS:2 TOTAL
33551 Sent 301 0 0 301
33552 Done 0 301 301 602
33553 Skipped 0 0 0 0
33554 Error 0 0 0 0
33555 Last Flush: 9754 10004 10004
33556 Downstairs last five completed: 9754 9753 9752 9751 9750
33557 Upstairs last five completed: 10055 10054 10053 10052 10051
33558 CLIENT: Up:0 ds:301 act:2
33559 [8810] got start action
33560 [8810] I am going to restart
33561 Starting downstairs at port 8810
33562 Make output file at "/tmp/dsc/downstairs-8810.txt"
33563 [8810][0] reports Starting
33564 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1232)
33565 [8810][0] reports Running
33566 ----------------------------------------------------------------
33567 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33568 States: Offline Active Active
33569 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33570 9755 Acked Read 2 Sent Done Done false
33571 9756 Acked Flush 0 Sent Done Done false
33572 9757 Acked Write 9 Sent Done Done false
33573 9758 Acked Read 4 Sent Done Done false
33574 9759 Acked Flush 0 Sent Done Done false
33575 9760 Acked Write 10 Sent Done Done false
33576 9761 Acked Read 7 Sent Done Done false
33577 9762 Acked Write 5 Sent Done Done false
33578 9763 Acked Read 8 Sent Done Done false
33579 9764 Acked Write 9 Sent Done Done false
33580 9765 Acked Flush 0 Sent Done Done false
33581 9766 Acked Write 8 Sent Done Done false
33582 9767 Acked Read 6 Sent Done Done false
33583 9768 Acked Read 9 Sent Done Done false
33584 9769 Acked Read 3 Sent Done Done false
33585 9770 Acked Write 1 Sent Done Done false
33586 9771 Acked Write 2 Sent Done Done false
33587 9772 Acked Flush 0 Sent Done Done false
33588 9773 Acked Read 9 Sent Done Done false
33589 9774 Acked Read 8 Sent Done Done false
33590 9775 Acked Read 5 Sent Done Done false
33591 9776 Acked Read 4 Sent Done Done false
33592 9777 Acked Read 3 Sent Done Done false
33593 9778 Acked Read 4 Sent Done Done false
33594 9779 Acked Flush 0 Sent Done Done false
33595 9780 Acked Read 9 Sent Done Done false
33596 9781 Acked Write 5 Sent Done Done false
33597 9782 Acked Read 2 Sent Done Done false
33598 9783 Acked Write 4 Sent Done Done false
33599 9784 Acked Write 3 Sent Done Done false
33600 9785 Acked Read 4 Sent Done Done false
33601 9786 Acked Read 1 Sent Done Done false
33602 9787 Acked Write 10 Sent Done Done false
33603 9788 Acked Flush 0 Sent Done Done false
33604 9789 Acked Read 2 Sent Done Done false
33605 9790 Acked Write 2 Sent Done Done false
33606 9791 Acked Read 10 Sent Done Done false
33607 9792 Acked Write 8 Sent Done Done false
33608 9793 Acked Read 8 Sent Done Done false
33609 9794 Acked Read 4 Sent Done Done false
33610 9795 Acked Flush 0 Sent Done Done false
33611 9796 Acked Write 6 Sent Done Done false
33612 9797 Acked Read 7 Sent Done Done false
33613 9798 Acked Write 1 Sent Done Done false
33614 9799 Acked Read 8 Sent Done Done false
33615 9800 Acked Read 5 Sent Done Done false
33616 9801 Acked Read 2 Sent Done Done false
33617 9802 Acked Flush 0 Sent Done Done false
33618 9803 Acked Flush 0 Sent Done Done false
33619 9804 Acked Read 5 Sent Done Done false
33620 9805 Acked Write 7 Sent Done Done false
33621 9806 Acked Read 9 Sent Done Done false
33622 9807 Acked Write 9 Sent Done Done false
33623 9808 Acked Read 4 Sent Done Done false
33624 9809 Acked Write 4 Sent Done Done false
33625 9810 Acked Write 4 Sent Done Done false
33626 9811 Acked Read 2 Sent Done Done false
33627 9812 Acked Read 8 Sent Done Done false
33628 9813 Acked Read 1 Sent Done Done false
33629 9814 Acked Flush 0 Sent Done Done false
33630 9815 Acked Read 2 Sent Done Done false
33631 9816 Acked Read 8 Sent Done Done false
33632 9817 Acked Read 6 Sent Done Done false
33633 9818 Acked Write 2 Sent Done Done false
33634 9819 Acked Read 5 Sent Done Done false
33635 9820 Acked Flush 0 Sent Done Done false
33636 9821 Acked Read 10 Sent Done Done false
33637 9822 Acked Read 5 Sent Done Done false
33638 9823 Acked Write 6 Sent Done Done false
33639 9824 Acked Read 9 Sent Done Done false
33640 9825 Acked Read 4 Sent Done Done false
33641 9826 Acked Read 4 Sent Done Done false
33642 9827 Acked Read 7 Sent Done Done false
33643 9828 Acked Write 3 Sent Done Done false
33644 9829 Acked Write 7 Sent Done Done false
33645 9830 Acked Read 3 Sent Done Done false
33646 9831 Acked Read 3 Sent Done Done false
33647 9832 Acked Read 9 Sent Done Done false
33648 9833 Acked Read 9 Sent Done Done false
33649 9834 Acked Write 8 Sent Done Done false
33650 9835 Acked Read 10 Sent Done Done false
33651 9836 Acked Read 6 Sent Done Done false
33652 9837 Acked Read 7 Sent Done Done false
33653 9838 Acked Write 4 Sent Done Done false
33654 9839 Acked Write 1 Sent Done Done false
33655 9840 Acked Flush 0 Sent Done Done false
33656 9841 Acked Write 8 Sent Done Done false
33657 9842 Acked Write 4 Sent Done Done false
33658 9843 Acked Read 2 Sent Done Done false
33659 9844 Acked Read 2 Sent Done Done false
33660 9845 Acked Read 10 Sent Done Done false
33661 9846 Acked Read 7 Sent Done Done false
33662 9847 Acked Write 5 Sent Done Done false
33663 9848 Acked Write 5 Sent Done Done false
33664 9849 Acked Flush 0 Sent Done Done false
33665 9850 Acked Write 9 Sent Done Done false
33666 9851 Acked Read 4 Sent Done Done false
33667 9852 Acked Write 6 Sent Done Done false
33668 9853 Acked Read 3 Sent Done Done false
33669 9854 Acked Write 10 Sent Done Done false
33670 9855 Acked Flush 0 Sent Done Done false
33671 9856 Acked Write 9 Sent Done Done false
33672 9857 Acked Read 8 Sent Done Done false
33673 9858 Acked Read 4 Sent Done Done false
33674 9859 Acked Write 8 Sent Done Done false
33675 9860 Acked Write 8 Sent Done Done false
33676 9861 Acked Read 2 Sent Done Done false
33677 9862 Acked Write 2 Sent Done Done false
33678 9863 Acked Read 6 Sent Done Done false
33679 9864 Acked Write 9 Sent Done Done false
33680 9865 Acked Write 9 Sent Done Done false
33681 9866 Acked Write 10 Sent Done Done false
33682 9867 Acked Write 10 Sent Done Done false
33683 9868 Acked Write 6 Sent Done Done false
33684 9869 Acked Write 9 Sent Done Done false
33685 9870 Acked Write 3 Sent Done Done false
33686 9871 Acked Read 9 Sent Done Done false
33687 9872 Acked Flush 0 Sent Done Done false
33688 9873 Acked Read 3 Sent Done Done false
33689 9874 Acked Write 8 Sent Done Done false
33690 9875 Acked Read 8 Sent Done Done false
33691 9876 Acked Flush 0 Sent Done Done false
33692 9877 Acked Write 3 Sent Done Done false
33693 9878 Acked Write 10 Sent Done Done false
33694 9879 Acked Read 4 Sent Done Done false
33695 9880 Acked Read 7 Sent Done Done false
33696 9881 Acked Read 8 Sent Done Done false
33697 9882 Acked Write 3 Sent Done Done false
33698 9883 Acked Write 4 Sent Done Done false
33699 9884 Acked Read 8 Sent Done Done false
33700 9885 Acked Write 4 Sent Done Done false
33701 9886 Acked Write 4 Sent Done Done false
33702 9887 Acked Read 7 Sent Done Done false
33703 9888 Acked Write 3 Sent Done Done false
33704 9889 Acked Read 9 Sent Done Done false
33705 9890 Acked Read 10 Sent Done Done false
33706 9891 Acked Write 8 Sent Done Done false
33707 9892 Acked Write 5 Sent Done Done false
33708 9893 Acked Write 6 Sent Done Done false
33709 9894 Acked Write 5 Sent Done Done false
33710 9895 Acked Write 2 Sent Done Done false
33711 9896 Acked Write 4 Sent Done Done false
33712 9897 Acked Read 10 Sent Done Done false
33713 9898 Acked Read 7 Sent Done Done false
33714 9899 Acked Write 4 Sent Done Done false
33715 9900 Acked Write 1 Sent Done Done false
33716 9901 Acked Write 2 Sent Done Done false
33717 9902 Acked Write 7 Sent Done Done false
33718 9903 Acked Read 9 Sent Done Done false
33719 9904 Acked Write 10 Sent Done Done false
33720 9905 Acked Read 10 Sent Done Done false
33721 9906 Acked Read 3 Sent Done Done false
33722 9907 Acked Write 6 Sent Done Done false
33723 9908 Acked Write 2 Sent Done Done false
33724 9909 Acked Read 7 Sent Done Done false
33725 9910 Acked Write 6 Sent Done Done false
33726 9911 Acked Write 8 Sent Done Done false
33727 9912 Acked Read 4 Sent Done Done false
33728 9913 Acked Read 10 Sent Done Done false
33729 9914 Acked Read 2 Sent Done Done false
33730 9915 Acked Read 10 Sent Done Done false
33731 9916 Acked Flush 0 Sent Done Done false
33732 9917 Acked Write 6 Sent Done Done false
33733 9918 Acked Write 5 Sent Done Done false
33734 9919 Acked Write 9 Sent Done Done false
33735 9920 Acked Read 3 Sent Done Done false
33736 9921 Acked Read 10 Sent Done Done false
33737 9922 Acked Read 2 Sent Done Done false
33738 9923 Acked Write 10 Sent Done Done false
33739 9924 Acked Write 3 Sent Done Done false
33740 9925 Acked Write 4 Sent Done Done false
33741 9926 Acked Read 4 Sent Done Done false
33742 9927 Acked Read 6 Sent Done Done false
33743 9928 Acked Flush 0 Sent Done Done false
33744 9929 Acked Read 4 Sent Done Done false
33745 9930 Acked Read 3 Sent Done Done false
33746 9931 Acked Read 3 Sent Done Done false
33747 9932 Acked Write 6 Sent Done Done false
33748 9933 Acked Read 3 Sent Done Done false
33749 9934 Acked Write 6 Sent Done Done false
33750 9935 Acked Write 9 Sent Done Done false
33751 9936 Acked Read 8 Sent Done Done false
33752 9937 Acked Write 10 Sent Done Done false
33753 9938 Acked Read 6 Sent Done Done false
33754 9939 Acked Read 7 Sent Done Done false
33755 9940 Acked Read 6 Sent Done Done false
33756 9941 Acked Read 10 Sent Done Done false
33757 9942 Acked Flush 0 Sent Done Done false
33758 9943 Acked Write 2 Sent Done Done false
33759 9944 Acked Write 2 Sent Done Done false
33760 9945 Acked Read 6 Sent Done Done false
33761 9946 Acked Write 6 Sent Done Done false
33762 9947 Acked Read 5 Sent Done Done false
33763 9948 Acked Read 4 Sent Done Done false
33764 9949 Acked Read 9 Sent Done Done false
33765 9950 Acked Read 2 Sent Done Done false
33766 9951 Acked Read 5 Sent Done Done false
33767 9952 Acked Flush 0 Sent Done Done false
33768 9953 Acked Read 6 Sent Done Done false
33769 9954 Acked Read 3 Sent Done Done false
33770 9955 Acked Write 8 Sent Done Done false
33771 9956 Acked Read 3 Sent Done Done false
33772 9957 Acked Read 8 Sent Done Done false
33773 9958 Acked Write 1 Sent Done Done false
33774 9959 Acked Read 2 Sent Done Done false
33775 9960 Acked Flush 0 Sent Done Done false
33776 9961 Acked Write 1 Sent Done Done false
33777 9962 Acked Read 8 Sent Done Done false
33778 9963 Acked Read 5 Sent Done Done false
33779 9964 Acked Read 2 Sent Done Done false
33780 9965 Acked Read 8 Sent Done Done false
33781 9966 Acked Write 2 Sent Done Done false
33782 9967 Acked Read 6 Sent Done Done false
33783 9968 Acked Read 2 Sent Done Done false
33784 9969 Acked Write 1 Sent Done Done false
33785 9970 Acked Write 3 Sent Done Done false
33786 9971 Acked Read 5 Sent Done Done false
33787 9972 Acked Write 7 Sent Done Done false
33788 9973 Acked Write 6 Sent Done Done false
33789 9974 Acked Read 3 Sent Done Done false
33790 9975 Acked Write 5 Sent Done Done false
33791 9976 Acked Write 8 Sent Done Done false
33792 9977 Acked Write 10 Sent Done Done false
33793 9978 Acked Write 7 Sent Done Done false
33794 9979 Acked Read 8 Sent Done Done false
33795 9980 Acked Read 6 Sent Done Done false
33796 9981 Acked Write 8 Sent Done Done false
33797 9982 Acked Read 9 Sent Done Done false
33798 9983 Acked Read 7 Sent Done Done false
33799 9984 Acked Write 3 Sent Done Done false
33800 9985 Acked Read 10 Sent Done Done false
33801 9986 Acked Read 8 Sent Done Done false
33802 9987 Acked Write 7 Sent Done Done false
33803 9988 Acked Write 9 Sent Done Done false
33804 9989 Acked Read 6 Sent Done Done false
33805 9990 Acked Read 3 Sent Done Done false
33806 9991 Acked Read 6 Sent Done Done false
33807 9992 Acked Read 1 Sent Done Done false
33808 9993 Acked Write 6 Sent Done Done false
33809 9994 Acked Read 8 Sent Done Done false
33810 9995 Acked Flush 0 Sent Done Done false
33811 9996 Acked Write 1 Sent Done Done false
33812 9997 Acked Write 8 Sent Done Done false
33813 9998 Acked Write 6 Sent Done Done false
33814 9999 Acked Read 3 Sent Done Done false
33815 10000 Acked Write 9 Sent Done Done false
33816 10001 Acked Read 4 Sent Done Done false
33817 10002 Acked Read 3 Sent Done Done false
33818 10003 Acked Read 1 Sent Done Done false
33819 10004 Acked Flush 0 Sent Done Done false
33820 10005 Acked Write 2 Sent Done Done false
33821 10006 Acked Write 5 Sent Done Done false
33822 10007 Acked Write 9 Sent Done Done false
33823 10008 Acked Read 5 Sent Done Done false
33824 10009 Acked Read 5 Sent Done Done false
33825 10010 Acked Write 1 Sent Done Done false
33826 10011 Acked Write 3 Sent Done Done false
33827 10012 Acked Read 8 Sent Done Done false
33828 10013 Acked Read 10 Sent Done Done false
33829 10014 Acked Write 1 Sent Done Done false
33830 10015 Acked Write 3 Sent Done Done false
33831 10016 Acked Write 3 Sent Done Done false
33832 10017 Acked Write 7 Sent Done Done false
33833 10018 Acked Write 6 Sent Done Done false
33834 10019 Acked Write 3 Sent Done Done false
33835 10020 Acked Read 9 Sent Done Done false
33836 10021 Acked Read 3 Sent Done Done false
33837 10022 Acked Write 2 Sent Done Done false
33838 10023 Acked Read 9 Sent Done Done false
33839 10024 Acked Read 4 Sent Done Done false
33840 10025 Acked Write 6 Sent Done Done false
33841 10026 Acked Read 2 Sent Done Done false
33842 10027 Acked Write 3 Sent Done Done false
33843 10028 Acked Write 8 Sent Done Done false
33844 10029 Acked Read 3 Sent Done Done false
33845 10030 Acked Read 10 Sent Done Done false
33846 10031 Acked Read 5 Sent Done Done false
33847 10032 Acked Read 8 Sent Done Done false
33848 10033 Acked Read 2 Sent Done Done false
33849 10034 Acked Read 6 Sent Done Done false
33850 10035 Acked Write 3 Sent Done Done false
33851 10036 Acked Write 3 Sent Done Done false
33852 10037 Acked Read 10 Sent Done Done false
33853 10038 Acked Read 6 Sent Done Done false
33854 10039 Acked Write 2 Sent Done Done false
33855 10040 Acked Write 1 Sent Done Done false
33856 10041 Acked Read 5 Sent Done Done false
33857 10042 Acked Read 7 Sent Done Done false
33858 10043 Acked Write 6 Sent Done Done false
33859 10044 Acked Read 3 Sent Done Done false
33860 10045 Acked Write 3 Sent Done Done false
33861 10046 Acked Write 7 Sent Done Done false
33862 10047 Acked Read 7 Sent Done Done false
33863 10048 Acked Read 10 Sent Done Done false
33864 10049 Acked Read 8 Sent Done Done false
33865 10050 Acked Read 3 Sent Done Done false
33866 10051 Acked Read 4 Sent Done Done false
33867 10052 Acked Read 10 Sent Done Done false
33868 10053 Acked Read 6 Sent Done Done false
33869 10054 Acked Write 1 Sent Done Done false
33870 10055 Acked Read 4 Sent Done Done false
33871 10056 Acked Flush 0 Sent Done Done false
33872 STATES DS:0 DS:1 DS:2 TOTAL
33873 Sent 302 0 0 302
33874 Done 0 302 302 604
33875 Skipped 0 0 0 0
33876 Error 0 0 0 0
33877 Last Flush: 9754 10056 10056
33878 Downstairs last five completed: 9754 9753 9752 9751 9750
33879 Upstairs last five completed: 10056 10055 10054 10053 10052
33880 CLIENT: Up:0 ds:302 act:2
338812024-11-19T17:15:04.887ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338822024-11-19T17:15:04.887ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55757) = io task client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338832024-11-19T17:15:04.888ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338842024-11-19T17:15:04.888ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338852024-11-19T17:15:04.888ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338862024-11-19T17:15:04.888ZINFOcrucible: send last flush ID to this DS: 9754 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338872024-11-19T17:15:04.889ZINFOcrucible: Replied this last flush ID: 9754 = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338882024-11-19T17:15:04.889ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338892024-11-19T17:15:04.889ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 05650a15-902e-469c-91e5-cba45f07470e
338902024-11-19T17:15:04.889ZINFOcrucible: [0] client re-new 302 jobs since flush 9754 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
338912024-11-19T17:15:04.889ZINFOcrucible: [0] Replayed 302 jobs since flush: 9754 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
33892 ----------------------------------------------------------------
33893 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33894 Downstairs last five completed: 10056 10055 10054 10053 10052
33895 Upstairs last five completed: 10056 10055 10054 10053 10052
33896 CLIENT: Up:0 ds:0 act:3
33897 Replay: All jobs finished
33898 Main task has work to do, go find it
33899 got dsc Stop(2)
33900 stop 2
33901 [8830] Got stop action so:false kr:false
33902 Nov 19 17:15:07.136 INFO request completed, latency_us: 392, response_code: 204, uri: /stop/cid/2, method: POST, req_id: e1d6db8d-364e-443d-8806-4af238324b5e, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
339032024-11-19T17:15:07.137ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
339042024-11-19T17:15:07.137ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
339052024-11-19T17:15:07.137ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 05650a15-902e-469c-91e5-cba45f07470e
339062024-11-19T17:15:07.137ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
339072024-11-19T17:15:07.137ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
33908 Nov 19 17:15:07.137 INFO request completed, latency_us: 147, response_code: 200, uri: /state/cid/2, method: GET, req_id: d53ca497-d7fb-4852-a7d3-1229c11d9755, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:15:07.137499492Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"05650a15-902e-469c-91e5-cba45f07470e"}
33909 1:9998
33910 [8830] Exited with: ExitStatus(unix_wait_status(9))
33911 [8830][2] reports Exit
33912 Nov 19 17:15:11.139 INFO request completed, latency_us: 199, response_code: 200, uri: /state/cid/2, method: GET, req_id: 483a4c67-507c-4e7b-ad24-859c59371672, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
33913 001/300 Flush
33914 002/300 Write block 3025 len 4096 data: 4
33915 003/300 Write block 4807 len 4096 data: 5
33916 004/300 Read block 1462 len 32768
33917 005/300 Read block 2459 len 16384
33918 006/300 Flush
33919 007/300 Read block 1156 len 20480
33920 008/300 Read block 4874 len 20480
33921 009/300 Read block 3508 len 12288
33922 010/300 Write block 1906 len 36864 data: 5 6 5 4 5 5 5 5 4
33923 011/300 Read block 1907 len 20480
33924 012/300 Flush
33925 013/300 Write block 71 len 28672 data: 5 6 5 5 5 5 5
33926 014/300 Flush
33927 015/300 Read block 3850 len 20480
33928 016/300 Write block 2707 len 24576 data: 12 9 11 9 9 8
33929 017/300 Write block 179 len 8192 data: 7 8
33930 018/300 Read block 1216 len 20480
33931 019/300 Read block 2193 len 12288
33932 020/300 Read block 36 len 8192
33933 021/300 Write block 1235 len 16384 data: 7 4 3 2
33934 022/300 Write block 2085 len 36864 data: 7 6 6 6 9 10 10 7 8
33935 023/300 Read block 1397 len 36864
33936 024/300 Read block 1231 len 12288
33937 025/300 Read block 3014 len 8192
33938 026/300 Read block 1021 len 12288
33939 027/300 Write block 2219 len 20480 data: 5 5 6 5 5
33940 028/300 Flush
33941 029/300 Read block 3217 len 12288
33942 030/300 Read block 4448 len 20480
33943 031/300 Flush
33944 032/300 Write block 3447 len 8192 data: 4 8
33945 033/300 Read block 4554 len 8192
33946 034/300 Read block 981 len 24576
33947 035/300 Flush
33948 036/300 Write block 1355 len 8192 data: 7 6
33949 037/300 Flush
33950 038/300 Read block 696 len 28672
33951 039/300 Write block 4549 len 12288 data: 6 7 6
33952 040/300 Write block 3801 len 8192 data: 9 7
33953 041/300 Write block 4591 len 28672 data: 10 9 7 7 7 7 6
33954 042/300 Read block 3271 len 8192
33955 043/300 Write block 1697 len 12288 data: 8 5 4
33956 044/300 Read block 2301 len 28672
33957 045/300 Read block 582 len 4096
33958 046/300 Read block 3150 len 32768
33959 047/300 Read block 4365 len 28672
33960 048/300 Write block 2607 len 12288 data: 6 7 6
33961 049/300 Read block 4352 len 28672
33962 050/300 Read block 4679 len 12288
33963 051/300 Write block 1651 len 12288 data: 6 6 5
33964 052/300 Write block 2078 len 12288 data: 9 9 6
33965 053/300 Read block 4364 len 4096
33966 054/300 Read block 4091 len 36864
33967 055/300 Read block 836 len 16384
33968 056/300 Write block 3871 len 24576 data: 7 7 8 8 7 4
33969 057/300 Flush
33970 058/300 Read block 1401 len 12288
33971 059/300 Write block 1648 len 20480 data: 4 6 6 7 7
33972 060/300 Read block 552 len 20480
33973 061/300 Read block 209 len 32768
33974 062/300 Write block 14 len 12288 data: 5 5 5
33975 063/300 Write block 185 len 20480 data: 3 3 4 7 6
33976 064/300 Write block 4286 len 8192 data: 4 4
33977 065/300 Read block 3974 len 40960
33978 066/300 Read block 2885 len 32768
33979 067/300 Write block 2416 len 20480 data: 5 2 2 2 3
33980 068/300 Write block 1571 len 28672 data: 4 3 3 4 6 4 4
33981 069/300 Write block 206 len 20480 data: 5 6 5 6 7
33982 070/300 Flush
33983 071/300 Write block 857 len 12288 data: 6 7 4
33984 072/300 Read block 4526 len 4096
33985 073/300 Flush
33986 074/300 Write block 1241 len 4096 data: 3
33987 075/300 Read block 4574 len 8192
33988 076/300 Read block 2787 len 20480
33989 077/300 Write block 2130 len 16384 data: 5 6 5 5
33990 078/300 Read block 1726 len 12288
33991 079/300 Read block 176 len 8192
33992 080/300 Read block 3677 len 36864
33993 081/300 Read block 3140 len 40960
33994 082/300 Read block 3453 len 4096
33995 083/300 Write block 594 len 12288 data: 3 4 5
33996 084/300 Flush
33997 085/300 Read block 2143 len 40960
33998 086/300 Read block 3179 len 4096
33999 087/300 Read block 1088 len 28672
34000 088/300 Read block 4619 len 16384
34001 089/300 Write block 4166 len 4096 data: 4
34002 090/300 Read block 666 len 20480
34003 091/300 Read block 905 len 36864
34004 092/300 Read block 3717 len 24576
34005 093/300 Read block 2198 len 28672
34006 094/300 Read block 2588 len 20480
34007 095/300 Read block 4342 len 20480
34008 096/300 Read block 374 len 20480
34009 097/300 Read block 1149 len 4096
34010 098/300 Write block 2928 len 8192 data: 5 6
34011 099/300 Write block 51 len 40960 data: 5 7 6 3 3 5 7 8 9 10
34012 100/300 Write block 1177 len 12288 data: 7 7 6
34013 101/300 Write block 2765 len 24576 data: 6 5 5 5 4 3
34014 102/300 Write block 578 len 40960 data: 3 4 6 5 5 4 4 5 6 6
34015 103/300 Flush
34016 104/300 Write block 1127 len 8192 data: 8 7
34017 105/300 Read block 2362 len 36864
34018 106/300 Read block 1495 len 40960
34019 107/300 Write block 533 len 28672 data: 6 6 5 5 5 7 7
34020 108/300 Read block 3922 len 8192
34021 109/300 Write block 3418 len 24576 data: 2 3 3 3 3 2
34022 110/300 Write block 1480 len 36864 data: 8 7 8 5 2 2 5 5 4
34023 111/300 Read block 3972 len 24576
34024 112/300 Flush
34025 113/300 Read block 1005 len 40960
34026 114/300 Read block 4704 len 20480
34027 115/300 Write block 2939 len 28672 data: 7 3 2 2 2 3 5
34028 116/300 Write block 4183 len 4096 data: 5
34029 117/300 Flush
34030 118/300 Write block 3764 len 8192 data: 6 7
34031 119/300 Read block 1619 len 12288
34032 120/300 Read block 4422 len 28672
34033 121/300 Write block 3885 len 16384 data: 6 5 4 4
34034 122/300 Read block 1315 len 36864
34035 123/300 Read block 372 len 16384
34036 124/300 Read block 3963 len 40960
34037 125/300 Read block 1688 len 16384
34038 126/300 Write block 1033 len 20480 data: 7 7 6 7 6
34039 127/300 Read block 658 len 36864
34040 128/300 Read block 1096 len 24576
34041 129/300 Flush
34042 130/300 Read block 1212 len 4096
34043 131/300 Read block 1400 len 24576
34044 132/300 Write block 2696 len 8192 data: 5 6
34045 133/300 Write block 1443 len 24576 data: 5 5 6 8 10 9
34046 134/300 Write block 1047 len 24576 data: 9 9 7 7 7 5
34047 135/300 Read block 4590 len 20480
34048 136/300 Write block 3522 len 28672 data: 4 4 3 4 3 4 5
34049 137/300 Write block 2318 len 8192 data: 5 5
34050 138/300 Write block 4598 len 24576 data: 4 4 4 6 6 6
34051 139/300 Write block 2046 len 28672 data: 7 7 6 5 7 5 4
34052 140/300 Write block 46 len 36864 data: 5 5 4 3 4 6 8 7 4
34053 141/300 Write block 4475 len 32768 data: 8 7 6 6 6 7 6 5
34054 142/300 Read block 3184 len 24576
34055 143/300 Write block 1862 len 12288 data: 5 5 4
34056 144/300 Read block 3326 len 40960
34057 145/300 Read block 3253 len 36864
34058 146/300 Read block 338 len 32768
34059 147/300 Write block 3569 len 12288 data: 7 6 7
34060 148/300 Read block 2005 len 12288
34061 149/300 Read block 4588 len 16384
34062 150/300 Write block 1818 len 40960 data: 9 8 5 5 7 8 8 7 8 8
34063 151/300 Write block 4126 len 4096 data: 4
34064 152/300 Read block 1012 len 36864
34065 153/300 Read block 3302 len 4096
34066 154/300 Read block 1055 len 36864
34067 155/300 Read block 665 len 36864
34068 156/300 Flush
34069 157/300 Read block 4175 len 16384
34070 158/300 Flush
34071 159/300 Read block 3510 len 16384
34072 160/300 Read block 2048 len 24576
34073 161/300 Write block 3512 len 20480 data: 5 6 6 6 6
34074 162/300 Write block 2593 len 20480 data: 7 7 6 6 5
34075 163/300 Write block 4291 len 16384 data: 4 5 5 6
34076 164/300 Write block 1860 len 36864 data: 3 4 6 6 5 4 6 8 8
34077 165/300 Read block 803 len 8192
34078 166/300 Read block 2304 len 20480
34079 167/300 Read block 1187 len 12288
34080 168/300 Write block 2216 len 24576 data: 6 6 6 6 6 7
34081 169/300 Write block 2177 len 8192 data: 3 4
34082 170/300 Read block 3719 len 12288
34083 171/300 Flush
34084 172/300 Read block 3737 len 40960
34085 173/300 Flush
34086 174/300 Write block 1932 len 12288 data: 9 9 7
34087 175/300 Write block 2896 len 4096 data: 10
34088 176/300 Write block 4390 len 24576 data: 6 5 7 7 6 5
34089 177/300 Read block 2606 len 8192
34090 178/300 Write block 1539 len 40960 data: 4 4 5 5 5 5 6 7 7 6
34091 179/300 Read block 3566 len 20480
34092 180/300 Flush
34093 181/300 Write block 4 len 16384 data: 4 5 6 7
34094 182/300 Read block 3018 len 28672
34095 183/300 Read block 1505 len 36864
34096 184/300 Flush
34097 185/300 Read block 790 len 32768
34098 186/300 Write block 23 len 40960 data: 3 3 4 4 4 5 5 7 8 9
34099 187/300 Read block 3125 len 8192
34100 188/300 Write block 4545 len 20480 data: 5 6 5 6 7
34101 189/300 Write block 2364 len 24576 data: 8 7 6 5 4 4
34102 190/300 Write block 245 len 12288 data: 5 5 4
34103 191/300 Write block 4888 len 36864 data: 5 6 9 9 11 12 11 10 11
34104 192/300 Read block 575 len 28672
34105 193/300 Read block 1408 len 16384
34106 194/300 Read block 4804 len 20480
34107 195/300 Read block 2696 len 4096
34108 196/300 Write block 1817 len 32768 data: 8 10 9 6 6 8 9 9
34109 197/300 Flush
34110 198/300 Read block 3507 len 24576
34111 199/300 Read block 2262 len 32768
34112 200/300 Write block 4296 len 28672 data: 6 7 7 6 7 5 5
34113 201/300 Write block 4672 len 20480 data: 8 8 7 7 7
34114 202/300 Write block 3564 len 20480 data: 8 9 7 7 8
34115 203/300 Flush
34116 204/300 Read block 2306 len 4096
34117 205/300 Read block 141 len 24576
34118 206/300 Write block 2033 len 12288 data: 3 3 4
34119 207/300 Read block 1072 len 36864
34120 208/300 Read block 4359 len 12288
34121 209/300 Write block 2959 len 12288 data: 3 3 3
34122 210/300 Read block 3487 len 20480
34123 211/300 Write block 997 len 28672 data: 7 6 6 5 5 6 4
34124 212/300 Write block 2950 len 4096 data: 4
34125 213/300 Write block 3308 len 16384 data: 8 8 5 4
34126 214/300 Read block 1159 len 8192
34127 215/300 Flush
34128 216/300 Read block 3201 len 16384
34129 217/300 Write block 2063 len 12288 data: 6 7 6
34130 218/300 Read block 1150 len 24576
34131 219/300 Flush
34132 220/300 Write block 478 len 4096 data: 6
34133 221/300 Flush
34134 222/300 Write block 3418 len 32768 data: 3 4 4 4 4 3 2 2
34135 223/300 Read block 3476 len 8192
34136 224/300 Write block 4681 len 24576 data: 4 4 4 5 5 7
34137 225/300 Read block 260 len 36864
34138 226/300 Read block 1314 len 8192
34139 227/300 Flush
34140 228/300 Flush
34141 229/300 Read block 830 len 4096
34142 230/300 Write block 1545 len 28672 data: 7 8 8 7 6 6 6
34143 231/300 Write block 4421 len 4096 data: 6
34144 232/300 Read block 1348 len 24576
34145 233/300 Write block 3169 len 36864 data: 6 8 8 5 4 5 5 5 4
34146 234/300 Flush
34147 235/300 Read block 1295 len 40960
34148 236/300 Read block 393 len 40960
34149 237/300 Read block 800 len 12288
34150 238/300 Flush
34151 239/300 Write block 4826 len 24576 data: 2 2 2 2 2 2
34152 240/300 Write block 4051 len 36864 data: 4 4 5 5 5 5 4 6 5
34153 241/300 Write block 1054 len 4096 data: 6
34154 242/300 Read block 3228 len 24576
34155 243/300 Write block 1238 len 24576 data: 3 3 3 4 5 5
34156 244/300 Write block 4087 len 12288 data: 6 5 7
34157 245/300 Write block 4063 len 36864 data: 6 5 5 5 6 7 8 6 6
34158 246/300 Read block 3171 len 16384
34159 247/300 Write block 247 len 40960 data: 5 4 4 7 7 7 6 7 8 8
34160 248/300 Write block 2515 len 36864 data: 6 5 4 5 5 4 4 5 5
34161 249/300 Write block 2152 len 4096 data: 6
34162 250/300 Write block 2102 len 28672 data: 6 6 5 4 5 3 3
34163 251/300 Write block 877 len 32768 data: 6 8 6 7 7 7 6 7
34164 252/300 Read block 168 len 40960
34165 253/300 Read block 1858 len 24576
34166 254/300 Write block 4233 len 12288 data: 3 3 3
34167 255/300 Read block 4839 len 16384
34168 256/300 Write block 3790 len 4096 data: 9
34169 257/300 Write block 1788 len 4096 data: 8
34170 258/300 Read block 4845 len 28672
34171 259/300 Write block 767 len 4096 data: 6
34172 260/300 Write block 1052 len 36864 data: 6 6 7 4 4 4 4 4 5
34173 261/300 Read block 3632 len 20480
34174 262/300 Write block 3346 len 4096 data: 4
34175 263/300 Write block 3990 len 40960 data: 9 9 9 11 9 7 5 7 7 7
34176 264/300 Write block 3228 len 8192 data: 7 7
34177 265/300 Read block 4224 len 32768
34178 266/300 Read block 2304 len 20480
34179 267/300 Read block 2226 len 12288
34180 268/300 Read block 4154 len 12288
34181 269/300 Write block 1558 len 28672 data: 5 6 4 7 9 9 10
34182 270/300 Write block 2121 len 32768 data: 3 4 4 6 5 4 4 4
34183 271/300 Write block 522 len 12288 data: 5 6 7
34184 272/300 Flush
34185 273/300 Write block 2177 len 4096 data: 4
34186 274/300 Write block 689 len 24576 data: 6 7 4 4 4 5
34187 275/300 Write block 4410 len 8192 data: 7 7
34188 276/300 Read block 162 len 16384
34189 277/300 Write block 3708 len 12288 data: 9 9 9
34190 278/300 Read block 1839 len 12288
34191 279/300 Write block 4099 len 4096 data: 5
34192 280/300 Read block 1216 len 20480
34193 281/300 Read block 2277 len 32768
34194 282/300 Read block 3920 len 4096
34195 283/300 Flush
34196 284/300 Read block 1194 len 16384
34197 285/300 Write block 2969 len 32768 data: 4 3 3 3 3 6 6 6
34198 286/300 Read block 662 len 16384
34199 287/300 Read block 3944 len 12288
34200 288/300 Flush
34201 289/300 Flush
34202 290/300 Read block 2190 len 4096
34203 291/300 Write block 4911 len 32768 data: 8 8 8 7 5 3 3 4
34204 292/300 Read block 3232 len 16384
34205 293/300 Flush
34206 294/300 Read block 1901 len 12288
34207 295/300 Write block 1111 len 32768 data: 6 7 6 8 7 8 7 7
34208 296/300 Write block 677 len 40960 data: 7 6 5 5 5 6 7 8 9 7
34209 297/300 Write block 3696 len 16384 data: 7 7 7 7
34210 298/300 Flush
34211 299/300 Read block 4134 len 28672
34212 300/300 Read block 308 len 36864
34213 301/300 Read block 1739 len 8192
34214 Main task has work to do, go find it
34215 got dsc Start(2)
34216 start 2
34217 Nov 19 17:15:11.364 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9d5bd308-0ac2-4f48-9d09-0fb658dec108, remote_addr: 127.0.0.1:64175, local_addr: 127.0.0.1:9998
34218 Replay: started 2, returned:Ok(())
34219 ----------------------------------------------------------------
34220 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34221 States: Active Active Offline
34222 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34223 10057 Acked Flush 0 Done Done Sent false
34224 10058 Acked Write 1 Done Done Sent false
34225 10059 Acked Write 1 Done Done Sent false
34226 10060 Acked Read 8 Done Done Sent false
34227 10061 Acked Read 4 Done Done Sent false
34228 10062 Acked Flush 0 Done Done Sent false
34229 10063 Acked Read 5 Done Done Sent false
34230 10064 Acked Read 5 Done Done Sent false
34231 10065 Acked Read 3 Done Done Sent false
34232 10066 Acked Write 9 Done Done Sent false
34233 10067 Acked Read 5 Done Done Sent false
34234 10068 Acked Flush 0 Done Done Sent false
34235 10069 Acked Write 7 Done Done Sent false
34236 10070 Acked Flush 0 Done Done Sent false
34237 10071 Acked Read 5 Done Done Sent false
34238 10072 Acked Write 6 Done Done Sent false
34239 10073 Acked Write 2 Done Done Sent false
34240 10074 Acked Read 5 Done Done Sent false
34241 10075 Acked Read 3 Done Done Sent false
34242 10076 Acked Read 2 Done Done Sent false
34243 10077 Acked Write 4 Done Done Sent false
34244 10078 Acked Write 9 Done Done Sent false
34245 10079 Acked Read 9 Done Done Sent false
34246 10080 Acked Read 3 Done Done Sent false
34247 10081 Acked Read 2 Done Done Sent false
34248 10082 Acked Read 3 Done Done Sent false
34249 10083 Acked Write 5 Done Done Sent false
34250 10084 Acked Flush 0 Done Done Sent false
34251 10085 Acked Read 3 Done Done Sent false
34252 10086 Acked Read 5 Done Done Sent false
34253 10087 Acked Flush 0 Done Done Sent false
34254 10088 Acked Write 2 Done Done Sent false
34255 10089 Acked Read 2 Done Done Sent false
34256 10090 Acked Read 6 Done Done Sent false
34257 10091 Acked Flush 0 Done Done Sent false
34258 10092 Acked Write 2 Done Done Sent false
34259 10093 Acked Flush 0 Done Done Sent false
34260 10094 Acked Read 7 Done Done Sent false
34261 10095 Acked Write 3 Done Done Sent false
34262 10096 Acked Write 2 Done Done Sent false
34263 10097 Acked Write 7 Done Done Sent false
34264 10098 Acked Read 2 Done Done Sent false
34265 10099 Acked Write 3 Done Done Sent false
34266 10100 Acked Read 7 Done Done Sent false
34267 10101 Acked Read 1 Done Done Sent false
34268 10102 Acked Read 8 Done Done Sent false
34269 10103 Acked Read 7 Done Done Sent false
34270 10104 Acked Write 3 Done Done Sent false
34271 10105 Acked Read 7 Done Done Sent false
34272 10106 Acked Read 3 Done Done Sent false
34273 10107 Acked Write 3 Done Done Sent false
34274 10108 Acked Write 3 Done Done Sent false
34275 10109 Acked Read 1 Done Done Sent false
34276 10110 Acked Read 9 Done Done Sent false
34277 10111 Acked Read 4 Done Done Sent false
34278 10112 Acked Write 6 Done Done Sent false
34279 10113 Acked Flush 0 Done Done Sent false
34280 10114 Acked Read 3 Done Done Sent false
34281 10115 Acked Write 5 Done Done Sent false
34282 10116 Acked Read 5 Done Done Sent false
34283 10117 Acked Read 8 Done Done Sent false
34284 10118 Acked Write 3 Done Done Sent false
34285 10119 Acked Write 5 Done Done Sent false
34286 10120 Acked Write 2 Done Done Sent false
34287 10121 Acked Read 10 Done Done Sent false
34288 10122 Acked Read 8 Done Done Sent false
34289 10123 Acked Write 5 Done Done Sent false
34290 10124 Acked Write 7 Done Done Sent false
34291 10125 Acked Write 5 Done Done Sent false
34292 10126 Acked Flush 0 Done Done Sent false
34293 10127 Acked Write 3 Done Done Sent false
34294 10128 Acked Read 1 Done Done Sent false
34295 10129 Acked Flush 0 Done Done Sent false
34296 10130 Acked Write 1 Done Done Sent false
34297 10131 Acked Read 2 Done Done Sent false
34298 10132 Acked Read 5 Done Done Sent false
34299 10133 Acked Write 4 Done Done Sent false
34300 10134 Acked Read 3 Done Done Sent false
34301 10135 Acked Read 2 Done Done Sent false
34302 10136 Acked Read 9 Done Done Sent false
34303 10137 Acked Read 10 Done Done Sent false
34304 10138 Acked Read 1 Done Done Sent false
34305 10139 Acked Write 3 Done Done Sent false
34306 10140 Acked Flush 0 Done Done Sent false
34307 10141 Acked Read 10 Done Done Sent false
34308 10142 Acked Read 1 Done Done Sent false
34309 10143 Acked Read 7 Done Done Sent false
34310 10144 Acked Read 4 Done Done Sent false
34311 10145 Acked Write 1 Done Done Sent false
34312 10146 Acked Read 5 Done Done Sent false
34313 10147 Acked Read 9 Done Done Sent false
34314 10148 Acked Read 6 Done Done Sent false
34315 10149 Acked Read 7 Done Done Sent false
34316 10150 Acked Read 5 Done Done Sent false
34317 10151 Acked Read 5 Done Done Sent false
34318 10152 Acked Read 5 Done Done Sent false
34319 10153 Acked Read 1 Done Done Sent false
34320 10154 Acked Write 2 Done Done Sent false
34321 10155 Acked Write 10 Done Done Sent false
34322 10156 Acked Write 3 Done Done Sent false
34323 10157 Acked Write 6 Done Done Sent false
34324 10158 Acked Write 10 Done Done Sent false
34325 10159 Acked Flush 0 Done Done Sent false
34326 10160 Acked Write 2 Done Done Sent false
34327 10161 Acked Read 9 Done Done Sent false
34328 10162 Acked Read 10 Done Done Sent false
34329 10163 Acked Write 7 Done Done Sent false
34330 10164 Acked Read 2 Done Done Sent false
34331 10165 Acked Write 6 Done Done Sent false
34332 10166 Acked Write 9 Done Done Sent false
34333 10167 Acked Read 6 Done Done Sent false
34334 10168 Acked Flush 0 Done Done Sent false
34335 10169 Acked Read 10 Done Done Sent false
34336 10170 Acked Read 5 Done Done Sent false
34337 10171 Acked Write 7 Done Done Sent false
34338 10172 Acked Write 1 Done Done Sent false
34339 10173 Acked Flush 0 Done Done Sent false
34340 10174 Acked Write 2 Done Done Sent false
34341 10175 Acked Read 3 Done Done Sent false
34342 10176 Acked Read 7 Done Done Sent false
34343 10177 Acked Write 4 Done Done Sent false
34344 10178 Acked Read 9 Done Done Sent false
34345 10179 Acked Read 4 Done Done Sent false
34346 10180 Acked Read 10 Done Done Sent false
34347 10181 Acked Read 4 Done Done Sent false
34348 10182 Acked Write 5 Done Done Sent false
34349 10183 Acked Read 9 Done Done Sent false
34350 10184 Acked Read 6 Done Done Sent false
34351 10185 Acked Flush 0 Done Done Sent false
34352 10186 Acked Read 1 Done Done Sent false
34353 10187 Acked Read 6 Done Done Sent false
34354 10188 Acked Write 2 Done Done Sent false
34355 10189 Acked Write 6 Done Done Sent false
34356 10190 Acked Write 6 Done Done Sent false
34357 10191 Acked Read 5 Done Done Sent false
34358 10192 Acked Write 7 Done Done Sent false
34359 10193 Acked Write 2 Done Done Sent false
34360 10194 Acked Write 6 Done Done Sent false
34361 10195 Acked Write 7 Done Done Sent false
34362 10196 Acked Write 9 Done Done Sent false
34363 10197 Acked Write 8 Done Done Sent false
34364 10198 Acked Read 6 Done Done Sent false
34365 10199 Acked Write 3 Done Done Sent false
34366 10200 Acked Read 10 Done Done Sent false
34367 10201 Acked Read 9 Done Done Sent false
34368 10202 Acked Read 8 Done Done Sent false
34369 10203 Acked Write 3 Done Done Sent false
34370 10204 Acked Read 3 Done Done Sent false
34371 10205 Acked Read 4 Done Done Sent false
34372 10206 Acked Write 10 Done Done Sent false
34373 10207 Acked Write 1 Done Done Sent false
34374 10208 Acked Read 9 Done Done Sent false
34375 10209 Acked Read 1 Done Done Sent false
34376 10210 Acked Read 9 Done Done Sent false
34377 10211 Acked Read 9 Done Done Sent false
34378 10212 Acked Flush 0 Done Done Sent false
34379 10213 Acked Read 4 Done Done Sent false
34380 10214 Acked Flush 0 Done Done Sent false
34381 10215 Acked Read 4 Done Done Sent false
34382 10216 Acked Read 6 Done Done Sent false
34383 10217 Acked Write 5 Done Done Sent false
34384 10218 Acked Write 5 Done Done Sent false
34385 10219 Acked Write 4 Done Done Sent false
34386 10220 Acked Write 9 Done Done Sent false
34387 10221 Acked Read 2 Done Done Sent false
34388 10222 Acked Read 5 Done Done Sent false
34389 10223 Acked Read 3 Done Done Sent false
34390 10224 Acked Write 6 Done Done Sent false
34391 10225 Acked Write 2 Done Done Sent false
34392 10226 Acked Read 3 Done Done Sent false
34393 10227 Acked Flush 0 Done Done Sent false
34394 10228 Acked Read 10 Done Done Sent false
34395 10229 Acked Flush 0 Done Done Sent false
34396 10230 Acked Write 3 Done Done Sent false
34397 10231 Acked Write 1 Done Done Sent false
34398 10232 Acked Write 6 Done Done Sent false
34399 10233 Acked Read 2 Done Done Sent false
34400 10234 Acked Write 10 Done Done Sent false
34401 10235 Acked Read 5 Done Done Sent false
34402 10236 Acked Flush 0 Done Done Sent false
34403 10237 Acked Write 4 Done Done Sent false
34404 10238 Acked Read 7 Done Done Sent false
34405 10239 Acked Read 9 Done Done Sent false
34406 10240 Acked Flush 0 Done Done Sent false
34407 10241 Acked Read 8 Done Done Sent false
34408 10242 Acked Write 10 Done Done Sent false
34409 10243 Acked Read 2 Done Done Sent false
34410 10244 Acked Write 5 Done Done Sent false
34411 10245 Acked Write 6 Done Done Sent false
34412 10246 Acked Write 3 Done Done Sent false
34413 10247 Acked Write 9 Done Done Sent false
34414 10248 Acked Read 7 Done Done Sent false
34415 10249 Acked Read 4 Done Done Sent false
34416 10250 Acked Read 5 Done Done Sent false
34417 10251 Acked Read 1 Done Done Sent false
34418 10252 Acked Write 8 Done Done Sent false
34419 10253 Acked Flush 0 Done Done Sent false
34420 10254 Acked Read 6 Done Done Sent false
34421 10255 Acked Read 8 Done Done Sent false
34422 10256 Acked Write 7 Done Done Sent false
34423 10257 Acked Write 5 Done Done Sent false
34424 10258 Acked Write 5 Done Done Sent false
34425 10259 Acked Flush 0 Done Done Sent false
34426 10260 Acked Read 1 Done Done Sent false
34427 10261 Acked Read 6 Done Done Sent false
34428 10262 Acked Write 3 Done Done Sent false
34429 10263 Acked Read 9 Done Done Sent false
34430 10264 Acked Read 3 Done Done Sent false
34431 10265 Acked Write 3 Done Done Sent false
34432 10266 Acked Read 5 Done Done Sent false
34433 10267 Acked Write 7 Done Done Sent false
34434 10268 Acked Write 1 Done Done Sent false
34435 10269 Acked Write 4 Done Done Sent false
34436 10270 Acked Read 2 Done Done Sent false
34437 10271 Acked Flush 0 Done Done Sent false
34438 10272 Acked Read 4 Done Done Sent false
34439 10273 Acked Write 3 Done Done Sent false
34440 10274 Acked Read 6 Done Done Sent false
34441 10275 Acked Flush 0 Done Done Sent false
34442 10276 Acked Write 1 Done Done Sent false
34443 10277 Acked Flush 0 Done Done Sent false
34444 10278 Acked Write 8 Done Done Sent false
34445 10279 Acked Read 2 Done Done Sent false
34446 10280 Acked Write 6 Done Done Sent false
34447 10281 Acked Read 9 Done Done Sent false
34448 10282 Acked Read 2 Done Done Sent false
34449 10283 Acked Flush 0 Done Done Sent false
34450 10284 Acked Flush 0 Done Done Sent false
34451 10285 Acked Read 1 Done Done Sent false
34452 10286 Acked Write 7 Done Done Sent false
34453 10287 Acked Write 1 Done Done Sent false
34454 10288 Acked Read 6 Done Done Sent false
34455 10289 Acked Write 9 Done Done Sent false
34456 10290 Acked Flush 0 Done Done Sent false
34457 10291 Acked Read 10 Done Done Sent false
34458 10292 Acked Read 10 Done Done Sent false
34459 10293 Acked Read 3 Done Done Sent false
34460 10294 Acked Flush 0 Done Done Sent false
34461 10295 Acked Write 6 Done Done Sent false
34462 10296 Acked Write 9 Done Done Sent false
34463 10297 Acked Write 1 Done Done Sent false
34464 10298 Acked Read 6 Done Done Sent false
34465 10299 Acked Write 6 Done Done Sent false
34466 10300 Acked Write 3 Done Done Sent false
34467 10301 Acked Write 9 Done Done Sent false
34468 10302 Acked Read 4 Done Done Sent false
34469 10303 Acked Write 10 Done Done Sent false
34470 10304 Acked Write 9 Done Done Sent false
34471 10305 Acked Write 1 Done Done Sent false
34472 10306 Acked Write 7 Done Done Sent false
34473 10307 Acked Write 8 Done Done Sent false
34474 10308 Acked Read 10 Done Done Sent false
34475 10309 Acked Read 6 Done Done Sent false
34476 10310 Acked Write 3 Done Done Sent false
34477 10311 Acked Read 4 Done Done Sent false
34478 10312 Acked Write 1 Done Done Sent false
34479 10313 Acked Write 1 Done Done Sent false
34480 10314 Acked Read 7 Done Done Sent false
34481 10315 Acked Write 1 Done Done Sent false
34482 10316 Acked Write 9 Done Done Sent false
34483 10317 Acked Read 5 Done Done Sent false
34484 10318 Acked Write 1 Done Done Sent false
34485 10319 Acked Write 10 Done Done Sent false
34486 10320 Acked Write 2 Done Done Sent false
34487 10321 Acked Read 8 Done Done Sent false
34488 10322 Acked Read 5 Done Done Sent false
34489 10323 Acked Read 3 Done Done Sent false
34490 10324 Acked Read 3 Done Done Sent false
34491 10325 Acked Write 7 Done Done Sent false
34492 10326 Acked Write 8 Done Done Sent false
34493 10327 Acked Write 3 Done Done Sent false
34494 10328 Acked Flush 0 Done Done Sent false
34495 10329 Acked Write 1 Done Done Sent false
34496 10330 Acked Write 6 Done Done Sent false
34497 10331 Acked Write 2 Done Done Sent false
34498 10332 Acked Read 4 Done Done Sent false
34499 10333 Acked Write 3 Done Done Sent false
34500 10334 Acked Read 3 Done Done Sent false
34501 10335 Acked Write 1 Done Done Sent false
34502 10336 Acked Read 5 Done Done Sent false
34503 10337 Acked Read 8 Done Done Sent false
34504 10338 Acked Read 1 Done Done Sent false
34505 10339 Acked Flush 0 Done Done Sent false
34506 10340 Acked Read 4 Done Done Sent false
34507 10341 Acked Write 8 Done Done Sent false
34508 10342 Acked Read 4 Done Done Sent false
34509 10343 Acked Read 3 Done Done Sent false
34510 10344 Acked Flush 0 Done Done Sent false
34511 10345 Acked Flush 0 Done Done Sent false
34512 10346 Acked Read 1 Done Done Sent false
34513 10347 Acked Write 8 Done Done Sent false
34514 10348 Acked Read 4 Done Done Sent false
34515 10349 Acked Flush 0 Done Done Sent false
34516 10350 Acked Read 3 Done Done Sent false
34517 10351 Acked Write 8 Done Done Sent false
34518 10352 Acked Write 10 Done Done Sent false
34519 10353 Acked Write 4 Done Done Sent false
34520 10354 Acked Flush 0 Done Done Sent false
34521 10355 Acked Read 7 Done Done Sent false
34522 10356 Acked Read 9 Done Done Sent false
34523 10357 Acked Read 2 Done Done Sent false
34524 STATES DS:0 DS:1 DS:2 TOTAL
34525 Sent 0 0 301 301
34526 Done 301 301 0 602
34527 Skipped 0 0 0 0
34528 Error 0 0 0 0
34529 Last Flush: 10354 10354 10056
34530 Downstairs last five completed: 10056 10055 10054 10053 10052
34531 Upstairs last five completed: 10357 10356 10355 10354 10353
34532 CLIENT: Up:0 ds:301 act:2
34533 [8830] got start action
34534 [8830] I am going to restart
34535 Starting downstairs at port 8830
34536 Make output file at "/tmp/dsc/downstairs-8830.txt"
34537 [8830][2] reports Starting
34538 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1233)
34539 [8830][2] reports Running
34540 ----------------------------------------------------------------
34541 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34542 States: Active Active Offline
34543 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34544 10057 Acked Flush 0 Done Done Sent false
34545 10058 Acked Write 1 Done Done Sent false
34546 10059 Acked Write 1 Done Done Sent false
34547 10060 Acked Read 8 Done Done Sent false
34548 10061 Acked Read 4 Done Done Sent false
34549 10062 Acked Flush 0 Done Done Sent false
34550 10063 Acked Read 5 Done Done Sent false
34551 10064 Acked Read 5 Done Done Sent false
34552 10065 Acked Read 3 Done Done Sent false
34553 10066 Acked Write 9 Done Done Sent false
34554 10067 Acked Read 5 Done Done Sent false
34555 10068 Acked Flush 0 Done Done Sent false
34556 10069 Acked Write 7 Done Done Sent false
34557 10070 Acked Flush 0 Done Done Sent false
34558 10071 Acked Read 5 Done Done Sent false
34559 10072 Acked Write 6 Done Done Sent false
34560 10073 Acked Write 2 Done Done Sent false
34561 10074 Acked Read 5 Done Done Sent false
34562 10075 Acked Read 3 Done Done Sent false
34563 10076 Acked Read 2 Done Done Sent false
34564 10077 Acked Write 4 Done Done Sent false
34565 10078 Acked Write 9 Done Done Sent false
34566 10079 Acked Read 9 Done Done Sent false
34567 10080 Acked Read 3 Done Done Sent false
34568 10081 Acked Read 2 Done Done Sent false
34569 10082 Acked Read 3 Done Done Sent false
34570 10083 Acked Write 5 Done Done Sent false
34571 10084 Acked Flush 0 Done Done Sent false
34572 10085 Acked Read 3 Done Done Sent false
34573 10086 Acked Read 5 Done Done Sent false
34574 10087 Acked Flush 0 Done Done Sent false
34575 10088 Acked Write 2 Done Done Sent false
34576 10089 Acked Read 2 Done Done Sent false
34577 10090 Acked Read 6 Done Done Sent false
34578 10091 Acked Flush 0 Done Done Sent false
34579 10092 Acked Write 2 Done Done Sent false
34580 10093 Acked Flush 0 Done Done Sent false
34581 10094 Acked Read 7 Done Done Sent false
34582 10095 Acked Write 3 Done Done Sent false
34583 10096 Acked Write 2 Done Done Sent false
34584 10097 Acked Write 7 Done Done Sent false
34585 10098 Acked Read 2 Done Done Sent false
34586 10099 Acked Write 3 Done Done Sent false
34587 10100 Acked Read 7 Done Done Sent false
34588 10101 Acked Read 1 Done Done Sent false
34589 10102 Acked Read 8 Done Done Sent false
34590 10103 Acked Read 7 Done Done Sent false
34591 10104 Acked Write 3 Done Done Sent false
34592 10105 Acked Read 7 Done Done Sent false
34593 10106 Acked Read 3 Done Done Sent false
34594 10107 Acked Write 3 Done Done Sent false
34595 10108 Acked Write 3 Done Done Sent false
34596 10109 Acked Read 1 Done Done Sent false
34597 10110 Acked Read 9 Done Done Sent false
34598 10111 Acked Read 4 Done Done Sent false
34599 10112 Acked Write 6 Done Done Sent false
34600 10113 Acked Flush 0 Done Done Sent false
34601 10114 Acked Read 3 Done Done Sent false
34602 10115 Acked Write 5 Done Done Sent false
34603 10116 Acked Read 5 Done Done Sent false
34604 10117 Acked Read 8 Done Done Sent false
34605 10118 Acked Write 3 Done Done Sent false
34606 10119 Acked Write 5 Done Done Sent false
34607 10120 Acked Write 2 Done Done Sent false
34608 10121 Acked Read 10 Done Done Sent false
34609 10122 Acked Read 8 Done Done Sent false
34610 10123 Acked Write 5 Done Done Sent false
34611 10124 Acked Write 7 Done Done Sent false
34612 10125 Acked Write 5 Done Done Sent false
34613 10126 Acked Flush 0 Done Done Sent false
34614 10127 Acked Write 3 Done Done Sent false
34615 10128 Acked Read 1 Done Done Sent false
34616 10129 Acked Flush 0 Done Done Sent false
34617 10130 Acked Write 1 Done Done Sent false
34618 10131 Acked Read 2 Done Done Sent false
34619 10132 Acked Read 5 Done Done Sent false
34620 10133 Acked Write 4 Done Done Sent false
34621 10134 Acked Read 3 Done Done Sent false
34622 10135 Acked Read 2 Done Done Sent false
34623 10136 Acked Read 9 Done Done Sent false
34624 10137 Acked Read 10 Done Done Sent false
34625 10138 Acked Read 1 Done Done Sent false
34626 10139 Acked Write 3 Done Done Sent false
34627 10140 Acked Flush 0 Done Done Sent false
34628 10141 Acked Read 10 Done Done Sent false
34629 10142 Acked Read 1 Done Done Sent false
34630 10143 Acked Read 7 Done Done Sent false
34631 10144 Acked Read 4 Done Done Sent false
34632 10145 Acked Write 1 Done Done Sent false
34633 10146 Acked Read 5 Done Done Sent false
34634 10147 Acked Read 9 Done Done Sent false
34635 10148 Acked Read 6 Done Done Sent false
34636 10149 Acked Read 7 Done Done Sent false
34637 10150 Acked Read 5 Done Done Sent false
34638 10151 Acked Read 5 Done Done Sent false
34639 10152 Acked Read 5 Done Done Sent false
34640 10153 Acked Read 1 Done Done Sent false
34641 10154 Acked Write 2 Done Done Sent false
34642 10155 Acked Write 10 Done Done Sent false
34643 10156 Acked Write 3 Done Done Sent false
34644 10157 Acked Write 6 Done Done Sent false
34645 10158 Acked Write 10 Done Done Sent false
34646 10159 Acked Flush 0 Done Done Sent false
34647 10160 Acked Write 2 Done Done Sent false
34648 10161 Acked Read 9 Done Done Sent false
34649 10162 Acked Read 10 Done Done Sent false
34650 10163 Acked Write 7 Done Done Sent false
34651 10164 Acked Read 2 Done Done Sent false
34652 10165 Acked Write 6 Done Done Sent false
34653 10166 Acked Write 9 Done Done Sent false
34654 10167 Acked Read 6 Done Done Sent false
34655 10168 Acked Flush 0 Done Done Sent false
34656 10169 Acked Read 10 Done Done Sent false
34657 10170 Acked Read 5 Done Done Sent false
34658 10171 Acked Write 7 Done Done Sent false
34659 10172 Acked Write 1 Done Done Sent false
34660 10173 Acked Flush 0 Done Done Sent false
34661 10174 Acked Write 2 Done Done Sent false
34662 10175 Acked Read 3 Done Done Sent false
34663 10176 Acked Read 7 Done Done Sent false
34664 10177 Acked Write 4 Done Done Sent false
34665 10178 Acked Read 9 Done Done Sent false
34666 10179 Acked Read 4 Done Done Sent false
34667 10180 Acked Read 10 Done Done Sent false
34668 10181 Acked Read 4 Done Done Sent false
34669 10182 Acked Write 5 Done Done Sent false
34670 10183 Acked Read 9 Done Done Sent false
34671 10184 Acked Read 6 Done Done Sent false
34672 10185 Acked Flush 0 Done Done Sent false
34673 10186 Acked Read 1 Done Done Sent false
34674 10187 Acked Read 6 Done Done Sent false
34675 10188 Acked Write 2 Done Done Sent false
34676 10189 Acked Write 6 Done Done Sent false
34677 10190 Acked Write 6 Done Done Sent false
34678 10191 Acked Read 5 Done Done Sent false
34679 10192 Acked Write 7 Done Done Sent false
34680 10193 Acked Write 2 Done Done Sent false
34681 10194 Acked Write 6 Done Done Sent false
34682 10195 Acked Write 7 Done Done Sent false
34683 10196 Acked Write 9 Done Done Sent false
34684 10197 Acked Write 8 Done Done Sent false
34685 10198 Acked Read 6 Done Done Sent false
34686 10199 Acked Write 3 Done Done Sent false
34687 10200 Acked Read 10 Done Done Sent false
34688 10201 Acked Read 9 Done Done Sent false
34689 10202 Acked Read 8 Done Done Sent false
34690 10203 Acked Write 3 Done Done Sent false
34691 10204 Acked Read 3 Done Done Sent false
34692 10205 Acked Read 4 Done Done Sent false
34693 10206 Acked Write 10 Done Done Sent false
34694 10207 Acked Write 1 Done Done Sent false
34695 10208 Acked Read 9 Done Done Sent false
34696 10209 Acked Read 1 Done Done Sent false
34697 10210 Acked Read 9 Done Done Sent false
34698 10211 Acked Read 9 Done Done Sent false
34699 10212 Acked Flush 0 Done Done Sent false
34700 10213 Acked Read 4 Done Done Sent false
34701 10214 Acked Flush 0 Done Done Sent false
34702 10215 Acked Read 4 Done Done Sent false
34703 10216 Acked Read 6 Done Done Sent false
34704 10217 Acked Write 5 Done Done Sent false
34705 10218 Acked Write 5 Done Done Sent false
34706 10219 Acked Write 4 Done Done Sent false
34707 10220 Acked Write 9 Done Done Sent false
34708 10221 Acked Read 2 Done Done Sent false
34709 10222 Acked Read 5 Done Done Sent false
34710 10223 Acked Read 3 Done Done Sent false
34711 10224 Acked Write 6 Done Done Sent false
34712 10225 Acked Write 2 Done Done Sent false
34713 10226 Acked Read 3 Done Done Sent false
34714 10227 Acked Flush 0 Done Done Sent false
34715 10228 Acked Read 10 Done Done Sent false
34716 10229 Acked Flush 0 Done Done Sent false
34717 10230 Acked Write 3 Done Done Sent false
34718 10231 Acked Write 1 Done Done Sent false
34719 10232 Acked Write 6 Done Done Sent false
34720 10233 Acked Read 2 Done Done Sent false
34721 10234 Acked Write 10 Done Done Sent false
34722 10235 Acked Read 5 Done Done Sent false
34723 10236 Acked Flush 0 Done Done Sent false
34724 10237 Acked Write 4 Done Done Sent false
34725 10238 Acked Read 7 Done Done Sent false
34726 10239 Acked Read 9 Done Done Sent false
34727 10240 Acked Flush 0 Done Done Sent false
34728 10241 Acked Read 8 Done Done Sent false
34729 10242 Acked Write 10 Done Done Sent false
34730 10243 Acked Read 2 Done Done Sent false
34731 10244 Acked Write 5 Done Done Sent false
34732 10245 Acked Write 6 Done Done Sent false
34733 10246 Acked Write 3 Done Done Sent false
34734 10247 Acked Write 9 Done Done Sent false
34735 10248 Acked Read 7 Done Done Sent false
34736 10249 Acked Read 4 Done Done Sent false
34737 10250 Acked Read 5 Done Done Sent false
34738 10251 Acked Read 1 Done Done Sent false
34739 10252 Acked Write 8 Done Done Sent false
34740 10253 Acked Flush 0 Done Done Sent false
34741 10254 Acked Read 6 Done Done Sent false
34742 10255 Acked Read 8 Done Done Sent false
34743 10256 Acked Write 7 Done Done Sent false
34744 10257 Acked Write 5 Done Done Sent false
34745 10258 Acked Write 5 Done Done Sent false
34746 10259 Acked Flush 0 Done Done Sent false
34747 10260 Acked Read 1 Done Done Sent false
34748 10261 Acked Read 6 Done Done Sent false
34749 10262 Acked Write 3 Done Done Sent false
34750 10263 Acked Read 9 Done Done Sent false
34751 10264 Acked Read 3 Done Done Sent false
34752 10265 Acked Write 3 Done Done Sent false
34753 10266 Acked Read 5 Done Done Sent false
34754 10267 Acked Write 7 Done Done Sent false
34755 10268 Acked Write 1 Done Done Sent false
34756 10269 Acked Write 4 Done Done Sent false
34757 10270 Acked Read 2 Done Done Sent false
34758 10271 Acked Flush 0 Done Done Sent false
34759 10272 Acked Read 4 Done Done Sent false
34760 10273 Acked Write 3 Done Done Sent false
34761 10274 Acked Read 6 Done Done Sent false
34762 10275 Acked Flush 0 Done Done Sent false
34763 10276 Acked Write 1 Done Done Sent false
34764 10277 Acked Flush 0 Done Done Sent false
34765 10278 Acked Write 8 Done Done Sent false
34766 10279 Acked Read 2 Done Done Sent false
34767 10280 Acked Write 6 Done Done Sent false
34768 10281 Acked Read 9 Done Done Sent false
34769 10282 Acked Read 2 Done Done Sent false
34770 10283 Acked Flush 0 Done Done Sent false
34771 10284 Acked Flush 0 Done Done Sent false
34772 10285 Acked Read 1 Done Done Sent false
34773 10286 Acked Write 7 Done Done Sent false
34774 10287 Acked Write 1 Done Done Sent false
34775 10288 Acked Read 6 Done Done Sent false
34776 10289 Acked Write 9 Done Done Sent false
34777 10290 Acked Flush 0 Done Done Sent false
34778 10291 Acked Read 10 Done Done Sent false
34779 10292 Acked Read 10 Done Done Sent false
34780 10293 Acked Read 3 Done Done Sent false
34781 10294 Acked Flush 0 Done Done Sent false
34782 10295 Acked Write 6 Done Done Sent false
34783 10296 Acked Write 9 Done Done Sent false
34784 10297 Acked Write 1 Done Done Sent false
34785 10298 Acked Read 6 Done Done Sent false
34786 10299 Acked Write 6 Done Done Sent false
34787 10300 Acked Write 3 Done Done Sent false
34788 10301 Acked Write 9 Done Done Sent false
34789 10302 Acked Read 4 Done Done Sent false
34790 10303 Acked Write 10 Done Done Sent false
34791 10304 Acked Write 9 Done Done Sent false
34792 10305 Acked Write 1 Done Done Sent false
34793 10306 Acked Write 7 Done Done Sent false
34794 10307 Acked Write 8 Done Done Sent false
34795 10308 Acked Read 10 Done Done Sent false
34796 10309 Acked Read 6 Done Done Sent false
34797 10310 Acked Write 3 Done Done Sent false
34798 10311 Acked Read 4 Done Done Sent false
34799 10312 Acked Write 1 Done Done Sent false
34800 10313 Acked Write 1 Done Done Sent false
34801 10314 Acked Read 7 Done Done Sent false
34802 10315 Acked Write 1 Done Done Sent false
34803 10316 Acked Write 9 Done Done Sent false
34804 10317 Acked Read 5 Done Done Sent false
34805 10318 Acked Write 1 Done Done Sent false
34806 10319 Acked Write 10 Done Done Sent false
34807 10320 Acked Write 2 Done Done Sent false
34808 10321 Acked Read 8 Done Done Sent false
34809 10322 Acked Read 5 Done Done Sent false
34810 10323 Acked Read 3 Done Done Sent false
34811 10324 Acked Read 3 Done Done Sent false
34812 10325 Acked Write 7 Done Done Sent false
34813 10326 Acked Write 8 Done Done Sent false
34814 10327 Acked Write 3 Done Done Sent false
34815 10328 Acked Flush 0 Done Done Sent false
34816 10329 Acked Write 1 Done Done Sent false
34817 10330 Acked Write 6 Done Done Sent false
34818 10331 Acked Write 2 Done Done Sent false
34819 10332 Acked Read 4 Done Done Sent false
34820 10333 Acked Write 3 Done Done Sent false
34821 10334 Acked Read 3 Done Done Sent false
34822 10335 Acked Write 1 Done Done Sent false
34823 10336 Acked Read 5 Done Done Sent false
34824 10337 Acked Read 8 Done Done Sent false
34825 10338 Acked Read 1 Done Done Sent false
34826 10339 Acked Flush 0 Done Done Sent false
34827 10340 Acked Read 4 Done Done Sent false
34828 10341 Acked Write 8 Done Done Sent false
34829 10342 Acked Read 4 Done Done Sent false
34830 10343 Acked Read 3 Done Done Sent false
34831 10344 Acked Flush 0 Done Done Sent false
34832 10345 Acked Flush 0 Done Done Sent false
34833 10346 Acked Read 1 Done Done Sent false
34834 10347 Acked Write 8 Done Done Sent false
34835 10348 Acked Read 4 Done Done Sent false
34836 10349 Acked Flush 0 Done Done Sent false
34837 10350 Acked Read 3 Done Done Sent false
34838 10351 Acked Write 8 Done Done Sent false
34839 10352 Acked Write 10 Done Done Sent false
34840 10353 Acked Write 4 Done Done Sent false
34841 10354 Acked Flush 0 Done Done Sent false
34842 10355 Acked Read 7 Done Done Sent false
34843 10356 Acked Read 9 Done Done Sent false
34844 10357 Acked Read 2 Done Done Sent false
34845 10358 Acked Flush 0 Done Done Sent false
34846 STATES DS:0 DS:1 DS:2 TOTAL
34847 Sent 0 0 302 302
34848 Done 302 302 0 604
34849 Skipped 0 0 0 0
34850 Error 0 0 0 0
34851 Last Flush: 10358 10358 10056
34852 Downstairs last five completed: 10056 10055 10054 10053 10052
34853 Upstairs last five completed: 10358 10357 10356 10355 10354
34854 CLIENT: Up:0 ds:302 act:2
348552024-11-19T17:15:17.138ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348562024-11-19T17:15:17.138ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34962) = io task client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348572024-11-19T17:15:17.139ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348582024-11-19T17:15:17.139ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348592024-11-19T17:15:17.139ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348602024-11-19T17:15:17.139ZINFOcrucible: send last flush ID to this DS: 10056 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348612024-11-19T17:15:17.140ZINFOcrucible: Replied this last flush ID: 10056 = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348622024-11-19T17:15:17.140ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348632024-11-19T17:15:17.140ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 05650a15-902e-469c-91e5-cba45f07470e
348642024-11-19T17:15:17.140ZINFOcrucible: [2] client re-new 302 jobs since flush 10056 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
348652024-11-19T17:15:17.140ZINFOcrucible: [2] Replayed 302 jobs since flush: 10056 = downstairs session_id = 05650a15-902e-469c-91e5-cba45f07470e
34866 ----------------------------------------------------------------
34867 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34868 Downstairs last five completed: 10358 10357 10356 10355 10354
34869 Upstairs last five completed: 10358 10357 10356 10355 10354
34870 CLIENT: Up:0 ds:0 act:3
34871 Replay: All jobs finished
34872 Test replay has completed
34873 Wrote out file "/tmp/test_replay_verify.log"
34874 CLIENT: Tests done. All submitted work has been ACK'd
34875 ----------------------------------------------------------------
34876 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34877 Downstairs last five completed: 10358 10357 10356 10355 10354
34878 Upstairs last five completed: 10358 10357 10356 10355 10354
34879 CLIENT: Up:0 ds:0 act:3
34880 CLIENT: All jobs finished, all DS active.
34881 Replays:30 time: 7:16
34882 Do final verify
34883 Nov 19 17:15:19.447 INFO accepted connection, remote_addr: 127.0.0.1:34640, local_addr: 127.0.0.1:9998
34884 Nov 19 17:15:19.447 INFO request completed, latency_us: 219, response_code: 200, uri: /regioninfo, method: GET, req_id: 2a4157ff-ae03-4322-8416-da3c523793a4, remote_addr: 127.0.0.1:34640, local_addr: 127.0.0.1:9998
348852024-11-19T17:15:19.448ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
34886 Nov 19 17:15:19.448 INFO request completed, latency_us: 108, response_code: 200, uri: /regioncount, method: GET, req_id: 3d879cbc-c030-4bac-b1ae-4db5a80d8dc7, remote_addr: 127.0.0.1:34640, local_addr: 127.0.0.1:9998
348872024-11-19T17:15:19.448ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
34888 Nov 19 17:15:19.449 INFO request completed, latency_us: 112, response_code: 200, uri: /port/cid/0, method: GET, req_id: c9b7fcd0-9186-4d75-b9db-5abb6f1f899d, remote_addr: 127.0.0.1:34640, local_addr: 127.0.0.1:9998
34889 Nov 19 17:15:19.449 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/1, method: GET, req_id: ead1a4c1-0d16-42a9-845a-ea41e4b6e338, remote_addr: 127.0.0.1:34640, local_addr: 127.0.0.1:9998
34890 Nov 19 17:15:19.450 INFO request completed, latency_us: 95, response_code: 200, uri: /port/cid/2, method: GET, req_id: 3b4e0845-1cea-4b13-997a-6f8e43db8373, remote_addr: 127.0.0.1:34640, local_addr: 127.0.0.1:9998
348912024-11-19T17:15:19.450ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
348922024-11-19T17:15:19.450ZINFOcrucible: Upstairs starts
348932024-11-19T17:15:19.450ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "775749f049384d6df9adbcf0df5fb1eb2a3a0b7e", git_commit_timestamp: "2024-11-19T16:38:47.000000000Z", git_branch: "mkeeter/early-io-rejection", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
348942024-11-19T17:15:19.450ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
348952024-11-19T17:15:19.451ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
348962024-11-19T17:15:19.451ZINFOcrucible: Crucible 9f30af27-6f84-4562-86ee-50a64d08d67d has session id: 3dcdb683-2f16-4498-91ac-a9976e131137 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
348972024-11-19T17:15:19.451ZINFOcrucible: Upstairs opts: Upstairs UUID: 9f30af27-6f84-4562-86ee-50a64d08d67d, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
348982024-11-19T17:15:19.451ZINFOcrucible: Crucible stats registered with UUID: 9f30af27-6f84-4562-86ee-50a64d08d67d session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
348992024-11-19T17:15:19.485ZINFOcrucible: Downstairs in volume = 3 task = crutest
349002024-11-19T17:15:19.485ZINFOcrucible: The guest has requested activation with gen:3
349012024-11-19T17:15:19.485ZINFOcrucible: 9f30af27-6f84-4562-86ee-50a64d08d67d active request set session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349022024-11-19T17:15:19.485ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349032024-11-19T17:15:19.485ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349042024-11-19T17:15:19.485ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349052024-11-19T17:15:19.485ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349062024-11-19T17:15:19.485ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349072024-11-19T17:15:19.485ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349082024-11-19T17:15:19.485ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349092024-11-19T17:15:19.485ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349102024-11-19T17:15:19.485ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349112024-11-19T17:15:19.485ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54146) = io task client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349122024-11-19T17:15:19.486ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62284) = io task client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349132024-11-19T17:15:19.486ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53248) = io task client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349142024-11-19T17:15:19.486ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349152024-11-19T17:15:19.486ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349162024-11-19T17:15:19.486ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349172024-11-19T17:15:19.486ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349182024-11-19T17:15:19.486ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349192024-11-19T17:15:19.486ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349202024-11-19T17:15:19.487ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349212024-11-19T17:15:19.487ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349222024-11-19T17:15:19.487ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349232024-11-19T17:15:19.488ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349242024-11-19T17:15:19.488ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349252024-11-19T17:15:19.488ZINFOcrucible: Waiting for more clients to be ready session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349262024-11-19T17:15:19.488ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349272024-11-19T17:15:19.488ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349282024-11-19T17:15:19.488ZINFOcrucible: Waiting for more clients to be ready session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349292024-11-19T17:15:19.488ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349302024-11-19T17:15:19.488ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349312024-11-19T17:15:19.488ZINFOcrucible: [0]R flush_numbers[0..12]: [949, 936, 958, 919, 953, 958, 963, 958, 958, 951, 958, 963] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349322024-11-19T17:15:19.488ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349332024-11-19T17:15:19.488ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349342024-11-19T17:15:19.488ZINFOcrucible: [1]R flush_numbers[0..12]: [949, 936, 958, 919, 953, 958, 963, 958, 958, 951, 958, 963] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349352024-11-19T17:15:19.488ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349362024-11-19T17:15:19.488ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349372024-11-19T17:15:19.488ZINFOcrucible: [2]R flush_numbers[0..12]: [949, 936, 958, 919, 953, 958, 963, 958, 958, 951, 958, 963] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349382024-11-19T17:15:19.488ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349392024-11-19T17:15:19.488ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349402024-11-19T17:15:19.488ZINFOcrucible: Max found gen is 3 = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
34941 {"msg":"Generation requested: 3 >= found:3","v":0,"name":"crucible","level":30,"time":"2024-11-19T17:15:19.488530303Z","hostname":"w-01JD2P7PDM2J8VSJ42EDFX8WH7","pid":1238,"":"downstairs","":"downstairs","Wait for a query_work_queue command to finish before sending IO
34942 session_id":"3dcdb683-2f16-4498-91ac-a9976e131137"}
349432024-11-19T17:15:19.488ZINFOcrucible: Next flush: 964 = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349442024-11-19T17:15:19.488ZINFOcrucible: All extents match = downstairs session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349452024-11-19T17:15:19.488ZINFOcrucible: No downstairs reconciliation required session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349462024-11-19T17:15:19.488ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349472024-11-19T17:15:19.488ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349482024-11-19T17:15:19.488ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349492024-11-19T17:15:19.488ZINFOcrucible: All required reconciliation work is completed session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349502024-11-19T17:15:19.488ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349512024-11-19T17:15:19.488ZINFOcrucible: 9f30af27-6f84-4562-86ee-50a64d08d67d is now active with session: 3dcdb683-2f16-4498-91ac-a9976e131137 session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349522024-11-19T17:15:19.488ZINFOcrucible: Set Active after no reconciliation session_id = 3dcdb683-2f16-4498-91ac-a9976e131137
349532024-11-19T17:15:19.488ZINFOcrucible: The guest has finished waiting for activation with:3
34954 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
34955 Loading write count information from file "/tmp/test_replay_verify.log"
34956 Read and Verify all blocks (0..5000 range:false)
34957 Wrote out file "/tmp/test_replay_verify.log"
34958 Verify test completed
34959 Wrote out file "/tmp/test_replay_verify.log"
34960 CLIENT: Tests done. All submitted work has been ACK'd
34961 ----------------------------------------------------------------
34962 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
34963 States: Active Active Active
34964 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34965 1000 Acked Read 256 Done Done Done false
34966 1001 Acked Read 256 Done Done Done false
34967 1002 Acked Read 256 Done Done Done false
34968 1003 Acked Read 256 Done Done Done false
34969 1004 Acked Read 256 Done Done Done false
34970 1005 Acked Read 256 Done Done Done false
34971 1006 Acked Read 256 Done Done Done false
34972 1007 Acked Read 256 Done Done Done false
34973 1008 Acked Read 256 Done Done Done false
34974 1009 Acked Read 256 Done Done Done false
34975 1010 Acked Read 256 Done Done Done false
34976 1011 Acked Read 256 Done Done Done false
34977 1012 Acked Read 256 Done Done Done false
34978 1013 Acked Read 256 Done Done Done false
34979 1014 Acked Read 256 Done Done Done false
34980 1015 Acked Read 256 Done Done Done false
34981 1016 Acked Read 256 Done Done Done false
34982 1017 Acked Read 256 Done Done Done false
34983 1018 Acked Read 256 Done Done Done false
34984 1019 Acked Read 136 Done Done Done false
34985 STATES DS:0 DS:1 DS:2 TOTAL
34986 Sent 0 0 0 0
34987 Done 20 20 20 60
34988 Skipped 0 0 0 0
34989 Error 0 0 0 0
34990 Last Flush: 0 0 0
34991 Downstairs last five completed:
34992 Upstairs last five completed: 1019 1018 1017 1016 1015
34993 CLIENT: Up:0 ds:20 act:3
34994 ----------------------------------------------------------------
34995 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
34996 Downstairs last five completed: 1020 1019 1018 1017 1016
34997 Upstairs last five completed: 1020 1019 1018 1017 1016
34998 CLIENT: Up:0 ds:0 act:3
34999 CLIENT: All crucible jobs finished, exiting program
35000 Nov 19 17:15:23.960 INFO accepted connection, remote_addr: 127.0.0.1:46785, local_addr: 127.0.0.1:9998
35001 Main task has work to do, go find it
35002 got dsc Shutdown
35003 Shutdown
35004 Nov 19 17:15:23.961Shut it down
35005 [8810] Disable keep_running
35006 INFO request completed, [8810] Got stop action so:false kr:false
35007 latency_us[8820] Disable keep_running
35008 : 171, response_code: 204, uri: /shutdown, method: POST, req_id: 27d8e90f-d331-4e06-8a9d-bbdfb189f33e, remote_addr: 127.0.0.1:46785, local_addr: 127.0.0.1:9998
35009 [8820] Got stop action so:false kr:false
35010 [8830] Disable keep_running
35011 [8830] Got stop action so:false kr:false
35012 [8810] Exited with: ExitStatus(unix_wait_status(9))
35013 [8820] Exited with: ExitStatus(unix_wait_status(9))
35014 [8810][0] reports Exit
35015 [8830] Exited with: ExitStatus(unix_wait_status(9))
35016 [8820][1] reports Exit
35017 [8830][2] reports Exit
35018 November 19, 2024 at 05:15:28 PM UTC Test ends with 0