Max level shown:
1 starting December 11, 2024 at 11:40:24 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.2758532
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.021514554
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.02011326
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start access at:127.0.0.1:9998
14 start ds: 8820
15 start ds: 8830
16 Starting downstairs at port 8810
17 Starting downstairs at port 8820
18 Make output file at "/tmp/dsc/downstairs-8810.txt"
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Starting downstairs at port 8830
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8810][0] initial start wait reports Starting
23 [8820][1] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1190)
26 [8830][2] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1191)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1192)
30 [8810][0] initial start wait reports Running
31 All downstairs are running
32 Dec 11 23:40:24.763 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Dec 11 23:40:29.833 INFO accepted connection, remote_addr: 127.0.0.1:56540, local_addr: 127.0.0.1:9998
36 Dec 11 23:40:29.837 INFO request completed, latency_us: 1302, response_code: 200, uri: /regioninfo, method: GET, req_id: e5bc7a21-957a-4254-863f-0ae83ff26c67, remote_addr: 127.0.0.1:56540, local_addr: 127.0.0.1:9998
37 Dec 11 23:40:29.839 INFO request completed, latency_us: 114, response_code: 200, uri: /regioncount, method: GET, req_id: a4908986-0810-49b7-a983-725c0ccea93d, remote_addr: 127.0.0.1:56540, local_addr: 127.0.0.1:9998
382024-12-11T23:40:29.839ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392024-12-11T23:40:29.840ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Dec 11 23:40:29.840 INFO request completed, latency_us: 187, response_code: 200, uri: /port/cid/0, method: GET, req_id: 9b53bd88-dac3-48fc-95de-9411390dfa45, remote_addr: 127.0.0.1:56540, local_addr: 127.0.0.1:9998
41 Dec 11 23:40:29.841 INFO request completed, latency_us: 101, response_code: 200, uri: /port/cid/1, method: GET, req_id: d744283d-da69-49d8-9380-7babca845107, remote_addr: 127.0.0.1:56540, local_addr: 127.0.0.1:9998
42 Dec 11 23:40:29.841 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/2, method: GET, req_id: e3e0d762-6f97-499f-9f0a-6f5a8aa0a1a7, remote_addr: 127.0.0.1:56540, local_addr: 127.0.0.1:9998
432024-12-11T23:40:29.842ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442024-12-11T23:40:29.842ZINFOcrucible: Upstairs starts
452024-12-11T23:40:29.843ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "d81f7fe7d3a87a2a3c58f4f5d619f7f37b0a05d0", git_commit_timestamp: "2024-12-11T23:24:14.000000000Z", git_branch: "mkeeter/separate-notify-task", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462024-12-11T23:40:29.843ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472024-12-11T23:40:29.843ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482024-12-11T23:40:29.843ZINFOcrucible: Crucible 97a67375-c677-4f56-81e4-386b5b47aa66 has session id: 0d15d2c4-fe62-4044-9d5f-787ecedca8e0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
492024-12-11T23:40:29.844ZINFOcrucible: Upstairs opts: Upstairs UUID: 97a67375-c677-4f56-81e4-386b5b47aa66, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
502024-12-11T23:40:29.844ZINFOcrucible: Crucible stats registered with UUID: 97a67375-c677-4f56-81e4-386b5b47aa66 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
512024-12-11T23:40:29.844ZWARNcrucible: could not find Downstairs address for Nexus session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
522024-12-11T23:40:29.845ZINFOcrucible: Downstairs in volume = 3 task = crutest
532024-12-11T23:40:29.846ZINFOcrucible: The guest has requested activation with gen:1
542024-12-11T23:40:29.846ZINFOcrucible: 97a67375-c677-4f56-81e4-386b5b47aa66 active request set session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
552024-12-11T23:40:29.846ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
562024-12-11T23:40:29.846ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
572024-12-11T23:40:29.846ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
582024-12-11T23:40:29.846ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
592024-12-11T23:40:29.846ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
602024-12-11T23:40:29.846ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
612024-12-11T23:40:29.846ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
622024-12-11T23:40:29.846ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
632024-12-11T23:40:29.846ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
642024-12-11T23:40:29.846ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60598) = io task client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
652024-12-11T23:40:29.846ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60560) = io task client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
662024-12-11T23:40:29.846ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51055) = io task client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
672024-12-11T23:40:29.848ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
682024-12-11T23:40:29.848ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
692024-12-11T23:40:29.848ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
702024-12-11T23:40:29.848ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
712024-12-11T23:40:29.849ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
722024-12-11T23:40:29.849ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
732024-12-11T23:40:29.850ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
742024-12-11T23:40:29.850ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
752024-12-11T23:40:29.850ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
762024-12-11T23:40:29.851ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
772024-12-11T23:40:29.851ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
782024-12-11T23:40:29.851ZINFOcrucible: Waiting for more clients to be ready session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
792024-12-11T23:40:29.851ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
802024-12-11T23:40:29.851ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
812024-12-11T23:40:29.851ZINFOcrucible: Waiting for more clients to be ready session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
822024-12-11T23:40:29.851ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
832024-12-11T23:40:29.851ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
842024-12-11T23:40:29.851ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
852024-12-11T23:40:29.851ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
862024-12-11T23:40:29.851ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
872024-12-11T23:40:29.851ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
882024-12-11T23:40:29.851ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
892024-12-11T23:40:29.851ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
902024-12-11T23:40:29.851ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
912024-12-11T23:40:29.851ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
922024-12-11T23:40:29.851ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
932024-12-11T23:40:29.851ZINFOcrucible: Max found gen is 1 = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
942024-12-11T23:40:29.851ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
952024-12-11T23:40:29.851ZINFOcrucible: Next flush: 1 = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
962024-12-11T23:40:29.851ZINFOcrucible: All extents match = downstairs session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
972024-12-11T23:40:29.851ZINFOcrucible: No downstairs reconciliation required session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
982024-12-11T23:40:29.851ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
992024-12-11T23:40:29.851ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
1002024-12-11T23:40:29.851ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
1012024-12-11T23:40:29.851ZINFOcrucible: All required reconciliation work is completed session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
1022024-12-11T23:40:29.851ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
1032024-12-11T23:40:29.851ZINFOcrucible: 97a67375-c677-4f56-81e4-386b5b47aa66 is now active with session: 0d15d2c4-fe62-4044-9d5f-787ecedca8e0 session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
1042024-12-11T23:40:29.851ZINFOcrucible: Set Active after no reconciliation session_id = 0d15d2c4-fe62-4044-9d5f-787ecedca8e0
1052024-12-11T23:40:29.851ZINFOcrucible: The guest has finished waiting for activation with:1
106 Activate successful
107 Wait for a query_work_queue command to finish before sending IO
108 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
109 Fill test
110 Read and Verify all blocks (0..5000 range:false)
111 Wrote out file "/tmp/test_replay_verify.log"
112 CLIENT: Tests done. All submitted work has been ACK'd
113 ----------------------------------------------------------------
114 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
115 States: Active Active Active
116 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
117 1021 Acked Read 256 Done Done Done false
118 1022 Acked Read 256 Done Done Done false
119 1023 Acked Read 256 Done Done Done false
120 1024 Acked Read 256 Done Done Done false
121 1025 Acked Read 256 Done Done Done false
122 1026 Acked Read 256 Done Done Done false
123 1027 Acked Read 256 Done Done Done false
124 1028 Acked Read 256 Done Done Done false
125 1029 Acked Read 256 Done Done Done false
126 1030 Acked Read 256 Done Done Done false
127 1031 Acked Read 256 Done Done Done false
128 1032 Acked Read 256 Done Done Done false
129 1033 Acked Read 256 Done Done Done false
130 1034 Acked Read 256 Done Done Done false
131 1035 Acked Read 256 Done Done Done false
132 1036 Acked Read 256 Done Done Done false
133 1037 Acked Read 256 Done Done Done false
134 1038 Acked Read 256 Done Done Done false
135 1039 Acked Read 256 Done Done Done false
136 1040 Acked Read 136 Done Done Done false
137 STATES DS:0 DS:1 DS:2 TOTAL
138 Sent 0 0 0 0
139 Done 20 20 20 60
140 Skipped 0 0 0 0
141 Error 0 0 0 0
142 Last Flush: 1020 1020 1020
143 Downstairs last five completed: 1020 1019 1018 1017 1016
144 Upstairs last five completed: 1040 1039 1038 1037 1036
145 CLIENT: Up:0 ds:20 act:3
146 ----------------------------------------------------------------
147 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
148 Downstairs last five completed: 1041 1040 1039 1038 1037
149 Upstairs last five completed: 1041 1040 1039 1038 1037
150 CLIENT: Up:0 ds:0 act:3
151 CLIENT: All crucible jobs finished, exiting program
1522024-12-11T23:40:34.406ZWARNcrucible: Guest handle has been dropped
153 Replay loop starts now December 11, 2024 at 11:40:34 PM UTC
154 Dec 11 23:40:34.481 INFO accepted connection, remote_addr: 127.0.0.1:64087, local_addr: 127.0.0.1:9998
155 Dec 11 23:40:34.482 INFO request completed, latency_us: 162, response_code: 200, uri: /regioninfo, method: GET, req_id: 22c2080c-ef8c-4175-9c6f-025bab57f6c9, remote_addr: 127.0.0.1:64087, local_addr: 127.0.0.1:9998
1562024-12-11T23:40:34.483ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
157 Dec 11 23:40:34.483 INFO request completed, latency_us: 105, response_code: 200, uri: /regioncount, method: GET, req_id: 70b8d87b-f034-4e8c-a433-5a86e64625f0, remote_addr: 127.0.0.1:64087, local_addr: 127.0.0.1:9998
1582024-12-11T23:40:34.483ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
159 Dec 11 23:40:34.484 INFO request completed, latency_us: 134, response_code: 200, uri: /port/cid/0, method: GET, req_id: 9bad4178-5ad4-4ea9-9a88-5e8e887ec1de, remote_addr: 127.0.0.1:64087, local_addr: 127.0.0.1:9998
160 Dec 11 23:40:34.484 INFO request completed, latency_us: 123, response_code: 200, uri: /port/cid/1, method: GET, req_id: 7962d090-2742-4e26-8bd4-744e1ed7b4cb, remote_addr: 127.0.0.1:64087, local_addr: 127.0.0.1:9998
161 Dec 11 23:40:34.485 INFO request completed, latency_us: 106, response_code: 200, uri: /port/cid/2, method: GET, req_id: 028093e0-8117-4d26-a071-3457b774f275, remote_addr: 127.0.0.1:64087, local_addr: 127.0.0.1:9998
1622024-12-11T23:40:34.485ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1632024-12-11T23:40:34.485ZINFOcrucible: Upstairs starts
1642024-12-11T23:40:34.485ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "d81f7fe7d3a87a2a3c58f4f5d619f7f37b0a05d0", git_commit_timestamp: "2024-12-11T23:24:14.000000000Z", git_branch: "mkeeter/separate-notify-task", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1652024-12-11T23:40:34.485ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1662024-12-11T23:40:34.485ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1672024-12-11T23:40:34.486ZINFOcrucible: Crucible ba9d6b53-efa3-47f7-abce-193512508430 has session id: b03dd6e3-2411-404a-b84f-1ea6c44cc21f session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1682024-12-11T23:40:34.486ZINFOcrucible: Upstairs opts: Upstairs UUID: ba9d6b53-efa3-47f7-abce-193512508430, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1692024-12-11T23:40:34.486ZINFOcrucible: Crucible stats registered with UUID: ba9d6b53-efa3-47f7-abce-193512508430 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1702024-12-11T23:40:34.486ZWARNcrucible: could not find Downstairs address for Nexus session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1712024-12-11T23:40:34.487ZINFOcrucible: Downstairs in volume = 3 task = crutest
1722024-12-11T23:40:34.487ZINFOcrucible: The guest has requested activation with gen:2
1732024-12-11T23:40:34.487ZINFOcrucible: ba9d6b53-efa3-47f7-abce-193512508430 active request set session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1742024-12-11T23:40:34.487ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1752024-12-11T23:40:34.487ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1762024-12-11T23:40:34.487ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1772024-12-11T23:40:34.487ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1782024-12-11T23:40:34.487ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1792024-12-11T23:40:34.487ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1802024-12-11T23:40:34.488ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1812024-12-11T23:40:34.488ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1822024-12-11T23:40:34.488ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1832024-12-11T23:40:34.488ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45741) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1842024-12-11T23:40:34.488ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61359) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1852024-12-11T23:40:34.488ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56210) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1862024-12-11T23:40:34.488ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1872024-12-11T23:40:34.488ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1882024-12-11T23:40:34.489ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1892024-12-11T23:40:34.489ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1902024-12-11T23:40:34.489ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1912024-12-11T23:40:34.489ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1922024-12-11T23:40:34.489ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1932024-12-11T23:40:34.489ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1942024-12-11T23:40:34.489ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1952024-12-11T23:40:34.490ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1962024-12-11T23:40:34.490ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1972024-12-11T23:40:34.490ZINFOcrucible: Waiting for more clients to be ready session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1982024-12-11T23:40:34.490ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1992024-12-11T23:40:34.490ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2002024-12-11T23:40:34.490ZINFOcrucible: Waiting for more clients to be ready session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2012024-12-11T23:40:34.490ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2022024-12-11T23:40:34.490ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2032024-12-11T23:40:34.490ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2042024-12-11T23:40:34.490ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2052024-12-11T23:40:34.490ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2062024-12-11T23:40:34.490ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2072024-12-11T23:40:34.490ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2082024-12-11T23:40:34.490ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2092024-12-11T23:40:34.490ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2102024-12-11T23:40:34.490ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2112024-12-11T23:40:34.490ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2122024-12-11T23:40:34.490ZINFOcrucible: Max found gen is 2 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2132024-12-11T23:40:34.490ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
214 {"msg":"Next flush: 2","v":0,"name":"crucible","level":30,"time":"Activate successful
215 Wait for a query_work_queue command to finish before sending IO
216 2024-12-11T23:40:34.490926456Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"downstairs","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
2172024-12-11T23:40:34.491ZINFOcrucible: All extents match = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2182024-12-11T23:40:34.491ZINFOcrucible: No downstairs reconciliation required session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2192024-12-11T23:40:34.491ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2202024-12-11T23:40:34.491ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2212024-12-11T23:40:34.491ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2222024-12-11T23:40:34.491ZINFOcrucible: All required reconciliation work is completed session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2232024-12-11T23:40:34.491ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
224 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
2252024-12-11T23:40:34.491ZINFOcrucible: ba9d6b53-efa3-47f7-abce-193512508430 is now active with session: b03dd6e3-2411-404a-b84f-1ea6c44cc21f session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2262024-12-11T23:40:34.491ZINFOcrucible: Set Active after no reconciliation session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2272024-12-11T23:40:34.491ZINFOcrucible: The guest has finished waiting for activation with:2
228 Loading write count information from file "/tmp/test_replay_verify.log"
229 Dec 11 23:40:34.526 INFO accepted connection, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
230 Main task has work to do, go find it
231 got dsc Stop(2)
232 Dec 11 23:40:34.526 INFO request completed, latency_us:stop 2
233 257, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 74f06f65-f547-4c17-9d2e-f7f9d67a3842, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
234 [8830] Got stop action so:false kr:false
2352024-12-11T23:40:34.527ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
236 Dec 11 23:40:34.527 INFO request completed, latency_us: 151, response_code: 200, uri: /state/cid/2, method: GET, req_id: 36a74dc4-619d-4df2-b6b4-7b8d6de666c1, remote_addr: 127.0.0.1:62969, local_addr: 127.0.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:40:34.527871998Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
237 0.1:9998
2382024-12-11T23:40:34.527ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2392024-12-11T23:40:34.527ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2402024-12-11T23:40:34.527ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2412024-12-11T23:40:34.528ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
242 [8830] Exited with: ExitStatus(unix_wait_status(9))
243 [8830][2] reports Exit
244 Dec 11 23:40:38.530 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/2, method: GET, req_id: df676da1-a944-47c8-b874-5d415a40c6dd, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
245 001/300 Read block 4934 len 40960
246 002/300 Read block 2952 len 4096
247 003/300 Read block 4657 len 32768
248 004/300 Write block 2610 len 32768 data: 2 2 2 2 2 2 2 2
249 005/300 Write block 2884 len 28672 data: 2 2 2 2 2 2 2
250 006/300 Write block 280 len 4096 data: 2
251 007/300 Read block 4662 len 20480
252 008/300 Flush
253 009/300 Write block 4585 len 40960 data: 2 2 2 2 2 2 2 2 2 2
254 010/300 Read block 4588 len 4096
255 011/300 Write block 3298 len 20480 data: 2 2 2 2 2
256 012/300 Read block 2824 len 12288
257 013/300 Read block 2844 len 12288
258 014/300 Write block 3841 len 4096 data: 2
259 015/300 Read block 4269 len 24576
260 016/300 Write block 1781 len 24576 data: 2 2 2 2 2 2
261 017/300 Write block 887 len 36864 data: 2 2 2 2 2 2 2 2 2
262 018/300 Read block 1857 len 32768
263 019/300 Write block 2573 len 28672 data: 2 2 2 2 2 2 2
264 020/300 Write block 1740 len 32768 data: 2 2 2 2 2 2 2 2
265 021/300 Read block 4066 len 8192
266 022/300 Flush
267 023/300 Write block 363 len 16384 data: 2 2 2 2
268 024/300 Read block 4791 len 32768
269 025/300 Write block 1979 len 28672 data: 2 2 2 2 2 2 2
270 026/300 Read block 2431 len 32768
271 027/300 Flush
272 028/300 Flush
273 029/300 Flush
274 030/300 Read block 3374 len 24576
275 031/300 Read block 499 len 24576
276 032/300 Flush
277 033/300 Read block 3727 len 40960
278 034/300 Read block 2701 len 24576
279 035/300 Write block 3856 len 4096 data: 2
280 036/300 Read block 4032 len 32768
281 037/300 Read block 3072 len 24576
282 038/300 Read block 942 len 4096
283 039/300 Write block 4657 len 24576 data: 2 2 2 2 2 2
284 040/300 Read block 4275 len 4096
285 041/300 Read block 1167 len 12288
286 042/300 Write block 118 len 32768 data: 2 2 2 2 2 2 2 2
287 043/300 Read block 3946 len 36864
288 044/300 Write block 1228 len 28672 data: 2 2 2 2 2 2 2
289 045/300 Write block 1429 len 4096 data: 2
290 046/300 Read block 2477 len 36864
291 047/300 Read block 2336 len 28672
292 048/300 Read block 2049 len 28672
293 049/300 Read block 951 len 12288
294 050/300 Write block 1161 len 28672 data: 2 2 2 2 2 2 2
295 051/300 Read block 2404 len 20480
296 052/300 Read block 2501 len 24576
297 053/300 Write block 2897 len 28672 data: 2 2 2 2 2 2 2
298 054/300 Flush
299 055/300 Read block 2466 len 20480
300 056/300 Write block 2470 len 12288 data: 2 2 2
301 057/300 Write block 3821 len 4096 data: 2
302 058/300 Write block 4 len 28672 data: 2 2 2 2 2 2 2
303 059/300 Read block 3998 len 24576
304 060/300 Read block 3916 len 20480
305 061/300 Read block 1521 len 28672
306 062/300 Read block 715 len 12288
307 063/300 Read block 4208 len 12288
308 064/300 Write block 1889 len 32768 data: 2 2 2 2 2 2 2 2
309 065/300 Flush
310 066/300 Read block 4960 len 40960
311 067/300 Read block 1907 len 12288
312 068/300 Write block 231 len 4096 data: 2
313 069/300 Read block 1599 len 24576
314 070/300 Write block 3828 len 32768 data: 2 2 2 2 2 2 2 2
315 071/300 Write block 2855 len 32768 data: 2 2 2 2 2 2 2 2
316 072/300 Read block 269 len 40960
317 073/300 Read block 4021 len 16384
318 074/300 Read block 1755 len 8192
319 075/300 Flush
320 076/300 Write block 4442 len 20480 data: 2 2 2 2 2
321 077/300 Write block 665 len 24576 data: 2 2 2 2 2 2
322 078/300 Write block 4528 len 20480 data: 2 2 2 2 2
323 079/300 Write block 2733 len 40960 data: 2 2 2 2 2 2 2 2 2 2
324 080/300 Read block 1472 len 16384
325 081/300 Read block 3815 len 20480
326 082/300 Read block 4142 len 20480
327 083/300 Write block 4378 len 32768 data: 2 2 2 2 2 2 2 2
328 084/300 Write block 4791 len 40960 data: 2 2 2 2 2 2 2 2 2 2
329 085/300 Read block 1333 len 32768
330 086/300 Read block 3366 len 4096
331 087/300 Read block 3886 len 8192
332 088/300 Write block 4399 len 24576 data: 2 2 2 2 2 2
333 089/300 Write block 2514 len 32768 data: 2 2 2 2 2 2 2 2
334 090/300 Read block 1763 len 20480
335 091/300 Read block 2155 len 32768
336 092/300 Write block 4857 len 36864 data: 2 2 2 2 2 2 2 2 2
337 093/300 Write block 1087 len 24576 data: 2 2 2 2 2 2
338 094/300 Write block 3573 len 4096 data: 2
339 095/300 Write block 2996 len 40960 data: 2 2 2 2 2 2 2 2 2 2
340 096/300 Write block 2637 len 4096 data: 2
341 097/300 Write block 770 len 12288 data: 2 2 2
342 098/300 Read block 1946 len 36864
343 099/300 Read block 3178 len 24576
344 100/300 Read block 4814 len 16384
345 101/300 Flush
346 102/300 Write block 1575 len 8192 data: 2 2
347 103/300 Write block 3493 len 36864 data: 2 2 2 2 2 2 2 2 2
348 104/300 Read block 731 len 40960
349 105/300 Write block 223 len 12288 data: 2 2 2
350 106/300 Write block 1380 len 12288 data: 2 2 2
351 107/300 Read block 2017 len 8192
352 108/300 Write block 3958 len 32768 data: 2 2 2 2 2 2 2 2
353 109/300 Flush
354 110/300 Read block 880 len 36864
355 111/300 Write block 3744 len 32768 data: 2 2 2 2 2 2 2 2
356 112/300 Read block 682 len 16384
357 113/300 Read block 1445 len 28672
358 114/300 Write block 2689 len 36864 data: 2 2 2 2 2 2 2 2 2
359 115/300 Read block 916 len 20480
360 116/300 Write block 3129 len 40960 data: 2 2 2 2 2 2 2 2 2 2
361 117/300 Read block 3901 len 8192
362 118/300 Read block 4937 len 24576
363 119/300 Write block 2113 len 20480 data: 2 2 2 2 2
364 120/300 Write block 1908 len 24576 data: 2 2 2 2 2 2
365 121/300 Read block 4235 len 36864
366 122/300 Read block 2817 len 20480
367 123/300 Write block 3054 len 20480 data: 2 2 2 2 2
368 124/300 Write block 3892 len 40960 data: 2 2 2 2 2 2 2 2 2 2
369 125/300 Read block 4169 len 32768
370 126/300 Write block 3385 len 36864 data: 2 2 2 2 2 2 2 2 2
371 127/300 Write block 4914 len 4096 data: 2
372 128/300 Read block 80 len 8192
373 129/300 Read block 4689 len 8192
374 130/300 Read block 3610 len 40960
375 131/300 Read block 4060 len 20480
376 132/300 Write block 1880 len 40960 data: 2 2 2 2 2 2 2 2 2 3
377 133/300 Write block 1014 len 24576 data: 2 2 2 2 2 2
378 134/300 Write block 1926 len 24576 data: 2 2 2 2 2 2
379 135/300 Write block 3660 len 36864 data: 2 2 2 2 2 2 2 2 2
380 136/300 Write block 2332 len 24576 data: 2 2 2 2 2 2
381 137/300 Write block 3587 len 8192 data: 2 2
382 138/300 Write block 3136 len 32768 data: 3 3 3 2 2 2 2 2
383 139/300 Read block 3964 len 4096
384 140/300 Write block 3617 len 20480 data: 2 2 2 2 2
385 141/300 Read block 3686 len 16384
386 142/300 Write block 3310 len 32768 data: 2 2 2 2 2 2 2 2
387 143/300 Read block 1753 len 20480
388 144/300 Flush
389 145/300 Read block 4033 len 8192
390 146/300 Read block 4570 len 40960
391 147/300 Read block 3679 len 32768
392 148/300 Read block 4009 len 28672
393 149/300 Write block 1127 len 12288 data: 2 2 2
394 150/300 Read block 4554 len 24576
395 151/300 Write block 3223 len 24576 data: 2 2 2 2 2 2
396 152/300 Read block 1129 len 12288
397 153/300 Flush
398 154/300 Write block 4530 len 32768 data: 3 3 3 2 2 2 2 2
399 155/300 Read block 4828 len 12288
400 156/300 Flush
401 157/300 Flush
402 158/300 Write block 2930 len 36864 data: 2 2 2 2 2 2 2 2 2
403 159/300 Read block 410 len 4096
404 160/300 Write block 4904 len 8192 data: 2 2
405 161/300 Read block 1801 len 40960
406 162/300 Write block 1948 len 8192 data: 2 2
407 163/300 Read block 943 len 24576
408 164/300 Read block 4225 len 40960
409 165/300 Read block 1436 len 32768
410 166/300 Read block 3645 len 40960
411 167/300 Read block 984 len 28672
412 168/300 Write block 1923 len 20480 data: 2 2 2 3 3
413 169/300 Read block 4907 len 24576
414 170/300 Read block 3720 len 12288
415 171/300 Write block 1071 len 32768 data: 2 2 2 2 2 2 2 2
416 172/300 Write block 1055 len 12288 data: 2 2 2
417 173/300 Read block 1587 len 16384
418 174/300 Write block 1213 len 36864 data: 2 2 2 2 2 2 2 2 2
419 175/300 Read block 1415 len 12288
420 176/300 Flush
421 177/300 Flush
422 178/300 Read block 1042 len 32768
423 179/300 Read block 1567 len 36864
424 180/300 Write block 3158 len 32768 data: 2 2 2 2 2 2 2 2
425 181/300 Read block 595 len 12288
426 182/300 Write block 38 len 36864 data: 2 2 2 2 2 2 2 2 2
427 183/300 Write block 4220 len 40960 data: 2 2 2 2 2 2 2 2 2 2
428 184/300 Read block 780 len 8192
429 185/300 Read block 1072 len 16384
430 186/300 Read block 3298 len 4096
431 187/300 Read block 2690 len 20480
432 188/300 Read block 3010 len 20480
433 189/300 Flush
434 190/300 Read block 3724 len 36864
435 191/300 Read block 4077 len 40960
436 192/300 Write block 4199 len 20480 data: 2 2 2 2 2
437 193/300 Write block 1256 len 20480 data: 2 2 2 2 2
438 194/300 Read block 1637 len 16384
439 195/300 Read block 555 len 40960
440 196/300 Read block 3548 len 4096
441 197/300 Write block 1612 len 12288 data: 2 2 2
442 198/300 Read block 2866 len 40960
443 199/300 Read block 4112 len 16384
444 200/300 Write block 1081 len 24576 data: 2 2 2 2 2 2
445 201/300 Read block 4024 len 24576
446 202/300 Write block 2600 len 24576 data: 2 2 2 2 2 2
447 203/300 Read block 1170 len 28672
448 204/300 Read block 4542 len 24576
449 205/300 Read block 3035 len 20480
450 206/300 Flush
451 207/300 Read block 3628 len 4096
452 208/300 Read block 882 len 36864
453 209/300 Write block 937 len 24576 data: 2 2 2 2 2 2
454 210/300 Write block 2594 len 32768 data: 2 2 2 2 2 2 3 3
455 211/300 Flush
456 212/300 Read block 4196 len 12288
457 213/300 Read block 604 len 12288
458 214/300 Read block 2459 len 36864
459 215/300 Write block 2649 len 4096 data: 2
460 216/300 Read block 4736 len 12288
461 217/300 Write block 1257 len 36864 data: 3 3 3 3 2 2 2 2 2
462 218/300 Read block 3179 len 36864
463 219/300 Flush
464 220/300 Read block 2505 len 28672
465 221/300 Read block 220 len 40960
466 222/300 Flush
467 223/300 Write block 3213 len 20480 data: 2 2 2 2 2
468 224/300 Read block 2795 len 24576
469 225/300 Read block 4870 len 40960
470 226/300 Read block 2242 len 16384
471 227/300 Read block 3327 len 16384
472 228/300 Read block 1264 len 4096
473 229/300 Write block 4516 len 4096 data: 2
474 230/300 Read block 445 len 4096
475 231/300 Read block 1029 len 36864
476 232/300 Read block 235 len 8192
477 233/300 Read block 328 len 24576
478 234/300 Flush
479 235/300 Write block 339 len 16384 data: 2 2 2 2
480 236/300 Write block 3089 len 40960 data: 2 2 2 2 2 2 2 2 2 2
481 237/300 Write block 1426 len 24576 data: 2 2 2 3 2 2
482 238/300 Write block 2687 len 8192 data: 2 2
483 239/300 Write block 1511 len 4096 data: 2
484 240/300 Read block 3930 len 32768
485 241/300 Read block 1340 len 12288
486 242/300 Read block 4033 len 32768
487 243/300 Read block 236 len 32768
488 244/300 Write block 445 len 36864 data: 2 2 2 2 2 2 2 2 2
489 245/300 Write block 4497 len 8192 data: 2 2
490 246/300 Read block 1615 len 8192
491 247/300 Flush
492 248/300 Read block 4117 len 12288
493 249/300 Read block 1200 len 16384
494 250/300 Read block 1516 len 36864
495 251/300 Write block 4802 len 16384 data: 2 2 2 2
496 252/300 Read block 2797 len 32768
497 253/300 Read block 2251 len 24576
498 254/300 Read block 1726 len 28672
499 255/300 Read block 2535 len 40960
500 256/300 Write block 2038 len 28672 data: 2 2 2 2 2 2 2
501 257/300 Write block 667 len 8192 data: 3 3
502 258/300 Write block 2676 len 20480 data: 2 2 2 2 2
503 259/300 Write block 1481 len 20480 data: 2 2 2 2 2
504 260/300 Read block 1296 len 32768
505 261/300 Write block 148 len 8192 data: 2 2
506 262/300 Flush
507 263/300 Write block 4204 len 8192 data: 2 2
508 264/300 Write block 3593 len 8192 data: 2 2
509 265/300 Read block 4837 len 20480
510 266/300 Read block 1552 len 28672
511 267/300 Write block 3174 len 12288 data: 2 2 2
512 268/300 Flush
513 269/300 Flush
514 270/300 Write block 2267 len 28672 data: 2 2 2 2 2 2 2
515 271/300 Read block 4328 len 12288
516 272/300 Read block 3356 len 12288
517 273/300 Write block 515 len 32768 data: 2 2 2 2 2 2 2 2
518 274/300 Read block 3421 len 8192
519 275/300 Read block 229 len 24576
520 276/300 Read block 2966 len 40960
521 277/300 Write block 1147 len 28672 data: 2 2 2 2 2 2 2
522 278/300 Write block 2742 len 36864 data: 3 2 2 2 2 2 2 2 2
523 279/300 Write block 429 len 16384 data: 2 2 2 2
524 280/300 Read block 3488 len 40960
525 281/300 Write block 3698 len 8192 data: 2 2
526 282/300 Read block 4936 len 16384
527 283/300 Read block 354 len 16384
528 284/300 Write block 3751 len 24576 data: 3 2 2 2 2 2
529 285/300 Read block 1150 len 4096
530 286/300 Flush
531 287/300 Read block 744 len 28672
532 288/300 Write block 3130 len 4096 data: 3
533 289/300 Read block 1889 len 24576
534 290/300 Read block 1729 len 40960
535 291/300 Write block 1929 len 16384 data: 3 3 3 2
536 292/300 Write block 380 len 40960 data: 2 2 2 2 2 2 2 2 2 2
537 293/300 Read block 1607 len 8192
538 294/300 Read block 1967 len 36864
539 295/300 Write block 1733 len 12288 data: 2 2 2
540 296/300 Read block 2035 len 12288
541 297/300 Read block 1565 len 32768
542 298/300 Write block 114 len 36864 data: 2 2 2 2 3 3 3 3 3
543 299/300 Read block 4572 len 20480
544 300/300 Write block 3616 len 20480 data: 2 3 3 3 3
545 301/300 Read block 3883 len 28672
546 Main task has work to do, go find it
547 got dsc Start(2)
548 start 2
549 Dec 11 23:40:38.778 INFO request completed, latency_us: 246, response_code: 204, uri: /start/cid/2, method: POST, req_id: 857ccb3e-2308-4598-ada4-adf08e5db9db, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
550 Replay: started 2, returned:Ok(())
551 ----------------------------------------------------------------
552 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
553 States: Active Active Offline
554 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
555 1000 Acked Read 10 Done Done Sent false
556 1001 Acked Read 1 Done Done Sent false
557 1002 Acked Read 8 Done Done Sent false
558 1003 Acked Write 8 Done Done Sent false
559 1004 Acked Write 7 Done Done Sent false
560 1005 Acked Write 1 Done Done Sent false
561 1006 Acked Read 5 Done Done Sent false
562 1007 Acked Flush 0 Done Done Sent false
563 1008 Acked Write 10 Done Done Sent false
564 1009 Acked Read 1 Done Done Sent false
565 1010 Acked Write 5 Done Done Sent false
566 1011 Acked Read 3 Done Done Sent false
567 1012 Acked Read 3 Done Done Sent false
568 1013 Acked Write 1 Done Done Sent false
569 1014 Acked Read 6 Done Done Sent false
570 1015 Acked Write 6 Done Done Sent false
571 1016 Acked Write 9 Done Done Sent false
572 1017 Acked Read 8 Done Done Sent false
573 1018 Acked Write 7 Done Done Sent false
574 1019 Acked Write 8 Done Done Sent false
575 1020 Acked Read 2 Done Done Sent false
576 1021 Acked Flush 0 Done Done Sent false
577 1022 Acked Write 4 Done Done Sent false
578 1023 Acked Read 8 Done Done Sent false
579 1024 Acked Write 7 Done Done Sent false
580 1025 Acked Read 8 Done Done Sent false
581 1026 Acked Flush 0 Done Done Sent false
582 1027 Acked Flush 0 Done Done Sent false
583 1028 Acked Flush 0 Done Done Sent false
584 1029 Acked Read 6 Done Done Sent false
585 1030 Acked Read 6 Done Done Sent false
586 1031 Acked Flush 0 Done Done Sent false
587 1032 Acked Read 10 Done Done Sent false
588 1033 Acked Read 6 Done Done Sent false
589 1034 Acked Write 1 Done Done Sent false
590 1035 Acked Read 8 Done Done Sent false
591 1036 Acked Read 6 Done Done Sent false
592 1037 Acked Read 1 Done Done Sent false
593 1038 Acked Write 6 Done Done Sent false
594 1039 Acked Read 1 Done Done Sent false
595 1040 Acked Read 3 Done Done Sent false
596 1041 Acked Write 8 Done Done Sent false
597 1042 Acked Read 9 Done Done Sent false
598 1043 Acked Write 7 Done Done Sent false
599 1044 Acked Write 1 Done Done Sent false
600 1045 Acked Read 9 Done Done Sent false
601 1046 Acked Read 7 Done Done Sent false
602 1047 Acked Read 7 Done Done Sent false
603 1048 Acked Read 3 Done Done Sent false
604 1049 Acked Write 7 Done Done Sent false
605 1050 Acked Read 5 Done Done Sent false
606 1051 Acked Read 6 Done Done Sent false
607 1052 Acked Write 7 Done Done Sent false
608 1053 Acked Flush 0 Done Done Sent false
609 1054 Acked Read 5 Done Done Sent false
610 1055 Acked Write 3 Done Done Sent false
611 1056 Acked Write 1 Done Done Sent false
612 1057 Acked Write 7 Done Done Sent false
613 1058 Acked Read 6 Done Done Sent false
614 1059 Acked Read 5 Done Done Sent false
615 1060 Acked Read 7 Done Done Sent false
616 1061 Acked Read 3 Done Done Sent false
617 1062 Acked Read 3 Done Done Sent false
618 1063 Acked Write 8 Done Done Sent false
619 1064 Acked Flush 0 Done Done Sent false
620 1065 Acked Read 10 Done Done Sent false
621 1066 Acked Read 3 Done Done Sent false
622 1067 Acked Write 1 Done Done Sent false
623 1068 Acked Read 6 Done Done Sent false
624 1069 Acked Write 8 Done Done Sent false
625 1070 Acked Write 8 Done Done Sent false
626 1071 Acked Read 10 Done Done Sent false
627 1072 Acked Read 4 Done Done Sent false
628 1073 Acked Read 2 Done Done Sent false
629 1074 Acked Flush 0 Done Done Sent false
630 1075 Acked Write 5 Done Done Sent false
631 1076 Acked Write 6 Done Done Sent false
632 1077 Acked Write 5 Done Done Sent false
633 1078 Acked Write 10 Done Done Sent false
634 1079 Acked Read 4 Done Done Sent false
635 1080 Acked Read 5 Done Done Sent false
636 1081 Acked Read 5 Done Done Sent false
637 1082 Acked Write 8 Done Done Sent false
638 1083 Acked Write 10 Done Done Sent false
639 1084 Acked Read 8 Done Done Sent false
640 1085 Acked Read 1 Done Done Sent false
641 1086 Acked Read 2 Done Done Sent false
642 1087 Acked Write 6 Done Done Sent false
643 1088 Acked Write 8 Done Done Sent false
644 1089 Acked Read 5 Done Done Sent false
645 1090 Acked Read 8 Done Done Sent false
646 1091 Acked Write 9 Done Done Sent false
647 1092 Acked Write 6 Done Done Sent false
648 1093 Acked Write 1 Done Done Sent false
649 1094 Acked Write 10 Done Done Sent false
650 1095 Acked Write 1 Done Done Sent false
651 1096 Acked Write 3 Done Done Sent false
652 1097 Acked Read 9 Done Done Sent false
653 1098 Acked Read 6 Done Done Sent false
654 1099 Acked Read 4 Done Done Sent false
655 1100 Acked Flush 0 Done Done Sent false
656 1101 Acked Write 2 Done Done Sent false
657 1102 Acked Write 9 Done Done Sent false
658 1103 Acked Read 10 Done Done Sent false
659 1104 Acked Write 3 Done Done Sent false
660 1105 Acked Write 3 Done Done Sent false
661 1106 Acked Read 2 Done Done Sent false
662 1107 Acked Write 8 Done Done Sent false
663 1108 Acked Flush 0 Done Done Sent false
664 1109 Acked Read 9 Done Done Sent false
665 1110 Acked Write 8 Done Done Sent false
666 1111 Acked Read 4 Done Done Sent false
667 1112 Acked Read 7 Done Done Sent false
668 1113 Acked Write 9 Done Done Sent false
669 1114 Acked Read 5 Done Done Sent false
670 1115 Acked Write 10 Done Done Sent false
671 1116 Acked Read 2 Done Done Sent false
672 1117 Acked Read 6 Done Done Sent false
673 1118 Acked Write 5 Done Done Sent false
674 1119 Acked Write 6 Done Done Sent false
675 1120 Acked Read 9 Done Done Sent false
676 1121 Acked Read 5 Done Done Sent false
677 1122 Acked Write 5 Done Done Sent false
678 1123 Acked Write 10 Done Done Sent false
679 1124 Acked Read 8 Done Done Sent false
680 1125 Acked Write 9 Done Done Sent false
681 1126 Acked Write 1 Done Done Sent false
682 1127 Acked Read 2 Done Done Sent false
683 1128 Acked Read 2 Done Done Sent false
684 1129 Acked Read 10 Done Done Sent false
685 1130 Acked Read 5 Done Done Sent false
686 1131 Acked Write 10 Done Done Sent false
687 1132 Acked Write 6 Done Done Sent false
688 1133 Acked Write 6 Done Done Sent false
689 1134 Acked Write 9 Done Done Sent false
690 1135 Acked Write 6 Done Done Sent false
691 1136 Acked Write 2 Done Done Sent false
692 1137 Acked Write 8 Done Done Sent false
693 1138 Acked Read 1 Done Done Sent false
694 1139 Acked Write 5 Done Done Sent false
695 1140 Acked Read 4 Done Done Sent false
696 1141 Acked Write 8 Done Done Sent false
697 1142 Acked Read 5 Done Done Sent false
698 1143 Acked Flush 0 Done Done Sent false
699 1144 Acked Read 2 Done Done Sent false
700 1145 Acked Read 10 Done Done Sent false
701 1146 Acked Read 8 Done Done Sent false
702 1147 Acked Read 7 Done Done Sent false
703 1148 Acked Write 3 Done Done Sent false
704 1149 Acked Read 6 Done Done Sent false
705 1150 Acked Write 6 Done Done Sent false
706 1151 Acked Read 3 Done Done Sent false
707 1152 Acked Flush 0 Done Done Sent false
708 1153 Acked Write 8 Done Done Sent false
709 1154 Acked Read 3 Done Done Sent false
710 1155 Acked Flush 0 Done Done Sent false
711 1156 Acked Flush 0 Done Done Sent false
712 1157 Acked Write 9 Done Done Sent false
713 1158 Acked Read 1 Done Done Sent false
714 1159 Acked Write 2 Done Done Sent false
715 1160 Acked Read 10 Done Done Sent false
716 1161 Acked Write 2 Done Done Sent false
717 1162 Acked Read 6 Done Done Sent false
718 1163 Acked Read 10 Done Done Sent false
719 1164 Acked Read 8 Done Done Sent false
720 1165 Acked Read 10 Done Done Sent false
721 1166 Acked Read 7 Done Done Sent false
722 1167 Acked Write 5 Done Done Sent false
723 1168 Acked Read 6 Done Done Sent false
724 1169 Acked Read 3 Done Done Sent false
725 1170 Acked Write 8 Done Done Sent false
726 1171 Acked Write 3 Done Done Sent false
727 1172 Acked Read 4 Done Done Sent false
728 1173 Acked Write 9 Done Done Sent false
729 1174 Acked Read 3 Done Done Sent false
730 1175 Acked Flush 0 Done Done Sent false
731 1176 Acked Flush 0 Done Done Sent false
732 1177 Acked Read 8 Done Done Sent false
733 1178 Acked Read 9 Done Done Sent false
734 1179 Acked Write 8 Done Done Sent false
735 1180 Acked Read 3 Done Done Sent false
736 1181 Acked Write 9 Done Done Sent false
737 1182 Acked Write 10 Done Done Sent false
738 1183 Acked Read 2 Done Done Sent false
739 1184 Acked Read 4 Done Done Sent false
740 1185 Acked Read 1 Done Done Sent false
741 1186 Acked Read 5 Done Done Sent false
742 1187 Acked Read 5 Done Done Sent false
743 1188 Acked Flush 0 Done Done Sent false
744 1189 Acked Read 9 Done Done Sent false
745 1190 Acked Read 10 Done Done Sent false
746 1191 Acked Write 5 Done Done Sent false
747 1192 Acked Write 5 Done Done Sent false
748 1193 Acked Read 4 Done Done Sent false
749 1194 Acked Read 10 Done Done Sent false
750 1195 Acked Read 1 Done Done Sent false
751 1196 Acked Write 3 Done Done Sent false
752 1197 Acked Read 10 Done Done Sent false
753 1198 Acked Read 4 Done Done Sent false
754 1199 Acked Write 6 Done Done Sent false
755 1200 Acked Read 6 Done Done Sent false
756 1201 Acked Write 6 Done Done Sent false
757 1202 Acked Read 7 Done Done Sent false
758 1203 Acked Read 6 Done Done Sent false
759 1204 Acked Read 5 Done Done Sent false
760 1205 Acked Flush 0 Done Done Sent false
761 1206 Acked Read 1 Done Done Sent false
762 1207 Acked Read 9 Done Done Sent false
763 1208 Acked Write 6 Done Done Sent false
764 1209 Acked Write 8 Done Done Sent false
765 1210 Acked Flush 0 Done Done Sent false
766 1211 Acked Read 3 Done Done Sent false
767 1212 Acked Read 3 Done Done Sent false
768 1213 Acked Read 9 Done Done Sent false
769 1214 Acked Write 1 Done Done Sent false
770 1215 Acked Read 3 Done Done Sent false
771 1216 Acked Write 9 Done Done Sent false
772 1217 Acked Read 9 Done Done Sent false
773 1218 Acked Flush 0 Done Done Sent false
774 1219 Acked Read 7 Done Done Sent false
775 1220 Acked Read 10 Done Done Sent false
776 1221 Acked Flush 0 Done Done Sent false
777 1222 Acked Write 5 Done Done Sent false
778 1223 Acked Read 6 Done Done Sent false
779 1224 Acked Read 10 Done Done Sent false
780 1225 Acked Read 4 Done Done Sent false
781 1226 Acked Read 4 Done Done Sent false
782 1227 Acked Read 1 Done Done Sent false
783 1228 Acked Write 1 Done Done Sent false
784 1229 Acked Read 1 Done Done Sent false
785 1230 Acked Read 9 Done Done Sent false
786 1231 Acked Read 2 Done Done Sent false
787 1232 Acked Read 6 Done Done Sent false
788 1233 Acked Flush 0 Done Done Sent false
789 1234 Acked Write 4 Done Done Sent false
790 1235 Acked Write 10 Done Done Sent false
791 1236 Acked Write 6 Done Done Sent false
792 1237 Acked Write 2 Done Done Sent false
793 1238 Acked Write 1 Done Done Sent false
794 1239 Acked Read 8 Done Done Sent false
795 1240 Acked Read 3 Done Done Sent false
796 1241 Acked Read 8 Done Done Sent false
797 1242 Acked Read 8 Done Done Sent false
798 1243 Acked Write 9 Done Done Sent false
799 1244 Acked Write 2 Done Done Sent false
800 1245 Acked Read 2 Done Done Sent false
801 1246 Acked Flush 0 Done Done Sent false
802 1247 Acked Read 3 Done Done Sent false
803 1248 Acked Read 4 Done Done Sent false
804 1249 Acked Read 9 Done Done Sent false
805 1250 Acked Write 4 Done Done Sent false
806 1251 Acked Read 8 Done Done Sent false
807 1252 Acked Read 6 Done Done Sent false
808 1253 Acked Read 7 Done Done Sent false
809 1254 Acked Read 10 Done Done Sent false
810 1255 Acked Write 7 Done Done Sent false
811 1256 Acked Write 2 Done Done Sent false
812 1257 Acked Write 5 Done Done Sent false
813 1258 Acked Write 5 Done Done Sent false
814 1259 Acked Read 8 Done Done Sent false
815 1260 Acked Write 2 Done Done Sent false
816 1261 Acked Flush 0 Done Done Sent false
817 1262 Acked Write 2 Done Done Sent false
818 1263 Acked Write 2 Done Done Sent false
819 1264 Acked Read 5 Done Done Sent false
820 1265 Acked Read 7 Done Done Sent false
821 1266 Acked Write 3 Done Done Sent false
822 1267 Acked Flush 0 Done Done Sent false
823 1268 Acked Flush 0 Done Done Sent false
824 1269 Acked Write 7 Done Done Sent false
825 1270 Acked Read 3 Done Done Sent false
826 1271 Acked Read 3 Done Done Sent false
827 1272 Acked Write 8 Done Done Sent false
828 1273 Acked Read 2 Done Done Sent false
829 1274 Acked Read 6 Done Done Sent false
830 1275 Acked Read 10 Done Done Sent false
831 1276 Acked Write 7 Done Done Sent false
832 1277 Acked Write 9 Done Done Sent false
833 1278 Acked Write 4 Done Done Sent false
834 1279 Acked Read 10 Done Done Sent false
835 1280 Acked Write 2 Done Done Sent false
836 1281 Acked Read 4 Done Done Sent false
837 1282 Acked Read 4 Done Done Sent false
838 1283 Acked Write 6 Done Done Sent false
839 1284 Acked Read 1 Done Done Sent false
840 1285 Acked Flush 0 Done Done Sent false
841 1286 Acked Read 7 Done Done Sent false
842 1287 Acked Write 1 Done Done Sent false
843 1288 Acked Read 6 Done Done Sent false
844 1289 Acked Read 10 Done Done Sent false
845 1290 Acked Write 4 Done Done Sent false
846 1291 Acked Write 10 Done Done Sent false
847 1292 Acked Read 2 Done Done Sent false
848 1293 Acked Read 9 Done Done Sent false
849 1294 Acked Write 3 Done Done Sent false
850 1295 Acked Read 3 Done Done Sent false
851 1296 Acked Read 8 Done Done Sent false
852 1297 Acked Write 9 Done Done Sent false
853 1298 Acked Read 5 Done Done Sent false
854 1299 Acked Write 5 Done Done Sent false
855 1300 Acked Read 7 Done Done Sent false
856 STATES DS:0 DS:1 DS:2 TOTAL
857 Sent 0 0 301 301
858 Done 301 301 0 602
859 Skipped 0 0 0 0
860 Error 0 0 0 0
861 Last Flush: 1285 1285 0
862 Downstairs last five completed:
863 Upstairs last five completed: 1300 1299 1298 1297 1296
864 CLIENT: Up:0 ds:301 act:2
865 [8830] got start action
866 [8830] I am going to restart
867 Starting downstairs at port 8830
868 Make output file at "/tmp/dsc/downstairs-8830.txt"
869 [8830][2] reports Starting
870 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1201)
871 [8830][2] reports Running
872 ----------------------------------------------------------------
873 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
874 States: Active Active Offline
875 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
876 1000 Acked Read 10 Done Done Sent false
877 1001 Acked Read 1 Done Done Sent false
878 1002 Acked Read 8 Done Done Sent false
879 1003 Acked Write 8 Done Done Sent false
880 1004 Acked Write 7 Done Done Sent false
881 1005 Acked Write 1 Done Done Sent false
882 1006 Acked Read 5 Done Done Sent false
883 1007 Acked Flush 0 Done Done Sent false
884 1008 Acked Write 10 Done Done Sent false
885 1009 Acked Read 1 Done Done Sent false
886 1010 Acked Write 5 Done Done Sent false
887 1011 Acked Read 3 Done Done Sent false
888 1012 Acked Read 3 Done Done Sent false
889 1013 Acked Write 1 Done Done Sent false
890 1014 Acked Read 6 Done Done Sent false
891 1015 Acked Write 6 Done Done Sent false
892 1016 Acked Write 9 Done Done Sent false
893 1017 Acked Read 8 Done Done Sent false
894 1018 Acked Write 7 Done Done Sent false
895 1019 Acked Write 8 Done Done Sent false
896 1020 Acked Read 2 Done Done Sent false
897 1021 Acked Flush 0 Done Done Sent false
898 1022 Acked Write 4 Done Done Sent false
899 1023 Acked Read 8 Done Done Sent false
900 1024 Acked Write 7 Done Done Sent false
901 1025 Acked Read 8 Done Done Sent false
902 1026 Acked Flush 0 Done Done Sent false
903 1027 Acked Flush 0 Done Done Sent false
904 1028 Acked Flush 0 Done Done Sent false
905 1029 Acked Read 6 Done Done Sent false
906 1030 Acked Read 6 Done Done Sent false
907 1031 Acked Flush 0 Done Done Sent false
908 1032 Acked Read 10 Done Done Sent false
909 1033 Acked Read 6 Done Done Sent false
910 1034 Acked Write 1 Done Done Sent false
911 1035 Acked Read 8 Done Done Sent false
912 1036 Acked Read 6 Done Done Sent false
913 1037 Acked Read 1 Done Done Sent false
914 1038 Acked Write 6 Done Done Sent false
915 1039 Acked Read 1 Done Done Sent false
916 1040 Acked Read 3 Done Done Sent false
917 1041 Acked Write 8 Done Done Sent false
918 1042 Acked Read 9 Done Done Sent false
919 1043 Acked Write 7 Done Done Sent false
920 1044 Acked Write 1 Done Done Sent false
921 1045 Acked Read 9 Done Done Sent false
922 1046 Acked Read 7 Done Done Sent false
923 1047 Acked Read 7 Done Done Sent false
924 1048 Acked Read 3 Done Done Sent false
925 1049 Acked Write 7 Done Done Sent false
926 1050 Acked Read 5 Done Done Sent false
927 1051 Acked Read 6 Done Done Sent false
928 1052 Acked Write 7 Done Done Sent false
929 1053 Acked Flush 0 Done Done Sent false
930 1054 Acked Read 5 Done Done Sent false
931 1055 Acked Write 3 Done Done Sent false
932 1056 Acked Write 1 Done Done Sent false
933 1057 Acked Write 7 Done Done Sent false
934 1058 Acked Read 6 Done Done Sent false
935 1059 Acked Read 5 Done Done Sent false
936 1060 Acked Read 7 Done Done Sent false
937 1061 Acked Read 3 Done Done Sent false
938 1062 Acked Read 3 Done Done Sent false
939 1063 Acked Write 8 Done Done Sent false
940 1064 Acked Flush 0 Done Done Sent false
941 1065 Acked Read 10 Done Done Sent false
942 1066 Acked Read 3 Done Done Sent false
943 1067 Acked Write 1 Done Done Sent false
944 1068 Acked Read 6 Done Done Sent false
945 1069 Acked Write 8 Done Done Sent false
946 1070 Acked Write 8 Done Done Sent false
947 1071 Acked Read 10 Done Done Sent false
948 1072 Acked Read 4 Done Done Sent false
949 1073 Acked Read 2 Done Done Sent false
950 1074 Acked Flush 0 Done Done Sent false
951 1075 Acked Write 5 Done Done Sent false
952 1076 Acked Write 6 Done Done Sent false
953 1077 Acked Write 5 Done Done Sent false
954 1078 Acked Write 10 Done Done Sent false
955 1079 Acked Read 4 Done Done Sent false
956 1080 Acked Read 5 Done Done Sent false
957 1081 Acked Read 5 Done Done Sent false
958 1082 Acked Write 8 Done Done Sent false
959 1083 Acked Write 10 Done Done Sent false
960 1084 Acked Read 8 Done Done Sent false
961 1085 Acked Read 1 Done Done Sent false
962 1086 Acked Read 2 Done Done Sent false
963 1087 Acked Write 6 Done Done Sent false
964 1088 Acked Write 8 Done Done Sent false
965 1089 Acked Read 5 Done Done Sent false
966 1090 Acked Read 8 Done Done Sent false
967 1091 Acked Write 9 Done Done Sent false
968 1092 Acked Write 6 Done Done Sent false
969 1093 Acked Write 1 Done Done Sent false
970 1094 Acked Write 10 Done Done Sent false
971 1095 Acked Write 1 Done Done Sent false
972 1096 Acked Write 3 Done Done Sent false
973 1097 Acked Read 9 Done Done Sent false
974 1098 Acked Read 6 Done Done Sent false
975 1099 Acked Read 4 Done Done Sent false
976 1100 Acked Flush 0 Done Done Sent false
977 1101 Acked Write 2 Done Done Sent false
978 1102 Acked Write 9 Done Done Sent false
979 1103 Acked Read 10 Done Done Sent false
980 1104 Acked Write 3 Done Done Sent false
981 1105 Acked Write 3 Done Done Sent false
982 1106 Acked Read 2 Done Done Sent false
983 1107 Acked Write 8 Done Done Sent false
984 1108 Acked Flush 0 Done Done Sent false
985 1109 Acked Read 9 Done Done Sent false
986 1110 Acked Write 8 Done Done Sent false
987 1111 Acked Read 4 Done Done Sent false
988 1112 Acked Read 7 Done Done Sent false
989 1113 Acked Write 9 Done Done Sent false
990 1114 Acked Read 5 Done Done Sent false
991 1115 Acked Write 10 Done Done Sent false
992 1116 Acked Read 2 Done Done Sent false
993 1117 Acked Read 6 Done Done Sent false
994 1118 Acked Write 5 Done Done Sent false
995 1119 Acked Write 6 Done Done Sent false
996 1120 Acked Read 9 Done Done Sent false
997 1121 Acked Read 5 Done Done Sent false
998 1122 Acked Write 5 Done Done Sent false
999 1123 Acked Write 10 Done Done Sent false
1000 1124 Acked Read 8 Done Done Sent false
1001 1125 Acked Write 9 Done Done Sent false
1002 1126 Acked Write 1 Done Done Sent false
1003 1127 Acked Read 2 Done Done Sent false
1004 1128 Acked Read 2 Done Done Sent false
1005 1129 Acked Read 10 Done Done Sent false
1006 1130 Acked Read 5 Done Done Sent false
1007 1131 Acked Write 10 Done Done Sent false
1008 1132 Acked Write 6 Done Done Sent false
1009 1133 Acked Write 6 Done Done Sent false
1010 1134 Acked Write 9 Done Done Sent false
1011 1135 Acked Write 6 Done Done Sent false
1012 1136 Acked Write 2 Done Done Sent false
1013 1137 Acked Write 8 Done Done Sent false
1014 1138 Acked Read 1 Done Done Sent false
1015 1139 Acked Write 5 Done Done Sent false
1016 1140 Acked Read 4 Done Done Sent false
1017 1141 Acked Write 8 Done Done Sent false
1018 1142 Acked Read 5 Done Done Sent false
1019 1143 Acked Flush 0 Done Done Sent false
1020 1144 Acked Read 2 Done Done Sent false
1021 1145 Acked Read 10 Done Done Sent false
1022 1146 Acked Read 8 Done Done Sent false
1023 1147 Acked Read 7 Done Done Sent false
1024 1148 Acked Write 3 Done Done Sent false
1025 1149 Acked Read 6 Done Done Sent false
1026 1150 Acked Write 6 Done Done Sent false
1027 1151 Acked Read 3 Done Done Sent false
1028 1152 Acked Flush 0 Done Done Sent false
1029 1153 Acked Write 8 Done Done Sent false
1030 1154 Acked Read 3 Done Done Sent false
1031 1155 Acked Flush 0 Done Done Sent false
1032 1156 Acked Flush 0 Done Done Sent false
1033 1157 Acked Write 9 Done Done Sent false
1034 1158 Acked Read 1 Done Done Sent false
1035 1159 Acked Write 2 Done Done Sent false
1036 1160 Acked Read 10 Done Done Sent false
1037 1161 Acked Write 2 Done Done Sent false
1038 1162 Acked Read 6 Done Done Sent false
1039 1163 Acked Read 10 Done Done Sent false
1040 1164 Acked Read 8 Done Done Sent false
1041 1165 Acked Read 10 Done Done Sent false
1042 1166 Acked Read 7 Done Done Sent false
1043 1167 Acked Write 5 Done Done Sent false
1044 1168 Acked Read 6 Done Done Sent false
1045 1169 Acked Read 3 Done Done Sent false
1046 1170 Acked Write 8 Done Done Sent false
1047 1171 Acked Write 3 Done Done Sent false
1048 1172 Acked Read 4 Done Done Sent false
1049 1173 Acked Write 9 Done Done Sent false
1050 1174 Acked Read 3 Done Done Sent false
1051 1175 Acked Flush 0 Done Done Sent false
1052 1176 Acked Flush 0 Done Done Sent false
1053 1177 Acked Read 8 Done Done Sent false
1054 1178 Acked Read 9 Done Done Sent false
1055 1179 Acked Write 8 Done Done Sent false
1056 1180 Acked Read 3 Done Done Sent false
1057 1181 Acked Write 9 Done Done Sent false
1058 1182 Acked Write 10 Done Done Sent false
1059 1183 Acked Read 2 Done Done Sent false
1060 1184 Acked Read 4 Done Done Sent false
1061 1185 Acked Read 1 Done Done Sent false
1062 1186 Acked Read 5 Done Done Sent false
1063 1187 Acked Read 5 Done Done Sent false
1064 1188 Acked Flush 0 Done Done Sent false
1065 1189 Acked Read 9 Done Done Sent false
1066 1190 Acked Read 10 Done Done Sent false
1067 1191 Acked Write 5 Done Done Sent false
1068 1192 Acked Write 5 Done Done Sent false
1069 1193 Acked Read 4 Done Done Sent false
1070 1194 Acked Read 10 Done Done Sent false
1071 1195 Acked Read 1 Done Done Sent false
1072 1196 Acked Write 3 Done Done Sent false
1073 1197 Acked Read 10 Done Done Sent false
1074 1198 Acked Read 4 Done Done Sent false
1075 1199 Acked Write 6 Done Done Sent false
1076 1200 Acked Read 6 Done Done Sent false
1077 1201 Acked Write 6 Done Done Sent false
1078 1202 Acked Read 7 Done Done Sent false
1079 1203 Acked Read 6 Done Done Sent false
1080 1204 Acked Read 5 Done Done Sent false
1081 1205 Acked Flush 0 Done Done Sent false
1082 1206 Acked Read 1 Done Done Sent false
1083 1207 Acked Read 9 Done Done Sent false
1084 1208 Acked Write 6 Done Done Sent false
1085 1209 Acked Write 8 Done Done Sent false
1086 1210 Acked Flush 0 Done Done Sent false
1087 1211 Acked Read 3 Done Done Sent false
1088 1212 Acked Read 3 Done Done Sent false
1089 1213 Acked Read 9 Done Done Sent false
1090 1214 Acked Write 1 Done Done Sent false
1091 1215 Acked Read 3 Done Done Sent false
1092 1216 Acked Write 9 Done Done Sent false
1093 1217 Acked Read 9 Done Done Sent false
1094 1218 Acked Flush 0 Done Done Sent false
1095 1219 Acked Read 7 Done Done Sent false
1096 1220 Acked Read 10 Done Done Sent false
1097 1221 Acked Flush 0 Done Done Sent false
1098 1222 Acked Write 5 Done Done Sent false
1099 1223 Acked Read 6 Done Done Sent false
1100 1224 Acked Read 10 Done Done Sent false
1101 1225 Acked Read 4 Done Done Sent false
1102 1226 Acked Read 4 Done Done Sent false
1103 1227 Acked Read 1 Done Done Sent false
1104 1228 Acked Write 1 Done Done Sent false
1105 1229 Acked Read 1 Done Done Sent false
1106 1230 Acked Read 9 Done Done Sent false
1107 1231 Acked Read 2 Done Done Sent false
1108 1232 Acked Read 6 Done Done Sent false
1109 1233 Acked Flush 0 Done Done Sent false
1110 1234 Acked Write 4 Done Done Sent false
1111 1235 Acked Write 10 Done Done Sent false
1112 1236 Acked Write 6 Done Done Sent false
1113 1237 Acked Write 2 Done Done Sent false
1114 1238 Acked Write 1 Done Done Sent false
1115 1239 Acked Read 8 Done Done Sent false
1116 1240 Acked Read 3 Done Done Sent false
1117 1241 Acked Read 8 Done Done Sent false
1118 1242 Acked Read 8 Done Done Sent false
1119 1243 Acked Write 9 Done Done Sent false
1120 1244 Acked Write 2 Done Done Sent false
1121 1245 Acked Read 2 Done Done Sent false
1122 1246 Acked Flush 0 Done Done Sent false
1123 1247 Acked Read 3 Done Done Sent false
1124 1248 Acked Read 4 Done Done Sent false
1125 1249 Acked Read 9 Done Done Sent false
1126 1250 Acked Write 4 Done Done Sent false
1127 1251 Acked Read 8 Done Done Sent false
1128 1252 Acked Read 6 Done Done Sent false
1129 1253 Acked Read 7 Done Done Sent false
1130 1254 Acked Read 10 Done Done Sent false
1131 1255 Acked Write 7 Done Done Sent false
1132 1256 Acked Write 2 Done Done Sent false
1133 1257 Acked Write 5 Done Done Sent false
1134 1258 Acked Write 5 Done Done Sent false
1135 1259 Acked Read 8 Done Done Sent false
1136 1260 Acked Write 2 Done Done Sent false
1137 1261 Acked Flush 0 Done Done Sent false
1138 1262 Acked Write 2 Done Done Sent false
1139 1263 Acked Write 2 Done Done Sent false
1140 1264 Acked Read 5 Done Done Sent false
1141 1265 Acked Read 7 Done Done Sent false
1142 1266 Acked Write 3 Done Done Sent false
1143 1267 Acked Flush 0 Done Done Sent false
1144 1268 Acked Flush 0 Done Done Sent false
1145 1269 Acked Write 7 Done Done Sent false
1146 1270 Acked Read 3 Done Done Sent false
1147 1271 Acked Read 3 Done Done Sent false
1148 1272 Acked Write 8 Done Done Sent false
1149 1273 Acked Read 2 Done Done Sent false
1150 1274 Acked Read 6 Done Done Sent false
1151 1275 Acked Read 10 Done Done Sent false
1152 1276 Acked Write 7 Done Done Sent false
1153 1277 Acked Write 9 Done Done Sent false
1154 1278 Acked Write 4 Done Done Sent false
1155 1279 Acked Read 10 Done Done Sent false
1156 1280 Acked Write 2 Done Done Sent false
1157 1281 Acked Read 4 Done Done Sent false
1158 1282 Acked Read 4 Done Done Sent false
1159 1283 Acked Write 6 Done Done Sent false
1160 1284 Acked Read 1 Done Done Sent false
1161 1285 Acked Flush 0 Done Done Sent false
1162 1286 Acked Read 7 Done Done Sent false
1163 1287 Acked Write 1 Done Done Sent false
1164 1288 Acked Read 6 Done Done Sent false
1165 1289 Acked Read 10 Done Done Sent false
1166 1290 Acked Write 4 Done Done Sent false
1167 1291 Acked Write 10 Done Done Sent false
1168 1292 Acked Read 2 Done Done Sent false
1169 1293 Acked Read 9 Done Done Sent false
1170 1294 Acked Write 3 Done Done Sent false
1171 1295 Acked Read 3 Done Done Sent false
1172 1296 Acked Read 8 Done Done Sent false
1173 1297 Acked Write 9 Done Done Sent false
1174 1298 Acked Read 5 Done Done Sent false
1175 1299 Acked Write 5 Done Done Sent false
1176 1300 Acked Read 7 Done Done Sent false
1177 1301 Acked Flush 0 Done Done Sent false
1178 STATES DS:0 DS:1 DS:2 TOTAL
1179 Sent 0 0 302 302
1180 Done 302 302 0 604
1181 Skipped 0 0 0 0
1182 Error 0 0 0 0
1183 Last Flush: 1301 1301 0
1184 Downstairs last five completed:
1185 Upstairs last five completed: 1301 1300 1299 1298 1297
1186 CLIENT: Up:0 ds:302 act:2
11872024-12-11T23:40:44.528ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11882024-12-11T23:40:44.528ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54324) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11892024-12-11T23:40:44.529ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11902024-12-11T23:40:44.530ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11912024-12-11T23:40:44.530ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11922024-12-11T23:40:44.530ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11932024-12-11T23:40:44.530ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11942024-12-11T23:40:44.530ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11952024-12-11T23:40:44.530ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11962024-12-11T23:40:44.530ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11972024-12-11T23:40:44.530ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1198 ----------------------------------------------------------------
1199 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1200 Downstairs last five completed: 1301 1300 1299 1298 1297
1201 Upstairs last five completed: 1301 1300 1299 1298 1297
1202 CLIENT: Up:0 ds:0 act:3
1203 Replay: All jobs finished
1204 Main task has work to do, go find it
1205 got dsc Stop(1)
1206 stop 1
1207 Dec 11 23:40:46.783 INFO request completed, latency_us: 402, response_code: 204, uri: /stop/cid/1[8820] Got stop action so:false kr:false
1208 , method: POST, req_id: b9292d23-dfc8-4540-9803-aa4bb6b9437b, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
12092024-12-11T23:40:46.784ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
12102024-12-11T23:40:46.784ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
12112024-12-11T23:40:46.784ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
12122024-12-11T23:40:46.784ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
1213 Dec 11 23:40:46.784 INFO request completed, latency_us:{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:40:46.78480757Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
1214 117, response_code: 200, uri: /state/cid/1, method: GET, req_id{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:40:46.784837251Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
1215 : 24c1c9fe-a77a-4def-8ab8-05436db10b89, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
1216 [8820] Exited with: ExitStatus(unix_wait_status(9))
1217 [8820][1] reports Exit
1218 Dec 11 23:40:50.786 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/1, method: GET, req_id: e2320eb0-02b1-4f7a-8344-1cd5f55cf25a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
1219 001/300 Read block 3255 len 28672
1220 002/300 Write block 3725 len 36864 data: 2 2 2 2 2 2 2 2 2
1221 003/300 Write block 1097 len 8192 data: 2 2
1222 004/300 Read block 4700 len 24576
1223 005/300 Read block 2846 len 16384
1224 006/300 Flush
1225 007/300 Write block 4840 len 12288 data: 2 2 2
1226 008/300 Write block 2648 len 4096 data: 2
1227 009/300 Read block 4118 len 8192
1228 010/300 Read block 1396 len 8192
1229 011/300 Write block 4135 len 12288 data: 2 2 2
1230 012/300 Read block 929 len 4096
1231 013/300 Write block 158 len 32768 data: 2 2 2 2 2 2 2 2
1232 014/300 Read block 1812 len 16384
1233 015/300 Write block 3364 len 20480 data: 2 2 2 2 2
1234 016/300 Read block 27 len 8192
1235 017/300 Flush
1236 018/300 Read block 4405 len 20480
1237 019/300 Flush
1238 020/300 Read block 1816 len 40960
1239 021/300 Read block 3236 len 28672
1240 022/300 Read block 1576 len 12288
1241 023/300 Write block 3862 len 32768 data: 2 2 2 2 2 2 2 2
1242 024/300 Write block 4881 len 16384 data: 2 2 2 2
1243 025/300 Write block 30 len 28672 data: 2 2 2 2 2 2 2
1244 026/300 Read block 3553 len 4096
1245 027/300 Read block 853 len 12288
1246 028/300 Read block 3345 len 24576
1247 029/300 Write block 763 len 4096 data: 2
1248 030/300 Flush
1249 031/300 Write block 4231 len 24576 data: 2 2 2 2 2 2
1250 032/300 Write block 2355 len 12288 data: 2 2 2
1251 033/300 Read block 3459 len 20480
1252 034/300 Read block 3635 len 28672
1253 035/300 Flush
1254 036/300 Write block 840 len 8192 data: 2 2
1255 037/300 Read block 2908 len 24576
1256 038/300 Write block 1440 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1257 039/300 Read block 2808 len 36864
1258 040/300 Write block 2482 len 32768 data: 2 2 2 2 2 2 2 2
1259 041/300 Flush
1260 042/300 Write block 1788 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1261 043/300 Write block 1463 len 16384 data: 2 2 2 2
1262 044/300 Read block 4353 len 12288
1263 045/300 Write block 4047 len 28672 data: 2 2 2 2 2 2 2
1264 046/300 Flush
1265 047/300 Write block 2581 len 24576 data: 2 2 2 2 2 2
1266 048/300 Read block 1819 len 12288
1267 049/300 Read block 2989 len 12288
1268 050/300 Write block 4433 len 36864 data: 2 2 2 2 2 2 2 2 2
1269 051/300 Read block 2822 len 40960
1270 052/300 Read block 651 len 28672
1271 053/300 Read block 4349 len 36864
1272 054/300 Write block 3697 len 28672 data: 2 3 3 2 2 2 2
1273 055/300 Flush
1274 056/300 Read block 1969 len 8192
1275 057/300 Read block 2439 len 8192
1276 058/300 Read block 2629 len 28672
1277 059/300 Write block 2063 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1278 060/300 Read block 2795 len 28672
1279 061/300 Flush
1280 062/300 Read block 2335 len 28672
1281 063/300 Read block 4195 len 16384
1282 064/300 Write block 3768 len 12288 data: 2 2 2
1283 065/300 Write block 2114 len 40960 data: 3 3 3 3 2 2 2 2 2 2
1284 066/300 Write block 2510 len 16384 data: 2 2 2 2
1285 067/300 Read block 3985 len 12288
1286 068/300 Write block 4493 len 36864 data: 2 2 2 2 3 3 2 2 2
1287 069/300 Read block 126 len 8192
1288 070/300 Write block 323 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1289 071/300 Write block 640 len 12288 data: 2 2 2
1290 072/300 Read block 2555 len 16384
1291 073/300 Write block 536 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1292 074/300 Write block 4714 len 20480 data: 2 2 2 2 2
1293 075/300 Read block 4290 len 28672
1294 076/300 Read block 4073 len 32768
1295 077/300 Write block 44 len 28672 data: 3 3 3 2 2 2 2
1296 078/300 Flush
1297 079/300 Write block 3205 len 4096 data: 2
1298 080/300 Read block 3699 len 32768
1299 081/300 Read block 2298 len 12288
1300 082/300 Write block 4390 len 8192 data: 2 2
1301 083/300 Write block 4518 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1302 084/300 Write block 3815 len 24576 data: 2 2 2 2 2 2
1303 085/300 Read block 806 len 24576
1304 086/300 Write block 4002 len 4096 data: 2
1305 087/300 Read block 3572 len 12288
1306 088/300 Read block 2769 len 40960
1307 089/300 Read block 2438 len 24576
1308 090/300 Read block 760 len 32768
1309 091/300 Flush
1310 092/300 Read block 254 len 24576
1311 093/300 Read block 2288 len 4096
1312 094/300 Read block 848 len 40960
1313 095/300 Flush
1314 096/300 Write block 3585 len 36864 data: 2 2 3 3 2 2 2 2 3
1315 097/300 Read block 1456 len 24576
1316 098/300 Write block 2976 len 28672 data: 2 2 2 2 2 2 2
1317 099/300 Write block 1435 len 28672 data: 2 2 2 2 2 3 3
1318 100/300 Read block 1897 len 32768
1319 101/300 Read block 1952 len 32768
1320 102/300 Write block 2197 len 24576 data: 2 2 2 2 2 2
1321 103/300 Write block 3190 len 28672 data: 2 2 2 2 2 2 2
1322 104/300 Read block 1895 len 12288
1323 105/300 Write block 2508 len 8192 data: 2 2
1324 106/300 Read block 2824 len 40960
1325 107/300 Read block 1268 len 16384
1326 108/300 Flush
1327 109/300 Read block 1239 len 24576
1328 110/300 Write block 102 len 16384 data: 2 2 2 2
1329 111/300 Write block 3268 len 4096 data: 2
1330 112/300 Flush
1331 113/300 Read block 4243 len 32768
1332 114/300 Read block 3844 len 24576
1333 115/300 Flush
1334 116/300 Write block 2655 len 16384 data: 2 2 2 2
1335 117/300 Write block 1426 len 4096 data: 3
1336 118/300 Read block 1748 len 20480
1337 119/300 Read block 4189 len 24576
1338 120/300 Write block 123 len 16384 data: 3 3 3 2
1339 121/300 Read block 4168 len 4096
1340 122/300 Write block 3826 len 24576 data: 2 2 3 3 3 3
1341 123/300 Write block 2495 len 28672 data: 2 2 2 2 2 2 2
1342 124/300 Read block 497 len 36864
1343 125/300 Read block 4809 len 40960
1344 126/300 Flush
1345 127/300 Write block 1053 len 8192 data: 2 2
1346 128/300 Flush
1347 129/300 Read block 3095 len 28672
1348 130/300 Flush
1349 131/300 Read block 1993 len 20480
1350 132/300 Read block 1141 len 20480
1351 133/300 Read block 4518 len 36864
1352 134/300 Write block 655 len 28672 data: 2 2 2 2 2 2 2
1353 135/300 Write block 3674 len 8192 data: 2 2
1354 136/300 Flush
1355 137/300 Read block 3508 len 32768
1356 138/300 Read block 937 len 32768
1357 139/300 Write block 3432 len 16384 data: 2 2 2 2
1358 140/300 Read block 1088 len 32768
1359 141/300 Read block 452 len 4096
1360 142/300 Read block 1775 len 36864
1361 143/300 Read block 2358 len 32768
1362 144/300 Write block 4780 len 24576 data: 2 2 2 2 2 2
1363 145/300 Read block 2277 len 24576
1364 146/300 Read block 1770 len 12288
1365 147/300 Flush
1366 148/300 Read block 922 len 32768
1367 149/300 Write block 4920 len 12288 data: 2 2 2
1368 150/300 Write block 3941 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1369 151/300 Flush
1370 152/300 Write block 4525 len 32768 data: 3 3 3 3 3 4 4 4
1371 153/300 Read block 4930 len 20480
1372 154/300 Read block 2205 len 12288
1373 155/300 Read block 1534 len 20480
1374 156/300 Read block 3366 len 12288
1375 157/300 Write block 4899 len 40960 data: 2 2 2 2 2 3 3 2 2 2
1376 158/300 Write block 3627 len 32768 data: 2 2 2 2 2 2 2 2
1377 159/300 Read block 2601 len 12288
1378 160/300 Read block 2686 len 32768
1379 161/300 Write block 1484 len 40960 data: 3 3 2 2 2 2 2 2 2 2
1380 162/300 Flush
1381 163/300 Read block 1107 len 12288
1382 164/300 Read block 4808 len 36864
1383 165/300 Read block 419 len 24576
1384 166/300 Write block 2647 len 16384 data: 2 3 3 2
1385 167/300 Write block 2094 len 4096 data: 2
1386 168/300 Read block 4640 len 8192
1387 169/300 Write block 2588 len 16384 data: 2 2 2 2
1388 170/300 Read block 4626 len 16384
1389 171/300 Read block 3285 len 8192
1390 172/300 Write block 1368 len 32768 data: 2 2 2 2 2 2 2 2
1391 173/300 Read block 2417 len 32768
1392 174/300 Write block 2236 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1393 175/300 Write block 973 len 12288 data: 2 2 2
1394 176/300 Flush
1395 177/300 Read block 1994 len 16384
1396 178/300 Read block 464 len 20480
1397 179/300 Write block 2302 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1398 180/300 Read block 3708 len 32768
1399 181/300 Write block 2659 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1400 182/300 Read block 573 len 12288
1401 183/300 Flush
1402 184/300 Write block 3949 len 20480 data: 3 3 2 2 2
1403 185/300 Write block 2734 len 20480 data: 3 3 3 3 3
1404 186/300 Write block 391 len 8192 data: 2 2
1405 187/300 Read block 4019 len 4096
1406 188/300 Write block 2898 len 20480 data: 3 3 3 3 3
1407 189/300 Read block 3818 len 28672
1408 190/300 Read block 3814 len 32768
1409 191/300 Read block 642 len 28672
1410 192/300 Read block 1276 len 40960
1411 193/300 Flush
1412 194/300 Read block 233 len 32768
1413 195/300 Read block 98 len 20480
1414 196/300 Read block 4193 len 8192
1415 197/300 Write block 58 len 32768 data: 2 2 2 2 2 2 2 2
1416 198/300 Write block 3575 len 8192 data: 2 2
1417 199/300 Write block 975 len 4096 data: 3
1418 200/300 Write block 3103 len 12288 data: 2 2 2
1419 201/300 Read block 888 len 28672
1420 202/300 Read block 4472 len 36864
1421 203/300 Read block 736 len 36864
1422 204/300 Write block 4922 len 12288 data: 3 2 2
1423 205/300 Write block 1246 len 12288 data: 2 2 2
1424 206/300 Read block 204 len 12288
1425 207/300 Write block 2287 len 8192 data: 2 2
1426 208/300 Write block 4520 len 28672 data: 3 3 3 3 3 4 4
1427 209/300 Read block 664 len 4096
1428 210/300 Read block 1651 len 24576
1429 211/300 Read block 4854 len 8192
1430 212/300 Flush
1431 213/300 Flush
1432 214/300 Write block 4424 len 24576 data: 2 2 2 2 2 2
1433 215/300 Read block 304 len 4096
1434 216/300 Read block 2432 len 24576
1435 217/300 Read block 896 len 12288
1436 218/300 Write block 4519 len 16384 data: 3 4 4 4
1437 219/300 Flush
1438 220/300 Write block 2157 len 20480 data: 2 2 2 2 2
1439 221/300 Write block 444 len 12288 data: 2 3 3
1440 222/300 Flush
1441 223/300 Read block 240 len 20480
1442 224/300 Write block 1263 len 20480 data: 3 3 3 2 2
1443 225/300 Write block 2898 len 20480 data: 4 4 4 4 4
1444 226/300 Read block 2585 len 24576
1445 227/300 Read block 1141 len 12288
1446 228/300 Flush
1447 229/300 Read block 2279 len 16384
1448 230/300 Read block 1893 len 12288
1449 231/300 Write block 3740 len 28672 data: 2 2 2 2 3 3 3
1450 232/300 Write block 208 len 12288 data: 2 2 2
1451 233/300 Write block 939 len 40960 data: 3 3 3 3 2 2 2 2 2 2
1452 234/300 Write block 764 len 12288 data: 2 2 2
1453 235/300 Write block 1196 len 12288 data: 2 2 2
1454 236/300 Flush
1455 237/300 Read block 2282 len 20480
1456 238/300 Write block 326 len 16384 data: 3 3 3 3
1457 239/300 Read block 686 len 16384
1458 240/300 Read block 4359 len 4096
1459 241/300 Read block 1810 len 40960
1460 242/300 Read block 4182 len 32768
1461 243/300 Write block 3081 len 36864 data: 2 2 2 2 2 2 2 2 3
1462 244/300 Write block 1095 len 28672 data: 2 2 3 3 2 2 2
1463 245/300 Write block 1195 len 12288 data: 2 3 3
1464 246/300 Write block 2815 len 4096 data: 2
1465 247/300 Read block 2468 len 20480
1466 248/300 Read block 271 len 32768
1467 249/300 Write block 4613 len 20480 data: 2 2 2 2 2
1468 250/300 Read block 4936 len 32768
1469 251/300 Read block 2646 len 32768
1470 252/300 Read block 2498 len 16384
1471 253/300 Write block 3850 len 32768 data: 2 2 2 2 2 2 3 2
1472 254/300 Read block 876 len 32768
1473 255/300 Write block 1638 len 36864 data: 2 2 2 2 2 2 2 2 2
1474 256/300 Read block 4170 len 12288
1475 257/300 Read block 4255 len 12288
1476 258/300 Read block 163 len 32768
1477 259/300 Write block 2151 len 24576 data: 2 2 2 2 2 2
1478 260/300 Read block 1309 len 8192
1479 261/300 Read block 4979 len 40960
1480 262/300 Read block 4694 len 8192
1481 263/300 Read block 345 len 28672
1482 264/300 Read block 4964 len 12288
1483 265/300 Flush
1484 266/300 Read block 4107 len 28672
1485 267/300 Flush
1486 268/300 Read block 4318 len 12288
1487 269/300 Write block 3943 len 36864 data: 3 3 3 3 3 3 4 4 3
1488 270/300 Read block 2887 len 8192
1489 271/300 Read block 1489 len 20480
1490 272/300 Write block 3488 len 12288 data: 2 2 2
1491 273/300 Read block 712 len 24576
1492 274/300 Read block 867 len 12288
1493 275/300 Write block 4180 len 36864 data: 2 2 2 2 2 2 2 2 2
1494 276/300 Write block 4296 len 24576 data: 2 2 2 2 2 2
1495 277/300 Read block 1048 len 12288
1496 278/300 Read block 3735 len 24576
1497 279/300 Read block 1193 len 8192
1498 280/300 Read block 2505 len 20480
1499 281/300 Write block 4746 len 32768 data: 2 2 2 2 2 2 2 2
1500 282/300 Read block 3610 len 12288
1501 283/300 Read block 4084 len 24576
1502 284/300 Read block 1758 len 4096
1503 285/300 Read block 1041 len 20480
1504 286/300 Read block 2905 len 20480
1505 287/300 Write block 1029 len 16384 data: 2 2 2 2
1506 288/300 Read block 2448 len 28672
1507 289/300 Write block 977 len 12288 data: 2 2 2
1508 290/300 Read block 2296 len 8192
1509 291/300 Read block 1589 len 12288
1510 292/300 Flush
1511 293/300 Write block 4797 len 8192 data: 3 3
1512 294/300 Read block 4218 len 20480
1513 295/300 Read block 2037 len 12288
1514 296/300 Read block 2860 len 20480
1515 297/300 Flush
1516 298/300 Read block 2913 len 12288
1517 299/300 Read block 4953 len 28672
1518 300/300 Read block 4444 len 16384
1519 301/300 Write block 1046 len 8192 data: 2 2
1520 Main task has work to do, go find it
1521 got dsc Start(1)
1522 start 1
1523 Dec 11 23:40:51.026 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/1, method: POST, req_id: 57c8f78f-f33f-4d84-97f8-f3c8f60467f6, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
1524 Replay: started 1, returned:Ok(())
1525 ----------------------------------------------------------------
1526 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1527 States: Active Offline Active
1528 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1529 1302 Acked Read 7 Done Sent Done false
1530 1303 Acked Write 9 Done Sent Done false
1531 1304 Acked Write 2 Done Sent Done false
1532 1305 Acked Read 6 Done Sent Done false
1533 1306 Acked Read 4 Done Sent Done false
1534 1307 Acked Flush 0 Done Sent Done false
1535 1308 Acked Write 3 Done Sent Done false
1536 1309 Acked Write 1 Done Sent Done false
1537 1310 Acked Read 2 Done Sent Done false
1538 1311 Acked Read 2 Done Sent Done false
1539 1312 Acked Write 3 Done Sent Done false
1540 1313 Acked Read 1 Done Sent Done false
1541 1314 Acked Write 8 Done Sent Done false
1542 1315 Acked Read 4 Done Sent Done false
1543 1316 Acked Write 5 Done Sent Done false
1544 1317 Acked Read 2 Done Sent Done false
1545 1318 Acked Flush 0 Done Sent Done false
1546 1319 Acked Read 5 Done Sent Done false
1547 1320 Acked Flush 0 Done Sent Done false
1548 1321 Acked Read 10 Done Sent Done false
1549 1322 Acked Read 7 Done Sent Done false
1550 1323 Acked Read 3 Done Sent Done false
1551 1324 Acked Write 8 Done Sent Done false
1552 1325 Acked Write 4 Done Sent Done false
1553 1326 Acked Write 7 Done Sent Done false
1554 1327 Acked Read 1 Done Sent Done false
1555 1328 Acked Read 3 Done Sent Done false
1556 1329 Acked Read 6 Done Sent Done false
1557 1330 Acked Write 1 Done Sent Done false
1558 1331 Acked Flush 0 Done Sent Done false
1559 1332 Acked Write 6 Done Sent Done false
1560 1333 Acked Write 3 Done Sent Done false
1561 1334 Acked Read 5 Done Sent Done false
1562 1335 Acked Read 7 Done Sent Done false
1563 1336 Acked Flush 0 Done Sent Done false
1564 1337 Acked Write 2 Done Sent Done false
1565 1338 Acked Read 6 Done Sent Done false
1566 1339 Acked Write 10 Done Sent Done false
1567 1340 Acked Read 9 Done Sent Done false
1568 1341 Acked Write 8 Done Sent Done false
1569 1342 Acked Flush 0 Done Sent Done false
1570 1343 Acked Write 10 Done Sent Done false
1571 1344 Acked Write 4 Done Sent Done false
1572 1345 Acked Read 3 Done Sent Done false
1573 1346 Acked Write 7 Done Sent Done false
1574 1347 Acked Flush 0 Done Sent Done false
1575 1348 Acked Write 6 Done Sent Done false
1576 1349 Acked Read 3 Done Sent Done false
1577 1350 Acked Read 3 Done Sent Done false
1578 1351 Acked Write 9 Done Sent Done false
1579 1352 Acked Read 10 Done Sent Done false
1580 1353 Acked Read 7 Done Sent Done false
1581 1354 Acked Read 9 Done Sent Done false
1582 1355 Acked Write 7 Done Sent Done false
1583 1356 Acked Flush 0 Done Sent Done false
1584 1357 Acked Read 2 Done Sent Done false
1585 1358 Acked Read 2 Done Sent Done false
1586 1359 Acked Read 7 Done Sent Done false
1587 1360 Acked Write 10 Done Sent Done false
1588 1361 Acked Read 7 Done Sent Done false
1589 1362 Acked Flush 0 Done Sent Done false
1590 1363 Acked Read 7 Done Sent Done false
1591 1364 Acked Read 4 Done Sent Done false
1592 1365 Acked Write 3 Done Sent Done false
1593 1366 Acked Write 10 Done Sent Done false
1594 1367 Acked Write 4 Done Sent Done false
1595 1368 Acked Read 3 Done Sent Done false
1596 1369 Acked Write 9 Done Sent Done false
1597 1370 Acked Read 2 Done Sent Done false
1598 1371 Acked Write 10 Done Sent Done false
1599 1372 Acked Write 3 Done Sent Done false
1600 1373 Acked Read 4 Done Sent Done false
1601 1374 Acked Write 10 Done Sent Done false
1602 1375 Acked Write 5 Done Sent Done false
1603 1376 Acked Read 7 Done Sent Done false
1604 1377 Acked Read 8 Done Sent Done false
1605 1378 Acked Write 7 Done Sent Done false
1606 1379 Acked Flush 0 Done Sent Done false
1607 1380 Acked Write 1 Done Sent Done false
1608 1381 Acked Read 8 Done Sent Done false
1609 1382 Acked Read 3 Done Sent Done false
1610 1383 Acked Write 2 Done Sent Done false
1611 1384 Acked Write 10 Done Sent Done false
1612 1385 Acked Write 6 Done Sent Done false
1613 1386 Acked Read 6 Done Sent Done false
1614 1387 Acked Write 1 Done Sent Done false
1615 1388 Acked Read 3 Done Sent Done false
1616 1389 Acked Read 10 Done Sent Done false
1617 1390 Acked Read 6 Done Sent Done false
1618 1391 Acked Read 8 Done Sent Done false
1619 1392 Acked Flush 0 Done Sent Done false
1620 1393 Acked Read 6 Done Sent Done false
1621 1394 Acked Read 1 Done Sent Done false
1622 1395 Acked Read 10 Done Sent Done false
1623 1396 Acked Flush 0 Done Sent Done false
1624 1397 Acked Write 9 Done Sent Done false
1625 1398 Acked Read 6 Done Sent Done false
1626 1399 Acked Write 7 Done Sent Done false
1627 1400 Acked Write 7 Done Sent Done false
1628 1401 Acked Read 8 Done Sent Done false
1629 1402 Acked Read 8 Done Sent Done false
1630 1403 Acked Write 6 Done Sent Done false
1631 1404 Acked Write 7 Done Sent Done false
1632 1405 Acked Read 3 Done Sent Done false
1633 1406 Acked Write 2 Done Sent Done false
1634 1407 Acked Read 10 Done Sent Done false
1635 1408 Acked Read 4 Done Sent Done false
1636 1409 Acked Flush 0 Done Sent Done false
1637 1410 Acked Read 6 Done Sent Done false
1638 1411 Acked Write 4 Done Sent Done false
1639 1412 Acked Write 1 Done Sent Done false
1640 1413 Acked Flush 0 Done Sent Done false
1641 1414 Acked Read 8 Done Sent Done false
1642 1415 Acked Read 6 Done Sent Done false
1643 1416 Acked Flush 0 Done Sent Done false
1644 1417 Acked Write 4 Done Sent Done false
1645 1418 Acked Write 1 Done Sent Done false
1646 1419 Acked Read 5 Done Sent Done false
1647 1420 Acked Read 6 Done Sent Done false
1648 1421 Acked Write 4 Done Sent Done false
1649 1422 Acked Read 1 Done Sent Done false
1650 1423 Acked Write 6 Done Sent Done false
1651 1424 Acked Write 7 Done Sent Done false
1652 1425 Acked Read 9 Done Sent Done false
1653 1426 Acked Read 10 Done Sent Done false
1654 1427 Acked Flush 0 Done Sent Done false
1655 1428 Acked Write 2 Done Sent Done false
1656 1429 Acked Flush 0 Done Sent Done false
1657 1430 Acked Read 7 Done Sent Done false
1658 1431 Acked Flush 0 Done Sent Done false
1659 1432 Acked Read 5 Done Sent Done false
1660 1433 Acked Read 5 Done Sent Done false
1661 1434 Acked Read 9 Done Sent Done false
1662 1435 Acked Write 7 Done Sent Done false
1663 1436 Acked Write 2 Done Sent Done false
1664 1437 Acked Flush 0 Done Sent Done false
1665 1438 Acked Read 8 Done Sent Done false
1666 1439 Acked Read 8 Done Sent Done false
1667 1440 Acked Write 4 Done Sent Done false
1668 1441 Acked Read 8 Done Sent Done false
1669 1442 Acked Read 1 Done Sent Done false
1670 1443 Acked Read 9 Done Sent Done false
1671 1444 Acked Read 8 Done Sent Done false
1672 1445 Acked Write 6 Done Sent Done false
1673 1446 Acked Read 6 Done Sent Done false
1674 1447 Acked Read 3 Done Sent Done false
1675 1448 Acked Flush 0 Done Sent Done false
1676 1449 Acked Read 8 Done Sent Done false
1677 1450 Acked Write 3 Done Sent Done false
1678 1451 Acked Write 10 Done Sent Done false
1679 1452 Acked Flush 0 Done Sent Done false
1680 1453 Acked Write 8 Done Sent Done false
1681 1454 Acked Read 5 Done Sent Done false
1682 1455 Acked Read 3 Done Sent Done false
1683 1456 Acked Read 5 Done Sent Done false
1684 1457 Acked Read 3 Done Sent Done false
1685 1458 Acked Write 10 Done Sent Done false
1686 1459 Acked Write 8 Done Sent Done false
1687 1460 Acked Read 3 Done Sent Done false
1688 1461 Acked Read 8 Done Sent Done false
1689 1462 Acked Write 10 Done Sent Done false
1690 1463 Acked Flush 0 Done Sent Done false
1691 1464 Acked Read 3 Done Sent Done false
1692 1465 Acked Read 9 Done Sent Done false
1693 1466 Acked Read 6 Done Sent Done false
1694 1467 Acked Write 4 Done Sent Done false
1695 1468 Acked Write 1 Done Sent Done false
1696 1469 Acked Read 2 Done Sent Done false
1697 1470 Acked Write 4 Done Sent Done false
1698 1471 Acked Read 4 Done Sent Done false
1699 1472 Acked Read 2 Done Sent Done false
1700 1473 Acked Write 8 Done Sent Done false
1701 1474 Acked Read 8 Done Sent Done false
1702 1475 Acked Write 10 Done Sent Done false
1703 1476 Acked Write 3 Done Sent Done false
1704 1477 Acked Flush 0 Done Sent Done false
1705 1478 Acked Read 4 Done Sent Done false
1706 1479 Acked Read 5 Done Sent Done false
1707 1480 Acked Write 10 Done Sent Done false
1708 1481 Acked Read 8 Done Sent Done false
1709 1482 Acked Write 10 Done Sent Done false
1710 1483 Acked Read 3 Done Sent Done false
1711 1484 Acked Flush 0 Done Sent Done false
1712 1485 Acked Write 5 Done Sent Done false
1713 1486 Acked Write 5 Done Sent Done false
1714 1487 Acked Write 2 Done Sent Done false
1715 1488 Acked Read 1 Done Sent Done false
1716 1489 Acked Write 5 Done Sent Done false
1717 1490 Acked Read 7 Done Sent Done false
1718 1491 Acked Read 8 Done Sent Done false
1719 1492 Acked Read 7 Done Sent Done false
1720 1493 Acked Read 10 Done Sent Done false
1721 1494 Acked Flush 0 Done Sent Done false
1722 1495 Acked Read 8 Done Sent Done false
1723 1496 Acked Read 5 Done Sent Done false
1724 1497 Acked Read 2 Done Sent Done false
1725 1498 Acked Write 8 Done Sent Done false
1726 1499 Acked Write 2 Done Sent Done false
1727 1500 Acked Write 1 Done Sent Done false
1728 1501 Acked Write 3 Done Sent Done false
1729 1502 Acked Read 7 Done Sent Done false
1730 1503 Acked Read 9 Done Sent Done false
1731 1504 Acked Read 9 Done Sent Done false
1732 1505 Acked Write 3 Done Sent Done false
1733 1506 Acked Write 3 Done Sent Done false
1734 1507 Acked Read 3 Done Sent Done false
1735 1508 Acked Write 2 Done Sent Done false
1736 1509 Acked Write 7 Done Sent Done false
1737 1510 Acked Read 1 Done Sent Done false
1738 1511 Acked Read 6 Done Sent Done false
1739 1512 Acked Read 2 Done Sent Done false
1740 1513 Acked Flush 0 Done Sent Done false
1741 1514 Acked Flush 0 Done Sent Done false
1742 1515 Acked Write 6 Done Sent Done false
1743 1516 Acked Read 1 Done Sent Done false
1744 1517 Acked Read 6 Done Sent Done false
1745 1518 Acked Read 3 Done Sent Done false
1746 1519 Acked Write 4 Done Sent Done false
1747 1520 Acked Flush 0 Done Sent Done false
1748 1521 Acked Write 5 Done Sent Done false
1749 1522 Acked Write 3 Done Sent Done false
1750 1523 Acked Flush 0 Done Sent Done false
1751 1524 Acked Read 5 Done Sent Done false
1752 1525 Acked Write 5 Done Sent Done false
1753 1526 Acked Write 5 Done Sent Done false
1754 1527 Acked Read 6 Done Sent Done false
1755 1528 Acked Read 3 Done Sent Done false
1756 1529 Acked Flush 0 Done Sent Done false
1757 1530 Acked Read 4 Done Sent Done false
1758 1531 Acked Read 3 Done Sent Done false
1759 1532 Acked Write 7 Done Sent Done false
1760 1533 Acked Write 3 Done Sent Done false
1761 1534 Acked Write 10 Done Sent Done false
1762 1535 Acked Write 3 Done Sent Done false
1763 1536 Acked Write 3 Done Sent Done false
1764 1537 Acked Flush 0 Done Sent Done false
1765 1538 Acked Read 5 Done Sent Done false
1766 1539 Acked Write 4 Done Sent Done false
1767 1540 Acked Read 4 Done Sent Done false
1768 1541 Acked Read 1 Done Sent Done false
1769 1542 Acked Read 10 Done Sent Done false
1770 1543 Acked Read 8 Done Sent Done false
1771 1544 Acked Write 9 Done Sent Done false
1772 1545 Acked Write 7 Done Sent Done false
1773 1546 Acked Write 3 Done Sent Done false
1774 1547 Acked Write 1 Done Sent Done false
1775 1548 Acked Read 5 Done Sent Done false
1776 1549 Acked Read 8 Done Sent Done false
1777 1550 Acked Write 5 Done Sent Done false
1778 1551 Acked Read 8 Done Sent Done false
1779 1552 Acked Read 8 Done Sent Done false
1780 1553 Acked Read 4 Done Sent Done false
1781 1554 Acked Write 8 Done Sent Done false
1782 1555 Acked Read 8 Done Sent Done false
1783 1556 Acked Write 9 Done Sent Done false
1784 1557 Acked Read 3 Done Sent Done false
1785 1558 Acked Read 3 Done Sent Done false
1786 1559 Acked Read 8 Done Sent Done false
1787 1560 Acked Write 6 Done Sent Done false
1788 1561 Acked Read 2 Done Sent Done false
1789 1562 Acked Read 10 Done Sent Done false
1790 1563 Acked Read 2 Done Sent Done false
1791 1564 Acked Read 7 Done Sent Done false
1792 1565 Acked Read 3 Done Sent Done false
1793 1566 Acked Flush 0 Done Sent Done false
1794 1567 Acked Read 7 Done Sent Done false
1795 1568 Acked Flush 0 Done Sent Done false
1796 1569 Acked Read 3 Done Sent Done false
1797 1570 Acked Write 9 Done Sent Done false
1798 1571 Acked Read 2 Done Sent Done false
1799 1572 Acked Read 5 Done Sent Done false
1800 1573 Acked Write 3 Done Sent Done false
1801 1574 Acked Read 6 Done Sent Done false
1802 1575 Acked Read 3 Done Sent Done false
1803 1576 Acked Write 9 Done Sent Done false
1804 1577 Acked Write 6 Done Sent Done false
1805 1578 Acked Read 3 Done Sent Done false
1806 1579 Acked Read 6 Done Sent Done false
1807 1580 Acked Read 2 Done Sent Done false
1808 1581 Acked Read 5 Done Sent Done false
1809 1582 Acked Write 8 Done Sent Done false
1810 1583 Acked Read 3 Done Sent Done false
1811 1584 Acked Read 6 Done Sent Done false
1812 1585 Acked Read 1 Done Sent Done false
1813 1586 Acked Read 5 Done Sent Done false
1814 1587 Acked Read 5 Done Sent Done false
1815 1588 Acked Write 4 Done Sent Done false
1816 1589 Acked Read 7 Done Sent Done false
1817 1590 Acked Write 3 Done Sent Done false
1818 1591 Acked Read 2 Done Sent Done false
1819 1592 Acked Read 3 Done Sent Done false
1820 1593 Acked Flush 0 Done Sent Done false
1821 1594 Acked Write 2 Done Sent Done false
1822 1595 Acked Read 5 Done Sent Done false
1823 1596 Acked Read 3 Done Sent Done false
1824 1597 Acked Read 5 Done Sent Done false
1825 1598 Acked Flush 0 Done Sent Done false
1826 1599 Acked Read 3 Done Sent Done false
1827 1600 Acked Read 7 Done Sent Done false
1828 1601 Acked Read 4 Done Sent Done false
1829 1602 Acked Write 2 Done Sent Done false
1830 STATES DS:0 DS:1 DS:2 TOTAL
1831 Sent 0 301 0 301
1832 Done 301 0 301 602
1833 Skipped 0 0 0 0
1834 Error 0 0 0 0
1835 Last Flush: 1598 1301 1598
1836 Downstairs last five completed: 1301 1300 1299 1298 1297
1837 Upstairs last five completed: 1602 1601 1600 1599 1598
1838 CLIENT: Up:0 ds:301 act:2
1839 [8820] got start action
1840 [8820] I am going to restart
1841 Starting downstairs at port 8820
1842 Make output file at "/tmp/dsc/downstairs-8820.txt"
1843 [8820][1] reports Starting
1844 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1202)
1845 [8820][1] reports Running
1846 ----------------------------------------------------------------
1847 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1848 States: Active Offline Active
1849 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1850 1302 Acked Read 7 Done Sent Done false
1851 1303 Acked Write 9 Done Sent Done false
1852 1304 Acked Write 2 Done Sent Done false
1853 1305 Acked Read 6 Done Sent Done false
1854 1306 Acked Read 4 Done Sent Done false
1855 1307 Acked Flush 0 Done Sent Done false
1856 1308 Acked Write 3 Done Sent Done false
1857 1309 Acked Write 1 Done Sent Done false
1858 1310 Acked Read 2 Done Sent Done false
1859 1311 Acked Read 2 Done Sent Done false
1860 1312 Acked Write 3 Done Sent Done false
1861 1313 Acked Read 1 Done Sent Done false
1862 1314 Acked Write 8 Done Sent Done false
1863 1315 Acked Read 4 Done Sent Done false
1864 1316 Acked Write 5 Done Sent Done false
1865 1317 Acked Read 2 Done Sent Done false
1866 1318 Acked Flush 0 Done Sent Done false
1867 1319 Acked Read 5 Done Sent Done false
1868 1320 Acked Flush 0 Done Sent Done false
1869 1321 Acked Read 10 Done Sent Done false
1870 1322 Acked Read 7 Done Sent Done false
1871 1323 Acked Read 3 Done Sent Done false
1872 1324 Acked Write 8 Done Sent Done false
1873 1325 Acked Write 4 Done Sent Done false
1874 1326 Acked Write 7 Done Sent Done false
1875 1327 Acked Read 1 Done Sent Done false
1876 1328 Acked Read 3 Done Sent Done false
1877 1329 Acked Read 6 Done Sent Done false
1878 1330 Acked Write 1 Done Sent Done false
1879 1331 Acked Flush 0 Done Sent Done false
1880 1332 Acked Write 6 Done Sent Done false
1881 1333 Acked Write 3 Done Sent Done false
1882 1334 Acked Read 5 Done Sent Done false
1883 1335 Acked Read 7 Done Sent Done false
1884 1336 Acked Flush 0 Done Sent Done false
1885 1337 Acked Write 2 Done Sent Done false
1886 1338 Acked Read 6 Done Sent Done false
1887 1339 Acked Write 10 Done Sent Done false
1888 1340 Acked Read 9 Done Sent Done false
1889 1341 Acked Write 8 Done Sent Done false
1890 1342 Acked Flush 0 Done Sent Done false
1891 1343 Acked Write 10 Done Sent Done false
1892 1344 Acked Write 4 Done Sent Done false
1893 1345 Acked Read 3 Done Sent Done false
1894 1346 Acked Write 7 Done Sent Done false
1895 1347 Acked Flush 0 Done Sent Done false
1896 1348 Acked Write 6 Done Sent Done false
1897 1349 Acked Read 3 Done Sent Done false
1898 1350 Acked Read 3 Done Sent Done false
1899 1351 Acked Write 9 Done Sent Done false
1900 1352 Acked Read 10 Done Sent Done false
1901 1353 Acked Read 7 Done Sent Done false
1902 1354 Acked Read 9 Done Sent Done false
1903 1355 Acked Write 7 Done Sent Done false
1904 1356 Acked Flush 0 Done Sent Done false
1905 1357 Acked Read 2 Done Sent Done false
1906 1358 Acked Read 2 Done Sent Done false
1907 1359 Acked Read 7 Done Sent Done false
1908 1360 Acked Write 10 Done Sent Done false
1909 1361 Acked Read 7 Done Sent Done false
1910 1362 Acked Flush 0 Done Sent Done false
1911 1363 Acked Read 7 Done Sent Done false
1912 1364 Acked Read 4 Done Sent Done false
1913 1365 Acked Write 3 Done Sent Done false
1914 1366 Acked Write 10 Done Sent Done false
1915 1367 Acked Write 4 Done Sent Done false
1916 1368 Acked Read 3 Done Sent Done false
1917 1369 Acked Write 9 Done Sent Done false
1918 1370 Acked Read 2 Done Sent Done false
1919 1371 Acked Write 10 Done Sent Done false
1920 1372 Acked Write 3 Done Sent Done false
1921 1373 Acked Read 4 Done Sent Done false
1922 1374 Acked Write 10 Done Sent Done false
1923 1375 Acked Write 5 Done Sent Done false
1924 1376 Acked Read 7 Done Sent Done false
1925 1377 Acked Read 8 Done Sent Done false
1926 1378 Acked Write 7 Done Sent Done false
1927 1379 Acked Flush 0 Done Sent Done false
1928 1380 Acked Write 1 Done Sent Done false
1929 1381 Acked Read 8 Done Sent Done false
1930 1382 Acked Read 3 Done Sent Done false
1931 1383 Acked Write 2 Done Sent Done false
1932 1384 Acked Write 10 Done Sent Done false
1933 1385 Acked Write 6 Done Sent Done false
1934 1386 Acked Read 6 Done Sent Done false
1935 1387 Acked Write 1 Done Sent Done false
1936 1388 Acked Read 3 Done Sent Done false
1937 1389 Acked Read 10 Done Sent Done false
1938 1390 Acked Read 6 Done Sent Done false
1939 1391 Acked Read 8 Done Sent Done false
1940 1392 Acked Flush 0 Done Sent Done false
1941 1393 Acked Read 6 Done Sent Done false
1942 1394 Acked Read 1 Done Sent Done false
1943 1395 Acked Read 10 Done Sent Done false
1944 1396 Acked Flush 0 Done Sent Done false
1945 1397 Acked Write 9 Done Sent Done false
1946 1398 Acked Read 6 Done Sent Done false
1947 1399 Acked Write 7 Done Sent Done false
1948 1400 Acked Write 7 Done Sent Done false
1949 1401 Acked Read 8 Done Sent Done false
1950 1402 Acked Read 8 Done Sent Done false
1951 1403 Acked Write 6 Done Sent Done false
1952 1404 Acked Write 7 Done Sent Done false
1953 1405 Acked Read 3 Done Sent Done false
1954 1406 Acked Write 2 Done Sent Done false
1955 1407 Acked Read 10 Done Sent Done false
1956 1408 Acked Read 4 Done Sent Done false
1957 1409 Acked Flush 0 Done Sent Done false
1958 1410 Acked Read 6 Done Sent Done false
1959 1411 Acked Write 4 Done Sent Done false
1960 1412 Acked Write 1 Done Sent Done false
1961 1413 Acked Flush 0 Done Sent Done false
1962 1414 Acked Read 8 Done Sent Done false
1963 1415 Acked Read 6 Done Sent Done false
1964 1416 Acked Flush 0 Done Sent Done false
1965 1417 Acked Write 4 Done Sent Done false
1966 1418 Acked Write 1 Done Sent Done false
1967 1419 Acked Read 5 Done Sent Done false
1968 1420 Acked Read 6 Done Sent Done false
1969 1421 Acked Write 4 Done Sent Done false
1970 1422 Acked Read 1 Done Sent Done false
1971 1423 Acked Write 6 Done Sent Done false
1972 1424 Acked Write 7 Done Sent Done false
1973 1425 Acked Read 9 Done Sent Done false
1974 1426 Acked Read 10 Done Sent Done false
1975 1427 Acked Flush 0 Done Sent Done false
1976 1428 Acked Write 2 Done Sent Done false
1977 1429 Acked Flush 0 Done Sent Done false
1978 1430 Acked Read 7 Done Sent Done false
1979 1431 Acked Flush 0 Done Sent Done false
1980 1432 Acked Read 5 Done Sent Done false
1981 1433 Acked Read 5 Done Sent Done false
1982 1434 Acked Read 9 Done Sent Done false
1983 1435 Acked Write 7 Done Sent Done false
1984 1436 Acked Write 2 Done Sent Done false
1985 1437 Acked Flush 0 Done Sent Done false
1986 1438 Acked Read 8 Done Sent Done false
1987 1439 Acked Read 8 Done Sent Done false
1988 1440 Acked Write 4 Done Sent Done false
1989 1441 Acked Read 8 Done Sent Done false
1990 1442 Acked Read 1 Done Sent Done false
1991 1443 Acked Read 9 Done Sent Done false
1992 1444 Acked Read 8 Done Sent Done false
1993 1445 Acked Write 6 Done Sent Done false
1994 1446 Acked Read 6 Done Sent Done false
1995 1447 Acked Read 3 Done Sent Done false
1996 1448 Acked Flush 0 Done Sent Done false
1997 1449 Acked Read 8 Done Sent Done false
1998 1450 Acked Write 3 Done Sent Done false
1999 1451 Acked Write 10 Done Sent Done false
2000 1452 Acked Flush 0 Done Sent Done false
2001 1453 Acked Write 8 Done Sent Done false
2002 1454 Acked Read 5 Done Sent Done false
2003 1455 Acked Read 3 Done Sent Done false
2004 1456 Acked Read 5 Done Sent Done false
2005 1457 Acked Read 3 Done Sent Done false
2006 1458 Acked Write 10 Done Sent Done false
2007 1459 Acked Write 8 Done Sent Done false
2008 1460 Acked Read 3 Done Sent Done false
2009 1461 Acked Read 8 Done Sent Done false
2010 1462 Acked Write 10 Done Sent Done false
2011 1463 Acked Flush 0 Done Sent Done false
2012 1464 Acked Read 3 Done Sent Done false
2013 1465 Acked Read 9 Done Sent Done false
2014 1466 Acked Read 6 Done Sent Done false
2015 1467 Acked Write 4 Done Sent Done false
2016 1468 Acked Write 1 Done Sent Done false
2017 1469 Acked Read 2 Done Sent Done false
2018 1470 Acked Write 4 Done Sent Done false
2019 1471 Acked Read 4 Done Sent Done false
2020 1472 Acked Read 2 Done Sent Done false
2021 1473 Acked Write 8 Done Sent Done false
2022 1474 Acked Read 8 Done Sent Done false
2023 1475 Acked Write 10 Done Sent Done false
2024 1476 Acked Write 3 Done Sent Done false
2025 1477 Acked Flush 0 Done Sent Done false
2026 1478 Acked Read 4 Done Sent Done false
2027 1479 Acked Read 5 Done Sent Done false
2028 1480 Acked Write 10 Done Sent Done false
2029 1481 Acked Read 8 Done Sent Done false
2030 1482 Acked Write 10 Done Sent Done false
2031 1483 Acked Read 3 Done Sent Done false
2032 1484 Acked Flush 0 Done Sent Done false
2033 1485 Acked Write 5 Done Sent Done false
2034 1486 Acked Write 5 Done Sent Done false
2035 1487 Acked Write 2 Done Sent Done false
2036 1488 Acked Read 1 Done Sent Done false
2037 1489 Acked Write 5 Done Sent Done false
2038 1490 Acked Read 7 Done Sent Done false
2039 1491 Acked Read 8 Done Sent Done false
2040 1492 Acked Read 7 Done Sent Done false
2041 1493 Acked Read 10 Done Sent Done false
2042 1494 Acked Flush 0 Done Sent Done false
2043 1495 Acked Read 8 Done Sent Done false
2044 1496 Acked Read 5 Done Sent Done false
2045 1497 Acked Read 2 Done Sent Done false
2046 1498 Acked Write 8 Done Sent Done false
2047 1499 Acked Write 2 Done Sent Done false
2048 1500 Acked Write 1 Done Sent Done false
2049 1501 Acked Write 3 Done Sent Done false
2050 1502 Acked Read 7 Done Sent Done false
2051 1503 Acked Read 9 Done Sent Done false
2052 1504 Acked Read 9 Done Sent Done false
2053 1505 Acked Write 3 Done Sent Done false
2054 1506 Acked Write 3 Done Sent Done false
2055 1507 Acked Read 3 Done Sent Done false
2056 1508 Acked Write 2 Done Sent Done false
2057 1509 Acked Write 7 Done Sent Done false
2058 1510 Acked Read 1 Done Sent Done false
2059 1511 Acked Read 6 Done Sent Done false
2060 1512 Acked Read 2 Done Sent Done false
2061 1513 Acked Flush 0 Done Sent Done false
2062 1514 Acked Flush 0 Done Sent Done false
2063 1515 Acked Write 6 Done Sent Done false
2064 1516 Acked Read 1 Done Sent Done false
2065 1517 Acked Read 6 Done Sent Done false
2066 1518 Acked Read 3 Done Sent Done false
2067 1519 Acked Write 4 Done Sent Done false
2068 1520 Acked Flush 0 Done Sent Done false
2069 1521 Acked Write 5 Done Sent Done false
2070 1522 Acked Write 3 Done Sent Done false
2071 1523 Acked Flush 0 Done Sent Done false
2072 1524 Acked Read 5 Done Sent Done false
2073 1525 Acked Write 5 Done Sent Done false
2074 1526 Acked Write 5 Done Sent Done false
2075 1527 Acked Read 6 Done Sent Done false
2076 1528 Acked Read 3 Done Sent Done false
2077 1529 Acked Flush 0 Done Sent Done false
2078 1530 Acked Read 4 Done Sent Done false
2079 1531 Acked Read 3 Done Sent Done false
2080 1532 Acked Write 7 Done Sent Done false
2081 1533 Acked Write 3 Done Sent Done false
2082 1534 Acked Write 10 Done Sent Done false
2083 1535 Acked Write 3 Done Sent Done false
2084 1536 Acked Write 3 Done Sent Done false
2085 1537 Acked Flush 0 Done Sent Done false
2086 1538 Acked Read 5 Done Sent Done false
2087 1539 Acked Write 4 Done Sent Done false
2088 1540 Acked Read 4 Done Sent Done false
2089 1541 Acked Read 1 Done Sent Done false
2090 1542 Acked Read 10 Done Sent Done false
2091 1543 Acked Read 8 Done Sent Done false
2092 1544 Acked Write 9 Done Sent Done false
2093 1545 Acked Write 7 Done Sent Done false
2094 1546 Acked Write 3 Done Sent Done false
2095 1547 Acked Write 1 Done Sent Done false
2096 1548 Acked Read 5 Done Sent Done false
2097 1549 Acked Read 8 Done Sent Done false
2098 1550 Acked Write 5 Done Sent Done false
2099 1551 Acked Read 8 Done Sent Done false
2100 1552 Acked Read 8 Done Sent Done false
2101 1553 Acked Read 4 Done Sent Done false
2102 1554 Acked Write 8 Done Sent Done false
2103 1555 Acked Read 8 Done Sent Done false
2104 1556 Acked Write 9 Done Sent Done false
2105 1557 Acked Read 3 Done Sent Done false
2106 1558 Acked Read 3 Done Sent Done false
2107 1559 Acked Read 8 Done Sent Done false
2108 1560 Acked Write 6 Done Sent Done false
2109 1561 Acked Read 2 Done Sent Done false
2110 1562 Acked Read 10 Done Sent Done false
2111 1563 Acked Read 2 Done Sent Done false
2112 1564 Acked Read 7 Done Sent Done false
2113 1565 Acked Read 3 Done Sent Done false
2114 1566 Acked Flush 0 Done Sent Done false
2115 1567 Acked Read 7 Done Sent Done false
2116 1568 Acked Flush 0 Done Sent Done false
2117 1569 Acked Read 3 Done Sent Done false
2118 1570 Acked Write 9 Done Sent Done false
2119 1571 Acked Read 2 Done Sent Done false
2120 1572 Acked Read 5 Done Sent Done false
2121 1573 Acked Write 3 Done Sent Done false
2122 1574 Acked Read 6 Done Sent Done false
2123 1575 Acked Read 3 Done Sent Done false
2124 1576 Acked Write 9 Done Sent Done false
2125 1577 Acked Write 6 Done Sent Done false
2126 1578 Acked Read 3 Done Sent Done false
2127 1579 Acked Read 6 Done Sent Done false
2128 1580 Acked Read 2 Done Sent Done false
2129 1581 Acked Read 5 Done Sent Done false
2130 1582 Acked Write 8 Done Sent Done false
2131 1583 Acked Read 3 Done Sent Done false
2132 1584 Acked Read 6 Done Sent Done false
2133 1585 Acked Read 1 Done Sent Done false
2134 1586 Acked Read 5 Done Sent Done false
2135 1587 Acked Read 5 Done Sent Done false
2136 1588 Acked Write 4 Done Sent Done false
2137 1589 Acked Read 7 Done Sent Done false
2138 1590 Acked Write 3 Done Sent Done false
2139 1591 Acked Read 2 Done Sent Done false
2140 1592 Acked Read 3 Done Sent Done false
2141 1593 Acked Flush 0 Done Sent Done false
2142 1594 Acked Write 2 Done Sent Done false
2143 1595 Acked Read 5 Done Sent Done false
2144 1596 Acked Read 3 Done Sent Done false
2145 1597 Acked Read 5 Done Sent Done false
2146 1598 Acked Flush 0 Done Sent Done false
2147 1599 Acked Read 3 Done Sent Done false
2148 1600 Acked Read 7 Done Sent Done false
2149 1601 Acked Read 4 Done Sent Done false
2150 1602 Acked Write 2 Done Sent Done false
2151 1603 Acked Flush 0 Done Sent Done false
2152 STATES DS:0 DS:1 DS:2 TOTAL
2153 Sent 0 302 0 302
2154 Done 302 0 302 604
2155 Skipped 0 0 0 0
2156 Error 0 0 0 0
2157 Last Flush: 1603 1301 1603
2158 Downstairs last five completed: 1301 1300 1299 1298 1297
2159 Upstairs last five completed: 1603 1602 1601 1600 1599
2160 CLIENT: Up:0 ds:302 act:2
21612024-12-11T23:40:56.786ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21622024-12-11T23:40:56.786ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43141) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21632024-12-11T23:40:56.787ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21642024-12-11T23:40:56.787ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21652024-12-11T23:40:56.787ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21662024-12-11T23:40:56.787ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21672024-12-11T23:40:56.788ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21682024-12-11T23:40:56.788ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21692024-12-11T23:40:56.788ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21702024-12-11T23:40:56.788ZINFOcrucible: [1] client re-new 302 jobs since flush 1301 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21712024-12-11T23:40:56.788ZINFOcrucible: [1] Replayed 302 jobs since flush: 1301 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2172 ----------------------------------------------------------------
2173 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2174 Downstairs last five completed: 1603 1602 1601 1600 1599
2175 Upstairs last five completed: 1603 1602 1601 1600 1599
2176 CLIENT: Up:0 ds:0 act:3
2177 Replay: All jobs finished
2178 Main task has work to do, go find it
2179 got dsc Stop(2)
2180 stop 2
2181 Dec 11 23:40:59.031 INFO request completed, latency_us:[8830] Got stop action so:false kr:false
2182 327, response_code: 204, uri: /stop/cid/2, method: POST, req_id: c3bd4f8c-3289-4fa1-8785-5047936fca66, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
21832024-12-11T23:40:59.032ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2184 Dec 11 23:40:59.032 INFO request completed, latency_us: 188, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7314756f-eef3-40ef-a603-afc45bcdc7cf, remote_addr: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:40:59.032410522Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
2185 127.0.0.1:62969, local_addr: 127.0.0.1:9998
21862024-12-11T23:40:59.032ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21872024-12-11T23:40:59.032ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21882024-12-11T23:40:59.032ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21892024-12-11T23:40:59.032ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
2190 [8830] Exited with: ExitStatus(unix_wait_status(9))
2191 [8830][2] reports Exit
2192 Dec 11 23:41:03.034 INFO request completed, latency_us: 231, response_code: 200, uri: /state/cid/2, method: GET, req_id: af0c9dd7-8bc8-4d4e-a218-ffb4cdbe88a3, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
2193 001/300 Read block 1600 len 32768
2194 002/300 Read block 209 len 32768
2195 003/300 Read block 4191 len 16384
2196 004/300 Write block 1723 len 16384 data: 2 2 2 2
2197 005/300 Read block 2357 len 24576
2198 006/300 Read block 1287 len 36864
2199 007/300 Read block 2814 len 40960
2200 008/300 Read block 3202 len 4096
2201 009/300 Write block 2710 len 28672 data: 2 2 2 2 2 2 2
2202 010/300 Flush
2203 011/300 Read block 203 len 8192
2204 012/300 Read block 4334 len 8192
2205 013/300 Write block 3587 len 20480 data: 4 4 3 3 3
2206 014/300 Read block 4285 len 20480
2207 015/300 Read block 2221 len 40960
2208 016/300 Read block 1099 len 32768
2209 017/300 Read block 2995 len 12288
2210 018/300 Read block 965 len 20480
2211 019/300 Write block 1992 len 4096 data: 2
2212 020/300 Read block 4492 len 28672
2213 021/300 Read block 4892 len 4096
2214 022/300 Read block 42 len 24576
2215 023/300 Read block 2514 len 32768
2216 024/300 Write block 3812 len 36864 data: 2 2 2 3 3 3 3 3 3
2217 025/300 Write block 2339 len 16384 data: 2 2 2 2
2218 026/300 Flush
2219 027/300 Read block 970 len 20480
2220 028/300 Write block 3297 len 40960 data: 2 3 3 3 3 3 2 2 2 2
2221 029/300 Write block 3370 len 20480 data: 2 2 2 2 2
2222 030/300 Read block 470 len 36864
2223 031/300 Read block 1712 len 16384
2224 032/300 Read block 2532 len 16384
2225 033/300 Write block 4629 len 28672 data: 2 2 2 2 2 2 2
2226 034/300 Write block 1246 len 12288 data: 3 3 3
2227 035/300 Flush
2228 036/300 Write block 4143 len 20480 data: 2 2 2 2 2
2229 037/300 Read block 1403 len 4096
2230 038/300 Read block 4847 len 16384
2231 039/300 Read block 4722 len 8192
2232 040/300 Write block 4596 len 36864 data: 2 2 2 2 2 2 2 2 2
2233 041/300 Read block 4071 len 20480
2234 042/300 Read block 3209 len 4096
2235 043/300 Read block 1113 len 12288
2236 044/300 Read block 1043 len 12288
2237 045/300 Read block 3080 len 28672
2238 046/300 Write block 3026 len 16384 data: 2 2 2 2
2239 047/300 Write block 3155 len 12288 data: 2 2 2
2240 048/300 Read block 2223 len 4096
2241 049/300 Read block 4318 len 20480
2242 050/300 Write block 4241 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2243 051/300 Read block 1968 len 36864
2244 052/300 Read block 1116 len 12288
2245 053/300 Read block 1472 len 16384
2246 054/300 Read block 1483 len 36864
2247 055/300 Read block 4210 len 28672
2248 056/300 Write block 547 len 28672 data: 2 2 2 2 2 2 2
2249 057/300 Write block 4253 len 16384 data: 2 2 2 2
2250 058/300 Write block 3573 len 36864 data: 3 2 3 3 2 2 2 2 2
2251 059/300 Write block 2184 len 24576 data: 2 2 2 2 2 2
2252 060/300 Read block 400 len 36864
2253 061/300 Flush
2254 062/300 Read block 611 len 20480
2255 063/300 Read block 3433 len 40960
2256 064/300 Write block 3314 len 16384 data: 3 3 3 3
2257 065/300 Read block 2046 len 40960
2258 066/300 Read block 4953 len 20480
2259 067/300 Write block 4227 len 40960 data: 3 3 3 2 3 3 3 3 3 3
2260 068/300 Read block 3516 len 4096
2261 069/300 Read block 189 len 24576
2262 070/300 Read block 2951 len 4096
2263 071/300 Flush
2264 072/300 Flush
2265 073/300 Write block 4124 len 12288 data: 2 2 2
2266 074/300 Read block 1678 len 36864
2267 075/300 Flush
2268 076/300 Read block 4967 len 28672
2269 077/300 Write block 3795 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2270 078/300 Write block 2786 len 32768 data: 2 2 2 2 2 2 2 2
2271 079/300 Read block 1961 len 16384
2272 080/300 Read block 2403 len 40960
2273 081/300 Read block 420 len 16384
2274 082/300 Write block 314 len 40960 data: 2 2 2 2 2 2 2 2 2 3
2275 083/300 Write block 4428 len 8192 data: 3 3
2276 084/300 Read block 3088 len 36864
2277 085/300 Read block 2841 len 36864
2278 086/300 Read block 3265 len 16384
2279 087/300 Write block 3560 len 16384 data: 2 2 2 2
2280 088/300 Write block 2360 len 28672 data: 2 2 2 2 2 2 2
2281 089/300 Flush
2282 090/300 Flush
2283 091/300 Read block 3944 len 8192
2284 092/300 Write block 3886 len 12288 data: 2 2 2
2285 093/300 Flush
2286 094/300 Write block 831 len 4096 data: 2
2287 095/300 Read block 4960 len 32768
2288 096/300 Write block 1133 len 36864 data: 2 2 2 2 2 2 2 2 2
2289 097/300 Write block 873 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2290 098/300 Flush
2291 099/300 Write block 2439 len 20480 data: 2 2 2 2 2
2292 100/300 Read block 3106 len 20480
2293 101/300 Write block 3791 len 24576 data: 2 2 2 2 3 3
2294 102/300 Read block 334 len 8192
2295 103/300 Write block 3922 len 8192 data: 2 2
2296 104/300 Read block 1580 len 8192
2297 105/300 Write block 32 len 20480 data: 3 3 3 3 3
2298 106/300 Read block 1959 len 20480
2299 107/300 Write block 2981 len 36864 data: 3 3 2 2 2 2 2 2 2
2300 108/300 Write block 4060 len 28672 data: 2 2 2 2 2 2 2
2301 109/300 Read block 594 len 24576
2302 110/300 Read block 2572 len 16384
2303 111/300 Write block 3856 len 40960 data: 4 3 2 2 2 2 3 3 3 3
2304 112/300 Flush
2305 113/300 Read block 3874 len 8192
2306 114/300 Write block 4733 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2307 115/300 Write block 3017 len 12288 data: 2 2 2
2308 116/300 Read block 3714 len 16384
2309 117/300 Write block 383 len 12288 data: 3 3 3
2310 118/300 Write block 866 len 4096 data: 2
2311 119/300 Read block 3365 len 8192
2312 120/300 Flush
2313 121/300 Write block 4887 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2314 122/300 Flush
2315 123/300 Read block 4544 len 12288
2316 124/300 Read block 1221 len 24576
2317 125/300 Write block 4994 len 20480 data: 2 2 2 2 2
2318 126/300 Write block 2716 len 4096 data: 3
2319 127/300 Read block 496 len 32768
2320 128/300 Write block 2649 len 16384 data: 4 3 2 2
2321 129/300 Read block 4153 len 36864
2322 130/300 Read block 3639 len 4096
2323 131/300 Write block 4563 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2324 132/300 Read block 3429 len 16384
2325 133/300 Write block 4087 len 32768 data: 2 2 2 2 2 2 2 2
2326 134/300 Write block 1984 len 8192 data: 3 3
2327 135/300 Flush
2328 136/300 Flush
2329 137/300 Write block 2142 len 4096 data: 2
2330 138/300 Write block 1182 len 20480 data: 2 2 2 2 2
2331 139/300 Write block 4192 len 8192 data: 2 2
2332 140/300 Write block 2423 len 12288 data: 2 2 2
2333 141/300 Write block 2828 len 32768 data: 2 2 2 2 2 2 2 2
2334 142/300 Write block 1438 len 40960 data: 3 3 4 4 3 3 3 3 3 3
2335 143/300 Write block 2211 len 24576 data: 2 2 2 2 2 2
2336 144/300 Write block 3279 len 16384 data: 2 2 2 2
2337 145/300 Read block 1828 len 8192
2338 146/300 Write block 2597 len 36864 data: 3 3 3 4 4 3 3 3 3
2339 147/300 Write block 2695 len 4096 data: 3
2340 148/300 Write block 344 len 16384 data: 2 2 2 2
2341 149/300 Flush
2342 150/300 Read block 1067 len 4096
2343 151/300 Read block 2006 len 36864
2344 152/300 Read block 3292 len 4096
2345 153/300 Write block 3975 len 28672 data: 2 2 2 2 2 2 2
2346 154/300 Write block 886 len 8192 data: 2 3
2347 155/300 Write block 4668 len 24576 data: 2 2 2 2 2 2
2348 156/300 Write block 3200 len 32768 data: 2 2 2 2 2 3 2 2
2349 157/300 Read block 2310 len 16384
2350 158/300 Flush
2351 159/300 Write block 3856 len 28672 data: 5 4 3 3 3 3 4
2352 160/300 Write block 2575 len 40960 data: 3 3 3 3 3 2 3 3 3 3
2353 161/300 Write block 2716 len 36864 data: 4 2 2 2 2 2 2 2 2
2354 162/300 Flush
2355 163/300 Write block 99 len 12288 data: 2 2 2
2356 164/300 Write block 3448 len 20480 data: 2 2 2 2 2
2357 165/300 Read block 3898 len 32768
2358 166/300 Read block 2248 len 4096
2359 167/300 Flush
2360 168/300 Read block 4062 len 28672
2361 169/300 Read block 2180 len 28672
2362 170/300 Read block 3227 len 20480
2363 171/300 Write block 2030 len 16384 data: 2 2 2 2
2364 172/300 Read block 1771 len 8192
2365 173/300 Read block 2319 len 8192
2366 174/300 Flush
2367 175/300 Flush
2368 176/300 Read block 4018 len 24576
2369 177/300 Write block 1975 len 20480 data: 2 2 2 2 3
2370 178/300 Write block 209 len 16384 data: 3 3 2 2
2371 179/300 Read block 685 len 24576
2372 180/300 Write block 3003 len 4096 data: 3
2373 181/300 Write block 2724 len 12288 data: 3 2 2
2374 182/300 Read block 2530 len 40960
2375 183/300 Write block 2062 len 8192 data: 2 3
2376 184/300 Write block 3041 len 24576 data: 2 2 2 2 2 2
2377 185/300 Read block 2720 len 12288
2378 186/300 Read block 3196 len 4096
2379 187/300 Flush
2380 188/300 Write block 2887 len 4096 data: 3
2381 189/300 Write block 3383 len 20480 data: 2 2 3 3 3
2382 190/300 Write block 3326 len 32768 data: 2 2 2 2 2 2 2 2
2383 191/300 Read block 3657 len 40960
2384 192/300 Read block 2596 len 8192
2385 193/300 Read block 809 len 24576
2386 194/300 Write block 2961 len 32768 data: 2 2 2 2 2 2 2 2
2387 195/300 Write block 4013 len 32768 data: 2 2 2 2 2 2 2 2
2388 196/300 Read block 3985 len 8192
2389 197/300 Read block 1555 len 32768
2390 198/300 Read block 3951 len 16384
2391 199/300 Read block 2938 len 24576
2392 200/300 Read block 1345 len 8192
2393 201/300 Read block 488 len 24576
2394 202/300 Read block 1752 len 20480
2395 203/300 Write block 3814 len 28672 data: 3 4 4 4 4 4 4
2396 204/300 Write block 2716 len 20480 data: 5 3 3 3 3
2397 205/300 Read block 476 len 36864
2398 206/300 Write block 2192 len 32768 data: 2 2 2 2 2 3 3 3
2399 207/300 Write block 1034 len 12288 data: 2 2 2
2400 208/300 Read block 720 len 36864
2401 209/300 Read block 1328 len 28672
2402 210/300 Write block 2194 len 28672 data: 3 3 3 4 4 4 3
2403 211/300 Write block 4309 len 4096 data: 2
2404 212/300 Write block 2802 len 20480 data: 2 2 2 2 2
2405 213/300 Read block 4920 len 16384
2406 214/300 Write block 818 len 28672 data: 2 2 2 2 2 2 2
2407 215/300 Read block 124 len 24576
2408 216/300 Write block 1932 len 4096 data: 3
2409 217/300 Write block 4871 len 24576 data: 2 2 2 2 2 2
2410 218/300 Read block 4531 len 32768
2411 219/300 Write block 833 len 28672 data: 2 2 2 2 2 2 2
2412 220/300 Read block 2021 len 12288
2413 221/300 Write block 1880 len 8192 data: 3 3
2414 222/300 Flush
2415 223/300 Flush
2416 224/300 Read block 2280 len 4096
2417 225/300 Read block 3404 len 40960
2418 226/300 Write block 893 len 36864 data: 3 3 3 2 2 2 2 2 2
2419 227/300 Flush
2420 228/300 Read block 1441 len 4096
2421 229/300 Read block 1650 len 20480
2422 230/300 Write block 9 len 20480 data: 3 3 2 2 2
2423 231/300 Write block 1850 len 32768 data: 2 2 2 2 2 2 2 2
2424 232/300 Write block 2823 len 8192 data: 2 2
2425 233/300 Read block 551 len 12288
2426 234/300 Write block 3418 len 28672 data: 2 2 2 2 2 2 2
2427 235/300 Write block 1194 len 8192 data: 2 3
2428 236/300 Read block 989 len 32768
2429 237/300 Read block 3940 len 40960
2430 238/300 Read block 4785 len 4096
2431 239/300 Write block 3943 len 20480 data: 4 4 4 4 4
2432 240/300 Read block 17 len 20480
2433 241/300 Read block 3059 len 24576
2434 242/300 Flush
2435 243/300 Write block 3600 len 20480 data: 2 2 2 2 2
2436 244/300 Write block 2045 len 32768 data: 2 2 2 2 2 2 2 2
2437 245/300 Flush
2438 246/300 Read block 1640 len 36864
2439 247/300 Write block 161 len 4096 data: 3
2440 248/300 Write block 3439 len 8192 data: 2 2
2441 249/300 Read block 1266 len 20480
2442 250/300 Write block 77 len 36864 data: 2 2 2 2 2 2 2 2 2
2443 251/300 Write block 2125 len 32768 data: 2 2 2 2 2 2 2 2
2444 252/300 Read block 3623 len 4096
2445 253/300 Write block 808 len 16384 data: 2 2 2 2
2446 254/300 Write block 511 len 8192 data: 2 2
2447 255/300 Read block 4949 len 28672
2448 256/300 Read block 3290 len 24576
2449 257/300 Write block 3226 len 28672 data: 3 3 3 2 2 2 2
2450 258/300 Write block 3168 len 12288 data: 2 2 2
2451 259/300 Write block 3063 len 36864 data: 2 2 2 2 2 2 2 2 2
2452 260/300 Read block 3437 len 12288
2453 261/300 Write block 147 len 36864 data: 2 3 3 2 2 2 2 2 2
2454 262/300 Read block 3262 len 8192
2455 263/300 Write block 1301 len 28672 data: 2 2 2 2 2 2 2
2456 264/300 Write block 1525 len 32768 data: 2 2 2 2 2 2 2 2
2457 265/300 Flush
2458 266/300 Write block 4450 len 4096 data: 2
2459 267/300 Read block 779 len 12288
2460 268/300 Read block 1108 len 24576
2461 269/300 Read block 2250 len 16384
2462 270/300 Write block 3561 len 4096 data: 3
2463 271/300 Write block 227 len 32768 data: 2 2 2 2 3 2 2 2
2464 272/300 Write block 2852 len 24576 data: 2 2 2 3 3 3
2465 273/300 Write block 1571 len 8192 data: 2 2
2466 274/300 Write block 2264 len 40960 data: 2 2 2 3 3 3 3 3 3 3
2467 275/300 Write block 1204 len 24576 data: 2 2 2 2 2 2
2468 276/300 Read block 4110 len 36864
2469 277/300 Read block 1618 len 36864
2470 278/300 Read block 3349 len 16384
2471 279/300 Write block 4340 len 12288 data: 2 2 2
2472 280/300 Read block 4040 len 24576
2473 281/300 Read block 3744 len 28672
2474 282/300 Write block 1410 len 20480 data: 2 2 2 2 2
2475 283/300 Read block 3482 len 28672
2476 284/300 Flush
2477 285/300 Write block 1114 len 12288 data: 2 2 2
2478 286/300 Write block 2511 len 28672 data: 3 3 3 3 3 3 3
2479 287/300 Write block 2718 len 4096 data: 4
2480 288/300 Read block 0 len 28672
2481 289/300 Write block 2943 len 20480 data: 2 2 2 2 2
2482 290/300 Read block 1829 len 8192
2483 291/300 Write block 3414 len 28672 data: 2 2 2 2 3 3 3
2484 292/300 Write block 4469 len 16384 data: 2 2 2 2
2485 293/300 Write block 4892 len 20480 data: 3 3 3 3 3
2486 294/300 Write block 2267 len 20480 data: 4 4 4 4 4
2487 295/300 Flush
2488 296/300 Write block 2650 len 4096 data: 4
2489 297/300 Read block 390 len 12288
2490 298/300 Read block 2950 len 24576
2491 299/300 Write block 3640 len 16384 data: 2 2 2 2
2492 300/300 Flush
2493 301/300 Read block 237 len 20480
2494 Main task has work to do, go find it
2495 got dsc Start(2)
2496 start 2
2497 Dec 11 23:41:03.258 INFO request completed, latency_us: 229, response_code: 204, uri: /start/cid/2, method: POST, req_id: 1552abf1-f421-4436-bac8-cc0a9e5eed8d, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
2498 Replay: started 2, returned:Ok(())
2499 ----------------------------------------------------------------
2500 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2501 States: Active Active Offline
2502 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2503 1604 Acked Read 8 Done Done Sent false
2504 1605 Acked Read 8 Done Done Sent false
2505 1606 Acked Read 4 Done Done Sent false
2506 1607 Acked Write 4 Done Done Sent false
2507 1608 Acked Read 6 Done Done Sent false
2508 1609 Acked Read 9 Done Done Sent false
2509 1610 Acked Read 10 Done Done Sent false
2510 1611 Acked Read 1 Done Done Sent false
2511 1612 Acked Write 7 Done Done Sent false
2512 1613 Acked Flush 0 Done Done Sent false
2513 1614 Acked Read 2 Done Done Sent false
2514 1615 Acked Read 2 Done Done Sent false
2515 1616 Acked Write 5 Done Done Sent false
2516 1617 Acked Read 5 Done Done Sent false
2517 1618 Acked Read 10 Done Done Sent false
2518 1619 Acked Read 8 Done Done Sent false
2519 1620 Acked Read 3 Done Done Sent false
2520 1621 Acked Read 5 Done Done Sent false
2521 1622 Acked Write 1 Done Done Sent false
2522 1623 Acked Read 7 Done Done Sent false
2523 1624 Acked Read 1 Done Done Sent false
2524 1625 Acked Read 6 Done Done Sent false
2525 1626 Acked Read 8 Done Done Sent false
2526 1627 Acked Write 9 Done Done Sent false
2527 1628 Acked Write 4 Done Done Sent false
2528 1629 Acked Flush 0 Done Done Sent false
2529 1630 Acked Read 5 Done Done Sent false
2530 1631 Acked Write 10 Done Done Sent false
2531 1632 Acked Write 5 Done Done Sent false
2532 1633 Acked Read 9 Done Done Sent false
2533 1634 Acked Read 4 Done Done Sent false
2534 1635 Acked Read 4 Done Done Sent false
2535 1636 Acked Write 7 Done Done Sent false
2536 1637 Acked Write 3 Done Done Sent false
2537 1638 Acked Flush 0 Done Done Sent false
2538 1639 Acked Write 5 Done Done Sent false
2539 1640 Acked Read 1 Done Done Sent false
2540 1641 Acked Read 4 Done Done Sent false
2541 1642 Acked Read 2 Done Done Sent false
2542 1643 Acked Write 9 Done Done Sent false
2543 1644 Acked Read 5 Done Done Sent false
2544 1645 Acked Read 1 Done Done Sent false
2545 1646 Acked Read 3 Done Done Sent false
2546 1647 Acked Read 3 Done Done Sent false
2547 1648 Acked Read 7 Done Done Sent false
2548 1649 Acked Write 4 Done Done Sent false
2549 1650 Acked Write 3 Done Done Sent false
2550 1651 Acked Read 1 Done Done Sent false
2551 1652 Acked Read 5 Done Done Sent false
2552 1653 Acked Write 10 Done Done Sent false
2553 1654 Acked Read 9 Done Done Sent false
2554 1655 Acked Read 3 Done Done Sent false
2555 1656 Acked Read 4 Done Done Sent false
2556 1657 Acked Read 9 Done Done Sent false
2557 1658 Acked Read 7 Done Done Sent false
2558 1659 Acked Write 7 Done Done Sent false
2559 1660 Acked Write 4 Done Done Sent false
2560 1661 Acked Write 9 Done Done Sent false
2561 1662 Acked Write 6 Done Done Sent false
2562 1663 Acked Read 9 Done Done Sent false
2563 1664 Acked Flush 0 Done Done Sent false
2564 1665 Acked Read 5 Done Done Sent false
2565 1666 Acked Read 10 Done Done Sent false
2566 1667 Acked Write 4 Done Done Sent false
2567 1668 Acked Read 10 Done Done Sent false
2568 1669 Acked Read 5 Done Done Sent false
2569 1670 Acked Write 10 Done Done Sent false
2570 1671 Acked Read 1 Done Done Sent false
2571 1672 Acked Read 6 Done Done Sent false
2572 1673 Acked Read 1 Done Done Sent false
2573 1674 Acked Flush 0 Done Done Sent false
2574 1675 Acked Flush 0 Done Done Sent false
2575 1676 Acked Write 3 Done Done Sent false
2576 1677 Acked Read 9 Done Done Sent false
2577 1678 Acked Flush 0 Done Done Sent false
2578 1679 Acked Read 7 Done Done Sent false
2579 1680 Acked Write 10 Done Done Sent false
2580 1681 Acked Write 8 Done Done Sent false
2581 1682 Acked Read 4 Done Done Sent false
2582 1683 Acked Read 10 Done Done Sent false
2583 1684 Acked Read 4 Done Done Sent false
2584 1685 Acked Write 10 Done Done Sent false
2585 1686 Acked Write 2 Done Done Sent false
2586 1687 Acked Read 9 Done Done Sent false
2587 1688 Acked Read 9 Done Done Sent false
2588 1689 Acked Read 4 Done Done Sent false
2589 1690 Acked Write 4 Done Done Sent false
2590 1691 Acked Write 7 Done Done Sent false
2591 1692 Acked Flush 0 Done Done Sent false
2592 1693 Acked Flush 0 Done Done Sent false
2593 1694 Acked Read 2 Done Done Sent false
2594 1695 Acked Write 3 Done Done Sent false
2595 1696 Acked Flush 0 Done Done Sent false
2596 1697 Acked Write 1 Done Done Sent false
2597 1698 Acked Read 8 Done Done Sent false
2598 1699 Acked Write 9 Done Done Sent false
2599 1700 Acked Write 10 Done Done Sent false
2600 1701 Acked Flush 0 Done Done Sent false
2601 1702 Acked Write 5 Done Done Sent false
2602 1703 Acked Read 5 Done Done Sent false
2603 1704 Acked Write 6 Done Done Sent false
2604 1705 Acked Read 2 Done Done Sent false
2605 1706 Acked Write 2 Done Done Sent false
2606 1707 Acked Read 2 Done Done Sent false
2607 1708 Acked Write 5 Done Done Sent false
2608 1709 Acked Read 5 Done Done Sent false
2609 1710 Acked Write 9 Done Done Sent false
2610 1711 Acked Write 7 Done Done Sent false
2611 1712 Acked Read 6 Done Done Sent false
2612 1713 Acked Read 4 Done Done Sent false
2613 1714 Acked Write 10 Done Done Sent false
2614 1715 Acked Flush 0 Done Done Sent false
2615 1716 Acked Read 2 Done Done Sent false
2616 1717 Acked Write 10 Done Done Sent false
2617 1718 Acked Write 3 Done Done Sent false
2618 1719 Acked Read 4 Done Done Sent false
2619 1720 Acked Write 3 Done Done Sent false
2620 1721 Acked Write 1 Done Done Sent false
2621 1722 Acked Read 2 Done Done Sent false
2622 1723 Acked Flush 0 Done Done Sent false
2623 1724 Acked Write 10 Done Done Sent false
2624 1725 Acked Flush 0 Done Done Sent false
2625 1726 Acked Read 3 Done Done Sent false
2626 1727 Acked Read 6 Done Done Sent false
2627 1728 Acked Write 5 Done Done Sent false
2628 1729 Acked Write 1 Done Done Sent false
2629 1730 Acked Read 8 Done Done Sent false
2630 1731 Acked Write 4 Done Done Sent false
2631 1732 Acked Read 9 Done Done Sent false
2632 1733 Acked Read 1 Done Done Sent false
2633 1734 Acked Write 10 Done Done Sent false
2634 1735 Acked Read 4 Done Done Sent false
2635 1736 Acked Write 8 Done Done Sent false
2636 1737 Acked Write 2 Done Done Sent false
2637 1738 Acked Flush 0 Done Done Sent false
2638 1739 Acked Flush 0 Done Done Sent false
2639 1740 Acked Write 1 Done Done Sent false
2640 1741 Acked Write 5 Done Done Sent false
2641 1742 Acked Write 2 Done Done Sent false
2642 1743 Acked Write 3 Done Done Sent false
2643 1744 Acked Write 8 Done Done Sent false
2644 1745 Acked Write 10 Done Done Sent false
2645 1746 Acked Write 6 Done Done Sent false
2646 1747 Acked Write 4 Done Done Sent false
2647 1748 Acked Read 2 Done Done Sent false
2648 1749 Acked Write 9 Done Done Sent false
2649 1750 Acked Write 1 Done Done Sent false
2650 1751 Acked Write 4 Done Done Sent false
2651 1752 Acked Flush 0 Done Done Sent false
2652 1753 Acked Read 1 Done Done Sent false
2653 1754 Acked Read 9 Done Done Sent false
2654 1755 Acked Read 1 Done Done Sent false
2655 1756 Acked Write 7 Done Done Sent false
2656 1757 Acked Write 2 Done Done Sent false
2657 1758 Acked Write 6 Done Done Sent false
2658 1759 Acked Write 8 Done Done Sent false
2659 1760 Acked Read 4 Done Done Sent false
2660 1761 Acked Flush 0 Done Done Sent false
2661 1762 Acked Write 7 Done Done Sent false
2662 1763 Acked Write 10 Done Done Sent false
2663 1764 Acked Write 9 Done Done Sent false
2664 1765 Acked Flush 0 Done Done Sent false
2665 1766 Acked Write 3 Done Done Sent false
2666 1767 Acked Write 5 Done Done Sent false
2667 1768 Acked Read 8 Done Done Sent false
2668 1769 Acked Read 1 Done Done Sent false
2669 1770 Acked Flush 0 Done Done Sent false
2670 1771 Acked Read 7 Done Done Sent false
2671 1772 Acked Read 7 Done Done Sent false
2672 1773 Acked Read 5 Done Done Sent false
2673 1774 Acked Write 4 Done Done Sent false
2674 1775 Acked Read 2 Done Done Sent false
2675 1776 Acked Read 2 Done Done Sent false
2676 1777 Acked Flush 0 Done Done Sent false
2677 1778 Acked Flush 0 Done Done Sent false
2678 1779 Acked Read 6 Done Done Sent false
2679 1780 Acked Write 5 Done Done Sent false
2680 1781 Acked Write 4 Done Done Sent false
2681 1782 Acked Read 6 Done Done Sent false
2682 1783 Acked Write 1 Done Done Sent false
2683 1784 Acked Write 3 Done Done Sent false
2684 1785 Acked Read 10 Done Done Sent false
2685 1786 Acked Write 2 Done Done Sent false
2686 1787 Acked Write 6 Done Done Sent false
2687 1788 Acked Read 3 Done Done Sent false
2688 1789 Acked Read 1 Done Done Sent false
2689 1790 Acked Flush 0 Done Done Sent false
2690 1791 Acked Write 1 Done Done Sent false
2691 1792 Acked Write 5 Done Done Sent false
2692 1793 Acked Write 8 Done Done Sent false
2693 1794 Acked Read 10 Done Done Sent false
2694 1795 Acked Read 2 Done Done Sent false
2695 1796 Acked Read 6 Done Done Sent false
2696 1797 Acked Write 8 Done Done Sent false
2697 1798 Acked Write 8 Done Done Sent false
2698 1799 Acked Read 2 Done Done Sent false
2699 1800 Acked Read 8 Done Done Sent false
2700 1801 Acked Read 4 Done Done Sent false
2701 1802 Acked Read 6 Done Done Sent false
2702 1803 Acked Read 2 Done Done Sent false
2703 1804 Acked Read 6 Done Done Sent false
2704 1805 Acked Read 5 Done Done Sent false
2705 1806 Acked Write 7 Done Done Sent false
2706 1807 Acked Write 5 Done Done Sent false
2707 1808 Acked Read 9 Done Done Sent false
2708 1809 Acked Write 8 Done Done Sent false
2709 1810 Acked Write 3 Done Done Sent false
2710 1811 Acked Read 9 Done Done Sent false
2711 1812 Acked Read 7 Done Done Sent false
2712 1813 Acked Write 7 Done Done Sent false
2713 1814 Acked Write 1 Done Done Sent false
2714 1815 Acked Write 5 Done Done Sent false
2715 1816 Acked Read 4 Done Done Sent false
2716 1817 Acked Write 7 Done Done Sent false
2717 1818 Acked Read 6 Done Done Sent false
2718 1819 Acked Write 1 Done Done Sent false
2719 1820 Acked Write 6 Done Done Sent false
2720 1821 Acked Read 8 Done Done Sent false
2721 1822 Acked Write 7 Done Done Sent false
2722 1823 Acked Read 3 Done Done Sent false
2723 1824 Acked Write 2 Done Done Sent false
2724 1825 Acked Flush 0 Done Done Sent false
2725 1826 Acked Flush 0 Done Done Sent false
2726 1827 Acked Read 1 Done Done Sent false
2727 1828 Acked Read 10 Done Done Sent false
2728 1829 Acked Write 9 Done Done Sent false
2729 1830 Acked Flush 0 Done Done Sent false
2730 1831 Acked Read 1 Done Done Sent false
2731 1832 Acked Read 5 Done Done Sent false
2732 1833 Acked Write 5 Done Done Sent false
2733 1834 Acked Write 8 Done Done Sent false
2734 1835 Acked Write 2 Done Done Sent false
2735 1836 Acked Read 3 Done Done Sent false
2736 1837 Acked Write 7 Done Done Sent false
2737 1838 Acked Write 2 Done Done Sent false
2738 1839 Acked Read 8 Done Done Sent false
2739 1840 Acked Read 10 Done Done Sent false
2740 1841 Acked Read 1 Done Done Sent false
2741 1842 Acked Write 5 Done Done Sent false
2742 1843 Acked Read 5 Done Done Sent false
2743 1844 Acked Read 6 Done Done Sent false
2744 1845 Acked Flush 0 Done Done Sent false
2745 1846 Acked Write 5 Done Done Sent false
2746 1847 Acked Write 8 Done Done Sent false
2747 1848 Acked Flush 0 Done Done Sent false
2748 1849 Acked Read 9 Done Done Sent false
2749 1850 Acked Write 1 Done Done Sent false
2750 1851 Acked Write 2 Done Done Sent false
2751 1852 Acked Read 5 Done Done Sent false
2752 1853 Acked Write 9 Done Done Sent false
2753 1854 Acked Write 8 Done Done Sent false
2754 1855 Acked Read 1 Done Done Sent false
2755 1856 Acked Write 4 Done Done Sent false
2756 1857 Acked Write 2 Done Done Sent false
2757 1858 Acked Read 7 Done Done Sent false
2758 1859 Acked Read 6 Done Done Sent false
2759 1860 Acked Write 7 Done Done Sent false
2760 1861 Acked Write 3 Done Done Sent false
2761 1862 Acked Write 9 Done Done Sent false
2762 1863 Acked Read 3 Done Done Sent false
2763 1864 Acked Write 9 Done Done Sent false
2764 1865 Acked Read 2 Done Done Sent false
2765 1866 Acked Write 7 Done Done Sent false
2766 1867 Acked Write 8 Done Done Sent false
2767 1868 Acked Flush 0 Done Done Sent false
2768 1869 Acked Write 1 Done Done Sent false
2769 1870 Acked Read 3 Done Done Sent false
2770 1871 Acked Read 6 Done Done Sent false
2771 1872 Acked Read 4 Done Done Sent false
2772 1873 Acked Write 1 Done Done Sent false
2773 1874 Acked Write 8 Done Done Sent false
2774 1875 Acked Write 6 Done Done Sent false
2775 1876 Acked Write 2 Done Done Sent false
2776 1877 Acked Write 10 Done Done Sent false
2777 1878 Acked Write 6 Done Done Sent false
2778 1879 Acked Read 9 Done Done Sent false
2779 1880 Acked Read 9 Done Done Sent false
2780 1881 Acked Read 4 Done Done Sent false
2781 1882 Acked Write 3 Done Done Sent false
2782 1883 Acked Read 6 Done Done Sent false
2783 1884 Acked Read 7 Done Done Sent false
2784 1885 Acked Write 5 Done Done Sent false
2785 1886 Acked Read 7 Done Done Sent false
2786 1887 Acked Flush 0 Done Done Sent false
2787 1888 Acked Write 3 Done Done Sent false
2788 1889 Acked Write 7 Done Done Sent false
2789 1890 Acked Write 1 Done Done Sent false
2790 1891 Acked Read 7 Done Done Sent false
2791 1892 Acked Write 5 Done Done Sent false
2792 1893 Acked Read 2 Done Done Sent false
2793 1894 Acked Write 7 Done Done Sent false
2794 1895 Acked Write 4 Done Done Sent false
2795 1896 Acked Write 5 Done Done Sent false
2796 1897 Acked Write 5 Done Done Sent false
2797 1898 Acked Flush 0 Done Done Sent false
2798 1899 Acked Write 1 Done Done Sent false
2799 1900 Acked Read 3 Done Done Sent false
2800 1901 Acked Read 6 Done Done Sent false
2801 1902 Acked Write 4 Done Done Sent false
2802 1903 Acked Flush 0 Done Done Sent false
2803 1904 Acked Read 5 Done Done Sent false
2804 STATES DS:0 DS:1 DS:2 TOTAL
2805 Sent 0 0 301 301
2806 Done 301 301 0 602
2807 Skipped 0 0 0 0
2808 Error 0 0 0 0
2809 Last Flush: 1903 1903 1603
2810 Downstairs last five completed: 1603 1602 1601 1600 1599
2811 Upstairs last five completed: 1904 1903 1902 1901 1900
2812 CLIENT: Up:0 ds:301 act:2
2813 ----------------------------------------------------------------
2814 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2815 States: Active Active Offline
2816 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2817 1604 Acked Read 8 Done Done Sent false
2818 1605 Acked Read 8 Done Done Sent false
2819 1606 Acked Read 4 Done Done Sent false
2820 1607 Acked Write 4 Done Done Sent false
2821 1608 Acked Read 6 Done Done Sent false
2822 1609 Acked Read 9 Done Done Sent false
2823 1610 Acked Read 10 Done Done Sent false
2824 1611 Acked Read 1 Done Done Sent false
2825 1612 Acked Write 7 Done Done Sent false
2826 1613 Acked Flush 0 Done Done Sent false
2827 1614 Acked Read 2 Done Done Sent false
2828 1615 Acked Read 2 Done Done Sent false
2829 1616 Acked Write 5 Done Done Sent false
2830 1617 Acked Read 5 Done Done Sent false
2831 1618 Acked Read 10 Done Done Sent false
2832 1619 Acked Read 8 Done Done Sent false
2833 1620 Acked Read 3 Done Done Sent false
2834 1621 Acked Read 5 Done Done Sent false
2835 1622 Acked Write 1 Done Done Sent false
2836 1623 Acked Read 7 Done Done Sent false
2837 1624 Acked Read 1 Done Done Sent false
2838 1625 Acked Read 6 Done Done Sent false
2839 1626 Acked Read 8 Done Done Sent false
2840 1627 Acked Write 9 Done Done Sent false
2841 1628 Acked Write 4 Done Done Sent false
2842 1629 Acked Flush 0 Done Done Sent false
2843 1630 Acked Read 5 Done Done Sent false
2844 1631 Acked Write 10 Done Done Sent false
2845 1632 Acked Write 5 Done Done Sent false
2846 1633 Acked Read 9 Done Done Sent false
2847 1634 Acked Read 4 Done Done Sent false
2848 1635 Acked Read 4 Done Done Sent false
2849 1636 Acked Write 7 Done Done Sent false
2850 1637 Acked Write 3 Done Done Sent false
2851 1638 Acked Flush 0 Done Done Sent false
2852 1639 Acked Write 5 Done Done Sent false
2853 1640 Acked Read 1 Done Done Sent false
2854 1641 Acked Read 4 Done Done Sent false
2855 1642 Acked Read 2 Done Done Sent false
2856 1643 Acked Write 9 Done Done Sent false
2857 1644 Acked Read 5 Done Done Sent false
2858 1645 Acked Read 1 Done Done Sent false
2859 1646 Acked Read 3 Done Done Sent false
2860 1647 Acked Read 3 Done Done Sent false
2861 1648 Acked Read 7 Done Done Sent false
2862 1649 Acked Write 4 Done Done Sent false
2863 1650 Acked Write 3 Done Done Sent false
2864 1651 Acked Read 1 Done Done Sent false
2865 1652 Acked Read 5 Done Done Sent false
2866 1653 Acked Write 10 Done Done Sent false
2867 1654 Acked Read 9 Done Done Sent false
2868 1655 Acked Read 3 Done Done Sent false
2869 1656 Acked Read 4 Done Done Sent false
2870 1657 Acked Read 9 Done Done Sent false
2871 1658 Acked Read 7 Done Done Sent false
2872 1659 Acked Write 7 Done Done Sent false
2873 1660 Acked Write 4 Done Done Sent false
2874 1661 Acked Write 9 Done Done Sent false
2875 1662 Acked Write 6 Done Done Sent false
2876 1663 Acked Read 9 Done Done Sent false
2877 1664 Acked Flush 0 Done Done Sent false
2878 1665 Acked Read 5 Done Done Sent false
2879 1666 Acked Read 10 Done Done Sent false
2880 1667 Acked Write 4 Done Done Sent false
2881 1668 Acked Read 10 Done Done Sent false
2882 1669 Acked Read 5 Done Done Sent false
2883 1670 Acked Write 10 Done Done Sent false
2884 1671 Acked Read 1 Done Done Sent false
2885 1672 Acked Read 6 Done Done Sent false
2886 1673 Acked Read 1 Done Done Sent false
2887 1674 Acked Flush 0 Done Done Sent false
2888 1675 Acked Flush 0 Done Done Sent false
2889 1676 Acked Write 3 Done Done Sent false
2890 1677 Acked Read 9 Done Done Sent false
2891 1678 Acked Flush 0 Done Done Sent false
2892 1679 Acked Read 7 Done Done Sent false
2893 1680 Acked Write 10 Done Done Sent false
2894 1681 Acked Write 8 Done Done Sent false
2895 1682 Acked Read 4 Done Done Sent false
2896 1683 Acked Read 10 Done Done Sent false
2897 1684 Acked Read 4 Done Done Sent false
2898 1685 Acked Write 10 Done Done Sent false
2899 1686 Acked Write 2 Done Done Sent false
2900 1687 Acked Read 9 Done Done Sent false
2901 1688 Acked Read 9 Done Done Sent false
2902 1689 Acked Read 4 Done Done Sent false
2903 1690 Acked Write 4 Done Done Sent false
2904 1691 Acked Write 7 Done Done Sent false
2905 1692 Acked Flush 0 Done Done Sent false
2906 1693 Acked Flush 0 Done Done Sent false
2907 1694 Acked Read 2 Done Done Sent false
2908 1695 Acked Write 3 Done Done Sent false
2909 1696 Acked Flush 0 Done Done Sent false
2910 1697 Acked Write 1 Done Done Sent false
2911 1698 Acked Read 8 Done Done Sent false
2912 1699 Acked Write 9 Done Done Sent false
2913 1700 Acked Write 10 Done Done Sent false
2914 1701 Acked Flush 0 Done Done Sent false
2915 1702 Acked Write 5 Done Done Sent false
2916 1703 Acked Read 5 Done Done Sent false
2917 1704 Acked Write 6 Done Done Sent false
2918 1705 Acked Read 2 Done Done Sent false
2919 1706 Acked Write 2 Done Done Sent false
2920 1707 Acked Read 2 Done Done Sent false
2921 1708 Acked Write 5 Done Done Sent false
2922 1709 Acked Read 5 Done Done Sent false
2923 1710 Acked Write 9 Done Done Sent false
2924 1711 Acked Write 7 Done Done Sent false
2925 1712 Acked Read 6 Done Done Sent false
2926 1713 Acked Read 4 Done Done Sent false
2927 1714 Acked Write 10 Done Done Sent false
2928 1715 Acked Flush 0 Done Done Sent false
2929 1716 Acked Read 2 Done Done Sent false
2930 1717 Acked Write 10 Done Done Sent false
2931 1718 Acked Write 3 Done Done Sent false
2932 1719 Acked Read 4 Done Done Sent false
2933 1720 Acked Write 3 Done Done Sent false
2934 1721 Acked Write 1 Done Done Sent false
2935 1722 Acked Read 2 Done Done Sent false
2936 1723 Acked Flush 0 Done Done Sent false
2937 1724 Acked Write 10 Done Done Sent false
2938 1725 Acked Flush 0 Done Done Sent false
2939 1726 Acked Read 3 Done Done Sent false
2940 1727 Acked Read 6 Done Done Sent false
2941 1728 Acked Write 5 Done Done Sent false
2942 1729 Acked Write 1 Done Done Sent false
2943 1730 Acked Read 8 Done Done Sent false
2944 1731 Acked Write 4 Done Done Sent false
2945 1732 Acked Read 9 Done Done Sent false
2946 1733 Acked Read 1 Done Done Sent false
2947 1734 Acked Write 10 Done Done Sent false
2948 1735 Acked Read 4 Done Done Sent false
2949 1736 Acked Write 8 Done Done Sent false
2950 1737 Acked Write 2 Done Done Sent false
2951 1738 Acked Flush 0 Done Done Sent false
2952 1739 Acked Flush 0 Done Done Sent false
2953 1740 Acked Write 1 Done Done Sent false
2954 1741 Acked Write 5 Done Done Sent false
2955 1742 Acked Write 2 Done Done Sent false
2956 1743 Acked Write 3 Done Done Sent false
2957 1744 Acked Write 8 Done Done Sent false
2958 1745 Acked Write 10 Done Done Sent false
2959 1746 Acked Write 6 Done Done Sent false
2960 1747 Acked Write 4 Done Done Sent false
2961 1748 Acked Read 2 Done Done Sent false
2962 1749 Acked Write 9 Done Done Sent false
2963 1750 Acked Write 1 Done Done Sent false
2964 1751 Acked Write 4 Done Done Sent false
2965 1752 Acked Flush 0 Done Done Sent false
2966 1753 Acked Read 1 Done Done Sent false
2967 1754 Acked Read 9 Done Done Sent false
2968 1755 Acked Read 1 Done Done Sent false
2969 1756 Acked Write 7 Done Done Sent false
2970 1757 Acked Write 2 Done Done Sent false
2971 1758 Acked Write 6 Done Done Sent false
2972 1759 Acked Write 8 Done Done Sent false
2973 1760 Acked Read 4 Done Done Sent false
2974 1761 Acked Flush 0 Done Done Sent false
2975 1762 Acked Write 7 Done Done Sent false
2976 1763 Acked Write 10 Done Done Sent false
2977 1764 Acked Write 9 Done Done Sent false
2978 1765 Acked Flush 0 Done Done Sent false
2979 1766 Acked Write 3 Done Done Sent false
2980 1767 Acked Write 5 Done Done Sent false
2981 1768 Acked Read 8 Done Done Sent false
2982 1769 Acked Read 1 Done Done Sent false
2983 1770 Acked Flush 0 Done Done Sent false
2984 1771 Acked Read 7 Done Done Sent false
2985 1772 Acked Read 7 Done Done Sent false
2986 1773 Acked Read 5 Done Done Sent false
2987 1774 Acked Write 4 Done Done Sent false
2988 1775 Acked Read 2 Done Done Sent false
2989 1776 Acked Read 2 Done Done Sent false
2990 1777 Acked Flush 0 Done Done Sent false
2991 1778 Acked Flush 0 Done Done Sent false
2992 1779 Acked Read 6 Done Done Sent false
2993 1780 Acked Write 5 Done Done Sent false
2994 1781 Acked Write 4 Done Done Sent false
2995 1782 Acked Read 6 Done Done Sent false
2996 1783 Acked Write 1 Done Done Sent false
2997 1784 Acked Write 3 Done Done Sent false
2998 1785 Acked Read 10 Done Done Sent false
2999 1786 Acked Write 2 Done Done Sent false
3000 1787 Acked Write 6 Done Done Sent false
3001 1788 Acked Read 3 Done Done Sent false
3002 1789 Acked Read 1 Done Done Sent false
3003 1790 Acked Flush 0 Done Done Sent false
3004 1791 Acked Write 1 Done Done Sent false
3005 1792 Acked Write 5 Done Done Sent false
3006 1793 Acked Write 8 Done Done Sent false
3007 1794 Acked Read 10 Done Done Sent false
3008 1795 Acked Read 2 Done Done Sent false
3009 1796 Acked Read 6 Done Done Sent false
3010 1797 Acked Write 8 Done Done Sent false
3011 1798 Acked Write 8 Done Done Sent false
3012 1799 Acked Read 2 Done Done Sent false
3013 1800 Acked Read 8 Done Done Sent false
3014 1801 Acked Read 4 Done Done Sent false
3015 1802 Acked Read 6 Done Done Sent false
3016 1803 Acked Read 2 Done Done Sent false
3017 1804 Acked Read 6 Done Done Sent false
3018 1805 Acked Read 5 Done Done Sent false
3019 1806 Acked Write 7 Done Done Sent false
3020 1807 Acked Write 5 Done Done Sent false
3021 1808 Acked Read 9 Done Done Sent false
3022 1809 Acked Write 8 Done Done Sent false
3023 1810 Acked Write 3 Done Done Sent false
3024 1811 Acked Read 9 Done Done Sent false
3025 1812 Acked Read 7 Done Done Sent false
3026 1813 Acked Write 7 Done Done Sent false
3027 1814 Acked Write 1 Done Done Sent false
3028 1815 Acked Write 5 Done Done Sent false
3029 1816 Acked Read 4 Done Done Sent false
3030 1817 Acked Write 7 Done Done Sent false
3031 1818 Acked Read 6 Done Done Sent false
3032 1819 Acked Write 1 Done Done Sent false
3033 1820 Acked Write 6 Done Done Sent false
3034 1821 Acked Read 8 Done Done Sent false
3035 1822 Acked Write 7 Done Done Sent false
3036 1823 Acked Read 3 Done Done Sent false
3037 1824 Acked Write 2 Done Done Sent false
3038 1825 Acked Flush 0 Done Done Sent false
3039 1826 Acked Flush 0 Done Done Sent false
3040 1827 Acked Read 1 Done Done Sent false
3041 1828 Acked Read 10 Done Done Sent false
3042 1829 Acked Write 9 Done Done Sent false
3043 1830 Acked Flush 0 Done Done Sent false
3044 1831 Acked Read 1 Done Done Sent false
3045 1832 Acked Read 5 Done Done Sent false
3046 1833 Acked Write 5 Done Done Sent false
3047 1834 Acked Write 8 Done Done Sent false
3048 1835 Acked Write 2 Done Done Sent false
3049 1836 Acked Read 3 Done Done Sent false
3050 1837 Acked Write 7 Done Done Sent false
3051 1838 Acked Write 2 Done Done Sent false
3052 1839 Acked Read 8 Done Done Sent false
3053 1840 Acked Read 10 Done Done Sent false
3054 1841 Acked Read 1 Done Done Sent false
3055 1842 Acked Write 5 Done Done Sent false
3056 1843 Acked Read 5 Done Done Sent false
3057 1844 Acked Read 6 Done Done Sent false
3058 1845 Acked Flush 0 Done Done Sent false
3059 1846 Acked Write 5 Done Done Sent false
3060 1847 Acked Write 8 Done Done Sent false
3061 1848 Acked Flush 0 Done Done Sent false
3062 1849 Acked Read 9 Done Done Sent false
3063 1850 Acked Write 1 Done Done Sent false
3064 1851 Acked Write 2 Done Done Sent false
3065 1852 Acked Read 5 Done Done Sent false
3066 1853 Acked Write 9 Done Done Sent false
3067 1854 Acked Write 8 Done Done Sent false
3068 1855 Acked Read 1 Done Done Sent false
3069 1856 Acked Write 4 Done Done Sent false
3070 1857 Acked Write 2 Done Done Sent false
3071 1858 Acked Read 7 Done Done Sent false
3072 1859 Acked Read 6 Done Done Sent false
3073 1860 Acked Write 7 Done Done Sent false
3074 1861 Acked Write 3 Done Done Sent false
3075 1862 Acked Write 9 Done Done Sent false
3076 1863 Acked Read 3 Done Done Sent false
3077 1864 Acked Write 9 Done Done Sent false
3078 1865 Acked Read 2 Done Done Sent false
3079 1866 Acked Write 7 Done Done Sent false
3080 1867 Acked Write 8 Done Done Sent false
3081 1868 Acked Flush 0 Done Done Sent false
3082 1869 Acked Write 1 Done Done Sent false
3083 1870 Acked Read 3 Done Done Sent false
3084 1871 Acked Read 6 Done Done Sent false
3085 1872 Acked Read 4 Done Done Sent false
3086 1873 Acked Write 1 Done Done Sent false
3087 1874 Acked Write 8 Done Done Sent false
3088 1875 Acked Write 6 Done Done Sent false
3089 1876 Acked Write 2 Done Done Sent false
3090 1877 Acked Write 10 Done Done Sent false
3091 1878 Acked Write 6 Done Done Sent false
3092 1879 Acked Read 9 Done Done Sent false
3093 1880 Acked Read 9 Done Done Sent false
3094 1881 Acked Read 4 Done Done Sent false
3095 1882 Acked Write 3 Done Done Sent false
3096 1883 Acked Read 6 Done Done Sent false
3097 1884 Acked Read 7 Done Done Sent false
3098 1885 Acked Write 5 Done Done Sent false
3099 1886 Acked Read 7 Done Done Sent false
3100 1887 Acked Flush 0 Done Done Sent false
3101 1888 Acked Write 3 Done Done Sent false
3102 1889 Acked Write 7 Done Done Sent false
3103 1890 Acked Write 1 Done Done Sent false
3104 1891 Acked Read 7 Done Done Sent false
3105 1892 Acked Write 5 Done Done Sent false
3106 1893 Acked Read 2 Done Done Sent false
3107 1894 Acked Write 7 Done Done Sent false
3108 1895 Acked Write 4 Done Done Sent false
3109 1896 Acked Write 5 Done Done Sent false
3110 1897 Acked Write 5 Done Done Sent false
3111 1898 Acked Flush 0 Done Done Sent false
3112 1899 Acked Write 1 Done Done Sent false
3113 1900 Acked Read 3 Done Done Sent false
3114 1901 Acked Read 6 Done Done Sent false
3115 1902 Acked Write 4 Done Done Sent false
3116 1903 Acked Flush 0 Done Done Sent false
3117 1904 Acked Read 5 Done Done Sent false
3118 1905 Acked Flush 0 Done Done Sent false
3119 STATES DS:0 DS:1 DS:2 TOTAL
3120 Sent 0 0 302 302
3121 Done 302 302 0 604
3122 Skipped 0 0 0 0
3123 Error 0 0 0 0
3124 Last Flush: 1905 1905 1603
3125 Downstairs last five completed: 1603 1602 1601 1600 1599
3126 Upstairs last five completed: 1905 1904 1903 1902 1901
3127 CLIENT: Up:0 ds:302 act:2
31282024-12-11T23:41:09.033ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
31292024-12-11T23:41:09.033ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
31302024-12-11T23:41:09.033ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
31312024-12-11T23:41:09.033ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
31322024-12-11T23:41:09.033ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
3133 ----------------------------------------------------------------
3134 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3135 States: Active Active Offline
3136 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3137 1604 Acked Read 8 Done Done Sent false
3138 1605 Acked Read 8 Done Done Sent false
3139 1606 Acked Read 4 Done Done Sent false
3140 1607 Acked Write 4 Done Done Sent false
3141 1608 Acked Read 6 Done Done Sent false
3142 1609 Acked Read 9 Done Done Sent false
3143 1610 Acked Read 10 Done Done Sent false
3144 1611 Acked Read 1 Done Done Sent false
3145 1612 Acked Write 7 Done Done Sent false
3146 1613 Acked Flush 0 Done Done Sent false
3147 1614 Acked Read 2 Done Done Sent false
3148 1615 Acked Read 2 Done Done Sent false
3149 1616 Acked Write 5 Done Done Sent false
3150 1617 Acked Read 5 Done Done Sent false
3151 1618 Acked Read 10 Done Done Sent false
3152 1619 Acked Read 8 Done Done Sent false
3153 1620 Acked Read 3 Done Done Sent false
3154 1621 Acked Read 5 Done Done Sent false
3155 1622 Acked Write 1 Done Done Sent false
3156 1623 Acked Read 7 Done Done Sent false
3157 1624 Acked Read 1 Done Done Sent false
3158 1625 Acked Read 6 Done Done Sent false
3159 1626 Acked Read 8 Done Done Sent false
3160 1627 Acked Write 9 Done Done Sent false
3161 1628 Acked Write 4 Done Done Sent false
3162 1629 Acked Flush 0 Done Done Sent false
3163 1630 Acked Read 5 Done Done Sent false
3164 1631 Acked Write 10 Done Done Sent false
3165 1632 Acked Write 5 Done Done Sent false
3166 1633 Acked Read 9 Done Done Sent false
3167 1634 Acked Read 4 Done Done Sent false
3168 1635 Acked Read 4 Done Done Sent false
3169 1636 Acked Write 7 Done Done Sent false
3170 1637 Acked Write 3 Done Done Sent false
3171 1638 Acked Flush 0 Done Done Sent false
3172 1639 Acked Write 5 Done Done Sent false
3173 1640 Acked Read 1 Done Done Sent false
3174 1641 Acked Read 4 Done Done Sent false
3175 1642 Acked Read 2 Done Done Sent false
3176 1643 Acked Write 9 Done Done Sent false
3177 1644 Acked Read 5 Done Done Sent false
3178 1645 Acked Read 1 Done Done Sent false
3179 1646 Acked Read 3 Done Done Sent false
3180 1647 Acked Read 3 Done Done Sent false
3181 1648 Acked Read 7 Done Done Sent false
3182 1649 Acked Write 4 Done Done Sent false
3183 1650 Acked Write 3 Done Done Sent false
3184 1651 Acked Read 1 Done Done Sent false
3185 1652 Acked Read 5 Done Done Sent false
3186 1653 Acked Write 10 Done Done Sent false
3187 1654 Acked Read 9 Done Done Sent false
3188 1655 Acked Read 3 Done Done Sent false
3189 1656 Acked Read 4 Done Done Sent false
3190 1657 Acked Read 9 Done Done Sent false
3191 1658 Acked Read 7 Done Done Sent false
3192 1659 Acked Write 7 Done Done Sent false
3193 1660 Acked Write 4 Done Done Sent false
3194 1661 Acked Write 9 Done Done Sent false
3195 1662 Acked Write 6 Done Done Sent false
3196 1663 Acked Read 9 Done Done Sent false
3197 1664 Acked Flush 0 Done Done Sent false
3198 1665 Acked Read 5 Done Done Sent false
3199 1666 Acked Read 10 Done Done Sent false
3200 1667 Acked Write 4 Done Done Sent false
3201 1668 Acked Read 10 Done Done Sent false
3202 1669 Acked Read 5 Done Done Sent false
3203 1670 Acked Write 10 Done Done Sent false
3204 1671 Acked Read 1 Done Done Sent false
3205 1672 Acked Read 6 Done Done Sent false
3206 1673 Acked Read 1 Done Done Sent false
3207 1674 Acked Flush 0 Done Done Sent false
3208 1675 Acked Flush 0 Done Done Sent false
3209 1676 Acked Write 3 Done Done Sent false
3210 1677 Acked Read 9 Done Done Sent false
3211 1678 Acked Flush 0 Done Done Sent false
3212 1679 Acked Read 7 Done Done Sent false
3213 1680 Acked Write 10 Done Done Sent false
3214 1681 Acked Write 8 Done Done Sent false
3215 1682 Acked Read 4 Done Done Sent false
3216 1683 Acked Read 10 Done Done Sent false
3217 1684 Acked Read 4 Done Done Sent false
3218 1685 Acked Write 10 Done Done Sent false
3219 1686 Acked Write 2 Done Done Sent false
3220 1687 Acked Read 9 Done Done Sent false
3221 1688 Acked Read 9 Done Done Sent false
3222 1689 Acked Read 4 Done Done Sent false
3223 1690 Acked Write 4 Done Done Sent false
3224 1691 Acked Write 7 Done Done Sent false
3225 1692 Acked Flush 0 Done Done Sent false
3226 1693 Acked Flush 0 Done Done Sent false
3227 1694 Acked Read 2 Done Done Sent false
3228 1695 Acked Write 3 Done Done Sent false
3229 1696 Acked Flush 0 Done Done Sent false
3230 1697 Acked Write 1 Done Done Sent false
3231 1698 Acked Read 8 Done Done Sent false
3232 1699 Acked Write 9 Done Done Sent false
3233 1700 Acked Write 10 Done Done Sent false
3234 1701 Acked Flush 0 Done Done Sent false
3235 1702 Acked Write 5 Done Done Sent false
3236 1703 Acked Read 5 Done Done Sent false
3237 1704 Acked Write 6 Done Done Sent false
3238 1705 Acked Read 2 Done Done Sent false
3239 1706 Acked Write 2 Done Done Sent false
3240 1707 Acked Read 2 Done Done Sent false
3241 1708 Acked Write 5 Done Done Sent false
3242 1709 Acked Read 5 Done Done Sent false
3243 1710 Acked Write 9 Done Done Sent false
3244 1711 Acked Write 7 Done Done Sent false
3245 1712 Acked Read 6 Done Done Sent false
3246 1713 Acked Read 4 Done Done Sent false
3247 1714 Acked Write 10 Done Done Sent false
3248 1715 Acked Flush 0 Done Done Sent false
3249 1716 Acked Read 2 Done Done Sent false
3250 1717 Acked Write 10 Done Done Sent false
3251 1718 Acked Write 3 Done Done Sent false
3252 1719 Acked Read 4 Done Done Sent false
3253 1720 Acked Write 3 Done Done Sent false
3254 1721 Acked Write 1 Done Done Sent false
3255 1722 Acked Read 2 Done Done Sent false
3256 1723 Acked Flush 0 Done Done Sent false
3257 1724 Acked Write 10 Done Done Sent false
3258 1725 Acked Flush 0 Done Done Sent false
3259 1726 Acked Read 3 Done Done Sent false
3260 1727 Acked Read 6 Done Done Sent false
3261 1728 Acked Write 5 Done Done Sent false
3262 1729 Acked Write 1 Done Done Sent false
3263 1730 Acked Read 8 Done Done Sent false
3264 1731 Acked Write 4 Done Done Sent false
3265 1732 Acked Read 9 Done Done Sent false
3266 1733 Acked Read 1 Done Done Sent false
3267 1734 Acked Write 10 Done Done Sent false
3268 1735 Acked Read 4 Done Done Sent false
3269 1736 Acked Write 8 Done Done Sent false
3270 1737 Acked Write 2 Done Done Sent false
3271 1738 Acked Flush 0 Done Done Sent false
3272 1739 Acked Flush 0 Done Done Sent false
3273 1740 Acked Write 1 Done Done Sent false
3274 1741 Acked Write 5 Done Done Sent false
3275 1742 Acked Write 2 Done Done Sent false
3276 1743 Acked Write 3 Done Done Sent false
3277 1744 Acked Write 8 Done Done Sent false
3278 1745 Acked Write 10 Done Done Sent false
3279 1746 Acked Write 6 Done Done Sent false
3280 1747 Acked Write 4 Done Done Sent false
3281 1748 Acked Read 2 Done Done Sent false
3282 1749 Acked Write 9 Done Done Sent false
3283 1750 Acked Write 1 Done Done Sent false
3284 1751 Acked Write 4 Done Done Sent false
3285 1752 Acked Flush 0 Done Done Sent false
3286 1753 Acked Read 1 Done Done Sent false
3287 1754 Acked Read 9 Done Done Sent false
3288 1755 Acked Read 1 Done Done Sent false
3289 1756 Acked Write 7 Done Done Sent false
3290 1757 Acked Write 2 Done Done Sent false
3291 1758 Acked Write 6 Done Done Sent false
3292 1759 Acked Write 8 Done Done Sent false
3293 1760 Acked Read 4 Done Done Sent false
3294 1761 Acked Flush 0 Done Done Sent false
3295 1762 Acked Write 7 Done Done Sent false
3296 1763 Acked Write 10 Done Done Sent false
3297 1764 Acked Write 9 Done Done Sent false
3298 1765 Acked Flush 0 Done Done Sent false
3299 1766 Acked Write 3 Done Done Sent false
3300 1767 Acked Write 5 Done Done Sent false
3301 1768 Acked Read 8 Done Done Sent false
3302 1769 Acked Read 1 Done Done Sent false
3303 1770 Acked Flush 0 Done Done Sent false
3304 1771 Acked Read 7 Done Done Sent false
3305 1772 Acked Read 7 Done Done Sent false
3306 1773 Acked Read 5 Done Done Sent false
3307 1774 Acked Write 4 Done Done Sent false
3308 1775 Acked Read 2 Done Done Sent false
3309 1776 Acked Read 2 Done Done Sent false
3310 1777 Acked Flush 0 Done Done Sent false
3311 1778 Acked Flush 0 Done Done Sent false
3312 1779 Acked Read 6 Done Done Sent false
3313 1780 Acked Write 5 Done Done Sent false
3314 1781 Acked Write 4 Done Done Sent false
3315 1782 Acked Read 6 Done Done Sent false
3316 1783 Acked Write 1 Done Done Sent false
3317 1784 Acked Write 3 Done Done Sent false
3318 1785 Acked Read 10 Done Done Sent false
3319 1786 Acked Write 2 Done Done Sent false
3320 1787 Acked Write 6 Done Done Sent false
3321 1788 Acked Read 3 Done Done Sent false
3322 1789 Acked Read 1 Done Done Sent false
3323 1790 Acked Flush 0 Done Done Sent false
3324 1791 Acked Write 1 Done Done Sent false
3325 1792 Acked Write 5 Done Done Sent false
3326 1793 Acked Write 8 Done Done Sent false
3327 1794 Acked Read 10 Done Done Sent false
3328 1795 Acked Read 2 Done Done Sent false
3329 1796 Acked Read 6 Done Done Sent false
3330 1797 Acked Write 8 Done Done Sent false
3331 1798 Acked Write 8 Done Done Sent false
3332 1799 Acked Read 2 Done Done Sent false
3333 1800 Acked Read 8 Done Done Sent false
3334 1801 Acked Read 4 Done Done Sent false
3335 1802 Acked Read 6 Done Done Sent false
3336 1803 Acked Read 2 Done Done Sent false
3337 1804 Acked Read 6 Done Done Sent false
3338 1805 Acked Read 5 Done Done Sent false
3339 1806 Acked Write 7 Done Done Sent false
3340 1807 Acked Write 5 Done Done Sent false
3341 1808 Acked Read 9 Done Done Sent false
3342 1809 Acked Write 8 Done Done Sent false
3343 1810 Acked Write 3 Done Done Sent false
3344 1811 Acked Read 9 Done Done Sent false
3345 1812 Acked Read 7 Done Done Sent false
3346 1813 Acked Write 7 Done Done Sent false
3347 1814 Acked Write 1 Done Done Sent false
3348 1815 Acked Write 5 Done Done Sent false
3349 1816 Acked Read 4 Done Done Sent false
3350 1817 Acked Write 7 Done Done Sent false
3351 1818 Acked Read 6 Done Done Sent false
3352 1819 Acked Write 1 Done Done Sent false
3353 1820 Acked Write 6 Done Done Sent false
3354 1821 Acked Read 8 Done Done Sent false
3355 1822 Acked Write 7 Done Done Sent false
3356 1823 Acked Read 3 Done Done Sent false
3357 1824 Acked Write 2 Done Done Sent false
3358 1825 Acked Flush 0 Done Done Sent false
3359 1826 Acked Flush 0 Done Done Sent false
3360 1827 Acked Read 1 Done Done Sent false
3361 1828 Acked Read 10 Done Done Sent false
3362 1829 Acked Write 9 Done Done Sent false
3363 1830 Acked Flush 0 Done Done Sent false
3364 1831 Acked Read 1 Done Done Sent false
3365 1832 Acked Read 5 Done Done Sent false
3366 1833 Acked Write 5 Done Done Sent false
3367 1834 Acked Write 8 Done Done Sent false
3368 1835 Acked Write 2 Done Done Sent false
3369 1836 Acked Read 3 Done Done Sent false
3370 1837 Acked Write 7 Done Done Sent false
3371 1838 Acked Write 2 Done Done Sent false
3372 1839 Acked Read 8 Done Done Sent false
3373 1840 Acked Read 10 Done Done Sent false
3374 1841 Acked Read 1 Done Done Sent false
3375 1842 Acked Write 5 Done Done Sent false
3376 1843 Acked Read 5 Done Done Sent false
3377 1844 Acked Read 6 Done Done Sent false
3378 1845 Acked Flush 0 Done Done Sent false
3379 1846 Acked Write 5 Done Done Sent false
3380 1847 Acked Write 8 Done Done Sent false
3381 1848 Acked Flush 0 Done Done Sent false
3382 1849 Acked Read 9 Done Done Sent false
3383 1850 Acked Write 1 Done Done Sent false
3384 1851 Acked Write 2 Done Done Sent false
3385 1852 Acked Read 5 Done Done Sent false
3386 1853 Acked Write 9 Done Done Sent false
3387 1854 Acked Write 8 Done Done Sent false
3388 1855 Acked Read 1 Done Done Sent false
3389 1856 Acked Write 4 Done Done Sent false
3390 1857 Acked Write 2 Done Done Sent false
3391 1858 Acked Read 7 Done Done Sent false
3392 1859 Acked Read 6 Done Done Sent false
3393 1860 Acked Write 7 Done Done Sent false
3394 1861 Acked Write 3 Done Done Sent false
3395 1862 Acked Write 9 Done Done Sent false
3396 1863 Acked Read 3 Done Done Sent false
3397 1864 Acked Write 9 Done Done Sent false
3398 1865 Acked Read 2 Done Done Sent false
3399 1866 Acked Write 7 Done Done Sent false
3400 1867 Acked Write 8 Done Done Sent false
3401 1868 Acked Flush 0 Done Done Sent false
3402 1869 Acked Write 1 Done Done Sent false
3403 1870 Acked Read 3 Done Done Sent false
3404 1871 Acked Read 6 Done Done Sent false
3405 1872 Acked Read 4 Done Done Sent false
3406 1873 Acked Write 1 Done Done Sent false
3407 1874 Acked Write 8 Done Done Sent false
3408 1875 Acked Write 6 Done Done Sent false
3409 1876 Acked Write 2 Done Done Sent false
3410 1877 Acked Write 10 Done Done Sent false
3411 1878 Acked Write 6 Done Done Sent false
3412 1879 Acked Read 9 Done Done Sent false
3413 1880 Acked Read 9 Done Done Sent false
3414 1881 Acked Read 4 Done Done Sent false
3415 1882 Acked Write 3 Done Done Sent false
3416 1883 Acked Read 6 Done Done Sent false
3417 1884 Acked Read 7 Done Done Sent false
3418 1885 Acked Write 5 Done Done Sent false
3419 1886 Acked Read 7 Done Done Sent false
3420 1887 Acked Flush 0 Done Done Sent false
3421 1888 Acked Write 3 Done Done Sent false
3422 1889 Acked Write 7 Done Done Sent false
3423 1890 Acked Write 1 Done Done Sent false
3424 1891 Acked Read 7 Done Done Sent false
3425 1892 Acked Write 5 Done Done Sent false
3426 1893 Acked Read 2 Done Done Sent false
3427 1894 Acked Write 7 Done Done Sent false
3428 1895 Acked Write 4 Done Done Sent false
3429 1896 Acked Write 5 Done Done Sent false
3430 1897 Acked Write 5 Done Done Sent false
3431 1898 Acked Flush 0 Done Done Sent false
3432 1899 Acked Write 1 Done Done Sent false
3433 1900 Acked Read 3 Done Done Sent false
3434 1901 Acked Read 6 Done Done Sent false
3435 1902 Acked Write 4 Done Done Sent false
3436 1903 Acked Flush 0 Done Done Sent false
3437 1904 Acked Read 5 Done Done Sent false
3438 1905 Acked Flush 0 Done Done Sent false
3439 STATES DS:0 DS:1 DS:2 TOTAL
3440 Sent 0 0 302 302
3441 Done 302 302 0 604
3442 Skipped 0 0 0 0
3443 Error 0 0 0 0
3444 Last Flush: 1905 1905 1603
3445 Downstairs last five completed: 1603 1602 1601 1600 1599
3446 Upstairs last five completed: 1905 1904 1903 1902 1901
3447 CLIENT: Up:0 ds:302 act:2
3448 [8830] got start action
3449 [8830] I am going to restart
3450 Starting downstairs at port 8830
3451 Make output file at "/tmp/dsc/downstairs-8830.txt"
3452 [8830][2] reports Starting
3453 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1203)
3454 [8830][2] reports Running
3455 ----------------------------------------------------------------
3456 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3457 States: Active Active Offline
3458 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3459 1604 Acked Read 8 Done Done Sent false
3460 1605 Acked Read 8 Done Done Sent false
3461 1606 Acked Read 4 Done Done Sent false
3462 1607 Acked Write 4 Done Done Sent false
3463 1608 Acked Read 6 Done Done Sent false
3464 1609 Acked Read 9 Done Done Sent false
3465 1610 Acked Read 10 Done Done Sent false
3466 1611 Acked Read 1 Done Done Sent false
3467 1612 Acked Write 7 Done Done Sent false
3468 1613 Acked Flush 0 Done Done Sent false
3469 1614 Acked Read 2 Done Done Sent false
3470 1615 Acked Read 2 Done Done Sent false
3471 1616 Acked Write 5 Done Done Sent false
3472 1617 Acked Read 5 Done Done Sent false
3473 1618 Acked Read 10 Done Done Sent false
3474 1619 Acked Read 8 Done Done Sent false
3475 1620 Acked Read 3 Done Done Sent false
3476 1621 Acked Read 5 Done Done Sent false
3477 1622 Acked Write 1 Done Done Sent false
3478 1623 Acked Read 7 Done Done Sent false
3479 1624 Acked Read 1 Done Done Sent false
3480 1625 Acked Read 6 Done Done Sent false
3481 1626 Acked Read 8 Done Done Sent false
3482 1627 Acked Write 9 Done Done Sent false
3483 1628 Acked Write 4 Done Done Sent false
3484 1629 Acked Flush 0 Done Done Sent false
3485 1630 Acked Read 5 Done Done Sent false
3486 1631 Acked Write 10 Done Done Sent false
3487 1632 Acked Write 5 Done Done Sent false
3488 1633 Acked Read 9 Done Done Sent false
3489 1634 Acked Read 4 Done Done Sent false
3490 1635 Acked Read 4 Done Done Sent false
3491 1636 Acked Write 7 Done Done Sent false
3492 1637 Acked Write 3 Done Done Sent false
3493 1638 Acked Flush 0 Done Done Sent false
3494 1639 Acked Write 5 Done Done Sent false
3495 1640 Acked Read 1 Done Done Sent false
3496 1641 Acked Read 4 Done Done Sent false
3497 1642 Acked Read 2 Done Done Sent false
3498 1643 Acked Write 9 Done Done Sent false
3499 1644 Acked Read 5 Done Done Sent false
3500 1645 Acked Read 1 Done Done Sent false
3501 1646 Acked Read 3 Done Done Sent false
3502 1647 Acked Read 3 Done Done Sent false
3503 1648 Acked Read 7 Done Done Sent false
3504 1649 Acked Write 4 Done Done Sent false
3505 1650 Acked Write 3 Done Done Sent false
3506 1651 Acked Read 1 Done Done Sent false
3507 1652 Acked Read 5 Done Done Sent false
3508 1653 Acked Write 10 Done Done Sent false
3509 1654 Acked Read 9 Done Done Sent false
3510 1655 Acked Read 3 Done Done Sent false
3511 1656 Acked Read 4 Done Done Sent false
3512 1657 Acked Read 9 Done Done Sent false
3513 1658 Acked Read 7 Done Done Sent false
3514 1659 Acked Write 7 Done Done Sent false
3515 1660 Acked Write 4 Done Done Sent false
3516 1661 Acked Write 9 Done Done Sent false
3517 1662 Acked Write 6 Done Done Sent false
3518 1663 Acked Read 9 Done Done Sent false
3519 1664 Acked Flush 0 Done Done Sent false
3520 1665 Acked Read 5 Done Done Sent false
3521 1666 Acked Read 10 Done Done Sent false
3522 1667 Acked Write 4 Done Done Sent false
3523 1668 Acked Read 10 Done Done Sent false
3524 1669 Acked Read 5 Done Done Sent false
3525 1670 Acked Write 10 Done Done Sent false
3526 1671 Acked Read 1 Done Done Sent false
3527 1672 Acked Read 6 Done Done Sent false
3528 1673 Acked Read 1 Done Done Sent false
3529 1674 Acked Flush 0 Done Done Sent false
3530 1675 Acked Flush 0 Done Done Sent false
3531 1676 Acked Write 3 Done Done Sent false
3532 1677 Acked Read 9 Done Done Sent false
3533 1678 Acked Flush 0 Done Done Sent false
3534 1679 Acked Read 7 Done Done Sent false
3535 1680 Acked Write 10 Done Done Sent false
3536 1681 Acked Write 8 Done Done Sent false
3537 1682 Acked Read 4 Done Done Sent false
3538 1683 Acked Read 10 Done Done Sent false
3539 1684 Acked Read 4 Done Done Sent false
3540 1685 Acked Write 10 Done Done Sent false
3541 1686 Acked Write 2 Done Done Sent false
3542 1687 Acked Read 9 Done Done Sent false
3543 1688 Acked Read 9 Done Done Sent false
3544 1689 Acked Read 4 Done Done Sent false
3545 1690 Acked Write 4 Done Done Sent false
3546 1691 Acked Write 7 Done Done Sent false
3547 1692 Acked Flush 0 Done Done Sent false
3548 1693 Acked Flush 0 Done Done Sent false
3549 1694 Acked Read 2 Done Done Sent false
3550 1695 Acked Write 3 Done Done Sent false
3551 1696 Acked Flush 0 Done Done Sent false
3552 1697 Acked Write 1 Done Done Sent false
3553 1698 Acked Read 8 Done Done Sent false
3554 1699 Acked Write 9 Done Done Sent false
3555 1700 Acked Write 10 Done Done Sent false
3556 1701 Acked Flush 0 Done Done Sent false
3557 1702 Acked Write 5 Done Done Sent false
3558 1703 Acked Read 5 Done Done Sent false
3559 1704 Acked Write 6 Done Done Sent false
3560 1705 Acked Read 2 Done Done Sent false
3561 1706 Acked Write 2 Done Done Sent false
3562 1707 Acked Read 2 Done Done Sent false
3563 1708 Acked Write 5 Done Done Sent false
3564 1709 Acked Read 5 Done Done Sent false
3565 1710 Acked Write 9 Done Done Sent false
3566 1711 Acked Write 7 Done Done Sent false
3567 1712 Acked Read 6 Done Done Sent false
3568 1713 Acked Read 4 Done Done Sent false
3569 1714 Acked Write 10 Done Done Sent false
3570 1715 Acked Flush 0 Done Done Sent false
3571 1716 Acked Read 2 Done Done Sent false
3572 1717 Acked Write 10 Done Done Sent false
3573 1718 Acked Write 3 Done Done Sent false
3574 1719 Acked Read 4 Done Done Sent false
3575 1720 Acked Write 3 Done Done Sent false
3576 1721 Acked Write 1 Done Done Sent false
3577 1722 Acked Read 2 Done Done Sent false
3578 1723 Acked Flush 0 Done Done Sent false
3579 1724 Acked Write 10 Done Done Sent false
3580 1725 Acked Flush 0 Done Done Sent false
3581 1726 Acked Read 3 Done Done Sent false
3582 1727 Acked Read 6 Done Done Sent false
3583 1728 Acked Write 5 Done Done Sent false
3584 1729 Acked Write 1 Done Done Sent false
3585 1730 Acked Read 8 Done Done Sent false
3586 1731 Acked Write 4 Done Done Sent false
3587 1732 Acked Read 9 Done Done Sent false
3588 1733 Acked Read 1 Done Done Sent false
3589 1734 Acked Write 10 Done Done Sent false
3590 1735 Acked Read 4 Done Done Sent false
3591 1736 Acked Write 8 Done Done Sent false
3592 1737 Acked Write 2 Done Done Sent false
3593 1738 Acked Flush 0 Done Done Sent false
3594 1739 Acked Flush 0 Done Done Sent false
3595 1740 Acked Write 1 Done Done Sent false
3596 1741 Acked Write 5 Done Done Sent false
3597 1742 Acked Write 2 Done Done Sent false
3598 1743 Acked Write 3 Done Done Sent false
3599 1744 Acked Write 8 Done Done Sent false
3600 1745 Acked Write 10 Done Done Sent false
3601 1746 Acked Write 6 Done Done Sent false
3602 1747 Acked Write 4 Done Done Sent false
3603 1748 Acked Read 2 Done Done Sent false
3604 1749 Acked Write 9 Done Done Sent false
3605 1750 Acked Write 1 Done Done Sent false
3606 1751 Acked Write 4 Done Done Sent false
3607 1752 Acked Flush 0 Done Done Sent false
3608 1753 Acked Read 1 Done Done Sent false
3609 1754 Acked Read 9 Done Done Sent false
3610 1755 Acked Read 1 Done Done Sent false
3611 1756 Acked Write 7 Done Done Sent false
3612 1757 Acked Write 2 Done Done Sent false
3613 1758 Acked Write 6 Done Done Sent false
3614 1759 Acked Write 8 Done Done Sent false
3615 1760 Acked Read 4 Done Done Sent false
3616 1761 Acked Flush 0 Done Done Sent false
3617 1762 Acked Write 7 Done Done Sent false
3618 1763 Acked Write 10 Done Done Sent false
3619 1764 Acked Write 9 Done Done Sent false
3620 1765 Acked Flush 0 Done Done Sent false
3621 1766 Acked Write 3 Done Done Sent false
3622 1767 Acked Write 5 Done Done Sent false
3623 1768 Acked Read 8 Done Done Sent false
3624 1769 Acked Read 1 Done Done Sent false
3625 1770 Acked Flush 0 Done Done Sent false
3626 1771 Acked Read 7 Done Done Sent false
3627 1772 Acked Read 7 Done Done Sent false
3628 1773 Acked Read 5 Done Done Sent false
3629 1774 Acked Write 4 Done Done Sent false
3630 1775 Acked Read 2 Done Done Sent false
3631 1776 Acked Read 2 Done Done Sent false
3632 1777 Acked Flush 0 Done Done Sent false
3633 1778 Acked Flush 0 Done Done Sent false
3634 1779 Acked Read 6 Done Done Sent false
3635 1780 Acked Write 5 Done Done Sent false
3636 1781 Acked Write 4 Done Done Sent false
3637 1782 Acked Read 6 Done Done Sent false
3638 1783 Acked Write 1 Done Done Sent false
3639 1784 Acked Write 3 Done Done Sent false
3640 1785 Acked Read 10 Done Done Sent false
3641 1786 Acked Write 2 Done Done Sent false
3642 1787 Acked Write 6 Done Done Sent false
3643 1788 Acked Read 3 Done Done Sent false
3644 1789 Acked Read 1 Done Done Sent false
3645 1790 Acked Flush 0 Done Done Sent false
3646 1791 Acked Write 1 Done Done Sent false
3647 1792 Acked Write 5 Done Done Sent false
3648 1793 Acked Write 8 Done Done Sent false
3649 1794 Acked Read 10 Done Done Sent false
3650 1795 Acked Read 2 Done Done Sent false
3651 1796 Acked Read 6 Done Done Sent false
3652 1797 Acked Write 8 Done Done Sent false
3653 1798 Acked Write 8 Done Done Sent false
3654 1799 Acked Read 2 Done Done Sent false
3655 1800 Acked Read 8 Done Done Sent false
3656 1801 Acked Read 4 Done Done Sent false
3657 1802 Acked Read 6 Done Done Sent false
3658 1803 Acked Read 2 Done Done Sent false
3659 1804 Acked Read 6 Done Done Sent false
3660 1805 Acked Read 5 Done Done Sent false
3661 1806 Acked Write 7 Done Done Sent false
3662 1807 Acked Write 5 Done Done Sent false
3663 1808 Acked Read 9 Done Done Sent false
3664 1809 Acked Write 8 Done Done Sent false
3665 1810 Acked Write 3 Done Done Sent false
3666 1811 Acked Read 9 Done Done Sent false
3667 1812 Acked Read 7 Done Done Sent false
3668 1813 Acked Write 7 Done Done Sent false
3669 1814 Acked Write 1 Done Done Sent false
3670 1815 Acked Write 5 Done Done Sent false
3671 1816 Acked Read 4 Done Done Sent false
3672 1817 Acked Write 7 Done Done Sent false
3673 1818 Acked Read 6 Done Done Sent false
3674 1819 Acked Write 1 Done Done Sent false
3675 1820 Acked Write 6 Done Done Sent false
3676 1821 Acked Read 8 Done Done Sent false
3677 1822 Acked Write 7 Done Done Sent false
3678 1823 Acked Read 3 Done Done Sent false
3679 1824 Acked Write 2 Done Done Sent false
3680 1825 Acked Flush 0 Done Done Sent false
3681 1826 Acked Flush 0 Done Done Sent false
3682 1827 Acked Read 1 Done Done Sent false
3683 1828 Acked Read 10 Done Done Sent false
3684 1829 Acked Write 9 Done Done Sent false
3685 1830 Acked Flush 0 Done Done Sent false
3686 1831 Acked Read 1 Done Done Sent false
3687 1832 Acked Read 5 Done Done Sent false
3688 1833 Acked Write 5 Done Done Sent false
3689 1834 Acked Write 8 Done Done Sent false
3690 1835 Acked Write 2 Done Done Sent false
3691 1836 Acked Read 3 Done Done Sent false
3692 1837 Acked Write 7 Done Done Sent false
3693 1838 Acked Write 2 Done Done Sent false
3694 1839 Acked Read 8 Done Done Sent false
3695 1840 Acked Read 10 Done Done Sent false
3696 1841 Acked Read 1 Done Done Sent false
3697 1842 Acked Write 5 Done Done Sent false
3698 1843 Acked Read 5 Done Done Sent false
3699 1844 Acked Read 6 Done Done Sent false
3700 1845 Acked Flush 0 Done Done Sent false
3701 1846 Acked Write 5 Done Done Sent false
3702 1847 Acked Write 8 Done Done Sent false
3703 1848 Acked Flush 0 Done Done Sent false
3704 1849 Acked Read 9 Done Done Sent false
3705 1850 Acked Write 1 Done Done Sent false
3706 1851 Acked Write 2 Done Done Sent false
3707 1852 Acked Read 5 Done Done Sent false
3708 1853 Acked Write 9 Done Done Sent false
3709 1854 Acked Write 8 Done Done Sent false
3710 1855 Acked Read 1 Done Done Sent false
3711 1856 Acked Write 4 Done Done Sent false
3712 1857 Acked Write 2 Done Done Sent false
3713 1858 Acked Read 7 Done Done Sent false
3714 1859 Acked Read 6 Done Done Sent false
3715 1860 Acked Write 7 Done Done Sent false
3716 1861 Acked Write 3 Done Done Sent false
3717 1862 Acked Write 9 Done Done Sent false
3718 1863 Acked Read 3 Done Done Sent false
3719 1864 Acked Write 9 Done Done Sent false
3720 1865 Acked Read 2 Done Done Sent false
3721 1866 Acked Write 7 Done Done Sent false
3722 1867 Acked Write 8 Done Done Sent false
3723 1868 Acked Flush 0 Done Done Sent false
3724 1869 Acked Write 1 Done Done Sent false
3725 1870 Acked Read 3 Done Done Sent false
3726 1871 Acked Read 6 Done Done Sent false
3727 1872 Acked Read 4 Done Done Sent false
3728 1873 Acked Write 1 Done Done Sent false
3729 1874 Acked Write 8 Done Done Sent false
3730 1875 Acked Write 6 Done Done Sent false
3731 1876 Acked Write 2 Done Done Sent false
3732 1877 Acked Write 10 Done Done Sent false
3733 1878 Acked Write 6 Done Done Sent false
3734 1879 Acked Read 9 Done Done Sent false
3735 1880 Acked Read 9 Done Done Sent false
3736 1881 Acked Read 4 Done Done Sent false
3737 1882 Acked Write 3 Done Done Sent false
3738 1883 Acked Read 6 Done Done Sent false
3739 1884 Acked Read 7 Done Done Sent false
3740 1885 Acked Write 5 Done Done Sent false
3741 1886 Acked Read 7 Done Done Sent false
3742 1887 Acked Flush 0 Done Done Sent false
3743 1888 Acked Write 3 Done Done Sent false
3744 1889 Acked Write 7 Done Done Sent false
3745 1890 Acked Write 1 Done Done Sent false
3746 1891 Acked Read 7 Done Done Sent false
3747 1892 Acked Write 5 Done Done Sent false
3748 1893 Acked Read 2 Done Done Sent false
3749 1894 Acked Write 7 Done Done Sent false
3750 1895 Acked Write 4 Done Done Sent false
3751 1896 Acked Write 5 Done Done Sent false
3752 1897 Acked Write 5 Done Done Sent false
3753 1898 Acked Flush 0 Done Done Sent false
3754 1899 Acked Write 1 Done Done Sent false
3755 1900 Acked Read 3 Done Done Sent false
3756 1901 Acked Read 6 Done Done Sent false
3757 1902 Acked Write 4 Done Done Sent false
3758 1903 Acked Flush 0 Done Done Sent false
3759 1904 Acked Read 5 Done Done Sent false
3760 1905 Acked Flush 0 Done Done Sent false
3761 STATES DS:0 DS:1 DS:2 TOTAL
3762 Sent 0 0 302 302
3763 Done 302 302 0 604
3764 Skipped 0 0 0 0
3765 Error 0 0 0 0
3766 Last Flush: 1905 1905 1603
3767 Downstairs last five completed: 1603 1602 1601 1600 1599
3768 Upstairs last five completed: 1905 1904 1903 1902 1901
3769 CLIENT: Up:0 ds:302 act:2
37702024-12-11T23:41:19.035ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37712024-12-11T23:41:19.035ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49705) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37722024-12-11T23:41:19.036ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37732024-12-11T23:41:19.036ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37742024-12-11T23:41:19.036ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37752024-12-11T23:41:19.036ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37762024-12-11T23:41:19.037ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37772024-12-11T23:41:19.037ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37782024-12-11T23:41:19.037ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37792024-12-11T23:41:19.037ZINFOcrucible: [2] client re-new 302 jobs since flush 1603 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37802024-12-11T23:41:19.037ZINFOcrucible: [2] Replayed 302 jobs since flush: 1603 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
3781 ----------------------------------------------------------------
3782 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3783 Downstairs last five completed: 1905 1904 1903 1902 1901
3784 Upstairs last five completed: 1905 1904 1903 1902 1901
3785 CLIENT: Up:0 ds:0 act:3
3786 Replay: All jobs finished
3787 Main task has work to do, go find it
3788 got dsc Stop(0)
3789 stop 0
3790 Dec 11 23:41:19.267 INFO request completed, latency_us: 356, response_code: 204, uri: /stop/cid/0, method: POST, req_id: c4584d35-5386-4355-b127-fd12df42923d, remote_addr: 127.0.0.1:62969, local_addr[8810] Got stop action so:false kr:false
3791 : 127.0.0.1:9998
37922024-12-11T23:41:19.268ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37932024-12-11T23:41:19.268ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37942024-12-11T23:41:19.268ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37952024-12-11T23:41:19.268ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37962024-12-11T23:41:19.268ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
3797 Dec 11 23:41:19.268 INFO request completed, latency_us: 176, response_code: 200, uri: /state/cid/0, method: GET, req_id: b27c42e3-501e-406b-afbb-0a10ecf8d8a2, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
37982024-12-11T23:41:19.268ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
3799 [8810] Exited with: ExitStatus(unix_wait_status(9))
3800 [8810][0] reports Exit
3801 Dec 11 23:41:23.270 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/0, method: GET, req_id: 91f523e4-b904-488c-b5b9-26d42431500e, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
3802 001/300 Write block 2698 len 28672 data: 2 2 2 2 2 2 2
3803 002/300 Read block 1676 len 28672
3804 003/300 Read block 4979 len 20480
3805 004/300 Write block 4103 len 24576 data: 2 2 2 2 2 2
3806 005/300 Read block 2438 len 4096
3807 006/300 Write block 3870 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3808 007/300 Read block 2498 len 36864
3809 008/300 Read block 4845 len 36864
3810 009/300 Write block 3086 len 36864 data: 3 3 3 4 3 3 3 3 3
3811 010/300 Read block 2166 len 24576
3812 011/300 Read block 1004 len 36864
3813 012/300 Read block 1280 len 8192
3814 013/300 Read block 481 len 28672
3815 014/300 Read block 4255 len 28672
3816 015/300 Read block 2830 len 28672
3817 016/300 Read block 2784 len 24576
3818 017/300 Write block 148 len 40960 data: 4 4 3 3 3 3 3 3 2 2
3819 018/300 Write block 134 len 8192 data: 2 2
3820 019/300 Read block 3327 len 20480
3821 020/300 Write block 2733 len 20480 data: 3 4 4 4 4
3822 021/300 Read block 4066 len 12288
3823 022/300 Read block 3968 len 12288
3824 023/300 Write block 2543 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3825 024/300 Write block 4993 len 4096 data: 2
3826 025/300 Write block 3006 len 16384 data: 2 2 2 2
3827 026/300 Write block 2329 len 28672 data: 2 2 2 3 3 3 3
3828 027/300 Read block 3789 len 16384
3829 028/300 Write block 1980 len 40960 data: 3 3 3 3 4 4 2 2 2 2
3830 029/300 Write block 4018 len 4096 data: 3
3831 030/300 Write block 3267 len 4096 data: 2
3832 031/300 Read block 3895 len 32768
3833 032/300 Read block 2941 len 20480
3834 033/300 Write block 154 len 24576 data: 4 4 3 3 3 3
3835 034/300 Read block 3654 len 20480
3836 035/300 Read block 4675 len 40960
3837 036/300 Write block 1922 len 16384 data: 2 3 3 3
3838 037/300 Flush
3839 038/300 Write block 3669 len 4096 data: 2
3840 039/300 Write block 3528 len 12288 data: 2 2 2
3841 040/300 Write block 4937 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3842 041/300 Write block 4380 len 4096 data: 3
3843 042/300 Read block 1471 len 12288
3844 043/300 Read block 687 len 20480
3845 044/300 Read block 607 len 40960
3846 045/300 Read block 2117 len 20480
3847 046/300 Read block 4326 len 36864
3848 047/300 Read block 2326 len 8192
3849 048/300 Flush
3850 049/300 Write block 3986 len 28672 data: 2 2 2 2 2 2 2
3851 050/300 Write block 1341 len 28672 data: 2 2 2 2 2 2 2
3852 051/300 Read block 2843 len 24576
3853 052/300 Read block 550 len 20480
3854 053/300 Write block 2024 len 28672 data: 2 2 2 2 2 2 3
3855 054/300 Read block 2636 len 4096
3856 055/300 Read block 996 len 16384
3857 056/300 Read block 3416 len 32768
3858 057/300 Write block 1941 len 24576 data: 2 2 2 2 2 2
3859 058/300 Write block 3522 len 32768 data: 2 2 2 2 2 2 3 3
3860 059/300 Read block 218 len 20480
3861 060/300 Read block 1897 len 4096
3862 061/300 Read block 2240 len 4096
3863 062/300 Read block 507 len 16384
3864 063/300 Read block 3410 len 40960
3865 064/300 Flush
3866 065/300 Write block 1821 len 4096 data: 2
3867 066/300 Read block 1465 len 8192
3868 067/300 Flush
3869 068/300 Read block 2098 len 32768
3870 069/300 Read block 4086 len 8192
3871 070/300 Read block 4525 len 16384
3872 071/300 Flush
3873 072/300 Write block 3831 len 28672 data: 4 3 3 3 3 2 2
3874 073/300 Read block 1085 len 20480
3875 074/300 Write block 3799 len 20480 data: 3 3 3 3 3
3876 075/300 Read block 1667 len 12288
3877 076/300 Write block 1490 len 36864 data: 3 3 3 3 2 2 2 2 2
3878 077/300 Read block 692 len 28672
3879 078/300 Write block 178 len 16384 data: 2 2 2 2
3880 079/300 Read block 1418 len 32768
3881 080/300 Write block 518 len 32768 data: 3 3 3 3 3 2 2 2
3882 081/300 Write block 3478 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3883 082/300 Read block 2385 len 36864
3884 083/300 Read block 2603 len 36864
3885 084/300 Write block 2309 len 28672 data: 3 3 3 2 2 2 2
3886 085/300 Write block 261 len 24576 data: 2 2 2 2 2 2
3887 086/300 Read block 4414 len 8192
3888 087/300 Write block 2796 len 36864 data: 2 2 2 2 2 2 3 3 3
3889 088/300 Read block 360 len 28672
3890 089/300 Write block 4956 len 28672 data: 2 2 2 2 2 2 2
3891 090/300 Read block 2791 len 12288
3892 091/300 Read block 1395 len 16384
3893 092/300 Read block 1325 len 36864
3894 093/300 Read block 138 len 16384
3895 094/300 Write block 3889 len 16384 data: 2 2 2 3
3896 095/300 Flush
3897 096/300 Read block 4054 len 4096
3898 097/300 Write block 4843 len 16384 data: 2 2 2 2
3899 098/300 Read block 2476 len 20480
3900 099/300 Read block 1250 len 32768
3901 100/300 Read block 2411 len 12288
3902 101/300 Read block 2666 len 4096
3903 102/300 Read block 3966 len 32768
3904 103/300 Read block 932 len 16384
3905 104/300 Read block 1640 len 16384
3906 105/300 Read block 5 len 8192
3907 106/300 Write block 1361 len 20480 data: 2 2 2 2 2
3908 107/300 Write block 3863 len 24576 data: 4 4 4 3 3 3
3909 108/300 Write block 2828 len 40960 data: 3 3 3 3 3 3 3 3 2 2
3910 109/300 Read block 2206 len 36864
3911 110/300 Read block 1822 len 40960
3912 111/300 Read block 2462 len 24576
3913 112/300 Write block 1641 len 36864 data: 3 3 3 3 3 3 2 2 2
3914 113/300 Read block 3597 len 32768
3915 114/300 Write block 2810 len 24576 data: 2 2 2 2 2 3
3916 115/300 Read block 1942 len 12288
3917 116/300 Read block 4346 len 40960
3918 117/300 Write block 3867 len 28672 data: 4 4 3 3 3 3 3
3919 118/300 Read block 2666 len 12288
3920 119/300 Write block 854 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3921 120/300 Read block 4231 len 32768
3922 121/300 Read block 2870 len 32768
3923 122/300 Read block 354 len 40960
3924 123/300 Write block 2187 len 40960 data: 3 3 3 2 2 3 3 4 4 4
3925 124/300 Write block 155 len 8192 data: 5 4
3926 125/300 Write block 3709 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3927 126/300 Read block 3742 len 8192
3928 127/300 Read block 2357 len 36864
3929 128/300 Write block 3009 len 24576 data: 3 2 2 2 2 2
3930 129/300 Read block 543 len 28672
3931 130/300 Read block 157 len 12288
3932 131/300 Read block 1439 len 24576
3933 132/300 Write block 4318 len 32768 data: 2 2 2 2 2 2 2 2
3934 133/300 Read block 2677 len 40960
3935 134/300 Write block 1678 len 28672 data: 2 2 2 2 2 2 2
3936 135/300 Write block 138 len 28672 data: 2 2 2 2 2 2 2
3937 136/300 Read block 3989 len 8192
3938 137/300 Read block 476 len 12288
3939 138/300 Write block 440 len 12288 data: 2 2 2
3940 139/300 Read block 1299 len 28672
3941 140/300 Read block 2841 len 20480
3942 141/300 Read block 3844 len 24576
3943 142/300 Read block 3229 len 12288
3944 143/300 Write block 2498 len 32768 data: 3 3 3 3 2 2 2 2
3945 144/300 Write block 2551 len 40960 data: 3 3 2 2 2 2 2 2 2 2
3946 145/300 Read block 3539 len 4096
3947 146/300 Read block 2714 len 20480
3948 147/300 Read block 3809 len 16384
3949 148/300 Write block 4202 len 28672 data: 3 3 3 3 2 2 2
3950 149/300 Read block 4853 len 28672
3951 150/300 Read block 4426 len 12288
3952 151/300 Write block 1009 len 12288 data: 2 2 2
3953 152/300 Write block 2933 len 36864 data: 3 3 3 3 3 3 2 2 2
3954 153/300 Read block 1264 len 24576
3955 154/300 Read block 3543 len 40960
3956 155/300 Read block 4457 len 32768
3957 156/300 Write block 4538 len 24576 data: 2 2 2 2 2 2
3958 157/300 Flush
3959 158/300 Write block 4677 len 16384 data: 2 2 2 2
3960 159/300 Write block 3619 len 20480 data: 4 4 3 2 2
3961 160/300 Read block 1231 len 4096
3962 161/300 Read block 3857 len 32768
3963 162/300 Write block 979 len 32768 data: 3 2 2 2 2 2 2 2
3964 163/300 Flush
3965 164/300 Read block 2753 len 12288
3966 165/300 Read block 1416 len 28672
3967 166/300 Read block 557 len 24576
3968 167/300 Write block 3243 len 20480 data: 2 2 2 2 2
3969 168/300 Write block 115 len 8192 data: 3 3
3970 169/300 Flush
3971 170/300 Read block 3934 len 36864
3972 171/300 Read block 2707 len 4096
3973 172/300 Flush
3974 173/300 Read block 19 len 4096
3975 174/300 Read block 2818 len 4096
3976 175/300 Read block 1791 len 8192
3977 176/300 Read block 2756 len 24576
3978 177/300 Write block 3822 len 36864 data: 2 2 2 2 3 3 4 4 4
3979 178/300 Flush
3980 179/300 Write block 129 len 20480 data: 2 2 2 2 2
3981 180/300 Write block 3345 len 4096 data: 2
3982 181/300 Write block 287 len 8192 data: 2 2
3983 182/300 Write block 2652 len 28672 data: 3 2 2 3 3 3 3
3984 183/300 Write block 2689 len 20480 data: 3 3 3 3 3
3985 184/300 Write block 2926 len 28672 data: 2 2 2 2 3 3 3
3986 185/300 Write block 2437 len 40960 data: 2 2 3 3 3 3 3 2 2 2
3987 186/300 Write block 4000 len 28672 data: 2 2 3 2 2 2 2
3988 187/300 Write block 2147 len 12288 data: 2 2 2
3989 188/300 Read block 4314 len 24576
3990 189/300 Read block 4965 len 8192
3991 190/300 Write block 4582 len 28672 data: 2 2 2 3 3 3 3
3992 191/300 Read block 4070 len 4096
3993 192/300 Write block 4034 len 32768 data: 2 2 2 2 2 2 2 2
3994 193/300 Write block 2271 len 20480 data: 5 4 4 2 2
3995 194/300 Write block 2422 len 28672 data: 2 3 3 3 2 2 2
3996 195/300 Read block 1842 len 4096
3997 196/300 Write block 3282 len 12288 data: 3 2 2
3998 197/300 Read block 4629 len 28672
3999 198/300 Write block 2798 len 28672 data: 3 3 3 3 4 4 4
4000 199/300 Read block 3975 len 20480
4001 200/300 Write block 4862 len 4096 data: 3
4002 201/300 Read block 3684 len 24576
4003 202/300 Read block 3443 len 36864
4004 203/300 Write block 1981 len 28672 data: 4 4 4 5 5 3 3
4005 204/300 Flush
4006 205/300 Read block 3405 len 16384
4007 206/300 Read block 1223 len 16384
4008 207/300 Read block 2580 len 8192
4009 208/300 Write block 391 len 24576 data: 3 3 2 2 2 2
4010 209/300 Write block 877 len 36864 data: 3 3 3 3 3 3 2 2 2
4011 210/300 Read block 1339 len 24576
4012 211/300 Write block 4111 len 36864 data: 2 2 2 2 2 2 2 2 2
4013 212/300 Write block 868 len 40960 data: 2 2 2 2 2 3 3 3 3 4
4014 213/300 Write block 3290 len 16384 data: 2 2 2 2
4015 214/300 Write block 1169 len 4096 data: 2
4016 215/300 Write block 3243 len 8192 data: 3 3
4017 216/300 Read block 4456 len 40960
4018 217/300 Write block 75 len 16384 data: 2 2 3 3
4019 218/300 Read block 4789 len 12288
4020 219/300 Write block 1439 len 32768 data: 4 5 5 4 4 4 4 4
4021 220/300 Read block 860 len 4096
4022 221/300 Read block 247 len 24576
4023 222/300 Read block 2949 len 24576
4024 223/300 Read block 253 len 24576
4025 224/300 Read block 3689 len 16384
4026 225/300 Read block 2264 len 40960
4027 226/300 Write block 1493 len 4096 data: 4
4028 227/300 Write block 3279 len 12288 data: 3 3 3
4029 228/300 Flush
4030 229/300 Write block 1575 len 12288 data: 3 3 2
4031 230/300 Read block 909 len 28672
4032 231/300 Write block 1926 len 36864 data: 4 4 3 4 4 4 4 2 2
4033 232/300 Write block 3085 len 24576 data: 3 4 4 4 5 4
4034 233/300 Write block 2721 len 20480 data: 3 3 3 4 3
4035 234/300 Read block 4315 len 24576
4036 235/300 Flush
4037 236/300 Flush
4038 237/300 Write block 1705 len 4096 data: 2
4039 238/300 Write block 1562 len 16384 data: 2 2 2 2
4040 239/300 Write block 3858 len 28672 data: 4 4 4 4 5 5 5
4041 240/300 Read block 3945 len 24576
4042 241/300 Write block 2504 len 20480 data: 3 3 2 2 3
4043 242/300 Write block 177 len 28672 data: 2 3 3 3 3 2 2
4044 243/300 Read block 1086 len 16384
4045 244/300 Read block 1561 len 24576
4046 245/300 Read block 1305 len 40960
4047 246/300 Write block 4821 len 4096 data: 2
4048 247/300 Write block 2172 len 20480 data: 2 2 2 2 2
4049 248/300 Write block 4207 len 32768 data: 3 3 2 2 2 2 2 2
4050 249/300 Read block 607 len 8192
4051 250/300 Read block 4806 len 36864
4052 251/300 Write block 1063 len 12288 data: 2 2 2
4053 252/300 Read block 3872 len 36864
4054 253/300 Read block 4066 len 40960
4055 254/300 Write block 4198 len 16384 data: 2 3 3 3
4056 255/300 Read block 482 len 4096
4057 256/300 Write block 3781 len 20480 data: 2 2 2 2 2
4058 257/300 Read block 3154 len 40960
4059 258/300 Read block 4047 len 28672
4060 259/300 Read block 1893 len 24576
4061 260/300 Read block 4209 len 40960
4062 261/300 Read block 4785 len 12288
4063 262/300 Flush
4064 263/300 Write block 4866 len 24576 data: 2 2 2 2 2 3
4065 264/300 Read block 3159 len 4096
4066 265/300 Write block 3960 len 32768 data: 3 3 3 3 3 3 2 2
4067 266/300 Write block 1906 len 36864 data: 2 2 3 3 3 3 3 3 2
4068 267/300 Read block 3781 len 4096
4069 268/300 Write block 1311 len 12288 data: 2 2 2
4070 269/300 Write block 398 len 24576 data: 2 2 2 2 2 2
4071 270/300 Read block 2460 len 36864
4072 271/300 Read block 4380 len 16384
4073 272/300 Read block 4052 len 4096
4074 273/300 Write block 2795 len 32768 data: 2 3 3 4 4 4 4 5
4075 274/300 Read block 145 len 24576
4076 275/300 Write block 492 len 24576 data: 2 2 2 2 2 2
4077 276/300 Read block 4638 len 4096
4078 277/300 Write block 4199 len 8192 data: 4 4
4079 278/300 Read block 1517 len 8192
4080 279/300 Write block 1693 len 24576 data: 2 2 2 2 2 2
4081 280/300 Read block 2637 len 20480
4082 281/300 Write block 2763 len 28672 data: 2 2 2 2 2 2 2
4083 282/300 Read block 4661 len 20480
4084 283/300 Write block 3638 len 36864 data: 2 2 3 3 3 3 2 2 2
4085 284/300 Read block 819 len 16384
4086 285/300 Read block 1244 len 4096
4087 286/300 Read block 3063 len 24576
4088 287/300 Read block 2179 len 32768
4089 288/300 Write block 3182 len 8192 data: 2 2
4090 289/300 Flush
4091 290/300 Write block 509 len 4096 data: 2
4092 291/300 Read block 2266 len 8192
4093 292/300 Flush
4094 293/300 Read block 270 len 12288
4095 294/300 Read block 2221 len 32768
4096 295/300 Read block 2781 len 4096
4097 296/300 Read block 3548 len 16384
4098 297/300 Read block 3134 len 8192
4099 298/300 Write block 196 len 24576 data: 2 2 2 2 2 2
4100 299/300 Read block 4439 len 12288
4101 300/300 Read block 3483 len 40960
4102 301/300 Read block 2870 len 20480
4103 Main task has work to do, go find it
4104 got dsc Start(0)
4105 start 0
4106 Dec 11 23:41:23.513 INFO request completed, latency_us: 234, response_code: 204, uri: /start/cid/0, method: POST, req_id: 67d3bb43-ca16-4036-8608-387255707e59, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
4107 Replay: started 0, returned:Ok(())
4108 ----------------------------------------------------------------
4109 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4110 States: Offline Active Active
4111 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4112 1906 Acked Write 7 Sent Done Done false
4113 1907 Acked Read 7 Sent Done Done false
4114 1908 Acked Read 5 Sent Done Done false
4115 1909 Acked Write 6 Sent Done Done false
4116 1910 Acked Read 1 Sent Done Done false
4117 1911 Acked Write 10 Sent Done Done false
4118 1912 Acked Read 9 Sent Done Done false
4119 1913 Acked Read 9 Sent Done Done false
4120 1914 Acked Write 9 Sent Done Done false
4121 1915 Acked Read 6 Sent Done Done false
4122 1916 Acked Read 9 Sent Done Done false
4123 1917 Acked Read 2 Sent Done Done false
4124 1918 Acked Read 7 Sent Done Done false
4125 1919 Acked Read 7 Sent Done Done false
4126 1920 Acked Read 7 Sent Done Done false
4127 1921 Acked Read 6 Sent Done Done false
4128 1922 Acked Write 10 Sent Done Done false
4129 1923 Acked Write 2 Sent Done Done false
4130 1924 Acked Read 5 Sent Done Done false
4131 1925 Acked Write 5 Sent Done Done false
4132 1926 Acked Read 3 Sent Done Done false
4133 1927 Acked Read 3 Sent Done Done false
4134 1928 Acked Write 10 Sent Done Done false
4135 1929 Acked Write 1 Sent Done Done false
4136 1930 Acked Write 4 Sent Done Done false
4137 1931 Acked Write 7 Sent Done Done false
4138 1932 Acked Read 4 Sent Done Done false
4139 1933 Acked Write 10 Sent Done Done false
4140 1934 Acked Write 1 Sent Done Done false
4141 1935 Acked Write 1 Sent Done Done false
4142 1936 Acked Read 8 Sent Done Done false
4143 1937 Acked Read 5 Sent Done Done false
4144 1938 Acked Write 6 Sent Done Done false
4145 1939 Acked Read 5 Sent Done Done false
4146 1940 Acked Read 10 Sent Done Done false
4147 1941 Acked Write 4 Sent Done Done false
4148 1942 Acked Flush 0 Sent Done Done false
4149 1943 Acked Write 1 Sent Done Done false
4150 1944 Acked Write 3 Sent Done Done false
4151 1945 Acked Write 10 Sent Done Done false
4152 1946 Acked Write 1 Sent Done Done false
4153 1947 Acked Read 3 Sent Done Done false
4154 1948 Acked Read 5 Sent Done Done false
4155 1949 Acked Read 10 Sent Done Done false
4156 1950 Acked Read 5 Sent Done Done false
4157 1951 Acked Read 9 Sent Done Done false
4158 1952 Acked Read 2 Sent Done Done false
4159 1953 Acked Flush 0 Sent Done Done false
4160 1954 Acked Write 7 Sent Done Done false
4161 1955 Acked Write 7 Sent Done Done false
4162 1956 Acked Read 6 Sent Done Done false
4163 1957 Acked Read 5 Sent Done Done false
4164 1958 Acked Write 7 Sent Done Done false
4165 1959 Acked Read 1 Sent Done Done false
4166 1960 Acked Read 4 Sent Done Done false
4167 1961 Acked Read 8 Sent Done Done false
4168 1962 Acked Write 6 Sent Done Done false
4169 1963 Acked Write 8 Sent Done Done false
4170 1964 Acked Read 5 Sent Done Done false
4171 1965 Acked Read 1 Sent Done Done false
4172 1966 Acked Read 1 Sent Done Done false
4173 1967 Acked Read 4 Sent Done Done false
4174 1968 Acked Read 10 Sent Done Done false
4175 1969 Acked Flush 0 Sent Done Done false
4176 1970 Acked Write 1 Sent Done Done false
4177 1971 Acked Read 2 Sent Done Done false
4178 1972 Acked Flush 0 Sent Done Done false
4179 1973 Acked Read 8 Sent Done Done false
4180 1974 Acked Read 2 Sent Done Done false
4181 1975 Acked Read 4 Sent Done Done false
4182 1976 Acked Flush 0 Sent Done Done false
4183 1977 Acked Write 7 Sent Done Done false
4184 1978 Acked Read 5 Sent Done Done false
4185 1979 Acked Write 5 Sent Done Done false
4186 1980 Acked Read 3 Sent Done Done false
4187 1981 Acked Write 9 Sent Done Done false
4188 1982 Acked Read 7 Sent Done Done false
4189 1983 Acked Write 4 Sent Done Done false
4190 1984 Acked Read 8 Sent Done Done false
4191 1985 Acked Write 8 Sent Done Done false
4192 1986 Acked Write 10 Sent Done Done false
4193 1987 Acked Read 9 Sent Done Done false
4194 1988 Acked Read 9 Sent Done Done false
4195 1989 Acked Write 7 Sent Done Done false
4196 1990 Acked Write 6 Sent Done Done false
4197 1991 Acked Read 2 Sent Done Done false
4198 1992 Acked Write 9 Sent Done Done false
4199 1993 Acked Read 7 Sent Done Done false
4200 1994 Acked Write 7 Sent Done Done false
4201 1995 Acked Read 3 Sent Done Done false
4202 1996 Acked Read 4 Sent Done Done false
4203 1997 Acked Read 9 Sent Done Done false
4204 1998 Acked Read 4 Sent Done Done false
4205 1999 Acked Write 4 Sent Done Done false
4206 2000 Acked Flush 0 Sent Done Done false
4207 2001 Acked Read 1 Sent Done Done false
4208 2002 Acked Write 4 Sent Done Done false
4209 2003 Acked Read 5 Sent Done Done false
4210 2004 Acked Read 8 Sent Done Done false
4211 2005 Acked Read 3 Sent Done Done false
4212 2006 Acked Read 1 Sent Done Done false
4213 2007 Acked Read 8 Sent Done Done false
4214 2008 Acked Read 4 Sent Done Done false
4215 2009 Acked Read 4 Sent Done Done false
4216 2010 Acked Read 2 Sent Done Done false
4217 2011 Acked Write 5 Sent Done Done false
4218 2012 Acked Write 6 Sent Done Done false
4219 2013 Acked Write 10 Sent Done Done false
4220 2014 Acked Read 9 Sent Done Done false
4221 2015 Acked Read 10 Sent Done Done false
4222 2016 Acked Read 6 Sent Done Done false
4223 2017 Acked Write 9 Sent Done Done false
4224 2018 Acked Read 8 Sent Done Done false
4225 2019 Acked Write 6 Sent Done Done false
4226 2020 Acked Read 3 Sent Done Done false
4227 2021 Acked Read 10 Sent Done Done false
4228 2022 Acked Write 7 Sent Done Done false
4229 2023 Acked Read 3 Sent Done Done false
4230 2024 Acked Write 10 Sent Done Done false
4231 2025 Acked Read 8 Sent Done Done false
4232 2026 Acked Read 8 Sent Done Done false
4233 2027 Acked Read 10 Sent Done Done false
4234 2028 Acked Write 10 Sent Done Done false
4235 2029 Acked Write 2 Sent Done Done false
4236 2030 Acked Write 10 Sent Done Done false
4237 2031 Acked Read 2 Sent Done Done false
4238 2032 Acked Read 9 Sent Done Done false
4239 2033 Acked Write 6 Sent Done Done false
4240 2034 Acked Read 7 Sent Done Done false
4241 2035 Acked Read 3 Sent Done Done false
4242 2036 Acked Read 6 Sent Done Done false
4243 2037 Acked Write 8 Sent Done Done false
4244 2038 Acked Read 10 Sent Done Done false
4245 2039 Acked Write 7 Sent Done Done false
4246 2040 Acked Write 7 Sent Done Done false
4247 2041 Acked Read 2 Sent Done Done false
4248 2042 Acked Read 3 Sent Done Done false
4249 2043 Acked Write 3 Sent Done Done false
4250 2044 Acked Read 7 Sent Done Done false
4251 2045 Acked Read 5 Sent Done Done false
4252 2046 Acked Read 6 Sent Done Done false
4253 2047 Acked Read 3 Sent Done Done false
4254 2048 Acked Write 8 Sent Done Done false
4255 2049 Acked Write 10 Sent Done Done false
4256 2050 Acked Read 1 Sent Done Done false
4257 2051 Acked Read 5 Sent Done Done false
4258 2052 Acked Read 4 Sent Done Done false
4259 2053 Acked Write 7 Sent Done Done false
4260 2054 Acked Read 7 Sent Done Done false
4261 2055 Acked Read 3 Sent Done Done false
4262 2056 Acked Write 3 Sent Done Done false
4263 2057 Acked Write 9 Sent Done Done false
4264 2058 Acked Read 6 Sent Done Done false
4265 2059 Acked Read 10 Sent Done Done false
4266 2060 Acked Read 8 Sent Done Done false
4267 2061 Acked Write 6 Sent Done Done false
4268 2062 Acked Flush 0 Sent Done Done false
4269 2063 Acked Write 4 Sent Done Done false
4270 2064 Acked Write 5 Sent Done Done false
4271 2065 Acked Read 1 Sent Done Done false
4272 2066 Acked Read 8 Sent Done Done false
4273 2067 Acked Write 8 Sent Done Done false
4274 2068 Acked Flush 0 Sent Done Done false
4275 2069 Acked Read 3 Sent Done Done false
4276 2070 Acked Read 7 Sent Done Done false
4277 2071 Acked Read 6 Sent Done Done false
4278 2072 Acked Write 5 Sent Done Done false
4279 2073 Acked Write 2 Sent Done Done false
4280 2074 Acked Flush 0 Sent Done Done false
4281 2075 Acked Read 9 Sent Done Done false
4282 2076 Acked Read 1 Sent Done Done false
4283 2077 Acked Flush 0 Sent Done Done false
4284 2078 Acked Read 1 Sent Done Done false
4285 2079 Acked Read 1 Sent Done Done false
4286 2080 Acked Read 2 Sent Done Done false
4287 2081 Acked Read 6 Sent Done Done false
4288 2082 Acked Write 9 Sent Done Done false
4289 2083 Acked Flush 0 Sent Done Done false
4290 2084 Acked Write 5 Sent Done Done false
4291 2085 Acked Write 1 Sent Done Done false
4292 2086 Acked Write 2 Sent Done Done false
4293 2087 Acked Write 7 Sent Done Done false
4294 2088 Acked Write 5 Sent Done Done false
4295 2089 Acked Write 7 Sent Done Done false
4296 2090 Acked Write 10 Sent Done Done false
4297 2091 Acked Write 7 Sent Done Done false
4298 2092 Acked Write 3 Sent Done Done false
4299 2093 Acked Read 6 Sent Done Done false
4300 2094 Acked Read 2 Sent Done Done false
4301 2095 Acked Write 7 Sent Done Done false
4302 2096 Acked Read 1 Sent Done Done false
4303 2097 Acked Write 8 Sent Done Done false
4304 2098 Acked Write 5 Sent Done Done false
4305 2099 Acked Write 7 Sent Done Done false
4306 2100 Acked Read 1 Sent Done Done false
4307 2101 Acked Write 3 Sent Done Done false
4308 2102 Acked Read 7 Sent Done Done false
4309 2103 Acked Write 7 Sent Done Done false
4310 2104 Acked Read 5 Sent Done Done false
4311 2105 Acked Write 1 Sent Done Done false
4312 2106 Acked Read 6 Sent Done Done false
4313 2107 Acked Read 9 Sent Done Done false
4314 2108 Acked Write 7 Sent Done Done false
4315 2109 Acked Flush 0 Sent Done Done false
4316 2110 Acked Read 4 Sent Done Done false
4317 2111 Acked Read 4 Sent Done Done false
4318 2112 Acked Read 2 Sent Done Done false
4319 2113 Acked Write 6 Sent Done Done false
4320 2114 Acked Write 9 Sent Done Done false
4321 2115 Acked Read 6 Sent Done Done false
4322 2116 Acked Write 9 Sent Done Done false
4323 2117 Acked Write 10 Sent Done Done false
4324 2118 Acked Write 4 Sent Done Done false
4325 2119 Acked Write 1 Sent Done Done false
4326 2120 Acked Write 2 Sent Done Done false
4327 2121 Acked Read 10 Sent Done Done false
4328 2122 Acked Write 4 Sent Done Done false
4329 2123 Acked Read 3 Sent Done Done false
4330 2124 Acked Write 8 Sent Done Done false
4331 2125 Acked Read 1 Sent Done Done false
4332 2126 Acked Read 6 Sent Done Done false
4333 2127 Acked Read 6 Sent Done Done false
4334 2128 Acked Read 6 Sent Done Done false
4335 2129 Acked Read 4 Sent Done Done false
4336 2130 Acked Read 10 Sent Done Done false
4337 2131 Acked Write 1 Sent Done Done false
4338 2132 Acked Write 3 Sent Done Done false
4339 2133 Acked Flush 0 Sent Done Done false
4340 2134 Acked Write 3 Sent Done Done false
4341 2135 Acked Read 7 Sent Done Done false
4342 2136 Acked Write 9 Sent Done Done false
4343 2137 Acked Write 6 Sent Done Done false
4344 2138 Acked Write 5 Sent Done Done false
4345 2139 Acked Read 6 Sent Done Done false
4346 2140 Acked Flush 0 Sent Done Done false
4347 2141 Acked Flush 0 Sent Done Done false
4348 2142 Acked Write 1 Sent Done Done false
4349 2143 Acked Write 4 Sent Done Done false
4350 2144 Acked Write 7 Sent Done Done false
4351 2145 Acked Read 6 Sent Done Done false
4352 2146 Acked Write 5 Sent Done Done false
4353 2147 Acked Write 7 Sent Done Done false
4354 2148 Acked Read 4 Sent Done Done false
4355 2149 Acked Read 6 Sent Done Done false
4356 2150 Acked Read 10 Sent Done Done false
4357 2151 Acked Write 1 Sent Done Done false
4358 2152 Acked Write 5 Sent Done Done false
4359 2153 Acked Write 8 Sent Done Done false
4360 2154 Acked Read 2 Sent Done Done false
4361 2155 Acked Read 9 Sent Done Done false
4362 2156 Acked Write 3 Sent Done Done false
4363 2157 Acked Read 9 Sent Done Done false
4364 2158 Acked Read 10 Sent Done Done false
4365 2159 Acked Write 4 Sent Done Done false
4366 2160 Acked Read 1 Sent Done Done false
4367 2161 Acked Write 5 Sent Done Done false
4368 2162 Acked Read 10 Sent Done Done false
4369 2163 Acked Read 7 Sent Done Done false
4370 2164 Acked Read 6 Sent Done Done false
4371 2165 Acked Read 10 Sent Done Done false
4372 2166 Acked Read 3 Sent Done Done false
4373 2167 Acked Flush 0 Sent Done Done false
4374 2168 Acked Write 6 Sent Done Done false
4375 2169 Acked Read 1 Sent Done Done false
4376 2170 Acked Write 8 Sent Done Done false
4377 2171 Acked Write 9 Sent Done Done false
4378 2172 Acked Read 1 Sent Done Done false
4379 2173 Acked Write 3 Sent Done Done false
4380 2174 Acked Write 6 Sent Done Done false
4381 2175 Acked Read 9 Sent Done Done false
4382 2176 Acked Read 4 Sent Done Done false
4383 2177 Acked Read 1 Sent Done Done false
4384 2178 Acked Write 8 Sent Done Done false
4385 2179 Acked Read 6 Sent Done Done false
4386 2180 Acked Write 6 Sent Done Done false
4387 2181 Acked Read 1 Sent Done Done false
4388 2182 Acked Write 2 Sent Done Done false
4389 2183 Acked Read 2 Sent Done Done false
4390 2184 Acked Write 6 Sent Done Done false
4391 2185 Acked Read 5 Sent Done Done false
4392 2186 Acked Write 7 Sent Done Done false
4393 2187 Acked Read 5 Sent Done Done false
4394 2188 Acked Write 9 Sent Done Done false
4395 2189 Acked Read 4 Sent Done Done false
4396 2190 Acked Read 1 Sent Done Done false
4397 2191 Acked Read 6 Sent Done Done false
4398 2192 Acked Read 8 Sent Done Done false
4399 2193 Acked Write 2 Sent Done Done false
4400 2194 Acked Flush 0 Sent Done Done false
4401 2195 Acked Write 1 Sent Done Done false
4402 2196 Acked Read 2 Sent Done Done false
4403 2197 Acked Flush 0 Sent Done Done false
4404 2198 Acked Read 3 Sent Done Done false
4405 2199 Acked Read 8 Sent Done Done false
4406 2200 Acked Read 1 Sent Done Done false
4407 2201 Acked Read 4 Sent Done Done false
4408 2202 Acked Read 2 Sent Done Done false
4409 2203 Acked Write 6 Sent Done Done false
4410 2204 Acked Read 3 Sent Done Done false
4411 2205 Acked Read 10 Sent Done Done false
4412 2206 Acked Read 5 Sent Done Done false
4413 STATES DS:0 DS:1 DS:2 TOTAL
4414 Sent 301 0 0 301
4415 Done 0 301 301 602
4416 Skipped 0 0 0 0
4417 Error 0 0 0 0
4418 Last Flush: 1905 2197 2197
4419 Downstairs last five completed: 1905 1904 1903 1902 1901
4420 Upstairs last five completed: 2206 2205 2204 2203 2202
4421 CLIENT: Up:0 ds:301 act:2
4422 [8810] got start action
4423 [8810] I am going to restart
4424 Starting downstairs at port 8810
4425 Make output file at "/tmp/dsc/downstairs-8810.txt"
4426 [8810][0] reports Starting
4427 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1204)
4428 [8810][0] reports Running
4429 ----------------------------------------------------------------
4430 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4431 States: Offline Active Active
4432 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4433 1906 Acked Write 7 Sent Done Done false
4434 1907 Acked Read 7 Sent Done Done false
4435 1908 Acked Read 5 Sent Done Done false
4436 1909 Acked Write 6 Sent Done Done false
4437 1910 Acked Read 1 Sent Done Done false
4438 1911 Acked Write 10 Sent Done Done false
4439 1912 Acked Read 9 Sent Done Done false
4440 1913 Acked Read 9 Sent Done Done false
4441 1914 Acked Write 9 Sent Done Done false
4442 1915 Acked Read 6 Sent Done Done false
4443 1916 Acked Read 9 Sent Done Done false
4444 1917 Acked Read 2 Sent Done Done false
4445 1918 Acked Read 7 Sent Done Done false
4446 1919 Acked Read 7 Sent Done Done false
4447 1920 Acked Read 7 Sent Done Done false
4448 1921 Acked Read 6 Sent Done Done false
4449 1922 Acked Write 10 Sent Done Done false
4450 1923 Acked Write 2 Sent Done Done false
4451 1924 Acked Read 5 Sent Done Done false
4452 1925 Acked Write 5 Sent Done Done false
4453 1926 Acked Read 3 Sent Done Done false
4454 1927 Acked Read 3 Sent Done Done false
4455 1928 Acked Write 10 Sent Done Done false
4456 1929 Acked Write 1 Sent Done Done false
4457 1930 Acked Write 4 Sent Done Done false
4458 1931 Acked Write 7 Sent Done Done false
4459 1932 Acked Read 4 Sent Done Done false
4460 1933 Acked Write 10 Sent Done Done false
4461 1934 Acked Write 1 Sent Done Done false
4462 1935 Acked Write 1 Sent Done Done false
4463 1936 Acked Read 8 Sent Done Done false
4464 1937 Acked Read 5 Sent Done Done false
4465 1938 Acked Write 6 Sent Done Done false
4466 1939 Acked Read 5 Sent Done Done false
4467 1940 Acked Read 10 Sent Done Done false
4468 1941 Acked Write 4 Sent Done Done false
4469 1942 Acked Flush 0 Sent Done Done false
4470 1943 Acked Write 1 Sent Done Done false
4471 1944 Acked Write 3 Sent Done Done false
4472 1945 Acked Write 10 Sent Done Done false
4473 1946 Acked Write 1 Sent Done Done false
4474 1947 Acked Read 3 Sent Done Done false
4475 1948 Acked Read 5 Sent Done Done false
4476 1949 Acked Read 10 Sent Done Done false
4477 1950 Acked Read 5 Sent Done Done false
4478 1951 Acked Read 9 Sent Done Done false
4479 1952 Acked Read 2 Sent Done Done false
4480 1953 Acked Flush 0 Sent Done Done false
4481 1954 Acked Write 7 Sent Done Done false
4482 1955 Acked Write 7 Sent Done Done false
4483 1956 Acked Read 6 Sent Done Done false
4484 1957 Acked Read 5 Sent Done Done false
4485 1958 Acked Write 7 Sent Done Done false
4486 1959 Acked Read 1 Sent Done Done false
4487 1960 Acked Read 4 Sent Done Done false
4488 1961 Acked Read 8 Sent Done Done false
4489 1962 Acked Write 6 Sent Done Done false
4490 1963 Acked Write 8 Sent Done Done false
4491 1964 Acked Read 5 Sent Done Done false
4492 1965 Acked Read 1 Sent Done Done false
4493 1966 Acked Read 1 Sent Done Done false
4494 1967 Acked Read 4 Sent Done Done false
4495 1968 Acked Read 10 Sent Done Done false
4496 1969 Acked Flush 0 Sent Done Done false
4497 1970 Acked Write 1 Sent Done Done false
4498 1971 Acked Read 2 Sent Done Done false
4499 1972 Acked Flush 0 Sent Done Done false
4500 1973 Acked Read 8 Sent Done Done false
4501 1974 Acked Read 2 Sent Done Done false
4502 1975 Acked Read 4 Sent Done Done false
4503 1976 Acked Flush 0 Sent Done Done false
4504 1977 Acked Write 7 Sent Done Done false
4505 1978 Acked Read 5 Sent Done Done false
4506 1979 Acked Write 5 Sent Done Done false
4507 1980 Acked Read 3 Sent Done Done false
4508 1981 Acked Write 9 Sent Done Done false
4509 1982 Acked Read 7 Sent Done Done false
4510 1983 Acked Write 4 Sent Done Done false
4511 1984 Acked Read 8 Sent Done Done false
4512 1985 Acked Write 8 Sent Done Done false
4513 1986 Acked Write 10 Sent Done Done false
4514 1987 Acked Read 9 Sent Done Done false
4515 1988 Acked Read 9 Sent Done Done false
4516 1989 Acked Write 7 Sent Done Done false
4517 1990 Acked Write 6 Sent Done Done false
4518 1991 Acked Read 2 Sent Done Done false
4519 1992 Acked Write 9 Sent Done Done false
4520 1993 Acked Read 7 Sent Done Done false
4521 1994 Acked Write 7 Sent Done Done false
4522 1995 Acked Read 3 Sent Done Done false
4523 1996 Acked Read 4 Sent Done Done false
4524 1997 Acked Read 9 Sent Done Done false
4525 1998 Acked Read 4 Sent Done Done false
4526 1999 Acked Write 4 Sent Done Done false
4527 2000 Acked Flush 0 Sent Done Done false
4528 2001 Acked Read 1 Sent Done Done false
4529 2002 Acked Write 4 Sent Done Done false
4530 2003 Acked Read 5 Sent Done Done false
4531 2004 Acked Read 8 Sent Done Done false
4532 2005 Acked Read 3 Sent Done Done false
4533 2006 Acked Read 1 Sent Done Done false
4534 2007 Acked Read 8 Sent Done Done false
4535 2008 Acked Read 4 Sent Done Done false
4536 2009 Acked Read 4 Sent Done Done false
4537 2010 Acked Read 2 Sent Done Done false
4538 2011 Acked Write 5 Sent Done Done false
4539 2012 Acked Write 6 Sent Done Done false
4540 2013 Acked Write 10 Sent Done Done false
4541 2014 Acked Read 9 Sent Done Done false
4542 2015 Acked Read 10 Sent Done Done false
4543 2016 Acked Read 6 Sent Done Done false
4544 2017 Acked Write 9 Sent Done Done false
4545 2018 Acked Read 8 Sent Done Done false
4546 2019 Acked Write 6 Sent Done Done false
4547 2020 Acked Read 3 Sent Done Done false
4548 2021 Acked Read 10 Sent Done Done false
4549 2022 Acked Write 7 Sent Done Done false
4550 2023 Acked Read 3 Sent Done Done false
4551 2024 Acked Write 10 Sent Done Done false
4552 2025 Acked Read 8 Sent Done Done false
4553 2026 Acked Read 8 Sent Done Done false
4554 2027 Acked Read 10 Sent Done Done false
4555 2028 Acked Write 10 Sent Done Done false
4556 2029 Acked Write 2 Sent Done Done false
4557 2030 Acked Write 10 Sent Done Done false
4558 2031 Acked Read 2 Sent Done Done false
4559 2032 Acked Read 9 Sent Done Done false
4560 2033 Acked Write 6 Sent Done Done false
4561 2034 Acked Read 7 Sent Done Done false
4562 2035 Acked Read 3 Sent Done Done false
4563 2036 Acked Read 6 Sent Done Done false
4564 2037 Acked Write 8 Sent Done Done false
4565 2038 Acked Read 10 Sent Done Done false
4566 2039 Acked Write 7 Sent Done Done false
4567 2040 Acked Write 7 Sent Done Done false
4568 2041 Acked Read 2 Sent Done Done false
4569 2042 Acked Read 3 Sent Done Done false
4570 2043 Acked Write 3 Sent Done Done false
4571 2044 Acked Read 7 Sent Done Done false
4572 2045 Acked Read 5 Sent Done Done false
4573 2046 Acked Read 6 Sent Done Done false
4574 2047 Acked Read 3 Sent Done Done false
4575 2048 Acked Write 8 Sent Done Done false
4576 2049 Acked Write 10 Sent Done Done false
4577 2050 Acked Read 1 Sent Done Done false
4578 2051 Acked Read 5 Sent Done Done false
4579 2052 Acked Read 4 Sent Done Done false
4580 2053 Acked Write 7 Sent Done Done false
4581 2054 Acked Read 7 Sent Done Done false
4582 2055 Acked Read 3 Sent Done Done false
4583 2056 Acked Write 3 Sent Done Done false
4584 2057 Acked Write 9 Sent Done Done false
4585 2058 Acked Read 6 Sent Done Done false
4586 2059 Acked Read 10 Sent Done Done false
4587 2060 Acked Read 8 Sent Done Done false
4588 2061 Acked Write 6 Sent Done Done false
4589 2062 Acked Flush 0 Sent Done Done false
4590 2063 Acked Write 4 Sent Done Done false
4591 2064 Acked Write 5 Sent Done Done false
4592 2065 Acked Read 1 Sent Done Done false
4593 2066 Acked Read 8 Sent Done Done false
4594 2067 Acked Write 8 Sent Done Done false
4595 2068 Acked Flush 0 Sent Done Done false
4596 2069 Acked Read 3 Sent Done Done false
4597 2070 Acked Read 7 Sent Done Done false
4598 2071 Acked Read 6 Sent Done Done false
4599 2072 Acked Write 5 Sent Done Done false
4600 2073 Acked Write 2 Sent Done Done false
4601 2074 Acked Flush 0 Sent Done Done false
4602 2075 Acked Read 9 Sent Done Done false
4603 2076 Acked Read 1 Sent Done Done false
4604 2077 Acked Flush 0 Sent Done Done false
4605 2078 Acked Read 1 Sent Done Done false
4606 2079 Acked Read 1 Sent Done Done false
4607 2080 Acked Read 2 Sent Done Done false
4608 2081 Acked Read 6 Sent Done Done false
4609 2082 Acked Write 9 Sent Done Done false
4610 2083 Acked Flush 0 Sent Done Done false
4611 2084 Acked Write 5 Sent Done Done false
4612 2085 Acked Write 1 Sent Done Done false
4613 2086 Acked Write 2 Sent Done Done false
4614 2087 Acked Write 7 Sent Done Done false
4615 2088 Acked Write 5 Sent Done Done false
4616 2089 Acked Write 7 Sent Done Done false
4617 2090 Acked Write 10 Sent Done Done false
4618 2091 Acked Write 7 Sent Done Done false
4619 2092 Acked Write 3 Sent Done Done false
4620 2093 Acked Read 6 Sent Done Done false
4621 2094 Acked Read 2 Sent Done Done false
4622 2095 Acked Write 7 Sent Done Done false
4623 2096 Acked Read 1 Sent Done Done false
4624 2097 Acked Write 8 Sent Done Done false
4625 2098 Acked Write 5 Sent Done Done false
4626 2099 Acked Write 7 Sent Done Done false
4627 2100 Acked Read 1 Sent Done Done false
4628 2101 Acked Write 3 Sent Done Done false
4629 2102 Acked Read 7 Sent Done Done false
4630 2103 Acked Write 7 Sent Done Done false
4631 2104 Acked Read 5 Sent Done Done false
4632 2105 Acked Write 1 Sent Done Done false
4633 2106 Acked Read 6 Sent Done Done false
4634 2107 Acked Read 9 Sent Done Done false
4635 2108 Acked Write 7 Sent Done Done false
4636 2109 Acked Flush 0 Sent Done Done false
4637 2110 Acked Read 4 Sent Done Done false
4638 2111 Acked Read 4 Sent Done Done false
4639 2112 Acked Read 2 Sent Done Done false
4640 2113 Acked Write 6 Sent Done Done false
4641 2114 Acked Write 9 Sent Done Done false
4642 2115 Acked Read 6 Sent Done Done false
4643 2116 Acked Write 9 Sent Done Done false
4644 2117 Acked Write 10 Sent Done Done false
4645 2118 Acked Write 4 Sent Done Done false
4646 2119 Acked Write 1 Sent Done Done false
4647 2120 Acked Write 2 Sent Done Done false
4648 2121 Acked Read 10 Sent Done Done false
4649 2122 Acked Write 4 Sent Done Done false
4650 2123 Acked Read 3 Sent Done Done false
4651 2124 Acked Write 8 Sent Done Done false
4652 2125 Acked Read 1 Sent Done Done false
4653 2126 Acked Read 6 Sent Done Done false
4654 2127 Acked Read 6 Sent Done Done false
4655 2128 Acked Read 6 Sent Done Done false
4656 2129 Acked Read 4 Sent Done Done false
4657 2130 Acked Read 10 Sent Done Done false
4658 2131 Acked Write 1 Sent Done Done false
4659 2132 Acked Write 3 Sent Done Done false
4660 2133 Acked Flush 0 Sent Done Done false
4661 2134 Acked Write 3 Sent Done Done false
4662 2135 Acked Read 7 Sent Done Done false
4663 2136 Acked Write 9 Sent Done Done false
4664 2137 Acked Write 6 Sent Done Done false
4665 2138 Acked Write 5 Sent Done Done false
4666 2139 Acked Read 6 Sent Done Done false
4667 2140 Acked Flush 0 Sent Done Done false
4668 2141 Acked Flush 0 Sent Done Done false
4669 2142 Acked Write 1 Sent Done Done false
4670 2143 Acked Write 4 Sent Done Done false
4671 2144 Acked Write 7 Sent Done Done false
4672 2145 Acked Read 6 Sent Done Done false
4673 2146 Acked Write 5 Sent Done Done false
4674 2147 Acked Write 7 Sent Done Done false
4675 2148 Acked Read 4 Sent Done Done false
4676 2149 Acked Read 6 Sent Done Done false
4677 2150 Acked Read 10 Sent Done Done false
4678 2151 Acked Write 1 Sent Done Done false
4679 2152 Acked Write 5 Sent Done Done false
4680 2153 Acked Write 8 Sent Done Done false
4681 2154 Acked Read 2 Sent Done Done false
4682 2155 Acked Read 9 Sent Done Done false
4683 2156 Acked Write 3 Sent Done Done false
4684 2157 Acked Read 9 Sent Done Done false
4685 2158 Acked Read 10 Sent Done Done false
4686 2159 Acked Write 4 Sent Done Done false
4687 2160 Acked Read 1 Sent Done Done false
4688 2161 Acked Write 5 Sent Done Done false
4689 2162 Acked Read 10 Sent Done Done false
4690 2163 Acked Read 7 Sent Done Done false
4691 2164 Acked Read 6 Sent Done Done false
4692 2165 Acked Read 10 Sent Done Done false
4693 2166 Acked Read 3 Sent Done Done false
4694 2167 Acked Flush 0 Sent Done Done false
4695 2168 Acked Write 6 Sent Done Done false
4696 2169 Acked Read 1 Sent Done Done false
4697 2170 Acked Write 8 Sent Done Done false
4698 2171 Acked Write 9 Sent Done Done false
4699 2172 Acked Read 1 Sent Done Done false
4700 2173 Acked Write 3 Sent Done Done false
4701 2174 Acked Write 6 Sent Done Done false
4702 2175 Acked Read 9 Sent Done Done false
4703 2176 Acked Read 4 Sent Done Done false
4704 2177 Acked Read 1 Sent Done Done false
4705 2178 Acked Write 8 Sent Done Done false
4706 2179 Acked Read 6 Sent Done Done false
4707 2180 Acked Write 6 Sent Done Done false
4708 2181 Acked Read 1 Sent Done Done false
4709 2182 Acked Write 2 Sent Done Done false
4710 2183 Acked Read 2 Sent Done Done false
4711 2184 Acked Write 6 Sent Done Done false
4712 2185 Acked Read 5 Sent Done Done false
4713 2186 Acked Write 7 Sent Done Done false
4714 2187 Acked Read 5 Sent Done Done false
4715 2188 Acked Write 9 Sent Done Done false
4716 2189 Acked Read 4 Sent Done Done false
4717 2190 Acked Read 1 Sent Done Done false
4718 2191 Acked Read 6 Sent Done Done false
4719 2192 Acked Read 8 Sent Done Done false
4720 2193 Acked Write 2 Sent Done Done false
4721 2194 Acked Flush 0 Sent Done Done false
4722 2195 Acked Write 1 Sent Done Done false
4723 2196 Acked Read 2 Sent Done Done false
4724 2197 Acked Flush 0 Sent Done Done false
4725 2198 Acked Read 3 Sent Done Done false
4726 2199 Acked Read 8 Sent Done Done false
4727 2200 Acked Read 1 Sent Done Done false
4728 2201 Acked Read 4 Sent Done Done false
4729 2202 Acked Read 2 Sent Done Done false
4730 2203 Acked Write 6 Sent Done Done false
4731 2204 Acked Read 3 Sent Done Done false
4732 2205 Acked Read 10 Sent Done Done false
4733 2206 Acked Read 5 Sent Done Done false
4734 2207 Acked Flush 0 Sent Done Done false
4735 STATES DS:0 DS:1 DS:2 TOTAL
4736 Sent 302 0 0 302
4737 Done 0 302 302 604
4738 Skipped 0 0 0 0
4739 Error 0 0 0 0
4740 Last Flush: 1905 2207 2207
4741 Downstairs last five completed: 1905 1904 1903 1902 1901
4742 Upstairs last five completed: 2207 2206 2205 2204 2203
4743 CLIENT: Up:0 ds:302 act:2
47442024-12-11T23:41:29.270ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47452024-12-11T23:41:29.270ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55412) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47462024-12-11T23:41:29.271ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47472024-12-11T23:41:29.271ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47482024-12-11T23:41:29.271ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47492024-12-11T23:41:29.271ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47502024-12-11T23:41:29.272ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47512024-12-11T23:41:29.272ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47522024-12-11T23:41:29.272ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47532024-12-11T23:41:29.272ZINFOcrucible: [0] client re-new 302 jobs since flush 1905 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47542024-12-11T23:41:29.272ZINFOcrucible: [0] Replayed 302 jobs since flush: 1905 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
4755 ----------------------------------------------------------------
4756 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4757 Downstairs last five completed: 2207 2206 2205 2204 2203
4758 Upstairs last five completed: 2207 2206 2205 2204 2203
4759 CLIENT: Up:0 ds:0 act:3
4760 Replay: All jobs finished
4761 Main task has work to do, go find it
4762 got dsc Stop(0)
4763 stop 0
4764 Dec 11 23:41:31.519 INFO request completed, latency_us: 369[8810] Got stop action so:false kr:false
4765 , response_code: 204, uri: /stop/cid/0, method: POST, req_id: b4cf4405-d31e-423c-9729-9a6b80a20bc7, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
47662024-12-11T23:41:31.520ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47672024-12-11T23:41:31.520ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
47682024-12-11T23:41:31.520ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
4769 Dec 11 23:41:31.520 INFO request completed, latency_us: 130, response_code: {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:41:31.520245397Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
4770 200, uri: /state/cid/0, method{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:41:31.520277548Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
4771 : GET, req_id: 76409450-5b69-419d-88a6-23458c1ecba3, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
47722024-12-11T23:41:31.520ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
4773 [8810] Exited with: ExitStatus(unix_wait_status(9))
4774 [8810][0] reports Exit
4775 Dec 11 23:41:35.522 INFO request completed, latency_us: 257, response_code: 200, uri: /state/cid/0, method: GET, req_id: a9b6f655-4749-40c9-bfef-2fcbd54d4938, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
4776 001/300 Write block 4456 len 4096 data: 2
4777 002/300 Write block 4852 len 32768 data: 2 2 2 2 2 3 3 3
4778 003/300 Read block 2575 len 36864
4779 004/300 Read block 1164 len 40960
4780 005/300 Write block 3166 len 28672 data: 2 2 3 3 3 2 2
4781 006/300 Flush
4782 007/300 Flush
4783 008/300 Read block 929 len 12288
4784 009/300 Read block 3612 len 28672
4785 010/300 Flush
4786 011/300 Write block 3284 len 40960 data: 3 2 2 2 2 2 3 3 3 3
4787 012/300 Read block 1226 len 28672
4788 013/300 Read block 2966 len 20480
4789 014/300 Read block 3970 len 8192
4790 015/300 Read block 1178 len 16384
4791 016/300 Read block 2670 len 32768
4792 017/300 Read block 1194 len 28672
4793 018/300 Read block 4637 len 24576
4794 019/300 Read block 10 len 20480
4795 020/300 Read block 4798 len 4096
4796 021/300 Read block 872 len 24576
4797 022/300 Read block 807 len 16384
4798 023/300 Read block 2716 len 36864
4799 024/300 Flush
4800 025/300 Read block 4669 len 8192
4801 026/300 Read block 2983 len 36864
4802 027/300 Write block 951 len 16384 data: 2 2 2 2
4803 028/300 Flush
4804 029/300 Read block 2033 len 8192
4805 030/300 Flush
4806 031/300 Write block 2952 len 32768 data: 2 2 2 2 2 2 2 2
4807 032/300 Read block 1916 len 4096
4808 033/300 Write block 2178 len 20480 data: 2 2 2 2 2
4809 034/300 Read block 4451 len 28672
4810 035/300 Read block 4114 len 28672
4811 036/300 Flush
4812 037/300 Write block 1638 len 8192 data: 3 3
4813 038/300 Read block 636 len 12288
4814 039/300 Write block 1792 len 12288 data: 3 3 3
4815 040/300 Read block 1182 len 12288
4816 041/300 Read block 193 len 32768
4817 042/300 Read block 4259 len 24576
4818 043/300 Write block 2644 len 12288 data: 2 2 2
4819 044/300 Write block 192 len 20480 data: 2 2 2 2 3
4820 045/300 Write block 3991 len 12288 data: 3 3 2
4821 046/300 Flush
4822 047/300 Write block 2931 len 20480 data: 4 4 4 4 4
4823 048/300 Write block 3172 len 8192 data: 3 2
4824 049/300 Read block 4235 len 12288
4825 050/300 Flush
4826 051/300 Write block 44 len 28672 data: 4 4 4 3 3 3 3
4827 052/300 Flush
4828 053/300 Write block 1624 len 32768 data: 2 2 2 2 2 2 2 2
4829 054/300 Read block 2128 len 28672
4830 055/300 Write block 983 len 20480 data: 3 3 3 3 2
4831 056/300 Write block 2478 len 32768 data: 2 2 2 2 3 3 3 3
4832 057/300 Write block 2464 len 24576 data: 2 2 2 2 2 2
4833 058/300 Read block 4488 len 16384
4834 059/300 Write block 3660 len 16384 data: 3 3 3 3
4835 060/300 Read block 3354 len 8192
4836 061/300 Read block 4713 len 16384
4837 062/300 Read block 3695 len 4096
4838 063/300 Read block 4302 len 24576
4839 064/300 Read block 3186 len 12288
4840 065/300 Read block 972 len 36864
4841 066/300 Read block 3532 len 24576
4842 067/300 Read block 2875 len 32768
4843 068/300 Write block 2931 len 12288 data: 5 5 5
4844 069/300 Read block 407 len 4096
4845 070/300 Read block 1020 len 32768
4846 071/300 Write block 85 len 28672 data: 3 2 2 2 2 2 2
4847 072/300 Write block 4926 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4848 073/300 Read block 2074 len 28672
4849 074/300 Write block 443 len 24576 data: 2 3 4 4 3 3
4850 075/300 Read block 4909 len 28672
4851 076/300 Write block 2091 len 40960 data: 2 2 2 3 2 2 2 2 2 2
4852 077/300 Read block 1588 len 4096
4853 078/300 Flush
4854 079/300 Read block 616 len 32768
4855 080/300 Read block 4317 len 4096
4856 081/300 Flush
4857 082/300 Read block 1717 len 16384
4858 083/300 Read block 1700 len 28672
4859 084/300 Flush
4860 085/300 Write block 718 len 36864 data: 2 2 2 2 2 2 2 2 2
4861 086/300 Write block 1850 len 12288 data: 3 3 3
4862 087/300 Read block 4048 len 16384
4863 088/300 Read block 3097 len 36864
4864 089/300 Read block 723 len 16384
4865 090/300 Read block 1749 len 28672
4866 091/300 Read block 2444 len 4096
4867 092/300 Flush
4868 093/300 Write block 800 len 36864 data: 2 2 2 2 2 2 2 2 3
4869 094/300 Write block 4177 len 40960 data: 2 2 2 3 3 3 3 3 3 3
4870 095/300 Write block 4054 len 36864 data: 2 2 2 2 2 2 3 3 3
4871 096/300 Read block 3022 len 20480
4872 097/300 Read block 2005 len 40960
4873 098/300 Read block 3208 len 20480
4874 099/300 Read block 617 len 12288
4875 100/300 Write block 1986 len 36864 data: 4 4 3 3 2 2 3 2 2
4876 101/300 Read block 1531 len 32768
4877 102/300 Flush
4878 103/300 Flush
4879 104/300 Write block 2621 len 4096 data: 2
4880 105/300 Read block 1606 len 4096
4881 106/300 Write block 4495 len 16384 data: 3 3 4 4
4882 107/300 Flush
4883 108/300 Read block 1543 len 24576
4884 109/300 Read block 4680 len 40960
4885 110/300 Read block 3026 len 24576
4886 111/300 Write block 2574 len 20480 data: 3 4 4 4 4
4887 112/300 Write block 573 len 28672 data: 2 2 2 2 2 2 2
4888 113/300 Read block 2437 len 16384
4889 114/300 Write block 4944 len 16384 data: 3 3 3 2
4890 115/300 Write block 104 len 8192 data: 3 3
4891 116/300 Read block 2128 len 28672
4892 117/300 Read block 497 len 4096
4893 118/300 Read block 1963 len 32768
4894 119/300 Read block 4441 len 20480
4895 120/300 Flush
4896 121/300 Write block 2224 len 36864 data: 2 2 2 2 2 2 2 2 2
4897 122/300 Read block 4628 len 16384
4898 123/300 Write block 2041 len 36864 data: 3 3 3 3 3 3 3 3 3
4899 124/300 Write block 2560 len 40960 data: 3 2 2 2 2 2 2 2 2 2
4900 125/300 Read block 4347 len 28672
4901 126/300 Read block 2901 len 28672
4902 127/300 Read block 526 len 20480
4903 128/300 Read block 746 len 12288
4904 129/300 Read block 923 len 40960
4905 130/300 Flush
4906 131/300 Write block 862 len 24576 data: 3 3 2 2 3 2
4907 132/300 Flush
4908 133/300 Read block 3281 len 36864
4909 134/300 Write block 912 len 4096 data: 2
4910 135/300 Read block 4402 len 16384
4911 136/300 Write block 54 len 4096 data: 2
4912 137/300 Write block 4233 len 36864 data: 4 4 4 4 2 2 2 2 3
4913 138/300 Read block 2845 len 24576
4914 139/300 Read block 3738 len 36864
4915 140/300 Flush
4916 141/300 Read block 1146 len 20480
4917 142/300 Read block 2693 len 16384
4918 143/300 Write block 4921 len 24576 data: 3 4 3 3 2 3
4919 144/300 Read block 333 len 20480
4920 145/300 Write block 2288 len 24576 data: 3 2 2 2 2 2
4921 146/300 Read block 2890 len 40960
4922 147/300 Read block 1721 len 8192
4923 148/300 Write block 1145 len 4096 data: 2
4924 149/300 Write block 4256 len 4096 data: 3
4925 150/300 Read block 4645 len 4096
4926 151/300 Read block 2388 len 4096
4927 152/300 Write block 1728 len 16384 data: 2 2 2 2
4928 153/300 Flush
4929 154/300 Flush
4930 155/300 Read block 806 len 12288
4931 156/300 Write block 3042 len 4096 data: 3
4932 157/300 Read block 790 len 36864
4933 158/300 Read block 2752 len 36864
4934 159/300 Write block 3177 len 8192 data: 2 2
4935 160/300 Write block 204 len 4096 data: 2
4936 161/300 Write block 2720 len 36864 data: 4 4 4 4 5 4 3 2 2
4937 162/300 Read block 310 len 12288
4938 163/300 Read block 912 len 20480
4939 164/300 Read block 4190 len 24576
4940 165/300 Read block 729 len 24576
4941 166/300 Write block 4968 len 16384 data: 2 2 2 2
4942 167/300 Write block 1147 len 20480 data: 3 3 3 3 3
4943 168/300 Write block 4261 len 16384 data: 2 2 2 2
4944 169/300 Read block 1123 len 4096
4945 170/300 Read block 419 len 16384
4946 171/300 Read block 2085 len 16384
4947 172/300 Read block 4801 len 8192
4948 173/300 Read block 3934 len 32768
4949 174/300 Read block 2452 len 36864
4950 175/300 Read block 2908 len 4096
4951 176/300 Write block 1275 len 24576 data: 2 2 2 2 2 2
4952 177/300 Write block 3001 len 36864 data: 3 3 4 3 3 3 3 3 4
4953 178/300 Read block 1226 len 20480
4954 179/300 Write block 2646 len 8192 data: 3 3
4955 180/300 Read block 36 len 16384
4956 181/300 Read block 1730 len 36864
4957 182/300 Write block 2467 len 20480 data: 3 3 3 3 3
4958 183/300 Read block 1623 len 12288
4959 184/300 Read block 3063 len 36864
4960 185/300 Flush
4961 186/300 Write block 2706 len 4096 data: 2
4962 187/300 Read block 185 len 8192
4963 188/300 Flush
4964 189/300 Flush
4965 190/300 Read block 2274 len 16384
4966 191/300 Read block 992 len 4096
4967 192/300 Read block 3130 len 8192
4968 193/300 Flush
4969 194/300 Write block 661 len 24576 data: 3 2 2 2 3 3
4970 195/300 Read block 992 len 36864
4971 196/300 Write block 2092 len 32768 data: 3 3 4 3 3 3 3 3
4972 197/300 Read block 4621 len 12288
4973 198/300 Write block 1190 len 16384 data: 2 2 2 2
4974 199/300 Write block 1824 len 8192 data: 2 2
4975 200/300 Read block 2645 len 12288
4976 201/300 Read block 2303 len 28672
4977 202/300 Read block 1145 len 4096
4978 203/300 Write block 1317 len 24576 data: 2 2 2 2 2 2
4979 204/300 Read block 1556 len 12288
4980 205/300 Read block 4306 len 8192
4981 206/300 Write block 3118 len 32768 data: 2 2 2 2 2 2 2 2
4982 207/300 Read block 1492 len 28672
4983 208/300 Flush
4984 209/300 Flush
4985 210/300 Write block 4122 len 28672 data: 2 2 3 3 3 2 2
4986 211/300 Write block 1055 len 8192 data: 3 3
4987 212/300 Read block 1369 len 32768
4988 213/300 Read block 1531 len 24576
4989 214/300 Read block 2565 len 4096
4990 215/300 Flush
4991 216/300 Flush
4992 217/300 Write block 4347 len 12288 data: 2 2 2
4993 218/300 Write block 2852 len 32768 data: 3 3 3 4 4 4 3 3
4994 219/300 Flush
4995 220/300 Write block 2365 len 8192 data: 3 3
4996 221/300 Write block 101 len 16384 data: 3 3 3 4
4997 222/300 Read block 1058 len 24576
4998 223/300 Flush
4999 224/300 Write block 2280 len 16384 data: 2 2 2 2
5000 225/300 Read block 909 len 8192
5001 226/300 Read block 1855 len 4096
5002 227/300 Read block 2859 len 8192
5003 228/300 Write block 1409 len 40960 data: 2 3 3 3 3 3 2 2 2 2
5004 229/300 Write block 3141 len 16384 data: 3 3 3 2
5005 230/300 Read block 529 len 12288
5006 231/300 Read block 2082 len 12288
5007 232/300 Read block 863 len 20480
5008 233/300 Read block 203 len 24576
5009 234/300 Read block 1830 len 16384
5010 235/300 Read block 1312 len 28672
5011 236/300 Read block 3763 len 40960
5012 237/300 Read block 916 len 12288
5013 238/300 Read block 940 len 16384
5014 239/300 Write block 4218 len 36864 data: 2 2 3 3 3 3 3 3 3
5015 240/300 Write block 1041 len 4096 data: 2
5016 241/300 Read block 945 len 24576
5017 242/300 Flush
5018 243/300 Read block 24 len 8192
5019 244/300 Read block 2058 len 24576
5020 245/300 Write block 3670 len 28672 data: 2 2 2 2 3 3 2
5021 246/300 Write block 1891 len 28672 data: 3 3 3 3 3 3 2
5022 247/300 Read block 2351 len 8192
5023 248/300 Write block 3848 len 28672 data: 2 2 3 3 3 3 3
5024 249/300 Write block 3506 len 20480 data: 2 2 2 2 2
5025 250/300 Write block 1134 len 28672 data: 3 3 3 3 3 3 3
5026 251/300 Read block 2699 len 20480
5027 252/300 Write block 4093 len 24576 data: 3 3 2 2 2 2
5028 253/300 Flush
5029 254/300 Flush
5030 255/300 Write block 847 len 36864 data: 2 2 2 2 2 2 2 3 3
5031 256/300 Read block 3339 len 40960
5032 257/300 Read block 1567 len 8192
5033 258/300 Flush
5034 259/300 Read block 451 len 40960
5035 260/300 Flush
5036 261/300 Read block 445 len 28672
5037 262/300 Write block 3616 len 36864 data: 3 4 4 5 5 4 3 3 2
5038 263/300 Flush
5039 264/300 Write block 3054 len 28672 data: 3 3 3 3 3 2 2
5040 265/300 Write block 2727 len 28672 data: 3 3 2 2 2 2 4
5041 266/300 Write block 3694 len 16384 data: 2 2 2 3
5042 267/300 Flush
5043 268/300 Read block 3801 len 20480
5044 269/300 Read block 64 len 32768
5045 270/300 Read block 319 len 12288
5046 271/300 Read block 2176 len 16384
5047 272/300 Read block 1158 len 24576
5048 273/300 Read block 4371 len 28672
5049 274/300 Read block 645 len 16384
5050 275/300 Write block 643 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5051 276/300 Read block 4014 len 32768
5052 277/300 Write block 4003 len 16384 data: 3 3 3 3
5053 278/300 Flush
5054 279/300 Write block 2525 len 4096 data: 2
5055 280/300 Write block 2914 len 16384 data: 2 2 2 2
5056 281/300 Flush
5057 282/300 Read block 4566 len 16384
5058 283/300 Write block 2626 len 24576 data: 2 2 2 2 2 2
5059 284/300 Read block 1452 len 8192
5060 285/300 Read block 3753 len 20480
5061 286/300 Write block 3107 len 16384 data: 2 2 2 2
5062 287/300 Read block 4040 len 40960
5063 288/300 Read block 2902 len 4096
5064 289/300 Read block 1890 len 20480
5065 290/300 Write block 4166 len 28672 data: 2 2 2 2 2 2 2
5066 291/300 Write block 812 len 12288 data: 2 2 2
5067 292/300 Write block 2105 len 8192 data: 2 2
5068 293/300 Read block 2178 len 40960
5069 294/300 Write block 4163 len 40960 data: 2 2 2 3 3 3 3 3 3 3
5070 295/300 Read block 857 len 20480
5071 296/300 Write block 1320 len 28672 data: 3 3 3 2 2 2 2
5072 297/300 Flush
5073 298/300 Write block 4530 len 20480 data: 5 5 5 3 3
5074 299/300 Read block 1721 len 12288
5075 300/300 Write block 2885 len 20480 data: 3 3 4 3 3
5076 301/300 Write block 2113 len 16384 data: 3 4 4 4
5077 Main task has work to do, go find it
5078 got dsc Start(0)
5079 start 0
5080 Dec 11 23:41:35.760 INFO request completed, latency_us: 290, response_code: 204, uri: /start/cid/0, method: POST, req_id: c6023c9f-59db-4e18-bc08-2c78c3bae757, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
5081 Replay: started 0, returned:Ok(())
5082 ----------------------------------------------------------------
5083 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5084 States: Offline Active Active
5085 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5086 2208 Acked Write 1 Sent Done Done false
5087 2209 Acked Write 8 Sent Done Done false
5088 2210 Acked Read 9 Sent Done Done false
5089 2211 Acked Read 10 Sent Done Done false
5090 2212 Acked Write 7 Sent Done Done false
5091 2213 Acked Flush 0 Sent Done Done false
5092 2214 Acked Flush 0 Sent Done Done false
5093 2215 Acked Read 3 Sent Done Done false
5094 2216 Acked Read 7 Sent Done Done false
5095 2217 Acked Flush 0 Sent Done Done false
5096 2218 Acked Write 10 Sent Done Done false
5097 2219 Acked Read 7 Sent Done Done false
5098 2220 Acked Read 5 Sent Done Done false
5099 2221 Acked Read 2 Sent Done Done false
5100 2222 Acked Read 4 Sent Done Done false
5101 2223 Acked Read 8 Sent Done Done false
5102 2224 Acked Read 7 Sent Done Done false
5103 2225 Acked Read 6 Sent Done Done false
5104 2226 Acked Read 5 Sent Done Done false
5105 2227 Acked Read 1 Sent Done Done false
5106 2228 Acked Read 6 Sent Done Done false
5107 2229 Acked Read 4 Sent Done Done false
5108 2230 Acked Read 9 Sent Done Done false
5109 2231 Acked Flush 0 Sent Done Done false
5110 2232 Acked Read 2 Sent Done Done false
5111 2233 Acked Read 9 Sent Done Done false
5112 2234 Acked Write 4 Sent Done Done false
5113 2235 Acked Flush 0 Sent Done Done false
5114 2236 Acked Read 2 Sent Done Done false
5115 2237 Acked Flush 0 Sent Done Done false
5116 2238 Acked Write 8 Sent Done Done false
5117 2239 Acked Read 1 Sent Done Done false
5118 2240 Acked Write 5 Sent Done Done false
5119 2241 Acked Read 7 Sent Done Done false
5120 2242 Acked Read 7 Sent Done Done false
5121 2243 Acked Flush 0 Sent Done Done false
5122 2244 Acked Write 2 Sent Done Done false
5123 2245 Acked Read 3 Sent Done Done false
5124 2246 Acked Write 3 Sent Done Done false
5125 2247 Acked Read 3 Sent Done Done false
5126 2248 Acked Read 8 Sent Done Done false
5127 2249 Acked Read 6 Sent Done Done false
5128 2250 Acked Write 3 Sent Done Done false
5129 2251 Acked Write 5 Sent Done Done false
5130 2252 Acked Write 3 Sent Done Done false
5131 2253 Acked Flush 0 Sent Done Done false
5132 2254 Acked Write 5 Sent Done Done false
5133 2255 Acked Write 2 Sent Done Done false
5134 2256 Acked Read 3 Sent Done Done false
5135 2257 Acked Flush 0 Sent Done Done false
5136 2258 Acked Write 7 Sent Done Done false
5137 2259 Acked Flush 0 Sent Done Done false
5138 2260 Acked Write 8 Sent Done Done false
5139 2261 Acked Read 7 Sent Done Done false
5140 2262 Acked Write 5 Sent Done Done false
5141 2263 Acked Write 8 Sent Done Done false
5142 2264 Acked Write 6 Sent Done Done false
5143 2265 Acked Read 4 Sent Done Done false
5144 2266 Acked Write 4 Sent Done Done false
5145 2267 Acked Read 2 Sent Done Done false
5146 2268 Acked Read 4 Sent Done Done false
5147 2269 Acked Read 1 Sent Done Done false
5148 2270 Acked Read 6 Sent Done Done false
5149 2271 Acked Read 3 Sent Done Done false
5150 2272 Acked Read 9 Sent Done Done false
5151 2273 Acked Read 6 Sent Done Done false
5152 2274 Acked Read 8 Sent Done Done false
5153 2275 Acked Write 3 Sent Done Done false
5154 2276 Acked Read 1 Sent Done Done false
5155 2277 Acked Read 8 Sent Done Done false
5156 2278 Acked Write 7 Sent Done Done false
5157 2279 Acked Write 10 Sent Done Done false
5158 2280 Acked Read 7 Sent Done Done false
5159 2281 Acked Write 6 Sent Done Done false
5160 2282 Acked Read 7 Sent Done Done false
5161 2283 Acked Write 10 Sent Done Done false
5162 2284 Acked Read 1 Sent Done Done false
5163 2285 Acked Flush 0 Sent Done Done false
5164 2286 Acked Read 8 Sent Done Done false
5165 2287 Acked Read 1 Sent Done Done false
5166 2288 Acked Flush 0 Sent Done Done false
5167 2289 Acked Read 4 Sent Done Done false
5168 2290 Acked Read 7 Sent Done Done false
5169 2291 Acked Flush 0 Sent Done Done false
5170 2292 Acked Write 9 Sent Done Done false
5171 2293 Acked Write 3 Sent Done Done false
5172 2294 Acked Read 4 Sent Done Done false
5173 2295 Acked Read 9 Sent Done Done false
5174 2296 Acked Read 4 Sent Done Done false
5175 2297 Acked Read 7 Sent Done Done false
5176 2298 Acked Read 1 Sent Done Done false
5177 2299 Acked Flush 0 Sent Done Done false
5178 2300 Acked Write 9 Sent Done Done false
5179 2301 Acked Write 10 Sent Done Done false
5180 2302 Acked Write 9 Sent Done Done false
5181 2303 Acked Read 5 Sent Done Done false
5182 2304 Acked Read 10 Sent Done Done false
5183 2305 Acked Read 5 Sent Done Done false
5184 2306 Acked Read 3 Sent Done Done false
5185 2307 Acked Write 9 Sent Done Done false
5186 2308 Acked Read 8 Sent Done Done false
5187 2309 Acked Flush 0 Sent Done Done false
5188 2310 Acked Flush 0 Sent Done Done false
5189 2311 Acked Write 1 Sent Done Done false
5190 2312 Acked Read 1 Sent Done Done false
5191 2313 Acked Write 4 Sent Done Done false
5192 2314 Acked Flush 0 Sent Done Done false
5193 2315 Acked Read 6 Sent Done Done false
5194 2316 Acked Read 10 Sent Done Done false
5195 2317 Acked Read 6 Sent Done Done false
5196 2318 Acked Write 5 Sent Done Done false
5197 2319 Acked Write 7 Sent Done Done false
5198 2320 Acked Read 4 Sent Done Done false
5199 2321 Acked Write 4 Sent Done Done false
5200 2322 Acked Write 2 Sent Done Done false
5201 2323 Acked Read 7 Sent Done Done false
5202 2324 Acked Read 1 Sent Done Done false
5203 2325 Acked Read 8 Sent Done Done false
5204 2326 Acked Read 5 Sent Done Done false
5205 2327 Acked Flush 0 Sent Done Done false
5206 2328 Acked Write 9 Sent Done Done false
5207 2329 Acked Read 4 Sent Done Done false
5208 2330 Acked Write 9 Sent Done Done false
5209 2331 Acked Write 10 Sent Done Done false
5210 2332 Acked Read 7 Sent Done Done false
5211 2333 Acked Read 7 Sent Done Done false
5212 2334 Acked Read 5 Sent Done Done false
5213 2335 Acked Read 3 Sent Done Done false
5214 2336 Acked Read 10 Sent Done Done false
5215 2337 Acked Flush 0 Sent Done Done false
5216 2338 Acked Write 6 Sent Done Done false
5217 2339 Acked Flush 0 Sent Done Done false
5218 2340 Acked Read 9 Sent Done Done false
5219 2341 Acked Write 1 Sent Done Done false
5220 2342 Acked Read 4 Sent Done Done false
5221 2343 Acked Write 1 Sent Done Done false
5222 2344 Acked Write 9 Sent Done Done false
5223 2345 Acked Read 6 Sent Done Done false
5224 2346 Acked Read 9 Sent Done Done false
5225 2347 Acked Flush 0 Sent Done Done false
5226 2348 Acked Read 5 Sent Done Done false
5227 2349 Acked Read 4 Sent Done Done false
5228 2350 Acked Write 6 Sent Done Done false
5229 2351 Acked Read 5 Sent Done Done false
5230 2352 Acked Write 6 Sent Done Done false
5231 2353 Acked Read 10 Sent Done Done false
5232 2354 Acked Read 2 Sent Done Done false
5233 2355 Acked Write 1 Sent Done Done false
5234 2356 Acked Write 1 Sent Done Done false
5235 2357 Acked Read 1 Sent Done Done false
5236 2358 Acked Read 1 Sent Done Done false
5237 2359 Acked Write 4 Sent Done Done false
5238 2360 Acked Flush 0 Sent Done Done false
5239 2361 Acked Flush 0 Sent Done Done false
5240 2362 Acked Read 3 Sent Done Done false
5241 2363 Acked Write 1 Sent Done Done false
5242 2364 Acked Read 9 Sent Done Done false
5243 2365 Acked Read 9 Sent Done Done false
5244 2366 Acked Write 2 Sent Done Done false
5245 2367 Acked Write 1 Sent Done Done false
5246 2368 Acked Write 9 Sent Done Done false
5247 2369 Acked Read 3 Sent Done Done false
5248 2370 Acked Read 5 Sent Done Done false
5249 2371 Acked Read 6 Sent Done Done false
5250 2372 Acked Read 6 Sent Done Done false
5251 2373 Acked Write 4 Sent Done Done false
5252 2374 Acked Write 5 Sent Done Done false
5253 2375 Acked Write 4 Sent Done Done false
5254 2376 Acked Read 1 Sent Done Done false
5255 2377 Acked Read 4 Sent Done Done false
5256 2378 Acked Read 4 Sent Done Done false
5257 2379 Acked Read 2 Sent Done Done false
5258 2380 Acked Read 8 Sent Done Done false
5259 2381 Acked Read 9 Sent Done Done false
5260 2382 Acked Read 1 Sent Done Done false
5261 2383 Acked Write 6 Sent Done Done false
5262 2384 Acked Write 9 Sent Done Done false
5263 2385 Acked Read 5 Sent Done Done false
5264 2386 Acked Write 2 Sent Done Done false
5265 2387 Acked Read 4 Sent Done Done false
5266 2388 Acked Read 9 Sent Done Done false
5267 2389 Acked Write 5 Sent Done Done false
5268 2390 Acked Read 3 Sent Done Done false
5269 2391 Acked Read 9 Sent Done Done false
5270 2392 Acked Flush 0 Sent Done Done false
5271 2393 Acked Write 1 Sent Done Done false
5272 2394 Acked Read 2 Sent Done Done false
5273 2395 Acked Flush 0 Sent Done Done false
5274 2396 Acked Flush 0 Sent Done Done false
5275 2397 Acked Read 4 Sent Done Done false
5276 2398 Acked Read 1 Sent Done Done false
5277 2399 Acked Read 2 Sent Done Done false
5278 2400 Acked Flush 0 Sent Done Done false
5279 2401 Acked Write 6 Sent Done Done false
5280 2402 Acked Read 9 Sent Done Done false
5281 2403 Acked Write 8 Sent Done Done false
5282 2404 Acked Read 3 Sent Done Done false
5283 2405 Acked Write 4 Sent Done Done false
5284 2406 Acked Write 2 Sent Done Done false
5285 2407 Acked Read 3 Sent Done Done false
5286 2408 Acked Read 7 Sent Done Done false
5287 2409 Acked Read 1 Sent Done Done false
5288 2410 Acked Write 6 Sent Done Done false
5289 2411 Acked Read 3 Sent Done Done false
5290 2412 Acked Read 2 Sent Done Done false
5291 2413 Acked Write 8 Sent Done Done false
5292 2414 Acked Read 7 Sent Done Done false
5293 2415 Acked Flush 0 Sent Done Done false
5294 2416 Acked Flush 0 Sent Done Done false
5295 2417 Acked Write 7 Sent Done Done false
5296 2418 Acked Write 2 Sent Done Done false
5297 2419 Acked Read 8 Sent Done Done false
5298 2420 Acked Read 6 Sent Done Done false
5299 2421 Acked Read 1 Sent Done Done false
5300 2422 Acked Flush 0 Sent Done Done false
5301 2423 Acked Flush 0 Sent Done Done false
5302 2424 Acked Write 3 Sent Done Done false
5303 2425 Acked Write 8 Sent Done Done false
5304 2426 Acked Flush 0 Sent Done Done false
5305 2427 Acked Write 2 Sent Done Done false
5306 2428 Acked Write 4 Sent Done Done false
5307 2429 Acked Read 6 Sent Done Done false
5308 2430 Acked Flush 0 Sent Done Done false
5309 2431 Acked Write 4 Sent Done Done false
5310 2432 Acked Read 2 Sent Done Done false
5311 2433 Acked Read 1 Sent Done Done false
5312 2434 Acked Read 2 Sent Done Done false
5313 2435 Acked Write 10 Sent Done Done false
5314 2436 Acked Write 4 Sent Done Done false
5315 2437 Acked Read 3 Sent Done Done false
5316 2438 Acked Read 3 Sent Done Done false
5317 2439 Acked Read 5 Sent Done Done false
5318 2440 Acked Read 6 Sent Done Done false
5319 2441 Acked Read 4 Sent Done Done false
5320 2442 Acked Read 7 Sent Done Done false
5321 2443 Acked Read 10 Sent Done Done false
5322 2444 Acked Read 3 Sent Done Done false
5323 2445 Acked Read 4 Sent Done Done false
5324 2446 Acked Write 9 Sent Done Done false
5325 2447 Acked Write 1 Sent Done Done false
5326 2448 Acked Read 6 Sent Done Done false
5327 2449 Acked Flush 0 Sent Done Done false
5328 2450 Acked Read 2 Sent Done Done false
5329 2451 Acked Read 6 Sent Done Done false
5330 2452 Acked Write 7 Sent Done Done false
5331 2453 Acked Write 7 Sent Done Done false
5332 2454 Acked Read 2 Sent Done Done false
5333 2455 Acked Write 7 Sent Done Done false
5334 2456 Acked Write 5 Sent Done Done false
5335 2457 Acked Write 7 Sent Done Done false
5336 2458 Acked Read 5 Sent Done Done false
5337 2459 Acked Write 6 Sent Done Done false
5338 2460 Acked Flush 0 Sent Done Done false
5339 2461 Acked Flush 0 Sent Done Done false
5340 2462 Acked Write 9 Sent Done Done false
5341 2463 Acked Read 10 Sent Done Done false
5342 2464 Acked Read 2 Sent Done Done false
5343 2465 Acked Flush 0 Sent Done Done false
5344 2466 Acked Read 10 Sent Done Done false
5345 2467 Acked Flush 0 Sent Done Done false
5346 2468 Acked Read 7 Sent Done Done false
5347 2469 Acked Write 9 Sent Done Done false
5348 2470 Acked Flush 0 Sent Done Done false
5349 2471 Acked Write 7 Sent Done Done false
5350 2472 Acked Write 7 Sent Done Done false
5351 2473 Acked Write 4 Sent Done Done false
5352 2474 Acked Flush 0 Sent Done Done false
5353 2475 Acked Read 5 Sent Done Done false
5354 2476 Acked Read 8 Sent Done Done false
5355 2477 Acked Read 3 Sent Done Done false
5356 2478 Acked Read 4 Sent Done Done false
5357 2479 Acked Read 6 Sent Done Done false
5358 2480 Acked Read 7 Sent Done Done false
5359 2481 Acked Read 4 Sent Done Done false
5360 2482 Acked Write 10 Sent Done Done false
5361 2483 Acked Read 8 Sent Done Done false
5362 2484 Acked Write 4 Sent Done Done false
5363 2485 Acked Flush 0 Sent Done Done false
5364 2486 Acked Write 1 Sent Done Done false
5365 2487 Acked Write 4 Sent Done Done false
5366 2488 Acked Flush 0 Sent Done Done false
5367 2489 Acked Read 4 Sent Done Done false
5368 2490 Acked Write 6 Sent Done Done false
5369 2491 Acked Read 2 Sent Done Done false
5370 2492 Acked Read 5 Sent Done Done false
5371 2493 Acked Write 4 Sent Done Done false
5372 2494 Acked Read 10 Sent Done Done false
5373 2495 Acked Read 1 Sent Done Done false
5374 2496 Acked Read 5 Sent Done Done false
5375 2497 Acked Write 7 Sent Done Done false
5376 2498 Acked Write 3 Sent Done Done false
5377 2499 Acked Write 2 Sent Done Done false
5378 2500 Acked Read 10 Sent Done Done false
5379 2501 Acked Write 10 Sent Done Done false
5380 2502 Acked Read 5 Sent Done Done false
5381 2503 Acked Write 7 Sent Done Done false
5382 2504 Acked Flush 0 Sent Done Done false
5383 2505 Acked Write 5 Sent Done Done false
5384 2506 Acked Read 3 Sent Done Done false
5385 2507 Acked Write 5 Sent Done Done false
5386 2508 Acked Write 4 Sent Done Done false
5387 STATES DS:0 DS:1 DS:2 TOTAL
5388 Sent 301 0 0 301
5389 Done 0 301 301 602
5390 Skipped 0 0 0 0
5391 Error 0 0 0 0
5392 Last Flush: 2207 2504 2504
5393 Downstairs last five completed: 2207 2206 2205 2204 2203
5394 Upstairs last five completed: 2508 2507 2506 2505 2504
5395 CLIENT: Up:0 ds:301 act:2
5396 [8810] got start action
5397 [8810] I am going to restart
5398 Starting downstairs at port 8810
5399 Make output file at "/tmp/dsc/downstairs-8810.txt"
5400 [8810][0] reports Starting
5401 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1205)
5402 [8810][0] reports Running
5403 ----------------------------------------------------------------
5404 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5405 States: Offline Active Active
5406 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5407 2208 Acked Write 1 Sent Done Done false
5408 2209 Acked Write 8 Sent Done Done false
5409 2210 Acked Read 9 Sent Done Done false
5410 2211 Acked Read 10 Sent Done Done false
5411 2212 Acked Write 7 Sent Done Done false
5412 2213 Acked Flush 0 Sent Done Done false
5413 2214 Acked Flush 0 Sent Done Done false
5414 2215 Acked Read 3 Sent Done Done false
5415 2216 Acked Read 7 Sent Done Done false
5416 2217 Acked Flush 0 Sent Done Done false
5417 2218 Acked Write 10 Sent Done Done false
5418 2219 Acked Read 7 Sent Done Done false
5419 2220 Acked Read 5 Sent Done Done false
5420 2221 Acked Read 2 Sent Done Done false
5421 2222 Acked Read 4 Sent Done Done false
5422 2223 Acked Read 8 Sent Done Done false
5423 2224 Acked Read 7 Sent Done Done false
5424 2225 Acked Read 6 Sent Done Done false
5425 2226 Acked Read 5 Sent Done Done false
5426 2227 Acked Read 1 Sent Done Done false
5427 2228 Acked Read 6 Sent Done Done false
5428 2229 Acked Read 4 Sent Done Done false
5429 2230 Acked Read 9 Sent Done Done false
5430 2231 Acked Flush 0 Sent Done Done false
5431 2232 Acked Read 2 Sent Done Done false
5432 2233 Acked Read 9 Sent Done Done false
5433 2234 Acked Write 4 Sent Done Done false
5434 2235 Acked Flush 0 Sent Done Done false
5435 2236 Acked Read 2 Sent Done Done false
5436 2237 Acked Flush 0 Sent Done Done false
5437 2238 Acked Write 8 Sent Done Done false
5438 2239 Acked Read 1 Sent Done Done false
5439 2240 Acked Write 5 Sent Done Done false
5440 2241 Acked Read 7 Sent Done Done false
5441 2242 Acked Read 7 Sent Done Done false
5442 2243 Acked Flush 0 Sent Done Done false
5443 2244 Acked Write 2 Sent Done Done false
5444 2245 Acked Read 3 Sent Done Done false
5445 2246 Acked Write 3 Sent Done Done false
5446 2247 Acked Read 3 Sent Done Done false
5447 2248 Acked Read 8 Sent Done Done false
5448 2249 Acked Read 6 Sent Done Done false
5449 2250 Acked Write 3 Sent Done Done false
5450 2251 Acked Write 5 Sent Done Done false
5451 2252 Acked Write 3 Sent Done Done false
5452 2253 Acked Flush 0 Sent Done Done false
5453 2254 Acked Write 5 Sent Done Done false
5454 2255 Acked Write 2 Sent Done Done false
5455 2256 Acked Read 3 Sent Done Done false
5456 2257 Acked Flush 0 Sent Done Done false
5457 2258 Acked Write 7 Sent Done Done false
5458 2259 Acked Flush 0 Sent Done Done false
5459 2260 Acked Write 8 Sent Done Done false
5460 2261 Acked Read 7 Sent Done Done false
5461 2262 Acked Write 5 Sent Done Done false
5462 2263 Acked Write 8 Sent Done Done false
5463 2264 Acked Write 6 Sent Done Done false
5464 2265 Acked Read 4 Sent Done Done false
5465 2266 Acked Write 4 Sent Done Done false
5466 2267 Acked Read 2 Sent Done Done false
5467 2268 Acked Read 4 Sent Done Done false
5468 2269 Acked Read 1 Sent Done Done false
5469 2270 Acked Read 6 Sent Done Done false
5470 2271 Acked Read 3 Sent Done Done false
5471 2272 Acked Read 9 Sent Done Done false
5472 2273 Acked Read 6 Sent Done Done false
5473 2274 Acked Read 8 Sent Done Done false
5474 2275 Acked Write 3 Sent Done Done false
5475 2276 Acked Read 1 Sent Done Done false
5476 2277 Acked Read 8 Sent Done Done false
5477 2278 Acked Write 7 Sent Done Done false
5478 2279 Acked Write 10 Sent Done Done false
5479 2280 Acked Read 7 Sent Done Done false
5480 2281 Acked Write 6 Sent Done Done false
5481 2282 Acked Read 7 Sent Done Done false
5482 2283 Acked Write 10 Sent Done Done false
5483 2284 Acked Read 1 Sent Done Done false
5484 2285 Acked Flush 0 Sent Done Done false
5485 2286 Acked Read 8 Sent Done Done false
5486 2287 Acked Read 1 Sent Done Done false
5487 2288 Acked Flush 0 Sent Done Done false
5488 2289 Acked Read 4 Sent Done Done false
5489 2290 Acked Read 7 Sent Done Done false
5490 2291 Acked Flush 0 Sent Done Done false
5491 2292 Acked Write 9 Sent Done Done false
5492 2293 Acked Write 3 Sent Done Done false
5493 2294 Acked Read 4 Sent Done Done false
5494 2295 Acked Read 9 Sent Done Done false
5495 2296 Acked Read 4 Sent Done Done false
5496 2297 Acked Read 7 Sent Done Done false
5497 2298 Acked Read 1 Sent Done Done false
5498 2299 Acked Flush 0 Sent Done Done false
5499 2300 Acked Write 9 Sent Done Done false
5500 2301 Acked Write 10 Sent Done Done false
5501 2302 Acked Write 9 Sent Done Done false
5502 2303 Acked Read 5 Sent Done Done false
5503 2304 Acked Read 10 Sent Done Done false
5504 2305 Acked Read 5 Sent Done Done false
5505 2306 Acked Read 3 Sent Done Done false
5506 2307 Acked Write 9 Sent Done Done false
5507 2308 Acked Read 8 Sent Done Done false
5508 2309 Acked Flush 0 Sent Done Done false
5509 2310 Acked Flush 0 Sent Done Done false
5510 2311 Acked Write 1 Sent Done Done false
5511 2312 Acked Read 1 Sent Done Done false
5512 2313 Acked Write 4 Sent Done Done false
5513 2314 Acked Flush 0 Sent Done Done false
5514 2315 Acked Read 6 Sent Done Done false
5515 2316 Acked Read 10 Sent Done Done false
5516 2317 Acked Read 6 Sent Done Done false
5517 2318 Acked Write 5 Sent Done Done false
5518 2319 Acked Write 7 Sent Done Done false
5519 2320 Acked Read 4 Sent Done Done false
5520 2321 Acked Write 4 Sent Done Done false
5521 2322 Acked Write 2 Sent Done Done false
5522 2323 Acked Read 7 Sent Done Done false
5523 2324 Acked Read 1 Sent Done Done false
5524 2325 Acked Read 8 Sent Done Done false
5525 2326 Acked Read 5 Sent Done Done false
5526 2327 Acked Flush 0 Sent Done Done false
5527 2328 Acked Write 9 Sent Done Done false
5528 2329 Acked Read 4 Sent Done Done false
5529 2330 Acked Write 9 Sent Done Done false
5530 2331 Acked Write 10 Sent Done Done false
5531 2332 Acked Read 7 Sent Done Done false
5532 2333 Acked Read 7 Sent Done Done false
5533 2334 Acked Read 5 Sent Done Done false
5534 2335 Acked Read 3 Sent Done Done false
5535 2336 Acked Read 10 Sent Done Done false
5536 2337 Acked Flush 0 Sent Done Done false
5537 2338 Acked Write 6 Sent Done Done false
5538 2339 Acked Flush 0 Sent Done Done false
5539 2340 Acked Read 9 Sent Done Done false
5540 2341 Acked Write 1 Sent Done Done false
5541 2342 Acked Read 4 Sent Done Done false
5542 2343 Acked Write 1 Sent Done Done false
5543 2344 Acked Write 9 Sent Done Done false
5544 2345 Acked Read 6 Sent Done Done false
5545 2346 Acked Read 9 Sent Done Done false
5546 2347 Acked Flush 0 Sent Done Done false
5547 2348 Acked Read 5 Sent Done Done false
5548 2349 Acked Read 4 Sent Done Done false
5549 2350 Acked Write 6 Sent Done Done false
5550 2351 Acked Read 5 Sent Done Done false
5551 2352 Acked Write 6 Sent Done Done false
5552 2353 Acked Read 10 Sent Done Done false
5553 2354 Acked Read 2 Sent Done Done false
5554 2355 Acked Write 1 Sent Done Done false
5555 2356 Acked Write 1 Sent Done Done false
5556 2357 Acked Read 1 Sent Done Done false
5557 2358 Acked Read 1 Sent Done Done false
5558 2359 Acked Write 4 Sent Done Done false
5559 2360 Acked Flush 0 Sent Done Done false
5560 2361 Acked Flush 0 Sent Done Done false
5561 2362 Acked Read 3 Sent Done Done false
5562 2363 Acked Write 1 Sent Done Done false
5563 2364 Acked Read 9 Sent Done Done false
5564 2365 Acked Read 9 Sent Done Done false
5565 2366 Acked Write 2 Sent Done Done false
5566 2367 Acked Write 1 Sent Done Done false
5567 2368 Acked Write 9 Sent Done Done false
5568 2369 Acked Read 3 Sent Done Done false
5569 2370 Acked Read 5 Sent Done Done false
5570 2371 Acked Read 6 Sent Done Done false
5571 2372 Acked Read 6 Sent Done Done false
5572 2373 Acked Write 4 Sent Done Done false
5573 2374 Acked Write 5 Sent Done Done false
5574 2375 Acked Write 4 Sent Done Done false
5575 2376 Acked Read 1 Sent Done Done false
5576 2377 Acked Read 4 Sent Done Done false
5577 2378 Acked Read 4 Sent Done Done false
5578 2379 Acked Read 2 Sent Done Done false
5579 2380 Acked Read 8 Sent Done Done false
5580 2381 Acked Read 9 Sent Done Done false
5581 2382 Acked Read 1 Sent Done Done false
5582 2383 Acked Write 6 Sent Done Done false
5583 2384 Acked Write 9 Sent Done Done false
5584 2385 Acked Read 5 Sent Done Done false
5585 2386 Acked Write 2 Sent Done Done false
5586 2387 Acked Read 4 Sent Done Done false
5587 2388 Acked Read 9 Sent Done Done false
5588 2389 Acked Write 5 Sent Done Done false
5589 2390 Acked Read 3 Sent Done Done false
5590 2391 Acked Read 9 Sent Done Done false
5591 2392 Acked Flush 0 Sent Done Done false
5592 2393 Acked Write 1 Sent Done Done false
5593 2394 Acked Read 2 Sent Done Done false
5594 2395 Acked Flush 0 Sent Done Done false
5595 2396 Acked Flush 0 Sent Done Done false
5596 2397 Acked Read 4 Sent Done Done false
5597 2398 Acked Read 1 Sent Done Done false
5598 2399 Acked Read 2 Sent Done Done false
5599 2400 Acked Flush 0 Sent Done Done false
5600 2401 Acked Write 6 Sent Done Done false
5601 2402 Acked Read 9 Sent Done Done false
5602 2403 Acked Write 8 Sent Done Done false
5603 2404 Acked Read 3 Sent Done Done false
5604 2405 Acked Write 4 Sent Done Done false
5605 2406 Acked Write 2 Sent Done Done false
5606 2407 Acked Read 3 Sent Done Done false
5607 2408 Acked Read 7 Sent Done Done false
5608 2409 Acked Read 1 Sent Done Done false
5609 2410 Acked Write 6 Sent Done Done false
5610 2411 Acked Read 3 Sent Done Done false
5611 2412 Acked Read 2 Sent Done Done false
5612 2413 Acked Write 8 Sent Done Done false
5613 2414 Acked Read 7 Sent Done Done false
5614 2415 Acked Flush 0 Sent Done Done false
5615 2416 Acked Flush 0 Sent Done Done false
5616 2417 Acked Write 7 Sent Done Done false
5617 2418 Acked Write 2 Sent Done Done false
5618 2419 Acked Read 8 Sent Done Done false
5619 2420 Acked Read 6 Sent Done Done false
5620 2421 Acked Read 1 Sent Done Done false
5621 2422 Acked Flush 0 Sent Done Done false
5622 2423 Acked Flush 0 Sent Done Done false
5623 2424 Acked Write 3 Sent Done Done false
5624 2425 Acked Write 8 Sent Done Done false
5625 2426 Acked Flush 0 Sent Done Done false
5626 2427 Acked Write 2 Sent Done Done false
5627 2428 Acked Write 4 Sent Done Done false
5628 2429 Acked Read 6 Sent Done Done false
5629 2430 Acked Flush 0 Sent Done Done false
5630 2431 Acked Write 4 Sent Done Done false
5631 2432 Acked Read 2 Sent Done Done false
5632 2433 Acked Read 1 Sent Done Done false
5633 2434 Acked Read 2 Sent Done Done false
5634 2435 Acked Write 10 Sent Done Done false
5635 2436 Acked Write 4 Sent Done Done false
5636 2437 Acked Read 3 Sent Done Done false
5637 2438 Acked Read 3 Sent Done Done false
5638 2439 Acked Read 5 Sent Done Done false
5639 2440 Acked Read 6 Sent Done Done false
5640 2441 Acked Read 4 Sent Done Done false
5641 2442 Acked Read 7 Sent Done Done false
5642 2443 Acked Read 10 Sent Done Done false
5643 2444 Acked Read 3 Sent Done Done false
5644 2445 Acked Read 4 Sent Done Done false
5645 2446 Acked Write 9 Sent Done Done false
5646 2447 Acked Write 1 Sent Done Done false
5647 2448 Acked Read 6 Sent Done Done false
5648 2449 Acked Flush 0 Sent Done Done false
5649 2450 Acked Read 2 Sent Done Done false
5650 2451 Acked Read 6 Sent Done Done false
5651 2452 Acked Write 7 Sent Done Done false
5652 2453 Acked Write 7 Sent Done Done false
5653 2454 Acked Read 2 Sent Done Done false
5654 2455 Acked Write 7 Sent Done Done false
5655 2456 Acked Write 5 Sent Done Done false
5656 2457 Acked Write 7 Sent Done Done false
5657 2458 Acked Read 5 Sent Done Done false
5658 2459 Acked Write 6 Sent Done Done false
5659 2460 Acked Flush 0 Sent Done Done false
5660 2461 Acked Flush 0 Sent Done Done false
5661 2462 Acked Write 9 Sent Done Done false
5662 2463 Acked Read 10 Sent Done Done false
5663 2464 Acked Read 2 Sent Done Done false
5664 2465 Acked Flush 0 Sent Done Done false
5665 2466 Acked Read 10 Sent Done Done false
5666 2467 Acked Flush 0 Sent Done Done false
5667 2468 Acked Read 7 Sent Done Done false
5668 2469 Acked Write 9 Sent Done Done false
5669 2470 Acked Flush 0 Sent Done Done false
5670 2471 Acked Write 7 Sent Done Done false
5671 2472 Acked Write 7 Sent Done Done false
5672 2473 Acked Write 4 Sent Done Done false
5673 2474 Acked Flush 0 Sent Done Done false
5674 2475 Acked Read 5 Sent Done Done false
5675 2476 Acked Read 8 Sent Done Done false
5676 2477 Acked Read 3 Sent Done Done false
5677 2478 Acked Read 4 Sent Done Done false
5678 2479 Acked Read 6 Sent Done Done false
5679 2480 Acked Read 7 Sent Done Done false
5680 2481 Acked Read 4 Sent Done Done false
5681 2482 Acked Write 10 Sent Done Done false
5682 2483 Acked Read 8 Sent Done Done false
5683 2484 Acked Write 4 Sent Done Done false
5684 2485 Acked Flush 0 Sent Done Done false
5685 2486 Acked Write 1 Sent Done Done false
5686 2487 Acked Write 4 Sent Done Done false
5687 2488 Acked Flush 0 Sent Done Done false
5688 2489 Acked Read 4 Sent Done Done false
5689 2490 Acked Write 6 Sent Done Done false
5690 2491 Acked Read 2 Sent Done Done false
5691 2492 Acked Read 5 Sent Done Done false
5692 2493 Acked Write 4 Sent Done Done false
5693 2494 Acked Read 10 Sent Done Done false
5694 2495 Acked Read 1 Sent Done Done false
5695 2496 Acked Read 5 Sent Done Done false
5696 2497 Acked Write 7 Sent Done Done false
5697 2498 Acked Write 3 Sent Done Done false
5698 2499 Acked Write 2 Sent Done Done false
5699 2500 Acked Read 10 Sent Done Done false
5700 2501 Acked Write 10 Sent Done Done false
5701 2502 Acked Read 5 Sent Done Done false
5702 2503 Acked Write 7 Sent Done Done false
5703 2504 Acked Flush 0 Sent Done Done false
5704 2505 Acked Write 5 Sent Done Done false
5705 2506 Acked Read 3 Sent Done Done false
5706 2507 Acked Write 5 Sent Done Done false
5707 2508 Acked Write 4 Sent Done Done false
5708 2509 Acked Flush 0 Sent Done Done false
5709 STATES DS:0 DS:1 DS:2 TOTAL
5710 Sent 302 0 0 302
5711 Done 0 302 302 604
5712 Skipped 0 0 0 0
5713 Error 0 0 0 0
5714 Last Flush: 2207 2509 2509
5715 Downstairs last five completed: 2207 2206 2205 2204 2203
5716 Upstairs last five completed: 2509 2508 2507 2506 2505
5717 CLIENT: Up:0 ds:302 act:2
57182024-12-11T23:41:41.521ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57192024-12-11T23:41:41.521ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39283) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57202024-12-11T23:41:41.522ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57212024-12-11T23:41:41.523ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57222024-12-11T23:41:41.523ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57232024-12-11T23:41:41.523ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57242024-12-11T23:41:41.523ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57252024-12-11T23:41:41.523ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57262024-12-11T23:41:41.523ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57272024-12-11T23:41:41.523ZINFOcrucible: [0] client re-new 302 jobs since flush 2207 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
57282024-12-11T23:41:41.524ZINFOcrucible: [0] Replayed 302 jobs since flush: 2207 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
5729 ----------------------------------------------------------------
5730 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5731 Downstairs last five completed: 2509 2508 2507 2506 2505
5732 Upstairs last five completed: 2509 2508 2507 2506 2505
5733 CLIENT: Up:0 ds:0 act:3
5734 Replay: All jobs finished
5735 Main task has work to do, go find it
5736 got dsc Stop(2)
5737 stop 2
5738 Dec 11 23:41:43.765 INFO request completed, [8830] Got stop action so:false kr:false
5739 latency_us: 329, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 4eaf214b-9b18-41c1-837c-48e575ccca2b, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
57402024-12-11T23:41:43.766ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
5741 Dec 11 23:41:43.766 {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:41:43.766390182Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
5742 INFO {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:41:43.766436703Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
5743 request completed, latency_us:{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:41:43.766459244Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
57442024-12-11T23:41:43.766ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
5745 189, response_code: 200{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:41:43.766558797Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
5746 , uri: /state/cid/2, method: GET, req_id: e089d322-4cd9-44e3-b5c5-b45193cd0a3e, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
5747 [8830] Exited with: ExitStatus(unix_wait_status(9))
5748 [8830][2] reports Exit
5749 Dec 11 23:41:47.768 INFO request completed, latency_us: 225, response_code: 200, uri: /state/cid/2, method: GET, req_id: 666d23e1-878a-4bc0-97c3-a7dba7f212ca, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
5750 001/300 Read block 3063 len 28672
5751 002/300 Write block 2997 len 8192 data: 3 3
5752 003/300 Write block 2066 len 40960 data: 3 3 3 3 3 3 3 2 2 2
5753 004/300 Read block 417 len 36864
5754 005/300 Read block 3380 len 8192
5755 006/300 Read block 2990 len 36864
5756 007/300 Read block 171 len 12288
5757 008/300 Read block 1895 len 16384
5758 009/300 Read block 3658 len 16384
5759 010/300 Flush
5760 011/300 Read block 3726 len 24576
5761 012/300 Read block 2056 len 36864
5762 013/300 Write block 4838 len 40960 data: 2 2 3 3 3 3 3 3 3 2
5763 014/300 Write block 1310 len 32768 data: 2 3 3 3 2 2 2 3
5764 015/300 Flush
5765 016/300 Read block 1956 len 4096
5766 017/300 Write block 3904 len 36864 data: 2 2 2 2 2 2 2 2 2
5767 018/300 Read block 2369 len 20480
5768 019/300 Read block 4546 len 36864
5769 020/300 Write block 4386 len 40960 data: 2 2 2 2 3 3 2 2 2 2
5770 021/300 Read block 4892 len 40960
5771 022/300 Write block 1981 len 16384 data: 5 5 5 6
5772 023/300 Write block 736 len 36864 data: 2 2 2 2 2 2 2 2 2
5773 024/300 Write block 2426 len 8192 data: 3 3
5774 025/300 Write block 178 len 40960 data: 4 4 4 4 3 3 2 2 2 2
5775 026/300 Write block 1270 len 12288 data: 2 2 2
5776 027/300 Read block 4285 len 24576
5777 028/300 Read block 3147 len 32768
5778 029/300 Read block 1183 len 24576
5779 030/300 Read block 1068 len 24576
5780 031/300 Flush
5781 032/300 Write block 3045 len 36864 data: 3 3 2 2 2 2 2 2 2
5782 033/300 Write block 4701 len 28672 data: 2 2 2 2 2 2 2
5783 034/300 Read block 98 len 8192
5784 035/300 Write block 4288 len 8192 data: 2 2
5785 036/300 Write block 2429 len 12288 data: 2 2 2
5786 037/300 Write block 637 len 36864 data: 2 2 2 3 3 3 3 3 3
5787 038/300 Read block 3467 len 16384
5788 039/300 Read block 99 len 36864
5789 040/300 Read block 3332 len 40960
5790 041/300 Write block 2319 len 8192 data: 2 2
5791 042/300 Write block 137 len 36864 data: 2 3 3 3 3 3 3 3 2
5792 043/300 Write block 1684 len 40960 data: 3 2 2 2 2 2 2 2 2 3
5793 044/300 Write block 2894 len 40960 data: 2 2 2 3 5 5 5 5 5 3
5794 045/300 Write block 2265 len 4096 data: 3
5795 046/300 Read block 1575 len 8192
5796 047/300 Read block 3383 len 28672
5797 048/300 Write block 1073 len 8192 data: 3 3
5798 049/300 Read block 1199 len 36864
5799 050/300 Write block 3045 len 12288 data: 4 4 3
5800 051/300 Read block 170 len 24576
5801 052/300 Read block 2215 len 4096
5802 053/300 Read block 3266 len 24576
5803 054/300 Write block 4042 len 20480 data: 2 2 2 2 2
5804 055/300 Read block 2290 len 12288
5805 056/300 Write block 2001 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5806 057/300 Read block 241 len 8192
5807 058/300 Read block 3620 len 36864
5808 059/300 Flush
5809 060/300 Write block 4237 len 20480 data: 3 3 3 3 4
5810 061/300 Write block 1802 len 24576 data: 2 2 2 2 2 2
5811 062/300 Write block 4226 len 12288 data: 4 4 4
5812 063/300 Write block 2429 len 12288 data: 3 3 3
5813 064/300 Write block 1085 len 40960 data: 3 3 3 3 3 3 3 3 2 2
5814 065/300 Write block 4813 len 4096 data: 2
5815 066/300 Read block 2427 len 4096
5816 067/300 Write block 3105 len 20480 data: 3 2 3 3 3
5817 068/300 Read block 1199 len 16384
5818 069/300 Read block 4747 len 24576
5819 070/300 Read block 3594 len 24576
5820 071/300 Write block 3647 len 28672 data: 2 2 2 2 2 2 2
5821 072/300 Read block 3527 len 36864
5822 073/300 Write block 4047 len 32768 data: 3 3 3 3 3 3 3 3
5823 074/300 Flush
5824 075/300 Read block 3513 len 8192
5825 076/300 Write block 4597 len 16384 data: 3 3 3 3
5826 077/300 Write block 3311 len 4096 data: 3
5827 078/300 Write block 1635 len 12288 data: 2 2 2
5828 079/300 Write block 2271 len 8192 data: 6 5
5829 080/300 Read block 2087 len 4096
5830 081/300 Flush
5831 082/300 Write block 2241 len 32768 data: 3 3 3 3 3 2 2 2
5832 083/300 Read block 763 len 40960
5833 084/300 Read block 430 len 40960
5834 085/300 Flush
5835 086/300 Write block 3719 len 36864 data: 2 2 2 2 2 2 3 3 3
5836 087/300 Write block 1134 len 36864 data: 4 4 4 4 4 4 4 3 2
5837 088/300 Read block 4636 len 4096
5838 089/300 Write block 1001 len 8192 data: 2 2
5839 090/300 Read block 4916 len 28672
5840 091/300 Write block 324 len 8192 data: 3 3
5841 092/300 Write block 4109 len 20480 data: 2 2 3 3 3
5842 093/300 Read block 137 len 40960
5843 094/300 Read block 3159 len 8192
5844 095/300 Write block 1854 len 36864 data: 3 3 3 3 2 2 2 2 2
5845 096/300 Write block 64 len 24576 data: 3 3 2 2 2 2
5846 097/300 Write block 3028 len 36864 data: 3 3 2 2 2 2 2 2 2
5847 098/300 Read block 4613 len 8192
5848 099/300 Write block 1435 len 8192 data: 3 3
5849 100/300 Read block 854 len 28672
5850 101/300 Write block 3192 len 4096 data: 3
5851 102/300 Write block 4233 len 4096 data: 5
5852 103/300 Read block 4694 len 36864
5853 104/300 Read block 4126 len 32768
5854 105/300 Read block 1523 len 40960
5855 106/300 Write block 1385 len 4096 data: 2
5856 107/300 Read block 3520 len 8192
5857 108/300 Read block 1822 len 12288
5858 109/300 Read block 2050 len 4096
5859 110/300 Write block 3947 len 40960 data: 5 4 5 5 4 3 3 2 2 2
5860 111/300 Write block 4924 len 36864 data: 4 3 4 3 3 3 3 3 3
5861 112/300 Read block 4353 len 8192
5862 113/300 Read block 3140 len 32768
5863 114/300 Write block 3907 len 28672 data: 3 3 3 3 3 3 2
5864 115/300 Write block 3510 len 24576 data: 3 2 2 2 2 2
5865 116/300 Flush
5866 117/300 Read block 2565 len 40960
5867 118/300 Write block 3083 len 8192 data: 3 3
5868 119/300 Read block 2044 len 12288
5869 120/300 Write block 3093 len 16384 data: 4 4 3 3
5870 121/300 Write block 1234 len 4096 data: 3
5871 122/300 Read block 4062 len 36864
5872 123/300 Write block 3127 len 28672 data: 2 2 3 4 3 3 3
5873 124/300 Write block 4071 len 24576 data: 2 2 2 2 2 2
5874 125/300 Read block 3343 len 8192
5875 126/300 Write block 3158 len 28672 data: 3 3 3 3 3 3 3
5876 127/300 Write block 3960 len 12288 data: 4 4 4
5877 128/300 Read block 2828 len 36864
5878 129/300 Read block 763 len 20480
5879 130/300 Write block 3688 len 16384 data: 2 2 2 2
5880 131/300 Write block 4667 len 12288 data: 2 3 3
5881 132/300 Read block 32 len 16384
5882 133/300 Write block 4170 len 28672 data: 4 4 4 2 2 2 2
5883 134/300 Read block 1108 len 16384
5884 135/300 Read block 733 len 12288
5885 136/300 Read block 3025 len 28672
5886 137/300 Read block 684 len 36864
5887 138/300 Read block 260 len 16384
5888 139/300 Write block 4649 len 36864 data: 2 2 2 2 2 2 2 2 3
5889 140/300 Write block 3135 len 12288 data: 3 4 4
5890 141/300 Read block 1697 len 16384
5891 142/300 Read block 3380 len 28672
5892 143/300 Write block 202 len 24576 data: 2 2 3 2 2 2
5893 144/300 Write block 2418 len 8192 data: 2 2
5894 145/300 Read block 4173 len 32768
5895 146/300 Write block 1031 len 20480 data: 3 3 2 3 3
5896 147/300 Write block 1373 len 4096 data: 3
5897 148/300 Write block 4771 len 36864 data: 2 2 2 2 2 2 2 2 2
5898 149/300 Read block 2149 len 40960
5899 150/300 Write block 2291 len 24576 data: 3 3 3 2 2 2
5900 151/300 Write block 2460 len 32768 data: 2 2 2 2 3 3 3 4
5901 152/300 Flush
5902 153/300 Write block 82 len 36864 data: 3 3 3 4 3 3 3 3 3
5903 154/300 Read block 441 len 16384
5904 155/300 Read block 650 len 40960
5905 156/300 Read block 2755 len 40960
5906 157/300 Read block 3800 len 12288
5907 158/300 Flush
5908 159/300 Read block 3422 len 8192
5909 160/300 Read block 1146 len 20480
5910 161/300 Read block 1292 len 40960
5911 162/300 Read block 2352 len 40960
5912 163/300 Flush
5913 164/300 Write block 3994 len 36864 data: 2 2 2 2 2 2 3 3 4
5914 165/300 Write block 4348 len 4096 data: 3
5915 166/300 Write block 426 len 4096 data: 2
5916 167/300 Write block 2537 len 20480 data: 2 2 2 2 2
5917 168/300 Read block 2631 len 28672
5918 169/300 Read block 4251 len 20480
5919 170/300 Read block 1806 len 4096
5920 171/300 Write block 2958 len 24576 data: 3 3 2 3 3 3
5921 172/300 Read block 4167 len 40960
5922 173/300 Read block 1617 len 32768
5923 174/300 Read block 3054 len 36864
5924 175/300 Read block 2059 len 32768
5925 176/300 Flush
5926 177/300 Read block 3610 len 32768
5927 178/300 Read block 1701 len 16384
5928 179/300 Write block 844 len 16384 data: 2 2 2 3
5929 180/300 Write block 2864 len 24576 data: 2 2 2 2 2 2
5930 181/300 Read block 1925 len 32768
5931 182/300 Write block 1831 len 12288 data: 2 2 2
5932 183/300 Read block 3139 len 12288
5933 184/300 Read block 3671 len 40960
5934 185/300 Read block 2688 len 8192
5935 186/300 Read block 1854 len 28672
5936 187/300 Read block 1487 len 40960
5937 188/300 Read block 204 len 28672
5938 189/300 Write block 1467 len 16384 data: 2 2 2 2
5939 190/300 Write block 587 len 24576 data: 2 2 2 2 2 2
5940 191/300 Write block 408 len 32768 data: 2 2 2 2 2 2 2 2
5941 192/300 Write block 4994 len 4096 data: 3
5942 193/300 Read block 2092 len 24576
5943 194/300 Write block 2685 len 16384 data: 2 2 3 3
5944 195/300 Read block 3392 len 40960
5945 196/300 Write block 1516 len 16384 data: 2 2 2 2
5946 197/300 Read block 4723 len 24576
5947 198/300 Write block 964 len 8192 data: 2 2
5948 199/300 Flush
5949 200/300 Flush
5950 201/300 Write block 2574 len 40960 data: 4 5 5 5 5 4 3 4 4 4
5951 202/300 Read block 4276 len 20480
5952 203/300 Read block 3848 len 8192
5953 204/300 Read block 2622 len 28672
5954 205/300 Flush
5955 206/300 Read block 4387 len 4096
5956 207/300 Read block 1019 len 20480
5957 208/300 Write block 4719 len 24576 data: 2 2 2 2 2 2
5958 209/300 Write block 1275 len 16384 data: 3 3 3 3
5959 210/300 Write block 4353 len 28672 data: 2 2 2 2 2 2 2
5960 211/300 Read block 3708 len 32768
5961 212/300 Read block 3257 len 40960
5962 213/300 Flush
5963 214/300 Read block 813 len 8192
5964 215/300 Read block 2957 len 36864
5965 216/300 Write block 2444 len 24576 data: 3 3 3 2 2 2
5966 217/300 Write block 3539 len 36864 data: 2 2 2 2 2 2 2 2 2
5967 218/300 Flush
5968 219/300 Write block 298 len 32768 data: 2 2 2 2 2 2 2 2
5969 220/300 Read block 3821 len 32768
5970 221/300 Read block 4141 len 16384
5971 222/300 Read block 4189 len 32768
5972 223/300 Write block 516 len 8192 data: 3 3
5973 224/300 Read block 311 len 4096
5974 225/300 Write block 657 len 24576 data: 3 3 3 3 4 3
5975 226/300 Write block 3913 len 16384 data: 3 2 2 2
5976 227/300 Flush
5977 228/300 Read block 2045 len 24576
5978 229/300 Write block 571 len 40960 data: 2 2 3 3 3 3 3 3 3 2
5979 230/300 Read block 3676 len 40960
5980 231/300 Write block 3646 len 4096 data: 3
5981 232/300 Read block 266 len 28672
5982 233/300 Write block 639 len 4096 data: 3
5983 234/300 Write block 4961 len 24576 data: 3 3 2 2 2 2
5984 235/300 Write block 1773 len 12288 data: 2 2 2
5985 236/300 Read block 746 len 32768
5986 237/300 Read block 3650 len 20480
5987 238/300 Write block 1626 len 24576 data: 3 3 3 3 3 3
5988 239/300 Write block 410 len 36864 data: 3 3 3 3 3 3 2 2 2
5989 240/300 Write block 2830 len 16384 data: 4 4 4 4
5990 241/300 Read block 4986 len 16384
5991 242/300 Read block 1813 len 40960
5992 243/300 Read block 1095 len 20480
5993 244/300 Write block 3383 len 24576 data: 3 3 4 4 4 3
5994 245/300 Flush
5995 246/300 Read block 1660 len 4096
5996 247/300 Read block 2687 len 28672
5997 248/300 Read block 972 len 8192
5998 249/300 Write block 3262 len 8192 data: 2 2
5999 250/300 Flush
6000 251/300 Write block 1077 len 8192 data: 3 3
6001 252/300 Read block 911 len 20480
6002 253/300 Read block 214 len 28672
6003 254/300 Read block 1281 len 16384
6004 255/300 Read block 3537 len 12288
6005 256/300 Write block 3935 len 20480 data: 2 2 2 2 2
6006 257/300 Read block 4883 len 36864
6007 258/300 Write block 3155 len 8192 data: 3 3
6008 259/300 Write block 4902 len 16384 data: 3 3 4 4
6009 260/300 Write block 1248 len 8192 data: 4 2
6010 261/300 Write block 3413 len 40960 data: 2 3 3 3 3 4 4 4 3 3
6011 262/300 Write block 4515 len 28672 data: 2 3 2 3 4 5 5
6012 263/300 Write block 766 len 32768 data: 3 2 2 2 3 3 3 2
6013 264/300 Write block 3204 len 20480 data: 3 4 3 3 2
6014 265/300 Flush
6015 266/300 Write block 4281 len 32768 data: 2 2 2 2 2 2 2 3
6016 267/300 Read block 123 len 28672
6017 268/300 Write block 174 len 28672 data: 2 2 2 3 5 5 5
6018 269/300 Read block 1250 len 36864
6019 270/300 Write block 2098 len 32768 data: 4 4 3 2 2 2 2 3
6020 271/300 Write block 408 len 8192 data: 3 3
6021 272/300 Read block 3568 len 24576
6022 273/300 Write block 1161 len 32768 data: 3 3 3 3 3 3 3 2
6023 274/300 Read block 1159 len 28672
6024 275/300 Read block 584 len 4096
6025 276/300 Read block 1840 len 8192
6026 277/300 Write block 2942 len 12288 data: 2 3 3
6027 278/300 Write block 4467 len 36864 data: 2 2 3 3 3 3 2 2 2
6028 279/300 Write block 2723 len 8192 data: 5 6
6029 280/300 Write block 3362 len 16384 data: 2 2 3 3
6030 281/300 Write block 448 len 4096 data: 4
6031 282/300 Write block 2588 len 36864 data: 3 3 3 3 2 2 3 3 3
6032 283/300 Write block 3078 len 4096 data: 2
6033 284/300 Write block 1439 len 36864 data: 5 6 6 5 5 5 5 5 4
6034 285/300 Read block 348 len 12288
6035 286/300 Read block 300 len 24576
6036 287/300 Read block 3634 len 40960
6037 288/300 Flush
6038 289/300 Write block 4074 len 32768 data: 3 3 3 2 2 2 2 2
6039 290/300 Read block 3779 len 12288
6040 291/300 Write block 2748 len 12288 data: 3 3 3
6041 292/300 Write block 3118 len 32768 data: 3 3 3 3 3 3 3 3
6042 293/300 Read block 761 len 28672
6043 294/300 Read block 1451 len 20480
6044 295/300 Write block 2936 len 8192 data: 4 4
6045 296/300 Read block 4143 len 24576
6046 297/300 Read block 1501 len 24576
6047 298/300 Write block 2093 len 24576 data: 4 5 4 4 4 5
6048 299/300 Read block 76 len 4096
6049 300/300 Read block 1275 len 24576
6050 301/300 Write block 3780 len 40960 data: 2 3 3 3 3 3 2 2 2 2
6051 Main task has work to do, go find it
6052 got dsc Start(2)
6053 start 2
6054 Dec 11 23:41:48.004 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/2, method: POST, req_id: 0430b57a-79df-4360-b4d5-234cc6c44983, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
6055 Replay: started 2, returned:Ok(())
6056 ----------------------------------------------------------------
6057 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6058 States: Active Active Offline
6059 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6060 2510 Acked Read 7 Done Done Sent false
6061 2511 Acked Write 2 Done Done Sent false
6062 2512 Acked Write 10 Done Done Sent false
6063 2513 Acked Read 9 Done Done Sent false
6064 2514 Acked Read 2 Done Done Sent false
6065 2515 Acked Read 9 Done Done Sent false
6066 2516 Acked Read 3 Done Done Sent false
6067 2517 Acked Read 4 Done Done Sent false
6068 2518 Acked Read 4 Done Done Sent false
6069 2519 Acked Flush 0 Done Done Sent false
6070 2520 Acked Read 6 Done Done Sent false
6071 2521 Acked Read 9 Done Done Sent false
6072 2522 Acked Write 10 Done Done Sent false
6073 2523 Acked Write 8 Done Done Sent false
6074 2524 Acked Flush 0 Done Done Sent false
6075 2525 Acked Read 1 Done Done Sent false
6076 2526 Acked Write 9 Done Done Sent false
6077 2527 Acked Read 5 Done Done Sent false
6078 2528 Acked Read 9 Done Done Sent false
6079 2529 Acked Write 10 Done Done Sent false
6080 2530 Acked Read 10 Done Done Sent false
6081 2531 Acked Write 4 Done Done Sent false
6082 2532 Acked Write 9 Done Done Sent false
6083 2533 Acked Write 2 Done Done Sent false
6084 2534 Acked Write 10 Done Done Sent false
6085 2535 Acked Write 3 Done Done Sent false
6086 2536 Acked Read 6 Done Done Sent false
6087 2537 Acked Read 8 Done Done Sent false
6088 2538 Acked Read 6 Done Done Sent false
6089 2539 Acked Read 6 Done Done Sent false
6090 2540 Acked Flush 0 Done Done Sent false
6091 2541 Acked Write 9 Done Done Sent false
6092 2542 Acked Write 7 Done Done Sent false
6093 2543 Acked Read 2 Done Done Sent false
6094 2544 Acked Write 2 Done Done Sent false
6095 2545 Acked Write 3 Done Done Sent false
6096 2546 Acked Write 9 Done Done Sent false
6097 2547 Acked Read 4 Done Done Sent false
6098 2548 Acked Read 9 Done Done Sent false
6099 2549 Acked Read 10 Done Done Sent false
6100 2550 Acked Write 2 Done Done Sent false
6101 2551 Acked Write 9 Done Done Sent false
6102 2552 Acked Write 10 Done Done Sent false
6103 2553 Acked Write 10 Done Done Sent false
6104 2554 Acked Write 1 Done Done Sent false
6105 2555 Acked Read 2 Done Done Sent false
6106 2556 Acked Read 7 Done Done Sent false
6107 2557 Acked Write 2 Done Done Sent false
6108 2558 Acked Read 9 Done Done Sent false
6109 2559 Acked Write 3 Done Done Sent false
6110 2560 Acked Read 6 Done Done Sent false
6111 2561 Acked Read 1 Done Done Sent false
6112 2562 Acked Read 6 Done Done Sent false
6113 2563 Acked Write 5 Done Done Sent false
6114 2564 Acked Read 3 Done Done Sent false
6115 2565 Acked Write 10 Done Done Sent false
6116 2566 Acked Read 2 Done Done Sent false
6117 2567 Acked Read 9 Done Done Sent false
6118 2568 Acked Flush 0 Done Done Sent false
6119 2569 Acked Write 5 Done Done Sent false
6120 2570 Acked Write 6 Done Done Sent false
6121 2571 Acked Write 3 Done Done Sent false
6122 2572 Acked Write 3 Done Done Sent false
6123 2573 Acked Write 10 Done Done Sent false
6124 2574 Acked Write 1 Done Done Sent false
6125 2575 Acked Read 1 Done Done Sent false
6126 2576 Acked Write 5 Done Done Sent false
6127 2577 Acked Read 4 Done Done Sent false
6128 2578 Acked Read 6 Done Done Sent false
6129 2579 Acked Read 6 Done Done Sent false
6130 2580 Acked Write 7 Done Done Sent false
6131 2581 Acked Read 9 Done Done Sent false
6132 2582 Acked Write 8 Done Done Sent false
6133 2583 Acked Flush 0 Done Done Sent false
6134 2584 Acked Read 2 Done Done Sent false
6135 2585 Acked Write 4 Done Done Sent false
6136 2586 Acked Write 1 Done Done Sent false
6137 2587 Acked Write 3 Done Done Sent false
6138 2588 Acked Write 2 Done Done Sent false
6139 2589 Acked Read 1 Done Done Sent false
6140 2590 Acked Flush 0 Done Done Sent false
6141 2591 Acked Write 8 Done Done Sent false
6142 2592 Acked Read 10 Done Done Sent false
6143 2593 Acked Read 10 Done Done Sent false
6144 2594 Acked Flush 0 Done Done Sent false
6145 2595 Acked Write 9 Done Done Sent false
6146 2596 Acked Write 9 Done Done Sent false
6147 2597 Acked Read 1 Done Done Sent false
6148 2598 Acked Write 2 Done Done Sent false
6149 2599 Acked Read 7 Done Done Sent false
6150 2600 Acked Write 2 Done Done Sent false
6151 2601 Acked Write 5 Done Done Sent false
6152 2602 Acked Read 10 Done Done Sent false
6153 2603 Acked Read 2 Done Done Sent false
6154 2604 Acked Write 9 Done Done Sent false
6155 2605 Acked Write 6 Done Done Sent false
6156 2606 Acked Write 9 Done Done Sent false
6157 2607 Acked Read 2 Done Done Sent false
6158 2608 Acked Write 2 Done Done Sent false
6159 2609 Acked Read 7 Done Done Sent false
6160 2610 Acked Write 1 Done Done Sent false
6161 2611 Acked Write 1 Done Done Sent false
6162 2612 Acked Read 9 Done Done Sent false
6163 2613 Acked Read 8 Done Done Sent false
6164 2614 Acked Read 10 Done Done Sent false
6165 2615 Acked Write 1 Done Done Sent false
6166 2616 Acked Read 2 Done Done Sent false
6167 2617 Acked Read 3 Done Done Sent false
6168 2618 Acked Read 1 Done Done Sent false
6169 2619 Acked Write 10 Done Done Sent false
6170 2620 Acked Write 9 Done Done Sent false
6171 2621 Acked Read 2 Done Done Sent false
6172 2622 Acked Read 8 Done Done Sent false
6173 2623 Acked Write 7 Done Done Sent false
6174 2624 Acked Write 6 Done Done Sent false
6175 2625 Acked Flush 0 Done Done Sent false
6176 2626 Acked Read 10 Done Done Sent false
6177 2627 Acked Write 2 Done Done Sent false
6178 2628 Acked Read 3 Done Done Sent false
6179 2629 Acked Write 4 Done Done Sent false
6180 2630 Acked Write 1 Done Done Sent false
6181 2631 Acked Read 9 Done Done Sent false
6182 2632 Acked Write 7 Done Done Sent false
6183 2633 Acked Write 6 Done Done Sent false
6184 2634 Acked Read 2 Done Done Sent false
6185 2635 Acked Write 7 Done Done Sent false
6186 2636 Acked Write 3 Done Done Sent false
6187 2637 Acked Read 9 Done Done Sent false
6188 2638 Acked Read 5 Done Done Sent false
6189 2639 Acked Write 4 Done Done Sent false
6190 2640 Acked Write 3 Done Done Sent false
6191 2641 Acked Read 4 Done Done Sent false
6192 2642 Acked Write 7 Done Done Sent false
6193 2643 Acked Read 4 Done Done Sent false
6194 2644 Acked Read 3 Done Done Sent false
6195 2645 Acked Read 7 Done Done Sent false
6196 2646 Acked Read 9 Done Done Sent false
6197 2647 Acked Read 4 Done Done Sent false
6198 2648 Acked Write 9 Done Done Sent false
6199 2649 Acked Write 3 Done Done Sent false
6200 2650 Acked Read 4 Done Done Sent false
6201 2651 Acked Read 7 Done Done Sent false
6202 2652 Acked Write 6 Done Done Sent false
6203 2653 Acked Write 2 Done Done Sent false
6204 2654 Acked Read 8 Done Done Sent false
6205 2655 Acked Write 5 Done Done Sent false
6206 2656 Acked Write 1 Done Done Sent false
6207 2657 Acked Write 9 Done Done Sent false
6208 2658 Acked Read 10 Done Done Sent false
6209 2659 Acked Write 6 Done Done Sent false
6210 2660 Acked Write 8 Done Done Sent false
6211 2661 Acked Flush 0 Done Done Sent false
6212 2662 Acked Write 9 Done Done Sent false
6213 2663 Acked Read 4 Done Done Sent false
6214 2664 Acked Read 10 Done Done Sent false
6215 2665 Acked Read 10 Done Done Sent false
6216 2666 Acked Read 3 Done Done Sent false
6217 2667 Acked Flush 0 Done Done Sent false
6218 2668 Acked Read 2 Done Done Sent false
6219 2669 Acked Read 5 Done Done Sent false
6220 2670 Acked Read 10 Done Done Sent false
6221 2671 Acked Read 10 Done Done Sent false
6222 2672 Acked Flush 0 Done Done Sent false
6223 2673 Acked Write 9 Done Done Sent false
6224 2674 Acked Write 1 Done Done Sent false
6225 2675 Acked Write 1 Done Done Sent false
6226 2676 Acked Write 5 Done Done Sent false
6227 2677 Acked Read 7 Done Done Sent false
6228 2678 Acked Read 5 Done Done Sent false
6229 2679 Acked Read 1 Done Done Sent false
6230 2680 Acked Write 6 Done Done Sent false
6231 2681 Acked Read 10 Done Done Sent false
6232 2682 Acked Read 8 Done Done Sent false
6233 2683 Acked Read 9 Done Done Sent false
6234 2684 Acked Read 8 Done Done Sent false
6235 2685 Acked Flush 0 Done Done Sent false
6236 2686 Acked Read 8 Done Done Sent false
6237 2687 Acked Read 4 Done Done Sent false
6238 2688 Acked Write 4 Done Done Sent false
6239 2689 Acked Write 6 Done Done Sent false
6240 2690 Acked Read 8 Done Done Sent false
6241 2691 Acked Write 3 Done Done Sent false
6242 2692 Acked Read 3 Done Done Sent false
6243 2693 Acked Read 10 Done Done Sent false
6244 2694 Acked Read 2 Done Done Sent false
6245 2695 Acked Read 7 Done Done Sent false
6246 2696 Acked Read 10 Done Done Sent false
6247 2697 Acked Read 7 Done Done Sent false
6248 2698 Acked Write 4 Done Done Sent false
6249 2699 Acked Write 6 Done Done Sent false
6250 2700 Acked Write 8 Done Done Sent false
6251 2701 Acked Write 1 Done Done Sent false
6252 2702 Acked Read 6 Done Done Sent false
6253 2703 Acked Write 4 Done Done Sent false
6254 2704 Acked Read 10 Done Done Sent false
6255 2705 Acked Write 4 Done Done Sent false
6256 2706 Acked Read 6 Done Done Sent false
6257 2707 Acked Write 2 Done Done Sent false
6258 2708 Acked Flush 0 Done Done Sent false
6259 2709 Acked Flush 0 Done Done Sent false
6260 2710 Acked Write 10 Done Done Sent false
6261 2711 Acked Read 5 Done Done Sent false
6262 2712 Acked Read 2 Done Done Sent false
6263 2713 Acked Read 7 Done Done Sent false
6264 2714 Acked Flush 0 Done Done Sent false
6265 2715 Acked Read 1 Done Done Sent false
6266 2716 Acked Read 5 Done Done Sent false
6267 2717 Acked Write 6 Done Done Sent false
6268 2718 Acked Write 4 Done Done Sent false
6269 2719 Acked Write 7 Done Done Sent false
6270 2720 Acked Read 8 Done Done Sent false
6271 2721 Acked Read 10 Done Done Sent false
6272 2722 Acked Flush 0 Done Done Sent false
6273 2723 Acked Read 2 Done Done Sent false
6274 2724 Acked Read 9 Done Done Sent false
6275 2725 Acked Write 6 Done Done Sent false
6276 2726 Acked Write 9 Done Done Sent false
6277 2727 Acked Flush 0 Done Done Sent false
6278 2728 Acked Write 8 Done Done Sent false
6279 2729 Acked Read 8 Done Done Sent false
6280 2730 Acked Read 4 Done Done Sent false
6281 2731 Acked Read 8 Done Done Sent false
6282 2732 Acked Write 2 Done Done Sent false
6283 2733 Acked Read 1 Done Done Sent false
6284 2734 Acked Write 6 Done Done Sent false
6285 2735 Acked Write 4 Done Done Sent false
6286 2736 Acked Flush 0 Done Done Sent false
6287 2737 Acked Read 6 Done Done Sent false
6288 2738 Acked Write 10 Done Done Sent false
6289 2739 Acked Read 10 Done Done Sent false
6290 2740 Acked Write 1 Done Done Sent false
6291 2741 Acked Read 7 Done Done Sent false
6292 2742 Acked Write 1 Done Done Sent false
6293 2743 Acked Write 6 Done Done Sent false
6294 2744 Acked Write 3 Done Done Sent false
6295 2745 Acked Read 8 Done Done Sent false
6296 2746 Acked Read 5 Done Done Sent false
6297 2747 Acked Write 6 Done Done Sent false
6298 2748 Acked Write 9 Done Done Sent false
6299 2749 Acked Write 4 Done Done Sent false
6300 2750 Acked Read 4 Done Done Sent false
6301 2751 Acked Read 10 Done Done Sent false
6302 2752 Acked Read 5 Done Done Sent false
6303 2753 Acked Write 6 Done Done Sent false
6304 2754 Acked Flush 0 Done Done Sent false
6305 2755 Acked Read 1 Done Done Sent false
6306 2756 Acked Read 7 Done Done Sent false
6307 2757 Acked Read 2 Done Done Sent false
6308 2758 Acked Write 2 Done Done Sent false
6309 2759 Acked Flush 0 Done Done Sent false
6310 2760 Acked Write 2 Done Done Sent false
6311 2761 Acked Read 5 Done Done Sent false
6312 2762 Acked Read 7 Done Done Sent false
6313 2763 Acked Read 4 Done Done Sent false
6314 2764 Acked Read 3 Done Done Sent false
6315 2765 Acked Write 5 Done Done Sent false
6316 2766 Acked Read 9 Done Done Sent false
6317 2767 Acked Write 2 Done Done Sent false
6318 2768 Acked Write 4 Done Done Sent false
6319 2769 Acked Write 2 Done Done Sent false
6320 2770 Acked Write 10 Done Done Sent false
6321 2771 Acked Write 7 Done Done Sent false
6322 2772 Acked Write 8 Done Done Sent false
6323 2773 Acked Write 5 Done Done Sent false
6324 2774 Acked Flush 0 Done Done Sent false
6325 2775 Acked Write 8 Done Done Sent false
6326 2776 Acked Read 7 Done Done Sent false
6327 2777 Acked Write 7 Done Done Sent false
6328 2778 Acked Read 9 Done Done Sent false
6329 2779 Acked Write 8 Done Done Sent false
6330 2780 Acked Write 2 Done Done Sent false
6331 2781 Acked Read 6 Done Done Sent false
6332 2782 Acked Write 8 Done Done Sent false
6333 2783 Acked Read 7 Done Done Sent false
6334 2784 Acked Read 1 Done Done Sent false
6335 2785 Acked Read 2 Done Done Sent false
6336 2786 Acked Write 3 Done Done Sent false
6337 2787 Acked Write 9 Done Done Sent false
6338 2788 Acked Write 2 Done Done Sent false
6339 2789 Acked Write 4 Done Done Sent false
6340 2790 Acked Write 1 Done Done Sent false
6341 2791 Acked Write 9 Done Done Sent false
6342 2792 Acked Write 1 Done Done Sent false
6343 2793 Acked Write 9 Done Done Sent false
6344 2794 Acked Read 3 Done Done Sent false
6345 2795 Acked Read 6 Done Done Sent false
6346 2796 Acked Read 10 Done Done Sent false
6347 2797 Acked Flush 0 Done Done Sent false
6348 2798 Acked Write 8 Done Done Sent false
6349 2799 Acked Read 3 Done Done Sent false
6350 2800 Acked Write 3 Done Done Sent false
6351 2801 Acked Write 8 Done Done Sent false
6352 2802 Acked Read 7 Done Done Sent false
6353 2803 Acked Read 5 Done Done Sent false
6354 2804 Acked Write 2 Done Done Sent false
6355 2805 Acked Read 6 Done Done Sent false
6356 2806 Acked Read 6 Done Done Sent false
6357 2807 Acked Write 6 Done Done Sent false
6358 2808 Acked Read 1 Done Done Sent false
6359 2809 Acked Read 6 Done Done Sent false
6360 2810 Acked Write 10 Done Done Sent false
6361 STATES DS:0 DS:1 DS:2 TOTAL
6362 Sent 0 0 301 301
6363 Done 301 301 0 602
6364 Skipped 0 0 0 0
6365 Error 0 0 0 0
6366 Last Flush: 2797 2797 2509
6367 Downstairs last five completed: 2509 2508 2507 2506 2505
6368 Upstairs last five completed: 2810 2809 2808 2807 2806
6369 CLIENT: Up:0 ds:301 act:2
6370 [8830] got start action
6371 [8830] I am going to restart
6372 Starting downstairs at port 8830
6373 Make output file at "/tmp/dsc/downstairs-8830.txt"
6374 [8830][2] reports Starting
6375 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1206)
6376 [8830][2] reports Running
6377 ----------------------------------------------------------------
6378 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6379 States: Active Active Offline
6380 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6381 2510 Acked Read 7 Done Done Sent false
6382 2511 Acked Write 2 Done Done Sent false
6383 2512 Acked Write 10 Done Done Sent false
6384 2513 Acked Read 9 Done Done Sent false
6385 2514 Acked Read 2 Done Done Sent false
6386 2515 Acked Read 9 Done Done Sent false
6387 2516 Acked Read 3 Done Done Sent false
6388 2517 Acked Read 4 Done Done Sent false
6389 2518 Acked Read 4 Done Done Sent false
6390 2519 Acked Flush 0 Done Done Sent false
6391 2520 Acked Read 6 Done Done Sent false
6392 2521 Acked Read 9 Done Done Sent false
6393 2522 Acked Write 10 Done Done Sent false
6394 2523 Acked Write 8 Done Done Sent false
6395 2524 Acked Flush 0 Done Done Sent false
6396 2525 Acked Read 1 Done Done Sent false
6397 2526 Acked Write 9 Done Done Sent false
6398 2527 Acked Read 5 Done Done Sent false
6399 2528 Acked Read 9 Done Done Sent false
6400 2529 Acked Write 10 Done Done Sent false
6401 2530 Acked Read 10 Done Done Sent false
6402 2531 Acked Write 4 Done Done Sent false
6403 2532 Acked Write 9 Done Done Sent false
6404 2533 Acked Write 2 Done Done Sent false
6405 2534 Acked Write 10 Done Done Sent false
6406 2535 Acked Write 3 Done Done Sent false
6407 2536 Acked Read 6 Done Done Sent false
6408 2537 Acked Read 8 Done Done Sent false
6409 2538 Acked Read 6 Done Done Sent false
6410 2539 Acked Read 6 Done Done Sent false
6411 2540 Acked Flush 0 Done Done Sent false
6412 2541 Acked Write 9 Done Done Sent false
6413 2542 Acked Write 7 Done Done Sent false
6414 2543 Acked Read 2 Done Done Sent false
6415 2544 Acked Write 2 Done Done Sent false
6416 2545 Acked Write 3 Done Done Sent false
6417 2546 Acked Write 9 Done Done Sent false
6418 2547 Acked Read 4 Done Done Sent false
6419 2548 Acked Read 9 Done Done Sent false
6420 2549 Acked Read 10 Done Done Sent false
6421 2550 Acked Write 2 Done Done Sent false
6422 2551 Acked Write 9 Done Done Sent false
6423 2552 Acked Write 10 Done Done Sent false
6424 2553 Acked Write 10 Done Done Sent false
6425 2554 Acked Write 1 Done Done Sent false
6426 2555 Acked Read 2 Done Done Sent false
6427 2556 Acked Read 7 Done Done Sent false
6428 2557 Acked Write 2 Done Done Sent false
6429 2558 Acked Read 9 Done Done Sent false
6430 2559 Acked Write 3 Done Done Sent false
6431 2560 Acked Read 6 Done Done Sent false
6432 2561 Acked Read 1 Done Done Sent false
6433 2562 Acked Read 6 Done Done Sent false
6434 2563 Acked Write 5 Done Done Sent false
6435 2564 Acked Read 3 Done Done Sent false
6436 2565 Acked Write 10 Done Done Sent false
6437 2566 Acked Read 2 Done Done Sent false
6438 2567 Acked Read 9 Done Done Sent false
6439 2568 Acked Flush 0 Done Done Sent false
6440 2569 Acked Write 5 Done Done Sent false
6441 2570 Acked Write 6 Done Done Sent false
6442 2571 Acked Write 3 Done Done Sent false
6443 2572 Acked Write 3 Done Done Sent false
6444 2573 Acked Write 10 Done Done Sent false
6445 2574 Acked Write 1 Done Done Sent false
6446 2575 Acked Read 1 Done Done Sent false
6447 2576 Acked Write 5 Done Done Sent false
6448 2577 Acked Read 4 Done Done Sent false
6449 2578 Acked Read 6 Done Done Sent false
6450 2579 Acked Read 6 Done Done Sent false
6451 2580 Acked Write 7 Done Done Sent false
6452 2581 Acked Read 9 Done Done Sent false
6453 2582 Acked Write 8 Done Done Sent false
6454 2583 Acked Flush 0 Done Done Sent false
6455 2584 Acked Read 2 Done Done Sent false
6456 2585 Acked Write 4 Done Done Sent false
6457 2586 Acked Write 1 Done Done Sent false
6458 2587 Acked Write 3 Done Done Sent false
6459 2588 Acked Write 2 Done Done Sent false
6460 2589 Acked Read 1 Done Done Sent false
6461 2590 Acked Flush 0 Done Done Sent false
6462 2591 Acked Write 8 Done Done Sent false
6463 2592 Acked Read 10 Done Done Sent false
6464 2593 Acked Read 10 Done Done Sent false
6465 2594 Acked Flush 0 Done Done Sent false
6466 2595 Acked Write 9 Done Done Sent false
6467 2596 Acked Write 9 Done Done Sent false
6468 2597 Acked Read 1 Done Done Sent false
6469 2598 Acked Write 2 Done Done Sent false
6470 2599 Acked Read 7 Done Done Sent false
6471 2600 Acked Write 2 Done Done Sent false
6472 2601 Acked Write 5 Done Done Sent false
6473 2602 Acked Read 10 Done Done Sent false
6474 2603 Acked Read 2 Done Done Sent false
6475 2604 Acked Write 9 Done Done Sent false
6476 2605 Acked Write 6 Done Done Sent false
6477 2606 Acked Write 9 Done Done Sent false
6478 2607 Acked Read 2 Done Done Sent false
6479 2608 Acked Write 2 Done Done Sent false
6480 2609 Acked Read 7 Done Done Sent false
6481 2610 Acked Write 1 Done Done Sent false
6482 2611 Acked Write 1 Done Done Sent false
6483 2612 Acked Read 9 Done Done Sent false
6484 2613 Acked Read 8 Done Done Sent false
6485 2614 Acked Read 10 Done Done Sent false
6486 2615 Acked Write 1 Done Done Sent false
6487 2616 Acked Read 2 Done Done Sent false
6488 2617 Acked Read 3 Done Done Sent false
6489 2618 Acked Read 1 Done Done Sent false
6490 2619 Acked Write 10 Done Done Sent false
6491 2620 Acked Write 9 Done Done Sent false
6492 2621 Acked Read 2 Done Done Sent false
6493 2622 Acked Read 8 Done Done Sent false
6494 2623 Acked Write 7 Done Done Sent false
6495 2624 Acked Write 6 Done Done Sent false
6496 2625 Acked Flush 0 Done Done Sent false
6497 2626 Acked Read 10 Done Done Sent false
6498 2627 Acked Write 2 Done Done Sent false
6499 2628 Acked Read 3 Done Done Sent false
6500 2629 Acked Write 4 Done Done Sent false
6501 2630 Acked Write 1 Done Done Sent false
6502 2631 Acked Read 9 Done Done Sent false
6503 2632 Acked Write 7 Done Done Sent false
6504 2633 Acked Write 6 Done Done Sent false
6505 2634 Acked Read 2 Done Done Sent false
6506 2635 Acked Write 7 Done Done Sent false
6507 2636 Acked Write 3 Done Done Sent false
6508 2637 Acked Read 9 Done Done Sent false
6509 2638 Acked Read 5 Done Done Sent false
6510 2639 Acked Write 4 Done Done Sent false
6511 2640 Acked Write 3 Done Done Sent false
6512 2641 Acked Read 4 Done Done Sent false
6513 2642 Acked Write 7 Done Done Sent false
6514 2643 Acked Read 4 Done Done Sent false
6515 2644 Acked Read 3 Done Done Sent false
6516 2645 Acked Read 7 Done Done Sent false
6517 2646 Acked Read 9 Done Done Sent false
6518 2647 Acked Read 4 Done Done Sent false
6519 2648 Acked Write 9 Done Done Sent false
6520 2649 Acked Write 3 Done Done Sent false
6521 2650 Acked Read 4 Done Done Sent false
6522 2651 Acked Read 7 Done Done Sent false
6523 2652 Acked Write 6 Done Done Sent false
6524 2653 Acked Write 2 Done Done Sent false
6525 2654 Acked Read 8 Done Done Sent false
6526 2655 Acked Write 5 Done Done Sent false
6527 2656 Acked Write 1 Done Done Sent false
6528 2657 Acked Write 9 Done Done Sent false
6529 2658 Acked Read 10 Done Done Sent false
6530 2659 Acked Write 6 Done Done Sent false
6531 2660 Acked Write 8 Done Done Sent false
6532 2661 Acked Flush 0 Done Done Sent false
6533 2662 Acked Write 9 Done Done Sent false
6534 2663 Acked Read 4 Done Done Sent false
6535 2664 Acked Read 10 Done Done Sent false
6536 2665 Acked Read 10 Done Done Sent false
6537 2666 Acked Read 3 Done Done Sent false
6538 2667 Acked Flush 0 Done Done Sent false
6539 2668 Acked Read 2 Done Done Sent false
6540 2669 Acked Read 5 Done Done Sent false
6541 2670 Acked Read 10 Done Done Sent false
6542 2671 Acked Read 10 Done Done Sent false
6543 2672 Acked Flush 0 Done Done Sent false
6544 2673 Acked Write 9 Done Done Sent false
6545 2674 Acked Write 1 Done Done Sent false
6546 2675 Acked Write 1 Done Done Sent false
6547 2676 Acked Write 5 Done Done Sent false
6548 2677 Acked Read 7 Done Done Sent false
6549 2678 Acked Read 5 Done Done Sent false
6550 2679 Acked Read 1 Done Done Sent false
6551 2680 Acked Write 6 Done Done Sent false
6552 2681 Acked Read 10 Done Done Sent false
6553 2682 Acked Read 8 Done Done Sent false
6554 2683 Acked Read 9 Done Done Sent false
6555 2684 Acked Read 8 Done Done Sent false
6556 2685 Acked Flush 0 Done Done Sent false
6557 2686 Acked Read 8 Done Done Sent false
6558 2687 Acked Read 4 Done Done Sent false
6559 2688 Acked Write 4 Done Done Sent false
6560 2689 Acked Write 6 Done Done Sent false
6561 2690 Acked Read 8 Done Done Sent false
6562 2691 Acked Write 3 Done Done Sent false
6563 2692 Acked Read 3 Done Done Sent false
6564 2693 Acked Read 10 Done Done Sent false
6565 2694 Acked Read 2 Done Done Sent false
6566 2695 Acked Read 7 Done Done Sent false
6567 2696 Acked Read 10 Done Done Sent false
6568 2697 Acked Read 7 Done Done Sent false
6569 2698 Acked Write 4 Done Done Sent false
6570 2699 Acked Write 6 Done Done Sent false
6571 2700 Acked Write 8 Done Done Sent false
6572 2701 Acked Write 1 Done Done Sent false
6573 2702 Acked Read 6 Done Done Sent false
6574 2703 Acked Write 4 Done Done Sent false
6575 2704 Acked Read 10 Done Done Sent false
6576 2705 Acked Write 4 Done Done Sent false
6577 2706 Acked Read 6 Done Done Sent false
6578 2707 Acked Write 2 Done Done Sent false
6579 2708 Acked Flush 0 Done Done Sent false
6580 2709 Acked Flush 0 Done Done Sent false
6581 2710 Acked Write 10 Done Done Sent false
6582 2711 Acked Read 5 Done Done Sent false
6583 2712 Acked Read 2 Done Done Sent false
6584 2713 Acked Read 7 Done Done Sent false
6585 2714 Acked Flush 0 Done Done Sent false
6586 2715 Acked Read 1 Done Done Sent false
6587 2716 Acked Read 5 Done Done Sent false
6588 2717 Acked Write 6 Done Done Sent false
6589 2718 Acked Write 4 Done Done Sent false
6590 2719 Acked Write 7 Done Done Sent false
6591 2720 Acked Read 8 Done Done Sent false
6592 2721 Acked Read 10 Done Done Sent false
6593 2722 Acked Flush 0 Done Done Sent false
6594 2723 Acked Read 2 Done Done Sent false
6595 2724 Acked Read 9 Done Done Sent false
6596 2725 Acked Write 6 Done Done Sent false
6597 2726 Acked Write 9 Done Done Sent false
6598 2727 Acked Flush 0 Done Done Sent false
6599 2728 Acked Write 8 Done Done Sent false
6600 2729 Acked Read 8 Done Done Sent false
6601 2730 Acked Read 4 Done Done Sent false
6602 2731 Acked Read 8 Done Done Sent false
6603 2732 Acked Write 2 Done Done Sent false
6604 2733 Acked Read 1 Done Done Sent false
6605 2734 Acked Write 6 Done Done Sent false
6606 2735 Acked Write 4 Done Done Sent false
6607 2736 Acked Flush 0 Done Done Sent false
6608 2737 Acked Read 6 Done Done Sent false
6609 2738 Acked Write 10 Done Done Sent false
6610 2739 Acked Read 10 Done Done Sent false
6611 2740 Acked Write 1 Done Done Sent false
6612 2741 Acked Read 7 Done Done Sent false
6613 2742 Acked Write 1 Done Done Sent false
6614 2743 Acked Write 6 Done Done Sent false
6615 2744 Acked Write 3 Done Done Sent false
6616 2745 Acked Read 8 Done Done Sent false
6617 2746 Acked Read 5 Done Done Sent false
6618 2747 Acked Write 6 Done Done Sent false
6619 2748 Acked Write 9 Done Done Sent false
6620 2749 Acked Write 4 Done Done Sent false
6621 2750 Acked Read 4 Done Done Sent false
6622 2751 Acked Read 10 Done Done Sent false
6623 2752 Acked Read 5 Done Done Sent false
6624 2753 Acked Write 6 Done Done Sent false
6625 2754 Acked Flush 0 Done Done Sent false
6626 2755 Acked Read 1 Done Done Sent false
6627 2756 Acked Read 7 Done Done Sent false
6628 2757 Acked Read 2 Done Done Sent false
6629 2758 Acked Write 2 Done Done Sent false
6630 2759 Acked Flush 0 Done Done Sent false
6631 2760 Acked Write 2 Done Done Sent false
6632 2761 Acked Read 5 Done Done Sent false
6633 2762 Acked Read 7 Done Done Sent false
6634 2763 Acked Read 4 Done Done Sent false
6635 2764 Acked Read 3 Done Done Sent false
6636 2765 Acked Write 5 Done Done Sent false
6637 2766 Acked Read 9 Done Done Sent false
6638 2767 Acked Write 2 Done Done Sent false
6639 2768 Acked Write 4 Done Done Sent false
6640 2769 Acked Write 2 Done Done Sent false
6641 2770 Acked Write 10 Done Done Sent false
6642 2771 Acked Write 7 Done Done Sent false
6643 2772 Acked Write 8 Done Done Sent false
6644 2773 Acked Write 5 Done Done Sent false
6645 2774 Acked Flush 0 Done Done Sent false
6646 2775 Acked Write 8 Done Done Sent false
6647 2776 Acked Read 7 Done Done Sent false
6648 2777 Acked Write 7 Done Done Sent false
6649 2778 Acked Read 9 Done Done Sent false
6650 2779 Acked Write 8 Done Done Sent false
6651 2780 Acked Write 2 Done Done Sent false
6652 2781 Acked Read 6 Done Done Sent false
6653 2782 Acked Write 8 Done Done Sent false
6654 2783 Acked Read 7 Done Done Sent false
6655 2784 Acked Read 1 Done Done Sent false
6656 2785 Acked Read 2 Done Done Sent false
6657 2786 Acked Write 3 Done Done Sent false
6658 2787 Acked Write 9 Done Done Sent false
6659 2788 Acked Write 2 Done Done Sent false
6660 2789 Acked Write 4 Done Done Sent false
6661 2790 Acked Write 1 Done Done Sent false
6662 2791 Acked Write 9 Done Done Sent false
6663 2792 Acked Write 1 Done Done Sent false
6664 2793 Acked Write 9 Done Done Sent false
6665 2794 Acked Read 3 Done Done Sent false
6666 2795 Acked Read 6 Done Done Sent false
6667 2796 Acked Read 10 Done Done Sent false
6668 2797 Acked Flush 0 Done Done Sent false
6669 2798 Acked Write 8 Done Done Sent false
6670 2799 Acked Read 3 Done Done Sent false
6671 2800 Acked Write 3 Done Done Sent false
6672 2801 Acked Write 8 Done Done Sent false
6673 2802 Acked Read 7 Done Done Sent false
6674 2803 Acked Read 5 Done Done Sent false
6675 2804 Acked Write 2 Done Done Sent false
6676 2805 Acked Read 6 Done Done Sent false
6677 2806 Acked Read 6 Done Done Sent false
6678 2807 Acked Write 6 Done Done Sent false
6679 2808 Acked Read 1 Done Done Sent false
6680 2809 Acked Read 6 Done Done Sent false
6681 2810 Acked Write 10 Done Done Sent false
6682 2811 Acked Flush 0 Done Done Sent false
6683 STATES DS:0 DS:1 DS:2 TOTAL
6684 Sent 0 0 302 302
6685 Done 302 302 0 604
6686 Skipped 0 0 0 0
6687 Error 0 0 0 0
6688 Last Flush: 2811 2811 2509
6689 Downstairs last five completed: 2509 2508 2507 2506 2505
6690 Upstairs last five completed: 2811 2810 2809 2808 2807
6691 CLIENT: Up:0 ds:302 act:2
66922024-12-11T23:41:53.767ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
66932024-12-11T23:41:53.767ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63789) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
66942024-12-11T23:41:53.768ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
66952024-12-11T23:41:53.769ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
66962024-12-11T23:41:53.769ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
66972024-12-11T23:41:53.769ZINFOcrucible: send last flush ID to this DS: 2509 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
66982024-12-11T23:41:53.769ZINFOcrucible: Replied this last flush ID: 2509 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
66992024-12-11T23:41:53.769ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
67002024-12-11T23:41:53.769ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
67012024-12-11T23:41:53.769ZINFOcrucible: [2] client re-new 302 jobs since flush 2509 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
67022024-12-11T23:41:53.769ZINFOcrucible: [2] Replayed 302 jobs since flush: 2509 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
6703 ----------------------------------------------------------------
6704 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6705 Downstairs last five completed: 2811 2810 2809 2808 2807
6706 Upstairs last five completed: 2811 2810 2809 2808 2807
6707 CLIENT: Up:0 ds:0 act:3
6708 Replay: All jobs finished
6709 Main task has work to do, go find it
6710 got dsc Stop(0)
6711 stop 0
6712 Dec 11 23:41:56.010 INFO request completed, latency_us: 364, response_code: 204, uri: /stop/cid/0, method: POST, req_id: d88f0899-2ae9-4fbc-85d3-3fa565873401, remote_addr: 127.[8810] Got stop action so:false kr:false
6713 0.0.1:62969, local_addr: 127.0.0.1:9998
67142024-12-11T23:41:56.010ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
67152024-12-11T23:41:56.010ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
67162024-12-11T23:41:56.010ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
6717 Dec 11 23:41:56.010 INFO request completed, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:41:56.010839001Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
6718 latency_us: {"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:41:56.010865332Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
6719 116, response_code: 200, uri: /state/cid/0, method: GET{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:41:56.010923693Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
6720 , req_id: d76fb846-f97f-4aa3-8119-f7b8c16b6626, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
6721 [8810] Exited with: ExitStatus(unix_wait_status(9))
6722 [8810][0] reports Exit
6723 Dec 11 23:42:00.012 INFO request completed, latency_us: 261, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7ed6a128-5f50-4f7d-87c4-a5f5727a31c4, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
6724 001/300 Write block 1311 len 12288 data: 4 4 4
6725 002/300 Read block 2419 len 36864
6726 003/300 Write block 4616 len 8192 data: 3 3
6727 004/300 Read block 3334 len 12288
6728 005/300 Write block 580 len 20480 data: 3 2 2 2 2
6729 006/300 Read block 1581 len 16384
6730 007/300 Write block 4715 len 12288 data: 3 3 3
6731 008/300 Read block 1472 len 4096
6732 009/300 Read block 2873 len 24576
6733 010/300 Write block 4214 len 24576 data: 3 2 2 2 3 3
6734 011/300 Flush
6735 012/300 Read block 2936 len 40960
6736 013/300 Write block 3853 len 12288 data: 4 4 3
6737 014/300 Flush
6738 015/300 Read block 3377 len 8192
6739 016/300 Write block 2109 len 32768 data: 2 2 2 2 4 5 5 5
6740 017/300 Flush
6741 018/300 Read block 1716 len 40960
6742 019/300 Flush
6743 020/300 Read block 3547 len 28672
6744 021/300 Read block 157 len 36864
6745 022/300 Write block 1006 len 36864 data: 2 2 2 3 3 3 2 2 3
6746 023/300 Write block 1456 len 36864 data: 2 2 2 2 2 2 2 3 3
6747 024/300 Write block 3211 len 4096 data: 2
6748 025/300 Flush
6749 026/300 Read block 3921 len 36864
6750 027/300 Read block 743 len 24576
6751 028/300 Write block 4839 len 12288 data: 3 4 4
6752 029/300 Read block 421 len 40960
6753 030/300 Read block 2266 len 20480
6754 031/300 Read block 2501 len 28672
6755 032/300 Write block 3881 len 28672 data: 2 2 2 2 2 3 3
6756 033/300 Write block 4948 len 16384 data: 2 2 2 2
6757 034/300 Flush
6758 035/300 Write block 784 len 12288 data: 2 2 2
6759 036/300 Read block 4435 len 32768
6760 037/300 Read block 1302 len 24576
6761 038/300 Read block 4771 len 32768
6762 039/300 Write block 3873 len 24576 data: 4 3 3 3 3 3
6763 040/300 Flush
6764 041/300 Read block 344 len 40960
6765 042/300 Write block 1383 len 12288 data: 2 2 3
6766 043/300 Write block 3132 len 24576 data: 4 4 3 4 5 5
6767 044/300 Write block 117 len 12288 data: 3 4 4
6768 045/300 Read block 484 len 4096
6769 046/300 Read block 2609 len 20480
6770 047/300 Read block 357 len 12288
6771 048/300 Flush
6772 049/300 Read block 2906 len 8192
6773 050/300 Write block 2855 len 24576 data: 5 5 5 4 4 3
6774 051/300 Read block 2361 len 12288
6775 052/300 Read block 3805 len 12288
6776 053/300 Read block 2393 len 12288
6777 054/300 Write block 238 len 16384 data: 2 2 2 2
6778 055/300 Write block 1117 len 16384 data: 2 2 2 2
6779 056/300 Write block 404 len 28672 data: 2 2 2 2 4 4 4
6780 057/300 Write block 1771 len 24576 data: 2 2 3 3 3 2
6781 058/300 Read block 3968 len 40960
6782 059/300 Read block 3317 len 32768
6783 060/300 Read block 3063 len 20480
6784 061/300 Read block 3421 len 16384
6785 062/300 Write block 104 len 12288 data: 5 4 2
6786 063/300 Read block 183 len 12288
6787 064/300 Flush
6788 065/300 Read block 728 len 4096
6789 066/300 Write block 482 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6790 067/300 Read block 4342 len 16384
6791 068/300 Read block 4430 len 36864
6792 069/300 Read block 1410 len 32768
6793 070/300 Read block 524 len 4096
6794 071/300 Read block 542 len 24576
6795 072/300 Read block 3561 len 36864
6796 073/300 Flush
6797 074/300 Read block 2766 len 16384
6798 075/300 Read block 2390 len 8192
6799 076/300 Read block 2175 len 24576
6800 077/300 Read block 3941 len 32768
6801 078/300 Write block 4557 len 24576 data: 2 2 2 2 2 2
6802 079/300 Read block 4306 len 16384
6803 080/300 Write block 1717 len 4096 data: 2
6804 081/300 Write block 4921 len 16384 data: 4 5 4 5
6805 082/300 Write block 1032 len 16384 data: 4 3 4 4
6806 083/300 Write block 3046 len 28672 data: 5 4 3 3 3 3 3
6807 084/300 Read block 4171 len 8192
6808 085/300 Flush
6809 086/300 Flush
6810 087/300 Read block 115 len 24576
6811 088/300 Flush
6812 089/300 Read block 2025 len 24576
6813 090/300 Flush
6814 091/300 Read block 1790 len 20480
6815 092/300 Read block 1438 len 8192
6816 093/300 Read block 4823 len 8192
6817 094/300 Write block 1913 len 20480 data: 4 3 2 2 2
6818 095/300 Write block 2731 len 8192 data: 3 3
6819 096/300 Read block 3586 len 32768
6820 097/300 Write block 4770 len 36864 data: 2 3 3 3 3 3 3 3 3
6821 098/300 Write block 3581 len 36864 data: 3 2 2 2 3 3 5 5 4
6822 099/300 Flush
6823 100/300 Write block 1116 len 24576 data: 3 3 3 3 3 2
6824 101/300 Write block 1188 len 12288 data: 2 2 3
6825 102/300 Write block 2833 len 40960 data: 5 4 4 3 3 2 2 2 2 2
6826 103/300 Read block 3235 len 16384
6827 104/300 Flush
6828 105/300 Write block 531 len 24576 data: 2 2 2 2 2 3
6829 106/300 Read block 260 len 24576
6830 107/300 Write block 3017 len 28672 data: 3 3 3 2 2 2 2
6831 108/300 Write block 4781 len 36864 data: 3 3 3 3 3 2 2 2 2
6832 109/300 Write block 2996 len 28672 data: 3 4 4 3 3 4 4
6833 110/300 Read block 2043 len 4096
6834 111/300 Write block 12 len 4096 data: 3
6835 112/300 Read block 3159 len 8192
6836 113/300 Flush
6837 114/300 Flush
6838 115/300 Read block 1161 len 12288
6839 116/300 Write block 3991 len 24576 data: 4 4 3 3 3 3
6840 117/300 Write block 1174 len 36864 data: 2 2 2 2 2 2 2 2 3
6841 118/300 Read block 2263 len 20480
6842 119/300 Read block 1670 len 28672
6843 120/300 Read block 681 len 20480
6844 121/300 Write block 2992 len 36864 data: 2 2 2 2 4 5 5 4 4
6845 122/300 Write block 577 len 24576 data: 4 4 4 4 3 3
6846 123/300 Write block 3979 len 40960 data: 3 3 3 2 2 2 2 3 3 3
6847 124/300 Write block 3757 len 24576 data: 2 2 2 2 2 2
6848 125/300 Flush
6849 126/300 Write block 4922 len 40960 data: 6 5 6 4 5 4 4 4 4 4
6850 127/300 Write block 4526 len 28672 data: 5 4 4 4 6 6 6
6851 128/300 Read block 1368 len 20480
6852 129/300 Read block 2233 len 36864
6853 130/300 Read block 4096 len 20480
6854 131/300 Flush
6855 132/300 Write block 2706 len 40960 data: 3 2 2 2 3 3 3 3 3 3
6856 133/300 Read block 3454 len 28672
6857 134/300 Write block 1233 len 32768 data: 3 4 2 2 2 2 2 2
6858 135/300 Write block 692 len 28672 data: 2 2 2 2 2 2 2
6859 136/300 Flush
6860 137/300 Read block 3071 len 16384
6861 138/300 Read block 3160 len 20480
6862 139/300 Read block 180 len 28672
6863 140/300 Read block 1704 len 40960
6864 141/300 Read block 4805 len 28672
6865 142/300 Write block 2206 len 28672 data: 2 2 2 2 2 3 3
6866 143/300 Read block 342 len 28672
6867 144/300 Write block 1937 len 24576 data: 2 2 2 2 3 3
6868 145/300 Read block 3563 len 8192
6869 146/300 Read block 3928 len 4096
6870 147/300 Read block 3911 len 24576
6871 148/300 Read block 2043 len 4096
6872 149/300 Read block 4678 len 36864
6873 150/300 Write block 4759 len 16384 data: 2 2 2 2
6874 151/300 Write block 2222 len 20480 data: 2 2 3 3 3
6875 152/300 Write block 3375 len 4096 data: 2
6876 153/300 Write block 3438 len 36864 data: 2 3 3 2 2 2 2 2 2
6877 154/300 Read block 2414 len 28672
6878 155/300 Read block 2560 len 8192
6879 156/300 Write block 4875 len 12288 data: 3 3 2
6880 157/300 Write block 353 len 20480 data: 2 2 2 2 2
6881 158/300 Write block 1694 len 36864 data: 3 3 3 3 3 2 2 2 2
6882 159/300 Read block 1438 len 40960
6883 160/300 Write block 3153 len 36864 data: 2 2 4 4 3 4 4 4 4
6884 161/300 Read block 2230 len 40960
6885 162/300 Read block 2810 len 8192
6886 163/300 Flush
6887 164/300 Write block 1683 len 32768 data: 3 4 3 3 3 3 3 3
6888 165/300 Write block 2998 len 8192 data: 6 5
6889 166/300 Write block 1924 len 4096 data: 4
6890 167/300 Write block 1884 len 4096 data: 3
6891 168/300 Read block 3852 len 8192
6892 169/300 Read block 4475 len 8192
6893 170/300 Read block 273 len 20480
6894 171/300 Write block 2550 len 20480 data: 3 4 4 3 3
6895 172/300 Read block 1550 len 16384
6896 173/300 Write block 906 len 40960 data: 2 2 2 2 2 2 3 2 2 2
6897 174/300 Read block 1798 len 28672
6898 175/300 Read block 532 len 32768
6899 176/300 Write block 2226 len 40960 data: 4 3 3 3 3 3 3 2 2 2
6900 177/300 Read block 3619 len 12288
6901 178/300 Read block 1350 len 28672
6902 179/300 Flush
6903 180/300 Read block 1749 len 32768
6904 181/300 Read block 872 len 8192
6905 182/300 Read block 634 len 20480
6906 183/300 Write block 3311 len 16384 data: 4 3 3 4
6907 184/300 Read block 166 len 28672
6908 185/300 Read block 1867 len 16384
6909 186/300 Write block 3100 len 4096 data: 2
6910 187/300 Write block 3432 len 32768 data: 3 3 3 3 2 2 3 4
6911 188/300 Write block 4261 len 36864 data: 3 3 3 3 2 2 2 2 2
6912 189/300 Write block 4911 len 36864 data: 2 2 2 3 2 2 2 2 2
6913 190/300 Write block 3379 len 12288 data: 2 2 2
6914 191/300 Write block 1297 len 8192 data: 2 2
6915 192/300 Read block 2359 len 4096
6916 193/300 Write block 2701 len 24576 data: 3 3 3 3 2 4
6917 194/300 Read block 4432 len 36864
6918 195/300 Read block 321 len 16384
6919 196/300 Read block 3760 len 28672
6920 197/300 Read block 2292 len 16384
6921 198/300 Read block 1959 len 24576
6922 199/300 Write block 304 len 32768 data: 3 3 2 2 2 2 2 2
6923 200/300 Read block 840 len 40960
6924 201/300 Read block 1103 len 12288
6925 202/300 Read block 2865 len 4096
6926 203/300 Read block 2412 len 32768
6927 204/300 Write block 3242 len 4096 data: 2
6928 205/300 Write block 2131 len 4096 data: 3
6929 206/300 Read block 3330 len 20480
6930 207/300 Read block 3400 len 20480
6931 208/300 Write block 2379 len 20480 data: 2 2 2 2 2
6932 209/300 Flush
6933 210/300 Flush
6934 211/300 Read block 3401 len 24576
6935 212/300 Write block 2122 len 36864 data: 3 3 2 3 3 3 3 3 3
6936 213/300 Read block 3173 len 32768
6937 214/300 Read block 2497 len 32768
6938 215/300 Read block 4949 len 16384
6939 216/300 Write block 1091 len 16384 data: 4 4 3 3
6940 217/300 Write block 49 len 12288 data: 4 4 2
6941 218/300 Read block 388 len 12288
6942 219/300 Read block 4349 len 32768
6943 220/300 Read block 2212 len 16384
6944 221/300 Read block 3097 len 40960
6945 222/300 Write block 1110 len 16384 data: 2 2 2 2
6946 223/300 Read block 4622 len 20480
6947 224/300 Read block 1314 len 16384
6948 225/300 Read block 2472 len 4096
6949 226/300 Flush
6950 227/300 Write block 1337 len 12288 data: 2 2 2
6951 228/300 Write block 3092 len 36864 data: 4 5 5 4 4 3 3 2 3
6952 229/300 Read block 2362 len 36864
6953 230/300 Write block 3263 len 16384 data: 3 2 2 2
6954 231/300 Write block 2080 len 16384 data: 2 2 2 2
6955 232/300 Read block 2269 len 8192
6956 233/300 Write block 4422 len 12288 data: 2 2 3
6957 234/300 Write block 1619 len 16384 data: 2 2 2 2
6958 235/300 Write block 490 len 40960 data: 3 3 3 3 3 3 3 3 2 2
6959 236/300 Read block 4213 len 32768
6960 237/300 Read block 4001 len 28672
6961 238/300 Read block 1735 len 24576
6962 239/300 Read block 1892 len 32768
6963 240/300 Read block 2698 len 8192
6964 241/300 Read block 3517 len 36864
6965 242/300 Write block 4594 len 28672 data: 3 2 3 4 4 4 4
6966 243/300 Read block 4852 len 36864
6967 244/300 Read block 2673 len 4096
6968 245/300 Read block 3649 len 20480
6969 246/300 Write block 2346 len 20480 data: 2 2 2 2 2
6970 247/300 Read block 4255 len 16384
6971 248/300 Write block 213 len 4096 data: 2
6972 249/300 Write block 85 len 4096 data: 5
6973 250/300 Read block 1715 len 4096
6974 251/300 Flush
6975 252/300 Read block 3250 len 40960
6976 253/300 Write block 831 len 40960 data: 3 2 3 3 3 3 3 3 3 3
6977 254/300 Write block 4314 len 28672 data: 2 2 2 2 3 3 3
6978 255/300 Read block 2282 len 36864
6979 256/300 Read block 672 len 36864
6980 257/300 Write block 3909 len 32768 data: 4 4 4 4 4 3 3 3
6981 258/300 Write block 3714 len 28672 data: 3 3 3 3 3 3 3
6982 259/300 Write block 3749 len 20480 data: 3 3 4 3 3
6983 260/300 Read block 660 len 4096
6984 261/300 Write block 2280 len 28672 data: 3 3 3 3 2 2 2
6985 262/300 Flush
6986 263/300 Write block 1882 len 36864 data: 3 3 4 3 3 3 3 4 3
6987 264/300 Write block 1092 len 24576 data: 5 4 4 3 3 4
6988 265/300 Read block 3143 len 12288
6989 266/300 Write block 4175 len 40960 data: 3 3 3 3 3 4 4 4 4 4
6990 267/300 Read block 1710 len 4096
6991 268/300 Read block 945 len 20480
6992 269/300 Read block 2664 len 16384
6993 270/300 Write block 2208 len 40960 data: 3 3 3 4 4 3 3 3 3 2
6994 271/300 Write block 3060 len 24576 data: 3 2 2 3 3 3
6995 272/300 Read block 1959 len 16384
6996 273/300 Flush
6997 274/300 Read block 815 len 24576
6998 275/300 Write block 3337 len 12288 data: 2 2 2
6999 276/300 Write block 4677 len 40960 data: 3 3 3 3 2 2 2 2 2 2
7000 277/300 Read block 1611 len 36864
7001 278/300 Write block 2062 len 24576 data: 3 4 3 3 4 4
7002 279/300 Read block 2730 len 20480
7003 280/300 Flush
7004 281/300 Flush
7005 282/300 Read block 3972 len 16384
7006 283/300 Read block 3335 len 8192
7007 284/300 Write block 589 len 40960 data: 3 3 3 3 2 2 2 2 2 2
7008 285/300 Write block 3556 len 8192 data: 2 2
7009 286/300 Read block 1042 len 8192
7010 287/300 Flush
7011 288/300 Write block 4288 len 20480 data: 4 3 2 2 2
7012 289/300 Read block 2308 len 40960
7013 290/300 Write block 1500 len 36864 data: 2 2 2 2 2 2 2 2 2
7014 291/300 Write block 123 len 32768 data: 4 4 4 3 2 2 3 3
7015 292/300 Write block 4131 len 8192 data: 2 2
7016 293/300 Read block 1737 len 24576
7017 294/300 Read block 2896 len 4096
7018 295/300 Read block 3332 len 12288
7019 296/300 Write block 4139 len 24576 data: 2 2 2 2 3 3
7020 297/300 Read block 2679 len 8192
7021 298/300 Read block 288 len 4096
7022 299/300 Read block 1887 len 8192
7023 300/300 Write block 1687 len 20480 data: 4 4 4 4 3
7024 301/300 Write block 1637 len 32768 data: 3 4 4 3 4 4 4 4
7025 Main task has work to do, go find it
7026 got dsc Start(0)
7027 start 0
7028 Dec 11 23:42:00.250 INFO request completed, latency_us: 231, response_code: 204, uri: /start/cid/0, method: POST, req_id: d8f6cc5b-8c06-4346-9912-b60dc791c610, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
7029 Replay: started 0, returned:Ok(())
7030 ----------------------------------------------------------------
7031 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7032 States: Offline Active Active
7033 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7034 2812 Acked Write 3 Sent Done Done false
7035 2813 Acked Read 9 Sent Done Done false
7036 2814 Acked Write 2 Sent Done Done false
7037 2815 Acked Read 3 Sent Done Done false
7038 2816 Acked Write 5 Sent Done Done false
7039 2817 Acked Read 4 Sent Done Done false
7040 2818 Acked Write 3 Sent Done Done false
7041 2819 Acked Read 1 Sent Done Done false
7042 2820 Acked Read 6 Sent Done Done false
7043 2821 Acked Write 6 Sent Done Done false
7044 2822 Acked Flush 0 Sent Done Done false
7045 2823 Acked Read 10 Sent Done Done false
7046 2824 Acked Write 3 Sent Done Done false
7047 2825 Acked Flush 0 Sent Done Done false
7048 2826 Acked Read 2 Sent Done Done false
7049 2827 Acked Write 8 Sent Done Done false
7050 2828 Acked Flush 0 Sent Done Done false
7051 2829 Acked Read 10 Sent Done Done false
7052 2830 Acked Flush 0 Sent Done Done false
7053 2831 Acked Read 7 Sent Done Done false
7054 2832 Acked Read 9 Sent Done Done false
7055 2833 Acked Write 9 Sent Done Done false
7056 2834 Acked Write 9 Sent Done Done false
7057 2835 Acked Write 1 Sent Done Done false
7058 2836 Acked Flush 0 Sent Done Done false
7059 2837 Acked Read 9 Sent Done Done false
7060 2838 Acked Read 6 Sent Done Done false
7061 2839 Acked Write 3 Sent Done Done false
7062 2840 Acked Read 10 Sent Done Done false
7063 2841 Acked Read 5 Sent Done Done false
7064 2842 Acked Read 7 Sent Done Done false
7065 2843 Acked Write 7 Sent Done Done false
7066 2844 Acked Write 4 Sent Done Done false
7067 2845 Acked Flush 0 Sent Done Done false
7068 2846 Acked Write 3 Sent Done Done false
7069 2847 Acked Read 8 Sent Done Done false
7070 2848 Acked Read 6 Sent Done Done false
7071 2849 Acked Read 8 Sent Done Done false
7072 2850 Acked Write 6 Sent Done Done false
7073 2851 Acked Flush 0 Sent Done Done false
7074 2852 Acked Read 10 Sent Done Done false
7075 2853 Acked Write 3 Sent Done Done false
7076 2854 Acked Write 6 Sent Done Done false
7077 2855 Acked Write 3 Sent Done Done false
7078 2856 Acked Read 1 Sent Done Done false
7079 2857 Acked Read 5 Sent Done Done false
7080 2858 Acked Read 3 Sent Done Done false
7081 2859 Acked Flush 0 Sent Done Done false
7082 2860 Acked Read 2 Sent Done Done false
7083 2861 Acked Write 6 Sent Done Done false
7084 2862 Acked Read 3 Sent Done Done false
7085 2863 Acked Read 3 Sent Done Done false
7086 2864 Acked Read 3 Sent Done Done false
7087 2865 Acked Write 4 Sent Done Done false
7088 2866 Acked Write 4 Sent Done Done false
7089 2867 Acked Write 7 Sent Done Done false
7090 2868 Acked Write 6 Sent Done Done false
7091 2869 Acked Read 10 Sent Done Done false
7092 2870 Acked Read 8 Sent Done Done false
7093 2871 Acked Read 5 Sent Done Done false
7094 2872 Acked Read 4 Sent Done Done false
7095 2873 Acked Write 3 Sent Done Done false
7096 2874 Acked Read 3 Sent Done Done false
7097 2875 Acked Flush 0 Sent Done Done false
7098 2876 Acked Read 1 Sent Done Done false
7099 2877 Acked Write 10 Sent Done Done false
7100 2878 Acked Read 4 Sent Done Done false
7101 2879 Acked Read 9 Sent Done Done false
7102 2880 Acked Read 8 Sent Done Done false
7103 2881 Acked Read 1 Sent Done Done false
7104 2882 Acked Read 6 Sent Done Done false
7105 2883 Acked Read 9 Sent Done Done false
7106 2884 Acked Flush 0 Sent Done Done false
7107 2885 Acked Read 4 Sent Done Done false
7108 2886 Acked Read 2 Sent Done Done false
7109 2887 Acked Read 6 Sent Done Done false
7110 2888 Acked Read 8 Sent Done Done false
7111 2889 Acked Write 6 Sent Done Done false
7112 2890 Acked Read 4 Sent Done Done false
7113 2891 Acked Write 1 Sent Done Done false
7114 2892 Acked Write 4 Sent Done Done false
7115 2893 Acked Write 4 Sent Done Done false
7116 2894 Acked Write 7 Sent Done Done false
7117 2895 Acked Read 2 Sent Done Done false
7118 2896 Acked Flush 0 Sent Done Done false
7119 2897 Acked Flush 0 Sent Done Done false
7120 2898 Acked Read 6 Sent Done Done false
7121 2899 Acked Flush 0 Sent Done Done false
7122 2900 Acked Read 6 Sent Done Done false
7123 2901 Acked Flush 0 Sent Done Done false
7124 2902 Acked Read 5 Sent Done Done false
7125 2903 Acked Read 2 Sent Done Done false
7126 2904 Acked Read 2 Sent Done Done false
7127 2905 Acked Write 5 Sent Done Done false
7128 2906 Acked Write 2 Sent Done Done false
7129 2907 Acked Read 8 Sent Done Done false
7130 2908 Acked Write 9 Sent Done Done false
7131 2909 Acked Write 9 Sent Done Done false
7132 2910 Acked Flush 0 Sent Done Done false
7133 2911 Acked Write 6 Sent Done Done false
7134 2912 Acked Write 3 Sent Done Done false
7135 2913 Acked Write 10 Sent Done Done false
7136 2914 Acked Read 4 Sent Done Done false
7137 2915 Acked Flush 0 Sent Done Done false
7138 2916 Acked Write 6 Sent Done Done false
7139 2917 Acked Read 6 Sent Done Done false
7140 2918 Acked Write 7 Sent Done Done false
7141 2919 Acked Write 9 Sent Done Done false
7142 2920 Acked Write 7 Sent Done Done false
7143 2921 Acked Read 1 Sent Done Done false
7144 2922 Acked Write 1 Sent Done Done false
7145 2923 Acked Read 2 Sent Done Done false
7146 2924 Acked Flush 0 Sent Done Done false
7147 2925 Acked Flush 0 Sent Done Done false
7148 2926 Acked Read 3 Sent Done Done false
7149 2927 Acked Write 6 Sent Done Done false
7150 2928 Acked Write 9 Sent Done Done false
7151 2929 Acked Read 5 Sent Done Done false
7152 2930 Acked Read 7 Sent Done Done false
7153 2931 Acked Read 5 Sent Done Done false
7154 2932 Acked Write 9 Sent Done Done false
7155 2933 Acked Write 6 Sent Done Done false
7156 2934 Acked Write 10 Sent Done Done false
7157 2935 Acked Write 6 Sent Done Done false
7158 2936 Acked Flush 0 Sent Done Done false
7159 2937 Acked Write 10 Sent Done Done false
7160 2938 Acked Write 7 Sent Done Done false
7161 2939 Acked Read 5 Sent Done Done false
7162 2940 Acked Read 9 Sent Done Done false
7163 2941 Acked Read 5 Sent Done Done false
7164 2942 Acked Flush 0 Sent Done Done false
7165 2943 Acked Write 10 Sent Done Done false
7166 2944 Acked Read 7 Sent Done Done false
7167 2945 Acked Write 8 Sent Done Done false
7168 2946 Acked Write 7 Sent Done Done false
7169 2947 Acked Flush 0 Sent Done Done false
7170 2948 Acked Read 4 Sent Done Done false
7171 2949 Acked Read 5 Sent Done Done false
7172 2950 Acked Read 7 Sent Done Done false
7173 2951 Acked Read 10 Sent Done Done false
7174 2952 Acked Read 7 Sent Done Done false
7175 2953 Acked Write 7 Sent Done Done false
7176 2954 Acked Read 7 Sent Done Done false
7177 2955 Acked Write 6 Sent Done Done false
7178 2956 Acked Read 2 Sent Done Done false
7179 2957 Acked Read 1 Sent Done Done false
7180 2958 Acked Read 6 Sent Done Done false
7181 2959 Acked Read 1 Sent Done Done false
7182 2960 Acked Read 9 Sent Done Done false
7183 2961 Acked Write 4 Sent Done Done false
7184 2962 Acked Write 5 Sent Done Done false
7185 2963 Acked Write 1 Sent Done Done false
7186 2964 Acked Write 9 Sent Done Done false
7187 2965 Acked Read 7 Sent Done Done false
7188 2966 Acked Read 2 Sent Done Done false
7189 2967 Acked Write 3 Sent Done Done false
7190 2968 Acked Write 5 Sent Done Done false
7191 2969 Acked Write 9 Sent Done Done false
7192 2970 Acked Read 10 Sent Done Done false
7193 2971 Acked Write 9 Sent Done Done false
7194 2972 Acked Read 10 Sent Done Done false
7195 2973 Acked Read 2 Sent Done Done false
7196 2974 Acked Flush 0 Sent Done Done false
7197 2975 Acked Write 8 Sent Done Done false
7198 2976 Acked Write 2 Sent Done Done false
7199 2977 Acked Write 1 Sent Done Done false
7200 2978 Acked Write 1 Sent Done Done false
7201 2979 Acked Read 2 Sent Done Done false
7202 2980 Acked Read 2 Sent Done Done false
7203 2981 Acked Read 5 Sent Done Done false
7204 2982 Acked Write 5 Sent Done Done false
7205 2983 Acked Read 4 Sent Done Done false
7206 2984 Acked Write 10 Sent Done Done false
7207 2985 Acked Read 7 Sent Done Done false
7208 2986 Acked Read 8 Sent Done Done false
7209 2987 Acked Write 10 Sent Done Done false
7210 2988 Acked Read 3 Sent Done Done false
7211 2989 Acked Read 7 Sent Done Done false
7212 2990 Acked Flush 0 Sent Done Done false
7213 2991 Acked Read 8 Sent Done Done false
7214 2992 Acked Read 2 Sent Done Done false
7215 2993 Acked Read 5 Sent Done Done false
7216 2994 Acked Write 4 Sent Done Done false
7217 2995 Acked Read 7 Sent Done Done false
7218 2996 Acked Read 4 Sent Done Done false
7219 2997 Acked Write 1 Sent Done Done false
7220 2998 Acked Write 8 Sent Done Done false
7221 2999 Acked Write 9 Sent Done Done false
7222 3000 Acked Write 9 Sent Done Done false
7223 3001 Acked Write 3 Sent Done Done false
7224 3002 Acked Write 2 Sent Done Done false
7225 3003 Acked Read 1 Sent Done Done false
7226 3004 Acked Write 6 Sent Done Done false
7227 3005 Acked Read 9 Sent Done Done false
7228 3006 Acked Read 4 Sent Done Done false
7229 3007 Acked Read 7 Sent Done Done false
7230 3008 Acked Read 4 Sent Done Done false
7231 3009 Acked Read 6 Sent Done Done false
7232 3010 Acked Write 8 Sent Done Done false
7233 3011 Acked Read 10 Sent Done Done false
7234 3012 Acked Read 3 Sent Done Done false
7235 3013 Acked Read 1 Sent Done Done false
7236 3014 Acked Read 8 Sent Done Done false
7237 3015 Acked Write 1 Sent Done Done false
7238 3016 Acked Write 1 Sent Done Done false
7239 3017 Acked Read 5 Sent Done Done false
7240 3018 Acked Read 5 Sent Done Done false
7241 3019 Acked Write 5 Sent Done Done false
7242 3020 Acked Flush 0 Sent Done Done false
7243 3021 Acked Flush 0 Sent Done Done false
7244 3022 Acked Read 6 Sent Done Done false
7245 3023 Acked Write 9 Sent Done Done false
7246 3024 Acked Read 8 Sent Done Done false
7247 3025 Acked Read 8 Sent Done Done false
7248 3026 Acked Read 4 Sent Done Done false
7249 3027 Acked Write 4 Sent Done Done false
7250 3028 Acked Write 3 Sent Done Done false
7251 3029 Acked Read 3 Sent Done Done false
7252 3030 Acked Read 8 Sent Done Done false
7253 3031 Acked Read 4 Sent Done Done false
7254 3032 Acked Read 10 Sent Done Done false
7255 3033 Acked Write 4 Sent Done Done false
7256 3034 Acked Read 5 Sent Done Done false
7257 3035 Acked Read 4 Sent Done Done false
7258 3036 Acked Read 1 Sent Done Done false
7259 3037 Acked Flush 0 Sent Done Done false
7260 3038 Acked Write 3 Sent Done Done false
7261 3039 Acked Write 9 Sent Done Done false
7262 3040 Acked Read 9 Sent Done Done false
7263 3041 Acked Write 4 Sent Done Done false
7264 3042 Acked Write 4 Sent Done Done false
7265 3043 Acked Read 2 Sent Done Done false
7266 3044 Acked Write 3 Sent Done Done false
7267 3045 Acked Write 4 Sent Done Done false
7268 3046 Acked Write 10 Sent Done Done false
7269 3047 Acked Read 8 Sent Done Done false
7270 3048 Acked Read 7 Sent Done Done false
7271 3049 Acked Read 6 Sent Done Done false
7272 3050 Acked Read 8 Sent Done Done false
7273 3051 Acked Read 2 Sent Done Done false
7274 3052 Acked Read 9 Sent Done Done false
7275 3053 Acked Write 7 Sent Done Done false
7276 3054 Acked Read 9 Sent Done Done false
7277 3055 Acked Read 1 Sent Done Done false
7278 3056 Acked Read 5 Sent Done Done false
7279 3057 Acked Write 5 Sent Done Done false
7280 3058 Acked Read 4 Sent Done Done false
7281 3059 Acked Write 1 Sent Done Done false
7282 3060 Acked Write 1 Sent Done Done false
7283 3061 Acked Read 1 Sent Done Done false
7284 3062 Acked Flush 0 Sent Done Done false
7285 3063 Acked Read 10 Sent Done Done false
7286 3064 Acked Write 10 Sent Done Done false
7287 3065 Acked Write 7 Sent Done Done false
7288 3066 Acked Read 9 Sent Done Done false
7289 3067 Acked Read 9 Sent Done Done false
7290 3068 Acked Write 8 Sent Done Done false
7291 3069 Acked Write 7 Sent Done Done false
7292 3070 Acked Write 5 Sent Done Done false
7293 3071 Acked Read 1 Sent Done Done false
7294 3072 Acked Write 7 Sent Done Done false
7295 3073 Acked Flush 0 Sent Done Done false
7296 3074 Acked Write 9 Sent Done Done false
7297 3075 Acked Write 6 Sent Done Done false
7298 3076 Acked Read 3 Sent Done Done false
7299 3077 Acked Write 10 Sent Done Done false
7300 3078 Acked Read 1 Sent Done Done false
7301 3079 Acked Read 5 Sent Done Done false
7302 3080 Acked Read 4 Sent Done Done false
7303 3081 Acked Write 10 Sent Done Done false
7304 3082 Acked Write 6 Sent Done Done false
7305 3083 Acked Read 4 Sent Done Done false
7306 3084 Acked Flush 0 Sent Done Done false
7307 3085 Acked Read 6 Sent Done Done false
7308 3086 Acked Write 3 Sent Done Done false
7309 3087 Acked Write 10 Sent Done Done false
7310 3088 Acked Read 9 Sent Done Done false
7311 3089 Acked Write 6 Sent Done Done false
7312 3090 Acked Read 5 Sent Done Done false
7313 3091 Acked Flush 0 Sent Done Done false
7314 3092 Acked Flush 0 Sent Done Done false
7315 3093 Acked Read 4 Sent Done Done false
7316 3094 Acked Read 2 Sent Done Done false
7317 3095 Acked Write 10 Sent Done Done false
7318 3096 Acked Write 2 Sent Done Done false
7319 3097 Acked Read 2 Sent Done Done false
7320 3098 Acked Flush 0 Sent Done Done false
7321 3099 Acked Write 5 Sent Done Done false
7322 3100 Acked Read 10 Sent Done Done false
7323 3101 Acked Write 9 Sent Done Done false
7324 3102 Acked Write 8 Sent Done Done false
7325 3103 Acked Write 2 Sent Done Done false
7326 3104 Acked Read 6 Sent Done Done false
7327 3105 Acked Read 1 Sent Done Done false
7328 3106 Acked Read 3 Sent Done Done false
7329 3107 Acked Write 6 Sent Done Done false
7330 3108 Acked Read 2 Sent Done Done false
7331 3109 Acked Read 1 Sent Done Done false
7332 3110 Acked Read 2 Sent Done Done false
7333 3111 Acked Write 5 Sent Done Done false
7334 3112 Acked Write 8 Sent Done Done false
7335 STATES DS:0 DS:1 DS:2 TOTAL
7336 Sent 301 0 0 301
7337 Done 0 301 301 602
7338 Skipped 0 0 0 0
7339 Error 0 0 0 0
7340 Last Flush: 2811 3098 3098
7341 Downstairs last five completed: 2811 2810 2809 2808 2807
7342 Upstairs last five completed: 3112 3111 3110 3109 3108
7343 CLIENT: Up:0 ds:301 act:2
7344 ----------------------------------------------------------------
7345 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7346 States: Offline Active Active
7347 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7348 2812 Acked Write 3 Sent Done Done false
7349 2813 Acked Read 9 Sent Done Done false
7350 2814 Acked Write 2 Sent Done Done false
7351 2815 Acked Read 3 Sent Done Done false
7352 2816 Acked Write 5 Sent Done Done false
7353 2817 Acked Read 4 Sent Done Done false
7354 2818 Acked Write 3 Sent Done Done false
7355 2819 Acked Read 1 Sent Done Done false
7356 2820 Acked Read 6 Sent Done Done false
7357 2821 Acked Write 6 Sent Done Done false
7358 2822 Acked Flush 0 Sent Done Done false
7359 2823 Acked Read 10 Sent Done Done false
7360 2824 Acked Write 3 Sent Done Done false
7361 2825 Acked Flush 0 Sent Done Done false
7362 2826 Acked Read 2 Sent Done Done false
7363 2827 Acked Write 8 Sent Done Done false
7364 2828 Acked Flush 0 Sent Done Done false
7365 2829 Acked Read 10 Sent Done Done false
7366 2830 Acked Flush 0 Sent Done Done false
7367 2831 Acked Read 7 Sent Done Done false
7368 2832 Acked Read 9 Sent Done Done false
7369 2833 Acked Write 9 Sent Done Done false
7370 2834 Acked Write 9 Sent Done Done false
7371 2835 Acked Write 1 Sent Done Done false
7372 2836 Acked Flush 0 Sent Done Done false
7373 2837 Acked Read 9 Sent Done Done false
7374 2838 Acked Read 6 Sent Done Done false
7375 2839 Acked Write 3 Sent Done Done false
7376 2840 Acked Read 10 Sent Done Done false
7377 2841 Acked Read 5 Sent Done Done false
7378 2842 Acked Read 7 Sent Done Done false
7379 2843 Acked Write 7 Sent Done Done false
7380 2844 Acked Write 4 Sent Done Done false
7381 2845 Acked Flush 0 Sent Done Done false
7382 2846 Acked Write 3 Sent Done Done false
7383 2847 Acked Read 8 Sent Done Done false
7384 2848 Acked Read 6 Sent Done Done false
7385 2849 Acked Read 8 Sent Done Done false
7386 2850 Acked Write 6 Sent Done Done false
7387 2851 Acked Flush 0 Sent Done Done false
7388 2852 Acked Read 10 Sent Done Done false
7389 2853 Acked Write 3 Sent Done Done false
7390 2854 Acked Write 6 Sent Done Done false
7391 2855 Acked Write 3 Sent Done Done false
7392 2856 Acked Read 1 Sent Done Done false
7393 2857 Acked Read 5 Sent Done Done false
7394 2858 Acked Read 3 Sent Done Done false
7395 2859 Acked Flush 0 Sent Done Done false
7396 2860 Acked Read 2 Sent Done Done false
7397 2861 Acked Write 6 Sent Done Done false
7398 2862 Acked Read 3 Sent Done Done false
7399 2863 Acked Read 3 Sent Done Done false
7400 2864 Acked Read 3 Sent Done Done false
7401 2865 Acked Write 4 Sent Done Done false
7402 2866 Acked Write 4 Sent Done Done false
7403 2867 Acked Write 7 Sent Done Done false
7404 2868 Acked Write 6 Sent Done Done false
7405 2869 Acked Read 10 Sent Done Done false
7406 2870 Acked Read 8 Sent Done Done false
7407 2871 Acked Read 5 Sent Done Done false
7408 2872 Acked Read 4 Sent Done Done false
7409 2873 Acked Write 3 Sent Done Done false
7410 2874 Acked Read 3 Sent Done Done false
7411 2875 Acked Flush 0 Sent Done Done false
7412 2876 Acked Read 1 Sent Done Done false
7413 2877 Acked Write 10 Sent Done Done false
7414 2878 Acked Read 4 Sent Done Done false
7415 2879 Acked Read 9 Sent Done Done false
7416 2880 Acked Read 8 Sent Done Done false
7417 2881 Acked Read 1 Sent Done Done false
7418 2882 Acked Read 6 Sent Done Done false
7419 2883 Acked Read 9 Sent Done Done false
7420 2884 Acked Flush 0 Sent Done Done false
7421 2885 Acked Read 4 Sent Done Done false
7422 2886 Acked Read 2 Sent Done Done false
7423 2887 Acked Read 6 Sent Done Done false
7424 2888 Acked Read 8 Sent Done Done false
7425 2889 Acked Write 6 Sent Done Done false
7426 2890 Acked Read 4 Sent Done Done false
7427 2891 Acked Write 1 Sent Done Done false
7428 2892 Acked Write 4 Sent Done Done false
7429 2893 Acked Write 4 Sent Done Done false
7430 2894 Acked Write 7 Sent Done Done false
7431 2895 Acked Read 2 Sent Done Done false
7432 2896 Acked Flush 0 Sent Done Done false
7433 2897 Acked Flush 0 Sent Done Done false
7434 2898 Acked Read 6 Sent Done Done false
7435 2899 Acked Flush 0 Sent Done Done false
7436 2900 Acked Read 6 Sent Done Done false
7437 2901 Acked Flush 0 Sent Done Done false
7438 2902 Acked Read 5 Sent Done Done false
7439 2903 Acked Read 2 Sent Done Done false
7440 2904 Acked Read 2 Sent Done Done false
7441 2905 Acked Write 5 Sent Done Done false
7442 2906 Acked Write 2 Sent Done Done false
7443 2907 Acked Read 8 Sent Done Done false
7444 2908 Acked Write 9 Sent Done Done false
7445 2909 Acked Write 9 Sent Done Done false
7446 2910 Acked Flush 0 Sent Done Done false
7447 2911 Acked Write 6 Sent Done Done false
7448 2912 Acked Write 3 Sent Done Done false
7449 2913 Acked Write 10 Sent Done Done false
7450 2914 Acked Read 4 Sent Done Done false
7451 2915 Acked Flush 0 Sent Done Done false
7452 2916 Acked Write 6 Sent Done Done false
7453 2917 Acked Read 6 Sent Done Done false
7454 2918 Acked Write 7 Sent Done Done false
7455 2919 Acked Write 9 Sent Done Done false
7456 2920 Acked Write 7 Sent Done Done false
7457 2921 Acked Read 1 Sent Done Done false
7458 2922 Acked Write 1 Sent Done Done false
7459 2923 Acked Read 2 Sent Done Done false
7460 2924 Acked Flush 0 Sent Done Done false
7461 2925 Acked Flush 0 Sent Done Done false
7462 2926 Acked Read 3 Sent Done Done false
7463 2927 Acked Write 6 Sent Done Done false
7464 2928 Acked Write 9 Sent Done Done false
7465 2929 Acked Read 5 Sent Done Done false
7466 2930 Acked Read 7 Sent Done Done false
7467 2931 Acked Read 5 Sent Done Done false
7468 2932 Acked Write 9 Sent Done Done false
7469 2933 Acked Write 6 Sent Done Done false
7470 2934 Acked Write 10 Sent Done Done false
7471 2935 Acked Write 6 Sent Done Done false
7472 2936 Acked Flush 0 Sent Done Done false
7473 2937 Acked Write 10 Sent Done Done false
7474 2938 Acked Write 7 Sent Done Done false
7475 2939 Acked Read 5 Sent Done Done false
7476 2940 Acked Read 9 Sent Done Done false
7477 2941 Acked Read 5 Sent Done Done false
7478 2942 Acked Flush 0 Sent Done Done false
7479 2943 Acked Write 10 Sent Done Done false
7480 2944 Acked Read 7 Sent Done Done false
7481 2945 Acked Write 8 Sent Done Done false
7482 2946 Acked Write 7 Sent Done Done false
7483 2947 Acked Flush 0 Sent Done Done false
7484 2948 Acked Read 4 Sent Done Done false
7485 2949 Acked Read 5 Sent Done Done false
7486 2950 Acked Read 7 Sent Done Done false
7487 2951 Acked Read 10 Sent Done Done false
7488 2952 Acked Read 7 Sent Done Done false
7489 2953 Acked Write 7 Sent Done Done false
7490 2954 Acked Read 7 Sent Done Done false
7491 2955 Acked Write 6 Sent Done Done false
7492 2956 Acked Read 2 Sent Done Done false
7493 2957 Acked Read 1 Sent Done Done false
7494 2958 Acked Read 6 Sent Done Done false
7495 2959 Acked Read 1 Sent Done Done false
7496 2960 Acked Read 9 Sent Done Done false
7497 2961 Acked Write 4 Sent Done Done false
7498 2962 Acked Write 5 Sent Done Done false
7499 2963 Acked Write 1 Sent Done Done false
7500 2964 Acked Write 9 Sent Done Done false
7501 2965 Acked Read 7 Sent Done Done false
7502 2966 Acked Read 2 Sent Done Done false
7503 2967 Acked Write 3 Sent Done Done false
7504 2968 Acked Write 5 Sent Done Done false
7505 2969 Acked Write 9 Sent Done Done false
7506 2970 Acked Read 10 Sent Done Done false
7507 2971 Acked Write 9 Sent Done Done false
7508 2972 Acked Read 10 Sent Done Done false
7509 2973 Acked Read 2 Sent Done Done false
7510 2974 Acked Flush 0 Sent Done Done false
7511 2975 Acked Write 8 Sent Done Done false
7512 2976 Acked Write 2 Sent Done Done false
7513 2977 Acked Write 1 Sent Done Done false
7514 2978 Acked Write 1 Sent Done Done false
7515 2979 Acked Read 2 Sent Done Done false
7516 2980 Acked Read 2 Sent Done Done false
7517 2981 Acked Read 5 Sent Done Done false
7518 2982 Acked Write 5 Sent Done Done false
7519 2983 Acked Read 4 Sent Done Done false
7520 2984 Acked Write 10 Sent Done Done false
7521 2985 Acked Read 7 Sent Done Done false
7522 2986 Acked Read 8 Sent Done Done false
7523 2987 Acked Write 10 Sent Done Done false
7524 2988 Acked Read 3 Sent Done Done false
7525 2989 Acked Read 7 Sent Done Done false
7526 2990 Acked Flush 0 Sent Done Done false
7527 2991 Acked Read 8 Sent Done Done false
7528 2992 Acked Read 2 Sent Done Done false
7529 2993 Acked Read 5 Sent Done Done false
7530 2994 Acked Write 4 Sent Done Done false
7531 2995 Acked Read 7 Sent Done Done false
7532 2996 Acked Read 4 Sent Done Done false
7533 2997 Acked Write 1 Sent Done Done false
7534 2998 Acked Write 8 Sent Done Done false
7535 2999 Acked Write 9 Sent Done Done false
7536 3000 Acked Write 9 Sent Done Done false
7537 3001 Acked Write 3 Sent Done Done false
7538 3002 Acked Write 2 Sent Done Done false
7539 3003 Acked Read 1 Sent Done Done false
7540 3004 Acked Write 6 Sent Done Done false
7541 3005 Acked Read 9 Sent Done Done false
7542 3006 Acked Read 4 Sent Done Done false
7543 3007 Acked Read 7 Sent Done Done false
7544 3008 Acked Read 4 Sent Done Done false
7545 3009 Acked Read 6 Sent Done Done false
7546 3010 Acked Write 8 Sent Done Done false
7547 3011 Acked Read 10 Sent Done Done false
7548 3012 Acked Read 3 Sent Done Done false
7549 3013 Acked Read 1 Sent Done Done false
7550 3014 Acked Read 8 Sent Done Done false
7551 3015 Acked Write 1 Sent Done Done false
7552 3016 Acked Write 1 Sent Done Done false
7553 3017 Acked Read 5 Sent Done Done false
7554 3018 Acked Read 5 Sent Done Done false
7555 3019 Acked Write 5 Sent Done Done false
7556 3020 Acked Flush 0 Sent Done Done false
7557 3021 Acked Flush 0 Sent Done Done false
7558 3022 Acked Read 6 Sent Done Done false
7559 3023 Acked Write 9 Sent Done Done false
7560 3024 Acked Read 8 Sent Done Done false
7561 3025 Acked Read 8 Sent Done Done false
7562 3026 Acked Read 4 Sent Done Done false
7563 3027 Acked Write 4 Sent Done Done false
7564 3028 Acked Write 3 Sent Done Done false
7565 3029 Acked Read 3 Sent Done Done false
7566 3030 Acked Read 8 Sent Done Done false
7567 3031 Acked Read 4 Sent Done Done false
7568 3032 Acked Read 10 Sent Done Done false
7569 3033 Acked Write 4 Sent Done Done false
7570 3034 Acked Read 5 Sent Done Done false
7571 3035 Acked Read 4 Sent Done Done false
7572 3036 Acked Read 1 Sent Done Done false
7573 3037 Acked Flush 0 Sent Done Done false
7574 3038 Acked Write 3 Sent Done Done false
7575 3039 Acked Write 9 Sent Done Done false
7576 3040 Acked Read 9 Sent Done Done false
7577 3041 Acked Write 4 Sent Done Done false
7578 3042 Acked Write 4 Sent Done Done false
7579 3043 Acked Read 2 Sent Done Done false
7580 3044 Acked Write 3 Sent Done Done false
7581 3045 Acked Write 4 Sent Done Done false
7582 3046 Acked Write 10 Sent Done Done false
7583 3047 Acked Read 8 Sent Done Done false
7584 3048 Acked Read 7 Sent Done Done false
7585 3049 Acked Read 6 Sent Done Done false
7586 3050 Acked Read 8 Sent Done Done false
7587 3051 Acked Read 2 Sent Done Done false
7588 3052 Acked Read 9 Sent Done Done false
7589 3053 Acked Write 7 Sent Done Done false
7590 3054 Acked Read 9 Sent Done Done false
7591 3055 Acked Read 1 Sent Done Done false
7592 3056 Acked Read 5 Sent Done Done false
7593 3057 Acked Write 5 Sent Done Done false
7594 3058 Acked Read 4 Sent Done Done false
7595 3059 Acked Write 1 Sent Done Done false
7596 3060 Acked Write 1 Sent Done Done false
7597 3061 Acked Read 1 Sent Done Done false
7598 3062 Acked Flush 0 Sent Done Done false
7599 3063 Acked Read 10 Sent Done Done false
7600 3064 Acked Write 10 Sent Done Done false
7601 3065 Acked Write 7 Sent Done Done false
7602 3066 Acked Read 9 Sent Done Done false
7603 3067 Acked Read 9 Sent Done Done false
7604 3068 Acked Write 8 Sent Done Done false
7605 3069 Acked Write 7 Sent Done Done false
7606 3070 Acked Write 5 Sent Done Done false
7607 3071 Acked Read 1 Sent Done Done false
7608 3072 Acked Write 7 Sent Done Done false
7609 3073 Acked Flush 0 Sent Done Done false
7610 3074 Acked Write 9 Sent Done Done false
7611 3075 Acked Write 6 Sent Done Done false
7612 3076 Acked Read 3 Sent Done Done false
7613 3077 Acked Write 10 Sent Done Done false
7614 3078 Acked Read 1 Sent Done Done false
7615 3079 Acked Read 5 Sent Done Done false
7616 3080 Acked Read 4 Sent Done Done false
7617 3081 Acked Write 10 Sent Done Done false
7618 3082 Acked Write 6 Sent Done Done false
7619 3083 Acked Read 4 Sent Done Done false
7620 3084 Acked Flush 0 Sent Done Done false
7621 3085 Acked Read 6 Sent Done Done false
7622 3086 Acked Write 3 Sent Done Done false
7623 3087 Acked Write 10 Sent Done Done false
7624 3088 Acked Read 9 Sent Done Done false
7625 3089 Acked Write 6 Sent Done Done false
7626 3090 Acked Read 5 Sent Done Done false
7627 3091 Acked Flush 0 Sent Done Done false
7628 3092 Acked Flush 0 Sent Done Done false
7629 3093 Acked Read 4 Sent Done Done false
7630 3094 Acked Read 2 Sent Done Done false
7631 3095 Acked Write 10 Sent Done Done false
7632 3096 Acked Write 2 Sent Done Done false
7633 3097 Acked Read 2 Sent Done Done false
7634 3098 Acked Flush 0 Sent Done Done false
7635 3099 Acked Write 5 Sent Done Done false
7636 3100 Acked Read 10 Sent Done Done false
7637 3101 Acked Write 9 Sent Done Done false
7638 3102 Acked Write 8 Sent Done Done false
7639 3103 Acked Write 2 Sent Done Done false
7640 3104 Acked Read 6 Sent Done Done false
7641 3105 Acked Read 1 Sent Done Done false
7642 3106 Acked Read 3 Sent Done Done false
7643 3107 Acked Write 6 Sent Done Done false
7644 3108 Acked Read 2 Sent Done Done false
7645 3109 Acked Read 1 Sent Done Done false
7646 3110 Acked Read 2 Sent Done Done false
7647 3111 Acked Write 5 Sent Done Done false
7648 3112 Acked Write 8 Sent Done Done false
7649 3113 Acked Flush 0 Sent Done Done false
7650 STATES DS:0 DS:1 DS:2 TOTAL
7651 Sent 302 0 0 302
7652 Done 0 302 302 604
7653 Skipped 0 0 0 0
7654 Error 0 0 0 0
7655 Last Flush: 2811 3113 3113
7656 Downstairs last five completed: 2811 2810 2809 2808 2807
7657 Upstairs last five completed: 3113 3112 3111 3110 3109
7658 CLIENT: Up:0 ds:302 act:2
76592024-12-11T23:42:06.011ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
76602024-12-11T23:42:06.012ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
76612024-12-11T23:42:06.012ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
76622024-12-11T23:42:06.012ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
76632024-12-11T23:42:06.012ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
7664 [8810] got start action
7665 [8810] I am going to restart
7666 Starting downstairs at port 8810
7667 Make output file at "/tmp/dsc/downstairs-8810.txt"
7668 [8810][0] reports Starting
7669 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
7670 [8810][0] reports Running
7671 ----------------------------------------------------------------
7672 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7673 States: Offline Active Active
7674 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7675 2812 Acked Write 3 Sent Done Done false
7676 2813 Acked Read 9 Sent Done Done false
7677 2814 Acked Write 2 Sent Done Done false
7678 2815 Acked Read 3 Sent Done Done false
7679 2816 Acked Write 5 Sent Done Done false
7680 2817 Acked Read 4 Sent Done Done false
7681 2818 Acked Write 3 Sent Done Done false
7682 2819 Acked Read 1 Sent Done Done false
7683 2820 Acked Read 6 Sent Done Done false
7684 2821 Acked Write 6 Sent Done Done false
7685 2822 Acked Flush 0 Sent Done Done false
7686 2823 Acked Read 10 Sent Done Done false
7687 2824 Acked Write 3 Sent Done Done false
7688 2825 Acked Flush 0 Sent Done Done false
7689 2826 Acked Read 2 Sent Done Done false
7690 2827 Acked Write 8 Sent Done Done false
7691 2828 Acked Flush 0 Sent Done Done false
7692 2829 Acked Read 10 Sent Done Done false
7693 2830 Acked Flush 0 Sent Done Done false
7694 2831 Acked Read 7 Sent Done Done false
7695 2832 Acked Read 9 Sent Done Done false
7696 2833 Acked Write 9 Sent Done Done false
7697 2834 Acked Write 9 Sent Done Done false
7698 2835 Acked Write 1 Sent Done Done false
7699 2836 Acked Flush 0 Sent Done Done false
7700 2837 Acked Read 9 Sent Done Done false
7701 2838 Acked Read 6 Sent Done Done false
7702 2839 Acked Write 3 Sent Done Done false
7703 2840 Acked Read 10 Sent Done Done false
7704 2841 Acked Read 5 Sent Done Done false
7705 2842 Acked Read 7 Sent Done Done false
7706 2843 Acked Write 7 Sent Done Done false
7707 2844 Acked Write 4 Sent Done Done false
7708 2845 Acked Flush 0 Sent Done Done false
7709 2846 Acked Write 3 Sent Done Done false
7710 2847 Acked Read 8 Sent Done Done false
7711 2848 Acked Read 6 Sent Done Done false
7712 2849 Acked Read 8 Sent Done Done false
7713 2850 Acked Write 6 Sent Done Done false
7714 2851 Acked Flush 0 Sent Done Done false
7715 2852 Acked Read 10 Sent Done Done false
7716 2853 Acked Write 3 Sent Done Done false
7717 2854 Acked Write 6 Sent Done Done false
7718 2855 Acked Write 3 Sent Done Done false
7719 2856 Acked Read 1 Sent Done Done false
7720 2857 Acked Read 5 Sent Done Done false
7721 2858 Acked Read 3 Sent Done Done false
7722 2859 Acked Flush 0 Sent Done Done false
7723 2860 Acked Read 2 Sent Done Done false
7724 2861 Acked Write 6 Sent Done Done false
7725 2862 Acked Read 3 Sent Done Done false
7726 2863 Acked Read 3 Sent Done Done false
7727 2864 Acked Read 3 Sent Done Done false
7728 2865 Acked Write 4 Sent Done Done false
7729 2866 Acked Write 4 Sent Done Done false
7730 2867 Acked Write 7 Sent Done Done false
7731 2868 Acked Write 6 Sent Done Done false
7732 2869 Acked Read 10 Sent Done Done false
7733 2870 Acked Read 8 Sent Done Done false
7734 2871 Acked Read 5 Sent Done Done false
7735 2872 Acked Read 4 Sent Done Done false
7736 2873 Acked Write 3 Sent Done Done false
7737 2874 Acked Read 3 Sent Done Done false
7738 2875 Acked Flush 0 Sent Done Done false
7739 2876 Acked Read 1 Sent Done Done false
7740 2877 Acked Write 10 Sent Done Done false
7741 2878 Acked Read 4 Sent Done Done false
7742 2879 Acked Read 9 Sent Done Done false
7743 2880 Acked Read 8 Sent Done Done false
7744 2881 Acked Read 1 Sent Done Done false
7745 2882 Acked Read 6 Sent Done Done false
7746 2883 Acked Read 9 Sent Done Done false
7747 2884 Acked Flush 0 Sent Done Done false
7748 2885 Acked Read 4 Sent Done Done false
7749 2886 Acked Read 2 Sent Done Done false
7750 2887 Acked Read 6 Sent Done Done false
7751 2888 Acked Read 8 Sent Done Done false
7752 2889 Acked Write 6 Sent Done Done false
7753 2890 Acked Read 4 Sent Done Done false
7754 2891 Acked Write 1 Sent Done Done false
7755 2892 Acked Write 4 Sent Done Done false
7756 2893 Acked Write 4 Sent Done Done false
7757 2894 Acked Write 7 Sent Done Done false
7758 2895 Acked Read 2 Sent Done Done false
7759 2896 Acked Flush 0 Sent Done Done false
7760 2897 Acked Flush 0 Sent Done Done false
7761 2898 Acked Read 6 Sent Done Done false
7762 2899 Acked Flush 0 Sent Done Done false
7763 2900 Acked Read 6 Sent Done Done false
7764 2901 Acked Flush 0 Sent Done Done false
7765 2902 Acked Read 5 Sent Done Done false
7766 2903 Acked Read 2 Sent Done Done false
7767 2904 Acked Read 2 Sent Done Done false
7768 2905 Acked Write 5 Sent Done Done false
7769 2906 Acked Write 2 Sent Done Done false
7770 2907 Acked Read 8 Sent Done Done false
7771 2908 Acked Write 9 Sent Done Done false
7772 2909 Acked Write 9 Sent Done Done false
7773 2910 Acked Flush 0 Sent Done Done false
7774 2911 Acked Write 6 Sent Done Done false
7775 2912 Acked Write 3 Sent Done Done false
7776 2913 Acked Write 10 Sent Done Done false
7777 2914 Acked Read 4 Sent Done Done false
7778 2915 Acked Flush 0 Sent Done Done false
7779 2916 Acked Write 6 Sent Done Done false
7780 2917 Acked Read 6 Sent Done Done false
7781 2918 Acked Write 7 Sent Done Done false
7782 2919 Acked Write 9 Sent Done Done false
7783 2920 Acked Write 7 Sent Done Done false
7784 2921 Acked Read 1 Sent Done Done false
7785 2922 Acked Write 1 Sent Done Done false
7786 2923 Acked Read 2 Sent Done Done false
7787 2924 Acked Flush 0 Sent Done Done false
7788 2925 Acked Flush 0 Sent Done Done false
7789 2926 Acked Read 3 Sent Done Done false
7790 2927 Acked Write 6 Sent Done Done false
7791 2928 Acked Write 9 Sent Done Done false
7792 2929 Acked Read 5 Sent Done Done false
7793 2930 Acked Read 7 Sent Done Done false
7794 2931 Acked Read 5 Sent Done Done false
7795 2932 Acked Write 9 Sent Done Done false
7796 2933 Acked Write 6 Sent Done Done false
7797 2934 Acked Write 10 Sent Done Done false
7798 2935 Acked Write 6 Sent Done Done false
7799 2936 Acked Flush 0 Sent Done Done false
7800 2937 Acked Write 10 Sent Done Done false
7801 2938 Acked Write 7 Sent Done Done false
7802 2939 Acked Read 5 Sent Done Done false
7803 2940 Acked Read 9 Sent Done Done false
7804 2941 Acked Read 5 Sent Done Done false
7805 2942 Acked Flush 0 Sent Done Done false
7806 2943 Acked Write 10 Sent Done Done false
7807 2944 Acked Read 7 Sent Done Done false
7808 2945 Acked Write 8 Sent Done Done false
7809 2946 Acked Write 7 Sent Done Done false
7810 2947 Acked Flush 0 Sent Done Done false
7811 2948 Acked Read 4 Sent Done Done false
7812 2949 Acked Read 5 Sent Done Done false
7813 2950 Acked Read 7 Sent Done Done false
7814 2951 Acked Read 10 Sent Done Done false
7815 2952 Acked Read 7 Sent Done Done false
7816 2953 Acked Write 7 Sent Done Done false
7817 2954 Acked Read 7 Sent Done Done false
7818 2955 Acked Write 6 Sent Done Done false
7819 2956 Acked Read 2 Sent Done Done false
7820 2957 Acked Read 1 Sent Done Done false
7821 2958 Acked Read 6 Sent Done Done false
7822 2959 Acked Read 1 Sent Done Done false
7823 2960 Acked Read 9 Sent Done Done false
7824 2961 Acked Write 4 Sent Done Done false
7825 2962 Acked Write 5 Sent Done Done false
7826 2963 Acked Write 1 Sent Done Done false
7827 2964 Acked Write 9 Sent Done Done false
7828 2965 Acked Read 7 Sent Done Done false
7829 2966 Acked Read 2 Sent Done Done false
7830 2967 Acked Write 3 Sent Done Done false
7831 2968 Acked Write 5 Sent Done Done false
7832 2969 Acked Write 9 Sent Done Done false
7833 2970 Acked Read 10 Sent Done Done false
7834 2971 Acked Write 9 Sent Done Done false
7835 2972 Acked Read 10 Sent Done Done false
7836 2973 Acked Read 2 Sent Done Done false
7837 2974 Acked Flush 0 Sent Done Done false
7838 2975 Acked Write 8 Sent Done Done false
7839 2976 Acked Write 2 Sent Done Done false
7840 2977 Acked Write 1 Sent Done Done false
7841 2978 Acked Write 1 Sent Done Done false
7842 2979 Acked Read 2 Sent Done Done false
7843 2980 Acked Read 2 Sent Done Done false
7844 2981 Acked Read 5 Sent Done Done false
7845 2982 Acked Write 5 Sent Done Done false
7846 2983 Acked Read 4 Sent Done Done false
7847 2984 Acked Write 10 Sent Done Done false
7848 2985 Acked Read 7 Sent Done Done false
7849 2986 Acked Read 8 Sent Done Done false
7850 2987 Acked Write 10 Sent Done Done false
7851 2988 Acked Read 3 Sent Done Done false
7852 2989 Acked Read 7 Sent Done Done false
7853 2990 Acked Flush 0 Sent Done Done false
7854 2991 Acked Read 8 Sent Done Done false
7855 2992 Acked Read 2 Sent Done Done false
7856 2993 Acked Read 5 Sent Done Done false
7857 2994 Acked Write 4 Sent Done Done false
7858 2995 Acked Read 7 Sent Done Done false
7859 2996 Acked Read 4 Sent Done Done false
7860 2997 Acked Write 1 Sent Done Done false
7861 2998 Acked Write 8 Sent Done Done false
7862 2999 Acked Write 9 Sent Done Done false
7863 3000 Acked Write 9 Sent Done Done false
7864 3001 Acked Write 3 Sent Done Done false
7865 3002 Acked Write 2 Sent Done Done false
7866 3003 Acked Read 1 Sent Done Done false
7867 3004 Acked Write 6 Sent Done Done false
7868 3005 Acked Read 9 Sent Done Done false
7869 3006 Acked Read 4 Sent Done Done false
7870 3007 Acked Read 7 Sent Done Done false
7871 3008 Acked Read 4 Sent Done Done false
7872 3009 Acked Read 6 Sent Done Done false
7873 3010 Acked Write 8 Sent Done Done false
7874 3011 Acked Read 10 Sent Done Done false
7875 3012 Acked Read 3 Sent Done Done false
7876 3013 Acked Read 1 Sent Done Done false
7877 3014 Acked Read 8 Sent Done Done false
7878 3015 Acked Write 1 Sent Done Done false
7879 3016 Acked Write 1 Sent Done Done false
7880 3017 Acked Read 5 Sent Done Done false
7881 3018 Acked Read 5 Sent Done Done false
7882 3019 Acked Write 5 Sent Done Done false
7883 3020 Acked Flush 0 Sent Done Done false
7884 3021 Acked Flush 0 Sent Done Done false
7885 3022 Acked Read 6 Sent Done Done false
7886 3023 Acked Write 9 Sent Done Done false
7887 3024 Acked Read 8 Sent Done Done false
7888 3025 Acked Read 8 Sent Done Done false
7889 3026 Acked Read 4 Sent Done Done false
7890 3027 Acked Write 4 Sent Done Done false
7891 3028 Acked Write 3 Sent Done Done false
7892 3029 Acked Read 3 Sent Done Done false
7893 3030 Acked Read 8 Sent Done Done false
7894 3031 Acked Read 4 Sent Done Done false
7895 3032 Acked Read 10 Sent Done Done false
7896 3033 Acked Write 4 Sent Done Done false
7897 3034 Acked Read 5 Sent Done Done false
7898 3035 Acked Read 4 Sent Done Done false
7899 3036 Acked Read 1 Sent Done Done false
7900 3037 Acked Flush 0 Sent Done Done false
7901 3038 Acked Write 3 Sent Done Done false
7902 3039 Acked Write 9 Sent Done Done false
7903 3040 Acked Read 9 Sent Done Done false
7904 3041 Acked Write 4 Sent Done Done false
7905 3042 Acked Write 4 Sent Done Done false
7906 3043 Acked Read 2 Sent Done Done false
7907 3044 Acked Write 3 Sent Done Done false
7908 3045 Acked Write 4 Sent Done Done false
7909 3046 Acked Write 10 Sent Done Done false
7910 3047 Acked Read 8 Sent Done Done false
7911 3048 Acked Read 7 Sent Done Done false
7912 3049 Acked Read 6 Sent Done Done false
7913 3050 Acked Read 8 Sent Done Done false
7914 3051 Acked Read 2 Sent Done Done false
7915 3052 Acked Read 9 Sent Done Done false
7916 3053 Acked Write 7 Sent Done Done false
7917 3054 Acked Read 9 Sent Done Done false
7918 3055 Acked Read 1 Sent Done Done false
7919 3056 Acked Read 5 Sent Done Done false
7920 3057 Acked Write 5 Sent Done Done false
7921 3058 Acked Read 4 Sent Done Done false
7922 3059 Acked Write 1 Sent Done Done false
7923 3060 Acked Write 1 Sent Done Done false
7924 3061 Acked Read 1 Sent Done Done false
7925 3062 Acked Flush 0 Sent Done Done false
7926 3063 Acked Read 10 Sent Done Done false
7927 3064 Acked Write 10 Sent Done Done false
7928 3065 Acked Write 7 Sent Done Done false
7929 3066 Acked Read 9 Sent Done Done false
7930 3067 Acked Read 9 Sent Done Done false
7931 3068 Acked Write 8 Sent Done Done false
7932 3069 Acked Write 7 Sent Done Done false
7933 3070 Acked Write 5 Sent Done Done false
7934 3071 Acked Read 1 Sent Done Done false
7935 3072 Acked Write 7 Sent Done Done false
7936 3073 Acked Flush 0 Sent Done Done false
7937 3074 Acked Write 9 Sent Done Done false
7938 3075 Acked Write 6 Sent Done Done false
7939 3076 Acked Read 3 Sent Done Done false
7940 3077 Acked Write 10 Sent Done Done false
7941 3078 Acked Read 1 Sent Done Done false
7942 3079 Acked Read 5 Sent Done Done false
7943 3080 Acked Read 4 Sent Done Done false
7944 3081 Acked Write 10 Sent Done Done false
7945 3082 Acked Write 6 Sent Done Done false
7946 3083 Acked Read 4 Sent Done Done false
7947 3084 Acked Flush 0 Sent Done Done false
7948 3085 Acked Read 6 Sent Done Done false
7949 3086 Acked Write 3 Sent Done Done false
7950 3087 Acked Write 10 Sent Done Done false
7951 3088 Acked Read 9 Sent Done Done false
7952 3089 Acked Write 6 Sent Done Done false
7953 3090 Acked Read 5 Sent Done Done false
7954 3091 Acked Flush 0 Sent Done Done false
7955 3092 Acked Flush 0 Sent Done Done false
7956 3093 Acked Read 4 Sent Done Done false
7957 3094 Acked Read 2 Sent Done Done false
7958 3095 Acked Write 10 Sent Done Done false
7959 3096 Acked Write 2 Sent Done Done false
7960 3097 Acked Read 2 Sent Done Done false
7961 3098 Acked Flush 0 Sent Done Done false
7962 3099 Acked Write 5 Sent Done Done false
7963 3100 Acked Read 10 Sent Done Done false
7964 3101 Acked Write 9 Sent Done Done false
7965 3102 Acked Write 8 Sent Done Done false
7966 3103 Acked Write 2 Sent Done Done false
7967 3104 Acked Read 6 Sent Done Done false
7968 3105 Acked Read 1 Sent Done Done false
7969 3106 Acked Read 3 Sent Done Done false
7970 3107 Acked Write 6 Sent Done Done false
7971 3108 Acked Read 2 Sent Done Done false
7972 3109 Acked Read 1 Sent Done Done false
7973 3110 Acked Read 2 Sent Done Done false
7974 3111 Acked Write 5 Sent Done Done false
7975 3112 Acked Write 8 Sent Done Done false
7976 3113 Acked Flush 0 Sent Done Done false
7977 STATES DS:0 DS:1 DS:2 TOTAL
7978 Sent 302 0 0 302
7979 Done 0 302 302 604
7980 Skipped 0 0 0 0
7981 Error 0 0 0 0
7982 Last Flush: 2811 3113 3113
7983 Downstairs last five completed: 2811 2810 2809 2808 2807
7984 Upstairs last five completed: 3113 3112 3111 3110 3109
7985 CLIENT: Up:0 ds:302 act:2
7986 ----------------------------------------------------------------
7987 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7988 States: Offline Active Active
7989 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7990 2812 Acked Write 3 Sent Done Done false
7991 2813 Acked Read 9 Sent Done Done false
7992 2814 Acked Write 2 Sent Done Done false
7993 2815 Acked Read 3 Sent Done Done false
7994 2816 Acked Write 5 Sent Done Done false
7995 2817 Acked Read 4 Sent Done Done false
7996 2818 Acked Write 3 Sent Done Done false
7997 2819 Acked Read 1 Sent Done Done false
7998 2820 Acked Read 6 Sent Done Done false
7999 2821 Acked Write 6 Sent Done Done false
8000 2822 Acked Flush 0 Sent Done Done false
8001 2823 Acked Read 10 Sent Done Done false
8002 2824 Acked Write 3 Sent Done Done false
8003 2825 Acked Flush 0 Sent Done Done false
8004 2826 Acked Read 2 Sent Done Done false
8005 2827 Acked Write 8 Sent Done Done false
8006 2828 Acked Flush 0 Sent Done Done false
8007 2829 Acked Read 10 Sent Done Done false
8008 2830 Acked Flush 0 Sent Done Done false
8009 2831 Acked Read 7 Sent Done Done false
8010 2832 Acked Read 9 Sent Done Done false
8011 2833 Acked Write 9 Sent Done Done false
8012 2834 Acked Write 9 Sent Done Done false
8013 2835 Acked Write 1 Sent Done Done false
8014 2836 Acked Flush 0 Sent Done Done false
8015 2837 Acked Read 9 Sent Done Done false
8016 2838 Acked Read 6 Sent Done Done false
8017 2839 Acked Write 3 Sent Done Done false
8018 2840 Acked Read 10 Sent Done Done false
8019 2841 Acked Read 5 Sent Done Done false
8020 2842 Acked Read 7 Sent Done Done false
8021 2843 Acked Write 7 Sent Done Done false
8022 2844 Acked Write 4 Sent Done Done false
8023 2845 Acked Flush 0 Sent Done Done false
8024 2846 Acked Write 3 Sent Done Done false
8025 2847 Acked Read 8 Sent Done Done false
8026 2848 Acked Read 6 Sent Done Done false
8027 2849 Acked Read 8 Sent Done Done false
8028 2850 Acked Write 6 Sent Done Done false
8029 2851 Acked Flush 0 Sent Done Done false
8030 2852 Acked Read 10 Sent Done Done false
8031 2853 Acked Write 3 Sent Done Done false
8032 2854 Acked Write 6 Sent Done Done false
8033 2855 Acked Write 3 Sent Done Done false
8034 2856 Acked Read 1 Sent Done Done false
8035 2857 Acked Read 5 Sent Done Done false
8036 2858 Acked Read 3 Sent Done Done false
8037 2859 Acked Flush 0 Sent Done Done false
8038 2860 Acked Read 2 Sent Done Done false
8039 2861 Acked Write 6 Sent Done Done false
8040 2862 Acked Read 3 Sent Done Done false
8041 2863 Acked Read 3 Sent Done Done false
8042 2864 Acked Read 3 Sent Done Done false
8043 2865 Acked Write 4 Sent Done Done false
8044 2866 Acked Write 4 Sent Done Done false
8045 2867 Acked Write 7 Sent Done Done false
8046 2868 Acked Write 6 Sent Done Done false
8047 2869 Acked Read 10 Sent Done Done false
8048 2870 Acked Read 8 Sent Done Done false
8049 2871 Acked Read 5 Sent Done Done false
8050 2872 Acked Read 4 Sent Done Done false
8051 2873 Acked Write 3 Sent Done Done false
8052 2874 Acked Read 3 Sent Done Done false
8053 2875 Acked Flush 0 Sent Done Done false
8054 2876 Acked Read 1 Sent Done Done false
8055 2877 Acked Write 10 Sent Done Done false
8056 2878 Acked Read 4 Sent Done Done false
8057 2879 Acked Read 9 Sent Done Done false
8058 2880 Acked Read 8 Sent Done Done false
8059 2881 Acked Read 1 Sent Done Done false
8060 2882 Acked Read 6 Sent Done Done false
8061 2883 Acked Read 9 Sent Done Done false
8062 2884 Acked Flush 0 Sent Done Done false
8063 2885 Acked Read 4 Sent Done Done false
8064 2886 Acked Read 2 Sent Done Done false
8065 2887 Acked Read 6 Sent Done Done false
8066 2888 Acked Read 8 Sent Done Done false
8067 2889 Acked Write 6 Sent Done Done false
8068 2890 Acked Read 4 Sent Done Done false
8069 2891 Acked Write 1 Sent Done Done false
8070 2892 Acked Write 4 Sent Done Done false
8071 2893 Acked Write 4 Sent Done Done false
8072 2894 Acked Write 7 Sent Done Done false
8073 2895 Acked Read 2 Sent Done Done false
8074 2896 Acked Flush 0 Sent Done Done false
8075 2897 Acked Flush 0 Sent Done Done false
8076 2898 Acked Read 6 Sent Done Done false
8077 2899 Acked Flush 0 Sent Done Done false
8078 2900 Acked Read 6 Sent Done Done false
8079 2901 Acked Flush 0 Sent Done Done false
8080 2902 Acked Read 5 Sent Done Done false
8081 2903 Acked Read 2 Sent Done Done false
8082 2904 Acked Read 2 Sent Done Done false
8083 2905 Acked Write 5 Sent Done Done false
8084 2906 Acked Write 2 Sent Done Done false
8085 2907 Acked Read 8 Sent Done Done false
8086 2908 Acked Write 9 Sent Done Done false
8087 2909 Acked Write 9 Sent Done Done false
8088 2910 Acked Flush 0 Sent Done Done false
8089 2911 Acked Write 6 Sent Done Done false
8090 2912 Acked Write 3 Sent Done Done false
8091 2913 Acked Write 10 Sent Done Done false
8092 2914 Acked Read 4 Sent Done Done false
8093 2915 Acked Flush 0 Sent Done Done false
8094 2916 Acked Write 6 Sent Done Done false
8095 2917 Acked Read 6 Sent Done Done false
8096 2918 Acked Write 7 Sent Done Done false
8097 2919 Acked Write 9 Sent Done Done false
8098 2920 Acked Write 7 Sent Done Done false
8099 2921 Acked Read 1 Sent Done Done false
8100 2922 Acked Write 1 Sent Done Done false
8101 2923 Acked Read 2 Sent Done Done false
8102 2924 Acked Flush 0 Sent Done Done false
8103 2925 Acked Flush 0 Sent Done Done false
8104 2926 Acked Read 3 Sent Done Done false
8105 2927 Acked Write 6 Sent Done Done false
8106 2928 Acked Write 9 Sent Done Done false
8107 2929 Acked Read 5 Sent Done Done false
8108 2930 Acked Read 7 Sent Done Done false
8109 2931 Acked Read 5 Sent Done Done false
8110 2932 Acked Write 9 Sent Done Done false
8111 2933 Acked Write 6 Sent Done Done false
8112 2934 Acked Write 10 Sent Done Done false
8113 2935 Acked Write 6 Sent Done Done false
8114 2936 Acked Flush 0 Sent Done Done false
8115 2937 Acked Write 10 Sent Done Done false
8116 2938 Acked Write 7 Sent Done Done false
8117 2939 Acked Read 5 Sent Done Done false
8118 2940 Acked Read 9 Sent Done Done false
8119 2941 Acked Read 5 Sent Done Done false
8120 2942 Acked Flush 0 Sent Done Done false
8121 2943 Acked Write 10 Sent Done Done false
8122 2944 Acked Read 7 Sent Done Done false
8123 2945 Acked Write 8 Sent Done Done false
8124 2946 Acked Write 7 Sent Done Done false
8125 2947 Acked Flush 0 Sent Done Done false
8126 2948 Acked Read 4 Sent Done Done false
8127 2949 Acked Read 5 Sent Done Done false
8128 2950 Acked Read 7 Sent Done Done false
8129 2951 Acked Read 10 Sent Done Done false
8130 2952 Acked Read 7 Sent Done Done false
8131 2953 Acked Write 7 Sent Done Done false
8132 2954 Acked Read 7 Sent Done Done false
8133 2955 Acked Write 6 Sent Done Done false
8134 2956 Acked Read 2 Sent Done Done false
8135 2957 Acked Read 1 Sent Done Done false
8136 2958 Acked Read 6 Sent Done Done false
8137 2959 Acked Read 1 Sent Done Done false
8138 2960 Acked Read 9 Sent Done Done false
8139 2961 Acked Write 4 Sent Done Done false
8140 2962 Acked Write 5 Sent Done Done false
8141 2963 Acked Write 1 Sent Done Done false
8142 2964 Acked Write 9 Sent Done Done false
8143 2965 Acked Read 7 Sent Done Done false
8144 2966 Acked Read 2 Sent Done Done false
8145 2967 Acked Write 3 Sent Done Done false
8146 2968 Acked Write 5 Sent Done Done false
8147 2969 Acked Write 9 Sent Done Done false
8148 2970 Acked Read 10 Sent Done Done false
8149 2971 Acked Write 9 Sent Done Done false
8150 2972 Acked Read 10 Sent Done Done false
8151 2973 Acked Read 2 Sent Done Done false
8152 2974 Acked Flush 0 Sent Done Done false
8153 2975 Acked Write 8 Sent Done Done false
8154 2976 Acked Write 2 Sent Done Done false
8155 2977 Acked Write 1 Sent Done Done false
8156 2978 Acked Write 1 Sent Done Done false
8157 2979 Acked Read 2 Sent Done Done false
8158 2980 Acked Read 2 Sent Done Done false
8159 2981 Acked Read 5 Sent Done Done false
8160 2982 Acked Write 5 Sent Done Done false
8161 2983 Acked Read 4 Sent Done Done false
8162 2984 Acked Write 10 Sent Done Done false
8163 2985 Acked Read 7 Sent Done Done false
8164 2986 Acked Read 8 Sent Done Done false
8165 2987 Acked Write 10 Sent Done Done false
8166 2988 Acked Read 3 Sent Done Done false
8167 2989 Acked Read 7 Sent Done Done false
8168 2990 Acked Flush 0 Sent Done Done false
8169 2991 Acked Read 8 Sent Done Done false
8170 2992 Acked Read 2 Sent Done Done false
8171 2993 Acked Read 5 Sent Done Done false
8172 2994 Acked Write 4 Sent Done Done false
8173 2995 Acked Read 7 Sent Done Done false
8174 2996 Acked Read 4 Sent Done Done false
8175 2997 Acked Write 1 Sent Done Done false
8176 2998 Acked Write 8 Sent Done Done false
8177 2999 Acked Write 9 Sent Done Done false
8178 3000 Acked Write 9 Sent Done Done false
8179 3001 Acked Write 3 Sent Done Done false
8180 3002 Acked Write 2 Sent Done Done false
8181 3003 Acked Read 1 Sent Done Done false
8182 3004 Acked Write 6 Sent Done Done false
8183 3005 Acked Read 9 Sent Done Done false
8184 3006 Acked Read 4 Sent Done Done false
8185 3007 Acked Read 7 Sent Done Done false
8186 3008 Acked Read 4 Sent Done Done false
8187 3009 Acked Read 6 Sent Done Done false
8188 3010 Acked Write 8 Sent Done Done false
8189 3011 Acked Read 10 Sent Done Done false
8190 3012 Acked Read 3 Sent Done Done false
8191 3013 Acked Read 1 Sent Done Done false
8192 3014 Acked Read 8 Sent Done Done false
8193 3015 Acked Write 1 Sent Done Done false
8194 3016 Acked Write 1 Sent Done Done false
8195 3017 Acked Read 5 Sent Done Done false
8196 3018 Acked Read 5 Sent Done Done false
8197 3019 Acked Write 5 Sent Done Done false
8198 3020 Acked Flush 0 Sent Done Done false
8199 3021 Acked Flush 0 Sent Done Done false
8200 3022 Acked Read 6 Sent Done Done false
8201 3023 Acked Write 9 Sent Done Done false
8202 3024 Acked Read 8 Sent Done Done false
8203 3025 Acked Read 8 Sent Done Done false
8204 3026 Acked Read 4 Sent Done Done false
8205 3027 Acked Write 4 Sent Done Done false
8206 3028 Acked Write 3 Sent Done Done false
8207 3029 Acked Read 3 Sent Done Done false
8208 3030 Acked Read 8 Sent Done Done false
8209 3031 Acked Read 4 Sent Done Done false
8210 3032 Acked Read 10 Sent Done Done false
8211 3033 Acked Write 4 Sent Done Done false
8212 3034 Acked Read 5 Sent Done Done false
8213 3035 Acked Read 4 Sent Done Done false
8214 3036 Acked Read 1 Sent Done Done false
8215 3037 Acked Flush 0 Sent Done Done false
8216 3038 Acked Write 3 Sent Done Done false
8217 3039 Acked Write 9 Sent Done Done false
8218 3040 Acked Read 9 Sent Done Done false
8219 3041 Acked Write 4 Sent Done Done false
8220 3042 Acked Write 4 Sent Done Done false
8221 3043 Acked Read 2 Sent Done Done false
8222 3044 Acked Write 3 Sent Done Done false
8223 3045 Acked Write 4 Sent Done Done false
8224 3046 Acked Write 10 Sent Done Done false
8225 3047 Acked Read 8 Sent Done Done false
8226 3048 Acked Read 7 Sent Done Done false
8227 3049 Acked Read 6 Sent Done Done false
8228 3050 Acked Read 8 Sent Done Done false
8229 3051 Acked Read 2 Sent Done Done false
8230 3052 Acked Read 9 Sent Done Done false
8231 3053 Acked Write 7 Sent Done Done false
8232 3054 Acked Read 9 Sent Done Done false
8233 3055 Acked Read 1 Sent Done Done false
8234 3056 Acked Read 5 Sent Done Done false
8235 3057 Acked Write 5 Sent Done Done false
8236 3058 Acked Read 4 Sent Done Done false
8237 3059 Acked Write 1 Sent Done Done false
8238 3060 Acked Write 1 Sent Done Done false
8239 3061 Acked Read 1 Sent Done Done false
8240 3062 Acked Flush 0 Sent Done Done false
8241 3063 Acked Read 10 Sent Done Done false
8242 3064 Acked Write 10 Sent Done Done false
8243 3065 Acked Write 7 Sent Done Done false
8244 3066 Acked Read 9 Sent Done Done false
8245 3067 Acked Read 9 Sent Done Done false
8246 3068 Acked Write 8 Sent Done Done false
8247 3069 Acked Write 7 Sent Done Done false
8248 3070 Acked Write 5 Sent Done Done false
8249 3071 Acked Read 1 Sent Done Done false
8250 3072 Acked Write 7 Sent Done Done false
8251 3073 Acked Flush 0 Sent Done Done false
8252 3074 Acked Write 9 Sent Done Done false
8253 3075 Acked Write 6 Sent Done Done false
8254 3076 Acked Read 3 Sent Done Done false
8255 3077 Acked Write 10 Sent Done Done false
8256 3078 Acked Read 1 Sent Done Done false
8257 3079 Acked Read 5 Sent Done Done false
8258 3080 Acked Read 4 Sent Done Done false
8259 3081 Acked Write 10 Sent Done Done false
8260 3082 Acked Write 6 Sent Done Done false
8261 3083 Acked Read 4 Sent Done Done false
8262 3084 Acked Flush 0 Sent Done Done false
8263 3085 Acked Read 6 Sent Done Done false
8264 3086 Acked Write 3 Sent Done Done false
8265 3087 Acked Write 10 Sent Done Done false
8266 3088 Acked Read 9 Sent Done Done false
8267 3089 Acked Write 6 Sent Done Done false
8268 3090 Acked Read 5 Sent Done Done false
8269 3091 Acked Flush 0 Sent Done Done false
8270 3092 Acked Flush 0 Sent Done Done false
8271 3093 Acked Read 4 Sent Done Done false
8272 3094 Acked Read 2 Sent Done Done false
8273 3095 Acked Write 10 Sent Done Done false
8274 3096 Acked Write 2 Sent Done Done false
8275 3097 Acked Read 2 Sent Done Done false
8276 3098 Acked Flush 0 Sent Done Done false
8277 3099 Acked Write 5 Sent Done Done false
8278 3100 Acked Read 10 Sent Done Done false
8279 3101 Acked Write 9 Sent Done Done false
8280 3102 Acked Write 8 Sent Done Done false
8281 3103 Acked Write 2 Sent Done Done false
8282 3104 Acked Read 6 Sent Done Done false
8283 3105 Acked Read 1 Sent Done Done false
8284 3106 Acked Read 3 Sent Done Done false
8285 3107 Acked Write 6 Sent Done Done false
8286 3108 Acked Read 2 Sent Done Done false
8287 3109 Acked Read 1 Sent Done Done false
8288 3110 Acked Read 2 Sent Done Done false
8289 3111 Acked Write 5 Sent Done Done false
8290 3112 Acked Write 8 Sent Done Done false
8291 3113 Acked Flush 0 Sent Done Done false
8292 STATES DS:0 DS:1 DS:2 TOTAL
8293 Sent 302 0 0 302
8294 Done 0 302 302 604
8295 Skipped 0 0 0 0
8296 Error 0 0 0 0
8297 Last Flush: 2811 3113 3113
8298 Downstairs last five completed: 2811 2810 2809 2808 2807
8299 Upstairs last five completed: 3113 3112 3111 3110 3109
8300 CLIENT: Up:0 ds:302 act:2
83012024-12-11T23:42:16.012ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83022024-12-11T23:42:16.012ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46303) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83032024-12-11T23:42:16.013ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83042024-12-11T23:42:16.014ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83052024-12-11T23:42:16.014ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83062024-12-11T23:42:16.014ZINFOcrucible: send last flush ID to this DS: 2811 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83072024-12-11T23:42:16.014ZINFOcrucible: Replied this last flush ID: 2811 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83082024-12-11T23:42:16.014ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83092024-12-11T23:42:16.014ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83102024-12-11T23:42:16.014ZINFOcrucible: [0] client re-new 302 jobs since flush 2811 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83112024-12-11T23:42:16.014ZINFOcrucible: [0] Replayed 302 jobs since flush: 2811 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
8312 ----------------------------------------------------------------
8313 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8314 Downstairs last five completed: 3113 3112 3111 3110 3109
8315 Upstairs last five completed: 3113 3112 3111 3110 3109
8316 CLIENT: Up:0 ds:0 act:3
8317 Replay: All jobs finished
8318 Main task has work to do, go find it
8319 got dsc Stop(0)
8320 stop 0
8321 Dec 11 23:42:16.259 [8810] Got stop action so:false kr:false
8322 INFO request completed, latency_us: 327, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 3a4643fc-1bec-43a6-a8fc-58445d4c22d2, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
83232024-12-11T23:42:16.260ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
8324 Dec 11 23:42:16.260 INFO request completed, latency_us: 125, response_code: 200, uri: /state/cid/0, method: GET{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:42:16.260420406Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
8325 , req_id: 9ec2c944-e67d-4328-bfaa-4d1d22f74db3, remote_addr: {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:42:16.260455537Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
8326 127.0.0.1:62969, local_addr: 127{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:42:16.260485008Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
8327 .0.0.1:9998
83282024-12-11T23:42:16.260ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
83292024-12-11T23:42:16.260ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
8330 [8810] Exited with: ExitStatus(unix_wait_status(9))
8331 [8810][0] reports Exit
8332 Dec 11 23:42:20.262 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/0, method: GET, req_id: e33a8887-6087-47ea-a933-a9ad125816f5, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
8333 001/300 Read block 2590 len 40960
8334 002/300 Read block 3355 len 40960
8335 003/300 Write block 2364 len 32768 data: 3 4 4 2 2 2 2 2
8336 004/300 Write block 2019 len 40960 data: 2 2 2 2 2 3 3 3 3 3
8337 005/300 Write block 781 len 28672 data: 2 2 2 3 3 3 2
8338 006/300 Write block 4614 len 40960 data: 3 3 4 4 2 2 2 2 2 2
8339 007/300 Read block 2874 len 16384
8340 008/300 Read block 4887 len 12288
8341 009/300 Write block 3928 len 4096 data: 2
8342 010/300 Read block 2680 len 4096
8343 011/300 Read block 2536 len 8192
8344 012/300 Flush
8345 013/300 Read block 2798 len 36864
8346 014/300 Read block 4312 len 24576
8347 015/300 Flush
8348 016/300 Write block 683 len 12288 data: 2 2 2
8349 017/300 Read block 300 len 4096
8350 018/300 Flush
8351 019/300 Write block 1541 len 24576 data: 2 2 2 2 2 2
8352 020/300 Write block 877 len 40960 data: 5 4 4 4 4 4 3 3 3 3
8353 021/300 Write block 204 len 4096 data: 4
8354 022/300 Write block 3209 len 12288 data: 2 2 3
8355 023/300 Read block 1775 len 36864
8356 024/300 Read block 962 len 20480
8357 025/300 Read block 86 len 8192
8358 026/300 Write block 4759 len 8192 data: 3 3
8359 027/300 Write block 968 len 12288 data: 2 2 2
8360 028/300 Read block 1874 len 24576
8361 029/300 Read block 141 len 20480
8362 030/300 Read block 4275 len 12288
8363 031/300 Read block 3663 len 32768
8364 032/300 Read block 1478 len 4096
8365 033/300 Write block 1403 len 4096 data: 2
8366 034/300 Write block 2610 len 24576 data: 3 3 3 3 3 3
8367 035/300 Read block 432 len 40960
8368 036/300 Write block 4777 len 16384 data: 4 4 3 3
8369 037/300 Read block 4934 len 12288
8370 038/300 Flush
8371 039/300 Write block 1769 len 32768 data: 2 2 3 3 4 4 4 3
8372 040/300 Flush
8373 041/300 Flush
8374 042/300 Write block 2231 len 24576 data: 4 4 3 3 3 3
8375 043/300 Read block 3618 len 12288
8376 044/300 Read block 4436 len 28672
8377 045/300 Flush
8378 046/300 Read block 1591 len 8192
8379 047/300 Read block 2716 len 36864
8380 048/300 Write block 161 len 28672 data: 4 3 3 3 3 2 2
8381 049/300 Write block 20 len 32768 data: 2 2 2 2 2 2 2 2
8382 050/300 Write block 983 len 32768 data: 4 4 4 4 3 2 2 2
8383 051/300 Write block 2570 len 12288 data: 2 2 2
8384 052/300 Read block 4432 len 20480
8385 053/300 Write block 4470 len 28672 data: 4 4 4 3 3 3 2
8386 054/300 Read block 3846 len 4096
8387 055/300 Flush
8388 056/300 Read block 3248 len 28672
8389 057/300 Write block 4905 len 40960 data: 5 3 3 3 2 2 3 3 3 4
8390 058/300 Write block 519 len 20480 data: 4 4 4 4 3
8391 059/300 Read block 645 len 4096
8392 060/300 Read block 465 len 4096
8393 061/300 Read block 4288 len 28672
8394 062/300 Read block 3233 len 20480
8395 063/300 Read block 260 len 12288
8396 064/300 Write block 3527 len 4096 data: 3
8397 065/300 Read block 1217 len 28672
8398 066/300 Read block 79 len 16384
8399 067/300 Read block 4048 len 24576
8400 068/300 Write block 1201 len 20480 data: 2 2 2 3 3
8401 069/300 Write block 1055 len 12288 data: 4 4 3
8402 070/300 Write block 1187 len 8192 data: 2 3
8403 071/300 Write block 3263 len 24576 data: 4 3 3 3 3 3
8404 072/300 Flush
8405 073/300 Flush
8406 074/300 Flush
8407 075/300 Write block 3427 len 16384 data: 2 2 2 2
8408 076/300 Write block 4103 len 8192 data: 3 3
8409 077/300 Read block 1221 len 8192
8410 078/300 Read block 4823 len 8192
8411 079/300 Read block 3511 len 28672
8412 080/300 Write block 4345 len 16384 data: 2 2 3 4
8413 081/300 Write block 1491 len 40960 data: 4 4 5 3 3 3 3 3 2 3
8414 082/300 Write block 4425 len 20480 data: 3 3 3 4 4
8415 083/300 Read block 1551 len 36864
8416 084/300 Write block 1190 len 4096 data: 4
8417 085/300 Write block 2814 len 8192 data: 3 4
8418 086/300 Write block 1294 len 12288 data: 2 2 2
8419 087/300 Read block 4116 len 8192
8420 088/300 Write block 2996 len 24576 data: 5 6 7 6 5 5
8421 089/300 Read block 1888 len 28672
8422 090/300 Read block 2435 len 40960
8423 091/300 Read block 1531 len 32768
8424 092/300 Write block 3327 len 16384 data: 3 3 3 3
8425 093/300 Flush
8426 094/300 Write block 1358 len 32768 data: 2 2 2 3 3 3 3 3
8427 095/300 Read block 2247 len 32768
8428 096/300 Flush
8429 097/300 Read block 2595 len 20480
8430 098/300 Read block 495 len 8192
8431 099/300 Read block 4635 len 8192
8432 100/300 Write block 2126 len 36864 data: 4 4 4 4 4 4 3 2 2
8433 101/300 Read block 4648 len 20480
8434 102/300 Read block 724 len 8192
8435 103/300 Read block 3731 len 28672
8436 104/300 Read block 4917 len 8192
8437 105/300 Read block 1714 len 32768
8438 106/300 Write block 4797 len 28672 data: 4 4 3 3 2 3 3
8439 107/300 Write block 297 len 32768 data: 2 3 3 3 3 3 3 4
8440 108/300 Write block 1095 len 40960 data: 4 4 5 4 3 3 3 2 2 2
8441 109/300 Flush
8442 110/300 Write block 1246 len 4096 data: 4
8443 111/300 Read block 1981 len 36864
8444 112/300 Read block 4767 len 32768
8445 113/300 Read block 2120 len 4096
8446 114/300 Write block 2829 len 28672 data: 4 5 5 5 6 5 5
8447 115/300 Write block 3281 len 28672 data: 4 4 3 4 3 3 3
8448 116/300 Read block 3878 len 8192
8449 117/300 Write block 2775 len 20480 data: 2 2 2 2 2
8450 118/300 Write block 685 len 4096 data: 3
8451 119/300 Read block 910 len 28672
8452 120/300 Read block 4342 len 12288
8453 121/300 Write block 1481 len 12288 data: 3 3 3
8454 122/300 Write block 2458 len 12288 data: 2 2 3
8455 123/300 Write block 2325 len 16384 data: 2 2 2 2
8456 124/300 Write block 2718 len 28672 data: 5 4 5 5 5 6 7
8457 125/300 Read block 360 len 16384
8458 126/300 Write block 354 len 40960 data: 3 3 3 3 2 2 2 2 2 3
8459 127/300 Write block 578 len 4096 data: 5
8460 128/300 Read block 4456 len 20480
8461 129/300 Read block 15 len 20480
8462 130/300 Write block 4605 len 24576 data: 2 2 2 2 2 2
8463 131/300 Read block 4786 len 8192
8464 132/300 Flush
8465 133/300 Read block 2546 len 32768
8466 134/300 Read block 2026 len 4096
8467 135/300 Read block 872 len 24576
8468 136/300 Read block 4897 len 24576
8469 137/300 Write block 3275 len 24576 data: 2 2 2 2 4 4
8470 138/300 Read block 625 len 32768
8471 139/300 Flush
8472 140/300 Read block 231 len 20480
8473 141/300 Read block 2583 len 24576
8474 142/300 Read block 2180 len 40960
8475 143/300 Read block 1195 len 16384
8476 144/300 Read block 2506 len 16384
8477 145/300 Read block 1088 len 40960
8478 146/300 Write block 4253 len 20480 data: 3 3 3 4 2
8479 147/300 Read block 2213 len 28672
8480 148/300 Write block 2334 len 28672 data: 4 4 3 3 2 3 3
8481 149/300 Write block 1944 len 36864 data: 3 3 3 2 3 3 2 2 2
8482 150/300 Write block 4464 len 36864 data: 2 2 2 3 3 4 5 5 5
8483 151/300 Read block 1033 len 12288
8484 152/300 Read block 4729 len 28672
8485 153/300 Read block 1242 len 16384
8486 154/300 Flush
8487 155/300 Write block 4378 len 4096 data: 3
8488 156/300 Read block 1709 len 20480
8489 157/300 Write block 1892 len 20480 data: 4 4 4 4 4
8490 158/300 Read block 2783 len 4096
8491 159/300 Flush
8492 160/300 Write block 1517 len 28672 data: 3 3 3 2 2 2 2
8493 161/300 Read block 2980 len 24576
8494 162/300 Read block 1783 len 16384
8495 163/300 Flush
8496 164/300 Read block 2339 len 32768
8497 165/300 Write block 890 len 8192 data: 3 3
8498 166/300 Read block 4266 len 12288
8499 167/300 Write block 3824 len 28672 data: 3 3 4 4 5 5 5
8500 168/300 Read block 4760 len 4096
8501 169/300 Write block 959 len 8192 data: 2 2
8502 170/300 Flush
8503 171/300 Write block 4927 len 16384 data: 5 5 5 5
8504 172/300 Write block 286 len 24576 data: 2 3 3 2 2 2
8505 173/300 Read block 2914 len 20480
8506 174/300 Flush
8507 175/300 Read block 4796 len 16384
8508 176/300 Write block 2936 len 28672 data: 5 5 4 3 3 3 3
8509 177/300 Read block 919 len 8192
8510 178/300 Write block 3695 len 32768 data: 3 3 4 4 4 3 3 3
8511 179/300 Read block 4061 len 16384
8512 180/300 Read block 2096 len 16384
8513 181/300 Read block 4975 len 20480
8514 182/300 Flush
8515 183/300 Write block 1757 len 36864 data: 2 2 2 2 2 2 2 2 2
8516 184/300 Read block 2306 len 24576
8517 185/300 Write block 2301 len 12288 data: 2 3 3
8518 186/300 Write block 322 len 40960 data: 3 4 4 4 4 4 4 4 3 3
8519 187/300 Write block 1062 len 36864 data: 2 3 3 3 2 2 2 2 2
8520 188/300 Write block 2012 len 40960 data: 2 2 2 2 2 2 2 3 3 3
8521 189/300 Read block 1142 len 36864
8522 190/300 Read block 1537 len 16384
8523 191/300 Read block 1656 len 20480
8524 192/300 Flush
8525 193/300 Write block 2681 len 28672 data: 2 2 2 2 3 3 4
8526 194/300 Read block 3335 len 28672
8527 195/300 Read block 3090 len 24576
8528 196/300 Read block 4242 len 20480
8529 197/300 Read block 3414 len 36864
8530 198/300 Read block 2426 len 36864
8531 199/300 Flush
8532 200/300 Read block 3891 len 28672
8533 201/300 Write block 343 len 16384 data: 2 3 3 3
8534 202/300 Write block 2346 len 8192 data: 3 3
8535 203/300 Read block 1356 len 24576
8536 204/300 Write block 1265 len 8192 data: 4 3
8537 205/300 Read block 2424 len 16384
8538 206/300 Read block 140 len 24576
8539 207/300 Write block 3549 len 24576 data: 2 2 2 2 2 2
8540 208/300 Flush
8541 209/300 Read block 4988 len 8192
8542 210/300 Read block 1524 len 32768
8543 211/300 Write block 195 len 16384 data: 3 4 3 3
8544 212/300 Write block 2960 len 28672 data: 3 4 4 4 3 3 3
8545 213/300 Read block 2900 len 36864
8546 214/300 Read block 1839 len 16384
8547 215/300 Read block 2885 len 40960
8548 216/300 Write block 583 len 8192 data: 3 3
8549 217/300 Read block 4959 len 12288
8550 218/300 Read block 1097 len 40960
8551 219/300 Read block 256 len 36864
8552 220/300 Flush
8553 221/300 Write block 4408 len 4096 data: 2
8554 222/300 Write block 795 len 40960 data: 2 2 2 2 2 3 3 3 3 3
8555 223/300 Read block 4597 len 20480
8556 224/300 Write block 4069 len 16384 data: 2 2 3 3
8557 225/300 Read block 1910 len 28672
8558 226/300 Read block 2110 len 16384
8559 227/300 Write block 3702 len 4096 data: 4
8560 228/300 Read block 268 len 28672
8561 229/300 Write block 3922 len 4096 data: 3
8562 230/300 Write block 3727 len 4096 data: 4
8563 231/300 Write block 4968 len 28672 data: 3 3 3 3 2 2 2
8564 232/300 Write block 267 len 32768 data: 2 2 2 2 2 2 2 2
8565 233/300 Read block 3697 len 20480
8566 234/300 Read block 3883 len 24576
8567 235/300 Read block 2289 len 28672
8568 236/300 Write block 4693 len 20480 data: 2 2 2 2 2
8569 237/300 Read block 379 len 28672
8570 238/300 Write block 1270 len 8192 data: 3 3
8571 239/300 Flush
8572 240/300 Write block 2712 len 32768 data: 4 4 4 4 6 4 6 5
8573 241/300 Write block 4478 len 36864 data: 2 2 2 2 2 2 2 2 2
8574 242/300 Read block 4047 len 4096
8575 243/300 Read block 2626 len 12288
8576 244/300 Read block 3007 len 8192
8577 245/300 Read block 646 len 32768
8578 246/300 Write block 1315 len 4096 data: 3
8579 247/300 Read block 3316 len 24576
8580 248/300 Write block 3992 len 8192 data: 5 4
8581 249/300 Flush
8582 250/300 Write block 2785 len 28672 data: 2 3 3 3 3 3 3
8583 251/300 Read block 583 len 40960
8584 252/300 Write block 1177 len 28672 data: 3 3 3 3 3 4 3
8585 253/300 Write block 138 len 32768 data: 4 4 4 4 4 4 4 3
8586 254/300 Read block 40 len 4096
8587 255/300 Read block 419 len 16384
8588 256/300 Read block 522 len 32768
8589 257/300 Write block 2017 len 28672 data: 3 3 4 4 4 3 3
8590 258/300 Read block 4869 len 40960
8591 259/300 Read block 1248 len 28672
8592 260/300 Write block 2821 len 28672 data: 2 2 3 3 2 2 2
8593 261/300 Read block 2293 len 12288
8594 262/300 Write block 2608 len 16384 data: 2 2 4 4
8595 263/300 Write block 4114 len 8192 data: 3 3
8596 264/300 Flush
8597 265/300 Read block 492 len 32768
8598 266/300 Write block 2168 len 36864 data: 2 2 2 2 3 3 3 3 3
8599 267/300 Read block 884 len 16384
8600 268/300 Write block 545 len 36864 data: 3 2 3 3 3 3 3 3 3
8601 269/300 Flush
8602 270/300 Write block 4419 len 12288 data: 2 2 2
8603 271/300 Write block 1405 len 40960 data: 2 2 2 2 3 4 4 4 4 4
8604 272/300 Read block 3066 len 16384
8605 273/300 Flush
8606 274/300 Read block 2530 len 8192
8607 275/300 Read block 400 len 20480
8608 276/300 Flush
8609 277/300 Read block 957 len 28672
8610 278/300 Write block 754 len 12288 data: 2 2 2
8611 279/300 Flush
8612 280/300 Read block 4455 len 32768
8613 281/300 Read block 2540 len 4096
8614 282/300 Read block 1557 len 12288
8615 283/300 Write block 2439 len 20480 data: 4 4 4 4 4
8616 284/300 Read block 2209 len 12288
8617 285/300 Write block 329 len 32768 data: 5 4 4 3 2 2 2 2
8618 286/300 Read block 929 len 4096
8619 287/300 Write block 2713 len 20480 data: 5 5 5 7 5
8620 288/300 Read block 562 len 4096
8621 289/300 Write block 840 len 4096 data: 4
8622 290/300 Read block 2703 len 40960
8623 291/300 Write block 4281 len 4096 data: 3
8624 292/300 Write block 1846 len 8192 data: 2 2
8625 293/300 Read block 3767 len 28672
8626 294/300 Write block 1723 len 4096 data: 3
8627 295/300 Flush
8628 296/300 Write block 3021 len 20480 data: 3 3 3 2 2
8629 297/300 Write block 3824 len 36864 data: 4 4 5 5 6 6 6 5 4
8630 298/300 Read block 4577 len 40960
8631 299/300 Read block 3557 len 4096
8632 300/300 Write block 2733 len 36864 data: 5 5 5 5 5 4 3 3 3
8633 301/300 Write block 4176 len 16384 data: 4 4 4 4
8634 Main task has work to do, go find it
8635 got dsc Start(0)
8636 start 0
8637 Dec 11 23:42:20.492 INFO request completed, latency_us: 279, response_code: 204, uri: /start/cid/0, method: POST, req_id: d44c8c62-9fa8-48ad-9e82-d052d941372f, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
8638 Replay: started 0, returned:Ok(())
8639 ----------------------------------------------------------------
8640 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8641 States: Offline Active Active
8642 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8643 3114 Acked Read 10 Sent Done Done false
8644 3115 Acked Read 10 Sent Done Done false
8645 3116 Acked Write 8 Sent Done Done false
8646 3117 Acked Write 10 Sent Done Done false
8647 3118 Acked Write 7 Sent Done Done false
8648 3119 Acked Write 10 Sent Done Done false
8649 3120 Acked Read 4 Sent Done Done false
8650 3121 Acked Read 3 Sent Done Done false
8651 3122 Acked Write 1 Sent Done Done false
8652 3123 Acked Read 1 Sent Done Done false
8653 3124 Acked Read 2 Sent Done Done false
8654 3125 Acked Flush 0 Sent Done Done false
8655 3126 Acked Read 9 Sent Done Done false
8656 3127 Acked Read 6 Sent Done Done false
8657 3128 Acked Flush 0 Sent Done Done false
8658 3129 Acked Write 3 Sent Done Done false
8659 3130 Acked Read 1 Sent Done Done false
8660 3131 Acked Flush 0 Sent Done Done false
8661 3132 Acked Write 6 Sent Done Done false
8662 3133 Acked Write 10 Sent Done Done false
8663 3134 Acked Write 1 Sent Done Done false
8664 3135 Acked Write 3 Sent Done Done false
8665 3136 Acked Read 9 Sent Done Done false
8666 3137 Acked Read 5 Sent Done Done false
8667 3138 Acked Read 2 Sent Done Done false
8668 3139 Acked Write 2 Sent Done Done false
8669 3140 Acked Write 3 Sent Done Done false
8670 3141 Acked Read 6 Sent Done Done false
8671 3142 Acked Read 5 Sent Done Done false
8672 3143 Acked Read 3 Sent Done Done false
8673 3144 Acked Read 8 Sent Done Done false
8674 3145 Acked Read 1 Sent Done Done false
8675 3146 Acked Write 1 Sent Done Done false
8676 3147 Acked Write 6 Sent Done Done false
8677 3148 Acked Read 10 Sent Done Done false
8678 3149 Acked Write 4 Sent Done Done false
8679 3150 Acked Read 3 Sent Done Done false
8680 3151 Acked Flush 0 Sent Done Done false
8681 3152 Acked Write 8 Sent Done Done false
8682 3153 Acked Flush 0 Sent Done Done false
8683 3154 Acked Flush 0 Sent Done Done false
8684 3155 Acked Write 6 Sent Done Done false
8685 3156 Acked Read 3 Sent Done Done false
8686 3157 Acked Read 7 Sent Done Done false
8687 3158 Acked Flush 0 Sent Done Done false
8688 3159 Acked Read 2 Sent Done Done false
8689 3160 Acked Read 9 Sent Done Done false
8690 3161 Acked Write 7 Sent Done Done false
8691 3162 Acked Write 8 Sent Done Done false
8692 3163 Acked Write 8 Sent Done Done false
8693 3164 Acked Write 3 Sent Done Done false
8694 3165 Acked Read 5 Sent Done Done false
8695 3166 Acked Write 7 Sent Done Done false
8696 3167 Acked Read 1 Sent Done Done false
8697 3168 Acked Flush 0 Sent Done Done false
8698 3169 Acked Read 7 Sent Done Done false
8699 3170 Acked Write 10 Sent Done Done false
8700 3171 Acked Write 5 Sent Done Done false
8701 3172 Acked Read 1 Sent Done Done false
8702 3173 Acked Read 1 Sent Done Done false
8703 3174 Acked Read 7 Sent Done Done false
8704 3175 Acked Read 5 Sent Done Done false
8705 3176 Acked Read 3 Sent Done Done false
8706 3177 Acked Write 1 Sent Done Done false
8707 3178 Acked Read 7 Sent Done Done false
8708 3179 Acked Read 4 Sent Done Done false
8709 3180 Acked Read 6 Sent Done Done false
8710 3181 Acked Write 5 Sent Done Done false
8711 3182 Acked Write 3 Sent Done Done false
8712 3183 Acked Write 2 Sent Done Done false
8713 3184 Acked Write 6 Sent Done Done false
8714 3185 Acked Flush 0 Sent Done Done false
8715 3186 Acked Flush 0 Sent Done Done false
8716 3187 Acked Flush 0 Sent Done Done false
8717 3188 Acked Write 4 Sent Done Done false
8718 3189 Acked Write 2 Sent Done Done false
8719 3190 Acked Read 2 Sent Done Done false
8720 3191 Acked Read 2 Sent Done Done false
8721 3192 Acked Read 7 Sent Done Done false
8722 3193 Acked Write 4 Sent Done Done false
8723 3194 Acked Write 10 Sent Done Done false
8724 3195 Acked Write 5 Sent Done Done false
8725 3196 Acked Read 9 Sent Done Done false
8726 3197 Acked Write 1 Sent Done Done false
8727 3198 Acked Write 2 Sent Done Done false
8728 3199 Acked Write 3 Sent Done Done false
8729 3200 Acked Read 2 Sent Done Done false
8730 3201 Acked Write 6 Sent Done Done false
8731 3202 Acked Read 7 Sent Done Done false
8732 3203 Acked Read 10 Sent Done Done false
8733 3204 Acked Read 8 Sent Done Done false
8734 3205 Acked Write 4 Sent Done Done false
8735 3206 Acked Flush 0 Sent Done Done false
8736 3207 Acked Write 8 Sent Done Done false
8737 3208 Acked Read 8 Sent Done Done false
8738 3209 Acked Flush 0 Sent Done Done false
8739 3210 Acked Read 5 Sent Done Done false
8740 3211 Acked Read 2 Sent Done Done false
8741 3212 Acked Read 2 Sent Done Done false
8742 3213 Acked Write 9 Sent Done Done false
8743 3214 Acked Read 5 Sent Done Done false
8744 3215 Acked Read 2 Sent Done Done false
8745 3216 Acked Read 7 Sent Done Done false
8746 3217 Acked Read 2 Sent Done Done false
8747 3218 Acked Read 8 Sent Done Done false
8748 3219 Acked Write 7 Sent Done Done false
8749 3220 Acked Write 8 Sent Done Done false
8750 3221 Acked Write 10 Sent Done Done false
8751 3222 Acked Flush 0 Sent Done Done false
8752 3223 Acked Write 1 Sent Done Done false
8753 3224 Acked Read 9 Sent Done Done false
8754 3225 Acked Read 8 Sent Done Done false
8755 3226 Acked Read 1 Sent Done Done false
8756 3227 Acked Write 7 Sent Done Done false
8757 3228 Acked Write 7 Sent Done Done false
8758 3229 Acked Read 2 Sent Done Done false
8759 3230 Acked Write 5 Sent Done Done false
8760 3231 Acked Write 1 Sent Done Done false
8761 3232 Acked Read 7 Sent Done Done false
8762 3233 Acked Read 3 Sent Done Done false
8763 3234 Acked Write 3 Sent Done Done false
8764 3235 Acked Write 3 Sent Done Done false
8765 3236 Acked Write 4 Sent Done Done false
8766 3237 Acked Write 7 Sent Done Done false
8767 3238 Acked Read 4 Sent Done Done false
8768 3239 Acked Write 10 Sent Done Done false
8769 3240 Acked Write 1 Sent Done Done false
8770 3241 Acked Read 5 Sent Done Done false
8771 3242 Acked Read 5 Sent Done Done false
8772 3243 Acked Write 6 Sent Done Done false
8773 3244 Acked Read 2 Sent Done Done false
8774 3245 Acked Flush 0 Sent Done Done false
8775 3246 Acked Read 8 Sent Done Done false
8776 3247 Acked Read 1 Sent Done Done false
8777 3248 Acked Read 6 Sent Done Done false
8778 3249 Acked Read 6 Sent Done Done false
8779 3250 Acked Write 6 Sent Done Done false
8780 3251 Acked Read 8 Sent Done Done false
8781 3252 Acked Flush 0 Sent Done Done false
8782 3253 Acked Read 5 Sent Done Done false
8783 3254 Acked Read 6 Sent Done Done false
8784 3255 Acked Read 10 Sent Done Done false
8785 3256 Acked Read 4 Sent Done Done false
8786 3257 Acked Read 4 Sent Done Done false
8787 3258 Acked Read 10 Sent Done Done false
8788 3259 Acked Write 5 Sent Done Done false
8789 3260 Acked Read 7 Sent Done Done false
8790 3261 Acked Write 7 Sent Done Done false
8791 3262 Acked Write 9 Sent Done Done false
8792 3263 Acked Write 9 Sent Done Done false
8793 3264 Acked Read 3 Sent Done Done false
8794 3265 Acked Read 7 Sent Done Done false
8795 3266 Acked Read 4 Sent Done Done false
8796 3267 Acked Flush 0 Sent Done Done false
8797 3268 Acked Write 1 Sent Done Done false
8798 3269 Acked Read 5 Sent Done Done false
8799 3270 Acked Write 5 Sent Done Done false
8800 3271 Acked Read 1 Sent Done Done false
8801 3272 Acked Flush 0 Sent Done Done false
8802 3273 Acked Write 7 Sent Done Done false
8803 3274 Acked Read 6 Sent Done Done false
8804 3275 Acked Read 4 Sent Done Done false
8805 3276 Acked Flush 0 Sent Done Done false
8806 3277 Acked Read 8 Sent Done Done false
8807 3278 Acked Write 2 Sent Done Done false
8808 3279 Acked Read 3 Sent Done Done false
8809 3280 Acked Write 7 Sent Done Done false
8810 3281 Acked Read 1 Sent Done Done false
8811 3282 Acked Write 2 Sent Done Done false
8812 3283 Acked Flush 0 Sent Done Done false
8813 3284 Acked Write 4 Sent Done Done false
8814 3285 Acked Write 6 Sent Done Done false
8815 3286 Acked Read 5 Sent Done Done false
8816 3287 Acked Flush 0 Sent Done Done false
8817 3288 Acked Read 4 Sent Done Done false
8818 3289 Acked Write 7 Sent Done Done false
8819 3290 Acked Read 2 Sent Done Done false
8820 3291 Acked Write 8 Sent Done Done false
8821 3292 Acked Read 4 Sent Done Done false
8822 3293 Acked Read 4 Sent Done Done false
8823 3294 Acked Read 5 Sent Done Done false
8824 3295 Acked Flush 0 Sent Done Done false
8825 3296 Acked Write 9 Sent Done Done false
8826 3297 Acked Read 6 Sent Done Done false
8827 3298 Acked Write 3 Sent Done Done false
8828 3299 Acked Write 10 Sent Done Done false
8829 3300 Acked Write 9 Sent Done Done false
8830 3301 Acked Write 10 Sent Done Done false
8831 3302 Acked Read 9 Sent Done Done false
8832 3303 Acked Read 4 Sent Done Done false
8833 3304 Acked Read 5 Sent Done Done false
8834 3305 Acked Flush 0 Sent Done Done false
8835 3306 Acked Write 7 Sent Done Done false
8836 3307 Acked Read 7 Sent Done Done false
8837 3308 Acked Read 6 Sent Done Done false
8838 3309 Acked Read 5 Sent Done Done false
8839 3310 Acked Read 9 Sent Done Done false
8840 3311 Acked Read 9 Sent Done Done false
8841 3312 Acked Flush 0 Sent Done Done false
8842 3313 Acked Read 7 Sent Done Done false
8843 3314 Acked Write 4 Sent Done Done false
8844 3315 Acked Write 2 Sent Done Done false
8845 3316 Acked Read 6 Sent Done Done false
8846 3317 Acked Write 2 Sent Done Done false
8847 3318 Acked Read 4 Sent Done Done false
8848 3319 Acked Read 6 Sent Done Done false
8849 3320 Acked Write 6 Sent Done Done false
8850 3321 Acked Flush 0 Sent Done Done false
8851 3322 Acked Read 2 Sent Done Done false
8852 3323 Acked Read 8 Sent Done Done false
8853 3324 Acked Write 4 Sent Done Done false
8854 3325 Acked Write 7 Sent Done Done false
8855 3326 Acked Read 9 Sent Done Done false
8856 3327 Acked Read 4 Sent Done Done false
8857 3328 Acked Read 10 Sent Done Done false
8858 3329 Acked Write 2 Sent Done Done false
8859 3330 Acked Read 3 Sent Done Done false
8860 3331 Acked Read 10 Sent Done Done false
8861 3332 Acked Read 9 Sent Done Done false
8862 3333 Acked Flush 0 Sent Done Done false
8863 3334 Acked Write 1 Sent Done Done false
8864 3335 Acked Write 10 Sent Done Done false
8865 3336 Acked Read 5 Sent Done Done false
8866 3337 Acked Write 4 Sent Done Done false
8867 3338 Acked Read 7 Sent Done Done false
8868 3339 Acked Read 4 Sent Done Done false
8869 3340 Acked Write 1 Sent Done Done false
8870 3341 Acked Read 7 Sent Done Done false
8871 3342 Acked Write 1 Sent Done Done false
8872 3343 Acked Write 1 Sent Done Done false
8873 3344 Acked Write 7 Sent Done Done false
8874 3345 Acked Write 8 Sent Done Done false
8875 3346 Acked Read 5 Sent Done Done false
8876 3347 Acked Read 6 Sent Done Done false
8877 3348 Acked Read 7 Sent Done Done false
8878 3349 Acked Write 5 Sent Done Done false
8879 3350 Acked Read 7 Sent Done Done false
8880 3351 Acked Write 2 Sent Done Done false
8881 3352 Acked Flush 0 Sent Done Done false
8882 3353 Acked Write 8 Sent Done Done false
8883 3354 Acked Write 9 Sent Done Done false
8884 3355 Acked Read 1 Sent Done Done false
8885 3356 Acked Read 3 Sent Done Done false
8886 3357 Acked Read 2 Sent Done Done false
8887 3358 Acked Read 8 Sent Done Done false
8888 3359 Acked Write 1 Sent Done Done false
8889 3360 Acked Read 6 Sent Done Done false
8890 3361 Acked Write 2 Sent Done Done false
8891 3362 Acked Flush 0 Sent Done Done false
8892 3363 Acked Write 7 Sent Done Done false
8893 3364 Acked Read 10 Sent Done Done false
8894 3365 Acked Write 7 Sent Done Done false
8895 3366 Acked Write 8 Sent Done Done false
8896 3367 Acked Read 1 Sent Done Done false
8897 3368 Acked Read 4 Sent Done Done false
8898 3369 Acked Read 8 Sent Done Done false
8899 3370 Acked Write 7 Sent Done Done false
8900 3371 Acked Read 10 Sent Done Done false
8901 3372 Acked Read 7 Sent Done Done false
8902 3373 Acked Write 7 Sent Done Done false
8903 3374 Acked Read 3 Sent Done Done false
8904 3375 Acked Write 4 Sent Done Done false
8905 3376 Acked Write 2 Sent Done Done false
8906 3377 Acked Flush 0 Sent Done Done false
8907 3378 Acked Read 8 Sent Done Done false
8908 3379 Acked Write 9 Sent Done Done false
8909 3380 Acked Read 4 Sent Done Done false
8910 3381 Acked Write 9 Sent Done Done false
8911 3382 Acked Flush 0 Sent Done Done false
8912 3383 Acked Write 3 Sent Done Done false
8913 3384 Acked Write 10 Sent Done Done false
8914 3385 Acked Read 4 Sent Done Done false
8915 3386 Acked Flush 0 Sent Done Done false
8916 3387 Acked Read 2 Sent Done Done false
8917 3388 Acked Read 5 Sent Done Done false
8918 3389 Acked Flush 0 Sent Done Done false
8919 3390 Acked Read 7 Sent Done Done false
8920 3391 Acked Write 3 Sent Done Done false
8921 3392 Acked Flush 0 Sent Done Done false
8922 3393 Acked Read 8 Sent Done Done false
8923 3394 Acked Read 1 Sent Done Done false
8924 3395 Acked Read 3 Sent Done Done false
8925 3396 Acked Write 5 Sent Done Done false
8926 3397 Acked Read 3 Sent Done Done false
8927 3398 Acked Write 8 Sent Done Done false
8928 3399 Acked Read 1 Sent Done Done false
8929 3400 Acked Write 5 Sent Done Done false
8930 3401 Acked Read 1 Sent Done Done false
8931 3402 Acked Write 1 Sent Done Done false
8932 3403 Acked Read 10 Sent Done Done false
8933 3404 Acked Write 1 Sent Done Done false
8934 3405 Acked Write 2 Sent Done Done false
8935 3406 Acked Read 7 Sent Done Done false
8936 3407 Acked Write 1 Sent Done Done false
8937 3408 Acked Flush 0 Sent Done Done false
8938 3409 Acked Write 5 Sent Done Done false
8939 3410 Acked Write 9 Sent Done Done false
8940 3411 Acked Read 10 Sent Done Done false
8941 3412 Acked Read 1 Sent Done Done false
8942 3413 Acked Write 9 Sent Done Done false
8943 3414 Acked Write 4 Sent Done Done false
8944 STATES DS:0 DS:1 DS:2 TOTAL
8945 Sent 301 0 0 301
8946 Done 0 301 301 602
8947 Skipped 0 0 0 0
8948 Error 0 0 0 0
8949 Last Flush: 3113 3408 3408
8950 Downstairs last five completed: 3113 3112 3111 3110 3109
8951 Upstairs last five completed: 3414 3413 3412 3411 3410
8952 CLIENT: Up:0 ds:301 act:2
8953 ----------------------------------------------------------------
8954 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8955 States: Offline Active Active
8956 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8957 3114 Acked Read 10 Sent Done Done false
8958 3115 Acked Read 10 Sent Done Done false
8959 3116 Acked Write 8 Sent Done Done false
8960 3117 Acked Write 10 Sent Done Done false
8961 3118 Acked Write 7 Sent Done Done false
8962 3119 Acked Write 10 Sent Done Done false
8963 3120 Acked Read 4 Sent Done Done false
8964 3121 Acked Read 3 Sent Done Done false
8965 3122 Acked Write 1 Sent Done Done false
8966 3123 Acked Read 1 Sent Done Done false
8967 3124 Acked Read 2 Sent Done Done false
8968 3125 Acked Flush 0 Sent Done Done false
8969 3126 Acked Read 9 Sent Done Done false
8970 3127 Acked Read 6 Sent Done Done false
8971 3128 Acked Flush 0 Sent Done Done false
8972 3129 Acked Write 3 Sent Done Done false
8973 3130 Acked Read 1 Sent Done Done false
8974 3131 Acked Flush 0 Sent Done Done false
8975 3132 Acked Write 6 Sent Done Done false
8976 3133 Acked Write 10 Sent Done Done false
8977 3134 Acked Write 1 Sent Done Done false
8978 3135 Acked Write 3 Sent Done Done false
8979 3136 Acked Read 9 Sent Done Done false
8980 3137 Acked Read 5 Sent Done Done false
8981 3138 Acked Read 2 Sent Done Done false
8982 3139 Acked Write 2 Sent Done Done false
8983 3140 Acked Write 3 Sent Done Done false
8984 3141 Acked Read 6 Sent Done Done false
8985 3142 Acked Read 5 Sent Done Done false
8986 3143 Acked Read 3 Sent Done Done false
8987 3144 Acked Read 8 Sent Done Done false
8988 3145 Acked Read 1 Sent Done Done false
8989 3146 Acked Write 1 Sent Done Done false
8990 3147 Acked Write 6 Sent Done Done false
8991 3148 Acked Read 10 Sent Done Done false
8992 3149 Acked Write 4 Sent Done Done false
8993 3150 Acked Read 3 Sent Done Done false
8994 3151 Acked Flush 0 Sent Done Done false
8995 3152 Acked Write 8 Sent Done Done false
8996 3153 Acked Flush 0 Sent Done Done false
8997 3154 Acked Flush 0 Sent Done Done false
8998 3155 Acked Write 6 Sent Done Done false
8999 3156 Acked Read 3 Sent Done Done false
9000 3157 Acked Read 7 Sent Done Done false
9001 3158 Acked Flush 0 Sent Done Done false
9002 3159 Acked Read 2 Sent Done Done false
9003 3160 Acked Read 9 Sent Done Done false
9004 3161 Acked Write 7 Sent Done Done false
9005 3162 Acked Write 8 Sent Done Done false
9006 3163 Acked Write 8 Sent Done Done false
9007 3164 Acked Write 3 Sent Done Done false
9008 3165 Acked Read 5 Sent Done Done false
9009 3166 Acked Write 7 Sent Done Done false
9010 3167 Acked Read 1 Sent Done Done false
9011 3168 Acked Flush 0 Sent Done Done false
9012 3169 Acked Read 7 Sent Done Done false
9013 3170 Acked Write 10 Sent Done Done false
9014 3171 Acked Write 5 Sent Done Done false
9015 3172 Acked Read 1 Sent Done Done false
9016 3173 Acked Read 1 Sent Done Done false
9017 3174 Acked Read 7 Sent Done Done false
9018 3175 Acked Read 5 Sent Done Done false
9019 3176 Acked Read 3 Sent Done Done false
9020 3177 Acked Write 1 Sent Done Done false
9021 3178 Acked Read 7 Sent Done Done false
9022 3179 Acked Read 4 Sent Done Done false
9023 3180 Acked Read 6 Sent Done Done false
9024 3181 Acked Write 5 Sent Done Done false
9025 3182 Acked Write 3 Sent Done Done false
9026 3183 Acked Write 2 Sent Done Done false
9027 3184 Acked Write 6 Sent Done Done false
9028 3185 Acked Flush 0 Sent Done Done false
9029 3186 Acked Flush 0 Sent Done Done false
9030 3187 Acked Flush 0 Sent Done Done false
9031 3188 Acked Write 4 Sent Done Done false
9032 3189 Acked Write 2 Sent Done Done false
9033 3190 Acked Read 2 Sent Done Done false
9034 3191 Acked Read 2 Sent Done Done false
9035 3192 Acked Read 7 Sent Done Done false
9036 3193 Acked Write 4 Sent Done Done false
9037 3194 Acked Write 10 Sent Done Done false
9038 3195 Acked Write 5 Sent Done Done false
9039 3196 Acked Read 9 Sent Done Done false
9040 3197 Acked Write 1 Sent Done Done false
9041 3198 Acked Write 2 Sent Done Done false
9042 3199 Acked Write 3 Sent Done Done false
9043 3200 Acked Read 2 Sent Done Done false
9044 3201 Acked Write 6 Sent Done Done false
9045 3202 Acked Read 7 Sent Done Done false
9046 3203 Acked Read 10 Sent Done Done false
9047 3204 Acked Read 8 Sent Done Done false
9048 3205 Acked Write 4 Sent Done Done false
9049 3206 Acked Flush 0 Sent Done Done false
9050 3207 Acked Write 8 Sent Done Done false
9051 3208 Acked Read 8 Sent Done Done false
9052 3209 Acked Flush 0 Sent Done Done false
9053 3210 Acked Read 5 Sent Done Done false
9054 3211 Acked Read 2 Sent Done Done false
9055 3212 Acked Read 2 Sent Done Done false
9056 3213 Acked Write 9 Sent Done Done false
9057 3214 Acked Read 5 Sent Done Done false
9058 3215 Acked Read 2 Sent Done Done false
9059 3216 Acked Read 7 Sent Done Done false
9060 3217 Acked Read 2 Sent Done Done false
9061 3218 Acked Read 8 Sent Done Done false
9062 3219 Acked Write 7 Sent Done Done false
9063 3220 Acked Write 8 Sent Done Done false
9064 3221 Acked Write 10 Sent Done Done false
9065 3222 Acked Flush 0 Sent Done Done false
9066 3223 Acked Write 1 Sent Done Done false
9067 3224 Acked Read 9 Sent Done Done false
9068 3225 Acked Read 8 Sent Done Done false
9069 3226 Acked Read 1 Sent Done Done false
9070 3227 Acked Write 7 Sent Done Done false
9071 3228 Acked Write 7 Sent Done Done false
9072 3229 Acked Read 2 Sent Done Done false
9073 3230 Acked Write 5 Sent Done Done false
9074 3231 Acked Write 1 Sent Done Done false
9075 3232 Acked Read 7 Sent Done Done false
9076 3233 Acked Read 3 Sent Done Done false
9077 3234 Acked Write 3 Sent Done Done false
9078 3235 Acked Write 3 Sent Done Done false
9079 3236 Acked Write 4 Sent Done Done false
9080 3237 Acked Write 7 Sent Done Done false
9081 3238 Acked Read 4 Sent Done Done false
9082 3239 Acked Write 10 Sent Done Done false
9083 3240 Acked Write 1 Sent Done Done false
9084 3241 Acked Read 5 Sent Done Done false
9085 3242 Acked Read 5 Sent Done Done false
9086 3243 Acked Write 6 Sent Done Done false
9087 3244 Acked Read 2 Sent Done Done false
9088 3245 Acked Flush 0 Sent Done Done false
9089 3246 Acked Read 8 Sent Done Done false
9090 3247 Acked Read 1 Sent Done Done false
9091 3248 Acked Read 6 Sent Done Done false
9092 3249 Acked Read 6 Sent Done Done false
9093 3250 Acked Write 6 Sent Done Done false
9094 3251 Acked Read 8 Sent Done Done false
9095 3252 Acked Flush 0 Sent Done Done false
9096 3253 Acked Read 5 Sent Done Done false
9097 3254 Acked Read 6 Sent Done Done false
9098 3255 Acked Read 10 Sent Done Done false
9099 3256 Acked Read 4 Sent Done Done false
9100 3257 Acked Read 4 Sent Done Done false
9101 3258 Acked Read 10 Sent Done Done false
9102 3259 Acked Write 5 Sent Done Done false
9103 3260 Acked Read 7 Sent Done Done false
9104 3261 Acked Write 7 Sent Done Done false
9105 3262 Acked Write 9 Sent Done Done false
9106 3263 Acked Write 9 Sent Done Done false
9107 3264 Acked Read 3 Sent Done Done false
9108 3265 Acked Read 7 Sent Done Done false
9109 3266 Acked Read 4 Sent Done Done false
9110 3267 Acked Flush 0 Sent Done Done false
9111 3268 Acked Write 1 Sent Done Done false
9112 3269 Acked Read 5 Sent Done Done false
9113 3270 Acked Write 5 Sent Done Done false
9114 3271 Acked Read 1 Sent Done Done false
9115 3272 Acked Flush 0 Sent Done Done false
9116 3273 Acked Write 7 Sent Done Done false
9117 3274 Acked Read 6 Sent Done Done false
9118 3275 Acked Read 4 Sent Done Done false
9119 3276 Acked Flush 0 Sent Done Done false
9120 3277 Acked Read 8 Sent Done Done false
9121 3278 Acked Write 2 Sent Done Done false
9122 3279 Acked Read 3 Sent Done Done false
9123 3280 Acked Write 7 Sent Done Done false
9124 3281 Acked Read 1 Sent Done Done false
9125 3282 Acked Write 2 Sent Done Done false
9126 3283 Acked Flush 0 Sent Done Done false
9127 3284 Acked Write 4 Sent Done Done false
9128 3285 Acked Write 6 Sent Done Done false
9129 3286 Acked Read 5 Sent Done Done false
9130 3287 Acked Flush 0 Sent Done Done false
9131 3288 Acked Read 4 Sent Done Done false
9132 3289 Acked Write 7 Sent Done Done false
9133 3290 Acked Read 2 Sent Done Done false
9134 3291 Acked Write 8 Sent Done Done false
9135 3292 Acked Read 4 Sent Done Done false
9136 3293 Acked Read 4 Sent Done Done false
9137 3294 Acked Read 5 Sent Done Done false
9138 3295 Acked Flush 0 Sent Done Done false
9139 3296 Acked Write 9 Sent Done Done false
9140 3297 Acked Read 6 Sent Done Done false
9141 3298 Acked Write 3 Sent Done Done false
9142 3299 Acked Write 10 Sent Done Done false
9143 3300 Acked Write 9 Sent Done Done false
9144 3301 Acked Write 10 Sent Done Done false
9145 3302 Acked Read 9 Sent Done Done false
9146 3303 Acked Read 4 Sent Done Done false
9147 3304 Acked Read 5 Sent Done Done false
9148 3305 Acked Flush 0 Sent Done Done false
9149 3306 Acked Write 7 Sent Done Done false
9150 3307 Acked Read 7 Sent Done Done false
9151 3308 Acked Read 6 Sent Done Done false
9152 3309 Acked Read 5 Sent Done Done false
9153 3310 Acked Read 9 Sent Done Done false
9154 3311 Acked Read 9 Sent Done Done false
9155 3312 Acked Flush 0 Sent Done Done false
9156 3313 Acked Read 7 Sent Done Done false
9157 3314 Acked Write 4 Sent Done Done false
9158 3315 Acked Write 2 Sent Done Done false
9159 3316 Acked Read 6 Sent Done Done false
9160 3317 Acked Write 2 Sent Done Done false
9161 3318 Acked Read 4 Sent Done Done false
9162 3319 Acked Read 6 Sent Done Done false
9163 3320 Acked Write 6 Sent Done Done false
9164 3321 Acked Flush 0 Sent Done Done false
9165 3322 Acked Read 2 Sent Done Done false
9166 3323 Acked Read 8 Sent Done Done false
9167 3324 Acked Write 4 Sent Done Done false
9168 3325 Acked Write 7 Sent Done Done false
9169 3326 Acked Read 9 Sent Done Done false
9170 3327 Acked Read 4 Sent Done Done false
9171 3328 Acked Read 10 Sent Done Done false
9172 3329 Acked Write 2 Sent Done Done false
9173 3330 Acked Read 3 Sent Done Done false
9174 3331 Acked Read 10 Sent Done Done false
9175 3332 Acked Read 9 Sent Done Done false
9176 3333 Acked Flush 0 Sent Done Done false
9177 3334 Acked Write 1 Sent Done Done false
9178 3335 Acked Write 10 Sent Done Done false
9179 3336 Acked Read 5 Sent Done Done false
9180 3337 Acked Write 4 Sent Done Done false
9181 3338 Acked Read 7 Sent Done Done false
9182 3339 Acked Read 4 Sent Done Done false
9183 3340 Acked Write 1 Sent Done Done false
9184 3341 Acked Read 7 Sent Done Done false
9185 3342 Acked Write 1 Sent Done Done false
9186 3343 Acked Write 1 Sent Done Done false
9187 3344 Acked Write 7 Sent Done Done false
9188 3345 Acked Write 8 Sent Done Done false
9189 3346 Acked Read 5 Sent Done Done false
9190 3347 Acked Read 6 Sent Done Done false
9191 3348 Acked Read 7 Sent Done Done false
9192 3349 Acked Write 5 Sent Done Done false
9193 3350 Acked Read 7 Sent Done Done false
9194 3351 Acked Write 2 Sent Done Done false
9195 3352 Acked Flush 0 Sent Done Done false
9196 3353 Acked Write 8 Sent Done Done false
9197 3354 Acked Write 9 Sent Done Done false
9198 3355 Acked Read 1 Sent Done Done false
9199 3356 Acked Read 3 Sent Done Done false
9200 3357 Acked Read 2 Sent Done Done false
9201 3358 Acked Read 8 Sent Done Done false
9202 3359 Acked Write 1 Sent Done Done false
9203 3360 Acked Read 6 Sent Done Done false
9204 3361 Acked Write 2 Sent Done Done false
9205 3362 Acked Flush 0 Sent Done Done false
9206 3363 Acked Write 7 Sent Done Done false
9207 3364 Acked Read 10 Sent Done Done false
9208 3365 Acked Write 7 Sent Done Done false
9209 3366 Acked Write 8 Sent Done Done false
9210 3367 Acked Read 1 Sent Done Done false
9211 3368 Acked Read 4 Sent Done Done false
9212 3369 Acked Read 8 Sent Done Done false
9213 3370 Acked Write 7 Sent Done Done false
9214 3371 Acked Read 10 Sent Done Done false
9215 3372 Acked Read 7 Sent Done Done false
9216 3373 Acked Write 7 Sent Done Done false
9217 3374 Acked Read 3 Sent Done Done false
9218 3375 Acked Write 4 Sent Done Done false
9219 3376 Acked Write 2 Sent Done Done false
9220 3377 Acked Flush 0 Sent Done Done false
9221 3378 Acked Read 8 Sent Done Done false
9222 3379 Acked Write 9 Sent Done Done false
9223 3380 Acked Read 4 Sent Done Done false
9224 3381 Acked Write 9 Sent Done Done false
9225 3382 Acked Flush 0 Sent Done Done false
9226 3383 Acked Write 3 Sent Done Done false
9227 3384 Acked Write 10 Sent Done Done false
9228 3385 Acked Read 4 Sent Done Done false
9229 3386 Acked Flush 0 Sent Done Done false
9230 3387 Acked Read 2 Sent Done Done false
9231 3388 Acked Read 5 Sent Done Done false
9232 3389 Acked Flush 0 Sent Done Done false
9233 3390 Acked Read 7 Sent Done Done false
9234 3391 Acked Write 3 Sent Done Done false
9235 3392 Acked Flush 0 Sent Done Done false
9236 3393 Acked Read 8 Sent Done Done false
9237 3394 Acked Read 1 Sent Done Done false
9238 3395 Acked Read 3 Sent Done Done false
9239 3396 Acked Write 5 Sent Done Done false
9240 3397 Acked Read 3 Sent Done Done false
9241 3398 Acked Write 8 Sent Done Done false
9242 3399 Acked Read 1 Sent Done Done false
9243 3400 Acked Write 5 Sent Done Done false
9244 3401 Acked Read 1 Sent Done Done false
9245 3402 Acked Write 1 Sent Done Done false
9246 3403 Acked Read 10 Sent Done Done false
9247 3404 Acked Write 1 Sent Done Done false
9248 3405 Acked Write 2 Sent Done Done false
9249 3406 Acked Read 7 Sent Done Done false
9250 3407 Acked Write 1 Sent Done Done false
9251 3408 Acked Flush 0 Sent Done Done false
9252 3409 Acked Write 5 Sent Done Done false
9253 3410 Acked Write 9 Sent Done Done false
9254 3411 Acked Read 10 Sent Done Done false
9255 3412 Acked Read 1 Sent Done Done false
9256 3413 Acked Write 9 Sent Done Done false
9257 3414 Acked Write 4 Sent Done Done false
9258 3415 Acked Flush 0 Sent Done Done false
9259 STATES DS:0 DS:1 DS:2 TOTAL
9260 Sent 302 0 0 302
9261 Done 0 302 302 604
9262 Skipped 0 0 0 0
9263 Error 0 0 0 0
9264 Last Flush: 3113 3415 3415
9265 Downstairs last five completed: 3113 3112 3111 3110 3109
9266 Upstairs last five completed: 3415 3414 3413 3412 3411
9267 CLIENT: Up:0 ds:302 act:2
92682024-12-11T23:42:26.262ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
92692024-12-11T23:42:26.262ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
92702024-12-11T23:42:26.262ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
92712024-12-11T23:42:26.262ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
92722024-12-11T23:42:26.262ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
9273 ----------------------------------------------------------------
9274 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9275 States: Offline Active Active
9276 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9277 3114 Acked Read 10 Sent Done Done false
9278 3115 Acked Read 10 Sent Done Done false
9279 3116 Acked Write 8 Sent Done Done false
9280 3117 Acked Write 10 Sent Done Done false
9281 3118 Acked Write 7 Sent Done Done false
9282 3119 Acked Write 10 Sent Done Done false
9283 3120 Acked Read 4 Sent Done Done false
9284 3121 Acked Read 3 Sent Done Done false
9285 3122 Acked Write 1 Sent Done Done false
9286 3123 Acked Read 1 Sent Done Done false
9287 3124 Acked Read 2 Sent Done Done false
9288 3125 Acked Flush 0 Sent Done Done false
9289 3126 Acked Read 9 Sent Done Done false
9290 3127 Acked Read 6 Sent Done Done false
9291 3128 Acked Flush 0 Sent Done Done false
9292 3129 Acked Write 3 Sent Done Done false
9293 3130 Acked Read 1 Sent Done Done false
9294 3131 Acked Flush 0 Sent Done Done false
9295 3132 Acked Write 6 Sent Done Done false
9296 3133 Acked Write 10 Sent Done Done false
9297 3134 Acked Write 1 Sent Done Done false
9298 3135 Acked Write 3 Sent Done Done false
9299 3136 Acked Read 9 Sent Done Done false
9300 3137 Acked Read 5 Sent Done Done false
9301 3138 Acked Read 2 Sent Done Done false
9302 3139 Acked Write 2 Sent Done Done false
9303 3140 Acked Write 3 Sent Done Done false
9304 3141 Acked Read 6 Sent Done Done false
9305 3142 Acked Read 5 Sent Done Done false
9306 3143 Acked Read 3 Sent Done Done false
9307 3144 Acked Read 8 Sent Done Done false
9308 3145 Acked Read 1 Sent Done Done false
9309 3146 Acked Write 1 Sent Done Done false
9310 3147 Acked Write 6 Sent Done Done false
9311 3148 Acked Read 10 Sent Done Done false
9312 3149 Acked Write 4 Sent Done Done false
9313 3150 Acked Read 3 Sent Done Done false
9314 3151 Acked Flush 0 Sent Done Done false
9315 3152 Acked Write 8 Sent Done Done false
9316 3153 Acked Flush 0 Sent Done Done false
9317 3154 Acked Flush 0 Sent Done Done false
9318 3155 Acked Write 6 Sent Done Done false
9319 3156 Acked Read 3 Sent Done Done false
9320 3157 Acked Read 7 Sent Done Done false
9321 3158 Acked Flush 0 Sent Done Done false
9322 3159 Acked Read 2 Sent Done Done false
9323 3160 Acked Read 9 Sent Done Done false
9324 3161 Acked Write 7 Sent Done Done false
9325 3162 Acked Write 8 Sent Done Done false
9326 3163 Acked Write 8 Sent Done Done false
9327 3164 Acked Write 3 Sent Done Done false
9328 3165 Acked Read 5 Sent Done Done false
9329 3166 Acked Write 7 Sent Done Done false
9330 3167 Acked Read 1 Sent Done Done false
9331 3168 Acked Flush 0 Sent Done Done false
9332 3169 Acked Read 7 Sent Done Done false
9333 3170 Acked Write 10 Sent Done Done false
9334 3171 Acked Write 5 Sent Done Done false
9335 3172 Acked Read 1 Sent Done Done false
9336 3173 Acked Read 1 Sent Done Done false
9337 3174 Acked Read 7 Sent Done Done false
9338 3175 Acked Read 5 Sent Done Done false
9339 3176 Acked Read 3 Sent Done Done false
9340 3177 Acked Write 1 Sent Done Done false
9341 3178 Acked Read 7 Sent Done Done false
9342 3179 Acked Read 4 Sent Done Done false
9343 3180 Acked Read 6 Sent Done Done false
9344 3181 Acked Write 5 Sent Done Done false
9345 3182 Acked Write 3 Sent Done Done false
9346 3183 Acked Write 2 Sent Done Done false
9347 3184 Acked Write 6 Sent Done Done false
9348 3185 Acked Flush 0 Sent Done Done false
9349 3186 Acked Flush 0 Sent Done Done false
9350 3187 Acked Flush 0 Sent Done Done false
9351 3188 Acked Write 4 Sent Done Done false
9352 3189 Acked Write 2 Sent Done Done false
9353 3190 Acked Read 2 Sent Done Done false
9354 3191 Acked Read 2 Sent Done Done false
9355 3192 Acked Read 7 Sent Done Done false
9356 3193 Acked Write 4 Sent Done Done false
9357 3194 Acked Write 10 Sent Done Done false
9358 3195 Acked Write 5 Sent Done Done false
9359 3196 Acked Read 9 Sent Done Done false
9360 3197 Acked Write 1 Sent Done Done false
9361 3198 Acked Write 2 Sent Done Done false
9362 3199 Acked Write 3 Sent Done Done false
9363 3200 Acked Read 2 Sent Done Done false
9364 3201 Acked Write 6 Sent Done Done false
9365 3202 Acked Read 7 Sent Done Done false
9366 3203 Acked Read 10 Sent Done Done false
9367 3204 Acked Read 8 Sent Done Done false
9368 3205 Acked Write 4 Sent Done Done false
9369 3206 Acked Flush 0 Sent Done Done false
9370 3207 Acked Write 8 Sent Done Done false
9371 3208 Acked Read 8 Sent Done Done false
9372 3209 Acked Flush 0 Sent Done Done false
9373 3210 Acked Read 5 Sent Done Done false
9374 3211 Acked Read 2 Sent Done Done false
9375 3212 Acked Read 2 Sent Done Done false
9376 3213 Acked Write 9 Sent Done Done false
9377 3214 Acked Read 5 Sent Done Done false
9378 3215 Acked Read 2 Sent Done Done false
9379 3216 Acked Read 7 Sent Done Done false
9380 3217 Acked Read 2 Sent Done Done false
9381 3218 Acked Read 8 Sent Done Done false
9382 3219 Acked Write 7 Sent Done Done false
9383 3220 Acked Write 8 Sent Done Done false
9384 3221 Acked Write 10 Sent Done Done false
9385 3222 Acked Flush 0 Sent Done Done false
9386 3223 Acked Write 1 Sent Done Done false
9387 3224 Acked Read 9 Sent Done Done false
9388 3225 Acked Read 8 Sent Done Done false
9389 3226 Acked Read 1 Sent Done Done false
9390 3227 Acked Write 7 Sent Done Done false
9391 3228 Acked Write 7 Sent Done Done false
9392 3229 Acked Read 2 Sent Done Done false
9393 3230 Acked Write 5 Sent Done Done false
9394 3231 Acked Write 1 Sent Done Done false
9395 3232 Acked Read 7 Sent Done Done false
9396 3233 Acked Read 3 Sent Done Done false
9397 3234 Acked Write 3 Sent Done Done false
9398 3235 Acked Write 3 Sent Done Done false
9399 3236 Acked Write 4 Sent Done Done false
9400 3237 Acked Write 7 Sent Done Done false
9401 3238 Acked Read 4 Sent Done Done false
9402 3239 Acked Write 10 Sent Done Done false
9403 3240 Acked Write 1 Sent Done Done false
9404 3241 Acked Read 5 Sent Done Done false
9405 3242 Acked Read 5 Sent Done Done false
9406 3243 Acked Write 6 Sent Done Done false
9407 3244 Acked Read 2 Sent Done Done false
9408 3245 Acked Flush 0 Sent Done Done false
9409 3246 Acked Read 8 Sent Done Done false
9410 3247 Acked Read 1 Sent Done Done false
9411 3248 Acked Read 6 Sent Done Done false
9412 3249 Acked Read 6 Sent Done Done false
9413 3250 Acked Write 6 Sent Done Done false
9414 3251 Acked Read 8 Sent Done Done false
9415 3252 Acked Flush 0 Sent Done Done false
9416 3253 Acked Read 5 Sent Done Done false
9417 3254 Acked Read 6 Sent Done Done false
9418 3255 Acked Read 10 Sent Done Done false
9419 3256 Acked Read 4 Sent Done Done false
9420 3257 Acked Read 4 Sent Done Done false
9421 3258 Acked Read 10 Sent Done Done false
9422 3259 Acked Write 5 Sent Done Done false
9423 3260 Acked Read 7 Sent Done Done false
9424 3261 Acked Write 7 Sent Done Done false
9425 3262 Acked Write 9 Sent Done Done false
9426 3263 Acked Write 9 Sent Done Done false
9427 3264 Acked Read 3 Sent Done Done false
9428 3265 Acked Read 7 Sent Done Done false
9429 3266 Acked Read 4 Sent Done Done false
9430 3267 Acked Flush 0 Sent Done Done false
9431 3268 Acked Write 1 Sent Done Done false
9432 3269 Acked Read 5 Sent Done Done false
9433 3270 Acked Write 5 Sent Done Done false
9434 3271 Acked Read 1 Sent Done Done false
9435 3272 Acked Flush 0 Sent Done Done false
9436 3273 Acked Write 7 Sent Done Done false
9437 3274 Acked Read 6 Sent Done Done false
9438 3275 Acked Read 4 Sent Done Done false
9439 3276 Acked Flush 0 Sent Done Done false
9440 3277 Acked Read 8 Sent Done Done false
9441 3278 Acked Write 2 Sent Done Done false
9442 3279 Acked Read 3 Sent Done Done false
9443 3280 Acked Write 7 Sent Done Done false
9444 3281 Acked Read 1 Sent Done Done false
9445 3282 Acked Write 2 Sent Done Done false
9446 3283 Acked Flush 0 Sent Done Done false
9447 3284 Acked Write 4 Sent Done Done false
9448 3285 Acked Write 6 Sent Done Done false
9449 3286 Acked Read 5 Sent Done Done false
9450 3287 Acked Flush 0 Sent Done Done false
9451 3288 Acked Read 4 Sent Done Done false
9452 3289 Acked Write 7 Sent Done Done false
9453 3290 Acked Read 2 Sent Done Done false
9454 3291 Acked Write 8 Sent Done Done false
9455 3292 Acked Read 4 Sent Done Done false
9456 3293 Acked Read 4 Sent Done Done false
9457 3294 Acked Read 5 Sent Done Done false
9458 3295 Acked Flush 0 Sent Done Done false
9459 3296 Acked Write 9 Sent Done Done false
9460 3297 Acked Read 6 Sent Done Done false
9461 3298 Acked Write 3 Sent Done Done false
9462 3299 Acked Write 10 Sent Done Done false
9463 3300 Acked Write 9 Sent Done Done false
9464 3301 Acked Write 10 Sent Done Done false
9465 3302 Acked Read 9 Sent Done Done false
9466 3303 Acked Read 4 Sent Done Done false
9467 3304 Acked Read 5 Sent Done Done false
9468 3305 Acked Flush 0 Sent Done Done false
9469 3306 Acked Write 7 Sent Done Done false
9470 3307 Acked Read 7 Sent Done Done false
9471 3308 Acked Read 6 Sent Done Done false
9472 3309 Acked Read 5 Sent Done Done false
9473 3310 Acked Read 9 Sent Done Done false
9474 3311 Acked Read 9 Sent Done Done false
9475 3312 Acked Flush 0 Sent Done Done false
9476 3313 Acked Read 7 Sent Done Done false
9477 3314 Acked Write 4 Sent Done Done false
9478 3315 Acked Write 2 Sent Done Done false
9479 3316 Acked Read 6 Sent Done Done false
9480 3317 Acked Write 2 Sent Done Done false
9481 3318 Acked Read 4 Sent Done Done false
9482 3319 Acked Read 6 Sent Done Done false
9483 3320 Acked Write 6 Sent Done Done false
9484 3321 Acked Flush 0 Sent Done Done false
9485 3322 Acked Read 2 Sent Done Done false
9486 3323 Acked Read 8 Sent Done Done false
9487 3324 Acked Write 4 Sent Done Done false
9488 3325 Acked Write 7 Sent Done Done false
9489 3326 Acked Read 9 Sent Done Done false
9490 3327 Acked Read 4 Sent Done Done false
9491 3328 Acked Read 10 Sent Done Done false
9492 3329 Acked Write 2 Sent Done Done false
9493 3330 Acked Read 3 Sent Done Done false
9494 3331 Acked Read 10 Sent Done Done false
9495 3332 Acked Read 9 Sent Done Done false
9496 3333 Acked Flush 0 Sent Done Done false
9497 3334 Acked Write 1 Sent Done Done false
9498 3335 Acked Write 10 Sent Done Done false
9499 3336 Acked Read 5 Sent Done Done false
9500 3337 Acked Write 4 Sent Done Done false
9501 3338 Acked Read 7 Sent Done Done false
9502 3339 Acked Read 4 Sent Done Done false
9503 3340 Acked Write 1 Sent Done Done false
9504 3341 Acked Read 7 Sent Done Done false
9505 3342 Acked Write 1 Sent Done Done false
9506 3343 Acked Write 1 Sent Done Done false
9507 3344 Acked Write 7 Sent Done Done false
9508 3345 Acked Write 8 Sent Done Done false
9509 3346 Acked Read 5 Sent Done Done false
9510 3347 Acked Read 6 Sent Done Done false
9511 3348 Acked Read 7 Sent Done Done false
9512 3349 Acked Write 5 Sent Done Done false
9513 3350 Acked Read 7 Sent Done Done false
9514 3351 Acked Write 2 Sent Done Done false
9515 3352 Acked Flush 0 Sent Done Done false
9516 3353 Acked Write 8 Sent Done Done false
9517 3354 Acked Write 9 Sent Done Done false
9518 3355 Acked Read 1 Sent Done Done false
9519 3356 Acked Read 3 Sent Done Done false
9520 3357 Acked Read 2 Sent Done Done false
9521 3358 Acked Read 8 Sent Done Done false
9522 3359 Acked Write 1 Sent Done Done false
9523 3360 Acked Read 6 Sent Done Done false
9524 3361 Acked Write 2 Sent Done Done false
9525 3362 Acked Flush 0 Sent Done Done false
9526 3363 Acked Write 7 Sent Done Done false
9527 3364 Acked Read 10 Sent Done Done false
9528 3365 Acked Write 7 Sent Done Done false
9529 3366 Acked Write 8 Sent Done Done false
9530 3367 Acked Read 1 Sent Done Done false
9531 3368 Acked Read 4 Sent Done Done false
9532 3369 Acked Read 8 Sent Done Done false
9533 3370 Acked Write 7 Sent Done Done false
9534 3371 Acked Read 10 Sent Done Done false
9535 3372 Acked Read 7 Sent Done Done false
9536 3373 Acked Write 7 Sent Done Done false
9537 3374 Acked Read 3 Sent Done Done false
9538 3375 Acked Write 4 Sent Done Done false
9539 3376 Acked Write 2 Sent Done Done false
9540 3377 Acked Flush 0 Sent Done Done false
9541 3378 Acked Read 8 Sent Done Done false
9542 3379 Acked Write 9 Sent Done Done false
9543 3380 Acked Read 4 Sent Done Done false
9544 3381 Acked Write 9 Sent Done Done false
9545 3382 Acked Flush 0 Sent Done Done false
9546 3383 Acked Write 3 Sent Done Done false
9547 3384 Acked Write 10 Sent Done Done false
9548 3385 Acked Read 4 Sent Done Done false
9549 3386 Acked Flush 0 Sent Done Done false
9550 3387 Acked Read 2 Sent Done Done false
9551 3388 Acked Read 5 Sent Done Done false
9552 3389 Acked Flush 0 Sent Done Done false
9553 3390 Acked Read 7 Sent Done Done false
9554 3391 Acked Write 3 Sent Done Done false
9555 3392 Acked Flush 0 Sent Done Done false
9556 3393 Acked Read 8 Sent Done Done false
9557 3394 Acked Read 1 Sent Done Done false
9558 3395 Acked Read 3 Sent Done Done false
9559 3396 Acked Write 5 Sent Done Done false
9560 3397 Acked Read 3 Sent Done Done false
9561 3398 Acked Write 8 Sent Done Done false
9562 3399 Acked Read 1 Sent Done Done false
9563 3400 Acked Write 5 Sent Done Done false
9564 3401 Acked Read 1 Sent Done Done false
9565 3402 Acked Write 1 Sent Done Done false
9566 3403 Acked Read 10 Sent Done Done false
9567 3404 Acked Write 1 Sent Done Done false
9568 3405 Acked Write 2 Sent Done Done false
9569 3406 Acked Read 7 Sent Done Done false
9570 3407 Acked Write 1 Sent Done Done false
9571 3408 Acked Flush 0 Sent Done Done false
9572 3409 Acked Write 5 Sent Done Done false
9573 3410 Acked Write 9 Sent Done Done false
9574 3411 Acked Read 10 Sent Done Done false
9575 3412 Acked Read 1 Sent Done Done false
9576 3413 Acked Write 9 Sent Done Done false
9577 3414 Acked Write 4 Sent Done Done false
9578 3415 Acked Flush 0 Sent Done Done false
9579 STATES DS:0 DS:1 DS:2 TOTAL
9580 Sent 302 0 0 302
9581 Done 0 302 302 604
9582 Skipped 0 0 0 0
9583 Error 0 0 0 0
9584 Last Flush: 3113 3415 3415
9585 Downstairs last five completed: 3113 3112 3111 3110 3109
9586 Upstairs last five completed: 3415 3414 3413 3412 3411
9587 CLIENT: Up:0 ds:302 act:2
9588 ----------------------------------------------------------------
9589 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9590 States: Offline Active Active
9591 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9592 3114 Acked Read 10 Sent Done Done false
9593 3115 Acked Read 10 Sent Done Done false
9594 3116 Acked Write 8 Sent Done Done false
9595 3117 Acked Write 10 Sent Done Done false
9596 3118 Acked Write 7 Sent Done Done false
9597 3119 Acked Write 10 Sent Done Done false
9598 3120 Acked Read 4 Sent Done Done false
9599 3121 Acked Read 3 Sent Done Done false
9600 3122 Acked Write 1 Sent Done Done false
9601 3123 Acked Read 1 Sent Done Done false
9602 3124 Acked Read 2 Sent Done Done false
9603 3125 Acked Flush 0 Sent Done Done false
9604 3126 Acked Read 9 Sent Done Done false
9605 3127 Acked Read 6 Sent Done Done false
9606 3128 Acked Flush 0 Sent Done Done false
9607 3129 Acked Write 3 Sent Done Done false
9608 3130 Acked Read 1 Sent Done Done false
9609 3131 Acked Flush 0 Sent Done Done false
9610 3132 Acked Write 6 Sent Done Done false
9611 3133 Acked Write 10 Sent Done Done false
9612 3134 Acked Write 1 Sent Done Done false
9613 3135 Acked Write 3 Sent Done Done false
9614 3136 Acked Read 9 Sent Done Done false
9615 3137 Acked Read 5 Sent Done Done false
9616 3138 Acked Read 2 Sent Done Done false
9617 3139 Acked Write 2 Sent Done Done false
9618 3140 Acked Write 3 Sent Done Done false
9619 3141 Acked Read 6 Sent Done Done false
9620 3142 Acked Read 5 Sent Done Done false
9621 3143 Acked Read 3 Sent Done Done false
9622 3144 Acked Read 8 Sent Done Done false
9623 3145 Acked Read 1 Sent Done Done false
9624 3146 Acked Write 1 Sent Done Done false
9625 3147 Acked Write 6 Sent Done Done false
9626 3148 Acked Read 10 Sent Done Done false
9627 3149 Acked Write 4 Sent Done Done false
9628 3150 Acked Read 3 Sent Done Done false
9629 3151 Acked Flush 0 Sent Done Done false
9630 3152 Acked Write 8 Sent Done Done false
9631 3153 Acked Flush 0 Sent Done Done false
9632 3154 Acked Flush 0 Sent Done Done false
9633 3155 Acked Write 6 Sent Done Done false
9634 3156 Acked Read 3 Sent Done Done false
9635 3157 Acked Read 7 Sent Done Done false
9636 3158 Acked Flush 0 Sent Done Done false
9637 3159 Acked Read 2 Sent Done Done false
9638 3160 Acked Read 9 Sent Done Done false
9639 3161 Acked Write 7 Sent Done Done false
9640 3162 Acked Write 8 Sent Done Done false
9641 3163 Acked Write 8 Sent Done Done false
9642 3164 Acked Write 3 Sent Done Done false
9643 3165 Acked Read 5 Sent Done Done false
9644 3166 Acked Write 7 Sent Done Done false
9645 3167 Acked Read 1 Sent Done Done false
9646 3168 Acked Flush 0 Sent Done Done false
9647 3169 Acked Read 7 Sent Done Done false
9648 3170 Acked Write 10 Sent Done Done false
9649 3171 Acked Write 5 Sent Done Done false
9650 3172 Acked Read 1 Sent Done Done false
9651 3173 Acked Read 1 Sent Done Done false
9652 3174 Acked Read 7 Sent Done Done false
9653 3175 Acked Read 5 Sent Done Done false
9654 3176 Acked Read 3 Sent Done Done false
9655 3177 Acked Write 1 Sent Done Done false
9656 3178 Acked Read 7 Sent Done Done false
9657 3179 Acked Read 4 Sent Done Done false
9658 3180 Acked Read 6 Sent Done Done false
9659 3181 Acked Write 5 Sent Done Done false
9660 3182 Acked Write 3 Sent Done Done false
9661 3183 Acked Write 2 Sent Done Done false
9662 3184 Acked Write 6 Sent Done Done false
9663 3185 Acked Flush 0 Sent Done Done false
9664 3186 Acked Flush 0 Sent Done Done false
9665 3187 Acked Flush 0 Sent Done Done false
9666 3188 Acked Write 4 Sent Done Done false
9667 3189 Acked Write 2 Sent Done Done false
9668 3190 Acked Read 2 Sent Done Done false
9669 3191 Acked Read 2 Sent Done Done false
9670 3192 Acked Read 7 Sent Done Done false
9671 3193 Acked Write 4 Sent Done Done false
9672 3194 Acked Write 10 Sent Done Done false
9673 3195 Acked Write 5 Sent Done Done false
9674 3196 Acked Read 9 Sent Done Done false
9675 3197 Acked Write 1 Sent Done Done false
9676 3198 Acked Write 2 Sent Done Done false
9677 3199 Acked Write 3 Sent Done Done false
9678 3200 Acked Read 2 Sent Done Done false
9679 3201 Acked Write 6 Sent Done Done false
9680 3202 Acked Read 7 Sent Done Done false
9681 3203 Acked Read 10 Sent Done Done false
9682 3204 Acked Read 8 Sent Done Done false
9683 3205 Acked Write 4 Sent Done Done false
9684 3206 Acked Flush 0 Sent Done Done false
9685 3207 Acked Write 8 Sent Done Done false
9686 3208 Acked Read 8 Sent Done Done false
9687 3209 Acked Flush 0 Sent Done Done false
9688 3210 Acked Read 5 Sent Done Done false
9689 3211 Acked Read 2 Sent Done Done false
9690 3212 Acked Read 2 Sent Done Done false
9691 3213 Acked Write 9 Sent Done Done false
9692 3214 Acked Read 5 Sent Done Done false
9693 3215 Acked Read 2 Sent Done Done false
9694 3216 Acked Read 7 Sent Done Done false
9695 3217 Acked Read 2 Sent Done Done false
9696 3218 Acked Read 8 Sent Done Done false
9697 3219 Acked Write 7 Sent Done Done false
9698 3220 Acked Write 8 Sent Done Done false
9699 3221 Acked Write 10 Sent Done Done false
9700 3222 Acked Flush 0 Sent Done Done false
9701 3223 Acked Write 1 Sent Done Done false
9702 3224 Acked Read 9 Sent Done Done false
9703 3225 Acked Read 8 Sent Done Done false
9704 3226 Acked Read 1 Sent Done Done false
9705 3227 Acked Write 7 Sent Done Done false
9706 3228 Acked Write 7 Sent Done Done false
9707 3229 Acked Read 2 Sent Done Done false
9708 3230 Acked Write 5 Sent Done Done false
9709 3231 Acked Write 1 Sent Done Done false
9710 3232 Acked Read 7 Sent Done Done false
9711 3233 Acked Read 3 Sent Done Done false
9712 3234 Acked Write 3 Sent Done Done false
9713 3235 Acked Write 3 Sent Done Done false
9714 3236 Acked Write 4 Sent Done Done false
9715 3237 Acked Write 7 Sent Done Done false
9716 3238 Acked Read 4 Sent Done Done false
9717 3239 Acked Write 10 Sent Done Done false
9718 3240 Acked Write 1 Sent Done Done false
9719 3241 Acked Read 5 Sent Done Done false
9720 3242 Acked Read 5 Sent Done Done false
9721 3243 Acked Write 6 Sent Done Done false
9722 3244 Acked Read 2 Sent Done Done false
9723 3245 Acked Flush 0 Sent Done Done false
9724 3246 Acked Read 8 Sent Done Done false
9725 3247 Acked Read 1 Sent Done Done false
9726 3248 Acked Read 6 Sent Done Done false
9727 3249 Acked Read 6 Sent Done Done false
9728 3250 Acked Write 6 Sent Done Done false
9729 3251 Acked Read 8 Sent Done Done false
9730 3252 Acked Flush 0 Sent Done Done false
9731 3253 Acked Read 5 Sent Done Done false
9732 3254 Acked Read 6 Sent Done Done false
9733 3255 Acked Read 10 Sent Done Done false
9734 3256 Acked Read 4 Sent Done Done false
9735 3257 Acked Read 4 Sent Done Done false
9736 3258 Acked Read 10 Sent Done Done false
9737 3259 Acked Write 5 Sent Done Done false
9738 3260 Acked Read 7 Sent Done Done false
9739 3261 Acked Write 7 Sent Done Done false
9740 3262 Acked Write 9 Sent Done Done false
9741 3263 Acked Write 9 Sent Done Done false
9742 3264 Acked Read 3 Sent Done Done false
9743 3265 Acked Read 7 Sent Done Done false
9744 3266 Acked Read 4 Sent Done Done false
9745 3267 Acked Flush 0 Sent Done Done false
9746 3268 Acked Write 1 Sent Done Done false
9747 3269 Acked Read 5 Sent Done Done false
9748 3270 Acked Write 5 Sent Done Done false
9749 3271 Acked Read 1 Sent Done Done false
9750 3272 Acked Flush 0 Sent Done Done false
9751 3273 Acked Write 7 Sent Done Done false
9752 3274 Acked Read 6 Sent Done Done false
9753 3275 Acked Read 4 Sent Done Done false
9754 3276 Acked Flush 0 Sent Done Done false
9755 3277 Acked Read 8 Sent Done Done false
9756 3278 Acked Write 2 Sent Done Done false
9757 3279 Acked Read 3 Sent Done Done false
9758 3280 Acked Write 7 Sent Done Done false
9759 3281 Acked Read 1 Sent Done Done false
9760 3282 Acked Write 2 Sent Done Done false
9761 3283 Acked Flush 0 Sent Done Done false
9762 3284 Acked Write 4 Sent Done Done false
9763 3285 Acked Write 6 Sent Done Done false
9764 3286 Acked Read 5 Sent Done Done false
9765 3287 Acked Flush 0 Sent Done Done false
9766 3288 Acked Read 4 Sent Done Done false
9767 3289 Acked Write 7 Sent Done Done false
9768 3290 Acked Read 2 Sent Done Done false
9769 3291 Acked Write 8 Sent Done Done false
9770 3292 Acked Read 4 Sent Done Done false
9771 3293 Acked Read 4 Sent Done Done false
9772 3294 Acked Read 5 Sent Done Done false
9773 3295 Acked Flush 0 Sent Done Done false
9774 3296 Acked Write 9 Sent Done Done false
9775 3297 Acked Read 6 Sent Done Done false
9776 3298 Acked Write 3 Sent Done Done false
9777 3299 Acked Write 10 Sent Done Done false
9778 3300 Acked Write 9 Sent Done Done false
9779 3301 Acked Write 10 Sent Done Done false
9780 3302 Acked Read 9 Sent Done Done false
9781 3303 Acked Read 4 Sent Done Done false
9782 3304 Acked Read 5 Sent Done Done false
9783 3305 Acked Flush 0 Sent Done Done false
9784 3306 Acked Write 7 Sent Done Done false
9785 3307 Acked Read 7 Sent Done Done false
9786 3308 Acked Read 6 Sent Done Done false
9787 3309 Acked Read 5 Sent Done Done false
9788 3310 Acked Read 9 Sent Done Done false
9789 3311 Acked Read 9 Sent Done Done false
9790 3312 Acked Flush 0 Sent Done Done false
9791 3313 Acked Read 7 Sent Done Done false
9792 3314 Acked Write 4 Sent Done Done false
9793 3315 Acked Write 2 Sent Done Done false
9794 3316 Acked Read 6 Sent Done Done false
9795 3317 Acked Write 2 Sent Done Done false
9796 3318 Acked Read 4 Sent Done Done false
9797 3319 Acked Read 6 Sent Done Done false
9798 3320 Acked Write 6 Sent Done Done false
9799 3321 Acked Flush 0 Sent Done Done false
9800 3322 Acked Read 2 Sent Done Done false
9801 3323 Acked Read 8 Sent Done Done false
9802 3324 Acked Write 4 Sent Done Done false
9803 3325 Acked Write 7 Sent Done Done false
9804 3326 Acked Read 9 Sent Done Done false
9805 3327 Acked Read 4 Sent Done Done false
9806 3328 Acked Read 10 Sent Done Done false
9807 3329 Acked Write 2 Sent Done Done false
9808 3330 Acked Read 3 Sent Done Done false
9809 3331 Acked Read 10 Sent Done Done false
9810 3332 Acked Read 9 Sent Done Done false
9811 3333 Acked Flush 0 Sent Done Done false
9812 3334 Acked Write 1 Sent Done Done false
9813 3335 Acked Write 10 Sent Done Done false
9814 3336 Acked Read 5 Sent Done Done false
9815 3337 Acked Write 4 Sent Done Done false
9816 3338 Acked Read 7 Sent Done Done false
9817 3339 Acked Read 4 Sent Done Done false
9818 3340 Acked Write 1 Sent Done Done false
9819 3341 Acked Read 7 Sent Done Done false
9820 3342 Acked Write 1 Sent Done Done false
9821 3343 Acked Write 1 Sent Done Done false
9822 3344 Acked Write 7 Sent Done Done false
9823 3345 Acked Write 8 Sent Done Done false
9824 3346 Acked Read 5 Sent Done Done false
9825 3347 Acked Read 6 Sent Done Done false
9826 3348 Acked Read 7 Sent Done Done false
9827 3349 Acked Write 5 Sent Done Done false
9828 3350 Acked Read 7 Sent Done Done false
9829 3351 Acked Write 2 Sent Done Done false
9830 3352 Acked Flush 0 Sent Done Done false
9831 3353 Acked Write 8 Sent Done Done false
9832 3354 Acked Write 9 Sent Done Done false
9833 3355 Acked Read 1 Sent Done Done false
9834 3356 Acked Read 3 Sent Done Done false
9835 3357 Acked Read 2 Sent Done Done false
9836 3358 Acked Read 8 Sent Done Done false
9837 3359 Acked Write 1 Sent Done Done false
9838 3360 Acked Read 6 Sent Done Done false
9839 3361 Acked Write 2 Sent Done Done false
9840 3362 Acked Flush 0 Sent Done Done false
9841 3363 Acked Write 7 Sent Done Done false
9842 3364 Acked Read 10 Sent Done Done false
9843 3365 Acked Write 7 Sent Done Done false
9844 3366 Acked Write 8 Sent Done Done false
9845 3367 Acked Read 1 Sent Done Done false
9846 3368 Acked Read 4 Sent Done Done false
9847 3369 Acked Read 8 Sent Done Done false
9848 3370 Acked Write 7 Sent Done Done false
9849 3371 Acked Read 10 Sent Done Done false
9850 3372 Acked Read 7 Sent Done Done false
9851 3373 Acked Write 7 Sent Done Done false
9852 3374 Acked Read 3 Sent Done Done false
9853 3375 Acked Write 4 Sent Done Done false
9854 3376 Acked Write 2 Sent Done Done false
9855 3377 Acked Flush 0 Sent Done Done false
9856 3378 Acked Read 8 Sent Done Done false
9857 3379 Acked Write 9 Sent Done Done false
9858 3380 Acked Read 4 Sent Done Done false
9859 3381 Acked Write 9 Sent Done Done false
9860 3382 Acked Flush 0 Sent Done Done false
9861 3383 Acked Write 3 Sent Done Done false
9862 3384 Acked Write 10 Sent Done Done false
9863 3385 Acked Read 4 Sent Done Done false
9864 3386 Acked Flush 0 Sent Done Done false
9865 3387 Acked Read 2 Sent Done Done false
9866 3388 Acked Read 5 Sent Done Done false
9867 3389 Acked Flush 0 Sent Done Done false
9868 3390 Acked Read 7 Sent Done Done false
9869 3391 Acked Write 3 Sent Done Done false
9870 3392 Acked Flush 0 Sent Done Done false
9871 3393 Acked Read 8 Sent Done Done false
9872 3394 Acked Read 1 Sent Done Done false
9873 3395 Acked Read 3 Sent Done Done false
9874 3396 Acked Write 5 Sent Done Done false
9875 3397 Acked Read 3 Sent Done Done false
9876 3398 Acked Write 8 Sent Done Done false
9877 3399 Acked Read 1 Sent Done Done false
9878 3400 Acked Write 5 Sent Done Done false
9879 3401 Acked Read 1 Sent Done Done false
9880 3402 Acked Write 1 Sent Done Done false
9881 3403 Acked Read 10 Sent Done Done false
9882 3404 Acked Write 1 Sent Done Done false
9883 3405 Acked Write 2 Sent Done Done false
9884 3406 Acked Read 7 Sent Done Done false
9885 3407 Acked Write 1 Sent Done Done false
9886 3408 Acked Flush 0 Sent Done Done false
9887 3409 Acked Write 5 Sent Done Done false
9888 3410 Acked Write 9 Sent Done Done false
9889 3411 Acked Read 10 Sent Done Done false
9890 3412 Acked Read 1 Sent Done Done false
9891 3413 Acked Write 9 Sent Done Done false
9892 3414 Acked Write 4 Sent Done Done false
9893 3415 Acked Flush 0 Sent Done Done false
9894 STATES DS:0 DS:1 DS:2 TOTAL
9895 Sent 302 0 0 302
9896 Done 0 302 302 604
9897 Skipped 0 0 0 0
9898 Error 0 0 0 0
9899 Last Flush: 3113 3415 3415
9900 Downstairs last five completed: 3113 3112 3111 3110 3109
9901 Upstairs last five completed: 3415 3414 3413 3412 3411
9902 CLIENT: Up:0 ds:302 act:2
9903 [8810] got start action
9904 [8810] I am going to restart
9905 Starting downstairs at port 8810
9906 Make output file at "/tmp/dsc/downstairs-8810.txt"
9907 [8810][0] reports Starting
9908 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1208)
9909 [8810][0] reports Running
99102024-12-11T23:42:36.263ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99112024-12-11T23:42:36.263ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44113) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99122024-12-11T23:42:36.264ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99132024-12-11T23:42:36.265ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99142024-12-11T23:42:36.265ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99152024-12-11T23:42:36.265ZINFOcrucible: send last flush ID to this DS: 3113 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99162024-12-11T23:42:36.265ZINFOcrucible: Replied this last flush ID: 3113 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99172024-12-11T23:42:36.265ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99182024-12-11T23:42:36.265ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99192024-12-11T23:42:36.265ZINFOcrucible: [0] client re-new 302 jobs since flush 3113 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99202024-12-11T23:42:36.265ZINFOcrucible: [0] Replayed 302 jobs since flush: 3113 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
9921 ----------------------------------------------------------------
9922 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9923 Downstairs last five completed: 3415 3414 3413 3412 3411
9924 Upstairs last five completed: 3415 3414 3413 3412 3411
9925 CLIENT: Up:0 ds:0 act:3
9926 Replay: All jobs finished
9927 Main task has work to do, go find it
9928 got dsc Stop(2)
9929 stop 2
9930 Dec 11 23:42:36.501 [8830] Got stop action so:false kr:false
9931 INFO request completed, latency_us: 359, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 22d3a2f0-58ed-4e33-a3f5-3772c6435fba, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
99322024-12-11T23:42:36.502ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99332024-12-11T23:42:36.502ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99342024-12-11T23:42:36.502ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99352024-12-11T23:42:36.502ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
99362024-12-11T23:42:36.502ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
9937 Dec 11 23:42:36.502 INFO request completed{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:42:36.502598512Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
9938 , latency_us: 167, response_code: 200, uri: /state/cid/2, method: GET, req_id: f254f528-ac65-4e10-ac92-58e8a4a5036c, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
9939 [8830] Exited with: ExitStatus(unix_wait_status(9))
9940 [8830][2] reports Exit
9941 Dec 11 23:42:40.504 INFO request completed, latency_us: 237, response_code: 200, uri: /state/cid/2, method: GET, req_id: 93d05036-45d3-4db7-a766-b5ca75e40693, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
9942 001/300 Write block 3939 len 4096 data: 3
9943 002/300 Read block 4210 len 20480
9944 003/300 Write block 1709 len 24576 data: 2 2 2 2 2 2
9945 004/300 Read block 365 len 40960
9946 005/300 Read block 4243 len 36864
9947 006/300 Write block 4622 len 16384 data: 3 3 2 2
9948 007/300 Read block 1617 len 28672
9949 008/300 Flush
9950 009/300 Read block 2768 len 12288
9951 010/300 Write block 3449 len 16384 data: 3 3 3 3
9952 011/300 Write block 4890 len 24576 data: 3 3 4 4 4 4
9953 012/300 Read block 1181 len 16384
9954 013/300 Write block 1629 len 20480 data: 4 4 4 2 2
9955 014/300 Read block 746 len 24576
9956 015/300 Read block 945 len 8192
9957 016/300 Write block 3357 len 12288 data: 2 2 2
9958 017/300 Write block 1961 len 32768 data: 2 2 2 2 2 2 2 2
9959 018/300 Write block 4737 len 4096 data: 3
9960 019/300 Write block 4847 len 40960 data: 3 2 2 2 2 3 3 3 3 3
9961 020/300 Write block 875 len 28672 data: 4 4 6 5 5 5 5
9962 021/300 Read block 1537 len 24576
9963 022/300 Write block 4381 len 40960 data: 3 3 3 3 3 3 3 3 3 4
9964 023/300 Write block 4139 len 4096 data: 3
9965 024/300 Write block 4971 len 12288 data: 4 3 3
9966 025/300 Write block 1065 len 28672 data: 4 3 3 3 3 3 3
9967 026/300 Read block 1409 len 40960
9968 027/300 Write block 2681 len 32768 data: 3 3 3 3 4 4 5 4
9969 028/300 Read block 3364 len 28672
9970 029/300 Flush
9971 030/300 Flush
9972 031/300 Read block 3557 len 8192
9973 032/300 Write block 3690 len 36864 data: 3 3 2 2 3 4 4 5 5
9974 033/300 Read block 4239 len 4096
9975 034/300 Write block 702 len 24576 data: 2 2 2 2 2 2
9976 035/300 Write block 2569 len 40960 data: 3 3 3 3 3 5 6 6 6 6
9977 036/300 Read block 2539 len 8192
9978 037/300 Read block 817 len 12288
9979 038/300 Read block 4569 len 20480
9980 039/300 Write block 638 len 12288 data: 3 4 4
9981 040/300 Write block 4922 len 32768 data: 7 6 7 5 6 6 6 6
9982 041/300 Read block 3636 len 24576
9983 042/300 Write block 4610 len 40960 data: 3 2 2 3 4 4 5 5 3 3
9984 043/300 Read block 4337 len 16384
9985 044/300 Write block 3899 len 24576 data: 3 3 3 2 2 3
9986 045/300 Flush
9987 046/300 Read block 1546 len 16384
9988 047/300 Read block 2714 len 24576
9989 048/300 Read block 3497 len 40960
9990 049/300 Write block 1124 len 4096 data: 2
9991 050/300 Read block 3087 len 32768
9992 051/300 Write block 2605 len 32768 data: 4 2 2 3 3 5 5 4
9993 052/300 Write block 3420 len 16384 data: 5 4 4 3
9994 053/300 Read block 4111 len 28672
9995 054/300 Write block 1692 len 8192 data: 3 4
9996 055/300 Read block 1067 len 8192
9997 056/300 Write block 4751 len 4096 data: 3
9998 057/300 Write block 2404 len 12288 data: 2 2 2
9999 058/300 Write block 1579 len 32768 data: 2 2 2 2 2 2 2 2
10000 059/300 Read block 1471 len 8192
10001 060/300 Read block 107 len 24576
10002 061/300 Read block 2223 len 20480
10003 062/300 Write block 1379 len 40960 data: 2 3 3 3 3 3 4 2 2 2
10004 063/300 Read block 3625 len 4096
10005 064/300 Write block 4936 len 40960 data: 2 3 3 3 3 3 3 3 4 4
10006 065/300 Write block 1632 len 20480 data: 3 3 2 3 3
10007 066/300 Read block 1077 len 24576
10008 067/300 Write block 767 len 8192 data: 3 3
10009 068/300 Read block 163 len 8192
10010 069/300 Read block 3778 len 28672
10011 070/300 Flush
10012 071/300 Read block 3754 len 12288
10013 072/300 Read block 474 len 16384
10014 073/300 Write block 2293 len 36864 data: 4 3 3 3 2 2 2 2 3
10015 074/300 Write block 1083 len 40960 data: 3 3 4 4 4 4 4 4 5 6
10016 075/300 Read block 1359 len 24576
10017 076/300 Write block 4042 len 24576 data: 3 3 3 3 3 4
10018 077/300 Write block 4968 len 40960 data: 4 4 4 5 4 4 3 2 2 2
10019 078/300 Write block 4347 len 28672 data: 4 5 3 2 2 2 3
10020 079/300 Read block 3341 len 4096
10021 080/300 Write block 692 len 4096 data: 3
10022 081/300 Read block 1513 len 20480
10023 082/300 Write block 2386 len 12288 data: 2 2 2
10024 083/300 Read block 2311 len 20480
10025 084/300 Read block 4082 len 36864
10026 085/300 Flush
10027 086/300 Write block 4038 len 40960 data: 3 3 3 3 4 4 4 4 4 5
10028 087/300 Write block 4522 len 36864 data: 5 4 4 5 6 5 5 5 7
10029 088/300 Write block 951 len 20480 data: 3 3 3 3 2
10030 089/300 Read block 1850 len 32768
10031 090/300 Write block 1751 len 36864 data: 2 2 2 2 2 2 3 3 3
10032 091/300 Read block 4857 len 28672
10033 092/300 Write block 1413 len 20480 data: 5 5 3 3 3
10034 093/300 Write block 2334 len 12288 data: 5 5 4
10035 094/300 Read block 1653 len 36864
10036 095/300 Flush
10037 096/300 Read block 1318 len 24576
10038 097/300 Write block 1108 len 12288 data: 2 2 3
10039 098/300 Read block 3833 len 32768
10040 099/300 Read block 51 len 4096
10041 100/300 Flush
10042 101/300 Read block 3562 len 8192
10043 102/300 Write block 3324 len 32768 data: 2 2 3 4 4 4 4 3
10044 103/300 Write block 943 len 12288 data: 3 3 3
10045 104/300 Write block 3156 len 16384 data: 5 4 5 5
10046 105/300 Read block 2890 len 32768
10047 106/300 Write block 4040 len 20480 data: 4 4 5 5 5
10048 107/300 Flush
10049 108/300 Read block 863 len 20480
10050 109/300 Write block 1451 len 28672 data: 2 2 2 2 2 3 3
10051 110/300 Write block 1082 len 40960 data: 3 4 4 5 5 5 5 5 5 6
10052 111/300 Read block 83 len 28672
10053 112/300 Write block 357 len 16384 data: 4 3 3 3
10054 113/300 Write block 2094 len 8192 data: 6 5
10055 114/300 Write block 4296 len 16384 data: 3 3 3 3
10056 115/300 Read block 4987 len 12288
10057 116/300 Read block 3727 len 8192
10058 117/300 Read block 543 len 8192
10059 118/300 Flush
10060 119/300 Write block 3511 len 20480 data: 3 3 3 3 3
10061 120/300 Write block 3893 len 24576 data: 3 3 3 3 3 3
10062 121/300 Read block 2113 len 12288
10063 122/300 Write block 3816 len 12288 data: 5 5 5
10064 123/300 Read block 402 len 24576
10065 124/300 Write block 524 len 16384 data: 3 3 2 2
10066 125/300 Flush
10067 126/300 Write block 3469 len 16384 data: 2 2 2 2
10068 127/300 Write block 76 len 16384 data: 3 4 4 3
10069 128/300 Read block 325 len 32768
10070 129/300 Read block 95 len 8192
10071 130/300 Write block 1351 len 28672 data: 2 2 2 2 2 2 2
10072 131/300 Read block 81 len 32768
10073 132/300 Flush
10074 133/300 Flush
10075 134/300 Write block 3918 len 36864 data: 2 2 2 2 4 3 2 2 2
10076 135/300 Write block 2855 len 4096 data: 6
10077 136/300 Read block 438 len 4096
10078 137/300 Read block 764 len 28672
10079 138/300 Flush
10080 139/300 Write block 4978 len 24576 data: 2 2 2 2 2 2
10081 140/300 Read block 4659 len 16384
10082 141/300 Write block 3638 len 32768 data: 3 3 4 4 4 4 3 3
10083 142/300 Write block 1860 len 28672 data: 3 3 3 2 2 2 2
10084 143/300 Flush
10085 144/300 Write block 1544 len 20480 data: 3 3 3 2 2
10086 145/300 Read block 695 len 12288
10087 146/300 Write block 25 len 4096 data: 3
10088 147/300 Write block 855 len 32768 data: 4 3 3 3 3 3 3 4
10089 148/300 Read block 4109 len 28672
10090 149/300 Read block 276 len 32768
10091 150/300 Write block 458 len 28672 data: 2 2 2 2 2 2 2
10092 151/300 Write block 1349 len 28672 data: 2 2 3 3 3 3 3
10093 152/300 Flush
10094 153/300 Flush
10095 154/300 Write block 3666 len 32768 data: 3 3 3 3 3 3 3 3
10096 155/300 Read block 785 len 8192
10097 156/300 Write block 2896 len 32768 data: 3 4 6 6 6 6 6 4
10098 157/300 Read block 2164 len 20480
10099 158/300 Write block 1483 len 8192 data: 4 4
10100 159/300 Write block 1537 len 20480 data: 2 2 2 2 3
10101 160/300 Read block 4587 len 16384
10102 161/300 Flush
10103 162/300 Write block 4469 len 32768 data: 5 6 6 6 4 4 4 3
10104 163/300 Read block 1034 len 28672
10105 164/300 Write block 3948 len 4096 data: 5
10106 165/300 Write block 1195 len 24576 data: 4 4 4 3 2 2
10107 166/300 Write block 2877 len 16384 data: 2 2 2 2
10108 167/300 Write block 1226 len 16384 data: 2 2 3 3
10109 168/300 Write block 1984 len 40960 data: 7 6 5 5 4 4 3 3 4 3
10110 169/300 Write block 2643 len 36864 data: 2 3 3 4 4 4 5 5 3
10111 170/300 Read block 3624 len 36864
10112 171/300 Write block 771 len 12288 data: 4 4 3
10113 172/300 Read block 985 len 4096
10114 173/300 Flush
10115 174/300 Read block 2698 len 20480
10116 175/300 Write block 3166 len 20480 data: 3 3 4 4 4
10117 176/300 Read block 4001 len 28672
10118 177/300 Read block 1484 len 24576
10119 178/300 Read block 488 len 4096
10120 179/300 Read block 4955 len 36864
10121 180/300 Read block 2844 len 24576
10122 181/300 Read block 715 len 4096
10123 182/300 Write block 614 len 32768 data: 2 2 2 2 2 2 2 2
10124 183/300 Read block 3593 len 20480
10125 184/300 Read block 3598 len 4096
10126 185/300 Flush
10127 186/300 Read block 3325 len 24576
10128 187/300 Read block 1845 len 8192
10129 188/300 Flush
10130 189/300 Read block 64 len 36864
10131 190/300 Read block 3567 len 20480
10132 191/300 Write block 2379 len 4096 data: 3
10133 192/300 Read block 3927 len 28672
10134 193/300 Read block 128 len 12288
10135 194/300 Read block 1549 len 40960
10136 195/300 Read block 1889 len 8192
10137 196/300 Read block 323 len 16384
10138 197/300 Flush
10139 198/300 Read block 1387 len 32768
10140 199/300 Flush
10141 200/300 Read block 2034 len 40960
10142 201/300 Read block 2858 len 8192
10143 202/300 Write block 3878 len 12288 data: 4 3 2
10144 203/300 Read block 4819 len 36864
10145 204/300 Write block 582 len 4096 data: 4
10146 205/300 Write block 183 len 28672 data: 4 3 3 3 3 2 2
10147 206/300 Read block 4311 len 20480
10148 207/300 Write block 2338 len 24576 data: 3 4 4 3 3 2
10149 208/300 Read block 471 len 28672
10150 209/300 Write block 1125 len 16384 data: 2 2 3 3
10151 210/300 Flush
10152 211/300 Write block 629 len 20480 data: 2 2 2 2 2
10153 212/300 Write block 4017 len 36864 data: 3 4 3 3 2 2 2 2 2
10154 213/300 Write block 4886 len 16384 data: 2 3 3 3
10155 214/300 Write block 99 len 20480 data: 3 3 4 4 4
10156 215/300 Read block 1997 len 40960
10157 216/300 Write block 4457 len 32768 data: 2 2 2 2 2 2 2 3
10158 217/300 Write block 4186 len 28672 data: 4 3 3 2 2 2 3
10159 218/300 Read block 2018 len 16384
10160 219/300 Write block 3777 len 16384 data: 2 2 2 3
10161 220/300 Write block 2835 len 16384 data: 6 4 4 3
10162 221/300 Read block 1128 len 28672
10163 222/300 Read block 3913 len 4096
10164 223/300 Write block 4958 len 20480 data: 3 3 3 4 4
10165 224/300 Write block 4147 len 28672 data: 3 2 2 2 2 2 2
10166 225/300 Read block 4255 len 28672
10167 226/300 Write block 584 len 8192 data: 4 2
10168 227/300 Read block 4459 len 32768
10169 228/300 Flush
10170 229/300 Write block 3201 len 20480 data: 3 3 3 4 5
10171 230/300 Write block 757 len 28672 data: 2 2 2 2 2 2 3
10172 231/300 Read block 4079 len 40960
10173 232/300 Write block 3861 len 32768 data: 5 6 6 6 5 4 5 5
10174 233/300 Write block 3682 len 32768 data: 2 2 2 2 2 2 3 3
10175 234/300 Read block 3604 len 28672
10176 235/300 Read block 2480 len 32768
10177 236/300 Read block 3635 len 4096
10178 237/300 Write block 4806 len 20480 data: 2 2 2 2 2
10179 238/300 Write block 1277 len 8192 data: 4 4
10180 239/300 Write block 4059 len 20480 data: 3 4 4 4 3
10181 240/300 Read block 3944 len 20480
10182 241/300 Read block 2225 len 16384
10183 242/300 Write block 2934 len 24576 data: 5 5 6 6 5 4
10184 243/300 Write block 4480 len 28672 data: 3 3 3 3 3 3 3
10185 244/300 Write block 914 len 4096 data: 3
10186 245/300 Write block 4148 len 32768 data: 3 3 3 3 3 3 2 2
10187 246/300 Read block 3107 len 28672
10188 247/300 Read block 1734 len 40960
10189 248/300 Write block 1349 len 32768 data: 3 3 4 4 4 4 4 3
10190 249/300 Write block 4764 len 4096 data: 2
10191 250/300 Read block 2438 len 20480
10192 251/300 Write block 202 len 12288 data: 3 3 5
10193 252/300 Write block 1108 len 36864 data: 3 3 4 3 3 3 3 3 4
10194 253/300 Write block 45 len 40960 data: 5 5 4 4 5 5 3 2 2 3
10195 254/300 Write block 582 len 20480 data: 5 4 5 3 2
10196 255/300 Write block 2857 len 36864 data: 6 5 5 4 3 3 2 3 3
10197 256/300 Write block 2113 len 28672 data: 5 6 6 6 4 3 3
10198 257/300 Read block 1738 len 32768
10199 258/300 Read block 206 len 16384
10200 259/300 Write block 2304 len 36864 data: 3 3 3 3 3 4 4 4 3
10201 260/300 Write block 3886 len 32768 data: 4 4 3 3 3 3 4 4
10202 261/300 Read block 1164 len 8192
10203 262/300 Read block 4107 len 24576
10204 263/300 Read block 3550 len 8192
10205 264/300 Read block 1164 len 20480
10206 265/300 Write block 3646 len 12288 data: 4 3 3
10207 266/300 Write block 2901 len 20480 data: 7 7 5 2 2
10208 267/300 Read block 3234 len 16384
10209 268/300 Write block 2155 len 20480 data: 3 3 3 3 3
10210 269/300 Read block 3173 len 36864
10211 270/300 Read block 3229 len 8192
10212 271/300 Flush
10213 272/300 Write block 3371 len 16384 data: 3 3 3 3
10214 273/300 Write block 2430 len 32768 data: 4 4 2 2 2 2 2 3
10215 274/300 Write block 1536 len 36864 data: 2 3 3 3 3 4 3 3 4
10216 275/300 Flush
10217 276/300 Read block 4085 len 16384
10218 277/300 Read block 3736 len 32768
10219 278/300 Write block 3857 len 24576 data: 5 5 5 5 6 7
10220 279/300 Read block 110 len 4096
10221 280/300 Read block 3693 len 20480
10222 281/300 Read block 1067 len 40960
10223 282/300 Read block 511 len 24576
10224 283/300 Write block 71 len 16384 data: 2 2 2 2
10225 284/300 Read block 4617 len 32768
10226 285/300 Write block 3530 len 24576 data: 3 2 2 2 2 2
10227 286/300 Read block 2500 len 20480
10228 287/300 Read block 2667 len 4096
10229 288/300 Write block 2806 len 24576 data: 3 2 2 2 3 3
10230 289/300 Read block 4282 len 40960
10231 290/300 Read block 1464 len 16384
10232 291/300 Write block 4310 len 28672 data: 2 2 2 2 3 3 3
10233 292/300 Write block 1603 len 32768 data: 2 2 2 2 2 2 2 2
10234 293/300 Read block 123 len 4096
10235 294/300 Read block 1900 len 20480
10236 295/300 Read block 1301 len 8192
10237 296/300 Write block 66 len 20480 data: 3 3 3 3 2
10238 297/300 Write block 1954 len 4096 data: 2
10239 298/300 Write block 3700 len 36864 data: 4 4 5 3 2 2 2 2 2
10240 299/300 Write block 3962 len 24576 data: 5 4 4 4 3 3
10241 300/300 Read block 1547 len 40960
10242 301/300 Write block 3326 len 16384 data: 4 5 5 5
10243 Main task has work to do, go find it
10244 got dsc Start(2)
10245 start 2
10246 Dec 11 23:42:40.728 INFO request completed, latency_us: 218, response_code: 204, uri: /start/cid/2, method: POST, req_id: b219c2b7-a0b3-41b9-a1b6-014600758480, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
10247 Replay: started 2, returned:Ok(())
10248 ----------------------------------------------------------------
10249 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10250 States: Active Active Offline
10251 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10252 3416 Acked Write 1 Done Done Sent false
10253 3417 Acked Read 5 Done Done Sent false
10254 3418 Acked Write 6 Done Done Sent false
10255 3419 Acked Read 10 Done Done Sent false
10256 3420 Acked Read 9 Done Done Sent false
10257 3421 Acked Write 4 Done Done Sent false
10258 3422 Acked Read 7 Done Done Sent false
10259 3423 Acked Flush 0 Done Done Sent false
10260 3424 Acked Read 3 Done Done Sent false
10261 3425 Acked Write 4 Done Done Sent false
10262 3426 Acked Write 6 Done Done Sent false
10263 3427 Acked Read 4 Done Done Sent false
10264 3428 Acked Write 5 Done Done Sent false
10265 3429 Acked Read 6 Done Done Sent false
10266 3430 Acked Read 2 Done Done Sent false
10267 3431 Acked Write 3 Done Done Sent false
10268 3432 Acked Write 8 Done Done Sent false
10269 3433 Acked Write 1 Done Done Sent false
10270 3434 Acked Write 10 Done Done Sent false
10271 3435 Acked Write 7 Done Done Sent false
10272 3436 Acked Read 6 Done Done Sent false
10273 3437 Acked Write 10 Done Done Sent false
10274 3438 Acked Write 1 Done Done Sent false
10275 3439 Acked Write 3 Done Done Sent false
10276 3440 Acked Write 7 Done Done Sent false
10277 3441 Acked Read 10 Done Done Sent false
10278 3442 Acked Write 8 Done Done Sent false
10279 3443 Acked Read 7 Done Done Sent false
10280 3444 Acked Flush 0 Done Done Sent false
10281 3445 Acked Flush 0 Done Done Sent false
10282 3446 Acked Read 2 Done Done Sent false
10283 3447 Acked Write 9 Done Done Sent false
10284 3448 Acked Read 1 Done Done Sent false
10285 3449 Acked Write 6 Done Done Sent false
10286 3450 Acked Write 10 Done Done Sent false
10287 3451 Acked Read 2 Done Done Sent false
10288 3452 Acked Read 3 Done Done Sent false
10289 3453 Acked Read 5 Done Done Sent false
10290 3454 Acked Write 3 Done Done Sent false
10291 3455 Acked Write 8 Done Done Sent false
10292 3456 Acked Read 6 Done Done Sent false
10293 3457 Acked Write 10 Done Done Sent false
10294 3458 Acked Read 4 Done Done Sent false
10295 3459 Acked Write 6 Done Done Sent false
10296 3460 Acked Flush 0 Done Done Sent false
10297 3461 Acked Read 4 Done Done Sent false
10298 3462 Acked Read 6 Done Done Sent false
10299 3463 Acked Read 10 Done Done Sent false
10300 3464 Acked Write 1 Done Done Sent false
10301 3465 Acked Read 8 Done Done Sent false
10302 3466 Acked Write 8 Done Done Sent false
10303 3467 Acked Write 4 Done Done Sent false
10304 3468 Acked Read 7 Done Done Sent false
10305 3469 Acked Write 2 Done Done Sent false
10306 3470 Acked Read 2 Done Done Sent false
10307 3471 Acked Write 1 Done Done Sent false
10308 3472 Acked Write 3 Done Done Sent false
10309 3473 Acked Write 8 Done Done Sent false
10310 3474 Acked Read 2 Done Done Sent false
10311 3475 Acked Read 6 Done Done Sent false
10312 3476 Acked Read 5 Done Done Sent false
10313 3477 Acked Write 10 Done Done Sent false
10314 3478 Acked Read 1 Done Done Sent false
10315 3479 Acked Write 10 Done Done Sent false
10316 3480 Acked Write 5 Done Done Sent false
10317 3481 Acked Read 6 Done Done Sent false
10318 3482 Acked Write 2 Done Done Sent false
10319 3483 Acked Read 2 Done Done Sent false
10320 3484 Acked Read 7 Done Done Sent false
10321 3485 Acked Flush 0 Done Done Sent false
10322 3486 Acked Read 3 Done Done Sent false
10323 3487 Acked Read 4 Done Done Sent false
10324 3488 Acked Write 9 Done Done Sent false
10325 3489 Acked Write 10 Done Done Sent false
10326 3490 Acked Read 6 Done Done Sent false
10327 3491 Acked Write 6 Done Done Sent false
10328 3492 Acked Write 10 Done Done Sent false
10329 3493 Acked Write 7 Done Done Sent false
10330 3494 Acked Read 1 Done Done Sent false
10331 3495 Acked Write 1 Done Done Sent false
10332 3496 Acked Read 5 Done Done Sent false
10333 3497 Acked Write 3 Done Done Sent false
10334 3498 Acked Read 5 Done Done Sent false
10335 3499 Acked Read 9 Done Done Sent false
10336 3500 Acked Flush 0 Done Done Sent false
10337 3501 Acked Write 10 Done Done Sent false
10338 3502 Acked Write 9 Done Done Sent false
10339 3503 Acked Write 5 Done Done Sent false
10340 3504 Acked Read 8 Done Done Sent false
10341 3505 Acked Write 9 Done Done Sent false
10342 3506 Acked Read 7 Done Done Sent false
10343 3507 Acked Write 5 Done Done Sent false
10344 3508 Acked Write 3 Done Done Sent false
10345 3509 Acked Read 9 Done Done Sent false
10346 3510 Acked Flush 0 Done Done Sent false
10347 3511 Acked Read 6 Done Done Sent false
10348 3512 Acked Write 3 Done Done Sent false
10349 3513 Acked Read 8 Done Done Sent false
10350 3514 Acked Read 1 Done Done Sent false
10351 3515 Acked Flush 0 Done Done Sent false
10352 3516 Acked Read 2 Done Done Sent false
10353 3517 Acked Write 8 Done Done Sent false
10354 3518 Acked Write 3 Done Done Sent false
10355 3519 Acked Write 4 Done Done Sent false
10356 3520 Acked Read 8 Done Done Sent false
10357 3521 Acked Write 5 Done Done Sent false
10358 3522 Acked Flush 0 Done Done Sent false
10359 3523 Acked Read 5 Done Done Sent false
10360 3524 Acked Write 7 Done Done Sent false
10361 3525 Acked Write 10 Done Done Sent false
10362 3526 Acked Read 7 Done Done Sent false
10363 3527 Acked Write 4 Done Done Sent false
10364 3528 Acked Write 2 Done Done Sent false
10365 3529 Acked Write 4 Done Done Sent false
10366 3530 Acked Read 3 Done Done Sent false
10367 3531 Acked Read 2 Done Done Sent false
10368 3532 Acked Read 2 Done Done Sent false
10369 3533 Acked Flush 0 Done Done Sent false
10370 3534 Acked Write 5 Done Done Sent false
10371 3535 Acked Write 6 Done Done Sent false
10372 3536 Acked Read 3 Done Done Sent false
10373 3537 Acked Write 3 Done Done Sent false
10374 3538 Acked Read 6 Done Done Sent false
10375 3539 Acked Write 4 Done Done Sent false
10376 3540 Acked Flush 0 Done Done Sent false
10377 3541 Acked Write 4 Done Done Sent false
10378 3542 Acked Write 4 Done Done Sent false
10379 3543 Acked Read 8 Done Done Sent false
10380 3544 Acked Read 2 Done Done Sent false
10381 3545 Acked Write 7 Done Done Sent false
10382 3546 Acked Read 8 Done Done Sent false
10383 3547 Acked Flush 0 Done Done Sent false
10384 3548 Acked Flush 0 Done Done Sent false
10385 3549 Acked Write 9 Done Done Sent false
10386 3550 Acked Write 1 Done Done Sent false
10387 3551 Acked Read 1 Done Done Sent false
10388 3552 Acked Read 7 Done Done Sent false
10389 3553 Acked Flush 0 Done Done Sent false
10390 3554 Acked Write 6 Done Done Sent false
10391 3555 Acked Read 4 Done Done Sent false
10392 3556 Acked Write 8 Done Done Sent false
10393 3557 Acked Write 7 Done Done Sent false
10394 3558 Acked Flush 0 Done Done Sent false
10395 3559 Acked Write 5 Done Done Sent false
10396 3560 Acked Read 3 Done Done Sent false
10397 3561 Acked Write 1 Done Done Sent false
10398 3562 Acked Write 8 Done Done Sent false
10399 3563 Acked Read 7 Done Done Sent false
10400 3564 Acked Read 8 Done Done Sent false
10401 3565 Acked Write 7 Done Done Sent false
10402 3566 Acked Write 7 Done Done Sent false
10403 3567 Acked Flush 0 Done Done Sent false
10404 3568 Acked Flush 0 Done Done Sent false
10405 3569 Acked Write 8 Done Done Sent false
10406 3570 Acked Read 2 Done Done Sent false
10407 3571 Acked Write 8 Done Done Sent false
10408 3572 Acked Read 5 Done Done Sent false
10409 3573 Acked Write 2 Done Done Sent false
10410 3574 Acked Write 5 Done Done Sent false
10411 3575 Acked Read 4 Done Done Sent false
10412 3576 Acked Flush 0 Done Done Sent false
10413 3577 Acked Write 8 Done Done Sent false
10414 3578 Acked Read 7 Done Done Sent false
10415 3579 Acked Write 1 Done Done Sent false
10416 3580 Acked Write 6 Done Done Sent false
10417 3581 Acked Write 4 Done Done Sent false
10418 3582 Acked Write 4 Done Done Sent false
10419 3583 Acked Write 10 Done Done Sent false
10420 3584 Acked Write 9 Done Done Sent false
10421 3585 Acked Read 9 Done Done Sent false
10422 3586 Acked Write 3 Done Done Sent false
10423 3587 Acked Read 1 Done Done Sent false
10424 3588 Acked Flush 0 Done Done Sent false
10425 3589 Acked Read 5 Done Done Sent false
10426 3590 Acked Write 5 Done Done Sent false
10427 3591 Acked Read 7 Done Done Sent false
10428 3592 Acked Read 6 Done Done Sent false
10429 3593 Acked Read 1 Done Done Sent false
10430 3594 Acked Read 9 Done Done Sent false
10431 3595 Acked Read 6 Done Done Sent false
10432 3596 Acked Read 1 Done Done Sent false
10433 3597 Acked Write 8 Done Done Sent false
10434 3598 Acked Read 5 Done Done Sent false
10435 3599 Acked Read 1 Done Done Sent false
10436 3600 Acked Flush 0 Done Done Sent false
10437 3601 Acked Read 6 Done Done Sent false
10438 3602 Acked Read 2 Done Done Sent false
10439 3603 Acked Flush 0 Done Done Sent false
10440 3604 Acked Read 9 Done Done Sent false
10441 3605 Acked Read 5 Done Done Sent false
10442 3606 Acked Write 1 Done Done Sent false
10443 3607 Acked Read 7 Done Done Sent false
10444 3608 Acked Read 3 Done Done Sent false
10445 3609 Acked Read 10 Done Done Sent false
10446 3610 Acked Read 2 Done Done Sent false
10447 3611 Acked Read 4 Done Done Sent false
10448 3612 Acked Flush 0 Done Done Sent false
10449 3613 Acked Read 8 Done Done Sent false
10450 3614 Acked Flush 0 Done Done Sent false
10451 3615 Acked Read 10 Done Done Sent false
10452 3616 Acked Read 2 Done Done Sent false
10453 3617 Acked Write 3 Done Done Sent false
10454 3618 Acked Read 9 Done Done Sent false
10455 3619 Acked Write 1 Done Done Sent false
10456 3620 Acked Write 7 Done Done Sent false
10457 3621 Acked Read 5 Done Done Sent false
10458 3622 Acked Write 6 Done Done Sent false
10459 3623 Acked Read 7 Done Done Sent false
10460 3624 Acked Write 4 Done Done Sent false
10461 3625 Acked Flush 0 Done Done Sent false
10462 3626 Acked Write 5 Done Done Sent false
10463 3627 Acked Write 9 Done Done Sent false
10464 3628 Acked Write 4 Done Done Sent false
10465 3629 Acked Write 5 Done Done Sent false
10466 3630 Acked Read 10 Done Done Sent false
10467 3631 Acked Write 8 Done Done Sent false
10468 3632 Acked Write 7 Done Done Sent false
10469 3633 Acked Read 4 Done Done Sent false
10470 3634 Acked Write 4 Done Done Sent false
10471 3635 Acked Write 4 Done Done Sent false
10472 3636 Acked Read 7 Done Done Sent false
10473 3637 Acked Read 1 Done Done Sent false
10474 3638 Acked Write 5 Done Done Sent false
10475 3639 Acked Write 7 Done Done Sent false
10476 3640 Acked Read 7 Done Done Sent false
10477 3641 Acked Write 2 Done Done Sent false
10478 3642 Acked Read 8 Done Done Sent false
10479 3643 Acked Flush 0 Done Done Sent false
10480 3644 Acked Write 5 Done Done Sent false
10481 3645 Acked Write 7 Done Done Sent false
10482 3646 Acked Read 10 Done Done Sent false
10483 3647 Acked Write 8 Done Done Sent false
10484 3648 Acked Write 8 Done Done Sent false
10485 3649 Acked Read 7 Done Done Sent false
10486 3650 Acked Read 8 Done Done Sent false
10487 3651 Acked Read 1 Done Done Sent false
10488 3652 Acked Write 5 Done Done Sent false
10489 3653 Acked Write 2 Done Done Sent false
10490 3654 Acked Write 5 Done Done Sent false
10491 3655 Acked Read 5 Done Done Sent false
10492 3656 Acked Read 4 Done Done Sent false
10493 3657 Acked Write 6 Done Done Sent false
10494 3658 Acked Write 7 Done Done Sent false
10495 3659 Acked Write 1 Done Done Sent false
10496 3660 Acked Write 8 Done Done Sent false
10497 3661 Acked Read 7 Done Done Sent false
10498 3662 Acked Read 10 Done Done Sent false
10499 3663 Acked Write 8 Done Done Sent false
10500 3664 Acked Write 1 Done Done Sent false
10501 3665 Acked Read 5 Done Done Sent false
10502 3666 Acked Write 3 Done Done Sent false
10503 3667 Acked Write 9 Done Done Sent false
10504 3668 Acked Write 10 Done Done Sent false
10505 3669 Acked Write 5 Done Done Sent false
10506 3670 Acked Write 9 Done Done Sent false
10507 3671 Acked Write 7 Done Done Sent false
10508 3672 Acked Read 8 Done Done Sent false
10509 3673 Acked Read 4 Done Done Sent false
10510 3674 Acked Write 9 Done Done Sent false
10511 3675 Acked Write 8 Done Done Sent false
10512 3676 Acked Read 2 Done Done Sent false
10513 3677 Acked Read 6 Done Done Sent false
10514 3678 Acked Read 2 Done Done Sent false
10515 3679 Acked Read 5 Done Done Sent false
10516 3680 Acked Write 3 Done Done Sent false
10517 3681 Acked Write 5 Done Done Sent false
10518 3682 Acked Read 4 Done Done Sent false
10519 3683 Acked Write 5 Done Done Sent false
10520 3684 Acked Read 9 Done Done Sent false
10521 3685 Acked Read 2 Done Done Sent false
10522 3686 Acked Flush 0 Done Done Sent false
10523 3687 Acked Write 4 Done Done Sent false
10524 3688 Acked Write 8 Done Done Sent false
10525 3689 Acked Write 9 Done Done Sent false
10526 3690 Acked Flush 0 Done Done Sent false
10527 3691 Acked Read 4 Done Done Sent false
10528 3692 Acked Read 8 Done Done Sent false
10529 3693 Acked Write 6 Done Done Sent false
10530 3694 Acked Read 1 Done Done Sent false
10531 3695 Acked Read 5 Done Done Sent false
10532 3696 Acked Read 10 Done Done Sent false
10533 3697 Acked Read 6 Done Done Sent false
10534 3698 Acked Write 4 Done Done Sent false
10535 3699 Acked Read 8 Done Done Sent false
10536 3700 Acked Write 6 Done Done Sent false
10537 3701 Acked Read 5 Done Done Sent false
10538 3702 Acked Read 1 Done Done Sent false
10539 3703 Acked Write 6 Done Done Sent false
10540 3704 Acked Read 10 Done Done Sent false
10541 3705 Acked Read 4 Done Done Sent false
10542 3706 Acked Write 7 Done Done Sent false
10543 3707 Acked Write 8 Done Done Sent false
10544 3708 Acked Read 1 Done Done Sent false
10545 3709 Acked Read 5 Done Done Sent false
10546 3710 Acked Read 2 Done Done Sent false
10547 3711 Acked Write 5 Done Done Sent false
10548 3712 Acked Write 1 Done Done Sent false
10549 3713 Acked Write 9 Done Done Sent false
10550 3714 Acked Write 6 Done Done Sent false
10551 3715 Acked Read 10 Done Done Sent false
10552 3716 Acked Write 4 Done Done Sent false
10553 STATES DS:0 DS:1 DS:2 TOTAL
10554 Sent 0 0 301 301
10555 Done 301 301 0 602
10556 Skipped 0 0 0 0
10557 Error 0 0 0 0
10558 Last Flush: 3690 3690 3415
10559 Downstairs last five completed: 3415 3414 3413 3412 3411
10560 Upstairs last five completed: 3716 3715 3714 3713 3712
10561 CLIENT: Up:0 ds:301 act:2
10562 [8830] got start action
10563 [8830] I am going to restart
10564 Starting downstairs at port 8830
10565 Make output file at "/tmp/dsc/downstairs-8830.txt"
10566 [8830][2] reports Starting
10567 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1209)
10568 [8830][2] reports Running
10569 ----------------------------------------------------------------
10570 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10571 States: Active Active Offline
10572 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10573 3416 Acked Write 1 Done Done Sent false
10574 3417 Acked Read 5 Done Done Sent false
10575 3418 Acked Write 6 Done Done Sent false
10576 3419 Acked Read 10 Done Done Sent false
10577 3420 Acked Read 9 Done Done Sent false
10578 3421 Acked Write 4 Done Done Sent false
10579 3422 Acked Read 7 Done Done Sent false
10580 3423 Acked Flush 0 Done Done Sent false
10581 3424 Acked Read 3 Done Done Sent false
10582 3425 Acked Write 4 Done Done Sent false
10583 3426 Acked Write 6 Done Done Sent false
10584 3427 Acked Read 4 Done Done Sent false
10585 3428 Acked Write 5 Done Done Sent false
10586 3429 Acked Read 6 Done Done Sent false
10587 3430 Acked Read 2 Done Done Sent false
10588 3431 Acked Write 3 Done Done Sent false
10589 3432 Acked Write 8 Done Done Sent false
10590 3433 Acked Write 1 Done Done Sent false
10591 3434 Acked Write 10 Done Done Sent false
10592 3435 Acked Write 7 Done Done Sent false
10593 3436 Acked Read 6 Done Done Sent false
10594 3437 Acked Write 10 Done Done Sent false
10595 3438 Acked Write 1 Done Done Sent false
10596 3439 Acked Write 3 Done Done Sent false
10597 3440 Acked Write 7 Done Done Sent false
10598 3441 Acked Read 10 Done Done Sent false
10599 3442 Acked Write 8 Done Done Sent false
10600 3443 Acked Read 7 Done Done Sent false
10601 3444 Acked Flush 0 Done Done Sent false
10602 3445 Acked Flush 0 Done Done Sent false
10603 3446 Acked Read 2 Done Done Sent false
10604 3447 Acked Write 9 Done Done Sent false
10605 3448 Acked Read 1 Done Done Sent false
10606 3449 Acked Write 6 Done Done Sent false
10607 3450 Acked Write 10 Done Done Sent false
10608 3451 Acked Read 2 Done Done Sent false
10609 3452 Acked Read 3 Done Done Sent false
10610 3453 Acked Read 5 Done Done Sent false
10611 3454 Acked Write 3 Done Done Sent false
10612 3455 Acked Write 8 Done Done Sent false
10613 3456 Acked Read 6 Done Done Sent false
10614 3457 Acked Write 10 Done Done Sent false
10615 3458 Acked Read 4 Done Done Sent false
10616 3459 Acked Write 6 Done Done Sent false
10617 3460 Acked Flush 0 Done Done Sent false
10618 3461 Acked Read 4 Done Done Sent false
10619 3462 Acked Read 6 Done Done Sent false
10620 3463 Acked Read 10 Done Done Sent false
10621 3464 Acked Write 1 Done Done Sent false
10622 3465 Acked Read 8 Done Done Sent false
10623 3466 Acked Write 8 Done Done Sent false
10624 3467 Acked Write 4 Done Done Sent false
10625 3468 Acked Read 7 Done Done Sent false
10626 3469 Acked Write 2 Done Done Sent false
10627 3470 Acked Read 2 Done Done Sent false
10628 3471 Acked Write 1 Done Done Sent false
10629 3472 Acked Write 3 Done Done Sent false
10630 3473 Acked Write 8 Done Done Sent false
10631 3474 Acked Read 2 Done Done Sent false
10632 3475 Acked Read 6 Done Done Sent false
10633 3476 Acked Read 5 Done Done Sent false
10634 3477 Acked Write 10 Done Done Sent false
10635 3478 Acked Read 1 Done Done Sent false
10636 3479 Acked Write 10 Done Done Sent false
10637 3480 Acked Write 5 Done Done Sent false
10638 3481 Acked Read 6 Done Done Sent false
10639 3482 Acked Write 2 Done Done Sent false
10640 3483 Acked Read 2 Done Done Sent false
10641 3484 Acked Read 7 Done Done Sent false
10642 3485 Acked Flush 0 Done Done Sent false
10643 3486 Acked Read 3 Done Done Sent false
10644 3487 Acked Read 4 Done Done Sent false
10645 3488 Acked Write 9 Done Done Sent false
10646 3489 Acked Write 10 Done Done Sent false
10647 3490 Acked Read 6 Done Done Sent false
10648 3491 Acked Write 6 Done Done Sent false
10649 3492 Acked Write 10 Done Done Sent false
10650 3493 Acked Write 7 Done Done Sent false
10651 3494 Acked Read 1 Done Done Sent false
10652 3495 Acked Write 1 Done Done Sent false
10653 3496 Acked Read 5 Done Done Sent false
10654 3497 Acked Write 3 Done Done Sent false
10655 3498 Acked Read 5 Done Done Sent false
10656 3499 Acked Read 9 Done Done Sent false
10657 3500 Acked Flush 0 Done Done Sent false
10658 3501 Acked Write 10 Done Done Sent false
10659 3502 Acked Write 9 Done Done Sent false
10660 3503 Acked Write 5 Done Done Sent false
10661 3504 Acked Read 8 Done Done Sent false
10662 3505 Acked Write 9 Done Done Sent false
10663 3506 Acked Read 7 Done Done Sent false
10664 3507 Acked Write 5 Done Done Sent false
10665 3508 Acked Write 3 Done Done Sent false
10666 3509 Acked Read 9 Done Done Sent false
10667 3510 Acked Flush 0 Done Done Sent false
10668 3511 Acked Read 6 Done Done Sent false
10669 3512 Acked Write 3 Done Done Sent false
10670 3513 Acked Read 8 Done Done Sent false
10671 3514 Acked Read 1 Done Done Sent false
10672 3515 Acked Flush 0 Done Done Sent false
10673 3516 Acked Read 2 Done Done Sent false
10674 3517 Acked Write 8 Done Done Sent false
10675 3518 Acked Write 3 Done Done Sent false
10676 3519 Acked Write 4 Done Done Sent false
10677 3520 Acked Read 8 Done Done Sent false
10678 3521 Acked Write 5 Done Done Sent false
10679 3522 Acked Flush 0 Done Done Sent false
10680 3523 Acked Read 5 Done Done Sent false
10681 3524 Acked Write 7 Done Done Sent false
10682 3525 Acked Write 10 Done Done Sent false
10683 3526 Acked Read 7 Done Done Sent false
10684 3527 Acked Write 4 Done Done Sent false
10685 3528 Acked Write 2 Done Done Sent false
10686 3529 Acked Write 4 Done Done Sent false
10687 3530 Acked Read 3 Done Done Sent false
10688 3531 Acked Read 2 Done Done Sent false
10689 3532 Acked Read 2 Done Done Sent false
10690 3533 Acked Flush 0 Done Done Sent false
10691 3534 Acked Write 5 Done Done Sent false
10692 3535 Acked Write 6 Done Done Sent false
10693 3536 Acked Read 3 Done Done Sent false
10694 3537 Acked Write 3 Done Done Sent false
10695 3538 Acked Read 6 Done Done Sent false
10696 3539 Acked Write 4 Done Done Sent false
10697 3540 Acked Flush 0 Done Done Sent false
10698 3541 Acked Write 4 Done Done Sent false
10699 3542 Acked Write 4 Done Done Sent false
10700 3543 Acked Read 8 Done Done Sent false
10701 3544 Acked Read 2 Done Done Sent false
10702 3545 Acked Write 7 Done Done Sent false
10703 3546 Acked Read 8 Done Done Sent false
10704 3547 Acked Flush 0 Done Done Sent false
10705 3548 Acked Flush 0 Done Done Sent false
10706 3549 Acked Write 9 Done Done Sent false
10707 3550 Acked Write 1 Done Done Sent false
10708 3551 Acked Read 1 Done Done Sent false
10709 3552 Acked Read 7 Done Done Sent false
10710 3553 Acked Flush 0 Done Done Sent false
10711 3554 Acked Write 6 Done Done Sent false
10712 3555 Acked Read 4 Done Done Sent false
10713 3556 Acked Write 8 Done Done Sent false
10714 3557 Acked Write 7 Done Done Sent false
10715 3558 Acked Flush 0 Done Done Sent false
10716 3559 Acked Write 5 Done Done Sent false
10717 3560 Acked Read 3 Done Done Sent false
10718 3561 Acked Write 1 Done Done Sent false
10719 3562 Acked Write 8 Done Done Sent false
10720 3563 Acked Read 7 Done Done Sent false
10721 3564 Acked Read 8 Done Done Sent false
10722 3565 Acked Write 7 Done Done Sent false
10723 3566 Acked Write 7 Done Done Sent false
10724 3567 Acked Flush 0 Done Done Sent false
10725 3568 Acked Flush 0 Done Done Sent false
10726 3569 Acked Write 8 Done Done Sent false
10727 3570 Acked Read 2 Done Done Sent false
10728 3571 Acked Write 8 Done Done Sent false
10729 3572 Acked Read 5 Done Done Sent false
10730 3573 Acked Write 2 Done Done Sent false
10731 3574 Acked Write 5 Done Done Sent false
10732 3575 Acked Read 4 Done Done Sent false
10733 3576 Acked Flush 0 Done Done Sent false
10734 3577 Acked Write 8 Done Done Sent false
10735 3578 Acked Read 7 Done Done Sent false
10736 3579 Acked Write 1 Done Done Sent false
10737 3580 Acked Write 6 Done Done Sent false
10738 3581 Acked Write 4 Done Done Sent false
10739 3582 Acked Write 4 Done Done Sent false
10740 3583 Acked Write 10 Done Done Sent false
10741 3584 Acked Write 9 Done Done Sent false
10742 3585 Acked Read 9 Done Done Sent false
10743 3586 Acked Write 3 Done Done Sent false
10744 3587 Acked Read 1 Done Done Sent false
10745 3588 Acked Flush 0 Done Done Sent false
10746 3589 Acked Read 5 Done Done Sent false
10747 3590 Acked Write 5 Done Done Sent false
10748 3591 Acked Read 7 Done Done Sent false
10749 3592 Acked Read 6 Done Done Sent false
10750 3593 Acked Read 1 Done Done Sent false
10751 3594 Acked Read 9 Done Done Sent false
10752 3595 Acked Read 6 Done Done Sent false
10753 3596 Acked Read 1 Done Done Sent false
10754 3597 Acked Write 8 Done Done Sent false
10755 3598 Acked Read 5 Done Done Sent false
10756 3599 Acked Read 1 Done Done Sent false
10757 3600 Acked Flush 0 Done Done Sent false
10758 3601 Acked Read 6 Done Done Sent false
10759 3602 Acked Read 2 Done Done Sent false
10760 3603 Acked Flush 0 Done Done Sent false
10761 3604 Acked Read 9 Done Done Sent false
10762 3605 Acked Read 5 Done Done Sent false
10763 3606 Acked Write 1 Done Done Sent false
10764 3607 Acked Read 7 Done Done Sent false
10765 3608 Acked Read 3 Done Done Sent false
10766 3609 Acked Read 10 Done Done Sent false
10767 3610 Acked Read 2 Done Done Sent false
10768 3611 Acked Read 4 Done Done Sent false
10769 3612 Acked Flush 0 Done Done Sent false
10770 3613 Acked Read 8 Done Done Sent false
10771 3614 Acked Flush 0 Done Done Sent false
10772 3615 Acked Read 10 Done Done Sent false
10773 3616 Acked Read 2 Done Done Sent false
10774 3617 Acked Write 3 Done Done Sent false
10775 3618 Acked Read 9 Done Done Sent false
10776 3619 Acked Write 1 Done Done Sent false
10777 3620 Acked Write 7 Done Done Sent false
10778 3621 Acked Read 5 Done Done Sent false
10779 3622 Acked Write 6 Done Done Sent false
10780 3623 Acked Read 7 Done Done Sent false
10781 3624 Acked Write 4 Done Done Sent false
10782 3625 Acked Flush 0 Done Done Sent false
10783 3626 Acked Write 5 Done Done Sent false
10784 3627 Acked Write 9 Done Done Sent false
10785 3628 Acked Write 4 Done Done Sent false
10786 3629 Acked Write 5 Done Done Sent false
10787 3630 Acked Read 10 Done Done Sent false
10788 3631 Acked Write 8 Done Done Sent false
10789 3632 Acked Write 7 Done Done Sent false
10790 3633 Acked Read 4 Done Done Sent false
10791 3634 Acked Write 4 Done Done Sent false
10792 3635 Acked Write 4 Done Done Sent false
10793 3636 Acked Read 7 Done Done Sent false
10794 3637 Acked Read 1 Done Done Sent false
10795 3638 Acked Write 5 Done Done Sent false
10796 3639 Acked Write 7 Done Done Sent false
10797 3640 Acked Read 7 Done Done Sent false
10798 3641 Acked Write 2 Done Done Sent false
10799 3642 Acked Read 8 Done Done Sent false
10800 3643 Acked Flush 0 Done Done Sent false
10801 3644 Acked Write 5 Done Done Sent false
10802 3645 Acked Write 7 Done Done Sent false
10803 3646 Acked Read 10 Done Done Sent false
10804 3647 Acked Write 8 Done Done Sent false
10805 3648 Acked Write 8 Done Done Sent false
10806 3649 Acked Read 7 Done Done Sent false
10807 3650 Acked Read 8 Done Done Sent false
10808 3651 Acked Read 1 Done Done Sent false
10809 3652 Acked Write 5 Done Done Sent false
10810 3653 Acked Write 2 Done Done Sent false
10811 3654 Acked Write 5 Done Done Sent false
10812 3655 Acked Read 5 Done Done Sent false
10813 3656 Acked Read 4 Done Done Sent false
10814 3657 Acked Write 6 Done Done Sent false
10815 3658 Acked Write 7 Done Done Sent false
10816 3659 Acked Write 1 Done Done Sent false
10817 3660 Acked Write 8 Done Done Sent false
10818 3661 Acked Read 7 Done Done Sent false
10819 3662 Acked Read 10 Done Done Sent false
10820 3663 Acked Write 8 Done Done Sent false
10821 3664 Acked Write 1 Done Done Sent false
10822 3665 Acked Read 5 Done Done Sent false
10823 3666 Acked Write 3 Done Done Sent false
10824 3667 Acked Write 9 Done Done Sent false
10825 3668 Acked Write 10 Done Done Sent false
10826 3669 Acked Write 5 Done Done Sent false
10827 3670 Acked Write 9 Done Done Sent false
10828 3671 Acked Write 7 Done Done Sent false
10829 3672 Acked Read 8 Done Done Sent false
10830 3673 Acked Read 4 Done Done Sent false
10831 3674 Acked Write 9 Done Done Sent false
10832 3675 Acked Write 8 Done Done Sent false
10833 3676 Acked Read 2 Done Done Sent false
10834 3677 Acked Read 6 Done Done Sent false
10835 3678 Acked Read 2 Done Done Sent false
10836 3679 Acked Read 5 Done Done Sent false
10837 3680 Acked Write 3 Done Done Sent false
10838 3681 Acked Write 5 Done Done Sent false
10839 3682 Acked Read 4 Done Done Sent false
10840 3683 Acked Write 5 Done Done Sent false
10841 3684 Acked Read 9 Done Done Sent false
10842 3685 Acked Read 2 Done Done Sent false
10843 3686 Acked Flush 0 Done Done Sent false
10844 3687 Acked Write 4 Done Done Sent false
10845 3688 Acked Write 8 Done Done Sent false
10846 3689 Acked Write 9 Done Done Sent false
10847 3690 Acked Flush 0 Done Done Sent false
10848 3691 Acked Read 4 Done Done Sent false
10849 3692 Acked Read 8 Done Done Sent false
10850 3693 Acked Write 6 Done Done Sent false
10851 3694 Acked Read 1 Done Done Sent false
10852 3695 Acked Read 5 Done Done Sent false
10853 3696 Acked Read 10 Done Done Sent false
10854 3697 Acked Read 6 Done Done Sent false
10855 3698 Acked Write 4 Done Done Sent false
10856 3699 Acked Read 8 Done Done Sent false
10857 3700 Acked Write 6 Done Done Sent false
10858 3701 Acked Read 5 Done Done Sent false
10859 3702 Acked Read 1 Done Done Sent false
10860 3703 Acked Write 6 Done Done Sent false
10861 3704 Acked Read 10 Done Done Sent false
10862 3705 Acked Read 4 Done Done Sent false
10863 3706 Acked Write 7 Done Done Sent false
10864 3707 Acked Write 8 Done Done Sent false
10865 3708 Acked Read 1 Done Done Sent false
10866 3709 Acked Read 5 Done Done Sent false
10867 3710 Acked Read 2 Done Done Sent false
10868 3711 Acked Write 5 Done Done Sent false
10869 3712 Acked Write 1 Done Done Sent false
10870 3713 Acked Write 9 Done Done Sent false
10871 3714 Acked Write 6 Done Done Sent false
10872 3715 Acked Read 10 Done Done Sent false
10873 3716 Acked Write 4 Done Done Sent false
10874 3717 Acked Flush 0 Done Done Sent false
10875 STATES DS:0 DS:1 DS:2 TOTAL
10876 Sent 0 0 302 302
10877 Done 302 302 0 604
10878 Skipped 0 0 0 0
10879 Error 0 0 0 0
10880 Last Flush: 3717 3717 3415
10881 Downstairs last five completed: 3415 3414 3413 3412 3411
10882 Upstairs last five completed: 3717 3716 3715 3714 3713
10883 CLIENT: Up:0 ds:302 act:2
108842024-12-11T23:42:46.503ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108852024-12-11T23:42:46.503ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59724) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108862024-12-11T23:42:46.504ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108872024-12-11T23:42:46.505ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108882024-12-11T23:42:46.505ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108892024-12-11T23:42:46.505ZINFOcrucible: send last flush ID to this DS: 3415 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108902024-12-11T23:42:46.505ZINFOcrucible: Replied this last flush ID: 3415 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108912024-12-11T23:42:46.505ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108922024-12-11T23:42:46.505ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108932024-12-11T23:42:46.505ZINFOcrucible: [2] client re-new 302 jobs since flush 3415 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
108942024-12-11T23:42:46.506ZINFOcrucible: [2] Replayed 302 jobs since flush: 3415 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
10895 ----------------------------------------------------------------
10896 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10897 Downstairs last five completed: 3717 3716 3715 3714 3713
10898 Upstairs last five completed: 3717 3716 3715 3714 3713
10899 CLIENT: Up:0 ds:0 act:3
10900 Replay: All jobs finished
10901 Main task has work to do, go find it
10902 got dsc Stop(1)
10903 stop 1
10904 Dec 11 23:42:48.734 INFO request completed, latency_us: 350, response_code: 204, uri: [8820] Got stop action so:false kr:false
10905 /stop/cid/1, method: POST, req_id: 6ac44028-27ca-4e2f-a21a-87bcc80beb92, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
109062024-12-11T23:42:48.734ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
109072024-12-11T23:42:48.734ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
109082024-12-11T23:42:48.734ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
109092024-12-11T23:42:48.734ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
109102024-12-11T23:42:48.735ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
10911 Dec 11 23:42:48.735 INFO request completed, latency_us: 169, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7969d064-02ce-4c41-b176-1d3c650bfebc, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
109122024-12-11T23:42:48.735ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
10913 [8820] Exited with: ExitStatus(unix_wait_status(9))
10914 [8820][1] reports Exit
10915 Dec 11 23:42:52.737 INFO request completed, latency_us: 229, response_code: 200, uri: /state/cid/1, method: GET, req_id: c1216984-1be5-4ee0-94bf-3fa97c9ecc9d, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
10916 001/300 Read block 4136 len 32768
10917 002/300 Write block 237 len 8192 data: 2 3
10918 003/300 Write block 205 len 8192 data: 3 3
10919 004/300 Read block 4773 len 28672
10920 005/300 Read block 1143 len 16384
10921 006/300 Read block 4037 len 12288
10922 007/300 Read block 1401 len 20480
10923 008/300 Write block 1305 len 32768 data: 3 3 3 2 2 3 5 5
10924 009/300 Read block 3520 len 28672
10925 010/300 Read block 2889 len 16384
10926 011/300 Write block 4898 len 40960 data: 2 3 3 3 4 4 5 6 4 4
10927 012/300 Write block 3362 len 32768 data: 3 3 4 4 3 3 3 2
10928 013/300 Write block 4015 len 12288 data: 3 3 4
10929 014/300 Read block 1828 len 40960
10930 015/300 Write block 3253 len 8192 data: 2 2
10931 016/300 Read block 2005 len 4096
10932 017/300 Write block 340 len 32768 data: 3 3 3 3 4 4 4 3
10933 018/300 Read block 2680 len 36864
10934 019/300 Read block 4964 len 40960
10935 020/300 Read block 463 len 4096
10936 021/300 Read block 2219 len 36864
10937 022/300 Write block 3345 len 4096 data: 3
10938 023/300 Read block 2393 len 12288
10939 024/300 Flush
10940 025/300 Read block 4203 len 8192
10941 026/300 Flush
10942 027/300 Read block 4741 len 36864
10943 028/300 Read block 2805 len 4096
10944 029/300 Write block 771 len 4096 data: 5
10945 030/300 Read block 4358 len 16384
10946 031/300 Read block 1320 len 32768
10947 032/300 Write block 3209 len 16384 data: 3 3 4 2
10948 033/300 Read block 1694 len 20480
10949 034/300 Write block 4272 len 12288 data: 2 2 2
10950 035/300 Write block 2970 len 12288 data: 2 2 2
10951 036/300 Write block 3311 len 16384 data: 5 4 4 5
10952 037/300 Read block 2482 len 36864
10953 038/300 Read block 4547 len 8192
10954 039/300 Write block 1347 len 32768 data: 3 2 4 4 5 5 5 5
10955 040/300 Read block 4242 len 12288
10956 041/300 Read block 3037 len 40960
10957 042/300 Read block 1949 len 28672
10958 043/300 Read block 2241 len 8192
10959 044/300 Read block 3378 len 8192
10960 045/300 Flush
10961 046/300 Read block 572 len 36864
10962 047/300 Write block 2326 len 36864 data: 3 3 3 3 3 3 4 4 6
10963 048/300 Flush
10964 049/300 Read block 4485 len 28672
10965 050/300 Write block 3111 len 28672 data: 2 2 2 2 2 2 2
10966 051/300 Write block 1283 len 36864 data: 2 2 2 2 2 2 2 2 2
10967 052/300 Write block 3179 len 12288 data: 2 2 2
10968 053/300 Flush
10969 054/300 Read block 165 len 12288
10970 055/300 Write block 3575 len 28672 data: 4 4 3 3 3 3 4
10971 056/300 Read block 1894 len 28672
10972 057/300 Read block 4374 len 16384
10973 058/300 Read block 3891 len 12288
10974 059/300 Write block 3806 len 28672 data: 2 2 2 2 2 2 3
10975 060/300 Write block 1173 len 4096 data: 2
10976 061/300 Write block 396 len 12288 data: 3 2 3
10977 062/300 Flush
10978 063/300 Write block 687 len 16384 data: 2 2 2 2
10979 064/300 Flush
10980 065/300 Write block 3034 len 12288 data: 3 3 3
10981 066/300 Write block 2029 len 32768 data: 3 4 3 3 3 2 2 2
10982 067/300 Write block 4362 len 20480 data: 2 2 2 2 2
10983 068/300 Read block 744 len 16384
10984 069/300 Read block 3198 len 32768
10985 070/300 Read block 3479 len 4096
10986 071/300 Flush
10987 072/300 Write block 4259 len 32768 data: 2 2 4 4 4 4 3 3
10988 073/300 Flush
10989 074/300 Write block 660 len 8192 data: 4 5
10990 075/300 Write block 4955 len 16384 data: 2 3 3 4
10991 076/300 Write block 1256 len 4096 data: 3
10992 077/300 Write block 2891 len 20480 data: 2 2 2 3 3
10993 078/300 Read block 1986 len 12288
10994 079/300 Read block 1535 len 12288
10995 080/300 Read block 1868 len 4096
10996 081/300 Write block 1167 len 24576 data: 4 3 3 2 2 2
10997 082/300 Write block 3556 len 12288 data: 3 3 2
10998 083/300 Read block 409 len 28672
10999 084/300 Read block 4969 len 12288
11000 085/300 Write block 174 len 8192 data: 3 3
11001 086/300 Read block 4140 len 28672
11002 087/300 Write block 4832 len 20480 data: 2 2 2 2 2
11003 088/300 Write block 568 len 20480 data: 2 2 2 3 3
11004 089/300 Read block 2472 len 16384
11005 090/300 Write block 4990 len 32768 data: 2 2 2 3 4 3 3 3
11006 091/300 Read block 3124 len 16384
11007 092/300 Read block 4217 len 28672
11008 093/300 Read block 1489 len 4096
11009 094/300 Read block 3852 len 24576
11010 095/300 Write block 2848 len 16384 data: 2 2 2 2
11011 096/300 Read block 2746 len 20480
11012 097/300 Read block 3316 len 32768
11013 098/300 Read block 1660 len 8192
11014 099/300 Read block 1455 len 32768
11015 100/300 Write block 4902 len 4096 data: 5
11016 101/300 Flush
11017 102/300 Write block 3675 len 28672 data: 4 3 2 2 2 2 2
11018 103/300 Read block 3113 len 20480
11019 104/300 Read block 4069 len 8192
11020 105/300 Write block 218 len 32768 data: 2 2 2 2 2 3 3 3
11021 106/300 Flush
11022 107/300 Write block 4398 len 16384 data: 2 3 3 3
11023 108/300 Read block 4258 len 24576
11024 109/300 Read block 1113 len 16384
11025 110/300 Read block 1097 len 8192
11026 111/300 Write block 2804 len 8192 data: 5 3
11027 112/300 Read block 3956 len 4096
11028 113/300 Write block 3155 len 4096 data: 5
11029 114/300 Flush
11030 115/300 Read block 4041 len 12288
11031 116/300 Flush
11032 117/300 Write block 317 len 32768 data: 3 3 3 3 3 4 5 5
11033 118/300 Read block 3844 len 20480
11034 119/300 Read block 3616 len 24576
11035 120/300 Write block 211 len 8192 data: 3 3
11036 121/300 Write block 3991 len 8192 data: 5 6
11037 122/300 Read block 1263 len 12288
11038 123/300 Write block 3897 len 20480 data: 4 4 4 4 4
11039 124/300 Read block 4966 len 40960
11040 125/300 Write block 4409 len 20480 data: 2 2 2 2 2
11041 126/300 Read block 2507 len 4096
11042 127/300 Flush
11043 128/300 Read block 932 len 4096
11044 129/300 Read block 857 len 20480
11045 130/300 Write block 3972 len 20480 data: 2 2 2 3 3
11046 131/300 Write block 3472 len 36864 data: 3 2 2 2 2 2 3 3 3
11047 132/300 Write block 1153 len 32768 data: 3 2 2 2 2 2 2 2
11048 133/300 Write block 2888 len 32768 data: 4 4 3 3 3 3 4 4
11049 134/300 Flush
11050 135/300 Read block 3440 len 8192
11051 136/300 Flush
11052 137/300 Read block 623 len 12288
11053 138/300 Read block 4077 len 36864
11054 139/300 Read block 2893 len 4096
11055 140/300 Flush
11056 141/300 Flush
11057 142/300 Read block 3578 len 20480
11058 143/300 Read block 3050 len 28672
11059 144/300 Write block 1393 len 28672 data: 2 2 2 2 2 2 2
11060 145/300 Read block 3362 len 40960
11061 146/300 Write block 1286 len 36864 data: 3 3 3 3 3 3 2 2 3
11062 147/300 Flush
11063 148/300 Write block 4010 len 16384 data: 2 2 2 3
11064 149/300 Write block 3975 len 12288 data: 4 4 3
11065 150/300 Read block 4889 len 28672
11066 151/300 Read block 4232 len 40960
11067 152/300 Read block 1430 len 32768
11068 153/300 Read block 4364 len 36864
11069 154/300 Read block 4828 len 32768
11070 155/300 Write block 1214 len 24576 data: 3 3 3 3 3 3
11071 156/300 Read block 1985 len 8192
11072 157/300 Flush
11073 158/300 Write block 1854 len 24576 data: 4 4 4 4 3 3
11074 159/300 Read block 1869 len 32768
11075 160/300 Write block 805 len 28672 data: 3 3 3 4 3 3 3
11076 161/300 Write block 2116 len 20480 data: 7 5 4 4 3
11077 162/300 Read block 2913 len 32768
11078 163/300 Read block 3656 len 8192
11079 164/300 Write block 4163 len 40960 data: 3 3 3 4 4 4 4 5 5 5
11080 165/300 Read block 4898 len 36864
11081 166/300 Write block 4444 len 4096 data: 3
11082 167/300 Read block 925 len 40960
11083 168/300 Read block 3918 len 12288
11084 169/300 Read block 3434 len 12288
11085 170/300 Read block 2820 len 8192
11086 171/300 Flush
11087 172/300 Write block 1986 len 4096 data: 6
11088 173/300 Flush
11089 174/300 Write block 265 len 28672 data: 3 3 3 3 3 3 3
11090 175/300 Read block 2598 len 32768
11091 176/300 Read block 2379 len 4096
11092 177/300 Write block 854 len 8192 data: 4 5
11093 178/300 Read block 1593 len 8192
11094 179/300 Read block 1406 len 4096
11095 180/300 Write block 2890 len 36864 data: 4 4 4 4 5 5 4 5 7
11096 181/300 Read block 1133 len 28672
11097 182/300 Read block 1149 len 24576
11098 183/300 Read block 3691 len 16384
11099 184/300 Flush
11100 185/300 Write block 2053 len 4096 data: 2
11101 186/300 Flush
11102 187/300 Write block 3885 len 12288 data: 3 5 5
11103 188/300 Read block 1221 len 20480
11104 189/300 Flush
11105 190/300 Read block 4555 len 40960
11106 191/300 Read block 4238 len 12288
11107 192/300 Read block 3202 len 16384
11108 193/300 Read block 561 len 16384
11109 194/300 Write block 2522 len 36864 data: 2 2 2 3 2 2 2 2 2
11110 195/300 Write block 968 len 28672 data: 3 3 3 2 2 3 3
11111 196/300 Read block 3680 len 8192
11112 197/300 Read block 2819 len 4096
11113 198/300 Write block 4507 len 20480 data: 2 2 2 2 2
11114 199/300 Read block 1291 len 40960
11115 200/300 Flush
11116 201/300 Write block 4536 len 40960 data: 3 3 3 3 3 3 3 3 2 2
11117 202/300 Read block 3156 len 16384
11118 203/300 Write block 3959 len 36864 data: 3 5 5 6 5 5 5 4 4
11119 204/300 Flush
11120 205/300 Read block 4243 len 16384
11121 206/300 Read block 2556 len 8192
11122 207/300 Write block 3460 len 16384 data: 2 2 2 2
11123 208/300 Flush
11124 209/300 Write block 774 len 16384 data: 2 2 2 2
11125 210/300 Write block 343 len 20480 data: 4 5 5 5 4
11126 211/300 Write block 2692 len 36864 data: 4 4 3 4 3 3 3 3 3
11127 212/300 Read block 4614 len 4096
11128 213/300 Read block 3323 len 36864
11129 214/300 Read block 3505 len 16384
11130 215/300 Read block 3006 len 20480
11131 216/300 Write block 1269 len 40960 data: 2 4 4 3 2 2 4 4 5 5
11132 217/300 Read block 2131 len 28672
11133 218/300 Read block 3273 len 40960
11134 219/300 Read block 2042 len 32768
11135 220/300 Read block 3499 len 20480
11136 221/300 Read block 3809 len 24576
11137 222/300 Write block 1462 len 4096 data: 3
11138 223/300 Read block 3776 len 24576
11139 224/300 Write block 1046 len 32768 data: 3 3 2 2 2 2 2 3
11140 225/300 Write block 2577 len 16384 data: 7 7 5 4
11141 226/300 Write block 244 len 4096 data: 2
11142 227/300 Read block 475 len 36864
11143 228/300 Write block 3382 len 32768 data: 2 4 4 5 5 5 4 3
11144 229/300 Write block 69 len 20480 data: 4 3 3 3 3
11145 230/300 Read block 1539 len 24576
11146 231/300 Write block 2375 len 32768 data: 2 2 2 2 4 3 3 3
11147 232/300 Write block 3648 len 20480 data: 4 3 3 3 3
11148 233/300 Read block 4066 len 24576
11149 234/300 Read block 2849 len 12288
11150 235/300 Write block 517 len 16384 data: 4 4 5 5
11151 236/300 Read block 284 len 16384
11152 237/300 Read block 3618 len 36864
11153 238/300 Write block 1947 len 36864 data: 3 4 4 3 3 3 2 3 2
11154 239/300 Write block 4134 len 4096 data: 2
11155 240/300 Flush
11156 241/300 Flush
11157 242/300 Write block 449 len 24576 data: 3 3 3 3 3 2
11158 243/300 Write block 2579 len 12288 data: 6 5 5
11159 244/300 Flush
11160 245/300 Write block 3977 len 12288 data: 4 3 4
11161 246/300 Read block 915 len 20480
11162 247/300 Write block 1290 len 36864 data: 4 4 3 3 4 3 3 3 3
11163 248/300 Read block 2022 len 40960
11164 249/300 Read block 4345 len 4096
11165 250/300 Write block 3129 len 36864 data: 4 5 4 5 5 4 5 6 6
11166 251/300 Read block 4576 len 12288
11167 252/300 Write block 4492 len 40960 data: 2 3 3 4 4 5 5 3 3 3
11168 253/300 Read block 3593 len 24576
11169 254/300 Flush
11170 255/300 Read block 4915 len 16384
11171 256/300 Read block 3613 len 12288
11172 257/300 Read block 239 len 24576
11173 258/300 Read block 3903 len 36864
11174 259/300 Read block 325 len 12288
11175 260/300 Flush
11176 261/300 Read block 4037 len 40960
11177 262/300 Write block 3368 len 20480 data: 4 3 3 4 4
11178 263/300 Read block 4242 len 16384
11179 264/300 Read block 966 len 36864
11180 265/300 Write block 752 len 40960 data: 2 2 3 3 3 3 3 3 3 3
11181 266/300 Flush
11182 267/300 Read block 1834 len 36864
11183 268/300 Read block 1721 len 40960
11184 269/300 Write block 4381 len 32768 data: 4 4 4 4 4 4 4 4
11185 270/300 Read block 1821 len 20480
11186 271/300 Read block 3466 len 16384
11187 272/300 Read block 4806 len 32768
11188 273/300 Write block 3628 len 16384 data: 3 3 3 3
11189 274/300 Write block 4505 len 8192 data: 2 2
11190 275/300 Read block 3376 len 16384
11191 276/300 Write block 2846 len 8192 data: 2 2
11192 277/300 Write block 822 len 32768 data: 3 3 3 2 2 2 2 2
11193 278/300 Write block 2525 len 16384 data: 4 3 3 3
11194 279/300 Write block 483 len 36864 data: 3 3 3 3 3 3 3 4 4
11195 280/300 Read block 3732 len 36864
11196 281/300 Write block 2103 len 28672 data: 3 3 4 3 2 2 3
11197 282/300 Read block 2869 len 12288
11198 283/300 Read block 2137 len 4096
11199 284/300 Write block 590 len 8192 data: 4 4
11200 285/300 Write block 4979 len 36864 data: 3 3 3 3 3 2 2 2 2
11201 286/300 Write block 3702 len 4096 data: 6
11202 287/300 Write block 2844 len 32768 data: 2 2 3 3 3 3 3 3
11203 288/300 Write block 3052 len 40960 data: 4 3 4 4 4 4 4 3 4 3
11204 289/300 Read block 1020 len 36864
11205 290/300 Read block 3422 len 28672
11206 291/300 Read block 4348 len 24576
11207 292/300 Read block 1773 len 36864
11208 293/300 Write block 3104 len 4096 data: 3
11209 294/300 Write block 1950 len 20480 data: 4 4 4 3 4
11210 295/300 Read block 1788 len 8192
11211 296/300 Read block 2438 len 20480
11212 297/300 Write block 637 len 16384 data: 3 4 5 5
11213 298/300 Read block 1936 len 8192
11214 299/300 Write block 321 len 36864 data: 4 5 6 6 5 5 5 5 6
11215 300/300 Flush
11216 301/300 Write block 2863 len 20480 data: 3 4 4 3 3
11217 Main task has work to do, go find it
11218 got dsc Start(1)
11219 start 1
11220 Dec 11 23:42:52.972 INFO request completed, latency_us: 269, response_code: 204, uri: /start/cid/1, method: POST, req_id: e6307c29-b69a-4b48-a235-86a10c3e857d, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
11221 Replay: started 1, returned:Ok(())
11222 ----------------------------------------------------------------
11223 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11224 States: Active Offline Active
11225 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11226 3718 Acked Read 8 Done Sent Done false
11227 3719 Acked Write 2 Done Sent Done false
11228 3720 Acked Write 2 Done Sent Done false
11229 3721 Acked Read 7 Done Sent Done false
11230 3722 Acked Read 4 Done Sent Done false
11231 3723 Acked Read 3 Done Sent Done false
11232 3724 Acked Read 5 Done Sent Done false
11233 3725 Acked Write 8 Done Sent Done false
11234 3726 Acked Read 7 Done Sent Done false
11235 3727 Acked Read 4 Done Sent Done false
11236 3728 Acked Write 10 Done Sent Done false
11237 3729 Acked Write 8 Done Sent Done false
11238 3730 Acked Write 3 Done Sent Done false
11239 3731 Acked Read 10 Done Sent Done false
11240 3732 Acked Write 2 Done Sent Done false
11241 3733 Acked Read 1 Done Sent Done false
11242 3734 Acked Write 8 Done Sent Done false
11243 3735 Acked Read 9 Done Sent Done false
11244 3736 Acked Read 10 Done Sent Done false
11245 3737 Acked Read 1 Done Sent Done false
11246 3738 Acked Read 9 Done Sent Done false
11247 3739 Acked Write 1 Done Sent Done false
11248 3740 Acked Read 3 Done Sent Done false
11249 3741 Acked Flush 0 Done Sent Done false
11250 3742 Acked Read 2 Done Sent Done false
11251 3743 Acked Flush 0 Done Sent Done false
11252 3744 Acked Read 9 Done Sent Done false
11253 3745 Acked Read 1 Done Sent Done false
11254 3746 Acked Write 1 Done Sent Done false
11255 3747 Acked Read 4 Done Sent Done false
11256 3748 Acked Read 8 Done Sent Done false
11257 3749 Acked Write 4 Done Sent Done false
11258 3750 Acked Read 5 Done Sent Done false
11259 3751 Acked Write 3 Done Sent Done false
11260 3752 Acked Write 3 Done Sent Done false
11261 3753 Acked Write 4 Done Sent Done false
11262 3754 Acked Read 9 Done Sent Done false
11263 3755 Acked Read 2 Done Sent Done false
11264 3756 Acked Write 8 Done Sent Done false
11265 3757 Acked Read 3 Done Sent Done false
11266 3758 Acked Read 10 Done Sent Done false
11267 3759 Acked Read 7 Done Sent Done false
11268 3760 Acked Read 2 Done Sent Done false
11269 3761 Acked Read 2 Done Sent Done false
11270 3762 Acked Flush 0 Done Sent Done false
11271 3763 Acked Read 9 Done Sent Done false
11272 3764 Acked Write 9 Done Sent Done false
11273 3765 Acked Flush 0 Done Sent Done false
11274 3766 Acked Read 7 Done Sent Done false
11275 3767 Acked Write 7 Done Sent Done false
11276 3768 Acked Write 9 Done Sent Done false
11277 3769 Acked Write 3 Done Sent Done false
11278 3770 Acked Flush 0 Done Sent Done false
11279 3771 Acked Read 3 Done Sent Done false
11280 3772 Acked Write 7 Done Sent Done false
11281 3773 Acked Read 7 Done Sent Done false
11282 3774 Acked Read 4 Done Sent Done false
11283 3775 Acked Read 3 Done Sent Done false
11284 3776 Acked Write 7 Done Sent Done false
11285 3777 Acked Write 1 Done Sent Done false
11286 3778 Acked Write 3 Done Sent Done false
11287 3779 Acked Flush 0 Done Sent Done false
11288 3780 Acked Write 4 Done Sent Done false
11289 3781 Acked Flush 0 Done Sent Done false
11290 3782 Acked Write 3 Done Sent Done false
11291 3783 Acked Write 8 Done Sent Done false
11292 3784 Acked Write 5 Done Sent Done false
11293 3785 Acked Read 4 Done Sent Done false
11294 3786 Acked Read 8 Done Sent Done false
11295 3787 Acked Read 1 Done Sent Done false
11296 3788 Acked Flush 0 Done Sent Done false
11297 3789 Acked Write 8 Done Sent Done false
11298 3790 Acked Flush 0 Done Sent Done false
11299 3791 Acked Write 2 Done Sent Done false
11300 3792 Acked Write 4 Done Sent Done false
11301 3793 Acked Write 1 Done Sent Done false
11302 3794 Acked Write 5 Done Sent Done false
11303 3795 Acked Read 3 Done Sent Done false
11304 3796 Acked Read 3 Done Sent Done false
11305 3797 Acked Read 1 Done Sent Done false
11306 3798 Acked Write 6 Done Sent Done false
11307 3799 Acked Write 3 Done Sent Done false
11308 3800 Acked Read 7 Done Sent Done false
11309 3801 Acked Read 3 Done Sent Done false
11310 3802 Acked Write 2 Done Sent Done false
11311 3803 Acked Read 7 Done Sent Done false
11312 3804 Acked Write 5 Done Sent Done false
11313 3805 Acked Write 5 Done Sent Done false
11314 3806 Acked Read 4 Done Sent Done false
11315 3807 Acked Write 8 Done Sent Done false
11316 3808 Acked Read 4 Done Sent Done false
11317 3809 Acked Read 7 Done Sent Done false
11318 3810 Acked Read 1 Done Sent Done false
11319 3811 Acked Read 6 Done Sent Done false
11320 3812 Acked Write 4 Done Sent Done false
11321 3813 Acked Read 5 Done Sent Done false
11322 3814 Acked Read 8 Done Sent Done false
11323 3815 Acked Read 2 Done Sent Done false
11324 3816 Acked Read 8 Done Sent Done false
11325 3817 Acked Write 1 Done Sent Done false
11326 3818 Acked Flush 0 Done Sent Done false
11327 3819 Acked Write 7 Done Sent Done false
11328 3820 Acked Read 5 Done Sent Done false
11329 3821 Acked Read 2 Done Sent Done false
11330 3822 Acked Write 8 Done Sent Done false
11331 3823 Acked Flush 0 Done Sent Done false
11332 3824 Acked Write 4 Done Sent Done false
11333 3825 Acked Read 6 Done Sent Done false
11334 3826 Acked Read 4 Done Sent Done false
11335 3827 Acked Read 2 Done Sent Done false
11336 3828 Acked Write 2 Done Sent Done false
11337 3829 Acked Read 1 Done Sent Done false
11338 3830 Acked Write 1 Done Sent Done false
11339 3831 Acked Flush 0 Done Sent Done false
11340 3832 Acked Read 3 Done Sent Done false
11341 3833 Acked Flush 0 Done Sent Done false
11342 3834 Acked Write 8 Done Sent Done false
11343 3835 Acked Read 5 Done Sent Done false
11344 3836 Acked Read 6 Done Sent Done false
11345 3837 Acked Write 2 Done Sent Done false
11346 3838 Acked Write 2 Done Sent Done false
11347 3839 Acked Read 3 Done Sent Done false
11348 3840 Acked Write 5 Done Sent Done false
11349 3841 Acked Read 10 Done Sent Done false
11350 3842 Acked Write 5 Done Sent Done false
11351 3843 Acked Read 1 Done Sent Done false
11352 3844 Acked Flush 0 Done Sent Done false
11353 3845 Acked Read 1 Done Sent Done false
11354 3846 Acked Read 5 Done Sent Done false
11355 3847 Acked Write 5 Done Sent Done false
11356 3848 Acked Write 9 Done Sent Done false
11357 3849 Acked Write 8 Done Sent Done false
11358 3850 Acked Write 8 Done Sent Done false
11359 3851 Acked Flush 0 Done Sent Done false
11360 3852 Acked Read 2 Done Sent Done false
11361 3853 Acked Flush 0 Done Sent Done false
11362 3854 Acked Read 3 Done Sent Done false
11363 3855 Acked Read 9 Done Sent Done false
11364 3856 Acked Read 1 Done Sent Done false
11365 3857 Acked Flush 0 Done Sent Done false
11366 3858 Acked Flush 0 Done Sent Done false
11367 3859 Acked Read 5 Done Sent Done false
11368 3860 Acked Read 7 Done Sent Done false
11369 3861 Acked Write 7 Done Sent Done false
11370 3862 Acked Read 10 Done Sent Done false
11371 3863 Acked Write 9 Done Sent Done false
11372 3864 Acked Flush 0 Done Sent Done false
11373 3865 Acked Write 4 Done Sent Done false
11374 3866 Acked Write 3 Done Sent Done false
11375 3867 Acked Read 7 Done Sent Done false
11376 3868 Acked Read 10 Done Sent Done false
11377 3869 Acked Read 8 Done Sent Done false
11378 3870 Acked Read 9 Done Sent Done false
11379 3871 Acked Read 8 Done Sent Done false
11380 3872 Acked Write 6 Done Sent Done false
11381 3873 Acked Read 2 Done Sent Done false
11382 3874 Acked Flush 0 Done Sent Done false
11383 3875 Acked Write 6 Done Sent Done false
11384 3876 Acked Read 8 Done Sent Done false
11385 3877 Acked Write 7 Done Sent Done false
11386 3878 Acked Write 5 Done Sent Done false
11387 3879 Acked Read 8 Done Sent Done false
11388 3880 Acked Read 2 Done Sent Done false
11389 3881 Acked Write 10 Done Sent Done false
11390 3882 Acked Read 9 Done Sent Done false
11391 3883 Acked Write 1 Done Sent Done false
11392 3884 Acked Read 10 Done Sent Done false
11393 3885 Acked Read 3 Done Sent Done false
11394 3886 Acked Read 3 Done Sent Done false
11395 3887 Acked Read 2 Done Sent Done false
11396 3888 Acked Flush 0 Done Sent Done false
11397 3889 Acked Write 1 Done Sent Done false
11398 3890 Acked Flush 0 Done Sent Done false
11399 3891 Acked Write 7 Done Sent Done false
11400 3892 Acked Read 8 Done Sent Done false
11401 3893 Acked Read 1 Done Sent Done false
11402 3894 Acked Write 2 Done Sent Done false
11403 3895 Acked Read 2 Done Sent Done false
11404 3896 Acked Read 1 Done Sent Done false
11405 3897 Acked Write 9 Done Sent Done false
11406 3898 Acked Read 7 Done Sent Done false
11407 3899 Acked Read 6 Done Sent Done false
11408 3900 Acked Read 4 Done Sent Done false
11409 3901 Acked Flush 0 Done Sent Done false
11410 3902 Acked Write 1 Done Sent Done false
11411 3903 Acked Flush 0 Done Sent Done false
11412 3904 Acked Write 3 Done Sent Done false
11413 3905 Acked Read 5 Done Sent Done false
11414 3906 Acked Flush 0 Done Sent Done false
11415 3907 Acked Read 10 Done Sent Done false
11416 3908 Acked Read 3 Done Sent Done false
11417 3909 Acked Read 4 Done Sent Done false
11418 3910 Acked Read 4 Done Sent Done false
11419 3911 Acked Write 9 Done Sent Done false
11420 3912 Acked Write 7 Done Sent Done false
11421 3913 Acked Read 2 Done Sent Done false
11422 3914 Acked Read 1 Done Sent Done false
11423 3915 Acked Write 5 Done Sent Done false
11424 3916 Acked Read 10 Done Sent Done false
11425 3917 Acked Flush 0 Done Sent Done false
11426 3918 Acked Write 10 Done Sent Done false
11427 3919 Acked Read 4 Done Sent Done false
11428 3920 Acked Write 9 Done Sent Done false
11429 3921 Acked Flush 0 Done Sent Done false
11430 3922 Acked Read 4 Done Sent Done false
11431 3923 Acked Read 2 Done Sent Done false
11432 3924 Acked Write 4 Done Sent Done false
11433 3925 Acked Flush 0 Done Sent Done false
11434 3926 Acked Write 4 Done Sent Done false
11435 3927 Acked Write 5 Done Sent Done false
11436 3928 Acked Write 9 Done Sent Done false
11437 3929 Acked Read 1 Done Sent Done false
11438 3930 Acked Read 9 Done Sent Done false
11439 3931 Acked Read 4 Done Sent Done false
11440 3932 Acked Read 5 Done Sent Done false
11441 3933 Acked Write 10 Done Sent Done false
11442 3934 Acked Read 7 Done Sent Done false
11443 3935 Acked Read 10 Done Sent Done false
11444 3936 Acked Read 8 Done Sent Done false
11445 3937 Acked Read 5 Done Sent Done false
11446 3938 Acked Read 6 Done Sent Done false
11447 3939 Acked Write 1 Done Sent Done false
11448 3940 Acked Read 6 Done Sent Done false
11449 3941 Acked Write 8 Done Sent Done false
11450 3942 Acked Write 4 Done Sent Done false
11451 3943 Acked Write 1 Done Sent Done false
11452 3944 Acked Read 9 Done Sent Done false
11453 3945 Acked Write 8 Done Sent Done false
11454 3946 Acked Write 5 Done Sent Done false
11455 3947 Acked Read 6 Done Sent Done false
11456 3948 Acked Write 8 Done Sent Done false
11457 3949 Acked Write 5 Done Sent Done false
11458 3950 Acked Read 6 Done Sent Done false
11459 3951 Acked Read 3 Done Sent Done false
11460 3952 Acked Write 4 Done Sent Done false
11461 3953 Acked Read 4 Done Sent Done false
11462 3954 Acked Read 9 Done Sent Done false
11463 3955 Acked Write 9 Done Sent Done false
11464 3956 Acked Write 1 Done Sent Done false
11465 3957 Acked Flush 0 Done Sent Done false
11466 3958 Acked Flush 0 Done Sent Done false
11467 3959 Acked Write 6 Done Sent Done false
11468 3960 Acked Write 3 Done Sent Done false
11469 3961 Acked Flush 0 Done Sent Done false
11470 3962 Acked Write 3 Done Sent Done false
11471 3963 Acked Read 5 Done Sent Done false
11472 3964 Acked Write 9 Done Sent Done false
11473 3965 Acked Read 10 Done Sent Done false
11474 3966 Acked Read 1 Done Sent Done false
11475 3967 Acked Write 9 Done Sent Done false
11476 3968 Acked Read 3 Done Sent Done false
11477 3969 Acked Write 10 Done Sent Done false
11478 3970 Acked Read 6 Done Sent Done false
11479 3971 Acked Flush 0 Done Sent Done false
11480 3972 Acked Read 4 Done Sent Done false
11481 3973 Acked Read 3 Done Sent Done false
11482 3974 Acked Read 6 Done Sent Done false
11483 3975 Acked Read 9 Done Sent Done false
11484 3976 Acked Read 3 Done Sent Done false
11485 3977 Acked Flush 0 Done Sent Done false
11486 3978 Acked Read 10 Done Sent Done false
11487 3979 Acked Write 5 Done Sent Done false
11488 3980 Acked Read 4 Done Sent Done false
11489 3981 Acked Read 9 Done Sent Done false
11490 3982 Acked Write 10 Done Sent Done false
11491 3983 Acked Flush 0 Done Sent Done false
11492 3984 Acked Read 9 Done Sent Done false
11493 3985 Acked Read 10 Done Sent Done false
11494 3986 Acked Write 8 Done Sent Done false
11495 3987 Acked Read 5 Done Sent Done false
11496 3988 Acked Read 4 Done Sent Done false
11497 3989 Acked Read 8 Done Sent Done false
11498 3990 Acked Write 4 Done Sent Done false
11499 3991 Acked Write 2 Done Sent Done false
11500 3992 Acked Read 4 Done Sent Done false
11501 3993 Acked Write 2 Done Sent Done false
11502 3994 Acked Write 8 Done Sent Done false
11503 3995 Acked Write 4 Done Sent Done false
11504 3996 Acked Write 9 Done Sent Done false
11505 3997 Acked Read 9 Done Sent Done false
11506 3998 Acked Write 7 Done Sent Done false
11507 3999 Acked Read 3 Done Sent Done false
11508 4000 Acked Read 1 Done Sent Done false
11509 4001 Acked Write 2 Done Sent Done false
11510 4002 Acked Write 9 Done Sent Done false
11511 4003 Acked Write 1 Done Sent Done false
11512 4004 Acked Write 8 Done Sent Done false
11513 4005 Acked Write 10 Done Sent Done false
11514 4006 Acked Read 9 Done Sent Done false
11515 4007 Acked Read 7 Done Sent Done false
11516 4008 Acked Read 6 Done Sent Done false
11517 4009 Acked Read 9 Done Sent Done false
11518 4010 Acked Write 1 Done Sent Done false
11519 4011 Acked Write 5 Done Sent Done false
11520 4012 Acked Read 2 Done Sent Done false
11521 4013 Acked Read 5 Done Sent Done false
11522 4014 Acked Write 4 Done Sent Done false
11523 4015 Acked Read 2 Done Sent Done false
11524 4016 Acked Write 9 Done Sent Done false
11525 4017 Acked Flush 0 Done Sent Done false
11526 4018 Acked Write 5 Done Sent Done false
11527 STATES DS:0 DS:1 DS:2 TOTAL
11528 Sent 0 301 0 301
11529 Done 301 0 301 602
11530 Skipped 0 0 0 0
11531 Error 0 0 0 0
11532 Last Flush: 4017 3717 4017
11533 Downstairs last five completed: 3717 3716 3715 3714 3713
11534 Upstairs last five completed: 4018 4017 4016 4015 4014
11535 CLIENT: Up:0 ds:301 act:2
11536 ----------------------------------------------------------------
11537 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11538 States: Active Offline Active
11539 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11540 3718 Acked Read 8 Done Sent Done false
11541 3719 Acked Write 2 Done Sent Done false
11542 3720 Acked Write 2 Done Sent Done false
11543 3721 Acked Read 7 Done Sent Done false
11544 3722 Acked Read 4 Done Sent Done false
11545 3723 Acked Read 3 Done Sent Done false
11546 3724 Acked Read 5 Done Sent Done false
11547 3725 Acked Write 8 Done Sent Done false
11548 3726 Acked Read 7 Done Sent Done false
11549 3727 Acked Read 4 Done Sent Done false
11550 3728 Acked Write 10 Done Sent Done false
11551 3729 Acked Write 8 Done Sent Done false
11552 3730 Acked Write 3 Done Sent Done false
11553 3731 Acked Read 10 Done Sent Done false
11554 3732 Acked Write 2 Done Sent Done false
11555 3733 Acked Read 1 Done Sent Done false
11556 3734 Acked Write 8 Done Sent Done false
11557 3735 Acked Read 9 Done Sent Done false
11558 3736 Acked Read 10 Done Sent Done false
11559 3737 Acked Read 1 Done Sent Done false
11560 3738 Acked Read 9 Done Sent Done false
11561 3739 Acked Write 1 Done Sent Done false
11562 3740 Acked Read 3 Done Sent Done false
11563 3741 Acked Flush 0 Done Sent Done false
11564 3742 Acked Read 2 Done Sent Done false
11565 3743 Acked Flush 0 Done Sent Done false
11566 3744 Acked Read 9 Done Sent Done false
11567 3745 Acked Read 1 Done Sent Done false
11568 3746 Acked Write 1 Done Sent Done false
11569 3747 Acked Read 4 Done Sent Done false
11570 3748 Acked Read 8 Done Sent Done false
11571 3749 Acked Write 4 Done Sent Done false
11572 3750 Acked Read 5 Done Sent Done false
11573 3751 Acked Write 3 Done Sent Done false
11574 3752 Acked Write 3 Done Sent Done false
11575 3753 Acked Write 4 Done Sent Done false
11576 3754 Acked Read 9 Done Sent Done false
11577 3755 Acked Read 2 Done Sent Done false
11578 3756 Acked Write 8 Done Sent Done false
11579 3757 Acked Read 3 Done Sent Done false
11580 3758 Acked Read 10 Done Sent Done false
11581 3759 Acked Read 7 Done Sent Done false
11582 3760 Acked Read 2 Done Sent Done false
11583 3761 Acked Read 2 Done Sent Done false
11584 3762 Acked Flush 0 Done Sent Done false
11585 3763 Acked Read 9 Done Sent Done false
11586 3764 Acked Write 9 Done Sent Done false
11587 3765 Acked Flush 0 Done Sent Done false
11588 3766 Acked Read 7 Done Sent Done false
11589 3767 Acked Write 7 Done Sent Done false
11590 3768 Acked Write 9 Done Sent Done false
11591 3769 Acked Write 3 Done Sent Done false
11592 3770 Acked Flush 0 Done Sent Done false
11593 3771 Acked Read 3 Done Sent Done false
11594 3772 Acked Write 7 Done Sent Done false
11595 3773 Acked Read 7 Done Sent Done false
11596 3774 Acked Read 4 Done Sent Done false
11597 3775 Acked Read 3 Done Sent Done false
11598 3776 Acked Write 7 Done Sent Done false
11599 3777 Acked Write 1 Done Sent Done false
11600 3778 Acked Write 3 Done Sent Done false
11601 3779 Acked Flush 0 Done Sent Done false
11602 3780 Acked Write 4 Done Sent Done false
11603 3781 Acked Flush 0 Done Sent Done false
11604 3782 Acked Write 3 Done Sent Done false
11605 3783 Acked Write 8 Done Sent Done false
11606 3784 Acked Write 5 Done Sent Done false
11607 3785 Acked Read 4 Done Sent Done false
11608 3786 Acked Read 8 Done Sent Done false
11609 3787 Acked Read 1 Done Sent Done false
11610 3788 Acked Flush 0 Done Sent Done false
11611 3789 Acked Write 8 Done Sent Done false
11612 3790 Acked Flush 0 Done Sent Done false
11613 3791 Acked Write 2 Done Sent Done false
11614 3792 Acked Write 4 Done Sent Done false
11615 3793 Acked Write 1 Done Sent Done false
11616 3794 Acked Write 5 Done Sent Done false
11617 3795 Acked Read 3 Done Sent Done false
11618 3796 Acked Read 3 Done Sent Done false
11619 3797 Acked Read 1 Done Sent Done false
11620 3798 Acked Write 6 Done Sent Done false
11621 3799 Acked Write 3 Done Sent Done false
11622 3800 Acked Read 7 Done Sent Done false
11623 3801 Acked Read 3 Done Sent Done false
11624 3802 Acked Write 2 Done Sent Done false
11625 3803 Acked Read 7 Done Sent Done false
11626 3804 Acked Write 5 Done Sent Done false
11627 3805 Acked Write 5 Done Sent Done false
11628 3806 Acked Read 4 Done Sent Done false
11629 3807 Acked Write 8 Done Sent Done false
11630 3808 Acked Read 4 Done Sent Done false
11631 3809 Acked Read 7 Done Sent Done false
11632 3810 Acked Read 1 Done Sent Done false
11633 3811 Acked Read 6 Done Sent Done false
11634 3812 Acked Write 4 Done Sent Done false
11635 3813 Acked Read 5 Done Sent Done false
11636 3814 Acked Read 8 Done Sent Done false
11637 3815 Acked Read 2 Done Sent Done false
11638 3816 Acked Read 8 Done Sent Done false
11639 3817 Acked Write 1 Done Sent Done false
11640 3818 Acked Flush 0 Done Sent Done false
11641 3819 Acked Write 7 Done Sent Done false
11642 3820 Acked Read 5 Done Sent Done false
11643 3821 Acked Read 2 Done Sent Done false
11644 3822 Acked Write 8 Done Sent Done false
11645 3823 Acked Flush 0 Done Sent Done false
11646 3824 Acked Write 4 Done Sent Done false
11647 3825 Acked Read 6 Done Sent Done false
11648 3826 Acked Read 4 Done Sent Done false
11649 3827 Acked Read 2 Done Sent Done false
11650 3828 Acked Write 2 Done Sent Done false
11651 3829 Acked Read 1 Done Sent Done false
11652 3830 Acked Write 1 Done Sent Done false
11653 3831 Acked Flush 0 Done Sent Done false
11654 3832 Acked Read 3 Done Sent Done false
11655 3833 Acked Flush 0 Done Sent Done false
11656 3834 Acked Write 8 Done Sent Done false
11657 3835 Acked Read 5 Done Sent Done false
11658 3836 Acked Read 6 Done Sent Done false
11659 3837 Acked Write 2 Done Sent Done false
11660 3838 Acked Write 2 Done Sent Done false
11661 3839 Acked Read 3 Done Sent Done false
11662 3840 Acked Write 5 Done Sent Done false
11663 3841 Acked Read 10 Done Sent Done false
11664 3842 Acked Write 5 Done Sent Done false
11665 3843 Acked Read 1 Done Sent Done false
11666 3844 Acked Flush 0 Done Sent Done false
11667 3845 Acked Read 1 Done Sent Done false
11668 3846 Acked Read 5 Done Sent Done false
11669 3847 Acked Write 5 Done Sent Done false
11670 3848 Acked Write 9 Done Sent Done false
11671 3849 Acked Write 8 Done Sent Done false
11672 3850 Acked Write 8 Done Sent Done false
11673 3851 Acked Flush 0 Done Sent Done false
11674 3852 Acked Read 2 Done Sent Done false
11675 3853 Acked Flush 0 Done Sent Done false
11676 3854 Acked Read 3 Done Sent Done false
11677 3855 Acked Read 9 Done Sent Done false
11678 3856 Acked Read 1 Done Sent Done false
11679 3857 Acked Flush 0 Done Sent Done false
11680 3858 Acked Flush 0 Done Sent Done false
11681 3859 Acked Read 5 Done Sent Done false
11682 3860 Acked Read 7 Done Sent Done false
11683 3861 Acked Write 7 Done Sent Done false
11684 3862 Acked Read 10 Done Sent Done false
11685 3863 Acked Write 9 Done Sent Done false
11686 3864 Acked Flush 0 Done Sent Done false
11687 3865 Acked Write 4 Done Sent Done false
11688 3866 Acked Write 3 Done Sent Done false
11689 3867 Acked Read 7 Done Sent Done false
11690 3868 Acked Read 10 Done Sent Done false
11691 3869 Acked Read 8 Done Sent Done false
11692 3870 Acked Read 9 Done Sent Done false
11693 3871 Acked Read 8 Done Sent Done false
11694 3872 Acked Write 6 Done Sent Done false
11695 3873 Acked Read 2 Done Sent Done false
11696 3874 Acked Flush 0 Done Sent Done false
11697 3875 Acked Write 6 Done Sent Done false
11698 3876 Acked Read 8 Done Sent Done false
11699 3877 Acked Write 7 Done Sent Done false
11700 3878 Acked Write 5 Done Sent Done false
11701 3879 Acked Read 8 Done Sent Done false
11702 3880 Acked Read 2 Done Sent Done false
11703 3881 Acked Write 10 Done Sent Done false
11704 3882 Acked Read 9 Done Sent Done false
11705 3883 Acked Write 1 Done Sent Done false
11706 3884 Acked Read 10 Done Sent Done false
11707 3885 Acked Read 3 Done Sent Done false
11708 3886 Acked Read 3 Done Sent Done false
11709 3887 Acked Read 2 Done Sent Done false
11710 3888 Acked Flush 0 Done Sent Done false
11711 3889 Acked Write 1 Done Sent Done false
11712 3890 Acked Flush 0 Done Sent Done false
11713 3891 Acked Write 7 Done Sent Done false
11714 3892 Acked Read 8 Done Sent Done false
11715 3893 Acked Read 1 Done Sent Done false
11716 3894 Acked Write 2 Done Sent Done false
11717 3895 Acked Read 2 Done Sent Done false
11718 3896 Acked Read 1 Done Sent Done false
11719 3897 Acked Write 9 Done Sent Done false
11720 3898 Acked Read 7 Done Sent Done false
11721 3899 Acked Read 6 Done Sent Done false
11722 3900 Acked Read 4 Done Sent Done false
11723 3901 Acked Flush 0 Done Sent Done false
11724 3902 Acked Write 1 Done Sent Done false
11725 3903 Acked Flush 0 Done Sent Done false
11726 3904 Acked Write 3 Done Sent Done false
11727 3905 Acked Read 5 Done Sent Done false
11728 3906 Acked Flush 0 Done Sent Done false
11729 3907 Acked Read 10 Done Sent Done false
11730 3908 Acked Read 3 Done Sent Done false
11731 3909 Acked Read 4 Done Sent Done false
11732 3910 Acked Read 4 Done Sent Done false
11733 3911 Acked Write 9 Done Sent Done false
11734 3912 Acked Write 7 Done Sent Done false
11735 3913 Acked Read 2 Done Sent Done false
11736 3914 Acked Read 1 Done Sent Done false
11737 3915 Acked Write 5 Done Sent Done false
11738 3916 Acked Read 10 Done Sent Done false
11739 3917 Acked Flush 0 Done Sent Done false
11740 3918 Acked Write 10 Done Sent Done false
11741 3919 Acked Read 4 Done Sent Done false
11742 3920 Acked Write 9 Done Sent Done false
11743 3921 Acked Flush 0 Done Sent Done false
11744 3922 Acked Read 4 Done Sent Done false
11745 3923 Acked Read 2 Done Sent Done false
11746 3924 Acked Write 4 Done Sent Done false
11747 3925 Acked Flush 0 Done Sent Done false
11748 3926 Acked Write 4 Done Sent Done false
11749 3927 Acked Write 5 Done Sent Done false
11750 3928 Acked Write 9 Done Sent Done false
11751 3929 Acked Read 1 Done Sent Done false
11752 3930 Acked Read 9 Done Sent Done false
11753 3931 Acked Read 4 Done Sent Done false
11754 3932 Acked Read 5 Done Sent Done false
11755 3933 Acked Write 10 Done Sent Done false
11756 3934 Acked Read 7 Done Sent Done false
11757 3935 Acked Read 10 Done Sent Done false
11758 3936 Acked Read 8 Done Sent Done false
11759 3937 Acked Read 5 Done Sent Done false
11760 3938 Acked Read 6 Done Sent Done false
11761 3939 Acked Write 1 Done Sent Done false
11762 3940 Acked Read 6 Done Sent Done false
11763 3941 Acked Write 8 Done Sent Done false
11764 3942 Acked Write 4 Done Sent Done false
11765 3943 Acked Write 1 Done Sent Done false
11766 3944 Acked Read 9 Done Sent Done false
11767 3945 Acked Write 8 Done Sent Done false
11768 3946 Acked Write 5 Done Sent Done false
11769 3947 Acked Read 6 Done Sent Done false
11770 3948 Acked Write 8 Done Sent Done false
11771 3949 Acked Write 5 Done Sent Done false
11772 3950 Acked Read 6 Done Sent Done false
11773 3951 Acked Read 3 Done Sent Done false
11774 3952 Acked Write 4 Done Sent Done false
11775 3953 Acked Read 4 Done Sent Done false
11776 3954 Acked Read 9 Done Sent Done false
11777 3955 Acked Write 9 Done Sent Done false
11778 3956 Acked Write 1 Done Sent Done false
11779 3957 Acked Flush 0 Done Sent Done false
11780 3958 Acked Flush 0 Done Sent Done false
11781 3959 Acked Write 6 Done Sent Done false
11782 3960 Acked Write 3 Done Sent Done false
11783 3961 Acked Flush 0 Done Sent Done false
11784 3962 Acked Write 3 Done Sent Done false
11785 3963 Acked Read 5 Done Sent Done false
11786 3964 Acked Write 9 Done Sent Done false
11787 3965 Acked Read 10 Done Sent Done false
11788 3966 Acked Read 1 Done Sent Done false
11789 3967 Acked Write 9 Done Sent Done false
11790 3968 Acked Read 3 Done Sent Done false
11791 3969 Acked Write 10 Done Sent Done false
11792 3970 Acked Read 6 Done Sent Done false
11793 3971 Acked Flush 0 Done Sent Done false
11794 3972 Acked Read 4 Done Sent Done false
11795 3973 Acked Read 3 Done Sent Done false
11796 3974 Acked Read 6 Done Sent Done false
11797 3975 Acked Read 9 Done Sent Done false
11798 3976 Acked Read 3 Done Sent Done false
11799 3977 Acked Flush 0 Done Sent Done false
11800 3978 Acked Read 10 Done Sent Done false
11801 3979 Acked Write 5 Done Sent Done false
11802 3980 Acked Read 4 Done Sent Done false
11803 3981 Acked Read 9 Done Sent Done false
11804 3982 Acked Write 10 Done Sent Done false
11805 3983 Acked Flush 0 Done Sent Done false
11806 3984 Acked Read 9 Done Sent Done false
11807 3985 Acked Read 10 Done Sent Done false
11808 3986 Acked Write 8 Done Sent Done false
11809 3987 Acked Read 5 Done Sent Done false
11810 3988 Acked Read 4 Done Sent Done false
11811 3989 Acked Read 8 Done Sent Done false
11812 3990 Acked Write 4 Done Sent Done false
11813 3991 Acked Write 2 Done Sent Done false
11814 3992 Acked Read 4 Done Sent Done false
11815 3993 Acked Write 2 Done Sent Done false
11816 3994 Acked Write 8 Done Sent Done false
11817 3995 Acked Write 4 Done Sent Done false
11818 3996 Acked Write 9 Done Sent Done false
11819 3997 Acked Read 9 Done Sent Done false
11820 3998 Acked Write 7 Done Sent Done false
11821 3999 Acked Read 3 Done Sent Done false
11822 4000 Acked Read 1 Done Sent Done false
11823 4001 Acked Write 2 Done Sent Done false
11824 4002 Acked Write 9 Done Sent Done false
11825 4003 Acked Write 1 Done Sent Done false
11826 4004 Acked Write 8 Done Sent Done false
11827 4005 Acked Write 10 Done Sent Done false
11828 4006 Acked Read 9 Done Sent Done false
11829 4007 Acked Read 7 Done Sent Done false
11830 4008 Acked Read 6 Done Sent Done false
11831 4009 Acked Read 9 Done Sent Done false
11832 4010 Acked Write 1 Done Sent Done false
11833 4011 Acked Write 5 Done Sent Done false
11834 4012 Acked Read 2 Done Sent Done false
11835 4013 Acked Read 5 Done Sent Done false
11836 4014 Acked Write 4 Done Sent Done false
11837 4015 Acked Read 2 Done Sent Done false
11838 4016 Acked Write 9 Done Sent Done false
11839 4017 Acked Flush 0 Done Sent Done false
11840 4018 Acked Write 5 Done Sent Done false
11841 4019 Acked Flush 0 Done Sent Done false
11842 STATES DS:0 DS:1 DS:2 TOTAL
11843 Sent 0 302 0 302
11844 Done 302 0 302 604
11845 Skipped 0 0 0 0
11846 Error 0 0 0 0
11847 Last Flush: 4019 3717 4019
11848 Downstairs last five completed: 3717 3716 3715 3714 3713
11849 Upstairs last five completed: 4019 4018 4017 4016 4015
11850 CLIENT: Up:0 ds:302 act:2
118512024-12-11T23:42:58.736ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
118522024-12-11T23:42:58.736ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
118532024-12-11T23:42:58.736ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
118542024-12-11T23:42:58.736ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
118552024-12-11T23:42:58.736ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
11856 [8820] got start action
11857 [8820] I am going to restart
11858 Starting downstairs at port 8820
11859 Make output file at "/tmp/dsc/downstairs-8820.txt"
11860 [8820][1] reports Starting
11861 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1210)
11862 [8820][1] reports Running
11863 ----------------------------------------------------------------
11864 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11865 States: Active Offline Active
11866 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11867 3718 Acked Read 8 Done Sent Done false
11868 3719 Acked Write 2 Done Sent Done false
11869 3720 Acked Write 2 Done Sent Done false
11870 3721 Acked Read 7 Done Sent Done false
11871 3722 Acked Read 4 Done Sent Done false
11872 3723 Acked Read 3 Done Sent Done false
11873 3724 Acked Read 5 Done Sent Done false
11874 3725 Acked Write 8 Done Sent Done false
11875 3726 Acked Read 7 Done Sent Done false
11876 3727 Acked Read 4 Done Sent Done false
11877 3728 Acked Write 10 Done Sent Done false
11878 3729 Acked Write 8 Done Sent Done false
11879 3730 Acked Write 3 Done Sent Done false
11880 3731 Acked Read 10 Done Sent Done false
11881 3732 Acked Write 2 Done Sent Done false
11882 3733 Acked Read 1 Done Sent Done false
11883 3734 Acked Write 8 Done Sent Done false
11884 3735 Acked Read 9 Done Sent Done false
11885 3736 Acked Read 10 Done Sent Done false
11886 3737 Acked Read 1 Done Sent Done false
11887 3738 Acked Read 9 Done Sent Done false
11888 3739 Acked Write 1 Done Sent Done false
11889 3740 Acked Read 3 Done Sent Done false
11890 3741 Acked Flush 0 Done Sent Done false
11891 3742 Acked Read 2 Done Sent Done false
11892 3743 Acked Flush 0 Done Sent Done false
11893 3744 Acked Read 9 Done Sent Done false
11894 3745 Acked Read 1 Done Sent Done false
11895 3746 Acked Write 1 Done Sent Done false
11896 3747 Acked Read 4 Done Sent Done false
11897 3748 Acked Read 8 Done Sent Done false
11898 3749 Acked Write 4 Done Sent Done false
11899 3750 Acked Read 5 Done Sent Done false
11900 3751 Acked Write 3 Done Sent Done false
11901 3752 Acked Write 3 Done Sent Done false
11902 3753 Acked Write 4 Done Sent Done false
11903 3754 Acked Read 9 Done Sent Done false
11904 3755 Acked Read 2 Done Sent Done false
11905 3756 Acked Write 8 Done Sent Done false
11906 3757 Acked Read 3 Done Sent Done false
11907 3758 Acked Read 10 Done Sent Done false
11908 3759 Acked Read 7 Done Sent Done false
11909 3760 Acked Read 2 Done Sent Done false
11910 3761 Acked Read 2 Done Sent Done false
11911 3762 Acked Flush 0 Done Sent Done false
11912 3763 Acked Read 9 Done Sent Done false
11913 3764 Acked Write 9 Done Sent Done false
11914 3765 Acked Flush 0 Done Sent Done false
11915 3766 Acked Read 7 Done Sent Done false
11916 3767 Acked Write 7 Done Sent Done false
11917 3768 Acked Write 9 Done Sent Done false
11918 3769 Acked Write 3 Done Sent Done false
11919 3770 Acked Flush 0 Done Sent Done false
11920 3771 Acked Read 3 Done Sent Done false
11921 3772 Acked Write 7 Done Sent Done false
11922 3773 Acked Read 7 Done Sent Done false
11923 3774 Acked Read 4 Done Sent Done false
11924 3775 Acked Read 3 Done Sent Done false
11925 3776 Acked Write 7 Done Sent Done false
11926 3777 Acked Write 1 Done Sent Done false
11927 3778 Acked Write 3 Done Sent Done false
11928 3779 Acked Flush 0 Done Sent Done false
11929 3780 Acked Write 4 Done Sent Done false
11930 3781 Acked Flush 0 Done Sent Done false
11931 3782 Acked Write 3 Done Sent Done false
11932 3783 Acked Write 8 Done Sent Done false
11933 3784 Acked Write 5 Done Sent Done false
11934 3785 Acked Read 4 Done Sent Done false
11935 3786 Acked Read 8 Done Sent Done false
11936 3787 Acked Read 1 Done Sent Done false
11937 3788 Acked Flush 0 Done Sent Done false
11938 3789 Acked Write 8 Done Sent Done false
11939 3790 Acked Flush 0 Done Sent Done false
11940 3791 Acked Write 2 Done Sent Done false
11941 3792 Acked Write 4 Done Sent Done false
11942 3793 Acked Write 1 Done Sent Done false
11943 3794 Acked Write 5 Done Sent Done false
11944 3795 Acked Read 3 Done Sent Done false
11945 3796 Acked Read 3 Done Sent Done false
11946 3797 Acked Read 1 Done Sent Done false
11947 3798 Acked Write 6 Done Sent Done false
11948 3799 Acked Write 3 Done Sent Done false
11949 3800 Acked Read 7 Done Sent Done false
11950 3801 Acked Read 3 Done Sent Done false
11951 3802 Acked Write 2 Done Sent Done false
11952 3803 Acked Read 7 Done Sent Done false
11953 3804 Acked Write 5 Done Sent Done false
11954 3805 Acked Write 5 Done Sent Done false
11955 3806 Acked Read 4 Done Sent Done false
11956 3807 Acked Write 8 Done Sent Done false
11957 3808 Acked Read 4 Done Sent Done false
11958 3809 Acked Read 7 Done Sent Done false
11959 3810 Acked Read 1 Done Sent Done false
11960 3811 Acked Read 6 Done Sent Done false
11961 3812 Acked Write 4 Done Sent Done false
11962 3813 Acked Read 5 Done Sent Done false
11963 3814 Acked Read 8 Done Sent Done false
11964 3815 Acked Read 2 Done Sent Done false
11965 3816 Acked Read 8 Done Sent Done false
11966 3817 Acked Write 1 Done Sent Done false
11967 3818 Acked Flush 0 Done Sent Done false
11968 3819 Acked Write 7 Done Sent Done false
11969 3820 Acked Read 5 Done Sent Done false
11970 3821 Acked Read 2 Done Sent Done false
11971 3822 Acked Write 8 Done Sent Done false
11972 3823 Acked Flush 0 Done Sent Done false
11973 3824 Acked Write 4 Done Sent Done false
11974 3825 Acked Read 6 Done Sent Done false
11975 3826 Acked Read 4 Done Sent Done false
11976 3827 Acked Read 2 Done Sent Done false
11977 3828 Acked Write 2 Done Sent Done false
11978 3829 Acked Read 1 Done Sent Done false
11979 3830 Acked Write 1 Done Sent Done false
11980 3831 Acked Flush 0 Done Sent Done false
11981 3832 Acked Read 3 Done Sent Done false
11982 3833 Acked Flush 0 Done Sent Done false
11983 3834 Acked Write 8 Done Sent Done false
11984 3835 Acked Read 5 Done Sent Done false
11985 3836 Acked Read 6 Done Sent Done false
11986 3837 Acked Write 2 Done Sent Done false
11987 3838 Acked Write 2 Done Sent Done false
11988 3839 Acked Read 3 Done Sent Done false
11989 3840 Acked Write 5 Done Sent Done false
11990 3841 Acked Read 10 Done Sent Done false
11991 3842 Acked Write 5 Done Sent Done false
11992 3843 Acked Read 1 Done Sent Done false
11993 3844 Acked Flush 0 Done Sent Done false
11994 3845 Acked Read 1 Done Sent Done false
11995 3846 Acked Read 5 Done Sent Done false
11996 3847 Acked Write 5 Done Sent Done false
11997 3848 Acked Write 9 Done Sent Done false
11998 3849 Acked Write 8 Done Sent Done false
11999 3850 Acked Write 8 Done Sent Done false
12000 3851 Acked Flush 0 Done Sent Done false
12001 3852 Acked Read 2 Done Sent Done false
12002 3853 Acked Flush 0 Done Sent Done false
12003 3854 Acked Read 3 Done Sent Done false
12004 3855 Acked Read 9 Done Sent Done false
12005 3856 Acked Read 1 Done Sent Done false
12006 3857 Acked Flush 0 Done Sent Done false
12007 3858 Acked Flush 0 Done Sent Done false
12008 3859 Acked Read 5 Done Sent Done false
12009 3860 Acked Read 7 Done Sent Done false
12010 3861 Acked Write 7 Done Sent Done false
12011 3862 Acked Read 10 Done Sent Done false
12012 3863 Acked Write 9 Done Sent Done false
12013 3864 Acked Flush 0 Done Sent Done false
12014 3865 Acked Write 4 Done Sent Done false
12015 3866 Acked Write 3 Done Sent Done false
12016 3867 Acked Read 7 Done Sent Done false
12017 3868 Acked Read 10 Done Sent Done false
12018 3869 Acked Read 8 Done Sent Done false
12019 3870 Acked Read 9 Done Sent Done false
12020 3871 Acked Read 8 Done Sent Done false
12021 3872 Acked Write 6 Done Sent Done false
12022 3873 Acked Read 2 Done Sent Done false
12023 3874 Acked Flush 0 Done Sent Done false
12024 3875 Acked Write 6 Done Sent Done false
12025 3876 Acked Read 8 Done Sent Done false
12026 3877 Acked Write 7 Done Sent Done false
12027 3878 Acked Write 5 Done Sent Done false
12028 3879 Acked Read 8 Done Sent Done false
12029 3880 Acked Read 2 Done Sent Done false
12030 3881 Acked Write 10 Done Sent Done false
12031 3882 Acked Read 9 Done Sent Done false
12032 3883 Acked Write 1 Done Sent Done false
12033 3884 Acked Read 10 Done Sent Done false
12034 3885 Acked Read 3 Done Sent Done false
12035 3886 Acked Read 3 Done Sent Done false
12036 3887 Acked Read 2 Done Sent Done false
12037 3888 Acked Flush 0 Done Sent Done false
12038 3889 Acked Write 1 Done Sent Done false
12039 3890 Acked Flush 0 Done Sent Done false
12040 3891 Acked Write 7 Done Sent Done false
12041 3892 Acked Read 8 Done Sent Done false
12042 3893 Acked Read 1 Done Sent Done false
12043 3894 Acked Write 2 Done Sent Done false
12044 3895 Acked Read 2 Done Sent Done false
12045 3896 Acked Read 1 Done Sent Done false
12046 3897 Acked Write 9 Done Sent Done false
12047 3898 Acked Read 7 Done Sent Done false
12048 3899 Acked Read 6 Done Sent Done false
12049 3900 Acked Read 4 Done Sent Done false
12050 3901 Acked Flush 0 Done Sent Done false
12051 3902 Acked Write 1 Done Sent Done false
12052 3903 Acked Flush 0 Done Sent Done false
12053 3904 Acked Write 3 Done Sent Done false
12054 3905 Acked Read 5 Done Sent Done false
12055 3906 Acked Flush 0 Done Sent Done false
12056 3907 Acked Read 10 Done Sent Done false
12057 3908 Acked Read 3 Done Sent Done false
12058 3909 Acked Read 4 Done Sent Done false
12059 3910 Acked Read 4 Done Sent Done false
12060 3911 Acked Write 9 Done Sent Done false
12061 3912 Acked Write 7 Done Sent Done false
12062 3913 Acked Read 2 Done Sent Done false
12063 3914 Acked Read 1 Done Sent Done false
12064 3915 Acked Write 5 Done Sent Done false
12065 3916 Acked Read 10 Done Sent Done false
12066 3917 Acked Flush 0 Done Sent Done false
12067 3918 Acked Write 10 Done Sent Done false
12068 3919 Acked Read 4 Done Sent Done false
12069 3920 Acked Write 9 Done Sent Done false
12070 3921 Acked Flush 0 Done Sent Done false
12071 3922 Acked Read 4 Done Sent Done false
12072 3923 Acked Read 2 Done Sent Done false
12073 3924 Acked Write 4 Done Sent Done false
12074 3925 Acked Flush 0 Done Sent Done false
12075 3926 Acked Write 4 Done Sent Done false
12076 3927 Acked Write 5 Done Sent Done false
12077 3928 Acked Write 9 Done Sent Done false
12078 3929 Acked Read 1 Done Sent Done false
12079 3930 Acked Read 9 Done Sent Done false
12080 3931 Acked Read 4 Done Sent Done false
12081 3932 Acked Read 5 Done Sent Done false
12082 3933 Acked Write 10 Done Sent Done false
12083 3934 Acked Read 7 Done Sent Done false
12084 3935 Acked Read 10 Done Sent Done false
12085 3936 Acked Read 8 Done Sent Done false
12086 3937 Acked Read 5 Done Sent Done false
12087 3938 Acked Read 6 Done Sent Done false
12088 3939 Acked Write 1 Done Sent Done false
12089 3940 Acked Read 6 Done Sent Done false
12090 3941 Acked Write 8 Done Sent Done false
12091 3942 Acked Write 4 Done Sent Done false
12092 3943 Acked Write 1 Done Sent Done false
12093 3944 Acked Read 9 Done Sent Done false
12094 3945 Acked Write 8 Done Sent Done false
12095 3946 Acked Write 5 Done Sent Done false
12096 3947 Acked Read 6 Done Sent Done false
12097 3948 Acked Write 8 Done Sent Done false
12098 3949 Acked Write 5 Done Sent Done false
12099 3950 Acked Read 6 Done Sent Done false
12100 3951 Acked Read 3 Done Sent Done false
12101 3952 Acked Write 4 Done Sent Done false
12102 3953 Acked Read 4 Done Sent Done false
12103 3954 Acked Read 9 Done Sent Done false
12104 3955 Acked Write 9 Done Sent Done false
12105 3956 Acked Write 1 Done Sent Done false
12106 3957 Acked Flush 0 Done Sent Done false
12107 3958 Acked Flush 0 Done Sent Done false
12108 3959 Acked Write 6 Done Sent Done false
12109 3960 Acked Write 3 Done Sent Done false
12110 3961 Acked Flush 0 Done Sent Done false
12111 3962 Acked Write 3 Done Sent Done false
12112 3963 Acked Read 5 Done Sent Done false
12113 3964 Acked Write 9 Done Sent Done false
12114 3965 Acked Read 10 Done Sent Done false
12115 3966 Acked Read 1 Done Sent Done false
12116 3967 Acked Write 9 Done Sent Done false
12117 3968 Acked Read 3 Done Sent Done false
12118 3969 Acked Write 10 Done Sent Done false
12119 3970 Acked Read 6 Done Sent Done false
12120 3971 Acked Flush 0 Done Sent Done false
12121 3972 Acked Read 4 Done Sent Done false
12122 3973 Acked Read 3 Done Sent Done false
12123 3974 Acked Read 6 Done Sent Done false
12124 3975 Acked Read 9 Done Sent Done false
12125 3976 Acked Read 3 Done Sent Done false
12126 3977 Acked Flush 0 Done Sent Done false
12127 3978 Acked Read 10 Done Sent Done false
12128 3979 Acked Write 5 Done Sent Done false
12129 3980 Acked Read 4 Done Sent Done false
12130 3981 Acked Read 9 Done Sent Done false
12131 3982 Acked Write 10 Done Sent Done false
12132 3983 Acked Flush 0 Done Sent Done false
12133 3984 Acked Read 9 Done Sent Done false
12134 3985 Acked Read 10 Done Sent Done false
12135 3986 Acked Write 8 Done Sent Done false
12136 3987 Acked Read 5 Done Sent Done false
12137 3988 Acked Read 4 Done Sent Done false
12138 3989 Acked Read 8 Done Sent Done false
12139 3990 Acked Write 4 Done Sent Done false
12140 3991 Acked Write 2 Done Sent Done false
12141 3992 Acked Read 4 Done Sent Done false
12142 3993 Acked Write 2 Done Sent Done false
12143 3994 Acked Write 8 Done Sent Done false
12144 3995 Acked Write 4 Done Sent Done false
12145 3996 Acked Write 9 Done Sent Done false
12146 3997 Acked Read 9 Done Sent Done false
12147 3998 Acked Write 7 Done Sent Done false
12148 3999 Acked Read 3 Done Sent Done false
12149 4000 Acked Read 1 Done Sent Done false
12150 4001 Acked Write 2 Done Sent Done false
12151 4002 Acked Write 9 Done Sent Done false
12152 4003 Acked Write 1 Done Sent Done false
12153 4004 Acked Write 8 Done Sent Done false
12154 4005 Acked Write 10 Done Sent Done false
12155 4006 Acked Read 9 Done Sent Done false
12156 4007 Acked Read 7 Done Sent Done false
12157 4008 Acked Read 6 Done Sent Done false
12158 4009 Acked Read 9 Done Sent Done false
12159 4010 Acked Write 1 Done Sent Done false
12160 4011 Acked Write 5 Done Sent Done false
12161 4012 Acked Read 2 Done Sent Done false
12162 4013 Acked Read 5 Done Sent Done false
12163 4014 Acked Write 4 Done Sent Done false
12164 4015 Acked Read 2 Done Sent Done false
12165 4016 Acked Write 9 Done Sent Done false
12166 4017 Acked Flush 0 Done Sent Done false
12167 4018 Acked Write 5 Done Sent Done false
12168 4019 Acked Flush 0 Done Sent Done false
12169 STATES DS:0 DS:1 DS:2 TOTAL
12170 Sent 0 302 0 302
12171 Done 302 0 302 604
12172 Skipped 0 0 0 0
12173 Error 0 0 0 0
12174 Last Flush: 4019 3717 4019
12175 Downstairs last five completed: 3717 3716 3715 3714 3713
12176 Upstairs last five completed: 4019 4018 4017 4016 4015
12177 CLIENT: Up:0 ds:302 act:2
12178 ----------------------------------------------------------------
12179 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12180 States: Active Offline Active
12181 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12182 3718 Acked Read 8 Done Sent Done false
12183 3719 Acked Write 2 Done Sent Done false
12184 3720 Acked Write 2 Done Sent Done false
12185 3721 Acked Read 7 Done Sent Done false
12186 3722 Acked Read 4 Done Sent Done false
12187 3723 Acked Read 3 Done Sent Done false
12188 3724 Acked Read 5 Done Sent Done false
12189 3725 Acked Write 8 Done Sent Done false
12190 3726 Acked Read 7 Done Sent Done false
12191 3727 Acked Read 4 Done Sent Done false
12192 3728 Acked Write 10 Done Sent Done false
12193 3729 Acked Write 8 Done Sent Done false
12194 3730 Acked Write 3 Done Sent Done false
12195 3731 Acked Read 10 Done Sent Done false
12196 3732 Acked Write 2 Done Sent Done false
12197 3733 Acked Read 1 Done Sent Done false
12198 3734 Acked Write 8 Done Sent Done false
12199 3735 Acked Read 9 Done Sent Done false
12200 3736 Acked Read 10 Done Sent Done false
12201 3737 Acked Read 1 Done Sent Done false
12202 3738 Acked Read 9 Done Sent Done false
12203 3739 Acked Write 1 Done Sent Done false
12204 3740 Acked Read 3 Done Sent Done false
12205 3741 Acked Flush 0 Done Sent Done false
12206 3742 Acked Read 2 Done Sent Done false
12207 3743 Acked Flush 0 Done Sent Done false
12208 3744 Acked Read 9 Done Sent Done false
12209 3745 Acked Read 1 Done Sent Done false
12210 3746 Acked Write 1 Done Sent Done false
12211 3747 Acked Read 4 Done Sent Done false
12212 3748 Acked Read 8 Done Sent Done false
12213 3749 Acked Write 4 Done Sent Done false
12214 3750 Acked Read 5 Done Sent Done false
12215 3751 Acked Write 3 Done Sent Done false
12216 3752 Acked Write 3 Done Sent Done false
12217 3753 Acked Write 4 Done Sent Done false
12218 3754 Acked Read 9 Done Sent Done false
12219 3755 Acked Read 2 Done Sent Done false
12220 3756 Acked Write 8 Done Sent Done false
12221 3757 Acked Read 3 Done Sent Done false
12222 3758 Acked Read 10 Done Sent Done false
12223 3759 Acked Read 7 Done Sent Done false
12224 3760 Acked Read 2 Done Sent Done false
12225 3761 Acked Read 2 Done Sent Done false
12226 3762 Acked Flush 0 Done Sent Done false
12227 3763 Acked Read 9 Done Sent Done false
12228 3764 Acked Write 9 Done Sent Done false
12229 3765 Acked Flush 0 Done Sent Done false
12230 3766 Acked Read 7 Done Sent Done false
12231 3767 Acked Write 7 Done Sent Done false
12232 3768 Acked Write 9 Done Sent Done false
12233 3769 Acked Write 3 Done Sent Done false
12234 3770 Acked Flush 0 Done Sent Done false
12235 3771 Acked Read 3 Done Sent Done false
12236 3772 Acked Write 7 Done Sent Done false
12237 3773 Acked Read 7 Done Sent Done false
12238 3774 Acked Read 4 Done Sent Done false
12239 3775 Acked Read 3 Done Sent Done false
12240 3776 Acked Write 7 Done Sent Done false
12241 3777 Acked Write 1 Done Sent Done false
12242 3778 Acked Write 3 Done Sent Done false
12243 3779 Acked Flush 0 Done Sent Done false
12244 3780 Acked Write 4 Done Sent Done false
12245 3781 Acked Flush 0 Done Sent Done false
12246 3782 Acked Write 3 Done Sent Done false
12247 3783 Acked Write 8 Done Sent Done false
12248 3784 Acked Write 5 Done Sent Done false
12249 3785 Acked Read 4 Done Sent Done false
12250 3786 Acked Read 8 Done Sent Done false
12251 3787 Acked Read 1 Done Sent Done false
12252 3788 Acked Flush 0 Done Sent Done false
12253 3789 Acked Write 8 Done Sent Done false
12254 3790 Acked Flush 0 Done Sent Done false
12255 3791 Acked Write 2 Done Sent Done false
12256 3792 Acked Write 4 Done Sent Done false
12257 3793 Acked Write 1 Done Sent Done false
12258 3794 Acked Write 5 Done Sent Done false
12259 3795 Acked Read 3 Done Sent Done false
12260 3796 Acked Read 3 Done Sent Done false
12261 3797 Acked Read 1 Done Sent Done false
12262 3798 Acked Write 6 Done Sent Done false
12263 3799 Acked Write 3 Done Sent Done false
12264 3800 Acked Read 7 Done Sent Done false
12265 3801 Acked Read 3 Done Sent Done false
12266 3802 Acked Write 2 Done Sent Done false
12267 3803 Acked Read 7 Done Sent Done false
12268 3804 Acked Write 5 Done Sent Done false
12269 3805 Acked Write 5 Done Sent Done false
12270 3806 Acked Read 4 Done Sent Done false
12271 3807 Acked Write 8 Done Sent Done false
12272 3808 Acked Read 4 Done Sent Done false
12273 3809 Acked Read 7 Done Sent Done false
12274 3810 Acked Read 1 Done Sent Done false
12275 3811 Acked Read 6 Done Sent Done false
12276 3812 Acked Write 4 Done Sent Done false
12277 3813 Acked Read 5 Done Sent Done false
12278 3814 Acked Read 8 Done Sent Done false
12279 3815 Acked Read 2 Done Sent Done false
12280 3816 Acked Read 8 Done Sent Done false
12281 3817 Acked Write 1 Done Sent Done false
12282 3818 Acked Flush 0 Done Sent Done false
12283 3819 Acked Write 7 Done Sent Done false
12284 3820 Acked Read 5 Done Sent Done false
12285 3821 Acked Read 2 Done Sent Done false
12286 3822 Acked Write 8 Done Sent Done false
12287 3823 Acked Flush 0 Done Sent Done false
12288 3824 Acked Write 4 Done Sent Done false
12289 3825 Acked Read 6 Done Sent Done false
12290 3826 Acked Read 4 Done Sent Done false
12291 3827 Acked Read 2 Done Sent Done false
12292 3828 Acked Write 2 Done Sent Done false
12293 3829 Acked Read 1 Done Sent Done false
12294 3830 Acked Write 1 Done Sent Done false
12295 3831 Acked Flush 0 Done Sent Done false
12296 3832 Acked Read 3 Done Sent Done false
12297 3833 Acked Flush 0 Done Sent Done false
12298 3834 Acked Write 8 Done Sent Done false
12299 3835 Acked Read 5 Done Sent Done false
12300 3836 Acked Read 6 Done Sent Done false
12301 3837 Acked Write 2 Done Sent Done false
12302 3838 Acked Write 2 Done Sent Done false
12303 3839 Acked Read 3 Done Sent Done false
12304 3840 Acked Write 5 Done Sent Done false
12305 3841 Acked Read 10 Done Sent Done false
12306 3842 Acked Write 5 Done Sent Done false
12307 3843 Acked Read 1 Done Sent Done false
12308 3844 Acked Flush 0 Done Sent Done false
12309 3845 Acked Read 1 Done Sent Done false
12310 3846 Acked Read 5 Done Sent Done false
12311 3847 Acked Write 5 Done Sent Done false
12312 3848 Acked Write 9 Done Sent Done false
12313 3849 Acked Write 8 Done Sent Done false
12314 3850 Acked Write 8 Done Sent Done false
12315 3851 Acked Flush 0 Done Sent Done false
12316 3852 Acked Read 2 Done Sent Done false
12317 3853 Acked Flush 0 Done Sent Done false
12318 3854 Acked Read 3 Done Sent Done false
12319 3855 Acked Read 9 Done Sent Done false
12320 3856 Acked Read 1 Done Sent Done false
12321 3857 Acked Flush 0 Done Sent Done false
12322 3858 Acked Flush 0 Done Sent Done false
12323 3859 Acked Read 5 Done Sent Done false
12324 3860 Acked Read 7 Done Sent Done false
12325 3861 Acked Write 7 Done Sent Done false
12326 3862 Acked Read 10 Done Sent Done false
12327 3863 Acked Write 9 Done Sent Done false
12328 3864 Acked Flush 0 Done Sent Done false
12329 3865 Acked Write 4 Done Sent Done false
12330 3866 Acked Write 3 Done Sent Done false
12331 3867 Acked Read 7 Done Sent Done false
12332 3868 Acked Read 10 Done Sent Done false
12333 3869 Acked Read 8 Done Sent Done false
12334 3870 Acked Read 9 Done Sent Done false
12335 3871 Acked Read 8 Done Sent Done false
12336 3872 Acked Write 6 Done Sent Done false
12337 3873 Acked Read 2 Done Sent Done false
12338 3874 Acked Flush 0 Done Sent Done false
12339 3875 Acked Write 6 Done Sent Done false
12340 3876 Acked Read 8 Done Sent Done false
12341 3877 Acked Write 7 Done Sent Done false
12342 3878 Acked Write 5 Done Sent Done false
12343 3879 Acked Read 8 Done Sent Done false
12344 3880 Acked Read 2 Done Sent Done false
12345 3881 Acked Write 10 Done Sent Done false
12346 3882 Acked Read 9 Done Sent Done false
12347 3883 Acked Write 1 Done Sent Done false
12348 3884 Acked Read 10 Done Sent Done false
12349 3885 Acked Read 3 Done Sent Done false
12350 3886 Acked Read 3 Done Sent Done false
12351 3887 Acked Read 2 Done Sent Done false
12352 3888 Acked Flush 0 Done Sent Done false
12353 3889 Acked Write 1 Done Sent Done false
12354 3890 Acked Flush 0 Done Sent Done false
12355 3891 Acked Write 7 Done Sent Done false
12356 3892 Acked Read 8 Done Sent Done false
12357 3893 Acked Read 1 Done Sent Done false
12358 3894 Acked Write 2 Done Sent Done false
12359 3895 Acked Read 2 Done Sent Done false
12360 3896 Acked Read 1 Done Sent Done false
12361 3897 Acked Write 9 Done Sent Done false
12362 3898 Acked Read 7 Done Sent Done false
12363 3899 Acked Read 6 Done Sent Done false
12364 3900 Acked Read 4 Done Sent Done false
12365 3901 Acked Flush 0 Done Sent Done false
12366 3902 Acked Write 1 Done Sent Done false
12367 3903 Acked Flush 0 Done Sent Done false
12368 3904 Acked Write 3 Done Sent Done false
12369 3905 Acked Read 5 Done Sent Done false
12370 3906 Acked Flush 0 Done Sent Done false
12371 3907 Acked Read 10 Done Sent Done false
12372 3908 Acked Read 3 Done Sent Done false
12373 3909 Acked Read 4 Done Sent Done false
12374 3910 Acked Read 4 Done Sent Done false
12375 3911 Acked Write 9 Done Sent Done false
12376 3912 Acked Write 7 Done Sent Done false
12377 3913 Acked Read 2 Done Sent Done false
12378 3914 Acked Read 1 Done Sent Done false
12379 3915 Acked Write 5 Done Sent Done false
12380 3916 Acked Read 10 Done Sent Done false
12381 3917 Acked Flush 0 Done Sent Done false
12382 3918 Acked Write 10 Done Sent Done false
12383 3919 Acked Read 4 Done Sent Done false
12384 3920 Acked Write 9 Done Sent Done false
12385 3921 Acked Flush 0 Done Sent Done false
12386 3922 Acked Read 4 Done Sent Done false
12387 3923 Acked Read 2 Done Sent Done false
12388 3924 Acked Write 4 Done Sent Done false
12389 3925 Acked Flush 0 Done Sent Done false
12390 3926 Acked Write 4 Done Sent Done false
12391 3927 Acked Write 5 Done Sent Done false
12392 3928 Acked Write 9 Done Sent Done false
12393 3929 Acked Read 1 Done Sent Done false
12394 3930 Acked Read 9 Done Sent Done false
12395 3931 Acked Read 4 Done Sent Done false
12396 3932 Acked Read 5 Done Sent Done false
12397 3933 Acked Write 10 Done Sent Done false
12398 3934 Acked Read 7 Done Sent Done false
12399 3935 Acked Read 10 Done Sent Done false
12400 3936 Acked Read 8 Done Sent Done false
12401 3937 Acked Read 5 Done Sent Done false
12402 3938 Acked Read 6 Done Sent Done false
12403 3939 Acked Write 1 Done Sent Done false
12404 3940 Acked Read 6 Done Sent Done false
12405 3941 Acked Write 8 Done Sent Done false
12406 3942 Acked Write 4 Done Sent Done false
12407 3943 Acked Write 1 Done Sent Done false
12408 3944 Acked Read 9 Done Sent Done false
12409 3945 Acked Write 8 Done Sent Done false
12410 3946 Acked Write 5 Done Sent Done false
12411 3947 Acked Read 6 Done Sent Done false
12412 3948 Acked Write 8 Done Sent Done false
12413 3949 Acked Write 5 Done Sent Done false
12414 3950 Acked Read 6 Done Sent Done false
12415 3951 Acked Read 3 Done Sent Done false
12416 3952 Acked Write 4 Done Sent Done false
12417 3953 Acked Read 4 Done Sent Done false
12418 3954 Acked Read 9 Done Sent Done false
12419 3955 Acked Write 9 Done Sent Done false
12420 3956 Acked Write 1 Done Sent Done false
12421 3957 Acked Flush 0 Done Sent Done false
12422 3958 Acked Flush 0 Done Sent Done false
12423 3959 Acked Write 6 Done Sent Done false
12424 3960 Acked Write 3 Done Sent Done false
12425 3961 Acked Flush 0 Done Sent Done false
12426 3962 Acked Write 3 Done Sent Done false
12427 3963 Acked Read 5 Done Sent Done false
12428 3964 Acked Write 9 Done Sent Done false
12429 3965 Acked Read 10 Done Sent Done false
12430 3966 Acked Read 1 Done Sent Done false
12431 3967 Acked Write 9 Done Sent Done false
12432 3968 Acked Read 3 Done Sent Done false
12433 3969 Acked Write 10 Done Sent Done false
12434 3970 Acked Read 6 Done Sent Done false
12435 3971 Acked Flush 0 Done Sent Done false
12436 3972 Acked Read 4 Done Sent Done false
12437 3973 Acked Read 3 Done Sent Done false
12438 3974 Acked Read 6 Done Sent Done false
12439 3975 Acked Read 9 Done Sent Done false
12440 3976 Acked Read 3 Done Sent Done false
12441 3977 Acked Flush 0 Done Sent Done false
12442 3978 Acked Read 10 Done Sent Done false
12443 3979 Acked Write 5 Done Sent Done false
12444 3980 Acked Read 4 Done Sent Done false
12445 3981 Acked Read 9 Done Sent Done false
12446 3982 Acked Write 10 Done Sent Done false
12447 3983 Acked Flush 0 Done Sent Done false
12448 3984 Acked Read 9 Done Sent Done false
12449 3985 Acked Read 10 Done Sent Done false
12450 3986 Acked Write 8 Done Sent Done false
12451 3987 Acked Read 5 Done Sent Done false
12452 3988 Acked Read 4 Done Sent Done false
12453 3989 Acked Read 8 Done Sent Done false
12454 3990 Acked Write 4 Done Sent Done false
12455 3991 Acked Write 2 Done Sent Done false
12456 3992 Acked Read 4 Done Sent Done false
12457 3993 Acked Write 2 Done Sent Done false
12458 3994 Acked Write 8 Done Sent Done false
12459 3995 Acked Write 4 Done Sent Done false
12460 3996 Acked Write 9 Done Sent Done false
12461 3997 Acked Read 9 Done Sent Done false
12462 3998 Acked Write 7 Done Sent Done false
12463 3999 Acked Read 3 Done Sent Done false
12464 4000 Acked Read 1 Done Sent Done false
12465 4001 Acked Write 2 Done Sent Done false
12466 4002 Acked Write 9 Done Sent Done false
12467 4003 Acked Write 1 Done Sent Done false
12468 4004 Acked Write 8 Done Sent Done false
12469 4005 Acked Write 10 Done Sent Done false
12470 4006 Acked Read 9 Done Sent Done false
12471 4007 Acked Read 7 Done Sent Done false
12472 4008 Acked Read 6 Done Sent Done false
12473 4009 Acked Read 9 Done Sent Done false
12474 4010 Acked Write 1 Done Sent Done false
12475 4011 Acked Write 5 Done Sent Done false
12476 4012 Acked Read 2 Done Sent Done false
12477 4013 Acked Read 5 Done Sent Done false
12478 4014 Acked Write 4 Done Sent Done false
12479 4015 Acked Read 2 Done Sent Done false
12480 4016 Acked Write 9 Done Sent Done false
12481 4017 Acked Flush 0 Done Sent Done false
12482 4018 Acked Write 5 Done Sent Done false
12483 4019 Acked Flush 0 Done Sent Done false
12484 STATES DS:0 DS:1 DS:2 TOTAL
12485 Sent 0 302 0 302
12486 Done 302 0 302 604
12487 Skipped 0 0 0 0
12488 Error 0 0 0 0
12489 Last Flush: 4019 3717 4019
12490 Downstairs last five completed: 3717 3716 3715 3714 3713
12491 Upstairs last five completed: 4019 4018 4017 4016 4015
12492 CLIENT: Up:0 ds:302 act:2
124932024-12-11T23:43:08.737ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
124942024-12-11T23:43:08.738ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54650) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
124952024-12-11T23:43:08.739ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
124962024-12-11T23:43:08.739ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
124972024-12-11T23:43:08.739ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
124982024-12-11T23:43:08.739ZINFOcrucible: send last flush ID to this DS: 3717 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
124992024-12-11T23:43:08.739ZINFOcrucible: Replied this last flush ID: 3717 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125002024-12-11T23:43:08.739ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125012024-12-11T23:43:08.740ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125022024-12-11T23:43:08.740ZINFOcrucible: [1] client re-new 302 jobs since flush 3717 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125032024-12-11T23:43:08.740ZINFOcrucible: [1] Replayed 302 jobs since flush: 3717 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
12504 ----------------------------------------------------------------
12505 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12506 Downstairs last five completed: 4019 4018 4017 4016 4015
12507 Upstairs last five completed: 4019 4018 4017 4016 4015
12508 CLIENT: Up:0 ds:0 act:3
12509 Replay: All jobs finished
12510 Main task has work to do, go find it
12511 got dsc Stop(1)
12512 stop 1
12513 Dec 11 23:43:08.982 INFO request completed, [8820] Got stop action so:false kr:false
12514 latency_us: 359, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 4f4ce104-cdf4-47ce-9b2e-9e43b0f8a218, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
12515 Dec 11 23:43:08.983 INFO request completed, latency_us: 125, response_code: 200, uri: /state/cid/1, method: GET, req_id: e6888024-1288-48f8-9f2d-d643abe542ae, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
125162024-12-11T23:43:08.983ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125172024-12-11T23:43:08.983ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125182024-12-11T23:43:08.983ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125192024-12-11T23:43:08.983ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125202024-12-11T23:43:08.983ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
125212024-12-11T23:43:08.983ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
12522 [8820] Exited with: ExitStatus(unix_wait_status(9))
12523 [8820][1] reports Exit
12524 Dec 11 23:43:12.985 INFO request completed, latency_us: 202, response_code: 200, uri: /state/cid/1, method: GET, req_id: 66bc7c20-e7b8-415c-a79d-918a4340102c, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
12525 001/300 Read block 493 len 24576
12526 002/300 Read block 1945 len 8192
12527 003/300 Write block 973 len 4096 data: 4
12528 004/300 Write block 778 len 8192 data: 2 2
12529 005/300 Read block 2441 len 16384
12530 006/300 Read block 2187 len 32768
12531 007/300 Flush
12532 008/300 Flush
12533 009/300 Read block 1443 len 20480
12534 010/300 Write block 2827 len 8192 data: 3 4
12535 011/300 Write block 4117 len 24576 data: 3 3 3 2 2 3
12536 012/300 Write block 3687 len 32768 data: 3 4 4 4 4 3 3 4
12537 013/300 Read block 3024 len 28672
12538 014/300 Flush
12539 015/300 Read block 1946 len 16384
12540 016/300 Write block 3808 len 28672 data: 3 3 3 3 4 3 4
12541 017/300 Write block 4017 len 16384 data: 5 5 4 4
12542 018/300 Write block 4172 len 20480 data: 6 3 3 4 5
12543 019/300 Read block 531 len 4096
12544 020/300 Read block 3647 len 8192
12545 021/300 Write block 2 len 4096 data: 2
12546 022/300 Write block 59 len 16384 data: 3 3 3 3
12547 023/300 Write block 1807 len 28672 data: 3 2 2 2 2 2 2
12548 024/300 Read block 281 len 8192
12549 025/300 Write block 4583 len 36864 data: 3 3 4 4 4 4 3 3 3
12550 026/300 Write block 1976 len 28672 data: 3 3 3 4 4 6 6
12551 027/300 Read block 2210 len 8192
12552 028/300 Read block 3943 len 40960
12553 029/300 Read block 1985 len 32768
12554 030/300 Flush
12555 031/300 Read block 3737 len 16384
12556 032/300 Write block 3489 len 4096 data: 3
12557 033/300 Flush
12558 034/300 Write block 2915 len 40960 data: 3 3 3 2 2 2 2 2 2 2
12559 035/300 Flush
12560 036/300 Read block 3764 len 24576
12561 037/300 Read block 1572 len 24576
12562 038/300 Read block 2231 len 20480
12563 039/300 Read block 2485 len 28672
12564 040/300 Write block 2990 len 8192 data: 2 2
12565 041/300 Read block 869 len 36864
12566 042/300 Write block 3801 len 20480 data: 4 4 4 3 2
12567 043/300 Write block 1882 len 8192 data: 4 4
12568 044/300 Read block 2085 len 20480
12569 045/300 Write block 2992 len 32768 data: 3 3 3 3 6 7 8 7
12570 046/300 Write block 4899 len 32768 data: 4 4 4 6 5 6 7 5
12571 047/300 Read block 3506 len 40960
12572 048/300 Write block 4987 len 24576 data: 3 2 2 3 3 3
12573 049/300 Read block 1956 len 16384
12574 050/300 Write block 285 len 8192 data: 2 3
12575 051/300 Write block 3540 len 40960 data: 3 3 3 3 3 3 3 3 2 3
12576 052/300 Read block 1787 len 36864
12577 053/300 Read block 1182 len 40960
12578 054/300 Write block 1288 len 40960 data: 4 4 5 5 4 4 5 4 4 4
12579 055/300 Read block 3568 len 24576
12580 056/300 Write block 4370 len 28672 data: 2 2 2 2 2 2 2
12581 057/300 Read block 1875 len 28672
12582 058/300 Read block 1090 len 12288
12583 059/300 Write block 3274 len 8192 data: 2 3
12584 060/300 Read block 285 len 8192
12585 061/300 Read block 90 len 32768
12586 062/300 Write block 888 len 20480 data: 3 3 4 4 3
12587 063/300 Read block 4400 len 32768
12588 064/300 Write block 2515 len 4096 data: 4
12589 065/300 Write block 2034 len 20480 data: 3 3 3 2 3
12590 066/300 Write block 4517 len 32768 data: 3 4 5 6 6 6 5 5
12591 067/300 Read block 1071 len 16384
12592 068/300 Read block 3396 len 40960
12593 069/300 Write block 3211 len 36864 data: 5 3 3 3 3 3 3 2 2
12594 070/300 Write block 18 len 20480 data: 2 2 3 3 3
12595 071/300 Read block 3467 len 8192
12596 072/300 Read block 4006 len 4096
12597 073/300 Read block 1273 len 40960
12598 074/300 Read block 827 len 36864
12599 075/300 Write block 1620 len 32768 data: 3 3 3 2 3 3 4 4
12600 076/300 Write block 3044 len 16384 data: 3 5 6 5
12601 077/300 Write block 1387 len 12288 data: 3 3 2
12602 078/300 Flush
12603 079/300 Read block 2319 len 12288
12604 080/300 Read block 3078 len 8192
12605 081/300 Read block 2095 len 12288
12606 082/300 Read block 3670 len 16384
12607 083/300 Read block 3571 len 12288
12608 084/300 Flush
12609 085/300 Read block 3556 len 24576
12610 086/300 Write block 2806 len 40960 data: 4 3 3 3 4 4 3 3 4 5
12611 087/300 Read block 3833 len 4096
12612 088/300 Read block 4844 len 40960
12613 089/300 Write block 1150 len 8192 data: 4 4
12614 090/300 Write block 2976 len 32768 data: 3 3 3 3 3 4 4 3
12615 091/300 Read block 2532 len 36864
12616 092/300 Read block 4321 len 12288
12617 093/300 Read block 754 len 24576
12618 094/300 Read block 1480 len 36864
12619 095/300 Write block 4213 len 28672 data: 3 4 3 3 3 4 4
12620 096/300 Write block 709 len 24576 data: 2 2 2 2 2 2
12621 097/300 Read block 3576 len 8192
12622 098/300 Read block 4664 len 16384
12623 099/300 Read block 4164 len 4096
12624 100/300 Flush
12625 101/300 Read block 11 len 4096
12626 102/300 Write block 3102 len 20480 data: 2 3 4 4 3
12627 103/300 Flush
12628 104/300 Write block 3565 len 16384 data: 2 2 2 2
12629 105/300 Read block 1929 len 24576
12630 106/300 Read block 4803 len 24576
12631 107/300 Write block 3090 len 24576 data: 5 4 5 6 6 5
12632 108/300 Read block 364 len 24576
12633 109/300 Write block 290 len 4096 data: 3
12634 110/300 Write block 3643 len 28672 data: 5 4 4 5 4 5 4
12635 111/300 Write block 3877 len 40960 data: 4 5 4 3 3 3 3 3 4 6
12636 112/300 Write block 2242 len 16384 data: 4 4 4 4
12637 113/300 Write block 2652 len 20480 data: 4 3 3 4 4
12638 114/300 Read block 3852 len 8192
12639 115/300 Read block 3009 len 24576
12640 116/300 Read block 2316 len 20480
12641 117/300 Write block 1161 len 4096 data: 4
12642 118/300 Write block 4347 len 16384 data: 5 6 4 3
12643 119/300 Flush
12644 120/300 Read block 3789 len 32768
12645 121/300 Write block 72 len 36864 data: 4 4 3 3 4 5 5 4 3
12646 122/300 Read block 4127 len 36864
12647 123/300 Write block 4693 len 24576 data: 3 3 3 3 3 2
12648 124/300 Read block 1942 len 24576
12649 125/300 Write block 2814 len 12288 data: 5 6 2
12650 126/300 Read block 4846 len 32768
12651 127/300 Read block 1208 len 24576
12652 128/300 Read block 2187 len 8192
12653 129/300 Read block 3820 len 20480
12654 130/300 Write block 3093 len 40960 data: 7 7 6 5 4 4 3 4 2 3
12655 131/300 Write block 1648 len 8192 data: 3 3
12656 132/300 Read block 926 len 40960
12657 133/300 Read block 2891 len 40960
12658 134/300 Read block 564 len 32768
12659 135/300 Write block 3456 len 8192 data: 2 2
12660 136/300 Write block 4394 len 36864 data: 3 3 2 2 3 4 4 4 3
12661 137/300 Flush
12662 138/300 Read block 2531 len 28672
12663 139/300 Read block 3941 len 4096
12664 140/300 Write block 4932 len 24576 data: 4 3 3 3 3 4
12665 141/300 Read block 4836 len 40960
12666 142/300 Write block 2039 len 28672 data: 3 3 4 4 4 4 4
12667 143/300 Flush
12668 144/300 Read block 2513 len 8192
12669 145/300 Write block 1092 len 16384 data: 7 5 5 5
12670 146/300 Read block 2359 len 20480
12671 147/300 Read block 200 len 12288
12672 148/300 Read block 4586 len 12288
12673 149/300 Write block 239 len 28672 data: 3 3 3 2 2 3 2
12674 150/300 Read block 479 len 24576
12675 151/300 Read block 4889 len 12288
12676 152/300 Write block 4477 len 12288 data: 2 3 3
12677 153/300 Read block 1697 len 8192
12678 154/300 Write block 4833 len 8192 data: 3 3
12679 155/300 Read block 4766 len 28672
12680 156/300 Write block 1457 len 20480 data: 4 3 3 3 3
12681 157/300 Write block 126 len 28672 data: 4 3 3 4 4 3 3
12682 158/300 Write block 2052 len 32768 data: 3 3 2 2 2 2 2 2
12683 159/300 Write block 3990 len 4096 data: 3
12684 160/300 Read block 4 len 28672
12685 161/300 Write block 185 len 24576 data: 4 4 4 3 3 2
12686 162/300 Write block 3960 len 32768 data: 6 6 7 6 6 6 5 5
12687 163/300 Flush
12688 164/300 Read block 2493 len 8192
12689 165/300 Read block 2812 len 28672
12690 166/300 Write block 2436 len 24576 data: 3 4 3 5 5 5
12691 167/300 Read block 2915 len 28672
12692 168/300 Read block 4475 len 20480
12693 169/300 Read block 2353 len 28672
12694 170/300 Flush
12695 171/300 Flush
12696 172/300 Read block 21 len 4096
12697 173/300 Read block 3582 len 20480
12698 174/300 Write block 811 len 32768 data: 4 3 3 3 2 2 2 3
12699 175/300 Read block 2823 len 8192
12700 176/300 Read block 1957 len 16384
12701 177/300 Read block 4783 len 4096
12702 178/300 Write block 4917 len 40960 data: 3 3 3 3 5 8 7 8 6 7
12703 179/300 Write block 1103 len 24576 data: 3 3 2 2 2 4
12704 180/300 Read block 1194 len 24576
12705 181/300 Write block 4980 len 16384 data: 4 4 4 4
12706 182/300 Read block 2577 len 8192
12707 183/300 Read block 2315 len 4096
12708 184/300 Flush
12709 185/300 Read block 2181 len 36864
12710 186/300 Write block 942 len 8192 data: 4 4
12711 187/300 Read block 4452 len 32768
12712 188/300 Read block 2822 len 16384
12713 189/300 Write block 4084 len 32768 data: 2 2 2 3 3 3 3 3
12714 190/300 Read block 1648 len 36864
12715 191/300 Write block 3629 len 28672 data: 4 4 4 3 3 3 2
12716 192/300 Write block 3174 len 32768 data: 3 3 3 3 3 3 3 3
12717 193/300 Flush
12718 194/300 Read block 2173 len 20480
12719 195/300 Read block 4924 len 4096
12720 196/300 Flush
12721 197/300 Read block 2127 len 36864
12722 198/300 Write block 4638 len 16384 data: 2 2 2 2
12723 199/300 Flush
12724 200/300 Read block 203 len 36864
12725 201/300 Write block 3549 len 40960 data: 4 3 3 3 3 3 2 4 4 3
12726 202/300 Read block 860 len 36864
12727 203/300 Read block 4471 len 16384
12728 204/300 Read block 340 len 4096
12729 205/300 Read block 910 len 12288
12730 206/300 Read block 1101 len 16384
12731 207/300 Write block 2461 len 12288 data: 3 3 3
12732 208/300 Write block 3400 len 4096 data: 2
12733 209/300 Write block 4378 len 12288 data: 4 3 4
12734 210/300 Read block 785 len 40960
12735 211/300 Write block 3527 len 24576 data: 4 4 4 4 3 3
12736 212/300 Read block 3167 len 20480
12737 213/300 Read block 2891 len 20480
12738 214/300 Read block 366 len 32768
12739 215/300 Write block 1021 len 28672 data: 2 2 2 2 2 2 2
12740 216/300 Write block 1602 len 8192 data: 2 3
12741 217/300 Flush
12742 218/300 Read block 1374 len 12288
12743 219/300 Read block 1971 len 4096
12744 220/300 Read block 2723 len 12288
12745 221/300 Read block 4239 len 4096
12746 222/300 Write block 3067 len 36864 data: 3 3 3 3 3 2 2 2 2
12747 223/300 Read block 1171 len 8192
12748 224/300 Read block 3310 len 12288
12749 225/300 Read block 4353 len 36864
12750 226/300 Write block 4929 len 28672 data: 7 6 5 5 4 4 4
12751 227/300 Read block 2219 len 36864
12752 228/300 Read block 879 len 20480
12753 229/300 Write block 1212 len 12288 data: 2 3 4
12754 230/300 Read block 631 len 4096
12755 231/300 Read block 621 len 16384
12756 232/300 Write block 2325 len 24576 data: 3 4 4 4 4 4
12757 233/300 Write block 3230 len 4096 data: 3
12758 234/300 Read block 155 len 12288
12759 235/300 Write block 586 len 8192 data: 3 3
12760 236/300 Read block 1888 len 24576
12761 237/300 Write block 2844 len 16384 data: 3 3 4 4
12762 238/300 Write block 1180 len 24576 data: 4 4 5 4 3 3
12763 239/300 Write block 2918 len 8192 data: 3 3
12764 240/300 Write block 826 len 32768 data: 3 3 3 3 2 4 3 4
12765 241/300 Write block 2234 len 16384 data: 4 4 4 3
12766 242/300 Read block 4313 len 8192
12767 243/300 Read block 3030 len 40960
12768 244/300 Read block 472 len 32768
12769 245/300 Read block 2960 len 20480
12770 246/300 Read block 2304 len 12288
12771 247/300 Write block 1112 len 40960 data: 4 4 4 4 5 4 4 4 4 3
12772 248/300 Read block 493 len 28672
12773 249/300 Read block 3357 len 28672
12774 250/300 Read block 659 len 36864
12775 251/300 Read block 4077 len 36864
12776 252/300 Write block 3139 len 40960 data: 3 3 4 4 4 3 2 2 2 2
12777 253/300 Write block 419 len 4096 data: 2
12778 254/300 Read block 1753 len 36864
12779 255/300 Read block 1086 len 20480
12780 256/300 Write block 2032 len 32768 data: 4 4 4 4 4 3 4 4
12781 257/300 Read block 2586 len 40960
12782 258/300 Write block 1432 len 32768 data: 2 2 2 4 4 3 4 6
12783 259/300 Read block 3947 len 28672
12784 260/300 Read block 1376 len 32768
12785 261/300 Write block 2135 len 20480 data: 2 2 2 2 2
12786 262/300 Read block 483 len 20480
12787 263/300 Read block 3803 len 16384
12788 264/300 Write block 3193 len 20480 data: 3 3 3 3 2
12789 265/300 Flush
12790 266/300 Write block 1842 len 4096 data: 2
12791 267/300 Read block 4185 len 16384
12792 268/300 Read block 2376 len 40960
12793 269/300 Read block 4865 len 28672
12794 270/300 Flush
12795 271/300 Read block 3541 len 20480
12796 272/300 Write block 172 len 20480 data: 2 2 4 4 3
12797 273/300 Read block 828 len 4096
12798 274/300 Write block 4121 len 20480 data: 3 4 3 4 4
12799 275/300 Write block 1625 len 36864 data: 4 5 5 4 5 5 5 4 4
12800 276/300 Read block 2760 len 8192
12801 277/300 Write block 3277 len 16384 data: 3 3 5 5
12802 278/300 Read block 1726 len 36864
12803 279/300 Write block 3011 len 32768 data: 3 3 3 3 2 2 4 4
12804 280/300 Flush
12805 281/300 Read block 3545 len 24576
12806 282/300 Write block 4717 len 28672 data: 4 3 3 3 3 3 3
12807 283/300 Write block 4488 len 40960 data: 2 2 2 2 3 4 4 5 5 6
12808 284/300 Read block 3898 len 28672
12809 285/300 Write block 4826 len 16384 data: 2 2 2 2
12810 286/300 Read block 1890 len 8192
12811 287/300 Write block 2931 len 16384 data: 6 6 6 6
12812 288/300 Read block 1817 len 40960
12813 289/300 Read block 3628 len 24576
12814 290/300 Write block 1446 len 12288 data: 6 5 3
12815 291/300 Read block 180 len 12288
12816 292/300 Read block 3402 len 4096
12817 293/300 Read block 1745 len 4096
12818 294/300 Write block 2609 len 12288 data: 4 6 6
12819 295/300 Read block 1926 len 32768
12820 296/300 Read block 2560 len 28672
12821 297/300 Read block 4437 len 12288
12822 298/300 Read block 1939 len 32768
12823 299/300 Flush
12824 300/300 Read block 1040 len 12288
12825 301/300 Read block 3330 len 8192
12826 Main task has work to do, go find it
12827 got dsc Start(1)
12828 start 1
12829 Dec 11 23:43:13.226 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/1, method: POST, req_id: fff733b8-a65a-40f1-bcac-f1a8508cb1c9, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
12830 Replay: started 1, returned:Ok(())
12831 ----------------------------------------------------------------
12832 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12833 States: Active Offline Active
12834 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12835 4020 Acked Read 6 Done Sent Done false
12836 4021 Acked Read 2 Done Sent Done false
12837 4022 Acked Write 1 Done Sent Done false
12838 4023 Acked Write 2 Done Sent Done false
12839 4024 Acked Read 4 Done Sent Done false
12840 4025 Acked Read 8 Done Sent Done false
12841 4026 Acked Flush 0 Done Sent Done false
12842 4027 Acked Flush 0 Done Sent Done false
12843 4028 Acked Read 5 Done Sent Done false
12844 4029 Acked Write 2 Done Sent Done false
12845 4030 Acked Write 6 Done Sent Done false
12846 4031 Acked Write 8 Done Sent Done false
12847 4032 Acked Read 7 Done Sent Done false
12848 4033 Acked Flush 0 Done Sent Done false
12849 4034 Acked Read 4 Done Sent Done false
12850 4035 Acked Write 7 Done Sent Done false
12851 4036 Acked Write 4 Done Sent Done false
12852 4037 Acked Write 5 Done Sent Done false
12853 4038 Acked Read 1 Done Sent Done false
12854 4039 Acked Read 2 Done Sent Done false
12855 4040 Acked Write 1 Done Sent Done false
12856 4041 Acked Write 4 Done Sent Done false
12857 4042 Acked Write 7 Done Sent Done false
12858 4043 Acked Read 2 Done Sent Done false
12859 4044 Acked Write 9 Done Sent Done false
12860 4045 Acked Write 7 Done Sent Done false
12861 4046 Acked Read 2 Done Sent Done false
12862 4047 Acked Read 10 Done Sent Done false
12863 4048 Acked Read 8 Done Sent Done false
12864 4049 Acked Flush 0 Done Sent Done false
12865 4050 Acked Read 4 Done Sent Done false
12866 4051 Acked Write 1 Done Sent Done false
12867 4052 Acked Flush 0 Done Sent Done false
12868 4053 Acked Write 10 Done Sent Done false
12869 4054 Acked Flush 0 Done Sent Done false
12870 4055 Acked Read 6 Done Sent Done false
12871 4056 Acked Read 6 Done Sent Done false
12872 4057 Acked Read 5 Done Sent Done false
12873 4058 Acked Read 7 Done Sent Done false
12874 4059 Acked Write 2 Done Sent Done false
12875 4060 Acked Read 9 Done Sent Done false
12876 4061 Acked Write 5 Done Sent Done false
12877 4062 Acked Write 2 Done Sent Done false
12878 4063 Acked Read 5 Done Sent Done false
12879 4064 Acked Write 8 Done Sent Done false
12880 4065 Acked Write 8 Done Sent Done false
12881 4066 Acked Read 10 Done Sent Done false
12882 4067 Acked Write 6 Done Sent Done false
12883 4068 Acked Read 4 Done Sent Done false
12884 4069 Acked Write 2 Done Sent Done false
12885 4070 Acked Write 10 Done Sent Done false
12886 4071 Acked Read 9 Done Sent Done false
12887 4072 Acked Read 10 Done Sent Done false
12888 4073 Acked Write 10 Done Sent Done false
12889 4074 Acked Read 6 Done Sent Done false
12890 4075 Acked Write 7 Done Sent Done false
12891 4076 Acked Read 7 Done Sent Done false
12892 4077 Acked Read 3 Done Sent Done false
12893 4078 Acked Write 2 Done Sent Done false
12894 4079 Acked Read 2 Done Sent Done false
12895 4080 Acked Read 8 Done Sent Done false
12896 4081 Acked Write 5 Done Sent Done false
12897 4082 Acked Read 8 Done Sent Done false
12898 4083 Acked Write 1 Done Sent Done false
12899 4084 Acked Write 5 Done Sent Done false
12900 4085 Acked Write 8 Done Sent Done false
12901 4086 Acked Read 4 Done Sent Done false
12902 4087 Acked Read 10 Done Sent Done false
12903 4088 Acked Write 9 Done Sent Done false
12904 4089 Acked Write 5 Done Sent Done false
12905 4090 Acked Read 2 Done Sent Done false
12906 4091 Acked Read 1 Done Sent Done false
12907 4092 Acked Read 10 Done Sent Done false
12908 4093 Acked Read 9 Done Sent Done false
12909 4094 Acked Write 8 Done Sent Done false
12910 4095 Acked Write 4 Done Sent Done false
12911 4096 Acked Write 3 Done Sent Done false
12912 4097 Acked Flush 0 Done Sent Done false
12913 4098 Acked Read 3 Done Sent Done false
12914 4099 Acked Read 2 Done Sent Done false
12915 4100 Acked Read 3 Done Sent Done false
12916 4101 Acked Read 4 Done Sent Done false
12917 4102 Acked Read 3 Done Sent Done false
12918 4103 Acked Flush 0 Done Sent Done false
12919 4104 Acked Read 6 Done Sent Done false
12920 4105 Acked Write 10 Done Sent Done false
12921 4106 Acked Read 1 Done Sent Done false
12922 4107 Acked Read 10 Done Sent Done false
12923 4108 Acked Write 2 Done Sent Done false
12924 4109 Acked Write 8 Done Sent Done false
12925 4110 Acked Read 9 Done Sent Done false
12926 4111 Acked Read 3 Done Sent Done false
12927 4112 Acked Read 6 Done Sent Done false
12928 4113 Acked Read 9 Done Sent Done false
12929 4114 Acked Write 7 Done Sent Done false
12930 4115 Acked Write 6 Done Sent Done false
12931 4116 Acked Read 2 Done Sent Done false
12932 4117 Acked Read 4 Done Sent Done false
12933 4118 Acked Read 1 Done Sent Done false
12934 4119 Acked Flush 0 Done Sent Done false
12935 4120 Acked Read 1 Done Sent Done false
12936 4121 Acked Write 5 Done Sent Done false
12937 4122 Acked Flush 0 Done Sent Done false
12938 4123 Acked Write 4 Done Sent Done false
12939 4124 Acked Read 6 Done Sent Done false
12940 4125 Acked Read 6 Done Sent Done false
12941 4126 Acked Write 6 Done Sent Done false
12942 4127 Acked Read 6 Done Sent Done false
12943 4128 Acked Write 1 Done Sent Done false
12944 4129 Acked Write 7 Done Sent Done false
12945 4130 Acked Write 10 Done Sent Done false
12946 4131 Acked Write 4 Done Sent Done false
12947 4132 Acked Write 5 Done Sent Done false
12948 4133 Acked Read 2 Done Sent Done false
12949 4134 Acked Read 6 Done Sent Done false
12950 4135 Acked Read 5 Done Sent Done false
12951 4136 Acked Write 1 Done Sent Done false
12952 4137 Acked Write 4 Done Sent Done false
12953 4138 Acked Flush 0 Done Sent Done false
12954 4139 Acked Read 8 Done Sent Done false
12955 4140 Acked Write 9 Done Sent Done false
12956 4141 Acked Read 9 Done Sent Done false
12957 4142 Acked Write 6 Done Sent Done false
12958 4143 Acked Read 6 Done Sent Done false
12959 4144 Acked Write 3 Done Sent Done false
12960 4145 Acked Read 8 Done Sent Done false
12961 4146 Acked Read 6 Done Sent Done false
12962 4147 Acked Read 2 Done Sent Done false
12963 4148 Acked Read 5 Done Sent Done false
12964 4149 Acked Write 10 Done Sent Done false
12965 4150 Acked Write 2 Done Sent Done false
12966 4151 Acked Read 10 Done Sent Done false
12967 4152 Acked Read 10 Done Sent Done false
12968 4153 Acked Read 8 Done Sent Done false
12969 4154 Acked Write 2 Done Sent Done false
12970 4155 Acked Write 9 Done Sent Done false
12971 4156 Acked Flush 0 Done Sent Done false
12972 4157 Acked Read 7 Done Sent Done false
12973 4158 Acked Read 1 Done Sent Done false
12974 4159 Acked Write 6 Done Sent Done false
12975 4160 Acked Read 10 Done Sent Done false
12976 4161 Acked Write 7 Done Sent Done false
12977 4162 Acked Flush 0 Done Sent Done false
12978 4163 Acked Read 2 Done Sent Done false
12979 4164 Acked Write 4 Done Sent Done false
12980 4165 Acked Read 5 Done Sent Done false
12981 4166 Acked Read 3 Done Sent Done false
12982 4167 Acked Read 3 Done Sent Done false
12983 4168 Acked Write 7 Done Sent Done false
12984 4169 Acked Read 6 Done Sent Done false
12985 4170 Acked Read 3 Done Sent Done false
12986 4171 Acked Write 3 Done Sent Done false
12987 4172 Acked Read 2 Done Sent Done false
12988 4173 Acked Write 2 Done Sent Done false
12989 4174 Acked Read 7 Done Sent Done false
12990 4175 Acked Write 5 Done Sent Done false
12991 4176 Acked Write 7 Done Sent Done false
12992 4177 Acked Write 8 Done Sent Done false
12993 4178 Acked Write 1 Done Sent Done false
12994 4179 Acked Read 7 Done Sent Done false
12995 4180 Acked Write 6 Done Sent Done false
12996 4181 Acked Write 8 Done Sent Done false
12997 4182 Acked Flush 0 Done Sent Done false
12998 4183 Acked Read 2 Done Sent Done false
12999 4184 Acked Read 7 Done Sent Done false
13000 4185 Acked Write 6 Done Sent Done false
13001 4186 Acked Read 7 Done Sent Done false
13002 4187 Acked Read 5 Done Sent Done false
13003 4188 Acked Read 7 Done Sent Done false
13004 4189 Acked Flush 0 Done Sent Done false
13005 4190 Acked Flush 0 Done Sent Done false
13006 4191 Acked Read 1 Done Sent Done false
13007 4192 Acked Read 5 Done Sent Done false
13008 4193 Acked Write 8 Done Sent Done false
13009 4194 Acked Read 2 Done Sent Done false
13010 4195 Acked Read 4 Done Sent Done false
13011 4196 Acked Read 1 Done Sent Done false
13012 4197 Acked Write 10 Done Sent Done false
13013 4198 Acked Write 6 Done Sent Done false
13014 4199 Acked Read 6 Done Sent Done false
13015 4200 Acked Write 4 Done Sent Done false
13016 4201 Acked Read 2 Done Sent Done false
13017 4202 Acked Read 1 Done Sent Done false
13018 4203 Acked Flush 0 Done Sent Done false
13019 4204 Acked Read 9 Done Sent Done false
13020 4205 Acked Write 2 Done Sent Done false
13021 4206 Acked Read 8 Done Sent Done false
13022 4207 Acked Read 4 Done Sent Done false
13023 4208 Acked Write 8 Done Sent Done false
13024 4209 Acked Read 9 Done Sent Done false
13025 4210 Acked Write 7 Done Sent Done false
13026 4211 Acked Write 8 Done Sent Done false
13027 4212 Acked Flush 0 Done Sent Done false
13028 4213 Acked Read 5 Done Sent Done false
13029 4214 Acked Read 1 Done Sent Done false
13030 4215 Acked Flush 0 Done Sent Done false
13031 4216 Acked Read 9 Done Sent Done false
13032 4217 Acked Write 4 Done Sent Done false
13033 4218 Acked Flush 0 Done Sent Done false
13034 4219 Acked Read 9 Done Sent Done false
13035 4220 Acked Write 10 Done Sent Done false
13036 4221 Acked Read 9 Done Sent Done false
13037 4222 Acked Read 4 Done Sent Done false
13038 4223 Acked Read 1 Done Sent Done false
13039 4224 Acked Read 3 Done Sent Done false
13040 4225 Acked Read 4 Done Sent Done false
13041 4226 Acked Write 3 Done Sent Done false
13042 4227 Acked Write 1 Done Sent Done false
13043 4228 Acked Write 3 Done Sent Done false
13044 4229 Acked Read 10 Done Sent Done false
13045 4230 Acked Write 6 Done Sent Done false
13046 4231 Acked Read 5 Done Sent Done false
13047 4232 Acked Read 5 Done Sent Done false
13048 4233 Acked Read 8 Done Sent Done false
13049 4234 Acked Write 7 Done Sent Done false
13050 4235 Acked Write 2 Done Sent Done false
13051 4236 Acked Flush 0 Done Sent Done false
13052 4237 Acked Read 3 Done Sent Done false
13053 4238 Acked Read 1 Done Sent Done false
13054 4239 Acked Read 3 Done Sent Done false
13055 4240 Acked Read 1 Done Sent Done false
13056 4241 Acked Write 9 Done Sent Done false
13057 4242 Acked Read 2 Done Sent Done false
13058 4243 Acked Read 3 Done Sent Done false
13059 4244 Acked Read 9 Done Sent Done false
13060 4245 Acked Write 7 Done Sent Done false
13061 4246 Acked Read 9 Done Sent Done false
13062 4247 Acked Read 5 Done Sent Done false
13063 4248 Acked Write 3 Done Sent Done false
13064 4249 Acked Read 1 Done Sent Done false
13065 4250 Acked Read 4 Done Sent Done false
13066 4251 Acked Write 6 Done Sent Done false
13067 4252 Acked Write 1 Done Sent Done false
13068 4253 Acked Read 3 Done Sent Done false
13069 4254 Acked Write 2 Done Sent Done false
13070 4255 Acked Read 6 Done Sent Done false
13071 4256 Acked Write 4 Done Sent Done false
13072 4257 Acked Write 6 Done Sent Done false
13073 4258 Acked Write 2 Done Sent Done false
13074 4259 Acked Write 8 Done Sent Done false
13075 4260 Acked Write 4 Done Sent Done false
13076 4261 Acked Read 2 Done Sent Done false
13077 4262 Acked Read 10 Done Sent Done false
13078 4263 Acked Read 8 Done Sent Done false
13079 4264 Acked Read 5 Done Sent Done false
13080 4265 Acked Read 3 Done Sent Done false
13081 4266 Acked Write 10 Done Sent Done false
13082 4267 Acked Read 7 Done Sent Done false
13083 4268 Acked Read 7 Done Sent Done false
13084 4269 Acked Read 9 Done Sent Done false
13085 4270 Acked Read 9 Done Sent Done false
13086 4271 Acked Write 10 Done Sent Done false
13087 4272 Acked Write 1 Done Sent Done false
13088 4273 Acked Read 9 Done Sent Done false
13089 4274 Acked Read 5 Done Sent Done false
13090 4275 Acked Write 8 Done Sent Done false
13091 4276 Acked Read 10 Done Sent Done false
13092 4277 Acked Write 8 Done Sent Done false
13093 4278 Acked Read 7 Done Sent Done false
13094 4279 Acked Read 8 Done Sent Done false
13095 4280 Acked Write 5 Done Sent Done false
13096 4281 Acked Read 5 Done Sent Done false
13097 4282 Acked Read 4 Done Sent Done false
13098 4283 Acked Write 5 Done Sent Done false
13099 4284 Acked Flush 0 Done Sent Done false
13100 4285 Acked Write 1 Done Sent Done false
13101 4286 Acked Read 4 Done Sent Done false
13102 4287 Acked Read 10 Done Sent Done false
13103 4288 Acked Read 7 Done Sent Done false
13104 4289 Acked Flush 0 Done Sent Done false
13105 4290 Acked Read 5 Done Sent Done false
13106 4291 Acked Write 5 Done Sent Done false
13107 4292 Acked Read 1 Done Sent Done false
13108 4293 Acked Write 5 Done Sent Done false
13109 4294 Acked Write 9 Done Sent Done false
13110 4295 Acked Read 2 Done Sent Done false
13111 4296 Acked Write 4 Done Sent Done false
13112 4297 Acked Read 9 Done Sent Done false
13113 4298 Acked Write 8 Done Sent Done false
13114 4299 Acked Flush 0 Done Sent Done false
13115 4300 Acked Read 6 Done Sent Done false
13116 4301 Acked Write 7 Done Sent Done false
13117 4302 Acked Write 10 Done Sent Done false
13118 4303 Acked Read 7 Done Sent Done false
13119 4304 Acked Write 4 Done Sent Done false
13120 4305 Acked Read 2 Done Sent Done false
13121 4306 Acked Write 4 Done Sent Done false
13122 4307 Acked Read 10 Done Sent Done false
13123 4308 Acked Read 6 Done Sent Done false
13124 4309 Acked Write 3 Done Sent Done false
13125 4310 Acked Read 3 Done Sent Done false
13126 4311 Acked Read 1 Done Sent Done false
13127 4312 Acked Read 1 Done Sent Done false
13128 4313 Acked Write 3 Done Sent Done false
13129 4314 Acked Read 8 Done Sent Done false
13130 4315 Acked Read 7 Done Sent Done false
13131 4316 Acked Read 3 Done Sent Done false
13132 4317 Acked Read 8 Done Sent Done false
13133 4318 Acked Flush 0 Done Sent Done false
13134 4319 Acked Read 3 Done Sent Done false
13135 4320 Acked Read 2 Done Sent Done false
13136 STATES DS:0 DS:1 DS:2 TOTAL
13137 Sent 0 301 0 301
13138 Done 301 0 301 602
13139 Skipped 0 0 0 0
13140 Error 0 0 0 0
13141 Last Flush: 4318 4019 4318
13142 Downstairs last five completed: 4019 4018 4017 4016 4015
13143 Upstairs last five completed: 4320 4319 4318 4317 4316
13144 CLIENT: Up:0 ds:301 act:2
13145 [8820] got start action
13146 [8820] I am going to restart
13147 Starting downstairs at port 8820
13148 Make output file at "/tmp/dsc/downstairs-8820.txt"
13149 [8820][1] reports Starting
13150 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1211)
13151 [8820][1] reports Running
13152 ----------------------------------------------------------------
13153 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13154 States: Active Offline Active
13155 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13156 4020 Acked Read 6 Done Sent Done false
13157 4021 Acked Read 2 Done Sent Done false
13158 4022 Acked Write 1 Done Sent Done false
13159 4023 Acked Write 2 Done Sent Done false
13160 4024 Acked Read 4 Done Sent Done false
13161 4025 Acked Read 8 Done Sent Done false
13162 4026 Acked Flush 0 Done Sent Done false
13163 4027 Acked Flush 0 Done Sent Done false
13164 4028 Acked Read 5 Done Sent Done false
13165 4029 Acked Write 2 Done Sent Done false
13166 4030 Acked Write 6 Done Sent Done false
13167 4031 Acked Write 8 Done Sent Done false
13168 4032 Acked Read 7 Done Sent Done false
13169 4033 Acked Flush 0 Done Sent Done false
13170 4034 Acked Read 4 Done Sent Done false
13171 4035 Acked Write 7 Done Sent Done false
13172 4036 Acked Write 4 Done Sent Done false
13173 4037 Acked Write 5 Done Sent Done false
13174 4038 Acked Read 1 Done Sent Done false
13175 4039 Acked Read 2 Done Sent Done false
13176 4040 Acked Write 1 Done Sent Done false
13177 4041 Acked Write 4 Done Sent Done false
13178 4042 Acked Write 7 Done Sent Done false
13179 4043 Acked Read 2 Done Sent Done false
13180 4044 Acked Write 9 Done Sent Done false
13181 4045 Acked Write 7 Done Sent Done false
13182 4046 Acked Read 2 Done Sent Done false
13183 4047 Acked Read 10 Done Sent Done false
13184 4048 Acked Read 8 Done Sent Done false
13185 4049 Acked Flush 0 Done Sent Done false
13186 4050 Acked Read 4 Done Sent Done false
13187 4051 Acked Write 1 Done Sent Done false
13188 4052 Acked Flush 0 Done Sent Done false
13189 4053 Acked Write 10 Done Sent Done false
13190 4054 Acked Flush 0 Done Sent Done false
13191 4055 Acked Read 6 Done Sent Done false
13192 4056 Acked Read 6 Done Sent Done false
13193 4057 Acked Read 5 Done Sent Done false
13194 4058 Acked Read 7 Done Sent Done false
13195 4059 Acked Write 2 Done Sent Done false
13196 4060 Acked Read 9 Done Sent Done false
13197 4061 Acked Write 5 Done Sent Done false
13198 4062 Acked Write 2 Done Sent Done false
13199 4063 Acked Read 5 Done Sent Done false
13200 4064 Acked Write 8 Done Sent Done false
13201 4065 Acked Write 8 Done Sent Done false
13202 4066 Acked Read 10 Done Sent Done false
13203 4067 Acked Write 6 Done Sent Done false
13204 4068 Acked Read 4 Done Sent Done false
13205 4069 Acked Write 2 Done Sent Done false
13206 4070 Acked Write 10 Done Sent Done false
13207 4071 Acked Read 9 Done Sent Done false
13208 4072 Acked Read 10 Done Sent Done false
13209 4073 Acked Write 10 Done Sent Done false
13210 4074 Acked Read 6 Done Sent Done false
13211 4075 Acked Write 7 Done Sent Done false
13212 4076 Acked Read 7 Done Sent Done false
13213 4077 Acked Read 3 Done Sent Done false
13214 4078 Acked Write 2 Done Sent Done false
13215 4079 Acked Read 2 Done Sent Done false
13216 4080 Acked Read 8 Done Sent Done false
13217 4081 Acked Write 5 Done Sent Done false
13218 4082 Acked Read 8 Done Sent Done false
13219 4083 Acked Write 1 Done Sent Done false
13220 4084 Acked Write 5 Done Sent Done false
13221 4085 Acked Write 8 Done Sent Done false
13222 4086 Acked Read 4 Done Sent Done false
13223 4087 Acked Read 10 Done Sent Done false
13224 4088 Acked Write 9 Done Sent Done false
13225 4089 Acked Write 5 Done Sent Done false
13226 4090 Acked Read 2 Done Sent Done false
13227 4091 Acked Read 1 Done Sent Done false
13228 4092 Acked Read 10 Done Sent Done false
13229 4093 Acked Read 9 Done Sent Done false
13230 4094 Acked Write 8 Done Sent Done false
13231 4095 Acked Write 4 Done Sent Done false
13232 4096 Acked Write 3 Done Sent Done false
13233 4097 Acked Flush 0 Done Sent Done false
13234 4098 Acked Read 3 Done Sent Done false
13235 4099 Acked Read 2 Done Sent Done false
13236 4100 Acked Read 3 Done Sent Done false
13237 4101 Acked Read 4 Done Sent Done false
13238 4102 Acked Read 3 Done Sent Done false
13239 4103 Acked Flush 0 Done Sent Done false
13240 4104 Acked Read 6 Done Sent Done false
13241 4105 Acked Write 10 Done Sent Done false
13242 4106 Acked Read 1 Done Sent Done false
13243 4107 Acked Read 10 Done Sent Done false
13244 4108 Acked Write 2 Done Sent Done false
13245 4109 Acked Write 8 Done Sent Done false
13246 4110 Acked Read 9 Done Sent Done false
13247 4111 Acked Read 3 Done Sent Done false
13248 4112 Acked Read 6 Done Sent Done false
13249 4113 Acked Read 9 Done Sent Done false
13250 4114 Acked Write 7 Done Sent Done false
13251 4115 Acked Write 6 Done Sent Done false
13252 4116 Acked Read 2 Done Sent Done false
13253 4117 Acked Read 4 Done Sent Done false
13254 4118 Acked Read 1 Done Sent Done false
13255 4119 Acked Flush 0 Done Sent Done false
13256 4120 Acked Read 1 Done Sent Done false
13257 4121 Acked Write 5 Done Sent Done false
13258 4122 Acked Flush 0 Done Sent Done false
13259 4123 Acked Write 4 Done Sent Done false
13260 4124 Acked Read 6 Done Sent Done false
13261 4125 Acked Read 6 Done Sent Done false
13262 4126 Acked Write 6 Done Sent Done false
13263 4127 Acked Read 6 Done Sent Done false
13264 4128 Acked Write 1 Done Sent Done false
13265 4129 Acked Write 7 Done Sent Done false
13266 4130 Acked Write 10 Done Sent Done false
13267 4131 Acked Write 4 Done Sent Done false
13268 4132 Acked Write 5 Done Sent Done false
13269 4133 Acked Read 2 Done Sent Done false
13270 4134 Acked Read 6 Done Sent Done false
13271 4135 Acked Read 5 Done Sent Done false
13272 4136 Acked Write 1 Done Sent Done false
13273 4137 Acked Write 4 Done Sent Done false
13274 4138 Acked Flush 0 Done Sent Done false
13275 4139 Acked Read 8 Done Sent Done false
13276 4140 Acked Write 9 Done Sent Done false
13277 4141 Acked Read 9 Done Sent Done false
13278 4142 Acked Write 6 Done Sent Done false
13279 4143 Acked Read 6 Done Sent Done false
13280 4144 Acked Write 3 Done Sent Done false
13281 4145 Acked Read 8 Done Sent Done false
13282 4146 Acked Read 6 Done Sent Done false
13283 4147 Acked Read 2 Done Sent Done false
13284 4148 Acked Read 5 Done Sent Done false
13285 4149 Acked Write 10 Done Sent Done false
13286 4150 Acked Write 2 Done Sent Done false
13287 4151 Acked Read 10 Done Sent Done false
13288 4152 Acked Read 10 Done Sent Done false
13289 4153 Acked Read 8 Done Sent Done false
13290 4154 Acked Write 2 Done Sent Done false
13291 4155 Acked Write 9 Done Sent Done false
13292 4156 Acked Flush 0 Done Sent Done false
13293 4157 Acked Read 7 Done Sent Done false
13294 4158 Acked Read 1 Done Sent Done false
13295 4159 Acked Write 6 Done Sent Done false
13296 4160 Acked Read 10 Done Sent Done false
13297 4161 Acked Write 7 Done Sent Done false
13298 4162 Acked Flush 0 Done Sent Done false
13299 4163 Acked Read 2 Done Sent Done false
13300 4164 Acked Write 4 Done Sent Done false
13301 4165 Acked Read 5 Done Sent Done false
13302 4166 Acked Read 3 Done Sent Done false
13303 4167 Acked Read 3 Done Sent Done false
13304 4168 Acked Write 7 Done Sent Done false
13305 4169 Acked Read 6 Done Sent Done false
13306 4170 Acked Read 3 Done Sent Done false
13307 4171 Acked Write 3 Done Sent Done false
13308 4172 Acked Read 2 Done Sent Done false
13309 4173 Acked Write 2 Done Sent Done false
13310 4174 Acked Read 7 Done Sent Done false
13311 4175 Acked Write 5 Done Sent Done false
13312 4176 Acked Write 7 Done Sent Done false
13313 4177 Acked Write 8 Done Sent Done false
13314 4178 Acked Write 1 Done Sent Done false
13315 4179 Acked Read 7 Done Sent Done false
13316 4180 Acked Write 6 Done Sent Done false
13317 4181 Acked Write 8 Done Sent Done false
13318 4182 Acked Flush 0 Done Sent Done false
13319 4183 Acked Read 2 Done Sent Done false
13320 4184 Acked Read 7 Done Sent Done false
13321 4185 Acked Write 6 Done Sent Done false
13322 4186 Acked Read 7 Done Sent Done false
13323 4187 Acked Read 5 Done Sent Done false
13324 4188 Acked Read 7 Done Sent Done false
13325 4189 Acked Flush 0 Done Sent Done false
13326 4190 Acked Flush 0 Done Sent Done false
13327 4191 Acked Read 1 Done Sent Done false
13328 4192 Acked Read 5 Done Sent Done false
13329 4193 Acked Write 8 Done Sent Done false
13330 4194 Acked Read 2 Done Sent Done false
13331 4195 Acked Read 4 Done Sent Done false
13332 4196 Acked Read 1 Done Sent Done false
13333 4197 Acked Write 10 Done Sent Done false
13334 4198 Acked Write 6 Done Sent Done false
13335 4199 Acked Read 6 Done Sent Done false
13336 4200 Acked Write 4 Done Sent Done false
13337 4201 Acked Read 2 Done Sent Done false
13338 4202 Acked Read 1 Done Sent Done false
13339 4203 Acked Flush 0 Done Sent Done false
13340 4204 Acked Read 9 Done Sent Done false
13341 4205 Acked Write 2 Done Sent Done false
13342 4206 Acked Read 8 Done Sent Done false
13343 4207 Acked Read 4 Done Sent Done false
13344 4208 Acked Write 8 Done Sent Done false
13345 4209 Acked Read 9 Done Sent Done false
13346 4210 Acked Write 7 Done Sent Done false
13347 4211 Acked Write 8 Done Sent Done false
13348 4212 Acked Flush 0 Done Sent Done false
13349 4213 Acked Read 5 Done Sent Done false
13350 4214 Acked Read 1 Done Sent Done false
13351 4215 Acked Flush 0 Done Sent Done false
13352 4216 Acked Read 9 Done Sent Done false
13353 4217 Acked Write 4 Done Sent Done false
13354 4218 Acked Flush 0 Done Sent Done false
13355 4219 Acked Read 9 Done Sent Done false
13356 4220 Acked Write 10 Done Sent Done false
13357 4221 Acked Read 9 Done Sent Done false
13358 4222 Acked Read 4 Done Sent Done false
13359 4223 Acked Read 1 Done Sent Done false
13360 4224 Acked Read 3 Done Sent Done false
13361 4225 Acked Read 4 Done Sent Done false
13362 4226 Acked Write 3 Done Sent Done false
13363 4227 Acked Write 1 Done Sent Done false
13364 4228 Acked Write 3 Done Sent Done false
13365 4229 Acked Read 10 Done Sent Done false
13366 4230 Acked Write 6 Done Sent Done false
13367 4231 Acked Read 5 Done Sent Done false
13368 4232 Acked Read 5 Done Sent Done false
13369 4233 Acked Read 8 Done Sent Done false
13370 4234 Acked Write 7 Done Sent Done false
13371 4235 Acked Write 2 Done Sent Done false
13372 4236 Acked Flush 0 Done Sent Done false
13373 4237 Acked Read 3 Done Sent Done false
13374 4238 Acked Read 1 Done Sent Done false
13375 4239 Acked Read 3 Done Sent Done false
13376 4240 Acked Read 1 Done Sent Done false
13377 4241 Acked Write 9 Done Sent Done false
13378 4242 Acked Read 2 Done Sent Done false
13379 4243 Acked Read 3 Done Sent Done false
13380 4244 Acked Read 9 Done Sent Done false
13381 4245 Acked Write 7 Done Sent Done false
13382 4246 Acked Read 9 Done Sent Done false
13383 4247 Acked Read 5 Done Sent Done false
13384 4248 Acked Write 3 Done Sent Done false
13385 4249 Acked Read 1 Done Sent Done false
13386 4250 Acked Read 4 Done Sent Done false
13387 4251 Acked Write 6 Done Sent Done false
13388 4252 Acked Write 1 Done Sent Done false
13389 4253 Acked Read 3 Done Sent Done false
13390 4254 Acked Write 2 Done Sent Done false
13391 4255 Acked Read 6 Done Sent Done false
13392 4256 Acked Write 4 Done Sent Done false
13393 4257 Acked Write 6 Done Sent Done false
13394 4258 Acked Write 2 Done Sent Done false
13395 4259 Acked Write 8 Done Sent Done false
13396 4260 Acked Write 4 Done Sent Done false
13397 4261 Acked Read 2 Done Sent Done false
13398 4262 Acked Read 10 Done Sent Done false
13399 4263 Acked Read 8 Done Sent Done false
13400 4264 Acked Read 5 Done Sent Done false
13401 4265 Acked Read 3 Done Sent Done false
13402 4266 Acked Write 10 Done Sent Done false
13403 4267 Acked Read 7 Done Sent Done false
13404 4268 Acked Read 7 Done Sent Done false
13405 4269 Acked Read 9 Done Sent Done false
13406 4270 Acked Read 9 Done Sent Done false
13407 4271 Acked Write 10 Done Sent Done false
13408 4272 Acked Write 1 Done Sent Done false
13409 4273 Acked Read 9 Done Sent Done false
13410 4274 Acked Read 5 Done Sent Done false
13411 4275 Acked Write 8 Done Sent Done false
13412 4276 Acked Read 10 Done Sent Done false
13413 4277 Acked Write 8 Done Sent Done false
13414 4278 Acked Read 7 Done Sent Done false
13415 4279 Acked Read 8 Done Sent Done false
13416 4280 Acked Write 5 Done Sent Done false
13417 4281 Acked Read 5 Done Sent Done false
13418 4282 Acked Read 4 Done Sent Done false
13419 4283 Acked Write 5 Done Sent Done false
13420 4284 Acked Flush 0 Done Sent Done false
13421 4285 Acked Write 1 Done Sent Done false
13422 4286 Acked Read 4 Done Sent Done false
13423 4287 Acked Read 10 Done Sent Done false
13424 4288 Acked Read 7 Done Sent Done false
13425 4289 Acked Flush 0 Done Sent Done false
13426 4290 Acked Read 5 Done Sent Done false
13427 4291 Acked Write 5 Done Sent Done false
13428 4292 Acked Read 1 Done Sent Done false
13429 4293 Acked Write 5 Done Sent Done false
13430 4294 Acked Write 9 Done Sent Done false
13431 4295 Acked Read 2 Done Sent Done false
13432 4296 Acked Write 4 Done Sent Done false
13433 4297 Acked Read 9 Done Sent Done false
13434 4298 Acked Write 8 Done Sent Done false
13435 4299 Acked Flush 0 Done Sent Done false
13436 4300 Acked Read 6 Done Sent Done false
13437 4301 Acked Write 7 Done Sent Done false
13438 4302 Acked Write 10 Done Sent Done false
13439 4303 Acked Read 7 Done Sent Done false
13440 4304 Acked Write 4 Done Sent Done false
13441 4305 Acked Read 2 Done Sent Done false
13442 4306 Acked Write 4 Done Sent Done false
13443 4307 Acked Read 10 Done Sent Done false
13444 4308 Acked Read 6 Done Sent Done false
13445 4309 Acked Write 3 Done Sent Done false
13446 4310 Acked Read 3 Done Sent Done false
13447 4311 Acked Read 1 Done Sent Done false
13448 4312 Acked Read 1 Done Sent Done false
13449 4313 Acked Write 3 Done Sent Done false
13450 4314 Acked Read 8 Done Sent Done false
13451 4315 Acked Read 7 Done Sent Done false
13452 4316 Acked Read 3 Done Sent Done false
13453 4317 Acked Read 8 Done Sent Done false
13454 4318 Acked Flush 0 Done Sent Done false
13455 4319 Acked Read 3 Done Sent Done false
13456 4320 Acked Read 2 Done Sent Done false
13457 4321 Acked Flush 0 Done Sent Done false
13458 STATES DS:0 DS:1 DS:2 TOTAL
13459 Sent 0 302 0 302
13460 Done 302 0 302 604
13461 Skipped 0 0 0 0
13462 Error 0 0 0 0
13463 Last Flush: 4321 4019 4321
13464 Downstairs last five completed: 4019 4018 4017 4016 4015
13465 Upstairs last five completed: 4321 4320 4319 4318 4317
13466 CLIENT: Up:0 ds:302 act:2
134672024-12-11T23:43:18.985ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134682024-12-11T23:43:18.985ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49836) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134692024-12-11T23:43:18.986ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134702024-12-11T23:43:18.986ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134712024-12-11T23:43:18.986ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134722024-12-11T23:43:18.986ZINFOcrucible: send last flush ID to this DS: 4019 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134732024-12-11T23:43:18.987ZINFOcrucible: Replied this last flush ID: 4019 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134742024-12-11T23:43:18.987ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134752024-12-11T23:43:18.987ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134762024-12-11T23:43:18.987ZINFOcrucible: [1] client re-new 302 jobs since flush 4019 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
134772024-12-11T23:43:18.987ZINFOcrucible: [1] Replayed 302 jobs since flush: 4019 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
13478 ----------------------------------------------------------------
13479 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13480 Downstairs last five completed: 4321 4320 4319 4318 4317
13481 Upstairs last five completed: 4321 4320 4319 4318 4317
13482 CLIENT: Up:0 ds:0 act:3
13483 Replay: All jobs finished
13484 Main task has work to do, go find it
13485 got dsc Stop(1)
13486 stop 1
13487 [8820] Got stop action so:false kr:false
13488 Dec 11 23:43:21.230 INFO request completed, latency_us: 329, response_code: 204, uri: /stop/cid/1, method: POST, req_id: a23f5691-91ae-4b23-aada-a1fa088a9cf4, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
13489 Dec 11 23:43:21.231 INFO request completed{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:43:21.231737505Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
13490 , latency_us: 144, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:43:21.231802857Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
13491 response_code{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:43:21.231840568Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
13492 : 200, uri: /state/cid/1, method: GET, req_id: ccbd1276-fea5-4bb8-b5ce-148ad1deea67, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:43:21.231856819Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
13493 remote_addr: 127.0.{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:43:21.23189183Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
13494 0.1:62969, local_addr: 127.0.0.1:9998
134952024-12-11T23:43:21.231ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
13496 [8820] Exited with: ExitStatus(unix_wait_status(9))
13497 [8820][1] reports Exit
13498 Dec 11 23:43:25.233 INFO request completed, latency_us: 232, response_code: 200, uri: /state/cid/1, method: GET, req_id: 203919ca-3c92-4eb4-bda6-4a4964a55596, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
13499 001/300 Read block 3526 len 12288
13500 002/300 Write block 4826 len 28672 data: 3 3 3 3 2 2 3
13501 003/300 Write block 2860 len 40960 data: 5 4 4 4 5 5 4 4 3 3
13502 004/300 Write block 3011 len 40960 data: 4 4 4 4 3 3 5 5 4 3
13503 005/300 Flush
13504 006/300 Read block 4574 len 40960
13505 007/300 Read block 439 len 16384
13506 008/300 Flush
13507 009/300 Write block 4751 len 20480 data: 4 3 3 2 2
13508 010/300 Write block 2744 len 4096 data: 3
13509 011/300 Read block 4864 len 28672
13510 012/300 Write block 2944 len 32768 data: 4 3 3 3 2 2 2 2
13511 013/300 Read block 3811 len 8192
13512 014/300 Read block 3622 len 8192
13513 015/300 Read block 2807 len 16384
13514 016/300 Write block 3144 len 40960 data: 4 3 3 3 3 2 2 2 2 3
13515 017/300 Write block 4401 len 8192 data: 5 4
13516 018/300 Read block 2872 len 12288
13517 019/300 Read block 3351 len 8192
13518 020/300 Read block 152 len 12288
13519 021/300 Write block 1321 len 12288 data: 4 4 3
13520 022/300 Read block 1234 len 4096
13521 023/300 Read block 2231 len 36864
13522 024/300 Write block 2372 len 32768 data: 2 2 2 3 3 3 3 5
13523 025/300 Read block 4288 len 40960
13524 026/300 Read block 859 len 4096
13525 027/300 Write block 1645 len 36864 data: 4 4 3 4 4 2 2 2 2
13526 028/300 Write block 4951 len 36864 data: 3 2 2 2 3 4 4 5 4
13527 029/300 Write block 4846 len 16384 data: 4 4 3 3
13528 030/300 Read block 2987 len 40960
13529 031/300 Read block 4570 len 8192
13530 032/300 Write block 484 len 4096 data: 4
13531 033/300 Read block 1090 len 20480
13532 034/300 Write block 4181 len 8192 data: 5 5
13533 035/300 Read block 2623 len 20480
13534 036/300 Write block 2285 len 8192 data: 3 3
13535 037/300 Read block 317 len 32768
13536 038/300 Write block 4469 len 36864 data: 6 7 7 7 5 5 5 4 3
13537 039/300 Write block 3541 len 32768 data: 4 4 4 4 4 4 4 3
13538 040/300 Write block 2130 len 20480 data: 5 5 4 3 3
13539 041/300 Read block 1492 len 24576
13540 042/300 Read block 3876 len 24576
13541 043/300 Write block 4283 len 16384 data: 3 3 3 3
13542 044/300 Read block 4150 len 16384
13543 045/300 Read block 3233 len 36864
13544 046/300 Read block 3894 len 24576
13545 047/300 Write block 709 len 20480 data: 3 3 3 3 3
13546 048/300 Read block 1968 len 24576
13547 049/300 Write block 2141 len 4096 data: 2
13548 050/300 Flush
13549 051/300 Read block 2199 len 32768
13550 052/300 Read block 25 len 32768
13551 053/300 Write block 2616 len 36864 data: 3 3 2 2 2 3 2 2 2
13552 054/300 Write block 1891 len 4096 data: 4
13553 055/300 Write block 1635 len 12288 data: 4 4 4
13554 056/300 Read block 3034 len 40960
13555 057/300 Read block 3996 len 24576
13556 058/300 Read block 1838 len 4096
13557 059/300 Read block 2868 len 28672
13558 060/300 Write block 605 len 4096 data: 2
13559 061/300 Write block 1500 len 36864 data: 4 3 3 3 3 3 3 3 3
13560 062/300 Write block 2543 len 28672 data: 3 3 3 3 3 3 3
13561 063/300 Read block 1416 len 16384
13562 064/300 Write block 3798 len 36864 data: 3 4 4 5 5 5 4 3 3
13563 065/300 Write block 4269 len 20480 data: 3 2 2 3 3
13564 066/300 Write block 3033 len 8192 data: 3 4
13565 067/300 Read block 2758 len 28672
13566 068/300 Read block 3316 len 40960
13567 069/300 Write block 4711 len 32768 data: 2 2 2 3 4 4 5 4
13568 070/300 Write block 3203 len 4096 data: 4
13569 071/300 Write block 2370 len 4096 data: 3
13570 072/300 Write block 406 len 36864 data: 3 3 5 5 5 4 4 4 4
13571 073/300 Flush
13572 074/300 Read block 330 len 12288
13573 075/300 Read block 3346 len 20480
13574 076/300 Write block 4441 len 12288 data: 3 3 3
13575 077/300 Read block 4896 len 12288
13576 078/300 Read block 3286 len 4096
13577 079/300 Read block 1645 len 20480
13578 080/300 Write block 1859 len 20480 data: 4 4 4 4 3
13579 081/300 Write block 1887 len 16384 data: 4 4 5 4
13580 082/300 Flush
13581 083/300 Flush
13582 084/300 Read block 3625 len 20480
13583 085/300 Write block 2701 len 36864 data: 4 4 4 4 3 5 3 3 3
13584 086/300 Write block 4568 len 40960 data: 3 3 3 3 3 2 2 2 2 2
13585 087/300 Write block 4557 len 40960 data: 3 3 3 3 3 3 3 3 3 3
13586 088/300 Write block 986 len 12288 data: 5 4 3
13587 089/300 Flush
13588 090/300 Read block 998 len 24576
13589 091/300 Write block 1145 len 12288 data: 3 2 4
13590 092/300 Read block 2823 len 40960
13591 093/300 Read block 1032 len 36864
13592 094/300 Write block 2408 len 4096 data: 2
13593 095/300 Read block 339 len 28672
13594 096/300 Write block 2837 len 24576 data: 5 4 3 3 3 3
13595 097/300 Write block 3940 len 12288 data: 2 3 3
13596 098/300 Read block 3047 len 24576
13597 099/300 Read block 939 len 28672
13598 100/300 Read block 3876 len 12288
13599 101/300 Read block 3010 len 28672
13600 102/300 Write block 4163 len 12288 data: 4 4 4
13601 103/300 Read block 2430 len 40960
13602 104/300 Flush
13603 105/300 Read block 3352 len 20480
13604 106/300 Write block 973 len 16384 data: 5 4 4 2
13605 107/300 Read block 2547 len 24576
13606 108/300 Flush
13607 109/300 Write block 1660 len 12288 data: 2 2 2
13608 110/300 Read block 1643 len 40960
13609 111/300 Flush
13610 112/300 Flush
13611 113/300 Write block 1745 len 20480 data: 3 3 3 2 2
13612 114/300 Write block 4728 len 24576 data: 2 2 2 2 2 3
13613 115/300 Read block 4665 len 36864
13614 116/300 Read block 903 len 16384
13615 117/300 Write block 4206 len 24576 data: 3 4 4 3 3 3
13616 118/300 Write block 2714 len 32768 data: 6 6 8 6 7 6 6 6
13617 119/300 Write block 3828 len 24576 data: 7 7 7 6 5 4
13618 120/300 Write block 4063 len 36864 data: 4 3 3 3 2 2 3 3 4
13619 121/300 Read block 299 len 24576
13620 122/300 Read block 1114 len 24576
13621 123/300 Read block 1824 len 40960
13622 124/300 Flush
13623 125/300 Write block 1696 len 12288 data: 4 4 4
13624 126/300 Read block 1969 len 16384
13625 127/300 Write block 4545 len 20480 data: 3 2 2 2 2
13626 128/300 Read block 3146 len 28672
13627 129/300 Write block 2454 len 24576 data: 2 2 2 2 3 3
13628 130/300 Write block 3998 len 4096 data: 3
13629 131/300 Read block 2050 len 24576
13630 132/300 Write block 3435 len 4096 data: 4
13631 133/300 Flush
13632 134/300 Read block 3141 len 32768
13633 135/300 Read block 2924 len 4096
13634 136/300 Read block 2644 len 28672
13635 137/300 Read block 4003 len 12288
13636 138/300 Write block 4433 len 28672 data: 3 3 3 3 3 3 3
13637 139/300 Read block 3122 len 16384
13638 140/300 Write block 881 len 28672 data: 6 5 4 4 4 4 4
13639 141/300 Write block 2314 len 4096 data: 3
13640 142/300 Write block 3271 len 28672 data: 2 2 2 3 4 3 4
13641 143/300 Read block 1534 len 24576
13642 144/300 Read block 1814 len 40960
13643 145/300 Read block 286 len 36864
13644 146/300 Read block 2011 len 12288
13645 147/300 Write block 3521 len 32768 data: 2 3 3 3 3 3 5 5
13646 148/300 Flush
13647 149/300 Flush
13648 150/300 Write block 684 len 12288 data: 3 4 2
13649 151/300 Read block 969 len 20480
13650 152/300 Flush
13651 153/300 Read block 2503 len 28672
13652 154/300 Read block 1591 len 40960
13653 155/300 Read block 4 len 20480
13654 156/300 Write block 1251 len 8192 data: 2 2
13655 157/300 Flush
13656 158/300 Read block 4901 len 20480
13657 159/300 Write block 4595 len 16384 data: 3 4 5 5
13658 160/300 Flush
13659 161/300 Read block 3959 len 36864
13660 162/300 Read block 1494 len 20480
13661 163/300 Write block 844 len 32768 data: 3 3 3 4 3 3 3 3
13662 164/300 Write block 250 len 24576 data: 2 2 2 2 2 2
13663 165/300 Read block 4073 len 28672
13664 166/300 Read block 3166 len 8192
13665 167/300 Flush
13666 168/300 Read block 136 len 24576
13667 169/300 Write block 2578 len 8192 data: 8 7
13668 170/300 Read block 3061 len 40960
13669 171/300 Write block 509 len 36864 data: 3 2 3 3 2 2 3 4 5
13670 172/300 Read block 1375 len 24576
13671 173/300 Write block 1646 len 36864 data: 5 4 5 5 3 3 3 3 2
13672 174/300 Read block 1558 len 36864
13673 175/300 Read block 3196 len 16384
13674 176/300 Write block 2907 len 36864 data: 2 2 2 2 2 2 2 3 4
13675 177/300 Read block 935 len 40960
13676 178/300 Read block 3422 len 12288
13677 179/300 Read block 764 len 24576
13678 180/300 Write block 4718 len 8192 data: 5 4
13679 181/300 Write block 4537 len 36864 data: 4 4 4 4 4 4 4 3 4
13680 182/300 Write block 3099 len 40960 data: 4 5 3 4 4 5 5 4 4 4
13681 183/300 Write block 2710 len 12288 data: 4 4 5
13682 184/300 Read block 3674 len 8192
13683 185/300 Write block 3677 len 36864 data: 3 3 3 3 3 3 3 3 3
13684 186/300 Write block 3259 len 28672 data: 2 2 2 3 5 4 4
13685 187/300 Write block 3233 len 20480 data: 2 2 2 2 2
13686 188/300 Read block 2249 len 16384
13687 189/300 Read block 3827 len 8192
13688 190/300 Write block 1902 len 28672 data: 2 2 2 2 3 3 4
13689 191/300 Write block 3086 len 28672 data: 5 5 5 6 6 5 6
13690 192/300 Write block 856 len 24576 data: 4 4 4 4 4 4
13691 193/300 Read block 2257 len 20480
13692 194/300 Write block 495 len 28672 data: 4 4 4 3 3 2 2
13693 195/300 Read block 293 len 8192
13694 196/300 Read block 3431 len 16384
13695 197/300 Read block 4983 len 8192
13696 198/300 Read block 3634 len 28672
13697 199/300 Write block 1736 len 32768 data: 2 2 2 2 3 3 3 3
13698 200/300 Write block 2126 len 4096 data: 5
13699 201/300 Write block 1997 len 16384 data: 2 2 2 2
13700 202/300 Write block 1100 len 12288 data: 4 4 3
13701 203/300 Write block 2995 len 16384 data: 4 7 8 9
13702 204/300 Read block 1241 len 32768
13703 205/300 Read block 322 len 8192
13704 206/300 Flush
13705 207/300 Read block 3352 len 32768
13706 208/300 Read block 3787 len 4096
13707 209/300 Read block 2262 len 12288
13708 210/300 Write block 223 len 36864 data: 4 4 4 2 3 3 3 3 4
13709 211/300 Read block 1715 len 28672
13710 212/300 Write block 4800 len 20480 data: 4 3 4 4 3
13711 213/300 Write block 3869 len 4096 data: 4
13712 214/300 Read block 3794 len 12288
13713 215/300 Write block 1187 len 12288 data: 3 4 3
13714 216/300 Read block 2572 len 16384
13715 217/300 Read block 987 len 16384
13716 218/300 Write block 4038 len 16384 data: 4 4 5 5
13717 219/300 Read block 4213 len 8192
13718 220/300 Write block 341 len 32768 data: 4 4 5 6 6 6 5 2
13719 221/300 Flush
13720 222/300 Write block 1039 len 36864 data: 2 2 3 2 2 2 2 4 4
13721 223/300 Write block 995 len 28672 data: 2 2 2 2 2 2 3
13722 224/300 Write block 2210 len 36864 data: 4 5 5 4 4 4 4 3 2
13723 225/300 Read block 1053 len 4096
13724 226/300 Read block 918 len 28672
13725 227/300 Write block 4446 len 20480 data: 3 2 2 2 3
13726 228/300 Read block 2107 len 28672
13727 229/300 Read block 3693 len 8192
13728 230/300 Read block 3028 len 12288
13729 231/300 Write block 293 len 28672 data: 2 2 2 2 3 4 4
13730 232/300 Write block 2427 len 20480 data: 4 3 4 5 5
13731 233/300 Read block 3161 len 12288
13732 234/300 Flush
13733 235/300 Read block 2203 len 12288
13734 236/300 Write block 576 len 28672 data: 4 5 6 5 5 4 6
13735 237/300 Read block 4946 len 24576
13736 238/300 Write block 337 len 8192 data: 2 2
13737 239/300 Write block 2328 len 32768 data: 5 5 5 4 5 5 7 6
13738 240/300 Write block 3170 len 20480 data: 5 3 4 3 4
13739 241/300 Read block 592 len 4096
13740 242/300 Read block 314 len 40960
13741 243/300 Read block 1799 len 12288
13742 244/300 Read block 3525 len 16384
13743 245/300 Read block 626 len 20480
13744 246/300 Read block 1347 len 32768
13745 247/300 Read block 134 len 32768
13746 248/300 Read block 587 len 24576
13747 249/300 Read block 2790 len 32768
13748 250/300 Write block 4463 len 16384 data: 3 4 3 3
13749 251/300 Flush
13750 252/300 Read block 789 len 4096
13751 253/300 Write block 1455 len 36864 data: 3 4 5 4 4 4 4 4 4
13752 254/300 Read block 4170 len 28672
13753 255/300 Write block 2259 len 16384 data: 2 2 2 2
13754 256/300 Read block 909 len 20480
13755 257/300 Read block 4152 len 4096
13756 258/300 Read block 3206 len 32768
13757 259/300 Read block 613 len 8192
13758 260/300 Write block 1374 len 12288 data: 3 3 2
13759 261/300 Write block 633 len 4096 data: 3
13760 262/300 Write block 3321 len 12288 data: 2 2 2
13761 263/300 Read block 481 len 40960
13762 264/300 Write block 1002 len 28672 data: 3 2 2 2 3 3 3
13763 265/300 Write block 2563 len 28672 data: 3 3 3 3 3 3 4
13764 266/300 Read block 4325 len 4096
13765 267/300 Flush
13766 268/300 Write block 357 len 4096 data: 5
13767 269/300 Write block 3889 len 4096 data: 4
13768 270/300 Read block 2355 len 20480
13769 271/300 Write block 1604 len 40960 data: 3 3 3 3 3 3 3 2 3 3
13770 272/300 Read block 2890 len 36864
13771 273/300 Read block 268 len 8192
13772 274/300 Read block 2896 len 40960
13773 275/300 Write block 3499 len 20480 data: 3 3 3 2 2
13774 276/300 Read block 38 len 8192
13775 277/300 Read block 1666 len 24576
13776 278/300 Read block 2522 len 12288
13777 279/300 Read block 2176 len 36864
13778 280/300 Read block 3264 len 16384
13779 281/300 Flush
13780 282/300 Read block 880 len 40960
13781 283/300 Write block 4380 len 8192 data: 5 5
13782 284/300 Read block 4670 len 32768
13783 285/300 Write block 1203 len 32768 data: 3 4 4 3 3 3 3 2
13784 286/300 Write block 607 len 32768 data: 2 2 2 2 2 2 2 3
13785 287/300 Read block 2077 len 28672
13786 288/300 Flush
13787 289/300 Write block 759 len 8192 data: 4 4
13788 290/300 Write block 1668 len 12288 data: 2 2 2
13789 291/300 Write block 1541 len 36864 data: 5 4 4 5 4 4 3 3 2
13790 292/300 Write block 1560 len 36864 data: 2 2 3 3 3 3 2 2 2
13791 293/300 Read block 813 len 12288
13792 294/300 Write block 4807 len 32768 data: 3 3 3 3 2 2 3 2
13793 295/300 Write block 855 len 8192 data: 6 5
13794 296/300 Read block 3716 len 24576
13795 297/300 Write block 3919 len 36864 data: 3 3 3 5 4 3 3 3 2
13796 298/300 Write block 3016 len 16384 data: 4 6 6 5
13797 299/300 Write block 49 len 24576 data: 6 6 4 3 3 4
13798 300/300 Read block 538 len 36864
13799 301/300 Write block 1871 len 36864 data: 2 2 2 2 2 2 2 2 2
13800 Main task has work to do, go find it
13801 got dsc Start(1)
13802 start 1
13803 Dec 11 23:43:25.466 INFO request completed, latency_us: 226, response_code: 204, uri: /start/cid/1, method: POST, req_id: 239cd3a4-bf19-4f2a-bd50-b65a4aaefb90, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
13804 Replay: started 1, returned:Ok(())
13805 ----------------------------------------------------------------
13806 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13807 States: Active Offline Active
13808 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13809 4322 Acked Read 3 Done Sent Done false
13810 4323 Acked Write 7 Done Sent Done false
13811 4324 Acked Write 10 Done Sent Done false
13812 4325 Acked Write 10 Done Sent Done false
13813 4326 Acked Flush 0 Done Sent Done false
13814 4327 Acked Read 10 Done Sent Done false
13815 4328 Acked Read 4 Done Sent Done false
13816 4329 Acked Flush 0 Done Sent Done false
13817 4330 Acked Write 5 Done Sent Done false
13818 4331 Acked Write 1 Done Sent Done false
13819 4332 Acked Read 7 Done Sent Done false
13820 4333 Acked Write 8 Done Sent Done false
13821 4334 Acked Read 2 Done Sent Done false
13822 4335 Acked Read 2 Done Sent Done false
13823 4336 Acked Read 4 Done Sent Done false
13824 4337 Acked Write 10 Done Sent Done false
13825 4338 Acked Write 2 Done Sent Done false
13826 4339 Acked Read 3 Done Sent Done false
13827 4340 Acked Read 2 Done Sent Done false
13828 4341 Acked Read 3 Done Sent Done false
13829 4342 Acked Write 3 Done Sent Done false
13830 4343 Acked Read 1 Done Sent Done false
13831 4344 Acked Read 9 Done Sent Done false
13832 4345 Acked Write 8 Done Sent Done false
13833 4346 Acked Read 10 Done Sent Done false
13834 4347 Acked Read 1 Done Sent Done false
13835 4348 Acked Write 9 Done Sent Done false
13836 4349 Acked Write 9 Done Sent Done false
13837 4350 Acked Write 4 Done Sent Done false
13838 4351 Acked Read 10 Done Sent Done false
13839 4352 Acked Read 2 Done Sent Done false
13840 4353 Acked Write 1 Done Sent Done false
13841 4354 Acked Read 5 Done Sent Done false
13842 4355 Acked Write 2 Done Sent Done false
13843 4356 Acked Read 5 Done Sent Done false
13844 4357 Acked Write 2 Done Sent Done false
13845 4358 Acked Read 8 Done Sent Done false
13846 4359 Acked Write 9 Done Sent Done false
13847 4360 Acked Write 8 Done Sent Done false
13848 4361 Acked Write 5 Done Sent Done false
13849 4362 Acked Read 6 Done Sent Done false
13850 4363 Acked Read 6 Done Sent Done false
13851 4364 Acked Write 4 Done Sent Done false
13852 4365 Acked Read 4 Done Sent Done false
13853 4366 Acked Read 9 Done Sent Done false
13854 4367 Acked Read 6 Done Sent Done false
13855 4368 Acked Write 5 Done Sent Done false
13856 4369 Acked Read 6 Done Sent Done false
13857 4370 Acked Write 1 Done Sent Done false
13858 4371 Acked Flush 0 Done Sent Done false
13859 4372 Acked Read 8 Done Sent Done false
13860 4373 Acked Read 8 Done Sent Done false
13861 4374 Acked Write 9 Done Sent Done false
13862 4375 Acked Write 1 Done Sent Done false
13863 4376 Acked Write 3 Done Sent Done false
13864 4377 Acked Read 10 Done Sent Done false
13865 4378 Acked Read 6 Done Sent Done false
13866 4379 Acked Read 1 Done Sent Done false
13867 4380 Acked Read 7 Done Sent Done false
13868 4381 Acked Write 1 Done Sent Done false
13869 4382 Acked Write 9 Done Sent Done false
13870 4383 Acked Write 7 Done Sent Done false
13871 4384 Acked Read 4 Done Sent Done false
13872 4385 Acked Write 9 Done Sent Done false
13873 4386 Acked Write 5 Done Sent Done false
13874 4387 Acked Write 2 Done Sent Done false
13875 4388 Acked Read 7 Done Sent Done false
13876 4389 Acked Read 10 Done Sent Done false
13877 4390 Acked Write 8 Done Sent Done false
13878 4391 Acked Write 1 Done Sent Done false
13879 4392 Acked Write 1 Done Sent Done false
13880 4393 Acked Write 9 Done Sent Done false
13881 4394 Acked Flush 0 Done Sent Done false
13882 4395 Acked Read 3 Done Sent Done false
13883 4396 Acked Read 5 Done Sent Done false
13884 4397 Acked Write 3 Done Sent Done false
13885 4398 Acked Read 3 Done Sent Done false
13886 4399 Acked Read 1 Done Sent Done false
13887 4400 Acked Read 5 Done Sent Done false
13888 4401 Acked Write 5 Done Sent Done false
13889 4402 Acked Write 4 Done Sent Done false
13890 4403 Acked Flush 0 Done Sent Done false
13891 4404 Acked Flush 0 Done Sent Done false
13892 4405 Acked Read 5 Done Sent Done false
13893 4406 Acked Write 9 Done Sent Done false
13894 4407 Acked Write 10 Done Sent Done false
13895 4408 Acked Write 10 Done Sent Done false
13896 4409 Acked Write 3 Done Sent Done false
13897 4410 Acked Flush 0 Done Sent Done false
13898 4411 Acked Read 6 Done Sent Done false
13899 4412 Acked Write 3 Done Sent Done false
13900 4413 Acked Read 10 Done Sent Done false
13901 4414 Acked Read 9 Done Sent Done false
13902 4415 Acked Write 1 Done Sent Done false
13903 4416 Acked Read 7 Done Sent Done false
13904 4417 Acked Write 6 Done Sent Done false
13905 4418 Acked Write 3 Done Sent Done false
13906 4419 Acked Read 6 Done Sent Done false
13907 4420 Acked Read 7 Done Sent Done false
13908 4421 Acked Read 3 Done Sent Done false
13909 4422 Acked Read 7 Done Sent Done false
13910 4423 Acked Write 3 Done Sent Done false
13911 4424 Acked Read 10 Done Sent Done false
13912 4425 Acked Flush 0 Done Sent Done false
13913 4426 Acked Read 5 Done Sent Done false
13914 4427 Acked Write 4 Done Sent Done false
13915 4428 Acked Read 6 Done Sent Done false
13916 4429 Acked Flush 0 Done Sent Done false
13917 4430 Acked Write 3 Done Sent Done false
13918 4431 Acked Read 10 Done Sent Done false
13919 4432 Acked Flush 0 Done Sent Done false
13920 4433 Acked Flush 0 Done Sent Done false
13921 4434 Acked Write 5 Done Sent Done false
13922 4435 Acked Write 6 Done Sent Done false
13923 4436 Acked Read 9 Done Sent Done false
13924 4437 Acked Read 4 Done Sent Done false
13925 4438 Acked Write 6 Done Sent Done false
13926 4439 Acked Write 8 Done Sent Done false
13927 4440 Acked Write 6 Done Sent Done false
13928 4441 Acked Write 9 Done Sent Done false
13929 4442 Acked Read 6 Done Sent Done false
13930 4443 Acked Read 6 Done Sent Done false
13931 4444 Acked Read 10 Done Sent Done false
13932 4445 Acked Flush 0 Done Sent Done false
13933 4446 Acked Write 3 Done Sent Done false
13934 4447 Acked Read 4 Done Sent Done false
13935 4448 Acked Write 5 Done Sent Done false
13936 4449 Acked Read 7 Done Sent Done false
13937 4450 Acked Write 6 Done Sent Done false
13938 4451 Acked Write 1 Done Sent Done false
13939 4452 Acked Read 6 Done Sent Done false
13940 4453 Acked Write 1 Done Sent Done false
13941 4454 Acked Flush 0 Done Sent Done false
13942 4455 Acked Read 8 Done Sent Done false
13943 4456 Acked Read 1 Done Sent Done false
13944 4457 Acked Read 7 Done Sent Done false
13945 4458 Acked Read 3 Done Sent Done false
13946 4459 Acked Write 7 Done Sent Done false
13947 4460 Acked Read 4 Done Sent Done false
13948 4461 Acked Write 7 Done Sent Done false
13949 4462 Acked Write 1 Done Sent Done false
13950 4463 Acked Write 7 Done Sent Done false
13951 4464 Acked Read 6 Done Sent Done false
13952 4465 Acked Read 10 Done Sent Done false
13953 4466 Acked Read 9 Done Sent Done false
13954 4467 Acked Read 3 Done Sent Done false
13955 4468 Acked Write 8 Done Sent Done false
13956 4469 Acked Flush 0 Done Sent Done false
13957 4470 Acked Flush 0 Done Sent Done false
13958 4471 Acked Write 3 Done Sent Done false
13959 4472 Acked Read 5 Done Sent Done false
13960 4473 Acked Flush 0 Done Sent Done false
13961 4474 Acked Read 7 Done Sent Done false
13962 4475 Acked Read 10 Done Sent Done false
13963 4476 Acked Read 5 Done Sent Done false
13964 4477 Acked Write 2 Done Sent Done false
13965 4478 Acked Flush 0 Done Sent Done false
13966 4479 Acked Read 5 Done Sent Done false
13967 4480 Acked Write 4 Done Sent Done false
13968 4481 Acked Flush 0 Done Sent Done false
13969 4482 Acked Read 9 Done Sent Done false
13970 4483 Acked Read 5 Done Sent Done false
13971 4484 Acked Write 8 Done Sent Done false
13972 4485 Acked Write 6 Done Sent Done false
13973 4486 Acked Read 7 Done Sent Done false
13974 4487 Acked Read 2 Done Sent Done false
13975 4488 Acked Flush 0 Done Sent Done false
13976 4489 Acked Read 6 Done Sent Done false
13977 4490 Acked Write 2 Done Sent Done false
13978 4491 Acked Read 10 Done Sent Done false
13979 4492 Acked Write 9 Done Sent Done false
13980 4493 Acked Read 6 Done Sent Done false
13981 4494 Acked Write 9 Done Sent Done false
13982 4495 Acked Read 9 Done Sent Done false
13983 4496 Acked Read 4 Done Sent Done false
13984 4497 Acked Write 9 Done Sent Done false
13985 4498 Acked Read 10 Done Sent Done false
13986 4499 Acked Read 3 Done Sent Done false
13987 4500 Acked Read 6 Done Sent Done false
13988 4501 Acked Write 2 Done Sent Done false
13989 4502 Acked Write 9 Done Sent Done false
13990 4503 Acked Write 10 Done Sent Done false
13991 4504 Acked Write 3 Done Sent Done false
13992 4505 Acked Read 2 Done Sent Done false
13993 4506 Acked Write 9 Done Sent Done false
13994 4507 Acked Write 7 Done Sent Done false
13995 4508 Acked Write 5 Done Sent Done false
13996 4509 Acked Read 4 Done Sent Done false
13997 4510 Acked Read 2 Done Sent Done false
13998 4511 Acked Write 7 Done Sent Done false
13999 4512 Acked Write 7 Done Sent Done false
14000 4513 Acked Write 6 Done Sent Done false
14001 4514 Acked Read 5 Done Sent Done false
14002 4515 Acked Write 7 Done Sent Done false
14003 4516 Acked Read 2 Done Sent Done false
14004 4517 Acked Read 4 Done Sent Done false
14005 4518 Acked Read 2 Done Sent Done false
14006 4519 Acked Read 7 Done Sent Done false
14007 4520 Acked Write 8 Done Sent Done false
14008 4521 Acked Write 1 Done Sent Done false
14009 4522 Acked Write 4 Done Sent Done false
14010 4523 Acked Write 3 Done Sent Done false
14011 4524 Acked Write 4 Done Sent Done false
14012 4525 Acked Read 8 Done Sent Done false
14013 4526 Acked Read 2 Done Sent Done false
14014 4527 Acked Flush 0 Done Sent Done false
14015 4528 Acked Read 8 Done Sent Done false
14016 4529 Acked Read 1 Done Sent Done false
14017 4530 Acked Read 3 Done Sent Done false
14018 4531 Acked Write 9 Done Sent Done false
14019 4532 Acked Read 7 Done Sent Done false
14020 4533 Acked Write 5 Done Sent Done false
14021 4534 Acked Write 1 Done Sent Done false
14022 4535 Acked Read 3 Done Sent Done false
14023 4536 Acked Write 3 Done Sent Done false
14024 4537 Acked Read 4 Done Sent Done false
14025 4538 Acked Read 4 Done Sent Done false
14026 4539 Acked Write 4 Done Sent Done false
14027 4540 Acked Read 2 Done Sent Done false
14028 4541 Acked Write 8 Done Sent Done false
14029 4542 Acked Flush 0 Done Sent Done false
14030 4543 Acked Write 9 Done Sent Done false
14031 4544 Acked Write 7 Done Sent Done false
14032 4545 Acked Write 9 Done Sent Done false
14033 4546 Acked Read 1 Done Sent Done false
14034 4547 Acked Read 7 Done Sent Done false
14035 4548 Acked Write 5 Done Sent Done false
14036 4549 Acked Read 7 Done Sent Done false
14037 4550 Acked Read 2 Done Sent Done false
14038 4551 Acked Read 3 Done Sent Done false
14039 4552 Acked Write 7 Done Sent Done false
14040 4553 Acked Write 5 Done Sent Done false
14041 4554 Acked Read 3 Done Sent Done false
14042 4555 Acked Flush 0 Done Sent Done false
14043 4556 Acked Read 3 Done Sent Done false
14044 4557 Acked Write 7 Done Sent Done false
14045 4558 Acked Read 6 Done Sent Done false
14046 4559 Acked Write 2 Done Sent Done false
14047 4560 Acked Write 8 Done Sent Done false
14048 4561 Acked Write 5 Done Sent Done false
14049 4562 Acked Read 1 Done Sent Done false
14050 4563 Acked Read 10 Done Sent Done false
14051 4564 Acked Read 3 Done Sent Done false
14052 4565 Acked Read 4 Done Sent Done false
14053 4566 Acked Read 5 Done Sent Done false
14054 4567 Acked Read 8 Done Sent Done false
14055 4568 Acked Read 8 Done Sent Done false
14056 4569 Acked Read 6 Done Sent Done false
14057 4570 Acked Read 8 Done Sent Done false
14058 4571 Acked Write 4 Done Sent Done false
14059 4572 Acked Flush 0 Done Sent Done false
14060 4573 Acked Read 1 Done Sent Done false
14061 4574 Acked Write 9 Done Sent Done false
14062 4575 Acked Read 7 Done Sent Done false
14063 4576 Acked Write 4 Done Sent Done false
14064 4577 Acked Read 5 Done Sent Done false
14065 4578 Acked Read 1 Done Sent Done false
14066 4579 Acked Read 8 Done Sent Done false
14067 4580 Acked Read 2 Done Sent Done false
14068 4581 Acked Write 3 Done Sent Done false
14069 4582 Acked Write 1 Done Sent Done false
14070 4583 Acked Write 3 Done Sent Done false
14071 4584 Acked Read 10 Done Sent Done false
14072 4585 Acked Write 7 Done Sent Done false
14073 4586 Acked Write 7 Done Sent Done false
14074 4587 Acked Read 1 Done Sent Done false
14075 4588 Acked Flush 0 Done Sent Done false
14076 4589 Acked Write 1 Done Sent Done false
14077 4590 Acked Write 1 Done Sent Done false
14078 4591 Acked Read 5 Done Sent Done false
14079 4592 Acked Write 10 Done Sent Done false
14080 4593 Acked Read 9 Done Sent Done false
14081 4594 Acked Read 2 Done Sent Done false
14082 4595 Acked Read 10 Done Sent Done false
14083 4596 Acked Write 5 Done Sent Done false
14084 4597 Acked Read 2 Done Sent Done false
14085 4598 Acked Read 6 Done Sent Done false
14086 4599 Acked Read 3 Done Sent Done false
14087 4600 Acked Read 9 Done Sent Done false
14088 4601 Acked Read 4 Done Sent Done false
14089 4602 Acked Flush 0 Done Sent Done false
14090 4603 Acked Read 10 Done Sent Done false
14091 4604 Acked Write 2 Done Sent Done false
14092 4605 Acked Read 8 Done Sent Done false
14093 4606 Acked Write 8 Done Sent Done false
14094 4607 Acked Write 8 Done Sent Done false
14095 4608 Acked Read 7 Done Sent Done false
14096 4609 Acked Flush 0 Done Sent Done false
14097 4610 Acked Write 2 Done Sent Done false
14098 4611 Acked Write 3 Done Sent Done false
14099 4612 Acked Write 9 Done Sent Done false
14100 4613 Acked Write 9 Done Sent Done false
14101 4614 Acked Read 3 Done Sent Done false
14102 4615 Acked Write 8 Done Sent Done false
14103 4616 Acked Write 2 Done Sent Done false
14104 4617 Acked Read 6 Done Sent Done false
14105 4618 Acked Write 9 Done Sent Done false
14106 4619 Acked Write 4 Done Sent Done false
14107 4620 Acked Write 6 Done Sent Done false
14108 4621 Acked Read 9 Done Sent Done false
14109 4622 Acked Write 9 Done Sent Done false
14110 STATES DS:0 DS:1 DS:2 TOTAL
14111 Sent 0 301 0 301
14112 Done 301 0 301 602
14113 Skipped 0 0 0 0
14114 Error 0 0 0 0
14115 Last Flush: 4609 4321 4609
14116 Downstairs last five completed: 4321 4320 4319 4318 4317
14117 Upstairs last five completed: 4622 4621 4620 4619 4618
14118 CLIENT: Up:0 ds:301 act:2
14119 [8820] got start action
14120 [8820] I am going to restart
14121 Starting downstairs at port 8820
14122 Make output file at "/tmp/dsc/downstairs-8820.txt"
14123 [8820][1] reports Starting
14124 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
14125 [8820][1] reports Running
14126 ----------------------------------------------------------------
14127 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14128 States: Active Offline Active
14129 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14130 4322 Acked Read 3 Done Sent Done false
14131 4323 Acked Write 7 Done Sent Done false
14132 4324 Acked Write 10 Done Sent Done false
14133 4325 Acked Write 10 Done Sent Done false
14134 4326 Acked Flush 0 Done Sent Done false
14135 4327 Acked Read 10 Done Sent Done false
14136 4328 Acked Read 4 Done Sent Done false
14137 4329 Acked Flush 0 Done Sent Done false
14138 4330 Acked Write 5 Done Sent Done false
14139 4331 Acked Write 1 Done Sent Done false
14140 4332 Acked Read 7 Done Sent Done false
14141 4333 Acked Write 8 Done Sent Done false
14142 4334 Acked Read 2 Done Sent Done false
14143 4335 Acked Read 2 Done Sent Done false
14144 4336 Acked Read 4 Done Sent Done false
14145 4337 Acked Write 10 Done Sent Done false
14146 4338 Acked Write 2 Done Sent Done false
14147 4339 Acked Read 3 Done Sent Done false
14148 4340 Acked Read 2 Done Sent Done false
14149 4341 Acked Read 3 Done Sent Done false
14150 4342 Acked Write 3 Done Sent Done false
14151 4343 Acked Read 1 Done Sent Done false
14152 4344 Acked Read 9 Done Sent Done false
14153 4345 Acked Write 8 Done Sent Done false
14154 4346 Acked Read 10 Done Sent Done false
14155 4347 Acked Read 1 Done Sent Done false
14156 4348 Acked Write 9 Done Sent Done false
14157 4349 Acked Write 9 Done Sent Done false
14158 4350 Acked Write 4 Done Sent Done false
14159 4351 Acked Read 10 Done Sent Done false
14160 4352 Acked Read 2 Done Sent Done false
14161 4353 Acked Write 1 Done Sent Done false
14162 4354 Acked Read 5 Done Sent Done false
14163 4355 Acked Write 2 Done Sent Done false
14164 4356 Acked Read 5 Done Sent Done false
14165 4357 Acked Write 2 Done Sent Done false
14166 4358 Acked Read 8 Done Sent Done false
14167 4359 Acked Write 9 Done Sent Done false
14168 4360 Acked Write 8 Done Sent Done false
14169 4361 Acked Write 5 Done Sent Done false
14170 4362 Acked Read 6 Done Sent Done false
14171 4363 Acked Read 6 Done Sent Done false
14172 4364 Acked Write 4 Done Sent Done false
14173 4365 Acked Read 4 Done Sent Done false
14174 4366 Acked Read 9 Done Sent Done false
14175 4367 Acked Read 6 Done Sent Done false
14176 4368 Acked Write 5 Done Sent Done false
14177 4369 Acked Read 6 Done Sent Done false
14178 4370 Acked Write 1 Done Sent Done false
14179 4371 Acked Flush 0 Done Sent Done false
14180 4372 Acked Read 8 Done Sent Done false
14181 4373 Acked Read 8 Done Sent Done false
14182 4374 Acked Write 9 Done Sent Done false
14183 4375 Acked Write 1 Done Sent Done false
14184 4376 Acked Write 3 Done Sent Done false
14185 4377 Acked Read 10 Done Sent Done false
14186 4378 Acked Read 6 Done Sent Done false
14187 4379 Acked Read 1 Done Sent Done false
14188 4380 Acked Read 7 Done Sent Done false
14189 4381 Acked Write 1 Done Sent Done false
14190 4382 Acked Write 9 Done Sent Done false
14191 4383 Acked Write 7 Done Sent Done false
14192 4384 Acked Read 4 Done Sent Done false
14193 4385 Acked Write 9 Done Sent Done false
14194 4386 Acked Write 5 Done Sent Done false
14195 4387 Acked Write 2 Done Sent Done false
14196 4388 Acked Read 7 Done Sent Done false
14197 4389 Acked Read 10 Done Sent Done false
14198 4390 Acked Write 8 Done Sent Done false
14199 4391 Acked Write 1 Done Sent Done false
14200 4392 Acked Write 1 Done Sent Done false
14201 4393 Acked Write 9 Done Sent Done false
14202 4394 Acked Flush 0 Done Sent Done false
14203 4395 Acked Read 3 Done Sent Done false
14204 4396 Acked Read 5 Done Sent Done false
14205 4397 Acked Write 3 Done Sent Done false
14206 4398 Acked Read 3 Done Sent Done false
14207 4399 Acked Read 1 Done Sent Done false
14208 4400 Acked Read 5 Done Sent Done false
14209 4401 Acked Write 5 Done Sent Done false
14210 4402 Acked Write 4 Done Sent Done false
14211 4403 Acked Flush 0 Done Sent Done false
14212 4404 Acked Flush 0 Done Sent Done false
14213 4405 Acked Read 5 Done Sent Done false
14214 4406 Acked Write 9 Done Sent Done false
14215 4407 Acked Write 10 Done Sent Done false
14216 4408 Acked Write 10 Done Sent Done false
14217 4409 Acked Write 3 Done Sent Done false
14218 4410 Acked Flush 0 Done Sent Done false
14219 4411 Acked Read 6 Done Sent Done false
14220 4412 Acked Write 3 Done Sent Done false
14221 4413 Acked Read 10 Done Sent Done false
14222 4414 Acked Read 9 Done Sent Done false
14223 4415 Acked Write 1 Done Sent Done false
14224 4416 Acked Read 7 Done Sent Done false
14225 4417 Acked Write 6 Done Sent Done false
14226 4418 Acked Write 3 Done Sent Done false
14227 4419 Acked Read 6 Done Sent Done false
14228 4420 Acked Read 7 Done Sent Done false
14229 4421 Acked Read 3 Done Sent Done false
14230 4422 Acked Read 7 Done Sent Done false
14231 4423 Acked Write 3 Done Sent Done false
14232 4424 Acked Read 10 Done Sent Done false
14233 4425 Acked Flush 0 Done Sent Done false
14234 4426 Acked Read 5 Done Sent Done false
14235 4427 Acked Write 4 Done Sent Done false
14236 4428 Acked Read 6 Done Sent Done false
14237 4429 Acked Flush 0 Done Sent Done false
14238 4430 Acked Write 3 Done Sent Done false
14239 4431 Acked Read 10 Done Sent Done false
14240 4432 Acked Flush 0 Done Sent Done false
14241 4433 Acked Flush 0 Done Sent Done false
14242 4434 Acked Write 5 Done Sent Done false
14243 4435 Acked Write 6 Done Sent Done false
14244 4436 Acked Read 9 Done Sent Done false
14245 4437 Acked Read 4 Done Sent Done false
14246 4438 Acked Write 6 Done Sent Done false
14247 4439 Acked Write 8 Done Sent Done false
14248 4440 Acked Write 6 Done Sent Done false
14249 4441 Acked Write 9 Done Sent Done false
14250 4442 Acked Read 6 Done Sent Done false
14251 4443 Acked Read 6 Done Sent Done false
14252 4444 Acked Read 10 Done Sent Done false
14253 4445 Acked Flush 0 Done Sent Done false
14254 4446 Acked Write 3 Done Sent Done false
14255 4447 Acked Read 4 Done Sent Done false
14256 4448 Acked Write 5 Done Sent Done false
14257 4449 Acked Read 7 Done Sent Done false
14258 4450 Acked Write 6 Done Sent Done false
14259 4451 Acked Write 1 Done Sent Done false
14260 4452 Acked Read 6 Done Sent Done false
14261 4453 Acked Write 1 Done Sent Done false
14262 4454 Acked Flush 0 Done Sent Done false
14263 4455 Acked Read 8 Done Sent Done false
14264 4456 Acked Read 1 Done Sent Done false
14265 4457 Acked Read 7 Done Sent Done false
14266 4458 Acked Read 3 Done Sent Done false
14267 4459 Acked Write 7 Done Sent Done false
14268 4460 Acked Read 4 Done Sent Done false
14269 4461 Acked Write 7 Done Sent Done false
14270 4462 Acked Write 1 Done Sent Done false
14271 4463 Acked Write 7 Done Sent Done false
14272 4464 Acked Read 6 Done Sent Done false
14273 4465 Acked Read 10 Done Sent Done false
14274 4466 Acked Read 9 Done Sent Done false
14275 4467 Acked Read 3 Done Sent Done false
14276 4468 Acked Write 8 Done Sent Done false
14277 4469 Acked Flush 0 Done Sent Done false
14278 4470 Acked Flush 0 Done Sent Done false
14279 4471 Acked Write 3 Done Sent Done false
14280 4472 Acked Read 5 Done Sent Done false
14281 4473 Acked Flush 0 Done Sent Done false
14282 4474 Acked Read 7 Done Sent Done false
14283 4475 Acked Read 10 Done Sent Done false
14284 4476 Acked Read 5 Done Sent Done false
14285 4477 Acked Write 2 Done Sent Done false
14286 4478 Acked Flush 0 Done Sent Done false
14287 4479 Acked Read 5 Done Sent Done false
14288 4480 Acked Write 4 Done Sent Done false
14289 4481 Acked Flush 0 Done Sent Done false
14290 4482 Acked Read 9 Done Sent Done false
14291 4483 Acked Read 5 Done Sent Done false
14292 4484 Acked Write 8 Done Sent Done false
14293 4485 Acked Write 6 Done Sent Done false
14294 4486 Acked Read 7 Done Sent Done false
14295 4487 Acked Read 2 Done Sent Done false
14296 4488 Acked Flush 0 Done Sent Done false
14297 4489 Acked Read 6 Done Sent Done false
14298 4490 Acked Write 2 Done Sent Done false
14299 4491 Acked Read 10 Done Sent Done false
14300 4492 Acked Write 9 Done Sent Done false
14301 4493 Acked Read 6 Done Sent Done false
14302 4494 Acked Write 9 Done Sent Done false
14303 4495 Acked Read 9 Done Sent Done false
14304 4496 Acked Read 4 Done Sent Done false
14305 4497 Acked Write 9 Done Sent Done false
14306 4498 Acked Read 10 Done Sent Done false
14307 4499 Acked Read 3 Done Sent Done false
14308 4500 Acked Read 6 Done Sent Done false
14309 4501 Acked Write 2 Done Sent Done false
14310 4502 Acked Write 9 Done Sent Done false
14311 4503 Acked Write 10 Done Sent Done false
14312 4504 Acked Write 3 Done Sent Done false
14313 4505 Acked Read 2 Done Sent Done false
14314 4506 Acked Write 9 Done Sent Done false
14315 4507 Acked Write 7 Done Sent Done false
14316 4508 Acked Write 5 Done Sent Done false
14317 4509 Acked Read 4 Done Sent Done false
14318 4510 Acked Read 2 Done Sent Done false
14319 4511 Acked Write 7 Done Sent Done false
14320 4512 Acked Write 7 Done Sent Done false
14321 4513 Acked Write 6 Done Sent Done false
14322 4514 Acked Read 5 Done Sent Done false
14323 4515 Acked Write 7 Done Sent Done false
14324 4516 Acked Read 2 Done Sent Done false
14325 4517 Acked Read 4 Done Sent Done false
14326 4518 Acked Read 2 Done Sent Done false
14327 4519 Acked Read 7 Done Sent Done false
14328 4520 Acked Write 8 Done Sent Done false
14329 4521 Acked Write 1 Done Sent Done false
14330 4522 Acked Write 4 Done Sent Done false
14331 4523 Acked Write 3 Done Sent Done false
14332 4524 Acked Write 4 Done Sent Done false
14333 4525 Acked Read 8 Done Sent Done false
14334 4526 Acked Read 2 Done Sent Done false
14335 4527 Acked Flush 0 Done Sent Done false
14336 4528 Acked Read 8 Done Sent Done false
14337 4529 Acked Read 1 Done Sent Done false
14338 4530 Acked Read 3 Done Sent Done false
14339 4531 Acked Write 9 Done Sent Done false
14340 4532 Acked Read 7 Done Sent Done false
14341 4533 Acked Write 5 Done Sent Done false
14342 4534 Acked Write 1 Done Sent Done false
14343 4535 Acked Read 3 Done Sent Done false
14344 4536 Acked Write 3 Done Sent Done false
14345 4537 Acked Read 4 Done Sent Done false
14346 4538 Acked Read 4 Done Sent Done false
14347 4539 Acked Write 4 Done Sent Done false
14348 4540 Acked Read 2 Done Sent Done false
14349 4541 Acked Write 8 Done Sent Done false
14350 4542 Acked Flush 0 Done Sent Done false
14351 4543 Acked Write 9 Done Sent Done false
14352 4544 Acked Write 7 Done Sent Done false
14353 4545 Acked Write 9 Done Sent Done false
14354 4546 Acked Read 1 Done Sent Done false
14355 4547 Acked Read 7 Done Sent Done false
14356 4548 Acked Write 5 Done Sent Done false
14357 4549 Acked Read 7 Done Sent Done false
14358 4550 Acked Read 2 Done Sent Done false
14359 4551 Acked Read 3 Done Sent Done false
14360 4552 Acked Write 7 Done Sent Done false
14361 4553 Acked Write 5 Done Sent Done false
14362 4554 Acked Read 3 Done Sent Done false
14363 4555 Acked Flush 0 Done Sent Done false
14364 4556 Acked Read 3 Done Sent Done false
14365 4557 Acked Write 7 Done Sent Done false
14366 4558 Acked Read 6 Done Sent Done false
14367 4559 Acked Write 2 Done Sent Done false
14368 4560 Acked Write 8 Done Sent Done false
14369 4561 Acked Write 5 Done Sent Done false
14370 4562 Acked Read 1 Done Sent Done false
14371 4563 Acked Read 10 Done Sent Done false
14372 4564 Acked Read 3 Done Sent Done false
14373 4565 Acked Read 4 Done Sent Done false
14374 4566 Acked Read 5 Done Sent Done false
14375 4567 Acked Read 8 Done Sent Done false
14376 4568 Acked Read 8 Done Sent Done false
14377 4569 Acked Read 6 Done Sent Done false
14378 4570 Acked Read 8 Done Sent Done false
14379 4571 Acked Write 4 Done Sent Done false
14380 4572 Acked Flush 0 Done Sent Done false
14381 4573 Acked Read 1 Done Sent Done false
14382 4574 Acked Write 9 Done Sent Done false
14383 4575 Acked Read 7 Done Sent Done false
14384 4576 Acked Write 4 Done Sent Done false
14385 4577 Acked Read 5 Done Sent Done false
14386 4578 Acked Read 1 Done Sent Done false
14387 4579 Acked Read 8 Done Sent Done false
14388 4580 Acked Read 2 Done Sent Done false
14389 4581 Acked Write 3 Done Sent Done false
14390 4582 Acked Write 1 Done Sent Done false
14391 4583 Acked Write 3 Done Sent Done false
14392 4584 Acked Read 10 Done Sent Done false
14393 4585 Acked Write 7 Done Sent Done false
14394 4586 Acked Write 7 Done Sent Done false
14395 4587 Acked Read 1 Done Sent Done false
14396 4588 Acked Flush 0 Done Sent Done false
14397 4589 Acked Write 1 Done Sent Done false
14398 4590 Acked Write 1 Done Sent Done false
14399 4591 Acked Read 5 Done Sent Done false
14400 4592 Acked Write 10 Done Sent Done false
14401 4593 Acked Read 9 Done Sent Done false
14402 4594 Acked Read 2 Done Sent Done false
14403 4595 Acked Read 10 Done Sent Done false
14404 4596 Acked Write 5 Done Sent Done false
14405 4597 Acked Read 2 Done Sent Done false
14406 4598 Acked Read 6 Done Sent Done false
14407 4599 Acked Read 3 Done Sent Done false
14408 4600 Acked Read 9 Done Sent Done false
14409 4601 Acked Read 4 Done Sent Done false
14410 4602 Acked Flush 0 Done Sent Done false
14411 4603 Acked Read 10 Done Sent Done false
14412 4604 Acked Write 2 Done Sent Done false
14413 4605 Acked Read 8 Done Sent Done false
14414 4606 Acked Write 8 Done Sent Done false
14415 4607 Acked Write 8 Done Sent Done false
14416 4608 Acked Read 7 Done Sent Done false
14417 4609 Acked Flush 0 Done Sent Done false
14418 4610 Acked Write 2 Done Sent Done false
14419 4611 Acked Write 3 Done Sent Done false
14420 4612 Acked Write 9 Done Sent Done false
14421 4613 Acked Write 9 Done Sent Done false
14422 4614 Acked Read 3 Done Sent Done false
14423 4615 Acked Write 8 Done Sent Done false
14424 4616 Acked Write 2 Done Sent Done false
14425 4617 Acked Read 6 Done Sent Done false
14426 4618 Acked Write 9 Done Sent Done false
14427 4619 Acked Write 4 Done Sent Done false
14428 4620 Acked Write 6 Done Sent Done false
14429 4621 Acked Read 9 Done Sent Done false
14430 4622 Acked Write 9 Done Sent Done false
14431 4623 Acked Flush 0 Done Sent Done false
14432 STATES DS:0 DS:1 DS:2 TOTAL
14433 Sent 0 302 0 302
14434 Done 302 0 302 604
14435 Skipped 0 0 0 0
14436 Error 0 0 0 0
14437 Last Flush: 4623 4321 4623
14438 Downstairs last five completed: 4321 4320 4319 4318 4317
14439 Upstairs last five completed: 4623 4622 4621 4620 4619
14440 CLIENT: Up:0 ds:302 act:2
144412024-12-11T23:43:31.232ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144422024-12-11T23:43:31.232ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37250) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144432024-12-11T23:43:31.233ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144442024-12-11T23:43:31.234ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144452024-12-11T23:43:31.234ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144462024-12-11T23:43:31.234ZINFOcrucible: send last flush ID to this DS: 4321 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144472024-12-11T23:43:31.234ZINFOcrucible: Replied this last flush ID: 4321 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144482024-12-11T23:43:31.234ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144492024-12-11T23:43:31.234ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144502024-12-11T23:43:31.234ZINFOcrucible: [1] client re-new 302 jobs since flush 4321 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144512024-12-11T23:43:31.234ZINFOcrucible: [1] Replayed 302 jobs since flush: 4321 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
14452 ----------------------------------------------------------------
14453 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14454 Downstairs last five completed: 4623 4622 4621 4620 4619
14455 Upstairs last five completed: 4623 4622 4621 4620 4619
14456 CLIENT: Up:0 ds:0 act:3
14457 Replay: All jobs finished
14458 Main task has work to do, go find it
14459 got dsc Stop(1)
14460 stop 1
14461 Dec 11 23:43:33.472 INFO [8820] Got stop action so:false kr:false
14462 request completed, latency_us: 342, response_code: 204, uri: /stop/cid/1, method: POST, req_id: cc7019f7-d4f9-4fbd-b664-0f3b02fd721d, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
14463 Dec 11 23:43:33.472 INFO request completed, latency_us: 122, response_code: 200, uri: /state/cid/1, method: GET, req_id: fa023932-9da1-4773-91a0-72c95d9c07d5, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
144642024-12-11T23:43:33.472ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144652024-12-11T23:43:33.472ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144662024-12-11T23:43:33.472ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144672024-12-11T23:43:33.472ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144682024-12-11T23:43:33.472ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
144692024-12-11T23:43:33.473ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
14470 [8820] Exited with: ExitStatus(unix_wait_status(9))
14471 [8820][1] reports Exit
14472 Dec 11 23:43:37.475 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/1, method: GET, req_id: 315d6bcb-cd61-43e4-9b20-e7e677eb281a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
14473 001/300 Write block 2765 len 32768 data: 3 3 3 3 3 2 2 2
14474 002/300 Read block 1949 len 28672
14475 003/300 Write block 3345 len 28672 data: 4 2 2 2 2 2 2
14476 004/300 Read block 4447 len 8192
14477 005/300 Read block 282 len 32768
14478 006/300 Read block 4438 len 16384
14479 007/300 Flush
14480 008/300 Read block 583 len 4096
14481 009/300 Read block 2763 len 8192
14482 010/300 Write block 248 len 36864 data: 2 2 3 3 3 3 3 3 2
14483 011/300 Write block 998 len 16384 data: 3 3 3 4
14484 012/300 Read block 2744 len 12288
14485 013/300 Write block 1017 len 24576 data: 3 3 3 2 3 3
14486 014/300 Read block 1846 len 16384
14487 015/300 Read block 3698 len 8192
14488 016/300 Read block 2873 len 16384
14489 017/300 Read block 1686 len 40960
14490 018/300 Read block 1658 len 8192
14491 019/300 Read block 1593 len 28672
14492 020/300 Write block 1669 len 16384 data: 3 3 2 2
14493 021/300 Read block 521 len 36864
14494 022/300 Read block 3485 len 16384
14495 023/300 Read block 1757 len 40960
14496 024/300 Read block 3521 len 16384
14497 025/300 Write block 917 len 36864 data: 2 2 2 2 2 2 2 2 2
14498 026/300 Write block 4438 len 32768 data: 4 4 3 4 4 4 4 3
14499 027/300 Flush
14500 028/300 Write block 145 len 12288 data: 4 2 3
14501 029/300 Read block 960 len 32768
14502 030/300 Read block 3353 len 24576
14503 031/300 Read block 2176 len 24576
14504 032/300 Read block 1369 len 28672
14505 033/300 Read block 4723 len 12288
14506 034/300 Flush
14507 035/300 Read block 1518 len 32768
14508 036/300 Read block 4045 len 12288
14509 037/300 Write block 3400 len 28672 data: 3 2 2 2 2 2 2
14510 038/300 Read block 3867 len 40960
14511 039/300 Read block 4096 len 4096
14512 040/300 Write block 4864 len 4096 data: 3
14513 041/300 Flush
14514 042/300 Flush
14515 043/300 Read block 4888 len 4096
14516 044/300 Read block 4500 len 16384
14517 045/300 Write block 1518 len 32768 data: 4 4 3 3 3 3 2 3
14518 046/300 Write block 4240 len 4096 data: 4
14519 047/300 Write block 2092 len 36864 data: 4 5 7 6 5 5 6 5 4
14520 048/300 Write block 68 len 16384 data: 4 5 4 4
14521 049/300 Write block 115 len 36864 data: 4 4 4 5 5 4 4 4 5
14522 050/300 Read block 1743 len 16384
14523 051/300 Write block 477 len 28672 data: 2 2 2 2 2 3 4
14524 052/300 Read block 3646 len 4096
14525 053/300 Read block 3869 len 32768
14526 054/300 Write block 2923 len 28672 data: 3 3 2 3 3 3 3
14527 055/300 Write block 3958 len 28672 data: 3 4 7 7 8 7 7
14528 056/300 Write block 991 len 12288 data: 2 2 2
14529 057/300 Write block 1847 len 12288 data: 3 2 2
14530 058/300 Read block 2387 len 12288
14531 059/300 Read block 4074 len 20480
14532 060/300 Read block 728 len 20480
14533 061/300 Flush
14534 062/300 Flush
14535 063/300 Read block 3253 len 12288
14536 064/300 Write block 2737 len 32768 data: 6 5 4 4 4 4 3 4
14537 065/300 Write block 22 len 36864 data: 4 3 3 4 3 3 2 2 3
14538 066/300 Write block 4426 len 32768 data: 4 4 5 5 2 2 2 4
14539 067/300 Read block 2515 len 4096
14540 068/300 Read block 4575 len 24576
14541 069/300 Write block 4910 len 24576 data: 3 4 4 4 5 3
14542 070/300 Read block 1473 len 8192
14543 071/300 Flush
14544 072/300 Read block 1544 len 12288
14545 073/300 Read block 4324 len 32768
14546 074/300 Write block 3294 len 12288 data: 2 2 2
14547 075/300 Read block 837 len 20480
14548 076/300 Write block 3758 len 16384 data: 3 3 3 3
14549 077/300 Read block 3881 len 36864
14550 078/300 Read block 2351 len 32768
14551 079/300 Flush
14552 080/300 Read block 1643 len 36864
14553 081/300 Flush
14554 082/300 Read block 4248 len 24576
14555 083/300 Read block 3960 len 28672
14556 084/300 Read block 4491 len 24576
14557 085/300 Write block 2037 len 20480 data: 4 5 5 4 5
14558 086/300 Write block 4956 len 16384 data: 5 5 6 5
14559 087/300 Read block 2143 len 8192
14560 088/300 Read block 2315 len 24576
14561 089/300 Read block 1745 len 36864
14562 090/300 Read block 920 len 4096
14563 091/300 Write block 4101 len 40960 data: 2 2 4 4 3 3 3 3 3 3
14564 092/300 Read block 506 len 4096
14565 093/300 Write block 1986 len 4096 data: 7
14566 094/300 Read block 1370 len 32768
14567 095/300 Write block 1064 len 8192 data: 4 5
14568 096/300 Flush
14569 097/300 Read block 238 len 16384
14570 098/300 Read block 1078 len 16384
14571 099/300 Read block 2388 len 4096
14572 100/300 Write block 106 len 36864 data: 3 2 2 2 2 2 2 2 3
14573 101/300 Write block 4678 len 32768 data: 4 4 4 3 3 3 3 3
14574 102/300 Write block 3115 len 24576 data: 3 3 3 4 4 4
14575 103/300 Write block 1840 len 24576 data: 2 2 3 2 2 2
14576 104/300 Write block 4753 len 12288 data: 4 3 3
14577 105/300 Read block 4878 len 8192
14578 106/300 Write block 2823 len 8192 data: 4 4
14579 107/300 Read block 2107 len 28672
14580 108/300 Write block 3577 len 36864 data: 4 4 4 4 5 3 3 3 4
14581 109/300 Write block 1005 len 36864 data: 3 4 4 4 4 4 4 3 3
14582 110/300 Write block 182 len 32768 data: 4 5 4 5 5 5 4 4
14583 111/300 Read block 21 len 20480
14584 112/300 Read block 2462 len 28672
14585 113/300 Read block 2587 len 12288
14586 114/300 Write block 4902 len 40960 data: 7 6 7 8 6 5 4 3 4 5
14587 115/300 Write block 1971 len 4096 data: 2
14588 116/300 Read block 3811 len 8192
14589 117/300 Read block 1126 len 4096
14590 118/300 Read block 1270 len 20480
14591 119/300 Read block 943 len 40960
14592 120/300 Write block 3525 len 16384 data: 4 4 6 6
14593 121/300 Read block 2802 len 36864
14594 122/300 Flush
14595 123/300 Read block 2873 len 24576
14596 124/300 Read block 301 len 12288
14597 125/300 Read block 934 len 16384
14598 126/300 Read block 1113 len 20480
14599 127/300 Read block 1036 len 24576
14600 128/300 Write block 474 len 28672 data: 2 2 2 3 3 3 3
14601 129/300 Read block 661 len 4096
14602 130/300 Read block 65 len 16384
14603 131/300 Flush
14604 132/300 Write block 1693 len 28672 data: 5 4 4 5 5 5 3
14605 133/300 Write block 3009 len 16384 data: 5 3 5 5
14606 134/300 Write block 3592 len 28672 data: 3 4 3 2 2 2 2
14607 135/300 Write block 611 len 16384 data: 3 3 3 4
14608 136/300 Write block 1682 len 36864 data: 3 4 5 4 4 5 5 5 5
14609 137/300 Write block 2153 len 32768 data: 3 3 4 4 4 4 4 3
14610 138/300 Read block 2619 len 8192
14611 139/300 Read block 1361 len 24576
14612 140/300 Read block 2987 len 32768
14613 141/300 Write block 3540 len 16384 data: 4 5 5 5
14614 142/300 Write block 2181 len 4096 data: 3
14615 143/300 Write block 685 len 32768 data: 5 3 3 3 3 3 2 4
14616 144/300 Read block 1754 len 20480
14617 145/300 Read block 1133 len 8192
14618 146/300 Read block 3762 len 16384
14619 147/300 Write block 1715 len 4096 data: 2
14620 148/300 Write block 4462 len 24576 data: 3 4 5 4 4 4
14621 149/300 Write block 1245 len 16384 data: 2 5 4 5
14622 150/300 Write block 3934 len 24576 data: 2 3 3 3 3 4
14623 151/300 Read block 911 len 32768
14624 152/300 Read block 809 len 12288
14625 153/300 Write block 4451 len 12288 data: 2 2 2
14626 154/300 Flush
14627 155/300 Read block 1841 len 32768
14628 156/300 Read block 1971 len 40960
14629 157/300 Flush
14630 158/300 Write block 4680 len 20480 data: 5 4 4 4 4
14631 159/300 Write block 4468 len 8192 data: 4 7
14632 160/300 Write block 286 len 36864 data: 4 4 4 3 4 3 2 3 3
14633 161/300 Read block 724 len 32768
14634 162/300 Read block 3732 len 24576
14635 163/300 Read block 2830 len 20480
14636 164/300 Flush
14637 165/300 Read block 4575 len 16384
14638 166/300 Read block 3535 len 4096
14639 167/300 Read block 992 len 40960
14640 168/300 Write block 3857 len 32768 data: 6 6 6 6 7 8 7 7
14641 169/300 Read block 2216 len 24576
14642 170/300 Read block 810 len 40960
14643 171/300 Write block 4333 len 4096 data: 2
14644 172/300 Read block 477 len 12288
14645 173/300 Flush
14646 174/300 Read block 2177 len 36864
14647 175/300 Flush
14648 176/300 Write block 331 len 8192 data: 5 4
14649 177/300 Read block 1412 len 40960
14650 178/300 Read block 2929 len 16384
14651 179/300 Write block 1744 len 12288 data: 3 4 4
14652 180/300 Write block 3621 len 4096 data: 5
14653 181/300 Read block 3214 len 8192
14654 182/300 Read block 988 len 20480
14655 183/300 Write block 1915 len 32768 data: 3 3 3 2 2 2 2 3
14656 184/300 Write block 1900 len 8192 data: 2 2
14657 185/300 Write block 1785 len 32768 data: 3 3 2 3 3 3 3 4
14658 186/300 Write block 3862 len 16384 data: 9 8 8 6
14659 187/300 Write block 2068 len 8192 data: 4 4
14660 188/300 Read block 1497 len 8192
14661 189/300 Read block 493 len 4096
14662 190/300 Write block 655 len 4096 data: 3
14663 191/300 Read block 3725 len 32768
14664 192/300 Read block 3310 len 32768
14665 193/300 Write block 1428 len 16384 data: 3 4 3 3
14666 194/300 Read block 2502 len 24576
14667 195/300 Flush
14668 196/300 Write block 1780 len 32768 data: 2 3 3 3 3 4 4 3
14669 197/300 Write block 2121 len 36864 data: 3 4 4 3 4 6 5 5 5
14670 198/300 Read block 1348 len 24576
14671 199/300 Read block 2570 len 24576
14672 200/300 Read block 2099 len 12288
14673 201/300 Read block 3438 len 40960
14674 202/300 Read block 1247 len 8192
14675 203/300 Write block 3093 len 32768 data: 8 8 7 6 5 5 5 6
14676 204/300 Read block 4379 len 8192
14677 205/300 Write block 1971 len 20480 data: 3 2 2 2 3
14678 206/300 Flush
14679 207/300 Write block 1311 len 20480 data: 6 6 5 3 4
14680 208/300 Write block 3215 len 12288 data: 4 4 4
14681 209/300 Write block 3814 len 4096 data: 5
14682 210/300 Write block 4602 len 12288 data: 3 3 3
14683 211/300 Read block 591 len 12288
14684 212/300 Write block 4057 len 24576 data: 3 3 4 5 5 5
14685 213/300 Read block 739 len 36864
14686 214/300 Read block 2925 len 36864
14687 215/300 Write block 3571 len 12288 data: 2 2 4
14688 216/300 Flush
14689 217/300 Read block 3610 len 36864
14690 218/300 Write block 1067 len 8192 data: 4 4
14691 219/300 Flush
14692 220/300 Read block 3598 len 8192
14693 221/300 Flush
14694 222/300 Write block 2057 len 4096 data: 3
14695 223/300 Write block 3389 len 12288 data: 4 3 3
14696 224/300 Write block 867 len 36864 data: 3 3 3 3 3 3 4 4 5
14697 225/300 Read block 2807 len 40960
14698 226/300 Flush
14699 227/300 Read block 4487 len 12288
14700 228/300 Read block 296 len 4096
14701 229/300 Write block 3865 len 4096 data: 7
14702 230/300 Write block 2439 len 12288 data: 6 6 6
14703 231/300 Write block 2192 len 24576 data: 4 4 5 5 5 5
14704 232/300 Read block 1901 len 40960
14705 233/300 Write block 4601 len 32768 data: 3 4 4 4 3 3 3 3
14706 234/300 Read block 4756 len 32768
14707 235/300 Read block 2627 len 8192
14708 236/300 Write block 2704 len 8192 data: 5 4
14709 237/300 Read block 4468 len 36864
14710 238/300 Write block 296 len 16384 data: 3 4 5 5
14711 239/300 Read block 1578 len 16384
14712 240/300 Read block 2898 len 8192
14713 241/300 Read block 4580 len 24576
14714 242/300 Flush
14715 243/300 Write block 1429 len 20480 data: 5 4 4 3 3
14716 244/300 Flush
14717 245/300 Flush
14718 246/300 Write block 1670 len 16384 data: 4 3 3 2
14719 247/300 Write block 3237 len 40960 data: 3 2 2 2 2 3 4 4 3 3
14720 248/300 Flush
14721 249/300 Read block 1622 len 8192
14722 250/300 Write block 410 len 8192 data: 6 5
14723 251/300 Write block 2802 len 32768 data: 6 5 6 4 5 4 4 4
14724 252/300 Write block 26 len 4096 data: 4
14725 253/300 Read block 4649 len 32768
14726 254/300 Flush
14727 255/300 Flush
14728 256/300 Write block 1956 len 40960 data: 2 2 2 2 2 3 3 3 3 3
14729 257/300 Read block 828 len 32768
14730 258/300 Flush
14731 259/300 Write block 4529 len 24576 data: 6 8 7 7 4 4
14732 260/300 Write block 2127 len 8192 data: 6 6
14733 261/300 Flush
14734 262/300 Read block 1179 len 8192
14735 263/300 Read block 4044 len 8192
14736 264/300 Read block 1554 len 36864
14737 265/300 Write block 3338 len 36864 data: 3 3 2 2 2 2 2 5 3
14738 266/300 Read block 727 len 8192
14739 267/300 Read block 4034 len 28672
14740 268/300 Read block 3580 len 36864
14741 269/300 Read block 1999 len 32768
14742 270/300 Read block 1111 len 20480
14743 271/300 Read block 3789 len 12288
14744 272/300 Read block 2952 len 40960
14745 273/300 Write block 2106 len 28672 data: 4 3 3 4 3 3 3
14746 274/300 Flush
14747 275/300 Write block 2344 len 36864 data: 2 2 4 4 3 3 3 2 2
14748 276/300 Flush
14749 277/300 Write block 759 len 36864 data: 5 5 4 3 4 3 3 4 4
14750 278/300 Read block 3477 len 20480
14751 279/300 Write block 4997 len 12288 data: 4 3 2
14752 280/300 Flush
14753 281/300 Read block 3997 len 28672
14754 282/300 Write block 4532 len 36864 data: 8 5 5 3 4 5 5 5 5
14755 283/300 Write block 93 len 20480 data: 2 2 2 2 2
14756 284/300 Flush
14757 285/300 Flush
14758 286/300 Read block 3024 len 12288
14759 287/300 Read block 569 len 28672
14760 288/300 Write block 4251 len 28672 data: 2 2 4 4 4 5 3
14761 289/300 Write block 2852 len 8192 data: 4 4
14762 290/300 Read block 4219 len 12288
14763 291/300 Read block 1113 len 28672
14764 292/300 Read block 3362 len 4096
14765 293/300 Flush
14766 294/300 Flush
14767 295/300 Flush
14768 296/300 Read block 992 len 40960
14769 297/300 Flush
14770 298/300 Flush
14771 299/300 Read block 1602 len 40960
14772 300/300 Read block 4301 len 4096
14773 301/300 Write block 3764 len 28672 data: 2 2 2 2 3 3 3
14774 Main task has work to do, go find it
14775 got dsc Start(1)
14776 start 1
14777 Dec 11 23:43:37.702 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/1, method: POST, req_id: 28769f93-54b1-4ac2-ad31-4df954d1c191, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
14778 Replay: started 1, returned:Ok(())
14779 ----------------------------------------------------------------
14780 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14781 States: Active Offline Active
14782 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14783 4624 Acked Write 8 Done Sent Done false
14784 4625 Acked Read 7 Done Sent Done false
14785 4626 Acked Write 7 Done Sent Done false
14786 4627 Acked Read 2 Done Sent Done false
14787 4628 Acked Read 8 Done Sent Done false
14788 4629 Acked Read 4 Done Sent Done false
14789 4630 Acked Flush 0 Done Sent Done false
14790 4631 Acked Read 1 Done Sent Done false
14791 4632 Acked Read 2 Done Sent Done false
14792 4633 Acked Write 9 Done Sent Done false
14793 4634 Acked Write 4 Done Sent Done false
14794 4635 Acked Read 3 Done Sent Done false
14795 4636 Acked Write 6 Done Sent Done false
14796 4637 Acked Read 4 Done Sent Done false
14797 4638 Acked Read 2 Done Sent Done false
14798 4639 Acked Read 4 Done Sent Done false
14799 4640 Acked Read 10 Done Sent Done false
14800 4641 Acked Read 2 Done Sent Done false
14801 4642 Acked Read 7 Done Sent Done false
14802 4643 Acked Write 4 Done Sent Done false
14803 4644 Acked Read 9 Done Sent Done false
14804 4645 Acked Read 4 Done Sent Done false
14805 4646 Acked Read 10 Done Sent Done false
14806 4647 Acked Read 4 Done Sent Done false
14807 4648 Acked Write 9 Done Sent Done false
14808 4649 Acked Write 8 Done Sent Done false
14809 4650 Acked Flush 0 Done Sent Done false
14810 4651 Acked Write 3 Done Sent Done false
14811 4652 Acked Read 8 Done Sent Done false
14812 4653 Acked Read 6 Done Sent Done false
14813 4654 Acked Read 6 Done Sent Done false
14814 4655 Acked Read 7 Done Sent Done false
14815 4656 Acked Read 3 Done Sent Done false
14816 4657 Acked Flush 0 Done Sent Done false
14817 4658 Acked Read 8 Done Sent Done false
14818 4659 Acked Read 3 Done Sent Done false
14819 4660 Acked Write 7 Done Sent Done false
14820 4661 Acked Read 10 Done Sent Done false
14821 4662 Acked Read 1 Done Sent Done false
14822 4663 Acked Write 1 Done Sent Done false
14823 4664 Acked Flush 0 Done Sent Done false
14824 4665 Acked Flush 0 Done Sent Done false
14825 4666 Acked Read 1 Done Sent Done false
14826 4667 Acked Read 4 Done Sent Done false
14827 4668 Acked Write 8 Done Sent Done false
14828 4669 Acked Write 1 Done Sent Done false
14829 4670 Acked Write 9 Done Sent Done false
14830 4671 Acked Write 4 Done Sent Done false
14831 4672 Acked Write 9 Done Sent Done false
14832 4673 Acked Read 4 Done Sent Done false
14833 4674 Acked Write 7 Done Sent Done false
14834 4675 Acked Read 1 Done Sent Done false
14835 4676 Acked Read 8 Done Sent Done false
14836 4677 Acked Write 7 Done Sent Done false
14837 4678 Acked Write 7 Done Sent Done false
14838 4679 Acked Write 3 Done Sent Done false
14839 4680 Acked Write 3 Done Sent Done false
14840 4681 Acked Read 3 Done Sent Done false
14841 4682 Acked Read 5 Done Sent Done false
14842 4683 Acked Read 5 Done Sent Done false
14843 4684 Acked Flush 0 Done Sent Done false
14844 4685 Acked Flush 0 Done Sent Done false
14845 4686 Acked Read 3 Done Sent Done false
14846 4687 Acked Write 8 Done Sent Done false
14847 4688 Acked Write 9 Done Sent Done false
14848 4689 Acked Write 8 Done Sent Done false
14849 4690 Acked Read 1 Done Sent Done false
14850 4691 Acked Read 6 Done Sent Done false
14851 4692 Acked Write 6 Done Sent Done false
14852 4693 Acked Read 2 Done Sent Done false
14853 4694 Acked Flush 0 Done Sent Done false
14854 4695 Acked Read 3 Done Sent Done false
14855 4696 Acked Read 8 Done Sent Done false
14856 4697 Acked Write 3 Done Sent Done false
14857 4698 Acked Read 5 Done Sent Done false
14858 4699 Acked Write 4 Done Sent Done false
14859 4700 Acked Read 9 Done Sent Done false
14860 4701 Acked Read 8 Done Sent Done false
14861 4702 Acked Flush 0 Done Sent Done false
14862 4703 Acked Read 9 Done Sent Done false
14863 4704 Acked Flush 0 Done Sent Done false
14864 4705 Acked Read 6 Done Sent Done false
14865 4706 Acked Read 7 Done Sent Done false
14866 4707 Acked Read 6 Done Sent Done false
14867 4708 Acked Write 5 Done Sent Done false
14868 4709 Acked Write 4 Done Sent Done false
14869 4710 Acked Read 2 Done Sent Done false
14870 4711 Acked Read 6 Done Sent Done false
14871 4712 Acked Read 9 Done Sent Done false
14872 4713 Acked Read 1 Done Sent Done false
14873 4714 Acked Write 10 Done Sent Done false
14874 4715 Acked Read 1 Done Sent Done false
14875 4716 Acked Write 1 Done Sent Done false
14876 4717 Acked Read 8 Done Sent Done false
14877 4718 Acked Write 2 Done Sent Done false
14878 4719 Acked Flush 0 Done Sent Done false
14879 4720 Acked Read 4 Done Sent Done false
14880 4721 Acked Read 4 Done Sent Done false
14881 4722 Acked Read 1 Done Sent Done false
14882 4723 Acked Write 9 Done Sent Done false
14883 4724 Acked Write 8 Done Sent Done false
14884 4725 Acked Write 6 Done Sent Done false
14885 4726 Acked Write 6 Done Sent Done false
14886 4727 Acked Write 3 Done Sent Done false
14887 4728 Acked Read 2 Done Sent Done false
14888 4729 Acked Write 2 Done Sent Done false
14889 4730 Acked Read 7 Done Sent Done false
14890 4731 Acked Write 9 Done Sent Done false
14891 4732 Acked Write 9 Done Sent Done false
14892 4733 Acked Write 8 Done Sent Done false
14893 4734 Acked Read 5 Done Sent Done false
14894 4735 Acked Read 7 Done Sent Done false
14895 4736 Acked Read 3 Done Sent Done false
14896 4737 Acked Write 10 Done Sent Done false
14897 4738 Acked Write 1 Done Sent Done false
14898 4739 Acked Read 2 Done Sent Done false
14899 4740 Acked Read 1 Done Sent Done false
14900 4741 Acked Read 5 Done Sent Done false
14901 4742 Acked Read 10 Done Sent Done false
14902 4743 Acked Write 4 Done Sent Done false
14903 4744 Acked Read 9 Done Sent Done false
14904 4745 Acked Flush 0 Done Sent Done false
14905 4746 Acked Read 6 Done Sent Done false
14906 4747 Acked Read 3 Done Sent Done false
14907 4748 Acked Read 4 Done Sent Done false
14908 4749 Acked Read 5 Done Sent Done false
14909 4750 Acked Read 6 Done Sent Done false
14910 4751 Acked Write 7 Done Sent Done false
14911 4752 Acked Read 1 Done Sent Done false
14912 4753 Acked Read 4 Done Sent Done false
14913 4754 Acked Flush 0 Done Sent Done false
14914 4755 Acked Write 7 Done Sent Done false
14915 4756 Acked Write 4 Done Sent Done false
14916 4757 Acked Write 7 Done Sent Done false
14917 4758 Acked Write 4 Done Sent Done false
14918 4759 Acked Write 9 Done Sent Done false
14919 4760 Acked Write 8 Done Sent Done false
14920 4761 Acked Read 2 Done Sent Done false
14921 4762 Acked Read 6 Done Sent Done false
14922 4763 Acked Read 8 Done Sent Done false
14923 4764 Acked Write 4 Done Sent Done false
14924 4765 Acked Write 1 Done Sent Done false
14925 4766 Acked Write 8 Done Sent Done false
14926 4767 Acked Read 5 Done Sent Done false
14927 4768 Acked Read 2 Done Sent Done false
14928 4769 Acked Read 4 Done Sent Done false
14929 4770 Acked Write 1 Done Sent Done false
14930 4771 Acked Write 6 Done Sent Done false
14931 4772 Acked Write 4 Done Sent Done false
14932 4773 Acked Write 6 Done Sent Done false
14933 4774 Acked Read 8 Done Sent Done false
14934 4775 Acked Read 3 Done Sent Done false
14935 4776 Acked Write 3 Done Sent Done false
14936 4777 Acked Flush 0 Done Sent Done false
14937 4778 Acked Read 8 Done Sent Done false
14938 4779 Acked Read 10 Done Sent Done false
14939 4780 Acked Flush 0 Done Sent Done false
14940 4781 Acked Write 5 Done Sent Done false
14941 4782 Acked Write 2 Done Sent Done false
14942 4783 Acked Write 9 Done Sent Done false
14943 4784 Acked Read 8 Done Sent Done false
14944 4785 Acked Read 6 Done Sent Done false
14945 4786 Acked Read 5 Done Sent Done false
14946 4787 Acked Flush 0 Done Sent Done false
14947 4788 Acked Read 4 Done Sent Done false
14948 4789 Acked Read 1 Done Sent Done false
14949 4790 Acked Read 10 Done Sent Done false
14950 4791 Acked Write 8 Done Sent Done false
14951 4792 Acked Read 6 Done Sent Done false
14952 4793 Acked Read 10 Done Sent Done false
14953 4794 Acked Write 1 Done Sent Done false
14954 4795 Acked Read 3 Done Sent Done false
14955 4796 Acked Flush 0 Done Sent Done false
14956 4797 Acked Read 9 Done Sent Done false
14957 4798 Acked Flush 0 Done Sent Done false
14958 4799 Acked Write 2 Done Sent Done false
14959 4800 Acked Read 10 Done Sent Done false
14960 4801 Acked Read 4 Done Sent Done false
14961 4802 Acked Write 3 Done Sent Done false
14962 4803 Acked Write 1 Done Sent Done false
14963 4804 Acked Read 2 Done Sent Done false
14964 4805 Acked Read 5 Done Sent Done false
14965 4806 Acked Write 8 Done Sent Done false
14966 4807 Acked Write 2 Done Sent Done false
14967 4808 Acked Write 8 Done Sent Done false
14968 4809 Acked Write 4 Done Sent Done false
14969 4810 Acked Write 2 Done Sent Done false
14970 4811 Acked Read 2 Done Sent Done false
14971 4812 Acked Read 1 Done Sent Done false
14972 4813 Acked Write 1 Done Sent Done false
14973 4814 Acked Read 8 Done Sent Done false
14974 4815 Acked Read 8 Done Sent Done false
14975 4816 Acked Write 4 Done Sent Done false
14976 4817 Acked Read 6 Done Sent Done false
14977 4818 Acked Flush 0 Done Sent Done false
14978 4819 Acked Write 8 Done Sent Done false
14979 4820 Acked Write 9 Done Sent Done false
14980 4821 Acked Read 6 Done Sent Done false
14981 4822 Acked Read 6 Done Sent Done false
14982 4823 Acked Read 3 Done Sent Done false
14983 4824 Acked Read 10 Done Sent Done false
14984 4825 Acked Read 2 Done Sent Done false
14985 4826 Acked Write 8 Done Sent Done false
14986 4827 Acked Read 2 Done Sent Done false
14987 4828 Acked Write 5 Done Sent Done false
14988 4829 Acked Flush 0 Done Sent Done false
14989 4830 Acked Write 5 Done Sent Done false
14990 4831 Acked Write 3 Done Sent Done false
14991 4832 Acked Write 1 Done Sent Done false
14992 4833 Acked Write 3 Done Sent Done false
14993 4834 Acked Read 3 Done Sent Done false
14994 4835 Acked Write 6 Done Sent Done false
14995 4836 Acked Read 9 Done Sent Done false
14996 4837 Acked Read 9 Done Sent Done false
14997 4838 Acked Write 3 Done Sent Done false
14998 4839 Acked Flush 0 Done Sent Done false
14999 4840 Acked Read 9 Done Sent Done false
15000 4841 Acked Write 2 Done Sent Done false
15001 4842 Acked Flush 0 Done Sent Done false
15002 4843 Acked Read 2 Done Sent Done false
15003 4844 Acked Flush 0 Done Sent Done false
15004 4845 Acked Write 1 Done Sent Done false
15005 4846 Acked Write 3 Done Sent Done false
15006 4847 Acked Write 9 Done Sent Done false
15007 4848 Acked Read 10 Done Sent Done false
15008 4849 Acked Flush 0 Done Sent Done false
15009 4850 Acked Read 3 Done Sent Done false
15010 4851 Acked Read 1 Done Sent Done false
15011 4852 Acked Write 1 Done Sent Done false
15012 4853 Acked Write 3 Done Sent Done false
15013 4854 Acked Write 6 Done Sent Done false
15014 4855 Acked Read 10 Done Sent Done false
15015 4856 Acked Write 8 Done Sent Done false
15016 4857 Acked Read 8 Done Sent Done false
15017 4858 Acked Read 2 Done Sent Done false
15018 4859 Acked Write 2 Done Sent Done false
15019 4860 Acked Read 9 Done Sent Done false
15020 4861 Acked Write 4 Done Sent Done false
15021 4862 Acked Read 4 Done Sent Done false
15022 4863 Acked Read 2 Done Sent Done false
15023 4864 Acked Read 6 Done Sent Done false
15024 4865 Acked Flush 0 Done Sent Done false
15025 4866 Acked Write 5 Done Sent Done false
15026 4867 Acked Flush 0 Done Sent Done false
15027 4868 Acked Flush 0 Done Sent Done false
15028 4869 Acked Write 4 Done Sent Done false
15029 4870 Acked Write 10 Done Sent Done false
15030 4871 Acked Flush 0 Done Sent Done false
15031 4872 Acked Read 2 Done Sent Done false
15032 4873 Acked Write 2 Done Sent Done false
15033 4874 Acked Write 8 Done Sent Done false
15034 4875 Acked Write 1 Done Sent Done false
15035 4876 Acked Read 8 Done Sent Done false
15036 4877 Acked Flush 0 Done Sent Done false
15037 4878 Acked Flush 0 Done Sent Done false
15038 4879 Acked Write 10 Done Sent Done false
15039 4880 Acked Read 8 Done Sent Done false
15040 4881 Acked Flush 0 Done Sent Done false
15041 4882 Acked Write 6 Done Sent Done false
15042 4883 Acked Write 2 Done Sent Done false
15043 4884 Acked Flush 0 Done Sent Done false
15044 4885 Acked Read 2 Done Sent Done false
15045 4886 Acked Read 2 Done Sent Done false
15046 4887 Acked Read 9 Done Sent Done false
15047 4888 Acked Write 9 Done Sent Done false
15048 4889 Acked Read 2 Done Sent Done false
15049 4890 Acked Read 7 Done Sent Done false
15050 4891 Acked Read 9 Done Sent Done false
15051 4892 Acked Read 8 Done Sent Done false
15052 4893 Acked Read 5 Done Sent Done false
15053 4894 Acked Read 3 Done Sent Done false
15054 4895 Acked Read 10 Done Sent Done false
15055 4896 Acked Write 7 Done Sent Done false
15056 4897 Acked Flush 0 Done Sent Done false
15057 4898 Acked Write 9 Done Sent Done false
15058 4899 Acked Flush 0 Done Sent Done false
15059 4900 Acked Write 9 Done Sent Done false
15060 4901 Acked Read 5 Done Sent Done false
15061 4902 Acked Write 3 Done Sent Done false
15062 4903 Acked Flush 0 Done Sent Done false
15063 4904 Acked Read 7 Done Sent Done false
15064 4905 Acked Write 9 Done Sent Done false
15065 4906 Acked Write 5 Done Sent Done false
15066 4907 Acked Flush 0 Done Sent Done false
15067 4908 Acked Flush 0 Done Sent Done false
15068 4909 Acked Read 3 Done Sent Done false
15069 4910 Acked Read 7 Done Sent Done false
15070 4911 Acked Write 7 Done Sent Done false
15071 4912 Acked Write 2 Done Sent Done false
15072 4913 Acked Read 3 Done Sent Done false
15073 4914 Acked Read 7 Done Sent Done false
15074 4915 Acked Read 1 Done Sent Done false
15075 4916 Acked Flush 0 Done Sent Done false
15076 4917 Acked Flush 0 Done Sent Done false
15077 4918 Acked Flush 0 Done Sent Done false
15078 4919 Acked Read 10 Done Sent Done false
15079 4920 Acked Flush 0 Done Sent Done false
15080 4921 Acked Flush 0 Done Sent Done false
15081 4922 Acked Read 10 Done Sent Done false
15082 4923 Acked Read 1 Done Sent Done false
15083 4924 Acked Write 7 Done Sent Done false
15084 STATES DS:0 DS:1 DS:2 TOTAL
15085 Sent 0 301 0 301
15086 Done 301 0 301 602
15087 Skipped 0 0 0 0
15088 Error 0 0 0 0
15089 Last Flush: 4921 4623 4921
15090 Downstairs last five completed: 4623 4622 4621 4620 4619
15091 Upstairs last five completed: 4924 4923 4922 4921 4920
15092 CLIENT: Up:0 ds:301 act:2
15093 [8820] got start action
15094 [8820] I am going to restart
15095 Starting downstairs at port 8820
15096 Make output file at "/tmp/dsc/downstairs-8820.txt"
15097 [8820][1] reports Starting
15098 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
15099 [8820][1] reports Running
15100 ----------------------------------------------------------------
15101 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15102 States: Active Offline Active
15103 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15104 4624 Acked Write 8 Done Sent Done false
15105 4625 Acked Read 7 Done Sent Done false
15106 4626 Acked Write 7 Done Sent Done false
15107 4627 Acked Read 2 Done Sent Done false
15108 4628 Acked Read 8 Done Sent Done false
15109 4629 Acked Read 4 Done Sent Done false
15110 4630 Acked Flush 0 Done Sent Done false
15111 4631 Acked Read 1 Done Sent Done false
15112 4632 Acked Read 2 Done Sent Done false
15113 4633 Acked Write 9 Done Sent Done false
15114 4634 Acked Write 4 Done Sent Done false
15115 4635 Acked Read 3 Done Sent Done false
15116 4636 Acked Write 6 Done Sent Done false
15117 4637 Acked Read 4 Done Sent Done false
15118 4638 Acked Read 2 Done Sent Done false
15119 4639 Acked Read 4 Done Sent Done false
15120 4640 Acked Read 10 Done Sent Done false
15121 4641 Acked Read 2 Done Sent Done false
15122 4642 Acked Read 7 Done Sent Done false
15123 4643 Acked Write 4 Done Sent Done false
15124 4644 Acked Read 9 Done Sent Done false
15125 4645 Acked Read 4 Done Sent Done false
15126 4646 Acked Read 10 Done Sent Done false
15127 4647 Acked Read 4 Done Sent Done false
15128 4648 Acked Write 9 Done Sent Done false
15129 4649 Acked Write 8 Done Sent Done false
15130 4650 Acked Flush 0 Done Sent Done false
15131 4651 Acked Write 3 Done Sent Done false
15132 4652 Acked Read 8 Done Sent Done false
15133 4653 Acked Read 6 Done Sent Done false
15134 4654 Acked Read 6 Done Sent Done false
15135 4655 Acked Read 7 Done Sent Done false
15136 4656 Acked Read 3 Done Sent Done false
15137 4657 Acked Flush 0 Done Sent Done false
15138 4658 Acked Read 8 Done Sent Done false
15139 4659 Acked Read 3 Done Sent Done false
15140 4660 Acked Write 7 Done Sent Done false
15141 4661 Acked Read 10 Done Sent Done false
15142 4662 Acked Read 1 Done Sent Done false
15143 4663 Acked Write 1 Done Sent Done false
15144 4664 Acked Flush 0 Done Sent Done false
15145 4665 Acked Flush 0 Done Sent Done false
15146 4666 Acked Read 1 Done Sent Done false
15147 4667 Acked Read 4 Done Sent Done false
15148 4668 Acked Write 8 Done Sent Done false
15149 4669 Acked Write 1 Done Sent Done false
15150 4670 Acked Write 9 Done Sent Done false
15151 4671 Acked Write 4 Done Sent Done false
15152 4672 Acked Write 9 Done Sent Done false
15153 4673 Acked Read 4 Done Sent Done false
15154 4674 Acked Write 7 Done Sent Done false
15155 4675 Acked Read 1 Done Sent Done false
15156 4676 Acked Read 8 Done Sent Done false
15157 4677 Acked Write 7 Done Sent Done false
15158 4678 Acked Write 7 Done Sent Done false
15159 4679 Acked Write 3 Done Sent Done false
15160 4680 Acked Write 3 Done Sent Done false
15161 4681 Acked Read 3 Done Sent Done false
15162 4682 Acked Read 5 Done Sent Done false
15163 4683 Acked Read 5 Done Sent Done false
15164 4684 Acked Flush 0 Done Sent Done false
15165 4685 Acked Flush 0 Done Sent Done false
15166 4686 Acked Read 3 Done Sent Done false
15167 4687 Acked Write 8 Done Sent Done false
15168 4688 Acked Write 9 Done Sent Done false
15169 4689 Acked Write 8 Done Sent Done false
15170 4690 Acked Read 1 Done Sent Done false
15171 4691 Acked Read 6 Done Sent Done false
15172 4692 Acked Write 6 Done Sent Done false
15173 4693 Acked Read 2 Done Sent Done false
15174 4694 Acked Flush 0 Done Sent Done false
15175 4695 Acked Read 3 Done Sent Done false
15176 4696 Acked Read 8 Done Sent Done false
15177 4697 Acked Write 3 Done Sent Done false
15178 4698 Acked Read 5 Done Sent Done false
15179 4699 Acked Write 4 Done Sent Done false
15180 4700 Acked Read 9 Done Sent Done false
15181 4701 Acked Read 8 Done Sent Done false
15182 4702 Acked Flush 0 Done Sent Done false
15183 4703 Acked Read 9 Done Sent Done false
15184 4704 Acked Flush 0 Done Sent Done false
15185 4705 Acked Read 6 Done Sent Done false
15186 4706 Acked Read 7 Done Sent Done false
15187 4707 Acked Read 6 Done Sent Done false
15188 4708 Acked Write 5 Done Sent Done false
15189 4709 Acked Write 4 Done Sent Done false
15190 4710 Acked Read 2 Done Sent Done false
15191 4711 Acked Read 6 Done Sent Done false
15192 4712 Acked Read 9 Done Sent Done false
15193 4713 Acked Read 1 Done Sent Done false
15194 4714 Acked Write 10 Done Sent Done false
15195 4715 Acked Read 1 Done Sent Done false
15196 4716 Acked Write 1 Done Sent Done false
15197 4717 Acked Read 8 Done Sent Done false
15198 4718 Acked Write 2 Done Sent Done false
15199 4719 Acked Flush 0 Done Sent Done false
15200 4720 Acked Read 4 Done Sent Done false
15201 4721 Acked Read 4 Done Sent Done false
15202 4722 Acked Read 1 Done Sent Done false
15203 4723 Acked Write 9 Done Sent Done false
15204 4724 Acked Write 8 Done Sent Done false
15205 4725 Acked Write 6 Done Sent Done false
15206 4726 Acked Write 6 Done Sent Done false
15207 4727 Acked Write 3 Done Sent Done false
15208 4728 Acked Read 2 Done Sent Done false
15209 4729 Acked Write 2 Done Sent Done false
15210 4730 Acked Read 7 Done Sent Done false
15211 4731 Acked Write 9 Done Sent Done false
15212 4732 Acked Write 9 Done Sent Done false
15213 4733 Acked Write 8 Done Sent Done false
15214 4734 Acked Read 5 Done Sent Done false
15215 4735 Acked Read 7 Done Sent Done false
15216 4736 Acked Read 3 Done Sent Done false
15217 4737 Acked Write 10 Done Sent Done false
15218 4738 Acked Write 1 Done Sent Done false
15219 4739 Acked Read 2 Done Sent Done false
15220 4740 Acked Read 1 Done Sent Done false
15221 4741 Acked Read 5 Done Sent Done false
15222 4742 Acked Read 10 Done Sent Done false
15223 4743 Acked Write 4 Done Sent Done false
15224 4744 Acked Read 9 Done Sent Done false
15225 4745 Acked Flush 0 Done Sent Done false
15226 4746 Acked Read 6 Done Sent Done false
15227 4747 Acked Read 3 Done Sent Done false
15228 4748 Acked Read 4 Done Sent Done false
15229 4749 Acked Read 5 Done Sent Done false
15230 4750 Acked Read 6 Done Sent Done false
15231 4751 Acked Write 7 Done Sent Done false
15232 4752 Acked Read 1 Done Sent Done false
15233 4753 Acked Read 4 Done Sent Done false
15234 4754 Acked Flush 0 Done Sent Done false
15235 4755 Acked Write 7 Done Sent Done false
15236 4756 Acked Write 4 Done Sent Done false
15237 4757 Acked Write 7 Done Sent Done false
15238 4758 Acked Write 4 Done Sent Done false
15239 4759 Acked Write 9 Done Sent Done false
15240 4760 Acked Write 8 Done Sent Done false
15241 4761 Acked Read 2 Done Sent Done false
15242 4762 Acked Read 6 Done Sent Done false
15243 4763 Acked Read 8 Done Sent Done false
15244 4764 Acked Write 4 Done Sent Done false
15245 4765 Acked Write 1 Done Sent Done false
15246 4766 Acked Write 8 Done Sent Done false
15247 4767 Acked Read 5 Done Sent Done false
15248 4768 Acked Read 2 Done Sent Done false
15249 4769 Acked Read 4 Done Sent Done false
15250 4770 Acked Write 1 Done Sent Done false
15251 4771 Acked Write 6 Done Sent Done false
15252 4772 Acked Write 4 Done Sent Done false
15253 4773 Acked Write 6 Done Sent Done false
15254 4774 Acked Read 8 Done Sent Done false
15255 4775 Acked Read 3 Done Sent Done false
15256 4776 Acked Write 3 Done Sent Done false
15257 4777 Acked Flush 0 Done Sent Done false
15258 4778 Acked Read 8 Done Sent Done false
15259 4779 Acked Read 10 Done Sent Done false
15260 4780 Acked Flush 0 Done Sent Done false
15261 4781 Acked Write 5 Done Sent Done false
15262 4782 Acked Write 2 Done Sent Done false
15263 4783 Acked Write 9 Done Sent Done false
15264 4784 Acked Read 8 Done Sent Done false
15265 4785 Acked Read 6 Done Sent Done false
15266 4786 Acked Read 5 Done Sent Done false
15267 4787 Acked Flush 0 Done Sent Done false
15268 4788 Acked Read 4 Done Sent Done false
15269 4789 Acked Read 1 Done Sent Done false
15270 4790 Acked Read 10 Done Sent Done false
15271 4791 Acked Write 8 Done Sent Done false
15272 4792 Acked Read 6 Done Sent Done false
15273 4793 Acked Read 10 Done Sent Done false
15274 4794 Acked Write 1 Done Sent Done false
15275 4795 Acked Read 3 Done Sent Done false
15276 4796 Acked Flush 0 Done Sent Done false
15277 4797 Acked Read 9 Done Sent Done false
15278 4798 Acked Flush 0 Done Sent Done false
15279 4799 Acked Write 2 Done Sent Done false
15280 4800 Acked Read 10 Done Sent Done false
15281 4801 Acked Read 4 Done Sent Done false
15282 4802 Acked Write 3 Done Sent Done false
15283 4803 Acked Write 1 Done Sent Done false
15284 4804 Acked Read 2 Done Sent Done false
15285 4805 Acked Read 5 Done Sent Done false
15286 4806 Acked Write 8 Done Sent Done false
15287 4807 Acked Write 2 Done Sent Done false
15288 4808 Acked Write 8 Done Sent Done false
15289 4809 Acked Write 4 Done Sent Done false
15290 4810 Acked Write 2 Done Sent Done false
15291 4811 Acked Read 2 Done Sent Done false
15292 4812 Acked Read 1 Done Sent Done false
15293 4813 Acked Write 1 Done Sent Done false
15294 4814 Acked Read 8 Done Sent Done false
15295 4815 Acked Read 8 Done Sent Done false
15296 4816 Acked Write 4 Done Sent Done false
15297 4817 Acked Read 6 Done Sent Done false
15298 4818 Acked Flush 0 Done Sent Done false
15299 4819 Acked Write 8 Done Sent Done false
15300 4820 Acked Write 9 Done Sent Done false
15301 4821 Acked Read 6 Done Sent Done false
15302 4822 Acked Read 6 Done Sent Done false
15303 4823 Acked Read 3 Done Sent Done false
15304 4824 Acked Read 10 Done Sent Done false
15305 4825 Acked Read 2 Done Sent Done false
15306 4826 Acked Write 8 Done Sent Done false
15307 4827 Acked Read 2 Done Sent Done false
15308 4828 Acked Write 5 Done Sent Done false
15309 4829 Acked Flush 0 Done Sent Done false
15310 4830 Acked Write 5 Done Sent Done false
15311 4831 Acked Write 3 Done Sent Done false
15312 4832 Acked Write 1 Done Sent Done false
15313 4833 Acked Write 3 Done Sent Done false
15314 4834 Acked Read 3 Done Sent Done false
15315 4835 Acked Write 6 Done Sent Done false
15316 4836 Acked Read 9 Done Sent Done false
15317 4837 Acked Read 9 Done Sent Done false
15318 4838 Acked Write 3 Done Sent Done false
15319 4839 Acked Flush 0 Done Sent Done false
15320 4840 Acked Read 9 Done Sent Done false
15321 4841 Acked Write 2 Done Sent Done false
15322 4842 Acked Flush 0 Done Sent Done false
15323 4843 Acked Read 2 Done Sent Done false
15324 4844 Acked Flush 0 Done Sent Done false
15325 4845 Acked Write 1 Done Sent Done false
15326 4846 Acked Write 3 Done Sent Done false
15327 4847 Acked Write 9 Done Sent Done false
15328 4848 Acked Read 10 Done Sent Done false
15329 4849 Acked Flush 0 Done Sent Done false
15330 4850 Acked Read 3 Done Sent Done false
15331 4851 Acked Read 1 Done Sent Done false
15332 4852 Acked Write 1 Done Sent Done false
15333 4853 Acked Write 3 Done Sent Done false
15334 4854 Acked Write 6 Done Sent Done false
15335 4855 Acked Read 10 Done Sent Done false
15336 4856 Acked Write 8 Done Sent Done false
15337 4857 Acked Read 8 Done Sent Done false
15338 4858 Acked Read 2 Done Sent Done false
15339 4859 Acked Write 2 Done Sent Done false
15340 4860 Acked Read 9 Done Sent Done false
15341 4861 Acked Write 4 Done Sent Done false
15342 4862 Acked Read 4 Done Sent Done false
15343 4863 Acked Read 2 Done Sent Done false
15344 4864 Acked Read 6 Done Sent Done false
15345 4865 Acked Flush 0 Done Sent Done false
15346 4866 Acked Write 5 Done Sent Done false
15347 4867 Acked Flush 0 Done Sent Done false
15348 4868 Acked Flush 0 Done Sent Done false
15349 4869 Acked Write 4 Done Sent Done false
15350 4870 Acked Write 10 Done Sent Done false
15351 4871 Acked Flush 0 Done Sent Done false
15352 4872 Acked Read 2 Done Sent Done false
15353 4873 Acked Write 2 Done Sent Done false
15354 4874 Acked Write 8 Done Sent Done false
15355 4875 Acked Write 1 Done Sent Done false
15356 4876 Acked Read 8 Done Sent Done false
15357 4877 Acked Flush 0 Done Sent Done false
15358 4878 Acked Flush 0 Done Sent Done false
15359 4879 Acked Write 10 Done Sent Done false
15360 4880 Acked Read 8 Done Sent Done false
15361 4881 Acked Flush 0 Done Sent Done false
15362 4882 Acked Write 6 Done Sent Done false
15363 4883 Acked Write 2 Done Sent Done false
15364 4884 Acked Flush 0 Done Sent Done false
15365 4885 Acked Read 2 Done Sent Done false
15366 4886 Acked Read 2 Done Sent Done false
15367 4887 Acked Read 9 Done Sent Done false
15368 4888 Acked Write 9 Done Sent Done false
15369 4889 Acked Read 2 Done Sent Done false
15370 4890 Acked Read 7 Done Sent Done false
15371 4891 Acked Read 9 Done Sent Done false
15372 4892 Acked Read 8 Done Sent Done false
15373 4893 Acked Read 5 Done Sent Done false
15374 4894 Acked Read 3 Done Sent Done false
15375 4895 Acked Read 10 Done Sent Done false
15376 4896 Acked Write 7 Done Sent Done false
15377 4897 Acked Flush 0 Done Sent Done false
15378 4898 Acked Write 9 Done Sent Done false
15379 4899 Acked Flush 0 Done Sent Done false
15380 4900 Acked Write 9 Done Sent Done false
15381 4901 Acked Read 5 Done Sent Done false
15382 4902 Acked Write 3 Done Sent Done false
15383 4903 Acked Flush 0 Done Sent Done false
15384 4904 Acked Read 7 Done Sent Done false
15385 4905 Acked Write 9 Done Sent Done false
15386 4906 Acked Write 5 Done Sent Done false
15387 4907 Acked Flush 0 Done Sent Done false
15388 4908 Acked Flush 0 Done Sent Done false
15389 4909 Acked Read 3 Done Sent Done false
15390 4910 Acked Read 7 Done Sent Done false
15391 4911 Acked Write 7 Done Sent Done false
15392 4912 Acked Write 2 Done Sent Done false
15393 4913 Acked Read 3 Done Sent Done false
15394 4914 Acked Read 7 Done Sent Done false
15395 4915 Acked Read 1 Done Sent Done false
15396 4916 Acked Flush 0 Done Sent Done false
15397 4917 Acked Flush 0 Done Sent Done false
15398 4918 Acked Flush 0 Done Sent Done false
15399 4919 Acked Read 10 Done Sent Done false
15400 4920 Acked Flush 0 Done Sent Done false
15401 4921 Acked Flush 0 Done Sent Done false
15402 4922 Acked Read 10 Done Sent Done false
15403 4923 Acked Read 1 Done Sent Done false
15404 4924 Acked Write 7 Done Sent Done false
15405 4925 Acked Flush 0 Done Sent Done false
15406 STATES DS:0 DS:1 DS:2 TOTAL
15407 Sent 0 302 0 302
15408 Done 302 0 302 604
15409 Skipped 0 0 0 0
15410 Error 0 0 0 0
15411 Last Flush: 4925 4623 4925
15412 Downstairs last five completed: 4623 4622 4621 4620 4619
15413 Upstairs last five completed: 4925 4924 4923 4922 4921
15414 CLIENT: Up:0 ds:302 act:2
154152024-12-11T23:43:43.474ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154162024-12-11T23:43:43.474ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53637) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154172024-12-11T23:43:43.475ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154182024-12-11T23:43:43.475ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154192024-12-11T23:43:43.475ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154202024-12-11T23:43:43.475ZINFOcrucible: send last flush ID to this DS: 4623 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154212024-12-11T23:43:43.475ZINFOcrucible: Replied this last flush ID: 4623 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154222024-12-11T23:43:43.475ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154232024-12-11T23:43:43.475ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154242024-12-11T23:43:43.475ZINFOcrucible: [1] client re-new 302 jobs since flush 4623 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154252024-12-11T23:43:43.476ZINFOcrucible: [1] Replayed 302 jobs since flush: 4623 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
15426 ----------------------------------------------------------------
15427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15428 Downstairs last five completed: 4925 4924 4923 4922 4921
15429 Upstairs last five completed: 4925 4924 4923 4922 4921
15430 CLIENT: Up:0 ds:0 act:3
15431 Replay: All jobs finished
15432 Main task has work to do, go find it
15433 got dsc Stop(0)
15434 stop 0
15435 Dec 11 23:43:45.706 INFO request completed, [8810] Got stop action so:false kr:false
15436 latency_us: 326, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 24dc8b09-fb00-4692-901b-ed8716bc919c, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
15437 Dec 11 23:43:45.707 INFO request completed, latency_us: 117, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6addcbbb-3324-4b7f-8cd3-e9551795169c, remote_addr{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:43:45.707267373Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
15438 : 127.0.0.1:62969{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:43:45.707378907Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
15439 , local_addr: 127.0.0.1{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:43:45.707425378Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
15440 :9998
154412024-12-11T23:43:45.707ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154422024-12-11T23:43:45.707ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
154432024-12-11T23:43:45.707ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
15444 [8810] Exited with: ExitStatus(unix_wait_status(9))
15445 [8810][0] reports Exit
15446 Dec 11 23:43:49.709 INFO request completed, latency_us: 219, response_code: 200, uri: /state/cid/0, method: GET, req_id: a56c6a7d-34c1-49ee-a8fb-0ea2a50ab69c, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
15447 001/300 Write block 1492 len 28672 data: 5 6 4 4 4 4 4
15448 002/300 Write block 4453 len 24576 data: 3 2 2 3 3 3
15449 003/300 Flush
15450 004/300 Write block 4674 len 4096 data: 2
15451 005/300 Write block 723 len 24576 data: 3 3 3 3 2 2
15452 006/300 Read block 1922 len 20480
15453 007/300 Read block 3442 len 24576
15454 008/300 Write block 665 len 20480 data: 4 4 4 4 3
15455 009/300 Read block 895 len 4096
15456 010/300 Read block 4064 len 12288
15457 011/300 Write block 4532 len 20480 data: 9 6 6 4 5
15458 012/300 Read block 2949 len 32768
15459 013/300 Read block 1907 len 16384
15460 014/300 Read block 4439 len 16384
15461 015/300 Read block 4980 len 20480
15462 016/300 Read block 3690 len 40960
15463 017/300 Read block 1138 len 12288
15464 018/300 Read block 2198 len 8192
15465 019/300 Write block 147 len 20480 data: 4 5 5 4 4
15466 020/300 Write block 3917 len 12288 data: 2 3 4
15467 021/300 Read block 3476 len 8192
15468 022/300 Read block 1820 len 40960
15469 023/300 Flush
15470 024/300 Write block 4699 len 32768 data: 2 2 3 3 3 3 3 3
15471 025/300 Write block 3878 len 20480 data: 6 5 4 4 4
15472 026/300 Read block 4096 len 40960
15473 027/300 Read block 770 len 16384
15474 028/300 Write block 3055 len 12288 data: 5 5 5
15475 029/300 Read block 841 len 40960
15476 030/300 Write block 688 len 8192 data: 4 4
15477 031/300 Read block 4295 len 28672
15478 032/300 Flush
15479 033/300 Write block 1642 len 4096 data: 5
15480 034/300 Read block 1617 len 40960
15481 035/300 Write block 8 len 16384 data: 3 4 4 3
15482 036/300 Read block 3573 len 40960
15483 037/300 Read block 2670 len 12288
15484 038/300 Write block 2684 len 36864 data: 4 5 5 6 5 4 4 4 5
15485 039/300 Read block 4174 len 4096
15486 040/300 Write block 1334 len 8192 data: 2 2
15487 041/300 Write block 460 len 24576 data: 3 3 3 3 3 2
15488 042/300 Write block 3475 len 4096 data: 3
15489 043/300 Read block 4196 len 16384
15490 044/300 Read block 636 len 24576
15491 045/300 Read block 3485 len 36864
15492 046/300 Read block 4035 len 40960
15493 047/300 Read block 2432 len 32768
15494 048/300 Read block 3563 len 16384
15495 049/300 Write block 2574 len 32768 data: 6 7 7 8 9 8 6 6
15496 050/300 Write block 2102 len 12288 data: 3 4 4
15497 051/300 Write block 4723 len 20480 data: 4 3 2 2 2
15498 052/300 Read block 2701 len 8192
15499 053/300 Read block 643 len 8192
15500 054/300 Write block 2065 len 28672 data: 4 5 5 5 5 4 4
15501 055/300 Write block 3487 len 8192 data: 3 3
15502 056/300 Flush
15503 057/300 Write block 4293 len 24576 data: 2 2 2 4 4 4
15504 058/300 Write block 722 len 20480 data: 3 4 4 4 4
15505 059/300 Write block 882 len 16384 data: 6 5 5 5
15506 060/300 Write block 4870 len 24576 data: 3 4 3 3 3 4
15507 061/300 Read block 1528 len 32768
15508 062/300 Write block 1292 len 24576 data: 5 5 6 5 5 5
15509 063/300 Write block 584 len 16384 data: 6 4 4 4
15510 064/300 Write block 381 len 16384 data: 3 3 4 4
15511 065/300 Read block 4845 len 36864
15512 066/300 Read block 2263 len 40960
15513 067/300 Read block 3819 len 4096
15514 068/300 Read block 2801 len 4096
15515 069/300 Write block 37 len 4096 data: 2
15516 070/300 Read block 1191 len 12288
15517 071/300 Write block 4424 len 40960 data: 4 4 5 5 6 6 3 3 3 5
15518 072/300 Flush
15519 073/300 Read block 1098 len 36864
15520 074/300 Write block 4982 len 40960 data: 5 5 3 3 3 4 3 3 4 4
15521 075/300 Write block 3023 len 36864 data: 4 3 3 3 3 4 4 3 3
15522 076/300 Read block 2326 len 12288
15523 077/300 Write block 1259 len 12288 data: 4 4 3
15524 078/300 Write block 3121 len 8192 data: 4 4
15525 079/300 Read block 475 len 12288
15526 080/300 Read block 1118 len 4096
15527 081/300 Read block 3578 len 4096
15528 082/300 Write block 3940 len 40960 data: 3 4 4 5 5 5 5 6 6 6
15529 083/300 Write block 2773 len 28672 data: 2 2 3 3 3 3 3
15530 084/300 Read block 3121 len 40960
15531 085/300 Write block 704 len 24576 data: 3 3 3 3 2 4
15532 086/300 Read block 4532 len 16384
15533 087/300 Write block 1590 len 32768 data: 2 2 2 2 2 2 2 2
15534 088/300 Write block 3975 len 8192 data: 5 5
15535 089/300 Read block 1292 len 16384
15536 090/300 Read block 4881 len 12288
15537 091/300 Read block 1664 len 8192
15538 092/300 Read block 469 len 12288
15539 093/300 Read block 2680 len 8192
15540 094/300 Write block 4676 len 24576 data: 2 4 5 5 6 5
15541 095/300 Write block 3307 len 8192 data: 2 2
15542 096/300 Read block 4674 len 24576
15543 097/300 Read block 706 len 24576
15544 098/300 Read block 2121 len 12288
15545 099/300 Write block 1195 len 24576 data: 5 5 5 4 3 3
15546 100/300 Read block 3615 len 8192
15547 101/300 Write block 1168 len 16384 data: 4 4 3 3
15548 102/300 Read block 3252 len 28672
15549 103/300 Read block 4328 len 20480
15550 104/300 Read block 4563 len 20480
15551 105/300 Read block 3153 len 12288
15552 106/300 Flush
15553 107/300 Write block 4918 len 4096 data: 4
15554 108/300 Read block 4292 len 24576
15555 109/300 Write block 1242 len 16384 data: 2 2 2 3
15556 110/300 Read block 730 len 8192
15557 111/300 Write block 776 len 16384 data: 3 3 3 3
15558 112/300 Write block 1265 len 28672 data: 5 4 3 2 3 5 5
15559 113/300 Flush
15560 114/300 Write block 3177 len 36864 data: 4 4 4 4 4 3 3 2 2
15561 115/300 Write block 1751 len 16384 data: 3 3 3 3
15562 116/300 Read block 1498 len 12288
15563 117/300 Read block 4461 len 36864
15564 118/300 Read block 1070 len 32768
15565 119/300 Write block 984 len 32768 data: 5 5 6 5 4 3 3 3
15566 120/300 Read block 280 len 8192
15567 121/300 Write block 3742 len 8192 data: 3 3
15568 122/300 Write block 4072 len 4096 data: 4
15569 123/300 Write block 1054 len 16384 data: 3 5 5 4
15570 124/300 Read block 1853 len 20480
15571 125/300 Read block 2161 len 8192
15572 126/300 Read block 1024 len 12288
15573 127/300 Write block 1223 len 36864 data: 2 2 2 3 3 4 4 3 3
15574 128/300 Write block 67 len 28672 data: 4 5 6 5 5 5 5
15575 129/300 Write block 4128 len 40960 data: 3 2 2 3 3 2 3 3 3 3
15576 130/300 Read block 3477 len 12288
15577 131/300 Read block 828 len 40960
15578 132/300 Read block 1638 len 8192
15579 133/300 Read block 3020 len 24576
15580 134/300 Write block 813 len 32768 data: 4 4 3 3 3 4 3 3
15581 135/300 Write block 3726 len 28672 data: 4 5 3 3 3 3 3
15582 136/300 Write block 1805 len 28672 data: 3 3 4 3 3 3 3
15583 137/300 Write block 3216 len 4096 data: 5
15584 138/300 Write block 281 len 40960 data: 2 2 2 2 3 5 5 5 4 5
15585 139/300 Read block 4780 len 12288
15586 140/300 Read block 1492 len 8192
15587 141/300 Read block 457 len 4096
15588 142/300 Write block 1428 len 16384 data: 4 6 5 5
15589 143/300 Write block 4039 len 36864 data: 5 6 6 6 6 6 5 5 6
15590 144/300 Read block 3424 len 24576
15591 145/300 Read block 1010 len 16384
15592 146/300 Read block 4257 len 28672
15593 147/300 Read block 4881 len 32768
15594 148/300 Write block 321 len 32768 data: 5 6 7 7 6 6 6 6
15595 149/300 Read block 1475 len 4096
15596 150/300 Read block 4865 len 24576
15597 151/300 Read block 4363 len 16384
15598 152/300 Write block 4637 len 24576 data: 2 3 3 3 3 2
15599 153/300 Write block 4267 len 8192 data: 3 3
15600 154/300 Write block 1773 len 4096 data: 5
15601 155/300 Write block 2200 len 8192 data: 4 3
15602 156/300 Write block 450 len 8192 data: 4 4
15603 157/300 Read block 4045 len 28672
15604 158/300 Read block 973 len 16384
15605 159/300 Read block 959 len 40960
15606 160/300 Read block 4711 len 8192
15607 161/300 Write block 385 len 28672 data: 4 3 3 3 3 2 4
15608 162/300 Read block 476 len 4096
15609 163/300 Read block 2436 len 40960
15610 164/300 Write block 1468 len 16384 data: 3 3 3 2
15611 165/300 Read block 3213 len 4096
15612 166/300 Write block 749 len 12288 data: 2 2 2
15613 167/300 Read block 286 len 40960
15614 168/300 Flush
15615 169/300 Write block 1450 len 12288 data: 2 3 3
15616 170/300 Read block 2095 len 36864
15617 171/300 Write block 3417 len 4096 data: 4
15618 172/300 Write block 3029 len 8192 data: 5 4
15619 173/300 Flush
15620 174/300 Read block 982 len 40960
15621 175/300 Read block 701 len 36864
15622 176/300 Read block 1349 len 24576
15623 177/300 Write block 2216 len 24576 data: 5 4 3 2 2 2
15624 178/300 Write block 4352 len 20480 data: 3 4 3 3 3
15625 179/300 Write block 383 len 12288 data: 5 5 5
15626 180/300 Flush
15627 181/300 Read block 4538 len 8192
15628 182/300 Write block 2416 len 28672 data: 2 2 3 3 2 2 3
15629 183/300 Write block 4686 len 16384 data: 3 2 2 2
15630 184/300 Read block 2540 len 36864
15631 185/300 Write block 96 len 20480 data: 3 3 2 4 4
15632 186/300 Read block 3903 len 28672
15633 187/300 Write block 4277 len 32768 data: 2 2 2 2 4 3 4 4
15634 188/300 Read block 391 len 16384
15635 189/300 Flush
15636 190/300 Read block 3014 len 8192
15637 191/300 Write block 2616 len 12288 data: 4 4 3
15638 192/300 Read block 4520 len 40960
15639 193/300 Write block 2413 len 32768 data: 2 2 2 3 3 4 4 3
15640 194/300 Read block 117 len 12288
15641 195/300 Read block 2422 len 24576
15642 196/300 Read block 1331 len 32768
15643 197/300 Write block 1712 len 36864 data: 3 3 3 3 2 3 2 2 2
15644 198/300 Read block 1108 len 32768
15645 199/300 Read block 3474 len 24576
15646 200/300 Read block 1462 len 32768
15647 201/300 Read block 147 len 28672
15648 202/300 Write block 2962 len 20480 data: 5 5 4 4 4
15649 203/300 Write block 3564 len 32768 data: 2 3 3 3 3 2 2 3
15650 204/300 Write block 2465 len 16384 data: 4 4 5 4
15651 205/300 Write block 3305 len 36864 data: 3 3 3 3 2 3 6 5 5
15652 206/300 Read block 561 len 28672
15653 207/300 Write block 3319 len 20480 data: 2 2 3 3 3
15654 208/300 Read block 1643 len 8192
15655 209/300 Read block 4515 len 4096
15656 210/300 Read block 3307 len 36864
15657 211/300 Read block 1199 len 8192
15658 212/300 Write block 3834 len 12288 data: 4 4 3
15659 213/300 Write block 818 len 40960 data: 5 4 4 3 4 4 4 3 4 4
15660 214/300 Read block 4710 len 28672
15661 215/300 Write block 1190 len 28672 data: 5 3 3 3 3 6 6
15662 216/300 Write block 4783 len 8192 data: 4 4
15663 217/300 Write block 4810 len 32768 data: 4 3 3 4 3 2 2 2
15664 218/300 Write block 2355 len 20480 data: 3 3 3 2 2
15665 219/300 Read block 663 len 4096
15666 220/300 Read block 2327 len 8192
15667 221/300 Write block 332 len 28672 data: 5 3 3 3 3 3 3
15668 222/300 Write block 4405 len 24576 data: 2 2 2 3 3 3
15669 223/300 Read block 1453 len 40960
15670 224/300 Write block 3884 len 32768 data: 4 5 7 6 4 5 4 4
15671 225/300 Read block 2149 len 28672
15672 226/300 Read block 4291 len 8192
15673 227/300 Read block 3276 len 8192
15674 228/300 Read block 4452 len 40960
15675 229/300 Read block 595 len 40960
15676 230/300 Write block 1498 len 28672 data: 5 3 5 4 4 4 4
15677 231/300 Read block 387 len 24576
15678 232/300 Write block 4438 len 16384 data: 5 5 4 5
15679 233/300 Read block 2913 len 20480
15680 234/300 Read block 3595 len 4096
15681 235/300 Read block 4427 len 4096
15682 236/300 Write block 2746 len 16384 data: 3 3 4 4
15683 237/300 Read block 2641 len 16384
15684 238/300 Read block 3717 len 40960
15685 239/300 Read block 3494 len 4096
15686 240/300 Write block 3880 len 8192 data: 5 5
15687 241/300 Read block 4970 len 40960
15688 242/300 Flush
15689 243/300 Write block 2086 len 16384 data: 2 2 2 2
15690 244/300 Read block 1835 len 24576
15691 245/300 Write block 4618 len 8192 data: 4 4
15692 246/300 Read block 610 len 28672
15693 247/300 Flush
15694 248/300 Write block 934 len 24576 data: 2 2 2 3 3 4
15695 249/300 Flush
15696 250/300 Write block 968 len 40960 data: 4 4 4 3 3 6 5 5 3 3
15697 251/300 Read block 3008 len 12288
15698 252/300 Read block 351 len 8192
15699 253/300 Read block 3558 len 8192
15700 254/300 Read block 820 len 20480
15701 255/300 Write block 1746 len 4096 data: 5
15702 256/300 Read block 3086 len 16384
15703 257/300 Flush
15704 258/300 Read block 1094 len 20480
15705 259/300 Write block 4542 len 20480 data: 5 5 4 5 3
15706 260/300 Write block 3150 len 8192 data: 3 3
15707 261/300 Write block 4455 len 12288 data: 3 4 4
15708 262/300 Write block 261 len 24576 data: 3 3 3 3 4 4
15709 263/300 Read block 1026 len 24576
15710 264/300 Write block 3241 len 20480 data: 3 4 5 5 4
15711 265/300 Write block 2769 len 4096 data: 4
15712 266/300 Write block 3655 len 16384 data: 2 2 2 2
15713 267/300 Write block 2622 len 4096 data: 3
15714 268/300 Read block 4425 len 20480
15715 269/300 Write block 4122 len 4096 data: 5
15716 270/300 Read block 119 len 16384
15717 271/300 Read block 2694 len 24576
15718 272/300 Write block 1518 len 20480 data: 5 5 4 4 4
15719 273/300 Flush
15720 274/300 Read block 1255 len 12288
15721 275/300 Write block 4117 len 24576 data: 4 4 4 3 4 6
15722 276/300 Write block 1256 len 40960 data: 4 4 4 5 5 4 3 4 4 6
15723 277/300 Write block 3395 len 32768 data: 2 2 2 2 2 4 3 3
15724 278/300 Write block 3892 len 28672 data: 5 5 4 4 4 5 5
15725 279/300 Write block 1567 len 36864 data: 3 3 2 2 3 3 2 2 4
15726 280/300 Read block 1721 len 28672
15727 281/300 Read block 1845 len 28672
15728 282/300 Read block 4417 len 28672
15729 283/300 Read block 783 len 36864
15730 284/300 Write block 3567 len 20480 data: 4 4 3 3 4
15731 285/300 Read block 4975 len 36864
15732 286/300 Read block 3482 len 32768
15733 287/300 Flush
15734 288/300 Read block 712 len 12288
15735 289/300 Read block 62 len 40960
15736 290/300 Read block 877 len 28672
15737 291/300 Read block 3886 len 36864
15738 292/300 Read block 1916 len 32768
15739 293/300 Write block 834 len 40960 data: 4 4 4 4 4 4 5 3 2 2
15740 294/300 Write block 1040 len 24576 data: 3 4 3 3 3 3
15741 295/300 Flush
15742 296/300 Write block 50 len 40960 data: 7 5 4 4 5 2 2 2 3 4
15743 297/300 Read block 2998 len 32768
15744 298/300 Read block 598 len 36864
15745 299/300 Write block 1698 len 32768 data: 6 4 3 3 3 2 2 3
15746 300/300 Read block 1843 len 32768
15747 301/300 Write block 3149 len 20480 data: 3 4 4 3 4
15748 Main task has work to do, go find it
15749 got dsc Start(0)
15750 start 0
15751 Dec 11 23:43:49.945 INFO request completed, latency_us: 273, response_code: 204, uri: /start/cid/0, method: POST, req_id: 4b497395-7d90-411d-b87e-eff41fa03588, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
15752 Replay: started 0, returned:Ok(())
15753 ----------------------------------------------------------------
15754 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15755 States: Offline Active Active
15756 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15757 4926 Acked Write 7 Sent Done Done false
15758 4927 Acked Write 6 Sent Done Done false
15759 4928 Acked Flush 0 Sent Done Done false
15760 4929 Acked Write 1 Sent Done Done false
15761 4930 Acked Write 6 Sent Done Done false
15762 4931 Acked Read 5 Sent Done Done false
15763 4932 Acked Read 6 Sent Done Done false
15764 4933 Acked Write 5 Sent Done Done false
15765 4934 Acked Read 1 Sent Done Done false
15766 4935 Acked Read 3 Sent Done Done false
15767 4936 Acked Write 5 Sent Done Done false
15768 4937 Acked Read 8 Sent Done Done false
15769 4938 Acked Read 4 Sent Done Done false
15770 4939 Acked Read 4 Sent Done Done false
15771 4940 Acked Read 5 Sent Done Done false
15772 4941 Acked Read 10 Sent Done Done false
15773 4942 Acked Read 3 Sent Done Done false
15774 4943 Acked Read 2 Sent Done Done false
15775 4944 Acked Write 5 Sent Done Done false
15776 4945 Acked Write 3 Sent Done Done false
15777 4946 Acked Read 2 Sent Done Done false
15778 4947 Acked Read 10 Sent Done Done false
15779 4948 Acked Flush 0 Sent Done Done false
15780 4949 Acked Write 8 Sent Done Done false
15781 4950 Acked Write 5 Sent Done Done false
15782 4951 Acked Read 10 Sent Done Done false
15783 4952 Acked Read 4 Sent Done Done false
15784 4953 Acked Write 3 Sent Done Done false
15785 4954 Acked Read 10 Sent Done Done false
15786 4955 Acked Write 2 Sent Done Done false
15787 4956 Acked Read 7 Sent Done Done false
15788 4957 Acked Flush 0 Sent Done Done false
15789 4958 Acked Write 1 Sent Done Done false
15790 4959 Acked Read 10 Sent Done Done false
15791 4960 Acked Write 4 Sent Done Done false
15792 4961 Acked Read 10 Sent Done Done false
15793 4962 Acked Read 3 Sent Done Done false
15794 4963 Acked Write 9 Sent Done Done false
15795 4964 Acked Read 1 Sent Done Done false
15796 4965 Acked Write 2 Sent Done Done false
15797 4966 Acked Write 6 Sent Done Done false
15798 4967 Acked Write 1 Sent Done Done false
15799 4968 Acked Read 4 Sent Done Done false
15800 4969 Acked Read 6 Sent Done Done false
15801 4970 Acked Read 9 Sent Done Done false
15802 4971 Acked Read 10 Sent Done Done false
15803 4972 Acked Read 8 Sent Done Done false
15804 4973 Acked Read 4 Sent Done Done false
15805 4974 Acked Write 8 Sent Done Done false
15806 4975 Acked Write 3 Sent Done Done false
15807 4976 Acked Write 5 Sent Done Done false
15808 4977 Acked Read 2 Sent Done Done false
15809 4978 Acked Read 2 Sent Done Done false
15810 4979 Acked Write 7 Sent Done Done false
15811 4980 Acked Write 2 Sent Done Done false
15812 4981 Acked Flush 0 Sent Done Done false
15813 4982 Acked Write 6 Sent Done Done false
15814 4983 Acked Write 5 Sent Done Done false
15815 4984 Acked Write 4 Sent Done Done false
15816 4985 Acked Write 6 Sent Done Done false
15817 4986 Acked Read 8 Sent Done Done false
15818 4987 Acked Write 6 Sent Done Done false
15819 4988 Acked Write 4 Sent Done Done false
15820 4989 Acked Write 4 Sent Done Done false
15821 4990 Acked Read 9 Sent Done Done false
15822 4991 Acked Read 10 Sent Done Done false
15823 4992 Acked Read 1 Sent Done Done false
15824 4993 Acked Read 1 Sent Done Done false
15825 4994 Acked Write 1 Sent Done Done false
15826 4995 Acked Read 3 Sent Done Done false
15827 4996 Acked Write 10 Sent Done Done false
15828 4997 Acked Flush 0 Sent Done Done false
15829 4998 Acked Read 9 Sent Done Done false
15830 4999 Acked Write 10 Sent Done Done false
15831 5000 Acked Write 9 Sent Done Done false
15832 5001 Acked Read 3 Sent Done Done false
15833 5002 Acked Write 3 Sent Done Done false
15834 5003 Acked Write 2 Sent Done Done false
15835 5004 Acked Read 3 Sent Done Done false
15836 5005 Acked Read 1 Sent Done Done false
15837 5006 Acked Read 1 Sent Done Done false
15838 5007 Acked Write 10 Sent Done Done false
15839 5008 Acked Write 7 Sent Done Done false
15840 5009 Acked Read 10 Sent Done Done false
15841 5010 Acked Write 6 Sent Done Done false
15842 5011 Acked Read 4 Sent Done Done false
15843 5012 Acked Write 8 Sent Done Done false
15844 5013 Acked Write 2 Sent Done Done false
15845 5014 Acked Read 4 Sent Done Done false
15846 5015 Acked Read 3 Sent Done Done false
15847 5016 Acked Read 2 Sent Done Done false
15848 5017 Acked Read 3 Sent Done Done false
15849 5018 Acked Read 2 Sent Done Done false
15850 5019 Acked Write 6 Sent Done Done false
15851 5020 Acked Write 2 Sent Done Done false
15852 5021 Acked Read 6 Sent Done Done false
15853 5022 Acked Read 6 Sent Done Done false
15854 5023 Acked Read 3 Sent Done Done false
15855 5024 Acked Write 6 Sent Done Done false
15856 5025 Acked Read 2 Sent Done Done false
15857 5026 Acked Write 4 Sent Done Done false
15858 5027 Acked Read 7 Sent Done Done false
15859 5028 Acked Read 5 Sent Done Done false
15860 5029 Acked Read 5 Sent Done Done false
15861 5030 Acked Read 3 Sent Done Done false
15862 5031 Acked Flush 0 Sent Done Done false
15863 5032 Acked Write 1 Sent Done Done false
15864 5033 Acked Read 6 Sent Done Done false
15865 5034 Acked Write 4 Sent Done Done false
15866 5035 Acked Read 2 Sent Done Done false
15867 5036 Acked Write 4 Sent Done Done false
15868 5037 Acked Write 7 Sent Done Done false
15869 5038 Acked Flush 0 Sent Done Done false
15870 5039 Acked Write 9 Sent Done Done false
15871 5040 Acked Write 4 Sent Done Done false
15872 5041 Acked Read 3 Sent Done Done false
15873 5042 Acked Read 9 Sent Done Done false
15874 5043 Acked Read 8 Sent Done Done false
15875 5044 Acked Write 8 Sent Done Done false
15876 5045 Acked Read 2 Sent Done Done false
15877 5046 Acked Write 2 Sent Done Done false
15878 5047 Acked Write 1 Sent Done Done false
15879 5048 Acked Write 4 Sent Done Done false
15880 5049 Acked Read 5 Sent Done Done false
15881 5050 Acked Read 2 Sent Done Done false
15882 5051 Acked Read 3 Sent Done Done false
15883 5052 Acked Write 9 Sent Done Done false
15884 5053 Acked Write 7 Sent Done Done false
15885 5054 Acked Write 10 Sent Done Done false
15886 5055 Acked Read 3 Sent Done Done false
15887 5056 Acked Read 10 Sent Done Done false
15888 5057 Acked Read 2 Sent Done Done false
15889 5058 Acked Read 6 Sent Done Done false
15890 5059 Acked Write 8 Sent Done Done false
15891 5060 Acked Write 7 Sent Done Done false
15892 5061 Acked Write 7 Sent Done Done false
15893 5062 Acked Write 1 Sent Done Done false
15894 5063 Acked Write 10 Sent Done Done false
15895 5064 Acked Read 3 Sent Done Done false
15896 5065 Acked Read 2 Sent Done Done false
15897 5066 Acked Read 1 Sent Done Done false
15898 5067 Acked Write 4 Sent Done Done false
15899 5068 Acked Write 9 Sent Done Done false
15900 5069 Acked Read 6 Sent Done Done false
15901 5070 Acked Read 4 Sent Done Done false
15902 5071 Acked Read 7 Sent Done Done false
15903 5072 Acked Read 8 Sent Done Done false
15904 5073 Acked Write 8 Sent Done Done false
15905 5074 Acked Read 1 Sent Done Done false
15906 5075 Acked Read 6 Sent Done Done false
15907 5076 Acked Read 4 Sent Done Done false
15908 5077 Acked Write 6 Sent Done Done false
15909 5078 Acked Write 2 Sent Done Done false
15910 5079 Acked Write 1 Sent Done Done false
15911 5080 Acked Write 2 Sent Done Done false
15912 5081 Acked Write 2 Sent Done Done false
15913 5082 Acked Read 7 Sent Done Done false
15914 5083 Acked Read 4 Sent Done Done false
15915 5084 Acked Read 10 Sent Done Done false
15916 5085 Acked Read 2 Sent Done Done false
15917 5086 Acked Write 7 Sent Done Done false
15918 5087 Acked Read 1 Sent Done Done false
15919 5088 Acked Read 10 Sent Done Done false
15920 5089 Acked Write 4 Sent Done Done false
15921 5090 Acked Read 1 Sent Done Done false
15922 5091 Acked Write 3 Sent Done Done false
15923 5092 Acked Read 10 Sent Done Done false
15924 5093 Acked Flush 0 Sent Done Done false
15925 5094 Acked Write 3 Sent Done Done false
15926 5095 Acked Read 9 Sent Done Done false
15927 5096 Acked Write 1 Sent Done Done false
15928 5097 Acked Write 2 Sent Done Done false
15929 5098 Acked Flush 0 Sent Done Done false
15930 5099 Acked Read 10 Sent Done Done false
15931 5100 Acked Read 9 Sent Done Done false
15932 5101 Acked Read 6 Sent Done Done false
15933 5102 Acked Write 6 Sent Done Done false
15934 5103 Acked Write 5 Sent Done Done false
15935 5104 Acked Write 3 Sent Done Done false
15936 5105 Acked Flush 0 Sent Done Done false
15937 5106 Acked Read 2 Sent Done Done false
15938 5107 Acked Write 7 Sent Done Done false
15939 5108 Acked Write 4 Sent Done Done false
15940 5109 Acked Read 9 Sent Done Done false
15941 5110 Acked Write 5 Sent Done Done false
15942 5111 Acked Read 7 Sent Done Done false
15943 5112 Acked Write 8 Sent Done Done false
15944 5113 Acked Read 4 Sent Done Done false
15945 5114 Acked Flush 0 Sent Done Done false
15946 5115 Acked Read 2 Sent Done Done false
15947 5116 Acked Write 3 Sent Done Done false
15948 5117 Acked Read 10 Sent Done Done false
15949 5118 Acked Write 8 Sent Done Done false
15950 5119 Acked Read 3 Sent Done Done false
15951 5120 Acked Read 6 Sent Done Done false
15952 5121 Acked Read 8 Sent Done Done false
15953 5122 Acked Write 9 Sent Done Done false
15954 5123 Acked Read 8 Sent Done Done false
15955 5124 Acked Read 6 Sent Done Done false
15956 5125 Acked Read 8 Sent Done Done false
15957 5126 Acked Read 7 Sent Done Done false
15958 5127 Acked Write 5 Sent Done Done false
15959 5128 Acked Write 8 Sent Done Done false
15960 5129 Acked Write 4 Sent Done Done false
15961 5130 Acked Write 9 Sent Done Done false
15962 5131 Acked Read 7 Sent Done Done false
15963 5132 Acked Write 5 Sent Done Done false
15964 5133 Acked Read 2 Sent Done Done false
15965 5134 Acked Read 1 Sent Done Done false
15966 5135 Acked Read 9 Sent Done Done false
15967 5136 Acked Read 2 Sent Done Done false
15968 5137 Acked Write 3 Sent Done Done false
15969 5138 Acked Write 10 Sent Done Done false
15970 5139 Acked Read 7 Sent Done Done false
15971 5140 Acked Write 7 Sent Done Done false
15972 5141 Acked Write 2 Sent Done Done false
15973 5142 Acked Write 8 Sent Done Done false
15974 5143 Acked Write 5 Sent Done Done false
15975 5144 Acked Read 1 Sent Done Done false
15976 5145 Acked Read 2 Sent Done Done false
15977 5146 Acked Write 7 Sent Done Done false
15978 5147 Acked Write 6 Sent Done Done false
15979 5148 Acked Read 10 Sent Done Done false
15980 5149 Acked Write 8 Sent Done Done false
15981 5150 Acked Read 7 Sent Done Done false
15982 5151 Acked Read 2 Sent Done Done false
15983 5152 Acked Read 2 Sent Done Done false
15984 5153 Acked Read 10 Sent Done Done false
15985 5154 Acked Read 10 Sent Done Done false
15986 5155 Acked Write 7 Sent Done Done false
15987 5156 Acked Read 6 Sent Done Done false
15988 5157 Acked Write 4 Sent Done Done false
15989 5158 Acked Read 5 Sent Done Done false
15990 5159 Acked Read 1 Sent Done Done false
15991 5160 Acked Read 1 Sent Done Done false
15992 5161 Acked Write 4 Sent Done Done false
15993 5162 Acked Read 4 Sent Done Done false
15994 5163 Acked Read 10 Sent Done Done false
15995 5164 Acked Read 1 Sent Done Done false
15996 5165 Acked Write 2 Sent Done Done false
15997 5166 Acked Read 10 Sent Done Done false
15998 5167 Acked Flush 0 Sent Done Done false
15999 5168 Acked Write 4 Sent Done Done false
16000 5169 Acked Read 6 Sent Done Done false
16001 5170 Acked Write 2 Sent Done Done false
16002 5171 Acked Read 7 Sent Done Done false
16003 5172 Acked Flush 0 Sent Done Done false
16004 5173 Acked Write 6 Sent Done Done false
16005 5174 Acked Flush 0 Sent Done Done false
16006 5175 Acked Write 10 Sent Done Done false
16007 5176 Acked Read 3 Sent Done Done false
16008 5177 Acked Read 2 Sent Done Done false
16009 5178 Acked Read 2 Sent Done Done false
16010 5179 Acked Read 5 Sent Done Done false
16011 5180 Acked Write 1 Sent Done Done false
16012 5181 Acked Read 4 Sent Done Done false
16013 5182 Acked Flush 0 Sent Done Done false
16014 5183 Acked Read 5 Sent Done Done false
16015 5184 Acked Write 5 Sent Done Done false
16016 5185 Acked Write 2 Sent Done Done false
16017 5186 Acked Write 3 Sent Done Done false
16018 5187 Acked Write 6 Sent Done Done false
16019 5188 Acked Read 6 Sent Done Done false
16020 5189 Acked Write 5 Sent Done Done false
16021 5190 Acked Write 1 Sent Done Done false
16022 5191 Acked Write 4 Sent Done Done false
16023 5192 Acked Write 1 Sent Done Done false
16024 5193 Acked Read 5 Sent Done Done false
16025 5194 Acked Write 1 Sent Done Done false
16026 5195 Acked Read 4 Sent Done Done false
16027 5196 Acked Read 6 Sent Done Done false
16028 5197 Acked Write 5 Sent Done Done false
16029 5198 Acked Flush 0 Sent Done Done false
16030 5199 Acked Read 3 Sent Done Done false
16031 5200 Acked Write 6 Sent Done Done false
16032 5201 Acked Write 10 Sent Done Done false
16033 5202 Acked Write 8 Sent Done Done false
16034 5203 Acked Write 7 Sent Done Done false
16035 5204 Acked Write 9 Sent Done Done false
16036 5205 Acked Read 7 Sent Done Done false
16037 5206 Acked Read 7 Sent Done Done false
16038 5207 Acked Read 7 Sent Done Done false
16039 5208 Acked Read 9 Sent Done Done false
16040 5209 Acked Write 5 Sent Done Done false
16041 5210 Acked Read 9 Sent Done Done false
16042 5211 Acked Read 8 Sent Done Done false
16043 5212 Acked Flush 0 Sent Done Done false
16044 5213 Acked Read 3 Sent Done Done false
16045 5214 Acked Read 10 Sent Done Done false
16046 5215 Acked Read 7 Sent Done Done false
16047 5216 Acked Read 9 Sent Done Done false
16048 5217 Acked Read 8 Sent Done Done false
16049 5218 Acked Write 10 Sent Done Done false
16050 5219 Acked Write 6 Sent Done Done false
16051 5220 Acked Flush 0 Sent Done Done false
16052 5221 Acked Write 10 Sent Done Done false
16053 5222 Acked Read 8 Sent Done Done false
16054 5223 Acked Read 9 Sent Done Done false
16055 5224 Acked Write 8 Sent Done Done false
16056 5225 Acked Read 8 Sent Done Done false
16057 5226 Acked Write 5 Sent Done Done false
16058 STATES DS:0 DS:1 DS:2 TOTAL
16059 Sent 301 0 0 301
16060 Done 0 301 301 602
16061 Skipped 0 0 0 0
16062 Error 0 0 0 0
16063 Last Flush: 4925 5220 5220
16064 Downstairs last five completed: 4925 4924 4923 4922 4921
16065 Upstairs last five completed: 5226 5225 5224 5223 5222
16066 CLIENT: Up:0 ds:301 act:2
16067 [8810] got start action
16068 [8810] I am going to restart
16069 Starting downstairs at port 8810
16070 Make output file at "/tmp/dsc/downstairs-8810.txt"
16071 [8810][0] reports Starting
16072 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1214)
16073 [8810][0] reports Running
16074 ----------------------------------------------------------------
16075 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16076 States: Offline Active Active
16077 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16078 4926 Acked Write 7 Sent Done Done false
16079 4927 Acked Write 6 Sent Done Done false
16080 4928 Acked Flush 0 Sent Done Done false
16081 4929 Acked Write 1 Sent Done Done false
16082 4930 Acked Write 6 Sent Done Done false
16083 4931 Acked Read 5 Sent Done Done false
16084 4932 Acked Read 6 Sent Done Done false
16085 4933 Acked Write 5 Sent Done Done false
16086 4934 Acked Read 1 Sent Done Done false
16087 4935 Acked Read 3 Sent Done Done false
16088 4936 Acked Write 5 Sent Done Done false
16089 4937 Acked Read 8 Sent Done Done false
16090 4938 Acked Read 4 Sent Done Done false
16091 4939 Acked Read 4 Sent Done Done false
16092 4940 Acked Read 5 Sent Done Done false
16093 4941 Acked Read 10 Sent Done Done false
16094 4942 Acked Read 3 Sent Done Done false
16095 4943 Acked Read 2 Sent Done Done false
16096 4944 Acked Write 5 Sent Done Done false
16097 4945 Acked Write 3 Sent Done Done false
16098 4946 Acked Read 2 Sent Done Done false
16099 4947 Acked Read 10 Sent Done Done false
16100 4948 Acked Flush 0 Sent Done Done false
16101 4949 Acked Write 8 Sent Done Done false
16102 4950 Acked Write 5 Sent Done Done false
16103 4951 Acked Read 10 Sent Done Done false
16104 4952 Acked Read 4 Sent Done Done false
16105 4953 Acked Write 3 Sent Done Done false
16106 4954 Acked Read 10 Sent Done Done false
16107 4955 Acked Write 2 Sent Done Done false
16108 4956 Acked Read 7 Sent Done Done false
16109 4957 Acked Flush 0 Sent Done Done false
16110 4958 Acked Write 1 Sent Done Done false
16111 4959 Acked Read 10 Sent Done Done false
16112 4960 Acked Write 4 Sent Done Done false
16113 4961 Acked Read 10 Sent Done Done false
16114 4962 Acked Read 3 Sent Done Done false
16115 4963 Acked Write 9 Sent Done Done false
16116 4964 Acked Read 1 Sent Done Done false
16117 4965 Acked Write 2 Sent Done Done false
16118 4966 Acked Write 6 Sent Done Done false
16119 4967 Acked Write 1 Sent Done Done false
16120 4968 Acked Read 4 Sent Done Done false
16121 4969 Acked Read 6 Sent Done Done false
16122 4970 Acked Read 9 Sent Done Done false
16123 4971 Acked Read 10 Sent Done Done false
16124 4972 Acked Read 8 Sent Done Done false
16125 4973 Acked Read 4 Sent Done Done false
16126 4974 Acked Write 8 Sent Done Done false
16127 4975 Acked Write 3 Sent Done Done false
16128 4976 Acked Write 5 Sent Done Done false
16129 4977 Acked Read 2 Sent Done Done false
16130 4978 Acked Read 2 Sent Done Done false
16131 4979 Acked Write 7 Sent Done Done false
16132 4980 Acked Write 2 Sent Done Done false
16133 4981 Acked Flush 0 Sent Done Done false
16134 4982 Acked Write 6 Sent Done Done false
16135 4983 Acked Write 5 Sent Done Done false
16136 4984 Acked Write 4 Sent Done Done false
16137 4985 Acked Write 6 Sent Done Done false
16138 4986 Acked Read 8 Sent Done Done false
16139 4987 Acked Write 6 Sent Done Done false
16140 4988 Acked Write 4 Sent Done Done false
16141 4989 Acked Write 4 Sent Done Done false
16142 4990 Acked Read 9 Sent Done Done false
16143 4991 Acked Read 10 Sent Done Done false
16144 4992 Acked Read 1 Sent Done Done false
16145 4993 Acked Read 1 Sent Done Done false
16146 4994 Acked Write 1 Sent Done Done false
16147 4995 Acked Read 3 Sent Done Done false
16148 4996 Acked Write 10 Sent Done Done false
16149 4997 Acked Flush 0 Sent Done Done false
16150 4998 Acked Read 9 Sent Done Done false
16151 4999 Acked Write 10 Sent Done Done false
16152 5000 Acked Write 9 Sent Done Done false
16153 5001 Acked Read 3 Sent Done Done false
16154 5002 Acked Write 3 Sent Done Done false
16155 5003 Acked Write 2 Sent Done Done false
16156 5004 Acked Read 3 Sent Done Done false
16157 5005 Acked Read 1 Sent Done Done false
16158 5006 Acked Read 1 Sent Done Done false
16159 5007 Acked Write 10 Sent Done Done false
16160 5008 Acked Write 7 Sent Done Done false
16161 5009 Acked Read 10 Sent Done Done false
16162 5010 Acked Write 6 Sent Done Done false
16163 5011 Acked Read 4 Sent Done Done false
16164 5012 Acked Write 8 Sent Done Done false
16165 5013 Acked Write 2 Sent Done Done false
16166 5014 Acked Read 4 Sent Done Done false
16167 5015 Acked Read 3 Sent Done Done false
16168 5016 Acked Read 2 Sent Done Done false
16169 5017 Acked Read 3 Sent Done Done false
16170 5018 Acked Read 2 Sent Done Done false
16171 5019 Acked Write 6 Sent Done Done false
16172 5020 Acked Write 2 Sent Done Done false
16173 5021 Acked Read 6 Sent Done Done false
16174 5022 Acked Read 6 Sent Done Done false
16175 5023 Acked Read 3 Sent Done Done false
16176 5024 Acked Write 6 Sent Done Done false
16177 5025 Acked Read 2 Sent Done Done false
16178 5026 Acked Write 4 Sent Done Done false
16179 5027 Acked Read 7 Sent Done Done false
16180 5028 Acked Read 5 Sent Done Done false
16181 5029 Acked Read 5 Sent Done Done false
16182 5030 Acked Read 3 Sent Done Done false
16183 5031 Acked Flush 0 Sent Done Done false
16184 5032 Acked Write 1 Sent Done Done false
16185 5033 Acked Read 6 Sent Done Done false
16186 5034 Acked Write 4 Sent Done Done false
16187 5035 Acked Read 2 Sent Done Done false
16188 5036 Acked Write 4 Sent Done Done false
16189 5037 Acked Write 7 Sent Done Done false
16190 5038 Acked Flush 0 Sent Done Done false
16191 5039 Acked Write 9 Sent Done Done false
16192 5040 Acked Write 4 Sent Done Done false
16193 5041 Acked Read 3 Sent Done Done false
16194 5042 Acked Read 9 Sent Done Done false
16195 5043 Acked Read 8 Sent Done Done false
16196 5044 Acked Write 8 Sent Done Done false
16197 5045 Acked Read 2 Sent Done Done false
16198 5046 Acked Write 2 Sent Done Done false
16199 5047 Acked Write 1 Sent Done Done false
16200 5048 Acked Write 4 Sent Done Done false
16201 5049 Acked Read 5 Sent Done Done false
16202 5050 Acked Read 2 Sent Done Done false
16203 5051 Acked Read 3 Sent Done Done false
16204 5052 Acked Write 9 Sent Done Done false
16205 5053 Acked Write 7 Sent Done Done false
16206 5054 Acked Write 10 Sent Done Done false
16207 5055 Acked Read 3 Sent Done Done false
16208 5056 Acked Read 10 Sent Done Done false
16209 5057 Acked Read 2 Sent Done Done false
16210 5058 Acked Read 6 Sent Done Done false
16211 5059 Acked Write 8 Sent Done Done false
16212 5060 Acked Write 7 Sent Done Done false
16213 5061 Acked Write 7 Sent Done Done false
16214 5062 Acked Write 1 Sent Done Done false
16215 5063 Acked Write 10 Sent Done Done false
16216 5064 Acked Read 3 Sent Done Done false
16217 5065 Acked Read 2 Sent Done Done false
16218 5066 Acked Read 1 Sent Done Done false
16219 5067 Acked Write 4 Sent Done Done false
16220 5068 Acked Write 9 Sent Done Done false
16221 5069 Acked Read 6 Sent Done Done false
16222 5070 Acked Read 4 Sent Done Done false
16223 5071 Acked Read 7 Sent Done Done false
16224 5072 Acked Read 8 Sent Done Done false
16225 5073 Acked Write 8 Sent Done Done false
16226 5074 Acked Read 1 Sent Done Done false
16227 5075 Acked Read 6 Sent Done Done false
16228 5076 Acked Read 4 Sent Done Done false
16229 5077 Acked Write 6 Sent Done Done false
16230 5078 Acked Write 2 Sent Done Done false
16231 5079 Acked Write 1 Sent Done Done false
16232 5080 Acked Write 2 Sent Done Done false
16233 5081 Acked Write 2 Sent Done Done false
16234 5082 Acked Read 7 Sent Done Done false
16235 5083 Acked Read 4 Sent Done Done false
16236 5084 Acked Read 10 Sent Done Done false
16237 5085 Acked Read 2 Sent Done Done false
16238 5086 Acked Write 7 Sent Done Done false
16239 5087 Acked Read 1 Sent Done Done false
16240 5088 Acked Read 10 Sent Done Done false
16241 5089 Acked Write 4 Sent Done Done false
16242 5090 Acked Read 1 Sent Done Done false
16243 5091 Acked Write 3 Sent Done Done false
16244 5092 Acked Read 10 Sent Done Done false
16245 5093 Acked Flush 0 Sent Done Done false
16246 5094 Acked Write 3 Sent Done Done false
16247 5095 Acked Read 9 Sent Done Done false
16248 5096 Acked Write 1 Sent Done Done false
16249 5097 Acked Write 2 Sent Done Done false
16250 5098 Acked Flush 0 Sent Done Done false
16251 5099 Acked Read 10 Sent Done Done false
16252 5100 Acked Read 9 Sent Done Done false
16253 5101 Acked Read 6 Sent Done Done false
16254 5102 Acked Write 6 Sent Done Done false
16255 5103 Acked Write 5 Sent Done Done false
16256 5104 Acked Write 3 Sent Done Done false
16257 5105 Acked Flush 0 Sent Done Done false
16258 5106 Acked Read 2 Sent Done Done false
16259 5107 Acked Write 7 Sent Done Done false
16260 5108 Acked Write 4 Sent Done Done false
16261 5109 Acked Read 9 Sent Done Done false
16262 5110 Acked Write 5 Sent Done Done false
16263 5111 Acked Read 7 Sent Done Done false
16264 5112 Acked Write 8 Sent Done Done false
16265 5113 Acked Read 4 Sent Done Done false
16266 5114 Acked Flush 0 Sent Done Done false
16267 5115 Acked Read 2 Sent Done Done false
16268 5116 Acked Write 3 Sent Done Done false
16269 5117 Acked Read 10 Sent Done Done false
16270 5118 Acked Write 8 Sent Done Done false
16271 5119 Acked Read 3 Sent Done Done false
16272 5120 Acked Read 6 Sent Done Done false
16273 5121 Acked Read 8 Sent Done Done false
16274 5122 Acked Write 9 Sent Done Done false
16275 5123 Acked Read 8 Sent Done Done false
16276 5124 Acked Read 6 Sent Done Done false
16277 5125 Acked Read 8 Sent Done Done false
16278 5126 Acked Read 7 Sent Done Done false
16279 5127 Acked Write 5 Sent Done Done false
16280 5128 Acked Write 8 Sent Done Done false
16281 5129 Acked Write 4 Sent Done Done false
16282 5130 Acked Write 9 Sent Done Done false
16283 5131 Acked Read 7 Sent Done Done false
16284 5132 Acked Write 5 Sent Done Done false
16285 5133 Acked Read 2 Sent Done Done false
16286 5134 Acked Read 1 Sent Done Done false
16287 5135 Acked Read 9 Sent Done Done false
16288 5136 Acked Read 2 Sent Done Done false
16289 5137 Acked Write 3 Sent Done Done false
16290 5138 Acked Write 10 Sent Done Done false
16291 5139 Acked Read 7 Sent Done Done false
16292 5140 Acked Write 7 Sent Done Done false
16293 5141 Acked Write 2 Sent Done Done false
16294 5142 Acked Write 8 Sent Done Done false
16295 5143 Acked Write 5 Sent Done Done false
16296 5144 Acked Read 1 Sent Done Done false
16297 5145 Acked Read 2 Sent Done Done false
16298 5146 Acked Write 7 Sent Done Done false
16299 5147 Acked Write 6 Sent Done Done false
16300 5148 Acked Read 10 Sent Done Done false
16301 5149 Acked Write 8 Sent Done Done false
16302 5150 Acked Read 7 Sent Done Done false
16303 5151 Acked Read 2 Sent Done Done false
16304 5152 Acked Read 2 Sent Done Done false
16305 5153 Acked Read 10 Sent Done Done false
16306 5154 Acked Read 10 Sent Done Done false
16307 5155 Acked Write 7 Sent Done Done false
16308 5156 Acked Read 6 Sent Done Done false
16309 5157 Acked Write 4 Sent Done Done false
16310 5158 Acked Read 5 Sent Done Done false
16311 5159 Acked Read 1 Sent Done Done false
16312 5160 Acked Read 1 Sent Done Done false
16313 5161 Acked Write 4 Sent Done Done false
16314 5162 Acked Read 4 Sent Done Done false
16315 5163 Acked Read 10 Sent Done Done false
16316 5164 Acked Read 1 Sent Done Done false
16317 5165 Acked Write 2 Sent Done Done false
16318 5166 Acked Read 10 Sent Done Done false
16319 5167 Acked Flush 0 Sent Done Done false
16320 5168 Acked Write 4 Sent Done Done false
16321 5169 Acked Read 6 Sent Done Done false
16322 5170 Acked Write 2 Sent Done Done false
16323 5171 Acked Read 7 Sent Done Done false
16324 5172 Acked Flush 0 Sent Done Done false
16325 5173 Acked Write 6 Sent Done Done false
16326 5174 Acked Flush 0 Sent Done Done false
16327 5175 Acked Write 10 Sent Done Done false
16328 5176 Acked Read 3 Sent Done Done false
16329 5177 Acked Read 2 Sent Done Done false
16330 5178 Acked Read 2 Sent Done Done false
16331 5179 Acked Read 5 Sent Done Done false
16332 5180 Acked Write 1 Sent Done Done false
16333 5181 Acked Read 4 Sent Done Done false
16334 5182 Acked Flush 0 Sent Done Done false
16335 5183 Acked Read 5 Sent Done Done false
16336 5184 Acked Write 5 Sent Done Done false
16337 5185 Acked Write 2 Sent Done Done false
16338 5186 Acked Write 3 Sent Done Done false
16339 5187 Acked Write 6 Sent Done Done false
16340 5188 Acked Read 6 Sent Done Done false
16341 5189 Acked Write 5 Sent Done Done false
16342 5190 Acked Write 1 Sent Done Done false
16343 5191 Acked Write 4 Sent Done Done false
16344 5192 Acked Write 1 Sent Done Done false
16345 5193 Acked Read 5 Sent Done Done false
16346 5194 Acked Write 1 Sent Done Done false
16347 5195 Acked Read 4 Sent Done Done false
16348 5196 Acked Read 6 Sent Done Done false
16349 5197 Acked Write 5 Sent Done Done false
16350 5198 Acked Flush 0 Sent Done Done false
16351 5199 Acked Read 3 Sent Done Done false
16352 5200 Acked Write 6 Sent Done Done false
16353 5201 Acked Write 10 Sent Done Done false
16354 5202 Acked Write 8 Sent Done Done false
16355 5203 Acked Write 7 Sent Done Done false
16356 5204 Acked Write 9 Sent Done Done false
16357 5205 Acked Read 7 Sent Done Done false
16358 5206 Acked Read 7 Sent Done Done false
16359 5207 Acked Read 7 Sent Done Done false
16360 5208 Acked Read 9 Sent Done Done false
16361 5209 Acked Write 5 Sent Done Done false
16362 5210 Acked Read 9 Sent Done Done false
16363 5211 Acked Read 8 Sent Done Done false
16364 5212 Acked Flush 0 Sent Done Done false
16365 5213 Acked Read 3 Sent Done Done false
16366 5214 Acked Read 10 Sent Done Done false
16367 5215 Acked Read 7 Sent Done Done false
16368 5216 Acked Read 9 Sent Done Done false
16369 5217 Acked Read 8 Sent Done Done false
16370 5218 Acked Write 10 Sent Done Done false
16371 5219 Acked Write 6 Sent Done Done false
16372 5220 Acked Flush 0 Sent Done Done false
16373 5221 Acked Write 10 Sent Done Done false
16374 5222 Acked Read 8 Sent Done Done false
16375 5223 Acked Read 9 Sent Done Done false
16376 5224 Acked Write 8 Sent Done Done false
16377 5225 Acked Read 8 Sent Done Done false
16378 5226 Acked Write 5 Sent Done Done false
16379 5227 Acked Flush 0 Sent Done Done false
16380 STATES DS:0 DS:1 DS:2 TOTAL
16381 Sent 302 0 0 302
16382 Done 0 302 302 604
16383 Skipped 0 0 0 0
16384 Error 0 0 0 0
16385 Last Flush: 4925 5227 5227
16386 Downstairs last five completed: 4925 4924 4923 4922 4921
16387 Upstairs last five completed: 5227 5226 5225 5224 5223
16388 CLIENT: Up:0 ds:302 act:2
163892024-12-11T23:43:55.708ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163902024-12-11T23:43:55.708ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62373) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163912024-12-11T23:43:55.709ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163922024-12-11T23:43:55.710ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163932024-12-11T23:43:55.710ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163942024-12-11T23:43:55.710ZINFOcrucible: send last flush ID to this DS: 4925 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163952024-12-11T23:43:55.710ZINFOcrucible: Replied this last flush ID: 4925 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163962024-12-11T23:43:55.710ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163972024-12-11T23:43:55.710ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163982024-12-11T23:43:55.710ZINFOcrucible: [0] client re-new 302 jobs since flush 4925 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
163992024-12-11T23:43:55.710ZINFOcrucible: [0] Replayed 302 jobs since flush: 4925 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
16400 ----------------------------------------------------------------
16401 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16402 Downstairs last five completed: 5227 5226 5225 5224 5223
16403 Upstairs last five completed: 5227 5226 5225 5224 5223
16404 CLIENT: Up:0 ds:0 act:3
16405 Replay: All jobs finished
16406 Main task has work to do, go find it
16407 got dsc Stop(0)
16408 stop 0
16409 Dec 11 23:43:57.951 INFO request completed, latency_us: 348, response_code:[8810] Got stop action so:false kr:false
16410 204, uri: /stop/cid/0, method: POST, req_id: 48415885-baa8-44ab-97cc-5443dbf68b50, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
164112024-12-11T23:43:57.951ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
164122024-12-11T23:43:57.951ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
164132024-12-11T23:43:57.951ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
164142024-12-11T23:43:57.951ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
16415 Dec 11 23:43:57.951 INFO request completed, latency_us: 142, response_code: 200, uri:{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:43:57.95196111Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
16416 /state/cid/0, method: GET, req_id: 629f7883-e395-40ba-a82a-c055099045a9, remote_addr: 127.0.0.1:62969, local_addr: 127{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:43:57.951990841Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
16417 .0.0.1:9998
16418 [8810] Exited with: ExitStatus(unix_wait_status(9))
16419 [8810][0] reports Exit
16420 Dec 11 23:44:01.955 INFO request completed, latency_us: 241, response_code: 200, uri: /state/cid/0, method: GET, req_id: 538441b5-7b41-4dbc-8b56-96b27bde1b62, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
16421 001/300 Write block 1815 len 4096 data: 2
16422 002/300 Read block 2381 len 20480
16423 003/300 Read block 3946 len 12288
16424 004/300 Flush
16425 005/300 Write block 1105 len 4096 data: 3
16426 006/300 Read block 131 len 28672
16427 007/300 Write block 2838 len 32768 data: 5 4 4 4 4 2 4 4
16428 008/300 Read block 4075 len 36864
16429 009/300 Flush
16430 010/300 Read block 1826 len 32768
16431 011/300 Write block 2033 len 20480 data: 5 5 5 5 5
16432 012/300 Read block 541 len 40960
16433 013/300 Write block 4721 len 32768 data: 4 4 5 4 3 3 3 3
16434 014/300 Read block 3344 len 16384
16435 015/300 Write block 388 len 12288 data: 4 4 3
16436 016/300 Read block 1029 len 12288
16437 017/300 Read block 312 len 40960
16438 018/300 Read block 4726 len 28672
16439 019/300 Write block 1368 len 4096 data: 3
16440 020/300 Read block 213 len 36864
16441 021/300 Write block 1018 len 28672 data: 4 4 3 4 4 3 3
16442 022/300 Read block 3251 len 40960
16443 023/300 Read block 1875 len 20480
16444 024/300 Read block 3120 len 8192
16445 025/300 Read block 2582 len 16384
16446 026/300 Read block 4405 len 8192
16447 027/300 Write block 3691 len 4096 data: 5
16448 028/300 Write block 3089 len 8192 data: 7 7
16449 029/300 Read block 4121 len 4096
16450 030/300 Read block 3505 len 4096
16451 031/300 Read block 3240 len 16384
16452 032/300 Write block 4206 len 16384 data: 4 5 5 4
16453 033/300 Write block 3445 len 4096 data: 3
16454 034/300 Write block 3182 len 4096 data: 4
16455 035/300 Read block 845 len 16384
16456 036/300 Write block 1308 len 32768 data: 3 3 4 7 7 6 4 5
16457 037/300 Read block 3328 len 12288
16458 038/300 Write block 3285 len 28672 data: 4 4 4 3 3 4 4
16459 039/300 Read block 2078 len 40960
16460 040/300 Write block 1258 len 12288 data: 5 6 6
16461 041/300 Read block 711 len 16384
16462 042/300 Read block 1644 len 16384
16463 043/300 Read block 4128 len 16384
16464 044/300 Read block 4514 len 28672
16465 045/300 Read block 2390 len 8192
16466 046/300 Flush
16467 047/300 Read block 2468 len 24576
16468 048/300 Read block 2400 len 12288
16469 049/300 Read block 2887 len 28672
16470 050/300 Read block 4267 len 28672
16471 051/300 Read block 4932 len 24576
16472 052/300 Read block 3481 len 4096
16473 053/300 Read block 1974 len 24576
16474 054/300 Write block 4693 len 24576 data: 4 4 4 4 4 3
16475 055/300 Read block 3012 len 32768
16476 056/300 Write block 3727 len 20480 data: 6 4 4 4 4
16477 057/300 Flush
16478 058/300 Write block 3581 len 28672 data: 6 4 4 4 5 4 6
16479 059/300 Flush
16480 060/300 Write block 2053 len 8192 data: 4 3
16481 061/300 Read block 564 len 20480
16482 062/300 Write block 4501 len 4096 data: 4
16483 063/300 Write block 2462 len 28672 data: 4 4 4 5 5 6 5
16484 064/300 Read block 4356 len 28672
16485 065/300 Write block 1131 len 36864 data: 2 2 3 5 5 5 5 5 5
16486 066/300 Write block 1620 len 32768 data: 4 4 4 3 4 5 6 6
16487 067/300 Read block 848 len 32768
16488 068/300 Flush
16489 069/300 Write block 3224 len 36864 data: 3 3 4 4 4 3 4 3 3
16490 070/300 Write block 1236 len 4096 data: 3
16491 071/300 Read block 4464 len 40960
16492 072/300 Read block 124 len 40960
16493 073/300 Write block 2693 len 4096 data: 5
16494 074/300 Flush
16495 075/300 Read block 58 len 24576
16496 076/300 Write block 156 len 8192 data: 5 4
16497 077/300 Read block 3148 len 32768
16498 078/300 Read block 4441 len 36864
16499 079/300 Write block 409 len 40960 data: 6 7 6 5 5 5 4 3 3 3
16500 080/300 Write block 551 len 4096 data: 4
16501 081/300 Read block 118 len 12288
16502 082/300 Flush
16503 083/300 Read block 694 len 4096
16504 084/300 Flush
16505 085/300 Write block 3380 len 36864 data: 3 3 3 5 5 6 6 6 5
16506 086/300 Flush
16507 087/300 Write block 3410 len 12288 data: 2 2 2
16508 088/300 Flush
16509 089/300 Write block 4784 len 16384 data: 5 4 3 3
16510 090/300 Read block 125 len 20480
16511 091/300 Read block 3740 len 4096
16512 092/300 Read block 4360 len 24576
16513 093/300 Read block 4090 len 36864
16514 094/300 Read block 2850 len 8192
16515 095/300 Write block 3009 len 32768 data: 6 4 6 6 5 5 4 5
16516 096/300 Write block 1683 len 32768 data: 5 6 5 5 6 6 6 6
16517 097/300 Flush
16518 098/300 Write block 2479 len 8192 data: 3 3
16519 099/300 Write block 1587 len 12288 data: 2 2 2
16520 100/300 Write block 1358 len 8192 data: 3 3
16521 101/300 Write block 2974 len 32768 data: 2 2 4 4 4 4 4 5
16522 102/300 Read block 612 len 32768
16523 103/300 Read block 1519 len 40960
16524 104/300 Read block 4314 len 36864
16525 105/300 Read block 1441 len 16384
16526 106/300 Read block 587 len 40960
16527 107/300 Read block 4531 len 24576
16528 108/300 Write block 2738 len 40960 data: 6 5 5 5 5 4 5 3 4 4
16529 109/300 Flush
16530 110/300 Flush
16531 111/300 Read block 2338 len 20480
16532 112/300 Read block 1417 len 12288
16533 113/300 Read block 3070 len 20480
16534 114/300 Flush
16535 115/300 Write block 3518 len 4096 data: 2
16536 116/300 Write block 2544 len 32768 data: 4 4 4 4 4 4 4 5
16537 117/300 Write block 3739 len 24576 data: 2 3 3 4 4 4
16538 118/300 Write block 3432 len 16384 data: 4 4 4 5
16539 119/300 Read block 437 len 24576
16540 120/300 Read block 4004 len 16384
16541 121/300 Read block 2981 len 12288
16542 122/300 Write block 2408 len 12288 data: 3 2 2
16543 123/300 Read block 4934 len 36864
16544 124/300 Write block 1025 len 20480 data: 3 3 3 2 3
16545 125/300 Read block 2908 len 8192
16546 126/300 Read block 4352 len 24576
16547 127/300 Read block 2080 len 8192
16548 128/300 Write block 3683 len 36864 data: 4 4 4 3 4 5 5 5 6
16549 129/300 Read block 2630 len 12288
16550 130/300 Write block 1138 len 24576 data: 6 6 5 4 3 2
16551 131/300 Read block 3590 len 12288
16552 132/300 Write block 3701 len 28672 data: 5 7 4 3 3 3 3
16553 133/300 Write block 2895 len 20480 data: 6 5 6 8 7
16554 134/300 Write block 1564 len 32768 data: 4 4 3 4 4 3 3 4
16555 135/300 Write block 395 len 16384 data: 3 4 3 4
16556 136/300 Write block 4173 len 12288 data: 4 4 5
16557 137/300 Flush
16558 138/300 Read block 4930 len 16384
16559 139/300 Read block 2238 len 12288
16560 140/300 Read block 3624 len 28672
16561 141/300 Read block 1248 len 20480
16562 142/300 Read block 2460 len 28672
16563 143/300 Flush
16564 144/300 Flush
16565 145/300 Write block 4592 len 36864 data: 3 3 4 4 5 6 6 5 5
16566 146/300 Write block 2903 len 40960 data: 6 3 3 2 3 3 3 3 3 3
16567 147/300 Read block 2479 len 24576
16568 148/300 Read block 4565 len 40960
16569 149/300 Write block 1000 len 40960 data: 4 5 4 3 3 4 5 5 5 5
16570 150/300 Read block 4141 len 36864
16571 151/300 Flush
16572 152/300 Read block 4016 len 32768
16573 153/300 Write block 2804 len 36864 data: 7 5 6 5 5 5 5 5 4
16574 154/300 Flush
16575 155/300 Write block 4561 len 20480 data: 4 4 4 4 4
16576 156/300 Read block 3996 len 40960
16577 157/300 Read block 884 len 8192
16578 158/300 Read block 4356 len 32768
16579 159/300 Write block 4440 len 36864 data: 5 6 5 5 5 4 4 3 3
16580 160/300 Read block 22 len 16384
16581 161/300 Write block 2382 len 16384 data: 4 3 2 2
16582 162/300 Read block 4009 len 16384
16583 163/300 Read block 632 len 16384
16584 164/300 Read block 2251 len 20480
16585 165/300 Write block 695 len 16384 data: 3 3 3 3
16586 166/300 Read block 3186 len 16384
16587 167/300 Flush
16588 168/300 Read block 1079 len 16384
16589 169/300 Write block 4337 len 36864 data: 2 2 2 3 3 3 2 2 3
16590 170/300 Read block 2033 len 40960
16591 171/300 Read block 1882 len 8192
16592 172/300 Write block 3113 len 12288 data: 3 3 4
16593 173/300 Write block 4355 len 20480 data: 4 4 3 3 3
16594 174/300 Read block 168 len 16384
16595 175/300 Write block 3496 len 28672 data: 3 3 3 4 4 4 3
16596 176/300 Flush
16597 177/300 Read block 3099 len 28672
16598 178/300 Flush
16599 179/300 Write block 1307 len 12288 data: 4 4 4
16600 180/300 Read block 3287 len 8192
16601 181/300 Write block 3844 len 20480 data: 2 2 2 2 3
16602 182/300 Write block 4095 len 20480 data: 3 3 3 3 2
16603 183/300 Read block 2048 len 8192
16604 184/300 Write block 1903 len 4096 data: 3
16605 185/300 Write block 3970 len 36864 data: 2 2 3 3 3 6 6 5 4
16606 186/300 Read block 4066 len 4096
16607 187/300 Read block 1601 len 4096
16608 188/300 Read block 2547 len 24576
16609 189/300 Read block 3424 len 4096
16610 190/300 Write block 4700 len 40960 data: 3 4 4 4 4 4 4 3 2 2
16611 191/300 Write block 473 len 40960 data: 2 3 3 3 4 4 4 4 3 4
16612 192/300 Write block 4813 len 36864 data: 5 4 3 3 3 2 2 2 3
16613 193/300 Read block 1285 len 36864
16614 194/300 Flush
16615 195/300 Flush
16616 196/300 Write block 4863 len 32768 data: 3 4 3 3 3 3 3 4
16617 197/300 Read block 993 len 8192
16618 198/300 Read block 1333 len 16384
16619 199/300 Write block 1424 len 16384 data: 2 2 4 3
16620 200/300 Flush
16621 201/300 Write block 118 len 36864 data: 6 6 5 5 5 6 5 5 5
16622 202/300 Read block 4419 len 4096
16623 203/300 Write block 1157 len 28672 data: 3 3 3 3 5 4 4
16624 204/300 Flush
16625 205/300 Write block 3130 len 4096 data: 6
16626 206/300 Read block 4577 len 4096
16627 207/300 Write block 288 len 32768 data: 6 5 6 4 3 4 4 3
16628 208/300 Read block 2328 len 12288
16629 209/300 Read block 3997 len 40960
16630 210/300 Read block 1024 len 28672
16631 211/300 Read block 161 len 20480
16632 212/300 Read block 4922 len 24576
16633 213/300 Write block 4690 len 24576 data: 2 2 2 5 5 5
16634 214/300 Read block 3049 len 20480
16635 215/300 Read block 4013 len 16384
16636 216/300 Read block 594 len 12288
16637 217/300 Read block 1390 len 12288
16638 218/300 Flush
16639 219/300 Read block 375 len 8192
16640 220/300 Read block 3412 len 16384
16641 221/300 Write block 2574 len 24576 data: 7 8 8 9 10 9
16642 222/300 Read block 87 len 24576
16643 223/300 Write block 645 len 4096 data: 4
16644 224/300 Read block 2814 len 16384
16645 225/300 Read block 1271 len 40960
16646 226/300 Read block 3110 len 12288
16647 227/300 Read block 2396 len 16384
16648 228/300 Flush
16649 229/300 Write block 2148 len 12288 data: 3 3 2
16650 230/300 Flush
16651 231/300 Write block 3282 len 8192 data: 5 4
16652 232/300 Write block 2737 len 12288 data: 7 7 6
16653 233/300 Write block 711 len 32768 data: 4 4 4 3 2 2 2 3
16654 234/300 Write block 311 len 32768 data: 3 2 2 3 3 3 4 4
16655 235/300 Read block 4978 len 12288
16656 236/300 Write block 3460 len 40960 data: 3 3 3 3 2 2 2 2 2 3
16657 237/300 Write block 4507 len 36864 data: 3 3 3 3 3 2 2 2 3
16658 238/300 Read block 4678 len 4096
16659 239/300 Read block 1395 len 16384
16660 240/300 Read block 3549 len 16384
16661 241/300 Write block 3031 len 8192 data: 4 3
16662 242/300 Read block 2089 len 28672
16663 243/300 Read block 1794 len 16384
16664 244/300 Write block 1851 len 8192 data: 4 4
16665 245/300 Read block 4548 len 40960
16666 246/300 Write block 1096 len 24576 data: 5 6 5 4 5 5
16667 247/300 Write block 1025 len 8192 data: 4 4
16668 248/300 Write block 800 len 24576 data: 4 4 4 4 4 4
16669 249/300 Flush
16670 250/300 Write block 3205 len 36864 data: 6 4 4 3 4 4 6 4 4
16671 251/300 Write block 3866 len 24576 data: 5 6 6 5 4 4
16672 252/300 Read block 968 len 12288
16673 253/300 Write block 2546 len 12288 data: 5 5 5
16674 254/300 Write block 932 len 16384 data: 2 2 3 3
16675 255/300 Read block 47 len 28672
16676 256/300 Flush
16677 257/300 Write block 4794 len 4096 data: 3
16678 258/300 Write block 1446 len 32768 data: 7 6 4 3 3 4 4 3
16679 259/300 Read block 878 len 12288
16680 260/300 Flush
16681 261/300 Read block 3106 len 8192
16682 262/300 Read block 4643 len 28672
16683 263/300 Write block 4401 len 20480 data: 6 5 3 3 3
16684 264/300 Read block 1951 len 24576
16685 265/300 Read block 1130 len 40960
16686 266/300 Write block 1148 len 20480 data: 4 4 5 5 3
16687 267/300 Read block 597 len 4096
16688 268/300 Read block 3039 len 24576
16689 269/300 Read block 590 len 12288
16690 270/300 Read block 3127 len 32768
16691 271/300 Read block 3949 len 16384
16692 272/300 Read block 3651 len 36864
16693 273/300 Write block 2417 len 40960 data: 4 5 5 4 3 4 4 4 4 4
16694 274/300 Read block 2524 len 28672
16695 275/300 Flush
16696 276/300 Flush
16697 277/300 Write block 1962 len 4096 data: 4
16698 278/300 Read block 715 len 24576
16699 279/300 Read block 3409 len 24576
16700 280/300 Read block 1274 len 8192
16701 281/300 Read block 1653 len 36864
16702 282/300 Flush
16703 283/300 Read block 4161 len 24576
16704 284/300 Write block 4271 len 16384 data: 3 4 4 3
16705 285/300 Read block 1077 len 8192
16706 286/300 Write block 197 len 28672 data: 4 4 3 3 3 4 4
16707 287/300 Write block 2241 len 24576 data: 4 5 5 5 5 3
16708 288/300 Write block 4765 len 20480 data: 2 2 2 2 2
16709 289/300 Write block 463 len 16384 data: 4 4 3 2
16710 290/300 Write block 3121 len 8192 data: 5 5
16711 291/300 Read block 699 len 4096
16712 292/300 Write block 4666 len 20480 data: 2 3 4 4 3
16713 293/300 Read block 2932 len 36864
16714 294/300 Write block 3509 len 12288 data: 3 4 4
16715 295/300 Write block 619 len 28672 data: 3 3 3 2 2 2 2
16716 296/300 Read block 4781 len 24576
16717 297/300 Read block 2580 len 16384
16718 298/300 Write block 1834 len 24576 data: 2 2 2 2 2 2
16719 299/300 Write block 4429 len 36864 data: 7 4 4 4 6 4 4 4 4
16720 300/300 Read block 173 len 20480
16721 301/300 Read block 72 len 20480
16722 Main task has work to do, go find it
16723 got dsc Start(0)
16724 start 0
16725 Dec 11 23:44:02.195 INFO request completed, latency_us: 229, response_code: 204, uri: /start/cid/0, method: POST, req_id: 19a94d0c-52c7-46e2-b904-b672ab963bf3, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
16726 Replay: started 0, returned:Ok(())
16727 ----------------------------------------------------------------
16728 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16729 States: Offline Active Active
16730 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16731 5228 Acked Write 1 Sent Done Done false
16732 5229 Acked Read 5 Sent Done Done false
16733 5230 Acked Read 3 Sent Done Done false
16734 5231 Acked Flush 0 Sent Done Done false
16735 5232 Acked Write 1 Sent Done Done false
16736 5233 Acked Read 7 Sent Done Done false
16737 5234 Acked Write 8 Sent Done Done false
16738 5235 Acked Read 9 Sent Done Done false
16739 5236 Acked Flush 0 Sent Done Done false
16740 5237 Acked Read 8 Sent Done Done false
16741 5238 Acked Write 5 Sent Done Done false
16742 5239 Acked Read 10 Sent Done Done false
16743 5240 Acked Write 8 Sent Done Done false
16744 5241 Acked Read 4 Sent Done Done false
16745 5242 Acked Write 3 Sent Done Done false
16746 5243 Acked Read 3 Sent Done Done false
16747 5244 Acked Read 10 Sent Done Done false
16748 5245 Acked Read 7 Sent Done Done false
16749 5246 Acked Write 1 Sent Done Done false
16750 5247 Acked Read 9 Sent Done Done false
16751 5248 Acked Write 7 Sent Done Done false
16752 5249 Acked Read 10 Sent Done Done false
16753 5250 Acked Read 5 Sent Done Done false
16754 5251 Acked Read 2 Sent Done Done false
16755 5252 Acked Read 4 Sent Done Done false
16756 5253 Acked Read 2 Sent Done Done false
16757 5254 Acked Write 1 Sent Done Done false
16758 5255 Acked Write 2 Sent Done Done false
16759 5256 Acked Read 1 Sent Done Done false
16760 5257 Acked Read 1 Sent Done Done false
16761 5258 Acked Read 4 Sent Done Done false
16762 5259 Acked Write 4 Sent Done Done false
16763 5260 Acked Write 1 Sent Done Done false
16764 5261 Acked Write 1 Sent Done Done false
16765 5262 Acked Read 4 Sent Done Done false
16766 5263 Acked Write 8 Sent Done Done false
16767 5264 Acked Read 3 Sent Done Done false
16768 5265 Acked Write 7 Sent Done Done false
16769 5266 Acked Read 10 Sent Done Done false
16770 5267 Acked Write 3 Sent Done Done false
16771 5268 Acked Read 4 Sent Done Done false
16772 5269 Acked Read 4 Sent Done Done false
16773 5270 Acked Read 4 Sent Done Done false
16774 5271 Acked Read 7 Sent Done Done false
16775 5272 Acked Read 2 Sent Done Done false
16776 5273 Acked Flush 0 Sent Done Done false
16777 5274 Acked Read 6 Sent Done Done false
16778 5275 Acked Read 3 Sent Done Done false
16779 5276 Acked Read 7 Sent Done Done false
16780 5277 Acked Read 7 Sent Done Done false
16781 5278 Acked Read 6 Sent Done Done false
16782 5279 Acked Read 1 Sent Done Done false
16783 5280 Acked Read 6 Sent Done Done false
16784 5281 Acked Write 6 Sent Done Done false
16785 5282 Acked Read 8 Sent Done Done false
16786 5283 Acked Write 5 Sent Done Done false
16787 5284 Acked Flush 0 Sent Done Done false
16788 5285 Acked Write 7 Sent Done Done false
16789 5286 Acked Flush 0 Sent Done Done false
16790 5287 Acked Write 2 Sent Done Done false
16791 5288 Acked Read 5 Sent Done Done false
16792 5289 Acked Write 1 Sent Done Done false
16793 5290 Acked Write 7 Sent Done Done false
16794 5291 Acked Read 7 Sent Done Done false
16795 5292 Acked Write 9 Sent Done Done false
16796 5293 Acked Write 8 Sent Done Done false
16797 5294 Acked Read 8 Sent Done Done false
16798 5295 Acked Flush 0 Sent Done Done false
16799 5296 Acked Write 9 Sent Done Done false
16800 5297 Acked Write 1 Sent Done Done false
16801 5298 Acked Read 10 Sent Done Done false
16802 5299 Acked Read 10 Sent Done Done false
16803 5300 Acked Write 1 Sent Done Done false
16804 5301 Acked Flush 0 Sent Done Done false
16805 5302 Acked Read 6 Sent Done Done false
16806 5303 Acked Write 2 Sent Done Done false
16807 5304 Acked Read 8 Sent Done Done false
16808 5305 Acked Read 9 Sent Done Done false
16809 5306 Acked Write 10 Sent Done Done false
16810 5307 Acked Write 1 Sent Done Done false
16811 5308 Acked Read 3 Sent Done Done false
16812 5309 Acked Flush 0 Sent Done Done false
16813 5310 Acked Read 1 Sent Done Done false
16814 5311 Acked Flush 0 Sent Done Done false
16815 5312 Acked Write 9 Sent Done Done false
16816 5313 Acked Flush 0 Sent Done Done false
16817 5314 Acked Write 3 Sent Done Done false
16818 5315 Acked Flush 0 Sent Done Done false
16819 5316 Acked Write 4 Sent Done Done false
16820 5317 Acked Read 5 Sent Done Done false
16821 5318 Acked Read 1 Sent Done Done false
16822 5319 Acked Read 6 Sent Done Done false
16823 5320 Acked Read 9 Sent Done Done false
16824 5321 Acked Read 2 Sent Done Done false
16825 5322 Acked Write 8 Sent Done Done false
16826 5323 Acked Write 8 Sent Done Done false
16827 5324 Acked Flush 0 Sent Done Done false
16828 5325 Acked Write 2 Sent Done Done false
16829 5326 Acked Write 3 Sent Done Done false
16830 5327 Acked Write 2 Sent Done Done false
16831 5328 Acked Write 8 Sent Done Done false
16832 5329 Acked Read 8 Sent Done Done false
16833 5330 Acked Read 10 Sent Done Done false
16834 5331 Acked Read 9 Sent Done Done false
16835 5332 Acked Read 4 Sent Done Done false
16836 5333 Acked Read 10 Sent Done Done false
16837 5334 Acked Read 6 Sent Done Done false
16838 5335 Acked Write 10 Sent Done Done false
16839 5336 Acked Flush 0 Sent Done Done false
16840 5337 Acked Flush 0 Sent Done Done false
16841 5338 Acked Read 5 Sent Done Done false
16842 5339 Acked Read 3 Sent Done Done false
16843 5340 Acked Read 5 Sent Done Done false
16844 5341 Acked Flush 0 Sent Done Done false
16845 5342 Acked Write 1 Sent Done Done false
16846 5343 Acked Write 8 Sent Done Done false
16847 5344 Acked Write 6 Sent Done Done false
16848 5345 Acked Write 4 Sent Done Done false
16849 5346 Acked Read 6 Sent Done Done false
16850 5347 Acked Read 4 Sent Done Done false
16851 5348 Acked Read 3 Sent Done Done false
16852 5349 Acked Write 3 Sent Done Done false
16853 5350 Acked Read 9 Sent Done Done false
16854 5351 Acked Write 5 Sent Done Done false
16855 5352 Acked Read 2 Sent Done Done false
16856 5353 Acked Read 6 Sent Done Done false
16857 5354 Acked Read 2 Sent Done Done false
16858 5355 Acked Write 9 Sent Done Done false
16859 5356 Acked Read 3 Sent Done Done false
16860 5357 Acked Write 6 Sent Done Done false
16861 5358 Acked Read 3 Sent Done Done false
16862 5359 Acked Write 7 Sent Done Done false
16863 5360 Acked Write 5 Sent Done Done false
16864 5361 Acked Write 8 Sent Done Done false
16865 5362 Acked Write 4 Sent Done Done false
16866 5363 Acked Write 3 Sent Done Done false
16867 5364 Acked Flush 0 Sent Done Done false
16868 5365 Acked Read 4 Sent Done Done false
16869 5366 Acked Read 3 Sent Done Done false
16870 5367 Acked Read 7 Sent Done Done false
16871 5368 Acked Read 5 Sent Done Done false
16872 5369 Acked Read 7 Sent Done Done false
16873 5370 Acked Flush 0 Sent Done Done false
16874 5371 Acked Flush 0 Sent Done Done false
16875 5372 Acked Write 9 Sent Done Done false
16876 5373 Acked Write 10 Sent Done Done false
16877 5374 Acked Read 6 Sent Done Done false
16878 5375 Acked Read 10 Sent Done Done false
16879 5376 Acked Write 10 Sent Done Done false
16880 5377 Acked Read 9 Sent Done Done false
16881 5378 Acked Flush 0 Sent Done Done false
16882 5379 Acked Read 8 Sent Done Done false
16883 5380 Acked Write 9 Sent Done Done false
16884 5381 Acked Flush 0 Sent Done Done false
16885 5382 Acked Write 5 Sent Done Done false
16886 5383 Acked Read 10 Sent Done Done false
16887 5384 Acked Read 2 Sent Done Done false
16888 5385 Acked Read 8 Sent Done Done false
16889 5386 Acked Write 9 Sent Done Done false
16890 5387 Acked Read 4 Sent Done Done false
16891 5388 Acked Write 4 Sent Done Done false
16892 5389 Acked Read 4 Sent Done Done false
16893 5390 Acked Read 4 Sent Done Done false
16894 5391 Acked Read 5 Sent Done Done false
16895 5392 Acked Write 4 Sent Done Done false
16896 5393 Acked Read 4 Sent Done Done false
16897 5394 Acked Flush 0 Sent Done Done false
16898 5395 Acked Read 4 Sent Done Done false
16899 5396 Acked Write 9 Sent Done Done false
16900 5397 Acked Read 10 Sent Done Done false
16901 5398 Acked Read 2 Sent Done Done false
16902 5399 Acked Write 3 Sent Done Done false
16903 5400 Acked Write 5 Sent Done Done false
16904 5401 Acked Read 4 Sent Done Done false
16905 5402 Acked Write 7 Sent Done Done false
16906 5403 Acked Flush 0 Sent Done Done false
16907 5404 Acked Read 7 Sent Done Done false
16908 5405 Acked Flush 0 Sent Done Done false
16909 5406 Acked Write 3 Sent Done Done false
16910 5407 Acked Read 2 Sent Done Done false
16911 5408 Acked Write 5 Sent Done Done false
16912 5409 Acked Write 5 Sent Done Done false
16913 5410 Acked Read 2 Sent Done Done false
16914 5411 Acked Write 1 Sent Done Done false
16915 5412 Acked Write 9 Sent Done Done false
16916 5413 Acked Read 1 Sent Done Done false
16917 5414 Acked Read 1 Sent Done Done false
16918 5415 Acked Read 6 Sent Done Done false
16919 5416 Acked Read 1 Sent Done Done false
16920 5417 Acked Write 10 Sent Done Done false
16921 5418 Acked Write 10 Sent Done Done false
16922 5419 Acked Write 9 Sent Done Done false
16923 5420 Acked Read 9 Sent Done Done false
16924 5421 Acked Flush 0 Sent Done Done false
16925 5422 Acked Flush 0 Sent Done Done false
16926 5423 Acked Write 8 Sent Done Done false
16927 5424 Acked Read 2 Sent Done Done false
16928 5425 Acked Read 4 Sent Done Done false
16929 5426 Acked Write 4 Sent Done Done false
16930 5427 Acked Flush 0 Sent Done Done false
16931 5428 Acked Write 9 Sent Done Done false
16932 5429 Acked Read 1 Sent Done Done false
16933 5430 Acked Write 7 Sent Done Done false
16934 5431 Acked Flush 0 Sent Done Done false
16935 5432 Acked Write 1 Sent Done Done false
16936 5433 Acked Read 1 Sent Done Done false
16937 5434 Acked Write 8 Sent Done Done false
16938 5435 Acked Read 3 Sent Done Done false
16939 5436 Acked Read 10 Sent Done Done false
16940 5437 Acked Read 7 Sent Done Done false
16941 5438 Acked Read 5 Sent Done Done false
16942 5439 Acked Read 6 Sent Done Done false
16943 5440 Acked Write 6 Sent Done Done false
16944 5441 Acked Read 5 Sent Done Done false
16945 5442 Acked Read 4 Sent Done Done false
16946 5443 Acked Read 3 Sent Done Done false
16947 5444 Acked Read 3 Sent Done Done false
16948 5445 Acked Flush 0 Sent Done Done false
16949 5446 Acked Read 2 Sent Done Done false
16950 5447 Acked Read 4 Sent Done Done false
16951 5448 Acked Write 6 Sent Done Done false
16952 5449 Acked Read 6 Sent Done Done false
16953 5450 Acked Write 1 Sent Done Done false
16954 5451 Acked Read 4 Sent Done Done false
16955 5452 Acked Read 10 Sent Done Done false
16956 5453 Acked Read 3 Sent Done Done false
16957 5454 Acked Read 4 Sent Done Done false
16958 5455 Acked Flush 0 Sent Done Done false
16959 5456 Acked Write 3 Sent Done Done false
16960 5457 Acked Flush 0 Sent Done Done false
16961 5458 Acked Write 2 Sent Done Done false
16962 5459 Acked Write 3 Sent Done Done false
16963 5460 Acked Write 8 Sent Done Done false
16964 5461 Acked Write 8 Sent Done Done false
16965 5462 Acked Read 3 Sent Done Done false
16966 5463 Acked Write 10 Sent Done Done false
16967 5464 Acked Write 9 Sent Done Done false
16968 5465 Acked Read 1 Sent Done Done false
16969 5466 Acked Read 4 Sent Done Done false
16970 5467 Acked Read 4 Sent Done Done false
16971 5468 Acked Write 2 Sent Done Done false
16972 5469 Acked Read 7 Sent Done Done false
16973 5470 Acked Read 4 Sent Done Done false
16974 5471 Acked Write 2 Sent Done Done false
16975 5472 Acked Read 10 Sent Done Done false
16976 5473 Acked Write 6 Sent Done Done false
16977 5474 Acked Write 2 Sent Done Done false
16978 5475 Acked Write 6 Sent Done Done false
16979 5476 Acked Flush 0 Sent Done Done false
16980 5477 Acked Write 9 Sent Done Done false
16981 5478 Acked Write 6 Sent Done Done false
16982 5479 Acked Read 3 Sent Done Done false
16983 5480 Acked Write 3 Sent Done Done false
16984 5481 Acked Write 4 Sent Done Done false
16985 5482 Acked Read 7 Sent Done Done false
16986 5483 Acked Flush 0 Sent Done Done false
16987 5484 Acked Write 1 Sent Done Done false
16988 5485 Acked Write 8 Sent Done Done false
16989 5486 Acked Read 3 Sent Done Done false
16990 5487 Acked Flush 0 Sent Done Done false
16991 5488 Acked Read 2 Sent Done Done false
16992 5489 Acked Read 7 Sent Done Done false
16993 5490 Acked Write 5 Sent Done Done false
16994 5491 Acked Read 6 Sent Done Done false
16995 5492 Acked Read 10 Sent Done Done false
16996 5493 Acked Write 5 Sent Done Done false
16997 5494 Acked Read 1 Sent Done Done false
16998 5495 Acked Read 6 Sent Done Done false
16999 5496 Acked Read 3 Sent Done Done false
17000 5497 Acked Read 8 Sent Done Done false
17001 5498 Acked Read 4 Sent Done Done false
17002 5499 Acked Read 9 Sent Done Done false
17003 5500 Acked Write 10 Sent Done Done false
17004 5501 Acked Read 7 Sent Done Done false
17005 5502 Acked Flush 0 Sent Done Done false
17006 5503 Acked Flush 0 Sent Done Done false
17007 5504 Acked Write 1 Sent Done Done false
17008 5505 Acked Read 6 Sent Done Done false
17009 5506 Acked Read 6 Sent Done Done false
17010 5507 Acked Read 2 Sent Done Done false
17011 5508 Acked Read 9 Sent Done Done false
17012 5509 Acked Flush 0 Sent Done Done false
17013 5510 Acked Read 6 Sent Done Done false
17014 5511 Acked Write 4 Sent Done Done false
17015 5512 Acked Read 2 Sent Done Done false
17016 5513 Acked Write 7 Sent Done Done false
17017 5514 Acked Write 6 Sent Done Done false
17018 5515 Acked Write 5 Sent Done Done false
17019 5516 Acked Write 4 Sent Done Done false
17020 5517 Acked Write 2 Sent Done Done false
17021 5518 Acked Read 1 Sent Done Done false
17022 5519 Acked Write 5 Sent Done Done false
17023 5520 Acked Read 9 Sent Done Done false
17024 5521 Acked Write 3 Sent Done Done false
17025 5522 Acked Write 7 Sent Done Done false
17026 5523 Acked Read 6 Sent Done Done false
17027 5524 Acked Read 4 Sent Done Done false
17028 5525 Acked Write 6 Sent Done Done false
17029 5526 Acked Write 9 Sent Done Done false
17030 5527 Acked Read 5 Sent Done Done false
17031 5528 Acked Read 5 Sent Done Done false
17032 STATES DS:0 DS:1 DS:2 TOTAL
17033 Sent 301 0 0 301
17034 Done 0 301 301 602
17035 Skipped 0 0 0 0
17036 Error 0 0 0 0
17037 Last Flush: 5227 5509 5509
17038 Downstairs last five completed: 5227 5226 5225 5224 5223
17039 Upstairs last five completed: 5528 5527 5526 5525 5524
17040 CLIENT: Up:0 ds:301 act:2
17041 [8810] got start action
17042 [8810] I am going to restart
17043 Starting downstairs at port 8810
17044 Make output file at "/tmp/dsc/downstairs-8810.txt"
17045 [8810][0] reports Starting
17046 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1215)
17047 [8810][0] reports Running
17048 ----------------------------------------------------------------
17049 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17050 States: Offline Active Active
17051 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17052 5228 Acked Write 1 Sent Done Done false
17053 5229 Acked Read 5 Sent Done Done false
17054 5230 Acked Read 3 Sent Done Done false
17055 5231 Acked Flush 0 Sent Done Done false
17056 5232 Acked Write 1 Sent Done Done false
17057 5233 Acked Read 7 Sent Done Done false
17058 5234 Acked Write 8 Sent Done Done false
17059 5235 Acked Read 9 Sent Done Done false
17060 5236 Acked Flush 0 Sent Done Done false
17061 5237 Acked Read 8 Sent Done Done false
17062 5238 Acked Write 5 Sent Done Done false
17063 5239 Acked Read 10 Sent Done Done false
17064 5240 Acked Write 8 Sent Done Done false
17065 5241 Acked Read 4 Sent Done Done false
17066 5242 Acked Write 3 Sent Done Done false
17067 5243 Acked Read 3 Sent Done Done false
17068 5244 Acked Read 10 Sent Done Done false
17069 5245 Acked Read 7 Sent Done Done false
17070 5246 Acked Write 1 Sent Done Done false
17071 5247 Acked Read 9 Sent Done Done false
17072 5248 Acked Write 7 Sent Done Done false
17073 5249 Acked Read 10 Sent Done Done false
17074 5250 Acked Read 5 Sent Done Done false
17075 5251 Acked Read 2 Sent Done Done false
17076 5252 Acked Read 4 Sent Done Done false
17077 5253 Acked Read 2 Sent Done Done false
17078 5254 Acked Write 1 Sent Done Done false
17079 5255 Acked Write 2 Sent Done Done false
17080 5256 Acked Read 1 Sent Done Done false
17081 5257 Acked Read 1 Sent Done Done false
17082 5258 Acked Read 4 Sent Done Done false
17083 5259 Acked Write 4 Sent Done Done false
17084 5260 Acked Write 1 Sent Done Done false
17085 5261 Acked Write 1 Sent Done Done false
17086 5262 Acked Read 4 Sent Done Done false
17087 5263 Acked Write 8 Sent Done Done false
17088 5264 Acked Read 3 Sent Done Done false
17089 5265 Acked Write 7 Sent Done Done false
17090 5266 Acked Read 10 Sent Done Done false
17091 5267 Acked Write 3 Sent Done Done false
17092 5268 Acked Read 4 Sent Done Done false
17093 5269 Acked Read 4 Sent Done Done false
17094 5270 Acked Read 4 Sent Done Done false
17095 5271 Acked Read 7 Sent Done Done false
17096 5272 Acked Read 2 Sent Done Done false
17097 5273 Acked Flush 0 Sent Done Done false
17098 5274 Acked Read 6 Sent Done Done false
17099 5275 Acked Read 3 Sent Done Done false
17100 5276 Acked Read 7 Sent Done Done false
17101 5277 Acked Read 7 Sent Done Done false
17102 5278 Acked Read 6 Sent Done Done false
17103 5279 Acked Read 1 Sent Done Done false
17104 5280 Acked Read 6 Sent Done Done false
17105 5281 Acked Write 6 Sent Done Done false
17106 5282 Acked Read 8 Sent Done Done false
17107 5283 Acked Write 5 Sent Done Done false
17108 5284 Acked Flush 0 Sent Done Done false
17109 5285 Acked Write 7 Sent Done Done false
17110 5286 Acked Flush 0 Sent Done Done false
17111 5287 Acked Write 2 Sent Done Done false
17112 5288 Acked Read 5 Sent Done Done false
17113 5289 Acked Write 1 Sent Done Done false
17114 5290 Acked Write 7 Sent Done Done false
17115 5291 Acked Read 7 Sent Done Done false
17116 5292 Acked Write 9 Sent Done Done false
17117 5293 Acked Write 8 Sent Done Done false
17118 5294 Acked Read 8 Sent Done Done false
17119 5295 Acked Flush 0 Sent Done Done false
17120 5296 Acked Write 9 Sent Done Done false
17121 5297 Acked Write 1 Sent Done Done false
17122 5298 Acked Read 10 Sent Done Done false
17123 5299 Acked Read 10 Sent Done Done false
17124 5300 Acked Write 1 Sent Done Done false
17125 5301 Acked Flush 0 Sent Done Done false
17126 5302 Acked Read 6 Sent Done Done false
17127 5303 Acked Write 2 Sent Done Done false
17128 5304 Acked Read 8 Sent Done Done false
17129 5305 Acked Read 9 Sent Done Done false
17130 5306 Acked Write 10 Sent Done Done false
17131 5307 Acked Write 1 Sent Done Done false
17132 5308 Acked Read 3 Sent Done Done false
17133 5309 Acked Flush 0 Sent Done Done false
17134 5310 Acked Read 1 Sent Done Done false
17135 5311 Acked Flush 0 Sent Done Done false
17136 5312 Acked Write 9 Sent Done Done false
17137 5313 Acked Flush 0 Sent Done Done false
17138 5314 Acked Write 3 Sent Done Done false
17139 5315 Acked Flush 0 Sent Done Done false
17140 5316 Acked Write 4 Sent Done Done false
17141 5317 Acked Read 5 Sent Done Done false
17142 5318 Acked Read 1 Sent Done Done false
17143 5319 Acked Read 6 Sent Done Done false
17144 5320 Acked Read 9 Sent Done Done false
17145 5321 Acked Read 2 Sent Done Done false
17146 5322 Acked Write 8 Sent Done Done false
17147 5323 Acked Write 8 Sent Done Done false
17148 5324 Acked Flush 0 Sent Done Done false
17149 5325 Acked Write 2 Sent Done Done false
17150 5326 Acked Write 3 Sent Done Done false
17151 5327 Acked Write 2 Sent Done Done false
17152 5328 Acked Write 8 Sent Done Done false
17153 5329 Acked Read 8 Sent Done Done false
17154 5330 Acked Read 10 Sent Done Done false
17155 5331 Acked Read 9 Sent Done Done false
17156 5332 Acked Read 4 Sent Done Done false
17157 5333 Acked Read 10 Sent Done Done false
17158 5334 Acked Read 6 Sent Done Done false
17159 5335 Acked Write 10 Sent Done Done false
17160 5336 Acked Flush 0 Sent Done Done false
17161 5337 Acked Flush 0 Sent Done Done false
17162 5338 Acked Read 5 Sent Done Done false
17163 5339 Acked Read 3 Sent Done Done false
17164 5340 Acked Read 5 Sent Done Done false
17165 5341 Acked Flush 0 Sent Done Done false
17166 5342 Acked Write 1 Sent Done Done false
17167 5343 Acked Write 8 Sent Done Done false
17168 5344 Acked Write 6 Sent Done Done false
17169 5345 Acked Write 4 Sent Done Done false
17170 5346 Acked Read 6 Sent Done Done false
17171 5347 Acked Read 4 Sent Done Done false
17172 5348 Acked Read 3 Sent Done Done false
17173 5349 Acked Write 3 Sent Done Done false
17174 5350 Acked Read 9 Sent Done Done false
17175 5351 Acked Write 5 Sent Done Done false
17176 5352 Acked Read 2 Sent Done Done false
17177 5353 Acked Read 6 Sent Done Done false
17178 5354 Acked Read 2 Sent Done Done false
17179 5355 Acked Write 9 Sent Done Done false
17180 5356 Acked Read 3 Sent Done Done false
17181 5357 Acked Write 6 Sent Done Done false
17182 5358 Acked Read 3 Sent Done Done false
17183 5359 Acked Write 7 Sent Done Done false
17184 5360 Acked Write 5 Sent Done Done false
17185 5361 Acked Write 8 Sent Done Done false
17186 5362 Acked Write 4 Sent Done Done false
17187 5363 Acked Write 3 Sent Done Done false
17188 5364 Acked Flush 0 Sent Done Done false
17189 5365 Acked Read 4 Sent Done Done false
17190 5366 Acked Read 3 Sent Done Done false
17191 5367 Acked Read 7 Sent Done Done false
17192 5368 Acked Read 5 Sent Done Done false
17193 5369 Acked Read 7 Sent Done Done false
17194 5370 Acked Flush 0 Sent Done Done false
17195 5371 Acked Flush 0 Sent Done Done false
17196 5372 Acked Write 9 Sent Done Done false
17197 5373 Acked Write 10 Sent Done Done false
17198 5374 Acked Read 6 Sent Done Done false
17199 5375 Acked Read 10 Sent Done Done false
17200 5376 Acked Write 10 Sent Done Done false
17201 5377 Acked Read 9 Sent Done Done false
17202 5378 Acked Flush 0 Sent Done Done false
17203 5379 Acked Read 8 Sent Done Done false
17204 5380 Acked Write 9 Sent Done Done false
17205 5381 Acked Flush 0 Sent Done Done false
17206 5382 Acked Write 5 Sent Done Done false
17207 5383 Acked Read 10 Sent Done Done false
17208 5384 Acked Read 2 Sent Done Done false
17209 5385 Acked Read 8 Sent Done Done false
17210 5386 Acked Write 9 Sent Done Done false
17211 5387 Acked Read 4 Sent Done Done false
17212 5388 Acked Write 4 Sent Done Done false
17213 5389 Acked Read 4 Sent Done Done false
17214 5390 Acked Read 4 Sent Done Done false
17215 5391 Acked Read 5 Sent Done Done false
17216 5392 Acked Write 4 Sent Done Done false
17217 5393 Acked Read 4 Sent Done Done false
17218 5394 Acked Flush 0 Sent Done Done false
17219 5395 Acked Read 4 Sent Done Done false
17220 5396 Acked Write 9 Sent Done Done false
17221 5397 Acked Read 10 Sent Done Done false
17222 5398 Acked Read 2 Sent Done Done false
17223 5399 Acked Write 3 Sent Done Done false
17224 5400 Acked Write 5 Sent Done Done false
17225 5401 Acked Read 4 Sent Done Done false
17226 5402 Acked Write 7 Sent Done Done false
17227 5403 Acked Flush 0 Sent Done Done false
17228 5404 Acked Read 7 Sent Done Done false
17229 5405 Acked Flush 0 Sent Done Done false
17230 5406 Acked Write 3 Sent Done Done false
17231 5407 Acked Read 2 Sent Done Done false
17232 5408 Acked Write 5 Sent Done Done false
17233 5409 Acked Write 5 Sent Done Done false
17234 5410 Acked Read 2 Sent Done Done false
17235 5411 Acked Write 1 Sent Done Done false
17236 5412 Acked Write 9 Sent Done Done false
17237 5413 Acked Read 1 Sent Done Done false
17238 5414 Acked Read 1 Sent Done Done false
17239 5415 Acked Read 6 Sent Done Done false
17240 5416 Acked Read 1 Sent Done Done false
17241 5417 Acked Write 10 Sent Done Done false
17242 5418 Acked Write 10 Sent Done Done false
17243 5419 Acked Write 9 Sent Done Done false
17244 5420 Acked Read 9 Sent Done Done false
17245 5421 Acked Flush 0 Sent Done Done false
17246 5422 Acked Flush 0 Sent Done Done false
17247 5423 Acked Write 8 Sent Done Done false
17248 5424 Acked Read 2 Sent Done Done false
17249 5425 Acked Read 4 Sent Done Done false
17250 5426 Acked Write 4 Sent Done Done false
17251 5427 Acked Flush 0 Sent Done Done false
17252 5428 Acked Write 9 Sent Done Done false
17253 5429 Acked Read 1 Sent Done Done false
17254 5430 Acked Write 7 Sent Done Done false
17255 5431 Acked Flush 0 Sent Done Done false
17256 5432 Acked Write 1 Sent Done Done false
17257 5433 Acked Read 1 Sent Done Done false
17258 5434 Acked Write 8 Sent Done Done false
17259 5435 Acked Read 3 Sent Done Done false
17260 5436 Acked Read 10 Sent Done Done false
17261 5437 Acked Read 7 Sent Done Done false
17262 5438 Acked Read 5 Sent Done Done false
17263 5439 Acked Read 6 Sent Done Done false
17264 5440 Acked Write 6 Sent Done Done false
17265 5441 Acked Read 5 Sent Done Done false
17266 5442 Acked Read 4 Sent Done Done false
17267 5443 Acked Read 3 Sent Done Done false
17268 5444 Acked Read 3 Sent Done Done false
17269 5445 Acked Flush 0 Sent Done Done false
17270 5446 Acked Read 2 Sent Done Done false
17271 5447 Acked Read 4 Sent Done Done false
17272 5448 Acked Write 6 Sent Done Done false
17273 5449 Acked Read 6 Sent Done Done false
17274 5450 Acked Write 1 Sent Done Done false
17275 5451 Acked Read 4 Sent Done Done false
17276 5452 Acked Read 10 Sent Done Done false
17277 5453 Acked Read 3 Sent Done Done false
17278 5454 Acked Read 4 Sent Done Done false
17279 5455 Acked Flush 0 Sent Done Done false
17280 5456 Acked Write 3 Sent Done Done false
17281 5457 Acked Flush 0 Sent Done Done false
17282 5458 Acked Write 2 Sent Done Done false
17283 5459 Acked Write 3 Sent Done Done false
17284 5460 Acked Write 8 Sent Done Done false
17285 5461 Acked Write 8 Sent Done Done false
17286 5462 Acked Read 3 Sent Done Done false
17287 5463 Acked Write 10 Sent Done Done false
17288 5464 Acked Write 9 Sent Done Done false
17289 5465 Acked Read 1 Sent Done Done false
17290 5466 Acked Read 4 Sent Done Done false
17291 5467 Acked Read 4 Sent Done Done false
17292 5468 Acked Write 2 Sent Done Done false
17293 5469 Acked Read 7 Sent Done Done false
17294 5470 Acked Read 4 Sent Done Done false
17295 5471 Acked Write 2 Sent Done Done false
17296 5472 Acked Read 10 Sent Done Done false
17297 5473 Acked Write 6 Sent Done Done false
17298 5474 Acked Write 2 Sent Done Done false
17299 5475 Acked Write 6 Sent Done Done false
17300 5476 Acked Flush 0 Sent Done Done false
17301 5477 Acked Write 9 Sent Done Done false
17302 5478 Acked Write 6 Sent Done Done false
17303 5479 Acked Read 3 Sent Done Done false
17304 5480 Acked Write 3 Sent Done Done false
17305 5481 Acked Write 4 Sent Done Done false
17306 5482 Acked Read 7 Sent Done Done false
17307 5483 Acked Flush 0 Sent Done Done false
17308 5484 Acked Write 1 Sent Done Done false
17309 5485 Acked Write 8 Sent Done Done false
17310 5486 Acked Read 3 Sent Done Done false
17311 5487 Acked Flush 0 Sent Done Done false
17312 5488 Acked Read 2 Sent Done Done false
17313 5489 Acked Read 7 Sent Done Done false
17314 5490 Acked Write 5 Sent Done Done false
17315 5491 Acked Read 6 Sent Done Done false
17316 5492 Acked Read 10 Sent Done Done false
17317 5493 Acked Write 5 Sent Done Done false
17318 5494 Acked Read 1 Sent Done Done false
17319 5495 Acked Read 6 Sent Done Done false
17320 5496 Acked Read 3 Sent Done Done false
17321 5497 Acked Read 8 Sent Done Done false
17322 5498 Acked Read 4 Sent Done Done false
17323 5499 Acked Read 9 Sent Done Done false
17324 5500 Acked Write 10 Sent Done Done false
17325 5501 Acked Read 7 Sent Done Done false
17326 5502 Acked Flush 0 Sent Done Done false
17327 5503 Acked Flush 0 Sent Done Done false
17328 5504 Acked Write 1 Sent Done Done false
17329 5505 Acked Read 6 Sent Done Done false
17330 5506 Acked Read 6 Sent Done Done false
17331 5507 Acked Read 2 Sent Done Done false
17332 5508 Acked Read 9 Sent Done Done false
17333 5509 Acked Flush 0 Sent Done Done false
17334 5510 Acked Read 6 Sent Done Done false
17335 5511 Acked Write 4 Sent Done Done false
17336 5512 Acked Read 2 Sent Done Done false
17337 5513 Acked Write 7 Sent Done Done false
17338 5514 Acked Write 6 Sent Done Done false
17339 5515 Acked Write 5 Sent Done Done false
17340 5516 Acked Write 4 Sent Done Done false
17341 5517 Acked Write 2 Sent Done Done false
17342 5518 Acked Read 1 Sent Done Done false
17343 5519 Acked Write 5 Sent Done Done false
17344 5520 Acked Read 9 Sent Done Done false
17345 5521 Acked Write 3 Sent Done Done false
17346 5522 Acked Write 7 Sent Done Done false
17347 5523 Acked Read 6 Sent Done Done false
17348 5524 Acked Read 4 Sent Done Done false
17349 5525 Acked Write 6 Sent Done Done false
17350 5526 Acked Write 9 Sent Done Done false
17351 5527 Acked Read 5 Sent Done Done false
17352 5528 Acked Read 5 Sent Done Done false
17353 5529 Acked Flush 0 Sent Done Done false
17354 STATES DS:0 DS:1 DS:2 TOTAL
17355 Sent 302 0 0 302
17356 Done 0 302 302 604
17357 Skipped 0 0 0 0
17358 Error 0 0 0 0
17359 Last Flush: 5227 5529 5529
17360 Downstairs last five completed: 5227 5226 5225 5224 5223
17361 Upstairs last five completed: 5529 5528 5527 5526 5525
17362 CLIENT: Up:0 ds:302 act:2
173632024-12-11T23:44:07.953ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173642024-12-11T23:44:07.953ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:32908) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173652024-12-11T23:44:07.954ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173662024-12-11T23:44:07.954ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173672024-12-11T23:44:07.954ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173682024-12-11T23:44:07.954ZINFOcrucible: send last flush ID to this DS: 5227 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173692024-12-11T23:44:07.955ZINFOcrucible: Replied this last flush ID: 5227 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173702024-12-11T23:44:07.955ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173712024-12-11T23:44:07.955ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173722024-12-11T23:44:07.955ZINFOcrucible: [0] client re-new 302 jobs since flush 5227 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173732024-12-11T23:44:07.955ZINFOcrucible: [0] Replayed 302 jobs since flush: 5227 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
17374 ----------------------------------------------------------------
17375 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17376 Downstairs last five completed: 5529 5528 5527 5526 5525
17377 Upstairs last five completed: 5529 5528 5527 5526 5525
17378 CLIENT: Up:0 ds:0 act:3
17379 Replay: All jobs finished
17380 Main task has work to do, go find it
17381 got dsc Stop(1)
17382 stop 1
17383 [8820] Got stop action so:false kr:false
17384 Dec 11 23:44:10.200 INFO request completed, latency_us: 322, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 780a9230-d18b-433b-9300-a6f2295c6197, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
173852024-12-11T23:44:10.201ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173862024-12-11T23:44:10.201ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173872024-12-11T23:44:10.201ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173882024-12-11T23:44:10.201ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
173892024-12-11T23:44:10.201ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
17390 Dec 11 23:44:10.201 INFO request completed, latency_us: 152, response_code: 200, uri: /state/cid/1, method: GET, req_id: 526ba87f-07de-4507-aac3-049cb66cc966, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
173912024-12-11T23:44:10.201ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
17392 [8820] Exited with: ExitStatus(unix_wait_status(9))
17393 [8820][1] reports Exit
17394 Dec 11 23:44:14.203 INFO request completed, latency_us: 221, response_code: 200, uri: /state/cid/1, method: GET, req_id: 5e7d1ac6-ec3f-40c0-b43c-e378df5351ab, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
17395 001/300 Write block 4122 len 24576 data: 7 4 5 5 4 3
17396 002/300 Read block 2875 len 20480
17397 003/300 Read block 302 len 12288
17398 004/300 Write block 9 len 24576 data: 5 5 4 4 3 2
17399 005/300 Write block 1571 len 4096 data: 5
17400 006/300 Read block 3980 len 28672
17401 007/300 Read block 3943 len 4096
17402 008/300 Read block 4540 len 12288
17403 009/300 Read block 4537 len 20480
17404 010/300 Write block 3206 len 36864 data: 5 5 4 5 5 7 5 5 4
17405 011/300 Read block 3942 len 16384
17406 012/300 Read block 4940 len 40960
17407 013/300 Write block 2719 len 36864 data: 7 7 7 6 7 8 5 4 4
17408 014/300 Flush
17409 015/300 Read block 2472 len 8192
17410 016/300 Write block 2289 len 20480 data: 3 3 4 4 5
17411 017/300 Read block 4389 len 24576
17412 018/300 Read block 3690 len 16384
17413 019/300 Read block 2545 len 8192
17414 020/300 Flush
17415 021/300 Write block 145 len 16384 data: 5 3 5 6
17416 022/300 Write block 1546 len 8192 data: 5 4
17417 023/300 Write block 1470 len 4096 data: 4
17418 024/300 Write block 4357 len 4096 data: 4
17419 025/300 Read block 4693 len 8192
17420 026/300 Read block 4153 len 12288
17421 027/300 Read block 2543 len 24576
17422 028/300 Read block 1757 len 32768
17423 029/300 Read block 2517 len 28672
17424 030/300 Read block 3648 len 28672
17425 031/300 Read block 4692 len 4096
17426 032/300 Flush
17427 033/300 Read block 596 len 4096
17428 034/300 Read block 3178 len 16384
17429 035/300 Write block 4977 len 24576 data: 3 3 4 5 5 6
17430 036/300 Write block 4217 len 12288 data: 4 5 5
17431 037/300 Write block 3525 len 28672 data: 5 5 7 7 5 5 4
17432 038/300 Read block 2214 len 12288
17433 039/300 Write block 1033 len 12288 data: 4 5 5
17434 040/300 Read block 2551 len 20480
17435 041/300 Write block 376 len 28672 data: 2 2 2 2 3 4 4
17436 042/300 Read block 2773 len 36864
17437 043/300 Write block 1969 len 40960 data: 2 2 4 3 3 3 4 4 4 4
17438 044/300 Read block 2359 len 12288
17439 045/300 Write block 4583 len 12288 data: 4 4 5
17440 046/300 Write block 1290 len 16384 data: 6 6 6 6
17441 047/300 Read block 632 len 32768
17442 048/300 Read block 2909 len 20480
17443 049/300 Read block 4008 len 8192
17444 050/300 Write block 141 len 4096 data: 5
17445 051/300 Write block 3620 len 16384 data: 6 6 4 4
17446 052/300 Read block 4748 len 28672
17447 053/300 Read block 3956 len 20480
17448 054/300 Write block 1809 len 36864 data: 4 4 4 3 3 2 3 2 2
17449 055/300 Read block 2611 len 4096
17450 056/300 Read block 3441 len 4096
17451 057/300 Read block 1910 len 32768
17452 058/300 Read block 3232 len 8192
17453 059/300 Read block 3317 len 28672
17454 060/300 Read block 515 len 20480
17455 061/300 Read block 606 len 28672
17456 062/300 Write block 974 len 36864 data: 6 6 4 4 3 4 3 3 3
17457 063/300 Read block 3454 len 16384
17458 064/300 Flush
17459 065/300 Write block 3829 len 16384 data: 8 8 7 6
17460 066/300 Read block 3015 len 32768
17461 067/300 Write block 1267 len 4096 data: 4
17462 068/300 Flush
17463 069/300 Write block 3682 len 28672 data: 4 5 5 5 4 5 6
17464 070/300 Write block 2888 len 12288 data: 5 5 5
17465 071/300 Read block 4290 len 12288
17466 072/300 Read block 2821 len 24576
17467 073/300 Read block 1073 len 12288
17468 074/300 Flush
17469 075/300 Read block 1952 len 40960
17470 076/300 Write block 4115 len 24576 data: 4 3 5 5 5 4
17471 077/300 Read block 3918 len 16384
17472 078/300 Write block 158 len 40960 data: 4 4 3 5 4 4 4 4 3 3
17473 079/300 Read block 3261 len 16384
17474 080/300 Read block 3847 len 32768
17475 081/300 Read block 3188 len 40960
17476 082/300 Read block 4547 len 4096
17477 083/300 Read block 2795 len 36864
17478 084/300 Read block 2620 len 32768
17479 085/300 Write block 688 len 32768 data: 5 5 4 3 5 3 3 4
17480 086/300 Flush
17481 087/300 Read block 3042 len 28672
17482 088/300 Write block 2957 len 20480 data: 3 4 4 4 5
17483 089/300 Write block 373 len 12288 data: 2 2 2
17484 090/300 Write block 4353 len 24576 data: 5 4 5 5 5 4
17485 091/300 Write block 70 len 40960 data: 6 6 6 6 4 4 5 6 6 5
17486 092/300 Write block 3532 len 4096 data: 4
17487 093/300 Read block 1308 len 16384
17488 094/300 Read block 2718 len 36864
17489 095/300 Read block 1317 len 12288
17490 096/300 Read block 4038 len 32768
17491 097/300 Read block 1352 len 28672
17492 098/300 Write block 447 len 40960 data: 4 5 4 5 5 4 4 3 2 2
17493 099/300 Read block 1370 len 12288
17494 100/300 Read block 3480 len 36864
17495 101/300 Write block 509 len 40960 data: 4 3 4 4 3 3 4 5 6 5
17496 102/300 Write block 4177 len 20480 data: 5 5 5 5 6
17497 103/300 Write block 3450 len 4096 data: 4
17498 104/300 Flush
17499 105/300 Flush
17500 106/300 Flush
17501 107/300 Write block 4995 len 16384 data: 4 4 5 4
17502 108/300 Write block 3514 len 40960 data: 4 4 2 2 3 2 2 3 4 4
17503 109/300 Flush
17504 110/300 Write block 811 len 28672 data: 5 4 5 5 4 4 4
17505 111/300 Write block 2021 len 36864 data: 5 4 4 4 4 4 4 4 4
17506 112/300 Read block 475 len 12288
17507 113/300 Read block 4771 len 32768
17508 114/300 Read block 3647 len 36864
17509 115/300 Read block 191 len 8192
17510 116/300 Read block 4011 len 28672
17511 117/300 Flush
17512 118/300 Write block 4881 len 40960 data: 3 3 3 3 2 3 4 4 4 4
17513 119/300 Flush
17514 120/300 Flush
17515 121/300 Read block 2578 len 32768
17516 122/300 Read block 472 len 20480
17517 123/300 Write block 4030 len 24576 data: 2 2 2 2 3 3
17518 124/300 Read block 4582 len 36864
17519 125/300 Write block 2706 len 20480 data: 6 4 4 4 5
17520 126/300 Read block 4462 len 16384
17521 127/300 Flush
17522 128/300 Write block 614 len 8192 data: 5 3
17523 129/300 Read block 4215 len 8192
17524 130/300 Write block 4972 len 28672 data: 5 5 4 3 3 4 4
17525 131/300 Read block 3781 len 20480
17526 132/300 Read block 2403 len 8192
17527 133/300 Read block 1142 len 16384
17528 134/300 Read block 3962 len 24576
17529 135/300 Read block 1322 len 4096
17530 136/300 Read block 3339 len 32768
17531 137/300 Read block 2264 len 20480
17532 138/300 Read block 4286 len 8192
17533 139/300 Read block 2974 len 8192
17534 140/300 Read block 2698 len 4096
17535 141/300 Read block 2853 len 20480
17536 142/300 Read block 2751 len 20480
17537 143/300 Read block 1940 len 40960
17538 144/300 Read block 4704 len 16384
17539 145/300 Write block 4424 len 40960 data: 5 5 6 6 7 8 5 5 5 7
17540 146/300 Read block 4774 len 36864
17541 147/300 Read block 458 len 32768
17542 148/300 Write block 1697 len 8192 data: 6 7
17543 149/300 Read block 719 len 36864
17544 150/300 Write block 3580 len 12288 data: 5 7 5
17545 151/300 Read block 3947 len 32768
17546 152/300 Write block 2163 len 40960 data: 2 2 2 2 2 3 3 3 3 4
17547 153/300 Flush
17548 154/300 Read block 3645 len 4096
17549 155/300 Read block 4372 len 40960
17550 156/300 Read block 1315 len 40960
17551 157/300 Read block 1734 len 16384
17552 158/300 Write block 667 len 40960 data: 5 5 4 3 2 2 2 2 2 2
17553 159/300 Write block 243 len 4096 data: 3
17554 160/300 Read block 3383 len 16384
17555 161/300 Read block 2603 len 4096
17556 162/300 Write block 1595 len 32768 data: 3 3 3 2 2 2 2 3
17557 163/300 Flush
17558 164/300 Flush
17559 165/300 Write block 2283 len 8192 data: 4 3
17560 166/300 Write block 1203 len 32768 data: 4 5 5 4 4 4 4 3
17561 167/300 Write block 2488 len 28672 data: 3 3 2 2 2 2 2
17562 168/300 Read block 1785 len 16384
17563 169/300 Write block 2727 len 40960 data: 5 4 3 3 4 4 6 6 6 6
17564 170/300 Write block 1307 len 12288 data: 5 5 5
17565 171/300 Read block 4859 len 28672
17566 172/300 Read block 2202 len 24576
17567 173/300 Read block 4329 len 8192
17568 174/300 Read block 4742 len 12288
17569 175/300 Read block 1771 len 28672
17570 176/300 Read block 3374 len 28672
17571 177/300 Write block 4629 len 4096 data: 3
17572 178/300 Write block 4313 len 8192 data: 3 4
17573 179/300 Read block 1280 len 40960
17574 180/300 Read block 2432 len 20480
17575 181/300 Read block 473 len 12288
17576 182/300 Read block 1793 len 12288
17577 183/300 Flush
17578 184/300 Read block 2679 len 28672
17579 185/300 Write block 1247 len 32768 data: 5 6 3 2 3 3 2 2
17580 186/300 Read block 1922 len 4096
17581 187/300 Write block 2010 len 8192 data: 3 2
17582 188/300 Read block 4733 len 40960
17583 189/300 Write block 1053 len 12288 data: 4 4 6
17584 190/300 Read block 1098 len 8192
17585 191/300 Flush
17586 192/300 Write block 1464 len 32768 data: 4 3 3 3 4 4 5 3
17587 193/300 Read block 2852 len 8192
17588 194/300 Read block 2473 len 8192
17589 195/300 Read block 1211 len 24576
17590 196/300 Read block 3672 len 12288
17591 197/300 Write block 3625 len 28672 data: 2 2 3 4 5 5 5
17592 198/300 Write block 2945 len 36864 data: 4 4 4 3 3 3 3 3 3
17593 199/300 Write block 3333 len 16384 data: 3 2 2 2
17594 200/300 Write block 1055 len 16384 data: 7 6 5 2
17595 201/300 Flush
17596 202/300 Write block 4061 len 20480 data: 6 6 5 4 4
17597 203/300 Write block 1491 len 36864 data: 5 6 7 5 5 5 5 6 4
17598 204/300 Write block 3863 len 40960 data: 9 9 8 6 7 7 6 5 5 4
17599 205/300 Write block 3648 len 4096 data: 6
17600 206/300 Read block 1870 len 20480
17601 207/300 Read block 2116 len 36864
17602 208/300 Write block 264 len 12288 data: 4 5 5
17603 209/300 Write block 600 len 8192 data: 2 2
17604 210/300 Write block 4110 len 40960 data: 4 4 4 4 4 5 4 6 6 6
17605 211/300 Read block 553 len 16384
17606 212/300 Write block 108 len 40960 data: 3 3 3 3 3 3 4 5 5 5
17607 213/300 Write block 4235 len 24576 data: 5 5 4 4 4 5
17608 214/300 Read block 2772 len 32768
17609 215/300 Write block 2822 len 40960 data: 3 5 5 3 3 4 5 5 6 6
17610 216/300 Write block 3222 len 28672 data: 2 3 4 4 5 5 5
17611 217/300 Read block 2796 len 28672
17612 218/300 Read block 2183 len 24576
17613 219/300 Write block 2126 len 16384 data: 7 7 7 6
17614 220/300 Read block 2215 len 40960
17615 221/300 Read block 4937 len 28672
17616 222/300 Read block 1652 len 4096
17617 223/300 Read block 1369 len 36864
17618 224/300 Write block 1795 len 40960 data: 3 3 3 2 2 2 2 3 3 3
17619 225/300 Read block 294 len 20480
17620 226/300 Write block 563 len 20480 data: 2 2 2 2 2
17621 227/300 Flush
17622 228/300 Flush
17623 229/300 Write block 1623 len 16384 data: 4 5 6 7
17624 230/300 Flush
17625 231/300 Read block 121 len 24576
17626 232/300 Write block 357 len 4096 data: 6
17627 233/300 Read block 1993 len 36864
17628 234/300 Read block 2664 len 4096
17629 235/300 Read block 888 len 28672
17630 236/300 Write block 3180 len 4096 data: 5
17631 237/300 Read block 2846 len 12288
17632 238/300 Read block 4896 len 4096
17633 239/300 Read block 1978 len 8192
17634 240/300 Write block 3486 len 40960 data: 3 4 4 4 3 2 2 3 3 3
17635 241/300 Write block 1621 len 24576 data: 5 5 5 6 7 8
17636 242/300 Write block 3171 len 24576 data: 4 5 4 5 4 4
17637 243/300 Read block 3011 len 28672
17638 244/300 Flush
17639 245/300 Flush
17640 246/300 Read block 2201 len 8192
17641 247/300 Write block 4937 len 28672 data: 5 4 4 4 4 4 4
17642 248/300 Write block 252 len 4096 data: 4
17643 249/300 Read block 4743 len 28672
17644 250/300 Read block 1412 len 24576
17645 251/300 Write block 4922 len 4096 data: 9
17646 252/300 Flush
17647 253/300 Write block 2292 len 20480 data: 5 6 4 4 4
17648 254/300 Write block 2000 len 4096 data: 3
17649 255/300 Read block 3925 len 12288
17650 256/300 Write block 900 len 16384 data: 3 3 2 2
17651 257/300 Write block 965 len 20480 data: 3 2 2 5 5
17652 258/300 Write block 3154 len 16384 data: 3 6 6 5
17653 259/300 Read block 3539 len 16384
17654 260/300 Write block 2448 len 24576 data: 3 3 2 2 2 2
17655 261/300 Write block 125 len 20480 data: 6 6 4 4 5
17656 262/300 Write block 3296 len 12288 data: 3 3 4
17657 263/300 Write block 2602 len 36864 data: 4 4 4 5 3 3 4 5 7
17658 264/300 Read block 868 len 32768
17659 265/300 Read block 4074 len 4096
17660 266/300 Write block 4719 len 24576 data: 5 4 5 5 6 5
17661 267/300 Read block 4286 len 40960
17662 268/300 Write block 4673 len 4096 data: 3
17663 269/300 Write block 1939 len 24576 data: 3 3 4 4 3 4
17664 270/300 Read block 4459 len 4096
17665 271/300 Write block 1295 len 8192 data: 6 6
17666 272/300 Flush
17667 273/300 Write block 304 len 24576 data: 5 4 3 3 3 3
17668 274/300 Write block 2273 len 32768 data: 5 3 3 2 2 2 2 4
17669 275/300 Read block 2598 len 40960
17670 276/300 Read block 1238 len 16384
17671 277/300 Read block 1963 len 20480
17672 278/300 Read block 3023 len 24576
17673 279/300 Read block 467 len 28672
17674 280/300 Read block 4833 len 20480
17675 281/300 Flush
17676 282/300 Read block 1559 len 20480
17677 283/300 Read block 2974 len 16384
17678 284/300 Read block 3574 len 4096
17679 285/300 Read block 1011 len 16384
17680 286/300 Write block 4359 len 36864 data: 4 2 2 3 3 3 3 3 2
17681 287/300 Flush
17682 288/300 Read block 4187 len 36864
17683 289/300 Read block 4352 len 4096
17684 290/300 Flush
17685 291/300 Write block 106 len 12288 data: 4 3 4
17686 292/300 Read block 4404 len 16384
17687 293/300 Write block 592 len 16384 data: 4 3 3 3
17688 294/300 Read block 4667 len 12288
17689 295/300 Write block 2048 len 24576 data: 4 4 3 3 4 5
17690 296/300 Flush
17691 297/300 Flush
17692 298/300 Flush
17693 299/300 Write block 2764 len 28672 data: 3 4 4 4 4 5 3
17694 300/300 Read block 4521 len 16384
17695 301/300 Write block 4957 len 12288 data: 6 7 6
17696 Main task has work to do, go find it
17697 got dsc Start(1)
17698 start 1
17699 Dec 11 23:44:14.439 INFO request completed, latency_us: 249, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7af25d73-82e4-4b2e-9e05-1b8cebb55163, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
17700 Replay: started 1, returned:Ok(())
17701 ----------------------------------------------------------------
17702 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17703 States: Active Offline Active
17704 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17705 5530 Acked Write 6 Done Sent Done false
17706 5531 Acked Read 5 Done Sent Done false
17707 5532 Acked Read 3 Done Sent Done false
17708 5533 Acked Write 6 Done Sent Done false
17709 5534 Acked Write 1 Done Sent Done false
17710 5535 Acked Read 7 Done Sent Done false
17711 5536 Acked Read 1 Done Sent Done false
17712 5537 Acked Read 3 Done Sent Done false
17713 5538 Acked Read 5 Done Sent Done false
17714 5539 Acked Write 9 Done Sent Done false
17715 5540 Acked Read 4 Done Sent Done false
17716 5541 Acked Read 10 Done Sent Done false
17717 5542 Acked Write 9 Done Sent Done false
17718 5543 Acked Flush 0 Done Sent Done false
17719 5544 Acked Read 2 Done Sent Done false
17720 5545 Acked Write 5 Done Sent Done false
17721 5546 Acked Read 6 Done Sent Done false
17722 5547 Acked Read 4 Done Sent Done false
17723 5548 Acked Read 2 Done Sent Done false
17724 5549 Acked Flush 0 Done Sent Done false
17725 5550 Acked Write 4 Done Sent Done false
17726 5551 Acked Write 2 Done Sent Done false
17727 5552 Acked Write 1 Done Sent Done false
17728 5553 Acked Write 1 Done Sent Done false
17729 5554 Acked Read 2 Done Sent Done false
17730 5555 Acked Read 3 Done Sent Done false
17731 5556 Acked Read 6 Done Sent Done false
17732 5557 Acked Read 8 Done Sent Done false
17733 5558 Acked Read 7 Done Sent Done false
17734 5559 Acked Read 7 Done Sent Done false
17735 5560 Acked Read 1 Done Sent Done false
17736 5561 Acked Flush 0 Done Sent Done false
17737 5562 Acked Read 1 Done Sent Done false
17738 5563 Acked Read 4 Done Sent Done false
17739 5564 Acked Write 6 Done Sent Done false
17740 5565 Acked Write 3 Done Sent Done false
17741 5566 Acked Write 7 Done Sent Done false
17742 5567 Acked Read 3 Done Sent Done false
17743 5568 Acked Write 3 Done Sent Done false
17744 5569 Acked Read 5 Done Sent Done false
17745 5570 Acked Write 7 Done Sent Done false
17746 5571 Acked Read 9 Done Sent Done false
17747 5572 Acked Write 10 Done Sent Done false
17748 5573 Acked Read 3 Done Sent Done false
17749 5574 Acked Write 3 Done Sent Done false
17750 5575 Acked Write 4 Done Sent Done false
17751 5576 Acked Read 8 Done Sent Done false
17752 5577 Acked Read 5 Done Sent Done false
17753 5578 Acked Read 2 Done Sent Done false
17754 5579 Acked Write 1 Done Sent Done false
17755 5580 Acked Write 4 Done Sent Done false
17756 5581 Acked Read 7 Done Sent Done false
17757 5582 Acked Read 5 Done Sent Done false
17758 5583 Acked Write 9 Done Sent Done false
17759 5584 Acked Read 1 Done Sent Done false
17760 5585 Acked Read 1 Done Sent Done false
17761 5586 Acked Read 8 Done Sent Done false
17762 5587 Acked Read 2 Done Sent Done false
17763 5588 Acked Read 7 Done Sent Done false
17764 5589 Acked Read 5 Done Sent Done false
17765 5590 Acked Read 7 Done Sent Done false
17766 5591 Acked Write 9 Done Sent Done false
17767 5592 Acked Read 4 Done Sent Done false
17768 5593 Acked Flush 0 Done Sent Done false
17769 5594 Acked Write 4 Done Sent Done false
17770 5595 Acked Read 8 Done Sent Done false
17771 5596 Acked Write 1 Done Sent Done false
17772 5597 Acked Flush 0 Done Sent Done false
17773 5598 Acked Write 7 Done Sent Done false
17774 5599 Acked Write 3 Done Sent Done false
17775 5600 Acked Read 3 Done Sent Done false
17776 5601 Acked Read 6 Done Sent Done false
17777 5602 Acked Read 3 Done Sent Done false
17778 5603 Acked Flush 0 Done Sent Done false
17779 5604 Acked Read 10 Done Sent Done false
17780 5605 Acked Write 6 Done Sent Done false
17781 5606 Acked Read 4 Done Sent Done false
17782 5607 Acked Write 10 Done Sent Done false
17783 5608 Acked Read 4 Done Sent Done false
17784 5609 Acked Read 8 Done Sent Done false
17785 5610 Acked Read 10 Done Sent Done false
17786 5611 Acked Read 1 Done Sent Done false
17787 5612 Acked Read 9 Done Sent Done false
17788 5613 Acked Read 8 Done Sent Done false
17789 5614 Acked Write 8 Done Sent Done false
17790 5615 Acked Flush 0 Done Sent Done false
17791 5616 Acked Read 7 Done Sent Done false
17792 5617 Acked Write 5 Done Sent Done false
17793 5618 Acked Write 3 Done Sent Done false
17794 5619 Acked Write 6 Done Sent Done false
17795 5620 Acked Write 10 Done Sent Done false
17796 5621 Acked Write 1 Done Sent Done false
17797 5622 Acked Read 4 Done Sent Done false
17798 5623 Acked Read 9 Done Sent Done false
17799 5624 Acked Read 3 Done Sent Done false
17800 5625 Acked Read 8 Done Sent Done false
17801 5626 Acked Read 7 Done Sent Done false
17802 5627 Acked Write 10 Done Sent Done false
17803 5628 Acked Read 3 Done Sent Done false
17804 5629 Acked Read 9 Done Sent Done false
17805 5630 Acked Write 10 Done Sent Done false
17806 5631 Acked Write 5 Done Sent Done false
17807 5632 Acked Write 1 Done Sent Done false
17808 5633 Acked Flush 0 Done Sent Done false
17809 5634 Acked Flush 0 Done Sent Done false
17810 5635 Acked Flush 0 Done Sent Done false
17811 5636 Acked Write 4 Done Sent Done false
17812 5637 Acked Write 10 Done Sent Done false
17813 5638 Acked Flush 0 Done Sent Done false
17814 5639 Acked Write 7 Done Sent Done false
17815 5640 Acked Write 9 Done Sent Done false
17816 5641 Acked Read 3 Done Sent Done false
17817 5642 Acked Read 8 Done Sent Done false
17818 5643 Acked Read 9 Done Sent Done false
17819 5644 Acked Read 2 Done Sent Done false
17820 5645 Acked Read 7 Done Sent Done false
17821 5646 Acked Flush 0 Done Sent Done false
17822 5647 Acked Write 10 Done Sent Done false
17823 5648 Acked Flush 0 Done Sent Done false
17824 5649 Acked Flush 0 Done Sent Done false
17825 5650 Acked Read 8 Done Sent Done false
17826 5651 Acked Read 5 Done Sent Done false
17827 5652 Acked Write 6 Done Sent Done false
17828 5653 Acked Read 9 Done Sent Done false
17829 5654 Acked Write 5 Done Sent Done false
17830 5655 Acked Read 4 Done Sent Done false
17831 5656 Acked Flush 0 Done Sent Done false
17832 5657 Acked Write 2 Done Sent Done false
17833 5658 Acked Read 2 Done Sent Done false
17834 5659 Acked Write 7 Done Sent Done false
17835 5660 Acked Read 5 Done Sent Done false
17836 5661 Acked Read 2 Done Sent Done false
17837 5662 Acked Read 4 Done Sent Done false
17838 5663 Acked Read 6 Done Sent Done false
17839 5664 Acked Read 1 Done Sent Done false
17840 5665 Acked Read 8 Done Sent Done false
17841 5666 Acked Read 5 Done Sent Done false
17842 5667 Acked Read 2 Done Sent Done false
17843 5668 Acked Read 2 Done Sent Done false
17844 5669 Acked Read 1 Done Sent Done false
17845 5670 Acked Read 5 Done Sent Done false
17846 5671 Acked Read 5 Done Sent Done false
17847 5672 Acked Read 10 Done Sent Done false
17848 5673 Acked Read 4 Done Sent Done false
17849 5674 Acked Write 10 Done Sent Done false
17850 5675 Acked Read 9 Done Sent Done false
17851 5676 Acked Read 8 Done Sent Done false
17852 5677 Acked Write 2 Done Sent Done false
17853 5678 Acked Read 9 Done Sent Done false
17854 5679 Acked Write 3 Done Sent Done false
17855 5680 Acked Read 8 Done Sent Done false
17856 5681 Acked Write 10 Done Sent Done false
17857 5682 Acked Flush 0 Done Sent Done false
17858 5683 Acked Read 1 Done Sent Done false
17859 5684 Acked Read 10 Done Sent Done false
17860 5685 Acked Read 10 Done Sent Done false
17861 5686 Acked Read 4 Done Sent Done false
17862 5687 Acked Write 10 Done Sent Done false
17863 5688 Acked Write 1 Done Sent Done false
17864 5689 Acked Read 4 Done Sent Done false
17865 5690 Acked Read 1 Done Sent Done false
17866 5691 Acked Write 8 Done Sent Done false
17867 5692 Acked Flush 0 Done Sent Done false
17868 5693 Acked Flush 0 Done Sent Done false
17869 5694 Acked Write 2 Done Sent Done false
17870 5695 Acked Write 8 Done Sent Done false
17871 5696 Acked Write 7 Done Sent Done false
17872 5697 Acked Read 4 Done Sent Done false
17873 5698 Acked Write 10 Done Sent Done false
17874 5699 Acked Write 3 Done Sent Done false
17875 5700 Acked Read 7 Done Sent Done false
17876 5701 Acked Read 6 Done Sent Done false
17877 5702 Acked Read 2 Done Sent Done false
17878 5703 Acked Read 3 Done Sent Done false
17879 5704 Acked Read 7 Done Sent Done false
17880 5705 Acked Read 7 Done Sent Done false
17881 5706 Acked Write 1 Done Sent Done false
17882 5707 Acked Write 2 Done Sent Done false
17883 5708 Acked Read 10 Done Sent Done false
17884 5709 Acked Read 5 Done Sent Done false
17885 5710 Acked Read 3 Done Sent Done false
17886 5711 Acked Read 3 Done Sent Done false
17887 5712 Acked Flush 0 Done Sent Done false
17888 5713 Acked Read 7 Done Sent Done false
17889 5714 Acked Write 8 Done Sent Done false
17890 5715 Acked Read 1 Done Sent Done false
17891 5716 Acked Write 2 Done Sent Done false
17892 5717 Acked Read 10 Done Sent Done false
17893 5718 Acked Write 3 Done Sent Done false
17894 5719 Acked Read 2 Done Sent Done false
17895 5720 Acked Flush 0 Done Sent Done false
17896 5721 Acked Write 8 Done Sent Done false
17897 5722 Acked Read 2 Done Sent Done false
17898 5723 Acked Read 2 Done Sent Done false
17899 5724 Acked Read 6 Done Sent Done false
17900 5725 Acked Read 3 Done Sent Done false
17901 5726 Acked Write 7 Done Sent Done false
17902 5727 Acked Write 9 Done Sent Done false
17903 5728 Acked Write 4 Done Sent Done false
17904 5729 Acked Write 4 Done Sent Done false
17905 5730 Acked Flush 0 Done Sent Done false
17906 5731 Acked Write 5 Done Sent Done false
17907 5732 Acked Write 9 Done Sent Done false
17908 5733 Acked Write 10 Done Sent Done false
17909 5734 Acked Write 1 Done Sent Done false
17910 5735 Acked Read 5 Done Sent Done false
17911 5736 Acked Read 9 Done Sent Done false
17912 5737 Acked Write 3 Done Sent Done false
17913 5738 Acked Write 2 Done Sent Done false
17914 5739 Acked Write 10 Done Sent Done false
17915 5740 Acked Read 4 Done Sent Done false
17916 5741 Acked Write 10 Done Sent Done false
17917 5742 Acked Write 6 Done Sent Done false
17918 5743 Acked Read 8 Done Sent Done false
17919 5744 Acked Write 10 Done Sent Done false
17920 5745 Acked Write 7 Done Sent Done false
17921 5746 Acked Read 7 Done Sent Done false
17922 5747 Acked Read 6 Done Sent Done false
17923 5748 Acked Write 4 Done Sent Done false
17924 5749 Acked Read 10 Done Sent Done false
17925 5750 Acked Read 7 Done Sent Done false
17926 5751 Acked Read 1 Done Sent Done false
17927 5752 Acked Read 9 Done Sent Done false
17928 5753 Acked Write 10 Done Sent Done false
17929 5754 Acked Read 5 Done Sent Done false
17930 5755 Acked Write 5 Done Sent Done false
17931 5756 Acked Flush 0 Done Sent Done false
17932 5757 Acked Flush 0 Done Sent Done false
17933 5758 Acked Write 4 Done Sent Done false
17934 5759 Acked Flush 0 Done Sent Done false
17935 5760 Acked Read 6 Done Sent Done false
17936 5761 Acked Write 1 Done Sent Done false
17937 5762 Acked Read 9 Done Sent Done false
17938 5763 Acked Read 1 Done Sent Done false
17939 5764 Acked Read 7 Done Sent Done false
17940 5765 Acked Write 1 Done Sent Done false
17941 5766 Acked Read 3 Done Sent Done false
17942 5767 Acked Read 1 Done Sent Done false
17943 5768 Acked Read 2 Done Sent Done false
17944 5769 Acked Write 10 Done Sent Done false
17945 5770 Acked Write 6 Done Sent Done false
17946 5771 Acked Write 6 Done Sent Done false
17947 5772 Acked Read 7 Done Sent Done false
17948 5773 Acked Flush 0 Done Sent Done false
17949 5774 Acked Flush 0 Done Sent Done false
17950 5775 Acked Read 2 Done Sent Done false
17951 5776 Acked Write 7 Done Sent Done false
17952 5777 Acked Write 1 Done Sent Done false
17953 5778 Acked Read 7 Done Sent Done false
17954 5779 Acked Read 6 Done Sent Done false
17955 5780 Acked Write 1 Done Sent Done false
17956 5781 Acked Flush 0 Done Sent Done false
17957 5782 Acked Write 5 Done Sent Done false
17958 5783 Acked Write 1 Done Sent Done false
17959 5784 Acked Read 3 Done Sent Done false
17960 5785 Acked Write 4 Done Sent Done false
17961 5786 Acked Write 5 Done Sent Done false
17962 5787 Acked Write 4 Done Sent Done false
17963 5788 Acked Read 4 Done Sent Done false
17964 5789 Acked Write 6 Done Sent Done false
17965 5790 Acked Write 5 Done Sent Done false
17966 5791 Acked Write 3 Done Sent Done false
17967 5792 Acked Write 9 Done Sent Done false
17968 5793 Acked Read 8 Done Sent Done false
17969 5794 Acked Read 1 Done Sent Done false
17970 5795 Acked Write 6 Done Sent Done false
17971 5796 Acked Read 10 Done Sent Done false
17972 5797 Acked Write 1 Done Sent Done false
17973 5798 Acked Write 6 Done Sent Done false
17974 5799 Acked Read 1 Done Sent Done false
17975 5800 Acked Write 2 Done Sent Done false
17976 5801 Acked Flush 0 Done Sent Done false
17977 5802 Acked Write 6 Done Sent Done false
17978 5803 Acked Write 8 Done Sent Done false
17979 5804 Acked Read 10 Done Sent Done false
17980 5805 Acked Read 4 Done Sent Done false
17981 5806 Acked Read 5 Done Sent Done false
17982 5807 Acked Read 6 Done Sent Done false
17983 5808 Acked Read 7 Done Sent Done false
17984 5809 Acked Read 5 Done Sent Done false
17985 5810 Acked Flush 0 Done Sent Done false
17986 5811 Acked Read 5 Done Sent Done false
17987 5812 Acked Read 4 Done Sent Done false
17988 5813 Acked Read 1 Done Sent Done false
17989 5814 Acked Read 4 Done Sent Done false
17990 5815 Acked Write 9 Done Sent Done false
17991 5816 Acked Flush 0 Done Sent Done false
17992 5817 Acked Read 9 Done Sent Done false
17993 5818 Acked Read 1 Done Sent Done false
17994 5819 Acked Flush 0 Done Sent Done false
17995 5820 Acked Write 3 Done Sent Done false
17996 5821 Acked Read 4 Done Sent Done false
17997 5822 Acked Write 4 Done Sent Done false
17998 5823 Acked Read 3 Done Sent Done false
17999 5824 Acked Write 6 Done Sent Done false
18000 5825 Acked Flush 0 Done Sent Done false
18001 5826 Acked Flush 0 Done Sent Done false
18002 5827 Acked Flush 0 Done Sent Done false
18003 5828 Acked Write 7 Done Sent Done false
18004 5829 Acked Read 4 Done Sent Done false
18005 5830 Acked Write 3 Done Sent Done false
18006 STATES DS:0 DS:1 DS:2 TOTAL
18007 Sent 0 301 0 301
18008 Done 301 0 301 602
18009 Skipped 0 0 0 0
18010 Error 0 0 0 0
18011 Last Flush: 5827 5529 5827
18012 Downstairs last five completed: 5529 5528 5527 5526 5525
18013 Upstairs last five completed: 5830 5829 5828 5827 5826
18014 CLIENT: Up:0 ds:301 act:2
18015 ----------------------------------------------------------------
18016 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18017 States: Active Offline Active
18018 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18019 5530 Acked Write 6 Done Sent Done false
18020 5531 Acked Read 5 Done Sent Done false
18021 5532 Acked Read 3 Done Sent Done false
18022 5533 Acked Write 6 Done Sent Done false
18023 5534 Acked Write 1 Done Sent Done false
18024 5535 Acked Read 7 Done Sent Done false
18025 5536 Acked Read 1 Done Sent Done false
18026 5537 Acked Read 3 Done Sent Done false
18027 5538 Acked Read 5 Done Sent Done false
18028 5539 Acked Write 9 Done Sent Done false
18029 5540 Acked Read 4 Done Sent Done false
18030 5541 Acked Read 10 Done Sent Done false
18031 5542 Acked Write 9 Done Sent Done false
18032 5543 Acked Flush 0 Done Sent Done false
18033 5544 Acked Read 2 Done Sent Done false
18034 5545 Acked Write 5 Done Sent Done false
18035 5546 Acked Read 6 Done Sent Done false
18036 5547 Acked Read 4 Done Sent Done false
18037 5548 Acked Read 2 Done Sent Done false
18038 5549 Acked Flush 0 Done Sent Done false
18039 5550 Acked Write 4 Done Sent Done false
18040 5551 Acked Write 2 Done Sent Done false
18041 5552 Acked Write 1 Done Sent Done false
18042 5553 Acked Write 1 Done Sent Done false
18043 5554 Acked Read 2 Done Sent Done false
18044 5555 Acked Read 3 Done Sent Done false
18045 5556 Acked Read 6 Done Sent Done false
18046 5557 Acked Read 8 Done Sent Done false
18047 5558 Acked Read 7 Done Sent Done false
18048 5559 Acked Read 7 Done Sent Done false
18049 5560 Acked Read 1 Done Sent Done false
18050 5561 Acked Flush 0 Done Sent Done false
18051 5562 Acked Read 1 Done Sent Done false
18052 5563 Acked Read 4 Done Sent Done false
18053 5564 Acked Write 6 Done Sent Done false
18054 5565 Acked Write 3 Done Sent Done false
18055 5566 Acked Write 7 Done Sent Done false
18056 5567 Acked Read 3 Done Sent Done false
18057 5568 Acked Write 3 Done Sent Done false
18058 5569 Acked Read 5 Done Sent Done false
18059 5570 Acked Write 7 Done Sent Done false
18060 5571 Acked Read 9 Done Sent Done false
18061 5572 Acked Write 10 Done Sent Done false
18062 5573 Acked Read 3 Done Sent Done false
18063 5574 Acked Write 3 Done Sent Done false
18064 5575 Acked Write 4 Done Sent Done false
18065 5576 Acked Read 8 Done Sent Done false
18066 5577 Acked Read 5 Done Sent Done false
18067 5578 Acked Read 2 Done Sent Done false
18068 5579 Acked Write 1 Done Sent Done false
18069 5580 Acked Write 4 Done Sent Done false
18070 5581 Acked Read 7 Done Sent Done false
18071 5582 Acked Read 5 Done Sent Done false
18072 5583 Acked Write 9 Done Sent Done false
18073 5584 Acked Read 1 Done Sent Done false
18074 5585 Acked Read 1 Done Sent Done false
18075 5586 Acked Read 8 Done Sent Done false
18076 5587 Acked Read 2 Done Sent Done false
18077 5588 Acked Read 7 Done Sent Done false
18078 5589 Acked Read 5 Done Sent Done false
18079 5590 Acked Read 7 Done Sent Done false
18080 5591 Acked Write 9 Done Sent Done false
18081 5592 Acked Read 4 Done Sent Done false
18082 5593 Acked Flush 0 Done Sent Done false
18083 5594 Acked Write 4 Done Sent Done false
18084 5595 Acked Read 8 Done Sent Done false
18085 5596 Acked Write 1 Done Sent Done false
18086 5597 Acked Flush 0 Done Sent Done false
18087 5598 Acked Write 7 Done Sent Done false
18088 5599 Acked Write 3 Done Sent Done false
18089 5600 Acked Read 3 Done Sent Done false
18090 5601 Acked Read 6 Done Sent Done false
18091 5602 Acked Read 3 Done Sent Done false
18092 5603 Acked Flush 0 Done Sent Done false
18093 5604 Acked Read 10 Done Sent Done false
18094 5605 Acked Write 6 Done Sent Done false
18095 5606 Acked Read 4 Done Sent Done false
18096 5607 Acked Write 10 Done Sent Done false
18097 5608 Acked Read 4 Done Sent Done false
18098 5609 Acked Read 8 Done Sent Done false
18099 5610 Acked Read 10 Done Sent Done false
18100 5611 Acked Read 1 Done Sent Done false
18101 5612 Acked Read 9 Done Sent Done false
18102 5613 Acked Read 8 Done Sent Done false
18103 5614 Acked Write 8 Done Sent Done false
18104 5615 Acked Flush 0 Done Sent Done false
18105 5616 Acked Read 7 Done Sent Done false
18106 5617 Acked Write 5 Done Sent Done false
18107 5618 Acked Write 3 Done Sent Done false
18108 5619 Acked Write 6 Done Sent Done false
18109 5620 Acked Write 10 Done Sent Done false
18110 5621 Acked Write 1 Done Sent Done false
18111 5622 Acked Read 4 Done Sent Done false
18112 5623 Acked Read 9 Done Sent Done false
18113 5624 Acked Read 3 Done Sent Done false
18114 5625 Acked Read 8 Done Sent Done false
18115 5626 Acked Read 7 Done Sent Done false
18116 5627 Acked Write 10 Done Sent Done false
18117 5628 Acked Read 3 Done Sent Done false
18118 5629 Acked Read 9 Done Sent Done false
18119 5630 Acked Write 10 Done Sent Done false
18120 5631 Acked Write 5 Done Sent Done false
18121 5632 Acked Write 1 Done Sent Done false
18122 5633 Acked Flush 0 Done Sent Done false
18123 5634 Acked Flush 0 Done Sent Done false
18124 5635 Acked Flush 0 Done Sent Done false
18125 5636 Acked Write 4 Done Sent Done false
18126 5637 Acked Write 10 Done Sent Done false
18127 5638 Acked Flush 0 Done Sent Done false
18128 5639 Acked Write 7 Done Sent Done false
18129 5640 Acked Write 9 Done Sent Done false
18130 5641 Acked Read 3 Done Sent Done false
18131 5642 Acked Read 8 Done Sent Done false
18132 5643 Acked Read 9 Done Sent Done false
18133 5644 Acked Read 2 Done Sent Done false
18134 5645 Acked Read 7 Done Sent Done false
18135 5646 Acked Flush 0 Done Sent Done false
18136 5647 Acked Write 10 Done Sent Done false
18137 5648 Acked Flush 0 Done Sent Done false
18138 5649 Acked Flush 0 Done Sent Done false
18139 5650 Acked Read 8 Done Sent Done false
18140 5651 Acked Read 5 Done Sent Done false
18141 5652 Acked Write 6 Done Sent Done false
18142 5653 Acked Read 9 Done Sent Done false
18143 5654 Acked Write 5 Done Sent Done false
18144 5655 Acked Read 4 Done Sent Done false
18145 5656 Acked Flush 0 Done Sent Done false
18146 5657 Acked Write 2 Done Sent Done false
18147 5658 Acked Read 2 Done Sent Done false
18148 5659 Acked Write 7 Done Sent Done false
18149 5660 Acked Read 5 Done Sent Done false
18150 5661 Acked Read 2 Done Sent Done false
18151 5662 Acked Read 4 Done Sent Done false
18152 5663 Acked Read 6 Done Sent Done false
18153 5664 Acked Read 1 Done Sent Done false
18154 5665 Acked Read 8 Done Sent Done false
18155 5666 Acked Read 5 Done Sent Done false
18156 5667 Acked Read 2 Done Sent Done false
18157 5668 Acked Read 2 Done Sent Done false
18158 5669 Acked Read 1 Done Sent Done false
18159 5670 Acked Read 5 Done Sent Done false
18160 5671 Acked Read 5 Done Sent Done false
18161 5672 Acked Read 10 Done Sent Done false
18162 5673 Acked Read 4 Done Sent Done false
18163 5674 Acked Write 10 Done Sent Done false
18164 5675 Acked Read 9 Done Sent Done false
18165 5676 Acked Read 8 Done Sent Done false
18166 5677 Acked Write 2 Done Sent Done false
18167 5678 Acked Read 9 Done Sent Done false
18168 5679 Acked Write 3 Done Sent Done false
18169 5680 Acked Read 8 Done Sent Done false
18170 5681 Acked Write 10 Done Sent Done false
18171 5682 Acked Flush 0 Done Sent Done false
18172 5683 Acked Read 1 Done Sent Done false
18173 5684 Acked Read 10 Done Sent Done false
18174 5685 Acked Read 10 Done Sent Done false
18175 5686 Acked Read 4 Done Sent Done false
18176 5687 Acked Write 10 Done Sent Done false
18177 5688 Acked Write 1 Done Sent Done false
18178 5689 Acked Read 4 Done Sent Done false
18179 5690 Acked Read 1 Done Sent Done false
18180 5691 Acked Write 8 Done Sent Done false
18181 5692 Acked Flush 0 Done Sent Done false
18182 5693 Acked Flush 0 Done Sent Done false
18183 5694 Acked Write 2 Done Sent Done false
18184 5695 Acked Write 8 Done Sent Done false
18185 5696 Acked Write 7 Done Sent Done false
18186 5697 Acked Read 4 Done Sent Done false
18187 5698 Acked Write 10 Done Sent Done false
18188 5699 Acked Write 3 Done Sent Done false
18189 5700 Acked Read 7 Done Sent Done false
18190 5701 Acked Read 6 Done Sent Done false
18191 5702 Acked Read 2 Done Sent Done false
18192 5703 Acked Read 3 Done Sent Done false
18193 5704 Acked Read 7 Done Sent Done false
18194 5705 Acked Read 7 Done Sent Done false
18195 5706 Acked Write 1 Done Sent Done false
18196 5707 Acked Write 2 Done Sent Done false
18197 5708 Acked Read 10 Done Sent Done false
18198 5709 Acked Read 5 Done Sent Done false
18199 5710 Acked Read 3 Done Sent Done false
18200 5711 Acked Read 3 Done Sent Done false
18201 5712 Acked Flush 0 Done Sent Done false
18202 5713 Acked Read 7 Done Sent Done false
18203 5714 Acked Write 8 Done Sent Done false
18204 5715 Acked Read 1 Done Sent Done false
18205 5716 Acked Write 2 Done Sent Done false
18206 5717 Acked Read 10 Done Sent Done false
18207 5718 Acked Write 3 Done Sent Done false
18208 5719 Acked Read 2 Done Sent Done false
18209 5720 Acked Flush 0 Done Sent Done false
18210 5721 Acked Write 8 Done Sent Done false
18211 5722 Acked Read 2 Done Sent Done false
18212 5723 Acked Read 2 Done Sent Done false
18213 5724 Acked Read 6 Done Sent Done false
18214 5725 Acked Read 3 Done Sent Done false
18215 5726 Acked Write 7 Done Sent Done false
18216 5727 Acked Write 9 Done Sent Done false
18217 5728 Acked Write 4 Done Sent Done false
18218 5729 Acked Write 4 Done Sent Done false
18219 5730 Acked Flush 0 Done Sent Done false
18220 5731 Acked Write 5 Done Sent Done false
18221 5732 Acked Write 9 Done Sent Done false
18222 5733 Acked Write 10 Done Sent Done false
18223 5734 Acked Write 1 Done Sent Done false
18224 5735 Acked Read 5 Done Sent Done false
18225 5736 Acked Read 9 Done Sent Done false
18226 5737 Acked Write 3 Done Sent Done false
18227 5738 Acked Write 2 Done Sent Done false
18228 5739 Acked Write 10 Done Sent Done false
18229 5740 Acked Read 4 Done Sent Done false
18230 5741 Acked Write 10 Done Sent Done false
18231 5742 Acked Write 6 Done Sent Done false
18232 5743 Acked Read 8 Done Sent Done false
18233 5744 Acked Write 10 Done Sent Done false
18234 5745 Acked Write 7 Done Sent Done false
18235 5746 Acked Read 7 Done Sent Done false
18236 5747 Acked Read 6 Done Sent Done false
18237 5748 Acked Write 4 Done Sent Done false
18238 5749 Acked Read 10 Done Sent Done false
18239 5750 Acked Read 7 Done Sent Done false
18240 5751 Acked Read 1 Done Sent Done false
18241 5752 Acked Read 9 Done Sent Done false
18242 5753 Acked Write 10 Done Sent Done false
18243 5754 Acked Read 5 Done Sent Done false
18244 5755 Acked Write 5 Done Sent Done false
18245 5756 Acked Flush 0 Done Sent Done false
18246 5757 Acked Flush 0 Done Sent Done false
18247 5758 Acked Write 4 Done Sent Done false
18248 5759 Acked Flush 0 Done Sent Done false
18249 5760 Acked Read 6 Done Sent Done false
18250 5761 Acked Write 1 Done Sent Done false
18251 5762 Acked Read 9 Done Sent Done false
18252 5763 Acked Read 1 Done Sent Done false
18253 5764 Acked Read 7 Done Sent Done false
18254 5765 Acked Write 1 Done Sent Done false
18255 5766 Acked Read 3 Done Sent Done false
18256 5767 Acked Read 1 Done Sent Done false
18257 5768 Acked Read 2 Done Sent Done false
18258 5769 Acked Write 10 Done Sent Done false
18259 5770 Acked Write 6 Done Sent Done false
18260 5771 Acked Write 6 Done Sent Done false
18261 5772 Acked Read 7 Done Sent Done false
18262 5773 Acked Flush 0 Done Sent Done false
18263 5774 Acked Flush 0 Done Sent Done false
18264 5775 Acked Read 2 Done Sent Done false
18265 5776 Acked Write 7 Done Sent Done false
18266 5777 Acked Write 1 Done Sent Done false
18267 5778 Acked Read 7 Done Sent Done false
18268 5779 Acked Read 6 Done Sent Done false
18269 5780 Acked Write 1 Done Sent Done false
18270 5781 Acked Flush 0 Done Sent Done false
18271 5782 Acked Write 5 Done Sent Done false
18272 5783 Acked Write 1 Done Sent Done false
18273 5784 Acked Read 3 Done Sent Done false
18274 5785 Acked Write 4 Done Sent Done false
18275 5786 Acked Write 5 Done Sent Done false
18276 5787 Acked Write 4 Done Sent Done false
18277 5788 Acked Read 4 Done Sent Done false
18278 5789 Acked Write 6 Done Sent Done false
18279 5790 Acked Write 5 Done Sent Done false
18280 5791 Acked Write 3 Done Sent Done false
18281 5792 Acked Write 9 Done Sent Done false
18282 5793 Acked Read 8 Done Sent Done false
18283 5794 Acked Read 1 Done Sent Done false
18284 5795 Acked Write 6 Done Sent Done false
18285 5796 Acked Read 10 Done Sent Done false
18286 5797 Acked Write 1 Done Sent Done false
18287 5798 Acked Write 6 Done Sent Done false
18288 5799 Acked Read 1 Done Sent Done false
18289 5800 Acked Write 2 Done Sent Done false
18290 5801 Acked Flush 0 Done Sent Done false
18291 5802 Acked Write 6 Done Sent Done false
18292 5803 Acked Write 8 Done Sent Done false
18293 5804 Acked Read 10 Done Sent Done false
18294 5805 Acked Read 4 Done Sent Done false
18295 5806 Acked Read 5 Done Sent Done false
18296 5807 Acked Read 6 Done Sent Done false
18297 5808 Acked Read 7 Done Sent Done false
18298 5809 Acked Read 5 Done Sent Done false
18299 5810 Acked Flush 0 Done Sent Done false
18300 5811 Acked Read 5 Done Sent Done false
18301 5812 Acked Read 4 Done Sent Done false
18302 5813 Acked Read 1 Done Sent Done false
18303 5814 Acked Read 4 Done Sent Done false
18304 5815 Acked Write 9 Done Sent Done false
18305 5816 Acked Flush 0 Done Sent Done false
18306 5817 Acked Read 9 Done Sent Done false
18307 5818 Acked Read 1 Done Sent Done false
18308 5819 Acked Flush 0 Done Sent Done false
18309 5820 Acked Write 3 Done Sent Done false
18310 5821 Acked Read 4 Done Sent Done false
18311 5822 Acked Write 4 Done Sent Done false
18312 5823 Acked Read 3 Done Sent Done false
18313 5824 Acked Write 6 Done Sent Done false
18314 5825 Acked Flush 0 Done Sent Done false
18315 5826 Acked Flush 0 Done Sent Done false
18316 5827 Acked Flush 0 Done Sent Done false
18317 5828 Acked Write 7 Done Sent Done false
18318 5829 Acked Read 4 Done Sent Done false
18319 5830 Acked Write 3 Done Sent Done false
18320 5831 Acked Flush 0 Done Sent Done false
18321 STATES DS:0 DS:1 DS:2 TOTAL
18322 Sent 0 302 0 302
18323 Done 302 0 302 604
18324 Skipped 0 0 0 0
18325 Error 0 0 0 0
18326 Last Flush: 5831 5529 5831
18327 Downstairs last five completed: 5529 5528 5527 5526 5525
18328 Upstairs last five completed: 5831 5830 5829 5828 5827
18329 CLIENT: Up:0 ds:302 act:2
183302024-12-11T23:44:20.202ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
183312024-12-11T23:44:20.202ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
183322024-12-11T23:44:20.202ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
183332024-12-11T23:44:20.202ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
183342024-12-11T23:44:20.202ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
18335 ----------------------------------------------------------------
18336 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18337 States: Active Offline Active
18338 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18339 5530 Acked Write 6 Done Sent Done false
18340 5531 Acked Read 5 Done Sent Done false
18341 5532 Acked Read 3 Done Sent Done false
18342 5533 Acked Write 6 Done Sent Done false
18343 5534 Acked Write 1 Done Sent Done false
18344 5535 Acked Read 7 Done Sent Done false
18345 5536 Acked Read 1 Done Sent Done false
18346 5537 Acked Read 3 Done Sent Done false
18347 5538 Acked Read 5 Done Sent Done false
18348 5539 Acked Write 9 Done Sent Done false
18349 5540 Acked Read 4 Done Sent Done false
18350 5541 Acked Read 10 Done Sent Done false
18351 5542 Acked Write 9 Done Sent Done false
18352 5543 Acked Flush 0 Done Sent Done false
18353 5544 Acked Read 2 Done Sent Done false
18354 5545 Acked Write 5 Done Sent Done false
18355 5546 Acked Read 6 Done Sent Done false
18356 5547 Acked Read 4 Done Sent Done false
18357 5548 Acked Read 2 Done Sent Done false
18358 5549 Acked Flush 0 Done Sent Done false
18359 5550 Acked Write 4 Done Sent Done false
18360 5551 Acked Write 2 Done Sent Done false
18361 5552 Acked Write 1 Done Sent Done false
18362 5553 Acked Write 1 Done Sent Done false
18363 5554 Acked Read 2 Done Sent Done false
18364 5555 Acked Read 3 Done Sent Done false
18365 5556 Acked Read 6 Done Sent Done false
18366 5557 Acked Read 8 Done Sent Done false
18367 5558 Acked Read 7 Done Sent Done false
18368 5559 Acked Read 7 Done Sent Done false
18369 5560 Acked Read 1 Done Sent Done false
18370 5561 Acked Flush 0 Done Sent Done false
18371 5562 Acked Read 1 Done Sent Done false
18372 5563 Acked Read 4 Done Sent Done false
18373 5564 Acked Write 6 Done Sent Done false
18374 5565 Acked Write 3 Done Sent Done false
18375 5566 Acked Write 7 Done Sent Done false
18376 5567 Acked Read 3 Done Sent Done false
18377 5568 Acked Write 3 Done Sent Done false
18378 5569 Acked Read 5 Done Sent Done false
18379 5570 Acked Write 7 Done Sent Done false
18380 5571 Acked Read 9 Done Sent Done false
18381 5572 Acked Write 10 Done Sent Done false
18382 5573 Acked Read 3 Done Sent Done false
18383 5574 Acked Write 3 Done Sent Done false
18384 5575 Acked Write 4 Done Sent Done false
18385 5576 Acked Read 8 Done Sent Done false
18386 5577 Acked Read 5 Done Sent Done false
18387 5578 Acked Read 2 Done Sent Done false
18388 5579 Acked Write 1 Done Sent Done false
18389 5580 Acked Write 4 Done Sent Done false
18390 5581 Acked Read 7 Done Sent Done false
18391 5582 Acked Read 5 Done Sent Done false
18392 5583 Acked Write 9 Done Sent Done false
18393 5584 Acked Read 1 Done Sent Done false
18394 5585 Acked Read 1 Done Sent Done false
18395 5586 Acked Read 8 Done Sent Done false
18396 5587 Acked Read 2 Done Sent Done false
18397 5588 Acked Read 7 Done Sent Done false
18398 5589 Acked Read 5 Done Sent Done false
18399 5590 Acked Read 7 Done Sent Done false
18400 5591 Acked Write 9 Done Sent Done false
18401 5592 Acked Read 4 Done Sent Done false
18402 5593 Acked Flush 0 Done Sent Done false
18403 5594 Acked Write 4 Done Sent Done false
18404 5595 Acked Read 8 Done Sent Done false
18405 5596 Acked Write 1 Done Sent Done false
18406 5597 Acked Flush 0 Done Sent Done false
18407 5598 Acked Write 7 Done Sent Done false
18408 5599 Acked Write 3 Done Sent Done false
18409 5600 Acked Read 3 Done Sent Done false
18410 5601 Acked Read 6 Done Sent Done false
18411 5602 Acked Read 3 Done Sent Done false
18412 5603 Acked Flush 0 Done Sent Done false
18413 5604 Acked Read 10 Done Sent Done false
18414 5605 Acked Write 6 Done Sent Done false
18415 5606 Acked Read 4 Done Sent Done false
18416 5607 Acked Write 10 Done Sent Done false
18417 5608 Acked Read 4 Done Sent Done false
18418 5609 Acked Read 8 Done Sent Done false
18419 5610 Acked Read 10 Done Sent Done false
18420 5611 Acked Read 1 Done Sent Done false
18421 5612 Acked Read 9 Done Sent Done false
18422 5613 Acked Read 8 Done Sent Done false
18423 5614 Acked Write 8 Done Sent Done false
18424 5615 Acked Flush 0 Done Sent Done false
18425 5616 Acked Read 7 Done Sent Done false
18426 5617 Acked Write 5 Done Sent Done false
18427 5618 Acked Write 3 Done Sent Done false
18428 5619 Acked Write 6 Done Sent Done false
18429 5620 Acked Write 10 Done Sent Done false
18430 5621 Acked Write 1 Done Sent Done false
18431 5622 Acked Read 4 Done Sent Done false
18432 5623 Acked Read 9 Done Sent Done false
18433 5624 Acked Read 3 Done Sent Done false
18434 5625 Acked Read 8 Done Sent Done false
18435 5626 Acked Read 7 Done Sent Done false
18436 5627 Acked Write 10 Done Sent Done false
18437 5628 Acked Read 3 Done Sent Done false
18438 5629 Acked Read 9 Done Sent Done false
18439 5630 Acked Write 10 Done Sent Done false
18440 5631 Acked Write 5 Done Sent Done false
18441 5632 Acked Write 1 Done Sent Done false
18442 5633 Acked Flush 0 Done Sent Done false
18443 5634 Acked Flush 0 Done Sent Done false
18444 5635 Acked Flush 0 Done Sent Done false
18445 5636 Acked Write 4 Done Sent Done false
18446 5637 Acked Write 10 Done Sent Done false
18447 5638 Acked Flush 0 Done Sent Done false
18448 5639 Acked Write 7 Done Sent Done false
18449 5640 Acked Write 9 Done Sent Done false
18450 5641 Acked Read 3 Done Sent Done false
18451 5642 Acked Read 8 Done Sent Done false
18452 5643 Acked Read 9 Done Sent Done false
18453 5644 Acked Read 2 Done Sent Done false
18454 5645 Acked Read 7 Done Sent Done false
18455 5646 Acked Flush 0 Done Sent Done false
18456 5647 Acked Write 10 Done Sent Done false
18457 5648 Acked Flush 0 Done Sent Done false
18458 5649 Acked Flush 0 Done Sent Done false
18459 5650 Acked Read 8 Done Sent Done false
18460 5651 Acked Read 5 Done Sent Done false
18461 5652 Acked Write 6 Done Sent Done false
18462 5653 Acked Read 9 Done Sent Done false
18463 5654 Acked Write 5 Done Sent Done false
18464 5655 Acked Read 4 Done Sent Done false
18465 5656 Acked Flush 0 Done Sent Done false
18466 5657 Acked Write 2 Done Sent Done false
18467 5658 Acked Read 2 Done Sent Done false
18468 5659 Acked Write 7 Done Sent Done false
18469 5660 Acked Read 5 Done Sent Done false
18470 5661 Acked Read 2 Done Sent Done false
18471 5662 Acked Read 4 Done Sent Done false
18472 5663 Acked Read 6 Done Sent Done false
18473 5664 Acked Read 1 Done Sent Done false
18474 5665 Acked Read 8 Done Sent Done false
18475 5666 Acked Read 5 Done Sent Done false
18476 5667 Acked Read 2 Done Sent Done false
18477 5668 Acked Read 2 Done Sent Done false
18478 5669 Acked Read 1 Done Sent Done false
18479 5670 Acked Read 5 Done Sent Done false
18480 5671 Acked Read 5 Done Sent Done false
18481 5672 Acked Read 10 Done Sent Done false
18482 5673 Acked Read 4 Done Sent Done false
18483 5674 Acked Write 10 Done Sent Done false
18484 5675 Acked Read 9 Done Sent Done false
18485 5676 Acked Read 8 Done Sent Done false
18486 5677 Acked Write 2 Done Sent Done false
18487 5678 Acked Read 9 Done Sent Done false
18488 5679 Acked Write 3 Done Sent Done false
18489 5680 Acked Read 8 Done Sent Done false
18490 5681 Acked Write 10 Done Sent Done false
18491 5682 Acked Flush 0 Done Sent Done false
18492 5683 Acked Read 1 Done Sent Done false
18493 5684 Acked Read 10 Done Sent Done false
18494 5685 Acked Read 10 Done Sent Done false
18495 5686 Acked Read 4 Done Sent Done false
18496 5687 Acked Write 10 Done Sent Done false
18497 5688 Acked Write 1 Done Sent Done false
18498 5689 Acked Read 4 Done Sent Done false
18499 5690 Acked Read 1 Done Sent Done false
18500 5691 Acked Write 8 Done Sent Done false
18501 5692 Acked Flush 0 Done Sent Done false
18502 5693 Acked Flush 0 Done Sent Done false
18503 5694 Acked Write 2 Done Sent Done false
18504 5695 Acked Write 8 Done Sent Done false
18505 5696 Acked Write 7 Done Sent Done false
18506 5697 Acked Read 4 Done Sent Done false
18507 5698 Acked Write 10 Done Sent Done false
18508 5699 Acked Write 3 Done Sent Done false
18509 5700 Acked Read 7 Done Sent Done false
18510 5701 Acked Read 6 Done Sent Done false
18511 5702 Acked Read 2 Done Sent Done false
18512 5703 Acked Read 3 Done Sent Done false
18513 5704 Acked Read 7 Done Sent Done false
18514 5705 Acked Read 7 Done Sent Done false
18515 5706 Acked Write 1 Done Sent Done false
18516 5707 Acked Write 2 Done Sent Done false
18517 5708 Acked Read 10 Done Sent Done false
18518 5709 Acked Read 5 Done Sent Done false
18519 5710 Acked Read 3 Done Sent Done false
18520 5711 Acked Read 3 Done Sent Done false
18521 5712 Acked Flush 0 Done Sent Done false
18522 5713 Acked Read 7 Done Sent Done false
18523 5714 Acked Write 8 Done Sent Done false
18524 5715 Acked Read 1 Done Sent Done false
18525 5716 Acked Write 2 Done Sent Done false
18526 5717 Acked Read 10 Done Sent Done false
18527 5718 Acked Write 3 Done Sent Done false
18528 5719 Acked Read 2 Done Sent Done false
18529 5720 Acked Flush 0 Done Sent Done false
18530 5721 Acked Write 8 Done Sent Done false
18531 5722 Acked Read 2 Done Sent Done false
18532 5723 Acked Read 2 Done Sent Done false
18533 5724 Acked Read 6 Done Sent Done false
18534 5725 Acked Read 3 Done Sent Done false
18535 5726 Acked Write 7 Done Sent Done false
18536 5727 Acked Write 9 Done Sent Done false
18537 5728 Acked Write 4 Done Sent Done false
18538 5729 Acked Write 4 Done Sent Done false
18539 5730 Acked Flush 0 Done Sent Done false
18540 5731 Acked Write 5 Done Sent Done false
18541 5732 Acked Write 9 Done Sent Done false
18542 5733 Acked Write 10 Done Sent Done false
18543 5734 Acked Write 1 Done Sent Done false
18544 5735 Acked Read 5 Done Sent Done false
18545 5736 Acked Read 9 Done Sent Done false
18546 5737 Acked Write 3 Done Sent Done false
18547 5738 Acked Write 2 Done Sent Done false
18548 5739 Acked Write 10 Done Sent Done false
18549 5740 Acked Read 4 Done Sent Done false
18550 5741 Acked Write 10 Done Sent Done false
18551 5742 Acked Write 6 Done Sent Done false
18552 5743 Acked Read 8 Done Sent Done false
18553 5744 Acked Write 10 Done Sent Done false
18554 5745 Acked Write 7 Done Sent Done false
18555 5746 Acked Read 7 Done Sent Done false
18556 5747 Acked Read 6 Done Sent Done false
18557 5748 Acked Write 4 Done Sent Done false
18558 5749 Acked Read 10 Done Sent Done false
18559 5750 Acked Read 7 Done Sent Done false
18560 5751 Acked Read 1 Done Sent Done false
18561 5752 Acked Read 9 Done Sent Done false
18562 5753 Acked Write 10 Done Sent Done false
18563 5754 Acked Read 5 Done Sent Done false
18564 5755 Acked Write 5 Done Sent Done false
18565 5756 Acked Flush 0 Done Sent Done false
18566 5757 Acked Flush 0 Done Sent Done false
18567 5758 Acked Write 4 Done Sent Done false
18568 5759 Acked Flush 0 Done Sent Done false
18569 5760 Acked Read 6 Done Sent Done false
18570 5761 Acked Write 1 Done Sent Done false
18571 5762 Acked Read 9 Done Sent Done false
18572 5763 Acked Read 1 Done Sent Done false
18573 5764 Acked Read 7 Done Sent Done false
18574 5765 Acked Write 1 Done Sent Done false
18575 5766 Acked Read 3 Done Sent Done false
18576 5767 Acked Read 1 Done Sent Done false
18577 5768 Acked Read 2 Done Sent Done false
18578 5769 Acked Write 10 Done Sent Done false
18579 5770 Acked Write 6 Done Sent Done false
18580 5771 Acked Write 6 Done Sent Done false
18581 5772 Acked Read 7 Done Sent Done false
18582 5773 Acked Flush 0 Done Sent Done false
18583 5774 Acked Flush 0 Done Sent Done false
18584 5775 Acked Read 2 Done Sent Done false
18585 5776 Acked Write 7 Done Sent Done false
18586 5777 Acked Write 1 Done Sent Done false
18587 5778 Acked Read 7 Done Sent Done false
18588 5779 Acked Read 6 Done Sent Done false
18589 5780 Acked Write 1 Done Sent Done false
18590 5781 Acked Flush 0 Done Sent Done false
18591 5782 Acked Write 5 Done Sent Done false
18592 5783 Acked Write 1 Done Sent Done false
18593 5784 Acked Read 3 Done Sent Done false
18594 5785 Acked Write 4 Done Sent Done false
18595 5786 Acked Write 5 Done Sent Done false
18596 5787 Acked Write 4 Done Sent Done false
18597 5788 Acked Read 4 Done Sent Done false
18598 5789 Acked Write 6 Done Sent Done false
18599 5790 Acked Write 5 Done Sent Done false
18600 5791 Acked Write 3 Done Sent Done false
18601 5792 Acked Write 9 Done Sent Done false
18602 5793 Acked Read 8 Done Sent Done false
18603 5794 Acked Read 1 Done Sent Done false
18604 5795 Acked Write 6 Done Sent Done false
18605 5796 Acked Read 10 Done Sent Done false
18606 5797 Acked Write 1 Done Sent Done false
18607 5798 Acked Write 6 Done Sent Done false
18608 5799 Acked Read 1 Done Sent Done false
18609 5800 Acked Write 2 Done Sent Done false
18610 5801 Acked Flush 0 Done Sent Done false
18611 5802 Acked Write 6 Done Sent Done false
18612 5803 Acked Write 8 Done Sent Done false
18613 5804 Acked Read 10 Done Sent Done false
18614 5805 Acked Read 4 Done Sent Done false
18615 5806 Acked Read 5 Done Sent Done false
18616 5807 Acked Read 6 Done Sent Done false
18617 5808 Acked Read 7 Done Sent Done false
18618 5809 Acked Read 5 Done Sent Done false
18619 5810 Acked Flush 0 Done Sent Done false
18620 5811 Acked Read 5 Done Sent Done false
18621 5812 Acked Read 4 Done Sent Done false
18622 5813 Acked Read 1 Done Sent Done false
18623 5814 Acked Read 4 Done Sent Done false
18624 5815 Acked Write 9 Done Sent Done false
18625 5816 Acked Flush 0 Done Sent Done false
18626 5817 Acked Read 9 Done Sent Done false
18627 5818 Acked Read 1 Done Sent Done false
18628 5819 Acked Flush 0 Done Sent Done false
18629 5820 Acked Write 3 Done Sent Done false
18630 5821 Acked Read 4 Done Sent Done false
18631 5822 Acked Write 4 Done Sent Done false
18632 5823 Acked Read 3 Done Sent Done false
18633 5824 Acked Write 6 Done Sent Done false
18634 5825 Acked Flush 0 Done Sent Done false
18635 5826 Acked Flush 0 Done Sent Done false
18636 5827 Acked Flush 0 Done Sent Done false
18637 5828 Acked Write 7 Done Sent Done false
18638 5829 Acked Read 4 Done Sent Done false
18639 5830 Acked Write 3 Done Sent Done false
18640 5831 Acked Flush 0 Done Sent Done false
18641 STATES DS:0 DS:1 DS:2 TOTAL
18642 Sent 0 302 0 302
18643 Done 302 0 302 604
18644 Skipped 0 0 0 0
18645 Error 0 0 0 0
18646 Last Flush: 5831 5529 5831
18647 Downstairs last five completed: 5529 5528 5527 5526 5525
18648 Upstairs last five completed: 5831 5830 5829 5828 5827
18649 CLIENT: Up:0 ds:302 act:2
18650 ----------------------------------------------------------------
18651 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18652 States: Active Offline Active
18653 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18654 5530 Acked Write 6 Done Sent Done false
18655 5531 Acked Read 5 Done Sent Done false
18656 5532 Acked Read 3 Done Sent Done false
18657 5533 Acked Write 6 Done Sent Done false
18658 5534 Acked Write 1 Done Sent Done false
18659 5535 Acked Read 7 Done Sent Done false
18660 5536 Acked Read 1 Done Sent Done false
18661 5537 Acked Read 3 Done Sent Done false
18662 5538 Acked Read 5 Done Sent Done false
18663 5539 Acked Write 9 Done Sent Done false
18664 5540 Acked Read 4 Done Sent Done false
18665 5541 Acked Read 10 Done Sent Done false
18666 5542 Acked Write 9 Done Sent Done false
18667 5543 Acked Flush 0 Done Sent Done false
18668 5544 Acked Read 2 Done Sent Done false
18669 5545 Acked Write 5 Done Sent Done false
18670 5546 Acked Read 6 Done Sent Done false
18671 5547 Acked Read 4 Done Sent Done false
18672 5548 Acked Read 2 Done Sent Done false
18673 5549 Acked Flush 0 Done Sent Done false
18674 5550 Acked Write 4 Done Sent Done false
18675 5551 Acked Write 2 Done Sent Done false
18676 5552 Acked Write 1 Done Sent Done false
18677 5553 Acked Write 1 Done Sent Done false
18678 5554 Acked Read 2 Done Sent Done false
18679 5555 Acked Read 3 Done Sent Done false
18680 5556 Acked Read 6 Done Sent Done false
18681 5557 Acked Read 8 Done Sent Done false
18682 5558 Acked Read 7 Done Sent Done false
18683 5559 Acked Read 7 Done Sent Done false
18684 5560 Acked Read 1 Done Sent Done false
18685 5561 Acked Flush 0 Done Sent Done false
18686 5562 Acked Read 1 Done Sent Done false
18687 5563 Acked Read 4 Done Sent Done false
18688 5564 Acked Write 6 Done Sent Done false
18689 5565 Acked Write 3 Done Sent Done false
18690 5566 Acked Write 7 Done Sent Done false
18691 5567 Acked Read 3 Done Sent Done false
18692 5568 Acked Write 3 Done Sent Done false
18693 5569 Acked Read 5 Done Sent Done false
18694 5570 Acked Write 7 Done Sent Done false
18695 5571 Acked Read 9 Done Sent Done false
18696 5572 Acked Write 10 Done Sent Done false
18697 5573 Acked Read 3 Done Sent Done false
18698 5574 Acked Write 3 Done Sent Done false
18699 5575 Acked Write 4 Done Sent Done false
18700 5576 Acked Read 8 Done Sent Done false
18701 5577 Acked Read 5 Done Sent Done false
18702 5578 Acked Read 2 Done Sent Done false
18703 5579 Acked Write 1 Done Sent Done false
18704 5580 Acked Write 4 Done Sent Done false
18705 5581 Acked Read 7 Done Sent Done false
18706 5582 Acked Read 5 Done Sent Done false
18707 5583 Acked Write 9 Done Sent Done false
18708 5584 Acked Read 1 Done Sent Done false
18709 5585 Acked Read 1 Done Sent Done false
18710 5586 Acked Read 8 Done Sent Done false
18711 5587 Acked Read 2 Done Sent Done false
18712 5588 Acked Read 7 Done Sent Done false
18713 5589 Acked Read 5 Done Sent Done false
18714 5590 Acked Read 7 Done Sent Done false
18715 5591 Acked Write 9 Done Sent Done false
18716 5592 Acked Read 4 Done Sent Done false
18717 5593 Acked Flush 0 Done Sent Done false
18718 5594 Acked Write 4 Done Sent Done false
18719 5595 Acked Read 8 Done Sent Done false
18720 5596 Acked Write 1 Done Sent Done false
18721 5597 Acked Flush 0 Done Sent Done false
18722 5598 Acked Write 7 Done Sent Done false
18723 5599 Acked Write 3 Done Sent Done false
18724 5600 Acked Read 3 Done Sent Done false
18725 5601 Acked Read 6 Done Sent Done false
18726 5602 Acked Read 3 Done Sent Done false
18727 5603 Acked Flush 0 Done Sent Done false
18728 5604 Acked Read 10 Done Sent Done false
18729 5605 Acked Write 6 Done Sent Done false
18730 5606 Acked Read 4 Done Sent Done false
18731 5607 Acked Write 10 Done Sent Done false
18732 5608 Acked Read 4 Done Sent Done false
18733 5609 Acked Read 8 Done Sent Done false
18734 5610 Acked Read 10 Done Sent Done false
18735 5611 Acked Read 1 Done Sent Done false
18736 5612 Acked Read 9 Done Sent Done false
18737 5613 Acked Read 8 Done Sent Done false
18738 5614 Acked Write 8 Done Sent Done false
18739 5615 Acked Flush 0 Done Sent Done false
18740 5616 Acked Read 7 Done Sent Done false
18741 5617 Acked Write 5 Done Sent Done false
18742 5618 Acked Write 3 Done Sent Done false
18743 5619 Acked Write 6 Done Sent Done false
18744 5620 Acked Write 10 Done Sent Done false
18745 5621 Acked Write 1 Done Sent Done false
18746 5622 Acked Read 4 Done Sent Done false
18747 5623 Acked Read 9 Done Sent Done false
18748 5624 Acked Read 3 Done Sent Done false
18749 5625 Acked Read 8 Done Sent Done false
18750 5626 Acked Read 7 Done Sent Done false
18751 5627 Acked Write 10 Done Sent Done false
18752 5628 Acked Read 3 Done Sent Done false
18753 5629 Acked Read 9 Done Sent Done false
18754 5630 Acked Write 10 Done Sent Done false
18755 5631 Acked Write 5 Done Sent Done false
18756 5632 Acked Write 1 Done Sent Done false
18757 5633 Acked Flush 0 Done Sent Done false
18758 5634 Acked Flush 0 Done Sent Done false
18759 5635 Acked Flush 0 Done Sent Done false
18760 5636 Acked Write 4 Done Sent Done false
18761 5637 Acked Write 10 Done Sent Done false
18762 5638 Acked Flush 0 Done Sent Done false
18763 5639 Acked Write 7 Done Sent Done false
18764 5640 Acked Write 9 Done Sent Done false
18765 5641 Acked Read 3 Done Sent Done false
18766 5642 Acked Read 8 Done Sent Done false
18767 5643 Acked Read 9 Done Sent Done false
18768 5644 Acked Read 2 Done Sent Done false
18769 5645 Acked Read 7 Done Sent Done false
18770 5646 Acked Flush 0 Done Sent Done false
18771 5647 Acked Write 10 Done Sent Done false
18772 5648 Acked Flush 0 Done Sent Done false
18773 5649 Acked Flush 0 Done Sent Done false
18774 5650 Acked Read 8 Done Sent Done false
18775 5651 Acked Read 5 Done Sent Done false
18776 5652 Acked Write 6 Done Sent Done false
18777 5653 Acked Read 9 Done Sent Done false
18778 5654 Acked Write 5 Done Sent Done false
18779 5655 Acked Read 4 Done Sent Done false
18780 5656 Acked Flush 0 Done Sent Done false
18781 5657 Acked Write 2 Done Sent Done false
18782 5658 Acked Read 2 Done Sent Done false
18783 5659 Acked Write 7 Done Sent Done false
18784 5660 Acked Read 5 Done Sent Done false
18785 5661 Acked Read 2 Done Sent Done false
18786 5662 Acked Read 4 Done Sent Done false
18787 5663 Acked Read 6 Done Sent Done false
18788 5664 Acked Read 1 Done Sent Done false
18789 5665 Acked Read 8 Done Sent Done false
18790 5666 Acked Read 5 Done Sent Done false
18791 5667 Acked Read 2 Done Sent Done false
18792 5668 Acked Read 2 Done Sent Done false
18793 5669 Acked Read 1 Done Sent Done false
18794 5670 Acked Read 5 Done Sent Done false
18795 5671 Acked Read 5 Done Sent Done false
18796 5672 Acked Read 10 Done Sent Done false
18797 5673 Acked Read 4 Done Sent Done false
18798 5674 Acked Write 10 Done Sent Done false
18799 5675 Acked Read 9 Done Sent Done false
18800 5676 Acked Read 8 Done Sent Done false
18801 5677 Acked Write 2 Done Sent Done false
18802 5678 Acked Read 9 Done Sent Done false
18803 5679 Acked Write 3 Done Sent Done false
18804 5680 Acked Read 8 Done Sent Done false
18805 5681 Acked Write 10 Done Sent Done false
18806 5682 Acked Flush 0 Done Sent Done false
18807 5683 Acked Read 1 Done Sent Done false
18808 5684 Acked Read 10 Done Sent Done false
18809 5685 Acked Read 10 Done Sent Done false
18810 5686 Acked Read 4 Done Sent Done false
18811 5687 Acked Write 10 Done Sent Done false
18812 5688 Acked Write 1 Done Sent Done false
18813 5689 Acked Read 4 Done Sent Done false
18814 5690 Acked Read 1 Done Sent Done false
18815 5691 Acked Write 8 Done Sent Done false
18816 5692 Acked Flush 0 Done Sent Done false
18817 5693 Acked Flush 0 Done Sent Done false
18818 5694 Acked Write 2 Done Sent Done false
18819 5695 Acked Write 8 Done Sent Done false
18820 5696 Acked Write 7 Done Sent Done false
18821 5697 Acked Read 4 Done Sent Done false
18822 5698 Acked Write 10 Done Sent Done false
18823 5699 Acked Write 3 Done Sent Done false
18824 5700 Acked Read 7 Done Sent Done false
18825 5701 Acked Read 6 Done Sent Done false
18826 5702 Acked Read 2 Done Sent Done false
18827 5703 Acked Read 3 Done Sent Done false
18828 5704 Acked Read 7 Done Sent Done false
18829 5705 Acked Read 7 Done Sent Done false
18830 5706 Acked Write 1 Done Sent Done false
18831 5707 Acked Write 2 Done Sent Done false
18832 5708 Acked Read 10 Done Sent Done false
18833 5709 Acked Read 5 Done Sent Done false
18834 5710 Acked Read 3 Done Sent Done false
18835 5711 Acked Read 3 Done Sent Done false
18836 5712 Acked Flush 0 Done Sent Done false
18837 5713 Acked Read 7 Done Sent Done false
18838 5714 Acked Write 8 Done Sent Done false
18839 5715 Acked Read 1 Done Sent Done false
18840 5716 Acked Write 2 Done Sent Done false
18841 5717 Acked Read 10 Done Sent Done false
18842 5718 Acked Write 3 Done Sent Done false
18843 5719 Acked Read 2 Done Sent Done false
18844 5720 Acked Flush 0 Done Sent Done false
18845 5721 Acked Write 8 Done Sent Done false
18846 5722 Acked Read 2 Done Sent Done false
18847 5723 Acked Read 2 Done Sent Done false
18848 5724 Acked Read 6 Done Sent Done false
18849 5725 Acked Read 3 Done Sent Done false
18850 5726 Acked Write 7 Done Sent Done false
18851 5727 Acked Write 9 Done Sent Done false
18852 5728 Acked Write 4 Done Sent Done false
18853 5729 Acked Write 4 Done Sent Done false
18854 5730 Acked Flush 0 Done Sent Done false
18855 5731 Acked Write 5 Done Sent Done false
18856 5732 Acked Write 9 Done Sent Done false
18857 5733 Acked Write 10 Done Sent Done false
18858 5734 Acked Write 1 Done Sent Done false
18859 5735 Acked Read 5 Done Sent Done false
18860 5736 Acked Read 9 Done Sent Done false
18861 5737 Acked Write 3 Done Sent Done false
18862 5738 Acked Write 2 Done Sent Done false
18863 5739 Acked Write 10 Done Sent Done false
18864 5740 Acked Read 4 Done Sent Done false
18865 5741 Acked Write 10 Done Sent Done false
18866 5742 Acked Write 6 Done Sent Done false
18867 5743 Acked Read 8 Done Sent Done false
18868 5744 Acked Write 10 Done Sent Done false
18869 5745 Acked Write 7 Done Sent Done false
18870 5746 Acked Read 7 Done Sent Done false
18871 5747 Acked Read 6 Done Sent Done false
18872 5748 Acked Write 4 Done Sent Done false
18873 5749 Acked Read 10 Done Sent Done false
18874 5750 Acked Read 7 Done Sent Done false
18875 5751 Acked Read 1 Done Sent Done false
18876 5752 Acked Read 9 Done Sent Done false
18877 5753 Acked Write 10 Done Sent Done false
18878 5754 Acked Read 5 Done Sent Done false
18879 5755 Acked Write 5 Done Sent Done false
18880 5756 Acked Flush 0 Done Sent Done false
18881 5757 Acked Flush 0 Done Sent Done false
18882 5758 Acked Write 4 Done Sent Done false
18883 5759 Acked Flush 0 Done Sent Done false
18884 5760 Acked Read 6 Done Sent Done false
18885 5761 Acked Write 1 Done Sent Done false
18886 5762 Acked Read 9 Done Sent Done false
18887 5763 Acked Read 1 Done Sent Done false
18888 5764 Acked Read 7 Done Sent Done false
18889 5765 Acked Write 1 Done Sent Done false
18890 5766 Acked Read 3 Done Sent Done false
18891 5767 Acked Read 1 Done Sent Done false
18892 5768 Acked Read 2 Done Sent Done false
18893 5769 Acked Write 10 Done Sent Done false
18894 5770 Acked Write 6 Done Sent Done false
18895 5771 Acked Write 6 Done Sent Done false
18896 5772 Acked Read 7 Done Sent Done false
18897 5773 Acked Flush 0 Done Sent Done false
18898 5774 Acked Flush 0 Done Sent Done false
18899 5775 Acked Read 2 Done Sent Done false
18900 5776 Acked Write 7 Done Sent Done false
18901 5777 Acked Write 1 Done Sent Done false
18902 5778 Acked Read 7 Done Sent Done false
18903 5779 Acked Read 6 Done Sent Done false
18904 5780 Acked Write 1 Done Sent Done false
18905 5781 Acked Flush 0 Done Sent Done false
18906 5782 Acked Write 5 Done Sent Done false
18907 5783 Acked Write 1 Done Sent Done false
18908 5784 Acked Read 3 Done Sent Done false
18909 5785 Acked Write 4 Done Sent Done false
18910 5786 Acked Write 5 Done Sent Done false
18911 5787 Acked Write 4 Done Sent Done false
18912 5788 Acked Read 4 Done Sent Done false
18913 5789 Acked Write 6 Done Sent Done false
18914 5790 Acked Write 5 Done Sent Done false
18915 5791 Acked Write 3 Done Sent Done false
18916 5792 Acked Write 9 Done Sent Done false
18917 5793 Acked Read 8 Done Sent Done false
18918 5794 Acked Read 1 Done Sent Done false
18919 5795 Acked Write 6 Done Sent Done false
18920 5796 Acked Read 10 Done Sent Done false
18921 5797 Acked Write 1 Done Sent Done false
18922 5798 Acked Write 6 Done Sent Done false
18923 5799 Acked Read 1 Done Sent Done false
18924 5800 Acked Write 2 Done Sent Done false
18925 5801 Acked Flush 0 Done Sent Done false
18926 5802 Acked Write 6 Done Sent Done false
18927 5803 Acked Write 8 Done Sent Done false
18928 5804 Acked Read 10 Done Sent Done false
18929 5805 Acked Read 4 Done Sent Done false
18930 5806 Acked Read 5 Done Sent Done false
18931 5807 Acked Read 6 Done Sent Done false
18932 5808 Acked Read 7 Done Sent Done false
18933 5809 Acked Read 5 Done Sent Done false
18934 5810 Acked Flush 0 Done Sent Done false
18935 5811 Acked Read 5 Done Sent Done false
18936 5812 Acked Read 4 Done Sent Done false
18937 5813 Acked Read 1 Done Sent Done false
18938 5814 Acked Read 4 Done Sent Done false
18939 5815 Acked Write 9 Done Sent Done false
18940 5816 Acked Flush 0 Done Sent Done false
18941 5817 Acked Read 9 Done Sent Done false
18942 5818 Acked Read 1 Done Sent Done false
18943 5819 Acked Flush 0 Done Sent Done false
18944 5820 Acked Write 3 Done Sent Done false
18945 5821 Acked Read 4 Done Sent Done false
18946 5822 Acked Write 4 Done Sent Done false
18947 5823 Acked Read 3 Done Sent Done false
18948 5824 Acked Write 6 Done Sent Done false
18949 5825 Acked Flush 0 Done Sent Done false
18950 5826 Acked Flush 0 Done Sent Done false
18951 5827 Acked Flush 0 Done Sent Done false
18952 5828 Acked Write 7 Done Sent Done false
18953 5829 Acked Read 4 Done Sent Done false
18954 5830 Acked Write 3 Done Sent Done false
18955 5831 Acked Flush 0 Done Sent Done false
18956 STATES DS:0 DS:1 DS:2 TOTAL
18957 Sent 0 302 0 302
18958 Done 302 0 302 604
18959 Skipped 0 0 0 0
18960 Error 0 0 0 0
18961 Last Flush: 5831 5529 5831
18962 Downstairs last five completed: 5529 5528 5527 5526 5525
18963 Upstairs last five completed: 5831 5830 5829 5828 5827
18964 CLIENT: Up:0 ds:302 act:2
18965 [8820] got start action
18966 [8820] I am going to restart
18967 Starting downstairs at port 8820
18968 Make output file at "/tmp/dsc/downstairs-8820.txt"
18969 [8820][1] reports Starting
18970 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1216)
18971 [8820][1] reports Running
189722024-12-11T23:44:30.204ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189732024-12-11T23:44:30.204ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58498) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189742024-12-11T23:44:30.205ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189752024-12-11T23:44:30.205ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189762024-12-11T23:44:30.205ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189772024-12-11T23:44:30.205ZINFOcrucible: send last flush ID to this DS: 5529 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189782024-12-11T23:44:30.206ZINFOcrucible: Replied this last flush ID: 5529 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189792024-12-11T23:44:30.206ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189802024-12-11T23:44:30.206ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189812024-12-11T23:44:30.206ZINFOcrucible: [1] client re-new 302 jobs since flush 5529 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189822024-12-11T23:44:30.206ZINFOcrucible: [1] Replayed 302 jobs since flush: 5529 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
18983 ----------------------------------------------------------------
18984 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18985 Downstairs last five completed: 5831 5830 5829 5828 5827
18986 Upstairs last five completed: 5831 5830 5829 5828 5827
18987 CLIENT: Up:0 ds:0 act:3
18988 Replay: All jobs finished
18989 Main task has work to do, go find it
18990 got dsc Stop(2)
18991 stop 2
18992 Dec 11 23:44:30.449 INFO request completed, latency_us: 327, response_code: [8830] Got stop action so:false kr:false
18993 204, uri: /stop/cid/2, method: POST, req_id: 9879f201-10bb-4e13-8b09-097551ff990a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
189942024-12-11T23:44:30.449ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189952024-12-11T23:44:30.449ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189962024-12-11T23:44:30.449ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189972024-12-11T23:44:30.449ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189982024-12-11T23:44:30.450ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
189992024-12-11T23:44:30.450ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
19000 Dec 11 23:44:30.450 INFO request completed, latency_us: 183, response_code: 200, uri: /state/cid/2, method: GET, req_id: 69450e20-28a4-4aa7-b28a-f770082bebd2, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
19001 [8830] Exited with: ExitStatus(unix_wait_status(9))
19002 [8830][2] reports Exit
19003 Dec 11 23:44:34.452 INFO request completed, latency_us: 243, response_code: 200, uri: /state/cid/2, method: GET, req_id: fddbce37-fd53-4ce0-8195-a11c69a18781, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
19004 001/300 Write block 2732 len 8192 data: 5 7
19005 002/300 Write block 2932 len 36864 data: 7 7 7 6 7 7 6 5 4
19006 003/300 Read block 2581 len 20480
19007 004/300 Read block 310 len 36864
19008 005/300 Flush
19009 006/300 Read block 1442 len 36864
19010 007/300 Write block 1832 len 36864 data: 3 3 3 3 3 3 3 3 3
19011 008/300 Flush
19012 009/300 Read block 3335 len 24576
19013 010/300 Flush
19014 011/300 Read block 4675 len 16384
19015 012/300 Write block 2782 len 16384 data: 2 2 2 3
19016 013/300 Flush
19017 014/300 Read block 3798 len 4096
19018 015/300 Write block 720 len 16384 data: 3 3 4 5
19019 016/300 Write block 1920 len 8192 data: 3 3
19020 017/300 Read block 2952 len 28672
19021 018/300 Read block 3637 len 20480
19022 019/300 Read block 4934 len 32768
19023 020/300 Read block 3544 len 32768
19024 021/300 Read block 1837 len 40960
19025 022/300 Write block 1522 len 12288 data: 5 4 3
19026 023/300 Read block 2332 len 20480
19027 024/300 Read block 4660 len 24576
19028 025/300 Read block 4112 len 16384
19029 026/300 Read block 1388 len 36864
19030 027/300 Flush
19031 028/300 Read block 1543 len 24576
19032 029/300 Read block 3982 len 36864
19033 030/300 Write block 2084 len 12288 data: 2 2 3
19034 031/300 Read block 4919 len 20480
19035 032/300 Read block 4856 len 32768
19036 033/300 Read block 2690 len 16384
19037 034/300 Read block 1255 len 32768
19038 035/300 Read block 2553 len 28672
19039 036/300 Write block 2754 len 20480 data: 2 2 2 2 2
19040 037/300 Write block 4525 len 32768 data: 6 7 6 6 7 9 8 10
19041 038/300 Read block 4925 len 12288
19042 039/300 Flush
19043 040/300 Write block 1908 len 36864 data: 5 4 4 4 4 5 4 4 4
19044 041/300 Read block 1951 len 16384
19045 042/300 Write block 2007 len 32768 data: 3 3 3 4 3 3 3 3
19046 043/300 Write block 2593 len 8192 data: 3 4
19047 044/300 Read block 1226 len 20480
19048 045/300 Write block 1117 len 28672 data: 5 5 5 5 4 2 2
19049 046/300 Read block 3164 len 4096
19050 047/300 Write block 1860 len 24576 data: 5 5 5 4 3 3
19051 048/300 Read block 1288 len 32768
19052 049/300 Write block 4660 len 40960 data: 3 3 3 2 2 2 3 4 5 5
19053 050/300 Read block 2420 len 36864
19054 051/300 Flush
19055 052/300 Read block 1250 len 16384
19056 053/300 Write block 1978 len 32768 data: 5 5 5 7 7 6 8 7
19057 054/300 Write block 686 len 12288 data: 4 4 6
19058 055/300 Read block 291 len 32768
19059 056/300 Write block 2105 len 8192 data: 5 5
19060 057/300 Write block 4099 len 40960 data: 3 2 3 3 5 5 4 4 4 4
19061 058/300 Write block 4210 len 4096 data: 4
19062 059/300 Read block 2313 len 12288
19063 060/300 Write block 635 len 20480 data: 2 2 4 5 6
19064 061/300 Read block 2706 len 24576
19065 062/300 Read block 2395 len 40960
19066 063/300 Write block 3739 len 16384 data: 3 4 4 5
19067 064/300 Write block 2338 len 40960 data: 4 5 5 4 4 3 3 3 5 5
19068 065/300 Write block 1417 len 8192 data: 4 3
19069 066/300 Write block 1779 len 12288 data: 2 3 4
19070 067/300 Write block 1084 len 20480 data: 5 6 6 6 6
19071 068/300 Read block 1421 len 36864
19072 069/300 Write block 4983 len 4096 data: 6
19073 070/300 Read block 1352 len 20480
19074 071/300 Write block 4655 len 28672 data: 3 3 4 3 3 4 4
19075 072/300 Write block 3852 len 28672 data: 4 5 5 4 6 7 7
19076 073/300 Read block 4088 len 40960
19077 074/300 Write block 1661 len 16384 data: 3 3 2 2
19078 075/300 Write block 3548 len 28672 data: 4 5 4 4 4 4 4
19079 076/300 Write block 4967 len 28672 data: 2 5 5 5 6 6 6
19080 077/300 Read block 3456 len 28672
19081 078/300 Read block 4337 len 12288
19082 079/300 Write block 1968 len 40960 data: 3 3 3 5 4 4 4 5 5 5
19083 080/300 Write block 4248 len 36864 data: 3 3 3 3 3 5 5 5 6
19084 081/300 Read block 4601 len 36864
19085 082/300 Read block 3186 len 12288
19086 083/300 Read block 4465 len 36864
19087 084/300 Write block 2959 len 20480 data: 5 5 6 6 6
19088 085/300 Read block 3608 len 4096
19089 086/300 Write block 2617 len 16384 data: 5 4 3 3
19090 087/300 Read block 2643 len 28672
19091 088/300 Read block 2261 len 4096
19092 089/300 Read block 15 len 36864
19093 090/300 Flush
19094 091/300 Write block 1366 len 12288 data: 2 2 4
19095 092/300 Flush
19096 093/300 Write block 2051 len 24576 data: 4 5 6 4 3 3
19097 094/300 Flush
19098 095/300 Read block 386 len 4096
19099 096/300 Write block 2962 len 28672 data: 7 7 5 5 5 3 3
19100 097/300 Write block 3487 len 20480 data: 5 5 5 4 3
19101 098/300 Write block 96 len 40960 data: 4 4 3 5 5 5 5 5 6 5
19102 099/300 Read block 2730 len 12288
19103 100/300 Read block 3199 len 28672
19104 101/300 Read block 1207 len 36864
19105 102/300 Read block 3417 len 24576
19106 103/300 Read block 2440 len 8192
19107 104/300 Write block 4631 len 4096 data: 3
19108 105/300 Write block 3811 len 32768 data: 4 5 4 6 5 6 6 6
19109 106/300 Read block 4567 len 16384
19110 107/300 Read block 3644 len 24576
19111 108/300 Read block 4320 len 20480
19112 109/300 Flush
19113 110/300 Read block 2708 len 28672
19114 111/300 Write block 3539 len 28672 data: 3 5 6 6 6 5 5
19115 112/300 Flush
19116 113/300 Write block 3306 len 16384 data: 4 4 4 3
19117 114/300 Write block 4283 len 12288 data: 5 5 4
19118 115/300 Write block 4590 len 8192 data: 4 4
19119 116/300 Read block 1425 len 36864
19120 117/300 Write block 1076 len 28672 data: 3 4 4 2 2 3 4
19121 118/300 Flush
19122 119/300 Write block 2483 len 36864 data: 4 4 4 3 3 4 4 3 3
19123 120/300 Write block 2492 len 32768 data: 3 3 3 3 3 3 4 4
19124 121/300 Write block 4282 len 40960 data: 4 6 6 5 4 3 5 4 3 3
19125 122/300 Read block 2321 len 12288
19126 123/300 Read block 3756 len 28672
19127 124/300 Write block 1466 len 32768 data: 4 4 5 5 6 4 2 2
19128 125/300 Write block 607 len 24576 data: 3 3 3 3 4 4
19129 126/300 Read block 4306 len 28672
19130 127/300 Read block 4210 len 20480
19131 128/300 Write block 4801 len 36864 data: 4 5 5 4 3 3 4 4 4
19132 129/300 Read block 2096 len 24576
19133 130/300 Read block 3561 len 16384
19134 131/300 Read block 3528 len 4096
19135 132/300 Write block 3317 len 28672 data: 4 2 3 3 4 4 4
19136 133/300 Write block 4555 len 28672 data: 2 2 4 4 4 4 5
19137 134/300 Write block 4049 len 36864 data: 4 4 4 4 4 4 3 3 4
19138 135/300 Read block 304 len 24576
19139 136/300 Read block 1213 len 12288
19140 137/300 Read block 2521 len 8192
19141 138/300 Write block 2900 len 8192 data: 7 8
19142 139/300 Read block 3086 len 32768
19143 140/300 Read block 1244 len 40960
19144 141/300 Write block 4124 len 28672 data: 6 6 5 4 4 3 3
19145 142/300 Read block 4001 len 24576
19146 143/300 Write block 3781 len 8192 data: 4 4
19147 144/300 Read block 382 len 40960
19148 145/300 Read block 2060 len 36864
19149 146/300 Write block 2962 len 16384 data: 8 8 6 6
19150 147/300 Write block 4043 len 28672 data: 7 7 6 6 7 4 5
19151 148/300 Write block 4591 len 20480 data: 5 4 4 5 5
19152 149/300 Write block 2355 len 40960 data: 4 4 4 3 3 3 3 3 3 4
19153 150/300 Read block 350 len 24576
19154 151/300 Write block 1616 len 28672 data: 2 2 2 3 5 6 6
19155 152/300 Write block 1671 len 20480 data: 4 4 3 2 2
19156 153/300 Read block 2011 len 24576
19157 154/300 Write block 3145 len 40960 data: 4 4 4 4 4 5 5 4 5 4
19158 155/300 Write block 2065 len 24576 data: 5 6 6 6 6 5
19159 156/300 Flush
19160 157/300 Write block 982 len 36864 data: 4 5 6 6 7 6 5 4 4
19161 158/300 Read block 1820 len 4096
19162 159/300 Write block 2835 len 40960 data: 7 5 6 6 5 5 5 5 3 5
19163 160/300 Write block 226 len 12288 data: 3 4 4
19164 161/300 Write block 4436 len 32768 data: 5 5 6 6 6 7 6 6
19165 162/300 Read block 1600 len 24576
19166 163/300 Flush
19167 164/300 Flush
19168 165/300 Write block 3838 len 28672 data: 2 2 2 3 2 2 3
19169 166/300 Read block 3356 len 8192
19170 167/300 Write block 3582 len 20480 data: 6 5 5 6 5
19171 168/300 Read block 679 len 36864
19172 169/300 Write block 381 len 4096 data: 5
19173 170/300 Write block 3825 len 40960 data: 5 6 6 8 9 9 8 7 5 5
19174 171/300 Write block 4097 len 16384 data: 4 4 4 3
19175 172/300 Write block 2210 len 36864 data: 5 6 6 5 5 5 6 5 4
19176 173/300 Write block 4306 len 4096 data: 2
19177 174/300 Write block 3954 len 36864 data: 3 3 3 2 4 5 8 8 9
19178 175/300 Read block 3259 len 40960
19179 176/300 Read block 4493 len 16384
19180 177/300 Write block 3934 len 40960 data: 3 4 4 4 4 5 4 5 5 6
19181 178/300 Read block 2839 len 20480
19182 179/300 Flush
19183 180/300 Read block 235 len 24576
19184 181/300 Read block 1615 len 12288
19185 182/300 Write block 368 len 4096 data: 2
19186 183/300 Write block 2225 len 24576 data: 4 5 4 4 4 4
19187 184/300 Write block 1085 len 36864 data: 7 7 7 7 6 6 7 8 6
19188 185/300 Write block 1552 len 24576 data: 2 2 2 2 2 2
19189 186/300 Write block 4470 len 8192 data: 8 8
19190 187/300 Flush
19191 188/300 Read block 3893 len 36864
19192 189/300 Flush
19193 190/300 Write block 1781 len 16384 data: 5 4 4 4
19194 191/300 Write block 3956 len 40960 data: 4 3 5 6 9 9 10 8 8 7
19195 192/300 Read block 110 len 16384
19196 193/300 Read block 2936 len 36864
19197 194/300 Write block 3483 len 40960 data: 3 3 3 4 6 6 6 5 4 3
19198 195/300 Write block 4583 len 4096 data: 5
19199 196/300 Write block 2546 len 8192 data: 6 6
19200 197/300 Read block 445 len 32768
19201 198/300 Read block 1369 len 20480
19202 199/300 Flush
19203 200/300 Read block 1432 len 20480
19204 201/300 Write block 2835 len 12288 data: 8 6 7
19205 202/300 Write block 3558 len 8192 data: 4 2
19206 203/300 Read block 3145 len 24576
19207 204/300 Flush
19208 205/300 Write block 403 len 40960 data: 3 3 3 4 4 6 7 8 7 6
19209 206/300 Write block 1704 len 24576 data: 3 4 2 2 2 3
19210 207/300 Read block 4666 len 8192
19211 208/300 Read block 4398 len 12288
19212 209/300 Write block 4240 len 20480 data: 6 5 3 3 3
19213 210/300 Read block 1085 len 4096
19214 211/300 Read block 2163 len 4096
19215 212/300 Write block 4765 len 20480 data: 3 3 3 3 3
19216 213/300 Write block 980 len 8192 data: 4 4
19217 214/300 Read block 1943 len 16384
19218 215/300 Read block 1497 len 8192
19219 216/300 Write block 1499 len 40960 data: 5 6 5 5 5 5 4 4 4 4
19220 217/300 Read block 648 len 40960
19221 218/300 Write block 169 len 12288 data: 2 2 2
19222 219/300 Read block 4636 len 24576
19223 220/300 Write block 3077 len 36864 data: 2 3 2 2 3 3 4 4 4
19224 221/300 Flush
19225 222/300 Read block 3921 len 4096
19226 223/300 Read block 75 len 28672
19227 224/300 Read block 4436 len 28672
19228 225/300 Read block 1901 len 4096
19229 226/300 Read block 736 len 32768
19230 227/300 Write block 771 len 8192 data: 6 5
19231 228/300 Read block 4020 len 20480
19232 229/300 Read block 4201 len 4096
19233 230/300 Read block 227 len 40960
19234 231/300 Read block 3953 len 28672
19235 232/300 Read block 3511 len 24576
19236 233/300 Read block 2999 len 28672
19237 234/300 Read block 1321 len 28672
19238 235/300 Write block 1628 len 8192 data: 5 6
19239 236/300 Read block 3659 len 32768
19240 237/300 Flush
19241 238/300 Write block 2449 len 12288 data: 4 3 3
19242 239/300 Read block 4614 len 36864
19243 240/300 Write block 1506 len 4096 data: 5
19244 241/300 Write block 781 len 40960 data: 3 3 3 4 4 4 3 2 2 2
19245 242/300 Write block 2895 len 24576 data: 7 6 7 9 8 8
19246 243/300 Read block 4523 len 8192
19247 244/300 Write block 3756 len 4096 data: 3
19248 245/300 Flush
19249 246/300 Write block 2309 len 12288 data: 5 5 5
19250 247/300 Write block 4750 len 36864 data: 3 5 4 5 4 4 2 2 2
19251 248/300 Read block 421 len 40960
19252 249/300 Read block 165 len 28672
19253 250/300 Read block 2888 len 20480
19254 251/300 Write block 1236 len 36864 data: 4 3 3 3 3 2 3 3 3
19255 252/300 Read block 1715 len 20480
19256 253/300 Write block 4296 len 40960 data: 5 5 5 4 3 3 2 2 2 2
19257 254/300 Read block 3605 len 4096
19258 255/300 Flush
19259 256/300 Write block 2261 len 16384 data: 3 3 2 3
19260 257/300 Write block 530 len 16384 data: 2 3 3 3
19261 258/300 Write block 2987 len 8192 data: 3 3
19262 259/300 Read block 688 len 28672
19263 260/300 Read block 431 len 24576
19264 261/300 Write block 1269 len 8192 data: 4 6
19265 262/300 Flush
19266 263/300 Write block 2949 len 28672 data: 4 4 4 4 4 3 3
19267 264/300 Read block 3330 len 4096
19268 265/300 Write block 3807 len 16384 data: 3 4 4 4
19269 266/300 Write block 2804 len 12288 data: 8 6 7
19270 267/300 Flush
19271 268/300 Read block 2061 len 36864
19272 269/300 Write block 803 len 40960 data: 5 5 5 4 4 5 4 4 6 5
19273 270/300 Write block 4729 len 4096 data: 3
19274 271/300 Read block 2810 len 24576
19275 272/300 Flush
19276 273/300 Read block 4937 len 8192
19277 274/300 Read block 2521 len 24576
19278 275/300 Read block 4526 len 36864
19279 276/300 Read block 1184 len 16384
19280 277/300 Read block 4477 len 28672
19281 278/300 Write block 2618 len 16384 data: 5 4 4 4
19282 279/300 Read block 3246 len 32768
19283 280/300 Flush
19284 281/300 Flush
19285 282/300 Read block 1802 len 16384
19286 283/300 Write block 2610 len 36864 data: 8 7 5 4 4 4 5 6 6
19287 284/300 Write block 3346 len 12288 data: 4 3 3
19288 285/300 Write block 2583 len 24576 data: 5 4 3 3 2 4
19289 286/300 Read block 1448 len 32768
19290 287/300 Write block 3320 len 40960 data: 4 5 5 5 3 3 5 6 6 6
19291 288/300 Write block 1499 len 24576 data: 6 7 6 6 6 6
19292 289/300 Flush
19293 290/300 Write block 3329 len 24576 data: 7 5 4 3 4 3
19294 291/300 Read block 1978 len 40960
19295 292/300 Write block 1441 len 8192 data: 7 6
19296 293/300 Flush
19297 294/300 Read block 1233 len 20480
19298 295/300 Read block 952 len 32768
19299 296/300 Read block 4985 len 8192
19300 297/300 Write block 2348 len 16384 data: 4 4 4 3
19301 298/300 Write block 1632 len 12288 data: 5 5 3
19302 299/300 Read block 2188 len 40960
19303 300/300 Read block 2291 len 8192
19304 301/300 Read block 3549 len 24576
19305 Main task has work to do, go find it
19306 got dsc Start(2)
19307 start 2
19308 Dec 11 23:44:34.689 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/2, method: POST, req_id: 0dda72f6-6c53-4897-8bcc-f634e1c504f2, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
19309 Replay: started 2, returned:Ok(())
19310 ----------------------------------------------------------------
19311 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19312 States: Active Active Offline
19313 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19314 5832 Acked Write 2 Done Done Sent false
19315 5833 Acked Write 9 Done Done Sent false
19316 5834 Acked Read 5 Done Done Sent false
19317 5835 Acked Read 9 Done Done Sent false
19318 5836 Acked Flush 0 Done Done Sent false
19319 5837 Acked Read 9 Done Done Sent false
19320 5838 Acked Write 9 Done Done Sent false
19321 5839 Acked Flush 0 Done Done Sent false
19322 5840 Acked Read 6 Done Done Sent false
19323 5841 Acked Flush 0 Done Done Sent false
19324 5842 Acked Read 4 Done Done Sent false
19325 5843 Acked Write 4 Done Done Sent false
19326 5844 Acked Flush 0 Done Done Sent false
19327 5845 Acked Read 1 Done Done Sent false
19328 5846 Acked Write 4 Done Done Sent false
19329 5847 Acked Write 2 Done Done Sent false
19330 5848 Acked Read 7 Done Done Sent false
19331 5849 Acked Read 5 Done Done Sent false
19332 5850 Acked Read 8 Done Done Sent false
19333 5851 Acked Read 8 Done Done Sent false
19334 5852 Acked Read 10 Done Done Sent false
19335 5853 Acked Write 3 Done Done Sent false
19336 5854 Acked Read 5 Done Done Sent false
19337 5855 Acked Read 6 Done Done Sent false
19338 5856 Acked Read 4 Done Done Sent false
19339 5857 Acked Read 9 Done Done Sent false
19340 5858 Acked Flush 0 Done Done Sent false
19341 5859 Acked Read 6 Done Done Sent false
19342 5860 Acked Read 9 Done Done Sent false
19343 5861 Acked Write 3 Done Done Sent false
19344 5862 Acked Read 5 Done Done Sent false
19345 5863 Acked Read 8 Done Done Sent false
19346 5864 Acked Read 4 Done Done Sent false
19347 5865 Acked Read 8 Done Done Sent false
19348 5866 Acked Read 7 Done Done Sent false
19349 5867 Acked Write 5 Done Done Sent false
19350 5868 Acked Write 8 Done Done Sent false
19351 5869 Acked Read 3 Done Done Sent false
19352 5870 Acked Flush 0 Done Done Sent false
19353 5871 Acked Write 9 Done Done Sent false
19354 5872 Acked Read 4 Done Done Sent false
19355 5873 Acked Write 8 Done Done Sent false
19356 5874 Acked Write 2 Done Done Sent false
19357 5875 Acked Read 5 Done Done Sent false
19358 5876 Acked Write 7 Done Done Sent false
19359 5877 Acked Read 1 Done Done Sent false
19360 5878 Acked Write 6 Done Done Sent false
19361 5879 Acked Read 8 Done Done Sent false
19362 5880 Acked Write 10 Done Done Sent false
19363 5881 Acked Read 9 Done Done Sent false
19364 5882 Acked Flush 0 Done Done Sent false
19365 5883 Acked Read 4 Done Done Sent false
19366 5884 Acked Write 8 Done Done Sent false
19367 5885 Acked Write 3 Done Done Sent false
19368 5886 Acked Read 8 Done Done Sent false
19369 5887 Acked Write 2 Done Done Sent false
19370 5888 Acked Write 10 Done Done Sent false
19371 5889 Acked Write 1 Done Done Sent false
19372 5890 Acked Read 3 Done Done Sent false
19373 5891 Acked Write 5 Done Done Sent false
19374 5892 Acked Read 6 Done Done Sent false
19375 5893 Acked Read 10 Done Done Sent false
19376 5894 Acked Write 4 Done Done Sent false
19377 5895 Acked Write 10 Done Done Sent false
19378 5896 Acked Write 2 Done Done Sent false
19379 5897 Acked Write 3 Done Done Sent false
19380 5898 Acked Write 5 Done Done Sent false
19381 5899 Acked Read 9 Done Done Sent false
19382 5900 Acked Write 1 Done Done Sent false
19383 5901 Acked Read 5 Done Done Sent false
19384 5902 Acked Write 7 Done Done Sent false
19385 5903 Acked Write 7 Done Done Sent false
19386 5904 Acked Read 10 Done Done Sent false
19387 5905 Acked Write 4 Done Done Sent false
19388 5906 Acked Write 7 Done Done Sent false
19389 5907 Acked Write 7 Done Done Sent false
19390 5908 Acked Read 7 Done Done Sent false
19391 5909 Acked Read 3 Done Done Sent false
19392 5910 Acked Write 10 Done Done Sent false
19393 5911 Acked Write 9 Done Done Sent false
19394 5912 Acked Read 9 Done Done Sent false
19395 5913 Acked Read 3 Done Done Sent false
19396 5914 Acked Read 9 Done Done Sent false
19397 5915 Acked Write 5 Done Done Sent false
19398 5916 Acked Read 1 Done Done Sent false
19399 5917 Acked Write 4 Done Done Sent false
19400 5918 Acked Read 7 Done Done Sent false
19401 5919 Acked Read 1 Done Done Sent false
19402 5920 Acked Read 9 Done Done Sent false
19403 5921 Acked Flush 0 Done Done Sent false
19404 5922 Acked Write 3 Done Done Sent false
19405 5923 Acked Flush 0 Done Done Sent false
19406 5924 Acked Write 6 Done Done Sent false
19407 5925 Acked Flush 0 Done Done Sent false
19408 5926 Acked Read 1 Done Done Sent false
19409 5927 Acked Write 7 Done Done Sent false
19410 5928 Acked Write 5 Done Done Sent false
19411 5929 Acked Write 10 Done Done Sent false
19412 5930 Acked Read 3 Done Done Sent false
19413 5931 Acked Read 7 Done Done Sent false
19414 5932 Acked Read 9 Done Done Sent false
19415 5933 Acked Read 6 Done Done Sent false
19416 5934 Acked Read 2 Done Done Sent false
19417 5935 Acked Write 1 Done Done Sent false
19418 5936 Acked Write 8 Done Done Sent false
19419 5937 Acked Read 4 Done Done Sent false
19420 5938 Acked Read 6 Done Done Sent false
19421 5939 Acked Read 5 Done Done Sent false
19422 5940 Acked Flush 0 Done Done Sent false
19423 5941 Acked Read 7 Done Done Sent false
19424 5942 Acked Write 7 Done Done Sent false
19425 5943 Acked Flush 0 Done Done Sent false
19426 5944 Acked Write 4 Done Done Sent false
19427 5945 Acked Write 3 Done Done Sent false
19428 5946 Acked Write 2 Done Done Sent false
19429 5947 Acked Read 9 Done Done Sent false
19430 5948 Acked Write 7 Done Done Sent false
19431 5949 Acked Flush 0 Done Done Sent false
19432 5950 Acked Write 9 Done Done Sent false
19433 5951 Acked Write 8 Done Done Sent false
19434 5952 Acked Write 10 Done Done Sent false
19435 5953 Acked Read 3 Done Done Sent false
19436 5954 Acked Read 7 Done Done Sent false
19437 5955 Acked Write 8 Done Done Sent false
19438 5956 Acked Write 6 Done Done Sent false
19439 5957 Acked Read 7 Done Done Sent false
19440 5958 Acked Read 5 Done Done Sent false
19441 5959 Acked Write 9 Done Done Sent false
19442 5960 Acked Read 6 Done Done Sent false
19443 5961 Acked Read 4 Done Done Sent false
19444 5962 Acked Read 1 Done Done Sent false
19445 5963 Acked Write 7 Done Done Sent false
19446 5964 Acked Write 7 Done Done Sent false
19447 5965 Acked Write 9 Done Done Sent false
19448 5966 Acked Read 6 Done Done Sent false
19449 5967 Acked Read 3 Done Done Sent false
19450 5968 Acked Read 2 Done Done Sent false
19451 5969 Acked Write 2 Done Done Sent false
19452 5970 Acked Read 8 Done Done Sent false
19453 5971 Acked Read 10 Done Done Sent false
19454 5972 Acked Write 7 Done Done Sent false
19455 5973 Acked Read 6 Done Done Sent false
19456 5974 Acked Write 2 Done Done Sent false
19457 5975 Acked Read 10 Done Done Sent false
19458 5976 Acked Read 9 Done Done Sent false
19459 5977 Acked Write 4 Done Done Sent false
19460 5978 Acked Write 7 Done Done Sent false
19461 5979 Acked Write 5 Done Done Sent false
19462 5980 Acked Write 10 Done Done Sent false
19463 5981 Acked Read 6 Done Done Sent false
19464 5982 Acked Write 7 Done Done Sent false
19465 5983 Acked Write 5 Done Done Sent false
19466 5984 Acked Read 6 Done Done Sent false
19467 5985 Acked Write 10 Done Done Sent false
19468 5986 Acked Write 6 Done Done Sent false
19469 5987 Acked Flush 0 Done Done Sent false
19470 5988 Acked Write 9 Done Done Sent false
19471 5989 Acked Read 1 Done Done Sent false
19472 5990 Acked Write 10 Done Done Sent false
19473 5991 Acked Write 3 Done Done Sent false
19474 5992 Acked Write 8 Done Done Sent false
19475 5993 Acked Read 6 Done Done Sent false
19476 5994 Acked Flush 0 Done Done Sent false
19477 5995 Acked Flush 0 Done Done Sent false
19478 5996 Acked Write 7 Done Done Sent false
19479 5997 Acked Read 2 Done Done Sent false
19480 5998 Acked Write 5 Done Done Sent false
19481 5999 Acked Read 9 Done Done Sent false
19482 6000 Acked Write 1 Done Done Sent false
19483 6001 Acked Write 10 Done Done Sent false
19484 6002 Acked Write 4 Done Done Sent false
19485 6003 Acked Write 9 Done Done Sent false
19486 6004 Acked Write 1 Done Done Sent false
19487 6005 Acked Write 9 Done Done Sent false
19488 6006 Acked Read 10 Done Done Sent false
19489 6007 Acked Read 4 Done Done Sent false
19490 6008 Acked Write 10 Done Done Sent false
19491 6009 Acked Read 5 Done Done Sent false
19492 6010 Acked Flush 0 Done Done Sent false
19493 6011 Acked Read 6 Done Done Sent false
19494 6012 Acked Read 3 Done Done Sent false
19495 6013 Acked Write 1 Done Done Sent false
19496 6014 Acked Write 6 Done Done Sent false
19497 6015 Acked Write 9 Done Done Sent false
19498 6016 Acked Write 6 Done Done Sent false
19499 6017 Acked Write 2 Done Done Sent false
19500 6018 Acked Flush 0 Done Done Sent false
19501 6019 Acked Read 9 Done Done Sent false
19502 6020 Acked Flush 0 Done Done Sent false
19503 6021 Acked Write 4 Done Done Sent false
19504 6022 Acked Write 10 Done Done Sent false
19505 6023 Acked Read 4 Done Done Sent false
19506 6024 Acked Read 9 Done Done Sent false
19507 6025 Acked Write 10 Done Done Sent false
19508 6026 Acked Write 1 Done Done Sent false
19509 6027 Acked Write 2 Done Done Sent false
19510 6028 Acked Read 8 Done Done Sent false
19511 6029 Acked Read 5 Done Done Sent false
19512 6030 Acked Flush 0 Done Done Sent false
19513 6031 Acked Read 5 Done Done Sent false
19514 6032 Acked Write 3 Done Done Sent false
19515 6033 Acked Write 2 Done Done Sent false
19516 6034 Acked Read 6 Done Done Sent false
19517 6035 Acked Flush 0 Done Done Sent false
19518 6036 Acked Write 10 Done Done Sent false
19519 6037 Acked Write 6 Done Done Sent false
19520 6038 Acked Read 2 Done Done Sent false
19521 6039 Acked Read 3 Done Done Sent false
19522 6040 Acked Write 5 Done Done Sent false
19523 6041 Acked Read 1 Done Done Sent false
19524 6042 Acked Read 1 Done Done Sent false
19525 6043 Acked Write 5 Done Done Sent false
19526 6044 Acked Write 2 Done Done Sent false
19527 6045 Acked Read 4 Done Done Sent false
19528 6046 Acked Read 2 Done Done Sent false
19529 6047 Acked Write 10 Done Done Sent false
19530 6048 Acked Read 10 Done Done Sent false
19531 6049 Acked Write 3 Done Done Sent false
19532 6050 Acked Read 6 Done Done Sent false
19533 6051 Acked Write 9 Done Done Sent false
19534 6052 Acked Flush 0 Done Done Sent false
19535 6053 Acked Read 1 Done Done Sent false
19536 6054 Acked Read 7 Done Done Sent false
19537 6055 Acked Read 7 Done Done Sent false
19538 6056 Acked Read 1 Done Done Sent false
19539 6057 Acked Read 8 Done Done Sent false
19540 6058 Acked Write 2 Done Done Sent false
19541 6059 Acked Read 5 Done Done Sent false
19542 6060 Acked Read 1 Done Done Sent false
19543 6061 Acked Read 10 Done Done Sent false
19544 6062 Acked Read 7 Done Done Sent false
19545 6063 Acked Read 6 Done Done Sent false
19546 6064 Acked Read 7 Done Done Sent false
19547 6065 Acked Read 7 Done Done Sent false
19548 6066 Acked Write 2 Done Done Sent false
19549 6067 Acked Read 8 Done Done Sent false
19550 6068 Acked Flush 0 Done Done Sent false
19551 6069 Acked Write 3 Done Done Sent false
19552 6070 Acked Read 9 Done Done Sent false
19553 6071 Acked Write 1 Done Done Sent false
19554 6072 Acked Write 10 Done Done Sent false
19555 6073 Acked Write 6 Done Done Sent false
19556 6074 Acked Read 2 Done Done Sent false
19557 6075 Acked Write 1 Done Done Sent false
19558 6076 Acked Flush 0 Done Done Sent false
19559 6077 Acked Write 3 Done Done Sent false
19560 6078 Acked Write 9 Done Done Sent false
19561 6079 Acked Read 10 Done Done Sent false
19562 6080 Acked Read 7 Done Done Sent false
19563 6081 Acked Read 5 Done Done Sent false
19564 6082 Acked Write 9 Done Done Sent false
19565 6083 Acked Read 5 Done Done Sent false
19566 6084 Acked Write 10 Done Done Sent false
19567 6085 Acked Read 1 Done Done Sent false
19568 6086 Acked Flush 0 Done Done Sent false
19569 6087 Acked Write 4 Done Done Sent false
19570 6088 Acked Write 4 Done Done Sent false
19571 6089 Acked Write 2 Done Done Sent false
19572 6090 Acked Read 7 Done Done Sent false
19573 6091 Acked Read 6 Done Done Sent false
19574 6092 Acked Write 2 Done Done Sent false
19575 6093 Acked Flush 0 Done Done Sent false
19576 6094 Acked Write 7 Done Done Sent false
19577 6095 Acked Read 1 Done Done Sent false
19578 6096 Acked Write 4 Done Done Sent false
19579 6097 Acked Write 3 Done Done Sent false
19580 6098 Acked Flush 0 Done Done Sent false
19581 6099 Acked Read 9 Done Done Sent false
19582 6100 Acked Write 10 Done Done Sent false
19583 6101 Acked Write 1 Done Done Sent false
19584 6102 Acked Read 6 Done Done Sent false
19585 6103 Acked Flush 0 Done Done Sent false
19586 6104 Acked Read 2 Done Done Sent false
19587 6105 Acked Read 6 Done Done Sent false
19588 6106 Acked Read 9 Done Done Sent false
19589 6107 Acked Read 4 Done Done Sent false
19590 6108 Acked Read 7 Done Done Sent false
19591 6109 Acked Write 4 Done Done Sent false
19592 6110 Acked Read 8 Done Done Sent false
19593 6111 Acked Flush 0 Done Done Sent false
19594 6112 Acked Flush 0 Done Done Sent false
19595 6113 Acked Read 4 Done Done Sent false
19596 6114 Acked Write 9 Done Done Sent false
19597 6115 Acked Write 3 Done Done Sent false
19598 6116 Acked Write 6 Done Done Sent false
19599 6117 Acked Read 8 Done Done Sent false
19600 6118 Acked Write 10 Done Done Sent false
19601 6119 Acked Write 6 Done Done Sent false
19602 6120 Acked Flush 0 Done Done Sent false
19603 6121 Acked Write 6 Done Done Sent false
19604 6122 Acked Read 10 Done Done Sent false
19605 6123 Acked Write 2 Done Done Sent false
19606 6124 Acked Flush 0 Done Done Sent false
19607 6125 Acked Read 5 Done Done Sent false
19608 6126 Acked Read 8 Done Done Sent false
19609 6127 Acked Read 2 Done Done Sent false
19610 6128 Acked Write 4 Done Done Sent false
19611 6129 Acked Write 3 Done Done Sent false
19612 6130 Acked Read 10 Done Done Sent false
19613 6131 Acked Read 2 Done Done Sent false
19614 6132 Acked Read 6 Done Done Sent false
19615 STATES DS:0 DS:1 DS:2 TOTAL
19616 Sent 0 0 301 301
19617 Done 301 301 0 602
19618 Skipped 0 0 0 0
19619 Error 0 0 0 0
19620 Last Flush: 6124 6124 5831
19621 Downstairs last five completed: 5831 5830 5829 5828 5827
19622 Upstairs last five completed: 6132 6131 6130 6129 6128
19623 CLIENT: Up:0 ds:301 act:2
19624 ----------------------------------------------------------------
19625 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19626 States: Active Active Offline
19627 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19628 5832 Acked Write 2 Done Done Sent false
19629 5833 Acked Write 9 Done Done Sent false
19630 5834 Acked Read 5 Done Done Sent false
19631 5835 Acked Read 9 Done Done Sent false
19632 5836 Acked Flush 0 Done Done Sent false
19633 5837 Acked Read 9 Done Done Sent false
19634 5838 Acked Write 9 Done Done Sent false
19635 5839 Acked Flush 0 Done Done Sent false
19636 5840 Acked Read 6 Done Done Sent false
19637 5841 Acked Flush 0 Done Done Sent false
19638 5842 Acked Read 4 Done Done Sent false
19639 5843 Acked Write 4 Done Done Sent false
19640 5844 Acked Flush 0 Done Done Sent false
19641 5845 Acked Read 1 Done Done Sent false
19642 5846 Acked Write 4 Done Done Sent false
19643 5847 Acked Write 2 Done Done Sent false
19644 5848 Acked Read 7 Done Done Sent false
19645 5849 Acked Read 5 Done Done Sent false
19646 5850 Acked Read 8 Done Done Sent false
19647 5851 Acked Read 8 Done Done Sent false
19648 5852 Acked Read 10 Done Done Sent false
19649 5853 Acked Write 3 Done Done Sent false
19650 5854 Acked Read 5 Done Done Sent false
19651 5855 Acked Read 6 Done Done Sent false
19652 5856 Acked Read 4 Done Done Sent false
19653 5857 Acked Read 9 Done Done Sent false
19654 5858 Acked Flush 0 Done Done Sent false
19655 5859 Acked Read 6 Done Done Sent false
19656 5860 Acked Read 9 Done Done Sent false
19657 5861 Acked Write 3 Done Done Sent false
19658 5862 Acked Read 5 Done Done Sent false
19659 5863 Acked Read 8 Done Done Sent false
19660 5864 Acked Read 4 Done Done Sent false
19661 5865 Acked Read 8 Done Done Sent false
19662 5866 Acked Read 7 Done Done Sent false
19663 5867 Acked Write 5 Done Done Sent false
19664 5868 Acked Write 8 Done Done Sent false
19665 5869 Acked Read 3 Done Done Sent false
19666 5870 Acked Flush 0 Done Done Sent false
19667 5871 Acked Write 9 Done Done Sent false
19668 5872 Acked Read 4 Done Done Sent false
19669 5873 Acked Write 8 Done Done Sent false
19670 5874 Acked Write 2 Done Done Sent false
19671 5875 Acked Read 5 Done Done Sent false
19672 5876 Acked Write 7 Done Done Sent false
19673 5877 Acked Read 1 Done Done Sent false
19674 5878 Acked Write 6 Done Done Sent false
19675 5879 Acked Read 8 Done Done Sent false
19676 5880 Acked Write 10 Done Done Sent false
19677 5881 Acked Read 9 Done Done Sent false
19678 5882 Acked Flush 0 Done Done Sent false
19679 5883 Acked Read 4 Done Done Sent false
19680 5884 Acked Write 8 Done Done Sent false
19681 5885 Acked Write 3 Done Done Sent false
19682 5886 Acked Read 8 Done Done Sent false
19683 5887 Acked Write 2 Done Done Sent false
19684 5888 Acked Write 10 Done Done Sent false
19685 5889 Acked Write 1 Done Done Sent false
19686 5890 Acked Read 3 Done Done Sent false
19687 5891 Acked Write 5 Done Done Sent false
19688 5892 Acked Read 6 Done Done Sent false
19689 5893 Acked Read 10 Done Done Sent false
19690 5894 Acked Write 4 Done Done Sent false
19691 5895 Acked Write 10 Done Done Sent false
19692 5896 Acked Write 2 Done Done Sent false
19693 5897 Acked Write 3 Done Done Sent false
19694 5898 Acked Write 5 Done Done Sent false
19695 5899 Acked Read 9 Done Done Sent false
19696 5900 Acked Write 1 Done Done Sent false
19697 5901 Acked Read 5 Done Done Sent false
19698 5902 Acked Write 7 Done Done Sent false
19699 5903 Acked Write 7 Done Done Sent false
19700 5904 Acked Read 10 Done Done Sent false
19701 5905 Acked Write 4 Done Done Sent false
19702 5906 Acked Write 7 Done Done Sent false
19703 5907 Acked Write 7 Done Done Sent false
19704 5908 Acked Read 7 Done Done Sent false
19705 5909 Acked Read 3 Done Done Sent false
19706 5910 Acked Write 10 Done Done Sent false
19707 5911 Acked Write 9 Done Done Sent false
19708 5912 Acked Read 9 Done Done Sent false
19709 5913 Acked Read 3 Done Done Sent false
19710 5914 Acked Read 9 Done Done Sent false
19711 5915 Acked Write 5 Done Done Sent false
19712 5916 Acked Read 1 Done Done Sent false
19713 5917 Acked Write 4 Done Done Sent false
19714 5918 Acked Read 7 Done Done Sent false
19715 5919 Acked Read 1 Done Done Sent false
19716 5920 Acked Read 9 Done Done Sent false
19717 5921 Acked Flush 0 Done Done Sent false
19718 5922 Acked Write 3 Done Done Sent false
19719 5923 Acked Flush 0 Done Done Sent false
19720 5924 Acked Write 6 Done Done Sent false
19721 5925 Acked Flush 0 Done Done Sent false
19722 5926 Acked Read 1 Done Done Sent false
19723 5927 Acked Write 7 Done Done Sent false
19724 5928 Acked Write 5 Done Done Sent false
19725 5929 Acked Write 10 Done Done Sent false
19726 5930 Acked Read 3 Done Done Sent false
19727 5931 Acked Read 7 Done Done Sent false
19728 5932 Acked Read 9 Done Done Sent false
19729 5933 Acked Read 6 Done Done Sent false
19730 5934 Acked Read 2 Done Done Sent false
19731 5935 Acked Write 1 Done Done Sent false
19732 5936 Acked Write 8 Done Done Sent false
19733 5937 Acked Read 4 Done Done Sent false
19734 5938 Acked Read 6 Done Done Sent false
19735 5939 Acked Read 5 Done Done Sent false
19736 5940 Acked Flush 0 Done Done Sent false
19737 5941 Acked Read 7 Done Done Sent false
19738 5942 Acked Write 7 Done Done Sent false
19739 5943 Acked Flush 0 Done Done Sent false
19740 5944 Acked Write 4 Done Done Sent false
19741 5945 Acked Write 3 Done Done Sent false
19742 5946 Acked Write 2 Done Done Sent false
19743 5947 Acked Read 9 Done Done Sent false
19744 5948 Acked Write 7 Done Done Sent false
19745 5949 Acked Flush 0 Done Done Sent false
19746 5950 Acked Write 9 Done Done Sent false
19747 5951 Acked Write 8 Done Done Sent false
19748 5952 Acked Write 10 Done Done Sent false
19749 5953 Acked Read 3 Done Done Sent false
19750 5954 Acked Read 7 Done Done Sent false
19751 5955 Acked Write 8 Done Done Sent false
19752 5956 Acked Write 6 Done Done Sent false
19753 5957 Acked Read 7 Done Done Sent false
19754 5958 Acked Read 5 Done Done Sent false
19755 5959 Acked Write 9 Done Done Sent false
19756 5960 Acked Read 6 Done Done Sent false
19757 5961 Acked Read 4 Done Done Sent false
19758 5962 Acked Read 1 Done Done Sent false
19759 5963 Acked Write 7 Done Done Sent false
19760 5964 Acked Write 7 Done Done Sent false
19761 5965 Acked Write 9 Done Done Sent false
19762 5966 Acked Read 6 Done Done Sent false
19763 5967 Acked Read 3 Done Done Sent false
19764 5968 Acked Read 2 Done Done Sent false
19765 5969 Acked Write 2 Done Done Sent false
19766 5970 Acked Read 8 Done Done Sent false
19767 5971 Acked Read 10 Done Done Sent false
19768 5972 Acked Write 7 Done Done Sent false
19769 5973 Acked Read 6 Done Done Sent false
19770 5974 Acked Write 2 Done Done Sent false
19771 5975 Acked Read 10 Done Done Sent false
19772 5976 Acked Read 9 Done Done Sent false
19773 5977 Acked Write 4 Done Done Sent false
19774 5978 Acked Write 7 Done Done Sent false
19775 5979 Acked Write 5 Done Done Sent false
19776 5980 Acked Write 10 Done Done Sent false
19777 5981 Acked Read 6 Done Done Sent false
19778 5982 Acked Write 7 Done Done Sent false
19779 5983 Acked Write 5 Done Done Sent false
19780 5984 Acked Read 6 Done Done Sent false
19781 5985 Acked Write 10 Done Done Sent false
19782 5986 Acked Write 6 Done Done Sent false
19783 5987 Acked Flush 0 Done Done Sent false
19784 5988 Acked Write 9 Done Done Sent false
19785 5989 Acked Read 1 Done Done Sent false
19786 5990 Acked Write 10 Done Done Sent false
19787 5991 Acked Write 3 Done Done Sent false
19788 5992 Acked Write 8 Done Done Sent false
19789 5993 Acked Read 6 Done Done Sent false
19790 5994 Acked Flush 0 Done Done Sent false
19791 5995 Acked Flush 0 Done Done Sent false
19792 5996 Acked Write 7 Done Done Sent false
19793 5997 Acked Read 2 Done Done Sent false
19794 5998 Acked Write 5 Done Done Sent false
19795 5999 Acked Read 9 Done Done Sent false
19796 6000 Acked Write 1 Done Done Sent false
19797 6001 Acked Write 10 Done Done Sent false
19798 6002 Acked Write 4 Done Done Sent false
19799 6003 Acked Write 9 Done Done Sent false
19800 6004 Acked Write 1 Done Done Sent false
19801 6005 Acked Write 9 Done Done Sent false
19802 6006 Acked Read 10 Done Done Sent false
19803 6007 Acked Read 4 Done Done Sent false
19804 6008 Acked Write 10 Done Done Sent false
19805 6009 Acked Read 5 Done Done Sent false
19806 6010 Acked Flush 0 Done Done Sent false
19807 6011 Acked Read 6 Done Done Sent false
19808 6012 Acked Read 3 Done Done Sent false
19809 6013 Acked Write 1 Done Done Sent false
19810 6014 Acked Write 6 Done Done Sent false
19811 6015 Acked Write 9 Done Done Sent false
19812 6016 Acked Write 6 Done Done Sent false
19813 6017 Acked Write 2 Done Done Sent false
19814 6018 Acked Flush 0 Done Done Sent false
19815 6019 Acked Read 9 Done Done Sent false
19816 6020 Acked Flush 0 Done Done Sent false
19817 6021 Acked Write 4 Done Done Sent false
19818 6022 Acked Write 10 Done Done Sent false
19819 6023 Acked Read 4 Done Done Sent false
19820 6024 Acked Read 9 Done Done Sent false
19821 6025 Acked Write 10 Done Done Sent false
19822 6026 Acked Write 1 Done Done Sent false
19823 6027 Acked Write 2 Done Done Sent false
19824 6028 Acked Read 8 Done Done Sent false
19825 6029 Acked Read 5 Done Done Sent false
19826 6030 Acked Flush 0 Done Done Sent false
19827 6031 Acked Read 5 Done Done Sent false
19828 6032 Acked Write 3 Done Done Sent false
19829 6033 Acked Write 2 Done Done Sent false
19830 6034 Acked Read 6 Done Done Sent false
19831 6035 Acked Flush 0 Done Done Sent false
19832 6036 Acked Write 10 Done Done Sent false
19833 6037 Acked Write 6 Done Done Sent false
19834 6038 Acked Read 2 Done Done Sent false
19835 6039 Acked Read 3 Done Done Sent false
19836 6040 Acked Write 5 Done Done Sent false
19837 6041 Acked Read 1 Done Done Sent false
19838 6042 Acked Read 1 Done Done Sent false
19839 6043 Acked Write 5 Done Done Sent false
19840 6044 Acked Write 2 Done Done Sent false
19841 6045 Acked Read 4 Done Done Sent false
19842 6046 Acked Read 2 Done Done Sent false
19843 6047 Acked Write 10 Done Done Sent false
19844 6048 Acked Read 10 Done Done Sent false
19845 6049 Acked Write 3 Done Done Sent false
19846 6050 Acked Read 6 Done Done Sent false
19847 6051 Acked Write 9 Done Done Sent false
19848 6052 Acked Flush 0 Done Done Sent false
19849 6053 Acked Read 1 Done Done Sent false
19850 6054 Acked Read 7 Done Done Sent false
19851 6055 Acked Read 7 Done Done Sent false
19852 6056 Acked Read 1 Done Done Sent false
19853 6057 Acked Read 8 Done Done Sent false
19854 6058 Acked Write 2 Done Done Sent false
19855 6059 Acked Read 5 Done Done Sent false
19856 6060 Acked Read 1 Done Done Sent false
19857 6061 Acked Read 10 Done Done Sent false
19858 6062 Acked Read 7 Done Done Sent false
19859 6063 Acked Read 6 Done Done Sent false
19860 6064 Acked Read 7 Done Done Sent false
19861 6065 Acked Read 7 Done Done Sent false
19862 6066 Acked Write 2 Done Done Sent false
19863 6067 Acked Read 8 Done Done Sent false
19864 6068 Acked Flush 0 Done Done Sent false
19865 6069 Acked Write 3 Done Done Sent false
19866 6070 Acked Read 9 Done Done Sent false
19867 6071 Acked Write 1 Done Done Sent false
19868 6072 Acked Write 10 Done Done Sent false
19869 6073 Acked Write 6 Done Done Sent false
19870 6074 Acked Read 2 Done Done Sent false
19871 6075 Acked Write 1 Done Done Sent false
19872 6076 Acked Flush 0 Done Done Sent false
19873 6077 Acked Write 3 Done Done Sent false
19874 6078 Acked Write 9 Done Done Sent false
19875 6079 Acked Read 10 Done Done Sent false
19876 6080 Acked Read 7 Done Done Sent false
19877 6081 Acked Read 5 Done Done Sent false
19878 6082 Acked Write 9 Done Done Sent false
19879 6083 Acked Read 5 Done Done Sent false
19880 6084 Acked Write 10 Done Done Sent false
19881 6085 Acked Read 1 Done Done Sent false
19882 6086 Acked Flush 0 Done Done Sent false
19883 6087 Acked Write 4 Done Done Sent false
19884 6088 Acked Write 4 Done Done Sent false
19885 6089 Acked Write 2 Done Done Sent false
19886 6090 Acked Read 7 Done Done Sent false
19887 6091 Acked Read 6 Done Done Sent false
19888 6092 Acked Write 2 Done Done Sent false
19889 6093 Acked Flush 0 Done Done Sent false
19890 6094 Acked Write 7 Done Done Sent false
19891 6095 Acked Read 1 Done Done Sent false
19892 6096 Acked Write 4 Done Done Sent false
19893 6097 Acked Write 3 Done Done Sent false
19894 6098 Acked Flush 0 Done Done Sent false
19895 6099 Acked Read 9 Done Done Sent false
19896 6100 Acked Write 10 Done Done Sent false
19897 6101 Acked Write 1 Done Done Sent false
19898 6102 Acked Read 6 Done Done Sent false
19899 6103 Acked Flush 0 Done Done Sent false
19900 6104 Acked Read 2 Done Done Sent false
19901 6105 Acked Read 6 Done Done Sent false
19902 6106 Acked Read 9 Done Done Sent false
19903 6107 Acked Read 4 Done Done Sent false
19904 6108 Acked Read 7 Done Done Sent false
19905 6109 Acked Write 4 Done Done Sent false
19906 6110 Acked Read 8 Done Done Sent false
19907 6111 Acked Flush 0 Done Done Sent false
19908 6112 Acked Flush 0 Done Done Sent false
19909 6113 Acked Read 4 Done Done Sent false
19910 6114 Acked Write 9 Done Done Sent false
19911 6115 Acked Write 3 Done Done Sent false
19912 6116 Acked Write 6 Done Done Sent false
19913 6117 Acked Read 8 Done Done Sent false
19914 6118 Acked Write 10 Done Done Sent false
19915 6119 Acked Write 6 Done Done Sent false
19916 6120 Acked Flush 0 Done Done Sent false
19917 6121 Acked Write 6 Done Done Sent false
19918 6122 Acked Read 10 Done Done Sent false
19919 6123 Acked Write 2 Done Done Sent false
19920 6124 Acked Flush 0 Done Done Sent false
19921 6125 Acked Read 5 Done Done Sent false
19922 6126 Acked Read 8 Done Done Sent false
19923 6127 Acked Read 2 Done Done Sent false
19924 6128 Acked Write 4 Done Done Sent false
19925 6129 Acked Write 3 Done Done Sent false
19926 6130 Acked Read 10 Done Done Sent false
19927 6131 Acked Read 2 Done Done Sent false
19928 6132 Acked Read 6 Done Done Sent false
19929 6133 Acked Flush 0 Done Done Sent false
19930 STATES DS:0 DS:1 DS:2 TOTAL
19931 Sent 0 0 302 302
19932 Done 302 302 0 604
19933 Skipped 0 0 0 0
19934 Error 0 0 0 0
19935 Last Flush: 6133 6133 5831
19936 Downstairs last five completed: 5831 5830 5829 5828 5827
19937 Upstairs last five completed: 6133 6132 6131 6130 6129
19938 CLIENT: Up:0 ds:302 act:2
199392024-12-11T23:44:40.450ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
199402024-12-11T23:44:40.450ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
199412024-12-11T23:44:40.450ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
199422024-12-11T23:44:40.450ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
199432024-12-11T23:44:40.450ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
19944 ----------------------------------------------------------------
19945 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19946 States: Active Active Offline
19947 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19948 5832 Acked Write 2 Done Done Sent false
19949 5833 Acked Write 9 Done Done Sent false
19950 5834 Acked Read 5 Done Done Sent false
19951 5835 Acked Read 9 Done Done Sent false
19952 5836 Acked Flush 0 Done Done Sent false
19953 5837 Acked Read 9 Done Done Sent false
19954 5838 Acked Write 9 Done Done Sent false
19955 5839 Acked Flush 0 Done Done Sent false
19956 5840 Acked Read 6 Done Done Sent false
19957 5841 Acked Flush 0 Done Done Sent false
19958 5842 Acked Read 4 Done Done Sent false
19959 5843 Acked Write 4 Done Done Sent false
19960 5844 Acked Flush 0 Done Done Sent false
19961 5845 Acked Read 1 Done Done Sent false
19962 5846 Acked Write 4 Done Done Sent false
19963 5847 Acked Write 2 Done Done Sent false
19964 5848 Acked Read 7 Done Done Sent false
19965 5849 Acked Read 5 Done Done Sent false
19966 5850 Acked Read 8 Done Done Sent false
19967 5851 Acked Read 8 Done Done Sent false
19968 5852 Acked Read 10 Done Done Sent false
19969 5853 Acked Write 3 Done Done Sent false
19970 5854 Acked Read 5 Done Done Sent false
19971 5855 Acked Read 6 Done Done Sent false
19972 5856 Acked Read 4 Done Done Sent false
19973 5857 Acked Read 9 Done Done Sent false
19974 5858 Acked Flush 0 Done Done Sent false
19975 5859 Acked Read 6 Done Done Sent false
19976 5860 Acked Read 9 Done Done Sent false
19977 5861 Acked Write 3 Done Done Sent false
19978 5862 Acked Read 5 Done Done Sent false
19979 5863 Acked Read 8 Done Done Sent false
19980 5864 Acked Read 4 Done Done Sent false
19981 5865 Acked Read 8 Done Done Sent false
19982 5866 Acked Read 7 Done Done Sent false
19983 5867 Acked Write 5 Done Done Sent false
19984 5868 Acked Write 8 Done Done Sent false
19985 5869 Acked Read 3 Done Done Sent false
19986 5870 Acked Flush 0 Done Done Sent false
19987 5871 Acked Write 9 Done Done Sent false
19988 5872 Acked Read 4 Done Done Sent false
19989 5873 Acked Write 8 Done Done Sent false
19990 5874 Acked Write 2 Done Done Sent false
19991 5875 Acked Read 5 Done Done Sent false
19992 5876 Acked Write 7 Done Done Sent false
19993 5877 Acked Read 1 Done Done Sent false
19994 5878 Acked Write 6 Done Done Sent false
19995 5879 Acked Read 8 Done Done Sent false
19996 5880 Acked Write 10 Done Done Sent false
19997 5881 Acked Read 9 Done Done Sent false
19998 5882 Acked Flush 0 Done Done Sent false
19999 5883 Acked Read 4 Done Done Sent false
20000 5884 Acked Write 8 Done Done Sent false
20001 5885 Acked Write 3 Done Done Sent false
20002 5886 Acked Read 8 Done Done Sent false
20003 5887 Acked Write 2 Done Done Sent false
20004 5888 Acked Write 10 Done Done Sent false
20005 5889 Acked Write 1 Done Done Sent false
20006 5890 Acked Read 3 Done Done Sent false
20007 5891 Acked Write 5 Done Done Sent false
20008 5892 Acked Read 6 Done Done Sent false
20009 5893 Acked Read 10 Done Done Sent false
20010 5894 Acked Write 4 Done Done Sent false
20011 5895 Acked Write 10 Done Done Sent false
20012 5896 Acked Write 2 Done Done Sent false
20013 5897 Acked Write 3 Done Done Sent false
20014 5898 Acked Write 5 Done Done Sent false
20015 5899 Acked Read 9 Done Done Sent false
20016 5900 Acked Write 1 Done Done Sent false
20017 5901 Acked Read 5 Done Done Sent false
20018 5902 Acked Write 7 Done Done Sent false
20019 5903 Acked Write 7 Done Done Sent false
20020 5904 Acked Read 10 Done Done Sent false
20021 5905 Acked Write 4 Done Done Sent false
20022 5906 Acked Write 7 Done Done Sent false
20023 5907 Acked Write 7 Done Done Sent false
20024 5908 Acked Read 7 Done Done Sent false
20025 5909 Acked Read 3 Done Done Sent false
20026 5910 Acked Write 10 Done Done Sent false
20027 5911 Acked Write 9 Done Done Sent false
20028 5912 Acked Read 9 Done Done Sent false
20029 5913 Acked Read 3 Done Done Sent false
20030 5914 Acked Read 9 Done Done Sent false
20031 5915 Acked Write 5 Done Done Sent false
20032 5916 Acked Read 1 Done Done Sent false
20033 5917 Acked Write 4 Done Done Sent false
20034 5918 Acked Read 7 Done Done Sent false
20035 5919 Acked Read 1 Done Done Sent false
20036 5920 Acked Read 9 Done Done Sent false
20037 5921 Acked Flush 0 Done Done Sent false
20038 5922 Acked Write 3 Done Done Sent false
20039 5923 Acked Flush 0 Done Done Sent false
20040 5924 Acked Write 6 Done Done Sent false
20041 5925 Acked Flush 0 Done Done Sent false
20042 5926 Acked Read 1 Done Done Sent false
20043 5927 Acked Write 7 Done Done Sent false
20044 5928 Acked Write 5 Done Done Sent false
20045 5929 Acked Write 10 Done Done Sent false
20046 5930 Acked Read 3 Done Done Sent false
20047 5931 Acked Read 7 Done Done Sent false
20048 5932 Acked Read 9 Done Done Sent false
20049 5933 Acked Read 6 Done Done Sent false
20050 5934 Acked Read 2 Done Done Sent false
20051 5935 Acked Write 1 Done Done Sent false
20052 5936 Acked Write 8 Done Done Sent false
20053 5937 Acked Read 4 Done Done Sent false
20054 5938 Acked Read 6 Done Done Sent false
20055 5939 Acked Read 5 Done Done Sent false
20056 5940 Acked Flush 0 Done Done Sent false
20057 5941 Acked Read 7 Done Done Sent false
20058 5942 Acked Write 7 Done Done Sent false
20059 5943 Acked Flush 0 Done Done Sent false
20060 5944 Acked Write 4 Done Done Sent false
20061 5945 Acked Write 3 Done Done Sent false
20062 5946 Acked Write 2 Done Done Sent false
20063 5947 Acked Read 9 Done Done Sent false
20064 5948 Acked Write 7 Done Done Sent false
20065 5949 Acked Flush 0 Done Done Sent false
20066 5950 Acked Write 9 Done Done Sent false
20067 5951 Acked Write 8 Done Done Sent false
20068 5952 Acked Write 10 Done Done Sent false
20069 5953 Acked Read 3 Done Done Sent false
20070 5954 Acked Read 7 Done Done Sent false
20071 5955 Acked Write 8 Done Done Sent false
20072 5956 Acked Write 6 Done Done Sent false
20073 5957 Acked Read 7 Done Done Sent false
20074 5958 Acked Read 5 Done Done Sent false
20075 5959 Acked Write 9 Done Done Sent false
20076 5960 Acked Read 6 Done Done Sent false
20077 5961 Acked Read 4 Done Done Sent false
20078 5962 Acked Read 1 Done Done Sent false
20079 5963 Acked Write 7 Done Done Sent false
20080 5964 Acked Write 7 Done Done Sent false
20081 5965 Acked Write 9 Done Done Sent false
20082 5966 Acked Read 6 Done Done Sent false
20083 5967 Acked Read 3 Done Done Sent false
20084 5968 Acked Read 2 Done Done Sent false
20085 5969 Acked Write 2 Done Done Sent false
20086 5970 Acked Read 8 Done Done Sent false
20087 5971 Acked Read 10 Done Done Sent false
20088 5972 Acked Write 7 Done Done Sent false
20089 5973 Acked Read 6 Done Done Sent false
20090 5974 Acked Write 2 Done Done Sent false
20091 5975 Acked Read 10 Done Done Sent false
20092 5976 Acked Read 9 Done Done Sent false
20093 5977 Acked Write 4 Done Done Sent false
20094 5978 Acked Write 7 Done Done Sent false
20095 5979 Acked Write 5 Done Done Sent false
20096 5980 Acked Write 10 Done Done Sent false
20097 5981 Acked Read 6 Done Done Sent false
20098 5982 Acked Write 7 Done Done Sent false
20099 5983 Acked Write 5 Done Done Sent false
20100 5984 Acked Read 6 Done Done Sent false
20101 5985 Acked Write 10 Done Done Sent false
20102 5986 Acked Write 6 Done Done Sent false
20103 5987 Acked Flush 0 Done Done Sent false
20104 5988 Acked Write 9 Done Done Sent false
20105 5989 Acked Read 1 Done Done Sent false
20106 5990 Acked Write 10 Done Done Sent false
20107 5991 Acked Write 3 Done Done Sent false
20108 5992 Acked Write 8 Done Done Sent false
20109 5993 Acked Read 6 Done Done Sent false
20110 5994 Acked Flush 0 Done Done Sent false
20111 5995 Acked Flush 0 Done Done Sent false
20112 5996 Acked Write 7 Done Done Sent false
20113 5997 Acked Read 2 Done Done Sent false
20114 5998 Acked Write 5 Done Done Sent false
20115 5999 Acked Read 9 Done Done Sent false
20116 6000 Acked Write 1 Done Done Sent false
20117 6001 Acked Write 10 Done Done Sent false
20118 6002 Acked Write 4 Done Done Sent false
20119 6003 Acked Write 9 Done Done Sent false
20120 6004 Acked Write 1 Done Done Sent false
20121 6005 Acked Write 9 Done Done Sent false
20122 6006 Acked Read 10 Done Done Sent false
20123 6007 Acked Read 4 Done Done Sent false
20124 6008 Acked Write 10 Done Done Sent false
20125 6009 Acked Read 5 Done Done Sent false
20126 6010 Acked Flush 0 Done Done Sent false
20127 6011 Acked Read 6 Done Done Sent false
20128 6012 Acked Read 3 Done Done Sent false
20129 6013 Acked Write 1 Done Done Sent false
20130 6014 Acked Write 6 Done Done Sent false
20131 6015 Acked Write 9 Done Done Sent false
20132 6016 Acked Write 6 Done Done Sent false
20133 6017 Acked Write 2 Done Done Sent false
20134 6018 Acked Flush 0 Done Done Sent false
20135 6019 Acked Read 9 Done Done Sent false
20136 6020 Acked Flush 0 Done Done Sent false
20137 6021 Acked Write 4 Done Done Sent false
20138 6022 Acked Write 10 Done Done Sent false
20139 6023 Acked Read 4 Done Done Sent false
20140 6024 Acked Read 9 Done Done Sent false
20141 6025 Acked Write 10 Done Done Sent false
20142 6026 Acked Write 1 Done Done Sent false
20143 6027 Acked Write 2 Done Done Sent false
20144 6028 Acked Read 8 Done Done Sent false
20145 6029 Acked Read 5 Done Done Sent false
20146 6030 Acked Flush 0 Done Done Sent false
20147 6031 Acked Read 5 Done Done Sent false
20148 6032 Acked Write 3 Done Done Sent false
20149 6033 Acked Write 2 Done Done Sent false
20150 6034 Acked Read 6 Done Done Sent false
20151 6035 Acked Flush 0 Done Done Sent false
20152 6036 Acked Write 10 Done Done Sent false
20153 6037 Acked Write 6 Done Done Sent false
20154 6038 Acked Read 2 Done Done Sent false
20155 6039 Acked Read 3 Done Done Sent false
20156 6040 Acked Write 5 Done Done Sent false
20157 6041 Acked Read 1 Done Done Sent false
20158 6042 Acked Read 1 Done Done Sent false
20159 6043 Acked Write 5 Done Done Sent false
20160 6044 Acked Write 2 Done Done Sent false
20161 6045 Acked Read 4 Done Done Sent false
20162 6046 Acked Read 2 Done Done Sent false
20163 6047 Acked Write 10 Done Done Sent false
20164 6048 Acked Read 10 Done Done Sent false
20165 6049 Acked Write 3 Done Done Sent false
20166 6050 Acked Read 6 Done Done Sent false
20167 6051 Acked Write 9 Done Done Sent false
20168 6052 Acked Flush 0 Done Done Sent false
20169 6053 Acked Read 1 Done Done Sent false
20170 6054 Acked Read 7 Done Done Sent false
20171 6055 Acked Read 7 Done Done Sent false
20172 6056 Acked Read 1 Done Done Sent false
20173 6057 Acked Read 8 Done Done Sent false
20174 6058 Acked Write 2 Done Done Sent false
20175 6059 Acked Read 5 Done Done Sent false
20176 6060 Acked Read 1 Done Done Sent false
20177 6061 Acked Read 10 Done Done Sent false
20178 6062 Acked Read 7 Done Done Sent false
20179 6063 Acked Read 6 Done Done Sent false
20180 6064 Acked Read 7 Done Done Sent false
20181 6065 Acked Read 7 Done Done Sent false
20182 6066 Acked Write 2 Done Done Sent false
20183 6067 Acked Read 8 Done Done Sent false
20184 6068 Acked Flush 0 Done Done Sent false
20185 6069 Acked Write 3 Done Done Sent false
20186 6070 Acked Read 9 Done Done Sent false
20187 6071 Acked Write 1 Done Done Sent false
20188 6072 Acked Write 10 Done Done Sent false
20189 6073 Acked Write 6 Done Done Sent false
20190 6074 Acked Read 2 Done Done Sent false
20191 6075 Acked Write 1 Done Done Sent false
20192 6076 Acked Flush 0 Done Done Sent false
20193 6077 Acked Write 3 Done Done Sent false
20194 6078 Acked Write 9 Done Done Sent false
20195 6079 Acked Read 10 Done Done Sent false
20196 6080 Acked Read 7 Done Done Sent false
20197 6081 Acked Read 5 Done Done Sent false
20198 6082 Acked Write 9 Done Done Sent false
20199 6083 Acked Read 5 Done Done Sent false
20200 6084 Acked Write 10 Done Done Sent false
20201 6085 Acked Read 1 Done Done Sent false
20202 6086 Acked Flush 0 Done Done Sent false
20203 6087 Acked Write 4 Done Done Sent false
20204 6088 Acked Write 4 Done Done Sent false
20205 6089 Acked Write 2 Done Done Sent false
20206 6090 Acked Read 7 Done Done Sent false
20207 6091 Acked Read 6 Done Done Sent false
20208 6092 Acked Write 2 Done Done Sent false
20209 6093 Acked Flush 0 Done Done Sent false
20210 6094 Acked Write 7 Done Done Sent false
20211 6095 Acked Read 1 Done Done Sent false
20212 6096 Acked Write 4 Done Done Sent false
20213 6097 Acked Write 3 Done Done Sent false
20214 6098 Acked Flush 0 Done Done Sent false
20215 6099 Acked Read 9 Done Done Sent false
20216 6100 Acked Write 10 Done Done Sent false
20217 6101 Acked Write 1 Done Done Sent false
20218 6102 Acked Read 6 Done Done Sent false
20219 6103 Acked Flush 0 Done Done Sent false
20220 6104 Acked Read 2 Done Done Sent false
20221 6105 Acked Read 6 Done Done Sent false
20222 6106 Acked Read 9 Done Done Sent false
20223 6107 Acked Read 4 Done Done Sent false
20224 6108 Acked Read 7 Done Done Sent false
20225 6109 Acked Write 4 Done Done Sent false
20226 6110 Acked Read 8 Done Done Sent false
20227 6111 Acked Flush 0 Done Done Sent false
20228 6112 Acked Flush 0 Done Done Sent false
20229 6113 Acked Read 4 Done Done Sent false
20230 6114 Acked Write 9 Done Done Sent false
20231 6115 Acked Write 3 Done Done Sent false
20232 6116 Acked Write 6 Done Done Sent false
20233 6117 Acked Read 8 Done Done Sent false
20234 6118 Acked Write 10 Done Done Sent false
20235 6119 Acked Write 6 Done Done Sent false
20236 6120 Acked Flush 0 Done Done Sent false
20237 6121 Acked Write 6 Done Done Sent false
20238 6122 Acked Read 10 Done Done Sent false
20239 6123 Acked Write 2 Done Done Sent false
20240 6124 Acked Flush 0 Done Done Sent false
20241 6125 Acked Read 5 Done Done Sent false
20242 6126 Acked Read 8 Done Done Sent false
20243 6127 Acked Read 2 Done Done Sent false
20244 6128 Acked Write 4 Done Done Sent false
20245 6129 Acked Write 3 Done Done Sent false
20246 6130 Acked Read 10 Done Done Sent false
20247 6131 Acked Read 2 Done Done Sent false
20248 6132 Acked Read 6 Done Done Sent false
20249 6133 Acked Flush 0 Done Done Sent false
20250 STATES DS:0 DS:1 DS:2 TOTAL
20251 Sent 0 0 302 302
20252 Done 302 302 0 604
20253 Skipped 0 0 0 0
20254 Error 0 0 0 0
20255 Last Flush: 6133 6133 5831
20256 Downstairs last five completed: 5831 5830 5829 5828 5827
20257 Upstairs last five completed: 6133 6132 6131 6130 6129
20258 CLIENT: Up:0 ds:302 act:2
20259 [8830] got start action
20260 [8830] I am going to restart
20261 Starting downstairs at port 8830
20262 Make output file at "/tmp/dsc/downstairs-8830.txt"
20263 [8830][2] reports Starting
20264 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1217)
20265 [8830][2] reports Running
20266 ----------------------------------------------------------------
20267 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20268 States: Active Active Offline
20269 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20270 5832 Acked Write 2 Done Done Sent false
20271 5833 Acked Write 9 Done Done Sent false
20272 5834 Acked Read 5 Done Done Sent false
20273 5835 Acked Read 9 Done Done Sent false
20274 5836 Acked Flush 0 Done Done Sent false
20275 5837 Acked Read 9 Done Done Sent false
20276 5838 Acked Write 9 Done Done Sent false
20277 5839 Acked Flush 0 Done Done Sent false
20278 5840 Acked Read 6 Done Done Sent false
20279 5841 Acked Flush 0 Done Done Sent false
20280 5842 Acked Read 4 Done Done Sent false
20281 5843 Acked Write 4 Done Done Sent false
20282 5844 Acked Flush 0 Done Done Sent false
20283 5845 Acked Read 1 Done Done Sent false
20284 5846 Acked Write 4 Done Done Sent false
20285 5847 Acked Write 2 Done Done Sent false
20286 5848 Acked Read 7 Done Done Sent false
20287 5849 Acked Read 5 Done Done Sent false
20288 5850 Acked Read 8 Done Done Sent false
20289 5851 Acked Read 8 Done Done Sent false
20290 5852 Acked Read 10 Done Done Sent false
20291 5853 Acked Write 3 Done Done Sent false
20292 5854 Acked Read 5 Done Done Sent false
20293 5855 Acked Read 6 Done Done Sent false
20294 5856 Acked Read 4 Done Done Sent false
20295 5857 Acked Read 9 Done Done Sent false
20296 5858 Acked Flush 0 Done Done Sent false
20297 5859 Acked Read 6 Done Done Sent false
20298 5860 Acked Read 9 Done Done Sent false
20299 5861 Acked Write 3 Done Done Sent false
20300 5862 Acked Read 5 Done Done Sent false
20301 5863 Acked Read 8 Done Done Sent false
20302 5864 Acked Read 4 Done Done Sent false
20303 5865 Acked Read 8 Done Done Sent false
20304 5866 Acked Read 7 Done Done Sent false
20305 5867 Acked Write 5 Done Done Sent false
20306 5868 Acked Write 8 Done Done Sent false
20307 5869 Acked Read 3 Done Done Sent false
20308 5870 Acked Flush 0 Done Done Sent false
20309 5871 Acked Write 9 Done Done Sent false
20310 5872 Acked Read 4 Done Done Sent false
20311 5873 Acked Write 8 Done Done Sent false
20312 5874 Acked Write 2 Done Done Sent false
20313 5875 Acked Read 5 Done Done Sent false
20314 5876 Acked Write 7 Done Done Sent false
20315 5877 Acked Read 1 Done Done Sent false
20316 5878 Acked Write 6 Done Done Sent false
20317 5879 Acked Read 8 Done Done Sent false
20318 5880 Acked Write 10 Done Done Sent false
20319 5881 Acked Read 9 Done Done Sent false
20320 5882 Acked Flush 0 Done Done Sent false
20321 5883 Acked Read 4 Done Done Sent false
20322 5884 Acked Write 8 Done Done Sent false
20323 5885 Acked Write 3 Done Done Sent false
20324 5886 Acked Read 8 Done Done Sent false
20325 5887 Acked Write 2 Done Done Sent false
20326 5888 Acked Write 10 Done Done Sent false
20327 5889 Acked Write 1 Done Done Sent false
20328 5890 Acked Read 3 Done Done Sent false
20329 5891 Acked Write 5 Done Done Sent false
20330 5892 Acked Read 6 Done Done Sent false
20331 5893 Acked Read 10 Done Done Sent false
20332 5894 Acked Write 4 Done Done Sent false
20333 5895 Acked Write 10 Done Done Sent false
20334 5896 Acked Write 2 Done Done Sent false
20335 5897 Acked Write 3 Done Done Sent false
20336 5898 Acked Write 5 Done Done Sent false
20337 5899 Acked Read 9 Done Done Sent false
20338 5900 Acked Write 1 Done Done Sent false
20339 5901 Acked Read 5 Done Done Sent false
20340 5902 Acked Write 7 Done Done Sent false
20341 5903 Acked Write 7 Done Done Sent false
20342 5904 Acked Read 10 Done Done Sent false
20343 5905 Acked Write 4 Done Done Sent false
20344 5906 Acked Write 7 Done Done Sent false
20345 5907 Acked Write 7 Done Done Sent false
20346 5908 Acked Read 7 Done Done Sent false
20347 5909 Acked Read 3 Done Done Sent false
20348 5910 Acked Write 10 Done Done Sent false
20349 5911 Acked Write 9 Done Done Sent false
20350 5912 Acked Read 9 Done Done Sent false
20351 5913 Acked Read 3 Done Done Sent false
20352 5914 Acked Read 9 Done Done Sent false
20353 5915 Acked Write 5 Done Done Sent false
20354 5916 Acked Read 1 Done Done Sent false
20355 5917 Acked Write 4 Done Done Sent false
20356 5918 Acked Read 7 Done Done Sent false
20357 5919 Acked Read 1 Done Done Sent false
20358 5920 Acked Read 9 Done Done Sent false
20359 5921 Acked Flush 0 Done Done Sent false
20360 5922 Acked Write 3 Done Done Sent false
20361 5923 Acked Flush 0 Done Done Sent false
20362 5924 Acked Write 6 Done Done Sent false
20363 5925 Acked Flush 0 Done Done Sent false
20364 5926 Acked Read 1 Done Done Sent false
20365 5927 Acked Write 7 Done Done Sent false
20366 5928 Acked Write 5 Done Done Sent false
20367 5929 Acked Write 10 Done Done Sent false
20368 5930 Acked Read 3 Done Done Sent false
20369 5931 Acked Read 7 Done Done Sent false
20370 5932 Acked Read 9 Done Done Sent false
20371 5933 Acked Read 6 Done Done Sent false
20372 5934 Acked Read 2 Done Done Sent false
20373 5935 Acked Write 1 Done Done Sent false
20374 5936 Acked Write 8 Done Done Sent false
20375 5937 Acked Read 4 Done Done Sent false
20376 5938 Acked Read 6 Done Done Sent false
20377 5939 Acked Read 5 Done Done Sent false
20378 5940 Acked Flush 0 Done Done Sent false
20379 5941 Acked Read 7 Done Done Sent false
20380 5942 Acked Write 7 Done Done Sent false
20381 5943 Acked Flush 0 Done Done Sent false
20382 5944 Acked Write 4 Done Done Sent false
20383 5945 Acked Write 3 Done Done Sent false
20384 5946 Acked Write 2 Done Done Sent false
20385 5947 Acked Read 9 Done Done Sent false
20386 5948 Acked Write 7 Done Done Sent false
20387 5949 Acked Flush 0 Done Done Sent false
20388 5950 Acked Write 9 Done Done Sent false
20389 5951 Acked Write 8 Done Done Sent false
20390 5952 Acked Write 10 Done Done Sent false
20391 5953 Acked Read 3 Done Done Sent false
20392 5954 Acked Read 7 Done Done Sent false
20393 5955 Acked Write 8 Done Done Sent false
20394 5956 Acked Write 6 Done Done Sent false
20395 5957 Acked Read 7 Done Done Sent false
20396 5958 Acked Read 5 Done Done Sent false
20397 5959 Acked Write 9 Done Done Sent false
20398 5960 Acked Read 6 Done Done Sent false
20399 5961 Acked Read 4 Done Done Sent false
20400 5962 Acked Read 1 Done Done Sent false
20401 5963 Acked Write 7 Done Done Sent false
20402 5964 Acked Write 7 Done Done Sent false
20403 5965 Acked Write 9 Done Done Sent false
20404 5966 Acked Read 6 Done Done Sent false
20405 5967 Acked Read 3 Done Done Sent false
20406 5968 Acked Read 2 Done Done Sent false
20407 5969 Acked Write 2 Done Done Sent false
20408 5970 Acked Read 8 Done Done Sent false
20409 5971 Acked Read 10 Done Done Sent false
20410 5972 Acked Write 7 Done Done Sent false
20411 5973 Acked Read 6 Done Done Sent false
20412 5974 Acked Write 2 Done Done Sent false
20413 5975 Acked Read 10 Done Done Sent false
20414 5976 Acked Read 9 Done Done Sent false
20415 5977 Acked Write 4 Done Done Sent false
20416 5978 Acked Write 7 Done Done Sent false
20417 5979 Acked Write 5 Done Done Sent false
20418 5980 Acked Write 10 Done Done Sent false
20419 5981 Acked Read 6 Done Done Sent false
20420 5982 Acked Write 7 Done Done Sent false
20421 5983 Acked Write 5 Done Done Sent false
20422 5984 Acked Read 6 Done Done Sent false
20423 5985 Acked Write 10 Done Done Sent false
20424 5986 Acked Write 6 Done Done Sent false
20425 5987 Acked Flush 0 Done Done Sent false
20426 5988 Acked Write 9 Done Done Sent false
20427 5989 Acked Read 1 Done Done Sent false
20428 5990 Acked Write 10 Done Done Sent false
20429 5991 Acked Write 3 Done Done Sent false
20430 5992 Acked Write 8 Done Done Sent false
20431 5993 Acked Read 6 Done Done Sent false
20432 5994 Acked Flush 0 Done Done Sent false
20433 5995 Acked Flush 0 Done Done Sent false
20434 5996 Acked Write 7 Done Done Sent false
20435 5997 Acked Read 2 Done Done Sent false
20436 5998 Acked Write 5 Done Done Sent false
20437 5999 Acked Read 9 Done Done Sent false
20438 6000 Acked Write 1 Done Done Sent false
20439 6001 Acked Write 10 Done Done Sent false
20440 6002 Acked Write 4 Done Done Sent false
20441 6003 Acked Write 9 Done Done Sent false
20442 6004 Acked Write 1 Done Done Sent false
20443 6005 Acked Write 9 Done Done Sent false
20444 6006 Acked Read 10 Done Done Sent false
20445 6007 Acked Read 4 Done Done Sent false
20446 6008 Acked Write 10 Done Done Sent false
20447 6009 Acked Read 5 Done Done Sent false
20448 6010 Acked Flush 0 Done Done Sent false
20449 6011 Acked Read 6 Done Done Sent false
20450 6012 Acked Read 3 Done Done Sent false
20451 6013 Acked Write 1 Done Done Sent false
20452 6014 Acked Write 6 Done Done Sent false
20453 6015 Acked Write 9 Done Done Sent false
20454 6016 Acked Write 6 Done Done Sent false
20455 6017 Acked Write 2 Done Done Sent false
20456 6018 Acked Flush 0 Done Done Sent false
20457 6019 Acked Read 9 Done Done Sent false
20458 6020 Acked Flush 0 Done Done Sent false
20459 6021 Acked Write 4 Done Done Sent false
20460 6022 Acked Write 10 Done Done Sent false
20461 6023 Acked Read 4 Done Done Sent false
20462 6024 Acked Read 9 Done Done Sent false
20463 6025 Acked Write 10 Done Done Sent false
20464 6026 Acked Write 1 Done Done Sent false
20465 6027 Acked Write 2 Done Done Sent false
20466 6028 Acked Read 8 Done Done Sent false
20467 6029 Acked Read 5 Done Done Sent false
20468 6030 Acked Flush 0 Done Done Sent false
20469 6031 Acked Read 5 Done Done Sent false
20470 6032 Acked Write 3 Done Done Sent false
20471 6033 Acked Write 2 Done Done Sent false
20472 6034 Acked Read 6 Done Done Sent false
20473 6035 Acked Flush 0 Done Done Sent false
20474 6036 Acked Write 10 Done Done Sent false
20475 6037 Acked Write 6 Done Done Sent false
20476 6038 Acked Read 2 Done Done Sent false
20477 6039 Acked Read 3 Done Done Sent false
20478 6040 Acked Write 5 Done Done Sent false
20479 6041 Acked Read 1 Done Done Sent false
20480 6042 Acked Read 1 Done Done Sent false
20481 6043 Acked Write 5 Done Done Sent false
20482 6044 Acked Write 2 Done Done Sent false
20483 6045 Acked Read 4 Done Done Sent false
20484 6046 Acked Read 2 Done Done Sent false
20485 6047 Acked Write 10 Done Done Sent false
20486 6048 Acked Read 10 Done Done Sent false
20487 6049 Acked Write 3 Done Done Sent false
20488 6050 Acked Read 6 Done Done Sent false
20489 6051 Acked Write 9 Done Done Sent false
20490 6052 Acked Flush 0 Done Done Sent false
20491 6053 Acked Read 1 Done Done Sent false
20492 6054 Acked Read 7 Done Done Sent false
20493 6055 Acked Read 7 Done Done Sent false
20494 6056 Acked Read 1 Done Done Sent false
20495 6057 Acked Read 8 Done Done Sent false
20496 6058 Acked Write 2 Done Done Sent false
20497 6059 Acked Read 5 Done Done Sent false
20498 6060 Acked Read 1 Done Done Sent false
20499 6061 Acked Read 10 Done Done Sent false
20500 6062 Acked Read 7 Done Done Sent false
20501 6063 Acked Read 6 Done Done Sent false
20502 6064 Acked Read 7 Done Done Sent false
20503 6065 Acked Read 7 Done Done Sent false
20504 6066 Acked Write 2 Done Done Sent false
20505 6067 Acked Read 8 Done Done Sent false
20506 6068 Acked Flush 0 Done Done Sent false
20507 6069 Acked Write 3 Done Done Sent false
20508 6070 Acked Read 9 Done Done Sent false
20509 6071 Acked Write 1 Done Done Sent false
20510 6072 Acked Write 10 Done Done Sent false
20511 6073 Acked Write 6 Done Done Sent false
20512 6074 Acked Read 2 Done Done Sent false
20513 6075 Acked Write 1 Done Done Sent false
20514 6076 Acked Flush 0 Done Done Sent false
20515 6077 Acked Write 3 Done Done Sent false
20516 6078 Acked Write 9 Done Done Sent false
20517 6079 Acked Read 10 Done Done Sent false
20518 6080 Acked Read 7 Done Done Sent false
20519 6081 Acked Read 5 Done Done Sent false
20520 6082 Acked Write 9 Done Done Sent false
20521 6083 Acked Read 5 Done Done Sent false
20522 6084 Acked Write 10 Done Done Sent false
20523 6085 Acked Read 1 Done Done Sent false
20524 6086 Acked Flush 0 Done Done Sent false
20525 6087 Acked Write 4 Done Done Sent false
20526 6088 Acked Write 4 Done Done Sent false
20527 6089 Acked Write 2 Done Done Sent false
20528 6090 Acked Read 7 Done Done Sent false
20529 6091 Acked Read 6 Done Done Sent false
20530 6092 Acked Write 2 Done Done Sent false
20531 6093 Acked Flush 0 Done Done Sent false
20532 6094 Acked Write 7 Done Done Sent false
20533 6095 Acked Read 1 Done Done Sent false
20534 6096 Acked Write 4 Done Done Sent false
20535 6097 Acked Write 3 Done Done Sent false
20536 6098 Acked Flush 0 Done Done Sent false
20537 6099 Acked Read 9 Done Done Sent false
20538 6100 Acked Write 10 Done Done Sent false
20539 6101 Acked Write 1 Done Done Sent false
20540 6102 Acked Read 6 Done Done Sent false
20541 6103 Acked Flush 0 Done Done Sent false
20542 6104 Acked Read 2 Done Done Sent false
20543 6105 Acked Read 6 Done Done Sent false
20544 6106 Acked Read 9 Done Done Sent false
20545 6107 Acked Read 4 Done Done Sent false
20546 6108 Acked Read 7 Done Done Sent false
20547 6109 Acked Write 4 Done Done Sent false
20548 6110 Acked Read 8 Done Done Sent false
20549 6111 Acked Flush 0 Done Done Sent false
20550 6112 Acked Flush 0 Done Done Sent false
20551 6113 Acked Read 4 Done Done Sent false
20552 6114 Acked Write 9 Done Done Sent false
20553 6115 Acked Write 3 Done Done Sent false
20554 6116 Acked Write 6 Done Done Sent false
20555 6117 Acked Read 8 Done Done Sent false
20556 6118 Acked Write 10 Done Done Sent false
20557 6119 Acked Write 6 Done Done Sent false
20558 6120 Acked Flush 0 Done Done Sent false
20559 6121 Acked Write 6 Done Done Sent false
20560 6122 Acked Read 10 Done Done Sent false
20561 6123 Acked Write 2 Done Done Sent false
20562 6124 Acked Flush 0 Done Done Sent false
20563 6125 Acked Read 5 Done Done Sent false
20564 6126 Acked Read 8 Done Done Sent false
20565 6127 Acked Read 2 Done Done Sent false
20566 6128 Acked Write 4 Done Done Sent false
20567 6129 Acked Write 3 Done Done Sent false
20568 6130 Acked Read 10 Done Done Sent false
20569 6131 Acked Read 2 Done Done Sent false
20570 6132 Acked Read 6 Done Done Sent false
20571 6133 Acked Flush 0 Done Done Sent false
20572 STATES DS:0 DS:1 DS:2 TOTAL
20573 Sent 0 0 302 302
20574 Done 302 302 0 604
20575 Skipped 0 0 0 0
20576 Error 0 0 0 0
20577 Last Flush: 6133 6133 5831
20578 Downstairs last five completed: 5831 5830 5829 5828 5827
20579 Upstairs last five completed: 6133 6132 6131 6130 6129
20580 CLIENT: Up:0 ds:302 act:2
205812024-12-11T23:44:50.452ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205822024-12-11T23:44:50.452ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55198) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205832024-12-11T23:44:50.453ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205842024-12-11T23:44:50.453ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205852024-12-11T23:44:50.453ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205862024-12-11T23:44:50.453ZINFOcrucible: send last flush ID to this DS: 5831 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205872024-12-11T23:44:50.453ZINFOcrucible: Replied this last flush ID: 5831 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205882024-12-11T23:44:50.453ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205892024-12-11T23:44:50.453ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205902024-12-11T23:44:50.453ZINFOcrucible: [2] client re-new 302 jobs since flush 5831 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
205912024-12-11T23:44:50.454ZINFOcrucible: [2] Replayed 302 jobs since flush: 5831 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
20592 ----------------------------------------------------------------
20593 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20594 Downstairs last five completed: 6133 6132 6131 6130 6129
20595 Upstairs last five completed: 6133 6132 6131 6130 6129
20596 CLIENT: Up:0 ds:0 act:3
20597 Replay: All jobs finished
20598 Main task has work to do, go find it
20599 got dsc Stop(1)
20600 stop 1
20601 Dec 11 23:44:50.699 INFO[8820] Got stop action so:false kr:false
20602 request completed, latency_us: 311, response_code: 204, uri: /stop/cid/1, method: POST, req_id: c62b2d54-9375-4f72-b5fe-479a0e4d19ab, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
20603 Dec 11 23:44:50.699 INFO request completed, latency_us:{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:44:50.699946982Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
20604 128, response_code: 200, uri: /state/cid/1, method: GET, req_id: 8497d488-cec4-48ea-95da-23326c95d3ed, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:44:50.700017244Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
20605 :9998
206062024-12-11T23:44:50.700ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
206072024-12-11T23:44:50.700ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
206082024-12-11T23:44:50.700ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
206092024-12-11T23:44:50.700ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
20610 [8820] Exited with: ExitStatus(unix_wait_status(9))
20611 [8820][1] reports Exit
20612 Dec 11 23:44:54.702 INFO request completed, latency_us: 222, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1d25527d-02b0-4154-ad42-b12bda216cd2, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
20613 001/300 Read block 1754 len 12288
20614 002/300 Write block 2943 len 24576 data: 4 5 5 5 5 4
20615 003/300 Write block 4154 len 16384 data: 3 3 2 2
20616 004/300 Write block 3777 len 12288 data: 3 3 3
20617 005/300 Read block 1435 len 24576
20618 006/300 Read block 1369 len 8192
20619 007/300 Read block 1499 len 12288
20620 008/300 Read block 2156 len 8192
20621 009/300 Read block 4671 len 28672
20622 010/300 Read block 835 len 4096
20623 011/300 Read block 4352 len 40960
20624 012/300 Write block 1001 len 16384 data: 6 5 4 4
20625 013/300 Write block 1299 len 40960 data: 2 2 3 3 3 3 4 4 6 6
20626 014/300 Read block 4323 len 12288
20627 015/300 Read block 1653 len 32768
20628 016/300 Read block 2298 len 40960
20629 017/300 Write block 2640 len 36864 data: 2 2 2 3 4 4 5 5 5
20630 018/300 Write block 2415 len 12288 data: 3 4 5
20631 019/300 Write block 1660 len 32768 data: 3 4 4 3 3 2 2 2
20632 020/300 Write block 4945 len 16384 data: 5 4 3 3
20633 021/300 Read block 1035 len 4096
20634 022/300 Read block 445 len 4096
20635 023/300 Write block 3365 len 4096 data: 5
20636 024/300 Write block 1897 len 4096 data: 3
20637 025/300 Read block 601 len 12288
20638 026/300 Read block 366 len 12288
20639 027/300 Read block 3578 len 40960
20640 028/300 Write block 4062 len 36864 data: 7 6 5 5 4 3 3 4 4
20641 029/300 Write block 3894 len 36864 data: 5 5 5 6 6 5 5 5 3
20642 030/300 Read block 4400 len 40960
20643 031/300 Write block 1120 len 20480 data: 6 5 3 3 3
20644 032/300 Write block 3903 len 36864 data: 3 4 3 3 4 4 5 5 5
20645 033/300 Write block 4755 len 12288 data: 5 3 3
20646 034/300 Write block 339 len 4096 data: 3
20647 035/300 Write block 770 len 28672 data: 4 7 6 4 3 3 4
20648 036/300 Write block 1775 len 12288 data: 5 4 2
20649 037/300 Read block 3937 len 28672
20650 038/300 Flush
20651 039/300 Flush
20652 040/300 Read block 1465 len 8192
20653 041/300 Write block 3462 len 16384 data: 4 4 3 3
20654 042/300 Write block 4140 len 40960 data: 3 3 3 4 4 3 3 4 4 4
20655 043/300 Read block 118 len 16384
20656 044/300 Write block 4825 len 8192 data: 2 4
20657 045/300 Read block 669 len 8192
20658 046/300 Flush
20659 047/300 Read block 3439 len 8192
20660 048/300 Read block 1717 len 8192
20661 049/300 Read block 2894 len 28672
20662 050/300 Write block 32 len 20480 data: 4 4 4 4 4
20663 051/300 Write block 2545 len 20480 data: 5 7 7 6 5
20664 052/300 Read block 4307 len 28672
20665 053/300 Write block 3422 len 36864 data: 5 4 3 2 2 3 3 3 3
20666 054/300 Write block 751 len 20480 data: 3 3 3 4 4
20667 055/300 Read block 1534 len 12288
20668 056/300 Read block 4823 len 8192
20669 057/300 Write block 4016 len 20480 data: 4 6 6 5 5
20670 058/300 Flush
20671 059/300 Read block 1718 len 40960
20672 060/300 Read block 219 len 4096
20673 061/300 Read block 4181 len 28672
20674 062/300 Write block 4673 len 8192 data: 4 3
20675 063/300 Read block 3255 len 20480
20676 064/300 Read block 393 len 24576
20677 065/300 Read block 1957 len 16384
20678 066/300 Read block 3062 len 8192
20679 067/300 Write block 1593 len 36864 data: 3 3 4 4 4 3 3 3 3
20680 068/300 Flush
20681 069/300 Write block 4235 len 28672 data: 6 6 5 5 5 7 6
20682 070/300 Write block 3336 len 4096 data: 3
20683 071/300 Read block 1571 len 36864
20684 072/300 Write block 868 len 32768 data: 4 4 4 4 4 5 5 6
20685 073/300 Read block 1892 len 4096
20686 074/300 Flush
20687 075/300 Flush
20688 076/300 Write block 4207 len 20480 data: 6 6 5 5 4
20689 077/300 Read block 4249 len 8192
20690 078/300 Write block 3353 len 4096 data: 2
20691 079/300 Read block 3102 len 40960
20692 080/300 Write block 3555 len 40960 data: 3 5 5 5 3 3 4 3 3 3
20693 081/300 Read block 3731 len 24576
20694 082/300 Read block 4203 len 12288
20695 083/300 Write block 3713 len 36864 data: 3 4 4 4 4 4 4 4 3
20696 084/300 Flush
20697 085/300 Flush
20698 086/300 Read block 4533 len 8192
20699 087/300 Read block 2223 len 8192
20700 088/300 Flush
20701 089/300 Read block 3092 len 16384
20702 090/300 Write block 747 len 32768 data: 2 2 3 3 4 4 4 5
20703 091/300 Read block 3994 len 4096
20704 092/300 Read block 1075 len 12288
20705 093/300 Write block 2553 len 40960 data: 4 4 3 3 3 3 3 4 3 3
20706 094/300 Write block 3628 len 24576 data: 5 6 6 6 4 4
20707 095/300 Read block 2700 len 28672
20708 096/300 Read block 2388 len 40960
20709 097/300 Read block 3135 len 20480
20710 098/300 Read block 2763 len 24576
20711 099/300 Write block 727 len 24576 data: 3 3 2 2 2 2
20712 100/300 Write block 1779 len 20480 data: 3 4 6 5 5
20713 101/300 Read block 1511 len 12288
20714 102/300 Read block 637 len 12288
20715 103/300 Read block 1196 len 40960
20716 104/300 Write block 3509 len 28672 data: 4 5 5 4 4 5 5
20717 105/300 Flush
20718 106/300 Read block 1428 len 4096
20719 107/300 Write block 2744 len 8192 data: 6 4
20720 108/300 Write block 2863 len 32768 data: 5 6 6 5 5 4 4 2
20721 109/300 Write block 555 len 40960 data: 2 2 2 2 2 2 2 2 3 3
20722 110/300 Write block 1918 len 28672 data: 3 3 4 4 4 4 5
20723 111/300 Read block 3049 len 12288
20724 112/300 Read block 4044 len 36864
20725 113/300 Read block 4069 len 28672
20726 114/300 Read block 2427 len 32768
20727 115/300 Write block 501 len 4096 data: 3
20728 116/300 Read block 1521 len 20480
20729 117/300 Write block 2674 len 12288 data: 2 2 3
20730 118/300 Flush
20731 119/300 Read block 1426 len 4096
20732 120/300 Flush
20733 121/300 Read block 4050 len 12288
20734 122/300 Write block 1157 len 40960 data: 4 4 4 4 6 5 5 4 4 4
20735 123/300 Read block 2474 len 40960
20736 124/300 Read block 4681 len 16384
20737 125/300 Write block 4001 len 24576 data: 4 5 4 4 4 4
20738 126/300 Read block 2381 len 16384
20739 127/300 Read block 2266 len 28672
20740 128/300 Read block 255 len 20480
20741 129/300 Read block 2050 len 12288
20742 130/300 Read block 3278 len 40960
20743 131/300 Read block 2007 len 20480
20744 132/300 Read block 3026 len 20480
20745 133/300 Read block 1351 len 12288
20746 134/300 Read block 3660 len 28672
20747 135/300 Read block 1387 len 16384
20748 136/300 Read block 372 len 24576
20749 137/300 Write block 4913 len 36864 data: 5 6 4 3 4 5 4 4 6
20750 138/300 Read block 2013 len 28672
20751 139/300 Read block 691 len 8192
20752 140/300 Write block 4912 len 40960 data: 5 6 7 5 4 5 6 5 5 7
20753 141/300 Write block 961 len 24576 data: 2 2 2 3 4 3
20754 142/300 Read block 420 len 28672
20755 143/300 Read block 2645 len 32768
20756 144/300 Flush
20757 145/300 Read block 919 len 40960
20758 146/300 Write block 1517 len 36864 data: 4 6 6 5 5 6 5 4 4
20759 147/300 Read block 3324 len 24576
20760 148/300 Write block 3063 len 36864 data: 4 4 4 3 4 4 4 4 4
20761 149/300 Write block 323 len 40960 data: 8 8 7 7 7 7 7 5 6 6
20762 150/300 Read block 536 len 28672
20763 151/300 Read block 732 len 24576
20764 152/300 Read block 198 len 28672
20765 153/300 Write block 705 len 16384 data: 4 4 4 3
20766 154/300 Flush
20767 155/300 Flush
20768 156/300 Write block 3936 len 8192 data: 5 5
20769 157/300 Flush
20770 158/300 Write block 1677 len 8192 data: 2 3
20771 159/300 Write block 2217 len 24576 data: 6 5 3 3 3 3
20772 160/300 Read block 167 len 32768
20773 161/300 Write block 4733 len 24576 data: 4 3 3 3 4 3
20774 162/300 Flush
20775 163/300 Flush
20776 164/300 Read block 4263 len 16384
20777 165/300 Write block 3656 len 28672 data: 3 3 3 2 4 4 4
20778 166/300 Read block 1940 len 4096
20779 167/300 Write block 1121 len 24576 data: 6 4 4 4 3 3
20780 168/300 Write block 2327 len 32768 data: 5 6 6 6 5 6 6 8
20781 169/300 Write block 4136 len 16384 data: 4 4 2 4
20782 170/300 Write block 1917 len 40960 data: 4 4 4 5 5 5 5 6 4 5
20783 171/300 Read block 2794 len 40960
20784 172/300 Write block 2677 len 16384 data: 3 3 3 3
20785 173/300 Flush
20786 174/300 Read block 4684 len 12288
20787 175/300 Read block 470 len 4096
20788 176/300 Flush
20789 177/300 Write block 3391 len 28672 data: 4 3 3 2 3 3 3
20790 178/300 Read block 4176 len 36864
20791 179/300 Flush
20792 180/300 Write block 4267 len 24576 data: 4 4 4 3 4 5
20793 181/300 Write block 2816 len 16384 data: 3 2 2 2
20794 182/300 Flush
20795 183/300 Read block 4198 len 40960
20796 184/300 Read block 4992 len 20480
20797 185/300 Write block 2311 len 40960 data: 6 4 3 4 3 2 2 2 3 3
20798 186/300 Write block 1487 len 12288 data: 3 3 3
20799 187/300 Read block 3771 len 12288
20800 188/300 Read block 1864 len 32768
20801 189/300 Flush
20802 190/300 Write block 1440 len 40960 data: 7 8 7 6 6 6 8 7 5 4
20803 191/300 Write block 1949 len 12288 data: 5 5 5
20804 192/300 Read block 2981 len 20480
20805 193/300 Read block 3976 len 24576
20806 194/300 Write block 1640 len 28672 data: 4 5 6 5 5 5 6
20807 195/300 Read block 463 len 12288
20808 196/300 Read block 3808 len 20480
20809 197/300 Read block 3524 len 20480
20810 198/300 Flush
20811 199/300 Flush
20812 200/300 Write block 3138 len 16384 data: 4 4 4 5
20813 201/300 Read block 76 len 8192
20814 202/300 Write block 1187 len 4096 data: 4
20815 203/300 Read block 393 len 4096
20816 204/300 Read block 4218 len 28672
20817 205/300 Read block 2643 len 20480
20818 206/300 Write block 4893 len 8192 data: 5 5
20819 207/300 Write block 3383 len 16384 data: 6 6 7 7
20820 208/300 Read block 4676 len 36864
20821 209/300 Write block 3335 len 16384 data: 3 4 3 4
20822 210/300 Read block 3519 len 4096
20823 211/300 Read block 4833 len 4096
20824 212/300 Write block 3348 len 16384 data: 4 3 3 3
20825 213/300 Write block 403 len 36864 data: 4 4 4 5 5 7 8 9 8
20826 214/300 Read block 1805 len 24576
20827 215/300 Read block 1328 len 32768
20828 216/300 Read block 941 len 12288
20829 217/300 Write block 1043 len 12288 data: 4 4 4
20830 218/300 Read block 1142 len 24576
20831 219/300 Read block 3455 len 24576
20832 220/300 Write block 3420 len 12288 data: 6 5 6
20833 221/300 Read block 3667 len 16384
20834 222/300 Flush
20835 223/300 Read block 3383 len 20480
20836 224/300 Read block 3775 len 12288
20837 225/300 Write block 4301 len 32768 data: 4 3 3 3 3 3 2 2
20838 226/300 Read block 2638 len 12288
20839 227/300 Read block 3063 len 32768
20840 228/300 Read block 268 len 32768
20841 229/300 Read block 48 len 32768
20842 230/300 Write block 4922 len 8192 data: 10 8
20843 231/300 Read block 2224 len 8192
20844 232/300 Read block 623 len 40960
20845 233/300 Read block 3993 len 24576
20846 234/300 Read block 346 len 12288
20847 235/300 Write block 883 len 24576 data: 6 6 6 5 5 4
20848 236/300 Write block 3541 len 40960 data: 7 7 7 6 6 5 5 5 6 5
20849 237/300 Flush
20850 238/300 Write block 178 len 16384 data: 6 6 6 5
20851 239/300 Read block 2235 len 36864
20852 240/300 Read block 4612 len 20480
20853 241/300 Write block 4149 len 28672 data: 5 4 4 4 4 4 4
20854 242/300 Read block 3789 len 16384
20855 243/300 Write block 4244 len 40960 data: 4 3 3 3 4 4 4 4 4 6
20856 244/300 Read block 1099 len 20480
20857 245/300 Flush
20858 246/300 Write block 937 len 12288 data: 4 4 5
20859 247/300 Read block 551 len 24576
20860 248/300 Write block 1895 len 8192 data: 5 5
20861 249/300 Read block 2189 len 20480
20862 250/300 Write block 479 len 20480 data: 5 5 4 5 5
20863 251/300 Flush
20864 252/300 Read block 2830 len 20480
20865 253/300 Write block 3168 len 40960 data: 5 5 6 5 6 5 6 5 5 5
20866 254/300 Read block 3614 len 36864
20867 255/300 Read block 2969 len 36864
20868 256/300 Write block 4427 len 32768 data: 7 8 9 6 6 6 8 5
20869 257/300 Write block 1952 len 20480 data: 5 4 5 3 3
20870 258/300 Read block 1407 len 8192
20871 259/300 Read block 517 len 28672
20872 260/300 Read block 548 len 12288
20873 261/300 Read block 4089 len 36864
20874 262/300 Write block 1452 len 32768 data: 5 4 3 4 5 6 5 5
20875 263/300 Read block 1080 len 4096
20876 264/300 Flush
20877 265/300 Flush
20878 266/300 Flush
20879 267/300 Read block 466 len 12288
20880 268/300 Write block 4525 len 20480 data: 7 8 7 7 8
20881 269/300 Write block 4629 len 28672 data: 4 3 4 3 3 3 3
20882 270/300 Read block 1960 len 40960
20883 271/300 Read block 4340 len 24576
20884 272/300 Read block 3259 len 20480
20885 273/300 Write block 2548 len 8192 data: 7 6
20886 274/300 Write block 2995 len 28672 data: 5 8 9 10 8 6 6
20887 275/300 Read block 1232 len 16384
20888 276/300 Read block 3311 len 28672
20889 277/300 Write block 1371 len 28672 data: 3 3 4 4 4 3 2
20890 278/300 Write block 3691 len 36864 data: 7 4 4 5 5 5 6 6 5
20891 279/300 Read block 126 len 32768
20892 280/300 Write block 3114 len 24576 data: 4 5 4 4 5 5
20893 281/300 Write block 2312 len 24576 data: 5 4 5 4 3 3
20894 282/300 Read block 1764 len 12288
20895 283/300 Write block 3718 len 36864 data: 5 5 5 4 3 3 3 4 5
20896 284/300 Write block 1905 len 12288 data: 3 4 4
20897 285/300 Read block 56 len 40960
20898 286/300 Write block 1260 len 4096 data: 7
20899 287/300 Read block 3188 len 20480
20900 288/300 Read block 4440 len 28672
20901 289/300 Flush
20902 290/300 Write block 3462 len 36864 data: 5 5 4 4 3 3 3 4 3
20903 291/300 Write block 2850 len 4096 data: 4
20904 292/300 Write block 3651 len 32768 data: 4 4 3 2 3 4 4 4
20905 293/300 Read block 3378 len 8192
20906 294/300 Read block 844 len 40960
20907 295/300 Write block 1832 len 8192 data: 4 4
20908 296/300 Write block 4362 len 8192 data: 4 4
20909 297/300 Flush
20910 298/300 Read block 1563 len 36864
20911 299/300 Read block 2591 len 4096
20912 300/300 Write block 3774 len 12288 data: 2 2 2
20913 301/300 Read block 606 len 20480
20914 Main task has work to do, go find it
20915 got dsc Start(1)
20916 start 1
20917 Dec 11 23:44:54.939 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/1, method: POST, req_id: 335fdf7a-eb8c-48c1-858b-81ccddb32d34, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
20918 Replay: started 1, returned:Ok(())
20919 ----------------------------------------------------------------
20920 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20921 States: Active Offline Active
20922 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20923 6134 Acked Read 3 Done Sent Done false
20924 6135 Acked Write 6 Done Sent Done false
20925 6136 Acked Write 4 Done Sent Done false
20926 6137 Acked Write 3 Done Sent Done false
20927 6138 Acked Read 6 Done Sent Done false
20928 6139 Acked Read 2 Done Sent Done false
20929 6140 Acked Read 3 Done Sent Done false
20930 6141 Acked Read 2 Done Sent Done false
20931 6142 Acked Read 7 Done Sent Done false
20932 6143 Acked Read 1 Done Sent Done false
20933 6144 Acked Read 10 Done Sent Done false
20934 6145 Acked Write 4 Done Sent Done false
20935 6146 Acked Write 10 Done Sent Done false
20936 6147 Acked Read 3 Done Sent Done false
20937 6148 Acked Read 8 Done Sent Done false
20938 6149 Acked Read 10 Done Sent Done false
20939 6150 Acked Write 9 Done Sent Done false
20940 6151 Acked Write 3 Done Sent Done false
20941 6152 Acked Write 8 Done Sent Done false
20942 6153 Acked Write 4 Done Sent Done false
20943 6154 Acked Read 1 Done Sent Done false
20944 6155 Acked Read 1 Done Sent Done false
20945 6156 Acked Write 1 Done Sent Done false
20946 6157 Acked Write 1 Done Sent Done false
20947 6158 Acked Read 3 Done Sent Done false
20948 6159 Acked Read 3 Done Sent Done false
20949 6160 Acked Read 10 Done Sent Done false
20950 6161 Acked Write 9 Done Sent Done false
20951 6162 Acked Write 9 Done Sent Done false
20952 6163 Acked Read 10 Done Sent Done false
20953 6164 Acked Write 5 Done Sent Done false
20954 6165 Acked Write 9 Done Sent Done false
20955 6166 Acked Write 3 Done Sent Done false
20956 6167 Acked Write 1 Done Sent Done false
20957 6168 Acked Write 7 Done Sent Done false
20958 6169 Acked Write 3 Done Sent Done false
20959 6170 Acked Read 7 Done Sent Done false
20960 6171 Acked Flush 0 Done Sent Done false
20961 6172 Acked Flush 0 Done Sent Done false
20962 6173 Acked Read 2 Done Sent Done false
20963 6174 Acked Write 4 Done Sent Done false
20964 6175 Acked Write 10 Done Sent Done false
20965 6176 Acked Read 4 Done Sent Done false
20966 6177 Acked Write 2 Done Sent Done false
20967 6178 Acked Read 2 Done Sent Done false
20968 6179 Acked Flush 0 Done Sent Done false
20969 6180 Acked Read 2 Done Sent Done false
20970 6181 Acked Read 2 Done Sent Done false
20971 6182 Acked Read 7 Done Sent Done false
20972 6183 Acked Write 5 Done Sent Done false
20973 6184 Acked Write 5 Done Sent Done false
20974 6185 Acked Read 7 Done Sent Done false
20975 6186 Acked Write 9 Done Sent Done false
20976 6187 Acked Write 5 Done Sent Done false
20977 6188 Acked Read 3 Done Sent Done false
20978 6189 Acked Read 2 Done Sent Done false
20979 6190 Acked Write 5 Done Sent Done false
20980 6191 Acked Flush 0 Done Sent Done false
20981 6192 Acked Read 10 Done Sent Done false
20982 6193 Acked Read 1 Done Sent Done false
20983 6194 Acked Read 7 Done Sent Done false
20984 6195 Acked Write 2 Done Sent Done false
20985 6196 Acked Read 5 Done Sent Done false
20986 6197 Acked Read 6 Done Sent Done false
20987 6198 Acked Read 4 Done Sent Done false
20988 6199 Acked Read 2 Done Sent Done false
20989 6200 Acked Write 9 Done Sent Done false
20990 6201 Acked Flush 0 Done Sent Done false
20991 6202 Acked Write 7 Done Sent Done false
20992 6203 Acked Write 1 Done Sent Done false
20993 6204 Acked Read 9 Done Sent Done false
20994 6205 Acked Write 8 Done Sent Done false
20995 6206 Acked Read 1 Done Sent Done false
20996 6207 Acked Flush 0 Done Sent Done false
20997 6208 Acked Flush 0 Done Sent Done false
20998 6209 Acked Write 5 Done Sent Done false
20999 6210 Acked Read 2 Done Sent Done false
21000 6211 Acked Write 1 Done Sent Done false
21001 6212 Acked Read 10 Done Sent Done false
21002 6213 Acked Write 10 Done Sent Done false
21003 6214 Acked Read 6 Done Sent Done false
21004 6215 Acked Read 3 Done Sent Done false
21005 6216 Acked Write 9 Done Sent Done false
21006 6217 Acked Flush 0 Done Sent Done false
21007 6218 Acked Flush 0 Done Sent Done false
21008 6219 Acked Read 2 Done Sent Done false
21009 6220 Acked Read 2 Done Sent Done false
21010 6221 Acked Flush 0 Done Sent Done false
21011 6222 Acked Read 4 Done Sent Done false
21012 6223 Acked Write 8 Done Sent Done false
21013 6224 Acked Read 1 Done Sent Done false
21014 6225 Acked Read 3 Done Sent Done false
21015 6226 Acked Write 10 Done Sent Done false
21016 6227 Acked Write 6 Done Sent Done false
21017 6228 Acked Read 7 Done Sent Done false
21018 6229 Acked Read 10 Done Sent Done false
21019 6230 Acked Read 5 Done Sent Done false
21020 6231 Acked Read 6 Done Sent Done false
21021 6232 Acked Write 6 Done Sent Done false
21022 6233 Acked Write 5 Done Sent Done false
21023 6234 Acked Read 3 Done Sent Done false
21024 6235 Acked Read 3 Done Sent Done false
21025 6236 Acked Read 10 Done Sent Done false
21026 6237 Acked Write 7 Done Sent Done false
21027 6238 Acked Flush 0 Done Sent Done false
21028 6239 Acked Read 1 Done Sent Done false
21029 6240 Acked Write 2 Done Sent Done false
21030 6241 Acked Write 8 Done Sent Done false
21031 6242 Acked Write 10 Done Sent Done false
21032 6243 Acked Write 7 Done Sent Done false
21033 6244 Acked Read 3 Done Sent Done false
21034 6245 Acked Read 9 Done Sent Done false
21035 6246 Acked Read 7 Done Sent Done false
21036 6247 Acked Read 8 Done Sent Done false
21037 6248 Acked Write 1 Done Sent Done false
21038 6249 Acked Read 5 Done Sent Done false
21039 6250 Acked Write 3 Done Sent Done false
21040 6251 Acked Flush 0 Done Sent Done false
21041 6252 Acked Read 1 Done Sent Done false
21042 6253 Acked Flush 0 Done Sent Done false
21043 6254 Acked Read 3 Done Sent Done false
21044 6255 Acked Write 10 Done Sent Done false
21045 6256 Acked Read 10 Done Sent Done false
21046 6257 Acked Read 4 Done Sent Done false
21047 6258 Acked Write 6 Done Sent Done false
21048 6259 Acked Read 4 Done Sent Done false
21049 6260 Acked Read 7 Done Sent Done false
21050 6261 Acked Read 5 Done Sent Done false
21051 6262 Acked Read 3 Done Sent Done false
21052 6263 Acked Read 10 Done Sent Done false
21053 6264 Acked Read 5 Done Sent Done false
21054 6265 Acked Read 5 Done Sent Done false
21055 6266 Acked Read 3 Done Sent Done false
21056 6267 Acked Read 7 Done Sent Done false
21057 6268 Acked Read 4 Done Sent Done false
21058 6269 Acked Read 6 Done Sent Done false
21059 6270 Acked Write 9 Done Sent Done false
21060 6271 Acked Read 7 Done Sent Done false
21061 6272 Acked Read 2 Done Sent Done false
21062 6273 Acked Write 10 Done Sent Done false
21063 6274 Acked Write 6 Done Sent Done false
21064 6275 Acked Read 7 Done Sent Done false
21065 6276 Acked Read 8 Done Sent Done false
21066 6277 Acked Flush 0 Done Sent Done false
21067 6278 Acked Read 10 Done Sent Done false
21068 6279 Acked Write 9 Done Sent Done false
21069 6280 Acked Read 6 Done Sent Done false
21070 6281 Acked Write 9 Done Sent Done false
21071 6282 Acked Write 10 Done Sent Done false
21072 6283 Acked Read 7 Done Sent Done false
21073 6284 Acked Read 6 Done Sent Done false
21074 6285 Acked Read 7 Done Sent Done false
21075 6286 Acked Write 4 Done Sent Done false
21076 6287 Acked Flush 0 Done Sent Done false
21077 6288 Acked Flush 0 Done Sent Done false
21078 6289 Acked Write 2 Done Sent Done false
21079 6290 Acked Flush 0 Done Sent Done false
21080 6291 Acked Write 2 Done Sent Done false
21081 6292 Acked Write 6 Done Sent Done false
21082 6293 Acked Read 8 Done Sent Done false
21083 6294 Acked Write 6 Done Sent Done false
21084 6295 Acked Flush 0 Done Sent Done false
21085 6296 Acked Flush 0 Done Sent Done false
21086 6297 Acked Read 4 Done Sent Done false
21087 6298 Acked Write 7 Done Sent Done false
21088 6299 Acked Read 1 Done Sent Done false
21089 6300 Acked Write 6 Done Sent Done false
21090 6301 Acked Write 8 Done Sent Done false
21091 6302 Acked Write 4 Done Sent Done false
21092 6303 Acked Write 10 Done Sent Done false
21093 6304 Acked Read 10 Done Sent Done false
21094 6305 Acked Write 4 Done Sent Done false
21095 6306 Acked Flush 0 Done Sent Done false
21096 6307 Acked Read 3 Done Sent Done false
21097 6308 Acked Read 1 Done Sent Done false
21098 6309 Acked Flush 0 Done Sent Done false
21099 6310 Acked Write 7 Done Sent Done false
21100 6311 Acked Read 9 Done Sent Done false
21101 6312 Acked Flush 0 Done Sent Done false
21102 6313 Acked Write 6 Done Sent Done false
21103 6314 Acked Write 4 Done Sent Done false
21104 6315 Acked Flush 0 Done Sent Done false
21105 6316 Acked Read 10 Done Sent Done false
21106 6317 Acked Read 5 Done Sent Done false
21107 6318 Acked Write 10 Done Sent Done false
21108 6319 Acked Write 3 Done Sent Done false
21109 6320 Acked Read 3 Done Sent Done false
21110 6321 Acked Read 8 Done Sent Done false
21111 6322 Acked Flush 0 Done Sent Done false
21112 6323 Acked Write 10 Done Sent Done false
21113 6324 Acked Write 3 Done Sent Done false
21114 6325 Acked Read 5 Done Sent Done false
21115 6326 Acked Read 6 Done Sent Done false
21116 6327 Acked Write 7 Done Sent Done false
21117 6328 Acked Read 3 Done Sent Done false
21118 6329 Acked Read 5 Done Sent Done false
21119 6330 Acked Read 5 Done Sent Done false
21120 6331 Acked Flush 0 Done Sent Done false
21121 6332 Acked Flush 0 Done Sent Done false
21122 6333 Acked Write 4 Done Sent Done false
21123 6334 Acked Read 2 Done Sent Done false
21124 6335 Acked Write 1 Done Sent Done false
21125 6336 Acked Read 1 Done Sent Done false
21126 6337 Acked Read 7 Done Sent Done false
21127 6338 Acked Read 5 Done Sent Done false
21128 6339 Acked Write 2 Done Sent Done false
21129 6340 Acked Write 4 Done Sent Done false
21130 6341 Acked Read 9 Done Sent Done false
21131 6342 Acked Write 4 Done Sent Done false
21132 6343 Acked Read 1 Done Sent Done false
21133 6344 Acked Read 1 Done Sent Done false
21134 6345 Acked Write 4 Done Sent Done false
21135 6346 Acked Write 9 Done Sent Done false
21136 6347 Acked Read 6 Done Sent Done false
21137 6348 Acked Read 8 Done Sent Done false
21138 6349 Acked Read 3 Done Sent Done false
21139 6350 Acked Write 3 Done Sent Done false
21140 6351 Acked Read 6 Done Sent Done false
21141 6352 Acked Read 6 Done Sent Done false
21142 6353 Acked Write 3 Done Sent Done false
21143 6354 Acked Read 4 Done Sent Done false
21144 6355 Acked Flush 0 Done Sent Done false
21145 6356 Acked Read 5 Done Sent Done false
21146 6357 Acked Read 3 Done Sent Done false
21147 6358 Acked Write 8 Done Sent Done false
21148 6359 Acked Read 3 Done Sent Done false
21149 6360 Acked Read 8 Done Sent Done false
21150 6361 Acked Read 8 Done Sent Done false
21151 6362 Acked Read 8 Done Sent Done false
21152 6363 Acked Write 2 Done Sent Done false
21153 6364 Acked Read 2 Done Sent Done false
21154 6365 Acked Read 10 Done Sent Done false
21155 6366 Acked Read 6 Done Sent Done false
21156 6367 Acked Read 3 Done Sent Done false
21157 6368 Acked Write 6 Done Sent Done false
21158 6369 Acked Write 10 Done Sent Done false
21159 6370 Acked Flush 0 Done Sent Done false
21160 6371 Acked Write 4 Done Sent Done false
21161 6372 Acked Read 9 Done Sent Done false
21162 6373 Acked Read 5 Done Sent Done false
21163 6374 Acked Write 7 Done Sent Done false
21164 6375 Acked Read 4 Done Sent Done false
21165 6376 Acked Write 10 Done Sent Done false
21166 6377 Acked Read 5 Done Sent Done false
21167 6378 Acked Flush 0 Done Sent Done false
21168 6379 Acked Write 3 Done Sent Done false
21169 6380 Acked Read 6 Done Sent Done false
21170 6381 Acked Write 2 Done Sent Done false
21171 6382 Acked Read 5 Done Sent Done false
21172 6383 Acked Write 5 Done Sent Done false
21173 6384 Acked Flush 0 Done Sent Done false
21174 6385 Acked Read 5 Done Sent Done false
21175 6386 Acked Write 10 Done Sent Done false
21176 6387 Acked Read 9 Done Sent Done false
21177 6388 Acked Read 9 Done Sent Done false
21178 6389 Acked Write 8 Done Sent Done false
21179 6390 Acked Write 5 Done Sent Done false
21180 6391 Acked Read 2 Done Sent Done false
21181 6392 Acked Read 7 Done Sent Done false
21182 6393 Acked Read 3 Done Sent Done false
21183 6394 Acked Read 9 Done Sent Done false
21184 6395 Acked Write 8 Done Sent Done false
21185 6396 Acked Read 1 Done Sent Done false
21186 6397 Acked Flush 0 Done Sent Done false
21187 6398 Acked Flush 0 Done Sent Done false
21188 6399 Acked Flush 0 Done Sent Done false
21189 6400 Acked Read 3 Done Sent Done false
21190 6401 Acked Write 5 Done Sent Done false
21191 6402 Acked Write 7 Done Sent Done false
21192 6403 Acked Read 10 Done Sent Done false
21193 6404 Acked Read 6 Done Sent Done false
21194 6405 Acked Read 5 Done Sent Done false
21195 6406 Acked Write 2 Done Sent Done false
21196 6407 Acked Write 7 Done Sent Done false
21197 6408 Acked Read 4 Done Sent Done false
21198 6409 Acked Read 7 Done Sent Done false
21199 6410 Acked Write 7 Done Sent Done false
21200 6411 Acked Write 9 Done Sent Done false
21201 6412 Acked Read 8 Done Sent Done false
21202 6413 Acked Write 6 Done Sent Done false
21203 6414 Acked Write 6 Done Sent Done false
21204 6415 Acked Read 3 Done Sent Done false
21205 6416 Acked Write 9 Done Sent Done false
21206 6417 Acked Write 3 Done Sent Done false
21207 6418 Acked Read 10 Done Sent Done false
21208 6419 Acked Write 1 Done Sent Done false
21209 6420 Acked Read 5 Done Sent Done false
21210 6421 Acked Read 7 Done Sent Done false
21211 6422 Acked Flush 0 Done Sent Done false
21212 6423 Acked Write 9 Done Sent Done false
21213 6424 Acked Write 1 Done Sent Done false
21214 6425 Acked Write 8 Done Sent Done false
21215 6426 Acked Read 2 Done Sent Done false
21216 6427 Acked Read 10 Done Sent Done false
21217 6428 Acked Write 2 Done Sent Done false
21218 6429 Acked Write 2 Done Sent Done false
21219 6430 Acked Flush 0 Done Sent Done false
21220 6431 Acked Read 9 Done Sent Done false
21221 6432 Acked Read 1 Done Sent Done false
21222 6433 Acked Write 3 Done Sent Done false
21223 6434 Acked Read 5 Done Sent Done false
21224 STATES DS:0 DS:1 DS:2 TOTAL
21225 Sent 0 301 0 301
21226 Done 301 0 301 602
21227 Skipped 0 0 0 0
21228 Error 0 0 0 0
21229 Last Flush: 6430 6133 6430
21230 Downstairs last five completed: 6133 6132 6131 6130 6129
21231 Upstairs last five completed: 6434 6433 6432 6431 6430
21232 CLIENT: Up:0 ds:301 act:2
21233 ----------------------------------------------------------------
21234 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21235 States: Active Offline Active
21236 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21237 6134 Acked Read 3 Done Sent Done false
21238 6135 Acked Write 6 Done Sent Done false
21239 6136 Acked Write 4 Done Sent Done false
21240 6137 Acked Write 3 Done Sent Done false
21241 6138 Acked Read 6 Done Sent Done false
21242 6139 Acked Read 2 Done Sent Done false
21243 6140 Acked Read 3 Done Sent Done false
21244 6141 Acked Read 2 Done Sent Done false
21245 6142 Acked Read 7 Done Sent Done false
21246 6143 Acked Read 1 Done Sent Done false
21247 6144 Acked Read 10 Done Sent Done false
21248 6145 Acked Write 4 Done Sent Done false
21249 6146 Acked Write 10 Done Sent Done false
21250 6147 Acked Read 3 Done Sent Done false
21251 6148 Acked Read 8 Done Sent Done false
21252 6149 Acked Read 10 Done Sent Done false
21253 6150 Acked Write 9 Done Sent Done false
21254 6151 Acked Write 3 Done Sent Done false
21255 6152 Acked Write 8 Done Sent Done false
21256 6153 Acked Write 4 Done Sent Done false
21257 6154 Acked Read 1 Done Sent Done false
21258 6155 Acked Read 1 Done Sent Done false
21259 6156 Acked Write 1 Done Sent Done false
21260 6157 Acked Write 1 Done Sent Done false
21261 6158 Acked Read 3 Done Sent Done false
21262 6159 Acked Read 3 Done Sent Done false
21263 6160 Acked Read 10 Done Sent Done false
21264 6161 Acked Write 9 Done Sent Done false
21265 6162 Acked Write 9 Done Sent Done false
21266 6163 Acked Read 10 Done Sent Done false
21267 6164 Acked Write 5 Done Sent Done false
21268 6165 Acked Write 9 Done Sent Done false
21269 6166 Acked Write 3 Done Sent Done false
21270 6167 Acked Write 1 Done Sent Done false
21271 6168 Acked Write 7 Done Sent Done false
21272 6169 Acked Write 3 Done Sent Done false
21273 6170 Acked Read 7 Done Sent Done false
21274 6171 Acked Flush 0 Done Sent Done false
21275 6172 Acked Flush 0 Done Sent Done false
21276 6173 Acked Read 2 Done Sent Done false
21277 6174 Acked Write 4 Done Sent Done false
21278 6175 Acked Write 10 Done Sent Done false
21279 6176 Acked Read 4 Done Sent Done false
21280 6177 Acked Write 2 Done Sent Done false
21281 6178 Acked Read 2 Done Sent Done false
21282 6179 Acked Flush 0 Done Sent Done false
21283 6180 Acked Read 2 Done Sent Done false
21284 6181 Acked Read 2 Done Sent Done false
21285 6182 Acked Read 7 Done Sent Done false
21286 6183 Acked Write 5 Done Sent Done false
21287 6184 Acked Write 5 Done Sent Done false
21288 6185 Acked Read 7 Done Sent Done false
21289 6186 Acked Write 9 Done Sent Done false
21290 6187 Acked Write 5 Done Sent Done false
21291 6188 Acked Read 3 Done Sent Done false
21292 6189 Acked Read 2 Done Sent Done false
21293 6190 Acked Write 5 Done Sent Done false
21294 6191 Acked Flush 0 Done Sent Done false
21295 6192 Acked Read 10 Done Sent Done false
21296 6193 Acked Read 1 Done Sent Done false
21297 6194 Acked Read 7 Done Sent Done false
21298 6195 Acked Write 2 Done Sent Done false
21299 6196 Acked Read 5 Done Sent Done false
21300 6197 Acked Read 6 Done Sent Done false
21301 6198 Acked Read 4 Done Sent Done false
21302 6199 Acked Read 2 Done Sent Done false
21303 6200 Acked Write 9 Done Sent Done false
21304 6201 Acked Flush 0 Done Sent Done false
21305 6202 Acked Write 7 Done Sent Done false
21306 6203 Acked Write 1 Done Sent Done false
21307 6204 Acked Read 9 Done Sent Done false
21308 6205 Acked Write 8 Done Sent Done false
21309 6206 Acked Read 1 Done Sent Done false
21310 6207 Acked Flush 0 Done Sent Done false
21311 6208 Acked Flush 0 Done Sent Done false
21312 6209 Acked Write 5 Done Sent Done false
21313 6210 Acked Read 2 Done Sent Done false
21314 6211 Acked Write 1 Done Sent Done false
21315 6212 Acked Read 10 Done Sent Done false
21316 6213 Acked Write 10 Done Sent Done false
21317 6214 Acked Read 6 Done Sent Done false
21318 6215 Acked Read 3 Done Sent Done false
21319 6216 Acked Write 9 Done Sent Done false
21320 6217 Acked Flush 0 Done Sent Done false
21321 6218 Acked Flush 0 Done Sent Done false
21322 6219 Acked Read 2 Done Sent Done false
21323 6220 Acked Read 2 Done Sent Done false
21324 6221 Acked Flush 0 Done Sent Done false
21325 6222 Acked Read 4 Done Sent Done false
21326 6223 Acked Write 8 Done Sent Done false
21327 6224 Acked Read 1 Done Sent Done false
21328 6225 Acked Read 3 Done Sent Done false
21329 6226 Acked Write 10 Done Sent Done false
21330 6227 Acked Write 6 Done Sent Done false
21331 6228 Acked Read 7 Done Sent Done false
21332 6229 Acked Read 10 Done Sent Done false
21333 6230 Acked Read 5 Done Sent Done false
21334 6231 Acked Read 6 Done Sent Done false
21335 6232 Acked Write 6 Done Sent Done false
21336 6233 Acked Write 5 Done Sent Done false
21337 6234 Acked Read 3 Done Sent Done false
21338 6235 Acked Read 3 Done Sent Done false
21339 6236 Acked Read 10 Done Sent Done false
21340 6237 Acked Write 7 Done Sent Done false
21341 6238 Acked Flush 0 Done Sent Done false
21342 6239 Acked Read 1 Done Sent Done false
21343 6240 Acked Write 2 Done Sent Done false
21344 6241 Acked Write 8 Done Sent Done false
21345 6242 Acked Write 10 Done Sent Done false
21346 6243 Acked Write 7 Done Sent Done false
21347 6244 Acked Read 3 Done Sent Done false
21348 6245 Acked Read 9 Done Sent Done false
21349 6246 Acked Read 7 Done Sent Done false
21350 6247 Acked Read 8 Done Sent Done false
21351 6248 Acked Write 1 Done Sent Done false
21352 6249 Acked Read 5 Done Sent Done false
21353 6250 Acked Write 3 Done Sent Done false
21354 6251 Acked Flush 0 Done Sent Done false
21355 6252 Acked Read 1 Done Sent Done false
21356 6253 Acked Flush 0 Done Sent Done false
21357 6254 Acked Read 3 Done Sent Done false
21358 6255 Acked Write 10 Done Sent Done false
21359 6256 Acked Read 10 Done Sent Done false
21360 6257 Acked Read 4 Done Sent Done false
21361 6258 Acked Write 6 Done Sent Done false
21362 6259 Acked Read 4 Done Sent Done false
21363 6260 Acked Read 7 Done Sent Done false
21364 6261 Acked Read 5 Done Sent Done false
21365 6262 Acked Read 3 Done Sent Done false
21366 6263 Acked Read 10 Done Sent Done false
21367 6264 Acked Read 5 Done Sent Done false
21368 6265 Acked Read 5 Done Sent Done false
21369 6266 Acked Read 3 Done Sent Done false
21370 6267 Acked Read 7 Done Sent Done false
21371 6268 Acked Read 4 Done Sent Done false
21372 6269 Acked Read 6 Done Sent Done false
21373 6270 Acked Write 9 Done Sent Done false
21374 6271 Acked Read 7 Done Sent Done false
21375 6272 Acked Read 2 Done Sent Done false
21376 6273 Acked Write 10 Done Sent Done false
21377 6274 Acked Write 6 Done Sent Done false
21378 6275 Acked Read 7 Done Sent Done false
21379 6276 Acked Read 8 Done Sent Done false
21380 6277 Acked Flush 0 Done Sent Done false
21381 6278 Acked Read 10 Done Sent Done false
21382 6279 Acked Write 9 Done Sent Done false
21383 6280 Acked Read 6 Done Sent Done false
21384 6281 Acked Write 9 Done Sent Done false
21385 6282 Acked Write 10 Done Sent Done false
21386 6283 Acked Read 7 Done Sent Done false
21387 6284 Acked Read 6 Done Sent Done false
21388 6285 Acked Read 7 Done Sent Done false
21389 6286 Acked Write 4 Done Sent Done false
21390 6287 Acked Flush 0 Done Sent Done false
21391 6288 Acked Flush 0 Done Sent Done false
21392 6289 Acked Write 2 Done Sent Done false
21393 6290 Acked Flush 0 Done Sent Done false
21394 6291 Acked Write 2 Done Sent Done false
21395 6292 Acked Write 6 Done Sent Done false
21396 6293 Acked Read 8 Done Sent Done false
21397 6294 Acked Write 6 Done Sent Done false
21398 6295 Acked Flush 0 Done Sent Done false
21399 6296 Acked Flush 0 Done Sent Done false
21400 6297 Acked Read 4 Done Sent Done false
21401 6298 Acked Write 7 Done Sent Done false
21402 6299 Acked Read 1 Done Sent Done false
21403 6300 Acked Write 6 Done Sent Done false
21404 6301 Acked Write 8 Done Sent Done false
21405 6302 Acked Write 4 Done Sent Done false
21406 6303 Acked Write 10 Done Sent Done false
21407 6304 Acked Read 10 Done Sent Done false
21408 6305 Acked Write 4 Done Sent Done false
21409 6306 Acked Flush 0 Done Sent Done false
21410 6307 Acked Read 3 Done Sent Done false
21411 6308 Acked Read 1 Done Sent Done false
21412 6309 Acked Flush 0 Done Sent Done false
21413 6310 Acked Write 7 Done Sent Done false
21414 6311 Acked Read 9 Done Sent Done false
21415 6312 Acked Flush 0 Done Sent Done false
21416 6313 Acked Write 6 Done Sent Done false
21417 6314 Acked Write 4 Done Sent Done false
21418 6315 Acked Flush 0 Done Sent Done false
21419 6316 Acked Read 10 Done Sent Done false
21420 6317 Acked Read 5 Done Sent Done false
21421 6318 Acked Write 10 Done Sent Done false
21422 6319 Acked Write 3 Done Sent Done false
21423 6320 Acked Read 3 Done Sent Done false
21424 6321 Acked Read 8 Done Sent Done false
21425 6322 Acked Flush 0 Done Sent Done false
21426 6323 Acked Write 10 Done Sent Done false
21427 6324 Acked Write 3 Done Sent Done false
21428 6325 Acked Read 5 Done Sent Done false
21429 6326 Acked Read 6 Done Sent Done false
21430 6327 Acked Write 7 Done Sent Done false
21431 6328 Acked Read 3 Done Sent Done false
21432 6329 Acked Read 5 Done Sent Done false
21433 6330 Acked Read 5 Done Sent Done false
21434 6331 Acked Flush 0 Done Sent Done false
21435 6332 Acked Flush 0 Done Sent Done false
21436 6333 Acked Write 4 Done Sent Done false
21437 6334 Acked Read 2 Done Sent Done false
21438 6335 Acked Write 1 Done Sent Done false
21439 6336 Acked Read 1 Done Sent Done false
21440 6337 Acked Read 7 Done Sent Done false
21441 6338 Acked Read 5 Done Sent Done false
21442 6339 Acked Write 2 Done Sent Done false
21443 6340 Acked Write 4 Done Sent Done false
21444 6341 Acked Read 9 Done Sent Done false
21445 6342 Acked Write 4 Done Sent Done false
21446 6343 Acked Read 1 Done Sent Done false
21447 6344 Acked Read 1 Done Sent Done false
21448 6345 Acked Write 4 Done Sent Done false
21449 6346 Acked Write 9 Done Sent Done false
21450 6347 Acked Read 6 Done Sent Done false
21451 6348 Acked Read 8 Done Sent Done false
21452 6349 Acked Read 3 Done Sent Done false
21453 6350 Acked Write 3 Done Sent Done false
21454 6351 Acked Read 6 Done Sent Done false
21455 6352 Acked Read 6 Done Sent Done false
21456 6353 Acked Write 3 Done Sent Done false
21457 6354 Acked Read 4 Done Sent Done false
21458 6355 Acked Flush 0 Done Sent Done false
21459 6356 Acked Read 5 Done Sent Done false
21460 6357 Acked Read 3 Done Sent Done false
21461 6358 Acked Write 8 Done Sent Done false
21462 6359 Acked Read 3 Done Sent Done false
21463 6360 Acked Read 8 Done Sent Done false
21464 6361 Acked Read 8 Done Sent Done false
21465 6362 Acked Read 8 Done Sent Done false
21466 6363 Acked Write 2 Done Sent Done false
21467 6364 Acked Read 2 Done Sent Done false
21468 6365 Acked Read 10 Done Sent Done false
21469 6366 Acked Read 6 Done Sent Done false
21470 6367 Acked Read 3 Done Sent Done false
21471 6368 Acked Write 6 Done Sent Done false
21472 6369 Acked Write 10 Done Sent Done false
21473 6370 Acked Flush 0 Done Sent Done false
21474 6371 Acked Write 4 Done Sent Done false
21475 6372 Acked Read 9 Done Sent Done false
21476 6373 Acked Read 5 Done Sent Done false
21477 6374 Acked Write 7 Done Sent Done false
21478 6375 Acked Read 4 Done Sent Done false
21479 6376 Acked Write 10 Done Sent Done false
21480 6377 Acked Read 5 Done Sent Done false
21481 6378 Acked Flush 0 Done Sent Done false
21482 6379 Acked Write 3 Done Sent Done false
21483 6380 Acked Read 6 Done Sent Done false
21484 6381 Acked Write 2 Done Sent Done false
21485 6382 Acked Read 5 Done Sent Done false
21486 6383 Acked Write 5 Done Sent Done false
21487 6384 Acked Flush 0 Done Sent Done false
21488 6385 Acked Read 5 Done Sent Done false
21489 6386 Acked Write 10 Done Sent Done false
21490 6387 Acked Read 9 Done Sent Done false
21491 6388 Acked Read 9 Done Sent Done false
21492 6389 Acked Write 8 Done Sent Done false
21493 6390 Acked Write 5 Done Sent Done false
21494 6391 Acked Read 2 Done Sent Done false
21495 6392 Acked Read 7 Done Sent Done false
21496 6393 Acked Read 3 Done Sent Done false
21497 6394 Acked Read 9 Done Sent Done false
21498 6395 Acked Write 8 Done Sent Done false
21499 6396 Acked Read 1 Done Sent Done false
21500 6397 Acked Flush 0 Done Sent Done false
21501 6398 Acked Flush 0 Done Sent Done false
21502 6399 Acked Flush 0 Done Sent Done false
21503 6400 Acked Read 3 Done Sent Done false
21504 6401 Acked Write 5 Done Sent Done false
21505 6402 Acked Write 7 Done Sent Done false
21506 6403 Acked Read 10 Done Sent Done false
21507 6404 Acked Read 6 Done Sent Done false
21508 6405 Acked Read 5 Done Sent Done false
21509 6406 Acked Write 2 Done Sent Done false
21510 6407 Acked Write 7 Done Sent Done false
21511 6408 Acked Read 4 Done Sent Done false
21512 6409 Acked Read 7 Done Sent Done false
21513 6410 Acked Write 7 Done Sent Done false
21514 6411 Acked Write 9 Done Sent Done false
21515 6412 Acked Read 8 Done Sent Done false
21516 6413 Acked Write 6 Done Sent Done false
21517 6414 Acked Write 6 Done Sent Done false
21518 6415 Acked Read 3 Done Sent Done false
21519 6416 Acked Write 9 Done Sent Done false
21520 6417 Acked Write 3 Done Sent Done false
21521 6418 Acked Read 10 Done Sent Done false
21522 6419 Acked Write 1 Done Sent Done false
21523 6420 Acked Read 5 Done Sent Done false
21524 6421 Acked Read 7 Done Sent Done false
21525 6422 Acked Flush 0 Done Sent Done false
21526 6423 Acked Write 9 Done Sent Done false
21527 6424 Acked Write 1 Done Sent Done false
21528 6425 Acked Write 8 Done Sent Done false
21529 6426 Acked Read 2 Done Sent Done false
21530 6427 Acked Read 10 Done Sent Done false
21531 6428 Acked Write 2 Done Sent Done false
21532 6429 Acked Write 2 Done Sent Done false
21533 6430 Acked Flush 0 Done Sent Done false
21534 6431 Acked Read 9 Done Sent Done false
21535 6432 Acked Read 1 Done Sent Done false
21536 6433 Acked Write 3 Done Sent Done false
21537 6434 Acked Read 5 Done Sent Done false
21538 6435 Acked Flush 0 Done Sent Done false
21539 STATES DS:0 DS:1 DS:2 TOTAL
21540 Sent 0 302 0 302
21541 Done 302 0 302 604
21542 Skipped 0 0 0 0
21543 Error 0 0 0 0
21544 Last Flush: 6435 6133 6435
21545 Downstairs last five completed: 6133 6132 6131 6130 6129
21546 Upstairs last five completed: 6435 6434 6433 6432 6431
21547 CLIENT: Up:0 ds:302 act:2
215482024-12-11T23:45:00.700ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
215492024-12-11T23:45:00.700ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
215502024-12-11T23:45:00.700ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
215512024-12-11T23:45:00.700ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
215522024-12-11T23:45:00.700ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
21553 [8820] got start action
21554 [8820] I am going to restart
21555 Starting downstairs at port 8820
21556 Make output file at "/tmp/dsc/downstairs-8820.txt"
21557 [8820][1] reports Starting
21558 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
21559 [8820][1] reports Running
21560 ----------------------------------------------------------------
21561 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21562 States: Active Offline Active
21563 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21564 6134 Acked Read 3 Done Sent Done false
21565 6135 Acked Write 6 Done Sent Done false
21566 6136 Acked Write 4 Done Sent Done false
21567 6137 Acked Write 3 Done Sent Done false
21568 6138 Acked Read 6 Done Sent Done false
21569 6139 Acked Read 2 Done Sent Done false
21570 6140 Acked Read 3 Done Sent Done false
21571 6141 Acked Read 2 Done Sent Done false
21572 6142 Acked Read 7 Done Sent Done false
21573 6143 Acked Read 1 Done Sent Done false
21574 6144 Acked Read 10 Done Sent Done false
21575 6145 Acked Write 4 Done Sent Done false
21576 6146 Acked Write 10 Done Sent Done false
21577 6147 Acked Read 3 Done Sent Done false
21578 6148 Acked Read 8 Done Sent Done false
21579 6149 Acked Read 10 Done Sent Done false
21580 6150 Acked Write 9 Done Sent Done false
21581 6151 Acked Write 3 Done Sent Done false
21582 6152 Acked Write 8 Done Sent Done false
21583 6153 Acked Write 4 Done Sent Done false
21584 6154 Acked Read 1 Done Sent Done false
21585 6155 Acked Read 1 Done Sent Done false
21586 6156 Acked Write 1 Done Sent Done false
21587 6157 Acked Write 1 Done Sent Done false
21588 6158 Acked Read 3 Done Sent Done false
21589 6159 Acked Read 3 Done Sent Done false
21590 6160 Acked Read 10 Done Sent Done false
21591 6161 Acked Write 9 Done Sent Done false
21592 6162 Acked Write 9 Done Sent Done false
21593 6163 Acked Read 10 Done Sent Done false
21594 6164 Acked Write 5 Done Sent Done false
21595 6165 Acked Write 9 Done Sent Done false
21596 6166 Acked Write 3 Done Sent Done false
21597 6167 Acked Write 1 Done Sent Done false
21598 6168 Acked Write 7 Done Sent Done false
21599 6169 Acked Write 3 Done Sent Done false
21600 6170 Acked Read 7 Done Sent Done false
21601 6171 Acked Flush 0 Done Sent Done false
21602 6172 Acked Flush 0 Done Sent Done false
21603 6173 Acked Read 2 Done Sent Done false
21604 6174 Acked Write 4 Done Sent Done false
21605 6175 Acked Write 10 Done Sent Done false
21606 6176 Acked Read 4 Done Sent Done false
21607 6177 Acked Write 2 Done Sent Done false
21608 6178 Acked Read 2 Done Sent Done false
21609 6179 Acked Flush 0 Done Sent Done false
21610 6180 Acked Read 2 Done Sent Done false
21611 6181 Acked Read 2 Done Sent Done false
21612 6182 Acked Read 7 Done Sent Done false
21613 6183 Acked Write 5 Done Sent Done false
21614 6184 Acked Write 5 Done Sent Done false
21615 6185 Acked Read 7 Done Sent Done false
21616 6186 Acked Write 9 Done Sent Done false
21617 6187 Acked Write 5 Done Sent Done false
21618 6188 Acked Read 3 Done Sent Done false
21619 6189 Acked Read 2 Done Sent Done false
21620 6190 Acked Write 5 Done Sent Done false
21621 6191 Acked Flush 0 Done Sent Done false
21622 6192 Acked Read 10 Done Sent Done false
21623 6193 Acked Read 1 Done Sent Done false
21624 6194 Acked Read 7 Done Sent Done false
21625 6195 Acked Write 2 Done Sent Done false
21626 6196 Acked Read 5 Done Sent Done false
21627 6197 Acked Read 6 Done Sent Done false
21628 6198 Acked Read 4 Done Sent Done false
21629 6199 Acked Read 2 Done Sent Done false
21630 6200 Acked Write 9 Done Sent Done false
21631 6201 Acked Flush 0 Done Sent Done false
21632 6202 Acked Write 7 Done Sent Done false
21633 6203 Acked Write 1 Done Sent Done false
21634 6204 Acked Read 9 Done Sent Done false
21635 6205 Acked Write 8 Done Sent Done false
21636 6206 Acked Read 1 Done Sent Done false
21637 6207 Acked Flush 0 Done Sent Done false
21638 6208 Acked Flush 0 Done Sent Done false
21639 6209 Acked Write 5 Done Sent Done false
21640 6210 Acked Read 2 Done Sent Done false
21641 6211 Acked Write 1 Done Sent Done false
21642 6212 Acked Read 10 Done Sent Done false
21643 6213 Acked Write 10 Done Sent Done false
21644 6214 Acked Read 6 Done Sent Done false
21645 6215 Acked Read 3 Done Sent Done false
21646 6216 Acked Write 9 Done Sent Done false
21647 6217 Acked Flush 0 Done Sent Done false
21648 6218 Acked Flush 0 Done Sent Done false
21649 6219 Acked Read 2 Done Sent Done false
21650 6220 Acked Read 2 Done Sent Done false
21651 6221 Acked Flush 0 Done Sent Done false
21652 6222 Acked Read 4 Done Sent Done false
21653 6223 Acked Write 8 Done Sent Done false
21654 6224 Acked Read 1 Done Sent Done false
21655 6225 Acked Read 3 Done Sent Done false
21656 6226 Acked Write 10 Done Sent Done false
21657 6227 Acked Write 6 Done Sent Done false
21658 6228 Acked Read 7 Done Sent Done false
21659 6229 Acked Read 10 Done Sent Done false
21660 6230 Acked Read 5 Done Sent Done false
21661 6231 Acked Read 6 Done Sent Done false
21662 6232 Acked Write 6 Done Sent Done false
21663 6233 Acked Write 5 Done Sent Done false
21664 6234 Acked Read 3 Done Sent Done false
21665 6235 Acked Read 3 Done Sent Done false
21666 6236 Acked Read 10 Done Sent Done false
21667 6237 Acked Write 7 Done Sent Done false
21668 6238 Acked Flush 0 Done Sent Done false
21669 6239 Acked Read 1 Done Sent Done false
21670 6240 Acked Write 2 Done Sent Done false
21671 6241 Acked Write 8 Done Sent Done false
21672 6242 Acked Write 10 Done Sent Done false
21673 6243 Acked Write 7 Done Sent Done false
21674 6244 Acked Read 3 Done Sent Done false
21675 6245 Acked Read 9 Done Sent Done false
21676 6246 Acked Read 7 Done Sent Done false
21677 6247 Acked Read 8 Done Sent Done false
21678 6248 Acked Write 1 Done Sent Done false
21679 6249 Acked Read 5 Done Sent Done false
21680 6250 Acked Write 3 Done Sent Done false
21681 6251 Acked Flush 0 Done Sent Done false
21682 6252 Acked Read 1 Done Sent Done false
21683 6253 Acked Flush 0 Done Sent Done false
21684 6254 Acked Read 3 Done Sent Done false
21685 6255 Acked Write 10 Done Sent Done false
21686 6256 Acked Read 10 Done Sent Done false
21687 6257 Acked Read 4 Done Sent Done false
21688 6258 Acked Write 6 Done Sent Done false
21689 6259 Acked Read 4 Done Sent Done false
21690 6260 Acked Read 7 Done Sent Done false
21691 6261 Acked Read 5 Done Sent Done false
21692 6262 Acked Read 3 Done Sent Done false
21693 6263 Acked Read 10 Done Sent Done false
21694 6264 Acked Read 5 Done Sent Done false
21695 6265 Acked Read 5 Done Sent Done false
21696 6266 Acked Read 3 Done Sent Done false
21697 6267 Acked Read 7 Done Sent Done false
21698 6268 Acked Read 4 Done Sent Done false
21699 6269 Acked Read 6 Done Sent Done false
21700 6270 Acked Write 9 Done Sent Done false
21701 6271 Acked Read 7 Done Sent Done false
21702 6272 Acked Read 2 Done Sent Done false
21703 6273 Acked Write 10 Done Sent Done false
21704 6274 Acked Write 6 Done Sent Done false
21705 6275 Acked Read 7 Done Sent Done false
21706 6276 Acked Read 8 Done Sent Done false
21707 6277 Acked Flush 0 Done Sent Done false
21708 6278 Acked Read 10 Done Sent Done false
21709 6279 Acked Write 9 Done Sent Done false
21710 6280 Acked Read 6 Done Sent Done false
21711 6281 Acked Write 9 Done Sent Done false
21712 6282 Acked Write 10 Done Sent Done false
21713 6283 Acked Read 7 Done Sent Done false
21714 6284 Acked Read 6 Done Sent Done false
21715 6285 Acked Read 7 Done Sent Done false
21716 6286 Acked Write 4 Done Sent Done false
21717 6287 Acked Flush 0 Done Sent Done false
21718 6288 Acked Flush 0 Done Sent Done false
21719 6289 Acked Write 2 Done Sent Done false
21720 6290 Acked Flush 0 Done Sent Done false
21721 6291 Acked Write 2 Done Sent Done false
21722 6292 Acked Write 6 Done Sent Done false
21723 6293 Acked Read 8 Done Sent Done false
21724 6294 Acked Write 6 Done Sent Done false
21725 6295 Acked Flush 0 Done Sent Done false
21726 6296 Acked Flush 0 Done Sent Done false
21727 6297 Acked Read 4 Done Sent Done false
21728 6298 Acked Write 7 Done Sent Done false
21729 6299 Acked Read 1 Done Sent Done false
21730 6300 Acked Write 6 Done Sent Done false
21731 6301 Acked Write 8 Done Sent Done false
21732 6302 Acked Write 4 Done Sent Done false
21733 6303 Acked Write 10 Done Sent Done false
21734 6304 Acked Read 10 Done Sent Done false
21735 6305 Acked Write 4 Done Sent Done false
21736 6306 Acked Flush 0 Done Sent Done false
21737 6307 Acked Read 3 Done Sent Done false
21738 6308 Acked Read 1 Done Sent Done false
21739 6309 Acked Flush 0 Done Sent Done false
21740 6310 Acked Write 7 Done Sent Done false
21741 6311 Acked Read 9 Done Sent Done false
21742 6312 Acked Flush 0 Done Sent Done false
21743 6313 Acked Write 6 Done Sent Done false
21744 6314 Acked Write 4 Done Sent Done false
21745 6315 Acked Flush 0 Done Sent Done false
21746 6316 Acked Read 10 Done Sent Done false
21747 6317 Acked Read 5 Done Sent Done false
21748 6318 Acked Write 10 Done Sent Done false
21749 6319 Acked Write 3 Done Sent Done false
21750 6320 Acked Read 3 Done Sent Done false
21751 6321 Acked Read 8 Done Sent Done false
21752 6322 Acked Flush 0 Done Sent Done false
21753 6323 Acked Write 10 Done Sent Done false
21754 6324 Acked Write 3 Done Sent Done false
21755 6325 Acked Read 5 Done Sent Done false
21756 6326 Acked Read 6 Done Sent Done false
21757 6327 Acked Write 7 Done Sent Done false
21758 6328 Acked Read 3 Done Sent Done false
21759 6329 Acked Read 5 Done Sent Done false
21760 6330 Acked Read 5 Done Sent Done false
21761 6331 Acked Flush 0 Done Sent Done false
21762 6332 Acked Flush 0 Done Sent Done false
21763 6333 Acked Write 4 Done Sent Done false
21764 6334 Acked Read 2 Done Sent Done false
21765 6335 Acked Write 1 Done Sent Done false
21766 6336 Acked Read 1 Done Sent Done false
21767 6337 Acked Read 7 Done Sent Done false
21768 6338 Acked Read 5 Done Sent Done false
21769 6339 Acked Write 2 Done Sent Done false
21770 6340 Acked Write 4 Done Sent Done false
21771 6341 Acked Read 9 Done Sent Done false
21772 6342 Acked Write 4 Done Sent Done false
21773 6343 Acked Read 1 Done Sent Done false
21774 6344 Acked Read 1 Done Sent Done false
21775 6345 Acked Write 4 Done Sent Done false
21776 6346 Acked Write 9 Done Sent Done false
21777 6347 Acked Read 6 Done Sent Done false
21778 6348 Acked Read 8 Done Sent Done false
21779 6349 Acked Read 3 Done Sent Done false
21780 6350 Acked Write 3 Done Sent Done false
21781 6351 Acked Read 6 Done Sent Done false
21782 6352 Acked Read 6 Done Sent Done false
21783 6353 Acked Write 3 Done Sent Done false
21784 6354 Acked Read 4 Done Sent Done false
21785 6355 Acked Flush 0 Done Sent Done false
21786 6356 Acked Read 5 Done Sent Done false
21787 6357 Acked Read 3 Done Sent Done false
21788 6358 Acked Write 8 Done Sent Done false
21789 6359 Acked Read 3 Done Sent Done false
21790 6360 Acked Read 8 Done Sent Done false
21791 6361 Acked Read 8 Done Sent Done false
21792 6362 Acked Read 8 Done Sent Done false
21793 6363 Acked Write 2 Done Sent Done false
21794 6364 Acked Read 2 Done Sent Done false
21795 6365 Acked Read 10 Done Sent Done false
21796 6366 Acked Read 6 Done Sent Done false
21797 6367 Acked Read 3 Done Sent Done false
21798 6368 Acked Write 6 Done Sent Done false
21799 6369 Acked Write 10 Done Sent Done false
21800 6370 Acked Flush 0 Done Sent Done false
21801 6371 Acked Write 4 Done Sent Done false
21802 6372 Acked Read 9 Done Sent Done false
21803 6373 Acked Read 5 Done Sent Done false
21804 6374 Acked Write 7 Done Sent Done false
21805 6375 Acked Read 4 Done Sent Done false
21806 6376 Acked Write 10 Done Sent Done false
21807 6377 Acked Read 5 Done Sent Done false
21808 6378 Acked Flush 0 Done Sent Done false
21809 6379 Acked Write 3 Done Sent Done false
21810 6380 Acked Read 6 Done Sent Done false
21811 6381 Acked Write 2 Done Sent Done false
21812 6382 Acked Read 5 Done Sent Done false
21813 6383 Acked Write 5 Done Sent Done false
21814 6384 Acked Flush 0 Done Sent Done false
21815 6385 Acked Read 5 Done Sent Done false
21816 6386 Acked Write 10 Done Sent Done false
21817 6387 Acked Read 9 Done Sent Done false
21818 6388 Acked Read 9 Done Sent Done false
21819 6389 Acked Write 8 Done Sent Done false
21820 6390 Acked Write 5 Done Sent Done false
21821 6391 Acked Read 2 Done Sent Done false
21822 6392 Acked Read 7 Done Sent Done false
21823 6393 Acked Read 3 Done Sent Done false
21824 6394 Acked Read 9 Done Sent Done false
21825 6395 Acked Write 8 Done Sent Done false
21826 6396 Acked Read 1 Done Sent Done false
21827 6397 Acked Flush 0 Done Sent Done false
21828 6398 Acked Flush 0 Done Sent Done false
21829 6399 Acked Flush 0 Done Sent Done false
21830 6400 Acked Read 3 Done Sent Done false
21831 6401 Acked Write 5 Done Sent Done false
21832 6402 Acked Write 7 Done Sent Done false
21833 6403 Acked Read 10 Done Sent Done false
21834 6404 Acked Read 6 Done Sent Done false
21835 6405 Acked Read 5 Done Sent Done false
21836 6406 Acked Write 2 Done Sent Done false
21837 6407 Acked Write 7 Done Sent Done false
21838 6408 Acked Read 4 Done Sent Done false
21839 6409 Acked Read 7 Done Sent Done false
21840 6410 Acked Write 7 Done Sent Done false
21841 6411 Acked Write 9 Done Sent Done false
21842 6412 Acked Read 8 Done Sent Done false
21843 6413 Acked Write 6 Done Sent Done false
21844 6414 Acked Write 6 Done Sent Done false
21845 6415 Acked Read 3 Done Sent Done false
21846 6416 Acked Write 9 Done Sent Done false
21847 6417 Acked Write 3 Done Sent Done false
21848 6418 Acked Read 10 Done Sent Done false
21849 6419 Acked Write 1 Done Sent Done false
21850 6420 Acked Read 5 Done Sent Done false
21851 6421 Acked Read 7 Done Sent Done false
21852 6422 Acked Flush 0 Done Sent Done false
21853 6423 Acked Write 9 Done Sent Done false
21854 6424 Acked Write 1 Done Sent Done false
21855 6425 Acked Write 8 Done Sent Done false
21856 6426 Acked Read 2 Done Sent Done false
21857 6427 Acked Read 10 Done Sent Done false
21858 6428 Acked Write 2 Done Sent Done false
21859 6429 Acked Write 2 Done Sent Done false
21860 6430 Acked Flush 0 Done Sent Done false
21861 6431 Acked Read 9 Done Sent Done false
21862 6432 Acked Read 1 Done Sent Done false
21863 6433 Acked Write 3 Done Sent Done false
21864 6434 Acked Read 5 Done Sent Done false
21865 6435 Acked Flush 0 Done Sent Done false
21866 STATES DS:0 DS:1 DS:2 TOTAL
21867 Sent 0 302 0 302
21868 Done 302 0 302 604
21869 Skipped 0 0 0 0
21870 Error 0 0 0 0
21871 Last Flush: 6435 6133 6435
21872 Downstairs last five completed: 6133 6132 6131 6130 6129
21873 Upstairs last five completed: 6435 6434 6433 6432 6431
21874 CLIENT: Up:0 ds:302 act:2
21875 ----------------------------------------------------------------
21876 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21877 States: Active Offline Active
21878 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21879 6134 Acked Read 3 Done Sent Done false
21880 6135 Acked Write 6 Done Sent Done false
21881 6136 Acked Write 4 Done Sent Done false
21882 6137 Acked Write 3 Done Sent Done false
21883 6138 Acked Read 6 Done Sent Done false
21884 6139 Acked Read 2 Done Sent Done false
21885 6140 Acked Read 3 Done Sent Done false
21886 6141 Acked Read 2 Done Sent Done false
21887 6142 Acked Read 7 Done Sent Done false
21888 6143 Acked Read 1 Done Sent Done false
21889 6144 Acked Read 10 Done Sent Done false
21890 6145 Acked Write 4 Done Sent Done false
21891 6146 Acked Write 10 Done Sent Done false
21892 6147 Acked Read 3 Done Sent Done false
21893 6148 Acked Read 8 Done Sent Done false
21894 6149 Acked Read 10 Done Sent Done false
21895 6150 Acked Write 9 Done Sent Done false
21896 6151 Acked Write 3 Done Sent Done false
21897 6152 Acked Write 8 Done Sent Done false
21898 6153 Acked Write 4 Done Sent Done false
21899 6154 Acked Read 1 Done Sent Done false
21900 6155 Acked Read 1 Done Sent Done false
21901 6156 Acked Write 1 Done Sent Done false
21902 6157 Acked Write 1 Done Sent Done false
21903 6158 Acked Read 3 Done Sent Done false
21904 6159 Acked Read 3 Done Sent Done false
21905 6160 Acked Read 10 Done Sent Done false
21906 6161 Acked Write 9 Done Sent Done false
21907 6162 Acked Write 9 Done Sent Done false
21908 6163 Acked Read 10 Done Sent Done false
21909 6164 Acked Write 5 Done Sent Done false
21910 6165 Acked Write 9 Done Sent Done false
21911 6166 Acked Write 3 Done Sent Done false
21912 6167 Acked Write 1 Done Sent Done false
21913 6168 Acked Write 7 Done Sent Done false
21914 6169 Acked Write 3 Done Sent Done false
21915 6170 Acked Read 7 Done Sent Done false
21916 6171 Acked Flush 0 Done Sent Done false
21917 6172 Acked Flush 0 Done Sent Done false
21918 6173 Acked Read 2 Done Sent Done false
21919 6174 Acked Write 4 Done Sent Done false
21920 6175 Acked Write 10 Done Sent Done false
21921 6176 Acked Read 4 Done Sent Done false
21922 6177 Acked Write 2 Done Sent Done false
21923 6178 Acked Read 2 Done Sent Done false
21924 6179 Acked Flush 0 Done Sent Done false
21925 6180 Acked Read 2 Done Sent Done false
21926 6181 Acked Read 2 Done Sent Done false
21927 6182 Acked Read 7 Done Sent Done false
21928 6183 Acked Write 5 Done Sent Done false
21929 6184 Acked Write 5 Done Sent Done false
21930 6185 Acked Read 7 Done Sent Done false
21931 6186 Acked Write 9 Done Sent Done false
21932 6187 Acked Write 5 Done Sent Done false
21933 6188 Acked Read 3 Done Sent Done false
21934 6189 Acked Read 2 Done Sent Done false
21935 6190 Acked Write 5 Done Sent Done false
21936 6191 Acked Flush 0 Done Sent Done false
21937 6192 Acked Read 10 Done Sent Done false
21938 6193 Acked Read 1 Done Sent Done false
21939 6194 Acked Read 7 Done Sent Done false
21940 6195 Acked Write 2 Done Sent Done false
21941 6196 Acked Read 5 Done Sent Done false
21942 6197 Acked Read 6 Done Sent Done false
21943 6198 Acked Read 4 Done Sent Done false
21944 6199 Acked Read 2 Done Sent Done false
21945 6200 Acked Write 9 Done Sent Done false
21946 6201 Acked Flush 0 Done Sent Done false
21947 6202 Acked Write 7 Done Sent Done false
21948 6203 Acked Write 1 Done Sent Done false
21949 6204 Acked Read 9 Done Sent Done false
21950 6205 Acked Write 8 Done Sent Done false
21951 6206 Acked Read 1 Done Sent Done false
21952 6207 Acked Flush 0 Done Sent Done false
21953 6208 Acked Flush 0 Done Sent Done false
21954 6209 Acked Write 5 Done Sent Done false
21955 6210 Acked Read 2 Done Sent Done false
21956 6211 Acked Write 1 Done Sent Done false
21957 6212 Acked Read 10 Done Sent Done false
21958 6213 Acked Write 10 Done Sent Done false
21959 6214 Acked Read 6 Done Sent Done false
21960 6215 Acked Read 3 Done Sent Done false
21961 6216 Acked Write 9 Done Sent Done false
21962 6217 Acked Flush 0 Done Sent Done false
21963 6218 Acked Flush 0 Done Sent Done false
21964 6219 Acked Read 2 Done Sent Done false
21965 6220 Acked Read 2 Done Sent Done false
21966 6221 Acked Flush 0 Done Sent Done false
21967 6222 Acked Read 4 Done Sent Done false
21968 6223 Acked Write 8 Done Sent Done false
21969 6224 Acked Read 1 Done Sent Done false
21970 6225 Acked Read 3 Done Sent Done false
21971 6226 Acked Write 10 Done Sent Done false
21972 6227 Acked Write 6 Done Sent Done false
21973 6228 Acked Read 7 Done Sent Done false
21974 6229 Acked Read 10 Done Sent Done false
21975 6230 Acked Read 5 Done Sent Done false
21976 6231 Acked Read 6 Done Sent Done false
21977 6232 Acked Write 6 Done Sent Done false
21978 6233 Acked Write 5 Done Sent Done false
21979 6234 Acked Read 3 Done Sent Done false
21980 6235 Acked Read 3 Done Sent Done false
21981 6236 Acked Read 10 Done Sent Done false
21982 6237 Acked Write 7 Done Sent Done false
21983 6238 Acked Flush 0 Done Sent Done false
21984 6239 Acked Read 1 Done Sent Done false
21985 6240 Acked Write 2 Done Sent Done false
21986 6241 Acked Write 8 Done Sent Done false
21987 6242 Acked Write 10 Done Sent Done false
21988 6243 Acked Write 7 Done Sent Done false
21989 6244 Acked Read 3 Done Sent Done false
21990 6245 Acked Read 9 Done Sent Done false
21991 6246 Acked Read 7 Done Sent Done false
21992 6247 Acked Read 8 Done Sent Done false
21993 6248 Acked Write 1 Done Sent Done false
21994 6249 Acked Read 5 Done Sent Done false
21995 6250 Acked Write 3 Done Sent Done false
21996 6251 Acked Flush 0 Done Sent Done false
21997 6252 Acked Read 1 Done Sent Done false
21998 6253 Acked Flush 0 Done Sent Done false
21999 6254 Acked Read 3 Done Sent Done false
22000 6255 Acked Write 10 Done Sent Done false
22001 6256 Acked Read 10 Done Sent Done false
22002 6257 Acked Read 4 Done Sent Done false
22003 6258 Acked Write 6 Done Sent Done false
22004 6259 Acked Read 4 Done Sent Done false
22005 6260 Acked Read 7 Done Sent Done false
22006 6261 Acked Read 5 Done Sent Done false
22007 6262 Acked Read 3 Done Sent Done false
22008 6263 Acked Read 10 Done Sent Done false
22009 6264 Acked Read 5 Done Sent Done false
22010 6265 Acked Read 5 Done Sent Done false
22011 6266 Acked Read 3 Done Sent Done false
22012 6267 Acked Read 7 Done Sent Done false
22013 6268 Acked Read 4 Done Sent Done false
22014 6269 Acked Read 6 Done Sent Done false
22015 6270 Acked Write 9 Done Sent Done false
22016 6271 Acked Read 7 Done Sent Done false
22017 6272 Acked Read 2 Done Sent Done false
22018 6273 Acked Write 10 Done Sent Done false
22019 6274 Acked Write 6 Done Sent Done false
22020 6275 Acked Read 7 Done Sent Done false
22021 6276 Acked Read 8 Done Sent Done false
22022 6277 Acked Flush 0 Done Sent Done false
22023 6278 Acked Read 10 Done Sent Done false
22024 6279 Acked Write 9 Done Sent Done false
22025 6280 Acked Read 6 Done Sent Done false
22026 6281 Acked Write 9 Done Sent Done false
22027 6282 Acked Write 10 Done Sent Done false
22028 6283 Acked Read 7 Done Sent Done false
22029 6284 Acked Read 6 Done Sent Done false
22030 6285 Acked Read 7 Done Sent Done false
22031 6286 Acked Write 4 Done Sent Done false
22032 6287 Acked Flush 0 Done Sent Done false
22033 6288 Acked Flush 0 Done Sent Done false
22034 6289 Acked Write 2 Done Sent Done false
22035 6290 Acked Flush 0 Done Sent Done false
22036 6291 Acked Write 2 Done Sent Done false
22037 6292 Acked Write 6 Done Sent Done false
22038 6293 Acked Read 8 Done Sent Done false
22039 6294 Acked Write 6 Done Sent Done false
22040 6295 Acked Flush 0 Done Sent Done false
22041 6296 Acked Flush 0 Done Sent Done false
22042 6297 Acked Read 4 Done Sent Done false
22043 6298 Acked Write 7 Done Sent Done false
22044 6299 Acked Read 1 Done Sent Done false
22045 6300 Acked Write 6 Done Sent Done false
22046 6301 Acked Write 8 Done Sent Done false
22047 6302 Acked Write 4 Done Sent Done false
22048 6303 Acked Write 10 Done Sent Done false
22049 6304 Acked Read 10 Done Sent Done false
22050 6305 Acked Write 4 Done Sent Done false
22051 6306 Acked Flush 0 Done Sent Done false
22052 6307 Acked Read 3 Done Sent Done false
22053 6308 Acked Read 1 Done Sent Done false
22054 6309 Acked Flush 0 Done Sent Done false
22055 6310 Acked Write 7 Done Sent Done false
22056 6311 Acked Read 9 Done Sent Done false
22057 6312 Acked Flush 0 Done Sent Done false
22058 6313 Acked Write 6 Done Sent Done false
22059 6314 Acked Write 4 Done Sent Done false
22060 6315 Acked Flush 0 Done Sent Done false
22061 6316 Acked Read 10 Done Sent Done false
22062 6317 Acked Read 5 Done Sent Done false
22063 6318 Acked Write 10 Done Sent Done false
22064 6319 Acked Write 3 Done Sent Done false
22065 6320 Acked Read 3 Done Sent Done false
22066 6321 Acked Read 8 Done Sent Done false
22067 6322 Acked Flush 0 Done Sent Done false
22068 6323 Acked Write 10 Done Sent Done false
22069 6324 Acked Write 3 Done Sent Done false
22070 6325 Acked Read 5 Done Sent Done false
22071 6326 Acked Read 6 Done Sent Done false
22072 6327 Acked Write 7 Done Sent Done false
22073 6328 Acked Read 3 Done Sent Done false
22074 6329 Acked Read 5 Done Sent Done false
22075 6330 Acked Read 5 Done Sent Done false
22076 6331 Acked Flush 0 Done Sent Done false
22077 6332 Acked Flush 0 Done Sent Done false
22078 6333 Acked Write 4 Done Sent Done false
22079 6334 Acked Read 2 Done Sent Done false
22080 6335 Acked Write 1 Done Sent Done false
22081 6336 Acked Read 1 Done Sent Done false
22082 6337 Acked Read 7 Done Sent Done false
22083 6338 Acked Read 5 Done Sent Done false
22084 6339 Acked Write 2 Done Sent Done false
22085 6340 Acked Write 4 Done Sent Done false
22086 6341 Acked Read 9 Done Sent Done false
22087 6342 Acked Write 4 Done Sent Done false
22088 6343 Acked Read 1 Done Sent Done false
22089 6344 Acked Read 1 Done Sent Done false
22090 6345 Acked Write 4 Done Sent Done false
22091 6346 Acked Write 9 Done Sent Done false
22092 6347 Acked Read 6 Done Sent Done false
22093 6348 Acked Read 8 Done Sent Done false
22094 6349 Acked Read 3 Done Sent Done false
22095 6350 Acked Write 3 Done Sent Done false
22096 6351 Acked Read 6 Done Sent Done false
22097 6352 Acked Read 6 Done Sent Done false
22098 6353 Acked Write 3 Done Sent Done false
22099 6354 Acked Read 4 Done Sent Done false
22100 6355 Acked Flush 0 Done Sent Done false
22101 6356 Acked Read 5 Done Sent Done false
22102 6357 Acked Read 3 Done Sent Done false
22103 6358 Acked Write 8 Done Sent Done false
22104 6359 Acked Read 3 Done Sent Done false
22105 6360 Acked Read 8 Done Sent Done false
22106 6361 Acked Read 8 Done Sent Done false
22107 6362 Acked Read 8 Done Sent Done false
22108 6363 Acked Write 2 Done Sent Done false
22109 6364 Acked Read 2 Done Sent Done false
22110 6365 Acked Read 10 Done Sent Done false
22111 6366 Acked Read 6 Done Sent Done false
22112 6367 Acked Read 3 Done Sent Done false
22113 6368 Acked Write 6 Done Sent Done false
22114 6369 Acked Write 10 Done Sent Done false
22115 6370 Acked Flush 0 Done Sent Done false
22116 6371 Acked Write 4 Done Sent Done false
22117 6372 Acked Read 9 Done Sent Done false
22118 6373 Acked Read 5 Done Sent Done false
22119 6374 Acked Write 7 Done Sent Done false
22120 6375 Acked Read 4 Done Sent Done false
22121 6376 Acked Write 10 Done Sent Done false
22122 6377 Acked Read 5 Done Sent Done false
22123 6378 Acked Flush 0 Done Sent Done false
22124 6379 Acked Write 3 Done Sent Done false
22125 6380 Acked Read 6 Done Sent Done false
22126 6381 Acked Write 2 Done Sent Done false
22127 6382 Acked Read 5 Done Sent Done false
22128 6383 Acked Write 5 Done Sent Done false
22129 6384 Acked Flush 0 Done Sent Done false
22130 6385 Acked Read 5 Done Sent Done false
22131 6386 Acked Write 10 Done Sent Done false
22132 6387 Acked Read 9 Done Sent Done false
22133 6388 Acked Read 9 Done Sent Done false
22134 6389 Acked Write 8 Done Sent Done false
22135 6390 Acked Write 5 Done Sent Done false
22136 6391 Acked Read 2 Done Sent Done false
22137 6392 Acked Read 7 Done Sent Done false
22138 6393 Acked Read 3 Done Sent Done false
22139 6394 Acked Read 9 Done Sent Done false
22140 6395 Acked Write 8 Done Sent Done false
22141 6396 Acked Read 1 Done Sent Done false
22142 6397 Acked Flush 0 Done Sent Done false
22143 6398 Acked Flush 0 Done Sent Done false
22144 6399 Acked Flush 0 Done Sent Done false
22145 6400 Acked Read 3 Done Sent Done false
22146 6401 Acked Write 5 Done Sent Done false
22147 6402 Acked Write 7 Done Sent Done false
22148 6403 Acked Read 10 Done Sent Done false
22149 6404 Acked Read 6 Done Sent Done false
22150 6405 Acked Read 5 Done Sent Done false
22151 6406 Acked Write 2 Done Sent Done false
22152 6407 Acked Write 7 Done Sent Done false
22153 6408 Acked Read 4 Done Sent Done false
22154 6409 Acked Read 7 Done Sent Done false
22155 6410 Acked Write 7 Done Sent Done false
22156 6411 Acked Write 9 Done Sent Done false
22157 6412 Acked Read 8 Done Sent Done false
22158 6413 Acked Write 6 Done Sent Done false
22159 6414 Acked Write 6 Done Sent Done false
22160 6415 Acked Read 3 Done Sent Done false
22161 6416 Acked Write 9 Done Sent Done false
22162 6417 Acked Write 3 Done Sent Done false
22163 6418 Acked Read 10 Done Sent Done false
22164 6419 Acked Write 1 Done Sent Done false
22165 6420 Acked Read 5 Done Sent Done false
22166 6421 Acked Read 7 Done Sent Done false
22167 6422 Acked Flush 0 Done Sent Done false
22168 6423 Acked Write 9 Done Sent Done false
22169 6424 Acked Write 1 Done Sent Done false
22170 6425 Acked Write 8 Done Sent Done false
22171 6426 Acked Read 2 Done Sent Done false
22172 6427 Acked Read 10 Done Sent Done false
22173 6428 Acked Write 2 Done Sent Done false
22174 6429 Acked Write 2 Done Sent Done false
22175 6430 Acked Flush 0 Done Sent Done false
22176 6431 Acked Read 9 Done Sent Done false
22177 6432 Acked Read 1 Done Sent Done false
22178 6433 Acked Write 3 Done Sent Done false
22179 6434 Acked Read 5 Done Sent Done false
22180 6435 Acked Flush 0 Done Sent Done false
22181 STATES DS:0 DS:1 DS:2 TOTAL
22182 Sent 0 302 0 302
22183 Done 302 0 302 604
22184 Skipped 0 0 0 0
22185 Error 0 0 0 0
22186 Last Flush: 6435 6133 6435
22187 Downstairs last five completed: 6133 6132 6131 6130 6129
22188 Upstairs last five completed: 6435 6434 6433 6432 6431
22189 CLIENT: Up:0 ds:302 act:2
221902024-12-11T23:45:10.701ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221912024-12-11T23:45:10.701ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62285) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221922024-12-11T23:45:10.702ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221932024-12-11T23:45:10.703ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221942024-12-11T23:45:10.703ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221952024-12-11T23:45:10.703ZINFOcrucible: send last flush ID to this DS: 6133 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221962024-12-11T23:45:10.703ZINFOcrucible: Replied this last flush ID: 6133 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221972024-12-11T23:45:10.703ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221982024-12-11T23:45:10.703ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
221992024-12-11T23:45:10.703ZINFOcrucible: [1] client re-new 302 jobs since flush 6133 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
222002024-12-11T23:45:10.703ZINFOcrucible: [1] Replayed 302 jobs since flush: 6133 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
22201 ----------------------------------------------------------------
22202 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22203 Downstairs last five completed: 6435 6434 6433 6432 6431
22204 Upstairs last five completed: 6435 6434 6433 6432 6431
22205 CLIENT: Up:0 ds:0 act:3
22206 Replay: All jobs finished
22207 Main task has work to do, go find it
22208 got dsc Stop(0)
22209 stop 0
22210 [8810] Got stop action so:false kr:false
22211 Dec 11 23:45:10.948 INFO request completed, latency_us: 410, response_code: 204, uri: /stop/cid/0, method: POST, req_id: f0739e22-f17b-48bc-8612-9f67b936eff5, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
22212 Dec 11 23:45:10.948 INFO accepted connection, remote_addr: 127.0.0.1:39246, local_addr: 127.0.0.1:9998
222132024-12-11T23:45:10.949ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
222142024-12-11T23:45:10.949ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
222152024-12-11T23:45:10.949ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
222162024-12-11T23:45:10.949ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
222172024-12-11T23:45:10.949ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
222182024-12-11T23:45:10.949ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
22219 Dec 11 23:45:10.949 INFO request completed, latency_us: 173, response_code: 200, uri: /state/cid/0, method: GET, req_id: 03ecc9da-ac9a-413d-baaf-7681e0da728d, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
22220 [8810] Exited with: ExitStatus(unix_wait_status(9))
22221 [8810][0] reports Exit
22222 Dec 11 23:45:14.951 INFO request completed, latency_us: 223, response_code: 200, uri: /state/cid/0, method: GET, req_id: afee4fbc-ec60-4865-bd55-6336e934b360, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
22223 001/300 Flush
22224 002/300 Write block 4786 len 4096 data: 4
22225 003/300 Read block 4667 len 28672
22226 004/300 Write block 130 len 32768 data: 5 4 4 3 3 3 2 3
22227 005/300 Write block 2531 len 12288 data: 2 2 2
22228 006/300 Write block 4521 len 12288 data: 7 7 6
22229 007/300 Write block 4353 len 24576 data: 6 5 6 6 6 5
22230 008/300 Write block 99 len 28672 data: 6 6 6 6 6 7 6
22231 009/300 Write block 962 len 24576 data: 3 3 4 5 4 3
22232 010/300 Flush
22233 011/300 Read block 206 len 12288
22234 012/300 Flush
22235 013/300 Read block 1642 len 12288
22236 014/300 Write block 1483 len 24576 data: 5 5 4 3 4 4
22237 015/300 Write block 3854 len 8192 data: 6 5
22238 016/300 Read block 2301 len 12288
22239 017/300 Write block 1026 len 20480 data: 5 4 3 4 3
22240 018/300 Read block 4953 len 8192
22241 019/300 Read block 16 len 16384
22242 020/300 Write block 560 len 4096 data: 3
22243 021/300 Read block 459 len 40960
22244 022/300 Read block 1031 len 36864
22245 023/300 Read block 222 len 4096
22246 024/300 Write block 355 len 36864 data: 4 4 7 4 4 4 3 3 4
22247 025/300 Write block 3288 len 8192 data: 4 4
22248 026/300 Read block 919 len 16384
22249 027/300 Read block 2520 len 20480
22250 028/300 Write block 4458 len 36864 data: 4 3 3 3 4 5 6 5 5
22251 029/300 Flush
22252 030/300 Write block 2239 len 24576 data: 3 3 5 6 6 6
22253 031/300 Write block 4015 len 36864 data: 4 5 7 7 6 6 3 3 3
22254 032/300 Write block 3276 len 36864 data: 4 5 4 6 6 5 6 5 5
22255 033/300 Read block 2590 len 20480
22256 034/300 Read block 4399 len 8192
22257 035/300 Flush
22258 036/300 Read block 570 len 24576
22259 037/300 Write block 339 len 40960 data: 4 4 5 5 6 7 7 7 6 3
22260 038/300 Write block 1365 len 24576 data: 4 3 3 5 3 3
22261 039/300 Flush
22262 040/300 Write block 499 len 40960 data: 4 3 4 2 2 2 2 2 2 2
22263 041/300 Write block 4101 len 16384 data: 4 4 6 6
22264 042/300 Read block 4118 len 12288
22265 043/300 Read block 4360 len 12288
22266 044/300 Write block 752 len 12288 data: 5 5 6
22267 045/300 Read block 3209 len 20480
22268 046/300 Read block 2436 len 12288
22269 047/300 Read block 35 len 36864
22270 048/300 Read block 772 len 40960
22271 049/300 Read block 168 len 4096
22272 050/300 Write block 3778 len 20480 data: 4 4 4 5 5
22273 051/300 Write block 3715 len 32768 data: 5 5 5 6 6 6 5 4
22274 052/300 Flush
22275 053/300 Write block 4210 len 12288 data: 6 5 3
22276 054/300 Flush
22277 055/300 Read block 2626 len 20480
22278 056/300 Read block 3249 len 4096
22279 057/300 Write block 1358 len 32768 data: 4 4 3 4 4 4 4 5
22280 058/300 Write block 2280 len 4096 data: 5
22281 059/300 Read block 4422 len 36864
22282 060/300 Write block 2423 len 28672 data: 5 5 5 5 5 4 5
22283 061/300 Write block 821 len 40960 data: 4 5 5 5 4 5 5 4 4 3
22284 062/300 Flush
22285 063/300 Read block 2620 len 24576
22286 064/300 Write block 633 len 32768 data: 4 2 3 3 5 6 7 6
22287 065/300 Write block 3685 len 4096 data: 6
22288 066/300 Read block 1729 len 36864
22289 067/300 Flush
22290 068/300 Write block 1889 len 32768 data: 6 5 5 5 5 5 6 6
22291 069/300 Read block 4404 len 32768
22292 070/300 Read block 831 len 8192
22293 071/300 Read block 3065 len 40960
22294 072/300 Read block 324 len 4096
22295 073/300 Read block 922 len 4096
22296 074/300 Read block 272 len 12288
22297 075/300 Read block 1811 len 28672
22298 076/300 Read block 3424 len 12288
22299 077/300 Write block 3567 len 28672 data: 5 5 4 4 5 3 5
22300 078/300 Write block 4411 len 8192 data: 3 3
22301 079/300 Flush
22302 080/300 Write block 1791 len 16384 data: 4 5 4 4
22303 081/300 Flush
22304 082/300 Write block 3030 len 36864 data: 5 5 4 4 5 4 4 2 2
22305 083/300 Write block 4445 len 8192 data: 5 5
22306 084/300 Read block 1946 len 36864
22307 085/300 Write block 3846 len 32768 data: 3 3 4 3 4 4 5 6
22308 086/300 Read block 4822 len 16384
22309 087/300 Write block 1910 len 36864 data: 5 5 5 6 5 5 5 5 5
22310 088/300 Flush
22311 089/300 Read block 598 len 36864
22312 090/300 Read block 4390 len 40960
22313 091/300 Write block 1027 len 40960 data: 5 4 5 4 4 5 5 6 6 3
22314 092/300 Read block 3999 len 4096
22315 093/300 Write block 180 len 20480 data: 7 6 5 6 5
22316 094/300 Read block 2652 len 4096
22317 095/300 Write block 4530 len 12288 data: 10 9 11
22318 096/300 Read block 4791 len 24576
22319 097/300 Flush
22320 098/300 Write block 4068 len 36864 data: 4 5 5 5 5 3 4 4 4
22321 099/300 Read block 4018 len 8192
22322 100/300 Read block 2862 len 4096
22323 101/300 Flush
22324 102/300 Write block 4812 len 8192 data: 4 6
22325 103/300 Read block 1796 len 36864
22326 104/300 Write block 3347 len 12288 data: 4 5 4
22327 105/300 Read block 2520 len 16384
22328 106/300 Flush
22329 107/300 Write block 897 len 12288 data: 3 3 3
22330 108/300 Write block 3760 len 24576 data: 4 4 3 2 3 3
22331 109/300 Read block 1148 len 4096
22332 110/300 Write block 322 len 12288 data: 7 9 9
22333 111/300 Flush
22334 112/300 Read block 2496 len 4096
22335 113/300 Read block 3545 len 28672
22336 114/300 Write block 3813 len 8192 data: 5 7
22337 115/300 Write block 2740 len 12288 data: 6 6 6
22338 116/300 Write block 2357 len 40960 data: 5 4 4 4 4 4 4 5 5 5
22339 117/300 Write block 2339 len 16384 data: 6 6 5 5
22340 118/300 Read block 4699 len 36864
22341 119/300 Read block 2262 len 36864
22342 120/300 Read block 412 len 36864
22343 121/300 Read block 3017 len 28672
22344 122/300 Read block 3269 len 36864
22345 123/300 Read block 285 len 8192
22346 124/300 Write block 1008 len 28672 data: 6 6 5 5 4 4 4
22347 125/300 Write block 2598 len 20480 data: 4 4 5 5 5
22348 126/300 Read block 2237 len 20480
22349 127/300 Read block 3244 len 12288
22350 128/300 Write block 2265 len 32768 data: 4 3 5 5 5 5 7 6
22351 129/300 Read block 2323 len 24576
22352 130/300 Write block 2279 len 12288 data: 3 6 4
22353 131/300 Read block 3898 len 8192
22354 132/300 Flush
22355 133/300 Read block 939 len 24576
22356 134/300 Flush
22357 135/300 Read block 4750 len 12288
22358 136/300 Read block 4545 len 16384
22359 137/300 Write block 3510 len 8192 data: 6 6
22360 138/300 Write block 309 len 36864 data: 4 3 4 3 3 4 4 4 5
22361 139/300 Write block 598 len 16384 data: 3 2 3 3
22362 140/300 Write block 1110 len 40960 data: 5 4 5 5 5 5 6 6 6 6
22363 141/300 Read block 4086 len 40960
22364 142/300 Write block 601 len 12288 data: 4 2 2
22365 143/300 Write block 3256 len 40960 data: 2 2 2 3 3 3 4 6 5 5
22366 144/300 Read block 1163 len 4096
22367 145/300 Write block 4815 len 28672 data: 4 4 4 3 3 3 4
22368 146/300 Read block 3037 len 32768
22369 147/300 Read block 257 len 28672
22370 148/300 Flush
22371 149/300 Write block 3135 len 20480 data: 6 7 7 5 5
22372 150/300 Read block 272 len 4096
22373 151/300 Write block 3754 len 12288 data: 3 3 4
22374 152/300 Read block 1319 len 32768
22375 153/300 Write block 1458 len 20480 data: 6 6 5 5 5
22376 154/300 Read block 1994 len 36864
22377 155/300 Read block 2107 len 36864
22378 156/300 Read block 3002 len 24576
22379 157/300 Write block 607 len 40960 data: 4 4 4 4 5 5 4 6 4 3
22380 158/300 Read block 3412 len 8192
22381 159/300 Write block 3436 len 28672 data: 3 3 4 5 4 3 3
22382 160/300 Read block 4373 len 8192
22383 161/300 Write block 427 len 12288 data: 2 2 3
22384 162/300 Read block 2307 len 20480
22385 163/300 Flush
22386 164/300 Write block 3961 len 40960 data: 10 11 9 9 8 6 6 2 2 3
22387 165/300 Write block 1685 len 32768 data: 6 6 7 7 7 7 4 4
22388 166/300 Read block 2437 len 16384
22389 167/300 Flush
22390 168/300 Write block 3207 len 40960 data: 6 5 6 6 8 6 6 5 5 6
22391 169/300 Read block 4991 len 4096
22392 170/300 Read block 4721 len 24576
22393 171/300 Read block 1194 len 32768
22394 172/300 Write block 4072 len 40960 data: 6 4 5 5 5 3 3 3 3 3
22395 173/300 Read block 3381 len 40960
22396 174/300 Write block 2625 len 20480 data: 2 3 3 3 3
22397 175/300 Write block 2517 len 32768 data: 4 3 3 3 3 3 3 3
22398 176/300 Read block 3784 len 40960
22399 177/300 Write block 426 len 36864 data: 3 3 3 4 3 3 3 2 2
22400 178/300 Flush
22401 179/300 Write block 4428 len 20480 data: 9 10 7 7 7
22402 180/300 Write block 503 len 12288 data: 3 3 3
22403 181/300 Write block 3240 len 28672 data: 3 4 5 6 6 5 4
22404 182/300 Write block 1002 len 24576 data: 6 5 5 5 6 6
22405 183/300 Read block 2019 len 40960
22406 184/300 Write block 1178 len 36864 data: 4 4 5 5 6 5 4 4 3
22407 185/300 Write block 1742 len 20480 data: 4 4 4 5 6
22408 186/300 Write block 1578 len 24576 data: 2 3 3 3 3 3
22409 187/300 Read block 4982 len 20480
22410 188/300 Read block 289 len 32768
22411 189/300 Write block 2710 len 12288 data: 6 5 6
22412 190/300 Read block 2177 len 36864
22413 191/300 Read block 1136 len 20480
22414 192/300 Flush
22415 193/300 Read block 2440 len 8192
22416 194/300 Flush
22417 195/300 Read block 624 len 36864
22418 196/300 Read block 4252 len 24576
22419 197/300 Write block 3837 len 8192 data: 3 3
22420 198/300 Read block 1947 len 20480
22421 199/300 Write block 4397 len 20480 data: 3 4 5 5 7
22422 200/300 Write block 1367 len 32768 data: 4 6 4 4 4 4 5 5
22423 201/300 Read block 1013 len 4096
22424 202/300 Read block 3075 len 16384
22425 203/300 Write block 4397 len 40960 data: 4 5 6 6 8 6 4 4 4 3
22426 204/300 Read block 2028 len 12288
22427 205/300 Write block 648 len 4096 data: 3
22428 206/300 Write block 3510 len 24576 data: 7 7 5 5 6 6
22429 207/300 Write block 2224 len 8192 data: 4 5
22430 208/300 Read block 3777 len 20480
22431 209/300 Flush
22432 210/300 Read block 1920 len 32768
22433 211/300 Read block 3863 len 16384
22434 212/300 Write block 598 len 8192 data: 4 3
22435 213/300 Write block 1316 len 32768 data: 3 4 3 3 4 5 5 4
22436 214/300 Read block 2390 len 4096
22437 215/300 Flush
22438 216/300 Read block 3132 len 40960
22439 217/300 Write block 1266 len 40960 data: 5 5 3 5 7 6 4 3 3 5
22440 218/300 Write block 4672 len 40960 data: 3 5 4 2 3 5 6 6 7 6
22441 219/300 Write block 3267 len 8192 data: 4 4
22442 220/300 Flush
22443 221/300 Read block 4513 len 24576
22444 222/300 Write block 2719 len 4096 data: 8
22445 223/300 Write block 242 len 32768 data: 3 4 4 3 2 2 3 3
22446 224/300 Write block 2353 len 16384 data: 2 2 5 5
22447 225/300 Write block 4084 len 20480 data: 3 3 3 4 4
22448 226/300 Read block 3720 len 36864
22449 227/300 Write block 3505 len 32768 data: 2 3 3 3 5 8 8 6
22450 228/300 Flush
22451 229/300 Read block 4065 len 16384
22452 230/300 Flush
22453 231/300 Write block 1642 len 24576 data: 7 6 6 6 7 5
22454 232/300 Read block 4047 len 12288
22455 233/300 Read block 382 len 20480
22456 234/300 Write block 415 len 16384 data: 5 4 4 4
22457 235/300 Write block 198 len 16384 data: 5 4 4 4
22458 236/300 Read block 286 len 40960
22459 237/300 Write block 2631 len 16384 data: 3 2 2 2
22460 238/300 Write block 1753 len 36864 data: 4 4 3 3 4 4 4 3 3
22461 239/300 Write block 2043 len 32768 data: 5 5 5 4 4 5 5 4
22462 240/300 Read block 1482 len 20480
22463 241/300 Read block 4761 len 12288
22464 242/300 Read block 905 len 20480
22465 243/300 Flush
22466 244/300 Flush
22467 245/300 Read block 1177 len 8192
22468 246/300 Read block 1111 len 36864
22469 247/300 Read block 1668 len 12288
22470 248/300 Write block 1747 len 8192 data: 4 3
22471 249/300 Flush
22472 250/300 Write block 1972 len 8192 data: 5 5
22473 251/300 Read block 3936 len 32768
22474 252/300 Write block 2085 len 40960 data: 3 4 3 3 3 2 3 5 6 8
22475 253/300 Read block 4649 len 12288
22476 254/300 Read block 4660 len 20480
22477 255/300 Write block 2164 len 12288 data: 3 3 3
22478 256/300 Read block 2734 len 8192
22479 257/300 Write block 763 len 4096 data: 5
22480 258/300 Write block 3052 len 12288 data: 5 4 5
22481 259/300 Read block 4147 len 36864
22482 260/300 Read block 3700 len 36864
22483 261/300 Write block 3035 len 4096 data: 5
22484 262/300 Write block 3328 len 36864 data: 7 8 6 5 4 5 4 4 5
22485 263/300 Write block 356 len 36864 data: 5 8 5 5 5 4 4 5 3
22486 264/300 Read block 1652 len 24576
22487 265/300 Read block 3190 len 24576
22488 266/300 Write block 2166 len 36864 data: 4 3 4 4 4 4 5 4 4
22489 267/300 Read block 4906 len 16384
22490 268/300 Write block 1139 len 40960 data: 7 6 5 4 3 2 4 3 5 5
22491 269/300 Write block 4647 len 40960 data: 2 2 3 3 3 3 3 3 4 4
22492 270/300 Read block 905 len 20480
22493 271/300 Read block 76 len 40960
22494 272/300 Read block 4293 len 40960
22495 273/300 Write block 4898 len 36864 data: 3 5 5 5 8 7 8 9 7
22496 274/300 Read block 3208 len 8192
22497 275/300 Flush
22498 276/300 Write block 4843 len 20480 data: 4 4 4 5 5
22499 277/300 Read block 423 len 8192
22500 278/300 Flush
22501 279/300 Write block 1456 len 16384 data: 6 7 7 7
22502 280/300 Write block 497 len 32768 data: 5 4 5 4 5 3 4 4
22503 281/300 Read block 794 len 12288
22504 282/300 Read block 3632 len 36864
22505 283/300 Read block 1796 len 24576
22506 284/300 Read block 4816 len 40960
22507 285/300 Read block 3907 len 24576
22508 286/300 Read block 4055 len 28672
22509 287/300 Read block 2786 len 8192
22510 288/300 Read block 1643 len 28672
22511 289/300 Write block 4783 len 24576 data: 5 6 5 5 4 3
22512 290/300 Read block 268 len 16384
22513 291/300 Write block 4777 len 12288 data: 5 5 4
22514 292/300 Read block 2904 len 24576
22515 293/300 Read block 3886 len 20480
22516 294/300 Write block 4511 len 4096 data: 4
22517 295/300 Flush
22518 296/300 Write block 871 len 28672 data: 5 5 6 6 7 5 7
22519 297/300 Read block 166 len 32768
22520 298/300 Write block 4743 len 20480 data: 2 2 2 3 3
22521 299/300 Write block 879 len 36864 data: 6 6 7 7 7 7 7 6 6
22522 300/300 Write block 4289 len 32768 data: 5 4 4 3 3 3 3 6
22523 301/300 Write block 1726 len 32768 data: 3 2 3 3 3 3 2 3
22524 Main task has work to do, go find it
22525 got dsc Start(0)
22526 start 0
22527 Dec 11 23:45:15.185 INFO request completed, latency_us: 279, response_code: 204, uri: /start/cid/0, method: POST, req_id: d55ab542-dba0-4793-a49f-3b269dd65e7a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
22528 Replay: started 0, returned:Ok(())
22529 ----------------------------------------------------------------
22530 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22531 States: Offline Active Active
22532 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22533 6436 Acked Flush 0 Sent Done Done false
22534 6437 Acked Write 1 Sent Done Done false
22535 6438 Acked Read 7 Sent Done Done false
22536 6439 Acked Write 8 Sent Done Done false
22537 6440 Acked Write 3 Sent Done Done false
22538 6441 Acked Write 3 Sent Done Done false
22539 6442 Acked Write 6 Sent Done Done false
22540 6443 Acked Write 7 Sent Done Done false
22541 6444 Acked Write 6 Sent Done Done false
22542 6445 Acked Flush 0 Sent Done Done false
22543 6446 Acked Read 3 Sent Done Done false
22544 6447 Acked Flush 0 Sent Done Done false
22545 6448 Acked Read 3 Sent Done Done false
22546 6449 Acked Write 6 Sent Done Done false
22547 6450 Acked Write 2 Sent Done Done false
22548 6451 Acked Read 3 Sent Done Done false
22549 6452 Acked Write 5 Sent Done Done false
22550 6453 Acked Read 2 Sent Done Done false
22551 6454 Acked Read 4 Sent Done Done false
22552 6455 Acked Write 1 Sent Done Done false
22553 6456 Acked Read 10 Sent Done Done false
22554 6457 Acked Read 9 Sent Done Done false
22555 6458 Acked Read 1 Sent Done Done false
22556 6459 Acked Write 9 Sent Done Done false
22557 6460 Acked Write 2 Sent Done Done false
22558 6461 Acked Read 4 Sent Done Done false
22559 6462 Acked Read 5 Sent Done Done false
22560 6463 Acked Write 9 Sent Done Done false
22561 6464 Acked Flush 0 Sent Done Done false
22562 6465 Acked Write 6 Sent Done Done false
22563 6466 Acked Write 9 Sent Done Done false
22564 6467 Acked Write 9 Sent Done Done false
22565 6468 Acked Read 5 Sent Done Done false
22566 6469 Acked Read 2 Sent Done Done false
22567 6470 Acked Flush 0 Sent Done Done false
22568 6471 Acked Read 6 Sent Done Done false
22569 6472 Acked Write 10 Sent Done Done false
22570 6473 Acked Write 6 Sent Done Done false
22571 6474 Acked Flush 0 Sent Done Done false
22572 6475 Acked Write 10 Sent Done Done false
22573 6476 Acked Write 4 Sent Done Done false
22574 6477 Acked Read 3 Sent Done Done false
22575 6478 Acked Read 3 Sent Done Done false
22576 6479 Acked Write 3 Sent Done Done false
22577 6480 Acked Read 5 Sent Done Done false
22578 6481 Acked Read 3 Sent Done Done false
22579 6482 Acked Read 9 Sent Done Done false
22580 6483 Acked Read 10 Sent Done Done false
22581 6484 Acked Read 1 Sent Done Done false
22582 6485 Acked Write 5 Sent Done Done false
22583 6486 Acked Write 8 Sent Done Done false
22584 6487 Acked Flush 0 Sent Done Done false
22585 6488 Acked Write 3 Sent Done Done false
22586 6489 Acked Flush 0 Sent Done Done false
22587 6490 Acked Read 5 Sent Done Done false
22588 6491 Acked Read 1 Sent Done Done false
22589 6492 Acked Write 8 Sent Done Done false
22590 6493 Acked Write 1 Sent Done Done false
22591 6494 Acked Read 9 Sent Done Done false
22592 6495 Acked Write 7 Sent Done Done false
22593 6496 Acked Write 10 Sent Done Done false
22594 6497 Acked Flush 0 Sent Done Done false
22595 6498 Acked Read 6 Sent Done Done false
22596 6499 Acked Write 8 Sent Done Done false
22597 6500 Acked Write 1 Sent Done Done false
22598 6501 Acked Read 9 Sent Done Done false
22599 6502 Acked Flush 0 Sent Done Done false
22600 6503 Acked Write 8 Sent Done Done false
22601 6504 Acked Read 8 Sent Done Done false
22602 6505 Acked Read 2 Sent Done Done false
22603 6506 Acked Read 10 Sent Done Done false
22604 6507 Acked Read 1 Sent Done Done false
22605 6508 Acked Read 1 Sent Done Done false
22606 6509 Acked Read 3 Sent Done Done false
22607 6510 Acked Read 7 Sent Done Done false
22608 6511 Acked Read 3 Sent Done Done false
22609 6512 Acked Write 7 Sent Done Done false
22610 6513 Acked Write 2 Sent Done Done false
22611 6514 Acked Flush 0 Sent Done Done false
22612 6515 Acked Write 4 Sent Done Done false
22613 6516 Acked Flush 0 Sent Done Done false
22614 6517 Acked Write 9 Sent Done Done false
22615 6518 Acked Write 2 Sent Done Done false
22616 6519 Acked Read 9 Sent Done Done false
22617 6520 Acked Write 8 Sent Done Done false
22618 6521 Acked Read 4 Sent Done Done false
22619 6522 Acked Write 9 Sent Done Done false
22620 6523 Acked Flush 0 Sent Done Done false
22621 6524 Acked Read 9 Sent Done Done false
22622 6525 Acked Read 10 Sent Done Done false
22623 6526 Acked Write 10 Sent Done Done false
22624 6527 Acked Read 1 Sent Done Done false
22625 6528 Acked Write 5 Sent Done Done false
22626 6529 Acked Read 1 Sent Done Done false
22627 6530 Acked Write 3 Sent Done Done false
22628 6531 Acked Read 6 Sent Done Done false
22629 6532 Acked Flush 0 Sent Done Done false
22630 6533 Acked Write 9 Sent Done Done false
22631 6534 Acked Read 2 Sent Done Done false
22632 6535 Acked Read 1 Sent Done Done false
22633 6536 Acked Flush 0 Sent Done Done false
22634 6537 Acked Write 2 Sent Done Done false
22635 6538 Acked Read 9 Sent Done Done false
22636 6539 Acked Write 3 Sent Done Done false
22637 6540 Acked Read 4 Sent Done Done false
22638 6541 Acked Flush 0 Sent Done Done false
22639 6542 Acked Write 3 Sent Done Done false
22640 6543 Acked Write 6 Sent Done Done false
22641 6544 Acked Read 1 Sent Done Done false
22642 6545 Acked Write 3 Sent Done Done false
22643 6546 Acked Flush 0 Sent Done Done false
22644 6547 Acked Read 1 Sent Done Done false
22645 6548 Acked Read 7 Sent Done Done false
22646 6549 Acked Write 2 Sent Done Done false
22647 6550 Acked Write 3 Sent Done Done false
22648 6551 Acked Write 10 Sent Done Done false
22649 6552 Acked Write 4 Sent Done Done false
22650 6553 Acked Read 9 Sent Done Done false
22651 6554 Acked Read 9 Sent Done Done false
22652 6555 Acked Read 9 Sent Done Done false
22653 6556 Acked Read 7 Sent Done Done false
22654 6557 Acked Read 9 Sent Done Done false
22655 6558 Acked Read 2 Sent Done Done false
22656 6559 Acked Write 7 Sent Done Done false
22657 6560 Acked Write 5 Sent Done Done false
22658 6561 Acked Read 5 Sent Done Done false
22659 6562 Acked Read 3 Sent Done Done false
22660 6563 Acked Write 8 Sent Done Done false
22661 6564 Acked Read 6 Sent Done Done false
22662 6565 Acked Write 3 Sent Done Done false
22663 6566 Acked Read 2 Sent Done Done false
22664 6567 Acked Flush 0 Sent Done Done false
22665 6568 Acked Read 6 Sent Done Done false
22666 6569 Acked Flush 0 Sent Done Done false
22667 6570 Acked Read 3 Sent Done Done false
22668 6571 Acked Read 4 Sent Done Done false
22669 6572 Acked Write 2 Sent Done Done false
22670 6573 Acked Write 9 Sent Done Done false
22671 6574 Acked Write 4 Sent Done Done false
22672 6575 Acked Write 10 Sent Done Done false
22673 6576 Acked Read 10 Sent Done Done false
22674 6577 Acked Write 3 Sent Done Done false
22675 6578 Acked Write 10 Sent Done Done false
22676 6579 Acked Read 1 Sent Done Done false
22677 6580 Acked Write 7 Sent Done Done false
22678 6581 Acked Read 8 Sent Done Done false
22679 6582 Acked Read 7 Sent Done Done false
22680 6583 Acked Flush 0 Sent Done Done false
22681 6584 Acked Write 5 Sent Done Done false
22682 6585 Acked Read 1 Sent Done Done false
22683 6586 Acked Write 3 Sent Done Done false
22684 6587 Acked Read 8 Sent Done Done false
22685 6588 Acked Write 5 Sent Done Done false
22686 6589 Acked Read 9 Sent Done Done false
22687 6590 Acked Read 9 Sent Done Done false
22688 6591 Acked Read 6 Sent Done Done false
22689 6592 Acked Write 10 Sent Done Done false
22690 6593 Acked Read 2 Sent Done Done false
22691 6594 Acked Write 7 Sent Done Done false
22692 6595 Acked Read 2 Sent Done Done false
22693 6596 Acked Write 3 Sent Done Done false
22694 6597 Acked Read 5 Sent Done Done false
22695 6598 Acked Flush 0 Sent Done Done false
22696 6599 Acked Write 10 Sent Done Done false
22697 6600 Acked Write 8 Sent Done Done false
22698 6601 Acked Read 4 Sent Done Done false
22699 6602 Acked Flush 0 Sent Done Done false
22700 6603 Acked Write 10 Sent Done Done false
22701 6604 Acked Read 1 Sent Done Done false
22702 6605 Acked Read 6 Sent Done Done false
22703 6606 Acked Read 8 Sent Done Done false
22704 6607 Acked Write 10 Sent Done Done false
22705 6608 Acked Read 10 Sent Done Done false
22706 6609 Acked Write 5 Sent Done Done false
22707 6610 Acked Write 8 Sent Done Done false
22708 6611 Acked Read 10 Sent Done Done false
22709 6612 Acked Write 9 Sent Done Done false
22710 6613 Acked Flush 0 Sent Done Done false
22711 6614 Acked Write 5 Sent Done Done false
22712 6615 Acked Write 3 Sent Done Done false
22713 6616 Acked Write 7 Sent Done Done false
22714 6617 Acked Write 6 Sent Done Done false
22715 6618 Acked Read 10 Sent Done Done false
22716 6619 Acked Write 9 Sent Done Done false
22717 6620 Acked Write 5 Sent Done Done false
22718 6621 Acked Write 6 Sent Done Done false
22719 6622 Acked Read 5 Sent Done Done false
22720 6623 Acked Read 8 Sent Done Done false
22721 6624 Acked Write 3 Sent Done Done false
22722 6625 Acked Read 9 Sent Done Done false
22723 6626 Acked Read 5 Sent Done Done false
22724 6627 Acked Flush 0 Sent Done Done false
22725 6628 Acked Read 2 Sent Done Done false
22726 6629 Acked Flush 0 Sent Done Done false
22727 6630 Acked Read 9 Sent Done Done false
22728 6631 Acked Read 6 Sent Done Done false
22729 6632 Acked Write 2 Sent Done Done false
22730 6633 Acked Read 5 Sent Done Done false
22731 6634 Acked Write 5 Sent Done Done false
22732 6635 Acked Write 8 Sent Done Done false
22733 6636 Acked Read 1 Sent Done Done false
22734 6637 Acked Read 4 Sent Done Done false
22735 6638 Acked Write 10 Sent Done Done false
22736 6639 Acked Read 3 Sent Done Done false
22737 6640 Acked Write 1 Sent Done Done false
22738 6641 Acked Write 6 Sent Done Done false
22739 6642 Acked Write 2 Sent Done Done false
22740 6643 Acked Read 5 Sent Done Done false
22741 6644 Acked Flush 0 Sent Done Done false
22742 6645 Acked Read 8 Sent Done Done false
22743 6646 Acked Read 4 Sent Done Done false
22744 6647 Acked Write 2 Sent Done Done false
22745 6648 Acked Write 8 Sent Done Done false
22746 6649 Acked Read 1 Sent Done Done false
22747 6650 Acked Flush 0 Sent Done Done false
22748 6651 Acked Read 10 Sent Done Done false
22749 6652 Acked Write 10 Sent Done Done false
22750 6653 Acked Write 10 Sent Done Done false
22751 6654 Acked Write 2 Sent Done Done false
22752 6655 Acked Flush 0 Sent Done Done false
22753 6656 Acked Read 6 Sent Done Done false
22754 6657 Acked Write 1 Sent Done Done false
22755 6658 Acked Write 8 Sent Done Done false
22756 6659 Acked Write 4 Sent Done Done false
22757 6660 Acked Write 5 Sent Done Done false
22758 6661 Acked Read 9 Sent Done Done false
22759 6662 Acked Write 8 Sent Done Done false
22760 6663 Acked Flush 0 Sent Done Done false
22761 6664 Acked Read 4 Sent Done Done false
22762 6665 Acked Flush 0 Sent Done Done false
22763 6666 Acked Write 6 Sent Done Done false
22764 6667 Acked Read 3 Sent Done Done false
22765 6668 Acked Read 5 Sent Done Done false
22766 6669 Acked Write 4 Sent Done Done false
22767 6670 Acked Write 4 Sent Done Done false
22768 6671 Acked Read 10 Sent Done Done false
22769 6672 Acked Write 4 Sent Done Done false
22770 6673 Acked Write 9 Sent Done Done false
22771 6674 Acked Write 8 Sent Done Done false
22772 6675 Acked Read 5 Sent Done Done false
22773 6676 Acked Read 3 Sent Done Done false
22774 6677 Acked Read 5 Sent Done Done false
22775 6678 Acked Flush 0 Sent Done Done false
22776 6679 Acked Flush 0 Sent Done Done false
22777 6680 Acked Read 2 Sent Done Done false
22778 6681 Acked Read 9 Sent Done Done false
22779 6682 Acked Read 3 Sent Done Done false
22780 6683 Acked Write 2 Sent Done Done false
22781 6684 Acked Flush 0 Sent Done Done false
22782 6685 Acked Write 2 Sent Done Done false
22783 6686 Acked Read 8 Sent Done Done false
22784 6687 Acked Write 10 Sent Done Done false
22785 6688 Acked Read 3 Sent Done Done false
22786 6689 Acked Read 5 Sent Done Done false
22787 6690 Acked Write 3 Sent Done Done false
22788 6691 Acked Read 2 Sent Done Done false
22789 6692 Acked Write 1 Sent Done Done false
22790 6693 Acked Write 3 Sent Done Done false
22791 6694 Acked Read 9 Sent Done Done false
22792 6695 Acked Read 9 Sent Done Done false
22793 6696 Acked Write 1 Sent Done Done false
22794 6697 Acked Write 9 Sent Done Done false
22795 6698 Acked Write 9 Sent Done Done false
22796 6699 Acked Read 6 Sent Done Done false
22797 6700 Acked Read 6 Sent Done Done false
22798 6701 Acked Write 9 Sent Done Done false
22799 6702 Acked Read 4 Sent Done Done false
22800 6703 Acked Write 10 Sent Done Done false
22801 6704 Acked Write 10 Sent Done Done false
22802 6705 Acked Read 5 Sent Done Done false
22803 6706 Acked Read 10 Sent Done Done false
22804 6707 Acked Read 10 Sent Done Done false
22805 6708 Acked Write 9 Sent Done Done false
22806 6709 Acked Read 2 Sent Done Done false
22807 6710 Acked Flush 0 Sent Done Done false
22808 6711 Acked Write 5 Sent Done Done false
22809 6712 Acked Read 2 Sent Done Done false
22810 6713 Acked Flush 0 Sent Done Done false
22811 6714 Acked Write 4 Sent Done Done false
22812 6715 Acked Write 8 Sent Done Done false
22813 6716 Acked Read 3 Sent Done Done false
22814 6717 Acked Read 9 Sent Done Done false
22815 6718 Acked Read 6 Sent Done Done false
22816 6719 Acked Read 10 Sent Done Done false
22817 6720 Acked Read 6 Sent Done Done false
22818 6721 Acked Read 7 Sent Done Done false
22819 6722 Acked Read 2 Sent Done Done false
22820 6723 Acked Read 7 Sent Done Done false
22821 6724 Acked Write 6 Sent Done Done false
22822 6725 Acked Read 4 Sent Done Done false
22823 6726 Acked Write 3 Sent Done Done false
22824 6727 Acked Read 6 Sent Done Done false
22825 6728 Acked Read 5 Sent Done Done false
22826 6729 Acked Write 1 Sent Done Done false
22827 6730 Acked Flush 0 Sent Done Done false
22828 6731 Acked Write 7 Sent Done Done false
22829 6732 Acked Read 8 Sent Done Done false
22830 6733 Acked Write 5 Sent Done Done false
22831 6734 Acked Write 9 Sent Done Done false
22832 6735 Acked Write 8 Sent Done Done false
22833 6736 Acked Write 8 Sent Done Done false
22834 STATES DS:0 DS:1 DS:2 TOTAL
22835 Sent 301 0 0 301
22836 Done 0 301 301 602
22837 Skipped 0 0 0 0
22838 Error 0 0 0 0
22839 Last Flush: 6435 6730 6730
22840 Downstairs last five completed: 6435 6434 6433 6432 6431
22841 Upstairs last five completed: 6736 6735 6734 6733 6732
22842 CLIENT: Up:0 ds:301 act:2
22843 [8810] got start action
22844 [8810] I am going to restart
22845 Starting downstairs at port 8810
22846 Make output file at "/tmp/dsc/downstairs-8810.txt"
22847 [8810][0] reports Starting
22848 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
22849 [8810][0] reports Running
22850 ----------------------------------------------------------------
22851 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22852 States: Offline Active Active
22853 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22854 6436 Acked Flush 0 Sent Done Done false
22855 6437 Acked Write 1 Sent Done Done false
22856 6438 Acked Read 7 Sent Done Done false
22857 6439 Acked Write 8 Sent Done Done false
22858 6440 Acked Write 3 Sent Done Done false
22859 6441 Acked Write 3 Sent Done Done false
22860 6442 Acked Write 6 Sent Done Done false
22861 6443 Acked Write 7 Sent Done Done false
22862 6444 Acked Write 6 Sent Done Done false
22863 6445 Acked Flush 0 Sent Done Done false
22864 6446 Acked Read 3 Sent Done Done false
22865 6447 Acked Flush 0 Sent Done Done false
22866 6448 Acked Read 3 Sent Done Done false
22867 6449 Acked Write 6 Sent Done Done false
22868 6450 Acked Write 2 Sent Done Done false
22869 6451 Acked Read 3 Sent Done Done false
22870 6452 Acked Write 5 Sent Done Done false
22871 6453 Acked Read 2 Sent Done Done false
22872 6454 Acked Read 4 Sent Done Done false
22873 6455 Acked Write 1 Sent Done Done false
22874 6456 Acked Read 10 Sent Done Done false
22875 6457 Acked Read 9 Sent Done Done false
22876 6458 Acked Read 1 Sent Done Done false
22877 6459 Acked Write 9 Sent Done Done false
22878 6460 Acked Write 2 Sent Done Done false
22879 6461 Acked Read 4 Sent Done Done false
22880 6462 Acked Read 5 Sent Done Done false
22881 6463 Acked Write 9 Sent Done Done false
22882 6464 Acked Flush 0 Sent Done Done false
22883 6465 Acked Write 6 Sent Done Done false
22884 6466 Acked Write 9 Sent Done Done false
22885 6467 Acked Write 9 Sent Done Done false
22886 6468 Acked Read 5 Sent Done Done false
22887 6469 Acked Read 2 Sent Done Done false
22888 6470 Acked Flush 0 Sent Done Done false
22889 6471 Acked Read 6 Sent Done Done false
22890 6472 Acked Write 10 Sent Done Done false
22891 6473 Acked Write 6 Sent Done Done false
22892 6474 Acked Flush 0 Sent Done Done false
22893 6475 Acked Write 10 Sent Done Done false
22894 6476 Acked Write 4 Sent Done Done false
22895 6477 Acked Read 3 Sent Done Done false
22896 6478 Acked Read 3 Sent Done Done false
22897 6479 Acked Write 3 Sent Done Done false
22898 6480 Acked Read 5 Sent Done Done false
22899 6481 Acked Read 3 Sent Done Done false
22900 6482 Acked Read 9 Sent Done Done false
22901 6483 Acked Read 10 Sent Done Done false
22902 6484 Acked Read 1 Sent Done Done false
22903 6485 Acked Write 5 Sent Done Done false
22904 6486 Acked Write 8 Sent Done Done false
22905 6487 Acked Flush 0 Sent Done Done false
22906 6488 Acked Write 3 Sent Done Done false
22907 6489 Acked Flush 0 Sent Done Done false
22908 6490 Acked Read 5 Sent Done Done false
22909 6491 Acked Read 1 Sent Done Done false
22910 6492 Acked Write 8 Sent Done Done false
22911 6493 Acked Write 1 Sent Done Done false
22912 6494 Acked Read 9 Sent Done Done false
22913 6495 Acked Write 7 Sent Done Done false
22914 6496 Acked Write 10 Sent Done Done false
22915 6497 Acked Flush 0 Sent Done Done false
22916 6498 Acked Read 6 Sent Done Done false
22917 6499 Acked Write 8 Sent Done Done false
22918 6500 Acked Write 1 Sent Done Done false
22919 6501 Acked Read 9 Sent Done Done false
22920 6502 Acked Flush 0 Sent Done Done false
22921 6503 Acked Write 8 Sent Done Done false
22922 6504 Acked Read 8 Sent Done Done false
22923 6505 Acked Read 2 Sent Done Done false
22924 6506 Acked Read 10 Sent Done Done false
22925 6507 Acked Read 1 Sent Done Done false
22926 6508 Acked Read 1 Sent Done Done false
22927 6509 Acked Read 3 Sent Done Done false
22928 6510 Acked Read 7 Sent Done Done false
22929 6511 Acked Read 3 Sent Done Done false
22930 6512 Acked Write 7 Sent Done Done false
22931 6513 Acked Write 2 Sent Done Done false
22932 6514 Acked Flush 0 Sent Done Done false
22933 6515 Acked Write 4 Sent Done Done false
22934 6516 Acked Flush 0 Sent Done Done false
22935 6517 Acked Write 9 Sent Done Done false
22936 6518 Acked Write 2 Sent Done Done false
22937 6519 Acked Read 9 Sent Done Done false
22938 6520 Acked Write 8 Sent Done Done false
22939 6521 Acked Read 4 Sent Done Done false
22940 6522 Acked Write 9 Sent Done Done false
22941 6523 Acked Flush 0 Sent Done Done false
22942 6524 Acked Read 9 Sent Done Done false
22943 6525 Acked Read 10 Sent Done Done false
22944 6526 Acked Write 10 Sent Done Done false
22945 6527 Acked Read 1 Sent Done Done false
22946 6528 Acked Write 5 Sent Done Done false
22947 6529 Acked Read 1 Sent Done Done false
22948 6530 Acked Write 3 Sent Done Done false
22949 6531 Acked Read 6 Sent Done Done false
22950 6532 Acked Flush 0 Sent Done Done false
22951 6533 Acked Write 9 Sent Done Done false
22952 6534 Acked Read 2 Sent Done Done false
22953 6535 Acked Read 1 Sent Done Done false
22954 6536 Acked Flush 0 Sent Done Done false
22955 6537 Acked Write 2 Sent Done Done false
22956 6538 Acked Read 9 Sent Done Done false
22957 6539 Acked Write 3 Sent Done Done false
22958 6540 Acked Read 4 Sent Done Done false
22959 6541 Acked Flush 0 Sent Done Done false
22960 6542 Acked Write 3 Sent Done Done false
22961 6543 Acked Write 6 Sent Done Done false
22962 6544 Acked Read 1 Sent Done Done false
22963 6545 Acked Write 3 Sent Done Done false
22964 6546 Acked Flush 0 Sent Done Done false
22965 6547 Acked Read 1 Sent Done Done false
22966 6548 Acked Read 7 Sent Done Done false
22967 6549 Acked Write 2 Sent Done Done false
22968 6550 Acked Write 3 Sent Done Done false
22969 6551 Acked Write 10 Sent Done Done false
22970 6552 Acked Write 4 Sent Done Done false
22971 6553 Acked Read 9 Sent Done Done false
22972 6554 Acked Read 9 Sent Done Done false
22973 6555 Acked Read 9 Sent Done Done false
22974 6556 Acked Read 7 Sent Done Done false
22975 6557 Acked Read 9 Sent Done Done false
22976 6558 Acked Read 2 Sent Done Done false
22977 6559 Acked Write 7 Sent Done Done false
22978 6560 Acked Write 5 Sent Done Done false
22979 6561 Acked Read 5 Sent Done Done false
22980 6562 Acked Read 3 Sent Done Done false
22981 6563 Acked Write 8 Sent Done Done false
22982 6564 Acked Read 6 Sent Done Done false
22983 6565 Acked Write 3 Sent Done Done false
22984 6566 Acked Read 2 Sent Done Done false
22985 6567 Acked Flush 0 Sent Done Done false
22986 6568 Acked Read 6 Sent Done Done false
22987 6569 Acked Flush 0 Sent Done Done false
22988 6570 Acked Read 3 Sent Done Done false
22989 6571 Acked Read 4 Sent Done Done false
22990 6572 Acked Write 2 Sent Done Done false
22991 6573 Acked Write 9 Sent Done Done false
22992 6574 Acked Write 4 Sent Done Done false
22993 6575 Acked Write 10 Sent Done Done false
22994 6576 Acked Read 10 Sent Done Done false
22995 6577 Acked Write 3 Sent Done Done false
22996 6578 Acked Write 10 Sent Done Done false
22997 6579 Acked Read 1 Sent Done Done false
22998 6580 Acked Write 7 Sent Done Done false
22999 6581 Acked Read 8 Sent Done Done false
23000 6582 Acked Read 7 Sent Done Done false
23001 6583 Acked Flush 0 Sent Done Done false
23002 6584 Acked Write 5 Sent Done Done false
23003 6585 Acked Read 1 Sent Done Done false
23004 6586 Acked Write 3 Sent Done Done false
23005 6587 Acked Read 8 Sent Done Done false
23006 6588 Acked Write 5 Sent Done Done false
23007 6589 Acked Read 9 Sent Done Done false
23008 6590 Acked Read 9 Sent Done Done false
23009 6591 Acked Read 6 Sent Done Done false
23010 6592 Acked Write 10 Sent Done Done false
23011 6593 Acked Read 2 Sent Done Done false
23012 6594 Acked Write 7 Sent Done Done false
23013 6595 Acked Read 2 Sent Done Done false
23014 6596 Acked Write 3 Sent Done Done false
23015 6597 Acked Read 5 Sent Done Done false
23016 6598 Acked Flush 0 Sent Done Done false
23017 6599 Acked Write 10 Sent Done Done false
23018 6600 Acked Write 8 Sent Done Done false
23019 6601 Acked Read 4 Sent Done Done false
23020 6602 Acked Flush 0 Sent Done Done false
23021 6603 Acked Write 10 Sent Done Done false
23022 6604 Acked Read 1 Sent Done Done false
23023 6605 Acked Read 6 Sent Done Done false
23024 6606 Acked Read 8 Sent Done Done false
23025 6607 Acked Write 10 Sent Done Done false
23026 6608 Acked Read 10 Sent Done Done false
23027 6609 Acked Write 5 Sent Done Done false
23028 6610 Acked Write 8 Sent Done Done false
23029 6611 Acked Read 10 Sent Done Done false
23030 6612 Acked Write 9 Sent Done Done false
23031 6613 Acked Flush 0 Sent Done Done false
23032 6614 Acked Write 5 Sent Done Done false
23033 6615 Acked Write 3 Sent Done Done false
23034 6616 Acked Write 7 Sent Done Done false
23035 6617 Acked Write 6 Sent Done Done false
23036 6618 Acked Read 10 Sent Done Done false
23037 6619 Acked Write 9 Sent Done Done false
23038 6620 Acked Write 5 Sent Done Done false
23039 6621 Acked Write 6 Sent Done Done false
23040 6622 Acked Read 5 Sent Done Done false
23041 6623 Acked Read 8 Sent Done Done false
23042 6624 Acked Write 3 Sent Done Done false
23043 6625 Acked Read 9 Sent Done Done false
23044 6626 Acked Read 5 Sent Done Done false
23045 6627 Acked Flush 0 Sent Done Done false
23046 6628 Acked Read 2 Sent Done Done false
23047 6629 Acked Flush 0 Sent Done Done false
23048 6630 Acked Read 9 Sent Done Done false
23049 6631 Acked Read 6 Sent Done Done false
23050 6632 Acked Write 2 Sent Done Done false
23051 6633 Acked Read 5 Sent Done Done false
23052 6634 Acked Write 5 Sent Done Done false
23053 6635 Acked Write 8 Sent Done Done false
23054 6636 Acked Read 1 Sent Done Done false
23055 6637 Acked Read 4 Sent Done Done false
23056 6638 Acked Write 10 Sent Done Done false
23057 6639 Acked Read 3 Sent Done Done false
23058 6640 Acked Write 1 Sent Done Done false
23059 6641 Acked Write 6 Sent Done Done false
23060 6642 Acked Write 2 Sent Done Done false
23061 6643 Acked Read 5 Sent Done Done false
23062 6644 Acked Flush 0 Sent Done Done false
23063 6645 Acked Read 8 Sent Done Done false
23064 6646 Acked Read 4 Sent Done Done false
23065 6647 Acked Write 2 Sent Done Done false
23066 6648 Acked Write 8 Sent Done Done false
23067 6649 Acked Read 1 Sent Done Done false
23068 6650 Acked Flush 0 Sent Done Done false
23069 6651 Acked Read 10 Sent Done Done false
23070 6652 Acked Write 10 Sent Done Done false
23071 6653 Acked Write 10 Sent Done Done false
23072 6654 Acked Write 2 Sent Done Done false
23073 6655 Acked Flush 0 Sent Done Done false
23074 6656 Acked Read 6 Sent Done Done false
23075 6657 Acked Write 1 Sent Done Done false
23076 6658 Acked Write 8 Sent Done Done false
23077 6659 Acked Write 4 Sent Done Done false
23078 6660 Acked Write 5 Sent Done Done false
23079 6661 Acked Read 9 Sent Done Done false
23080 6662 Acked Write 8 Sent Done Done false
23081 6663 Acked Flush 0 Sent Done Done false
23082 6664 Acked Read 4 Sent Done Done false
23083 6665 Acked Flush 0 Sent Done Done false
23084 6666 Acked Write 6 Sent Done Done false
23085 6667 Acked Read 3 Sent Done Done false
23086 6668 Acked Read 5 Sent Done Done false
23087 6669 Acked Write 4 Sent Done Done false
23088 6670 Acked Write 4 Sent Done Done false
23089 6671 Acked Read 10 Sent Done Done false
23090 6672 Acked Write 4 Sent Done Done false
23091 6673 Acked Write 9 Sent Done Done false
23092 6674 Acked Write 8 Sent Done Done false
23093 6675 Acked Read 5 Sent Done Done false
23094 6676 Acked Read 3 Sent Done Done false
23095 6677 Acked Read 5 Sent Done Done false
23096 6678 Acked Flush 0 Sent Done Done false
23097 6679 Acked Flush 0 Sent Done Done false
23098 6680 Acked Read 2 Sent Done Done false
23099 6681 Acked Read 9 Sent Done Done false
23100 6682 Acked Read 3 Sent Done Done false
23101 6683 Acked Write 2 Sent Done Done false
23102 6684 Acked Flush 0 Sent Done Done false
23103 6685 Acked Write 2 Sent Done Done false
23104 6686 Acked Read 8 Sent Done Done false
23105 6687 Acked Write 10 Sent Done Done false
23106 6688 Acked Read 3 Sent Done Done false
23107 6689 Acked Read 5 Sent Done Done false
23108 6690 Acked Write 3 Sent Done Done false
23109 6691 Acked Read 2 Sent Done Done false
23110 6692 Acked Write 1 Sent Done Done false
23111 6693 Acked Write 3 Sent Done Done false
23112 6694 Acked Read 9 Sent Done Done false
23113 6695 Acked Read 9 Sent Done Done false
23114 6696 Acked Write 1 Sent Done Done false
23115 6697 Acked Write 9 Sent Done Done false
23116 6698 Acked Write 9 Sent Done Done false
23117 6699 Acked Read 6 Sent Done Done false
23118 6700 Acked Read 6 Sent Done Done false
23119 6701 Acked Write 9 Sent Done Done false
23120 6702 Acked Read 4 Sent Done Done false
23121 6703 Acked Write 10 Sent Done Done false
23122 6704 Acked Write 10 Sent Done Done false
23123 6705 Acked Read 5 Sent Done Done false
23124 6706 Acked Read 10 Sent Done Done false
23125 6707 Acked Read 10 Sent Done Done false
23126 6708 Acked Write 9 Sent Done Done false
23127 6709 Acked Read 2 Sent Done Done false
23128 6710 Acked Flush 0 Sent Done Done false
23129 6711 Acked Write 5 Sent Done Done false
23130 6712 Acked Read 2 Sent Done Done false
23131 6713 Acked Flush 0 Sent Done Done false
23132 6714 Acked Write 4 Sent Done Done false
23133 6715 Acked Write 8 Sent Done Done false
23134 6716 Acked Read 3 Sent Done Done false
23135 6717 Acked Read 9 Sent Done Done false
23136 6718 Acked Read 6 Sent Done Done false
23137 6719 Acked Read 10 Sent Done Done false
23138 6720 Acked Read 6 Sent Done Done false
23139 6721 Acked Read 7 Sent Done Done false
23140 6722 Acked Read 2 Sent Done Done false
23141 6723 Acked Read 7 Sent Done Done false
23142 6724 Acked Write 6 Sent Done Done false
23143 6725 Acked Read 4 Sent Done Done false
23144 6726 Acked Write 3 Sent Done Done false
23145 6727 Acked Read 6 Sent Done Done false
23146 6728 Acked Read 5 Sent Done Done false
23147 6729 Acked Write 1 Sent Done Done false
23148 6730 Acked Flush 0 Sent Done Done false
23149 6731 Acked Write 7 Sent Done Done false
23150 6732 Acked Read 8 Sent Done Done false
23151 6733 Acked Write 5 Sent Done Done false
23152 6734 Acked Write 9 Sent Done Done false
23153 6735 Acked Write 8 Sent Done Done false
23154 6736 Acked Write 8 Sent Done Done false
23155 6737 Acked Flush 0 Sent Done Done false
23156 STATES DS:0 DS:1 DS:2 TOTAL
23157 Sent 302 0 0 302
23158 Done 0 302 302 604
23159 Skipped 0 0 0 0
23160 Error 0 0 0 0
23161 Last Flush: 6435 6737 6737
23162 Downstairs last five completed: 6435 6434 6433 6432 6431
23163 Upstairs last five completed: 6737 6736 6735 6734 6733
23164 CLIENT: Up:0 ds:302 act:2
231652024-12-11T23:45:20.950ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231662024-12-11T23:45:20.950ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57162) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231672024-12-11T23:45:20.951ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231682024-12-11T23:45:20.952ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231692024-12-11T23:45:20.952ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231702024-12-11T23:45:20.952ZINFOcrucible: send last flush ID to this DS: 6435 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231712024-12-11T23:45:20.952ZINFOcrucible: Replied this last flush ID: 6435 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231722024-12-11T23:45:20.952ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231732024-12-11T23:45:20.952ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231742024-12-11T23:45:20.952ZINFOcrucible: [0] client re-new 302 jobs since flush 6435 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231752024-12-11T23:45:20.953ZINFOcrucible: [0] Replayed 302 jobs since flush: 6435 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
23176 ----------------------------------------------------------------
23177 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23178 Downstairs last five completed: 6737 6736 6735 6734 6733
23179 Upstairs last five completed: 6737 6736 6735 6734 6733
23180 CLIENT: Up:0 ds:0 act:3
23181 Replay: All jobs finished
23182 Main task has work to do, go find it
23183 got dsc Stop(0)
23184 stop 0
23185 Dec 11 23:45:23.189 INFO request completed[8810] Got stop action so:false kr:false
23186 , latency_us: 354, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 17ec975f-0cb1-45e4-9912-3a84d1eb1fec, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
23187 Dec 11 23:45:23.190 INFO request completed, latency_us: 115, response_code: 200, uri: /state/cid/0, method: GET, req_id: fd6d6cf4-70ef-42f1-b36f-3a7e14154e03, remote_addr: 127.{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:45:23.190706411Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
23188 0.0.1:62969, local_addr: 127.0.0.1:9998
231892024-12-11T23:45:23.190ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231902024-12-11T23:45:23.190ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231912024-12-11T23:45:23.190ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231922024-12-11T23:45:23.190ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
231932024-12-11T23:45:23.190ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
23194 [8810] Exited with: ExitStatus(unix_wait_status(9))
23195 [8810][0] reports Exit
23196 Dec 11 23:45:27.192 INFO request completed, latency_us: 263, response_code: 200, uri: /state/cid/0, method: GET, req_id: b6c3a696-38e3-41cd-8441-c455c4cba023, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
23197 001/300 Write block 4394 len 32768 data: 4 4 3 5 6 7 7 9
23198 002/300 Write block 4375 len 24576 data: 3 3 2 5 4 6
23199 003/300 Read block 3005 len 32768
23200 004/300 Read block 3123 len 28672
23201 005/300 Write block 1577 len 36864 data: 3 3 4 4 4 4 4 3 3
23202 006/300 Read block 2455 len 40960
23203 007/300 Read block 4770 len 36864
23204 008/300 Read block 3728 len 32768
23205 009/300 Read block 2199 len 40960
23206 010/300 Write block 3297 len 12288 data: 4 5 4
23207 011/300 Write block 2205 len 40960 data: 2 3 3 4 4 6 7 7 6 6
23208 012/300 Write block 3930 len 4096 data: 2
23209 013/300 Read block 39 len 8192
23210 014/300 Write block 4484 len 24576 data: 4 4 4 2 3 3
23211 015/300 Write block 1044 len 24576 data: 5 5 5 5 3 3
23212 016/300 Flush
23213 017/300 Write block 1346 len 8192 data: 3 4
23214 018/300 Write block 3445 len 4096 data: 4
23215 019/300 Write block 1638 len 24576 data: 5 5 5 6 8 7
23216 020/300 Read block 4394 len 24576
23217 021/300 Write block 271 len 32768 data: 4 3 3 3 2 2 2 2
23218 022/300 Write block 3575 len 16384 data: 5 5 5 5
23219 023/300 Read block 2818 len 36864
23220 024/300 Flush
23221 025/300 Flush
23222 026/300 Write block 1723 len 32768 data: 4 3 3 4 3 4 4 4
23223 027/300 Write block 4779 len 24576 data: 5 4 4 4 6 7
23224 028/300 Write block 1540 len 28672 data: 4 6 5 5 6 5 6
23225 029/300 Flush
23226 030/300 Read block 3043 len 20480
23227 031/300 Write block 2576 len 16384 data: 9 10 11 10
23228 032/300 Read block 935 len 24576
23229 033/300 Write block 1196 len 4096 data: 7
23230 034/300 Write block 2252 len 40960 data: 2 2 2 2 2 2 2 3 3 4
23231 035/300 Write block 1183 len 8192 data: 6 5
23232 036/300 Read block 1572 len 4096
23233 037/300 Write block 1744 len 16384 data: 5 6 7 5
23234 038/300 Read block 924 len 24576
23235 039/300 Read block 4741 len 20480
23236 040/300 Write block 1943 len 16384 data: 4 5 4 4
23237 041/300 Write block 2816 len 24576 data: 4 3 3 3 2 3
23238 042/300 Read block 1469 len 16384
23239 043/300 Write block 3090 len 28672 data: 8 6 7 9 9 8 7
23240 044/300 Write block 300 len 28672 data: 4 4 4 4 6 5 4
23241 045/300 Write block 3 len 12288 data: 2 3 3
23242 046/300 Write block 4783 len 36864 data: 7 8 6 6 5 4 3 2 3
23243 047/300 Read block 2550 len 32768
23244 048/300 Read block 3233 len 32768
23245 049/300 Write block 1563 len 20480 data: 4 5 5 4 5
23246 050/300 Flush
23247 051/300 Write block 4120 len 24576 data: 5 5 8 5 7 7
23248 052/300 Read block 4130 len 16384
23249 053/300 Write block 3231 len 32768 data: 4 4 3 3 3 3 4 3
23250 054/300 Read block 4814 len 20480
23251 055/300 Write block 2501 len 12288 data: 4 3 3
23252 056/300 Read block 82 len 20480
23253 057/300 Flush
23254 058/300 Read block 3845 len 40960
23255 059/300 Flush
23256 060/300 Read block 485 len 20480
23257 061/300 Write block 12 len 20480 data: 5 4 3 2 2
23258 062/300 Read block 2186 len 32768
23259 063/300 Write block 64 len 8192 data: 4 4
23260 064/300 Write block 1204 len 12288 data: 6 6 5
23261 065/300 Write block 1874 len 24576 data: 3 3 3 3 3 3
23262 066/300 Write block 4382 len 32768 data: 5 5 5 5 5 5 5 4
23263 067/300 Read block 2175 len 40960
23264 068/300 Write block 334 len 4096 data: 4
23265 069/300 Write block 2358 len 36864 data: 5 5 5 5 5 5 6 6 6
23266 070/300 Write block 2312 len 36864 data: 6 5 6 5 4 4 3 4 4
23267 071/300 Write block 4775 len 8192 data: 4 4
23268 072/300 Write block 2262 len 4096 data: 4
23269 073/300 Read block 531 len 4096
23270 074/300 Read block 4293 len 32768
23271 075/300 Write block 3324 len 36864 data: 4 4 6 7 8 9 7 6 5
23272 076/300 Write block 1393 len 36864 data: 3 3 3 3 3 3 3 2 2
23273 077/300 Write block 3952 len 32768 data: 4 4 4 4 5 4 6 7
23274 078/300 Write block 2885 len 40960 data: 4 4 5 6 6 6 5 5 5 6
23275 079/300 Flush
23276 080/300 Read block 4906 len 16384
23277 081/300 Read block 2163 len 20480
23278 082/300 Write block 1632 len 24576 data: 6 6 4 5 5 5
23279 083/300 Write block 1018 len 8192 data: 5 5
23280 084/300 Read block 4124 len 32768
23281 085/300 Flush
23282 086/300 Write block 4341 len 28672 data: 4 4 3 3 4 3 6
23283 087/300 Read block 3483 len 8192
23284 088/300 Read block 4890 len 24576
23285 089/300 Read block 607 len 36864
23286 090/300 Write block 102 len 28672 data: 7 7 8 7 5 4 5
23287 091/300 Write block 2716 len 32768 data: 9 7 8 9 8 8 7 8
23288 092/300 Flush
23289 093/300 Read block 3552 len 4096
23290 094/300 Write block 4890 len 24576 data: 5 4 5 6 6 5
23291 095/300 Flush
23292 096/300 Read block 3509 len 28672
23293 097/300 Write block 2708 len 28672 data: 5 5 7 6 7 6 7
23294 098/300 Write block 2160 len 12288 data: 4 3 2
23295 099/300 Read block 3087 len 20480
23296 100/300 Write block 695 len 16384 data: 5 4 4 4
23297 101/300 Read block 2030 len 12288
23298 102/300 Write block 2690 len 16384 data: 5 5 6 6
23299 103/300 Flush
23300 104/300 Read block 4258 len 4096
23301 105/300 Write block 2660 len 4096 data: 3
23302 106/300 Write block 3202 len 12288 data: 4 5 5
23303 107/300 Flush
23304 108/300 Write block 2595 len 24576 data: 4 4 4 5 5 6
23305 109/300 Write block 4340 len 28672 data: 4 5 5 4 4 5 4
23306 110/300 Read block 204 len 20480
23307 111/300 Write block 63 len 40960 data: 3 5 5 4 5 6 7 7 7 7
23308 112/300 Write block 3814 len 24576 data: 8 6 7 7 7 5
23309 113/300 Flush
23310 114/300 Flush
23311 115/300 Write block 4160 len 20480 data: 2 2 2 5 5
23312 116/300 Read block 3253 len 36864
23313 117/300 Read block 101 len 32768
23314 118/300 Read block 262 len 4096
23315 119/300 Write block 2784 len 36864 data: 3 4 4 4 4 4 4 4 3
23316 120/300 Read block 3364 len 24576
23317 121/300 Read block 205 len 36864
23318 122/300 Read block 2779 len 4096
23319 123/300 Write block 4385 len 20480 data: 6 6 6 6 5
23320 124/300 Write block 1070 len 36864 data: 4 4 3 4 4 3 4 5 5
23321 125/300 Write block 787 len 28672 data: 4 3 3 3 2 2 2
23322 126/300 Flush
23323 127/300 Write block 1570 len 28672 data: 4 6 4 3 3 5 4
23324 128/300 Read block 404 len 16384
23325 129/300 Read block 579 len 32768
23326 130/300 Read block 139 len 32768
23327 131/300 Read block 1760 len 8192
23328 132/300 Flush
23329 133/300 Write block 883 len 32768 data: 8 8 8 7 7 5 4 5
23330 134/300 Flush
23331 135/300 Read block 2936 len 40960
23332 136/300 Read block 4318 len 12288
23333 137/300 Read block 233 len 4096
23334 138/300 Write block 2549 len 8192 data: 7 5
23335 139/300 Read block 35 len 40960
23336 140/300 Read block 4495 len 28672
23337 141/300 Write block 4336 len 32768 data: 2 3 3 3 5 6 6 5
23338 142/300 Read block 1185 len 16384
23339 143/300 Write block 3085 len 20480 data: 5 6 6 6 8
23340 144/300 Write block 752 len 4096 data: 6
23341 145/300 Write block 3915 len 4096 data: 4
23342 146/300 Flush
23343 147/300 Write block 313 len 28672 data: 4 5 5 5 6 5 4
23344 148/300 Write block 4132 len 36864 data: 4 3 4 4 5 5 3 5 4
23345 149/300 Read block 1677 len 32768
23346 150/300 Write block 3865 len 4096 data: 9
23347 151/300 Read block 2669 len 28672
23348 152/300 Read block 4801 len 12288
23349 153/300 Write block 3300 len 12288 data: 4 4 4
23350 154/300 Read block 566 len 36864
23351 155/300 Read block 4168 len 16384
23352 156/300 Read block 3568 len 8192
23353 157/300 Write block 3151 len 32768 data: 6 5 6 5 7 7 6 6
23354 158/300 Read block 2467 len 36864
23355 159/300 Write block 1498 len 24576 data: 7 7 8 7 7 7
23356 160/300 Write block 640 len 32768 data: 7 4 4 4 4 5 3 3
23357 161/300 Read block 1293 len 20480
23358 162/300 Flush
23359 163/300 Read block 552 len 40960
23360 164/300 Write block 778 len 4096 data: 4
23361 165/300 Read block 4270 len 40960
23362 166/300 Write block 1492 len 4096 data: 7
23363 167/300 Read block 1056 len 36864
23364 168/300 Write block 1949 len 4096 data: 6
23365 169/300 Read block 1032 len 32768
23366 170/300 Read block 4939 len 16384
23367 171/300 Flush
23368 172/300 Write block 3652 len 28672 data: 5 4 3 4 5 5 5
23369 173/300 Read block 601 len 20480
23370 174/300 Read block 1472 len 16384
23371 175/300 Write block 2783 len 4096 data: 3
23372 176/300 Write block 931 len 36864 data: 2 3 3 4 4 3 5 5 6
23373 177/300 Read block 1295 len 40960
23374 178/300 Write block 3001 len 4096 data: 7
23375 179/300 Read block 88 len 32768
23376 180/300 Write block 2330 len 4096 data: 7
23377 181/300 Read block 2323 len 32768
23378 182/300 Flush
23379 183/300 Write block 4253 len 16384 data: 7 6 6 7
23380 184/300 Write block 3645 len 36864 data: 5 6 5 7 5 4 5 6 5
23381 185/300 Read block 2201 len 36864
23382 186/300 Read block 1515 len 28672
23383 187/300 Write block 3208 len 8192 data: 6 7
23384 188/300 Read block 2969 len 4096
23385 189/300 Write block 1852 len 16384 data: 5 3 5 5
23386 190/300 Write block 4091 len 8192 data: 4 3
23387 191/300 Write block 2821 len 36864 data: 4 4 6 6 4 4 5 6 6
23388 192/300 Write block 1842 len 36864 data: 4 3 3 3 3 4 3 3 4
23389 193/300 Read block 3032 len 20480
23390 194/300 Write block 3956 len 24576 data: 6 5 7 8 10 11
23391 195/300 Read block 2009 len 24576
23392 196/300 Read block 2922 len 24576
23393 197/300 Flush
23394 198/300 Write block 502 len 12288 data: 4 5 5
23395 199/300 Read block 910 len 8192
23396 200/300 Write block 1776 len 28672 data: 5 3 2 4 5 7 6
23397 201/300 Read block 2716 len 12288
23398 202/300 Flush
23399 203/300 Write block 1875 len 36864 data: 4 4 4 4 4 4 4 5 5
23400 204/300 Read block 2408 len 36864
23401 205/300 Write block 906 len 20480 data: 3 3 3 3 3
23402 206/300 Read block 2927 len 12288
23403 207/300 Read block 4427 len 36864
23404 208/300 Read block 1709 len 4096
23405 209/300 Write block 836 len 12288 data: 5 5 5
23406 210/300 Flush
23407 211/300 Write block 2152 len 32768 data: 3 4 4 5 5 5 5 5
23408 212/300 Write block 367 len 12288 data: 2 3 2
23409 213/300 Read block 113 len 16384
23410 214/300 Read block 4535 len 32768
23411 215/300 Write block 2346 len 40960 data: 6 6 5 5 5 4 3 3 3 6
23412 216/300 Flush
23413 217/300 Flush
23414 218/300 Flush
23415 219/300 Write block 4231 len 24576 data: 4 4 6 5 7 7
23416 220/300 Flush
23417 221/300 Read block 2448 len 4096
23418 222/300 Flush
23419 223/300 Read block 2979 len 36864
23420 224/300 Write block 4617 len 16384 data: 6 5 5 3
23421 225/300 Write block 3947 len 28672 data: 7 7 7 6 5 5 5
23422 226/300 Write block 833 len 28672 data: 5 5 5 6 6 6 5
23423 227/300 Write block 4017 len 40960 data: 8 8 7 7 4 4 4 3 3 2
23424 228/300 Write block 3859 len 24576 data: 7 7 8 10 10 10
23425 229/300 Flush
23426 230/300 Read block 3228 len 32768
23427 231/300 Write block 3244 len 32768 data: 7 6 5 3 2 2 2 2
23428 232/300 Write block 4822 len 24576 data: 2 2 2 3 5 4
23429 233/300 Read block 1833 len 36864
23430 234/300 Write block 2328 len 20480 data: 7 7 8 6 7
23431 235/300 Write block 2770 len 24576 data: 4 3 3 3 3 4
23432 236/300 Read block 3422 len 16384
23433 237/300 Read block 1610 len 16384
23434 238/300 Read block 2514 len 20480
23435 239/300 Write block 1904 len 4096 data: 3
23436 240/300 Read block 521 len 36864
23437 241/300 Read block 1239 len 40960
23438 242/300 Read block 3298 len 20480
23439 243/300 Write block 4328 len 12288 data: 2 2 2
23440 244/300 Write block 995 len 16384 data: 3 3 3 4
23441 245/300 Write block 4755 len 28672 data: 6 4 4 3 4 4 3
23442 246/300 Write block 3516 len 8192 data: 3 3
23443 247/300 Flush
23444 248/300 Read block 2700 len 20480
23445 249/300 Write block 3695 len 12288 data: 6 6 7
23446 250/300 Read block 4186 len 4096
23447 251/300 Read block 3527 len 28672
23448 252/300 Write block 2381 len 36864 data: 4 5 4 3 3 3 3 3 2
23449 253/300 Read block 1153 len 32768
23450 254/300 Flush
23451 255/300 Read block 4743 len 16384
23452 256/300 Write block 1680 len 20480 data: 3 3 4 6 7
23453 257/300 Flush
23454 258/300 Read block 1640 len 32768
23455 259/300 Write block 3371 len 40960 data: 5 5 4 4 3 2 2 2 3 4
23456 260/300 Flush
23457 261/300 Write block 835 len 20480 data: 6 7 7 7 6
23458 262/300 Write block 2962 len 40960 data: 9 9 7 7 6 4 4 2 3 3
23459 263/300 Read block 139 len 28672
23460 264/300 Read block 1264 len 32768
23461 265/300 Write block 4723 len 28672 data: 7 6 4 4 4 4 4
23462 266/300 Read block 3094 len 20480
23463 267/300 Write block 1571 len 16384 data: 7 5 4 4
23464 268/300 Write block 2014 len 4096 data: 4
23465 269/300 Write block 2923 len 20480 data: 4 4 3 4 4
23466 270/300 Flush
23467 271/300 Write block 2214 len 40960 data: 7 6 7 7 6 4 4 4 4 3
23468 272/300 Read block 4897 len 20480
23469 273/300 Read block 271 len 36864
23470 274/300 Write block 2710 len 28672 data: 8 7 8 7 8 7 10
23471 275/300 Write block 4736 len 24576 data: 4 5 4 3 3 3
23472 276/300 Read block 3916 len 4096
23473 277/300 Read block 178 len 32768
23474 278/300 Flush
23475 279/300 Write block 3636 len 4096 data: 2
23476 280/300 Read block 304 len 4096
23477 281/300 Read block 2996 len 40960
23478 282/300 Read block 4154 len 12288
23479 283/300 Read block 1031 len 20480
23480 284/300 Read block 3016 len 12288
23481 285/300 Write block 510 len 20480 data: 4 5 5 4 4
23482 286/300 Write block 1359 len 16384 data: 5 4 5 5
23483 287/300 Read block 4639 len 8192
23484 288/300 Read block 2340 len 20480
23485 289/300 Write block 534 len 8192 data: 3 3
23486 290/300 Read block 3725 len 20480
23487 291/300 Write block 4140 len 36864 data: 5 4 4 5 5 4 4 5 5
23488 292/300 Read block 640 len 36864
23489 293/300 Write block 3131 len 4096 data: 5
23490 294/300 Write block 2552 len 16384 data: 5 5 5 4
23491 295/300 Read block 3797 len 28672
23492 296/300 Read block 1384 len 36864
23493 297/300 Write block 2709 len 8192 data: 6 9
23494 298/300 Read block 3360 len 12288
23495 299/300 Read block 183 len 36864
23496 300/300 Read block 1901 len 40960
23497 301/300 Read block 3061 len 36864
23498 Main task has work to do, go find it
23499 got dsc Start(0)
23500 start 0
23501 Dec 11 23:45:27.422 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/0, method: POST, req_id: 210f4cb5-0c11-4ce1-9d14-8048dc47681e, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
23502 Replay: started 0, returned:Ok(())
23503 ----------------------------------------------------------------
23504 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23505 States: Offline Active Active
23506 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23507 6738 Acked Write 8 Sent Done Done false
23508 6739 Acked Write 6 Sent Done Done false
23509 6740 Acked Read 8 Sent Done Done false
23510 6741 Acked Read 7 Sent Done Done false
23511 6742 Acked Write 9 Sent Done Done false
23512 6743 Acked Read 10 Sent Done Done false
23513 6744 Acked Read 9 Sent Done Done false
23514 6745 Acked Read 8 Sent Done Done false
23515 6746 Acked Read 10 Sent Done Done false
23516 6747 Acked Write 3 Sent Done Done false
23517 6748 Acked Write 10 Sent Done Done false
23518 6749 Acked Write 1 Sent Done Done false
23519 6750 Acked Read 2 Sent Done Done false
23520 6751 Acked Write 6 Sent Done Done false
23521 6752 Acked Write 6 Sent Done Done false
23522 6753 Acked Flush 0 Sent Done Done false
23523 6754 Acked Write 2 Sent Done Done false
23524 6755 Acked Write 1 Sent Done Done false
23525 6756 Acked Write 6 Sent Done Done false
23526 6757 Acked Read 6 Sent Done Done false
23527 6758 Acked Write 8 Sent Done Done false
23528 6759 Acked Write 4 Sent Done Done false
23529 6760 Acked Read 9 Sent Done Done false
23530 6761 Acked Flush 0 Sent Done Done false
23531 6762 Acked Flush 0 Sent Done Done false
23532 6763 Acked Write 8 Sent Done Done false
23533 6764 Acked Write 6 Sent Done Done false
23534 6765 Acked Write 7 Sent Done Done false
23535 6766 Acked Flush 0 Sent Done Done false
23536 6767 Acked Read 5 Sent Done Done false
23537 6768 Acked Write 4 Sent Done Done false
23538 6769 Acked Read 6 Sent Done Done false
23539 6770 Acked Write 1 Sent Done Done false
23540 6771 Acked Write 10 Sent Done Done false
23541 6772 Acked Write 2 Sent Done Done false
23542 6773 Acked Read 1 Sent Done Done false
23543 6774 Acked Write 4 Sent Done Done false
23544 6775 Acked Read 6 Sent Done Done false
23545 6776 Acked Read 5 Sent Done Done false
23546 6777 Acked Write 4 Sent Done Done false
23547 6778 Acked Write 6 Sent Done Done false
23548 6779 Acked Read 4 Sent Done Done false
23549 6780 Acked Write 7 Sent Done Done false
23550 6781 Acked Write 7 Sent Done Done false
23551 6782 Acked Write 3 Sent Done Done false
23552 6783 Acked Write 9 Sent Done Done false
23553 6784 Acked Read 8 Sent Done Done false
23554 6785 Acked Read 8 Sent Done Done false
23555 6786 Acked Write 5 Sent Done Done false
23556 6787 Acked Flush 0 Sent Done Done false
23557 6788 Acked Write 6 Sent Done Done false
23558 6789 Acked Read 4 Sent Done Done false
23559 6790 Acked Write 8 Sent Done Done false
23560 6791 Acked Read 5 Sent Done Done false
23561 6792 Acked Write 3 Sent Done Done false
23562 6793 Acked Read 5 Sent Done Done false
23563 6794 Acked Flush 0 Sent Done Done false
23564 6795 Acked Read 10 Sent Done Done false
23565 6796 Acked Flush 0 Sent Done Done false
23566 6797 Acked Read 5 Sent Done Done false
23567 6798 Acked Write 5 Sent Done Done false
23568 6799 Acked Read 8 Sent Done Done false
23569 6800 Acked Write 2 Sent Done Done false
23570 6801 Acked Write 3 Sent Done Done false
23571 6802 Acked Write 6 Sent Done Done false
23572 6803 Acked Write 8 Sent Done Done false
23573 6804 Acked Read 10 Sent Done Done false
23574 6805 Acked Write 1 Sent Done Done false
23575 6806 Acked Write 9 Sent Done Done false
23576 6807 Acked Write 9 Sent Done Done false
23577 6808 Acked Write 2 Sent Done Done false
23578 6809 Acked Write 1 Sent Done Done false
23579 6810 Acked Read 1 Sent Done Done false
23580 6811 Acked Read 8 Sent Done Done false
23581 6812 Acked Write 9 Sent Done Done false
23582 6813 Acked Write 9 Sent Done Done false
23583 6814 Acked Write 8 Sent Done Done false
23584 6815 Acked Write 10 Sent Done Done false
23585 6816 Acked Flush 0 Sent Done Done false
23586 6817 Acked Read 4 Sent Done Done false
23587 6818 Acked Read 5 Sent Done Done false
23588 6819 Acked Write 6 Sent Done Done false
23589 6820 Acked Write 2 Sent Done Done false
23590 6821 Acked Read 8 Sent Done Done false
23591 6822 Acked Flush 0 Sent Done Done false
23592 6823 Acked Write 7 Sent Done Done false
23593 6824 Acked Read 2 Sent Done Done false
23594 6825 Acked Read 6 Sent Done Done false
23595 6826 Acked Read 9 Sent Done Done false
23596 6827 Acked Write 7 Sent Done Done false
23597 6828 Acked Write 8 Sent Done Done false
23598 6829 Acked Flush 0 Sent Done Done false
23599 6830 Acked Read 1 Sent Done Done false
23600 6831 Acked Write 6 Sent Done Done false
23601 6832 Acked Flush 0 Sent Done Done false
23602 6833 Acked Read 7 Sent Done Done false
23603 6834 Acked Write 7 Sent Done Done false
23604 6835 Acked Write 3 Sent Done Done false
23605 6836 Acked Read 5 Sent Done Done false
23606 6837 Acked Write 4 Sent Done Done false
23607 6838 Acked Read 3 Sent Done Done false
23608 6839 Acked Write 4 Sent Done Done false
23609 6840 Acked Flush 0 Sent Done Done false
23610 6841 Acked Read 1 Sent Done Done false
23611 6842 Acked Write 1 Sent Done Done false
23612 6843 Acked Write 3 Sent Done Done false
23613 6844 Acked Flush 0 Sent Done Done false
23614 6845 Acked Write 6 Sent Done Done false
23615 6846 Acked Write 7 Sent Done Done false
23616 6847 Acked Read 5 Sent Done Done false
23617 6848 Acked Write 10 Sent Done Done false
23618 6849 Acked Write 6 Sent Done Done false
23619 6850 Acked Flush 0 Sent Done Done false
23620 6851 Acked Flush 0 Sent Done Done false
23621 6852 Acked Write 5 Sent Done Done false
23622 6853 Acked Read 9 Sent Done Done false
23623 6854 Acked Read 8 Sent Done Done false
23624 6855 Acked Read 1 Sent Done Done false
23625 6856 Acked Write 9 Sent Done Done false
23626 6857 Acked Read 6 Sent Done Done false
23627 6858 Acked Read 9 Sent Done Done false
23628 6859 Acked Read 1 Sent Done Done false
23629 6860 Acked Write 5 Sent Done Done false
23630 6861 Acked Write 9 Sent Done Done false
23631 6862 Acked Write 7 Sent Done Done false
23632 6863 Acked Flush 0 Sent Done Done false
23633 6864 Acked Write 7 Sent Done Done false
23634 6865 Acked Read 4 Sent Done Done false
23635 6866 Acked Read 8 Sent Done Done false
23636 6867 Acked Read 8 Sent Done Done false
23637 6868 Acked Read 2 Sent Done Done false
23638 6869 Acked Flush 0 Sent Done Done false
23639 6870 Acked Write 8 Sent Done Done false
23640 6871 Acked Flush 0 Sent Done Done false
23641 6872 Acked Read 10 Sent Done Done false
23642 6873 Acked Read 3 Sent Done Done false
23643 6874 Acked Read 1 Sent Done Done false
23644 6875 Acked Write 2 Sent Done Done false
23645 6876 Acked Read 10 Sent Done Done false
23646 6877 Acked Read 7 Sent Done Done false
23647 6878 Acked Write 8 Sent Done Done false
23648 6879 Acked Read 4 Sent Done Done false
23649 6880 Acked Write 5 Sent Done Done false
23650 6881 Acked Write 1 Sent Done Done false
23651 6882 Acked Write 1 Sent Done Done false
23652 6883 Acked Flush 0 Sent Done Done false
23653 6884 Acked Write 7 Sent Done Done false
23654 6885 Acked Write 9 Sent Done Done false
23655 6886 Acked Read 8 Sent Done Done false
23656 6887 Acked Write 1 Sent Done Done false
23657 6888 Acked Read 7 Sent Done Done false
23658 6889 Acked Read 3 Sent Done Done false
23659 6890 Acked Write 3 Sent Done Done false
23660 6891 Acked Read 9 Sent Done Done false
23661 6892 Acked Read 4 Sent Done Done false
23662 6893 Acked Read 2 Sent Done Done false
23663 6894 Acked Write 8 Sent Done Done false
23664 6895 Acked Read 9 Sent Done Done false
23665 6896 Acked Write 6 Sent Done Done false
23666 6897 Acked Write 8 Sent Done Done false
23667 6898 Acked Read 5 Sent Done Done false
23668 6899 Acked Flush 0 Sent Done Done false
23669 6900 Acked Read 10 Sent Done Done false
23670 6901 Acked Write 1 Sent Done Done false
23671 6902 Acked Read 10 Sent Done Done false
23672 6903 Acked Write 1 Sent Done Done false
23673 6904 Acked Read 9 Sent Done Done false
23674 6905 Acked Write 1 Sent Done Done false
23675 6906 Acked Read 8 Sent Done Done false
23676 6907 Acked Read 4 Sent Done Done false
23677 6908 Acked Flush 0 Sent Done Done false
23678 6909 Acked Write 7 Sent Done Done false
23679 6910 Acked Read 5 Sent Done Done false
23680 6911 Acked Read 4 Sent Done Done false
23681 6912 Acked Write 1 Sent Done Done false
23682 6913 Acked Write 9 Sent Done Done false
23683 6914 Acked Read 10 Sent Done Done false
23684 6915 Acked Write 1 Sent Done Done false
23685 6916 Acked Read 8 Sent Done Done false
23686 6917 Acked Write 1 Sent Done Done false
23687 6918 Acked Read 8 Sent Done Done false
23688 6919 Acked Flush 0 Sent Done Done false
23689 6920 Acked Write 4 Sent Done Done false
23690 6921 Acked Write 9 Sent Done Done false
23691 6922 Acked Read 9 Sent Done Done false
23692 6923 Acked Read 7 Sent Done Done false
23693 6924 Acked Write 2 Sent Done Done false
23694 6925 Acked Read 1 Sent Done Done false
23695 6926 Acked Write 4 Sent Done Done false
23696 6927 Acked Write 2 Sent Done Done false
23697 6928 Acked Write 9 Sent Done Done false
23698 6929 Acked Write 9 Sent Done Done false
23699 6930 Acked Read 5 Sent Done Done false
23700 6931 Acked Write 6 Sent Done Done false
23701 6932 Acked Read 6 Sent Done Done false
23702 6933 Acked Read 6 Sent Done Done false
23703 6934 Acked Flush 0 Sent Done Done false
23704 6935 Acked Write 3 Sent Done Done false
23705 6936 Acked Read 2 Sent Done Done false
23706 6937 Acked Write 7 Sent Done Done false
23707 6938 Acked Read 3 Sent Done Done false
23708 6939 Acked Flush 0 Sent Done Done false
23709 6940 Acked Write 9 Sent Done Done false
23710 6941 Acked Read 9 Sent Done Done false
23711 6942 Acked Write 5 Sent Done Done false
23712 6943 Acked Read 3 Sent Done Done false
23713 6944 Acked Read 9 Sent Done Done false
23714 6945 Acked Read 1 Sent Done Done false
23715 6946 Acked Write 3 Sent Done Done false
23716 6947 Acked Flush 0 Sent Done Done false
23717 6948 Acked Write 8 Sent Done Done false
23718 6949 Acked Write 3 Sent Done Done false
23719 6950 Acked Read 4 Sent Done Done false
23720 6951 Acked Read 8 Sent Done Done false
23721 6952 Acked Write 10 Sent Done Done false
23722 6953 Acked Flush 0 Sent Done Done false
23723 6954 Acked Flush 0 Sent Done Done false
23724 6955 Acked Flush 0 Sent Done Done false
23725 6956 Acked Write 6 Sent Done Done false
23726 6957 Acked Flush 0 Sent Done Done false
23727 6958 Acked Read 1 Sent Done Done false
23728 6959 Acked Flush 0 Sent Done Done false
23729 6960 Acked Read 9 Sent Done Done false
23730 6961 Acked Write 4 Sent Done Done false
23731 6962 Acked Write 7 Sent Done Done false
23732 6963 Acked Write 7 Sent Done Done false
23733 6964 Acked Write 10 Sent Done Done false
23734 6965 Acked Write 6 Sent Done Done false
23735 6966 Acked Flush 0 Sent Done Done false
23736 6967 Acked Read 8 Sent Done Done false
23737 6968 Acked Write 8 Sent Done Done false
23738 6969 Acked Write 6 Sent Done Done false
23739 6970 Acked Read 9 Sent Done Done false
23740 6971 Acked Write 5 Sent Done Done false
23741 6972 Acked Write 6 Sent Done Done false
23742 6973 Acked Read 4 Sent Done Done false
23743 6974 Acked Read 4 Sent Done Done false
23744 6975 Acked Read 5 Sent Done Done false
23745 6976 Acked Write 1 Sent Done Done false
23746 6977 Acked Read 9 Sent Done Done false
23747 6978 Acked Read 10 Sent Done Done false
23748 6979 Acked Read 5 Sent Done Done false
23749 6980 Acked Write 3 Sent Done Done false
23750 6981 Acked Write 4 Sent Done Done false
23751 6982 Acked Write 7 Sent Done Done false
23752 6983 Acked Write 2 Sent Done Done false
23753 6984 Acked Flush 0 Sent Done Done false
23754 6985 Acked Read 5 Sent Done Done false
23755 6986 Acked Write 3 Sent Done Done false
23756 6987 Acked Read 1 Sent Done Done false
23757 6988 Acked Read 7 Sent Done Done false
23758 6989 Acked Write 9 Sent Done Done false
23759 6990 Acked Read 8 Sent Done Done false
23760 6991 Acked Flush 0 Sent Done Done false
23761 6992 Acked Read 4 Sent Done Done false
23762 6993 Acked Write 5 Sent Done Done false
23763 6994 Acked Flush 0 Sent Done Done false
23764 6995 Acked Read 8 Sent Done Done false
23765 6996 Acked Write 10 Sent Done Done false
23766 6997 Acked Flush 0 Sent Done Done false
23767 6998 Acked Write 5 Sent Done Done false
23768 6999 Acked Write 10 Sent Done Done false
23769 7000 Acked Read 7 Sent Done Done false
23770 7001 Acked Read 8 Sent Done Done false
23771 7002 Acked Write 7 Sent Done Done false
23772 7003 Acked Read 5 Sent Done Done false
23773 7004 Acked Write 4 Sent Done Done false
23774 7005 Acked Write 1 Sent Done Done false
23775 7006 Acked Write 5 Sent Done Done false
23776 7007 Acked Flush 0 Sent Done Done false
23777 7008 Acked Write 10 Sent Done Done false
23778 7009 Acked Read 5 Sent Done Done false
23779 7010 Acked Read 9 Sent Done Done false
23780 7011 Acked Write 7 Sent Done Done false
23781 7012 Acked Write 6 Sent Done Done false
23782 7013 Acked Read 1 Sent Done Done false
23783 7014 Acked Read 8 Sent Done Done false
23784 7015 Acked Flush 0 Sent Done Done false
23785 7016 Acked Write 1 Sent Done Done false
23786 7017 Acked Read 1 Sent Done Done false
23787 7018 Acked Read 10 Sent Done Done false
23788 7019 Acked Read 3 Sent Done Done false
23789 7020 Acked Read 5 Sent Done Done false
23790 7021 Acked Read 3 Sent Done Done false
23791 7022 Acked Write 5 Sent Done Done false
23792 7023 Acked Write 4 Sent Done Done false
23793 7024 Acked Read 2 Sent Done Done false
23794 7025 Acked Read 5 Sent Done Done false
23795 7026 Acked Write 2 Sent Done Done false
23796 7027 Acked Read 5 Sent Done Done false
23797 7028 Acked Write 9 Sent Done Done false
23798 7029 Acked Read 9 Sent Done Done false
23799 7030 Acked Write 1 Sent Done Done false
23800 7031 Acked Write 4 Sent Done Done false
23801 7032 Acked Read 7 Sent Done Done false
23802 7033 Acked Read 9 Sent Done Done false
23803 7034 Acked Write 2 Sent Done Done false
23804 7035 Acked Read 3 Sent Done Done false
23805 7036 Acked Read 9 Sent Done Done false
23806 7037 Acked Read 10 Sent Done Done false
23807 7038 Acked Read 9 Sent Done Done false
23808 STATES DS:0 DS:1 DS:2 TOTAL
23809 Sent 301 0 0 301
23810 Done 0 301 301 602
23811 Skipped 0 0 0 0
23812 Error 0 0 0 0
23813 Last Flush: 6737 7015 7015
23814 Downstairs last five completed: 6737 6736 6735 6734 6733
23815 Upstairs last five completed: 7038 7037 7036 7035 7034
23816 CLIENT: Up:0 ds:301 act:2
23817 [8810] got start action
23818 [8810] I am going to restart
23819 Starting downstairs at port 8810
23820 Make output file at "/tmp/dsc/downstairs-8810.txt"
23821 [8810][0] reports Starting
23822 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1220)
23823 [8810][0] reports Running
23824 ----------------------------------------------------------------
23825 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23826 States: Offline Active Active
23827 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23828 6738 Acked Write 8 Sent Done Done false
23829 6739 Acked Write 6 Sent Done Done false
23830 6740 Acked Read 8 Sent Done Done false
23831 6741 Acked Read 7 Sent Done Done false
23832 6742 Acked Write 9 Sent Done Done false
23833 6743 Acked Read 10 Sent Done Done false
23834 6744 Acked Read 9 Sent Done Done false
23835 6745 Acked Read 8 Sent Done Done false
23836 6746 Acked Read 10 Sent Done Done false
23837 6747 Acked Write 3 Sent Done Done false
23838 6748 Acked Write 10 Sent Done Done false
23839 6749 Acked Write 1 Sent Done Done false
23840 6750 Acked Read 2 Sent Done Done false
23841 6751 Acked Write 6 Sent Done Done false
23842 6752 Acked Write 6 Sent Done Done false
23843 6753 Acked Flush 0 Sent Done Done false
23844 6754 Acked Write 2 Sent Done Done false
23845 6755 Acked Write 1 Sent Done Done false
23846 6756 Acked Write 6 Sent Done Done false
23847 6757 Acked Read 6 Sent Done Done false
23848 6758 Acked Write 8 Sent Done Done false
23849 6759 Acked Write 4 Sent Done Done false
23850 6760 Acked Read 9 Sent Done Done false
23851 6761 Acked Flush 0 Sent Done Done false
23852 6762 Acked Flush 0 Sent Done Done false
23853 6763 Acked Write 8 Sent Done Done false
23854 6764 Acked Write 6 Sent Done Done false
23855 6765 Acked Write 7 Sent Done Done false
23856 6766 Acked Flush 0 Sent Done Done false
23857 6767 Acked Read 5 Sent Done Done false
23858 6768 Acked Write 4 Sent Done Done false
23859 6769 Acked Read 6 Sent Done Done false
23860 6770 Acked Write 1 Sent Done Done false
23861 6771 Acked Write 10 Sent Done Done false
23862 6772 Acked Write 2 Sent Done Done false
23863 6773 Acked Read 1 Sent Done Done false
23864 6774 Acked Write 4 Sent Done Done false
23865 6775 Acked Read 6 Sent Done Done false
23866 6776 Acked Read 5 Sent Done Done false
23867 6777 Acked Write 4 Sent Done Done false
23868 6778 Acked Write 6 Sent Done Done false
23869 6779 Acked Read 4 Sent Done Done false
23870 6780 Acked Write 7 Sent Done Done false
23871 6781 Acked Write 7 Sent Done Done false
23872 6782 Acked Write 3 Sent Done Done false
23873 6783 Acked Write 9 Sent Done Done false
23874 6784 Acked Read 8 Sent Done Done false
23875 6785 Acked Read 8 Sent Done Done false
23876 6786 Acked Write 5 Sent Done Done false
23877 6787 Acked Flush 0 Sent Done Done false
23878 6788 Acked Write 6 Sent Done Done false
23879 6789 Acked Read 4 Sent Done Done false
23880 6790 Acked Write 8 Sent Done Done false
23881 6791 Acked Read 5 Sent Done Done false
23882 6792 Acked Write 3 Sent Done Done false
23883 6793 Acked Read 5 Sent Done Done false
23884 6794 Acked Flush 0 Sent Done Done false
23885 6795 Acked Read 10 Sent Done Done false
23886 6796 Acked Flush 0 Sent Done Done false
23887 6797 Acked Read 5 Sent Done Done false
23888 6798 Acked Write 5 Sent Done Done false
23889 6799 Acked Read 8 Sent Done Done false
23890 6800 Acked Write 2 Sent Done Done false
23891 6801 Acked Write 3 Sent Done Done false
23892 6802 Acked Write 6 Sent Done Done false
23893 6803 Acked Write 8 Sent Done Done false
23894 6804 Acked Read 10 Sent Done Done false
23895 6805 Acked Write 1 Sent Done Done false
23896 6806 Acked Write 9 Sent Done Done false
23897 6807 Acked Write 9 Sent Done Done false
23898 6808 Acked Write 2 Sent Done Done false
23899 6809 Acked Write 1 Sent Done Done false
23900 6810 Acked Read 1 Sent Done Done false
23901 6811 Acked Read 8 Sent Done Done false
23902 6812 Acked Write 9 Sent Done Done false
23903 6813 Acked Write 9 Sent Done Done false
23904 6814 Acked Write 8 Sent Done Done false
23905 6815 Acked Write 10 Sent Done Done false
23906 6816 Acked Flush 0 Sent Done Done false
23907 6817 Acked Read 4 Sent Done Done false
23908 6818 Acked Read 5 Sent Done Done false
23909 6819 Acked Write 6 Sent Done Done false
23910 6820 Acked Write 2 Sent Done Done false
23911 6821 Acked Read 8 Sent Done Done false
23912 6822 Acked Flush 0 Sent Done Done false
23913 6823 Acked Write 7 Sent Done Done false
23914 6824 Acked Read 2 Sent Done Done false
23915 6825 Acked Read 6 Sent Done Done false
23916 6826 Acked Read 9 Sent Done Done false
23917 6827 Acked Write 7 Sent Done Done false
23918 6828 Acked Write 8 Sent Done Done false
23919 6829 Acked Flush 0 Sent Done Done false
23920 6830 Acked Read 1 Sent Done Done false
23921 6831 Acked Write 6 Sent Done Done false
23922 6832 Acked Flush 0 Sent Done Done false
23923 6833 Acked Read 7 Sent Done Done false
23924 6834 Acked Write 7 Sent Done Done false
23925 6835 Acked Write 3 Sent Done Done false
23926 6836 Acked Read 5 Sent Done Done false
23927 6837 Acked Write 4 Sent Done Done false
23928 6838 Acked Read 3 Sent Done Done false
23929 6839 Acked Write 4 Sent Done Done false
23930 6840 Acked Flush 0 Sent Done Done false
23931 6841 Acked Read 1 Sent Done Done false
23932 6842 Acked Write 1 Sent Done Done false
23933 6843 Acked Write 3 Sent Done Done false
23934 6844 Acked Flush 0 Sent Done Done false
23935 6845 Acked Write 6 Sent Done Done false
23936 6846 Acked Write 7 Sent Done Done false
23937 6847 Acked Read 5 Sent Done Done false
23938 6848 Acked Write 10 Sent Done Done false
23939 6849 Acked Write 6 Sent Done Done false
23940 6850 Acked Flush 0 Sent Done Done false
23941 6851 Acked Flush 0 Sent Done Done false
23942 6852 Acked Write 5 Sent Done Done false
23943 6853 Acked Read 9 Sent Done Done false
23944 6854 Acked Read 8 Sent Done Done false
23945 6855 Acked Read 1 Sent Done Done false
23946 6856 Acked Write 9 Sent Done Done false
23947 6857 Acked Read 6 Sent Done Done false
23948 6858 Acked Read 9 Sent Done Done false
23949 6859 Acked Read 1 Sent Done Done false
23950 6860 Acked Write 5 Sent Done Done false
23951 6861 Acked Write 9 Sent Done Done false
23952 6862 Acked Write 7 Sent Done Done false
23953 6863 Acked Flush 0 Sent Done Done false
23954 6864 Acked Write 7 Sent Done Done false
23955 6865 Acked Read 4 Sent Done Done false
23956 6866 Acked Read 8 Sent Done Done false
23957 6867 Acked Read 8 Sent Done Done false
23958 6868 Acked Read 2 Sent Done Done false
23959 6869 Acked Flush 0 Sent Done Done false
23960 6870 Acked Write 8 Sent Done Done false
23961 6871 Acked Flush 0 Sent Done Done false
23962 6872 Acked Read 10 Sent Done Done false
23963 6873 Acked Read 3 Sent Done Done false
23964 6874 Acked Read 1 Sent Done Done false
23965 6875 Acked Write 2 Sent Done Done false
23966 6876 Acked Read 10 Sent Done Done false
23967 6877 Acked Read 7 Sent Done Done false
23968 6878 Acked Write 8 Sent Done Done false
23969 6879 Acked Read 4 Sent Done Done false
23970 6880 Acked Write 5 Sent Done Done false
23971 6881 Acked Write 1 Sent Done Done false
23972 6882 Acked Write 1 Sent Done Done false
23973 6883 Acked Flush 0 Sent Done Done false
23974 6884 Acked Write 7 Sent Done Done false
23975 6885 Acked Write 9 Sent Done Done false
23976 6886 Acked Read 8 Sent Done Done false
23977 6887 Acked Write 1 Sent Done Done false
23978 6888 Acked Read 7 Sent Done Done false
23979 6889 Acked Read 3 Sent Done Done false
23980 6890 Acked Write 3 Sent Done Done false
23981 6891 Acked Read 9 Sent Done Done false
23982 6892 Acked Read 4 Sent Done Done false
23983 6893 Acked Read 2 Sent Done Done false
23984 6894 Acked Write 8 Sent Done Done false
23985 6895 Acked Read 9 Sent Done Done false
23986 6896 Acked Write 6 Sent Done Done false
23987 6897 Acked Write 8 Sent Done Done false
23988 6898 Acked Read 5 Sent Done Done false
23989 6899 Acked Flush 0 Sent Done Done false
23990 6900 Acked Read 10 Sent Done Done false
23991 6901 Acked Write 1 Sent Done Done false
23992 6902 Acked Read 10 Sent Done Done false
23993 6903 Acked Write 1 Sent Done Done false
23994 6904 Acked Read 9 Sent Done Done false
23995 6905 Acked Write 1 Sent Done Done false
23996 6906 Acked Read 8 Sent Done Done false
23997 6907 Acked Read 4 Sent Done Done false
23998 6908 Acked Flush 0 Sent Done Done false
23999 6909 Acked Write 7 Sent Done Done false
24000 6910 Acked Read 5 Sent Done Done false
24001 6911 Acked Read 4 Sent Done Done false
24002 6912 Acked Write 1 Sent Done Done false
24003 6913 Acked Write 9 Sent Done Done false
24004 6914 Acked Read 10 Sent Done Done false
24005 6915 Acked Write 1 Sent Done Done false
24006 6916 Acked Read 8 Sent Done Done false
24007 6917 Acked Write 1 Sent Done Done false
24008 6918 Acked Read 8 Sent Done Done false
24009 6919 Acked Flush 0 Sent Done Done false
24010 6920 Acked Write 4 Sent Done Done false
24011 6921 Acked Write 9 Sent Done Done false
24012 6922 Acked Read 9 Sent Done Done false
24013 6923 Acked Read 7 Sent Done Done false
24014 6924 Acked Write 2 Sent Done Done false
24015 6925 Acked Read 1 Sent Done Done false
24016 6926 Acked Write 4 Sent Done Done false
24017 6927 Acked Write 2 Sent Done Done false
24018 6928 Acked Write 9 Sent Done Done false
24019 6929 Acked Write 9 Sent Done Done false
24020 6930 Acked Read 5 Sent Done Done false
24021 6931 Acked Write 6 Sent Done Done false
24022 6932 Acked Read 6 Sent Done Done false
24023 6933 Acked Read 6 Sent Done Done false
24024 6934 Acked Flush 0 Sent Done Done false
24025 6935 Acked Write 3 Sent Done Done false
24026 6936 Acked Read 2 Sent Done Done false
24027 6937 Acked Write 7 Sent Done Done false
24028 6938 Acked Read 3 Sent Done Done false
24029 6939 Acked Flush 0 Sent Done Done false
24030 6940 Acked Write 9 Sent Done Done false
24031 6941 Acked Read 9 Sent Done Done false
24032 6942 Acked Write 5 Sent Done Done false
24033 6943 Acked Read 3 Sent Done Done false
24034 6944 Acked Read 9 Sent Done Done false
24035 6945 Acked Read 1 Sent Done Done false
24036 6946 Acked Write 3 Sent Done Done false
24037 6947 Acked Flush 0 Sent Done Done false
24038 6948 Acked Write 8 Sent Done Done false
24039 6949 Acked Write 3 Sent Done Done false
24040 6950 Acked Read 4 Sent Done Done false
24041 6951 Acked Read 8 Sent Done Done false
24042 6952 Acked Write 10 Sent Done Done false
24043 6953 Acked Flush 0 Sent Done Done false
24044 6954 Acked Flush 0 Sent Done Done false
24045 6955 Acked Flush 0 Sent Done Done false
24046 6956 Acked Write 6 Sent Done Done false
24047 6957 Acked Flush 0 Sent Done Done false
24048 6958 Acked Read 1 Sent Done Done false
24049 6959 Acked Flush 0 Sent Done Done false
24050 6960 Acked Read 9 Sent Done Done false
24051 6961 Acked Write 4 Sent Done Done false
24052 6962 Acked Write 7 Sent Done Done false
24053 6963 Acked Write 7 Sent Done Done false
24054 6964 Acked Write 10 Sent Done Done false
24055 6965 Acked Write 6 Sent Done Done false
24056 6966 Acked Flush 0 Sent Done Done false
24057 6967 Acked Read 8 Sent Done Done false
24058 6968 Acked Write 8 Sent Done Done false
24059 6969 Acked Write 6 Sent Done Done false
24060 6970 Acked Read 9 Sent Done Done false
24061 6971 Acked Write 5 Sent Done Done false
24062 6972 Acked Write 6 Sent Done Done false
24063 6973 Acked Read 4 Sent Done Done false
24064 6974 Acked Read 4 Sent Done Done false
24065 6975 Acked Read 5 Sent Done Done false
24066 6976 Acked Write 1 Sent Done Done false
24067 6977 Acked Read 9 Sent Done Done false
24068 6978 Acked Read 10 Sent Done Done false
24069 6979 Acked Read 5 Sent Done Done false
24070 6980 Acked Write 3 Sent Done Done false
24071 6981 Acked Write 4 Sent Done Done false
24072 6982 Acked Write 7 Sent Done Done false
24073 6983 Acked Write 2 Sent Done Done false
24074 6984 Acked Flush 0 Sent Done Done false
24075 6985 Acked Read 5 Sent Done Done false
24076 6986 Acked Write 3 Sent Done Done false
24077 6987 Acked Read 1 Sent Done Done false
24078 6988 Acked Read 7 Sent Done Done false
24079 6989 Acked Write 9 Sent Done Done false
24080 6990 Acked Read 8 Sent Done Done false
24081 6991 Acked Flush 0 Sent Done Done false
24082 6992 Acked Read 4 Sent Done Done false
24083 6993 Acked Write 5 Sent Done Done false
24084 6994 Acked Flush 0 Sent Done Done false
24085 6995 Acked Read 8 Sent Done Done false
24086 6996 Acked Write 10 Sent Done Done false
24087 6997 Acked Flush 0 Sent Done Done false
24088 6998 Acked Write 5 Sent Done Done false
24089 6999 Acked Write 10 Sent Done Done false
24090 7000 Acked Read 7 Sent Done Done false
24091 7001 Acked Read 8 Sent Done Done false
24092 7002 Acked Write 7 Sent Done Done false
24093 7003 Acked Read 5 Sent Done Done false
24094 7004 Acked Write 4 Sent Done Done false
24095 7005 Acked Write 1 Sent Done Done false
24096 7006 Acked Write 5 Sent Done Done false
24097 7007 Acked Flush 0 Sent Done Done false
24098 7008 Acked Write 10 Sent Done Done false
24099 7009 Acked Read 5 Sent Done Done false
24100 7010 Acked Read 9 Sent Done Done false
24101 7011 Acked Write 7 Sent Done Done false
24102 7012 Acked Write 6 Sent Done Done false
24103 7013 Acked Read 1 Sent Done Done false
24104 7014 Acked Read 8 Sent Done Done false
24105 7015 Acked Flush 0 Sent Done Done false
24106 7016 Acked Write 1 Sent Done Done false
24107 7017 Acked Read 1 Sent Done Done false
24108 7018 Acked Read 10 Sent Done Done false
24109 7019 Acked Read 3 Sent Done Done false
24110 7020 Acked Read 5 Sent Done Done false
24111 7021 Acked Read 3 Sent Done Done false
24112 7022 Acked Write 5 Sent Done Done false
24113 7023 Acked Write 4 Sent Done Done false
24114 7024 Acked Read 2 Sent Done Done false
24115 7025 Acked Read 5 Sent Done Done false
24116 7026 Acked Write 2 Sent Done Done false
24117 7027 Acked Read 5 Sent Done Done false
24118 7028 Acked Write 9 Sent Done Done false
24119 7029 Acked Read 9 Sent Done Done false
24120 7030 Acked Write 1 Sent Done Done false
24121 7031 Acked Write 4 Sent Done Done false
24122 7032 Acked Read 7 Sent Done Done false
24123 7033 Acked Read 9 Sent Done Done false
24124 7034 Acked Write 2 Sent Done Done false
24125 7035 Acked Read 3 Sent Done Done false
24126 7036 Acked Read 9 Sent Done Done false
24127 7037 Acked Read 10 Sent Done Done false
24128 7038 Acked Read 9 Sent Done Done false
24129 7039 Acked Flush 0 Sent Done Done false
24130 STATES DS:0 DS:1 DS:2 TOTAL
24131 Sent 302 0 0 302
24132 Done 0 302 302 604
24133 Skipped 0 0 0 0
24134 Error 0 0 0 0
24135 Last Flush: 6737 7039 7039
24136 Downstairs last five completed: 6737 6736 6735 6734 6733
24137 Upstairs last five completed: 7039 7038 7037 7036 7035
24138 CLIENT: Up:0 ds:302 act:2
241392024-12-11T23:45:33.191ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241402024-12-11T23:45:33.191ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50466) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241412024-12-11T23:45:33.192ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241422024-12-11T23:45:33.193ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241432024-12-11T23:45:33.193ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241442024-12-11T23:45:33.193ZINFOcrucible: send last flush ID to this DS: 6737 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241452024-12-11T23:45:33.193ZINFOcrucible: Replied this last flush ID: 6737 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241462024-12-11T23:45:33.193ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241472024-12-11T23:45:33.193ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241482024-12-11T23:45:33.193ZINFOcrucible: [0] client re-new 302 jobs since flush 6737 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241492024-12-11T23:45:33.193ZINFOcrucible: [0] Replayed 302 jobs since flush: 6737 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
24150 ----------------------------------------------------------------
24151 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24152 Downstairs last five completed: 7039 7038 7037 7036 7035
24153 Upstairs last five completed: 7039 7038 7037 7036 7035
24154 CLIENT: Up:0 ds:0 act:3
24155 Replay: All jobs finished
24156 Main task has work to do, go find it
24157 got dsc Stop(1)
24158 stop 1
24159 Dec 11 23:45:35.427 INFO request completed, latency_us: 352, response_code: 204[8820] Got stop action so:false kr:false
24160 , uri: /stop/cid/1, method: POST, req_id: 07d73928-4e05-4ead-bcf9-c9321fcc01e0, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
241612024-12-11T23:45:35.428ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
24162 Dec 11 23:45:35.428 INFO request completed, latency_us: 106, response_code: 200, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:45:35.428504186Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
24163 uri: /state/cid/1, method: GET, req_id: acb2b456-9a39-4c56-9243-87c2e109edef, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:45:35.428547697Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
24164 .1:9998
241652024-12-11T23:45:35.428ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241662024-12-11T23:45:35.428ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
241672024-12-11T23:45:35.428ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
24168 [8820] Exited with: ExitStatus(unix_wait_status(9))
24169 [8820][1] reports Exit
24170 Dec 11 23:45:39.430 INFO request completed, latency_us: 227, response_code: 200, uri: /state/cid/1, method: GET, req_id: c3a286d1-b090-4446-8c29-45266758d069, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
24171 001/300 Write block 1312 len 36864 data: 8 7 5 6 4 5 4 4 5
24172 002/300 Write block 4720 len 16384 data: 5 6 6 8
24173 003/300 Write block 3662 len 20480 data: 5 4 3 3 4
24174 004/300 Read block 3970 len 40960
24175 005/300 Read block 4063 len 20480
24176 006/300 Write block 1793 len 20480 data: 5 5 4 4 4
24177 007/300 Read block 4539 len 28672
24178 008/300 Write block 1855 len 4096 data: 6
24179 009/300 Read block 3264 len 40960
24180 010/300 Read block 2503 len 36864
24181 011/300 Read block 4069 len 12288
24182 012/300 Read block 3438 len 8192
24183 013/300 Write block 3970 len 8192 data: 4 3
24184 014/300 Flush
24185 015/300 Read block 1434 len 12288
24186 016/300 Read block 971 len 12288
24187 017/300 Read block 2701 len 28672
24188 018/300 Read block 1365 len 40960
24189 019/300 Write block 2242 len 36864 data: 7 7 7 6 4 3 3 2 2
24190 020/300 Write block 813 len 24576 data: 6 6 5 5 5 6
24191 021/300 Read block 2848 len 12288
24192 022/300 Read block 2652 len 36864
24193 023/300 Read block 3036 len 20480
24194 024/300 Write block 1395 len 40960 data: 4 4 4 4 4 3 3 2 3 2
24195 025/300 Read block 2079 len 32768
24196 026/300 Read block 1344 len 28672
24197 027/300 Read block 4156 len 12288
24198 028/300 Write block 4695 len 32768 data: 6 5 5 4 3 4 5 5
24199 029/300 Read block 915 len 8192
24200 030/300 Read block 2637 len 20480
24201 031/300 Write block 883 len 36864 data: 9 9 9 8 8 6 5 6 5
24202 032/300 Write block 580 len 28672 data: 6 5 7 5 7 5 5
24203 033/300 Read block 4612 len 24576
24204 034/300 Read block 4204 len 12288
24205 035/300 Read block 860 len 8192
24206 036/300 Read block 2064 len 32768
24207 037/300 Write block 3194 len 28672 data: 4 4 4 3 2 2 3
24208 038/300 Write block 4773 len 8192 data: 4 4
24209 039/300 Read block 1166 len 16384
24210 040/300 Write block 86 len 32768 data: 4 4 4 4 4 3 2 3
24211 041/300 Write block 1667 len 20480 data: 3 3 4 5 5
24212 042/300 Read block 1975 len 12288
24213 043/300 Write block 3490 len 20480 data: 6 5 4 4 4
24214 044/300 Read block 1546 len 12288
24215 045/300 Read block 4716 len 40960
24216 046/300 Read block 568 len 40960
24217 047/300 Write block 1443 len 40960 data: 7 7 7 9 8 6 5 4 5 6
24218 048/300 Read block 2961 len 20480
24219 049/300 Read block 3589 len 32768
24220 050/300 Read block 4608 len 24576
24221 051/300 Write block 444 len 16384 data: 4 5 5 5
24222 052/300 Read block 4206 len 20480
24223 053/300 Read block 2158 len 32768
24224 054/300 Flush
24225 055/300 Read block 3643 len 4096
24226 056/300 Read block 261 len 4096
24227 057/300 Read block 3792 len 40960
24228 058/300 Read block 1003 len 16384
24229 059/300 Read block 673 len 12288
24230 060/300 Read block 1878 len 4096
24231 061/300 Read block 66 len 24576
24232 062/300 Read block 884 len 24576
24233 063/300 Write block 3870 len 12288 data: 6 6 5
24234 064/300 Write block 2310 len 8192 data: 6 7
24235 065/300 Read block 3747 len 16384
24236 066/300 Write block 1137 len 12288 data: 6 7 8
24237 067/300 Read block 3661 len 16384
24238 068/300 Read block 2662 len 36864
24239 069/300 Write block 1180 len 40960 data: 6 6 7 7 6 5 4 5 5 4
24240 070/300 Write block 1307 len 40960 data: 7 7 6 5 8 9 8 6 7 5
24241 071/300 Read block 1309 len 16384
24242 072/300 Write block 4640 len 20480 data: 4 4 3 2 2
24243 073/300 Write block 1031 len 28672 data: 5 6 6 7 7 4 2
24244 074/300 Read block 55 len 36864
24245 075/300 Write block 471 len 28672 data: 2 2 3 4 4 4 5
24246 076/300 Write block 308 len 40960 data: 4 5 4 5 4 5 6 6 6 7
24247 077/300 Write block 4578 len 20480 data: 2 2 2 2 3
24248 078/300 Read block 4624 len 28672
24249 079/300 Read block 3812 len 28672
24250 080/300 Flush
24251 081/300 Write block 4766 len 20480 data: 4 4 4 4 3
24252 082/300 Write block 4599 len 40960 data: 6 6 4 5 5 5 4 4 4 4
24253 083/300 Read block 4191 len 12288
24254 084/300 Write block 2222 len 36864 data: 5 4 5 6 6 5 5 5 5
24255 085/300 Write block 4828 len 36864 data: 4 4 3 3 4 4 4 3 3
24256 086/300 Write block 1704 len 28672 data: 4 5 3 3 3 4 3
24257 087/300 Read block 2026 len 24576
24258 088/300 Read block 3853 len 28672
24259 089/300 Read block 3877 len 4096
24260 090/300 Write block 2402 len 24576 data: 2 2 3 3 3 2
24261 091/300 Read block 259 len 8192
24262 092/300 Write block 1927 len 28672 data: 5 4 5 5 5 5 3
24263 093/300 Write block 3969 len 40960 data: 3 5 4 4 4 4 7 7 6 5
24264 094/300 Read block 1388 len 28672
24265 095/300 Flush
24266 096/300 Read block 4110 len 4096
24267 097/300 Read block 3667 len 40960
24268 098/300 Read block 267 len 4096
24269 099/300 Write block 1198 len 24576 data: 5 4 4 3 3 5
24270 100/300 Write block 2707 len 32768 data: 5 6 7 10 8 9 8 9
24271 101/300 Read block 1431 len 24576
24272 102/300 Read block 4308 len 24576
24273 103/300 Read block 2709 len 4096
24274 104/300 Read block 4234 len 4096
24275 105/300 Read block 169 len 28672
24276 106/300 Write block 1204 len 28672 data: 7 7 6 5 5 5 4
24277 107/300 Flush
24278 108/300 Write block 3424 len 12288 data: 4 3 3
24279 109/300 Write block 1544 len 8192 data: 7 6
24280 110/300 Flush
24281 111/300 Read block 2648 len 36864
24282 112/300 Read block 4229 len 24576
24283 113/300 Write block 4640 len 40960 data: 5 5 4 3 3 2 2 3 3 4
24284 114/300 Write block 22 len 40960 data: 5 4 4 5 5 4 3 3 4 3
24285 115/300 Read block 2231 len 32768
24286 116/300 Write block 4482 len 28672 data: 4 4 5 5 5 3 4
24287 117/300 Read block 3555 len 8192
24288 118/300 Read block 3830 len 36864
24289 119/300 Write block 1056 len 36864 data: 7 6 3 2 2 2 3 4 5
24290 120/300 Write block 2937 len 8192 data: 8 7
24291 121/300 Read block 872 len 24576
24292 122/300 Write block 3286 len 24576 data: 5 5 5 5 5 5
24293 123/300 Write block 357 len 24576 data: 9 6 6 6 5 5
24294 124/300 Read block 364 len 16384
24295 125/300 Write block 2641 len 4096 data: 3
24296 126/300 Write block 3473 len 32768 data: 3 3 4 3 3 4 4 4
24297 127/300 Write block 1510 len 8192 data: 2 3
24298 128/300 Write block 2896 len 4096 data: 7
24299 129/300 Write block 3924 len 16384 data: 4 4 4 3
24300 130/300 Write block 2887 len 12288 data: 6 7 7
24301 131/300 Write block 4456 len 20480 data: 5 5 5 4 4
24302 132/300 Read block 329 len 4096
24303 133/300 Write block 858 len 28672 data: 5 5 5 5 5 4 3
24304 134/300 Read block 1276 len 32768
24305 135/300 Read block 1611 len 40960
24306 136/300 Read block 3176 len 28672
24307 137/300 Write block 3200 len 40960 data: 4 4 5 6 6 7 6 7 7 8
24308 138/300 Write block 779 len 36864 data: 4 2 4 4 4 5 5 5 5
24309 139/300 Write block 1365 len 20480 data: 6 4 5 7 5
24310 140/300 Write block 4631 len 40960 data: 5 4 4 4 4 2 3 4 4 6
24311 141/300 Write block 723 len 16384 data: 6 5 5 5
24312 142/300 Write block 333 len 8192 data: 4 5
24313 143/300 Flush
24314 144/300 Read block 2062 len 24576
24315 145/300 Read block 4777 len 4096
24316 146/300 Read block 3733 len 20480
24317 147/300 Read block 4982 len 40960
24318 148/300 Write block 1236 len 32768 data: 5 4 4 4 4 3 4 4
24319 149/300 Write block 2096 len 12288 data: 6 6 7
24320 150/300 Write block 2150 len 32768 data: 3 3 4 5 5 6 6 6
24321 151/300 Write block 2890 len 28672 data: 7 6 6 6 7 8 8
24322 152/300 Read block 1736 len 20480
24323 153/300 Write block 1713 len 36864 data: 4 4 4 3 4 3 3 3 2
24324 154/300 Write block 1564 len 16384 data: 6 6 5 6
24325 155/300 Write block 3883 len 4096 data: 4
24326 156/300 Write block 4273 len 8192 data: 5 4
24327 157/300 Read block 2319 len 32768
24328 158/300 Write block 3949 len 12288 data: 8 7 6
24329 159/300 Write block 1836 len 12288 data: 4 4 4
24330 160/300 Flush
24331 161/300 Read block 4763 len 36864
24332 162/300 Write block 1222 len 16384 data: 2 3 3 3
24333 163/300 Write block 4941 len 8192 data: 5 5
24334 164/300 Read block 4829 len 36864
24335 165/300 Read block 975 len 40960
24336 166/300 Write block 636 len 36864 data: 4 6 7 8 8 5 5 5 5
24337 167/300 Read block 3278 len 36864
24338 168/300 Write block 4196 len 12288 data: 2 2 3
24339 169/300 Read block 456 len 20480
24340 170/300 Read block 4133 len 36864
24341 171/300 Read block 1037 len 8192
24342 172/300 Write block 2030 len 28672 data: 5 4 5 6 6 6 6
24343 173/300 Read block 2863 len 20480
24344 174/300 Read block 3234 len 40960
24345 175/300 Read block 497 len 36864
24346 176/300 Flush
24347 177/300 Write block 3882 len 40960 data: 5 5 5 6 8 7 5 6 5 5
24348 178/300 Write block 1556 len 36864 data: 3 3 2 2 3 3 4 5 7
24349 179/300 Write block 3534 len 20480 data: 3 3 2 2 2
24350 180/300 Write block 389 len 20480 data: 5 4 5 4 3
24351 181/300 Write block 3007 len 12288 data: 4 4 7
24352 182/300 Read block 1588 len 36864
24353 183/300 Write block 770 len 8192 data: 5 8
24354 184/300 Read block 2321 len 4096
24355 185/300 Read block 4949 len 28672
24356 186/300 Flush
24357 187/300 Flush
24358 188/300 Read block 2960 len 40960
24359 189/300 Write block 576 len 40960 data: 5 6 7 6 7 6 8 6 8 6
24360 190/300 Write block 3323 len 16384 data: 6 5 5 7
24361 191/300 Read block 3902 len 40960
24362 192/300 Read block 987 len 12288
24363 193/300 Flush
24364 194/300 Write block 2119 len 28672 data: 5 4 4 5 5 4 5
24365 195/300 Write block 4236 len 8192 data: 8 6
24366 196/300 Read block 3480 len 40960
24367 197/300 Write block 3103 len 32768 data: 5 6 6 5 5 5 4 3
24368 198/300 Read block 2789 len 8192
24369 199/300 Read block 1534 len 28672
24370 200/300 Read block 2294 len 16384
24371 201/300 Write block 1581 len 36864 data: 5 5 5 4 4 3 3 3 3
24372 202/300 Read block 2629 len 28672
24373 203/300 Write block 3661 len 8192 data: 5 6
24374 204/300 Write block 4561 len 12288 data: 6 5 5
24375 205/300 Flush
24376 206/300 Read block 1664 len 4096
24377 207/300 Read block 926 len 36864
24378 208/300 Flush
24379 209/300 Write block 4449 len 36864 data: 3 4 3 3 4 3 4 6 6
24380 210/300 Write block 2605 len 8192 data: 6 4
24381 211/300 Write block 5 len 12288 data: 4 3 3
24382 212/300 Write block 2130 len 8192 data: 6 6
24383 213/300 Flush
24384 214/300 Write block 3152 len 16384 data: 6 7 6 8
24385 215/300 Read block 2035 len 12288
24386 216/300 Read block 3459 len 36864
24387 217/300 Write block 1372 len 8192 data: 5 6
24388 218/300 Flush
24389 219/300 Read block 4471 len 32768
24390 220/300 Write block 1769 len 8192 data: 3 3
24391 221/300 Write block 1036 len 4096 data: 5
24392 222/300 Read block 4417 len 36864
24393 223/300 Read block 3655 len 16384
24394 224/300 Read block 4015 len 4096
24395 225/300 Flush
24396 226/300 Write block 2910 len 16384 data: 4 4 4 3
24397 227/300 Write block 4434 len 8192 data: 6 5
24398 228/300 Read block 4284 len 12288
24399 229/300 Read block 1407 len 36864
24400 230/300 Read block 4021 len 12288
24401 231/300 Flush
24402 232/300 Read block 3823 len 40960
24403 233/300 Read block 1475 len 20480
24404 234/300 Read block 1844 len 28672
24405 235/300 Read block 3104 len 12288
24406 236/300 Read block 2305 len 12288
24407 237/300 Read block 3419 len 4096
24408 238/300 Write block 3862 len 24576 data: 11 11 11 10 7 8
24409 239/300 Write block 1315 len 36864 data: 8 6 6 5 5 6 6 6 5
24410 240/300 Read block 2644 len 20480
24411 241/300 Write block 3856 len 36864 data: 7 8 8 8 8 9 12 12 12
24412 242/300 Write block 2136 len 4096 data: 3
24413 243/300 Flush
24414 244/300 Write block 4591 len 32768 data: 6 5 5 6 6 6 7 7
24415 245/300 Flush
24416 246/300 Read block 4915 len 8192
24417 247/300 Flush
24418 248/300 Write block 4620 len 36864 data: 4 3 4 4 3 3 2 2 2
24419 249/300 Flush
24420 250/300 Flush
24421 251/300 Read block 1947 len 32768
24422 252/300 Write block 1543 len 40960 data: 6 8 7 7 5 4 3 2 2 3
24423 253/300 Read block 3787 len 16384
24424 254/300 Read block 881 len 32768
24425 255/300 Read block 3392 len 8192
24426 256/300 Read block 1487 len 8192
24427 257/300 Write block 3907 len 20480 data: 5 5 6 6 6
24428 258/300 Read block 3842 len 12288
24429 259/300 Read block 3908 len 16384
24430 260/300 Flush
24431 261/300 Write block 3053 len 28672 data: 5 6 6 6 6 5 4
24432 262/300 Flush
24433 263/300 Write block 3441 len 12288 data: 4 4 3
24434 264/300 Read block 4275 len 16384
24435 265/300 Write block 3894 len 40960 data: 6 6 6 7 7 6 6 6 4 4
24436 266/300 Read block 4850 len 8192
24437 267/300 Read block 817 len 16384
24438 268/300 Write block 1187 len 16384 data: 6 6 5 6
24439 269/300 Write block 4791 len 8192 data: 4 3
24440 270/300 Read block 3654 len 16384
24441 271/300 Write block 3577 len 8192 data: 6 6
24442 272/300 Read block 4139 len 8192
24443 273/300 Read block 1043 len 4096
24444 274/300 Write block 1261 len 12288 data: 5 4 5
24445 275/300 Read block 2879 len 12288
24446 276/300 Read block 2506 len 24576
24447 277/300 Read block 1603 len 32768
24448 278/300 Read block 2452 len 12288
24449 279/300 Flush
24450 280/300 Read block 2312 len 8192
24451 281/300 Write block 1404 len 32768 data: 3 3 3 3 3 4 5 5
24452 282/300 Write block 2766 len 8192 data: 5 5
24453 283/300 Write block 1558 len 4096 data: 3
24454 284/300 Read block 3281 len 32768
24455 285/300 Flush
24456 286/300 Write block 4802 len 24576 data: 6 6 5 4 4 5
24457 287/300 Read block 3546 len 28672
24458 288/300 Read block 231 len 16384
24459 289/300 Read block 2154 len 24576
24460 290/300 Write block 1550 len 16384 data: 3 3 4 3
24461 291/300 Read block 154 len 24576
24462 292/300 Write block 1515 len 28672 data: 2 3 5 7 7 6 6
24463 293/300 Write block 445 len 12288 data: 6 6 6
24464 294/300 Read block 287 len 8192
24465 295/300 Flush
24466 296/300 Write block 3450 len 12288 data: 5 4 4
24467 297/300 Write block 3047 len 24576 data: 6 4 4 4 4 6
24468 298/300 Read block 116 len 24576
24469 299/300 Write block 767 len 12288 data: 5 4 3
24470 300/300 Write block 3188 len 4096 data: 2
24471 301/300 Write block 3449 len 24576 data: 4 6 5 5 2 2
24472 Main task has work to do, go find it
24473 got dsc Start(1)
24474 start 1
24475 Dec 11 23:45:39.668 INFO request completed, latency_us: 269, response_code: 204, uri: /start/cid/1, method: POST, req_id: 084e169e-3843-433c-bf59-3b605ec51536, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
24476 Replay: started 1, returned:Ok(())
24477 ----------------------------------------------------------------
24478 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24479 States: Active Offline Active
24480 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24481 7040 Acked Write 9 Done Sent Done false
24482 7041 Acked Write 4 Done Sent Done false
24483 7042 Acked Write 5 Done Sent Done false
24484 7043 Acked Read 10 Done Sent Done false
24485 7044 Acked Read 5 Done Sent Done false
24486 7045 Acked Write 5 Done Sent Done false
24487 7046 Acked Read 7 Done Sent Done false
24488 7047 Acked Write 1 Done Sent Done false
24489 7048 Acked Read 10 Done Sent Done false
24490 7049 Acked Read 9 Done Sent Done false
24491 7050 Acked Read 3 Done Sent Done false
24492 7051 Acked Read 2 Done Sent Done false
24493 7052 Acked Write 2 Done Sent Done false
24494 7053 Acked Flush 0 Done Sent Done false
24495 7054 Acked Read 3 Done Sent Done false
24496 7055 Acked Read 3 Done Sent Done false
24497 7056 Acked Read 7 Done Sent Done false
24498 7057 Acked Read 10 Done Sent Done false
24499 7058 Acked Write 9 Done Sent Done false
24500 7059 Acked Write 6 Done Sent Done false
24501 7060 Acked Read 3 Done Sent Done false
24502 7061 Acked Read 9 Done Sent Done false
24503 7062 Acked Read 5 Done Sent Done false
24504 7063 Acked Write 10 Done Sent Done false
24505 7064 Acked Read 8 Done Sent Done false
24506 7065 Acked Read 7 Done Sent Done false
24507 7066 Acked Read 3 Done Sent Done false
24508 7067 Acked Write 8 Done Sent Done false
24509 7068 Acked Read 2 Done Sent Done false
24510 7069 Acked Read 5 Done Sent Done false
24511 7070 Acked Write 9 Done Sent Done false
24512 7071 Acked Write 7 Done Sent Done false
24513 7072 Acked Read 6 Done Sent Done false
24514 7073 Acked Read 3 Done Sent Done false
24515 7074 Acked Read 2 Done Sent Done false
24516 7075 Acked Read 8 Done Sent Done false
24517 7076 Acked Write 7 Done Sent Done false
24518 7077 Acked Write 2 Done Sent Done false
24519 7078 Acked Read 4 Done Sent Done false
24520 7079 Acked Write 8 Done Sent Done false
24521 7080 Acked Write 5 Done Sent Done false
24522 7081 Acked Read 3 Done Sent Done false
24523 7082 Acked Write 5 Done Sent Done false
24524 7083 Acked Read 3 Done Sent Done false
24525 7084 Acked Read 10 Done Sent Done false
24526 7085 Acked Read 10 Done Sent Done false
24527 7086 Acked Write 10 Done Sent Done false
24528 7087 Acked Read 5 Done Sent Done false
24529 7088 Acked Read 8 Done Sent Done false
24530 7089 Acked Read 6 Done Sent Done false
24531 7090 Acked Write 4 Done Sent Done false
24532 7091 Acked Read 5 Done Sent Done false
24533 7092 Acked Read 8 Done Sent Done false
24534 7093 Acked Flush 0 Done Sent Done false
24535 7094 Acked Read 1 Done Sent Done false
24536 7095 Acked Read 1 Done Sent Done false
24537 7096 Acked Read 10 Done Sent Done false
24538 7097 Acked Read 4 Done Sent Done false
24539 7098 Acked Read 3 Done Sent Done false
24540 7099 Acked Read 1 Done Sent Done false
24541 7100 Acked Read 6 Done Sent Done false
24542 7101 Acked Read 6 Done Sent Done false
24543 7102 Acked Write 3 Done Sent Done false
24544 7103 Acked Write 2 Done Sent Done false
24545 7104 Acked Read 4 Done Sent Done false
24546 7105 Acked Write 3 Done Sent Done false
24547 7106 Acked Read 4 Done Sent Done false
24548 7107 Acked Read 9 Done Sent Done false
24549 7108 Acked Write 10 Done Sent Done false
24550 7109 Acked Write 10 Done Sent Done false
24551 7110 Acked Read 4 Done Sent Done false
24552 7111 Acked Write 5 Done Sent Done false
24553 7112 Acked Write 7 Done Sent Done false
24554 7113 Acked Read 9 Done Sent Done false
24555 7114 Acked Write 7 Done Sent Done false
24556 7115 Acked Write 10 Done Sent Done false
24557 7116 Acked Write 5 Done Sent Done false
24558 7117 Acked Read 7 Done Sent Done false
24559 7118 Acked Read 7 Done Sent Done false
24560 7119 Acked Flush 0 Done Sent Done false
24561 7120 Acked Write 5 Done Sent Done false
24562 7121 Acked Write 10 Done Sent Done false
24563 7122 Acked Read 3 Done Sent Done false
24564 7123 Acked Write 9 Done Sent Done false
24565 7124 Acked Write 9 Done Sent Done false
24566 7125 Acked Write 7 Done Sent Done false
24567 7126 Acked Read 6 Done Sent Done false
24568 7127 Acked Read 7 Done Sent Done false
24569 7128 Acked Read 1 Done Sent Done false
24570 7129 Acked Write 6 Done Sent Done false
24571 7130 Acked Read 2 Done Sent Done false
24572 7131 Acked Write 7 Done Sent Done false
24573 7132 Acked Write 10 Done Sent Done false
24574 7133 Acked Read 7 Done Sent Done false
24575 7134 Acked Flush 0 Done Sent Done false
24576 7135 Acked Read 1 Done Sent Done false
24577 7136 Acked Read 10 Done Sent Done false
24578 7137 Acked Read 1 Done Sent Done false
24579 7138 Acked Write 6 Done Sent Done false
24580 7139 Acked Write 8 Done Sent Done false
24581 7140 Acked Read 6 Done Sent Done false
24582 7141 Acked Read 6 Done Sent Done false
24583 7142 Acked Read 1 Done Sent Done false
24584 7143 Acked Read 1 Done Sent Done false
24585 7144 Acked Read 7 Done Sent Done false
24586 7145 Acked Write 7 Done Sent Done false
24587 7146 Acked Flush 0 Done Sent Done false
24588 7147 Acked Write 3 Done Sent Done false
24589 7148 Acked Write 2 Done Sent Done false
24590 7149 Acked Flush 0 Done Sent Done false
24591 7150 Acked Read 9 Done Sent Done false
24592 7151 Acked Read 6 Done Sent Done false
24593 7152 Acked Write 10 Done Sent Done false
24594 7153 Acked Write 10 Done Sent Done false
24595 7154 Acked Read 8 Done Sent Done false
24596 7155 Acked Write 7 Done Sent Done false
24597 7156 Acked Read 2 Done Sent Done false
24598 7157 Acked Read 9 Done Sent Done false
24599 7158 Acked Write 9 Done Sent Done false
24600 7159 Acked Write 2 Done Sent Done false
24601 7160 Acked Read 6 Done Sent Done false
24602 7161 Acked Write 6 Done Sent Done false
24603 7162 Acked Write 6 Done Sent Done false
24604 7163 Acked Read 4 Done Sent Done false
24605 7164 Acked Write 1 Done Sent Done false
24606 7165 Acked Write 8 Done Sent Done false
24607 7166 Acked Write 2 Done Sent Done false
24608 7167 Acked Write 1 Done Sent Done false
24609 7168 Acked Write 4 Done Sent Done false
24610 7169 Acked Write 3 Done Sent Done false
24611 7170 Acked Write 5 Done Sent Done false
24612 7171 Acked Read 1 Done Sent Done false
24613 7172 Acked Write 7 Done Sent Done false
24614 7173 Acked Read 8 Done Sent Done false
24615 7174 Acked Read 10 Done Sent Done false
24616 7175 Acked Read 7 Done Sent Done false
24617 7176 Acked Write 10 Done Sent Done false
24618 7177 Acked Write 9 Done Sent Done false
24619 7178 Acked Write 5 Done Sent Done false
24620 7179 Acked Write 10 Done Sent Done false
24621 7180 Acked Write 4 Done Sent Done false
24622 7181 Acked Write 2 Done Sent Done false
24623 7182 Acked Flush 0 Done Sent Done false
24624 7183 Acked Read 6 Done Sent Done false
24625 7184 Acked Read 1 Done Sent Done false
24626 7185 Acked Read 5 Done Sent Done false
24627 7186 Acked Read 10 Done Sent Done false
24628 7187 Acked Write 8 Done Sent Done false
24629 7188 Acked Write 3 Done Sent Done false
24630 7189 Acked Write 8 Done Sent Done false
24631 7190 Acked Write 7 Done Sent Done false
24632 7191 Acked Read 5 Done Sent Done false
24633 7192 Acked Write 9 Done Sent Done false
24634 7193 Acked Write 4 Done Sent Done false
24635 7194 Acked Write 1 Done Sent Done false
24636 7195 Acked Write 2 Done Sent Done false
24637 7196 Acked Read 8 Done Sent Done false
24638 7197 Acked Write 3 Done Sent Done false
24639 7198 Acked Write 3 Done Sent Done false
24640 7199 Acked Flush 0 Done Sent Done false
24641 7200 Acked Read 9 Done Sent Done false
24642 7201 Acked Write 4 Done Sent Done false
24643 7202 Acked Write 2 Done Sent Done false
24644 7203 Acked Read 9 Done Sent Done false
24645 7204 Acked Read 10 Done Sent Done false
24646 7205 Acked Write 9 Done Sent Done false
24647 7206 Acked Read 9 Done Sent Done false
24648 7207 Acked Write 3 Done Sent Done false
24649 7208 Acked Read 5 Done Sent Done false
24650 7209 Acked Read 9 Done Sent Done false
24651 7210 Acked Read 2 Done Sent Done false
24652 7211 Acked Write 7 Done Sent Done false
24653 7212 Acked Read 5 Done Sent Done false
24654 7213 Acked Read 10 Done Sent Done false
24655 7214 Acked Read 9 Done Sent Done false
24656 7215 Acked Flush 0 Done Sent Done false
24657 7216 Acked Write 10 Done Sent Done false
24658 7217 Acked Write 9 Done Sent Done false
24659 7218 Acked Write 5 Done Sent Done false
24660 7219 Acked Write 5 Done Sent Done false
24661 7220 Acked Write 3 Done Sent Done false
24662 7221 Acked Read 9 Done Sent Done false
24663 7222 Acked Write 2 Done Sent Done false
24664 7223 Acked Read 1 Done Sent Done false
24665 7224 Acked Read 7 Done Sent Done false
24666 7225 Acked Flush 0 Done Sent Done false
24667 7226 Acked Flush 0 Done Sent Done false
24668 7227 Acked Read 10 Done Sent Done false
24669 7228 Acked Write 10 Done Sent Done false
24670 7229 Acked Write 4 Done Sent Done false
24671 7230 Acked Read 10 Done Sent Done false
24672 7231 Acked Read 3 Done Sent Done false
24673 7232 Acked Flush 0 Done Sent Done false
24674 7233 Acked Write 7 Done Sent Done false
24675 7234 Acked Write 2 Done Sent Done false
24676 7235 Acked Read 10 Done Sent Done false
24677 7236 Acked Write 8 Done Sent Done false
24678 7237 Acked Read 2 Done Sent Done false
24679 7238 Acked Read 7 Done Sent Done false
24680 7239 Acked Read 4 Done Sent Done false
24681 7240 Acked Write 9 Done Sent Done false
24682 7241 Acked Read 7 Done Sent Done false
24683 7242 Acked Write 2 Done Sent Done false
24684 7243 Acked Write 3 Done Sent Done false
24685 7244 Acked Flush 0 Done Sent Done false
24686 7245 Acked Read 1 Done Sent Done false
24687 7246 Acked Read 9 Done Sent Done false
24688 7247 Acked Flush 0 Done Sent Done false
24689 7248 Acked Write 9 Done Sent Done false
24690 7249 Acked Write 2 Done Sent Done false
24691 7250 Acked Write 3 Done Sent Done false
24692 7251 Acked Write 2 Done Sent Done false
24693 7252 Acked Flush 0 Done Sent Done false
24694 7253 Acked Write 4 Done Sent Done false
24695 7254 Acked Read 3 Done Sent Done false
24696 7255 Acked Read 9 Done Sent Done false
24697 7256 Acked Write 2 Done Sent Done false
24698 7257 Acked Flush 0 Done Sent Done false
24699 7258 Acked Read 8 Done Sent Done false
24700 7259 Acked Write 2 Done Sent Done false
24701 7260 Acked Write 1 Done Sent Done false
24702 7261 Acked Read 9 Done Sent Done false
24703 7262 Acked Read 4 Done Sent Done false
24704 7263 Acked Read 1 Done Sent Done false
24705 7264 Acked Flush 0 Done Sent Done false
24706 7265 Acked Write 4 Done Sent Done false
24707 7266 Acked Write 2 Done Sent Done false
24708 7267 Acked Read 3 Done Sent Done false
24709 7268 Acked Read 9 Done Sent Done false
24710 7269 Acked Read 3 Done Sent Done false
24711 7270 Acked Flush 0 Done Sent Done false
24712 7271 Acked Read 10 Done Sent Done false
24713 7272 Acked Read 5 Done Sent Done false
24714 7273 Acked Read 7 Done Sent Done false
24715 7274 Acked Read 3 Done Sent Done false
24716 7275 Acked Read 3 Done Sent Done false
24717 7276 Acked Read 1 Done Sent Done false
24718 7277 Acked Write 6 Done Sent Done false
24719 7278 Acked Write 9 Done Sent Done false
24720 7279 Acked Read 5 Done Sent Done false
24721 7280 Acked Write 9 Done Sent Done false
24722 7281 Acked Write 1 Done Sent Done false
24723 7282 Acked Flush 0 Done Sent Done false
24724 7283 Acked Write 8 Done Sent Done false
24725 7284 Acked Flush 0 Done Sent Done false
24726 7285 Acked Read 2 Done Sent Done false
24727 7286 Acked Flush 0 Done Sent Done false
24728 7287 Acked Write 9 Done Sent Done false
24729 7288 Acked Flush 0 Done Sent Done false
24730 7289 Acked Flush 0 Done Sent Done false
24731 7290 Acked Read 8 Done Sent Done false
24732 7291 Acked Write 10 Done Sent Done false
24733 7292 Acked Read 4 Done Sent Done false
24734 7293 Acked Read 8 Done Sent Done false
24735 7294 Acked Read 2 Done Sent Done false
24736 7295 Acked Read 2 Done Sent Done false
24737 7296 Acked Write 5 Done Sent Done false
24738 7297 Acked Read 3 Done Sent Done false
24739 7298 Acked Read 4 Done Sent Done false
24740 7299 Acked Flush 0 Done Sent Done false
24741 7300 Acked Write 7 Done Sent Done false
24742 7301 Acked Flush 0 Done Sent Done false
24743 7302 Acked Write 3 Done Sent Done false
24744 7303 Acked Read 4 Done Sent Done false
24745 7304 Acked Write 10 Done Sent Done false
24746 7305 Acked Read 2 Done Sent Done false
24747 7306 Acked Read 4 Done Sent Done false
24748 7307 Acked Write 4 Done Sent Done false
24749 7308 Acked Write 2 Done Sent Done false
24750 7309 Acked Read 4 Done Sent Done false
24751 7310 Acked Write 2 Done Sent Done false
24752 7311 Acked Read 2 Done Sent Done false
24753 7312 Acked Read 1 Done Sent Done false
24754 7313 Acked Write 3 Done Sent Done false
24755 7314 Acked Read 3 Done Sent Done false
24756 7315 Acked Read 6 Done Sent Done false
24757 7316 Acked Read 8 Done Sent Done false
24758 7317 Acked Read 3 Done Sent Done false
24759 7318 Acked Flush 0 Done Sent Done false
24760 7319 Acked Read 2 Done Sent Done false
24761 7320 Acked Write 8 Done Sent Done false
24762 7321 Acked Write 2 Done Sent Done false
24763 7322 Acked Write 1 Done Sent Done false
24764 7323 Acked Read 8 Done Sent Done false
24765 7324 Acked Flush 0 Done Sent Done false
24766 7325 Acked Write 6 Done Sent Done false
24767 7326 Acked Read 7 Done Sent Done false
24768 7327 Acked Read 4 Done Sent Done false
24769 7328 Acked Read 6 Done Sent Done false
24770 7329 Acked Write 4 Done Sent Done false
24771 7330 Acked Read 6 Done Sent Done false
24772 7331 Acked Write 7 Done Sent Done false
24773 7332 Acked Write 3 Done Sent Done false
24774 7333 Acked Read 2 Done Sent Done false
24775 7334 Acked Flush 0 Done Sent Done false
24776 7335 Acked Write 3 Done Sent Done false
24777 7336 Acked Write 6 Done Sent Done false
24778 7337 Acked Read 6 Done Sent Done false
24779 7338 Acked Write 3 Done Sent Done false
24780 7339 Acked Write 1 Done Sent Done false
24781 7340 Acked Write 6 Done Sent Done false
24782 STATES DS:0 DS:1 DS:2 TOTAL
24783 Sent 0 301 0 301
24784 Done 301 0 301 602
24785 Skipped 0 0 0 0
24786 Error 0 0 0 0
24787 Last Flush: 7334 7039 7334
24788 Downstairs last five completed: 7039 7038 7037 7036 7035
24789 Upstairs last five completed: 7340 7339 7338 7337 7336
24790 CLIENT: Up:0 ds:301 act:2
24791 [8820] got start action
24792 [8820] I am going to restart
24793 Starting downstairs at port 8820
24794 Make output file at "/tmp/dsc/downstairs-8820.txt"
24795 [8820][1] reports Starting
24796 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1221)
24797 [8820][1] reports Running
24798 ----------------------------------------------------------------
24799 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24800 States: Active Offline Active
24801 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24802 7040 Acked Write 9 Done Sent Done false
24803 7041 Acked Write 4 Done Sent Done false
24804 7042 Acked Write 5 Done Sent Done false
24805 7043 Acked Read 10 Done Sent Done false
24806 7044 Acked Read 5 Done Sent Done false
24807 7045 Acked Write 5 Done Sent Done false
24808 7046 Acked Read 7 Done Sent Done false
24809 7047 Acked Write 1 Done Sent Done false
24810 7048 Acked Read 10 Done Sent Done false
24811 7049 Acked Read 9 Done Sent Done false
24812 7050 Acked Read 3 Done Sent Done false
24813 7051 Acked Read 2 Done Sent Done false
24814 7052 Acked Write 2 Done Sent Done false
24815 7053 Acked Flush 0 Done Sent Done false
24816 7054 Acked Read 3 Done Sent Done false
24817 7055 Acked Read 3 Done Sent Done false
24818 7056 Acked Read 7 Done Sent Done false
24819 7057 Acked Read 10 Done Sent Done false
24820 7058 Acked Write 9 Done Sent Done false
24821 7059 Acked Write 6 Done Sent Done false
24822 7060 Acked Read 3 Done Sent Done false
24823 7061 Acked Read 9 Done Sent Done false
24824 7062 Acked Read 5 Done Sent Done false
24825 7063 Acked Write 10 Done Sent Done false
24826 7064 Acked Read 8 Done Sent Done false
24827 7065 Acked Read 7 Done Sent Done false
24828 7066 Acked Read 3 Done Sent Done false
24829 7067 Acked Write 8 Done Sent Done false
24830 7068 Acked Read 2 Done Sent Done false
24831 7069 Acked Read 5 Done Sent Done false
24832 7070 Acked Write 9 Done Sent Done false
24833 7071 Acked Write 7 Done Sent Done false
24834 7072 Acked Read 6 Done Sent Done false
24835 7073 Acked Read 3 Done Sent Done false
24836 7074 Acked Read 2 Done Sent Done false
24837 7075 Acked Read 8 Done Sent Done false
24838 7076 Acked Write 7 Done Sent Done false
24839 7077 Acked Write 2 Done Sent Done false
24840 7078 Acked Read 4 Done Sent Done false
24841 7079 Acked Write 8 Done Sent Done false
24842 7080 Acked Write 5 Done Sent Done false
24843 7081 Acked Read 3 Done Sent Done false
24844 7082 Acked Write 5 Done Sent Done false
24845 7083 Acked Read 3 Done Sent Done false
24846 7084 Acked Read 10 Done Sent Done false
24847 7085 Acked Read 10 Done Sent Done false
24848 7086 Acked Write 10 Done Sent Done false
24849 7087 Acked Read 5 Done Sent Done false
24850 7088 Acked Read 8 Done Sent Done false
24851 7089 Acked Read 6 Done Sent Done false
24852 7090 Acked Write 4 Done Sent Done false
24853 7091 Acked Read 5 Done Sent Done false
24854 7092 Acked Read 8 Done Sent Done false
24855 7093 Acked Flush 0 Done Sent Done false
24856 7094 Acked Read 1 Done Sent Done false
24857 7095 Acked Read 1 Done Sent Done false
24858 7096 Acked Read 10 Done Sent Done false
24859 7097 Acked Read 4 Done Sent Done false
24860 7098 Acked Read 3 Done Sent Done false
24861 7099 Acked Read 1 Done Sent Done false
24862 7100 Acked Read 6 Done Sent Done false
24863 7101 Acked Read 6 Done Sent Done false
24864 7102 Acked Write 3 Done Sent Done false
24865 7103 Acked Write 2 Done Sent Done false
24866 7104 Acked Read 4 Done Sent Done false
24867 7105 Acked Write 3 Done Sent Done false
24868 7106 Acked Read 4 Done Sent Done false
24869 7107 Acked Read 9 Done Sent Done false
24870 7108 Acked Write 10 Done Sent Done false
24871 7109 Acked Write 10 Done Sent Done false
24872 7110 Acked Read 4 Done Sent Done false
24873 7111 Acked Write 5 Done Sent Done false
24874 7112 Acked Write 7 Done Sent Done false
24875 7113 Acked Read 9 Done Sent Done false
24876 7114 Acked Write 7 Done Sent Done false
24877 7115 Acked Write 10 Done Sent Done false
24878 7116 Acked Write 5 Done Sent Done false
24879 7117 Acked Read 7 Done Sent Done false
24880 7118 Acked Read 7 Done Sent Done false
24881 7119 Acked Flush 0 Done Sent Done false
24882 7120 Acked Write 5 Done Sent Done false
24883 7121 Acked Write 10 Done Sent Done false
24884 7122 Acked Read 3 Done Sent Done false
24885 7123 Acked Write 9 Done Sent Done false
24886 7124 Acked Write 9 Done Sent Done false
24887 7125 Acked Write 7 Done Sent Done false
24888 7126 Acked Read 6 Done Sent Done false
24889 7127 Acked Read 7 Done Sent Done false
24890 7128 Acked Read 1 Done Sent Done false
24891 7129 Acked Write 6 Done Sent Done false
24892 7130 Acked Read 2 Done Sent Done false
24893 7131 Acked Write 7 Done Sent Done false
24894 7132 Acked Write 10 Done Sent Done false
24895 7133 Acked Read 7 Done Sent Done false
24896 7134 Acked Flush 0 Done Sent Done false
24897 7135 Acked Read 1 Done Sent Done false
24898 7136 Acked Read 10 Done Sent Done false
24899 7137 Acked Read 1 Done Sent Done false
24900 7138 Acked Write 6 Done Sent Done false
24901 7139 Acked Write 8 Done Sent Done false
24902 7140 Acked Read 6 Done Sent Done false
24903 7141 Acked Read 6 Done Sent Done false
24904 7142 Acked Read 1 Done Sent Done false
24905 7143 Acked Read 1 Done Sent Done false
24906 7144 Acked Read 7 Done Sent Done false
24907 7145 Acked Write 7 Done Sent Done false
24908 7146 Acked Flush 0 Done Sent Done false
24909 7147 Acked Write 3 Done Sent Done false
24910 7148 Acked Write 2 Done Sent Done false
24911 7149 Acked Flush 0 Done Sent Done false
24912 7150 Acked Read 9 Done Sent Done false
24913 7151 Acked Read 6 Done Sent Done false
24914 7152 Acked Write 10 Done Sent Done false
24915 7153 Acked Write 10 Done Sent Done false
24916 7154 Acked Read 8 Done Sent Done false
24917 7155 Acked Write 7 Done Sent Done false
24918 7156 Acked Read 2 Done Sent Done false
24919 7157 Acked Read 9 Done Sent Done false
24920 7158 Acked Write 9 Done Sent Done false
24921 7159 Acked Write 2 Done Sent Done false
24922 7160 Acked Read 6 Done Sent Done false
24923 7161 Acked Write 6 Done Sent Done false
24924 7162 Acked Write 6 Done Sent Done false
24925 7163 Acked Read 4 Done Sent Done false
24926 7164 Acked Write 1 Done Sent Done false
24927 7165 Acked Write 8 Done Sent Done false
24928 7166 Acked Write 2 Done Sent Done false
24929 7167 Acked Write 1 Done Sent Done false
24930 7168 Acked Write 4 Done Sent Done false
24931 7169 Acked Write 3 Done Sent Done false
24932 7170 Acked Write 5 Done Sent Done false
24933 7171 Acked Read 1 Done Sent Done false
24934 7172 Acked Write 7 Done Sent Done false
24935 7173 Acked Read 8 Done Sent Done false
24936 7174 Acked Read 10 Done Sent Done false
24937 7175 Acked Read 7 Done Sent Done false
24938 7176 Acked Write 10 Done Sent Done false
24939 7177 Acked Write 9 Done Sent Done false
24940 7178 Acked Write 5 Done Sent Done false
24941 7179 Acked Write 10 Done Sent Done false
24942 7180 Acked Write 4 Done Sent Done false
24943 7181 Acked Write 2 Done Sent Done false
24944 7182 Acked Flush 0 Done Sent Done false
24945 7183 Acked Read 6 Done Sent Done false
24946 7184 Acked Read 1 Done Sent Done false
24947 7185 Acked Read 5 Done Sent Done false
24948 7186 Acked Read 10 Done Sent Done false
24949 7187 Acked Write 8 Done Sent Done false
24950 7188 Acked Write 3 Done Sent Done false
24951 7189 Acked Write 8 Done Sent Done false
24952 7190 Acked Write 7 Done Sent Done false
24953 7191 Acked Read 5 Done Sent Done false
24954 7192 Acked Write 9 Done Sent Done false
24955 7193 Acked Write 4 Done Sent Done false
24956 7194 Acked Write 1 Done Sent Done false
24957 7195 Acked Write 2 Done Sent Done false
24958 7196 Acked Read 8 Done Sent Done false
24959 7197 Acked Write 3 Done Sent Done false
24960 7198 Acked Write 3 Done Sent Done false
24961 7199 Acked Flush 0 Done Sent Done false
24962 7200 Acked Read 9 Done Sent Done false
24963 7201 Acked Write 4 Done Sent Done false
24964 7202 Acked Write 2 Done Sent Done false
24965 7203 Acked Read 9 Done Sent Done false
24966 7204 Acked Read 10 Done Sent Done false
24967 7205 Acked Write 9 Done Sent Done false
24968 7206 Acked Read 9 Done Sent Done false
24969 7207 Acked Write 3 Done Sent Done false
24970 7208 Acked Read 5 Done Sent Done false
24971 7209 Acked Read 9 Done Sent Done false
24972 7210 Acked Read 2 Done Sent Done false
24973 7211 Acked Write 7 Done Sent Done false
24974 7212 Acked Read 5 Done Sent Done false
24975 7213 Acked Read 10 Done Sent Done false
24976 7214 Acked Read 9 Done Sent Done false
24977 7215 Acked Flush 0 Done Sent Done false
24978 7216 Acked Write 10 Done Sent Done false
24979 7217 Acked Write 9 Done Sent Done false
24980 7218 Acked Write 5 Done Sent Done false
24981 7219 Acked Write 5 Done Sent Done false
24982 7220 Acked Write 3 Done Sent Done false
24983 7221 Acked Read 9 Done Sent Done false
24984 7222 Acked Write 2 Done Sent Done false
24985 7223 Acked Read 1 Done Sent Done false
24986 7224 Acked Read 7 Done Sent Done false
24987 7225 Acked Flush 0 Done Sent Done false
24988 7226 Acked Flush 0 Done Sent Done false
24989 7227 Acked Read 10 Done Sent Done false
24990 7228 Acked Write 10 Done Sent Done false
24991 7229 Acked Write 4 Done Sent Done false
24992 7230 Acked Read 10 Done Sent Done false
24993 7231 Acked Read 3 Done Sent Done false
24994 7232 Acked Flush 0 Done Sent Done false
24995 7233 Acked Write 7 Done Sent Done false
24996 7234 Acked Write 2 Done Sent Done false
24997 7235 Acked Read 10 Done Sent Done false
24998 7236 Acked Write 8 Done Sent Done false
24999 7237 Acked Read 2 Done Sent Done false
25000 7238 Acked Read 7 Done Sent Done false
25001 7239 Acked Read 4 Done Sent Done false
25002 7240 Acked Write 9 Done Sent Done false
25003 7241 Acked Read 7 Done Sent Done false
25004 7242 Acked Write 2 Done Sent Done false
25005 7243 Acked Write 3 Done Sent Done false
25006 7244 Acked Flush 0 Done Sent Done false
25007 7245 Acked Read 1 Done Sent Done false
25008 7246 Acked Read 9 Done Sent Done false
25009 7247 Acked Flush 0 Done Sent Done false
25010 7248 Acked Write 9 Done Sent Done false
25011 7249 Acked Write 2 Done Sent Done false
25012 7250 Acked Write 3 Done Sent Done false
25013 7251 Acked Write 2 Done Sent Done false
25014 7252 Acked Flush 0 Done Sent Done false
25015 7253 Acked Write 4 Done Sent Done false
25016 7254 Acked Read 3 Done Sent Done false
25017 7255 Acked Read 9 Done Sent Done false
25018 7256 Acked Write 2 Done Sent Done false
25019 7257 Acked Flush 0 Done Sent Done false
25020 7258 Acked Read 8 Done Sent Done false
25021 7259 Acked Write 2 Done Sent Done false
25022 7260 Acked Write 1 Done Sent Done false
25023 7261 Acked Read 9 Done Sent Done false
25024 7262 Acked Read 4 Done Sent Done false
25025 7263 Acked Read 1 Done Sent Done false
25026 7264 Acked Flush 0 Done Sent Done false
25027 7265 Acked Write 4 Done Sent Done false
25028 7266 Acked Write 2 Done Sent Done false
25029 7267 Acked Read 3 Done Sent Done false
25030 7268 Acked Read 9 Done Sent Done false
25031 7269 Acked Read 3 Done Sent Done false
25032 7270 Acked Flush 0 Done Sent Done false
25033 7271 Acked Read 10 Done Sent Done false
25034 7272 Acked Read 5 Done Sent Done false
25035 7273 Acked Read 7 Done Sent Done false
25036 7274 Acked Read 3 Done Sent Done false
25037 7275 Acked Read 3 Done Sent Done false
25038 7276 Acked Read 1 Done Sent Done false
25039 7277 Acked Write 6 Done Sent Done false
25040 7278 Acked Write 9 Done Sent Done false
25041 7279 Acked Read 5 Done Sent Done false
25042 7280 Acked Write 9 Done Sent Done false
25043 7281 Acked Write 1 Done Sent Done false
25044 7282 Acked Flush 0 Done Sent Done false
25045 7283 Acked Write 8 Done Sent Done false
25046 7284 Acked Flush 0 Done Sent Done false
25047 7285 Acked Read 2 Done Sent Done false
25048 7286 Acked Flush 0 Done Sent Done false
25049 7287 Acked Write 9 Done Sent Done false
25050 7288 Acked Flush 0 Done Sent Done false
25051 7289 Acked Flush 0 Done Sent Done false
25052 7290 Acked Read 8 Done Sent Done false
25053 7291 Acked Write 10 Done Sent Done false
25054 7292 Acked Read 4 Done Sent Done false
25055 7293 Acked Read 8 Done Sent Done false
25056 7294 Acked Read 2 Done Sent Done false
25057 7295 Acked Read 2 Done Sent Done false
25058 7296 Acked Write 5 Done Sent Done false
25059 7297 Acked Read 3 Done Sent Done false
25060 7298 Acked Read 4 Done Sent Done false
25061 7299 Acked Flush 0 Done Sent Done false
25062 7300 Acked Write 7 Done Sent Done false
25063 7301 Acked Flush 0 Done Sent Done false
25064 7302 Acked Write 3 Done Sent Done false
25065 7303 Acked Read 4 Done Sent Done false
25066 7304 Acked Write 10 Done Sent Done false
25067 7305 Acked Read 2 Done Sent Done false
25068 7306 Acked Read 4 Done Sent Done false
25069 7307 Acked Write 4 Done Sent Done false
25070 7308 Acked Write 2 Done Sent Done false
25071 7309 Acked Read 4 Done Sent Done false
25072 7310 Acked Write 2 Done Sent Done false
25073 7311 Acked Read 2 Done Sent Done false
25074 7312 Acked Read 1 Done Sent Done false
25075 7313 Acked Write 3 Done Sent Done false
25076 7314 Acked Read 3 Done Sent Done false
25077 7315 Acked Read 6 Done Sent Done false
25078 7316 Acked Read 8 Done Sent Done false
25079 7317 Acked Read 3 Done Sent Done false
25080 7318 Acked Flush 0 Done Sent Done false
25081 7319 Acked Read 2 Done Sent Done false
25082 7320 Acked Write 8 Done Sent Done false
25083 7321 Acked Write 2 Done Sent Done false
25084 7322 Acked Write 1 Done Sent Done false
25085 7323 Acked Read 8 Done Sent Done false
25086 7324 Acked Flush 0 Done Sent Done false
25087 7325 Acked Write 6 Done Sent Done false
25088 7326 Acked Read 7 Done Sent Done false
25089 7327 Acked Read 4 Done Sent Done false
25090 7328 Acked Read 6 Done Sent Done false
25091 7329 Acked Write 4 Done Sent Done false
25092 7330 Acked Read 6 Done Sent Done false
25093 7331 Acked Write 7 Done Sent Done false
25094 7332 Acked Write 3 Done Sent Done false
25095 7333 Acked Read 2 Done Sent Done false
25096 7334 Acked Flush 0 Done Sent Done false
25097 7335 Acked Write 3 Done Sent Done false
25098 7336 Acked Write 6 Done Sent Done false
25099 7337 Acked Read 6 Done Sent Done false
25100 7338 Acked Write 3 Done Sent Done false
25101 7339 Acked Write 1 Done Sent Done false
25102 7340 Acked Write 6 Done Sent Done false
25103 7341 Acked Flush 0 Done Sent Done false
25104 STATES DS:0 DS:1 DS:2 TOTAL
25105 Sent 0 302 0 302
25106 Done 302 0 302 604
25107 Skipped 0 0 0 0
25108 Error 0 0 0 0
25109 Last Flush: 7341 7039 7341
25110 Downstairs last five completed: 7039 7038 7037 7036 7035
25111 Upstairs last five completed: 7341 7340 7339 7338 7337
25112 CLIENT: Up:0 ds:302 act:2
251132024-12-11T23:45:45.430ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251142024-12-11T23:45:45.430ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49244) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251152024-12-11T23:45:45.431ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251162024-12-11T23:45:45.431ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251172024-12-11T23:45:45.431ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251182024-12-11T23:45:45.431ZINFOcrucible: send last flush ID to this DS: 7039 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251192024-12-11T23:45:45.431ZINFOcrucible: Replied this last flush ID: 7039 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251202024-12-11T23:45:45.431ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251212024-12-11T23:45:45.431ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251222024-12-11T23:45:45.431ZINFOcrucible: [1] client re-new 302 jobs since flush 7039 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251232024-12-11T23:45:45.432ZINFOcrucible: [1] Replayed 302 jobs since flush: 7039 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
25124 ----------------------------------------------------------------
25125 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25126 Downstairs last five completed: 7341 7340 7339 7338 7337
25127 Upstairs last five completed: 7341 7340 7339 7338 7337
25128 CLIENT: Up:0 ds:0 act:3
25129 Replay: All jobs finished
25130 Main task has work to do, go find it
25131 got dsc Stop(2)
25132 stop 2
25133 Dec 11 23:45:47.672 INFO [8830] Got stop action so:false kr:false
25134 request completed, latency_us: 325, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 699e5cb4-9e9e-47b9-916f-6abfd85cfed8, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
251352024-12-11T23:45:47.673ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
25136 Dec 11 23:45:47.673 INFO request completed, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:45:47.673611037Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
25137 latency_us: 126, response_code: 200, uri: /state/cid/2, method: GET, req_id: 1732d3e8-180f-4788-b7e4-b0f8229dc624, remote_addr{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:45:47.673642878Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
25138 : 127.0.0.1:62969, local_addr: 127.0.0{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:45:47.67369302Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"2","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
25139 .1:9998
251402024-12-11T23:45:47.673ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
251412024-12-11T23:45:47.673ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
25142 [8830] Exited with: ExitStatus(unix_wait_status(9))
25143 [8830][2] reports Exit
25144 Dec 11 23:45:51.675 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6b04b9c1-0a75-478a-ab46-57b92696daab, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
25145 001/300 Write block 144 len 4096 data: 5
25146 002/300 Write block 2500 len 36864 data: 4 5 4 4 4 4 3 3 4
25147 003/300 Write block 3517 len 4096 data: 4
25148 004/300 Read block 3158 len 8192
25149 005/300 Write block 4891 len 12288 data: 5 6 7
25150 006/300 Read block 2297 len 16384
25151 007/300 Write block 2729 len 40960 data: 4 4 5 6 8 7 7 7 8 8
25152 008/300 Flush
25153 009/300 Read block 2056 len 24576
25154 010/300 Read block 2264 len 4096
25155 011/300 Read block 4515 len 4096
25156 012/300 Read block 2635 len 8192
25157 013/300 Write block 545 len 24576 data: 4 3 4 4 4 4
25158 014/300 Read block 3612 len 16384
25159 015/300 Write block 3741 len 12288 data: 5 6 5
25160 016/300 Flush
25161 017/300 Read block 2991 len 12288
25162 018/300 Flush
25163 019/300 Read block 4272 len 24576
25164 020/300 Write block 1378 len 36864 data: 2 3 4 4 4 4 4 5 3
25165 021/300 Read block 3351 len 4096
25166 022/300 Read block 4063 len 32768
25167 023/300 Write block 3887 len 24576 data: 8 6 7 6 6 6
25168 024/300 Read block 4117 len 8192
25169 025/300 Write block 4764 len 40960 data: 3 4 5 5 5 5 4 4 4 5
25170 026/300 Write block 524 len 24576 data: 4 4 3 3 2 2
25171 027/300 Read block 1835 len 40960
25172 028/300 Read block 2297 len 28672
25173 029/300 Read block 530 len 32768
25174 030/300 Read block 1206 len 40960
25175 031/300 Write block 991 len 8192 data: 4 3
25176 032/300 Write block 3164 len 24576 data: 4 3 4 4 6 6
25177 033/300 Read block 1708 len 4096
25178 034/300 Read block 1366 len 12288
25179 035/300 Read block 2382 len 28672
25180 036/300 Read block 4527 len 28672
25181 037/300 Write block 3600 len 4096 data: 3
25182 038/300 Flush
25183 039/300 Write block 4209 len 32768 data: 6 7 6 4 4 5 4 4
25184 040/300 Read block 3606 len 36864
25185 041/300 Read block 4613 len 28672
25186 042/300 Write block 1595 len 16384 data: 5 5 5 4
25187 043/300 Read block 4319 len 40960
25188 044/300 Read block 755 len 32768
25189 045/300 Write block 1063 len 20480 data: 5 6 6 4 5
25190 046/300 Write block 4355 len 40960 data: 7 7 7 6 5 3 3 5 5 4
25191 047/300 Flush
25192 048/300 Read block 3874 len 32768
25193 049/300 Read block 3683 len 20480
25194 050/300 Read block 4276 len 8192
25195 051/300 Write block 1954 len 8192 data: 6 4
25196 052/300 Write block 2009 len 24576 data: 4 5 4 4 4 5
25197 053/300 Read block 1252 len 24576
25198 054/300 Read block 3121 len 4096
25199 055/300 Write block 1834 len 40960 data: 4 4 5 5 5 4 4 3 5 4
25200 056/300 Read block 3573 len 40960
25201 057/300 Read block 1419 len 28672
25202 058/300 Write block 2714 len 36864 data: 10 8 11 8 9 10 9 9 8
25203 059/300 Read block 1606 len 28672
25204 060/300 Write block 1652 len 16384 data: 4 4 3 2
25205 061/300 Write block 2301 len 32768 data: 4 4 4 4 4 4 4 4
25206 062/300 Read block 4621 len 20480
25207 063/300 Write block 3042 len 32768 data: 4 3 4 6 7 7 5 5
25208 064/300 Write block 3070 len 12288 data: 5 5 3
25209 065/300 Read block 4711 len 28672
25210 066/300 Read block 348 len 4096
25211 067/300 Read block 3024 len 32768
25212 068/300 Write block 1751 len 40960 data: 4 4 5 5 4 4 5 5 5 4
25213 069/300 Write block 3863 len 12288 data: 13 13 11
25214 070/300 Write block 3926 len 20480 data: 5 4 3 2 3
25215 071/300 Write block 4348 len 28672 data: 7 5 4 3 4 7 6
25216 072/300 Flush
25217 073/300 Write block 3142 len 40960 data: 5 5 5 5 5 5 5 5 6 7
25218 074/300 Read block 3388 len 36864
25219 075/300 Write block 4209 len 28672 data: 7 8 7 5 5 6 5
25220 076/300 Write block 2435 len 20480 data: 3 4 5 4 7
25221 077/300 Read block 1456 len 24576
25222 078/300 Read block 3632 len 40960
25223 079/300 Flush
25224 080/300 Write block 4254 len 8192 data: 7 7
25225 081/300 Read block 4819 len 40960
25226 082/300 Write block 2375 len 40960 data: 4 4 4 4 6 4 5 6 5 4
25227 083/300 Write block 3144 len 4096 data: 6
25228 084/300 Read block 3939 len 40960
25229 085/300 Write block 4857 len 24576 data: 4 4 4 3 3 4
25230 086/300 Write block 3936 len 12288 data: 6 6 5
25231 087/300 Read block 2821 len 24576
25232 088/300 Read block 3488 len 12288
25233 089/300 Write block 2618 len 20480 data: 7 5 5 5 4
25234 090/300 Write block 3408 len 12288 data: 2 2 3
25235 091/300 Read block 2004 len 40960
25236 092/300 Read block 2205 len 12288
25237 093/300 Write block 4693 len 12288 data: 6 6 7
25238 094/300 Read block 1662 len 20480
25239 095/300 Flush
25240 096/300 Read block 3762 len 28672
25241 097/300 Write block 965 len 24576 data: 6 5 4 6 6 5
25242 098/300 Write block 3406 len 24576 data: 3 2 3 3 4 3
25243 099/300 Flush
25244 100/300 Read block 3462 len 20480
25245 101/300 Write block 4590 len 12288 data: 5 7 6
25246 102/300 Read block 4593 len 4096
25247 103/300 Read block 1140 len 8192
25248 104/300 Write block 990 len 8192 data: 5 5
25249 105/300 Write block 1103 len 20480 data: 4 4 4 3 3
25250 106/300 Write block 1026 len 12288 data: 6 6 5
25251 107/300 Write block 1926 len 28672 data: 6 6 5 6 6 6 6
25252 108/300 Write block 4558 len 4096 data: 5
25253 109/300 Write block 3632 len 28672 data: 5 5 4 3 3 2 4
25254 110/300 Write block 4096 len 32768 data: 4 5 5 5 4 5 5 7
25255 111/300 Flush
25256 112/300 Read block 1144 len 20480
25257 113/300 Write block 1824 len 32768 data: 3 3 2 2 2 2 2 3
25258 114/300 Write block 3587 len 8192 data: 7 6
25259 115/300 Read block 2626 len 32768
25260 116/300 Read block 1634 len 32768
25261 117/300 Write block 3822 len 16384 data: 3 3 5 6
25262 118/300 Write block 3657 len 24576 data: 6 6 3 5 6 7
25263 119/300 Write block 3258 len 24576 data: 3 4 4 4 5 7
25264 120/300 Write block 1865 len 24576 data: 4 3 2 2 2 2
25265 121/300 Read block 4445 len 40960
25266 122/300 Write block 4286 len 24576 data: 5 4 6 6 5 5
25267 123/300 Read block 856 len 20480
25268 124/300 Write block 4557 len 8192 data: 5 6
25269 125/300 Read block 3765 len 16384
25270 126/300 Read block 3196 len 20480
25271 127/300 Write block 4760 len 24576 data: 5 4 3 2 4 5
25272 128/300 Write block 1231 len 24576 data: 4 3 4 5 3 6
25273 129/300 Write block 3723 len 20480 data: 4 4 5 6 7
25274 130/300 Read block 2578 len 40960
25275 131/300 Write block 2728 len 36864 data: 5 5 5 6 7 9 8 8 8
25276 132/300 Read block 3464 len 28672
25277 133/300 Write block 4577 len 32768 data: 3 3 3 3 3 4 6 5
25278 134/300 Read block 3194 len 12288
25279 135/300 Read block 4742 len 36864
25280 136/300 Flush
25281 137/300 Flush
25282 138/300 Write block 4794 len 8192 data: 4 3
25283 139/300 Write block 2733 len 32768 data: 10 9 9 9 9 9 7 7
25284 140/300 Write block 2863 len 16384 data: 6 7 7 6
25285 141/300 Write block 2346 len 20480 data: 7 7 6 6 6
25286 142/300 Read block 3064 len 28672
25287 143/300 Write block 2335 len 4096 data: 7
25288 144/300 Read block 973 len 28672
25289 145/300 Write block 4024 len 16384 data: 4 4 3 2
25290 146/300 Flush
25291 147/300 Flush
25292 148/300 Read block 1131 len 16384
25293 149/300 Read block 2960 len 24576
25294 150/300 Write block 3487 len 40960 data: 7 7 7 7 6 5 5 5 4 4
25295 151/300 Write block 2706 len 24576 data: 7 6 7 8 11 9
25296 152/300 Read block 280 len 36864
25297 153/300 Flush
25298 154/300 Read block 692 len 24576
25299 155/300 Write block 2648 len 36864 data: 6 6 6 4 5 4 4 5 5
25300 156/300 Flush
25301 157/300 Read block 983 len 16384
25302 158/300 Read block 1201 len 32768
25303 159/300 Read block 2052 len 40960
25304 160/300 Write block 2820 len 24576 data: 3 5 5 7 7 5
25305 161/300 Write block 3597 len 36864 data: 3 3 2 4 3 3 3 3 2
25306 162/300 Read block 965 len 20480
25307 163/300 Read block 4333 len 24576
25308 164/300 Write block 3866 len 28672 data: 8 9 8 7 7 7 6
25309 165/300 Flush
25310 166/300 Write block 4264 len 32768 data: 5 4 4 5 5 5 4 5
25311 167/300 Read block 1944 len 36864
25312 168/300 Write block 932 len 4096 data: 4
25313 169/300 Write block 2755 len 8192 data: 3 3
25314 170/300 Write block 2735 len 36864 data: 10 10 10 10 8 8 7 7 5
25315 171/300 Write block 2451 len 40960 data: 4 3 3 3 3 3 3 4 4 4
25316 172/300 Read block 3799 len 4096
25317 173/300 Write block 874 len 4096 data: 7
25318 174/300 Read block 3302 len 20480
25319 175/300 Write block 3301 len 16384 data: 5 5 3 3
25320 176/300 Read block 4718 len 28672
25321 177/300 Read block 591 len 20480
25322 178/300 Read block 379 len 24576
25323 179/300 Flush
25324 180/300 Read block 1466 len 32768
25325 181/300 Read block 2217 len 28672
25326 182/300 Read block 1778 len 16384
25327 183/300 Read block 470 len 16384
25328 184/300 Write block 4090 len 40960 data: 4 5 4 4 4 4 5 6 6 6
25329 185/300 Write block 4899 len 40960 data: 6 6 6 9 8 9 10 8 6 5
25330 186/300 Write block 1041 len 36864 data: 5 4 5 6 6 6 6 4 4
25331 187/300 Read block 643 len 36864
25332 188/300 Read block 377 len 24576
25333 189/300 Read block 21 len 4096
25334 190/300 Read block 4480 len 36864
25335 191/300 Write block 2387 len 32768 data: 4 4 3 2 2 2 2 2
25336 192/300 Write block 3705 len 32768 data: 4 4 4 3 3 3 3 3
25337 193/300 Write block 4790 len 12288 data: 3 5 4
25338 194/300 Read block 2408 len 24576
25339 195/300 Write block 3432 len 40960 data: 5 5 5 6 4 4 5 6 5 5
25340 196/300 Read block 4423 len 36864
25341 197/300 Write block 4143 len 24576 data: 6 6 5 5 6 6
25342 198/300 Read block 1641 len 20480
25343 199/300 Flush
25344 200/300 Flush
25345 201/300 Read block 3161 len 8192
25346 202/300 Read block 76 len 4096
25347 203/300 Read block 4250 len 8192
25348 204/300 Write block 342 len 24576 data: 6 7 8 8 8 7
25349 205/300 Write block 1659 len 20480 data: 2 4 5 5 4
25350 206/300 Read block 514 len 36864
25351 207/300 Write block 3510 len 36864 data: 9 9 7 6 7 7 4 5 4
25352 208/300 Write block 45 len 12288 data: 6 6 5
25353 209/300 Read block 3317 len 16384
25354 210/300 Write block 1963 len 20480 data: 4 4 4 3 3
25355 211/300 Read block 334 len 36864
25356 212/300 Write block 621 len 40960 data: 4 3 3 3 3 2 2 2 3 3
25357 213/300 Read block 1716 len 8192
25358 214/300 Read block 1033 len 40960
25359 215/300 Read block 2655 len 36864
25360 216/300 Read block 2509 len 12288
25361 217/300 Write block 4598 len 28672 data: 8 7 7 5 6 6 6
25362 218/300 Read block 3895 len 4096
25363 219/300 Write block 575 len 32768 data: 4 6 7 8 7 8 7 9
25364 220/300 Write block 1168 len 24576 data: 5 5 4 4 3 3
25365 221/300 Read block 476 len 20480
25366 222/300 Write block 3989 len 12288 data: 3 4 6
25367 223/300 Flush
25368 224/300 Write block 2815 len 20480 data: 7 5 4 4 4
25369 225/300 Read block 2954 len 36864
25370 226/300 Read block 3693 len 4096
25371 227/300 Read block 4737 len 12288
25372 228/300 Read block 3534 len 32768
25373 229/300 Read block 1766 len 40960
25374 230/300 Write block 3097 len 4096 data: 6
25375 231/300 Write block 738 len 28672 data: 3 3 3 3 3 3 3
25376 232/300 Read block 892 len 24576
25377 233/300 Write block 920 len 28672 data: 3 3 3 3 3 3 2
25378 234/300 Read block 1092 len 4096
25379 235/300 Read block 989 len 4096
25380 236/300 Read block 3128 len 4096
25381 237/300 Flush
25382 238/300 Read block 4108 len 16384
25383 239/300 Read block 1779 len 16384
25384 240/300 Read block 1819 len 36864
25385 241/300 Read block 3067 len 36864
25386 242/300 Write block 2307 len 28672 data: 5 5 6 7 8 7 6
25387 243/300 Write block 3834 len 16384 data: 6 5 4 4
25388 244/300 Read block 1395 len 20480
25389 245/300 Write block 1577 len 32768 data: 4 4 5 5 6 6 6 5
25390 246/300 Write block 4802 len 40960 data: 7 7 6 5 5 6 5 5 5 4
25391 247/300 Write block 4827 len 12288 data: 5 5 5
25392 248/300 Write block 3255 len 32768 data: 2 3 3 4 5 5 5 6
25393 249/300 Read block 1610 len 36864
25394 250/300 Read block 3366 len 32768
25395 251/300 Read block 707 len 20480
25396 252/300 Read block 4394 len 16384
25397 253/300 Read block 4686 len 20480
25398 254/300 Write block 3007 len 12288 data: 5 5 8
25399 255/300 Flush
25400 256/300 Read block 4174 len 20480
25401 257/300 Write block 1239 len 24576 data: 5 5 4 5 5 4
25402 258/300 Flush
25403 259/300 Write block 2234 len 32768 data: 5 5 5 4 3 4 4 6
25404 260/300 Write block 1756 len 20480 data: 5 6 6 6 5
25405 261/300 Read block 1472 len 12288
25406 262/300 Read block 3923 len 24576
25407 263/300 Read block 1517 len 36864
25408 264/300 Write block 3483 len 24576 data: 4 4 4 5 8 8
25409 265/300 Write block 4767 len 36864 data: 6 6 6 5 5 5 6 5 5
25410 266/300 Flush
25411 267/300 Flush
25412 268/300 Write block 1467 len 28672 data: 5 6 6 7 5 3 3
25413 269/300 Write block 1926 len 28672 data: 7 7 6 7 7 7 7
25414 270/300 Read block 1076 len 16384
25415 271/300 Read block 4549 len 36864
25416 272/300 Flush
25417 273/300 Read block 975 len 4096
25418 274/300 Read block 415 len 12288
25419 275/300 Write block 3685 len 8192 data: 7 5
25420 276/300 Write block 930 len 32768 data: 2 3 5 4 5 5 4 6
25421 277/300 Read block 369 len 12288
25422 278/300 Read block 2151 len 20480
25423 279/300 Flush
25424 280/300 Write block 4335 len 40960 data: 2 3 4 4 4 6 7 7 6 5
25425 281/300 Read block 4404 len 4096
25426 282/300 Write block 3956 len 12288 data: 7 6 8
25427 283/300 Write block 4790 len 16384 data: 4 6 5 3
25428 284/300 Read block 4755 len 20480
25429 285/300 Read block 3303 len 8192
25430 286/300 Write block 4646 len 32768 data: 3 4 4 5 4 4 4 4
25431 287/300 Write block 573 len 32768 data: 4 4 5 7 8 9 8 9
25432 288/300 Flush
25433 289/300 Flush
25434 290/300 Read block 3289 len 4096
25435 291/300 Write block 3186 len 16384 data: 2 2 3 2
25436 292/300 Read block 4426 len 32768
25437 293/300 Write block 2609 len 12288 data: 6 9 8
25438 294/300 Read block 3612 len 8192
25439 295/300 Write block 2067 len 8192 data: 7 7
25440 296/300 Read block 4631 len 36864
25441 297/300 Flush
25442 298/300 Read block 4868 len 20480
25443 299/300 Read block 2135 len 40960
25444 300/300 Flush
25445 301/300 Write block 3233 len 40960 data: 4 4 4 4 5 4 3 4 5 6
25446 Main task has work to do, go find it
25447 got dsc Start(2)
25448 start 2
25449 Dec 11 23:45:51.910 INFO request completed, latency_us: 231, response_code: 204, uri: /start/cid/2, method: POST, req_id: 73bba0cd-7830-4b26-96f6-1a92dac1b1ad, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
25450 Replay: started 2, returned:Ok(())
25451 ----------------------------------------------------------------
25452 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25453 States: Active Active Offline
25454 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25455 7342 Acked Write 1 Done Done Sent false
25456 7343 Acked Write 9 Done Done Sent false
25457 7344 Acked Write 1 Done Done Sent false
25458 7345 Acked Read 2 Done Done Sent false
25459 7346 Acked Write 3 Done Done Sent false
25460 7347 Acked Read 4 Done Done Sent false
25461 7348 Acked Write 10 Done Done Sent false
25462 7349 Acked Flush 0 Done Done Sent false
25463 7350 Acked Read 6 Done Done Sent false
25464 7351 Acked Read 1 Done Done Sent false
25465 7352 Acked Read 1 Done Done Sent false
25466 7353 Acked Read 2 Done Done Sent false
25467 7354 Acked Write 6 Done Done Sent false
25468 7355 Acked Read 4 Done Done Sent false
25469 7356 Acked Write 3 Done Done Sent false
25470 7357 Acked Flush 0 Done Done Sent false
25471 7358 Acked Read 3 Done Done Sent false
25472 7359 Acked Flush 0 Done Done Sent false
25473 7360 Acked Read 6 Done Done Sent false
25474 7361 Acked Write 9 Done Done Sent false
25475 7362 Acked Read 1 Done Done Sent false
25476 7363 Acked Read 8 Done Done Sent false
25477 7364 Acked Write 6 Done Done Sent false
25478 7365 Acked Read 2 Done Done Sent false
25479 7366 Acked Write 10 Done Done Sent false
25480 7367 Acked Write 6 Done Done Sent false
25481 7368 Acked Read 10 Done Done Sent false
25482 7369 Acked Read 7 Done Done Sent false
25483 7370 Acked Read 8 Done Done Sent false
25484 7371 Acked Read 10 Done Done Sent false
25485 7372 Acked Write 2 Done Done Sent false
25486 7373 Acked Write 6 Done Done Sent false
25487 7374 Acked Read 1 Done Done Sent false
25488 7375 Acked Read 3 Done Done Sent false
25489 7376 Acked Read 7 Done Done Sent false
25490 7377 Acked Read 7 Done Done Sent false
25491 7378 Acked Write 1 Done Done Sent false
25492 7379 Acked Flush 0 Done Done Sent false
25493 7380 Acked Write 8 Done Done Sent false
25494 7381 Acked Read 9 Done Done Sent false
25495 7382 Acked Read 7 Done Done Sent false
25496 7383 Acked Write 4 Done Done Sent false
25497 7384 Acked Read 10 Done Done Sent false
25498 7385 Acked Read 8 Done Done Sent false
25499 7386 Acked Write 5 Done Done Sent false
25500 7387 Acked Write 10 Done Done Sent false
25501 7388 Acked Flush 0 Done Done Sent false
25502 7389 Acked Read 8 Done Done Sent false
25503 7390 Acked Read 5 Done Done Sent false
25504 7391 Acked Read 2 Done Done Sent false
25505 7392 Acked Write 2 Done Done Sent false
25506 7393 Acked Write 6 Done Done Sent false
25507 7394 Acked Read 6 Done Done Sent false
25508 7395 Acked Read 1 Done Done Sent false
25509 7396 Acked Write 10 Done Done Sent false
25510 7397 Acked Read 10 Done Done Sent false
25511 7398 Acked Read 7 Done Done Sent false
25512 7399 Acked Write 9 Done Done Sent false
25513 7400 Acked Read 7 Done Done Sent false
25514 7401 Acked Write 4 Done Done Sent false
25515 7402 Acked Write 8 Done Done Sent false
25516 7403 Acked Read 5 Done Done Sent false
25517 7404 Acked Write 8 Done Done Sent false
25518 7405 Acked Write 3 Done Done Sent false
25519 7406 Acked Read 7 Done Done Sent false
25520 7407 Acked Read 1 Done Done Sent false
25521 7408 Acked Read 8 Done Done Sent false
25522 7409 Acked Write 10 Done Done Sent false
25523 7410 Acked Write 3 Done Done Sent false
25524 7411 Acked Write 5 Done Done Sent false
25525 7412 Acked Write 7 Done Done Sent false
25526 7413 Acked Flush 0 Done Done Sent false
25527 7414 Acked Write 10 Done Done Sent false
25528 7415 Acked Read 9 Done Done Sent false
25529 7416 Acked Write 7 Done Done Sent false
25530 7417 Acked Write 5 Done Done Sent false
25531 7418 Acked Read 6 Done Done Sent false
25532 7419 Acked Read 10 Done Done Sent false
25533 7420 Acked Flush 0 Done Done Sent false
25534 7421 Acked Write 2 Done Done Sent false
25535 7422 Acked Read 10 Done Done Sent false
25536 7423 Acked Write 10 Done Done Sent false
25537 7424 Acked Write 1 Done Done Sent false
25538 7425 Acked Read 10 Done Done Sent false
25539 7426 Acked Write 6 Done Done Sent false
25540 7427 Acked Write 3 Done Done Sent false
25541 7428 Acked Read 6 Done Done Sent false
25542 7429 Acked Read 3 Done Done Sent false
25543 7430 Acked Write 5 Done Done Sent false
25544 7431 Acked Write 3 Done Done Sent false
25545 7432 Acked Read 10 Done Done Sent false
25546 7433 Acked Read 3 Done Done Sent false
25547 7434 Acked Write 3 Done Done Sent false
25548 7435 Acked Read 5 Done Done Sent false
25549 7436 Acked Flush 0 Done Done Sent false
25550 7437 Acked Read 7 Done Done Sent false
25551 7438 Acked Write 6 Done Done Sent false
25552 7439 Acked Write 6 Done Done Sent false
25553 7440 Acked Flush 0 Done Done Sent false
25554 7441 Acked Read 5 Done Done Sent false
25555 7442 Acked Write 3 Done Done Sent false
25556 7443 Acked Read 1 Done Done Sent false
25557 7444 Acked Read 2 Done Done Sent false
25558 7445 Acked Write 2 Done Done Sent false
25559 7446 Acked Write 5 Done Done Sent false
25560 7447 Acked Write 3 Done Done Sent false
25561 7448 Acked Write 7 Done Done Sent false
25562 7449 Acked Write 1 Done Done Sent false
25563 7450 Acked Write 7 Done Done Sent false
25564 7451 Acked Write 8 Done Done Sent false
25565 7452 Acked Flush 0 Done Done Sent false
25566 7453 Acked Read 5 Done Done Sent false
25567 7454 Acked Write 8 Done Done Sent false
25568 7455 Acked Write 2 Done Done Sent false
25569 7456 Acked Read 8 Done Done Sent false
25570 7457 Acked Read 8 Done Done Sent false
25571 7458 Acked Write 4 Done Done Sent false
25572 7459 Acked Write 6 Done Done Sent false
25573 7460 Acked Write 6 Done Done Sent false
25574 7461 Acked Write 6 Done Done Sent false
25575 7462 Acked Read 10 Done Done Sent false
25576 7463 Acked Write 6 Done Done Sent false
25577 7464 Acked Read 5 Done Done Sent false
25578 7465 Acked Write 2 Done Done Sent false
25579 7466 Acked Read 4 Done Done Sent false
25580 7467 Acked Read 5 Done Done Sent false
25581 7468 Acked Write 6 Done Done Sent false
25582 7469 Acked Write 6 Done Done Sent false
25583 7470 Acked Write 5 Done Done Sent false
25584 7471 Acked Read 10 Done Done Sent false
25585 7472 Acked Write 9 Done Done Sent false
25586 7473 Acked Read 7 Done Done Sent false
25587 7474 Acked Write 8 Done Done Sent false
25588 7475 Acked Read 3 Done Done Sent false
25589 7476 Acked Read 9 Done Done Sent false
25590 7477 Acked Flush 0 Done Done Sent false
25591 7478 Acked Flush 0 Done Done Sent false
25592 7479 Acked Write 2 Done Done Sent false
25593 7480 Acked Write 8 Done Done Sent false
25594 7481 Acked Write 4 Done Done Sent false
25595 7482 Acked Write 5 Done Done Sent false
25596 7483 Acked Read 7 Done Done Sent false
25597 7484 Acked Write 1 Done Done Sent false
25598 7485 Acked Read 7 Done Done Sent false
25599 7486 Acked Write 4 Done Done Sent false
25600 7487 Acked Flush 0 Done Done Sent false
25601 7488 Acked Flush 0 Done Done Sent false
25602 7489 Acked Read 4 Done Done Sent false
25603 7490 Acked Read 6 Done Done Sent false
25604 7491 Acked Write 10 Done Done Sent false
25605 7492 Acked Write 6 Done Done Sent false
25606 7493 Acked Read 9 Done Done Sent false
25607 7494 Acked Flush 0 Done Done Sent false
25608 7495 Acked Read 6 Done Done Sent false
25609 7496 Acked Write 9 Done Done Sent false
25610 7497 Acked Flush 0 Done Done Sent false
25611 7498 Acked Read 4 Done Done Sent false
25612 7499 Acked Read 8 Done Done Sent false
25613 7500 Acked Read 10 Done Done Sent false
25614 7501 Acked Write 6 Done Done Sent false
25615 7502 Acked Write 9 Done Done Sent false
25616 7503 Acked Read 5 Done Done Sent false
25617 7504 Acked Read 6 Done Done Sent false
25618 7505 Acked Write 7 Done Done Sent false
25619 7506 Acked Flush 0 Done Done Sent false
25620 7507 Acked Write 8 Done Done Sent false
25621 7508 Acked Read 9 Done Done Sent false
25622 7509 Acked Write 1 Done Done Sent false
25623 7510 Acked Write 2 Done Done Sent false
25624 7511 Acked Write 9 Done Done Sent false
25625 7512 Acked Write 10 Done Done Sent false
25626 7513 Acked Read 1 Done Done Sent false
25627 7514 Acked Write 1 Done Done Sent false
25628 7515 Acked Read 5 Done Done Sent false
25629 7516 Acked Write 4 Done Done Sent false
25630 7517 Acked Read 7 Done Done Sent false
25631 7518 Acked Read 5 Done Done Sent false
25632 7519 Acked Read 6 Done Done Sent false
25633 7520 Acked Flush 0 Done Done Sent false
25634 7521 Acked Read 8 Done Done Sent false
25635 7522 Acked Read 7 Done Done Sent false
25636 7523 Acked Read 4 Done Done Sent false
25637 7524 Acked Read 4 Done Done Sent false
25638 7525 Acked Write 10 Done Done Sent false
25639 7526 Acked Write 10 Done Done Sent false
25640 7527 Acked Write 9 Done Done Sent false
25641 7528 Acked Read 9 Done Done Sent false
25642 7529 Acked Read 6 Done Done Sent false
25643 7530 Acked Read 1 Done Done Sent false
25644 7531 Acked Read 9 Done Done Sent false
25645 7532 Acked Write 8 Done Done Sent false
25646 7533 Acked Write 8 Done Done Sent false
25647 7534 Acked Write 3 Done Done Sent false
25648 7535 Acked Read 6 Done Done Sent false
25649 7536 Acked Write 10 Done Done Sent false
25650 7537 Acked Read 9 Done Done Sent false
25651 7538 Acked Write 6 Done Done Sent false
25652 7539 Acked Read 5 Done Done Sent false
25653 7540 Acked Flush 0 Done Done Sent false
25654 7541 Acked Flush 0 Done Done Sent false
25655 7542 Acked Read 2 Done Done Sent false
25656 7543 Acked Read 1 Done Done Sent false
25657 7544 Acked Read 2 Done Done Sent false
25658 7545 Acked Write 6 Done Done Sent false
25659 7546 Acked Write 5 Done Done Sent false
25660 7547 Acked Read 9 Done Done Sent false
25661 7548 Acked Write 9 Done Done Sent false
25662 7549 Acked Write 3 Done Done Sent false
25663 7550 Acked Read 4 Done Done Sent false
25664 7551 Acked Write 5 Done Done Sent false
25665 7552 Acked Read 9 Done Done Sent false
25666 7553 Acked Write 10 Done Done Sent false
25667 7554 Acked Read 2 Done Done Sent false
25668 7555 Acked Read 10 Done Done Sent false
25669 7556 Acked Read 9 Done Done Sent false
25670 7557 Acked Read 3 Done Done Sent false
25671 7558 Acked Write 7 Done Done Sent false
25672 7559 Acked Read 1 Done Done Sent false
25673 7560 Acked Write 8 Done Done Sent false
25674 7561 Acked Write 6 Done Done Sent false
25675 7562 Acked Read 5 Done Done Sent false
25676 7563 Acked Write 3 Done Done Sent false
25677 7564 Acked Flush 0 Done Done Sent false
25678 7565 Acked Write 5 Done Done Sent false
25679 7566 Acked Read 9 Done Done Sent false
25680 7567 Acked Read 1 Done Done Sent false
25681 7568 Acked Read 3 Done Done Sent false
25682 7569 Acked Read 8 Done Done Sent false
25683 7570 Acked Read 10 Done Done Sent false
25684 7571 Acked Write 1 Done Done Sent false
25685 7572 Acked Write 7 Done Done Sent false
25686 7573 Acked Read 6 Done Done Sent false
25687 7574 Acked Write 7 Done Done Sent false
25688 7575 Acked Read 1 Done Done Sent false
25689 7576 Acked Read 1 Done Done Sent false
25690 7577 Acked Read 1 Done Done Sent false
25691 7578 Acked Flush 0 Done Done Sent false
25692 7579 Acked Read 4 Done Done Sent false
25693 7580 Acked Read 4 Done Done Sent false
25694 7581 Acked Read 9 Done Done Sent false
25695 7582 Acked Read 9 Done Done Sent false
25696 7583 Acked Write 7 Done Done Sent false
25697 7584 Acked Write 4 Done Done Sent false
25698 7585 Acked Read 5 Done Done Sent false
25699 7586 Acked Write 8 Done Done Sent false
25700 7587 Acked Write 10 Done Done Sent false
25701 7588 Acked Write 3 Done Done Sent false
25702 7589 Acked Write 8 Done Done Sent false
25703 7590 Acked Read 9 Done Done Sent false
25704 7591 Acked Read 8 Done Done Sent false
25705 7592 Acked Read 5 Done Done Sent false
25706 7593 Acked Read 4 Done Done Sent false
25707 7594 Acked Read 5 Done Done Sent false
25708 7595 Acked Write 3 Done Done Sent false
25709 7596 Acked Flush 0 Done Done Sent false
25710 7597 Acked Read 5 Done Done Sent false
25711 7598 Acked Write 6 Done Done Sent false
25712 7599 Acked Flush 0 Done Done Sent false
25713 7600 Acked Write 8 Done Done Sent false
25714 7601 Acked Write 5 Done Done Sent false
25715 7602 Acked Read 3 Done Done Sent false
25716 7603 Acked Read 6 Done Done Sent false
25717 7604 Acked Read 9 Done Done Sent false
25718 7605 Acked Write 6 Done Done Sent false
25719 7606 Acked Write 9 Done Done Sent false
25720 7607 Acked Flush 0 Done Done Sent false
25721 7608 Acked Flush 0 Done Done Sent false
25722 7609 Acked Write 7 Done Done Sent false
25723 7610 Acked Write 7 Done Done Sent false
25724 7611 Acked Read 4 Done Done Sent false
25725 7612 Acked Read 9 Done Done Sent false
25726 7613 Acked Flush 0 Done Done Sent false
25727 7614 Acked Read 1 Done Done Sent false
25728 7615 Acked Read 3 Done Done Sent false
25729 7616 Acked Write 2 Done Done Sent false
25730 7617 Acked Write 8 Done Done Sent false
25731 7618 Acked Read 3 Done Done Sent false
25732 7619 Acked Read 5 Done Done Sent false
25733 7620 Acked Flush 0 Done Done Sent false
25734 7621 Acked Write 10 Done Done Sent false
25735 7622 Acked Read 1 Done Done Sent false
25736 7623 Acked Write 3 Done Done Sent false
25737 7624 Acked Write 4 Done Done Sent false
25738 7625 Acked Read 5 Done Done Sent false
25739 7626 Acked Read 2 Done Done Sent false
25740 7627 Acked Write 8 Done Done Sent false
25741 7628 Acked Write 8 Done Done Sent false
25742 7629 Acked Flush 0 Done Done Sent false
25743 7630 Acked Flush 0 Done Done Sent false
25744 7631 Acked Read 1 Done Done Sent false
25745 7632 Acked Write 4 Done Done Sent false
25746 7633 Acked Read 8 Done Done Sent false
25747 7634 Acked Write 3 Done Done Sent false
25748 7635 Acked Read 2 Done Done Sent false
25749 7636 Acked Write 2 Done Done Sent false
25750 7637 Acked Read 9 Done Done Sent false
25751 7638 Acked Flush 0 Done Done Sent false
25752 7639 Acked Read 5 Done Done Sent false
25753 7640 Acked Read 10 Done Done Sent false
25754 7641 Acked Flush 0 Done Done Sent false
25755 7642 Acked Write 10 Done Done Sent false
25756 STATES DS:0 DS:1 DS:2 TOTAL
25757 Sent 0 0 301 301
25758 Done 301 301 0 602
25759 Skipped 0 0 0 0
25760 Error 0 0 0 0
25761 Last Flush: 7641 7641 7341
25762 Downstairs last five completed: 7341 7340 7339 7338 7337
25763 Upstairs last five completed: 7642 7641 7640 7639 7638
25764 CLIENT: Up:0 ds:301 act:2
25765 ----------------------------------------------------------------
25766 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25767 States: Active Active Offline
25768 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25769 7342 Acked Write 1 Done Done Sent false
25770 7343 Acked Write 9 Done Done Sent false
25771 7344 Acked Write 1 Done Done Sent false
25772 7345 Acked Read 2 Done Done Sent false
25773 7346 Acked Write 3 Done Done Sent false
25774 7347 Acked Read 4 Done Done Sent false
25775 7348 Acked Write 10 Done Done Sent false
25776 7349 Acked Flush 0 Done Done Sent false
25777 7350 Acked Read 6 Done Done Sent false
25778 7351 Acked Read 1 Done Done Sent false
25779 7352 Acked Read 1 Done Done Sent false
25780 7353 Acked Read 2 Done Done Sent false
25781 7354 Acked Write 6 Done Done Sent false
25782 7355 Acked Read 4 Done Done Sent false
25783 7356 Acked Write 3 Done Done Sent false
25784 7357 Acked Flush 0 Done Done Sent false
25785 7358 Acked Read 3 Done Done Sent false
25786 7359 Acked Flush 0 Done Done Sent false
25787 7360 Acked Read 6 Done Done Sent false
25788 7361 Acked Write 9 Done Done Sent false
25789 7362 Acked Read 1 Done Done Sent false
25790 7363 Acked Read 8 Done Done Sent false
25791 7364 Acked Write 6 Done Done Sent false
25792 7365 Acked Read 2 Done Done Sent false
25793 7366 Acked Write 10 Done Done Sent false
25794 7367 Acked Write 6 Done Done Sent false
25795 7368 Acked Read 10 Done Done Sent false
25796 7369 Acked Read 7 Done Done Sent false
25797 7370 Acked Read 8 Done Done Sent false
25798 7371 Acked Read 10 Done Done Sent false
25799 7372 Acked Write 2 Done Done Sent false
25800 7373 Acked Write 6 Done Done Sent false
25801 7374 Acked Read 1 Done Done Sent false
25802 7375 Acked Read 3 Done Done Sent false
25803 7376 Acked Read 7 Done Done Sent false
25804 7377 Acked Read 7 Done Done Sent false
25805 7378 Acked Write 1 Done Done Sent false
25806 7379 Acked Flush 0 Done Done Sent false
25807 7380 Acked Write 8 Done Done Sent false
25808 7381 Acked Read 9 Done Done Sent false
25809 7382 Acked Read 7 Done Done Sent false
25810 7383 Acked Write 4 Done Done Sent false
25811 7384 Acked Read 10 Done Done Sent false
25812 7385 Acked Read 8 Done Done Sent false
25813 7386 Acked Write 5 Done Done Sent false
25814 7387 Acked Write 10 Done Done Sent false
25815 7388 Acked Flush 0 Done Done Sent false
25816 7389 Acked Read 8 Done Done Sent false
25817 7390 Acked Read 5 Done Done Sent false
25818 7391 Acked Read 2 Done Done Sent false
25819 7392 Acked Write 2 Done Done Sent false
25820 7393 Acked Write 6 Done Done Sent false
25821 7394 Acked Read 6 Done Done Sent false
25822 7395 Acked Read 1 Done Done Sent false
25823 7396 Acked Write 10 Done Done Sent false
25824 7397 Acked Read 10 Done Done Sent false
25825 7398 Acked Read 7 Done Done Sent false
25826 7399 Acked Write 9 Done Done Sent false
25827 7400 Acked Read 7 Done Done Sent false
25828 7401 Acked Write 4 Done Done Sent false
25829 7402 Acked Write 8 Done Done Sent false
25830 7403 Acked Read 5 Done Done Sent false
25831 7404 Acked Write 8 Done Done Sent false
25832 7405 Acked Write 3 Done Done Sent false
25833 7406 Acked Read 7 Done Done Sent false
25834 7407 Acked Read 1 Done Done Sent false
25835 7408 Acked Read 8 Done Done Sent false
25836 7409 Acked Write 10 Done Done Sent false
25837 7410 Acked Write 3 Done Done Sent false
25838 7411 Acked Write 5 Done Done Sent false
25839 7412 Acked Write 7 Done Done Sent false
25840 7413 Acked Flush 0 Done Done Sent false
25841 7414 Acked Write 10 Done Done Sent false
25842 7415 Acked Read 9 Done Done Sent false
25843 7416 Acked Write 7 Done Done Sent false
25844 7417 Acked Write 5 Done Done Sent false
25845 7418 Acked Read 6 Done Done Sent false
25846 7419 Acked Read 10 Done Done Sent false
25847 7420 Acked Flush 0 Done Done Sent false
25848 7421 Acked Write 2 Done Done Sent false
25849 7422 Acked Read 10 Done Done Sent false
25850 7423 Acked Write 10 Done Done Sent false
25851 7424 Acked Write 1 Done Done Sent false
25852 7425 Acked Read 10 Done Done Sent false
25853 7426 Acked Write 6 Done Done Sent false
25854 7427 Acked Write 3 Done Done Sent false
25855 7428 Acked Read 6 Done Done Sent false
25856 7429 Acked Read 3 Done Done Sent false
25857 7430 Acked Write 5 Done Done Sent false
25858 7431 Acked Write 3 Done Done Sent false
25859 7432 Acked Read 10 Done Done Sent false
25860 7433 Acked Read 3 Done Done Sent false
25861 7434 Acked Write 3 Done Done Sent false
25862 7435 Acked Read 5 Done Done Sent false
25863 7436 Acked Flush 0 Done Done Sent false
25864 7437 Acked Read 7 Done Done Sent false
25865 7438 Acked Write 6 Done Done Sent false
25866 7439 Acked Write 6 Done Done Sent false
25867 7440 Acked Flush 0 Done Done Sent false
25868 7441 Acked Read 5 Done Done Sent false
25869 7442 Acked Write 3 Done Done Sent false
25870 7443 Acked Read 1 Done Done Sent false
25871 7444 Acked Read 2 Done Done Sent false
25872 7445 Acked Write 2 Done Done Sent false
25873 7446 Acked Write 5 Done Done Sent false
25874 7447 Acked Write 3 Done Done Sent false
25875 7448 Acked Write 7 Done Done Sent false
25876 7449 Acked Write 1 Done Done Sent false
25877 7450 Acked Write 7 Done Done Sent false
25878 7451 Acked Write 8 Done Done Sent false
25879 7452 Acked Flush 0 Done Done Sent false
25880 7453 Acked Read 5 Done Done Sent false
25881 7454 Acked Write 8 Done Done Sent false
25882 7455 Acked Write 2 Done Done Sent false
25883 7456 Acked Read 8 Done Done Sent false
25884 7457 Acked Read 8 Done Done Sent false
25885 7458 Acked Write 4 Done Done Sent false
25886 7459 Acked Write 6 Done Done Sent false
25887 7460 Acked Write 6 Done Done Sent false
25888 7461 Acked Write 6 Done Done Sent false
25889 7462 Acked Read 10 Done Done Sent false
25890 7463 Acked Write 6 Done Done Sent false
25891 7464 Acked Read 5 Done Done Sent false
25892 7465 Acked Write 2 Done Done Sent false
25893 7466 Acked Read 4 Done Done Sent false
25894 7467 Acked Read 5 Done Done Sent false
25895 7468 Acked Write 6 Done Done Sent false
25896 7469 Acked Write 6 Done Done Sent false
25897 7470 Acked Write 5 Done Done Sent false
25898 7471 Acked Read 10 Done Done Sent false
25899 7472 Acked Write 9 Done Done Sent false
25900 7473 Acked Read 7 Done Done Sent false
25901 7474 Acked Write 8 Done Done Sent false
25902 7475 Acked Read 3 Done Done Sent false
25903 7476 Acked Read 9 Done Done Sent false
25904 7477 Acked Flush 0 Done Done Sent false
25905 7478 Acked Flush 0 Done Done Sent false
25906 7479 Acked Write 2 Done Done Sent false
25907 7480 Acked Write 8 Done Done Sent false
25908 7481 Acked Write 4 Done Done Sent false
25909 7482 Acked Write 5 Done Done Sent false
25910 7483 Acked Read 7 Done Done Sent false
25911 7484 Acked Write 1 Done Done Sent false
25912 7485 Acked Read 7 Done Done Sent false
25913 7486 Acked Write 4 Done Done Sent false
25914 7487 Acked Flush 0 Done Done Sent false
25915 7488 Acked Flush 0 Done Done Sent false
25916 7489 Acked Read 4 Done Done Sent false
25917 7490 Acked Read 6 Done Done Sent false
25918 7491 Acked Write 10 Done Done Sent false
25919 7492 Acked Write 6 Done Done Sent false
25920 7493 Acked Read 9 Done Done Sent false
25921 7494 Acked Flush 0 Done Done Sent false
25922 7495 Acked Read 6 Done Done Sent false
25923 7496 Acked Write 9 Done Done Sent false
25924 7497 Acked Flush 0 Done Done Sent false
25925 7498 Acked Read 4 Done Done Sent false
25926 7499 Acked Read 8 Done Done Sent false
25927 7500 Acked Read 10 Done Done Sent false
25928 7501 Acked Write 6 Done Done Sent false
25929 7502 Acked Write 9 Done Done Sent false
25930 7503 Acked Read 5 Done Done Sent false
25931 7504 Acked Read 6 Done Done Sent false
25932 7505 Acked Write 7 Done Done Sent false
25933 7506 Acked Flush 0 Done Done Sent false
25934 7507 Acked Write 8 Done Done Sent false
25935 7508 Acked Read 9 Done Done Sent false
25936 7509 Acked Write 1 Done Done Sent false
25937 7510 Acked Write 2 Done Done Sent false
25938 7511 Acked Write 9 Done Done Sent false
25939 7512 Acked Write 10 Done Done Sent false
25940 7513 Acked Read 1 Done Done Sent false
25941 7514 Acked Write 1 Done Done Sent false
25942 7515 Acked Read 5 Done Done Sent false
25943 7516 Acked Write 4 Done Done Sent false
25944 7517 Acked Read 7 Done Done Sent false
25945 7518 Acked Read 5 Done Done Sent false
25946 7519 Acked Read 6 Done Done Sent false
25947 7520 Acked Flush 0 Done Done Sent false
25948 7521 Acked Read 8 Done Done Sent false
25949 7522 Acked Read 7 Done Done Sent false
25950 7523 Acked Read 4 Done Done Sent false
25951 7524 Acked Read 4 Done Done Sent false
25952 7525 Acked Write 10 Done Done Sent false
25953 7526 Acked Write 10 Done Done Sent false
25954 7527 Acked Write 9 Done Done Sent false
25955 7528 Acked Read 9 Done Done Sent false
25956 7529 Acked Read 6 Done Done Sent false
25957 7530 Acked Read 1 Done Done Sent false
25958 7531 Acked Read 9 Done Done Sent false
25959 7532 Acked Write 8 Done Done Sent false
25960 7533 Acked Write 8 Done Done Sent false
25961 7534 Acked Write 3 Done Done Sent false
25962 7535 Acked Read 6 Done Done Sent false
25963 7536 Acked Write 10 Done Done Sent false
25964 7537 Acked Read 9 Done Done Sent false
25965 7538 Acked Write 6 Done Done Sent false
25966 7539 Acked Read 5 Done Done Sent false
25967 7540 Acked Flush 0 Done Done Sent false
25968 7541 Acked Flush 0 Done Done Sent false
25969 7542 Acked Read 2 Done Done Sent false
25970 7543 Acked Read 1 Done Done Sent false
25971 7544 Acked Read 2 Done Done Sent false
25972 7545 Acked Write 6 Done Done Sent false
25973 7546 Acked Write 5 Done Done Sent false
25974 7547 Acked Read 9 Done Done Sent false
25975 7548 Acked Write 9 Done Done Sent false
25976 7549 Acked Write 3 Done Done Sent false
25977 7550 Acked Read 4 Done Done Sent false
25978 7551 Acked Write 5 Done Done Sent false
25979 7552 Acked Read 9 Done Done Sent false
25980 7553 Acked Write 10 Done Done Sent false
25981 7554 Acked Read 2 Done Done Sent false
25982 7555 Acked Read 10 Done Done Sent false
25983 7556 Acked Read 9 Done Done Sent false
25984 7557 Acked Read 3 Done Done Sent false
25985 7558 Acked Write 7 Done Done Sent false
25986 7559 Acked Read 1 Done Done Sent false
25987 7560 Acked Write 8 Done Done Sent false
25988 7561 Acked Write 6 Done Done Sent false
25989 7562 Acked Read 5 Done Done Sent false
25990 7563 Acked Write 3 Done Done Sent false
25991 7564 Acked Flush 0 Done Done Sent false
25992 7565 Acked Write 5 Done Done Sent false
25993 7566 Acked Read 9 Done Done Sent false
25994 7567 Acked Read 1 Done Done Sent false
25995 7568 Acked Read 3 Done Done Sent false
25996 7569 Acked Read 8 Done Done Sent false
25997 7570 Acked Read 10 Done Done Sent false
25998 7571 Acked Write 1 Done Done Sent false
25999 7572 Acked Write 7 Done Done Sent false
26000 7573 Acked Read 6 Done Done Sent false
26001 7574 Acked Write 7 Done Done Sent false
26002 7575 Acked Read 1 Done Done Sent false
26003 7576 Acked Read 1 Done Done Sent false
26004 7577 Acked Read 1 Done Done Sent false
26005 7578 Acked Flush 0 Done Done Sent false
26006 7579 Acked Read 4 Done Done Sent false
26007 7580 Acked Read 4 Done Done Sent false
26008 7581 Acked Read 9 Done Done Sent false
26009 7582 Acked Read 9 Done Done Sent false
26010 7583 Acked Write 7 Done Done Sent false
26011 7584 Acked Write 4 Done Done Sent false
26012 7585 Acked Read 5 Done Done Sent false
26013 7586 Acked Write 8 Done Done Sent false
26014 7587 Acked Write 10 Done Done Sent false
26015 7588 Acked Write 3 Done Done Sent false
26016 7589 Acked Write 8 Done Done Sent false
26017 7590 Acked Read 9 Done Done Sent false
26018 7591 Acked Read 8 Done Done Sent false
26019 7592 Acked Read 5 Done Done Sent false
26020 7593 Acked Read 4 Done Done Sent false
26021 7594 Acked Read 5 Done Done Sent false
26022 7595 Acked Write 3 Done Done Sent false
26023 7596 Acked Flush 0 Done Done Sent false
26024 7597 Acked Read 5 Done Done Sent false
26025 7598 Acked Write 6 Done Done Sent false
26026 7599 Acked Flush 0 Done Done Sent false
26027 7600 Acked Write 8 Done Done Sent false
26028 7601 Acked Write 5 Done Done Sent false
26029 7602 Acked Read 3 Done Done Sent false
26030 7603 Acked Read 6 Done Done Sent false
26031 7604 Acked Read 9 Done Done Sent false
26032 7605 Acked Write 6 Done Done Sent false
26033 7606 Acked Write 9 Done Done Sent false
26034 7607 Acked Flush 0 Done Done Sent false
26035 7608 Acked Flush 0 Done Done Sent false
26036 7609 Acked Write 7 Done Done Sent false
26037 7610 Acked Write 7 Done Done Sent false
26038 7611 Acked Read 4 Done Done Sent false
26039 7612 Acked Read 9 Done Done Sent false
26040 7613 Acked Flush 0 Done Done Sent false
26041 7614 Acked Read 1 Done Done Sent false
26042 7615 Acked Read 3 Done Done Sent false
26043 7616 Acked Write 2 Done Done Sent false
26044 7617 Acked Write 8 Done Done Sent false
26045 7618 Acked Read 3 Done Done Sent false
26046 7619 Acked Read 5 Done Done Sent false
26047 7620 Acked Flush 0 Done Done Sent false
26048 7621 Acked Write 10 Done Done Sent false
26049 7622 Acked Read 1 Done Done Sent false
26050 7623 Acked Write 3 Done Done Sent false
26051 7624 Acked Write 4 Done Done Sent false
26052 7625 Acked Read 5 Done Done Sent false
26053 7626 Acked Read 2 Done Done Sent false
26054 7627 Acked Write 8 Done Done Sent false
26055 7628 Acked Write 8 Done Done Sent false
26056 7629 Acked Flush 0 Done Done Sent false
26057 7630 Acked Flush 0 Done Done Sent false
26058 7631 Acked Read 1 Done Done Sent false
26059 7632 Acked Write 4 Done Done Sent false
26060 7633 Acked Read 8 Done Done Sent false
26061 7634 Acked Write 3 Done Done Sent false
26062 7635 Acked Read 2 Done Done Sent false
26063 7636 Acked Write 2 Done Done Sent false
26064 7637 Acked Read 9 Done Done Sent false
26065 7638 Acked Flush 0 Done Done Sent false
26066 7639 Acked Read 5 Done Done Sent false
26067 7640 Acked Read 10 Done Done Sent false
26068 7641 Acked Flush 0 Done Done Sent false
26069 7642 Acked Write 10 Done Done Sent false
26070 7643 Acked Flush 0 Done Done Sent false
26071 STATES DS:0 DS:1 DS:2 TOTAL
26072 Sent 0 0 302 302
26073 Done 302 302 0 604
26074 Skipped 0 0 0 0
26075 Error 0 0 0 0
26076 Last Flush: 7643 7643 7341
26077 Downstairs last five completed: 7341 7340 7339 7338 7337
26078 Upstairs last five completed: 7643 7642 7641 7640 7639
26079 CLIENT: Up:0 ds:302 act:2
260802024-12-11T23:45:57.674ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
260812024-12-11T23:45:57.674ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
260822024-12-11T23:45:57.674ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
260832024-12-11T23:45:57.674ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
260842024-12-11T23:45:57.674ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
26085 [8830] got start action
26086 [8830] I am going to restart
26087 Starting downstairs at port 8830
26088 Make output file at "/tmp/dsc/downstairs-8830.txt"
26089 [8830][2] reports Starting
26090 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1222)
26091 [8830][2] reports Running
26092 ----------------------------------------------------------------
26093 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26094 States: Active Active Offline
26095 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26096 7342 Acked Write 1 Done Done Sent false
26097 7343 Acked Write 9 Done Done Sent false
26098 7344 Acked Write 1 Done Done Sent false
26099 7345 Acked Read 2 Done Done Sent false
26100 7346 Acked Write 3 Done Done Sent false
26101 7347 Acked Read 4 Done Done Sent false
26102 7348 Acked Write 10 Done Done Sent false
26103 7349 Acked Flush 0 Done Done Sent false
26104 7350 Acked Read 6 Done Done Sent false
26105 7351 Acked Read 1 Done Done Sent false
26106 7352 Acked Read 1 Done Done Sent false
26107 7353 Acked Read 2 Done Done Sent false
26108 7354 Acked Write 6 Done Done Sent false
26109 7355 Acked Read 4 Done Done Sent false
26110 7356 Acked Write 3 Done Done Sent false
26111 7357 Acked Flush 0 Done Done Sent false
26112 7358 Acked Read 3 Done Done Sent false
26113 7359 Acked Flush 0 Done Done Sent false
26114 7360 Acked Read 6 Done Done Sent false
26115 7361 Acked Write 9 Done Done Sent false
26116 7362 Acked Read 1 Done Done Sent false
26117 7363 Acked Read 8 Done Done Sent false
26118 7364 Acked Write 6 Done Done Sent false
26119 7365 Acked Read 2 Done Done Sent false
26120 7366 Acked Write 10 Done Done Sent false
26121 7367 Acked Write 6 Done Done Sent false
26122 7368 Acked Read 10 Done Done Sent false
26123 7369 Acked Read 7 Done Done Sent false
26124 7370 Acked Read 8 Done Done Sent false
26125 7371 Acked Read 10 Done Done Sent false
26126 7372 Acked Write 2 Done Done Sent false
26127 7373 Acked Write 6 Done Done Sent false
26128 7374 Acked Read 1 Done Done Sent false
26129 7375 Acked Read 3 Done Done Sent false
26130 7376 Acked Read 7 Done Done Sent false
26131 7377 Acked Read 7 Done Done Sent false
26132 7378 Acked Write 1 Done Done Sent false
26133 7379 Acked Flush 0 Done Done Sent false
26134 7380 Acked Write 8 Done Done Sent false
26135 7381 Acked Read 9 Done Done Sent false
26136 7382 Acked Read 7 Done Done Sent false
26137 7383 Acked Write 4 Done Done Sent false
26138 7384 Acked Read 10 Done Done Sent false
26139 7385 Acked Read 8 Done Done Sent false
26140 7386 Acked Write 5 Done Done Sent false
26141 7387 Acked Write 10 Done Done Sent false
26142 7388 Acked Flush 0 Done Done Sent false
26143 7389 Acked Read 8 Done Done Sent false
26144 7390 Acked Read 5 Done Done Sent false
26145 7391 Acked Read 2 Done Done Sent false
26146 7392 Acked Write 2 Done Done Sent false
26147 7393 Acked Write 6 Done Done Sent false
26148 7394 Acked Read 6 Done Done Sent false
26149 7395 Acked Read 1 Done Done Sent false
26150 7396 Acked Write 10 Done Done Sent false
26151 7397 Acked Read 10 Done Done Sent false
26152 7398 Acked Read 7 Done Done Sent false
26153 7399 Acked Write 9 Done Done Sent false
26154 7400 Acked Read 7 Done Done Sent false
26155 7401 Acked Write 4 Done Done Sent false
26156 7402 Acked Write 8 Done Done Sent false
26157 7403 Acked Read 5 Done Done Sent false
26158 7404 Acked Write 8 Done Done Sent false
26159 7405 Acked Write 3 Done Done Sent false
26160 7406 Acked Read 7 Done Done Sent false
26161 7407 Acked Read 1 Done Done Sent false
26162 7408 Acked Read 8 Done Done Sent false
26163 7409 Acked Write 10 Done Done Sent false
26164 7410 Acked Write 3 Done Done Sent false
26165 7411 Acked Write 5 Done Done Sent false
26166 7412 Acked Write 7 Done Done Sent false
26167 7413 Acked Flush 0 Done Done Sent false
26168 7414 Acked Write 10 Done Done Sent false
26169 7415 Acked Read 9 Done Done Sent false
26170 7416 Acked Write 7 Done Done Sent false
26171 7417 Acked Write 5 Done Done Sent false
26172 7418 Acked Read 6 Done Done Sent false
26173 7419 Acked Read 10 Done Done Sent false
26174 7420 Acked Flush 0 Done Done Sent false
26175 7421 Acked Write 2 Done Done Sent false
26176 7422 Acked Read 10 Done Done Sent false
26177 7423 Acked Write 10 Done Done Sent false
26178 7424 Acked Write 1 Done Done Sent false
26179 7425 Acked Read 10 Done Done Sent false
26180 7426 Acked Write 6 Done Done Sent false
26181 7427 Acked Write 3 Done Done Sent false
26182 7428 Acked Read 6 Done Done Sent false
26183 7429 Acked Read 3 Done Done Sent false
26184 7430 Acked Write 5 Done Done Sent false
26185 7431 Acked Write 3 Done Done Sent false
26186 7432 Acked Read 10 Done Done Sent false
26187 7433 Acked Read 3 Done Done Sent false
26188 7434 Acked Write 3 Done Done Sent false
26189 7435 Acked Read 5 Done Done Sent false
26190 7436 Acked Flush 0 Done Done Sent false
26191 7437 Acked Read 7 Done Done Sent false
26192 7438 Acked Write 6 Done Done Sent false
26193 7439 Acked Write 6 Done Done Sent false
26194 7440 Acked Flush 0 Done Done Sent false
26195 7441 Acked Read 5 Done Done Sent false
26196 7442 Acked Write 3 Done Done Sent false
26197 7443 Acked Read 1 Done Done Sent false
26198 7444 Acked Read 2 Done Done Sent false
26199 7445 Acked Write 2 Done Done Sent false
26200 7446 Acked Write 5 Done Done Sent false
26201 7447 Acked Write 3 Done Done Sent false
26202 7448 Acked Write 7 Done Done Sent false
26203 7449 Acked Write 1 Done Done Sent false
26204 7450 Acked Write 7 Done Done Sent false
26205 7451 Acked Write 8 Done Done Sent false
26206 7452 Acked Flush 0 Done Done Sent false
26207 7453 Acked Read 5 Done Done Sent false
26208 7454 Acked Write 8 Done Done Sent false
26209 7455 Acked Write 2 Done Done Sent false
26210 7456 Acked Read 8 Done Done Sent false
26211 7457 Acked Read 8 Done Done Sent false
26212 7458 Acked Write 4 Done Done Sent false
26213 7459 Acked Write 6 Done Done Sent false
26214 7460 Acked Write 6 Done Done Sent false
26215 7461 Acked Write 6 Done Done Sent false
26216 7462 Acked Read 10 Done Done Sent false
26217 7463 Acked Write 6 Done Done Sent false
26218 7464 Acked Read 5 Done Done Sent false
26219 7465 Acked Write 2 Done Done Sent false
26220 7466 Acked Read 4 Done Done Sent false
26221 7467 Acked Read 5 Done Done Sent false
26222 7468 Acked Write 6 Done Done Sent false
26223 7469 Acked Write 6 Done Done Sent false
26224 7470 Acked Write 5 Done Done Sent false
26225 7471 Acked Read 10 Done Done Sent false
26226 7472 Acked Write 9 Done Done Sent false
26227 7473 Acked Read 7 Done Done Sent false
26228 7474 Acked Write 8 Done Done Sent false
26229 7475 Acked Read 3 Done Done Sent false
26230 7476 Acked Read 9 Done Done Sent false
26231 7477 Acked Flush 0 Done Done Sent false
26232 7478 Acked Flush 0 Done Done Sent false
26233 7479 Acked Write 2 Done Done Sent false
26234 7480 Acked Write 8 Done Done Sent false
26235 7481 Acked Write 4 Done Done Sent false
26236 7482 Acked Write 5 Done Done Sent false
26237 7483 Acked Read 7 Done Done Sent false
26238 7484 Acked Write 1 Done Done Sent false
26239 7485 Acked Read 7 Done Done Sent false
26240 7486 Acked Write 4 Done Done Sent false
26241 7487 Acked Flush 0 Done Done Sent false
26242 7488 Acked Flush 0 Done Done Sent false
26243 7489 Acked Read 4 Done Done Sent false
26244 7490 Acked Read 6 Done Done Sent false
26245 7491 Acked Write 10 Done Done Sent false
26246 7492 Acked Write 6 Done Done Sent false
26247 7493 Acked Read 9 Done Done Sent false
26248 7494 Acked Flush 0 Done Done Sent false
26249 7495 Acked Read 6 Done Done Sent false
26250 7496 Acked Write 9 Done Done Sent false
26251 7497 Acked Flush 0 Done Done Sent false
26252 7498 Acked Read 4 Done Done Sent false
26253 7499 Acked Read 8 Done Done Sent false
26254 7500 Acked Read 10 Done Done Sent false
26255 7501 Acked Write 6 Done Done Sent false
26256 7502 Acked Write 9 Done Done Sent false
26257 7503 Acked Read 5 Done Done Sent false
26258 7504 Acked Read 6 Done Done Sent false
26259 7505 Acked Write 7 Done Done Sent false
26260 7506 Acked Flush 0 Done Done Sent false
26261 7507 Acked Write 8 Done Done Sent false
26262 7508 Acked Read 9 Done Done Sent false
26263 7509 Acked Write 1 Done Done Sent false
26264 7510 Acked Write 2 Done Done Sent false
26265 7511 Acked Write 9 Done Done Sent false
26266 7512 Acked Write 10 Done Done Sent false
26267 7513 Acked Read 1 Done Done Sent false
26268 7514 Acked Write 1 Done Done Sent false
26269 7515 Acked Read 5 Done Done Sent false
26270 7516 Acked Write 4 Done Done Sent false
26271 7517 Acked Read 7 Done Done Sent false
26272 7518 Acked Read 5 Done Done Sent false
26273 7519 Acked Read 6 Done Done Sent false
26274 7520 Acked Flush 0 Done Done Sent false
26275 7521 Acked Read 8 Done Done Sent false
26276 7522 Acked Read 7 Done Done Sent false
26277 7523 Acked Read 4 Done Done Sent false
26278 7524 Acked Read 4 Done Done Sent false
26279 7525 Acked Write 10 Done Done Sent false
26280 7526 Acked Write 10 Done Done Sent false
26281 7527 Acked Write 9 Done Done Sent false
26282 7528 Acked Read 9 Done Done Sent false
26283 7529 Acked Read 6 Done Done Sent false
26284 7530 Acked Read 1 Done Done Sent false
26285 7531 Acked Read 9 Done Done Sent false
26286 7532 Acked Write 8 Done Done Sent false
26287 7533 Acked Write 8 Done Done Sent false
26288 7534 Acked Write 3 Done Done Sent false
26289 7535 Acked Read 6 Done Done Sent false
26290 7536 Acked Write 10 Done Done Sent false
26291 7537 Acked Read 9 Done Done Sent false
26292 7538 Acked Write 6 Done Done Sent false
26293 7539 Acked Read 5 Done Done Sent false
26294 7540 Acked Flush 0 Done Done Sent false
26295 7541 Acked Flush 0 Done Done Sent false
26296 7542 Acked Read 2 Done Done Sent false
26297 7543 Acked Read 1 Done Done Sent false
26298 7544 Acked Read 2 Done Done Sent false
26299 7545 Acked Write 6 Done Done Sent false
26300 7546 Acked Write 5 Done Done Sent false
26301 7547 Acked Read 9 Done Done Sent false
26302 7548 Acked Write 9 Done Done Sent false
26303 7549 Acked Write 3 Done Done Sent false
26304 7550 Acked Read 4 Done Done Sent false
26305 7551 Acked Write 5 Done Done Sent false
26306 7552 Acked Read 9 Done Done Sent false
26307 7553 Acked Write 10 Done Done Sent false
26308 7554 Acked Read 2 Done Done Sent false
26309 7555 Acked Read 10 Done Done Sent false
26310 7556 Acked Read 9 Done Done Sent false
26311 7557 Acked Read 3 Done Done Sent false
26312 7558 Acked Write 7 Done Done Sent false
26313 7559 Acked Read 1 Done Done Sent false
26314 7560 Acked Write 8 Done Done Sent false
26315 7561 Acked Write 6 Done Done Sent false
26316 7562 Acked Read 5 Done Done Sent false
26317 7563 Acked Write 3 Done Done Sent false
26318 7564 Acked Flush 0 Done Done Sent false
26319 7565 Acked Write 5 Done Done Sent false
26320 7566 Acked Read 9 Done Done Sent false
26321 7567 Acked Read 1 Done Done Sent false
26322 7568 Acked Read 3 Done Done Sent false
26323 7569 Acked Read 8 Done Done Sent false
26324 7570 Acked Read 10 Done Done Sent false
26325 7571 Acked Write 1 Done Done Sent false
26326 7572 Acked Write 7 Done Done Sent false
26327 7573 Acked Read 6 Done Done Sent false
26328 7574 Acked Write 7 Done Done Sent false
26329 7575 Acked Read 1 Done Done Sent false
26330 7576 Acked Read 1 Done Done Sent false
26331 7577 Acked Read 1 Done Done Sent false
26332 7578 Acked Flush 0 Done Done Sent false
26333 7579 Acked Read 4 Done Done Sent false
26334 7580 Acked Read 4 Done Done Sent false
26335 7581 Acked Read 9 Done Done Sent false
26336 7582 Acked Read 9 Done Done Sent false
26337 7583 Acked Write 7 Done Done Sent false
26338 7584 Acked Write 4 Done Done Sent false
26339 7585 Acked Read 5 Done Done Sent false
26340 7586 Acked Write 8 Done Done Sent false
26341 7587 Acked Write 10 Done Done Sent false
26342 7588 Acked Write 3 Done Done Sent false
26343 7589 Acked Write 8 Done Done Sent false
26344 7590 Acked Read 9 Done Done Sent false
26345 7591 Acked Read 8 Done Done Sent false
26346 7592 Acked Read 5 Done Done Sent false
26347 7593 Acked Read 4 Done Done Sent false
26348 7594 Acked Read 5 Done Done Sent false
26349 7595 Acked Write 3 Done Done Sent false
26350 7596 Acked Flush 0 Done Done Sent false
26351 7597 Acked Read 5 Done Done Sent false
26352 7598 Acked Write 6 Done Done Sent false
26353 7599 Acked Flush 0 Done Done Sent false
26354 7600 Acked Write 8 Done Done Sent false
26355 7601 Acked Write 5 Done Done Sent false
26356 7602 Acked Read 3 Done Done Sent false
26357 7603 Acked Read 6 Done Done Sent false
26358 7604 Acked Read 9 Done Done Sent false
26359 7605 Acked Write 6 Done Done Sent false
26360 7606 Acked Write 9 Done Done Sent false
26361 7607 Acked Flush 0 Done Done Sent false
26362 7608 Acked Flush 0 Done Done Sent false
26363 7609 Acked Write 7 Done Done Sent false
26364 7610 Acked Write 7 Done Done Sent false
26365 7611 Acked Read 4 Done Done Sent false
26366 7612 Acked Read 9 Done Done Sent false
26367 7613 Acked Flush 0 Done Done Sent false
26368 7614 Acked Read 1 Done Done Sent false
26369 7615 Acked Read 3 Done Done Sent false
26370 7616 Acked Write 2 Done Done Sent false
26371 7617 Acked Write 8 Done Done Sent false
26372 7618 Acked Read 3 Done Done Sent false
26373 7619 Acked Read 5 Done Done Sent false
26374 7620 Acked Flush 0 Done Done Sent false
26375 7621 Acked Write 10 Done Done Sent false
26376 7622 Acked Read 1 Done Done Sent false
26377 7623 Acked Write 3 Done Done Sent false
26378 7624 Acked Write 4 Done Done Sent false
26379 7625 Acked Read 5 Done Done Sent false
26380 7626 Acked Read 2 Done Done Sent false
26381 7627 Acked Write 8 Done Done Sent false
26382 7628 Acked Write 8 Done Done Sent false
26383 7629 Acked Flush 0 Done Done Sent false
26384 7630 Acked Flush 0 Done Done Sent false
26385 7631 Acked Read 1 Done Done Sent false
26386 7632 Acked Write 4 Done Done Sent false
26387 7633 Acked Read 8 Done Done Sent false
26388 7634 Acked Write 3 Done Done Sent false
26389 7635 Acked Read 2 Done Done Sent false
26390 7636 Acked Write 2 Done Done Sent false
26391 7637 Acked Read 9 Done Done Sent false
26392 7638 Acked Flush 0 Done Done Sent false
26393 7639 Acked Read 5 Done Done Sent false
26394 7640 Acked Read 10 Done Done Sent false
26395 7641 Acked Flush 0 Done Done Sent false
26396 7642 Acked Write 10 Done Done Sent false
26397 7643 Acked Flush 0 Done Done Sent false
26398 STATES DS:0 DS:1 DS:2 TOTAL
26399 Sent 0 0 302 302
26400 Done 302 302 0 604
26401 Skipped 0 0 0 0
26402 Error 0 0 0 0
26403 Last Flush: 7643 7643 7341
26404 Downstairs last five completed: 7341 7340 7339 7338 7337
26405 Upstairs last five completed: 7643 7642 7641 7640 7639
26406 CLIENT: Up:0 ds:302 act:2
26407 ----------------------------------------------------------------
26408 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26409 States: Active Active Offline
26410 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26411 7342 Acked Write 1 Done Done Sent false
26412 7343 Acked Write 9 Done Done Sent false
26413 7344 Acked Write 1 Done Done Sent false
26414 7345 Acked Read 2 Done Done Sent false
26415 7346 Acked Write 3 Done Done Sent false
26416 7347 Acked Read 4 Done Done Sent false
26417 7348 Acked Write 10 Done Done Sent false
26418 7349 Acked Flush 0 Done Done Sent false
26419 7350 Acked Read 6 Done Done Sent false
26420 7351 Acked Read 1 Done Done Sent false
26421 7352 Acked Read 1 Done Done Sent false
26422 7353 Acked Read 2 Done Done Sent false
26423 7354 Acked Write 6 Done Done Sent false
26424 7355 Acked Read 4 Done Done Sent false
26425 7356 Acked Write 3 Done Done Sent false
26426 7357 Acked Flush 0 Done Done Sent false
26427 7358 Acked Read 3 Done Done Sent false
26428 7359 Acked Flush 0 Done Done Sent false
26429 7360 Acked Read 6 Done Done Sent false
26430 7361 Acked Write 9 Done Done Sent false
26431 7362 Acked Read 1 Done Done Sent false
26432 7363 Acked Read 8 Done Done Sent false
26433 7364 Acked Write 6 Done Done Sent false
26434 7365 Acked Read 2 Done Done Sent false
26435 7366 Acked Write 10 Done Done Sent false
26436 7367 Acked Write 6 Done Done Sent false
26437 7368 Acked Read 10 Done Done Sent false
26438 7369 Acked Read 7 Done Done Sent false
26439 7370 Acked Read 8 Done Done Sent false
26440 7371 Acked Read 10 Done Done Sent false
26441 7372 Acked Write 2 Done Done Sent false
26442 7373 Acked Write 6 Done Done Sent false
26443 7374 Acked Read 1 Done Done Sent false
26444 7375 Acked Read 3 Done Done Sent false
26445 7376 Acked Read 7 Done Done Sent false
26446 7377 Acked Read 7 Done Done Sent false
26447 7378 Acked Write 1 Done Done Sent false
26448 7379 Acked Flush 0 Done Done Sent false
26449 7380 Acked Write 8 Done Done Sent false
26450 7381 Acked Read 9 Done Done Sent false
26451 7382 Acked Read 7 Done Done Sent false
26452 7383 Acked Write 4 Done Done Sent false
26453 7384 Acked Read 10 Done Done Sent false
26454 7385 Acked Read 8 Done Done Sent false
26455 7386 Acked Write 5 Done Done Sent false
26456 7387 Acked Write 10 Done Done Sent false
26457 7388 Acked Flush 0 Done Done Sent false
26458 7389 Acked Read 8 Done Done Sent false
26459 7390 Acked Read 5 Done Done Sent false
26460 7391 Acked Read 2 Done Done Sent false
26461 7392 Acked Write 2 Done Done Sent false
26462 7393 Acked Write 6 Done Done Sent false
26463 7394 Acked Read 6 Done Done Sent false
26464 7395 Acked Read 1 Done Done Sent false
26465 7396 Acked Write 10 Done Done Sent false
26466 7397 Acked Read 10 Done Done Sent false
26467 7398 Acked Read 7 Done Done Sent false
26468 7399 Acked Write 9 Done Done Sent false
26469 7400 Acked Read 7 Done Done Sent false
26470 7401 Acked Write 4 Done Done Sent false
26471 7402 Acked Write 8 Done Done Sent false
26472 7403 Acked Read 5 Done Done Sent false
26473 7404 Acked Write 8 Done Done Sent false
26474 7405 Acked Write 3 Done Done Sent false
26475 7406 Acked Read 7 Done Done Sent false
26476 7407 Acked Read 1 Done Done Sent false
26477 7408 Acked Read 8 Done Done Sent false
26478 7409 Acked Write 10 Done Done Sent false
26479 7410 Acked Write 3 Done Done Sent false
26480 7411 Acked Write 5 Done Done Sent false
26481 7412 Acked Write 7 Done Done Sent false
26482 7413 Acked Flush 0 Done Done Sent false
26483 7414 Acked Write 10 Done Done Sent false
26484 7415 Acked Read 9 Done Done Sent false
26485 7416 Acked Write 7 Done Done Sent false
26486 7417 Acked Write 5 Done Done Sent false
26487 7418 Acked Read 6 Done Done Sent false
26488 7419 Acked Read 10 Done Done Sent false
26489 7420 Acked Flush 0 Done Done Sent false
26490 7421 Acked Write 2 Done Done Sent false
26491 7422 Acked Read 10 Done Done Sent false
26492 7423 Acked Write 10 Done Done Sent false
26493 7424 Acked Write 1 Done Done Sent false
26494 7425 Acked Read 10 Done Done Sent false
26495 7426 Acked Write 6 Done Done Sent false
26496 7427 Acked Write 3 Done Done Sent false
26497 7428 Acked Read 6 Done Done Sent false
26498 7429 Acked Read 3 Done Done Sent false
26499 7430 Acked Write 5 Done Done Sent false
26500 7431 Acked Write 3 Done Done Sent false
26501 7432 Acked Read 10 Done Done Sent false
26502 7433 Acked Read 3 Done Done Sent false
26503 7434 Acked Write 3 Done Done Sent false
26504 7435 Acked Read 5 Done Done Sent false
26505 7436 Acked Flush 0 Done Done Sent false
26506 7437 Acked Read 7 Done Done Sent false
26507 7438 Acked Write 6 Done Done Sent false
26508 7439 Acked Write 6 Done Done Sent false
26509 7440 Acked Flush 0 Done Done Sent false
26510 7441 Acked Read 5 Done Done Sent false
26511 7442 Acked Write 3 Done Done Sent false
26512 7443 Acked Read 1 Done Done Sent false
26513 7444 Acked Read 2 Done Done Sent false
26514 7445 Acked Write 2 Done Done Sent false
26515 7446 Acked Write 5 Done Done Sent false
26516 7447 Acked Write 3 Done Done Sent false
26517 7448 Acked Write 7 Done Done Sent false
26518 7449 Acked Write 1 Done Done Sent false
26519 7450 Acked Write 7 Done Done Sent false
26520 7451 Acked Write 8 Done Done Sent false
26521 7452 Acked Flush 0 Done Done Sent false
26522 7453 Acked Read 5 Done Done Sent false
26523 7454 Acked Write 8 Done Done Sent false
26524 7455 Acked Write 2 Done Done Sent false
26525 7456 Acked Read 8 Done Done Sent false
26526 7457 Acked Read 8 Done Done Sent false
26527 7458 Acked Write 4 Done Done Sent false
26528 7459 Acked Write 6 Done Done Sent false
26529 7460 Acked Write 6 Done Done Sent false
26530 7461 Acked Write 6 Done Done Sent false
26531 7462 Acked Read 10 Done Done Sent false
26532 7463 Acked Write 6 Done Done Sent false
26533 7464 Acked Read 5 Done Done Sent false
26534 7465 Acked Write 2 Done Done Sent false
26535 7466 Acked Read 4 Done Done Sent false
26536 7467 Acked Read 5 Done Done Sent false
26537 7468 Acked Write 6 Done Done Sent false
26538 7469 Acked Write 6 Done Done Sent false
26539 7470 Acked Write 5 Done Done Sent false
26540 7471 Acked Read 10 Done Done Sent false
26541 7472 Acked Write 9 Done Done Sent false
26542 7473 Acked Read 7 Done Done Sent false
26543 7474 Acked Write 8 Done Done Sent false
26544 7475 Acked Read 3 Done Done Sent false
26545 7476 Acked Read 9 Done Done Sent false
26546 7477 Acked Flush 0 Done Done Sent false
26547 7478 Acked Flush 0 Done Done Sent false
26548 7479 Acked Write 2 Done Done Sent false
26549 7480 Acked Write 8 Done Done Sent false
26550 7481 Acked Write 4 Done Done Sent false
26551 7482 Acked Write 5 Done Done Sent false
26552 7483 Acked Read 7 Done Done Sent false
26553 7484 Acked Write 1 Done Done Sent false
26554 7485 Acked Read 7 Done Done Sent false
26555 7486 Acked Write 4 Done Done Sent false
26556 7487 Acked Flush 0 Done Done Sent false
26557 7488 Acked Flush 0 Done Done Sent false
26558 7489 Acked Read 4 Done Done Sent false
26559 7490 Acked Read 6 Done Done Sent false
26560 7491 Acked Write 10 Done Done Sent false
26561 7492 Acked Write 6 Done Done Sent false
26562 7493 Acked Read 9 Done Done Sent false
26563 7494 Acked Flush 0 Done Done Sent false
26564 7495 Acked Read 6 Done Done Sent false
26565 7496 Acked Write 9 Done Done Sent false
26566 7497 Acked Flush 0 Done Done Sent false
26567 7498 Acked Read 4 Done Done Sent false
26568 7499 Acked Read 8 Done Done Sent false
26569 7500 Acked Read 10 Done Done Sent false
26570 7501 Acked Write 6 Done Done Sent false
26571 7502 Acked Write 9 Done Done Sent false
26572 7503 Acked Read 5 Done Done Sent false
26573 7504 Acked Read 6 Done Done Sent false
26574 7505 Acked Write 7 Done Done Sent false
26575 7506 Acked Flush 0 Done Done Sent false
26576 7507 Acked Write 8 Done Done Sent false
26577 7508 Acked Read 9 Done Done Sent false
26578 7509 Acked Write 1 Done Done Sent false
26579 7510 Acked Write 2 Done Done Sent false
26580 7511 Acked Write 9 Done Done Sent false
26581 7512 Acked Write 10 Done Done Sent false
26582 7513 Acked Read 1 Done Done Sent false
26583 7514 Acked Write 1 Done Done Sent false
26584 7515 Acked Read 5 Done Done Sent false
26585 7516 Acked Write 4 Done Done Sent false
26586 7517 Acked Read 7 Done Done Sent false
26587 7518 Acked Read 5 Done Done Sent false
26588 7519 Acked Read 6 Done Done Sent false
26589 7520 Acked Flush 0 Done Done Sent false
26590 7521 Acked Read 8 Done Done Sent false
26591 7522 Acked Read 7 Done Done Sent false
26592 7523 Acked Read 4 Done Done Sent false
26593 7524 Acked Read 4 Done Done Sent false
26594 7525 Acked Write 10 Done Done Sent false
26595 7526 Acked Write 10 Done Done Sent false
26596 7527 Acked Write 9 Done Done Sent false
26597 7528 Acked Read 9 Done Done Sent false
26598 7529 Acked Read 6 Done Done Sent false
26599 7530 Acked Read 1 Done Done Sent false
26600 7531 Acked Read 9 Done Done Sent false
26601 7532 Acked Write 8 Done Done Sent false
26602 7533 Acked Write 8 Done Done Sent false
26603 7534 Acked Write 3 Done Done Sent false
26604 7535 Acked Read 6 Done Done Sent false
26605 7536 Acked Write 10 Done Done Sent false
26606 7537 Acked Read 9 Done Done Sent false
26607 7538 Acked Write 6 Done Done Sent false
26608 7539 Acked Read 5 Done Done Sent false
26609 7540 Acked Flush 0 Done Done Sent false
26610 7541 Acked Flush 0 Done Done Sent false
26611 7542 Acked Read 2 Done Done Sent false
26612 7543 Acked Read 1 Done Done Sent false
26613 7544 Acked Read 2 Done Done Sent false
26614 7545 Acked Write 6 Done Done Sent false
26615 7546 Acked Write 5 Done Done Sent false
26616 7547 Acked Read 9 Done Done Sent false
26617 7548 Acked Write 9 Done Done Sent false
26618 7549 Acked Write 3 Done Done Sent false
26619 7550 Acked Read 4 Done Done Sent false
26620 7551 Acked Write 5 Done Done Sent false
26621 7552 Acked Read 9 Done Done Sent false
26622 7553 Acked Write 10 Done Done Sent false
26623 7554 Acked Read 2 Done Done Sent false
26624 7555 Acked Read 10 Done Done Sent false
26625 7556 Acked Read 9 Done Done Sent false
26626 7557 Acked Read 3 Done Done Sent false
26627 7558 Acked Write 7 Done Done Sent false
26628 7559 Acked Read 1 Done Done Sent false
26629 7560 Acked Write 8 Done Done Sent false
26630 7561 Acked Write 6 Done Done Sent false
26631 7562 Acked Read 5 Done Done Sent false
26632 7563 Acked Write 3 Done Done Sent false
26633 7564 Acked Flush 0 Done Done Sent false
26634 7565 Acked Write 5 Done Done Sent false
26635 7566 Acked Read 9 Done Done Sent false
26636 7567 Acked Read 1 Done Done Sent false
26637 7568 Acked Read 3 Done Done Sent false
26638 7569 Acked Read 8 Done Done Sent false
26639 7570 Acked Read 10 Done Done Sent false
26640 7571 Acked Write 1 Done Done Sent false
26641 7572 Acked Write 7 Done Done Sent false
26642 7573 Acked Read 6 Done Done Sent false
26643 7574 Acked Write 7 Done Done Sent false
26644 7575 Acked Read 1 Done Done Sent false
26645 7576 Acked Read 1 Done Done Sent false
26646 7577 Acked Read 1 Done Done Sent false
26647 7578 Acked Flush 0 Done Done Sent false
26648 7579 Acked Read 4 Done Done Sent false
26649 7580 Acked Read 4 Done Done Sent false
26650 7581 Acked Read 9 Done Done Sent false
26651 7582 Acked Read 9 Done Done Sent false
26652 7583 Acked Write 7 Done Done Sent false
26653 7584 Acked Write 4 Done Done Sent false
26654 7585 Acked Read 5 Done Done Sent false
26655 7586 Acked Write 8 Done Done Sent false
26656 7587 Acked Write 10 Done Done Sent false
26657 7588 Acked Write 3 Done Done Sent false
26658 7589 Acked Write 8 Done Done Sent false
26659 7590 Acked Read 9 Done Done Sent false
26660 7591 Acked Read 8 Done Done Sent false
26661 7592 Acked Read 5 Done Done Sent false
26662 7593 Acked Read 4 Done Done Sent false
26663 7594 Acked Read 5 Done Done Sent false
26664 7595 Acked Write 3 Done Done Sent false
26665 7596 Acked Flush 0 Done Done Sent false
26666 7597 Acked Read 5 Done Done Sent false
26667 7598 Acked Write 6 Done Done Sent false
26668 7599 Acked Flush 0 Done Done Sent false
26669 7600 Acked Write 8 Done Done Sent false
26670 7601 Acked Write 5 Done Done Sent false
26671 7602 Acked Read 3 Done Done Sent false
26672 7603 Acked Read 6 Done Done Sent false
26673 7604 Acked Read 9 Done Done Sent false
26674 7605 Acked Write 6 Done Done Sent false
26675 7606 Acked Write 9 Done Done Sent false
26676 7607 Acked Flush 0 Done Done Sent false
26677 7608 Acked Flush 0 Done Done Sent false
26678 7609 Acked Write 7 Done Done Sent false
26679 7610 Acked Write 7 Done Done Sent false
26680 7611 Acked Read 4 Done Done Sent false
26681 7612 Acked Read 9 Done Done Sent false
26682 7613 Acked Flush 0 Done Done Sent false
26683 7614 Acked Read 1 Done Done Sent false
26684 7615 Acked Read 3 Done Done Sent false
26685 7616 Acked Write 2 Done Done Sent false
26686 7617 Acked Write 8 Done Done Sent false
26687 7618 Acked Read 3 Done Done Sent false
26688 7619 Acked Read 5 Done Done Sent false
26689 7620 Acked Flush 0 Done Done Sent false
26690 7621 Acked Write 10 Done Done Sent false
26691 7622 Acked Read 1 Done Done Sent false
26692 7623 Acked Write 3 Done Done Sent false
26693 7624 Acked Write 4 Done Done Sent false
26694 7625 Acked Read 5 Done Done Sent false
26695 7626 Acked Read 2 Done Done Sent false
26696 7627 Acked Write 8 Done Done Sent false
26697 7628 Acked Write 8 Done Done Sent false
26698 7629 Acked Flush 0 Done Done Sent false
26699 7630 Acked Flush 0 Done Done Sent false
26700 7631 Acked Read 1 Done Done Sent false
26701 7632 Acked Write 4 Done Done Sent false
26702 7633 Acked Read 8 Done Done Sent false
26703 7634 Acked Write 3 Done Done Sent false
26704 7635 Acked Read 2 Done Done Sent false
26705 7636 Acked Write 2 Done Done Sent false
26706 7637 Acked Read 9 Done Done Sent false
26707 7638 Acked Flush 0 Done Done Sent false
26708 7639 Acked Read 5 Done Done Sent false
26709 7640 Acked Read 10 Done Done Sent false
26710 7641 Acked Flush 0 Done Done Sent false
26711 7642 Acked Write 10 Done Done Sent false
26712 7643 Acked Flush 0 Done Done Sent false
26713 STATES DS:0 DS:1 DS:2 TOTAL
26714 Sent 0 0 302 302
26715 Done 302 302 0 604
26716 Skipped 0 0 0 0
26717 Error 0 0 0 0
26718 Last Flush: 7643 7643 7341
26719 Downstairs last five completed: 7341 7340 7339 7338 7337
26720 Upstairs last five completed: 7643 7642 7641 7640 7639
26721 CLIENT: Up:0 ds:302 act:2
267222024-12-11T23:46:07.676ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267232024-12-11T23:46:07.676ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33512) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267242024-12-11T23:46:07.677ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267252024-12-11T23:46:07.677ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267262024-12-11T23:46:07.677ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267272024-12-11T23:46:07.677ZINFOcrucible: send last flush ID to this DS: 7341 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267282024-12-11T23:46:07.678ZINFOcrucible: Replied this last flush ID: 7341 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267292024-12-11T23:46:07.678ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267302024-12-11T23:46:07.678ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267312024-12-11T23:46:07.678ZINFOcrucible: [2] client re-new 302 jobs since flush 7341 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267322024-12-11T23:46:07.678ZINFOcrucible: [2] Replayed 302 jobs since flush: 7341 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
26733 ----------------------------------------------------------------
26734 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26735 Downstairs last five completed: 7643 7642 7641 7640 7639
26736 Upstairs last five completed: 7643 7642 7641 7640 7639
26737 CLIENT: Up:0 ds:0 act:3
26738 Replay: All jobs finished
26739 Main task has work to do, go find it
26740 got dsc Stop(2)
26741 stop 2
26742 [8830] Got stop action so:false kr:false
26743 Dec 11 23:46:07.920 INFO request completed, latency_us: 335, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 6261d985-4054-4e2a-9c89-6cbe2b80e108, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
267442024-12-11T23:46:07.920ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267452024-12-11T23:46:07.920ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267462024-12-11T23:46:07.920ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267472024-12-11T23:46:07.920ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267482024-12-11T23:46:07.920ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
267492024-12-11T23:46:07.921ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
26750 Dec 11 23:46:07.921 INFO request completed, latency_us: 159, response_code: 200, uri: /state/cid/2, method: GET, req_id: 5bf4b7f3-b628-4a20-9e8c-1a6b253de254, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
26751 [8830] Exited with: ExitStatus(unix_wait_status(9))
26752 [8830][2] reports Exit
26753 Dec 11 23:46:11.923 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/2, method: GET, req_id: 76ad8c78-0ce7-4c74-b1fe-2de3f11aeb7d, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
26754 001/300 Write block 3550 len 36864 data: 6 5 5 5 5 4 6 6 6
26755 002/300 Write block 3938 len 40960 data: 6 6 5 6 6 7 6 6 6 8
26756 003/300 Write block 3741 len 28672 data: 6 7 6 5 4 4 3
26757 004/300 Write block 507 len 16384 data: 3 3 5 5
26758 005/300 Read block 1127 len 12288
26759 006/300 Flush
26760 007/300 Write block 4419 len 28672 data: 3 3 3 3 3 6 6
26761 008/300 Read block 4591 len 20480
26762 009/300 Write block 2762 len 32768 data: 2 3 4 5 6 6 5 6
26763 010/300 Read block 3551 len 8192
26764 011/300 Write block 3998 len 28672 data: 4 3 4 5 6 5 5
26765 012/300 Read block 4860 len 40960
26766 013/300 Read block 3607 len 24576
26767 014/300 Flush
26768 015/300 Read block 3346 len 32768
26769 016/300 Write block 4021 len 4096 data: 5
26770 017/300 Write block 4310 len 32768 data: 3 3 3 4 5 4 4 3
26771 018/300 Read block 1985 len 24576
26772 019/300 Read block 2623 len 40960
26773 020/300 Read block 945 len 4096
26774 021/300 Read block 3562 len 32768
26775 022/300 Read block 3435 len 36864
26776 023/300 Read block 3148 len 32768
26777 024/300 Read block 1520 len 4096
26778 025/300 Read block 2519 len 8192
26779 026/300 Write block 1227 len 20480 data: 4 5 5 4 5
26780 027/300 Write block 2985 len 32768 data: 3 3 4 4 3 3 3 4
26781 028/300 Read block 3440 len 12288
26782 029/300 Read block 581 len 32768
26783 030/300 Read block 581 len 24576
26784 031/300 Read block 2220 len 28672
26785 032/300 Flush
26786 033/300 Write block 1185 len 40960 data: 6 5 7 7 6 7 4 4 4 4
26787 034/300 Read block 522 len 8192
26788 035/300 Write block 164 len 16384 data: 5 5 4 4
26789 036/300 Read block 2858 len 4096
26790 037/300 Write block 4215 len 36864 data: 6 5 5 6 6 4 4 4 4
26791 038/300 Flush
26792 039/300 Flush
26793 040/300 Write block 503 len 20480 data: 6 6 4 3 4
26794 041/300 Read block 1645 len 40960
26795 042/300 Write block 1175 len 8192 data: 3 3
26796 043/300 Flush
26797 044/300 Read block 4285 len 28672
26798 045/300 Read block 3930 len 24576
26799 046/300 Read block 4300 len 24576
26800 047/300 Read block 1715 len 20480
26801 048/300 Read block 4076 len 16384
26802 049/300 Write block 1440 len 4096 data: 8
26803 050/300 Read block 435 len 36864
26804 051/300 Write block 307 len 12288 data: 4 5 6
26805 052/300 Read block 1838 len 28672
26806 053/300 Read block 855 len 4096
26807 054/300 Write block 4827 len 4096 data: 6
26808 055/300 Read block 1884 len 24576
26809 056/300 Write block 2851 len 8192 data: 4 5
26810 057/300 Read block 2395 len 16384
26811 058/300 Write block 2559 len 40960 data: 4 5 4 4 4 4 4 4 4 4
26812 059/300 Write block 2209 len 4096 data: 5
26813 060/300 Write block 4128 len 8192 data: 5 4
26814 061/300 Write block 551 len 4096 data: 5
26815 062/300 Write block 2314 len 36864 data: 7 6 5 5 4 5 5 2 2
26816 063/300 Flush
26817 064/300 Flush
26818 065/300 Read block 4447 len 12288
26819 066/300 Write block 358 len 12288 data: 7 7 7
26820 067/300 Write block 1371 len 20480 data: 5 6 7 6 5
26821 068/300 Write block 4718 len 40960 data: 6 6 6 7 7 9 7 5 5 5
26822 069/300 Write block 3056 len 16384 data: 7 7 6 5
26823 070/300 Read block 2391 len 24576
26824 071/300 Read block 81 len 4096
26825 072/300 Read block 2164 len 28672
26826 073/300 Read block 1258 len 40960
26827 074/300 Read block 2561 len 20480
26828 075/300 Read block 4826 len 8192
26829 076/300 Read block 909 len 8192
26830 077/300 Write block 4020 len 36864 data: 8 6 5 5 5 5 4 3 2
26831 078/300 Write block 138 len 24576 data: 5 5 5 6 5 5
26832 079/300 Write block 4731 len 32768 data: 3 3 5 4 4 5 6 5
26833 080/300 Read block 4435 len 20480
26834 081/300 Write block 636 len 16384 data: 5 7 8 9
26835 082/300 Read block 3422 len 32768
26836 083/300 Write block 4241 len 16384 data: 7 4 4 5
26837 084/300 Read block 3823 len 36864
26838 085/300 Write block 2981 len 24576 data: 6 5 4 3 4 4
26839 086/300 Flush
26840 087/300 Write block 309 len 24576 data: 7 5 6 5 6 7
26841 088/300 Read block 1666 len 32768
26842 089/300 Flush
26843 090/300 Read block 2880 len 20480
26844 091/300 Read block 3386 len 40960
26845 092/300 Flush
26846 093/300 Read block 256 len 24576
26847 094/300 Write block 4117 len 36864 data: 7 7 7 6 6 9 6 8 8
26848 095/300 Write block 2055 len 4096 data: 4
26849 096/300 Read block 1146 len 8192
26850 097/300 Read block 574 len 8192
26851 098/300 Read block 220 len 20480
26852 099/300 Read block 3340 len 28672
26853 100/300 Write block 940 len 32768 data: 4 4 5 5 4 4 3 3
26854 101/300 Write block 3154 len 4096 data: 7
26855 102/300 Read block 3175 len 24576
26856 103/300 Write block 4255 len 8192 data: 8 8
26857 104/300 Write block 1900 len 16384 data: 3 3 3 4
26858 105/300 Read block 4890 len 40960
26859 106/300 Write block 1656 len 36864 data: 2 2 2 3 5 6 6 5 4
26860 107/300 Write block 4828 len 4096 data: 6
26861 108/300 Read block 160 len 16384
26862 109/300 Read block 2260 len 8192
26863 110/300 Flush
26864 111/300 Write block 3880 len 32768 data: 6 6 6 6 6 7 9 9
26865 112/300 Read block 486 len 16384
26866 113/300 Write block 1378 len 28672 data: 3 4 5 5 5 5 5
26867 114/300 Read block 2332 len 24576
26868 115/300 Write block 3354 len 32768 data: 2 2 2 3 3 3 2 2
26869 116/300 Write block 1148 len 32768 data: 6 5 6 6 4 4 3 3
26870 117/300 Read block 2904 len 36864
26871 118/300 Read block 4133 len 20480
26872 119/300 Write block 268 len 40960 data: 4 4 4 5 4 4 4 3 3 3
26873 120/300 Read block 2510 len 20480
26874 121/300 Read block 2093 len 16384
26875 122/300 Read block 36 len 4096
26876 123/300 Read block 314 len 36864
26877 124/300 Write block 3645 len 40960 data: 6 7 6 8 6 5 6 7 6 4
26878 125/300 Read block 4340 len 4096
26879 126/300 Read block 3836 len 40960
26880 127/300 Read block 4854 len 24576
26881 128/300 Read block 4303 len 40960
26882 129/300 Read block 3243 len 20480
26883 130/300 Write block 1781 len 32768 data: 8 7 6 5 5 5 4 4
26884 131/300 Write block 2334 len 8192 data: 9 8
26885 132/300 Write block 3668 len 36864 data: 4 4 4 4 4 4 4 5 4
26886 133/300 Write block 1098 len 32768 data: 6 5 6 6 4 5 5 5
26887 134/300 Write block 2156 len 16384 data: 7 7 6 6
26888 135/300 Write block 3426 len 24576 data: 4 4 4 4 4 2
26889 136/300 Read block 3542 len 36864
26890 137/300 Read block 3228 len 40960
26891 138/300 Flush
26892 139/300 Write block 2966 len 4096 data: 7
26893 140/300 Write block 3339 len 20480 data: 4 3 3 3 3
26894 141/300 Read block 2450 len 32768
26895 142/300 Write block 4732 len 20480 data: 4 6 5 5 6
26896 143/300 Read block 2028 len 12288
26897 144/300 Read block 1403 len 16384
26898 145/300 Read block 732 len 16384
26899 146/300 Write block 4749 len 40960 data: 3 4 6 5 6 5 7 5 5 4
26900 147/300 Flush
26901 148/300 Read block 1621 len 16384
26902 149/300 Write block 3422 len 16384 data: 7 5 5 4
26903 150/300 Read block 2228 len 28672
26904 151/300 Write block 2774 len 16384 data: 4 5 4 4
26905 152/300 Read block 2779 len 4096
26906 153/300 Read block 2257 len 16384
26907 154/300 Read block 3925 len 4096
26908 155/300 Write block 3728 len 36864 data: 5 5 5 5 4 3 2 2 2
26909 156/300 Write block 3272 len 24576 data: 3 3 4 5 5 6
26910 157/300 Read block 841 len 32768
26911 158/300 Read block 4985 len 40960
26912 159/300 Read block 4695 len 16384
26913 160/300 Write block 2532 len 16384 data: 3 3 2 2
26914 161/300 Read block 390 len 32768
26915 162/300 Read block 577 len 12288
26916 163/300 Read block 1043 len 24576
26917 164/300 Write block 639 len 16384 data: 10 9 6 6
26918 165/300 Write block 892 len 8192 data: 4 4
26919 166/300 Write block 462 len 24576 data: 4 5 5 4 3 2
26920 167/300 Read block 3207 len 40960
26921 168/300 Write block 4533 len 28672 data: 7 7 5 6 6 6 6
26922 169/300 Read block 2208 len 12288
26923 170/300 Read block 4707 len 4096
26924 171/300 Read block 2986 len 36864
26925 172/300 Read block 4781 len 12288
26926 173/300 Write block 4404 len 20480 data: 5 5 4 3 4
26927 174/300 Flush
26928 175/300 Read block 2679 len 4096
26929 176/300 Write block 1288 len 32768 data: 5 5 7 7 7 7 7 7
26930 177/300 Read block 3980 len 36864
26931 178/300 Read block 3504 len 12288
26932 179/300 Write block 2734 len 24576 data: 10 11 11 11 11 9
26933 180/300 Read block 51 len 24576
26934 181/300 Write block 1147 len 40960 data: 6 7 6 7 7 5 5 4 4 3
26935 182/300 Read block 4613 len 24576
26936 183/300 Read block 3841 len 40960
26937 184/300 Write block 3532 len 8192 data: 5 3
26938 185/300 Read block 2712 len 28672
26939 186/300 Read block 1462 len 32768
26940 187/300 Write block 1834 len 40960 data: 5 5 6 6 6 5 5 4 6 5
26941 188/300 Write block 3162 len 28672 data: 4 4 5 4 5 5 7
26942 189/300 Read block 973 len 16384
26943 190/300 Read block 1335 len 8192
26944 191/300 Write block 4145 len 8192 data: 6 6
26945 192/300 Read block 3892 len 8192
26946 193/300 Flush
26947 194/300 Write block 804 len 8192 data: 6 6
26948 195/300 Flush
26949 196/300 Write block 155 len 8192 data: 6 6
26950 197/300 Read block 1634 len 36864
26951 198/300 Write block 1617 len 36864 data: 3 3 4 6 7 7 6 7 8
26952 199/300 Write block 4341 len 4096 data: 8
26953 200/300 Read block 2198 len 40960
26954 201/300 Write block 644 len 16384 data: 6 6 4 4
26955 202/300 Write block 4045 len 8192 data: 7 7
26956 203/300 Write block 1531 len 20480 data: 3 3 2 2 2
26957 204/300 Write block 2235 len 4096 data: 6
26958 205/300 Write block 1869 len 12288 data: 3 3 3
26959 206/300 Read block 2805 len 12288
26960 207/300 Write block 1526 len 8192 data: 3 3
26961 208/300 Read block 3917 len 8192
26962 209/300 Flush
26963 210/300 Read block 305 len 32768
26964 211/300 Read block 4268 len 12288
26965 212/300 Write block 1516 len 16384 data: 4 6 8 8
26966 213/300 Write block 2314 len 24576 data: 8 7 6 6 5 6
26967 214/300 Write block 2831 len 12288 data: 7 6 7
26968 215/300 Write block 4159 len 32768 data: 2 3 3 3 6 6 5 5
26969 216/300 Read block 1942 len 8192
26970 217/300 Read block 2963 len 4096
26971 218/300 Write block 392 len 4096 data: 5
26972 219/300 Read block 1722 len 24576
26973 220/300 Read block 3428 len 40960
26974 221/300 Read block 816 len 28672
26975 222/300 Write block 70 len 28672 data: 8 8 8 7 5 5 6
26976 223/300 Read block 1698 len 36864
26977 224/300 Flush
26978 225/300 Read block 1068 len 40960
26979 226/300 Read block 1037 len 12288
26980 227/300 Write block 1345 len 16384 data: 3 4 5 3
26981 228/300 Write block 3611 len 28672 data: 2 2 2 2 2 4 5
26982 229/300 Write block 3211 len 8192 data: 9 7
26983 230/300 Write block 4222 len 16384 data: 5 5 4 4
26984 231/300 Read block 2219 len 8192
26985 232/300 Write block 2815 len 16384 data: 8 6 5 5
26986 233/300 Read block 1623 len 16384
26987 234/300 Write block 4610 len 32768 data: 4 3 3 4 5 5 6 7
26988 235/300 Read block 1586 len 8192
26989 236/300 Write block 3060 len 4096 data: 5
26990 237/300 Write block 1371 len 4096 data: 6
26991 238/300 Read block 4020 len 20480
26992 239/300 Read block 1734 len 20480
26993 240/300 Write block 3278 len 32768 data: 5 7 7 6 7 6 6 5
26994 241/300 Read block 1906 len 4096
26995 242/300 Read block 2551 len 32768
26996 243/300 Read block 4775 len 40960
26997 244/300 Write block 3992 len 20480 data: 7 5 4 4 4
26998 245/300 Write block 898 len 32768 data: 4 4 4 4 3 3 2 2
26999 246/300 Write block 675 len 40960 data: 3 3 2 2 2 2 2 2 3 4
27000 247/300 Read block 4709 len 36864
27001 248/300 Read block 4155 len 20480
27002 249/300 Read block 4637 len 32768
27003 250/300 Write block 1808 len 4096 data: 4
27004 251/300 Read block 3555 len 12288
27005 252/300 Write block 3131 len 36864 data: 6 6 6 5 7 8 8 6 6
27006 253/300 Flush
27007 254/300 Write block 3615 len 20480 data: 3 5 6 5 6
27008 255/300 Read block 4243 len 28672
27009 256/300 Write block 1696 len 40960 data: 6 7 8 5 4 4 4 3 5 6
27010 257/300 Read block 1945 len 28672
27011 258/300 Write block 1880 len 12288 data: 5 5 6
27012 259/300 Write block 3486 len 16384 data: 6 9 9 8
27013 260/300 Read block 4056 len 12288
27014 261/300 Write block 4388 len 36864 data: 7 6 5 4 3 3 5 5 4
27015 262/300 Read block 3299 len 12288
27016 263/300 Flush
27017 264/300 Read block 3621 len 8192
27018 265/300 Write block 2511 len 24576 data: 4 4 4 4 5 4
27019 266/300 Read block 3609 len 36864
27020 267/300 Write block 2542 len 12288 data: 2 4 5
27021 268/300 Write block 1697 len 12288 data: 8 9 6
27022 269/300 Read block 3902 len 20480
27023 270/300 Write block 2771 len 12288 data: 4 4 4
27024 271/300 Read block 4921 len 12288
27025 272/300 Write block 1432 len 24576 data: 4 4 3 5 5 4
27026 273/300 Read block 1894 len 12288
27027 274/300 Read block 95 len 32768
27028 275/300 Read block 4576 len 8192
27029 276/300 Read block 2808 len 36864
27030 277/300 Flush
27031 278/300 Write block 63 len 28672 data: 4 6 6 5 6 7 8
27032 279/300 Read block 4553 len 20480
27033 280/300 Read block 4828 len 4096
27034 281/300 Write block 3880 len 4096 data: 7
27035 282/300 Read block 3106 len 32768
27036 283/300 Flush
27037 284/300 Flush
27038 285/300 Write block 2573 len 32768 data: 4 8 9 10 11 12 11 7
27039 286/300 Write block 2584 len 4096 data: 5
27040 287/300 Read block 3702 len 12288
27041 288/300 Read block 3423 len 28672
27042 289/300 Read block 1390 len 12288
27043 290/300 Write block 4142 len 32768 data: 5 7 7 7 7 7 7 6
27044 291/300 Write block 1505 len 20480 data: 5 6 5 5 2
27045 292/300 Read block 960 len 40960
27046 293/300 Read block 3927 len 20480
27047 294/300 Flush
27048 295/300 Read block 3465 len 36864
27049 296/300 Write block 3208 len 20480 data: 8 9 7 10 8
27050 297/300 Read block 994 len 32768
27051 298/300 Write block 1100 len 28672 data: 7 7 5 6 6 6 4
27052 299/300 Read block 3441 len 16384
27053 300/300 Read block 823 len 32768
27054 301/300 Write block 1297 len 12288 data: 6 4 3
27055 Main task has work to do, go find it
27056 got dsc Start(2)
27057 start 2
27058 Dec 11 23:46:12.159 INFO request completed, latency_us: 264, response_code: 204, uri: /start/cid/2, method: POST, req_id: e4f6c548-f99f-4a82-8dec-d79f2561d381, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
27059 Replay: started 2, returned:Ok(())
27060 ----------------------------------------------------------------
27061 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27062 States: Active Active Offline
27063 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27064 7644 Acked Write 9 Done Done Sent false
27065 7645 Acked Write 10 Done Done Sent false
27066 7646 Acked Write 7 Done Done Sent false
27067 7647 Acked Write 4 Done Done Sent false
27068 7648 Acked Read 3 Done Done Sent false
27069 7649 Acked Flush 0 Done Done Sent false
27070 7650 Acked Write 7 Done Done Sent false
27071 7651 Acked Read 5 Done Done Sent false
27072 7652 Acked Write 8 Done Done Sent false
27073 7653 Acked Read 2 Done Done Sent false
27074 7654 Acked Write 7 Done Done Sent false
27075 7655 Acked Read 10 Done Done Sent false
27076 7656 Acked Read 6 Done Done Sent false
27077 7657 Acked Flush 0 Done Done Sent false
27078 7658 Acked Read 8 Done Done Sent false
27079 7659 Acked Write 1 Done Done Sent false
27080 7660 Acked Write 8 Done Done Sent false
27081 7661 Acked Read 6 Done Done Sent false
27082 7662 Acked Read 10 Done Done Sent false
27083 7663 Acked Read 1 Done Done Sent false
27084 7664 Acked Read 8 Done Done Sent false
27085 7665 Acked Read 9 Done Done Sent false
27086 7666 Acked Read 8 Done Done Sent false
27087 7667 Acked Read 1 Done Done Sent false
27088 7668 Acked Read 2 Done Done Sent false
27089 7669 Acked Write 5 Done Done Sent false
27090 7670 Acked Write 8 Done Done Sent false
27091 7671 Acked Read 3 Done Done Sent false
27092 7672 Acked Read 8 Done Done Sent false
27093 7673 Acked Read 6 Done Done Sent false
27094 7674 Acked Read 7 Done Done Sent false
27095 7675 Acked Flush 0 Done Done Sent false
27096 7676 Acked Write 10 Done Done Sent false
27097 7677 Acked Read 2 Done Done Sent false
27098 7678 Acked Write 4 Done Done Sent false
27099 7679 Acked Read 1 Done Done Sent false
27100 7680 Acked Write 9 Done Done Sent false
27101 7681 Acked Flush 0 Done Done Sent false
27102 7682 Acked Flush 0 Done Done Sent false
27103 7683 Acked Write 5 Done Done Sent false
27104 7684 Acked Read 10 Done Done Sent false
27105 7685 Acked Write 2 Done Done Sent false
27106 7686 Acked Flush 0 Done Done Sent false
27107 7687 Acked Read 7 Done Done Sent false
27108 7688 Acked Read 6 Done Done Sent false
27109 7689 Acked Read 6 Done Done Sent false
27110 7690 Acked Read 5 Done Done Sent false
27111 7691 Acked Read 4 Done Done Sent false
27112 7692 Acked Write 1 Done Done Sent false
27113 7693 Acked Read 9 Done Done Sent false
27114 7694 Acked Write 3 Done Done Sent false
27115 7695 Acked Read 7 Done Done Sent false
27116 7696 Acked Read 1 Done Done Sent false
27117 7697 Acked Write 1 Done Done Sent false
27118 7698 Acked Read 6 Done Done Sent false
27119 7699 Acked Write 2 Done Done Sent false
27120 7700 Acked Read 4 Done Done Sent false
27121 7701 Acked Write 10 Done Done Sent false
27122 7702 Acked Write 1 Done Done Sent false
27123 7703 Acked Write 2 Done Done Sent false
27124 7704 Acked Write 1 Done Done Sent false
27125 7705 Acked Write 9 Done Done Sent false
27126 7706 Acked Flush 0 Done Done Sent false
27127 7707 Acked Flush 0 Done Done Sent false
27128 7708 Acked Read 3 Done Done Sent false
27129 7709 Acked Write 3 Done Done Sent false
27130 7710 Acked Write 5 Done Done Sent false
27131 7711 Acked Write 10 Done Done Sent false
27132 7712 Acked Write 4 Done Done Sent false
27133 7713 Acked Read 6 Done Done Sent false
27134 7714 Acked Read 1 Done Done Sent false
27135 7715 Acked Read 7 Done Done Sent false
27136 7716 Acked Read 10 Done Done Sent false
27137 7717 Acked Read 5 Done Done Sent false
27138 7718 Acked Read 2 Done Done Sent false
27139 7719 Acked Read 2 Done Done Sent false
27140 7720 Acked Write 9 Done Done Sent false
27141 7721 Acked Write 6 Done Done Sent false
27142 7722 Acked Write 8 Done Done Sent false
27143 7723 Acked Read 5 Done Done Sent false
27144 7724 Acked Write 4 Done Done Sent false
27145 7725 Acked Read 8 Done Done Sent false
27146 7726 Acked Write 4 Done Done Sent false
27147 7727 Acked Read 9 Done Done Sent false
27148 7728 Acked Write 6 Done Done Sent false
27149 7729 Acked Flush 0 Done Done Sent false
27150 7730 Acked Write 6 Done Done Sent false
27151 7731 Acked Read 8 Done Done Sent false
27152 7732 Acked Flush 0 Done Done Sent false
27153 7733 Acked Read 5 Done Done Sent false
27154 7734 Acked Read 10 Done Done Sent false
27155 7735 Acked Flush 0 Done Done Sent false
27156 7736 Acked Read 6 Done Done Sent false
27157 7737 Acked Write 9 Done Done Sent false
27158 7738 Acked Write 1 Done Done Sent false
27159 7739 Acked Read 2 Done Done Sent false
27160 7740 Acked Read 2 Done Done Sent false
27161 7741 Acked Read 5 Done Done Sent false
27162 7742 Acked Read 7 Done Done Sent false
27163 7743 Acked Write 8 Done Done Sent false
27164 7744 Acked Write 1 Done Done Sent false
27165 7745 Acked Read 6 Done Done Sent false
27166 7746 Acked Write 2 Done Done Sent false
27167 7747 Acked Write 4 Done Done Sent false
27168 7748 Acked Read 10 Done Done Sent false
27169 7749 Acked Write 9 Done Done Sent false
27170 7750 Acked Write 1 Done Done Sent false
27171 7751 Acked Read 4 Done Done Sent false
27172 7752 Acked Read 2 Done Done Sent false
27173 7753 Acked Flush 0 Done Done Sent false
27174 7754 Acked Write 8 Done Done Sent false
27175 7755 Acked Read 4 Done Done Sent false
27176 7756 Acked Write 7 Done Done Sent false
27177 7757 Acked Read 6 Done Done Sent false
27178 7758 Acked Write 8 Done Done Sent false
27179 7759 Acked Write 8 Done Done Sent false
27180 7760 Acked Read 9 Done Done Sent false
27181 7761 Acked Read 5 Done Done Sent false
27182 7762 Acked Write 10 Done Done Sent false
27183 7763 Acked Read 5 Done Done Sent false
27184 7764 Acked Read 4 Done Done Sent false
27185 7765 Acked Read 1 Done Done Sent false
27186 7766 Acked Read 9 Done Done Sent false
27187 7767 Acked Write 10 Done Done Sent false
27188 7768 Acked Read 1 Done Done Sent false
27189 7769 Acked Read 10 Done Done Sent false
27190 7770 Acked Read 6 Done Done Sent false
27191 7771 Acked Read 10 Done Done Sent false
27192 7772 Acked Read 5 Done Done Sent false
27193 7773 Acked Write 8 Done Done Sent false
27194 7774 Acked Write 2 Done Done Sent false
27195 7775 Acked Write 9 Done Done Sent false
27196 7776 Acked Write 8 Done Done Sent false
27197 7777 Acked Write 4 Done Done Sent false
27198 7778 Acked Write 6 Done Done Sent false
27199 7779 Acked Read 9 Done Done Sent false
27200 7780 Acked Read 10 Done Done Sent false
27201 7781 Acked Flush 0 Done Done Sent false
27202 7782 Acked Write 1 Done Done Sent false
27203 7783 Acked Write 5 Done Done Sent false
27204 7784 Acked Read 8 Done Done Sent false
27205 7785 Acked Write 5 Done Done Sent false
27206 7786 Acked Read 3 Done Done Sent false
27207 7787 Acked Read 4 Done Done Sent false
27208 7788 Acked Read 4 Done Done Sent false
27209 7789 Acked Write 10 Done Done Sent false
27210 7790 Acked Flush 0 Done Done Sent false
27211 7791 Acked Read 4 Done Done Sent false
27212 7792 Acked Write 4 Done Done Sent false
27213 7793 Acked Read 7 Done Done Sent false
27214 7794 Acked Write 4 Done Done Sent false
27215 7795 Acked Read 1 Done Done Sent false
27216 7796 Acked Read 4 Done Done Sent false
27217 7797 Acked Read 1 Done Done Sent false
27218 7798 Acked Write 9 Done Done Sent false
27219 7799 Acked Write 6 Done Done Sent false
27220 7800 Acked Read 8 Done Done Sent false
27221 7801 Acked Read 10 Done Done Sent false
27222 7802 Acked Read 4 Done Done Sent false
27223 7803 Acked Write 4 Done Done Sent false
27224 7804 Acked Read 8 Done Done Sent false
27225 7805 Acked Read 3 Done Done Sent false
27226 7806 Acked Read 6 Done Done Sent false
27227 7807 Acked Write 4 Done Done Sent false
27228 7808 Acked Write 2 Done Done Sent false
27229 7809 Acked Write 6 Done Done Sent false
27230 7810 Acked Read 10 Done Done Sent false
27231 7811 Acked Write 7 Done Done Sent false
27232 7812 Acked Read 3 Done Done Sent false
27233 7813 Acked Read 1 Done Done Sent false
27234 7814 Acked Read 9 Done Done Sent false
27235 7815 Acked Read 3 Done Done Sent false
27236 7816 Acked Write 5 Done Done Sent false
27237 7817 Acked Flush 0 Done Done Sent false
27238 7818 Acked Read 1 Done Done Sent false
27239 7819 Acked Write 8 Done Done Sent false
27240 7820 Acked Read 9 Done Done Sent false
27241 7821 Acked Read 3 Done Done Sent false
27242 7822 Acked Write 6 Done Done Sent false
27243 7823 Acked Read 6 Done Done Sent false
27244 7824 Acked Write 10 Done Done Sent false
27245 7825 Acked Read 6 Done Done Sent false
27246 7826 Acked Read 10 Done Done Sent false
27247 7827 Acked Write 2 Done Done Sent false
27248 7828 Acked Read 7 Done Done Sent false
27249 7829 Acked Read 8 Done Done Sent false
27250 7830 Acked Write 10 Done Done Sent false
27251 7831 Acked Write 7 Done Done Sent false
27252 7832 Acked Read 4 Done Done Sent false
27253 7833 Acked Read 2 Done Done Sent false
27254 7834 Acked Write 2 Done Done Sent false
27255 7835 Acked Read 2 Done Done Sent false
27256 7836 Acked Flush 0 Done Done Sent false
27257 7837 Acked Write 2 Done Done Sent false
27258 7838 Acked Flush 0 Done Done Sent false
27259 7839 Acked Write 2 Done Done Sent false
27260 7840 Acked Read 9 Done Done Sent false
27261 7841 Acked Write 9 Done Done Sent false
27262 7842 Acked Write 1 Done Done Sent false
27263 7843 Acked Read 10 Done Done Sent false
27264 7844 Acked Write 4 Done Done Sent false
27265 7845 Acked Write 2 Done Done Sent false
27266 7846 Acked Write 5 Done Done Sent false
27267 7847 Acked Write 1 Done Done Sent false
27268 7848 Acked Write 3 Done Done Sent false
27269 7849 Acked Read 3 Done Done Sent false
27270 7850 Acked Write 2 Done Done Sent false
27271 7851 Acked Read 2 Done Done Sent false
27272 7852 Acked Flush 0 Done Done Sent false
27273 7853 Acked Read 8 Done Done Sent false
27274 7854 Acked Read 3 Done Done Sent false
27275 7855 Acked Write 4 Done Done Sent false
27276 7856 Acked Write 6 Done Done Sent false
27277 7857 Acked Write 3 Done Done Sent false
27278 7858 Acked Write 8 Done Done Sent false
27279 7859 Acked Read 2 Done Done Sent false
27280 7860 Acked Read 1 Done Done Sent false
27281 7861 Acked Write 1 Done Done Sent false
27282 7862 Acked Read 6 Done Done Sent false
27283 7863 Acked Read 10 Done Done Sent false
27284 7864 Acked Read 7 Done Done Sent false
27285 7865 Acked Write 7 Done Done Sent false
27286 7866 Acked Read 9 Done Done Sent false
27287 7867 Acked Flush 0 Done Done Sent false
27288 7868 Acked Read 10 Done Done Sent false
27289 7869 Acked Read 3 Done Done Sent false
27290 7870 Acked Write 4 Done Done Sent false
27291 7871 Acked Write 7 Done Done Sent false
27292 7872 Acked Write 2 Done Done Sent false
27293 7873 Acked Write 4 Done Done Sent false
27294 7874 Acked Read 2 Done Done Sent false
27295 7875 Acked Write 4 Done Done Sent false
27296 7876 Acked Read 4 Done Done Sent false
27297 7877 Acked Write 8 Done Done Sent false
27298 7878 Acked Read 2 Done Done Sent false
27299 7879 Acked Write 1 Done Done Sent false
27300 7880 Acked Write 1 Done Done Sent false
27301 7881 Acked Read 5 Done Done Sent false
27302 7882 Acked Read 5 Done Done Sent false
27303 7883 Acked Write 8 Done Done Sent false
27304 7884 Acked Read 1 Done Done Sent false
27305 7885 Acked Read 8 Done Done Sent false
27306 7886 Acked Read 10 Done Done Sent false
27307 7887 Acked Write 5 Done Done Sent false
27308 7888 Acked Write 8 Done Done Sent false
27309 7889 Acked Write 10 Done Done Sent false
27310 7890 Acked Read 9 Done Done Sent false
27311 7891 Acked Read 5 Done Done Sent false
27312 7892 Acked Read 8 Done Done Sent false
27313 7893 Acked Write 1 Done Done Sent false
27314 7894 Acked Read 3 Done Done Sent false
27315 7895 Acked Write 9 Done Done Sent false
27316 7896 Acked Flush 0 Done Done Sent false
27317 7897 Acked Write 5 Done Done Sent false
27318 7898 Acked Read 7 Done Done Sent false
27319 7899 Acked Write 10 Done Done Sent false
27320 7900 Acked Read 7 Done Done Sent false
27321 7901 Acked Write 3 Done Done Sent false
27322 7902 Acked Write 4 Done Done Sent false
27323 7903 Acked Read 3 Done Done Sent false
27324 7904 Acked Write 9 Done Done Sent false
27325 7905 Acked Read 3 Done Done Sent false
27326 7906 Acked Flush 0 Done Done Sent false
27327 7907 Acked Read 2 Done Done Sent false
27328 7908 Acked Write 6 Done Done Sent false
27329 7909 Acked Read 9 Done Done Sent false
27330 7910 Acked Write 3 Done Done Sent false
27331 7911 Acked Write 3 Done Done Sent false
27332 7912 Acked Read 5 Done Done Sent false
27333 7913 Acked Write 3 Done Done Sent false
27334 7914 Acked Read 3 Done Done Sent false
27335 7915 Acked Write 6 Done Done Sent false
27336 7916 Acked Read 3 Done Done Sent false
27337 7917 Acked Read 8 Done Done Sent false
27338 7918 Acked Read 2 Done Done Sent false
27339 7919 Acked Read 9 Done Done Sent false
27340 7920 Acked Flush 0 Done Done Sent false
27341 7921 Acked Write 7 Done Done Sent false
27342 7922 Acked Read 5 Done Done Sent false
27343 7923 Acked Read 1 Done Done Sent false
27344 7924 Acked Write 1 Done Done Sent false
27345 7925 Acked Read 8 Done Done Sent false
27346 7926 Acked Flush 0 Done Done Sent false
27347 7927 Acked Flush 0 Done Done Sent false
27348 7928 Acked Write 8 Done Done Sent false
27349 7929 Acked Write 1 Done Done Sent false
27350 7930 Acked Read 3 Done Done Sent false
27351 7931 Acked Read 7 Done Done Sent false
27352 7932 Acked Read 3 Done Done Sent false
27353 7933 Acked Write 8 Done Done Sent false
27354 7934 Acked Write 5 Done Done Sent false
27355 7935 Acked Read 10 Done Done Sent false
27356 7936 Acked Read 5 Done Done Sent false
27357 7937 Acked Flush 0 Done Done Sent false
27358 7938 Acked Read 9 Done Done Sent false
27359 7939 Acked Write 5 Done Done Sent false
27360 7940 Acked Read 8 Done Done Sent false
27361 7941 Acked Write 7 Done Done Sent false
27362 7942 Acked Read 4 Done Done Sent false
27363 7943 Acked Read 8 Done Done Sent false
27364 7944 Acked Write 3 Done Done Sent false
27365 STATES DS:0 DS:1 DS:2 TOTAL
27366 Sent 0 0 301 301
27367 Done 301 301 0 602
27368 Skipped 0 0 0 0
27369 Error 0 0 0 0
27370 Last Flush: 7937 7937 7643
27371 Downstairs last five completed: 7643 7642 7641 7640 7639
27372 Upstairs last five completed: 7944 7943 7942 7941 7940
27373 CLIENT: Up:0 ds:301 act:2
27374 ----------------------------------------------------------------
27375 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27376 States: Active Active Offline
27377 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27378 7644 Acked Write 9 Done Done Sent false
27379 7645 Acked Write 10 Done Done Sent false
27380 7646 Acked Write 7 Done Done Sent false
27381 7647 Acked Write 4 Done Done Sent false
27382 7648 Acked Read 3 Done Done Sent false
27383 7649 Acked Flush 0 Done Done Sent false
27384 7650 Acked Write 7 Done Done Sent false
27385 7651 Acked Read 5 Done Done Sent false
27386 7652 Acked Write 8 Done Done Sent false
27387 7653 Acked Read 2 Done Done Sent false
27388 7654 Acked Write 7 Done Done Sent false
27389 7655 Acked Read 10 Done Done Sent false
27390 7656 Acked Read 6 Done Done Sent false
27391 7657 Acked Flush 0 Done Done Sent false
27392 7658 Acked Read 8 Done Done Sent false
27393 7659 Acked Write 1 Done Done Sent false
27394 7660 Acked Write 8 Done Done Sent false
27395 7661 Acked Read 6 Done Done Sent false
27396 7662 Acked Read 10 Done Done Sent false
27397 7663 Acked Read 1 Done Done Sent false
27398 7664 Acked Read 8 Done Done Sent false
27399 7665 Acked Read 9 Done Done Sent false
27400 7666 Acked Read 8 Done Done Sent false
27401 7667 Acked Read 1 Done Done Sent false
27402 7668 Acked Read 2 Done Done Sent false
27403 7669 Acked Write 5 Done Done Sent false
27404 7670 Acked Write 8 Done Done Sent false
27405 7671 Acked Read 3 Done Done Sent false
27406 7672 Acked Read 8 Done Done Sent false
27407 7673 Acked Read 6 Done Done Sent false
27408 7674 Acked Read 7 Done Done Sent false
27409 7675 Acked Flush 0 Done Done Sent false
27410 7676 Acked Write 10 Done Done Sent false
27411 7677 Acked Read 2 Done Done Sent false
27412 7678 Acked Write 4 Done Done Sent false
27413 7679 Acked Read 1 Done Done Sent false
27414 7680 Acked Write 9 Done Done Sent false
27415 7681 Acked Flush 0 Done Done Sent false
27416 7682 Acked Flush 0 Done Done Sent false
27417 7683 Acked Write 5 Done Done Sent false
27418 7684 Acked Read 10 Done Done Sent false
27419 7685 Acked Write 2 Done Done Sent false
27420 7686 Acked Flush 0 Done Done Sent false
27421 7687 Acked Read 7 Done Done Sent false
27422 7688 Acked Read 6 Done Done Sent false
27423 7689 Acked Read 6 Done Done Sent false
27424 7690 Acked Read 5 Done Done Sent false
27425 7691 Acked Read 4 Done Done Sent false
27426 7692 Acked Write 1 Done Done Sent false
27427 7693 Acked Read 9 Done Done Sent false
27428 7694 Acked Write 3 Done Done Sent false
27429 7695 Acked Read 7 Done Done Sent false
27430 7696 Acked Read 1 Done Done Sent false
27431 7697 Acked Write 1 Done Done Sent false
27432 7698 Acked Read 6 Done Done Sent false
27433 7699 Acked Write 2 Done Done Sent false
27434 7700 Acked Read 4 Done Done Sent false
27435 7701 Acked Write 10 Done Done Sent false
27436 7702 Acked Write 1 Done Done Sent false
27437 7703 Acked Write 2 Done Done Sent false
27438 7704 Acked Write 1 Done Done Sent false
27439 7705 Acked Write 9 Done Done Sent false
27440 7706 Acked Flush 0 Done Done Sent false
27441 7707 Acked Flush 0 Done Done Sent false
27442 7708 Acked Read 3 Done Done Sent false
27443 7709 Acked Write 3 Done Done Sent false
27444 7710 Acked Write 5 Done Done Sent false
27445 7711 Acked Write 10 Done Done Sent false
27446 7712 Acked Write 4 Done Done Sent false
27447 7713 Acked Read 6 Done Done Sent false
27448 7714 Acked Read 1 Done Done Sent false
27449 7715 Acked Read 7 Done Done Sent false
27450 7716 Acked Read 10 Done Done Sent false
27451 7717 Acked Read 5 Done Done Sent false
27452 7718 Acked Read 2 Done Done Sent false
27453 7719 Acked Read 2 Done Done Sent false
27454 7720 Acked Write 9 Done Done Sent false
27455 7721 Acked Write 6 Done Done Sent false
27456 7722 Acked Write 8 Done Done Sent false
27457 7723 Acked Read 5 Done Done Sent false
27458 7724 Acked Write 4 Done Done Sent false
27459 7725 Acked Read 8 Done Done Sent false
27460 7726 Acked Write 4 Done Done Sent false
27461 7727 Acked Read 9 Done Done Sent false
27462 7728 Acked Write 6 Done Done Sent false
27463 7729 Acked Flush 0 Done Done Sent false
27464 7730 Acked Write 6 Done Done Sent false
27465 7731 Acked Read 8 Done Done Sent false
27466 7732 Acked Flush 0 Done Done Sent false
27467 7733 Acked Read 5 Done Done Sent false
27468 7734 Acked Read 10 Done Done Sent false
27469 7735 Acked Flush 0 Done Done Sent false
27470 7736 Acked Read 6 Done Done Sent false
27471 7737 Acked Write 9 Done Done Sent false
27472 7738 Acked Write 1 Done Done Sent false
27473 7739 Acked Read 2 Done Done Sent false
27474 7740 Acked Read 2 Done Done Sent false
27475 7741 Acked Read 5 Done Done Sent false
27476 7742 Acked Read 7 Done Done Sent false
27477 7743 Acked Write 8 Done Done Sent false
27478 7744 Acked Write 1 Done Done Sent false
27479 7745 Acked Read 6 Done Done Sent false
27480 7746 Acked Write 2 Done Done Sent false
27481 7747 Acked Write 4 Done Done Sent false
27482 7748 Acked Read 10 Done Done Sent false
27483 7749 Acked Write 9 Done Done Sent false
27484 7750 Acked Write 1 Done Done Sent false
27485 7751 Acked Read 4 Done Done Sent false
27486 7752 Acked Read 2 Done Done Sent false
27487 7753 Acked Flush 0 Done Done Sent false
27488 7754 Acked Write 8 Done Done Sent false
27489 7755 Acked Read 4 Done Done Sent false
27490 7756 Acked Write 7 Done Done Sent false
27491 7757 Acked Read 6 Done Done Sent false
27492 7758 Acked Write 8 Done Done Sent false
27493 7759 Acked Write 8 Done Done Sent false
27494 7760 Acked Read 9 Done Done Sent false
27495 7761 Acked Read 5 Done Done Sent false
27496 7762 Acked Write 10 Done Done Sent false
27497 7763 Acked Read 5 Done Done Sent false
27498 7764 Acked Read 4 Done Done Sent false
27499 7765 Acked Read 1 Done Done Sent false
27500 7766 Acked Read 9 Done Done Sent false
27501 7767 Acked Write 10 Done Done Sent false
27502 7768 Acked Read 1 Done Done Sent false
27503 7769 Acked Read 10 Done Done Sent false
27504 7770 Acked Read 6 Done Done Sent false
27505 7771 Acked Read 10 Done Done Sent false
27506 7772 Acked Read 5 Done Done Sent false
27507 7773 Acked Write 8 Done Done Sent false
27508 7774 Acked Write 2 Done Done Sent false
27509 7775 Acked Write 9 Done Done Sent false
27510 7776 Acked Write 8 Done Done Sent false
27511 7777 Acked Write 4 Done Done Sent false
27512 7778 Acked Write 6 Done Done Sent false
27513 7779 Acked Read 9 Done Done Sent false
27514 7780 Acked Read 10 Done Done Sent false
27515 7781 Acked Flush 0 Done Done Sent false
27516 7782 Acked Write 1 Done Done Sent false
27517 7783 Acked Write 5 Done Done Sent false
27518 7784 Acked Read 8 Done Done Sent false
27519 7785 Acked Write 5 Done Done Sent false
27520 7786 Acked Read 3 Done Done Sent false
27521 7787 Acked Read 4 Done Done Sent false
27522 7788 Acked Read 4 Done Done Sent false
27523 7789 Acked Write 10 Done Done Sent false
27524 7790 Acked Flush 0 Done Done Sent false
27525 7791 Acked Read 4 Done Done Sent false
27526 7792 Acked Write 4 Done Done Sent false
27527 7793 Acked Read 7 Done Done Sent false
27528 7794 Acked Write 4 Done Done Sent false
27529 7795 Acked Read 1 Done Done Sent false
27530 7796 Acked Read 4 Done Done Sent false
27531 7797 Acked Read 1 Done Done Sent false
27532 7798 Acked Write 9 Done Done Sent false
27533 7799 Acked Write 6 Done Done Sent false
27534 7800 Acked Read 8 Done Done Sent false
27535 7801 Acked Read 10 Done Done Sent false
27536 7802 Acked Read 4 Done Done Sent false
27537 7803 Acked Write 4 Done Done Sent false
27538 7804 Acked Read 8 Done Done Sent false
27539 7805 Acked Read 3 Done Done Sent false
27540 7806 Acked Read 6 Done Done Sent false
27541 7807 Acked Write 4 Done Done Sent false
27542 7808 Acked Write 2 Done Done Sent false
27543 7809 Acked Write 6 Done Done Sent false
27544 7810 Acked Read 10 Done Done Sent false
27545 7811 Acked Write 7 Done Done Sent false
27546 7812 Acked Read 3 Done Done Sent false
27547 7813 Acked Read 1 Done Done Sent false
27548 7814 Acked Read 9 Done Done Sent false
27549 7815 Acked Read 3 Done Done Sent false
27550 7816 Acked Write 5 Done Done Sent false
27551 7817 Acked Flush 0 Done Done Sent false
27552 7818 Acked Read 1 Done Done Sent false
27553 7819 Acked Write 8 Done Done Sent false
27554 7820 Acked Read 9 Done Done Sent false
27555 7821 Acked Read 3 Done Done Sent false
27556 7822 Acked Write 6 Done Done Sent false
27557 7823 Acked Read 6 Done Done Sent false
27558 7824 Acked Write 10 Done Done Sent false
27559 7825 Acked Read 6 Done Done Sent false
27560 7826 Acked Read 10 Done Done Sent false
27561 7827 Acked Write 2 Done Done Sent false
27562 7828 Acked Read 7 Done Done Sent false
27563 7829 Acked Read 8 Done Done Sent false
27564 7830 Acked Write 10 Done Done Sent false
27565 7831 Acked Write 7 Done Done Sent false
27566 7832 Acked Read 4 Done Done Sent false
27567 7833 Acked Read 2 Done Done Sent false
27568 7834 Acked Write 2 Done Done Sent false
27569 7835 Acked Read 2 Done Done Sent false
27570 7836 Acked Flush 0 Done Done Sent false
27571 7837 Acked Write 2 Done Done Sent false
27572 7838 Acked Flush 0 Done Done Sent false
27573 7839 Acked Write 2 Done Done Sent false
27574 7840 Acked Read 9 Done Done Sent false
27575 7841 Acked Write 9 Done Done Sent false
27576 7842 Acked Write 1 Done Done Sent false
27577 7843 Acked Read 10 Done Done Sent false
27578 7844 Acked Write 4 Done Done Sent false
27579 7845 Acked Write 2 Done Done Sent false
27580 7846 Acked Write 5 Done Done Sent false
27581 7847 Acked Write 1 Done Done Sent false
27582 7848 Acked Write 3 Done Done Sent false
27583 7849 Acked Read 3 Done Done Sent false
27584 7850 Acked Write 2 Done Done Sent false
27585 7851 Acked Read 2 Done Done Sent false
27586 7852 Acked Flush 0 Done Done Sent false
27587 7853 Acked Read 8 Done Done Sent false
27588 7854 Acked Read 3 Done Done Sent false
27589 7855 Acked Write 4 Done Done Sent false
27590 7856 Acked Write 6 Done Done Sent false
27591 7857 Acked Write 3 Done Done Sent false
27592 7858 Acked Write 8 Done Done Sent false
27593 7859 Acked Read 2 Done Done Sent false
27594 7860 Acked Read 1 Done Done Sent false
27595 7861 Acked Write 1 Done Done Sent false
27596 7862 Acked Read 6 Done Done Sent false
27597 7863 Acked Read 10 Done Done Sent false
27598 7864 Acked Read 7 Done Done Sent false
27599 7865 Acked Write 7 Done Done Sent false
27600 7866 Acked Read 9 Done Done Sent false
27601 7867 Acked Flush 0 Done Done Sent false
27602 7868 Acked Read 10 Done Done Sent false
27603 7869 Acked Read 3 Done Done Sent false
27604 7870 Acked Write 4 Done Done Sent false
27605 7871 Acked Write 7 Done Done Sent false
27606 7872 Acked Write 2 Done Done Sent false
27607 7873 Acked Write 4 Done Done Sent false
27608 7874 Acked Read 2 Done Done Sent false
27609 7875 Acked Write 4 Done Done Sent false
27610 7876 Acked Read 4 Done Done Sent false
27611 7877 Acked Write 8 Done Done Sent false
27612 7878 Acked Read 2 Done Done Sent false
27613 7879 Acked Write 1 Done Done Sent false
27614 7880 Acked Write 1 Done Done Sent false
27615 7881 Acked Read 5 Done Done Sent false
27616 7882 Acked Read 5 Done Done Sent false
27617 7883 Acked Write 8 Done Done Sent false
27618 7884 Acked Read 1 Done Done Sent false
27619 7885 Acked Read 8 Done Done Sent false
27620 7886 Acked Read 10 Done Done Sent false
27621 7887 Acked Write 5 Done Done Sent false
27622 7888 Acked Write 8 Done Done Sent false
27623 7889 Acked Write 10 Done Done Sent false
27624 7890 Acked Read 9 Done Done Sent false
27625 7891 Acked Read 5 Done Done Sent false
27626 7892 Acked Read 8 Done Done Sent false
27627 7893 Acked Write 1 Done Done Sent false
27628 7894 Acked Read 3 Done Done Sent false
27629 7895 Acked Write 9 Done Done Sent false
27630 7896 Acked Flush 0 Done Done Sent false
27631 7897 Acked Write 5 Done Done Sent false
27632 7898 Acked Read 7 Done Done Sent false
27633 7899 Acked Write 10 Done Done Sent false
27634 7900 Acked Read 7 Done Done Sent false
27635 7901 Acked Write 3 Done Done Sent false
27636 7902 Acked Write 4 Done Done Sent false
27637 7903 Acked Read 3 Done Done Sent false
27638 7904 Acked Write 9 Done Done Sent false
27639 7905 Acked Read 3 Done Done Sent false
27640 7906 Acked Flush 0 Done Done Sent false
27641 7907 Acked Read 2 Done Done Sent false
27642 7908 Acked Write 6 Done Done Sent false
27643 7909 Acked Read 9 Done Done Sent false
27644 7910 Acked Write 3 Done Done Sent false
27645 7911 Acked Write 3 Done Done Sent false
27646 7912 Acked Read 5 Done Done Sent false
27647 7913 Acked Write 3 Done Done Sent false
27648 7914 Acked Read 3 Done Done Sent false
27649 7915 Acked Write 6 Done Done Sent false
27650 7916 Acked Read 3 Done Done Sent false
27651 7917 Acked Read 8 Done Done Sent false
27652 7918 Acked Read 2 Done Done Sent false
27653 7919 Acked Read 9 Done Done Sent false
27654 7920 Acked Flush 0 Done Done Sent false
27655 7921 Acked Write 7 Done Done Sent false
27656 7922 Acked Read 5 Done Done Sent false
27657 7923 Acked Read 1 Done Done Sent false
27658 7924 Acked Write 1 Done Done Sent false
27659 7925 Acked Read 8 Done Done Sent false
27660 7926 Acked Flush 0 Done Done Sent false
27661 7927 Acked Flush 0 Done Done Sent false
27662 7928 Acked Write 8 Done Done Sent false
27663 7929 Acked Write 1 Done Done Sent false
27664 7930 Acked Read 3 Done Done Sent false
27665 7931 Acked Read 7 Done Done Sent false
27666 7932 Acked Read 3 Done Done Sent false
27667 7933 Acked Write 8 Done Done Sent false
27668 7934 Acked Write 5 Done Done Sent false
27669 7935 Acked Read 10 Done Done Sent false
27670 7936 Acked Read 5 Done Done Sent false
27671 7937 Acked Flush 0 Done Done Sent false
27672 7938 Acked Read 9 Done Done Sent false
27673 7939 Acked Write 5 Done Done Sent false
27674 7940 Acked Read 8 Done Done Sent false
27675 7941 Acked Write 7 Done Done Sent false
27676 7942 Acked Read 4 Done Done Sent false
27677 7943 Acked Read 8 Done Done Sent false
27678 7944 Acked Write 3 Done Done Sent false
27679 7945 Acked Flush 0 Done Done Sent false
27680 STATES DS:0 DS:1 DS:2 TOTAL
27681 Sent 0 0 302 302
27682 Done 302 302 0 604
27683 Skipped 0 0 0 0
27684 Error 0 0 0 0
27685 Last Flush: 7945 7945 7643
27686 Downstairs last five completed: 7643 7642 7641 7640 7639
27687 Upstairs last five completed: 7945 7944 7943 7942 7941
27688 CLIENT: Up:0 ds:302 act:2
276892024-12-11T23:46:17.922ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
276902024-12-11T23:46:17.922ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
276912024-12-11T23:46:17.922ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
276922024-12-11T23:46:17.922ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
276932024-12-11T23:46:17.922ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
27694 [8830] got start action
27695 [8830] I am going to restart
27696 Starting downstairs at port 8830
27697 Make output file at "/tmp/dsc/downstairs-8830.txt"
27698 [8830][2] reports Starting
27699 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
27700 [8830][2] reports Running
27701 ----------------------------------------------------------------
27702 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27703 States: Active Active Offline
27704 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27705 7644 Acked Write 9 Done Done Sent false
27706 7645 Acked Write 10 Done Done Sent false
27707 7646 Acked Write 7 Done Done Sent false
27708 7647 Acked Write 4 Done Done Sent false
27709 7648 Acked Read 3 Done Done Sent false
27710 7649 Acked Flush 0 Done Done Sent false
27711 7650 Acked Write 7 Done Done Sent false
27712 7651 Acked Read 5 Done Done Sent false
27713 7652 Acked Write 8 Done Done Sent false
27714 7653 Acked Read 2 Done Done Sent false
27715 7654 Acked Write 7 Done Done Sent false
27716 7655 Acked Read 10 Done Done Sent false
27717 7656 Acked Read 6 Done Done Sent false
27718 7657 Acked Flush 0 Done Done Sent false
27719 7658 Acked Read 8 Done Done Sent false
27720 7659 Acked Write 1 Done Done Sent false
27721 7660 Acked Write 8 Done Done Sent false
27722 7661 Acked Read 6 Done Done Sent false
27723 7662 Acked Read 10 Done Done Sent false
27724 7663 Acked Read 1 Done Done Sent false
27725 7664 Acked Read 8 Done Done Sent false
27726 7665 Acked Read 9 Done Done Sent false
27727 7666 Acked Read 8 Done Done Sent false
27728 7667 Acked Read 1 Done Done Sent false
27729 7668 Acked Read 2 Done Done Sent false
27730 7669 Acked Write 5 Done Done Sent false
27731 7670 Acked Write 8 Done Done Sent false
27732 7671 Acked Read 3 Done Done Sent false
27733 7672 Acked Read 8 Done Done Sent false
27734 7673 Acked Read 6 Done Done Sent false
27735 7674 Acked Read 7 Done Done Sent false
27736 7675 Acked Flush 0 Done Done Sent false
27737 7676 Acked Write 10 Done Done Sent false
27738 7677 Acked Read 2 Done Done Sent false
27739 7678 Acked Write 4 Done Done Sent false
27740 7679 Acked Read 1 Done Done Sent false
27741 7680 Acked Write 9 Done Done Sent false
27742 7681 Acked Flush 0 Done Done Sent false
27743 7682 Acked Flush 0 Done Done Sent false
27744 7683 Acked Write 5 Done Done Sent false
27745 7684 Acked Read 10 Done Done Sent false
27746 7685 Acked Write 2 Done Done Sent false
27747 7686 Acked Flush 0 Done Done Sent false
27748 7687 Acked Read 7 Done Done Sent false
27749 7688 Acked Read 6 Done Done Sent false
27750 7689 Acked Read 6 Done Done Sent false
27751 7690 Acked Read 5 Done Done Sent false
27752 7691 Acked Read 4 Done Done Sent false
27753 7692 Acked Write 1 Done Done Sent false
27754 7693 Acked Read 9 Done Done Sent false
27755 7694 Acked Write 3 Done Done Sent false
27756 7695 Acked Read 7 Done Done Sent false
27757 7696 Acked Read 1 Done Done Sent false
27758 7697 Acked Write 1 Done Done Sent false
27759 7698 Acked Read 6 Done Done Sent false
27760 7699 Acked Write 2 Done Done Sent false
27761 7700 Acked Read 4 Done Done Sent false
27762 7701 Acked Write 10 Done Done Sent false
27763 7702 Acked Write 1 Done Done Sent false
27764 7703 Acked Write 2 Done Done Sent false
27765 7704 Acked Write 1 Done Done Sent false
27766 7705 Acked Write 9 Done Done Sent false
27767 7706 Acked Flush 0 Done Done Sent false
27768 7707 Acked Flush 0 Done Done Sent false
27769 7708 Acked Read 3 Done Done Sent false
27770 7709 Acked Write 3 Done Done Sent false
27771 7710 Acked Write 5 Done Done Sent false
27772 7711 Acked Write 10 Done Done Sent false
27773 7712 Acked Write 4 Done Done Sent false
27774 7713 Acked Read 6 Done Done Sent false
27775 7714 Acked Read 1 Done Done Sent false
27776 7715 Acked Read 7 Done Done Sent false
27777 7716 Acked Read 10 Done Done Sent false
27778 7717 Acked Read 5 Done Done Sent false
27779 7718 Acked Read 2 Done Done Sent false
27780 7719 Acked Read 2 Done Done Sent false
27781 7720 Acked Write 9 Done Done Sent false
27782 7721 Acked Write 6 Done Done Sent false
27783 7722 Acked Write 8 Done Done Sent false
27784 7723 Acked Read 5 Done Done Sent false
27785 7724 Acked Write 4 Done Done Sent false
27786 7725 Acked Read 8 Done Done Sent false
27787 7726 Acked Write 4 Done Done Sent false
27788 7727 Acked Read 9 Done Done Sent false
27789 7728 Acked Write 6 Done Done Sent false
27790 7729 Acked Flush 0 Done Done Sent false
27791 7730 Acked Write 6 Done Done Sent false
27792 7731 Acked Read 8 Done Done Sent false
27793 7732 Acked Flush 0 Done Done Sent false
27794 7733 Acked Read 5 Done Done Sent false
27795 7734 Acked Read 10 Done Done Sent false
27796 7735 Acked Flush 0 Done Done Sent false
27797 7736 Acked Read 6 Done Done Sent false
27798 7737 Acked Write 9 Done Done Sent false
27799 7738 Acked Write 1 Done Done Sent false
27800 7739 Acked Read 2 Done Done Sent false
27801 7740 Acked Read 2 Done Done Sent false
27802 7741 Acked Read 5 Done Done Sent false
27803 7742 Acked Read 7 Done Done Sent false
27804 7743 Acked Write 8 Done Done Sent false
27805 7744 Acked Write 1 Done Done Sent false
27806 7745 Acked Read 6 Done Done Sent false
27807 7746 Acked Write 2 Done Done Sent false
27808 7747 Acked Write 4 Done Done Sent false
27809 7748 Acked Read 10 Done Done Sent false
27810 7749 Acked Write 9 Done Done Sent false
27811 7750 Acked Write 1 Done Done Sent false
27812 7751 Acked Read 4 Done Done Sent false
27813 7752 Acked Read 2 Done Done Sent false
27814 7753 Acked Flush 0 Done Done Sent false
27815 7754 Acked Write 8 Done Done Sent false
27816 7755 Acked Read 4 Done Done Sent false
27817 7756 Acked Write 7 Done Done Sent false
27818 7757 Acked Read 6 Done Done Sent false
27819 7758 Acked Write 8 Done Done Sent false
27820 7759 Acked Write 8 Done Done Sent false
27821 7760 Acked Read 9 Done Done Sent false
27822 7761 Acked Read 5 Done Done Sent false
27823 7762 Acked Write 10 Done Done Sent false
27824 7763 Acked Read 5 Done Done Sent false
27825 7764 Acked Read 4 Done Done Sent false
27826 7765 Acked Read 1 Done Done Sent false
27827 7766 Acked Read 9 Done Done Sent false
27828 7767 Acked Write 10 Done Done Sent false
27829 7768 Acked Read 1 Done Done Sent false
27830 7769 Acked Read 10 Done Done Sent false
27831 7770 Acked Read 6 Done Done Sent false
27832 7771 Acked Read 10 Done Done Sent false
27833 7772 Acked Read 5 Done Done Sent false
27834 7773 Acked Write 8 Done Done Sent false
27835 7774 Acked Write 2 Done Done Sent false
27836 7775 Acked Write 9 Done Done Sent false
27837 7776 Acked Write 8 Done Done Sent false
27838 7777 Acked Write 4 Done Done Sent false
27839 7778 Acked Write 6 Done Done Sent false
27840 7779 Acked Read 9 Done Done Sent false
27841 7780 Acked Read 10 Done Done Sent false
27842 7781 Acked Flush 0 Done Done Sent false
27843 7782 Acked Write 1 Done Done Sent false
27844 7783 Acked Write 5 Done Done Sent false
27845 7784 Acked Read 8 Done Done Sent false
27846 7785 Acked Write 5 Done Done Sent false
27847 7786 Acked Read 3 Done Done Sent false
27848 7787 Acked Read 4 Done Done Sent false
27849 7788 Acked Read 4 Done Done Sent false
27850 7789 Acked Write 10 Done Done Sent false
27851 7790 Acked Flush 0 Done Done Sent false
27852 7791 Acked Read 4 Done Done Sent false
27853 7792 Acked Write 4 Done Done Sent false
27854 7793 Acked Read 7 Done Done Sent false
27855 7794 Acked Write 4 Done Done Sent false
27856 7795 Acked Read 1 Done Done Sent false
27857 7796 Acked Read 4 Done Done Sent false
27858 7797 Acked Read 1 Done Done Sent false
27859 7798 Acked Write 9 Done Done Sent false
27860 7799 Acked Write 6 Done Done Sent false
27861 7800 Acked Read 8 Done Done Sent false
27862 7801 Acked Read 10 Done Done Sent false
27863 7802 Acked Read 4 Done Done Sent false
27864 7803 Acked Write 4 Done Done Sent false
27865 7804 Acked Read 8 Done Done Sent false
27866 7805 Acked Read 3 Done Done Sent false
27867 7806 Acked Read 6 Done Done Sent false
27868 7807 Acked Write 4 Done Done Sent false
27869 7808 Acked Write 2 Done Done Sent false
27870 7809 Acked Write 6 Done Done Sent false
27871 7810 Acked Read 10 Done Done Sent false
27872 7811 Acked Write 7 Done Done Sent false
27873 7812 Acked Read 3 Done Done Sent false
27874 7813 Acked Read 1 Done Done Sent false
27875 7814 Acked Read 9 Done Done Sent false
27876 7815 Acked Read 3 Done Done Sent false
27877 7816 Acked Write 5 Done Done Sent false
27878 7817 Acked Flush 0 Done Done Sent false
27879 7818 Acked Read 1 Done Done Sent false
27880 7819 Acked Write 8 Done Done Sent false
27881 7820 Acked Read 9 Done Done Sent false
27882 7821 Acked Read 3 Done Done Sent false
27883 7822 Acked Write 6 Done Done Sent false
27884 7823 Acked Read 6 Done Done Sent false
27885 7824 Acked Write 10 Done Done Sent false
27886 7825 Acked Read 6 Done Done Sent false
27887 7826 Acked Read 10 Done Done Sent false
27888 7827 Acked Write 2 Done Done Sent false
27889 7828 Acked Read 7 Done Done Sent false
27890 7829 Acked Read 8 Done Done Sent false
27891 7830 Acked Write 10 Done Done Sent false
27892 7831 Acked Write 7 Done Done Sent false
27893 7832 Acked Read 4 Done Done Sent false
27894 7833 Acked Read 2 Done Done Sent false
27895 7834 Acked Write 2 Done Done Sent false
27896 7835 Acked Read 2 Done Done Sent false
27897 7836 Acked Flush 0 Done Done Sent false
27898 7837 Acked Write 2 Done Done Sent false
27899 7838 Acked Flush 0 Done Done Sent false
27900 7839 Acked Write 2 Done Done Sent false
27901 7840 Acked Read 9 Done Done Sent false
27902 7841 Acked Write 9 Done Done Sent false
27903 7842 Acked Write 1 Done Done Sent false
27904 7843 Acked Read 10 Done Done Sent false
27905 7844 Acked Write 4 Done Done Sent false
27906 7845 Acked Write 2 Done Done Sent false
27907 7846 Acked Write 5 Done Done Sent false
27908 7847 Acked Write 1 Done Done Sent false
27909 7848 Acked Write 3 Done Done Sent false
27910 7849 Acked Read 3 Done Done Sent false
27911 7850 Acked Write 2 Done Done Sent false
27912 7851 Acked Read 2 Done Done Sent false
27913 7852 Acked Flush 0 Done Done Sent false
27914 7853 Acked Read 8 Done Done Sent false
27915 7854 Acked Read 3 Done Done Sent false
27916 7855 Acked Write 4 Done Done Sent false
27917 7856 Acked Write 6 Done Done Sent false
27918 7857 Acked Write 3 Done Done Sent false
27919 7858 Acked Write 8 Done Done Sent false
27920 7859 Acked Read 2 Done Done Sent false
27921 7860 Acked Read 1 Done Done Sent false
27922 7861 Acked Write 1 Done Done Sent false
27923 7862 Acked Read 6 Done Done Sent false
27924 7863 Acked Read 10 Done Done Sent false
27925 7864 Acked Read 7 Done Done Sent false
27926 7865 Acked Write 7 Done Done Sent false
27927 7866 Acked Read 9 Done Done Sent false
27928 7867 Acked Flush 0 Done Done Sent false
27929 7868 Acked Read 10 Done Done Sent false
27930 7869 Acked Read 3 Done Done Sent false
27931 7870 Acked Write 4 Done Done Sent false
27932 7871 Acked Write 7 Done Done Sent false
27933 7872 Acked Write 2 Done Done Sent false
27934 7873 Acked Write 4 Done Done Sent false
27935 7874 Acked Read 2 Done Done Sent false
27936 7875 Acked Write 4 Done Done Sent false
27937 7876 Acked Read 4 Done Done Sent false
27938 7877 Acked Write 8 Done Done Sent false
27939 7878 Acked Read 2 Done Done Sent false
27940 7879 Acked Write 1 Done Done Sent false
27941 7880 Acked Write 1 Done Done Sent false
27942 7881 Acked Read 5 Done Done Sent false
27943 7882 Acked Read 5 Done Done Sent false
27944 7883 Acked Write 8 Done Done Sent false
27945 7884 Acked Read 1 Done Done Sent false
27946 7885 Acked Read 8 Done Done Sent false
27947 7886 Acked Read 10 Done Done Sent false
27948 7887 Acked Write 5 Done Done Sent false
27949 7888 Acked Write 8 Done Done Sent false
27950 7889 Acked Write 10 Done Done Sent false
27951 7890 Acked Read 9 Done Done Sent false
27952 7891 Acked Read 5 Done Done Sent false
27953 7892 Acked Read 8 Done Done Sent false
27954 7893 Acked Write 1 Done Done Sent false
27955 7894 Acked Read 3 Done Done Sent false
27956 7895 Acked Write 9 Done Done Sent false
27957 7896 Acked Flush 0 Done Done Sent false
27958 7897 Acked Write 5 Done Done Sent false
27959 7898 Acked Read 7 Done Done Sent false
27960 7899 Acked Write 10 Done Done Sent false
27961 7900 Acked Read 7 Done Done Sent false
27962 7901 Acked Write 3 Done Done Sent false
27963 7902 Acked Write 4 Done Done Sent false
27964 7903 Acked Read 3 Done Done Sent false
27965 7904 Acked Write 9 Done Done Sent false
27966 7905 Acked Read 3 Done Done Sent false
27967 7906 Acked Flush 0 Done Done Sent false
27968 7907 Acked Read 2 Done Done Sent false
27969 7908 Acked Write 6 Done Done Sent false
27970 7909 Acked Read 9 Done Done Sent false
27971 7910 Acked Write 3 Done Done Sent false
27972 7911 Acked Write 3 Done Done Sent false
27973 7912 Acked Read 5 Done Done Sent false
27974 7913 Acked Write 3 Done Done Sent false
27975 7914 Acked Read 3 Done Done Sent false
27976 7915 Acked Write 6 Done Done Sent false
27977 7916 Acked Read 3 Done Done Sent false
27978 7917 Acked Read 8 Done Done Sent false
27979 7918 Acked Read 2 Done Done Sent false
27980 7919 Acked Read 9 Done Done Sent false
27981 7920 Acked Flush 0 Done Done Sent false
27982 7921 Acked Write 7 Done Done Sent false
27983 7922 Acked Read 5 Done Done Sent false
27984 7923 Acked Read 1 Done Done Sent false
27985 7924 Acked Write 1 Done Done Sent false
27986 7925 Acked Read 8 Done Done Sent false
27987 7926 Acked Flush 0 Done Done Sent false
27988 7927 Acked Flush 0 Done Done Sent false
27989 7928 Acked Write 8 Done Done Sent false
27990 7929 Acked Write 1 Done Done Sent false
27991 7930 Acked Read 3 Done Done Sent false
27992 7931 Acked Read 7 Done Done Sent false
27993 7932 Acked Read 3 Done Done Sent false
27994 7933 Acked Write 8 Done Done Sent false
27995 7934 Acked Write 5 Done Done Sent false
27996 7935 Acked Read 10 Done Done Sent false
27997 7936 Acked Read 5 Done Done Sent false
27998 7937 Acked Flush 0 Done Done Sent false
27999 7938 Acked Read 9 Done Done Sent false
28000 7939 Acked Write 5 Done Done Sent false
28001 7940 Acked Read 8 Done Done Sent false
28002 7941 Acked Write 7 Done Done Sent false
28003 7942 Acked Read 4 Done Done Sent false
28004 7943 Acked Read 8 Done Done Sent false
28005 7944 Acked Write 3 Done Done Sent false
28006 7945 Acked Flush 0 Done Done Sent false
28007 STATES DS:0 DS:1 DS:2 TOTAL
28008 Sent 0 0 302 302
28009 Done 302 302 0 604
28010 Skipped 0 0 0 0
28011 Error 0 0 0 0
28012 Last Flush: 7945 7945 7643
28013 Downstairs last five completed: 7643 7642 7641 7640 7639
28014 Upstairs last five completed: 7945 7944 7943 7942 7941
28015 CLIENT: Up:0 ds:302 act:2
28016 ----------------------------------------------------------------
28017 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28018 States: Active Active Offline
28019 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28020 7644 Acked Write 9 Done Done Sent false
28021 7645 Acked Write 10 Done Done Sent false
28022 7646 Acked Write 7 Done Done Sent false
28023 7647 Acked Write 4 Done Done Sent false
28024 7648 Acked Read 3 Done Done Sent false
28025 7649 Acked Flush 0 Done Done Sent false
28026 7650 Acked Write 7 Done Done Sent false
28027 7651 Acked Read 5 Done Done Sent false
28028 7652 Acked Write 8 Done Done Sent false
28029 7653 Acked Read 2 Done Done Sent false
28030 7654 Acked Write 7 Done Done Sent false
28031 7655 Acked Read 10 Done Done Sent false
28032 7656 Acked Read 6 Done Done Sent false
28033 7657 Acked Flush 0 Done Done Sent false
28034 7658 Acked Read 8 Done Done Sent false
28035 7659 Acked Write 1 Done Done Sent false
28036 7660 Acked Write 8 Done Done Sent false
28037 7661 Acked Read 6 Done Done Sent false
28038 7662 Acked Read 10 Done Done Sent false
28039 7663 Acked Read 1 Done Done Sent false
28040 7664 Acked Read 8 Done Done Sent false
28041 7665 Acked Read 9 Done Done Sent false
28042 7666 Acked Read 8 Done Done Sent false
28043 7667 Acked Read 1 Done Done Sent false
28044 7668 Acked Read 2 Done Done Sent false
28045 7669 Acked Write 5 Done Done Sent false
28046 7670 Acked Write 8 Done Done Sent false
28047 7671 Acked Read 3 Done Done Sent false
28048 7672 Acked Read 8 Done Done Sent false
28049 7673 Acked Read 6 Done Done Sent false
28050 7674 Acked Read 7 Done Done Sent false
28051 7675 Acked Flush 0 Done Done Sent false
28052 7676 Acked Write 10 Done Done Sent false
28053 7677 Acked Read 2 Done Done Sent false
28054 7678 Acked Write 4 Done Done Sent false
28055 7679 Acked Read 1 Done Done Sent false
28056 7680 Acked Write 9 Done Done Sent false
28057 7681 Acked Flush 0 Done Done Sent false
28058 7682 Acked Flush 0 Done Done Sent false
28059 7683 Acked Write 5 Done Done Sent false
28060 7684 Acked Read 10 Done Done Sent false
28061 7685 Acked Write 2 Done Done Sent false
28062 7686 Acked Flush 0 Done Done Sent false
28063 7687 Acked Read 7 Done Done Sent false
28064 7688 Acked Read 6 Done Done Sent false
28065 7689 Acked Read 6 Done Done Sent false
28066 7690 Acked Read 5 Done Done Sent false
28067 7691 Acked Read 4 Done Done Sent false
28068 7692 Acked Write 1 Done Done Sent false
28069 7693 Acked Read 9 Done Done Sent false
28070 7694 Acked Write 3 Done Done Sent false
28071 7695 Acked Read 7 Done Done Sent false
28072 7696 Acked Read 1 Done Done Sent false
28073 7697 Acked Write 1 Done Done Sent false
28074 7698 Acked Read 6 Done Done Sent false
28075 7699 Acked Write 2 Done Done Sent false
28076 7700 Acked Read 4 Done Done Sent false
28077 7701 Acked Write 10 Done Done Sent false
28078 7702 Acked Write 1 Done Done Sent false
28079 7703 Acked Write 2 Done Done Sent false
28080 7704 Acked Write 1 Done Done Sent false
28081 7705 Acked Write 9 Done Done Sent false
28082 7706 Acked Flush 0 Done Done Sent false
28083 7707 Acked Flush 0 Done Done Sent false
28084 7708 Acked Read 3 Done Done Sent false
28085 7709 Acked Write 3 Done Done Sent false
28086 7710 Acked Write 5 Done Done Sent false
28087 7711 Acked Write 10 Done Done Sent false
28088 7712 Acked Write 4 Done Done Sent false
28089 7713 Acked Read 6 Done Done Sent false
28090 7714 Acked Read 1 Done Done Sent false
28091 7715 Acked Read 7 Done Done Sent false
28092 7716 Acked Read 10 Done Done Sent false
28093 7717 Acked Read 5 Done Done Sent false
28094 7718 Acked Read 2 Done Done Sent false
28095 7719 Acked Read 2 Done Done Sent false
28096 7720 Acked Write 9 Done Done Sent false
28097 7721 Acked Write 6 Done Done Sent false
28098 7722 Acked Write 8 Done Done Sent false
28099 7723 Acked Read 5 Done Done Sent false
28100 7724 Acked Write 4 Done Done Sent false
28101 7725 Acked Read 8 Done Done Sent false
28102 7726 Acked Write 4 Done Done Sent false
28103 7727 Acked Read 9 Done Done Sent false
28104 7728 Acked Write 6 Done Done Sent false
28105 7729 Acked Flush 0 Done Done Sent false
28106 7730 Acked Write 6 Done Done Sent false
28107 7731 Acked Read 8 Done Done Sent false
28108 7732 Acked Flush 0 Done Done Sent false
28109 7733 Acked Read 5 Done Done Sent false
28110 7734 Acked Read 10 Done Done Sent false
28111 7735 Acked Flush 0 Done Done Sent false
28112 7736 Acked Read 6 Done Done Sent false
28113 7737 Acked Write 9 Done Done Sent false
28114 7738 Acked Write 1 Done Done Sent false
28115 7739 Acked Read 2 Done Done Sent false
28116 7740 Acked Read 2 Done Done Sent false
28117 7741 Acked Read 5 Done Done Sent false
28118 7742 Acked Read 7 Done Done Sent false
28119 7743 Acked Write 8 Done Done Sent false
28120 7744 Acked Write 1 Done Done Sent false
28121 7745 Acked Read 6 Done Done Sent false
28122 7746 Acked Write 2 Done Done Sent false
28123 7747 Acked Write 4 Done Done Sent false
28124 7748 Acked Read 10 Done Done Sent false
28125 7749 Acked Write 9 Done Done Sent false
28126 7750 Acked Write 1 Done Done Sent false
28127 7751 Acked Read 4 Done Done Sent false
28128 7752 Acked Read 2 Done Done Sent false
28129 7753 Acked Flush 0 Done Done Sent false
28130 7754 Acked Write 8 Done Done Sent false
28131 7755 Acked Read 4 Done Done Sent false
28132 7756 Acked Write 7 Done Done Sent false
28133 7757 Acked Read 6 Done Done Sent false
28134 7758 Acked Write 8 Done Done Sent false
28135 7759 Acked Write 8 Done Done Sent false
28136 7760 Acked Read 9 Done Done Sent false
28137 7761 Acked Read 5 Done Done Sent false
28138 7762 Acked Write 10 Done Done Sent false
28139 7763 Acked Read 5 Done Done Sent false
28140 7764 Acked Read 4 Done Done Sent false
28141 7765 Acked Read 1 Done Done Sent false
28142 7766 Acked Read 9 Done Done Sent false
28143 7767 Acked Write 10 Done Done Sent false
28144 7768 Acked Read 1 Done Done Sent false
28145 7769 Acked Read 10 Done Done Sent false
28146 7770 Acked Read 6 Done Done Sent false
28147 7771 Acked Read 10 Done Done Sent false
28148 7772 Acked Read 5 Done Done Sent false
28149 7773 Acked Write 8 Done Done Sent false
28150 7774 Acked Write 2 Done Done Sent false
28151 7775 Acked Write 9 Done Done Sent false
28152 7776 Acked Write 8 Done Done Sent false
28153 7777 Acked Write 4 Done Done Sent false
28154 7778 Acked Write 6 Done Done Sent false
28155 7779 Acked Read 9 Done Done Sent false
28156 7780 Acked Read 10 Done Done Sent false
28157 7781 Acked Flush 0 Done Done Sent false
28158 7782 Acked Write 1 Done Done Sent false
28159 7783 Acked Write 5 Done Done Sent false
28160 7784 Acked Read 8 Done Done Sent false
28161 7785 Acked Write 5 Done Done Sent false
28162 7786 Acked Read 3 Done Done Sent false
28163 7787 Acked Read 4 Done Done Sent false
28164 7788 Acked Read 4 Done Done Sent false
28165 7789 Acked Write 10 Done Done Sent false
28166 7790 Acked Flush 0 Done Done Sent false
28167 7791 Acked Read 4 Done Done Sent false
28168 7792 Acked Write 4 Done Done Sent false
28169 7793 Acked Read 7 Done Done Sent false
28170 7794 Acked Write 4 Done Done Sent false
28171 7795 Acked Read 1 Done Done Sent false
28172 7796 Acked Read 4 Done Done Sent false
28173 7797 Acked Read 1 Done Done Sent false
28174 7798 Acked Write 9 Done Done Sent false
28175 7799 Acked Write 6 Done Done Sent false
28176 7800 Acked Read 8 Done Done Sent false
28177 7801 Acked Read 10 Done Done Sent false
28178 7802 Acked Read 4 Done Done Sent false
28179 7803 Acked Write 4 Done Done Sent false
28180 7804 Acked Read 8 Done Done Sent false
28181 7805 Acked Read 3 Done Done Sent false
28182 7806 Acked Read 6 Done Done Sent false
28183 7807 Acked Write 4 Done Done Sent false
28184 7808 Acked Write 2 Done Done Sent false
28185 7809 Acked Write 6 Done Done Sent false
28186 7810 Acked Read 10 Done Done Sent false
28187 7811 Acked Write 7 Done Done Sent false
28188 7812 Acked Read 3 Done Done Sent false
28189 7813 Acked Read 1 Done Done Sent false
28190 7814 Acked Read 9 Done Done Sent false
28191 7815 Acked Read 3 Done Done Sent false
28192 7816 Acked Write 5 Done Done Sent false
28193 7817 Acked Flush 0 Done Done Sent false
28194 7818 Acked Read 1 Done Done Sent false
28195 7819 Acked Write 8 Done Done Sent false
28196 7820 Acked Read 9 Done Done Sent false
28197 7821 Acked Read 3 Done Done Sent false
28198 7822 Acked Write 6 Done Done Sent false
28199 7823 Acked Read 6 Done Done Sent false
28200 7824 Acked Write 10 Done Done Sent false
28201 7825 Acked Read 6 Done Done Sent false
28202 7826 Acked Read 10 Done Done Sent false
28203 7827 Acked Write 2 Done Done Sent false
28204 7828 Acked Read 7 Done Done Sent false
28205 7829 Acked Read 8 Done Done Sent false
28206 7830 Acked Write 10 Done Done Sent false
28207 7831 Acked Write 7 Done Done Sent false
28208 7832 Acked Read 4 Done Done Sent false
28209 7833 Acked Read 2 Done Done Sent false
28210 7834 Acked Write 2 Done Done Sent false
28211 7835 Acked Read 2 Done Done Sent false
28212 7836 Acked Flush 0 Done Done Sent false
28213 7837 Acked Write 2 Done Done Sent false
28214 7838 Acked Flush 0 Done Done Sent false
28215 7839 Acked Write 2 Done Done Sent false
28216 7840 Acked Read 9 Done Done Sent false
28217 7841 Acked Write 9 Done Done Sent false
28218 7842 Acked Write 1 Done Done Sent false
28219 7843 Acked Read 10 Done Done Sent false
28220 7844 Acked Write 4 Done Done Sent false
28221 7845 Acked Write 2 Done Done Sent false
28222 7846 Acked Write 5 Done Done Sent false
28223 7847 Acked Write 1 Done Done Sent false
28224 7848 Acked Write 3 Done Done Sent false
28225 7849 Acked Read 3 Done Done Sent false
28226 7850 Acked Write 2 Done Done Sent false
28227 7851 Acked Read 2 Done Done Sent false
28228 7852 Acked Flush 0 Done Done Sent false
28229 7853 Acked Read 8 Done Done Sent false
28230 7854 Acked Read 3 Done Done Sent false
28231 7855 Acked Write 4 Done Done Sent false
28232 7856 Acked Write 6 Done Done Sent false
28233 7857 Acked Write 3 Done Done Sent false
28234 7858 Acked Write 8 Done Done Sent false
28235 7859 Acked Read 2 Done Done Sent false
28236 7860 Acked Read 1 Done Done Sent false
28237 7861 Acked Write 1 Done Done Sent false
28238 7862 Acked Read 6 Done Done Sent false
28239 7863 Acked Read 10 Done Done Sent false
28240 7864 Acked Read 7 Done Done Sent false
28241 7865 Acked Write 7 Done Done Sent false
28242 7866 Acked Read 9 Done Done Sent false
28243 7867 Acked Flush 0 Done Done Sent false
28244 7868 Acked Read 10 Done Done Sent false
28245 7869 Acked Read 3 Done Done Sent false
28246 7870 Acked Write 4 Done Done Sent false
28247 7871 Acked Write 7 Done Done Sent false
28248 7872 Acked Write 2 Done Done Sent false
28249 7873 Acked Write 4 Done Done Sent false
28250 7874 Acked Read 2 Done Done Sent false
28251 7875 Acked Write 4 Done Done Sent false
28252 7876 Acked Read 4 Done Done Sent false
28253 7877 Acked Write 8 Done Done Sent false
28254 7878 Acked Read 2 Done Done Sent false
28255 7879 Acked Write 1 Done Done Sent false
28256 7880 Acked Write 1 Done Done Sent false
28257 7881 Acked Read 5 Done Done Sent false
28258 7882 Acked Read 5 Done Done Sent false
28259 7883 Acked Write 8 Done Done Sent false
28260 7884 Acked Read 1 Done Done Sent false
28261 7885 Acked Read 8 Done Done Sent false
28262 7886 Acked Read 10 Done Done Sent false
28263 7887 Acked Write 5 Done Done Sent false
28264 7888 Acked Write 8 Done Done Sent false
28265 7889 Acked Write 10 Done Done Sent false
28266 7890 Acked Read 9 Done Done Sent false
28267 7891 Acked Read 5 Done Done Sent false
28268 7892 Acked Read 8 Done Done Sent false
28269 7893 Acked Write 1 Done Done Sent false
28270 7894 Acked Read 3 Done Done Sent false
28271 7895 Acked Write 9 Done Done Sent false
28272 7896 Acked Flush 0 Done Done Sent false
28273 7897 Acked Write 5 Done Done Sent false
28274 7898 Acked Read 7 Done Done Sent false
28275 7899 Acked Write 10 Done Done Sent false
28276 7900 Acked Read 7 Done Done Sent false
28277 7901 Acked Write 3 Done Done Sent false
28278 7902 Acked Write 4 Done Done Sent false
28279 7903 Acked Read 3 Done Done Sent false
28280 7904 Acked Write 9 Done Done Sent false
28281 7905 Acked Read 3 Done Done Sent false
28282 7906 Acked Flush 0 Done Done Sent false
28283 7907 Acked Read 2 Done Done Sent false
28284 7908 Acked Write 6 Done Done Sent false
28285 7909 Acked Read 9 Done Done Sent false
28286 7910 Acked Write 3 Done Done Sent false
28287 7911 Acked Write 3 Done Done Sent false
28288 7912 Acked Read 5 Done Done Sent false
28289 7913 Acked Write 3 Done Done Sent false
28290 7914 Acked Read 3 Done Done Sent false
28291 7915 Acked Write 6 Done Done Sent false
28292 7916 Acked Read 3 Done Done Sent false
28293 7917 Acked Read 8 Done Done Sent false
28294 7918 Acked Read 2 Done Done Sent false
28295 7919 Acked Read 9 Done Done Sent false
28296 7920 Acked Flush 0 Done Done Sent false
28297 7921 Acked Write 7 Done Done Sent false
28298 7922 Acked Read 5 Done Done Sent false
28299 7923 Acked Read 1 Done Done Sent false
28300 7924 Acked Write 1 Done Done Sent false
28301 7925 Acked Read 8 Done Done Sent false
28302 7926 Acked Flush 0 Done Done Sent false
28303 7927 Acked Flush 0 Done Done Sent false
28304 7928 Acked Write 8 Done Done Sent false
28305 7929 Acked Write 1 Done Done Sent false
28306 7930 Acked Read 3 Done Done Sent false
28307 7931 Acked Read 7 Done Done Sent false
28308 7932 Acked Read 3 Done Done Sent false
28309 7933 Acked Write 8 Done Done Sent false
28310 7934 Acked Write 5 Done Done Sent false
28311 7935 Acked Read 10 Done Done Sent false
28312 7936 Acked Read 5 Done Done Sent false
28313 7937 Acked Flush 0 Done Done Sent false
28314 7938 Acked Read 9 Done Done Sent false
28315 7939 Acked Write 5 Done Done Sent false
28316 7940 Acked Read 8 Done Done Sent false
28317 7941 Acked Write 7 Done Done Sent false
28318 7942 Acked Read 4 Done Done Sent false
28319 7943 Acked Read 8 Done Done Sent false
28320 7944 Acked Write 3 Done Done Sent false
28321 7945 Acked Flush 0 Done Done Sent false
28322 STATES DS:0 DS:1 DS:2 TOTAL
28323 Sent 0 0 302 302
28324 Done 302 302 0 604
28325 Skipped 0 0 0 0
28326 Error 0 0 0 0
28327 Last Flush: 7945 7945 7643
28328 Downstairs last five completed: 7643 7642 7641 7640 7639
28329 Upstairs last five completed: 7945 7944 7943 7942 7941
28330 CLIENT: Up:0 ds:302 act:2
283312024-12-11T23:46:27.924ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283322024-12-11T23:46:27.924ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64949) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283332024-12-11T23:46:27.925ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283342024-12-11T23:46:27.926ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283352024-12-11T23:46:27.926ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283362024-12-11T23:46:27.926ZINFOcrucible: send last flush ID to this DS: 7643 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283372024-12-11T23:46:27.926ZINFOcrucible: Replied this last flush ID: 7643 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283382024-12-11T23:46:27.926ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283392024-12-11T23:46:27.926ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283402024-12-11T23:46:27.926ZINFOcrucible: [2] client re-new 302 jobs since flush 7643 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283412024-12-11T23:46:27.926ZINFOcrucible: [2] Replayed 302 jobs since flush: 7643 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
28342 ----------------------------------------------------------------
28343 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28344 Downstairs last five completed: 7945 7944 7943 7942 7941
28345 Upstairs last five completed: 7945 7944 7943 7942 7941
28346 CLIENT: Up:0 ds:0 act:3
28347 Replay: All jobs finished
28348 Main task has work to do, go find it
28349 got dsc Stop(1)
28350 stop 1
28351 [8820] Got stop action so:false kr:false
28352 Dec 11 23:46:28.169 INFO request completed, latency_us: 326, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 6f7f8d68-e84d-447a-aee1-75c0731f0cf1, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
283532024-12-11T23:46:28.169ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283542024-12-11T23:46:28.169ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283552024-12-11T23:46:28.169ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283562024-12-11T23:46:28.170ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283572024-12-11T23:46:28.170ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
283582024-12-11T23:46:28.170ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
28359 Dec 11 23:46:28.170 INFO request completed, latency_us: 235, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3c3eb953-d9dc-4c9e-a31b-eaf11d2b0ebe, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
28360 [8820] Exited with: ExitStatus(unix_wait_status(9))
28361 [8820][1] reports Exit
28362 Dec 11 23:46:32.172 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1976cd86-6baa-4202-b298-78993925e051, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
28363 001/300 Write block 4577 len 4096 data: 4
28364 002/300 Write block 4831 len 12288 data: 4 5 5
28365 003/300 Read block 2431 len 28672
28366 004/300 Write block 1011 len 4096 data: 6
28367 005/300 Read block 4446 len 32768
28368 006/300 Write block 3529 len 28672 data: 6 6 5 6 4 4 4
28369 007/300 Read block 4169 len 28672
28370 008/300 Read block 4385 len 4096
28371 009/300 Read block 2380 len 32768
28372 010/300 Write block 1727 len 4096 data: 4
28373 011/300 Flush
28374 012/300 Write block 2030 len 4096 data: 6
28375 013/300 Read block 1264 len 16384
28376 014/300 Read block 581 len 4096
28377 015/300 Read block 497 len 16384
28378 016/300 Read block 2679 len 4096
28379 017/300 Read block 1818 len 16384
28380 018/300 Read block 4540 len 8192
28381 019/300 Write block 3176 len 12288 data: 6 6 5
28382 020/300 Read block 4924 len 40960
28383 021/300 Read block 4510 len 36864
28384 022/300 Write block 2468 len 4096 data: 6
28385 023/300 Write block 3413 len 8192 data: 3 4
28386 024/300 Write block 510 len 24576 data: 6 6 6 5 5 5
28387 025/300 Write block 4841 len 20480 data: 5 4 5 5 5
28388 026/300 Write block 1200 len 8192 data: 5 4
28389 027/300 Read block 4284 len 8192
28390 028/300 Write block 4626 len 36864 data: 3 3 3 5 4 6 5 5 5
28391 029/300 Read block 4472 len 12288
28392 030/300 Write block 3615 len 40960 data: 4 6 7 6 7 7 7 5 5 3
28393 031/300 Read block 4687 len 20480
28394 032/300 Write block 2824 len 40960 data: 8 6 5 6 7 7 7 8 7 8
28395 033/300 Write block 2923 len 40960 data: 5 5 4 5 5 4 4 4 7 8
28396 034/300 Write block 897 len 40960 data: 4 5 5 5 5 4 4 3 3 4
28397 035/300 Read block 1420 len 12288
28398 036/300 Flush
28399 037/300 Write block 3741 len 36864 data: 7 8 7 6 5 5 4 3 4
28400 038/300 Read block 2581 len 4096
28401 039/300 Flush
28402 040/300 Flush
28403 041/300 Read block 1203 len 24576
28404 042/300 Write block 3636 len 24576 data: 4 3 5 4 5 5
28405 043/300 Write block 4722 len 32768 data: 8 10 8 6 6 6 5 5
28406 044/300 Write block 23 len 36864 data: 5 5 6 6 5 4 4 5 4
28407 045/300 Write block 4680 len 12288 data: 8 7 5
28408 046/300 Read block 4515 len 8192
28409 047/300 Read block 1627 len 40960
28410 048/300 Write block 853 len 8192 data: 3 5
28411 049/300 Write block 4408 len 8192 data: 5 4
28412 050/300 Read block 2382 len 12288
28413 051/300 Flush
28414 052/300 Write block 973 len 8192 data: 7 7
28415 053/300 Read block 4226 len 8192
28416 054/300 Flush
28417 055/300 Write block 1834 len 36864 data: 6 6 7 7 7 6 6 5 7
28418 056/300 Write block 1736 len 4096 data: 3
28419 057/300 Read block 4471 len 8192
28420 058/300 Write block 1229 len 4096 data: 6
28421 059/300 Flush
28422 060/300 Write block 1522 len 36864 data: 7 6 5 5 4 4 3 3 3
28423 061/300 Write block 3442 len 40960 data: 5 4 3 5 3 2 3 5 7 6
28424 062/300 Read block 1937 len 36864
28425 063/300 Write block 3072 len 4096 data: 4
28426 064/300 Write block 4084 len 32768 data: 4 4 4 5 5 4 5 6
28427 065/300 Flush
28428 066/300 Read block 1239 len 40960
28429 067/300 Flush
28430 068/300 Read block 2479 len 28672
28431 069/300 Write block 681 len 12288 data: 3 3 4
28432 070/300 Read block 243 len 12288
28433 071/300 Write block 3490 len 20480 data: 8 7 6 6 6
28434 072/300 Write block 2946 len 24576 data: 6 6 5 5 5 5
28435 073/300 Read block 924 len 36864
28436 074/300 Read block 4893 len 24576
28437 075/300 Read block 4061 len 40960
28438 076/300 Read block 3562 len 24576
28439 077/300 Read block 4187 len 20480
28440 078/300 Write block 3297 len 8192 data: 5 6
28441 079/300 Read block 2174 len 40960
28442 080/300 Write block 1998 len 12288 data: 3 3 4
28443 081/300 Read block 1249 len 20480
28444 082/300 Read block 2668 len 36864
28445 083/300 Write block 3911 len 32768 data: 7 5 5 4 5 4 3 4
28446 084/300 Write block 1819 len 8192 data: 2 2
28447 085/300 Read block 1959 len 20480
28448 086/300 Write block 2173 len 32768 data: 5 5 4 4 2 3 3 3
28449 087/300 Write block 2879 len 36864 data: 3 3 2 2 2 3 5 5 7
28450 088/300 Write block 2934 len 4096 data: 8
28451 089/300 Read block 3959 len 32768
28452 090/300 Read block 3611 len 4096
28453 091/300 Write block 189 len 8192 data: 5 3
28454 092/300 Read block 1219 len 4096
28455 093/300 Read block 1410 len 4096
28456 094/300 Write block 4098 len 32768 data: 7 7 5 6 6 8 7 5
28457 095/300 Write block 3566 len 28672 data: 4 6 6 5 5 6 4
28458 096/300 Flush
28459 097/300 Write block 1891 len 4096 data: 6
28460 098/300 Read block 161 len 40960
28461 099/300 Read block 1380 len 36864
28462 100/300 Write block 3234 len 24576 data: 5 5 5 6 5 4
28463 101/300 Write block 3858 len 4096 data: 9
28464 102/300 Read block 671 len 16384
28465 103/300 Write block 3685 len 16384 data: 8 6 6 7
28466 104/300 Flush
28467 105/300 Read block 894 len 16384
28468 106/300 Write block 2675 len 36864 data: 3 4 4 4 4 4 4 4 4
28469 107/300 Read block 359 len 8192
28470 108/300 Read block 1205 len 40960
28471 109/300 Read block 506 len 24576
28472 110/300 Write block 4402 len 28672 data: 7 5 6 6 5 4 6
28473 111/300 Write block 1590 len 12288 data: 3 3 3
28474 112/300 Read block 1382 len 32768
28475 113/300 Read block 1359 len 32768
28476 114/300 Write block 4268 len 24576 data: 6 6 5 6 6 6
28477 115/300 Read block 3325 len 24576
28478 116/300 Read block 4148 len 8192
28479 117/300 Write block 3933 len 12288 data: 2 4 5
28480 118/300 Write block 481 len 16384 data: 5 6 6 5
28481 119/300 Read block 2362 len 36864
28482 120/300 Write block 4919 len 28672 data: 6 6 8 11 9 9 7
28483 121/300 Write block 4601 len 28672 data: 6 7 7 7 5 5 5
28484 122/300 Write block 4605 len 4096 data: 6
28485 123/300 Read block 4878 len 4096
28486 124/300 Flush
28487 125/300 Write block 874 len 12288 data: 8 8 6
28488 126/300 Write block 4337 len 24576 data: 5 5 5 7 9 8
28489 127/300 Read block 220 len 36864
28490 128/300 Flush
28491 129/300 Flush
28492 130/300 Read block 824 len 20480
28493 131/300 Read block 587 len 20480
28494 132/300 Flush
28495 133/300 Read block 685 len 24576
28496 134/300 Read block 137 len 32768
28497 135/300 Read block 1953 len 28672
28498 136/300 Read block 4962 len 24576
28499 137/300 Write block 2993 len 32768 data: 4 4 6 9 10 11 9 7
28500 138/300 Write block 3644 len 40960 data: 5 7 8 7 9 7 6 7 8 7
28501 139/300 Write block 3797 len 20480 data: 3 4 5 5 6
28502 140/300 Write block 3765 len 32768 data: 4 3 3 4 4 4 2 2
28503 141/300 Write block 4348 len 36864 data: 8 6 5 4 5 8 7 8 8
28504 142/300 Write block 1246 len 40960 data: 6 6 7 4 3 4 4 3 3 2
28505 143/300 Read block 324 len 4096
28506 144/300 Flush
28507 145/300 Write block 910 len 28672 data: 4 3 4 3 4 3 2
28508 146/300 Read block 1067 len 8192
28509 147/300 Write block 104 len 28672 data: 9 8 6 5 6 4 4
28510 148/300 Write block 3975 len 12288 data: 8 8 7
28511 149/300 Write block 4064 len 16384 data: 6 6 5 4
28512 150/300 Read block 4434 len 8192
28513 151/300 Read block 3583 len 16384
28514 152/300 Read block 2609 len 12288
28515 153/300 Write block 495 len 24576 data: 5 5 6 5 6 5
28516 154/300 Read block 4365 len 28672
28517 155/300 Read block 4530 len 8192
28518 156/300 Write block 1374 len 16384 data: 7 6 4 3
28519 157/300 Read block 1900 len 8192
28520 158/300 Read block 555 len 16384
28521 159/300 Read block 3437 len 28672
28522 160/300 Flush
28523 161/300 Read block 3310 len 16384
28524 162/300 Read block 151 len 4096
28525 163/300 Write block 4854 len 20480 data: 4 4 4 5 5
28526 164/300 Write block 3004 len 8192 data: 4 4
28527 165/300 Write block 4391 len 8192 data: 5 4
28528 166/300 Read block 1706 len 24576
28529 167/300 Read block 2541 len 4096
28530 168/300 Flush
28531 169/300 Read block 1201 len 20480
28532 170/300 Read block 1253 len 40960
28533 171/300 Read block 3800 len 20480
28534 172/300 Read block 3721 len 32768
28535 173/300 Write block 3793 len 32768 data: 3 3 4 4 4 5 6 6
28536 174/300 Read block 3194 len 8192
28537 175/300 Read block 3058 len 16384
28538 176/300 Read block 2564 len 4096
28539 177/300 Write block 292 len 40960 data: 4 5 5 4 4 5 6 6 5 5
28540 178/300 Read block 4453 len 36864
28541 179/300 Read block 1256 len 36864
28542 180/300 Write block 2435 len 20480 data: 4 5 6 5 8
28543 181/300 Read block 3759 len 8192
28544 182/300 Write block 3482 len 36864 data: 3 5 5 5 7 10 10 9 9
28545 183/300 Read block 4557 len 12288
28546 184/300 Flush
28547 185/300 Read block 1556 len 40960
28548 186/300 Write block 4048 len 20480 data: 5 6 5 5 5
28549 187/300 Read block 2681 len 16384
28550 188/300 Write block 824 len 40960 data: 6 5 6 6 5 5 4 5 4 6
28551 189/300 Write block 72 len 32768 data: 9 8 6 6 7 7 7 6
28552 190/300 Write block 43 len 20480 data: 3 5 7 7 6
28553 191/300 Read block 4454 len 12288
28554 192/300 Write block 3637 len 4096 data: 4
28555 193/300 Read block 1919 len 12288
28556 194/300 Flush
28557 195/300 Write block 3210 len 12288 data: 8 11 9
28558 196/300 Write block 1646 len 40960 data: 8 6 6 6 4 4 5 5 4 3
28559 197/300 Read block 4650 len 28672
28560 198/300 Write block 2538 len 12288 data: 3 3 3
28561 199/300 Flush
28562 200/300 Read block 3488 len 36864
28563 201/300 Read block 4703 len 4096
28564 202/300 Write block 1187 len 28672 data: 8 8 7 8 5 5 5
28565 203/300 Flush
28566 204/300 Read block 725 len 36864
28567 205/300 Write block 2631 len 24576 data: 4 3 3 3 2 2
28568 206/300 Read block 3754 len 40960
28569 207/300 Write block 1624 len 40960 data: 8 9 9 7 6 7 6 6 7 7
28570 208/300 Flush
28571 209/300 Write block 156 len 24576 data: 7 5 5 5 4 6
28572 210/300 Read block 2845 len 8192
28573 211/300 Write block 1452 len 40960 data: 7 5 4 5 7 8 8 8 6 6
28574 212/300 Write block 3483 len 4096 data: 6
28575 213/300 Write block 3589 len 12288 data: 5 4 4
28576 214/300 Read block 4076 len 36864
28577 215/300 Write block 4870 len 32768 data: 5 5 4 4 4 5 4 3
28578 216/300 Write block 2286 len 8192 data: 4 3
28579 217/300 Read block 953 len 24576
28580 218/300 Read block 2452 len 40960
28581 219/300 Write block 2889 len 24576 data: 8 8 7 7 7 8
28582 220/300 Write block 1660 len 12288 data: 6 7 7
28583 221/300 Write block 3512 len 24576 data: 8 7 8 8 5 6
28584 222/300 Read block 3808 len 32768
28585 223/300 Read block 3328 len 40960
28586 224/300 Write block 4965 len 40960 data: 3 3 3 6 6 6 7 7 7 5
28587 225/300 Flush
28588 226/300 Read block 3417 len 40960
28589 227/300 Read block 3223 len 4096
28590 228/300 Read block 1145 len 4096
28591 229/300 Read block 3037 len 12288
28592 230/300 Write block 1269 len 32768 data: 6 8 7 5 4 4 6 5
28593 231/300 Read block 2446 len 40960
28594 232/300 Write block 45 len 32768 data: 8 8 7 5 7 8 6 5
28595 233/300 Flush
28596 234/300 Read block 4384 len 16384
28597 235/300 Write block 541 len 28672 data: 3 3 3 3 5 4 5
28598 236/300 Write block 273 len 8192 data: 5 5
28599 237/300 Write block 491 len 12288 data: 5 4 4
28600 238/300 Write block 322 len 12288 data: 8 10 10
28601 239/300 Write block 2395 len 36864 data: 2 2 2 2 2 2 2 3 3
28602 240/300 Write block 821 len 8192 data: 5 6
28603 241/300 Write block 1289 len 28672 data: 6 8 8 8 8 8 8
28604 242/300 Write block 4811 len 20480 data: 5 5 7 5 5
28605 243/300 Read block 27 len 16384
28606 244/300 Write block 4861 len 12288 data: 4 5 4
28607 245/300 Flush
28608 246/300 Write block 1873 len 36864 data: 3 4 5 5 5 5 5 6 6
28609 247/300 Read block 488 len 40960
28610 248/300 Write block 3494 len 28672 data: 7 5 5 4 4 5 5
28611 249/300 Read block 3042 len 16384
28612 250/300 Read block 869 len 36864
28613 251/300 Read block 402 len 8192
28614 252/300 Read block 3855 len 4096
28615 253/300 Write block 4556 len 20480 data: 3 6 7 5 5
28616 254/300 Read block 4577 len 40960
28617 255/300 Read block 3680 len 20480
28618 256/300 Read block 2259 len 36864
28619 257/300 Read block 2127 len 8192
28620 258/300 Read block 2205 len 36864
28621 259/300 Read block 3009 len 8192
28622 260/300 Read block 3194 len 28672
28623 261/300 Flush
28624 262/300 Flush
28625 263/300 Read block 4000 len 16384
28626 264/300 Write block 1177 len 4096 data: 4
28627 265/300 Write block 782 len 20480 data: 5 5 6 6 6
28628 266/300 Read block 608 len 40960
28629 267/300 Write block 3770 len 28672 data: 5 3 3 2 3 3 3
28630 268/300 Flush
28631 269/300 Read block 2914 len 32768
28632 270/300 Write block 4623 len 36864 data: 5 4 4 4 4 4 6 5 7
28633 271/300 Write block 340 len 32768 data: 5 6 7 8 9 9 9 8
28634 272/300 Write block 1352 len 16384 data: 6 6 6 5
28635 273/300 Flush
28636 274/300 Flush
28637 275/300 Read block 4206 len 12288
28638 276/300 Write block 2109 len 8192 data: 5 4
28639 277/300 Write block 2193 len 4096 data: 5
28640 278/300 Read block 4397 len 24576
28641 279/300 Write block 1435 len 40960 data: 6 6 5 5 7 9 9 8 8 8
28642 280/300 Write block 2988 len 28672 data: 5 4 4 4 5 5 5
28643 281/300 Write block 3701 len 40960 data: 6 8 5 4 5 5 5 4 4 4
28644 282/300 Write block 1989 len 40960 data: 5 4 4 5 4 3 2 2 3 4
28645 283/300 Read block 2077 len 32768
28646 284/300 Write block 3066 len 36864 data: 4 5 5 5 6 6 5 3 3
28647 285/300 Write block 2993 len 20480 data: 6 6 7 10 11
28648 286/300 Read block 2512 len 8192
28649 287/300 Read block 2224 len 32768
28650 288/300 Write block 1679 len 28672 data: 3 4 4 5 7 8 7
28651 289/300 Write block 975 len 28672 data: 7 5 5 4 5 5 5
28652 290/300 Flush
28653 291/300 Read block 1546 len 12288
28654 292/300 Flush
28655 293/300 Write block 4568 len 32768 data: 4 4 4 4 4 3 3 3
28656 294/300 Write block 3652 len 32768 data: 9 8 5 5 6 7 7 4
28657 295/300 Flush
28658 296/300 Read block 1953 len 16384
28659 297/300 Write block 1214 len 24576 data: 5 4 4 4 4 4
28660 298/300 Read block 204 len 36864
28661 299/300 Flush
28662 300/300 Flush
28663 301/300 Read block 3907 len 12288
28664 Main task has work to do, go find it
28665 got dsc Start(1)
28666 start 1
28667 Dec 11 23:46:32.396 INFO request completed, latency_us: 221, response_code: 204, uri: /start/cid/1, method: POST, req_id: a350c765-74ae-4172-af25-4219dd64b96a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
28668 Replay: started 1, returned:Ok(())
28669 ----------------------------------------------------------------
28670 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28671 States: Active Offline Active
28672 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28673 7946 Acked Write 1 Done Sent Done false
28674 7947 Acked Write 3 Done Sent Done false
28675 7948 Acked Read 7 Done Sent Done false
28676 7949 Acked Write 1 Done Sent Done false
28677 7950 Acked Read 8 Done Sent Done false
28678 7951 Acked Write 7 Done Sent Done false
28679 7952 Acked Read 7 Done Sent Done false
28680 7953 Acked Read 1 Done Sent Done false
28681 7954 Acked Read 8 Done Sent Done false
28682 7955 Acked Write 1 Done Sent Done false
28683 7956 Acked Flush 0 Done Sent Done false
28684 7957 Acked Write 1 Done Sent Done false
28685 7958 Acked Read 4 Done Sent Done false
28686 7959 Acked Read 1 Done Sent Done false
28687 7960 Acked Read 4 Done Sent Done false
28688 7961 Acked Read 1 Done Sent Done false
28689 7962 Acked Read 4 Done Sent Done false
28690 7963 Acked Read 2 Done Sent Done false
28691 7964 Acked Write 3 Done Sent Done false
28692 7965 Acked Read 10 Done Sent Done false
28693 7966 Acked Read 9 Done Sent Done false
28694 7967 Acked Write 1 Done Sent Done false
28695 7968 Acked Write 2 Done Sent Done false
28696 7969 Acked Write 6 Done Sent Done false
28697 7970 Acked Write 5 Done Sent Done false
28698 7971 Acked Write 2 Done Sent Done false
28699 7972 Acked Read 2 Done Sent Done false
28700 7973 Acked Write 9 Done Sent Done false
28701 7974 Acked Read 3 Done Sent Done false
28702 7975 Acked Write 10 Done Sent Done false
28703 7976 Acked Read 5 Done Sent Done false
28704 7977 Acked Write 10 Done Sent Done false
28705 7978 Acked Write 10 Done Sent Done false
28706 7979 Acked Write 10 Done Sent Done false
28707 7980 Acked Read 3 Done Sent Done false
28708 7981 Acked Flush 0 Done Sent Done false
28709 7982 Acked Write 9 Done Sent Done false
28710 7983 Acked Read 1 Done Sent Done false
28711 7984 Acked Flush 0 Done Sent Done false
28712 7985 Acked Flush 0 Done Sent Done false
28713 7986 Acked Read 6 Done Sent Done false
28714 7987 Acked Write 6 Done Sent Done false
28715 7988 Acked Write 8 Done Sent Done false
28716 7989 Acked Write 9 Done Sent Done false
28717 7990 Acked Write 3 Done Sent Done false
28718 7991 Acked Read 2 Done Sent Done false
28719 7992 Acked Read 10 Done Sent Done false
28720 7993 Acked Write 2 Done Sent Done false
28721 7994 Acked Write 2 Done Sent Done false
28722 7995 Acked Read 3 Done Sent Done false
28723 7996 Acked Flush 0 Done Sent Done false
28724 7997 Acked Write 2 Done Sent Done false
28725 7998 Acked Read 2 Done Sent Done false
28726 7999 Acked Flush 0 Done Sent Done false
28727 8000 Acked Write 9 Done Sent Done false
28728 8001 Acked Write 1 Done Sent Done false
28729 8002 Acked Read 2 Done Sent Done false
28730 8003 Acked Write 1 Done Sent Done false
28731 8004 Acked Flush 0 Done Sent Done false
28732 8005 Acked Write 9 Done Sent Done false
28733 8006 Acked Write 10 Done Sent Done false
28734 8007 Acked Read 9 Done Sent Done false
28735 8008 Acked Write 1 Done Sent Done false
28736 8009 Acked Write 8 Done Sent Done false
28737 8010 Acked Flush 0 Done Sent Done false
28738 8011 Acked Read 10 Done Sent Done false
28739 8012 Acked Flush 0 Done Sent Done false
28740 8013 Acked Read 7 Done Sent Done false
28741 8014 Acked Write 3 Done Sent Done false
28742 8015 Acked Read 3 Done Sent Done false
28743 8016 Acked Write 5 Done Sent Done false
28744 8017 Acked Write 6 Done Sent Done false
28745 8018 Acked Read 9 Done Sent Done false
28746 8019 Acked Read 6 Done Sent Done false
28747 8020 Acked Read 10 Done Sent Done false
28748 8021 Acked Read 6 Done Sent Done false
28749 8022 Acked Read 5 Done Sent Done false
28750 8023 Acked Write 2 Done Sent Done false
28751 8024 Acked Read 10 Done Sent Done false
28752 8025 Acked Write 3 Done Sent Done false
28753 8026 Acked Read 5 Done Sent Done false
28754 8027 Acked Read 9 Done Sent Done false
28755 8028 Acked Write 8 Done Sent Done false
28756 8029 Acked Write 2 Done Sent Done false
28757 8030 Acked Read 5 Done Sent Done false
28758 8031 Acked Write 8 Done Sent Done false
28759 8032 Acked Write 9 Done Sent Done false
28760 8033 Acked Write 1 Done Sent Done false
28761 8034 Acked Read 8 Done Sent Done false
28762 8035 Acked Read 1 Done Sent Done false
28763 8036 Acked Write 2 Done Sent Done false
28764 8037 Acked Read 1 Done Sent Done false
28765 8038 Acked Read 1 Done Sent Done false
28766 8039 Acked Write 8 Done Sent Done false
28767 8040 Acked Write 7 Done Sent Done false
28768 8041 Acked Flush 0 Done Sent Done false
28769 8042 Acked Write 1 Done Sent Done false
28770 8043 Acked Read 10 Done Sent Done false
28771 8044 Acked Read 9 Done Sent Done false
28772 8045 Acked Write 6 Done Sent Done false
28773 8046 Acked Write 1 Done Sent Done false
28774 8047 Acked Read 4 Done Sent Done false
28775 8048 Acked Write 4 Done Sent Done false
28776 8049 Acked Flush 0 Done Sent Done false
28777 8050 Acked Read 4 Done Sent Done false
28778 8051 Acked Write 9 Done Sent Done false
28779 8052 Acked Read 2 Done Sent Done false
28780 8053 Acked Read 10 Done Sent Done false
28781 8054 Acked Read 6 Done Sent Done false
28782 8055 Acked Write 7 Done Sent Done false
28783 8056 Acked Write 3 Done Sent Done false
28784 8057 Acked Read 8 Done Sent Done false
28785 8058 Acked Read 8 Done Sent Done false
28786 8059 Acked Write 6 Done Sent Done false
28787 8060 Acked Read 6 Done Sent Done false
28788 8061 Acked Read 2 Done Sent Done false
28789 8062 Acked Write 3 Done Sent Done false
28790 8063 Acked Write 4 Done Sent Done false
28791 8064 Acked Read 9 Done Sent Done false
28792 8065 Acked Write 7 Done Sent Done false
28793 8066 Acked Write 7 Done Sent Done false
28794 8067 Acked Write 1 Done Sent Done false
28795 8068 Acked Read 1 Done Sent Done false
28796 8069 Acked Flush 0 Done Sent Done false
28797 8070 Acked Write 3 Done Sent Done false
28798 8071 Acked Write 6 Done Sent Done false
28799 8072 Acked Read 9 Done Sent Done false
28800 8073 Acked Flush 0 Done Sent Done false
28801 8074 Acked Flush 0 Done Sent Done false
28802 8075 Acked Read 5 Done Sent Done false
28803 8076 Acked Read 5 Done Sent Done false
28804 8077 Acked Flush 0 Done Sent Done false
28805 8078 Acked Read 6 Done Sent Done false
28806 8079 Acked Read 8 Done Sent Done false
28807 8080 Acked Read 7 Done Sent Done false
28808 8081 Acked Read 6 Done Sent Done false
28809 8082 Acked Write 8 Done Sent Done false
28810 8083 Acked Write 10 Done Sent Done false
28811 8084 Acked Write 5 Done Sent Done false
28812 8085 Acked Write 8 Done Sent Done false
28813 8086 Acked Write 9 Done Sent Done false
28814 8087 Acked Write 10 Done Sent Done false
28815 8088 Acked Read 1 Done Sent Done false
28816 8089 Acked Flush 0 Done Sent Done false
28817 8090 Acked Write 7 Done Sent Done false
28818 8091 Acked Read 2 Done Sent Done false
28819 8092 Acked Write 7 Done Sent Done false
28820 8093 Acked Write 3 Done Sent Done false
28821 8094 Acked Write 4 Done Sent Done false
28822 8095 Acked Read 2 Done Sent Done false
28823 8096 Acked Read 4 Done Sent Done false
28824 8097 Acked Read 3 Done Sent Done false
28825 8098 Acked Write 6 Done Sent Done false
28826 8099 Acked Read 7 Done Sent Done false
28827 8100 Acked Read 2 Done Sent Done false
28828 8101 Acked Write 4 Done Sent Done false
28829 8102 Acked Read 2 Done Sent Done false
28830 8103 Acked Read 4 Done Sent Done false
28831 8104 Acked Read 7 Done Sent Done false
28832 8105 Acked Flush 0 Done Sent Done false
28833 8106 Acked Read 4 Done Sent Done false
28834 8107 Acked Read 1 Done Sent Done false
28835 8108 Acked Write 5 Done Sent Done false
28836 8109 Acked Write 2 Done Sent Done false
28837 8110 Acked Write 2 Done Sent Done false
28838 8111 Acked Read 6 Done Sent Done false
28839 8112 Acked Read 1 Done Sent Done false
28840 8113 Acked Flush 0 Done Sent Done false
28841 8114 Acked Read 5 Done Sent Done false
28842 8115 Acked Read 10 Done Sent Done false
28843 8116 Acked Read 5 Done Sent Done false
28844 8117 Acked Read 8 Done Sent Done false
28845 8118 Acked Write 8 Done Sent Done false
28846 8119 Acked Read 2 Done Sent Done false
28847 8120 Acked Read 4 Done Sent Done false
28848 8121 Acked Read 1 Done Sent Done false
28849 8122 Acked Write 10 Done Sent Done false
28850 8123 Acked Read 9 Done Sent Done false
28851 8124 Acked Read 9 Done Sent Done false
28852 8125 Acked Write 5 Done Sent Done false
28853 8126 Acked Read 2 Done Sent Done false
28854 8127 Acked Write 9 Done Sent Done false
28855 8128 Acked Read 3 Done Sent Done false
28856 8129 Acked Flush 0 Done Sent Done false
28857 8130 Acked Read 10 Done Sent Done false
28858 8131 Acked Write 5 Done Sent Done false
28859 8132 Acked Read 4 Done Sent Done false
28860 8133 Acked Write 10 Done Sent Done false
28861 8134 Acked Write 8 Done Sent Done false
28862 8135 Acked Write 5 Done Sent Done false
28863 8136 Acked Read 3 Done Sent Done false
28864 8137 Acked Write 1 Done Sent Done false
28865 8138 Acked Read 3 Done Sent Done false
28866 8139 Acked Flush 0 Done Sent Done false
28867 8140 Acked Write 3 Done Sent Done false
28868 8141 Acked Write 10 Done Sent Done false
28869 8142 Acked Read 7 Done Sent Done false
28870 8143 Acked Write 3 Done Sent Done false
28871 8144 Acked Flush 0 Done Sent Done false
28872 8145 Acked Read 9 Done Sent Done false
28873 8146 Acked Read 1 Done Sent Done false
28874 8147 Acked Write 7 Done Sent Done false
28875 8148 Acked Flush 0 Done Sent Done false
28876 8149 Acked Read 9 Done Sent Done false
28877 8150 Acked Write 6 Done Sent Done false
28878 8151 Acked Read 10 Done Sent Done false
28879 8152 Acked Write 10 Done Sent Done false
28880 8153 Acked Flush 0 Done Sent Done false
28881 8154 Acked Write 6 Done Sent Done false
28882 8155 Acked Read 2 Done Sent Done false
28883 8156 Acked Write 10 Done Sent Done false
28884 8157 Acked Write 1 Done Sent Done false
28885 8158 Acked Write 3 Done Sent Done false
28886 8159 Acked Read 9 Done Sent Done false
28887 8160 Acked Write 8 Done Sent Done false
28888 8161 Acked Write 2 Done Sent Done false
28889 8162 Acked Read 6 Done Sent Done false
28890 8163 Acked Read 10 Done Sent Done false
28891 8164 Acked Write 6 Done Sent Done false
28892 8165 Acked Write 3 Done Sent Done false
28893 8166 Acked Write 6 Done Sent Done false
28894 8167 Acked Read 8 Done Sent Done false
28895 8168 Acked Read 10 Done Sent Done false
28896 8169 Acked Write 10 Done Sent Done false
28897 8170 Acked Flush 0 Done Sent Done false
28898 8171 Acked Read 10 Done Sent Done false
28899 8172 Acked Read 1 Done Sent Done false
28900 8173 Acked Read 1 Done Sent Done false
28901 8174 Acked Read 3 Done Sent Done false
28902 8175 Acked Write 8 Done Sent Done false
28903 8176 Acked Read 10 Done Sent Done false
28904 8177 Acked Write 8 Done Sent Done false
28905 8178 Acked Flush 0 Done Sent Done false
28906 8179 Acked Read 4 Done Sent Done false
28907 8180 Acked Write 7 Done Sent Done false
28908 8181 Acked Write 2 Done Sent Done false
28909 8182 Acked Write 3 Done Sent Done false
28910 8183 Acked Write 3 Done Sent Done false
28911 8184 Acked Write 9 Done Sent Done false
28912 8185 Acked Write 2 Done Sent Done false
28913 8186 Acked Write 7 Done Sent Done false
28914 8187 Acked Write 5 Done Sent Done false
28915 8188 Acked Read 4 Done Sent Done false
28916 8189 Acked Write 3 Done Sent Done false
28917 8190 Acked Flush 0 Done Sent Done false
28918 8191 Acked Write 9 Done Sent Done false
28919 8192 Acked Read 10 Done Sent Done false
28920 8193 Acked Write 7 Done Sent Done false
28921 8194 Acked Read 4 Done Sent Done false
28922 8195 Acked Read 9 Done Sent Done false
28923 8196 Acked Read 2 Done Sent Done false
28924 8197 Acked Read 1 Done Sent Done false
28925 8198 Acked Write 5 Done Sent Done false
28926 8199 Acked Read 10 Done Sent Done false
28927 8200 Acked Read 5 Done Sent Done false
28928 8201 Acked Read 9 Done Sent Done false
28929 8202 Acked Read 2 Done Sent Done false
28930 8203 Acked Read 9 Done Sent Done false
28931 8204 Acked Read 2 Done Sent Done false
28932 8205 Acked Read 7 Done Sent Done false
28933 8206 Acked Flush 0 Done Sent Done false
28934 8207 Acked Flush 0 Done Sent Done false
28935 8208 Acked Read 4 Done Sent Done false
28936 8209 Acked Write 1 Done Sent Done false
28937 8210 Acked Write 5 Done Sent Done false
28938 8211 Acked Read 10 Done Sent Done false
28939 8212 Acked Write 7 Done Sent Done false
28940 8213 Acked Flush 0 Done Sent Done false
28941 8214 Acked Read 8 Done Sent Done false
28942 8215 Acked Write 9 Done Sent Done false
28943 8216 Acked Write 8 Done Sent Done false
28944 8217 Acked Write 4 Done Sent Done false
28945 8218 Acked Flush 0 Done Sent Done false
28946 8219 Acked Flush 0 Done Sent Done false
28947 8220 Acked Read 3 Done Sent Done false
28948 8221 Acked Write 2 Done Sent Done false
28949 8222 Acked Write 1 Done Sent Done false
28950 8223 Acked Read 6 Done Sent Done false
28951 8224 Acked Write 10 Done Sent Done false
28952 8225 Acked Write 7 Done Sent Done false
28953 8226 Acked Write 10 Done Sent Done false
28954 8227 Acked Write 10 Done Sent Done false
28955 8228 Acked Read 8 Done Sent Done false
28956 8229 Acked Write 9 Done Sent Done false
28957 8230 Acked Write 5 Done Sent Done false
28958 8231 Acked Read 2 Done Sent Done false
28959 8232 Acked Read 8 Done Sent Done false
28960 8233 Acked Write 7 Done Sent Done false
28961 8234 Acked Write 7 Done Sent Done false
28962 8235 Acked Flush 0 Done Sent Done false
28963 8236 Acked Read 3 Done Sent Done false
28964 8237 Acked Flush 0 Done Sent Done false
28965 8238 Acked Write 8 Done Sent Done false
28966 8239 Acked Write 8 Done Sent Done false
28967 8240 Acked Flush 0 Done Sent Done false
28968 8241 Acked Read 4 Done Sent Done false
28969 8242 Acked Write 6 Done Sent Done false
28970 8243 Acked Read 9 Done Sent Done false
28971 8244 Acked Flush 0 Done Sent Done false
28972 8245 Acked Flush 0 Done Sent Done false
28973 8246 Acked Read 3 Done Sent Done false
28974 STATES DS:0 DS:1 DS:2 TOTAL
28975 Sent 0 301 0 301
28976 Done 301 0 301 602
28977 Skipped 0 0 0 0
28978 Error 0 0 0 0
28979 Last Flush: 8245 7945 8245
28980 Downstairs last five completed: 7945 7944 7943 7942 7941
28981 Upstairs last five completed: 8246 8245 8244 8243 8242
28982 CLIENT: Up:0 ds:301 act:2
28983 [8820] got start action
28984 [8820] I am going to restart
28985 Starting downstairs at port 8820
28986 Make output file at "/tmp/dsc/downstairs-8820.txt"
28987 [8820][1] reports Starting
28988 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1224)
28989 [8820][1] reports Running
28990 ----------------------------------------------------------------
28991 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28992 States: Active Offline Active
28993 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28994 7946 Acked Write 1 Done Sent Done false
28995 7947 Acked Write 3 Done Sent Done false
28996 7948 Acked Read 7 Done Sent Done false
28997 7949 Acked Write 1 Done Sent Done false
28998 7950 Acked Read 8 Done Sent Done false
28999 7951 Acked Write 7 Done Sent Done false
29000 7952 Acked Read 7 Done Sent Done false
29001 7953 Acked Read 1 Done Sent Done false
29002 7954 Acked Read 8 Done Sent Done false
29003 7955 Acked Write 1 Done Sent Done false
29004 7956 Acked Flush 0 Done Sent Done false
29005 7957 Acked Write 1 Done Sent Done false
29006 7958 Acked Read 4 Done Sent Done false
29007 7959 Acked Read 1 Done Sent Done false
29008 7960 Acked Read 4 Done Sent Done false
29009 7961 Acked Read 1 Done Sent Done false
29010 7962 Acked Read 4 Done Sent Done false
29011 7963 Acked Read 2 Done Sent Done false
29012 7964 Acked Write 3 Done Sent Done false
29013 7965 Acked Read 10 Done Sent Done false
29014 7966 Acked Read 9 Done Sent Done false
29015 7967 Acked Write 1 Done Sent Done false
29016 7968 Acked Write 2 Done Sent Done false
29017 7969 Acked Write 6 Done Sent Done false
29018 7970 Acked Write 5 Done Sent Done false
29019 7971 Acked Write 2 Done Sent Done false
29020 7972 Acked Read 2 Done Sent Done false
29021 7973 Acked Write 9 Done Sent Done false
29022 7974 Acked Read 3 Done Sent Done false
29023 7975 Acked Write 10 Done Sent Done false
29024 7976 Acked Read 5 Done Sent Done false
29025 7977 Acked Write 10 Done Sent Done false
29026 7978 Acked Write 10 Done Sent Done false
29027 7979 Acked Write 10 Done Sent Done false
29028 7980 Acked Read 3 Done Sent Done false
29029 7981 Acked Flush 0 Done Sent Done false
29030 7982 Acked Write 9 Done Sent Done false
29031 7983 Acked Read 1 Done Sent Done false
29032 7984 Acked Flush 0 Done Sent Done false
29033 7985 Acked Flush 0 Done Sent Done false
29034 7986 Acked Read 6 Done Sent Done false
29035 7987 Acked Write 6 Done Sent Done false
29036 7988 Acked Write 8 Done Sent Done false
29037 7989 Acked Write 9 Done Sent Done false
29038 7990 Acked Write 3 Done Sent Done false
29039 7991 Acked Read 2 Done Sent Done false
29040 7992 Acked Read 10 Done Sent Done false
29041 7993 Acked Write 2 Done Sent Done false
29042 7994 Acked Write 2 Done Sent Done false
29043 7995 Acked Read 3 Done Sent Done false
29044 7996 Acked Flush 0 Done Sent Done false
29045 7997 Acked Write 2 Done Sent Done false
29046 7998 Acked Read 2 Done Sent Done false
29047 7999 Acked Flush 0 Done Sent Done false
29048 8000 Acked Write 9 Done Sent Done false
29049 8001 Acked Write 1 Done Sent Done false
29050 8002 Acked Read 2 Done Sent Done false
29051 8003 Acked Write 1 Done Sent Done false
29052 8004 Acked Flush 0 Done Sent Done false
29053 8005 Acked Write 9 Done Sent Done false
29054 8006 Acked Write 10 Done Sent Done false
29055 8007 Acked Read 9 Done Sent Done false
29056 8008 Acked Write 1 Done Sent Done false
29057 8009 Acked Write 8 Done Sent Done false
29058 8010 Acked Flush 0 Done Sent Done false
29059 8011 Acked Read 10 Done Sent Done false
29060 8012 Acked Flush 0 Done Sent Done false
29061 8013 Acked Read 7 Done Sent Done false
29062 8014 Acked Write 3 Done Sent Done false
29063 8015 Acked Read 3 Done Sent Done false
29064 8016 Acked Write 5 Done Sent Done false
29065 8017 Acked Write 6 Done Sent Done false
29066 8018 Acked Read 9 Done Sent Done false
29067 8019 Acked Read 6 Done Sent Done false
29068 8020 Acked Read 10 Done Sent Done false
29069 8021 Acked Read 6 Done Sent Done false
29070 8022 Acked Read 5 Done Sent Done false
29071 8023 Acked Write 2 Done Sent Done false
29072 8024 Acked Read 10 Done Sent Done false
29073 8025 Acked Write 3 Done Sent Done false
29074 8026 Acked Read 5 Done Sent Done false
29075 8027 Acked Read 9 Done Sent Done false
29076 8028 Acked Write 8 Done Sent Done false
29077 8029 Acked Write 2 Done Sent Done false
29078 8030 Acked Read 5 Done Sent Done false
29079 8031 Acked Write 8 Done Sent Done false
29080 8032 Acked Write 9 Done Sent Done false
29081 8033 Acked Write 1 Done Sent Done false
29082 8034 Acked Read 8 Done Sent Done false
29083 8035 Acked Read 1 Done Sent Done false
29084 8036 Acked Write 2 Done Sent Done false
29085 8037 Acked Read 1 Done Sent Done false
29086 8038 Acked Read 1 Done Sent Done false
29087 8039 Acked Write 8 Done Sent Done false
29088 8040 Acked Write 7 Done Sent Done false
29089 8041 Acked Flush 0 Done Sent Done false
29090 8042 Acked Write 1 Done Sent Done false
29091 8043 Acked Read 10 Done Sent Done false
29092 8044 Acked Read 9 Done Sent Done false
29093 8045 Acked Write 6 Done Sent Done false
29094 8046 Acked Write 1 Done Sent Done false
29095 8047 Acked Read 4 Done Sent Done false
29096 8048 Acked Write 4 Done Sent Done false
29097 8049 Acked Flush 0 Done Sent Done false
29098 8050 Acked Read 4 Done Sent Done false
29099 8051 Acked Write 9 Done Sent Done false
29100 8052 Acked Read 2 Done Sent Done false
29101 8053 Acked Read 10 Done Sent Done false
29102 8054 Acked Read 6 Done Sent Done false
29103 8055 Acked Write 7 Done Sent Done false
29104 8056 Acked Write 3 Done Sent Done false
29105 8057 Acked Read 8 Done Sent Done false
29106 8058 Acked Read 8 Done Sent Done false
29107 8059 Acked Write 6 Done Sent Done false
29108 8060 Acked Read 6 Done Sent Done false
29109 8061 Acked Read 2 Done Sent Done false
29110 8062 Acked Write 3 Done Sent Done false
29111 8063 Acked Write 4 Done Sent Done false
29112 8064 Acked Read 9 Done Sent Done false
29113 8065 Acked Write 7 Done Sent Done false
29114 8066 Acked Write 7 Done Sent Done false
29115 8067 Acked Write 1 Done Sent Done false
29116 8068 Acked Read 1 Done Sent Done false
29117 8069 Acked Flush 0 Done Sent Done false
29118 8070 Acked Write 3 Done Sent Done false
29119 8071 Acked Write 6 Done Sent Done false
29120 8072 Acked Read 9 Done Sent Done false
29121 8073 Acked Flush 0 Done Sent Done false
29122 8074 Acked Flush 0 Done Sent Done false
29123 8075 Acked Read 5 Done Sent Done false
29124 8076 Acked Read 5 Done Sent Done false
29125 8077 Acked Flush 0 Done Sent Done false
29126 8078 Acked Read 6 Done Sent Done false
29127 8079 Acked Read 8 Done Sent Done false
29128 8080 Acked Read 7 Done Sent Done false
29129 8081 Acked Read 6 Done Sent Done false
29130 8082 Acked Write 8 Done Sent Done false
29131 8083 Acked Write 10 Done Sent Done false
29132 8084 Acked Write 5 Done Sent Done false
29133 8085 Acked Write 8 Done Sent Done false
29134 8086 Acked Write 9 Done Sent Done false
29135 8087 Acked Write 10 Done Sent Done false
29136 8088 Acked Read 1 Done Sent Done false
29137 8089 Acked Flush 0 Done Sent Done false
29138 8090 Acked Write 7 Done Sent Done false
29139 8091 Acked Read 2 Done Sent Done false
29140 8092 Acked Write 7 Done Sent Done false
29141 8093 Acked Write 3 Done Sent Done false
29142 8094 Acked Write 4 Done Sent Done false
29143 8095 Acked Read 2 Done Sent Done false
29144 8096 Acked Read 4 Done Sent Done false
29145 8097 Acked Read 3 Done Sent Done false
29146 8098 Acked Write 6 Done Sent Done false
29147 8099 Acked Read 7 Done Sent Done false
29148 8100 Acked Read 2 Done Sent Done false
29149 8101 Acked Write 4 Done Sent Done false
29150 8102 Acked Read 2 Done Sent Done false
29151 8103 Acked Read 4 Done Sent Done false
29152 8104 Acked Read 7 Done Sent Done false
29153 8105 Acked Flush 0 Done Sent Done false
29154 8106 Acked Read 4 Done Sent Done false
29155 8107 Acked Read 1 Done Sent Done false
29156 8108 Acked Write 5 Done Sent Done false
29157 8109 Acked Write 2 Done Sent Done false
29158 8110 Acked Write 2 Done Sent Done false
29159 8111 Acked Read 6 Done Sent Done false
29160 8112 Acked Read 1 Done Sent Done false
29161 8113 Acked Flush 0 Done Sent Done false
29162 8114 Acked Read 5 Done Sent Done false
29163 8115 Acked Read 10 Done Sent Done false
29164 8116 Acked Read 5 Done Sent Done false
29165 8117 Acked Read 8 Done Sent Done false
29166 8118 Acked Write 8 Done Sent Done false
29167 8119 Acked Read 2 Done Sent Done false
29168 8120 Acked Read 4 Done Sent Done false
29169 8121 Acked Read 1 Done Sent Done false
29170 8122 Acked Write 10 Done Sent Done false
29171 8123 Acked Read 9 Done Sent Done false
29172 8124 Acked Read 9 Done Sent Done false
29173 8125 Acked Write 5 Done Sent Done false
29174 8126 Acked Read 2 Done Sent Done false
29175 8127 Acked Write 9 Done Sent Done false
29176 8128 Acked Read 3 Done Sent Done false
29177 8129 Acked Flush 0 Done Sent Done false
29178 8130 Acked Read 10 Done Sent Done false
29179 8131 Acked Write 5 Done Sent Done false
29180 8132 Acked Read 4 Done Sent Done false
29181 8133 Acked Write 10 Done Sent Done false
29182 8134 Acked Write 8 Done Sent Done false
29183 8135 Acked Write 5 Done Sent Done false
29184 8136 Acked Read 3 Done Sent Done false
29185 8137 Acked Write 1 Done Sent Done false
29186 8138 Acked Read 3 Done Sent Done false
29187 8139 Acked Flush 0 Done Sent Done false
29188 8140 Acked Write 3 Done Sent Done false
29189 8141 Acked Write 10 Done Sent Done false
29190 8142 Acked Read 7 Done Sent Done false
29191 8143 Acked Write 3 Done Sent Done false
29192 8144 Acked Flush 0 Done Sent Done false
29193 8145 Acked Read 9 Done Sent Done false
29194 8146 Acked Read 1 Done Sent Done false
29195 8147 Acked Write 7 Done Sent Done false
29196 8148 Acked Flush 0 Done Sent Done false
29197 8149 Acked Read 9 Done Sent Done false
29198 8150 Acked Write 6 Done Sent Done false
29199 8151 Acked Read 10 Done Sent Done false
29200 8152 Acked Write 10 Done Sent Done false
29201 8153 Acked Flush 0 Done Sent Done false
29202 8154 Acked Write 6 Done Sent Done false
29203 8155 Acked Read 2 Done Sent Done false
29204 8156 Acked Write 10 Done Sent Done false
29205 8157 Acked Write 1 Done Sent Done false
29206 8158 Acked Write 3 Done Sent Done false
29207 8159 Acked Read 9 Done Sent Done false
29208 8160 Acked Write 8 Done Sent Done false
29209 8161 Acked Write 2 Done Sent Done false
29210 8162 Acked Read 6 Done Sent Done false
29211 8163 Acked Read 10 Done Sent Done false
29212 8164 Acked Write 6 Done Sent Done false
29213 8165 Acked Write 3 Done Sent Done false
29214 8166 Acked Write 6 Done Sent Done false
29215 8167 Acked Read 8 Done Sent Done false
29216 8168 Acked Read 10 Done Sent Done false
29217 8169 Acked Write 10 Done Sent Done false
29218 8170 Acked Flush 0 Done Sent Done false
29219 8171 Acked Read 10 Done Sent Done false
29220 8172 Acked Read 1 Done Sent Done false
29221 8173 Acked Read 1 Done Sent Done false
29222 8174 Acked Read 3 Done Sent Done false
29223 8175 Acked Write 8 Done Sent Done false
29224 8176 Acked Read 10 Done Sent Done false
29225 8177 Acked Write 8 Done Sent Done false
29226 8178 Acked Flush 0 Done Sent Done false
29227 8179 Acked Read 4 Done Sent Done false
29228 8180 Acked Write 7 Done Sent Done false
29229 8181 Acked Write 2 Done Sent Done false
29230 8182 Acked Write 3 Done Sent Done false
29231 8183 Acked Write 3 Done Sent Done false
29232 8184 Acked Write 9 Done Sent Done false
29233 8185 Acked Write 2 Done Sent Done false
29234 8186 Acked Write 7 Done Sent Done false
29235 8187 Acked Write 5 Done Sent Done false
29236 8188 Acked Read 4 Done Sent Done false
29237 8189 Acked Write 3 Done Sent Done false
29238 8190 Acked Flush 0 Done Sent Done false
29239 8191 Acked Write 9 Done Sent Done false
29240 8192 Acked Read 10 Done Sent Done false
29241 8193 Acked Write 7 Done Sent Done false
29242 8194 Acked Read 4 Done Sent Done false
29243 8195 Acked Read 9 Done Sent Done false
29244 8196 Acked Read 2 Done Sent Done false
29245 8197 Acked Read 1 Done Sent Done false
29246 8198 Acked Write 5 Done Sent Done false
29247 8199 Acked Read 10 Done Sent Done false
29248 8200 Acked Read 5 Done Sent Done false
29249 8201 Acked Read 9 Done Sent Done false
29250 8202 Acked Read 2 Done Sent Done false
29251 8203 Acked Read 9 Done Sent Done false
29252 8204 Acked Read 2 Done Sent Done false
29253 8205 Acked Read 7 Done Sent Done false
29254 8206 Acked Flush 0 Done Sent Done false
29255 8207 Acked Flush 0 Done Sent Done false
29256 8208 Acked Read 4 Done Sent Done false
29257 8209 Acked Write 1 Done Sent Done false
29258 8210 Acked Write 5 Done Sent Done false
29259 8211 Acked Read 10 Done Sent Done false
29260 8212 Acked Write 7 Done Sent Done false
29261 8213 Acked Flush 0 Done Sent Done false
29262 8214 Acked Read 8 Done Sent Done false
29263 8215 Acked Write 9 Done Sent Done false
29264 8216 Acked Write 8 Done Sent Done false
29265 8217 Acked Write 4 Done Sent Done false
29266 8218 Acked Flush 0 Done Sent Done false
29267 8219 Acked Flush 0 Done Sent Done false
29268 8220 Acked Read 3 Done Sent Done false
29269 8221 Acked Write 2 Done Sent Done false
29270 8222 Acked Write 1 Done Sent Done false
29271 8223 Acked Read 6 Done Sent Done false
29272 8224 Acked Write 10 Done Sent Done false
29273 8225 Acked Write 7 Done Sent Done false
29274 8226 Acked Write 10 Done Sent Done false
29275 8227 Acked Write 10 Done Sent Done false
29276 8228 Acked Read 8 Done Sent Done false
29277 8229 Acked Write 9 Done Sent Done false
29278 8230 Acked Write 5 Done Sent Done false
29279 8231 Acked Read 2 Done Sent Done false
29280 8232 Acked Read 8 Done Sent Done false
29281 8233 Acked Write 7 Done Sent Done false
29282 8234 Acked Write 7 Done Sent Done false
29283 8235 Acked Flush 0 Done Sent Done false
29284 8236 Acked Read 3 Done Sent Done false
29285 8237 Acked Flush 0 Done Sent Done false
29286 8238 Acked Write 8 Done Sent Done false
29287 8239 Acked Write 8 Done Sent Done false
29288 8240 Acked Flush 0 Done Sent Done false
29289 8241 Acked Read 4 Done Sent Done false
29290 8242 Acked Write 6 Done Sent Done false
29291 8243 Acked Read 9 Done Sent Done false
29292 8244 Acked Flush 0 Done Sent Done false
29293 8245 Acked Flush 0 Done Sent Done false
29294 8246 Acked Read 3 Done Sent Done false
29295 8247 Acked Flush 0 Done Sent Done false
29296 STATES DS:0 DS:1 DS:2 TOTAL
29297 Sent 0 302 0 302
29298 Done 302 0 302 604
29299 Skipped 0 0 0 0
29300 Error 0 0 0 0
29301 Last Flush: 8247 7945 8247
29302 Downstairs last five completed: 7945 7944 7943 7942 7941
29303 Upstairs last five completed: 8247 8246 8245 8244 8243
29304 CLIENT: Up:0 ds:302 act:2
293052024-12-11T23:46:38.171ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293062024-12-11T23:46:38.171ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55734) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293072024-12-11T23:46:38.172ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293082024-12-11T23:46:38.172ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293092024-12-11T23:46:38.172ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293102024-12-11T23:46:38.172ZINFOcrucible: send last flush ID to this DS: 7945 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293112024-12-11T23:46:38.173ZINFOcrucible: Replied this last flush ID: 7945 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293122024-12-11T23:46:38.173ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293132024-12-11T23:46:38.173ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293142024-12-11T23:46:38.173ZINFOcrucible: [1] client re-new 302 jobs since flush 7945 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293152024-12-11T23:46:38.173ZINFOcrucible: [1] Replayed 302 jobs since flush: 7945 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
29316 ----------------------------------------------------------------
29317 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29318 Downstairs last five completed: 8247 8246 8245 8244 8243
29319 Upstairs last five completed: 8247 8246 8245 8244 8243
29320 CLIENT: Up:0 ds:0 act:3
29321 Replay: All jobs finished
29322 Main task has work to do, go find it
29323 got dsc Stop(1)
29324 stop 1
29325 [8820] Got stop action so:false kr:false
29326 Dec 11 23:46:40.401 INFO request completed, latency_us: 345, response_code: 204, uri: /stop/cid/1, method: POST, req_id: fbde2b5d-c1cc-4b95-a197-b998bd0514b4, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
29327 Dec 11 23:46:40.402 INFO request completed, latency_us: 138, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3080df83-64cd-4146-9970-f67838ef5f44, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
293282024-12-11T23:46:40.402ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293292024-12-11T23:46:40.402ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293302024-12-11T23:46:40.402ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293312024-12-11T23:46:40.402ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293322024-12-11T23:46:40.402ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
293332024-12-11T23:46:40.402ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
29334 [8820] Exited with: ExitStatus(unix_wait_status(9))
29335 [8820][1] reports Exit
29336 Dec 11 23:46:44.404 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/1, method: GET, req_id: 89fa8721-6f5d-46b9-b35e-9d9951cf0fe5, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
29337 001/300 Write block 4880 len 16384 data: 2 4 4 4
29338 002/300 Read block 4569 len 8192
29339 003/300 Write block 2549 len 8192 data: 8 6
29340 004/300 Read block 1429 len 16384
29341 005/300 Read block 358 len 8192
29342 006/300 Flush
29343 007/300 Read block 25 len 4096
29344 008/300 Flush
29345 009/300 Write block 1145 len 4096 data: 5
29346 010/300 Write block 1331 len 28672 data: 2 2 2 3 3 2 3
29347 011/300 Write block 2833 len 12288 data: 9 6 9
29348 012/300 Write block 293 len 8192 data: 6 6
29349 013/300 Flush
29350 014/300 Write block 4085 len 36864 data: 5 5 6 6 5 6 7 5 5
29351 015/300 Read block 2738 len 32768
29352 016/300 Read block 2760 len 4096
29353 017/300 Read block 4707 len 36864
29354 018/300 Read block 186 len 20480
29355 019/300 Write block 2846 len 32768 data: 5 5 4 4 5 5 6 5
29356 020/300 Write block 3652 len 20480 data: 10 9 6 6 7
29357 021/300 Read block 1162 len 24576
29358 022/300 Read block 4231 len 8192
29359 023/300 Read block 3930 len 20480
29360 024/300 Write block 1065 len 28672 data: 7 5 6 5 4 5 5
29361 025/300 Read block 2625 len 24576
29362 026/300 Flush
29363 027/300 Write block 1710 len 28672 data: 4 3 4 5 5 5 4
29364 028/300 Read block 360 len 28672
29365 029/300 Write block 2364 len 4096 data: 7
29366 030/300 Write block 1699 len 24576 data: 7 5 5 5 4 6
29367 031/300 Read block 2244 len 32768
29368 032/300 Read block 2086 len 20480
29369 033/300 Write block 4389 len 24576 data: 7 6 6 5 4 6
29370 034/300 Read block 1321 len 16384
29371 035/300 Read block 919 len 8192
29372 036/300 Read block 2993 len 32768
29373 037/300 Read block 3759 len 4096
29374 038/300 Write block 1777 len 28672 data: 4 3 5 6 9 8 7
29375 039/300 Read block 10 len 12288
29376 040/300 Read block 4534 len 20480
29377 041/300 Read block 3264 len 20480
29378 042/300 Read block 3149 len 40960
29379 043/300 Read block 1555 len 4096
29380 044/300 Write block 3032 len 24576 data: 5 5 6 6 5 3
29381 045/300 Write block 3403 len 28672 data: 3 3 3 4 3 4 4
29382 046/300 Flush
29383 047/300 Read block 2018 len 24576
29384 048/300 Write block 4504 len 28672 data: 2 3 3 4 4 4 4
29385 049/300 Read block 1720 len 40960
29386 050/300 Read block 315 len 8192
29387 051/300 Read block 8 len 4096
29388 052/300 Flush
29389 053/300 Flush
29390 054/300 Read block 3569 len 20480
29391 055/300 Write block 4711 len 32768 data: 3 3 3 4 5 5 6 7
29392 056/300 Read block 2688 len 36864
29393 057/300 Write block 3006 len 8192 data: 4 6
29394 058/300 Write block 2476 len 28672 data: 2 2 3 4 4 3 4
29395 059/300 Read block 3980 len 36864
29396 060/300 Write block 1092 len 40960 data: 9 7 6 6 6 7 7 6 8 8
29397 061/300 Write block 4151 len 32768 data: 5 5 5 5 5 3 3 2
29398 062/300 Write block 107 len 8192 data: 6 7
29399 063/300 Write block 669 len 36864 data: 5 4 3 3 3 3 4 4 3
29400 064/300 Write block 2133 len 8192 data: 4 4
29401 065/300 Write block 1589 len 28672 data: 4 4 4 4 4 4 6
29402 066/300 Write block 3626 len 36864 data: 3 4 6 7 7 7 6 6 5
29403 067/300 Read block 4401 len 24576
29404 068/300 Write block 2507 len 36864 data: 4 5 3 3 5 5 5 5 6
29405 069/300 Read block 4336 len 32768
29406 070/300 Read block 3461 len 20480
29407 071/300 Flush
29408 072/300 Read block 2359 len 28672
29409 073/300 Flush
29410 074/300 Read block 3927 len 32768
29411 075/300 Write block 3530 len 36864 data: 7 6 7 5 5 5 3 3 3
29412 076/300 Read block 2280 len 32768
29413 077/300 Write block 1490 len 40960 data: 4 6 8 8 6 6 6 6 8 8
29414 078/300 Flush
29415 079/300 Write block 4584 len 24576 data: 6 6 5 5 5 4
29416 080/300 Write block 1338 len 12288 data: 3 3 2
29417 081/300 Write block 4498 len 24576 data: 6 4 4 5 2 2
29418 082/300 Write block 2767 len 40960 data: 7 6 7 5 5 5 5 5 6 5
29419 083/300 Write block 398 len 16384 data: 5 3 3 3
29420 084/300 Write block 2509 len 40960 data: 4 4 6 6 6 6 7 5 5 4
29421 085/300 Read block 3669 len 28672
29422 086/300 Write block 2539 len 4096 data: 4
29423 087/300 Write block 4966 len 40960 data: 4 4 7 7 7 8 8 8 6 4
29424 088/300 Write block 3181 len 16384 data: 5 5 4 3
29425 089/300 Write block 3253 len 32768 data: 3 3 3 4 4 5 6 6
29426 090/300 Write block 2276 len 4096 data: 3
29427 091/300 Read block 882 len 20480
29428 092/300 Write block 4565 len 8192 data: 5 4
29429 093/300 Read block 1453 len 20480
29430 094/300 Read block 91 len 8192
29431 095/300 Write block 4421 len 16384 data: 4 4 4 7
29432 096/300 Read block 1901 len 12288
29433 097/300 Flush
29434 098/300 Read block 2102 len 16384
29435 099/300 Read block 135 len 40960
29436 100/300 Write block 1264 len 4096 data: 5
29437 101/300 Write block 274 len 24576 data: 6 4 4 4 3 2
29438 102/300 Write block 3706 len 16384 data: 6 6 5 5
29439 103/300 Read block 590 len 40960
29440 104/300 Read block 2096 len 4096
29441 105/300 Read block 323 len 40960
29442 106/300 Write block 1002 len 40960 data: 7 6 6 6 7 7 7 7 6 7
29443 107/300 Read block 3006 len 4096
29444 108/300 Read block 3323 len 36864
29445 109/300 Write block 490 len 12288 data: 5 6 5
29446 110/300 Read block 3925 len 28672
29447 111/300 Flush
29448 112/300 Read block 4668 len 32768
29449 113/300 Write block 1524 len 40960 data: 6 6 5 5 4 4 4 4 4 3
29450 114/300 Write block 943 len 12288 data: 6 5 5
29451 115/300 Write block 748 len 28672 data: 3 4 4 5 7 6 7
29452 116/300 Write block 1539 len 40960 data: 4 5 7 6 7 9 8 8 6 5
29453 117/300 Write block 2541 len 28672 data: 3 3 5 6 6 8 8
29454 118/300 Read block 1449 len 20480
29455 119/300 Write block 1439 len 8192 data: 8 10
29456 120/300 Write block 3373 len 16384 data: 5 5 4 3
29457 121/300 Read block 2844 len 40960
29458 122/300 Write block 4604 len 12288 data: 8 7 6
29459 123/300 Flush
29460 124/300 Read block 3562 len 36864
29461 125/300 Read block 2066 len 12288
29462 126/300 Write block 3442 len 4096 data: 6
29463 127/300 Read block 4467 len 12288
29464 128/300 Write block 2729 len 36864 data: 6 6 7 8 11 11 12 12 12
29465 129/300 Write block 4124 len 40960 data: 9 9 6 5 6 5 4 4 5 4
29466 130/300 Write block 1502 len 8192 data: 8 8
29467 131/300 Write block 3893 len 36864 data: 6 7 7 7 8 8 7 7 7
29468 132/300 Write block 1450 len 20480 data: 5 6 8 6 5
29469 133/300 Flush
29470 134/300 Write block 2471 len 36864 data: 4 3 2 2 2 3 3 4 5
29471 135/300 Write block 3920 len 36864 data: 4 4 6 5 5 5 6 5 4
29472 136/300 Write block 4722 len 40960 data: 9 11 9 7 7 7 6 6 3 4
29473 137/300 Write block 4025 len 24576 data: 6 5 4 3 2 3
29474 138/300 Read block 3021 len 28672
29475 139/300 Write block 737 len 16384 data: 3 4 4 4
29476 140/300 Read block 1022 len 40960
29477 141/300 Flush
29478 142/300 Write block 1992 len 12288 data: 6 5 4
29479 143/300 Read block 3784 len 36864
29480 144/300 Read block 4531 len 32768
29481 145/300 Flush
29482 146/300 Write block 1031 len 40960 data: 6 7 7 8 8 6 3 2 3 4
29483 147/300 Write block 4946 len 24576 data: 5 4 4 3 3 4
29484 148/300 Read block 4304 len 20480
29485 149/300 Flush
29486 150/300 Write block 1611 len 4096 data: 3
29487 151/300 Read block 1505 len 20480
29488 152/300 Write block 74 len 16384 data: 7 7 8 8
29489 153/300 Write block 1217 len 12288 data: 5 5 5
29490 154/300 Read block 316 len 8192
29491 155/300 Write block 2583 len 40960 data: 6 6 4 4 3 5 4 4 4 3
29492 156/300 Write block 3437 len 4096 data: 5
29493 157/300 Read block 1065 len 4096
29494 158/300 Read block 3273 len 28672
29495 159/300 Flush
29496 160/300 Write block 4132 len 4096 data: 6
29497 161/300 Write block 401 len 40960 data: 4 3 5 5 5 6 6 8 9 10
29498 162/300 Write block 764 len 16384 data: 4 4 5 6
29499 163/300 Write block 1749 len 8192 data: 3 2
29500 164/300 Read block 4805 len 12288
29501 165/300 Read block 1147 len 32768
29502 166/300 Write block 2836 len 4096 data: 7
29503 167/300 Read block 1537 len 4096
29504 168/300 Flush
29505 169/300 Read block 4872 len 28672
29506 170/300 Read block 3384 len 8192
29507 171/300 Flush
29508 172/300 Write block 1657 len 16384 data: 3 3 4 7
29509 173/300 Read block 1477 len 24576
29510 174/300 Read block 3531 len 4096
29511 175/300 Flush
29512 176/300 Read block 4663 len 40960
29513 177/300 Write block 1245 len 40960 data: 4 7 7 8 5 4 5 5 4 4
29514 178/300 Flush
29515 179/300 Write block 3864 len 36864 data: 14 12 9 10 9 8 8 8 7
29516 180/300 Read block 4201 len 8192
29517 181/300 Write block 2714 len 32768 data: 11 9 12 9 10 11 10 10
29518 182/300 Write block 812 len 16384 data: 6 7 7 6
29519 183/300 Flush
29520 184/300 Flush
29521 185/300 Write block 4259 len 8192 data: 3 3
29522 186/300 Read block 4640 len 32768
29523 187/300 Read block 3733 len 8192
29524 188/300 Read block 411 len 20480
29525 189/300 Write block 1532 len 8192 data: 5 4
29526 190/300 Read block 3263 len 20480
29527 191/300 Read block 761 len 40960
29528 192/300 Flush
29529 193/300 Read block 3405 len 32768
29530 194/300 Read block 1679 len 28672
29531 195/300 Read block 332 len 36864
29532 196/300 Write block 2260 len 40960 data: 4 5 5 3 4 5 4 6 6 6
29533 197/300 Read block 4578 len 8192
29534 198/300 Flush
29535 199/300 Write block 2342 len 36864 data: 6 4 4 4 8 8 7 7 7
29536 200/300 Read block 4175 len 32768
29537 201/300 Write block 3369 len 36864 data: 4 4 6 6 6 6 5 4 3
29538 202/300 Read block 1761 len 36864
29539 203/300 Read block 2057 len 8192
29540 204/300 Write block 1599 len 8192 data: 4 4
29541 205/300 Flush
29542 206/300 Read block 4293 len 4096
29543 207/300 Read block 378 len 8192
29544 208/300 Read block 1128 len 24576
29545 209/300 Read block 3408 len 8192
29546 210/300 Read block 4447 len 4096
29547 211/300 Read block 3228 len 36864
29548 212/300 Read block 4259 len 4096
29549 213/300 Write block 4053 len 24576 data: 5 5 4 4 5 4
29550 214/300 Read block 2765 len 8192
29551 215/300 Read block 4239 len 8192
29552 216/300 Read block 4318 len 12288
29553 217/300 Write block 1623 len 32768 data: 7 9 10 10 8 7 8 7
29554 218/300 Write block 417 len 12288 data: 5 5 3
29555 219/300 Read block 3682 len 20480
29556 220/300 Write block 1961 len 8192 data: 4 5
29557 221/300 Read block 1543 len 12288
29558 222/300 Write block 2110 len 20480 data: 5 4 4 6 7
29559 223/300 Read block 3743 len 4096
29560 224/300 Read block 223 len 20480
29561 225/300 Read block 2736 len 28672
29562 226/300 Write block 3885 len 4096 data: 8
29563 227/300 Flush
29564 228/300 Flush
29565 229/300 Write block 4562 len 28672 data: 6 6 5 6 5 3 5
29566 230/300 Read block 4918 len 40960
29567 231/300 Flush
29568 232/300 Read block 489 len 40960
29569 233/300 Write block 2061 len 24576 data: 2 4 5 4 6 7
29570 234/300 Write block 672 len 28672 data: 4 4 4 5 5 4 3
29571 235/300 Write block 997 len 8192 data: 4 5
29572 236/300 Write block 4565 len 4096 data: 7
29573 237/300 Read block 4822 len 36864
29574 238/300 Flush
29575 239/300 Read block 2548 len 12288
29576 240/300 Write block 4044 len 20480 data: 8 8 8 8 6
29577 241/300 Read block 2106 len 12288
29578 242/300 Read block 4198 len 4096
29579 243/300 Read block 2964 len 8192
29580 244/300 Read block 691 len 28672
29581 245/300 Write block 583 len 12288 data: 7 9 7
29582 246/300 Read block 1863 len 12288
29583 247/300 Write block 3425 len 20480 data: 5 5 5 5 5
29584 248/300 Read block 992 len 36864
29585 249/300 Read block 1471 len 8192
29586 250/300 Read block 1643 len 20480
29587 251/300 Read block 4345 len 24576
29588 252/300 Read block 3661 len 16384
29589 253/300 Read block 596 len 8192
29590 254/300 Write block 1017 len 32768 data: 4 6 6 4 5 5 4 4
29591 255/300 Read block 3740 len 28672
29592 256/300 Read block 4026 len 16384
29593 257/300 Read block 1237 len 40960
29594 258/300 Write block 4659 len 8192 data: 4 5
29595 259/300 Write block 3660 len 24576 data: 6 7 8 5 4 4
29596 260/300 Write block 417 len 36864 data: 6 6 4 2 2 2 2 2 2
29597 261/300 Read block 4346 len 24576
29598 262/300 Read block 852 len 36864
29599 263/300 Read block 4892 len 32768
29600 264/300 Read block 3318 len 32768
29601 265/300 Read block 4533 len 16384
29602 266/300 Read block 4430 len 36864
29603 267/300 Write block 1696 len 12288 data: 7 9 10
29604 268/300 Read block 4650 len 40960
29605 269/300 Read block 3962 len 12288
29606 270/300 Read block 1044 len 12288
29607 271/300 Read block 4472 len 24576
29608 272/300 Read block 4751 len 16384
29609 273/300 Flush
29610 274/300 Write block 4954 len 20480 data: 3 4 6 7 8
29611 275/300 Flush
29612 276/300 Write block 3321 len 36864 data: 6 6 7 6 6 8 8 9 10
29613 277/300 Write block 529 len 20480 data: 3 3 4 4 4
29614 278/300 Read block 3845 len 40960
29615 279/300 Write block 4179 len 32768 data: 6 6 7 6 5 5 4 5
29616 280/300 Write block 590 len 24576 data: 5 5 5 4 4 4
29617 281/300 Write block 1836 len 20480 data: 8 8 8 7 7
29618 282/300 Read block 1175 len 8192
29619 283/300 Write block 1973 len 36864 data: 6 5 6 6 6 6 6 6 8
29620 284/300 Flush
29621 285/300 Read block 454 len 40960
29622 286/300 Write block 1010 len 12288 data: 7 8 5
29623 287/300 Flush
29624 288/300 Write block 1821 len 32768 data: 3 2 2 4 4 3 3 3
29625 289/300 Read block 3895 len 20480
29626 290/300 Read block 7 len 16384
29627 291/300 Write block 2740 len 32768 data: 9 8 8 6 7 5 5 5
29628 292/300 Read block 4829 len 40960
29629 293/300 Write block 1071 len 12288 data: 6 4 5
29630 294/300 Write block 1656 len 8192 data: 3 4
29631 295/300 Write block 4605 len 20480 data: 8 7 6 5 3
29632 296/300 Read block 1783 len 28672
29633 297/300 Read block 3404 len 8192
29634 298/300 Flush
29635 299/300 Read block 1680 len 16384
29636 300/300 Read block 4766 len 36864
29637 301/300 Read block 3186 len 16384
29638 Main task has work to do, go find it
29639 got dsc Start(1)
29640 start 1
29641 Dec 11 23:46:44.639 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6d2723c5-f56d-4c4b-b565-cc0e1c6dbb2d, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
29642 Replay: started 1, returned:Ok(())
29643 ----------------------------------------------------------------
29644 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29645 States: Active Offline Active
29646 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29647 8248 Acked Write 4 Done Sent Done false
29648 8249 Acked Read 2 Done Sent Done false
29649 8250 Acked Write 2 Done Sent Done false
29650 8251 Acked Read 4 Done Sent Done false
29651 8252 Acked Read 2 Done Sent Done false
29652 8253 Acked Flush 0 Done Sent Done false
29653 8254 Acked Read 1 Done Sent Done false
29654 8255 Acked Flush 0 Done Sent Done false
29655 8256 Acked Write 1 Done Sent Done false
29656 8257 Acked Write 7 Done Sent Done false
29657 8258 Acked Write 3 Done Sent Done false
29658 8259 Acked Write 2 Done Sent Done false
29659 8260 Acked Flush 0 Done Sent Done false
29660 8261 Acked Write 9 Done Sent Done false
29661 8262 Acked Read 8 Done Sent Done false
29662 8263 Acked Read 1 Done Sent Done false
29663 8264 Acked Read 9 Done Sent Done false
29664 8265 Acked Read 5 Done Sent Done false
29665 8266 Acked Write 8 Done Sent Done false
29666 8267 Acked Write 5 Done Sent Done false
29667 8268 Acked Read 6 Done Sent Done false
29668 8269 Acked Read 2 Done Sent Done false
29669 8270 Acked Read 5 Done Sent Done false
29670 8271 Acked Write 7 Done Sent Done false
29671 8272 Acked Read 6 Done Sent Done false
29672 8273 Acked Flush 0 Done Sent Done false
29673 8274 Acked Write 7 Done Sent Done false
29674 8275 Acked Read 7 Done Sent Done false
29675 8276 Acked Write 1 Done Sent Done false
29676 8277 Acked Write 6 Done Sent Done false
29677 8278 Acked Read 8 Done Sent Done false
29678 8279 Acked Read 5 Done Sent Done false
29679 8280 Acked Write 6 Done Sent Done false
29680 8281 Acked Read 4 Done Sent Done false
29681 8282 Acked Read 2 Done Sent Done false
29682 8283 Acked Read 8 Done Sent Done false
29683 8284 Acked Read 1 Done Sent Done false
29684 8285 Acked Write 7 Done Sent Done false
29685 8286 Acked Read 3 Done Sent Done false
29686 8287 Acked Read 5 Done Sent Done false
29687 8288 Acked Read 5 Done Sent Done false
29688 8289 Acked Read 10 Done Sent Done false
29689 8290 Acked Read 1 Done Sent Done false
29690 8291 Acked Write 6 Done Sent Done false
29691 8292 Acked Write 7 Done Sent Done false
29692 8293 Acked Flush 0 Done Sent Done false
29693 8294 Acked Read 6 Done Sent Done false
29694 8295 Acked Write 7 Done Sent Done false
29695 8296 Acked Read 10 Done Sent Done false
29696 8297 Acked Read 2 Done Sent Done false
29697 8298 Acked Read 1 Done Sent Done false
29698 8299 Acked Flush 0 Done Sent Done false
29699 8300 Acked Flush 0 Done Sent Done false
29700 8301 Acked Read 5 Done Sent Done false
29701 8302 Acked Write 8 Done Sent Done false
29702 8303 Acked Read 9 Done Sent Done false
29703 8304 Acked Write 2 Done Sent Done false
29704 8305 Acked Write 7 Done Sent Done false
29705 8306 Acked Read 9 Done Sent Done false
29706 8307 Acked Write 10 Done Sent Done false
29707 8308 Acked Write 8 Done Sent Done false
29708 8309 Acked Write 2 Done Sent Done false
29709 8310 Acked Write 9 Done Sent Done false
29710 8311 Acked Write 2 Done Sent Done false
29711 8312 Acked Write 7 Done Sent Done false
29712 8313 Acked Write 9 Done Sent Done false
29713 8314 Acked Read 6 Done Sent Done false
29714 8315 Acked Write 9 Done Sent Done false
29715 8316 Acked Read 8 Done Sent Done false
29716 8317 Acked Read 5 Done Sent Done false
29717 8318 Acked Flush 0 Done Sent Done false
29718 8319 Acked Read 7 Done Sent Done false
29719 8320 Acked Flush 0 Done Sent Done false
29720 8321 Acked Read 8 Done Sent Done false
29721 8322 Acked Write 9 Done Sent Done false
29722 8323 Acked Read 8 Done Sent Done false
29723 8324 Acked Write 10 Done Sent Done false
29724 8325 Acked Flush 0 Done Sent Done false
29725 8326 Acked Write 6 Done Sent Done false
29726 8327 Acked Write 3 Done Sent Done false
29727 8328 Acked Write 6 Done Sent Done false
29728 8329 Acked Write 10 Done Sent Done false
29729 8330 Acked Write 4 Done Sent Done false
29730 8331 Acked Write 10 Done Sent Done false
29731 8332 Acked Read 7 Done Sent Done false
29732 8333 Acked Write 1 Done Sent Done false
29733 8334 Acked Write 10 Done Sent Done false
29734 8335 Acked Write 4 Done Sent Done false
29735 8336 Acked Write 8 Done Sent Done false
29736 8337 Acked Write 1 Done Sent Done false
29737 8338 Acked Read 5 Done Sent Done false
29738 8339 Acked Write 2 Done Sent Done false
29739 8340 Acked Read 5 Done Sent Done false
29740 8341 Acked Read 2 Done Sent Done false
29741 8342 Acked Write 4 Done Sent Done false
29742 8343 Acked Read 3 Done Sent Done false
29743 8344 Acked Flush 0 Done Sent Done false
29744 8345 Acked Read 4 Done Sent Done false
29745 8346 Acked Read 10 Done Sent Done false
29746 8347 Acked Write 1 Done Sent Done false
29747 8348 Acked Write 6 Done Sent Done false
29748 8349 Acked Write 4 Done Sent Done false
29749 8350 Acked Read 10 Done Sent Done false
29750 8351 Acked Read 1 Done Sent Done false
29751 8352 Acked Read 10 Done Sent Done false
29752 8353 Acked Write 10 Done Sent Done false
29753 8354 Acked Read 1 Done Sent Done false
29754 8355 Acked Read 9 Done Sent Done false
29755 8356 Acked Write 3 Done Sent Done false
29756 8357 Acked Read 7 Done Sent Done false
29757 8358 Acked Flush 0 Done Sent Done false
29758 8359 Acked Read 8 Done Sent Done false
29759 8360 Acked Write 10 Done Sent Done false
29760 8361 Acked Write 3 Done Sent Done false
29761 8362 Acked Write 7 Done Sent Done false
29762 8363 Acked Write 10 Done Sent Done false
29763 8364 Acked Write 7 Done Sent Done false
29764 8365 Acked Read 5 Done Sent Done false
29765 8366 Acked Write 2 Done Sent Done false
29766 8367 Acked Write 4 Done Sent Done false
29767 8368 Acked Read 10 Done Sent Done false
29768 8369 Acked Write 3 Done Sent Done false
29769 8370 Acked Flush 0 Done Sent Done false
29770 8371 Acked Read 9 Done Sent Done false
29771 8372 Acked Read 3 Done Sent Done false
29772 8373 Acked Write 1 Done Sent Done false
29773 8374 Acked Read 3 Done Sent Done false
29774 8375 Acked Write 9 Done Sent Done false
29775 8376 Acked Write 10 Done Sent Done false
29776 8377 Acked Write 2 Done Sent Done false
29777 8378 Acked Write 9 Done Sent Done false
29778 8379 Acked Write 5 Done Sent Done false
29779 8380 Acked Flush 0 Done Sent Done false
29780 8381 Acked Write 9 Done Sent Done false
29781 8382 Acked Write 9 Done Sent Done false
29782 8383 Acked Write 10 Done Sent Done false
29783 8384 Acked Write 6 Done Sent Done false
29784 8385 Acked Read 7 Done Sent Done false
29785 8386 Acked Write 4 Done Sent Done false
29786 8387 Acked Read 10 Done Sent Done false
29787 8388 Acked Flush 0 Done Sent Done false
29788 8389 Acked Write 3 Done Sent Done false
29789 8390 Acked Read 9 Done Sent Done false
29790 8391 Acked Read 8 Done Sent Done false
29791 8392 Acked Flush 0 Done Sent Done false
29792 8393 Acked Write 10 Done Sent Done false
29793 8394 Acked Write 6 Done Sent Done false
29794 8395 Acked Read 5 Done Sent Done false
29795 8396 Acked Flush 0 Done Sent Done false
29796 8397 Acked Write 1 Done Sent Done false
29797 8398 Acked Read 5 Done Sent Done false
29798 8399 Acked Write 4 Done Sent Done false
29799 8400 Acked Write 3 Done Sent Done false
29800 8401 Acked Read 2 Done Sent Done false
29801 8402 Acked Write 10 Done Sent Done false
29802 8403 Acked Write 1 Done Sent Done false
29803 8404 Acked Read 1 Done Sent Done false
29804 8405 Acked Read 7 Done Sent Done false
29805 8406 Acked Flush 0 Done Sent Done false
29806 8407 Acked Write 1 Done Sent Done false
29807 8408 Acked Write 10 Done Sent Done false
29808 8409 Acked Write 4 Done Sent Done false
29809 8410 Acked Write 2 Done Sent Done false
29810 8411 Acked Read 3 Done Sent Done false
29811 8412 Acked Read 8 Done Sent Done false
29812 8413 Acked Write 1 Done Sent Done false
29813 8414 Acked Read 1 Done Sent Done false
29814 8415 Acked Flush 0 Done Sent Done false
29815 8416 Acked Read 7 Done Sent Done false
29816 8417 Acked Read 2 Done Sent Done false
29817 8418 Acked Flush 0 Done Sent Done false
29818 8419 Acked Write 4 Done Sent Done false
29819 8420 Acked Read 6 Done Sent Done false
29820 8421 Acked Read 1 Done Sent Done false
29821 8422 Acked Flush 0 Done Sent Done false
29822 8423 Acked Read 10 Done Sent Done false
29823 8424 Acked Write 10 Done Sent Done false
29824 8425 Acked Flush 0 Done Sent Done false
29825 8426 Acked Write 9 Done Sent Done false
29826 8427 Acked Read 2 Done Sent Done false
29827 8428 Acked Write 8 Done Sent Done false
29828 8429 Acked Write 4 Done Sent Done false
29829 8430 Acked Flush 0 Done Sent Done false
29830 8431 Acked Flush 0 Done Sent Done false
29831 8432 Acked Write 2 Done Sent Done false
29832 8433 Acked Read 8 Done Sent Done false
29833 8434 Acked Read 2 Done Sent Done false
29834 8435 Acked Read 5 Done Sent Done false
29835 8436 Acked Write 2 Done Sent Done false
29836 8437 Acked Read 5 Done Sent Done false
29837 8438 Acked Read 10 Done Sent Done false
29838 8439 Acked Flush 0 Done Sent Done false
29839 8440 Acked Read 8 Done Sent Done false
29840 8441 Acked Read 7 Done Sent Done false
29841 8442 Acked Read 9 Done Sent Done false
29842 8443 Acked Write 10 Done Sent Done false
29843 8444 Acked Read 2 Done Sent Done false
29844 8445 Acked Flush 0 Done Sent Done false
29845 8446 Acked Write 9 Done Sent Done false
29846 8447 Acked Read 8 Done Sent Done false
29847 8448 Acked Write 9 Done Sent Done false
29848 8449 Acked Read 9 Done Sent Done false
29849 8450 Acked Read 2 Done Sent Done false
29850 8451 Acked Write 2 Done Sent Done false
29851 8452 Acked Flush 0 Done Sent Done false
29852 8453 Acked Read 1 Done Sent Done false
29853 8454 Acked Read 2 Done Sent Done false
29854 8455 Acked Read 6 Done Sent Done false
29855 8456 Acked Read 2 Done Sent Done false
29856 8457 Acked Read 1 Done Sent Done false
29857 8458 Acked Read 9 Done Sent Done false
29858 8459 Acked Read 1 Done Sent Done false
29859 8460 Acked Write 6 Done Sent Done false
29860 8461 Acked Read 2 Done Sent Done false
29861 8462 Acked Read 2 Done Sent Done false
29862 8463 Acked Read 3 Done Sent Done false
29863 8464 Acked Write 8 Done Sent Done false
29864 8465 Acked Write 3 Done Sent Done false
29865 8466 Acked Read 5 Done Sent Done false
29866 8467 Acked Write 2 Done Sent Done false
29867 8468 Acked Read 3 Done Sent Done false
29868 8469 Acked Write 5 Done Sent Done false
29869 8470 Acked Read 1 Done Sent Done false
29870 8471 Acked Read 5 Done Sent Done false
29871 8472 Acked Read 7 Done Sent Done false
29872 8473 Acked Write 1 Done Sent Done false
29873 8474 Acked Flush 0 Done Sent Done false
29874 8475 Acked Flush 0 Done Sent Done false
29875 8476 Acked Write 7 Done Sent Done false
29876 8477 Acked Read 10 Done Sent Done false
29877 8478 Acked Flush 0 Done Sent Done false
29878 8479 Acked Read 10 Done Sent Done false
29879 8480 Acked Write 6 Done Sent Done false
29880 8481 Acked Write 7 Done Sent Done false
29881 8482 Acked Write 2 Done Sent Done false
29882 8483 Acked Write 1 Done Sent Done false
29883 8484 Acked Read 9 Done Sent Done false
29884 8485 Acked Flush 0 Done Sent Done false
29885 8486 Acked Read 3 Done Sent Done false
29886 8487 Acked Write 5 Done Sent Done false
29887 8488 Acked Read 3 Done Sent Done false
29888 8489 Acked Read 1 Done Sent Done false
29889 8490 Acked Read 2 Done Sent Done false
29890 8491 Acked Read 7 Done Sent Done false
29891 8492 Acked Write 3 Done Sent Done false
29892 8493 Acked Read 3 Done Sent Done false
29893 8494 Acked Write 5 Done Sent Done false
29894 8495 Acked Read 9 Done Sent Done false
29895 8496 Acked Read 2 Done Sent Done false
29896 8497 Acked Read 5 Done Sent Done false
29897 8498 Acked Read 6 Done Sent Done false
29898 8499 Acked Read 4 Done Sent Done false
29899 8500 Acked Read 2 Done Sent Done false
29900 8501 Acked Write 8 Done Sent Done false
29901 8502 Acked Read 7 Done Sent Done false
29902 8503 Acked Read 4 Done Sent Done false
29903 8504 Acked Read 10 Done Sent Done false
29904 8505 Acked Write 2 Done Sent Done false
29905 8506 Acked Write 6 Done Sent Done false
29906 8507 Acked Write 9 Done Sent Done false
29907 8508 Acked Read 6 Done Sent Done false
29908 8509 Acked Read 9 Done Sent Done false
29909 8510 Acked Read 8 Done Sent Done false
29910 8511 Acked Read 8 Done Sent Done false
29911 8512 Acked Read 4 Done Sent Done false
29912 8513 Acked Read 9 Done Sent Done false
29913 8514 Acked Write 3 Done Sent Done false
29914 8515 Acked Read 10 Done Sent Done false
29915 8516 Acked Read 3 Done Sent Done false
29916 8517 Acked Read 3 Done Sent Done false
29917 8518 Acked Read 6 Done Sent Done false
29918 8519 Acked Read 4 Done Sent Done false
29919 8520 Acked Flush 0 Done Sent Done false
29920 8521 Acked Write 5 Done Sent Done false
29921 8522 Acked Flush 0 Done Sent Done false
29922 8523 Acked Write 9 Done Sent Done false
29923 8524 Acked Write 5 Done Sent Done false
29924 8525 Acked Read 10 Done Sent Done false
29925 8526 Acked Write 8 Done Sent Done false
29926 8527 Acked Write 6 Done Sent Done false
29927 8528 Acked Write 5 Done Sent Done false
29928 8529 Acked Read 2 Done Sent Done false
29929 8530 Acked Write 9 Done Sent Done false
29930 8531 Acked Flush 0 Done Sent Done false
29931 8532 Acked Read 10 Done Sent Done false
29932 8533 Acked Write 3 Done Sent Done false
29933 8534 Acked Flush 0 Done Sent Done false
29934 8535 Acked Write 8 Done Sent Done false
29935 8536 Acked Read 5 Done Sent Done false
29936 8537 Acked Read 4 Done Sent Done false
29937 8538 Acked Write 8 Done Sent Done false
29938 8539 Acked Read 10 Done Sent Done false
29939 8540 Acked Write 3 Done Sent Done false
29940 8541 Acked Write 2 Done Sent Done false
29941 8542 Acked Write 5 Done Sent Done false
29942 8543 Acked Read 7 Done Sent Done false
29943 8544 Acked Read 2 Done Sent Done false
29944 8545 Acked Flush 0 Done Sent Done false
29945 8546 Acked Read 4 Done Sent Done false
29946 8547 Acked Read 9 Done Sent Done false
29947 8548 Acked Read 4 Done Sent Done false
29948 STATES DS:0 DS:1 DS:2 TOTAL
29949 Sent 0 301 0 301
29950 Done 301 0 301 602
29951 Skipped 0 0 0 0
29952 Error 0 0 0 0
29953 Last Flush: 8545 8247 8545
29954 Downstairs last five completed: 8247 8246 8245 8244 8243
29955 Upstairs last five completed: 8548 8547 8546 8545 8544
29956 CLIENT: Up:0 ds:301 act:2
29957 ----------------------------------------------------------------
29958 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29959 States: Active Offline Active
29960 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29961 8248 Acked Write 4 Done Sent Done false
29962 8249 Acked Read 2 Done Sent Done false
29963 8250 Acked Write 2 Done Sent Done false
29964 8251 Acked Read 4 Done Sent Done false
29965 8252 Acked Read 2 Done Sent Done false
29966 8253 Acked Flush 0 Done Sent Done false
29967 8254 Acked Read 1 Done Sent Done false
29968 8255 Acked Flush 0 Done Sent Done false
29969 8256 Acked Write 1 Done Sent Done false
29970 8257 Acked Write 7 Done Sent Done false
29971 8258 Acked Write 3 Done Sent Done false
29972 8259 Acked Write 2 Done Sent Done false
29973 8260 Acked Flush 0 Done Sent Done false
29974 8261 Acked Write 9 Done Sent Done false
29975 8262 Acked Read 8 Done Sent Done false
29976 8263 Acked Read 1 Done Sent Done false
29977 8264 Acked Read 9 Done Sent Done false
29978 8265 Acked Read 5 Done Sent Done false
29979 8266 Acked Write 8 Done Sent Done false
29980 8267 Acked Write 5 Done Sent Done false
29981 8268 Acked Read 6 Done Sent Done false
29982 8269 Acked Read 2 Done Sent Done false
29983 8270 Acked Read 5 Done Sent Done false
29984 8271 Acked Write 7 Done Sent Done false
29985 8272 Acked Read 6 Done Sent Done false
29986 8273 Acked Flush 0 Done Sent Done false
29987 8274 Acked Write 7 Done Sent Done false
29988 8275 Acked Read 7 Done Sent Done false
29989 8276 Acked Write 1 Done Sent Done false
29990 8277 Acked Write 6 Done Sent Done false
29991 8278 Acked Read 8 Done Sent Done false
29992 8279 Acked Read 5 Done Sent Done false
29993 8280 Acked Write 6 Done Sent Done false
29994 8281 Acked Read 4 Done Sent Done false
29995 8282 Acked Read 2 Done Sent Done false
29996 8283 Acked Read 8 Done Sent Done false
29997 8284 Acked Read 1 Done Sent Done false
29998 8285 Acked Write 7 Done Sent Done false
29999 8286 Acked Read 3 Done Sent Done false
30000 8287 Acked Read 5 Done Sent Done false
30001 8288 Acked Read 5 Done Sent Done false
30002 8289 Acked Read 10 Done Sent Done false
30003 8290 Acked Read 1 Done Sent Done false
30004 8291 Acked Write 6 Done Sent Done false
30005 8292 Acked Write 7 Done Sent Done false
30006 8293 Acked Flush 0 Done Sent Done false
30007 8294 Acked Read 6 Done Sent Done false
30008 8295 Acked Write 7 Done Sent Done false
30009 8296 Acked Read 10 Done Sent Done false
30010 8297 Acked Read 2 Done Sent Done false
30011 8298 Acked Read 1 Done Sent Done false
30012 8299 Acked Flush 0 Done Sent Done false
30013 8300 Acked Flush 0 Done Sent Done false
30014 8301 Acked Read 5 Done Sent Done false
30015 8302 Acked Write 8 Done Sent Done false
30016 8303 Acked Read 9 Done Sent Done false
30017 8304 Acked Write 2 Done Sent Done false
30018 8305 Acked Write 7 Done Sent Done false
30019 8306 Acked Read 9 Done Sent Done false
30020 8307 Acked Write 10 Done Sent Done false
30021 8308 Acked Write 8 Done Sent Done false
30022 8309 Acked Write 2 Done Sent Done false
30023 8310 Acked Write 9 Done Sent Done false
30024 8311 Acked Write 2 Done Sent Done false
30025 8312 Acked Write 7 Done Sent Done false
30026 8313 Acked Write 9 Done Sent Done false
30027 8314 Acked Read 6 Done Sent Done false
30028 8315 Acked Write 9 Done Sent Done false
30029 8316 Acked Read 8 Done Sent Done false
30030 8317 Acked Read 5 Done Sent Done false
30031 8318 Acked Flush 0 Done Sent Done false
30032 8319 Acked Read 7 Done Sent Done false
30033 8320 Acked Flush 0 Done Sent Done false
30034 8321 Acked Read 8 Done Sent Done false
30035 8322 Acked Write 9 Done Sent Done false
30036 8323 Acked Read 8 Done Sent Done false
30037 8324 Acked Write 10 Done Sent Done false
30038 8325 Acked Flush 0 Done Sent Done false
30039 8326 Acked Write 6 Done Sent Done false
30040 8327 Acked Write 3 Done Sent Done false
30041 8328 Acked Write 6 Done Sent Done false
30042 8329 Acked Write 10 Done Sent Done false
30043 8330 Acked Write 4 Done Sent Done false
30044 8331 Acked Write 10 Done Sent Done false
30045 8332 Acked Read 7 Done Sent Done false
30046 8333 Acked Write 1 Done Sent Done false
30047 8334 Acked Write 10 Done Sent Done false
30048 8335 Acked Write 4 Done Sent Done false
30049 8336 Acked Write 8 Done Sent Done false
30050 8337 Acked Write 1 Done Sent Done false
30051 8338 Acked Read 5 Done Sent Done false
30052 8339 Acked Write 2 Done Sent Done false
30053 8340 Acked Read 5 Done Sent Done false
30054 8341 Acked Read 2 Done Sent Done false
30055 8342 Acked Write 4 Done Sent Done false
30056 8343 Acked Read 3 Done Sent Done false
30057 8344 Acked Flush 0 Done Sent Done false
30058 8345 Acked Read 4 Done Sent Done false
30059 8346 Acked Read 10 Done Sent Done false
30060 8347 Acked Write 1 Done Sent Done false
30061 8348 Acked Write 6 Done Sent Done false
30062 8349 Acked Write 4 Done Sent Done false
30063 8350 Acked Read 10 Done Sent Done false
30064 8351 Acked Read 1 Done Sent Done false
30065 8352 Acked Read 10 Done Sent Done false
30066 8353 Acked Write 10 Done Sent Done false
30067 8354 Acked Read 1 Done Sent Done false
30068 8355 Acked Read 9 Done Sent Done false
30069 8356 Acked Write 3 Done Sent Done false
30070 8357 Acked Read 7 Done Sent Done false
30071 8358 Acked Flush 0 Done Sent Done false
30072 8359 Acked Read 8 Done Sent Done false
30073 8360 Acked Write 10 Done Sent Done false
30074 8361 Acked Write 3 Done Sent Done false
30075 8362 Acked Write 7 Done Sent Done false
30076 8363 Acked Write 10 Done Sent Done false
30077 8364 Acked Write 7 Done Sent Done false
30078 8365 Acked Read 5 Done Sent Done false
30079 8366 Acked Write 2 Done Sent Done false
30080 8367 Acked Write 4 Done Sent Done false
30081 8368 Acked Read 10 Done Sent Done false
30082 8369 Acked Write 3 Done Sent Done false
30083 8370 Acked Flush 0 Done Sent Done false
30084 8371 Acked Read 9 Done Sent Done false
30085 8372 Acked Read 3 Done Sent Done false
30086 8373 Acked Write 1 Done Sent Done false
30087 8374 Acked Read 3 Done Sent Done false
30088 8375 Acked Write 9 Done Sent Done false
30089 8376 Acked Write 10 Done Sent Done false
30090 8377 Acked Write 2 Done Sent Done false
30091 8378 Acked Write 9 Done Sent Done false
30092 8379 Acked Write 5 Done Sent Done false
30093 8380 Acked Flush 0 Done Sent Done false
30094 8381 Acked Write 9 Done Sent Done false
30095 8382 Acked Write 9 Done Sent Done false
30096 8383 Acked Write 10 Done Sent Done false
30097 8384 Acked Write 6 Done Sent Done false
30098 8385 Acked Read 7 Done Sent Done false
30099 8386 Acked Write 4 Done Sent Done false
30100 8387 Acked Read 10 Done Sent Done false
30101 8388 Acked Flush 0 Done Sent Done false
30102 8389 Acked Write 3 Done Sent Done false
30103 8390 Acked Read 9 Done Sent Done false
30104 8391 Acked Read 8 Done Sent Done false
30105 8392 Acked Flush 0 Done Sent Done false
30106 8393 Acked Write 10 Done Sent Done false
30107 8394 Acked Write 6 Done Sent Done false
30108 8395 Acked Read 5 Done Sent Done false
30109 8396 Acked Flush 0 Done Sent Done false
30110 8397 Acked Write 1 Done Sent Done false
30111 8398 Acked Read 5 Done Sent Done false
30112 8399 Acked Write 4 Done Sent Done false
30113 8400 Acked Write 3 Done Sent Done false
30114 8401 Acked Read 2 Done Sent Done false
30115 8402 Acked Write 10 Done Sent Done false
30116 8403 Acked Write 1 Done Sent Done false
30117 8404 Acked Read 1 Done Sent Done false
30118 8405 Acked Read 7 Done Sent Done false
30119 8406 Acked Flush 0 Done Sent Done false
30120 8407 Acked Write 1 Done Sent Done false
30121 8408 Acked Write 10 Done Sent Done false
30122 8409 Acked Write 4 Done Sent Done false
30123 8410 Acked Write 2 Done Sent Done false
30124 8411 Acked Read 3 Done Sent Done false
30125 8412 Acked Read 8 Done Sent Done false
30126 8413 Acked Write 1 Done Sent Done false
30127 8414 Acked Read 1 Done Sent Done false
30128 8415 Acked Flush 0 Done Sent Done false
30129 8416 Acked Read 7 Done Sent Done false
30130 8417 Acked Read 2 Done Sent Done false
30131 8418 Acked Flush 0 Done Sent Done false
30132 8419 Acked Write 4 Done Sent Done false
30133 8420 Acked Read 6 Done Sent Done false
30134 8421 Acked Read 1 Done Sent Done false
30135 8422 Acked Flush 0 Done Sent Done false
30136 8423 Acked Read 10 Done Sent Done false
30137 8424 Acked Write 10 Done Sent Done false
30138 8425 Acked Flush 0 Done Sent Done false
30139 8426 Acked Write 9 Done Sent Done false
30140 8427 Acked Read 2 Done Sent Done false
30141 8428 Acked Write 8 Done Sent Done false
30142 8429 Acked Write 4 Done Sent Done false
30143 8430 Acked Flush 0 Done Sent Done false
30144 8431 Acked Flush 0 Done Sent Done false
30145 8432 Acked Write 2 Done Sent Done false
30146 8433 Acked Read 8 Done Sent Done false
30147 8434 Acked Read 2 Done Sent Done false
30148 8435 Acked Read 5 Done Sent Done false
30149 8436 Acked Write 2 Done Sent Done false
30150 8437 Acked Read 5 Done Sent Done false
30151 8438 Acked Read 10 Done Sent Done false
30152 8439 Acked Flush 0 Done Sent Done false
30153 8440 Acked Read 8 Done Sent Done false
30154 8441 Acked Read 7 Done Sent Done false
30155 8442 Acked Read 9 Done Sent Done false
30156 8443 Acked Write 10 Done Sent Done false
30157 8444 Acked Read 2 Done Sent Done false
30158 8445 Acked Flush 0 Done Sent Done false
30159 8446 Acked Write 9 Done Sent Done false
30160 8447 Acked Read 8 Done Sent Done false
30161 8448 Acked Write 9 Done Sent Done false
30162 8449 Acked Read 9 Done Sent Done false
30163 8450 Acked Read 2 Done Sent Done false
30164 8451 Acked Write 2 Done Sent Done false
30165 8452 Acked Flush 0 Done Sent Done false
30166 8453 Acked Read 1 Done Sent Done false
30167 8454 Acked Read 2 Done Sent Done false
30168 8455 Acked Read 6 Done Sent Done false
30169 8456 Acked Read 2 Done Sent Done false
30170 8457 Acked Read 1 Done Sent Done false
30171 8458 Acked Read 9 Done Sent Done false
30172 8459 Acked Read 1 Done Sent Done false
30173 8460 Acked Write 6 Done Sent Done false
30174 8461 Acked Read 2 Done Sent Done false
30175 8462 Acked Read 2 Done Sent Done false
30176 8463 Acked Read 3 Done Sent Done false
30177 8464 Acked Write 8 Done Sent Done false
30178 8465 Acked Write 3 Done Sent Done false
30179 8466 Acked Read 5 Done Sent Done false
30180 8467 Acked Write 2 Done Sent Done false
30181 8468 Acked Read 3 Done Sent Done false
30182 8469 Acked Write 5 Done Sent Done false
30183 8470 Acked Read 1 Done Sent Done false
30184 8471 Acked Read 5 Done Sent Done false
30185 8472 Acked Read 7 Done Sent Done false
30186 8473 Acked Write 1 Done Sent Done false
30187 8474 Acked Flush 0 Done Sent Done false
30188 8475 Acked Flush 0 Done Sent Done false
30189 8476 Acked Write 7 Done Sent Done false
30190 8477 Acked Read 10 Done Sent Done false
30191 8478 Acked Flush 0 Done Sent Done false
30192 8479 Acked Read 10 Done Sent Done false
30193 8480 Acked Write 6 Done Sent Done false
30194 8481 Acked Write 7 Done Sent Done false
30195 8482 Acked Write 2 Done Sent Done false
30196 8483 Acked Write 1 Done Sent Done false
30197 8484 Acked Read 9 Done Sent Done false
30198 8485 Acked Flush 0 Done Sent Done false
30199 8486 Acked Read 3 Done Sent Done false
30200 8487 Acked Write 5 Done Sent Done false
30201 8488 Acked Read 3 Done Sent Done false
30202 8489 Acked Read 1 Done Sent Done false
30203 8490 Acked Read 2 Done Sent Done false
30204 8491 Acked Read 7 Done Sent Done false
30205 8492 Acked Write 3 Done Sent Done false
30206 8493 Acked Read 3 Done Sent Done false
30207 8494 Acked Write 5 Done Sent Done false
30208 8495 Acked Read 9 Done Sent Done false
30209 8496 Acked Read 2 Done Sent Done false
30210 8497 Acked Read 5 Done Sent Done false
30211 8498 Acked Read 6 Done Sent Done false
30212 8499 Acked Read 4 Done Sent Done false
30213 8500 Acked Read 2 Done Sent Done false
30214 8501 Acked Write 8 Done Sent Done false
30215 8502 Acked Read 7 Done Sent Done false
30216 8503 Acked Read 4 Done Sent Done false
30217 8504 Acked Read 10 Done Sent Done false
30218 8505 Acked Write 2 Done Sent Done false
30219 8506 Acked Write 6 Done Sent Done false
30220 8507 Acked Write 9 Done Sent Done false
30221 8508 Acked Read 6 Done Sent Done false
30222 8509 Acked Read 9 Done Sent Done false
30223 8510 Acked Read 8 Done Sent Done false
30224 8511 Acked Read 8 Done Sent Done false
30225 8512 Acked Read 4 Done Sent Done false
30226 8513 Acked Read 9 Done Sent Done false
30227 8514 Acked Write 3 Done Sent Done false
30228 8515 Acked Read 10 Done Sent Done false
30229 8516 Acked Read 3 Done Sent Done false
30230 8517 Acked Read 3 Done Sent Done false
30231 8518 Acked Read 6 Done Sent Done false
30232 8519 Acked Read 4 Done Sent Done false
30233 8520 Acked Flush 0 Done Sent Done false
30234 8521 Acked Write 5 Done Sent Done false
30235 8522 Acked Flush 0 Done Sent Done false
30236 8523 Acked Write 9 Done Sent Done false
30237 8524 Acked Write 5 Done Sent Done false
30238 8525 Acked Read 10 Done Sent Done false
30239 8526 Acked Write 8 Done Sent Done false
30240 8527 Acked Write 6 Done Sent Done false
30241 8528 Acked Write 5 Done Sent Done false
30242 8529 Acked Read 2 Done Sent Done false
30243 8530 Acked Write 9 Done Sent Done false
30244 8531 Acked Flush 0 Done Sent Done false
30245 8532 Acked Read 10 Done Sent Done false
30246 8533 Acked Write 3 Done Sent Done false
30247 8534 Acked Flush 0 Done Sent Done false
30248 8535 Acked Write 8 Done Sent Done false
30249 8536 Acked Read 5 Done Sent Done false
30250 8537 Acked Read 4 Done Sent Done false
30251 8538 Acked Write 8 Done Sent Done false
30252 8539 Acked Read 10 Done Sent Done false
30253 8540 Acked Write 3 Done Sent Done false
30254 8541 Acked Write 2 Done Sent Done false
30255 8542 Acked Write 5 Done Sent Done false
30256 8543 Acked Read 7 Done Sent Done false
30257 8544 Acked Read 2 Done Sent Done false
30258 8545 Acked Flush 0 Done Sent Done false
30259 8546 Acked Read 4 Done Sent Done false
30260 8547 Acked Read 9 Done Sent Done false
30261 8548 Acked Read 4 Done Sent Done false
30262 8549 Acked Flush 0 Done Sent Done false
30263 STATES DS:0 DS:1 DS:2 TOTAL
30264 Sent 0 302 0 302
30265 Done 302 0 302 604
30266 Skipped 0 0 0 0
30267 Error 0 0 0 0
30268 Last Flush: 8549 8247 8549
30269 Downstairs last five completed: 8247 8246 8245 8244 8243
30270 Upstairs last five completed: 8549 8548 8547 8546 8545
30271 CLIENT: Up:0 ds:302 act:2
302722024-12-11T23:46:50.403ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
302732024-12-11T23:46:50.403ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
302742024-12-11T23:46:50.404ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
302752024-12-11T23:46:50.404ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
302762024-12-11T23:46:50.404ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
30277 ----------------------------------------------------------------
30278 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30279 States: Active Offline Active
30280 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30281 8248 Acked Write 4 Done Sent Done false
30282 8249 Acked Read 2 Done Sent Done false
30283 8250 Acked Write 2 Done Sent Done false
30284 8251 Acked Read 4 Done Sent Done false
30285 8252 Acked Read 2 Done Sent Done false
30286 8253 Acked Flush 0 Done Sent Done false
30287 8254 Acked Read 1 Done Sent Done false
30288 8255 Acked Flush 0 Done Sent Done false
30289 8256 Acked Write 1 Done Sent Done false
30290 8257 Acked Write 7 Done Sent Done false
30291 8258 Acked Write 3 Done Sent Done false
30292 8259 Acked Write 2 Done Sent Done false
30293 8260 Acked Flush 0 Done Sent Done false
30294 8261 Acked Write 9 Done Sent Done false
30295 8262 Acked Read 8 Done Sent Done false
30296 8263 Acked Read 1 Done Sent Done false
30297 8264 Acked Read 9 Done Sent Done false
30298 8265 Acked Read 5 Done Sent Done false
30299 8266 Acked Write 8 Done Sent Done false
30300 8267 Acked Write 5 Done Sent Done false
30301 8268 Acked Read 6 Done Sent Done false
30302 8269 Acked Read 2 Done Sent Done false
30303 8270 Acked Read 5 Done Sent Done false
30304 8271 Acked Write 7 Done Sent Done false
30305 8272 Acked Read 6 Done Sent Done false
30306 8273 Acked Flush 0 Done Sent Done false
30307 8274 Acked Write 7 Done Sent Done false
30308 8275 Acked Read 7 Done Sent Done false
30309 8276 Acked Write 1 Done Sent Done false
30310 8277 Acked Write 6 Done Sent Done false
30311 8278 Acked Read 8 Done Sent Done false
30312 8279 Acked Read 5 Done Sent Done false
30313 8280 Acked Write 6 Done Sent Done false
30314 8281 Acked Read 4 Done Sent Done false
30315 8282 Acked Read 2 Done Sent Done false
30316 8283 Acked Read 8 Done Sent Done false
30317 8284 Acked Read 1 Done Sent Done false
30318 8285 Acked Write 7 Done Sent Done false
30319 8286 Acked Read 3 Done Sent Done false
30320 8287 Acked Read 5 Done Sent Done false
30321 8288 Acked Read 5 Done Sent Done false
30322 8289 Acked Read 10 Done Sent Done false
30323 8290 Acked Read 1 Done Sent Done false
30324 8291 Acked Write 6 Done Sent Done false
30325 8292 Acked Write 7 Done Sent Done false
30326 8293 Acked Flush 0 Done Sent Done false
30327 8294 Acked Read 6 Done Sent Done false
30328 8295 Acked Write 7 Done Sent Done false
30329 8296 Acked Read 10 Done Sent Done false
30330 8297 Acked Read 2 Done Sent Done false
30331 8298 Acked Read 1 Done Sent Done false
30332 8299 Acked Flush 0 Done Sent Done false
30333 8300 Acked Flush 0 Done Sent Done false
30334 8301 Acked Read 5 Done Sent Done false
30335 8302 Acked Write 8 Done Sent Done false
30336 8303 Acked Read 9 Done Sent Done false
30337 8304 Acked Write 2 Done Sent Done false
30338 8305 Acked Write 7 Done Sent Done false
30339 8306 Acked Read 9 Done Sent Done false
30340 8307 Acked Write 10 Done Sent Done false
30341 8308 Acked Write 8 Done Sent Done false
30342 8309 Acked Write 2 Done Sent Done false
30343 8310 Acked Write 9 Done Sent Done false
30344 8311 Acked Write 2 Done Sent Done false
30345 8312 Acked Write 7 Done Sent Done false
30346 8313 Acked Write 9 Done Sent Done false
30347 8314 Acked Read 6 Done Sent Done false
30348 8315 Acked Write 9 Done Sent Done false
30349 8316 Acked Read 8 Done Sent Done false
30350 8317 Acked Read 5 Done Sent Done false
30351 8318 Acked Flush 0 Done Sent Done false
30352 8319 Acked Read 7 Done Sent Done false
30353 8320 Acked Flush 0 Done Sent Done false
30354 8321 Acked Read 8 Done Sent Done false
30355 8322 Acked Write 9 Done Sent Done false
30356 8323 Acked Read 8 Done Sent Done false
30357 8324 Acked Write 10 Done Sent Done false
30358 8325 Acked Flush 0 Done Sent Done false
30359 8326 Acked Write 6 Done Sent Done false
30360 8327 Acked Write 3 Done Sent Done false
30361 8328 Acked Write 6 Done Sent Done false
30362 8329 Acked Write 10 Done Sent Done false
30363 8330 Acked Write 4 Done Sent Done false
30364 8331 Acked Write 10 Done Sent Done false
30365 8332 Acked Read 7 Done Sent Done false
30366 8333 Acked Write 1 Done Sent Done false
30367 8334 Acked Write 10 Done Sent Done false
30368 8335 Acked Write 4 Done Sent Done false
30369 8336 Acked Write 8 Done Sent Done false
30370 8337 Acked Write 1 Done Sent Done false
30371 8338 Acked Read 5 Done Sent Done false
30372 8339 Acked Write 2 Done Sent Done false
30373 8340 Acked Read 5 Done Sent Done false
30374 8341 Acked Read 2 Done Sent Done false
30375 8342 Acked Write 4 Done Sent Done false
30376 8343 Acked Read 3 Done Sent Done false
30377 8344 Acked Flush 0 Done Sent Done false
30378 8345 Acked Read 4 Done Sent Done false
30379 8346 Acked Read 10 Done Sent Done false
30380 8347 Acked Write 1 Done Sent Done false
30381 8348 Acked Write 6 Done Sent Done false
30382 8349 Acked Write 4 Done Sent Done false
30383 8350 Acked Read 10 Done Sent Done false
30384 8351 Acked Read 1 Done Sent Done false
30385 8352 Acked Read 10 Done Sent Done false
30386 8353 Acked Write 10 Done Sent Done false
30387 8354 Acked Read 1 Done Sent Done false
30388 8355 Acked Read 9 Done Sent Done false
30389 8356 Acked Write 3 Done Sent Done false
30390 8357 Acked Read 7 Done Sent Done false
30391 8358 Acked Flush 0 Done Sent Done false
30392 8359 Acked Read 8 Done Sent Done false
30393 8360 Acked Write 10 Done Sent Done false
30394 8361 Acked Write 3 Done Sent Done false
30395 8362 Acked Write 7 Done Sent Done false
30396 8363 Acked Write 10 Done Sent Done false
30397 8364 Acked Write 7 Done Sent Done false
30398 8365 Acked Read 5 Done Sent Done false
30399 8366 Acked Write 2 Done Sent Done false
30400 8367 Acked Write 4 Done Sent Done false
30401 8368 Acked Read 10 Done Sent Done false
30402 8369 Acked Write 3 Done Sent Done false
30403 8370 Acked Flush 0 Done Sent Done false
30404 8371 Acked Read 9 Done Sent Done false
30405 8372 Acked Read 3 Done Sent Done false
30406 8373 Acked Write 1 Done Sent Done false
30407 8374 Acked Read 3 Done Sent Done false
30408 8375 Acked Write 9 Done Sent Done false
30409 8376 Acked Write 10 Done Sent Done false
30410 8377 Acked Write 2 Done Sent Done false
30411 8378 Acked Write 9 Done Sent Done false
30412 8379 Acked Write 5 Done Sent Done false
30413 8380 Acked Flush 0 Done Sent Done false
30414 8381 Acked Write 9 Done Sent Done false
30415 8382 Acked Write 9 Done Sent Done false
30416 8383 Acked Write 10 Done Sent Done false
30417 8384 Acked Write 6 Done Sent Done false
30418 8385 Acked Read 7 Done Sent Done false
30419 8386 Acked Write 4 Done Sent Done false
30420 8387 Acked Read 10 Done Sent Done false
30421 8388 Acked Flush 0 Done Sent Done false
30422 8389 Acked Write 3 Done Sent Done false
30423 8390 Acked Read 9 Done Sent Done false
30424 8391 Acked Read 8 Done Sent Done false
30425 8392 Acked Flush 0 Done Sent Done false
30426 8393 Acked Write 10 Done Sent Done false
30427 8394 Acked Write 6 Done Sent Done false
30428 8395 Acked Read 5 Done Sent Done false
30429 8396 Acked Flush 0 Done Sent Done false
30430 8397 Acked Write 1 Done Sent Done false
30431 8398 Acked Read 5 Done Sent Done false
30432 8399 Acked Write 4 Done Sent Done false
30433 8400 Acked Write 3 Done Sent Done false
30434 8401 Acked Read 2 Done Sent Done false
30435 8402 Acked Write 10 Done Sent Done false
30436 8403 Acked Write 1 Done Sent Done false
30437 8404 Acked Read 1 Done Sent Done false
30438 8405 Acked Read 7 Done Sent Done false
30439 8406 Acked Flush 0 Done Sent Done false
30440 8407 Acked Write 1 Done Sent Done false
30441 8408 Acked Write 10 Done Sent Done false
30442 8409 Acked Write 4 Done Sent Done false
30443 8410 Acked Write 2 Done Sent Done false
30444 8411 Acked Read 3 Done Sent Done false
30445 8412 Acked Read 8 Done Sent Done false
30446 8413 Acked Write 1 Done Sent Done false
30447 8414 Acked Read 1 Done Sent Done false
30448 8415 Acked Flush 0 Done Sent Done false
30449 8416 Acked Read 7 Done Sent Done false
30450 8417 Acked Read 2 Done Sent Done false
30451 8418 Acked Flush 0 Done Sent Done false
30452 8419 Acked Write 4 Done Sent Done false
30453 8420 Acked Read 6 Done Sent Done false
30454 8421 Acked Read 1 Done Sent Done false
30455 8422 Acked Flush 0 Done Sent Done false
30456 8423 Acked Read 10 Done Sent Done false
30457 8424 Acked Write 10 Done Sent Done false
30458 8425 Acked Flush 0 Done Sent Done false
30459 8426 Acked Write 9 Done Sent Done false
30460 8427 Acked Read 2 Done Sent Done false
30461 8428 Acked Write 8 Done Sent Done false
30462 8429 Acked Write 4 Done Sent Done false
30463 8430 Acked Flush 0 Done Sent Done false
30464 8431 Acked Flush 0 Done Sent Done false
30465 8432 Acked Write 2 Done Sent Done false
30466 8433 Acked Read 8 Done Sent Done false
30467 8434 Acked Read 2 Done Sent Done false
30468 8435 Acked Read 5 Done Sent Done false
30469 8436 Acked Write 2 Done Sent Done false
30470 8437 Acked Read 5 Done Sent Done false
30471 8438 Acked Read 10 Done Sent Done false
30472 8439 Acked Flush 0 Done Sent Done false
30473 8440 Acked Read 8 Done Sent Done false
30474 8441 Acked Read 7 Done Sent Done false
30475 8442 Acked Read 9 Done Sent Done false
30476 8443 Acked Write 10 Done Sent Done false
30477 8444 Acked Read 2 Done Sent Done false
30478 8445 Acked Flush 0 Done Sent Done false
30479 8446 Acked Write 9 Done Sent Done false
30480 8447 Acked Read 8 Done Sent Done false
30481 8448 Acked Write 9 Done Sent Done false
30482 8449 Acked Read 9 Done Sent Done false
30483 8450 Acked Read 2 Done Sent Done false
30484 8451 Acked Write 2 Done Sent Done false
30485 8452 Acked Flush 0 Done Sent Done false
30486 8453 Acked Read 1 Done Sent Done false
30487 8454 Acked Read 2 Done Sent Done false
30488 8455 Acked Read 6 Done Sent Done false
30489 8456 Acked Read 2 Done Sent Done false
30490 8457 Acked Read 1 Done Sent Done false
30491 8458 Acked Read 9 Done Sent Done false
30492 8459 Acked Read 1 Done Sent Done false
30493 8460 Acked Write 6 Done Sent Done false
30494 8461 Acked Read 2 Done Sent Done false
30495 8462 Acked Read 2 Done Sent Done false
30496 8463 Acked Read 3 Done Sent Done false
30497 8464 Acked Write 8 Done Sent Done false
30498 8465 Acked Write 3 Done Sent Done false
30499 8466 Acked Read 5 Done Sent Done false
30500 8467 Acked Write 2 Done Sent Done false
30501 8468 Acked Read 3 Done Sent Done false
30502 8469 Acked Write 5 Done Sent Done false
30503 8470 Acked Read 1 Done Sent Done false
30504 8471 Acked Read 5 Done Sent Done false
30505 8472 Acked Read 7 Done Sent Done false
30506 8473 Acked Write 1 Done Sent Done false
30507 8474 Acked Flush 0 Done Sent Done false
30508 8475 Acked Flush 0 Done Sent Done false
30509 8476 Acked Write 7 Done Sent Done false
30510 8477 Acked Read 10 Done Sent Done false
30511 8478 Acked Flush 0 Done Sent Done false
30512 8479 Acked Read 10 Done Sent Done false
30513 8480 Acked Write 6 Done Sent Done false
30514 8481 Acked Write 7 Done Sent Done false
30515 8482 Acked Write 2 Done Sent Done false
30516 8483 Acked Write 1 Done Sent Done false
30517 8484 Acked Read 9 Done Sent Done false
30518 8485 Acked Flush 0 Done Sent Done false
30519 8486 Acked Read 3 Done Sent Done false
30520 8487 Acked Write 5 Done Sent Done false
30521 8488 Acked Read 3 Done Sent Done false
30522 8489 Acked Read 1 Done Sent Done false
30523 8490 Acked Read 2 Done Sent Done false
30524 8491 Acked Read 7 Done Sent Done false
30525 8492 Acked Write 3 Done Sent Done false
30526 8493 Acked Read 3 Done Sent Done false
30527 8494 Acked Write 5 Done Sent Done false
30528 8495 Acked Read 9 Done Sent Done false
30529 8496 Acked Read 2 Done Sent Done false
30530 8497 Acked Read 5 Done Sent Done false
30531 8498 Acked Read 6 Done Sent Done false
30532 8499 Acked Read 4 Done Sent Done false
30533 8500 Acked Read 2 Done Sent Done false
30534 8501 Acked Write 8 Done Sent Done false
30535 8502 Acked Read 7 Done Sent Done false
30536 8503 Acked Read 4 Done Sent Done false
30537 8504 Acked Read 10 Done Sent Done false
30538 8505 Acked Write 2 Done Sent Done false
30539 8506 Acked Write 6 Done Sent Done false
30540 8507 Acked Write 9 Done Sent Done false
30541 8508 Acked Read 6 Done Sent Done false
30542 8509 Acked Read 9 Done Sent Done false
30543 8510 Acked Read 8 Done Sent Done false
30544 8511 Acked Read 8 Done Sent Done false
30545 8512 Acked Read 4 Done Sent Done false
30546 8513 Acked Read 9 Done Sent Done false
30547 8514 Acked Write 3 Done Sent Done false
30548 8515 Acked Read 10 Done Sent Done false
30549 8516 Acked Read 3 Done Sent Done false
30550 8517 Acked Read 3 Done Sent Done false
30551 8518 Acked Read 6 Done Sent Done false
30552 8519 Acked Read 4 Done Sent Done false
30553 8520 Acked Flush 0 Done Sent Done false
30554 8521 Acked Write 5 Done Sent Done false
30555 8522 Acked Flush 0 Done Sent Done false
30556 8523 Acked Write 9 Done Sent Done false
30557 8524 Acked Write 5 Done Sent Done false
30558 8525 Acked Read 10 Done Sent Done false
30559 8526 Acked Write 8 Done Sent Done false
30560 8527 Acked Write 6 Done Sent Done false
30561 8528 Acked Write 5 Done Sent Done false
30562 8529 Acked Read 2 Done Sent Done false
30563 8530 Acked Write 9 Done Sent Done false
30564 8531 Acked Flush 0 Done Sent Done false
30565 8532 Acked Read 10 Done Sent Done false
30566 8533 Acked Write 3 Done Sent Done false
30567 8534 Acked Flush 0 Done Sent Done false
30568 8535 Acked Write 8 Done Sent Done false
30569 8536 Acked Read 5 Done Sent Done false
30570 8537 Acked Read 4 Done Sent Done false
30571 8538 Acked Write 8 Done Sent Done false
30572 8539 Acked Read 10 Done Sent Done false
30573 8540 Acked Write 3 Done Sent Done false
30574 8541 Acked Write 2 Done Sent Done false
30575 8542 Acked Write 5 Done Sent Done false
30576 8543 Acked Read 7 Done Sent Done false
30577 8544 Acked Read 2 Done Sent Done false
30578 8545 Acked Flush 0 Done Sent Done false
30579 8546 Acked Read 4 Done Sent Done false
30580 8547 Acked Read 9 Done Sent Done false
30581 8548 Acked Read 4 Done Sent Done false
30582 8549 Acked Flush 0 Done Sent Done false
30583 STATES DS:0 DS:1 DS:2 TOTAL
30584 Sent 0 302 0 302
30585 Done 302 0 302 604
30586 Skipped 0 0 0 0
30587 Error 0 0 0 0
30588 Last Flush: 8549 8247 8549
30589 Downstairs last five completed: 8247 8246 8245 8244 8243
30590 Upstairs last five completed: 8549 8548 8547 8546 8545
30591 CLIENT: Up:0 ds:302 act:2
30592 ----------------------------------------------------------------
30593 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30594 States: Active Offline Active
30595 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30596 8248 Acked Write 4 Done Sent Done false
30597 8249 Acked Read 2 Done Sent Done false
30598 8250 Acked Write 2 Done Sent Done false
30599 8251 Acked Read 4 Done Sent Done false
30600 8252 Acked Read 2 Done Sent Done false
30601 8253 Acked Flush 0 Done Sent Done false
30602 8254 Acked Read 1 Done Sent Done false
30603 8255 Acked Flush 0 Done Sent Done false
30604 8256 Acked Write 1 Done Sent Done false
30605 8257 Acked Write 7 Done Sent Done false
30606 8258 Acked Write 3 Done Sent Done false
30607 8259 Acked Write 2 Done Sent Done false
30608 8260 Acked Flush 0 Done Sent Done false
30609 8261 Acked Write 9 Done Sent Done false
30610 8262 Acked Read 8 Done Sent Done false
30611 8263 Acked Read 1 Done Sent Done false
30612 8264 Acked Read 9 Done Sent Done false
30613 8265 Acked Read 5 Done Sent Done false
30614 8266 Acked Write 8 Done Sent Done false
30615 8267 Acked Write 5 Done Sent Done false
30616 8268 Acked Read 6 Done Sent Done false
30617 8269 Acked Read 2 Done Sent Done false
30618 8270 Acked Read 5 Done Sent Done false
30619 8271 Acked Write 7 Done Sent Done false
30620 8272 Acked Read 6 Done Sent Done false
30621 8273 Acked Flush 0 Done Sent Done false
30622 8274 Acked Write 7 Done Sent Done false
30623 8275 Acked Read 7 Done Sent Done false
30624 8276 Acked Write 1 Done Sent Done false
30625 8277 Acked Write 6 Done Sent Done false
30626 8278 Acked Read 8 Done Sent Done false
30627 8279 Acked Read 5 Done Sent Done false
30628 8280 Acked Write 6 Done Sent Done false
30629 8281 Acked Read 4 Done Sent Done false
30630 8282 Acked Read 2 Done Sent Done false
30631 8283 Acked Read 8 Done Sent Done false
30632 8284 Acked Read 1 Done Sent Done false
30633 8285 Acked Write 7 Done Sent Done false
30634 8286 Acked Read 3 Done Sent Done false
30635 8287 Acked Read 5 Done Sent Done false
30636 8288 Acked Read 5 Done Sent Done false
30637 8289 Acked Read 10 Done Sent Done false
30638 8290 Acked Read 1 Done Sent Done false
30639 8291 Acked Write 6 Done Sent Done false
30640 8292 Acked Write 7 Done Sent Done false
30641 8293 Acked Flush 0 Done Sent Done false
30642 8294 Acked Read 6 Done Sent Done false
30643 8295 Acked Write 7 Done Sent Done false
30644 8296 Acked Read 10 Done Sent Done false
30645 8297 Acked Read 2 Done Sent Done false
30646 8298 Acked Read 1 Done Sent Done false
30647 8299 Acked Flush 0 Done Sent Done false
30648 8300 Acked Flush 0 Done Sent Done false
30649 8301 Acked Read 5 Done Sent Done false
30650 8302 Acked Write 8 Done Sent Done false
30651 8303 Acked Read 9 Done Sent Done false
30652 8304 Acked Write 2 Done Sent Done false
30653 8305 Acked Write 7 Done Sent Done false
30654 8306 Acked Read 9 Done Sent Done false
30655 8307 Acked Write 10 Done Sent Done false
30656 8308 Acked Write 8 Done Sent Done false
30657 8309 Acked Write 2 Done Sent Done false
30658 8310 Acked Write 9 Done Sent Done false
30659 8311 Acked Write 2 Done Sent Done false
30660 8312 Acked Write 7 Done Sent Done false
30661 8313 Acked Write 9 Done Sent Done false
30662 8314 Acked Read 6 Done Sent Done false
30663 8315 Acked Write 9 Done Sent Done false
30664 8316 Acked Read 8 Done Sent Done false
30665 8317 Acked Read 5 Done Sent Done false
30666 8318 Acked Flush 0 Done Sent Done false
30667 8319 Acked Read 7 Done Sent Done false
30668 8320 Acked Flush 0 Done Sent Done false
30669 8321 Acked Read 8 Done Sent Done false
30670 8322 Acked Write 9 Done Sent Done false
30671 8323 Acked Read 8 Done Sent Done false
30672 8324 Acked Write 10 Done Sent Done false
30673 8325 Acked Flush 0 Done Sent Done false
30674 8326 Acked Write 6 Done Sent Done false
30675 8327 Acked Write 3 Done Sent Done false
30676 8328 Acked Write 6 Done Sent Done false
30677 8329 Acked Write 10 Done Sent Done false
30678 8330 Acked Write 4 Done Sent Done false
30679 8331 Acked Write 10 Done Sent Done false
30680 8332 Acked Read 7 Done Sent Done false
30681 8333 Acked Write 1 Done Sent Done false
30682 8334 Acked Write 10 Done Sent Done false
30683 8335 Acked Write 4 Done Sent Done false
30684 8336 Acked Write 8 Done Sent Done false
30685 8337 Acked Write 1 Done Sent Done false
30686 8338 Acked Read 5 Done Sent Done false
30687 8339 Acked Write 2 Done Sent Done false
30688 8340 Acked Read 5 Done Sent Done false
30689 8341 Acked Read 2 Done Sent Done false
30690 8342 Acked Write 4 Done Sent Done false
30691 8343 Acked Read 3 Done Sent Done false
30692 8344 Acked Flush 0 Done Sent Done false
30693 8345 Acked Read 4 Done Sent Done false
30694 8346 Acked Read 10 Done Sent Done false
30695 8347 Acked Write 1 Done Sent Done false
30696 8348 Acked Write 6 Done Sent Done false
30697 8349 Acked Write 4 Done Sent Done false
30698 8350 Acked Read 10 Done Sent Done false
30699 8351 Acked Read 1 Done Sent Done false
30700 8352 Acked Read 10 Done Sent Done false
30701 8353 Acked Write 10 Done Sent Done false
30702 8354 Acked Read 1 Done Sent Done false
30703 8355 Acked Read 9 Done Sent Done false
30704 8356 Acked Write 3 Done Sent Done false
30705 8357 Acked Read 7 Done Sent Done false
30706 8358 Acked Flush 0 Done Sent Done false
30707 8359 Acked Read 8 Done Sent Done false
30708 8360 Acked Write 10 Done Sent Done false
30709 8361 Acked Write 3 Done Sent Done false
30710 8362 Acked Write 7 Done Sent Done false
30711 8363 Acked Write 10 Done Sent Done false
30712 8364 Acked Write 7 Done Sent Done false
30713 8365 Acked Read 5 Done Sent Done false
30714 8366 Acked Write 2 Done Sent Done false
30715 8367 Acked Write 4 Done Sent Done false
30716 8368 Acked Read 10 Done Sent Done false
30717 8369 Acked Write 3 Done Sent Done false
30718 8370 Acked Flush 0 Done Sent Done false
30719 8371 Acked Read 9 Done Sent Done false
30720 8372 Acked Read 3 Done Sent Done false
30721 8373 Acked Write 1 Done Sent Done false
30722 8374 Acked Read 3 Done Sent Done false
30723 8375 Acked Write 9 Done Sent Done false
30724 8376 Acked Write 10 Done Sent Done false
30725 8377 Acked Write 2 Done Sent Done false
30726 8378 Acked Write 9 Done Sent Done false
30727 8379 Acked Write 5 Done Sent Done false
30728 8380 Acked Flush 0 Done Sent Done false
30729 8381 Acked Write 9 Done Sent Done false
30730 8382 Acked Write 9 Done Sent Done false
30731 8383 Acked Write 10 Done Sent Done false
30732 8384 Acked Write 6 Done Sent Done false
30733 8385 Acked Read 7 Done Sent Done false
30734 8386 Acked Write 4 Done Sent Done false
30735 8387 Acked Read 10 Done Sent Done false
30736 8388 Acked Flush 0 Done Sent Done false
30737 8389 Acked Write 3 Done Sent Done false
30738 8390 Acked Read 9 Done Sent Done false
30739 8391 Acked Read 8 Done Sent Done false
30740 8392 Acked Flush 0 Done Sent Done false
30741 8393 Acked Write 10 Done Sent Done false
30742 8394 Acked Write 6 Done Sent Done false
30743 8395 Acked Read 5 Done Sent Done false
30744 8396 Acked Flush 0 Done Sent Done false
30745 8397 Acked Write 1 Done Sent Done false
30746 8398 Acked Read 5 Done Sent Done false
30747 8399 Acked Write 4 Done Sent Done false
30748 8400 Acked Write 3 Done Sent Done false
30749 8401 Acked Read 2 Done Sent Done false
30750 8402 Acked Write 10 Done Sent Done false
30751 8403 Acked Write 1 Done Sent Done false
30752 8404 Acked Read 1 Done Sent Done false
30753 8405 Acked Read 7 Done Sent Done false
30754 8406 Acked Flush 0 Done Sent Done false
30755 8407 Acked Write 1 Done Sent Done false
30756 8408 Acked Write 10 Done Sent Done false
30757 8409 Acked Write 4 Done Sent Done false
30758 8410 Acked Write 2 Done Sent Done false
30759 8411 Acked Read 3 Done Sent Done false
30760 8412 Acked Read 8 Done Sent Done false
30761 8413 Acked Write 1 Done Sent Done false
30762 8414 Acked Read 1 Done Sent Done false
30763 8415 Acked Flush 0 Done Sent Done false
30764 8416 Acked Read 7 Done Sent Done false
30765 8417 Acked Read 2 Done Sent Done false
30766 8418 Acked Flush 0 Done Sent Done false
30767 8419 Acked Write 4 Done Sent Done false
30768 8420 Acked Read 6 Done Sent Done false
30769 8421 Acked Read 1 Done Sent Done false
30770 8422 Acked Flush 0 Done Sent Done false
30771 8423 Acked Read 10 Done Sent Done false
30772 8424 Acked Write 10 Done Sent Done false
30773 8425 Acked Flush 0 Done Sent Done false
30774 8426 Acked Write 9 Done Sent Done false
30775 8427 Acked Read 2 Done Sent Done false
30776 8428 Acked Write 8 Done Sent Done false
30777 8429 Acked Write 4 Done Sent Done false
30778 8430 Acked Flush 0 Done Sent Done false
30779 8431 Acked Flush 0 Done Sent Done false
30780 8432 Acked Write 2 Done Sent Done false
30781 8433 Acked Read 8 Done Sent Done false
30782 8434 Acked Read 2 Done Sent Done false
30783 8435 Acked Read 5 Done Sent Done false
30784 8436 Acked Write 2 Done Sent Done false
30785 8437 Acked Read 5 Done Sent Done false
30786 8438 Acked Read 10 Done Sent Done false
30787 8439 Acked Flush 0 Done Sent Done false
30788 8440 Acked Read 8 Done Sent Done false
30789 8441 Acked Read 7 Done Sent Done false
30790 8442 Acked Read 9 Done Sent Done false
30791 8443 Acked Write 10 Done Sent Done false
30792 8444 Acked Read 2 Done Sent Done false
30793 8445 Acked Flush 0 Done Sent Done false
30794 8446 Acked Write 9 Done Sent Done false
30795 8447 Acked Read 8 Done Sent Done false
30796 8448 Acked Write 9 Done Sent Done false
30797 8449 Acked Read 9 Done Sent Done false
30798 8450 Acked Read 2 Done Sent Done false
30799 8451 Acked Write 2 Done Sent Done false
30800 8452 Acked Flush 0 Done Sent Done false
30801 8453 Acked Read 1 Done Sent Done false
30802 8454 Acked Read 2 Done Sent Done false
30803 8455 Acked Read 6 Done Sent Done false
30804 8456 Acked Read 2 Done Sent Done false
30805 8457 Acked Read 1 Done Sent Done false
30806 8458 Acked Read 9 Done Sent Done false
30807 8459 Acked Read 1 Done Sent Done false
30808 8460 Acked Write 6 Done Sent Done false
30809 8461 Acked Read 2 Done Sent Done false
30810 8462 Acked Read 2 Done Sent Done false
30811 8463 Acked Read 3 Done Sent Done false
30812 8464 Acked Write 8 Done Sent Done false
30813 8465 Acked Write 3 Done Sent Done false
30814 8466 Acked Read 5 Done Sent Done false
30815 8467 Acked Write 2 Done Sent Done false
30816 8468 Acked Read 3 Done Sent Done false
30817 8469 Acked Write 5 Done Sent Done false
30818 8470 Acked Read 1 Done Sent Done false
30819 8471 Acked Read 5 Done Sent Done false
30820 8472 Acked Read 7 Done Sent Done false
30821 8473 Acked Write 1 Done Sent Done false
30822 8474 Acked Flush 0 Done Sent Done false
30823 8475 Acked Flush 0 Done Sent Done false
30824 8476 Acked Write 7 Done Sent Done false
30825 8477 Acked Read 10 Done Sent Done false
30826 8478 Acked Flush 0 Done Sent Done false
30827 8479 Acked Read 10 Done Sent Done false
30828 8480 Acked Write 6 Done Sent Done false
30829 8481 Acked Write 7 Done Sent Done false
30830 8482 Acked Write 2 Done Sent Done false
30831 8483 Acked Write 1 Done Sent Done false
30832 8484 Acked Read 9 Done Sent Done false
30833 8485 Acked Flush 0 Done Sent Done false
30834 8486 Acked Read 3 Done Sent Done false
30835 8487 Acked Write 5 Done Sent Done false
30836 8488 Acked Read 3 Done Sent Done false
30837 8489 Acked Read 1 Done Sent Done false
30838 8490 Acked Read 2 Done Sent Done false
30839 8491 Acked Read 7 Done Sent Done false
30840 8492 Acked Write 3 Done Sent Done false
30841 8493 Acked Read 3 Done Sent Done false
30842 8494 Acked Write 5 Done Sent Done false
30843 8495 Acked Read 9 Done Sent Done false
30844 8496 Acked Read 2 Done Sent Done false
30845 8497 Acked Read 5 Done Sent Done false
30846 8498 Acked Read 6 Done Sent Done false
30847 8499 Acked Read 4 Done Sent Done false
30848 8500 Acked Read 2 Done Sent Done false
30849 8501 Acked Write 8 Done Sent Done false
30850 8502 Acked Read 7 Done Sent Done false
30851 8503 Acked Read 4 Done Sent Done false
30852 8504 Acked Read 10 Done Sent Done false
30853 8505 Acked Write 2 Done Sent Done false
30854 8506 Acked Write 6 Done Sent Done false
30855 8507 Acked Write 9 Done Sent Done false
30856 8508 Acked Read 6 Done Sent Done false
30857 8509 Acked Read 9 Done Sent Done false
30858 8510 Acked Read 8 Done Sent Done false
30859 8511 Acked Read 8 Done Sent Done false
30860 8512 Acked Read 4 Done Sent Done false
30861 8513 Acked Read 9 Done Sent Done false
30862 8514 Acked Write 3 Done Sent Done false
30863 8515 Acked Read 10 Done Sent Done false
30864 8516 Acked Read 3 Done Sent Done false
30865 8517 Acked Read 3 Done Sent Done false
30866 8518 Acked Read 6 Done Sent Done false
30867 8519 Acked Read 4 Done Sent Done false
30868 8520 Acked Flush 0 Done Sent Done false
30869 8521 Acked Write 5 Done Sent Done false
30870 8522 Acked Flush 0 Done Sent Done false
30871 8523 Acked Write 9 Done Sent Done false
30872 8524 Acked Write 5 Done Sent Done false
30873 8525 Acked Read 10 Done Sent Done false
30874 8526 Acked Write 8 Done Sent Done false
30875 8527 Acked Write 6 Done Sent Done false
30876 8528 Acked Write 5 Done Sent Done false
30877 8529 Acked Read 2 Done Sent Done false
30878 8530 Acked Write 9 Done Sent Done false
30879 8531 Acked Flush 0 Done Sent Done false
30880 8532 Acked Read 10 Done Sent Done false
30881 8533 Acked Write 3 Done Sent Done false
30882 8534 Acked Flush 0 Done Sent Done false
30883 8535 Acked Write 8 Done Sent Done false
30884 8536 Acked Read 5 Done Sent Done false
30885 8537 Acked Read 4 Done Sent Done false
30886 8538 Acked Write 8 Done Sent Done false
30887 8539 Acked Read 10 Done Sent Done false
30888 8540 Acked Write 3 Done Sent Done false
30889 8541 Acked Write 2 Done Sent Done false
30890 8542 Acked Write 5 Done Sent Done false
30891 8543 Acked Read 7 Done Sent Done false
30892 8544 Acked Read 2 Done Sent Done false
30893 8545 Acked Flush 0 Done Sent Done false
30894 8546 Acked Read 4 Done Sent Done false
30895 8547 Acked Read 9 Done Sent Done false
30896 8548 Acked Read 4 Done Sent Done false
30897 8549 Acked Flush 0 Done Sent Done false
30898 STATES DS:0 DS:1 DS:2 TOTAL
30899 Sent 0 302 0 302
30900 Done 302 0 302 604
30901 Skipped 0 0 0 0
30902 Error 0 0 0 0
30903 Last Flush: 8549 8247 8549
30904 Downstairs last five completed: 8247 8246 8245 8244 8243
30905 Upstairs last five completed: 8549 8548 8547 8546 8545
30906 CLIENT: Up:0 ds:302 act:2
309072024-12-11T23:47:00.404ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
309082024-12-11T23:47:00.404ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
309092024-12-11T23:47:00.404ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
309102024-12-11T23:47:00.404ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
309112024-12-11T23:47:00.404ZINFOcrucible: client task is exiting = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
30912 [8820] got start action
30913 [8820] I am going to restart
30914 Starting downstairs at port 8820
30915 Make output file at "/tmp/dsc/downstairs-8820.txt"
30916 [8820][1] reports Starting
30917 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
30918 [8820][1] reports Running
30919 ----------------------------------------------------------------
30920 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30921 States: Active Offline Active
30922 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30923 8248 Acked Write 4 Done Sent Done false
30924 8249 Acked Read 2 Done Sent Done false
30925 8250 Acked Write 2 Done Sent Done false
30926 8251 Acked Read 4 Done Sent Done false
30927 8252 Acked Read 2 Done Sent Done false
30928 8253 Acked Flush 0 Done Sent Done false
30929 8254 Acked Read 1 Done Sent Done false
30930 8255 Acked Flush 0 Done Sent Done false
30931 8256 Acked Write 1 Done Sent Done false
30932 8257 Acked Write 7 Done Sent Done false
30933 8258 Acked Write 3 Done Sent Done false
30934 8259 Acked Write 2 Done Sent Done false
30935 8260 Acked Flush 0 Done Sent Done false
30936 8261 Acked Write 9 Done Sent Done false
30937 8262 Acked Read 8 Done Sent Done false
30938 8263 Acked Read 1 Done Sent Done false
30939 8264 Acked Read 9 Done Sent Done false
30940 8265 Acked Read 5 Done Sent Done false
30941 8266 Acked Write 8 Done Sent Done false
30942 8267 Acked Write 5 Done Sent Done false
30943 8268 Acked Read 6 Done Sent Done false
30944 8269 Acked Read 2 Done Sent Done false
30945 8270 Acked Read 5 Done Sent Done false
30946 8271 Acked Write 7 Done Sent Done false
30947 8272 Acked Read 6 Done Sent Done false
30948 8273 Acked Flush 0 Done Sent Done false
30949 8274 Acked Write 7 Done Sent Done false
30950 8275 Acked Read 7 Done Sent Done false
30951 8276 Acked Write 1 Done Sent Done false
30952 8277 Acked Write 6 Done Sent Done false
30953 8278 Acked Read 8 Done Sent Done false
30954 8279 Acked Read 5 Done Sent Done false
30955 8280 Acked Write 6 Done Sent Done false
30956 8281 Acked Read 4 Done Sent Done false
30957 8282 Acked Read 2 Done Sent Done false
30958 8283 Acked Read 8 Done Sent Done false
30959 8284 Acked Read 1 Done Sent Done false
30960 8285 Acked Write 7 Done Sent Done false
30961 8286 Acked Read 3 Done Sent Done false
30962 8287 Acked Read 5 Done Sent Done false
30963 8288 Acked Read 5 Done Sent Done false
30964 8289 Acked Read 10 Done Sent Done false
30965 8290 Acked Read 1 Done Sent Done false
30966 8291 Acked Write 6 Done Sent Done false
30967 8292 Acked Write 7 Done Sent Done false
30968 8293 Acked Flush 0 Done Sent Done false
30969 8294 Acked Read 6 Done Sent Done false
30970 8295 Acked Write 7 Done Sent Done false
30971 8296 Acked Read 10 Done Sent Done false
30972 8297 Acked Read 2 Done Sent Done false
30973 8298 Acked Read 1 Done Sent Done false
30974 8299 Acked Flush 0 Done Sent Done false
30975 8300 Acked Flush 0 Done Sent Done false
30976 8301 Acked Read 5 Done Sent Done false
30977 8302 Acked Write 8 Done Sent Done false
30978 8303 Acked Read 9 Done Sent Done false
30979 8304 Acked Write 2 Done Sent Done false
30980 8305 Acked Write 7 Done Sent Done false
30981 8306 Acked Read 9 Done Sent Done false
30982 8307 Acked Write 10 Done Sent Done false
30983 8308 Acked Write 8 Done Sent Done false
30984 8309 Acked Write 2 Done Sent Done false
30985 8310 Acked Write 9 Done Sent Done false
30986 8311 Acked Write 2 Done Sent Done false
30987 8312 Acked Write 7 Done Sent Done false
30988 8313 Acked Write 9 Done Sent Done false
30989 8314 Acked Read 6 Done Sent Done false
30990 8315 Acked Write 9 Done Sent Done false
30991 8316 Acked Read 8 Done Sent Done false
30992 8317 Acked Read 5 Done Sent Done false
30993 8318 Acked Flush 0 Done Sent Done false
30994 8319 Acked Read 7 Done Sent Done false
30995 8320 Acked Flush 0 Done Sent Done false
30996 8321 Acked Read 8 Done Sent Done false
30997 8322 Acked Write 9 Done Sent Done false
30998 8323 Acked Read 8 Done Sent Done false
30999 8324 Acked Write 10 Done Sent Done false
31000 8325 Acked Flush 0 Done Sent Done false
31001 8326 Acked Write 6 Done Sent Done false
31002 8327 Acked Write 3 Done Sent Done false
31003 8328 Acked Write 6 Done Sent Done false
31004 8329 Acked Write 10 Done Sent Done false
31005 8330 Acked Write 4 Done Sent Done false
31006 8331 Acked Write 10 Done Sent Done false
31007 8332 Acked Read 7 Done Sent Done false
31008 8333 Acked Write 1 Done Sent Done false
31009 8334 Acked Write 10 Done Sent Done false
31010 8335 Acked Write 4 Done Sent Done false
31011 8336 Acked Write 8 Done Sent Done false
31012 8337 Acked Write 1 Done Sent Done false
31013 8338 Acked Read 5 Done Sent Done false
31014 8339 Acked Write 2 Done Sent Done false
31015 8340 Acked Read 5 Done Sent Done false
31016 8341 Acked Read 2 Done Sent Done false
31017 8342 Acked Write 4 Done Sent Done false
31018 8343 Acked Read 3 Done Sent Done false
31019 8344 Acked Flush 0 Done Sent Done false
31020 8345 Acked Read 4 Done Sent Done false
31021 8346 Acked Read 10 Done Sent Done false
31022 8347 Acked Write 1 Done Sent Done false
31023 8348 Acked Write 6 Done Sent Done false
31024 8349 Acked Write 4 Done Sent Done false
31025 8350 Acked Read 10 Done Sent Done false
31026 8351 Acked Read 1 Done Sent Done false
31027 8352 Acked Read 10 Done Sent Done false
31028 8353 Acked Write 10 Done Sent Done false
31029 8354 Acked Read 1 Done Sent Done false
31030 8355 Acked Read 9 Done Sent Done false
31031 8356 Acked Write 3 Done Sent Done false
31032 8357 Acked Read 7 Done Sent Done false
31033 8358 Acked Flush 0 Done Sent Done false
31034 8359 Acked Read 8 Done Sent Done false
31035 8360 Acked Write 10 Done Sent Done false
31036 8361 Acked Write 3 Done Sent Done false
31037 8362 Acked Write 7 Done Sent Done false
31038 8363 Acked Write 10 Done Sent Done false
31039 8364 Acked Write 7 Done Sent Done false
31040 8365 Acked Read 5 Done Sent Done false
31041 8366 Acked Write 2 Done Sent Done false
31042 8367 Acked Write 4 Done Sent Done false
31043 8368 Acked Read 10 Done Sent Done false
31044 8369 Acked Write 3 Done Sent Done false
31045 8370 Acked Flush 0 Done Sent Done false
31046 8371 Acked Read 9 Done Sent Done false
31047 8372 Acked Read 3 Done Sent Done false
31048 8373 Acked Write 1 Done Sent Done false
31049 8374 Acked Read 3 Done Sent Done false
31050 8375 Acked Write 9 Done Sent Done false
31051 8376 Acked Write 10 Done Sent Done false
31052 8377 Acked Write 2 Done Sent Done false
31053 8378 Acked Write 9 Done Sent Done false
31054 8379 Acked Write 5 Done Sent Done false
31055 8380 Acked Flush 0 Done Sent Done false
31056 8381 Acked Write 9 Done Sent Done false
31057 8382 Acked Write 9 Done Sent Done false
31058 8383 Acked Write 10 Done Sent Done false
31059 8384 Acked Write 6 Done Sent Done false
31060 8385 Acked Read 7 Done Sent Done false
31061 8386 Acked Write 4 Done Sent Done false
31062 8387 Acked Read 10 Done Sent Done false
31063 8388 Acked Flush 0 Done Sent Done false
31064 8389 Acked Write 3 Done Sent Done false
31065 8390 Acked Read 9 Done Sent Done false
31066 8391 Acked Read 8 Done Sent Done false
31067 8392 Acked Flush 0 Done Sent Done false
31068 8393 Acked Write 10 Done Sent Done false
31069 8394 Acked Write 6 Done Sent Done false
31070 8395 Acked Read 5 Done Sent Done false
31071 8396 Acked Flush 0 Done Sent Done false
31072 8397 Acked Write 1 Done Sent Done false
31073 8398 Acked Read 5 Done Sent Done false
31074 8399 Acked Write 4 Done Sent Done false
31075 8400 Acked Write 3 Done Sent Done false
31076 8401 Acked Read 2 Done Sent Done false
31077 8402 Acked Write 10 Done Sent Done false
31078 8403 Acked Write 1 Done Sent Done false
31079 8404 Acked Read 1 Done Sent Done false
31080 8405 Acked Read 7 Done Sent Done false
31081 8406 Acked Flush 0 Done Sent Done false
31082 8407 Acked Write 1 Done Sent Done false
31083 8408 Acked Write 10 Done Sent Done false
31084 8409 Acked Write 4 Done Sent Done false
31085 8410 Acked Write 2 Done Sent Done false
31086 8411 Acked Read 3 Done Sent Done false
31087 8412 Acked Read 8 Done Sent Done false
31088 8413 Acked Write 1 Done Sent Done false
31089 8414 Acked Read 1 Done Sent Done false
31090 8415 Acked Flush 0 Done Sent Done false
31091 8416 Acked Read 7 Done Sent Done false
31092 8417 Acked Read 2 Done Sent Done false
31093 8418 Acked Flush 0 Done Sent Done false
31094 8419 Acked Write 4 Done Sent Done false
31095 8420 Acked Read 6 Done Sent Done false
31096 8421 Acked Read 1 Done Sent Done false
31097 8422 Acked Flush 0 Done Sent Done false
31098 8423 Acked Read 10 Done Sent Done false
31099 8424 Acked Write 10 Done Sent Done false
31100 8425 Acked Flush 0 Done Sent Done false
31101 8426 Acked Write 9 Done Sent Done false
31102 8427 Acked Read 2 Done Sent Done false
31103 8428 Acked Write 8 Done Sent Done false
31104 8429 Acked Write 4 Done Sent Done false
31105 8430 Acked Flush 0 Done Sent Done false
31106 8431 Acked Flush 0 Done Sent Done false
31107 8432 Acked Write 2 Done Sent Done false
31108 8433 Acked Read 8 Done Sent Done false
31109 8434 Acked Read 2 Done Sent Done false
31110 8435 Acked Read 5 Done Sent Done false
31111 8436 Acked Write 2 Done Sent Done false
31112 8437 Acked Read 5 Done Sent Done false
31113 8438 Acked Read 10 Done Sent Done false
31114 8439 Acked Flush 0 Done Sent Done false
31115 8440 Acked Read 8 Done Sent Done false
31116 8441 Acked Read 7 Done Sent Done false
31117 8442 Acked Read 9 Done Sent Done false
31118 8443 Acked Write 10 Done Sent Done false
31119 8444 Acked Read 2 Done Sent Done false
31120 8445 Acked Flush 0 Done Sent Done false
31121 8446 Acked Write 9 Done Sent Done false
31122 8447 Acked Read 8 Done Sent Done false
31123 8448 Acked Write 9 Done Sent Done false
31124 8449 Acked Read 9 Done Sent Done false
31125 8450 Acked Read 2 Done Sent Done false
31126 8451 Acked Write 2 Done Sent Done false
31127 8452 Acked Flush 0 Done Sent Done false
31128 8453 Acked Read 1 Done Sent Done false
31129 8454 Acked Read 2 Done Sent Done false
31130 8455 Acked Read 6 Done Sent Done false
31131 8456 Acked Read 2 Done Sent Done false
31132 8457 Acked Read 1 Done Sent Done false
31133 8458 Acked Read 9 Done Sent Done false
31134 8459 Acked Read 1 Done Sent Done false
31135 8460 Acked Write 6 Done Sent Done false
31136 8461 Acked Read 2 Done Sent Done false
31137 8462 Acked Read 2 Done Sent Done false
31138 8463 Acked Read 3 Done Sent Done false
31139 8464 Acked Write 8 Done Sent Done false
31140 8465 Acked Write 3 Done Sent Done false
31141 8466 Acked Read 5 Done Sent Done false
31142 8467 Acked Write 2 Done Sent Done false
31143 8468 Acked Read 3 Done Sent Done false
31144 8469 Acked Write 5 Done Sent Done false
31145 8470 Acked Read 1 Done Sent Done false
31146 8471 Acked Read 5 Done Sent Done false
31147 8472 Acked Read 7 Done Sent Done false
31148 8473 Acked Write 1 Done Sent Done false
31149 8474 Acked Flush 0 Done Sent Done false
31150 8475 Acked Flush 0 Done Sent Done false
31151 8476 Acked Write 7 Done Sent Done false
31152 8477 Acked Read 10 Done Sent Done false
31153 8478 Acked Flush 0 Done Sent Done false
31154 8479 Acked Read 10 Done Sent Done false
31155 8480 Acked Write 6 Done Sent Done false
31156 8481 Acked Write 7 Done Sent Done false
31157 8482 Acked Write 2 Done Sent Done false
31158 8483 Acked Write 1 Done Sent Done false
31159 8484 Acked Read 9 Done Sent Done false
31160 8485 Acked Flush 0 Done Sent Done false
31161 8486 Acked Read 3 Done Sent Done false
31162 8487 Acked Write 5 Done Sent Done false
31163 8488 Acked Read 3 Done Sent Done false
31164 8489 Acked Read 1 Done Sent Done false
31165 8490 Acked Read 2 Done Sent Done false
31166 8491 Acked Read 7 Done Sent Done false
31167 8492 Acked Write 3 Done Sent Done false
31168 8493 Acked Read 3 Done Sent Done false
31169 8494 Acked Write 5 Done Sent Done false
31170 8495 Acked Read 9 Done Sent Done false
31171 8496 Acked Read 2 Done Sent Done false
31172 8497 Acked Read 5 Done Sent Done false
31173 8498 Acked Read 6 Done Sent Done false
31174 8499 Acked Read 4 Done Sent Done false
31175 8500 Acked Read 2 Done Sent Done false
31176 8501 Acked Write 8 Done Sent Done false
31177 8502 Acked Read 7 Done Sent Done false
31178 8503 Acked Read 4 Done Sent Done false
31179 8504 Acked Read 10 Done Sent Done false
31180 8505 Acked Write 2 Done Sent Done false
31181 8506 Acked Write 6 Done Sent Done false
31182 8507 Acked Write 9 Done Sent Done false
31183 8508 Acked Read 6 Done Sent Done false
31184 8509 Acked Read 9 Done Sent Done false
31185 8510 Acked Read 8 Done Sent Done false
31186 8511 Acked Read 8 Done Sent Done false
31187 8512 Acked Read 4 Done Sent Done false
31188 8513 Acked Read 9 Done Sent Done false
31189 8514 Acked Write 3 Done Sent Done false
31190 8515 Acked Read 10 Done Sent Done false
31191 8516 Acked Read 3 Done Sent Done false
31192 8517 Acked Read 3 Done Sent Done false
31193 8518 Acked Read 6 Done Sent Done false
31194 8519 Acked Read 4 Done Sent Done false
31195 8520 Acked Flush 0 Done Sent Done false
31196 8521 Acked Write 5 Done Sent Done false
31197 8522 Acked Flush 0 Done Sent Done false
31198 8523 Acked Write 9 Done Sent Done false
31199 8524 Acked Write 5 Done Sent Done false
31200 8525 Acked Read 10 Done Sent Done false
31201 8526 Acked Write 8 Done Sent Done false
31202 8527 Acked Write 6 Done Sent Done false
31203 8528 Acked Write 5 Done Sent Done false
31204 8529 Acked Read 2 Done Sent Done false
31205 8530 Acked Write 9 Done Sent Done false
31206 8531 Acked Flush 0 Done Sent Done false
31207 8532 Acked Read 10 Done Sent Done false
31208 8533 Acked Write 3 Done Sent Done false
31209 8534 Acked Flush 0 Done Sent Done false
31210 8535 Acked Write 8 Done Sent Done false
31211 8536 Acked Read 5 Done Sent Done false
31212 8537 Acked Read 4 Done Sent Done false
31213 8538 Acked Write 8 Done Sent Done false
31214 8539 Acked Read 10 Done Sent Done false
31215 8540 Acked Write 3 Done Sent Done false
31216 8541 Acked Write 2 Done Sent Done false
31217 8542 Acked Write 5 Done Sent Done false
31218 8543 Acked Read 7 Done Sent Done false
31219 8544 Acked Read 2 Done Sent Done false
31220 8545 Acked Flush 0 Done Sent Done false
31221 8546 Acked Read 4 Done Sent Done false
31222 8547 Acked Read 9 Done Sent Done false
31223 8548 Acked Read 4 Done Sent Done false
31224 8549 Acked Flush 0 Done Sent Done false
31225 STATES DS:0 DS:1 DS:2 TOTAL
31226 Sent 0 302 0 302
31227 Done 302 0 302 604
31228 Skipped 0 0 0 0
31229 Error 0 0 0 0
31230 Last Flush: 8549 8247 8549
31231 Downstairs last five completed: 8247 8246 8245 8244 8243
31232 Upstairs last five completed: 8549 8548 8547 8546 8545
31233 CLIENT: Up:0 ds:302 act:2
31234 ----------------------------------------------------------------
31235 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31236 States: Active Offline Active
31237 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31238 8248 Acked Write 4 Done Sent Done false
31239 8249 Acked Read 2 Done Sent Done false
31240 8250 Acked Write 2 Done Sent Done false
31241 8251 Acked Read 4 Done Sent Done false
31242 8252 Acked Read 2 Done Sent Done false
31243 8253 Acked Flush 0 Done Sent Done false
31244 8254 Acked Read 1 Done Sent Done false
31245 8255 Acked Flush 0 Done Sent Done false
31246 8256 Acked Write 1 Done Sent Done false
31247 8257 Acked Write 7 Done Sent Done false
31248 8258 Acked Write 3 Done Sent Done false
31249 8259 Acked Write 2 Done Sent Done false
31250 8260 Acked Flush 0 Done Sent Done false
31251 8261 Acked Write 9 Done Sent Done false
31252 8262 Acked Read 8 Done Sent Done false
31253 8263 Acked Read 1 Done Sent Done false
31254 8264 Acked Read 9 Done Sent Done false
31255 8265 Acked Read 5 Done Sent Done false
31256 8266 Acked Write 8 Done Sent Done false
31257 8267 Acked Write 5 Done Sent Done false
31258 8268 Acked Read 6 Done Sent Done false
31259 8269 Acked Read 2 Done Sent Done false
31260 8270 Acked Read 5 Done Sent Done false
31261 8271 Acked Write 7 Done Sent Done false
31262 8272 Acked Read 6 Done Sent Done false
31263 8273 Acked Flush 0 Done Sent Done false
31264 8274 Acked Write 7 Done Sent Done false
31265 8275 Acked Read 7 Done Sent Done false
31266 8276 Acked Write 1 Done Sent Done false
31267 8277 Acked Write 6 Done Sent Done false
31268 8278 Acked Read 8 Done Sent Done false
31269 8279 Acked Read 5 Done Sent Done false
31270 8280 Acked Write 6 Done Sent Done false
31271 8281 Acked Read 4 Done Sent Done false
31272 8282 Acked Read 2 Done Sent Done false
31273 8283 Acked Read 8 Done Sent Done false
31274 8284 Acked Read 1 Done Sent Done false
31275 8285 Acked Write 7 Done Sent Done false
31276 8286 Acked Read 3 Done Sent Done false
31277 8287 Acked Read 5 Done Sent Done false
31278 8288 Acked Read 5 Done Sent Done false
31279 8289 Acked Read 10 Done Sent Done false
31280 8290 Acked Read 1 Done Sent Done false
31281 8291 Acked Write 6 Done Sent Done false
31282 8292 Acked Write 7 Done Sent Done false
31283 8293 Acked Flush 0 Done Sent Done false
31284 8294 Acked Read 6 Done Sent Done false
31285 8295 Acked Write 7 Done Sent Done false
31286 8296 Acked Read 10 Done Sent Done false
31287 8297 Acked Read 2 Done Sent Done false
31288 8298 Acked Read 1 Done Sent Done false
31289 8299 Acked Flush 0 Done Sent Done false
31290 8300 Acked Flush 0 Done Sent Done false
31291 8301 Acked Read 5 Done Sent Done false
31292 8302 Acked Write 8 Done Sent Done false
31293 8303 Acked Read 9 Done Sent Done false
31294 8304 Acked Write 2 Done Sent Done false
31295 8305 Acked Write 7 Done Sent Done false
31296 8306 Acked Read 9 Done Sent Done false
31297 8307 Acked Write 10 Done Sent Done false
31298 8308 Acked Write 8 Done Sent Done false
31299 8309 Acked Write 2 Done Sent Done false
31300 8310 Acked Write 9 Done Sent Done false
31301 8311 Acked Write 2 Done Sent Done false
31302 8312 Acked Write 7 Done Sent Done false
31303 8313 Acked Write 9 Done Sent Done false
31304 8314 Acked Read 6 Done Sent Done false
31305 8315 Acked Write 9 Done Sent Done false
31306 8316 Acked Read 8 Done Sent Done false
31307 8317 Acked Read 5 Done Sent Done false
31308 8318 Acked Flush 0 Done Sent Done false
31309 8319 Acked Read 7 Done Sent Done false
31310 8320 Acked Flush 0 Done Sent Done false
31311 8321 Acked Read 8 Done Sent Done false
31312 8322 Acked Write 9 Done Sent Done false
31313 8323 Acked Read 8 Done Sent Done false
31314 8324 Acked Write 10 Done Sent Done false
31315 8325 Acked Flush 0 Done Sent Done false
31316 8326 Acked Write 6 Done Sent Done false
31317 8327 Acked Write 3 Done Sent Done false
31318 8328 Acked Write 6 Done Sent Done false
31319 8329 Acked Write 10 Done Sent Done false
31320 8330 Acked Write 4 Done Sent Done false
31321 8331 Acked Write 10 Done Sent Done false
31322 8332 Acked Read 7 Done Sent Done false
31323 8333 Acked Write 1 Done Sent Done false
31324 8334 Acked Write 10 Done Sent Done false
31325 8335 Acked Write 4 Done Sent Done false
31326 8336 Acked Write 8 Done Sent Done false
31327 8337 Acked Write 1 Done Sent Done false
31328 8338 Acked Read 5 Done Sent Done false
31329 8339 Acked Write 2 Done Sent Done false
31330 8340 Acked Read 5 Done Sent Done false
31331 8341 Acked Read 2 Done Sent Done false
31332 8342 Acked Write 4 Done Sent Done false
31333 8343 Acked Read 3 Done Sent Done false
31334 8344 Acked Flush 0 Done Sent Done false
31335 8345 Acked Read 4 Done Sent Done false
31336 8346 Acked Read 10 Done Sent Done false
31337 8347 Acked Write 1 Done Sent Done false
31338 8348 Acked Write 6 Done Sent Done false
31339 8349 Acked Write 4 Done Sent Done false
31340 8350 Acked Read 10 Done Sent Done false
31341 8351 Acked Read 1 Done Sent Done false
31342 8352 Acked Read 10 Done Sent Done false
31343 8353 Acked Write 10 Done Sent Done false
31344 8354 Acked Read 1 Done Sent Done false
31345 8355 Acked Read 9 Done Sent Done false
31346 8356 Acked Write 3 Done Sent Done false
31347 8357 Acked Read 7 Done Sent Done false
31348 8358 Acked Flush 0 Done Sent Done false
31349 8359 Acked Read 8 Done Sent Done false
31350 8360 Acked Write 10 Done Sent Done false
31351 8361 Acked Write 3 Done Sent Done false
31352 8362 Acked Write 7 Done Sent Done false
31353 8363 Acked Write 10 Done Sent Done false
31354 8364 Acked Write 7 Done Sent Done false
31355 8365 Acked Read 5 Done Sent Done false
31356 8366 Acked Write 2 Done Sent Done false
31357 8367 Acked Write 4 Done Sent Done false
31358 8368 Acked Read 10 Done Sent Done false
31359 8369 Acked Write 3 Done Sent Done false
31360 8370 Acked Flush 0 Done Sent Done false
31361 8371 Acked Read 9 Done Sent Done false
31362 8372 Acked Read 3 Done Sent Done false
31363 8373 Acked Write 1 Done Sent Done false
31364 8374 Acked Read 3 Done Sent Done false
31365 8375 Acked Write 9 Done Sent Done false
31366 8376 Acked Write 10 Done Sent Done false
31367 8377 Acked Write 2 Done Sent Done false
31368 8378 Acked Write 9 Done Sent Done false
31369 8379 Acked Write 5 Done Sent Done false
31370 8380 Acked Flush 0 Done Sent Done false
31371 8381 Acked Write 9 Done Sent Done false
31372 8382 Acked Write 9 Done Sent Done false
31373 8383 Acked Write 10 Done Sent Done false
31374 8384 Acked Write 6 Done Sent Done false
31375 8385 Acked Read 7 Done Sent Done false
31376 8386 Acked Write 4 Done Sent Done false
31377 8387 Acked Read 10 Done Sent Done false
31378 8388 Acked Flush 0 Done Sent Done false
31379 8389 Acked Write 3 Done Sent Done false
31380 8390 Acked Read 9 Done Sent Done false
31381 8391 Acked Read 8 Done Sent Done false
31382 8392 Acked Flush 0 Done Sent Done false
31383 8393 Acked Write 10 Done Sent Done false
31384 8394 Acked Write 6 Done Sent Done false
31385 8395 Acked Read 5 Done Sent Done false
31386 8396 Acked Flush 0 Done Sent Done false
31387 8397 Acked Write 1 Done Sent Done false
31388 8398 Acked Read 5 Done Sent Done false
31389 8399 Acked Write 4 Done Sent Done false
31390 8400 Acked Write 3 Done Sent Done false
31391 8401 Acked Read 2 Done Sent Done false
31392 8402 Acked Write 10 Done Sent Done false
31393 8403 Acked Write 1 Done Sent Done false
31394 8404 Acked Read 1 Done Sent Done false
31395 8405 Acked Read 7 Done Sent Done false
31396 8406 Acked Flush 0 Done Sent Done false
31397 8407 Acked Write 1 Done Sent Done false
31398 8408 Acked Write 10 Done Sent Done false
31399 8409 Acked Write 4 Done Sent Done false
31400 8410 Acked Write 2 Done Sent Done false
31401 8411 Acked Read 3 Done Sent Done false
31402 8412 Acked Read 8 Done Sent Done false
31403 8413 Acked Write 1 Done Sent Done false
31404 8414 Acked Read 1 Done Sent Done false
31405 8415 Acked Flush 0 Done Sent Done false
31406 8416 Acked Read 7 Done Sent Done false
31407 8417 Acked Read 2 Done Sent Done false
31408 8418 Acked Flush 0 Done Sent Done false
31409 8419 Acked Write 4 Done Sent Done false
31410 8420 Acked Read 6 Done Sent Done false
31411 8421 Acked Read 1 Done Sent Done false
31412 8422 Acked Flush 0 Done Sent Done false
31413 8423 Acked Read 10 Done Sent Done false
31414 8424 Acked Write 10 Done Sent Done false
31415 8425 Acked Flush 0 Done Sent Done false
31416 8426 Acked Write 9 Done Sent Done false
31417 8427 Acked Read 2 Done Sent Done false
31418 8428 Acked Write 8 Done Sent Done false
31419 8429 Acked Write 4 Done Sent Done false
31420 8430 Acked Flush 0 Done Sent Done false
31421 8431 Acked Flush 0 Done Sent Done false
31422 8432 Acked Write 2 Done Sent Done false
31423 8433 Acked Read 8 Done Sent Done false
31424 8434 Acked Read 2 Done Sent Done false
31425 8435 Acked Read 5 Done Sent Done false
31426 8436 Acked Write 2 Done Sent Done false
31427 8437 Acked Read 5 Done Sent Done false
31428 8438 Acked Read 10 Done Sent Done false
31429 8439 Acked Flush 0 Done Sent Done false
31430 8440 Acked Read 8 Done Sent Done false
31431 8441 Acked Read 7 Done Sent Done false
31432 8442 Acked Read 9 Done Sent Done false
31433 8443 Acked Write 10 Done Sent Done false
31434 8444 Acked Read 2 Done Sent Done false
31435 8445 Acked Flush 0 Done Sent Done false
31436 8446 Acked Write 9 Done Sent Done false
31437 8447 Acked Read 8 Done Sent Done false
31438 8448 Acked Write 9 Done Sent Done false
31439 8449 Acked Read 9 Done Sent Done false
31440 8450 Acked Read 2 Done Sent Done false
31441 8451 Acked Write 2 Done Sent Done false
31442 8452 Acked Flush 0 Done Sent Done false
31443 8453 Acked Read 1 Done Sent Done false
31444 8454 Acked Read 2 Done Sent Done false
31445 8455 Acked Read 6 Done Sent Done false
31446 8456 Acked Read 2 Done Sent Done false
31447 8457 Acked Read 1 Done Sent Done false
31448 8458 Acked Read 9 Done Sent Done false
31449 8459 Acked Read 1 Done Sent Done false
31450 8460 Acked Write 6 Done Sent Done false
31451 8461 Acked Read 2 Done Sent Done false
31452 8462 Acked Read 2 Done Sent Done false
31453 8463 Acked Read 3 Done Sent Done false
31454 8464 Acked Write 8 Done Sent Done false
31455 8465 Acked Write 3 Done Sent Done false
31456 8466 Acked Read 5 Done Sent Done false
31457 8467 Acked Write 2 Done Sent Done false
31458 8468 Acked Read 3 Done Sent Done false
31459 8469 Acked Write 5 Done Sent Done false
31460 8470 Acked Read 1 Done Sent Done false
31461 8471 Acked Read 5 Done Sent Done false
31462 8472 Acked Read 7 Done Sent Done false
31463 8473 Acked Write 1 Done Sent Done false
31464 8474 Acked Flush 0 Done Sent Done false
31465 8475 Acked Flush 0 Done Sent Done false
31466 8476 Acked Write 7 Done Sent Done false
31467 8477 Acked Read 10 Done Sent Done false
31468 8478 Acked Flush 0 Done Sent Done false
31469 8479 Acked Read 10 Done Sent Done false
31470 8480 Acked Write 6 Done Sent Done false
31471 8481 Acked Write 7 Done Sent Done false
31472 8482 Acked Write 2 Done Sent Done false
31473 8483 Acked Write 1 Done Sent Done false
31474 8484 Acked Read 9 Done Sent Done false
31475 8485 Acked Flush 0 Done Sent Done false
31476 8486 Acked Read 3 Done Sent Done false
31477 8487 Acked Write 5 Done Sent Done false
31478 8488 Acked Read 3 Done Sent Done false
31479 8489 Acked Read 1 Done Sent Done false
31480 8490 Acked Read 2 Done Sent Done false
31481 8491 Acked Read 7 Done Sent Done false
31482 8492 Acked Write 3 Done Sent Done false
31483 8493 Acked Read 3 Done Sent Done false
31484 8494 Acked Write 5 Done Sent Done false
31485 8495 Acked Read 9 Done Sent Done false
31486 8496 Acked Read 2 Done Sent Done false
31487 8497 Acked Read 5 Done Sent Done false
31488 8498 Acked Read 6 Done Sent Done false
31489 8499 Acked Read 4 Done Sent Done false
31490 8500 Acked Read 2 Done Sent Done false
31491 8501 Acked Write 8 Done Sent Done false
31492 8502 Acked Read 7 Done Sent Done false
31493 8503 Acked Read 4 Done Sent Done false
31494 8504 Acked Read 10 Done Sent Done false
31495 8505 Acked Write 2 Done Sent Done false
31496 8506 Acked Write 6 Done Sent Done false
31497 8507 Acked Write 9 Done Sent Done false
31498 8508 Acked Read 6 Done Sent Done false
31499 8509 Acked Read 9 Done Sent Done false
31500 8510 Acked Read 8 Done Sent Done false
31501 8511 Acked Read 8 Done Sent Done false
31502 8512 Acked Read 4 Done Sent Done false
31503 8513 Acked Read 9 Done Sent Done false
31504 8514 Acked Write 3 Done Sent Done false
31505 8515 Acked Read 10 Done Sent Done false
31506 8516 Acked Read 3 Done Sent Done false
31507 8517 Acked Read 3 Done Sent Done false
31508 8518 Acked Read 6 Done Sent Done false
31509 8519 Acked Read 4 Done Sent Done false
31510 8520 Acked Flush 0 Done Sent Done false
31511 8521 Acked Write 5 Done Sent Done false
31512 8522 Acked Flush 0 Done Sent Done false
31513 8523 Acked Write 9 Done Sent Done false
31514 8524 Acked Write 5 Done Sent Done false
31515 8525 Acked Read 10 Done Sent Done false
31516 8526 Acked Write 8 Done Sent Done false
31517 8527 Acked Write 6 Done Sent Done false
31518 8528 Acked Write 5 Done Sent Done false
31519 8529 Acked Read 2 Done Sent Done false
31520 8530 Acked Write 9 Done Sent Done false
31521 8531 Acked Flush 0 Done Sent Done false
31522 8532 Acked Read 10 Done Sent Done false
31523 8533 Acked Write 3 Done Sent Done false
31524 8534 Acked Flush 0 Done Sent Done false
31525 8535 Acked Write 8 Done Sent Done false
31526 8536 Acked Read 5 Done Sent Done false
31527 8537 Acked Read 4 Done Sent Done false
31528 8538 Acked Write 8 Done Sent Done false
31529 8539 Acked Read 10 Done Sent Done false
31530 8540 Acked Write 3 Done Sent Done false
31531 8541 Acked Write 2 Done Sent Done false
31532 8542 Acked Write 5 Done Sent Done false
31533 8543 Acked Read 7 Done Sent Done false
31534 8544 Acked Read 2 Done Sent Done false
31535 8545 Acked Flush 0 Done Sent Done false
31536 8546 Acked Read 4 Done Sent Done false
31537 8547 Acked Read 9 Done Sent Done false
31538 8548 Acked Read 4 Done Sent Done false
31539 8549 Acked Flush 0 Done Sent Done false
31540 STATES DS:0 DS:1 DS:2 TOTAL
31541 Sent 0 302 0 302
31542 Done 302 0 302 604
31543 Skipped 0 0 0 0
31544 Error 0 0 0 0
31545 Last Flush: 8549 8247 8549
31546 Downstairs last five completed: 8247 8246 8245 8244 8243
31547 Upstairs last five completed: 8549 8548 8547 8546 8545
31548 CLIENT: Up:0 ds:302 act:2
31549 ----------------------------------------------------------------
31550 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31551 States: Active Offline Active
31552 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31553 8248 Acked Write 4 Done Sent Done false
31554 8249 Acked Read 2 Done Sent Done false
31555 8250 Acked Write 2 Done Sent Done false
31556 8251 Acked Read 4 Done Sent Done false
31557 8252 Acked Read 2 Done Sent Done false
31558 8253 Acked Flush 0 Done Sent Done false
31559 8254 Acked Read 1 Done Sent Done false
31560 8255 Acked Flush 0 Done Sent Done false
31561 8256 Acked Write 1 Done Sent Done false
31562 8257 Acked Write 7 Done Sent Done false
31563 8258 Acked Write 3 Done Sent Done false
31564 8259 Acked Write 2 Done Sent Done false
31565 8260 Acked Flush 0 Done Sent Done false
31566 8261 Acked Write 9 Done Sent Done false
31567 8262 Acked Read 8 Done Sent Done false
31568 8263 Acked Read 1 Done Sent Done false
31569 8264 Acked Read 9 Done Sent Done false
31570 8265 Acked Read 5 Done Sent Done false
31571 8266 Acked Write 8 Done Sent Done false
31572 8267 Acked Write 5 Done Sent Done false
31573 8268 Acked Read 6 Done Sent Done false
31574 8269 Acked Read 2 Done Sent Done false
31575 8270 Acked Read 5 Done Sent Done false
31576 8271 Acked Write 7 Done Sent Done false
31577 8272 Acked Read 6 Done Sent Done false
31578 8273 Acked Flush 0 Done Sent Done false
31579 8274 Acked Write 7 Done Sent Done false
31580 8275 Acked Read 7 Done Sent Done false
31581 8276 Acked Write 1 Done Sent Done false
31582 8277 Acked Write 6 Done Sent Done false
31583 8278 Acked Read 8 Done Sent Done false
31584 8279 Acked Read 5 Done Sent Done false
31585 8280 Acked Write 6 Done Sent Done false
31586 8281 Acked Read 4 Done Sent Done false
31587 8282 Acked Read 2 Done Sent Done false
31588 8283 Acked Read 8 Done Sent Done false
31589 8284 Acked Read 1 Done Sent Done false
31590 8285 Acked Write 7 Done Sent Done false
31591 8286 Acked Read 3 Done Sent Done false
31592 8287 Acked Read 5 Done Sent Done false
31593 8288 Acked Read 5 Done Sent Done false
31594 8289 Acked Read 10 Done Sent Done false
31595 8290 Acked Read 1 Done Sent Done false
31596 8291 Acked Write 6 Done Sent Done false
31597 8292 Acked Write 7 Done Sent Done false
31598 8293 Acked Flush 0 Done Sent Done false
31599 8294 Acked Read 6 Done Sent Done false
31600 8295 Acked Write 7 Done Sent Done false
31601 8296 Acked Read 10 Done Sent Done false
31602 8297 Acked Read 2 Done Sent Done false
31603 8298 Acked Read 1 Done Sent Done false
31604 8299 Acked Flush 0 Done Sent Done false
31605 8300 Acked Flush 0 Done Sent Done false
31606 8301 Acked Read 5 Done Sent Done false
31607 8302 Acked Write 8 Done Sent Done false
31608 8303 Acked Read 9 Done Sent Done false
31609 8304 Acked Write 2 Done Sent Done false
31610 8305 Acked Write 7 Done Sent Done false
31611 8306 Acked Read 9 Done Sent Done false
31612 8307 Acked Write 10 Done Sent Done false
31613 8308 Acked Write 8 Done Sent Done false
31614 8309 Acked Write 2 Done Sent Done false
31615 8310 Acked Write 9 Done Sent Done false
31616 8311 Acked Write 2 Done Sent Done false
31617 8312 Acked Write 7 Done Sent Done false
31618 8313 Acked Write 9 Done Sent Done false
31619 8314 Acked Read 6 Done Sent Done false
31620 8315 Acked Write 9 Done Sent Done false
31621 8316 Acked Read 8 Done Sent Done false
31622 8317 Acked Read 5 Done Sent Done false
31623 8318 Acked Flush 0 Done Sent Done false
31624 8319 Acked Read 7 Done Sent Done false
31625 8320 Acked Flush 0 Done Sent Done false
31626 8321 Acked Read 8 Done Sent Done false
31627 8322 Acked Write 9 Done Sent Done false
31628 8323 Acked Read 8 Done Sent Done false
31629 8324 Acked Write 10 Done Sent Done false
31630 8325 Acked Flush 0 Done Sent Done false
31631 8326 Acked Write 6 Done Sent Done false
31632 8327 Acked Write 3 Done Sent Done false
31633 8328 Acked Write 6 Done Sent Done false
31634 8329 Acked Write 10 Done Sent Done false
31635 8330 Acked Write 4 Done Sent Done false
31636 8331 Acked Write 10 Done Sent Done false
31637 8332 Acked Read 7 Done Sent Done false
31638 8333 Acked Write 1 Done Sent Done false
31639 8334 Acked Write 10 Done Sent Done false
31640 8335 Acked Write 4 Done Sent Done false
31641 8336 Acked Write 8 Done Sent Done false
31642 8337 Acked Write 1 Done Sent Done false
31643 8338 Acked Read 5 Done Sent Done false
31644 8339 Acked Write 2 Done Sent Done false
31645 8340 Acked Read 5 Done Sent Done false
31646 8341 Acked Read 2 Done Sent Done false
31647 8342 Acked Write 4 Done Sent Done false
31648 8343 Acked Read 3 Done Sent Done false
31649 8344 Acked Flush 0 Done Sent Done false
31650 8345 Acked Read 4 Done Sent Done false
31651 8346 Acked Read 10 Done Sent Done false
31652 8347 Acked Write 1 Done Sent Done false
31653 8348 Acked Write 6 Done Sent Done false
31654 8349 Acked Write 4 Done Sent Done false
31655 8350 Acked Read 10 Done Sent Done false
31656 8351 Acked Read 1 Done Sent Done false
31657 8352 Acked Read 10 Done Sent Done false
31658 8353 Acked Write 10 Done Sent Done false
31659 8354 Acked Read 1 Done Sent Done false
31660 8355 Acked Read 9 Done Sent Done false
31661 8356 Acked Write 3 Done Sent Done false
31662 8357 Acked Read 7 Done Sent Done false
31663 8358 Acked Flush 0 Done Sent Done false
31664 8359 Acked Read 8 Done Sent Done false
31665 8360 Acked Write 10 Done Sent Done false
31666 8361 Acked Write 3 Done Sent Done false
31667 8362 Acked Write 7 Done Sent Done false
31668 8363 Acked Write 10 Done Sent Done false
31669 8364 Acked Write 7 Done Sent Done false
31670 8365 Acked Read 5 Done Sent Done false
31671 8366 Acked Write 2 Done Sent Done false
31672 8367 Acked Write 4 Done Sent Done false
31673 8368 Acked Read 10 Done Sent Done false
31674 8369 Acked Write 3 Done Sent Done false
31675 8370 Acked Flush 0 Done Sent Done false
31676 8371 Acked Read 9 Done Sent Done false
31677 8372 Acked Read 3 Done Sent Done false
31678 8373 Acked Write 1 Done Sent Done false
31679 8374 Acked Read 3 Done Sent Done false
31680 8375 Acked Write 9 Done Sent Done false
31681 8376 Acked Write 10 Done Sent Done false
31682 8377 Acked Write 2 Done Sent Done false
31683 8378 Acked Write 9 Done Sent Done false
31684 8379 Acked Write 5 Done Sent Done false
31685 8380 Acked Flush 0 Done Sent Done false
31686 8381 Acked Write 9 Done Sent Done false
31687 8382 Acked Write 9 Done Sent Done false
31688 8383 Acked Write 10 Done Sent Done false
31689 8384 Acked Write 6 Done Sent Done false
31690 8385 Acked Read 7 Done Sent Done false
31691 8386 Acked Write 4 Done Sent Done false
31692 8387 Acked Read 10 Done Sent Done false
31693 8388 Acked Flush 0 Done Sent Done false
31694 8389 Acked Write 3 Done Sent Done false
31695 8390 Acked Read 9 Done Sent Done false
31696 8391 Acked Read 8 Done Sent Done false
31697 8392 Acked Flush 0 Done Sent Done false
31698 8393 Acked Write 10 Done Sent Done false
31699 8394 Acked Write 6 Done Sent Done false
31700 8395 Acked Read 5 Done Sent Done false
31701 8396 Acked Flush 0 Done Sent Done false
31702 8397 Acked Write 1 Done Sent Done false
31703 8398 Acked Read 5 Done Sent Done false
31704 8399 Acked Write 4 Done Sent Done false
31705 8400 Acked Write 3 Done Sent Done false
31706 8401 Acked Read 2 Done Sent Done false
31707 8402 Acked Write 10 Done Sent Done false
31708 8403 Acked Write 1 Done Sent Done false
31709 8404 Acked Read 1 Done Sent Done false
31710 8405 Acked Read 7 Done Sent Done false
31711 8406 Acked Flush 0 Done Sent Done false
31712 8407 Acked Write 1 Done Sent Done false
31713 8408 Acked Write 10 Done Sent Done false
31714 8409 Acked Write 4 Done Sent Done false
31715 8410 Acked Write 2 Done Sent Done false
31716 8411 Acked Read 3 Done Sent Done false
31717 8412 Acked Read 8 Done Sent Done false
31718 8413 Acked Write 1 Done Sent Done false
31719 8414 Acked Read 1 Done Sent Done false
31720 8415 Acked Flush 0 Done Sent Done false
31721 8416 Acked Read 7 Done Sent Done false
31722 8417 Acked Read 2 Done Sent Done false
31723 8418 Acked Flush 0 Done Sent Done false
31724 8419 Acked Write 4 Done Sent Done false
31725 8420 Acked Read 6 Done Sent Done false
31726 8421 Acked Read 1 Done Sent Done false
31727 8422 Acked Flush 0 Done Sent Done false
31728 8423 Acked Read 10 Done Sent Done false
31729 8424 Acked Write 10 Done Sent Done false
31730 8425 Acked Flush 0 Done Sent Done false
31731 8426 Acked Write 9 Done Sent Done false
31732 8427 Acked Read 2 Done Sent Done false
31733 8428 Acked Write 8 Done Sent Done false
31734 8429 Acked Write 4 Done Sent Done false
31735 8430 Acked Flush 0 Done Sent Done false
31736 8431 Acked Flush 0 Done Sent Done false
31737 8432 Acked Write 2 Done Sent Done false
31738 8433 Acked Read 8 Done Sent Done false
31739 8434 Acked Read 2 Done Sent Done false
31740 8435 Acked Read 5 Done Sent Done false
31741 8436 Acked Write 2 Done Sent Done false
31742 8437 Acked Read 5 Done Sent Done false
31743 8438 Acked Read 10 Done Sent Done false
31744 8439 Acked Flush 0 Done Sent Done false
31745 8440 Acked Read 8 Done Sent Done false
31746 8441 Acked Read 7 Done Sent Done false
31747 8442 Acked Read 9 Done Sent Done false
31748 8443 Acked Write 10 Done Sent Done false
31749 8444 Acked Read 2 Done Sent Done false
31750 8445 Acked Flush 0 Done Sent Done false
31751 8446 Acked Write 9 Done Sent Done false
31752 8447 Acked Read 8 Done Sent Done false
31753 8448 Acked Write 9 Done Sent Done false
31754 8449 Acked Read 9 Done Sent Done false
31755 8450 Acked Read 2 Done Sent Done false
31756 8451 Acked Write 2 Done Sent Done false
31757 8452 Acked Flush 0 Done Sent Done false
31758 8453 Acked Read 1 Done Sent Done false
31759 8454 Acked Read 2 Done Sent Done false
31760 8455 Acked Read 6 Done Sent Done false
31761 8456 Acked Read 2 Done Sent Done false
31762 8457 Acked Read 1 Done Sent Done false
31763 8458 Acked Read 9 Done Sent Done false
31764 8459 Acked Read 1 Done Sent Done false
31765 8460 Acked Write 6 Done Sent Done false
31766 8461 Acked Read 2 Done Sent Done false
31767 8462 Acked Read 2 Done Sent Done false
31768 8463 Acked Read 3 Done Sent Done false
31769 8464 Acked Write 8 Done Sent Done false
31770 8465 Acked Write 3 Done Sent Done false
31771 8466 Acked Read 5 Done Sent Done false
31772 8467 Acked Write 2 Done Sent Done false
31773 8468 Acked Read 3 Done Sent Done false
31774 8469 Acked Write 5 Done Sent Done false
31775 8470 Acked Read 1 Done Sent Done false
31776 8471 Acked Read 5 Done Sent Done false
31777 8472 Acked Read 7 Done Sent Done false
31778 8473 Acked Write 1 Done Sent Done false
31779 8474 Acked Flush 0 Done Sent Done false
31780 8475 Acked Flush 0 Done Sent Done false
31781 8476 Acked Write 7 Done Sent Done false
31782 8477 Acked Read 10 Done Sent Done false
31783 8478 Acked Flush 0 Done Sent Done false
31784 8479 Acked Read 10 Done Sent Done false
31785 8480 Acked Write 6 Done Sent Done false
31786 8481 Acked Write 7 Done Sent Done false
31787 8482 Acked Write 2 Done Sent Done false
31788 8483 Acked Write 1 Done Sent Done false
31789 8484 Acked Read 9 Done Sent Done false
31790 8485 Acked Flush 0 Done Sent Done false
31791 8486 Acked Read 3 Done Sent Done false
31792 8487 Acked Write 5 Done Sent Done false
31793 8488 Acked Read 3 Done Sent Done false
31794 8489 Acked Read 1 Done Sent Done false
31795 8490 Acked Read 2 Done Sent Done false
31796 8491 Acked Read 7 Done Sent Done false
31797 8492 Acked Write 3 Done Sent Done false
31798 8493 Acked Read 3 Done Sent Done false
31799 8494 Acked Write 5 Done Sent Done false
31800 8495 Acked Read 9 Done Sent Done false
31801 8496 Acked Read 2 Done Sent Done false
31802 8497 Acked Read 5 Done Sent Done false
31803 8498 Acked Read 6 Done Sent Done false
31804 8499 Acked Read 4 Done Sent Done false
31805 8500 Acked Read 2 Done Sent Done false
31806 8501 Acked Write 8 Done Sent Done false
31807 8502 Acked Read 7 Done Sent Done false
31808 8503 Acked Read 4 Done Sent Done false
31809 8504 Acked Read 10 Done Sent Done false
31810 8505 Acked Write 2 Done Sent Done false
31811 8506 Acked Write 6 Done Sent Done false
31812 8507 Acked Write 9 Done Sent Done false
31813 8508 Acked Read 6 Done Sent Done false
31814 8509 Acked Read 9 Done Sent Done false
31815 8510 Acked Read 8 Done Sent Done false
31816 8511 Acked Read 8 Done Sent Done false
31817 8512 Acked Read 4 Done Sent Done false
31818 8513 Acked Read 9 Done Sent Done false
31819 8514 Acked Write 3 Done Sent Done false
31820 8515 Acked Read 10 Done Sent Done false
31821 8516 Acked Read 3 Done Sent Done false
31822 8517 Acked Read 3 Done Sent Done false
31823 8518 Acked Read 6 Done Sent Done false
31824 8519 Acked Read 4 Done Sent Done false
31825 8520 Acked Flush 0 Done Sent Done false
31826 8521 Acked Write 5 Done Sent Done false
31827 8522 Acked Flush 0 Done Sent Done false
31828 8523 Acked Write 9 Done Sent Done false
31829 8524 Acked Write 5 Done Sent Done false
31830 8525 Acked Read 10 Done Sent Done false
31831 8526 Acked Write 8 Done Sent Done false
31832 8527 Acked Write 6 Done Sent Done false
31833 8528 Acked Write 5 Done Sent Done false
31834 8529 Acked Read 2 Done Sent Done false
31835 8530 Acked Write 9 Done Sent Done false
31836 8531 Acked Flush 0 Done Sent Done false
31837 8532 Acked Read 10 Done Sent Done false
31838 8533 Acked Write 3 Done Sent Done false
31839 8534 Acked Flush 0 Done Sent Done false
31840 8535 Acked Write 8 Done Sent Done false
31841 8536 Acked Read 5 Done Sent Done false
31842 8537 Acked Read 4 Done Sent Done false
31843 8538 Acked Write 8 Done Sent Done false
31844 8539 Acked Read 10 Done Sent Done false
31845 8540 Acked Write 3 Done Sent Done false
31846 8541 Acked Write 2 Done Sent Done false
31847 8542 Acked Write 5 Done Sent Done false
31848 8543 Acked Read 7 Done Sent Done false
31849 8544 Acked Read 2 Done Sent Done false
31850 8545 Acked Flush 0 Done Sent Done false
31851 8546 Acked Read 4 Done Sent Done false
31852 8547 Acked Read 9 Done Sent Done false
31853 8548 Acked Read 4 Done Sent Done false
31854 8549 Acked Flush 0 Done Sent Done false
31855 STATES DS:0 DS:1 DS:2 TOTAL
31856 Sent 0 302 0 302
31857 Done 302 0 302 604
31858 Skipped 0 0 0 0
31859 Error 0 0 0 0
31860 Last Flush: 8549 8247 8549
31861 Downstairs last five completed: 8247 8246 8245 8244 8243
31862 Upstairs last five completed: 8549 8548 8547 8546 8545
31863 CLIENT: Up:0 ds:302 act:2
318642024-12-11T23:47:10.405ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318652024-12-11T23:47:10.405ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41696) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318662024-12-11T23:47:10.406ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318672024-12-11T23:47:10.407ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318682024-12-11T23:47:10.407ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318692024-12-11T23:47:10.407ZINFOcrucible: send last flush ID to this DS: 8247 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318702024-12-11T23:47:10.407ZINFOcrucible: Replied this last flush ID: 8247 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318712024-12-11T23:47:10.407ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318722024-12-11T23:47:10.407ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318732024-12-11T23:47:10.407ZINFOcrucible: [1] client re-new 302 jobs since flush 8247 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318742024-12-11T23:47:10.407ZINFOcrucible: [1] Replayed 302 jobs since flush: 8247 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
31875 ----------------------------------------------------------------
31876 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31877 Downstairs last five completed: 8549 8548 8547 8546 8545
31878 Upstairs last five completed: 8549 8548 8547 8546 8545
31879 CLIENT: Up:0 ds:0 act:3
31880 Replay: All jobs finished
31881 Main task has work to do, go find it
31882 got dsc Stop(0)
31883 stop 0
31884 Dec 11 23:47:12.654[8810] Got stop action so:false kr:false
31885 INFO request completed, latency_us: 348, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 2a1d360f-0623-4ad0-86ed-e7c2200069ee, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
31886 Dec 11 23:47:12.654 INFO request completed, latency_us: 110, response_code: 200, uri: /state/cid/0, method: GET, req_id: baa8e438-a2f8-430e-bb78-9ea1342bfd98, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
318872024-12-11T23:47:12.654ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318882024-12-11T23:47:12.654ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318892024-12-11T23:47:12.654ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318902024-12-11T23:47:12.654ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318912024-12-11T23:47:12.655ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
318922024-12-11T23:47:12.655ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
31893 [8810] Exited with: ExitStatus(unix_wait_status(9))
31894 [8810][0] reports Exit
31895 Dec 11 23:47:16.657 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/0, method: GET, req_id: 13abc75a-c384-44d8-9e4d-bffdb9cc5b4e, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
31896 001/300 Write block 3967 len 24576 data: 7 3 4 6 5 5
31897 002/300 Read block 998 len 16384
31898 003/300 Write block 3978 len 40960 data: 6 5 4 4 3 3 3 3 4 4
31899 004/300 Read block 2566 len 24576
31900 005/300 Read block 1793 len 4096
31901 006/300 Write block 3368 len 8192 data: 5 5
31902 007/300 Write block 297 len 8192 data: 6 7
31903 008/300 Write block 4212 len 12288 data: 6 6 7
31904 009/300 Write block 3869 len 28672 data: 9 9 9 8 5 4 4
31905 010/300 Read block 3628 len 8192
31906 011/300 Read block 97 len 16384
31907 012/300 Read block 1614 len 20480
31908 013/300 Write block 4811 len 36864 data: 6 6 8 6 6 5 5 4 4
31909 014/300 Write block 4420 len 24576 data: 4 5 5 5 8 7
31910 015/300 Flush
31911 016/300 Read block 3083 len 4096
31912 017/300 Read block 4947 len 40960
31913 018/300 Read block 3599 len 40960
31914 019/300 Read block 4797 len 36864
31915 020/300 Read block 1826 len 40960
31916 021/300 Write block 4711 len 28672 data: 4 4 4 5 6 6 7
31917 022/300 Read block 4895 len 8192
31918 023/300 Read block 3310 len 24576
31919 024/300 Read block 2235 len 4096
31920 025/300 Write block 2004 len 16384 data: 3 3 3 4
31921 026/300 Write block 4085 len 24576 data: 6 6 7 7 6 7
31922 027/300 Write block 3917 len 28672 data: 4 5 5 5 5 7 6
31923 028/300 Read block 2200 len 8192
31924 029/300 Flush
31925 030/300 Write block 3170 len 8192 data: 7 6
31926 031/300 Read block 1916 len 40960
31927 032/300 Write block 551 len 36864 data: 6 4 4 2 3 3 3 3 3
31928 033/300 Read block 1711 len 16384
31929 034/300 Write block 2027 len 28672 data: 5 5 5 7 5 6 7
31930 035/300 Read block 3569 len 28672
31931 036/300 Write block 4467 len 20480 data: 5 5 8 9 9
31932 037/300 Flush
31933 038/300 Read block 2152 len 36864
31934 039/300 Read block 2992 len 24576
31935 040/300 Read block 2746 len 8192
31936 041/300 Write block 2693 len 4096 data: 7
31937 042/300 Read block 3887 len 16384
31938 043/300 Write block 4506 len 16384 data: 4 5 5 5
31939 044/300 Write block 4921 len 24576 data: 9 12 10 10 8 8
31940 045/300 Read block 4653 len 12288
31941 046/300 Read block 1270 len 36864
31942 047/300 Read block 982 len 8192
31943 048/300 Read block 1015 len 28672
31944 049/300 Flush
31945 050/300 Read block 2668 len 36864
31946 051/300 Flush
31947 052/300 Read block 308 len 32768
31948 053/300 Write block 4294 len 4096 data: 4
31949 054/300 Write block 4350 len 8192 data: 6 5
31950 055/300 Flush
31951 056/300 Write block 3486 len 28672 data: 8 11 11 10 10 8 7
31952 057/300 Read block 3377 len 28672
31953 058/300 Write block 2161 len 36864 data: 4 3 3 4 4 5 4 5 5
31954 059/300 Write block 1790 len 20480 data: 4 5 6 6 6
31955 060/300 Write block 81 len 32768 data: 3 4 4 4 6 5 5 5
31956 061/300 Write block 642 len 4096 data: 7
31957 062/300 Flush
31958 063/300 Read block 1499 len 20480
31959 064/300 Write block 922 len 4096 data: 4
31960 065/300 Write block 3764 len 8192 data: 4 5
31961 066/300 Write block 4785 len 20480 data: 7 7 6 5 4
31962 067/300 Write block 3087 len 20480 data: 7 7 9 9 7
31963 068/300 Read block 4778 len 16384
31964 069/300 Read block 4576 len 16384
31965 070/300 Read block 783 len 20480
31966 071/300 Flush
31967 072/300 Read block 4355 len 32768
31968 073/300 Read block 1498 len 12288
31969 074/300 Read block 3492 len 4096
31970 075/300 Write block 3103 len 40960 data: 6 7 7 6 6 6 5 4 3 3
31971 076/300 Read block 4053 len 28672
31972 077/300 Write block 2169 len 40960 data: 6 5 5 6 6 6 5 5 3 4
31973 078/300 Flush
31974 079/300 Write block 252 len 16384 data: 5 4 4 4
31975 080/300 Read block 873 len 40960
31976 081/300 Read block 1503 len 4096
31977 082/300 Read block 3912 len 20480
31978 083/300 Read block 3811 len 40960
31979 084/300 Write block 4848 len 8192 data: 4 4
31980 085/300 Write block 979 len 16384 data: 6 6 6 5
31981 086/300 Write block 2323 len 36864 data: 2 2 4 5 6 8 8 9 7
31982 087/300 Read block 2916 len 16384
31983 088/300 Read block 3748 len 28672
31984 089/300 Read block 3448 len 8192
31985 090/300 Read block 1168 len 8192
31986 091/300 Write block 1196 len 20480 data: 8 6 6 5 6
31987 092/300 Write block 3706 len 28672 data: 7 7 6 6 5 4 4
31988 093/300 Write block 431 len 36864 data: 4 4 3 3 2 2 2 2 2
31989 094/300 Read block 3128 len 36864
31990 095/300 Write block 2954 len 28672 data: 4 4 3 4 5 6 6
31991 096/300 Write block 1975 len 24576 data: 7 7 7 7 7 7
31992 097/300 Write block 1177 len 16384 data: 5 5 5 7
31993 098/300 Flush
31994 099/300 Read block 1996 len 12288
31995 100/300 Write block 1605 len 4096 data: 4
31996 101/300 Read block 978 len 28672
31997 102/300 Write block 4296 len 24576 data: 7 6 6 5 4 5
31998 103/300 Read block 1231 len 40960
31999 104/300 Write block 3973 len 8192 data: 5 5
32000 105/300 Write block 3247 len 36864 data: 4 3 3 3 3 2 4 4 4
32001 106/300 Read block 2668 len 28672
32002 107/300 Write block 32 len 4096 data: 5
32003 108/300 Read block 573 len 24576
32004 109/300 Write block 3803 len 12288 data: 6 5 4
32005 110/300 Write block 2083 len 24576 data: 3 3 4 5 4 4
32006 111/300 Read block 3955 len 36864
32007 112/300 Write block 4181 len 16384 data: 8 7 6 6
32008 113/300 Write block 879 len 24576 data: 7 7 8 8 10 10
32009 114/300 Write block 2039 len 16384 data: 6 5 6 5
32010 115/300 Write block 666 len 12288 data: 5 6 6
32011 116/300 Read block 277 len 28672
32012 117/300 Read block 1656 len 32768
32013 118/300 Write block 3742 len 20480 data: 9 8 7 6 6
32014 119/300 Read block 817 len 28672
32015 120/300 Write block 633 len 20480 data: 5 3 4 6 8
32016 121/300 Read block 2336 len 16384
32017 122/300 Read block 3223 len 28672
32018 123/300 Write block 743 len 40960 data: 4 4 2 2 3 4 5 5 6 8
32019 124/300 Read block 3407 len 16384
32020 125/300 Read block 4420 len 12288
32021 126/300 Write block 2239 len 24576 data: 5 5 7 8 8 8
32022 127/300 Write block 4361 len 20480 data: 4 6 6 5 4
32023 128/300 Read block 4004 len 20480
32024 129/300 Write block 1225 len 12288 data: 4 4 5
32025 130/300 Read block 4970 len 20480
32026 131/300 Read block 2647 len 16384
32027 132/300 Write block 3468 len 16384 data: 4 5 4 3
32028 133/300 Read block 2296 len 4096
32029 134/300 Read block 3491 len 24576
32030 135/300 Read block 2985 len 8192
32031 136/300 Read block 2527 len 36864
32032 137/300 Write block 436 len 4096 data: 3
32033 138/300 Write block 635 len 36864 data: 5 7 9 9 11 10 7 8 6
32034 139/300 Write block 3025 len 8192 data: 4 4
32035 140/300 Read block 4523 len 28672
32036 141/300 Read block 3384 len 24576
32037 142/300 Read block 3948 len 40960
32038 143/300 Read block 4004 len 20480
32039 144/300 Read block 261 len 24576
32040 145/300 Read block 501 len 36864
32041 146/300 Read block 1575 len 4096
32042 147/300 Read block 1523 len 4096
32043 148/300 Write block 3715 len 32768 data: 6 6 6 7 7 7 6 5
32044 149/300 Read block 2496 len 16384
32045 150/300 Read block 4483 len 16384
32046 151/300 Flush
32047 152/300 Read block 4796 len 40960
32048 153/300 Write block 3284 len 16384 data: 7 6 6 6
32049 154/300 Read block 3309 len 4096
32050 155/300 Write block 4825 len 36864 data: 4 6 7 7 6 4 5 6 6
32051 156/300 Write block 2876 len 24576 data: 2 3 3 4 4 3
32052 157/300 Read block 1142 len 12288
32053 158/300 Write block 1481 len 24576 data: 4 4 6 6 5 4
32054 159/300 Read block 4777 len 16384
32055 160/300 Write block 4562 len 28672 data: 7 7 6 8 6 4 6
32056 161/300 Write block 3838 len 32768 data: 4 3 3 4 3 3 4 3
32057 162/300 Flush
32058 163/300 Write block 4119 len 16384 data: 8 7 7 10
32059 164/300 Read block 3008 len 20480
32060 165/300 Write block 1020 len 8192 data: 5 6
32061 166/300 Read block 2528 len 32768
32062 167/300 Read block 3359 len 20480
32063 168/300 Write block 4369 len 40960 data: 2 3 3 3 3 3 4 4 3 6
32064 169/300 Write block 3240 len 12288 data: 5 6 7
32065 170/300 Write block 3820 len 8192 data: 5 3
32066 171/300 Flush
32067 172/300 Read block 2742 len 28672
32068 173/300 Read block 2218 len 24576
32069 174/300 Read block 2866 len 36864
32070 175/300 Write block 3562 len 8192 data: 4 4
32071 176/300 Write block 2220 len 20480 data: 5 5 6 5 6
32072 177/300 Write block 1489 len 20480 data: 4 5 7 9 9
32073 178/300 Read block 1676 len 36864
32074 179/300 Read block 4698 len 40960
32075 180/300 Read block 312 len 36864
32076 181/300 Write block 341 len 24576 data: 7 8 9 10 10 10
32077 182/300 Write block 870 len 16384 data: 5 6 6 7
32078 183/300 Read block 1563 len 36864
32079 184/300 Read block 3103 len 12288
32080 185/300 Read block 1424 len 4096
32081 186/300 Write block 2012 len 24576 data: 5 5 6 3 3 4
32082 187/300 Read block 4788 len 32768
32083 188/300 Write block 4476 len 20480 data: 5 4 4 4 4
32084 189/300 Read block 3001 len 8192
32085 190/300 Flush
32086 191/300 Read block 2821 len 8192
32087 192/300 Read block 400 len 40960
32088 193/300 Flush
32089 194/300 Flush
32090 195/300 Read block 2903 len 36864
32091 196/300 Write block 4302 len 16384 data: 4 4 4 4
32092 197/300 Read block 3551 len 12288
32093 198/300 Write block 3889 len 32768 data: 8 7 7 7 7 8 8 8
32094 199/300 Write block 1904 len 20480 data: 4 4 5 5 6
32095 200/300 Read block 3846 len 24576
32096 201/300 Read block 1981 len 12288
32097 202/300 Write block 3979 len 32768 data: 6 5 5 4 4 4 4 5
32098 203/300 Write block 4632 len 16384 data: 6 6 6 5
32099 204/300 Flush
32100 205/300 Write block 4367 len 28672 data: 3 2 3 4 4 4 4
32101 206/300 Read block 3912 len 4096
32102 207/300 Write block 2751 len 8192 data: 2 2
32103 208/300 Write block 3019 len 20480 data: 6 4 4 4 5
32104 209/300 Read block 2201 len 36864
32105 210/300 Read block 2524 len 24576
32106 211/300 Read block 1714 len 12288
32107 212/300 Write block 4085 len 36864 data: 7 7 8 8 7 8 8 6 6
32108 213/300 Write block 2076 len 16384 data: 2 2 2 2
32109 214/300 Write block 2597 len 24576 data: 5 6 6 7 6 6
32110 215/300 Write block 3621 len 40960 data: 8 6 6 4 3 4 5 7 8 8
32111 216/300 Read block 4774 len 40960
32112 217/300 Read block 1899 len 20480
32113 218/300 Write block 1565 len 36864 data: 7 6 7 5 4 5 8 6 5
32114 219/300 Read block 640 len 24576
32115 220/300 Read block 3397 len 40960
32116 221/300 Write block 3993 len 24576 data: 6 5 5 5 3 5
32117 222/300 Write block 4211 len 40960 data: 8 7 7 8 7 6 6 7 7 5
32118 223/300 Write block 4453 len 8192 data: 5 4
32119 224/300 Read block 4955 len 32768
32120 225/300 Write block 1017 len 4096 data: 5
32121 226/300 Write block 3354 len 24576 data: 3 3 3 4 4 4
32122 227/300 Read block 1929 len 20480
32123 228/300 Write block 3579 len 8192 data: 5 6
32124 229/300 Read block 3683 len 20480
32125 230/300 Read block 2692 len 20480
32126 231/300 Read block 3987 len 28672
32127 232/300 Read block 1358 len 40960
32128 233/300 Read block 2151 len 20480
32129 234/300 Write block 3123 len 4096 data: 4
32130 235/300 Write block 4275 len 28672 data: 2 2 3 3 3 3 5
32131 236/300 Read block 959 len 16384
32132 237/300 Flush
32133 238/300 Flush
32134 239/300 Read block 4949 len 40960
32135 240/300 Flush
32136 241/300 Write block 1731 len 36864 data: 4 3 4 3 3 4 3 3 3
32137 242/300 Flush
32138 243/300 Read block 1998 len 36864
32139 244/300 Write block 1805 len 40960 data: 4 4 5 5 5 5 5 4 4 3
32140 245/300 Write block 2966 len 32768 data: 8 5 5 3 4 4 3 2
32141 246/300 Read block 4041 len 32768
32142 247/300 Write block 98 len 4096 data: 4
32143 248/300 Read block 2351 len 32768
32144 249/300 Write block 328 len 40960 data: 8 8 6 7 7 5 6 4 4 4
32145 250/300 Read block 848 len 36864
32146 251/300 Read block 2379 len 32768
32147 252/300 Write block 1172 len 8192 data: 4 4
32148 253/300 Write block 103 len 8192 data: 8 10
32149 254/300 Read block 4236 len 28672
32150 255/300 Read block 4087 len 20480
32151 256/300 Write block 3696 len 16384 data: 7 8 7 6
32152 257/300 Read block 2868 len 24576
32153 258/300 Read block 2239 len 36864
32154 259/300 Flush
32155 260/300 Read block 725 len 36864
32156 261/300 Write block 3467 len 36864 data: 4 5 6 5 4 4 4 4 5
32157 262/300 Read block 1149 len 4096
32158 263/300 Read block 2917 len 28672
32159 264/300 Write block 1469 len 4096 data: 7
32160 265/300 Write block 4369 len 28672 data: 4 5 5 5 5 4 5
32161 266/300 Write block 1497 len 40960 data: 7 9 9 9 8 9 9 7 6 7
32162 267/300 Write block 158 len 32768 data: 6 6 5 7 5 5 6 6
32163 268/300 Read block 3149 len 24576
32164 269/300 Read block 4538 len 4096
32165 270/300 Write block 3186 len 4096 data: 3
32166 271/300 Read block 3263 len 40960
32167 272/300 Read block 4470 len 28672
32168 273/300 Read block 1293 len 16384
32169 274/300 Read block 4419 len 24576
32170 275/300 Read block 1885 len 16384
32171 276/300 Read block 1944 len 36864
32172 277/300 Read block 541 len 36864
32173 278/300 Flush
32174 279/300 Write block 544 len 28672 data: 4 6 5 6 5 5 5
32175 280/300 Read block 1396 len 24576
32176 281/300 Write block 48 len 36864 data: 6 8 9 7 6 5 6 3 3
32177 282/300 Flush
32178 283/300 Read block 2439 len 8192
32179 284/300 Read block 342 len 40960
32180 285/300 Read block 3787 len 12288
32181 286/300 Flush
32182 287/300 Read block 4938 len 8192
32183 288/300 Write block 1425 len 24576 data: 3 5 4 5 7 6
32184 289/300 Write block 2484 len 16384 data: 5 5 4 4
32185 290/300 Write block 2111 len 36864 data: 5 5 7 8 7 8 6 5 6
32186 291/300 Read block 1393 len 20480
32187 292/300 Read block 4978 len 28672
32188 293/300 Read block 1896 len 40960
32189 294/300 Read block 4669 len 12288
32190 295/300 Read block 2390 len 36864
32191 296/300 Read block 264 len 8192
32192 297/300 Read block 1482 len 32768
32193 298/300 Read block 4410 len 36864
32194 299/300 Write block 2534 len 28672 data: 3 3 2 3 4 5 4
32195 300/300 Write block 4832 len 20480 data: 7 7 5 4 4
32196 301/300 Flush
32197 Main task has work to do, go find it
32198 got dsc Start(0)
32199 start 0
32200 Dec 11 23:47:16.901 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/0, method: POST, req_id: a891239a-4c2e-4cb9-8320-291694e5ef33, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
32201 Replay: started 0, returned:Ok(())
32202 ----------------------------------------------------------------
32203 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32204 States: Offline Active Active
32205 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32206 8550 Acked Write 6 Sent Done Done false
32207 8551 Acked Read 4 Sent Done Done false
32208 8552 Acked Write 10 Sent Done Done false
32209 8553 Acked Read 6 Sent Done Done false
32210 8554 Acked Read 1 Sent Done Done false
32211 8555 Acked Write 2 Sent Done Done false
32212 8556 Acked Write 2 Sent Done Done false
32213 8557 Acked Write 3 Sent Done Done false
32214 8558 Acked Write 7 Sent Done Done false
32215 8559 Acked Read 2 Sent Done Done false
32216 8560 Acked Read 4 Sent Done Done false
32217 8561 Acked Read 5 Sent Done Done false
32218 8562 Acked Write 9 Sent Done Done false
32219 8563 Acked Write 6 Sent Done Done false
32220 8564 Acked Flush 0 Sent Done Done false
32221 8565 Acked Read 1 Sent Done Done false
32222 8566 Acked Read 10 Sent Done Done false
32223 8567 Acked Read 10 Sent Done Done false
32224 8568 Acked Read 9 Sent Done Done false
32225 8569 Acked Read 10 Sent Done Done false
32226 8570 Acked Write 7 Sent Done Done false
32227 8571 Acked Read 2 Sent Done Done false
32228 8572 Acked Read 6 Sent Done Done false
32229 8573 Acked Read 1 Sent Done Done false
32230 8574 Acked Write 4 Sent Done Done false
32231 8575 Acked Write 6 Sent Done Done false
32232 8576 Acked Write 7 Sent Done Done false
32233 8577 Acked Read 2 Sent Done Done false
32234 8578 Acked Flush 0 Sent Done Done false
32235 8579 Acked Write 2 Sent Done Done false
32236 8580 Acked Read 10 Sent Done Done false
32237 8581 Acked Write 9 Sent Done Done false
32238 8582 Acked Read 4 Sent Done Done false
32239 8583 Acked Write 7 Sent Done Done false
32240 8584 Acked Read 7 Sent Done Done false
32241 8585 Acked Write 5 Sent Done Done false
32242 8586 Acked Flush 0 Sent Done Done false
32243 8587 Acked Read 9 Sent Done Done false
32244 8588 Acked Read 6 Sent Done Done false
32245 8589 Acked Read 2 Sent Done Done false
32246 8590 Acked Write 1 Sent Done Done false
32247 8591 Acked Read 4 Sent Done Done false
32248 8592 Acked Write 4 Sent Done Done false
32249 8593 Acked Write 6 Sent Done Done false
32250 8594 Acked Read 3 Sent Done Done false
32251 8595 Acked Read 9 Sent Done Done false
32252 8596 Acked Read 2 Sent Done Done false
32253 8597 Acked Read 7 Sent Done Done false
32254 8598 Acked Flush 0 Sent Done Done false
32255 8599 Acked Read 9 Sent Done Done false
32256 8600 Acked Flush 0 Sent Done Done false
32257 8601 Acked Read 8 Sent Done Done false
32258 8602 Acked Write 1 Sent Done Done false
32259 8603 Acked Write 2 Sent Done Done false
32260 8604 Acked Flush 0 Sent Done Done false
32261 8605 Acked Write 7 Sent Done Done false
32262 8606 Acked Read 7 Sent Done Done false
32263 8607 Acked Write 9 Sent Done Done false
32264 8608 Acked Write 5 Sent Done Done false
32265 8609 Acked Write 8 Sent Done Done false
32266 8610 Acked Write 1 Sent Done Done false
32267 8611 Acked Flush 0 Sent Done Done false
32268 8612 Acked Read 5 Sent Done Done false
32269 8613 Acked Write 1 Sent Done Done false
32270 8614 Acked Write 2 Sent Done Done false
32271 8615 Acked Write 5 Sent Done Done false
32272 8616 Acked Write 5 Sent Done Done false
32273 8617 Acked Read 4 Sent Done Done false
32274 8618 Acked Read 4 Sent Done Done false
32275 8619 Acked Read 5 Sent Done Done false
32276 8620 Acked Flush 0 Sent Done Done false
32277 8621 Acked Read 8 Sent Done Done false
32278 8622 Acked Read 3 Sent Done Done false
32279 8623 Acked Read 1 Sent Done Done false
32280 8624 Acked Write 10 Sent Done Done false
32281 8625 Acked Read 7 Sent Done Done false
32282 8626 Acked Write 10 Sent Done Done false
32283 8627 Acked Flush 0 Sent Done Done false
32284 8628 Acked Write 4 Sent Done Done false
32285 8629 Acked Read 10 Sent Done Done false
32286 8630 Acked Read 1 Sent Done Done false
32287 8631 Acked Read 5 Sent Done Done false
32288 8632 Acked Read 10 Sent Done Done false
32289 8633 Acked Write 2 Sent Done Done false
32290 8634 Acked Write 4 Sent Done Done false
32291 8635 Acked Write 9 Sent Done Done false
32292 8636 Acked Read 4 Sent Done Done false
32293 8637 Acked Read 7 Sent Done Done false
32294 8638 Acked Read 2 Sent Done Done false
32295 8639 Acked Read 2 Sent Done Done false
32296 8640 Acked Write 5 Sent Done Done false
32297 8641 Acked Write 7 Sent Done Done false
32298 8642 Acked Write 9 Sent Done Done false
32299 8643 Acked Read 9 Sent Done Done false
32300 8644 Acked Write 7 Sent Done Done false
32301 8645 Acked Write 6 Sent Done Done false
32302 8646 Acked Write 4 Sent Done Done false
32303 8647 Acked Flush 0 Sent Done Done false
32304 8648 Acked Read 3 Sent Done Done false
32305 8649 Acked Write 1 Sent Done Done false
32306 8650 Acked Read 7 Sent Done Done false
32307 8651 Acked Write 6 Sent Done Done false
32308 8652 Acked Read 10 Sent Done Done false
32309 8653 Acked Write 2 Sent Done Done false
32310 8654 Acked Write 9 Sent Done Done false
32311 8655 Acked Read 7 Sent Done Done false
32312 8656 Acked Write 1 Sent Done Done false
32313 8657 Acked Read 6 Sent Done Done false
32314 8658 Acked Write 3 Sent Done Done false
32315 8659 Acked Write 6 Sent Done Done false
32316 8660 Acked Read 9 Sent Done Done false
32317 8661 Acked Write 4 Sent Done Done false
32318 8662 Acked Write 6 Sent Done Done false
32319 8663 Acked Write 4 Sent Done Done false
32320 8664 Acked Write 3 Sent Done Done false
32321 8665 Acked Read 7 Sent Done Done false
32322 8666 Acked Read 8 Sent Done Done false
32323 8667 Acked Write 5 Sent Done Done false
32324 8668 Acked Read 7 Sent Done Done false
32325 8669 Acked Write 5 Sent Done Done false
32326 8670 Acked Read 4 Sent Done Done false
32327 8671 Acked Read 7 Sent Done Done false
32328 8672 Acked Write 10 Sent Done Done false
32329 8673 Acked Read 4 Sent Done Done false
32330 8674 Acked Read 3 Sent Done Done false
32331 8675 Acked Write 6 Sent Done Done false
32332 8676 Acked Write 5 Sent Done Done false
32333 8677 Acked Read 5 Sent Done Done false
32334 8678 Acked Write 3 Sent Done Done false
32335 8679 Acked Read 5 Sent Done Done false
32336 8680 Acked Read 4 Sent Done Done false
32337 8681 Acked Write 4 Sent Done Done false
32338 8682 Acked Read 1 Sent Done Done false
32339 8683 Acked Read 6 Sent Done Done false
32340 8684 Acked Read 2 Sent Done Done false
32341 8685 Acked Read 9 Sent Done Done false
32342 8686 Acked Write 1 Sent Done Done false
32343 8687 Acked Write 9 Sent Done Done false
32344 8688 Acked Write 2 Sent Done Done false
32345 8689 Acked Read 7 Sent Done Done false
32346 8690 Acked Read 6 Sent Done Done false
32347 8691 Acked Read 10 Sent Done Done false
32348 8692 Acked Read 5 Sent Done Done false
32349 8693 Acked Read 6 Sent Done Done false
32350 8694 Acked Read 9 Sent Done Done false
32351 8695 Acked Read 1 Sent Done Done false
32352 8696 Acked Read 1 Sent Done Done false
32353 8697 Acked Write 8 Sent Done Done false
32354 8698 Acked Read 4 Sent Done Done false
32355 8699 Acked Read 4 Sent Done Done false
32356 8700 Acked Flush 0 Sent Done Done false
32357 8701 Acked Read 10 Sent Done Done false
32358 8702 Acked Write 4 Sent Done Done false
32359 8703 Acked Read 1 Sent Done Done false
32360 8704 Acked Write 9 Sent Done Done false
32361 8705 Acked Write 6 Sent Done Done false
32362 8706 Acked Read 3 Sent Done Done false
32363 8707 Acked Write 6 Sent Done Done false
32364 8708 Acked Read 4 Sent Done Done false
32365 8709 Acked Write 7 Sent Done Done false
32366 8710 Acked Write 8 Sent Done Done false
32367 8711 Acked Flush 0 Sent Done Done false
32368 8712 Acked Write 4 Sent Done Done false
32369 8713 Acked Read 5 Sent Done Done false
32370 8714 Acked Write 2 Sent Done Done false
32371 8715 Acked Read 8 Sent Done Done false
32372 8716 Acked Read 5 Sent Done Done false
32373 8717 Acked Write 10 Sent Done Done false
32374 8718 Acked Write 3 Sent Done Done false
32375 8719 Acked Write 2 Sent Done Done false
32376 8720 Acked Flush 0 Sent Done Done false
32377 8721 Acked Read 7 Sent Done Done false
32378 8722 Acked Read 6 Sent Done Done false
32379 8723 Acked Read 9 Sent Done Done false
32380 8724 Acked Write 2 Sent Done Done false
32381 8725 Acked Write 5 Sent Done Done false
32382 8726 Acked Write 5 Sent Done Done false
32383 8727 Acked Read 9 Sent Done Done false
32384 8728 Acked Read 10 Sent Done Done false
32385 8729 Acked Read 9 Sent Done Done false
32386 8730 Acked Write 6 Sent Done Done false
32387 8731 Acked Write 4 Sent Done Done false
32388 8732 Acked Read 9 Sent Done Done false
32389 8733 Acked Read 3 Sent Done Done false
32390 8734 Acked Read 1 Sent Done Done false
32391 8735 Acked Write 6 Sent Done Done false
32392 8736 Acked Read 8 Sent Done Done false
32393 8737 Acked Write 5 Sent Done Done false
32394 8738 Acked Read 2 Sent Done Done false
32395 8739 Acked Flush 0 Sent Done Done false
32396 8740 Acked Read 2 Sent Done Done false
32397 8741 Acked Read 10 Sent Done Done false
32398 8742 Acked Flush 0 Sent Done Done false
32399 8743 Acked Flush 0 Sent Done Done false
32400 8744 Acked Read 9 Sent Done Done false
32401 8745 Acked Write 4 Sent Done Done false
32402 8746 Acked Read 3 Sent Done Done false
32403 8747 Acked Write 8 Sent Done Done false
32404 8748 Acked Write 5 Sent Done Done false
32405 8749 Acked Read 6 Sent Done Done false
32406 8750 Acked Read 3 Sent Done Done false
32407 8751 Acked Write 8 Sent Done Done false
32408 8752 Acked Write 4 Sent Done Done false
32409 8753 Acked Flush 0 Sent Done Done false
32410 8754 Acked Write 7 Sent Done Done false
32411 8755 Acked Read 1 Sent Done Done false
32412 8756 Acked Write 2 Sent Done Done false
32413 8757 Acked Write 5 Sent Done Done false
32414 8758 Acked Read 9 Sent Done Done false
32415 8759 Acked Read 6 Sent Done Done false
32416 8760 Acked Read 3 Sent Done Done false
32417 8761 Acked Write 9 Sent Done Done false
32418 8762 Acked Write 4 Sent Done Done false
32419 8763 Acked Write 6 Sent Done Done false
32420 8764 Acked Write 10 Sent Done Done false
32421 8765 Acked Read 10 Sent Done Done false
32422 8766 Acked Read 5 Sent Done Done false
32423 8767 Acked Write 9 Sent Done Done false
32424 8768 Acked Read 6 Sent Done Done false
32425 8769 Acked Read 10 Sent Done Done false
32426 8770 Acked Write 6 Sent Done Done false
32427 8771 Acked Write 10 Sent Done Done false
32428 8772 Acked Write 2 Sent Done Done false
32429 8773 Acked Read 8 Sent Done Done false
32430 8774 Acked Write 1 Sent Done Done false
32431 8775 Acked Write 6 Sent Done Done false
32432 8776 Acked Read 5 Sent Done Done false
32433 8777 Acked Write 2 Sent Done Done false
32434 8778 Acked Read 5 Sent Done Done false
32435 8779 Acked Read 5 Sent Done Done false
32436 8780 Acked Read 7 Sent Done Done false
32437 8781 Acked Read 10 Sent Done Done false
32438 8782 Acked Read 5 Sent Done Done false
32439 8783 Acked Write 1 Sent Done Done false
32440 8784 Acked Write 7 Sent Done Done false
32441 8785 Acked Read 4 Sent Done Done false
32442 8786 Acked Flush 0 Sent Done Done false
32443 8787 Acked Flush 0 Sent Done Done false
32444 8788 Acked Read 10 Sent Done Done false
32445 8789 Acked Flush 0 Sent Done Done false
32446 8790 Acked Write 9 Sent Done Done false
32447 8791 Acked Flush 0 Sent Done Done false
32448 8792 Acked Read 9 Sent Done Done false
32449 8793 Acked Write 10 Sent Done Done false
32450 8794 Acked Write 8 Sent Done Done false
32451 8795 Acked Read 8 Sent Done Done false
32452 8796 Acked Write 1 Sent Done Done false
32453 8797 Acked Read 8 Sent Done Done false
32454 8798 Acked Write 10 Sent Done Done false
32455 8799 Acked Read 9 Sent Done Done false
32456 8800 Acked Read 8 Sent Done Done false
32457 8801 Acked Write 2 Sent Done Done false
32458 8802 Acked Write 2 Sent Done Done false
32459 8803 Acked Read 7 Sent Done Done false
32460 8804 Acked Read 5 Sent Done Done false
32461 8805 Acked Write 4 Sent Done Done false
32462 8806 Acked Read 6 Sent Done Done false
32463 8807 Acked Read 9 Sent Done Done false
32464 8808 Acked Flush 0 Sent Done Done false
32465 8809 Acked Read 9 Sent Done Done false
32466 8810 Acked Write 9 Sent Done Done false
32467 8811 Acked Read 1 Sent Done Done false
32468 8812 Acked Read 7 Sent Done Done false
32469 8813 Acked Write 1 Sent Done Done false
32470 8814 Acked Write 7 Sent Done Done false
32471 8815 Acked Write 10 Sent Done Done false
32472 8816 Acked Write 8 Sent Done Done false
32473 8817 Acked Read 6 Sent Done Done false
32474 8818 Acked Read 1 Sent Done Done false
32475 8819 Acked Write 1 Sent Done Done false
32476 8820 Acked Read 10 Sent Done Done false
32477 8821 Acked Read 7 Sent Done Done false
32478 8822 Acked Read 4 Sent Done Done false
32479 8823 Acked Read 6 Sent Done Done false
32480 8824 Acked Read 4 Sent Done Done false
32481 8825 Acked Read 9 Sent Done Done false
32482 8826 Acked Read 9 Sent Done Done false
32483 8827 Acked Flush 0 Sent Done Done false
32484 8828 Acked Write 7 Sent Done Done false
32485 8829 Acked Read 6 Sent Done Done false
32486 8830 Acked Write 9 Sent Done Done false
32487 8831 Acked Flush 0 Sent Done Done false
32488 8832 Acked Read 2 Sent Done Done false
32489 8833 Acked Read 10 Sent Done Done false
32490 8834 Acked Read 3 Sent Done Done false
32491 8835 Acked Flush 0 Sent Done Done false
32492 8836 Acked Read 2 Sent Done Done false
32493 8837 Acked Write 6 Sent Done Done false
32494 8838 Acked Write 4 Sent Done Done false
32495 8839 Acked Write 9 Sent Done Done false
32496 8840 Acked Read 5 Sent Done Done false
32497 8841 Acked Read 7 Sent Done Done false
32498 8842 Acked Read 10 Sent Done Done false
32499 8843 Acked Read 3 Sent Done Done false
32500 8844 Acked Read 9 Sent Done Done false
32501 8845 Acked Read 2 Sent Done Done false
32502 8846 Acked Read 8 Sent Done Done false
32503 8847 Acked Read 9 Sent Done Done false
32504 8848 Acked Write 7 Sent Done Done false
32505 8849 Acked Write 5 Sent Done Done false
32506 8850 Acked Flush 0 Sent Done Done false
32507 STATES DS:0 DS:1 DS:2 TOTAL
32508 Sent 301 0 0 301
32509 Done 0 301 301 602
32510 Skipped 0 0 0 0
32511 Error 0 0 0 0
32512 Last Flush: 8549 8850 8850
32513 Downstairs last five completed: 8549 8548 8547 8546 8545
32514 Upstairs last five completed: 8850 8849 8848 8847 8846
32515 CLIENT: Up:0 ds:301 act:2
32516 [8810] got start action
32517 [8810] I am going to restart
32518 Starting downstairs at port 8810
32519 Make output file at "/tmp/dsc/downstairs-8810.txt"
32520 [8810][0] reports Starting
32521 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1226)
32522 [8810][0] reports Running
32523 ----------------------------------------------------------------
32524 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32525 States: Offline Active Active
32526 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32527 8550 Acked Write 6 Sent Done Done false
32528 8551 Acked Read 4 Sent Done Done false
32529 8552 Acked Write 10 Sent Done Done false
32530 8553 Acked Read 6 Sent Done Done false
32531 8554 Acked Read 1 Sent Done Done false
32532 8555 Acked Write 2 Sent Done Done false
32533 8556 Acked Write 2 Sent Done Done false
32534 8557 Acked Write 3 Sent Done Done false
32535 8558 Acked Write 7 Sent Done Done false
32536 8559 Acked Read 2 Sent Done Done false
32537 8560 Acked Read 4 Sent Done Done false
32538 8561 Acked Read 5 Sent Done Done false
32539 8562 Acked Write 9 Sent Done Done false
32540 8563 Acked Write 6 Sent Done Done false
32541 8564 Acked Flush 0 Sent Done Done false
32542 8565 Acked Read 1 Sent Done Done false
32543 8566 Acked Read 10 Sent Done Done false
32544 8567 Acked Read 10 Sent Done Done false
32545 8568 Acked Read 9 Sent Done Done false
32546 8569 Acked Read 10 Sent Done Done false
32547 8570 Acked Write 7 Sent Done Done false
32548 8571 Acked Read 2 Sent Done Done false
32549 8572 Acked Read 6 Sent Done Done false
32550 8573 Acked Read 1 Sent Done Done false
32551 8574 Acked Write 4 Sent Done Done false
32552 8575 Acked Write 6 Sent Done Done false
32553 8576 Acked Write 7 Sent Done Done false
32554 8577 Acked Read 2 Sent Done Done false
32555 8578 Acked Flush 0 Sent Done Done false
32556 8579 Acked Write 2 Sent Done Done false
32557 8580 Acked Read 10 Sent Done Done false
32558 8581 Acked Write 9 Sent Done Done false
32559 8582 Acked Read 4 Sent Done Done false
32560 8583 Acked Write 7 Sent Done Done false
32561 8584 Acked Read 7 Sent Done Done false
32562 8585 Acked Write 5 Sent Done Done false
32563 8586 Acked Flush 0 Sent Done Done false
32564 8587 Acked Read 9 Sent Done Done false
32565 8588 Acked Read 6 Sent Done Done false
32566 8589 Acked Read 2 Sent Done Done false
32567 8590 Acked Write 1 Sent Done Done false
32568 8591 Acked Read 4 Sent Done Done false
32569 8592 Acked Write 4 Sent Done Done false
32570 8593 Acked Write 6 Sent Done Done false
32571 8594 Acked Read 3 Sent Done Done false
32572 8595 Acked Read 9 Sent Done Done false
32573 8596 Acked Read 2 Sent Done Done false
32574 8597 Acked Read 7 Sent Done Done false
32575 8598 Acked Flush 0 Sent Done Done false
32576 8599 Acked Read 9 Sent Done Done false
32577 8600 Acked Flush 0 Sent Done Done false
32578 8601 Acked Read 8 Sent Done Done false
32579 8602 Acked Write 1 Sent Done Done false
32580 8603 Acked Write 2 Sent Done Done false
32581 8604 Acked Flush 0 Sent Done Done false
32582 8605 Acked Write 7 Sent Done Done false
32583 8606 Acked Read 7 Sent Done Done false
32584 8607 Acked Write 9 Sent Done Done false
32585 8608 Acked Write 5 Sent Done Done false
32586 8609 Acked Write 8 Sent Done Done false
32587 8610 Acked Write 1 Sent Done Done false
32588 8611 Acked Flush 0 Sent Done Done false
32589 8612 Acked Read 5 Sent Done Done false
32590 8613 Acked Write 1 Sent Done Done false
32591 8614 Acked Write 2 Sent Done Done false
32592 8615 Acked Write 5 Sent Done Done false
32593 8616 Acked Write 5 Sent Done Done false
32594 8617 Acked Read 4 Sent Done Done false
32595 8618 Acked Read 4 Sent Done Done false
32596 8619 Acked Read 5 Sent Done Done false
32597 8620 Acked Flush 0 Sent Done Done false
32598 8621 Acked Read 8 Sent Done Done false
32599 8622 Acked Read 3 Sent Done Done false
32600 8623 Acked Read 1 Sent Done Done false
32601 8624 Acked Write 10 Sent Done Done false
32602 8625 Acked Read 7 Sent Done Done false
32603 8626 Acked Write 10 Sent Done Done false
32604 8627 Acked Flush 0 Sent Done Done false
32605 8628 Acked Write 4 Sent Done Done false
32606 8629 Acked Read 10 Sent Done Done false
32607 8630 Acked Read 1 Sent Done Done false
32608 8631 Acked Read 5 Sent Done Done false
32609 8632 Acked Read 10 Sent Done Done false
32610 8633 Acked Write 2 Sent Done Done false
32611 8634 Acked Write 4 Sent Done Done false
32612 8635 Acked Write 9 Sent Done Done false
32613 8636 Acked Read 4 Sent Done Done false
32614 8637 Acked Read 7 Sent Done Done false
32615 8638 Acked Read 2 Sent Done Done false
32616 8639 Acked Read 2 Sent Done Done false
32617 8640 Acked Write 5 Sent Done Done false
32618 8641 Acked Write 7 Sent Done Done false
32619 8642 Acked Write 9 Sent Done Done false
32620 8643 Acked Read 9 Sent Done Done false
32621 8644 Acked Write 7 Sent Done Done false
32622 8645 Acked Write 6 Sent Done Done false
32623 8646 Acked Write 4 Sent Done Done false
32624 8647 Acked Flush 0 Sent Done Done false
32625 8648 Acked Read 3 Sent Done Done false
32626 8649 Acked Write 1 Sent Done Done false
32627 8650 Acked Read 7 Sent Done Done false
32628 8651 Acked Write 6 Sent Done Done false
32629 8652 Acked Read 10 Sent Done Done false
32630 8653 Acked Write 2 Sent Done Done false
32631 8654 Acked Write 9 Sent Done Done false
32632 8655 Acked Read 7 Sent Done Done false
32633 8656 Acked Write 1 Sent Done Done false
32634 8657 Acked Read 6 Sent Done Done false
32635 8658 Acked Write 3 Sent Done Done false
32636 8659 Acked Write 6 Sent Done Done false
32637 8660 Acked Read 9 Sent Done Done false
32638 8661 Acked Write 4 Sent Done Done false
32639 8662 Acked Write 6 Sent Done Done false
32640 8663 Acked Write 4 Sent Done Done false
32641 8664 Acked Write 3 Sent Done Done false
32642 8665 Acked Read 7 Sent Done Done false
32643 8666 Acked Read 8 Sent Done Done false
32644 8667 Acked Write 5 Sent Done Done false
32645 8668 Acked Read 7 Sent Done Done false
32646 8669 Acked Write 5 Sent Done Done false
32647 8670 Acked Read 4 Sent Done Done false
32648 8671 Acked Read 7 Sent Done Done false
32649 8672 Acked Write 10 Sent Done Done false
32650 8673 Acked Read 4 Sent Done Done false
32651 8674 Acked Read 3 Sent Done Done false
32652 8675 Acked Write 6 Sent Done Done false
32653 8676 Acked Write 5 Sent Done Done false
32654 8677 Acked Read 5 Sent Done Done false
32655 8678 Acked Write 3 Sent Done Done false
32656 8679 Acked Read 5 Sent Done Done false
32657 8680 Acked Read 4 Sent Done Done false
32658 8681 Acked Write 4 Sent Done Done false
32659 8682 Acked Read 1 Sent Done Done false
32660 8683 Acked Read 6 Sent Done Done false
32661 8684 Acked Read 2 Sent Done Done false
32662 8685 Acked Read 9 Sent Done Done false
32663 8686 Acked Write 1 Sent Done Done false
32664 8687 Acked Write 9 Sent Done Done false
32665 8688 Acked Write 2 Sent Done Done false
32666 8689 Acked Read 7 Sent Done Done false
32667 8690 Acked Read 6 Sent Done Done false
32668 8691 Acked Read 10 Sent Done Done false
32669 8692 Acked Read 5 Sent Done Done false
32670 8693 Acked Read 6 Sent Done Done false
32671 8694 Acked Read 9 Sent Done Done false
32672 8695 Acked Read 1 Sent Done Done false
32673 8696 Acked Read 1 Sent Done Done false
32674 8697 Acked Write 8 Sent Done Done false
32675 8698 Acked Read 4 Sent Done Done false
32676 8699 Acked Read 4 Sent Done Done false
32677 8700 Acked Flush 0 Sent Done Done false
32678 8701 Acked Read 10 Sent Done Done false
32679 8702 Acked Write 4 Sent Done Done false
32680 8703 Acked Read 1 Sent Done Done false
32681 8704 Acked Write 9 Sent Done Done false
32682 8705 Acked Write 6 Sent Done Done false
32683 8706 Acked Read 3 Sent Done Done false
32684 8707 Acked Write 6 Sent Done Done false
32685 8708 Acked Read 4 Sent Done Done false
32686 8709 Acked Write 7 Sent Done Done false
32687 8710 Acked Write 8 Sent Done Done false
32688 8711 Acked Flush 0 Sent Done Done false
32689 8712 Acked Write 4 Sent Done Done false
32690 8713 Acked Read 5 Sent Done Done false
32691 8714 Acked Write 2 Sent Done Done false
32692 8715 Acked Read 8 Sent Done Done false
32693 8716 Acked Read 5 Sent Done Done false
32694 8717 Acked Write 10 Sent Done Done false
32695 8718 Acked Write 3 Sent Done Done false
32696 8719 Acked Write 2 Sent Done Done false
32697 8720 Acked Flush 0 Sent Done Done false
32698 8721 Acked Read 7 Sent Done Done false
32699 8722 Acked Read 6 Sent Done Done false
32700 8723 Acked Read 9 Sent Done Done false
32701 8724 Acked Write 2 Sent Done Done false
32702 8725 Acked Write 5 Sent Done Done false
32703 8726 Acked Write 5 Sent Done Done false
32704 8727 Acked Read 9 Sent Done Done false
32705 8728 Acked Read 10 Sent Done Done false
32706 8729 Acked Read 9 Sent Done Done false
32707 8730 Acked Write 6 Sent Done Done false
32708 8731 Acked Write 4 Sent Done Done false
32709 8732 Acked Read 9 Sent Done Done false
32710 8733 Acked Read 3 Sent Done Done false
32711 8734 Acked Read 1 Sent Done Done false
32712 8735 Acked Write 6 Sent Done Done false
32713 8736 Acked Read 8 Sent Done Done false
32714 8737 Acked Write 5 Sent Done Done false
32715 8738 Acked Read 2 Sent Done Done false
32716 8739 Acked Flush 0 Sent Done Done false
32717 8740 Acked Read 2 Sent Done Done false
32718 8741 Acked Read 10 Sent Done Done false
32719 8742 Acked Flush 0 Sent Done Done false
32720 8743 Acked Flush 0 Sent Done Done false
32721 8744 Acked Read 9 Sent Done Done false
32722 8745 Acked Write 4 Sent Done Done false
32723 8746 Acked Read 3 Sent Done Done false
32724 8747 Acked Write 8 Sent Done Done false
32725 8748 Acked Write 5 Sent Done Done false
32726 8749 Acked Read 6 Sent Done Done false
32727 8750 Acked Read 3 Sent Done Done false
32728 8751 Acked Write 8 Sent Done Done false
32729 8752 Acked Write 4 Sent Done Done false
32730 8753 Acked Flush 0 Sent Done Done false
32731 8754 Acked Write 7 Sent Done Done false
32732 8755 Acked Read 1 Sent Done Done false
32733 8756 Acked Write 2 Sent Done Done false
32734 8757 Acked Write 5 Sent Done Done false
32735 8758 Acked Read 9 Sent Done Done false
32736 8759 Acked Read 6 Sent Done Done false
32737 8760 Acked Read 3 Sent Done Done false
32738 8761 Acked Write 9 Sent Done Done false
32739 8762 Acked Write 4 Sent Done Done false
32740 8763 Acked Write 6 Sent Done Done false
32741 8764 Acked Write 10 Sent Done Done false
32742 8765 Acked Read 10 Sent Done Done false
32743 8766 Acked Read 5 Sent Done Done false
32744 8767 Acked Write 9 Sent Done Done false
32745 8768 Acked Read 6 Sent Done Done false
32746 8769 Acked Read 10 Sent Done Done false
32747 8770 Acked Write 6 Sent Done Done false
32748 8771 Acked Write 10 Sent Done Done false
32749 8772 Acked Write 2 Sent Done Done false
32750 8773 Acked Read 8 Sent Done Done false
32751 8774 Acked Write 1 Sent Done Done false
32752 8775 Acked Write 6 Sent Done Done false
32753 8776 Acked Read 5 Sent Done Done false
32754 8777 Acked Write 2 Sent Done Done false
32755 8778 Acked Read 5 Sent Done Done false
32756 8779 Acked Read 5 Sent Done Done false
32757 8780 Acked Read 7 Sent Done Done false
32758 8781 Acked Read 10 Sent Done Done false
32759 8782 Acked Read 5 Sent Done Done false
32760 8783 Acked Write 1 Sent Done Done false
32761 8784 Acked Write 7 Sent Done Done false
32762 8785 Acked Read 4 Sent Done Done false
32763 8786 Acked Flush 0 Sent Done Done false
32764 8787 Acked Flush 0 Sent Done Done false
32765 8788 Acked Read 10 Sent Done Done false
32766 8789 Acked Flush 0 Sent Done Done false
32767 8790 Acked Write 9 Sent Done Done false
32768 8791 Acked Flush 0 Sent Done Done false
32769 8792 Acked Read 9 Sent Done Done false
32770 8793 Acked Write 10 Sent Done Done false
32771 8794 Acked Write 8 Sent Done Done false
32772 8795 Acked Read 8 Sent Done Done false
32773 8796 Acked Write 1 Sent Done Done false
32774 8797 Acked Read 8 Sent Done Done false
32775 8798 Acked Write 10 Sent Done Done false
32776 8799 Acked Read 9 Sent Done Done false
32777 8800 Acked Read 8 Sent Done Done false
32778 8801 Acked Write 2 Sent Done Done false
32779 8802 Acked Write 2 Sent Done Done false
32780 8803 Acked Read 7 Sent Done Done false
32781 8804 Acked Read 5 Sent Done Done false
32782 8805 Acked Write 4 Sent Done Done false
32783 8806 Acked Read 6 Sent Done Done false
32784 8807 Acked Read 9 Sent Done Done false
32785 8808 Acked Flush 0 Sent Done Done false
32786 8809 Acked Read 9 Sent Done Done false
32787 8810 Acked Write 9 Sent Done Done false
32788 8811 Acked Read 1 Sent Done Done false
32789 8812 Acked Read 7 Sent Done Done false
32790 8813 Acked Write 1 Sent Done Done false
32791 8814 Acked Write 7 Sent Done Done false
32792 8815 Acked Write 10 Sent Done Done false
32793 8816 Acked Write 8 Sent Done Done false
32794 8817 Acked Read 6 Sent Done Done false
32795 8818 Acked Read 1 Sent Done Done false
32796 8819 Acked Write 1 Sent Done Done false
32797 8820 Acked Read 10 Sent Done Done false
32798 8821 Acked Read 7 Sent Done Done false
32799 8822 Acked Read 4 Sent Done Done false
32800 8823 Acked Read 6 Sent Done Done false
32801 8824 Acked Read 4 Sent Done Done false
32802 8825 Acked Read 9 Sent Done Done false
32803 8826 Acked Read 9 Sent Done Done false
32804 8827 Acked Flush 0 Sent Done Done false
32805 8828 Acked Write 7 Sent Done Done false
32806 8829 Acked Read 6 Sent Done Done false
32807 8830 Acked Write 9 Sent Done Done false
32808 8831 Acked Flush 0 Sent Done Done false
32809 8832 Acked Read 2 Sent Done Done false
32810 8833 Acked Read 10 Sent Done Done false
32811 8834 Acked Read 3 Sent Done Done false
32812 8835 Acked Flush 0 Sent Done Done false
32813 8836 Acked Read 2 Sent Done Done false
32814 8837 Acked Write 6 Sent Done Done false
32815 8838 Acked Write 4 Sent Done Done false
32816 8839 Acked Write 9 Sent Done Done false
32817 8840 Acked Read 5 Sent Done Done false
32818 8841 Acked Read 7 Sent Done Done false
32819 8842 Acked Read 10 Sent Done Done false
32820 8843 Acked Read 3 Sent Done Done false
32821 8844 Acked Read 9 Sent Done Done false
32822 8845 Acked Read 2 Sent Done Done false
32823 8846 Acked Read 8 Sent Done Done false
32824 8847 Acked Read 9 Sent Done Done false
32825 8848 Acked Write 7 Sent Done Done false
32826 8849 Acked Write 5 Sent Done Done false
32827 8850 Acked Flush 0 Sent Done Done false
32828 STATES DS:0 DS:1 DS:2 TOTAL
32829 Sent 301 0 0 301
32830 Done 0 301 301 602
32831 Skipped 0 0 0 0
32832 Error 0 0 0 0
32833 Last Flush: 8549 8850 8850
32834 Downstairs last five completed: 8549 8548 8547 8546 8545
32835 Upstairs last five completed: 8850 8849 8848 8847 8846
32836 CLIENT: Up:0 ds:301 act:2
328372024-12-11T23:47:22.655ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328382024-12-11T23:47:22.656ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43285) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328392024-12-11T23:47:22.657ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328402024-12-11T23:47:22.657ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328412024-12-11T23:47:22.657ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328422024-12-11T23:47:22.657ZINFOcrucible: send last flush ID to this DS: 8549 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328432024-12-11T23:47:22.658ZINFOcrucible: Replied this last flush ID: 8549 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328442024-12-11T23:47:22.658ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328452024-12-11T23:47:22.658ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328462024-12-11T23:47:22.658ZINFOcrucible: [0] client re-new 301 jobs since flush 8549 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328472024-12-11T23:47:22.658ZINFOcrucible: [0] Replayed 301 jobs since flush: 8549 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
32848 ----------------------------------------------------------------
32849 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32850 Downstairs last five completed: 8850 8849 8848 8847 8846
32851 Upstairs last five completed: 8850 8849 8848 8847 8846
32852 CLIENT: Up:0 ds:0 act:3
32853 Replay: All jobs finished
32854 Main task has work to do, go find it
32855 got dsc Stop(0)
32856 stop 0
32857 Dec 11 23:47:24.906 INFO request completed[8810] Got stop action so:false kr:false
32858 , latency_us: 380, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 150815ef-b5e4-4862-b04c-5448a9ae6b48, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
328592024-12-11T23:47:24.906ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328602024-12-11T23:47:24.906ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328612024-12-11T23:47:24.906ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328622024-12-11T23:47:24.906ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
328632024-12-11T23:47:24.906ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
32864 Dec 11 23:47:24.907{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:47:24.907016235Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
32865 INFO request completed, latency_us: 154, response_code: 200, uri: /state/cid/0, method: GET, req_id: 37077b30-6c17-4eab-949f-1bc7c4b38647, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
32866 [8810] Exited with: ExitStatus(unix_wait_status(9))
32867 [8810][0] reports Exit
32868 Dec 11 23:47:28.909 INFO request completed, latency_us: 221, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8b47a2cd-7c3c-4040-b828-98d90449f3ab, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
32869 001/300 Write block 4039 len 36864 data: 6 7 7 7 8 9 9 9 9
32870 002/300 Flush
32871 003/300 Write block 4823 len 28672 data: 3 3 5 7 8 8 7
32872 004/300 Write block 2374 len 40960 data: 3 5 5 5 5 7 5 6 7 6
32873 005/300 Write block 1296 len 24576 data: 7 7 5 4 3 4
32874 006/300 Read block 3468 len 4096
32875 007/300 Write block 4322 len 4096 data: 3
32876 008/300 Write block 1398 len 4096 data: 5
32877 009/300 Read block 1358 len 20480
32878 010/300 Read block 775 len 24576
32879 011/300 Read block 1302 len 24576
32880 012/300 Read block 3566 len 20480
32881 013/300 Write block 3274 len 4096 data: 5
32882 014/300 Write block 4546 len 4096 data: 4
32883 015/300 Write block 1251 len 4096 data: 6
32884 016/300 Read block 399 len 12288
32885 017/300 Read block 4827 len 40960
32886 018/300 Write block 1559 len 24576 data: 3 4 4 5 6 8
32887 019/300 Write block 2516 len 12288 data: 6 6 5
32888 020/300 Read block 2836 len 12288
32889 021/300 Write block 3074 len 4096 data: 4
32890 022/300 Read block 2443 len 16384
32891 023/300 Read block 164 len 40960
32892 024/300 Read block 3378 len 36864
32893 025/300 Write block 2267 len 20480 data: 7 7 7 6 8
32894 026/300 Read block 116 len 4096
32895 027/300 Read block 4671 len 40960
32896 028/300 Read block 1946 len 12288
32897 029/300 Read block 1114 len 4096
32898 030/300 Read block 4366 len 24576
32899 031/300 Write block 349 len 28672 data: 2 2 2 2 3 4 5
32900 032/300 Flush
32901 033/300 Write block 2390 len 16384 data: 3 3 3 3
32902 034/300 Write block 37 len 24576 data: 3 3 3 3 3 3
32903 035/300 Read block 3839 len 28672
32904 036/300 Write block 2694 len 32768 data: 4 5 4 4 4 4 4 5
32905 037/300 Flush
32906 038/300 Write block 701 len 8192 data: 2 3
32907 039/300 Read block 2461 len 28672
32908 040/300 Read block 2900 len 36864
32909 041/300 Read block 4441 len 8192
32910 042/300 Read block 4346 len 36864
32911 043/300 Read block 2383 len 8192
32912 044/300 Read block 2885 len 16384
32913 045/300 Read block 1044 len 12288
32914 046/300 Write block 170 len 12288 data: 3 3 3
32915 047/300 Read block 4098 len 32768
32916 048/300 Read block 2781 len 4096
32917 049/300 Write block 2515 len 36864 data: 8 7 7 6 4 4 4 4 4
32918 050/300 Write block 740 len 40960 data: 5 4 4 5 5 3 3 4 5 6
32919 051/300 Read block 4818 len 32768
32920 052/300 Read block 1401 len 28672
32921 053/300 Write block 578 len 12288 data: 10 9 10
32922 054/300 Write block 602 len 4096 data: 3
32923 055/300 Read block 3098 len 32768
32924 056/300 Read block 4177 len 28672
32925 057/300 Flush
32926 058/300 Read block 710 len 16384
32927 059/300 Flush
32928 060/300 Write block 4247 len 20480 data: 4 5 5 5 5
32929 061/300 Flush
32930 062/300 Write block 1903 len 36864 data: 5 5 5 6 6 7 5 6 6
32931 063/300 Read block 3940 len 20480
32932 064/300 Flush
32933 065/300 Write block 891 len 12288 data: 6 5 5
32934 066/300 Write block 1100 len 12288 data: 9 9 6
32935 067/300 Read block 1209 len 8192
32936 068/300 Flush
32937 069/300 Write block 1747 len 4096 data: 6
32938 070/300 Read block 1633 len 24576
32939 071/300 Read block 954 len 40960
32940 072/300 Read block 1632 len 16384
32941 073/300 Write block 3387 len 40960 data: 7 6 5 4 5 4 4 3 4 4
32942 074/300 Flush
32943 075/300 Write block 3388 len 4096 data: 7
32944 076/300 Write block 1079 len 16384 data: 3 3 4 5
32945 077/300 Read block 1702 len 32768
32946 078/300 Write block 3837 len 28672 data: 5 5 4 4 5 4 4
32947 079/300 Read block 1126 len 28672
32948 080/300 Write block 3852 len 32768 data: 6 7 7 6 8 9 10 9
32949 081/300 Read block 354 len 32768
32950 082/300 Read block 2988 len 4096
32951 083/300 Write block 1828 len 36864 data: 4 3 3 4 5 5 7 7 9
32952 084/300 Read block 279 len 24576
32953 085/300 Read block 2865 len 12288
32954 086/300 Flush
32955 087/300 Write block 859 len 40960 data: 6 6 6 6 5 4 3 4 4 5
32956 088/300 Read block 1027 len 20480
32957 089/300 Write block 2493 len 8192 data: 4 4
32958 090/300 Write block 4415 len 8192 data: 2 2
32959 091/300 Write block 735 len 36864 data: 2 3 4 5 5 6 5 5 6
32960 092/300 Read block 1362 len 12288
32961 093/300 Read block 1130 len 32768
32962 094/300 Write block 290 len 4096 data: 7
32963 095/300 Write block 2928 len 16384 data: 5 5 5 8
32964 096/300 Write block 4069 len 24576 data: 6 6 6 7 5 6
32965 097/300 Read block 1893 len 28672
32966 098/300 Write block 842 len 36864 data: 3 3 4 4 4 5 4 4 4
32967 099/300 Write block 3074 len 24576 data: 5 3 2 3 4 3
32968 100/300 Flush
32969 101/300 Read block 3864 len 4096
32970 102/300 Write block 3059 len 16384 data: 6 6 4 3
32971 103/300 Write block 3021 len 12288 data: 5 5 6
32972 104/300 Write block 1537 len 40960 data: 4 4 5 6 8 7 8 10 9 9
32973 105/300 Write block 2775 len 24576 data: 7 6 5 4 4 2
32974 106/300 Write block 4193 len 28672 data: 3 2 2 3 3 4 5
32975 107/300 Write block 557 len 40960 data: 4 4 4 4 3 3 4 4 3 3
32976 108/300 Flush
32977 109/300 Read block 2313 len 40960
32978 110/300 Write block 3111 len 16384 data: 4 4 4 5
32979 111/300 Read block 910 len 36864
32980 112/300 Read block 2269 len 32768
32981 113/300 Flush
32982 114/300 Read block 2446 len 32768
32983 115/300 Write block 2125 len 4096 data: 6
32984 116/300 Read block 2112 len 16384
32985 117/300 Flush
32986 118/300 Read block 240 len 20480
32987 119/300 Read block 24 len 16384
32988 120/300 Write block 1676 len 32768 data: 2 3 4 4 5 5 6 8
32989 121/300 Read block 4969 len 24576
32990 122/300 Read block 3081 len 24576
32991 123/300 Read block 4042 len 4096
32992 124/300 Write block 2179 len 20480 data: 4 4 4 3 2
32993 125/300 Read block 3534 len 40960
32994 126/300 Write block 1301 len 20480 data: 5 4 4 4 5
32995 127/300 Read block 4706 len 24576
32996 128/300 Write block 2279 len 4096 data: 4
32997 129/300 Read block 3279 len 32768
32998 130/300 Read block 2734 len 16384
32999 131/300 Flush
33000 132/300 Write block 4315 len 32768 data: 5 5 4 4 4 4 3 4
33001 133/300 Read block 6 len 36864
33002 134/300 Write block 4089 len 12288 data: 8 9 9
33003 135/300 Write block 3983 len 16384 data: 5 5 5 6
33004 136/300 Write block 4277 len 28672 data: 4 4 4 4 6 5 7
33005 137/300 Read block 2286 len 40960
33006 138/300 Flush
33007 139/300 Read block 1253 len 8192
33008 140/300 Write block 4309 len 20480 data: 3 4 4 4 5
33009 141/300 Write block 4263 len 16384 data: 5 6 5 5
33010 142/300 Write block 2711 len 16384 data: 10 10 9 12
33011 143/300 Write block 3991 len 32768 data: 7 8 7 6 6 6 4 6
33012 144/300 Write block 4221 len 40960 data: 5 6 6 5 5 5 5 5 4 3
33013 145/300 Read block 3372 len 36864
33014 146/300 Read block 4721 len 12288
33015 147/300 Write block 3061 len 40960 data: 5 4 5 5 5 5 6 6 6 7
33016 148/300 Flush
33017 149/300 Read block 4301 len 4096
33018 150/300 Write block 357 len 20480 data: 10 8 8 8 6
33019 151/300 Read block 4122 len 28672
33020 152/300 Read block 2587 len 20480
33021 153/300 Read block 625 len 28672
33022 154/300 Flush
33023 155/300 Read block 3851 len 36864
33024 156/300 Read block 317 len 8192
33025 157/300 Read block 4646 len 40960
33026 158/300 Read block 2610 len 8192
33027 159/300 Write block 3814 len 40960 data: 9 7 8 8 8 6 6 4 4 4
33028 160/300 Write block 338 len 32768 data: 4 5 6 8 9 10 11 11
33029 161/300 Flush
33030 162/300 Read block 1836 len 12288
33031 163/300 Write block 3026 len 8192 data: 5 4
33032 164/300 Write block 1293 len 28672 data: 9 9 9 8 8 6 5
33033 165/300 Read block 4926 len 16384
33034 166/300 Write block 495 len 24576 data: 6 6 7 6 7 6
33035 167/300 Read block 2896 len 40960
33036 168/300 Write block 1084 len 20480 data: 6 8 8 8 8
33037 169/300 Flush
33038 170/300 Read block 2336 len 36864
33039 171/300 Flush
33040 172/300 Read block 3517 len 24576
33041 173/300 Write block 1165 len 24576 data: 5 5 5 6 6 5
33042 174/300 Write block 959 len 28672 data: 3 3 3 4 4 5 7
33043 175/300 Read block 3542 len 16384
33044 176/300 Read block 433 len 16384
33045 177/300 Read block 3560 len 28672
33046 178/300 Read block 4180 len 12288
33047 179/300 Read block 545 len 40960
33048 180/300 Flush
33049 181/300 Read block 1803 len 20480
33050 182/300 Read block 3988 len 12288
33051 183/300 Write block 41 len 12288 data: 4 4 4
33052 184/300 Write block 1942 len 40960 data: 5 5 6 5 5 4 5 7 6 6
33053 185/300 Flush
33054 186/300 Read block 361 len 12288
33055 187/300 Read block 4365 len 40960
33056 188/300 Read block 467 len 40960
33057 189/300 Flush
33058 190/300 Read block 4383 len 24576
33059 191/300 Read block 3727 len 16384
33060 192/300 Read block 4444 len 12288
33061 193/300 Write block 3695 len 8192 data: 7 8
33062 194/300 Write block 1413 len 32768 data: 6 6 4 4 5 4 2 2
33063 195/300 Read block 4888 len 20480
33064 196/300 Write block 2761 len 20480 data: 2 3 4 5 6
33065 197/300 Write block 4083 len 28672 data: 2 5 8 8 9 9 9
33066 198/300 Read block 3307 len 24576
33067 199/300 Write block 4904 len 8192 data: 10 11
33068 200/300 Write block 1203 len 20480 data: 6 8 8 7 6
33069 201/300 Write block 2367 len 28672 data: 3 3 3 4 3 3 3
33070 202/300 Flush
33071 203/300 Write block 1853 len 40960 data: 4 6 7 5 5 4 5 6 6 6
33072 204/300 Read block 1383 len 28672
33073 205/300 Write block 887 len 16384 data: 9 7 6 7
33074 206/300 Write block 1959 len 8192 data: 3 3
33075 207/300 Read block 4781 len 12288
33076 208/300 Write block 3033 len 36864 data: 6 7 7 6 4 3 2 2 3
33077 209/300 Write block 356 len 16384 data: 6 11 9 9
33078 210/300 Read block 3223 len 16384
33079 211/300 Write block 171 len 36864 data: 4 4 3 5 5 4 4 7 7
33080 212/300 Read block 2258 len 40960
33081 213/300 Write block 2343 len 36864 data: 5 5 5 9 9 8 8 8 5
33082 214/300 Write block 149 len 12288 data: 6 5 5
33083 215/300 Read block 4538 len 4096
33084 216/300 Read block 2581 len 20480
33085 217/300 Read block 4791 len 24576
33086 218/300 Write block 964 len 12288 data: 6 8 6
33087 219/300 Read block 1171 len 8192
33088 220/300 Write block 1854 len 4096 data: 7
33089 221/300 Write block 3936 len 32768 data: 7 7 7 7 6 7 7 8
33090 222/300 Write block 329 len 36864 data: 9 7 8 8 6 7 5 5 5
33091 223/300 Write block 1644 len 24576 data: 7 7 9 7 7 7
33092 224/300 Read block 528 len 32768
33093 225/300 Write block 3005 len 16384 data: 5 5 7 6
33094 226/300 Write block 4365 len 36864 data: 5 4 4 3 5 6 6 6 6
33095 227/300 Read block 2013 len 20480
33096 228/300 Read block 3407 len 8192
33097 229/300 Flush
33098 230/300 Write block 3273 len 32768 data: 4 6 6 6 7 6 8 8
33099 231/300 Read block 1537 len 20480
33100 232/300 Read block 2680 len 32768
33101 233/300 Read block 1421 len 28672
33102 234/300 Flush
33103 235/300 Write block 1354 len 8192 data: 7 6
33104 236/300 Write block 254 len 36864 data: 5 5 3 2 2 2 2 4 4
33105 237/300 Read block 1732 len 16384
33106 238/300 Read block 441 len 28672
33107 239/300 Write block 3545 len 40960 data: 7 6 6 6 7 7 6 6 6 6
33108 240/300 Flush
33109 241/300 Write block 3564 len 4096 data: 4
33110 242/300 Write block 4593 len 8192 data: 6 7
33111 243/300 Flush
33112 244/300 Read block 3105 len 36864
33113 245/300 Read block 4644 len 16384
33114 246/300 Write block 4189 len 12288 data: 3 3 3
33115 247/300 Read block 998 len 8192
33116 248/300 Write block 2887 len 32768 data: 8 8 9 9 8 8 8 9
33117 249/300 Write block 2290 len 12288 data: 4 5 6
33118 250/300 Flush
33119 251/300 Write block 2980 len 24576 data: 5 7 6 5 4 5
33120 252/300 Flush
33121 253/300 Read block 2898 len 8192
33122 254/300 Flush
33123 255/300 Read block 1001 len 32768
33124 256/300 Read block 101 len 20480
33125 257/300 Read block 4163 len 8192
33126 258/300 Flush
33127 259/300 Read block 1802 len 8192
33128 260/300 Write block 1614 len 12288 data: 3 2 3
33129 261/300 Read block 562 len 32768
33130 262/300 Write block 2830 len 4096 data: 8
33131 263/300 Read block 4870 len 4096
33132 264/300 Flush
33133 265/300 Write block 4867 len 4096 data: 4
33134 266/300 Read block 477 len 8192
33135 267/300 Write block 3512 len 32768 data: 9 8 9 9 6 7 5 3
33136 268/300 Read block 2437 len 32768
33137 269/300 Read block 3073 len 4096
33138 270/300 Read block 513 len 36864
33139 271/300 Read block 1236 len 4096
33140 272/300 Read block 544 len 28672
33141 273/300 Write block 2413 len 8192 data: 3 3
33142 274/300 Write block 1545 len 4096 data: 10
33143 275/300 Read block 572 len 40960
33144 276/300 Read block 2689 len 32768
33145 277/300 Write block 3130 len 20480 data: 7 7 7 7 6
33146 278/300 Read block 2212 len 12288
33147 279/300 Write block 1172 len 16384 data: 5 5 3 4
33148 280/300 Write block 4884 len 24576 data: 4 3 4 5 5 5
33149 281/300 Read block 2941 len 16384
33150 282/300 Read block 506 len 32768
33151 283/300 Read block 479 len 8192
33152 284/300 Read block 1160 len 36864
33153 285/300 Read block 1953 len 24576
33154 286/300 Read block 172 len 20480
33155 287/300 Write block 1478 len 28672 data: 2 2 2 5 5 7 7
33156 288/300 Write block 2278 len 16384 data: 3 5 7 5
33157 289/300 Read block 1835 len 12288
33158 290/300 Read block 1474 len 4096
33159 291/300 Read block 2726 len 16384
33160 292/300 Read block 2955 len 8192
33161 293/300 Write block 1313 len 4096 data: 9
33162 294/300 Read block 1178 len 36864
33163 295/300 Read block 4698 len 32768
33164 296/300 Read block 1323 len 32768
33165 297/300 Read block 4339 len 28672
33166 298/300 Write block 3878 len 32768 data: 7 6 8 7 7 7 7 9
33167 299/300 Write block 1143 len 12288 data: 4 3 6
33168 300/300 Read block 19 len 40960
33169 301/300 Write block 4247 len 24576 data: 5 6 6 6 6 5
33170 Main task has work to do, go find it
33171 got dsc Start(0)
33172 start 0
33173 Dec 11 23:47:29.145 INFO request completed, latency_us: 245, response_code: 204, uri: /start/cid/0, method: POST, req_id: cbceb8d7-973f-471e-9e90-361f02bded99, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
33174 Replay: started 0, returned:Ok(())
33175 ----------------------------------------------------------------
33176 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33177 States: Offline Active Active
33178 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33179 8851 Acked Write 9 Sent Done Done false
33180 8852 Acked Flush 0 Sent Done Done false
33181 8853 Acked Write 7 Sent Done Done false
33182 8854 Acked Write 10 Sent Done Done false
33183 8855 Acked Write 6 Sent Done Done false
33184 8856 Acked Read 1 Sent Done Done false
33185 8857 Acked Write 1 Sent Done Done false
33186 8858 Acked Write 1 Sent Done Done false
33187 8859 Acked Read 5 Sent Done Done false
33188 8860 Acked Read 6 Sent Done Done false
33189 8861 Acked Read 6 Sent Done Done false
33190 8862 Acked Read 5 Sent Done Done false
33191 8863 Acked Write 1 Sent Done Done false
33192 8864 Acked Write 1 Sent Done Done false
33193 8865 Acked Write 1 Sent Done Done false
33194 8866 Acked Read 3 Sent Done Done false
33195 8867 Acked Read 10 Sent Done Done false
33196 8868 Acked Write 6 Sent Done Done false
33197 8869 Acked Write 3 Sent Done Done false
33198 8870 Acked Read 3 Sent Done Done false
33199 8871 Acked Write 1 Sent Done Done false
33200 8872 Acked Read 4 Sent Done Done false
33201 8873 Acked Read 10 Sent Done Done false
33202 8874 Acked Read 9 Sent Done Done false
33203 8875 Acked Write 5 Sent Done Done false
33204 8876 Acked Read 1 Sent Done Done false
33205 8877 Acked Read 10 Sent Done Done false
33206 8878 Acked Read 3 Sent Done Done false
33207 8879 Acked Read 1 Sent Done Done false
33208 8880 Acked Read 6 Sent Done Done false
33209 8881 Acked Write 7 Sent Done Done false
33210 8882 Acked Flush 0 Sent Done Done false
33211 8883 Acked Write 4 Sent Done Done false
33212 8884 Acked Write 6 Sent Done Done false
33213 8885 Acked Read 7 Sent Done Done false
33214 8886 Acked Write 8 Sent Done Done false
33215 8887 Acked Flush 0 Sent Done Done false
33216 8888 Acked Write 2 Sent Done Done false
33217 8889 Acked Read 7 Sent Done Done false
33218 8890 Acked Read 9 Sent Done Done false
33219 8891 Acked Read 2 Sent Done Done false
33220 8892 Acked Read 9 Sent Done Done false
33221 8893 Acked Read 2 Sent Done Done false
33222 8894 Acked Read 4 Sent Done Done false
33223 8895 Acked Read 3 Sent Done Done false
33224 8896 Acked Write 3 Sent Done Done false
33225 8897 Acked Read 8 Sent Done Done false
33226 8898 Acked Read 1 Sent Done Done false
33227 8899 Acked Write 9 Sent Done Done false
33228 8900 Acked Write 10 Sent Done Done false
33229 8901 Acked Read 8 Sent Done Done false
33230 8902 Acked Read 7 Sent Done Done false
33231 8903 Acked Write 3 Sent Done Done false
33232 8904 Acked Write 1 Sent Done Done false
33233 8905 Acked Read 8 Sent Done Done false
33234 8906 Acked Read 7 Sent Done Done false
33235 8907 Acked Flush 0 Sent Done Done false
33236 8908 Acked Read 4 Sent Done Done false
33237 8909 Acked Flush 0 Sent Done Done false
33238 8910 Acked Write 5 Sent Done Done false
33239 8911 Acked Flush 0 Sent Done Done false
33240 8912 Acked Write 9 Sent Done Done false
33241 8913 Acked Read 5 Sent Done Done false
33242 8914 Acked Flush 0 Sent Done Done false
33243 8915 Acked Write 3 Sent Done Done false
33244 8916 Acked Write 3 Sent Done Done false
33245 8917 Acked Read 2 Sent Done Done false
33246 8918 Acked Flush 0 Sent Done Done false
33247 8919 Acked Write 1 Sent Done Done false
33248 8920 Acked Read 6 Sent Done Done false
33249 8921 Acked Read 10 Sent Done Done false
33250 8922 Acked Read 4 Sent Done Done false
33251 8923 Acked Write 10 Sent Done Done false
33252 8924 Acked Flush 0 Sent Done Done false
33253 8925 Acked Write 1 Sent Done Done false
33254 8926 Acked Write 4 Sent Done Done false
33255 8927 Acked Read 8 Sent Done Done false
33256 8928 Acked Write 7 Sent Done Done false
33257 8929 Acked Read 7 Sent Done Done false
33258 8930 Acked Write 8 Sent Done Done false
33259 8931 Acked Read 8 Sent Done Done false
33260 8932 Acked Read 1 Sent Done Done false
33261 8933 Acked Write 9 Sent Done Done false
33262 8934 Acked Read 6 Sent Done Done false
33263 8935 Acked Read 3 Sent Done Done false
33264 8936 Acked Flush 0 Sent Done Done false
33265 8937 Acked Write 10 Sent Done Done false
33266 8938 Acked Read 5 Sent Done Done false
33267 8939 Acked Write 2 Sent Done Done false
33268 8940 Acked Write 2 Sent Done Done false
33269 8941 Acked Write 9 Sent Done Done false
33270 8942 Acked Read 3 Sent Done Done false
33271 8943 Acked Read 8 Sent Done Done false
33272 8944 Acked Write 1 Sent Done Done false
33273 8945 Acked Write 4 Sent Done Done false
33274 8946 Acked Write 6 Sent Done Done false
33275 8947 Acked Read 7 Sent Done Done false
33276 8948 Acked Write 9 Sent Done Done false
33277 8949 Acked Write 6 Sent Done Done false
33278 8950 Acked Flush 0 Sent Done Done false
33279 8951 Acked Read 1 Sent Done Done false
33280 8952 Acked Write 4 Sent Done Done false
33281 8953 Acked Write 3 Sent Done Done false
33282 8954 Acked Write 10 Sent Done Done false
33283 8955 Acked Write 6 Sent Done Done false
33284 8956 Acked Write 7 Sent Done Done false
33285 8957 Acked Write 10 Sent Done Done false
33286 8958 Acked Flush 0 Sent Done Done false
33287 8959 Acked Read 10 Sent Done Done false
33288 8960 Acked Write 4 Sent Done Done false
33289 8961 Acked Read 9 Sent Done Done false
33290 8962 Acked Read 8 Sent Done Done false
33291 8963 Acked Flush 0 Sent Done Done false
33292 8964 Acked Read 8 Sent Done Done false
33293 8965 Acked Write 1 Sent Done Done false
33294 8966 Acked Read 4 Sent Done Done false
33295 8967 Acked Flush 0 Sent Done Done false
33296 8968 Acked Read 5 Sent Done Done false
33297 8969 Acked Read 4 Sent Done Done false
33298 8970 Acked Write 8 Sent Done Done false
33299 8971 Acked Read 6 Sent Done Done false
33300 8972 Acked Read 6 Sent Done Done false
33301 8973 Acked Read 1 Sent Done Done false
33302 8974 Acked Write 5 Sent Done Done false
33303 8975 Acked Read 10 Sent Done Done false
33304 8976 Acked Write 5 Sent Done Done false
33305 8977 Acked Read 6 Sent Done Done false
33306 8978 Acked Write 1 Sent Done Done false
33307 8979 Acked Read 8 Sent Done Done false
33308 8980 Acked Read 4 Sent Done Done false
33309 8981 Acked Flush 0 Sent Done Done false
33310 8982 Acked Write 8 Sent Done Done false
33311 8983 Acked Read 9 Sent Done Done false
33312 8984 Acked Write 3 Sent Done Done false
33313 8985 Acked Write 4 Sent Done Done false
33314 8986 Acked Write 7 Sent Done Done false
33315 8987 Acked Read 10 Sent Done Done false
33316 8988 Acked Flush 0 Sent Done Done false
33317 8989 Acked Read 2 Sent Done Done false
33318 8990 Acked Write 5 Sent Done Done false
33319 8991 Acked Write 4 Sent Done Done false
33320 8992 Acked Write 4 Sent Done Done false
33321 8993 Acked Write 8 Sent Done Done false
33322 8994 Acked Write 10 Sent Done Done false
33323 8995 Acked Read 9 Sent Done Done false
33324 8996 Acked Read 3 Sent Done Done false
33325 8997 Acked Write 10 Sent Done Done false
33326 8998 Acked Flush 0 Sent Done Done false
33327 8999 Acked Read 1 Sent Done Done false
33328 9000 Acked Write 5 Sent Done Done false
33329 9001 Acked Read 7 Sent Done Done false
33330 9002 Acked Read 5 Sent Done Done false
33331 9003 Acked Read 7 Sent Done Done false
33332 9004 Acked Flush 0 Sent Done Done false
33333 9005 Acked Read 9 Sent Done Done false
33334 9006 Acked Read 2 Sent Done Done false
33335 9007 Acked Read 10 Sent Done Done false
33336 9008 Acked Read 2 Sent Done Done false
33337 9009 Acked Write 10 Sent Done Done false
33338 9010 Acked Write 8 Sent Done Done false
33339 9011 Acked Flush 0 Sent Done Done false
33340 9012 Acked Read 3 Sent Done Done false
33341 9013 Acked Write 2 Sent Done Done false
33342 9014 Acked Write 7 Sent Done Done false
33343 9015 Acked Read 4 Sent Done Done false
33344 9016 Acked Write 6 Sent Done Done false
33345 9017 Acked Read 10 Sent Done Done false
33346 9018 Acked Write 5 Sent Done Done false
33347 9019 Acked Flush 0 Sent Done Done false
33348 9020 Acked Read 9 Sent Done Done false
33349 9021 Acked Flush 0 Sent Done Done false
33350 9022 Acked Read 6 Sent Done Done false
33351 9023 Acked Write 6 Sent Done Done false
33352 9024 Acked Write 7 Sent Done Done false
33353 9025 Acked Read 4 Sent Done Done false
33354 9026 Acked Read 4 Sent Done Done false
33355 9027 Acked Read 7 Sent Done Done false
33356 9028 Acked Read 3 Sent Done Done false
33357 9029 Acked Read 10 Sent Done Done false
33358 9030 Acked Flush 0 Sent Done Done false
33359 9031 Acked Read 5 Sent Done Done false
33360 9032 Acked Read 3 Sent Done Done false
33361 9033 Acked Write 3 Sent Done Done false
33362 9034 Acked Write 10 Sent Done Done false
33363 9035 Acked Flush 0 Sent Done Done false
33364 9036 Acked Read 3 Sent Done Done false
33365 9037 Acked Read 10 Sent Done Done false
33366 9038 Acked Read 10 Sent Done Done false
33367 9039 Acked Flush 0 Sent Done Done false
33368 9040 Acked Read 6 Sent Done Done false
33369 9041 Acked Read 4 Sent Done Done false
33370 9042 Acked Read 3 Sent Done Done false
33371 9043 Acked Write 2 Sent Done Done false
33372 9044 Acked Write 8 Sent Done Done false
33373 9045 Acked Read 5 Sent Done Done false
33374 9046 Acked Write 5 Sent Done Done false
33375 9047 Acked Write 7 Sent Done Done false
33376 9048 Acked Read 6 Sent Done Done false
33377 9049 Acked Write 2 Sent Done Done false
33378 9050 Acked Write 5 Sent Done Done false
33379 9051 Acked Write 7 Sent Done Done false
33380 9052 Acked Flush 0 Sent Done Done false
33381 9053 Acked Write 10 Sent Done Done false
33382 9054 Acked Read 7 Sent Done Done false
33383 9055 Acked Write 4 Sent Done Done false
33384 9056 Acked Write 2 Sent Done Done false
33385 9057 Acked Read 3 Sent Done Done false
33386 9058 Acked Write 9 Sent Done Done false
33387 9059 Acked Write 4 Sent Done Done false
33388 9060 Acked Read 4 Sent Done Done false
33389 9061 Acked Write 9 Sent Done Done false
33390 9062 Acked Read 10 Sent Done Done false
33391 9063 Acked Write 9 Sent Done Done false
33392 9064 Acked Write 3 Sent Done Done false
33393 9065 Acked Read 1 Sent Done Done false
33394 9066 Acked Read 5 Sent Done Done false
33395 9067 Acked Read 6 Sent Done Done false
33396 9068 Acked Write 3 Sent Done Done false
33397 9069 Acked Read 2 Sent Done Done false
33398 9070 Acked Write 1 Sent Done Done false
33399 9071 Acked Write 8 Sent Done Done false
33400 9072 Acked Write 9 Sent Done Done false
33401 9073 Acked Write 6 Sent Done Done false
33402 9074 Acked Read 8 Sent Done Done false
33403 9075 Acked Write 4 Sent Done Done false
33404 9076 Acked Write 9 Sent Done Done false
33405 9077 Acked Read 5 Sent Done Done false
33406 9078 Acked Read 2 Sent Done Done false
33407 9079 Acked Flush 0 Sent Done Done false
33408 9080 Acked Write 8 Sent Done Done false
33409 9081 Acked Read 5 Sent Done Done false
33410 9082 Acked Read 8 Sent Done Done false
33411 9083 Acked Read 7 Sent Done Done false
33412 9084 Acked Flush 0 Sent Done Done false
33413 9085 Acked Write 2 Sent Done Done false
33414 9086 Acked Write 9 Sent Done Done false
33415 9087 Acked Read 4 Sent Done Done false
33416 9088 Acked Read 7 Sent Done Done false
33417 9089 Acked Write 10 Sent Done Done false
33418 9090 Acked Flush 0 Sent Done Done false
33419 9091 Acked Write 1 Sent Done Done false
33420 9092 Acked Write 2 Sent Done Done false
33421 9093 Acked Flush 0 Sent Done Done false
33422 9094 Acked Read 9 Sent Done Done false
33423 9095 Acked Read 4 Sent Done Done false
33424 9096 Acked Write 3 Sent Done Done false
33425 9097 Acked Read 2 Sent Done Done false
33426 9098 Acked Write 8 Sent Done Done false
33427 9099 Acked Write 3 Sent Done Done false
33428 9100 Acked Flush 0 Sent Done Done false
33429 9101 Acked Write 6 Sent Done Done false
33430 9102 Acked Flush 0 Sent Done Done false
33431 9103 Acked Read 2 Sent Done Done false
33432 9104 Acked Flush 0 Sent Done Done false
33433 9105 Acked Read 8 Sent Done Done false
33434 9106 Acked Read 5 Sent Done Done false
33435 9107 Acked Read 2 Sent Done Done false
33436 9108 Acked Flush 0 Sent Done Done false
33437 9109 Acked Read 2 Sent Done Done false
33438 9110 Acked Write 3 Sent Done Done false
33439 9111 Acked Read 8 Sent Done Done false
33440 9112 Acked Write 1 Sent Done Done false
33441 9113 Acked Read 1 Sent Done Done false
33442 9114 Acked Flush 0 Sent Done Done false
33443 9115 Acked Write 1 Sent Done Done false
33444 9116 Acked Read 2 Sent Done Done false
33445 9117 Acked Write 8 Sent Done Done false
33446 9118 Acked Read 8 Sent Done Done false
33447 9119 Acked Read 1 Sent Done Done false
33448 9120 Acked Read 9 Sent Done Done false
33449 9121 Acked Read 1 Sent Done Done false
33450 9122 Acked Read 7 Sent Done Done false
33451 9123 Acked Write 2 Sent Done Done false
33452 9124 Acked Write 1 Sent Done Done false
33453 9125 Acked Read 10 Sent Done Done false
33454 9126 Acked Read 8 Sent Done Done false
33455 9127 Acked Write 5 Sent Done Done false
33456 9128 Acked Read 3 Sent Done Done false
33457 9129 Acked Write 4 Sent Done Done false
33458 9130 Acked Write 6 Sent Done Done false
33459 9131 Acked Read 4 Sent Done Done false
33460 9132 Acked Read 8 Sent Done Done false
33461 9133 Acked Read 2 Sent Done Done false
33462 9134 Acked Read 9 Sent Done Done false
33463 9135 Acked Read 6 Sent Done Done false
33464 9136 Acked Read 5 Sent Done Done false
33465 9137 Acked Write 7 Sent Done Done false
33466 9138 Acked Write 4 Sent Done Done false
33467 9139 Acked Read 3 Sent Done Done false
33468 9140 Acked Read 1 Sent Done Done false
33469 9141 Acked Read 4 Sent Done Done false
33470 9142 Acked Read 2 Sent Done Done false
33471 9143 Acked Write 1 Sent Done Done false
33472 9144 Acked Read 9 Sent Done Done false
33473 9145 Acked Read 8 Sent Done Done false
33474 9146 Acked Read 8 Sent Done Done false
33475 9147 Acked Read 7 Sent Done Done false
33476 9148 Acked Write 8 Sent Done Done false
33477 9149 Acked Write 3 Sent Done Done false
33478 9150 Acked Read 10 Sent Done Done false
33479 9151 Acked Write 6 Sent Done Done false
33480 STATES DS:0 DS:1 DS:2 TOTAL
33481 Sent 301 0 0 301
33482 Done 0 301 301 602
33483 Skipped 0 0 0 0
33484 Error 0 0 0 0
33485 Last Flush: 8850 9114 9114
33486 Downstairs last five completed: 8850 8849 8848 8847 8846
33487 Upstairs last five completed: 9151 9150 9149 9148 9147
33488 CLIENT: Up:0 ds:301 act:2
33489 [8810] got start action
33490 [8810] I am going to restart
33491 Starting downstairs at port 8810
33492 Make output file at "/tmp/dsc/downstairs-8810.txt"
33493 [8810][0] reports Starting
33494 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1227)
33495 [8810][0] reports Running
33496 ----------------------------------------------------------------
33497 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33498 States: Offline Active Active
33499 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33500 8851 Acked Write 9 Sent Done Done false
33501 8852 Acked Flush 0 Sent Done Done false
33502 8853 Acked Write 7 Sent Done Done false
33503 8854 Acked Write 10 Sent Done Done false
33504 8855 Acked Write 6 Sent Done Done false
33505 8856 Acked Read 1 Sent Done Done false
33506 8857 Acked Write 1 Sent Done Done false
33507 8858 Acked Write 1 Sent Done Done false
33508 8859 Acked Read 5 Sent Done Done false
33509 8860 Acked Read 6 Sent Done Done false
33510 8861 Acked Read 6 Sent Done Done false
33511 8862 Acked Read 5 Sent Done Done false
33512 8863 Acked Write 1 Sent Done Done false
33513 8864 Acked Write 1 Sent Done Done false
33514 8865 Acked Write 1 Sent Done Done false
33515 8866 Acked Read 3 Sent Done Done false
33516 8867 Acked Read 10 Sent Done Done false
33517 8868 Acked Write 6 Sent Done Done false
33518 8869 Acked Write 3 Sent Done Done false
33519 8870 Acked Read 3 Sent Done Done false
33520 8871 Acked Write 1 Sent Done Done false
33521 8872 Acked Read 4 Sent Done Done false
33522 8873 Acked Read 10 Sent Done Done false
33523 8874 Acked Read 9 Sent Done Done false
33524 8875 Acked Write 5 Sent Done Done false
33525 8876 Acked Read 1 Sent Done Done false
33526 8877 Acked Read 10 Sent Done Done false
33527 8878 Acked Read 3 Sent Done Done false
33528 8879 Acked Read 1 Sent Done Done false
33529 8880 Acked Read 6 Sent Done Done false
33530 8881 Acked Write 7 Sent Done Done false
33531 8882 Acked Flush 0 Sent Done Done false
33532 8883 Acked Write 4 Sent Done Done false
33533 8884 Acked Write 6 Sent Done Done false
33534 8885 Acked Read 7 Sent Done Done false
33535 8886 Acked Write 8 Sent Done Done false
33536 8887 Acked Flush 0 Sent Done Done false
33537 8888 Acked Write 2 Sent Done Done false
33538 8889 Acked Read 7 Sent Done Done false
33539 8890 Acked Read 9 Sent Done Done false
33540 8891 Acked Read 2 Sent Done Done false
33541 8892 Acked Read 9 Sent Done Done false
33542 8893 Acked Read 2 Sent Done Done false
33543 8894 Acked Read 4 Sent Done Done false
33544 8895 Acked Read 3 Sent Done Done false
33545 8896 Acked Write 3 Sent Done Done false
33546 8897 Acked Read 8 Sent Done Done false
33547 8898 Acked Read 1 Sent Done Done false
33548 8899 Acked Write 9 Sent Done Done false
33549 8900 Acked Write 10 Sent Done Done false
33550 8901 Acked Read 8 Sent Done Done false
33551 8902 Acked Read 7 Sent Done Done false
33552 8903 Acked Write 3 Sent Done Done false
33553 8904 Acked Write 1 Sent Done Done false
33554 8905 Acked Read 8 Sent Done Done false
33555 8906 Acked Read 7 Sent Done Done false
33556 8907 Acked Flush 0 Sent Done Done false
33557 8908 Acked Read 4 Sent Done Done false
33558 8909 Acked Flush 0 Sent Done Done false
33559 8910 Acked Write 5 Sent Done Done false
33560 8911 Acked Flush 0 Sent Done Done false
33561 8912 Acked Write 9 Sent Done Done false
33562 8913 Acked Read 5 Sent Done Done false
33563 8914 Acked Flush 0 Sent Done Done false
33564 8915 Acked Write 3 Sent Done Done false
33565 8916 Acked Write 3 Sent Done Done false
33566 8917 Acked Read 2 Sent Done Done false
33567 8918 Acked Flush 0 Sent Done Done false
33568 8919 Acked Write 1 Sent Done Done false
33569 8920 Acked Read 6 Sent Done Done false
33570 8921 Acked Read 10 Sent Done Done false
33571 8922 Acked Read 4 Sent Done Done false
33572 8923 Acked Write 10 Sent Done Done false
33573 8924 Acked Flush 0 Sent Done Done false
33574 8925 Acked Write 1 Sent Done Done false
33575 8926 Acked Write 4 Sent Done Done false
33576 8927 Acked Read 8 Sent Done Done false
33577 8928 Acked Write 7 Sent Done Done false
33578 8929 Acked Read 7 Sent Done Done false
33579 8930 Acked Write 8 Sent Done Done false
33580 8931 Acked Read 8 Sent Done Done false
33581 8932 Acked Read 1 Sent Done Done false
33582 8933 Acked Write 9 Sent Done Done false
33583 8934 Acked Read 6 Sent Done Done false
33584 8935 Acked Read 3 Sent Done Done false
33585 8936 Acked Flush 0 Sent Done Done false
33586 8937 Acked Write 10 Sent Done Done false
33587 8938 Acked Read 5 Sent Done Done false
33588 8939 Acked Write 2 Sent Done Done false
33589 8940 Acked Write 2 Sent Done Done false
33590 8941 Acked Write 9 Sent Done Done false
33591 8942 Acked Read 3 Sent Done Done false
33592 8943 Acked Read 8 Sent Done Done false
33593 8944 Acked Write 1 Sent Done Done false
33594 8945 Acked Write 4 Sent Done Done false
33595 8946 Acked Write 6 Sent Done Done false
33596 8947 Acked Read 7 Sent Done Done false
33597 8948 Acked Write 9 Sent Done Done false
33598 8949 Acked Write 6 Sent Done Done false
33599 8950 Acked Flush 0 Sent Done Done false
33600 8951 Acked Read 1 Sent Done Done false
33601 8952 Acked Write 4 Sent Done Done false
33602 8953 Acked Write 3 Sent Done Done false
33603 8954 Acked Write 10 Sent Done Done false
33604 8955 Acked Write 6 Sent Done Done false
33605 8956 Acked Write 7 Sent Done Done false
33606 8957 Acked Write 10 Sent Done Done false
33607 8958 Acked Flush 0 Sent Done Done false
33608 8959 Acked Read 10 Sent Done Done false
33609 8960 Acked Write 4 Sent Done Done false
33610 8961 Acked Read 9 Sent Done Done false
33611 8962 Acked Read 8 Sent Done Done false
33612 8963 Acked Flush 0 Sent Done Done false
33613 8964 Acked Read 8 Sent Done Done false
33614 8965 Acked Write 1 Sent Done Done false
33615 8966 Acked Read 4 Sent Done Done false
33616 8967 Acked Flush 0 Sent Done Done false
33617 8968 Acked Read 5 Sent Done Done false
33618 8969 Acked Read 4 Sent Done Done false
33619 8970 Acked Write 8 Sent Done Done false
33620 8971 Acked Read 6 Sent Done Done false
33621 8972 Acked Read 6 Sent Done Done false
33622 8973 Acked Read 1 Sent Done Done false
33623 8974 Acked Write 5 Sent Done Done false
33624 8975 Acked Read 10 Sent Done Done false
33625 8976 Acked Write 5 Sent Done Done false
33626 8977 Acked Read 6 Sent Done Done false
33627 8978 Acked Write 1 Sent Done Done false
33628 8979 Acked Read 8 Sent Done Done false
33629 8980 Acked Read 4 Sent Done Done false
33630 8981 Acked Flush 0 Sent Done Done false
33631 8982 Acked Write 8 Sent Done Done false
33632 8983 Acked Read 9 Sent Done Done false
33633 8984 Acked Write 3 Sent Done Done false
33634 8985 Acked Write 4 Sent Done Done false
33635 8986 Acked Write 7 Sent Done Done false
33636 8987 Acked Read 10 Sent Done Done false
33637 8988 Acked Flush 0 Sent Done Done false
33638 8989 Acked Read 2 Sent Done Done false
33639 8990 Acked Write 5 Sent Done Done false
33640 8991 Acked Write 4 Sent Done Done false
33641 8992 Acked Write 4 Sent Done Done false
33642 8993 Acked Write 8 Sent Done Done false
33643 8994 Acked Write 10 Sent Done Done false
33644 8995 Acked Read 9 Sent Done Done false
33645 8996 Acked Read 3 Sent Done Done false
33646 8997 Acked Write 10 Sent Done Done false
33647 8998 Acked Flush 0 Sent Done Done false
33648 8999 Acked Read 1 Sent Done Done false
33649 9000 Acked Write 5 Sent Done Done false
33650 9001 Acked Read 7 Sent Done Done false
33651 9002 Acked Read 5 Sent Done Done false
33652 9003 Acked Read 7 Sent Done Done false
33653 9004 Acked Flush 0 Sent Done Done false
33654 9005 Acked Read 9 Sent Done Done false
33655 9006 Acked Read 2 Sent Done Done false
33656 9007 Acked Read 10 Sent Done Done false
33657 9008 Acked Read 2 Sent Done Done false
33658 9009 Acked Write 10 Sent Done Done false
33659 9010 Acked Write 8 Sent Done Done false
33660 9011 Acked Flush 0 Sent Done Done false
33661 9012 Acked Read 3 Sent Done Done false
33662 9013 Acked Write 2 Sent Done Done false
33663 9014 Acked Write 7 Sent Done Done false
33664 9015 Acked Read 4 Sent Done Done false
33665 9016 Acked Write 6 Sent Done Done false
33666 9017 Acked Read 10 Sent Done Done false
33667 9018 Acked Write 5 Sent Done Done false
33668 9019 Acked Flush 0 Sent Done Done false
33669 9020 Acked Read 9 Sent Done Done false
33670 9021 Acked Flush 0 Sent Done Done false
33671 9022 Acked Read 6 Sent Done Done false
33672 9023 Acked Write 6 Sent Done Done false
33673 9024 Acked Write 7 Sent Done Done false
33674 9025 Acked Read 4 Sent Done Done false
33675 9026 Acked Read 4 Sent Done Done false
33676 9027 Acked Read 7 Sent Done Done false
33677 9028 Acked Read 3 Sent Done Done false
33678 9029 Acked Read 10 Sent Done Done false
33679 9030 Acked Flush 0 Sent Done Done false
33680 9031 Acked Read 5 Sent Done Done false
33681 9032 Acked Read 3 Sent Done Done false
33682 9033 Acked Write 3 Sent Done Done false
33683 9034 Acked Write 10 Sent Done Done false
33684 9035 Acked Flush 0 Sent Done Done false
33685 9036 Acked Read 3 Sent Done Done false
33686 9037 Acked Read 10 Sent Done Done false
33687 9038 Acked Read 10 Sent Done Done false
33688 9039 Acked Flush 0 Sent Done Done false
33689 9040 Acked Read 6 Sent Done Done false
33690 9041 Acked Read 4 Sent Done Done false
33691 9042 Acked Read 3 Sent Done Done false
33692 9043 Acked Write 2 Sent Done Done false
33693 9044 Acked Write 8 Sent Done Done false
33694 9045 Acked Read 5 Sent Done Done false
33695 9046 Acked Write 5 Sent Done Done false
33696 9047 Acked Write 7 Sent Done Done false
33697 9048 Acked Read 6 Sent Done Done false
33698 9049 Acked Write 2 Sent Done Done false
33699 9050 Acked Write 5 Sent Done Done false
33700 9051 Acked Write 7 Sent Done Done false
33701 9052 Acked Flush 0 Sent Done Done false
33702 9053 Acked Write 10 Sent Done Done false
33703 9054 Acked Read 7 Sent Done Done false
33704 9055 Acked Write 4 Sent Done Done false
33705 9056 Acked Write 2 Sent Done Done false
33706 9057 Acked Read 3 Sent Done Done false
33707 9058 Acked Write 9 Sent Done Done false
33708 9059 Acked Write 4 Sent Done Done false
33709 9060 Acked Read 4 Sent Done Done false
33710 9061 Acked Write 9 Sent Done Done false
33711 9062 Acked Read 10 Sent Done Done false
33712 9063 Acked Write 9 Sent Done Done false
33713 9064 Acked Write 3 Sent Done Done false
33714 9065 Acked Read 1 Sent Done Done false
33715 9066 Acked Read 5 Sent Done Done false
33716 9067 Acked Read 6 Sent Done Done false
33717 9068 Acked Write 3 Sent Done Done false
33718 9069 Acked Read 2 Sent Done Done false
33719 9070 Acked Write 1 Sent Done Done false
33720 9071 Acked Write 8 Sent Done Done false
33721 9072 Acked Write 9 Sent Done Done false
33722 9073 Acked Write 6 Sent Done Done false
33723 9074 Acked Read 8 Sent Done Done false
33724 9075 Acked Write 4 Sent Done Done false
33725 9076 Acked Write 9 Sent Done Done false
33726 9077 Acked Read 5 Sent Done Done false
33727 9078 Acked Read 2 Sent Done Done false
33728 9079 Acked Flush 0 Sent Done Done false
33729 9080 Acked Write 8 Sent Done Done false
33730 9081 Acked Read 5 Sent Done Done false
33731 9082 Acked Read 8 Sent Done Done false
33732 9083 Acked Read 7 Sent Done Done false
33733 9084 Acked Flush 0 Sent Done Done false
33734 9085 Acked Write 2 Sent Done Done false
33735 9086 Acked Write 9 Sent Done Done false
33736 9087 Acked Read 4 Sent Done Done false
33737 9088 Acked Read 7 Sent Done Done false
33738 9089 Acked Write 10 Sent Done Done false
33739 9090 Acked Flush 0 Sent Done Done false
33740 9091 Acked Write 1 Sent Done Done false
33741 9092 Acked Write 2 Sent Done Done false
33742 9093 Acked Flush 0 Sent Done Done false
33743 9094 Acked Read 9 Sent Done Done false
33744 9095 Acked Read 4 Sent Done Done false
33745 9096 Acked Write 3 Sent Done Done false
33746 9097 Acked Read 2 Sent Done Done false
33747 9098 Acked Write 8 Sent Done Done false
33748 9099 Acked Write 3 Sent Done Done false
33749 9100 Acked Flush 0 Sent Done Done false
33750 9101 Acked Write 6 Sent Done Done false
33751 9102 Acked Flush 0 Sent Done Done false
33752 9103 Acked Read 2 Sent Done Done false
33753 9104 Acked Flush 0 Sent Done Done false
33754 9105 Acked Read 8 Sent Done Done false
33755 9106 Acked Read 5 Sent Done Done false
33756 9107 Acked Read 2 Sent Done Done false
33757 9108 Acked Flush 0 Sent Done Done false
33758 9109 Acked Read 2 Sent Done Done false
33759 9110 Acked Write 3 Sent Done Done false
33760 9111 Acked Read 8 Sent Done Done false
33761 9112 Acked Write 1 Sent Done Done false
33762 9113 Acked Read 1 Sent Done Done false
33763 9114 Acked Flush 0 Sent Done Done false
33764 9115 Acked Write 1 Sent Done Done false
33765 9116 Acked Read 2 Sent Done Done false
33766 9117 Acked Write 8 Sent Done Done false
33767 9118 Acked Read 8 Sent Done Done false
33768 9119 Acked Read 1 Sent Done Done false
33769 9120 Acked Read 9 Sent Done Done false
33770 9121 Acked Read 1 Sent Done Done false
33771 9122 Acked Read 7 Sent Done Done false
33772 9123 Acked Write 2 Sent Done Done false
33773 9124 Acked Write 1 Sent Done Done false
33774 9125 Acked Read 10 Sent Done Done false
33775 9126 Acked Read 8 Sent Done Done false
33776 9127 Acked Write 5 Sent Done Done false
33777 9128 Acked Read 3 Sent Done Done false
33778 9129 Acked Write 4 Sent Done Done false
33779 9130 Acked Write 6 Sent Done Done false
33780 9131 Acked Read 4 Sent Done Done false
33781 9132 Acked Read 8 Sent Done Done false
33782 9133 Acked Read 2 Sent Done Done false
33783 9134 Acked Read 9 Sent Done Done false
33784 9135 Acked Read 6 Sent Done Done false
33785 9136 Acked Read 5 Sent Done Done false
33786 9137 Acked Write 7 Sent Done Done false
33787 9138 Acked Write 4 Sent Done Done false
33788 9139 Acked Read 3 Sent Done Done false
33789 9140 Acked Read 1 Sent Done Done false
33790 9141 Acked Read 4 Sent Done Done false
33791 9142 Acked Read 2 Sent Done Done false
33792 9143 Acked Write 1 Sent Done Done false
33793 9144 Acked Read 9 Sent Done Done false
33794 9145 Acked Read 8 Sent Done Done false
33795 9146 Acked Read 8 Sent Done Done false
33796 9147 Acked Read 7 Sent Done Done false
33797 9148 Acked Write 8 Sent Done Done false
33798 9149 Acked Write 3 Sent Done Done false
33799 9150 Acked Read 10 Sent Done Done false
33800 9151 Acked Write 6 Sent Done Done false
33801 9152 Acked Flush 0 Sent Done Done false
33802 STATES DS:0 DS:1 DS:2 TOTAL
33803 Sent 302 0 0 302
33804 Done 0 302 302 604
33805 Skipped 0 0 0 0
33806 Error 0 0 0 0
33807 Last Flush: 8850 9152 9152
33808 Downstairs last five completed: 8850 8849 8848 8847 8846
33809 Upstairs last five completed: 9152 9151 9150 9149 9148
33810 CLIENT: Up:0 ds:302 act:2
338112024-12-11T23:47:34.908ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338122024-12-11T23:47:34.908ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54239) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338132024-12-11T23:47:34.909ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338142024-12-11T23:47:34.909ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338152024-12-11T23:47:34.909ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338162024-12-11T23:47:34.909ZINFOcrucible: send last flush ID to this DS: 8850 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338172024-12-11T23:47:34.910ZINFOcrucible: Replied this last flush ID: 8850 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338182024-12-11T23:47:34.910ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338192024-12-11T23:47:34.910ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338202024-12-11T23:47:34.910ZINFOcrucible: [0] client re-new 302 jobs since flush 8850 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338212024-12-11T23:47:34.910ZINFOcrucible: [0] Replayed 302 jobs since flush: 8850 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
33822 ----------------------------------------------------------------
33823 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33824 Downstairs last five completed: 9152 9151 9150 9149 9148
33825 Upstairs last five completed: 9152 9151 9150 9149 9148
33826 CLIENT: Up:0 ds:0 act:3
33827 Replay: All jobs finished
33828 Main task has work to do, go find it
33829 got dsc Stop(2)
33830 stop 2
33831 Dec 11 23:47:37.151 INFO request completed[8830] Got stop action so:false kr:false
33832 , latency_us: 374, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 860d2a53-2a69-4133-b7d2-1c09be81fc6f, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
33833 Dec 11 23:47:37.152 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/2, method: GET, req_id: 26b70aba-1dbb-435d-a188-60aca9209053, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
338342024-12-11T23:47:37.152ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338352024-12-11T23:47:37.152ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338362024-12-11T23:47:37.152ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338372024-12-11T23:47:37.152ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338382024-12-11T23:47:37.152ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
338392024-12-11T23:47:37.152ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
33840 [8830] Exited with: ExitStatus(unix_wait_status(9))
33841 [8830][2] reports Exit
33842 Dec 11 23:47:41.154 INFO request completed, latency_us: 222, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7af4a459-81b8-466a-bc08-089a3c85102c, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
33843 001/300 Read block 2716 len 20480
33844 002/300 Write block 1754 len 40960 data: 6 5 6 7 7 7 6 4 3 3
33845 003/300 Read block 3351 len 20480
33846 004/300 Write block 2564 len 40960 data: 5 5 5 5 5 5 4 4 4 5
33847 005/300 Write block 3742 len 12288 data: 10 9 8
33848 006/300 Read block 1130 len 16384
33849 007/300 Read block 3418 len 4096
33850 008/300 Write block 929 len 20480 data: 2 3 4 6 5
33851 009/300 Flush
33852 010/300 Flush
33853 011/300 Read block 725 len 36864
33854 012/300 Read block 4262 len 20480
33855 013/300 Read block 4565 len 4096
33856 014/300 Write block 260 len 28672 data: 3 5 5 4 5 6 6
33857 015/300 Read block 4681 len 32768
33858 016/300 Read block 701 len 4096
33859 017/300 Read block 2513 len 40960
33860 018/300 Write block 3251 len 12288 data: 4 3 5
33861 019/300 Read block 3438 len 20480
33862 020/300 Read block 3984 len 28672
33863 021/300 Write block 4028 len 36864 data: 4 3 4 3 3 3 4 4 3
33864 022/300 Flush
33865 023/300 Write block 4742 len 36864 data: 3 3 3 3 4 4 3 4 5
33866 024/300 Write block 839 len 36864 data: 7 6 4 4 4 5 5 5 6
33867 025/300 Write block 285 len 28672 data: 4 6 6 7 6 8 5
33868 026/300 Read block 906 len 12288
33869 027/300 Write block 1710 len 28672 data: 5 4 5 6 6 6 5
33870 028/300 Write block 616 len 20480 data: 4 3 3 4 4
33871 029/300 Read block 2402 len 28672
33872 030/300 Flush
33873 031/300 Read block 2354 len 12288
33874 032/300 Read block 1708 len 8192
33875 033/300 Write block 606 len 12288 data: 2 5 5
33876 034/300 Read block 1597 len 4096
33877 035/300 Write block 2408 len 40960 data: 4 3 3 2 2 4 4 4 5 6
33878 036/300 Read block 2784 len 28672
33879 037/300 Write block 585 len 24576 data: 8 6 5 3 4 6
33880 038/300 Write block 4171 len 20480 data: 6 7 5 5 6
33881 039/300 Flush
33882 040/300 Write block 4259 len 8192 data: 4 4
33883 041/300 Read block 1036 len 32768
33884 042/300 Write block 2792 len 4096 data: 4
33885 043/300 Read block 3111 len 28672
33886 044/300 Write block 193 len 32768 data: 3 3 4 5 5 6 5 5
33887 045/300 Read block 4527 len 36864
33888 046/300 Read block 3611 len 28672
33889 047/300 Read block 2483 len 32768
33890 048/300 Read block 4547 len 28672
33891 049/300 Read block 3938 len 8192
33892 050/300 Read block 558 len 32768
33893 051/300 Write block 251 len 32768 data: 4 6 5 6 6 4 3 3
33894 052/300 Read block 960 len 8192
33895 053/300 Read block 186 len 32768
33896 054/300 Read block 735 len 4096
33897 055/300 Read block 4635 len 24576
33898 056/300 Read block 2768 len 40960
33899 057/300 Write block 4131 len 40960 data: 5 7 5 5 5 6 6 4 6 6
33900 058/300 Write block 2521 len 36864 data: 5 5 5 4 5 4 4 4 3
33901 059/300 Read block 206 len 36864
33902 060/300 Read block 1383 len 12288
33903 061/300 Flush
33904 062/300 Read block 2128 len 16384
33905 063/300 Read block 3033 len 36864
33906 064/300 Write block 2490 len 36864 data: 4 4 4 5 5 4 4 4 5
33907 065/300 Read block 4252 len 28672
33908 066/300 Read block 3016 len 32768
33909 067/300 Write block 4120 len 32768 data: 8 8 11 7 10 10 7 6
33910 068/300 Read block 3885 len 28672
33911 069/300 Read block 1363 len 40960
33912 070/300 Read block 1643 len 12288
33913 071/300 Flush
33914 072/300 Write block 3368 len 32768 data: 6 6 5 7 7 7 7 6
33915 073/300 Read block 4029 len 12288
33916 074/300 Flush
33917 075/300 Write block 4687 len 24576 data: 3 3 3 3 3 3
33918 076/300 Read block 3308 len 28672
33919 077/300 Read block 4310 len 20480
33920 078/300 Write block 2544 len 12288 data: 7 7 9
33921 079/300 Flush
33922 080/300 Write block 1485 len 20480 data: 6 5 5 5 5
33923 081/300 Read block 4281 len 16384
33924 082/300 Write block 845 len 36864 data: 6 6 7 5 5 5 4 3 4
33925 083/300 Read block 3219 len 40960
33926 084/300 Read block 1586 len 32768
33927 085/300 Read block 977 len 16384
33928 086/300 Read block 270 len 8192
33929 087/300 Write block 4954 len 12288 data: 4 5 7
33930 088/300 Read block 2746 len 20480
33931 089/300 Read block 944 len 20480
33932 090/300 Read block 1015 len 24576
33933 091/300 Write block 1813 len 16384 data: 5 4 4 3
33934 092/300 Write block 3572 len 4096 data: 5
33935 093/300 Write block 184 len 36864 data: 6 6 6 6 5 6 4 2 3
33936 094/300 Read block 2353 len 24576
33937 095/300 Flush
33938 096/300 Read block 3073 len 36864
33939 097/300 Flush
33940 098/300 Read block 235 len 16384
33941 099/300 Write block 2170 len 20480 data: 6 6 7 7 7
33942 100/300 Flush
33943 101/300 Write block 2598 len 4096 data: 7
33944 102/300 Read block 809 len 40960
33945 103/300 Read block 1942 len 24576
33946 104/300 Read block 752 len 8192
33947 105/300 Read block 3558 len 16384
33948 106/300 Read block 1293 len 24576
33949 107/300 Write block 928 len 28672 data: 2 3 4 5 7 6 6
33950 108/300 Write block 1190 len 8192 data: 9 6
33951 109/300 Read block 1117 len 4096
33952 110/300 Write block 3668 len 16384 data: 5 5 5 5
33953 111/300 Write block 4877 len 36864 data: 4 2 2 3 5 5 5 5 4
33954 112/300 Read block 3368 len 20480
33955 113/300 Write block 306 len 16384 data: 5 5 6 8
33956 114/300 Read block 3794 len 16384
33957 115/300 Write block 184 len 4096 data: 7
33958 116/300 Read block 3929 len 32768
33959 117/300 Read block 3941 len 12288
33960 118/300 Read block 860 len 40960
33961 119/300 Read block 1833 len 12288
33962 120/300 Write block 2240 len 4096 data: 6
33963 121/300 Flush
33964 122/300 Write block 389 len 28672 data: 6 5 6 6 4 3 4
33965 123/300 Write block 3805 len 36864 data: 5 4 4 5 5 5 5 6 6
33966 124/300 Read block 470 len 24576
33967 125/300 Read block 2856 len 16384
33968 126/300 Write block 1463 len 32768 data: 5 5 4 5 6 7 8 8
33969 127/300 Flush
33970 128/300 Read block 924 len 28672
33971 129/300 Read block 2530 len 36864
33972 130/300 Write block 3064 len 8192 data: 6 6
33973 131/300 Flush
33974 132/300 Write block 428 len 20480 data: 4 5 4 5 5
33975 133/300 Read block 532 len 28672
33976 134/300 Read block 2198 len 28672
33977 135/300 Read block 1835 len 4096
33978 136/300 Read block 4056 len 16384
33979 137/300 Read block 4054 len 20480
33980 138/300 Write block 2656 len 36864 data: 6 4 4 3 4 3 3 3 3
33981 139/300 Read block 1006 len 4096
33982 140/300 Write block 4102 len 24576 data: 7 9 8 6 5 5
33983 141/300 Write block 168 len 40960 data: 2 3 4 5 5 4 6 6 5 5
33984 142/300 Read block 4493 len 24576
33985 143/300 Read block 2663 len 32768
33986 144/300 Read block 1050 len 36864
33987 145/300 Read block 4914 len 40960
33988 146/300 Read block 301 len 32768
33989 147/300 Read block 3284 len 4096
33990 148/300 Read block 4223 len 4096
33991 149/300 Read block 1546 len 8192
33992 150/300 Read block 637 len 24576
33993 151/300 Read block 3548 len 8192
33994 152/300 Write block 834 len 8192 data: 6 7
33995 153/300 Read block 1478 len 8192
33996 154/300 Read block 3812 len 36864
33997 155/300 Read block 514 len 8192
33998 156/300 Write block 2195 len 8192 data: 6 6
33999 157/300 Read block 3656 len 36864
34000 158/300 Write block 1186 len 40960 data: 6 9 9 8 10 7 6 6 5 7
34001 159/300 Read block 974 len 16384
34002 160/300 Write block 633 len 28672 data: 6 4 6 8 10 10 12
34003 161/300 Read block 28 len 24576
34004 162/300 Flush
34005 163/300 Write block 2002 len 28672 data: 3 3 4 4 4 5 4
34006 164/300 Write block 438 len 16384 data: 3 3 3 3
34007 165/300 Read block 1420 len 24576
34008 166/300 Flush
34009 167/300 Write block 1069 len 12288 data: 5 6 7
34010 168/300 Write block 2727 len 20480 data: 6 6 7 7 8
34011 169/300 Read block 3609 len 36864
34012 170/300 Read block 4514 len 8192
34013 171/300 Read block 4907 len 12288
34014 172/300 Read block 4202 len 16384
34015 173/300 Read block 1219 len 20480
34016 174/300 Write block 4142 len 40960 data: 6 8 8 8 8 8 8 7 5 6
34017 175/300 Write block 1253 len 4096 data: 5
34018 176/300 Write block 466 len 28672 data: 4 3 2 2 2 3 3
34019 177/300 Write block 4003 len 8192 data: 6 6
34020 178/300 Write block 4323 len 12288 data: 3 3 3
34021 179/300 Read block 2959 len 20480
34022 180/300 Write block 1597 len 16384 data: 6 5 5 5
34023 181/300 Write block 2917 len 28672 data: 4 4 4 3 3 3 6
34024 182/300 Write block 1965 len 32768 data: 5 4 4 4 4 4 6 6
34025 183/300 Read block 3996 len 8192
34026 184/300 Flush
34027 185/300 Read block 3059 len 40960
34028 186/300 Read block 1754 len 28672
34029 187/300 Read block 4702 len 4096
34030 188/300 Flush
34031 189/300 Read block 4044 len 16384
34032 190/300 Read block 1428 len 4096
34033 191/300 Read block 1513 len 4096
34034 192/300 Read block 4091 len 24576
34035 193/300 Write block 2135 len 20480 data: 3 4 3 3 3
34036 194/300 Write block 2555 len 24576 data: 5 4 4 4 5 6
34037 195/300 Write block 783 len 4096 data: 6
34038 196/300 Read block 445 len 32768
34039 197/300 Read block 2020 len 28672
34040 198/300 Read block 3011 len 28672
34041 199/300 Read block 48 len 12288
34042 200/300 Write block 1979 len 28672 data: 8 8 9 8 7 9 8
34043 201/300 Read block 4403 len 12288
34044 202/300 Write block 3304 len 8192 data: 4 4
34045 203/300 Read block 64 len 32768
34046 204/300 Write block 3770 len 40960 data: 6 4 4 3 4 4 4 4 5 5
34047 205/300 Read block 1630 len 8192
34048 206/300 Write block 4871 len 8192 data: 6 5
34049 207/300 Read block 1015 len 20480
34050 208/300 Read block 2267 len 28672
34051 209/300 Flush
34052 210/300 Write block 320 len 24576 data: 4 6 9 11 11 8
34053 211/300 Write block 835 len 36864 data: 8 8 8 8 8 7 5 5 5
34054 212/300 Read block 1182 len 40960
34055 213/300 Read block 2703 len 36864
34056 214/300 Flush
34057 215/300 Read block 2473 len 16384
34058 216/300 Read block 1080 len 40960
34059 217/300 Write block 3609 len 16384 data: 2 2 3 3
34060 218/300 Read block 3918 len 24576
34061 219/300 Write block 816 len 8192 data: 6 6
34062 220/300 Read block 1525 len 32768
34063 221/300 Write block 4905 len 24576 data: 12 9 7 6 4 5
34064 222/300 Read block 679 len 20480
34065 223/300 Write block 4024 len 28672 data: 6 7 6 5 5 4 5
34066 224/300 Read block 3806 len 16384
34067 225/300 Read block 191 len 28672
34068 226/300 Read block 7 len 28672
34069 227/300 Read block 1861 len 24576
34070 228/300 Write block 3707 len 8192 data: 8 7
34071 229/300 Read block 2150 len 32768
34072 230/300 Write block 3405 len 40960 data: 4 5 4 5 5 5 4 3 4 5
34073 231/300 Write block 2312 len 20480 data: 8 7 9 8 7
34074 232/300 Write block 3226 len 20480 data: 6 6 6 4 5
34075 233/300 Read block 3229 len 36864
34076 234/300 Read block 967 len 24576
34077 235/300 Write block 3048 len 32768 data: 6 6 5 5 7 6 7 7
34078 236/300 Write block 3844 len 8192 data: 5 4
34079 237/300 Write block 4752 len 28672 data: 6 7 6 8 6 6 5
34080 238/300 Read block 4581 len 20480
34081 239/300 Read block 211 len 8192
34082 240/300 Read block 1122 len 40960
34083 241/300 Read block 1920 len 4096
34084 242/300 Write block 3986 len 40960 data: 7 5 4 4 5 8 9 8 7 7
34085 243/300 Read block 4791 len 24576
34086 244/300 Read block 381 len 32768
34087 245/300 Write block 1789 len 8192 data: 4 5
34088 246/300 Read block 1520 len 4096
34089 247/300 Write block 4633 len 16384 data: 7 7 6 3
34090 248/300 Write block 4343 len 16384 data: 7 6 6 5
34091 249/300 Write block 593 len 24576 data: 5 5 5 3 3 5
34092 250/300 Read block 1567 len 36864
34093 251/300 Flush
34094 252/300 Write block 2129 len 32768 data: 7 7 7 5 5 5 4 5
34095 253/300 Write block 1282 len 16384 data: 2 3 3 3
34096 254/300 Write block 2857 len 16384 data: 7 6 6 6
34097 255/300 Write block 3489 len 4096 data: 11
34098 256/300 Read block 4886 len 40960
34099 257/300 Read block 33 len 20480
34100 258/300 Write block 32 len 12288 data: 6 5 5
34101 259/300 Write block 2900 len 16384 data: 9 9 8 7
34102 260/300 Read block 499 len 36864
34103 261/300 Write block 2261 len 28672 data: 6 6 4 5 6 5 8
34104 262/300 Write block 4482 len 36864 data: 5 5 6 6 6 4 5 4 3
34105 263/300 Write block 3845 len 36864 data: 5 4 4 5 4 5 5 7 8
34106 264/300 Write block 12 len 24576 data: 6 5 4 3 3 2
34107 265/300 Read block 4399 len 28672
34108 266/300 Read block 2429 len 32768
34109 267/300 Write block 1693 len 28672 data: 6 5 5 8 10 11 8
34110 268/300 Write block 2812 len 8192 data: 5 4
34111 269/300 Write block 1263 len 4096 data: 6
34112 270/300 Flush
34113 271/300 Write block 892 len 32768 data: 6 6 4 4 3 5 6 6
34114 272/300 Write block 3420 len 4096 data: 7
34115 273/300 Write block 4405 len 40960 data: 7 6 5 7 5 4 4 4 3 2
34116 274/300 Read block 3902 len 24576
34117 275/300 Write block 2489 len 4096 data: 5
34118 276/300 Write block 3761 len 20480 data: 5 4 3 5 6
34119 277/300 Write block 4155 len 20480 data: 6 4 4 3 3
34120 278/300 Read block 4352 len 24576
34121 279/300 Read block 2936 len 12288
34122 280/300 Write block 4802 len 36864 data: 8 8 7 6 6 7 6 6 6
34123 281/300 Read block 526 len 12288
34124 282/300 Write block 4058 len 8192 data: 5 5
34125 283/300 Read block 1912 len 12288
34126 284/300 Read block 1152 len 16384
34127 285/300 Write block 3556 len 40960 data: 7 7 7 4 4 5 5 5 5 4
34128 286/300 Flush
34129 287/300 Read block 4515 len 28672
34130 288/300 Flush
34131 289/300 Read block 4086 len 28672
34132 290/300 Read block 4220 len 12288
34133 291/300 Read block 4898 len 8192
34134 292/300 Write block 2082 len 24576 data: 3 4 4 5 6 5
34135 293/300 Write block 106 len 36864 data: 7 7 8 5 5 4 4 4 5
34136 294/300 Write block 1170 len 28672 data: 6 5 6 6 4 5 4
34137 295/300 Read block 3216 len 16384
34138 296/300 Write block 4733 len 8192 data: 7 6
34139 297/300 Write block 1054 len 40960 data: 5 8 8 7 4 3 3 3 4 6
34140 298/300 Read block 4503 len 36864
34141 299/300 Write block 3547 len 8192 data: 7 7
34142 300/300 Read block 417 len 28672
34143 301/300 Read block 2771 len 24576
34144 Main task has work to do, go find it
34145 got dsc Start(2)
34146 start 2
34147 Dec 11 23:47:41.398 INFO request completed, latency_us: 227, response_code: 204, uri: /start/cid/2, method: POST, req_id: 952cb9a8-7e9a-4ded-aa2d-903d41b21466, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
34148 Replay: started 2, returned:Ok(())
34149 ----------------------------------------------------------------
34150 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34151 States: Active Active Offline
34152 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34153 9153 Acked Read 5 Done Done Sent false
34154 9154 Acked Write 10 Done Done Sent false
34155 9155 Acked Read 5 Done Done Sent false
34156 9156 Acked Write 10 Done Done Sent false
34157 9157 Acked Write 3 Done Done Sent false
34158 9158 Acked Read 4 Done Done Sent false
34159 9159 Acked Read 1 Done Done Sent false
34160 9160 Acked Write 5 Done Done Sent false
34161 9161 Acked Flush 0 Done Done Sent false
34162 9162 Acked Flush 0 Done Done Sent false
34163 9163 Acked Read 9 Done Done Sent false
34164 9164 Acked Read 5 Done Done Sent false
34165 9165 Acked Read 1 Done Done Sent false
34166 9166 Acked Write 7 Done Done Sent false
34167 9167 Acked Read 8 Done Done Sent false
34168 9168 Acked Read 1 Done Done Sent false
34169 9169 Acked Read 10 Done Done Sent false
34170 9170 Acked Write 3 Done Done Sent false
34171 9171 Acked Read 5 Done Done Sent false
34172 9172 Acked Read 7 Done Done Sent false
34173 9173 Acked Write 9 Done Done Sent false
34174 9174 Acked Flush 0 Done Done Sent false
34175 9175 Acked Write 9 Done Done Sent false
34176 9176 Acked Write 9 Done Done Sent false
34177 9177 Acked Write 7 Done Done Sent false
34178 9178 Acked Read 3 Done Done Sent false
34179 9179 Acked Write 7 Done Done Sent false
34180 9180 Acked Write 5 Done Done Sent false
34181 9181 Acked Read 7 Done Done Sent false
34182 9182 Acked Flush 0 Done Done Sent false
34183 9183 Acked Read 3 Done Done Sent false
34184 9184 Acked Read 2 Done Done Sent false
34185 9185 Acked Write 3 Done Done Sent false
34186 9186 Acked Read 1 Done Done Sent false
34187 9187 Acked Write 10 Done Done Sent false
34188 9188 Acked Read 7 Done Done Sent false
34189 9189 Acked Write 6 Done Done Sent false
34190 9190 Acked Write 5 Done Done Sent false
34191 9191 Acked Flush 0 Done Done Sent false
34192 9192 Acked Write 2 Done Done Sent false
34193 9193 Acked Read 8 Done Done Sent false
34194 9194 Acked Write 1 Done Done Sent false
34195 9195 Acked Read 7 Done Done Sent false
34196 9196 Acked Write 8 Done Done Sent false
34197 9197 Acked Read 9 Done Done Sent false
34198 9198 Acked Read 7 Done Done Sent false
34199 9199 Acked Read 8 Done Done Sent false
34200 9200 Acked Read 7 Done Done Sent false
34201 9201 Acked Read 2 Done Done Sent false
34202 9202 Acked Read 8 Done Done Sent false
34203 9203 Acked Write 8 Done Done Sent false
34204 9204 Acked Read 2 Done Done Sent false
34205 9205 Acked Read 8 Done Done Sent false
34206 9206 Acked Read 1 Done Done Sent false
34207 9207 Acked Read 6 Done Done Sent false
34208 9208 Acked Read 10 Done Done Sent false
34209 9209 Acked Write 10 Done Done Sent false
34210 9210 Acked Write 9 Done Done Sent false
34211 9211 Acked Read 9 Done Done Sent false
34212 9212 Acked Read 3 Done Done Sent false
34213 9213 Acked Flush 0 Done Done Sent false
34214 9214 Acked Read 4 Done Done Sent false
34215 9215 Acked Read 9 Done Done Sent false
34216 9216 Acked Write 9 Done Done Sent false
34217 9217 Acked Read 7 Done Done Sent false
34218 9218 Acked Read 8 Done Done Sent false
34219 9219 Acked Write 8 Done Done Sent false
34220 9220 Acked Read 7 Done Done Sent false
34221 9221 Acked Read 10 Done Done Sent false
34222 9222 Acked Read 3 Done Done Sent false
34223 9223 Acked Flush 0 Done Done Sent false
34224 9224 Acked Write 8 Done Done Sent false
34225 9225 Acked Read 3 Done Done Sent false
34226 9226 Acked Flush 0 Done Done Sent false
34227 9227 Acked Write 6 Done Done Sent false
34228 9228 Acked Read 7 Done Done Sent false
34229 9229 Acked Read 5 Done Done Sent false
34230 9230 Acked Write 3 Done Done Sent false
34231 9231 Acked Flush 0 Done Done Sent false
34232 9232 Acked Write 5 Done Done Sent false
34233 9233 Acked Read 4 Done Done Sent false
34234 9234 Acked Write 9 Done Done Sent false
34235 9235 Acked Read 10 Done Done Sent false
34236 9236 Acked Read 8 Done Done Sent false
34237 9237 Acked Read 4 Done Done Sent false
34238 9238 Acked Read 2 Done Done Sent false
34239 9239 Acked Write 3 Done Done Sent false
34240 9240 Acked Read 5 Done Done Sent false
34241 9241 Acked Read 5 Done Done Sent false
34242 9242 Acked Read 6 Done Done Sent false
34243 9243 Acked Write 4 Done Done Sent false
34244 9244 Acked Write 1 Done Done Sent false
34245 9245 Acked Write 9 Done Done Sent false
34246 9246 Acked Read 6 Done Done Sent false
34247 9247 Acked Flush 0 Done Done Sent false
34248 9248 Acked Read 9 Done Done Sent false
34249 9249 Acked Flush 0 Done Done Sent false
34250 9250 Acked Read 4 Done Done Sent false
34251 9251 Acked Write 5 Done Done Sent false
34252 9252 Acked Flush 0 Done Done Sent false
34253 9253 Acked Write 1 Done Done Sent false
34254 9254 Acked Read 10 Done Done Sent false
34255 9255 Acked Read 6 Done Done Sent false
34256 9256 Acked Read 2 Done Done Sent false
34257 9257 Acked Read 4 Done Done Sent false
34258 9258 Acked Read 6 Done Done Sent false
34259 9259 Acked Write 7 Done Done Sent false
34260 9260 Acked Write 2 Done Done Sent false
34261 9261 Acked Read 1 Done Done Sent false
34262 9262 Acked Write 4 Done Done Sent false
34263 9263 Acked Write 9 Done Done Sent false
34264 9264 Acked Read 5 Done Done Sent false
34265 9265 Acked Write 4 Done Done Sent false
34266 9266 Acked Read 4 Done Done Sent false
34267 9267 Acked Write 1 Done Done Sent false
34268 9268 Acked Read 8 Done Done Sent false
34269 9269 Acked Read 3 Done Done Sent false
34270 9270 Acked Read 10 Done Done Sent false
34271 9271 Acked Read 3 Done Done Sent false
34272 9272 Acked Write 1 Done Done Sent false
34273 9273 Acked Flush 0 Done Done Sent false
34274 9274 Acked Write 7 Done Done Sent false
34275 9275 Acked Write 9 Done Done Sent false
34276 9276 Acked Read 6 Done Done Sent false
34277 9277 Acked Read 4 Done Done Sent false
34278 9278 Acked Write 8 Done Done Sent false
34279 9279 Acked Flush 0 Done Done Sent false
34280 9280 Acked Read 7 Done Done Sent false
34281 9281 Acked Read 9 Done Done Sent false
34282 9282 Acked Write 2 Done Done Sent false
34283 9283 Acked Flush 0 Done Done Sent false
34284 9284 Acked Write 5 Done Done Sent false
34285 9285 Acked Read 7 Done Done Sent false
34286 9286 Acked Read 7 Done Done Sent false
34287 9287 Acked Read 1 Done Done Sent false
34288 9288 Acked Read 4 Done Done Sent false
34289 9289 Acked Read 5 Done Done Sent false
34290 9290 Acked Write 9 Done Done Sent false
34291 9291 Acked Read 1 Done Done Sent false
34292 9292 Acked Write 6 Done Done Sent false
34293 9293 Acked Write 10 Done Done Sent false
34294 9294 Acked Read 6 Done Done Sent false
34295 9295 Acked Read 8 Done Done Sent false
34296 9296 Acked Read 9 Done Done Sent false
34297 9297 Acked Read 10 Done Done Sent false
34298 9298 Acked Read 8 Done Done Sent false
34299 9299 Acked Read 1 Done Done Sent false
34300 9300 Acked Read 1 Done Done Sent false
34301 9301 Acked Read 2 Done Done Sent false
34302 9302 Acked Read 6 Done Done Sent false
34303 9303 Acked Read 2 Done Done Sent false
34304 9304 Acked Write 2 Done Done Sent false
34305 9305 Acked Read 2 Done Done Sent false
34306 9306 Acked Read 9 Done Done Sent false
34307 9307 Acked Read 2 Done Done Sent false
34308 9308 Acked Write 2 Done Done Sent false
34309 9309 Acked Read 9 Done Done Sent false
34310 9310 Acked Write 10 Done Done Sent false
34311 9311 Acked Read 4 Done Done Sent false
34312 9312 Acked Write 7 Done Done Sent false
34313 9313 Acked Read 6 Done Done Sent false
34314 9314 Acked Flush 0 Done Done Sent false
34315 9315 Acked Write 7 Done Done Sent false
34316 9316 Acked Write 4 Done Done Sent false
34317 9317 Acked Read 6 Done Done Sent false
34318 9318 Acked Flush 0 Done Done Sent false
34319 9319 Acked Write 3 Done Done Sent false
34320 9320 Acked Write 5 Done Done Sent false
34321 9321 Acked Read 9 Done Done Sent false
34322 9322 Acked Read 2 Done Done Sent false
34323 9323 Acked Read 3 Done Done Sent false
34324 9324 Acked Read 4 Done Done Sent false
34325 9325 Acked Read 5 Done Done Sent false
34326 9326 Acked Write 10 Done Done Sent false
34327 9327 Acked Write 1 Done Done Sent false
34328 9328 Acked Write 7 Done Done Sent false
34329 9329 Acked Write 2 Done Done Sent false
34330 9330 Acked Write 3 Done Done Sent false
34331 9331 Acked Read 5 Done Done Sent false
34332 9332 Acked Write 4 Done Done Sent false
34333 9333 Acked Write 7 Done Done Sent false
34334 9334 Acked Write 8 Done Done Sent false
34335 9335 Acked Read 2 Done Done Sent false
34336 9336 Acked Flush 0 Done Done Sent false
34337 9337 Acked Read 10 Done Done Sent false
34338 9338 Acked Read 7 Done Done Sent false
34339 9339 Acked Read 1 Done Done Sent false
34340 9340 Acked Flush 0 Done Done Sent false
34341 9341 Acked Read 4 Done Done Sent false
34342 9342 Acked Read 1 Done Done Sent false
34343 9343 Acked Read 1 Done Done Sent false
34344 9344 Acked Read 6 Done Done Sent false
34345 9345 Acked Write 5 Done Done Sent false
34346 9346 Acked Write 6 Done Done Sent false
34347 9347 Acked Write 1 Done Done Sent false
34348 9348 Acked Read 8 Done Done Sent false
34349 9349 Acked Read 7 Done Done Sent false
34350 9350 Acked Read 7 Done Done Sent false
34351 9351 Acked Read 3 Done Done Sent false
34352 9352 Acked Write 7 Done Done Sent false
34353 9353 Acked Read 3 Done Done Sent false
34354 9354 Acked Write 2 Done Done Sent false
34355 9355 Acked Read 8 Done Done Sent false
34356 9356 Acked Write 10 Done Done Sent false
34357 9357 Acked Read 2 Done Done Sent false
34358 9358 Acked Write 2 Done Done Sent false
34359 9359 Acked Read 5 Done Done Sent false
34360 9360 Acked Read 7 Done Done Sent false
34361 9361 Acked Flush 0 Done Done Sent false
34362 9362 Acked Write 6 Done Done Sent false
34363 9363 Acked Write 9 Done Done Sent false
34364 9364 Acked Read 10 Done Done Sent false
34365 9365 Acked Read 9 Done Done Sent false
34366 9366 Acked Flush 0 Done Done Sent false
34367 9367 Acked Read 4 Done Done Sent false
34368 9368 Acked Read 10 Done Done Sent false
34369 9369 Acked Write 4 Done Done Sent false
34370 9370 Acked Read 6 Done Done Sent false
34371 9371 Acked Write 2 Done Done Sent false
34372 9372 Acked Read 8 Done Done Sent false
34373 9373 Acked Write 6 Done Done Sent false
34374 9374 Acked Read 5 Done Done Sent false
34375 9375 Acked Write 7 Done Done Sent false
34376 9376 Acked Read 4 Done Done Sent false
34377 9377 Acked Read 7 Done Done Sent false
34378 9378 Acked Read 7 Done Done Sent false
34379 9379 Acked Read 6 Done Done Sent false
34380 9380 Acked Write 2 Done Done Sent false
34381 9381 Acked Read 8 Done Done Sent false
34382 9382 Acked Write 10 Done Done Sent false
34383 9383 Acked Write 5 Done Done Sent false
34384 9384 Acked Write 5 Done Done Sent false
34385 9385 Acked Read 9 Done Done Sent false
34386 9386 Acked Read 6 Done Done Sent false
34387 9387 Acked Write 8 Done Done Sent false
34388 9388 Acked Write 2 Done Done Sent false
34389 9389 Acked Write 7 Done Done Sent false
34390 9390 Acked Read 5 Done Done Sent false
34391 9391 Acked Read 2 Done Done Sent false
34392 9392 Acked Read 10 Done Done Sent false
34393 9393 Acked Read 1 Done Done Sent false
34394 9394 Acked Write 10 Done Done Sent false
34395 9395 Acked Read 6 Done Done Sent false
34396 9396 Acked Read 8 Done Done Sent false
34397 9397 Acked Write 2 Done Done Sent false
34398 9398 Acked Read 1 Done Done Sent false
34399 9399 Acked Write 4 Done Done Sent false
34400 9400 Acked Write 4 Done Done Sent false
34401 9401 Acked Write 6 Done Done Sent false
34402 9402 Acked Read 9 Done Done Sent false
34403 9403 Acked Flush 0 Done Done Sent false
34404 9404 Acked Write 8 Done Done Sent false
34405 9405 Acked Write 4 Done Done Sent false
34406 9406 Acked Write 4 Done Done Sent false
34407 9407 Acked Write 1 Done Done Sent false
34408 9408 Acked Read 10 Done Done Sent false
34409 9409 Acked Read 5 Done Done Sent false
34410 9410 Acked Write 3 Done Done Sent false
34411 9411 Acked Write 4 Done Done Sent false
34412 9412 Acked Read 9 Done Done Sent false
34413 9413 Acked Write 7 Done Done Sent false
34414 9414 Acked Write 9 Done Done Sent false
34415 9415 Acked Write 9 Done Done Sent false
34416 9416 Acked Write 6 Done Done Sent false
34417 9417 Acked Read 7 Done Done Sent false
34418 9418 Acked Read 8 Done Done Sent false
34419 9419 Acked Write 7 Done Done Sent false
34420 9420 Acked Write 2 Done Done Sent false
34421 9421 Acked Write 1 Done Done Sent false
34422 9422 Acked Flush 0 Done Done Sent false
34423 9423 Acked Write 8 Done Done Sent false
34424 9424 Acked Write 1 Done Done Sent false
34425 9425 Acked Write 10 Done Done Sent false
34426 9426 Acked Read 6 Done Done Sent false
34427 9427 Acked Write 1 Done Done Sent false
34428 9428 Acked Write 5 Done Done Sent false
34429 9429 Acked Write 5 Done Done Sent false
34430 9430 Acked Read 6 Done Done Sent false
34431 9431 Acked Read 3 Done Done Sent false
34432 9432 Acked Write 9 Done Done Sent false
34433 9433 Acked Read 3 Done Done Sent false
34434 9434 Acked Write 2 Done Done Sent false
34435 9435 Acked Read 3 Done Done Sent false
34436 9436 Acked Read 4 Done Done Sent false
34437 9437 Acked Write 10 Done Done Sent false
34438 9438 Acked Flush 0 Done Done Sent false
34439 9439 Acked Read 7 Done Done Sent false
34440 9440 Acked Flush 0 Done Done Sent false
34441 9441 Acked Read 7 Done Done Sent false
34442 9442 Acked Read 3 Done Done Sent false
34443 9443 Acked Read 2 Done Done Sent false
34444 9444 Acked Write 6 Done Done Sent false
34445 9445 Acked Write 9 Done Done Sent false
34446 9446 Acked Write 7 Done Done Sent false
34447 9447 Acked Read 4 Done Done Sent false
34448 9448 Acked Write 2 Done Done Sent false
34449 9449 Acked Write 10 Done Done Sent false
34450 9450 Acked Read 9 Done Done Sent false
34451 9451 Acked Write 2 Done Done Sent false
34452 9452 Acked Read 7 Done Done Sent false
34453 9453 Acked Read 6 Done Done Sent false
34454 STATES DS:0 DS:1 DS:2 TOTAL
34455 Sent 0 0 301 301
34456 Done 301 301 0 602
34457 Skipped 0 0 0 0
34458 Error 0 0 0 0
34459 Last Flush: 9440 9440 9152
34460 Downstairs last five completed: 9152 9151 9150 9149 9148
34461 Upstairs last five completed: 9453 9452 9451 9450 9449
34462 CLIENT: Up:0 ds:301 act:2
34463 [8830] got start action
34464 [8830] I am going to restart
34465 Starting downstairs at port 8830
34466 Make output file at "/tmp/dsc/downstairs-8830.txt"
34467 [8830][2] reports Starting
34468 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1228)
34469 [8830][2] reports Running
34470 ----------------------------------------------------------------
34471 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34472 States: Active Active Offline
34473 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34474 9153 Acked Read 5 Done Done Sent false
34475 9154 Acked Write 10 Done Done Sent false
34476 9155 Acked Read 5 Done Done Sent false
34477 9156 Acked Write 10 Done Done Sent false
34478 9157 Acked Write 3 Done Done Sent false
34479 9158 Acked Read 4 Done Done Sent false
34480 9159 Acked Read 1 Done Done Sent false
34481 9160 Acked Write 5 Done Done Sent false
34482 9161 Acked Flush 0 Done Done Sent false
34483 9162 Acked Flush 0 Done Done Sent false
34484 9163 Acked Read 9 Done Done Sent false
34485 9164 Acked Read 5 Done Done Sent false
34486 9165 Acked Read 1 Done Done Sent false
34487 9166 Acked Write 7 Done Done Sent false
34488 9167 Acked Read 8 Done Done Sent false
34489 9168 Acked Read 1 Done Done Sent false
34490 9169 Acked Read 10 Done Done Sent false
34491 9170 Acked Write 3 Done Done Sent false
34492 9171 Acked Read 5 Done Done Sent false
34493 9172 Acked Read 7 Done Done Sent false
34494 9173 Acked Write 9 Done Done Sent false
34495 9174 Acked Flush 0 Done Done Sent false
34496 9175 Acked Write 9 Done Done Sent false
34497 9176 Acked Write 9 Done Done Sent false
34498 9177 Acked Write 7 Done Done Sent false
34499 9178 Acked Read 3 Done Done Sent false
34500 9179 Acked Write 7 Done Done Sent false
34501 9180 Acked Write 5 Done Done Sent false
34502 9181 Acked Read 7 Done Done Sent false
34503 9182 Acked Flush 0 Done Done Sent false
34504 9183 Acked Read 3 Done Done Sent false
34505 9184 Acked Read 2 Done Done Sent false
34506 9185 Acked Write 3 Done Done Sent false
34507 9186 Acked Read 1 Done Done Sent false
34508 9187 Acked Write 10 Done Done Sent false
34509 9188 Acked Read 7 Done Done Sent false
34510 9189 Acked Write 6 Done Done Sent false
34511 9190 Acked Write 5 Done Done Sent false
34512 9191 Acked Flush 0 Done Done Sent false
34513 9192 Acked Write 2 Done Done Sent false
34514 9193 Acked Read 8 Done Done Sent false
34515 9194 Acked Write 1 Done Done Sent false
34516 9195 Acked Read 7 Done Done Sent false
34517 9196 Acked Write 8 Done Done Sent false
34518 9197 Acked Read 9 Done Done Sent false
34519 9198 Acked Read 7 Done Done Sent false
34520 9199 Acked Read 8 Done Done Sent false
34521 9200 Acked Read 7 Done Done Sent false
34522 9201 Acked Read 2 Done Done Sent false
34523 9202 Acked Read 8 Done Done Sent false
34524 9203 Acked Write 8 Done Done Sent false
34525 9204 Acked Read 2 Done Done Sent false
34526 9205 Acked Read 8 Done Done Sent false
34527 9206 Acked Read 1 Done Done Sent false
34528 9207 Acked Read 6 Done Done Sent false
34529 9208 Acked Read 10 Done Done Sent false
34530 9209 Acked Write 10 Done Done Sent false
34531 9210 Acked Write 9 Done Done Sent false
34532 9211 Acked Read 9 Done Done Sent false
34533 9212 Acked Read 3 Done Done Sent false
34534 9213 Acked Flush 0 Done Done Sent false
34535 9214 Acked Read 4 Done Done Sent false
34536 9215 Acked Read 9 Done Done Sent false
34537 9216 Acked Write 9 Done Done Sent false
34538 9217 Acked Read 7 Done Done Sent false
34539 9218 Acked Read 8 Done Done Sent false
34540 9219 Acked Write 8 Done Done Sent false
34541 9220 Acked Read 7 Done Done Sent false
34542 9221 Acked Read 10 Done Done Sent false
34543 9222 Acked Read 3 Done Done Sent false
34544 9223 Acked Flush 0 Done Done Sent false
34545 9224 Acked Write 8 Done Done Sent false
34546 9225 Acked Read 3 Done Done Sent false
34547 9226 Acked Flush 0 Done Done Sent false
34548 9227 Acked Write 6 Done Done Sent false
34549 9228 Acked Read 7 Done Done Sent false
34550 9229 Acked Read 5 Done Done Sent false
34551 9230 Acked Write 3 Done Done Sent false
34552 9231 Acked Flush 0 Done Done Sent false
34553 9232 Acked Write 5 Done Done Sent false
34554 9233 Acked Read 4 Done Done Sent false
34555 9234 Acked Write 9 Done Done Sent false
34556 9235 Acked Read 10 Done Done Sent false
34557 9236 Acked Read 8 Done Done Sent false
34558 9237 Acked Read 4 Done Done Sent false
34559 9238 Acked Read 2 Done Done Sent false
34560 9239 Acked Write 3 Done Done Sent false
34561 9240 Acked Read 5 Done Done Sent false
34562 9241 Acked Read 5 Done Done Sent false
34563 9242 Acked Read 6 Done Done Sent false
34564 9243 Acked Write 4 Done Done Sent false
34565 9244 Acked Write 1 Done Done Sent false
34566 9245 Acked Write 9 Done Done Sent false
34567 9246 Acked Read 6 Done Done Sent false
34568 9247 Acked Flush 0 Done Done Sent false
34569 9248 Acked Read 9 Done Done Sent false
34570 9249 Acked Flush 0 Done Done Sent false
34571 9250 Acked Read 4 Done Done Sent false
34572 9251 Acked Write 5 Done Done Sent false
34573 9252 Acked Flush 0 Done Done Sent false
34574 9253 Acked Write 1 Done Done Sent false
34575 9254 Acked Read 10 Done Done Sent false
34576 9255 Acked Read 6 Done Done Sent false
34577 9256 Acked Read 2 Done Done Sent false
34578 9257 Acked Read 4 Done Done Sent false
34579 9258 Acked Read 6 Done Done Sent false
34580 9259 Acked Write 7 Done Done Sent false
34581 9260 Acked Write 2 Done Done Sent false
34582 9261 Acked Read 1 Done Done Sent false
34583 9262 Acked Write 4 Done Done Sent false
34584 9263 Acked Write 9 Done Done Sent false
34585 9264 Acked Read 5 Done Done Sent false
34586 9265 Acked Write 4 Done Done Sent false
34587 9266 Acked Read 4 Done Done Sent false
34588 9267 Acked Write 1 Done Done Sent false
34589 9268 Acked Read 8 Done Done Sent false
34590 9269 Acked Read 3 Done Done Sent false
34591 9270 Acked Read 10 Done Done Sent false
34592 9271 Acked Read 3 Done Done Sent false
34593 9272 Acked Write 1 Done Done Sent false
34594 9273 Acked Flush 0 Done Done Sent false
34595 9274 Acked Write 7 Done Done Sent false
34596 9275 Acked Write 9 Done Done Sent false
34597 9276 Acked Read 6 Done Done Sent false
34598 9277 Acked Read 4 Done Done Sent false
34599 9278 Acked Write 8 Done Done Sent false
34600 9279 Acked Flush 0 Done Done Sent false
34601 9280 Acked Read 7 Done Done Sent false
34602 9281 Acked Read 9 Done Done Sent false
34603 9282 Acked Write 2 Done Done Sent false
34604 9283 Acked Flush 0 Done Done Sent false
34605 9284 Acked Write 5 Done Done Sent false
34606 9285 Acked Read 7 Done Done Sent false
34607 9286 Acked Read 7 Done Done Sent false
34608 9287 Acked Read 1 Done Done Sent false
34609 9288 Acked Read 4 Done Done Sent false
34610 9289 Acked Read 5 Done Done Sent false
34611 9290 Acked Write 9 Done Done Sent false
34612 9291 Acked Read 1 Done Done Sent false
34613 9292 Acked Write 6 Done Done Sent false
34614 9293 Acked Write 10 Done Done Sent false
34615 9294 Acked Read 6 Done Done Sent false
34616 9295 Acked Read 8 Done Done Sent false
34617 9296 Acked Read 9 Done Done Sent false
34618 9297 Acked Read 10 Done Done Sent false
34619 9298 Acked Read 8 Done Done Sent false
34620 9299 Acked Read 1 Done Done Sent false
34621 9300 Acked Read 1 Done Done Sent false
34622 9301 Acked Read 2 Done Done Sent false
34623 9302 Acked Read 6 Done Done Sent false
34624 9303 Acked Read 2 Done Done Sent false
34625 9304 Acked Write 2 Done Done Sent false
34626 9305 Acked Read 2 Done Done Sent false
34627 9306 Acked Read 9 Done Done Sent false
34628 9307 Acked Read 2 Done Done Sent false
34629 9308 Acked Write 2 Done Done Sent false
34630 9309 Acked Read 9 Done Done Sent false
34631 9310 Acked Write 10 Done Done Sent false
34632 9311 Acked Read 4 Done Done Sent false
34633 9312 Acked Write 7 Done Done Sent false
34634 9313 Acked Read 6 Done Done Sent false
34635 9314 Acked Flush 0 Done Done Sent false
34636 9315 Acked Write 7 Done Done Sent false
34637 9316 Acked Write 4 Done Done Sent false
34638 9317 Acked Read 6 Done Done Sent false
34639 9318 Acked Flush 0 Done Done Sent false
34640 9319 Acked Write 3 Done Done Sent false
34641 9320 Acked Write 5 Done Done Sent false
34642 9321 Acked Read 9 Done Done Sent false
34643 9322 Acked Read 2 Done Done Sent false
34644 9323 Acked Read 3 Done Done Sent false
34645 9324 Acked Read 4 Done Done Sent false
34646 9325 Acked Read 5 Done Done Sent false
34647 9326 Acked Write 10 Done Done Sent false
34648 9327 Acked Write 1 Done Done Sent false
34649 9328 Acked Write 7 Done Done Sent false
34650 9329 Acked Write 2 Done Done Sent false
34651 9330 Acked Write 3 Done Done Sent false
34652 9331 Acked Read 5 Done Done Sent false
34653 9332 Acked Write 4 Done Done Sent false
34654 9333 Acked Write 7 Done Done Sent false
34655 9334 Acked Write 8 Done Done Sent false
34656 9335 Acked Read 2 Done Done Sent false
34657 9336 Acked Flush 0 Done Done Sent false
34658 9337 Acked Read 10 Done Done Sent false
34659 9338 Acked Read 7 Done Done Sent false
34660 9339 Acked Read 1 Done Done Sent false
34661 9340 Acked Flush 0 Done Done Sent false
34662 9341 Acked Read 4 Done Done Sent false
34663 9342 Acked Read 1 Done Done Sent false
34664 9343 Acked Read 1 Done Done Sent false
34665 9344 Acked Read 6 Done Done Sent false
34666 9345 Acked Write 5 Done Done Sent false
34667 9346 Acked Write 6 Done Done Sent false
34668 9347 Acked Write 1 Done Done Sent false
34669 9348 Acked Read 8 Done Done Sent false
34670 9349 Acked Read 7 Done Done Sent false
34671 9350 Acked Read 7 Done Done Sent false
34672 9351 Acked Read 3 Done Done Sent false
34673 9352 Acked Write 7 Done Done Sent false
34674 9353 Acked Read 3 Done Done Sent false
34675 9354 Acked Write 2 Done Done Sent false
34676 9355 Acked Read 8 Done Done Sent false
34677 9356 Acked Write 10 Done Done Sent false
34678 9357 Acked Read 2 Done Done Sent false
34679 9358 Acked Write 2 Done Done Sent false
34680 9359 Acked Read 5 Done Done Sent false
34681 9360 Acked Read 7 Done Done Sent false
34682 9361 Acked Flush 0 Done Done Sent false
34683 9362 Acked Write 6 Done Done Sent false
34684 9363 Acked Write 9 Done Done Sent false
34685 9364 Acked Read 10 Done Done Sent false
34686 9365 Acked Read 9 Done Done Sent false
34687 9366 Acked Flush 0 Done Done Sent false
34688 9367 Acked Read 4 Done Done Sent false
34689 9368 Acked Read 10 Done Done Sent false
34690 9369 Acked Write 4 Done Done Sent false
34691 9370 Acked Read 6 Done Done Sent false
34692 9371 Acked Write 2 Done Done Sent false
34693 9372 Acked Read 8 Done Done Sent false
34694 9373 Acked Write 6 Done Done Sent false
34695 9374 Acked Read 5 Done Done Sent false
34696 9375 Acked Write 7 Done Done Sent false
34697 9376 Acked Read 4 Done Done Sent false
34698 9377 Acked Read 7 Done Done Sent false
34699 9378 Acked Read 7 Done Done Sent false
34700 9379 Acked Read 6 Done Done Sent false
34701 9380 Acked Write 2 Done Done Sent false
34702 9381 Acked Read 8 Done Done Sent false
34703 9382 Acked Write 10 Done Done Sent false
34704 9383 Acked Write 5 Done Done Sent false
34705 9384 Acked Write 5 Done Done Sent false
34706 9385 Acked Read 9 Done Done Sent false
34707 9386 Acked Read 6 Done Done Sent false
34708 9387 Acked Write 8 Done Done Sent false
34709 9388 Acked Write 2 Done Done Sent false
34710 9389 Acked Write 7 Done Done Sent false
34711 9390 Acked Read 5 Done Done Sent false
34712 9391 Acked Read 2 Done Done Sent false
34713 9392 Acked Read 10 Done Done Sent false
34714 9393 Acked Read 1 Done Done Sent false
34715 9394 Acked Write 10 Done Done Sent false
34716 9395 Acked Read 6 Done Done Sent false
34717 9396 Acked Read 8 Done Done Sent false
34718 9397 Acked Write 2 Done Done Sent false
34719 9398 Acked Read 1 Done Done Sent false
34720 9399 Acked Write 4 Done Done Sent false
34721 9400 Acked Write 4 Done Done Sent false
34722 9401 Acked Write 6 Done Done Sent false
34723 9402 Acked Read 9 Done Done Sent false
34724 9403 Acked Flush 0 Done Done Sent false
34725 9404 Acked Write 8 Done Done Sent false
34726 9405 Acked Write 4 Done Done Sent false
34727 9406 Acked Write 4 Done Done Sent false
34728 9407 Acked Write 1 Done Done Sent false
34729 9408 Acked Read 10 Done Done Sent false
34730 9409 Acked Read 5 Done Done Sent false
34731 9410 Acked Write 3 Done Done Sent false
34732 9411 Acked Write 4 Done Done Sent false
34733 9412 Acked Read 9 Done Done Sent false
34734 9413 Acked Write 7 Done Done Sent false
34735 9414 Acked Write 9 Done Done Sent false
34736 9415 Acked Write 9 Done Done Sent false
34737 9416 Acked Write 6 Done Done Sent false
34738 9417 Acked Read 7 Done Done Sent false
34739 9418 Acked Read 8 Done Done Sent false
34740 9419 Acked Write 7 Done Done Sent false
34741 9420 Acked Write 2 Done Done Sent false
34742 9421 Acked Write 1 Done Done Sent false
34743 9422 Acked Flush 0 Done Done Sent false
34744 9423 Acked Write 8 Done Done Sent false
34745 9424 Acked Write 1 Done Done Sent false
34746 9425 Acked Write 10 Done Done Sent false
34747 9426 Acked Read 6 Done Done Sent false
34748 9427 Acked Write 1 Done Done Sent false
34749 9428 Acked Write 5 Done Done Sent false
34750 9429 Acked Write 5 Done Done Sent false
34751 9430 Acked Read 6 Done Done Sent false
34752 9431 Acked Read 3 Done Done Sent false
34753 9432 Acked Write 9 Done Done Sent false
34754 9433 Acked Read 3 Done Done Sent false
34755 9434 Acked Write 2 Done Done Sent false
34756 9435 Acked Read 3 Done Done Sent false
34757 9436 Acked Read 4 Done Done Sent false
34758 9437 Acked Write 10 Done Done Sent false
34759 9438 Acked Flush 0 Done Done Sent false
34760 9439 Acked Read 7 Done Done Sent false
34761 9440 Acked Flush 0 Done Done Sent false
34762 9441 Acked Read 7 Done Done Sent false
34763 9442 Acked Read 3 Done Done Sent false
34764 9443 Acked Read 2 Done Done Sent false
34765 9444 Acked Write 6 Done Done Sent false
34766 9445 Acked Write 9 Done Done Sent false
34767 9446 Acked Write 7 Done Done Sent false
34768 9447 Acked Read 4 Done Done Sent false
34769 9448 Acked Write 2 Done Done Sent false
34770 9449 Acked Write 10 Done Done Sent false
34771 9450 Acked Read 9 Done Done Sent false
34772 9451 Acked Write 2 Done Done Sent false
34773 9452 Acked Read 7 Done Done Sent false
34774 9453 Acked Read 6 Done Done Sent false
34775 9454 Acked Flush 0 Done Done Sent false
34776 STATES DS:0 DS:1 DS:2 TOTAL
34777 Sent 0 0 302 302
34778 Done 302 302 0 604
34779 Skipped 0 0 0 0
34780 Error 0 0 0 0
34781 Last Flush: 9454 9454 9152
34782 Downstairs last five completed: 9152 9151 9150 9149 9148
34783 Upstairs last five completed: 9454 9453 9452 9451 9450
34784 CLIENT: Up:0 ds:302 act:2
347852024-12-11T23:47:47.153ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347862024-12-11T23:47:47.153ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57769) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347872024-12-11T23:47:47.154ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347882024-12-11T23:47:47.155ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347892024-12-11T23:47:47.155ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347902024-12-11T23:47:47.155ZINFOcrucible: send last flush ID to this DS: 9152 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347912024-12-11T23:47:47.155ZINFOcrucible: Replied this last flush ID: 9152 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347922024-12-11T23:47:47.155ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347932024-12-11T23:47:47.155ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347942024-12-11T23:47:47.155ZINFOcrucible: [2] client re-new 302 jobs since flush 9152 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
347952024-12-11T23:47:47.155ZINFOcrucible: [2] Replayed 302 jobs since flush: 9152 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
34796 ----------------------------------------------------------------
34797 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34798 Downstairs last five completed: 9454 9453 9452 9451 9450
34799 Upstairs last five completed: 9454 9453 9452 9451 9450
34800 CLIENT: Up:0 ds:0 act:3
34801 Replay: All jobs finished
34802 Main task has work to do, go find it
34803 got dsc Stop(0)
34804 stop 0
34805 Dec 11 23:47:49.403 INFO request completed, [8810] Got stop action so:false kr:false
34806 latency_us: 361, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 62c74ee2-9010-4dee-a4cb-bab6a5af2cde, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
348072024-12-11T23:47:49.404ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
34808 Dec 11 23:47:49.404 INFO request completed, latency_us: 114{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:47:49.404525462Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
34809 , response_code: 200, uri:{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-11T23:47:49.404553843Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
34810 /state/cid/0{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:47:49.404579814Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
34811 , method: GET, req_id: 86f1ef4f-75b1-4b56-912b-47746725a56c, {"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:47:49.404619575Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
34812 remote_addr: 127.0.0.1:62969{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:47:49.404664286Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
34813 , local_addr: 127.0.0.1:9998
34814 [8810] Exited with: ExitStatus(unix_wait_status(9))
34815 [8810][0] reports Exit
34816 Dec 11 23:47:53.406 INFO request completed, latency_us: 239, response_code: 200, uri: /state/cid/0, method: GET, req_id: bc924dbd-e7f4-4a24-b61a-21f9a578404a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
34817 001/300 Read block 2429 len 8192
34818 002/300 Write block 1277 len 24576 data: 6 6 3 3 2 3
34819 003/300 Read block 959 len 36864
34820 004/300 Read block 4082 len 8192
34821 005/300 Flush
34822 006/300 Write block 3417 len 8192 data: 5 5
34823 007/300 Read block 3319 len 12288
34824 008/300 Read block 4284 len 36864
34825 009/300 Write block 1305 len 28672 data: 6 5 8 8 7 6 9
34826 010/300 Write block 4263 len 20480 data: 6 7 6 6 6
34827 011/300 Write block 1171 len 36864 data: 6 7 7 5 6 5 6 6 6
34828 012/300 Read block 597 len 12288
34829 013/300 Write block 4794 len 28672 data: 5 4 3 5 5 4 5
34830 014/300 Flush
34831 015/300 Read block 1028 len 24576
34832 016/300 Write block 542 len 28672 data: 4 4 5 7 6 7 6
34833 017/300 Write block 3264 len 24576 data: 6 6 4 5 5 2
34834 018/300 Read block 2991 len 40960
34835 019/300 Read block 1980 len 4096
34836 020/300 Write block 2598 len 4096 data: 8
34837 021/300 Write block 1961 len 20480 data: 5 6 5 5 6
34838 022/300 Read block 1336 len 28672
34839 023/300 Read block 3467 len 36864
34840 024/300 Read block 2169 len 4096
34841 025/300 Write block 1970 len 8192 data: 5 7
34842 026/300 Write block 2886 len 16384 data: 6 9 9 10
34843 027/300 Write block 1524 len 32768 data: 7 7 6 6 5 5 5 5
34844 028/300 Read block 3150 len 12288
34845 029/300 Read block 1939 len 28672
34846 030/300 Flush
34847 031/300 Write block 2117 len 36864 data: 7 6 7 5 5 6 6 5 7
34848 032/300 Write block 1096 len 36864 data: 7 8 8 7 10 10 7 7 7
34849 033/300 Read block 3586 len 28672
34850 034/300 Read block 678 len 32768
34851 035/300 Read block 2336 len 24576
34852 036/300 Write block 4676 len 24576 data: 4 6 7 7 9 8
34853 037/300 Write block 1332 len 12288 data: 3 3 4
34854 038/300 Read block 3738 len 36864
34855 039/300 Read block 4386 len 16384
34856 040/300 Write block 3722 len 32768 data: 6 5 5 6 7 8 6 6
34857 041/300 Read block 2107 len 24576
34858 042/300 Read block 1257 len 36864
34859 043/300 Write block 3115 len 16384 data: 6 5 5 6
34860 044/300 Write block 3584 len 36864 data: 6 7 6 8 7 6 5 5 4
34861 045/300 Read block 2195 len 4096
34862 046/300 Read block 3794 len 4096
34863 047/300 Write block 3314 len 12288 data: 6 4 4
34864 048/300 Flush
34865 049/300 Read block 1589 len 20480
34866 050/300 Read block 3873 len 28672
34867 051/300 Write block 2471 len 24576 data: 5 4 3 3 3 4
34868 052/300 Read block 1647 len 8192
34869 053/300 Write block 3446 len 8192 data: 4 3
34870 054/300 Read block 3099 len 16384
34871 055/300 Flush
34872 056/300 Write block 3721 len 20480 data: 7 7 6 6 7
34873 057/300 Read block 4699 len 4096
34874 058/300 Read block 2059 len 24576
34875 059/300 Write block 1132 len 40960 data: 3 4 6 6 6 7 8 9 7 6
34876 060/300 Write block 4731 len 24576 data: 5 5 8 7 6 7
34877 061/300 Read block 2072 len 28672
34878 062/300 Read block 1475 len 20480
34879 063/300 Read block 1520 len 32768
34880 064/300 Write block 3866 len 28672 data: 10 11 10 10 10 10 9
34881 065/300 Read block 3409 len 28672
34882 066/300 Write block 2594 len 20480 data: 5 5 5 6 9
34883 067/300 Write block 499 len 12288 data: 8 7 6
34884 068/300 Flush
34885 069/300 Write block 558 len 32768 data: 5 5 5 4 4 5 5 4
34886 070/300 Read block 4897 len 28672
34887 071/300 Write block 2324 len 28672 data: 3 5 6 7 9 9 10
34888 072/300 Read block 4673 len 40960
34889 073/300 Write block 2303 len 36864 data: 5 5 5 5 6 6 7 8 9
34890 074/300 Write block 194 len 36864 data: 4 5 6 6 7 6 6 5 5
34891 075/300 Read block 4904 len 24576
34892 076/300 Read block 696 len 36864
34893 077/300 Read block 4966 len 16384
34894 078/300 Read block 4404 len 8192
34895 079/300 Read block 4711 len 16384
34896 080/300 Read block 3588 len 8192
34897 081/300 Write block 2088 len 4096 data: 5
34898 082/300 Write block 4044 len 12288 data: 10 10 10
34899 083/300 Flush
34900 084/300 Write block 439 len 4096 data: 4
34901 085/300 Read block 789 len 36864
34902 086/300 Write block 179 len 16384 data: 8 8 7 6
34903 087/300 Write block 3485 len 4096 data: 6
34904 088/300 Read block 2229 len 40960
34905 089/300 Write block 2219 len 36864 data: 5 6 6 7 6 7 7 7 6
34906 090/300 Read block 1358 len 4096
34907 091/300 Write block 4776 len 8192 data: 5 6
34908 092/300 Flush
34909 093/300 Read block 4127 len 36864
34910 094/300 Write block 4400 len 20480 data: 8 10 8 6 7
34911 095/300 Write block 4599 len 20480 data: 8 8 7 8 8
34912 096/300 Write block 1034 len 12288 data: 9 9 7
34913 097/300 Write block 348 len 32768 data: 4 3 3 3 3 4 5 6
34914 098/300 Write block 2799 len 28672 data: 5 5 5 7 6 9 7
34915 099/300 Read block 2880 len 8192
34916 100/300 Read block 1544 len 32768
34917 101/300 Read block 2996 len 28672
34918 102/300 Write block 1527 len 32768 data: 7 6 6 6 6 6 5 3
34919 103/300 Flush
34920 104/300 Read block 266 len 28672
34921 105/300 Flush
34922 106/300 Write block 4990 len 40960 data: 5 5 4 4 5 5 5 6 5 3
34923 107/300 Write block 3456 len 4096 data: 3
34924 108/300 Write block 575 len 40960 data: 6 8 9 11 10 11 8 10 8 10
34925 109/300 Read block 872 len 8192
34926 110/300 Write block 3701 len 20480 data: 7 9 6 5 6
34927 111/300 Read block 2532 len 24576
34928 112/300 Read block 744 len 4096
34929 113/300 Read block 4161 len 32768
34930 114/300 Write block 349 len 4096 data: 4
34931 115/300 Read block 1131 len 36864
34932 116/300 Read block 2523 len 24576
34933 117/300 Write block 2817 len 24576 data: 6 6 5 4 6 6
34934 118/300 Write block 3028 len 32768 data: 5 6 6 6 6 7 8 8
34935 119/300 Read block 2428 len 36864
34936 120/300 Write block 2533 len 12288 data: 4 4 4
34937 121/300 Write block 4560 len 4096 data: 6
34938 122/300 Write block 4912 len 12288 data: 6 7 8
34939 123/300 Write block 2479 len 16384 data: 6 5 4 5
34940 124/300 Write block 4992 len 8192 data: 5 5
34941 125/300 Read block 409 len 16384
34942 126/300 Write block 968 len 20480 data: 7 7 6 4 4
34943 127/300 Write block 1771 len 4096 data: 4
34944 128/300 Write block 2082 len 36864 data: 4 5 5 6 7 6 6 4 3
34945 129/300 Read block 1858 len 36864
34946 130/300 Flush
34947 131/300 Read block 3075 len 8192
34948 132/300 Read block 405 len 20480
34949 133/300 Read block 1793 len 16384
34950 134/300 Flush
34951 135/300 Read block 4761 len 12288
34952 136/300 Flush
34953 137/300 Write block 2746 len 24576 data: 6 6 5 5 4 3
34954 138/300 Read block 4249 len 24576
34955 139/300 Read block 3028 len 20480
34956 140/300 Flush
34957 141/300 Write block 4432 len 4096 data: 8
34958 142/300 Flush
34959 143/300 Flush
34960 144/300 Read block 3775 len 36864
34961 145/300 Read block 2515 len 32768
34962 146/300 Write block 3736 len 4096 data: 3
34963 147/300 Flush
34964 148/300 Write block 1323 len 8192 data: 6 3
34965 149/300 Read block 4400 len 32768
34966 150/300 Read block 2398 len 32768
34967 151/300 Write block 109 len 24576 data: 6 6 5 5 5 6
34968 152/300 Write block 3657 len 16384 data: 8 8 5 7
34969 153/300 Flush
34970 154/300 Write block 914 len 36864 data: 5 4 3 3 3 3 4 4 5
34971 155/300 Write block 3171 len 32768 data: 7 7 6 7 6 7 7 6
34972 156/300 Flush
34973 157/300 Read block 422 len 4096
34974 158/300 Write block 1155 len 24576 data: 5 4 5 5 5 5
34975 159/300 Write block 2621 len 4096 data: 6
34976 160/300 Write block 1474 len 40960 data: 2 2 2 2 3 3 3 6 6 8
34977 161/300 Read block 3446 len 36864
34978 162/300 Write block 305 len 16384 data: 6 6 6 7
34979 163/300 Write block 2334 len 24576 data: 10 9 5 4 5 7
34980 164/300 Read block 4771 len 32768
34981 165/300 Write block 3213 len 40960 data: 7 6 6 7 5 3 3 2 2 3
34982 166/300 Flush
34983 167/300 Write block 1075 len 40960 data: 4 5 6 6 4 4 5 6 5 7
34984 168/300 Read block 3253 len 16384
34985 169/300 Write block 4057 len 40960 data: 6 6 6 6 7 8 7 7 7 6
34986 170/300 Read block 346 len 36864
34987 171/300 Write block 1414 len 28672 data: 7 5 5 6 5 3 3
34988 172/300 Read block 1993 len 12288
34989 173/300 Read block 936 len 36864
34990 174/300 Read block 3277 len 12288
34991 175/300 Read block 2554 len 16384
34992 176/300 Read block 1323 len 28672
34993 177/300 Read block 1445 len 40960
34994 178/300 Write block 2500 len 20480 data: 5 6 5 5 5
34995 179/300 Read block 3117 len 40960
34996 180/300 Write block 1352 len 16384 data: 7 7 8 7
34997 181/300 Flush
34998 182/300 Read block 4389 len 12288
34999 183/300 Flush
35000 184/300 Write block 1313 len 28672 data: 10 7 9 7 7 6 6
35001 185/300 Write block 4386 len 20480 data: 7 7 8 8 7
35002 186/300 Write block 4017 len 36864 data: 9 9 8 9 7 6 6 7 8
35003 187/300 Flush
35004 188/300 Write block 646 len 24576 data: 5 5 4 3 3 3
35005 189/300 Read block 4708 len 36864
35006 190/300 Read block 106 len 12288
35007 191/300 Read block 1445 len 4096
35008 192/300 Write block 1201 len 12288 data: 5 4 7
35009 193/300 Flush
35010 194/300 Read block 1943 len 40960
35011 195/300 Read block 3884 len 4096
35012 196/300 Write block 4154 len 40960 data: 6 7 5 5 4 4 4 4 4 7
35013 197/300 Read block 3436 len 40960
35014 198/300 Write block 3193 len 12288 data: 4 5 5
35015 199/300 Read block 4268 len 24576
35016 200/300 Write block 465 len 8192 data: 5 5
35017 201/300 Read block 1918 len 16384
35018 202/300 Write block 4439 len 28672 data: 7 7 8 7 7 6 6
35019 203/300 Write block 1776 len 36864 data: 6 5 4 6 7 10 9 8 6
35020 204/300 Read block 4337 len 24576
35021 205/300 Write block 1363 len 28672 data: 5 5 7 5 6 8 6
35022 206/300 Flush
35023 207/300 Read block 2490 len 36864
35024 208/300 Write block 152 len 16384 data: 4 4 5 7
35025 209/300 Write block 948 len 32768 data: 3 2 2 4 4 4 4 3
35026 210/300 Read block 1105 len 4096
35027 211/300 Write block 2227 len 36864 data: 7 6 6 6 5 5 4 6 7
35028 212/300 Write block 3677 len 12288 data: 4 4 4
35029 213/300 Read block 4455 len 24576
35030 214/300 Read block 475 len 24576
35031 215/300 Write block 2035 len 12288 data: 7 7 6
35032 216/300 Write block 3898 len 40960 data: 9 8 8 8 5 5 5 4 4 6
35033 217/300 Read block 2350 len 28672
35034 218/300 Read block 741 len 4096
35035 219/300 Flush
35036 220/300 Write block 2890 len 28672 data: 10 9 9 9 10 9 9
35037 221/300 Write block 3571 len 32768 data: 7 6 6 3 6 6 7 7
35038 222/300 Write block 2767 len 8192 data: 8 7
35039 223/300 Read block 215 len 20480
35040 224/300 Write block 1925 len 8192 data: 5 8
35041 225/300 Write block 1244 len 36864 data: 5 5 8 8 9 6 5 7 6
35042 226/300 Read block 3962 len 20480
35043 227/300 Read block 3567 len 8192
35044 228/300 Read block 4758 len 20480
35045 229/300 Read block 1989 len 20480
35046 230/300 Read block 1696 len 12288
35047 231/300 Read block 2375 len 32768
35048 232/300 Read block 4595 len 24576
35049 233/300 Flush
35050 234/300 Write block 2855 len 16384 data: 7 6 8 7
35051 235/300 Write block 4862 len 20480 data: 6 5 5 4 4
35052 236/300 Read block 3790 len 32768
35053 237/300 Write block 2356 len 28672 data: 6 6 6 6 6 6 6
35054 238/300 Write block 300 len 40960 data: 6 6 5 5 7 7 7 7 8 9
35055 239/300 Write block 1270 len 4096 data: 9
35056 240/300 Read block 4697 len 4096
35057 241/300 Read block 3550 len 36864
35058 242/300 Read block 4195 len 8192
35059 243/300 Read block 3080 len 40960
35060 244/300 Write block 1341 len 4096 data: 3
35061 245/300 Read block 4466 len 12288
35062 246/300 Read block 56 len 32768
35063 247/300 Read block 3371 len 8192
35064 248/300 Read block 1240 len 32768
35065 249/300 Write block 1644 len 24576 data: 8 8 10 8 8 8
35066 250/300 Write block 2298 len 4096 data: 3
35067 251/300 Read block 1538 len 32768
35068 252/300 Read block 1849 len 40960
35069 253/300 Write block 2315 len 32768 data: 9 8 7 6 7 6 3 3
35070 254/300 Read block 3196 len 24576
35071 255/300 Flush
35072 256/300 Read block 914 len 32768
35073 257/300 Read block 1466 len 40960
35074 258/300 Read block 1254 len 24576
35075 259/300 Write block 201 len 28672 data: 6 6 5 6 4 4 3
35076 260/300 Write block 4088 len 8192 data: 10 10
35077 261/300 Write block 3974 len 28672 data: 6 9 9 8 7 7 6
35078 262/300 Write block 3329 len 4096 data: 11
35079 263/300 Read block 3720 len 40960
35080 264/300 Write block 4296 len 12288 data: 8 7 7
35081 265/300 Read block 2228 len 12288
35082 266/300 Write block 4935 len 12288 data: 5 4 6
35083 267/300 Read block 2432 len 12288
35084 268/300 Write block 3760 len 8192 data: 5 6
35085 269/300 Write block 184 len 32768 data: 8 7 7 7 6 7 5 3
35086 270/300 Read block 2351 len 24576
35087 271/300 Write block 712 len 36864 data: 5 5 4 3 3 3 4 3 4
35088 272/300 Read block 4254 len 28672
35089 273/300 Read block 3929 len 12288
35090 274/300 Read block 4092 len 32768
35091 275/300 Write block 993 len 4096 data: 3
35092 276/300 Read block 3953 len 4096
35093 277/300 Read block 4873 len 32768
35094 278/300 Read block 4314 len 40960
35095 279/300 Read block 2998 len 40960
35096 280/300 Read block 942 len 8192
35097 281/300 Write block 2696 len 20480 data: 5 5 5 5 5
35098 282/300 Read block 1373 len 16384
35099 283/300 Write block 2069 len 36864 data: 7 6 5 4 3 3 3 3 3
35100 284/300 Write block 4792 len 36864 data: 6 4 6 5 4 6 6 5 6
35101 285/300 Write block 4778 len 24576 data: 6 6 5 5 5 8
35102 286/300 Write block 2908 len 20480 data: 4 4 5 5 5
35103 287/300 Read block 119 len 24576
35104 288/300 Write block 2897 len 4096 data: 8
35105 289/300 Read block 4518 len 40960
35106 290/300 Read block 223 len 36864
35107 291/300 Read block 3612 len 16384
35108 292/300 Read block 3383 len 28672
35109 293/300 Read block 1938 len 8192
35110 294/300 Flush
35111 295/300 Read block 50 len 28672
35112 296/300 Write block 3394 len 20480 data: 4 5 5 4 3
35113 297/300 Write block 1398 len 36864 data: 6 5 4 4 3 4 4 4 4
35114 298/300 Write block 751 len 8192 data: 7 9
35115 299/300 Read block 664 len 28672
35116 300/300 Read block 1211 len 36864
35117 301/300 Flush
35118 Main task has work to do, go find it
35119 got dsc Start(0)
35120 start 0
35121 Dec 11 23:47:53.642 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/0, method: POST, req_id: 3e375ca3-02ca-4732-bf9a-243b58ad8317, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
35122 Replay: started 0, returned:Ok(())
35123 ----------------------------------------------------------------
35124 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35125 States: Offline Active Active
35126 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35127 9455 Acked Read 2 Sent Done Done false
35128 9456 Acked Write 6 Sent Done Done false
35129 9457 Acked Read 9 Sent Done Done false
35130 9458 Acked Read 2 Sent Done Done false
35131 9459 Acked Flush 0 Sent Done Done false
35132 9460 Acked Write 2 Sent Done Done false
35133 9461 Acked Read 3 Sent Done Done false
35134 9462 Acked Read 9 Sent Done Done false
35135 9463 Acked Write 7 Sent Done Done false
35136 9464 Acked Write 5 Sent Done Done false
35137 9465 Acked Write 9 Sent Done Done false
35138 9466 Acked Read 3 Sent Done Done false
35139 9467 Acked Write 7 Sent Done Done false
35140 9468 Acked Flush 0 Sent Done Done false
35141 9469 Acked Read 6 Sent Done Done false
35142 9470 Acked Write 7 Sent Done Done false
35143 9471 Acked Write 6 Sent Done Done false
35144 9472 Acked Read 10 Sent Done Done false
35145 9473 Acked Read 1 Sent Done Done false
35146 9474 Acked Write 1 Sent Done Done false
35147 9475 Acked Write 5 Sent Done Done false
35148 9476 Acked Read 7 Sent Done Done false
35149 9477 Acked Read 9 Sent Done Done false
35150 9478 Acked Read 1 Sent Done Done false
35151 9479 Acked Write 2 Sent Done Done false
35152 9480 Acked Write 4 Sent Done Done false
35153 9481 Acked Write 8 Sent Done Done false
35154 9482 Acked Read 3 Sent Done Done false
35155 9483 Acked Read 7 Sent Done Done false
35156 9484 Acked Flush 0 Sent Done Done false
35157 9485 Acked Write 9 Sent Done Done false
35158 9486 Acked Write 9 Sent Done Done false
35159 9487 Acked Read 7 Sent Done Done false
35160 9488 Acked Read 8 Sent Done Done false
35161 9489 Acked Read 6 Sent Done Done false
35162 9490 Acked Write 6 Sent Done Done false
35163 9491 Acked Write 3 Sent Done Done false
35164 9492 Acked Read 9 Sent Done Done false
35165 9493 Acked Read 4 Sent Done Done false
35166 9494 Acked Write 8 Sent Done Done false
35167 9495 Acked Read 6 Sent Done Done false
35168 9496 Acked Read 9 Sent Done Done false
35169 9497 Acked Write 4 Sent Done Done false
35170 9498 Acked Write 9 Sent Done Done false
35171 9499 Acked Read 1 Sent Done Done false
35172 9500 Acked Read 1 Sent Done Done false
35173 9501 Acked Write 3 Sent Done Done false
35174 9502 Acked Flush 0 Sent Done Done false
35175 9503 Acked Read 5 Sent Done Done false
35176 9504 Acked Read 7 Sent Done Done false
35177 9505 Acked Write 6 Sent Done Done false
35178 9506 Acked Read 2 Sent Done Done false
35179 9507 Acked Write 2 Sent Done Done false
35180 9508 Acked Read 4 Sent Done Done false
35181 9509 Acked Flush 0 Sent Done Done false
35182 9510 Acked Write 5 Sent Done Done false
35183 9511 Acked Read 1 Sent Done Done false
35184 9512 Acked Read 6 Sent Done Done false
35185 9513 Acked Write 10 Sent Done Done false
35186 9514 Acked Write 6 Sent Done Done false
35187 9515 Acked Read 7 Sent Done Done false
35188 9516 Acked Read 5 Sent Done Done false
35189 9517 Acked Read 8 Sent Done Done false
35190 9518 Acked Write 7 Sent Done Done false
35191 9519 Acked Read 7 Sent Done Done false
35192 9520 Acked Write 5 Sent Done Done false
35193 9521 Acked Write 3 Sent Done Done false
35194 9522 Acked Flush 0 Sent Done Done false
35195 9523 Acked Write 8 Sent Done Done false
35196 9524 Acked Read 7 Sent Done Done false
35197 9525 Acked Write 7 Sent Done Done false
35198 9526 Acked Read 10 Sent Done Done false
35199 9527 Acked Write 9 Sent Done Done false
35200 9528 Acked Write 9 Sent Done Done false
35201 9529 Acked Read 6 Sent Done Done false
35202 9530 Acked Read 9 Sent Done Done false
35203 9531 Acked Read 4 Sent Done Done false
35204 9532 Acked Read 2 Sent Done Done false
35205 9533 Acked Read 4 Sent Done Done false
35206 9534 Acked Read 2 Sent Done Done false
35207 9535 Acked Write 1 Sent Done Done false
35208 9536 Acked Write 3 Sent Done Done false
35209 9537 Acked Flush 0 Sent Done Done false
35210 9538 Acked Write 1 Sent Done Done false
35211 9539 Acked Read 9 Sent Done Done false
35212 9540 Acked Write 4 Sent Done Done false
35213 9541 Acked Write 1 Sent Done Done false
35214 9542 Acked Read 10 Sent Done Done false
35215 9543 Acked Write 9 Sent Done Done false
35216 9544 Acked Read 1 Sent Done Done false
35217 9545 Acked Write 2 Sent Done Done false
35218 9546 Acked Flush 0 Sent Done Done false
35219 9547 Acked Read 9 Sent Done Done false
35220 9548 Acked Write 5 Sent Done Done false
35221 9549 Acked Write 5 Sent Done Done false
35222 9550 Acked Write 3 Sent Done Done false
35223 9551 Acked Write 8 Sent Done Done false
35224 9552 Acked Write 7 Sent Done Done false
35225 9553 Acked Read 2 Sent Done Done false
35226 9554 Acked Read 8 Sent Done Done false
35227 9555 Acked Read 7 Sent Done Done false
35228 9556 Acked Write 8 Sent Done Done false
35229 9557 Acked Flush 0 Sent Done Done false
35230 9558 Acked Read 7 Sent Done Done false
35231 9559 Acked Flush 0 Sent Done Done false
35232 9560 Acked Write 10 Sent Done Done false
35233 9561 Acked Write 1 Sent Done Done false
35234 9562 Acked Write 10 Sent Done Done false
35235 9563 Acked Read 2 Sent Done Done false
35236 9564 Acked Write 5 Sent Done Done false
35237 9565 Acked Read 6 Sent Done Done false
35238 9566 Acked Read 1 Sent Done Done false
35239 9567 Acked Read 8 Sent Done Done false
35240 9568 Acked Write 1 Sent Done Done false
35241 9569 Acked Read 9 Sent Done Done false
35242 9570 Acked Read 6 Sent Done Done false
35243 9571 Acked Write 6 Sent Done Done false
35244 9572 Acked Write 8 Sent Done Done false
35245 9573 Acked Read 9 Sent Done Done false
35246 9574 Acked Write 3 Sent Done Done false
35247 9575 Acked Write 1 Sent Done Done false
35248 9576 Acked Write 3 Sent Done Done false
35249 9577 Acked Write 4 Sent Done Done false
35250 9578 Acked Write 2 Sent Done Done false
35251 9579 Acked Read 4 Sent Done Done false
35252 9580 Acked Write 5 Sent Done Done false
35253 9581 Acked Write 1 Sent Done Done false
35254 9582 Acked Write 9 Sent Done Done false
35255 9583 Acked Read 9 Sent Done Done false
35256 9584 Acked Flush 0 Sent Done Done false
35257 9585 Acked Read 2 Sent Done Done false
35258 9586 Acked Read 5 Sent Done Done false
35259 9587 Acked Read 4 Sent Done Done false
35260 9588 Acked Flush 0 Sent Done Done false
35261 9589 Acked Read 3 Sent Done Done false
35262 9590 Acked Flush 0 Sent Done Done false
35263 9591 Acked Write 6 Sent Done Done false
35264 9592 Acked Read 6 Sent Done Done false
35265 9593 Acked Read 5 Sent Done Done false
35266 9594 Acked Flush 0 Sent Done Done false
35267 9595 Acked Write 1 Sent Done Done false
35268 9596 Acked Flush 0 Sent Done Done false
35269 9597 Acked Flush 0 Sent Done Done false
35270 9598 Acked Read 9 Sent Done Done false
35271 9599 Acked Read 8 Sent Done Done false
35272 9600 Acked Write 1 Sent Done Done false
35273 9601 Acked Flush 0 Sent Done Done false
35274 9602 Acked Write 2 Sent Done Done false
35275 9603 Acked Read 8 Sent Done Done false
35276 9604 Acked Read 8 Sent Done Done false
35277 9605 Acked Write 6 Sent Done Done false
35278 9606 Acked Write 4 Sent Done Done false
35279 9607 Acked Flush 0 Sent Done Done false
35280 9608 Acked Write 9 Sent Done Done false
35281 9609 Acked Write 8 Sent Done Done false
35282 9610 Acked Flush 0 Sent Done Done false
35283 9611 Acked Read 1 Sent Done Done false
35284 9612 Acked Write 6 Sent Done Done false
35285 9613 Acked Write 1 Sent Done Done false
35286 9614 Acked Write 10 Sent Done Done false
35287 9615 Acked Read 9 Sent Done Done false
35288 9616 Acked Write 4 Sent Done Done false
35289 9617 Acked Write 6 Sent Done Done false
35290 9618 Acked Read 8 Sent Done Done false
35291 9619 Acked Write 10 Sent Done Done false
35292 9620 Acked Flush 0 Sent Done Done false
35293 9621 Acked Write 10 Sent Done Done false
35294 9622 Acked Read 4 Sent Done Done false
35295 9623 Acked Write 10 Sent Done Done false
35296 9624 Acked Read 9 Sent Done Done false
35297 9625 Acked Write 7 Sent Done Done false
35298 9626 Acked Read 3 Sent Done Done false
35299 9627 Acked Read 9 Sent Done Done false
35300 9628 Acked Read 3 Sent Done Done false
35301 9629 Acked Read 4 Sent Done Done false
35302 9630 Acked Read 7 Sent Done Done false
35303 9631 Acked Read 10 Sent Done Done false
35304 9632 Acked Write 5 Sent Done Done false
35305 9633 Acked Read 10 Sent Done Done false
35306 9634 Acked Write 4 Sent Done Done false
35307 9635 Acked Flush 0 Sent Done Done false
35308 9636 Acked Read 3 Sent Done Done false
35309 9637 Acked Flush 0 Sent Done Done false
35310 9638 Acked Write 7 Sent Done Done false
35311 9639 Acked Write 5 Sent Done Done false
35312 9640 Acked Write 9 Sent Done Done false
35313 9641 Acked Flush 0 Sent Done Done false
35314 9642 Acked Write 6 Sent Done Done false
35315 9643 Acked Read 9 Sent Done Done false
35316 9644 Acked Read 3 Sent Done Done false
35317 9645 Acked Read 1 Sent Done Done false
35318 9646 Acked Write 3 Sent Done Done false
35319 9647 Acked Flush 0 Sent Done Done false
35320 9648 Acked Read 10 Sent Done Done false
35321 9649 Acked Read 1 Sent Done Done false
35322 9650 Acked Write 10 Sent Done Done false
35323 9651 Acked Read 10 Sent Done Done false
35324 9652 Acked Write 3 Sent Done Done false
35325 9653 Acked Read 6 Sent Done Done false
35326 9654 Acked Write 2 Sent Done Done false
35327 9655 Acked Read 4 Sent Done Done false
35328 9656 Acked Write 7 Sent Done Done false
35329 9657 Acked Write 9 Sent Done Done false
35330 9658 Acked Read 6 Sent Done Done false
35331 9659 Acked Write 7 Sent Done Done false
35332 9660 Acked Flush 0 Sent Done Done false
35333 9661 Acked Read 9 Sent Done Done false
35334 9662 Acked Write 4 Sent Done Done false
35335 9663 Acked Write 8 Sent Done Done false
35336 9664 Acked Read 1 Sent Done Done false
35337 9665 Acked Write 9 Sent Done Done false
35338 9666 Acked Write 3 Sent Done Done false
35339 9667 Acked Read 6 Sent Done Done false
35340 9668 Acked Read 6 Sent Done Done false
35341 9669 Acked Write 3 Sent Done Done false
35342 9670 Acked Write 10 Sent Done Done false
35343 9671 Acked Read 7 Sent Done Done false
35344 9672 Acked Read 1 Sent Done Done false
35345 9673 Acked Flush 0 Sent Done Done false
35346 9674 Acked Write 7 Sent Done Done false
35347 9675 Acked Write 8 Sent Done Done false
35348 9676 Acked Write 2 Sent Done Done false
35349 9677 Acked Read 5 Sent Done Done false
35350 9678 Acked Write 2 Sent Done Done false
35351 9679 Acked Write 9 Sent Done Done false
35352 9680 Acked Read 5 Sent Done Done false
35353 9681 Acked Read 2 Sent Done Done false
35354 9682 Acked Read 5 Sent Done Done false
35355 9683 Acked Read 5 Sent Done Done false
35356 9684 Acked Read 3 Sent Done Done false
35357 9685 Acked Read 8 Sent Done Done false
35358 9686 Acked Read 6 Sent Done Done false
35359 9687 Acked Flush 0 Sent Done Done false
35360 9688 Acked Write 4 Sent Done Done false
35361 9689 Acked Write 5 Sent Done Done false
35362 9690 Acked Read 8 Sent Done Done false
35363 9691 Acked Write 7 Sent Done Done false
35364 9692 Acked Write 10 Sent Done Done false
35365 9693 Acked Write 1 Sent Done Done false
35366 9694 Acked Read 1 Sent Done Done false
35367 9695 Acked Read 9 Sent Done Done false
35368 9696 Acked Read 2 Sent Done Done false
35369 9697 Acked Read 10 Sent Done Done false
35370 9698 Acked Write 1 Sent Done Done false
35371 9699 Acked Read 3 Sent Done Done false
35372 9700 Acked Read 8 Sent Done Done false
35373 9701 Acked Read 2 Sent Done Done false
35374 9702 Acked Read 8 Sent Done Done false
35375 9703 Acked Write 6 Sent Done Done false
35376 9704 Acked Write 1 Sent Done Done false
35377 9705 Acked Read 8 Sent Done Done false
35378 9706 Acked Read 10 Sent Done Done false
35379 9707 Acked Write 8 Sent Done Done false
35380 9708 Acked Read 6 Sent Done Done false
35381 9709 Acked Flush 0 Sent Done Done false
35382 9710 Acked Read 8 Sent Done Done false
35383 9711 Acked Read 10 Sent Done Done false
35384 9712 Acked Read 6 Sent Done Done false
35385 9713 Acked Write 7 Sent Done Done false
35386 9714 Acked Write 2 Sent Done Done false
35387 9715 Acked Write 7 Sent Done Done false
35388 9716 Acked Write 1 Sent Done Done false
35389 9717 Acked Read 10 Sent Done Done false
35390 9718 Acked Write 3 Sent Done Done false
35391 9719 Acked Read 3 Sent Done Done false
35392 9720 Acked Write 3 Sent Done Done false
35393 9721 Acked Read 3 Sent Done Done false
35394 9722 Acked Write 2 Sent Done Done false
35395 9723 Acked Write 8 Sent Done Done false
35396 9724 Acked Read 6 Sent Done Done false
35397 9725 Acked Write 9 Sent Done Done false
35398 9726 Acked Read 7 Sent Done Done false
35399 9727 Acked Read 3 Sent Done Done false
35400 9728 Acked Read 8 Sent Done Done false
35401 9729 Acked Write 1 Sent Done Done false
35402 9730 Acked Read 1 Sent Done Done false
35403 9731 Acked Read 8 Sent Done Done false
35404 9732 Acked Read 10 Sent Done Done false
35405 9733 Acked Read 10 Sent Done Done false
35406 9734 Acked Read 2 Sent Done Done false
35407 9735 Acked Write 5 Sent Done Done false
35408 9736 Acked Read 4 Sent Done Done false
35409 9737 Acked Write 9 Sent Done Done false
35410 9738 Acked Write 9 Sent Done Done false
35411 9739 Acked Write 6 Sent Done Done false
35412 9740 Acked Write 5 Sent Done Done false
35413 9741 Acked Read 6 Sent Done Done false
35414 9742 Acked Write 1 Sent Done Done false
35415 9743 Acked Read 10 Sent Done Done false
35416 9744 Acked Read 9 Sent Done Done false
35417 9745 Acked Read 4 Sent Done Done false
35418 9746 Acked Read 7 Sent Done Done false
35419 9747 Acked Read 2 Sent Done Done false
35420 9748 Acked Flush 0 Sent Done Done false
35421 9749 Acked Read 7 Sent Done Done false
35422 9750 Acked Write 5 Sent Done Done false
35423 9751 Acked Write 9 Sent Done Done false
35424 9752 Acked Write 2 Sent Done Done false
35425 9753 Acked Read 7 Sent Done Done false
35426 9754 Acked Read 9 Sent Done Done false
35427 9755 Acked Flush 0 Sent Done Done false
35428 STATES DS:0 DS:1 DS:2 TOTAL
35429 Sent 301 0 0 301
35430 Done 0 301 301 602
35431 Skipped 0 0 0 0
35432 Error 0 0 0 0
35433 Last Flush: 9454 9755 9755
35434 Downstairs last five completed: 9454 9453 9452 9451 9450
35435 Upstairs last five completed: 9755 9754 9753 9752 9751
35436 CLIENT: Up:0 ds:301 act:2
35437 ----------------------------------------------------------------
35438 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35439 States: Offline Active Active
35440 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35441 9455 Acked Read 2 Sent Done Done false
35442 9456 Acked Write 6 Sent Done Done false
35443 9457 Acked Read 9 Sent Done Done false
35444 9458 Acked Read 2 Sent Done Done false
35445 9459 Acked Flush 0 Sent Done Done false
35446 9460 Acked Write 2 Sent Done Done false
35447 9461 Acked Read 3 Sent Done Done false
35448 9462 Acked Read 9 Sent Done Done false
35449 9463 Acked Write 7 Sent Done Done false
35450 9464 Acked Write 5 Sent Done Done false
35451 9465 Acked Write 9 Sent Done Done false
35452 9466 Acked Read 3 Sent Done Done false
35453 9467 Acked Write 7 Sent Done Done false
35454 9468 Acked Flush 0 Sent Done Done false
35455 9469 Acked Read 6 Sent Done Done false
35456 9470 Acked Write 7 Sent Done Done false
35457 9471 Acked Write 6 Sent Done Done false
35458 9472 Acked Read 10 Sent Done Done false
35459 9473 Acked Read 1 Sent Done Done false
35460 9474 Acked Write 1 Sent Done Done false
35461 9475 Acked Write 5 Sent Done Done false
35462 9476 Acked Read 7 Sent Done Done false
35463 9477 Acked Read 9 Sent Done Done false
35464 9478 Acked Read 1 Sent Done Done false
35465 9479 Acked Write 2 Sent Done Done false
35466 9480 Acked Write 4 Sent Done Done false
35467 9481 Acked Write 8 Sent Done Done false
35468 9482 Acked Read 3 Sent Done Done false
35469 9483 Acked Read 7 Sent Done Done false
35470 9484 Acked Flush 0 Sent Done Done false
35471 9485 Acked Write 9 Sent Done Done false
35472 9486 Acked Write 9 Sent Done Done false
35473 9487 Acked Read 7 Sent Done Done false
35474 9488 Acked Read 8 Sent Done Done false
35475 9489 Acked Read 6 Sent Done Done false
35476 9490 Acked Write 6 Sent Done Done false
35477 9491 Acked Write 3 Sent Done Done false
35478 9492 Acked Read 9 Sent Done Done false
35479 9493 Acked Read 4 Sent Done Done false
35480 9494 Acked Write 8 Sent Done Done false
35481 9495 Acked Read 6 Sent Done Done false
35482 9496 Acked Read 9 Sent Done Done false
35483 9497 Acked Write 4 Sent Done Done false
35484 9498 Acked Write 9 Sent Done Done false
35485 9499 Acked Read 1 Sent Done Done false
35486 9500 Acked Read 1 Sent Done Done false
35487 9501 Acked Write 3 Sent Done Done false
35488 9502 Acked Flush 0 Sent Done Done false
35489 9503 Acked Read 5 Sent Done Done false
35490 9504 Acked Read 7 Sent Done Done false
35491 9505 Acked Write 6 Sent Done Done false
35492 9506 Acked Read 2 Sent Done Done false
35493 9507 Acked Write 2 Sent Done Done false
35494 9508 Acked Read 4 Sent Done Done false
35495 9509 Acked Flush 0 Sent Done Done false
35496 9510 Acked Write 5 Sent Done Done false
35497 9511 Acked Read 1 Sent Done Done false
35498 9512 Acked Read 6 Sent Done Done false
35499 9513 Acked Write 10 Sent Done Done false
35500 9514 Acked Write 6 Sent Done Done false
35501 9515 Acked Read 7 Sent Done Done false
35502 9516 Acked Read 5 Sent Done Done false
35503 9517 Acked Read 8 Sent Done Done false
35504 9518 Acked Write 7 Sent Done Done false
35505 9519 Acked Read 7 Sent Done Done false
35506 9520 Acked Write 5 Sent Done Done false
35507 9521 Acked Write 3 Sent Done Done false
35508 9522 Acked Flush 0 Sent Done Done false
35509 9523 Acked Write 8 Sent Done Done false
35510 9524 Acked Read 7 Sent Done Done false
35511 9525 Acked Write 7 Sent Done Done false
35512 9526 Acked Read 10 Sent Done Done false
35513 9527 Acked Write 9 Sent Done Done false
35514 9528 Acked Write 9 Sent Done Done false
35515 9529 Acked Read 6 Sent Done Done false
35516 9530 Acked Read 9 Sent Done Done false
35517 9531 Acked Read 4 Sent Done Done false
35518 9532 Acked Read 2 Sent Done Done false
35519 9533 Acked Read 4 Sent Done Done false
35520 9534 Acked Read 2 Sent Done Done false
35521 9535 Acked Write 1 Sent Done Done false
35522 9536 Acked Write 3 Sent Done Done false
35523 9537 Acked Flush 0 Sent Done Done false
35524 9538 Acked Write 1 Sent Done Done false
35525 9539 Acked Read 9 Sent Done Done false
35526 9540 Acked Write 4 Sent Done Done false
35527 9541 Acked Write 1 Sent Done Done false
35528 9542 Acked Read 10 Sent Done Done false
35529 9543 Acked Write 9 Sent Done Done false
35530 9544 Acked Read 1 Sent Done Done false
35531 9545 Acked Write 2 Sent Done Done false
35532 9546 Acked Flush 0 Sent Done Done false
35533 9547 Acked Read 9 Sent Done Done false
35534 9548 Acked Write 5 Sent Done Done false
35535 9549 Acked Write 5 Sent Done Done false
35536 9550 Acked Write 3 Sent Done Done false
35537 9551 Acked Write 8 Sent Done Done false
35538 9552 Acked Write 7 Sent Done Done false
35539 9553 Acked Read 2 Sent Done Done false
35540 9554 Acked Read 8 Sent Done Done false
35541 9555 Acked Read 7 Sent Done Done false
35542 9556 Acked Write 8 Sent Done Done false
35543 9557 Acked Flush 0 Sent Done Done false
35544 9558 Acked Read 7 Sent Done Done false
35545 9559 Acked Flush 0 Sent Done Done false
35546 9560 Acked Write 10 Sent Done Done false
35547 9561 Acked Write 1 Sent Done Done false
35548 9562 Acked Write 10 Sent Done Done false
35549 9563 Acked Read 2 Sent Done Done false
35550 9564 Acked Write 5 Sent Done Done false
35551 9565 Acked Read 6 Sent Done Done false
35552 9566 Acked Read 1 Sent Done Done false
35553 9567 Acked Read 8 Sent Done Done false
35554 9568 Acked Write 1 Sent Done Done false
35555 9569 Acked Read 9 Sent Done Done false
35556 9570 Acked Read 6 Sent Done Done false
35557 9571 Acked Write 6 Sent Done Done false
35558 9572 Acked Write 8 Sent Done Done false
35559 9573 Acked Read 9 Sent Done Done false
35560 9574 Acked Write 3 Sent Done Done false
35561 9575 Acked Write 1 Sent Done Done false
35562 9576 Acked Write 3 Sent Done Done false
35563 9577 Acked Write 4 Sent Done Done false
35564 9578 Acked Write 2 Sent Done Done false
35565 9579 Acked Read 4 Sent Done Done false
35566 9580 Acked Write 5 Sent Done Done false
35567 9581 Acked Write 1 Sent Done Done false
35568 9582 Acked Write 9 Sent Done Done false
35569 9583 Acked Read 9 Sent Done Done false
35570 9584 Acked Flush 0 Sent Done Done false
35571 9585 Acked Read 2 Sent Done Done false
35572 9586 Acked Read 5 Sent Done Done false
35573 9587 Acked Read 4 Sent Done Done false
35574 9588 Acked Flush 0 Sent Done Done false
35575 9589 Acked Read 3 Sent Done Done false
35576 9590 Acked Flush 0 Sent Done Done false
35577 9591 Acked Write 6 Sent Done Done false
35578 9592 Acked Read 6 Sent Done Done false
35579 9593 Acked Read 5 Sent Done Done false
35580 9594 Acked Flush 0 Sent Done Done false
35581 9595 Acked Write 1 Sent Done Done false
35582 9596 Acked Flush 0 Sent Done Done false
35583 9597 Acked Flush 0 Sent Done Done false
35584 9598 Acked Read 9 Sent Done Done false
35585 9599 Acked Read 8 Sent Done Done false
35586 9600 Acked Write 1 Sent Done Done false
35587 9601 Acked Flush 0 Sent Done Done false
35588 9602 Acked Write 2 Sent Done Done false
35589 9603 Acked Read 8 Sent Done Done false
35590 9604 Acked Read 8 Sent Done Done false
35591 9605 Acked Write 6 Sent Done Done false
35592 9606 Acked Write 4 Sent Done Done false
35593 9607 Acked Flush 0 Sent Done Done false
35594 9608 Acked Write 9 Sent Done Done false
35595 9609 Acked Write 8 Sent Done Done false
35596 9610 Acked Flush 0 Sent Done Done false
35597 9611 Acked Read 1 Sent Done Done false
35598 9612 Acked Write 6 Sent Done Done false
35599 9613 Acked Write 1 Sent Done Done false
35600 9614 Acked Write 10 Sent Done Done false
35601 9615 Acked Read 9 Sent Done Done false
35602 9616 Acked Write 4 Sent Done Done false
35603 9617 Acked Write 6 Sent Done Done false
35604 9618 Acked Read 8 Sent Done Done false
35605 9619 Acked Write 10 Sent Done Done false
35606 9620 Acked Flush 0 Sent Done Done false
35607 9621 Acked Write 10 Sent Done Done false
35608 9622 Acked Read 4 Sent Done Done false
35609 9623 Acked Write 10 Sent Done Done false
35610 9624 Acked Read 9 Sent Done Done false
35611 9625 Acked Write 7 Sent Done Done false
35612 9626 Acked Read 3 Sent Done Done false
35613 9627 Acked Read 9 Sent Done Done false
35614 9628 Acked Read 3 Sent Done Done false
35615 9629 Acked Read 4 Sent Done Done false
35616 9630 Acked Read 7 Sent Done Done false
35617 9631 Acked Read 10 Sent Done Done false
35618 9632 Acked Write 5 Sent Done Done false
35619 9633 Acked Read 10 Sent Done Done false
35620 9634 Acked Write 4 Sent Done Done false
35621 9635 Acked Flush 0 Sent Done Done false
35622 9636 Acked Read 3 Sent Done Done false
35623 9637 Acked Flush 0 Sent Done Done false
35624 9638 Acked Write 7 Sent Done Done false
35625 9639 Acked Write 5 Sent Done Done false
35626 9640 Acked Write 9 Sent Done Done false
35627 9641 Acked Flush 0 Sent Done Done false
35628 9642 Acked Write 6 Sent Done Done false
35629 9643 Acked Read 9 Sent Done Done false
35630 9644 Acked Read 3 Sent Done Done false
35631 9645 Acked Read 1 Sent Done Done false
35632 9646 Acked Write 3 Sent Done Done false
35633 9647 Acked Flush 0 Sent Done Done false
35634 9648 Acked Read 10 Sent Done Done false
35635 9649 Acked Read 1 Sent Done Done false
35636 9650 Acked Write 10 Sent Done Done false
35637 9651 Acked Read 10 Sent Done Done false
35638 9652 Acked Write 3 Sent Done Done false
35639 9653 Acked Read 6 Sent Done Done false
35640 9654 Acked Write 2 Sent Done Done false
35641 9655 Acked Read 4 Sent Done Done false
35642 9656 Acked Write 7 Sent Done Done false
35643 9657 Acked Write 9 Sent Done Done false
35644 9658 Acked Read 6 Sent Done Done false
35645 9659 Acked Write 7 Sent Done Done false
35646 9660 Acked Flush 0 Sent Done Done false
35647 9661 Acked Read 9 Sent Done Done false
35648 9662 Acked Write 4 Sent Done Done false
35649 9663 Acked Write 8 Sent Done Done false
35650 9664 Acked Read 1 Sent Done Done false
35651 9665 Acked Write 9 Sent Done Done false
35652 9666 Acked Write 3 Sent Done Done false
35653 9667 Acked Read 6 Sent Done Done false
35654 9668 Acked Read 6 Sent Done Done false
35655 9669 Acked Write 3 Sent Done Done false
35656 9670 Acked Write 10 Sent Done Done false
35657 9671 Acked Read 7 Sent Done Done false
35658 9672 Acked Read 1 Sent Done Done false
35659 9673 Acked Flush 0 Sent Done Done false
35660 9674 Acked Write 7 Sent Done Done false
35661 9675 Acked Write 8 Sent Done Done false
35662 9676 Acked Write 2 Sent Done Done false
35663 9677 Acked Read 5 Sent Done Done false
35664 9678 Acked Write 2 Sent Done Done false
35665 9679 Acked Write 9 Sent Done Done false
35666 9680 Acked Read 5 Sent Done Done false
35667 9681 Acked Read 2 Sent Done Done false
35668 9682 Acked Read 5 Sent Done Done false
35669 9683 Acked Read 5 Sent Done Done false
35670 9684 Acked Read 3 Sent Done Done false
35671 9685 Acked Read 8 Sent Done Done false
35672 9686 Acked Read 6 Sent Done Done false
35673 9687 Acked Flush 0 Sent Done Done false
35674 9688 Acked Write 4 Sent Done Done false
35675 9689 Acked Write 5 Sent Done Done false
35676 9690 Acked Read 8 Sent Done Done false
35677 9691 Acked Write 7 Sent Done Done false
35678 9692 Acked Write 10 Sent Done Done false
35679 9693 Acked Write 1 Sent Done Done false
35680 9694 Acked Read 1 Sent Done Done false
35681 9695 Acked Read 9 Sent Done Done false
35682 9696 Acked Read 2 Sent Done Done false
35683 9697 Acked Read 10 Sent Done Done false
35684 9698 Acked Write 1 Sent Done Done false
35685 9699 Acked Read 3 Sent Done Done false
35686 9700 Acked Read 8 Sent Done Done false
35687 9701 Acked Read 2 Sent Done Done false
35688 9702 Acked Read 8 Sent Done Done false
35689 9703 Acked Write 6 Sent Done Done false
35690 9704 Acked Write 1 Sent Done Done false
35691 9705 Acked Read 8 Sent Done Done false
35692 9706 Acked Read 10 Sent Done Done false
35693 9707 Acked Write 8 Sent Done Done false
35694 9708 Acked Read 6 Sent Done Done false
35695 9709 Acked Flush 0 Sent Done Done false
35696 9710 Acked Read 8 Sent Done Done false
35697 9711 Acked Read 10 Sent Done Done false
35698 9712 Acked Read 6 Sent Done Done false
35699 9713 Acked Write 7 Sent Done Done false
35700 9714 Acked Write 2 Sent Done Done false
35701 9715 Acked Write 7 Sent Done Done false
35702 9716 Acked Write 1 Sent Done Done false
35703 9717 Acked Read 10 Sent Done Done false
35704 9718 Acked Write 3 Sent Done Done false
35705 9719 Acked Read 3 Sent Done Done false
35706 9720 Acked Write 3 Sent Done Done false
35707 9721 Acked Read 3 Sent Done Done false
35708 9722 Acked Write 2 Sent Done Done false
35709 9723 Acked Write 8 Sent Done Done false
35710 9724 Acked Read 6 Sent Done Done false
35711 9725 Acked Write 9 Sent Done Done false
35712 9726 Acked Read 7 Sent Done Done false
35713 9727 Acked Read 3 Sent Done Done false
35714 9728 Acked Read 8 Sent Done Done false
35715 9729 Acked Write 1 Sent Done Done false
35716 9730 Acked Read 1 Sent Done Done false
35717 9731 Acked Read 8 Sent Done Done false
35718 9732 Acked Read 10 Sent Done Done false
35719 9733 Acked Read 10 Sent Done Done false
35720 9734 Acked Read 2 Sent Done Done false
35721 9735 Acked Write 5 Sent Done Done false
35722 9736 Acked Read 4 Sent Done Done false
35723 9737 Acked Write 9 Sent Done Done false
35724 9738 Acked Write 9 Sent Done Done false
35725 9739 Acked Write 6 Sent Done Done false
35726 9740 Acked Write 5 Sent Done Done false
35727 9741 Acked Read 6 Sent Done Done false
35728 9742 Acked Write 1 Sent Done Done false
35729 9743 Acked Read 10 Sent Done Done false
35730 9744 Acked Read 9 Sent Done Done false
35731 9745 Acked Read 4 Sent Done Done false
35732 9746 Acked Read 7 Sent Done Done false
35733 9747 Acked Read 2 Sent Done Done false
35734 9748 Acked Flush 0 Sent Done Done false
35735 9749 Acked Read 7 Sent Done Done false
35736 9750 Acked Write 5 Sent Done Done false
35737 9751 Acked Write 9 Sent Done Done false
35738 9752 Acked Write 2 Sent Done Done false
35739 9753 Acked Read 7 Sent Done Done false
35740 9754 Acked Read 9 Sent Done Done false
35741 9755 Acked Flush 0 Sent Done Done false
35742 STATES DS:0 DS:1 DS:2 TOTAL
35743 Sent 301 0 0 301
35744 Done 0 301 301 602
35745 Skipped 0 0 0 0
35746 Error 0 0 0 0
35747 Last Flush: 9454 9755 9755
35748 Downstairs last five completed: 9454 9453 9452 9451 9450
35749 Upstairs last five completed: 9755 9754 9753 9752 9751
35750 CLIENT: Up:0 ds:301 act:2
357512024-12-11T23:47:59.406ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
357522024-12-11T23:47:59.406ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
357532024-12-11T23:47:59.406ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
357542024-12-11T23:47:59.406ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
357552024-12-11T23:47:59.406ZINFOcrucible: client task is exiting = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
35756 [8810] got start action
35757 [8810] I am going to restart
35758 Starting downstairs at port 8810
35759 Make output file at "/tmp/dsc/downstairs-8810.txt"
35760 [8810][0] reports Starting
35761 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1229)
35762 [8810][0] reports Running
35763 ----------------------------------------------------------------
35764 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35765 States: Offline Active Active
35766 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35767 9455 Acked Read 2 Sent Done Done false
35768 9456 Acked Write 6 Sent Done Done false
35769 9457 Acked Read 9 Sent Done Done false
35770 9458 Acked Read 2 Sent Done Done false
35771 9459 Acked Flush 0 Sent Done Done false
35772 9460 Acked Write 2 Sent Done Done false
35773 9461 Acked Read 3 Sent Done Done false
35774 9462 Acked Read 9 Sent Done Done false
35775 9463 Acked Write 7 Sent Done Done false
35776 9464 Acked Write 5 Sent Done Done false
35777 9465 Acked Write 9 Sent Done Done false
35778 9466 Acked Read 3 Sent Done Done false
35779 9467 Acked Write 7 Sent Done Done false
35780 9468 Acked Flush 0 Sent Done Done false
35781 9469 Acked Read 6 Sent Done Done false
35782 9470 Acked Write 7 Sent Done Done false
35783 9471 Acked Write 6 Sent Done Done false
35784 9472 Acked Read 10 Sent Done Done false
35785 9473 Acked Read 1 Sent Done Done false
35786 9474 Acked Write 1 Sent Done Done false
35787 9475 Acked Write 5 Sent Done Done false
35788 9476 Acked Read 7 Sent Done Done false
35789 9477 Acked Read 9 Sent Done Done false
35790 9478 Acked Read 1 Sent Done Done false
35791 9479 Acked Write 2 Sent Done Done false
35792 9480 Acked Write 4 Sent Done Done false
35793 9481 Acked Write 8 Sent Done Done false
35794 9482 Acked Read 3 Sent Done Done false
35795 9483 Acked Read 7 Sent Done Done false
35796 9484 Acked Flush 0 Sent Done Done false
35797 9485 Acked Write 9 Sent Done Done false
35798 9486 Acked Write 9 Sent Done Done false
35799 9487 Acked Read 7 Sent Done Done false
35800 9488 Acked Read 8 Sent Done Done false
35801 9489 Acked Read 6 Sent Done Done false
35802 9490 Acked Write 6 Sent Done Done false
35803 9491 Acked Write 3 Sent Done Done false
35804 9492 Acked Read 9 Sent Done Done false
35805 9493 Acked Read 4 Sent Done Done false
35806 9494 Acked Write 8 Sent Done Done false
35807 9495 Acked Read 6 Sent Done Done false
35808 9496 Acked Read 9 Sent Done Done false
35809 9497 Acked Write 4 Sent Done Done false
35810 9498 Acked Write 9 Sent Done Done false
35811 9499 Acked Read 1 Sent Done Done false
35812 9500 Acked Read 1 Sent Done Done false
35813 9501 Acked Write 3 Sent Done Done false
35814 9502 Acked Flush 0 Sent Done Done false
35815 9503 Acked Read 5 Sent Done Done false
35816 9504 Acked Read 7 Sent Done Done false
35817 9505 Acked Write 6 Sent Done Done false
35818 9506 Acked Read 2 Sent Done Done false
35819 9507 Acked Write 2 Sent Done Done false
35820 9508 Acked Read 4 Sent Done Done false
35821 9509 Acked Flush 0 Sent Done Done false
35822 9510 Acked Write 5 Sent Done Done false
35823 9511 Acked Read 1 Sent Done Done false
35824 9512 Acked Read 6 Sent Done Done false
35825 9513 Acked Write 10 Sent Done Done false
35826 9514 Acked Write 6 Sent Done Done false
35827 9515 Acked Read 7 Sent Done Done false
35828 9516 Acked Read 5 Sent Done Done false
35829 9517 Acked Read 8 Sent Done Done false
35830 9518 Acked Write 7 Sent Done Done false
35831 9519 Acked Read 7 Sent Done Done false
35832 9520 Acked Write 5 Sent Done Done false
35833 9521 Acked Write 3 Sent Done Done false
35834 9522 Acked Flush 0 Sent Done Done false
35835 9523 Acked Write 8 Sent Done Done false
35836 9524 Acked Read 7 Sent Done Done false
35837 9525 Acked Write 7 Sent Done Done false
35838 9526 Acked Read 10 Sent Done Done false
35839 9527 Acked Write 9 Sent Done Done false
35840 9528 Acked Write 9 Sent Done Done false
35841 9529 Acked Read 6 Sent Done Done false
35842 9530 Acked Read 9 Sent Done Done false
35843 9531 Acked Read 4 Sent Done Done false
35844 9532 Acked Read 2 Sent Done Done false
35845 9533 Acked Read 4 Sent Done Done false
35846 9534 Acked Read 2 Sent Done Done false
35847 9535 Acked Write 1 Sent Done Done false
35848 9536 Acked Write 3 Sent Done Done false
35849 9537 Acked Flush 0 Sent Done Done false
35850 9538 Acked Write 1 Sent Done Done false
35851 9539 Acked Read 9 Sent Done Done false
35852 9540 Acked Write 4 Sent Done Done false
35853 9541 Acked Write 1 Sent Done Done false
35854 9542 Acked Read 10 Sent Done Done false
35855 9543 Acked Write 9 Sent Done Done false
35856 9544 Acked Read 1 Sent Done Done false
35857 9545 Acked Write 2 Sent Done Done false
35858 9546 Acked Flush 0 Sent Done Done false
35859 9547 Acked Read 9 Sent Done Done false
35860 9548 Acked Write 5 Sent Done Done false
35861 9549 Acked Write 5 Sent Done Done false
35862 9550 Acked Write 3 Sent Done Done false
35863 9551 Acked Write 8 Sent Done Done false
35864 9552 Acked Write 7 Sent Done Done false
35865 9553 Acked Read 2 Sent Done Done false
35866 9554 Acked Read 8 Sent Done Done false
35867 9555 Acked Read 7 Sent Done Done false
35868 9556 Acked Write 8 Sent Done Done false
35869 9557 Acked Flush 0 Sent Done Done false
35870 9558 Acked Read 7 Sent Done Done false
35871 9559 Acked Flush 0 Sent Done Done false
35872 9560 Acked Write 10 Sent Done Done false
35873 9561 Acked Write 1 Sent Done Done false
35874 9562 Acked Write 10 Sent Done Done false
35875 9563 Acked Read 2 Sent Done Done false
35876 9564 Acked Write 5 Sent Done Done false
35877 9565 Acked Read 6 Sent Done Done false
35878 9566 Acked Read 1 Sent Done Done false
35879 9567 Acked Read 8 Sent Done Done false
35880 9568 Acked Write 1 Sent Done Done false
35881 9569 Acked Read 9 Sent Done Done false
35882 9570 Acked Read 6 Sent Done Done false
35883 9571 Acked Write 6 Sent Done Done false
35884 9572 Acked Write 8 Sent Done Done false
35885 9573 Acked Read 9 Sent Done Done false
35886 9574 Acked Write 3 Sent Done Done false
35887 9575 Acked Write 1 Sent Done Done false
35888 9576 Acked Write 3 Sent Done Done false
35889 9577 Acked Write 4 Sent Done Done false
35890 9578 Acked Write 2 Sent Done Done false
35891 9579 Acked Read 4 Sent Done Done false
35892 9580 Acked Write 5 Sent Done Done false
35893 9581 Acked Write 1 Sent Done Done false
35894 9582 Acked Write 9 Sent Done Done false
35895 9583 Acked Read 9 Sent Done Done false
35896 9584 Acked Flush 0 Sent Done Done false
35897 9585 Acked Read 2 Sent Done Done false
35898 9586 Acked Read 5 Sent Done Done false
35899 9587 Acked Read 4 Sent Done Done false
35900 9588 Acked Flush 0 Sent Done Done false
35901 9589 Acked Read 3 Sent Done Done false
35902 9590 Acked Flush 0 Sent Done Done false
35903 9591 Acked Write 6 Sent Done Done false
35904 9592 Acked Read 6 Sent Done Done false
35905 9593 Acked Read 5 Sent Done Done false
35906 9594 Acked Flush 0 Sent Done Done false
35907 9595 Acked Write 1 Sent Done Done false
35908 9596 Acked Flush 0 Sent Done Done false
35909 9597 Acked Flush 0 Sent Done Done false
35910 9598 Acked Read 9 Sent Done Done false
35911 9599 Acked Read 8 Sent Done Done false
35912 9600 Acked Write 1 Sent Done Done false
35913 9601 Acked Flush 0 Sent Done Done false
35914 9602 Acked Write 2 Sent Done Done false
35915 9603 Acked Read 8 Sent Done Done false
35916 9604 Acked Read 8 Sent Done Done false
35917 9605 Acked Write 6 Sent Done Done false
35918 9606 Acked Write 4 Sent Done Done false
35919 9607 Acked Flush 0 Sent Done Done false
35920 9608 Acked Write 9 Sent Done Done false
35921 9609 Acked Write 8 Sent Done Done false
35922 9610 Acked Flush 0 Sent Done Done false
35923 9611 Acked Read 1 Sent Done Done false
35924 9612 Acked Write 6 Sent Done Done false
35925 9613 Acked Write 1 Sent Done Done false
35926 9614 Acked Write 10 Sent Done Done false
35927 9615 Acked Read 9 Sent Done Done false
35928 9616 Acked Write 4 Sent Done Done false
35929 9617 Acked Write 6 Sent Done Done false
35930 9618 Acked Read 8 Sent Done Done false
35931 9619 Acked Write 10 Sent Done Done false
35932 9620 Acked Flush 0 Sent Done Done false
35933 9621 Acked Write 10 Sent Done Done false
35934 9622 Acked Read 4 Sent Done Done false
35935 9623 Acked Write 10 Sent Done Done false
35936 9624 Acked Read 9 Sent Done Done false
35937 9625 Acked Write 7 Sent Done Done false
35938 9626 Acked Read 3 Sent Done Done false
35939 9627 Acked Read 9 Sent Done Done false
35940 9628 Acked Read 3 Sent Done Done false
35941 9629 Acked Read 4 Sent Done Done false
35942 9630 Acked Read 7 Sent Done Done false
35943 9631 Acked Read 10 Sent Done Done false
35944 9632 Acked Write 5 Sent Done Done false
35945 9633 Acked Read 10 Sent Done Done false
35946 9634 Acked Write 4 Sent Done Done false
35947 9635 Acked Flush 0 Sent Done Done false
35948 9636 Acked Read 3 Sent Done Done false
35949 9637 Acked Flush 0 Sent Done Done false
35950 9638 Acked Write 7 Sent Done Done false
35951 9639 Acked Write 5 Sent Done Done false
35952 9640 Acked Write 9 Sent Done Done false
35953 9641 Acked Flush 0 Sent Done Done false
35954 9642 Acked Write 6 Sent Done Done false
35955 9643 Acked Read 9 Sent Done Done false
35956 9644 Acked Read 3 Sent Done Done false
35957 9645 Acked Read 1 Sent Done Done false
35958 9646 Acked Write 3 Sent Done Done false
35959 9647 Acked Flush 0 Sent Done Done false
35960 9648 Acked Read 10 Sent Done Done false
35961 9649 Acked Read 1 Sent Done Done false
35962 9650 Acked Write 10 Sent Done Done false
35963 9651 Acked Read 10 Sent Done Done false
35964 9652 Acked Write 3 Sent Done Done false
35965 9653 Acked Read 6 Sent Done Done false
35966 9654 Acked Write 2 Sent Done Done false
35967 9655 Acked Read 4 Sent Done Done false
35968 9656 Acked Write 7 Sent Done Done false
35969 9657 Acked Write 9 Sent Done Done false
35970 9658 Acked Read 6 Sent Done Done false
35971 9659 Acked Write 7 Sent Done Done false
35972 9660 Acked Flush 0 Sent Done Done false
35973 9661 Acked Read 9 Sent Done Done false
35974 9662 Acked Write 4 Sent Done Done false
35975 9663 Acked Write 8 Sent Done Done false
35976 9664 Acked Read 1 Sent Done Done false
35977 9665 Acked Write 9 Sent Done Done false
35978 9666 Acked Write 3 Sent Done Done false
35979 9667 Acked Read 6 Sent Done Done false
35980 9668 Acked Read 6 Sent Done Done false
35981 9669 Acked Write 3 Sent Done Done false
35982 9670 Acked Write 10 Sent Done Done false
35983 9671 Acked Read 7 Sent Done Done false
35984 9672 Acked Read 1 Sent Done Done false
35985 9673 Acked Flush 0 Sent Done Done false
35986 9674 Acked Write 7 Sent Done Done false
35987 9675 Acked Write 8 Sent Done Done false
35988 9676 Acked Write 2 Sent Done Done false
35989 9677 Acked Read 5 Sent Done Done false
35990 9678 Acked Write 2 Sent Done Done false
35991 9679 Acked Write 9 Sent Done Done false
35992 9680 Acked Read 5 Sent Done Done false
35993 9681 Acked Read 2 Sent Done Done false
35994 9682 Acked Read 5 Sent Done Done false
35995 9683 Acked Read 5 Sent Done Done false
35996 9684 Acked Read 3 Sent Done Done false
35997 9685 Acked Read 8 Sent Done Done false
35998 9686 Acked Read 6 Sent Done Done false
35999 9687 Acked Flush 0 Sent Done Done false
36000 9688 Acked Write 4 Sent Done Done false
36001 9689 Acked Write 5 Sent Done Done false
36002 9690 Acked Read 8 Sent Done Done false
36003 9691 Acked Write 7 Sent Done Done false
36004 9692 Acked Write 10 Sent Done Done false
36005 9693 Acked Write 1 Sent Done Done false
36006 9694 Acked Read 1 Sent Done Done false
36007 9695 Acked Read 9 Sent Done Done false
36008 9696 Acked Read 2 Sent Done Done false
36009 9697 Acked Read 10 Sent Done Done false
36010 9698 Acked Write 1 Sent Done Done false
36011 9699 Acked Read 3 Sent Done Done false
36012 9700 Acked Read 8 Sent Done Done false
36013 9701 Acked Read 2 Sent Done Done false
36014 9702 Acked Read 8 Sent Done Done false
36015 9703 Acked Write 6 Sent Done Done false
36016 9704 Acked Write 1 Sent Done Done false
36017 9705 Acked Read 8 Sent Done Done false
36018 9706 Acked Read 10 Sent Done Done false
36019 9707 Acked Write 8 Sent Done Done false
36020 9708 Acked Read 6 Sent Done Done false
36021 9709 Acked Flush 0 Sent Done Done false
36022 9710 Acked Read 8 Sent Done Done false
36023 9711 Acked Read 10 Sent Done Done false
36024 9712 Acked Read 6 Sent Done Done false
36025 9713 Acked Write 7 Sent Done Done false
36026 9714 Acked Write 2 Sent Done Done false
36027 9715 Acked Write 7 Sent Done Done false
36028 9716 Acked Write 1 Sent Done Done false
36029 9717 Acked Read 10 Sent Done Done false
36030 9718 Acked Write 3 Sent Done Done false
36031 9719 Acked Read 3 Sent Done Done false
36032 9720 Acked Write 3 Sent Done Done false
36033 9721 Acked Read 3 Sent Done Done false
36034 9722 Acked Write 2 Sent Done Done false
36035 9723 Acked Write 8 Sent Done Done false
36036 9724 Acked Read 6 Sent Done Done false
36037 9725 Acked Write 9 Sent Done Done false
36038 9726 Acked Read 7 Sent Done Done false
36039 9727 Acked Read 3 Sent Done Done false
36040 9728 Acked Read 8 Sent Done Done false
36041 9729 Acked Write 1 Sent Done Done false
36042 9730 Acked Read 1 Sent Done Done false
36043 9731 Acked Read 8 Sent Done Done false
36044 9732 Acked Read 10 Sent Done Done false
36045 9733 Acked Read 10 Sent Done Done false
36046 9734 Acked Read 2 Sent Done Done false
36047 9735 Acked Write 5 Sent Done Done false
36048 9736 Acked Read 4 Sent Done Done false
36049 9737 Acked Write 9 Sent Done Done false
36050 9738 Acked Write 9 Sent Done Done false
36051 9739 Acked Write 6 Sent Done Done false
36052 9740 Acked Write 5 Sent Done Done false
36053 9741 Acked Read 6 Sent Done Done false
36054 9742 Acked Write 1 Sent Done Done false
36055 9743 Acked Read 10 Sent Done Done false
36056 9744 Acked Read 9 Sent Done Done false
36057 9745 Acked Read 4 Sent Done Done false
36058 9746 Acked Read 7 Sent Done Done false
36059 9747 Acked Read 2 Sent Done Done false
36060 9748 Acked Flush 0 Sent Done Done false
36061 9749 Acked Read 7 Sent Done Done false
36062 9750 Acked Write 5 Sent Done Done false
36063 9751 Acked Write 9 Sent Done Done false
36064 9752 Acked Write 2 Sent Done Done false
36065 9753 Acked Read 7 Sent Done Done false
36066 9754 Acked Read 9 Sent Done Done false
36067 9755 Acked Flush 0 Sent Done Done false
36068 STATES DS:0 DS:1 DS:2 TOTAL
36069 Sent 301 0 0 301
36070 Done 0 301 301 602
36071 Skipped 0 0 0 0
36072 Error 0 0 0 0
36073 Last Flush: 9454 9755 9755
36074 Downstairs last five completed: 9454 9453 9452 9451 9450
36075 Upstairs last five completed: 9755 9754 9753 9752 9751
36076 CLIENT: Up:0 ds:301 act:2
36077 ----------------------------------------------------------------
36078 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
36079 States: Offline Active Active
36080 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36081 9455 Acked Read 2 Sent Done Done false
36082 9456 Acked Write 6 Sent Done Done false
36083 9457 Acked Read 9 Sent Done Done false
36084 9458 Acked Read 2 Sent Done Done false
36085 9459 Acked Flush 0 Sent Done Done false
36086 9460 Acked Write 2 Sent Done Done false
36087 9461 Acked Read 3 Sent Done Done false
36088 9462 Acked Read 9 Sent Done Done false
36089 9463 Acked Write 7 Sent Done Done false
36090 9464 Acked Write 5 Sent Done Done false
36091 9465 Acked Write 9 Sent Done Done false
36092 9466 Acked Read 3 Sent Done Done false
36093 9467 Acked Write 7 Sent Done Done false
36094 9468 Acked Flush 0 Sent Done Done false
36095 9469 Acked Read 6 Sent Done Done false
36096 9470 Acked Write 7 Sent Done Done false
36097 9471 Acked Write 6 Sent Done Done false
36098 9472 Acked Read 10 Sent Done Done false
36099 9473 Acked Read 1 Sent Done Done false
36100 9474 Acked Write 1 Sent Done Done false
36101 9475 Acked Write 5 Sent Done Done false
36102 9476 Acked Read 7 Sent Done Done false
36103 9477 Acked Read 9 Sent Done Done false
36104 9478 Acked Read 1 Sent Done Done false
36105 9479 Acked Write 2 Sent Done Done false
36106 9480 Acked Write 4 Sent Done Done false
36107 9481 Acked Write 8 Sent Done Done false
36108 9482 Acked Read 3 Sent Done Done false
36109 9483 Acked Read 7 Sent Done Done false
36110 9484 Acked Flush 0 Sent Done Done false
36111 9485 Acked Write 9 Sent Done Done false
36112 9486 Acked Write 9 Sent Done Done false
36113 9487 Acked Read 7 Sent Done Done false
36114 9488 Acked Read 8 Sent Done Done false
36115 9489 Acked Read 6 Sent Done Done false
36116 9490 Acked Write 6 Sent Done Done false
36117 9491 Acked Write 3 Sent Done Done false
36118 9492 Acked Read 9 Sent Done Done false
36119 9493 Acked Read 4 Sent Done Done false
36120 9494 Acked Write 8 Sent Done Done false
36121 9495 Acked Read 6 Sent Done Done false
36122 9496 Acked Read 9 Sent Done Done false
36123 9497 Acked Write 4 Sent Done Done false
36124 9498 Acked Write 9 Sent Done Done false
36125 9499 Acked Read 1 Sent Done Done false
36126 9500 Acked Read 1 Sent Done Done false
36127 9501 Acked Write 3 Sent Done Done false
36128 9502 Acked Flush 0 Sent Done Done false
36129 9503 Acked Read 5 Sent Done Done false
36130 9504 Acked Read 7 Sent Done Done false
36131 9505 Acked Write 6 Sent Done Done false
36132 9506 Acked Read 2 Sent Done Done false
36133 9507 Acked Write 2 Sent Done Done false
36134 9508 Acked Read 4 Sent Done Done false
36135 9509 Acked Flush 0 Sent Done Done false
36136 9510 Acked Write 5 Sent Done Done false
36137 9511 Acked Read 1 Sent Done Done false
36138 9512 Acked Read 6 Sent Done Done false
36139 9513 Acked Write 10 Sent Done Done false
36140 9514 Acked Write 6 Sent Done Done false
36141 9515 Acked Read 7 Sent Done Done false
36142 9516 Acked Read 5 Sent Done Done false
36143 9517 Acked Read 8 Sent Done Done false
36144 9518 Acked Write 7 Sent Done Done false
36145 9519 Acked Read 7 Sent Done Done false
36146 9520 Acked Write 5 Sent Done Done false
36147 9521 Acked Write 3 Sent Done Done false
36148 9522 Acked Flush 0 Sent Done Done false
36149 9523 Acked Write 8 Sent Done Done false
36150 9524 Acked Read 7 Sent Done Done false
36151 9525 Acked Write 7 Sent Done Done false
36152 9526 Acked Read 10 Sent Done Done false
36153 9527 Acked Write 9 Sent Done Done false
36154 9528 Acked Write 9 Sent Done Done false
36155 9529 Acked Read 6 Sent Done Done false
36156 9530 Acked Read 9 Sent Done Done false
36157 9531 Acked Read 4 Sent Done Done false
36158 9532 Acked Read 2 Sent Done Done false
36159 9533 Acked Read 4 Sent Done Done false
36160 9534 Acked Read 2 Sent Done Done false
36161 9535 Acked Write 1 Sent Done Done false
36162 9536 Acked Write 3 Sent Done Done false
36163 9537 Acked Flush 0 Sent Done Done false
36164 9538 Acked Write 1 Sent Done Done false
36165 9539 Acked Read 9 Sent Done Done false
36166 9540 Acked Write 4 Sent Done Done false
36167 9541 Acked Write 1 Sent Done Done false
36168 9542 Acked Read 10 Sent Done Done false
36169 9543 Acked Write 9 Sent Done Done false
36170 9544 Acked Read 1 Sent Done Done false
36171 9545 Acked Write 2 Sent Done Done false
36172 9546 Acked Flush 0 Sent Done Done false
36173 9547 Acked Read 9 Sent Done Done false
36174 9548 Acked Write 5 Sent Done Done false
36175 9549 Acked Write 5 Sent Done Done false
36176 9550 Acked Write 3 Sent Done Done false
36177 9551 Acked Write 8 Sent Done Done false
36178 9552 Acked Write 7 Sent Done Done false
36179 9553 Acked Read 2 Sent Done Done false
36180 9554 Acked Read 8 Sent Done Done false
36181 9555 Acked Read 7 Sent Done Done false
36182 9556 Acked Write 8 Sent Done Done false
36183 9557 Acked Flush 0 Sent Done Done false
36184 9558 Acked Read 7 Sent Done Done false
36185 9559 Acked Flush 0 Sent Done Done false
36186 9560 Acked Write 10 Sent Done Done false
36187 9561 Acked Write 1 Sent Done Done false
36188 9562 Acked Write 10 Sent Done Done false
36189 9563 Acked Read 2 Sent Done Done false
36190 9564 Acked Write 5 Sent Done Done false
36191 9565 Acked Read 6 Sent Done Done false
36192 9566 Acked Read 1 Sent Done Done false
36193 9567 Acked Read 8 Sent Done Done false
36194 9568 Acked Write 1 Sent Done Done false
36195 9569 Acked Read 9 Sent Done Done false
36196 9570 Acked Read 6 Sent Done Done false
36197 9571 Acked Write 6 Sent Done Done false
36198 9572 Acked Write 8 Sent Done Done false
36199 9573 Acked Read 9 Sent Done Done false
36200 9574 Acked Write 3 Sent Done Done false
36201 9575 Acked Write 1 Sent Done Done false
36202 9576 Acked Write 3 Sent Done Done false
36203 9577 Acked Write 4 Sent Done Done false
36204 9578 Acked Write 2 Sent Done Done false
36205 9579 Acked Read 4 Sent Done Done false
36206 9580 Acked Write 5 Sent Done Done false
36207 9581 Acked Write 1 Sent Done Done false
36208 9582 Acked Write 9 Sent Done Done false
36209 9583 Acked Read 9 Sent Done Done false
36210 9584 Acked Flush 0 Sent Done Done false
36211 9585 Acked Read 2 Sent Done Done false
36212 9586 Acked Read 5 Sent Done Done false
36213 9587 Acked Read 4 Sent Done Done false
36214 9588 Acked Flush 0 Sent Done Done false
36215 9589 Acked Read 3 Sent Done Done false
36216 9590 Acked Flush 0 Sent Done Done false
36217 9591 Acked Write 6 Sent Done Done false
36218 9592 Acked Read 6 Sent Done Done false
36219 9593 Acked Read 5 Sent Done Done false
36220 9594 Acked Flush 0 Sent Done Done false
36221 9595 Acked Write 1 Sent Done Done false
36222 9596 Acked Flush 0 Sent Done Done false
36223 9597 Acked Flush 0 Sent Done Done false
36224 9598 Acked Read 9 Sent Done Done false
36225 9599 Acked Read 8 Sent Done Done false
36226 9600 Acked Write 1 Sent Done Done false
36227 9601 Acked Flush 0 Sent Done Done false
36228 9602 Acked Write 2 Sent Done Done false
36229 9603 Acked Read 8 Sent Done Done false
36230 9604 Acked Read 8 Sent Done Done false
36231 9605 Acked Write 6 Sent Done Done false
36232 9606 Acked Write 4 Sent Done Done false
36233 9607 Acked Flush 0 Sent Done Done false
36234 9608 Acked Write 9 Sent Done Done false
36235 9609 Acked Write 8 Sent Done Done false
36236 9610 Acked Flush 0 Sent Done Done false
36237 9611 Acked Read 1 Sent Done Done false
36238 9612 Acked Write 6 Sent Done Done false
36239 9613 Acked Write 1 Sent Done Done false
36240 9614 Acked Write 10 Sent Done Done false
36241 9615 Acked Read 9 Sent Done Done false
36242 9616 Acked Write 4 Sent Done Done false
36243 9617 Acked Write 6 Sent Done Done false
36244 9618 Acked Read 8 Sent Done Done false
36245 9619 Acked Write 10 Sent Done Done false
36246 9620 Acked Flush 0 Sent Done Done false
36247 9621 Acked Write 10 Sent Done Done false
36248 9622 Acked Read 4 Sent Done Done false
36249 9623 Acked Write 10 Sent Done Done false
36250 9624 Acked Read 9 Sent Done Done false
36251 9625 Acked Write 7 Sent Done Done false
36252 9626 Acked Read 3 Sent Done Done false
36253 9627 Acked Read 9 Sent Done Done false
36254 9628 Acked Read 3 Sent Done Done false
36255 9629 Acked Read 4 Sent Done Done false
36256 9630 Acked Read 7 Sent Done Done false
36257 9631 Acked Read 10 Sent Done Done false
36258 9632 Acked Write 5 Sent Done Done false
36259 9633 Acked Read 10 Sent Done Done false
36260 9634 Acked Write 4 Sent Done Done false
36261 9635 Acked Flush 0 Sent Done Done false
36262 9636 Acked Read 3 Sent Done Done false
36263 9637 Acked Flush 0 Sent Done Done false
36264 9638 Acked Write 7 Sent Done Done false
36265 9639 Acked Write 5 Sent Done Done false
36266 9640 Acked Write 9 Sent Done Done false
36267 9641 Acked Flush 0 Sent Done Done false
36268 9642 Acked Write 6 Sent Done Done false
36269 9643 Acked Read 9 Sent Done Done false
36270 9644 Acked Read 3 Sent Done Done false
36271 9645 Acked Read 1 Sent Done Done false
36272 9646 Acked Write 3 Sent Done Done false
36273 9647 Acked Flush 0 Sent Done Done false
36274 9648 Acked Read 10 Sent Done Done false
36275 9649 Acked Read 1 Sent Done Done false
36276 9650 Acked Write 10 Sent Done Done false
36277 9651 Acked Read 10 Sent Done Done false
36278 9652 Acked Write 3 Sent Done Done false
36279 9653 Acked Read 6 Sent Done Done false
36280 9654 Acked Write 2 Sent Done Done false
36281 9655 Acked Read 4 Sent Done Done false
36282 9656 Acked Write 7 Sent Done Done false
36283 9657 Acked Write 9 Sent Done Done false
36284 9658 Acked Read 6 Sent Done Done false
36285 9659 Acked Write 7 Sent Done Done false
36286 9660 Acked Flush 0 Sent Done Done false
36287 9661 Acked Read 9 Sent Done Done false
36288 9662 Acked Write 4 Sent Done Done false
36289 9663 Acked Write 8 Sent Done Done false
36290 9664 Acked Read 1 Sent Done Done false
36291 9665 Acked Write 9 Sent Done Done false
36292 9666 Acked Write 3 Sent Done Done false
36293 9667 Acked Read 6 Sent Done Done false
36294 9668 Acked Read 6 Sent Done Done false
36295 9669 Acked Write 3 Sent Done Done false
36296 9670 Acked Write 10 Sent Done Done false
36297 9671 Acked Read 7 Sent Done Done false
36298 9672 Acked Read 1 Sent Done Done false
36299 9673 Acked Flush 0 Sent Done Done false
36300 9674 Acked Write 7 Sent Done Done false
36301 9675 Acked Write 8 Sent Done Done false
36302 9676 Acked Write 2 Sent Done Done false
36303 9677 Acked Read 5 Sent Done Done false
36304 9678 Acked Write 2 Sent Done Done false
36305 9679 Acked Write 9 Sent Done Done false
36306 9680 Acked Read 5 Sent Done Done false
36307 9681 Acked Read 2 Sent Done Done false
36308 9682 Acked Read 5 Sent Done Done false
36309 9683 Acked Read 5 Sent Done Done false
36310 9684 Acked Read 3 Sent Done Done false
36311 9685 Acked Read 8 Sent Done Done false
36312 9686 Acked Read 6 Sent Done Done false
36313 9687 Acked Flush 0 Sent Done Done false
36314 9688 Acked Write 4 Sent Done Done false
36315 9689 Acked Write 5 Sent Done Done false
36316 9690 Acked Read 8 Sent Done Done false
36317 9691 Acked Write 7 Sent Done Done false
36318 9692 Acked Write 10 Sent Done Done false
36319 9693 Acked Write 1 Sent Done Done false
36320 9694 Acked Read 1 Sent Done Done false
36321 9695 Acked Read 9 Sent Done Done false
36322 9696 Acked Read 2 Sent Done Done false
36323 9697 Acked Read 10 Sent Done Done false
36324 9698 Acked Write 1 Sent Done Done false
36325 9699 Acked Read 3 Sent Done Done false
36326 9700 Acked Read 8 Sent Done Done false
36327 9701 Acked Read 2 Sent Done Done false
36328 9702 Acked Read 8 Sent Done Done false
36329 9703 Acked Write 6 Sent Done Done false
36330 9704 Acked Write 1 Sent Done Done false
36331 9705 Acked Read 8 Sent Done Done false
36332 9706 Acked Read 10 Sent Done Done false
36333 9707 Acked Write 8 Sent Done Done false
36334 9708 Acked Read 6 Sent Done Done false
36335 9709 Acked Flush 0 Sent Done Done false
36336 9710 Acked Read 8 Sent Done Done false
36337 9711 Acked Read 10 Sent Done Done false
36338 9712 Acked Read 6 Sent Done Done false
36339 9713 Acked Write 7 Sent Done Done false
36340 9714 Acked Write 2 Sent Done Done false
36341 9715 Acked Write 7 Sent Done Done false
36342 9716 Acked Write 1 Sent Done Done false
36343 9717 Acked Read 10 Sent Done Done false
36344 9718 Acked Write 3 Sent Done Done false
36345 9719 Acked Read 3 Sent Done Done false
36346 9720 Acked Write 3 Sent Done Done false
36347 9721 Acked Read 3 Sent Done Done false
36348 9722 Acked Write 2 Sent Done Done false
36349 9723 Acked Write 8 Sent Done Done false
36350 9724 Acked Read 6 Sent Done Done false
36351 9725 Acked Write 9 Sent Done Done false
36352 9726 Acked Read 7 Sent Done Done false
36353 9727 Acked Read 3 Sent Done Done false
36354 9728 Acked Read 8 Sent Done Done false
36355 9729 Acked Write 1 Sent Done Done false
36356 9730 Acked Read 1 Sent Done Done false
36357 9731 Acked Read 8 Sent Done Done false
36358 9732 Acked Read 10 Sent Done Done false
36359 9733 Acked Read 10 Sent Done Done false
36360 9734 Acked Read 2 Sent Done Done false
36361 9735 Acked Write 5 Sent Done Done false
36362 9736 Acked Read 4 Sent Done Done false
36363 9737 Acked Write 9 Sent Done Done false
36364 9738 Acked Write 9 Sent Done Done false
36365 9739 Acked Write 6 Sent Done Done false
36366 9740 Acked Write 5 Sent Done Done false
36367 9741 Acked Read 6 Sent Done Done false
36368 9742 Acked Write 1 Sent Done Done false
36369 9743 Acked Read 10 Sent Done Done false
36370 9744 Acked Read 9 Sent Done Done false
36371 9745 Acked Read 4 Sent Done Done false
36372 9746 Acked Read 7 Sent Done Done false
36373 9747 Acked Read 2 Sent Done Done false
36374 9748 Acked Flush 0 Sent Done Done false
36375 9749 Acked Read 7 Sent Done Done false
36376 9750 Acked Write 5 Sent Done Done false
36377 9751 Acked Write 9 Sent Done Done false
36378 9752 Acked Write 2 Sent Done Done false
36379 9753 Acked Read 7 Sent Done Done false
36380 9754 Acked Read 9 Sent Done Done false
36381 9755 Acked Flush 0 Sent Done Done false
36382 STATES DS:0 DS:1 DS:2 TOTAL
36383 Sent 301 0 0 301
36384 Done 0 301 301 602
36385 Skipped 0 0 0 0
36386 Error 0 0 0 0
36387 Last Flush: 9454 9755 9755
36388 Downstairs last five completed: 9454 9453 9452 9451 9450
36389 Upstairs last five completed: 9755 9754 9753 9752 9751
36390 CLIENT: Up:0 ds:301 act:2
363912024-12-11T23:48:09.407ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363922024-12-11T23:48:09.407ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51444) = io task client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363932024-12-11T23:48:09.409ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363942024-12-11T23:48:09.409ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363952024-12-11T23:48:09.409ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363962024-12-11T23:48:09.409ZINFOcrucible: send last flush ID to this DS: 9454 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363972024-12-11T23:48:09.409ZINFOcrucible: Replied this last flush ID: 9454 = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363982024-12-11T23:48:09.409ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
363992024-12-11T23:48:09.409ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
364002024-12-11T23:48:09.409ZINFOcrucible: [0] client re-new 301 jobs since flush 9454 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
364012024-12-11T23:48:09.410ZINFOcrucible: [0] Replayed 301 jobs since flush: 9454 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
36402 ----------------------------------------------------------------
36403 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
36404 Downstairs last five completed: 9755 9754 9753 9752 9751
36405 Upstairs last five completed: 9755 9754 9753 9752 9751
36406 CLIENT: Up:0 ds:0 act:3
36407 Replay: All jobs finished
36408 Main task has work to do, go find it
36409 got dsc Stop(2)
36410 stop 2
36411 Dec 11 23:48:09.652 INFO request completed, latency_us: 339, response_code[8830] Got stop action so:false kr:false
36412 : 204, uri: /stop/cid/2, method: POST, req_id: 8e3e74d3-14ef-4611-aee0-2b16f21ce02a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
364132024-12-11T23:48:09.653ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
364142024-12-11T23:48:09.653ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
364152024-12-11T23:48:09.653ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
364162024-12-11T23:48:09.653ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
364172024-12-11T23:48:09.653ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
364182024-12-11T23:48:09.653ZINFOcrucible: client task is exiting = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
36419 Dec 11 23:48:09.653 INFO request completed, latency_us: 173, response_code: 200, uri: /state/cid/2, method: GET, req_id: 5d170535-6a7b-4a8a-8d5d-2ffd362e381a, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
36420 [8830] Exited with: ExitStatus(unix_wait_status(9))
36421 [8830][2] reports Exit
36422 Dec 11 23:48:13.655 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/2, method: GET, req_id: 5ef8bee2-1343-491b-b975-731b13f04210, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
36423 001/300 Read block 4230 len 16384
36424 002/300 Read block 4065 len 40960
36425 003/300 Write block 1142 len 24576 data: 5 5 4 7 4 7
36426 004/300 Read block 532 len 28672
36427 005/300 Read block 3310 len 36864
36428 006/300 Write block 2331 len 16384 data: 8 8 7 11
36429 007/300 Read block 941 len 8192
36430 008/300 Read block 1820 len 20480
36431 009/300 Write block 2287 len 36864 data: 4 4 4 5 6 7 7 5 5
36432 010/300 Read block 2422 len 16384
36433 011/300 Read block 3585 len 12288
36434 012/300 Write block 3073 len 36864 data: 4 6 4 3 4 5 4 3 4
36435 013/300 Write block 3277 len 40960 data: 8 7 9 9 7 8 7 8 7 7
36436 014/300 Write block 2224 len 36864 data: 8 8 8 8 7 7 7 6 6
36437 015/300 Write block 4153 len 24576 data: 6 7 8 6 6 5
36438 016/300 Flush
36439 017/300 Read block 2539 len 12288
36440 018/300 Read block 452 len 16384
36441 019/300 Write block 2414 len 4096 data: 5
36442 020/300 Read block 3731 len 24576
36443 021/300 Read block 2803 len 28672
36444 022/300 Read block 911 len 40960
36445 023/300 Read block 199 len 40960
36446 024/300 Write block 4955 len 24576 data: 6 8 8 9 7 4
36447 025/300 Read block 2066 len 28672
36448 026/300 Read block 601 len 20480
36449 027/300 Read block 1230 len 4096
36450 028/300 Read block 3179 len 12288
36451 029/300 Read block 1952 len 24576
36452 030/300 Write block 2775 len 16384 data: 8 7 6 5
36453 031/300 Read block 224 len 28672
36454 032/300 Flush
36455 033/300 Read block 3439 len 12288
36456 034/300 Read block 2600 len 24576
36457 035/300 Flush
36458 036/300 Write block 846 len 20480 data: 7 8 6 6 6
36459 037/300 Read block 4152 len 20480
36460 038/300 Flush
36461 039/300 Read block 4866 len 32768
36462 040/300 Write block 1670 len 20480 data: 6 6 5 4 3
36463 041/300 Read block 2078 len 12288
36464 042/300 Flush
36465 043/300 Read block 2001 len 24576
36466 044/300 Flush
36467 045/300 Read block 4885 len 4096
36468 046/300 Read block 684 len 40960
36469 047/300 Write block 1075 len 36864 data: 5 6 7 7 5 5 6 7 6
36470 048/300 Flush
36471 049/300 Write block 3443 len 28672 data: 5 4 6 5 4 4 6
36472 050/300 Write block 737 len 24576 data: 5 6 6 7 6 6
36473 051/300 Write block 2356 len 4096 data: 7
36474 052/300 Read block 1534 len 36864
36475 053/300 Write block 4559 len 8192 data: 6 7
36476 054/300 Write block 2361 len 4096 data: 7
36477 055/300 Read block 1317 len 12288
36478 056/300 Read block 1045 len 20480
36479 057/300 Write block 4391 len 20480 data: 7 6 5 7 6
36480 058/300 Write block 3633 len 28672 data: 7 6 4 5 5 6 5
36481 059/300 Write block 4658 len 8192 data: 4 5
36482 060/300 Write block 2365 len 40960 data: 7 7 4 4 4 5 4 4 4 4
36483 061/300 Read block 536 len 4096
36484 062/300 Read block 2005 len 32768
36485 063/300 Read block 2030 len 40960
36486 064/300 Read block 3804 len 20480
36487 065/300 Write block 423 len 24576 data: 3 3 3 4 4 5
36488 066/300 Write block 4789 len 8192 data: 5 5
36489 067/300 Read block 2450 len 40960
36490 068/300 Read block 877 len 36864
36491 069/300 Read block 2042 len 16384
36492 070/300 Write block 2973 len 40960 data: 3 3 3 5 5 5 5 6 8 7
36493 071/300 Flush
36494 072/300 Write block 4124 len 12288 data: 11 11 8
36495 073/300 Write block 3780 len 4096 data: 5
36496 074/300 Read block 23 len 4096
36497 075/300 Read block 2948 len 20480
36498 076/300 Read block 2632 len 8192
36499 077/300 Read block 3700 len 4096
36500 078/300 Read block 1838 len 24576
36501 079/300 Write block 3643 len 8192 data: 6 6
36502 080/300 Write block 4077 len 12288 data: 4 4 4
36503 081/300 Write block 2030 len 20480 data: 8 6 7 8 7
36504 082/300 Read block 1512 len 12288
36505 083/300 Read block 3205 len 16384
36506 084/300 Read block 60 len 32768
36507 085/300 Read block 3607 len 16384
36508 086/300 Read block 4416 len 28672
36509 087/300 Read block 4440 len 36864
36510 088/300 Write block 1582 len 36864 data: 7 7 6 5 4 4 4 5 5
36511 089/300 Read block 3667 len 32768
36512 090/300 Write block 2546 len 12288 data: 10 9 8
36513 091/300 Read block 1711 len 16384
36514 092/300 Read block 141 len 20480
36515 093/300 Read block 4032 len 36864
36516 094/300 Write block 1944 len 32768 data: 7 6 6 5 6 8 7 7
36517 095/300 Write block 1800 len 12288 data: 3 3 4
36518 096/300 Read block 2012 len 20480
36519 097/300 Write block 3351 len 24576 data: 4 2 3 4 4 4
36520 098/300 Write block 3893 len 24576 data: 8 9 9 9 9 10
36521 099/300 Read block 2814 len 12288
36522 100/300 Read block 2172 len 20480
36523 101/300 Write block 4562 len 12288 data: 8 8 7
36524 102/300 Write block 2543 len 36864 data: 6 8 8 11 10 9 9 7 6
36525 103/300 Write block 1448 len 36864 data: 7 6 6 7 9 7 6 6 8
36526 104/300 Write block 2954 len 24576 data: 5 5 4 5 6 7
36527 105/300 Write block 4750 len 12288 data: 6 7 7
36528 106/300 Write block 2109 len 8192 data: 6 6
36529 107/300 Read block 1036 len 8192
36530 108/300 Read block 1135 len 24576
36531 109/300 Read block 4842 len 32768
36532 110/300 Read block 3225 len 32768
36533 111/300 Write block 345 len 4096 data: 12
36534 112/300 Read block 630 len 16384
36535 113/300 Read block 3797 len 8192
36536 114/300 Write block 4766 len 4096 data: 6
36537 115/300 Read block 3921 len 24576
36538 116/300 Write block 917 len 32768 data: 4 4 4 5 5 6 4 4
36539 117/300 Read block 3267 len 8192
36540 118/300 Flush
36541 119/300 Flush
36542 120/300 Write block 2935 len 8192 data: 7 8
36543 121/300 Write block 1291 len 4096 data: 9
36544 122/300 Read block 531 len 12288
36545 123/300 Read block 1017 len 40960
36546 124/300 Read block 3793 len 40960
36547 125/300 Write block 1332 len 12288 data: 4 4 5
36548 126/300 Read block 787 len 20480
36549 127/300 Read block 1907 len 32768
36550 128/300 Read block 658 len 28672
36551 129/300 Write block 2211 len 20480 data: 8 8 7 8 7
36552 130/300 Write block 1956 len 36864 data: 4 3 3 4 4 6 7 6 6
36553 131/300 Read block 1863 len 32768
36554 132/300 Read block 2006 len 40960
36555 133/300 Write block 3403 len 8192 data: 4 4
36556 134/300 Read block 2951 len 16384
36557 135/300 Write block 661 len 24576 data: 6 4 3 3 5 6
36558 136/300 Read block 1239 len 4096
36559 137/300 Write block 1806 len 8192 data: 5 6
36560 138/300 Write block 2324 len 32768 data: 4 6 7 8 10 10 11 9
36561 139/300 Read block 3654 len 32768
36562 140/300 Read block 1435 len 40960
36563 141/300 Read block 3766 len 24576
36564 142/300 Write block 4003 len 40960 data: 7 7 5 5 2 2 2 3 3 3
36565 143/300 Read block 4459 len 36864
36566 144/300 Read block 1940 len 12288
36567 145/300 Read block 3533 len 36864
36568 146/300 Read block 4373 len 32768
36569 147/300 Write block 87 len 32768 data: 6 6 5 5 4 3 4 3
36570 148/300 Read block 3506 len 24576
36571 149/300 Write block 4818 len 20480 data: 5 5 4 5 3
36572 150/300 Read block 4366 len 8192
36573 151/300 Read block 4100 len 40960
36574 152/300 Read block 3555 len 20480
36575 153/300 Write block 3706 len 8192 data: 8 9
36576 154/300 Read block 3333 len 32768
36577 155/300 Write block 4195 len 16384 data: 3 4 4 5
36578 156/300 Read block 2614 len 24576
36579 157/300 Flush
36580 158/300 Read block 2967 len 40960
36581 159/300 Write block 38 len 40960 data: 4 4 4 5 5 5 6 9 9 8
36582 160/300 Read block 2510 len 40960
36583 161/300 Flush
36584 162/300 Read block 1852 len 28672
36585 163/300 Write block 3079 len 8192 data: 5 4
36586 164/300 Write block 2911 len 16384 data: 6 6 4 4
36587 165/300 Read block 1300 len 24576
36588 166/300 Read block 3772 len 8192
36589 167/300 Write block 2046 len 28672 data: 5 5 6 6 5 5 6
36590 168/300 Read block 3530 len 4096
36591 169/300 Write block 1897 len 24576 data: 4 2 2 4 4 4
36592 170/300 Read block 1753 len 20480
36593 171/300 Write block 2480 len 8192 data: 6 5
36594 172/300 Read block 4684 len 4096
36595 173/300 Write block 147 len 32768 data: 6 7 7 6 6 5 5 6
36596 174/300 Write block 3926 len 12288 data: 7 6 5
36597 175/300 Read block 2497 len 8192
36598 176/300 Read block 201 len 40960
36599 177/300 Read block 3035 len 8192
36600 178/300 Read block 1619 len 28672
36601 179/300 Read block 3338 len 36864
36602 180/300 Write block 4808 len 36864 data: 7 7 7 7 7 9 7 7 6
36603 181/300 Read block 1585 len 4096
36604 182/300 Write block 3992 len 24576 data: 10 9 8 8 7 5
36605 183/300 Write block 3949 len 40960 data: 9 8 7 6 6 5 5 8 7 9
36606 184/300 Write block 2168 len 16384 data: 6 7 7 7
36607 185/300 Write block 4236 len 4096 data: 9
36608 186/300 Read block 1346 len 36864
36609 187/300 Read block 320 len 4096
36610 188/300 Flush
36611 189/300 Read block 1240 len 4096
36612 190/300 Write block 3436 len 32768 data: 5 6 6 7 6 6 7 6
36613 191/300 Read block 3718 len 16384
36614 192/300 Read block 351 len 40960
36615 193/300 Write block 3341 len 28672 data: 4 4 4 3 6 5 5
36616 194/300 Write block 4677 len 24576 data: 7 8 8 10 9 6
36617 195/300 Write block 3011 len 20480 data: 7 7 6 6 5
36618 196/300 Write block 3908 len 12288 data: 6 7 7
36619 197/300 Read block 1796 len 36864
36620 198/300 Write block 3267 len 12288 data: 6 6 3
36621 199/300 Write block 3854 len 36864 data: 8 7 9 10 11 10 9 10 13
36622 200/300 Read block 3552 len 24576
36623 201/300 Write block 2872 len 12288 data: 2 2 2
36624 202/300 Read block 1308 len 20480
36625 203/300 Read block 762 len 16384
36626 204/300 Write block 693 len 12288 data: 4 4 6
36627 205/300 Write block 1956 len 24576 data: 5 4 4 5 5 7
36628 206/300 Read block 1684 len 32768
36629 207/300 Read block 2552 len 24576
36630 208/300 Write block 2783 len 28672 data: 4 4 5 5 5 5 5
36631 209/300 Read block 419 len 28672
36632 210/300 Write block 3612 len 16384 data: 4 3 3 5
36633 211/300 Read block 781 len 32768
36634 212/300 Read block 3841 len 36864
36635 213/300 Read block 3942 len 16384
36636 214/300 Write block 447 len 20480 data: 7 6 5 6 6
36637 215/300 Read block 1133 len 40960
36638 216/300 Write block 1664 len 40960 data: 5 3 3 4 4 5 7 7 6 5
36639 217/300 Read block 2870 len 32768
36640 218/300 Write block 3456 len 40960 data: 4 3 2 2 4 4 6 6 5 5
36641 219/300 Read block 4598 len 12288
36642 220/300 Read block 747 len 4096
36643 221/300 Read block 2025 len 4096
36644 222/300 Write block 2970 len 20480 data: 5 5 4 4 4
36645 223/300 Read block 3722 len 32768
36646 224/300 Write block 3196 len 40960 data: 5 4 3 3 5 5 6 7 7 8
36647 225/300 Write block 2878 len 4096 data: 4
36648 226/300 Read block 2845 len 8192
36649 227/300 Write block 1972 len 40960 data: 7 7 6 8 8 8 8 9 9 10
36650 228/300 Write block 2713 len 24576 data: 10 13 10 13 10 11
36651 229/300 Read block 706 len 16384
36652 230/300 Write block 4512 len 24576 data: 3 3 3 4 4 4
36653 231/300 Flush
36654 232/300 Write block 1414 len 16384 data: 8 6 6 7
36655 233/300 Read block 678 len 8192
36656 234/300 Read block 1005 len 16384
36657 235/300 Read block 303 len 28672
36658 236/300 Write block 2961 len 8192 data: 7 10
36659 237/300 Read block 3882 len 20480
36660 238/300 Read block 2815 len 32768
36661 239/300 Read block 3814 len 12288
36662 240/300 Read block 2337 len 4096
36663 241/300 Write block 1476 len 28672 data: 3 3 4 4 4 7 7
36664 242/300 Write block 3861 len 8192 data: 11 14
36665 243/300 Read block 719 len 8192
36666 244/300 Write block 2969 len 28672 data: 4 6 6 5 5 5 4
36667 245/300 Write block 1847 len 16384 data: 5 4 4 5
36668 246/300 Write block 2482 len 32768 data: 6 5 6 6 5 5 5 6
36669 247/300 Write block 1780 len 12288 data: 8 11 10
36670 248/300 Read block 3579 len 4096
36671 249/300 Write block 371 len 36864 data: 2 2 3 3 3 3 3 3 3
36672 250/300 Write block 2221 len 24576 data: 7 8 7 9 9 9
36673 251/300 Flush
36674 252/300 Write block 1085 len 20480 data: 9 9 9 9 7
36675 253/300 Write block 867 len 40960 data: 5 6 5 6 7 7 8 9 9 7
36676 254/300 Write block 1454 len 4096 data: 7
36677 255/300 Read block 616 len 16384
36678 256/300 Read block 3248 len 12288
36679 257/300 Read block 3761 len 8192
36680 258/300 Write block 2572 len 12288 data: 5 6 9
36681 259/300 Read block 2163 len 24576
36682 260/300 Read block 3807 len 20480
36683 261/300 Write block 1975 len 36864 data: 9 9 9 9 10 10 11 9 8
36684 262/300 Flush
36685 263/300 Read block 2692 len 28672
36686 264/300 Write block 2873 len 16384 data: 3 3 2 3
36687 265/300 Read block 617 len 28672
36688 266/300 Read block 300 len 24576
36689 267/300 Write block 514 len 28672 data: 6 6 6 7 6 6 6
36690 268/300 Write block 3771 len 28672 data: 5 5 4 5 5 5 5
36691 269/300 Read block 3136 len 20480
36692 270/300 Read block 536 len 32768
36693 271/300 Flush
36694 272/300 Write block 4431 len 40960 data: 8 9 9 7 6 6 6 7 8 8
36695 273/300 Write block 3342 len 20480 data: 5 5 4 7 6
36696 274/300 Read block 4232 len 40960
36697 275/300 Write block 846 len 24576 data: 8 9 7 7 7 5
36698 276/300 Write block 4028 len 20480 data: 6 5 6 4 4
36699 277/300 Read block 1042 len 8192
36700 278/300 Write block 105 len 12288 data: 9 8 8
36701 279/300 Read block 2052 len 28672
36702 280/300 Read block 368 len 32768
36703 281/300 Read block 2645 len 8192
36704 282/300 Read block 595 len 36864
36705 283/300 Read block 2594 len 24576
36706 284/300 Read block 2952 len 4096
36707 285/300 Read block 3164 len 16384
36708 286/300 Write block 4605 len 8192 data: 9 8
36709 287/300 Write block 4721 len 8192 data: 8 10
36710 288/300 Write block 271 len 40960 data: 6 5 6 7 5 5 5 4 3 3
36711 289/300 Read block 1108 len 20480
36712 290/300 Read block 4624 len 12288
36713 291/300 Write block 4433 len 12288 data: 10 8 7
36714 292/300 Read block 3864 len 24576
36715 293/300 Flush
36716 294/300 Read block 493 len 8192
36717 295/300 Write block 1803 len 4096 data: 4
36718 296/300 Write block 3367 len 20480 data: 4 7 7 6 8
36719 297/300 Write block 4904 len 12288 data: 11 13 10
36720 298/300 Write block 1084 len 28672 data: 8 10 10 10 10 8 7
36721 299/300 Flush
36722 300/300 Write block 711 len 40960 data: 5 6 6 5 4 4 4 5 4 5
36723 301/300 Write block 799 len 12288 data: 3 5 5
36724 Main task has work to do, go find it
36725 got dsc Start(2)
36726 start 2
36727 Dec 11 23:48:13.905 INFO request completed, latency_us: 230, response_code: 204, uri: /start/cid/2, method: POST, req_id: d098ba15-06d0-4ebe-9b01-f5c3717971a9, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
36728 Replay: started 2, returned:Ok(())
36729 ----------------------------------------------------------------
36730 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
36731 States: Active Active Offline
36732 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
36733 9756 Acked Read 4 Done Done Sent false
36734 9757 Acked Read 10 Done Done Sent false
36735 9758 Acked Write 6 Done Done Sent false
36736 9759 Acked Read 7 Done Done Sent false
36737 9760 Acked Read 9 Done Done Sent false
36738 9761 Acked Write 4 Done Done Sent false
36739 9762 Acked Read 2 Done Done Sent false
36740 9763 Acked Read 5 Done Done Sent false
36741 9764 Acked Write 9 Done Done Sent false
36742 9765 Acked Read 4 Done Done Sent false
36743 9766 Acked Read 3 Done Done Sent false
36744 9767 Acked Write 9 Done Done Sent false
36745 9768 Acked Write 10 Done Done Sent false
36746 9769 Acked Write 9 Done Done Sent false
36747 9770 Acked Write 6 Done Done Sent false
36748 9771 Acked Flush 0 Done Done Sent false
36749 9772 Acked Read 3 Done Done Sent false
36750 9773 Acked Read 4 Done Done Sent false
36751 9774 Acked Write 1 Done Done Sent false
36752 9775 Acked Read 6 Done Done Sent false
36753 9776 Acked Read 7 Done Done Sent false
36754 9777 Acked Read 10 Done Done Sent false
36755 9778 Acked Read 10 Done Done Sent false
36756 9779 Acked Write 6 Done Done Sent false
36757 9780 Acked Read 7 Done Done Sent false
36758 9781 Acked Read 5 Done Done Sent false
36759 9782 Acked Read 1 Done Done Sent false
36760 9783 Acked Read 3 Done Done Sent false
36761 9784 Acked Read 6 Done Done Sent false
36762 9785 Acked Write 4 Done Done Sent false
36763 9786 Acked Read 7 Done Done Sent false
36764 9787 Acked Flush 0 Done Done Sent false
36765 9788 Acked Read 3 Done Done Sent false
36766 9789 Acked Read 6 Done Done Sent false
36767 9790 Acked Flush 0 Done Done Sent false
36768 9791 Acked Write 5 Done Done Sent false
36769 9792 Acked Read 5 Done Done Sent false
36770 9793 Acked Flush 0 Done Done Sent false
36771 9794 Acked Read 8 Done Done Sent false
36772 9795 Acked Write 5 Done Done Sent false
36773 9796 Acked Read 3 Done Done Sent false
36774 9797 Acked Flush 0 Done Done Sent false
36775 9798 Acked Read 6 Done Done Sent false
36776 9799 Acked Flush 0 Done Done Sent false
36777 9800 Acked Read 1 Done Done Sent false
36778 9801 Acked Read 10 Done Done Sent false
36779 9802 Acked Write 9 Done Done Sent false
36780 9803 Acked Flush 0 Done Done Sent false
36781 9804 Acked Write 7 Done Done Sent false
36782 9805 Acked Write 6 Done Done Sent false
36783 9806 Acked Write 1 Done Done Sent false
36784 9807 Acked Read 9 Done Done Sent false
36785 9808 Acked Write 2 Done Done Sent false
36786 9809 Acked Write 1 Done Done Sent false
36787 9810 Acked Read 3 Done Done Sent false
36788 9811 Acked Read 5 Done Done Sent false
36789 9812 Acked Write 5 Done Done Sent false
36790 9813 Acked Write 7 Done Done Sent false
36791 9814 Acked Write 2 Done Done Sent false
36792 9815 Acked Write 10 Done Done Sent false
36793 9816 Acked Read 1 Done Done Sent false
36794 9817 Acked Read 8 Done Done Sent false
36795 9818 Acked Read 10 Done Done Sent false
36796 9819 Acked Read 5 Done Done Sent false
36797 9820 Acked Write 6 Done Done Sent false
36798 9821 Acked Write 2 Done Done Sent false
36799 9822 Acked Read 10 Done Done Sent false
36800 9823 Acked Read 9 Done Done Sent false
36801 9824 Acked Read 4 Done Done Sent false
36802 9825 Acked Write 10 Done Done Sent false
36803 9826 Acked Flush 0 Done Done Sent false
36804 9827 Acked Write 3 Done Done Sent false
36805 9828 Acked Write 1 Done Done Sent false
36806 9829 Acked Read 1 Done Done Sent false
36807 9830 Acked Read 5 Done Done Sent false
36808 9831 Acked Read 2 Done Done Sent false
36809 9832 Acked Read 1 Done Done Sent false
36810 9833 Acked Read 6 Done Done Sent false
36811 9834 Acked Write 2 Done Done Sent false
36812 9835 Acked Write 3 Done Done Sent false
36813 9836 Acked Write 5 Done Done Sent false
36814 9837 Acked Read 3 Done Done Sent false
36815 9838 Acked Read 4 Done Done Sent false
36816 9839 Acked Read 8 Done Done Sent false
36817 9840 Acked Read 4 Done Done Sent false
36818 9841 Acked Read 7 Done Done Sent false
36819 9842 Acked Read 9 Done Done Sent false
36820 9843 Acked Write 9 Done Done Sent false
36821 9844 Acked Read 8 Done Done Sent false
36822 9845 Acked Write 3 Done Done Sent false
36823 9846 Acked Read 4 Done Done Sent false
36824 9847 Acked Read 5 Done Done Sent false
36825 9848 Acked Read 9 Done Done Sent false
36826 9849 Acked Write 8 Done Done Sent false
36827 9850 Acked Write 3 Done Done Sent false
36828 9851 Acked Read 5 Done Done Sent false
36829 9852 Acked Write 6 Done Done Sent false
36830 9853 Acked Write 6 Done Done Sent false
36831 9854 Acked Read 3 Done Done Sent false
36832 9855 Acked Read 5 Done Done Sent false
36833 9856 Acked Write 3 Done Done Sent false
36834 9857 Acked Write 9 Done Done Sent false
36835 9858 Acked Write 9 Done Done Sent false
36836 9859 Acked Write 6 Done Done Sent false
36837 9860 Acked Write 3 Done Done Sent false
36838 9861 Acked Write 2 Done Done Sent false
36839 9862 Acked Read 2 Done Done Sent false
36840 9863 Acked Read 6 Done Done Sent false
36841 9864 Acked Read 8 Done Done Sent false
36842 9865 Acked Read 8 Done Done Sent false
36843 9866 Acked Write 1 Done Done Sent false
36844 9867 Acked Read 4 Done Done Sent false
36845 9868 Acked Read 2 Done Done Sent false
36846 9869 Acked Write 1 Done Done Sent false
36847 9870 Acked Read 6 Done Done Sent false
36848 9871 Acked Write 8 Done Done Sent false
36849 9872 Acked Read 2 Done Done Sent false
36850 9873 Acked Flush 0 Done Done Sent false
36851 9874 Acked Flush 0 Done Done Sent false
36852 9875 Acked Write 2 Done Done Sent false
36853 9876 Acked Write 1 Done Done Sent false
36854 9877 Acked Read 3 Done Done Sent false
36855 9878 Acked Read 10 Done Done Sent false
36856 9879 Acked Read 10 Done Done Sent false
36857 9880 Acked Write 3 Done Done Sent false
36858 9881 Acked Read 5 Done Done Sent false
36859 9882 Acked Read 8 Done Done Sent false
36860 9883 Acked Read 7 Done Done Sent false
36861 9884 Acked Write 5 Done Done Sent false
36862 9885 Acked Write 9 Done Done Sent false
36863 9886 Acked Read 8 Done Done Sent false
36864 9887 Acked Read 10 Done Done Sent false
36865 9888 Acked Write 2 Done Done Sent false
36866 9889 Acked Read 4 Done Done Sent false
36867 9890 Acked Write 6 Done Done Sent false
36868 9891 Acked Read 1 Done Done Sent false
36869 9892 Acked Write 2 Done Done Sent false
36870 9893 Acked Write 8 Done Done Sent false
36871 9894 Acked Read 8 Done Done Sent false
36872 9895 Acked Read 10 Done Done Sent false
36873 9896 Acked Read 6 Done Done Sent false
36874 9897 Acked Write 10 Done Done Sent false
36875 9898 Acked Read 9 Done Done Sent false
36876 9899 Acked Read 3 Done Done Sent false
36877 9900 Acked Read 9 Done Done Sent false
36878 9901 Acked Read 8 Done Done Sent false
36879 9902 Acked Write 8 Done Done Sent false
36880 9903 Acked Read 6 Done Done Sent false
36881 9904 Acked Write 5 Done Done Sent false
36882 9905 Acked Read 2 Done Done Sent false
36883 9906 Acked Read 10 Done Done Sent false
36884 9907 Acked Read 5 Done Done Sent false
36885 9908 Acked Write 2 Done Done Sent false
36886 9909 Acked Read 8 Done Done Sent false
36887 9910 Acked Write 4 Done Done Sent false
36888 9911 Acked Read 6 Done Done Sent false
36889 9912 Acked Flush 0 Done Done Sent false
36890 9913 Acked Read 10 Done Done Sent false
36891 9914 Acked Write 10 Done Done Sent false
36892 9915 Acked Read 10 Done Done Sent false
36893 9916 Acked Flush 0 Done Done Sent false
36894 9917 Acked Read 7 Done Done Sent false
36895 9918 Acked Write 2 Done Done Sent false
36896 9919 Acked Write 4 Done Done Sent false
36897 9920 Acked Read 6 Done Done Sent false
36898 9921 Acked Read 2 Done Done Sent false
36899 9922 Acked Write 7 Done Done Sent false
36900 9923 Acked Read 1 Done Done Sent false
36901 9924 Acked Write 6 Done Done Sent false
36902 9925 Acked Read 5 Done Done Sent false
36903 9926 Acked Write 2 Done Done Sent false
36904 9927 Acked Read 1 Done Done Sent false
36905 9928 Acked Write 8 Done Done Sent false
36906 9929 Acked Write 3 Done Done Sent false
36907 9930 Acked Read 2 Done Done Sent false
36908 9931 Acked Read 10 Done Done Sent false
36909 9932 Acked Read 2 Done Done Sent false
36910 9933 Acked Read 7 Done Done Sent false
36911 9934 Acked Read 9 Done Done Sent false
36912 9935 Acked Write 9 Done Done Sent false
36913 9936 Acked Read 1 Done Done Sent false
36914 9937 Acked Write 6 Done Done Sent false
36915 9938 Acked Write 10 Done Done Sent false
36916 9939 Acked Write 4 Done Done Sent false
36917 9940 Acked Write 1 Done Done Sent false
36918 9941 Acked Read 9 Done Done Sent false
36919 9942 Acked Read 1 Done Done Sent false
36920 9943 Acked Flush 0 Done Done Sent false
36921 9944 Acked Read 1 Done Done Sent false
36922 9945 Acked Write 8 Done Done Sent false
36923 9946 Acked Read 4 Done Done Sent false
36924 9947 Acked Read 10 Done Done Sent false
36925 9948 Acked Write 7 Done Done Sent false
36926 9949 Acked Write 6 Done Done Sent false
36927 9950 Acked Write 5 Done Done Sent false
36928 9951 Acked Write 3 Done Done Sent false
36929 9952 Acked Read 9 Done Done Sent false
36930 9953 Acked Write 3 Done Done Sent false
36931 9954 Acked Write 9 Done Done Sent false
36932 9955 Acked Read 6 Done Done Sent false
36933 9956 Acked Write 3 Done Done Sent false
36934 9957 Acked Read 5 Done Done Sent false
36935 9958 Acked Read 4 Done Done Sent false
36936 9959 Acked Write 3 Done Done Sent false
36937 9960 Acked Write 6 Done Done Sent false
36938 9961 Acked Read 8 Done Done Sent false
36939 9962 Acked Read 6 Done Done Sent false
36940 9963 Acked Write 7 Done Done Sent false
36941 9964 Acked Read 7 Done Done Sent false
36942 9965 Acked Write 4 Done Done Sent false
36943 9966 Acked Read 8 Done Done Sent false
36944 9967 Acked Read 9 Done Done Sent false
36945 9968 Acked Read 4 Done Done Sent false
36946 9969 Acked Write 5 Done Done Sent false
36947 9970 Acked Read 10 Done Done Sent false
36948 9971 Acked Write 10 Done Done Sent false
36949 9972 Acked Read 8 Done Done Sent false
36950 9973 Acked Write 10 Done Done Sent false
36951 9974 Acked Read 3 Done Done Sent false
36952 9975 Acked Read 1 Done Done Sent false
36953 9976 Acked Read 1 Done Done Sent false
36954 9977 Acked Write 5 Done Done Sent false
36955 9978 Acked Read 8 Done Done Sent false
36956 9979 Acked Write 10 Done Done Sent false
36957 9980 Acked Write 1 Done Done Sent false
36958 9981 Acked Read 2 Done Done Sent false
36959 9982 Acked Write 10 Done Done Sent false
36960 9983 Acked Write 6 Done Done Sent false
36961 9984 Acked Read 4 Done Done Sent false
36962 9985 Acked Write 6 Done Done Sent false
36963 9986 Acked Flush 0 Done Done Sent false
36964 9987 Acked Write 4 Done Done Sent false
36965 9988 Acked Read 2 Done Done Sent false
36966 9989 Acked Read 4 Done Done Sent false
36967 9990 Acked Read 7 Done Done Sent false
36968 9991 Acked Write 2 Done Done Sent false
36969 9992 Acked Read 5 Done Done Sent false
36970 9993 Acked Read 8 Done Done Sent false
36971 9994 Acked Read 3 Done Done Sent false
36972 9995 Acked Read 1 Done Done Sent false
36973 9996 Acked Write 7 Done Done Sent false
36974 9997 Acked Write 2 Done Done Sent false
36975 9998 Acked Read 2 Done Done Sent false
36976 9999 Acked Write 7 Done Done Sent false
36977 10000 Acked Write 4 Done Done Sent false
36978 10001 Acked Write 8 Done Done Sent false
36979 10002 Acked Write 3 Done Done Sent false
36980 10003 Acked Read 1 Done Done Sent false
36981 10004 Acked Write 9 Done Done Sent false
36982 10005 Acked Write 6 Done Done Sent false
36983 10006 Acked Flush 0 Done Done Sent false
36984 10007 Acked Write 5 Done Done Sent false
36985 10008 Acked Write 10 Done Done Sent false
36986 10009 Acked Write 1 Done Done Sent false
36987 10010 Acked Read 4 Done Done Sent false
36988 10011 Acked Read 3 Done Done Sent false
36989 10012 Acked Read 2 Done Done Sent false
36990 10013 Acked Write 3 Done Done Sent false
36991 10014 Acked Read 6 Done Done Sent false
36992 10015 Acked Read 5 Done Done Sent false
36993 10016 Acked Write 9 Done Done Sent false
36994 10017 Acked Flush 0 Done Done Sent false
36995 10018 Acked Read 7 Done Done Sent false
36996 10019 Acked Write 4 Done Done Sent false
36997 10020 Acked Read 7 Done Done Sent false
36998 10021 Acked Read 6 Done Done Sent false
36999 10022 Acked Write 7 Done Done Sent false
37000 10023 Acked Write 7 Done Done Sent false
37001 10024 Acked Read 5 Done Done Sent false
37002 10025 Acked Read 8 Done Done Sent false
37003 10026 Acked Flush 0 Done Done Sent false
37004 10027 Acked Write 10 Done Done Sent false
37005 10028 Acked Write 5 Done Done Sent false
37006 10029 Acked Read 10 Done Done Sent false
37007 10030 Acked Write 6 Done Done Sent false
37008 10031 Acked Write 5 Done Done Sent false
37009 10032 Acked Read 2 Done Done Sent false
37010 10033 Acked Write 3 Done Done Sent false
37011 10034 Acked Read 7 Done Done Sent false
37012 10035 Acked Read 8 Done Done Sent false
37013 10036 Acked Read 2 Done Done Sent false
37014 10037 Acked Read 9 Done Done Sent false
37015 10038 Acked Read 6 Done Done Sent false
37016 10039 Acked Read 1 Done Done Sent false
37017 10040 Acked Read 4 Done Done Sent false
37018 10041 Acked Write 2 Done Done Sent false
37019 10042 Acked Write 2 Done Done Sent false
37020 10043 Acked Write 10 Done Done Sent false
37021 10044 Acked Read 5 Done Done Sent false
37022 10045 Acked Read 3 Done Done Sent false
37023 10046 Acked Write 3 Done Done Sent false
37024 10047 Acked Read 6 Done Done Sent false
37025 10048 Acked Flush 0 Done Done Sent false
37026 10049 Acked Read 2 Done Done Sent false
37027 10050 Acked Write 1 Done Done Sent false
37028 10051 Acked Write 5 Done Done Sent false
37029 10052 Acked Write 3 Done Done Sent false
37030 10053 Acked Write 7 Done Done Sent false
37031 10054 Acked Flush 0 Done Done Sent false
37032 10055 Acked Write 10 Done Done Sent false
37033 10056 Acked Write 3 Done Done Sent false
37034 STATES DS:0 DS:1 DS:2 TOTAL
37035 Sent 0 0 301 301
37036 Done 301 301 0 602
37037 Skipped 0 0 0 0
37038 Error 0 0 0 0
37039 Last Flush: 10054 10054 9755
37040 Downstairs last five completed: 9755 9754 9753 9752 9751
37041 Upstairs last five completed: 10056 10055 10054 10053 10052
37042 CLIENT: Up:0 ds:301 act:2
37043 [8830] got start action
37044 [8830] I am going to restart
37045 Starting downstairs at port 8830
37046 Make output file at "/tmp/dsc/downstairs-8830.txt"
37047 [8830][2] reports Starting
37048 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1230)
37049 [8830][2] reports Running
37050 ----------------------------------------------------------------
37051 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
37052 States: Active Active Offline
37053 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37054 9756 Acked Read 4 Done Done Sent false
37055 9757 Acked Read 10 Done Done Sent false
37056 9758 Acked Write 6 Done Done Sent false
37057 9759 Acked Read 7 Done Done Sent false
37058 9760 Acked Read 9 Done Done Sent false
37059 9761 Acked Write 4 Done Done Sent false
37060 9762 Acked Read 2 Done Done Sent false
37061 9763 Acked Read 5 Done Done Sent false
37062 9764 Acked Write 9 Done Done Sent false
37063 9765 Acked Read 4 Done Done Sent false
37064 9766 Acked Read 3 Done Done Sent false
37065 9767 Acked Write 9 Done Done Sent false
37066 9768 Acked Write 10 Done Done Sent false
37067 9769 Acked Write 9 Done Done Sent false
37068 9770 Acked Write 6 Done Done Sent false
37069 9771 Acked Flush 0 Done Done Sent false
37070 9772 Acked Read 3 Done Done Sent false
37071 9773 Acked Read 4 Done Done Sent false
37072 9774 Acked Write 1 Done Done Sent false
37073 9775 Acked Read 6 Done Done Sent false
37074 9776 Acked Read 7 Done Done Sent false
37075 9777 Acked Read 10 Done Done Sent false
37076 9778 Acked Read 10 Done Done Sent false
37077 9779 Acked Write 6 Done Done Sent false
37078 9780 Acked Read 7 Done Done Sent false
37079 9781 Acked Read 5 Done Done Sent false
37080 9782 Acked Read 1 Done Done Sent false
37081 9783 Acked Read 3 Done Done Sent false
37082 9784 Acked Read 6 Done Done Sent false
37083 9785 Acked Write 4 Done Done Sent false
37084 9786 Acked Read 7 Done Done Sent false
37085 9787 Acked Flush 0 Done Done Sent false
37086 9788 Acked Read 3 Done Done Sent false
37087 9789 Acked Read 6 Done Done Sent false
37088 9790 Acked Flush 0 Done Done Sent false
37089 9791 Acked Write 5 Done Done Sent false
37090 9792 Acked Read 5 Done Done Sent false
37091 9793 Acked Flush 0 Done Done Sent false
37092 9794 Acked Read 8 Done Done Sent false
37093 9795 Acked Write 5 Done Done Sent false
37094 9796 Acked Read 3 Done Done Sent false
37095 9797 Acked Flush 0 Done Done Sent false
37096 9798 Acked Read 6 Done Done Sent false
37097 9799 Acked Flush 0 Done Done Sent false
37098 9800 Acked Read 1 Done Done Sent false
37099 9801 Acked Read 10 Done Done Sent false
37100 9802 Acked Write 9 Done Done Sent false
37101 9803 Acked Flush 0 Done Done Sent false
37102 9804 Acked Write 7 Done Done Sent false
37103 9805 Acked Write 6 Done Done Sent false
37104 9806 Acked Write 1 Done Done Sent false
37105 9807 Acked Read 9 Done Done Sent false
37106 9808 Acked Write 2 Done Done Sent false
37107 9809 Acked Write 1 Done Done Sent false
37108 9810 Acked Read 3 Done Done Sent false
37109 9811 Acked Read 5 Done Done Sent false
37110 9812 Acked Write 5 Done Done Sent false
37111 9813 Acked Write 7 Done Done Sent false
37112 9814 Acked Write 2 Done Done Sent false
37113 9815 Acked Write 10 Done Done Sent false
37114 9816 Acked Read 1 Done Done Sent false
37115 9817 Acked Read 8 Done Done Sent false
37116 9818 Acked Read 10 Done Done Sent false
37117 9819 Acked Read 5 Done Done Sent false
37118 9820 Acked Write 6 Done Done Sent false
37119 9821 Acked Write 2 Done Done Sent false
37120 9822 Acked Read 10 Done Done Sent false
37121 9823 Acked Read 9 Done Done Sent false
37122 9824 Acked Read 4 Done Done Sent false
37123 9825 Acked Write 10 Done Done Sent false
37124 9826 Acked Flush 0 Done Done Sent false
37125 9827 Acked Write 3 Done Done Sent false
37126 9828 Acked Write 1 Done Done Sent false
37127 9829 Acked Read 1 Done Done Sent false
37128 9830 Acked Read 5 Done Done Sent false
37129 9831 Acked Read 2 Done Done Sent false
37130 9832 Acked Read 1 Done Done Sent false
37131 9833 Acked Read 6 Done Done Sent false
37132 9834 Acked Write 2 Done Done Sent false
37133 9835 Acked Write 3 Done Done Sent false
37134 9836 Acked Write 5 Done Done Sent false
37135 9837 Acked Read 3 Done Done Sent false
37136 9838 Acked Read 4 Done Done Sent false
37137 9839 Acked Read 8 Done Done Sent false
37138 9840 Acked Read 4 Done Done Sent false
37139 9841 Acked Read 7 Done Done Sent false
37140 9842 Acked Read 9 Done Done Sent false
37141 9843 Acked Write 9 Done Done Sent false
37142 9844 Acked Read 8 Done Done Sent false
37143 9845 Acked Write 3 Done Done Sent false
37144 9846 Acked Read 4 Done Done Sent false
37145 9847 Acked Read 5 Done Done Sent false
37146 9848 Acked Read 9 Done Done Sent false
37147 9849 Acked Write 8 Done Done Sent false
37148 9850 Acked Write 3 Done Done Sent false
37149 9851 Acked Read 5 Done Done Sent false
37150 9852 Acked Write 6 Done Done Sent false
37151 9853 Acked Write 6 Done Done Sent false
37152 9854 Acked Read 3 Done Done Sent false
37153 9855 Acked Read 5 Done Done Sent false
37154 9856 Acked Write 3 Done Done Sent false
37155 9857 Acked Write 9 Done Done Sent false
37156 9858 Acked Write 9 Done Done Sent false
37157 9859 Acked Write 6 Done Done Sent false
37158 9860 Acked Write 3 Done Done Sent false
37159 9861 Acked Write 2 Done Done Sent false
37160 9862 Acked Read 2 Done Done Sent false
37161 9863 Acked Read 6 Done Done Sent false
37162 9864 Acked Read 8 Done Done Sent false
37163 9865 Acked Read 8 Done Done Sent false
37164 9866 Acked Write 1 Done Done Sent false
37165 9867 Acked Read 4 Done Done Sent false
37166 9868 Acked Read 2 Done Done Sent false
37167 9869 Acked Write 1 Done Done Sent false
37168 9870 Acked Read 6 Done Done Sent false
37169 9871 Acked Write 8 Done Done Sent false
37170 9872 Acked Read 2 Done Done Sent false
37171 9873 Acked Flush 0 Done Done Sent false
37172 9874 Acked Flush 0 Done Done Sent false
37173 9875 Acked Write 2 Done Done Sent false
37174 9876 Acked Write 1 Done Done Sent false
37175 9877 Acked Read 3 Done Done Sent false
37176 9878 Acked Read 10 Done Done Sent false
37177 9879 Acked Read 10 Done Done Sent false
37178 9880 Acked Write 3 Done Done Sent false
37179 9881 Acked Read 5 Done Done Sent false
37180 9882 Acked Read 8 Done Done Sent false
37181 9883 Acked Read 7 Done Done Sent false
37182 9884 Acked Write 5 Done Done Sent false
37183 9885 Acked Write 9 Done Done Sent false
37184 9886 Acked Read 8 Done Done Sent false
37185 9887 Acked Read 10 Done Done Sent false
37186 9888 Acked Write 2 Done Done Sent false
37187 9889 Acked Read 4 Done Done Sent false
37188 9890 Acked Write 6 Done Done Sent false
37189 9891 Acked Read 1 Done Done Sent false
37190 9892 Acked Write 2 Done Done Sent false
37191 9893 Acked Write 8 Done Done Sent false
37192 9894 Acked Read 8 Done Done Sent false
37193 9895 Acked Read 10 Done Done Sent false
37194 9896 Acked Read 6 Done Done Sent false
37195 9897 Acked Write 10 Done Done Sent false
37196 9898 Acked Read 9 Done Done Sent false
37197 9899 Acked Read 3 Done Done Sent false
37198 9900 Acked Read 9 Done Done Sent false
37199 9901 Acked Read 8 Done Done Sent false
37200 9902 Acked Write 8 Done Done Sent false
37201 9903 Acked Read 6 Done Done Sent false
37202 9904 Acked Write 5 Done Done Sent false
37203 9905 Acked Read 2 Done Done Sent false
37204 9906 Acked Read 10 Done Done Sent false
37205 9907 Acked Read 5 Done Done Sent false
37206 9908 Acked Write 2 Done Done Sent false
37207 9909 Acked Read 8 Done Done Sent false
37208 9910 Acked Write 4 Done Done Sent false
37209 9911 Acked Read 6 Done Done Sent false
37210 9912 Acked Flush 0 Done Done Sent false
37211 9913 Acked Read 10 Done Done Sent false
37212 9914 Acked Write 10 Done Done Sent false
37213 9915 Acked Read 10 Done Done Sent false
37214 9916 Acked Flush 0 Done Done Sent false
37215 9917 Acked Read 7 Done Done Sent false
37216 9918 Acked Write 2 Done Done Sent false
37217 9919 Acked Write 4 Done Done Sent false
37218 9920 Acked Read 6 Done Done Sent false
37219 9921 Acked Read 2 Done Done Sent false
37220 9922 Acked Write 7 Done Done Sent false
37221 9923 Acked Read 1 Done Done Sent false
37222 9924 Acked Write 6 Done Done Sent false
37223 9925 Acked Read 5 Done Done Sent false
37224 9926 Acked Write 2 Done Done Sent false
37225 9927 Acked Read 1 Done Done Sent false
37226 9928 Acked Write 8 Done Done Sent false
37227 9929 Acked Write 3 Done Done Sent false
37228 9930 Acked Read 2 Done Done Sent false
37229 9931 Acked Read 10 Done Done Sent false
37230 9932 Acked Read 2 Done Done Sent false
37231 9933 Acked Read 7 Done Done Sent false
37232 9934 Acked Read 9 Done Done Sent false
37233 9935 Acked Write 9 Done Done Sent false
37234 9936 Acked Read 1 Done Done Sent false
37235 9937 Acked Write 6 Done Done Sent false
37236 9938 Acked Write 10 Done Done Sent false
37237 9939 Acked Write 4 Done Done Sent false
37238 9940 Acked Write 1 Done Done Sent false
37239 9941 Acked Read 9 Done Done Sent false
37240 9942 Acked Read 1 Done Done Sent false
37241 9943 Acked Flush 0 Done Done Sent false
37242 9944 Acked Read 1 Done Done Sent false
37243 9945 Acked Write 8 Done Done Sent false
37244 9946 Acked Read 4 Done Done Sent false
37245 9947 Acked Read 10 Done Done Sent false
37246 9948 Acked Write 7 Done Done Sent false
37247 9949 Acked Write 6 Done Done Sent false
37248 9950 Acked Write 5 Done Done Sent false
37249 9951 Acked Write 3 Done Done Sent false
37250 9952 Acked Read 9 Done Done Sent false
37251 9953 Acked Write 3 Done Done Sent false
37252 9954 Acked Write 9 Done Done Sent false
37253 9955 Acked Read 6 Done Done Sent false
37254 9956 Acked Write 3 Done Done Sent false
37255 9957 Acked Read 5 Done Done Sent false
37256 9958 Acked Read 4 Done Done Sent false
37257 9959 Acked Write 3 Done Done Sent false
37258 9960 Acked Write 6 Done Done Sent false
37259 9961 Acked Read 8 Done Done Sent false
37260 9962 Acked Read 6 Done Done Sent false
37261 9963 Acked Write 7 Done Done Sent false
37262 9964 Acked Read 7 Done Done Sent false
37263 9965 Acked Write 4 Done Done Sent false
37264 9966 Acked Read 8 Done Done Sent false
37265 9967 Acked Read 9 Done Done Sent false
37266 9968 Acked Read 4 Done Done Sent false
37267 9969 Acked Write 5 Done Done Sent false
37268 9970 Acked Read 10 Done Done Sent false
37269 9971 Acked Write 10 Done Done Sent false
37270 9972 Acked Read 8 Done Done Sent false
37271 9973 Acked Write 10 Done Done Sent false
37272 9974 Acked Read 3 Done Done Sent false
37273 9975 Acked Read 1 Done Done Sent false
37274 9976 Acked Read 1 Done Done Sent false
37275 9977 Acked Write 5 Done Done Sent false
37276 9978 Acked Read 8 Done Done Sent false
37277 9979 Acked Write 10 Done Done Sent false
37278 9980 Acked Write 1 Done Done Sent false
37279 9981 Acked Read 2 Done Done Sent false
37280 9982 Acked Write 10 Done Done Sent false
37281 9983 Acked Write 6 Done Done Sent false
37282 9984 Acked Read 4 Done Done Sent false
37283 9985 Acked Write 6 Done Done Sent false
37284 9986 Acked Flush 0 Done Done Sent false
37285 9987 Acked Write 4 Done Done Sent false
37286 9988 Acked Read 2 Done Done Sent false
37287 9989 Acked Read 4 Done Done Sent false
37288 9990 Acked Read 7 Done Done Sent false
37289 9991 Acked Write 2 Done Done Sent false
37290 9992 Acked Read 5 Done Done Sent false
37291 9993 Acked Read 8 Done Done Sent false
37292 9994 Acked Read 3 Done Done Sent false
37293 9995 Acked Read 1 Done Done Sent false
37294 9996 Acked Write 7 Done Done Sent false
37295 9997 Acked Write 2 Done Done Sent false
37296 9998 Acked Read 2 Done Done Sent false
37297 9999 Acked Write 7 Done Done Sent false
37298 10000 Acked Write 4 Done Done Sent false
37299 10001 Acked Write 8 Done Done Sent false
37300 10002 Acked Write 3 Done Done Sent false
37301 10003 Acked Read 1 Done Done Sent false
37302 10004 Acked Write 9 Done Done Sent false
37303 10005 Acked Write 6 Done Done Sent false
37304 10006 Acked Flush 0 Done Done Sent false
37305 10007 Acked Write 5 Done Done Sent false
37306 10008 Acked Write 10 Done Done Sent false
37307 10009 Acked Write 1 Done Done Sent false
37308 10010 Acked Read 4 Done Done Sent false
37309 10011 Acked Read 3 Done Done Sent false
37310 10012 Acked Read 2 Done Done Sent false
37311 10013 Acked Write 3 Done Done Sent false
37312 10014 Acked Read 6 Done Done Sent false
37313 10015 Acked Read 5 Done Done Sent false
37314 10016 Acked Write 9 Done Done Sent false
37315 10017 Acked Flush 0 Done Done Sent false
37316 10018 Acked Read 7 Done Done Sent false
37317 10019 Acked Write 4 Done Done Sent false
37318 10020 Acked Read 7 Done Done Sent false
37319 10021 Acked Read 6 Done Done Sent false
37320 10022 Acked Write 7 Done Done Sent false
37321 10023 Acked Write 7 Done Done Sent false
37322 10024 Acked Read 5 Done Done Sent false
37323 10025 Acked Read 8 Done Done Sent false
37324 10026 Acked Flush 0 Done Done Sent false
37325 10027 Acked Write 10 Done Done Sent false
37326 10028 Acked Write 5 Done Done Sent false
37327 10029 Acked Read 10 Done Done Sent false
37328 10030 Acked Write 6 Done Done Sent false
37329 10031 Acked Write 5 Done Done Sent false
37330 10032 Acked Read 2 Done Done Sent false
37331 10033 Acked Write 3 Done Done Sent false
37332 10034 Acked Read 7 Done Done Sent false
37333 10035 Acked Read 8 Done Done Sent false
37334 10036 Acked Read 2 Done Done Sent false
37335 10037 Acked Read 9 Done Done Sent false
37336 10038 Acked Read 6 Done Done Sent false
37337 10039 Acked Read 1 Done Done Sent false
37338 10040 Acked Read 4 Done Done Sent false
37339 10041 Acked Write 2 Done Done Sent false
37340 10042 Acked Write 2 Done Done Sent false
37341 10043 Acked Write 10 Done Done Sent false
37342 10044 Acked Read 5 Done Done Sent false
37343 10045 Acked Read 3 Done Done Sent false
37344 10046 Acked Write 3 Done Done Sent false
37345 10047 Acked Read 6 Done Done Sent false
37346 10048 Acked Flush 0 Done Done Sent false
37347 10049 Acked Read 2 Done Done Sent false
37348 10050 Acked Write 1 Done Done Sent false
37349 10051 Acked Write 5 Done Done Sent false
37350 10052 Acked Write 3 Done Done Sent false
37351 10053 Acked Write 7 Done Done Sent false
37352 10054 Acked Flush 0 Done Done Sent false
37353 10055 Acked Write 10 Done Done Sent false
37354 10056 Acked Write 3 Done Done Sent false
37355 10057 Acked Flush 0 Done Done Sent false
37356 STATES DS:0 DS:1 DS:2 TOTAL
37357 Sent 0 0 302 302
37358 Done 302 302 0 604
37359 Skipped 0 0 0 0
37360 Error 0 0 0 0
37361 Last Flush: 10057 10057 9755
37362 Downstairs last five completed: 9755 9754 9753 9752 9751
37363 Upstairs last five completed: 10057 10056 10055 10054 10053
37364 CLIENT: Up:0 ds:302 act:2
373652024-12-11T23:48:19.655ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373662024-12-11T23:48:19.655ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38404) = io task client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373672024-12-11T23:48:19.656ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373682024-12-11T23:48:19.656ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373692024-12-11T23:48:19.656ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373702024-12-11T23:48:19.656ZINFOcrucible: send last flush ID to this DS: 9755 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373712024-12-11T23:48:19.657ZINFOcrucible: Replied this last flush ID: 9755 = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373722024-12-11T23:48:19.657ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373732024-12-11T23:48:19.657ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373742024-12-11T23:48:19.657ZINFOcrucible: [2] client re-new 302 jobs since flush 9755 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373752024-12-11T23:48:19.657ZINFOcrucible: [2] Replayed 302 jobs since flush: 9755 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37376 ----------------------------------------------------------------
37377 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
37378 Downstairs last five completed: 10057 10056 10055 10054 10053
37379 Upstairs last five completed: 10057 10056 10055 10054 10053
37380 CLIENT: Up:0 ds:0 act:3
37381 Replay: All jobs finished
37382 Main task has work to do, go find it
37383 got dsc Stop(1)
37384 stop 1
37385 [8820] Got stop action so:false kr:false
37386 Dec 11 23:48:21.910 INFO request completed, latency_us: 390, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 94558c36-f44a-4d5c-8fbe-9bb52194c735, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
373872024-12-11T23:48:21.911ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373882024-12-11T23:48:21.911ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
373892024-12-11T23:48:21.911ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
37390 Dec 11 23:48:21.911 INFO request completed, latency_us: {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:48:21.911620878Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
37391 112, response_code: 200, uri: /state/cid/1, method{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:48:21.911644309Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
37392 : GET, req_id: 353b7cfd-e65c-4ff8-8e81-59fc040efbe2, remote_addr: 127.0.0.1:62969, local_addr:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-11T23:48:21.91168226Z","hostname":"w-01JEW1EB65A54HBHY7XVKMG26C","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"b03dd6e3-2411-404a-b84f-1ea6c44cc21f"}
37393 127.0.0.1:9998
37394 [8820] Exited with: ExitStatus(unix_wait_status(9))
37395 [8820][1] reports Exit
37396 Dec 11 23:48:25.913 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7334e10c-98e6-4297-a632-2578c35f51eb, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
37397 001/300 Flush
37398 002/300 Read block 3323 len 4096
37399 003/300 Read block 4284 len 28672
37400 004/300 Read block 8 len 36864
37401 005/300 Write block 4129 len 36864 data: 6 5 6 8 6 6 6 7 7
37402 006/300 Write block 2624 len 24576 data: 3 3 4 4 4 4
37403 007/300 Flush
37404 008/300 Read block 400 len 24576
37405 009/300 Read block 3289 len 36864
37406 010/300 Read block 1594 len 28672
37407 011/300 Write block 3890 len 32768 data: 8 8 8 9 10 10 10 10
37408 012/300 Write block 630 len 20480 data: 4 3 3 7 5
37409 013/300 Write block 394 len 12288 data: 4 5 5
37410 014/300 Flush
37411 015/300 Read block 3824 len 4096
37412 016/300 Read block 4570 len 8192
37413 017/300 Read block 537 len 40960
37414 018/300 Read block 171 len 12288
37415 019/300 Read block 4826 len 36864
37416 020/300 Write block 847 len 40960 data: 10 8 8 8 6 4 5 6 7 6
37417 021/300 Write block 3810 len 12288 data: 6 6 7
37418 022/300 Write block 2703 len 16384 data: 5 6 5 8
37419 023/300 Read block 1392 len 24576
37420 024/300 Read block 257 len 16384
37421 025/300 Read block 1478 len 4096
37422 026/300 Write block 2540 len 16384 data: 5 4 4 7
37423 027/300 Write block 3312 len 12288 data: 6 6 7
37424 028/300 Read block 2740 len 32768
37425 029/300 Flush
37426 030/300 Write block 2538 len 28672 data: 5 6 6 5 5 8 9
37427 031/300 Write block 2640 len 4096 data: 3
37428 032/300 Write block 1248 len 16384 data: 10 7 6 8
37429 033/300 Write block 4877 len 36864 data: 5 3 3 4 6 6 6 6 5
37430 034/300 Write block 3432 len 4096 data: 6
37431 035/300 Read block 4117 len 8192
37432 036/300 Read block 3379 len 8192
37433 037/300 Read block 208 len 20480
37434 038/300 Flush
37435 039/300 Flush
37436 040/300 Write block 1932 len 4096 data: 8
37437 041/300 Read block 2722 len 40960
37438 042/300 Write block 3724 len 4096 data: 7
37439 043/300 Read block 4448 len 40960
37440 044/300 Read block 942 len 24576
37441 045/300 Write block 2042 len 4096 data: 6
37442 046/300 Write block 3117 len 12288 data: 6 7 6
37443 047/300 Write block 3203 len 4096 data: 8
37444 048/300 Write block 495 len 32768 data: 7 7 8 7 9 8 7 5
37445 049/300 Read block 1250 len 16384
37446 050/300 Write block 4921 len 40960 data: 10 13 11 11 9 9 7 7 8 7
37447 051/300 Read block 356 len 4096
37448 052/300 Read block 2910 len 8192
37449 053/300 Read block 4703 len 12288
37450 054/300 Read block 3870 len 32768
37451 055/300 Write block 1398 len 8192 data: 7 6
37452 056/300 Read block 4468 len 20480
37453 057/300 Read block 2372 len 12288
37454 058/300 Read block 217 len 20480
37455 059/300 Read block 3718 len 12288
37456 060/300 Flush
37457 061/300 Read block 3488 len 24576
37458 062/300 Read block 405 len 20480
37459 063/300 Read block 2607 len 20480
37460 064/300 Read block 3137 len 32768
37461 065/300 Flush
37462 066/300 Read block 54 len 8192
37463 067/300 Write block 2392 len 32768 data: 4 4 3 3 3 3 3 3
37464 068/300 Write block 4557 len 24576 data: 7 8 7 8 7 9
37465 069/300 Read block 673 len 40960
37466 070/300 Write block 2217 len 4096 data: 8
37467 071/300 Write block 4293 len 8192 data: 4 5
37468 072/300 Read block 1981 len 16384
37469 073/300 Read block 4294 len 16384
37470 074/300 Read block 2395 len 4096
37471 075/300 Read block 2046 len 36864
37472 076/300 Read block 2693 len 20480
37473 077/300 Write block 4360 len 32768 data: 4 5 7 7 6 6 5 5
37474 078/300 Read block 2640 len 16384
37475 079/300 Read block 1831 len 32768
37476 080/300 Write block 1179 len 12288 data: 7 8 7
37477 081/300 Read block 4013 len 16384
37478 082/300 Read block 1857 len 12288
37479 083/300 Read block 4006 len 4096
37480 084/300 Read block 3396 len 4096
37481 085/300 Write block 2681 len 32768 data: 5 5 5 5 6 6 7 6
37482 086/300 Read block 4560 len 20480
37483 087/300 Write block 754 len 20480 data: 8 5 4 4 4
37484 088/300 Read block 3805 len 28672
37485 089/300 Read block 852 len 24576
37486 090/300 Write block 1768 len 12288 data: 2 4 4
37487 091/300 Flush
37488 092/300 Write block 1877 len 36864 data: 6 6 6 7 7 7 6 5 4
37489 093/300 Flush
37490 094/300 Flush
37491 095/300 Read block 3229 len 36864
37492 096/300 Read block 1219 len 28672
37493 097/300 Write block 3161 len 16384 data: 5 5 5 6
37494 098/300 Read block 4893 len 28672
37495 099/300 Read block 610 len 32768
37496 100/300 Write block 4274 len 32768 data: 5 3 3 5 5 5 5 7
37497 101/300 Read block 1677 len 12288
37498 102/300 Read block 4815 len 28672
37499 103/300 Read block 2433 len 4096
37500 104/300 Read block 1384 len 24576
37501 105/300 Read block 1856 len 40960
37502 106/300 Read block 1973 len 12288
37503 107/300 Write block 1448 len 24576 data: 8 7 7 8 10 8
37504 108/300 Write block 1076 len 32768 data: 7 8 8 6 6 7 8 7
37505 109/300 Write block 220 len 12288 data: 3 3 3
37506 110/300 Write block 230 len 8192 data: 4 5
37507 111/300 Read block 2698 len 8192
37508 112/300 Read block 4789 len 12288
37509 113/300 Read block 1512 len 8192
37510 114/300 Read block 1083 len 32768
37511 115/300 Read block 1062 len 8192
37512 116/300 Flush
37513 117/300 Write block 328 len 8192 data: 9 10
37514 118/300 Write block 446 len 12288 data: 7 8 7
37515 119/300 Read block 2384 len 16384
37516 120/300 Write block 755 len 16384 data: 6 5 5 5
37517 121/300 Write block 4097 len 28672 data: 7 8 8 6 7 8 10
37518 122/300 Write block 4081 len 8192 data: 4 2
37519 123/300 Read block 3131 len 40960
37520 124/300 Flush
37521 125/300 Write block 2955 len 20480 data: 6 5 6 7 8
37522 126/300 Read block 728 len 4096
37523 127/300 Write block 1585 len 8192 data: 6 5
37524 128/300 Write block 2389 len 36864 data: 4 4 4 5 5 4 4 4 4
37525 129/300 Write block 2401 len 16384 data: 3 4 4 4
37526 130/300 Flush
37527 131/300 Write block 4903 len 20480 data: 9 12 14 11 8
37528 132/300 Flush
37529 133/300 Write block 2823 len 16384 data: 8 9 7 6
37530 134/300 Flush
37531 135/300 Write block 2508 len 24576 data: 6 5 5 7 7 7
37532 136/300 Write block 4697 len 8192 data: 6 5
37533 137/300 Read block 515 len 16384
37534 138/300 Write block 3936 len 20480 data: 8 8 8 8 7
37535 139/300 Flush
37536 140/300 Read block 1131 len 12288
37537 141/300 Read block 2940 len 4096
37538 142/300 Write block 4163 len 40960 data: 8 7 6 6 5 5 5 6 7 8
37539 143/300 Flush
37540 144/300 Read block 2355 len 24576
37541 145/300 Read block 4188 len 12288
37542 146/300 Flush
37543 147/300 Read block 2152 len 8192
37544 148/300 Read block 1365 len 8192
37545 149/300 Read block 2181 len 32768
37546 150/300 Read block 173 len 8192
37547 151/300 Write block 2460 len 20480 data: 5 4 5 5 5
37548 152/300 Write block 3496 len 32768 data: 6 5 5 6 6 5 4 3
37549 153/300 Write block 69 len 32768 data: 9 9 9 10 9 8 8 9
37550 154/300 Write block 1199 len 8192 data: 6 7
37551 155/300 Read block 725 len 4096
37552 156/300 Write block 63 len 36864 data: 5 7 7 6 7 8 10 10 10
37553 157/300 Read block 3267 len 12288
37554 158/300 Write block 1483 len 28672 data: 9 8 7 6 6 6 6
37555 159/300 Read block 1629 len 12288
37556 160/300 Read block 3873 len 8192
37557 161/300 Read block 4130 len 28672
37558 162/300 Write block 3937 len 4096 data: 9
37559 163/300 Write block 3366 len 16384 data: 4 5 8 8
37560 164/300 Read block 2672 len 20480
37561 165/300 Read block 2885 len 36864
37562 166/300 Write block 1557 len 24576 data: 4 4 4 5 5 6
37563 167/300 Flush
37564 168/300 Read block 4668 len 8192
37565 169/300 Flush
37566 170/300 Read block 4689 len 32768
37567 171/300 Write block 545 len 36864 data: 8 7 8 7 6 6 7 5 5
37568 172/300 Read block 2081 len 20480
37569 173/300 Write block 1216 len 12288 data: 5 6 6
37570 174/300 Read block 4333 len 16384
37571 175/300 Write block 3462 len 20480 data: 7 7 6 6 4
37572 176/300 Read block 3385 len 28672
37573 177/300 Flush
37574 178/300 Read block 3943 len 20480
37575 179/300 Write block 4894 len 32768 data: 7 6 4 2 4 7 7 7
37576 180/300 Write block 3030 len 20480 data: 7 7 7 8 9
37577 181/300 Write block 592 len 16384 data: 6 6 6 6
37578 182/300 Flush
37579 183/300 Read block 2108 len 4096
37580 184/300 Read block 4646 len 8192
37581 185/300 Read block 2378 len 28672
37582 186/300 Write block 1949 len 12288 data: 9 8 8
37583 187/300 Read block 2837 len 24576
37584 188/300 Read block 922 len 24576
37585 189/300 Write block 2206 len 32768 data: 4 4 5 6 7 9 9 8
37586 190/300 Flush
37587 191/300 Read block 1648 len 36864
37588 192/300 Read block 4552 len 12288
37589 193/300 Write block 4616 len 32768 data: 7 8 6 6 5 4 5 6
37590 194/300 Read block 4371 len 28672
37591 195/300 Flush
37592 196/300 Read block 1799 len 4096
37593 197/300 Read block 1026 len 20480
37594 198/300 Flush
37595 199/300 Read block 1751 len 40960
37596 200/300 Write block 121 len 8192 data: 6 6
37597 201/300 Write block 1677 len 4096 data: 4
37598 202/300 Write block 1921 len 4096 data: 6
37599 203/300 Write block 3911 len 40960 data: 8 6 6 5 6 5 5 6 6 6
37600 204/300 Write block 3766 len 12288 data: 4 4 5
37601 205/300 Write block 566 len 12288 data: 4 3 3
37602 206/300 Write block 4695 len 36864 data: 8 6 7 6 4 5 6 6 5
37603 207/300 Write block 1996 len 4096 data: 3
37604 208/300 Read block 2637 len 28672
37605 209/300 Write block 3237 len 24576 data: 7 6 5 6 7 8
37606 210/300 Write block 3151 len 36864 data: 8 7 8 8 9 8 7 7 6
37607 211/300 Read block 358 len 4096
37608 212/300 Write block 4503 len 24576 data: 3 3 4 5 6 6
37609 213/300 Read block 1898 len 28672
37610 214/300 Read block 4937 len 8192
37611 215/300 Flush
37612 216/300 Write block 1256 len 36864 data: 5 5 6 7 8 6 5 7 6
37613 217/300 Read block 361 len 4096
37614 218/300 Read block 1197 len 16384
37615 219/300 Read block 2766 len 12288
37616 220/300 Read block 2402 len 8192
37617 221/300 Write block 1026 len 24576 data: 7 7 6 6 5 7
37618 222/300 Write block 403 len 40960 data: 6 6 6 7 7 9 10 11 9 7
37619 223/300 Read block 3239 len 28672
37620 224/300 Flush
37621 225/300 Write block 915 len 36864 data: 5 4 5 5 5 6 6 7 5
37622 226/300 Read block 2106 len 32768
37623 227/300 Read block 195 len 12288
37624 228/300 Write block 3150 len 4096 data: 7
37625 229/300 Flush
37626 230/300 Read block 4059 len 4096
37627 231/300 Read block 3642 len 8192
37628 232/300 Read block 4052 len 28672
37629 233/300 Read block 1343 len 40960
37630 234/300 Write block 2097 len 28672 data: 7 8 6 5 3 4 5
37631 235/300 Read block 1644 len 8192
37632 236/300 Flush
37633 237/300 Write block 2353 len 28672 data: 4 4 7 8 7 7 7
37634 238/300 Write block 3801 len 12288 data: 7 6 7
37635 239/300 Flush
37636 240/300 Read block 745 len 8192
37637 241/300 Read block 4229 len 36864
37638 242/300 Read block 2063 len 36864
37639 243/300 Read block 3880 len 20480
37640 244/300 Read block 2346 len 8192
37641 245/300 Read block 4572 len 16384
37642 246/300 Write block 1569 len 12288 data: 5 6 9
37643 247/300 Read block 1324 len 16384
37644 248/300 Read block 2873 len 32768
37645 249/300 Write block 4986 len 8192 data: 4 5
37646 250/300 Flush
37647 251/300 Read block 832 len 8192
37648 252/300 Read block 1799 len 36864
37649 253/300 Write block 1883 len 28672 data: 7 6 5 4 5 5 7
37650 254/300 Write block 4438 len 40960 data: 8 9 9 9 8 8 7 7 6 4
37651 255/300 Write block 1224 len 40960 data: 4 5 5 6 6 7 5 6 4 5
37652 256/300 Write block 2054 len 16384 data: 5 5 4 4
37653 257/300 Read block 4471 len 4096
37654 258/300 Read block 4289 len 28672
37655 259/300 Write block 4912 len 12288 data: 7 8 9
37656 260/300 Write block 4965 len 40960 data: 4 5 5 8 8 8 9 9 9 7
37657 261/300 Read block 3907 len 28672
37658 262/300 Read block 3438 len 24576
37659 263/300 Write block 1751 len 8192 data: 5 5
37660 264/300 Write block 454 len 40960 data: 4 3 3 2 3 3 4 4 5 6
37661 265/300 Write block 592 len 16384 data: 7 7 7 7
37662 266/300 Read block 2144 len 24576
37663 267/300 Read block 2807 len 24576
37664 268/300 Flush
37665 269/300 Write block 2177 len 12288 data: 4 5 5
37666 270/300 Write block 3192 len 40960 data: 4 5 6 6 6 5 4 4 6 6
37667 271/300 Read block 153 len 20480
37668 272/300 Read block 1284 len 4096
37669 273/300 Flush
37670 274/300 Write block 3648 len 36864 data: 10 8 7 8 11 10 7 7 8
37671 275/300 Read block 2454 len 36864
37672 276/300 Write block 3078 len 12288 data: 6 6 5
37673 277/300 Write block 2062 len 36864 data: 5 6 5 7 8 8 8 8 7
37674 278/300 Read block 4433 len 20480
37675 279/300 Read block 1433 len 40960
37676 280/300 Read block 4723 len 40960
37677 281/300 Write block 3156 len 4096 data: 9
37678 282/300 Read block 4896 len 24576
37679 283/300 Write block 3017 len 20480 data: 7 7 7 5 6
37680 284/300 Read block 4301 len 12288
37681 285/300 Write block 1659 len 16384 data: 5 8 8 8
37682 286/300 Read block 0 len 28672
37683 287/300 Read block 1883 len 4096
37684 288/300 Read block 3238 len 40960
37685 289/300 Write block 3867 len 20480 data: 12 11 11 11 11
37686 290/300 Read block 4226 len 36864
37687 291/300 Write block 1813 len 40960 data: 6 5 5 4 3 2 3 3 4 3
37688 292/300 Write block 979 len 20480 data: 7 7 7 6 6
37689 293/300 Read block 751 len 4096
37690 294/300 Read block 3432 len 16384
37691 295/300 Read block 1774 len 12288
37692 296/300 Read block 1511 len 24576
37693 297/300 Write block 1205 len 20480 data: 9 8 7 6 6
37694 298/300 Read block 2376 len 20480
37695 299/300 Read block 512 len 28672
37696 300/300 Write block 532 len 20480 data: 5 5 4 4 4
37697 301/300 Read block 439 len 4096
37698 Main task has work to do, go find it
37699 got dsc Start(1)
37700 start 1
37701 Dec 11 23:48:26.142 INFO request completed, latency_us: 272, response_code: 204, uri: /start/cid/1, method: POST, req_id: 65d55b8f-6e0c-4386-8d9e-535757720306, remote_addr: 127.0.0.1:62969, local_addr: 127.0.0.1:9998
37702 Replay: started 1, returned:Ok(())
37703 ----------------------------------------------------------------
37704 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
37705 States: Active Offline Active
37706 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
37707 10058 Acked Flush 0 Done Sent Done false
37708 10059 Acked Read 1 Done Sent Done false
37709 10060 Acked Read 7 Done Sent Done false
37710 10061 Acked Read 9 Done Sent Done false
37711 10062 Acked Write 9 Done Sent Done false
37712 10063 Acked Write 6 Done Sent Done false
37713 10064 Acked Flush 0 Done Sent Done false
37714 10065 Acked Read 6 Done Sent Done false
37715 10066 Acked Read 9 Done Sent Done false
37716 10067 Acked Read 7 Done Sent Done false
37717 10068 Acked Write 8 Done Sent Done false
37718 10069 Acked Write 5 Done Sent Done false
37719 10070 Acked Write 3 Done Sent Done false
37720 10071 Acked Flush 0 Done Sent Done false
37721 10072 Acked Read 1 Done Sent Done false
37722 10073 Acked Read 2 Done Sent Done false
37723 10074 Acked Read 10 Done Sent Done false
37724 10075 Acked Read 3 Done Sent Done false
37725 10076 Acked Read 9 Done Sent Done false
37726 10077 Acked Write 10 Done Sent Done false
37727 10078 Acked Write 3 Done Sent Done false
37728 10079 Acked Write 4 Done Sent Done false
37729 10080 Acked Read 6 Done Sent Done false
37730 10081 Acked Read 4 Done Sent Done false
37731 10082 Acked Read 1 Done Sent Done false
37732 10083 Acked Write 4 Done Sent Done false
37733 10084 Acked Write 3 Done Sent Done false
37734 10085 Acked Read 8 Done Sent Done false
37735 10086 Acked Flush 0 Done Sent Done false
37736 10087 Acked Write 7 Done Sent Done false
37737 10088 Acked Write 1 Done Sent Done false
37738 10089 Acked Write 4 Done Sent Done false
37739 10090 Acked Write 9 Done Sent Done false
37740 10091 Acked Write 1 Done Sent Done false
37741 10092 Acked Read 2 Done Sent Done false
37742 10093 Acked Read 2 Done Sent Done false
37743 10094 Acked Read 5 Done Sent Done false
37744 10095 Acked Flush 0 Done Sent Done false
37745 10096 Acked Flush 0 Done Sent Done false
37746 10097 Acked Write 1 Done Sent Done false
37747 10098 Acked Read 10 Done Sent Done false
37748 10099 Acked Write 1 Done Sent Done false
37749 10100 Acked Read 10 Done Sent Done false
37750 10101 Acked Read 6 Done Sent Done false
37751 10102 Acked Write 1 Done Sent Done false
37752 10103 Acked Write 3 Done Sent Done false
37753 10104 Acked Write 1 Done Sent Done false
37754 10105 Acked Write 8 Done Sent Done false
37755 10106 Acked Read 4 Done Sent Done false
37756 10107 Acked Write 10 Done Sent Done false
37757 10108 Acked Read 1 Done Sent Done false
37758 10109 Acked Read 2 Done Sent Done false
37759 10110 Acked Read 3 Done Sent Done false
37760 10111 Acked Read 8 Done Sent Done false
37761 10112 Acked Write 2 Done Sent Done false
37762 10113 Acked Read 5 Done Sent Done false
37763 10114 Acked Read 3 Done Sent Done false
37764 10115 Acked Read 5 Done Sent Done false
37765 10116 Acked Read 3 Done Sent Done false
37766 10117 Acked Flush 0 Done Sent Done false
37767 10118 Acked Read 6 Done Sent Done false
37768 10119 Acked Read 5 Done Sent Done false
37769 10120 Acked Read 5 Done Sent Done false
37770 10121 Acked Read 8 Done Sent Done false
37771 10122 Acked Flush 0 Done Sent Done false
37772 10123 Acked Read 2 Done Sent Done false
37773 10124 Acked Write 8 Done Sent Done false
37774 10125 Acked Write 6 Done Sent Done false
37775 10126 Acked Read 10 Done Sent Done false
37776 10127 Acked Write 1 Done Sent Done false
37777 10128 Acked Write 2 Done Sent Done false
37778 10129 Acked Read 4 Done Sent Done false
37779 10130 Acked Read 4 Done Sent Done false
37780 10131 Acked Read 1 Done Sent Done false
37781 10132 Acked Read 9 Done Sent Done false
37782 10133 Acked Read 5 Done Sent Done false
37783 10134 Acked Write 8 Done Sent Done false
37784 10135 Acked Read 4 Done Sent Done false
37785 10136 Acked Read 8 Done Sent Done false
37786 10137 Acked Write 3 Done Sent Done false
37787 10138 Acked Read 4 Done Sent Done false
37788 10139 Acked Read 3 Done Sent Done false
37789 10140 Acked Read 1 Done Sent Done false
37790 10141 Acked Read 1 Done Sent Done false
37791 10142 Acked Write 8 Done Sent Done false
37792 10143 Acked Read 5 Done Sent Done false
37793 10144 Acked Write 5 Done Sent Done false
37794 10145 Acked Read 7 Done Sent Done false
37795 10146 Acked Read 6 Done Sent Done false
37796 10147 Acked Write 3 Done Sent Done false
37797 10148 Acked Flush 0 Done Sent Done false
37798 10149 Acked Write 9 Done Sent Done false
37799 10150 Acked Flush 0 Done Sent Done false
37800 10151 Acked Flush 0 Done Sent Done false
37801 10152 Acked Read 9 Done Sent Done false
37802 10153 Acked Read 7 Done Sent Done false
37803 10154 Acked Write 4 Done Sent Done false
37804 10155 Acked Read 7 Done Sent Done false
37805 10156 Acked Read 8 Done Sent Done false
37806 10157 Acked Write 8 Done Sent Done false
37807 10158 Acked Read 3 Done Sent Done false
37808 10159 Acked Read 7 Done Sent Done false
37809 10160 Acked Read 1 Done Sent Done false
37810 10161 Acked Read 6 Done Sent Done false
37811 10162 Acked Read 10 Done Sent Done false
37812 10163 Acked Read 3 Done Sent Done false
37813 10164 Acked Write 6 Done Sent Done false
37814 10165 Acked Write 8 Done Sent Done false
37815 10166 Acked Write 3 Done Sent Done false
37816 10167 Acked Write 2 Done Sent Done false
37817 10168 Acked Read 2 Done Sent Done false
37818 10169 Acked Read 3 Done Sent Done false
37819 10170 Acked Read 2 Done Sent Done false
37820 10171 Acked Read 8 Done Sent Done false
37821 10172 Acked Read 2 Done Sent Done false
37822 10173 Acked Flush 0 Done Sent Done false
37823 10174 Acked Write 2 Done Sent Done false
37824 10175 Acked Write 3 Done Sent Done false
37825 10176 Acked Read 4 Done Sent Done false
37826 10177 Acked Write 4 Done Sent Done false
37827 10178 Acked Write 7 Done Sent Done false
37828 10179 Acked Write 2 Done Sent Done false
37829 10180 Acked Read 10 Done Sent Done false
37830 10181 Acked Flush 0 Done Sent Done false
37831 10182 Acked Write 5 Done Sent Done false
37832 10183 Acked Read 1 Done Sent Done false
37833 10184 Acked Write 2 Done Sent Done false
37834 10185 Acked Write 9 Done Sent Done false
37835 10186 Acked Write 4 Done Sent Done false
37836 10187 Acked Flush 0 Done Sent Done false
37837 10188 Acked Write 5 Done Sent Done false
37838 10189 Acked Flush 0 Done Sent Done false
37839 10190 Acked Write 4 Done Sent Done false
37840 10191 Acked Flush 0 Done Sent Done false
37841 10192 Acked Write 6 Done Sent Done false
37842 10193 Acked Write 2 Done Sent Done false
37843 10194 Acked Read 4 Done Sent Done false
37844 10195 Acked Write 5 Done Sent Done false
37845 10196 Acked Flush 0 Done Sent Done false
37846 10197 Acked Read 3 Done Sent Done false
37847 10198 Acked Read 1 Done Sent Done false
37848 10199 Acked Write 10 Done Sent Done false
37849 10200 Acked Flush 0 Done Sent Done false
37850 10201 Acked Read 6 Done Sent Done false
37851 10202 Acked Read 3 Done Sent Done false
37852 10203 Acked Flush 0 Done Sent Done false
37853 10204 Acked Read 2 Done Sent Done false
37854 10205 Acked Read 2 Done Sent Done false
37855 10206 Acked Read 8 Done Sent Done false
37856 10207 Acked Read 2 Done Sent Done false
37857 10208 Acked Write 5 Done Sent Done false
37858 10209 Acked Write 8 Done Sent Done false
37859 10210 Acked Write 8 Done Sent Done false
37860 10211 Acked Write 2 Done Sent Done false
37861 10212 Acked Read 1 Done Sent Done false
37862 10213 Acked Write 9 Done Sent Done false
37863 10214 Acked Read 3 Done Sent Done false
37864 10215 Acked Write 7 Done Sent Done false
37865 10216 Acked Read 3 Done Sent Done false
37866 10217 Acked Read 2 Done Sent Done false
37867 10218 Acked Read 7 Done Sent Done false
37868 10219 Acked Write 1 Done Sent Done false
37869 10220 Acked Write 4 Done Sent Done false
37870 10221 Acked Read 5 Done Sent Done false
37871 10222 Acked Read 9 Done Sent Done false
37872 10223 Acked Write 6 Done Sent Done false
37873 10224 Acked Flush 0 Done Sent Done false
37874 10225 Acked Read 2 Done Sent Done false
37875 10226 Acked Flush 0 Done Sent Done false
37876 10227 Acked Read 8 Done Sent Done false
37877 10228 Acked Write 9 Done Sent Done false
37878 10229 Acked Read 5 Done Sent Done false
37879 10230 Acked Write 3 Done Sent Done false
37880 10231 Acked Read 4 Done Sent Done false
37881 10232 Acked Write 5 Done Sent Done false
37882 10233 Acked Read 7 Done Sent Done false
37883 10234 Acked Flush 0 Done Sent Done false
37884 10235 Acked Read 5 Done Sent Done false
37885 10236 Acked Write 8 Done Sent Done false
37886 10237 Acked Write 5 Done Sent Done false
37887 10238 Acked Write 4 Done Sent Done false
37888 10239 Acked Flush 0 Done Sent Done false
37889 10240 Acked Read 1 Done Sent Done false
37890 10241 Acked Read 2 Done Sent Done false
37891 10242 Acked Read 7 Done Sent Done false
37892 10243 Acked Write 3 Done Sent Done false
37893 10244 Acked Read 6 Done Sent Done false
37894 10245 Acked Read 6 Done Sent Done false
37895 10246 Acked Write 8 Done Sent Done false
37896 10247 Acked Flush 0 Done Sent Done false
37897 10248 Acked Read 9 Done Sent Done false
37898 10249 Acked Read 3 Done Sent Done false
37899 10250 Acked Write 8 Done Sent Done false
37900 10251 Acked Read 7 Done Sent Done false
37901 10252 Acked Flush 0 Done Sent Done false
37902 10253 Acked Read 1 Done Sent Done false
37903 10254 Acked Read 5 Done Sent Done false
37904 10255 Acked Flush 0 Done Sent Done false
37905 10256 Acked Read 10 Done Sent Done false
37906 10257 Acked Write 2 Done Sent Done false
37907 10258 Acked Write 1 Done Sent Done false
37908 10259 Acked Write 1 Done Sent Done false
37909 10260 Acked Write 10 Done Sent Done false
37910 10261 Acked Write 3 Done Sent Done false
37911 10262 Acked Write 3 Done Sent Done false
37912 10263 Acked Write 9 Done Sent Done false
37913 10264 Acked Write 1 Done Sent Done false
37914 10265 Acked Read 7 Done Sent Done false
37915 10266 Acked Write 6 Done Sent Done false
37916 10267 Acked Write 9 Done Sent Done false
37917 10268 Acked Read 1 Done Sent Done false
37918 10269 Acked Write 6 Done Sent Done false
37919 10270 Acked Read 7 Done Sent Done false
37920 10271 Acked Read 2 Done Sent Done false
37921 10272 Acked Flush 0 Done Sent Done false
37922 10273 Acked Write 9 Done Sent Done false
37923 10274 Acked Read 1 Done Sent Done false
37924 10275 Acked Read 4 Done Sent Done false
37925 10276 Acked Read 3 Done Sent Done false
37926 10277 Acked Read 2 Done Sent Done false
37927 10278 Acked Write 6 Done Sent Done false
37928 10279 Acked Write 10 Done Sent Done false
37929 10280 Acked Read 7 Done Sent Done false
37930 10281 Acked Flush 0 Done Sent Done false
37931 10282 Acked Write 9 Done Sent Done false
37932 10283 Acked Read 8 Done Sent Done false
37933 10284 Acked Read 3 Done Sent Done false
37934 10285 Acked Write 1 Done Sent Done false
37935 10286 Acked Flush 0 Done Sent Done false
37936 10287 Acked Read 1 Done Sent Done false
37937 10288 Acked Read 2 Done Sent Done false
37938 10289 Acked Read 7 Done Sent Done false
37939 10290 Acked Read 10 Done Sent Done false
37940 10291 Acked Write 7 Done Sent Done false
37941 10292 Acked Read 2 Done Sent Done false
37942 10293 Acked Flush 0 Done Sent Done false
37943 10294 Acked Write 7 Done Sent Done false
37944 10295 Acked Write 3 Done Sent Done false
37945 10296 Acked Flush 0 Done Sent Done false
37946 10297 Acked Read 2 Done Sent Done false
37947 10298 Acked Read 9 Done Sent Done false
37948 10299 Acked Read 9 Done Sent Done false
37949 10300 Acked Read 5 Done Sent Done false
37950 10301 Acked Read 2 Done Sent Done false
37951 10302 Acked Read 4 Done Sent Done false
37952 10303 Acked Write 3 Done Sent Done false
37953 10304 Acked Read 4 Done Sent Done false
37954 10305 Acked Read 8 Done Sent Done false
37955 10306 Acked Write 2 Done Sent Done false
37956 10307 Acked Flush 0 Done Sent Done false
37957 10308 Acked Read 2 Done Sent Done false
37958 10309 Acked Read 9 Done Sent Done false
37959 10310 Acked Write 7 Done Sent Done false
37960 10311 Acked Write 10 Done Sent Done false
37961 10312 Acked Write 10 Done Sent Done false
37962 10313 Acked Write 4 Done Sent Done false
37963 10314 Acked Read 1 Done Sent Done false
37964 10315 Acked Read 7 Done Sent Done false
37965 10316 Acked Write 3 Done Sent Done false
37966 10317 Acked Write 10 Done Sent Done false
37967 10318 Acked Read 7 Done Sent Done false
37968 10319 Acked Read 6 Done Sent Done false
37969 10320 Acked Write 2 Done Sent Done false
37970 10321 Acked Write 10 Done Sent Done false
37971 10322 Acked Write 4 Done Sent Done false
37972 10323 Acked Read 6 Done Sent Done false
37973 10324 Acked Read 6 Done Sent Done false
37974 10325 Acked Flush 0 Done Sent Done false
37975 10326 Acked Write 3 Done Sent Done false
37976 10327 Acked Write 10 Done Sent Done false
37977 10328 Acked Read 5 Done Sent Done false
37978 10329 Acked Read 1 Done Sent Done false
37979 10330 Acked Flush 0 Done Sent Done false
37980 10331 Acked Write 9 Done Sent Done false
37981 10332 Acked Read 9 Done Sent Done false
37982 10333 Acked Write 3 Done Sent Done false
37983 10334 Acked Write 9 Done Sent Done false
37984 10335 Acked Read 5 Done Sent Done false
37985 10336 Acked Read 10 Done Sent Done false
37986 10337 Acked Read 10 Done Sent Done false
37987 10338 Acked Write 1 Done Sent Done false
37988 10339 Acked Read 6 Done Sent Done false
37989 10340 Acked Write 5 Done Sent Done false
37990 10341 Acked Read 3 Done Sent Done false
37991 10342 Acked Write 4 Done Sent Done false
37992 10343 Acked Read 7 Done Sent Done false
37993 10344 Acked Read 1 Done Sent Done false
37994 10345 Acked Read 10 Done Sent Done false
37995 10346 Acked Write 5 Done Sent Done false
37996 10347 Acked Read 9 Done Sent Done false
37997 10348 Acked Write 10 Done Sent Done false
37998 10349 Acked Write 5 Done Sent Done false
37999 10350 Acked Read 1 Done Sent Done false
38000 10351 Acked Read 4 Done Sent Done false
38001 10352 Acked Read 3 Done Sent Done false
38002 10353 Acked Read 6 Done Sent Done false
38003 10354 Acked Write 5 Done Sent Done false
38004 10355 Acked Read 5 Done Sent Done false
38005 10356 Acked Read 7 Done Sent Done false
38006 10357 Acked Write 5 Done Sent Done false
38007 10358 Acked Read 1 Done Sent Done false
38008 STATES DS:0 DS:1 DS:2 TOTAL
38009 Sent 0 301 0 301
38010 Done 301 0 301 602
38011 Skipped 0 0 0 0
38012 Error 0 0 0 0
38013 Last Flush: 10330 10057 10330
38014 Downstairs last five completed: 10057 10056 10055 10054 10053
38015 Upstairs last five completed: 10358 10357 10356 10355 10354
38016 CLIENT: Up:0 ds:301 act:2
38017 [8820] got start action
38018 [8820] I am going to restart
38019 Starting downstairs at port 8820
38020 Make output file at "/tmp/dsc/downstairs-8820.txt"
38021 [8820][1] reports Starting
38022 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1231)
38023 [8820][1] reports Running
38024 ----------------------------------------------------------------
38025 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
38026 States: Active Offline Active
38027 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
38028 10058 Acked Flush 0 Done Sent Done false
38029 10059 Acked Read 1 Done Sent Done false
38030 10060 Acked Read 7 Done Sent Done false
38031 10061 Acked Read 9 Done Sent Done false
38032 10062 Acked Write 9 Done Sent Done false
38033 10063 Acked Write 6 Done Sent Done false
38034 10064 Acked Flush 0 Done Sent Done false
38035 10065 Acked Read 6 Done Sent Done false
38036 10066 Acked Read 9 Done Sent Done false
38037 10067 Acked Read 7 Done Sent Done false
38038 10068 Acked Write 8 Done Sent Done false
38039 10069 Acked Write 5 Done Sent Done false
38040 10070 Acked Write 3 Done Sent Done false
38041 10071 Acked Flush 0 Done Sent Done false
38042 10072 Acked Read 1 Done Sent Done false
38043 10073 Acked Read 2 Done Sent Done false
38044 10074 Acked Read 10 Done Sent Done false
38045 10075 Acked Read 3 Done Sent Done false
38046 10076 Acked Read 9 Done Sent Done false
38047 10077 Acked Write 10 Done Sent Done false
38048 10078 Acked Write 3 Done Sent Done false
38049 10079 Acked Write 4 Done Sent Done false
38050 10080 Acked Read 6 Done Sent Done false
38051 10081 Acked Read 4 Done Sent Done false
38052 10082 Acked Read 1 Done Sent Done false
38053 10083 Acked Write 4 Done Sent Done false
38054 10084 Acked Write 3 Done Sent Done false
38055 10085 Acked Read 8 Done Sent Done false
38056 10086 Acked Flush 0 Done Sent Done false
38057 10087 Acked Write 7 Done Sent Done false
38058 10088 Acked Write 1 Done Sent Done false
38059 10089 Acked Write 4 Done Sent Done false
38060 10090 Acked Write 9 Done Sent Done false
38061 10091 Acked Write 1 Done Sent Done false
38062 10092 Acked Read 2 Done Sent Done false
38063 10093 Acked Read 2 Done Sent Done false
38064 10094 Acked Read 5 Done Sent Done false
38065 10095 Acked Flush 0 Done Sent Done false
38066 10096 Acked Flush 0 Done Sent Done false
38067 10097 Acked Write 1 Done Sent Done false
38068 10098 Acked Read 10 Done Sent Done false
38069 10099 Acked Write 1 Done Sent Done false
38070 10100 Acked Read 10 Done Sent Done false
38071 10101 Acked Read 6 Done Sent Done false
38072 10102 Acked Write 1 Done Sent Done false
38073 10103 Acked Write 3 Done Sent Done false
38074 10104 Acked Write 1 Done Sent Done false
38075 10105 Acked Write 8 Done Sent Done false
38076 10106 Acked Read 4 Done Sent Done false
38077 10107 Acked Write 10 Done Sent Done false
38078 10108 Acked Read 1 Done Sent Done false
38079 10109 Acked Read 2 Done Sent Done false
38080 10110 Acked Read 3 Done Sent Done false
38081 10111 Acked Read 8 Done Sent Done false
38082 10112 Acked Write 2 Done Sent Done false
38083 10113 Acked Read 5 Done Sent Done false
38084 10114 Acked Read 3 Done Sent Done false
38085 10115 Acked Read 5 Done Sent Done false
38086 10116 Acked Read 3 Done Sent Done false
38087 10117 Acked Flush 0 Done Sent Done false
38088 10118 Acked Read 6 Done Sent Done false
38089 10119 Acked Read 5 Done Sent Done false
38090 10120 Acked Read 5 Done Sent Done false
38091 10121 Acked Read 8 Done Sent Done false
38092 10122 Acked Flush 0 Done Sent Done false
38093 10123 Acked Read 2 Done Sent Done false
38094 10124 Acked Write 8 Done Sent Done false
38095 10125 Acked Write 6 Done Sent Done false
38096 10126 Acked Read 10 Done Sent Done false
38097 10127 Acked Write 1 Done Sent Done false
38098 10128 Acked Write 2 Done Sent Done false
38099 10129 Acked Read 4 Done Sent Done false
38100 10130 Acked Read 4 Done Sent Done false
38101 10131 Acked Read 1 Done Sent Done false
38102 10132 Acked Read 9 Done Sent Done false
38103 10133 Acked Read 5 Done Sent Done false
38104 10134 Acked Write 8 Done Sent Done false
38105 10135 Acked Read 4 Done Sent Done false
38106 10136 Acked Read 8 Done Sent Done false
38107 10137 Acked Write 3 Done Sent Done false
38108 10138 Acked Read 4 Done Sent Done false
38109 10139 Acked Read 3 Done Sent Done false
38110 10140 Acked Read 1 Done Sent Done false
38111 10141 Acked Read 1 Done Sent Done false
38112 10142 Acked Write 8 Done Sent Done false
38113 10143 Acked Read 5 Done Sent Done false
38114 10144 Acked Write 5 Done Sent Done false
38115 10145 Acked Read 7 Done Sent Done false
38116 10146 Acked Read 6 Done Sent Done false
38117 10147 Acked Write 3 Done Sent Done false
38118 10148 Acked Flush 0 Done Sent Done false
38119 10149 Acked Write 9 Done Sent Done false
38120 10150 Acked Flush 0 Done Sent Done false
38121 10151 Acked Flush 0 Done Sent Done false
38122 10152 Acked Read 9 Done Sent Done false
38123 10153 Acked Read 7 Done Sent Done false
38124 10154 Acked Write 4 Done Sent Done false
38125 10155 Acked Read 7 Done Sent Done false
38126 10156 Acked Read 8 Done Sent Done false
38127 10157 Acked Write 8 Done Sent Done false
38128 10158 Acked Read 3 Done Sent Done false
38129 10159 Acked Read 7 Done Sent Done false
38130 10160 Acked Read 1 Done Sent Done false
38131 10161 Acked Read 6 Done Sent Done false
38132 10162 Acked Read 10 Done Sent Done false
38133 10163 Acked Read 3 Done Sent Done false
38134 10164 Acked Write 6 Done Sent Done false
38135 10165 Acked Write 8 Done Sent Done false
38136 10166 Acked Write 3 Done Sent Done false
38137 10167 Acked Write 2 Done Sent Done false
38138 10168 Acked Read 2 Done Sent Done false
38139 10169 Acked Read 3 Done Sent Done false
38140 10170 Acked Read 2 Done Sent Done false
38141 10171 Acked Read 8 Done Sent Done false
38142 10172 Acked Read 2 Done Sent Done false
38143 10173 Acked Flush 0 Done Sent Done false
38144 10174 Acked Write 2 Done Sent Done false
38145 10175 Acked Write 3 Done Sent Done false
38146 10176 Acked Read 4 Done Sent Done false
38147 10177 Acked Write 4 Done Sent Done false
38148 10178 Acked Write 7 Done Sent Done false
38149 10179 Acked Write 2 Done Sent Done false
38150 10180 Acked Read 10 Done Sent Done false
38151 10181 Acked Flush 0 Done Sent Done false
38152 10182 Acked Write 5 Done Sent Done false
38153 10183 Acked Read 1 Done Sent Done false
38154 10184 Acked Write 2 Done Sent Done false
38155 10185 Acked Write 9 Done Sent Done false
38156 10186 Acked Write 4 Done Sent Done false
38157 10187 Acked Flush 0 Done Sent Done false
38158 10188 Acked Write 5 Done Sent Done false
38159 10189 Acked Flush 0 Done Sent Done false
38160 10190 Acked Write 4 Done Sent Done false
38161 10191 Acked Flush 0 Done Sent Done false
38162 10192 Acked Write 6 Done Sent Done false
38163 10193 Acked Write 2 Done Sent Done false
38164 10194 Acked Read 4 Done Sent Done false
38165 10195 Acked Write 5 Done Sent Done false
38166 10196 Acked Flush 0 Done Sent Done false
38167 10197 Acked Read 3 Done Sent Done false
38168 10198 Acked Read 1 Done Sent Done false
38169 10199 Acked Write 10 Done Sent Done false
38170 10200 Acked Flush 0 Done Sent Done false
38171 10201 Acked Read 6 Done Sent Done false
38172 10202 Acked Read 3 Done Sent Done false
38173 10203 Acked Flush 0 Done Sent Done false
38174 10204 Acked Read 2 Done Sent Done false
38175 10205 Acked Read 2 Done Sent Done false
38176 10206 Acked Read 8 Done Sent Done false
38177 10207 Acked Read 2 Done Sent Done false
38178 10208 Acked Write 5 Done Sent Done false
38179 10209 Acked Write 8 Done Sent Done false
38180 10210 Acked Write 8 Done Sent Done false
38181 10211 Acked Write 2 Done Sent Done false
38182 10212 Acked Read 1 Done Sent Done false
38183 10213 Acked Write 9 Done Sent Done false
38184 10214 Acked Read 3 Done Sent Done false
38185 10215 Acked Write 7 Done Sent Done false
38186 10216 Acked Read 3 Done Sent Done false
38187 10217 Acked Read 2 Done Sent Done false
38188 10218 Acked Read 7 Done Sent Done false
38189 10219 Acked Write 1 Done Sent Done false
38190 10220 Acked Write 4 Done Sent Done false
38191 10221 Acked Read 5 Done Sent Done false
38192 10222 Acked Read 9 Done Sent Done false
38193 10223 Acked Write 6 Done Sent Done false
38194 10224 Acked Flush 0 Done Sent Done false
38195 10225 Acked Read 2 Done Sent Done false
38196 10226 Acked Flush 0 Done Sent Done false
38197 10227 Acked Read 8 Done Sent Done false
38198 10228 Acked Write 9 Done Sent Done false
38199 10229 Acked Read 5 Done Sent Done false
38200 10230 Acked Write 3 Done Sent Done false
38201 10231 Acked Read 4 Done Sent Done false
38202 10232 Acked Write 5 Done Sent Done false
38203 10233 Acked Read 7 Done Sent Done false
38204 10234 Acked Flush 0 Done Sent Done false
38205 10235 Acked Read 5 Done Sent Done false
38206 10236 Acked Write 8 Done Sent Done false
38207 10237 Acked Write 5 Done Sent Done false
38208 10238 Acked Write 4 Done Sent Done false
38209 10239 Acked Flush 0 Done Sent Done false
38210 10240 Acked Read 1 Done Sent Done false
38211 10241 Acked Read 2 Done Sent Done false
38212 10242 Acked Read 7 Done Sent Done false
38213 10243 Acked Write 3 Done Sent Done false
38214 10244 Acked Read 6 Done Sent Done false
38215 10245 Acked Read 6 Done Sent Done false
38216 10246 Acked Write 8 Done Sent Done false
38217 10247 Acked Flush 0 Done Sent Done false
38218 10248 Acked Read 9 Done Sent Done false
38219 10249 Acked Read 3 Done Sent Done false
38220 10250 Acked Write 8 Done Sent Done false
38221 10251 Acked Read 7 Done Sent Done false
38222 10252 Acked Flush 0 Done Sent Done false
38223 10253 Acked Read 1 Done Sent Done false
38224 10254 Acked Read 5 Done Sent Done false
38225 10255 Acked Flush 0 Done Sent Done false
38226 10256 Acked Read 10 Done Sent Done false
38227 10257 Acked Write 2 Done Sent Done false
38228 10258 Acked Write 1 Done Sent Done false
38229 10259 Acked Write 1 Done Sent Done false
38230 10260 Acked Write 10 Done Sent Done false
38231 10261 Acked Write 3 Done Sent Done false
38232 10262 Acked Write 3 Done Sent Done false
38233 10263 Acked Write 9 Done Sent Done false
38234 10264 Acked Write 1 Done Sent Done false
38235 10265 Acked Read 7 Done Sent Done false
38236 10266 Acked Write 6 Done Sent Done false
38237 10267 Acked Write 9 Done Sent Done false
38238 10268 Acked Read 1 Done Sent Done false
38239 10269 Acked Write 6 Done Sent Done false
38240 10270 Acked Read 7 Done Sent Done false
38241 10271 Acked Read 2 Done Sent Done false
38242 10272 Acked Flush 0 Done Sent Done false
38243 10273 Acked Write 9 Done Sent Done false
38244 10274 Acked Read 1 Done Sent Done false
38245 10275 Acked Read 4 Done Sent Done false
38246 10276 Acked Read 3 Done Sent Done false
38247 10277 Acked Read 2 Done Sent Done false
38248 10278 Acked Write 6 Done Sent Done false
38249 10279 Acked Write 10 Done Sent Done false
38250 10280 Acked Read 7 Done Sent Done false
38251 10281 Acked Flush 0 Done Sent Done false
38252 10282 Acked Write 9 Done Sent Done false
38253 10283 Acked Read 8 Done Sent Done false
38254 10284 Acked Read 3 Done Sent Done false
38255 10285 Acked Write 1 Done Sent Done false
38256 10286 Acked Flush 0 Done Sent Done false
38257 10287 Acked Read 1 Done Sent Done false
38258 10288 Acked Read 2 Done Sent Done false
38259 10289 Acked Read 7 Done Sent Done false
38260 10290 Acked Read 10 Done Sent Done false
38261 10291 Acked Write 7 Done Sent Done false
38262 10292 Acked Read 2 Done Sent Done false
38263 10293 Acked Flush 0 Done Sent Done false
38264 10294 Acked Write 7 Done Sent Done false
38265 10295 Acked Write 3 Done Sent Done false
38266 10296 Acked Flush 0 Done Sent Done false
38267 10297 Acked Read 2 Done Sent Done false
38268 10298 Acked Read 9 Done Sent Done false
38269 10299 Acked Read 9 Done Sent Done false
38270 10300 Acked Read 5 Done Sent Done false
38271 10301 Acked Read 2 Done Sent Done false
38272 10302 Acked Read 4 Done Sent Done false
38273 10303 Acked Write 3 Done Sent Done false
38274 10304 Acked Read 4 Done Sent Done false
38275 10305 Acked Read 8 Done Sent Done false
38276 10306 Acked Write 2 Done Sent Done false
38277 10307 Acked Flush 0 Done Sent Done false
38278 10308 Acked Read 2 Done Sent Done false
38279 10309 Acked Read 9 Done Sent Done false
38280 10310 Acked Write 7 Done Sent Done false
38281 10311 Acked Write 10 Done Sent Done false
38282 10312 Acked Write 10 Done Sent Done false
38283 10313 Acked Write 4 Done Sent Done false
38284 10314 Acked Read 1 Done Sent Done false
38285 10315 Acked Read 7 Done Sent Done false
38286 10316 Acked Write 3 Done Sent Done false
38287 10317 Acked Write 10 Done Sent Done false
38288 10318 Acked Read 7 Done Sent Done false
38289 10319 Acked Read 6 Done Sent Done false
38290 10320 Acked Write 2 Done Sent Done false
38291 10321 Acked Write 10 Done Sent Done false
38292 10322 Acked Write 4 Done Sent Done false
38293 10323 Acked Read 6 Done Sent Done false
38294 10324 Acked Read 6 Done Sent Done false
38295 10325 Acked Flush 0 Done Sent Done false
38296 10326 Acked Write 3 Done Sent Done false
38297 10327 Acked Write 10 Done Sent Done false
38298 10328 Acked Read 5 Done Sent Done false
38299 10329 Acked Read 1 Done Sent Done false
38300 10330 Acked Flush 0 Done Sent Done false
38301 10331 Acked Write 9 Done Sent Done false
38302 10332 Acked Read 9 Done Sent Done false
38303 10333 Acked Write 3 Done Sent Done false
38304 10334 Acked Write 9 Done Sent Done false
38305 10335 Acked Read 5 Done Sent Done false
38306 10336 Acked Read 10 Done Sent Done false
38307 10337 Acked Read 10 Done Sent Done false
38308 10338 Acked Write 1 Done Sent Done false
38309 10339 Acked Read 6 Done Sent Done false
38310 10340 Acked Write 5 Done Sent Done false
38311 10341 Acked Read 3 Done Sent Done false
38312 10342 Acked Write 4 Done Sent Done false
38313 10343 Acked Read 7 Done Sent Done false
38314 10344 Acked Read 1 Done Sent Done false
38315 10345 Acked Read 10 Done Sent Done false
38316 10346 Acked Write 5 Done Sent Done false
38317 10347 Acked Read 9 Done Sent Done false
38318 10348 Acked Write 10 Done Sent Done false
38319 10349 Acked Write 5 Done Sent Done false
38320 10350 Acked Read 1 Done Sent Done false
38321 10351 Acked Read 4 Done Sent Done false
38322 10352 Acked Read 3 Done Sent Done false
38323 10353 Acked Read 6 Done Sent Done false
38324 10354 Acked Write 5 Done Sent Done false
38325 10355 Acked Read 5 Done Sent Done false
38326 10356 Acked Read 7 Done Sent Done false
38327 10357 Acked Write 5 Done Sent Done false
38328 10358 Acked Read 1 Done Sent Done false
38329 10359 Acked Flush 0 Done Sent Done false
38330 STATES DS:0 DS:1 DS:2 TOTAL
38331 Sent 0 302 0 302
38332 Done 302 0 302 604
38333 Skipped 0 0 0 0
38334 Error 0 0 0 0
38335 Last Flush: 10359 10057 10359
38336 Downstairs last five completed: 10057 10056 10055 10054 10053
38337 Upstairs last five completed: 10359 10358 10357 10356 10355
38338 CLIENT: Up:0 ds:302 act:2
383392024-12-11T23:48:31.912ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383402024-12-11T23:48:31.912ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63693) = io task client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383412024-12-11T23:48:31.913ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383422024-12-11T23:48:31.914ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383432024-12-11T23:48:31.914ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383442024-12-11T23:48:31.914ZINFOcrucible: send last flush ID to this DS: 10057 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383452024-12-11T23:48:31.914ZINFOcrucible: Replied this last flush ID: 10057 = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383462024-12-11T23:48:31.914ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383472024-12-11T23:48:31.914ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383482024-12-11T23:48:31.914ZINFOcrucible: [1] client re-new 302 jobs since flush 10057 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
383492024-12-11T23:48:31.914ZINFOcrucible: [1] Replayed 302 jobs since flush: 10057 = downstairs session_id = b03dd6e3-2411-404a-b84f-1ea6c44cc21f
38350 ----------------------------------------------------------------
38351 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
38352 Downstairs last five completed: 10359 10358 10357 10356 10355
38353 Upstairs last five completed: 10359 10358 10357 10356 10355
38354 CLIENT: Up:0 ds:0 act:3
38355 Replay: All jobs finished
38356 Test replay has completed
38357 Wrote out file "/tmp/test_replay_verify.log"
38358 CLIENT: Tests done. All submitted work has been ACK'd
38359 ----------------------------------------------------------------
38360 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
38361 Downstairs last five completed: 10359 10358 10357 10356 10355
38362 Upstairs last five completed: 10359 10358 10357 10356 10355
38363 CLIENT: Up:0 ds:0 act:3
38364 CLIENT: All jobs finished, all DS active.
383652024-12-11T23:48:34.149ZWARNcrucible: Guest handle has been dropped
38366 Replays:30 time: 8:00
38367 Do final verify
38368 Dec 11 23:48:34.222 INFO accepted connection, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
38369 Dec 11 23:48:34.222 INFO request completed, latency_us: 213, response_code: 200, uri: /regioninfo, method: GET, req_id: 66cf4fb6-de0c-4848-910b-2b4868583b9c, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
383702024-12-11T23:48:34.223ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
38371 Dec 11 23:48:34.223 INFO request completed, latency_us: 99, response_code: 200, uri: /regioncount, method: GET, req_id: 99f23bb4-6542-4f3b-8748-80f631aaa3e0, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
383722024-12-11T23:48:34.223ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
38373 Dec 11 23:48:34.224 INFO request completed, latency_us: 116, response_code: 200, uri: /port/cid/0, method: GET, req_id: 36b5623d-c640-4788-b23e-7f3a130b29c8, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
38374 Dec 11 23:48:34.224 INFO request completed, latency_us: 95, response_code: 200, uri: /port/cid/1, method: GET, req_id: 3ee00911-fe8a-4026-a88a-bb1e16c8317a, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
38375 Dec 11 23:48:34.225 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/2, method: GET, req_id: 7a91823d-6932-40ea-a417-9a4a9fb41aaf, remote_addr: 127.0.0.1:57191, local_addr: 127.0.0.1:9998
383762024-12-11T23:48:34.225ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
383772024-12-11T23:48:34.225ZINFOcrucible: Upstairs starts
383782024-12-11T23:48:34.225ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "d81f7fe7d3a87a2a3c58f4f5d619f7f37b0a05d0", git_commit_timestamp: "2024-12-11T23:24:14.000000000Z", git_branch: "mkeeter/separate-notify-task", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
383792024-12-11T23:48:34.225ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
383802024-12-11T23:48:34.225ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
383812024-12-11T23:48:34.226ZINFOcrucible: Crucible 7e13a35b-2275-404b-9c3f-e8a51caec9dd has session id: e77583dd-7a65-4e71-8dfd-f1d9aa2a2796 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383822024-12-11T23:48:34.226ZINFOcrucible: Upstairs opts: Upstairs UUID: 7e13a35b-2275-404b-9c3f-e8a51caec9dd, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383832024-12-11T23:48:34.226ZINFOcrucible: Crucible stats registered with UUID: 7e13a35b-2275-404b-9c3f-e8a51caec9dd session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383842024-12-11T23:48:34.226ZWARNcrucible: could not find Downstairs address for Nexus session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383852024-12-11T23:48:34.227ZINFOcrucible: Downstairs in volume = 3 task = crutest
383862024-12-11T23:48:34.227ZINFOcrucible: The guest has requested activation with gen:3
383872024-12-11T23:48:34.227ZINFOcrucible: 7e13a35b-2275-404b-9c3f-e8a51caec9dd active request set session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383882024-12-11T23:48:34.227ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383892024-12-11T23:48:34.227ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383902024-12-11T23:48:34.227ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383912024-12-11T23:48:34.227ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383922024-12-11T23:48:34.227ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383932024-12-11T23:48:34.227ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383942024-12-11T23:48:34.227ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383952024-12-11T23:48:34.227ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383962024-12-11T23:48:34.227ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383972024-12-11T23:48:34.227ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36172) = io task client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383982024-12-11T23:48:34.227ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61398) = io task client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
383992024-12-11T23:48:34.227ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:32950) = io task client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384002024-12-11T23:48:34.228ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384012024-12-11T23:48:34.228ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384022024-12-11T23:48:34.228ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384032024-12-11T23:48:34.228ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384042024-12-11T23:48:34.228ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384052024-12-11T23:48:34.228ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384062024-12-11T23:48:34.229ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384072024-12-11T23:48:34.229ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384082024-12-11T23:48:34.229ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384092024-12-11T23:48:34.229ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384102024-12-11T23:48:34.229ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384112024-12-11T23:48:34.229ZINFOcrucible: Waiting for more clients to be ready session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384122024-12-11T23:48:34.229ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384132024-12-11T23:48:34.229ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384142024-12-11T23:48:34.229ZINFOcrucible: Waiting for more clients to be ready session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384152024-12-11T23:48:34.230ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384162024-12-11T23:48:34.230ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384172024-12-11T23:48:34.230ZINFOcrucible: [0]R flush_numbers[0..12]: [967, 974, 959, 960, 980, 982, 950, 960, 951, 982, 975, 967] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384182024-12-11T23:48:34.230ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384192024-12-11T23:48:34.230ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384202024-12-11T23:48:34.230ZINFOcrucible: [1]R flush_numbers[0..12]: [967, 974, 959, 960, 980, 982, 950, 960, 951, 982, 975, 967] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384212024-12-11T23:48:34.230ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384222024-12-11T23:48:34.230ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384232024-12-11T23:48:34.230ZINFOcrucible: [2]R flush_numbers[0..12]: [967, 974, 959, 960, 980, 982, 950, 960, 951, 982, 975, 967] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384242024-12-11T23:48:34.230ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384252024-12-11T23:48:34.230ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384262024-12-11T23:48:34.230ZINFOcrucible: Max found gen is 3 = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384272024-12-11T23:48:34.230ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384282024-12-11T23:48:34.230ZINFOcrucible: Next flush: 983 = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384292024-12-11T23:48:34.230ZINFOcrucible: All extents match = downstairs session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384302024-12-11T23:48:34.230ZINFOcrucible: No downstairs reconciliation required session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384312024-12-11T23:48:34.230ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
38432 Wait for a query_work_queue command to finish before sending IO
384332024-12-11T23:48:34.230ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384342024-12-11T23:48:34.230ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384352024-12-11T23:48:34.230ZINFOcrucible: All required reconciliation work is completed session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384362024-12-11T23:48:34.230ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384372024-12-11T23:48:34.230ZINFOcrucible: 7e13a35b-2275-404b-9c3f-e8a51caec9dd is now active with session: e77583dd-7a65-4e71-8dfd-f1d9aa2a2796 session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384382024-12-11T23:48:34.230ZINFOcrucible: Set Active after no reconciliation session_id = e77583dd-7a65-4e71-8dfd-f1d9aa2a2796
384392024-12-11T23:48:34.230ZINFOcrucible: The guest has finished waiting for activation with:3
38440 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
38441 Loading write count information from file "/tmp/test_replay_verify.log"
38442 Read and Verify all blocks (0..5000 range:false)
38443 Wrote out file "/tmp/test_replay_verify.log"
38444 Verify test completed
38445 Wrote out file "/tmp/test_replay_verify.log"
38446 CLIENT: Tests done. All submitted work has been ACK'd
38447 ----------------------------------------------------------------
38448 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
38449 States: Active Active Active
38450 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
38451 1000 Acked Read 256 Done Done Done false
38452 1001 Acked Read 256 Done Done Done false
38453 1002 Acked Read 256 Done Done Done false
38454 1003 Acked Read 256 Done Done Done false
38455 1004 Acked Read 256 Done Done Done false
38456 1005 Acked Read 256 Done Done Done false
38457 1006 Acked Read 256 Done Done Done false
38458 1007 Acked Read 256 Done Done Done false
38459 1008 Acked Read 256 Done Done Done false
38460 1009 Acked Read 256 Done Done Done false
38461 1010 Acked Read 256 Done Done Done false
38462 1011 Acked Read 256 Done Done Done false
38463 1012 Acked Read 256 Done Done Done false
38464 1013 Acked Read 256 Done Done Done false
38465 1014 Acked Read 256 Done Done Done false
38466 1015 Acked Read 256 Done Done Done false
38467 1016 Acked Read 256 Done Done Done false
38468 1017 Acked Read 136 Done Done Done false
38469 1018 Acked Read 256 Done Done Done false
38470 1019 Acked Read 256 Done Done Done false
38471 STATES DS:0 DS:1 DS:2 TOTAL
38472 Sent 0 0 0 0
38473 Done 20 20 20 60
38474 Skipped 0 0 0 0
38475 Error 0 0 0 0
38476 Last Flush: 0 0 0
38477 Downstairs last five completed:
38478 Upstairs last five completed: 1019 1018 1017 1016 1015
38479 CLIENT: Up:0 ds:20 act:3
38480 ----------------------------------------------------------------
38481 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
38482 Downstairs last five completed: 1020 1019 1018 1017 1016
38483 Upstairs last five completed: 1020 1019 1018 1017 1016
38484 CLIENT: Up:0 ds:0 act:3
38485 CLIENT: All crucible jobs finished, exiting program
384862024-12-11T23:48:38.637ZWARNcrucible: Guest handle has been dropped
38487 Dec 11 23:48:38.710 INFO accepted connection, remote_addr: 127.0.0.1:64725, local_addr: 127.0.0.1:9998
38488 Main task has work to do, go find it
38489 got dsc Shutdown
38490 Shutdown
38491 Dec 11 23:48:38.711 INFO request completed, latency_us: 165, response_code: 204, uri: /shutdown, method: POST, req_id: 8b85830a-88b4-46ff-87de-8d0bb468642a, remote_addr: 127[8810] Disable keep_running
38492 .Shut it down
38493 0.0.1:64725, local_addr: 127.0.0.1:9998[8810] Got stop action so:false kr:false
38494 
38495 [8820] Disable keep_running
38496 [8820] Got stop action so:false kr:false
38497 [8830] Disable keep_running
38498 [8830] Got stop action so:false kr:false
38499 [8810] Exited with: ExitStatus(unix_wait_status(9))
38500 [8810][0] reports Exit
38501 [8820] Exited with: ExitStatus(unix_wait_status(9))
38502 [8820][1] reports Exit
38503 [8830] Exited with: ExitStatus(unix_wait_status(9))
38504 [8830][2] reports Exit
38505 December 11, 2024 at 11:48:43 PM UTC Test ends with 0