Max level shown:
1 starting December 4, 2024 at 01:30:35 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.26461485
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.02067219
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.018952224
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start access at:127.0.0.1:9998
14 start ds: 8820
15 start ds: 8830
16 Starting downstairs at port 8810
17 Make output file at "/tmp/dsc/downstairs-8810.txt"
18 Starting downstairs at port 8820
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 [8810][0] initial start wait reports Starting
21 [8820][1] initial start wait reports Starting
22 Starting downstairs at port 8830
23 Make output file at "/tmp/dsc/downstairs-8830.txt"
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1193)
26 [8810][0] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1194)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1195)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Dec 04 13:30:36.313 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Dec 04 13:30:41.391 INFO accepted connection, remote_addr: 127.0.0.1:49967, local_addr: 127.0.0.1:9998
36 Dec 04 13:30:41.393 INFO request completed, latency_us: 432, response_code: 200, uri: /regioninfo, method: GET, req_id: 8a8ee706-453f-4675-8486-c0ddc408d6ef, remote_addr: 127.0.0.1:49967, local_addr: 127.0.0.1:9998
37 Dec 04 13:30:41.395 INFO request completed, latency_us: 93, response_code: 200, uri: /regioncount, method: GET, req_id: a1f84872-c003-44ef-91e8-bfa8d8a48d75, remote_addr: 127.0.0.1:49967, local_addr: 127.0.0.1:9998
382024-12-04T13:30:41.394ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392024-12-04T13:30:41.395ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Dec 04 13:30:41.395 INFO request completed, latency_us: 132, response_code: 200, uri: /port/cid/0, method: GET, req_id: ccbacd4d-683f-4cac-b86f-de68defdd224, remote_addr: 127.0.0.1:49967, local_addr: 127.0.0.1:9998
41 Dec 04 13:30:41.396 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/1, method: GET, req_id: 33a49047-90ff-4d22-81ce-0284bbc58b8f, remote_addr: 127.0.0.1:49967, local_addr: 127.0.0.1:9998
42 Dec 04 13:30:41.396 INFO request completed, latency_us: 92, response_code: 200, uri: /port/cid/2, method: GET, req_id: a0991501-0c41-4442-b625-c165e1613825, remote_addr: 127.0.0.1:49967, local_addr: 127.0.0.1:9998
432024-12-04T13:30:41.397ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442024-12-04T13:30:41.397ZINFOcrucible: Upstairs starts
452024-12-04T13:30:41.397ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "0e1f7f60aff9f26b0d737c97120445edbded1ad3", git_commit_timestamp: "2024-12-04T13:14:36.000000000Z", git_branch: "renovate/tokio-util-0.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462024-12-04T13:30:41.397ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472024-12-04T13:30:41.398ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482024-12-04T13:30:41.398ZINFOcrucible: Crucible 6173079c-3dbe-45ca-90e5-fdf45f49928a has session id: cee19b1e-6987-482d-9805-42f298ce5d3a session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
492024-12-04T13:30:41.398ZINFOcrucible: Upstairs opts: Upstairs UUID: 6173079c-3dbe-45ca-90e5-fdf45f49928a, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
502024-12-04T13:30:41.398ZINFOcrucible: Crucible stats registered with UUID: 6173079c-3dbe-45ca-90e5-fdf45f49928a session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
512024-12-04T13:30:41.433ZINFOcrucible: Downstairs in volume = 3 task = crutest
522024-12-04T13:30:41.433ZINFOcrucible: The guest has requested activation with gen:1
532024-12-04T13:30:41.433ZINFOcrucible: 6173079c-3dbe-45ca-90e5-fdf45f49928a active request set session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
542024-12-04T13:30:41.433ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
552024-12-04T13:30:41.433ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
562024-12-04T13:30:41.433ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
572024-12-04T13:30:41.433ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
582024-12-04T13:30:41.433ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
592024-12-04T13:30:41.433ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
602024-12-04T13:30:41.433ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
612024-12-04T13:30:41.433ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
622024-12-04T13:30:41.433ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
632024-12-04T13:30:41.433ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43809) = io task client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
642024-12-04T13:30:41.433ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36179) = io task client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
652024-12-04T13:30:41.433ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42865) = io task client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
662024-12-04T13:30:41.435ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
672024-12-04T13:30:41.435ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
682024-12-04T13:30:41.435ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
692024-12-04T13:30:41.435ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
702024-12-04T13:30:41.435ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
712024-12-04T13:30:41.435ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
722024-12-04T13:30:41.436ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
732024-12-04T13:30:41.437ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
742024-12-04T13:30:41.437ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
752024-12-04T13:30:41.437ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
762024-12-04T13:30:41.437ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
772024-12-04T13:30:41.437ZINFOcrucible: Waiting for more clients to be ready session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
782024-12-04T13:30:41.437ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
792024-12-04T13:30:41.437ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
802024-12-04T13:30:41.437ZINFOcrucible: Waiting for more clients to be ready session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
812024-12-04T13:30:41.438ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
822024-12-04T13:30:41.438ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
832024-12-04T13:30:41.438ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
842024-12-04T13:30:41.438ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
852024-12-04T13:30:41.438ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
862024-12-04T13:30:41.438ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
872024-12-04T13:30:41.438ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
882024-12-04T13:30:41.438ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
892024-12-04T13:30:41.438ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
902024-12-04T13:30:41.438ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
912024-12-04T13:30:41.438ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
922024-12-04T13:30:41.438ZINFOcrucible: Max found gen is 1 = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
932024-12-04T13:30:41.438ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
942024-12-04T13:30:41.438ZINFOcrucible: Next flush: 1 = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
952024-12-04T13:30:41.438ZINFOcrucible: All extents match = downstairs session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
962024-12-04T13:30:41.438ZINFOcrucible: No downstairs reconciliation required session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
972024-12-04T13:30:41.438ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
982024-12-04T13:30:41.438ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
992024-12-04T13:30:41.438ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
1002024-12-04T13:30:41.438ZINFOcrucible: All required reconciliation work is completed session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
1012024-12-04T13:30:41.438ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
1022024-12-04T13:30:41.438ZINFOcrucible: 6173079c-3dbe-45ca-90e5-fdf45f49928a is now active with session: cee19b1e-6987-482d-9805-42f298ce5d3a session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
1032024-12-04T13:30:41.438ZINFOcrucible: Set Active after no reconciliation session_id = cee19b1e-6987-482d-9805-42f298ce5d3a
1042024-12-04T13:30:41.438ZINFOcrucible: The guest has finished waiting for activation with:1
105 Activate successful
106 Wait for a query_work_queue command to finish before sending IO
107 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1021 Acked Read 256 Done Done Done false
117 1022 Acked Read 256 Done Done Done false
118 1023 Acked Read 256 Done Done Done false
119 1024 Acked Read 256 Done Done Done false
120 1025 Acked Read 256 Done Done Done false
121 1026 Acked Read 256 Done Done Done false
122 1027 Acked Read 256 Done Done Done false
123 1028 Acked Read 256 Done Done Done false
124 1029 Acked Read 256 Done Done Done false
125 1030 Acked Read 256 Done Done Done false
126 1031 Acked Read 256 Done Done Done false
127 1032 Acked Read 256 Done Done Done false
128 1033 Acked Read 256 Done Done Done false
129 1034 Acked Read 256 Done Done Done false
130 1035 Acked Read 256 Done Done Done false
131 1036 Acked Read 256 Done Done Done false
132 1037 Acked Read 256 Done Done Done false
133 1038 Acked Read 136 Done Done Done false
134 1039 Acked Read 256 Done Done Done false
135 1040 Acked Read 256 Done Done Done false
136 STATES DS:0 DS:1 DS:2 TOTAL
137 Sent 0 0 0 0
138 Done 20 20 20 60
139 Skipped 0 0 0 0
140 Error 0 0 0 0
141 Last Flush: 1020 1020 1020
142 Downstairs last five completed: 1020 1019 1018 1017 1016
143 Upstairs last five completed: 1040 1039 1038 1037 1036
144 CLIENT: Up:0 ds:20 act:3
145 ----------------------------------------------------------------
146 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
147 Downstairs last five completed: 1041 1040 1039 1038 1037
148 Upstairs last five completed: 1041 1040 1039 1038 1037
149 CLIENT: Up:0 ds:0 act:3
150 CLIENT: All crucible jobs finished, exiting program
151 Replay loop starts now December 4, 2024 at 01:30:46 PM UTC
152 Dec 04 13:30:46.079 INFO accepted connection, remote_addr: 127.0.0.1:33230, local_addr: 127.0.0.1:9998
153 Dec 04 13:30:46.080 INFO request completed, latency_us: 172, response_code: 200, uri: /regioninfo, method: GET, req_id: 7ef9829e-12d5-4c56-8ede-1022d7aa5d56, remote_addr: 127.0.0.1:33230, local_addr: 127.0.0.1:9998
1542024-12-04T13:30:46.081ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
155 Dec 04 13:30:46.081 INFO request completed, latency_us: 91, response_code: 200, uri: /regioncount, method: GET, req_id: fb286497-ce00-4227-b30a-43b650e5299f, remote_addr: 127.0.0.1:33230, local_addr: 127.0.0.1:9998
1562024-12-04T13:30:46.081ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
157 Dec 04 13:30:46.082 INFO request completed, latency_us: 119, response_code: 200, uri: /port/cid/0, method: GET, req_id: 4852aedb-714e-4669-9230-e14f7bec68bd, remote_addr: 127.0.0.1:33230, local_addr: 127.0.0.1:9998
158 Dec 04 13:30:46.082 INFO request completed, latency_us: 95, response_code: 200, uri: /port/cid/1, method: GET, req_id: ee3bf06f-0a71-435c-9912-83e09cebe0a9, remote_addr: 127.0.0.1:33230, local_addr: 127.0.0.1:9998
159 Dec 04 13:30:46.083 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/2, method: GET, req_id: 162c1481-ff81-48a3-839a-4497fc797625, remote_addr: 127.0.0.1:33230, local_addr: 127.0.0.1:9998
1602024-12-04T13:30:46.083ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1612024-12-04T13:30:46.083ZINFOcrucible: Upstairs starts
1622024-12-04T13:30:46.083ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "0e1f7f60aff9f26b0d737c97120445edbded1ad3", git_commit_timestamp: "2024-12-04T13:14:36.000000000Z", git_branch: "renovate/tokio-util-0.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1632024-12-04T13:30:46.083ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1642024-12-04T13:30:46.083ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1652024-12-04T13:30:46.084ZINFOcrucible: Crucible 8599ae8a-2d59-489f-9c00-7e1af64307e4 has session id: 01834428-904b-4e36-8fe6-2e6a94afb6ff session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1662024-12-04T13:30:46.084ZINFOcrucible: Upstairs opts: Upstairs UUID: 8599ae8a-2d59-489f-9c00-7e1af64307e4, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1672024-12-04T13:30:46.084ZINFOcrucible: Crucible stats registered with UUID: 8599ae8a-2d59-489f-9c00-7e1af64307e4 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1682024-12-04T13:30:46.117ZINFOcrucible: Downstairs in volume = 3 task = crutest
1692024-12-04T13:30:46.117ZINFOcrucible: The guest has requested activation with gen:2
1702024-12-04T13:30:46.117ZINFOcrucible: 8599ae8a-2d59-489f-9c00-7e1af64307e4 active request set session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1712024-12-04T13:30:46.117ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1722024-12-04T13:30:46.117ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1732024-12-04T13:30:46.117ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1742024-12-04T13:30:46.117ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1752024-12-04T13:30:46.117ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1762024-12-04T13:30:46.117ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1772024-12-04T13:30:46.117ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1782024-12-04T13:30:46.117ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1792024-12-04T13:30:46.117ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1802024-12-04T13:30:46.117ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36703) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1812024-12-04T13:30:46.117ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52430) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1822024-12-04T13:30:46.117ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61078) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1832024-12-04T13:30:46.118ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1842024-12-04T13:30:46.118ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1852024-12-04T13:30:46.118ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1862024-12-04T13:30:46.118ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1872024-12-04T13:30:46.118ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1882024-12-04T13:30:46.118ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1892024-12-04T13:30:46.118ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1902024-12-04T13:30:46.118ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1912024-12-04T13:30:46.118ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1922024-12-04T13:30:46.119ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1932024-12-04T13:30:46.119ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1942024-12-04T13:30:46.119ZINFOcrucible: Waiting for more clients to be ready session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1952024-12-04T13:30:46.119ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1962024-12-04T13:30:46.119ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1972024-12-04T13:30:46.119ZINFOcrucible: Waiting for more clients to be ready session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1982024-12-04T13:30:46.119ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1992024-12-04T13:30:46.119ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2002024-12-04T13:30:46.119ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2012024-12-04T13:30:46.119ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2022024-12-04T13:30:46.119ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2032024-12-04T13:30:46.119ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2042024-12-04T13:30:46.119ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
205 {"msg":"[1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:30:46.119923954Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"downstairs","":"downstairs"Activate successful
206 ,Wait for a query_work_queue command to finish before sending IO
207 "session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
2082024-12-04T13:30:46.119ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2092024-12-04T13:30:46.119ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2102024-12-04T13:30:46.120ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2112024-12-04T13:30:46.120ZINFOcrucible: Max found gen is 2 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2122024-12-04T13:30:46.120ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2132024-12-04T13:30:46.120ZINFOcrucible: Next flush: 2 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
214 {"msg":"All extents match","v":0,"name":"crucible","level":30Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
215 ,"time":"2024-12-04T13:30:46.120152016Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"downstairs","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
2162024-12-04T13:30:46.120ZINFOcrucible: No downstairs reconciliation required session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2172024-12-04T13:30:46.120ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2182024-12-04T13:30:46.120ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2192024-12-04T13:30:46.120ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2202024-12-04T13:30:46.120ZINFOcrucible: All required reconciliation work is completed session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2212024-12-04T13:30:46.120ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2222024-12-04T13:30:46.120ZINFOcrucible: 8599ae8a-2d59-489f-9c00-7e1af64307e4 is now active with session: 01834428-904b-4e36-8fe6-2e6a94afb6ff session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2232024-12-04T13:30:46.120ZINFOcrucible: Set Active after no reconciliation session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2242024-12-04T13:30:46.120ZINFOcrucible: The guest has finished waiting for activation with:2
225 Loading write count information from file "/tmp/test_replay_verify.log"
226 Dec 04 13:30:46.152 INFO accepted connection, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
227 Main task has work to do, go find it
228 got dsc Stop(0)
229 stop 0
230 Dec 04 13:30:46.152 INFO request completed, latency_us: 167, response_code: 204, uri: /stop/cid/0, method: POST, req_id: d36c14c9-4436-451f-86fd-ded10b91b908, remote_addr: 127.[8810] Got stop action so:false kr:false
231 0.0.1:37804, local_addr: 127.0.0.1:9998
2322024-12-04T13:30:46.153ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2332024-12-04T13:30:46.153ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2342024-12-04T13:30:46.153ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2352024-12-04T13:30:46.153ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2362024-12-04T13:30:46.153ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
237 Dec 04 13:30:46.153 INFO request completed, latency_us: 124, response_code: 200, uri: /state/cid/0, method: GET, req_id: 038956b6-f66f-450c-8a9c-2f1ba4e440bb, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
2382024-12-04T13:30:46.153ZINFOcrucible: client task is exiting = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
239 [8810] Exited with: ExitStatus(unix_wait_status(9))
240 [8810][0] reports Exit
241 Dec 04 13:30:50.155 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/0, method: GET, req_id: 9cca4bf5-cc5e-4552-bc7c-c1a38e9bef01, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
242 001/300 Read block 2839 len 20480
243 002/300 Write block 1929 len 40960 data: 2 2 2 2 2 2 2 2 2 2
244 003/300 Write block 1 len 16384 data: 2 2 2 2
245 004/300 Read block 732 len 20480
246 005/300 Read block 1228 len 4096
247 006/300 Write block 4633 len 4096 data: 2
248 007/300 Write block 1622 len 4096 data: 2
249 008/300 Write block 4057 len 40960 data: 2 2 2 2 2 2 2 2 2 2
250 009/300 Read block 3160 len 24576
251 010/300 Read block 2967 len 32768
252 011/300 Read block 4585 len 20480
253 012/300 Write block 1741 len 36864 data: 2 2 2 2 2 2 2 2 2
254 013/300 Read block 2672 len 20480
255 014/300 Write block 1216 len 12288 data: 2 2 2
256 015/300 Read block 2609 len 20480
257 016/300 Read block 1873 len 28672
258 017/300 Flush
259 018/300 Flush
260 019/300 Read block 2620 len 12288
261 020/300 Write block 1859 len 20480 data: 2 2 2 2 2
262 021/300 Flush
263 022/300 Read block 3318 len 16384
264 023/300 Read block 4360 len 4096
265 024/300 Flush
266 025/300 Write block 2139 len 20480 data: 2 2 2 2 2
267 026/300 Write block 3101 len 20480 data: 2 2 2 2 2
268 027/300 Read block 3848 len 4096
269 028/300 Read block 4162 len 16384
270 029/300 Read block 2611 len 24576
271 030/300 Write block 770 len 12288 data: 2 2 2
272 031/300 Read block 2613 len 32768
273 032/300 Read block 972 len 16384
274 033/300 Write block 3569 len 16384 data: 2 2 2 2
275 034/300 Write block 804 len 32768 data: 2 2 2 2 2 2 2 2
276 035/300 Write block 164 len 32768 data: 2 2 2 2 2 2 2 2
277 036/300 Write block 153 len 40960 data: 2 2 2 2 2 2 2 2 2 2
278 037/300 Write block 3050 len 36864 data: 2 2 2 2 2 2 2 2 2
279 038/300 Write block 1518 len 28672 data: 2 2 2 2 2 2 2
280 039/300 Flush
281 040/300 Write block 1479 len 28672 data: 2 2 2 2 2 2 2
282 041/300 Read block 2698 len 36864
283 042/300 Write block 2456 len 36864 data: 2 2 2 2 2 2 2 2 2
284 043/300 Flush
285 044/300 Write block 2540 len 32768 data: 2 2 2 2 2 2 2 2
286 045/300 Read block 364 len 36864
287 046/300 Write block 40 len 24576 data: 2 2 2 2 2 2
288 047/300 Write block 1398 len 12288 data: 2 2 2
289 048/300 Write block 1781 len 16384 data: 2 2 2 2
290 049/300 Read block 1594 len 24576
291 050/300 Write block 1505 len 32768 data: 2 2 2 2 2 2 2 2
292 051/300 Write block 3715 len 32768 data: 2 2 2 2 2 2 2 2
293 052/300 Read block 3715 len 4096
294 053/300 Read block 250 len 8192
295 054/300 Write block 2631 len 16384 data: 2 2 2 2
296 055/300 Write block 4053 len 36864 data: 2 2 2 2 3 3 3 3 3
297 056/300 Write block 2 len 4096 data: 3
298 057/300 Read block 4644 len 28672
299 058/300 Write block 1807 len 36864 data: 2 2 2 2 2 2 2 2 2
300 059/300 Read block 4065 len 40960
301 060/300 Read block 3435 len 16384
302 061/300 Read block 638 len 32768
303 062/300 Flush
304 063/300 Write block 1242 len 36864 data: 2 2 2 2 2 2 2 2 2
305 064/300 Read block 4962 len 36864
306 065/300 Flush
307 066/300 Read block 2732 len 12288
308 067/300 Read block 389 len 8192
309 068/300 Read block 1289 len 36864
310 069/300 Read block 2891 len 4096
311 070/300 Read block 3691 len 12288
312 071/300 Read block 3314 len 24576
313 072/300 Read block 1552 len 16384
314 073/300 Write block 2395 len 20480 data: 2 2 2 2 2
315 074/300 Read block 1868 len 20480
316 075/300 Read block 2113 len 4096
317 076/300 Read block 4238 len 16384
318 077/300 Write block 1994 len 4096 data: 2
319 078/300 Write block 3151 len 24576 data: 2 2 2 2 2 2
320 079/300 Read block 1613 len 36864
321 080/300 Flush
322 081/300 Write block 3360 len 28672 data: 2 2 2 2 2 2 2
323 082/300 Write block 2851 len 8192 data: 2 2
324 083/300 Read block 542 len 4096
325 084/300 Read block 766 len 16384
326 085/300 Write block 3362 len 36864 data: 3 3 3 3 3 2 2 2 2
327 086/300 Write block 2545 len 20480 data: 3 3 3 2 2
328 087/300 Read block 536 len 24576
329 088/300 Read block 3256 len 4096
330 089/300 Write block 4274 len 24576 data: 2 2 2 2 2 2
331 090/300 Write block 2738 len 16384 data: 2 2 2 2
332 091/300 Write block 486 len 24576 data: 2 2 2 2 2 2
333 092/300 Read block 4929 len 24576
334 093/300 Write block 337 len 16384 data: 2 2 2 2
335 094/300 Read block 2939 len 36864
336 095/300 Read block 1956 len 8192
337 096/300 Write block 745 len 20480 data: 2 2 2 2 2
338 097/300 Read block 1136 len 24576
339 098/300 Read block 1894 len 40960
340 099/300 Read block 3608 len 32768
341 100/300 Write block 4462 len 36864 data: 2 2 2 2 2 2 2 2 2
342 101/300 Read block 1835 len 28672
343 102/300 Write block 147 len 12288 data: 2 2 2
344 103/300 Read block 4566 len 12288
345 104/300 Write block 4605 len 28672 data: 2 2 2 2 2 2 2
346 105/300 Read block 4011 len 20480
347 106/300 Write block 4155 len 36864 data: 2 2 2 2 2 2 2 2 2
348 107/300 Flush
349 108/300 Read block 1761 len 12288
350 109/300 Flush
351 110/300 Write block 4926 len 4096 data: 2
352 111/300 Write block 446 len 20480 data: 2 2 2 2 2
353 112/300 Read block 4669 len 28672
354 113/300 Write block 2589 len 8192 data: 2 2
355 114/300 Read block 4971 len 28672
356 115/300 Read block 724 len 32768
357 116/300 Flush
358 117/300 Read block 4550 len 28672
359 118/300 Read block 3295 len 28672
360 119/300 Read block 4348 len 20480
361 120/300 Write block 2368 len 24576 data: 2 2 2 2 2 2
362 121/300 Read block 4732 len 40960
363 122/300 Read block 2783 len 16384
364 123/300 Read block 3830 len 8192
365 124/300 Write block 4867 len 12288 data: 2 2 2
366 125/300 Read block 187 len 36864
367 126/300 Read block 4207 len 24576
368 127/300 Read block 422 len 40960
369 128/300 Write block 2120 len 36864 data: 2 2 2 2 2 2 2 2 2
370 129/300 Flush
371 130/300 Read block 2838 len 36864
372 131/300 Read block 131 len 8192
373 132/300 Write block 3945 len 24576 data: 2 2 2 2 2 2
374 133/300 Write block 3802 len 4096 data: 2
375 134/300 Read block 3086 len 4096
376 135/300 Write block 600 len 16384 data: 2 2 2 2
377 136/300 Write block 1318 len 36864 data: 2 2 2 2 2 2 2 2 2
378 137/300 Read block 4125 len 32768
379 138/300 Read block 1546 len 8192
380 139/300 Read block 791 len 16384
381 140/300 Read block 3902 len 32768
382 141/300 Write block 4746 len 32768 data: 2 2 2 2 2 2 2 2
383 142/300 Write block 2352 len 8192 data: 2 2
384 143/300 Write block 2631 len 40960 data: 3 3 3 3 2 2 2 2 2 2
385 144/300 Read block 2578 len 12288
386 145/300 Read block 1655 len 8192
387 146/300 Write block 2918 len 8192 data: 2 2
388 147/300 Write block 2624 len 24576 data: 2 2 2 2 2 2
389 148/300 Flush
390 149/300 Read block 3387 len 28672
391 150/300 Write block 3295 len 36864 data: 2 2 2 2 2 2 2 2 2
392 151/300 Write block 2804 len 12288 data: 2 2 2
393 152/300 Write block 1403 len 20480 data: 2 2 2 2 2
394 153/300 Write block 3530 len 24576 data: 2 2 2 2 2 2
395 154/300 Write block 2772 len 16384 data: 2 2 2 2
396 155/300 Write block 2832 len 8192 data: 2 2
397 156/300 Write block 1742 len 16384 data: 3 3 3 3
398 157/300 Read block 805 len 16384
399 158/300 Read block 3703 len 4096
400 159/300 Write block 443 len 12288 data: 2 2 2
401 160/300 Write block 2134 len 4096 data: 2
402 161/300 Write block 1337 len 20480 data: 2 2 2 2 2
403 162/300 Read block 322 len 12288
404 163/300 Read block 2962 len 8192
405 164/300 Read block 3792 len 12288
406 165/300 Write block 1317 len 16384 data: 2 3 3 3
407 166/300 Read block 4235 len 8192
408 167/300 Write block 4826 len 40960 data: 2 2 2 2 2 2 2 2 2 2
409 168/300 Flush
410 169/300 Write block 1768 len 24576 data: 2 2 2 2 2 2
411 170/300 Read block 4468 len 20480
412 171/300 Write block 1021 len 20480 data: 2 2 2 2 2
413 172/300 Write block 4522 len 24576 data: 2 2 2 2 2 2
414 173/300 Flush
415 174/300 Read block 1582 len 20480
416 175/300 Read block 911 len 16384
417 176/300 Write block 2894 len 32768 data: 2 2 2 2 2 2 2 2
418 177/300 Write block 3688 len 8192 data: 2 2
419 178/300 Write block 794 len 4096 data: 2
420 179/300 Write block 1979 len 24576 data: 2 2 2 2 2 2
421 180/300 Write block 3212 len 20480 data: 2 2 2 2 2
422 181/300 Write block 369 len 32768 data: 2 2 2 2 2 2 2 2
423 182/300 Read block 4270 len 16384
424 183/300 Write block 455 len 20480 data: 2 2 2 2 2
425 184/300 Read block 338 len 16384
426 185/300 Write block 1414 len 12288 data: 2 2 2
427 186/300 Read block 4549 len 16384
428 187/300 Write block 2211 len 12288 data: 2 2 2
429 188/300 Read block 535 len 36864
430 189/300 Read block 2525 len 12288
431 190/300 Read block 974 len 24576
432 191/300 Write block 252 len 32768 data: 2 2 2 2 2 2 2 2
433 192/300 Read block 3906 len 36864
434 193/300 Write block 194 len 4096 data: 2
435 194/300 Read block 1229 len 12288
436 195/300 Flush
437 196/300 Read block 1242 len 36864
438 197/300 Write block 2247 len 12288 data: 2 2 2
439 198/300 Read block 1819 len 28672
440 199/300 Read block 855 len 4096
441 200/300 Read block 3217 len 40960
442 201/300 Read block 2322 len 36864
443 202/300 Write block 3652 len 28672 data: 2 2 2 2 2 2 2
444 203/300 Write block 3378 len 24576 data: 2 2 2 2 2 2
445 204/300 Read block 603 len 8192
446 205/300 Read block 3340 len 12288
447 206/300 Read block 2089 len 24576
448 207/300 Write block 4597 len 36864 data: 2 2 2 2 2 2 2 2 3
449 208/300 Read block 887 len 20480
450 209/300 Read block 2102 len 36864
451 210/300 Write block 3873 len 32768 data: 2 2 2 2 2 2 2 2
452 211/300 Flush
453 212/300 Write block 1804 len 32768 data: 2 2 2 3 3 3 3 3
454 213/300 Flush
455 214/300 Read block 2004 len 40960
456 215/300 Read block 2231 len 12288
457 216/300 Write block 2459 len 28672 data: 3 3 3 3 3 3 2
458 217/300 Read block 3830 len 20480
459 218/300 Read block 3100 len 8192
460 219/300 Read block 4882 len 32768
461 220/300 Write block 1077 len 8192 data: 2 2
462 221/300 Read block 3678 len 12288
463 222/300 Read block 4193 len 20480
464 223/300 Read block 2102 len 8192
465 224/300 Read block 1828 len 24576
466 225/300 Read block 4927 len 36864
467 226/300 Read block 89 len 40960
468 227/300 Read block 4502 len 40960
469 228/300 Read block 2355 len 24576
470 229/300 Read block 716 len 24576
471 230/300 Write block 1372 len 32768 data: 2 2 2 2 2 2 2 2
472 231/300 Write block 2836 len 32768 data: 2 2 2 2 2 2 2 2
473 232/300 Read block 2747 len 8192
474 233/300 Read block 2161 len 40960
475 234/300 Write block 375 len 16384 data: 3 3 2 2
476 235/300 Write block 1896 len 20480 data: 2 2 2 2 2
477 236/300 Read block 3555 len 28672
478 237/300 Write block 4532 len 40960 data: 2 2 2 2 2 2 2 2 2 2
479 238/300 Write block 473 len 20480 data: 2 2 2 2 2
480 239/300 Read block 1660 len 4096
481 240/300 Read block 2071 len 4096
482 241/300 Write block 2410 len 20480 data: 2 2 2 2 2
483 242/300 Read block 3573 len 20480
484 243/300 Write block 1886 len 28672 data: 2 2 2 2 2 2 2
485 244/300 Read block 1094 len 4096
486 245/300 Read block 2701 len 32768
487 246/300 Write block 3584 len 24576 data: 2 2 2 2 2 2
488 247/300 Read block 3764 len 20480
489 248/300 Read block 4492 len 16384
490 249/300 Write block 2301 len 16384 data: 2 2 2 2
491 250/300 Write block 3211 len 32768 data: 2 3 3 3 3 3 2 2
492 251/300 Write block 1674 len 32768 data: 2 2 2 2 2 2 2 2
493 252/300 Read block 4245 len 8192
494 253/300 Read block 2365 len 20480
495 254/300 Write block 851 len 24576 data: 2 2 2 2 2 2
496 255/300 Write block 3292 len 16384 data: 2 2 2 3
497 256/300 Read block 3405 len 8192
498 257/300 Read block 777 len 28672
499 258/300 Read block 733 len 4096
500 259/300 Read block 1355 len 24576
501 260/300 Read block 482 len 32768
502 261/300 Read block 99 len 12288
503 262/300 Read block 292 len 12288
504 263/300 Read block 900 len 32768
505 264/300 Read block 4179 len 32768
506 265/300 Write block 4573 len 4096 data: 2
507 266/300 Read block 3084 len 28672
508 267/300 Write block 2935 len 8192 data: 2 2
509 268/300 Write block 3021 len 36864 data: 2 2 2 2 2 2 2 2 2
510 269/300 Flush
511 270/300 Write block 4468 len 20480 data: 3 3 3 2 2
512 271/300 Read block 2325 len 24576
513 272/300 Flush
514 273/300 Read block 811 len 12288
515 274/300 Write block 938 len 24576 data: 2 2 2 2 2 2
516 275/300 Read block 3171 len 28672
517 276/300 Flush
518 277/300 Read block 129 len 8192
519 278/300 Read block 4516 len 32768
520 279/300 Read block 216 len 24576
521 280/300 Flush
522 281/300 Read block 2440 len 32768
523 282/300 Read block 677 len 8192
524 283/300 Read block 2688 len 36864
525 284/300 Write block 2345 len 16384 data: 2 2 2 2
526 285/300 Read block 751 len 20480
527 286/300 Read block 4324 len 16384
528 287/300 Read block 651 len 36864
529 288/300 Read block 1545 len 4096
530 289/300 Write block 3509 len 32768 data: 2 2 2 2 2 2 2 2
531 290/300 Read block 4360 len 36864
532 291/300 Write block 1000 len 36864 data: 2 2 2 2 2 2 2 2 2
533 292/300 Write block 4898 len 28672 data: 2 2 2 2 2 2 2
534 293/300 Write block 3361 len 36864 data: 3 4 4 4 4 4 3 3 3
535 294/300 Write block 2664 len 40960 data: 2 2 2 2 2 2 2 2 2 2
536 295/300 Write block 4925 len 16384 data: 2 3 2 2
537 296/300 Read block 1008 len 32768
538 297/300 Read block 810 len 16384
539 298/300 Read block 3682 len 32768
540 299/300 Read block 249 len 40960
541 300/300 Read block 4281 len 4096
542 301/300 Write block 4454 len 24576 data: 2 2 2 2 2 2
543 Main task has work to do, go find it
544 got dsc Start(0)
545 start 0
546 Dec 04 13:30:50.392 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/0, method: POST, req_id: 077ed565-388d-44b3-ad6d-0f607ecd8467, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
547 Replay: started 0, returned:Ok(())
548 ----------------------------------------------------------------
549 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
550 States: Offline Active Active
551 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
552 1000 Acked Read 5 Sent Done Done false
553 1001 Acked Write 10 Sent Done Done false
554 1002 Acked Write 4 Sent Done Done false
555 1003 Acked Read 5 Sent Done Done false
556 1004 Acked Read 1 Sent Done Done false
557 1005 Acked Write 1 Sent Done Done false
558 1006 Acked Write 1 Sent Done Done false
559 1007 Acked Write 10 Sent Done Done false
560 1008 Acked Read 6 Sent Done Done false
561 1009 Acked Read 8 Sent Done Done false
562 1010 Acked Read 5 Sent Done Done false
563 1011 Acked Write 9 Sent Done Done false
564 1012 Acked Read 5 Sent Done Done false
565 1013 Acked Write 3 Sent Done Done false
566 1014 Acked Read 5 Sent Done Done false
567 1015 Acked Read 7 Sent Done Done false
568 1016 Acked Flush 0 Sent Done Done false
569 1017 Acked Flush 0 Sent Done Done false
570 1018 Acked Read 3 Sent Done Done false
571 1019 Acked Write 5 Sent Done Done false
572 1020 Acked Flush 0 Sent Done Done false
573 1021 Acked Read 4 Sent Done Done false
574 1022 Acked Read 1 Sent Done Done false
575 1023 Acked Flush 0 Sent Done Done false
576 1024 Acked Write 5 Sent Done Done false
577 1025 Acked Write 5 Sent Done Done false
578 1026 Acked Read 1 Sent Done Done false
579 1027 Acked Read 4 Sent Done Done false
580 1028 Acked Read 6 Sent Done Done false
581 1029 Acked Write 3 Sent Done Done false
582 1030 Acked Read 8 Sent Done Done false
583 1031 Acked Read 4 Sent Done Done false
584 1032 Acked Write 4 Sent Done Done false
585 1033 Acked Write 8 Sent Done Done false
586 1034 Acked Write 8 Sent Done Done false
587 1035 Acked Write 10 Sent Done Done false
588 1036 Acked Write 9 Sent Done Done false
589 1037 Acked Write 7 Sent Done Done false
590 1038 Acked Flush 0 Sent Done Done false
591 1039 Acked Write 7 Sent Done Done false
592 1040 Acked Read 9 Sent Done Done false
593 1041 Acked Write 9 Sent Done Done false
594 1042 Acked Flush 0 Sent Done Done false
595 1043 Acked Write 8 Sent Done Done false
596 1044 Acked Read 9 Sent Done Done false
597 1045 Acked Write 6 Sent Done Done false
598 1046 Acked Write 3 Sent Done Done false
599 1047 Acked Write 4 Sent Done Done false
600 1048 Acked Read 6 Sent Done Done false
601 1049 Acked Write 8 Sent Done Done false
602 1050 Acked Write 8 Sent Done Done false
603 1051 Acked Read 1 Sent Done Done false
604 1052 Acked Read 2 Sent Done Done false
605 1053 Acked Write 4 Sent Done Done false
606 1054 Acked Write 9 Sent Done Done false
607 1055 Acked Write 1 Sent Done Done false
608 1056 Acked Read 7 Sent Done Done false
609 1057 Acked Write 9 Sent Done Done false
610 1058 Acked Read 10 Sent Done Done false
611 1059 Acked Read 4 Sent Done Done false
612 1060 Acked Read 8 Sent Done Done false
613 1061 Acked Flush 0 Sent Done Done false
614 1062 Acked Write 9 Sent Done Done false
615 1063 Acked Read 9 Sent Done Done false
616 1064 Acked Flush 0 Sent Done Done false
617 1065 Acked Read 3 Sent Done Done false
618 1066 Acked Read 2 Sent Done Done false
619 1067 Acked Read 9 Sent Done Done false
620 1068 Acked Read 1 Sent Done Done false
621 1069 Acked Read 3 Sent Done Done false
622 1070 Acked Read 6 Sent Done Done false
623 1071 Acked Read 4 Sent Done Done false
624 1072 Acked Write 5 Sent Done Done false
625 1073 Acked Read 5 Sent Done Done false
626 1074 Acked Read 1 Sent Done Done false
627 1075 Acked Read 4 Sent Done Done false
628 1076 Acked Write 1 Sent Done Done false
629 1077 Acked Write 6 Sent Done Done false
630 1078 Acked Read 9 Sent Done Done false
631 1079 Acked Flush 0 Sent Done Done false
632 1080 Acked Write 7 Sent Done Done false
633 1081 Acked Write 2 Sent Done Done false
634 1082 Acked Read 1 Sent Done Done false
635 1083 Acked Read 4 Sent Done Done false
636 1084 Acked Write 9 Sent Done Done false
637 1085 Acked Write 5 Sent Done Done false
638 1086 Acked Read 6 Sent Done Done false
639 1087 Acked Read 1 Sent Done Done false
640 1088 Acked Write 6 Sent Done Done false
641 1089 Acked Write 4 Sent Done Done false
642 1090 Acked Write 6 Sent Done Done false
643 1091 Acked Read 6 Sent Done Done false
644 1092 Acked Write 4 Sent Done Done false
645 1093 Acked Read 9 Sent Done Done false
646 1094 Acked Read 2 Sent Done Done false
647 1095 Acked Write 5 Sent Done Done false
648 1096 Acked Read 6 Sent Done Done false
649 1097 Acked Read 10 Sent Done Done false
650 1098 Acked Read 8 Sent Done Done false
651 1099 Acked Write 9 Sent Done Done false
652 1100 Acked Read 7 Sent Done Done false
653 1101 Acked Write 3 Sent Done Done false
654 1102 Acked Read 3 Sent Done Done false
655 1103 Acked Write 7 Sent Done Done false
656 1104 Acked Read 5 Sent Done Done false
657 1105 Acked Write 9 Sent Done Done false
658 1106 Acked Flush 0 Sent Done Done false
659 1107 Acked Read 3 Sent Done Done false
660 1108 Acked Flush 0 Sent Done Done false
661 1109 Acked Write 1 Sent Done Done false
662 1110 Acked Write 5 Sent Done Done false
663 1111 Acked Read 7 Sent Done Done false
664 1112 Acked Write 2 Sent Done Done false
665 1113 Acked Read 7 Sent Done Done false
666 1114 Acked Read 8 Sent Done Done false
667 1115 Acked Flush 0 Sent Done Done false
668 1116 Acked Read 7 Sent Done Done false
669 1117 Acked Read 7 Sent Done Done false
670 1118 Acked Read 5 Sent Done Done false
671 1119 Acked Write 6 Sent Done Done false
672 1120 Acked Read 10 Sent Done Done false
673 1121 Acked Read 4 Sent Done Done false
674 1122 Acked Read 2 Sent Done Done false
675 1123 Acked Write 3 Sent Done Done false
676 1124 Acked Read 9 Sent Done Done false
677 1125 Acked Read 6 Sent Done Done false
678 1126 Acked Read 10 Sent Done Done false
679 1127 Acked Write 9 Sent Done Done false
680 1128 Acked Flush 0 Sent Done Done false
681 1129 Acked Read 9 Sent Done Done false
682 1130 Acked Read 2 Sent Done Done false
683 1131 Acked Write 6 Sent Done Done false
684 1132 Acked Write 1 Sent Done Done false
685 1133 Acked Read 1 Sent Done Done false
686 1134 Acked Write 4 Sent Done Done false
687 1135 Acked Write 9 Sent Done Done false
688 1136 Acked Read 8 Sent Done Done false
689 1137 Acked Read 2 Sent Done Done false
690 1138 Acked Read 4 Sent Done Done false
691 1139 Acked Read 8 Sent Done Done false
692 1140 Acked Write 8 Sent Done Done false
693 1141 Acked Write 2 Sent Done Done false
694 1142 Acked Write 10 Sent Done Done false
695 1143 Acked Read 3 Sent Done Done false
696 1144 Acked Read 2 Sent Done Done false
697 1145 Acked Write 2 Sent Done Done false
698 1146 Acked Write 6 Sent Done Done false
699 1147 Acked Flush 0 Sent Done Done false
700 1148 Acked Read 7 Sent Done Done false
701 1149 Acked Write 9 Sent Done Done false
702 1150 Acked Write 3 Sent Done Done false
703 1151 Acked Write 5 Sent Done Done false
704 1152 Acked Write 6 Sent Done Done false
705 1153 Acked Write 4 Sent Done Done false
706 1154 Acked Write 2 Sent Done Done false
707 1155 Acked Write 4 Sent Done Done false
708 1156 Acked Read 4 Sent Done Done false
709 1157 Acked Read 1 Sent Done Done false
710 1158 Acked Write 3 Sent Done Done false
711 1159 Acked Write 1 Sent Done Done false
712 1160 Acked Write 5 Sent Done Done false
713 1161 Acked Read 3 Sent Done Done false
714 1162 Acked Read 2 Sent Done Done false
715 1163 Acked Read 3 Sent Done Done false
716 1164 Acked Write 4 Sent Done Done false
717 1165 Acked Read 2 Sent Done Done false
718 1166 Acked Write 10 Sent Done Done false
719 1167 Acked Flush 0 Sent Done Done false
720 1168 Acked Write 6 Sent Done Done false
721 1169 Acked Read 5 Sent Done Done false
722 1170 Acked Write 5 Sent Done Done false
723 1171 Acked Write 6 Sent Done Done false
724 1172 Acked Flush 0 Sent Done Done false
725 1173 Acked Read 5 Sent Done Done false
726 1174 Acked Read 4 Sent Done Done false
727 1175 Acked Write 8 Sent Done Done false
728 1176 Acked Write 2 Sent Done Done false
729 1177 Acked Write 1 Sent Done Done false
730 1178 Acked Write 6 Sent Done Done false
731 1179 Acked Write 5 Sent Done Done false
732 1180 Acked Write 8 Sent Done Done false
733 1181 Acked Read 4 Sent Done Done false
734 1182 Acked Write 5 Sent Done Done false
735 1183 Acked Read 4 Sent Done Done false
736 1184 Acked Write 3 Sent Done Done false
737 1185 Acked Read 4 Sent Done Done false
738 1186 Acked Write 3 Sent Done Done false
739 1187 Acked Read 9 Sent Done Done false
740 1188 Acked Read 3 Sent Done Done false
741 1189 Acked Read 6 Sent Done Done false
742 1190 Acked Write 8 Sent Done Done false
743 1191 Acked Read 9 Sent Done Done false
744 1192 Acked Write 1 Sent Done Done false
745 1193 Acked Read 3 Sent Done Done false
746 1194 Acked Flush 0 Sent Done Done false
747 1195 Acked Read 9 Sent Done Done false
748 1196 Acked Write 3 Sent Done Done false
749 1197 Acked Read 7 Sent Done Done false
750 1198 Acked Read 1 Sent Done Done false
751 1199 Acked Read 10 Sent Done Done false
752 1200 Acked Read 9 Sent Done Done false
753 1201 Acked Write 7 Sent Done Done false
754 1202 Acked Write 6 Sent Done Done false
755 1203 Acked Read 2 Sent Done Done false
756 1204 Acked Read 3 Sent Done Done false
757 1205 Acked Read 6 Sent Done Done false
758 1206 Acked Write 9 Sent Done Done false
759 1207 Acked Read 5 Sent Done Done false
760 1208 Acked Read 9 Sent Done Done false
761 1209 Acked Write 8 Sent Done Done false
762 1210 Acked Flush 0 Sent Done Done false
763 1211 Acked Write 8 Sent Done Done false
764 1212 Acked Flush 0 Sent Done Done false
765 1213 Acked Read 10 Sent Done Done false
766 1214 Acked Read 3 Sent Done Done false
767 1215 Acked Write 7 Sent Done Done false
768 1216 Acked Read 5 Sent Done Done false
769 1217 Acked Read 2 Sent Done Done false
770 1218 Acked Read 8 Sent Done Done false
771 1219 Acked Write 2 Sent Done Done false
772 1220 Acked Read 3 Sent Done Done false
773 1221 Acked Read 5 Sent Done Done false
774 1222 Acked Read 2 Sent Done Done false
775 1223 Acked Read 6 Sent Done Done false
776 1224 Acked Read 9 Sent Done Done false
777 1225 Acked Read 10 Sent Done Done false
778 1226 Acked Read 10 Sent Done Done false
779 1227 Acked Read 6 Sent Done Done false
780 1228 Acked Read 6 Sent Done Done false
781 1229 Acked Write 8 Sent Done Done false
782 1230 Acked Write 8 Sent Done Done false
783 1231 Acked Read 2 Sent Done Done false
784 1232 Acked Read 10 Sent Done Done false
785 1233 Acked Write 4 Sent Done Done false
786 1234 Acked Write 5 Sent Done Done false
787 1235 Acked Read 7 Sent Done Done false
788 1236 Acked Write 10 Sent Done Done false
789 1237 Acked Write 5 Sent Done Done false
790 1238 Acked Read 1 Sent Done Done false
791 1239 Acked Read 1 Sent Done Done false
792 1240 Acked Write 5 Sent Done Done false
793 1241 Acked Read 5 Sent Done Done false
794 1242 Acked Write 7 Sent Done Done false
795 1243 Acked Read 1 Sent Done Done false
796 1244 Acked Read 8 Sent Done Done false
797 1245 Acked Write 6 Sent Done Done false
798 1246 Acked Read 5 Sent Done Done false
799 1247 Acked Read 4 Sent Done Done false
800 1248 Acked Write 4 Sent Done Done false
801 1249 Acked Write 8 Sent Done Done false
802 1250 Acked Write 8 Sent Done Done false
803 1251 Acked Read 2 Sent Done Done false
804 1252 Acked Read 5 Sent Done Done false
805 1253 Acked Write 6 Sent Done Done false
806 1254 Acked Write 4 Sent Done Done false
807 1255 Acked Read 2 Sent Done Done false
808 1256 Acked Read 7 Sent Done Done false
809 1257 Acked Read 1 Sent Done Done false
810 1258 Acked Read 6 Sent Done Done false
811 1259 Acked Read 8 Sent Done Done false
812 1260 Acked Read 3 Sent Done Done false
813 1261 Acked Read 3 Sent Done Done false
814 1262 Acked Read 8 Sent Done Done false
815 1263 Acked Read 8 Sent Done Done false
816 1264 Acked Write 1 Sent Done Done false
817 1265 Acked Read 7 Sent Done Done false
818 1266 Acked Write 2 Sent Done Done false
819 1267 Acked Write 9 Sent Done Done false
820 1268 Acked Flush 0 Sent Done Done false
821 1269 Acked Write 5 Sent Done Done false
822 1270 Acked Read 6 Sent Done Done false
823 1271 Acked Flush 0 Sent Done Done false
824 1272 Acked Read 3 Sent Done Done false
825 1273 Acked Write 6 Sent Done Done false
826 1274 Acked Read 7 Sent Done Done false
827 1275 Acked Flush 0 Sent Done Done false
828 1276 Acked Read 2 Sent Done Done false
829 1277 Acked Read 8 Sent Done Done false
830 1278 Acked Read 6 Sent Done Done false
831 1279 Acked Flush 0 Sent Done Done false
832 1280 Acked Read 8 Sent Done Done false
833 1281 Acked Read 2 Sent Done Done false
834 1282 Acked Read 9 Sent Done Done false
835 1283 Acked Write 4 Sent Done Done false
836 1284 Acked Read 5 Sent Done Done false
837 1285 Acked Read 4 Sent Done Done false
838 1286 Acked Read 9 Sent Done Done false
839 1287 Acked Read 1 Sent Done Done false
840 1288 Acked Write 8 Sent Done Done false
841 1289 Acked Read 9 Sent Done Done false
842 1290 Acked Write 9 Sent Done Done false
843 1291 Acked Write 7 Sent Done Done false
844 1292 Acked Write 9 Sent Done Done false
845 1293 Acked Write 10 Sent Done Done false
846 1294 Acked Write 4 Sent Done Done false
847 1295 Acked Read 8 Sent Done Done false
848 1296 Acked Read 4 Sent Done Done false
849 1297 Acked Read 8 Sent Done Done false
850 1298 Acked Read 10 Sent Done Done false
851 1299 Acked Read 1 Sent Done Done false
852 1300 Acked Write 6 Sent Done Done false
853 STATES DS:0 DS:1 DS:2 TOTAL
854 Sent 301 0 0 301
855 Done 0 301 301 602
856 Skipped 0 0 0 0
857 Error 0 0 0 0
858 Last Flush: 0 1279 1279
859 Downstairs last five completed:
860 Upstairs last five completed: 1300 1299 1298 1297 1296
861 CLIENT: Up:0 ds:301 act:2
862 [8810] got start action
863 [8810] I am going to restart
864 Starting downstairs at port 8810
865 Make output file at "/tmp/dsc/downstairs-8810.txt"
866 [8810][0] reports Starting
867 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1204)
868 [8810][0] reports Running
869 ----------------------------------------------------------------
870 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
871 States: Offline Active Active
872 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
873 1000 Acked Read 5 Sent Done Done false
874 1001 Acked Write 10 Sent Done Done false
875 1002 Acked Write 4 Sent Done Done false
876 1003 Acked Read 5 Sent Done Done false
877 1004 Acked Read 1 Sent Done Done false
878 1005 Acked Write 1 Sent Done Done false
879 1006 Acked Write 1 Sent Done Done false
880 1007 Acked Write 10 Sent Done Done false
881 1008 Acked Read 6 Sent Done Done false
882 1009 Acked Read 8 Sent Done Done false
883 1010 Acked Read 5 Sent Done Done false
884 1011 Acked Write 9 Sent Done Done false
885 1012 Acked Read 5 Sent Done Done false
886 1013 Acked Write 3 Sent Done Done false
887 1014 Acked Read 5 Sent Done Done false
888 1015 Acked Read 7 Sent Done Done false
889 1016 Acked Flush 0 Sent Done Done false
890 1017 Acked Flush 0 Sent Done Done false
891 1018 Acked Read 3 Sent Done Done false
892 1019 Acked Write 5 Sent Done Done false
893 1020 Acked Flush 0 Sent Done Done false
894 1021 Acked Read 4 Sent Done Done false
895 1022 Acked Read 1 Sent Done Done false
896 1023 Acked Flush 0 Sent Done Done false
897 1024 Acked Write 5 Sent Done Done false
898 1025 Acked Write 5 Sent Done Done false
899 1026 Acked Read 1 Sent Done Done false
900 1027 Acked Read 4 Sent Done Done false
901 1028 Acked Read 6 Sent Done Done false
902 1029 Acked Write 3 Sent Done Done false
903 1030 Acked Read 8 Sent Done Done false
904 1031 Acked Read 4 Sent Done Done false
905 1032 Acked Write 4 Sent Done Done false
906 1033 Acked Write 8 Sent Done Done false
907 1034 Acked Write 8 Sent Done Done false
908 1035 Acked Write 10 Sent Done Done false
909 1036 Acked Write 9 Sent Done Done false
910 1037 Acked Write 7 Sent Done Done false
911 1038 Acked Flush 0 Sent Done Done false
912 1039 Acked Write 7 Sent Done Done false
913 1040 Acked Read 9 Sent Done Done false
914 1041 Acked Write 9 Sent Done Done false
915 1042 Acked Flush 0 Sent Done Done false
916 1043 Acked Write 8 Sent Done Done false
917 1044 Acked Read 9 Sent Done Done false
918 1045 Acked Write 6 Sent Done Done false
919 1046 Acked Write 3 Sent Done Done false
920 1047 Acked Write 4 Sent Done Done false
921 1048 Acked Read 6 Sent Done Done false
922 1049 Acked Write 8 Sent Done Done false
923 1050 Acked Write 8 Sent Done Done false
924 1051 Acked Read 1 Sent Done Done false
925 1052 Acked Read 2 Sent Done Done false
926 1053 Acked Write 4 Sent Done Done false
927 1054 Acked Write 9 Sent Done Done false
928 1055 Acked Write 1 Sent Done Done false
929 1056 Acked Read 7 Sent Done Done false
930 1057 Acked Write 9 Sent Done Done false
931 1058 Acked Read 10 Sent Done Done false
932 1059 Acked Read 4 Sent Done Done false
933 1060 Acked Read 8 Sent Done Done false
934 1061 Acked Flush 0 Sent Done Done false
935 1062 Acked Write 9 Sent Done Done false
936 1063 Acked Read 9 Sent Done Done false
937 1064 Acked Flush 0 Sent Done Done false
938 1065 Acked Read 3 Sent Done Done false
939 1066 Acked Read 2 Sent Done Done false
940 1067 Acked Read 9 Sent Done Done false
941 1068 Acked Read 1 Sent Done Done false
942 1069 Acked Read 3 Sent Done Done false
943 1070 Acked Read 6 Sent Done Done false
944 1071 Acked Read 4 Sent Done Done false
945 1072 Acked Write 5 Sent Done Done false
946 1073 Acked Read 5 Sent Done Done false
947 1074 Acked Read 1 Sent Done Done false
948 1075 Acked Read 4 Sent Done Done false
949 1076 Acked Write 1 Sent Done Done false
950 1077 Acked Write 6 Sent Done Done false
951 1078 Acked Read 9 Sent Done Done false
952 1079 Acked Flush 0 Sent Done Done false
953 1080 Acked Write 7 Sent Done Done false
954 1081 Acked Write 2 Sent Done Done false
955 1082 Acked Read 1 Sent Done Done false
956 1083 Acked Read 4 Sent Done Done false
957 1084 Acked Write 9 Sent Done Done false
958 1085 Acked Write 5 Sent Done Done false
959 1086 Acked Read 6 Sent Done Done false
960 1087 Acked Read 1 Sent Done Done false
961 1088 Acked Write 6 Sent Done Done false
962 1089 Acked Write 4 Sent Done Done false
963 1090 Acked Write 6 Sent Done Done false
964 1091 Acked Read 6 Sent Done Done false
965 1092 Acked Write 4 Sent Done Done false
966 1093 Acked Read 9 Sent Done Done false
967 1094 Acked Read 2 Sent Done Done false
968 1095 Acked Write 5 Sent Done Done false
969 1096 Acked Read 6 Sent Done Done false
970 1097 Acked Read 10 Sent Done Done false
971 1098 Acked Read 8 Sent Done Done false
972 1099 Acked Write 9 Sent Done Done false
973 1100 Acked Read 7 Sent Done Done false
974 1101 Acked Write 3 Sent Done Done false
975 1102 Acked Read 3 Sent Done Done false
976 1103 Acked Write 7 Sent Done Done false
977 1104 Acked Read 5 Sent Done Done false
978 1105 Acked Write 9 Sent Done Done false
979 1106 Acked Flush 0 Sent Done Done false
980 1107 Acked Read 3 Sent Done Done false
981 1108 Acked Flush 0 Sent Done Done false
982 1109 Acked Write 1 Sent Done Done false
983 1110 Acked Write 5 Sent Done Done false
984 1111 Acked Read 7 Sent Done Done false
985 1112 Acked Write 2 Sent Done Done false
986 1113 Acked Read 7 Sent Done Done false
987 1114 Acked Read 8 Sent Done Done false
988 1115 Acked Flush 0 Sent Done Done false
989 1116 Acked Read 7 Sent Done Done false
990 1117 Acked Read 7 Sent Done Done false
991 1118 Acked Read 5 Sent Done Done false
992 1119 Acked Write 6 Sent Done Done false
993 1120 Acked Read 10 Sent Done Done false
994 1121 Acked Read 4 Sent Done Done false
995 1122 Acked Read 2 Sent Done Done false
996 1123 Acked Write 3 Sent Done Done false
997 1124 Acked Read 9 Sent Done Done false
998 1125 Acked Read 6 Sent Done Done false
999 1126 Acked Read 10 Sent Done Done false
1000 1127 Acked Write 9 Sent Done Done false
1001 1128 Acked Flush 0 Sent Done Done false
1002 1129 Acked Read 9 Sent Done Done false
1003 1130 Acked Read 2 Sent Done Done false
1004 1131 Acked Write 6 Sent Done Done false
1005 1132 Acked Write 1 Sent Done Done false
1006 1133 Acked Read 1 Sent Done Done false
1007 1134 Acked Write 4 Sent Done Done false
1008 1135 Acked Write 9 Sent Done Done false
1009 1136 Acked Read 8 Sent Done Done false
1010 1137 Acked Read 2 Sent Done Done false
1011 1138 Acked Read 4 Sent Done Done false
1012 1139 Acked Read 8 Sent Done Done false
1013 1140 Acked Write 8 Sent Done Done false
1014 1141 Acked Write 2 Sent Done Done false
1015 1142 Acked Write 10 Sent Done Done false
1016 1143 Acked Read 3 Sent Done Done false
1017 1144 Acked Read 2 Sent Done Done false
1018 1145 Acked Write 2 Sent Done Done false
1019 1146 Acked Write 6 Sent Done Done false
1020 1147 Acked Flush 0 Sent Done Done false
1021 1148 Acked Read 7 Sent Done Done false
1022 1149 Acked Write 9 Sent Done Done false
1023 1150 Acked Write 3 Sent Done Done false
1024 1151 Acked Write 5 Sent Done Done false
1025 1152 Acked Write 6 Sent Done Done false
1026 1153 Acked Write 4 Sent Done Done false
1027 1154 Acked Write 2 Sent Done Done false
1028 1155 Acked Write 4 Sent Done Done false
1029 1156 Acked Read 4 Sent Done Done false
1030 1157 Acked Read 1 Sent Done Done false
1031 1158 Acked Write 3 Sent Done Done false
1032 1159 Acked Write 1 Sent Done Done false
1033 1160 Acked Write 5 Sent Done Done false
1034 1161 Acked Read 3 Sent Done Done false
1035 1162 Acked Read 2 Sent Done Done false
1036 1163 Acked Read 3 Sent Done Done false
1037 1164 Acked Write 4 Sent Done Done false
1038 1165 Acked Read 2 Sent Done Done false
1039 1166 Acked Write 10 Sent Done Done false
1040 1167 Acked Flush 0 Sent Done Done false
1041 1168 Acked Write 6 Sent Done Done false
1042 1169 Acked Read 5 Sent Done Done false
1043 1170 Acked Write 5 Sent Done Done false
1044 1171 Acked Write 6 Sent Done Done false
1045 1172 Acked Flush 0 Sent Done Done false
1046 1173 Acked Read 5 Sent Done Done false
1047 1174 Acked Read 4 Sent Done Done false
1048 1175 Acked Write 8 Sent Done Done false
1049 1176 Acked Write 2 Sent Done Done false
1050 1177 Acked Write 1 Sent Done Done false
1051 1178 Acked Write 6 Sent Done Done false
1052 1179 Acked Write 5 Sent Done Done false
1053 1180 Acked Write 8 Sent Done Done false
1054 1181 Acked Read 4 Sent Done Done false
1055 1182 Acked Write 5 Sent Done Done false
1056 1183 Acked Read 4 Sent Done Done false
1057 1184 Acked Write 3 Sent Done Done false
1058 1185 Acked Read 4 Sent Done Done false
1059 1186 Acked Write 3 Sent Done Done false
1060 1187 Acked Read 9 Sent Done Done false
1061 1188 Acked Read 3 Sent Done Done false
1062 1189 Acked Read 6 Sent Done Done false
1063 1190 Acked Write 8 Sent Done Done false
1064 1191 Acked Read 9 Sent Done Done false
1065 1192 Acked Write 1 Sent Done Done false
1066 1193 Acked Read 3 Sent Done Done false
1067 1194 Acked Flush 0 Sent Done Done false
1068 1195 Acked Read 9 Sent Done Done false
1069 1196 Acked Write 3 Sent Done Done false
1070 1197 Acked Read 7 Sent Done Done false
1071 1198 Acked Read 1 Sent Done Done false
1072 1199 Acked Read 10 Sent Done Done false
1073 1200 Acked Read 9 Sent Done Done false
1074 1201 Acked Write 7 Sent Done Done false
1075 1202 Acked Write 6 Sent Done Done false
1076 1203 Acked Read 2 Sent Done Done false
1077 1204 Acked Read 3 Sent Done Done false
1078 1205 Acked Read 6 Sent Done Done false
1079 1206 Acked Write 9 Sent Done Done false
1080 1207 Acked Read 5 Sent Done Done false
1081 1208 Acked Read 9 Sent Done Done false
1082 1209 Acked Write 8 Sent Done Done false
1083 1210 Acked Flush 0 Sent Done Done false
1084 1211 Acked Write 8 Sent Done Done false
1085 1212 Acked Flush 0 Sent Done Done false
1086 1213 Acked Read 10 Sent Done Done false
1087 1214 Acked Read 3 Sent Done Done false
1088 1215 Acked Write 7 Sent Done Done false
1089 1216 Acked Read 5 Sent Done Done false
1090 1217 Acked Read 2 Sent Done Done false
1091 1218 Acked Read 8 Sent Done Done false
1092 1219 Acked Write 2 Sent Done Done false
1093 1220 Acked Read 3 Sent Done Done false
1094 1221 Acked Read 5 Sent Done Done false
1095 1222 Acked Read 2 Sent Done Done false
1096 1223 Acked Read 6 Sent Done Done false
1097 1224 Acked Read 9 Sent Done Done false
1098 1225 Acked Read 10 Sent Done Done false
1099 1226 Acked Read 10 Sent Done Done false
1100 1227 Acked Read 6 Sent Done Done false
1101 1228 Acked Read 6 Sent Done Done false
1102 1229 Acked Write 8 Sent Done Done false
1103 1230 Acked Write 8 Sent Done Done false
1104 1231 Acked Read 2 Sent Done Done false
1105 1232 Acked Read 10 Sent Done Done false
1106 1233 Acked Write 4 Sent Done Done false
1107 1234 Acked Write 5 Sent Done Done false
1108 1235 Acked Read 7 Sent Done Done false
1109 1236 Acked Write 10 Sent Done Done false
1110 1237 Acked Write 5 Sent Done Done false
1111 1238 Acked Read 1 Sent Done Done false
1112 1239 Acked Read 1 Sent Done Done false
1113 1240 Acked Write 5 Sent Done Done false
1114 1241 Acked Read 5 Sent Done Done false
1115 1242 Acked Write 7 Sent Done Done false
1116 1243 Acked Read 1 Sent Done Done false
1117 1244 Acked Read 8 Sent Done Done false
1118 1245 Acked Write 6 Sent Done Done false
1119 1246 Acked Read 5 Sent Done Done false
1120 1247 Acked Read 4 Sent Done Done false
1121 1248 Acked Write 4 Sent Done Done false
1122 1249 Acked Write 8 Sent Done Done false
1123 1250 Acked Write 8 Sent Done Done false
1124 1251 Acked Read 2 Sent Done Done false
1125 1252 Acked Read 5 Sent Done Done false
1126 1253 Acked Write 6 Sent Done Done false
1127 1254 Acked Write 4 Sent Done Done false
1128 1255 Acked Read 2 Sent Done Done false
1129 1256 Acked Read 7 Sent Done Done false
1130 1257 Acked Read 1 Sent Done Done false
1131 1258 Acked Read 6 Sent Done Done false
1132 1259 Acked Read 8 Sent Done Done false
1133 1260 Acked Read 3 Sent Done Done false
1134 1261 Acked Read 3 Sent Done Done false
1135 1262 Acked Read 8 Sent Done Done false
1136 1263 Acked Read 8 Sent Done Done false
1137 1264 Acked Write 1 Sent Done Done false
1138 1265 Acked Read 7 Sent Done Done false
1139 1266 Acked Write 2 Sent Done Done false
1140 1267 Acked Write 9 Sent Done Done false
1141 1268 Acked Flush 0 Sent Done Done false
1142 1269 Acked Write 5 Sent Done Done false
1143 1270 Acked Read 6 Sent Done Done false
1144 1271 Acked Flush 0 Sent Done Done false
1145 1272 Acked Read 3 Sent Done Done false
1146 1273 Acked Write 6 Sent Done Done false
1147 1274 Acked Read 7 Sent Done Done false
1148 1275 Acked Flush 0 Sent Done Done false
1149 1276 Acked Read 2 Sent Done Done false
1150 1277 Acked Read 8 Sent Done Done false
1151 1278 Acked Read 6 Sent Done Done false
1152 1279 Acked Flush 0 Sent Done Done false
1153 1280 Acked Read 8 Sent Done Done false
1154 1281 Acked Read 2 Sent Done Done false
1155 1282 Acked Read 9 Sent Done Done false
1156 1283 Acked Write 4 Sent Done Done false
1157 1284 Acked Read 5 Sent Done Done false
1158 1285 Acked Read 4 Sent Done Done false
1159 1286 Acked Read 9 Sent Done Done false
1160 1287 Acked Read 1 Sent Done Done false
1161 1288 Acked Write 8 Sent Done Done false
1162 1289 Acked Read 9 Sent Done Done false
1163 1290 Acked Write 9 Sent Done Done false
1164 1291 Acked Write 7 Sent Done Done false
1165 1292 Acked Write 9 Sent Done Done false
1166 1293 Acked Write 10 Sent Done Done false
1167 1294 Acked Write 4 Sent Done Done false
1168 1295 Acked Read 8 Sent Done Done false
1169 1296 Acked Read 4 Sent Done Done false
1170 1297 Acked Read 8 Sent Done Done false
1171 1298 Acked Read 10 Sent Done Done false
1172 1299 Acked Read 1 Sent Done Done false
1173 1300 Acked Write 6 Sent Done Done false
1174 1301 Acked Flush 0 Sent Done Done false
1175 STATES DS:0 DS:1 DS:2 TOTAL
1176 Sent 302 0 0 302
1177 Done 0 302 302 604
1178 Skipped 0 0 0 0
1179 Error 0 0 0 0
1180 Last Flush: 0 1301 1301
1181 Downstairs last five completed:
1182 Upstairs last five completed: 1301 1300 1299 1298 1297
1183 CLIENT: Up:0 ds:302 act:2
11842024-12-04T13:30:56.155ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11852024-12-04T13:30:56.155ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64952) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11862024-12-04T13:30:56.156ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11872024-12-04T13:30:56.157ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11882024-12-04T13:30:56.157ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11892024-12-04T13:30:56.157ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11902024-12-04T13:30:56.157ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11912024-12-04T13:30:56.157ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11922024-12-04T13:30:56.157ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11932024-12-04T13:30:56.157ZINFOcrucible: [0] client re-new 302 jobs since flush 0 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11942024-12-04T13:30:56.157ZINFOcrucible: [0] Replayed 302 jobs since flush: 0 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1195 ----------------------------------------------------------------
1196 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1197 Downstairs last five completed: 1301 1300 1299 1298 1297
1198 Upstairs last five completed: 1301 1300 1299 1298 1297
1199 CLIENT: Up:0 ds:0 act:3
1200 Replay: All jobs finished
1201 Main task has work to do, go find it
1202 got dsc Stop(1)
1203 stop 1
1204 Dec 04 13:30:58.396 [8820] Got stop action so:false kr:false
1205 INFO request completed, latency_us: 347, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 4c8c787d-69aa-4fed-bccd-88d5728ee600, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
12062024-12-04T13:30:58.396ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12072024-12-04T13:30:58.396ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12082024-12-04T13:30:58.396ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12092024-12-04T13:30:58.396ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12102024-12-04T13:30:58.396ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12112024-12-04T13:30:58.396ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
1212 Dec 04 13:30:58.397 INFO request completed, latency_us: 166, response_code: 200, uri: /state/cid/1, method: GET, req_id: 91b5de0d-84a1-44c9-9a16-ac35d8faa402, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
1213 [8820] Exited with: ExitStatus(unix_wait_status(9))
1214 [8820][1] reports Exit
1215 Dec 04 13:31:02.398 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/1, method: GET, req_id: 12a2b743-07eb-4f95-b83b-eab8126a1b29, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
1216 001/300 Write block 1525 len 24576 data: 2 2 2 2 2 2
1217 002/300 Write block 1413 len 36864 data: 2 3 3 3 2 2 2 2 2
1218 003/300 Read block 183 len 4096
1219 004/300 Read block 2707 len 8192
1220 005/300 Read block 325 len 20480
1221 006/300 Flush
1222 007/300 Read block 1752 len 36864
1223 008/300 Read block 1340 len 20480
1224 009/300 Write block 371 len 16384 data: 3 3 3 3
1225 010/300 Read block 81 len 16384
1226 011/300 Write block 3776 len 32768 data: 2 2 2 2 2 2 2 2
1227 012/300 Write block 978 len 12288 data: 2 2 2
1228 013/300 Write block 4200 len 16384 data: 2 2 2 2
1229 014/300 Read block 3464 len 8192
1230 015/300 Read block 706 len 32768
1231 016/300 Write block 4963 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1232 017/300 Write block 2415 len 4096 data: 2
1233 018/300 Read block 4169 len 36864
1234 019/300 Write block 2148 len 8192 data: 2 2
1235 020/300 Read block 119 len 20480
1236 021/300 Flush
1237 022/300 Write block 3315 len 36864 data: 2 2 2 2 2 2 2 2 2
1238 023/300 Write block 3485 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1239 024/300 Read block 4040 len 28672
1240 025/300 Write block 152 len 16384 data: 2 3 3 3
1241 026/300 Write block 3990 len 16384 data: 2 2 2 2
1242 027/300 Read block 4275 len 16384
1243 028/300 Read block 4537 len 28672
1244 029/300 Read block 4342 len 28672
1245 030/300 Read block 2940 len 28672
1246 031/300 Write block 1612 len 16384 data: 2 2 2 2
1247 032/300 Read block 1692 len 20480
1248 033/300 Write block 1503 len 12288 data: 2 2 3
1249 034/300 Read block 673 len 28672
1250 035/300 Write block 2968 len 24576 data: 2 2 2 2 2 2
1251 036/300 Read block 2875 len 28672
1252 037/300 Read block 4739 len 4096
1253 038/300 Read block 1610 len 12288
1254 039/300 Write block 935 len 4096 data: 2
1255 040/300 Read block 3155 len 40960
1256 041/300 Read block 2362 len 12288
1257 042/300 Read block 2448 len 16384
1258 043/300 Read block 1099 len 16384
1259 044/300 Read block 4251 len 32768
1260 045/300 Write block 404 len 28672 data: 2 2 2 2 2 2 2
1261 046/300 Read block 1832 len 12288
1262 047/300 Read block 873 len 12288
1263 048/300 Read block 2132 len 16384
1264 049/300 Read block 1227 len 24576
1265 050/300 Write block 4459 len 8192 data: 3 2
1266 051/300 Flush
1267 052/300 Read block 3040 len 40960
1268 053/300 Read block 3854 len 20480
1269 054/300 Read block 201 len 4096
1270 055/300 Read block 676 len 32768
1271 056/300 Write block 2844 len 24576 data: 2 2 2 2 2 2
1272 057/300 Read block 1763 len 28672
1273 058/300 Read block 1502 len 8192
1274 059/300 Flush
1275 060/300 Read block 3482 len 12288
1276 061/300 Read block 4771 len 32768
1277 062/300 Write block 113 len 24576 data: 2 2 2 2 2 2
1278 063/300 Flush
1279 064/300 Read block 4343 len 28672
1280 065/300 Write block 3732 len 12288 data: 2 2 2
1281 066/300 Read block 3262 len 36864
1282 067/300 Read block 112 len 40960
1283 068/300 Write block 3000 len 36864 data: 2 2 2 2 2 2 2 2 2
1284 069/300 Read block 3489 len 40960
1285 070/300 Flush
1286 071/300 Write block 1383 len 36864 data: 2 2 2 2 2 2 2 2 2
1287 072/300 Read block 3734 len 12288
1288 073/300 Write block 1898 len 4096 data: 3
1289 074/300 Read block 1429 len 36864
1290 075/300 Flush
1291 076/300 Write block 708 len 4096 data: 2
1292 077/300 Read block 1221 len 28672
1293 078/300 Read block 2527 len 12288
1294 079/300 Read block 4454 len 20480
1295 080/300 Read block 2821 len 40960
1296 081/300 Read block 3904 len 24576
1297 082/300 Write block 4659 len 28672 data: 2 2 2 2 2 2 2
1298 083/300 Read block 2290 len 4096
1299 084/300 Read block 2278 len 32768
1300 085/300 Write block 4271 len 28672 data: 2 2 2 3 3 3 3
1301 086/300 Read block 2727 len 16384
1302 087/300 Read block 914 len 8192
1303 088/300 Read block 4098 len 12288
1304 089/300 Read block 193 len 28672
1305 090/300 Read block 914 len 20480
1306 091/300 Write block 1502 len 40960 data: 2 3 3 4 3 3 3 3 3 3
1307 092/300 Write block 754 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1308 093/300 Write block 2570 len 8192 data: 2 2
1309 094/300 Read block 4962 len 20480
1310 095/300 Write block 2037 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1311 096/300 Write block 2457 len 36864 data: 3 3 4 4 4 4 4 4 3
1312 097/300 Write block 1945 len 4096 data: 2
1313 098/300 Read block 4490 len 12288
1314 099/300 Read block 1446 len 4096
1315 100/300 Write block 2760 len 32768 data: 2 2 2 2 2 2 2 2
1316 101/300 Flush
1317 102/300 Read block 1050 len 12288
1318 103/300 Write block 2742 len 4096 data: 2
1319 104/300 Read block 2509 len 36864
1320 105/300 Read block 4760 len 24576
1321 106/300 Write block 1741 len 36864 data: 3 4 4 4 4 3 3 3 3
1322 107/300 Read block 4709 len 28672
1323 108/300 Read block 2078 len 32768
1324 109/300 Read block 1416 len 20480
1325 110/300 Write block 2424 len 12288 data: 2 2 2
1326 111/300 Read block 268 len 24576
1327 112/300 Write block 3716 len 24576 data: 3 3 3 3 3 3
1328 113/300 Read block 322 len 20480
1329 114/300 Flush
1330 115/300 Read block 3225 len 4096
1331 116/300 Read block 4769 len 4096
1332 117/300 Read block 2837 len 28672
1333 118/300 Write block 2433 len 4096 data: 2
1334 119/300 Read block 2437 len 32768
1335 120/300 Read block 2950 len 12288
1336 121/300 Read block 4582 len 40960
1337 122/300 Read block 4030 len 24576
1338 123/300 Read block 704 len 24576
1339 124/300 Read block 4635 len 36864
1340 125/300 Write block 2821 len 28672 data: 2 2 2 2 2 2 2
1341 126/300 Read block 1000 len 16384
1342 127/300 Write block 2044 len 12288 data: 3 3 3
1343 128/300 Read block 1407 len 20480
1344 129/300 Flush
1345 130/300 Write block 3625 len 4096 data: 2
1346 131/300 Read block 4639 len 12288
1347 132/300 Read block 387 len 28672
1348 133/300 Read block 3918 len 40960
1349 134/300 Write block 3320 len 8192 data: 3 3
1350 135/300 Read block 570 len 24576
1351 136/300 Flush
1352 137/300 Read block 1587 len 16384
1353 138/300 Write block 1457 len 28672 data: 2 2 2 2 2 2 2
1354 139/300 Write block 4922 len 12288 data: 2 2 2
1355 140/300 Read block 3169 len 24576
1356 141/300 Read block 2383 len 40960
1357 142/300 Write block 3296 len 24576 data: 3 3 3 3 3 3
1358 143/300 Read block 2830 len 24576
1359 144/300 Write block 538 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1360 145/300 Read block 4990 len 16384
1361 146/300 Write block 1014 len 28672 data: 2 2 2 2 2 2 2
1362 147/300 Read block 3616 len 12288
1363 148/300 Write block 3824 len 24576 data: 2 2 2 2 2 2
1364 149/300 Write block 3226 len 4096 data: 2
1365 150/300 Read block 413 len 28672
1366 151/300 Read block 2446 len 36864
1367 152/300 Read block 1977 len 32768
1368 153/300 Read block 3140 len 40960
1369 154/300 Write block 2089 len 16384 data: 2 2 2 2
1370 155/300 Read block 2659 len 4096
1371 156/300 Write block 2287 len 36864 data: 2 2 2 2 2 2 2 2 2
1372 157/300 Flush
1373 158/300 Read block 2128 len 16384
1374 159/300 Read block 1856 len 12288
1375 160/300 Write block 2745 len 24576 data: 2 2 2 2 2 2
1376 161/300 Flush
1377 162/300 Read block 4642 len 16384
1378 163/300 Read block 431 len 4096
1379 164/300 Write block 4269 len 12288 data: 2 2 3
1380 165/300 Write block 1248 len 40960 data: 3 3 3 2 2 2 2 2 2 2
1381 166/300 Read block 771 len 40960
1382 167/300 Read block 520 len 4096
1383 168/300 Write block 4244 len 28672 data: 2 2 2 2 2 2 2
1384 169/300 Write block 1180 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1385 170/300 Read block 2263 len 24576
1386 171/300 Write block 121 len 24576 data: 2 2 2 2 2 2
1387 172/300 Write block 3941 len 20480 data: 2 2 2 2 3
1388 173/300 Read block 3396 len 24576
1389 174/300 Write block 2418 len 24576 data: 2 2 2 2 2 2
1390 175/300 Write block 2825 len 32768 data: 3 3 3 2 2 2 2 3
1391 176/300 Read block 3936 len 8192
1392 177/300 Read block 4548 len 4096
1393 178/300 Write block 1682 len 36864 data: 2 2 2 2 2 2 2 2 2
1394 179/300 Read block 4798 len 12288
1395 180/300 Read block 113 len 4096
1396 181/300 Write block 2135 len 8192 data: 2 2
1397 182/300 Write block 2191 len 32768 data: 2 2 2 2 2 2 2 2
1398 183/300 Read block 4858 len 4096
1399 184/300 Flush
1400 185/300 Read block 1711 len 24576
1401 186/300 Write block 3933 len 40960 data: 2 2 2 2 2 2 2 2 3 3
1402 187/300 Write block 4946 len 8192 data: 2 2
1403 188/300 Read block 3970 len 24576
1404 189/300 Read block 2894 len 4096
1405 190/300 Write block 1057 len 20480 data: 2 2 2 2 2
1406 191/300 Write block 1958 len 28672 data: 2 2 2 2 2 2 2
1407 192/300 Write block 1284 len 20480 data: 2 2 2 2 2
1408 193/300 Read block 1102 len 12288
1409 194/300 Read block 15 len 12288
1410 195/300 Read block 73 len 32768
1411 196/300 Write block 3001 len 32768 data: 3 3 3 3 3 3 3 3
1412 197/300 Flush
1413 198/300 Flush
1414 199/300 Read block 893 len 12288
1415 200/300 Read block 1871 len 4096
1416 201/300 Read block 3500 len 4096
1417 202/300 Read block 4090 len 12288
1418 203/300 Read block 1123 len 4096
1419 204/300 Read block 4876 len 24576
1420 205/300 Read block 1822 len 4096
1421 206/300 Write block 3956 len 32768 data: 2 2 2 2 2 2 2 2
1422 207/300 Read block 1341 len 12288
1423 208/300 Flush
1424 209/300 Flush
1425 210/300 Write block 4696 len 36864 data: 2 2 2 2 2 2 2 2 2
1426 211/300 Read block 2240 len 24576
1427 212/300 Write block 3816 len 24576 data: 2 2 2 2 2 2
1428 213/300 Write block 3671 len 12288 data: 2 2 2
1429 214/300 Write block 1409 len 4096 data: 2
1430 215/300 Read block 6 len 24576
1431 216/300 Read block 2505 len 16384
1432 217/300 Flush
1433 218/300 Read block 4059 len 4096
1434 219/300 Read block 4459 len 12288
1435 220/300 Flush
1436 221/300 Read block 1555 len 20480
1437 222/300 Read block 2614 len 32768
1438 223/300 Read block 2550 len 8192
1439 224/300 Read block 4374 len 20480
1440 225/300 Read block 3071 len 12288
1441 226/300 Read block 4179 len 36864
1442 227/300 Flush
1443 228/300 Read block 94 len 16384
1444 229/300 Flush
1445 230/300 Write block 2050 len 16384 data: 2 2 2 2
1446 231/300 Write block 55 len 28672 data: 2 2 2 2 2 2 2
1447 232/300 Write block 2653 len 4096 data: 2
1448 233/300 Write block 1863 len 20480 data: 3 2 2 2 2
1449 234/300 Read block 2127 len 4096
1450 235/300 Write block 4904 len 24576 data: 3 2 2 2 2 2
1451 236/300 Write block 4075 len 12288 data: 2 2 2
1452 237/300 Write block 3315 len 40960 data: 3 3 3 3 3 4 4 3 3 2
1453 238/300 Read block 4114 len 32768
1454 239/300 Write block 4 len 16384 data: 3 2 2 2
1455 240/300 Read block 926 len 8192
1456 241/300 Write block 511 len 28672 data: 2 2 2 2 2 2 2
1457 242/300 Write block 3346 len 16384 data: 2 2 2 2
1458 243/300 Flush
1459 244/300 Flush
1460 245/300 Write block 4406 len 20480 data: 2 2 2 2 2
1461 246/300 Read block 2811 len 28672
1462 247/300 Write block 3294 len 36864 data: 3 4 4 4 4 4 4 4 3
1463 248/300 Write block 2359 len 8192 data: 2 2
1464 249/300 Read block 2433 len 20480
1465 250/300 Read block 537 len 32768
1466 251/300 Read block 4274 len 24576
1467 252/300 Read block 3195 len 36864
1468 253/300 Read block 692 len 36864
1469 254/300 Flush
1470 255/300 Read block 1109 len 28672
1471 256/300 Flush
1472 257/300 Read block 891 len 36864
1473 258/300 Read block 2518 len 24576
1474 259/300 Write block 849 len 4096 data: 2
1475 260/300 Write block 1511 len 24576 data: 4 3 2 2 2 2
1476 261/300 Read block 2631 len 24576
1477 262/300 Read block 267 len 8192
1478 263/300 Write block 2846 len 16384 data: 3 3 3 3
1479 264/300 Write block 1065 len 24576 data: 2 2 2 2 2 2
1480 265/300 Read block 3990 len 16384
1481 266/300 Write block 1692 len 20480 data: 2 2 2 2 2
1482 267/300 Read block 2232 len 32768
1483 268/300 Write block 4509 len 36864 data: 2 2 2 2 2 2 2 2 2
1484 269/300 Write block 509 len 8192 data: 2 2
1485 270/300 Read block 1602 len 8192
1486 271/300 Read block 2328 len 40960
1487 272/300 Read block 3898 len 8192
1488 273/300 Read block 4804 len 8192
1489 274/300 Read block 1447 len 8192
1490 275/300 Read block 336 len 8192
1491 276/300 Read block 1888 len 36864
1492 277/300 Write block 4442 len 20480 data: 2 2 2 2 2
1493 278/300 Read block 4280 len 36864
1494 279/300 Flush
1495 280/300 Flush
1496 281/300 Write block 4191 len 16384 data: 2 2 2 2
1497 282/300 Read block 552 len 36864
1498 283/300 Read block 1160 len 32768
1499 284/300 Write block 693 len 36864 data: 2 2 2 2 2 2 2 2 2
1500 285/300 Read block 1325 len 12288
1501 286/300 Read block 4168 len 16384
1502 287/300 Write block 3943 len 24576 data: 3 3 4 3 3 3
1503 288/300 Read block 2830 len 8192
1504 289/300 Read block 175 len 4096
1505 290/300 Read block 1939 len 32768
1506 291/300 Flush
1507 292/300 Read block 966 len 28672
1508 293/300 Read block 4963 len 28672
1509 294/300 Read block 2368 len 24576
1510 295/300 Write block 1512 len 32768 data: 4 3 3 3 3 2 3 3
1511 296/300 Write block 2201 len 16384 data: 2 2 2 2
1512 297/300 Read block 4349 len 8192
1513 298/300 Read block 2736 len 4096
1514 299/300 Read block 4193 len 24576
1515 300/300 Read block 4058 len 40960
1516 301/300 Read block 288 len 8192
1517 Main task has work to do, go find it
1518 got dsc Start(1)
1519 start 1
1520 Dec 04 13:31:02.637 INFO request completed, latency_us: 176, response_code: 204, uri: /start/cid/1, method: POST, req_id: 12eda8cb-1807-4fbc-8433-4c65ab4dc8c5, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
1521 Replay: started 1, returned:Ok(())
1522 ----------------------------------------------------------------
1523 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1524 States: Active Offline Active
1525 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1526 1302 Acked Write 6 Done Sent Done false
1527 1303 Acked Write 9 Done Sent Done false
1528 1304 Acked Read 1 Done Sent Done false
1529 1305 Acked Read 2 Done Sent Done false
1530 1306 Acked Read 5 Done Sent Done false
1531 1307 Acked Flush 0 Done Sent Done false
1532 1308 Acked Read 9 Done Sent Done false
1533 1309 Acked Read 5 Done Sent Done false
1534 1310 Acked Write 4 Done Sent Done false
1535 1311 Acked Read 4 Done Sent Done false
1536 1312 Acked Write 8 Done Sent Done false
1537 1313 Acked Write 3 Done Sent Done false
1538 1314 Acked Write 4 Done Sent Done false
1539 1315 Acked Read 2 Done Sent Done false
1540 1316 Acked Read 8 Done Sent Done false
1541 1317 Acked Write 10 Done Sent Done false
1542 1318 Acked Write 1 Done Sent Done false
1543 1319 Acked Read 9 Done Sent Done false
1544 1320 Acked Write 2 Done Sent Done false
1545 1321 Acked Read 5 Done Sent Done false
1546 1322 Acked Flush 0 Done Sent Done false
1547 1323 Acked Write 9 Done Sent Done false
1548 1324 Acked Write 10 Done Sent Done false
1549 1325 Acked Read 7 Done Sent Done false
1550 1326 Acked Write 4 Done Sent Done false
1551 1327 Acked Write 4 Done Sent Done false
1552 1328 Acked Read 4 Done Sent Done false
1553 1329 Acked Read 7 Done Sent Done false
1554 1330 Acked Read 7 Done Sent Done false
1555 1331 Acked Read 7 Done Sent Done false
1556 1332 Acked Write 4 Done Sent Done false
1557 1333 Acked Read 5 Done Sent Done false
1558 1334 Acked Write 3 Done Sent Done false
1559 1335 Acked Read 7 Done Sent Done false
1560 1336 Acked Write 6 Done Sent Done false
1561 1337 Acked Read 7 Done Sent Done false
1562 1338 Acked Read 1 Done Sent Done false
1563 1339 Acked Read 3 Done Sent Done false
1564 1340 Acked Write 1 Done Sent Done false
1565 1341 Acked Read 10 Done Sent Done false
1566 1342 Acked Read 3 Done Sent Done false
1567 1343 Acked Read 4 Done Sent Done false
1568 1344 Acked Read 4 Done Sent Done false
1569 1345 Acked Read 8 Done Sent Done false
1570 1346 Acked Write 7 Done Sent Done false
1571 1347 Acked Read 3 Done Sent Done false
1572 1348 Acked Read 3 Done Sent Done false
1573 1349 Acked Read 4 Done Sent Done false
1574 1350 Acked Read 6 Done Sent Done false
1575 1351 Acked Write 2 Done Sent Done false
1576 1352 Acked Flush 0 Done Sent Done false
1577 1353 Acked Read 10 Done Sent Done false
1578 1354 Acked Read 5 Done Sent Done false
1579 1355 Acked Read 1 Done Sent Done false
1580 1356 Acked Read 8 Done Sent Done false
1581 1357 Acked Write 6 Done Sent Done false
1582 1358 Acked Read 7 Done Sent Done false
1583 1359 Acked Read 2 Done Sent Done false
1584 1360 Acked Flush 0 Done Sent Done false
1585 1361 Acked Read 3 Done Sent Done false
1586 1362 Acked Read 8 Done Sent Done false
1587 1363 Acked Write 6 Done Sent Done false
1588 1364 Acked Flush 0 Done Sent Done false
1589 1365 Acked Read 7 Done Sent Done false
1590 1366 Acked Write 3 Done Sent Done false
1591 1367 Acked Read 9 Done Sent Done false
1592 1368 Acked Read 10 Done Sent Done false
1593 1369 Acked Write 9 Done Sent Done false
1594 1370 Acked Read 10 Done Sent Done false
1595 1371 Acked Flush 0 Done Sent Done false
1596 1372 Acked Write 9 Done Sent Done false
1597 1373 Acked Read 3 Done Sent Done false
1598 1374 Acked Write 1 Done Sent Done false
1599 1375 Acked Read 9 Done Sent Done false
1600 1376 Acked Flush 0 Done Sent Done false
1601 1377 Acked Write 1 Done Sent Done false
1602 1378 Acked Read 7 Done Sent Done false
1603 1379 Acked Read 3 Done Sent Done false
1604 1380 Acked Read 5 Done Sent Done false
1605 1381 Acked Read 10 Done Sent Done false
1606 1382 Acked Read 6 Done Sent Done false
1607 1383 Acked Write 7 Done Sent Done false
1608 1384 Acked Read 1 Done Sent Done false
1609 1385 Acked Read 8 Done Sent Done false
1610 1386 Acked Write 7 Done Sent Done false
1611 1387 Acked Read 4 Done Sent Done false
1612 1388 Acked Read 2 Done Sent Done false
1613 1389 Acked Read 3 Done Sent Done false
1614 1390 Acked Read 7 Done Sent Done false
1615 1391 Acked Read 5 Done Sent Done false
1616 1392 Acked Write 10 Done Sent Done false
1617 1393 Acked Write 10 Done Sent Done false
1618 1394 Acked Write 2 Done Sent Done false
1619 1395 Acked Read 5 Done Sent Done false
1620 1396 Acked Write 10 Done Sent Done false
1621 1397 Acked Write 9 Done Sent Done false
1622 1398 Acked Write 1 Done Sent Done false
1623 1399 Acked Read 3 Done Sent Done false
1624 1400 Acked Read 1 Done Sent Done false
1625 1401 Acked Write 8 Done Sent Done false
1626 1402 Acked Flush 0 Done Sent Done false
1627 1403 Acked Read 3 Done Sent Done false
1628 1404 Acked Write 1 Done Sent Done false
1629 1405 Acked Read 9 Done Sent Done false
1630 1406 Acked Read 6 Done Sent Done false
1631 1407 Acked Write 9 Done Sent Done false
1632 1408 Acked Read 7 Done Sent Done false
1633 1409 Acked Read 8 Done Sent Done false
1634 1410 Acked Read 5 Done Sent Done false
1635 1411 Acked Write 3 Done Sent Done false
1636 1412 Acked Read 6 Done Sent Done false
1637 1413 Acked Write 6 Done Sent Done false
1638 1414 Acked Read 5 Done Sent Done false
1639 1415 Acked Flush 0 Done Sent Done false
1640 1416 Acked Read 1 Done Sent Done false
1641 1417 Acked Read 1 Done Sent Done false
1642 1418 Acked Read 7 Done Sent Done false
1643 1419 Acked Write 1 Done Sent Done false
1644 1420 Acked Read 8 Done Sent Done false
1645 1421 Acked Read 3 Done Sent Done false
1646 1422 Acked Read 10 Done Sent Done false
1647 1423 Acked Read 6 Done Sent Done false
1648 1424 Acked Read 6 Done Sent Done false
1649 1425 Acked Read 9 Done Sent Done false
1650 1426 Acked Write 7 Done Sent Done false
1651 1427 Acked Read 4 Done Sent Done false
1652 1428 Acked Write 3 Done Sent Done false
1653 1429 Acked Read 5 Done Sent Done false
1654 1430 Acked Flush 0 Done Sent Done false
1655 1431 Acked Write 1 Done Sent Done false
1656 1432 Acked Read 3 Done Sent Done false
1657 1433 Acked Read 7 Done Sent Done false
1658 1434 Acked Read 10 Done Sent Done false
1659 1435 Acked Write 2 Done Sent Done false
1660 1436 Acked Read 6 Done Sent Done false
1661 1437 Acked Flush 0 Done Sent Done false
1662 1438 Acked Read 4 Done Sent Done false
1663 1439 Acked Write 7 Done Sent Done false
1664 1440 Acked Write 3 Done Sent Done false
1665 1441 Acked Read 6 Done Sent Done false
1666 1442 Acked Read 10 Done Sent Done false
1667 1443 Acked Write 6 Done Sent Done false
1668 1444 Acked Read 6 Done Sent Done false
1669 1445 Acked Write 10 Done Sent Done false
1670 1446 Acked Read 4 Done Sent Done false
1671 1447 Acked Write 7 Done Sent Done false
1672 1448 Acked Read 3 Done Sent Done false
1673 1449 Acked Write 6 Done Sent Done false
1674 1450 Acked Write 1 Done Sent Done false
1675 1451 Acked Read 7 Done Sent Done false
1676 1452 Acked Read 9 Done Sent Done false
1677 1453 Acked Read 8 Done Sent Done false
1678 1454 Acked Read 10 Done Sent Done false
1679 1455 Acked Write 4 Done Sent Done false
1680 1456 Acked Read 1 Done Sent Done false
1681 1457 Acked Write 9 Done Sent Done false
1682 1458 Acked Flush 0 Done Sent Done false
1683 1459 Acked Read 4 Done Sent Done false
1684 1460 Acked Read 3 Done Sent Done false
1685 1461 Acked Write 6 Done Sent Done false
1686 1462 Acked Flush 0 Done Sent Done false
1687 1463 Acked Read 4 Done Sent Done false
1688 1464 Acked Read 1 Done Sent Done false
1689 1465 Acked Write 3 Done Sent Done false
1690 1466 Acked Write 10 Done Sent Done false
1691 1467 Acked Read 10 Done Sent Done false
1692 1468 Acked Read 1 Done Sent Done false
1693 1469 Acked Write 7 Done Sent Done false
1694 1470 Acked Write 10 Done Sent Done false
1695 1471 Acked Read 6 Done Sent Done false
1696 1472 Acked Write 6 Done Sent Done false
1697 1473 Acked Write 5 Done Sent Done false
1698 1474 Acked Read 6 Done Sent Done false
1699 1475 Acked Write 6 Done Sent Done false
1700 1476 Acked Write 8 Done Sent Done false
1701 1477 Acked Read 2 Done Sent Done false
1702 1478 Acked Read 1 Done Sent Done false
1703 1479 Acked Write 9 Done Sent Done false
1704 1480 Acked Read 3 Done Sent Done false
1705 1481 Acked Read 1 Done Sent Done false
1706 1482 Acked Write 2 Done Sent Done false
1707 1483 Acked Write 8 Done Sent Done false
1708 1484 Acked Read 1 Done Sent Done false
1709 1485 Acked Flush 0 Done Sent Done false
1710 1486 Acked Read 6 Done Sent Done false
1711 1487 Acked Write 10 Done Sent Done false
1712 1488 Acked Write 2 Done Sent Done false
1713 1489 Acked Read 6 Done Sent Done false
1714 1490 Acked Read 1 Done Sent Done false
1715 1491 Acked Write 5 Done Sent Done false
1716 1492 Acked Write 7 Done Sent Done false
1717 1493 Acked Write 5 Done Sent Done false
1718 1494 Acked Read 3 Done Sent Done false
1719 1495 Acked Read 3 Done Sent Done false
1720 1496 Acked Read 8 Done Sent Done false
1721 1497 Acked Write 8 Done Sent Done false
1722 1498 Acked Flush 0 Done Sent Done false
1723 1499 Acked Flush 0 Done Sent Done false
1724 1500 Acked Read 3 Done Sent Done false
1725 1501 Acked Read 1 Done Sent Done false
1726 1502 Acked Read 1 Done Sent Done false
1727 1503 Acked Read 3 Done Sent Done false
1728 1504 Acked Read 1 Done Sent Done false
1729 1505 Acked Read 6 Done Sent Done false
1730 1506 Acked Read 1 Done Sent Done false
1731 1507 Acked Write 8 Done Sent Done false
1732 1508 Acked Read 3 Done Sent Done false
1733 1509 Acked Flush 0 Done Sent Done false
1734 1510 Acked Flush 0 Done Sent Done false
1735 1511 Acked Write 9 Done Sent Done false
1736 1512 Acked Read 6 Done Sent Done false
1737 1513 Acked Write 6 Done Sent Done false
1738 1514 Acked Write 3 Done Sent Done false
1739 1515 Acked Write 1 Done Sent Done false
1740 1516 Acked Read 6 Done Sent Done false
1741 1517 Acked Read 4 Done Sent Done false
1742 1518 Acked Flush 0 Done Sent Done false
1743 1519 Acked Read 1 Done Sent Done false
1744 1520 Acked Read 3 Done Sent Done false
1745 1521 Acked Flush 0 Done Sent Done false
1746 1522 Acked Read 5 Done Sent Done false
1747 1523 Acked Read 8 Done Sent Done false
1748 1524 Acked Read 2 Done Sent Done false
1749 1525 Acked Read 5 Done Sent Done false
1750 1526 Acked Read 3 Done Sent Done false
1751 1527 Acked Read 9 Done Sent Done false
1752 1528 Acked Flush 0 Done Sent Done false
1753 1529 Acked Read 4 Done Sent Done false
1754 1530 Acked Flush 0 Done Sent Done false
1755 1531 Acked Write 4 Done Sent Done false
1756 1532 Acked Write 7 Done Sent Done false
1757 1533 Acked Write 1 Done Sent Done false
1758 1534 Acked Write 5 Done Sent Done false
1759 1535 Acked Read 1 Done Sent Done false
1760 1536 Acked Write 6 Done Sent Done false
1761 1537 Acked Write 3 Done Sent Done false
1762 1538 Acked Write 10 Done Sent Done false
1763 1539 Acked Read 8 Done Sent Done false
1764 1540 Acked Write 4 Done Sent Done false
1765 1541 Acked Read 2 Done Sent Done false
1766 1542 Acked Write 7 Done Sent Done false
1767 1543 Acked Write 4 Done Sent Done false
1768 1544 Acked Flush 0 Done Sent Done false
1769 1545 Acked Flush 0 Done Sent Done false
1770 1546 Acked Write 5 Done Sent Done false
1771 1547 Acked Read 7 Done Sent Done false
1772 1548 Acked Write 9 Done Sent Done false
1773 1549 Acked Write 2 Done Sent Done false
1774 1550 Acked Read 5 Done Sent Done false
1775 1551 Acked Read 8 Done Sent Done false
1776 1552 Acked Read 6 Done Sent Done false
1777 1553 Acked Read 9 Done Sent Done false
1778 1554 Acked Read 9 Done Sent Done false
1779 1555 Acked Flush 0 Done Sent Done false
1780 1556 Acked Read 7 Done Sent Done false
1781 1557 Acked Flush 0 Done Sent Done false
1782 1558 Acked Read 9 Done Sent Done false
1783 1559 Acked Read 6 Done Sent Done false
1784 1560 Acked Write 1 Done Sent Done false
1785 1561 Acked Write 6 Done Sent Done false
1786 1562 Acked Read 6 Done Sent Done false
1787 1563 Acked Read 2 Done Sent Done false
1788 1564 Acked Write 4 Done Sent Done false
1789 1565 Acked Write 6 Done Sent Done false
1790 1566 Acked Read 4 Done Sent Done false
1791 1567 Acked Write 5 Done Sent Done false
1792 1568 Acked Read 8 Done Sent Done false
1793 1569 Acked Write 9 Done Sent Done false
1794 1570 Acked Write 2 Done Sent Done false
1795 1571 Acked Read 2 Done Sent Done false
1796 1572 Acked Read 10 Done Sent Done false
1797 1573 Acked Read 2 Done Sent Done false
1798 1574 Acked Read 2 Done Sent Done false
1799 1575 Acked Read 2 Done Sent Done false
1800 1576 Acked Read 2 Done Sent Done false
1801 1577 Acked Read 9 Done Sent Done false
1802 1578 Acked Write 5 Done Sent Done false
1803 1579 Acked Read 9 Done Sent Done false
1804 1580 Acked Flush 0 Done Sent Done false
1805 1581 Acked Flush 0 Done Sent Done false
1806 1582 Acked Write 4 Done Sent Done false
1807 1583 Acked Read 9 Done Sent Done false
1808 1584 Acked Read 8 Done Sent Done false
1809 1585 Acked Write 9 Done Sent Done false
1810 1586 Acked Read 3 Done Sent Done false
1811 1587 Acked Read 4 Done Sent Done false
1812 1588 Acked Write 6 Done Sent Done false
1813 1589 Acked Read 2 Done Sent Done false
1814 1590 Acked Read 1 Done Sent Done false
1815 1591 Acked Read 8 Done Sent Done false
1816 1592 Acked Flush 0 Done Sent Done false
1817 1593 Acked Read 7 Done Sent Done false
1818 1594 Acked Read 7 Done Sent Done false
1819 1595 Acked Read 6 Done Sent Done false
1820 1596 Acked Write 8 Done Sent Done false
1821 1597 Acked Write 4 Done Sent Done false
1822 1598 Acked Read 2 Done Sent Done false
1823 1599 Acked Read 1 Done Sent Done false
1824 1600 Acked Read 6 Done Sent Done false
1825 1601 Acked Read 10 Done Sent Done false
1826 1602 Acked Read 2 Done Sent Done false
1827 STATES DS:0 DS:1 DS:2 TOTAL
1828 Sent 0 301 0 301
1829 Done 301 0 301 602
1830 Skipped 0 0 0 0
1831 Error 0 0 0 0
1832 Last Flush: 1592 1301 1592
1833 Downstairs last five completed: 1301 1300 1299 1298 1297
1834 Upstairs last five completed: 1602 1601 1600 1599 1598
1835 CLIENT: Up:0 ds:301 act:2
1836 [8820] got start action
1837 [8820] I am going to restart
1838 Starting downstairs at port 8820
1839 Make output file at "/tmp/dsc/downstairs-8820.txt"
1840 [8820][1] reports Starting
1841 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1205)
1842 [8820][1] reports Running
1843 ----------------------------------------------------------------
1844 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1845 States: Active Offline Active
1846 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1847 1302 Acked Write 6 Done Sent Done false
1848 1303 Acked Write 9 Done Sent Done false
1849 1304 Acked Read 1 Done Sent Done false
1850 1305 Acked Read 2 Done Sent Done false
1851 1306 Acked Read 5 Done Sent Done false
1852 1307 Acked Flush 0 Done Sent Done false
1853 1308 Acked Read 9 Done Sent Done false
1854 1309 Acked Read 5 Done Sent Done false
1855 1310 Acked Write 4 Done Sent Done false
1856 1311 Acked Read 4 Done Sent Done false
1857 1312 Acked Write 8 Done Sent Done false
1858 1313 Acked Write 3 Done Sent Done false
1859 1314 Acked Write 4 Done Sent Done false
1860 1315 Acked Read 2 Done Sent Done false
1861 1316 Acked Read 8 Done Sent Done false
1862 1317 Acked Write 10 Done Sent Done false
1863 1318 Acked Write 1 Done Sent Done false
1864 1319 Acked Read 9 Done Sent Done false
1865 1320 Acked Write 2 Done Sent Done false
1866 1321 Acked Read 5 Done Sent Done false
1867 1322 Acked Flush 0 Done Sent Done false
1868 1323 Acked Write 9 Done Sent Done false
1869 1324 Acked Write 10 Done Sent Done false
1870 1325 Acked Read 7 Done Sent Done false
1871 1326 Acked Write 4 Done Sent Done false
1872 1327 Acked Write 4 Done Sent Done false
1873 1328 Acked Read 4 Done Sent Done false
1874 1329 Acked Read 7 Done Sent Done false
1875 1330 Acked Read 7 Done Sent Done false
1876 1331 Acked Read 7 Done Sent Done false
1877 1332 Acked Write 4 Done Sent Done false
1878 1333 Acked Read 5 Done Sent Done false
1879 1334 Acked Write 3 Done Sent Done false
1880 1335 Acked Read 7 Done Sent Done false
1881 1336 Acked Write 6 Done Sent Done false
1882 1337 Acked Read 7 Done Sent Done false
1883 1338 Acked Read 1 Done Sent Done false
1884 1339 Acked Read 3 Done Sent Done false
1885 1340 Acked Write 1 Done Sent Done false
1886 1341 Acked Read 10 Done Sent Done false
1887 1342 Acked Read 3 Done Sent Done false
1888 1343 Acked Read 4 Done Sent Done false
1889 1344 Acked Read 4 Done Sent Done false
1890 1345 Acked Read 8 Done Sent Done false
1891 1346 Acked Write 7 Done Sent Done false
1892 1347 Acked Read 3 Done Sent Done false
1893 1348 Acked Read 3 Done Sent Done false
1894 1349 Acked Read 4 Done Sent Done false
1895 1350 Acked Read 6 Done Sent Done false
1896 1351 Acked Write 2 Done Sent Done false
1897 1352 Acked Flush 0 Done Sent Done false
1898 1353 Acked Read 10 Done Sent Done false
1899 1354 Acked Read 5 Done Sent Done false
1900 1355 Acked Read 1 Done Sent Done false
1901 1356 Acked Read 8 Done Sent Done false
1902 1357 Acked Write 6 Done Sent Done false
1903 1358 Acked Read 7 Done Sent Done false
1904 1359 Acked Read 2 Done Sent Done false
1905 1360 Acked Flush 0 Done Sent Done false
1906 1361 Acked Read 3 Done Sent Done false
1907 1362 Acked Read 8 Done Sent Done false
1908 1363 Acked Write 6 Done Sent Done false
1909 1364 Acked Flush 0 Done Sent Done false
1910 1365 Acked Read 7 Done Sent Done false
1911 1366 Acked Write 3 Done Sent Done false
1912 1367 Acked Read 9 Done Sent Done false
1913 1368 Acked Read 10 Done Sent Done false
1914 1369 Acked Write 9 Done Sent Done false
1915 1370 Acked Read 10 Done Sent Done false
1916 1371 Acked Flush 0 Done Sent Done false
1917 1372 Acked Write 9 Done Sent Done false
1918 1373 Acked Read 3 Done Sent Done false
1919 1374 Acked Write 1 Done Sent Done false
1920 1375 Acked Read 9 Done Sent Done false
1921 1376 Acked Flush 0 Done Sent Done false
1922 1377 Acked Write 1 Done Sent Done false
1923 1378 Acked Read 7 Done Sent Done false
1924 1379 Acked Read 3 Done Sent Done false
1925 1380 Acked Read 5 Done Sent Done false
1926 1381 Acked Read 10 Done Sent Done false
1927 1382 Acked Read 6 Done Sent Done false
1928 1383 Acked Write 7 Done Sent Done false
1929 1384 Acked Read 1 Done Sent Done false
1930 1385 Acked Read 8 Done Sent Done false
1931 1386 Acked Write 7 Done Sent Done false
1932 1387 Acked Read 4 Done Sent Done false
1933 1388 Acked Read 2 Done Sent Done false
1934 1389 Acked Read 3 Done Sent Done false
1935 1390 Acked Read 7 Done Sent Done false
1936 1391 Acked Read 5 Done Sent Done false
1937 1392 Acked Write 10 Done Sent Done false
1938 1393 Acked Write 10 Done Sent Done false
1939 1394 Acked Write 2 Done Sent Done false
1940 1395 Acked Read 5 Done Sent Done false
1941 1396 Acked Write 10 Done Sent Done false
1942 1397 Acked Write 9 Done Sent Done false
1943 1398 Acked Write 1 Done Sent Done false
1944 1399 Acked Read 3 Done Sent Done false
1945 1400 Acked Read 1 Done Sent Done false
1946 1401 Acked Write 8 Done Sent Done false
1947 1402 Acked Flush 0 Done Sent Done false
1948 1403 Acked Read 3 Done Sent Done false
1949 1404 Acked Write 1 Done Sent Done false
1950 1405 Acked Read 9 Done Sent Done false
1951 1406 Acked Read 6 Done Sent Done false
1952 1407 Acked Write 9 Done Sent Done false
1953 1408 Acked Read 7 Done Sent Done false
1954 1409 Acked Read 8 Done Sent Done false
1955 1410 Acked Read 5 Done Sent Done false
1956 1411 Acked Write 3 Done Sent Done false
1957 1412 Acked Read 6 Done Sent Done false
1958 1413 Acked Write 6 Done Sent Done false
1959 1414 Acked Read 5 Done Sent Done false
1960 1415 Acked Flush 0 Done Sent Done false
1961 1416 Acked Read 1 Done Sent Done false
1962 1417 Acked Read 1 Done Sent Done false
1963 1418 Acked Read 7 Done Sent Done false
1964 1419 Acked Write 1 Done Sent Done false
1965 1420 Acked Read 8 Done Sent Done false
1966 1421 Acked Read 3 Done Sent Done false
1967 1422 Acked Read 10 Done Sent Done false
1968 1423 Acked Read 6 Done Sent Done false
1969 1424 Acked Read 6 Done Sent Done false
1970 1425 Acked Read 9 Done Sent Done false
1971 1426 Acked Write 7 Done Sent Done false
1972 1427 Acked Read 4 Done Sent Done false
1973 1428 Acked Write 3 Done Sent Done false
1974 1429 Acked Read 5 Done Sent Done false
1975 1430 Acked Flush 0 Done Sent Done false
1976 1431 Acked Write 1 Done Sent Done false
1977 1432 Acked Read 3 Done Sent Done false
1978 1433 Acked Read 7 Done Sent Done false
1979 1434 Acked Read 10 Done Sent Done false
1980 1435 Acked Write 2 Done Sent Done false
1981 1436 Acked Read 6 Done Sent Done false
1982 1437 Acked Flush 0 Done Sent Done false
1983 1438 Acked Read 4 Done Sent Done false
1984 1439 Acked Write 7 Done Sent Done false
1985 1440 Acked Write 3 Done Sent Done false
1986 1441 Acked Read 6 Done Sent Done false
1987 1442 Acked Read 10 Done Sent Done false
1988 1443 Acked Write 6 Done Sent Done false
1989 1444 Acked Read 6 Done Sent Done false
1990 1445 Acked Write 10 Done Sent Done false
1991 1446 Acked Read 4 Done Sent Done false
1992 1447 Acked Write 7 Done Sent Done false
1993 1448 Acked Read 3 Done Sent Done false
1994 1449 Acked Write 6 Done Sent Done false
1995 1450 Acked Write 1 Done Sent Done false
1996 1451 Acked Read 7 Done Sent Done false
1997 1452 Acked Read 9 Done Sent Done false
1998 1453 Acked Read 8 Done Sent Done false
1999 1454 Acked Read 10 Done Sent Done false
2000 1455 Acked Write 4 Done Sent Done false
2001 1456 Acked Read 1 Done Sent Done false
2002 1457 Acked Write 9 Done Sent Done false
2003 1458 Acked Flush 0 Done Sent Done false
2004 1459 Acked Read 4 Done Sent Done false
2005 1460 Acked Read 3 Done Sent Done false
2006 1461 Acked Write 6 Done Sent Done false
2007 1462 Acked Flush 0 Done Sent Done false
2008 1463 Acked Read 4 Done Sent Done false
2009 1464 Acked Read 1 Done Sent Done false
2010 1465 Acked Write 3 Done Sent Done false
2011 1466 Acked Write 10 Done Sent Done false
2012 1467 Acked Read 10 Done Sent Done false
2013 1468 Acked Read 1 Done Sent Done false
2014 1469 Acked Write 7 Done Sent Done false
2015 1470 Acked Write 10 Done Sent Done false
2016 1471 Acked Read 6 Done Sent Done false
2017 1472 Acked Write 6 Done Sent Done false
2018 1473 Acked Write 5 Done Sent Done false
2019 1474 Acked Read 6 Done Sent Done false
2020 1475 Acked Write 6 Done Sent Done false
2021 1476 Acked Write 8 Done Sent Done false
2022 1477 Acked Read 2 Done Sent Done false
2023 1478 Acked Read 1 Done Sent Done false
2024 1479 Acked Write 9 Done Sent Done false
2025 1480 Acked Read 3 Done Sent Done false
2026 1481 Acked Read 1 Done Sent Done false
2027 1482 Acked Write 2 Done Sent Done false
2028 1483 Acked Write 8 Done Sent Done false
2029 1484 Acked Read 1 Done Sent Done false
2030 1485 Acked Flush 0 Done Sent Done false
2031 1486 Acked Read 6 Done Sent Done false
2032 1487 Acked Write 10 Done Sent Done false
2033 1488 Acked Write 2 Done Sent Done false
2034 1489 Acked Read 6 Done Sent Done false
2035 1490 Acked Read 1 Done Sent Done false
2036 1491 Acked Write 5 Done Sent Done false
2037 1492 Acked Write 7 Done Sent Done false
2038 1493 Acked Write 5 Done Sent Done false
2039 1494 Acked Read 3 Done Sent Done false
2040 1495 Acked Read 3 Done Sent Done false
2041 1496 Acked Read 8 Done Sent Done false
2042 1497 Acked Write 8 Done Sent Done false
2043 1498 Acked Flush 0 Done Sent Done false
2044 1499 Acked Flush 0 Done Sent Done false
2045 1500 Acked Read 3 Done Sent Done false
2046 1501 Acked Read 1 Done Sent Done false
2047 1502 Acked Read 1 Done Sent Done false
2048 1503 Acked Read 3 Done Sent Done false
2049 1504 Acked Read 1 Done Sent Done false
2050 1505 Acked Read 6 Done Sent Done false
2051 1506 Acked Read 1 Done Sent Done false
2052 1507 Acked Write 8 Done Sent Done false
2053 1508 Acked Read 3 Done Sent Done false
2054 1509 Acked Flush 0 Done Sent Done false
2055 1510 Acked Flush 0 Done Sent Done false
2056 1511 Acked Write 9 Done Sent Done false
2057 1512 Acked Read 6 Done Sent Done false
2058 1513 Acked Write 6 Done Sent Done false
2059 1514 Acked Write 3 Done Sent Done false
2060 1515 Acked Write 1 Done Sent Done false
2061 1516 Acked Read 6 Done Sent Done false
2062 1517 Acked Read 4 Done Sent Done false
2063 1518 Acked Flush 0 Done Sent Done false
2064 1519 Acked Read 1 Done Sent Done false
2065 1520 Acked Read 3 Done Sent Done false
2066 1521 Acked Flush 0 Done Sent Done false
2067 1522 Acked Read 5 Done Sent Done false
2068 1523 Acked Read 8 Done Sent Done false
2069 1524 Acked Read 2 Done Sent Done false
2070 1525 Acked Read 5 Done Sent Done false
2071 1526 Acked Read 3 Done Sent Done false
2072 1527 Acked Read 9 Done Sent Done false
2073 1528 Acked Flush 0 Done Sent Done false
2074 1529 Acked Read 4 Done Sent Done false
2075 1530 Acked Flush 0 Done Sent Done false
2076 1531 Acked Write 4 Done Sent Done false
2077 1532 Acked Write 7 Done Sent Done false
2078 1533 Acked Write 1 Done Sent Done false
2079 1534 Acked Write 5 Done Sent Done false
2080 1535 Acked Read 1 Done Sent Done false
2081 1536 Acked Write 6 Done Sent Done false
2082 1537 Acked Write 3 Done Sent Done false
2083 1538 Acked Write 10 Done Sent Done false
2084 1539 Acked Read 8 Done Sent Done false
2085 1540 Acked Write 4 Done Sent Done false
2086 1541 Acked Read 2 Done Sent Done false
2087 1542 Acked Write 7 Done Sent Done false
2088 1543 Acked Write 4 Done Sent Done false
2089 1544 Acked Flush 0 Done Sent Done false
2090 1545 Acked Flush 0 Done Sent Done false
2091 1546 Acked Write 5 Done Sent Done false
2092 1547 Acked Read 7 Done Sent Done false
2093 1548 Acked Write 9 Done Sent Done false
2094 1549 Acked Write 2 Done Sent Done false
2095 1550 Acked Read 5 Done Sent Done false
2096 1551 Acked Read 8 Done Sent Done false
2097 1552 Acked Read 6 Done Sent Done false
2098 1553 Acked Read 9 Done Sent Done false
2099 1554 Acked Read 9 Done Sent Done false
2100 1555 Acked Flush 0 Done Sent Done false
2101 1556 Acked Read 7 Done Sent Done false
2102 1557 Acked Flush 0 Done Sent Done false
2103 1558 Acked Read 9 Done Sent Done false
2104 1559 Acked Read 6 Done Sent Done false
2105 1560 Acked Write 1 Done Sent Done false
2106 1561 Acked Write 6 Done Sent Done false
2107 1562 Acked Read 6 Done Sent Done false
2108 1563 Acked Read 2 Done Sent Done false
2109 1564 Acked Write 4 Done Sent Done false
2110 1565 Acked Write 6 Done Sent Done false
2111 1566 Acked Read 4 Done Sent Done false
2112 1567 Acked Write 5 Done Sent Done false
2113 1568 Acked Read 8 Done Sent Done false
2114 1569 Acked Write 9 Done Sent Done false
2115 1570 Acked Write 2 Done Sent Done false
2116 1571 Acked Read 2 Done Sent Done false
2117 1572 Acked Read 10 Done Sent Done false
2118 1573 Acked Read 2 Done Sent Done false
2119 1574 Acked Read 2 Done Sent Done false
2120 1575 Acked Read 2 Done Sent Done false
2121 1576 Acked Read 2 Done Sent Done false
2122 1577 Acked Read 9 Done Sent Done false
2123 1578 Acked Write 5 Done Sent Done false
2124 1579 Acked Read 9 Done Sent Done false
2125 1580 Acked Flush 0 Done Sent Done false
2126 1581 Acked Flush 0 Done Sent Done false
2127 1582 Acked Write 4 Done Sent Done false
2128 1583 Acked Read 9 Done Sent Done false
2129 1584 Acked Read 8 Done Sent Done false
2130 1585 Acked Write 9 Done Sent Done false
2131 1586 Acked Read 3 Done Sent Done false
2132 1587 Acked Read 4 Done Sent Done false
2133 1588 Acked Write 6 Done Sent Done false
2134 1589 Acked Read 2 Done Sent Done false
2135 1590 Acked Read 1 Done Sent Done false
2136 1591 Acked Read 8 Done Sent Done false
2137 1592 Acked Flush 0 Done Sent Done false
2138 1593 Acked Read 7 Done Sent Done false
2139 1594 Acked Read 7 Done Sent Done false
2140 1595 Acked Read 6 Done Sent Done false
2141 1596 Acked Write 8 Done Sent Done false
2142 1597 Acked Write 4 Done Sent Done false
2143 1598 Acked Read 2 Done Sent Done false
2144 1599 Acked Read 1 Done Sent Done false
2145 1600 Acked Read 6 Done Sent Done false
2146 1601 Acked Read 10 Done Sent Done false
2147 1602 Acked Read 2 Done Sent Done false
2148 1603 Acked Flush 0 Done Sent Done false
2149 STATES DS:0 DS:1 DS:2 TOTAL
2150 Sent 0 302 0 302
2151 Done 302 0 302 604
2152 Skipped 0 0 0 0
2153 Error 0 0 0 0
2154 Last Flush: 1603 1301 1603
2155 Downstairs last five completed: 1301 1300 1299 1298 1297
2156 Upstairs last five completed: 1603 1602 1601 1600 1599
2157 CLIENT: Up:0 ds:302 act:2
21582024-12-04T13:31:08.398ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21592024-12-04T13:31:08.398ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49332) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21602024-12-04T13:31:08.399ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21612024-12-04T13:31:08.399ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21622024-12-04T13:31:08.399ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21632024-12-04T13:31:08.399ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21642024-12-04T13:31:08.399ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21652024-12-04T13:31:08.399ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21662024-12-04T13:31:08.399ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21672024-12-04T13:31:08.399ZINFOcrucible: [1] client re-new 302 jobs since flush 1301 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21682024-12-04T13:31:08.400ZINFOcrucible: [1] Replayed 302 jobs since flush: 1301 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2169 ----------------------------------------------------------------
2170 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2171 Downstairs last five completed: 1603 1602 1601 1600 1599
2172 Upstairs last five completed: 1603 1602 1601 1600 1599
2173 CLIENT: Up:0 ds:0 act:3
2174 Replay: All jobs finished
2175 Main task has work to do, go find it
2176 got dsc Stop(2)
2177 stop 2
2178 Dec 04 13:31:10.641 INFO request completed, latency_us: 310, response_code: 204, uri: /stop/cid/2, method: POST[8830] Got stop action so:false kr:false
2179 , req_id: a4bf0d53-a97a-40de-bc8c-aa2a45380535, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
2180 Dec 04 13:31:10.642 INFO request completed, latency_us: 127, response_code: 200, uri: /state/cid/2, method: GET, req_id: 150a4cd4-9413-42c8-bfdf-20620ad7908e, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
21812024-12-04T13:31:10.641ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21822024-12-04T13:31:10.642ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21832024-12-04T13:31:10.642ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21842024-12-04T13:31:10.642ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21852024-12-04T13:31:10.642ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21862024-12-04T13:31:10.642ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
2187 [8830] Exited with: ExitStatus(unix_wait_status(9))
2188 [8830][2] reports Exit
2189 Dec 04 13:31:14.644 INFO request completed, latency_us: 250, response_code: 200, uri: /state/cid/2, method: GET, req_id: cabd0941-80be-4e70-867a-80352453d489, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
2190 001/300 Write block 196 len 12288 data: 2 2 2
2191 002/300 Write block 1888 len 12288 data: 3 3 3
2192 003/300 Flush
2193 004/300 Write block 3255 len 4096 data: 2
2194 005/300 Flush
2195 006/300 Read block 3664 len 16384
2196 007/300 Write block 1293 len 4096 data: 2
2197 008/300 Write block 4719 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2198 009/300 Read block 4224 len 4096
2199 010/300 Read block 740 len 4096
2200 011/300 Read block 3003 len 40960
2201 012/300 Read block 529 len 8192
2202 013/300 Read block 2421 len 36864
2203 014/300 Write block 773 len 12288 data: 2 2 2
2204 015/300 Write block 2727 len 36864 data: 2 2 2 2 2 2 2 2 2
2205 016/300 Write block 4159 len 36864 data: 3 3 3 3 3 2 2 2 2
2206 017/300 Read block 3053 len 40960
2207 018/300 Write block 3327 len 12288 data: 2 2 2
2208 019/300 Read block 3283 len 8192
2209 020/300 Read block 3136 len 28672
2210 021/300 Flush
2211 022/300 Write block 260 len 20480 data: 2 2 2 2 2
2212 023/300 Flush
2213 024/300 Write block 2313 len 4096 data: 2
2214 025/300 Flush
2215 026/300 Read block 3876 len 12288
2216 027/300 Write block 962 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2217 028/300 Read block 4836 len 4096
2218 029/300 Write block 683 len 20480 data: 2 2 2 2 2
2219 030/300 Write block 3401 len 16384 data: 2 2 2 2
2220 031/300 Write block 294 len 20480 data: 2 2 2 2 2
2221 032/300 Read block 3829 len 24576
2222 033/300 Write block 2134 len 20480 data: 3 3 3 2 2
2223 034/300 Write block 1136 len 12288 data: 2 2 2
2224 035/300 Write block 126 len 32768 data: 3 2 2 2 2 2 2 2
2225 036/300 Write block 3018 len 24576 data: 2 2 2 3 3 3
2226 037/300 Read block 1711 len 4096
2227 038/300 Flush
2228 039/300 Read block 3519 len 8192
2229 040/300 Read block 3311 len 12288
2230 041/300 Read block 1196 len 36864
2231 042/300 Read block 2121 len 28672
2232 043/300 Read block 1738 len 20480
2233 044/300 Write block 1847 len 28672 data: 2 2 2 2 2 2 2
2234 045/300 Write block 4405 len 40960 data: 2 3 3 3 3 3 2 2 2 2
2235 046/300 Flush
2236 047/300 Write block 582 len 8192 data: 2 2
2237 048/300 Write block 3594 len 8192 data: 2 2
2238 049/300 Read block 395 len 20480
2239 050/300 Read block 2283 len 24576
2240 051/300 Read block 4095 len 32768
2241 052/300 Write block 3988 len 20480 data: 2 2 3 3 3
2242 053/300 Read block 4154 len 4096
2243 054/300 Write block 185 len 32768 data: 2 2 2 2 2 2 2 2
2244 055/300 Read block 1906 len 36864
2245 056/300 Write block 4798 len 20480 data: 2 2 2 2 2
2246 057/300 Read block 4799 len 28672
2247 058/300 Read block 3870 len 32768
2248 059/300 Read block 358 len 4096
2249 060/300 Read block 3821 len 40960
2250 061/300 Read block 1030 len 32768
2251 062/300 Read block 325 len 16384
2252 063/300 Read block 4714 len 8192
2253 064/300 Read block 4433 len 12288
2254 065/300 Write block 214 len 12288 data: 2 2 2
2255 066/300 Flush
2256 067/300 Write block 2664 len 16384 data: 3 3 3 3
2257 068/300 Flush
2258 069/300 Write block 498 len 16384 data: 2 2 2 2
2259 070/300 Read block 2526 len 8192
2260 071/300 Read block 392 len 32768
2261 072/300 Write block 4162 len 8192 data: 4 4
2262 073/300 Read block 3253 len 20480
2263 074/300 Write block 4131 len 4096 data: 2
2264 075/300 Read block 4039 len 24576
2265 076/300 Write block 1053 len 12288 data: 2 2 2
2266 077/300 Flush
2267 078/300 Read block 2664 len 32768
2268 079/300 Write block 330 len 8192 data: 2 2
2269 080/300 Write block 3979 len 16384 data: 2 2 2 2
2270 081/300 Read block 4492 len 20480
2271 082/300 Read block 4660 len 24576
2272 083/300 Read block 4817 len 32768
2273 084/300 Read block 950 len 4096
2274 085/300 Write block 1189 len 24576 data: 3 2 2 2 2 2
2275 086/300 Read block 3242 len 12288
2276 087/300 Read block 1394 len 28672
2277 088/300 Read block 4978 len 4096
2278 089/300 Read block 2305 len 32768
2279 090/300 Read block 2803 len 12288
2280 091/300 Flush
2281 092/300 Flush
2282 093/300 Write block 3609 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2283 094/300 Flush
2284 095/300 Write block 1867 len 32768 data: 3 2 2 2 2 2 2 2
2285 096/300 Read block 611 len 28672
2286 097/300 Read block 4291 len 8192
2287 098/300 Flush
2288 099/300 Read block 2313 len 12288
2289 100/300 Write block 166 len 40960 data: 3 3 3 3 3 3 2 2 2 2
2290 101/300 Flush
2291 102/300 Read block 1109 len 20480
2292 103/300 Read block 3449 len 24576
2293 104/300 Read block 1037 len 16384
2294 105/300 Read block 2435 len 12288
2295 106/300 Read block 3818 len 32768
2296 107/300 Write block 2646 len 12288 data: 2 2 2
2297 108/300 Read block 4381 len 40960
2298 109/300 Read block 3215 len 40960
2299 110/300 Read block 816 len 4096
2300 111/300 Read block 4456 len 12288
2301 112/300 Read block 2663 len 12288
2302 113/300 Write block 3034 len 24576 data: 2 2 2 2 2 2
2303 114/300 Write block 1 len 24576 data: 3 4 3 4 3 3
2304 115/300 Read block 3361 len 12288
2305 116/300 Write block 442 len 4096 data: 2
2306 117/300 Read block 702 len 36864
2307 118/300 Read block 803 len 12288
2308 119/300 Write block 360 len 20480 data: 2 2 2 2 2
2309 120/300 Read block 3259 len 8192
2310 121/300 Write block 4461 len 16384 data: 2 3 3 3
2311 122/300 Write block 4338 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2312 123/300 Read block 1187 len 12288
2313 124/300 Read block 3364 len 24576
2314 125/300 Read block 618 len 36864
2315 126/300 Read block 2985 len 28672
2316 127/300 Write block 947 len 20480 data: 2 2 2 2 2
2317 128/300 Read block 1954 len 8192
2318 129/300 Write block 1112 len 16384 data: 2 2 2 2
2319 130/300 Read block 2037 len 16384
2320 131/300 Write block 3216 len 28672 data: 4 3 3 2 2 2 2
2321 132/300 Read block 1907 len 16384
2322 133/300 Write block 3242 len 12288 data: 2 2 2
2323 134/300 Write block 1753 len 24576 data: 2 2 2 2 2 2
2324 135/300 Read block 458 len 28672
2325 136/300 Write block 1692 len 40960 data: 3 3 3 3 3 2 2 2 2 2
2326 137/300 Read block 466 len 32768
2327 138/300 Write block 4121 len 16384 data: 2 2 2 2
2328 139/300 Write block 4435 len 40960 data: 2 2 2 2 2 2 2 3 3 3
2329 140/300 Write block 1147 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2330 141/300 Write block 4844 len 32768 data: 2 2 2 2 2 2 2 2
2331 142/300 Write block 4781 len 16384 data: 2 2 2 2
2332 143/300 Write block 4465 len 32768 data: 3 3 3 4 4 4 3 3
2333 144/300 Read block 4612 len 24576
2334 145/300 Read block 2839 len 4096
2335 146/300 Read block 2827 len 16384
2336 147/300 Read block 1855 len 32768
2337 148/300 Write block 3594 len 12288 data: 3 3 2
2338 149/300 Flush
2339 150/300 Flush
2340 151/300 Read block 1306 len 28672
2341 152/300 Read block 3265 len 12288
2342 153/300 Read block 1115 len 32768
2343 154/300 Read block 2103 len 24576
2344 155/300 Flush
2345 156/300 Flush
2346 157/300 Read block 3453 len 28672
2347 158/300 Write block 562 len 16384 data: 2 2 2 2
2348 159/300 Read block 483 len 16384
2349 160/300 Write block 2071 len 12288 data: 2 2 2
2350 161/300 Write block 3090 len 12288 data: 2 2 2
2351 162/300 Write block 1737 len 24576 data: 2 2 2 2 4 5
2352 163/300 Read block 3512 len 36864
2353 164/300 Write block 1905 len 4096 data: 2
2354 165/300 Read block 4042 len 28672
2355 166/300 Flush
2356 167/300 Write block 920 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2357 168/300 Write block 3438 len 20480 data: 2 2 2 2 2
2358 169/300 Read block 1554 len 36864
2359 170/300 Read block 2192 len 24576
2360 171/300 Read block 2428 len 24576
2361 172/300 Write block 489 len 12288 data: 3 3 3
2362 173/300 Read block 1226 len 4096
2363 174/300 Read block 1329 len 36864
2364 175/300 Flush
2365 176/300 Read block 653 len 32768
2366 177/300 Read block 4360 len 8192
2367 178/300 Read block 1190 len 40960
2368 179/300 Read block 833 len 36864
2369 180/300 Read block 540 len 36864
2370 181/300 Flush
2371 182/300 Read block 2281 len 24576
2372 183/300 Write block 3133 len 24576 data: 2 2 2 2 2 2
2373 184/300 Write block 1945 len 28672 data: 3 2 2 2 2 2 2
2374 185/300 Write block 3065 len 4096 data: 2
2375 186/300 Write block 2677 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2376 187/300 Flush
2377 188/300 Read block 2121 len 32768
2378 189/300 Read block 2370 len 24576
2379 190/300 Write block 3623 len 28672 data: 2 2 3 2 2 2 2
2380 191/300 Read block 993 len 16384
2381 192/300 Write block 2042 len 24576 data: 3 3 4 4 4 2
2382 193/300 Read block 4425 len 20480
2383 194/300 Write block 1761 len 36864 data: 2 2 2 2 2 2 2 3 3
2384 195/300 Read block 3737 len 4096
2385 196/300 Flush
2386 197/300 Write block 3377 len 40960 data: 2 3 3 3 3 3 3 2 2 2
2387 198/300 Write block 1474 len 20480 data: 2 2 2 2 2
2388 199/300 Write block 2771 len 28672 data: 2 3 3 3 3 2 2
2389 200/300 Write block 3442 len 12288 data: 3 2 2
2390 201/300 Read block 3465 len 32768
2391 202/300 Read block 1452 len 12288
2392 203/300 Read block 1492 len 8192
2393 204/300 Write block 3736 len 36864 data: 2 2 2 2 2 2 2 2 2
2394 205/300 Read block 4465 len 20480
2395 206/300 Read block 4810 len 8192
2396 207/300 Write block 4972 len 36864 data: 3 2 2 2 2 2 2 2 2
2397 208/300 Write block 1128 len 4096 data: 2
2398 209/300 Read block 3906 len 16384
2399 210/300 Write block 3691 len 12288 data: 2 2 2
2400 211/300 Write block 3347 len 20480 data: 3 3 3 2 2
2401 212/300 Read block 1088 len 12288
2402 213/300 Read block 3192 len 12288
2403 214/300 Read block 3183 len 32768
2404 215/300 Write block 1036 len 28672 data: 2 2 2 2 2 2 2
2405 216/300 Write block 3390 len 28672 data: 2 2 2 2 2 2 2
2406 217/300 Flush
2407 218/300 Read block 4200 len 4096
2408 219/300 Write block 2154 len 20480 data: 2 2 2 2 2
2409 220/300 Read block 1728 len 8192
2410 221/300 Read block 2891 len 36864
2411 222/300 Read block 2895 len 20480
2412 223/300 Flush
2413 224/300 Write block 2364 len 32768 data: 2 2 2 2 3 3 3 3
2414 225/300 Write block 3108 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2415 226/300 Write block 4968 len 8192 data: 3 3
2416 227/300 Read block 1873 len 32768
2417 228/300 Read block 4164 len 4096
2418 229/300 Read block 1742 len 20480
2419 230/300 Write block 2941 len 16384 data: 2 2 2 2
2420 231/300 Write block 3921 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2421 232/300 Read block 1216 len 20480
2422 233/300 Read block 2132 len 20480
2423 234/300 Write block 3373 len 32768 data: 2 2 2 2 3 4 4 4
2424 235/300 Write block 1525 len 8192 data: 3 3
2425 236/300 Write block 4128 len 40960 data: 2 2 2 3 2 2 2 2 2 2
2426 237/300 Write block 2004 len 36864 data: 2 2 2 2 2 2 2 2 2
2427 238/300 Read block 2023 len 28672
2428 239/300 Read block 4831 len 24576
2429 240/300 Read block 3988 len 20480
2430 241/300 Write block 1296 len 12288 data: 2 2 2
2431 242/300 Read block 1064 len 4096
2432 243/300 Read block 4095 len 36864
2433 244/300 Write block 1852 len 16384 data: 3 3 2 2
2434 245/300 Read block 1873 len 20480
2435 246/300 Write block 97 len 24576 data: 2 2 2 2 2 2
2436 247/300 Write block 1332 len 40960 data: 2 2 2 2 2 3 3 3 3 3
2437 248/300 Write block 3827 len 28672 data: 3 3 3 2 2 2 2
2438 249/300 Read block 2079 len 32768
2439 250/300 Flush
2440 251/300 Write block 3218 len 32768 data: 4 3 3 3 3 2 2 2
2441 252/300 Write block 67 len 36864 data: 2 2 2 2 2 2 2 2 2
2442 253/300 Read block 4498 len 8192
2443 254/300 Write block 3192 len 28672 data: 2 2 2 2 2 2 2
2444 255/300 Read block 3880 len 16384
2445 256/300 Read block 2987 len 36864
2446 257/300 Flush
2447 258/300 Write block 2054 len 32768 data: 2 2 2 2 2 2 2 2
2448 259/300 Read block 1791 len 12288
2449 260/300 Read block 1897 len 40960
2450 261/300 Read block 4959 len 24576
2451 262/300 Write block 1056 len 36864 data: 2 3 3 3 3 3 2 2 2
2452 263/300 Read block 2955 len 8192
2453 264/300 Read block 1748 len 20480
2454 265/300 Read block 662 len 8192
2455 266/300 Read block 1393 len 20480
2456 267/300 Write block 86 len 28672 data: 2 2 2 2 2 2 2
2457 268/300 Write block 796 len 20480 data: 2 2 2 2 2
2458 269/300 Write block 2980 len 24576 data: 2 2 2 2 2 2
2459 270/300 Flush
2460 271/300 Flush
2461 272/300 Write block 3761 len 24576 data: 2 2 2 2 2 2
2462 273/300 Read block 2758 len 20480
2463 274/300 Read block 4967 len 24576
2464 275/300 Read block 3168 len 16384
2465 276/300 Read block 874 len 20480
2466 277/300 Read block 3877 len 8192
2467 278/300 Read block 2913 len 36864
2468 279/300 Write block 2537 len 20480 data: 2 2 2 3 3
2469 280/300 Write block 2170 len 4096 data: 2
2470 281/300 Write block 4497 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2471 282/300 Write block 3168 len 20480 data: 2 2 2 2 2
2472 283/300 Write block 1371 len 8192 data: 2 3
2473 284/300 Write block 2950 len 20480 data: 2 2 2 2 2
2474 285/300 Read block 4398 len 40960
2475 286/300 Read block 4906 len 12288
2476 287/300 Write block 4916 len 32768 data: 2 2 2 2 2 2 3 3
2477 288/300 Read block 4182 len 12288
2478 289/300 Flush
2479 290/300 Read block 1574 len 36864
2480 291/300 Read block 1837 len 28672
2481 292/300 Read block 4936 len 32768
2482 293/300 Write block 4153 len 16384 data: 2 2 3 3
2483 294/300 Write block 918 len 12288 data: 2 2 3
2484 295/300 Read block 4874 len 8192
2485 296/300 Read block 3396 len 20480
2486 297/300 Read block 2884 len 32768
2487 298/300 Flush
2488 299/300 Read block 2984 len 20480
2489 300/300 Read block 4705 len 40960
2490 301/300 Read block 1724 len 32768
2491 Main task has work to do, go find it
2492 got dsc Start(2)
2493 start 2
2494 Dec 04 13:31:14.877 INFO request completed, latency_us: 203, response_code: 204, uri: /start/cid/2, method: POST, req_id: be848e92-73ea-4eda-87f4-a6f191db381e, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
2495 Replay: started 2, returned:Ok(())
2496 ----------------------------------------------------------------
2497 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2498 States: Active Active Offline
2499 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2500 1604 Acked Write 3 Done Done Sent false
2501 1605 Acked Write 3 Done Done Sent false
2502 1606 Acked Flush 0 Done Done Sent false
2503 1607 Acked Write 1 Done Done Sent false
2504 1608 Acked Flush 0 Done Done Sent false
2505 1609 Acked Read 4 Done Done Sent false
2506 1610 Acked Write 1 Done Done Sent false
2507 1611 Acked Write 10 Done Done Sent false
2508 1612 Acked Read 1 Done Done Sent false
2509 1613 Acked Read 1 Done Done Sent false
2510 1614 Acked Read 10 Done Done Sent false
2511 1615 Acked Read 2 Done Done Sent false
2512 1616 Acked Read 9 Done Done Sent false
2513 1617 Acked Write 3 Done Done Sent false
2514 1618 Acked Write 9 Done Done Sent false
2515 1619 Acked Write 9 Done Done Sent false
2516 1620 Acked Read 10 Done Done Sent false
2517 1621 Acked Write 3 Done Done Sent false
2518 1622 Acked Read 2 Done Done Sent false
2519 1623 Acked Read 7 Done Done Sent false
2520 1624 Acked Flush 0 Done Done Sent false
2521 1625 Acked Write 5 Done Done Sent false
2522 1626 Acked Flush 0 Done Done Sent false
2523 1627 Acked Write 1 Done Done Sent false
2524 1628 Acked Flush 0 Done Done Sent false
2525 1629 Acked Read 3 Done Done Sent false
2526 1630 Acked Write 10 Done Done Sent false
2527 1631 Acked Read 1 Done Done Sent false
2528 1632 Acked Write 5 Done Done Sent false
2529 1633 Acked Write 4 Done Done Sent false
2530 1634 Acked Write 5 Done Done Sent false
2531 1635 Acked Read 6 Done Done Sent false
2532 1636 Acked Write 5 Done Done Sent false
2533 1637 Acked Write 3 Done Done Sent false
2534 1638 Acked Write 8 Done Done Sent false
2535 1639 Acked Write 6 Done Done Sent false
2536 1640 Acked Read 1 Done Done Sent false
2537 1641 Acked Flush 0 Done Done Sent false
2538 1642 Acked Read 2 Done Done Sent false
2539 1643 Acked Read 3 Done Done Sent false
2540 1644 Acked Read 9 Done Done Sent false
2541 1645 Acked Read 7 Done Done Sent false
2542 1646 Acked Read 5 Done Done Sent false
2543 1647 Acked Write 7 Done Done Sent false
2544 1648 Acked Write 10 Done Done Sent false
2545 1649 Acked Flush 0 Done Done Sent false
2546 1650 Acked Write 2 Done Done Sent false
2547 1651 Acked Write 2 Done Done Sent false
2548 1652 Acked Read 5 Done Done Sent false
2549 1653 Acked Read 6 Done Done Sent false
2550 1654 Acked Read 8 Done Done Sent false
2551 1655 Acked Write 5 Done Done Sent false
2552 1656 Acked Read 1 Done Done Sent false
2553 1657 Acked Write 8 Done Done Sent false
2554 1658 Acked Read 9 Done Done Sent false
2555 1659 Acked Write 5 Done Done Sent false
2556 1660 Acked Read 7 Done Done Sent false
2557 1661 Acked Read 8 Done Done Sent false
2558 1662 Acked Read 1 Done Done Sent false
2559 1663 Acked Read 10 Done Done Sent false
2560 1664 Acked Read 8 Done Done Sent false
2561 1665 Acked Read 4 Done Done Sent false
2562 1666 Acked Read 2 Done Done Sent false
2563 1667 Acked Read 3 Done Done Sent false
2564 1668 Acked Write 3 Done Done Sent false
2565 1669 Acked Flush 0 Done Done Sent false
2566 1670 Acked Write 4 Done Done Sent false
2567 1671 Acked Flush 0 Done Done Sent false
2568 1672 Acked Write 4 Done Done Sent false
2569 1673 Acked Read 2 Done Done Sent false
2570 1674 Acked Read 8 Done Done Sent false
2571 1675 Acked Write 2 Done Done Sent false
2572 1676 Acked Read 5 Done Done Sent false
2573 1677 Acked Write 1 Done Done Sent false
2574 1678 Acked Read 6 Done Done Sent false
2575 1679 Acked Write 3 Done Done Sent false
2576 1680 Acked Flush 0 Done Done Sent false
2577 1681 Acked Read 8 Done Done Sent false
2578 1682 Acked Write 2 Done Done Sent false
2579 1683 Acked Write 4 Done Done Sent false
2580 1684 Acked Read 5 Done Done Sent false
2581 1685 Acked Read 6 Done Done Sent false
2582 1686 Acked Read 8 Done Done Sent false
2583 1687 Acked Read 1 Done Done Sent false
2584 1688 Acked Write 6 Done Done Sent false
2585 1689 Acked Read 3 Done Done Sent false
2586 1690 Acked Read 7 Done Done Sent false
2587 1691 Acked Read 1 Done Done Sent false
2588 1692 Acked Read 8 Done Done Sent false
2589 1693 Acked Read 3 Done Done Sent false
2590 1694 Acked Flush 0 Done Done Sent false
2591 1695 Acked Flush 0 Done Done Sent false
2592 1696 Acked Write 10 Done Done Sent false
2593 1697 Acked Flush 0 Done Done Sent false
2594 1698 Acked Write 8 Done Done Sent false
2595 1699 Acked Read 7 Done Done Sent false
2596 1700 Acked Read 2 Done Done Sent false
2597 1701 Acked Flush 0 Done Done Sent false
2598 1702 Acked Read 3 Done Done Sent false
2599 1703 Acked Write 10 Done Done Sent false
2600 1704 Acked Flush 0 Done Done Sent false
2601 1705 Acked Read 5 Done Done Sent false
2602 1706 Acked Read 6 Done Done Sent false
2603 1707 Acked Read 4 Done Done Sent false
2604 1708 Acked Read 3 Done Done Sent false
2605 1709 Acked Read 8 Done Done Sent false
2606 1710 Acked Write 3 Done Done Sent false
2607 1711 Acked Read 10 Done Done Sent false
2608 1712 Acked Read 10 Done Done Sent false
2609 1713 Acked Read 1 Done Done Sent false
2610 1714 Acked Read 3 Done Done Sent false
2611 1715 Acked Read 3 Done Done Sent false
2612 1716 Acked Write 6 Done Done Sent false
2613 1717 Acked Write 6 Done Done Sent false
2614 1718 Acked Read 3 Done Done Sent false
2615 1719 Acked Write 1 Done Done Sent false
2616 1720 Acked Read 9 Done Done Sent false
2617 1721 Acked Read 3 Done Done Sent false
2618 1722 Acked Write 5 Done Done Sent false
2619 1723 Acked Read 2 Done Done Sent false
2620 1724 Acked Write 4 Done Done Sent false
2621 1725 Acked Write 10 Done Done Sent false
2622 1726 Acked Read 3 Done Done Sent false
2623 1727 Acked Read 6 Done Done Sent false
2624 1728 Acked Read 9 Done Done Sent false
2625 1729 Acked Read 7 Done Done Sent false
2626 1730 Acked Write 5 Done Done Sent false
2627 1731 Acked Read 2 Done Done Sent false
2628 1732 Acked Write 4 Done Done Sent false
2629 1733 Acked Read 4 Done Done Sent false
2630 1734 Acked Write 7 Done Done Sent false
2631 1735 Acked Read 4 Done Done Sent false
2632 1736 Acked Write 3 Done Done Sent false
2633 1737 Acked Write 6 Done Done Sent false
2634 1738 Acked Read 7 Done Done Sent false
2635 1739 Acked Write 10 Done Done Sent false
2636 1740 Acked Read 8 Done Done Sent false
2637 1741 Acked Write 4 Done Done Sent false
2638 1742 Acked Write 10 Done Done Sent false
2639 1743 Acked Write 10 Done Done Sent false
2640 1744 Acked Write 8 Done Done Sent false
2641 1745 Acked Write 4 Done Done Sent false
2642 1746 Acked Write 8 Done Done Sent false
2643 1747 Acked Read 6 Done Done Sent false
2644 1748 Acked Read 1 Done Done Sent false
2645 1749 Acked Read 4 Done Done Sent false
2646 1750 Acked Read 8 Done Done Sent false
2647 1751 Acked Write 3 Done Done Sent false
2648 1752 Acked Flush 0 Done Done Sent false
2649 1753 Acked Flush 0 Done Done Sent false
2650 1754 Acked Read 7 Done Done Sent false
2651 1755 Acked Read 3 Done Done Sent false
2652 1756 Acked Read 8 Done Done Sent false
2653 1757 Acked Read 6 Done Done Sent false
2654 1758 Acked Flush 0 Done Done Sent false
2655 1759 Acked Flush 0 Done Done Sent false
2656 1760 Acked Read 7 Done Done Sent false
2657 1761 Acked Write 4 Done Done Sent false
2658 1762 Acked Read 4 Done Done Sent false
2659 1763 Acked Write 3 Done Done Sent false
2660 1764 Acked Write 3 Done Done Sent false
2661 1765 Acked Write 6 Done Done Sent false
2662 1766 Acked Read 9 Done Done Sent false
2663 1767 Acked Write 1 Done Done Sent false
2664 1768 Acked Read 7 Done Done Sent false
2665 1769 Acked Flush 0 Done Done Sent false
2666 1770 Acked Write 10 Done Done Sent false
2667 1771 Acked Write 5 Done Done Sent false
2668 1772 Acked Read 9 Done Done Sent false
2669 1773 Acked Read 6 Done Done Sent false
2670 1774 Acked Read 6 Done Done Sent false
2671 1775 Acked Write 3 Done Done Sent false
2672 1776 Acked Read 1 Done Done Sent false
2673 1777 Acked Read 9 Done Done Sent false
2674 1778 Acked Flush 0 Done Done Sent false
2675 1779 Acked Read 8 Done Done Sent false
2676 1780 Acked Read 2 Done Done Sent false
2677 1781 Acked Read 10 Done Done Sent false
2678 1782 Acked Read 9 Done Done Sent false
2679 1783 Acked Read 9 Done Done Sent false
2680 1784 Acked Flush 0 Done Done Sent false
2681 1785 Acked Read 6 Done Done Sent false
2682 1786 Acked Write 6 Done Done Sent false
2683 1787 Acked Write 7 Done Done Sent false
2684 1788 Acked Write 1 Done Done Sent false
2685 1789 Acked Write 10 Done Done Sent false
2686 1790 Acked Flush 0 Done Done Sent false
2687 1791 Acked Read 8 Done Done Sent false
2688 1792 Acked Read 6 Done Done Sent false
2689 1793 Acked Write 7 Done Done Sent false
2690 1794 Acked Read 4 Done Done Sent false
2691 1795 Acked Write 6 Done Done Sent false
2692 1796 Acked Read 5 Done Done Sent false
2693 1797 Acked Write 9 Done Done Sent false
2694 1798 Acked Read 1 Done Done Sent false
2695 1799 Acked Flush 0 Done Done Sent false
2696 1800 Acked Write 10 Done Done Sent false
2697 1801 Acked Write 5 Done Done Sent false
2698 1802 Acked Write 7 Done Done Sent false
2699 1803 Acked Write 3 Done Done Sent false
2700 1804 Acked Read 8 Done Done Sent false
2701 1805 Acked Read 3 Done Done Sent false
2702 1806 Acked Read 2 Done Done Sent false
2703 1807 Acked Write 9 Done Done Sent false
2704 1808 Acked Read 5 Done Done Sent false
2705 1809 Acked Read 2 Done Done Sent false
2706 1810 Acked Write 9 Done Done Sent false
2707 1811 Acked Write 1 Done Done Sent false
2708 1812 Acked Read 4 Done Done Sent false
2709 1813 Acked Write 3 Done Done Sent false
2710 1814 Acked Write 5 Done Done Sent false
2711 1815 Acked Read 3 Done Done Sent false
2712 1816 Acked Read 3 Done Done Sent false
2713 1817 Acked Read 8 Done Done Sent false
2714 1818 Acked Write 7 Done Done Sent false
2715 1819 Acked Write 7 Done Done Sent false
2716 1820 Acked Flush 0 Done Done Sent false
2717 1821 Acked Read 1 Done Done Sent false
2718 1822 Acked Write 5 Done Done Sent false
2719 1823 Acked Read 2 Done Done Sent false
2720 1824 Acked Read 9 Done Done Sent false
2721 1825 Acked Read 5 Done Done Sent false
2722 1826 Acked Flush 0 Done Done Sent false
2723 1827 Acked Write 8 Done Done Sent false
2724 1828 Acked Write 10 Done Done Sent false
2725 1829 Acked Write 2 Done Done Sent false
2726 1830 Acked Read 8 Done Done Sent false
2727 1831 Acked Read 1 Done Done Sent false
2728 1832 Acked Read 5 Done Done Sent false
2729 1833 Acked Write 4 Done Done Sent false
2730 1834 Acked Write 10 Done Done Sent false
2731 1835 Acked Read 5 Done Done Sent false
2732 1836 Acked Read 5 Done Done Sent false
2733 1837 Acked Write 8 Done Done Sent false
2734 1838 Acked Write 2 Done Done Sent false
2735 1839 Acked Write 10 Done Done Sent false
2736 1840 Acked Write 9 Done Done Sent false
2737 1841 Acked Read 7 Done Done Sent false
2738 1842 Acked Read 6 Done Done Sent false
2739 1843 Acked Read 5 Done Done Sent false
2740 1844 Acked Write 3 Done Done Sent false
2741 1845 Acked Read 1 Done Done Sent false
2742 1846 Acked Read 9 Done Done Sent false
2743 1847 Acked Write 4 Done Done Sent false
2744 1848 Acked Read 5 Done Done Sent false
2745 1849 Acked Write 6 Done Done Sent false
2746 1850 Acked Write 10 Done Done Sent false
2747 1851 Acked Write 7 Done Done Sent false
2748 1852 Acked Read 8 Done Done Sent false
2749 1853 Acked Flush 0 Done Done Sent false
2750 1854 Acked Write 8 Done Done Sent false
2751 1855 Acked Write 9 Done Done Sent false
2752 1856 Acked Read 2 Done Done Sent false
2753 1857 Acked Write 7 Done Done Sent false
2754 1858 Acked Read 4 Done Done Sent false
2755 1859 Acked Read 9 Done Done Sent false
2756 1860 Acked Flush 0 Done Done Sent false
2757 1861 Acked Write 8 Done Done Sent false
2758 1862 Acked Read 3 Done Done Sent false
2759 1863 Acked Read 10 Done Done Sent false
2760 1864 Acked Read 6 Done Done Sent false
2761 1865 Acked Write 9 Done Done Sent false
2762 1866 Acked Read 2 Done Done Sent false
2763 1867 Acked Read 5 Done Done Sent false
2764 1868 Acked Read 2 Done Done Sent false
2765 1869 Acked Read 5 Done Done Sent false
2766 1870 Acked Write 7 Done Done Sent false
2767 1871 Acked Write 5 Done Done Sent false
2768 1872 Acked Write 6 Done Done Sent false
2769 1873 Acked Flush 0 Done Done Sent false
2770 1874 Acked Flush 0 Done Done Sent false
2771 1875 Acked Write 6 Done Done Sent false
2772 1876 Acked Read 5 Done Done Sent false
2773 1877 Acked Read 6 Done Done Sent false
2774 1878 Acked Read 4 Done Done Sent false
2775 1879 Acked Read 5 Done Done Sent false
2776 1880 Acked Read 2 Done Done Sent false
2777 1881 Acked Read 9 Done Done Sent false
2778 1882 Acked Write 5 Done Done Sent false
2779 1883 Acked Write 1 Done Done Sent false
2780 1884 Acked Write 10 Done Done Sent false
2781 1885 Acked Write 5 Done Done Sent false
2782 1886 Acked Write 2 Done Done Sent false
2783 1887 Acked Write 5 Done Done Sent false
2784 1888 Acked Read 10 Done Done Sent false
2785 1889 Acked Read 3 Done Done Sent false
2786 1890 Acked Write 8 Done Done Sent false
2787 1891 Acked Read 3 Done Done Sent false
2788 1892 Acked Flush 0 Done Done Sent false
2789 1893 Acked Read 9 Done Done Sent false
2790 1894 Acked Read 7 Done Done Sent false
2791 1895 Acked Read 8 Done Done Sent false
2792 1896 Acked Write 4 Done Done Sent false
2793 1897 Acked Write 3 Done Done Sent false
2794 1898 Acked Read 2 Done Done Sent false
2795 1899 Acked Read 5 Done Done Sent false
2796 1900 Acked Read 8 Done Done Sent false
2797 1901 Acked Flush 0 Done Done Sent false
2798 1902 Acked Read 5 Done Done Sent false
2799 1903 Acked Read 10 Done Done Sent false
2800 1904 Acked Read 8 Done Done Sent false
2801 STATES DS:0 DS:1 DS:2 TOTAL
2802 Sent 0 0 301 301
2803 Done 301 301 0 602
2804 Skipped 0 0 0 0
2805 Error 0 0 0 0
2806 Last Flush: 1901 1901 1603
2807 Downstairs last five completed: 1603 1602 1601 1600 1599
2808 Upstairs last five completed: 1904 1903 1902 1901 1900
2809 CLIENT: Up:0 ds:301 act:2
2810 [8830] got start action
2811 [8830] I am going to restart
2812 Starting downstairs at port 8830
2813 Make output file at "/tmp/dsc/downstairs-8830.txt"
2814 [8830][2] reports Starting
2815 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1206)
2816 [8830][2] reports Running
2817 ----------------------------------------------------------------
2818 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2819 States: Active Active Offline
2820 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2821 1604 Acked Write 3 Done Done Sent false
2822 1605 Acked Write 3 Done Done Sent false
2823 1606 Acked Flush 0 Done Done Sent false
2824 1607 Acked Write 1 Done Done Sent false
2825 1608 Acked Flush 0 Done Done Sent false
2826 1609 Acked Read 4 Done Done Sent false
2827 1610 Acked Write 1 Done Done Sent false
2828 1611 Acked Write 10 Done Done Sent false
2829 1612 Acked Read 1 Done Done Sent false
2830 1613 Acked Read 1 Done Done Sent false
2831 1614 Acked Read 10 Done Done Sent false
2832 1615 Acked Read 2 Done Done Sent false
2833 1616 Acked Read 9 Done Done Sent false
2834 1617 Acked Write 3 Done Done Sent false
2835 1618 Acked Write 9 Done Done Sent false
2836 1619 Acked Write 9 Done Done Sent false
2837 1620 Acked Read 10 Done Done Sent false
2838 1621 Acked Write 3 Done Done Sent false
2839 1622 Acked Read 2 Done Done Sent false
2840 1623 Acked Read 7 Done Done Sent false
2841 1624 Acked Flush 0 Done Done Sent false
2842 1625 Acked Write 5 Done Done Sent false
2843 1626 Acked Flush 0 Done Done Sent false
2844 1627 Acked Write 1 Done Done Sent false
2845 1628 Acked Flush 0 Done Done Sent false
2846 1629 Acked Read 3 Done Done Sent false
2847 1630 Acked Write 10 Done Done Sent false
2848 1631 Acked Read 1 Done Done Sent false
2849 1632 Acked Write 5 Done Done Sent false
2850 1633 Acked Write 4 Done Done Sent false
2851 1634 Acked Write 5 Done Done Sent false
2852 1635 Acked Read 6 Done Done Sent false
2853 1636 Acked Write 5 Done Done Sent false
2854 1637 Acked Write 3 Done Done Sent false
2855 1638 Acked Write 8 Done Done Sent false
2856 1639 Acked Write 6 Done Done Sent false
2857 1640 Acked Read 1 Done Done Sent false
2858 1641 Acked Flush 0 Done Done Sent false
2859 1642 Acked Read 2 Done Done Sent false
2860 1643 Acked Read 3 Done Done Sent false
2861 1644 Acked Read 9 Done Done Sent false
2862 1645 Acked Read 7 Done Done Sent false
2863 1646 Acked Read 5 Done Done Sent false
2864 1647 Acked Write 7 Done Done Sent false
2865 1648 Acked Write 10 Done Done Sent false
2866 1649 Acked Flush 0 Done Done Sent false
2867 1650 Acked Write 2 Done Done Sent false
2868 1651 Acked Write 2 Done Done Sent false
2869 1652 Acked Read 5 Done Done Sent false
2870 1653 Acked Read 6 Done Done Sent false
2871 1654 Acked Read 8 Done Done Sent false
2872 1655 Acked Write 5 Done Done Sent false
2873 1656 Acked Read 1 Done Done Sent false
2874 1657 Acked Write 8 Done Done Sent false
2875 1658 Acked Read 9 Done Done Sent false
2876 1659 Acked Write 5 Done Done Sent false
2877 1660 Acked Read 7 Done Done Sent false
2878 1661 Acked Read 8 Done Done Sent false
2879 1662 Acked Read 1 Done Done Sent false
2880 1663 Acked Read 10 Done Done Sent false
2881 1664 Acked Read 8 Done Done Sent false
2882 1665 Acked Read 4 Done Done Sent false
2883 1666 Acked Read 2 Done Done Sent false
2884 1667 Acked Read 3 Done Done Sent false
2885 1668 Acked Write 3 Done Done Sent false
2886 1669 Acked Flush 0 Done Done Sent false
2887 1670 Acked Write 4 Done Done Sent false
2888 1671 Acked Flush 0 Done Done Sent false
2889 1672 Acked Write 4 Done Done Sent false
2890 1673 Acked Read 2 Done Done Sent false
2891 1674 Acked Read 8 Done Done Sent false
2892 1675 Acked Write 2 Done Done Sent false
2893 1676 Acked Read 5 Done Done Sent false
2894 1677 Acked Write 1 Done Done Sent false
2895 1678 Acked Read 6 Done Done Sent false
2896 1679 Acked Write 3 Done Done Sent false
2897 1680 Acked Flush 0 Done Done Sent false
2898 1681 Acked Read 8 Done Done Sent false
2899 1682 Acked Write 2 Done Done Sent false
2900 1683 Acked Write 4 Done Done Sent false
2901 1684 Acked Read 5 Done Done Sent false
2902 1685 Acked Read 6 Done Done Sent false
2903 1686 Acked Read 8 Done Done Sent false
2904 1687 Acked Read 1 Done Done Sent false
2905 1688 Acked Write 6 Done Done Sent false
2906 1689 Acked Read 3 Done Done Sent false
2907 1690 Acked Read 7 Done Done Sent false
2908 1691 Acked Read 1 Done Done Sent false
2909 1692 Acked Read 8 Done Done Sent false
2910 1693 Acked Read 3 Done Done Sent false
2911 1694 Acked Flush 0 Done Done Sent false
2912 1695 Acked Flush 0 Done Done Sent false
2913 1696 Acked Write 10 Done Done Sent false
2914 1697 Acked Flush 0 Done Done Sent false
2915 1698 Acked Write 8 Done Done Sent false
2916 1699 Acked Read 7 Done Done Sent false
2917 1700 Acked Read 2 Done Done Sent false
2918 1701 Acked Flush 0 Done Done Sent false
2919 1702 Acked Read 3 Done Done Sent false
2920 1703 Acked Write 10 Done Done Sent false
2921 1704 Acked Flush 0 Done Done Sent false
2922 1705 Acked Read 5 Done Done Sent false
2923 1706 Acked Read 6 Done Done Sent false
2924 1707 Acked Read 4 Done Done Sent false
2925 1708 Acked Read 3 Done Done Sent false
2926 1709 Acked Read 8 Done Done Sent false
2927 1710 Acked Write 3 Done Done Sent false
2928 1711 Acked Read 10 Done Done Sent false
2929 1712 Acked Read 10 Done Done Sent false
2930 1713 Acked Read 1 Done Done Sent false
2931 1714 Acked Read 3 Done Done Sent false
2932 1715 Acked Read 3 Done Done Sent false
2933 1716 Acked Write 6 Done Done Sent false
2934 1717 Acked Write 6 Done Done Sent false
2935 1718 Acked Read 3 Done Done Sent false
2936 1719 Acked Write 1 Done Done Sent false
2937 1720 Acked Read 9 Done Done Sent false
2938 1721 Acked Read 3 Done Done Sent false
2939 1722 Acked Write 5 Done Done Sent false
2940 1723 Acked Read 2 Done Done Sent false
2941 1724 Acked Write 4 Done Done Sent false
2942 1725 Acked Write 10 Done Done Sent false
2943 1726 Acked Read 3 Done Done Sent false
2944 1727 Acked Read 6 Done Done Sent false
2945 1728 Acked Read 9 Done Done Sent false
2946 1729 Acked Read 7 Done Done Sent false
2947 1730 Acked Write 5 Done Done Sent false
2948 1731 Acked Read 2 Done Done Sent false
2949 1732 Acked Write 4 Done Done Sent false
2950 1733 Acked Read 4 Done Done Sent false
2951 1734 Acked Write 7 Done Done Sent false
2952 1735 Acked Read 4 Done Done Sent false
2953 1736 Acked Write 3 Done Done Sent false
2954 1737 Acked Write 6 Done Done Sent false
2955 1738 Acked Read 7 Done Done Sent false
2956 1739 Acked Write 10 Done Done Sent false
2957 1740 Acked Read 8 Done Done Sent false
2958 1741 Acked Write 4 Done Done Sent false
2959 1742 Acked Write 10 Done Done Sent false
2960 1743 Acked Write 10 Done Done Sent false
2961 1744 Acked Write 8 Done Done Sent false
2962 1745 Acked Write 4 Done Done Sent false
2963 1746 Acked Write 8 Done Done Sent false
2964 1747 Acked Read 6 Done Done Sent false
2965 1748 Acked Read 1 Done Done Sent false
2966 1749 Acked Read 4 Done Done Sent false
2967 1750 Acked Read 8 Done Done Sent false
2968 1751 Acked Write 3 Done Done Sent false
2969 1752 Acked Flush 0 Done Done Sent false
2970 1753 Acked Flush 0 Done Done Sent false
2971 1754 Acked Read 7 Done Done Sent false
2972 1755 Acked Read 3 Done Done Sent false
2973 1756 Acked Read 8 Done Done Sent false
2974 1757 Acked Read 6 Done Done Sent false
2975 1758 Acked Flush 0 Done Done Sent false
2976 1759 Acked Flush 0 Done Done Sent false
2977 1760 Acked Read 7 Done Done Sent false
2978 1761 Acked Write 4 Done Done Sent false
2979 1762 Acked Read 4 Done Done Sent false
2980 1763 Acked Write 3 Done Done Sent false
2981 1764 Acked Write 3 Done Done Sent false
2982 1765 Acked Write 6 Done Done Sent false
2983 1766 Acked Read 9 Done Done Sent false
2984 1767 Acked Write 1 Done Done Sent false
2985 1768 Acked Read 7 Done Done Sent false
2986 1769 Acked Flush 0 Done Done Sent false
2987 1770 Acked Write 10 Done Done Sent false
2988 1771 Acked Write 5 Done Done Sent false
2989 1772 Acked Read 9 Done Done Sent false
2990 1773 Acked Read 6 Done Done Sent false
2991 1774 Acked Read 6 Done Done Sent false
2992 1775 Acked Write 3 Done Done Sent false
2993 1776 Acked Read 1 Done Done Sent false
2994 1777 Acked Read 9 Done Done Sent false
2995 1778 Acked Flush 0 Done Done Sent false
2996 1779 Acked Read 8 Done Done Sent false
2997 1780 Acked Read 2 Done Done Sent false
2998 1781 Acked Read 10 Done Done Sent false
2999 1782 Acked Read 9 Done Done Sent false
3000 1783 Acked Read 9 Done Done Sent false
3001 1784 Acked Flush 0 Done Done Sent false
3002 1785 Acked Read 6 Done Done Sent false
3003 1786 Acked Write 6 Done Done Sent false
3004 1787 Acked Write 7 Done Done Sent false
3005 1788 Acked Write 1 Done Done Sent false
3006 1789 Acked Write 10 Done Done Sent false
3007 1790 Acked Flush 0 Done Done Sent false
3008 1791 Acked Read 8 Done Done Sent false
3009 1792 Acked Read 6 Done Done Sent false
3010 1793 Acked Write 7 Done Done Sent false
3011 1794 Acked Read 4 Done Done Sent false
3012 1795 Acked Write 6 Done Done Sent false
3013 1796 Acked Read 5 Done Done Sent false
3014 1797 Acked Write 9 Done Done Sent false
3015 1798 Acked Read 1 Done Done Sent false
3016 1799 Acked Flush 0 Done Done Sent false
3017 1800 Acked Write 10 Done Done Sent false
3018 1801 Acked Write 5 Done Done Sent false
3019 1802 Acked Write 7 Done Done Sent false
3020 1803 Acked Write 3 Done Done Sent false
3021 1804 Acked Read 8 Done Done Sent false
3022 1805 Acked Read 3 Done Done Sent false
3023 1806 Acked Read 2 Done Done Sent false
3024 1807 Acked Write 9 Done Done Sent false
3025 1808 Acked Read 5 Done Done Sent false
3026 1809 Acked Read 2 Done Done Sent false
3027 1810 Acked Write 9 Done Done Sent false
3028 1811 Acked Write 1 Done Done Sent false
3029 1812 Acked Read 4 Done Done Sent false
3030 1813 Acked Write 3 Done Done Sent false
3031 1814 Acked Write 5 Done Done Sent false
3032 1815 Acked Read 3 Done Done Sent false
3033 1816 Acked Read 3 Done Done Sent false
3034 1817 Acked Read 8 Done Done Sent false
3035 1818 Acked Write 7 Done Done Sent false
3036 1819 Acked Write 7 Done Done Sent false
3037 1820 Acked Flush 0 Done Done Sent false
3038 1821 Acked Read 1 Done Done Sent false
3039 1822 Acked Write 5 Done Done Sent false
3040 1823 Acked Read 2 Done Done Sent false
3041 1824 Acked Read 9 Done Done Sent false
3042 1825 Acked Read 5 Done Done Sent false
3043 1826 Acked Flush 0 Done Done Sent false
3044 1827 Acked Write 8 Done Done Sent false
3045 1828 Acked Write 10 Done Done Sent false
3046 1829 Acked Write 2 Done Done Sent false
3047 1830 Acked Read 8 Done Done Sent false
3048 1831 Acked Read 1 Done Done Sent false
3049 1832 Acked Read 5 Done Done Sent false
3050 1833 Acked Write 4 Done Done Sent false
3051 1834 Acked Write 10 Done Done Sent false
3052 1835 Acked Read 5 Done Done Sent false
3053 1836 Acked Read 5 Done Done Sent false
3054 1837 Acked Write 8 Done Done Sent false
3055 1838 Acked Write 2 Done Done Sent false
3056 1839 Acked Write 10 Done Done Sent false
3057 1840 Acked Write 9 Done Done Sent false
3058 1841 Acked Read 7 Done Done Sent false
3059 1842 Acked Read 6 Done Done Sent false
3060 1843 Acked Read 5 Done Done Sent false
3061 1844 Acked Write 3 Done Done Sent false
3062 1845 Acked Read 1 Done Done Sent false
3063 1846 Acked Read 9 Done Done Sent false
3064 1847 Acked Write 4 Done Done Sent false
3065 1848 Acked Read 5 Done Done Sent false
3066 1849 Acked Write 6 Done Done Sent false
3067 1850 Acked Write 10 Done Done Sent false
3068 1851 Acked Write 7 Done Done Sent false
3069 1852 Acked Read 8 Done Done Sent false
3070 1853 Acked Flush 0 Done Done Sent false
3071 1854 Acked Write 8 Done Done Sent false
3072 1855 Acked Write 9 Done Done Sent false
3073 1856 Acked Read 2 Done Done Sent false
3074 1857 Acked Write 7 Done Done Sent false
3075 1858 Acked Read 4 Done Done Sent false
3076 1859 Acked Read 9 Done Done Sent false
3077 1860 Acked Flush 0 Done Done Sent false
3078 1861 Acked Write 8 Done Done Sent false
3079 1862 Acked Read 3 Done Done Sent false
3080 1863 Acked Read 10 Done Done Sent false
3081 1864 Acked Read 6 Done Done Sent false
3082 1865 Acked Write 9 Done Done Sent false
3083 1866 Acked Read 2 Done Done Sent false
3084 1867 Acked Read 5 Done Done Sent false
3085 1868 Acked Read 2 Done Done Sent false
3086 1869 Acked Read 5 Done Done Sent false
3087 1870 Acked Write 7 Done Done Sent false
3088 1871 Acked Write 5 Done Done Sent false
3089 1872 Acked Write 6 Done Done Sent false
3090 1873 Acked Flush 0 Done Done Sent false
3091 1874 Acked Flush 0 Done Done Sent false
3092 1875 Acked Write 6 Done Done Sent false
3093 1876 Acked Read 5 Done Done Sent false
3094 1877 Acked Read 6 Done Done Sent false
3095 1878 Acked Read 4 Done Done Sent false
3096 1879 Acked Read 5 Done Done Sent false
3097 1880 Acked Read 2 Done Done Sent false
3098 1881 Acked Read 9 Done Done Sent false
3099 1882 Acked Write 5 Done Done Sent false
3100 1883 Acked Write 1 Done Done Sent false
3101 1884 Acked Write 10 Done Done Sent false
3102 1885 Acked Write 5 Done Done Sent false
3103 1886 Acked Write 2 Done Done Sent false
3104 1887 Acked Write 5 Done Done Sent false
3105 1888 Acked Read 10 Done Done Sent false
3106 1889 Acked Read 3 Done Done Sent false
3107 1890 Acked Write 8 Done Done Sent false
3108 1891 Acked Read 3 Done Done Sent false
3109 1892 Acked Flush 0 Done Done Sent false
3110 1893 Acked Read 9 Done Done Sent false
3111 1894 Acked Read 7 Done Done Sent false
3112 1895 Acked Read 8 Done Done Sent false
3113 1896 Acked Write 4 Done Done Sent false
3114 1897 Acked Write 3 Done Done Sent false
3115 1898 Acked Read 2 Done Done Sent false
3116 1899 Acked Read 5 Done Done Sent false
3117 1900 Acked Read 8 Done Done Sent false
3118 1901 Acked Flush 0 Done Done Sent false
3119 1902 Acked Read 5 Done Done Sent false
3120 1903 Acked Read 10 Done Done Sent false
3121 1904 Acked Read 8 Done Done Sent false
3122 1905 Acked Flush 0 Done Done Sent false
3123 STATES DS:0 DS:1 DS:2 TOTAL
3124 Sent 0 0 302 302
3125 Done 302 302 0 604
3126 Skipped 0 0 0 0
3127 Error 0 0 0 0
3128 Last Flush: 1905 1905 1603
3129 Downstairs last five completed: 1603 1602 1601 1600 1599
3130 Upstairs last five completed: 1905 1904 1903 1902 1901
3131 CLIENT: Up:0 ds:302 act:2
31322024-12-04T13:31:20.643ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31332024-12-04T13:31:20.643ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59531) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31342024-12-04T13:31:20.644ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31352024-12-04T13:31:20.645ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31362024-12-04T13:31:20.645ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31372024-12-04T13:31:20.645ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31382024-12-04T13:31:20.645ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31392024-12-04T13:31:20.645ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31402024-12-04T13:31:20.645ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31412024-12-04T13:31:20.645ZINFOcrucible: [2] client re-new 302 jobs since flush 1603 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31422024-12-04T13:31:20.645ZINFOcrucible: [2] Replayed 302 jobs since flush: 1603 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
3143 ----------------------------------------------------------------
3144 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3145 Downstairs last five completed: 1905 1904 1903 1902 1901
3146 Upstairs last five completed: 1905 1904 1903 1902 1901
3147 CLIENT: Up:0 ds:0 act:3
3148 Replay: All jobs finished
3149 Main task has work to do, go find it
3150 got dsc Stop(0)
3151 stop 0
3152 Dec 04 13:31:22.882 INFO request completed[8810] Got stop action so:false kr:false
3153 , latency_us: 303, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 29079513-a292-4974-8bf7-cd285e200da8, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
31542024-12-04T13:31:22.883ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31552024-12-04T13:31:22.883ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31562024-12-04T13:31:22.883ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31572024-12-04T13:31:22.883ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31582024-12-04T13:31:22.883ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
3159 Dec 04 13:31:22.883 INFO request completed, latency_us: 176, response_code: 200, uri: /state/cid/0, method: GET, req_id: 0c27d474-8fbc-4f31-aec8-5acb69196963, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
31602024-12-04T13:31:22.883ZINFOcrucible: client task is exiting = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
3161 [8810] Exited with: ExitStatus(unix_wait_status(9))
3162 [8810][0] reports Exit
3163 Dec 04 13:31:26.886 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6439ec12-ace3-4f5b-b786-65e6043bd43e, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
3164 001/300 Write block 4722 len 4096 data: 3
3165 002/300 Read block 3156 len 8192
3166 003/300 Write block 1550 len 16384 data: 2 2 2 2
3167 004/300 Read block 2899 len 24576
3168 005/300 Write block 2798 len 16384 data: 2 2 2 2
3169 006/300 Write block 4242 len 12288 data: 2 2 3
3170 007/300 Flush
3171 008/300 Read block 2435 len 28672
3172 009/300 Read block 2516 len 36864
3173 010/300 Flush
3174 011/300 Read block 3751 len 20480
3175 012/300 Read block 3717 len 20480
3176 013/300 Write block 2172 len 8192 data: 2 2
3177 014/300 Write block 3659 len 24576 data: 2 2 2 2 2 2
3178 015/300 Write block 4241 len 8192 data: 2 3
3179 016/300 Write block 2469 len 8192 data: 2 2
3180 017/300 Read block 858 len 12288
3181 018/300 Read block 3176 len 24576
3182 019/300 Read block 978 len 4096
3183 020/300 Write block 3462 len 28672 data: 2 2 2 2 2 2 2
3184 021/300 Write block 4447 len 16384 data: 2 2 2 2
3185 022/300 Write block 1254 len 24576 data: 3 3 3 3 2 2
3186 023/300 Read block 3082 len 16384
3187 024/300 Read block 3574 len 20480
3188 025/300 Flush
3189 026/300 Write block 719 len 24576 data: 2 2 2 2 2 2
3190 027/300 Flush
3191 028/300 Read block 2047 len 12288
3192 029/300 Read block 3072 len 4096
3193 030/300 Write block 4299 len 12288 data: 2 2 2
3194 031/300 Write block 2788 len 32768 data: 2 2 2 2 2 2 2 2
3195 032/300 Write block 2123 len 28672 data: 3 3 3 3 3 3 2
3196 033/300 Write block 3860 len 4096 data: 2
3197 034/300 Read block 1444 len 36864
3198 035/300 Read block 1055 len 4096
3199 036/300 Read block 4159 len 32768
3200 037/300 Read block 4787 len 28672
3201 038/300 Read block 2761 len 28672
3202 039/300 Read block 4736 len 8192
3203 040/300 Write block 4351 len 8192 data: 2 2
3204 041/300 Read block 3644 len 32768
3205 042/300 Read block 4784 len 20480
3206 043/300 Read block 4436 len 20480
3207 044/300 Flush
3208 045/300 Write block 2736 len 12288 data: 2 2 3
3209 046/300 Write block 1068 len 20480 data: 3 3 3 2 2
3210 047/300 Read block 519 len 24576
3211 048/300 Read block 4796 len 12288
3212 049/300 Write block 3771 len 20480 data: 2 2 2 2 2
3213 050/300 Read block 633 len 20480
3214 051/300 Read block 4374 len 32768
3215 052/300 Read block 1144 len 28672
3216 053/300 Read block 633 len 24576
3217 054/300 Flush
3218 055/300 Flush
3219 056/300 Write block 428 len 4096 data: 2
3220 057/300 Read block 25 len 4096
3221 058/300 Write block 920 len 8192 data: 4 3
3222 059/300 Read block 4764 len 32768
3223 060/300 Read block 3721 len 24576
3224 061/300 Read block 2124 len 28672
3225 062/300 Read block 998 len 36864
3226 063/300 Read block 2961 len 36864
3227 064/300 Write block 1433 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3228 065/300 Read block 3887 len 8192
3229 066/300 Write block 3171 len 28672 data: 3 3 2 2 2 2 2
3230 067/300 Read block 1728 len 24576
3231 068/300 Write block 2852 len 16384 data: 3 2 2 2
3232 069/300 Write block 3146 len 32768 data: 2 2 2 2 2 3 3 3
3233 070/300 Flush
3234 071/300 Read block 4554 len 20480
3235 072/300 Read block 1440 len 40960
3236 073/300 Write block 1717 len 16384 data: 2 2 2 2
3237 074/300 Write block 1859 len 20480 data: 3 3 3 3 4
3238 075/300 Write block 2601 len 12288 data: 2 2 2
3239 076/300 Write block 1004 len 24576 data: 3 3 3 3 3 2
3240 077/300 Read block 2877 len 8192
3241 078/300 Write block 3475 len 24576 data: 2 2 2 2 2 2
3242 079/300 Write block 3171 len 32768 data: 4 4 3 3 3 3 3 2
3243 080/300 Write block 3236 len 28672 data: 2 2 2 2 2 2 3
3244 081/300 Write block 3197 len 4096 data: 3
3245 082/300 Read block 3384 len 28672
3246 083/300 Read block 588 len 28672
3247 084/300 Write block 7 len 32768 data: 3 2 2 2 2 2 2 2
3248 085/300 Write block 392 len 32768 data: 2 2 2 2 2 2 2 2
3249 086/300 Read block 740 len 40960
3250 087/300 Read block 2924 len 32768
3251 088/300 Read block 742 len 4096
3252 089/300 Write block 216 len 24576 data: 3 2 2 2 2 2
3253 090/300 Flush
3254 091/300 Read block 2602 len 20480
3255 092/300 Read block 2470 len 4096
3256 093/300 Write block 4734 len 12288 data: 2 2 2
3257 094/300 Write block 1481 len 20480 data: 3 3 3 3 3
3258 095/300 Write block 3153 len 8192 data: 4 3
3259 096/300 Read block 4640 len 28672
3260 097/300 Read block 2616 len 4096
3261 098/300 Read block 1007 len 20480
3262 099/300 Write block 4040 len 8192 data: 2 2
3263 100/300 Flush
3264 101/300 Write block 4107 len 12288 data: 2 2 2
3265 102/300 Read block 1872 len 4096
3266 103/300 Flush
3267 104/300 Write block 3632 len 28672 data: 2 2 2 2 2 2 2
3268 105/300 Read block 337 len 16384
3269 106/300 Read block 4858 len 8192
3270 107/300 Flush
3271 108/300 Read block 4606 len 32768
3272 109/300 Write block 3960 len 28672 data: 3 3 3 3 2 2 2
3273 110/300 Read block 191 len 20480
3274 111/300 Read block 3201 len 16384
3275 112/300 Read block 122 len 12288
3276 113/300 Read block 1642 len 40960
3277 114/300 Read block 1638 len 16384
3278 115/300 Write block 1678 len 40960 data: 3 3 3 3 3 3 3 3 3 3
3279 116/300 Write block 3787 len 12288 data: 2 2 2
3280 117/300 Write block 3460 len 12288 data: 2 2 3
3281 118/300 Read block 4668 len 32768
3282 119/300 Read block 4435 len 20480
3283 120/300 Read block 2574 len 24576
3284 121/300 Read block 3461 len 32768
3285 122/300 Read block 3423 len 12288
3286 123/300 Read block 4520 len 36864
3287 124/300 Write block 1628 len 28672 data: 2 2 2 2 2 2 2
3288 125/300 Read block 3618 len 36864
3289 126/300 Write block 3809 len 36864 data: 2 2 2 2 2 2 2 3 3
3290 127/300 Write block 1576 len 32768 data: 2 2 2 2 2 2 2 2
3291 128/300 Read block 3527 len 40960
3292 129/300 Read block 1123 len 32768
3293 130/300 Read block 1740 len 20480
3294 131/300 Write block 1743 len 8192 data: 5 5
3295 132/300 Flush
3296 133/300 Read block 3189 len 4096
3297 134/300 Read block 3970 len 24576
3298 135/300 Read block 208 len 12288
3299 136/300 Write block 2708 len 24576 data: 2 2 2 2 2 2
3300 137/300 Read block 3090 len 12288
3301 138/300 Read block 4537 len 32768
3302 139/300 Read block 1306 len 36864
3303 140/300 Write block 882 len 28672 data: 2 2 2 2 2 2 2
3304 141/300 Flush
3305 142/300 Read block 1776 len 24576
3306 143/300 Read block 3612 len 8192
3307 144/300 Flush
3308 145/300 Write block 2548 len 20480 data: 3 3 2 2 2
3309 146/300 Read block 3625 len 20480
3310 147/300 Read block 3778 len 28672
3311 148/300 Read block 567 len 40960
3312 149/300 Read block 2975 len 8192
3313 150/300 Read block 650 len 8192
3314 151/300 Read block 1439 len 28672
3315 152/300 Write block 1691 len 32768 data: 2 4 4 4 4 4 3 3
3316 153/300 Write block 537 len 32768 data: 2 3 3 3 3 3 3 3
3317 154/300 Write block 3363 len 4096 data: 5
3318 155/300 Read block 2267 len 12288
3319 156/300 Read block 3147 len 4096
3320 157/300 Write block 4470 len 32768 data: 5 4 4 2 2 2 2 2
3321 158/300 Read block 2221 len 20480
3322 159/300 Flush
3323 160/300 Read block 2162 len 8192
3324 161/300 Read block 3838 len 32768
3325 162/300 Write block 4465 len 8192 data: 4 4
3326 163/300 Read block 481 len 12288
3327 164/300 Read block 713 len 4096
3328 165/300 Read block 3686 len 32768
3329 166/300 Write block 4167 len 16384 data: 3 2 2 2
3330 167/300 Flush
3331 168/300 Write block 2835 len 24576 data: 2 3 3 3 3 3
3332 169/300 Write block 1079 len 28672 data: 2 2 2 2 2 2 2
3333 170/300 Read block 2945 len 12288
3334 171/300 Read block 4755 len 28672
3335 172/300 Read block 2639 len 4096
3336 173/300 Read block 4244 len 40960
3337 174/300 Read block 1347 len 20480
3338 175/300 Write block 2131 len 28672 data: 2 2 2 4 4 4 3
3339 176/300 Write block 1400 len 4096 data: 3
3340 177/300 Read block 3775 len 32768
3341 178/300 Read block 2012 len 16384
3342 179/300 Read block 839 len 12288
3343 180/300 Flush
3344 181/300 Read block 3504 len 32768
3345 182/300 Write block 2651 len 4096 data: 2
3346 183/300 Write block 391 len 16384 data: 2 3 3 3
3347 184/300 Write block 4855 len 4096 data: 2
3348 185/300 Read block 3188 len 12288
3349 186/300 Write block 3660 len 32768 data: 3 3 3 3 3 2 2 2
3350 187/300 Write block 1730 len 4096 data: 2
3351 188/300 Write block 3529 len 4096 data: 2
3352 189/300 Read block 4695 len 20480
3353 190/300 Read block 3440 len 12288
3354 191/300 Write block 4461 len 24576 data: 3 4 4 4 5 5
3355 192/300 Write block 303 len 20480 data: 2 2 2 2 2
3356 193/300 Read block 4813 len 28672
3357 194/300 Write block 2610 len 4096 data: 2
3358 195/300 Write block 2503 len 24576 data: 2 2 2 2 2 2
3359 196/300 Write block 1849 len 16384 data: 3 3 3 4
3360 197/300 Flush
3361 198/300 Read block 4490 len 40960
3362 199/300 Write block 2219 len 4096 data: 2
3363 200/300 Read block 1717 len 40960
3364 201/300 Read block 2421 len 28672
3365 202/300 Flush
3366 203/300 Read block 541 len 8192
3367 204/300 Read block 980 len 4096
3368 205/300 Read block 3536 len 24576
3369 206/300 Read block 1614 len 12288
3370 207/300 Write block 2921 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3371 208/300 Write block 4564 len 28672 data: 2 2 2 2 2 2 2
3372 209/300 Read block 3907 len 32768
3373 210/300 Read block 10 len 12288
3374 211/300 Write block 1360 len 20480 data: 2 2 2 2 2
3375 212/300 Read block 3122 len 40960
3376 213/300 Write block 1691 len 12288 data: 3 5 5
3377 214/300 Write block 4354 len 16384 data: 2 2 2 2
3378 215/300 Write block 2589 len 4096 data: 3
3379 216/300 Read block 1977 len 20480
3380 217/300 Read block 335 len 36864
3381 218/300 Read block 533 len 40960
3382 219/300 Read block 4722 len 20480
3383 220/300 Read block 4022 len 12288
3384 221/300 Flush
3385 222/300 Read block 4213 len 4096
3386 223/300 Read block 4837 len 24576
3387 224/300 Read block 1345 len 4096
3388 225/300 Read block 383 len 20480
3389 226/300 Read block 3379 len 4096
3390 227/300 Read block 2853 len 32768
3391 228/300 Read block 3589 len 16384
3392 229/300 Write block 352 len 24576 data: 2 2 2 2 2 2
3393 230/300 Write block 2277 len 12288 data: 2 2 2
3394 231/300 Read block 1408 len 36864
3395 232/300 Read block 810 len 36864
3396 233/300 Read block 3080 len 36864
3397 234/300 Write block 2048 len 4096 data: 2
3398 235/300 Write block 4331 len 28672 data: 2 2 2 2 2 2 2
3399 236/300 Read block 2151 len 24576
3400 237/300 Write block 3501 len 12288 data: 2 2 2
3401 238/300 Read block 3317 len 36864
3402 239/300 Read block 4817 len 32768
3403 240/300 Read block 4712 len 28672
3404 241/300 Read block 957 len 16384
3405 242/300 Write block 2645 len 20480 data: 2 3 3 3 2
3406 243/300 Read block 698 len 4096
3407 244/300 Flush
3408 245/300 Write block 2768 len 32768 data: 2 2 2 3 4 4 4 4
3409 246/300 Read block 1733 len 24576
3410 247/300 Write block 856 len 20480 data: 3 2 2 2 2
3411 248/300 Write block 4 len 4096 data: 5
3412 249/300 Write block 629 len 12288 data: 2 2 2
3413 250/300 Read block 916 len 28672
3414 251/300 Read block 2649 len 8192
3415 252/300 Read block 128 len 12288
3416 253/300 Read block 914 len 36864
3417 254/300 Read block 4241 len 16384
3418 255/300 Read block 871 len 16384
3419 256/300 Read block 2699 len 12288
3420 257/300 Read block 2899 len 16384
3421 258/300 Flush
3422 259/300 Write block 2165 len 40960 data: 2 2 2 2 2 3 2 3 3 2
3423 260/300 Read block 3918 len 28672
3424 261/300 Write block 2499 len 12288 data: 2 2 2
3425 262/300 Read block 4527 len 28672
3426 263/300 Read block 2199 len 20480
3427 264/300 Write block 1631 len 28672 data: 3 3 3 3 2 2 2
3428 265/300 Write block 4043 len 24576 data: 2 2 2 2 2 2
3429 266/300 Write block 1020 len 4096 data: 3
3430 267/300 Read block 4944 len 28672
3431 268/300 Write block 4108 len 8192 data: 3 3
3432 269/300 Read block 3747 len 32768
3433 270/300 Write block 3687 len 20480 data: 2 3 3 2 3
3434 271/300 Read block 3429 len 16384
3435 272/300 Read block 2587 len 8192
3436 273/300 Read block 4264 len 36864
3437 274/300 Read block 902 len 24576
3438 275/300 Read block 1 len 20480
3439 276/300 Read block 1745 len 4096
3440 277/300 Read block 1431 len 32768
3441 278/300 Read block 1289 len 20480
3442 279/300 Read block 2394 len 12288
3443 280/300 Read block 422 len 32768
3444 281/300 Read block 3039 len 24576
3445 282/300 Read block 248 len 4096
3446 283/300 Read block 3313 len 12288
3447 284/300 Write block 2640 len 36864 data: 3 2 2 2 2 3 4 4 4
3448 285/300 Flush
3449 286/300 Flush
3450 287/300 Read block 3820 len 40960
3451 288/300 Write block 1708 len 28672 data: 2 2 2 2 2 2 2
3452 289/300 Read block 2435 len 36864
3453 290/300 Read block 4075 len 36864
3454 291/300 Write block 26 len 16384 data: 2 2 2 2
3455 292/300 Write block 2221 len 12288 data: 2 2 2
3456 293/300 Read block 3177 len 28672
3457 294/300 Write block 4296 len 32768 data: 2 2 2 3 3 3 2 2
3458 295/300 Write block 1262 len 32768 data: 2 2 2 2 2 2 2 2
3459 296/300 Read block 1351 len 12288
3460 297/300 Read block 975 len 4096
3461 298/300 Read block 1307 len 24576
3462 299/300 Write block 2943 len 16384 data: 3 3 2 2
3463 300/300 Read block 3125 len 28672
3464 301/300 Flush
3465 Main task has work to do, go find it
3466 got dsc Start(0)
3467 start 0
3468 Dec 04 13:31:27.124 INFO request completed, latency_us: 203, response_code: 204, uri: /start/cid/0, method: POST, req_id: b0fdba31-6cb9-4fe2-892d-b94b5becbea3, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
3469 Replay: started 0, returned:Ok(())
3470 ----------------------------------------------------------------
3471 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3472 States: Offline Active Active
3473 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3474 1906 Acked Write 1 Sent Done Done false
3475 1907 Acked Read 2 Sent Done Done false
3476 1908 Acked Write 4 Sent Done Done false
3477 1909 Acked Read 6 Sent Done Done false
3478 1910 Acked Write 4 Sent Done Done false
3479 1911 Acked Write 3 Sent Done Done false
3480 1912 Acked Flush 0 Sent Done Done false
3481 1913 Acked Read 7 Sent Done Done false
3482 1914 Acked Read 9 Sent Done Done false
3483 1915 Acked Flush 0 Sent Done Done false
3484 1916 Acked Read 5 Sent Done Done false
3485 1917 Acked Read 5 Sent Done Done false
3486 1918 Acked Write 2 Sent Done Done false
3487 1919 Acked Write 6 Sent Done Done false
3488 1920 Acked Write 2 Sent Done Done false
3489 1921 Acked Write 2 Sent Done Done false
3490 1922 Acked Read 3 Sent Done Done false
3491 1923 Acked Read 6 Sent Done Done false
3492 1924 Acked Read 1 Sent Done Done false
3493 1925 Acked Write 7 Sent Done Done false
3494 1926 Acked Write 4 Sent Done Done false
3495 1927 Acked Write 6 Sent Done Done false
3496 1928 Acked Read 4 Sent Done Done false
3497 1929 Acked Read 5 Sent Done Done false
3498 1930 Acked Flush 0 Sent Done Done false
3499 1931 Acked Write 6 Sent Done Done false
3500 1932 Acked Flush 0 Sent Done Done false
3501 1933 Acked Read 3 Sent Done Done false
3502 1934 Acked Read 1 Sent Done Done false
3503 1935 Acked Write 3 Sent Done Done false
3504 1936 Acked Write 8 Sent Done Done false
3505 1937 Acked Write 7 Sent Done Done false
3506 1938 Acked Write 1 Sent Done Done false
3507 1939 Acked Read 9 Sent Done Done false
3508 1940 Acked Read 1 Sent Done Done false
3509 1941 Acked Read 8 Sent Done Done false
3510 1942 Acked Read 7 Sent Done Done false
3511 1943 Acked Read 7 Sent Done Done false
3512 1944 Acked Read 2 Sent Done Done false
3513 1945 Acked Write 2 Sent Done Done false
3514 1946 Acked Read 8 Sent Done Done false
3515 1947 Acked Read 5 Sent Done Done false
3516 1948 Acked Read 5 Sent Done Done false
3517 1949 Acked Flush 0 Sent Done Done false
3518 1950 Acked Write 3 Sent Done Done false
3519 1951 Acked Write 5 Sent Done Done false
3520 1952 Acked Read 6 Sent Done Done false
3521 1953 Acked Read 3 Sent Done Done false
3522 1954 Acked Write 5 Sent Done Done false
3523 1955 Acked Read 5 Sent Done Done false
3524 1956 Acked Read 8 Sent Done Done false
3525 1957 Acked Read 7 Sent Done Done false
3526 1958 Acked Read 6 Sent Done Done false
3527 1959 Acked Flush 0 Sent Done Done false
3528 1960 Acked Flush 0 Sent Done Done false
3529 1961 Acked Write 1 Sent Done Done false
3530 1962 Acked Read 1 Sent Done Done false
3531 1963 Acked Write 2 Sent Done Done false
3532 1964 Acked Read 8 Sent Done Done false
3533 1965 Acked Read 6 Sent Done Done false
3534 1966 Acked Read 7 Sent Done Done false
3535 1967 Acked Read 9 Sent Done Done false
3536 1968 Acked Read 9 Sent Done Done false
3537 1969 Acked Write 10 Sent Done Done false
3538 1970 Acked Read 2 Sent Done Done false
3539 1971 Acked Write 7 Sent Done Done false
3540 1972 Acked Read 6 Sent Done Done false
3541 1973 Acked Write 4 Sent Done Done false
3542 1974 Acked Write 8 Sent Done Done false
3543 1975 Acked Flush 0 Sent Done Done false
3544 1976 Acked Read 5 Sent Done Done false
3545 1977 Acked Read 10 Sent Done Done false
3546 1978 Acked Write 4 Sent Done Done false
3547 1979 Acked Write 5 Sent Done Done false
3548 1980 Acked Write 3 Sent Done Done false
3549 1981 Acked Write 6 Sent Done Done false
3550 1982 Acked Read 2 Sent Done Done false
3551 1983 Acked Write 6 Sent Done Done false
3552 1984 Acked Write 8 Sent Done Done false
3553 1985 Acked Write 7 Sent Done Done false
3554 1986 Acked Write 1 Sent Done Done false
3555 1987 Acked Read 7 Sent Done Done false
3556 1988 Acked Read 7 Sent Done Done false
3557 1989 Acked Write 8 Sent Done Done false
3558 1990 Acked Write 8 Sent Done Done false
3559 1991 Acked Read 10 Sent Done Done false
3560 1992 Acked Read 8 Sent Done Done false
3561 1993 Acked Read 1 Sent Done Done false
3562 1994 Acked Write 6 Sent Done Done false
3563 1995 Acked Flush 0 Sent Done Done false
3564 1996 Acked Read 5 Sent Done Done false
3565 1997 Acked Read 1 Sent Done Done false
3566 1998 Acked Write 3 Sent Done Done false
3567 1999 Acked Write 5 Sent Done Done false
3568 2000 Acked Write 2 Sent Done Done false
3569 2001 Acked Read 7 Sent Done Done false
3570 2002 Acked Read 1 Sent Done Done false
3571 2003 Acked Read 5 Sent Done Done false
3572 2004 Acked Write 2 Sent Done Done false
3573 2005 Acked Flush 0 Sent Done Done false
3574 2006 Acked Write 3 Sent Done Done false
3575 2007 Acked Read 1 Sent Done Done false
3576 2008 Acked Flush 0 Sent Done Done false
3577 2009 Acked Write 7 Sent Done Done false
3578 2010 Acked Read 4 Sent Done Done false
3579 2011 Acked Read 2 Sent Done Done false
3580 2012 Acked Flush 0 Sent Done Done false
3581 2013 Acked Read 8 Sent Done Done false
3582 2014 Acked Write 7 Sent Done Done false
3583 2015 Acked Read 5 Sent Done Done false
3584 2016 Acked Read 4 Sent Done Done false
3585 2017 Acked Read 3 Sent Done Done false
3586 2018 Acked Read 10 Sent Done Done false
3587 2019 Acked Read 4 Sent Done Done false
3588 2020 Acked Write 10 Sent Done Done false
3589 2021 Acked Write 3 Sent Done Done false
3590 2022 Acked Write 3 Sent Done Done false
3591 2023 Acked Read 8 Sent Done Done false
3592 2024 Acked Read 5 Sent Done Done false
3593 2025 Acked Read 6 Sent Done Done false
3594 2026 Acked Read 8 Sent Done Done false
3595 2027 Acked Read 3 Sent Done Done false
3596 2028 Acked Read 9 Sent Done Done false
3597 2029 Acked Write 7 Sent Done Done false
3598 2030 Acked Read 9 Sent Done Done false
3599 2031 Acked Write 9 Sent Done Done false
3600 2032 Acked Write 8 Sent Done Done false
3601 2033 Acked Read 10 Sent Done Done false
3602 2034 Acked Read 8 Sent Done Done false
3603 2035 Acked Read 5 Sent Done Done false
3604 2036 Acked Write 2 Sent Done Done false
3605 2037 Acked Flush 0 Sent Done Done false
3606 2038 Acked Read 1 Sent Done Done false
3607 2039 Acked Read 6 Sent Done Done false
3608 2040 Acked Read 3 Sent Done Done false
3609 2041 Acked Write 6 Sent Done Done false
3610 2042 Acked Read 3 Sent Done Done false
3611 2043 Acked Read 8 Sent Done Done false
3612 2044 Acked Read 9 Sent Done Done false
3613 2045 Acked Write 7 Sent Done Done false
3614 2046 Acked Flush 0 Sent Done Done false
3615 2047 Acked Read 6 Sent Done Done false
3616 2048 Acked Read 2 Sent Done Done false
3617 2049 Acked Flush 0 Sent Done Done false
3618 2050 Acked Write 5 Sent Done Done false
3619 2051 Acked Read 5 Sent Done Done false
3620 2052 Acked Read 7 Sent Done Done false
3621 2053 Acked Read 10 Sent Done Done false
3622 2054 Acked Read 2 Sent Done Done false
3623 2055 Acked Read 2 Sent Done Done false
3624 2056 Acked Read 7 Sent Done Done false
3625 2057 Acked Write 8 Sent Done Done false
3626 2058 Acked Write 8 Sent Done Done false
3627 2059 Acked Write 1 Sent Done Done false
3628 2060 Acked Read 3 Sent Done Done false
3629 2061 Acked Read 1 Sent Done Done false
3630 2062 Acked Write 8 Sent Done Done false
3631 2063 Acked Read 5 Sent Done Done false
3632 2064 Acked Flush 0 Sent Done Done false
3633 2065 Acked Read 2 Sent Done Done false
3634 2066 Acked Read 8 Sent Done Done false
3635 2067 Acked Write 2 Sent Done Done false
3636 2068 Acked Read 3 Sent Done Done false
3637 2069 Acked Read 1 Sent Done Done false
3638 2070 Acked Read 8 Sent Done Done false
3639 2071 Acked Write 4 Sent Done Done false
3640 2072 Acked Flush 0 Sent Done Done false
3641 2073 Acked Write 6 Sent Done Done false
3642 2074 Acked Write 7 Sent Done Done false
3643 2075 Acked Read 3 Sent Done Done false
3644 2076 Acked Read 7 Sent Done Done false
3645 2077 Acked Read 1 Sent Done Done false
3646 2078 Acked Read 10 Sent Done Done false
3647 2079 Acked Read 5 Sent Done Done false
3648 2080 Acked Write 7 Sent Done Done false
3649 2081 Acked Write 1 Sent Done Done false
3650 2082 Acked Read 8 Sent Done Done false
3651 2083 Acked Read 4 Sent Done Done false
3652 2084 Acked Read 3 Sent Done Done false
3653 2085 Acked Flush 0 Sent Done Done false
3654 2086 Acked Read 8 Sent Done Done false
3655 2087 Acked Write 1 Sent Done Done false
3656 2088 Acked Write 4 Sent Done Done false
3657 2089 Acked Write 1 Sent Done Done false
3658 2090 Acked Read 3 Sent Done Done false
3659 2091 Acked Write 8 Sent Done Done false
3660 2092 Acked Write 1 Sent Done Done false
3661 2093 Acked Write 1 Sent Done Done false
3662 2094 Acked Read 5 Sent Done Done false
3663 2095 Acked Read 3 Sent Done Done false
3664 2096 Acked Write 6 Sent Done Done false
3665 2097 Acked Write 5 Sent Done Done false
3666 2098 Acked Read 7 Sent Done Done false
3667 2099 Acked Write 1 Sent Done Done false
3668 2100 Acked Write 6 Sent Done Done false
3669 2101 Acked Write 4 Sent Done Done false
3670 2102 Acked Flush 0 Sent Done Done false
3671 2103 Acked Read 10 Sent Done Done false
3672 2104 Acked Write 1 Sent Done Done false
3673 2105 Acked Read 10 Sent Done Done false
3674 2106 Acked Read 7 Sent Done Done false
3675 2107 Acked Flush 0 Sent Done Done false
3676 2108 Acked Read 2 Sent Done Done false
3677 2109 Acked Read 1 Sent Done Done false
3678 2110 Acked Read 6 Sent Done Done false
3679 2111 Acked Read 3 Sent Done Done false
3680 2112 Acked Write 10 Sent Done Done false
3681 2113 Acked Write 7 Sent Done Done false
3682 2114 Acked Read 8 Sent Done Done false
3683 2115 Acked Read 3 Sent Done Done false
3684 2116 Acked Write 5 Sent Done Done false
3685 2117 Acked Read 10 Sent Done Done false
3686 2118 Acked Write 3 Sent Done Done false
3687 2119 Acked Write 4 Sent Done Done false
3688 2120 Acked Write 1 Sent Done Done false
3689 2121 Acked Read 5 Sent Done Done false
3690 2122 Acked Read 9 Sent Done Done false
3691 2123 Acked Read 10 Sent Done Done false
3692 2124 Acked Read 5 Sent Done Done false
3693 2125 Acked Read 3 Sent Done Done false
3694 2126 Acked Flush 0 Sent Done Done false
3695 2127 Acked Read 1 Sent Done Done false
3696 2128 Acked Read 6 Sent Done Done false
3697 2129 Acked Read 1 Sent Done Done false
3698 2130 Acked Read 5 Sent Done Done false
3699 2131 Acked Read 1 Sent Done Done false
3700 2132 Acked Read 8 Sent Done Done false
3701 2133 Acked Read 4 Sent Done Done false
3702 2134 Acked Write 6 Sent Done Done false
3703 2135 Acked Write 3 Sent Done Done false
3704 2136 Acked Read 9 Sent Done Done false
3705 2137 Acked Read 9 Sent Done Done false
3706 2138 Acked Read 9 Sent Done Done false
3707 2139 Acked Write 1 Sent Done Done false
3708 2140 Acked Write 7 Sent Done Done false
3709 2141 Acked Read 6 Sent Done Done false
3710 2142 Acked Write 3 Sent Done Done false
3711 2143 Acked Read 9 Sent Done Done false
3712 2144 Acked Read 8 Sent Done Done false
3713 2145 Acked Read 7 Sent Done Done false
3714 2146 Acked Read 4 Sent Done Done false
3715 2147 Acked Write 5 Sent Done Done false
3716 2148 Acked Read 1 Sent Done Done false
3717 2149 Acked Flush 0 Sent Done Done false
3718 2150 Acked Write 8 Sent Done Done false
3719 2151 Acked Read 6 Sent Done Done false
3720 2152 Acked Write 5 Sent Done Done false
3721 2153 Acked Write 1 Sent Done Done false
3722 2154 Acked Write 3 Sent Done Done false
3723 2155 Acked Read 7 Sent Done Done false
3724 2156 Acked Read 2 Sent Done Done false
3725 2157 Acked Read 3 Sent Done Done false
3726 2158 Acked Read 9 Sent Done Done false
3727 2159 Acked Read 4 Sent Done Done false
3728 2160 Acked Read 4 Sent Done Done false
3729 2161 Acked Read 3 Sent Done Done false
3730 2162 Acked Read 4 Sent Done Done false
3731 2163 Acked Flush 0 Sent Done Done false
3732 2164 Acked Write 10 Sent Done Done false
3733 2165 Acked Read 7 Sent Done Done false
3734 2166 Acked Write 3 Sent Done Done false
3735 2167 Acked Read 7 Sent Done Done false
3736 2168 Acked Read 5 Sent Done Done false
3737 2169 Acked Write 7 Sent Done Done false
3738 2170 Acked Write 6 Sent Done Done false
3739 2171 Acked Write 1 Sent Done Done false
3740 2172 Acked Read 7 Sent Done Done false
3741 2173 Acked Write 2 Sent Done Done false
3742 2174 Acked Read 8 Sent Done Done false
3743 2175 Acked Write 5 Sent Done Done false
3744 2176 Acked Read 4 Sent Done Done false
3745 2177 Acked Read 2 Sent Done Done false
3746 2178 Acked Read 9 Sent Done Done false
3747 2179 Acked Read 6 Sent Done Done false
3748 2180 Acked Read 5 Sent Done Done false
3749 2181 Acked Read 1 Sent Done Done false
3750 2182 Acked Read 8 Sent Done Done false
3751 2183 Acked Read 5 Sent Done Done false
3752 2184 Acked Read 3 Sent Done Done false
3753 2185 Acked Read 8 Sent Done Done false
3754 2186 Acked Read 6 Sent Done Done false
3755 2187 Acked Read 1 Sent Done Done false
3756 2188 Acked Read 3 Sent Done Done false
3757 2189 Acked Write 9 Sent Done Done false
3758 2190 Acked Flush 0 Sent Done Done false
3759 2191 Acked Flush 0 Sent Done Done false
3760 2192 Acked Read 10 Sent Done Done false
3761 2193 Acked Write 7 Sent Done Done false
3762 2194 Acked Read 9 Sent Done Done false
3763 2195 Acked Read 9 Sent Done Done false
3764 2196 Acked Write 4 Sent Done Done false
3765 2197 Acked Write 3 Sent Done Done false
3766 2198 Acked Read 7 Sent Done Done false
3767 2199 Acked Write 8 Sent Done Done false
3768 2200 Acked Write 8 Sent Done Done false
3769 2201 Acked Read 3 Sent Done Done false
3770 2202 Acked Read 1 Sent Done Done false
3771 2203 Acked Read 6 Sent Done Done false
3772 2204 Acked Write 4 Sent Done Done false
3773 2205 Acked Read 7 Sent Done Done false
3774 2206 Acked Flush 0 Sent Done Done false
3775 STATES DS:0 DS:1 DS:2 TOTAL
3776 Sent 301 0 0 301
3777 Done 0 301 301 602
3778 Skipped 0 0 0 0
3779 Error 0 0 0 0
3780 Last Flush: 1905 2206 2206
3781 Downstairs last five completed: 1905 1904 1903 1902 1901
3782 Upstairs last five completed: 2206 2205 2204 2203 2202
3783 CLIENT: Up:0 ds:301 act:2
3784 [8810] got start action
3785 [8810] I am going to restart
3786 Starting downstairs at port 8810
3787 Make output file at "/tmp/dsc/downstairs-8810.txt"
3788 [8810][0] reports Starting
3789 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
3790 [8810][0] reports Running
3791 ----------------------------------------------------------------
3792 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3793 States: Offline Active Active
3794 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3795 1906 Acked Write 1 Sent Done Done false
3796 1907 Acked Read 2 Sent Done Done false
3797 1908 Acked Write 4 Sent Done Done false
3798 1909 Acked Read 6 Sent Done Done false
3799 1910 Acked Write 4 Sent Done Done false
3800 1911 Acked Write 3 Sent Done Done false
3801 1912 Acked Flush 0 Sent Done Done false
3802 1913 Acked Read 7 Sent Done Done false
3803 1914 Acked Read 9 Sent Done Done false
3804 1915 Acked Flush 0 Sent Done Done false
3805 1916 Acked Read 5 Sent Done Done false
3806 1917 Acked Read 5 Sent Done Done false
3807 1918 Acked Write 2 Sent Done Done false
3808 1919 Acked Write 6 Sent Done Done false
3809 1920 Acked Write 2 Sent Done Done false
3810 1921 Acked Write 2 Sent Done Done false
3811 1922 Acked Read 3 Sent Done Done false
3812 1923 Acked Read 6 Sent Done Done false
3813 1924 Acked Read 1 Sent Done Done false
3814 1925 Acked Write 7 Sent Done Done false
3815 1926 Acked Write 4 Sent Done Done false
3816 1927 Acked Write 6 Sent Done Done false
3817 1928 Acked Read 4 Sent Done Done false
3818 1929 Acked Read 5 Sent Done Done false
3819 1930 Acked Flush 0 Sent Done Done false
3820 1931 Acked Write 6 Sent Done Done false
3821 1932 Acked Flush 0 Sent Done Done false
3822 1933 Acked Read 3 Sent Done Done false
3823 1934 Acked Read 1 Sent Done Done false
3824 1935 Acked Write 3 Sent Done Done false
3825 1936 Acked Write 8 Sent Done Done false
3826 1937 Acked Write 7 Sent Done Done false
3827 1938 Acked Write 1 Sent Done Done false
3828 1939 Acked Read 9 Sent Done Done false
3829 1940 Acked Read 1 Sent Done Done false
3830 1941 Acked Read 8 Sent Done Done false
3831 1942 Acked Read 7 Sent Done Done false
3832 1943 Acked Read 7 Sent Done Done false
3833 1944 Acked Read 2 Sent Done Done false
3834 1945 Acked Write 2 Sent Done Done false
3835 1946 Acked Read 8 Sent Done Done false
3836 1947 Acked Read 5 Sent Done Done false
3837 1948 Acked Read 5 Sent Done Done false
3838 1949 Acked Flush 0 Sent Done Done false
3839 1950 Acked Write 3 Sent Done Done false
3840 1951 Acked Write 5 Sent Done Done false
3841 1952 Acked Read 6 Sent Done Done false
3842 1953 Acked Read 3 Sent Done Done false
3843 1954 Acked Write 5 Sent Done Done false
3844 1955 Acked Read 5 Sent Done Done false
3845 1956 Acked Read 8 Sent Done Done false
3846 1957 Acked Read 7 Sent Done Done false
3847 1958 Acked Read 6 Sent Done Done false
3848 1959 Acked Flush 0 Sent Done Done false
3849 1960 Acked Flush 0 Sent Done Done false
3850 1961 Acked Write 1 Sent Done Done false
3851 1962 Acked Read 1 Sent Done Done false
3852 1963 Acked Write 2 Sent Done Done false
3853 1964 Acked Read 8 Sent Done Done false
3854 1965 Acked Read 6 Sent Done Done false
3855 1966 Acked Read 7 Sent Done Done false
3856 1967 Acked Read 9 Sent Done Done false
3857 1968 Acked Read 9 Sent Done Done false
3858 1969 Acked Write 10 Sent Done Done false
3859 1970 Acked Read 2 Sent Done Done false
3860 1971 Acked Write 7 Sent Done Done false
3861 1972 Acked Read 6 Sent Done Done false
3862 1973 Acked Write 4 Sent Done Done false
3863 1974 Acked Write 8 Sent Done Done false
3864 1975 Acked Flush 0 Sent Done Done false
3865 1976 Acked Read 5 Sent Done Done false
3866 1977 Acked Read 10 Sent Done Done false
3867 1978 Acked Write 4 Sent Done Done false
3868 1979 Acked Write 5 Sent Done Done false
3869 1980 Acked Write 3 Sent Done Done false
3870 1981 Acked Write 6 Sent Done Done false
3871 1982 Acked Read 2 Sent Done Done false
3872 1983 Acked Write 6 Sent Done Done false
3873 1984 Acked Write 8 Sent Done Done false
3874 1985 Acked Write 7 Sent Done Done false
3875 1986 Acked Write 1 Sent Done Done false
3876 1987 Acked Read 7 Sent Done Done false
3877 1988 Acked Read 7 Sent Done Done false
3878 1989 Acked Write 8 Sent Done Done false
3879 1990 Acked Write 8 Sent Done Done false
3880 1991 Acked Read 10 Sent Done Done false
3881 1992 Acked Read 8 Sent Done Done false
3882 1993 Acked Read 1 Sent Done Done false
3883 1994 Acked Write 6 Sent Done Done false
3884 1995 Acked Flush 0 Sent Done Done false
3885 1996 Acked Read 5 Sent Done Done false
3886 1997 Acked Read 1 Sent Done Done false
3887 1998 Acked Write 3 Sent Done Done false
3888 1999 Acked Write 5 Sent Done Done false
3889 2000 Acked Write 2 Sent Done Done false
3890 2001 Acked Read 7 Sent Done Done false
3891 2002 Acked Read 1 Sent Done Done false
3892 2003 Acked Read 5 Sent Done Done false
3893 2004 Acked Write 2 Sent Done Done false
3894 2005 Acked Flush 0 Sent Done Done false
3895 2006 Acked Write 3 Sent Done Done false
3896 2007 Acked Read 1 Sent Done Done false
3897 2008 Acked Flush 0 Sent Done Done false
3898 2009 Acked Write 7 Sent Done Done false
3899 2010 Acked Read 4 Sent Done Done false
3900 2011 Acked Read 2 Sent Done Done false
3901 2012 Acked Flush 0 Sent Done Done false
3902 2013 Acked Read 8 Sent Done Done false
3903 2014 Acked Write 7 Sent Done Done false
3904 2015 Acked Read 5 Sent Done Done false
3905 2016 Acked Read 4 Sent Done Done false
3906 2017 Acked Read 3 Sent Done Done false
3907 2018 Acked Read 10 Sent Done Done false
3908 2019 Acked Read 4 Sent Done Done false
3909 2020 Acked Write 10 Sent Done Done false
3910 2021 Acked Write 3 Sent Done Done false
3911 2022 Acked Write 3 Sent Done Done false
3912 2023 Acked Read 8 Sent Done Done false
3913 2024 Acked Read 5 Sent Done Done false
3914 2025 Acked Read 6 Sent Done Done false
3915 2026 Acked Read 8 Sent Done Done false
3916 2027 Acked Read 3 Sent Done Done false
3917 2028 Acked Read 9 Sent Done Done false
3918 2029 Acked Write 7 Sent Done Done false
3919 2030 Acked Read 9 Sent Done Done false
3920 2031 Acked Write 9 Sent Done Done false
3921 2032 Acked Write 8 Sent Done Done false
3922 2033 Acked Read 10 Sent Done Done false
3923 2034 Acked Read 8 Sent Done Done false
3924 2035 Acked Read 5 Sent Done Done false
3925 2036 Acked Write 2 Sent Done Done false
3926 2037 Acked Flush 0 Sent Done Done false
3927 2038 Acked Read 1 Sent Done Done false
3928 2039 Acked Read 6 Sent Done Done false
3929 2040 Acked Read 3 Sent Done Done false
3930 2041 Acked Write 6 Sent Done Done false
3931 2042 Acked Read 3 Sent Done Done false
3932 2043 Acked Read 8 Sent Done Done false
3933 2044 Acked Read 9 Sent Done Done false
3934 2045 Acked Write 7 Sent Done Done false
3935 2046 Acked Flush 0 Sent Done Done false
3936 2047 Acked Read 6 Sent Done Done false
3937 2048 Acked Read 2 Sent Done Done false
3938 2049 Acked Flush 0 Sent Done Done false
3939 2050 Acked Write 5 Sent Done Done false
3940 2051 Acked Read 5 Sent Done Done false
3941 2052 Acked Read 7 Sent Done Done false
3942 2053 Acked Read 10 Sent Done Done false
3943 2054 Acked Read 2 Sent Done Done false
3944 2055 Acked Read 2 Sent Done Done false
3945 2056 Acked Read 7 Sent Done Done false
3946 2057 Acked Write 8 Sent Done Done false
3947 2058 Acked Write 8 Sent Done Done false
3948 2059 Acked Write 1 Sent Done Done false
3949 2060 Acked Read 3 Sent Done Done false
3950 2061 Acked Read 1 Sent Done Done false
3951 2062 Acked Write 8 Sent Done Done false
3952 2063 Acked Read 5 Sent Done Done false
3953 2064 Acked Flush 0 Sent Done Done false
3954 2065 Acked Read 2 Sent Done Done false
3955 2066 Acked Read 8 Sent Done Done false
3956 2067 Acked Write 2 Sent Done Done false
3957 2068 Acked Read 3 Sent Done Done false
3958 2069 Acked Read 1 Sent Done Done false
3959 2070 Acked Read 8 Sent Done Done false
3960 2071 Acked Write 4 Sent Done Done false
3961 2072 Acked Flush 0 Sent Done Done false
3962 2073 Acked Write 6 Sent Done Done false
3963 2074 Acked Write 7 Sent Done Done false
3964 2075 Acked Read 3 Sent Done Done false
3965 2076 Acked Read 7 Sent Done Done false
3966 2077 Acked Read 1 Sent Done Done false
3967 2078 Acked Read 10 Sent Done Done false
3968 2079 Acked Read 5 Sent Done Done false
3969 2080 Acked Write 7 Sent Done Done false
3970 2081 Acked Write 1 Sent Done Done false
3971 2082 Acked Read 8 Sent Done Done false
3972 2083 Acked Read 4 Sent Done Done false
3973 2084 Acked Read 3 Sent Done Done false
3974 2085 Acked Flush 0 Sent Done Done false
3975 2086 Acked Read 8 Sent Done Done false
3976 2087 Acked Write 1 Sent Done Done false
3977 2088 Acked Write 4 Sent Done Done false
3978 2089 Acked Write 1 Sent Done Done false
3979 2090 Acked Read 3 Sent Done Done false
3980 2091 Acked Write 8 Sent Done Done false
3981 2092 Acked Write 1 Sent Done Done false
3982 2093 Acked Write 1 Sent Done Done false
3983 2094 Acked Read 5 Sent Done Done false
3984 2095 Acked Read 3 Sent Done Done false
3985 2096 Acked Write 6 Sent Done Done false
3986 2097 Acked Write 5 Sent Done Done false
3987 2098 Acked Read 7 Sent Done Done false
3988 2099 Acked Write 1 Sent Done Done false
3989 2100 Acked Write 6 Sent Done Done false
3990 2101 Acked Write 4 Sent Done Done false
3991 2102 Acked Flush 0 Sent Done Done false
3992 2103 Acked Read 10 Sent Done Done false
3993 2104 Acked Write 1 Sent Done Done false
3994 2105 Acked Read 10 Sent Done Done false
3995 2106 Acked Read 7 Sent Done Done false
3996 2107 Acked Flush 0 Sent Done Done false
3997 2108 Acked Read 2 Sent Done Done false
3998 2109 Acked Read 1 Sent Done Done false
3999 2110 Acked Read 6 Sent Done Done false
4000 2111 Acked Read 3 Sent Done Done false
4001 2112 Acked Write 10 Sent Done Done false
4002 2113 Acked Write 7 Sent Done Done false
4003 2114 Acked Read 8 Sent Done Done false
4004 2115 Acked Read 3 Sent Done Done false
4005 2116 Acked Write 5 Sent Done Done false
4006 2117 Acked Read 10 Sent Done Done false
4007 2118 Acked Write 3 Sent Done Done false
4008 2119 Acked Write 4 Sent Done Done false
4009 2120 Acked Write 1 Sent Done Done false
4010 2121 Acked Read 5 Sent Done Done false
4011 2122 Acked Read 9 Sent Done Done false
4012 2123 Acked Read 10 Sent Done Done false
4013 2124 Acked Read 5 Sent Done Done false
4014 2125 Acked Read 3 Sent Done Done false
4015 2126 Acked Flush 0 Sent Done Done false
4016 2127 Acked Read 1 Sent Done Done false
4017 2128 Acked Read 6 Sent Done Done false
4018 2129 Acked Read 1 Sent Done Done false
4019 2130 Acked Read 5 Sent Done Done false
4020 2131 Acked Read 1 Sent Done Done false
4021 2132 Acked Read 8 Sent Done Done false
4022 2133 Acked Read 4 Sent Done Done false
4023 2134 Acked Write 6 Sent Done Done false
4024 2135 Acked Write 3 Sent Done Done false
4025 2136 Acked Read 9 Sent Done Done false
4026 2137 Acked Read 9 Sent Done Done false
4027 2138 Acked Read 9 Sent Done Done false
4028 2139 Acked Write 1 Sent Done Done false
4029 2140 Acked Write 7 Sent Done Done false
4030 2141 Acked Read 6 Sent Done Done false
4031 2142 Acked Write 3 Sent Done Done false
4032 2143 Acked Read 9 Sent Done Done false
4033 2144 Acked Read 8 Sent Done Done false
4034 2145 Acked Read 7 Sent Done Done false
4035 2146 Acked Read 4 Sent Done Done false
4036 2147 Acked Write 5 Sent Done Done false
4037 2148 Acked Read 1 Sent Done Done false
4038 2149 Acked Flush 0 Sent Done Done false
4039 2150 Acked Write 8 Sent Done Done false
4040 2151 Acked Read 6 Sent Done Done false
4041 2152 Acked Write 5 Sent Done Done false
4042 2153 Acked Write 1 Sent Done Done false
4043 2154 Acked Write 3 Sent Done Done false
4044 2155 Acked Read 7 Sent Done Done false
4045 2156 Acked Read 2 Sent Done Done false
4046 2157 Acked Read 3 Sent Done Done false
4047 2158 Acked Read 9 Sent Done Done false
4048 2159 Acked Read 4 Sent Done Done false
4049 2160 Acked Read 4 Sent Done Done false
4050 2161 Acked Read 3 Sent Done Done false
4051 2162 Acked Read 4 Sent Done Done false
4052 2163 Acked Flush 0 Sent Done Done false
4053 2164 Acked Write 10 Sent Done Done false
4054 2165 Acked Read 7 Sent Done Done false
4055 2166 Acked Write 3 Sent Done Done false
4056 2167 Acked Read 7 Sent Done Done false
4057 2168 Acked Read 5 Sent Done Done false
4058 2169 Acked Write 7 Sent Done Done false
4059 2170 Acked Write 6 Sent Done Done false
4060 2171 Acked Write 1 Sent Done Done false
4061 2172 Acked Read 7 Sent Done Done false
4062 2173 Acked Write 2 Sent Done Done false
4063 2174 Acked Read 8 Sent Done Done false
4064 2175 Acked Write 5 Sent Done Done false
4065 2176 Acked Read 4 Sent Done Done false
4066 2177 Acked Read 2 Sent Done Done false
4067 2178 Acked Read 9 Sent Done Done false
4068 2179 Acked Read 6 Sent Done Done false
4069 2180 Acked Read 5 Sent Done Done false
4070 2181 Acked Read 1 Sent Done Done false
4071 2182 Acked Read 8 Sent Done Done false
4072 2183 Acked Read 5 Sent Done Done false
4073 2184 Acked Read 3 Sent Done Done false
4074 2185 Acked Read 8 Sent Done Done false
4075 2186 Acked Read 6 Sent Done Done false
4076 2187 Acked Read 1 Sent Done Done false
4077 2188 Acked Read 3 Sent Done Done false
4078 2189 Acked Write 9 Sent Done Done false
4079 2190 Acked Flush 0 Sent Done Done false
4080 2191 Acked Flush 0 Sent Done Done false
4081 2192 Acked Read 10 Sent Done Done false
4082 2193 Acked Write 7 Sent Done Done false
4083 2194 Acked Read 9 Sent Done Done false
4084 2195 Acked Read 9 Sent Done Done false
4085 2196 Acked Write 4 Sent Done Done false
4086 2197 Acked Write 3 Sent Done Done false
4087 2198 Acked Read 7 Sent Done Done false
4088 2199 Acked Write 8 Sent Done Done false
4089 2200 Acked Write 8 Sent Done Done false
4090 2201 Acked Read 3 Sent Done Done false
4091 2202 Acked Read 1 Sent Done Done false
4092 2203 Acked Read 6 Sent Done Done false
4093 2204 Acked Write 4 Sent Done Done false
4094 2205 Acked Read 7 Sent Done Done false
4095 2206 Acked Flush 0 Sent Done Done false
4096 STATES DS:0 DS:1 DS:2 TOTAL
4097 Sent 301 0 0 301
4098 Done 0 301 301 602
4099 Skipped 0 0 0 0
4100 Error 0 0 0 0
4101 Last Flush: 1905 2206 2206
4102 Downstairs last five completed: 1905 1904 1903 1902 1901
4103 Upstairs last five completed: 2206 2205 2204 2203 2202
4104 CLIENT: Up:0 ds:301 act:2
41052024-12-04T13:31:32.884ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41062024-12-04T13:31:32.884ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:65275) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41072024-12-04T13:31:32.885ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41082024-12-04T13:31:32.885ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41092024-12-04T13:31:32.885ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41102024-12-04T13:31:32.885ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41112024-12-04T13:31:32.886ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41122024-12-04T13:31:32.886ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41132024-12-04T13:31:32.886ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41142024-12-04T13:31:32.886ZINFOcrucible: [0] client re-new 301 jobs since flush 1905 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41152024-12-04T13:31:32.886ZINFOcrucible: [0] Replayed 301 jobs since flush: 1905 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
4116 ----------------------------------------------------------------
4117 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4118 Downstairs last five completed: 2206 2205 2204 2203 2202
4119 Upstairs last five completed: 2206 2205 2204 2203 2202
4120 CLIENT: Up:0 ds:0 act:3
4121 Replay: All jobs finished
4122 Main task has work to do, go find it
4123 got dsc Stop(2)
4124 stop 2
4125 Dec 04 13:31:35.129 INFO request completed[8830] Got stop action so:false kr:false
4126 , latency_us: 314, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 863502b9-bde5-4b37-89b3-3c57384358b1, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
4127 Dec 04 13:31:35.130 INFO request completed, latency_us: 128, response_code: 200, uri: /state/cid/2, method: GET, req_id: 194d8b3a-ad0e-4bb0-b61b-5dd02637813a, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
41282024-12-04T13:31:35.130ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41292024-12-04T13:31:35.130ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41302024-12-04T13:31:35.130ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41312024-12-04T13:31:35.130ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41322024-12-04T13:31:35.130ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
41332024-12-04T13:31:35.130ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
4134 [8830] Exited with: ExitStatus(unix_wait_status(9))
4135 [8830][2] reports Exit
4136 Dec 04 13:31:39.132 INFO request completed, latency_us: 233, response_code: 200, uri: /state/cid/2, method: GET, req_id: ec5b6330-2848-418b-ba70-9e151af6509c, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
4137 001/300 Read block 919 len 16384
4138 002/300 Write block 4413 len 32768 data: 3 3 2 2 2 2 2 2
4139 003/300 Write block 4542 len 12288 data: 2 2 2
4140 004/300 Read block 2795 len 36864
4141 005/300 Read block 4238 len 4096
4142 006/300 Write block 4828 len 20480 data: 3 3 3 3 3
4143 007/300 Write block 1873 len 32768 data: 3 3 2 2 2 2 2 2
4144 008/300 Write block 4596 len 36864 data: 2 3 3 3 3 3 3 3 3
4145 009/300 Read block 3377 len 16384
4146 010/300 Write block 4754 len 16384 data: 2 2 2 2
4147 011/300 Flush
4148 012/300 Write block 4119 len 40960 data: 2 2 3 3 3 3 2 2 2 3
4149 013/300 Read block 1795 len 16384
4150 014/300 Flush
4151 015/300 Write block 4167 len 8192 data: 4 3
4152 016/300 Read block 1851 len 12288
4153 017/300 Flush
4154 018/300 Read block 803 len 24576
4155 019/300 Read block 4172 len 12288
4156 020/300 Read block 4129 len 16384
4157 021/300 Flush
4158 022/300 Write block 4536 len 32768 data: 3 3 3 3 3 3 3 3
4159 023/300 Write block 1944 len 32768 data: 2 4 3 3 3 3 3 3
4160 024/300 Read block 2035 len 32768
4161 025/300 Read block 1742 len 12288
4162 026/300 Read block 1445 len 20480
4163 027/300 Flush
4164 028/300 Read block 4086 len 12288
4165 029/300 Write block 3005 len 4096 data: 4
4166 030/300 Read block 3803 len 40960
4167 031/300 Flush
4168 032/300 Read block 4376 len 4096
4169 033/300 Write block 601 len 20480 data: 3 3 3 2 2
4170 034/300 Flush
4171 035/300 Write block 2893 len 16384 data: 2 3 3 3
4172 036/300 Flush
4173 037/300 Write block 4519 len 40960 data: 2 2 2 3 3 3 3 3 3 2
4174 038/300 Write block 2425 len 16384 data: 3 3 2 2
4175 039/300 Write block 2528 len 8192 data: 2 2
4176 040/300 Write block 539 len 40960 data: 4 4 4 4 4 4 3 3 3 2
4177 041/300 Write block 161 len 40960 data: 3 3 2 3 3 4 4 4 4 4
4178 042/300 Write block 3269 len 24576 data: 2 2 2 2 2 2
4179 043/300 Read block 4559 len 8192
4180 044/300 Read block 4598 len 28672
4181 045/300 Read block 4893 len 16384
4182 046/300 Read block 4987 len 40960
4183 047/300 Read block 1287 len 32768
4184 048/300 Write block 1815 len 4096 data: 3
4185 049/300 Write block 1470 len 16384 data: 2 2 2 2
4186 050/300 Write block 2380 len 4096 data: 2
4187 051/300 Write block 1528 len 20480 data: 3 3 3 2 2
4188 052/300 Read block 3975 len 24576
4189 053/300 Read block 1426 len 40960
4190 054/300 Write block 1888 len 28672 data: 4 4 4 3 3 2 2
4191 055/300 Read block 4187 len 16384
4192 056/300 Read block 2139 len 40960
4193 057/300 Write block 959 len 24576 data: 2 2 2 3 3 3
4194 058/300 Read block 2394 len 28672
4195 059/300 Flush
4196 060/300 Read block 534 len 4096
4197 061/300 Read block 2846 len 28672
4198 062/300 Write block 3773 len 36864 data: 3 3 3 3 3 3 3 3 3
4199 063/300 Read block 142 len 40960
4200 064/300 Read block 4033 len 28672
4201 065/300 Flush
4202 066/300 Read block 1745 len 40960
4203 067/300 Write block 2343 len 40960 data: 2 2 3 3 3 3 2 2 2 3
4204 068/300 Write block 2477 len 36864 data: 2 2 2 2 2 2 2 2 2
4205 069/300 Read block 1600 len 16384
4206 070/300 Read block 4173 len 4096
4207 071/300 Read block 3292 len 12288
4208 072/300 Read block 3608 len 36864
4209 073/300 Read block 990 len 4096
4210 074/300 Write block 691 len 20480 data: 2 2 3 3 3
4211 075/300 Read block 2689 len 16384
4212 076/300 Read block 4702 len 36864
4213 077/300 Read block 2222 len 4096
4214 078/300 Write block 4518 len 28672 data: 2 3 3 3 4 4 4
4215 079/300 Read block 1453 len 4096
4216 080/300 Write block 2416 len 28672 data: 2 2 3 3 3 3 3
4217 081/300 Write block 2712 len 4096 data: 3
4218 082/300 Write block 2376 len 4096 data: 2
4219 083/300 Read block 4436 len 40960
4220 084/300 Read block 2106 len 8192
4221 085/300 Read block 4095 len 40960
4222 086/300 Write block 3546 len 32768 data: 2 2 2 2 2 2 2 2
4223 087/300 Read block 779 len 32768
4224 088/300 Read block 2739 len 16384
4225 089/300 Write block 294 len 24576 data: 3 3 3 3 3 2
4226 090/300 Read block 718 len 8192
4227 091/300 Read block 3570 len 4096
4228 092/300 Read block 596 len 36864
4229 093/300 Write block 2424 len 24576 data: 3 4 4 3 3 2
4230 094/300 Write block 3062 len 8192 data: 2 2
4231 095/300 Read block 4700 len 40960
4232 096/300 Write block 520 len 28672 data: 2 2 2 2 2 2 2
4233 097/300 Read block 1859 len 24576
4234 098/300 Read block 66 len 28672
4235 099/300 Read block 287 len 40960
4236 100/300 Flush
4237 101/300 Flush
4238 102/300 Read block 3677 len 12288
4239 103/300 Read block 1673 len 16384
4240 104/300 Flush
4241 105/300 Read block 514 len 24576
4242 106/300 Read block 300 len 4096
4243 107/300 Flush
4244 108/300 Read block 3185 len 4096
4245 109/300 Read block 2427 len 32768
4246 110/300 Flush
4247 111/300 Read block 1933 len 36864
4248 112/300 Write block 2826 len 24576 data: 4 4 3 3 3 3
4249 113/300 Read block 1757 len 24576
4250 114/300 Read block 919 len 40960
4251 115/300 Read block 3568 len 16384
4252 116/300 Read block 1674 len 28672
4253 117/300 Read block 2010 len 32768
4254 118/300 Flush
4255 119/300 Read block 3760 len 36864
4256 120/300 Flush
4257 121/300 Read block 1465 len 36864
4258 122/300 Read block 2138 len 12288
4259 123/300 Write block 2868 len 36864 data: 2 2 2 2 2 2 2 2 2
4260 124/300 Write block 4577 len 8192 data: 2 2
4261 125/300 Read block 2011 len 40960
4262 126/300 Write block 152 len 20480 data: 3 4 4 4 3
4263 127/300 Write block 4026 len 28672 data: 2 2 2 2 2 2 2
4264 128/300 Read block 2997 len 32768
4265 129/300 Write block 364 len 28672 data: 3 2 2 2 2 3 3
4266 130/300 Write block 279 len 8192 data: 2 2
4267 131/300 Read block 3391 len 28672
4268 132/300 Read block 3269 len 32768
4269 133/300 Read block 4350 len 16384
4270 134/300 Write block 4704 len 8192 data: 3 2
4271 135/300 Write block 1717 len 4096 data: 3
4272 136/300 Write block 2337 len 8192 data: 2 2
4273 137/300 Write block 1737 len 4096 data: 3
4274 138/300 Write block 3347 len 20480 data: 4 4 4 3 3
4275 139/300 Read block 1796 len 16384
4276 140/300 Write block 2537 len 40960 data: 3 3 3 4 4 3 3 3 4 4
4277 141/300 Read block 1248 len 28672
4278 142/300 Write block 2432 len 12288 data: 2 3 2
4279 143/300 Write block 3513 len 28672 data: 3 3 3 3 2 2 2
4280 144/300 Write block 3909 len 28672 data: 2 2 2 2 2 2 2
4281 145/300 Write block 1332 len 8192 data: 3 3
4282 146/300 Write block 4225 len 28672 data: 2 2 2 2 2 2 2
4283 147/300 Read block 2978 len 16384
4284 148/300 Write block 2557 len 24576 data: 2 2 2 2 2 2
4285 149/300 Read block 3573 len 24576
4286 150/300 Flush
4287 151/300 Read block 2373 len 28672
4288 152/300 Flush
4289 153/300 Write block 4459 len 4096 data: 4
4290 154/300 Read block 2435 len 4096
4291 155/300 Flush
4292 156/300 Write block 1051 len 24576 data: 2 2 3 3 3 3
4293 157/300 Write block 1525 len 32768 data: 4 4 3 4 4 4 3 3
4294 158/300 Read block 3389 len 16384
4295 159/300 Flush
4296 160/300 Flush
4297 161/300 Flush
4298 162/300 Write block 2172 len 4096 data: 4
4299 163/300 Read block 1071 len 16384
4300 164/300 Read block 3985 len 28672
4301 165/300 Read block 1242 len 16384
4302 166/300 Read block 1021 len 20480
4303 167/300 Read block 2816 len 12288
4304 168/300 Read block 4272 len 12288
4305 169/300 Read block 2286 len 16384
4306 170/300 Write block 3654 len 36864 data: 3 3 3 3 3 3 4 4 4
4307 171/300 Write block 1114 len 20480 data: 3 3 2 2 2
4308 172/300 Write block 1472 len 40960 data: 3 3 3 3 3 3 3 3 3 4
4309 173/300 Write block 3473 len 32768 data: 2 2 3 3 3 3 3 3
4310 174/300 Read block 931 len 36864
4311 175/300 Read block 1572 len 28672
4312 176/300 Read block 4721 len 36864
4313 177/300 Read block 1004 len 40960
4314 178/300 Flush
4315 179/300 Read block 1273 len 4096
4316 180/300 Read block 2265 len 8192
4317 181/300 Read block 1845 len 40960
4318 182/300 Write block 1425 len 16384 data: 2 2 2 2
4319 183/300 Read block 2219 len 8192
4320 184/300 Read block 4188 len 40960
4321 185/300 Write block 4106 len 32768 data: 2 3 4 4 2 2 2 2
4322 186/300 Write block 230 len 8192 data: 2 2
4323 187/300 Read block 4557 len 8192
4324 188/300 Write block 3004 len 12288 data: 4 5 4
4325 189/300 Write block 2883 len 8192 data: 2 2
4326 190/300 Read block 1226 len 40960
4327 191/300 Read block 1692 len 28672
4328 192/300 Write block 4981 len 4096 data: 2
4329 193/300 Write block 4120 len 20480 data: 3 4 4 4 4
4330 194/300 Write block 4785 len 24576 data: 2 2 2 2 2 2
4331 195/300 Write block 3601 len 40960 data: 2 2 2 2 2 2 2 2 3 3
4332 196/300 Read block 4194 len 24576
4333 197/300 Write block 3526 len 24576 data: 2 2 2 3 3 3
4334 198/300 Flush
4335 199/300 Write block 1690 len 16384 data: 3 4 6 6
4336 200/300 Read block 3745 len 16384
4337 201/300 Write block 3949 len 8192 data: 3 3
4338 202/300 Read block 4969 len 12288
4339 203/300 Read block 1433 len 36864
4340 204/300 Flush
4341 205/300 Write block 1044 len 12288 data: 2 2 2
4342 206/300 Read block 1584 len 40960
4343 207/300 Read block 3678 len 36864
4344 208/300 Write block 438 len 36864 data: 2 2 2 2 3 3 3 3 3
4345 209/300 Write block 3476 len 4096 data: 4
4346 210/300 Write block 4884 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4347 211/300 Read block 641 len 36864
4348 212/300 Read block 499 len 20480
4349 213/300 Read block 4314 len 40960
4350 214/300 Write block 1111 len 32768 data: 2 3 3 4 4 3 3 3
4351 215/300 Read block 2221 len 16384
4352 216/300 Write block 1940 len 16384 data: 2 2 2 2
4353 217/300 Read block 1468 len 24576
4354 218/300 Read block 988 len 20480
4355 219/300 Write block 4729 len 32768 data: 2 2 2 2 2 3 3 3
4356 220/300 Read block 2967 len 20480
4357 221/300 Write block 3001 len 8192 data: 4 4
4358 222/300 Read block 3420 len 8192
4359 223/300 Write block 2298 len 12288 data: 2 2 2
4360 224/300 Read block 4635 len 4096
4361 225/300 Read block 4507 len 4096
4362 226/300 Read block 14 len 24576
4363 227/300 Read block 2593 len 40960
4364 228/300 Flush
4365 229/300 Read block 181 len 28672
4366 230/300 Read block 4680 len 16384
4367 231/300 Write block 4019 len 36864 data: 2 2 2 2 2 2 2 3 3
4368 232/300 Write block 3805 len 8192 data: 2 2
4369 233/300 Write block 3974 len 8192 data: 2 2
4370 234/300 Write block 1130 len 8192 data: 2 2
4371 235/300 Write block 1114 len 24576 data: 5 5 4 4 4 2
4372 236/300 Flush
4373 237/300 Flush
4374 238/300 Write block 2400 len 36864 data: 2 2 2 2 2 2 2 2 2
4375 239/300 Write block 3903 len 36864 data: 2 2 2 2 2 2 3 3 3
4376 240/300 Read block 3842 len 36864
4377 241/300 Write block 1255 len 24576 data: 4 4 4 3 3 2
4378 242/300 Read block 1035 len 36864
4379 243/300 Write block 3400 len 16384 data: 2 3 3 3
4380 244/300 Read block 2821 len 12288
4381 245/300 Read block 3098 len 40960
4382 246/300 Write block 1345 len 32768 data: 2 2 2 2 2 2 2 2
4383 247/300 Flush
4384 248/300 Read block 1585 len 16384
4385 249/300 Write block 408 len 24576 data: 3 3 3 2 2 2
4386 250/300 Write block 206 len 12288 data: 2 2 2
4387 251/300 Read block 2269 len 4096
4388 252/300 Read block 2670 len 28672
4389 253/300 Write block 738 len 16384 data: 2 2 2 2
4390 254/300 Read block 4065 len 16384
4391 255/300 Read block 1573 len 40960
4392 256/300 Write block 3661 len 12288 data: 5 5 4
4393 257/300 Read block 1607 len 16384
4394 258/300 Read block 4359 len 40960
4395 259/300 Read block 4058 len 12288
4396 260/300 Read block 4706 len 28672
4397 261/300 Flush
4398 262/300 Write block 1359 len 36864 data: 2 3 3 3 3 3 2 2 2
4399 263/300 Read block 4384 len 36864
4400 264/300 Write block 2825 len 32768 data: 4 5 5 4 4 4 4 4
4401 265/300 Read block 1477 len 12288
4402 266/300 Read block 1321 len 16384
4403 267/300 Write block 3701 len 28672 data: 2 2 2 2 2 2 2
4404 268/300 Read block 4635 len 28672
4405 269/300 Flush
4406 270/300 Read block 1255 len 24576
4407 271/300 Read block 754 len 36864
4408 272/300 Read block 665 len 36864
4409 273/300 Flush
4410 274/300 Write block 1974 len 16384 data: 2 2 2 2
4411 275/300 Read block 2238 len 24576
4412 276/300 Write block 1658 len 12288 data: 2 2 2
4413 277/300 Write block 3715 len 4096 data: 3
4414 278/300 Write block 1835 len 36864 data: 2 2 2 2 2 2 2 2 2
4415 279/300 Flush
4416 280/300 Write block 2597 len 4096 data: 2
4417 281/300 Write block 633 len 32768 data: 2 2 2 2 2 2 2 2
4418 282/300 Write block 465 len 28672 data: 2 2 2 2 2 2 2
4419 283/300 Read block 877 len 4096
4420 284/300 Write block 3109 len 20480 data: 3 3 3 3 3
4421 285/300 Read block 681 len 36864
4422 286/300 Read block 4185 len 20480
4423 287/300 Write block 3283 len 8192 data: 2 2
4424 288/300 Flush
4425 289/300 Write block 3475 len 32768 data: 4 5 4 4 4 4 2 2
4426 290/300 Read block 1558 len 4096
4427 291/300 Write block 2021 len 12288 data: 2 2 2
4428 292/300 Read block 164 len 12288
4429 293/300 Read block 1733 len 20480
4430 294/300 Flush
4431 295/300 Read block 2158 len 40960
4432 296/300 Read block 2040 len 12288
4433 297/300 Read block 1549 len 36864
4434 298/300 Read block 5 len 4096
4435 299/300 Write block 3396 len 32768 data: 3 2 2 2 3 4 4 4
4436 300/300 Read block 4467 len 8192
4437 301/300 Write block 1389 len 4096 data: 3
4438 Main task has work to do, go find it
4439 got dsc Start(2)
4440 start 2
4441 Dec 04 13:31:39.368 INFO request completed, latency_us: 187, response_code: 204, uri: /start/cid/2, method: POST, req_id: af2fd366-92e8-4c70-87ce-e3b1cee336a8, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
4442 Replay: started 2, returned:Ok(())
4443 ----------------------------------------------------------------
4444 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4445 States: Active Active Offline
4446 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4447 2207 Acked Read 4 Done Done Sent false
4448 2208 Acked Write 8 Done Done Sent false
4449 2209 Acked Write 3 Done Done Sent false
4450 2210 Acked Read 9 Done Done Sent false
4451 2211 Acked Read 1 Done Done Sent false
4452 2212 Acked Write 5 Done Done Sent false
4453 2213 Acked Write 8 Done Done Sent false
4454 2214 Acked Write 9 Done Done Sent false
4455 2215 Acked Read 4 Done Done Sent false
4456 2216 Acked Write 4 Done Done Sent false
4457 2217 Acked Flush 0 Done Done Sent false
4458 2218 Acked Write 10 Done Done Sent false
4459 2219 Acked Read 4 Done Done Sent false
4460 2220 Acked Flush 0 Done Done Sent false
4461 2221 Acked Write 2 Done Done Sent false
4462 2222 Acked Read 3 Done Done Sent false
4463 2223 Acked Flush 0 Done Done Sent false
4464 2224 Acked Read 6 Done Done Sent false
4465 2225 Acked Read 3 Done Done Sent false
4466 2226 Acked Read 4 Done Done Sent false
4467 2227 Acked Flush 0 Done Done Sent false
4468 2228 Acked Write 8 Done Done Sent false
4469 2229 Acked Write 8 Done Done Sent false
4470 2230 Acked Read 8 Done Done Sent false
4471 2231 Acked Read 3 Done Done Sent false
4472 2232 Acked Read 5 Done Done Sent false
4473 2233 Acked Flush 0 Done Done Sent false
4474 2234 Acked Read 3 Done Done Sent false
4475 2235 Acked Write 1 Done Done Sent false
4476 2236 Acked Read 10 Done Done Sent false
4477 2237 Acked Flush 0 Done Done Sent false
4478 2238 Acked Read 1 Done Done Sent false
4479 2239 Acked Write 5 Done Done Sent false
4480 2240 Acked Flush 0 Done Done Sent false
4481 2241 Acked Write 4 Done Done Sent false
4482 2242 Acked Flush 0 Done Done Sent false
4483 2243 Acked Write 10 Done Done Sent false
4484 2244 Acked Write 4 Done Done Sent false
4485 2245 Acked Write 2 Done Done Sent false
4486 2246 Acked Write 10 Done Done Sent false
4487 2247 Acked Write 10 Done Done Sent false
4488 2248 Acked Write 6 Done Done Sent false
4489 2249 Acked Read 2 Done Done Sent false
4490 2250 Acked Read 7 Done Done Sent false
4491 2251 Acked Read 4 Done Done Sent false
4492 2252 Acked Read 10 Done Done Sent false
4493 2253 Acked Read 8 Done Done Sent false
4494 2254 Acked Write 1 Done Done Sent false
4495 2255 Acked Write 4 Done Done Sent false
4496 2256 Acked Write 1 Done Done Sent false
4497 2257 Acked Write 5 Done Done Sent false
4498 2258 Acked Read 6 Done Done Sent false
4499 2259 Acked Read 10 Done Done Sent false
4500 2260 Acked Write 7 Done Done Sent false
4501 2261 Acked Read 4 Done Done Sent false
4502 2262 Acked Read 10 Done Done Sent false
4503 2263 Acked Write 6 Done Done Sent false
4504 2264 Acked Read 7 Done Done Sent false
4505 2265 Acked Flush 0 Done Done Sent false
4506 2266 Acked Read 1 Done Done Sent false
4507 2267 Acked Read 7 Done Done Sent false
4508 2268 Acked Write 9 Done Done Sent false
4509 2269 Acked Read 10 Done Done Sent false
4510 2270 Acked Read 7 Done Done Sent false
4511 2271 Acked Flush 0 Done Done Sent false
4512 2272 Acked Read 10 Done Done Sent false
4513 2273 Acked Write 10 Done Done Sent false
4514 2274 Acked Write 9 Done Done Sent false
4515 2275 Acked Read 4 Done Done Sent false
4516 2276 Acked Read 1 Done Done Sent false
4517 2277 Acked Read 3 Done Done Sent false
4518 2278 Acked Read 9 Done Done Sent false
4519 2279 Acked Read 1 Done Done Sent false
4520 2280 Acked Write 5 Done Done Sent false
4521 2281 Acked Read 4 Done Done Sent false
4522 2282 Acked Read 9 Done Done Sent false
4523 2283 Acked Read 1 Done Done Sent false
4524 2284 Acked Write 7 Done Done Sent false
4525 2285 Acked Read 1 Done Done Sent false
4526 2286 Acked Write 7 Done Done Sent false
4527 2287 Acked Write 1 Done Done Sent false
4528 2288 Acked Write 1 Done Done Sent false
4529 2289 Acked Read 10 Done Done Sent false
4530 2290 Acked Read 2 Done Done Sent false
4531 2291 Acked Read 10 Done Done Sent false
4532 2292 Acked Write 8 Done Done Sent false
4533 2293 Acked Read 8 Done Done Sent false
4534 2294 Acked Read 4 Done Done Sent false
4535 2295 Acked Write 6 Done Done Sent false
4536 2296 Acked Read 2 Done Done Sent false
4537 2297 Acked Read 1 Done Done Sent false
4538 2298 Acked Read 9 Done Done Sent false
4539 2299 Acked Write 6 Done Done Sent false
4540 2300 Acked Write 2 Done Done Sent false
4541 2301 Acked Read 10 Done Done Sent false
4542 2302 Acked Write 7 Done Done Sent false
4543 2303 Acked Read 6 Done Done Sent false
4544 2304 Acked Read 7 Done Done Sent false
4545 2305 Acked Read 10 Done Done Sent false
4546 2306 Acked Flush 0 Done Done Sent false
4547 2307 Acked Flush 0 Done Done Sent false
4548 2308 Acked Read 3 Done Done Sent false
4549 2309 Acked Read 4 Done Done Sent false
4550 2310 Acked Flush 0 Done Done Sent false
4551 2311 Acked Read 6 Done Done Sent false
4552 2312 Acked Read 1 Done Done Sent false
4553 2313 Acked Flush 0 Done Done Sent false
4554 2314 Acked Read 1 Done Done Sent false
4555 2315 Acked Read 8 Done Done Sent false
4556 2316 Acked Flush 0 Done Done Sent false
4557 2317 Acked Read 9 Done Done Sent false
4558 2318 Acked Write 6 Done Done Sent false
4559 2319 Acked Read 6 Done Done Sent false
4560 2320 Acked Read 10 Done Done Sent false
4561 2321 Acked Read 4 Done Done Sent false
4562 2322 Acked Read 7 Done Done Sent false
4563 2323 Acked Read 8 Done Done Sent false
4564 2324 Acked Flush 0 Done Done Sent false
4565 2325 Acked Read 9 Done Done Sent false
4566 2326 Acked Flush 0 Done Done Sent false
4567 2327 Acked Read 9 Done Done Sent false
4568 2328 Acked Read 3 Done Done Sent false
4569 2329 Acked Write 9 Done Done Sent false
4570 2330 Acked Write 2 Done Done Sent false
4571 2331 Acked Read 10 Done Done Sent false
4572 2332 Acked Write 5 Done Done Sent false
4573 2333 Acked Write 7 Done Done Sent false
4574 2334 Acked Read 8 Done Done Sent false
4575 2335 Acked Write 7 Done Done Sent false
4576 2336 Acked Write 2 Done Done Sent false
4577 2337 Acked Read 7 Done Done Sent false
4578 2338 Acked Read 8 Done Done Sent false
4579 2339 Acked Read 4 Done Done Sent false
4580 2340 Acked Write 2 Done Done Sent false
4581 2341 Acked Write 1 Done Done Sent false
4582 2342 Acked Write 2 Done Done Sent false
4583 2343 Acked Write 1 Done Done Sent false
4584 2344 Acked Write 5 Done Done Sent false
4585 2345 Acked Read 4 Done Done Sent false
4586 2346 Acked Write 10 Done Done Sent false
4587 2347 Acked Read 7 Done Done Sent false
4588 2348 Acked Write 3 Done Done Sent false
4589 2349 Acked Write 7 Done Done Sent false
4590 2350 Acked Write 7 Done Done Sent false
4591 2351 Acked Write 2 Done Done Sent false
4592 2352 Acked Write 7 Done Done Sent false
4593 2353 Acked Read 4 Done Done Sent false
4594 2354 Acked Write 6 Done Done Sent false
4595 2355 Acked Read 6 Done Done Sent false
4596 2356 Acked Flush 0 Done Done Sent false
4597 2357 Acked Read 7 Done Done Sent false
4598 2358 Acked Flush 0 Done Done Sent false
4599 2359 Acked Write 1 Done Done Sent false
4600 2360 Acked Read 1 Done Done Sent false
4601 2361 Acked Flush 0 Done Done Sent false
4602 2362 Acked Write 6 Done Done Sent false
4603 2363 Acked Write 8 Done Done Sent false
4604 2364 Acked Read 4 Done Done Sent false
4605 2365 Acked Flush 0 Done Done Sent false
4606 2366 Acked Flush 0 Done Done Sent false
4607 2367 Acked Flush 0 Done Done Sent false
4608 2368 Acked Write 1 Done Done Sent false
4609 2369 Acked Read 4 Done Done Sent false
4610 2370 Acked Read 7 Done Done Sent false
4611 2371 Acked Read 4 Done Done Sent false
4612 2372 Acked Read 5 Done Done Sent false
4613 2373 Acked Read 3 Done Done Sent false
4614 2374 Acked Read 3 Done Done Sent false
4615 2375 Acked Read 4 Done Done Sent false
4616 2376 Acked Write 9 Done Done Sent false
4617 2377 Acked Write 5 Done Done Sent false
4618 2378 Acked Write 10 Done Done Sent false
4619 2379 Acked Write 8 Done Done Sent false
4620 2380 Acked Read 9 Done Done Sent false
4621 2381 Acked Read 7 Done Done Sent false
4622 2382 Acked Read 9 Done Done Sent false
4623 2383 Acked Read 10 Done Done Sent false
4624 2384 Acked Flush 0 Done Done Sent false
4625 2385 Acked Read 1 Done Done Sent false
4626 2386 Acked Read 2 Done Done Sent false
4627 2387 Acked Read 10 Done Done Sent false
4628 2388 Acked Write 4 Done Done Sent false
4629 2389 Acked Read 2 Done Done Sent false
4630 2390 Acked Read 10 Done Done Sent false
4631 2391 Acked Write 8 Done Done Sent false
4632 2392 Acked Write 2 Done Done Sent false
4633 2393 Acked Read 2 Done Done Sent false
4634 2394 Acked Write 3 Done Done Sent false
4635 2395 Acked Write 2 Done Done Sent false
4636 2396 Acked Read 10 Done Done Sent false
4637 2397 Acked Read 7 Done Done Sent false
4638 2398 Acked Write 1 Done Done Sent false
4639 2399 Acked Write 5 Done Done Sent false
4640 2400 Acked Write 6 Done Done Sent false
4641 2401 Acked Write 10 Done Done Sent false
4642 2402 Acked Read 6 Done Done Sent false
4643 2403 Acked Write 6 Done Done Sent false
4644 2404 Acked Flush 0 Done Done Sent false
4645 2405 Acked Write 4 Done Done Sent false
4646 2406 Acked Read 4 Done Done Sent false
4647 2407 Acked Write 2 Done Done Sent false
4648 2408 Acked Read 3 Done Done Sent false
4649 2409 Acked Read 9 Done Done Sent false
4650 2410 Acked Flush 0 Done Done Sent false
4651 2411 Acked Write 3 Done Done Sent false
4652 2412 Acked Read 10 Done Done Sent false
4653 2413 Acked Read 9 Done Done Sent false
4654 2414 Acked Write 9 Done Done Sent false
4655 2415 Acked Write 1 Done Done Sent false
4656 2416 Acked Write 10 Done Done Sent false
4657 2417 Acked Read 9 Done Done Sent false
4658 2418 Acked Read 5 Done Done Sent false
4659 2419 Acked Read 10 Done Done Sent false
4660 2420 Acked Write 8 Done Done Sent false
4661 2421 Acked Read 4 Done Done Sent false
4662 2422 Acked Write 4 Done Done Sent false
4663 2423 Acked Read 6 Done Done Sent false
4664 2424 Acked Read 5 Done Done Sent false
4665 2425 Acked Write 8 Done Done Sent false
4666 2426 Acked Read 5 Done Done Sent false
4667 2427 Acked Write 2 Done Done Sent false
4668 2428 Acked Read 2 Done Done Sent false
4669 2429 Acked Write 3 Done Done Sent false
4670 2430 Acked Read 1 Done Done Sent false
4671 2431 Acked Read 1 Done Done Sent false
4672 2432 Acked Read 6 Done Done Sent false
4673 2433 Acked Read 10 Done Done Sent false
4674 2434 Acked Flush 0 Done Done Sent false
4675 2435 Acked Read 7 Done Done Sent false
4676 2436 Acked Read 4 Done Done Sent false
4677 2437 Acked Write 9 Done Done Sent false
4678 2438 Acked Write 2 Done Done Sent false
4679 2439 Acked Write 2 Done Done Sent false
4680 2440 Acked Write 2 Done Done Sent false
4681 2441 Acked Write 6 Done Done Sent false
4682 2442 Acked Flush 0 Done Done Sent false
4683 2443 Acked Flush 0 Done Done Sent false
4684 2444 Acked Write 9 Done Done Sent false
4685 2445 Acked Write 9 Done Done Sent false
4686 2446 Acked Read 9 Done Done Sent false
4687 2447 Acked Write 6 Done Done Sent false
4688 2448 Acked Read 9 Done Done Sent false
4689 2449 Acked Write 4 Done Done Sent false
4690 2450 Acked Read 3 Done Done Sent false
4691 2451 Acked Read 10 Done Done Sent false
4692 2452 Acked Write 8 Done Done Sent false
4693 2453 Acked Flush 0 Done Done Sent false
4694 2454 Acked Read 4 Done Done Sent false
4695 2455 Acked Write 6 Done Done Sent false
4696 2456 Acked Write 3 Done Done Sent false
4697 2457 Acked Read 1 Done Done Sent false
4698 2458 Acked Read 7 Done Done Sent false
4699 2459 Acked Write 4 Done Done Sent false
4700 2460 Acked Read 4 Done Done Sent false
4701 2461 Acked Read 10 Done Done Sent false
4702 2462 Acked Write 3 Done Done Sent false
4703 2463 Acked Read 4 Done Done Sent false
4704 2464 Acked Read 10 Done Done Sent false
4705 2465 Acked Read 3 Done Done Sent false
4706 2466 Acked Read 7 Done Done Sent false
4707 2467 Acked Flush 0 Done Done Sent false
4708 2468 Acked Write 9 Done Done Sent false
4709 2469 Acked Read 9 Done Done Sent false
4710 2470 Acked Write 8 Done Done Sent false
4711 2471 Acked Read 3 Done Done Sent false
4712 2472 Acked Read 4 Done Done Sent false
4713 2473 Acked Write 7 Done Done Sent false
4714 2474 Acked Read 7 Done Done Sent false
4715 2475 Acked Flush 0 Done Done Sent false
4716 2476 Acked Read 6 Done Done Sent false
4717 2477 Acked Read 9 Done Done Sent false
4718 2478 Acked Read 9 Done Done Sent false
4719 2479 Acked Flush 0 Done Done Sent false
4720 2480 Acked Write 4 Done Done Sent false
4721 2481 Acked Read 6 Done Done Sent false
4722 2482 Acked Write 3 Done Done Sent false
4723 2483 Acked Write 1 Done Done Sent false
4724 2484 Acked Write 9 Done Done Sent false
4725 2485 Acked Flush 0 Done Done Sent false
4726 2486 Acked Write 1 Done Done Sent false
4727 2487 Acked Write 8 Done Done Sent false
4728 2488 Acked Write 7 Done Done Sent false
4729 2489 Acked Read 1 Done Done Sent false
4730 2490 Acked Write 5 Done Done Sent false
4731 2491 Acked Read 9 Done Done Sent false
4732 2492 Acked Read 5 Done Done Sent false
4733 2493 Acked Write 2 Done Done Sent false
4734 2494 Acked Flush 0 Done Done Sent false
4735 2495 Acked Write 8 Done Done Sent false
4736 2496 Acked Read 1 Done Done Sent false
4737 2497 Acked Write 3 Done Done Sent false
4738 2498 Acked Read 3 Done Done Sent false
4739 2499 Acked Read 5 Done Done Sent false
4740 2500 Acked Flush 0 Done Done Sent false
4741 2501 Acked Read 10 Done Done Sent false
4742 2502 Acked Read 3 Done Done Sent false
4743 2503 Acked Read 9 Done Done Sent false
4744 2504 Acked Read 1 Done Done Sent false
4745 2505 Acked Write 8 Done Done Sent false
4746 2506 Acked Read 2 Done Done Sent false
4747 2507 Acked Write 1 Done Done Sent false
4748 STATES DS:0 DS:1 DS:2 TOTAL
4749 Sent 0 0 301 301
4750 Done 301 301 0 602
4751 Skipped 0 0 0 0
4752 Error 0 0 0 0
4753 Last Flush: 2500 2500 2206
4754 Downstairs last five completed: 2206 2205 2204 2203 2202
4755 Upstairs last five completed: 2507 2506 2505 2504 2503
4756 CLIENT: Up:0 ds:301 act:2
4757 [8830] got start action
4758 [8830] I am going to restart
4759 Starting downstairs at port 8830
4760 Make output file at "/tmp/dsc/downstairs-8830.txt"
4761 [8830][2] reports Starting
4762 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1208)
4763 [8830][2] reports Running
4764 ----------------------------------------------------------------
4765 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4766 States: Active Active Offline
4767 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4768 2207 Acked Read 4 Done Done Sent false
4769 2208 Acked Write 8 Done Done Sent false
4770 2209 Acked Write 3 Done Done Sent false
4771 2210 Acked Read 9 Done Done Sent false
4772 2211 Acked Read 1 Done Done Sent false
4773 2212 Acked Write 5 Done Done Sent false
4774 2213 Acked Write 8 Done Done Sent false
4775 2214 Acked Write 9 Done Done Sent false
4776 2215 Acked Read 4 Done Done Sent false
4777 2216 Acked Write 4 Done Done Sent false
4778 2217 Acked Flush 0 Done Done Sent false
4779 2218 Acked Write 10 Done Done Sent false
4780 2219 Acked Read 4 Done Done Sent false
4781 2220 Acked Flush 0 Done Done Sent false
4782 2221 Acked Write 2 Done Done Sent false
4783 2222 Acked Read 3 Done Done Sent false
4784 2223 Acked Flush 0 Done Done Sent false
4785 2224 Acked Read 6 Done Done Sent false
4786 2225 Acked Read 3 Done Done Sent false
4787 2226 Acked Read 4 Done Done Sent false
4788 2227 Acked Flush 0 Done Done Sent false
4789 2228 Acked Write 8 Done Done Sent false
4790 2229 Acked Write 8 Done Done Sent false
4791 2230 Acked Read 8 Done Done Sent false
4792 2231 Acked Read 3 Done Done Sent false
4793 2232 Acked Read 5 Done Done Sent false
4794 2233 Acked Flush 0 Done Done Sent false
4795 2234 Acked Read 3 Done Done Sent false
4796 2235 Acked Write 1 Done Done Sent false
4797 2236 Acked Read 10 Done Done Sent false
4798 2237 Acked Flush 0 Done Done Sent false
4799 2238 Acked Read 1 Done Done Sent false
4800 2239 Acked Write 5 Done Done Sent false
4801 2240 Acked Flush 0 Done Done Sent false
4802 2241 Acked Write 4 Done Done Sent false
4803 2242 Acked Flush 0 Done Done Sent false
4804 2243 Acked Write 10 Done Done Sent false
4805 2244 Acked Write 4 Done Done Sent false
4806 2245 Acked Write 2 Done Done Sent false
4807 2246 Acked Write 10 Done Done Sent false
4808 2247 Acked Write 10 Done Done Sent false
4809 2248 Acked Write 6 Done Done Sent false
4810 2249 Acked Read 2 Done Done Sent false
4811 2250 Acked Read 7 Done Done Sent false
4812 2251 Acked Read 4 Done Done Sent false
4813 2252 Acked Read 10 Done Done Sent false
4814 2253 Acked Read 8 Done Done Sent false
4815 2254 Acked Write 1 Done Done Sent false
4816 2255 Acked Write 4 Done Done Sent false
4817 2256 Acked Write 1 Done Done Sent false
4818 2257 Acked Write 5 Done Done Sent false
4819 2258 Acked Read 6 Done Done Sent false
4820 2259 Acked Read 10 Done Done Sent false
4821 2260 Acked Write 7 Done Done Sent false
4822 2261 Acked Read 4 Done Done Sent false
4823 2262 Acked Read 10 Done Done Sent false
4824 2263 Acked Write 6 Done Done Sent false
4825 2264 Acked Read 7 Done Done Sent false
4826 2265 Acked Flush 0 Done Done Sent false
4827 2266 Acked Read 1 Done Done Sent false
4828 2267 Acked Read 7 Done Done Sent false
4829 2268 Acked Write 9 Done Done Sent false
4830 2269 Acked Read 10 Done Done Sent false
4831 2270 Acked Read 7 Done Done Sent false
4832 2271 Acked Flush 0 Done Done Sent false
4833 2272 Acked Read 10 Done Done Sent false
4834 2273 Acked Write 10 Done Done Sent false
4835 2274 Acked Write 9 Done Done Sent false
4836 2275 Acked Read 4 Done Done Sent false
4837 2276 Acked Read 1 Done Done Sent false
4838 2277 Acked Read 3 Done Done Sent false
4839 2278 Acked Read 9 Done Done Sent false
4840 2279 Acked Read 1 Done Done Sent false
4841 2280 Acked Write 5 Done Done Sent false
4842 2281 Acked Read 4 Done Done Sent false
4843 2282 Acked Read 9 Done Done Sent false
4844 2283 Acked Read 1 Done Done Sent false
4845 2284 Acked Write 7 Done Done Sent false
4846 2285 Acked Read 1 Done Done Sent false
4847 2286 Acked Write 7 Done Done Sent false
4848 2287 Acked Write 1 Done Done Sent false
4849 2288 Acked Write 1 Done Done Sent false
4850 2289 Acked Read 10 Done Done Sent false
4851 2290 Acked Read 2 Done Done Sent false
4852 2291 Acked Read 10 Done Done Sent false
4853 2292 Acked Write 8 Done Done Sent false
4854 2293 Acked Read 8 Done Done Sent false
4855 2294 Acked Read 4 Done Done Sent false
4856 2295 Acked Write 6 Done Done Sent false
4857 2296 Acked Read 2 Done Done Sent false
4858 2297 Acked Read 1 Done Done Sent false
4859 2298 Acked Read 9 Done Done Sent false
4860 2299 Acked Write 6 Done Done Sent false
4861 2300 Acked Write 2 Done Done Sent false
4862 2301 Acked Read 10 Done Done Sent false
4863 2302 Acked Write 7 Done Done Sent false
4864 2303 Acked Read 6 Done Done Sent false
4865 2304 Acked Read 7 Done Done Sent false
4866 2305 Acked Read 10 Done Done Sent false
4867 2306 Acked Flush 0 Done Done Sent false
4868 2307 Acked Flush 0 Done Done Sent false
4869 2308 Acked Read 3 Done Done Sent false
4870 2309 Acked Read 4 Done Done Sent false
4871 2310 Acked Flush 0 Done Done Sent false
4872 2311 Acked Read 6 Done Done Sent false
4873 2312 Acked Read 1 Done Done Sent false
4874 2313 Acked Flush 0 Done Done Sent false
4875 2314 Acked Read 1 Done Done Sent false
4876 2315 Acked Read 8 Done Done Sent false
4877 2316 Acked Flush 0 Done Done Sent false
4878 2317 Acked Read 9 Done Done Sent false
4879 2318 Acked Write 6 Done Done Sent false
4880 2319 Acked Read 6 Done Done Sent false
4881 2320 Acked Read 10 Done Done Sent false
4882 2321 Acked Read 4 Done Done Sent false
4883 2322 Acked Read 7 Done Done Sent false
4884 2323 Acked Read 8 Done Done Sent false
4885 2324 Acked Flush 0 Done Done Sent false
4886 2325 Acked Read 9 Done Done Sent false
4887 2326 Acked Flush 0 Done Done Sent false
4888 2327 Acked Read 9 Done Done Sent false
4889 2328 Acked Read 3 Done Done Sent false
4890 2329 Acked Write 9 Done Done Sent false
4891 2330 Acked Write 2 Done Done Sent false
4892 2331 Acked Read 10 Done Done Sent false
4893 2332 Acked Write 5 Done Done Sent false
4894 2333 Acked Write 7 Done Done Sent false
4895 2334 Acked Read 8 Done Done Sent false
4896 2335 Acked Write 7 Done Done Sent false
4897 2336 Acked Write 2 Done Done Sent false
4898 2337 Acked Read 7 Done Done Sent false
4899 2338 Acked Read 8 Done Done Sent false
4900 2339 Acked Read 4 Done Done Sent false
4901 2340 Acked Write 2 Done Done Sent false
4902 2341 Acked Write 1 Done Done Sent false
4903 2342 Acked Write 2 Done Done Sent false
4904 2343 Acked Write 1 Done Done Sent false
4905 2344 Acked Write 5 Done Done Sent false
4906 2345 Acked Read 4 Done Done Sent false
4907 2346 Acked Write 10 Done Done Sent false
4908 2347 Acked Read 7 Done Done Sent false
4909 2348 Acked Write 3 Done Done Sent false
4910 2349 Acked Write 7 Done Done Sent false
4911 2350 Acked Write 7 Done Done Sent false
4912 2351 Acked Write 2 Done Done Sent false
4913 2352 Acked Write 7 Done Done Sent false
4914 2353 Acked Read 4 Done Done Sent false
4915 2354 Acked Write 6 Done Done Sent false
4916 2355 Acked Read 6 Done Done Sent false
4917 2356 Acked Flush 0 Done Done Sent false
4918 2357 Acked Read 7 Done Done Sent false
4919 2358 Acked Flush 0 Done Done Sent false
4920 2359 Acked Write 1 Done Done Sent false
4921 2360 Acked Read 1 Done Done Sent false
4922 2361 Acked Flush 0 Done Done Sent false
4923 2362 Acked Write 6 Done Done Sent false
4924 2363 Acked Write 8 Done Done Sent false
4925 2364 Acked Read 4 Done Done Sent false
4926 2365 Acked Flush 0 Done Done Sent false
4927 2366 Acked Flush 0 Done Done Sent false
4928 2367 Acked Flush 0 Done Done Sent false
4929 2368 Acked Write 1 Done Done Sent false
4930 2369 Acked Read 4 Done Done Sent false
4931 2370 Acked Read 7 Done Done Sent false
4932 2371 Acked Read 4 Done Done Sent false
4933 2372 Acked Read 5 Done Done Sent false
4934 2373 Acked Read 3 Done Done Sent false
4935 2374 Acked Read 3 Done Done Sent false
4936 2375 Acked Read 4 Done Done Sent false
4937 2376 Acked Write 9 Done Done Sent false
4938 2377 Acked Write 5 Done Done Sent false
4939 2378 Acked Write 10 Done Done Sent false
4940 2379 Acked Write 8 Done Done Sent false
4941 2380 Acked Read 9 Done Done Sent false
4942 2381 Acked Read 7 Done Done Sent false
4943 2382 Acked Read 9 Done Done Sent false
4944 2383 Acked Read 10 Done Done Sent false
4945 2384 Acked Flush 0 Done Done Sent false
4946 2385 Acked Read 1 Done Done Sent false
4947 2386 Acked Read 2 Done Done Sent false
4948 2387 Acked Read 10 Done Done Sent false
4949 2388 Acked Write 4 Done Done Sent false
4950 2389 Acked Read 2 Done Done Sent false
4951 2390 Acked Read 10 Done Done Sent false
4952 2391 Acked Write 8 Done Done Sent false
4953 2392 Acked Write 2 Done Done Sent false
4954 2393 Acked Read 2 Done Done Sent false
4955 2394 Acked Write 3 Done Done Sent false
4956 2395 Acked Write 2 Done Done Sent false
4957 2396 Acked Read 10 Done Done Sent false
4958 2397 Acked Read 7 Done Done Sent false
4959 2398 Acked Write 1 Done Done Sent false
4960 2399 Acked Write 5 Done Done Sent false
4961 2400 Acked Write 6 Done Done Sent false
4962 2401 Acked Write 10 Done Done Sent false
4963 2402 Acked Read 6 Done Done Sent false
4964 2403 Acked Write 6 Done Done Sent false
4965 2404 Acked Flush 0 Done Done Sent false
4966 2405 Acked Write 4 Done Done Sent false
4967 2406 Acked Read 4 Done Done Sent false
4968 2407 Acked Write 2 Done Done Sent false
4969 2408 Acked Read 3 Done Done Sent false
4970 2409 Acked Read 9 Done Done Sent false
4971 2410 Acked Flush 0 Done Done Sent false
4972 2411 Acked Write 3 Done Done Sent false
4973 2412 Acked Read 10 Done Done Sent false
4974 2413 Acked Read 9 Done Done Sent false
4975 2414 Acked Write 9 Done Done Sent false
4976 2415 Acked Write 1 Done Done Sent false
4977 2416 Acked Write 10 Done Done Sent false
4978 2417 Acked Read 9 Done Done Sent false
4979 2418 Acked Read 5 Done Done Sent false
4980 2419 Acked Read 10 Done Done Sent false
4981 2420 Acked Write 8 Done Done Sent false
4982 2421 Acked Read 4 Done Done Sent false
4983 2422 Acked Write 4 Done Done Sent false
4984 2423 Acked Read 6 Done Done Sent false
4985 2424 Acked Read 5 Done Done Sent false
4986 2425 Acked Write 8 Done Done Sent false
4987 2426 Acked Read 5 Done Done Sent false
4988 2427 Acked Write 2 Done Done Sent false
4989 2428 Acked Read 2 Done Done Sent false
4990 2429 Acked Write 3 Done Done Sent false
4991 2430 Acked Read 1 Done Done Sent false
4992 2431 Acked Read 1 Done Done Sent false
4993 2432 Acked Read 6 Done Done Sent false
4994 2433 Acked Read 10 Done Done Sent false
4995 2434 Acked Flush 0 Done Done Sent false
4996 2435 Acked Read 7 Done Done Sent false
4997 2436 Acked Read 4 Done Done Sent false
4998 2437 Acked Write 9 Done Done Sent false
4999 2438 Acked Write 2 Done Done Sent false
5000 2439 Acked Write 2 Done Done Sent false
5001 2440 Acked Write 2 Done Done Sent false
5002 2441 Acked Write 6 Done Done Sent false
5003 2442 Acked Flush 0 Done Done Sent false
5004 2443 Acked Flush 0 Done Done Sent false
5005 2444 Acked Write 9 Done Done Sent false
5006 2445 Acked Write 9 Done Done Sent false
5007 2446 Acked Read 9 Done Done Sent false
5008 2447 Acked Write 6 Done Done Sent false
5009 2448 Acked Read 9 Done Done Sent false
5010 2449 Acked Write 4 Done Done Sent false
5011 2450 Acked Read 3 Done Done Sent false
5012 2451 Acked Read 10 Done Done Sent false
5013 2452 Acked Write 8 Done Done Sent false
5014 2453 Acked Flush 0 Done Done Sent false
5015 2454 Acked Read 4 Done Done Sent false
5016 2455 Acked Write 6 Done Done Sent false
5017 2456 Acked Write 3 Done Done Sent false
5018 2457 Acked Read 1 Done Done Sent false
5019 2458 Acked Read 7 Done Done Sent false
5020 2459 Acked Write 4 Done Done Sent false
5021 2460 Acked Read 4 Done Done Sent false
5022 2461 Acked Read 10 Done Done Sent false
5023 2462 Acked Write 3 Done Done Sent false
5024 2463 Acked Read 4 Done Done Sent false
5025 2464 Acked Read 10 Done Done Sent false
5026 2465 Acked Read 3 Done Done Sent false
5027 2466 Acked Read 7 Done Done Sent false
5028 2467 Acked Flush 0 Done Done Sent false
5029 2468 Acked Write 9 Done Done Sent false
5030 2469 Acked Read 9 Done Done Sent false
5031 2470 Acked Write 8 Done Done Sent false
5032 2471 Acked Read 3 Done Done Sent false
5033 2472 Acked Read 4 Done Done Sent false
5034 2473 Acked Write 7 Done Done Sent false
5035 2474 Acked Read 7 Done Done Sent false
5036 2475 Acked Flush 0 Done Done Sent false
5037 2476 Acked Read 6 Done Done Sent false
5038 2477 Acked Read 9 Done Done Sent false
5039 2478 Acked Read 9 Done Done Sent false
5040 2479 Acked Flush 0 Done Done Sent false
5041 2480 Acked Write 4 Done Done Sent false
5042 2481 Acked Read 6 Done Done Sent false
5043 2482 Acked Write 3 Done Done Sent false
5044 2483 Acked Write 1 Done Done Sent false
5045 2484 Acked Write 9 Done Done Sent false
5046 2485 Acked Flush 0 Done Done Sent false
5047 2486 Acked Write 1 Done Done Sent false
5048 2487 Acked Write 8 Done Done Sent false
5049 2488 Acked Write 7 Done Done Sent false
5050 2489 Acked Read 1 Done Done Sent false
5051 2490 Acked Write 5 Done Done Sent false
5052 2491 Acked Read 9 Done Done Sent false
5053 2492 Acked Read 5 Done Done Sent false
5054 2493 Acked Write 2 Done Done Sent false
5055 2494 Acked Flush 0 Done Done Sent false
5056 2495 Acked Write 8 Done Done Sent false
5057 2496 Acked Read 1 Done Done Sent false
5058 2497 Acked Write 3 Done Done Sent false
5059 2498 Acked Read 3 Done Done Sent false
5060 2499 Acked Read 5 Done Done Sent false
5061 2500 Acked Flush 0 Done Done Sent false
5062 2501 Acked Read 10 Done Done Sent false
5063 2502 Acked Read 3 Done Done Sent false
5064 2503 Acked Read 9 Done Done Sent false
5065 2504 Acked Read 1 Done Done Sent false
5066 2505 Acked Write 8 Done Done Sent false
5067 2506 Acked Read 2 Done Done Sent false
5068 2507 Acked Write 1 Done Done Sent false
5069 2508 Acked Flush 0 Done Done Sent false
5070 STATES DS:0 DS:1 DS:2 TOTAL
5071 Sent 0 0 302 302
5072 Done 302 302 0 604
5073 Skipped 0 0 0 0
5074 Error 0 0 0 0
5075 Last Flush: 2508 2508 2206
5076 Downstairs last five completed: 2206 2205 2204 2203 2202
5077 Upstairs last five completed: 2508 2507 2506 2505 2504
5078 CLIENT: Up:0 ds:302 act:2
50792024-12-04T13:31:45.130ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50802024-12-04T13:31:45.130ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64675) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50812024-12-04T13:31:45.131ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50822024-12-04T13:31:45.132ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50832024-12-04T13:31:45.132ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50842024-12-04T13:31:45.132ZINFOcrucible: send last flush ID to this DS: 2206 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50852024-12-04T13:31:45.132ZINFOcrucible: Replied this last flush ID: 2206 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50862024-12-04T13:31:45.132ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50872024-12-04T13:31:45.132ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50882024-12-04T13:31:45.132ZINFOcrucible: [2] client re-new 302 jobs since flush 2206 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
50892024-12-04T13:31:45.132ZINFOcrucible: [2] Replayed 302 jobs since flush: 2206 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
5090 ----------------------------------------------------------------
5091 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5092 Downstairs last five completed: 2508 2507 2506 2505 2504
5093 Upstairs last five completed: 2508 2507 2506 2505 2504
5094 CLIENT: Up:0 ds:0 act:3
5095 Replay: All jobs finished
5096 Main task has work to do, go find it
5097 got dsc Stop(1)
5098 stop 1
5099 Dec 04 13:31:47.373 INFO request completed, latency_us: 345, response_code: 204, uri: /stop/cid/1, method: POST, [8820] Got stop action so:false kr:false
5100 req_id: e67f8d79-76de-4fca-b606-fab9e68d007a, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
51012024-12-04T13:31:47.373ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
51022024-12-04T13:31:47.373ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
51032024-12-04T13:31:47.373ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
51042024-12-04T13:31:47.374ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
51052024-12-04T13:31:47.374ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
51062024-12-04T13:31:47.374ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
5107 Dec 04 13:31:47.374 INFO request completed, latency_us: 181, response_code: 200, uri: /state/cid/1, method: GET, req_id: d7aeae3d-632d-4361-8382-1903407211c7, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
5108 [8820] Exited with: ExitStatus(unix_wait_status(9))
5109 [8820][1] reports Exit
5110 Dec 04 13:31:51.375 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: ba6acf58-d1e4-4aba-b58f-5f142acff216, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
5111 001/300 Read block 4106 len 28672
5112 002/300 Read block 1738 len 8192
5113 003/300 Write block 4268 len 4096 data: 2
5114 004/300 Read block 1917 len 8192
5115 005/300 Write block 4196 len 12288 data: 2 2 2
5116 006/300 Write block 3369 len 12288 data: 4 3 2
5117 007/300 Flush
5118 008/300 Read block 494 len 4096
5119 009/300 Write block 4015 len 20480 data: 2 2 2 2 3
5120 010/300 Read block 4734 len 12288
5121 011/300 Read block 4761 len 36864
5122 012/300 Write block 766 len 24576 data: 2 2 2 2 3 3
5123 013/300 Read block 2254 len 24576
5124 014/300 Read block 2435 len 12288
5125 015/300 Read block 3704 len 36864
5126 016/300 Read block 2109 len 16384
5127 017/300 Write block 3845 len 24576 data: 2 2 2 2 2 2
5128 018/300 Read block 4542 len 40960
5129 019/300 Write block 186 len 28672 data: 3 3 3 3 3 3 3
5130 020/300 Write block 1451 len 28672 data: 2 2 2 2 2 2 3
5131 021/300 Flush
5132 022/300 Read block 105 len 24576
5133 023/300 Read block 2448 len 24576
5134 024/300 Write block 2781 len 16384 data: 2 2 2 2
5135 025/300 Write block 1761 len 8192 data: 3 3
5136 026/300 Write block 169 len 24576 data: 5 5 4 3 3 3
5137 027/300 Read block 1030 len 32768
5138 028/300 Write block 3689 len 28672 data: 4 3 4 3 3 2 2
5139 029/300 Write block 3250 len 24576 data: 2 2 2 2 2 3
5140 030/300 Read block 669 len 36864
5141 031/300 Read block 2022 len 40960
5142 032/300 Flush
5143 033/300 Write block 482 len 32768 data: 2 2 2 2 3 3 3 4
5144 034/300 Write block 3209 len 36864 data: 2 2 3 4 4 4 4 5 4
5145 035/300 Flush
5146 036/300 Read block 165 len 36864
5147 037/300 Read block 2246 len 8192
5148 038/300 Write block 197 len 32768 data: 3 3 2 2 2 2 2 2
5149 039/300 Read block 3626 len 16384
5150 040/300 Read block 14 len 24576
5151 041/300 Write block 726 len 32768 data: 2 2 2 2 2 2 2 2
5152 042/300 Read block 852 len 8192
5153 043/300 Read block 4696 len 16384
5154 044/300 Read block 1580 len 4096
5155 045/300 Flush
5156 046/300 Write block 2140 len 28672 data: 3 3 3 3 2 2 2
5157 047/300 Read block 4952 len 36864
5158 048/300 Write block 1057 len 24576 data: 4 4 4 4 4 3
5159 049/300 Read block 4718 len 12288
5160 050/300 Write block 883 len 32768 data: 3 3 3 3 3 3 2 2
5161 051/300 Write block 2107 len 20480 data: 2 2 2 2 2
5162 052/300 Read block 1246 len 40960
5163 053/300 Flush
5164 054/300 Flush
5165 055/300 Read block 84 len 32768
5166 056/300 Read block 3559 len 40960
5167 057/300 Read block 1217 len 24576
5168 058/300 Write block 2368 len 20480 data: 4 4 4 4 3
5169 059/300 Write block 3474 len 40960 data: 3 5 6 5 5 5 5 3 3 2
5170 060/300 Flush
5171 061/300 Read block 3787 len 4096
5172 062/300 Write block 3055 len 36864 data: 3 3 3 3 2 2 2 3 3
5173 063/300 Write block 91 len 20480 data: 3 3 2 2 2
5174 064/300 Write block 3179 len 4096 data: 2
5175 065/300 Write block 1166 len 4096 data: 2
5176 066/300 Write block 539 len 12288 data: 5 5 5
5177 067/300 Write block 2242 len 32768 data: 2 2 2 2 2 3 3 3
5178 068/300 Read block 2628 len 32768
5179 069/300 Write block 277 len 12288 data: 2 2 3
5180 070/300 Read block 1059 len 8192
5181 071/300 Read block 2172 len 8192
5182 072/300 Write block 2707 len 4096 data: 2
5183 073/300 Read block 3613 len 24576
5184 074/300 Flush
5185 075/300 Read block 2260 len 8192
5186 076/300 Read block 1277 len 24576
5187 077/300 Write block 4946 len 32768 data: 3 3 2 2 2 2 2 2
5188 078/300 Flush
5189 079/300 Read block 237 len 12288
5190 080/300 Write block 3948 len 16384 data: 4 4 4 2
5191 081/300 Flush
5192 082/300 Flush
5193 083/300 Read block 3345 len 40960
5194 084/300 Flush
5195 085/300 Write block 4108 len 28672 data: 5 5 3 3 3 3 2
5196 086/300 Read block 285 len 32768
5197 087/300 Flush
5198 088/300 Write block 3157 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5199 089/300 Write block 3646 len 4096 data: 2
5200 090/300 Read block 4537 len 4096
5201 091/300 Read block 2847 len 12288
5202 092/300 Read block 3569 len 36864
5203 093/300 Write block 1257 len 28672 data: 5 4 4 3 2 3 3
5204 094/300 Write block 3678 len 4096 data: 2
5205 095/300 Read block 3616 len 28672
5206 096/300 Read block 1469 len 20480
5207 097/300 Read block 3165 len 28672
5208 098/300 Read block 2741 len 28672
5209 099/300 Read block 3501 len 4096
5210 100/300 Write block 1023 len 20480 data: 3 3 3 2 2
5211 101/300 Flush
5212 102/300 Read block 202 len 40960
5213 103/300 Read block 4208 len 16384
5214 104/300 Read block 4506 len 40960
5215 105/300 Read block 4303 len 40960
5216 106/300 Read block 4918 len 12288
5217 107/300 Write block 2563 len 36864 data: 2 2 2 2 2 2 2 3 3
5218 108/300 Write block 3663 len 12288 data: 5 4 3
5219 109/300 Read block 4646 len 4096
5220 110/300 Read block 2870 len 40960
5221 111/300 Write block 3934 len 32768 data: 3 3 3 3 3 3 3 4
5222 112/300 Write block 3320 len 12288 data: 5 5 4
5223 113/300 Read block 3651 len 12288
5224 114/300 Write block 1658 len 24576 data: 3 3 3 2 2 2
5225 115/300 Write block 669 len 28672 data: 2 2 2 2 2 2 2
5226 116/300 Write block 514 len 8192 data: 3 3
5227 117/300 Read block 541 len 20480
5228 118/300 Write block 4474 len 12288 data: 3 3 3
5229 119/300 Read block 2543 len 16384
5230 120/300 Read block 2393 len 4096
5231 121/300 Read block 2094 len 16384
5232 122/300 Write block 4057 len 24576 data: 4 4 4 4 4 3
5233 123/300 Read block 1158 len 16384
5234 124/300 Write block 1293 len 36864 data: 3 2 2 3 3 3 2 2 2
5235 125/300 Read block 2224 len 36864
5236 126/300 Read block 3331 len 36864
5237 127/300 Write block 1066 len 36864 data: 3 3 4 4 4 3 3 2 2
5238 128/300 Read block 2117 len 12288
5239 129/300 Read block 3227 len 40960
5240 130/300 Flush
5241 131/300 Flush
5242 132/300 Read block 1898 len 36864
5243 133/300 Read block 619 len 36864
5244 134/300 Write block 4326 len 8192 data: 2 2
5245 135/300 Read block 226 len 20480
5246 136/300 Write block 4027 len 36864 data: 4 3 3 3 3 3 2 2 2
5247 137/300 Flush
5248 138/300 Flush
5249 139/300 Write block 2732 len 40960 data: 3 3 3 3 3 3 4 3 3 3
5250 140/300 Write block 2380 len 40960 data: 3 2 2 2 2 2 2 2 2 2
5251 141/300 Read block 1210 len 4096
5252 142/300 Write block 4754 len 20480 data: 3 3 3 3 2
5253 143/300 Flush
5254 144/300 Write block 1672 len 28672 data: 2 2 3 3 3 3 4
5255 145/300 Read block 1220 len 28672
5256 146/300 Read block 1965 len 12288
5257 147/300 Read block 321 len 8192
5258 148/300 Write block 4235 len 20480 data: 2 2 2 2 2
5259 149/300 Read block 4503 len 16384
5260 150/300 Read block 1953 len 4096
5261 151/300 Read block 4239 len 32768
5262 152/300 Write block 3735 len 24576 data: 2 3 3 3 3 3
5263 153/300 Write block 3566 len 8192 data: 2 2
5264 154/300 Write block 2771 len 8192 data: 4 5
5265 155/300 Write block 1297 len 24576 data: 4 4 3 3 3 2
5266 156/300 Write block 2321 len 4096 data: 2
5267 157/300 Read block 1580 len 12288
5268 158/300 Write block 1707 len 4096 data: 2
5269 159/300 Write block 4618 len 32768 data: 2 2 2 2 2 2 2 2
5270 160/300 Read block 2357 len 40960
5271 161/300 Read block 4554 len 36864
5272 162/300 Read block 123 len 36864
5273 163/300 Write block 2322 len 28672 data: 2 2 2 2 2 2 2
5274 164/300 Read block 2375 len 4096
5275 165/300 Read block 2484 len 32768
5276 166/300 Write block 3 len 28672 data: 4 6 4 4 4 3 3
5277 167/300 Read block 500 len 8192
5278 168/300 Read block 1316 len 40960
5279 169/300 Write block 936 len 8192 data: 2 2
5280 170/300 Write block 1071 len 4096 data: 4
5281 171/300 Write block 1592 len 20480 data: 2 2 2 2 2
5282 172/300 Write block 2437 len 8192 data: 2 2
5283 173/300 Write block 4249 len 32768 data: 3 3 2 2 2 2 2 2
5284 174/300 Write block 2187 len 40960 data: 2 2 2 2 3 3 3 3 3 3
5285 175/300 Read block 7 len 16384
5286 176/300 Flush
5287 177/300 Write block 3750 len 16384 data: 2 2 2 2
5288 178/300 Read block 4833 len 8192
5289 179/300 Read block 4685 len 4096
5290 180/300 Write block 4894 len 12288 data: 2 2 2
5291 181/300 Write block 4171 len 24576 data: 2 2 2 2 2 2
5292 182/300 Read block 2069 len 4096
5293 183/300 Write block 4159 len 24576 data: 4 4 4 5 5 3
5294 184/300 Read block 4341 len 32768
5295 185/300 Read block 41 len 16384
5296 186/300 Read block 103 len 36864
5297 187/300 Read block 2181 len 32768
5298 188/300 Write block 4086 len 12288 data: 2 2 2
5299 189/300 Read block 1946 len 4096
5300 190/300 Read block 3579 len 40960
5301 191/300 Read block 4913 len 4096
5302 192/300 Flush
5303 193/300 Write block 2642 len 40960 data: 3 3 3 4 5 5 5 3 2 3
5304 194/300 Read block 3292 len 28672
5305 195/300 Read block 4106 len 40960
5306 196/300 Read block 277 len 4096
5307 197/300 Write block 3886 len 16384 data: 2 2 2 2
5308 198/300 Read block 4984 len 12288
5309 199/300 Read block 881 len 16384
5310 200/300 Read block 312 len 32768
5311 201/300 Read block 3454 len 16384
5312 202/300 Read block 3630 len 20480
5313 203/300 Flush
5314 204/300 Read block 3263 len 36864
5315 205/300 Read block 1583 len 8192
5316 206/300 Write block 3663 len 40960 data: 6 5 4 3 3 2 2 2 3 3
5317 207/300 Write block 1576 len 12288 data: 3 3 3
5318 208/300 Write block 4782 len 8192 data: 3 3
5319 209/300 Read block 37 len 24576
5320 210/300 Write block 2319 len 24576 data: 2 2 3 3 3 3
5321 211/300 Write block 3898 len 36864 data: 2 2 2 2 2 3 3 3 3
5322 212/300 Read block 1357 len 12288
5323 213/300 Read block 3258 len 28672
5324 214/300 Read block 4 len 4096
5325 215/300 Read block 1948 len 28672
5326 216/300 Write block 783 len 36864 data: 2 2 2 2 2 2 2 2 2
5327 217/300 Write block 2153 len 16384 data: 2 3 3 3
5328 218/300 Write block 1984 len 40960 data: 3 2 2 2 2 2 2 2 2 2
5329 219/300 Write block 1098 len 36864 data: 2 2 2 2 2 2 2 2 2
5330 220/300 Read block 1181 len 24576
5331 221/300 Read block 4091 len 16384
5332 222/300 Write block 1397 len 40960 data: 2 3 3 4 2 2 3 3 3 3
5333 223/300 Read block 2654 len 20480
5334 224/300 Write block 3527 len 12288 data: 3 3 4
5335 225/300 Read block 418 len 8192
5336 226/300 Read block 2078 len 24576
5337 227/300 Write block 1581 len 36864 data: 3 3 3 2 2 2 2 2 2
5338 228/300 Read block 2249 len 40960
5339 229/300 Read block 1231 len 16384
5340 230/300 Read block 4204 len 16384
5341 231/300 Write block 286 len 12288 data: 2 2 2
5342 232/300 Write block 860 len 8192 data: 3 2
5343 233/300 Flush
5344 234/300 Read block 2614 len 16384
5345 235/300 Write block 3867 len 20480 data: 2 2 2 2 2
5346 236/300 Read block 947 len 16384
5347 237/300 Write block 3315 len 36864 data: 4 4 4 4 4 6 6 5 4
5348 238/300 Write block 1651 len 40960 data: 2 2 2 2 2 2 2 4 4 4
5349 239/300 Write block 227 len 20480 data: 2 2 2 3 3
5350 240/300 Read block 732 len 16384
5351 241/300 Write block 1152 len 32768 data: 3 3 3 3 3 2 2 2
5352 242/300 Read block 2116 len 36864
5353 243/300 Read block 2838 len 4096
5354 244/300 Write block 1911 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5355 245/300 Read block 2969 len 20480
5356 246/300 Write block 3416 len 20480 data: 2 2 2 2 2
5357 247/300 Read block 1303 len 32768
5358 248/300 Write block 4901 len 40960 data: 3 3 3 4 3 3 3 3 3 2
5359 249/300 Read block 3143 len 20480
5360 250/300 Write block 4009 len 40960 data: 2 2 2 2 2 2 3 3 3 3
5361 251/300 Read block 4231 len 24576
5362 252/300 Write block 4803 len 16384 data: 2 2 2 2
5363 253/300 Read block 2727 len 24576
5364 254/300 Read block 1146 len 20480
5365 255/300 Write block 4887 len 28672 data: 3 3 3 3 3 3 3
5366 256/300 Write block 2313 len 12288 data: 3 2 2
5367 257/300 Read block 605 len 28672
5368 258/300 Write block 1464 len 12288 data: 2 2 2
5369 259/300 Write block 4613 len 36864 data: 2 2 2 2 2 3 3 3 3
5370 260/300 Write block 4837 len 28672 data: 2 2 2 2 2 2 2
5371 261/300 Write block 3434 len 32768 data: 2 2 2 2 3 3 3 3
5372 262/300 Read block 4154 len 8192
5373 263/300 Write block 4034 len 12288 data: 3 3 2
5374 264/300 Read block 374 len 16384
5375 265/300 Read block 4437 len 24576
5376 266/300 Write block 12 len 20480 data: 3 3 3 2 2
5377 267/300 Read block 1025 len 24576
5378 268/300 Write block 4388 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5379 269/300 Write block 3667 len 40960 data: 4 3 3 3 4 4 3 2 2 2
5380 270/300 Write block 4787 len 24576 data: 3 3 3 3 2 2
5381 271/300 Flush
5382 272/300 Write block 277 len 40960 data: 3 3 4 3 2 2 2 2 2 3
5383 273/300 Read block 4067 len 12288
5384 274/300 Read block 2296 len 20480
5385 275/300 Read block 3114 len 40960
5386 276/300 Read block 4538 len 28672
5387 277/300 Read block 4627 len 12288
5388 278/300 Write block 384 len 16384 data: 2 2 2 2
5389 279/300 Read block 3259 len 4096
5390 280/300 Read block 1953 len 16384
5391 281/300 Read block 2157 len 12288
5392 282/300 Write block 2026 len 16384 data: 2 2 2 2
5393 283/300 Read block 4913 len 8192
5394 284/300 Write block 1204 len 4096 data: 2
5395 285/300 Read block 785 len 28672
5396 286/300 Write block 2276 len 36864 data: 2 3 3 3 2 2 2 2 2
5397 287/300 Read block 2656 len 36864
5398 288/300 Read block 4414 len 4096
5399 289/300 Write block 2667 len 32768 data: 4 3 3 3 3 3 3 2
5400 290/300 Read block 4884 len 28672
5401 291/300 Write block 4120 len 12288 data: 4 5 5
5402 292/300 Write block 2186 len 4096 data: 2
5403 293/300 Write block 1526 len 36864 data: 5 4 5 5 5 4 4 2 2
5404 294/300 Write block 435 len 36864 data: 2 2 2 3 3 3 3 4 4
5405 295/300 Read block 4257 len 12288
5406 296/300 Read block 4404 len 28672
5407 297/300 Read block 4566 len 16384
5408 298/300 Read block 4629 len 20480
5409 299/300 Write block 111 len 20480 data: 2 2 3 3 3
5410 300/300 Read block 2915 len 8192
5411 301/300 Write block 2520 len 40960 data: 2 2 2 2 2 2 2 2 3 3
5412 Main task has work to do, go find it
5413 got dsc Start(1)
5414 start 1
5415 Dec 04 13:31:51.608 INFO request completed, latency_us: 235, response_code: 204, uri: /start/cid/1, method: POST, req_id: 69c8ee3f-2c5d-446c-8c79-b1341b62c135, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
5416 Replay: started 1, returned:Ok(())
5417 ----------------------------------------------------------------
5418 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5419 States: Active Offline Active
5420 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5421 2509 Acked Read 7 Done Sent Done false
5422 2510 Acked Read 2 Done Sent Done false
5423 2511 Acked Write 1 Done Sent Done false
5424 2512 Acked Read 2 Done Sent Done false
5425 2513 Acked Write 3 Done Sent Done false
5426 2514 Acked Write 3 Done Sent Done false
5427 2515 Acked Flush 0 Done Sent Done false
5428 2516 Acked Read 1 Done Sent Done false
5429 2517 Acked Write 5 Done Sent Done false
5430 2518 Acked Read 3 Done Sent Done false
5431 2519 Acked Read 9 Done Sent Done false
5432 2520 Acked Write 6 Done Sent Done false
5433 2521 Acked Read 6 Done Sent Done false
5434 2522 Acked Read 3 Done Sent Done false
5435 2523 Acked Read 9 Done Sent Done false
5436 2524 Acked Read 4 Done Sent Done false
5437 2525 Acked Write 6 Done Sent Done false
5438 2526 Acked Read 10 Done Sent Done false
5439 2527 Acked Write 7 Done Sent Done false
5440 2528 Acked Write 7 Done Sent Done false
5441 2529 Acked Flush 0 Done Sent Done false
5442 2530 Acked Read 6 Done Sent Done false
5443 2531 Acked Read 6 Done Sent Done false
5444 2532 Acked Write 4 Done Sent Done false
5445 2533 Acked Write 2 Done Sent Done false
5446 2534 Acked Write 6 Done Sent Done false
5447 2535 Acked Read 8 Done Sent Done false
5448 2536 Acked Write 7 Done Sent Done false
5449 2537 Acked Write 6 Done Sent Done false
5450 2538 Acked Read 9 Done Sent Done false
5451 2539 Acked Read 10 Done Sent Done false
5452 2540 Acked Flush 0 Done Sent Done false
5453 2541 Acked Write 8 Done Sent Done false
5454 2542 Acked Write 9 Done Sent Done false
5455 2543 Acked Flush 0 Done Sent Done false
5456 2544 Acked Read 9 Done Sent Done false
5457 2545 Acked Read 2 Done Sent Done false
5458 2546 Acked Write 8 Done Sent Done false
5459 2547 Acked Read 4 Done Sent Done false
5460 2548 Acked Read 6 Done Sent Done false
5461 2549 Acked Write 8 Done Sent Done false
5462 2550 Acked Read 2 Done Sent Done false
5463 2551 Acked Read 4 Done Sent Done false
5464 2552 Acked Read 1 Done Sent Done false
5465 2553 Acked Flush 0 Done Sent Done false
5466 2554 Acked Write 7 Done Sent Done false
5467 2555 Acked Read 9 Done Sent Done false
5468 2556 Acked Write 6 Done Sent Done false
5469 2557 Acked Read 3 Done Sent Done false
5470 2558 Acked Write 8 Done Sent Done false
5471 2559 Acked Write 5 Done Sent Done false
5472 2560 Acked Read 10 Done Sent Done false
5473 2561 Acked Flush 0 Done Sent Done false
5474 2562 Acked Flush 0 Done Sent Done false
5475 2563 Acked Read 8 Done Sent Done false
5476 2564 Acked Read 10 Done Sent Done false
5477 2565 Acked Read 6 Done Sent Done false
5478 2566 Acked Write 5 Done Sent Done false
5479 2567 Acked Write 10 Done Sent Done false
5480 2568 Acked Flush 0 Done Sent Done false
5481 2569 Acked Read 1 Done Sent Done false
5482 2570 Acked Write 9 Done Sent Done false
5483 2571 Acked Write 5 Done Sent Done false
5484 2572 Acked Write 1 Done Sent Done false
5485 2573 Acked Write 1 Done Sent Done false
5486 2574 Acked Write 3 Done Sent Done false
5487 2575 Acked Write 8 Done Sent Done false
5488 2576 Acked Read 8 Done Sent Done false
5489 2577 Acked Write 3 Done Sent Done false
5490 2578 Acked Read 2 Done Sent Done false
5491 2579 Acked Read 2 Done Sent Done false
5492 2580 Acked Write 1 Done Sent Done false
5493 2581 Acked Read 6 Done Sent Done false
5494 2582 Acked Flush 0 Done Sent Done false
5495 2583 Acked Read 2 Done Sent Done false
5496 2584 Acked Read 6 Done Sent Done false
5497 2585 Acked Write 8 Done Sent Done false
5498 2586 Acked Flush 0 Done Sent Done false
5499 2587 Acked Read 3 Done Sent Done false
5500 2588 Acked Write 4 Done Sent Done false
5501 2589 Acked Flush 0 Done Sent Done false
5502 2590 Acked Flush 0 Done Sent Done false
5503 2591 Acked Read 10 Done Sent Done false
5504 2592 Acked Flush 0 Done Sent Done false
5505 2593 Acked Write 7 Done Sent Done false
5506 2594 Acked Read 8 Done Sent Done false
5507 2595 Acked Flush 0 Done Sent Done false
5508 2596 Acked Write 10 Done Sent Done false
5509 2597 Acked Write 1 Done Sent Done false
5510 2598 Acked Read 1 Done Sent Done false
5511 2599 Acked Read 3 Done Sent Done false
5512 2600 Acked Read 9 Done Sent Done false
5513 2601 Acked Write 7 Done Sent Done false
5514 2602 Acked Write 1 Done Sent Done false
5515 2603 Acked Read 7 Done Sent Done false
5516 2604 Acked Read 5 Done Sent Done false
5517 2605 Acked Read 7 Done Sent Done false
5518 2606 Acked Read 7 Done Sent Done false
5519 2607 Acked Read 1 Done Sent Done false
5520 2608 Acked Write 5 Done Sent Done false
5521 2609 Acked Flush 0 Done Sent Done false
5522 2610 Acked Read 10 Done Sent Done false
5523 2611 Acked Read 4 Done Sent Done false
5524 2612 Acked Read 10 Done Sent Done false
5525 2613 Acked Read 10 Done Sent Done false
5526 2614 Acked Read 3 Done Sent Done false
5527 2615 Acked Write 9 Done Sent Done false
5528 2616 Acked Write 3 Done Sent Done false
5529 2617 Acked Read 1 Done Sent Done false
5530 2618 Acked Read 10 Done Sent Done false
5531 2619 Acked Write 8 Done Sent Done false
5532 2620 Acked Write 3 Done Sent Done false
5533 2621 Acked Read 3 Done Sent Done false
5534 2622 Acked Write 6 Done Sent Done false
5535 2623 Acked Write 7 Done Sent Done false
5536 2624 Acked Write 2 Done Sent Done false
5537 2625 Acked Read 5 Done Sent Done false
5538 2626 Acked Write 3 Done Sent Done false
5539 2627 Acked Read 4 Done Sent Done false
5540 2628 Acked Read 1 Done Sent Done false
5541 2629 Acked Read 4 Done Sent Done false
5542 2630 Acked Write 6 Done Sent Done false
5543 2631 Acked Read 4 Done Sent Done false
5544 2632 Acked Write 9 Done Sent Done false
5545 2633 Acked Read 9 Done Sent Done false
5546 2634 Acked Read 9 Done Sent Done false
5547 2635 Acked Write 9 Done Sent Done false
5548 2636 Acked Read 3 Done Sent Done false
5549 2637 Acked Read 10 Done Sent Done false
5550 2638 Acked Flush 0 Done Sent Done false
5551 2639 Acked Flush 0 Done Sent Done false
5552 2640 Acked Read 9 Done Sent Done false
5553 2641 Acked Read 9 Done Sent Done false
5554 2642 Acked Write 2 Done Sent Done false
5555 2643 Acked Read 5 Done Sent Done false
5556 2644 Acked Write 9 Done Sent Done false
5557 2645 Acked Flush 0 Done Sent Done false
5558 2646 Acked Flush 0 Done Sent Done false
5559 2647 Acked Write 10 Done Sent Done false
5560 2648 Acked Write 10 Done Sent Done false
5561 2649 Acked Read 1 Done Sent Done false
5562 2650 Acked Write 5 Done Sent Done false
5563 2651 Acked Flush 0 Done Sent Done false
5564 2652 Acked Write 7 Done Sent Done false
5565 2653 Acked Read 7 Done Sent Done false
5566 2654 Acked Read 3 Done Sent Done false
5567 2655 Acked Read 2 Done Sent Done false
5568 2656 Acked Write 5 Done Sent Done false
5569 2657 Acked Read 4 Done Sent Done false
5570 2658 Acked Read 1 Done Sent Done false
5571 2659 Acked Read 8 Done Sent Done false
5572 2660 Acked Write 6 Done Sent Done false
5573 2661 Acked Write 2 Done Sent Done false
5574 2662 Acked Write 2 Done Sent Done false
5575 2663 Acked Write 6 Done Sent Done false
5576 2664 Acked Write 1 Done Sent Done false
5577 2665 Acked Read 3 Done Sent Done false
5578 2666 Acked Write 1 Done Sent Done false
5579 2667 Acked Write 8 Done Sent Done false
5580 2668 Acked Read 10 Done Sent Done false
5581 2669 Acked Read 9 Done Sent Done false
5582 2670 Acked Read 9 Done Sent Done false
5583 2671 Acked Write 7 Done Sent Done false
5584 2672 Acked Read 1 Done Sent Done false
5585 2673 Acked Read 8 Done Sent Done false
5586 2674 Acked Write 7 Done Sent Done false
5587 2675 Acked Read 2 Done Sent Done false
5588 2676 Acked Read 10 Done Sent Done false
5589 2677 Acked Write 2 Done Sent Done false
5590 2678 Acked Write 1 Done Sent Done false
5591 2679 Acked Write 5 Done Sent Done false
5592 2680 Acked Write 2 Done Sent Done false
5593 2681 Acked Write 8 Done Sent Done false
5594 2682 Acked Write 10 Done Sent Done false
5595 2683 Acked Read 4 Done Sent Done false
5596 2684 Acked Flush 0 Done Sent Done false
5597 2685 Acked Write 4 Done Sent Done false
5598 2686 Acked Read 2 Done Sent Done false
5599 2687 Acked Read 1 Done Sent Done false
5600 2688 Acked Write 3 Done Sent Done false
5601 2689 Acked Write 6 Done Sent Done false
5602 2690 Acked Read 1 Done Sent Done false
5603 2691 Acked Write 6 Done Sent Done false
5604 2692 Acked Read 8 Done Sent Done false
5605 2693 Acked Read 4 Done Sent Done false
5606 2694 Acked Read 9 Done Sent Done false
5607 2695 Acked Read 8 Done Sent Done false
5608 2696 Acked Write 3 Done Sent Done false
5609 2697 Acked Read 1 Done Sent Done false
5610 2698 Acked Read 10 Done Sent Done false
5611 2699 Acked Read 1 Done Sent Done false
5612 2700 Acked Flush 0 Done Sent Done false
5613 2701 Acked Write 10 Done Sent Done false
5614 2702 Acked Read 7 Done Sent Done false
5615 2703 Acked Read 10 Done Sent Done false
5616 2704 Acked Read 1 Done Sent Done false
5617 2705 Acked Write 4 Done Sent Done false
5618 2706 Acked Read 3 Done Sent Done false
5619 2707 Acked Read 4 Done Sent Done false
5620 2708 Acked Read 8 Done Sent Done false
5621 2709 Acked Read 4 Done Sent Done false
5622 2710 Acked Read 5 Done Sent Done false
5623 2711 Acked Flush 0 Done Sent Done false
5624 2712 Acked Read 9 Done Sent Done false
5625 2713 Acked Read 2 Done Sent Done false
5626 2714 Acked Write 10 Done Sent Done false
5627 2715 Acked Write 3 Done Sent Done false
5628 2716 Acked Write 2 Done Sent Done false
5629 2717 Acked Read 6 Done Sent Done false
5630 2718 Acked Write 6 Done Sent Done false
5631 2719 Acked Write 9 Done Sent Done false
5632 2720 Acked Read 3 Done Sent Done false
5633 2721 Acked Read 7 Done Sent Done false
5634 2722 Acked Read 1 Done Sent Done false
5635 2723 Acked Read 7 Done Sent Done false
5636 2724 Acked Write 9 Done Sent Done false
5637 2725 Acked Write 4 Done Sent Done false
5638 2726 Acked Write 10 Done Sent Done false
5639 2727 Acked Write 9 Done Sent Done false
5640 2728 Acked Read 6 Done Sent Done false
5641 2729 Acked Read 4 Done Sent Done false
5642 2730 Acked Write 10 Done Sent Done false
5643 2731 Acked Read 5 Done Sent Done false
5644 2732 Acked Write 3 Done Sent Done false
5645 2733 Acked Read 2 Done Sent Done false
5646 2734 Acked Read 6 Done Sent Done false
5647 2735 Acked Write 9 Done Sent Done false
5648 2736 Acked Read 10 Done Sent Done false
5649 2737 Acked Read 4 Done Sent Done false
5650 2738 Acked Read 4 Done Sent Done false
5651 2739 Acked Write 3 Done Sent Done false
5652 2740 Acked Write 2 Done Sent Done false
5653 2741 Acked Flush 0 Done Sent Done false
5654 2742 Acked Read 4 Done Sent Done false
5655 2743 Acked Write 5 Done Sent Done false
5656 2744 Acked Read 4 Done Sent Done false
5657 2745 Acked Write 9 Done Sent Done false
5658 2746 Acked Write 10 Done Sent Done false
5659 2747 Acked Write 5 Done Sent Done false
5660 2748 Acked Read 4 Done Sent Done false
5661 2749 Acked Write 8 Done Sent Done false
5662 2750 Acked Read 9 Done Sent Done false
5663 2751 Acked Read 1 Done Sent Done false
5664 2752 Acked Write 10 Done Sent Done false
5665 2753 Acked Read 5 Done Sent Done false
5666 2754 Acked Write 5 Done Sent Done false
5667 2755 Acked Read 8 Done Sent Done false
5668 2756 Acked Write 10 Done Sent Done false
5669 2757 Acked Read 5 Done Sent Done false
5670 2758 Acked Write 10 Done Sent Done false
5671 2759 Acked Read 6 Done Sent Done false
5672 2760 Acked Write 4 Done Sent Done false
5673 2761 Acked Read 6 Done Sent Done false
5674 2762 Acked Read 5 Done Sent Done false
5675 2763 Acked Write 7 Done Sent Done false
5676 2764 Acked Write 3 Done Sent Done false
5677 2765 Acked Read 7 Done Sent Done false
5678 2766 Acked Write 3 Done Sent Done false
5679 2767 Acked Write 9 Done Sent Done false
5680 2768 Acked Write 7 Done Sent Done false
5681 2769 Acked Write 8 Done Sent Done false
5682 2770 Acked Read 2 Done Sent Done false
5683 2771 Acked Write 3 Done Sent Done false
5684 2772 Acked Read 4 Done Sent Done false
5685 2773 Acked Read 6 Done Sent Done false
5686 2774 Acked Write 5 Done Sent Done false
5687 2775 Acked Read 6 Done Sent Done false
5688 2776 Acked Write 10 Done Sent Done false
5689 2777 Acked Write 10 Done Sent Done false
5690 2778 Acked Write 6 Done Sent Done false
5691 2779 Acked Flush 0 Done Sent Done false
5692 2780 Acked Write 10 Done Sent Done false
5693 2781 Acked Read 3 Done Sent Done false
5694 2782 Acked Read 5 Done Sent Done false
5695 2783 Acked Read 10 Done Sent Done false
5696 2784 Acked Read 7 Done Sent Done false
5697 2785 Acked Read 3 Done Sent Done false
5698 2786 Acked Write 4 Done Sent Done false
5699 2787 Acked Read 1 Done Sent Done false
5700 2788 Acked Read 4 Done Sent Done false
5701 2789 Acked Read 3 Done Sent Done false
5702 2790 Acked Write 4 Done Sent Done false
5703 2791 Acked Read 2 Done Sent Done false
5704 2792 Acked Write 1 Done Sent Done false
5705 2793 Acked Read 7 Done Sent Done false
5706 2794 Acked Write 9 Done Sent Done false
5707 2795 Acked Read 9 Done Sent Done false
5708 2796 Acked Read 1 Done Sent Done false
5709 2797 Acked Write 8 Done Sent Done false
5710 2798 Acked Read 7 Done Sent Done false
5711 2799 Acked Write 3 Done Sent Done false
5712 2800 Acked Write 1 Done Sent Done false
5713 2801 Acked Write 9 Done Sent Done false
5714 2802 Acked Write 9 Done Sent Done false
5715 2803 Acked Read 3 Done Sent Done false
5716 2804 Acked Read 7 Done Sent Done false
5717 2805 Acked Read 4 Done Sent Done false
5718 2806 Acked Read 5 Done Sent Done false
5719 2807 Acked Write 5 Done Sent Done false
5720 2808 Acked Read 2 Done Sent Done false
5721 2809 Acked Write 10 Done Sent Done false
5722 STATES DS:0 DS:1 DS:2 TOTAL
5723 Sent 0 301 0 301
5724 Done 301 0 301 602
5725 Skipped 0 0 0 0
5726 Error 0 0 0 0
5727 Last Flush: 2779 2508 2779
5728 Downstairs last five completed: 2508 2507 2506 2505 2504
5729 Upstairs last five completed: 2809 2808 2807 2806 2805
5730 CLIENT: Up:0 ds:301 act:2
5731 [8820] got start action
5732 [8820] I am going to restart
5733 Starting downstairs at port 8820
5734 Make output file at "/tmp/dsc/downstairs-8820.txt"
5735 [8820][1] reports Starting
5736 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1209)
5737 [8820][1] reports Running
5738 ----------------------------------------------------------------
5739 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5740 States: Active Offline Active
5741 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5742 2509 Acked Read 7 Done Sent Done false
5743 2510 Acked Read 2 Done Sent Done false
5744 2511 Acked Write 1 Done Sent Done false
5745 2512 Acked Read 2 Done Sent Done false
5746 2513 Acked Write 3 Done Sent Done false
5747 2514 Acked Write 3 Done Sent Done false
5748 2515 Acked Flush 0 Done Sent Done false
5749 2516 Acked Read 1 Done Sent Done false
5750 2517 Acked Write 5 Done Sent Done false
5751 2518 Acked Read 3 Done Sent Done false
5752 2519 Acked Read 9 Done Sent Done false
5753 2520 Acked Write 6 Done Sent Done false
5754 2521 Acked Read 6 Done Sent Done false
5755 2522 Acked Read 3 Done Sent Done false
5756 2523 Acked Read 9 Done Sent Done false
5757 2524 Acked Read 4 Done Sent Done false
5758 2525 Acked Write 6 Done Sent Done false
5759 2526 Acked Read 10 Done Sent Done false
5760 2527 Acked Write 7 Done Sent Done false
5761 2528 Acked Write 7 Done Sent Done false
5762 2529 Acked Flush 0 Done Sent Done false
5763 2530 Acked Read 6 Done Sent Done false
5764 2531 Acked Read 6 Done Sent Done false
5765 2532 Acked Write 4 Done Sent Done false
5766 2533 Acked Write 2 Done Sent Done false
5767 2534 Acked Write 6 Done Sent Done false
5768 2535 Acked Read 8 Done Sent Done false
5769 2536 Acked Write 7 Done Sent Done false
5770 2537 Acked Write 6 Done Sent Done false
5771 2538 Acked Read 9 Done Sent Done false
5772 2539 Acked Read 10 Done Sent Done false
5773 2540 Acked Flush 0 Done Sent Done false
5774 2541 Acked Write 8 Done Sent Done false
5775 2542 Acked Write 9 Done Sent Done false
5776 2543 Acked Flush 0 Done Sent Done false
5777 2544 Acked Read 9 Done Sent Done false
5778 2545 Acked Read 2 Done Sent Done false
5779 2546 Acked Write 8 Done Sent Done false
5780 2547 Acked Read 4 Done Sent Done false
5781 2548 Acked Read 6 Done Sent Done false
5782 2549 Acked Write 8 Done Sent Done false
5783 2550 Acked Read 2 Done Sent Done false
5784 2551 Acked Read 4 Done Sent Done false
5785 2552 Acked Read 1 Done Sent Done false
5786 2553 Acked Flush 0 Done Sent Done false
5787 2554 Acked Write 7 Done Sent Done false
5788 2555 Acked Read 9 Done Sent Done false
5789 2556 Acked Write 6 Done Sent Done false
5790 2557 Acked Read 3 Done Sent Done false
5791 2558 Acked Write 8 Done Sent Done false
5792 2559 Acked Write 5 Done Sent Done false
5793 2560 Acked Read 10 Done Sent Done false
5794 2561 Acked Flush 0 Done Sent Done false
5795 2562 Acked Flush 0 Done Sent Done false
5796 2563 Acked Read 8 Done Sent Done false
5797 2564 Acked Read 10 Done Sent Done false
5798 2565 Acked Read 6 Done Sent Done false
5799 2566 Acked Write 5 Done Sent Done false
5800 2567 Acked Write 10 Done Sent Done false
5801 2568 Acked Flush 0 Done Sent Done false
5802 2569 Acked Read 1 Done Sent Done false
5803 2570 Acked Write 9 Done Sent Done false
5804 2571 Acked Write 5 Done Sent Done false
5805 2572 Acked Write 1 Done Sent Done false
5806 2573 Acked Write 1 Done Sent Done false
5807 2574 Acked Write 3 Done Sent Done false
5808 2575 Acked Write 8 Done Sent Done false
5809 2576 Acked Read 8 Done Sent Done false
5810 2577 Acked Write 3 Done Sent Done false
5811 2578 Acked Read 2 Done Sent Done false
5812 2579 Acked Read 2 Done Sent Done false
5813 2580 Acked Write 1 Done Sent Done false
5814 2581 Acked Read 6 Done Sent Done false
5815 2582 Acked Flush 0 Done Sent Done false
5816 2583 Acked Read 2 Done Sent Done false
5817 2584 Acked Read 6 Done Sent Done false
5818 2585 Acked Write 8 Done Sent Done false
5819 2586 Acked Flush 0 Done Sent Done false
5820 2587 Acked Read 3 Done Sent Done false
5821 2588 Acked Write 4 Done Sent Done false
5822 2589 Acked Flush 0 Done Sent Done false
5823 2590 Acked Flush 0 Done Sent Done false
5824 2591 Acked Read 10 Done Sent Done false
5825 2592 Acked Flush 0 Done Sent Done false
5826 2593 Acked Write 7 Done Sent Done false
5827 2594 Acked Read 8 Done Sent Done false
5828 2595 Acked Flush 0 Done Sent Done false
5829 2596 Acked Write 10 Done Sent Done false
5830 2597 Acked Write 1 Done Sent Done false
5831 2598 Acked Read 1 Done Sent Done false
5832 2599 Acked Read 3 Done Sent Done false
5833 2600 Acked Read 9 Done Sent Done false
5834 2601 Acked Write 7 Done Sent Done false
5835 2602 Acked Write 1 Done Sent Done false
5836 2603 Acked Read 7 Done Sent Done false
5837 2604 Acked Read 5 Done Sent Done false
5838 2605 Acked Read 7 Done Sent Done false
5839 2606 Acked Read 7 Done Sent Done false
5840 2607 Acked Read 1 Done Sent Done false
5841 2608 Acked Write 5 Done Sent Done false
5842 2609 Acked Flush 0 Done Sent Done false
5843 2610 Acked Read 10 Done Sent Done false
5844 2611 Acked Read 4 Done Sent Done false
5845 2612 Acked Read 10 Done Sent Done false
5846 2613 Acked Read 10 Done Sent Done false
5847 2614 Acked Read 3 Done Sent Done false
5848 2615 Acked Write 9 Done Sent Done false
5849 2616 Acked Write 3 Done Sent Done false
5850 2617 Acked Read 1 Done Sent Done false
5851 2618 Acked Read 10 Done Sent Done false
5852 2619 Acked Write 8 Done Sent Done false
5853 2620 Acked Write 3 Done Sent Done false
5854 2621 Acked Read 3 Done Sent Done false
5855 2622 Acked Write 6 Done Sent Done false
5856 2623 Acked Write 7 Done Sent Done false
5857 2624 Acked Write 2 Done Sent Done false
5858 2625 Acked Read 5 Done Sent Done false
5859 2626 Acked Write 3 Done Sent Done false
5860 2627 Acked Read 4 Done Sent Done false
5861 2628 Acked Read 1 Done Sent Done false
5862 2629 Acked Read 4 Done Sent Done false
5863 2630 Acked Write 6 Done Sent Done false
5864 2631 Acked Read 4 Done Sent Done false
5865 2632 Acked Write 9 Done Sent Done false
5866 2633 Acked Read 9 Done Sent Done false
5867 2634 Acked Read 9 Done Sent Done false
5868 2635 Acked Write 9 Done Sent Done false
5869 2636 Acked Read 3 Done Sent Done false
5870 2637 Acked Read 10 Done Sent Done false
5871 2638 Acked Flush 0 Done Sent Done false
5872 2639 Acked Flush 0 Done Sent Done false
5873 2640 Acked Read 9 Done Sent Done false
5874 2641 Acked Read 9 Done Sent Done false
5875 2642 Acked Write 2 Done Sent Done false
5876 2643 Acked Read 5 Done Sent Done false
5877 2644 Acked Write 9 Done Sent Done false
5878 2645 Acked Flush 0 Done Sent Done false
5879 2646 Acked Flush 0 Done Sent Done false
5880 2647 Acked Write 10 Done Sent Done false
5881 2648 Acked Write 10 Done Sent Done false
5882 2649 Acked Read 1 Done Sent Done false
5883 2650 Acked Write 5 Done Sent Done false
5884 2651 Acked Flush 0 Done Sent Done false
5885 2652 Acked Write 7 Done Sent Done false
5886 2653 Acked Read 7 Done Sent Done false
5887 2654 Acked Read 3 Done Sent Done false
5888 2655 Acked Read 2 Done Sent Done false
5889 2656 Acked Write 5 Done Sent Done false
5890 2657 Acked Read 4 Done Sent Done false
5891 2658 Acked Read 1 Done Sent Done false
5892 2659 Acked Read 8 Done Sent Done false
5893 2660 Acked Write 6 Done Sent Done false
5894 2661 Acked Write 2 Done Sent Done false
5895 2662 Acked Write 2 Done Sent Done false
5896 2663 Acked Write 6 Done Sent Done false
5897 2664 Acked Write 1 Done Sent Done false
5898 2665 Acked Read 3 Done Sent Done false
5899 2666 Acked Write 1 Done Sent Done false
5900 2667 Acked Write 8 Done Sent Done false
5901 2668 Acked Read 10 Done Sent Done false
5902 2669 Acked Read 9 Done Sent Done false
5903 2670 Acked Read 9 Done Sent Done false
5904 2671 Acked Write 7 Done Sent Done false
5905 2672 Acked Read 1 Done Sent Done false
5906 2673 Acked Read 8 Done Sent Done false
5907 2674 Acked Write 7 Done Sent Done false
5908 2675 Acked Read 2 Done Sent Done false
5909 2676 Acked Read 10 Done Sent Done false
5910 2677 Acked Write 2 Done Sent Done false
5911 2678 Acked Write 1 Done Sent Done false
5912 2679 Acked Write 5 Done Sent Done false
5913 2680 Acked Write 2 Done Sent Done false
5914 2681 Acked Write 8 Done Sent Done false
5915 2682 Acked Write 10 Done Sent Done false
5916 2683 Acked Read 4 Done Sent Done false
5917 2684 Acked Flush 0 Done Sent Done false
5918 2685 Acked Write 4 Done Sent Done false
5919 2686 Acked Read 2 Done Sent Done false
5920 2687 Acked Read 1 Done Sent Done false
5921 2688 Acked Write 3 Done Sent Done false
5922 2689 Acked Write 6 Done Sent Done false
5923 2690 Acked Read 1 Done Sent Done false
5924 2691 Acked Write 6 Done Sent Done false
5925 2692 Acked Read 8 Done Sent Done false
5926 2693 Acked Read 4 Done Sent Done false
5927 2694 Acked Read 9 Done Sent Done false
5928 2695 Acked Read 8 Done Sent Done false
5929 2696 Acked Write 3 Done Sent Done false
5930 2697 Acked Read 1 Done Sent Done false
5931 2698 Acked Read 10 Done Sent Done false
5932 2699 Acked Read 1 Done Sent Done false
5933 2700 Acked Flush 0 Done Sent Done false
5934 2701 Acked Write 10 Done Sent Done false
5935 2702 Acked Read 7 Done Sent Done false
5936 2703 Acked Read 10 Done Sent Done false
5937 2704 Acked Read 1 Done Sent Done false
5938 2705 Acked Write 4 Done Sent Done false
5939 2706 Acked Read 3 Done Sent Done false
5940 2707 Acked Read 4 Done Sent Done false
5941 2708 Acked Read 8 Done Sent Done false
5942 2709 Acked Read 4 Done Sent Done false
5943 2710 Acked Read 5 Done Sent Done false
5944 2711 Acked Flush 0 Done Sent Done false
5945 2712 Acked Read 9 Done Sent Done false
5946 2713 Acked Read 2 Done Sent Done false
5947 2714 Acked Write 10 Done Sent Done false
5948 2715 Acked Write 3 Done Sent Done false
5949 2716 Acked Write 2 Done Sent Done false
5950 2717 Acked Read 6 Done Sent Done false
5951 2718 Acked Write 6 Done Sent Done false
5952 2719 Acked Write 9 Done Sent Done false
5953 2720 Acked Read 3 Done Sent Done false
5954 2721 Acked Read 7 Done Sent Done false
5955 2722 Acked Read 1 Done Sent Done false
5956 2723 Acked Read 7 Done Sent Done false
5957 2724 Acked Write 9 Done Sent Done false
5958 2725 Acked Write 4 Done Sent Done false
5959 2726 Acked Write 10 Done Sent Done false
5960 2727 Acked Write 9 Done Sent Done false
5961 2728 Acked Read 6 Done Sent Done false
5962 2729 Acked Read 4 Done Sent Done false
5963 2730 Acked Write 10 Done Sent Done false
5964 2731 Acked Read 5 Done Sent Done false
5965 2732 Acked Write 3 Done Sent Done false
5966 2733 Acked Read 2 Done Sent Done false
5967 2734 Acked Read 6 Done Sent Done false
5968 2735 Acked Write 9 Done Sent Done false
5969 2736 Acked Read 10 Done Sent Done false
5970 2737 Acked Read 4 Done Sent Done false
5971 2738 Acked Read 4 Done Sent Done false
5972 2739 Acked Write 3 Done Sent Done false
5973 2740 Acked Write 2 Done Sent Done false
5974 2741 Acked Flush 0 Done Sent Done false
5975 2742 Acked Read 4 Done Sent Done false
5976 2743 Acked Write 5 Done Sent Done false
5977 2744 Acked Read 4 Done Sent Done false
5978 2745 Acked Write 9 Done Sent Done false
5979 2746 Acked Write 10 Done Sent Done false
5980 2747 Acked Write 5 Done Sent Done false
5981 2748 Acked Read 4 Done Sent Done false
5982 2749 Acked Write 8 Done Sent Done false
5983 2750 Acked Read 9 Done Sent Done false
5984 2751 Acked Read 1 Done Sent Done false
5985 2752 Acked Write 10 Done Sent Done false
5986 2753 Acked Read 5 Done Sent Done false
5987 2754 Acked Write 5 Done Sent Done false
5988 2755 Acked Read 8 Done Sent Done false
5989 2756 Acked Write 10 Done Sent Done false
5990 2757 Acked Read 5 Done Sent Done false
5991 2758 Acked Write 10 Done Sent Done false
5992 2759 Acked Read 6 Done Sent Done false
5993 2760 Acked Write 4 Done Sent Done false
5994 2761 Acked Read 6 Done Sent Done false
5995 2762 Acked Read 5 Done Sent Done false
5996 2763 Acked Write 7 Done Sent Done false
5997 2764 Acked Write 3 Done Sent Done false
5998 2765 Acked Read 7 Done Sent Done false
5999 2766 Acked Write 3 Done Sent Done false
6000 2767 Acked Write 9 Done Sent Done false
6001 2768 Acked Write 7 Done Sent Done false
6002 2769 Acked Write 8 Done Sent Done false
6003 2770 Acked Read 2 Done Sent Done false
6004 2771 Acked Write 3 Done Sent Done false
6005 2772 Acked Read 4 Done Sent Done false
6006 2773 Acked Read 6 Done Sent Done false
6007 2774 Acked Write 5 Done Sent Done false
6008 2775 Acked Read 6 Done Sent Done false
6009 2776 Acked Write 10 Done Sent Done false
6010 2777 Acked Write 10 Done Sent Done false
6011 2778 Acked Write 6 Done Sent Done false
6012 2779 Acked Flush 0 Done Sent Done false
6013 2780 Acked Write 10 Done Sent Done false
6014 2781 Acked Read 3 Done Sent Done false
6015 2782 Acked Read 5 Done Sent Done false
6016 2783 Acked Read 10 Done Sent Done false
6017 2784 Acked Read 7 Done Sent Done false
6018 2785 Acked Read 3 Done Sent Done false
6019 2786 Acked Write 4 Done Sent Done false
6020 2787 Acked Read 1 Done Sent Done false
6021 2788 Acked Read 4 Done Sent Done false
6022 2789 Acked Read 3 Done Sent Done false
6023 2790 Acked Write 4 Done Sent Done false
6024 2791 Acked Read 2 Done Sent Done false
6025 2792 Acked Write 1 Done Sent Done false
6026 2793 Acked Read 7 Done Sent Done false
6027 2794 Acked Write 9 Done Sent Done false
6028 2795 Acked Read 9 Done Sent Done false
6029 2796 Acked Read 1 Done Sent Done false
6030 2797 Acked Write 8 Done Sent Done false
6031 2798 Acked Read 7 Done Sent Done false
6032 2799 Acked Write 3 Done Sent Done false
6033 2800 Acked Write 1 Done Sent Done false
6034 2801 Acked Write 9 Done Sent Done false
6035 2802 Acked Write 9 Done Sent Done false
6036 2803 Acked Read 3 Done Sent Done false
6037 2804 Acked Read 7 Done Sent Done false
6038 2805 Acked Read 4 Done Sent Done false
6039 2806 Acked Read 5 Done Sent Done false
6040 2807 Acked Write 5 Done Sent Done false
6041 2808 Acked Read 2 Done Sent Done false
6042 2809 Acked Write 10 Done Sent Done false
6043 2810 Acked Flush 0 Done Sent Done false
6044 STATES DS:0 DS:1 DS:2 TOTAL
6045 Sent 0 302 0 302
6046 Done 302 0 302 604
6047 Skipped 0 0 0 0
6048 Error 0 0 0 0
6049 Last Flush: 2810 2508 2810
6050 Downstairs last five completed: 2508 2507 2506 2505 2504
6051 Upstairs last five completed: 2810 2809 2808 2807 2806
6052 CLIENT: Up:0 ds:302 act:2
60532024-12-04T13:31:57.375ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60542024-12-04T13:31:57.375ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38057) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60552024-12-04T13:31:57.376ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60562024-12-04T13:31:57.376ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60572024-12-04T13:31:57.376ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60582024-12-04T13:31:57.376ZINFOcrucible: send last flush ID to this DS: 2508 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60592024-12-04T13:31:57.376ZINFOcrucible: Replied this last flush ID: 2508 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60602024-12-04T13:31:57.376ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60612024-12-04T13:31:57.376ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60622024-12-04T13:31:57.377ZINFOcrucible: [1] client re-new 302 jobs since flush 2508 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
60632024-12-04T13:31:57.377ZINFOcrucible: [1] Replayed 302 jobs since flush: 2508 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
6064 ----------------------------------------------------------------
6065 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6066 Downstairs last five completed: 2810 2809 2808 2807 2806
6067 Upstairs last five completed: 2810 2809 2808 2807 2806
6068 CLIENT: Up:0 ds:0 act:3
6069 Replay: All jobs finished
6070 Main task has work to do, go find it
6071 got dsc Stop(0)
6072 stop 0
6073 Dec 04 13:31:59.613 INFO request completed[8810] Got stop action so:false kr:false
6074 , latency_us: 332, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 64a615b0-d065-444d-ba05-a4b5a45838e3, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
60752024-12-04T13:31:59.614ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
6076 Dec 04 13:31:59.614 INFO request completed, latency_us: 111, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:31:59.614391192Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
6077 response_code: 200, uri: /state/cid/0, method: GET, req_id: {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:31:59.614435232Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
6078 36bfdfb6-5512-4ba1-ace5-be4b68268168, remote_addr: 127.0.0.1:37804, local_addr{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:31:59.614523303Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
6079 : 127.0.0.1{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:31:59.614556803Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
6080 :9998
60812024-12-04T13:31:59.614ZINFOcrucible: client task is exiting = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
6082 [8810] Exited with: ExitStatus(unix_wait_status(9))
6083 [8810][0] reports Exit
6084 Dec 04 13:32:03.616 INFO request completed, latency_us: 240, response_code: 200, uri: /state/cid/0, method: GET, req_id: c6967494-d1f8-49b2-8fd0-69bf90a4b614, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
6085 001/300 Read block 2145 len 28672
6086 002/300 Read block 3852 len 8192
6087 003/300 Read block 1058 len 28672
6088 004/300 Write block 1927 len 16384 data: 2 2 3 3
6089 005/300 Read block 1693 len 32768
6090 006/300 Read block 4349 len 40960
6091 007/300 Read block 3371 len 28672
6092 008/300 Read block 906 len 24576
6093 009/300 Read block 1811 len 20480
6094 010/300 Read block 1517 len 32768
6095 011/300 Read block 3167 len 20480
6096 012/300 Read block 619 len 20480
6097 013/300 Write block 4157 len 20480 data: 3 3 5 5 5
6098 014/300 Write block 1169 len 12288 data: 2 2 2
6099 015/300 Read block 4576 len 28672
6100 016/300 Write block 2343 len 4096 data: 3
6101 017/300 Read block 4748 len 32768
6102 018/300 Read block 1419 len 28672
6103 019/300 Write block 4967 len 20480 data: 3 4 4 3 3
6104 020/300 Write block 3209 len 40960 data: 3 3 4 5 5 5 5 6 5 5
6105 021/300 Read block 2201 len 32768
6106 022/300 Read block 4077 len 12288
6107 023/300 Read block 4187 len 8192
6108 024/300 Read block 784 len 20480
6109 025/300 Flush
6110 026/300 Flush
6111 027/300 Read block 2904 len 24576
6112 028/300 Read block 417 len 16384
6113 029/300 Read block 3303 len 8192
6114 030/300 Read block 1335 len 36864
6115 031/300 Read block 648 len 24576
6116 032/300 Flush
6117 033/300 Write block 1611 len 4096 data: 2
6118 034/300 Write block 3571 len 24576 data: 3 3 2 2 2 2
6119 035/300 Read block 2219 len 4096
6120 036/300 Read block 2129 len 36864
6121 037/300 Read block 1193 len 24576
6122 038/300 Read block 1634 len 8192
6123 039/300 Write block 977 len 40960 data: 2 3 3 3 2 2 2 2 2 2
6124 040/300 Write block 3430 len 40960 data: 2 2 2 2 3 3 3 3 4 4
6125 041/300 Write block 560 len 24576 data: 2 2 3 3 3 3
6126 042/300 Write block 1960 len 20480 data: 3 3 3 3 3
6127 043/300 Read block 1682 len 16384
6128 044/300 Write block 2053 len 24576 data: 3 3 3 3 3 3
6129 045/300 Flush
6130 046/300 Write block 3517 len 20480 data: 3 3 3 2 2
6131 047/300 Write block 440 len 24576 data: 4 4 5 5 4 4
6132 048/300 Write block 967 len 8192 data: 3 3
6133 049/300 Read block 2222 len 4096
6134 050/300 Read block 103 len 36864
6135 051/300 Read block 2117 len 32768
6136 052/300 Flush
6137 053/300 Write block 303 len 4096 data: 3
6138 054/300 Read block 2367 len 24576
6139 055/300 Read block 4355 len 40960
6140 056/300 Read block 7 len 32768
6141 057/300 Read block 2375 len 12288
6142 058/300 Read block 1514 len 16384
6143 059/300 Write block 1045 len 36864 data: 3 3 2 2 2 2 3 3 4
6144 060/300 Flush
6145 061/300 Write block 1836 len 4096 data: 3
6146 062/300 Write block 2616 len 28672 data: 2 2 2 2 2 2 2
6147 063/300 Write block 3681 len 36864 data: 2 2 2 2 2 2 3 4 5
6148 064/300 Read block 3355 len 4096
6149 065/300 Flush
6150 066/300 Flush
6151 067/300 Write block 2469 len 28672 data: 3 3 2 2 2 2 2
6152 068/300 Write block 160 len 24576 data: 3 4 4 3 4 4
6153 069/300 Read block 789 len 32768
6154 070/300 Read block 2696 len 16384
6155 071/300 Read block 1228 len 28672
6156 072/300 Read block 4134 len 40960
6157 073/300 Write block 3348 len 20480 data: 5 5 4 4 2
6158 074/300 Write block 24 len 4096 data: 2
6159 075/300 Read block 935 len 28672
6160 076/300 Read block 728 len 36864
6161 077/300 Write block 4158 len 36864 data: 4 6 6 6 6 6 4 3 3
6162 078/300 Read block 2061 len 20480
6163 079/300 Write block 839 len 28672 data: 2 2 2 2 2 2 2
6164 080/300 Write block 3597 len 36864 data: 2 2 2 2 3 3 3 3 3
6165 081/300 Flush
6166 082/300 Write block 735 len 24576 data: 2 2 2 3 3 3
6167 083/300 Flush
6168 084/300 Write block 2334 len 8192 data: 2 2
6169 085/300 Read block 2424 len 36864
6170 086/300 Read block 4467 len 12288
6171 087/300 Read block 391 len 4096
6172 088/300 Read block 10 len 28672
6173 089/300 Write block 891 len 16384 data: 2 2 2 2
6174 090/300 Write block 2301 len 8192 data: 3 3
6175 091/300 Flush
6176 092/300 Read block 1576 len 8192
6177 093/300 Read block 4915 len 4096
6178 094/300 Read block 808 len 28672
6179 095/300 Write block 563 len 8192 data: 4 4
6180 096/300 Write block 316 len 36864 data: 2 2 2 2 2 2 2 2 2
6181 097/300 Write block 2512 len 4096 data: 2
6182 098/300 Read block 4643 len 4096
6183 099/300 Write block 371 len 28672 data: 4 4 4 4 4 4 3
6184 100/300 Flush
6185 101/300 Write block 453 len 36864 data: 2 2 3 3 3 3 3 2 2
6186 102/300 Read block 769 len 40960
6187 103/300 Read block 3042 len 20480
6188 104/300 Read block 88 len 24576
6189 105/300 Read block 1257 len 16384
6190 106/300 Read block 3587 len 8192
6191 107/300 Write block 2925 len 12288 data: 3 3 3
6192 108/300 Read block 3898 len 16384
6193 109/300 Read block 2106 len 36864
6194 110/300 Write block 3999 len 32768 data: 2 2 2 2 2 2 2 2
6195 111/300 Write block 3543 len 28672 data: 2 2 2 3 3 3 3
6196 112/300 Read block 3193 len 24576
6197 113/300 Read block 2044 len 32768
6198 114/300 Write block 1761 len 20480 data: 4 4 3 3 3
6199 115/300 Write block 1839 len 8192 data: 3 3
6200 116/300 Read block 3593 len 40960
6201 117/300 Write block 4739 len 16384 data: 2 2 2 2
6202 118/300 Read block 13 len 40960
6203 119/300 Read block 495 len 12288
6204 120/300 Read block 3730 len 32768
6205 121/300 Write block 4391 len 32768 data: 3 3 3 3 3 3 3 2
6206 122/300 Read block 2057 len 36864
6207 123/300 Write block 4312 len 20480 data: 2 2 2 2 2
6208 124/300 Flush
6209 125/300 Read block 768 len 8192
6210 126/300 Write block 2094 len 12288 data: 2 2 2
6211 127/300 Write block 3638 len 8192 data: 3 2
6212 128/300 Write block 1455 len 20480 data: 3 3 4 3 3
6213 129/300 Read block 1624 len 16384
6214 130/300 Write block 1235 len 32768 data: 2 2 2 2 2 2 2 3
6215 131/300 Write block 3296 len 8192 data: 5 5
6216 132/300 Read block 1094 len 36864
6217 133/300 Flush
6218 134/300 Write block 4740 len 4096 data: 3
6219 135/300 Read block 3947 len 28672
6220 136/300 Write block 362 len 32768 data: 3 3 4 3 3 3 3 4
6221 137/300 Write block 3377 len 20480 data: 4 5 5 5 4
6222 138/300 Read block 2589 len 8192
6223 139/300 Write block 328 len 20480 data: 2 2 3 3 2
6224 140/300 Read block 3388 len 36864
6225 141/300 Read block 2837 len 32768
6226 142/300 Read block 3147 len 40960
6227 143/300 Flush
6228 144/300 Read block 4485 len 12288
6229 145/300 Write block 3676 len 32768 data: 3 2 3 2 2 3 3 3
6230 146/300 Write block 1269 len 16384 data: 3 2 2 2
6231 147/300 Write block 1909 len 36864 data: 2 2 3 3 3 3 3 3 3
6232 148/300 Read block 2405 len 28672
6233 149/300 Read block 959 len 24576
6234 150/300 Write block 1949 len 16384 data: 4 4 4 2
6235 151/300 Write block 2164 len 32768 data: 2 3 3 3 3 3 4 3
6236 152/300 Read block 3706 len 24576
6237 153/300 Read block 473 len 28672
6238 154/300 Write block 2031 len 24576 data: 2 2 2 2 2 2
6239 155/300 Write block 1594 len 28672 data: 3 3 3 2 2 2 2
6240 156/300 Read block 26 len 36864
6241 157/300 Write block 16 len 12288 data: 3 2 2
6242 158/300 Read block 1986 len 36864
6243 159/300 Write block 1893 len 4096 data: 3
6244 160/300 Write block 2501 len 8192 data: 3 2
6245 161/300 Read block 4633 len 36864
6246 162/300 Read block 505 len 28672
6247 163/300 Write block 985 len 8192 data: 3 3
6248 164/300 Read block 646 len 32768
6249 165/300 Write block 3005 len 8192 data: 6 5
6250 166/300 Read block 2875 len 28672
6251 167/300 Read block 2698 len 12288
6252 168/300 Write block 99 len 32768 data: 3 3 3 3 2 2 2 2
6253 169/300 Write block 994 len 4096 data: 2
6254 170/300 Read block 940 len 20480
6255 171/300 Read block 762 len 16384
6256 172/300 Read block 1401 len 12288
6257 173/300 Read block 2019 len 24576
6258 174/300 Write block 3526 len 12288 data: 3 4 4
6259 175/300 Read block 2616 len 20480
6260 176/300 Read block 1491 len 4096
6261 177/300 Write block 4456 len 12288 data: 3 3 3
6262 178/300 Read block 3778 len 16384
6263 179/300 Flush
6264 180/300 Read block 183 len 4096
6265 181/300 Write block 4153 len 40960 data: 3 3 4 4 4 5 7 7 7 7
6266 182/300 Write block 1885 len 28672 data: 2 3 3 5 5 5 4
6267 183/300 Read block 2334 len 4096
6268 184/300 Read block 159 len 8192
6269 185/300 Read block 4501 len 40960
6270 186/300 Read block 1992 len 24576
6271 187/300 Read block 3880 len 28672
6272 188/300 Read block 33 len 32768
6273 189/300 Read block 1459 len 24576
6274 190/300 Write block 3950 len 24576 data: 5 3 2 2 2 2
6275 191/300 Read block 2318 len 20480
6276 192/300 Write block 2942 len 16384 data: 3 4 4 3
6277 193/300 Read block 3186 len 36864
6278 194/300 Write block 3359 len 16384 data: 2 3 4 5
6279 195/300 Read block 2106 len 32768
6280 196/300 Write block 2433 len 8192 data: 4 3
6281 197/300 Read block 1245 len 20480
6282 198/300 Write block 305 len 36864 data: 3 3 3 2 2 2 2 2 2
6283 199/300 Read block 2171 len 12288
6284 200/300 Read block 4553 len 24576
6285 201/300 Read block 1893 len 40960
6286 202/300 Write block 592 len 40960 data: 2 2 2 2 2 2 2 2 3 4
6287 203/300 Write block 806 len 40960 data: 3 3 3 3 3 3 2 2 2 2
6288 204/300 Read block 3798 len 16384
6289 205/300 Read block 1074 len 40960
6290 206/300 Read block 1584 len 24576
6291 207/300 Write block 156 len 32768 data: 4 3 3 3 4 5 5 4
6292 208/300 Read block 4323 len 12288
6293 209/300 Read block 2390 len 32768
6294 210/300 Write block 347 len 24576 data: 2 2 2 2 2 3
6295 211/300 Read block 3754 len 12288
6296 212/300 Write block 3742 len 24576 data: 3 3 3 2 2 2
6297 213/300 Read block 4806 len 32768
6298 214/300 Read block 542 len 36864
6299 215/300 Write block 2463 len 12288 data: 5 5 4
6300 216/300 Read block 1638 len 4096
6301 217/300 Write block 3053 len 40960 data: 3 3 4 4 4 4 3 3 3 4
6302 218/300 Read block 4401 len 12288
6303 219/300 Write block 1344 len 12288 data: 2 3 3
6304 220/300 Read block 3061 len 28672
6305 221/300 Read block 3188 len 12288
6306 222/300 Write block 3462 len 36864 data: 4 3 3 3 3 3 3 2 2
6307 223/300 Write block 3298 len 24576 data: 5 5 5 5 4 3
6308 224/300 Read block 4277 len 40960
6309 225/300 Write block 2161 len 32768 data: 2 2 2 3 4 4 4 4
6310 226/300 Write block 3399 len 8192 data: 3 4
6311 227/300 Write block 1117 len 32768 data: 5 5 3 2 2 2 2 2
6312 228/300 Write block 3928 len 24576 data: 3 3 3 2 2 3
6313 229/300 Write block 11 len 28672 data: 3 4 4 4 3 4 3
6314 230/300 Read block 3374 len 20480
6315 231/300 Read block 2072 len 12288
6316 232/300 Write block 4172 len 4096 data: 3
6317 233/300 Read block 3117 len 36864
6318 234/300 Read block 3453 len 8192
6319 235/300 Read block 1001 len 40960
6320 236/300 Read block 1638 len 16384
6321 237/300 Write block 1741 len 40960 data: 5 6 6 6 5 4 4 4 4 2
6322 238/300 Read block 1131 len 20480
6323 239/300 Read block 752 len 40960
6324 240/300 Read block 3045 len 40960
6325 241/300 Write block 4546 len 32768 data: 2 2 2 2 2 2 2 2
6326 242/300 Read block 323 len 24576
6327 243/300 Write block 4664 len 4096 data: 3
6328 244/300 Write block 1458 len 20480 data: 4 4 3 3 3
6329 245/300 Flush
6330 246/300 Write block 4191 len 40960 data: 3 3 3 3 2 3 3 3 2 3
6331 247/300 Read block 665 len 24576
6332 248/300 Read block 3457 len 20480
6333 249/300 Write block 1070 len 32768 data: 5 5 4 3 3 2 2 3
6334 250/300 Read block 3976 len 24576
6335 251/300 Read block 1821 len 32768
6336 252/300 Read block 4606 len 24576
6337 253/300 Write block 391 len 32768 data: 3 4 4 4 3 3 3 3
6338 254/300 Read block 2493 len 12288
6339 255/300 Read block 3617 len 8192
6340 256/300 Read block 89 len 16384
6341 257/300 Write block 4090 len 24576 data: 2 2 2 2 2 2
6342 258/300 Flush
6343 259/300 Write block 1278 len 4096 data: 2
6344 260/300 Write block 2312 len 16384 data: 2 4 3 3
6345 261/300 Read block 3404 len 20480
6346 262/300 Read block 4939 len 20480
6347 263/300 Flush
6348 264/300 Write block 262 len 36864 data: 3 3 3 2 2 2 2 2 2
6349 265/300 Read block 3071 len 32768
6350 266/300 Write block 1841 len 16384 data: 3 3 3 2
6351 267/300 Read block 1277 len 12288
6352 268/300 Read block 3160 len 4096
6353 269/300 Read block 3927 len 24576
6354 270/300 Write block 1658 len 24576 data: 5 5 5 3 3 3
6355 271/300 Write block 2227 len 4096 data: 2
6356 272/300 Write block 3175 len 32768 data: 4 4 4 3 3 2 2 2
6357 273/300 Read block 1791 len 28672
6358 274/300 Flush
6359 275/300 Write block 2045 len 40960 data: 5 5 3 3 2 3 3 3 4 4
6360 276/300 Write block 769 len 8192 data: 3 4
6361 277/300 Flush
6362 278/300 Flush
6363 279/300 Write block 813 len 8192 data: 3 3
6364 280/300 Read block 3649 len 36864
6365 281/300 Flush
6366 282/300 Write block 1047 len 24576 data: 3 3 3 3 4 4
6367 283/300 Read block 4894 len 4096
6368 284/300 Read block 4957 len 12288
6369 285/300 Write block 1677 len 36864 data: 4 5 4 4 4 4 4 4 4
6370 286/300 Read block 3823 len 16384
6371 287/300 Write block 1501 len 32768 data: 2 3 4 4 5 4 4 4
6372 288/300 Write block 4713 len 24576 data: 2 2 2 2 2 2
6373 289/300 Read block 2524 len 20480
6374 290/300 Read block 3393 len 8192
6375 291/300 Read block 4398 len 16384
6376 292/300 Read block 2377 len 36864
6377 293/300 Write block 295 len 12288 data: 4 4 4
6378 294/300 Read block 4378 len 8192
6379 295/300 Read block 4587 len 40960
6380 296/300 Write block 3598 len 12288 data: 3 3 3
6381 297/300 Read block 2030 len 8192
6382 298/300 Read block 4927 len 12288
6383 299/300 Read block 1062 len 16384
6384 300/300 Read block 3133 len 20480
6385 301/300 Read block 3378 len 8192
6386 Main task has work to do, go find it
6387 got dsc Start(0)
6388 start 0
6389 Dec 04 13:32:03.859 INFO request completed, latency_us: 201, response_code: 204, uri: /start/cid/0, method: POST, req_id: b4a00ea0-7232-4bd1-9b51-792a1696c464, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
6390 Replay: started 0, returned:Ok(())
6391 ----------------------------------------------------------------
6392 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6393 States: Offline Active Active
6394 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6395 2811 Acked Read 7 Sent Done Done false
6396 2812 Acked Read 2 Sent Done Done false
6397 2813 Acked Read 7 Sent Done Done false
6398 2814 Acked Write 4 Sent Done Done false
6399 2815 Acked Read 8 Sent Done Done false
6400 2816 Acked Read 10 Sent Done Done false
6401 2817 Acked Read 7 Sent Done Done false
6402 2818 Acked Read 6 Sent Done Done false
6403 2819 Acked Read 5 Sent Done Done false
6404 2820 Acked Read 8 Sent Done Done false
6405 2821 Acked Read 5 Sent Done Done false
6406 2822 Acked Read 5 Sent Done Done false
6407 2823 Acked Write 5 Sent Done Done false
6408 2824 Acked Write 3 Sent Done Done false
6409 2825 Acked Read 7 Sent Done Done false
6410 2826 Acked Write 1 Sent Done Done false
6411 2827 Acked Read 8 Sent Done Done false
6412 2828 Acked Read 7 Sent Done Done false
6413 2829 Acked Write 5 Sent Done Done false
6414 2830 Acked Write 10 Sent Done Done false
6415 2831 Acked Read 8 Sent Done Done false
6416 2832 Acked Read 3 Sent Done Done false
6417 2833 Acked Read 2 Sent Done Done false
6418 2834 Acked Read 5 Sent Done Done false
6419 2835 Acked Flush 0 Sent Done Done false
6420 2836 Acked Flush 0 Sent Done Done false
6421 2837 Acked Read 6 Sent Done Done false
6422 2838 Acked Read 4 Sent Done Done false
6423 2839 Acked Read 2 Sent Done Done false
6424 2840 Acked Read 9 Sent Done Done false
6425 2841 Acked Read 6 Sent Done Done false
6426 2842 Acked Flush 0 Sent Done Done false
6427 2843 Acked Write 1 Sent Done Done false
6428 2844 Acked Write 6 Sent Done Done false
6429 2845 Acked Read 1 Sent Done Done false
6430 2846 Acked Read 9 Sent Done Done false
6431 2847 Acked Read 6 Sent Done Done false
6432 2848 Acked Read 2 Sent Done Done false
6433 2849 Acked Write 10 Sent Done Done false
6434 2850 Acked Write 10 Sent Done Done false
6435 2851 Acked Write 6 Sent Done Done false
6436 2852 Acked Write 5 Sent Done Done false
6437 2853 Acked Read 4 Sent Done Done false
6438 2854 Acked Write 6 Sent Done Done false
6439 2855 Acked Flush 0 Sent Done Done false
6440 2856 Acked Write 5 Sent Done Done false
6441 2857 Acked Write 6 Sent Done Done false
6442 2858 Acked Write 2 Sent Done Done false
6443 2859 Acked Read 1 Sent Done Done false
6444 2860 Acked Read 9 Sent Done Done false
6445 2861 Acked Read 8 Sent Done Done false
6446 2862 Acked Flush 0 Sent Done Done false
6447 2863 Acked Write 1 Sent Done Done false
6448 2864 Acked Read 6 Sent Done Done false
6449 2865 Acked Read 10 Sent Done Done false
6450 2866 Acked Read 8 Sent Done Done false
6451 2867 Acked Read 3 Sent Done Done false
6452 2868 Acked Read 4 Sent Done Done false
6453 2869 Acked Write 9 Sent Done Done false
6454 2870 Acked Flush 0 Sent Done Done false
6455 2871 Acked Write 1 Sent Done Done false
6456 2872 Acked Write 7 Sent Done Done false
6457 2873 Acked Write 9 Sent Done Done false
6458 2874 Acked Read 1 Sent Done Done false
6459 2875 Acked Flush 0 Sent Done Done false
6460 2876 Acked Flush 0 Sent Done Done false
6461 2877 Acked Write 7 Sent Done Done false
6462 2878 Acked Write 6 Sent Done Done false
6463 2879 Acked Read 8 Sent Done Done false
6464 2880 Acked Read 4 Sent Done Done false
6465 2881 Acked Read 7 Sent Done Done false
6466 2882 Acked Read 10 Sent Done Done false
6467 2883 Acked Write 5 Sent Done Done false
6468 2884 Acked Write 1 Sent Done Done false
6469 2885 Acked Read 7 Sent Done Done false
6470 2886 Acked Read 9 Sent Done Done false
6471 2887 Acked Write 9 Sent Done Done false
6472 2888 Acked Read 5 Sent Done Done false
6473 2889 Acked Write 7 Sent Done Done false
6474 2890 Acked Write 9 Sent Done Done false
6475 2891 Acked Flush 0 Sent Done Done false
6476 2892 Acked Write 6 Sent Done Done false
6477 2893 Acked Flush 0 Sent Done Done false
6478 2894 Acked Write 2 Sent Done Done false
6479 2895 Acked Read 9 Sent Done Done false
6480 2896 Acked Read 3 Sent Done Done false
6481 2897 Acked Read 1 Sent Done Done false
6482 2898 Acked Read 7 Sent Done Done false
6483 2899 Acked Write 4 Sent Done Done false
6484 2900 Acked Write 2 Sent Done Done false
6485 2901 Acked Flush 0 Sent Done Done false
6486 2902 Acked Read 2 Sent Done Done false
6487 2903 Acked Read 1 Sent Done Done false
6488 2904 Acked Read 7 Sent Done Done false
6489 2905 Acked Write 2 Sent Done Done false
6490 2906 Acked Write 9 Sent Done Done false
6491 2907 Acked Write 1 Sent Done Done false
6492 2908 Acked Read 1 Sent Done Done false
6493 2909 Acked Write 7 Sent Done Done false
6494 2910 Acked Flush 0 Sent Done Done false
6495 2911 Acked Write 9 Sent Done Done false
6496 2912 Acked Read 10 Sent Done Done false
6497 2913 Acked Read 5 Sent Done Done false
6498 2914 Acked Read 6 Sent Done Done false
6499 2915 Acked Read 4 Sent Done Done false
6500 2916 Acked Read 2 Sent Done Done false
6501 2917 Acked Write 3 Sent Done Done false
6502 2918 Acked Read 4 Sent Done Done false
6503 2919 Acked Read 9 Sent Done Done false
6504 2920 Acked Write 8 Sent Done Done false
6505 2921 Acked Write 7 Sent Done Done false
6506 2922 Acked Read 6 Sent Done Done false
6507 2923 Acked Read 8 Sent Done Done false
6508 2924 Acked Write 5 Sent Done Done false
6509 2925 Acked Write 2 Sent Done Done false
6510 2926 Acked Read 10 Sent Done Done false
6511 2927 Acked Write 4 Sent Done Done false
6512 2928 Acked Read 10 Sent Done Done false
6513 2929 Acked Read 3 Sent Done Done false
6514 2930 Acked Read 8 Sent Done Done false
6515 2931 Acked Write 8 Sent Done Done false
6516 2932 Acked Read 9 Sent Done Done false
6517 2933 Acked Write 5 Sent Done Done false
6518 2934 Acked Flush 0 Sent Done Done false
6519 2935 Acked Read 2 Sent Done Done false
6520 2936 Acked Write 3 Sent Done Done false
6521 2937 Acked Write 2 Sent Done Done false
6522 2938 Acked Write 5 Sent Done Done false
6523 2939 Acked Read 4 Sent Done Done false
6524 2940 Acked Write 8 Sent Done Done false
6525 2941 Acked Write 2 Sent Done Done false
6526 2942 Acked Read 9 Sent Done Done false
6527 2943 Acked Flush 0 Sent Done Done false
6528 2944 Acked Write 1 Sent Done Done false
6529 2945 Acked Read 7 Sent Done Done false
6530 2946 Acked Write 8 Sent Done Done false
6531 2947 Acked Write 5 Sent Done Done false
6532 2948 Acked Read 2 Sent Done Done false
6533 2949 Acked Write 5 Sent Done Done false
6534 2950 Acked Read 9 Sent Done Done false
6535 2951 Acked Read 8 Sent Done Done false
6536 2952 Acked Read 10 Sent Done Done false
6537 2953 Acked Flush 0 Sent Done Done false
6538 2954 Acked Read 3 Sent Done Done false
6539 2955 Acked Write 8 Sent Done Done false
6540 2956 Acked Write 4 Sent Done Done false
6541 2957 Acked Write 9 Sent Done Done false
6542 2958 Acked Read 7 Sent Done Done false
6543 2959 Acked Read 6 Sent Done Done false
6544 2960 Acked Write 4 Sent Done Done false
6545 2961 Acked Write 8 Sent Done Done false
6546 2962 Acked Read 6 Sent Done Done false
6547 2963 Acked Read 7 Sent Done Done false
6548 2964 Acked Write 6 Sent Done Done false
6549 2965 Acked Write 7 Sent Done Done false
6550 2966 Acked Read 9 Sent Done Done false
6551 2967 Acked Write 3 Sent Done Done false
6552 2968 Acked Read 9 Sent Done Done false
6553 2969 Acked Write 1 Sent Done Done false
6554 2970 Acked Write 2 Sent Done Done false
6555 2971 Acked Read 9 Sent Done Done false
6556 2972 Acked Read 7 Sent Done Done false
6557 2973 Acked Write 2 Sent Done Done false
6558 2974 Acked Read 8 Sent Done Done false
6559 2975 Acked Write 2 Sent Done Done false
6560 2976 Acked Read 7 Sent Done Done false
6561 2977 Acked Read 3 Sent Done Done false
6562 2978 Acked Write 8 Sent Done Done false
6563 2979 Acked Write 1 Sent Done Done false
6564 2980 Acked Read 5 Sent Done Done false
6565 2981 Acked Read 4 Sent Done Done false
6566 2982 Acked Read 3 Sent Done Done false
6567 2983 Acked Read 6 Sent Done Done false
6568 2984 Acked Write 3 Sent Done Done false
6569 2985 Acked Read 5 Sent Done Done false
6570 2986 Acked Read 1 Sent Done Done false
6571 2987 Acked Write 3 Sent Done Done false
6572 2988 Acked Read 4 Sent Done Done false
6573 2989 Acked Flush 0 Sent Done Done false
6574 2990 Acked Read 1 Sent Done Done false
6575 2991 Acked Write 10 Sent Done Done false
6576 2992 Acked Write 7 Sent Done Done false
6577 2993 Acked Read 1 Sent Done Done false
6578 2994 Acked Read 2 Sent Done Done false
6579 2995 Acked Read 10 Sent Done Done false
6580 2996 Acked Read 6 Sent Done Done false
6581 2997 Acked Read 7 Sent Done Done false
6582 2998 Acked Read 8 Sent Done Done false
6583 2999 Acked Read 6 Sent Done Done false
6584 3000 Acked Write 6 Sent Done Done false
6585 3001 Acked Read 5 Sent Done Done false
6586 3002 Acked Write 4 Sent Done Done false
6587 3003 Acked Read 9 Sent Done Done false
6588 3004 Acked Write 4 Sent Done Done false
6589 3005 Acked Read 8 Sent Done Done false
6590 3006 Acked Write 2 Sent Done Done false
6591 3007 Acked Read 5 Sent Done Done false
6592 3008 Acked Write 9 Sent Done Done false
6593 3009 Acked Read 3 Sent Done Done false
6594 3010 Acked Read 6 Sent Done Done false
6595 3011 Acked Read 10 Sent Done Done false
6596 3012 Acked Write 10 Sent Done Done false
6597 3013 Acked Write 10 Sent Done Done false
6598 3014 Acked Read 4 Sent Done Done false
6599 3015 Acked Read 10 Sent Done Done false
6600 3016 Acked Read 6 Sent Done Done false
6601 3017 Acked Write 8 Sent Done Done false
6602 3018 Acked Read 3 Sent Done Done false
6603 3019 Acked Read 8 Sent Done Done false
6604 3020 Acked Write 6 Sent Done Done false
6605 3021 Acked Read 3 Sent Done Done false
6606 3022 Acked Write 6 Sent Done Done false
6607 3023 Acked Read 8 Sent Done Done false
6608 3024 Acked Read 9 Sent Done Done false
6609 3025 Acked Write 3 Sent Done Done false
6610 3026 Acked Read 1 Sent Done Done false
6611 3027 Acked Write 10 Sent Done Done false
6612 3028 Acked Read 3 Sent Done Done false
6613 3029 Acked Write 3 Sent Done Done false
6614 3030 Acked Read 7 Sent Done Done false
6615 3031 Acked Read 3 Sent Done Done false
6616 3032 Acked Write 9 Sent Done Done false
6617 3033 Acked Write 6 Sent Done Done false
6618 3034 Acked Read 10 Sent Done Done false
6619 3035 Acked Write 8 Sent Done Done false
6620 3036 Acked Write 2 Sent Done Done false
6621 3037 Acked Write 8 Sent Done Done false
6622 3038 Acked Write 6 Sent Done Done false
6623 3039 Acked Write 7 Sent Done Done false
6624 3040 Acked Read 5 Sent Done Done false
6625 3041 Acked Read 3 Sent Done Done false
6626 3042 Acked Write 1 Sent Done Done false
6627 3043 Acked Read 9 Sent Done Done false
6628 3044 Acked Read 2 Sent Done Done false
6629 3045 Acked Read 10 Sent Done Done false
6630 3046 Acked Read 4 Sent Done Done false
6631 3047 Acked Write 10 Sent Done Done false
6632 3048 Acked Read 5 Sent Done Done false
6633 3049 Acked Read 10 Sent Done Done false
6634 3050 Acked Read 10 Sent Done Done false
6635 3051 Acked Write 8 Sent Done Done false
6636 3052 Acked Read 6 Sent Done Done false
6637 3053 Acked Write 1 Sent Done Done false
6638 3054 Acked Write 5 Sent Done Done false
6639 3055 Acked Flush 0 Sent Done Done false
6640 3056 Acked Write 10 Sent Done Done false
6641 3057 Acked Read 6 Sent Done Done false
6642 3058 Acked Read 5 Sent Done Done false
6643 3059 Acked Write 8 Sent Done Done false
6644 3060 Acked Read 6 Sent Done Done false
6645 3061 Acked Read 8 Sent Done Done false
6646 3062 Acked Read 6 Sent Done Done false
6647 3063 Acked Write 8 Sent Done Done false
6648 3064 Acked Read 3 Sent Done Done false
6649 3065 Acked Read 2 Sent Done Done false
6650 3066 Acked Read 4 Sent Done Done false
6651 3067 Acked Write 6 Sent Done Done false
6652 3068 Acked Flush 0 Sent Done Done false
6653 3069 Acked Write 1 Sent Done Done false
6654 3070 Acked Write 4 Sent Done Done false
6655 3071 Acked Read 5 Sent Done Done false
6656 3072 Acked Read 5 Sent Done Done false
6657 3073 Acked Flush 0 Sent Done Done false
6658 3074 Acked Write 9 Sent Done Done false
6659 3075 Acked Read 8 Sent Done Done false
6660 3076 Acked Write 4 Sent Done Done false
6661 3077 Acked Read 3 Sent Done Done false
6662 3078 Acked Read 1 Sent Done Done false
6663 3079 Acked Read 6 Sent Done Done false
6664 3080 Acked Write 6 Sent Done Done false
6665 3081 Acked Write 1 Sent Done Done false
6666 3082 Acked Write 8 Sent Done Done false
6667 3083 Acked Read 7 Sent Done Done false
6668 3084 Acked Flush 0 Sent Done Done false
6669 3085 Acked Write 10 Sent Done Done false
6670 3086 Acked Write 2 Sent Done Done false
6671 3087 Acked Flush 0 Sent Done Done false
6672 3088 Acked Flush 0 Sent Done Done false
6673 3089 Acked Write 2 Sent Done Done false
6674 3090 Acked Read 9 Sent Done Done false
6675 3091 Acked Flush 0 Sent Done Done false
6676 3092 Acked Write 6 Sent Done Done false
6677 3093 Acked Read 1 Sent Done Done false
6678 3094 Acked Read 3 Sent Done Done false
6679 3095 Acked Write 9 Sent Done Done false
6680 3096 Acked Read 4 Sent Done Done false
6681 3097 Acked Write 8 Sent Done Done false
6682 3098 Acked Write 6 Sent Done Done false
6683 3099 Acked Read 5 Sent Done Done false
6684 3100 Acked Read 2 Sent Done Done false
6685 3101 Acked Read 4 Sent Done Done false
6686 3102 Acked Read 9 Sent Done Done false
6687 3103 Acked Write 3 Sent Done Done false
6688 3104 Acked Read 2 Sent Done Done false
6689 3105 Acked Read 10 Sent Done Done false
6690 3106 Acked Write 3 Sent Done Done false
6691 3107 Acked Read 2 Sent Done Done false
6692 3108 Acked Read 3 Sent Done Done false
6693 3109 Acked Read 4 Sent Done Done false
6694 3110 Acked Read 5 Sent Done Done false
6695 3111 Acked Read 2 Sent Done Done false
6696 STATES DS:0 DS:1 DS:2 TOTAL
6697 Sent 301 0 0 301
6698 Done 0 301 301 602
6699 Skipped 0 0 0 0
6700 Error 0 0 0 0
6701 Last Flush: 2810 3091 3091
6702 Downstairs last five completed: 2810 2809 2808 2807 2806
6703 Upstairs last five completed: 3111 3110 3109 3108 3107
6704 CLIENT: Up:0 ds:301 act:2
6705 [8810] got start action
6706 [8810] I am going to restart
6707 Starting downstairs at port 8810
6708 Make output file at "/tmp/dsc/downstairs-8810.txt"
6709 [8810][0] reports Starting
6710 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1210)
6711 [8810][0] reports Running
6712 ----------------------------------------------------------------
6713 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6714 States: Offline Active Active
6715 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6716 2811 Acked Read 7 Sent Done Done false
6717 2812 Acked Read 2 Sent Done Done false
6718 2813 Acked Read 7 Sent Done Done false
6719 2814 Acked Write 4 Sent Done Done false
6720 2815 Acked Read 8 Sent Done Done false
6721 2816 Acked Read 10 Sent Done Done false
6722 2817 Acked Read 7 Sent Done Done false
6723 2818 Acked Read 6 Sent Done Done false
6724 2819 Acked Read 5 Sent Done Done false
6725 2820 Acked Read 8 Sent Done Done false
6726 2821 Acked Read 5 Sent Done Done false
6727 2822 Acked Read 5 Sent Done Done false
6728 2823 Acked Write 5 Sent Done Done false
6729 2824 Acked Write 3 Sent Done Done false
6730 2825 Acked Read 7 Sent Done Done false
6731 2826 Acked Write 1 Sent Done Done false
6732 2827 Acked Read 8 Sent Done Done false
6733 2828 Acked Read 7 Sent Done Done false
6734 2829 Acked Write 5 Sent Done Done false
6735 2830 Acked Write 10 Sent Done Done false
6736 2831 Acked Read 8 Sent Done Done false
6737 2832 Acked Read 3 Sent Done Done false
6738 2833 Acked Read 2 Sent Done Done false
6739 2834 Acked Read 5 Sent Done Done false
6740 2835 Acked Flush 0 Sent Done Done false
6741 2836 Acked Flush 0 Sent Done Done false
6742 2837 Acked Read 6 Sent Done Done false
6743 2838 Acked Read 4 Sent Done Done false
6744 2839 Acked Read 2 Sent Done Done false
6745 2840 Acked Read 9 Sent Done Done false
6746 2841 Acked Read 6 Sent Done Done false
6747 2842 Acked Flush 0 Sent Done Done false
6748 2843 Acked Write 1 Sent Done Done false
6749 2844 Acked Write 6 Sent Done Done false
6750 2845 Acked Read 1 Sent Done Done false
6751 2846 Acked Read 9 Sent Done Done false
6752 2847 Acked Read 6 Sent Done Done false
6753 2848 Acked Read 2 Sent Done Done false
6754 2849 Acked Write 10 Sent Done Done false
6755 2850 Acked Write 10 Sent Done Done false
6756 2851 Acked Write 6 Sent Done Done false
6757 2852 Acked Write 5 Sent Done Done false
6758 2853 Acked Read 4 Sent Done Done false
6759 2854 Acked Write 6 Sent Done Done false
6760 2855 Acked Flush 0 Sent Done Done false
6761 2856 Acked Write 5 Sent Done Done false
6762 2857 Acked Write 6 Sent Done Done false
6763 2858 Acked Write 2 Sent Done Done false
6764 2859 Acked Read 1 Sent Done Done false
6765 2860 Acked Read 9 Sent Done Done false
6766 2861 Acked Read 8 Sent Done Done false
6767 2862 Acked Flush 0 Sent Done Done false
6768 2863 Acked Write 1 Sent Done Done false
6769 2864 Acked Read 6 Sent Done Done false
6770 2865 Acked Read 10 Sent Done Done false
6771 2866 Acked Read 8 Sent Done Done false
6772 2867 Acked Read 3 Sent Done Done false
6773 2868 Acked Read 4 Sent Done Done false
6774 2869 Acked Write 9 Sent Done Done false
6775 2870 Acked Flush 0 Sent Done Done false
6776 2871 Acked Write 1 Sent Done Done false
6777 2872 Acked Write 7 Sent Done Done false
6778 2873 Acked Write 9 Sent Done Done false
6779 2874 Acked Read 1 Sent Done Done false
6780 2875 Acked Flush 0 Sent Done Done false
6781 2876 Acked Flush 0 Sent Done Done false
6782 2877 Acked Write 7 Sent Done Done false
6783 2878 Acked Write 6 Sent Done Done false
6784 2879 Acked Read 8 Sent Done Done false
6785 2880 Acked Read 4 Sent Done Done false
6786 2881 Acked Read 7 Sent Done Done false
6787 2882 Acked Read 10 Sent Done Done false
6788 2883 Acked Write 5 Sent Done Done false
6789 2884 Acked Write 1 Sent Done Done false
6790 2885 Acked Read 7 Sent Done Done false
6791 2886 Acked Read 9 Sent Done Done false
6792 2887 Acked Write 9 Sent Done Done false
6793 2888 Acked Read 5 Sent Done Done false
6794 2889 Acked Write 7 Sent Done Done false
6795 2890 Acked Write 9 Sent Done Done false
6796 2891 Acked Flush 0 Sent Done Done false
6797 2892 Acked Write 6 Sent Done Done false
6798 2893 Acked Flush 0 Sent Done Done false
6799 2894 Acked Write 2 Sent Done Done false
6800 2895 Acked Read 9 Sent Done Done false
6801 2896 Acked Read 3 Sent Done Done false
6802 2897 Acked Read 1 Sent Done Done false
6803 2898 Acked Read 7 Sent Done Done false
6804 2899 Acked Write 4 Sent Done Done false
6805 2900 Acked Write 2 Sent Done Done false
6806 2901 Acked Flush 0 Sent Done Done false
6807 2902 Acked Read 2 Sent Done Done false
6808 2903 Acked Read 1 Sent Done Done false
6809 2904 Acked Read 7 Sent Done Done false
6810 2905 Acked Write 2 Sent Done Done false
6811 2906 Acked Write 9 Sent Done Done false
6812 2907 Acked Write 1 Sent Done Done false
6813 2908 Acked Read 1 Sent Done Done false
6814 2909 Acked Write 7 Sent Done Done false
6815 2910 Acked Flush 0 Sent Done Done false
6816 2911 Acked Write 9 Sent Done Done false
6817 2912 Acked Read 10 Sent Done Done false
6818 2913 Acked Read 5 Sent Done Done false
6819 2914 Acked Read 6 Sent Done Done false
6820 2915 Acked Read 4 Sent Done Done false
6821 2916 Acked Read 2 Sent Done Done false
6822 2917 Acked Write 3 Sent Done Done false
6823 2918 Acked Read 4 Sent Done Done false
6824 2919 Acked Read 9 Sent Done Done false
6825 2920 Acked Write 8 Sent Done Done false
6826 2921 Acked Write 7 Sent Done Done false
6827 2922 Acked Read 6 Sent Done Done false
6828 2923 Acked Read 8 Sent Done Done false
6829 2924 Acked Write 5 Sent Done Done false
6830 2925 Acked Write 2 Sent Done Done false
6831 2926 Acked Read 10 Sent Done Done false
6832 2927 Acked Write 4 Sent Done Done false
6833 2928 Acked Read 10 Sent Done Done false
6834 2929 Acked Read 3 Sent Done Done false
6835 2930 Acked Read 8 Sent Done Done false
6836 2931 Acked Write 8 Sent Done Done false
6837 2932 Acked Read 9 Sent Done Done false
6838 2933 Acked Write 5 Sent Done Done false
6839 2934 Acked Flush 0 Sent Done Done false
6840 2935 Acked Read 2 Sent Done Done false
6841 2936 Acked Write 3 Sent Done Done false
6842 2937 Acked Write 2 Sent Done Done false
6843 2938 Acked Write 5 Sent Done Done false
6844 2939 Acked Read 4 Sent Done Done false
6845 2940 Acked Write 8 Sent Done Done false
6846 2941 Acked Write 2 Sent Done Done false
6847 2942 Acked Read 9 Sent Done Done false
6848 2943 Acked Flush 0 Sent Done Done false
6849 2944 Acked Write 1 Sent Done Done false
6850 2945 Acked Read 7 Sent Done Done false
6851 2946 Acked Write 8 Sent Done Done false
6852 2947 Acked Write 5 Sent Done Done false
6853 2948 Acked Read 2 Sent Done Done false
6854 2949 Acked Write 5 Sent Done Done false
6855 2950 Acked Read 9 Sent Done Done false
6856 2951 Acked Read 8 Sent Done Done false
6857 2952 Acked Read 10 Sent Done Done false
6858 2953 Acked Flush 0 Sent Done Done false
6859 2954 Acked Read 3 Sent Done Done false
6860 2955 Acked Write 8 Sent Done Done false
6861 2956 Acked Write 4 Sent Done Done false
6862 2957 Acked Write 9 Sent Done Done false
6863 2958 Acked Read 7 Sent Done Done false
6864 2959 Acked Read 6 Sent Done Done false
6865 2960 Acked Write 4 Sent Done Done false
6866 2961 Acked Write 8 Sent Done Done false
6867 2962 Acked Read 6 Sent Done Done false
6868 2963 Acked Read 7 Sent Done Done false
6869 2964 Acked Write 6 Sent Done Done false
6870 2965 Acked Write 7 Sent Done Done false
6871 2966 Acked Read 9 Sent Done Done false
6872 2967 Acked Write 3 Sent Done Done false
6873 2968 Acked Read 9 Sent Done Done false
6874 2969 Acked Write 1 Sent Done Done false
6875 2970 Acked Write 2 Sent Done Done false
6876 2971 Acked Read 9 Sent Done Done false
6877 2972 Acked Read 7 Sent Done Done false
6878 2973 Acked Write 2 Sent Done Done false
6879 2974 Acked Read 8 Sent Done Done false
6880 2975 Acked Write 2 Sent Done Done false
6881 2976 Acked Read 7 Sent Done Done false
6882 2977 Acked Read 3 Sent Done Done false
6883 2978 Acked Write 8 Sent Done Done false
6884 2979 Acked Write 1 Sent Done Done false
6885 2980 Acked Read 5 Sent Done Done false
6886 2981 Acked Read 4 Sent Done Done false
6887 2982 Acked Read 3 Sent Done Done false
6888 2983 Acked Read 6 Sent Done Done false
6889 2984 Acked Write 3 Sent Done Done false
6890 2985 Acked Read 5 Sent Done Done false
6891 2986 Acked Read 1 Sent Done Done false
6892 2987 Acked Write 3 Sent Done Done false
6893 2988 Acked Read 4 Sent Done Done false
6894 2989 Acked Flush 0 Sent Done Done false
6895 2990 Acked Read 1 Sent Done Done false
6896 2991 Acked Write 10 Sent Done Done false
6897 2992 Acked Write 7 Sent Done Done false
6898 2993 Acked Read 1 Sent Done Done false
6899 2994 Acked Read 2 Sent Done Done false
6900 2995 Acked Read 10 Sent Done Done false
6901 2996 Acked Read 6 Sent Done Done false
6902 2997 Acked Read 7 Sent Done Done false
6903 2998 Acked Read 8 Sent Done Done false
6904 2999 Acked Read 6 Sent Done Done false
6905 3000 Acked Write 6 Sent Done Done false
6906 3001 Acked Read 5 Sent Done Done false
6907 3002 Acked Write 4 Sent Done Done false
6908 3003 Acked Read 9 Sent Done Done false
6909 3004 Acked Write 4 Sent Done Done false
6910 3005 Acked Read 8 Sent Done Done false
6911 3006 Acked Write 2 Sent Done Done false
6912 3007 Acked Read 5 Sent Done Done false
6913 3008 Acked Write 9 Sent Done Done false
6914 3009 Acked Read 3 Sent Done Done false
6915 3010 Acked Read 6 Sent Done Done false
6916 3011 Acked Read 10 Sent Done Done false
6917 3012 Acked Write 10 Sent Done Done false
6918 3013 Acked Write 10 Sent Done Done false
6919 3014 Acked Read 4 Sent Done Done false
6920 3015 Acked Read 10 Sent Done Done false
6921 3016 Acked Read 6 Sent Done Done false
6922 3017 Acked Write 8 Sent Done Done false
6923 3018 Acked Read 3 Sent Done Done false
6924 3019 Acked Read 8 Sent Done Done false
6925 3020 Acked Write 6 Sent Done Done false
6926 3021 Acked Read 3 Sent Done Done false
6927 3022 Acked Write 6 Sent Done Done false
6928 3023 Acked Read 8 Sent Done Done false
6929 3024 Acked Read 9 Sent Done Done false
6930 3025 Acked Write 3 Sent Done Done false
6931 3026 Acked Read 1 Sent Done Done false
6932 3027 Acked Write 10 Sent Done Done false
6933 3028 Acked Read 3 Sent Done Done false
6934 3029 Acked Write 3 Sent Done Done false
6935 3030 Acked Read 7 Sent Done Done false
6936 3031 Acked Read 3 Sent Done Done false
6937 3032 Acked Write 9 Sent Done Done false
6938 3033 Acked Write 6 Sent Done Done false
6939 3034 Acked Read 10 Sent Done Done false
6940 3035 Acked Write 8 Sent Done Done false
6941 3036 Acked Write 2 Sent Done Done false
6942 3037 Acked Write 8 Sent Done Done false
6943 3038 Acked Write 6 Sent Done Done false
6944 3039 Acked Write 7 Sent Done Done false
6945 3040 Acked Read 5 Sent Done Done false
6946 3041 Acked Read 3 Sent Done Done false
6947 3042 Acked Write 1 Sent Done Done false
6948 3043 Acked Read 9 Sent Done Done false
6949 3044 Acked Read 2 Sent Done Done false
6950 3045 Acked Read 10 Sent Done Done false
6951 3046 Acked Read 4 Sent Done Done false
6952 3047 Acked Write 10 Sent Done Done false
6953 3048 Acked Read 5 Sent Done Done false
6954 3049 Acked Read 10 Sent Done Done false
6955 3050 Acked Read 10 Sent Done Done false
6956 3051 Acked Write 8 Sent Done Done false
6957 3052 Acked Read 6 Sent Done Done false
6958 3053 Acked Write 1 Sent Done Done false
6959 3054 Acked Write 5 Sent Done Done false
6960 3055 Acked Flush 0 Sent Done Done false
6961 3056 Acked Write 10 Sent Done Done false
6962 3057 Acked Read 6 Sent Done Done false
6963 3058 Acked Read 5 Sent Done Done false
6964 3059 Acked Write 8 Sent Done Done false
6965 3060 Acked Read 6 Sent Done Done false
6966 3061 Acked Read 8 Sent Done Done false
6967 3062 Acked Read 6 Sent Done Done false
6968 3063 Acked Write 8 Sent Done Done false
6969 3064 Acked Read 3 Sent Done Done false
6970 3065 Acked Read 2 Sent Done Done false
6971 3066 Acked Read 4 Sent Done Done false
6972 3067 Acked Write 6 Sent Done Done false
6973 3068 Acked Flush 0 Sent Done Done false
6974 3069 Acked Write 1 Sent Done Done false
6975 3070 Acked Write 4 Sent Done Done false
6976 3071 Acked Read 5 Sent Done Done false
6977 3072 Acked Read 5 Sent Done Done false
6978 3073 Acked Flush 0 Sent Done Done false
6979 3074 Acked Write 9 Sent Done Done false
6980 3075 Acked Read 8 Sent Done Done false
6981 3076 Acked Write 4 Sent Done Done false
6982 3077 Acked Read 3 Sent Done Done false
6983 3078 Acked Read 1 Sent Done Done false
6984 3079 Acked Read 6 Sent Done Done false
6985 3080 Acked Write 6 Sent Done Done false
6986 3081 Acked Write 1 Sent Done Done false
6987 3082 Acked Write 8 Sent Done Done false
6988 3083 Acked Read 7 Sent Done Done false
6989 3084 Acked Flush 0 Sent Done Done false
6990 3085 Acked Write 10 Sent Done Done false
6991 3086 Acked Write 2 Sent Done Done false
6992 3087 Acked Flush 0 Sent Done Done false
6993 3088 Acked Flush 0 Sent Done Done false
6994 3089 Acked Write 2 Sent Done Done false
6995 3090 Acked Read 9 Sent Done Done false
6996 3091 Acked Flush 0 Sent Done Done false
6997 3092 Acked Write 6 Sent Done Done false
6998 3093 Acked Read 1 Sent Done Done false
6999 3094 Acked Read 3 Sent Done Done false
7000 3095 Acked Write 9 Sent Done Done false
7001 3096 Acked Read 4 Sent Done Done false
7002 3097 Acked Write 8 Sent Done Done false
7003 3098 Acked Write 6 Sent Done Done false
7004 3099 Acked Read 5 Sent Done Done false
7005 3100 Acked Read 2 Sent Done Done false
7006 3101 Acked Read 4 Sent Done Done false
7007 3102 Acked Read 9 Sent Done Done false
7008 3103 Acked Write 3 Sent Done Done false
7009 3104 Acked Read 2 Sent Done Done false
7010 3105 Acked Read 10 Sent Done Done false
7011 3106 Acked Write 3 Sent Done Done false
7012 3107 Acked Read 2 Sent Done Done false
7013 3108 Acked Read 3 Sent Done Done false
7014 3109 Acked Read 4 Sent Done Done false
7015 3110 Acked Read 5 Sent Done Done false
7016 3111 Acked Read 2 Sent Done Done false
7017 3112 Acked Flush 0 Sent Done Done false
7018 STATES DS:0 DS:1 DS:2 TOTAL
7019 Sent 302 0 0 302
7020 Done 0 302 302 604
7021 Skipped 0 0 0 0
7022 Error 0 0 0 0
7023 Last Flush: 2810 3112 3112
7024 Downstairs last five completed: 2810 2809 2808 2807 2806
7025 Upstairs last five completed: 3112 3111 3110 3109 3108
7026 CLIENT: Up:0 ds:302 act:2
70272024-12-04T13:32:09.615ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70282024-12-04T13:32:09.615ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49562) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70292024-12-04T13:32:09.616ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70302024-12-04T13:32:09.617ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70312024-12-04T13:32:09.617ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70322024-12-04T13:32:09.617ZINFOcrucible: send last flush ID to this DS: 2810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70332024-12-04T13:32:09.617ZINFOcrucible: Replied this last flush ID: 2810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70342024-12-04T13:32:09.617ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70352024-12-04T13:32:09.617ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70362024-12-04T13:32:09.617ZINFOcrucible: [0] client re-new 302 jobs since flush 2810 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70372024-12-04T13:32:09.617ZINFOcrucible: [0] Replayed 302 jobs since flush: 2810 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
7038 ----------------------------------------------------------------
7039 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7040 Downstairs last five completed: 3112 3111 3110 3109 3108
7041 Upstairs last five completed: 3112 3111 3110 3109 3108
7042 CLIENT: Up:0 ds:0 act:3
7043 Replay: All jobs finished
7044 Main task has work to do, go find it
7045 got dsc Stop(1)
7046 stop 1
7047 Dec 04 13:32:11.863[8820] Got stop action so:false kr:false
7048 INFO request completed, latency_us: 325, response_code: 204, uri: /stop/cid/1, method: POST, req_id: bebc65ce-5ac4-4f65-bf1a-fc0628729da4, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
70492024-12-04T13:32:11.864ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70502024-12-04T13:32:11.864ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70512024-12-04T13:32:11.864ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70522024-12-04T13:32:11.864ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70532024-12-04T13:32:11.864ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
70542024-12-04T13:32:11.864ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
7055 Dec 04 13:32:11.864 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/1, method: GET, req_id: ed9bdaf3-390b-4c3b-a79a-b6f4ef95a82c, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
7056 [8820] Exited with: ExitStatus(unix_wait_status(9))
7057 [8820][1] reports Exit
7058 Dec 04 13:32:15.867 INFO request completed, latency_us: 221, response_code: 200, uri: /state/cid/1, method: GET, req_id: 924b9733-523a-4d22-b944-90212e43a764, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
7059 001/300 Read block 342 len 4096
7060 002/300 Read block 3699 len 24576
7061 003/300 Write block 762 len 28672 data: 3 3 2 2 3 3 3
7062 004/300 Read block 3408 len 12288
7063 005/300 Read block 4295 len 24576
7064 006/300 Read block 3125 len 20480
7065 007/300 Write block 2044 len 24576 data: 5 6 6 4 4 3
7066 008/300 Read block 515 len 8192
7067 009/300 Read block 2397 len 40960
7068 010/300 Flush
7069 011/300 Write block 2342 len 36864 data: 2 4 3 4 4 4 4 3 3
7070 012/300 Write block 207 len 24576 data: 3 3 2 2 2 2
7071 013/300 Write block 3435 len 12288 data: 4 4 4
7072 014/300 Write block 266 len 16384 data: 3 3 3 3
7073 015/300 Read block 613 len 8192
7074 016/300 Read block 3973 len 28672
7075 017/300 Write block 2174 len 36864 data: 3 2 2 2 2 2 2 2 2
7076 018/300 Read block 447 len 32768
7077 019/300 Write block 160 len 24576 data: 5 6 6 5 5 5
7078 020/300 Write block 3645 len 36864 data: 2 3 2 2 2 2 2 3 3
7079 021/300 Read block 254 len 20480
7080 022/300 Write block 4311 len 16384 data: 2 3 3 3
7081 023/300 Write block 4864 len 28672 data: 2 2 2 3 3 3 2
7082 024/300 Read block 2223 len 36864
7083 025/300 Read block 221 len 8192
7084 026/300 Read block 1078 len 28672
7085 027/300 Read block 3621 len 4096
7086 028/300 Write block 2234 len 28672 data: 2 2 2 2 2 2 2
7087 029/300 Read block 88 len 12288
7088 030/300 Write block 1887 len 40960 data: 4 6 6 6 5 4 4 3 2 3
7089 031/300 Read block 3584 len 4096
7090 032/300 Read block 2771 len 36864
7091 033/300 Read block 1713 len 40960
7092 034/300 Read block 1481 len 20480
7093 035/300 Flush
7094 036/300 Write block 833 len 20480 data: 2 2 2 2 2
7095 037/300 Read block 4858 len 32768
7096 038/300 Write block 4129 len 40960 data: 3 3 4 3 3 3 3 3 3 2
7097 039/300 Read block 1186 len 36864
7098 040/300 Flush
7099 041/300 Write block 3633 len 40960 data: 3 3 3 3 3 4 3 2 2 2
7100 042/300 Read block 2338 len 16384
7101 043/300 Write block 3846 len 16384 data: 3 3 3 3
7102 044/300 Write block 4903 len 32768 data: 4 5 4 4 4 4 4 3
7103 045/300 Read block 1877 len 20480
7104 046/300 Write block 4378 len 8192 data: 2 2
7105 047/300 Read block 4153 len 16384
7106 048/300 Read block 1698 len 36864
7107 049/300 Read block 2944 len 8192
7108 050/300 Write block 2002 len 12288 data: 2 2 3
7109 051/300 Read block 4013 len 16384
7110 052/300 Read block 2338 len 24576
7111 053/300 Write block 2878 len 20480 data: 2 2 2 2 2
7112 054/300 Write block 3537 len 36864 data: 2 2 2 2 2 2 3 3 3
7113 055/300 Read block 1492 len 24576
7114 056/300 Read block 4167 len 4096
7115 057/300 Read block 1710 len 40960
7116 058/300 Read block 74 len 36864
7117 059/300 Read block 1866 len 28672
7118 060/300 Write block 2082 len 16384 data: 2 2 2 2
7119 061/300 Read block 4784 len 20480
7120 062/300 Read block 1672 len 40960
7121 063/300 Write block 3716 len 8192 data: 4 4
7122 064/300 Flush
7123 065/300 Read block 1363 len 12288
7124 066/300 Write block 4111 len 36864 data: 4 4 4 3 2 2 2 2 3
7125 067/300 Flush
7126 068/300 Write block 769 len 20480 data: 4 5 4 3 3
7127 069/300 Read block 4452 len 24576
7128 070/300 Write block 2838 len 28672 data: 4 4 4 3 3 3 3
7129 071/300 Write block 2558 len 28672 data: 3 3 3 3 3 3 3
7130 072/300 Read block 4902 len 4096
7131 073/300 Read block 1008 len 12288
7132 074/300 Read block 4375 len 36864
7133 075/300 Write block 4985 len 20480 data: 2 2 2 2 2
7134 076/300 Write block 1650 len 32768 data: 2 3 3 3 3 3 3 3
7135 077/300 Read block 3285 len 4096
7136 078/300 Flush
7137 079/300 Write block 2857 len 24576 data: 2 2 2 2 2 2
7138 080/300 Write block 4155 len 36864 data: 5 5 5 6 8 8 8 8 7
7139 081/300 Read block 2595 len 20480
7140 082/300 Read block 33 len 20480
7141 083/300 Read block 2678 len 40960
7142 084/300 Write block 4860 len 40960 data: 2 2 2 2 3 3 3 4 4 4
7143 085/300 Write block 1420 len 28672 data: 3 3 2 2 2 3 3
7144 086/300 Write block 4690 len 36864 data: 2 2 2 2 2 2 3 3 3
7145 087/300 Write block 388 len 16384 data: 2 2 2 4
7146 088/300 Read block 1068 len 12288
7147 089/300 Write block 1881 len 4096 data: 2
7148 090/300 Flush
7149 091/300 Write block 3678 len 20480 data: 4 3 3 4 4
7150 092/300 Read block 3469 len 4096
7151 093/300 Write block 1264 len 36864 data: 3 3 3 3 3 4 3 3 3
7152 094/300 Write block 3074 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7153 095/300 Flush
7154 096/300 Read block 500 len 16384
7155 097/300 Read block 187 len 20480
7156 098/300 Flush
7157 099/300 Flush
7158 100/300 Write block 1723 len 24576 data: 2 2 2 2 2 2
7159 101/300 Read block 3875 len 4096
7160 102/300 Read block 2634 len 16384
7161 103/300 Read block 3703 len 40960
7162 104/300 Read block 4316 len 12288
7163 105/300 Write block 3724 len 8192 data: 2 2
7164 106/300 Read block 1251 len 24576
7165 107/300 Read block 3433 len 32768
7166 108/300 Read block 4265 len 12288
7167 109/300 Read block 2985 len 20480
7168 110/300 Write block 3763 len 32768 data: 3 3 3 3 2 2 2 2
7169 111/300 Write block 485 len 28672 data: 3 4 4 4 5 4 4
7170 112/300 Write block 604 len 36864 data: 3 3 2 2 2 2 2 2 2
7171 113/300 Read block 4961 len 40960
7172 114/300 Read block 1200 len 28672
7173 115/300 Read block 1079 len 32768
7174 116/300 Read block 4603 len 28672
7175 117/300 Read block 3997 len 24576
7176 118/300 Flush
7177 119/300 Write block 1022 len 32768 data: 3 4 4 4 3 3 2 2
7178 120/300 Flush
7179 121/300 Write block 3446 len 8192 data: 2 2
7180 122/300 Write block 2782 len 20480 data: 3 3 3 2 2
7181 123/300 Read block 3101 len 20480
7182 124/300 Write block 4706 len 32768 data: 2 2 2 2 2 2 2 3
7183 125/300 Read block 914 len 32768
7184 126/300 Write block 206 len 4096 data: 3
7185 127/300 Read block 4458 len 28672
7186 128/300 Write block 1424 len 40960 data: 3 4 4 3 3 2 2 2 2 3
7187 129/300 Write block 4329 len 28672 data: 2 2 3 3 3 3 3
7188 130/300 Write block 3188 len 32768 data: 2 2 2 2 3 3 3 3
7189 131/300 Flush
7190 132/300 Write block 3320 len 28672 data: 7 7 6 5 3 2 2
7191 133/300 Read block 753 len 28672
7192 134/300 Read block 2070 len 16384
7193 135/300 Write block 47 len 4096 data: 2
7194 136/300 Read block 335 len 32768
7195 137/300 Read block 662 len 12288
7196 138/300 Read block 2442 len 40960
7197 139/300 Read block 3043 len 12288
7198 140/300 Write block 4655 len 8192 data: 2 2
7199 141/300 Read block 114 len 32768
7200 142/300 Read block 2226 len 24576
7201 143/300 Write block 259 len 12288 data: 3 3 3
7202 144/300 Flush
7203 145/300 Read block 1087 len 36864
7204 146/300 Write block 1010 len 4096 data: 2
7205 147/300 Read block 253 len 24576
7206 148/300 Read block 928 len 28672
7207 149/300 Read block 2558 len 16384
7208 150/300 Write block 1232 len 20480 data: 2 2 2 3 3
7209 151/300 Read block 2573 len 36864
7210 152/300 Read block 149 len 40960
7211 153/300 Read block 4565 len 16384
7212 154/300 Read block 1331 len 28672
7213 155/300 Read block 4164 len 8192
7214 156/300 Write block 1098 len 40960 data: 3 3 3 3 3 3 3 3 3 2
7215 157/300 Write block 2760 len 32768 data: 3 3 3 3 3 3 3 3
7216 158/300 Write block 2445 len 20480 data: 2 2 2 2 2
7217 159/300 Read block 927 len 24576
7218 160/300 Read block 4437 len 16384
7219 161/300 Read block 1367 len 12288
7220 162/300 Read block 4584 len 4096
7221 163/300 Read block 1698 len 32768
7222 164/300 Write block 2483 len 28672 data: 3 3 3 2 2 2 2
7223 165/300 Write block 1511 len 28672 data: 5 5 4 4 4 4 3
7224 166/300 Read block 1255 len 28672
7225 167/300 Read block 637 len 24576
7226 168/300 Write block 2666 len 40960 data: 4 5 4 4 4 4 4 4 3 2
7227 169/300 Read block 1591 len 28672
7228 170/300 Read block 1068 len 40960
7229 171/300 Write block 799 len 8192 data: 3 3
7230 172/300 Read block 3476 len 28672
7231 173/300 Write block 3432 len 28672 data: 3 3 4 5 5 5 5
7232 174/300 Write block 2591 len 32768 data: 2 2 2 2 2 2 3 2
7233 175/300 Write block 2183 len 24576 data: 2 2 2 3 3 3
7234 176/300 Read block 445 len 40960
7235 177/300 Read block 2417 len 12288
7236 178/300 Read block 4967 len 24576
7237 179/300 Read block 4094 len 4096
7238 180/300 Read block 2641 len 16384
7239 181/300 Write block 1761 len 40960 data: 5 5 4 4 4 3 3 4 4 3
7240 182/300 Write block 243 len 40960 data: 2 2 2 2 2 2 2 2 2 3
7241 183/300 Read block 1321 len 24576
7242 184/300 Write block 505 len 28672 data: 2 2 2 2 3 3 3
7243 185/300 Flush
7244 186/300 Read block 3964 len 8192
7245 187/300 Read block 1037 len 28672
7246 188/300 Read block 3292 len 12288
7247 189/300 Write block 1831 len 32768 data: 2 2 2 2 3 4 3 3
7248 190/300 Read block 4074 len 4096
7249 191/300 Write block 4090 len 32768 data: 3 3 3 3 3 3 2 2
7250 192/300 Write block 2555 len 16384 data: 2 2 3 4
7251 193/300 Read block 381 len 32768
7252 194/300 Read block 2192 len 24576
7253 195/300 Flush
7254 196/300 Write block 1856 len 16384 data: 2 2 2 4
7255 197/300 Read block 4471 len 32768
7256 198/300 Read block 4353 len 16384
7257 199/300 Write block 821 len 16384 data: 2 2 2 2
7258 200/300 Write block 803 len 36864 data: 2 3 3 4 4 4 4 4 4
7259 201/300 Read block 4650 len 20480
7260 202/300 Read block 4161 len 32768
7261 203/300 Write block 1419 len 36864 data: 3 4 4 3 3 4 5 5 4
7262 204/300 Write block 79 len 28672 data: 2 2 2 2 2 2 2
7263 205/300 Write block 1808 len 12288 data: 4 4 4
7264 206/300 Read block 3180 len 8192
7265 207/300 Write block 2651 len 32768 data: 4 2 3 2 2 2 2 2
7266 208/300 Read block 1786 len 32768
7267 209/300 Read block 3921 len 28672
7268 210/300 Read block 4004 len 36864
7269 211/300 Read block 2499 len 16384
7270 212/300 Read block 2250 len 4096
7271 213/300 Write block 4758 len 32768 data: 3 2 2 2 2 2 2 2
7272 214/300 Write block 3752 len 36864 data: 3 3 2 2 2 2 2 2 2
7273 215/300 Read block 3984 len 8192
7274 216/300 Read block 3532 len 36864
7275 217/300 Write block 1783 len 8192 data: 3 3
7276 218/300 Read block 2328 len 20480
7277 219/300 Read block 2784 len 28672
7278 220/300 Read block 2798 len 16384
7279 221/300 Read block 4363 len 16384
7280 222/300 Write block 2756 len 8192 data: 2 2
7281 223/300 Read block 3644 len 40960
7282 224/300 Read block 2843 len 8192
7283 225/300 Read block 4734 len 32768
7284 226/300 Read block 3382 len 8192
7285 227/300 Read block 1132 len 12288
7286 228/300 Read block 691 len 28672
7287 229/300 Read block 322 len 8192
7288 230/300 Read block 980 len 40960
7289 231/300 Read block 4067 len 28672
7290 232/300 Read block 4049 len 32768
7291 233/300 Write block 4706 len 24576 data: 3 3 3 3 3 3
7292 234/300 Read block 2441 len 32768
7293 235/300 Write block 3398 len 8192 data: 3 4
7294 236/300 Read block 2200 len 20480
7295 237/300 Write block 3809 len 40960 data: 3 3 3 3 3 3 3 4 4 3
7296 238/300 Write block 3070 len 20480 data: 2 2 2 2 3
7297 239/300 Read block 3995 len 28672
7298 240/300 Read block 4231 len 24576
7299 241/300 Read block 4986 len 8192
7300 242/300 Read block 1590 len 28672
7301 243/300 Read block 839 len 12288
7302 244/300 Write block 4070 len 12288 data: 2 2 2
7303 245/300 Write block 1417 len 32768 data: 3 3 4 5 5 4 4 5
7304 246/300 Read block 4230 len 8192
7305 247/300 Write block 3296 len 24576 data: 6 6 6 6 6 6
7306 248/300 Flush
7307 249/300 Read block 821 len 32768
7308 250/300 Read block 2188 len 4096
7309 251/300 Read block 3629 len 28672
7310 252/300 Read block 3005 len 28672
7311 253/300 Read block 3108 len 40960
7312 254/300 Write block 4905 len 20480 data: 5 5 5 5 5
7313 255/300 Flush
7314 256/300 Flush
7315 257/300 Write block 4683 len 32768 data: 2 2 2 2 2 2 2 3
7316 258/300 Read block 1905 len 16384
7317 259/300 Write block 3125 len 12288 data: 2 2 2
7318 260/300 Read block 3481 len 8192
7319 261/300 Read block 4315 len 36864
7320 262/300 Read block 3976 len 32768
7321 263/300 Read block 3983 len 4096
7322 264/300 Read block 22 len 36864
7323 265/300 Flush
7324 266/300 Write block 1877 len 4096 data: 3
7325 267/300 Write block 4677 len 12288 data: 2 2 2
7326 268/300 Flush
7327 269/300 Read block 3715 len 40960
7328 270/300 Flush
7329 271/300 Read block 1621 len 4096
7330 272/300 Write block 919 len 32768 data: 3 5 4 3 3 3 3 3
7331 273/300 Flush
7332 274/300 Read block 1898 len 32768
7333 275/300 Read block 3494 len 24576
7334 276/300 Write block 3493 len 24576 data: 3 3 2 2 2 2
7335 277/300 Write block 2101 len 8192 data: 2 2
7336 278/300 Write block 77 len 24576 data: 2 2 3 3 3 3
7337 279/300 Flush
7338 280/300 Read block 4325 len 28672
7339 281/300 Flush
7340 282/300 Write block 548 len 12288 data: 3 2 2
7341 283/300 Read block 2477 len 24576
7342 284/300 Read block 1217 len 8192
7343 285/300 Write block 2340 len 8192 data: 2 2
7344 286/300 Write block 2981 len 40960 data: 3 3 3 3 3 2 2 2 2 2
7345 287/300 Flush
7346 288/300 Write block 2400 len 28672 data: 3 3 3 3 3 3 3
7347 289/300 Write block 4248 len 40960 data: 3 4 4 3 3 3 3 3 3 2
7348 290/300 Flush
7349 291/300 Read block 293 len 12288
7350 292/300 Write block 3541 len 28672 data: 3 3 4 4 4 4 4
7351 293/300 Read block 1737 len 32768
7352 294/300 Flush
7353 295/300 Read block 1948 len 8192
7354 296/300 Read block 2848 len 24576
7355 297/300 Write block 2167 len 36864 data: 5 5 4 5 4 5 4 4 3
7356 298/300 Read block 3851 len 20480
7357 299/300 Read block 1126 len 8192
7358 300/300 Read block 4175 len 20480
7359 301/300 Read block 357 len 36864
7360 Main task has work to do, go find it
7361 got dsc Start(1)
7362 start 1
7363 Dec 04 13:32:16.116 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/1, method: POST, req_id: bb164bdb-c9cb-4f03-a5ff-88fb3a25acf9, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
7364 Replay: started 1, returned:Ok(())
7365 ----------------------------------------------------------------
7366 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7367 States: Active Offline Active
7368 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7369 3113 Acked Read 1 Done Sent Done false
7370 3114 Acked Read 6 Done Sent Done false
7371 3115 Acked Write 7 Done Sent Done false
7372 3116 Acked Read 3 Done Sent Done false
7373 3117 Acked Read 6 Done Sent Done false
7374 3118 Acked Read 5 Done Sent Done false
7375 3119 Acked Write 6 Done Sent Done false
7376 3120 Acked Read 2 Done Sent Done false
7377 3121 Acked Read 10 Done Sent Done false
7378 3122 Acked Flush 0 Done Sent Done false
7379 3123 Acked Write 9 Done Sent Done false
7380 3124 Acked Write 6 Done Sent Done false
7381 3125 Acked Write 3 Done Sent Done false
7382 3126 Acked Write 4 Done Sent Done false
7383 3127 Acked Read 2 Done Sent Done false
7384 3128 Acked Read 7 Done Sent Done false
7385 3129 Acked Write 9 Done Sent Done false
7386 3130 Acked Read 8 Done Sent Done false
7387 3131 Acked Write 6 Done Sent Done false
7388 3132 Acked Write 9 Done Sent Done false
7389 3133 Acked Read 5 Done Sent Done false
7390 3134 Acked Write 4 Done Sent Done false
7391 3135 Acked Write 7 Done Sent Done false
7392 3136 Acked Read 9 Done Sent Done false
7393 3137 Acked Read 2 Done Sent Done false
7394 3138 Acked Read 7 Done Sent Done false
7395 3139 Acked Read 1 Done Sent Done false
7396 3140 Acked Write 7 Done Sent Done false
7397 3141 Acked Read 3 Done Sent Done false
7398 3142 Acked Write 10 Done Sent Done false
7399 3143 Acked Read 1 Done Sent Done false
7400 3144 Acked Read 9 Done Sent Done false
7401 3145 Acked Read 10 Done Sent Done false
7402 3146 Acked Read 5 Done Sent Done false
7403 3147 Acked Flush 0 Done Sent Done false
7404 3148 Acked Write 5 Done Sent Done false
7405 3149 Acked Read 8 Done Sent Done false
7406 3150 Acked Write 10 Done Sent Done false
7407 3151 Acked Read 9 Done Sent Done false
7408 3152 Acked Flush 0 Done Sent Done false
7409 3153 Acked Write 10 Done Sent Done false
7410 3154 Acked Read 4 Done Sent Done false
7411 3155 Acked Write 4 Done Sent Done false
7412 3156 Acked Write 8 Done Sent Done false
7413 3157 Acked Read 5 Done Sent Done false
7414 3158 Acked Write 2 Done Sent Done false
7415 3159 Acked Read 4 Done Sent Done false
7416 3160 Acked Read 9 Done Sent Done false
7417 3161 Acked Read 2 Done Sent Done false
7418 3162 Acked Write 3 Done Sent Done false
7419 3163 Acked Read 4 Done Sent Done false
7420 3164 Acked Read 6 Done Sent Done false
7421 3165 Acked Write 5 Done Sent Done false
7422 3166 Acked Write 9 Done Sent Done false
7423 3167 Acked Read 6 Done Sent Done false
7424 3168 Acked Read 1 Done Sent Done false
7425 3169 Acked Read 10 Done Sent Done false
7426 3170 Acked Read 9 Done Sent Done false
7427 3171 Acked Read 7 Done Sent Done false
7428 3172 Acked Write 4 Done Sent Done false
7429 3173 Acked Read 5 Done Sent Done false
7430 3174 Acked Read 10 Done Sent Done false
7431 3175 Acked Write 2 Done Sent Done false
7432 3176 Acked Flush 0 Done Sent Done false
7433 3177 Acked Read 3 Done Sent Done false
7434 3178 Acked Write 9 Done Sent Done false
7435 3179 Acked Flush 0 Done Sent Done false
7436 3180 Acked Write 5 Done Sent Done false
7437 3181 Acked Read 6 Done Sent Done false
7438 3182 Acked Write 7 Done Sent Done false
7439 3183 Acked Write 7 Done Sent Done false
7440 3184 Acked Read 1 Done Sent Done false
7441 3185 Acked Read 3 Done Sent Done false
7442 3186 Acked Read 9 Done Sent Done false
7443 3187 Acked Write 5 Done Sent Done false
7444 3188 Acked Write 8 Done Sent Done false
7445 3189 Acked Read 1 Done Sent Done false
7446 3190 Acked Flush 0 Done Sent Done false
7447 3191 Acked Write 6 Done Sent Done false
7448 3192 Acked Write 9 Done Sent Done false
7449 3193 Acked Read 5 Done Sent Done false
7450 3194 Acked Read 5 Done Sent Done false
7451 3195 Acked Read 10 Done Sent Done false
7452 3196 Acked Write 10 Done Sent Done false
7453 3197 Acked Write 7 Done Sent Done false
7454 3198 Acked Write 9 Done Sent Done false
7455 3199 Acked Write 4 Done Sent Done false
7456 3200 Acked Read 3 Done Sent Done false
7457 3201 Acked Write 1 Done Sent Done false
7458 3202 Acked Flush 0 Done Sent Done false
7459 3203 Acked Write 5 Done Sent Done false
7460 3204 Acked Read 1 Done Sent Done false
7461 3205 Acked Write 9 Done Sent Done false
7462 3206 Acked Write 10 Done Sent Done false
7463 3207 Acked Flush 0 Done Sent Done false
7464 3208 Acked Read 4 Done Sent Done false
7465 3209 Acked Read 5 Done Sent Done false
7466 3210 Acked Flush 0 Done Sent Done false
7467 3211 Acked Flush 0 Done Sent Done false
7468 3212 Acked Write 6 Done Sent Done false
7469 3213 Acked Read 1 Done Sent Done false
7470 3214 Acked Read 4 Done Sent Done false
7471 3215 Acked Read 10 Done Sent Done false
7472 3216 Acked Read 3 Done Sent Done false
7473 3217 Acked Write 2 Done Sent Done false
7474 3218 Acked Read 6 Done Sent Done false
7475 3219 Acked Read 8 Done Sent Done false
7476 3220 Acked Read 3 Done Sent Done false
7477 3221 Acked Read 5 Done Sent Done false
7478 3222 Acked Write 8 Done Sent Done false
7479 3223 Acked Write 7 Done Sent Done false
7480 3224 Acked Write 9 Done Sent Done false
7481 3225 Acked Read 10 Done Sent Done false
7482 3226 Acked Read 7 Done Sent Done false
7483 3227 Acked Read 8 Done Sent Done false
7484 3228 Acked Read 7 Done Sent Done false
7485 3229 Acked Read 6 Done Sent Done false
7486 3230 Acked Flush 0 Done Sent Done false
7487 3231 Acked Write 8 Done Sent Done false
7488 3232 Acked Flush 0 Done Sent Done false
7489 3233 Acked Write 2 Done Sent Done false
7490 3234 Acked Write 5 Done Sent Done false
7491 3235 Acked Read 5 Done Sent Done false
7492 3236 Acked Write 8 Done Sent Done false
7493 3237 Acked Read 8 Done Sent Done false
7494 3238 Acked Write 1 Done Sent Done false
7495 3239 Acked Read 7 Done Sent Done false
7496 3240 Acked Write 10 Done Sent Done false
7497 3241 Acked Write 7 Done Sent Done false
7498 3242 Acked Write 8 Done Sent Done false
7499 3243 Acked Flush 0 Done Sent Done false
7500 3244 Acked Write 7 Done Sent Done false
7501 3245 Acked Read 7 Done Sent Done false
7502 3246 Acked Read 4 Done Sent Done false
7503 3247 Acked Write 1 Done Sent Done false
7504 3248 Acked Read 8 Done Sent Done false
7505 3249 Acked Read 3 Done Sent Done false
7506 3250 Acked Read 10 Done Sent Done false
7507 3251 Acked Read 3 Done Sent Done false
7508 3252 Acked Write 2 Done Sent Done false
7509 3253 Acked Read 8 Done Sent Done false
7510 3254 Acked Read 6 Done Sent Done false
7511 3255 Acked Write 3 Done Sent Done false
7512 3256 Acked Flush 0 Done Sent Done false
7513 3257 Acked Read 9 Done Sent Done false
7514 3258 Acked Write 1 Done Sent Done false
7515 3259 Acked Read 6 Done Sent Done false
7516 3260 Acked Read 7 Done Sent Done false
7517 3261 Acked Read 4 Done Sent Done false
7518 3262 Acked Write 5 Done Sent Done false
7519 3263 Acked Read 9 Done Sent Done false
7520 3264 Acked Read 10 Done Sent Done false
7521 3265 Acked Read 4 Done Sent Done false
7522 3266 Acked Read 7 Done Sent Done false
7523 3267 Acked Read 2 Done Sent Done false
7524 3268 Acked Write 10 Done Sent Done false
7525 3269 Acked Write 8 Done Sent Done false
7526 3270 Acked Write 5 Done Sent Done false
7527 3271 Acked Read 6 Done Sent Done false
7528 3272 Acked Read 4 Done Sent Done false
7529 3273 Acked Read 3 Done Sent Done false
7530 3274 Acked Read 1 Done Sent Done false
7531 3275 Acked Read 8 Done Sent Done false
7532 3276 Acked Write 7 Done Sent Done false
7533 3277 Acked Write 7 Done Sent Done false
7534 3278 Acked Read 7 Done Sent Done false
7535 3279 Acked Read 6 Done Sent Done false
7536 3280 Acked Write 10 Done Sent Done false
7537 3281 Acked Read 7 Done Sent Done false
7538 3282 Acked Read 10 Done Sent Done false
7539 3283 Acked Write 2 Done Sent Done false
7540 3284 Acked Read 7 Done Sent Done false
7541 3285 Acked Write 7 Done Sent Done false
7542 3286 Acked Write 8 Done Sent Done false
7543 3287 Acked Write 6 Done Sent Done false
7544 3288 Acked Read 10 Done Sent Done false
7545 3289 Acked Read 3 Done Sent Done false
7546 3290 Acked Read 6 Done Sent Done false
7547 3291 Acked Read 1 Done Sent Done false
7548 3292 Acked Read 4 Done Sent Done false
7549 3293 Acked Write 10 Done Sent Done false
7550 3294 Acked Write 10 Done Sent Done false
7551 3295 Acked Read 6 Done Sent Done false
7552 3296 Acked Write 7 Done Sent Done false
7553 3297 Acked Flush 0 Done Sent Done false
7554 3298 Acked Read 2 Done Sent Done false
7555 3299 Acked Read 7 Done Sent Done false
7556 3300 Acked Read 3 Done Sent Done false
7557 3301 Acked Write 8 Done Sent Done false
7558 3302 Acked Read 1 Done Sent Done false
7559 3303 Acked Write 8 Done Sent Done false
7560 3304 Acked Write 4 Done Sent Done false
7561 3305 Acked Read 8 Done Sent Done false
7562 3306 Acked Read 6 Done Sent Done false
7563 3307 Acked Flush 0 Done Sent Done false
7564 3308 Acked Write 4 Done Sent Done false
7565 3309 Acked Read 8 Done Sent Done false
7566 3310 Acked Read 4 Done Sent Done false
7567 3311 Acked Write 4 Done Sent Done false
7568 3312 Acked Write 9 Done Sent Done false
7569 3313 Acked Read 5 Done Sent Done false
7570 3314 Acked Read 8 Done Sent Done false
7571 3315 Acked Write 9 Done Sent Done false
7572 3316 Acked Write 7 Done Sent Done false
7573 3317 Acked Write 3 Done Sent Done false
7574 3318 Acked Read 2 Done Sent Done false
7575 3319 Acked Write 8 Done Sent Done false
7576 3320 Acked Read 8 Done Sent Done false
7577 3321 Acked Read 7 Done Sent Done false
7578 3322 Acked Read 9 Done Sent Done false
7579 3323 Acked Read 4 Done Sent Done false
7580 3324 Acked Read 1 Done Sent Done false
7581 3325 Acked Write 8 Done Sent Done false
7582 3326 Acked Write 9 Done Sent Done false
7583 3327 Acked Read 2 Done Sent Done false
7584 3328 Acked Read 9 Done Sent Done false
7585 3329 Acked Write 2 Done Sent Done false
7586 3330 Acked Read 5 Done Sent Done false
7587 3331 Acked Read 7 Done Sent Done false
7588 3332 Acked Read 4 Done Sent Done false
7589 3333 Acked Read 4 Done Sent Done false
7590 3334 Acked Write 2 Done Sent Done false
7591 3335 Acked Read 10 Done Sent Done false
7592 3336 Acked Read 2 Done Sent Done false
7593 3337 Acked Read 8 Done Sent Done false
7594 3338 Acked Read 2 Done Sent Done false
7595 3339 Acked Read 3 Done Sent Done false
7596 3340 Acked Read 7 Done Sent Done false
7597 3341 Acked Read 2 Done Sent Done false
7598 3342 Acked Read 10 Done Sent Done false
7599 3343 Acked Read 7 Done Sent Done false
7600 3344 Acked Read 8 Done Sent Done false
7601 3345 Acked Write 6 Done Sent Done false
7602 3346 Acked Read 8 Done Sent Done false
7603 3347 Acked Write 2 Done Sent Done false
7604 3348 Acked Read 5 Done Sent Done false
7605 3349 Acked Write 10 Done Sent Done false
7606 3350 Acked Write 5 Done Sent Done false
7607 3351 Acked Read 7 Done Sent Done false
7608 3352 Acked Read 6 Done Sent Done false
7609 3353 Acked Read 2 Done Sent Done false
7610 3354 Acked Read 7 Done Sent Done false
7611 3355 Acked Read 3 Done Sent Done false
7612 3356 Acked Write 3 Done Sent Done false
7613 3357 Acked Write 8 Done Sent Done false
7614 3358 Acked Read 2 Done Sent Done false
7615 3359 Acked Write 6 Done Sent Done false
7616 3360 Acked Flush 0 Done Sent Done false
7617 3361 Acked Read 8 Done Sent Done false
7618 3362 Acked Read 1 Done Sent Done false
7619 3363 Acked Read 7 Done Sent Done false
7620 3364 Acked Read 7 Done Sent Done false
7621 3365 Acked Read 10 Done Sent Done false
7622 3366 Acked Write 5 Done Sent Done false
7623 3367 Acked Flush 0 Done Sent Done false
7624 3368 Acked Flush 0 Done Sent Done false
7625 3369 Acked Write 8 Done Sent Done false
7626 3370 Acked Read 4 Done Sent Done false
7627 3371 Acked Write 3 Done Sent Done false
7628 3372 Acked Read 2 Done Sent Done false
7629 3373 Acked Read 9 Done Sent Done false
7630 3374 Acked Read 8 Done Sent Done false
7631 3375 Acked Read 1 Done Sent Done false
7632 3376 Acked Read 9 Done Sent Done false
7633 3377 Acked Flush 0 Done Sent Done false
7634 3378 Acked Write 1 Done Sent Done false
7635 3379 Acked Write 3 Done Sent Done false
7636 3380 Acked Flush 0 Done Sent Done false
7637 3381 Acked Read 10 Done Sent Done false
7638 3382 Acked Flush 0 Done Sent Done false
7639 3383 Acked Read 1 Done Sent Done false
7640 3384 Acked Write 8 Done Sent Done false
7641 3385 Acked Flush 0 Done Sent Done false
7642 3386 Acked Read 8 Done Sent Done false
7643 3387 Acked Read 6 Done Sent Done false
7644 3388 Acked Write 6 Done Sent Done false
7645 3389 Acked Write 2 Done Sent Done false
7646 3390 Acked Write 6 Done Sent Done false
7647 3391 Acked Flush 0 Done Sent Done false
7648 3392 Acked Read 7 Done Sent Done false
7649 3393 Acked Flush 0 Done Sent Done false
7650 3394 Acked Write 3 Done Sent Done false
7651 3395 Acked Read 6 Done Sent Done false
7652 3396 Acked Read 2 Done Sent Done false
7653 3397 Acked Write 2 Done Sent Done false
7654 3398 Acked Write 10 Done Sent Done false
7655 3399 Acked Flush 0 Done Sent Done false
7656 3400 Acked Write 7 Done Sent Done false
7657 3401 Acked Write 10 Done Sent Done false
7658 3402 Acked Flush 0 Done Sent Done false
7659 3403 Acked Read 3 Done Sent Done false
7660 3404 Acked Write 7 Done Sent Done false
7661 3405 Acked Read 8 Done Sent Done false
7662 3406 Acked Flush 0 Done Sent Done false
7663 3407 Acked Read 2 Done Sent Done false
7664 3408 Acked Read 6 Done Sent Done false
7665 3409 Acked Write 9 Done Sent Done false
7666 3410 Acked Read 5 Done Sent Done false
7667 3411 Acked Read 2 Done Sent Done false
7668 3412 Acked Read 5 Done Sent Done false
7669 3413 Acked Read 9 Done Sent Done false
7670 STATES DS:0 DS:1 DS:2 TOTAL
7671 Sent 0 301 0 301
7672 Done 301 0 301 602
7673 Skipped 0 0 0 0
7674 Error 0 0 0 0
7675 Last Flush: 3406 3112 3406
7676 Downstairs last five completed: 3112 3111 3110 3109 3108
7677 Upstairs last five completed: 3413 3412 3411 3410 3409
7678 CLIENT: Up:0 ds:301 act:2
7679 [8820] got start action
7680 [8820] I am going to restart
7681 Starting downstairs at port 8820
7682 Make output file at "/tmp/dsc/downstairs-8820.txt"
7683 [8820][1] reports Starting
7684 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1211)
7685 [8820][1] reports Running
7686 ----------------------------------------------------------------
7687 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7688 States: Active Offline Active
7689 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7690 3113 Acked Read 1 Done Sent Done false
7691 3114 Acked Read 6 Done Sent Done false
7692 3115 Acked Write 7 Done Sent Done false
7693 3116 Acked Read 3 Done Sent Done false
7694 3117 Acked Read 6 Done Sent Done false
7695 3118 Acked Read 5 Done Sent Done false
7696 3119 Acked Write 6 Done Sent Done false
7697 3120 Acked Read 2 Done Sent Done false
7698 3121 Acked Read 10 Done Sent Done false
7699 3122 Acked Flush 0 Done Sent Done false
7700 3123 Acked Write 9 Done Sent Done false
7701 3124 Acked Write 6 Done Sent Done false
7702 3125 Acked Write 3 Done Sent Done false
7703 3126 Acked Write 4 Done Sent Done false
7704 3127 Acked Read 2 Done Sent Done false
7705 3128 Acked Read 7 Done Sent Done false
7706 3129 Acked Write 9 Done Sent Done false
7707 3130 Acked Read 8 Done Sent Done false
7708 3131 Acked Write 6 Done Sent Done false
7709 3132 Acked Write 9 Done Sent Done false
7710 3133 Acked Read 5 Done Sent Done false
7711 3134 Acked Write 4 Done Sent Done false
7712 3135 Acked Write 7 Done Sent Done false
7713 3136 Acked Read 9 Done Sent Done false
7714 3137 Acked Read 2 Done Sent Done false
7715 3138 Acked Read 7 Done Sent Done false
7716 3139 Acked Read 1 Done Sent Done false
7717 3140 Acked Write 7 Done Sent Done false
7718 3141 Acked Read 3 Done Sent Done false
7719 3142 Acked Write 10 Done Sent Done false
7720 3143 Acked Read 1 Done Sent Done false
7721 3144 Acked Read 9 Done Sent Done false
7722 3145 Acked Read 10 Done Sent Done false
7723 3146 Acked Read 5 Done Sent Done false
7724 3147 Acked Flush 0 Done Sent Done false
7725 3148 Acked Write 5 Done Sent Done false
7726 3149 Acked Read 8 Done Sent Done false
7727 3150 Acked Write 10 Done Sent Done false
7728 3151 Acked Read 9 Done Sent Done false
7729 3152 Acked Flush 0 Done Sent Done false
7730 3153 Acked Write 10 Done Sent Done false
7731 3154 Acked Read 4 Done Sent Done false
7732 3155 Acked Write 4 Done Sent Done false
7733 3156 Acked Write 8 Done Sent Done false
7734 3157 Acked Read 5 Done Sent Done false
7735 3158 Acked Write 2 Done Sent Done false
7736 3159 Acked Read 4 Done Sent Done false
7737 3160 Acked Read 9 Done Sent Done false
7738 3161 Acked Read 2 Done Sent Done false
7739 3162 Acked Write 3 Done Sent Done false
7740 3163 Acked Read 4 Done Sent Done false
7741 3164 Acked Read 6 Done Sent Done false
7742 3165 Acked Write 5 Done Sent Done false
7743 3166 Acked Write 9 Done Sent Done false
7744 3167 Acked Read 6 Done Sent Done false
7745 3168 Acked Read 1 Done Sent Done false
7746 3169 Acked Read 10 Done Sent Done false
7747 3170 Acked Read 9 Done Sent Done false
7748 3171 Acked Read 7 Done Sent Done false
7749 3172 Acked Write 4 Done Sent Done false
7750 3173 Acked Read 5 Done Sent Done false
7751 3174 Acked Read 10 Done Sent Done false
7752 3175 Acked Write 2 Done Sent Done false
7753 3176 Acked Flush 0 Done Sent Done false
7754 3177 Acked Read 3 Done Sent Done false
7755 3178 Acked Write 9 Done Sent Done false
7756 3179 Acked Flush 0 Done Sent Done false
7757 3180 Acked Write 5 Done Sent Done false
7758 3181 Acked Read 6 Done Sent Done false
7759 3182 Acked Write 7 Done Sent Done false
7760 3183 Acked Write 7 Done Sent Done false
7761 3184 Acked Read 1 Done Sent Done false
7762 3185 Acked Read 3 Done Sent Done false
7763 3186 Acked Read 9 Done Sent Done false
7764 3187 Acked Write 5 Done Sent Done false
7765 3188 Acked Write 8 Done Sent Done false
7766 3189 Acked Read 1 Done Sent Done false
7767 3190 Acked Flush 0 Done Sent Done false
7768 3191 Acked Write 6 Done Sent Done false
7769 3192 Acked Write 9 Done Sent Done false
7770 3193 Acked Read 5 Done Sent Done false
7771 3194 Acked Read 5 Done Sent Done false
7772 3195 Acked Read 10 Done Sent Done false
7773 3196 Acked Write 10 Done Sent Done false
7774 3197 Acked Write 7 Done Sent Done false
7775 3198 Acked Write 9 Done Sent Done false
7776 3199 Acked Write 4 Done Sent Done false
7777 3200 Acked Read 3 Done Sent Done false
7778 3201 Acked Write 1 Done Sent Done false
7779 3202 Acked Flush 0 Done Sent Done false
7780 3203 Acked Write 5 Done Sent Done false
7781 3204 Acked Read 1 Done Sent Done false
7782 3205 Acked Write 9 Done Sent Done false
7783 3206 Acked Write 10 Done Sent Done false
7784 3207 Acked Flush 0 Done Sent Done false
7785 3208 Acked Read 4 Done Sent Done false
7786 3209 Acked Read 5 Done Sent Done false
7787 3210 Acked Flush 0 Done Sent Done false
7788 3211 Acked Flush 0 Done Sent Done false
7789 3212 Acked Write 6 Done Sent Done false
7790 3213 Acked Read 1 Done Sent Done false
7791 3214 Acked Read 4 Done Sent Done false
7792 3215 Acked Read 10 Done Sent Done false
7793 3216 Acked Read 3 Done Sent Done false
7794 3217 Acked Write 2 Done Sent Done false
7795 3218 Acked Read 6 Done Sent Done false
7796 3219 Acked Read 8 Done Sent Done false
7797 3220 Acked Read 3 Done Sent Done false
7798 3221 Acked Read 5 Done Sent Done false
7799 3222 Acked Write 8 Done Sent Done false
7800 3223 Acked Write 7 Done Sent Done false
7801 3224 Acked Write 9 Done Sent Done false
7802 3225 Acked Read 10 Done Sent Done false
7803 3226 Acked Read 7 Done Sent Done false
7804 3227 Acked Read 8 Done Sent Done false
7805 3228 Acked Read 7 Done Sent Done false
7806 3229 Acked Read 6 Done Sent Done false
7807 3230 Acked Flush 0 Done Sent Done false
7808 3231 Acked Write 8 Done Sent Done false
7809 3232 Acked Flush 0 Done Sent Done false
7810 3233 Acked Write 2 Done Sent Done false
7811 3234 Acked Write 5 Done Sent Done false
7812 3235 Acked Read 5 Done Sent Done false
7813 3236 Acked Write 8 Done Sent Done false
7814 3237 Acked Read 8 Done Sent Done false
7815 3238 Acked Write 1 Done Sent Done false
7816 3239 Acked Read 7 Done Sent Done false
7817 3240 Acked Write 10 Done Sent Done false
7818 3241 Acked Write 7 Done Sent Done false
7819 3242 Acked Write 8 Done Sent Done false
7820 3243 Acked Flush 0 Done Sent Done false
7821 3244 Acked Write 7 Done Sent Done false
7822 3245 Acked Read 7 Done Sent Done false
7823 3246 Acked Read 4 Done Sent Done false
7824 3247 Acked Write 1 Done Sent Done false
7825 3248 Acked Read 8 Done Sent Done false
7826 3249 Acked Read 3 Done Sent Done false
7827 3250 Acked Read 10 Done Sent Done false
7828 3251 Acked Read 3 Done Sent Done false
7829 3252 Acked Write 2 Done Sent Done false
7830 3253 Acked Read 8 Done Sent Done false
7831 3254 Acked Read 6 Done Sent Done false
7832 3255 Acked Write 3 Done Sent Done false
7833 3256 Acked Flush 0 Done Sent Done false
7834 3257 Acked Read 9 Done Sent Done false
7835 3258 Acked Write 1 Done Sent Done false
7836 3259 Acked Read 6 Done Sent Done false
7837 3260 Acked Read 7 Done Sent Done false
7838 3261 Acked Read 4 Done Sent Done false
7839 3262 Acked Write 5 Done Sent Done false
7840 3263 Acked Read 9 Done Sent Done false
7841 3264 Acked Read 10 Done Sent Done false
7842 3265 Acked Read 4 Done Sent Done false
7843 3266 Acked Read 7 Done Sent Done false
7844 3267 Acked Read 2 Done Sent Done false
7845 3268 Acked Write 10 Done Sent Done false
7846 3269 Acked Write 8 Done Sent Done false
7847 3270 Acked Write 5 Done Sent Done false
7848 3271 Acked Read 6 Done Sent Done false
7849 3272 Acked Read 4 Done Sent Done false
7850 3273 Acked Read 3 Done Sent Done false
7851 3274 Acked Read 1 Done Sent Done false
7852 3275 Acked Read 8 Done Sent Done false
7853 3276 Acked Write 7 Done Sent Done false
7854 3277 Acked Write 7 Done Sent Done false
7855 3278 Acked Read 7 Done Sent Done false
7856 3279 Acked Read 6 Done Sent Done false
7857 3280 Acked Write 10 Done Sent Done false
7858 3281 Acked Read 7 Done Sent Done false
7859 3282 Acked Read 10 Done Sent Done false
7860 3283 Acked Write 2 Done Sent Done false
7861 3284 Acked Read 7 Done Sent Done false
7862 3285 Acked Write 7 Done Sent Done false
7863 3286 Acked Write 8 Done Sent Done false
7864 3287 Acked Write 6 Done Sent Done false
7865 3288 Acked Read 10 Done Sent Done false
7866 3289 Acked Read 3 Done Sent Done false
7867 3290 Acked Read 6 Done Sent Done false
7868 3291 Acked Read 1 Done Sent Done false
7869 3292 Acked Read 4 Done Sent Done false
7870 3293 Acked Write 10 Done Sent Done false
7871 3294 Acked Write 10 Done Sent Done false
7872 3295 Acked Read 6 Done Sent Done false
7873 3296 Acked Write 7 Done Sent Done false
7874 3297 Acked Flush 0 Done Sent Done false
7875 3298 Acked Read 2 Done Sent Done false
7876 3299 Acked Read 7 Done Sent Done false
7877 3300 Acked Read 3 Done Sent Done false
7878 3301 Acked Write 8 Done Sent Done false
7879 3302 Acked Read 1 Done Sent Done false
7880 3303 Acked Write 8 Done Sent Done false
7881 3304 Acked Write 4 Done Sent Done false
7882 3305 Acked Read 8 Done Sent Done false
7883 3306 Acked Read 6 Done Sent Done false
7884 3307 Acked Flush 0 Done Sent Done false
7885 3308 Acked Write 4 Done Sent Done false
7886 3309 Acked Read 8 Done Sent Done false
7887 3310 Acked Read 4 Done Sent Done false
7888 3311 Acked Write 4 Done Sent Done false
7889 3312 Acked Write 9 Done Sent Done false
7890 3313 Acked Read 5 Done Sent Done false
7891 3314 Acked Read 8 Done Sent Done false
7892 3315 Acked Write 9 Done Sent Done false
7893 3316 Acked Write 7 Done Sent Done false
7894 3317 Acked Write 3 Done Sent Done false
7895 3318 Acked Read 2 Done Sent Done false
7896 3319 Acked Write 8 Done Sent Done false
7897 3320 Acked Read 8 Done Sent Done false
7898 3321 Acked Read 7 Done Sent Done false
7899 3322 Acked Read 9 Done Sent Done false
7900 3323 Acked Read 4 Done Sent Done false
7901 3324 Acked Read 1 Done Sent Done false
7902 3325 Acked Write 8 Done Sent Done false
7903 3326 Acked Write 9 Done Sent Done false
7904 3327 Acked Read 2 Done Sent Done false
7905 3328 Acked Read 9 Done Sent Done false
7906 3329 Acked Write 2 Done Sent Done false
7907 3330 Acked Read 5 Done Sent Done false
7908 3331 Acked Read 7 Done Sent Done false
7909 3332 Acked Read 4 Done Sent Done false
7910 3333 Acked Read 4 Done Sent Done false
7911 3334 Acked Write 2 Done Sent Done false
7912 3335 Acked Read 10 Done Sent Done false
7913 3336 Acked Read 2 Done Sent Done false
7914 3337 Acked Read 8 Done Sent Done false
7915 3338 Acked Read 2 Done Sent Done false
7916 3339 Acked Read 3 Done Sent Done false
7917 3340 Acked Read 7 Done Sent Done false
7918 3341 Acked Read 2 Done Sent Done false
7919 3342 Acked Read 10 Done Sent Done false
7920 3343 Acked Read 7 Done Sent Done false
7921 3344 Acked Read 8 Done Sent Done false
7922 3345 Acked Write 6 Done Sent Done false
7923 3346 Acked Read 8 Done Sent Done false
7924 3347 Acked Write 2 Done Sent Done false
7925 3348 Acked Read 5 Done Sent Done false
7926 3349 Acked Write 10 Done Sent Done false
7927 3350 Acked Write 5 Done Sent Done false
7928 3351 Acked Read 7 Done Sent Done false
7929 3352 Acked Read 6 Done Sent Done false
7930 3353 Acked Read 2 Done Sent Done false
7931 3354 Acked Read 7 Done Sent Done false
7932 3355 Acked Read 3 Done Sent Done false
7933 3356 Acked Write 3 Done Sent Done false
7934 3357 Acked Write 8 Done Sent Done false
7935 3358 Acked Read 2 Done Sent Done false
7936 3359 Acked Write 6 Done Sent Done false
7937 3360 Acked Flush 0 Done Sent Done false
7938 3361 Acked Read 8 Done Sent Done false
7939 3362 Acked Read 1 Done Sent Done false
7940 3363 Acked Read 7 Done Sent Done false
7941 3364 Acked Read 7 Done Sent Done false
7942 3365 Acked Read 10 Done Sent Done false
7943 3366 Acked Write 5 Done Sent Done false
7944 3367 Acked Flush 0 Done Sent Done false
7945 3368 Acked Flush 0 Done Sent Done false
7946 3369 Acked Write 8 Done Sent Done false
7947 3370 Acked Read 4 Done Sent Done false
7948 3371 Acked Write 3 Done Sent Done false
7949 3372 Acked Read 2 Done Sent Done false
7950 3373 Acked Read 9 Done Sent Done false
7951 3374 Acked Read 8 Done Sent Done false
7952 3375 Acked Read 1 Done Sent Done false
7953 3376 Acked Read 9 Done Sent Done false
7954 3377 Acked Flush 0 Done Sent Done false
7955 3378 Acked Write 1 Done Sent Done false
7956 3379 Acked Write 3 Done Sent Done false
7957 3380 Acked Flush 0 Done Sent Done false
7958 3381 Acked Read 10 Done Sent Done false
7959 3382 Acked Flush 0 Done Sent Done false
7960 3383 Acked Read 1 Done Sent Done false
7961 3384 Acked Write 8 Done Sent Done false
7962 3385 Acked Flush 0 Done Sent Done false
7963 3386 Acked Read 8 Done Sent Done false
7964 3387 Acked Read 6 Done Sent Done false
7965 3388 Acked Write 6 Done Sent Done false
7966 3389 Acked Write 2 Done Sent Done false
7967 3390 Acked Write 6 Done Sent Done false
7968 3391 Acked Flush 0 Done Sent Done false
7969 3392 Acked Read 7 Done Sent Done false
7970 3393 Acked Flush 0 Done Sent Done false
7971 3394 Acked Write 3 Done Sent Done false
7972 3395 Acked Read 6 Done Sent Done false
7973 3396 Acked Read 2 Done Sent Done false
7974 3397 Acked Write 2 Done Sent Done false
7975 3398 Acked Write 10 Done Sent Done false
7976 3399 Acked Flush 0 Done Sent Done false
7977 3400 Acked Write 7 Done Sent Done false
7978 3401 Acked Write 10 Done Sent Done false
7979 3402 Acked Flush 0 Done Sent Done false
7980 3403 Acked Read 3 Done Sent Done false
7981 3404 Acked Write 7 Done Sent Done false
7982 3405 Acked Read 8 Done Sent Done false
7983 3406 Acked Flush 0 Done Sent Done false
7984 3407 Acked Read 2 Done Sent Done false
7985 3408 Acked Read 6 Done Sent Done false
7986 3409 Acked Write 9 Done Sent Done false
7987 3410 Acked Read 5 Done Sent Done false
7988 3411 Acked Read 2 Done Sent Done false
7989 3412 Acked Read 5 Done Sent Done false
7990 3413 Acked Read 9 Done Sent Done false
7991 3414 Acked Flush 0 Done Sent Done false
7992 STATES DS:0 DS:1 DS:2 TOTAL
7993 Sent 0 302 0 302
7994 Done 302 0 302 604
7995 Skipped 0 0 0 0
7996 Error 0 0 0 0
7997 Last Flush: 3414 3112 3414
7998 Downstairs last five completed: 3112 3111 3110 3109 3108
7999 Upstairs last five completed: 3414 3413 3412 3411 3410
8000 CLIENT: Up:0 ds:302 act:2
80012024-12-04T13:32:21.866ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80022024-12-04T13:32:21.866ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56260) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80032024-12-04T13:32:21.867ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80042024-12-04T13:32:21.867ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80052024-12-04T13:32:21.867ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80062024-12-04T13:32:21.867ZINFOcrucible: send last flush ID to this DS: 3112 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80072024-12-04T13:32:21.867ZINFOcrucible: Replied this last flush ID: 3112 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80082024-12-04T13:32:21.868ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80092024-12-04T13:32:21.868ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80102024-12-04T13:32:21.868ZINFOcrucible: [1] client re-new 302 jobs since flush 3112 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80112024-12-04T13:32:21.868ZINFOcrucible: [1] Replayed 302 jobs since flush: 3112 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
8012 ----------------------------------------------------------------
8013 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8014 Downstairs last five completed: 3414 3413 3412 3411 3410
8015 Upstairs last five completed: 3414 3413 3412 3411 3410
8016 CLIENT: Up:0 ds:0 act:3
8017 Replay: All jobs finished
8018 Main task has work to do, go find it
8019 got dsc Stop(1)
8020 stop 1
8021 Dec 04 13:32:24.121 INFO request completed[8820] Got stop action so:false kr:false
8022 , latency_us: 313, response_code: 204, uri: /stop/cid/1, method: POST, req_id: fd12c97f-f94d-40ed-bdb9-6b1f035909bf, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
80232024-12-04T13:32:24.121ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80242024-12-04T13:32:24.121ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80252024-12-04T13:32:24.122ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80262024-12-04T13:32:24.122ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80272024-12-04T13:32:24.122ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
80282024-12-04T13:32:24.122ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
8029 Dec 04 13:32:24.122 INFO request completed, latency_us: 140, response_code: 200, uri: /state/cid/1, method: GET, req_id: 179aaec4-a653-486d-a189-23b59056b953, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
8030 [8820] Exited with: ExitStatus(unix_wait_status(9))
8031 [8820][1] reports Exit
8032 Dec 04 13:32:28.124 INFO request completed, latency_us: 222, response_code: 200, uri: /state/cid/1, method: GET, req_id: b755e1d5-2c9d-4e2f-ada0-85688e0825a0, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
8033 001/300 Read block 1986 len 36864
8034 002/300 Write block 3100 len 4096 data: 2
8035 003/300 Read block 390 len 28672
8036 004/300 Read block 4124 len 24576
8037 005/300 Read block 1341 len 12288
8038 006/300 Write block 2348 len 4096 data: 5
8039 007/300 Read block 3775 len 8192
8040 008/300 Read block 443 len 12288
8041 009/300 Write block 1497 len 28672 data: 2 2 2 2 3 4 5
8042 010/300 Write block 4584 len 32768 data: 2 2 2 2 2 2 2 2
8043 011/300 Write block 3115 len 36864 data: 3 3 3 2 2 2 2 2 2
8044 012/300 Read block 1141 len 8192
8045 013/300 Read block 4596 len 20480
8046 014/300 Flush
8047 015/300 Read block 3176 len 8192
8048 016/300 Read block 4420 len 24576
8049 017/300 Write block 167 len 40960 data: 5 5 6 6 5 4 4 4 3 2
8050 018/300 Read block 4407 len 16384
8051 019/300 Read block 938 len 32768
8052 020/300 Write block 929 len 4096 data: 3
8053 021/300 Write block 3484 len 8192 data: 2 3
8054 022/300 Write block 1695 len 28672 data: 5 5 4 4 3 3 3
8055 023/300 Read block 3968 len 4096
8056 024/300 Read block 3166 len 36864
8057 025/300 Read block 2745 len 40960
8058 026/300 Write block 1142 len 40960 data: 2 2 2 2 2 3 3 3 3 3
8059 027/300 Flush
8060 028/300 Read block 1519 len 16384
8061 029/300 Flush
8062 030/300 Write block 1478 len 12288 data: 4 4 4
8063 031/300 Write block 670 len 4096 data: 3
8064 032/300 Read block 1363 len 16384
8065 033/300 Flush
8066 034/300 Read block 3376 len 40960
8067 035/300 Read block 2298 len 36864
8068 036/300 Flush
8069 037/300 Write block 2425 len 32768 data: 5 5 4 4 3 2 2 3
8070 038/300 Read block 3960 len 32768
8071 039/300 Write block 4758 len 4096 data: 4
8072 040/300 Read block 3417 len 20480
8073 041/300 Read block 828 len 28672
8074 042/300 Flush
8075 043/300 Read block 4007 len 36864
8076 044/300 Read block 1155 len 16384
8077 045/300 Read block 1853 len 4096
8078 046/300 Read block 4684 len 4096
8079 047/300 Read block 1707 len 16384
8080 048/300 Read block 4664 len 12288
8081 049/300 Write block 1112 len 28672 data: 4 4 6 6 5 6 6
8082 050/300 Flush
8083 051/300 Read block 1324 len 20480
8084 052/300 Read block 2923 len 24576
8085 053/300 Write block 1440 len 12288 data: 3 3 3
8086 054/300 Write block 4510 len 8192 data: 3 3
8087 055/300 Flush
8088 056/300 Read block 3249 len 4096
8089 057/300 Write block 1104 len 28672 data: 4 4 4 3 2 2 2
8090 058/300 Write block 2184 len 20480 data: 3 3 4 4 4
8091 059/300 Read block 773 len 28672
8092 060/300 Read block 2776 len 36864
8093 061/300 Read block 2959 len 4096
8094 062/300 Read block 384 len 8192
8095 063/300 Read block 110 len 36864
8096 064/300 Read block 2119 len 20480
8097 065/300 Flush
8098 066/300 Write block 3495 len 40960 data: 3 3 3 3 2 2 3 3 3 2
8099 067/300 Read block 4782 len 8192
8100 068/300 Write block 1446 len 12288 data: 2 2 2
8101 069/300 Read block 3209 len 28672
8102 070/300 Read block 2421 len 8192
8103 071/300 Read block 733 len 24576
8104 072/300 Write block 3859 len 40960 data: 2 3 2 2 2 2 2 2 3 3
8105 073/300 Read block 1841 len 20480
8106 074/300 Flush
8107 075/300 Write block 766 len 28672 data: 4 4 4 5 6 5 4
8108 076/300 Write block 2289 len 32768 data: 3 3 3 3 3 3 3 2
8109 077/300 Write block 4722 len 4096 data: 4
8110 078/300 Read block 1409 len 36864
8111 079/300 Read block 2320 len 24576
8112 080/300 Write block 1497 len 36864 data: 3 3 3 3 4 5 6 5 6
8113 081/300 Write block 999 len 40960 data: 2 3 3 3 3 4 4 4 4 4
8114 082/300 Read block 4270 len 32768
8115 083/300 Flush
8116 084/300 Read block 422 len 40960
8117 085/300 Read block 2357 len 12288
8118 086/300 Flush
8119 087/300 Write block 4693 len 32768 data: 3 3 3 4 4 4 3 3
8120 088/300 Read block 1020 len 24576
8121 089/300 Read block 4617 len 12288
8122 090/300 Read block 2963 len 12288
8123 091/300 Write block 3343 len 8192 data: 2 2
8124 092/300 Read block 4482 len 12288
8125 093/300 Write block 3677 len 4096 data: 3
8126 094/300 Read block 2594 len 4096
8127 095/300 Write block 2647 len 4096 data: 6
8128 096/300 Write block 4129 len 16384 data: 4 4 5 4
8129 097/300 Read block 3762 len 32768
8130 098/300 Write block 2389 len 32768 data: 3 2 2 2 2 2 3 3
8131 099/300 Read block 1715 len 4096
8132 100/300 Read block 1931 len 12288
8133 101/300 Read block 2358 len 32768
8134 102/300 Write block 4526 len 12288 data: 4 4 3
8135 103/300 Write block 4026 len 32768 data: 4 5 4 4 4 4 4 3
8136 104/300 Read block 1807 len 36864
8137 105/300 Write block 3234 len 20480 data: 2 2 3 3 3
8138 106/300 Read block 2956 len 32768
8139 107/300 Read block 1207 len 12288
8140 108/300 Read block 1491 len 8192
8141 109/300 Write block 2478 len 12288 data: 3 3 3
8142 110/300 Write block 4306 len 20480 data: 2 2 2 2 2
8143 111/300 Read block 931 len 4096
8144 112/300 Write block 3788 len 36864 data: 3 3 2 2 2 2 2 2 2
8145 113/300 Write block 731 len 28672 data: 3 3 3 2 3 3 3
8146 114/300 Write block 3138 len 24576 data: 3 2 2 2 2 2
8147 115/300 Flush
8148 116/300 Flush
8149 117/300 Write block 3125 len 28672 data: 3 3 3 2 2 2 2
8150 118/300 Read block 173 len 8192
8151 119/300 Read block 468 len 16384
8152 120/300 Write block 685 len 4096 data: 3
8153 121/300 Write block 4544 len 20480 data: 3 2 3 3 3
8154 122/300 Write block 2352 len 24576 data: 4 3 2 2 2 2
8155 123/300 Write block 2448 len 4096 data: 3
8156 124/300 Write block 3962 len 12288 data: 4 4 3
8157 125/300 Write block 3662 len 4096 data: 6
8158 126/300 Read block 4613 len 36864
8159 127/300 Read block 1357 len 8192
8160 128/300 Flush
8161 129/300 Read block 599 len 32768
8162 130/300 Read block 2873 len 20480
8163 131/300 Write block 1357 len 24576 data: 2 2 3 4 4 4
8164 132/300 Write block 4182 len 16384 data: 2 2 2 2
8165 133/300 Write block 2376 len 16384 data: 3 2 2 2
8166 134/300 Write block 3229 len 32768 data: 2 2 2 2 2 3 3 4
8167 135/300 Read block 1702 len 20480
8168 136/300 Read block 3560 len 40960
8169 137/300 Write block 4772 len 32768 data: 2 2 2 2 2 2 2 2
8170 138/300 Flush
8171 139/300 Read block 592 len 40960
8172 140/300 Read block 4182 len 24576
8173 141/300 Read block 1042 len 16384
8174 142/300 Read block 405 len 24576
8175 143/300 Read block 1296 len 4096
8176 144/300 Read block 1375 len 32768
8177 145/300 Read block 2353 len 24576
8178 146/300 Read block 858 len 20480
8179 147/300 Flush
8180 148/300 Write block 2809 len 8192 data: 2 2
8181 149/300 Flush
8182 150/300 Write block 1063 len 36864 data: 3 3 3 4 4 5 5 6 6
8183 151/300 Read block 3577 len 4096
8184 152/300 Write block 1510 len 28672 data: 4 6 6 5 5 5 5
8185 153/300 Read block 4466 len 28672
8186 154/300 Read block 4329 len 28672
8187 155/300 Write block 1671 len 40960 data: 2 3 3 4 4 4 5 6 5 5
8188 156/300 Read block 3574 len 40960
8189 157/300 Read block 2384 len 12288
8190 158/300 Write block 1057 len 12288 data: 5 5 5
8191 159/300 Write block 4456 len 12288 data: 4 4 4
8192 160/300 Read block 725 len 20480
8193 161/300 Flush
8194 162/300 Read block 2400 len 36864
8195 163/300 Read block 2072 len 24576
8196 164/300 Read block 4757 len 8192
8197 165/300 Flush
8198 166/300 Flush
8199 167/300 Write block 3179 len 20480 data: 4 3 3 3 2
8200 168/300 Flush
8201 169/300 Read block 797 len 40960
8202 170/300 Read block 2922 len 40960
8203 171/300 Read block 1585 len 40960
8204 172/300 Write block 940 len 36864 data: 3 3 3 3 2 2 2 3 3
8205 173/300 Write block 3096 len 32768 data: 2 2 2 2 3 3 3 3
8206 174/300 Write block 3010 len 40960 data: 2 2 2 2 2 2 2 2 3 3
8207 175/300 Write block 2615 len 36864 data: 2 3 3 3 3 3 3 3 2
8208 176/300 Read block 4976 len 24576
8209 177/300 Write block 1894 len 12288 data: 4 3 4
8210 178/300 Write block 3883 len 12288 data: 2 2 2
8211 179/300 Read block 1737 len 36864
8212 180/300 Read block 2938 len 36864
8213 181/300 Write block 1095 len 16384 data: 2 2 2 4
8214 182/300 Write block 3400 len 32768 data: 5 5 5 5 3 2 2 2
8215 183/300 Read block 2007 len 24576
8216 184/300 Read block 4216 len 4096
8217 185/300 Write block 3356 len 20480 data: 2 2 2 3 4
8218 186/300 Write block 805 len 40960 data: 4 5 5 5 5 5 5 3 4 4
8219 187/300 Write block 322 len 16384 data: 3 3 3 2
8220 188/300 Read block 1586 len 16384
8221 189/300 Write block 1970 len 36864 data: 2 2 2 2 3 3 3 3 2
8222 190/300 Write block 2720 len 24576 data: 2 2 2 2 2 2
8223 191/300 Write block 3084 len 4096 data: 2
8224 192/300 Write block 1470 len 24576 data: 3 3 4 4 4 4
8225 193/300 Read block 2513 len 20480
8226 194/300 Read block 3646 len 28672
8227 195/300 Write block 3775 len 36864 data: 4 4 4 4 4 4 4 3 3
8228 196/300 Write block 21 len 28672 data: 2 2 2 3 2 3 3
8229 197/300 Write block 1091 len 8192 data: 2 2
8230 198/300 Read block 3928 len 16384
8231 199/300 Read block 2545 len 24576
8232 200/300 Flush
8233 201/300 Write block 3488 len 8192 data: 3 3
8234 202/300 Read block 1641 len 12288
8235 203/300 Read block 4868 len 36864
8236 204/300 Read block 4060 len 8192
8237 205/300 Write block 3878 len 32768 data: 3 3 3 2 2 3 3 3
8238 206/300 Flush
8239 207/300 Read block 2531 len 32768
8240 208/300 Write block 2865 len 40960 data: 2 2 2 3 3 3 3 3 3 3
8241 209/300 Write block 1138 len 16384 data: 3 2 2 2
8242 210/300 Read block 3292 len 40960
8243 211/300 Write block 3126 len 40960 data: 4 4 3 3 3 3 2 3 3 3
8244 212/300 Read block 3454 len 28672
8245 213/300 Read block 2926 len 36864
8246 214/300 Read block 3912 len 24576
8247 215/300 Read block 3767 len 8192
8248 216/300 Write block 662 len 36864 data: 2 2 2 2 2 2 2 3 4
8249 217/300 Flush
8250 218/300 Write block 642 len 24576 data: 2 2 2 2 2 2
8251 219/300 Read block 3742 len 12288
8252 220/300 Read block 84 len 16384
8253 221/300 Read block 3962 len 36864
8254 222/300 Write block 2474 len 12288 data: 3 3 2
8255 223/300 Write block 352 len 12288 data: 4 3 3
8256 224/300 Read block 1635 len 40960
8257 225/300 Flush
8258 226/300 Write block 3765 len 28672 data: 4 4 3 3 3 3 3
8259 227/300 Write block 3936 len 24576 data: 4 4 4 4 4 5
8260 228/300 Read block 450 len 36864
8261 229/300 Read block 3395 len 36864
8262 230/300 Read block 2095 len 4096
8263 231/300 Write block 1942 len 20480 data: 3 3 3 5 4
8264 232/300 Read block 2766 len 40960
8265 233/300 Read block 1745 len 16384
8266 234/300 Write block 867 len 8192 data: 2 2
8267 235/300 Read block 531 len 8192
8268 236/300 Read block 4675 len 8192
8269 237/300 Read block 2011 len 40960
8270 238/300 Read block 111 len 16384
8271 239/300 Read block 1387 len 24576
8272 240/300 Read block 4082 len 20480
8273 241/300 Read block 4467 len 40960
8274 242/300 Flush
8275 243/300 Read block 4159 len 24576
8276 244/300 Read block 2522 len 20480
8277 245/300 Read block 797 len 36864
8278 246/300 Write block 738 len 8192 data: 4 4
8279 247/300 Flush
8280 248/300 Write block 1220 len 28672 data: 2 2 2 2 2 2 2
8281 249/300 Write block 2483 len 8192 data: 4 4
8282 250/300 Write block 983 len 24576 data: 3 3 4 4 2 2
8283 251/300 Read block 274 len 16384
8284 252/300 Read block 3033 len 8192
8285 253/300 Write block 7 len 24576 data: 5 4 4 3 4 5
8286 254/300 Read block 2031 len 16384
8287 255/300 Read block 877 len 12288
8288 256/300 Write block 2794 len 20480 data: 3 3 2 2 3
8289 257/300 Read block 4649 len 8192
8290 258/300 Read block 534 len 36864
8291 259/300 Write block 1474 len 12288 data: 5 5 4
8292 260/300 Write block 403 len 12288 data: 2 3 3
8293 261/300 Read block 3672 len 24576
8294 262/300 Write block 4674 len 28672 data: 2 2 2 3 3 3 2
8295 263/300 Write block 2600 len 28672 data: 2 3 3 3 2 2 2
8296 264/300 Read block 49 len 4096
8297 265/300 Read block 985 len 20480
8298 266/300 Flush
8299 267/300 Write block 4259 len 32768 data: 2 2 2 2 2 2 2 2
8300 268/300 Write block 1374 len 4096 data: 3
8301 269/300 Read block 2510 len 16384
8302 270/300 Read block 3217 len 12288
8303 271/300 Flush
8304 272/300 Read block 3973 len 32768
8305 273/300 Read block 4380 len 40960
8306 274/300 Write block 4706 len 4096 data: 4
8307 275/300 Flush
8308 276/300 Read block 2177 len 4096
8309 277/300 Write block 159 len 28672 data: 4 6 7 7 6 6 6
8310 278/300 Read block 2604 len 20480
8311 279/300 Write block 247 len 40960 data: 3 3 3 3 3 4 3 3 3 3
8312 280/300 Read block 1949 len 16384
8313 281/300 Write block 2921 len 36864 data: 3 3 3 3 4 4 4 3 3
8314 282/300 Write block 4702 len 28672 data: 3 3 4 3 5 4 4
8315 283/300 Read block 102 len 32768
8316 284/300 Flush
8317 285/300 Read block 3306 len 36864
8318 286/300 Write block 3160 len 4096 data: 3
8319 287/300 Read block 160 len 36864
8320 288/300 Read block 3222 len 40960
8321 289/300 Read block 3157 len 12288
8322 290/300 Write block 2670 len 8192 data: 5 5
8323 291/300 Read block 4748 len 28672
8324 292/300 Write block 4890 len 12288 data: 4 4 4
8325 293/300 Write block 53 len 12288 data: 2 2 3
8326 294/300 Write block 1140 len 16384 data: 3 3 3 3
8327 295/300 Write block 4812 len 32768 data: 2 2 2 2 2 2 2 2
8328 296/300 Read block 57 len 12288
8329 297/300 Read block 798 len 12288
8330 298/300 Read block 3660 len 20480
8331 299/300 Read block 3683 len 40960
8332 300/300 Read block 1408 len 36864
8333 301/300 Write block 1471 len 4096 data: 4
8334 Main task has work to do, go find it
8335 got dsc Start(1)
8336 start 1
8337 Dec 04 13:32:28.360 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/1, method: POST, req_id: be883f95-f253-4711-af9f-9901af1aafbc, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
8338 Replay: started 1, returned:Ok(())
8339 ----------------------------------------------------------------
8340 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8341 States: Active Offline Active
8342 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8343 3415 Acked Read 9 Done Sent Done false
8344 3416 Acked Write 1 Done Sent Done false
8345 3417 Acked Read 7 Done Sent Done false
8346 3418 Acked Read 6 Done Sent Done false
8347 3419 Acked Read 3 Done Sent Done false
8348 3420 Acked Write 1 Done Sent Done false
8349 3421 Acked Read 2 Done Sent Done false
8350 3422 Acked Read 3 Done Sent Done false
8351 3423 Acked Write 7 Done Sent Done false
8352 3424 Acked Write 8 Done Sent Done false
8353 3425 Acked Write 9 Done Sent Done false
8354 3426 Acked Read 2 Done Sent Done false
8355 3427 Acked Read 5 Done Sent Done false
8356 3428 Acked Flush 0 Done Sent Done false
8357 3429 Acked Read 2 Done Sent Done false
8358 3430 Acked Read 6 Done Sent Done false
8359 3431 Acked Write 10 Done Sent Done false
8360 3432 Acked Read 4 Done Sent Done false
8361 3433 Acked Read 8 Done Sent Done false
8362 3434 Acked Write 1 Done Sent Done false
8363 3435 Acked Write 2 Done Sent Done false
8364 3436 Acked Write 7 Done Sent Done false
8365 3437 Acked Read 1 Done Sent Done false
8366 3438 Acked Read 9 Done Sent Done false
8367 3439 Acked Read 10 Done Sent Done false
8368 3440 Acked Write 10 Done Sent Done false
8369 3441 Acked Flush 0 Done Sent Done false
8370 3442 Acked Read 4 Done Sent Done false
8371 3443 Acked Flush 0 Done Sent Done false
8372 3444 Acked Write 3 Done Sent Done false
8373 3445 Acked Write 1 Done Sent Done false
8374 3446 Acked Read 4 Done Sent Done false
8375 3447 Acked Flush 0 Done Sent Done false
8376 3448 Acked Read 10 Done Sent Done false
8377 3449 Acked Read 9 Done Sent Done false
8378 3450 Acked Flush 0 Done Sent Done false
8379 3451 Acked Write 8 Done Sent Done false
8380 3452 Acked Read 8 Done Sent Done false
8381 3453 Acked Write 1 Done Sent Done false
8382 3454 Acked Read 5 Done Sent Done false
8383 3455 Acked Read 7 Done Sent Done false
8384 3456 Acked Flush 0 Done Sent Done false
8385 3457 Acked Read 9 Done Sent Done false
8386 3458 Acked Read 4 Done Sent Done false
8387 3459 Acked Read 1 Done Sent Done false
8388 3460 Acked Read 1 Done Sent Done false
8389 3461 Acked Read 4 Done Sent Done false
8390 3462 Acked Read 3 Done Sent Done false
8391 3463 Acked Write 7 Done Sent Done false
8392 3464 Acked Flush 0 Done Sent Done false
8393 3465 Acked Read 5 Done Sent Done false
8394 3466 Acked Read 6 Done Sent Done false
8395 3467 Acked Write 3 Done Sent Done false
8396 3468 Acked Write 2 Done Sent Done false
8397 3469 Acked Flush 0 Done Sent Done false
8398 3470 Acked Read 1 Done Sent Done false
8399 3471 Acked Write 7 Done Sent Done false
8400 3472 Acked Write 5 Done Sent Done false
8401 3473 Acked Read 7 Done Sent Done false
8402 3474 Acked Read 9 Done Sent Done false
8403 3475 Acked Read 1 Done Sent Done false
8404 3476 Acked Read 2 Done Sent Done false
8405 3477 Acked Read 9 Done Sent Done false
8406 3478 Acked Read 5 Done Sent Done false
8407 3479 Acked Flush 0 Done Sent Done false
8408 3480 Acked Write 10 Done Sent Done false
8409 3481 Acked Read 2 Done Sent Done false
8410 3482 Acked Write 3 Done Sent Done false
8411 3483 Acked Read 7 Done Sent Done false
8412 3484 Acked Read 2 Done Sent Done false
8413 3485 Acked Read 6 Done Sent Done false
8414 3486 Acked Write 10 Done Sent Done false
8415 3487 Acked Read 5 Done Sent Done false
8416 3488 Acked Flush 0 Done Sent Done false
8417 3489 Acked Write 7 Done Sent Done false
8418 3490 Acked Write 8 Done Sent Done false
8419 3491 Acked Write 1 Done Sent Done false
8420 3492 Acked Read 9 Done Sent Done false
8421 3493 Acked Read 6 Done Sent Done false
8422 3494 Acked Write 9 Done Sent Done false
8423 3495 Acked Write 10 Done Sent Done false
8424 3496 Acked Read 8 Done Sent Done false
8425 3497 Acked Flush 0 Done Sent Done false
8426 3498 Acked Read 10 Done Sent Done false
8427 3499 Acked Read 3 Done Sent Done false
8428 3500 Acked Flush 0 Done Sent Done false
8429 3501 Acked Write 8 Done Sent Done false
8430 3502 Acked Read 6 Done Sent Done false
8431 3503 Acked Read 3 Done Sent Done false
8432 3504 Acked Read 3 Done Sent Done false
8433 3505 Acked Write 2 Done Sent Done false
8434 3506 Acked Read 3 Done Sent Done false
8435 3507 Acked Write 1 Done Sent Done false
8436 3508 Acked Read 1 Done Sent Done false
8437 3509 Acked Write 1 Done Sent Done false
8438 3510 Acked Write 4 Done Sent Done false
8439 3511 Acked Read 8 Done Sent Done false
8440 3512 Acked Write 8 Done Sent Done false
8441 3513 Acked Read 1 Done Sent Done false
8442 3514 Acked Read 3 Done Sent Done false
8443 3515 Acked Read 8 Done Sent Done false
8444 3516 Acked Write 3 Done Sent Done false
8445 3517 Acked Write 8 Done Sent Done false
8446 3518 Acked Read 9 Done Sent Done false
8447 3519 Acked Write 5 Done Sent Done false
8448 3520 Acked Read 8 Done Sent Done false
8449 3521 Acked Read 3 Done Sent Done false
8450 3522 Acked Read 2 Done Sent Done false
8451 3523 Acked Write 3 Done Sent Done false
8452 3524 Acked Write 5 Done Sent Done false
8453 3525 Acked Read 1 Done Sent Done false
8454 3526 Acked Write 9 Done Sent Done false
8455 3527 Acked Write 7 Done Sent Done false
8456 3528 Acked Write 6 Done Sent Done false
8457 3529 Acked Flush 0 Done Sent Done false
8458 3530 Acked Flush 0 Done Sent Done false
8459 3531 Acked Write 7 Done Sent Done false
8460 3532 Acked Read 2 Done Sent Done false
8461 3533 Acked Read 4 Done Sent Done false
8462 3534 Acked Write 1 Done Sent Done false
8463 3535 Acked Write 5 Done Sent Done false
8464 3536 Acked Write 6 Done Sent Done false
8465 3537 Acked Write 1 Done Sent Done false
8466 3538 Acked Write 3 Done Sent Done false
8467 3539 Acked Write 1 Done Sent Done false
8468 3540 Acked Read 9 Done Sent Done false
8469 3541 Acked Read 2 Done Sent Done false
8470 3542 Acked Flush 0 Done Sent Done false
8471 3543 Acked Read 8 Done Sent Done false
8472 3544 Acked Read 5 Done Sent Done false
8473 3545 Acked Write 6 Done Sent Done false
8474 3546 Acked Write 4 Done Sent Done false
8475 3547 Acked Write 4 Done Sent Done false
8476 3548 Acked Write 8 Done Sent Done false
8477 3549 Acked Read 5 Done Sent Done false
8478 3550 Acked Read 10 Done Sent Done false
8479 3551 Acked Write 8 Done Sent Done false
8480 3552 Acked Flush 0 Done Sent Done false
8481 3553 Acked Read 10 Done Sent Done false
8482 3554 Acked Read 6 Done Sent Done false
8483 3555 Acked Read 4 Done Sent Done false
8484 3556 Acked Read 6 Done Sent Done false
8485 3557 Acked Read 1 Done Sent Done false
8486 3558 Acked Read 8 Done Sent Done false
8487 3559 Acked Read 6 Done Sent Done false
8488 3560 Acked Read 5 Done Sent Done false
8489 3561 Acked Flush 0 Done Sent Done false
8490 3562 Acked Write 2 Done Sent Done false
8491 3563 Acked Flush 0 Done Sent Done false
8492 3564 Acked Write 9 Done Sent Done false
8493 3565 Acked Read 1 Done Sent Done false
8494 3566 Acked Write 7 Done Sent Done false
8495 3567 Acked Read 7 Done Sent Done false
8496 3568 Acked Read 7 Done Sent Done false
8497 3569 Acked Write 10 Done Sent Done false
8498 3570 Acked Read 10 Done Sent Done false
8499 3571 Acked Read 3 Done Sent Done false
8500 3572 Acked Write 3 Done Sent Done false
8501 3573 Acked Write 3 Done Sent Done false
8502 3574 Acked Read 5 Done Sent Done false
8503 3575 Acked Flush 0 Done Sent Done false
8504 3576 Acked Read 9 Done Sent Done false
8505 3577 Acked Read 6 Done Sent Done false
8506 3578 Acked Read 2 Done Sent Done false
8507 3579 Acked Flush 0 Done Sent Done false
8508 3580 Acked Flush 0 Done Sent Done false
8509 3581 Acked Write 5 Done Sent Done false
8510 3582 Acked Flush 0 Done Sent Done false
8511 3583 Acked Read 10 Done Sent Done false
8512 3584 Acked Read 10 Done Sent Done false
8513 3585 Acked Read 10 Done Sent Done false
8514 3586 Acked Write 9 Done Sent Done false
8515 3587 Acked Write 8 Done Sent Done false
8516 3588 Acked Write 10 Done Sent Done false
8517 3589 Acked Write 9 Done Sent Done false
8518 3590 Acked Read 6 Done Sent Done false
8519 3591 Acked Write 3 Done Sent Done false
8520 3592 Acked Write 3 Done Sent Done false
8521 3593 Acked Read 9 Done Sent Done false
8522 3594 Acked Read 9 Done Sent Done false
8523 3595 Acked Write 4 Done Sent Done false
8524 3596 Acked Write 8 Done Sent Done false
8525 3597 Acked Read 6 Done Sent Done false
8526 3598 Acked Read 1 Done Sent Done false
8527 3599 Acked Write 5 Done Sent Done false
8528 3600 Acked Write 10 Done Sent Done false
8529 3601 Acked Write 4 Done Sent Done false
8530 3602 Acked Read 4 Done Sent Done false
8531 3603 Acked Write 9 Done Sent Done false
8532 3604 Acked Write 6 Done Sent Done false
8533 3605 Acked Write 1 Done Sent Done false
8534 3606 Acked Write 6 Done Sent Done false
8535 3607 Acked Read 5 Done Sent Done false
8536 3608 Acked Read 7 Done Sent Done false
8537 3609 Acked Write 9 Done Sent Done false
8538 3610 Acked Write 7 Done Sent Done false
8539 3611 Acked Write 2 Done Sent Done false
8540 3612 Acked Read 4 Done Sent Done false
8541 3613 Acked Read 6 Done Sent Done false
8542 3614 Acked Flush 0 Done Sent Done false
8543 3615 Acked Write 2 Done Sent Done false
8544 3616 Acked Read 3 Done Sent Done false
8545 3617 Acked Read 9 Done Sent Done false
8546 3618 Acked Read 2 Done Sent Done false
8547 3619 Acked Write 8 Done Sent Done false
8548 3620 Acked Flush 0 Done Sent Done false
8549 3621 Acked Read 8 Done Sent Done false
8550 3622 Acked Write 10 Done Sent Done false
8551 3623 Acked Write 4 Done Sent Done false
8552 3624 Acked Read 10 Done Sent Done false
8553 3625 Acked Write 10 Done Sent Done false
8554 3626 Acked Read 7 Done Sent Done false
8555 3627 Acked Read 9 Done Sent Done false
8556 3628 Acked Read 6 Done Sent Done false
8557 3629 Acked Read 2 Done Sent Done false
8558 3630 Acked Write 9 Done Sent Done false
8559 3631 Acked Flush 0 Done Sent Done false
8560 3632 Acked Write 6 Done Sent Done false
8561 3633 Acked Read 3 Done Sent Done false
8562 3634 Acked Read 4 Done Sent Done false
8563 3635 Acked Read 9 Done Sent Done false
8564 3636 Acked Write 3 Done Sent Done false
8565 3637 Acked Write 3 Done Sent Done false
8566 3638 Acked Read 10 Done Sent Done false
8567 3639 Acked Flush 0 Done Sent Done false
8568 3640 Acked Write 7 Done Sent Done false
8569 3641 Acked Write 6 Done Sent Done false
8570 3642 Acked Read 9 Done Sent Done false
8571 3643 Acked Read 9 Done Sent Done false
8572 3644 Acked Read 1 Done Sent Done false
8573 3645 Acked Write 5 Done Sent Done false
8574 3646 Acked Read 10 Done Sent Done false
8575 3647 Acked Read 4 Done Sent Done false
8576 3648 Acked Write 2 Done Sent Done false
8577 3649 Acked Read 2 Done Sent Done false
8578 3650 Acked Read 2 Done Sent Done false
8579 3651 Acked Read 10 Done Sent Done false
8580 3652 Acked Read 4 Done Sent Done false
8581 3653 Acked Read 6 Done Sent Done false
8582 3654 Acked Read 5 Done Sent Done false
8583 3655 Acked Read 10 Done Sent Done false
8584 3656 Acked Flush 0 Done Sent Done false
8585 3657 Acked Read 6 Done Sent Done false
8586 3658 Acked Read 5 Done Sent Done false
8587 3659 Acked Read 9 Done Sent Done false
8588 3660 Acked Write 2 Done Sent Done false
8589 3661 Acked Flush 0 Done Sent Done false
8590 3662 Acked Write 7 Done Sent Done false
8591 3663 Acked Write 2 Done Sent Done false
8592 3664 Acked Write 6 Done Sent Done false
8593 3665 Acked Read 4 Done Sent Done false
8594 3666 Acked Read 2 Done Sent Done false
8595 3667 Acked Write 6 Done Sent Done false
8596 3668 Acked Read 4 Done Sent Done false
8597 3669 Acked Read 3 Done Sent Done false
8598 3670 Acked Write 5 Done Sent Done false
8599 3671 Acked Read 2 Done Sent Done false
8600 3672 Acked Read 9 Done Sent Done false
8601 3673 Acked Write 3 Done Sent Done false
8602 3674 Acked Write 3 Done Sent Done false
8603 3675 Acked Read 6 Done Sent Done false
8604 3676 Acked Write 7 Done Sent Done false
8605 3677 Acked Write 7 Done Sent Done false
8606 3678 Acked Read 1 Done Sent Done false
8607 3679 Acked Read 5 Done Sent Done false
8608 3680 Acked Flush 0 Done Sent Done false
8609 3681 Acked Write 8 Done Sent Done false
8610 3682 Acked Write 1 Done Sent Done false
8611 3683 Acked Read 4 Done Sent Done false
8612 3684 Acked Read 3 Done Sent Done false
8613 3685 Acked Flush 0 Done Sent Done false
8614 3686 Acked Read 8 Done Sent Done false
8615 3687 Acked Read 10 Done Sent Done false
8616 3688 Acked Write 1 Done Sent Done false
8617 3689 Acked Flush 0 Done Sent Done false
8618 3690 Acked Read 1 Done Sent Done false
8619 3691 Acked Write 7 Done Sent Done false
8620 3692 Acked Read 5 Done Sent Done false
8621 3693 Acked Write 10 Done Sent Done false
8622 3694 Acked Read 4 Done Sent Done false
8623 3695 Acked Write 9 Done Sent Done false
8624 3696 Acked Write 7 Done Sent Done false
8625 3697 Acked Read 8 Done Sent Done false
8626 3698 Acked Flush 0 Done Sent Done false
8627 3699 Acked Read 9 Done Sent Done false
8628 3700 Acked Write 1 Done Sent Done false
8629 3701 Acked Read 9 Done Sent Done false
8630 3702 Acked Read 10 Done Sent Done false
8631 3703 Acked Read 3 Done Sent Done false
8632 3704 Acked Write 2 Done Sent Done false
8633 3705 Acked Read 7 Done Sent Done false
8634 3706 Acked Write 3 Done Sent Done false
8635 3707 Acked Write 3 Done Sent Done false
8636 3708 Acked Write 4 Done Sent Done false
8637 3709 Acked Write 8 Done Sent Done false
8638 3710 Acked Read 3 Done Sent Done false
8639 3711 Acked Read 3 Done Sent Done false
8640 3712 Acked Read 5 Done Sent Done false
8641 3713 Acked Read 10 Done Sent Done false
8642 3714 Acked Read 9 Done Sent Done false
8643 3715 Acked Write 1 Done Sent Done false
8644 STATES DS:0 DS:1 DS:2 TOTAL
8645 Sent 0 301 0 301
8646 Done 301 0 301 602
8647 Skipped 0 0 0 0
8648 Error 0 0 0 0
8649 Last Flush: 3698 3414 3698
8650 Downstairs last five completed: 3414 3413 3412 3411 3410
8651 Upstairs last five completed: 3715 3714 3713 3712 3711
8652 CLIENT: Up:0 ds:301 act:2
8653 [8820] got start action
8654 [8820] I am going to restart
8655 Starting downstairs at port 8820
8656 Make output file at "/tmp/dsc/downstairs-8820.txt"
8657 [8820][1] reports Starting
8658 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
8659 [8820][1] reports Running
8660 ----------------------------------------------------------------
8661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8662 States: Active Offline Active
8663 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8664 3415 Acked Read 9 Done Sent Done false
8665 3416 Acked Write 1 Done Sent Done false
8666 3417 Acked Read 7 Done Sent Done false
8667 3418 Acked Read 6 Done Sent Done false
8668 3419 Acked Read 3 Done Sent Done false
8669 3420 Acked Write 1 Done Sent Done false
8670 3421 Acked Read 2 Done Sent Done false
8671 3422 Acked Read 3 Done Sent Done false
8672 3423 Acked Write 7 Done Sent Done false
8673 3424 Acked Write 8 Done Sent Done false
8674 3425 Acked Write 9 Done Sent Done false
8675 3426 Acked Read 2 Done Sent Done false
8676 3427 Acked Read 5 Done Sent Done false
8677 3428 Acked Flush 0 Done Sent Done false
8678 3429 Acked Read 2 Done Sent Done false
8679 3430 Acked Read 6 Done Sent Done false
8680 3431 Acked Write 10 Done Sent Done false
8681 3432 Acked Read 4 Done Sent Done false
8682 3433 Acked Read 8 Done Sent Done false
8683 3434 Acked Write 1 Done Sent Done false
8684 3435 Acked Write 2 Done Sent Done false
8685 3436 Acked Write 7 Done Sent Done false
8686 3437 Acked Read 1 Done Sent Done false
8687 3438 Acked Read 9 Done Sent Done false
8688 3439 Acked Read 10 Done Sent Done false
8689 3440 Acked Write 10 Done Sent Done false
8690 3441 Acked Flush 0 Done Sent Done false
8691 3442 Acked Read 4 Done Sent Done false
8692 3443 Acked Flush 0 Done Sent Done false
8693 3444 Acked Write 3 Done Sent Done false
8694 3445 Acked Write 1 Done Sent Done false
8695 3446 Acked Read 4 Done Sent Done false
8696 3447 Acked Flush 0 Done Sent Done false
8697 3448 Acked Read 10 Done Sent Done false
8698 3449 Acked Read 9 Done Sent Done false
8699 3450 Acked Flush 0 Done Sent Done false
8700 3451 Acked Write 8 Done Sent Done false
8701 3452 Acked Read 8 Done Sent Done false
8702 3453 Acked Write 1 Done Sent Done false
8703 3454 Acked Read 5 Done Sent Done false
8704 3455 Acked Read 7 Done Sent Done false
8705 3456 Acked Flush 0 Done Sent Done false
8706 3457 Acked Read 9 Done Sent Done false
8707 3458 Acked Read 4 Done Sent Done false
8708 3459 Acked Read 1 Done Sent Done false
8709 3460 Acked Read 1 Done Sent Done false
8710 3461 Acked Read 4 Done Sent Done false
8711 3462 Acked Read 3 Done Sent Done false
8712 3463 Acked Write 7 Done Sent Done false
8713 3464 Acked Flush 0 Done Sent Done false
8714 3465 Acked Read 5 Done Sent Done false
8715 3466 Acked Read 6 Done Sent Done false
8716 3467 Acked Write 3 Done Sent Done false
8717 3468 Acked Write 2 Done Sent Done false
8718 3469 Acked Flush 0 Done Sent Done false
8719 3470 Acked Read 1 Done Sent Done false
8720 3471 Acked Write 7 Done Sent Done false
8721 3472 Acked Write 5 Done Sent Done false
8722 3473 Acked Read 7 Done Sent Done false
8723 3474 Acked Read 9 Done Sent Done false
8724 3475 Acked Read 1 Done Sent Done false
8725 3476 Acked Read 2 Done Sent Done false
8726 3477 Acked Read 9 Done Sent Done false
8727 3478 Acked Read 5 Done Sent Done false
8728 3479 Acked Flush 0 Done Sent Done false
8729 3480 Acked Write 10 Done Sent Done false
8730 3481 Acked Read 2 Done Sent Done false
8731 3482 Acked Write 3 Done Sent Done false
8732 3483 Acked Read 7 Done Sent Done false
8733 3484 Acked Read 2 Done Sent Done false
8734 3485 Acked Read 6 Done Sent Done false
8735 3486 Acked Write 10 Done Sent Done false
8736 3487 Acked Read 5 Done Sent Done false
8737 3488 Acked Flush 0 Done Sent Done false
8738 3489 Acked Write 7 Done Sent Done false
8739 3490 Acked Write 8 Done Sent Done false
8740 3491 Acked Write 1 Done Sent Done false
8741 3492 Acked Read 9 Done Sent Done false
8742 3493 Acked Read 6 Done Sent Done false
8743 3494 Acked Write 9 Done Sent Done false
8744 3495 Acked Write 10 Done Sent Done false
8745 3496 Acked Read 8 Done Sent Done false
8746 3497 Acked Flush 0 Done Sent Done false
8747 3498 Acked Read 10 Done Sent Done false
8748 3499 Acked Read 3 Done Sent Done false
8749 3500 Acked Flush 0 Done Sent Done false
8750 3501 Acked Write 8 Done Sent Done false
8751 3502 Acked Read 6 Done Sent Done false
8752 3503 Acked Read 3 Done Sent Done false
8753 3504 Acked Read 3 Done Sent Done false
8754 3505 Acked Write 2 Done Sent Done false
8755 3506 Acked Read 3 Done Sent Done false
8756 3507 Acked Write 1 Done Sent Done false
8757 3508 Acked Read 1 Done Sent Done false
8758 3509 Acked Write 1 Done Sent Done false
8759 3510 Acked Write 4 Done Sent Done false
8760 3511 Acked Read 8 Done Sent Done false
8761 3512 Acked Write 8 Done Sent Done false
8762 3513 Acked Read 1 Done Sent Done false
8763 3514 Acked Read 3 Done Sent Done false
8764 3515 Acked Read 8 Done Sent Done false
8765 3516 Acked Write 3 Done Sent Done false
8766 3517 Acked Write 8 Done Sent Done false
8767 3518 Acked Read 9 Done Sent Done false
8768 3519 Acked Write 5 Done Sent Done false
8769 3520 Acked Read 8 Done Sent Done false
8770 3521 Acked Read 3 Done Sent Done false
8771 3522 Acked Read 2 Done Sent Done false
8772 3523 Acked Write 3 Done Sent Done false
8773 3524 Acked Write 5 Done Sent Done false
8774 3525 Acked Read 1 Done Sent Done false
8775 3526 Acked Write 9 Done Sent Done false
8776 3527 Acked Write 7 Done Sent Done false
8777 3528 Acked Write 6 Done Sent Done false
8778 3529 Acked Flush 0 Done Sent Done false
8779 3530 Acked Flush 0 Done Sent Done false
8780 3531 Acked Write 7 Done Sent Done false
8781 3532 Acked Read 2 Done Sent Done false
8782 3533 Acked Read 4 Done Sent Done false
8783 3534 Acked Write 1 Done Sent Done false
8784 3535 Acked Write 5 Done Sent Done false
8785 3536 Acked Write 6 Done Sent Done false
8786 3537 Acked Write 1 Done Sent Done false
8787 3538 Acked Write 3 Done Sent Done false
8788 3539 Acked Write 1 Done Sent Done false
8789 3540 Acked Read 9 Done Sent Done false
8790 3541 Acked Read 2 Done Sent Done false
8791 3542 Acked Flush 0 Done Sent Done false
8792 3543 Acked Read 8 Done Sent Done false
8793 3544 Acked Read 5 Done Sent Done false
8794 3545 Acked Write 6 Done Sent Done false
8795 3546 Acked Write 4 Done Sent Done false
8796 3547 Acked Write 4 Done Sent Done false
8797 3548 Acked Write 8 Done Sent Done false
8798 3549 Acked Read 5 Done Sent Done false
8799 3550 Acked Read 10 Done Sent Done false
8800 3551 Acked Write 8 Done Sent Done false
8801 3552 Acked Flush 0 Done Sent Done false
8802 3553 Acked Read 10 Done Sent Done false
8803 3554 Acked Read 6 Done Sent Done false
8804 3555 Acked Read 4 Done Sent Done false
8805 3556 Acked Read 6 Done Sent Done false
8806 3557 Acked Read 1 Done Sent Done false
8807 3558 Acked Read 8 Done Sent Done false
8808 3559 Acked Read 6 Done Sent Done false
8809 3560 Acked Read 5 Done Sent Done false
8810 3561 Acked Flush 0 Done Sent Done false
8811 3562 Acked Write 2 Done Sent Done false
8812 3563 Acked Flush 0 Done Sent Done false
8813 3564 Acked Write 9 Done Sent Done false
8814 3565 Acked Read 1 Done Sent Done false
8815 3566 Acked Write 7 Done Sent Done false
8816 3567 Acked Read 7 Done Sent Done false
8817 3568 Acked Read 7 Done Sent Done false
8818 3569 Acked Write 10 Done Sent Done false
8819 3570 Acked Read 10 Done Sent Done false
8820 3571 Acked Read 3 Done Sent Done false
8821 3572 Acked Write 3 Done Sent Done false
8822 3573 Acked Write 3 Done Sent Done false
8823 3574 Acked Read 5 Done Sent Done false
8824 3575 Acked Flush 0 Done Sent Done false
8825 3576 Acked Read 9 Done Sent Done false
8826 3577 Acked Read 6 Done Sent Done false
8827 3578 Acked Read 2 Done Sent Done false
8828 3579 Acked Flush 0 Done Sent Done false
8829 3580 Acked Flush 0 Done Sent Done false
8830 3581 Acked Write 5 Done Sent Done false
8831 3582 Acked Flush 0 Done Sent Done false
8832 3583 Acked Read 10 Done Sent Done false
8833 3584 Acked Read 10 Done Sent Done false
8834 3585 Acked Read 10 Done Sent Done false
8835 3586 Acked Write 9 Done Sent Done false
8836 3587 Acked Write 8 Done Sent Done false
8837 3588 Acked Write 10 Done Sent Done false
8838 3589 Acked Write 9 Done Sent Done false
8839 3590 Acked Read 6 Done Sent Done false
8840 3591 Acked Write 3 Done Sent Done false
8841 3592 Acked Write 3 Done Sent Done false
8842 3593 Acked Read 9 Done Sent Done false
8843 3594 Acked Read 9 Done Sent Done false
8844 3595 Acked Write 4 Done Sent Done false
8845 3596 Acked Write 8 Done Sent Done false
8846 3597 Acked Read 6 Done Sent Done false
8847 3598 Acked Read 1 Done Sent Done false
8848 3599 Acked Write 5 Done Sent Done false
8849 3600 Acked Write 10 Done Sent Done false
8850 3601 Acked Write 4 Done Sent Done false
8851 3602 Acked Read 4 Done Sent Done false
8852 3603 Acked Write 9 Done Sent Done false
8853 3604 Acked Write 6 Done Sent Done false
8854 3605 Acked Write 1 Done Sent Done false
8855 3606 Acked Write 6 Done Sent Done false
8856 3607 Acked Read 5 Done Sent Done false
8857 3608 Acked Read 7 Done Sent Done false
8858 3609 Acked Write 9 Done Sent Done false
8859 3610 Acked Write 7 Done Sent Done false
8860 3611 Acked Write 2 Done Sent Done false
8861 3612 Acked Read 4 Done Sent Done false
8862 3613 Acked Read 6 Done Sent Done false
8863 3614 Acked Flush 0 Done Sent Done false
8864 3615 Acked Write 2 Done Sent Done false
8865 3616 Acked Read 3 Done Sent Done false
8866 3617 Acked Read 9 Done Sent Done false
8867 3618 Acked Read 2 Done Sent Done false
8868 3619 Acked Write 8 Done Sent Done false
8869 3620 Acked Flush 0 Done Sent Done false
8870 3621 Acked Read 8 Done Sent Done false
8871 3622 Acked Write 10 Done Sent Done false
8872 3623 Acked Write 4 Done Sent Done false
8873 3624 Acked Read 10 Done Sent Done false
8874 3625 Acked Write 10 Done Sent Done false
8875 3626 Acked Read 7 Done Sent Done false
8876 3627 Acked Read 9 Done Sent Done false
8877 3628 Acked Read 6 Done Sent Done false
8878 3629 Acked Read 2 Done Sent Done false
8879 3630 Acked Write 9 Done Sent Done false
8880 3631 Acked Flush 0 Done Sent Done false
8881 3632 Acked Write 6 Done Sent Done false
8882 3633 Acked Read 3 Done Sent Done false
8883 3634 Acked Read 4 Done Sent Done false
8884 3635 Acked Read 9 Done Sent Done false
8885 3636 Acked Write 3 Done Sent Done false
8886 3637 Acked Write 3 Done Sent Done false
8887 3638 Acked Read 10 Done Sent Done false
8888 3639 Acked Flush 0 Done Sent Done false
8889 3640 Acked Write 7 Done Sent Done false
8890 3641 Acked Write 6 Done Sent Done false
8891 3642 Acked Read 9 Done Sent Done false
8892 3643 Acked Read 9 Done Sent Done false
8893 3644 Acked Read 1 Done Sent Done false
8894 3645 Acked Write 5 Done Sent Done false
8895 3646 Acked Read 10 Done Sent Done false
8896 3647 Acked Read 4 Done Sent Done false
8897 3648 Acked Write 2 Done Sent Done false
8898 3649 Acked Read 2 Done Sent Done false
8899 3650 Acked Read 2 Done Sent Done false
8900 3651 Acked Read 10 Done Sent Done false
8901 3652 Acked Read 4 Done Sent Done false
8902 3653 Acked Read 6 Done Sent Done false
8903 3654 Acked Read 5 Done Sent Done false
8904 3655 Acked Read 10 Done Sent Done false
8905 3656 Acked Flush 0 Done Sent Done false
8906 3657 Acked Read 6 Done Sent Done false
8907 3658 Acked Read 5 Done Sent Done false
8908 3659 Acked Read 9 Done Sent Done false
8909 3660 Acked Write 2 Done Sent Done false
8910 3661 Acked Flush 0 Done Sent Done false
8911 3662 Acked Write 7 Done Sent Done false
8912 3663 Acked Write 2 Done Sent Done false
8913 3664 Acked Write 6 Done Sent Done false
8914 3665 Acked Read 4 Done Sent Done false
8915 3666 Acked Read 2 Done Sent Done false
8916 3667 Acked Write 6 Done Sent Done false
8917 3668 Acked Read 4 Done Sent Done false
8918 3669 Acked Read 3 Done Sent Done false
8919 3670 Acked Write 5 Done Sent Done false
8920 3671 Acked Read 2 Done Sent Done false
8921 3672 Acked Read 9 Done Sent Done false
8922 3673 Acked Write 3 Done Sent Done false
8923 3674 Acked Write 3 Done Sent Done false
8924 3675 Acked Read 6 Done Sent Done false
8925 3676 Acked Write 7 Done Sent Done false
8926 3677 Acked Write 7 Done Sent Done false
8927 3678 Acked Read 1 Done Sent Done false
8928 3679 Acked Read 5 Done Sent Done false
8929 3680 Acked Flush 0 Done Sent Done false
8930 3681 Acked Write 8 Done Sent Done false
8931 3682 Acked Write 1 Done Sent Done false
8932 3683 Acked Read 4 Done Sent Done false
8933 3684 Acked Read 3 Done Sent Done false
8934 3685 Acked Flush 0 Done Sent Done false
8935 3686 Acked Read 8 Done Sent Done false
8936 3687 Acked Read 10 Done Sent Done false
8937 3688 Acked Write 1 Done Sent Done false
8938 3689 Acked Flush 0 Done Sent Done false
8939 3690 Acked Read 1 Done Sent Done false
8940 3691 Acked Write 7 Done Sent Done false
8941 3692 Acked Read 5 Done Sent Done false
8942 3693 Acked Write 10 Done Sent Done false
8943 3694 Acked Read 4 Done Sent Done false
8944 3695 Acked Write 9 Done Sent Done false
8945 3696 Acked Write 7 Done Sent Done false
8946 3697 Acked Read 8 Done Sent Done false
8947 3698 Acked Flush 0 Done Sent Done false
8948 3699 Acked Read 9 Done Sent Done false
8949 3700 Acked Write 1 Done Sent Done false
8950 3701 Acked Read 9 Done Sent Done false
8951 3702 Acked Read 10 Done Sent Done false
8952 3703 Acked Read 3 Done Sent Done false
8953 3704 Acked Write 2 Done Sent Done false
8954 3705 Acked Read 7 Done Sent Done false
8955 3706 Acked Write 3 Done Sent Done false
8956 3707 Acked Write 3 Done Sent Done false
8957 3708 Acked Write 4 Done Sent Done false
8958 3709 Acked Write 8 Done Sent Done false
8959 3710 Acked Read 3 Done Sent Done false
8960 3711 Acked Read 3 Done Sent Done false
8961 3712 Acked Read 5 Done Sent Done false
8962 3713 Acked Read 10 Done Sent Done false
8963 3714 Acked Read 9 Done Sent Done false
8964 3715 Acked Write 1 Done Sent Done false
8965 3716 Acked Flush 0 Done Sent Done false
8966 STATES DS:0 DS:1 DS:2 TOTAL
8967 Sent 0 302 0 302
8968 Done 302 0 302 604
8969 Skipped 0 0 0 0
8970 Error 0 0 0 0
8971 Last Flush: 3716 3414 3716
8972 Downstairs last five completed: 3414 3413 3412 3411 3410
8973 Upstairs last five completed: 3716 3715 3714 3713 3712
8974 CLIENT: Up:0 ds:302 act:2
89752024-12-04T13:32:34.123ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89762024-12-04T13:32:34.123ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40414) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89772024-12-04T13:32:34.124ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89782024-12-04T13:32:34.124ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89792024-12-04T13:32:34.124ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89802024-12-04T13:32:34.124ZINFOcrucible: send last flush ID to this DS: 3414 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89812024-12-04T13:32:34.125ZINFOcrucible: Replied this last flush ID: 3414 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89822024-12-04T13:32:34.125ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89832024-12-04T13:32:34.125ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89842024-12-04T13:32:34.125ZINFOcrucible: [1] client re-new 302 jobs since flush 3414 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
89852024-12-04T13:32:34.125ZINFOcrucible: [1] Replayed 302 jobs since flush: 3414 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
8986 ----------------------------------------------------------------
8987 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8988 Downstairs last five completed: 3716 3715 3714 3713 3712
8989 Upstairs last five completed: 3716 3715 3714 3713 3712
8990 CLIENT: Up:0 ds:0 act:3
8991 Replay: All jobs finished
8992 Main task has work to do, go find it
8993 got dsc Stop(1)
8994 stop 1
8995 Dec 04 13:32:36.365 INFO request completed, latency_us: 344, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 18fb73be-3d59-406b-a3d0-40a90df9614e[8820] Got stop action so:false kr:false
8996 , remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
89972024-12-04T13:32:36.365ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
8998 Dec 04 13:32:36.365 INFO request completed, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:32:36.365884987Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
8999 latency_us: 124, response_code: 200, uri: /state/cid/1, method: GET, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:32:36.365922678Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
9000 req_id: 15911f2c-f729-44c6-b10d-4eb7380b5e07, remote_addr: 127.0.0.1:37804, local_addr:{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:32:36.365953958Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
9001 127.0.0.1:9998
90022024-12-04T13:32:36.365ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
90032024-12-04T13:32:36.366ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
9004 [8820] Exited with: ExitStatus(unix_wait_status(9))
9005 [8820][1] reports Exit
9006 Dec 04 13:32:40.367 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/1, method: GET, req_id: 96de6c56-7349-44e4-b2bd-e1bec385b5f0, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
9007 001/300 Write block 1451 len 36864 data: 3 3 3 3 4 4 5 5 5
9008 002/300 Read block 686 len 8192
9009 003/300 Flush
9010 004/300 Read block 4438 len 36864
9011 005/300 Read block 204 len 24576
9012 006/300 Write block 3212 len 28672 data: 6 6 6 6 7 6 6
9013 007/300 Write block 689 len 20480 data: 2 2 3 3 4
9014 008/300 Read block 2174 len 28672
9015 009/300 Flush
9016 010/300 Write block 1163 len 4096 data: 2
9017 011/300 Write block 2284 len 8192 data: 3 2
9018 012/300 Read block 4054 len 24576
9019 013/300 Write block 2270 len 40960 data: 2 2 2 2 2 2 3 4 4 4
9020 014/300 Read block 4719 len 28672
9021 015/300 Write block 41 len 4096 data: 3
9022 016/300 Write block 4419 len 36864 data: 3 3 2 2 2 2 2 2 2
9023 017/300 Read block 2198 len 40960
9024 018/300 Read block 2082 len 36864
9025 019/300 Write block 2166 len 4096 data: 5
9026 020/300 Read block 1863 len 20480
9027 021/300 Write block 4202 len 4096 data: 3
9028 022/300 Read block 3624 len 20480
9029 023/300 Write block 1121 len 4096 data: 3
9030 024/300 Read block 4380 len 24576
9031 025/300 Write block 2596 len 32768 data: 3 4 3 2 3 4 4 4
9032 026/300 Flush
9033 027/300 Read block 354 len 12288
9034 028/300 Read block 1136 len 32768
9035 029/300 Read block 4935 len 24576
9036 030/300 Write block 1728 len 20480 data: 3 2 3 2 2
9037 031/300 Write block 477 len 20480 data: 3 2 2 2 2
9038 032/300 Read block 662 len 24576
9039 033/300 Write block 3159 len 36864 data: 3 4 3 3 3 3 3 3 2
9040 034/300 Read block 3283 len 20480
9041 035/300 Read block 625 len 32768
9042 036/300 Write block 774 len 28672 data: 3 3 2 2 2 2 2
9043 037/300 Read block 882 len 24576
9044 038/300 Read block 2399 len 24576
9045 039/300 Read block 4432 len 4096
9046 040/300 Read block 785 len 32768
9047 041/300 Read block 928 len 8192
9048 042/300 Write block 3494 len 16384 data: 4 4 4 4
9049 043/300 Read block 455 len 36864
9050 044/300 Flush
9051 045/300 Read block 1474 len 12288
9052 046/300 Write block 285 len 12288 data: 3 4 3
9053 047/300 Read block 1308 len 40960
9054 048/300 Write block 2390 len 40960 data: 3 3 3 3 3 4 4 3 3 3
9055 049/300 Read block 235 len 32768
9056 050/300 Read block 84 len 36864
9057 051/300 Flush
9058 052/300 Write block 2888 len 8192 data: 2 2
9059 053/300 Read block 1355 len 4096
9060 054/300 Write block 4483 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9061 055/300 Flush
9062 056/300 Read block 4717 len 4096
9063 057/300 Write block 567 len 28672 data: 2 2 2 2 2 2 2
9064 058/300 Write block 4953 len 28672 data: 3 2 2 2 2 2 2
9065 059/300 Flush
9066 060/300 Write block 272 len 20480 data: 2 2 2 2 2
9067 061/300 Write block 4870 len 12288 data: 3 2 2
9068 062/300 Read block 1829 len 16384
9069 063/300 Write block 4442 len 40960 data: 4 4 4 3 3 3 3 3 3 2
9070 064/300 Read block 1628 len 16384
9071 065/300 Write block 3877 len 24576 data: 3 4 4 4 3 3
9072 066/300 Read block 2748 len 40960
9073 067/300 Read block 2431 len 28672
9074 068/300 Write block 3241 len 12288 data: 3 4 3
9075 069/300 Write block 4265 len 24576 data: 3 3 2 3 3 3
9076 070/300 Write block 4467 len 40960 data: 4 5 5 6 5 5 3 4 4 4
9077 071/300 Read block 1175 len 16384
9078 072/300 Read block 4647 len 32768
9079 073/300 Flush
9080 074/300 Write block 4572 len 12288 data: 2 3 2
9081 075/300 Read block 3101 len 32768
9082 076/300 Read block 797 len 28672
9083 077/300 Write block 3087 len 28672 data: 2 2 2 3 3 3 2
9084 078/300 Write block 3172 len 8192 data: 5 4
9085 079/300 Write block 2786 len 28672 data: 3 2 3 3 3 3 3
9086 080/300 Read block 3426 len 16384
9087 081/300 Write block 2863 len 4096 data: 2
9088 082/300 Read block 3707 len 24576
9089 083/300 Read block 301 len 12288
9090 084/300 Read block 370 len 40960
9091 085/300 Write block 3379 len 8192 data: 6 6
9092 086/300 Flush
9093 087/300 Write block 3672 len 8192 data: 5 4
9094 088/300 Write block 948 len 16384 data: 4 3 3 3
9095 089/300 Write block 1607 len 8192 data: 2 2
9096 090/300 Read block 2801 len 8192
9097 091/300 Write block 2747 len 40960 data: 3 3 3 3 2 2 2 2 2 3
9098 092/300 Read block 484 len 12288
9099 093/300 Flush
9100 094/300 Write block 2478 len 32768 data: 4 4 4 3 3 5 5 4
9101 095/300 Write block 2478 len 16384 data: 5 5 5 4
9102 096/300 Write block 1215 len 4096 data: 2
9103 097/300 Write block 2852 len 16384 data: 4 3 3 3
9104 098/300 Read block 2515 len 24576
9105 099/300 Write block 2335 len 8192 data: 3 2
9106 100/300 Write block 123 len 40960 data: 3 3 3 4 3 3 3 3 3 3
9107 101/300 Write block 2121 len 32768 data: 3 3 4 4 4 4 4 4
9108 102/300 Write block 3432 len 32768 data: 4 4 5 6 6 6 6 5
9109 103/300 Read block 4766 len 8192
9110 104/300 Read block 3930 len 16384
9111 105/300 Write block 1343 len 24576 data: 2 3 4 4 3 3
9112 106/300 Read block 2141 len 36864
9113 107/300 Write block 2829 len 4096 data: 5
9114 108/300 Read block 4415 len 20480
9115 109/300 Read block 790 len 28672
9116 110/300 Read block 544 len 36864
9117 111/300 Write block 2125 len 12288 data: 5 5 5
9118 112/300 Read block 3317 len 8192
9119 113/300 Write block 2707 len 24576 data: 3 3 3 3 3 4
9120 114/300 Read block 1434 len 40960
9121 115/300 Write block 2465 len 24576 data: 5 2 2 2 4 4
9122 116/300 Read block 3718 len 24576
9123 117/300 Read block 3939 len 36864
9124 118/300 Read block 2191 len 32768
9125 119/300 Read block 4111 len 28672
9126 120/300 Write block 1205 len 20480 data: 2 2 2 2 2
9127 121/300 Flush
9128 122/300 Write block 2132 len 36864 data: 3 3 5 5 5 4 3 3 4
9129 123/300 Read block 792 len 20480
9130 124/300 Write block 64 len 12288 data: 2 2 2
9131 125/300 Write block 2924 len 32768 data: 4 5 5 5 4 4 3 2
9132 126/300 Flush
9133 127/300 Read block 3837 len 32768
9134 128/300 Read block 2376 len 28672
9135 129/300 Read block 4011 len 40960
9136 130/300 Write block 1911 len 32768 data: 4 4 4 4 4 4 4 3
9137 131/300 Read block 2356 len 4096
9138 132/300 Write block 4478 len 40960 data: 2 2 2 2 2 3 3 3 3 3
9139 133/300 Read block 2195 len 20480
9140 134/300 Read block 4130 len 12288
9141 135/300 Write block 3051 len 40960 data: 3 3 4 4 5 5 5 5 4 4
9142 136/300 Flush
9143 137/300 Read block 742 len 28672
9144 138/300 Read block 69 len 12288
9145 139/300 Write block 3418 len 8192 data: 3 3
9146 140/300 Write block 248 len 32768 data: 4 4 4 4 5 4 4 4
9147 141/300 Read block 1067 len 4096
9148 142/300 Read block 4921 len 8192
9149 143/300 Write block 2692 len 8192 data: 2 2
9150 144/300 Read block 1507 len 4096
9151 145/300 Write block 2106 len 20480 data: 2 3 3 3 3
9152 146/300 Write block 2088 len 8192 data: 2 3
9153 147/300 Read block 3258 len 12288
9154 148/300 Flush
9155 149/300 Read block 4357 len 4096
9156 150/300 Read block 3491 len 28672
9157 151/300 Write block 4080 len 32768 data: 2 2 2 2 2 2 3 3
9158 152/300 Flush
9159 153/300 Write block 3400 len 40960 data: 6 6 6 6 4 3 3 3 2 2
9160 154/300 Read block 3616 len 4096
9161 155/300 Write block 1824 len 8192 data: 2 2
9162 156/300 Read block 2399 len 8192
9163 157/300 Write block 4159 len 28672 data: 9 9 9 9 8 5 4
9164 158/300 Flush
9165 159/300 Write block 4917 len 32768 data: 3 3 3 3 3 4 4 3
9166 160/300 Read block 1541 len 40960
9167 161/300 Write block 2861 len 40960 data: 3 3 3 2 3 3 3 4 4 4
9168 162/300 Flush
9169 163/300 Write block 727 len 32768 data: 3 3 3 3 4 4 4 3
9170 164/300 Write block 1421 len 4096 data: 6
9171 165/300 Read block 411 len 32768
9172 166/300 Write block 1814 len 40960 data: 3 4 2 2 2 2 2 2 2 2
9173 167/300 Read block 2226 len 8192
9174 168/300 Read block 373 len 32768
9175 169/300 Write block 2600 len 4096 data: 4
9176 170/300 Read block 2865 len 28672
9177 171/300 Write block 4857 len 40960 data: 2 2 2 3 3 3 3 4 4 4
9178 172/300 Flush
9179 173/300 Read block 699 len 40960
9180 174/300 Read block 4610 len 40960
9181 175/300 Write block 4269 len 16384 data: 4 4 4 3
9182 176/300 Write block 1940 len 40960 data: 3 3 4 4 4 6 5 4 4 5
9183 177/300 Write block 4671 len 8192 data: 2 2
9184 178/300 Flush
9185 179/300 Read block 3938 len 8192
9186 180/300 Flush
9187 181/300 Read block 330 len 24576
9188 182/300 Read block 4274 len 24576
9189 183/300 Write block 3883 len 16384 data: 4 4 4 3
9190 184/300 Write block 3155 len 40960 data: 3 3 3 3 4 5 4 4 4 4
9191 185/300 Read block 4331 len 40960
9192 186/300 Write block 502 len 24576 data: 2 2 2 3 3 3
9193 187/300 Read block 570 len 40960
9194 188/300 Write block 837 len 20480 data: 3 2 3 3 3
9195 189/300 Read block 3079 len 4096
9196 190/300 Read block 1473 len 28672
9197 191/300 Write block 4483 len 24576 data: 4 4 4 4 4 3
9198 192/300 Read block 3801 len 28672
9199 193/300 Flush
9200 194/300 Flush
9201 195/300 Write block 4871 len 32768 data: 3 3 2 2 2 2 2 2
9202 196/300 Write block 4703 len 8192 data: 4 5
9203 197/300 Write block 3692 len 8192 data: 4 4
9204 198/300 Write block 2370 len 40960 data: 5 5 4 3 2 2 4 3 3 3
9205 199/300 Flush
9206 200/300 Write block 3229 len 40960 data: 3 3 3 3 3 4 4 5 4 4
9207 201/300 Read block 3637 len 40960
9208 202/300 Read block 4017 len 32768
9209 203/300 Write block 4890 len 28672 data: 5 5 5 4 3 3 3
9210 204/300 Flush
9211 205/300 Write block 2996 len 28672 data: 2 2 2 2 3 5 5
9212 206/300 Write block 4989 len 36864 data: 3 2 2 2 2 2 2 2 2
9213 207/300 Write block 2750 len 20480 data: 4 3 3 3 3
9214 208/300 Write block 3869 len 20480 data: 3 3 3 2 3
9215 209/300 Write block 1815 len 28672 data: 5 3 3 3 3 3 3
9216 210/300 Write block 343 len 8192 data: 2 2
9217 211/300 Read block 2504 len 16384
9218 212/300 Write block 393 len 20480 data: 5 5 4 4 4
9219 213/300 Read block 1649 len 4096
9220 214/300 Flush
9221 215/300 Read block 3128 len 40960
9222 216/300 Write block 3739 len 40960 data: 4 4 3 4 4 4 3 3 3 2
9223 217/300 Read block 224 len 12288
9224 218/300 Write block 3276 len 4096 data: 2
9225 219/300 Read block 4855 len 12288
9226 220/300 Read block 4018 len 24576
9227 221/300 Read block 2546 len 20480
9228 222/300 Read block 345 len 32768
9229 223/300 Read block 3615 len 4096
9230 224/300 Write block 4851 len 32768 data: 3 2 2 2 3 2 3 3
9231 225/300 Write block 923 len 20480 data: 4 4 4 4 3
9232 226/300 Write block 2486 len 12288 data: 3 3 3
9233 227/300 Write block 596 len 32768 data: 3 3 3 3 4 5 4 4
9234 228/300 Write block 1180 len 24576 data: 3 3 3 3 3 3
9235 229/300 Write block 1940 len 36864 data: 4 4 5 5 5 7 6 5 5
9236 230/300 Read block 3063 len 28672
9237 231/300 Flush
9238 232/300 Write block 3480 len 36864 data: 6 4 4 3 3 4 3 3 4
9239 233/300 Write block 1351 len 4096 data: 3
9240 234/300 Read block 3981 len 8192
9241 235/300 Write block 2480 len 32768 data: 6 5 4 6 6 5 4 4
9242 236/300 Write block 445 len 8192 data: 5 4
9243 237/300 Read block 1742 len 40960
9244 238/300 Read block 4029 len 16384
9245 239/300 Read block 4815 len 8192
9246 240/300 Read block 1076 len 8192
9247 241/300 Write block 4370 len 12288 data: 2 2 2
9248 242/300 Write block 983 len 32768 data: 4 4 5 5 3 3 2 2
9249 243/300 Write block 786 len 4096 data: 3
9250 244/300 Write block 2706 len 28672 data: 2 4 4 4 4 4 5
9251 245/300 Read block 2175 len 40960
9252 246/300 Read block 3257 len 16384
9253 247/300 Write block 3647 len 4096 data: 3
9254 248/300 Read block 4714 len 24576
9255 249/300 Read block 4387 len 32768
9256 250/300 Flush
9257 251/300 Read block 4524 len 40960
9258 252/300 Write block 1663 len 12288 data: 4 2 2
9259 253/300 Read block 4135 len 4096
9260 254/300 Flush
9261 255/300 Write block 4552 len 28672 data: 3 3 2 2 2 2 2
9262 256/300 Read block 1580 len 36864
9263 257/300 Read block 4011 len 32768
9264 258/300 Write block 72 len 20480 data: 3 3 3 3 2
9265 259/300 Flush
9266 260/300 Read block 833 len 36864
9267 261/300 Read block 3902 len 4096
9268 262/300 Read block 4486 len 4096
9269 263/300 Write block 3426 len 32768 data: 2 2 2 2 3 3 5 5
9270 264/300 Write block 1762 len 8192 data: 6 5
9271 265/300 Write block 1680 len 36864 data: 6 5 5 5 5 5 4 4 3
9272 266/300 Read block 988 len 32768
9273 267/300 Read block 4035 len 4096
9274 268/300 Read block 3708 len 40960
9275 269/300 Read block 2834 len 28672
9276 270/300 Read block 4928 len 40960
9277 271/300 Write block 2438 len 36864 data: 3 2 2 2 2 2 2 3 3
9278 272/300 Read block 1921 len 36864
9279 273/300 Write block 1497 len 12288 data: 4 4 4
9280 274/300 Read block 3958 len 40960
9281 275/300 Read block 3042 len 24576
9282 276/300 Write block 1947 len 28672 data: 6 6 6 5 5 3 2
9283 277/300 Flush
9284 278/300 Flush
9285 279/300 Read block 2479 len 40960
9286 280/300 Read block 4923 len 8192
9287 281/300 Read block 1544 len 12288
9288 282/300 Read block 2805 len 12288
9289 283/300 Read block 4547 len 8192
9290 284/300 Read block 4973 len 16384
9291 285/300 Read block 2240 len 20480
9292 286/300 Flush
9293 287/300 Write block 582 len 8192 data: 3 3
9294 288/300 Read block 2255 len 36864
9295 289/300 Write block 85 len 28672 data: 3 3 3 3 3 3 4
9296 290/300 Read block 4698 len 16384
9297 291/300 Write block 2025 len 28672 data: 2 3 3 3 3 2 3
9298 292/300 Write block 2286 len 36864 data: 2 3 3 4 4 4 4 4 4
9299 293/300 Read block 2596 len 16384
9300 294/300 Read block 130 len 32768
9301 295/300 Read block 4722 len 8192
9302 296/300 Write block 3264 len 12288 data: 2 2 2
9303 297/300 Read block 2836 len 8192
9304 298/300 Read block 4593 len 28672
9305 299/300 Read block 2313 len 20480
9306 300/300 Read block 3494 len 24576
9307 301/300 Write block 274 len 24576 data: 3 3 3 4 4 5
9308 Main task has work to do, go find it
9309 got dsc Start(1)
9310 start 1
9311 Dec 04 13:32:40.600 INFO request completed, latency_us: 238, response_code: 204, uri: /start/cid/1, method: POST, req_id: ba868f4c-0497-43ec-b7fb-bd650c9c7f3d, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
9312 Replay: started 1, returned:Ok(())
9313 ----------------------------------------------------------------
9314 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9315 States: Active Offline Active
9316 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9317 3717 Acked Write 9 Done Sent Done false
9318 3718 Acked Read 2 Done Sent Done false
9319 3719 Acked Flush 0 Done Sent Done false
9320 3720 Acked Read 9 Done Sent Done false
9321 3721 Acked Read 6 Done Sent Done false
9322 3722 Acked Write 7 Done Sent Done false
9323 3723 Acked Write 5 Done Sent Done false
9324 3724 Acked Read 7 Done Sent Done false
9325 3725 Acked Flush 0 Done Sent Done false
9326 3726 Acked Write 1 Done Sent Done false
9327 3727 Acked Write 2 Done Sent Done false
9328 3728 Acked Read 6 Done Sent Done false
9329 3729 Acked Write 10 Done Sent Done false
9330 3730 Acked Read 7 Done Sent Done false
9331 3731 Acked Write 1 Done Sent Done false
9332 3732 Acked Write 9 Done Sent Done false
9333 3733 Acked Read 10 Done Sent Done false
9334 3734 Acked Read 9 Done Sent Done false
9335 3735 Acked Write 1 Done Sent Done false
9336 3736 Acked Read 5 Done Sent Done false
9337 3737 Acked Write 1 Done Sent Done false
9338 3738 Acked Read 5 Done Sent Done false
9339 3739 Acked Write 1 Done Sent Done false
9340 3740 Acked Read 6 Done Sent Done false
9341 3741 Acked Write 8 Done Sent Done false
9342 3742 Acked Flush 0 Done Sent Done false
9343 3743 Acked Read 3 Done Sent Done false
9344 3744 Acked Read 8 Done Sent Done false
9345 3745 Acked Read 6 Done Sent Done false
9346 3746 Acked Write 5 Done Sent Done false
9347 3747 Acked Write 5 Done Sent Done false
9348 3748 Acked Read 6 Done Sent Done false
9349 3749 Acked Write 9 Done Sent Done false
9350 3750 Acked Read 5 Done Sent Done false
9351 3751 Acked Read 8 Done Sent Done false
9352 3752 Acked Write 7 Done Sent Done false
9353 3753 Acked Read 6 Done Sent Done false
9354 3754 Acked Read 6 Done Sent Done false
9355 3755 Acked Read 1 Done Sent Done false
9356 3756 Acked Read 8 Done Sent Done false
9357 3757 Acked Read 2 Done Sent Done false
9358 3758 Acked Write 4 Done Sent Done false
9359 3759 Acked Read 9 Done Sent Done false
9360 3760 Acked Flush 0 Done Sent Done false
9361 3761 Acked Read 3 Done Sent Done false
9362 3762 Acked Write 3 Done Sent Done false
9363 3763 Acked Read 10 Done Sent Done false
9364 3764 Acked Write 10 Done Sent Done false
9365 3765 Acked Read 8 Done Sent Done false
9366 3766 Acked Read 9 Done Sent Done false
9367 3767 Acked Flush 0 Done Sent Done false
9368 3768 Acked Write 2 Done Sent Done false
9369 3769 Acked Read 1 Done Sent Done false
9370 3770 Acked Write 10 Done Sent Done false
9371 3771 Acked Flush 0 Done Sent Done false
9372 3772 Acked Read 1 Done Sent Done false
9373 3773 Acked Write 7 Done Sent Done false
9374 3774 Acked Write 7 Done Sent Done false
9375 3775 Acked Flush 0 Done Sent Done false
9376 3776 Acked Write 5 Done Sent Done false
9377 3777 Acked Write 3 Done Sent Done false
9378 3778 Acked Read 4 Done Sent Done false
9379 3779 Acked Write 10 Done Sent Done false
9380 3780 Acked Read 4 Done Sent Done false
9381 3781 Acked Write 6 Done Sent Done false
9382 3782 Acked Read 10 Done Sent Done false
9383 3783 Acked Read 7 Done Sent Done false
9384 3784 Acked Write 3 Done Sent Done false
9385 3785 Acked Write 6 Done Sent Done false
9386 3786 Acked Write 10 Done Sent Done false
9387 3787 Acked Read 4 Done Sent Done false
9388 3788 Acked Read 8 Done Sent Done false
9389 3789 Acked Flush 0 Done Sent Done false
9390 3790 Acked Write 3 Done Sent Done false
9391 3791 Acked Read 8 Done Sent Done false
9392 3792 Acked Read 7 Done Sent Done false
9393 3793 Acked Write 7 Done Sent Done false
9394 3794 Acked Write 2 Done Sent Done false
9395 3795 Acked Write 7 Done Sent Done false
9396 3796 Acked Read 4 Done Sent Done false
9397 3797 Acked Write 1 Done Sent Done false
9398 3798 Acked Read 6 Done Sent Done false
9399 3799 Acked Read 3 Done Sent Done false
9400 3800 Acked Read 10 Done Sent Done false
9401 3801 Acked Write 2 Done Sent Done false
9402 3802 Acked Flush 0 Done Sent Done false
9403 3803 Acked Write 2 Done Sent Done false
9404 3804 Acked Write 4 Done Sent Done false
9405 3805 Acked Write 2 Done Sent Done false
9406 3806 Acked Read 2 Done Sent Done false
9407 3807 Acked Write 10 Done Sent Done false
9408 3808 Acked Read 3 Done Sent Done false
9409 3809 Acked Flush 0 Done Sent Done false
9410 3810 Acked Write 8 Done Sent Done false
9411 3811 Acked Write 4 Done Sent Done false
9412 3812 Acked Write 1 Done Sent Done false
9413 3813 Acked Write 4 Done Sent Done false
9414 3814 Acked Read 6 Done Sent Done false
9415 3815 Acked Write 2 Done Sent Done false
9416 3816 Acked Write 10 Done Sent Done false
9417 3817 Acked Write 8 Done Sent Done false
9418 3818 Acked Write 8 Done Sent Done false
9419 3819 Acked Read 2 Done Sent Done false
9420 3820 Acked Read 4 Done Sent Done false
9421 3821 Acked Write 6 Done Sent Done false
9422 3822 Acked Read 9 Done Sent Done false
9423 3823 Acked Write 1 Done Sent Done false
9424 3824 Acked Read 5 Done Sent Done false
9425 3825 Acked Read 7 Done Sent Done false
9426 3826 Acked Read 9 Done Sent Done false
9427 3827 Acked Write 3 Done Sent Done false
9428 3828 Acked Read 2 Done Sent Done false
9429 3829 Acked Write 6 Done Sent Done false
9430 3830 Acked Read 10 Done Sent Done false
9431 3831 Acked Write 6 Done Sent Done false
9432 3832 Acked Read 6 Done Sent Done false
9433 3833 Acked Read 9 Done Sent Done false
9434 3834 Acked Read 8 Done Sent Done false
9435 3835 Acked Read 7 Done Sent Done false
9436 3836 Acked Write 5 Done Sent Done false
9437 3837 Acked Flush 0 Done Sent Done false
9438 3838 Acked Write 9 Done Sent Done false
9439 3839 Acked Read 5 Done Sent Done false
9440 3840 Acked Write 3 Done Sent Done false
9441 3841 Acked Write 8 Done Sent Done false
9442 3842 Acked Flush 0 Done Sent Done false
9443 3843 Acked Read 8 Done Sent Done false
9444 3844 Acked Read 7 Done Sent Done false
9445 3845 Acked Read 10 Done Sent Done false
9446 3846 Acked Write 8 Done Sent Done false
9447 3847 Acked Read 1 Done Sent Done false
9448 3848 Acked Write 10 Done Sent Done false
9449 3849 Acked Read 5 Done Sent Done false
9450 3850 Acked Read 3 Done Sent Done false
9451 3851 Acked Write 10 Done Sent Done false
9452 3852 Acked Flush 0 Done Sent Done false
9453 3853 Acked Read 7 Done Sent Done false
9454 3854 Acked Read 3 Done Sent Done false
9455 3855 Acked Write 2 Done Sent Done false
9456 3856 Acked Write 8 Done Sent Done false
9457 3857 Acked Read 1 Done Sent Done false
9458 3858 Acked Read 2 Done Sent Done false
9459 3859 Acked Write 2 Done Sent Done false
9460 3860 Acked Read 1 Done Sent Done false
9461 3861 Acked Write 5 Done Sent Done false
9462 3862 Acked Write 2 Done Sent Done false
9463 3863 Acked Read 3 Done Sent Done false
9464 3864 Acked Flush 0 Done Sent Done false
9465 3865 Acked Read 1 Done Sent Done false
9466 3866 Acked Read 7 Done Sent Done false
9467 3867 Acked Write 8 Done Sent Done false
9468 3868 Acked Flush 0 Done Sent Done false
9469 3869 Acked Write 10 Done Sent Done false
9470 3870 Acked Read 1 Done Sent Done false
9471 3871 Acked Write 2 Done Sent Done false
9472 3872 Acked Read 2 Done Sent Done false
9473 3873 Acked Write 7 Done Sent Done false
9474 3874 Acked Flush 0 Done Sent Done false
9475 3875 Acked Write 8 Done Sent Done false
9476 3876 Acked Read 10 Done Sent Done false
9477 3877 Acked Write 10 Done Sent Done false
9478 3878 Acked Flush 0 Done Sent Done false
9479 3879 Acked Write 8 Done Sent Done false
9480 3880 Acked Write 1 Done Sent Done false
9481 3881 Acked Read 8 Done Sent Done false
9482 3882 Acked Write 10 Done Sent Done false
9483 3883 Acked Read 2 Done Sent Done false
9484 3884 Acked Read 8 Done Sent Done false
9485 3885 Acked Write 1 Done Sent Done false
9486 3886 Acked Read 7 Done Sent Done false
9487 3887 Acked Write 10 Done Sent Done false
9488 3888 Acked Flush 0 Done Sent Done false
9489 3889 Acked Read 10 Done Sent Done false
9490 3890 Acked Read 10 Done Sent Done false
9491 3891 Acked Write 4 Done Sent Done false
9492 3892 Acked Write 10 Done Sent Done false
9493 3893 Acked Write 2 Done Sent Done false
9494 3894 Acked Flush 0 Done Sent Done false
9495 3895 Acked Read 2 Done Sent Done false
9496 3896 Acked Flush 0 Done Sent Done false
9497 3897 Acked Read 6 Done Sent Done false
9498 3898 Acked Read 6 Done Sent Done false
9499 3899 Acked Write 4 Done Sent Done false
9500 3900 Acked Write 10 Done Sent Done false
9501 3901 Acked Read 10 Done Sent Done false
9502 3902 Acked Write 6 Done Sent Done false
9503 3903 Acked Read 10 Done Sent Done false
9504 3904 Acked Write 5 Done Sent Done false
9505 3905 Acked Read 1 Done Sent Done false
9506 3906 Acked Read 7 Done Sent Done false
9507 3907 Acked Write 6 Done Sent Done false
9508 3908 Acked Read 7 Done Sent Done false
9509 3909 Acked Flush 0 Done Sent Done false
9510 3910 Acked Flush 0 Done Sent Done false
9511 3911 Acked Write 8 Done Sent Done false
9512 3912 Acked Write 2 Done Sent Done false
9513 3913 Acked Write 2 Done Sent Done false
9514 3914 Acked Write 10 Done Sent Done false
9515 3915 Acked Flush 0 Done Sent Done false
9516 3916 Acked Write 10 Done Sent Done false
9517 3917 Acked Read 10 Done Sent Done false
9518 3918 Acked Read 8 Done Sent Done false
9519 3919 Acked Write 7 Done Sent Done false
9520 3920 Acked Flush 0 Done Sent Done false
9521 3921 Acked Write 7 Done Sent Done false
9522 3922 Acked Write 9 Done Sent Done false
9523 3923 Acked Write 5 Done Sent Done false
9524 3924 Acked Write 5 Done Sent Done false
9525 3925 Acked Write 7 Done Sent Done false
9526 3926 Acked Write 2 Done Sent Done false
9527 3927 Acked Read 4 Done Sent Done false
9528 3928 Acked Write 5 Done Sent Done false
9529 3929 Acked Read 1 Done Sent Done false
9530 3930 Acked Flush 0 Done Sent Done false
9531 3931 Acked Read 10 Done Sent Done false
9532 3932 Acked Write 10 Done Sent Done false
9533 3933 Acked Read 3 Done Sent Done false
9534 3934 Acked Write 1 Done Sent Done false
9535 3935 Acked Read 3 Done Sent Done false
9536 3936 Acked Read 6 Done Sent Done false
9537 3937 Acked Read 5 Done Sent Done false
9538 3938 Acked Read 8 Done Sent Done false
9539 3939 Acked Read 1 Done Sent Done false
9540 3940 Acked Write 8 Done Sent Done false
9541 3941 Acked Write 5 Done Sent Done false
9542 3942 Acked Write 3 Done Sent Done false
9543 3943 Acked Write 8 Done Sent Done false
9544 3944 Acked Write 6 Done Sent Done false
9545 3945 Acked Write 9 Done Sent Done false
9546 3946 Acked Read 7 Done Sent Done false
9547 3947 Acked Flush 0 Done Sent Done false
9548 3948 Acked Write 9 Done Sent Done false
9549 3949 Acked Write 1 Done Sent Done false
9550 3950 Acked Read 2 Done Sent Done false
9551 3951 Acked Write 8 Done Sent Done false
9552 3952 Acked Write 2 Done Sent Done false
9553 3953 Acked Read 10 Done Sent Done false
9554 3954 Acked Read 4 Done Sent Done false
9555 3955 Acked Read 2 Done Sent Done false
9556 3956 Acked Read 2 Done Sent Done false
9557 3957 Acked Write 3 Done Sent Done false
9558 3958 Acked Write 8 Done Sent Done false
9559 3959 Acked Write 1 Done Sent Done false
9560 3960 Acked Write 7 Done Sent Done false
9561 3961 Acked Read 10 Done Sent Done false
9562 3962 Acked Read 4 Done Sent Done false
9563 3963 Acked Write 1 Done Sent Done false
9564 3964 Acked Read 6 Done Sent Done false
9565 3965 Acked Read 8 Done Sent Done false
9566 3966 Acked Flush 0 Done Sent Done false
9567 3967 Acked Read 10 Done Sent Done false
9568 3968 Acked Write 3 Done Sent Done false
9569 3969 Acked Read 1 Done Sent Done false
9570 3970 Acked Flush 0 Done Sent Done false
9571 3971 Acked Write 7 Done Sent Done false
9572 3972 Acked Read 9 Done Sent Done false
9573 3973 Acked Read 8 Done Sent Done false
9574 3974 Acked Write 5 Done Sent Done false
9575 3975 Acked Flush 0 Done Sent Done false
9576 3976 Acked Read 9 Done Sent Done false
9577 3977 Acked Read 1 Done Sent Done false
9578 3978 Acked Read 1 Done Sent Done false
9579 3979 Acked Write 8 Done Sent Done false
9580 3980 Acked Write 2 Done Sent Done false
9581 3981 Acked Write 9 Done Sent Done false
9582 3982 Acked Read 8 Done Sent Done false
9583 3983 Acked Read 1 Done Sent Done false
9584 3984 Acked Read 10 Done Sent Done false
9585 3985 Acked Read 7 Done Sent Done false
9586 3986 Acked Read 10 Done Sent Done false
9587 3987 Acked Write 9 Done Sent Done false
9588 3988 Acked Read 9 Done Sent Done false
9589 3989 Acked Write 3 Done Sent Done false
9590 3990 Acked Read 10 Done Sent Done false
9591 3991 Acked Read 6 Done Sent Done false
9592 3992 Acked Write 7 Done Sent Done false
9593 3993 Acked Flush 0 Done Sent Done false
9594 3994 Acked Flush 0 Done Sent Done false
9595 3995 Acked Read 10 Done Sent Done false
9596 3996 Acked Read 2 Done Sent Done false
9597 3997 Acked Read 3 Done Sent Done false
9598 3998 Acked Read 3 Done Sent Done false
9599 3999 Acked Read 2 Done Sent Done false
9600 4000 Acked Read 4 Done Sent Done false
9601 4001 Acked Read 5 Done Sent Done false
9602 4002 Acked Flush 0 Done Sent Done false
9603 4003 Acked Write 2 Done Sent Done false
9604 4004 Acked Read 9 Done Sent Done false
9605 4005 Acked Write 7 Done Sent Done false
9606 4006 Acked Read 4 Done Sent Done false
9607 4007 Acked Write 7 Done Sent Done false
9608 4008 Acked Write 9 Done Sent Done false
9609 4009 Acked Read 4 Done Sent Done false
9610 4010 Acked Read 8 Done Sent Done false
9611 4011 Acked Read 2 Done Sent Done false
9612 4012 Acked Write 3 Done Sent Done false
9613 4013 Acked Read 2 Done Sent Done false
9614 4014 Acked Read 7 Done Sent Done false
9615 4015 Acked Read 5 Done Sent Done false
9616 4016 Acked Read 6 Done Sent Done false
9617 4017 Acked Write 6 Done Sent Done false
9618 STATES DS:0 DS:1 DS:2 TOTAL
9619 Sent 0 301 0 301
9620 Done 301 0 301 602
9621 Skipped 0 0 0 0
9622 Error 0 0 0 0
9623 Last Flush: 4002 3716 4002
9624 Downstairs last five completed: 3716 3715 3714 3713 3712
9625 Upstairs last five completed: 4017 4016 4015 4014 4013
9626 CLIENT: Up:0 ds:301 act:2
9627 [8820] got start action
9628 [8820] I am going to restart
9629 Starting downstairs at port 8820
9630 Make output file at "/tmp/dsc/downstairs-8820.txt"
9631 [8820][1] reports Starting
9632 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
9633 [8820][1] reports Running
9634 ----------------------------------------------------------------
9635 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9636 States: Active Offline Active
9637 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9638 3717 Acked Write 9 Done Sent Done false
9639 3718 Acked Read 2 Done Sent Done false
9640 3719 Acked Flush 0 Done Sent Done false
9641 3720 Acked Read 9 Done Sent Done false
9642 3721 Acked Read 6 Done Sent Done false
9643 3722 Acked Write 7 Done Sent Done false
9644 3723 Acked Write 5 Done Sent Done false
9645 3724 Acked Read 7 Done Sent Done false
9646 3725 Acked Flush 0 Done Sent Done false
9647 3726 Acked Write 1 Done Sent Done false
9648 3727 Acked Write 2 Done Sent Done false
9649 3728 Acked Read 6 Done Sent Done false
9650 3729 Acked Write 10 Done Sent Done false
9651 3730 Acked Read 7 Done Sent Done false
9652 3731 Acked Write 1 Done Sent Done false
9653 3732 Acked Write 9 Done Sent Done false
9654 3733 Acked Read 10 Done Sent Done false
9655 3734 Acked Read 9 Done Sent Done false
9656 3735 Acked Write 1 Done Sent Done false
9657 3736 Acked Read 5 Done Sent Done false
9658 3737 Acked Write 1 Done Sent Done false
9659 3738 Acked Read 5 Done Sent Done false
9660 3739 Acked Write 1 Done Sent Done false
9661 3740 Acked Read 6 Done Sent Done false
9662 3741 Acked Write 8 Done Sent Done false
9663 3742 Acked Flush 0 Done Sent Done false
9664 3743 Acked Read 3 Done Sent Done false
9665 3744 Acked Read 8 Done Sent Done false
9666 3745 Acked Read 6 Done Sent Done false
9667 3746 Acked Write 5 Done Sent Done false
9668 3747 Acked Write 5 Done Sent Done false
9669 3748 Acked Read 6 Done Sent Done false
9670 3749 Acked Write 9 Done Sent Done false
9671 3750 Acked Read 5 Done Sent Done false
9672 3751 Acked Read 8 Done Sent Done false
9673 3752 Acked Write 7 Done Sent Done false
9674 3753 Acked Read 6 Done Sent Done false
9675 3754 Acked Read 6 Done Sent Done false
9676 3755 Acked Read 1 Done Sent Done false
9677 3756 Acked Read 8 Done Sent Done false
9678 3757 Acked Read 2 Done Sent Done false
9679 3758 Acked Write 4 Done Sent Done false
9680 3759 Acked Read 9 Done Sent Done false
9681 3760 Acked Flush 0 Done Sent Done false
9682 3761 Acked Read 3 Done Sent Done false
9683 3762 Acked Write 3 Done Sent Done false
9684 3763 Acked Read 10 Done Sent Done false
9685 3764 Acked Write 10 Done Sent Done false
9686 3765 Acked Read 8 Done Sent Done false
9687 3766 Acked Read 9 Done Sent Done false
9688 3767 Acked Flush 0 Done Sent Done false
9689 3768 Acked Write 2 Done Sent Done false
9690 3769 Acked Read 1 Done Sent Done false
9691 3770 Acked Write 10 Done Sent Done false
9692 3771 Acked Flush 0 Done Sent Done false
9693 3772 Acked Read 1 Done Sent Done false
9694 3773 Acked Write 7 Done Sent Done false
9695 3774 Acked Write 7 Done Sent Done false
9696 3775 Acked Flush 0 Done Sent Done false
9697 3776 Acked Write 5 Done Sent Done false
9698 3777 Acked Write 3 Done Sent Done false
9699 3778 Acked Read 4 Done Sent Done false
9700 3779 Acked Write 10 Done Sent Done false
9701 3780 Acked Read 4 Done Sent Done false
9702 3781 Acked Write 6 Done Sent Done false
9703 3782 Acked Read 10 Done Sent Done false
9704 3783 Acked Read 7 Done Sent Done false
9705 3784 Acked Write 3 Done Sent Done false
9706 3785 Acked Write 6 Done Sent Done false
9707 3786 Acked Write 10 Done Sent Done false
9708 3787 Acked Read 4 Done Sent Done false
9709 3788 Acked Read 8 Done Sent Done false
9710 3789 Acked Flush 0 Done Sent Done false
9711 3790 Acked Write 3 Done Sent Done false
9712 3791 Acked Read 8 Done Sent Done false
9713 3792 Acked Read 7 Done Sent Done false
9714 3793 Acked Write 7 Done Sent Done false
9715 3794 Acked Write 2 Done Sent Done false
9716 3795 Acked Write 7 Done Sent Done false
9717 3796 Acked Read 4 Done Sent Done false
9718 3797 Acked Write 1 Done Sent Done false
9719 3798 Acked Read 6 Done Sent Done false
9720 3799 Acked Read 3 Done Sent Done false
9721 3800 Acked Read 10 Done Sent Done false
9722 3801 Acked Write 2 Done Sent Done false
9723 3802 Acked Flush 0 Done Sent Done false
9724 3803 Acked Write 2 Done Sent Done false
9725 3804 Acked Write 4 Done Sent Done false
9726 3805 Acked Write 2 Done Sent Done false
9727 3806 Acked Read 2 Done Sent Done false
9728 3807 Acked Write 10 Done Sent Done false
9729 3808 Acked Read 3 Done Sent Done false
9730 3809 Acked Flush 0 Done Sent Done false
9731 3810 Acked Write 8 Done Sent Done false
9732 3811 Acked Write 4 Done Sent Done false
9733 3812 Acked Write 1 Done Sent Done false
9734 3813 Acked Write 4 Done Sent Done false
9735 3814 Acked Read 6 Done Sent Done false
9736 3815 Acked Write 2 Done Sent Done false
9737 3816 Acked Write 10 Done Sent Done false
9738 3817 Acked Write 8 Done Sent Done false
9739 3818 Acked Write 8 Done Sent Done false
9740 3819 Acked Read 2 Done Sent Done false
9741 3820 Acked Read 4 Done Sent Done false
9742 3821 Acked Write 6 Done Sent Done false
9743 3822 Acked Read 9 Done Sent Done false
9744 3823 Acked Write 1 Done Sent Done false
9745 3824 Acked Read 5 Done Sent Done false
9746 3825 Acked Read 7 Done Sent Done false
9747 3826 Acked Read 9 Done Sent Done false
9748 3827 Acked Write 3 Done Sent Done false
9749 3828 Acked Read 2 Done Sent Done false
9750 3829 Acked Write 6 Done Sent Done false
9751 3830 Acked Read 10 Done Sent Done false
9752 3831 Acked Write 6 Done Sent Done false
9753 3832 Acked Read 6 Done Sent Done false
9754 3833 Acked Read 9 Done Sent Done false
9755 3834 Acked Read 8 Done Sent Done false
9756 3835 Acked Read 7 Done Sent Done false
9757 3836 Acked Write 5 Done Sent Done false
9758 3837 Acked Flush 0 Done Sent Done false
9759 3838 Acked Write 9 Done Sent Done false
9760 3839 Acked Read 5 Done Sent Done false
9761 3840 Acked Write 3 Done Sent Done false
9762 3841 Acked Write 8 Done Sent Done false
9763 3842 Acked Flush 0 Done Sent Done false
9764 3843 Acked Read 8 Done Sent Done false
9765 3844 Acked Read 7 Done Sent Done false
9766 3845 Acked Read 10 Done Sent Done false
9767 3846 Acked Write 8 Done Sent Done false
9768 3847 Acked Read 1 Done Sent Done false
9769 3848 Acked Write 10 Done Sent Done false
9770 3849 Acked Read 5 Done Sent Done false
9771 3850 Acked Read 3 Done Sent Done false
9772 3851 Acked Write 10 Done Sent Done false
9773 3852 Acked Flush 0 Done Sent Done false
9774 3853 Acked Read 7 Done Sent Done false
9775 3854 Acked Read 3 Done Sent Done false
9776 3855 Acked Write 2 Done Sent Done false
9777 3856 Acked Write 8 Done Sent Done false
9778 3857 Acked Read 1 Done Sent Done false
9779 3858 Acked Read 2 Done Sent Done false
9780 3859 Acked Write 2 Done Sent Done false
9781 3860 Acked Read 1 Done Sent Done false
9782 3861 Acked Write 5 Done Sent Done false
9783 3862 Acked Write 2 Done Sent Done false
9784 3863 Acked Read 3 Done Sent Done false
9785 3864 Acked Flush 0 Done Sent Done false
9786 3865 Acked Read 1 Done Sent Done false
9787 3866 Acked Read 7 Done Sent Done false
9788 3867 Acked Write 8 Done Sent Done false
9789 3868 Acked Flush 0 Done Sent Done false
9790 3869 Acked Write 10 Done Sent Done false
9791 3870 Acked Read 1 Done Sent Done false
9792 3871 Acked Write 2 Done Sent Done false
9793 3872 Acked Read 2 Done Sent Done false
9794 3873 Acked Write 7 Done Sent Done false
9795 3874 Acked Flush 0 Done Sent Done false
9796 3875 Acked Write 8 Done Sent Done false
9797 3876 Acked Read 10 Done Sent Done false
9798 3877 Acked Write 10 Done Sent Done false
9799 3878 Acked Flush 0 Done Sent Done false
9800 3879 Acked Write 8 Done Sent Done false
9801 3880 Acked Write 1 Done Sent Done false
9802 3881 Acked Read 8 Done Sent Done false
9803 3882 Acked Write 10 Done Sent Done false
9804 3883 Acked Read 2 Done Sent Done false
9805 3884 Acked Read 8 Done Sent Done false
9806 3885 Acked Write 1 Done Sent Done false
9807 3886 Acked Read 7 Done Sent Done false
9808 3887 Acked Write 10 Done Sent Done false
9809 3888 Acked Flush 0 Done Sent Done false
9810 3889 Acked Read 10 Done Sent Done false
9811 3890 Acked Read 10 Done Sent Done false
9812 3891 Acked Write 4 Done Sent Done false
9813 3892 Acked Write 10 Done Sent Done false
9814 3893 Acked Write 2 Done Sent Done false
9815 3894 Acked Flush 0 Done Sent Done false
9816 3895 Acked Read 2 Done Sent Done false
9817 3896 Acked Flush 0 Done Sent Done false
9818 3897 Acked Read 6 Done Sent Done false
9819 3898 Acked Read 6 Done Sent Done false
9820 3899 Acked Write 4 Done Sent Done false
9821 3900 Acked Write 10 Done Sent Done false
9822 3901 Acked Read 10 Done Sent Done false
9823 3902 Acked Write 6 Done Sent Done false
9824 3903 Acked Read 10 Done Sent Done false
9825 3904 Acked Write 5 Done Sent Done false
9826 3905 Acked Read 1 Done Sent Done false
9827 3906 Acked Read 7 Done Sent Done false
9828 3907 Acked Write 6 Done Sent Done false
9829 3908 Acked Read 7 Done Sent Done false
9830 3909 Acked Flush 0 Done Sent Done false
9831 3910 Acked Flush 0 Done Sent Done false
9832 3911 Acked Write 8 Done Sent Done false
9833 3912 Acked Write 2 Done Sent Done false
9834 3913 Acked Write 2 Done Sent Done false
9835 3914 Acked Write 10 Done Sent Done false
9836 3915 Acked Flush 0 Done Sent Done false
9837 3916 Acked Write 10 Done Sent Done false
9838 3917 Acked Read 10 Done Sent Done false
9839 3918 Acked Read 8 Done Sent Done false
9840 3919 Acked Write 7 Done Sent Done false
9841 3920 Acked Flush 0 Done Sent Done false
9842 3921 Acked Write 7 Done Sent Done false
9843 3922 Acked Write 9 Done Sent Done false
9844 3923 Acked Write 5 Done Sent Done false
9845 3924 Acked Write 5 Done Sent Done false
9846 3925 Acked Write 7 Done Sent Done false
9847 3926 Acked Write 2 Done Sent Done false
9848 3927 Acked Read 4 Done Sent Done false
9849 3928 Acked Write 5 Done Sent Done false
9850 3929 Acked Read 1 Done Sent Done false
9851 3930 Acked Flush 0 Done Sent Done false
9852 3931 Acked Read 10 Done Sent Done false
9853 3932 Acked Write 10 Done Sent Done false
9854 3933 Acked Read 3 Done Sent Done false
9855 3934 Acked Write 1 Done Sent Done false
9856 3935 Acked Read 3 Done Sent Done false
9857 3936 Acked Read 6 Done Sent Done false
9858 3937 Acked Read 5 Done Sent Done false
9859 3938 Acked Read 8 Done Sent Done false
9860 3939 Acked Read 1 Done Sent Done false
9861 3940 Acked Write 8 Done Sent Done false
9862 3941 Acked Write 5 Done Sent Done false
9863 3942 Acked Write 3 Done Sent Done false
9864 3943 Acked Write 8 Done Sent Done false
9865 3944 Acked Write 6 Done Sent Done false
9866 3945 Acked Write 9 Done Sent Done false
9867 3946 Acked Read 7 Done Sent Done false
9868 3947 Acked Flush 0 Done Sent Done false
9869 3948 Acked Write 9 Done Sent Done false
9870 3949 Acked Write 1 Done Sent Done false
9871 3950 Acked Read 2 Done Sent Done false
9872 3951 Acked Write 8 Done Sent Done false
9873 3952 Acked Write 2 Done Sent Done false
9874 3953 Acked Read 10 Done Sent Done false
9875 3954 Acked Read 4 Done Sent Done false
9876 3955 Acked Read 2 Done Sent Done false
9877 3956 Acked Read 2 Done Sent Done false
9878 3957 Acked Write 3 Done Sent Done false
9879 3958 Acked Write 8 Done Sent Done false
9880 3959 Acked Write 1 Done Sent Done false
9881 3960 Acked Write 7 Done Sent Done false
9882 3961 Acked Read 10 Done Sent Done false
9883 3962 Acked Read 4 Done Sent Done false
9884 3963 Acked Write 1 Done Sent Done false
9885 3964 Acked Read 6 Done Sent Done false
9886 3965 Acked Read 8 Done Sent Done false
9887 3966 Acked Flush 0 Done Sent Done false
9888 3967 Acked Read 10 Done Sent Done false
9889 3968 Acked Write 3 Done Sent Done false
9890 3969 Acked Read 1 Done Sent Done false
9891 3970 Acked Flush 0 Done Sent Done false
9892 3971 Acked Write 7 Done Sent Done false
9893 3972 Acked Read 9 Done Sent Done false
9894 3973 Acked Read 8 Done Sent Done false
9895 3974 Acked Write 5 Done Sent Done false
9896 3975 Acked Flush 0 Done Sent Done false
9897 3976 Acked Read 9 Done Sent Done false
9898 3977 Acked Read 1 Done Sent Done false
9899 3978 Acked Read 1 Done Sent Done false
9900 3979 Acked Write 8 Done Sent Done false
9901 3980 Acked Write 2 Done Sent Done false
9902 3981 Acked Write 9 Done Sent Done false
9903 3982 Acked Read 8 Done Sent Done false
9904 3983 Acked Read 1 Done Sent Done false
9905 3984 Acked Read 10 Done Sent Done false
9906 3985 Acked Read 7 Done Sent Done false
9907 3986 Acked Read 10 Done Sent Done false
9908 3987 Acked Write 9 Done Sent Done false
9909 3988 Acked Read 9 Done Sent Done false
9910 3989 Acked Write 3 Done Sent Done false
9911 3990 Acked Read 10 Done Sent Done false
9912 3991 Acked Read 6 Done Sent Done false
9913 3992 Acked Write 7 Done Sent Done false
9914 3993 Acked Flush 0 Done Sent Done false
9915 3994 Acked Flush 0 Done Sent Done false
9916 3995 Acked Read 10 Done Sent Done false
9917 3996 Acked Read 2 Done Sent Done false
9918 3997 Acked Read 3 Done Sent Done false
9919 3998 Acked Read 3 Done Sent Done false
9920 3999 Acked Read 2 Done Sent Done false
9921 4000 Acked Read 4 Done Sent Done false
9922 4001 Acked Read 5 Done Sent Done false
9923 4002 Acked Flush 0 Done Sent Done false
9924 4003 Acked Write 2 Done Sent Done false
9925 4004 Acked Read 9 Done Sent Done false
9926 4005 Acked Write 7 Done Sent Done false
9927 4006 Acked Read 4 Done Sent Done false
9928 4007 Acked Write 7 Done Sent Done false
9929 4008 Acked Write 9 Done Sent Done false
9930 4009 Acked Read 4 Done Sent Done false
9931 4010 Acked Read 8 Done Sent Done false
9932 4011 Acked Read 2 Done Sent Done false
9933 4012 Acked Write 3 Done Sent Done false
9934 4013 Acked Read 2 Done Sent Done false
9935 4014 Acked Read 7 Done Sent Done false
9936 4015 Acked Read 5 Done Sent Done false
9937 4016 Acked Read 6 Done Sent Done false
9938 4017 Acked Write 6 Done Sent Done false
9939 4018 Acked Flush 0 Done Sent Done false
9940 STATES DS:0 DS:1 DS:2 TOTAL
9941 Sent 0 302 0 302
9942 Done 302 0 302 604
9943 Skipped 0 0 0 0
9944 Error 0 0 0 0
9945 Last Flush: 4018 3716 4018
9946 Downstairs last five completed: 3716 3715 3714 3713 3712
9947 Upstairs last five completed: 4018 4017 4016 4015 4014
9948 CLIENT: Up:0 ds:302 act:2
99492024-12-04T13:32:46.367ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99502024-12-04T13:32:46.367ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41480) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99512024-12-04T13:32:46.368ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99522024-12-04T13:32:46.369ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99532024-12-04T13:32:46.369ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99542024-12-04T13:32:46.369ZINFOcrucible: send last flush ID to this DS: 3716 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99552024-12-04T13:32:46.369ZINFOcrucible: Replied this last flush ID: 3716 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99562024-12-04T13:32:46.369ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99572024-12-04T13:32:46.369ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99582024-12-04T13:32:46.369ZINFOcrucible: [1] client re-new 302 jobs since flush 3716 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99592024-12-04T13:32:46.369ZINFOcrucible: [1] Replayed 302 jobs since flush: 3716 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
9960 ----------------------------------------------------------------
9961 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9962 Downstairs last five completed: 4018 4017 4016 4015 4014
9963 Upstairs last five completed: 4018 4017 4016 4015 4014
9964 CLIENT: Up:0 ds:0 act:3
9965 Replay: All jobs finished
9966 Main task has work to do, go find it
9967 got dsc Stop(1)
9968 stop 1
9969 Dec 04 13:32:48.605 INFO request completed, latency_us: 339, response_code: 204, uri: /stop/cid/1, method[8820] Got stop action so:false kr:false
9970 : POST, req_id: 5fed81ce-1903-4870-94db-5c0d07765072, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
9971 Dec 04 13:32:48.606 INFO request completed, latency_us: 154, response_code: 200, uri: /state/cid/1, method: GET, req_id: b3ac1523-c041-409b-abac-b22de4fa270e, remote_addr: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:32:48.606119083Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
9972 127.0.0.1:37804, local_addr: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:32:48.606197944Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
9973 127.0.0.1:9998
99742024-12-04T13:32:48.606ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99752024-12-04T13:32:48.606ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99762024-12-04T13:32:48.606ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
99772024-12-04T13:32:48.606ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
9978 [8820] Exited with: ExitStatus(unix_wait_status(9))
9979 [8820][1] reports Exit
9980 Dec 04 13:32:52.608 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/1, method: GET, req_id: 9c261aeb-3d63-4984-95a1-c88214e842ae, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
9981 001/300 Write block 116 len 12288 data: 3 3 3
9982 002/300 Write block 1953 len 8192 data: 3 2
9983 003/300 Read block 224 len 28672
9984 004/300 Write block 1762 len 28672 data: 7 6 5 5 4 4 5
9985 005/300 Read block 4362 len 32768
9986 006/300 Write block 648 len 16384 data: 2 2 2 2
9987 007/300 Write block 3814 len 40960 data: 4 4 5 5 4 3 3 3 2 2
9988 008/300 Read block 233 len 32768
9989 009/300 Read block 3448 len 24576
9990 010/300 Write block 4402 len 32768 data: 2 2 2 3 4 4 4 4
9991 011/300 Read block 4637 len 8192
9992 012/300 Write block 1561 len 24576 data: 2 2 2 2 2 2
9993 013/300 Flush
9994 014/300 Write block 4723 len 12288 data: 3 3 3
9995 015/300 Read block 4064 len 16384
9996 016/300 Flush
9997 017/300 Read block 1529 len 12288
9998 018/300 Read block 932 len 24576
9999 019/300 Write block 327 len 16384 data: 2 3 3 4
10000 020/300 Read block 2952 len 16384
10001 021/300 Read block 1852 len 4096
10002 022/300 Write block 4146 len 16384 data: 2 2 2 2
10003 023/300 Write block 196 len 4096 data: 3
10004 024/300 Read block 4475 len 24576
10005 025/300 Read block 3559 len 8192
10006 026/300 Write block 977 len 32768 data: 3 4 4 4 3 3 5 5
10007 027/300 Read block 983 len 8192
10008 028/300 Read block 4788 len 16384
10009 029/300 Write block 32 len 32768 data: 2 2 2 2 2 2 2 2
10010 030/300 Flush
10011 031/300 Read block 2620 len 40960
10012 032/300 Write block 1070 len 36864 data: 7 7 5 4 4 3 3 4 3
10013 033/300 Read block 4210 len 16384
10014 034/300 Read block 44 len 8192
10015 035/300 Read block 686 len 32768
10016 036/300 Read block 3234 len 32768
10017 037/300 Read block 4098 len 24576
10018 038/300 Write block 3402 len 20480 data: 7 7 5 4 4
10019 039/300 Write block 4269 len 16384 data: 5 5 5 4
10020 040/300 Read block 924 len 12288
10021 041/300 Write block 4237 len 36864 data: 3 3 3 2 3 4 3 4 3
10022 042/300 Read block 1045 len 36864
10023 043/300 Write block 4969 len 20480 data: 5 4 4 4 3
10024 044/300 Write block 393 len 8192 data: 6 6
10025 045/300 Write block 1031 len 16384 data: 2 2 2 2
10026 046/300 Flush
10027 047/300 Read block 123 len 4096
10028 048/300 Read block 3062 len 40960
10029 049/300 Read block 2529 len 20480
10030 050/300 Write block 1529 len 36864 data: 6 6 5 5 3 3 2 2 2
10031 051/300 Write block 3824 len 40960 data: 3 3 3 4 4 4 3 3 3 3
10032 052/300 Write block 4215 len 40960 data: 2 2 2 2 2 2 2 2 2 2
10033 053/300 Write block 1390 len 12288 data: 3 3 2
10034 054/300 Write block 579 len 28672 data: 2 2 2 4 4 2 2
10035 055/300 Write block 2587 len 24576 data: 2 2 4 3 3 3
10036 056/300 Write block 4486 len 28672 data: 5 5 4 3 3 3 3
10037 057/300 Write block 2234 len 36864 data: 3 3 3 3 3 3 3 2 3
10038 058/300 Read block 1221 len 4096
10039 059/300 Read block 2622 len 32768
10040 060/300 Read block 1464 len 40960
10041 061/300 Read block 3989 len 40960
10042 062/300 Write block 1904 len 20480 data: 2 3 2 2 2
10043 063/300 Write block 975 len 8192 data: 2 2
10044 064/300 Write block 1655 len 12288 data: 4 4 4
10045 065/300 Read block 3537 len 12288
10046 066/300 Write block 2882 len 36864 data: 3 3 3 2 2 2 3 3 2
10047 067/300 Read block 689 len 36864
10048 068/300 Read block 4849 len 16384
10049 069/300 Flush
10050 070/300 Write block 992 len 12288 data: 2 2 3
10051 071/300 Write block 2263 len 36864 data: 2 2 2 2 2 2 2 3 3
10052 072/300 Write block 1785 len 16384 data: 2 2 2 2
10053 073/300 Read block 4895 len 40960
10054 074/300 Read block 3387 len 28672
10055 075/300 Read block 1126 len 12288
10056 076/300 Write block 0 len 16384 data: 2 4 5 5
10057 077/300 Write block 2188 len 24576 data: 5 3 3 4 4 4
10058 078/300 Flush
10059 079/300 Read block 2900 len 36864
10060 080/300 Write block 2665 len 28672 data: 4 5 6 5 5 6 6
10061 081/300 Read block 4943 len 24576
10062 082/300 Read block 2952 len 16384
10063 083/300 Write block 441 len 32768 data: 5 6 6 5 6 5 3 3
10064 084/300 Write block 3483 len 32768 data: 4 4 5 4 4 5 4 3
10065 085/300 Read block 4662 len 20480
10066 086/300 Flush
10067 087/300 Read block 1943 len 28672
10068 088/300 Write block 4601 len 40960 data: 4 4 4 4 4 3 3 3 3 3
10069 089/300 Read block 4465 len 28672
10070 090/300 Read block 1516 len 12288
10071 091/300 Write block 4398 len 40960 data: 3 2 2 2 3 3 3 4 5 5
10072 092/300 Write block 415 len 28672 data: 2 2 2 2 2 2 2
10073 093/300 Read block 4997 len 8192
10074 094/300 Write block 3229 len 36864 data: 4 4 4 4 4 5 5 6 5
10075 095/300 Read block 1791 len 16384
10076 096/300 Read block 1044 len 24576
10077 097/300 Read block 2059 len 4096
10078 098/300 Flush
10079 099/300 Read block 2657 len 20480
10080 100/300 Write block 700 len 8192 data: 3 3
10081 101/300 Write block 1802 len 24576 data: 2 2 3 3 3 4
10082 102/300 Write block 1746 len 16384 data: 5 5 5 5
10083 103/300 Read block 4258 len 32768
10084 104/300 Read block 3538 len 24576
10085 105/300 Read block 2019 len 16384
10086 106/300 Read block 2293 len 28672
10087 107/300 Read block 881 len 28672
10088 108/300 Flush
10089 109/300 Read block 970 len 32768
10090 110/300 Read block 4782 len 12288
10091 111/300 Write block 275 len 8192 data: 4 4
10092 112/300 Read block 3808 len 28672
10093 113/300 Read block 3219 len 8192
10094 114/300 Write block 809 len 16384 data: 6 6 6 4
10095 115/300 Read block 3527 len 28672
10096 116/300 Read block 742 len 28672
10097 117/300 Read block 4053 len 16384
10098 118/300 Read block 3589 len 20480
10099 119/300 Read block 500 len 12288
10100 120/300 Read block 3313 len 40960
10101 121/300 Write block 1787 len 12288 data: 3 3 2
10102 122/300 Write block 4293 len 36864 data: 2 2 2 3 3 3 4 4 4
10103 123/300 Read block 2302 len 8192
10104 124/300 Read block 1469 len 12288
10105 125/300 Write block 1210 len 36864 data: 2 2 2 2 2 3 3 3 3
10106 126/300 Read block 2461 len 32768
10107 127/300 Read block 4220 len 32768
10108 128/300 Read block 2393 len 32768
10109 129/300 Write block 4231 len 40960 data: 3 2 2 2 3 3 4 4 4 3
10110 130/300 Flush
10111 131/300 Read block 1681 len 32768
10112 132/300 Write block 1375 len 24576 data: 3 3 3 3 3 2
10113 133/300 Write block 3800 len 24576 data: 2 2 3 2 2 3
10114 134/300 Read block 2752 len 36864
10115 135/300 Write block 886 len 28672 data: 4 4 4 3 3 3 3
10116 136/300 Read block 3495 len 32768
10117 137/300 Write block 2636 len 24576 data: 3 3 3 3 4 3
10118 138/300 Read block 2796 len 32768
10119 139/300 Write block 3929 len 40960 data: 4 4 3 3 4 4 4 5 5 5
10120 140/300 Read block 1114 len 20480
10121 141/300 Read block 605 len 4096
10122 142/300 Write block 537 len 4096 data: 3
10123 143/300 Write block 2072 len 40960 data: 3 3 2 2 2 2 2 2 2 2
10124 144/300 Write block 3120 len 8192 data: 3 3
10125 145/300 Read block 2643 len 28672
10126 146/300 Read block 4663 len 20480
10127 147/300 Read block 2000 len 36864
10128 148/300 Write block 2467 len 36864 data: 3 3 5 5 3 3 3 4 4
10129 149/300 Flush
10130 150/300 Read block 231 len 32768
10131 151/300 Read block 4641 len 28672
10132 152/300 Flush
10133 153/300 Write block 1509 len 40960 data: 4 5 7 7 6 6 6 6 4 4
10134 154/300 Read block 3352 len 28672
10135 155/300 Read block 3182 len 36864
10136 156/300 Write block 4132 len 4096 data: 5
10137 157/300 Write block 2988 len 28672 data: 3 3 3 2 2 2 2
10138 158/300 Write block 1845 len 12288 data: 2 2 3
10139 159/300 Write block 438 len 4096 data: 4
10140 160/300 Read block 3196 len 40960
10141 161/300 Read block 2085 len 20480
10142 162/300 Write block 4393 len 32768 data: 4 4 4 4 4 4 3 3
10143 163/300 Read block 2641 len 8192
10144 164/300 Flush
10145 165/300 Read block 1782 len 40960
10146 166/300 Write block 1551 len 24576 data: 3 3 3 2 2 2
10147 167/300 Write block 1047 len 40960 data: 4 4 4 4 5 5 5 4 4 4
10148 168/300 Write block 4329 len 16384 data: 3 3 4 4
10149 169/300 Write block 817 len 8192 data: 2 2
10150 170/300 Flush
10151 171/300 Read block 4944 len 20480
10152 172/300 Write block 2292 len 28672 data: 5 5 5 4 3 2 3
10153 173/300 Flush
10154 174/300 Flush
10155 175/300 Read block 3601 len 32768
10156 176/300 Read block 2858 len 8192
10157 177/300 Read block 3987 len 28672
10158 178/300 Flush
10159 179/300 Write block 255 len 12288 data: 5 4 3
10160 180/300 Write block 3861 len 24576 data: 3 3 3 3 3 3
10161 181/300 Write block 1232 len 20480 data: 3 3 3 4 4
10162 182/300 Write block 1899 len 20480 data: 3 3 2 2 2
10163 183/300 Read block 3670 len 28672
10164 184/300 Read block 4815 len 36864
10165 185/300 Write block 4774 len 4096 data: 3
10166 186/300 Write block 3612 len 16384 data: 3 3 3 3
10167 187/300 Write block 623 len 32768 data: 2 2 2 2 2 2 3 3
10168 188/300 Read block 886 len 24576
10169 189/300 Read block 1832 len 28672
10170 190/300 Read block 534 len 8192
10171 191/300 Write block 673 len 36864 data: 3 3 3 2 2 2 2 2 2
10172 192/300 Write block 2769 len 32768 data: 3 3 5 6 5 5 5 3
10173 193/300 Write block 1471 len 24576 data: 5 5 5 6 6 5
10174 194/300 Write block 4219 len 20480 data: 3 3 3 3 3
10175 195/300 Write block 565 len 24576 data: 4 2 3 3 3 3
10176 196/300 Write block 4604 len 12288 data: 5 5 4
10177 197/300 Write block 1627 len 8192 data: 2 3
10178 198/300 Read block 4680 len 16384
10179 199/300 Read block 4327 len 12288
10180 200/300 Read block 380 len 36864
10181 201/300 Read block 3542 len 16384
10182 202/300 Read block 2450 len 16384
10183 203/300 Read block 3818 len 8192
10184 204/300 Read block 4541 len 20480
10185 205/300 Read block 1296 len 20480
10186 206/300 Read block 4746 len 8192
10187 207/300 Read block 2580 len 4096
10188 208/300 Flush
10189 209/300 Write block 4457 len 12288 data: 5 5 5
10190 210/300 Write block 2056 len 8192 data: 4 4
10191 211/300 Flush
10192 212/300 Write block 4718 len 16384 data: 3 3 3 3
10193 213/300 Flush
10194 214/300 Read block 75 len 32768
10195 215/300 Read block 393 len 32768
10196 216/300 Read block 360 len 36864
10197 217/300 Flush
10198 218/300 Read block 2773 len 36864
10199 219/300 Read block 4265 len 8192
10200 220/300 Write block 2920 len 16384 data: 2 4 4 4
10201 221/300 Read block 3674 len 20480
10202 222/300 Read block 3681 len 28672
10203 223/300 Write block 4995 len 16384 data: 3 3 3 2
10204 224/300 Read block 3729 len 40960
10205 225/300 Read block 3644 len 28672
10206 226/300 Read block 4333 len 12288
10207 227/300 Read block 1896 len 20480
10208 228/300 Read block 2493 len 28672
10209 229/300 Write block 611 len 8192 data: 3 3
10210 230/300 Write block 3657 len 8192 data: 4 4
10211 231/300 Flush
10212 232/300 Write block 3717 len 4096 data: 5
10213 233/300 Write block 3003 len 40960 data: 4 5 7 6 4 4 2 3 3 3
10214 234/300 Read block 896 len 40960
10215 235/300 Read block 599 len 40960
10216 236/300 Write block 4048 len 8192 data: 3 2
10217 237/300 Write block 4586 len 40960 data: 3 3 3 3 3 3 2 2 2 2
10218 238/300 Read block 2127 len 36864
10219 239/300 Write block 2183 len 24576 data: 3 4 4 5 5 6
10220 240/300 Read block 4256 len 32768
10221 241/300 Read block 2188 len 40960
10222 242/300 Flush
10223 243/300 Read block 128 len 24576
10224 244/300 Read block 1814 len 40960
10225 245/300 Read block 1913 len 8192
10226 246/300 Flush
10227 247/300 Read block 289 len 8192
10228 248/300 Read block 1981 len 16384
10229 249/300 Read block 4140 len 20480
10230 250/300 Read block 1563 len 36864
10231 251/300 Write block 4739 len 24576 data: 3 4 3 3 2 2
10232 252/300 Write block 2010 len 28672 data: 3 3 3 2 2 2 2
10233 253/300 Flush
10234 254/300 Read block 3781 len 8192
10235 255/300 Write block 1748 len 20480 data: 6 6 3 2 2
10236 256/300 Flush
10237 257/300 Write block 2777 len 16384 data: 3 2 2 2
10238 258/300 Write block 514 len 20480 data: 4 4 3 3 2
10239 259/300 Read block 1472 len 16384
10240 260/300 Write block 192 len 12288 data: 4 2 3
10241 261/300 Read block 2318 len 12288
10242 262/300 Write block 3877 len 4096 data: 4
10243 263/300 Read block 144 len 8192
10244 264/300 Read block 888 len 40960
10245 265/300 Write block 3551 len 16384 data: 3 3 3 2
10246 266/300 Write block 4331 len 8192 data: 5 5
10247 267/300 Write block 831 len 40960 data: 2 2 3 3 3 3 4 3 4 4
10248 268/300 Read block 191 len 8192
10249 269/300 Read block 986 len 28672
10250 270/300 Write block 2521 len 28672 data: 3 3 3 3 3 3 3
10251 271/300 Read block 1211 len 20480
10252 272/300 Read block 4896 len 4096
10253 273/300 Read block 3768 len 36864
10254 274/300 Read block 915 len 16384
10255 275/300 Write block 2408 len 20480 data: 3 2 3 3 3
10256 276/300 Flush
10257 277/300 Write block 4633 len 16384 data: 3 2 2 2
10258 278/300 Read block 2221 len 24576
10259 279/300 Read block 3494 len 32768
10260 280/300 Flush
10261 281/300 Read block 703 len 28672
10262 282/300 Read block 1205 len 8192
10263 283/300 Write block 442 len 8192 data: 7 7
10264 284/300 Write block 869 len 12288 data: 2 2 2
10265 285/300 Write block 1866 len 24576 data: 3 4 3 3 3 3
10266 286/300 Flush
10267 287/300 Read block 4062 len 12288
10268 288/300 Read block 3931 len 36864
10269 289/300 Write block 4005 len 24576 data: 3 3 2 2 3 3
10270 290/300 Read block 3604 len 28672
10271 291/300 Write block 3589 len 4096 data: 3
10272 292/300 Read block 3815 len 16384
10273 293/300 Read block 3040 len 28672
10274 294/300 Read block 3466 len 4096
10275 295/300 Read block 2506 len 28672
10276 296/300 Read block 1590 len 8192
10277 297/300 Write block 2247 len 32768 data: 4 4 4 2 2 2 2 2
10278 298/300 Write block 952 len 36864 data: 2 2 2 2 2 2 2 3 3
10279 299/300 Write block 2868 len 32768 data: 5 5 5 4 4 4 4 3
10280 300/300 Read block 3646 len 40960
10281 301/300 Read block 1268 len 20480
10282 Main task has work to do, go find it
10283 got dsc Start(1)
10284 start 1
10285 Dec 04 13:32:52.849 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: ce67ee89-b2db-47c0-bc3f-ac55fdf95f65, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
10286 Replay: started 1, returned:Ok(())
10287 ----------------------------------------------------------------
10288 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10289 States: Active Offline Active
10290 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10291 4019 Acked Write 3 Done Sent Done false
10292 4020 Acked Write 2 Done Sent Done false
10293 4021 Acked Read 7 Done Sent Done false
10294 4022 Acked Write 7 Done Sent Done false
10295 4023 Acked Read 8 Done Sent Done false
10296 4024 Acked Write 4 Done Sent Done false
10297 4025 Acked Write 10 Done Sent Done false
10298 4026 Acked Read 8 Done Sent Done false
10299 4027 Acked Read 6 Done Sent Done false
10300 4028 Acked Write 8 Done Sent Done false
10301 4029 Acked Read 2 Done Sent Done false
10302 4030 Acked Write 6 Done Sent Done false
10303 4031 Acked Flush 0 Done Sent Done false
10304 4032 Acked Write 3 Done Sent Done false
10305 4033 Acked Read 4 Done Sent Done false
10306 4034 Acked Flush 0 Done Sent Done false
10307 4035 Acked Read 3 Done Sent Done false
10308 4036 Acked Read 6 Done Sent Done false
10309 4037 Acked Write 4 Done Sent Done false
10310 4038 Acked Read 4 Done Sent Done false
10311 4039 Acked Read 1 Done Sent Done false
10312 4040 Acked Write 4 Done Sent Done false
10313 4041 Acked Write 1 Done Sent Done false
10314 4042 Acked Read 6 Done Sent Done false
10315 4043 Acked Read 2 Done Sent Done false
10316 4044 Acked Write 8 Done Sent Done false
10317 4045 Acked Read 2 Done Sent Done false
10318 4046 Acked Read 4 Done Sent Done false
10319 4047 Acked Write 8 Done Sent Done false
10320 4048 Acked Flush 0 Done Sent Done false
10321 4049 Acked Read 10 Done Sent Done false
10322 4050 Acked Write 9 Done Sent Done false
10323 4051 Acked Read 4 Done Sent Done false
10324 4052 Acked Read 2 Done Sent Done false
10325 4053 Acked Read 8 Done Sent Done false
10326 4054 Acked Read 8 Done Sent Done false
10327 4055 Acked Read 6 Done Sent Done false
10328 4056 Acked Write 5 Done Sent Done false
10329 4057 Acked Write 4 Done Sent Done false
10330 4058 Acked Read 3 Done Sent Done false
10331 4059 Acked Write 9 Done Sent Done false
10332 4060 Acked Read 9 Done Sent Done false
10333 4061 Acked Write 5 Done Sent Done false
10334 4062 Acked Write 2 Done Sent Done false
10335 4063 Acked Write 4 Done Sent Done false
10336 4064 Acked Flush 0 Done Sent Done false
10337 4065 Acked Read 1 Done Sent Done false
10338 4066 Acked Read 10 Done Sent Done false
10339 4067 Acked Read 5 Done Sent Done false
10340 4068 Acked Write 9 Done Sent Done false
10341 4069 Acked Write 10 Done Sent Done false
10342 4070 Acked Write 10 Done Sent Done false
10343 4071 Acked Write 3 Done Sent Done false
10344 4072 Acked Write 7 Done Sent Done false
10345 4073 Acked Write 6 Done Sent Done false
10346 4074 Acked Write 7 Done Sent Done false
10347 4075 Acked Write 9 Done Sent Done false
10348 4076 Acked Read 1 Done Sent Done false
10349 4077 Acked Read 8 Done Sent Done false
10350 4078 Acked Read 10 Done Sent Done false
10351 4079 Acked Read 10 Done Sent Done false
10352 4080 Acked Write 5 Done Sent Done false
10353 4081 Acked Write 2 Done Sent Done false
10354 4082 Acked Write 3 Done Sent Done false
10355 4083 Acked Read 3 Done Sent Done false
10356 4084 Acked Write 9 Done Sent Done false
10357 4085 Acked Read 9 Done Sent Done false
10358 4086 Acked Read 4 Done Sent Done false
10359 4087 Acked Flush 0 Done Sent Done false
10360 4088 Acked Write 3 Done Sent Done false
10361 4089 Acked Write 9 Done Sent Done false
10362 4090 Acked Write 4 Done Sent Done false
10363 4091 Acked Read 10 Done Sent Done false
10364 4092 Acked Read 7 Done Sent Done false
10365 4093 Acked Read 3 Done Sent Done false
10366 4094 Acked Write 4 Done Sent Done false
10367 4095 Acked Write 6 Done Sent Done false
10368 4096 Acked Flush 0 Done Sent Done false
10369 4097 Acked Read 9 Done Sent Done false
10370 4098 Acked Write 7 Done Sent Done false
10371 4099 Acked Read 6 Done Sent Done false
10372 4100 Acked Read 4 Done Sent Done false
10373 4101 Acked Write 8 Done Sent Done false
10374 4102 Acked Write 8 Done Sent Done false
10375 4103 Acked Read 5 Done Sent Done false
10376 4104 Acked Flush 0 Done Sent Done false
10377 4105 Acked Read 7 Done Sent Done false
10378 4106 Acked Write 10 Done Sent Done false
10379 4107 Acked Read 7 Done Sent Done false
10380 4108 Acked Read 3 Done Sent Done false
10381 4109 Acked Write 10 Done Sent Done false
10382 4110 Acked Write 7 Done Sent Done false
10383 4111 Acked Read 2 Done Sent Done false
10384 4112 Acked Write 9 Done Sent Done false
10385 4113 Acked Read 4 Done Sent Done false
10386 4114 Acked Read 6 Done Sent Done false
10387 4115 Acked Read 1 Done Sent Done false
10388 4116 Acked Flush 0 Done Sent Done false
10389 4117 Acked Read 5 Done Sent Done false
10390 4118 Acked Write 2 Done Sent Done false
10391 4119 Acked Write 6 Done Sent Done false
10392 4120 Acked Write 4 Done Sent Done false
10393 4121 Acked Read 8 Done Sent Done false
10394 4122 Acked Read 6 Done Sent Done false
10395 4123 Acked Read 4 Done Sent Done false
10396 4124 Acked Read 7 Done Sent Done false
10397 4125 Acked Read 7 Done Sent Done false
10398 4126 Acked Flush 0 Done Sent Done false
10399 4127 Acked Read 8 Done Sent Done false
10400 4128 Acked Read 3 Done Sent Done false
10401 4129 Acked Write 2 Done Sent Done false
10402 4130 Acked Read 7 Done Sent Done false
10403 4131 Acked Read 2 Done Sent Done false
10404 4132 Acked Write 4 Done Sent Done false
10405 4133 Acked Read 7 Done Sent Done false
10406 4134 Acked Read 7 Done Sent Done false
10407 4135 Acked Read 4 Done Sent Done false
10408 4136 Acked Read 5 Done Sent Done false
10409 4137 Acked Read 3 Done Sent Done false
10410 4138 Acked Read 10 Done Sent Done false
10411 4139 Acked Write 3 Done Sent Done false
10412 4140 Acked Write 9 Done Sent Done false
10413 4141 Acked Read 2 Done Sent Done false
10414 4142 Acked Read 3 Done Sent Done false
10415 4143 Acked Write 9 Done Sent Done false
10416 4144 Acked Read 8 Done Sent Done false
10417 4145 Acked Read 8 Done Sent Done false
10418 4146 Acked Read 8 Done Sent Done false
10419 4147 Acked Write 10 Done Sent Done false
10420 4148 Acked Flush 0 Done Sent Done false
10421 4149 Acked Read 8 Done Sent Done false
10422 4150 Acked Write 6 Done Sent Done false
10423 4151 Acked Write 6 Done Sent Done false
10424 4152 Acked Read 9 Done Sent Done false
10425 4153 Acked Write 7 Done Sent Done false
10426 4154 Acked Read 8 Done Sent Done false
10427 4155 Acked Write 6 Done Sent Done false
10428 4156 Acked Read 8 Done Sent Done false
10429 4157 Acked Write 10 Done Sent Done false
10430 4158 Acked Read 5 Done Sent Done false
10431 4159 Acked Read 1 Done Sent Done false
10432 4160 Acked Write 1 Done Sent Done false
10433 4161 Acked Write 10 Done Sent Done false
10434 4162 Acked Write 2 Done Sent Done false
10435 4163 Acked Read 7 Done Sent Done false
10436 4164 Acked Read 5 Done Sent Done false
10437 4165 Acked Read 9 Done Sent Done false
10438 4166 Acked Write 9 Done Sent Done false
10439 4167 Acked Flush 0 Done Sent Done false
10440 4168 Acked Read 8 Done Sent Done false
10441 4169 Acked Read 7 Done Sent Done false
10442 4170 Acked Flush 0 Done Sent Done false
10443 4171 Acked Write 10 Done Sent Done false
10444 4172 Acked Read 7 Done Sent Done false
10445 4173 Acked Read 9 Done Sent Done false
10446 4174 Acked Write 1 Done Sent Done false
10447 4175 Acked Write 7 Done Sent Done false
10448 4176 Acked Write 3 Done Sent Done false
10449 4177 Acked Write 1 Done Sent Done false
10450 4178 Acked Read 10 Done Sent Done false
10451 4179 Acked Read 5 Done Sent Done false
10452 4180 Acked Write 8 Done Sent Done false
10453 4181 Acked Read 2 Done Sent Done false
10454 4182 Acked Flush 0 Done Sent Done false
10455 4183 Acked Read 10 Done Sent Done false
10456 4184 Acked Write 6 Done Sent Done false
10457 4185 Acked Write 10 Done Sent Done false
10458 4186 Acked Write 4 Done Sent Done false
10459 4187 Acked Write 2 Done Sent Done false
10460 4188 Acked Flush 0 Done Sent Done false
10461 4189 Acked Read 5 Done Sent Done false
10462 4190 Acked Write 7 Done Sent Done false
10463 4191 Acked Flush 0 Done Sent Done false
10464 4192 Acked Flush 0 Done Sent Done false
10465 4193 Acked Read 8 Done Sent Done false
10466 4194 Acked Read 2 Done Sent Done false
10467 4195 Acked Read 7 Done Sent Done false
10468 4196 Acked Flush 0 Done Sent Done false
10469 4197 Acked Write 3 Done Sent Done false
10470 4198 Acked Write 6 Done Sent Done false
10471 4199 Acked Write 5 Done Sent Done false
10472 4200 Acked Write 5 Done Sent Done false
10473 4201 Acked Read 7 Done Sent Done false
10474 4202 Acked Read 9 Done Sent Done false
10475 4203 Acked Write 1 Done Sent Done false
10476 4204 Acked Write 4 Done Sent Done false
10477 4205 Acked Write 8 Done Sent Done false
10478 4206 Acked Read 6 Done Sent Done false
10479 4207 Acked Read 7 Done Sent Done false
10480 4208 Acked Read 2 Done Sent Done false
10481 4209 Acked Write 9 Done Sent Done false
10482 4210 Acked Write 8 Done Sent Done false
10483 4211 Acked Write 6 Done Sent Done false
10484 4212 Acked Write 5 Done Sent Done false
10485 4213 Acked Write 6 Done Sent Done false
10486 4214 Acked Write 3 Done Sent Done false
10487 4215 Acked Write 2 Done Sent Done false
10488 4216 Acked Read 4 Done Sent Done false
10489 4217 Acked Read 3 Done Sent Done false
10490 4218 Acked Read 9 Done Sent Done false
10491 4219 Acked Read 4 Done Sent Done false
10492 4220 Acked Read 4 Done Sent Done false
10493 4221 Acked Read 2 Done Sent Done false
10494 4222 Acked Read 5 Done Sent Done false
10495 4223 Acked Read 5 Done Sent Done false
10496 4224 Acked Read 2 Done Sent Done false
10497 4225 Acked Read 1 Done Sent Done false
10498 4226 Acked Flush 0 Done Sent Done false
10499 4227 Acked Write 3 Done Sent Done false
10500 4228 Acked Write 2 Done Sent Done false
10501 4229 Acked Flush 0 Done Sent Done false
10502 4230 Acked Write 4 Done Sent Done false
10503 4231 Acked Flush 0 Done Sent Done false
10504 4232 Acked Read 8 Done Sent Done false
10505 4233 Acked Read 8 Done Sent Done false
10506 4234 Acked Read 9 Done Sent Done false
10507 4235 Acked Flush 0 Done Sent Done false
10508 4236 Acked Read 9 Done Sent Done false
10509 4237 Acked Read 2 Done Sent Done false
10510 4238 Acked Write 4 Done Sent Done false
10511 4239 Acked Read 5 Done Sent Done false
10512 4240 Acked Read 7 Done Sent Done false
10513 4241 Acked Write 4 Done Sent Done false
10514 4242 Acked Read 10 Done Sent Done false
10515 4243 Acked Read 7 Done Sent Done false
10516 4244 Acked Read 3 Done Sent Done false
10517 4245 Acked Read 5 Done Sent Done false
10518 4246 Acked Read 7 Done Sent Done false
10519 4247 Acked Write 2 Done Sent Done false
10520 4248 Acked Write 2 Done Sent Done false
10521 4249 Acked Flush 0 Done Sent Done false
10522 4250 Acked Write 1 Done Sent Done false
10523 4251 Acked Write 10 Done Sent Done false
10524 4252 Acked Read 10 Done Sent Done false
10525 4253 Acked Read 10 Done Sent Done false
10526 4254 Acked Write 2 Done Sent Done false
10527 4255 Acked Write 10 Done Sent Done false
10528 4256 Acked Read 9 Done Sent Done false
10529 4257 Acked Write 6 Done Sent Done false
10530 4258 Acked Read 8 Done Sent Done false
10531 4259 Acked Read 10 Done Sent Done false
10532 4260 Acked Flush 0 Done Sent Done false
10533 4261 Acked Read 6 Done Sent Done false
10534 4262 Acked Read 10 Done Sent Done false
10535 4263 Acked Read 2 Done Sent Done false
10536 4264 Acked Flush 0 Done Sent Done false
10537 4265 Acked Read 2 Done Sent Done false
10538 4266 Acked Read 4 Done Sent Done false
10539 4267 Acked Read 5 Done Sent Done false
10540 4268 Acked Read 9 Done Sent Done false
10541 4269 Acked Write 6 Done Sent Done false
10542 4270 Acked Write 7 Done Sent Done false
10543 4271 Acked Flush 0 Done Sent Done false
10544 4272 Acked Read 2 Done Sent Done false
10545 4273 Acked Write 5 Done Sent Done false
10546 4274 Acked Flush 0 Done Sent Done false
10547 4275 Acked Write 4 Done Sent Done false
10548 4276 Acked Write 5 Done Sent Done false
10549 4277 Acked Read 4 Done Sent Done false
10550 4278 Acked Write 3 Done Sent Done false
10551 4279 Acked Read 3 Done Sent Done false
10552 4280 Acked Write 1 Done Sent Done false
10553 4281 Acked Read 2 Done Sent Done false
10554 4282 Acked Read 10 Done Sent Done false
10555 4283 Acked Write 4 Done Sent Done false
10556 4284 Acked Write 2 Done Sent Done false
10557 4285 Acked Write 10 Done Sent Done false
10558 4286 Acked Read 2 Done Sent Done false
10559 4287 Acked Read 7 Done Sent Done false
10560 4288 Acked Write 7 Done Sent Done false
10561 4289 Acked Read 5 Done Sent Done false
10562 4290 Acked Read 1 Done Sent Done false
10563 4291 Acked Read 9 Done Sent Done false
10564 4292 Acked Read 4 Done Sent Done false
10565 4293 Acked Write 5 Done Sent Done false
10566 4294 Acked Flush 0 Done Sent Done false
10567 4295 Acked Write 4 Done Sent Done false
10568 4296 Acked Read 6 Done Sent Done false
10569 4297 Acked Read 8 Done Sent Done false
10570 4298 Acked Flush 0 Done Sent Done false
10571 4299 Acked Read 7 Done Sent Done false
10572 4300 Acked Read 2 Done Sent Done false
10573 4301 Acked Write 2 Done Sent Done false
10574 4302 Acked Write 3 Done Sent Done false
10575 4303 Acked Write 6 Done Sent Done false
10576 4304 Acked Flush 0 Done Sent Done false
10577 4305 Acked Read 3 Done Sent Done false
10578 4306 Acked Read 9 Done Sent Done false
10579 4307 Acked Write 6 Done Sent Done false
10580 4308 Acked Read 7 Done Sent Done false
10581 4309 Acked Write 1 Done Sent Done false
10582 4310 Acked Read 4 Done Sent Done false
10583 4311 Acked Read 7 Done Sent Done false
10584 4312 Acked Read 1 Done Sent Done false
10585 4313 Acked Read 7 Done Sent Done false
10586 4314 Acked Read 2 Done Sent Done false
10587 4315 Acked Write 8 Done Sent Done false
10588 4316 Acked Write 9 Done Sent Done false
10589 4317 Acked Write 8 Done Sent Done false
10590 4318 Acked Read 10 Done Sent Done false
10591 4319 Acked Read 5 Done Sent Done false
10592 STATES DS:0 DS:1 DS:2 TOTAL
10593 Sent 0 301 0 301
10594 Done 301 0 301 602
10595 Skipped 0 0 0 0
10596 Error 0 0 0 0
10597 Last Flush: 4304 4018 4304
10598 Downstairs last five completed: 4018 4017 4016 4015 4014
10599 Upstairs last five completed: 4319 4318 4317 4316 4315
10600 CLIENT: Up:0 ds:301 act:2
10601 ----------------------------------------------------------------
10602 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10603 States: Active Offline Active
10604 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10605 4019 Acked Write 3 Done Sent Done false
10606 4020 Acked Write 2 Done Sent Done false
10607 4021 Acked Read 7 Done Sent Done false
10608 4022 Acked Write 7 Done Sent Done false
10609 4023 Acked Read 8 Done Sent Done false
10610 4024 Acked Write 4 Done Sent Done false
10611 4025 Acked Write 10 Done Sent Done false
10612 4026 Acked Read 8 Done Sent Done false
10613 4027 Acked Read 6 Done Sent Done false
10614 4028 Acked Write 8 Done Sent Done false
10615 4029 Acked Read 2 Done Sent Done false
10616 4030 Acked Write 6 Done Sent Done false
10617 4031 Acked Flush 0 Done Sent Done false
10618 4032 Acked Write 3 Done Sent Done false
10619 4033 Acked Read 4 Done Sent Done false
10620 4034 Acked Flush 0 Done Sent Done false
10621 4035 Acked Read 3 Done Sent Done false
10622 4036 Acked Read 6 Done Sent Done false
10623 4037 Acked Write 4 Done Sent Done false
10624 4038 Acked Read 4 Done Sent Done false
10625 4039 Acked Read 1 Done Sent Done false
10626 4040 Acked Write 4 Done Sent Done false
10627 4041 Acked Write 1 Done Sent Done false
10628 4042 Acked Read 6 Done Sent Done false
10629 4043 Acked Read 2 Done Sent Done false
10630 4044 Acked Write 8 Done Sent Done false
10631 4045 Acked Read 2 Done Sent Done false
10632 4046 Acked Read 4 Done Sent Done false
10633 4047 Acked Write 8 Done Sent Done false
10634 4048 Acked Flush 0 Done Sent Done false
10635 4049 Acked Read 10 Done Sent Done false
10636 4050 Acked Write 9 Done Sent Done false
10637 4051 Acked Read 4 Done Sent Done false
10638 4052 Acked Read 2 Done Sent Done false
10639 4053 Acked Read 8 Done Sent Done false
10640 4054 Acked Read 8 Done Sent Done false
10641 4055 Acked Read 6 Done Sent Done false
10642 4056 Acked Write 5 Done Sent Done false
10643 4057 Acked Write 4 Done Sent Done false
10644 4058 Acked Read 3 Done Sent Done false
10645 4059 Acked Write 9 Done Sent Done false
10646 4060 Acked Read 9 Done Sent Done false
10647 4061 Acked Write 5 Done Sent Done false
10648 4062 Acked Write 2 Done Sent Done false
10649 4063 Acked Write 4 Done Sent Done false
10650 4064 Acked Flush 0 Done Sent Done false
10651 4065 Acked Read 1 Done Sent Done false
10652 4066 Acked Read 10 Done Sent Done false
10653 4067 Acked Read 5 Done Sent Done false
10654 4068 Acked Write 9 Done Sent Done false
10655 4069 Acked Write 10 Done Sent Done false
10656 4070 Acked Write 10 Done Sent Done false
10657 4071 Acked Write 3 Done Sent Done false
10658 4072 Acked Write 7 Done Sent Done false
10659 4073 Acked Write 6 Done Sent Done false
10660 4074 Acked Write 7 Done Sent Done false
10661 4075 Acked Write 9 Done Sent Done false
10662 4076 Acked Read 1 Done Sent Done false
10663 4077 Acked Read 8 Done Sent Done false
10664 4078 Acked Read 10 Done Sent Done false
10665 4079 Acked Read 10 Done Sent Done false
10666 4080 Acked Write 5 Done Sent Done false
10667 4081 Acked Write 2 Done Sent Done false
10668 4082 Acked Write 3 Done Sent Done false
10669 4083 Acked Read 3 Done Sent Done false
10670 4084 Acked Write 9 Done Sent Done false
10671 4085 Acked Read 9 Done Sent Done false
10672 4086 Acked Read 4 Done Sent Done false
10673 4087 Acked Flush 0 Done Sent Done false
10674 4088 Acked Write 3 Done Sent Done false
10675 4089 Acked Write 9 Done Sent Done false
10676 4090 Acked Write 4 Done Sent Done false
10677 4091 Acked Read 10 Done Sent Done false
10678 4092 Acked Read 7 Done Sent Done false
10679 4093 Acked Read 3 Done Sent Done false
10680 4094 Acked Write 4 Done Sent Done false
10681 4095 Acked Write 6 Done Sent Done false
10682 4096 Acked Flush 0 Done Sent Done false
10683 4097 Acked Read 9 Done Sent Done false
10684 4098 Acked Write 7 Done Sent Done false
10685 4099 Acked Read 6 Done Sent Done false
10686 4100 Acked Read 4 Done Sent Done false
10687 4101 Acked Write 8 Done Sent Done false
10688 4102 Acked Write 8 Done Sent Done false
10689 4103 Acked Read 5 Done Sent Done false
10690 4104 Acked Flush 0 Done Sent Done false
10691 4105 Acked Read 7 Done Sent Done false
10692 4106 Acked Write 10 Done Sent Done false
10693 4107 Acked Read 7 Done Sent Done false
10694 4108 Acked Read 3 Done Sent Done false
10695 4109 Acked Write 10 Done Sent Done false
10696 4110 Acked Write 7 Done Sent Done false
10697 4111 Acked Read 2 Done Sent Done false
10698 4112 Acked Write 9 Done Sent Done false
10699 4113 Acked Read 4 Done Sent Done false
10700 4114 Acked Read 6 Done Sent Done false
10701 4115 Acked Read 1 Done Sent Done false
10702 4116 Acked Flush 0 Done Sent Done false
10703 4117 Acked Read 5 Done Sent Done false
10704 4118 Acked Write 2 Done Sent Done false
10705 4119 Acked Write 6 Done Sent Done false
10706 4120 Acked Write 4 Done Sent Done false
10707 4121 Acked Read 8 Done Sent Done false
10708 4122 Acked Read 6 Done Sent Done false
10709 4123 Acked Read 4 Done Sent Done false
10710 4124 Acked Read 7 Done Sent Done false
10711 4125 Acked Read 7 Done Sent Done false
10712 4126 Acked Flush 0 Done Sent Done false
10713 4127 Acked Read 8 Done Sent Done false
10714 4128 Acked Read 3 Done Sent Done false
10715 4129 Acked Write 2 Done Sent Done false
10716 4130 Acked Read 7 Done Sent Done false
10717 4131 Acked Read 2 Done Sent Done false
10718 4132 Acked Write 4 Done Sent Done false
10719 4133 Acked Read 7 Done Sent Done false
10720 4134 Acked Read 7 Done Sent Done false
10721 4135 Acked Read 4 Done Sent Done false
10722 4136 Acked Read 5 Done Sent Done false
10723 4137 Acked Read 3 Done Sent Done false
10724 4138 Acked Read 10 Done Sent Done false
10725 4139 Acked Write 3 Done Sent Done false
10726 4140 Acked Write 9 Done Sent Done false
10727 4141 Acked Read 2 Done Sent Done false
10728 4142 Acked Read 3 Done Sent Done false
10729 4143 Acked Write 9 Done Sent Done false
10730 4144 Acked Read 8 Done Sent Done false
10731 4145 Acked Read 8 Done Sent Done false
10732 4146 Acked Read 8 Done Sent Done false
10733 4147 Acked Write 10 Done Sent Done false
10734 4148 Acked Flush 0 Done Sent Done false
10735 4149 Acked Read 8 Done Sent Done false
10736 4150 Acked Write 6 Done Sent Done false
10737 4151 Acked Write 6 Done Sent Done false
10738 4152 Acked Read 9 Done Sent Done false
10739 4153 Acked Write 7 Done Sent Done false
10740 4154 Acked Read 8 Done Sent Done false
10741 4155 Acked Write 6 Done Sent Done false
10742 4156 Acked Read 8 Done Sent Done false
10743 4157 Acked Write 10 Done Sent Done false
10744 4158 Acked Read 5 Done Sent Done false
10745 4159 Acked Read 1 Done Sent Done false
10746 4160 Acked Write 1 Done Sent Done false
10747 4161 Acked Write 10 Done Sent Done false
10748 4162 Acked Write 2 Done Sent Done false
10749 4163 Acked Read 7 Done Sent Done false
10750 4164 Acked Read 5 Done Sent Done false
10751 4165 Acked Read 9 Done Sent Done false
10752 4166 Acked Write 9 Done Sent Done false
10753 4167 Acked Flush 0 Done Sent Done false
10754 4168 Acked Read 8 Done Sent Done false
10755 4169 Acked Read 7 Done Sent Done false
10756 4170 Acked Flush 0 Done Sent Done false
10757 4171 Acked Write 10 Done Sent Done false
10758 4172 Acked Read 7 Done Sent Done false
10759 4173 Acked Read 9 Done Sent Done false
10760 4174 Acked Write 1 Done Sent Done false
10761 4175 Acked Write 7 Done Sent Done false
10762 4176 Acked Write 3 Done Sent Done false
10763 4177 Acked Write 1 Done Sent Done false
10764 4178 Acked Read 10 Done Sent Done false
10765 4179 Acked Read 5 Done Sent Done false
10766 4180 Acked Write 8 Done Sent Done false
10767 4181 Acked Read 2 Done Sent Done false
10768 4182 Acked Flush 0 Done Sent Done false
10769 4183 Acked Read 10 Done Sent Done false
10770 4184 Acked Write 6 Done Sent Done false
10771 4185 Acked Write 10 Done Sent Done false
10772 4186 Acked Write 4 Done Sent Done false
10773 4187 Acked Write 2 Done Sent Done false
10774 4188 Acked Flush 0 Done Sent Done false
10775 4189 Acked Read 5 Done Sent Done false
10776 4190 Acked Write 7 Done Sent Done false
10777 4191 Acked Flush 0 Done Sent Done false
10778 4192 Acked Flush 0 Done Sent Done false
10779 4193 Acked Read 8 Done Sent Done false
10780 4194 Acked Read 2 Done Sent Done false
10781 4195 Acked Read 7 Done Sent Done false
10782 4196 Acked Flush 0 Done Sent Done false
10783 4197 Acked Write 3 Done Sent Done false
10784 4198 Acked Write 6 Done Sent Done false
10785 4199 Acked Write 5 Done Sent Done false
10786 4200 Acked Write 5 Done Sent Done false
10787 4201 Acked Read 7 Done Sent Done false
10788 4202 Acked Read 9 Done Sent Done false
10789 4203 Acked Write 1 Done Sent Done false
10790 4204 Acked Write 4 Done Sent Done false
10791 4205 Acked Write 8 Done Sent Done false
10792 4206 Acked Read 6 Done Sent Done false
10793 4207 Acked Read 7 Done Sent Done false
10794 4208 Acked Read 2 Done Sent Done false
10795 4209 Acked Write 9 Done Sent Done false
10796 4210 Acked Write 8 Done Sent Done false
10797 4211 Acked Write 6 Done Sent Done false
10798 4212 Acked Write 5 Done Sent Done false
10799 4213 Acked Write 6 Done Sent Done false
10800 4214 Acked Write 3 Done Sent Done false
10801 4215 Acked Write 2 Done Sent Done false
10802 4216 Acked Read 4 Done Sent Done false
10803 4217 Acked Read 3 Done Sent Done false
10804 4218 Acked Read 9 Done Sent Done false
10805 4219 Acked Read 4 Done Sent Done false
10806 4220 Acked Read 4 Done Sent Done false
10807 4221 Acked Read 2 Done Sent Done false
10808 4222 Acked Read 5 Done Sent Done false
10809 4223 Acked Read 5 Done Sent Done false
10810 4224 Acked Read 2 Done Sent Done false
10811 4225 Acked Read 1 Done Sent Done false
10812 4226 Acked Flush 0 Done Sent Done false
10813 4227 Acked Write 3 Done Sent Done false
10814 4228 Acked Write 2 Done Sent Done false
10815 4229 Acked Flush 0 Done Sent Done false
10816 4230 Acked Write 4 Done Sent Done false
10817 4231 Acked Flush 0 Done Sent Done false
10818 4232 Acked Read 8 Done Sent Done false
10819 4233 Acked Read 8 Done Sent Done false
10820 4234 Acked Read 9 Done Sent Done false
10821 4235 Acked Flush 0 Done Sent Done false
10822 4236 Acked Read 9 Done Sent Done false
10823 4237 Acked Read 2 Done Sent Done false
10824 4238 Acked Write 4 Done Sent Done false
10825 4239 Acked Read 5 Done Sent Done false
10826 4240 Acked Read 7 Done Sent Done false
10827 4241 Acked Write 4 Done Sent Done false
10828 4242 Acked Read 10 Done Sent Done false
10829 4243 Acked Read 7 Done Sent Done false
10830 4244 Acked Read 3 Done Sent Done false
10831 4245 Acked Read 5 Done Sent Done false
10832 4246 Acked Read 7 Done Sent Done false
10833 4247 Acked Write 2 Done Sent Done false
10834 4248 Acked Write 2 Done Sent Done false
10835 4249 Acked Flush 0 Done Sent Done false
10836 4250 Acked Write 1 Done Sent Done false
10837 4251 Acked Write 10 Done Sent Done false
10838 4252 Acked Read 10 Done Sent Done false
10839 4253 Acked Read 10 Done Sent Done false
10840 4254 Acked Write 2 Done Sent Done false
10841 4255 Acked Write 10 Done Sent Done false
10842 4256 Acked Read 9 Done Sent Done false
10843 4257 Acked Write 6 Done Sent Done false
10844 4258 Acked Read 8 Done Sent Done false
10845 4259 Acked Read 10 Done Sent Done false
10846 4260 Acked Flush 0 Done Sent Done false
10847 4261 Acked Read 6 Done Sent Done false
10848 4262 Acked Read 10 Done Sent Done false
10849 4263 Acked Read 2 Done Sent Done false
10850 4264 Acked Flush 0 Done Sent Done false
10851 4265 Acked Read 2 Done Sent Done false
10852 4266 Acked Read 4 Done Sent Done false
10853 4267 Acked Read 5 Done Sent Done false
10854 4268 Acked Read 9 Done Sent Done false
10855 4269 Acked Write 6 Done Sent Done false
10856 4270 Acked Write 7 Done Sent Done false
10857 4271 Acked Flush 0 Done Sent Done false
10858 4272 Acked Read 2 Done Sent Done false
10859 4273 Acked Write 5 Done Sent Done false
10860 4274 Acked Flush 0 Done Sent Done false
10861 4275 Acked Write 4 Done Sent Done false
10862 4276 Acked Write 5 Done Sent Done false
10863 4277 Acked Read 4 Done Sent Done false
10864 4278 Acked Write 3 Done Sent Done false
10865 4279 Acked Read 3 Done Sent Done false
10866 4280 Acked Write 1 Done Sent Done false
10867 4281 Acked Read 2 Done Sent Done false
10868 4282 Acked Read 10 Done Sent Done false
10869 4283 Acked Write 4 Done Sent Done false
10870 4284 Acked Write 2 Done Sent Done false
10871 4285 Acked Write 10 Done Sent Done false
10872 4286 Acked Read 2 Done Sent Done false
10873 4287 Acked Read 7 Done Sent Done false
10874 4288 Acked Write 7 Done Sent Done false
10875 4289 Acked Read 5 Done Sent Done false
10876 4290 Acked Read 1 Done Sent Done false
10877 4291 Acked Read 9 Done Sent Done false
10878 4292 Acked Read 4 Done Sent Done false
10879 4293 Acked Write 5 Done Sent Done false
10880 4294 Acked Flush 0 Done Sent Done false
10881 4295 Acked Write 4 Done Sent Done false
10882 4296 Acked Read 6 Done Sent Done false
10883 4297 Acked Read 8 Done Sent Done false
10884 4298 Acked Flush 0 Done Sent Done false
10885 4299 Acked Read 7 Done Sent Done false
10886 4300 Acked Read 2 Done Sent Done false
10887 4301 Acked Write 2 Done Sent Done false
10888 4302 Acked Write 3 Done Sent Done false
10889 4303 Acked Write 6 Done Sent Done false
10890 4304 Acked Flush 0 Done Sent Done false
10891 4305 Acked Read 3 Done Sent Done false
10892 4306 Acked Read 9 Done Sent Done false
10893 4307 Acked Write 6 Done Sent Done false
10894 4308 Acked Read 7 Done Sent Done false
10895 4309 Acked Write 1 Done Sent Done false
10896 4310 Acked Read 4 Done Sent Done false
10897 4311 Acked Read 7 Done Sent Done false
10898 4312 Acked Read 1 Done Sent Done false
10899 4313 Acked Read 7 Done Sent Done false
10900 4314 Acked Read 2 Done Sent Done false
10901 4315 Acked Write 8 Done Sent Done false
10902 4316 Acked Write 9 Done Sent Done false
10903 4317 Acked Write 8 Done Sent Done false
10904 4318 Acked Read 10 Done Sent Done false
10905 4319 Acked Read 5 Done Sent Done false
10906 4320 Acked Flush 0 Done Sent Done false
10907 STATES DS:0 DS:1 DS:2 TOTAL
10908 Sent 0 302 0 302
10909 Done 302 0 302 604
10910 Skipped 0 0 0 0
10911 Error 0 0 0 0
10912 Last Flush: 4320 4018 4320
10913 Downstairs last five completed: 4018 4017 4016 4015 4014
10914 Upstairs last five completed: 4320 4319 4318 4317 4316
10915 CLIENT: Up:0 ds:302 act:2
109162024-12-04T13:32:58.607ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
109172024-12-04T13:32:58.607ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
109182024-12-04T13:32:58.607ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
109192024-12-04T13:32:58.607ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
109202024-12-04T13:32:58.607ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
10921 [8820] got start action
10922 [8820] I am going to restart
10923 Starting downstairs at port 8820
10924 Make output file at "/tmp/dsc/downstairs-8820.txt"
10925 [8820][1] reports Starting
10926 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1214)
10927 [8820][1] reports Running
10928 ----------------------------------------------------------------
10929 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10930 States: Active Offline Active
10931 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10932 4019 Acked Write 3 Done Sent Done false
10933 4020 Acked Write 2 Done Sent Done false
10934 4021 Acked Read 7 Done Sent Done false
10935 4022 Acked Write 7 Done Sent Done false
10936 4023 Acked Read 8 Done Sent Done false
10937 4024 Acked Write 4 Done Sent Done false
10938 4025 Acked Write 10 Done Sent Done false
10939 4026 Acked Read 8 Done Sent Done false
10940 4027 Acked Read 6 Done Sent Done false
10941 4028 Acked Write 8 Done Sent Done false
10942 4029 Acked Read 2 Done Sent Done false
10943 4030 Acked Write 6 Done Sent Done false
10944 4031 Acked Flush 0 Done Sent Done false
10945 4032 Acked Write 3 Done Sent Done false
10946 4033 Acked Read 4 Done Sent Done false
10947 4034 Acked Flush 0 Done Sent Done false
10948 4035 Acked Read 3 Done Sent Done false
10949 4036 Acked Read 6 Done Sent Done false
10950 4037 Acked Write 4 Done Sent Done false
10951 4038 Acked Read 4 Done Sent Done false
10952 4039 Acked Read 1 Done Sent Done false
10953 4040 Acked Write 4 Done Sent Done false
10954 4041 Acked Write 1 Done Sent Done false
10955 4042 Acked Read 6 Done Sent Done false
10956 4043 Acked Read 2 Done Sent Done false
10957 4044 Acked Write 8 Done Sent Done false
10958 4045 Acked Read 2 Done Sent Done false
10959 4046 Acked Read 4 Done Sent Done false
10960 4047 Acked Write 8 Done Sent Done false
10961 4048 Acked Flush 0 Done Sent Done false
10962 4049 Acked Read 10 Done Sent Done false
10963 4050 Acked Write 9 Done Sent Done false
10964 4051 Acked Read 4 Done Sent Done false
10965 4052 Acked Read 2 Done Sent Done false
10966 4053 Acked Read 8 Done Sent Done false
10967 4054 Acked Read 8 Done Sent Done false
10968 4055 Acked Read 6 Done Sent Done false
10969 4056 Acked Write 5 Done Sent Done false
10970 4057 Acked Write 4 Done Sent Done false
10971 4058 Acked Read 3 Done Sent Done false
10972 4059 Acked Write 9 Done Sent Done false
10973 4060 Acked Read 9 Done Sent Done false
10974 4061 Acked Write 5 Done Sent Done false
10975 4062 Acked Write 2 Done Sent Done false
10976 4063 Acked Write 4 Done Sent Done false
10977 4064 Acked Flush 0 Done Sent Done false
10978 4065 Acked Read 1 Done Sent Done false
10979 4066 Acked Read 10 Done Sent Done false
10980 4067 Acked Read 5 Done Sent Done false
10981 4068 Acked Write 9 Done Sent Done false
10982 4069 Acked Write 10 Done Sent Done false
10983 4070 Acked Write 10 Done Sent Done false
10984 4071 Acked Write 3 Done Sent Done false
10985 4072 Acked Write 7 Done Sent Done false
10986 4073 Acked Write 6 Done Sent Done false
10987 4074 Acked Write 7 Done Sent Done false
10988 4075 Acked Write 9 Done Sent Done false
10989 4076 Acked Read 1 Done Sent Done false
10990 4077 Acked Read 8 Done Sent Done false
10991 4078 Acked Read 10 Done Sent Done false
10992 4079 Acked Read 10 Done Sent Done false
10993 4080 Acked Write 5 Done Sent Done false
10994 4081 Acked Write 2 Done Sent Done false
10995 4082 Acked Write 3 Done Sent Done false
10996 4083 Acked Read 3 Done Sent Done false
10997 4084 Acked Write 9 Done Sent Done false
10998 4085 Acked Read 9 Done Sent Done false
10999 4086 Acked Read 4 Done Sent Done false
11000 4087 Acked Flush 0 Done Sent Done false
11001 4088 Acked Write 3 Done Sent Done false
11002 4089 Acked Write 9 Done Sent Done false
11003 4090 Acked Write 4 Done Sent Done false
11004 4091 Acked Read 10 Done Sent Done false
11005 4092 Acked Read 7 Done Sent Done false
11006 4093 Acked Read 3 Done Sent Done false
11007 4094 Acked Write 4 Done Sent Done false
11008 4095 Acked Write 6 Done Sent Done false
11009 4096 Acked Flush 0 Done Sent Done false
11010 4097 Acked Read 9 Done Sent Done false
11011 4098 Acked Write 7 Done Sent Done false
11012 4099 Acked Read 6 Done Sent Done false
11013 4100 Acked Read 4 Done Sent Done false
11014 4101 Acked Write 8 Done Sent Done false
11015 4102 Acked Write 8 Done Sent Done false
11016 4103 Acked Read 5 Done Sent Done false
11017 4104 Acked Flush 0 Done Sent Done false
11018 4105 Acked Read 7 Done Sent Done false
11019 4106 Acked Write 10 Done Sent Done false
11020 4107 Acked Read 7 Done Sent Done false
11021 4108 Acked Read 3 Done Sent Done false
11022 4109 Acked Write 10 Done Sent Done false
11023 4110 Acked Write 7 Done Sent Done false
11024 4111 Acked Read 2 Done Sent Done false
11025 4112 Acked Write 9 Done Sent Done false
11026 4113 Acked Read 4 Done Sent Done false
11027 4114 Acked Read 6 Done Sent Done false
11028 4115 Acked Read 1 Done Sent Done false
11029 4116 Acked Flush 0 Done Sent Done false
11030 4117 Acked Read 5 Done Sent Done false
11031 4118 Acked Write 2 Done Sent Done false
11032 4119 Acked Write 6 Done Sent Done false
11033 4120 Acked Write 4 Done Sent Done false
11034 4121 Acked Read 8 Done Sent Done false
11035 4122 Acked Read 6 Done Sent Done false
11036 4123 Acked Read 4 Done Sent Done false
11037 4124 Acked Read 7 Done Sent Done false
11038 4125 Acked Read 7 Done Sent Done false
11039 4126 Acked Flush 0 Done Sent Done false
11040 4127 Acked Read 8 Done Sent Done false
11041 4128 Acked Read 3 Done Sent Done false
11042 4129 Acked Write 2 Done Sent Done false
11043 4130 Acked Read 7 Done Sent Done false
11044 4131 Acked Read 2 Done Sent Done false
11045 4132 Acked Write 4 Done Sent Done false
11046 4133 Acked Read 7 Done Sent Done false
11047 4134 Acked Read 7 Done Sent Done false
11048 4135 Acked Read 4 Done Sent Done false
11049 4136 Acked Read 5 Done Sent Done false
11050 4137 Acked Read 3 Done Sent Done false
11051 4138 Acked Read 10 Done Sent Done false
11052 4139 Acked Write 3 Done Sent Done false
11053 4140 Acked Write 9 Done Sent Done false
11054 4141 Acked Read 2 Done Sent Done false
11055 4142 Acked Read 3 Done Sent Done false
11056 4143 Acked Write 9 Done Sent Done false
11057 4144 Acked Read 8 Done Sent Done false
11058 4145 Acked Read 8 Done Sent Done false
11059 4146 Acked Read 8 Done Sent Done false
11060 4147 Acked Write 10 Done Sent Done false
11061 4148 Acked Flush 0 Done Sent Done false
11062 4149 Acked Read 8 Done Sent Done false
11063 4150 Acked Write 6 Done Sent Done false
11064 4151 Acked Write 6 Done Sent Done false
11065 4152 Acked Read 9 Done Sent Done false
11066 4153 Acked Write 7 Done Sent Done false
11067 4154 Acked Read 8 Done Sent Done false
11068 4155 Acked Write 6 Done Sent Done false
11069 4156 Acked Read 8 Done Sent Done false
11070 4157 Acked Write 10 Done Sent Done false
11071 4158 Acked Read 5 Done Sent Done false
11072 4159 Acked Read 1 Done Sent Done false
11073 4160 Acked Write 1 Done Sent Done false
11074 4161 Acked Write 10 Done Sent Done false
11075 4162 Acked Write 2 Done Sent Done false
11076 4163 Acked Read 7 Done Sent Done false
11077 4164 Acked Read 5 Done Sent Done false
11078 4165 Acked Read 9 Done Sent Done false
11079 4166 Acked Write 9 Done Sent Done false
11080 4167 Acked Flush 0 Done Sent Done false
11081 4168 Acked Read 8 Done Sent Done false
11082 4169 Acked Read 7 Done Sent Done false
11083 4170 Acked Flush 0 Done Sent Done false
11084 4171 Acked Write 10 Done Sent Done false
11085 4172 Acked Read 7 Done Sent Done false
11086 4173 Acked Read 9 Done Sent Done false
11087 4174 Acked Write 1 Done Sent Done false
11088 4175 Acked Write 7 Done Sent Done false
11089 4176 Acked Write 3 Done Sent Done false
11090 4177 Acked Write 1 Done Sent Done false
11091 4178 Acked Read 10 Done Sent Done false
11092 4179 Acked Read 5 Done Sent Done false
11093 4180 Acked Write 8 Done Sent Done false
11094 4181 Acked Read 2 Done Sent Done false
11095 4182 Acked Flush 0 Done Sent Done false
11096 4183 Acked Read 10 Done Sent Done false
11097 4184 Acked Write 6 Done Sent Done false
11098 4185 Acked Write 10 Done Sent Done false
11099 4186 Acked Write 4 Done Sent Done false
11100 4187 Acked Write 2 Done Sent Done false
11101 4188 Acked Flush 0 Done Sent Done false
11102 4189 Acked Read 5 Done Sent Done false
11103 4190 Acked Write 7 Done Sent Done false
11104 4191 Acked Flush 0 Done Sent Done false
11105 4192 Acked Flush 0 Done Sent Done false
11106 4193 Acked Read 8 Done Sent Done false
11107 4194 Acked Read 2 Done Sent Done false
11108 4195 Acked Read 7 Done Sent Done false
11109 4196 Acked Flush 0 Done Sent Done false
11110 4197 Acked Write 3 Done Sent Done false
11111 4198 Acked Write 6 Done Sent Done false
11112 4199 Acked Write 5 Done Sent Done false
11113 4200 Acked Write 5 Done Sent Done false
11114 4201 Acked Read 7 Done Sent Done false
11115 4202 Acked Read 9 Done Sent Done false
11116 4203 Acked Write 1 Done Sent Done false
11117 4204 Acked Write 4 Done Sent Done false
11118 4205 Acked Write 8 Done Sent Done false
11119 4206 Acked Read 6 Done Sent Done false
11120 4207 Acked Read 7 Done Sent Done false
11121 4208 Acked Read 2 Done Sent Done false
11122 4209 Acked Write 9 Done Sent Done false
11123 4210 Acked Write 8 Done Sent Done false
11124 4211 Acked Write 6 Done Sent Done false
11125 4212 Acked Write 5 Done Sent Done false
11126 4213 Acked Write 6 Done Sent Done false
11127 4214 Acked Write 3 Done Sent Done false
11128 4215 Acked Write 2 Done Sent Done false
11129 4216 Acked Read 4 Done Sent Done false
11130 4217 Acked Read 3 Done Sent Done false
11131 4218 Acked Read 9 Done Sent Done false
11132 4219 Acked Read 4 Done Sent Done false
11133 4220 Acked Read 4 Done Sent Done false
11134 4221 Acked Read 2 Done Sent Done false
11135 4222 Acked Read 5 Done Sent Done false
11136 4223 Acked Read 5 Done Sent Done false
11137 4224 Acked Read 2 Done Sent Done false
11138 4225 Acked Read 1 Done Sent Done false
11139 4226 Acked Flush 0 Done Sent Done false
11140 4227 Acked Write 3 Done Sent Done false
11141 4228 Acked Write 2 Done Sent Done false
11142 4229 Acked Flush 0 Done Sent Done false
11143 4230 Acked Write 4 Done Sent Done false
11144 4231 Acked Flush 0 Done Sent Done false
11145 4232 Acked Read 8 Done Sent Done false
11146 4233 Acked Read 8 Done Sent Done false
11147 4234 Acked Read 9 Done Sent Done false
11148 4235 Acked Flush 0 Done Sent Done false
11149 4236 Acked Read 9 Done Sent Done false
11150 4237 Acked Read 2 Done Sent Done false
11151 4238 Acked Write 4 Done Sent Done false
11152 4239 Acked Read 5 Done Sent Done false
11153 4240 Acked Read 7 Done Sent Done false
11154 4241 Acked Write 4 Done Sent Done false
11155 4242 Acked Read 10 Done Sent Done false
11156 4243 Acked Read 7 Done Sent Done false
11157 4244 Acked Read 3 Done Sent Done false
11158 4245 Acked Read 5 Done Sent Done false
11159 4246 Acked Read 7 Done Sent Done false
11160 4247 Acked Write 2 Done Sent Done false
11161 4248 Acked Write 2 Done Sent Done false
11162 4249 Acked Flush 0 Done Sent Done false
11163 4250 Acked Write 1 Done Sent Done false
11164 4251 Acked Write 10 Done Sent Done false
11165 4252 Acked Read 10 Done Sent Done false
11166 4253 Acked Read 10 Done Sent Done false
11167 4254 Acked Write 2 Done Sent Done false
11168 4255 Acked Write 10 Done Sent Done false
11169 4256 Acked Read 9 Done Sent Done false
11170 4257 Acked Write 6 Done Sent Done false
11171 4258 Acked Read 8 Done Sent Done false
11172 4259 Acked Read 10 Done Sent Done false
11173 4260 Acked Flush 0 Done Sent Done false
11174 4261 Acked Read 6 Done Sent Done false
11175 4262 Acked Read 10 Done Sent Done false
11176 4263 Acked Read 2 Done Sent Done false
11177 4264 Acked Flush 0 Done Sent Done false
11178 4265 Acked Read 2 Done Sent Done false
11179 4266 Acked Read 4 Done Sent Done false
11180 4267 Acked Read 5 Done Sent Done false
11181 4268 Acked Read 9 Done Sent Done false
11182 4269 Acked Write 6 Done Sent Done false
11183 4270 Acked Write 7 Done Sent Done false
11184 4271 Acked Flush 0 Done Sent Done false
11185 4272 Acked Read 2 Done Sent Done false
11186 4273 Acked Write 5 Done Sent Done false
11187 4274 Acked Flush 0 Done Sent Done false
11188 4275 Acked Write 4 Done Sent Done false
11189 4276 Acked Write 5 Done Sent Done false
11190 4277 Acked Read 4 Done Sent Done false
11191 4278 Acked Write 3 Done Sent Done false
11192 4279 Acked Read 3 Done Sent Done false
11193 4280 Acked Write 1 Done Sent Done false
11194 4281 Acked Read 2 Done Sent Done false
11195 4282 Acked Read 10 Done Sent Done false
11196 4283 Acked Write 4 Done Sent Done false
11197 4284 Acked Write 2 Done Sent Done false
11198 4285 Acked Write 10 Done Sent Done false
11199 4286 Acked Read 2 Done Sent Done false
11200 4287 Acked Read 7 Done Sent Done false
11201 4288 Acked Write 7 Done Sent Done false
11202 4289 Acked Read 5 Done Sent Done false
11203 4290 Acked Read 1 Done Sent Done false
11204 4291 Acked Read 9 Done Sent Done false
11205 4292 Acked Read 4 Done Sent Done false
11206 4293 Acked Write 5 Done Sent Done false
11207 4294 Acked Flush 0 Done Sent Done false
11208 4295 Acked Write 4 Done Sent Done false
11209 4296 Acked Read 6 Done Sent Done false
11210 4297 Acked Read 8 Done Sent Done false
11211 4298 Acked Flush 0 Done Sent Done false
11212 4299 Acked Read 7 Done Sent Done false
11213 4300 Acked Read 2 Done Sent Done false
11214 4301 Acked Write 2 Done Sent Done false
11215 4302 Acked Write 3 Done Sent Done false
11216 4303 Acked Write 6 Done Sent Done false
11217 4304 Acked Flush 0 Done Sent Done false
11218 4305 Acked Read 3 Done Sent Done false
11219 4306 Acked Read 9 Done Sent Done false
11220 4307 Acked Write 6 Done Sent Done false
11221 4308 Acked Read 7 Done Sent Done false
11222 4309 Acked Write 1 Done Sent Done false
11223 4310 Acked Read 4 Done Sent Done false
11224 4311 Acked Read 7 Done Sent Done false
11225 4312 Acked Read 1 Done Sent Done false
11226 4313 Acked Read 7 Done Sent Done false
11227 4314 Acked Read 2 Done Sent Done false
11228 4315 Acked Write 8 Done Sent Done false
11229 4316 Acked Write 9 Done Sent Done false
11230 4317 Acked Write 8 Done Sent Done false
11231 4318 Acked Read 10 Done Sent Done false
11232 4319 Acked Read 5 Done Sent Done false
11233 4320 Acked Flush 0 Done Sent Done false
11234 STATES DS:0 DS:1 DS:2 TOTAL
11235 Sent 0 302 0 302
11236 Done 302 0 302 604
11237 Skipped 0 0 0 0
11238 Error 0 0 0 0
11239 Last Flush: 4320 4018 4320
11240 Downstairs last five completed: 4018 4017 4016 4015 4014
11241 Upstairs last five completed: 4320 4319 4318 4317 4316
11242 CLIENT: Up:0 ds:302 act:2
11243 ----------------------------------------------------------------
11244 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11245 States: Active Offline Active
11246 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11247 4019 Acked Write 3 Done Sent Done false
11248 4020 Acked Write 2 Done Sent Done false
11249 4021 Acked Read 7 Done Sent Done false
11250 4022 Acked Write 7 Done Sent Done false
11251 4023 Acked Read 8 Done Sent Done false
11252 4024 Acked Write 4 Done Sent Done false
11253 4025 Acked Write 10 Done Sent Done false
11254 4026 Acked Read 8 Done Sent Done false
11255 4027 Acked Read 6 Done Sent Done false
11256 4028 Acked Write 8 Done Sent Done false
11257 4029 Acked Read 2 Done Sent Done false
11258 4030 Acked Write 6 Done Sent Done false
11259 4031 Acked Flush 0 Done Sent Done false
11260 4032 Acked Write 3 Done Sent Done false
11261 4033 Acked Read 4 Done Sent Done false
11262 4034 Acked Flush 0 Done Sent Done false
11263 4035 Acked Read 3 Done Sent Done false
11264 4036 Acked Read 6 Done Sent Done false
11265 4037 Acked Write 4 Done Sent Done false
11266 4038 Acked Read 4 Done Sent Done false
11267 4039 Acked Read 1 Done Sent Done false
11268 4040 Acked Write 4 Done Sent Done false
11269 4041 Acked Write 1 Done Sent Done false
11270 4042 Acked Read 6 Done Sent Done false
11271 4043 Acked Read 2 Done Sent Done false
11272 4044 Acked Write 8 Done Sent Done false
11273 4045 Acked Read 2 Done Sent Done false
11274 4046 Acked Read 4 Done Sent Done false
11275 4047 Acked Write 8 Done Sent Done false
11276 4048 Acked Flush 0 Done Sent Done false
11277 4049 Acked Read 10 Done Sent Done false
11278 4050 Acked Write 9 Done Sent Done false
11279 4051 Acked Read 4 Done Sent Done false
11280 4052 Acked Read 2 Done Sent Done false
11281 4053 Acked Read 8 Done Sent Done false
11282 4054 Acked Read 8 Done Sent Done false
11283 4055 Acked Read 6 Done Sent Done false
11284 4056 Acked Write 5 Done Sent Done false
11285 4057 Acked Write 4 Done Sent Done false
11286 4058 Acked Read 3 Done Sent Done false
11287 4059 Acked Write 9 Done Sent Done false
11288 4060 Acked Read 9 Done Sent Done false
11289 4061 Acked Write 5 Done Sent Done false
11290 4062 Acked Write 2 Done Sent Done false
11291 4063 Acked Write 4 Done Sent Done false
11292 4064 Acked Flush 0 Done Sent Done false
11293 4065 Acked Read 1 Done Sent Done false
11294 4066 Acked Read 10 Done Sent Done false
11295 4067 Acked Read 5 Done Sent Done false
11296 4068 Acked Write 9 Done Sent Done false
11297 4069 Acked Write 10 Done Sent Done false
11298 4070 Acked Write 10 Done Sent Done false
11299 4071 Acked Write 3 Done Sent Done false
11300 4072 Acked Write 7 Done Sent Done false
11301 4073 Acked Write 6 Done Sent Done false
11302 4074 Acked Write 7 Done Sent Done false
11303 4075 Acked Write 9 Done Sent Done false
11304 4076 Acked Read 1 Done Sent Done false
11305 4077 Acked Read 8 Done Sent Done false
11306 4078 Acked Read 10 Done Sent Done false
11307 4079 Acked Read 10 Done Sent Done false
11308 4080 Acked Write 5 Done Sent Done false
11309 4081 Acked Write 2 Done Sent Done false
11310 4082 Acked Write 3 Done Sent Done false
11311 4083 Acked Read 3 Done Sent Done false
11312 4084 Acked Write 9 Done Sent Done false
11313 4085 Acked Read 9 Done Sent Done false
11314 4086 Acked Read 4 Done Sent Done false
11315 4087 Acked Flush 0 Done Sent Done false
11316 4088 Acked Write 3 Done Sent Done false
11317 4089 Acked Write 9 Done Sent Done false
11318 4090 Acked Write 4 Done Sent Done false
11319 4091 Acked Read 10 Done Sent Done false
11320 4092 Acked Read 7 Done Sent Done false
11321 4093 Acked Read 3 Done Sent Done false
11322 4094 Acked Write 4 Done Sent Done false
11323 4095 Acked Write 6 Done Sent Done false
11324 4096 Acked Flush 0 Done Sent Done false
11325 4097 Acked Read 9 Done Sent Done false
11326 4098 Acked Write 7 Done Sent Done false
11327 4099 Acked Read 6 Done Sent Done false
11328 4100 Acked Read 4 Done Sent Done false
11329 4101 Acked Write 8 Done Sent Done false
11330 4102 Acked Write 8 Done Sent Done false
11331 4103 Acked Read 5 Done Sent Done false
11332 4104 Acked Flush 0 Done Sent Done false
11333 4105 Acked Read 7 Done Sent Done false
11334 4106 Acked Write 10 Done Sent Done false
11335 4107 Acked Read 7 Done Sent Done false
11336 4108 Acked Read 3 Done Sent Done false
11337 4109 Acked Write 10 Done Sent Done false
11338 4110 Acked Write 7 Done Sent Done false
11339 4111 Acked Read 2 Done Sent Done false
11340 4112 Acked Write 9 Done Sent Done false
11341 4113 Acked Read 4 Done Sent Done false
11342 4114 Acked Read 6 Done Sent Done false
11343 4115 Acked Read 1 Done Sent Done false
11344 4116 Acked Flush 0 Done Sent Done false
11345 4117 Acked Read 5 Done Sent Done false
11346 4118 Acked Write 2 Done Sent Done false
11347 4119 Acked Write 6 Done Sent Done false
11348 4120 Acked Write 4 Done Sent Done false
11349 4121 Acked Read 8 Done Sent Done false
11350 4122 Acked Read 6 Done Sent Done false
11351 4123 Acked Read 4 Done Sent Done false
11352 4124 Acked Read 7 Done Sent Done false
11353 4125 Acked Read 7 Done Sent Done false
11354 4126 Acked Flush 0 Done Sent Done false
11355 4127 Acked Read 8 Done Sent Done false
11356 4128 Acked Read 3 Done Sent Done false
11357 4129 Acked Write 2 Done Sent Done false
11358 4130 Acked Read 7 Done Sent Done false
11359 4131 Acked Read 2 Done Sent Done false
11360 4132 Acked Write 4 Done Sent Done false
11361 4133 Acked Read 7 Done Sent Done false
11362 4134 Acked Read 7 Done Sent Done false
11363 4135 Acked Read 4 Done Sent Done false
11364 4136 Acked Read 5 Done Sent Done false
11365 4137 Acked Read 3 Done Sent Done false
11366 4138 Acked Read 10 Done Sent Done false
11367 4139 Acked Write 3 Done Sent Done false
11368 4140 Acked Write 9 Done Sent Done false
11369 4141 Acked Read 2 Done Sent Done false
11370 4142 Acked Read 3 Done Sent Done false
11371 4143 Acked Write 9 Done Sent Done false
11372 4144 Acked Read 8 Done Sent Done false
11373 4145 Acked Read 8 Done Sent Done false
11374 4146 Acked Read 8 Done Sent Done false
11375 4147 Acked Write 10 Done Sent Done false
11376 4148 Acked Flush 0 Done Sent Done false
11377 4149 Acked Read 8 Done Sent Done false
11378 4150 Acked Write 6 Done Sent Done false
11379 4151 Acked Write 6 Done Sent Done false
11380 4152 Acked Read 9 Done Sent Done false
11381 4153 Acked Write 7 Done Sent Done false
11382 4154 Acked Read 8 Done Sent Done false
11383 4155 Acked Write 6 Done Sent Done false
11384 4156 Acked Read 8 Done Sent Done false
11385 4157 Acked Write 10 Done Sent Done false
11386 4158 Acked Read 5 Done Sent Done false
11387 4159 Acked Read 1 Done Sent Done false
11388 4160 Acked Write 1 Done Sent Done false
11389 4161 Acked Write 10 Done Sent Done false
11390 4162 Acked Write 2 Done Sent Done false
11391 4163 Acked Read 7 Done Sent Done false
11392 4164 Acked Read 5 Done Sent Done false
11393 4165 Acked Read 9 Done Sent Done false
11394 4166 Acked Write 9 Done Sent Done false
11395 4167 Acked Flush 0 Done Sent Done false
11396 4168 Acked Read 8 Done Sent Done false
11397 4169 Acked Read 7 Done Sent Done false
11398 4170 Acked Flush 0 Done Sent Done false
11399 4171 Acked Write 10 Done Sent Done false
11400 4172 Acked Read 7 Done Sent Done false
11401 4173 Acked Read 9 Done Sent Done false
11402 4174 Acked Write 1 Done Sent Done false
11403 4175 Acked Write 7 Done Sent Done false
11404 4176 Acked Write 3 Done Sent Done false
11405 4177 Acked Write 1 Done Sent Done false
11406 4178 Acked Read 10 Done Sent Done false
11407 4179 Acked Read 5 Done Sent Done false
11408 4180 Acked Write 8 Done Sent Done false
11409 4181 Acked Read 2 Done Sent Done false
11410 4182 Acked Flush 0 Done Sent Done false
11411 4183 Acked Read 10 Done Sent Done false
11412 4184 Acked Write 6 Done Sent Done false
11413 4185 Acked Write 10 Done Sent Done false
11414 4186 Acked Write 4 Done Sent Done false
11415 4187 Acked Write 2 Done Sent Done false
11416 4188 Acked Flush 0 Done Sent Done false
11417 4189 Acked Read 5 Done Sent Done false
11418 4190 Acked Write 7 Done Sent Done false
11419 4191 Acked Flush 0 Done Sent Done false
11420 4192 Acked Flush 0 Done Sent Done false
11421 4193 Acked Read 8 Done Sent Done false
11422 4194 Acked Read 2 Done Sent Done false
11423 4195 Acked Read 7 Done Sent Done false
11424 4196 Acked Flush 0 Done Sent Done false
11425 4197 Acked Write 3 Done Sent Done false
11426 4198 Acked Write 6 Done Sent Done false
11427 4199 Acked Write 5 Done Sent Done false
11428 4200 Acked Write 5 Done Sent Done false
11429 4201 Acked Read 7 Done Sent Done false
11430 4202 Acked Read 9 Done Sent Done false
11431 4203 Acked Write 1 Done Sent Done false
11432 4204 Acked Write 4 Done Sent Done false
11433 4205 Acked Write 8 Done Sent Done false
11434 4206 Acked Read 6 Done Sent Done false
11435 4207 Acked Read 7 Done Sent Done false
11436 4208 Acked Read 2 Done Sent Done false
11437 4209 Acked Write 9 Done Sent Done false
11438 4210 Acked Write 8 Done Sent Done false
11439 4211 Acked Write 6 Done Sent Done false
11440 4212 Acked Write 5 Done Sent Done false
11441 4213 Acked Write 6 Done Sent Done false
11442 4214 Acked Write 3 Done Sent Done false
11443 4215 Acked Write 2 Done Sent Done false
11444 4216 Acked Read 4 Done Sent Done false
11445 4217 Acked Read 3 Done Sent Done false
11446 4218 Acked Read 9 Done Sent Done false
11447 4219 Acked Read 4 Done Sent Done false
11448 4220 Acked Read 4 Done Sent Done false
11449 4221 Acked Read 2 Done Sent Done false
11450 4222 Acked Read 5 Done Sent Done false
11451 4223 Acked Read 5 Done Sent Done false
11452 4224 Acked Read 2 Done Sent Done false
11453 4225 Acked Read 1 Done Sent Done false
11454 4226 Acked Flush 0 Done Sent Done false
11455 4227 Acked Write 3 Done Sent Done false
11456 4228 Acked Write 2 Done Sent Done false
11457 4229 Acked Flush 0 Done Sent Done false
11458 4230 Acked Write 4 Done Sent Done false
11459 4231 Acked Flush 0 Done Sent Done false
11460 4232 Acked Read 8 Done Sent Done false
11461 4233 Acked Read 8 Done Sent Done false
11462 4234 Acked Read 9 Done Sent Done false
11463 4235 Acked Flush 0 Done Sent Done false
11464 4236 Acked Read 9 Done Sent Done false
11465 4237 Acked Read 2 Done Sent Done false
11466 4238 Acked Write 4 Done Sent Done false
11467 4239 Acked Read 5 Done Sent Done false
11468 4240 Acked Read 7 Done Sent Done false
11469 4241 Acked Write 4 Done Sent Done false
11470 4242 Acked Read 10 Done Sent Done false
11471 4243 Acked Read 7 Done Sent Done false
11472 4244 Acked Read 3 Done Sent Done false
11473 4245 Acked Read 5 Done Sent Done false
11474 4246 Acked Read 7 Done Sent Done false
11475 4247 Acked Write 2 Done Sent Done false
11476 4248 Acked Write 2 Done Sent Done false
11477 4249 Acked Flush 0 Done Sent Done false
11478 4250 Acked Write 1 Done Sent Done false
11479 4251 Acked Write 10 Done Sent Done false
11480 4252 Acked Read 10 Done Sent Done false
11481 4253 Acked Read 10 Done Sent Done false
11482 4254 Acked Write 2 Done Sent Done false
11483 4255 Acked Write 10 Done Sent Done false
11484 4256 Acked Read 9 Done Sent Done false
11485 4257 Acked Write 6 Done Sent Done false
11486 4258 Acked Read 8 Done Sent Done false
11487 4259 Acked Read 10 Done Sent Done false
11488 4260 Acked Flush 0 Done Sent Done false
11489 4261 Acked Read 6 Done Sent Done false
11490 4262 Acked Read 10 Done Sent Done false
11491 4263 Acked Read 2 Done Sent Done false
11492 4264 Acked Flush 0 Done Sent Done false
11493 4265 Acked Read 2 Done Sent Done false
11494 4266 Acked Read 4 Done Sent Done false
11495 4267 Acked Read 5 Done Sent Done false
11496 4268 Acked Read 9 Done Sent Done false
11497 4269 Acked Write 6 Done Sent Done false
11498 4270 Acked Write 7 Done Sent Done false
11499 4271 Acked Flush 0 Done Sent Done false
11500 4272 Acked Read 2 Done Sent Done false
11501 4273 Acked Write 5 Done Sent Done false
11502 4274 Acked Flush 0 Done Sent Done false
11503 4275 Acked Write 4 Done Sent Done false
11504 4276 Acked Write 5 Done Sent Done false
11505 4277 Acked Read 4 Done Sent Done false
11506 4278 Acked Write 3 Done Sent Done false
11507 4279 Acked Read 3 Done Sent Done false
11508 4280 Acked Write 1 Done Sent Done false
11509 4281 Acked Read 2 Done Sent Done false
11510 4282 Acked Read 10 Done Sent Done false
11511 4283 Acked Write 4 Done Sent Done false
11512 4284 Acked Write 2 Done Sent Done false
11513 4285 Acked Write 10 Done Sent Done false
11514 4286 Acked Read 2 Done Sent Done false
11515 4287 Acked Read 7 Done Sent Done false
11516 4288 Acked Write 7 Done Sent Done false
11517 4289 Acked Read 5 Done Sent Done false
11518 4290 Acked Read 1 Done Sent Done false
11519 4291 Acked Read 9 Done Sent Done false
11520 4292 Acked Read 4 Done Sent Done false
11521 4293 Acked Write 5 Done Sent Done false
11522 4294 Acked Flush 0 Done Sent Done false
11523 4295 Acked Write 4 Done Sent Done false
11524 4296 Acked Read 6 Done Sent Done false
11525 4297 Acked Read 8 Done Sent Done false
11526 4298 Acked Flush 0 Done Sent Done false
11527 4299 Acked Read 7 Done Sent Done false
11528 4300 Acked Read 2 Done Sent Done false
11529 4301 Acked Write 2 Done Sent Done false
11530 4302 Acked Write 3 Done Sent Done false
11531 4303 Acked Write 6 Done Sent Done false
11532 4304 Acked Flush 0 Done Sent Done false
11533 4305 Acked Read 3 Done Sent Done false
11534 4306 Acked Read 9 Done Sent Done false
11535 4307 Acked Write 6 Done Sent Done false
11536 4308 Acked Read 7 Done Sent Done false
11537 4309 Acked Write 1 Done Sent Done false
11538 4310 Acked Read 4 Done Sent Done false
11539 4311 Acked Read 7 Done Sent Done false
11540 4312 Acked Read 1 Done Sent Done false
11541 4313 Acked Read 7 Done Sent Done false
11542 4314 Acked Read 2 Done Sent Done false
11543 4315 Acked Write 8 Done Sent Done false
11544 4316 Acked Write 9 Done Sent Done false
11545 4317 Acked Write 8 Done Sent Done false
11546 4318 Acked Read 10 Done Sent Done false
11547 4319 Acked Read 5 Done Sent Done false
11548 4320 Acked Flush 0 Done Sent Done false
11549 STATES DS:0 DS:1 DS:2 TOTAL
11550 Sent 0 302 0 302
11551 Done 302 0 302 604
11552 Skipped 0 0 0 0
11553 Error 0 0 0 0
11554 Last Flush: 4320 4018 4320
11555 Downstairs last five completed: 4018 4017 4016 4015 4014
11556 Upstairs last five completed: 4320 4319 4318 4317 4316
11557 CLIENT: Up:0 ds:302 act:2
115582024-12-04T13:33:08.609ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115592024-12-04T13:33:08.609ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48162) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115602024-12-04T13:33:08.610ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115612024-12-04T13:33:08.610ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115622024-12-04T13:33:08.610ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115632024-12-04T13:33:08.610ZINFOcrucible: send last flush ID to this DS: 4018 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115642024-12-04T13:33:08.611ZINFOcrucible: Replied this last flush ID: 4018 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115652024-12-04T13:33:08.611ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115662024-12-04T13:33:08.611ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115672024-12-04T13:33:08.611ZINFOcrucible: [1] client re-new 302 jobs since flush 4018 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115682024-12-04T13:33:08.611ZINFOcrucible: [1] Replayed 302 jobs since flush: 4018 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11569 ----------------------------------------------------------------
11570 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11571 Downstairs last five completed: 4320 4319 4318 4317 4316
11572 Upstairs last five completed: 4320 4319 4318 4317 4316
11573 CLIENT: Up:0 ds:0 act:3
11574 Replay: All jobs finished
11575 Main task has work to do, go find it
11576 got dsc Stop(1)
11577 stop 1
11578 Dec 04 13:33:08.857 INFO[8820] Got stop action so:false kr:false
11579 request completed, latency_us: 327, response_code: 204, uri: /stop/cid/1, method: POST, req_id: f4447d2f-0fa5-4719-91f5-bbcb99aa3cf8, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
11580 Dec 04 13:33:08.858 INFO request completed, latency_us: 120, response_code:{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:33:08.858059974Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
11581 200, uri: /state/cid/1, method: GET, req_id: 3a6f6d14-48d4-4b05-9958-c4383ae7592c, remote_addr: 127.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:33:08.858128594Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
11582 0.0.1:37804, local_addr{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:33:08.858165595Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
11583 : 127.0.0.1:9998{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:33:08.858209935Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
11584 
115852024-12-04T13:33:08.858ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
115862024-12-04T13:33:08.858ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
11587 [8820] Exited with: ExitStatus(unix_wait_status(9))
11588 [8820][1] reports Exit
11589 Dec 04 13:33:12.860 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/1, method: GET, req_id: b883d13e-d14d-43fc-978a-753b19dcd65a, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
11590 001/300 Write block 1016 len 24576 data: 3 3 3 3 4 3
11591 002/300 Write block 2987 len 12288 data: 3 4 4
11592 003/300 Read block 538 len 20480
11593 004/300 Read block 3127 len 40960
11594 005/300 Write block 2988 len 40960 data: 5 5 4 3 3 3 3 2 3 3
11595 006/300 Write block 4214 len 24576 data: 2 3 3 3 3 4
11596 007/300 Write block 4982 len 40960 data: 2 2 2 3 3 3 3 4 3 3
11597 008/300 Write block 3454 len 24576 data: 2 2 2 2 2 2
11598 009/300 Read block 1596 len 16384
11599 010/300 Read block 1327 len 4096
11600 011/300 Read block 100 len 36864
11601 012/300 Read block 2092 len 24576
11602 013/300 Write block 4523 len 32768 data: 5 5 4 5 5 4 2 2
11603 014/300 Flush
11604 015/300 Read block 671 len 40960
11605 016/300 Read block 739 len 12288
11606 017/300 Read block 442 len 40960
11607 018/300 Write block 1649 len 40960 data: 2 3 4 4 4 4 5 5 5 6
11608 019/300 Read block 2695 len 12288
11609 020/300 Read block 950 len 28672
11610 021/300 Read block 351 len 16384
11611 022/300 Read block 4274 len 32768
11612 023/300 Flush
11613 024/300 Write block 1053 len 32768 data: 6 5 5 5 6 6 6 5
11614 025/300 Read block 1480 len 12288
11615 026/300 Write block 4764 len 20480 data: 3 3 2 2 2
11616 027/300 Write block 619 len 28672 data: 2 2 2 2 3 3 3
11617 028/300 Read block 4256 len 8192
11618 029/300 Read block 4761 len 4096
11619 030/300 Read block 1063 len 40960
11620 031/300 Read block 4050 len 16384
11621 032/300 Flush
11622 033/300 Read block 2374 len 20480
11623 034/300 Read block 4039 len 8192
11624 035/300 Flush
11625 036/300 Write block 1562 len 40960 data: 3 3 3 3 3 2 2 2 2 2
11626 037/300 Write block 3915 len 32768 data: 3 2 2 2 2 2 3 3
11627 038/300 Write block 1728 len 36864 data: 4 3 4 3 3 2 2 2 2
11628 039/300 Write block 2320 len 32768 data: 3 4 4 4 4 3 3 3
11629 040/300 Read block 1711 len 8192
11630 041/300 Read block 4829 len 20480
11631 042/300 Flush
11632 043/300 Write block 4840 len 32768 data: 3 3 3 3 3 3 3 3
11633 044/300 Read block 934 len 28672
11634 045/300 Write block 1043 len 4096 data: 2
11635 046/300 Read block 2084 len 12288
11636 047/300 Write block 4730 len 40960 data: 3 3 3 3 4 4 4 2 2 4
11637 048/300 Read block 4658 len 8192
11638 049/300 Flush
11639 050/300 Write block 2211 len 32768 data: 3 3 3 2 2 2 2 2
11640 051/300 Read block 2273 len 12288
11641 052/300 Read block 858 len 32768
11642 053/300 Read block 889 len 4096
11643 054/300 Read block 908 len 20480
11644 055/300 Read block 1509 len 16384
11645 056/300 Write block 4945 len 36864 data: 2 4 4 3 3 3 3 3 4
11646 057/300 Write block 4357 len 40960 data: 3 2 2 2 2 2 2 2 2 2
11647 058/300 Write block 3700 len 40960 data: 2 3 3 3 3 3 3 3 2 2
11648 059/300 Write block 835 len 4096 data: 4
11649 060/300 Write block 2906 len 4096 data: 2
11650 061/300 Read block 4869 len 16384
11651 062/300 Read block 1519 len 20480
11652 063/300 Read block 540 len 32768
11653 064/300 Read block 2346 len 16384
11654 065/300 Write block 2337 len 16384 data: 3 3 2 3
11655 066/300 Write block 3759 len 32768 data: 3 3 3 3 4 4 5 5
11656 067/300 Write block 4764 len 12288 data: 4 4 3
11657 068/300 Read block 4263 len 20480
11658 069/300 Read block 1975 len 20480
11659 070/300 Write block 1065 len 28672 data: 4 5 5 6 6 8 8
11660 071/300 Read block 1452 len 24576
11661 072/300 Flush
11662 073/300 Read block 4172 len 8192
11663 074/300 Write block 3925 len 20480 data: 3 3 3 4 5
11664 075/300 Write block 2629 len 20480 data: 3 2 4 4 4
11665 076/300 Write block 1455 len 16384 data: 5 5 6 6
11666 077/300 Read block 3641 len 16384
11667 078/300 Write block 60 len 40960 data: 3 3 2 2 3 3 3 3 3 3
11668 079/300 Read block 3157 len 36864
11669 080/300 Read block 4362 len 20480
11670 081/300 Read block 3375 len 12288
11671 082/300 Read block 2046 len 8192
11672 083/300 Read block 1611 len 32768
11673 084/300 Read block 1992 len 40960
11674 085/300 Read block 3370 len 12288
11675 086/300 Read block 2754 len 40960
11676 087/300 Write block 4374 len 20480 data: 2 2 2 2 3
11677 088/300 Read block 248 len 4096
11678 089/300 Write block 4979 len 4096 data: 3
11679 090/300 Read block 3346 len 8192
11680 091/300 Flush
11681 092/300 Write block 2948 len 24576 data: 2 2 3 3 3 3
11682 093/300 Read block 4551 len 16384
11683 094/300 Read block 431 len 32768
11684 095/300 Write block 4826 len 24576 data: 3 3 4 4 4 4
11685 096/300 Read block 2550 len 8192
11686 097/300 Write block 3364 len 36864 data: 5 5 5 4 4 5 4 3 2
11687 098/300 Read block 3800 len 32768
11688 099/300 Write block 1438 len 20480 data: 3 3 4 4 4
11689 100/300 Read block 1699 len 32768
11690 101/300 Write block 4200 len 12288 data: 4 3 4
11691 102/300 Write block 896 len 36864 data: 2 2 2 2 2 2 2 2 2
11692 103/300 Flush
11693 104/300 Read block 2869 len 32768
11694 105/300 Write block 729 len 12288 data: 4 4 5
11695 106/300 Write block 4022 len 8192 data: 3 3
11696 107/300 Read block 2758 len 36864
11697 108/300 Write block 3420 len 24576 data: 3 2 2 2 2 2
11698 109/300 Read block 1339 len 8192
11699 110/300 Read block 2531 len 36864
11700 111/300 Write block 2410 len 32768 data: 4 4 4 3 3 3 3 3
11701 112/300 Read block 74 len 12288
11702 113/300 Write block 3698 len 24576 data: 2 2 3 4 4 4
11703 114/300 Flush
11704 115/300 Write block 2874 len 28672 data: 5 4 3 2 3 3 3
11705 116/300 Write block 3435 len 24576 data: 7 7 7 7 6 4
11706 117/300 Read block 3685 len 8192
11707 118/300 Read block 1331 len 24576
11708 119/300 Read block 2183 len 16384
11709 120/300 Flush
11710 121/300 Write block 2301 len 4096 data: 4
11711 122/300 Write block 167 len 28672 data: 6 6 7 7 6 5 5
11712 123/300 Write block 3884 len 4096 data: 5
11713 124/300 Write block 1077 len 28672 data: 5 4 3 3 3 3 3
11714 125/300 Read block 112 len 20480
11715 126/300 Read block 446 len 20480
11716 127/300 Flush
11717 128/300 Write block 940 len 36864 data: 4 4 4 4 3 3 3 4 5
11718 129/300 Read block 396 len 32768
11719 130/300 Write block 4245 len 20480 data: 4 3 3 4 5
11720 131/300 Read block 4753 len 4096
11721 132/300 Read block 721 len 16384
11722 133/300 Read block 2095 len 24576
11723 134/300 Read block 2052 len 12288
11724 135/300 Write block 3349 len 8192 data: 6 5
11725 136/300 Write block 1193 len 40960 data: 3 3 2 2 2 2 2 2 2 2
11726 137/300 Read block 2596 len 36864
11727 138/300 Read block 2476 len 24576
11728 139/300 Read block 1641 len 16384
11729 140/300 Read block 3193 len 20480
11730 141/300 Read block 4906 len 20480
11731 142/300 Read block 3149 len 32768
11732 143/300 Write block 1613 len 28672 data: 3 3 3 2 2 2 2
11733 144/300 Read block 3090 len 20480
11734 145/300 Read block 2739 len 28672
11735 146/300 Read block 1174 len 4096
11736 147/300 Read block 2311 len 28672
11737 148/300 Flush
11738 149/300 Read block 119 len 4096
11739 150/300 Read block 1515 len 8192
11740 151/300 Read block 2279 len 4096
11741 152/300 Read block 1532 len 4096
11742 153/300 Flush
11743 154/300 Write block 1938 len 20480 data: 3 2 5 5 6
11744 155/300 Write block 186 len 40960 data: 4 4 4 4 4 4 5 3 4 2
11745 156/300 Write block 3116 len 12288 data: 4 4 3
11746 157/300 Write block 2844 len 8192 data: 4 3
11747 158/300 Write block 800 len 32768 data: 4 2 2 3 4 5 6 6
11748 159/300 Write block 3521 len 8192 data: 3 2
11749 160/300 Read block 2877 len 16384
11750 161/300 Read block 1666 len 4096
11751 162/300 Read block 1901 len 4096
11752 163/300 Write block 2344 len 40960 data: 4 5 5 5 6 4 4 3 5 4
11753 164/300 Write block 2190 len 28672 data: 4 5 5 5 4 4 4
11754 165/300 Write block 3001 len 20480 data: 6 6 5 6 8
11755 166/300 Write block 3050 len 28672 data: 3 4 4 5 5 6 6
11756 167/300 Read block 1220 len 16384
11757 168/300 Write block 3977 len 32768 data: 2 2 3 3 3 3 2 2
11758 169/300 Write block 4210 len 8192 data: 2 2
11759 170/300 Read block 1701 len 24576
11760 171/300 Write block 1683 len 16384 data: 6 6 6 5
11761 172/300 Read block 4774 len 16384
11762 173/300 Write block 3866 len 20480 data: 4 4 4 4 4
11763 174/300 Write block 2106 len 4096 data: 3
11764 175/300 Write block 1540 len 28672 data: 2 2 2 2 2 2 2
11765 176/300 Read block 238 len 8192
11766 177/300 Flush
11767 178/300 Read block 1345 len 32768
11768 179/300 Write block 2758 len 32768 data: 2 2 4 4 4 4 4 4
11769 180/300 Read block 2430 len 40960
11770 181/300 Read block 4530 len 4096
11771 182/300 Write block 3690 len 32768 data: 4 5 5 5 3 3 2 2
11772 183/300 Write block 4484 len 40960 data: 5 5 6 6 5 4 4 4 4 2
11773 184/300 Read block 3690 len 40960
11774 185/300 Write block 3043 len 4096 data: 2
11775 186/300 Flush
11776 187/300 Read block 471 len 40960
11777 188/300 Read block 1804 len 16384
11778 189/300 Read block 2466 len 20480
11779 190/300 Write block 223 len 12288 data: 2 2 2
11780 191/300 Read block 2650 len 16384
11781 192/300 Read block 4792 len 12288
11782 193/300 Flush
11783 194/300 Read block 1414 len 36864
11784 195/300 Write block 4339 len 32768 data: 3 3 3 3 3 3 3 3
11785 196/300 Read block 4087 len 16384
11786 197/300 Write block 1565 len 36864 data: 4 4 3 3 3 3 3 2 2
11787 198/300 Flush
11788 199/300 Write block 4685 len 4096 data: 3
11789 200/300 Read block 2219 len 4096
11790 201/300 Write block 1565 len 12288 data: 5 5 4
11791 202/300 Write block 826 len 36864 data: 2 2 2 2 2 3 3 4 4
11792 203/300 Write block 3942 len 20480 data: 4 4 4 5 4
11793 204/300 Read block 3423 len 40960
11794 205/300 Read block 4017 len 12288
11795 206/300 Write block 263 len 8192 data: 4 4
11796 207/300 Read block 524 len 28672
11797 208/300 Read block 3776 len 40960
11798 209/300 Write block 4171 len 16384 data: 3 4 3 3
11799 210/300 Write block 984 len 28672 data: 6 6 6 4 4 3 3
11800 211/300 Write block 3624 len 32768 data: 3 4 3 3 3 3 2 2
11801 212/300 Read block 4989 len 16384
11802 213/300 Flush
11803 214/300 Write block 54 len 20480 data: 3 4 3 3 3
11804 215/300 Read block 4900 len 28672
11805 216/300 Read block 1154 len 20480
11806 217/300 Write block 4385 len 16384 data: 2 2 2 3
11807 218/300 Write block 2719 len 16384 data: 2 3 3 3
11808 219/300 Read block 1037 len 24576
11809 220/300 Read block 1104 len 28672
11810 221/300 Write block 3677 len 16384 data: 4 5 4 4
11811 222/300 Read block 1678 len 36864
11812 223/300 Read block 4595 len 16384
11813 224/300 Read block 2554 len 36864
11814 225/300 Read block 4923 len 36864
11815 226/300 Write block 3404 len 8192 data: 6 5
11816 227/300 Read block 4951 len 32768
11817 228/300 Write block 3304 len 8192 data: 2 2
11818 229/300 Write block 1761 len 16384 data: 6 8 7 6
11819 230/300 Flush
11820 231/300 Write block 1876 len 36864 data: 3 4 3 3 3 3 2 2 2
11821 232/300 Read block 2401 len 28672
11822 233/300 Write block 618 len 16384 data: 2 3 3 3
11823 234/300 Read block 3396 len 12288
11824 235/300 Write block 2146 len 24576 data: 3 2 3 3 2 2
11825 236/300 Write block 3466 len 12288 data: 4 4 4
11826 237/300 Write block 1751 len 16384 data: 3 3 3 3
11827 238/300 Flush
11828 239/300 Flush
11829 240/300 Read block 4762 len 12288
11830 241/300 Read block 3618 len 24576
11831 242/300 Read block 3605 len 4096
11832 243/300 Write block 3399 len 36864 data: 5 7 7 8 8 7 6 5 4
11833 244/300 Write block 2180 len 32768 data: 3 3 3 4 5 5 6 6
11834 245/300 Write block 3735 len 4096 data: 3
11835 246/300 Flush
11836 247/300 Write block 1410 len 12288 data: 2 2 2
11837 248/300 Read block 254 len 8192
11838 249/300 Read block 866 len 8192
11839 250/300 Read block 2628 len 20480
11840 251/300 Read block 3951 len 4096
11841 252/300 Write block 2049 len 28672 data: 4 4 4 4 5 5 4
11842 253/300 Write block 4351 len 20480 data: 3 3 2 3 3
11843 254/300 Write block 4240 len 40960 data: 4 4 5 4 5 5 4 4 5 6
11844 255/300 Read block 1653 len 24576
11845 256/300 Write block 600 len 36864 data: 5 6 5 5 4 4 3 3 3
11846 257/300 Read block 4618 len 16384
11847 258/300 Read block 4842 len 16384
11848 259/300 Read block 720 len 40960
11849 260/300 Read block 1014 len 32768
11850 261/300 Write block 522 len 4096 data: 3
11851 262/300 Write block 2928 len 40960 data: 5 5 4 3 2 2 2 3 3 2
11852 263/300 Read block 1559 len 36864
11853 264/300 Read block 64 len 28672
11854 265/300 Read block 2521 len 4096
11855 266/300 Write block 1601 len 12288 data: 2 2 2
11856 267/300 Read block 4834 len 36864
11857 268/300 Read block 4536 len 16384
11858 269/300 Write block 4632 len 36864 data: 2 4 3 3 3 2 2 2 2
11859 270/300 Read block 2436 len 32768
11860 271/300 Flush
11861 272/300 Read block 2432 len 8192
11862 273/300 Read block 3723 len 4096
11863 274/300 Flush
11864 275/300 Write block 4744 len 12288 data: 3 2 3
11865 276/300 Read block 4123 len 40960
11866 277/300 Read block 2054 len 20480
11867 278/300 Read block 2334 len 24576
11868 279/300 Write block 1826 len 36864 data: 2 2 2 2 2 3 3 3 3
11869 280/300 Read block 4088 len 16384
11870 281/300 Write block 568 len 28672 data: 4 4 4 3 3 3 2
11871 282/300 Read block 2554 len 16384
11872 283/300 Write block 3424 len 4096 data: 3
11873 284/300 Write block 3669 len 28672 data: 4 4 5 6 5 3 3
11874 285/300 Read block 599 len 36864
11875 286/300 Write block 2470 len 40960 data: 6 4 4 4 5 5 3 3 6 6
11876 287/300 Read block 3656 len 36864
11877 288/300 Write block 1902 len 20480 data: 3 3 3 4 3
11878 289/300 Read block 4446 len 36864
11879 290/300 Write block 446 len 24576 data: 6 4 4 3 3 2
11880 291/300 Write block 2785 len 28672 data: 3 4 3 4 4 4 4
11881 292/300 Write block 485 len 4096 data: 4
11882 293/300 Read block 2461 len 40960
11883 294/300 Read block 3245 len 8192
11884 295/300 Write block 2769 len 36864 data: 4 4 6 7 6 6 6 4 4
11885 296/300 Read block 81 len 32768
11886 297/300 Read block 4644 len 32768
11887 298/300 Flush
11888 299/300 Read block 410 len 4096
11889 300/300 Write block 800 len 20480 data: 5 3 3 4 5
11890 301/300 Flush
11891 Main task has work to do, go find it
11892 got dsc Start(1)
11893 start 1
11894 Dec 04 13:33:13.094 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: 77841b58-f5bc-4fae-88df-ca774aca57a8, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
11895 Replay: started 1, returned:Ok(())
11896 ----------------------------------------------------------------
11897 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11898 States: Active Offline Active
11899 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11900 4321 Acked Write 6 Done Sent Done false
11901 4322 Acked Write 3 Done Sent Done false
11902 4323 Acked Read 5 Done Sent Done false
11903 4324 Acked Read 10 Done Sent Done false
11904 4325 Acked Write 10 Done Sent Done false
11905 4326 Acked Write 6 Done Sent Done false
11906 4327 Acked Write 10 Done Sent Done false
11907 4328 Acked Write 6 Done Sent Done false
11908 4329 Acked Read 4 Done Sent Done false
11909 4330 Acked Read 1 Done Sent Done false
11910 4331 Acked Read 9 Done Sent Done false
11911 4332 Acked Read 6 Done Sent Done false
11912 4333 Acked Write 8 Done Sent Done false
11913 4334 Acked Flush 0 Done Sent Done false
11914 4335 Acked Read 10 Done Sent Done false
11915 4336 Acked Read 3 Done Sent Done false
11916 4337 Acked Read 10 Done Sent Done false
11917 4338 Acked Write 10 Done Sent Done false
11918 4339 Acked Read 3 Done Sent Done false
11919 4340 Acked Read 7 Done Sent Done false
11920 4341 Acked Read 4 Done Sent Done false
11921 4342 Acked Read 8 Done Sent Done false
11922 4343 Acked Flush 0 Done Sent Done false
11923 4344 Acked Write 8 Done Sent Done false
11924 4345 Acked Read 3 Done Sent Done false
11925 4346 Acked Write 5 Done Sent Done false
11926 4347 Acked Write 7 Done Sent Done false
11927 4348 Acked Read 2 Done Sent Done false
11928 4349 Acked Read 1 Done Sent Done false
11929 4350 Acked Read 10 Done Sent Done false
11930 4351 Acked Read 4 Done Sent Done false
11931 4352 Acked Flush 0 Done Sent Done false
11932 4353 Acked Read 5 Done Sent Done false
11933 4354 Acked Read 2 Done Sent Done false
11934 4355 Acked Flush 0 Done Sent Done false
11935 4356 Acked Write 10 Done Sent Done false
11936 4357 Acked Write 8 Done Sent Done false
11937 4358 Acked Write 9 Done Sent Done false
11938 4359 Acked Write 8 Done Sent Done false
11939 4360 Acked Read 2 Done Sent Done false
11940 4361 Acked Read 5 Done Sent Done false
11941 4362 Acked Flush 0 Done Sent Done false
11942 4363 Acked Write 8 Done Sent Done false
11943 4364 Acked Read 7 Done Sent Done false
11944 4365 Acked Write 1 Done Sent Done false
11945 4366 Acked Read 3 Done Sent Done false
11946 4367 Acked Write 10 Done Sent Done false
11947 4368 Acked Read 2 Done Sent Done false
11948 4369 Acked Flush 0 Done Sent Done false
11949 4370 Acked Write 8 Done Sent Done false
11950 4371 Acked Read 3 Done Sent Done false
11951 4372 Acked Read 8 Done Sent Done false
11952 4373 Acked Read 1 Done Sent Done false
11953 4374 Acked Read 5 Done Sent Done false
11954 4375 Acked Read 4 Done Sent Done false
11955 4376 Acked Write 9 Done Sent Done false
11956 4377 Acked Write 10 Done Sent Done false
11957 4378 Acked Write 10 Done Sent Done false
11958 4379 Acked Write 1 Done Sent Done false
11959 4380 Acked Write 1 Done Sent Done false
11960 4381 Acked Read 4 Done Sent Done false
11961 4382 Acked Read 5 Done Sent Done false
11962 4383 Acked Read 8 Done Sent Done false
11963 4384 Acked Read 4 Done Sent Done false
11964 4385 Acked Write 4 Done Sent Done false
11965 4386 Acked Write 8 Done Sent Done false
11966 4387 Acked Write 3 Done Sent Done false
11967 4388 Acked Read 5 Done Sent Done false
11968 4389 Acked Read 5 Done Sent Done false
11969 4390 Acked Write 7 Done Sent Done false
11970 4391 Acked Read 6 Done Sent Done false
11971 4392 Acked Flush 0 Done Sent Done false
11972 4393 Acked Read 2 Done Sent Done false
11973 4394 Acked Write 5 Done Sent Done false
11974 4395 Acked Write 5 Done Sent Done false
11975 4396 Acked Write 4 Done Sent Done false
11976 4397 Acked Read 4 Done Sent Done false
11977 4398 Acked Write 10 Done Sent Done false
11978 4399 Acked Read 9 Done Sent Done false
11979 4400 Acked Read 5 Done Sent Done false
11980 4401 Acked Read 3 Done Sent Done false
11981 4402 Acked Read 2 Done Sent Done false
11982 4403 Acked Read 8 Done Sent Done false
11983 4404 Acked Read 10 Done Sent Done false
11984 4405 Acked Read 3 Done Sent Done false
11985 4406 Acked Read 10 Done Sent Done false
11986 4407 Acked Write 5 Done Sent Done false
11987 4408 Acked Read 1 Done Sent Done false
11988 4409 Acked Write 1 Done Sent Done false
11989 4410 Acked Read 2 Done Sent Done false
11990 4411 Acked Flush 0 Done Sent Done false
11991 4412 Acked Write 6 Done Sent Done false
11992 4413 Acked Read 4 Done Sent Done false
11993 4414 Acked Read 8 Done Sent Done false
11994 4415 Acked Write 6 Done Sent Done false
11995 4416 Acked Read 2 Done Sent Done false
11996 4417 Acked Write 9 Done Sent Done false
11997 4418 Acked Read 8 Done Sent Done false
11998 4419 Acked Write 5 Done Sent Done false
11999 4420 Acked Read 8 Done Sent Done false
12000 4421 Acked Write 3 Done Sent Done false
12001 4422 Acked Write 9 Done Sent Done false
12002 4423 Acked Flush 0 Done Sent Done false
12003 4424 Acked Read 8 Done Sent Done false
12004 4425 Acked Write 3 Done Sent Done false
12005 4426 Acked Write 2 Done Sent Done false
12006 4427 Acked Read 9 Done Sent Done false
12007 4428 Acked Write 6 Done Sent Done false
12008 4429 Acked Read 2 Done Sent Done false
12009 4430 Acked Read 9 Done Sent Done false
12010 4431 Acked Write 8 Done Sent Done false
12011 4432 Acked Read 3 Done Sent Done false
12012 4433 Acked Write 6 Done Sent Done false
12013 4434 Acked Flush 0 Done Sent Done false
12014 4435 Acked Write 7 Done Sent Done false
12015 4436 Acked Write 6 Done Sent Done false
12016 4437 Acked Read 2 Done Sent Done false
12017 4438 Acked Read 6 Done Sent Done false
12018 4439 Acked Read 4 Done Sent Done false
12019 4440 Acked Flush 0 Done Sent Done false
12020 4441 Acked Write 1 Done Sent Done false
12021 4442 Acked Write 7 Done Sent Done false
12022 4443 Acked Write 1 Done Sent Done false
12023 4444 Acked Write 7 Done Sent Done false
12024 4445 Acked Read 5 Done Sent Done false
12025 4446 Acked Read 5 Done Sent Done false
12026 4447 Acked Flush 0 Done Sent Done false
12027 4448 Acked Write 9 Done Sent Done false
12028 4449 Acked Read 8 Done Sent Done false
12029 4450 Acked Write 5 Done Sent Done false
12030 4451 Acked Read 1 Done Sent Done false
12031 4452 Acked Read 4 Done Sent Done false
12032 4453 Acked Read 6 Done Sent Done false
12033 4454 Acked Read 3 Done Sent Done false
12034 4455 Acked Write 2 Done Sent Done false
12035 4456 Acked Write 10 Done Sent Done false
12036 4457 Acked Read 9 Done Sent Done false
12037 4458 Acked Read 6 Done Sent Done false
12038 4459 Acked Read 4 Done Sent Done false
12039 4460 Acked Read 5 Done Sent Done false
12040 4461 Acked Read 5 Done Sent Done false
12041 4462 Acked Read 8 Done Sent Done false
12042 4463 Acked Write 7 Done Sent Done false
12043 4464 Acked Read 5 Done Sent Done false
12044 4465 Acked Read 7 Done Sent Done false
12045 4466 Acked Read 1 Done Sent Done false
12046 4467 Acked Read 7 Done Sent Done false
12047 4468 Acked Flush 0 Done Sent Done false
12048 4469 Acked Read 1 Done Sent Done false
12049 4470 Acked Read 2 Done Sent Done false
12050 4471 Acked Read 1 Done Sent Done false
12051 4472 Acked Read 1 Done Sent Done false
12052 4473 Acked Flush 0 Done Sent Done false
12053 4474 Acked Write 5 Done Sent Done false
12054 4475 Acked Write 10 Done Sent Done false
12055 4476 Acked Write 3 Done Sent Done false
12056 4477 Acked Write 2 Done Sent Done false
12057 4478 Acked Write 8 Done Sent Done false
12058 4479 Acked Write 2 Done Sent Done false
12059 4480 Acked Read 4 Done Sent Done false
12060 4481 Acked Read 1 Done Sent Done false
12061 4482 Acked Read 1 Done Sent Done false
12062 4483 Acked Write 10 Done Sent Done false
12063 4484 Acked Write 7 Done Sent Done false
12064 4485 Acked Write 5 Done Sent Done false
12065 4486 Acked Write 7 Done Sent Done false
12066 4487 Acked Read 4 Done Sent Done false
12067 4488 Acked Write 8 Done Sent Done false
12068 4489 Acked Write 2 Done Sent Done false
12069 4490 Acked Read 6 Done Sent Done false
12070 4491 Acked Write 4 Done Sent Done false
12071 4492 Acked Read 4 Done Sent Done false
12072 4493 Acked Write 5 Done Sent Done false
12073 4494 Acked Write 1 Done Sent Done false
12074 4495 Acked Write 7 Done Sent Done false
12075 4496 Acked Read 2 Done Sent Done false
12076 4497 Acked Flush 0 Done Sent Done false
12077 4498 Acked Read 8 Done Sent Done false
12078 4499 Acked Write 8 Done Sent Done false
12079 4500 Acked Read 10 Done Sent Done false
12080 4501 Acked Read 1 Done Sent Done false
12081 4502 Acked Write 8 Done Sent Done false
12082 4503 Acked Write 10 Done Sent Done false
12083 4504 Acked Read 10 Done Sent Done false
12084 4505 Acked Write 1 Done Sent Done false
12085 4506 Acked Flush 0 Done Sent Done false
12086 4507 Acked Read 10 Done Sent Done false
12087 4508 Acked Read 4 Done Sent Done false
12088 4509 Acked Read 5 Done Sent Done false
12089 4510 Acked Write 3 Done Sent Done false
12090 4511 Acked Read 4 Done Sent Done false
12091 4512 Acked Read 3 Done Sent Done false
12092 4513 Acked Flush 0 Done Sent Done false
12093 4514 Acked Read 9 Done Sent Done false
12094 4515 Acked Write 8 Done Sent Done false
12095 4516 Acked Read 4 Done Sent Done false
12096 4517 Acked Write 9 Done Sent Done false
12097 4518 Acked Flush 0 Done Sent Done false
12098 4519 Acked Write 1 Done Sent Done false
12099 4520 Acked Read 1 Done Sent Done false
12100 4521 Acked Write 3 Done Sent Done false
12101 4522 Acked Write 9 Done Sent Done false
12102 4523 Acked Write 5 Done Sent Done false
12103 4524 Acked Read 10 Done Sent Done false
12104 4525 Acked Read 3 Done Sent Done false
12105 4526 Acked Write 2 Done Sent Done false
12106 4527 Acked Read 7 Done Sent Done false
12107 4528 Acked Read 10 Done Sent Done false
12108 4529 Acked Write 4 Done Sent Done false
12109 4530 Acked Write 7 Done Sent Done false
12110 4531 Acked Write 8 Done Sent Done false
12111 4532 Acked Read 4 Done Sent Done false
12112 4533 Acked Flush 0 Done Sent Done false
12113 4534 Acked Write 5 Done Sent Done false
12114 4535 Acked Read 7 Done Sent Done false
12115 4536 Acked Read 5 Done Sent Done false
12116 4537 Acked Write 4 Done Sent Done false
12117 4538 Acked Write 4 Done Sent Done false
12118 4539 Acked Read 6 Done Sent Done false
12119 4540 Acked Read 7 Done Sent Done false
12120 4541 Acked Write 4 Done Sent Done false
12121 4542 Acked Read 9 Done Sent Done false
12122 4543 Acked Read 4 Done Sent Done false
12123 4544 Acked Read 9 Done Sent Done false
12124 4545 Acked Read 9 Done Sent Done false
12125 4546 Acked Write 2 Done Sent Done false
12126 4547 Acked Read 8 Done Sent Done false
12127 4548 Acked Write 2 Done Sent Done false
12128 4549 Acked Write 4 Done Sent Done false
12129 4550 Acked Flush 0 Done Sent Done false
12130 4551 Acked Write 9 Done Sent Done false
12131 4552 Acked Read 7 Done Sent Done false
12132 4553 Acked Write 4 Done Sent Done false
12133 4554 Acked Read 3 Done Sent Done false
12134 4555 Acked Write 6 Done Sent Done false
12135 4556 Acked Write 3 Done Sent Done false
12136 4557 Acked Write 4 Done Sent Done false
12137 4558 Acked Flush 0 Done Sent Done false
12138 4559 Acked Flush 0 Done Sent Done false
12139 4560 Acked Read 3 Done Sent Done false
12140 4561 Acked Read 6 Done Sent Done false
12141 4562 Acked Read 1 Done Sent Done false
12142 4563 Acked Write 9 Done Sent Done false
12143 4564 Acked Write 8 Done Sent Done false
12144 4565 Acked Write 1 Done Sent Done false
12145 4566 Acked Flush 0 Done Sent Done false
12146 4567 Acked Write 3 Done Sent Done false
12147 4568 Acked Read 2 Done Sent Done false
12148 4569 Acked Read 2 Done Sent Done false
12149 4570 Acked Read 5 Done Sent Done false
12150 4571 Acked Read 1 Done Sent Done false
12151 4572 Acked Write 7 Done Sent Done false
12152 4573 Acked Write 5 Done Sent Done false
12153 4574 Acked Write 10 Done Sent Done false
12154 4575 Acked Read 6 Done Sent Done false
12155 4576 Acked Write 9 Done Sent Done false
12156 4577 Acked Read 4 Done Sent Done false
12157 4578 Acked Read 4 Done Sent Done false
12158 4579 Acked Read 10 Done Sent Done false
12159 4580 Acked Read 8 Done Sent Done false
12160 4581 Acked Write 1 Done Sent Done false
12161 4582 Acked Write 10 Done Sent Done false
12162 4583 Acked Read 9 Done Sent Done false
12163 4584 Acked Read 7 Done Sent Done false
12164 4585 Acked Read 1 Done Sent Done false
12165 4586 Acked Write 3 Done Sent Done false
12166 4587 Acked Read 9 Done Sent Done false
12167 4588 Acked Read 4 Done Sent Done false
12168 4589 Acked Write 9 Done Sent Done false
12169 4590 Acked Read 8 Done Sent Done false
12170 4591 Acked Flush 0 Done Sent Done false
12171 4592 Acked Read 2 Done Sent Done false
12172 4593 Acked Read 1 Done Sent Done false
12173 4594 Acked Flush 0 Done Sent Done false
12174 4595 Acked Write 3 Done Sent Done false
12175 4596 Acked Read 10 Done Sent Done false
12176 4597 Acked Read 5 Done Sent Done false
12177 4598 Acked Read 6 Done Sent Done false
12178 4599 Acked Write 9 Done Sent Done false
12179 4600 Acked Read 4 Done Sent Done false
12180 4601 Acked Write 7 Done Sent Done false
12181 4602 Acked Read 4 Done Sent Done false
12182 4603 Acked Write 1 Done Sent Done false
12183 4604 Acked Write 7 Done Sent Done false
12184 4605 Acked Read 9 Done Sent Done false
12185 4606 Acked Write 10 Done Sent Done false
12186 4607 Acked Read 9 Done Sent Done false
12187 4608 Acked Write 5 Done Sent Done false
12188 4609 Acked Read 9 Done Sent Done false
12189 4610 Acked Write 6 Done Sent Done false
12190 4611 Acked Write 7 Done Sent Done false
12191 4612 Acked Write 1 Done Sent Done false
12192 4613 Acked Read 10 Done Sent Done false
12193 4614 Acked Read 2 Done Sent Done false
12194 4615 Acked Write 9 Done Sent Done false
12195 4616 Acked Read 8 Done Sent Done false
12196 4617 Acked Read 8 Done Sent Done false
12197 4618 Acked Flush 0 Done Sent Done false
12198 4619 Acked Read 1 Done Sent Done false
12199 4620 Acked Write 5 Done Sent Done false
12200 4621 Acked Flush 0 Done Sent Done false
12201 STATES DS:0 DS:1 DS:2 TOTAL
12202 Sent 0 301 0 301
12203 Done 301 0 301 602
12204 Skipped 0 0 0 0
12205 Error 0 0 0 0
12206 Last Flush: 4621 4320 4621
12207 Downstairs last five completed: 4320 4319 4318 4317 4316
12208 Upstairs last five completed: 4621 4620 4619 4618 4617
12209 CLIENT: Up:0 ds:301 act:2
12210 [8820] got start action
12211 [8820] I am going to restart
12212 Starting downstairs at port 8820
12213 Make output file at "/tmp/dsc/downstairs-8820.txt"
12214 [8820][1] reports Starting
12215 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1215)
12216 [8820][1] reports Running
12217 ----------------------------------------------------------------
12218 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12219 States: Active Offline Active
12220 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12221 4321 Acked Write 6 Done Sent Done false
12222 4322 Acked Write 3 Done Sent Done false
12223 4323 Acked Read 5 Done Sent Done false
12224 4324 Acked Read 10 Done Sent Done false
12225 4325 Acked Write 10 Done Sent Done false
12226 4326 Acked Write 6 Done Sent Done false
12227 4327 Acked Write 10 Done Sent Done false
12228 4328 Acked Write 6 Done Sent Done false
12229 4329 Acked Read 4 Done Sent Done false
12230 4330 Acked Read 1 Done Sent Done false
12231 4331 Acked Read 9 Done Sent Done false
12232 4332 Acked Read 6 Done Sent Done false
12233 4333 Acked Write 8 Done Sent Done false
12234 4334 Acked Flush 0 Done Sent Done false
12235 4335 Acked Read 10 Done Sent Done false
12236 4336 Acked Read 3 Done Sent Done false
12237 4337 Acked Read 10 Done Sent Done false
12238 4338 Acked Write 10 Done Sent Done false
12239 4339 Acked Read 3 Done Sent Done false
12240 4340 Acked Read 7 Done Sent Done false
12241 4341 Acked Read 4 Done Sent Done false
12242 4342 Acked Read 8 Done Sent Done false
12243 4343 Acked Flush 0 Done Sent Done false
12244 4344 Acked Write 8 Done Sent Done false
12245 4345 Acked Read 3 Done Sent Done false
12246 4346 Acked Write 5 Done Sent Done false
12247 4347 Acked Write 7 Done Sent Done false
12248 4348 Acked Read 2 Done Sent Done false
12249 4349 Acked Read 1 Done Sent Done false
12250 4350 Acked Read 10 Done Sent Done false
12251 4351 Acked Read 4 Done Sent Done false
12252 4352 Acked Flush 0 Done Sent Done false
12253 4353 Acked Read 5 Done Sent Done false
12254 4354 Acked Read 2 Done Sent Done false
12255 4355 Acked Flush 0 Done Sent Done false
12256 4356 Acked Write 10 Done Sent Done false
12257 4357 Acked Write 8 Done Sent Done false
12258 4358 Acked Write 9 Done Sent Done false
12259 4359 Acked Write 8 Done Sent Done false
12260 4360 Acked Read 2 Done Sent Done false
12261 4361 Acked Read 5 Done Sent Done false
12262 4362 Acked Flush 0 Done Sent Done false
12263 4363 Acked Write 8 Done Sent Done false
12264 4364 Acked Read 7 Done Sent Done false
12265 4365 Acked Write 1 Done Sent Done false
12266 4366 Acked Read 3 Done Sent Done false
12267 4367 Acked Write 10 Done Sent Done false
12268 4368 Acked Read 2 Done Sent Done false
12269 4369 Acked Flush 0 Done Sent Done false
12270 4370 Acked Write 8 Done Sent Done false
12271 4371 Acked Read 3 Done Sent Done false
12272 4372 Acked Read 8 Done Sent Done false
12273 4373 Acked Read 1 Done Sent Done false
12274 4374 Acked Read 5 Done Sent Done false
12275 4375 Acked Read 4 Done Sent Done false
12276 4376 Acked Write 9 Done Sent Done false
12277 4377 Acked Write 10 Done Sent Done false
12278 4378 Acked Write 10 Done Sent Done false
12279 4379 Acked Write 1 Done Sent Done false
12280 4380 Acked Write 1 Done Sent Done false
12281 4381 Acked Read 4 Done Sent Done false
12282 4382 Acked Read 5 Done Sent Done false
12283 4383 Acked Read 8 Done Sent Done false
12284 4384 Acked Read 4 Done Sent Done false
12285 4385 Acked Write 4 Done Sent Done false
12286 4386 Acked Write 8 Done Sent Done false
12287 4387 Acked Write 3 Done Sent Done false
12288 4388 Acked Read 5 Done Sent Done false
12289 4389 Acked Read 5 Done Sent Done false
12290 4390 Acked Write 7 Done Sent Done false
12291 4391 Acked Read 6 Done Sent Done false
12292 4392 Acked Flush 0 Done Sent Done false
12293 4393 Acked Read 2 Done Sent Done false
12294 4394 Acked Write 5 Done Sent Done false
12295 4395 Acked Write 5 Done Sent Done false
12296 4396 Acked Write 4 Done Sent Done false
12297 4397 Acked Read 4 Done Sent Done false
12298 4398 Acked Write 10 Done Sent Done false
12299 4399 Acked Read 9 Done Sent Done false
12300 4400 Acked Read 5 Done Sent Done false
12301 4401 Acked Read 3 Done Sent Done false
12302 4402 Acked Read 2 Done Sent Done false
12303 4403 Acked Read 8 Done Sent Done false
12304 4404 Acked Read 10 Done Sent Done false
12305 4405 Acked Read 3 Done Sent Done false
12306 4406 Acked Read 10 Done Sent Done false
12307 4407 Acked Write 5 Done Sent Done false
12308 4408 Acked Read 1 Done Sent Done false
12309 4409 Acked Write 1 Done Sent Done false
12310 4410 Acked Read 2 Done Sent Done false
12311 4411 Acked Flush 0 Done Sent Done false
12312 4412 Acked Write 6 Done Sent Done false
12313 4413 Acked Read 4 Done Sent Done false
12314 4414 Acked Read 8 Done Sent Done false
12315 4415 Acked Write 6 Done Sent Done false
12316 4416 Acked Read 2 Done Sent Done false
12317 4417 Acked Write 9 Done Sent Done false
12318 4418 Acked Read 8 Done Sent Done false
12319 4419 Acked Write 5 Done Sent Done false
12320 4420 Acked Read 8 Done Sent Done false
12321 4421 Acked Write 3 Done Sent Done false
12322 4422 Acked Write 9 Done Sent Done false
12323 4423 Acked Flush 0 Done Sent Done false
12324 4424 Acked Read 8 Done Sent Done false
12325 4425 Acked Write 3 Done Sent Done false
12326 4426 Acked Write 2 Done Sent Done false
12327 4427 Acked Read 9 Done Sent Done false
12328 4428 Acked Write 6 Done Sent Done false
12329 4429 Acked Read 2 Done Sent Done false
12330 4430 Acked Read 9 Done Sent Done false
12331 4431 Acked Write 8 Done Sent Done false
12332 4432 Acked Read 3 Done Sent Done false
12333 4433 Acked Write 6 Done Sent Done false
12334 4434 Acked Flush 0 Done Sent Done false
12335 4435 Acked Write 7 Done Sent Done false
12336 4436 Acked Write 6 Done Sent Done false
12337 4437 Acked Read 2 Done Sent Done false
12338 4438 Acked Read 6 Done Sent Done false
12339 4439 Acked Read 4 Done Sent Done false
12340 4440 Acked Flush 0 Done Sent Done false
12341 4441 Acked Write 1 Done Sent Done false
12342 4442 Acked Write 7 Done Sent Done false
12343 4443 Acked Write 1 Done Sent Done false
12344 4444 Acked Write 7 Done Sent Done false
12345 4445 Acked Read 5 Done Sent Done false
12346 4446 Acked Read 5 Done Sent Done false
12347 4447 Acked Flush 0 Done Sent Done false
12348 4448 Acked Write 9 Done Sent Done false
12349 4449 Acked Read 8 Done Sent Done false
12350 4450 Acked Write 5 Done Sent Done false
12351 4451 Acked Read 1 Done Sent Done false
12352 4452 Acked Read 4 Done Sent Done false
12353 4453 Acked Read 6 Done Sent Done false
12354 4454 Acked Read 3 Done Sent Done false
12355 4455 Acked Write 2 Done Sent Done false
12356 4456 Acked Write 10 Done Sent Done false
12357 4457 Acked Read 9 Done Sent Done false
12358 4458 Acked Read 6 Done Sent Done false
12359 4459 Acked Read 4 Done Sent Done false
12360 4460 Acked Read 5 Done Sent Done false
12361 4461 Acked Read 5 Done Sent Done false
12362 4462 Acked Read 8 Done Sent Done false
12363 4463 Acked Write 7 Done Sent Done false
12364 4464 Acked Read 5 Done Sent Done false
12365 4465 Acked Read 7 Done Sent Done false
12366 4466 Acked Read 1 Done Sent Done false
12367 4467 Acked Read 7 Done Sent Done false
12368 4468 Acked Flush 0 Done Sent Done false
12369 4469 Acked Read 1 Done Sent Done false
12370 4470 Acked Read 2 Done Sent Done false
12371 4471 Acked Read 1 Done Sent Done false
12372 4472 Acked Read 1 Done Sent Done false
12373 4473 Acked Flush 0 Done Sent Done false
12374 4474 Acked Write 5 Done Sent Done false
12375 4475 Acked Write 10 Done Sent Done false
12376 4476 Acked Write 3 Done Sent Done false
12377 4477 Acked Write 2 Done Sent Done false
12378 4478 Acked Write 8 Done Sent Done false
12379 4479 Acked Write 2 Done Sent Done false
12380 4480 Acked Read 4 Done Sent Done false
12381 4481 Acked Read 1 Done Sent Done false
12382 4482 Acked Read 1 Done Sent Done false
12383 4483 Acked Write 10 Done Sent Done false
12384 4484 Acked Write 7 Done Sent Done false
12385 4485 Acked Write 5 Done Sent Done false
12386 4486 Acked Write 7 Done Sent Done false
12387 4487 Acked Read 4 Done Sent Done false
12388 4488 Acked Write 8 Done Sent Done false
12389 4489 Acked Write 2 Done Sent Done false
12390 4490 Acked Read 6 Done Sent Done false
12391 4491 Acked Write 4 Done Sent Done false
12392 4492 Acked Read 4 Done Sent Done false
12393 4493 Acked Write 5 Done Sent Done false
12394 4494 Acked Write 1 Done Sent Done false
12395 4495 Acked Write 7 Done Sent Done false
12396 4496 Acked Read 2 Done Sent Done false
12397 4497 Acked Flush 0 Done Sent Done false
12398 4498 Acked Read 8 Done Sent Done false
12399 4499 Acked Write 8 Done Sent Done false
12400 4500 Acked Read 10 Done Sent Done false
12401 4501 Acked Read 1 Done Sent Done false
12402 4502 Acked Write 8 Done Sent Done false
12403 4503 Acked Write 10 Done Sent Done false
12404 4504 Acked Read 10 Done Sent Done false
12405 4505 Acked Write 1 Done Sent Done false
12406 4506 Acked Flush 0 Done Sent Done false
12407 4507 Acked Read 10 Done Sent Done false
12408 4508 Acked Read 4 Done Sent Done false
12409 4509 Acked Read 5 Done Sent Done false
12410 4510 Acked Write 3 Done Sent Done false
12411 4511 Acked Read 4 Done Sent Done false
12412 4512 Acked Read 3 Done Sent Done false
12413 4513 Acked Flush 0 Done Sent Done false
12414 4514 Acked Read 9 Done Sent Done false
12415 4515 Acked Write 8 Done Sent Done false
12416 4516 Acked Read 4 Done Sent Done false
12417 4517 Acked Write 9 Done Sent Done false
12418 4518 Acked Flush 0 Done Sent Done false
12419 4519 Acked Write 1 Done Sent Done false
12420 4520 Acked Read 1 Done Sent Done false
12421 4521 Acked Write 3 Done Sent Done false
12422 4522 Acked Write 9 Done Sent Done false
12423 4523 Acked Write 5 Done Sent Done false
12424 4524 Acked Read 10 Done Sent Done false
12425 4525 Acked Read 3 Done Sent Done false
12426 4526 Acked Write 2 Done Sent Done false
12427 4527 Acked Read 7 Done Sent Done false
12428 4528 Acked Read 10 Done Sent Done false
12429 4529 Acked Write 4 Done Sent Done false
12430 4530 Acked Write 7 Done Sent Done false
12431 4531 Acked Write 8 Done Sent Done false
12432 4532 Acked Read 4 Done Sent Done false
12433 4533 Acked Flush 0 Done Sent Done false
12434 4534 Acked Write 5 Done Sent Done false
12435 4535 Acked Read 7 Done Sent Done false
12436 4536 Acked Read 5 Done Sent Done false
12437 4537 Acked Write 4 Done Sent Done false
12438 4538 Acked Write 4 Done Sent Done false
12439 4539 Acked Read 6 Done Sent Done false
12440 4540 Acked Read 7 Done Sent Done false
12441 4541 Acked Write 4 Done Sent Done false
12442 4542 Acked Read 9 Done Sent Done false
12443 4543 Acked Read 4 Done Sent Done false
12444 4544 Acked Read 9 Done Sent Done false
12445 4545 Acked Read 9 Done Sent Done false
12446 4546 Acked Write 2 Done Sent Done false
12447 4547 Acked Read 8 Done Sent Done false
12448 4548 Acked Write 2 Done Sent Done false
12449 4549 Acked Write 4 Done Sent Done false
12450 4550 Acked Flush 0 Done Sent Done false
12451 4551 Acked Write 9 Done Sent Done false
12452 4552 Acked Read 7 Done Sent Done false
12453 4553 Acked Write 4 Done Sent Done false
12454 4554 Acked Read 3 Done Sent Done false
12455 4555 Acked Write 6 Done Sent Done false
12456 4556 Acked Write 3 Done Sent Done false
12457 4557 Acked Write 4 Done Sent Done false
12458 4558 Acked Flush 0 Done Sent Done false
12459 4559 Acked Flush 0 Done Sent Done false
12460 4560 Acked Read 3 Done Sent Done false
12461 4561 Acked Read 6 Done Sent Done false
12462 4562 Acked Read 1 Done Sent Done false
12463 4563 Acked Write 9 Done Sent Done false
12464 4564 Acked Write 8 Done Sent Done false
12465 4565 Acked Write 1 Done Sent Done false
12466 4566 Acked Flush 0 Done Sent Done false
12467 4567 Acked Write 3 Done Sent Done false
12468 4568 Acked Read 2 Done Sent Done false
12469 4569 Acked Read 2 Done Sent Done false
12470 4570 Acked Read 5 Done Sent Done false
12471 4571 Acked Read 1 Done Sent Done false
12472 4572 Acked Write 7 Done Sent Done false
12473 4573 Acked Write 5 Done Sent Done false
12474 4574 Acked Write 10 Done Sent Done false
12475 4575 Acked Read 6 Done Sent Done false
12476 4576 Acked Write 9 Done Sent Done false
12477 4577 Acked Read 4 Done Sent Done false
12478 4578 Acked Read 4 Done Sent Done false
12479 4579 Acked Read 10 Done Sent Done false
12480 4580 Acked Read 8 Done Sent Done false
12481 4581 Acked Write 1 Done Sent Done false
12482 4582 Acked Write 10 Done Sent Done false
12483 4583 Acked Read 9 Done Sent Done false
12484 4584 Acked Read 7 Done Sent Done false
12485 4585 Acked Read 1 Done Sent Done false
12486 4586 Acked Write 3 Done Sent Done false
12487 4587 Acked Read 9 Done Sent Done false
12488 4588 Acked Read 4 Done Sent Done false
12489 4589 Acked Write 9 Done Sent Done false
12490 4590 Acked Read 8 Done Sent Done false
12491 4591 Acked Flush 0 Done Sent Done false
12492 4592 Acked Read 2 Done Sent Done false
12493 4593 Acked Read 1 Done Sent Done false
12494 4594 Acked Flush 0 Done Sent Done false
12495 4595 Acked Write 3 Done Sent Done false
12496 4596 Acked Read 10 Done Sent Done false
12497 4597 Acked Read 5 Done Sent Done false
12498 4598 Acked Read 6 Done Sent Done false
12499 4599 Acked Write 9 Done Sent Done false
12500 4600 Acked Read 4 Done Sent Done false
12501 4601 Acked Write 7 Done Sent Done false
12502 4602 Acked Read 4 Done Sent Done false
12503 4603 Acked Write 1 Done Sent Done false
12504 4604 Acked Write 7 Done Sent Done false
12505 4605 Acked Read 9 Done Sent Done false
12506 4606 Acked Write 10 Done Sent Done false
12507 4607 Acked Read 9 Done Sent Done false
12508 4608 Acked Write 5 Done Sent Done false
12509 4609 Acked Read 9 Done Sent Done false
12510 4610 Acked Write 6 Done Sent Done false
12511 4611 Acked Write 7 Done Sent Done false
12512 4612 Acked Write 1 Done Sent Done false
12513 4613 Acked Read 10 Done Sent Done false
12514 4614 Acked Read 2 Done Sent Done false
12515 4615 Acked Write 9 Done Sent Done false
12516 4616 Acked Read 8 Done Sent Done false
12517 4617 Acked Read 8 Done Sent Done false
12518 4618 Acked Flush 0 Done Sent Done false
12519 4619 Acked Read 1 Done Sent Done false
12520 4620 Acked Write 5 Done Sent Done false
12521 4621 Acked Flush 0 Done Sent Done false
12522 STATES DS:0 DS:1 DS:2 TOTAL
12523 Sent 0 301 0 301
12524 Done 301 0 301 602
12525 Skipped 0 0 0 0
12526 Error 0 0 0 0
12527 Last Flush: 4621 4320 4621
12528 Downstairs last five completed: 4320 4319 4318 4317 4316
12529 Upstairs last five completed: 4621 4620 4619 4618 4617
12530 CLIENT: Up:0 ds:301 act:2
125312024-12-04T13:33:18.858ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125322024-12-04T13:33:18.859ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35954) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125332024-12-04T13:33:18.859ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125342024-12-04T13:33:18.860ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125352024-12-04T13:33:18.860ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125362024-12-04T13:33:18.860ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125372024-12-04T13:33:18.860ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125382024-12-04T13:33:18.860ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125392024-12-04T13:33:18.860ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125402024-12-04T13:33:18.860ZINFOcrucible: [1] client re-new 301 jobs since flush 4320 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125412024-12-04T13:33:18.861ZINFOcrucible: [1] Replayed 301 jobs since flush: 4320 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12542 ----------------------------------------------------------------
12543 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12544 Downstairs last five completed: 4621 4620 4619 4618 4617
12545 Upstairs last five completed: 4621 4620 4619 4618 4617
12546 CLIENT: Up:0 ds:0 act:3
12547 Replay: All jobs finished
12548 Main task has work to do, go find it
12549 got dsc Stop(2)
12550 stop 2
12551 Dec 04 13:33:21.099 INFO request completed, latency_us: [8830] Got stop action so:false kr:false
12552 328, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 328d3b0c-6464-427e-8986-5f59b36b56c4, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
125532024-12-04T13:33:21.100ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125542024-12-04T13:33:21.100ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125552024-12-04T13:33:21.100ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125562024-12-04T13:33:21.100ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
125572024-12-04T13:33:21.100ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12558 Dec 04 13:33:21.100 INFO request completed, latency_us: 144, response_code: 200, uri: /state/cid/2, method: GET, req_id: 10a9184f-507d-4bc5-89ac-92f84e6368b9, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
125592024-12-04T13:33:21.100ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
12560 [8830] Exited with: ExitStatus(unix_wait_status(9))
12561 [8830][2] reports Exit
12562 Dec 04 13:33:25.103 INFO request completed, latency_us: 202, response_code: 200, uri: /state/cid/2, method: GET, req_id: 10014cd9-7b6b-4fba-b004-b9dd5eac0158, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
12563 001/300 Read block 1948 len 28672
12564 002/300 Read block 3286 len 8192
12565 003/300 Write block 3757 len 12288 data: 3 3 4
12566 004/300 Read block 777 len 12288
12567 005/300 Read block 2411 len 32768
12568 006/300 Read block 2395 len 16384
12569 007/300 Read block 973 len 16384
12570 008/300 Flush
12571 009/300 Write block 1005 len 4096 data: 5
12572 010/300 Write block 3060 len 16384 data: 5 4 5 4
12573 011/300 Write block 3148 len 36864 data: 3 3 3 4 4 5 4 4 4
12574 012/300 Write block 267 len 32768 data: 4 4 4 3 2 3 3 4
12575 013/300 Write block 4259 len 32768 data: 3 3 3 3 3 3 4 4
12576 014/300 Write block 1201 len 12288 data: 3 3 2
12577 015/300 Write block 3527 len 36864 data: 5 5 5 4 4 3 3 3 3
12578 016/300 Read block 1774 len 28672
12579 017/300 Write block 4246 len 16384 data: 5 5 6 7
12580 018/300 Write block 1947 len 24576 data: 7 7 7 6 6 4
12581 019/300 Flush
12582 020/300 Write block 912 len 28672 data: 2 2 2 2 2 2 3
12583 021/300 Read block 1912 len 36864
12584 022/300 Read block 4986 len 36864
12585 023/300 Read block 3229 len 4096
12586 024/300 Read block 2272 len 24576
12587 025/300 Read block 796 len 40960
12588 026/300 Read block 4236 len 32768
12589 027/300 Write block 2901 len 40960 data: 3 2 2 2 2 3 2 2 2 2
12590 028/300 Write block 1160 len 40960 data: 2 2 2 3 2 2 3 2 2 3
12591 029/300 Read block 4576 len 40960
12592 030/300 Write block 1160 len 4096 data: 3
12593 031/300 Write block 2577 len 20480 data: 2 2 2 2 2
12594 032/300 Read block 1277 len 20480
12595 033/300 Write block 3758 len 40960 data: 4 5 4 4 4 5 5 6 6 4
12596 034/300 Read block 1503 len 4096
12597 035/300 Read block 1894 len 16384
12598 036/300 Write block 2804 len 28672 data: 3 3 3 2 2 3 3
12599 037/300 Flush
12600 038/300 Flush
12601 039/300 Write block 591 len 20480 data: 2 3 3 3 3
12602 040/300 Read block 2065 len 32768
12603 041/300 Write block 120 len 4096 data: 2
12604 042/300 Read block 4247 len 4096
12605 043/300 Write block 2134 len 4096 data: 6
12606 044/300 Read block 4833 len 40960
12607 045/300 Write block 81 len 4096 data: 4
12608 046/300 Read block 2489 len 40960
12609 047/300 Read block 1941 len 28672
12610 048/300 Write block 1023 len 12288 data: 5 5 5
12611 049/300 Write block 897 len 20480 data: 3 3 3 3 3
12612 050/300 Flush
12613 051/300 Write block 961 len 12288 data: 3 4 4
12614 052/300 Write block 4387 len 28672 data: 3 4 3 3 4 4 5
12615 053/300 Write block 4104 len 8192 data: 2 2
12616 054/300 Write block 240 len 24576 data: 2 2 2 3 3 3
12617 055/300 Write block 806 len 24576 data: 7 7 6 7 7 7
12618 056/300 Write block 3565 len 36864 data: 2 3 3 2 3 3 4 4 3
12619 057/300 Read block 2233 len 8192
12620 058/300 Write block 3415 len 32768 data: 2 3 3 4 4 4 3 3
12621 059/300 Read block 4059 len 20480
12622 060/300 Read block 2400 len 40960
12623 061/300 Read block 4249 len 40960
12624 062/300 Read block 4077 len 40960
12625 063/300 Read block 3719 len 20480
12626 064/300 Read block 4335 len 20480
12627 065/300 Read block 1046 len 40960
12628 066/300 Read block 691 len 32768
12629 067/300 Flush
12630 068/300 Read block 3753 len 24576
12631 069/300 Read block 842 len 12288
12632 070/300 Read block 1552 len 32768
12633 071/300 Read block 3877 len 20480
12634 072/300 Write block 651 len 28672 data: 3 2 2 2 2 2 2
12635 073/300 Write block 1161 len 36864 data: 3 3 4 3 3 4 3 3 4
12636 074/300 Write block 4379 len 36864 data: 3 2 2 2 2 2 3 3 4
12637 075/300 Read block 3128 len 4096
12638 076/300 Read block 4523 len 32768
12639 077/300 Write block 3416 len 12288 data: 4 4 5
12640 078/300 Write block 2568 len 28672 data: 3 3 4 4 2 2 2
12641 079/300 Write block 4871 len 8192 data: 4 4
12642 080/300 Read block 223 len 20480
12643 081/300 Write block 4171 len 12288 data: 4 5 4
12644 082/300 Write block 2377 len 28672 data: 4 4 4 4 3 3 3
12645 083/300 Write block 577 len 16384 data: 2 2 3 3
12646 084/300 Write block 3517 len 4096 data: 4
12647 085/300 Read block 1399 len 28672
12648 086/300 Flush
12649 087/300 Flush
12650 088/300 Write block 3205 len 24576 data: 2 2 2 2 4 4
12651 089/300 Flush
12652 090/300 Read block 3716 len 8192
12653 091/300 Write block 3399 len 24576 data: 6 8 8 9 9 8
12654 092/300 Write block 690 len 8192 data: 3 4
12655 093/300 Write block 365 len 24576 data: 4 4 4 4 5 4
12656 094/300 Read block 261 len 20480
12657 095/300 Flush
12658 096/300 Read block 4827 len 8192
12659 097/300 Read block 103 len 32768
12660 098/300 Flush
12661 099/300 Write block 4869 len 24576 data: 5 4 5 5 3 3
12662 100/300 Write block 1289 len 32768 data: 2 2 2 2 4 3 3 4
12663 101/300 Read block 1475 len 4096
12664 102/300 Write block 3696 len 36864 data: 3 3 3 3 4 5 5 5 4
12665 103/300 Read block 2151 len 4096
12666 104/300 Read block 2301 len 4096
12667 105/300 Write block 2415 len 4096 data: 4
12668 106/300 Read block 1289 len 12288
12669 107/300 Read block 1952 len 12288
12670 108/300 Read block 1220 len 40960
12671 109/300 Write block 1468 len 20480 data: 2 2 4 6 6
12672 110/300 Write block 4451 len 8192 data: 3 2
12673 111/300 Read block 1481 len 12288
12674 112/300 Write block 1804 len 12288 data: 4 4 4
12675 113/300 Write block 3966 len 40960 data: 3 2 2 2 2 2 2 2 3 3
12676 114/300 Read block 4066 len 32768
12677 115/300 Write block 3175 len 8192 data: 5 5
12678 116/300 Write block 4518 len 16384 data: 3 4 4 4
12679 117/300 Write block 3115 len 8192 data: 4 5
12680 118/300 Read block 4936 len 40960
12681 119/300 Read block 4150 len 28672
12682 120/300 Write block 712 len 16384 data: 2 2 2 2
12683 121/300 Flush
12684 122/300 Write block 932 len 36864 data: 2 2 2 3 3 3 3 3 5
12685 123/300 Read block 502 len 4096
12686 124/300 Read block 2689 len 8192
12687 125/300 Write block 1160 len 20480 data: 4 4 4 5 4
12688 126/300 Read block 4310 len 32768
12689 127/300 Read block 2611 len 32768
12690 128/300 Write block 1062 len 36864 data: 4 4 4 5 6 6 7 7 9
12691 129/300 Write block 3451 len 12288 data: 2 2 2
12692 130/300 Write block 3293 len 24576 data: 3 4 5 7 7 7
12693 131/300 Write block 3418 len 12288 data: 6 5 5
12694 132/300 Read block 624 len 8192
12695 133/300 Write block 477 len 28672 data: 4 3 3 3 3 3 3
12696 134/300 Flush
12697 135/300 Read block 1395 len 8192
12698 136/300 Write block 3895 len 36864 data: 2 2 2 3 3 3 3 3 4
12699 137/300 Write block 2261 len 36864 data: 2 2 3 3 3 3 3 3 3
12700 138/300 Read block 4797 len 36864
12701 139/300 Read block 2933 len 36864
12702 140/300 Write block 3995 len 4096 data: 2
12703 141/300 Write block 2654 len 32768 data: 3 3 3 3 3 2 2 2
12704 142/300 Write block 1305 len 4096 data: 2
12705 143/300 Write block 3055 len 12288 data: 7 7 6
12706 144/300 Read block 4981 len 12288
12707 145/300 Read block 652 len 4096
12708 146/300 Read block 3649 len 24576
12709 147/300 Read block 3918 len 32768
12710 148/300 Read block 3593 len 16384
12711 149/300 Flush
12712 150/300 Write block 988 len 40960 data: 5 4 4 2 3 3 4 2 2 2
12713 151/300 Read block 1492 len 20480
12714 152/300 Write block 3564 len 36864 data: 2 3 4 4 3 4 4 5 5
12715 153/300 Flush
12716 154/300 Read block 1450 len 36864
12717 155/300 Read block 1144 len 4096
12718 156/300 Read block 3634 len 12288
12719 157/300 Read block 1830 len 28672
12720 158/300 Read block 2415 len 16384
12721 159/300 Write block 2477 len 4096 data: 4
12722 160/300 Read block 2993 len 36864
12723 161/300 Read block 2351 len 32768
12724 162/300 Write block 456 len 16384 data: 4 4 4 4
12725 163/300 Read block 3917 len 24576
12726 164/300 Flush
12727 165/300 Read block 943 len 28672
12728 166/300 Read block 2694 len 20480
12729 167/300 Read block 4797 len 4096
12730 168/300 Write block 4750 len 28672 data: 3 3 3 3 4 4 4
12731 169/300 Read block 2814 len 4096
12732 170/300 Read block 4447 len 32768
12733 171/300 Read block 747 len 24576
12734 172/300 Read block 3319 len 8192
12735 173/300 Read block 3404 len 32768
12736 174/300 Write block 4646 len 36864 data: 2 2 2 2 2 2 2 2 2
12737 175/300 Write block 1343 len 32768 data: 3 4 5 5 4 4 3 3
12738 176/300 Write block 3458 len 28672 data: 3 3 3 3 5 4 4
12739 177/300 Write block 3373 len 32768 data: 3 3 3 3 5 6 7 7
12740 178/300 Read block 700 len 8192
12741 179/300 Read block 2842 len 4096
12742 180/300 Read block 1052 len 8192
12743 181/300 Write block 3623 len 32768 data: 3 4 5 4 4 4 4 3
12744 182/300 Flush
12745 183/300 Write block 1686 len 20480 data: 6 5 4 3 4
12746 184/300 Flush
12747 185/300 Read block 1020 len 12288
12748 186/300 Write block 2245 len 12288 data: 3 3 5
12749 187/300 Write block 1742 len 28672 data: 7 7 7 6 6 6 7
12750 188/300 Write block 1437 len 4096 data: 3
12751 189/300 Read block 4201 len 20480
12752 190/300 Write block 1880 len 12288 data: 4 4 3
12753 191/300 Read block 897 len 4096
12754 192/300 Flush
12755 193/300 Write block 3739 len 36864 data: 5 5 4 5 5 5 4 4 4
12756 194/300 Read block 3822 len 4096
12757 195/300 Write block 1402 len 24576 data: 3 4 4 4 4 3
12758 196/300 Read block 3018 len 4096
12759 197/300 Write block 4609 len 8192 data: 4 4
12760 198/300 Read block 780 len 16384
12761 199/300 Write block 386 len 20480 data: 3 3 3 3 3
12762 200/300 Read block 3392 len 20480
12763 201/300 Write block 2136 len 36864 data: 6 5 4 4 5 4 4 4 3
12764 202/300 Write block 2863 len 28672 data: 4 3 4 4 4 6 6
12765 203/300 Read block 3087 len 28672
12766 204/300 Read block 3960 len 16384
12767 205/300 Write block 707 len 12288 data: 2 3 2
12768 206/300 Write block 891 len 40960 data: 4 4 3 3 2 3 4 4 4 4
12769 207/300 Write block 2945 len 12288 data: 4 3 2
12770 208/300 Flush
12771 209/300 Read block 1277 len 20480
12772 210/300 Read block 2138 len 20480
12773 211/300 Read block 1399 len 36864
12774 212/300 Read block 3124 len 12288
12775 213/300 Write block 2866 len 8192 data: 5 5
12776 214/300 Write block 2374 len 4096 data: 3
12777 215/300 Write block 2227 len 40960 data: 3 2 2 2 2 2 2 4 4 4
12778 216/300 Read block 852 len 40960
12779 217/300 Read block 3974 len 32768
12780 218/300 Write block 3610 len 8192 data: 4 3
12781 219/300 Write block 1470 len 36864 data: 5 7 7 6 7 7 6 4 5
12782 220/300 Flush
12783 221/300 Write block 273 len 16384 data: 4 5 5 5
12784 222/300 Read block 1641 len 40960
12785 223/300 Write block 3388 len 36864 data: 2 2 3 3 3 3 3 3 4
12786 224/300 Read block 1721 len 20480
12787 225/300 Read block 2600 len 4096
12788 226/300 Read block 866 len 8192
12789 227/300 Read block 2662 len 4096
12790 228/300 Write block 1781 len 32768 data: 3 3 4 4 3 3 4 4
12791 229/300 Write block 15 len 12288 data: 4 5 4
12792 230/300 Write block 2056 len 36864 data: 5 5 4 3 3 3 2 2 2
12793 231/300 Read block 4854 len 16384
12794 232/300 Read block 259 len 20480
12795 233/300 Read block 2982 len 24576
12796 234/300 Write block 2935 len 16384 data: 4 4 3 2
12797 235/300 Read block 2642 len 24576
12798 236/300 Read block 769 len 20480
12799 237/300 Read block 1654 len 8192
12800 238/300 Flush
12801 239/300 Write block 1532 len 36864 data: 6 4 4 3 3 3 2 2 3
12802 240/300 Write block 1243 len 20480 data: 3 3 3 3 3
12803 241/300 Read block 1309 len 8192
12804 242/300 Write block 1327 len 4096 data: 2
12805 243/300 Read block 3690 len 28672
12806 244/300 Read block 4624 len 12288
12807 245/300 Write block 3032 len 4096 data: 2
12808 246/300 Read block 2893 len 16384
12809 247/300 Read block 2192 len 16384
12810 248/300 Write block 1182 len 36864 data: 4 4 4 4 3 3 3 4 3
12811 249/300 Read block 2245 len 4096
12812 250/300 Read block 4165 len 4096
12813 251/300 Flush
12814 252/300 Read block 494 len 36864
12815 253/300 Write block 1290 len 36864 data: 3 3 3 5 4 4 5 5 5
12816 254/300 Flush
12817 255/300 Read block 1929 len 40960
12818 256/300 Write block 1595 len 16384 data: 4 4 3 3
12819 257/300 Write block 1464 len 4096 data: 3
12820 258/300 Flush
12821 259/300 Write block 591 len 28672 data: 3 4 4 4 4 4 4
12822 260/300 Read block 2995 len 24576
12823 261/300 Write block 972 len 40960 data: 2 2 2 3 3 4 5 5 5 4
12824 262/300 Flush
12825 263/300 Flush
12826 264/300 Read block 4080 len 12288
12827 265/300 Read block 3227 len 20480
12828 266/300 Read block 1877 len 24576
12829 267/300 Write block 3477 len 40960 data: 6 6 6 7 5 5 5 5 6 5
12830 268/300 Write block 2743 len 8192 data: 2 2
12831 269/300 Write block 3630 len 40960 data: 4 3 3 4 4 4 4 4 5 4
12832 270/300 Write block 3714 len 20480 data: 2 4 5 6 4
12833 271/300 Read block 78 len 16384
12834 272/300 Write block 3716 len 40960 data: 6 7 5 4 4 4 3 2 3 3
12835 273/300 Write block 520 len 36864 data: 3 3 4 3 3 3 3 2 2
12836 274/300 Write block 2984 len 20480 data: 4 4 3 4 6
12837 275/300 Read block 1253 len 16384
12838 276/300 Read block 2981 len 8192
12839 277/300 Write block 1202 len 36864 data: 4 3 3 3 3 3 3 3 3
12840 278/300 Flush
12841 279/300 Read block 1171 len 12288
12842 280/300 Write block 290 len 40960 data: 2 2 2 2 4 5 5 5 4 3
12843 281/300 Read block 1752 len 20480
12844 282/300 Read block 699 len 8192
12845 283/300 Read block 2098 len 4096
12846 284/300 Read block 1977 len 12288
12847 285/300 Write block 4977 len 36864 data: 3 3 4 3 3 3 3 3 4
12848 286/300 Flush
12849 287/300 Read block 4936 len 28672
12850 288/300 Read block 2494 len 36864
12851 289/300 Read block 1534 len 4096
12852 290/300 Write block 1632 len 20480 data: 4 4 4 3 3
12853 291/300 Write block 4149 len 32768 data: 3 2 2 2 4 4 6 6
12854 292/300 Read block 2500 len 12288
12855 293/300 Read block 213 len 20480
12856 294/300 Write block 4851 len 4096 data: 4
12857 295/300 Write block 4180 len 36864 data: 2 2 3 3 3 3 2 2 2
12858 296/300 Read block 3079 len 16384
12859 297/300 Read block 3442 len 4096
12860 298/300 Write block 1525 len 12288 data: 5 6 5
12861 299/300 Read block 4708 len 8192
12862 300/300 Read block 895 len 16384
12863 301/300 Read block 373 len 16384
12864 Main task has work to do, go find it
12865 got dsc Start(2)
12866 start 2
12867 Dec 04 13:33:25.328 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/2, method: POST, req_id: 31e0f249-c541-4963-875a-a3b88aab4acd, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
12868 Replay: started 2, returned:Ok(())
12869 ----------------------------------------------------------------
12870 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12871 States: Active Active Offline
12872 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12873 4622 Acked Read 7 Done Done Sent false
12874 4623 Acked Read 2 Done Done Sent false
12875 4624 Acked Write 3 Done Done Sent false
12876 4625 Acked Read 3 Done Done Sent false
12877 4626 Acked Read 8 Done Done Sent false
12878 4627 Acked Read 4 Done Done Sent false
12879 4628 Acked Read 4 Done Done Sent false
12880 4629 Acked Flush 0 Done Done Sent false
12881 4630 Acked Write 1 Done Done Sent false
12882 4631 Acked Write 4 Done Done Sent false
12883 4632 Acked Write 9 Done Done Sent false
12884 4633 Acked Write 8 Done Done Sent false
12885 4634 Acked Write 8 Done Done Sent false
12886 4635 Acked Write 3 Done Done Sent false
12887 4636 Acked Write 9 Done Done Sent false
12888 4637 Acked Read 7 Done Done Sent false
12889 4638 Acked Write 4 Done Done Sent false
12890 4639 Acked Write 6 Done Done Sent false
12891 4640 Acked Flush 0 Done Done Sent false
12892 4641 Acked Write 7 Done Done Sent false
12893 4642 Acked Read 9 Done Done Sent false
12894 4643 Acked Read 9 Done Done Sent false
12895 4644 Acked Read 1 Done Done Sent false
12896 4645 Acked Read 6 Done Done Sent false
12897 4646 Acked Read 10 Done Done Sent false
12898 4647 Acked Read 8 Done Done Sent false
12899 4648 Acked Write 10 Done Done Sent false
12900 4649 Acked Write 10 Done Done Sent false
12901 4650 Acked Read 10 Done Done Sent false
12902 4651 Acked Write 1 Done Done Sent false
12903 4652 Acked Write 5 Done Done Sent false
12904 4653 Acked Read 5 Done Done Sent false
12905 4654 Acked Write 10 Done Done Sent false
12906 4655 Acked Read 1 Done Done Sent false
12907 4656 Acked Read 4 Done Done Sent false
12908 4657 Acked Write 7 Done Done Sent false
12909 4658 Acked Flush 0 Done Done Sent false
12910 4659 Acked Flush 0 Done Done Sent false
12911 4660 Acked Write 5 Done Done Sent false
12912 4661 Acked Read 8 Done Done Sent false
12913 4662 Acked Write 1 Done Done Sent false
12914 4663 Acked Read 1 Done Done Sent false
12915 4664 Acked Write 1 Done Done Sent false
12916 4665 Acked Read 10 Done Done Sent false
12917 4666 Acked Write 1 Done Done Sent false
12918 4667 Acked Read 10 Done Done Sent false
12919 4668 Acked Read 7 Done Done Sent false
12920 4669 Acked Write 3 Done Done Sent false
12921 4670 Acked Write 5 Done Done Sent false
12922 4671 Acked Flush 0 Done Done Sent false
12923 4672 Acked Write 3 Done Done Sent false
12924 4673 Acked Write 7 Done Done Sent false
12925 4674 Acked Write 2 Done Done Sent false
12926 4675 Acked Write 6 Done Done Sent false
12927 4676 Acked Write 6 Done Done Sent false
12928 4677 Acked Write 9 Done Done Sent false
12929 4678 Acked Read 2 Done Done Sent false
12930 4679 Acked Write 8 Done Done Sent false
12931 4680 Acked Read 5 Done Done Sent false
12932 4681 Acked Read 10 Done Done Sent false
12933 4682 Acked Read 10 Done Done Sent false
12934 4683 Acked Read 10 Done Done Sent false
12935 4684 Acked Read 5 Done Done Sent false
12936 4685 Acked Read 5 Done Done Sent false
12937 4686 Acked Read 10 Done Done Sent false
12938 4687 Acked Read 8 Done Done Sent false
12939 4688 Acked Flush 0 Done Done Sent false
12940 4689 Acked Read 6 Done Done Sent false
12941 4690 Acked Read 3 Done Done Sent false
12942 4691 Acked Read 8 Done Done Sent false
12943 4692 Acked Read 5 Done Done Sent false
12944 4693 Acked Write 7 Done Done Sent false
12945 4694 Acked Write 9 Done Done Sent false
12946 4695 Acked Write 9 Done Done Sent false
12947 4696 Acked Read 1 Done Done Sent false
12948 4697 Acked Read 8 Done Done Sent false
12949 4698 Acked Write 3 Done Done Sent false
12950 4699 Acked Write 7 Done Done Sent false
12951 4700 Acked Write 2 Done Done Sent false
12952 4701 Acked Read 5 Done Done Sent false
12953 4702 Acked Write 3 Done Done Sent false
12954 4703 Acked Write 7 Done Done Sent false
12955 4704 Acked Write 4 Done Done Sent false
12956 4705 Acked Write 1 Done Done Sent false
12957 4706 Acked Read 7 Done Done Sent false
12958 4707 Acked Flush 0 Done Done Sent false
12959 4708 Acked Flush 0 Done Done Sent false
12960 4709 Acked Write 6 Done Done Sent false
12961 4710 Acked Flush 0 Done Done Sent false
12962 4711 Acked Read 2 Done Done Sent false
12963 4712 Acked Write 6 Done Done Sent false
12964 4713 Acked Write 2 Done Done Sent false
12965 4714 Acked Write 6 Done Done Sent false
12966 4715 Acked Read 5 Done Done Sent false
12967 4716 Acked Flush 0 Done Done Sent false
12968 4717 Acked Read 2 Done Done Sent false
12969 4718 Acked Read 8 Done Done Sent false
12970 4719 Acked Flush 0 Done Done Sent false
12971 4720 Acked Write 6 Done Done Sent false
12972 4721 Acked Write 8 Done Done Sent false
12973 4722 Acked Read 1 Done Done Sent false
12974 4723 Acked Write 9 Done Done Sent false
12975 4724 Acked Read 1 Done Done Sent false
12976 4725 Acked Read 1 Done Done Sent false
12977 4726 Acked Write 1 Done Done Sent false
12978 4727 Acked Read 3 Done Done Sent false
12979 4728 Acked Read 3 Done Done Sent false
12980 4729 Acked Read 10 Done Done Sent false
12981 4730 Acked Write 5 Done Done Sent false
12982 4731 Acked Write 2 Done Done Sent false
12983 4732 Acked Read 3 Done Done Sent false
12984 4733 Acked Write 3 Done Done Sent false
12985 4734 Acked Write 10 Done Done Sent false
12986 4735 Acked Read 8 Done Done Sent false
12987 4736 Acked Write 2 Done Done Sent false
12988 4737 Acked Write 4 Done Done Sent false
12989 4738 Acked Write 2 Done Done Sent false
12990 4739 Acked Read 10 Done Done Sent false
12991 4740 Acked Read 7 Done Done Sent false
12992 4741 Acked Write 4 Done Done Sent false
12993 4742 Acked Flush 0 Done Done Sent false
12994 4743 Acked Write 9 Done Done Sent false
12995 4744 Acked Read 1 Done Done Sent false
12996 4745 Acked Read 2 Done Done Sent false
12997 4746 Acked Write 5 Done Done Sent false
12998 4747 Acked Read 8 Done Done Sent false
12999 4748 Acked Read 8 Done Done Sent false
13000 4749 Acked Write 9 Done Done Sent false
13001 4750 Acked Write 3 Done Done Sent false
13002 4751 Acked Write 6 Done Done Sent false
13003 4752 Acked Write 3 Done Done Sent false
13004 4753 Acked Read 2 Done Done Sent false
13005 4754 Acked Write 7 Done Done Sent false
13006 4755 Acked Flush 0 Done Done Sent false
13007 4756 Acked Read 2 Done Done Sent false
13008 4757 Acked Write 9 Done Done Sent false
13009 4758 Acked Write 9 Done Done Sent false
13010 4759 Acked Read 9 Done Done Sent false
13011 4760 Acked Read 9 Done Done Sent false
13012 4761 Acked Write 1 Done Done Sent false
13013 4762 Acked Write 8 Done Done Sent false
13014 4763 Acked Write 1 Done Done Sent false
13015 4764 Acked Write 3 Done Done Sent false
13016 4765 Acked Read 3 Done Done Sent false
13017 4766 Acked Read 1 Done Done Sent false
13018 4767 Acked Read 6 Done Done Sent false
13019 4768 Acked Read 8 Done Done Sent false
13020 4769 Acked Read 4 Done Done Sent false
13021 4770 Acked Flush 0 Done Done Sent false
13022 4771 Acked Write 10 Done Done Sent false
13023 4772 Acked Read 5 Done Done Sent false
13024 4773 Acked Write 9 Done Done Sent false
13025 4774 Acked Flush 0 Done Done Sent false
13026 4775 Acked Read 9 Done Done Sent false
13027 4776 Acked Read 1 Done Done Sent false
13028 4777 Acked Read 3 Done Done Sent false
13029 4778 Acked Read 7 Done Done Sent false
13030 4779 Acked Read 4 Done Done Sent false
13031 4780 Acked Write 1 Done Done Sent false
13032 4781 Acked Read 9 Done Done Sent false
13033 4782 Acked Read 8 Done Done Sent false
13034 4783 Acked Write 4 Done Done Sent false
13035 4784 Acked Read 6 Done Done Sent false
13036 4785 Acked Flush 0 Done Done Sent false
13037 4786 Acked Read 7 Done Done Sent false
13038 4787 Acked Read 5 Done Done Sent false
13039 4788 Acked Read 1 Done Done Sent false
13040 4789 Acked Write 7 Done Done Sent false
13041 4790 Acked Read 1 Done Done Sent false
13042 4791 Acked Read 8 Done Done Sent false
13043 4792 Acked Read 6 Done Done Sent false
13044 4793 Acked Read 2 Done Done Sent false
13045 4794 Acked Read 8 Done Done Sent false
13046 4795 Acked Write 9 Done Done Sent false
13047 4796 Acked Write 8 Done Done Sent false
13048 4797 Acked Write 7 Done Done Sent false
13049 4798 Acked Write 8 Done Done Sent false
13050 4799 Acked Read 2 Done Done Sent false
13051 4800 Acked Read 1 Done Done Sent false
13052 4801 Acked Read 2 Done Done Sent false
13053 4802 Acked Write 8 Done Done Sent false
13054 4803 Acked Flush 0 Done Done Sent false
13055 4804 Acked Write 5 Done Done Sent false
13056 4805 Acked Flush 0 Done Done Sent false
13057 4806 Acked Read 3 Done Done Sent false
13058 4807 Acked Write 3 Done Done Sent false
13059 4808 Acked Write 7 Done Done Sent false
13060 4809 Acked Write 1 Done Done Sent false
13061 4810 Acked Read 5 Done Done Sent false
13062 4811 Acked Write 3 Done Done Sent false
13063 4812 Acked Read 1 Done Done Sent false
13064 4813 Acked Flush 0 Done Done Sent false
13065 4814 Acked Write 9 Done Done Sent false
13066 4815 Acked Read 1 Done Done Sent false
13067 4816 Acked Write 6 Done Done Sent false
13068 4817 Acked Read 1 Done Done Sent false
13069 4818 Acked Write 2 Done Done Sent false
13070 4819 Acked Read 4 Done Done Sent false
13071 4820 Acked Write 5 Done Done Sent false
13072 4821 Acked Read 5 Done Done Sent false
13073 4822 Acked Write 9 Done Done Sent false
13074 4823 Acked Write 7 Done Done Sent false
13075 4824 Acked Read 7 Done Done Sent false
13076 4825 Acked Read 4 Done Done Sent false
13077 4826 Acked Write 3 Done Done Sent false
13078 4827 Acked Write 10 Done Done Sent false
13079 4828 Acked Write 3 Done Done Sent false
13080 4829 Acked Flush 0 Done Done Sent false
13081 4830 Acked Read 5 Done Done Sent false
13082 4831 Acked Read 5 Done Done Sent false
13083 4832 Acked Read 9 Done Done Sent false
13084 4833 Acked Read 3 Done Done Sent false
13085 4834 Acked Write 2 Done Done Sent false
13086 4835 Acked Write 1 Done Done Sent false
13087 4836 Acked Write 10 Done Done Sent false
13088 4837 Acked Read 10 Done Done Sent false
13089 4838 Acked Read 8 Done Done Sent false
13090 4839 Acked Write 2 Done Done Sent false
13091 4840 Acked Write 9 Done Done Sent false
13092 4841 Acked Flush 0 Done Done Sent false
13093 4842 Acked Write 4 Done Done Sent false
13094 4843 Acked Read 10 Done Done Sent false
13095 4844 Acked Write 9 Done Done Sent false
13096 4845 Acked Read 5 Done Done Sent false
13097 4846 Acked Read 1 Done Done Sent false
13098 4847 Acked Read 2 Done Done Sent false
13099 4848 Acked Read 1 Done Done Sent false
13100 4849 Acked Write 8 Done Done Sent false
13101 4850 Acked Write 3 Done Done Sent false
13102 4851 Acked Write 9 Done Done Sent false
13103 4852 Acked Read 4 Done Done Sent false
13104 4853 Acked Read 5 Done Done Sent false
13105 4854 Acked Read 6 Done Done Sent false
13106 4855 Acked Write 4 Done Done Sent false
13107 4856 Acked Read 6 Done Done Sent false
13108 4857 Acked Read 5 Done Done Sent false
13109 4858 Acked Read 2 Done Done Sent false
13110 4859 Acked Flush 0 Done Done Sent false
13111 4860 Acked Write 9 Done Done Sent false
13112 4861 Acked Write 5 Done Done Sent false
13113 4862 Acked Read 2 Done Done Sent false
13114 4863 Acked Write 1 Done Done Sent false
13115 4864 Acked Read 7 Done Done Sent false
13116 4865 Acked Read 3 Done Done Sent false
13117 4866 Acked Write 1 Done Done Sent false
13118 4867 Acked Read 4 Done Done Sent false
13119 4868 Acked Read 4 Done Done Sent false
13120 4869 Acked Write 9 Done Done Sent false
13121 4870 Acked Read 1 Done Done Sent false
13122 4871 Acked Read 1 Done Done Sent false
13123 4872 Acked Flush 0 Done Done Sent false
13124 4873 Acked Read 9 Done Done Sent false
13125 4874 Acked Write 9 Done Done Sent false
13126 4875 Acked Flush 0 Done Done Sent false
13127 4876 Acked Read 10 Done Done Sent false
13128 4877 Acked Write 4 Done Done Sent false
13129 4878 Acked Write 1 Done Done Sent false
13130 4879 Acked Flush 0 Done Done Sent false
13131 4880 Acked Write 7 Done Done Sent false
13132 4881 Acked Read 6 Done Done Sent false
13133 4882 Acked Write 10 Done Done Sent false
13134 4883 Acked Flush 0 Done Done Sent false
13135 4884 Acked Flush 0 Done Done Sent false
13136 4885 Acked Read 3 Done Done Sent false
13137 4886 Acked Read 5 Done Done Sent false
13138 4887 Acked Read 6 Done Done Sent false
13139 4888 Acked Write 10 Done Done Sent false
13140 4889 Acked Write 2 Done Done Sent false
13141 4890 Acked Write 10 Done Done Sent false
13142 4891 Acked Write 5 Done Done Sent false
13143 4892 Acked Read 4 Done Done Sent false
13144 4893 Acked Write 10 Done Done Sent false
13145 4894 Acked Write 9 Done Done Sent false
13146 4895 Acked Write 5 Done Done Sent false
13147 4896 Acked Read 4 Done Done Sent false
13148 4897 Acked Read 2 Done Done Sent false
13149 4898 Acked Write 9 Done Done Sent false
13150 4899 Acked Flush 0 Done Done Sent false
13151 4900 Acked Read 3 Done Done Sent false
13152 4901 Acked Write 10 Done Done Sent false
13153 4902 Acked Read 5 Done Done Sent false
13154 4903 Acked Read 2 Done Done Sent false
13155 4904 Acked Read 1 Done Done Sent false
13156 4905 Acked Read 3 Done Done Sent false
13157 4906 Acked Write 9 Done Done Sent false
13158 4907 Acked Flush 0 Done Done Sent false
13159 4908 Acked Read 7 Done Done Sent false
13160 4909 Acked Read 9 Done Done Sent false
13161 4910 Acked Read 1 Done Done Sent false
13162 4911 Acked Write 5 Done Done Sent false
13163 4912 Acked Write 8 Done Done Sent false
13164 4913 Acked Read 3 Done Done Sent false
13165 4914 Acked Read 5 Done Done Sent false
13166 4915 Acked Write 1 Done Done Sent false
13167 4916 Acked Write 9 Done Done Sent false
13168 4917 Acked Read 4 Done Done Sent false
13169 4918 Acked Read 1 Done Done Sent false
13170 4919 Acked Write 3 Done Done Sent false
13171 4920 Acked Read 2 Done Done Sent false
13172 4921 Acked Read 4 Done Done Sent false
13173 4922 Acked Read 4 Done Done Sent false
13174 STATES DS:0 DS:1 DS:2 TOTAL
13175 Sent 0 0 301 301
13176 Done 301 301 0 602
13177 Skipped 0 0 0 0
13178 Error 0 0 0 0
13179 Last Flush: 4907 4907 4621
13180 Downstairs last five completed: 4621 4620 4619 4618 4617
13181 Upstairs last five completed: 4922 4921 4920 4919 4918
13182 CLIENT: Up:0 ds:301 act:2
13183 [8830] got start action
13184 [8830] I am going to restart
13185 Starting downstairs at port 8830
13186 Make output file at "/tmp/dsc/downstairs-8830.txt"
13187 [8830][2] reports Starting
13188 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1216)
13189 [8830][2] reports Running
13190 ----------------------------------------------------------------
13191 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13192 States: Active Active Offline
13193 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13194 4622 Acked Read 7 Done Done Sent false
13195 4623 Acked Read 2 Done Done Sent false
13196 4624 Acked Write 3 Done Done Sent false
13197 4625 Acked Read 3 Done Done Sent false
13198 4626 Acked Read 8 Done Done Sent false
13199 4627 Acked Read 4 Done Done Sent false
13200 4628 Acked Read 4 Done Done Sent false
13201 4629 Acked Flush 0 Done Done Sent false
13202 4630 Acked Write 1 Done Done Sent false
13203 4631 Acked Write 4 Done Done Sent false
13204 4632 Acked Write 9 Done Done Sent false
13205 4633 Acked Write 8 Done Done Sent false
13206 4634 Acked Write 8 Done Done Sent false
13207 4635 Acked Write 3 Done Done Sent false
13208 4636 Acked Write 9 Done Done Sent false
13209 4637 Acked Read 7 Done Done Sent false
13210 4638 Acked Write 4 Done Done Sent false
13211 4639 Acked Write 6 Done Done Sent false
13212 4640 Acked Flush 0 Done Done Sent false
13213 4641 Acked Write 7 Done Done Sent false
13214 4642 Acked Read 9 Done Done Sent false
13215 4643 Acked Read 9 Done Done Sent false
13216 4644 Acked Read 1 Done Done Sent false
13217 4645 Acked Read 6 Done Done Sent false
13218 4646 Acked Read 10 Done Done Sent false
13219 4647 Acked Read 8 Done Done Sent false
13220 4648 Acked Write 10 Done Done Sent false
13221 4649 Acked Write 10 Done Done Sent false
13222 4650 Acked Read 10 Done Done Sent false
13223 4651 Acked Write 1 Done Done Sent false
13224 4652 Acked Write 5 Done Done Sent false
13225 4653 Acked Read 5 Done Done Sent false
13226 4654 Acked Write 10 Done Done Sent false
13227 4655 Acked Read 1 Done Done Sent false
13228 4656 Acked Read 4 Done Done Sent false
13229 4657 Acked Write 7 Done Done Sent false
13230 4658 Acked Flush 0 Done Done Sent false
13231 4659 Acked Flush 0 Done Done Sent false
13232 4660 Acked Write 5 Done Done Sent false
13233 4661 Acked Read 8 Done Done Sent false
13234 4662 Acked Write 1 Done Done Sent false
13235 4663 Acked Read 1 Done Done Sent false
13236 4664 Acked Write 1 Done Done Sent false
13237 4665 Acked Read 10 Done Done Sent false
13238 4666 Acked Write 1 Done Done Sent false
13239 4667 Acked Read 10 Done Done Sent false
13240 4668 Acked Read 7 Done Done Sent false
13241 4669 Acked Write 3 Done Done Sent false
13242 4670 Acked Write 5 Done Done Sent false
13243 4671 Acked Flush 0 Done Done Sent false
13244 4672 Acked Write 3 Done Done Sent false
13245 4673 Acked Write 7 Done Done Sent false
13246 4674 Acked Write 2 Done Done Sent false
13247 4675 Acked Write 6 Done Done Sent false
13248 4676 Acked Write 6 Done Done Sent false
13249 4677 Acked Write 9 Done Done Sent false
13250 4678 Acked Read 2 Done Done Sent false
13251 4679 Acked Write 8 Done Done Sent false
13252 4680 Acked Read 5 Done Done Sent false
13253 4681 Acked Read 10 Done Done Sent false
13254 4682 Acked Read 10 Done Done Sent false
13255 4683 Acked Read 10 Done Done Sent false
13256 4684 Acked Read 5 Done Done Sent false
13257 4685 Acked Read 5 Done Done Sent false
13258 4686 Acked Read 10 Done Done Sent false
13259 4687 Acked Read 8 Done Done Sent false
13260 4688 Acked Flush 0 Done Done Sent false
13261 4689 Acked Read 6 Done Done Sent false
13262 4690 Acked Read 3 Done Done Sent false
13263 4691 Acked Read 8 Done Done Sent false
13264 4692 Acked Read 5 Done Done Sent false
13265 4693 Acked Write 7 Done Done Sent false
13266 4694 Acked Write 9 Done Done Sent false
13267 4695 Acked Write 9 Done Done Sent false
13268 4696 Acked Read 1 Done Done Sent false
13269 4697 Acked Read 8 Done Done Sent false
13270 4698 Acked Write 3 Done Done Sent false
13271 4699 Acked Write 7 Done Done Sent false
13272 4700 Acked Write 2 Done Done Sent false
13273 4701 Acked Read 5 Done Done Sent false
13274 4702 Acked Write 3 Done Done Sent false
13275 4703 Acked Write 7 Done Done Sent false
13276 4704 Acked Write 4 Done Done Sent false
13277 4705 Acked Write 1 Done Done Sent false
13278 4706 Acked Read 7 Done Done Sent false
13279 4707 Acked Flush 0 Done Done Sent false
13280 4708 Acked Flush 0 Done Done Sent false
13281 4709 Acked Write 6 Done Done Sent false
13282 4710 Acked Flush 0 Done Done Sent false
13283 4711 Acked Read 2 Done Done Sent false
13284 4712 Acked Write 6 Done Done Sent false
13285 4713 Acked Write 2 Done Done Sent false
13286 4714 Acked Write 6 Done Done Sent false
13287 4715 Acked Read 5 Done Done Sent false
13288 4716 Acked Flush 0 Done Done Sent false
13289 4717 Acked Read 2 Done Done Sent false
13290 4718 Acked Read 8 Done Done Sent false
13291 4719 Acked Flush 0 Done Done Sent false
13292 4720 Acked Write 6 Done Done Sent false
13293 4721 Acked Write 8 Done Done Sent false
13294 4722 Acked Read 1 Done Done Sent false
13295 4723 Acked Write 9 Done Done Sent false
13296 4724 Acked Read 1 Done Done Sent false
13297 4725 Acked Read 1 Done Done Sent false
13298 4726 Acked Write 1 Done Done Sent false
13299 4727 Acked Read 3 Done Done Sent false
13300 4728 Acked Read 3 Done Done Sent false
13301 4729 Acked Read 10 Done Done Sent false
13302 4730 Acked Write 5 Done Done Sent false
13303 4731 Acked Write 2 Done Done Sent false
13304 4732 Acked Read 3 Done Done Sent false
13305 4733 Acked Write 3 Done Done Sent false
13306 4734 Acked Write 10 Done Done Sent false
13307 4735 Acked Read 8 Done Done Sent false
13308 4736 Acked Write 2 Done Done Sent false
13309 4737 Acked Write 4 Done Done Sent false
13310 4738 Acked Write 2 Done Done Sent false
13311 4739 Acked Read 10 Done Done Sent false
13312 4740 Acked Read 7 Done Done Sent false
13313 4741 Acked Write 4 Done Done Sent false
13314 4742 Acked Flush 0 Done Done Sent false
13315 4743 Acked Write 9 Done Done Sent false
13316 4744 Acked Read 1 Done Done Sent false
13317 4745 Acked Read 2 Done Done Sent false
13318 4746 Acked Write 5 Done Done Sent false
13319 4747 Acked Read 8 Done Done Sent false
13320 4748 Acked Read 8 Done Done Sent false
13321 4749 Acked Write 9 Done Done Sent false
13322 4750 Acked Write 3 Done Done Sent false
13323 4751 Acked Write 6 Done Done Sent false
13324 4752 Acked Write 3 Done Done Sent false
13325 4753 Acked Read 2 Done Done Sent false
13326 4754 Acked Write 7 Done Done Sent false
13327 4755 Acked Flush 0 Done Done Sent false
13328 4756 Acked Read 2 Done Done Sent false
13329 4757 Acked Write 9 Done Done Sent false
13330 4758 Acked Write 9 Done Done Sent false
13331 4759 Acked Read 9 Done Done Sent false
13332 4760 Acked Read 9 Done Done Sent false
13333 4761 Acked Write 1 Done Done Sent false
13334 4762 Acked Write 8 Done Done Sent false
13335 4763 Acked Write 1 Done Done Sent false
13336 4764 Acked Write 3 Done Done Sent false
13337 4765 Acked Read 3 Done Done Sent false
13338 4766 Acked Read 1 Done Done Sent false
13339 4767 Acked Read 6 Done Done Sent false
13340 4768 Acked Read 8 Done Done Sent false
13341 4769 Acked Read 4 Done Done Sent false
13342 4770 Acked Flush 0 Done Done Sent false
13343 4771 Acked Write 10 Done Done Sent false
13344 4772 Acked Read 5 Done Done Sent false
13345 4773 Acked Write 9 Done Done Sent false
13346 4774 Acked Flush 0 Done Done Sent false
13347 4775 Acked Read 9 Done Done Sent false
13348 4776 Acked Read 1 Done Done Sent false
13349 4777 Acked Read 3 Done Done Sent false
13350 4778 Acked Read 7 Done Done Sent false
13351 4779 Acked Read 4 Done Done Sent false
13352 4780 Acked Write 1 Done Done Sent false
13353 4781 Acked Read 9 Done Done Sent false
13354 4782 Acked Read 8 Done Done Sent false
13355 4783 Acked Write 4 Done Done Sent false
13356 4784 Acked Read 6 Done Done Sent false
13357 4785 Acked Flush 0 Done Done Sent false
13358 4786 Acked Read 7 Done Done Sent false
13359 4787 Acked Read 5 Done Done Sent false
13360 4788 Acked Read 1 Done Done Sent false
13361 4789 Acked Write 7 Done Done Sent false
13362 4790 Acked Read 1 Done Done Sent false
13363 4791 Acked Read 8 Done Done Sent false
13364 4792 Acked Read 6 Done Done Sent false
13365 4793 Acked Read 2 Done Done Sent false
13366 4794 Acked Read 8 Done Done Sent false
13367 4795 Acked Write 9 Done Done Sent false
13368 4796 Acked Write 8 Done Done Sent false
13369 4797 Acked Write 7 Done Done Sent false
13370 4798 Acked Write 8 Done Done Sent false
13371 4799 Acked Read 2 Done Done Sent false
13372 4800 Acked Read 1 Done Done Sent false
13373 4801 Acked Read 2 Done Done Sent false
13374 4802 Acked Write 8 Done Done Sent false
13375 4803 Acked Flush 0 Done Done Sent false
13376 4804 Acked Write 5 Done Done Sent false
13377 4805 Acked Flush 0 Done Done Sent false
13378 4806 Acked Read 3 Done Done Sent false
13379 4807 Acked Write 3 Done Done Sent false
13380 4808 Acked Write 7 Done Done Sent false
13381 4809 Acked Write 1 Done Done Sent false
13382 4810 Acked Read 5 Done Done Sent false
13383 4811 Acked Write 3 Done Done Sent false
13384 4812 Acked Read 1 Done Done Sent false
13385 4813 Acked Flush 0 Done Done Sent false
13386 4814 Acked Write 9 Done Done Sent false
13387 4815 Acked Read 1 Done Done Sent false
13388 4816 Acked Write 6 Done Done Sent false
13389 4817 Acked Read 1 Done Done Sent false
13390 4818 Acked Write 2 Done Done Sent false
13391 4819 Acked Read 4 Done Done Sent false
13392 4820 Acked Write 5 Done Done Sent false
13393 4821 Acked Read 5 Done Done Sent false
13394 4822 Acked Write 9 Done Done Sent false
13395 4823 Acked Write 7 Done Done Sent false
13396 4824 Acked Read 7 Done Done Sent false
13397 4825 Acked Read 4 Done Done Sent false
13398 4826 Acked Write 3 Done Done Sent false
13399 4827 Acked Write 10 Done Done Sent false
13400 4828 Acked Write 3 Done Done Sent false
13401 4829 Acked Flush 0 Done Done Sent false
13402 4830 Acked Read 5 Done Done Sent false
13403 4831 Acked Read 5 Done Done Sent false
13404 4832 Acked Read 9 Done Done Sent false
13405 4833 Acked Read 3 Done Done Sent false
13406 4834 Acked Write 2 Done Done Sent false
13407 4835 Acked Write 1 Done Done Sent false
13408 4836 Acked Write 10 Done Done Sent false
13409 4837 Acked Read 10 Done Done Sent false
13410 4838 Acked Read 8 Done Done Sent false
13411 4839 Acked Write 2 Done Done Sent false
13412 4840 Acked Write 9 Done Done Sent false
13413 4841 Acked Flush 0 Done Done Sent false
13414 4842 Acked Write 4 Done Done Sent false
13415 4843 Acked Read 10 Done Done Sent false
13416 4844 Acked Write 9 Done Done Sent false
13417 4845 Acked Read 5 Done Done Sent false
13418 4846 Acked Read 1 Done Done Sent false
13419 4847 Acked Read 2 Done Done Sent false
13420 4848 Acked Read 1 Done Done Sent false
13421 4849 Acked Write 8 Done Done Sent false
13422 4850 Acked Write 3 Done Done Sent false
13423 4851 Acked Write 9 Done Done Sent false
13424 4852 Acked Read 4 Done Done Sent false
13425 4853 Acked Read 5 Done Done Sent false
13426 4854 Acked Read 6 Done Done Sent false
13427 4855 Acked Write 4 Done Done Sent false
13428 4856 Acked Read 6 Done Done Sent false
13429 4857 Acked Read 5 Done Done Sent false
13430 4858 Acked Read 2 Done Done Sent false
13431 4859 Acked Flush 0 Done Done Sent false
13432 4860 Acked Write 9 Done Done Sent false
13433 4861 Acked Write 5 Done Done Sent false
13434 4862 Acked Read 2 Done Done Sent false
13435 4863 Acked Write 1 Done Done Sent false
13436 4864 Acked Read 7 Done Done Sent false
13437 4865 Acked Read 3 Done Done Sent false
13438 4866 Acked Write 1 Done Done Sent false
13439 4867 Acked Read 4 Done Done Sent false
13440 4868 Acked Read 4 Done Done Sent false
13441 4869 Acked Write 9 Done Done Sent false
13442 4870 Acked Read 1 Done Done Sent false
13443 4871 Acked Read 1 Done Done Sent false
13444 4872 Acked Flush 0 Done Done Sent false
13445 4873 Acked Read 9 Done Done Sent false
13446 4874 Acked Write 9 Done Done Sent false
13447 4875 Acked Flush 0 Done Done Sent false
13448 4876 Acked Read 10 Done Done Sent false
13449 4877 Acked Write 4 Done Done Sent false
13450 4878 Acked Write 1 Done Done Sent false
13451 4879 Acked Flush 0 Done Done Sent false
13452 4880 Acked Write 7 Done Done Sent false
13453 4881 Acked Read 6 Done Done Sent false
13454 4882 Acked Write 10 Done Done Sent false
13455 4883 Acked Flush 0 Done Done Sent false
13456 4884 Acked Flush 0 Done Done Sent false
13457 4885 Acked Read 3 Done Done Sent false
13458 4886 Acked Read 5 Done Done Sent false
13459 4887 Acked Read 6 Done Done Sent false
13460 4888 Acked Write 10 Done Done Sent false
13461 4889 Acked Write 2 Done Done Sent false
13462 4890 Acked Write 10 Done Done Sent false
13463 4891 Acked Write 5 Done Done Sent false
13464 4892 Acked Read 4 Done Done Sent false
13465 4893 Acked Write 10 Done Done Sent false
13466 4894 Acked Write 9 Done Done Sent false
13467 4895 Acked Write 5 Done Done Sent false
13468 4896 Acked Read 4 Done Done Sent false
13469 4897 Acked Read 2 Done Done Sent false
13470 4898 Acked Write 9 Done Done Sent false
13471 4899 Acked Flush 0 Done Done Sent false
13472 4900 Acked Read 3 Done Done Sent false
13473 4901 Acked Write 10 Done Done Sent false
13474 4902 Acked Read 5 Done Done Sent false
13475 4903 Acked Read 2 Done Done Sent false
13476 4904 Acked Read 1 Done Done Sent false
13477 4905 Acked Read 3 Done Done Sent false
13478 4906 Acked Write 9 Done Done Sent false
13479 4907 Acked Flush 0 Done Done Sent false
13480 4908 Acked Read 7 Done Done Sent false
13481 4909 Acked Read 9 Done Done Sent false
13482 4910 Acked Read 1 Done Done Sent false
13483 4911 Acked Write 5 Done Done Sent false
13484 4912 Acked Write 8 Done Done Sent false
13485 4913 Acked Read 3 Done Done Sent false
13486 4914 Acked Read 5 Done Done Sent false
13487 4915 Acked Write 1 Done Done Sent false
13488 4916 Acked Write 9 Done Done Sent false
13489 4917 Acked Read 4 Done Done Sent false
13490 4918 Acked Read 1 Done Done Sent false
13491 4919 Acked Write 3 Done Done Sent false
13492 4920 Acked Read 2 Done Done Sent false
13493 4921 Acked Read 4 Done Done Sent false
13494 4922 Acked Read 4 Done Done Sent false
13495 4923 Acked Flush 0 Done Done Sent false
13496 STATES DS:0 DS:1 DS:2 TOTAL
13497 Sent 0 0 302 302
13498 Done 302 302 0 604
13499 Skipped 0 0 0 0
13500 Error 0 0 0 0
13501 Last Flush: 4923 4923 4621
13502 Downstairs last five completed: 4621 4620 4619 4618 4617
13503 Upstairs last five completed: 4923 4922 4921 4920 4919
13504 CLIENT: Up:0 ds:302 act:2
135052024-12-04T13:33:31.101ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135062024-12-04T13:33:31.101ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50090) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135072024-12-04T13:33:31.102ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135082024-12-04T13:33:31.103ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135092024-12-04T13:33:31.103ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135102024-12-04T13:33:31.103ZINFOcrucible: send last flush ID to this DS: 4621 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135112024-12-04T13:33:31.103ZINFOcrucible: Replied this last flush ID: 4621 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135122024-12-04T13:33:31.103ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135132024-12-04T13:33:31.103ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135142024-12-04T13:33:31.103ZINFOcrucible: [2] client re-new 302 jobs since flush 4621 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135152024-12-04T13:33:31.103ZINFOcrucible: [2] Replayed 302 jobs since flush: 4621 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
13516 ----------------------------------------------------------------
13517 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13518 Downstairs last five completed: 4923 4922 4921 4920 4919
13519 Upstairs last five completed: 4923 4922 4921 4920 4919
13520 CLIENT: Up:0 ds:0 act:3
13521 Replay: All jobs finished
13522 Main task has work to do, go find it
13523 got dsc Stop(1)
13524 stop 1
13525 Dec 04 13:33:33.332 INFO request completed, latency_us: 337[8820] Got stop action so:false kr:false
13526 , response_code: 204, uri: /stop/cid/1, method: POST, req_id: dd64fec1-9e55-484a-a333-032b8107c339, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
13527 Dec 04 13:33:33.333 INFO request completed, latency_us: 112, response_code: 200, uri: /state/cid/1, method: GET, req_id: c7789509-6357-4878-bacd-bb72a3de1f4c, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
135282024-12-04T13:33:33.333ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135292024-12-04T13:33:33.333ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135302024-12-04T13:33:33.333ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135312024-12-04T13:33:33.333ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135322024-12-04T13:33:33.333ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
135332024-12-04T13:33:33.333ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
13534 [8820] Exited with: ExitStatus(unix_wait_status(9))
13535 [8820][1] reports Exit
13536 Dec 04 13:33:37.335 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: 427f3780-b061-476a-a0d8-86bcac72d1ec, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
13537 001/300 Flush
13538 002/300 Flush
13539 003/300 Read block 3680 len 4096
13540 004/300 Flush
13541 005/300 Read block 384 len 20480
13542 006/300 Write block 4601 len 40960 data: 5 5 5 6 6 5 4 4 5 5
13543 007/300 Flush
13544 008/300 Flush
13545 009/300 Read block 1083 len 32768
13546 010/300 Write block 191 len 24576 data: 5 6 4 5 3 4
13547 011/300 Write block 3943 len 40960 data: 5 5 6 5 4 5 5 6 4 3
13548 012/300 Read block 1092 len 32768
13549 013/300 Flush
13550 014/300 Write block 621 len 12288 data: 4 3 4
13551 015/300 Write block 4616 len 36864 data: 3 3 4 4 4 4 3 3 3
13552 016/300 Flush
13553 017/300 Read block 578 len 40960
13554 018/300 Write block 3323 len 8192 data: 6 4
13555 019/300 Write block 3714 len 40960 data: 3 5 7 8 6 5 5 5 4 3
13556 020/300 Write block 2042 len 4096 data: 4
13557 021/300 Read block 3898 len 28672
13558 022/300 Flush
13559 023/300 Write block 3546 len 36864 data: 5 5 4 4 3 4 4 4 3
13560 024/300 Write block 2178 len 16384 data: 3 3 4 4
13561 025/300 Write block 662 len 24576 data: 3 3 3 3 3 3
13562 026/300 Read block 2849 len 4096
13563 027/300 Flush
13564 028/300 Read block 4259 len 28672
13565 029/300 Read block 2506 len 4096
13566 030/300 Read block 2148 len 12288
13567 031/300 Read block 3504 len 28672
13568 032/300 Read block 3683 len 28672
13569 033/300 Write block 3627 len 28672 data: 5 5 5 5 4 4 5
13570 034/300 Read block 4299 len 12288
13571 035/300 Read block 1863 len 12288
13572 036/300 Read block 82 len 16384
13573 037/300 Read block 2317 len 24576
13574 038/300 Read block 726 len 16384
13575 039/300 Write block 2256 len 24576 data: 2 2 2 2 2 3
13576 040/300 Read block 3903 len 16384
13577 041/300 Read block 2545 len 16384
13578 042/300 Read block 2004 len 12288
13579 043/300 Read block 3335 len 16384
13580 044/300 Read block 1747 len 8192
13581 045/300 Read block 1935 len 4096
13582 046/300 Write block 2103 len 40960 data: 2 2 2 4 4 4 4 4 3 2
13583 047/300 Flush
13584 048/300 Read block 2269 len 40960
13585 049/300 Read block 595 len 8192
13586 050/300 Write block 4348 len 28672 data: 2 2 2 4 4 3 4
13587 051/300 Write block 3200 len 20480 data: 2 2 2 2 2
13588 052/300 Read block 1128 len 28672
13589 053/300 Write block 44 len 28672 data: 3 3 2 3 2 2 2
13590 054/300 Read block 48 len 8192
13591 055/300 Write block 1607 len 32768 data: 3 3 2 2 3 3 4 4
13592 056/300 Write block 2377 len 4096 data: 5
13593 057/300 Write block 2137 len 8192 data: 6 5
13594 058/300 Flush
13595 059/300 Write block 1199 len 32768 data: 3 3 4 5 4 4 4 4
13596 060/300 Write block 1779 len 8192 data: 2 2
13597 061/300 Read block 1485 len 20480
13598 062/300 Flush
13599 063/300 Read block 3782 len 40960
13600 064/300 Write block 913 len 8192 data: 3 3
13601 065/300 Write block 138 len 8192 data: 2 2
13602 066/300 Read block 646 len 20480
13603 067/300 Read block 2321 len 36864
13604 068/300 Flush
13605 069/300 Flush
13606 070/300 Read block 4918 len 20480
13607 071/300 Flush
13608 072/300 Read block 1627 len 28672
13609 073/300 Read block 3057 len 24576
13610 074/300 Read block 35 len 8192
13611 075/300 Write block 4433 len 32768 data: 2 2 3 3 3 3 3 3
13612 076/300 Read block 428 len 12288
13613 077/300 Read block 4272 len 8192
13614 078/300 Read block 1037 len 4096
13615 079/300 Read block 3782 len 16384
13616 080/300 Write block 4204 len 24576 data: 2 2 2 2 2 2
13617 081/300 Read block 3597 len 16384
13618 082/300 Write block 1025 len 12288 data: 6 4 4
13619 083/300 Read block 2851 len 24576
13620 084/300 Write block 1420 len 24576 data: 6 7 5 5 6 6
13621 085/300 Read block 3799 len 36864
13622 086/300 Read block 4147 len 32768
13623 087/300 Flush
13624 088/300 Read block 1174 len 32768
13625 089/300 Flush
13626 090/300 Read block 3073 len 12288
13627 091/300 Write block 1592 len 32768 data: 3 3 4 5 5 4 4 3
13628 092/300 Write block 157 len 8192 data: 4 4
13629 093/300 Read block 3138 len 16384
13630 094/300 Read block 1836 len 12288
13631 095/300 Read block 3549 len 28672
13632 096/300 Read block 1139 len 20480
13633 097/300 Write block 301 len 28672 data: 2 2 4 3 4 4 4
13634 098/300 Write block 3671 len 24576 data: 6 7 6 4 4 4
13635 099/300 Write block 4244 len 32768 data: 6 6 6 6 7 8 5 4
13636 100/300 Read block 2535 len 8192
13637 101/300 Write block 2907 len 4096 data: 3
13638 102/300 Write block 4516 len 24576 data: 3 3 4 5 5 5
13639 103/300 Write block 4175 len 28672 data: 3 3 2 2 2 3 3
13640 104/300 Read block 3181 len 12288
13641 105/300 Write block 4566 len 40960 data: 3 3 3 3 3 2 3 4 3 2
13642 106/300 Read block 3045 len 36864
13643 107/300 Read block 1889 len 24576
13644 108/300 Read block 2575 len 24576
13645 109/300 Read block 2030 len 4096
13646 110/300 Read block 1654 len 8192
13647 111/300 Read block 1729 len 28672
13648 112/300 Read block 287 len 28672
13649 113/300 Read block 423 len 12288
13650 114/300 Read block 4686 len 4096
13651 115/300 Write block 381 len 16384 data: 2 2 2 3
13652 116/300 Read block 3258 len 28672
13653 117/300 Write block 4439 len 16384 data: 4 4 3 5
13654 118/300 Write block 2290 len 20480 data: 5 5 6 6 6
13655 119/300 Write block 2522 len 24576 data: 4 4 4 4 4 4
13656 120/300 Write block 2702 len 40960 data: 2 2 2 2 3 5 5 5 5 5
13657 121/300 Write block 2846 len 28672 data: 4 4 4 4 2 3 5
13658 122/300 Read block 3852 len 40960
13659 123/300 Write block 2119 len 16384 data: 2 3 4 4
13660 124/300 Write block 2905 len 4096 data: 3
13661 125/300 Flush
13662 126/300 Read block 497 len 32768
13663 127/300 Read block 1472 len 36864
13664 128/300 Write block 4125 len 32768 data: 3 3 3 4 5 5 6 6
13665 129/300 Write block 102 len 28672 data: 4 3 3 3 3 2 2
13666 130/300 Write block 2178 len 20480 data: 4 4 5 5 4
13667 131/300 Write block 3722 len 16384 data: 5 4 4 4
13668 132/300 Write block 1173 len 24576 data: 2 2 2 2 2 2
13669 133/300 Flush
13670 134/300 Read block 4840 len 12288
13671 135/300 Write block 190 len 20480 data: 5 6 7 5 6
13672 136/300 Write block 4326 len 36864 data: 3 3 2 4 4 6 6 4 4
13673 137/300 Write block 3512 len 32768 data: 3 4 4 4 4 5 4 4
13674 138/300 Read block 1423 len 16384
13675 139/300 Write block 2895 len 16384 data: 4 4 3 3
13676 140/300 Flush
13677 141/300 Write block 1249 len 12288 data: 4 4 3
13678 142/300 Flush
13679 143/300 Write block 1838 len 36864 data: 4 4 4 4 4 4 3 3 3
13680 144/300 Read block 4012 len 4096
13681 145/300 Write block 4267 len 40960 data: 3 4 6 6 6 5 3 4 4 4
13682 146/300 Read block 3353 len 24576
13683 147/300 Write block 366 len 40960 data: 5 5 5 6 5 5 5 5 5 5
13684 148/300 Read block 2487 len 36864
13685 149/300 Write block 2238 len 40960 data: 4 4 4 3 4 3 3 4 4 6
13686 150/300 Write block 4118 len 40960 data: 3 4 5 6 6 5 5 4 4 4
13687 151/300 Read block 4460 len 36864
13688 152/300 Write block 753 len 8192 data: 2 3
13689 153/300 Read block 3631 len 12288
13690 154/300 Write block 4398 len 16384 data: 5 4 4 3
13691 155/300 Read block 2971 len 16384
13692 156/300 Read block 2581 len 36864
13693 157/300 Write block 1065 len 24576 data: 6 7 7 8 8 10
13694 158/300 Write block 1648 len 8192 data: 2 3
13695 159/300 Read block 4049 len 12288
13696 160/300 Read block 2504 len 24576
13697 161/300 Read block 3848 len 24576
13698 162/300 Flush
13699 163/300 Write block 2130 len 20480 data: 2 3 4 4 7
13700 164/300 Read block 4749 len 4096
13701 165/300 Read block 3329 len 12288
13702 166/300 Read block 1495 len 20480
13703 167/300 Write block 1077 len 28672 data: 6 5 4 4 4 4 4
13704 168/300 Flush
13705 169/300 Read block 975 len 16384
13706 170/300 Write block 2136 len 8192 data: 7 7
13707 171/300 Write block 82 len 16384 data: 4 3 3 4
13708 172/300 Read block 1270 len 16384
13709 173/300 Write block 705 len 36864 data: 2 2 3 4 3 2 2 3 3
13710 174/300 Read block 3474 len 24576
13711 175/300 Read block 102 len 28672
13712 176/300 Read block 1720 len 32768
13713 177/300 Write block 686 len 12288 data: 3 3 2
13714 178/300 Write block 329 len 28672 data: 4 5 4 3 2 2 2
13715 179/300 Read block 3361 len 20480
13716 180/300 Write block 4717 len 16384 data: 3 4 4 4
13717 181/300 Read block 2095 len 4096
13718 182/300 Write block 1477 len 16384 data: 5 6 5 5
13719 183/300 Write block 1192 len 4096 data: 3
13720 184/300 Read block 2573 len 12288
13721 185/300 Read block 628 len 4096
13722 186/300 Read block 1151 len 12288
13723 187/300 Read block 2098 len 8192
13724 188/300 Read block 2309 len 24576
13725 189/300 Write block 4929 len 36864 data: 2 2 2 2 2 2 2 2 2
13726 190/300 Write block 2378 len 36864 data: 5 5 5 4 4 4 3 3 3
13727 191/300 Write block 4017 len 16384 data: 4 4 4 3
13728 192/300 Write block 1368 len 20480 data: 2 2 2 3 4
13729 193/300 Read block 201 len 12288
13730 194/300 Read block 4372 len 4096
13731 195/300 Write block 4179 len 4096 data: 3
13732 196/300 Write block 2689 len 12288 data: 2 2 2
13733 197/300 Write block 3305 len 40960 data: 3 2 2 2 2 2 2 2 2 2
13734 198/300 Write block 2604 len 12288 data: 3 3 3
13735 199/300 Read block 3833 len 40960
13736 200/300 Write block 4304 len 32768 data: 2 2 3 3 3 3 3 3
13737 201/300 Read block 2002 len 24576
13738 202/300 Write block 4909 len 32768 data: 6 4 2 2 2 2 2 3
13739 203/300 Write block 3405 len 32768 data: 7 6 5 3 3 2 2 2
13740 204/300 Write block 3319 len 12288 data: 5 8 8
13741 205/300 Write block 4660 len 12288 data: 3 3 3
13742 206/300 Flush
13743 207/300 Read block 4266 len 16384
13744 208/300 Write block 3149 len 24576 data: 4 4 5 5 6 5
13745 209/300 Read block 4533 len 28672
13746 210/300 Read block 4087 len 20480
13747 211/300 Flush
13748 212/300 Write block 770 len 8192 data: 7 6
13749 213/300 Read block 2368 len 4096
13750 214/300 Read block 3111 len 24576
13751 215/300 Read block 4168 len 24576
13752 216/300 Write block 3165 len 12288 data: 4 4 3
13753 217/300 Read block 128 len 28672
13754 218/300 Read block 661 len 8192
13755 219/300 Write block 3105 len 20480 data: 3 2 2 3 4
13756 220/300 Write block 3026 len 4096 data: 3
13757 221/300 Write block 476 len 28672 data: 3 5 4 4 4 4 4
13758 222/300 Write block 541 len 24576 data: 6 5 5 5 4 4
13759 223/300 Read block 2585 len 28672
13760 224/300 Read block 4986 len 4096
13761 225/300 Write block 2786 len 28672 data: 5 4 5 5 5 5 4
13762 226/300 Read block 902 len 16384
13763 227/300 Read block 4427 len 8192
13764 228/300 Write block 1486 len 32768 data: 2 2 2 2 2 2 2 2
13765 229/300 Write block 4795 len 36864 data: 2 2 2 3 3 3 3 3 3
13766 230/300 Read block 4899 len 28672
13767 231/300 Flush
13768 232/300 Write block 503 len 36864 data: 3 3 4 4 4 3 4 4 4
13769 233/300 Write block 3119 len 24576 data: 3 4 4 3 3 2
13770 234/300 Write block 550 len 4096 data: 3
13771 235/300 Read block 3100 len 24576
13772 236/300 Write block 400 len 28672 data: 2 2 2 3 4 4 3
13773 237/300 Write block 2315 len 32768 data: 4 2 2 2 3 4 5 5
13774 238/300 Write block 3529 len 20480 data: 6 5 5 4 4
13775 239/300 Write block 4930 len 8192 data: 3 3
13776 240/300 Write block 234 len 20480 data: 2 2 2 2 2
13777 241/300 Flush
13778 242/300 Write block 1209 len 32768 data: 4 4 3 3 3 3 4 4
13779 243/300 Flush
13780 244/300 Write block 2586 len 36864 data: 2 3 3 5 4 4 4 3 3
13781 245/300 Read block 3014 len 20480
13782 246/300 Read block 2685 len 8192
13783 247/300 Write block 3826 len 24576 data: 4 5 5 5 4 4
13784 248/300 Read block 3014 len 12288
13785 249/300 Write block 3467 len 32768 data: 5 5 3 3 2 2 3 4
13786 250/300 Read block 4052 len 28672
13787 251/300 Read block 2754 len 12288
13788 252/300 Read block 3152 len 40960
13789 253/300 Read block 1352 len 12288
13790 254/300 Read block 3648 len 8192
13791 255/300 Write block 991 len 24576 data: 3 4 4 5 3 3
13792 256/300 Read block 1886 len 40960
13793 257/300 Write block 4520 len 16384 data: 6 6 5 6
13794 258/300 Read block 2691 len 24576
13795 259/300 Read block 4029 len 28672
13796 260/300 Write block 1684 len 4096 data: 7
13797 261/300 Read block 1392 len 36864
13798 262/300 Write block 3051 len 4096 data: 5
13799 263/300 Flush
13800 264/300 Write block 4962 len 36864 data: 2 3 3 3 3 4 5 6 5
13801 265/300 Write block 2489 len 4096 data: 3
13802 266/300 Write block 534 len 12288 data: 2 2 2
13803 267/300 Read block 4812 len 4096
13804 268/300 Read block 594 len 28672
13805 269/300 Flush
13806 270/300 Read block 935 len 20480
13807 271/300 Write block 4592 len 36864 data: 3 3 3 3 3 4 4 4 4
13808 272/300 Write block 3105 len 28672 data: 4 3 3 4 5 4 4
13809 273/300 Read block 2596 len 8192
13810 274/300 Read block 711 len 4096
13811 275/300 Write block 4781 len 12288 data: 3 4 4
13812 276/300 Read block 4414 len 24576
13813 277/300 Write block 2426 len 24576 data: 6 5 5 4 3 3
13814 278/300 Flush
13815 279/300 Write block 2217 len 36864 data: 3 3 3 2 3 3 3 2 2
13816 280/300 Write block 2276 len 20480 data: 4 5 5 5 3
13817 281/300 Read block 3905 len 32768
13818 282/300 Flush
13819 283/300 Write block 80 len 40960 data: 4 5 5 4 4 5 4 4 4 4
13820 284/300 Read block 4280 len 36864
13821 285/300 Write block 2018 len 24576 data: 2 2 2 3 3 3
13822 286/300 Flush
13823 287/300 Flush
13824 288/300 Write block 4469 len 32768 data: 6 7 6 6 4 5 5 5
13825 289/300 Read block 3512 len 40960
13826 290/300 Read block 600 len 40960
13827 291/300 Write block 2102 len 36864 data: 3 3 3 3 5 5 5 5 5
13828 292/300 Write block 1567 len 20480 data: 5 4 4 4 4
13829 293/300 Read block 3319 len 28672
13830 294/300 Write block 4109 len 4096 data: 6
13831 295/300 Read block 1417 len 24576
13832 296/300 Read block 3890 len 24576
13833 297/300 Read block 4146 len 24576
13834 298/300 Flush
13835 299/300 Flush
13836 300/300 Flush
13837 301/300 Write block 4197 len 4096 data: 4
13838 Main task has work to do, go find it
13839 got dsc Start(1)
13840 start 1
13841 Dec 04 13:33:37.559 INFO request completed, latency_us: 184, response_code: 204, uri: /start/cid/1, method: POST, req_id: 5a1b74ab-c651-4336-a825-01a79f3100d4, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
13842 Replay: started 1, returned:Ok(())
13843 ----------------------------------------------------------------
13844 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13845 States: Active Offline Active
13846 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13847 4924 Acked Flush 0 Done Sent Done false
13848 4925 Acked Flush 0 Done Sent Done false
13849 4926 Acked Read 1 Done Sent Done false
13850 4927 Acked Flush 0 Done Sent Done false
13851 4928 Acked Read 5 Done Sent Done false
13852 4929 Acked Write 10 Done Sent Done false
13853 4930 Acked Flush 0 Done Sent Done false
13854 4931 Acked Flush 0 Done Sent Done false
13855 4932 Acked Read 8 Done Sent Done false
13856 4933 Acked Write 6 Done Sent Done false
13857 4934 Acked Write 10 Done Sent Done false
13858 4935 Acked Read 8 Done Sent Done false
13859 4936 Acked Flush 0 Done Sent Done false
13860 4937 Acked Write 3 Done Sent Done false
13861 4938 Acked Write 9 Done Sent Done false
13862 4939 Acked Flush 0 Done Sent Done false
13863 4940 Acked Read 10 Done Sent Done false
13864 4941 Acked Write 2 Done Sent Done false
13865 4942 Acked Write 10 Done Sent Done false
13866 4943 Acked Write 1 Done Sent Done false
13867 4944 Acked Read 7 Done Sent Done false
13868 4945 Acked Flush 0 Done Sent Done false
13869 4946 Acked Write 9 Done Sent Done false
13870 4947 Acked Write 4 Done Sent Done false
13871 4948 Acked Write 6 Done Sent Done false
13872 4949 Acked Read 1 Done Sent Done false
13873 4950 Acked Flush 0 Done Sent Done false
13874 4951 Acked Read 7 Done Sent Done false
13875 4952 Acked Read 1 Done Sent Done false
13876 4953 Acked Read 3 Done Sent Done false
13877 4954 Acked Read 7 Done Sent Done false
13878 4955 Acked Read 7 Done Sent Done false
13879 4956 Acked Write 7 Done Sent Done false
13880 4957 Acked Read 3 Done Sent Done false
13881 4958 Acked Read 3 Done Sent Done false
13882 4959 Acked Read 4 Done Sent Done false
13883 4960 Acked Read 6 Done Sent Done false
13884 4961 Acked Read 4 Done Sent Done false
13885 4962 Acked Write 6 Done Sent Done false
13886 4963 Acked Read 4 Done Sent Done false
13887 4964 Acked Read 4 Done Sent Done false
13888 4965 Acked Read 3 Done Sent Done false
13889 4966 Acked Read 4 Done Sent Done false
13890 4967 Acked Read 2 Done Sent Done false
13891 4968 Acked Read 1 Done Sent Done false
13892 4969 Acked Write 10 Done Sent Done false
13893 4970 Acked Flush 0 Done Sent Done false
13894 4971 Acked Read 10 Done Sent Done false
13895 4972 Acked Read 2 Done Sent Done false
13896 4973 Acked Write 7 Done Sent Done false
13897 4974 Acked Write 5 Done Sent Done false
13898 4975 Acked Read 7 Done Sent Done false
13899 4976 Acked Write 7 Done Sent Done false
13900 4977 Acked Read 2 Done Sent Done false
13901 4978 Acked Write 8 Done Sent Done false
13902 4979 Acked Write 1 Done Sent Done false
13903 4980 Acked Write 2 Done Sent Done false
13904 4981 Acked Flush 0 Done Sent Done false
13905 4982 Acked Write 8 Done Sent Done false
13906 4983 Acked Write 2 Done Sent Done false
13907 4984 Acked Read 5 Done Sent Done false
13908 4985 Acked Flush 0 Done Sent Done false
13909 4986 Acked Read 10 Done Sent Done false
13910 4987 Acked Write 2 Done Sent Done false
13911 4988 Acked Write 2 Done Sent Done false
13912 4989 Acked Read 5 Done Sent Done false
13913 4990 Acked Read 9 Done Sent Done false
13914 4991 Acked Flush 0 Done Sent Done false
13915 4992 Acked Flush 0 Done Sent Done false
13916 4993 Acked Read 5 Done Sent Done false
13917 4994 Acked Flush 0 Done Sent Done false
13918 4995 Acked Read 7 Done Sent Done false
13919 4996 Acked Read 6 Done Sent Done false
13920 4997 Acked Read 2 Done Sent Done false
13921 4998 Acked Write 8 Done Sent Done false
13922 4999 Acked Read 3 Done Sent Done false
13923 5000 Acked Read 2 Done Sent Done false
13924 5001 Acked Read 1 Done Sent Done false
13925 5002 Acked Read 4 Done Sent Done false
13926 5003 Acked Write 6 Done Sent Done false
13927 5004 Acked Read 4 Done Sent Done false
13928 5005 Acked Write 3 Done Sent Done false
13929 5006 Acked Read 6 Done Sent Done false
13930 5007 Acked Write 6 Done Sent Done false
13931 5008 Acked Read 9 Done Sent Done false
13932 5009 Acked Read 8 Done Sent Done false
13933 5010 Acked Flush 0 Done Sent Done false
13934 5011 Acked Read 8 Done Sent Done false
13935 5012 Acked Flush 0 Done Sent Done false
13936 5013 Acked Read 3 Done Sent Done false
13937 5014 Acked Write 8 Done Sent Done false
13938 5015 Acked Write 2 Done Sent Done false
13939 5016 Acked Read 4 Done Sent Done false
13940 5017 Acked Read 3 Done Sent Done false
13941 5018 Acked Read 7 Done Sent Done false
13942 5019 Acked Read 5 Done Sent Done false
13943 5020 Acked Write 7 Done Sent Done false
13944 5021 Acked Write 6 Done Sent Done false
13945 5022 Acked Write 8 Done Sent Done false
13946 5023 Acked Read 2 Done Sent Done false
13947 5024 Acked Write 1 Done Sent Done false
13948 5025 Acked Write 6 Done Sent Done false
13949 5026 Acked Write 7 Done Sent Done false
13950 5027 Acked Read 3 Done Sent Done false
13951 5028 Acked Write 10 Done Sent Done false
13952 5029 Acked Read 9 Done Sent Done false
13953 5030 Acked Read 6 Done Sent Done false
13954 5031 Acked Read 6 Done Sent Done false
13955 5032 Acked Read 1 Done Sent Done false
13956 5033 Acked Read 2 Done Sent Done false
13957 5034 Acked Read 7 Done Sent Done false
13958 5035 Acked Read 7 Done Sent Done false
13959 5036 Acked Read 3 Done Sent Done false
13960 5037 Acked Read 1 Done Sent Done false
13961 5038 Acked Write 4 Done Sent Done false
13962 5039 Acked Read 7 Done Sent Done false
13963 5040 Acked Write 4 Done Sent Done false
13964 5041 Acked Write 5 Done Sent Done false
13965 5042 Acked Write 6 Done Sent Done false
13966 5043 Acked Write 10 Done Sent Done false
13967 5044 Acked Write 7 Done Sent Done false
13968 5045 Acked Read 10 Done Sent Done false
13969 5046 Acked Write 4 Done Sent Done false
13970 5047 Acked Write 1 Done Sent Done false
13971 5048 Acked Flush 0 Done Sent Done false
13972 5049 Acked Read 8 Done Sent Done false
13973 5050 Acked Read 9 Done Sent Done false
13974 5051 Acked Write 8 Done Sent Done false
13975 5052 Acked Write 7 Done Sent Done false
13976 5053 Acked Write 5 Done Sent Done false
13977 5054 Acked Write 4 Done Sent Done false
13978 5055 Acked Write 6 Done Sent Done false
13979 5056 Acked Flush 0 Done Sent Done false
13980 5057 Acked Read 3 Done Sent Done false
13981 5058 Acked Write 5 Done Sent Done false
13982 5059 Acked Write 9 Done Sent Done false
13983 5060 Acked Write 8 Done Sent Done false
13984 5061 Acked Read 4 Done Sent Done false
13985 5062 Acked Write 4 Done Sent Done false
13986 5063 Acked Flush 0 Done Sent Done false
13987 5064 Acked Write 3 Done Sent Done false
13988 5065 Acked Flush 0 Done Sent Done false
13989 5066 Acked Write 9 Done Sent Done false
13990 5067 Acked Read 1 Done Sent Done false
13991 5068 Acked Write 10 Done Sent Done false
13992 5069 Acked Read 6 Done Sent Done false
13993 5070 Acked Write 10 Done Sent Done false
13994 5071 Acked Read 9 Done Sent Done false
13995 5072 Acked Write 10 Done Sent Done false
13996 5073 Acked Write 10 Done Sent Done false
13997 5074 Acked Read 9 Done Sent Done false
13998 5075 Acked Write 2 Done Sent Done false
13999 5076 Acked Read 3 Done Sent Done false
14000 5077 Acked Write 4 Done Sent Done false
14001 5078 Acked Read 4 Done Sent Done false
14002 5079 Acked Read 9 Done Sent Done false
14003 5080 Acked Write 6 Done Sent Done false
14004 5081 Acked Write 2 Done Sent Done false
14005 5082 Acked Read 3 Done Sent Done false
14006 5083 Acked Read 6 Done Sent Done false
14007 5084 Acked Read 6 Done Sent Done false
14008 5085 Acked Flush 0 Done Sent Done false
14009 5086 Acked Write 5 Done Sent Done false
14010 5087 Acked Read 1 Done Sent Done false
14011 5088 Acked Read 3 Done Sent Done false
14012 5089 Acked Read 5 Done Sent Done false
14013 5090 Acked Write 7 Done Sent Done false
14014 5091 Acked Flush 0 Done Sent Done false
14015 5092 Acked Read 4 Done Sent Done false
14016 5093 Acked Write 2 Done Sent Done false
14017 5094 Acked Write 4 Done Sent Done false
14018 5095 Acked Read 4 Done Sent Done false
14019 5096 Acked Write 9 Done Sent Done false
14020 5097 Acked Read 6 Done Sent Done false
14021 5098 Acked Read 7 Done Sent Done false
14022 5099 Acked Read 8 Done Sent Done false
14023 5100 Acked Write 3 Done Sent Done false
14024 5101 Acked Write 7 Done Sent Done false
14025 5102 Acked Read 5 Done Sent Done false
14026 5103 Acked Write 4 Done Sent Done false
14027 5104 Acked Read 1 Done Sent Done false
14028 5105 Acked Write 4 Done Sent Done false
14029 5106 Acked Write 1 Done Sent Done false
14030 5107 Acked Read 3 Done Sent Done false
14031 5108 Acked Read 1 Done Sent Done false
14032 5109 Acked Read 3 Done Sent Done false
14033 5110 Acked Read 2 Done Sent Done false
14034 5111 Acked Read 6 Done Sent Done false
14035 5112 Acked Write 9 Done Sent Done false
14036 5113 Acked Write 9 Done Sent Done false
14037 5114 Acked Write 4 Done Sent Done false
14038 5115 Acked Write 5 Done Sent Done false
14039 5116 Acked Read 3 Done Sent Done false
14040 5117 Acked Read 1 Done Sent Done false
14041 5118 Acked Write 1 Done Sent Done false
14042 5119 Acked Write 3 Done Sent Done false
14043 5120 Acked Write 10 Done Sent Done false
14044 5121 Acked Write 3 Done Sent Done false
14045 5122 Acked Read 10 Done Sent Done false
14046 5123 Acked Write 8 Done Sent Done false
14047 5124 Acked Read 6 Done Sent Done false
14048 5125 Acked Write 8 Done Sent Done false
14049 5126 Acked Write 8 Done Sent Done false
14050 5127 Acked Write 3 Done Sent Done false
14051 5128 Acked Write 3 Done Sent Done false
14052 5129 Acked Flush 0 Done Sent Done false
14053 5130 Acked Read 4 Done Sent Done false
14054 5131 Acked Write 6 Done Sent Done false
14055 5132 Acked Read 7 Done Sent Done false
14056 5133 Acked Read 5 Done Sent Done false
14057 5134 Acked Flush 0 Done Sent Done false
14058 5135 Acked Write 2 Done Sent Done false
14059 5136 Acked Read 1 Done Sent Done false
14060 5137 Acked Read 6 Done Sent Done false
14061 5138 Acked Read 6 Done Sent Done false
14062 5139 Acked Write 3 Done Sent Done false
14063 5140 Acked Read 7 Done Sent Done false
14064 5141 Acked Read 2 Done Sent Done false
14065 5142 Acked Write 5 Done Sent Done false
14066 5143 Acked Write 1 Done Sent Done false
14067 5144 Acked Write 7 Done Sent Done false
14068 5145 Acked Write 6 Done Sent Done false
14069 5146 Acked Read 7 Done Sent Done false
14070 5147 Acked Read 1 Done Sent Done false
14071 5148 Acked Write 7 Done Sent Done false
14072 5149 Acked Read 4 Done Sent Done false
14073 5150 Acked Read 2 Done Sent Done false
14074 5151 Acked Write 8 Done Sent Done false
14075 5152 Acked Write 9 Done Sent Done false
14076 5153 Acked Read 7 Done Sent Done false
14077 5154 Acked Flush 0 Done Sent Done false
14078 5155 Acked Write 9 Done Sent Done false
14079 5156 Acked Write 6 Done Sent Done false
14080 5157 Acked Write 1 Done Sent Done false
14081 5158 Acked Read 6 Done Sent Done false
14082 5159 Acked Write 7 Done Sent Done false
14083 5160 Acked Write 8 Done Sent Done false
14084 5161 Acked Write 5 Done Sent Done false
14085 5162 Acked Write 2 Done Sent Done false
14086 5163 Acked Write 5 Done Sent Done false
14087 5164 Acked Flush 0 Done Sent Done false
14088 5165 Acked Write 8 Done Sent Done false
14089 5166 Acked Flush 0 Done Sent Done false
14090 5167 Acked Write 9 Done Sent Done false
14091 5168 Acked Read 5 Done Sent Done false
14092 5169 Acked Read 2 Done Sent Done false
14093 5170 Acked Write 6 Done Sent Done false
14094 5171 Acked Read 3 Done Sent Done false
14095 5172 Acked Write 8 Done Sent Done false
14096 5173 Acked Read 7 Done Sent Done false
14097 5174 Acked Read 3 Done Sent Done false
14098 5175 Acked Read 10 Done Sent Done false
14099 5176 Acked Read 3 Done Sent Done false
14100 5177 Acked Read 2 Done Sent Done false
14101 5178 Acked Write 6 Done Sent Done false
14102 5179 Acked Read 10 Done Sent Done false
14103 5180 Acked Write 4 Done Sent Done false
14104 5181 Acked Read 6 Done Sent Done false
14105 5182 Acked Read 7 Done Sent Done false
14106 5183 Acked Write 1 Done Sent Done false
14107 5184 Acked Read 9 Done Sent Done false
14108 5185 Acked Write 1 Done Sent Done false
14109 5186 Acked Flush 0 Done Sent Done false
14110 5187 Acked Write 9 Done Sent Done false
14111 5188 Acked Write 1 Done Sent Done false
14112 5189 Acked Write 3 Done Sent Done false
14113 5190 Acked Read 1 Done Sent Done false
14114 5191 Acked Read 7 Done Sent Done false
14115 5192 Acked Flush 0 Done Sent Done false
14116 5193 Acked Read 5 Done Sent Done false
14117 5194 Acked Write 9 Done Sent Done false
14118 5195 Acked Write 7 Done Sent Done false
14119 5196 Acked Read 2 Done Sent Done false
14120 5197 Acked Read 1 Done Sent Done false
14121 5198 Acked Write 3 Done Sent Done false
14122 5199 Acked Read 6 Done Sent Done false
14123 5200 Acked Write 6 Done Sent Done false
14124 5201 Acked Flush 0 Done Sent Done false
14125 5202 Acked Write 9 Done Sent Done false
14126 5203 Acked Write 5 Done Sent Done false
14127 5204 Acked Read 8 Done Sent Done false
14128 5205 Acked Flush 0 Done Sent Done false
14129 5206 Acked Write 10 Done Sent Done false
14130 5207 Acked Read 9 Done Sent Done false
14131 5208 Acked Write 6 Done Sent Done false
14132 5209 Acked Flush 0 Done Sent Done false
14133 5210 Acked Flush 0 Done Sent Done false
14134 5211 Acked Write 8 Done Sent Done false
14135 5212 Acked Read 10 Done Sent Done false
14136 5213 Acked Read 10 Done Sent Done false
14137 5214 Acked Write 9 Done Sent Done false
14138 5215 Acked Write 5 Done Sent Done false
14139 5216 Acked Read 7 Done Sent Done false
14140 5217 Acked Write 1 Done Sent Done false
14141 5218 Acked Read 6 Done Sent Done false
14142 5219 Acked Read 6 Done Sent Done false
14143 5220 Acked Read 6 Done Sent Done false
14144 5221 Acked Flush 0 Done Sent Done false
14145 5222 Acked Flush 0 Done Sent Done false
14146 5223 Acked Flush 0 Done Sent Done false
14147 5224 Acked Write 1 Done Sent Done false
14148 STATES DS:0 DS:1 DS:2 TOTAL
14149 Sent 0 301 0 301
14150 Done 301 0 301 602
14151 Skipped 0 0 0 0
14152 Error 0 0 0 0
14153 Last Flush: 5223 4923 5223
14154 Downstairs last five completed: 4923 4922 4921 4920 4919
14155 Upstairs last five completed: 5224 5223 5222 5221 5220
14156 CLIENT: Up:0 ds:301 act:2
14157 [8820] got start action
14158 [8820] I am going to restart
14159 Starting downstairs at port 8820
14160 Make output file at "/tmp/dsc/downstairs-8820.txt"
14161 [8820][1] reports Starting
14162 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1217)
14163 [8820][1] reports Running
14164 ----------------------------------------------------------------
14165 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14166 States: Active Offline Active
14167 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14168 4924 Acked Flush 0 Done Sent Done false
14169 4925 Acked Flush 0 Done Sent Done false
14170 4926 Acked Read 1 Done Sent Done false
14171 4927 Acked Flush 0 Done Sent Done false
14172 4928 Acked Read 5 Done Sent Done false
14173 4929 Acked Write 10 Done Sent Done false
14174 4930 Acked Flush 0 Done Sent Done false
14175 4931 Acked Flush 0 Done Sent Done false
14176 4932 Acked Read 8 Done Sent Done false
14177 4933 Acked Write 6 Done Sent Done false
14178 4934 Acked Write 10 Done Sent Done false
14179 4935 Acked Read 8 Done Sent Done false
14180 4936 Acked Flush 0 Done Sent Done false
14181 4937 Acked Write 3 Done Sent Done false
14182 4938 Acked Write 9 Done Sent Done false
14183 4939 Acked Flush 0 Done Sent Done false
14184 4940 Acked Read 10 Done Sent Done false
14185 4941 Acked Write 2 Done Sent Done false
14186 4942 Acked Write 10 Done Sent Done false
14187 4943 Acked Write 1 Done Sent Done false
14188 4944 Acked Read 7 Done Sent Done false
14189 4945 Acked Flush 0 Done Sent Done false
14190 4946 Acked Write 9 Done Sent Done false
14191 4947 Acked Write 4 Done Sent Done false
14192 4948 Acked Write 6 Done Sent Done false
14193 4949 Acked Read 1 Done Sent Done false
14194 4950 Acked Flush 0 Done Sent Done false
14195 4951 Acked Read 7 Done Sent Done false
14196 4952 Acked Read 1 Done Sent Done false
14197 4953 Acked Read 3 Done Sent Done false
14198 4954 Acked Read 7 Done Sent Done false
14199 4955 Acked Read 7 Done Sent Done false
14200 4956 Acked Write 7 Done Sent Done false
14201 4957 Acked Read 3 Done Sent Done false
14202 4958 Acked Read 3 Done Sent Done false
14203 4959 Acked Read 4 Done Sent Done false
14204 4960 Acked Read 6 Done Sent Done false
14205 4961 Acked Read 4 Done Sent Done false
14206 4962 Acked Write 6 Done Sent Done false
14207 4963 Acked Read 4 Done Sent Done false
14208 4964 Acked Read 4 Done Sent Done false
14209 4965 Acked Read 3 Done Sent Done false
14210 4966 Acked Read 4 Done Sent Done false
14211 4967 Acked Read 2 Done Sent Done false
14212 4968 Acked Read 1 Done Sent Done false
14213 4969 Acked Write 10 Done Sent Done false
14214 4970 Acked Flush 0 Done Sent Done false
14215 4971 Acked Read 10 Done Sent Done false
14216 4972 Acked Read 2 Done Sent Done false
14217 4973 Acked Write 7 Done Sent Done false
14218 4974 Acked Write 5 Done Sent Done false
14219 4975 Acked Read 7 Done Sent Done false
14220 4976 Acked Write 7 Done Sent Done false
14221 4977 Acked Read 2 Done Sent Done false
14222 4978 Acked Write 8 Done Sent Done false
14223 4979 Acked Write 1 Done Sent Done false
14224 4980 Acked Write 2 Done Sent Done false
14225 4981 Acked Flush 0 Done Sent Done false
14226 4982 Acked Write 8 Done Sent Done false
14227 4983 Acked Write 2 Done Sent Done false
14228 4984 Acked Read 5 Done Sent Done false
14229 4985 Acked Flush 0 Done Sent Done false
14230 4986 Acked Read 10 Done Sent Done false
14231 4987 Acked Write 2 Done Sent Done false
14232 4988 Acked Write 2 Done Sent Done false
14233 4989 Acked Read 5 Done Sent Done false
14234 4990 Acked Read 9 Done Sent Done false
14235 4991 Acked Flush 0 Done Sent Done false
14236 4992 Acked Flush 0 Done Sent Done false
14237 4993 Acked Read 5 Done Sent Done false
14238 4994 Acked Flush 0 Done Sent Done false
14239 4995 Acked Read 7 Done Sent Done false
14240 4996 Acked Read 6 Done Sent Done false
14241 4997 Acked Read 2 Done Sent Done false
14242 4998 Acked Write 8 Done Sent Done false
14243 4999 Acked Read 3 Done Sent Done false
14244 5000 Acked Read 2 Done Sent Done false
14245 5001 Acked Read 1 Done Sent Done false
14246 5002 Acked Read 4 Done Sent Done false
14247 5003 Acked Write 6 Done Sent Done false
14248 5004 Acked Read 4 Done Sent Done false
14249 5005 Acked Write 3 Done Sent Done false
14250 5006 Acked Read 6 Done Sent Done false
14251 5007 Acked Write 6 Done Sent Done false
14252 5008 Acked Read 9 Done Sent Done false
14253 5009 Acked Read 8 Done Sent Done false
14254 5010 Acked Flush 0 Done Sent Done false
14255 5011 Acked Read 8 Done Sent Done false
14256 5012 Acked Flush 0 Done Sent Done false
14257 5013 Acked Read 3 Done Sent Done false
14258 5014 Acked Write 8 Done Sent Done false
14259 5015 Acked Write 2 Done Sent Done false
14260 5016 Acked Read 4 Done Sent Done false
14261 5017 Acked Read 3 Done Sent Done false
14262 5018 Acked Read 7 Done Sent Done false
14263 5019 Acked Read 5 Done Sent Done false
14264 5020 Acked Write 7 Done Sent Done false
14265 5021 Acked Write 6 Done Sent Done false
14266 5022 Acked Write 8 Done Sent Done false
14267 5023 Acked Read 2 Done Sent Done false
14268 5024 Acked Write 1 Done Sent Done false
14269 5025 Acked Write 6 Done Sent Done false
14270 5026 Acked Write 7 Done Sent Done false
14271 5027 Acked Read 3 Done Sent Done false
14272 5028 Acked Write 10 Done Sent Done false
14273 5029 Acked Read 9 Done Sent Done false
14274 5030 Acked Read 6 Done Sent Done false
14275 5031 Acked Read 6 Done Sent Done false
14276 5032 Acked Read 1 Done Sent Done false
14277 5033 Acked Read 2 Done Sent Done false
14278 5034 Acked Read 7 Done Sent Done false
14279 5035 Acked Read 7 Done Sent Done false
14280 5036 Acked Read 3 Done Sent Done false
14281 5037 Acked Read 1 Done Sent Done false
14282 5038 Acked Write 4 Done Sent Done false
14283 5039 Acked Read 7 Done Sent Done false
14284 5040 Acked Write 4 Done Sent Done false
14285 5041 Acked Write 5 Done Sent Done false
14286 5042 Acked Write 6 Done Sent Done false
14287 5043 Acked Write 10 Done Sent Done false
14288 5044 Acked Write 7 Done Sent Done false
14289 5045 Acked Read 10 Done Sent Done false
14290 5046 Acked Write 4 Done Sent Done false
14291 5047 Acked Write 1 Done Sent Done false
14292 5048 Acked Flush 0 Done Sent Done false
14293 5049 Acked Read 8 Done Sent Done false
14294 5050 Acked Read 9 Done Sent Done false
14295 5051 Acked Write 8 Done Sent Done false
14296 5052 Acked Write 7 Done Sent Done false
14297 5053 Acked Write 5 Done Sent Done false
14298 5054 Acked Write 4 Done Sent Done false
14299 5055 Acked Write 6 Done Sent Done false
14300 5056 Acked Flush 0 Done Sent Done false
14301 5057 Acked Read 3 Done Sent Done false
14302 5058 Acked Write 5 Done Sent Done false
14303 5059 Acked Write 9 Done Sent Done false
14304 5060 Acked Write 8 Done Sent Done false
14305 5061 Acked Read 4 Done Sent Done false
14306 5062 Acked Write 4 Done Sent Done false
14307 5063 Acked Flush 0 Done Sent Done false
14308 5064 Acked Write 3 Done Sent Done false
14309 5065 Acked Flush 0 Done Sent Done false
14310 5066 Acked Write 9 Done Sent Done false
14311 5067 Acked Read 1 Done Sent Done false
14312 5068 Acked Write 10 Done Sent Done false
14313 5069 Acked Read 6 Done Sent Done false
14314 5070 Acked Write 10 Done Sent Done false
14315 5071 Acked Read 9 Done Sent Done false
14316 5072 Acked Write 10 Done Sent Done false
14317 5073 Acked Write 10 Done Sent Done false
14318 5074 Acked Read 9 Done Sent Done false
14319 5075 Acked Write 2 Done Sent Done false
14320 5076 Acked Read 3 Done Sent Done false
14321 5077 Acked Write 4 Done Sent Done false
14322 5078 Acked Read 4 Done Sent Done false
14323 5079 Acked Read 9 Done Sent Done false
14324 5080 Acked Write 6 Done Sent Done false
14325 5081 Acked Write 2 Done Sent Done false
14326 5082 Acked Read 3 Done Sent Done false
14327 5083 Acked Read 6 Done Sent Done false
14328 5084 Acked Read 6 Done Sent Done false
14329 5085 Acked Flush 0 Done Sent Done false
14330 5086 Acked Write 5 Done Sent Done false
14331 5087 Acked Read 1 Done Sent Done false
14332 5088 Acked Read 3 Done Sent Done false
14333 5089 Acked Read 5 Done Sent Done false
14334 5090 Acked Write 7 Done Sent Done false
14335 5091 Acked Flush 0 Done Sent Done false
14336 5092 Acked Read 4 Done Sent Done false
14337 5093 Acked Write 2 Done Sent Done false
14338 5094 Acked Write 4 Done Sent Done false
14339 5095 Acked Read 4 Done Sent Done false
14340 5096 Acked Write 9 Done Sent Done false
14341 5097 Acked Read 6 Done Sent Done false
14342 5098 Acked Read 7 Done Sent Done false
14343 5099 Acked Read 8 Done Sent Done false
14344 5100 Acked Write 3 Done Sent Done false
14345 5101 Acked Write 7 Done Sent Done false
14346 5102 Acked Read 5 Done Sent Done false
14347 5103 Acked Write 4 Done Sent Done false
14348 5104 Acked Read 1 Done Sent Done false
14349 5105 Acked Write 4 Done Sent Done false
14350 5106 Acked Write 1 Done Sent Done false
14351 5107 Acked Read 3 Done Sent Done false
14352 5108 Acked Read 1 Done Sent Done false
14353 5109 Acked Read 3 Done Sent Done false
14354 5110 Acked Read 2 Done Sent Done false
14355 5111 Acked Read 6 Done Sent Done false
14356 5112 Acked Write 9 Done Sent Done false
14357 5113 Acked Write 9 Done Sent Done false
14358 5114 Acked Write 4 Done Sent Done false
14359 5115 Acked Write 5 Done Sent Done false
14360 5116 Acked Read 3 Done Sent Done false
14361 5117 Acked Read 1 Done Sent Done false
14362 5118 Acked Write 1 Done Sent Done false
14363 5119 Acked Write 3 Done Sent Done false
14364 5120 Acked Write 10 Done Sent Done false
14365 5121 Acked Write 3 Done Sent Done false
14366 5122 Acked Read 10 Done Sent Done false
14367 5123 Acked Write 8 Done Sent Done false
14368 5124 Acked Read 6 Done Sent Done false
14369 5125 Acked Write 8 Done Sent Done false
14370 5126 Acked Write 8 Done Sent Done false
14371 5127 Acked Write 3 Done Sent Done false
14372 5128 Acked Write 3 Done Sent Done false
14373 5129 Acked Flush 0 Done Sent Done false
14374 5130 Acked Read 4 Done Sent Done false
14375 5131 Acked Write 6 Done Sent Done false
14376 5132 Acked Read 7 Done Sent Done false
14377 5133 Acked Read 5 Done Sent Done false
14378 5134 Acked Flush 0 Done Sent Done false
14379 5135 Acked Write 2 Done Sent Done false
14380 5136 Acked Read 1 Done Sent Done false
14381 5137 Acked Read 6 Done Sent Done false
14382 5138 Acked Read 6 Done Sent Done false
14383 5139 Acked Write 3 Done Sent Done false
14384 5140 Acked Read 7 Done Sent Done false
14385 5141 Acked Read 2 Done Sent Done false
14386 5142 Acked Write 5 Done Sent Done false
14387 5143 Acked Write 1 Done Sent Done false
14388 5144 Acked Write 7 Done Sent Done false
14389 5145 Acked Write 6 Done Sent Done false
14390 5146 Acked Read 7 Done Sent Done false
14391 5147 Acked Read 1 Done Sent Done false
14392 5148 Acked Write 7 Done Sent Done false
14393 5149 Acked Read 4 Done Sent Done false
14394 5150 Acked Read 2 Done Sent Done false
14395 5151 Acked Write 8 Done Sent Done false
14396 5152 Acked Write 9 Done Sent Done false
14397 5153 Acked Read 7 Done Sent Done false
14398 5154 Acked Flush 0 Done Sent Done false
14399 5155 Acked Write 9 Done Sent Done false
14400 5156 Acked Write 6 Done Sent Done false
14401 5157 Acked Write 1 Done Sent Done false
14402 5158 Acked Read 6 Done Sent Done false
14403 5159 Acked Write 7 Done Sent Done false
14404 5160 Acked Write 8 Done Sent Done false
14405 5161 Acked Write 5 Done Sent Done false
14406 5162 Acked Write 2 Done Sent Done false
14407 5163 Acked Write 5 Done Sent Done false
14408 5164 Acked Flush 0 Done Sent Done false
14409 5165 Acked Write 8 Done Sent Done false
14410 5166 Acked Flush 0 Done Sent Done false
14411 5167 Acked Write 9 Done Sent Done false
14412 5168 Acked Read 5 Done Sent Done false
14413 5169 Acked Read 2 Done Sent Done false
14414 5170 Acked Write 6 Done Sent Done false
14415 5171 Acked Read 3 Done Sent Done false
14416 5172 Acked Write 8 Done Sent Done false
14417 5173 Acked Read 7 Done Sent Done false
14418 5174 Acked Read 3 Done Sent Done false
14419 5175 Acked Read 10 Done Sent Done false
14420 5176 Acked Read 3 Done Sent Done false
14421 5177 Acked Read 2 Done Sent Done false
14422 5178 Acked Write 6 Done Sent Done false
14423 5179 Acked Read 10 Done Sent Done false
14424 5180 Acked Write 4 Done Sent Done false
14425 5181 Acked Read 6 Done Sent Done false
14426 5182 Acked Read 7 Done Sent Done false
14427 5183 Acked Write 1 Done Sent Done false
14428 5184 Acked Read 9 Done Sent Done false
14429 5185 Acked Write 1 Done Sent Done false
14430 5186 Acked Flush 0 Done Sent Done false
14431 5187 Acked Write 9 Done Sent Done false
14432 5188 Acked Write 1 Done Sent Done false
14433 5189 Acked Write 3 Done Sent Done false
14434 5190 Acked Read 1 Done Sent Done false
14435 5191 Acked Read 7 Done Sent Done false
14436 5192 Acked Flush 0 Done Sent Done false
14437 5193 Acked Read 5 Done Sent Done false
14438 5194 Acked Write 9 Done Sent Done false
14439 5195 Acked Write 7 Done Sent Done false
14440 5196 Acked Read 2 Done Sent Done false
14441 5197 Acked Read 1 Done Sent Done false
14442 5198 Acked Write 3 Done Sent Done false
14443 5199 Acked Read 6 Done Sent Done false
14444 5200 Acked Write 6 Done Sent Done false
14445 5201 Acked Flush 0 Done Sent Done false
14446 5202 Acked Write 9 Done Sent Done false
14447 5203 Acked Write 5 Done Sent Done false
14448 5204 Acked Read 8 Done Sent Done false
14449 5205 Acked Flush 0 Done Sent Done false
14450 5206 Acked Write 10 Done Sent Done false
14451 5207 Acked Read 9 Done Sent Done false
14452 5208 Acked Write 6 Done Sent Done false
14453 5209 Acked Flush 0 Done Sent Done false
14454 5210 Acked Flush 0 Done Sent Done false
14455 5211 Acked Write 8 Done Sent Done false
14456 5212 Acked Read 10 Done Sent Done false
14457 5213 Acked Read 10 Done Sent Done false
14458 5214 Acked Write 9 Done Sent Done false
14459 5215 Acked Write 5 Done Sent Done false
14460 5216 Acked Read 7 Done Sent Done false
14461 5217 Acked Write 1 Done Sent Done false
14462 5218 Acked Read 6 Done Sent Done false
14463 5219 Acked Read 6 Done Sent Done false
14464 5220 Acked Read 6 Done Sent Done false
14465 5221 Acked Flush 0 Done Sent Done false
14466 5222 Acked Flush 0 Done Sent Done false
14467 5223 Acked Flush 0 Done Sent Done false
14468 5224 Acked Write 1 Done Sent Done false
14469 5225 Acked Flush 0 Done Sent Done false
14470 STATES DS:0 DS:1 DS:2 TOTAL
14471 Sent 0 302 0 302
14472 Done 302 0 302 604
14473 Skipped 0 0 0 0
14474 Error 0 0 0 0
14475 Last Flush: 5225 4923 5225
14476 Downstairs last five completed: 4923 4922 4921 4920 4919
14477 Upstairs last five completed: 5225 5224 5223 5222 5221
14478 CLIENT: Up:0 ds:302 act:2
144792024-12-04T13:33:43.333ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144802024-12-04T13:33:43.334ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40196) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144812024-12-04T13:33:43.334ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144822024-12-04T13:33:43.335ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144832024-12-04T13:33:43.335ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144842024-12-04T13:33:43.335ZINFOcrucible: send last flush ID to this DS: 4923 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144852024-12-04T13:33:43.335ZINFOcrucible: Replied this last flush ID: 4923 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144862024-12-04T13:33:43.335ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144872024-12-04T13:33:43.335ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144882024-12-04T13:33:43.335ZINFOcrucible: [1] client re-new 302 jobs since flush 4923 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
144892024-12-04T13:33:43.336ZINFOcrucible: [1] Replayed 302 jobs since flush: 4923 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
14490 ----------------------------------------------------------------
14491 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14492 Downstairs last five completed: 5225 5224 5223 5222 5221
14493 Upstairs last five completed: 5225 5224 5223 5222 5221
14494 CLIENT: Up:0 ds:0 act:3
14495 Replay: All jobs finished
14496 Main task has work to do, go find it
14497 got dsc Stop(0)
14498 stop 0
14499 Dec 04 13:33:45.564 INFO[8810] Got stop action so:false kr:false
14500 request completed, latency_us: 352, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 2221bc73-a42e-4e7d-9872-dd4e00ed1202, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
145012024-12-04T13:33:45.565ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
145022024-12-04T13:33:45.565ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
145032024-12-04T13:33:45.565ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
145042024-12-04T13:33:45.565ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
145052024-12-04T13:33:45.565ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
145062024-12-04T13:33:45.565ZINFOcrucible: client task is exiting = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
14507 Dec 04 13:33:45.565 INFO request completed, latency_us: 107, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2272a212-f7b2-45d5-91cc-2b7a826ab5ad, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
14508 [8810] Exited with: ExitStatus(unix_wait_status(9))
14509 [8810][0] reports Exit
14510 Dec 04 13:33:49.566 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/0, method: GET, req_id: d9bc2c21-bae0-4b6c-8003-4e937e112ce5, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
14511 001/300 Write block 3960 len 8192 data: 4 4
14512 002/300 Write block 3301 len 12288 data: 7 5 4
14513 003/300 Write block 4281 len 36864 data: 2 2 2 2 2 2 2 2 2
14514 004/300 Write block 1726 len 40960 data: 3 3 5 4 5 4 4 3 3 3
14515 005/300 Read block 3508 len 32768
14516 006/300 Read block 4958 len 20480
14517 007/300 Write block 1431 len 28672 data: 3 3 4 3 3 3 4
14518 008/300 Read block 1498 len 12288
14519 009/300 Read block 1060 len 36864
14520 010/300 Write block 1010 len 12288 data: 3 2 2
14521 011/300 Write block 4866 len 8192 data: 5 5
14522 012/300 Write block 1558 len 16384 data: 2 2 2 3
14523 013/300 Write block 1991 len 40960 data: 3 3 3 3 2 2 2 2 2 2
14524 014/300 Write block 3587 len 20480 data: 3 3 4 2 2
14525 015/300 Read block 988 len 20480
14526 016/300 Write block 3009 len 36864 data: 3 4 4 4 3 3 3 3 3
14527 017/300 Write block 2424 len 36864 data: 4 6 7 6 6 5 4 4 4
14528 018/300 Read block 2807 len 32768
14529 019/300 Write block 3577 len 40960 data: 2 2 2 2 2 2 2 3 3 3
14530 020/300 Write block 2385 len 32768 data: 4 4 3 3 4 4 4 4
14531 021/300 Write block 4801 len 28672 data: 4 4 4 3 3 3 2
14532 022/300 Read block 1426 len 36864
14533 023/300 Read block 2157 len 8192
14534 024/300 Write block 2781 len 16384 data: 3 4 4 4
14535 025/300 Write block 2019 len 40960 data: 3 3 4 4 4 2 3 4 4 4
14536 026/300 Write block 3959 len 40960 data: 3 5 5 5 5 4 3 4 3 3
14537 027/300 Read block 2376 len 20480
14538 028/300 Write block 2593 len 16384 data: 4 4 3 4
14539 029/300 Write block 3488 len 36864 data: 6 5 4 3 3 4 5 5 5
14540 030/300 Read block 3813 len 36864
14541 031/300 Write block 144 len 12288 data: 2 2 2
14542 032/300 Write block 2436 len 28672 data: 2 3 4 3 3 3 3
14543 033/300 Read block 426 len 8192
14544 034/300 Read block 1703 len 4096
14545 035/300 Write block 564 len 20480 data: 5 5 3 4 5
14546 036/300 Flush
14547 037/300 Write block 4755 len 40960 data: 5 5 4 5 3 3 3 3 3 5
14548 038/300 Read block 157 len 8192
14549 039/300 Write block 2346 len 40960 data: 6 6 7 5 5 4 6 5 3 3
14550 040/300 Read block 2449 len 24576
14551 041/300 Read block 1886 len 28672
14552 042/300 Read block 953 len 20480
14553 043/300 Write block 608 len 20480 data: 4 3 3 4 4
14554 044/300 Read block 1786 len 16384
14555 045/300 Read block 431 len 16384
14556 046/300 Write block 1271 len 32768 data: 4 4 2 2 2 2 2 3
14557 047/300 Write block 1816 len 12288 data: 4 4 4
14558 048/300 Write block 2166 len 32768 data: 6 6 6 5 6 5 6 5
14559 049/300 Write block 2762 len 32768 data: 5 5 5 5 4 4 3 5
14560 050/300 Flush
14561 051/300 Read block 1713 len 32768
14562 052/300 Flush
14563 053/300 Read block 995 len 36864
14564 054/300 Write block 296 len 4096 data: 6
14565 055/300 Read block 3508 len 32768
14566 056/300 Read block 3747 len 4096
14567 057/300 Read block 3096 len 20480
14568 058/300 Read block 3138 len 20480
14569 059/300 Read block 355 len 28672
14570 060/300 Read block 92 len 20480
14571 061/300 Read block 967 len 28672
14572 062/300 Read block 444 len 24576
14573 063/300 Write block 4003 len 40960 data: 3 3 4 4 3 3 4 4 3 3
14574 064/300 Write block 3681 len 8192 data: 5 5
14575 065/300 Read block 1586 len 40960
14576 066/300 Flush
14577 067/300 Read block 220 len 16384
14578 068/300 Write block 4560 len 8192 data: 2 2
14579 069/300 Flush
14580 070/300 Write block 3014 len 4096 data: 4
14581 071/300 Read block 4295 len 16384
14582 072/300 Read block 3169 len 36864
14583 073/300 Read block 4002 len 24576
14584 074/300 Read block 1960 len 32768
14585 075/300 Read block 4324 len 40960
14586 076/300 Write block 1348 len 20480 data: 5 4 4 4 3
14587 077/300 Write block 4153 len 32768 data: 5 5 7 7 6 7 10 10
14588 078/300 Write block 1988 len 16384 data: 3 3 3 4
14589 079/300 Read block 1321 len 28672
14590 080/300 Read block 4658 len 28672
14591 081/300 Write block 820 len 20480 data: 2 3 3 3 3
14592 082/300 Write block 1605 len 32768 data: 2 2 4 4 3 3 4 4
14593 083/300 Read block 3720 len 16384
14594 084/300 Write block 4779 len 4096 data: 3
14595 085/300 Flush
14596 086/300 Write block 1895 len 20480 data: 4 5 3 4 4
14597 087/300 Write block 1031 len 4096 data: 3
14598 088/300 Write block 4057 len 32768 data: 5 5 5 5 5 4 3 3
14599 089/300 Write block 1465 len 28672 data: 3 3 2 3 3 6 8
14600 090/300 Read block 254 len 16384
14601 091/300 Read block 2178 len 28672
14602 092/300 Read block 2414 len 4096
14603 093/300 Write block 501 len 4096 data: 3
14604 094/300 Read block 3750 len 20480
14605 095/300 Read block 1995 len 28672
14606 096/300 Write block 2978 len 36864 data: 2 2 3 4 4 4 5 5 4
14607 097/300 Read block 3454 len 20480
14608 098/300 Write block 16 len 16384 data: 6 5 3 2
14609 099/300 Write block 3475 len 12288 data: 6 7 7
14610 100/300 Read block 2757 len 16384
14611 101/300 Write block 4780 len 4096 data: 2
14612 102/300 Read block 113 len 12288
14613 103/300 Write block 1225 len 40960 data: 3 3 2 2 2 2 2 4 4 4
14614 104/300 Flush
14615 105/300 Read block 40 len 8192
14616 106/300 Write block 3593 len 12288 data: 2 4 4
14617 107/300 Read block 776 len 4096
14618 108/300 Read block 1632 len 8192
14619 109/300 Read block 2704 len 16384
14620 110/300 Write block 2869 len 32768 data: 7 6 5 5 5 6 5 4
14621 111/300 Read block 2295 len 16384
14622 112/300 Write block 1097 len 40960 data: 3 5 4 4 4 4 4 5 5 5
14623 113/300 Flush
14624 114/300 Flush
14625 115/300 Write block 2651 len 4096 data: 5
14626 116/300 Write block 1380 len 24576 data: 3 2 2 3 3 3
14627 117/300 Read block 1710 len 40960
14628 118/300 Write block 1543 len 24576 data: 3 3 3 3 2 2
14629 119/300 Read block 1442 len 20480
14630 120/300 Read block 3250 len 8192
14631 121/300 Write block 3549 len 36864 data: 5 4 5 5 5 4 2 2 2
14632 122/300 Read block 4101 len 36864
14633 123/300 Read block 1257 len 28672
14634 124/300 Read block 2457 len 20480
14635 125/300 Write block 1306 len 4096 data: 2
14636 126/300 Read block 1626 len 20480
14637 127/300 Write block 2247 len 4096 data: 7
14638 128/300 Write block 2710 len 24576 data: 6 6 6 3 2 2
14639 129/300 Write block 1334 len 32768 data: 3 3 3 4 4 4 4 4
14640 130/300 Write block 3757 len 4096 data: 4
14641 131/300 Write block 4798 len 32768 data: 4 4 4 5 5 5 4 4
14642 132/300 Write block 3443 len 12288 data: 3 3 2
14643 133/300 Read block 1087 len 32768
14644 134/300 Read block 1630 len 28672
14645 135/300 Flush
14646 136/300 Read block 675 len 24576
14647 137/300 Read block 3866 len 36864
14648 138/300 Read block 2244 len 36864
14649 139/300 Read block 2583 len 32768
14650 140/300 Read block 2441 len 8192
14651 141/300 Flush
14652 142/300 Write block 4336 len 24576 data: 3 3 3 4 4 4
14653 143/300 Read block 4054 len 32768
14654 144/300 Read block 4505 len 32768
14655 145/300 Flush
14656 146/300 Read block 3215 len 36864
14657 147/300 Read block 641 len 20480
14658 148/300 Write block 4647 len 20480 data: 3 3 3 3 3
14659 149/300 Write block 3872 len 4096 data: 3
14660 150/300 Write block 2078 len 36864 data: 3 3 3 3 3 3 3 3 2
14661 151/300 Flush
14662 152/300 Flush
14663 153/300 Read block 4246 len 24576
14664 154/300 Read block 1394 len 12288
14665 155/300 Read block 870 len 8192
14666 156/300 Write block 3412 len 16384 data: 3 2 2 3
14667 157/300 Write block 4969 len 12288 data: 7 6 5
14668 158/300 Read block 2479 len 36864
14669 159/300 Read block 4152 len 20480
14670 160/300 Write block 1684 len 20480 data: 8 7 7 6 5
14671 161/300 Read block 1112 len 16384
14672 162/300 Read block 2273 len 36864
14673 163/300 Read block 3723 len 28672
14674 164/300 Read block 1237 len 8192
14675 165/300 Write block 1244 len 8192 data: 4 4
14676 166/300 Flush
14677 167/300 Write block 4467 len 28672 data: 5 6 7 8 7 7 5
14678 168/300 Read block 1020 len 36864
14679 169/300 Write block 4577 len 36864 data: 3 3 2 2 2 2 2 3 3
14680 170/300 Read block 811 len 36864
14681 171/300 Read block 4662 len 28672
14682 172/300 Write block 509 len 24576 data: 5 5 5 3 3 5
14683 173/300 Write block 4896 len 28672 data: 4 2 3 3 3 4 4
14684 174/300 Write block 3436 len 4096 data: 8
14685 175/300 Read block 1548 len 28672
14686 176/300 Read block 849 len 28672
14687 177/300 Write block 3153 len 24576 data: 7 6 5 5 4 4
14688 178/300 Read block 4796 len 8192
14689 179/300 Read block 3337 len 20480
14690 180/300 Read block 1094 len 4096
14691 181/300 Write block 3817 len 4096 data: 6
14692 182/300 Write block 545 len 12288 data: 5 5 4
14693 183/300 Read block 2201 len 40960
14694 184/300 Write block 4217 len 24576 data: 4 4 5 4 4 4
14695 185/300 Write block 1537 len 32768 data: 4 3 3 4 3 3 4 4
14696 186/300 Write block 1762 len 36864 data: 9 8 7 6 5 5 6 5 4
14697 187/300 Read block 4400 len 32768
14698 188/300 Write block 4157 len 28672 data: 7 8 11 11 10 10 9
14699 189/300 Write block 1397 len 32768 data: 3 4 4 5 3 4 5 5
14700 190/300 Read block 4208 len 28672
14701 191/300 Read block 363 len 28672
14702 192/300 Write block 3974 len 16384 data: 4 4 2 3
14703 193/300 Write block 702 len 20480 data: 2 2 2 3 3
14704 194/300 Read block 4913 len 40960
14705 195/300 Write block 4197 len 16384 data: 5 4 3 5
14706 196/300 Flush
14707 197/300 Read block 826 len 40960
14708 198/300 Write block 3089 len 32768 data: 3 4 4 4 3 2 2 3
14709 199/300 Read block 2469 len 36864
14710 200/300 Read block 3846 len 16384
14711 201/300 Write block 1030 len 28672 data: 2 4 3 3 3 2 3
14712 202/300 Write block 4525 len 4096 data: 5
14713 203/300 Read block 2945 len 12288
14714 204/300 Write block 4332 len 32768 data: 7 5 5 4 4 4 4 5
14715 205/300 Read block 1770 len 20480
14716 206/300 Read block 2063 len 36864
14717 207/300 Read block 1696 len 28672
14718 208/300 Read block 1817 len 36864
14719 209/300 Write block 949 len 20480 data: 4 4 4 3 3
14720 210/300 Flush
14721 211/300 Write block 4348 len 40960 data: 3 3 3 5 5 4 5 4 3 4
14722 212/300 Write block 2333 len 20480 data: 2 3 4 3 4
14723 213/300 Read block 4277 len 40960
14724 214/300 Read block 1211 len 20480
14725 215/300 Read block 2330 len 36864
14726 216/300 Read block 605 len 28672
14727 217/300 Read block 3181 len 12288
14728 218/300 Write block 833 len 40960 data: 5 5 5 4 5 4 5 5 4 3
14729 219/300 Write block 1101 len 12288 data: 5 5 5
14730 220/300 Read block 23 len 32768
14731 221/300 Write block 72 len 16384 data: 4 4 4 4
14732 222/300 Read block 3310 len 16384
14733 223/300 Read block 2262 len 32768
14734 224/300 Read block 4646 len 36864
14735 225/300 Read block 1314 len 16384
14736 226/300 Write block 65 len 12288 data: 4 4 4
14737 227/300 Read block 4678 len 24576
14738 228/300 Write block 417 len 40960 data: 3 3 3 3 3 2 2 2 2 2
14739 229/300 Write block 2998 len 24576 data: 3 3 4 7 7 6
14740 230/300 Write block 1691 len 12288 data: 5 7 7
14741 231/300 Read block 2399 len 4096
14742 232/300 Read block 1363 len 36864
14743 233/300 Write block 2481 len 40960 data: 6 5 7 7 6 5 5 4 4 2
14744 234/300 Read block 4096 len 28672
14745 235/300 Write block 450 len 36864 data: 4 3 2 3 3 4 5 5 5
14746 236/300 Read block 1395 len 24576
14747 237/300 Read block 3960 len 24576
14748 238/300 Read block 3092 len 4096
14749 239/300 Read block 3535 len 32768
14750 240/300 Read block 346 len 4096
14751 241/300 Write block 3417 len 40960 data: 5 7 6 6 4 4 3 4 3 3
14752 242/300 Write block 2402 len 12288 data: 4 4 4
14753 243/300 Write block 771 len 28672 data: 7 5 4 4 4 3 3
14754 244/300 Read block 1767 len 40960
14755 245/300 Write block 4274 len 24576 data: 5 5 5 4 3 3
14756 246/300 Read block 3849 len 32768
14757 247/300 Read block 1380 len 12288
14758 248/300 Flush
14759 249/300 Read block 4703 len 36864
14760 250/300 Read block 2535 len 12288
14761 251/300 Read block 2775 len 12288
14762 252/300 Write block 4091 len 20480 data: 4 4 4 4 4
14763 253/300 Read block 3804 len 36864
14764 254/300 Write block 1839 len 36864 data: 5 5 5 5 5 4 4 4 4
14765 255/300 Read block 1407 len 32768
14766 256/300 Write block 1250 len 20480 data: 5 4 3 3 4
14767 257/300 Read block 1827 len 4096
14768 258/300 Read block 2061 len 24576
14769 259/300 Read block 4907 len 40960
14770 260/300 Write block 1721 len 16384 data: 2 2 3 3
14771 261/300 Read block 767 len 12288
14772 262/300 Read block 4058 len 12288
14773 263/300 Write block 1852 len 8192 data: 5 4
14774 264/300 Read block 435 len 24576
14775 265/300 Read block 2731 len 28672
14776 266/300 Read block 3307 len 36864
14777 267/300 Write block 1055 len 16384 data: 6 6 7 7
14778 268/300 Write block 2299 len 36864 data: 3 3 5 4 3 3 2 2 2
14779 269/300 Flush
14780 270/300 Read block 229 len 32768
14781 271/300 Read block 323 len 20480
14782 272/300 Read block 1167 len 28672
14783 273/300 Flush
14784 274/300 Read block 4632 len 32768
14785 275/300 Write block 4482 len 40960 data: 3 5 6 6 7 7 6 5 5 5
14786 276/300 Write block 4424 len 20480 data: 3 3 3 3 2
14787 277/300 Read block 4564 len 24576
14788 278/300 Read block 4806 len 20480
14789 279/300 Read block 1994 len 40960
14790 280/300 Write block 2290 len 36864 data: 6 6 7 7 7 5 4 3 4
14791 281/300 Read block 2460 len 4096
14792 282/300 Read block 3156 len 4096
14793 283/300 Write block 1289 len 16384 data: 3 4 4 4
14794 284/300 Flush
14795 285/300 Write block 4076 len 32768 data: 3 3 2 2 3 3 3 3
14796 286/300 Flush
14797 287/300 Read block 1245 len 24576
14798 288/300 Read block 868 len 12288
14799 289/300 Read block 4979 len 20480
14800 290/300 Write block 175 len 28672 data: 4 3 2 2 2 2 2
14801 291/300 Read block 920 len 24576
14802 292/300 Write block 4175 len 16384 data: 4 4 3 3
14803 293/300 Read block 1190 len 24576
14804 294/300 Read block 2933 len 40960
14805 295/300 Write block 1450 len 40960 data: 2 4 4 4 4 6 6 7 7 6
14806 296/300 Read block 4337 len 16384
14807 297/300 Read block 1868 len 8192
14808 298/300 Write block 4457 len 4096 data: 6
14809 299/300 Read block 2569 len 40960
14810 300/300 Read block 3908 len 28672
14811 301/300 Flush
14812 Main task has work to do, go find it
14813 got dsc Start(0)
14814 start 0
14815 Dec 04 13:33:49.810 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/0, method: POST, req_id: 8e864bf6-bdd9-48fb-838a-b7318426bd36, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
14816 Replay: started 0, returned:Ok(())
14817 ----------------------------------------------------------------
14818 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14819 States: Offline Active Active
14820 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14821 5226 Acked Write 2 Sent Done Done false
14822 5227 Acked Write 3 Sent Done Done false
14823 5228 Acked Write 9 Sent Done Done false
14824 5229 Acked Write 10 Sent Done Done false
14825 5230 Acked Read 8 Sent Done Done false
14826 5231 Acked Read 5 Sent Done Done false
14827 5232 Acked Write 7 Sent Done Done false
14828 5233 Acked Read 3 Sent Done Done false
14829 5234 Acked Read 9 Sent Done Done false
14830 5235 Acked Write 3 Sent Done Done false
14831 5236 Acked Write 2 Sent Done Done false
14832 5237 Acked Write 4 Sent Done Done false
14833 5238 Acked Write 10 Sent Done Done false
14834 5239 Acked Write 5 Sent Done Done false
14835 5240 Acked Read 5 Sent Done Done false
14836 5241 Acked Write 9 Sent Done Done false
14837 5242 Acked Write 9 Sent Done Done false
14838 5243 Acked Read 8 Sent Done Done false
14839 5244 Acked Write 10 Sent Done Done false
14840 5245 Acked Write 8 Sent Done Done false
14841 5246 Acked Write 7 Sent Done Done false
14842 5247 Acked Read 9 Sent Done Done false
14843 5248 Acked Read 2 Sent Done Done false
14844 5249 Acked Write 4 Sent Done Done false
14845 5250 Acked Write 10 Sent Done Done false
14846 5251 Acked Write 10 Sent Done Done false
14847 5252 Acked Read 5 Sent Done Done false
14848 5253 Acked Write 4 Sent Done Done false
14849 5254 Acked Write 9 Sent Done Done false
14850 5255 Acked Read 9 Sent Done Done false
14851 5256 Acked Write 3 Sent Done Done false
14852 5257 Acked Write 7 Sent Done Done false
14853 5258 Acked Read 2 Sent Done Done false
14854 5259 Acked Read 1 Sent Done Done false
14855 5260 Acked Write 5 Sent Done Done false
14856 5261 Acked Flush 0 Sent Done Done false
14857 5262 Acked Write 10 Sent Done Done false
14858 5263 Acked Read 2 Sent Done Done false
14859 5264 Acked Write 10 Sent Done Done false
14860 5265 Acked Read 6 Sent Done Done false
14861 5266 Acked Read 7 Sent Done Done false
14862 5267 Acked Read 5 Sent Done Done false
14863 5268 Acked Write 5 Sent Done Done false
14864 5269 Acked Read 4 Sent Done Done false
14865 5270 Acked Read 4 Sent Done Done false
14866 5271 Acked Write 8 Sent Done Done false
14867 5272 Acked Write 3 Sent Done Done false
14868 5273 Acked Write 8 Sent Done Done false
14869 5274 Acked Write 8 Sent Done Done false
14870 5275 Acked Flush 0 Sent Done Done false
14871 5276 Acked Read 8 Sent Done Done false
14872 5277 Acked Flush 0 Sent Done Done false
14873 5278 Acked Read 9 Sent Done Done false
14874 5279 Acked Write 1 Sent Done Done false
14875 5280 Acked Read 8 Sent Done Done false
14876 5281 Acked Read 1 Sent Done Done false
14877 5282 Acked Read 5 Sent Done Done false
14878 5283 Acked Read 5 Sent Done Done false
14879 5284 Acked Read 7 Sent Done Done false
14880 5285 Acked Read 5 Sent Done Done false
14881 5286 Acked Read 7 Sent Done Done false
14882 5287 Acked Read 6 Sent Done Done false
14883 5288 Acked Write 10 Sent Done Done false
14884 5289 Acked Write 2 Sent Done Done false
14885 5290 Acked Read 10 Sent Done Done false
14886 5291 Acked Flush 0 Sent Done Done false
14887 5292 Acked Read 4 Sent Done Done false
14888 5293 Acked Write 2 Sent Done Done false
14889 5294 Acked Flush 0 Sent Done Done false
14890 5295 Acked Write 1 Sent Done Done false
14891 5296 Acked Read 4 Sent Done Done false
14892 5297 Acked Read 9 Sent Done Done false
14893 5298 Acked Read 6 Sent Done Done false
14894 5299 Acked Read 8 Sent Done Done false
14895 5300 Acked Read 10 Sent Done Done false
14896 5301 Acked Write 5 Sent Done Done false
14897 5302 Acked Write 8 Sent Done Done false
14898 5303 Acked Write 4 Sent Done Done false
14899 5304 Acked Read 7 Sent Done Done false
14900 5305 Acked Read 7 Sent Done Done false
14901 5306 Acked Write 5 Sent Done Done false
14902 5307 Acked Write 8 Sent Done Done false
14903 5308 Acked Read 4 Sent Done Done false
14904 5309 Acked Write 1 Sent Done Done false
14905 5310 Acked Flush 0 Sent Done Done false
14906 5311 Acked Write 5 Sent Done Done false
14907 5312 Acked Write 1 Sent Done Done false
14908 5313 Acked Write 8 Sent Done Done false
14909 5314 Acked Write 7 Sent Done Done false
14910 5315 Acked Read 4 Sent Done Done false
14911 5316 Acked Read 7 Sent Done Done false
14912 5317 Acked Read 1 Sent Done Done false
14913 5318 Acked Write 1 Sent Done Done false
14914 5319 Acked Read 5 Sent Done Done false
14915 5320 Acked Read 7 Sent Done Done false
14916 5321 Acked Write 9 Sent Done Done false
14917 5322 Acked Read 5 Sent Done Done false
14918 5323 Acked Write 4 Sent Done Done false
14919 5324 Acked Write 3 Sent Done Done false
14920 5325 Acked Read 4 Sent Done Done false
14921 5326 Acked Write 1 Sent Done Done false
14922 5327 Acked Read 3 Sent Done Done false
14923 5328 Acked Write 10 Sent Done Done false
14924 5329 Acked Flush 0 Sent Done Done false
14925 5330 Acked Read 2 Sent Done Done false
14926 5331 Acked Write 3 Sent Done Done false
14927 5332 Acked Read 1 Sent Done Done false
14928 5333 Acked Read 2 Sent Done Done false
14929 5334 Acked Read 4 Sent Done Done false
14930 5335 Acked Write 8 Sent Done Done false
14931 5336 Acked Read 4 Sent Done Done false
14932 5337 Acked Write 10 Sent Done Done false
14933 5338 Acked Flush 0 Sent Done Done false
14934 5339 Acked Flush 0 Sent Done Done false
14935 5340 Acked Write 1 Sent Done Done false
14936 5341 Acked Write 6 Sent Done Done false
14937 5342 Acked Read 10 Sent Done Done false
14938 5343 Acked Write 6 Sent Done Done false
14939 5344 Acked Read 5 Sent Done Done false
14940 5345 Acked Read 2 Sent Done Done false
14941 5346 Acked Write 9 Sent Done Done false
14942 5347 Acked Read 9 Sent Done Done false
14943 5348 Acked Read 7 Sent Done Done false
14944 5349 Acked Read 5 Sent Done Done false
14945 5350 Acked Write 1 Sent Done Done false
14946 5351 Acked Read 5 Sent Done Done false
14947 5352 Acked Write 1 Sent Done Done false
14948 5353 Acked Write 6 Sent Done Done false
14949 5354 Acked Write 8 Sent Done Done false
14950 5355 Acked Write 1 Sent Done Done false
14951 5356 Acked Write 8 Sent Done Done false
14952 5357 Acked Write 3 Sent Done Done false
14953 5358 Acked Read 8 Sent Done Done false
14954 5359 Acked Read 7 Sent Done Done false
14955 5360 Acked Flush 0 Sent Done Done false
14956 5361 Acked Read 6 Sent Done Done false
14957 5362 Acked Read 9 Sent Done Done false
14958 5363 Acked Read 9 Sent Done Done false
14959 5364 Acked Read 8 Sent Done Done false
14960 5365 Acked Read 2 Sent Done Done false
14961 5366 Acked Flush 0 Sent Done Done false
14962 5367 Acked Write 6 Sent Done Done false
14963 5368 Acked Read 8 Sent Done Done false
14964 5369 Acked Read 8 Sent Done Done false
14965 5370 Acked Flush 0 Sent Done Done false
14966 5371 Acked Read 9 Sent Done Done false
14967 5372 Acked Read 5 Sent Done Done false
14968 5373 Acked Write 5 Sent Done Done false
14969 5374 Acked Write 1 Sent Done Done false
14970 5375 Acked Write 9 Sent Done Done false
14971 5376 Acked Flush 0 Sent Done Done false
14972 5377 Acked Flush 0 Sent Done Done false
14973 5378 Acked Read 6 Sent Done Done false
14974 5379 Acked Read 3 Sent Done Done false
14975 5380 Acked Read 2 Sent Done Done false
14976 5381 Acked Write 4 Sent Done Done false
14977 5382 Acked Write 3 Sent Done Done false
14978 5383 Acked Read 9 Sent Done Done false
14979 5384 Acked Read 5 Sent Done Done false
14980 5385 Acked Write 5 Sent Done Done false
14981 5386 Acked Read 4 Sent Done Done false
14982 5387 Acked Read 9 Sent Done Done false
14983 5388 Acked Read 7 Sent Done Done false
14984 5389 Acked Read 2 Sent Done Done false
14985 5390 Acked Write 2 Sent Done Done false
14986 5391 Acked Flush 0 Sent Done Done false
14987 5392 Acked Write 7 Sent Done Done false
14988 5393 Acked Read 9 Sent Done Done false
14989 5394 Acked Write 9 Sent Done Done false
14990 5395 Acked Read 9 Sent Done Done false
14991 5396 Acked Read 7 Sent Done Done false
14992 5397 Acked Write 6 Sent Done Done false
14993 5398 Acked Write 7 Sent Done Done false
14994 5399 Acked Write 1 Sent Done Done false
14995 5400 Acked Read 7 Sent Done Done false
14996 5401 Acked Read 7 Sent Done Done false
14997 5402 Acked Write 6 Sent Done Done false
14998 5403 Acked Read 2 Sent Done Done false
14999 5404 Acked Read 5 Sent Done Done false
15000 5405 Acked Read 1 Sent Done Done false
15001 5406 Acked Write 1 Sent Done Done false
15002 5407 Acked Write 3 Sent Done Done false
15003 5408 Acked Read 10 Sent Done Done false
15004 5409 Acked Write 6 Sent Done Done false
15005 5410 Acked Write 8 Sent Done Done false
15006 5411 Acked Write 9 Sent Done Done false
15007 5412 Acked Read 8 Sent Done Done false
15008 5413 Acked Write 7 Sent Done Done false
15009 5414 Acked Write 8 Sent Done Done false
15010 5415 Acked Read 7 Sent Done Done false
15011 5416 Acked Read 7 Sent Done Done false
15012 5417 Acked Write 4 Sent Done Done false
15013 5418 Acked Write 5 Sent Done Done false
15014 5419 Acked Read 10 Sent Done Done false
15015 5420 Acked Write 4 Sent Done Done false
15016 5421 Acked Flush 0 Sent Done Done false
15017 5422 Acked Read 10 Sent Done Done false
15018 5423 Acked Write 8 Sent Done Done false
15019 5424 Acked Read 9 Sent Done Done false
15020 5425 Acked Read 4 Sent Done Done false
15021 5426 Acked Write 7 Sent Done Done false
15022 5427 Acked Write 1 Sent Done Done false
15023 5428 Acked Read 3 Sent Done Done false
15024 5429 Acked Write 8 Sent Done Done false
15025 5430 Acked Read 5 Sent Done Done false
15026 5431 Acked Read 9 Sent Done Done false
15027 5432 Acked Read 7 Sent Done Done false
15028 5433 Acked Read 9 Sent Done Done false
15029 5434 Acked Write 5 Sent Done Done false
15030 5435 Acked Flush 0 Sent Done Done false
15031 5436 Acked Write 10 Sent Done Done false
15032 5437 Acked Write 5 Sent Done Done false
15033 5438 Acked Read 10 Sent Done Done false
15034 5439 Acked Read 5 Sent Done Done false
15035 5440 Acked Read 9 Sent Done Done false
15036 5441 Acked Read 7 Sent Done Done false
15037 5442 Acked Read 3 Sent Done Done false
15038 5443 Acked Write 10 Sent Done Done false
15039 5444 Acked Write 3 Sent Done Done false
15040 5445 Acked Read 8 Sent Done Done false
15041 5446 Acked Write 4 Sent Done Done false
15042 5447 Acked Read 4 Sent Done Done false
15043 5448 Acked Read 8 Sent Done Done false
15044 5449 Acked Read 9 Sent Done Done false
15045 5450 Acked Read 4 Sent Done Done false
15046 5451 Acked Write 3 Sent Done Done false
15047 5452 Acked Read 6 Sent Done Done false
15048 5453 Acked Write 10 Sent Done Done false
15049 5454 Acked Write 6 Sent Done Done false
15050 5455 Acked Write 3 Sent Done Done false
15051 5456 Acked Read 1 Sent Done Done false
15052 5457 Acked Read 9 Sent Done Done false
15053 5458 Acked Write 10 Sent Done Done false
15054 5459 Acked Read 7 Sent Done Done false
15055 5460 Acked Write 9 Sent Done Done false
15056 5461 Acked Read 6 Sent Done Done false
15057 5462 Acked Read 6 Sent Done Done false
15058 5463 Acked Read 1 Sent Done Done false
15059 5464 Acked Read 8 Sent Done Done false
15060 5465 Acked Read 1 Sent Done Done false
15061 5466 Acked Write 10 Sent Done Done false
15062 5467 Acked Write 3 Sent Done Done false
15063 5468 Acked Write 7 Sent Done Done false
15064 5469 Acked Read 10 Sent Done Done false
15065 5470 Acked Write 6 Sent Done Done false
15066 5471 Acked Read 8 Sent Done Done false
15067 5472 Acked Read 3 Sent Done Done false
15068 5473 Acked Flush 0 Sent Done Done false
15069 5474 Acked Read 9 Sent Done Done false
15070 5475 Acked Read 3 Sent Done Done false
15071 5476 Acked Read 3 Sent Done Done false
15072 5477 Acked Write 5 Sent Done Done false
15073 5478 Acked Read 9 Sent Done Done false
15074 5479 Acked Write 9 Sent Done Done false
15075 5480 Acked Read 8 Sent Done Done false
15076 5481 Acked Write 5 Sent Done Done false
15077 5482 Acked Read 1 Sent Done Done false
15078 5483 Acked Read 6 Sent Done Done false
15079 5484 Acked Read 10 Sent Done Done false
15080 5485 Acked Write 4 Sent Done Done false
15081 5486 Acked Read 3 Sent Done Done false
15082 5487 Acked Read 3 Sent Done Done false
15083 5488 Acked Write 2 Sent Done Done false
15084 5489 Acked Read 6 Sent Done Done false
15085 5490 Acked Read 7 Sent Done Done false
15086 5491 Acked Read 9 Sent Done Done false
15087 5492 Acked Write 4 Sent Done Done false
15088 5493 Acked Write 9 Sent Done Done false
15089 5494 Acked Flush 0 Sent Done Done false
15090 5495 Acked Read 8 Sent Done Done false
15091 5496 Acked Read 5 Sent Done Done false
15092 5497 Acked Read 7 Sent Done Done false
15093 5498 Acked Flush 0 Sent Done Done false
15094 5499 Acked Read 8 Sent Done Done false
15095 5500 Acked Write 10 Sent Done Done false
15096 5501 Acked Write 5 Sent Done Done false
15097 5502 Acked Read 6 Sent Done Done false
15098 5503 Acked Read 5 Sent Done Done false
15099 5504 Acked Read 10 Sent Done Done false
15100 5505 Acked Write 9 Sent Done Done false
15101 5506 Acked Read 1 Sent Done Done false
15102 5507 Acked Read 1 Sent Done Done false
15103 5508 Acked Write 4 Sent Done Done false
15104 5509 Acked Flush 0 Sent Done Done false
15105 5510 Acked Write 8 Sent Done Done false
15106 5511 Acked Flush 0 Sent Done Done false
15107 5512 Acked Read 6 Sent Done Done false
15108 5513 Acked Read 3 Sent Done Done false
15109 5514 Acked Read 5 Sent Done Done false
15110 5515 Acked Write 7 Sent Done Done false
15111 5516 Acked Read 6 Sent Done Done false
15112 5517 Acked Write 4 Sent Done Done false
15113 5518 Acked Read 6 Sent Done Done false
15114 5519 Acked Read 10 Sent Done Done false
15115 5520 Acked Write 10 Sent Done Done false
15116 5521 Acked Read 4 Sent Done Done false
15117 5522 Acked Read 2 Sent Done Done false
15118 5523 Acked Write 1 Sent Done Done false
15119 5524 Acked Read 10 Sent Done Done false
15120 5525 Acked Read 7 Sent Done Done false
15121 5526 Acked Flush 0 Sent Done Done false
15122 STATES DS:0 DS:1 DS:2 TOTAL
15123 Sent 301 0 0 301
15124 Done 0 301 301 602
15125 Skipped 0 0 0 0
15126 Error 0 0 0 0
15127 Last Flush: 5225 5526 5526
15128 Downstairs last five completed: 5225 5224 5223 5222 5221
15129 Upstairs last five completed: 5526 5525 5524 5523 5522
15130 CLIENT: Up:0 ds:301 act:2
15131 [8810] got start action
15132 [8810] I am going to restart
15133 Starting downstairs at port 8810
15134 Make output file at "/tmp/dsc/downstairs-8810.txt"
15135 [8810][0] reports Starting
15136 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1218)
15137 [8810][0] reports Running
15138 ----------------------------------------------------------------
15139 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15140 States: Offline Active Active
15141 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15142 5226 Acked Write 2 Sent Done Done false
15143 5227 Acked Write 3 Sent Done Done false
15144 5228 Acked Write 9 Sent Done Done false
15145 5229 Acked Write 10 Sent Done Done false
15146 5230 Acked Read 8 Sent Done Done false
15147 5231 Acked Read 5 Sent Done Done false
15148 5232 Acked Write 7 Sent Done Done false
15149 5233 Acked Read 3 Sent Done Done false
15150 5234 Acked Read 9 Sent Done Done false
15151 5235 Acked Write 3 Sent Done Done false
15152 5236 Acked Write 2 Sent Done Done false
15153 5237 Acked Write 4 Sent Done Done false
15154 5238 Acked Write 10 Sent Done Done false
15155 5239 Acked Write 5 Sent Done Done false
15156 5240 Acked Read 5 Sent Done Done false
15157 5241 Acked Write 9 Sent Done Done false
15158 5242 Acked Write 9 Sent Done Done false
15159 5243 Acked Read 8 Sent Done Done false
15160 5244 Acked Write 10 Sent Done Done false
15161 5245 Acked Write 8 Sent Done Done false
15162 5246 Acked Write 7 Sent Done Done false
15163 5247 Acked Read 9 Sent Done Done false
15164 5248 Acked Read 2 Sent Done Done false
15165 5249 Acked Write 4 Sent Done Done false
15166 5250 Acked Write 10 Sent Done Done false
15167 5251 Acked Write 10 Sent Done Done false
15168 5252 Acked Read 5 Sent Done Done false
15169 5253 Acked Write 4 Sent Done Done false
15170 5254 Acked Write 9 Sent Done Done false
15171 5255 Acked Read 9 Sent Done Done false
15172 5256 Acked Write 3 Sent Done Done false
15173 5257 Acked Write 7 Sent Done Done false
15174 5258 Acked Read 2 Sent Done Done false
15175 5259 Acked Read 1 Sent Done Done false
15176 5260 Acked Write 5 Sent Done Done false
15177 5261 Acked Flush 0 Sent Done Done false
15178 5262 Acked Write 10 Sent Done Done false
15179 5263 Acked Read 2 Sent Done Done false
15180 5264 Acked Write 10 Sent Done Done false
15181 5265 Acked Read 6 Sent Done Done false
15182 5266 Acked Read 7 Sent Done Done false
15183 5267 Acked Read 5 Sent Done Done false
15184 5268 Acked Write 5 Sent Done Done false
15185 5269 Acked Read 4 Sent Done Done false
15186 5270 Acked Read 4 Sent Done Done false
15187 5271 Acked Write 8 Sent Done Done false
15188 5272 Acked Write 3 Sent Done Done false
15189 5273 Acked Write 8 Sent Done Done false
15190 5274 Acked Write 8 Sent Done Done false
15191 5275 Acked Flush 0 Sent Done Done false
15192 5276 Acked Read 8 Sent Done Done false
15193 5277 Acked Flush 0 Sent Done Done false
15194 5278 Acked Read 9 Sent Done Done false
15195 5279 Acked Write 1 Sent Done Done false
15196 5280 Acked Read 8 Sent Done Done false
15197 5281 Acked Read 1 Sent Done Done false
15198 5282 Acked Read 5 Sent Done Done false
15199 5283 Acked Read 5 Sent Done Done false
15200 5284 Acked Read 7 Sent Done Done false
15201 5285 Acked Read 5 Sent Done Done false
15202 5286 Acked Read 7 Sent Done Done false
15203 5287 Acked Read 6 Sent Done Done false
15204 5288 Acked Write 10 Sent Done Done false
15205 5289 Acked Write 2 Sent Done Done false
15206 5290 Acked Read 10 Sent Done Done false
15207 5291 Acked Flush 0 Sent Done Done false
15208 5292 Acked Read 4 Sent Done Done false
15209 5293 Acked Write 2 Sent Done Done false
15210 5294 Acked Flush 0 Sent Done Done false
15211 5295 Acked Write 1 Sent Done Done false
15212 5296 Acked Read 4 Sent Done Done false
15213 5297 Acked Read 9 Sent Done Done false
15214 5298 Acked Read 6 Sent Done Done false
15215 5299 Acked Read 8 Sent Done Done false
15216 5300 Acked Read 10 Sent Done Done false
15217 5301 Acked Write 5 Sent Done Done false
15218 5302 Acked Write 8 Sent Done Done false
15219 5303 Acked Write 4 Sent Done Done false
15220 5304 Acked Read 7 Sent Done Done false
15221 5305 Acked Read 7 Sent Done Done false
15222 5306 Acked Write 5 Sent Done Done false
15223 5307 Acked Write 8 Sent Done Done false
15224 5308 Acked Read 4 Sent Done Done false
15225 5309 Acked Write 1 Sent Done Done false
15226 5310 Acked Flush 0 Sent Done Done false
15227 5311 Acked Write 5 Sent Done Done false
15228 5312 Acked Write 1 Sent Done Done false
15229 5313 Acked Write 8 Sent Done Done false
15230 5314 Acked Write 7 Sent Done Done false
15231 5315 Acked Read 4 Sent Done Done false
15232 5316 Acked Read 7 Sent Done Done false
15233 5317 Acked Read 1 Sent Done Done false
15234 5318 Acked Write 1 Sent Done Done false
15235 5319 Acked Read 5 Sent Done Done false
15236 5320 Acked Read 7 Sent Done Done false
15237 5321 Acked Write 9 Sent Done Done false
15238 5322 Acked Read 5 Sent Done Done false
15239 5323 Acked Write 4 Sent Done Done false
15240 5324 Acked Write 3 Sent Done Done false
15241 5325 Acked Read 4 Sent Done Done false
15242 5326 Acked Write 1 Sent Done Done false
15243 5327 Acked Read 3 Sent Done Done false
15244 5328 Acked Write 10 Sent Done Done false
15245 5329 Acked Flush 0 Sent Done Done false
15246 5330 Acked Read 2 Sent Done Done false
15247 5331 Acked Write 3 Sent Done Done false
15248 5332 Acked Read 1 Sent Done Done false
15249 5333 Acked Read 2 Sent Done Done false
15250 5334 Acked Read 4 Sent Done Done false
15251 5335 Acked Write 8 Sent Done Done false
15252 5336 Acked Read 4 Sent Done Done false
15253 5337 Acked Write 10 Sent Done Done false
15254 5338 Acked Flush 0 Sent Done Done false
15255 5339 Acked Flush 0 Sent Done Done false
15256 5340 Acked Write 1 Sent Done Done false
15257 5341 Acked Write 6 Sent Done Done false
15258 5342 Acked Read 10 Sent Done Done false
15259 5343 Acked Write 6 Sent Done Done false
15260 5344 Acked Read 5 Sent Done Done false
15261 5345 Acked Read 2 Sent Done Done false
15262 5346 Acked Write 9 Sent Done Done false
15263 5347 Acked Read 9 Sent Done Done false
15264 5348 Acked Read 7 Sent Done Done false
15265 5349 Acked Read 5 Sent Done Done false
15266 5350 Acked Write 1 Sent Done Done false
15267 5351 Acked Read 5 Sent Done Done false
15268 5352 Acked Write 1 Sent Done Done false
15269 5353 Acked Write 6 Sent Done Done false
15270 5354 Acked Write 8 Sent Done Done false
15271 5355 Acked Write 1 Sent Done Done false
15272 5356 Acked Write 8 Sent Done Done false
15273 5357 Acked Write 3 Sent Done Done false
15274 5358 Acked Read 8 Sent Done Done false
15275 5359 Acked Read 7 Sent Done Done false
15276 5360 Acked Flush 0 Sent Done Done false
15277 5361 Acked Read 6 Sent Done Done false
15278 5362 Acked Read 9 Sent Done Done false
15279 5363 Acked Read 9 Sent Done Done false
15280 5364 Acked Read 8 Sent Done Done false
15281 5365 Acked Read 2 Sent Done Done false
15282 5366 Acked Flush 0 Sent Done Done false
15283 5367 Acked Write 6 Sent Done Done false
15284 5368 Acked Read 8 Sent Done Done false
15285 5369 Acked Read 8 Sent Done Done false
15286 5370 Acked Flush 0 Sent Done Done false
15287 5371 Acked Read 9 Sent Done Done false
15288 5372 Acked Read 5 Sent Done Done false
15289 5373 Acked Write 5 Sent Done Done false
15290 5374 Acked Write 1 Sent Done Done false
15291 5375 Acked Write 9 Sent Done Done false
15292 5376 Acked Flush 0 Sent Done Done false
15293 5377 Acked Flush 0 Sent Done Done false
15294 5378 Acked Read 6 Sent Done Done false
15295 5379 Acked Read 3 Sent Done Done false
15296 5380 Acked Read 2 Sent Done Done false
15297 5381 Acked Write 4 Sent Done Done false
15298 5382 Acked Write 3 Sent Done Done false
15299 5383 Acked Read 9 Sent Done Done false
15300 5384 Acked Read 5 Sent Done Done false
15301 5385 Acked Write 5 Sent Done Done false
15302 5386 Acked Read 4 Sent Done Done false
15303 5387 Acked Read 9 Sent Done Done false
15304 5388 Acked Read 7 Sent Done Done false
15305 5389 Acked Read 2 Sent Done Done false
15306 5390 Acked Write 2 Sent Done Done false
15307 5391 Acked Flush 0 Sent Done Done false
15308 5392 Acked Write 7 Sent Done Done false
15309 5393 Acked Read 9 Sent Done Done false
15310 5394 Acked Write 9 Sent Done Done false
15311 5395 Acked Read 9 Sent Done Done false
15312 5396 Acked Read 7 Sent Done Done false
15313 5397 Acked Write 6 Sent Done Done false
15314 5398 Acked Write 7 Sent Done Done false
15315 5399 Acked Write 1 Sent Done Done false
15316 5400 Acked Read 7 Sent Done Done false
15317 5401 Acked Read 7 Sent Done Done false
15318 5402 Acked Write 6 Sent Done Done false
15319 5403 Acked Read 2 Sent Done Done false
15320 5404 Acked Read 5 Sent Done Done false
15321 5405 Acked Read 1 Sent Done Done false
15322 5406 Acked Write 1 Sent Done Done false
15323 5407 Acked Write 3 Sent Done Done false
15324 5408 Acked Read 10 Sent Done Done false
15325 5409 Acked Write 6 Sent Done Done false
15326 5410 Acked Write 8 Sent Done Done false
15327 5411 Acked Write 9 Sent Done Done false
15328 5412 Acked Read 8 Sent Done Done false
15329 5413 Acked Write 7 Sent Done Done false
15330 5414 Acked Write 8 Sent Done Done false
15331 5415 Acked Read 7 Sent Done Done false
15332 5416 Acked Read 7 Sent Done Done false
15333 5417 Acked Write 4 Sent Done Done false
15334 5418 Acked Write 5 Sent Done Done false
15335 5419 Acked Read 10 Sent Done Done false
15336 5420 Acked Write 4 Sent Done Done false
15337 5421 Acked Flush 0 Sent Done Done false
15338 5422 Acked Read 10 Sent Done Done false
15339 5423 Acked Write 8 Sent Done Done false
15340 5424 Acked Read 9 Sent Done Done false
15341 5425 Acked Read 4 Sent Done Done false
15342 5426 Acked Write 7 Sent Done Done false
15343 5427 Acked Write 1 Sent Done Done false
15344 5428 Acked Read 3 Sent Done Done false
15345 5429 Acked Write 8 Sent Done Done false
15346 5430 Acked Read 5 Sent Done Done false
15347 5431 Acked Read 9 Sent Done Done false
15348 5432 Acked Read 7 Sent Done Done false
15349 5433 Acked Read 9 Sent Done Done false
15350 5434 Acked Write 5 Sent Done Done false
15351 5435 Acked Flush 0 Sent Done Done false
15352 5436 Acked Write 10 Sent Done Done false
15353 5437 Acked Write 5 Sent Done Done false
15354 5438 Acked Read 10 Sent Done Done false
15355 5439 Acked Read 5 Sent Done Done false
15356 5440 Acked Read 9 Sent Done Done false
15357 5441 Acked Read 7 Sent Done Done false
15358 5442 Acked Read 3 Sent Done Done false
15359 5443 Acked Write 10 Sent Done Done false
15360 5444 Acked Write 3 Sent Done Done false
15361 5445 Acked Read 8 Sent Done Done false
15362 5446 Acked Write 4 Sent Done Done false
15363 5447 Acked Read 4 Sent Done Done false
15364 5448 Acked Read 8 Sent Done Done false
15365 5449 Acked Read 9 Sent Done Done false
15366 5450 Acked Read 4 Sent Done Done false
15367 5451 Acked Write 3 Sent Done Done false
15368 5452 Acked Read 6 Sent Done Done false
15369 5453 Acked Write 10 Sent Done Done false
15370 5454 Acked Write 6 Sent Done Done false
15371 5455 Acked Write 3 Sent Done Done false
15372 5456 Acked Read 1 Sent Done Done false
15373 5457 Acked Read 9 Sent Done Done false
15374 5458 Acked Write 10 Sent Done Done false
15375 5459 Acked Read 7 Sent Done Done false
15376 5460 Acked Write 9 Sent Done Done false
15377 5461 Acked Read 6 Sent Done Done false
15378 5462 Acked Read 6 Sent Done Done false
15379 5463 Acked Read 1 Sent Done Done false
15380 5464 Acked Read 8 Sent Done Done false
15381 5465 Acked Read 1 Sent Done Done false
15382 5466 Acked Write 10 Sent Done Done false
15383 5467 Acked Write 3 Sent Done Done false
15384 5468 Acked Write 7 Sent Done Done false
15385 5469 Acked Read 10 Sent Done Done false
15386 5470 Acked Write 6 Sent Done Done false
15387 5471 Acked Read 8 Sent Done Done false
15388 5472 Acked Read 3 Sent Done Done false
15389 5473 Acked Flush 0 Sent Done Done false
15390 5474 Acked Read 9 Sent Done Done false
15391 5475 Acked Read 3 Sent Done Done false
15392 5476 Acked Read 3 Sent Done Done false
15393 5477 Acked Write 5 Sent Done Done false
15394 5478 Acked Read 9 Sent Done Done false
15395 5479 Acked Write 9 Sent Done Done false
15396 5480 Acked Read 8 Sent Done Done false
15397 5481 Acked Write 5 Sent Done Done false
15398 5482 Acked Read 1 Sent Done Done false
15399 5483 Acked Read 6 Sent Done Done false
15400 5484 Acked Read 10 Sent Done Done false
15401 5485 Acked Write 4 Sent Done Done false
15402 5486 Acked Read 3 Sent Done Done false
15403 5487 Acked Read 3 Sent Done Done false
15404 5488 Acked Write 2 Sent Done Done false
15405 5489 Acked Read 6 Sent Done Done false
15406 5490 Acked Read 7 Sent Done Done false
15407 5491 Acked Read 9 Sent Done Done false
15408 5492 Acked Write 4 Sent Done Done false
15409 5493 Acked Write 9 Sent Done Done false
15410 5494 Acked Flush 0 Sent Done Done false
15411 5495 Acked Read 8 Sent Done Done false
15412 5496 Acked Read 5 Sent Done Done false
15413 5497 Acked Read 7 Sent Done Done false
15414 5498 Acked Flush 0 Sent Done Done false
15415 5499 Acked Read 8 Sent Done Done false
15416 5500 Acked Write 10 Sent Done Done false
15417 5501 Acked Write 5 Sent Done Done false
15418 5502 Acked Read 6 Sent Done Done false
15419 5503 Acked Read 5 Sent Done Done false
15420 5504 Acked Read 10 Sent Done Done false
15421 5505 Acked Write 9 Sent Done Done false
15422 5506 Acked Read 1 Sent Done Done false
15423 5507 Acked Read 1 Sent Done Done false
15424 5508 Acked Write 4 Sent Done Done false
15425 5509 Acked Flush 0 Sent Done Done false
15426 5510 Acked Write 8 Sent Done Done false
15427 5511 Acked Flush 0 Sent Done Done false
15428 5512 Acked Read 6 Sent Done Done false
15429 5513 Acked Read 3 Sent Done Done false
15430 5514 Acked Read 5 Sent Done Done false
15431 5515 Acked Write 7 Sent Done Done false
15432 5516 Acked Read 6 Sent Done Done false
15433 5517 Acked Write 4 Sent Done Done false
15434 5518 Acked Read 6 Sent Done Done false
15435 5519 Acked Read 10 Sent Done Done false
15436 5520 Acked Write 10 Sent Done Done false
15437 5521 Acked Read 4 Sent Done Done false
15438 5522 Acked Read 2 Sent Done Done false
15439 5523 Acked Write 1 Sent Done Done false
15440 5524 Acked Read 10 Sent Done Done false
15441 5525 Acked Read 7 Sent Done Done false
15442 5526 Acked Flush 0 Sent Done Done false
15443 STATES DS:0 DS:1 DS:2 TOTAL
15444 Sent 301 0 0 301
15445 Done 0 301 301 602
15446 Skipped 0 0 0 0
15447 Error 0 0 0 0
15448 Last Flush: 5225 5526 5526
15449 Downstairs last five completed: 5225 5224 5223 5222 5221
15450 Upstairs last five completed: 5526 5525 5524 5523 5522
15451 CLIENT: Up:0 ds:301 act:2
154522024-12-04T13:33:55.566ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154532024-12-04T13:33:55.566ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48752) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154542024-12-04T13:33:55.567ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154552024-12-04T13:33:55.567ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154562024-12-04T13:33:55.567ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154572024-12-04T13:33:55.567ZINFOcrucible: send last flush ID to this DS: 5225 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154582024-12-04T13:33:55.568ZINFOcrucible: Replied this last flush ID: 5225 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154592024-12-04T13:33:55.568ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154602024-12-04T13:33:55.568ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154612024-12-04T13:33:55.568ZINFOcrucible: [0] client re-new 301 jobs since flush 5225 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154622024-12-04T13:33:55.568ZINFOcrucible: [0] Replayed 301 jobs since flush: 5225 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
15463 ----------------------------------------------------------------
15464 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15465 Downstairs last five completed: 5526 5525 5524 5523 5522
15466 Upstairs last five completed: 5526 5525 5524 5523 5522
15467 CLIENT: Up:0 ds:0 act:3
15468 Replay: All jobs finished
15469 Main task has work to do, go find it
15470 got dsc Stop(0)
15471 stop 0
15472 Dec 04 13:33:57.816 [8810] Got stop action so:false kr:false
15473 INFO request completed, latency_us: 310, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 7d697754-8859-4c5f-9649-48b4a8298d20, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
154742024-12-04T13:33:57.816ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154752024-12-04T13:33:57.816ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154762024-12-04T13:33:57.816ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154772024-12-04T13:33:57.816ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154782024-12-04T13:33:57.816ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
154792024-12-04T13:33:57.816ZINFOcrucible: client task is exiting = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
15480 Dec 04 13:33:57.816 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8b8bafa7-9786-4e64-924b-d1ab315d0664, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
15481 [8810] Exited with: ExitStatus(unix_wait_status(9))
15482 [8810][0] reports Exit
15483 Dec 04 13:34:01.819 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8f61d0c6-378d-49f5-909a-852a9c5490e0, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
15484 001/300 Write block 2452 len 24576 data: 2 2 2 2 3 4
15485 002/300 Read block 2862 len 4096
15486 003/300 Read block 68 len 20480
15487 004/300 Read block 3459 len 16384
15488 005/300 Write block 4457 len 8192 data: 7 6
15489 006/300 Write block 3306 len 40960 data: 3 3 3 3 3 3 3 3 3 5
15490 007/300 Read block 3578 len 40960
15491 008/300 Write block 3124 len 28672 data: 3 4 5 5 4 4 4
15492 009/300 Read block 2599 len 24576
15493 010/300 Write block 2537 len 4096 data: 4
15494 011/300 Read block 3666 len 16384
15495 012/300 Read block 4649 len 4096
15496 013/300 Write block 2208 len 4096 data: 2
15497 014/300 Write block 246 len 40960 data: 3 4 5 5 5 5 6 5 5 6
15498 015/300 Read block 1636 len 40960
15499 016/300 Read block 4182 len 20480
15500 017/300 Write block 1366 len 28672 data: 3 3 3 3 3 4 5
15501 018/300 Read block 3261 len 8192
15502 019/300 Read block 3363 len 32768
15503 020/300 Write block 4095 len 20480 data: 5 3 3 2 2
15504 021/300 Write block 3450 len 4096 data: 2
15505 022/300 Read block 598 len 28672
15506 023/300 Read block 4007 len 24576
15507 024/300 Read block 2426 len 36864
15508 025/300 Write block 4803 len 28672 data: 6 5 5 4 3 2 2
15509 026/300 Write block 3013 len 28672 data: 4 5 4 4 4 4 4
15510 027/300 Write block 4016 len 12288 data: 4 5 5
15511 028/300 Read block 744 len 28672
15512 029/300 Read block 4671 len 36864
15513 030/300 Read block 3408 len 8192
15514 031/300 Write block 4325 len 28672 data: 2 4 4 3 5 5 7
15515 032/300 Flush
15516 033/300 Write block 3356 len 8192 data: 3 3
15517 034/300 Read block 3473 len 32768
15518 035/300 Write block 2313 len 32768 data: 5 4 5 3 3 3 4 5
15519 036/300 Read block 4757 len 32768
15520 037/300 Flush
15521 038/300 Write block 4908 len 36864 data: 6 7 5 3 3 3 3 3 4
15522 039/300 Read block 2637 len 24576
15523 040/300 Read block 4485 len 20480
15524 041/300 Flush
15525 042/300 Read block 2793 len 36864
15526 043/300 Flush
15527 044/300 Read block 4483 len 20480
15528 045/300 Read block 1145 len 4096
15529 046/300 Write block 2536 len 20480 data: 2 5 4 4 5
15530 047/300 Write block 739 len 32768 data: 5 4 3 2 2 2 3 3
15531 048/300 Read block 1484 len 28672
15532 049/300 Read block 2165 len 40960
15533 050/300 Read block 4632 len 16384
15534 051/300 Write block 2395 len 24576 data: 5 5 4 4 4 4
15535 052/300 Flush
15536 053/300 Write block 3432 len 16384 data: 6 6 6 8
15537 054/300 Write block 1838 len 8192 data: 5 6
15538 055/300 Flush
15539 056/300 Read block 4442 len 28672
15540 057/300 Write block 3224 len 28672 data: 3 3 3 2 2 5 5
15541 058/300 Read block 4255 len 32768
15542 059/300 Flush
15543 060/300 Read block 3520 len 40960
15544 061/300 Read block 2684 len 36864
15545 062/300 Flush
15546 063/300 Write block 2199 len 4096 data: 2
15547 064/300 Write block 3551 len 8192 data: 6 6
15548 065/300 Write block 2097 len 12288 data: 2 2 2
15549 066/300 Write block 2138 len 40960 data: 6 5 6 5 5 5 4 3 4 3
15550 067/300 Write block 2791 len 36864 data: 6 5 3 4 4 3 3 4 3
15551 068/300 Write block 4339 len 4096 data: 6
15552 069/300 Write block 1346 len 4096 data: 6
15553 070/300 Read block 2273 len 16384
15554 071/300 Write block 2762 len 28672 data: 6 6 6 6 5 5 4
15555 072/300 Flush
15556 073/300 Read block 2907 len 16384
15557 074/300 Write block 2980 len 32768 data: 4 5 5 5 6 6 5 5
15558 075/300 Read block 796 len 40960
15559 076/300 Write block 1790 len 20480 data: 2 2 2 2 2
15560 077/300 Write block 4471 len 4096 data: 8
15561 078/300 Write block 4494 len 12288 data: 2 2 2
15562 079/300 Write block 4582 len 40960 data: 3 3 4 4 4 4 4 4 4 4
15563 080/300 Read block 955 len 28672
15564 081/300 Write block 2846 len 40960 data: 5 5 5 5 3 4 6 4 4 4
15565 082/300 Write block 4343 len 32768 data: 4 4 4 4 3 4 4 4
15566 083/300 Flush
15567 084/300 Write block 1099 len 8192 data: 5 5
15568 085/300 Read block 2476 len 36864
15569 086/300 Read block 4400 len 40960
15570 087/300 Read block 2172 len 12288
15571 088/300 Read block 875 len 16384
15572 089/300 Write block 1321 len 8192 data: 3 3
15573 090/300 Read block 3740 len 20480
15574 091/300 Read block 2213 len 8192
15575 092/300 Read block 3888 len 12288
15576 093/300 Write block 1322 len 40960 data: 4 3 3 3 3 3 2 2 2 2
15577 094/300 Write block 644 len 24576 data: 3 3 3 3 3 3
15578 095/300 Write block 3331 len 24576 data: 2 2 2 2 2 2
15579 096/300 Write block 3258 len 16384 data: 2 2 2 2
15580 097/300 Write block 136 len 16384 data: 2 2 3 3
15581 098/300 Write block 137 len 36864 data: 3 4 4 2 2 2 2 3 3
15582 099/300 Read block 4323 len 36864
15583 100/300 Read block 3540 len 32768
15584 101/300 Read block 774 len 20480
15585 102/300 Read block 4047 len 40960
15586 103/300 Read block 503 len 4096
15587 104/300 Read block 2476 len 28672
15588 105/300 Read block 731 len 20480
15589 106/300 Read block 1084 len 24576
15590 107/300 Read block 2209 len 8192
15591 108/300 Read block 4947 len 20480
15592 109/300 Read block 1698 len 20480
15593 110/300 Write block 1910 len 28672 data: 3 5 5 5 5 5 5
15594 111/300 Write block 4936 len 4096 data: 3
15595 112/300 Flush
15596 113/300 Write block 2095 len 20480 data: 3 3 3 3 3
15597 114/300 Read block 3845 len 12288
15598 115/300 Read block 3534 len 16384
15599 116/300 Write block 2283 len 20480 data: 3 4 3 3 4
15600 117/300 Read block 2662 len 4096
15601 118/300 Write block 2513 len 36864 data: 2 2 2 2 2 2 2 3 4
15602 119/300 Write block 2585 len 24576 data: 2 3 4 4 6 5
15603 120/300 Flush
15604 121/300 Write block 2515 len 40960 data: 3 3 3 3 3 4 5 5 5 5
15605 122/300 Write block 1135 len 32768 data: 2 3 3 4 3 4 4 4
15606 123/300 Read block 77 len 8192
15607 124/300 Read block 543 len 8192
15608 125/300 Write block 4008 len 12288 data: 4 5 5
15609 126/300 Write block 3103 len 8192 data: 4 3
15610 127/300 Write block 2654 len 36864 data: 4 4 4 4 4 3 3 3 2
15611 128/300 Write block 3484 len 28672 data: 6 7 6 5 7 6 5
15612 129/300 Write block 4399 len 8192 data: 5 5
15613 130/300 Read block 52 len 8192
15614 131/300 Write block 3140 len 40960 data: 3 3 3 3 2 2 3 3 4 5
15615 132/300 Read block 3670 len 20480
15616 133/300 Write block 1997 len 24576 data: 3 3 3 3 2 3
15617 134/300 Write block 1523 len 20480 data: 3 3 6 7 6
15618 135/300 Write block 3226 len 4096 data: 4
15619 136/300 Write block 4479 len 40960 data: 3 3 3 4 6 7 7 8 8 7
15620 137/300 Write block 2196 len 8192 data: 5 3
15621 138/300 Write block 3326 len 24576 data: 3 3 3 3 2 3
15622 139/300 Write block 4112 len 28672 data: 5 5 4 3 3 3 4
15623 140/300 Read block 4865 len 20480
15624 141/300 Flush
15625 142/300 Read block 4294 len 36864
15626 143/300 Read block 3200 len 4096
15627 144/300 Read block 4364 len 28672
15628 145/300 Flush
15629 146/300 Read block 2936 len 4096
15630 147/300 Flush
15631 148/300 Read block 874 len 36864
15632 149/300 Read block 2834 len 32768
15633 150/300 Read block 452 len 40960
15634 151/300 Write block 855 len 32768 data: 3 4 3 3 3 4 3 2
15635 152/300 Read block 2468 len 16384
15636 153/300 Flush
15637 154/300 Read block 3933 len 8192
15638 155/300 Read block 3789 len 16384
15639 156/300 Write block 3478 len 16384 data: 7 7 8 6
15640 157/300 Read block 3125 len 16384
15641 158/300 Flush
15642 159/300 Read block 2051 len 28672
15643 160/300 Read block 2255 len 12288
15644 161/300 Read block 1715 len 8192
15645 162/300 Read block 4543 len 16384
15646 163/300 Flush
15647 164/300 Write block 3532 len 40960 data: 5 5 4 4 2 3 3 3 3 4
15648 165/300 Write block 1405 len 8192 data: 5 5
15649 166/300 Flush
15650 167/300 Read block 4429 len 8192
15651 168/300 Read block 3192 len 4096
15652 169/300 Write block 4510 len 4096 data: 4
15653 170/300 Read block 3088 len 12288
15654 171/300 Read block 3465 len 40960
15655 172/300 Write block 3662 len 20480 data: 7 7 6 5 4
15656 173/300 Read block 681 len 32768
15657 174/300 Read block 681 len 28672
15658 175/300 Read block 3996 len 28672
15659 176/300 Write block 3194 len 12288 data: 4 4 3
15660 177/300 Read block 309 len 20480
15661 178/300 Write block 4792 len 28672 data: 3 2 2 3 3 3 5
15662 179/300 Read block 1732 len 24576
15663 180/300 Flush
15664 181/300 Write block 615 len 32768 data: 2 2 2 3 4 4 5 4
15665 182/300 Read block 1215 len 16384
15666 183/300 Read block 600 len 40960
15667 184/300 Read block 1186 len 16384
15668 185/300 Read block 4970 len 20480
15669 186/300 Write block 997 len 12288 data: 3 2 3
15670 187/300 Read block 4440 len 20480
15671 188/300 Read block 3794 len 24576
15672 189/300 Read block 398 len 4096
15673 190/300 Read block 3734 len 32768
15674 191/300 Read block 3250 len 36864
15675 192/300 Flush
15676 193/300 Write block 2964 len 36864 data: 2 2 2 2 3 3 3 3 3
15677 194/300 Write block 1207 len 12288 data: 4 4 5
15678 195/300 Write block 2973 len 4096 data: 3
15679 196/300 Write block 4065 len 16384 data: 3 3 2 2
15680 197/300 Write block 4330 len 28672 data: 6 8 8 6 6 5 5
15681 198/300 Read block 3750 len 20480
15682 199/300 Write block 3663 len 32768 data: 8 7 6 5 5 4 5 5
15683 200/300 Write block 3698 len 8192 data: 4 4
15684 201/300 Read block 1178 len 40960
15685 202/300 Flush
15686 203/300 Write block 3856 len 40960 data: 2 2 2 3 4 4 4 4 4 4
15687 204/300 Read block 2729 len 36864
15688 205/300 Write block 4005 len 28672 data: 5 5 4 5 6 6 4
15689 206/300 Read block 3498 len 8192
15690 207/300 Read block 2441 len 16384
15691 208/300 Read block 3136 len 12288
15692 209/300 Read block 2780 len 8192
15693 210/300 Read block 497 len 36864
15694 211/300 Read block 1863 len 40960
15695 212/300 Read block 275 len 24576
15696 213/300 Read block 3270 len 20480
15697 214/300 Write block 1493 len 16384 data: 3 2 2 2
15698 215/300 Write block 3302 len 16384 data: 6 5 3 4
15699 216/300 Read block 755 len 40960
15700 217/300 Write block 92 len 12288 data: 4 3 3
15701 218/300 Read block 4472 len 20480
15702 219/300 Write block 1439 len 20480 data: 4 5 5 5 2
15703 220/300 Read block 3237 len 4096
15704 221/300 Write block 1012 len 20480 data: 3 2 3 3 4
15705 222/300 Read block 2590 len 12288
15706 223/300 Write block 3088 len 8192 data: 3 4
15707 224/300 Flush
15708 225/300 Write block 3507 len 20480 data: 2 2 3 3 3
15709 226/300 Flush
15710 227/300 Read block 3133 len 8192
15711 228/300 Read block 2046 len 40960
15712 229/300 Write block 1618 len 36864 data: 3 3 2 2 3 2 2 2 2
15713 230/300 Read block 753 len 28672
15714 231/300 Write block 204 len 20480 data: 3 2 4 4 4
15715 232/300 Write block 65 len 12288 data: 5 5 5
15716 233/300 Flush
15717 234/300 Read block 2218 len 12288
15718 235/300 Read block 4750 len 12288
15719 236/300 Write block 1766 len 28672 data: 6 6 7 6 5 3 3
15720 237/300 Flush
15721 238/300 Read block 4037 len 16384
15722 239/300 Write block 1076 len 28672 data: 4 7 6 5 5 5 5
15723 240/300 Flush
15724 241/300 Write block 626 len 8192 data: 3 3
15725 242/300 Write block 1494 len 24576 data: 3 3 3 5 5 5
15726 243/300 Read block 1091 len 20480
15727 244/300 Flush
15728 245/300 Write block 3939 len 20480 data: 5 5 6 5 6
15729 246/300 Read block 2985 len 40960
15730 247/300 Read block 998 len 12288
15731 248/300 Write block 3618 len 8192 data: 3 2
15732 249/300 Read block 1563 len 8192
15733 250/300 Read block 3845 len 40960
15734 251/300 Read block 3771 len 28672
15735 252/300 Write block 55 len 32768 data: 5 4 4 4 3 4 4 3
15736 253/300 Write block 1135 len 20480 data: 3 4 4 5 4
15737 254/300 Read block 1279 len 16384
15738 255/300 Flush
15739 256/300 Write block 3389 len 40960 data: 3 4 4 4 4 4 4 5 3 4
15740 257/300 Read block 3592 len 24576
15741 258/300 Read block 1413 len 28672
15742 259/300 Read block 3870 len 36864
15743 260/300 Write block 1580 len 28672 data: 3 4 4 4 3 3 3
15744 261/300 Write block 2602 len 8192 data: 5 5
15745 262/300 Write block 1281 len 36864 data: 2 2 2 3 3 3 3 3 4
15746 263/300 Write block 3241 len 4096 data: 4
15747 264/300 Write block 1951 len 8192 data: 7 5
15748 265/300 Read block 1660 len 28672
15749 266/300 Write block 1725 len 24576 data: 3 4 4 6 5 6
15750 267/300 Write block 1510 len 24576 data: 6 8 8 7 7 7
15751 268/300 Flush
15752 269/300 Write block 3499 len 20480 data: 3 3 4 4 4
15753 270/300 Write block 315 len 16384 data: 2 3 3 3
15754 271/300 Read block 1173 len 24576
15755 272/300 Read block 3964 len 28672
15756 273/300 Write block 725 len 40960 data: 2 3 4 4 5 5 6 5 5 4
15757 274/300 Write block 390 len 16384 data: 4 5 5 7
15758 275/300 Flush
15759 276/300 Flush
15760 277/300 Read block 494 len 4096
15761 278/300 Write block 227 len 24576 data: 3 3 3 4 4 2
15762 279/300 Read block 30 len 28672
15763 280/300 Write block 989 len 12288 data: 5 5 4
15764 281/300 Write block 1478 len 4096 data: 7
15765 282/300 Read block 1125 len 4096
15766 283/300 Read block 3670 len 28672
15767 284/300 Write block 4767 len 24576 data: 3 3 2 2 2 3
15768 285/300 Write block 2436 len 20480 data: 3 4 5 4 4
15769 286/300 Write block 3140 len 12288 data: 4 4 4
15770 287/300 Read block 12 len 12288
15771 288/300 Write block 2634 len 24576 data: 4 3 4 4 4 4
15772 289/300 Write block 3443 len 40960 data: 4 4 3 3 3 2 2 3 3 3
15773 290/300 Read block 3070 len 32768
15774 291/300 Flush
15775 292/300 Read block 4535 len 40960
15776 293/300 Flush
15777 294/300 Read block 2995 len 4096
15778 295/300 Read block 1215 len 12288
15779 296/300 Write block 4543 len 8192 data: 4 4
15780 297/300 Read block 963 len 4096
15781 298/300 Write block 635 len 12288 data: 3 3 3
15782 299/300 Read block 2357 len 16384
15783 300/300 Read block 1417 len 20480
15784 301/300 Read block 4156 len 32768
15785 Main task has work to do, go find it
15786 got dsc Start(0)
15787 start 0
15788 Dec 04 13:34:02.047 INFO request completed, latency_us: 245, response_code: 204, uri: /start/cid/0, method: POST, req_id: fae56023-44cf-4665-8700-dbeceea64c1a, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
15789 Replay: started 0, returned:Ok(())
15790 ----------------------------------------------------------------
15791 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15792 States: Offline Active Active
15793 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15794 5527 Acked Write 6 Sent Done Done false
15795 5528 Acked Read 1 Sent Done Done false
15796 5529 Acked Read 5 Sent Done Done false
15797 5530 Acked Read 4 Sent Done Done false
15798 5531 Acked Write 2 Sent Done Done false
15799 5532 Acked Write 10 Sent Done Done false
15800 5533 Acked Read 10 Sent Done Done false
15801 5534 Acked Write 7 Sent Done Done false
15802 5535 Acked Read 6 Sent Done Done false
15803 5536 Acked Write 1 Sent Done Done false
15804 5537 Acked Read 4 Sent Done Done false
15805 5538 Acked Read 1 Sent Done Done false
15806 5539 Acked Write 1 Sent Done Done false
15807 5540 Acked Write 10 Sent Done Done false
15808 5541 Acked Read 10 Sent Done Done false
15809 5542 Acked Read 5 Sent Done Done false
15810 5543 Acked Write 7 Sent Done Done false
15811 5544 Acked Read 2 Sent Done Done false
15812 5545 Acked Read 8 Sent Done Done false
15813 5546 Acked Write 5 Sent Done Done false
15814 5547 Acked Write 1 Sent Done Done false
15815 5548 Acked Read 7 Sent Done Done false
15816 5549 Acked Read 6 Sent Done Done false
15817 5550 Acked Read 9 Sent Done Done false
15818 5551 Acked Write 7 Sent Done Done false
15819 5552 Acked Write 7 Sent Done Done false
15820 5553 Acked Write 3 Sent Done Done false
15821 5554 Acked Read 7 Sent Done Done false
15822 5555 Acked Read 9 Sent Done Done false
15823 5556 Acked Read 2 Sent Done Done false
15824 5557 Acked Write 7 Sent Done Done false
15825 5558 Acked Flush 0 Sent Done Done false
15826 5559 Acked Write 2 Sent Done Done false
15827 5560 Acked Read 8 Sent Done Done false
15828 5561 Acked Write 8 Sent Done Done false
15829 5562 Acked Read 8 Sent Done Done false
15830 5563 Acked Flush 0 Sent Done Done false
15831 5564 Acked Write 9 Sent Done Done false
15832 5565 Acked Read 6 Sent Done Done false
15833 5566 Acked Read 5 Sent Done Done false
15834 5567 Acked Flush 0 Sent Done Done false
15835 5568 Acked Read 9 Sent Done Done false
15836 5569 Acked Flush 0 Sent Done Done false
15837 5570 Acked Read 5 Sent Done Done false
15838 5571 Acked Read 1 Sent Done Done false
15839 5572 Acked Write 5 Sent Done Done false
15840 5573 Acked Write 8 Sent Done Done false
15841 5574 Acked Read 7 Sent Done Done false
15842 5575 Acked Read 10 Sent Done Done false
15843 5576 Acked Read 4 Sent Done Done false
15844 5577 Acked Write 6 Sent Done Done false
15845 5578 Acked Flush 0 Sent Done Done false
15846 5579 Acked Write 4 Sent Done Done false
15847 5580 Acked Write 2 Sent Done Done false
15848 5581 Acked Flush 0 Sent Done Done false
15849 5582 Acked Read 7 Sent Done Done false
15850 5583 Acked Write 7 Sent Done Done false
15851 5584 Acked Read 8 Sent Done Done false
15852 5585 Acked Flush 0 Sent Done Done false
15853 5586 Acked Read 10 Sent Done Done false
15854 5587 Acked Read 9 Sent Done Done false
15855 5588 Acked Flush 0 Sent Done Done false
15856 5589 Acked Write 1 Sent Done Done false
15857 5590 Acked Write 2 Sent Done Done false
15858 5591 Acked Write 3 Sent Done Done false
15859 5592 Acked Write 10 Sent Done Done false
15860 5593 Acked Write 9 Sent Done Done false
15861 5594 Acked Write 1 Sent Done Done false
15862 5595 Acked Write 1 Sent Done Done false
15863 5596 Acked Read 4 Sent Done Done false
15864 5597 Acked Write 7 Sent Done Done false
15865 5598 Acked Flush 0 Sent Done Done false
15866 5599 Acked Read 4 Sent Done Done false
15867 5600 Acked Write 8 Sent Done Done false
15868 5601 Acked Read 10 Sent Done Done false
15869 5602 Acked Write 5 Sent Done Done false
15870 5603 Acked Write 1 Sent Done Done false
15871 5604 Acked Write 3 Sent Done Done false
15872 5605 Acked Write 10 Sent Done Done false
15873 5606 Acked Read 7 Sent Done Done false
15874 5607 Acked Write 10 Sent Done Done false
15875 5608 Acked Write 8 Sent Done Done false
15876 5609 Acked Flush 0 Sent Done Done false
15877 5610 Acked Write 2 Sent Done Done false
15878 5611 Acked Read 9 Sent Done Done false
15879 5612 Acked Read 10 Sent Done Done false
15880 5613 Acked Read 3 Sent Done Done false
15881 5614 Acked Read 4 Sent Done Done false
15882 5615 Acked Write 2 Sent Done Done false
15883 5616 Acked Read 5 Sent Done Done false
15884 5617 Acked Read 2 Sent Done Done false
15885 5618 Acked Read 3 Sent Done Done false
15886 5619 Acked Write 10 Sent Done Done false
15887 5620 Acked Write 6 Sent Done Done false
15888 5621 Acked Write 6 Sent Done Done false
15889 5622 Acked Write 4 Sent Done Done false
15890 5623 Acked Write 4 Sent Done Done false
15891 5624 Acked Write 9 Sent Done Done false
15892 5625 Acked Read 9 Sent Done Done false
15893 5626 Acked Read 8 Sent Done Done false
15894 5627 Acked Read 5 Sent Done Done false
15895 5628 Acked Read 10 Sent Done Done false
15896 5629 Acked Read 1 Sent Done Done false
15897 5630 Acked Read 7 Sent Done Done false
15898 5631 Acked Read 5 Sent Done Done false
15899 5632 Acked Read 6 Sent Done Done false
15900 5633 Acked Read 2 Sent Done Done false
15901 5634 Acked Read 5 Sent Done Done false
15902 5635 Acked Read 5 Sent Done Done false
15903 5636 Acked Write 7 Sent Done Done false
15904 5637 Acked Write 1 Sent Done Done false
15905 5638 Acked Flush 0 Sent Done Done false
15906 5639 Acked Write 5 Sent Done Done false
15907 5640 Acked Read 3 Sent Done Done false
15908 5641 Acked Read 4 Sent Done Done false
15909 5642 Acked Write 5 Sent Done Done false
15910 5643 Acked Read 1 Sent Done Done false
15911 5644 Acked Write 9 Sent Done Done false
15912 5645 Acked Write 6 Sent Done Done false
15913 5646 Acked Flush 0 Sent Done Done false
15914 5647 Acked Write 10 Sent Done Done false
15915 5648 Acked Write 8 Sent Done Done false
15916 5649 Acked Read 2 Sent Done Done false
15917 5650 Acked Read 2 Sent Done Done false
15918 5651 Acked Write 3 Sent Done Done false
15919 5652 Acked Write 2 Sent Done Done false
15920 5653 Acked Write 9 Sent Done Done false
15921 5654 Acked Write 7 Sent Done Done false
15922 5655 Acked Write 2 Sent Done Done false
15923 5656 Acked Read 2 Sent Done Done false
15924 5657 Acked Write 10 Sent Done Done false
15925 5658 Acked Read 5 Sent Done Done false
15926 5659 Acked Write 6 Sent Done Done false
15927 5660 Acked Write 5 Sent Done Done false
15928 5661 Acked Write 1 Sent Done Done false
15929 5662 Acked Write 10 Sent Done Done false
15930 5663 Acked Write 2 Sent Done Done false
15931 5664 Acked Write 6 Sent Done Done false
15932 5665 Acked Write 7 Sent Done Done false
15933 5666 Acked Read 5 Sent Done Done false
15934 5667 Acked Flush 0 Sent Done Done false
15935 5668 Acked Read 9 Sent Done Done false
15936 5669 Acked Read 1 Sent Done Done false
15937 5670 Acked Read 7 Sent Done Done false
15938 5671 Acked Flush 0 Sent Done Done false
15939 5672 Acked Read 1 Sent Done Done false
15940 5673 Acked Flush 0 Sent Done Done false
15941 5674 Acked Read 9 Sent Done Done false
15942 5675 Acked Read 8 Sent Done Done false
15943 5676 Acked Read 10 Sent Done Done false
15944 5677 Acked Write 8 Sent Done Done false
15945 5678 Acked Read 4 Sent Done Done false
15946 5679 Acked Flush 0 Sent Done Done false
15947 5680 Acked Read 2 Sent Done Done false
15948 5681 Acked Read 4 Sent Done Done false
15949 5682 Acked Write 4 Sent Done Done false
15950 5683 Acked Read 4 Sent Done Done false
15951 5684 Acked Flush 0 Sent Done Done false
15952 5685 Acked Read 7 Sent Done Done false
15953 5686 Acked Read 3 Sent Done Done false
15954 5687 Acked Read 2 Sent Done Done false
15955 5688 Acked Read 4 Sent Done Done false
15956 5689 Acked Flush 0 Sent Done Done false
15957 5690 Acked Write 10 Sent Done Done false
15958 5691 Acked Write 2 Sent Done Done false
15959 5692 Acked Flush 0 Sent Done Done false
15960 5693 Acked Read 2 Sent Done Done false
15961 5694 Acked Read 1 Sent Done Done false
15962 5695 Acked Write 1 Sent Done Done false
15963 5696 Acked Read 3 Sent Done Done false
15964 5697 Acked Read 10 Sent Done Done false
15965 5698 Acked Write 5 Sent Done Done false
15966 5699 Acked Read 8 Sent Done Done false
15967 5700 Acked Read 7 Sent Done Done false
15968 5701 Acked Read 7 Sent Done Done false
15969 5702 Acked Write 3 Sent Done Done false
15970 5703 Acked Read 5 Sent Done Done false
15971 5704 Acked Write 7 Sent Done Done false
15972 5705 Acked Read 6 Sent Done Done false
15973 5706 Acked Flush 0 Sent Done Done false
15974 5707 Acked Write 8 Sent Done Done false
15975 5708 Acked Read 4 Sent Done Done false
15976 5709 Acked Read 10 Sent Done Done false
15977 5710 Acked Read 4 Sent Done Done false
15978 5711 Acked Read 5 Sent Done Done false
15979 5712 Acked Write 3 Sent Done Done false
15980 5713 Acked Read 5 Sent Done Done false
15981 5714 Acked Read 6 Sent Done Done false
15982 5715 Acked Read 1 Sent Done Done false
15983 5716 Acked Read 8 Sent Done Done false
15984 5717 Acked Read 9 Sent Done Done false
15985 5718 Acked Flush 0 Sent Done Done false
15986 5719 Acked Write 9 Sent Done Done false
15987 5720 Acked Write 3 Sent Done Done false
15988 5721 Acked Write 1 Sent Done Done false
15989 5722 Acked Write 4 Sent Done Done false
15990 5723 Acked Write 7 Sent Done Done false
15991 5724 Acked Read 5 Sent Done Done false
15992 5725 Acked Write 8 Sent Done Done false
15993 5726 Acked Write 2 Sent Done Done false
15994 5727 Acked Read 10 Sent Done Done false
15995 5728 Acked Flush 0 Sent Done Done false
15996 5729 Acked Write 10 Sent Done Done false
15997 5730 Acked Read 9 Sent Done Done false
15998 5731 Acked Write 7 Sent Done Done false
15999 5732 Acked Read 2 Sent Done Done false
16000 5733 Acked Read 4 Sent Done Done false
16001 5734 Acked Read 3 Sent Done Done false
16002 5735 Acked Read 2 Sent Done Done false
16003 5736 Acked Read 9 Sent Done Done false
16004 5737 Acked Read 10 Sent Done Done false
16005 5738 Acked Read 6 Sent Done Done false
16006 5739 Acked Read 5 Sent Done Done false
16007 5740 Acked Write 4 Sent Done Done false
16008 5741 Acked Write 4 Sent Done Done false
16009 5742 Acked Read 10 Sent Done Done false
16010 5743 Acked Write 3 Sent Done Done false
16011 5744 Acked Read 5 Sent Done Done false
16012 5745 Acked Write 5 Sent Done Done false
16013 5746 Acked Read 1 Sent Done Done false
16014 5747 Acked Write 5 Sent Done Done false
16015 5748 Acked Read 3 Sent Done Done false
16016 5749 Acked Write 2 Sent Done Done false
16017 5750 Acked Flush 0 Sent Done Done false
16018 5751 Acked Write 5 Sent Done Done false
16019 5752 Acked Flush 0 Sent Done Done false
16020 5753 Acked Read 2 Sent Done Done false
16021 5754 Acked Read 10 Sent Done Done false
16022 5755 Acked Write 9 Sent Done Done false
16023 5756 Acked Read 7 Sent Done Done false
16024 5757 Acked Write 5 Sent Done Done false
16025 5758 Acked Write 3 Sent Done Done false
16026 5759 Acked Flush 0 Sent Done Done false
16027 5760 Acked Read 3 Sent Done Done false
16028 5761 Acked Read 3 Sent Done Done false
16029 5762 Acked Write 7 Sent Done Done false
16030 5763 Acked Flush 0 Sent Done Done false
16031 5764 Acked Read 4 Sent Done Done false
16032 5765 Acked Write 7 Sent Done Done false
16033 5766 Acked Flush 0 Sent Done Done false
16034 5767 Acked Write 2 Sent Done Done false
16035 5768 Acked Write 6 Sent Done Done false
16036 5769 Acked Read 5 Sent Done Done false
16037 5770 Acked Flush 0 Sent Done Done false
16038 5771 Acked Write 5 Sent Done Done false
16039 5772 Acked Read 10 Sent Done Done false
16040 5773 Acked Read 3 Sent Done Done false
16041 5774 Acked Write 2 Sent Done Done false
16042 5775 Acked Read 2 Sent Done Done false
16043 5776 Acked Read 10 Sent Done Done false
16044 5777 Acked Read 7 Sent Done Done false
16045 5778 Acked Write 8 Sent Done Done false
16046 5779 Acked Write 5 Sent Done Done false
16047 5780 Acked Read 4 Sent Done Done false
16048 5781 Acked Flush 0 Sent Done Done false
16049 5782 Acked Write 10 Sent Done Done false
16050 5783 Acked Read 6 Sent Done Done false
16051 5784 Acked Read 7 Sent Done Done false
16052 5785 Acked Read 9 Sent Done Done false
16053 5786 Acked Write 7 Sent Done Done false
16054 5787 Acked Write 2 Sent Done Done false
16055 5788 Acked Write 9 Sent Done Done false
16056 5789 Acked Write 1 Sent Done Done false
16057 5790 Acked Write 2 Sent Done Done false
16058 5791 Acked Read 7 Sent Done Done false
16059 5792 Acked Write 6 Sent Done Done false
16060 5793 Acked Write 6 Sent Done Done false
16061 5794 Acked Flush 0 Sent Done Done false
16062 5795 Acked Write 5 Sent Done Done false
16063 5796 Acked Write 4 Sent Done Done false
16064 5797 Acked Read 6 Sent Done Done false
16065 5798 Acked Read 7 Sent Done Done false
16066 5799 Acked Write 10 Sent Done Done false
16067 5800 Acked Write 4 Sent Done Done false
16068 5801 Acked Flush 0 Sent Done Done false
16069 5802 Acked Flush 0 Sent Done Done false
16070 5803 Acked Read 1 Sent Done Done false
16071 5804 Acked Write 6 Sent Done Done false
16072 5805 Acked Read 7 Sent Done Done false
16073 5806 Acked Write 3 Sent Done Done false
16074 5807 Acked Write 1 Sent Done Done false
16075 5808 Acked Read 1 Sent Done Done false
16076 5809 Acked Read 7 Sent Done Done false
16077 5810 Acked Write 6 Sent Done Done false
16078 5811 Acked Write 5 Sent Done Done false
16079 5812 Acked Write 3 Sent Done Done false
16080 5813 Acked Read 3 Sent Done Done false
16081 5814 Acked Write 6 Sent Done Done false
16082 5815 Acked Write 10 Sent Done Done false
16083 5816 Acked Read 8 Sent Done Done false
16084 5817 Acked Flush 0 Sent Done Done false
16085 5818 Acked Read 10 Sent Done Done false
16086 5819 Acked Flush 0 Sent Done Done false
16087 5820 Acked Read 1 Sent Done Done false
16088 5821 Acked Read 3 Sent Done Done false
16089 5822 Acked Write 2 Sent Done Done false
16090 5823 Acked Read 1 Sent Done Done false
16091 5824 Acked Write 3 Sent Done Done false
16092 5825 Acked Read 4 Sent Done Done false
16093 5826 Acked Read 5 Sent Done Done false
16094 5827 Acked Read 8 Sent Done Done false
16095 STATES DS:0 DS:1 DS:2 TOTAL
16096 Sent 301 0 0 301
16097 Done 0 301 301 602
16098 Skipped 0 0 0 0
16099 Error 0 0 0 0
16100 Last Flush: 5526 5819 5819
16101 Downstairs last five completed: 5526 5525 5524 5523 5522
16102 Upstairs last five completed: 5827 5826 5825 5824 5823
16103 CLIENT: Up:0 ds:301 act:2
16104 [8810] got start action
16105 [8810] I am going to restart
16106 Starting downstairs at port 8810
16107 Make output file at "/tmp/dsc/downstairs-8810.txt"
16108 [8810][0] reports Starting
16109 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
16110 [8810][0] reports Running
16111 ----------------------------------------------------------------
16112 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16113 States: Offline Active Active
16114 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16115 5527 Acked Write 6 Sent Done Done false
16116 5528 Acked Read 1 Sent Done Done false
16117 5529 Acked Read 5 Sent Done Done false
16118 5530 Acked Read 4 Sent Done Done false
16119 5531 Acked Write 2 Sent Done Done false
16120 5532 Acked Write 10 Sent Done Done false
16121 5533 Acked Read 10 Sent Done Done false
16122 5534 Acked Write 7 Sent Done Done false
16123 5535 Acked Read 6 Sent Done Done false
16124 5536 Acked Write 1 Sent Done Done false
16125 5537 Acked Read 4 Sent Done Done false
16126 5538 Acked Read 1 Sent Done Done false
16127 5539 Acked Write 1 Sent Done Done false
16128 5540 Acked Write 10 Sent Done Done false
16129 5541 Acked Read 10 Sent Done Done false
16130 5542 Acked Read 5 Sent Done Done false
16131 5543 Acked Write 7 Sent Done Done false
16132 5544 Acked Read 2 Sent Done Done false
16133 5545 Acked Read 8 Sent Done Done false
16134 5546 Acked Write 5 Sent Done Done false
16135 5547 Acked Write 1 Sent Done Done false
16136 5548 Acked Read 7 Sent Done Done false
16137 5549 Acked Read 6 Sent Done Done false
16138 5550 Acked Read 9 Sent Done Done false
16139 5551 Acked Write 7 Sent Done Done false
16140 5552 Acked Write 7 Sent Done Done false
16141 5553 Acked Write 3 Sent Done Done false
16142 5554 Acked Read 7 Sent Done Done false
16143 5555 Acked Read 9 Sent Done Done false
16144 5556 Acked Read 2 Sent Done Done false
16145 5557 Acked Write 7 Sent Done Done false
16146 5558 Acked Flush 0 Sent Done Done false
16147 5559 Acked Write 2 Sent Done Done false
16148 5560 Acked Read 8 Sent Done Done false
16149 5561 Acked Write 8 Sent Done Done false
16150 5562 Acked Read 8 Sent Done Done false
16151 5563 Acked Flush 0 Sent Done Done false
16152 5564 Acked Write 9 Sent Done Done false
16153 5565 Acked Read 6 Sent Done Done false
16154 5566 Acked Read 5 Sent Done Done false
16155 5567 Acked Flush 0 Sent Done Done false
16156 5568 Acked Read 9 Sent Done Done false
16157 5569 Acked Flush 0 Sent Done Done false
16158 5570 Acked Read 5 Sent Done Done false
16159 5571 Acked Read 1 Sent Done Done false
16160 5572 Acked Write 5 Sent Done Done false
16161 5573 Acked Write 8 Sent Done Done false
16162 5574 Acked Read 7 Sent Done Done false
16163 5575 Acked Read 10 Sent Done Done false
16164 5576 Acked Read 4 Sent Done Done false
16165 5577 Acked Write 6 Sent Done Done false
16166 5578 Acked Flush 0 Sent Done Done false
16167 5579 Acked Write 4 Sent Done Done false
16168 5580 Acked Write 2 Sent Done Done false
16169 5581 Acked Flush 0 Sent Done Done false
16170 5582 Acked Read 7 Sent Done Done false
16171 5583 Acked Write 7 Sent Done Done false
16172 5584 Acked Read 8 Sent Done Done false
16173 5585 Acked Flush 0 Sent Done Done false
16174 5586 Acked Read 10 Sent Done Done false
16175 5587 Acked Read 9 Sent Done Done false
16176 5588 Acked Flush 0 Sent Done Done false
16177 5589 Acked Write 1 Sent Done Done false
16178 5590 Acked Write 2 Sent Done Done false
16179 5591 Acked Write 3 Sent Done Done false
16180 5592 Acked Write 10 Sent Done Done false
16181 5593 Acked Write 9 Sent Done Done false
16182 5594 Acked Write 1 Sent Done Done false
16183 5595 Acked Write 1 Sent Done Done false
16184 5596 Acked Read 4 Sent Done Done false
16185 5597 Acked Write 7 Sent Done Done false
16186 5598 Acked Flush 0 Sent Done Done false
16187 5599 Acked Read 4 Sent Done Done false
16188 5600 Acked Write 8 Sent Done Done false
16189 5601 Acked Read 10 Sent Done Done false
16190 5602 Acked Write 5 Sent Done Done false
16191 5603 Acked Write 1 Sent Done Done false
16192 5604 Acked Write 3 Sent Done Done false
16193 5605 Acked Write 10 Sent Done Done false
16194 5606 Acked Read 7 Sent Done Done false
16195 5607 Acked Write 10 Sent Done Done false
16196 5608 Acked Write 8 Sent Done Done false
16197 5609 Acked Flush 0 Sent Done Done false
16198 5610 Acked Write 2 Sent Done Done false
16199 5611 Acked Read 9 Sent Done Done false
16200 5612 Acked Read 10 Sent Done Done false
16201 5613 Acked Read 3 Sent Done Done false
16202 5614 Acked Read 4 Sent Done Done false
16203 5615 Acked Write 2 Sent Done Done false
16204 5616 Acked Read 5 Sent Done Done false
16205 5617 Acked Read 2 Sent Done Done false
16206 5618 Acked Read 3 Sent Done Done false
16207 5619 Acked Write 10 Sent Done Done false
16208 5620 Acked Write 6 Sent Done Done false
16209 5621 Acked Write 6 Sent Done Done false
16210 5622 Acked Write 4 Sent Done Done false
16211 5623 Acked Write 4 Sent Done Done false
16212 5624 Acked Write 9 Sent Done Done false
16213 5625 Acked Read 9 Sent Done Done false
16214 5626 Acked Read 8 Sent Done Done false
16215 5627 Acked Read 5 Sent Done Done false
16216 5628 Acked Read 10 Sent Done Done false
16217 5629 Acked Read 1 Sent Done Done false
16218 5630 Acked Read 7 Sent Done Done false
16219 5631 Acked Read 5 Sent Done Done false
16220 5632 Acked Read 6 Sent Done Done false
16221 5633 Acked Read 2 Sent Done Done false
16222 5634 Acked Read 5 Sent Done Done false
16223 5635 Acked Read 5 Sent Done Done false
16224 5636 Acked Write 7 Sent Done Done false
16225 5637 Acked Write 1 Sent Done Done false
16226 5638 Acked Flush 0 Sent Done Done false
16227 5639 Acked Write 5 Sent Done Done false
16228 5640 Acked Read 3 Sent Done Done false
16229 5641 Acked Read 4 Sent Done Done false
16230 5642 Acked Write 5 Sent Done Done false
16231 5643 Acked Read 1 Sent Done Done false
16232 5644 Acked Write 9 Sent Done Done false
16233 5645 Acked Write 6 Sent Done Done false
16234 5646 Acked Flush 0 Sent Done Done false
16235 5647 Acked Write 10 Sent Done Done false
16236 5648 Acked Write 8 Sent Done Done false
16237 5649 Acked Read 2 Sent Done Done false
16238 5650 Acked Read 2 Sent Done Done false
16239 5651 Acked Write 3 Sent Done Done false
16240 5652 Acked Write 2 Sent Done Done false
16241 5653 Acked Write 9 Sent Done Done false
16242 5654 Acked Write 7 Sent Done Done false
16243 5655 Acked Write 2 Sent Done Done false
16244 5656 Acked Read 2 Sent Done Done false
16245 5657 Acked Write 10 Sent Done Done false
16246 5658 Acked Read 5 Sent Done Done false
16247 5659 Acked Write 6 Sent Done Done false
16248 5660 Acked Write 5 Sent Done Done false
16249 5661 Acked Write 1 Sent Done Done false
16250 5662 Acked Write 10 Sent Done Done false
16251 5663 Acked Write 2 Sent Done Done false
16252 5664 Acked Write 6 Sent Done Done false
16253 5665 Acked Write 7 Sent Done Done false
16254 5666 Acked Read 5 Sent Done Done false
16255 5667 Acked Flush 0 Sent Done Done false
16256 5668 Acked Read 9 Sent Done Done false
16257 5669 Acked Read 1 Sent Done Done false
16258 5670 Acked Read 7 Sent Done Done false
16259 5671 Acked Flush 0 Sent Done Done false
16260 5672 Acked Read 1 Sent Done Done false
16261 5673 Acked Flush 0 Sent Done Done false
16262 5674 Acked Read 9 Sent Done Done false
16263 5675 Acked Read 8 Sent Done Done false
16264 5676 Acked Read 10 Sent Done Done false
16265 5677 Acked Write 8 Sent Done Done false
16266 5678 Acked Read 4 Sent Done Done false
16267 5679 Acked Flush 0 Sent Done Done false
16268 5680 Acked Read 2 Sent Done Done false
16269 5681 Acked Read 4 Sent Done Done false
16270 5682 Acked Write 4 Sent Done Done false
16271 5683 Acked Read 4 Sent Done Done false
16272 5684 Acked Flush 0 Sent Done Done false
16273 5685 Acked Read 7 Sent Done Done false
16274 5686 Acked Read 3 Sent Done Done false
16275 5687 Acked Read 2 Sent Done Done false
16276 5688 Acked Read 4 Sent Done Done false
16277 5689 Acked Flush 0 Sent Done Done false
16278 5690 Acked Write 10 Sent Done Done false
16279 5691 Acked Write 2 Sent Done Done false
16280 5692 Acked Flush 0 Sent Done Done false
16281 5693 Acked Read 2 Sent Done Done false
16282 5694 Acked Read 1 Sent Done Done false
16283 5695 Acked Write 1 Sent Done Done false
16284 5696 Acked Read 3 Sent Done Done false
16285 5697 Acked Read 10 Sent Done Done false
16286 5698 Acked Write 5 Sent Done Done false
16287 5699 Acked Read 8 Sent Done Done false
16288 5700 Acked Read 7 Sent Done Done false
16289 5701 Acked Read 7 Sent Done Done false
16290 5702 Acked Write 3 Sent Done Done false
16291 5703 Acked Read 5 Sent Done Done false
16292 5704 Acked Write 7 Sent Done Done false
16293 5705 Acked Read 6 Sent Done Done false
16294 5706 Acked Flush 0 Sent Done Done false
16295 5707 Acked Write 8 Sent Done Done false
16296 5708 Acked Read 4 Sent Done Done false
16297 5709 Acked Read 10 Sent Done Done false
16298 5710 Acked Read 4 Sent Done Done false
16299 5711 Acked Read 5 Sent Done Done false
16300 5712 Acked Write 3 Sent Done Done false
16301 5713 Acked Read 5 Sent Done Done false
16302 5714 Acked Read 6 Sent Done Done false
16303 5715 Acked Read 1 Sent Done Done false
16304 5716 Acked Read 8 Sent Done Done false
16305 5717 Acked Read 9 Sent Done Done false
16306 5718 Acked Flush 0 Sent Done Done false
16307 5719 Acked Write 9 Sent Done Done false
16308 5720 Acked Write 3 Sent Done Done false
16309 5721 Acked Write 1 Sent Done Done false
16310 5722 Acked Write 4 Sent Done Done false
16311 5723 Acked Write 7 Sent Done Done false
16312 5724 Acked Read 5 Sent Done Done false
16313 5725 Acked Write 8 Sent Done Done false
16314 5726 Acked Write 2 Sent Done Done false
16315 5727 Acked Read 10 Sent Done Done false
16316 5728 Acked Flush 0 Sent Done Done false
16317 5729 Acked Write 10 Sent Done Done false
16318 5730 Acked Read 9 Sent Done Done false
16319 5731 Acked Write 7 Sent Done Done false
16320 5732 Acked Read 2 Sent Done Done false
16321 5733 Acked Read 4 Sent Done Done false
16322 5734 Acked Read 3 Sent Done Done false
16323 5735 Acked Read 2 Sent Done Done false
16324 5736 Acked Read 9 Sent Done Done false
16325 5737 Acked Read 10 Sent Done Done false
16326 5738 Acked Read 6 Sent Done Done false
16327 5739 Acked Read 5 Sent Done Done false
16328 5740 Acked Write 4 Sent Done Done false
16329 5741 Acked Write 4 Sent Done Done false
16330 5742 Acked Read 10 Sent Done Done false
16331 5743 Acked Write 3 Sent Done Done false
16332 5744 Acked Read 5 Sent Done Done false
16333 5745 Acked Write 5 Sent Done Done false
16334 5746 Acked Read 1 Sent Done Done false
16335 5747 Acked Write 5 Sent Done Done false
16336 5748 Acked Read 3 Sent Done Done false
16337 5749 Acked Write 2 Sent Done Done false
16338 5750 Acked Flush 0 Sent Done Done false
16339 5751 Acked Write 5 Sent Done Done false
16340 5752 Acked Flush 0 Sent Done Done false
16341 5753 Acked Read 2 Sent Done Done false
16342 5754 Acked Read 10 Sent Done Done false
16343 5755 Acked Write 9 Sent Done Done false
16344 5756 Acked Read 7 Sent Done Done false
16345 5757 Acked Write 5 Sent Done Done false
16346 5758 Acked Write 3 Sent Done Done false
16347 5759 Acked Flush 0 Sent Done Done false
16348 5760 Acked Read 3 Sent Done Done false
16349 5761 Acked Read 3 Sent Done Done false
16350 5762 Acked Write 7 Sent Done Done false
16351 5763 Acked Flush 0 Sent Done Done false
16352 5764 Acked Read 4 Sent Done Done false
16353 5765 Acked Write 7 Sent Done Done false
16354 5766 Acked Flush 0 Sent Done Done false
16355 5767 Acked Write 2 Sent Done Done false
16356 5768 Acked Write 6 Sent Done Done false
16357 5769 Acked Read 5 Sent Done Done false
16358 5770 Acked Flush 0 Sent Done Done false
16359 5771 Acked Write 5 Sent Done Done false
16360 5772 Acked Read 10 Sent Done Done false
16361 5773 Acked Read 3 Sent Done Done false
16362 5774 Acked Write 2 Sent Done Done false
16363 5775 Acked Read 2 Sent Done Done false
16364 5776 Acked Read 10 Sent Done Done false
16365 5777 Acked Read 7 Sent Done Done false
16366 5778 Acked Write 8 Sent Done Done false
16367 5779 Acked Write 5 Sent Done Done false
16368 5780 Acked Read 4 Sent Done Done false
16369 5781 Acked Flush 0 Sent Done Done false
16370 5782 Acked Write 10 Sent Done Done false
16371 5783 Acked Read 6 Sent Done Done false
16372 5784 Acked Read 7 Sent Done Done false
16373 5785 Acked Read 9 Sent Done Done false
16374 5786 Acked Write 7 Sent Done Done false
16375 5787 Acked Write 2 Sent Done Done false
16376 5788 Acked Write 9 Sent Done Done false
16377 5789 Acked Write 1 Sent Done Done false
16378 5790 Acked Write 2 Sent Done Done false
16379 5791 Acked Read 7 Sent Done Done false
16380 5792 Acked Write 6 Sent Done Done false
16381 5793 Acked Write 6 Sent Done Done false
16382 5794 Acked Flush 0 Sent Done Done false
16383 5795 Acked Write 5 Sent Done Done false
16384 5796 Acked Write 4 Sent Done Done false
16385 5797 Acked Read 6 Sent Done Done false
16386 5798 Acked Read 7 Sent Done Done false
16387 5799 Acked Write 10 Sent Done Done false
16388 5800 Acked Write 4 Sent Done Done false
16389 5801 Acked Flush 0 Sent Done Done false
16390 5802 Acked Flush 0 Sent Done Done false
16391 5803 Acked Read 1 Sent Done Done false
16392 5804 Acked Write 6 Sent Done Done false
16393 5805 Acked Read 7 Sent Done Done false
16394 5806 Acked Write 3 Sent Done Done false
16395 5807 Acked Write 1 Sent Done Done false
16396 5808 Acked Read 1 Sent Done Done false
16397 5809 Acked Read 7 Sent Done Done false
16398 5810 Acked Write 6 Sent Done Done false
16399 5811 Acked Write 5 Sent Done Done false
16400 5812 Acked Write 3 Sent Done Done false
16401 5813 Acked Read 3 Sent Done Done false
16402 5814 Acked Write 6 Sent Done Done false
16403 5815 Acked Write 10 Sent Done Done false
16404 5816 Acked Read 8 Sent Done Done false
16405 5817 Acked Flush 0 Sent Done Done false
16406 5818 Acked Read 10 Sent Done Done false
16407 5819 Acked Flush 0 Sent Done Done false
16408 5820 Acked Read 1 Sent Done Done false
16409 5821 Acked Read 3 Sent Done Done false
16410 5822 Acked Write 2 Sent Done Done false
16411 5823 Acked Read 1 Sent Done Done false
16412 5824 Acked Write 3 Sent Done Done false
16413 5825 Acked Read 4 Sent Done Done false
16414 5826 Acked Read 5 Sent Done Done false
16415 5827 Acked Read 8 Sent Done Done false
16416 5828 Acked Flush 0 Sent Done Done false
16417 STATES DS:0 DS:1 DS:2 TOTAL
16418 Sent 302 0 0 302
16419 Done 0 302 302 604
16420 Skipped 0 0 0 0
16421 Error 0 0 0 0
16422 Last Flush: 5526 5828 5828
16423 Downstairs last five completed: 5526 5525 5524 5523 5522
16424 Upstairs last five completed: 5828 5827 5826 5825 5824
16425 CLIENT: Up:0 ds:302 act:2
164262024-12-04T13:34:07.818ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164272024-12-04T13:34:07.818ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52181) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164282024-12-04T13:34:07.819ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164292024-12-04T13:34:07.820ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164302024-12-04T13:34:07.820ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164312024-12-04T13:34:07.820ZINFOcrucible: send last flush ID to this DS: 5526 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164322024-12-04T13:34:07.820ZINFOcrucible: Replied this last flush ID: 5526 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164332024-12-04T13:34:07.820ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164342024-12-04T13:34:07.820ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164352024-12-04T13:34:07.820ZINFOcrucible: [0] client re-new 302 jobs since flush 5526 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164362024-12-04T13:34:07.820ZINFOcrucible: [0] Replayed 302 jobs since flush: 5526 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
16437 ----------------------------------------------------------------
16438 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16439 Downstairs last five completed: 5828 5827 5826 5825 5824
16440 Upstairs last five completed: 5828 5827 5826 5825 5824
16441 CLIENT: Up:0 ds:0 act:3
16442 Replay: All jobs finished
16443 Main task has work to do, go find it
16444 got dsc Stop(2)
16445 stop 2
16446 [8830] Got stop action so:false kr:false
16447 Dec 04 13:34:10.052 INFO request completed, latency_us: 309, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 05a664ef-1e67-424e-b1e7-58af47ce7c95, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
164482024-12-04T13:34:10.053ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164492024-12-04T13:34:10.053ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164502024-12-04T13:34:10.053ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164512024-12-04T13:34:10.053ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
164522024-12-04T13:34:10.053ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
16453 Dec 04 13:34:10.053 INFO request completed, latency_us: 127, response_code: 200, uri: /state/cid/2, method: GET, req_id: 281f80db-4dda-4f64-86ec-ad5255e6325e, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
164542024-12-04T13:34:10.053ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
16455 [8830] Exited with: ExitStatus(unix_wait_status(9))
16456 [8830][2] reports Exit
16457 Dec 04 13:34:14.055 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/2, method: GET, req_id: 3d1b5bff-7da0-46cd-a9b2-f1886ca2def2, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
16458 001/300 Read block 4015 len 24576
16459 002/300 Flush
16460 003/300 Read block 4921 len 8192
16461 004/300 Write block 130 len 16384 data: 4 4 4 3
16462 005/300 Write block 1845 len 32768 data: 5 5 5 3 4 4 4 6
16463 006/300 Read block 1499 len 4096
16464 007/300 Read block 3987 len 40960
16465 008/300 Write block 4462 len 16384 data: 5 5 5 6
16466 009/300 Read block 2009 len 8192
16467 010/300 Write block 23 len 36864 data: 3 4 3 4 4 3 3 2 2
16468 011/300 Read block 4755 len 12288
16469 012/300 Write block 1545 len 40960 data: 4 4 3 3 2 3 4 4 4 3
16470 013/300 Read block 2373 len 16384
16471 014/300 Read block 3352 len 12288
16472 015/300 Read block 104 len 40960
16473 016/300 Write block 1829 len 4096 data: 3
16474 017/300 Write block 3135 len 40960 data: 4 3 3 4 3 5 5 5 4 3
16475 018/300 Read block 2790 len 32768
16476 019/300 Read block 2245 len 16384
16477 020/300 Read block 3521 len 24576
16478 021/300 Read block 2534 len 4096
16479 022/300 Read block 2696 len 36864
16480 023/300 Read block 864 len 8192
16481 024/300 Read block 4186 len 20480
16482 025/300 Write block 4231 len 24576 data: 4 3 3 3 4 4
16483 026/300 Flush
16484 027/300 Read block 3160 len 28672
16485 028/300 Write block 2085 len 28672 data: 4 3 2 3 4 3 3
16486 029/300 Flush
16487 030/300 Write block 1978 len 36864 data: 3 3 3 3 3 3 4 3 3
16488 031/300 Read block 140 len 40960
16489 032/300 Read block 4644 len 20480
16490 033/300 Read block 4661 len 12288
16491 034/300 Read block 557 len 20480
16492 035/300 Write block 1159 len 36864 data: 3 5 5 5 6 5 4 5 4
16493 036/300 Read block 4013 len 36864
16494 037/300 Read block 3158 len 36864
16495 038/300 Write block 189 len 4096 data: 5
16496 039/300 Write block 789 len 20480 data: 3 3 3 2 2
16497 040/300 Flush
16498 041/300 Write block 2201 len 28672 data: 3 3 3 3 2 2 2
16499 042/300 Write block 31 len 32768 data: 3 3 3 3 3 3 3 3
16500 043/300 Read block 3980 len 20480
16501 044/300 Write block 1239 len 36864 data: 3 3 3 4 4 5 5 4 4
16502 045/300 Write block 3002 len 32768 data: 8 7 7 9 7 5 5 4
16503 046/300 Write block 2273 len 40960 data: 3 3 3 5 6 6 6 4 3 3
16504 047/300 Write block 4400 len 8192 data: 6 4
16505 048/300 Read block 1986 len 28672
16506 049/300 Flush
16507 050/300 Write block 4395 len 36864 data: 5 5 5 6 6 7 5 4 4
16508 051/300 Write block 4974 len 32768 data: 3 3 3 4 4 5 4 4
16509 052/300 Write block 428 len 16384 data: 3 2 2 2
16510 053/300 Write block 3661 len 16384 data: 6 8 9 8
16511 054/300 Read block 1893 len 36864
16512 055/300 Read block 4309 len 28672
16513 056/300 Write block 4766 len 32768 data: 4 4 4 3 3 3 4 3
16514 057/300 Read block 2909 len 32768
16515 058/300 Read block 1599 len 32768
16516 059/300 Read block 169 len 40960
16517 060/300 Read block 1655 len 28672
16518 061/300 Read block 202 len 12288
16519 062/300 Read block 4606 len 12288
16520 063/300 Read block 514 len 40960
16521 064/300 Write block 1210 len 36864 data: 5 4 4 4 4 5 5 4 4
16522 065/300 Read block 258 len 32768
16523 066/300 Read block 3392 len 8192
16524 067/300 Write block 1120 len 32768 data: 3 4 3 3 3 2 2 2
16525 068/300 Write block 398 len 40960 data: 4 3 3 3 3 4 5 5 4 3
16526 069/300 Flush
16527 070/300 Read block 510 len 32768
16528 071/300 Write block 2348 len 28672 data: 8 6 6 5 7 6 4
16529 072/300 Write block 3644 len 8192 data: 2 3
16530 073/300 Write block 3683 len 32768 data: 4 3 3 3 4 5 6 5
16531 074/300 Write block 817 len 32768 data: 3 3 2 3 4 4 4 4
16532 075/300 Write block 1849 len 40960 data: 5 5 5 7 5 3 3 3 3 3
16533 076/300 Read block 4013 len 40960
16534 077/300 Read block 1366 len 12288
16535 078/300 Read block 3286 len 16384
16536 079/300 Flush
16537 080/300 Read block 996 len 8192
16538 081/300 Read block 2785 len 32768
16539 082/300 Read block 2941 len 12288
16540 083/300 Read block 357 len 28672
16541 084/300 Read block 648 len 24576
16542 085/300 Write block 12 len 24576 data: 6 5 5 5 7 6
16543 086/300 Write block 4236 len 12288 data: 5 5 5
16544 087/300 Read block 3532 len 4096
16545 088/300 Read block 411 len 16384
16546 089/300 Read block 4309 len 28672
16547 090/300 Write block 1323 len 20480 data: 4 4 4 4 4
16548 091/300 Read block 1852 len 24576
16549 092/300 Read block 4405 len 28672
16550 093/300 Write block 3940 len 16384 data: 6 7 6 7
16551 094/300 Write block 943 len 40960 data: 5 4 4 4 5 6 5 5 5 4
16552 095/300 Read block 2208 len 12288
16553 096/300 Write block 417 len 40960 data: 4 4 4 4 4 3 3 3 3 3
16554 097/300 Read block 2920 len 20480
16555 098/300 Write block 4811 len 4096 data: 2
16556 099/300 Write block 3687 len 12288 data: 5 6 7
16557 100/300 Write block 2207 len 12288 data: 3 3 2
16558 101/300 Read block 3683 len 40960
16559 102/300 Read block 2488 len 32768
16560 103/300 Read block 3102 len 12288
16561 104/300 Write block 3772 len 20480 data: 3 4 4 5 5
16562 105/300 Read block 2441 len 36864
16563 106/300 Read block 4177 len 4096
16564 107/300 Write block 3785 len 24576 data: 2 2 3 4 4 3
16565 108/300 Write block 3873 len 28672 data: 4 3 3 3 5 5 5
16566 109/300 Write block 2678 len 36864 data: 3 3 3 3 3 3 3 3 3
16567 110/300 Write block 1111 len 16384 data: 3 5 5 7
16568 111/300 Read block 2303 len 40960
16569 112/300 Read block 2279 len 20480
16570 113/300 Flush
16571 114/300 Write block 2958 len 32768 data: 2 2 2 2 2 2 3 3
16572 115/300 Read block 1512 len 20480
16573 116/300 Read block 2826 len 32768
16574 117/300 Write block 1203 len 28672 data: 5 5 5 5 5 5 6
16575 118/300 Read block 1084 len 12288
16576 119/300 Write block 4434 len 16384 data: 3 4 4 4
16577 120/300 Read block 3185 len 8192
16578 121/300 Write block 3673 len 12288 data: 7 5 5
16579 122/300 Flush
16580 123/300 Flush
16581 124/300 Read block 3209 len 36864
16582 125/300 Flush
16583 126/300 Read block 747 len 16384
16584 127/300 Flush
16585 128/300 Read block 4635 len 40960
16586 129/300 Read block 4359 len 20480
16587 130/300 Write block 2320 len 40960 data: 6 6 6 5 5 4 4 4 3 2
16588 131/300 Write block 1355 len 4096 data: 2
16589 132/300 Write block 4459 len 40960 data: 6 3 4 6 6 6 7 6 6 7
16590 133/300 Write block 2897 len 28672 data: 4 4 3 3 4 3 3
16591 134/300 Read block 2593 len 40960
16592 135/300 Read block 4670 len 40960
16593 136/300 Write block 3692 len 24576 data: 6 6 4 4 4 4
16594 137/300 Read block 4397 len 4096
16595 138/300 Read block 4711 len 20480
16596 139/300 Flush
16597 140/300 Read block 1275 len 16384
16598 141/300 Read block 3671 len 36864
16599 142/300 Read block 1923 len 16384
16600 143/300 Read block 775 len 24576
16601 144/300 Write block 3802 len 32768 data: 4 3 3 4 3 2 2 4
16602 145/300 Write block 1165 len 12288 data: 5 6 5
16603 146/300 Read block 1612 len 20480
16604 147/300 Read block 1830 len 16384
16605 148/300 Write block 4784 len 12288 data: 3 3 3
16606 149/300 Write block 4027 len 36864 data: 6 5 5 5 5 5 4 4 4
16607 150/300 Write block 824 len 40960 data: 5 2 3 3 3 3 3 4 4 6
16608 151/300 Write block 3046 len 28672 data: 2 2 2 2 4 6 5
16609 152/300 Read block 3502 len 16384
16610 153/300 Read block 2332 len 8192
16611 154/300 Write block 1498 len 32768 data: 6 6 4 5 6 7 6 7
16612 155/300 Read block 3469 len 12288
16613 156/300 Read block 2616 len 24576
16614 157/300 Read block 3320 len 28672
16615 158/300 Read block 4859 len 8192
16616 159/300 Flush
16617 160/300 Read block 3332 len 12288
16618 161/300 Write block 16 len 12288 data: 8 7 4
16619 162/300 Read block 983 len 28672
16620 163/300 Write block 2363 len 36864 data: 2 3 3 3 3 5 5 6 6
16621 164/300 Write block 681 len 4096 data: 3
16622 165/300 Write block 4334 len 24576 data: 7 6 6 5 5 7
16623 166/300 Read block 4434 len 12288
16624 167/300 Read block 2809 len 32768
16625 168/300 Flush
16626 169/300 Read block 4488 len 40960
16627 170/300 Write block 4614 len 24576 data: 3 3 4 4 5 5
16628 171/300 Write block 4573 len 4096 data: 5
16629 172/300 Write block 2174 len 24576 data: 5 4 3 3 5 5
16630 173/300 Read block 61 len 20480
16631 174/300 Read block 3129 len 36864
16632 175/300 Write block 2838 len 32768 data: 5 5 5 4 4 4 5 4
16633 176/300 Read block 4685 len 16384
16634 177/300 Read block 2444 len 36864
16635 178/300 Read block 1262 len 40960
16636 179/300 Read block 4516 len 20480
16637 180/300 Read block 2698 len 40960
16638 181/300 Write block 4316 len 12288 data: 3 2 2
16639 182/300 Write block 2275 len 16384 data: 4 6 7 7
16640 183/300 Write block 250 len 20480 data: 6 6 7 6 6
16641 184/300 Flush
16642 185/300 Read block 1052 len 40960
16643 186/300 Flush
16644 187/300 Write block 791 len 8192 data: 4 3
16645 188/300 Read block 4354 len 32768
16646 189/300 Read block 3072 len 36864
16647 190/300 Read block 1346 len 36864
16648 191/300 Write block 3439 len 32768 data: 7 5 4 4 5 5 4 4
16649 192/300 Read block 1689 len 16384
16650 193/300 Read block 1844 len 8192
16651 194/300 Read block 3784 len 16384
16652 195/300 Write block 4752 len 20480 data: 4 4 5 6 6
16653 196/300 Read block 3210 len 16384
16654 197/300 Read block 4082 len 8192
16655 198/300 Read block 753 len 20480
16656 199/300 Flush
16657 200/300 Read block 3806 len 4096
16658 201/300 Write block 4490 len 16384 data: 6 6 5 3
16659 202/300 Read block 2705 len 12288
16660 203/300 Write block 1593 len 16384 data: 4 5 6 6
16661 204/300 Write block 3328 len 32768 data: 4 4 3 4 3 3 3 3
16662 205/300 Read block 1702 len 16384
16663 206/300 Read block 392 len 28672
16664 207/300 Read block 2621 len 36864
16665 208/300 Write block 2171 len 32768 data: 6 7 6 6 5 4 4 6
16666 209/300 Flush
16667 210/300 Write block 3614 len 12288 data: 4 4 3
16668 211/300 Read block 433 len 4096
16669 212/300 Write block 4659 len 36864 data: 3 4 4 4 3 4 3 2 2
16670 213/300 Write block 2159 len 4096 data: 2
16671 214/300 Read block 2076 len 32768
16672 215/300 Write block 813 len 20480 data: 5 5 3 2 4
16673 216/300 Read block 481 len 20480
16674 217/300 Write block 2254 len 12288 data: 3 2 3
16675 218/300 Write block 1938 len 4096 data: 4
16676 219/300 Read block 942 len 24576
16677 220/300 Write block 3993 len 24576 data: 3 2 3 2 2 2
16678 221/300 Write block 2024 len 32768 data: 3 4 5 5 5 4 3 4
16679 222/300 Read block 3320 len 16384
16680 223/300 Read block 2305 len 40960
16681 224/300 Read block 4145 len 40960
16682 225/300 Flush
16683 226/300 Write block 4376 len 12288 data: 3 3 4
16684 227/300 Write block 4175 len 32768 data: 5 5 4 4 4 4 4 4
16685 228/300 Write block 1932 len 28672 data: 3 3 3 3 3 3 5
16686 229/300 Read block 4509 len 4096
16687 230/300 Flush
16688 231/300 Read block 3925 len 20480
16689 232/300 Read block 3349 len 12288
16690 233/300 Write block 3215 len 8192 data: 7 8
16691 234/300 Read block 450 len 12288
16692 235/300 Read block 4597 len 36864
16693 236/300 Write block 208 len 4096 data: 5
16694 237/300 Read block 4173 len 36864
16695 238/300 Write block 4693 len 20480 data: 4 4 4 5 5
16696 239/300 Read block 1248 len 36864
16697 240/300 Write block 2274 len 24576 data: 4 5 7 8 8 7
16698 241/300 Write block 3729 len 16384 data: 2 2 2 3
16699 242/300 Write block 1711 len 8192 data: 3 3
16700 243/300 Write block 1892 len 32768 data: 5 5 5 5 6 4 5 5
16701 244/300 Read block 4893 len 40960
16702 245/300 Write block 4606 len 40960 data: 6 5 5 6 6 3 2 3 4 4
16703 246/300 Read block 1407 len 4096
16704 247/300 Write block 3793 len 12288 data: 3 3 3
16705 248/300 Read block 3284 len 20480
16706 249/300 Read block 427 len 12288
16707 250/300 Write block 95 len 8192 data: 3 2
16708 251/300 Write block 2632 len 36864 data: 5 5 5 4 5 5 5 5 5
16709 252/300 Write block 4327 len 28672 data: 5 4 6 7 9 9 7
16710 253/300 Write block 2757 len 16384 data: 3 3 3 5
16711 254/300 Read block 3218 len 4096
16712 255/300 Read block 2933 len 8192
16713 256/300 Read block 338 len 20480
16714 257/300 Read block 133 len 32768
16715 258/300 Write block 2243 len 20480 data: 4 4 5 5 8
16716 259/300 Read block 741 len 36864
16717 260/300 Read block 2293 len 4096
16718 261/300 Read block 4147 len 4096
16719 262/300 Read block 4083 len 40960
16720 263/300 Read block 4986 len 20480
16721 264/300 Write block 2081 len 4096 data: 4
16722 265/300 Write block 2347 len 40960 data: 7 9 7 7 6 8 7 5 4 3
16723 266/300 Write block 10 len 32768 data: 4 5 7 6 6 6 9 8
16724 267/300 Write block 389 len 8192 data: 4 5
16725 268/300 Write block 627 len 36864 data: 4 3 4 4 3 2 3 3 4
16726 269/300 Read block 4772 len 12288
16727 270/300 Read block 416 len 40960
16728 271/300 Write block 3224 len 16384 data: 4 4 5 3
16729 272/300 Read block 1643 len 4096
16730 273/300 Flush
16731 274/300 Write block 1827 len 24576 data: 3 3 4 3 4 4
16732 275/300 Read block 3112 len 20480
16733 276/300 Flush
16734 277/300 Write block 1298 len 32768 data: 6 4 4 4 3 2 2 3
16735 278/300 Read block 2008 len 8192
16736 279/300 Write block 669 len 32768 data: 4 5 3 3 4 4 4 3
16737 280/300 Read block 1654 len 12288
16738 281/300 Write block 1329 len 36864 data: 3 3 3 4 4 4 4 4 5
16739 282/300 Read block 4408 len 4096
16740 283/300 Read block 1605 len 32768
16741 284/300 Write block 2820 len 16384 data: 2 3 3 3
16742 285/300 Write block 342 len 36864 data: 2 3 3 2 2 3 3 3 3
16743 286/300 Read block 2836 len 20480
16744 287/300 Write block 3501 len 20480 data: 5 5 5 3 2
16745 288/300 Read block 1746 len 28672
16746 289/300 Write block 344 len 12288 data: 4 3 3
16747 290/300 Read block 4092 len 36864
16748 291/300 Write block 3400 len 40960 data: 9 9 10 10 9 8 7 6 4 4
16749 292/300 Read block 4547 len 20480
16750 293/300 Read block 2957 len 36864
16751 294/300 Write block 2817 len 32768 data: 2 2 2 3 4 4 4 3
16752 295/300 Write block 2173 len 32768 data: 7 7 6 5 5 7 6 6
16753 296/300 Write block 3969 len 28672 data: 3 3 3 3 3 5 5
16754 297/300 Read block 1818 len 28672
16755 298/300 Flush
16756 299/300 Read block 541 len 20480
16757 300/300 Write block 2952 len 20480 data: 4 4 3 2 2
16758 301/300 Read block 1752 len 24576
16759 Main task has work to do, go find it
16760 got dsc Start(2)
16761 start 2
16762 Dec 04 13:34:14.298 INFO request completed, latency_us: 193, response_code: 204, uri: /start/cid/2, method: POST, req_id: 4265c532-5c46-4634-9927-fec2e4c136ba, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
16763 Replay: started 2, returned:Ok(())
16764 ----------------------------------------------------------------
16765 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16766 States: Active Active Offline
16767 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16768 5829 Acked Read 6 Done Done Sent false
16769 5830 Acked Flush 0 Done Done Sent false
16770 5831 Acked Read 2 Done Done Sent false
16771 5832 Acked Write 4 Done Done Sent false
16772 5833 Acked Write 8 Done Done Sent false
16773 5834 Acked Read 1 Done Done Sent false
16774 5835 Acked Read 10 Done Done Sent false
16775 5836 Acked Write 4 Done Done Sent false
16776 5837 Acked Read 2 Done Done Sent false
16777 5838 Acked Write 9 Done Done Sent false
16778 5839 Acked Read 3 Done Done Sent false
16779 5840 Acked Write 10 Done Done Sent false
16780 5841 Acked Read 4 Done Done Sent false
16781 5842 Acked Read 3 Done Done Sent false
16782 5843 Acked Read 10 Done Done Sent false
16783 5844 Acked Write 1 Done Done Sent false
16784 5845 Acked Write 10 Done Done Sent false
16785 5846 Acked Read 8 Done Done Sent false
16786 5847 Acked Read 4 Done Done Sent false
16787 5848 Acked Read 6 Done Done Sent false
16788 5849 Acked Read 1 Done Done Sent false
16789 5850 Acked Read 9 Done Done Sent false
16790 5851 Acked Read 2 Done Done Sent false
16791 5852 Acked Read 5 Done Done Sent false
16792 5853 Acked Write 6 Done Done Sent false
16793 5854 Acked Flush 0 Done Done Sent false
16794 5855 Acked Read 7 Done Done Sent false
16795 5856 Acked Write 7 Done Done Sent false
16796 5857 Acked Flush 0 Done Done Sent false
16797 5858 Acked Write 9 Done Done Sent false
16798 5859 Acked Read 10 Done Done Sent false
16799 5860 Acked Read 5 Done Done Sent false
16800 5861 Acked Read 3 Done Done Sent false
16801 5862 Acked Read 5 Done Done Sent false
16802 5863 Acked Write 9 Done Done Sent false
16803 5864 Acked Read 9 Done Done Sent false
16804 5865 Acked Read 9 Done Done Sent false
16805 5866 Acked Write 1 Done Done Sent false
16806 5867 Acked Write 5 Done Done Sent false
16807 5868 Acked Flush 0 Done Done Sent false
16808 5869 Acked Write 7 Done Done Sent false
16809 5870 Acked Write 8 Done Done Sent false
16810 5871 Acked Read 5 Done Done Sent false
16811 5872 Acked Write 9 Done Done Sent false
16812 5873 Acked Write 8 Done Done Sent false
16813 5874 Acked Write 10 Done Done Sent false
16814 5875 Acked Write 2 Done Done Sent false
16815 5876 Acked Read 7 Done Done Sent false
16816 5877 Acked Flush 0 Done Done Sent false
16817 5878 Acked Write 9 Done Done Sent false
16818 5879 Acked Write 8 Done Done Sent false
16819 5880 Acked Write 4 Done Done Sent false
16820 5881 Acked Write 4 Done Done Sent false
16821 5882 Acked Read 9 Done Done Sent false
16822 5883 Acked Read 7 Done Done Sent false
16823 5884 Acked Write 8 Done Done Sent false
16824 5885 Acked Read 8 Done Done Sent false
16825 5886 Acked Read 8 Done Done Sent false
16826 5887 Acked Read 10 Done Done Sent false
16827 5888 Acked Read 7 Done Done Sent false
16828 5889 Acked Read 3 Done Done Sent false
16829 5890 Acked Read 3 Done Done Sent false
16830 5891 Acked Read 10 Done Done Sent false
16831 5892 Acked Write 9 Done Done Sent false
16832 5893 Acked Read 8 Done Done Sent false
16833 5894 Acked Read 2 Done Done Sent false
16834 5895 Acked Write 8 Done Done Sent false
16835 5896 Acked Write 10 Done Done Sent false
16836 5897 Acked Flush 0 Done Done Sent false
16837 5898 Acked Read 8 Done Done Sent false
16838 5899 Acked Write 7 Done Done Sent false
16839 5900 Acked Write 2 Done Done Sent false
16840 5901 Acked Write 8 Done Done Sent false
16841 5902 Acked Write 8 Done Done Sent false
16842 5903 Acked Write 10 Done Done Sent false
16843 5904 Acked Read 10 Done Done Sent false
16844 5905 Acked Read 3 Done Done Sent false
16845 5906 Acked Read 4 Done Done Sent false
16846 5907 Acked Flush 0 Done Done Sent false
16847 5908 Acked Read 2 Done Done Sent false
16848 5909 Acked Read 8 Done Done Sent false
16849 5910 Acked Read 3 Done Done Sent false
16850 5911 Acked Read 7 Done Done Sent false
16851 5912 Acked Read 6 Done Done Sent false
16852 5913 Acked Write 6 Done Done Sent false
16853 5914 Acked Write 3 Done Done Sent false
16854 5915 Acked Read 1 Done Done Sent false
16855 5916 Acked Read 4 Done Done Sent false
16856 5917 Acked Read 7 Done Done Sent false
16857 5918 Acked Write 5 Done Done Sent false
16858 5919 Acked Read 6 Done Done Sent false
16859 5920 Acked Read 7 Done Done Sent false
16860 5921 Acked Write 4 Done Done Sent false
16861 5922 Acked Write 10 Done Done Sent false
16862 5923 Acked Read 3 Done Done Sent false
16863 5924 Acked Write 10 Done Done Sent false
16864 5925 Acked Read 5 Done Done Sent false
16865 5926 Acked Write 1 Done Done Sent false
16866 5927 Acked Write 3 Done Done Sent false
16867 5928 Acked Write 3 Done Done Sent false
16868 5929 Acked Read 10 Done Done Sent false
16869 5930 Acked Read 8 Done Done Sent false
16870 5931 Acked Read 3 Done Done Sent false
16871 5932 Acked Write 5 Done Done Sent false
16872 5933 Acked Read 9 Done Done Sent false
16873 5934 Acked Read 1 Done Done Sent false
16874 5935 Acked Write 6 Done Done Sent false
16875 5936 Acked Write 7 Done Done Sent false
16876 5937 Acked Write 9 Done Done Sent false
16877 5938 Acked Write 4 Done Done Sent false
16878 5939 Acked Read 10 Done Done Sent false
16879 5940 Acked Read 5 Done Done Sent false
16880 5941 Acked Flush 0 Done Done Sent false
16881 5942 Acked Write 8 Done Done Sent false
16882 5943 Acked Read 5 Done Done Sent false
16883 5944 Acked Read 8 Done Done Sent false
16884 5945 Acked Write 7 Done Done Sent false
16885 5946 Acked Read 3 Done Done Sent false
16886 5947 Acked Write 4 Done Done Sent false
16887 5948 Acked Read 2 Done Done Sent false
16888 5949 Acked Write 3 Done Done Sent false
16889 5950 Acked Flush 0 Done Done Sent false
16890 5951 Acked Flush 0 Done Done Sent false
16891 5952 Acked Read 9 Done Done Sent false
16892 5953 Acked Flush 0 Done Done Sent false
16893 5954 Acked Read 4 Done Done Sent false
16894 5955 Acked Flush 0 Done Done Sent false
16895 5956 Acked Read 10 Done Done Sent false
16896 5957 Acked Read 5 Done Done Sent false
16897 5958 Acked Write 10 Done Done Sent false
16898 5959 Acked Write 1 Done Done Sent false
16899 5960 Acked Write 10 Done Done Sent false
16900 5961 Acked Write 7 Done Done Sent false
16901 5962 Acked Read 10 Done Done Sent false
16902 5963 Acked Read 10 Done Done Sent false
16903 5964 Acked Write 6 Done Done Sent false
16904 5965 Acked Read 1 Done Done Sent false
16905 5966 Acked Read 5 Done Done Sent false
16906 5967 Acked Flush 0 Done Done Sent false
16907 5968 Acked Read 4 Done Done Sent false
16908 5969 Acked Read 9 Done Done Sent false
16909 5970 Acked Read 4 Done Done Sent false
16910 5971 Acked Read 6 Done Done Sent false
16911 5972 Acked Write 8 Done Done Sent false
16912 5973 Acked Write 3 Done Done Sent false
16913 5974 Acked Read 5 Done Done Sent false
16914 5975 Acked Read 4 Done Done Sent false
16915 5976 Acked Write 3 Done Done Sent false
16916 5977 Acked Write 9 Done Done Sent false
16917 5978 Acked Write 10 Done Done Sent false
16918 5979 Acked Write 7 Done Done Sent false
16919 5980 Acked Read 4 Done Done Sent false
16920 5981 Acked Read 2 Done Done Sent false
16921 5982 Acked Write 8 Done Done Sent false
16922 5983 Acked Read 3 Done Done Sent false
16923 5984 Acked Read 6 Done Done Sent false
16924 5985 Acked Read 7 Done Done Sent false
16925 5986 Acked Read 2 Done Done Sent false
16926 5987 Acked Flush 0 Done Done Sent false
16927 5988 Acked Read 3 Done Done Sent false
16928 5989 Acked Write 3 Done Done Sent false
16929 5990 Acked Read 7 Done Done Sent false
16930 5991 Acked Write 9 Done Done Sent false
16931 5992 Acked Write 1 Done Done Sent false
16932 5993 Acked Write 6 Done Done Sent false
16933 5994 Acked Read 3 Done Done Sent false
16934 5995 Acked Read 8 Done Done Sent false
16935 5996 Acked Flush 0 Done Done Sent false
16936 5997 Acked Read 10 Done Done Sent false
16937 5998 Acked Write 6 Done Done Sent false
16938 5999 Acked Write 1 Done Done Sent false
16939 6000 Acked Write 6 Done Done Sent false
16940 6001 Acked Read 5 Done Done Sent false
16941 6002 Acked Read 9 Done Done Sent false
16942 6003 Acked Write 8 Done Done Sent false
16943 6004 Acked Read 4 Done Done Sent false
16944 6005 Acked Read 9 Done Done Sent false
16945 6006 Acked Read 10 Done Done Sent false
16946 6007 Acked Read 5 Done Done Sent false
16947 6008 Acked Read 10 Done Done Sent false
16948 6009 Acked Write 3 Done Done Sent false
16949 6010 Acked Write 4 Done Done Sent false
16950 6011 Acked Write 5 Done Done Sent false
16951 6012 Acked Flush 0 Done Done Sent false
16952 6013 Acked Read 10 Done Done Sent false
16953 6014 Acked Flush 0 Done Done Sent false
16954 6015 Acked Write 2 Done Done Sent false
16955 6016 Acked Read 8 Done Done Sent false
16956 6017 Acked Read 9 Done Done Sent false
16957 6018 Acked Read 9 Done Done Sent false
16958 6019 Acked Write 8 Done Done Sent false
16959 6020 Acked Read 4 Done Done Sent false
16960 6021 Acked Read 2 Done Done Sent false
16961 6022 Acked Read 4 Done Done Sent false
16962 6023 Acked Write 5 Done Done Sent false
16963 6024 Acked Read 4 Done Done Sent false
16964 6025 Acked Read 2 Done Done Sent false
16965 6026 Acked Read 5 Done Done Sent false
16966 6027 Acked Flush 0 Done Done Sent false
16967 6028 Acked Read 1 Done Done Sent false
16968 6029 Acked Write 4 Done Done Sent false
16969 6030 Acked Read 3 Done Done Sent false
16970 6031 Acked Write 4 Done Done Sent false
16971 6032 Acked Write 8 Done Done Sent false
16972 6033 Acked Read 4 Done Done Sent false
16973 6034 Acked Read 7 Done Done Sent false
16974 6035 Acked Read 9 Done Done Sent false
16975 6036 Acked Write 8 Done Done Sent false
16976 6037 Acked Flush 0 Done Done Sent false
16977 6038 Acked Write 3 Done Done Sent false
16978 6039 Acked Read 1 Done Done Sent false
16979 6040 Acked Write 9 Done Done Sent false
16980 6041 Acked Write 1 Done Done Sent false
16981 6042 Acked Read 8 Done Done Sent false
16982 6043 Acked Write 5 Done Done Sent false
16983 6044 Acked Read 5 Done Done Sent false
16984 6045 Acked Write 3 Done Done Sent false
16985 6046 Acked Write 1 Done Done Sent false
16986 6047 Acked Read 6 Done Done Sent false
16987 6048 Acked Write 6 Done Done Sent false
16988 6049 Acked Write 8 Done Done Sent false
16989 6050 Acked Read 4 Done Done Sent false
16990 6051 Acked Read 10 Done Done Sent false
16991 6052 Acked Read 10 Done Done Sent false
16992 6053 Acked Flush 0 Done Done Sent false
16993 6054 Acked Write 3 Done Done Sent false
16994 6055 Acked Write 8 Done Done Sent false
16995 6056 Acked Write 7 Done Done Sent false
16996 6057 Acked Read 1 Done Done Sent false
16997 6058 Acked Flush 0 Done Done Sent false
16998 6059 Acked Read 5 Done Done Sent false
16999 6060 Acked Read 3 Done Done Sent false
17000 6061 Acked Write 2 Done Done Sent false
17001 6062 Acked Read 3 Done Done Sent false
17002 6063 Acked Read 9 Done Done Sent false
17003 6064 Acked Write 1 Done Done Sent false
17004 6065 Acked Read 9 Done Done Sent false
17005 6066 Acked Write 5 Done Done Sent false
17006 6067 Acked Read 9 Done Done Sent false
17007 6068 Acked Write 6 Done Done Sent false
17008 6069 Acked Write 4 Done Done Sent false
17009 6070 Acked Write 2 Done Done Sent false
17010 6071 Acked Write 8 Done Done Sent false
17011 6072 Acked Read 10 Done Done Sent false
17012 6073 Acked Write 10 Done Done Sent false
17013 6074 Acked Read 1 Done Done Sent false
17014 6075 Acked Write 3 Done Done Sent false
17015 6076 Acked Read 5 Done Done Sent false
17016 6077 Acked Read 3 Done Done Sent false
17017 6078 Acked Write 2 Done Done Sent false
17018 6079 Acked Write 9 Done Done Sent false
17019 6080 Acked Write 7 Done Done Sent false
17020 6081 Acked Write 4 Done Done Sent false
17021 6082 Acked Read 1 Done Done Sent false
17022 6083 Acked Read 2 Done Done Sent false
17023 6084 Acked Read 5 Done Done Sent false
17024 6085 Acked Read 8 Done Done Sent false
17025 6086 Acked Write 5 Done Done Sent false
17026 6087 Acked Read 9 Done Done Sent false
17027 6088 Acked Read 1 Done Done Sent false
17028 6089 Acked Read 1 Done Done Sent false
17029 6090 Acked Read 10 Done Done Sent false
17030 6091 Acked Read 5 Done Done Sent false
17031 6092 Acked Write 1 Done Done Sent false
17032 6093 Acked Write 10 Done Done Sent false
17033 6094 Acked Write 8 Done Done Sent false
17034 6095 Acked Write 2 Done Done Sent false
17035 6096 Acked Write 9 Done Done Sent false
17036 6097 Acked Read 3 Done Done Sent false
17037 6098 Acked Read 10 Done Done Sent false
17038 6099 Acked Write 4 Done Done Sent false
17039 6100 Acked Read 1 Done Done Sent false
17040 6101 Acked Flush 0 Done Done Sent false
17041 6102 Acked Write 6 Done Done Sent false
17042 6103 Acked Read 5 Done Done Sent false
17043 6104 Acked Flush 0 Done Done Sent false
17044 6105 Acked Write 8 Done Done Sent false
17045 6106 Acked Read 2 Done Done Sent false
17046 6107 Acked Write 8 Done Done Sent false
17047 6108 Acked Read 3 Done Done Sent false
17048 6109 Acked Write 9 Done Done Sent false
17049 6110 Acked Read 1 Done Done Sent false
17050 6111 Acked Read 8 Done Done Sent false
17051 6112 Acked Write 4 Done Done Sent false
17052 6113 Acked Write 9 Done Done Sent false
17053 6114 Acked Read 5 Done Done Sent false
17054 6115 Acked Write 5 Done Done Sent false
17055 6116 Acked Read 7 Done Done Sent false
17056 6117 Acked Write 3 Done Done Sent false
17057 6118 Acked Read 9 Done Done Sent false
17058 6119 Acked Write 10 Done Done Sent false
17059 6120 Acked Read 5 Done Done Sent false
17060 6121 Acked Read 9 Done Done Sent false
17061 6122 Acked Write 8 Done Done Sent false
17062 6123 Acked Write 8 Done Done Sent false
17063 6124 Acked Write 7 Done Done Sent false
17064 6125 Acked Read 7 Done Done Sent false
17065 6126 Acked Flush 0 Done Done Sent false
17066 6127 Acked Read 5 Done Done Sent false
17067 6128 Acked Write 5 Done Done Sent false
17068 6129 Acked Read 6 Done Done Sent false
17069 STATES DS:0 DS:1 DS:2 TOTAL
17070 Sent 0 0 301 301
17071 Done 301 301 0 602
17072 Skipped 0 0 0 0
17073 Error 0 0 0 0
17074 Last Flush: 6126 6126 5828
17075 Downstairs last five completed: 5828 5827 5826 5825 5824
17076 Upstairs last five completed: 6129 6128 6127 6126 6125
17077 CLIENT: Up:0 ds:301 act:2
17078 ----------------------------------------------------------------
17079 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17080 States: Active Active Offline
17081 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17082 5829 Acked Read 6 Done Done Sent false
17083 5830 Acked Flush 0 Done Done Sent false
17084 5831 Acked Read 2 Done Done Sent false
17085 5832 Acked Write 4 Done Done Sent false
17086 5833 Acked Write 8 Done Done Sent false
17087 5834 Acked Read 1 Done Done Sent false
17088 5835 Acked Read 10 Done Done Sent false
17089 5836 Acked Write 4 Done Done Sent false
17090 5837 Acked Read 2 Done Done Sent false
17091 5838 Acked Write 9 Done Done Sent false
17092 5839 Acked Read 3 Done Done Sent false
17093 5840 Acked Write 10 Done Done Sent false
17094 5841 Acked Read 4 Done Done Sent false
17095 5842 Acked Read 3 Done Done Sent false
17096 5843 Acked Read 10 Done Done Sent false
17097 5844 Acked Write 1 Done Done Sent false
17098 5845 Acked Write 10 Done Done Sent false
17099 5846 Acked Read 8 Done Done Sent false
17100 5847 Acked Read 4 Done Done Sent false
17101 5848 Acked Read 6 Done Done Sent false
17102 5849 Acked Read 1 Done Done Sent false
17103 5850 Acked Read 9 Done Done Sent false
17104 5851 Acked Read 2 Done Done Sent false
17105 5852 Acked Read 5 Done Done Sent false
17106 5853 Acked Write 6 Done Done Sent false
17107 5854 Acked Flush 0 Done Done Sent false
17108 5855 Acked Read 7 Done Done Sent false
17109 5856 Acked Write 7 Done Done Sent false
17110 5857 Acked Flush 0 Done Done Sent false
17111 5858 Acked Write 9 Done Done Sent false
17112 5859 Acked Read 10 Done Done Sent false
17113 5860 Acked Read 5 Done Done Sent false
17114 5861 Acked Read 3 Done Done Sent false
17115 5862 Acked Read 5 Done Done Sent false
17116 5863 Acked Write 9 Done Done Sent false
17117 5864 Acked Read 9 Done Done Sent false
17118 5865 Acked Read 9 Done Done Sent false
17119 5866 Acked Write 1 Done Done Sent false
17120 5867 Acked Write 5 Done Done Sent false
17121 5868 Acked Flush 0 Done Done Sent false
17122 5869 Acked Write 7 Done Done Sent false
17123 5870 Acked Write 8 Done Done Sent false
17124 5871 Acked Read 5 Done Done Sent false
17125 5872 Acked Write 9 Done Done Sent false
17126 5873 Acked Write 8 Done Done Sent false
17127 5874 Acked Write 10 Done Done Sent false
17128 5875 Acked Write 2 Done Done Sent false
17129 5876 Acked Read 7 Done Done Sent false
17130 5877 Acked Flush 0 Done Done Sent false
17131 5878 Acked Write 9 Done Done Sent false
17132 5879 Acked Write 8 Done Done Sent false
17133 5880 Acked Write 4 Done Done Sent false
17134 5881 Acked Write 4 Done Done Sent false
17135 5882 Acked Read 9 Done Done Sent false
17136 5883 Acked Read 7 Done Done Sent false
17137 5884 Acked Write 8 Done Done Sent false
17138 5885 Acked Read 8 Done Done Sent false
17139 5886 Acked Read 8 Done Done Sent false
17140 5887 Acked Read 10 Done Done Sent false
17141 5888 Acked Read 7 Done Done Sent false
17142 5889 Acked Read 3 Done Done Sent false
17143 5890 Acked Read 3 Done Done Sent false
17144 5891 Acked Read 10 Done Done Sent false
17145 5892 Acked Write 9 Done Done Sent false
17146 5893 Acked Read 8 Done Done Sent false
17147 5894 Acked Read 2 Done Done Sent false
17148 5895 Acked Write 8 Done Done Sent false
17149 5896 Acked Write 10 Done Done Sent false
17150 5897 Acked Flush 0 Done Done Sent false
17151 5898 Acked Read 8 Done Done Sent false
17152 5899 Acked Write 7 Done Done Sent false
17153 5900 Acked Write 2 Done Done Sent false
17154 5901 Acked Write 8 Done Done Sent false
17155 5902 Acked Write 8 Done Done Sent false
17156 5903 Acked Write 10 Done Done Sent false
17157 5904 Acked Read 10 Done Done Sent false
17158 5905 Acked Read 3 Done Done Sent false
17159 5906 Acked Read 4 Done Done Sent false
17160 5907 Acked Flush 0 Done Done Sent false
17161 5908 Acked Read 2 Done Done Sent false
17162 5909 Acked Read 8 Done Done Sent false
17163 5910 Acked Read 3 Done Done Sent false
17164 5911 Acked Read 7 Done Done Sent false
17165 5912 Acked Read 6 Done Done Sent false
17166 5913 Acked Write 6 Done Done Sent false
17167 5914 Acked Write 3 Done Done Sent false
17168 5915 Acked Read 1 Done Done Sent false
17169 5916 Acked Read 4 Done Done Sent false
17170 5917 Acked Read 7 Done Done Sent false
17171 5918 Acked Write 5 Done Done Sent false
17172 5919 Acked Read 6 Done Done Sent false
17173 5920 Acked Read 7 Done Done Sent false
17174 5921 Acked Write 4 Done Done Sent false
17175 5922 Acked Write 10 Done Done Sent false
17176 5923 Acked Read 3 Done Done Sent false
17177 5924 Acked Write 10 Done Done Sent false
17178 5925 Acked Read 5 Done Done Sent false
17179 5926 Acked Write 1 Done Done Sent false
17180 5927 Acked Write 3 Done Done Sent false
17181 5928 Acked Write 3 Done Done Sent false
17182 5929 Acked Read 10 Done Done Sent false
17183 5930 Acked Read 8 Done Done Sent false
17184 5931 Acked Read 3 Done Done Sent false
17185 5932 Acked Write 5 Done Done Sent false
17186 5933 Acked Read 9 Done Done Sent false
17187 5934 Acked Read 1 Done Done Sent false
17188 5935 Acked Write 6 Done Done Sent false
17189 5936 Acked Write 7 Done Done Sent false
17190 5937 Acked Write 9 Done Done Sent false
17191 5938 Acked Write 4 Done Done Sent false
17192 5939 Acked Read 10 Done Done Sent false
17193 5940 Acked Read 5 Done Done Sent false
17194 5941 Acked Flush 0 Done Done Sent false
17195 5942 Acked Write 8 Done Done Sent false
17196 5943 Acked Read 5 Done Done Sent false
17197 5944 Acked Read 8 Done Done Sent false
17198 5945 Acked Write 7 Done Done Sent false
17199 5946 Acked Read 3 Done Done Sent false
17200 5947 Acked Write 4 Done Done Sent false
17201 5948 Acked Read 2 Done Done Sent false
17202 5949 Acked Write 3 Done Done Sent false
17203 5950 Acked Flush 0 Done Done Sent false
17204 5951 Acked Flush 0 Done Done Sent false
17205 5952 Acked Read 9 Done Done Sent false
17206 5953 Acked Flush 0 Done Done Sent false
17207 5954 Acked Read 4 Done Done Sent false
17208 5955 Acked Flush 0 Done Done Sent false
17209 5956 Acked Read 10 Done Done Sent false
17210 5957 Acked Read 5 Done Done Sent false
17211 5958 Acked Write 10 Done Done Sent false
17212 5959 Acked Write 1 Done Done Sent false
17213 5960 Acked Write 10 Done Done Sent false
17214 5961 Acked Write 7 Done Done Sent false
17215 5962 Acked Read 10 Done Done Sent false
17216 5963 Acked Read 10 Done Done Sent false
17217 5964 Acked Write 6 Done Done Sent false
17218 5965 Acked Read 1 Done Done Sent false
17219 5966 Acked Read 5 Done Done Sent false
17220 5967 Acked Flush 0 Done Done Sent false
17221 5968 Acked Read 4 Done Done Sent false
17222 5969 Acked Read 9 Done Done Sent false
17223 5970 Acked Read 4 Done Done Sent false
17224 5971 Acked Read 6 Done Done Sent false
17225 5972 Acked Write 8 Done Done Sent false
17226 5973 Acked Write 3 Done Done Sent false
17227 5974 Acked Read 5 Done Done Sent false
17228 5975 Acked Read 4 Done Done Sent false
17229 5976 Acked Write 3 Done Done Sent false
17230 5977 Acked Write 9 Done Done Sent false
17231 5978 Acked Write 10 Done Done Sent false
17232 5979 Acked Write 7 Done Done Sent false
17233 5980 Acked Read 4 Done Done Sent false
17234 5981 Acked Read 2 Done Done Sent false
17235 5982 Acked Write 8 Done Done Sent false
17236 5983 Acked Read 3 Done Done Sent false
17237 5984 Acked Read 6 Done Done Sent false
17238 5985 Acked Read 7 Done Done Sent false
17239 5986 Acked Read 2 Done Done Sent false
17240 5987 Acked Flush 0 Done Done Sent false
17241 5988 Acked Read 3 Done Done Sent false
17242 5989 Acked Write 3 Done Done Sent false
17243 5990 Acked Read 7 Done Done Sent false
17244 5991 Acked Write 9 Done Done Sent false
17245 5992 Acked Write 1 Done Done Sent false
17246 5993 Acked Write 6 Done Done Sent false
17247 5994 Acked Read 3 Done Done Sent false
17248 5995 Acked Read 8 Done Done Sent false
17249 5996 Acked Flush 0 Done Done Sent false
17250 5997 Acked Read 10 Done Done Sent false
17251 5998 Acked Write 6 Done Done Sent false
17252 5999 Acked Write 1 Done Done Sent false
17253 6000 Acked Write 6 Done Done Sent false
17254 6001 Acked Read 5 Done Done Sent false
17255 6002 Acked Read 9 Done Done Sent false
17256 6003 Acked Write 8 Done Done Sent false
17257 6004 Acked Read 4 Done Done Sent false
17258 6005 Acked Read 9 Done Done Sent false
17259 6006 Acked Read 10 Done Done Sent false
17260 6007 Acked Read 5 Done Done Sent false
17261 6008 Acked Read 10 Done Done Sent false
17262 6009 Acked Write 3 Done Done Sent false
17263 6010 Acked Write 4 Done Done Sent false
17264 6011 Acked Write 5 Done Done Sent false
17265 6012 Acked Flush 0 Done Done Sent false
17266 6013 Acked Read 10 Done Done Sent false
17267 6014 Acked Flush 0 Done Done Sent false
17268 6015 Acked Write 2 Done Done Sent false
17269 6016 Acked Read 8 Done Done Sent false
17270 6017 Acked Read 9 Done Done Sent false
17271 6018 Acked Read 9 Done Done Sent false
17272 6019 Acked Write 8 Done Done Sent false
17273 6020 Acked Read 4 Done Done Sent false
17274 6021 Acked Read 2 Done Done Sent false
17275 6022 Acked Read 4 Done Done Sent false
17276 6023 Acked Write 5 Done Done Sent false
17277 6024 Acked Read 4 Done Done Sent false
17278 6025 Acked Read 2 Done Done Sent false
17279 6026 Acked Read 5 Done Done Sent false
17280 6027 Acked Flush 0 Done Done Sent false
17281 6028 Acked Read 1 Done Done Sent false
17282 6029 Acked Write 4 Done Done Sent false
17283 6030 Acked Read 3 Done Done Sent false
17284 6031 Acked Write 4 Done Done Sent false
17285 6032 Acked Write 8 Done Done Sent false
17286 6033 Acked Read 4 Done Done Sent false
17287 6034 Acked Read 7 Done Done Sent false
17288 6035 Acked Read 9 Done Done Sent false
17289 6036 Acked Write 8 Done Done Sent false
17290 6037 Acked Flush 0 Done Done Sent false
17291 6038 Acked Write 3 Done Done Sent false
17292 6039 Acked Read 1 Done Done Sent false
17293 6040 Acked Write 9 Done Done Sent false
17294 6041 Acked Write 1 Done Done Sent false
17295 6042 Acked Read 8 Done Done Sent false
17296 6043 Acked Write 5 Done Done Sent false
17297 6044 Acked Read 5 Done Done Sent false
17298 6045 Acked Write 3 Done Done Sent false
17299 6046 Acked Write 1 Done Done Sent false
17300 6047 Acked Read 6 Done Done Sent false
17301 6048 Acked Write 6 Done Done Sent false
17302 6049 Acked Write 8 Done Done Sent false
17303 6050 Acked Read 4 Done Done Sent false
17304 6051 Acked Read 10 Done Done Sent false
17305 6052 Acked Read 10 Done Done Sent false
17306 6053 Acked Flush 0 Done Done Sent false
17307 6054 Acked Write 3 Done Done Sent false
17308 6055 Acked Write 8 Done Done Sent false
17309 6056 Acked Write 7 Done Done Sent false
17310 6057 Acked Read 1 Done Done Sent false
17311 6058 Acked Flush 0 Done Done Sent false
17312 6059 Acked Read 5 Done Done Sent false
17313 6060 Acked Read 3 Done Done Sent false
17314 6061 Acked Write 2 Done Done Sent false
17315 6062 Acked Read 3 Done Done Sent false
17316 6063 Acked Read 9 Done Done Sent false
17317 6064 Acked Write 1 Done Done Sent false
17318 6065 Acked Read 9 Done Done Sent false
17319 6066 Acked Write 5 Done Done Sent false
17320 6067 Acked Read 9 Done Done Sent false
17321 6068 Acked Write 6 Done Done Sent false
17322 6069 Acked Write 4 Done Done Sent false
17323 6070 Acked Write 2 Done Done Sent false
17324 6071 Acked Write 8 Done Done Sent false
17325 6072 Acked Read 10 Done Done Sent false
17326 6073 Acked Write 10 Done Done Sent false
17327 6074 Acked Read 1 Done Done Sent false
17328 6075 Acked Write 3 Done Done Sent false
17329 6076 Acked Read 5 Done Done Sent false
17330 6077 Acked Read 3 Done Done Sent false
17331 6078 Acked Write 2 Done Done Sent false
17332 6079 Acked Write 9 Done Done Sent false
17333 6080 Acked Write 7 Done Done Sent false
17334 6081 Acked Write 4 Done Done Sent false
17335 6082 Acked Read 1 Done Done Sent false
17336 6083 Acked Read 2 Done Done Sent false
17337 6084 Acked Read 5 Done Done Sent false
17338 6085 Acked Read 8 Done Done Sent false
17339 6086 Acked Write 5 Done Done Sent false
17340 6087 Acked Read 9 Done Done Sent false
17341 6088 Acked Read 1 Done Done Sent false
17342 6089 Acked Read 1 Done Done Sent false
17343 6090 Acked Read 10 Done Done Sent false
17344 6091 Acked Read 5 Done Done Sent false
17345 6092 Acked Write 1 Done Done Sent false
17346 6093 Acked Write 10 Done Done Sent false
17347 6094 Acked Write 8 Done Done Sent false
17348 6095 Acked Write 2 Done Done Sent false
17349 6096 Acked Write 9 Done Done Sent false
17350 6097 Acked Read 3 Done Done Sent false
17351 6098 Acked Read 10 Done Done Sent false
17352 6099 Acked Write 4 Done Done Sent false
17353 6100 Acked Read 1 Done Done Sent false
17354 6101 Acked Flush 0 Done Done Sent false
17355 6102 Acked Write 6 Done Done Sent false
17356 6103 Acked Read 5 Done Done Sent false
17357 6104 Acked Flush 0 Done Done Sent false
17358 6105 Acked Write 8 Done Done Sent false
17359 6106 Acked Read 2 Done Done Sent false
17360 6107 Acked Write 8 Done Done Sent false
17361 6108 Acked Read 3 Done Done Sent false
17362 6109 Acked Write 9 Done Done Sent false
17363 6110 Acked Read 1 Done Done Sent false
17364 6111 Acked Read 8 Done Done Sent false
17365 6112 Acked Write 4 Done Done Sent false
17366 6113 Acked Write 9 Done Done Sent false
17367 6114 Acked Read 5 Done Done Sent false
17368 6115 Acked Write 5 Done Done Sent false
17369 6116 Acked Read 7 Done Done Sent false
17370 6117 Acked Write 3 Done Done Sent false
17371 6118 Acked Read 9 Done Done Sent false
17372 6119 Acked Write 10 Done Done Sent false
17373 6120 Acked Read 5 Done Done Sent false
17374 6121 Acked Read 9 Done Done Sent false
17375 6122 Acked Write 8 Done Done Sent false
17376 6123 Acked Write 8 Done Done Sent false
17377 6124 Acked Write 7 Done Done Sent false
17378 6125 Acked Read 7 Done Done Sent false
17379 6126 Acked Flush 0 Done Done Sent false
17380 6127 Acked Read 5 Done Done Sent false
17381 6128 Acked Write 5 Done Done Sent false
17382 6129 Acked Read 6 Done Done Sent false
17383 6130 Acked Flush 0 Done Done Sent false
17384 STATES DS:0 DS:1 DS:2 TOTAL
17385 Sent 0 0 302 302
17386 Done 302 302 0 604
17387 Skipped 0 0 0 0
17388 Error 0 0 0 0
17389 Last Flush: 6130 6130 5828
17390 Downstairs last five completed: 5828 5827 5826 5825 5824
17391 Upstairs last five completed: 6130 6129 6128 6127 6126
17392 CLIENT: Up:0 ds:302 act:2
173932024-12-04T13:34:20.054ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
173942024-12-04T13:34:20.054ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
173952024-12-04T13:34:20.054ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
173962024-12-04T13:34:20.054ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
173972024-12-04T13:34:20.054ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
17398 [8830] got start action
17399 [8830] I am going to restart
17400 Starting downstairs at port 8830
17401 Make output file at "/tmp/dsc/downstairs-8830.txt"
17402 [8830][2] reports Starting
17403 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1220)
17404 [8830][2] reports Running
17405 ----------------------------------------------------------------
17406 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17407 States: Active Active Offline
17408 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17409 5829 Acked Read 6 Done Done Sent false
17410 5830 Acked Flush 0 Done Done Sent false
17411 5831 Acked Read 2 Done Done Sent false
17412 5832 Acked Write 4 Done Done Sent false
17413 5833 Acked Write 8 Done Done Sent false
17414 5834 Acked Read 1 Done Done Sent false
17415 5835 Acked Read 10 Done Done Sent false
17416 5836 Acked Write 4 Done Done Sent false
17417 5837 Acked Read 2 Done Done Sent false
17418 5838 Acked Write 9 Done Done Sent false
17419 5839 Acked Read 3 Done Done Sent false
17420 5840 Acked Write 10 Done Done Sent false
17421 5841 Acked Read 4 Done Done Sent false
17422 5842 Acked Read 3 Done Done Sent false
17423 5843 Acked Read 10 Done Done Sent false
17424 5844 Acked Write 1 Done Done Sent false
17425 5845 Acked Write 10 Done Done Sent false
17426 5846 Acked Read 8 Done Done Sent false
17427 5847 Acked Read 4 Done Done Sent false
17428 5848 Acked Read 6 Done Done Sent false
17429 5849 Acked Read 1 Done Done Sent false
17430 5850 Acked Read 9 Done Done Sent false
17431 5851 Acked Read 2 Done Done Sent false
17432 5852 Acked Read 5 Done Done Sent false
17433 5853 Acked Write 6 Done Done Sent false
17434 5854 Acked Flush 0 Done Done Sent false
17435 5855 Acked Read 7 Done Done Sent false
17436 5856 Acked Write 7 Done Done Sent false
17437 5857 Acked Flush 0 Done Done Sent false
17438 5858 Acked Write 9 Done Done Sent false
17439 5859 Acked Read 10 Done Done Sent false
17440 5860 Acked Read 5 Done Done Sent false
17441 5861 Acked Read 3 Done Done Sent false
17442 5862 Acked Read 5 Done Done Sent false
17443 5863 Acked Write 9 Done Done Sent false
17444 5864 Acked Read 9 Done Done Sent false
17445 5865 Acked Read 9 Done Done Sent false
17446 5866 Acked Write 1 Done Done Sent false
17447 5867 Acked Write 5 Done Done Sent false
17448 5868 Acked Flush 0 Done Done Sent false
17449 5869 Acked Write 7 Done Done Sent false
17450 5870 Acked Write 8 Done Done Sent false
17451 5871 Acked Read 5 Done Done Sent false
17452 5872 Acked Write 9 Done Done Sent false
17453 5873 Acked Write 8 Done Done Sent false
17454 5874 Acked Write 10 Done Done Sent false
17455 5875 Acked Write 2 Done Done Sent false
17456 5876 Acked Read 7 Done Done Sent false
17457 5877 Acked Flush 0 Done Done Sent false
17458 5878 Acked Write 9 Done Done Sent false
17459 5879 Acked Write 8 Done Done Sent false
17460 5880 Acked Write 4 Done Done Sent false
17461 5881 Acked Write 4 Done Done Sent false
17462 5882 Acked Read 9 Done Done Sent false
17463 5883 Acked Read 7 Done Done Sent false
17464 5884 Acked Write 8 Done Done Sent false
17465 5885 Acked Read 8 Done Done Sent false
17466 5886 Acked Read 8 Done Done Sent false
17467 5887 Acked Read 10 Done Done Sent false
17468 5888 Acked Read 7 Done Done Sent false
17469 5889 Acked Read 3 Done Done Sent false
17470 5890 Acked Read 3 Done Done Sent false
17471 5891 Acked Read 10 Done Done Sent false
17472 5892 Acked Write 9 Done Done Sent false
17473 5893 Acked Read 8 Done Done Sent false
17474 5894 Acked Read 2 Done Done Sent false
17475 5895 Acked Write 8 Done Done Sent false
17476 5896 Acked Write 10 Done Done Sent false
17477 5897 Acked Flush 0 Done Done Sent false
17478 5898 Acked Read 8 Done Done Sent false
17479 5899 Acked Write 7 Done Done Sent false
17480 5900 Acked Write 2 Done Done Sent false
17481 5901 Acked Write 8 Done Done Sent false
17482 5902 Acked Write 8 Done Done Sent false
17483 5903 Acked Write 10 Done Done Sent false
17484 5904 Acked Read 10 Done Done Sent false
17485 5905 Acked Read 3 Done Done Sent false
17486 5906 Acked Read 4 Done Done Sent false
17487 5907 Acked Flush 0 Done Done Sent false
17488 5908 Acked Read 2 Done Done Sent false
17489 5909 Acked Read 8 Done Done Sent false
17490 5910 Acked Read 3 Done Done Sent false
17491 5911 Acked Read 7 Done Done Sent false
17492 5912 Acked Read 6 Done Done Sent false
17493 5913 Acked Write 6 Done Done Sent false
17494 5914 Acked Write 3 Done Done Sent false
17495 5915 Acked Read 1 Done Done Sent false
17496 5916 Acked Read 4 Done Done Sent false
17497 5917 Acked Read 7 Done Done Sent false
17498 5918 Acked Write 5 Done Done Sent false
17499 5919 Acked Read 6 Done Done Sent false
17500 5920 Acked Read 7 Done Done Sent false
17501 5921 Acked Write 4 Done Done Sent false
17502 5922 Acked Write 10 Done Done Sent false
17503 5923 Acked Read 3 Done Done Sent false
17504 5924 Acked Write 10 Done Done Sent false
17505 5925 Acked Read 5 Done Done Sent false
17506 5926 Acked Write 1 Done Done Sent false
17507 5927 Acked Write 3 Done Done Sent false
17508 5928 Acked Write 3 Done Done Sent false
17509 5929 Acked Read 10 Done Done Sent false
17510 5930 Acked Read 8 Done Done Sent false
17511 5931 Acked Read 3 Done Done Sent false
17512 5932 Acked Write 5 Done Done Sent false
17513 5933 Acked Read 9 Done Done Sent false
17514 5934 Acked Read 1 Done Done Sent false
17515 5935 Acked Write 6 Done Done Sent false
17516 5936 Acked Write 7 Done Done Sent false
17517 5937 Acked Write 9 Done Done Sent false
17518 5938 Acked Write 4 Done Done Sent false
17519 5939 Acked Read 10 Done Done Sent false
17520 5940 Acked Read 5 Done Done Sent false
17521 5941 Acked Flush 0 Done Done Sent false
17522 5942 Acked Write 8 Done Done Sent false
17523 5943 Acked Read 5 Done Done Sent false
17524 5944 Acked Read 8 Done Done Sent false
17525 5945 Acked Write 7 Done Done Sent false
17526 5946 Acked Read 3 Done Done Sent false
17527 5947 Acked Write 4 Done Done Sent false
17528 5948 Acked Read 2 Done Done Sent false
17529 5949 Acked Write 3 Done Done Sent false
17530 5950 Acked Flush 0 Done Done Sent false
17531 5951 Acked Flush 0 Done Done Sent false
17532 5952 Acked Read 9 Done Done Sent false
17533 5953 Acked Flush 0 Done Done Sent false
17534 5954 Acked Read 4 Done Done Sent false
17535 5955 Acked Flush 0 Done Done Sent false
17536 5956 Acked Read 10 Done Done Sent false
17537 5957 Acked Read 5 Done Done Sent false
17538 5958 Acked Write 10 Done Done Sent false
17539 5959 Acked Write 1 Done Done Sent false
17540 5960 Acked Write 10 Done Done Sent false
17541 5961 Acked Write 7 Done Done Sent false
17542 5962 Acked Read 10 Done Done Sent false
17543 5963 Acked Read 10 Done Done Sent false
17544 5964 Acked Write 6 Done Done Sent false
17545 5965 Acked Read 1 Done Done Sent false
17546 5966 Acked Read 5 Done Done Sent false
17547 5967 Acked Flush 0 Done Done Sent false
17548 5968 Acked Read 4 Done Done Sent false
17549 5969 Acked Read 9 Done Done Sent false
17550 5970 Acked Read 4 Done Done Sent false
17551 5971 Acked Read 6 Done Done Sent false
17552 5972 Acked Write 8 Done Done Sent false
17553 5973 Acked Write 3 Done Done Sent false
17554 5974 Acked Read 5 Done Done Sent false
17555 5975 Acked Read 4 Done Done Sent false
17556 5976 Acked Write 3 Done Done Sent false
17557 5977 Acked Write 9 Done Done Sent false
17558 5978 Acked Write 10 Done Done Sent false
17559 5979 Acked Write 7 Done Done Sent false
17560 5980 Acked Read 4 Done Done Sent false
17561 5981 Acked Read 2 Done Done Sent false
17562 5982 Acked Write 8 Done Done Sent false
17563 5983 Acked Read 3 Done Done Sent false
17564 5984 Acked Read 6 Done Done Sent false
17565 5985 Acked Read 7 Done Done Sent false
17566 5986 Acked Read 2 Done Done Sent false
17567 5987 Acked Flush 0 Done Done Sent false
17568 5988 Acked Read 3 Done Done Sent false
17569 5989 Acked Write 3 Done Done Sent false
17570 5990 Acked Read 7 Done Done Sent false
17571 5991 Acked Write 9 Done Done Sent false
17572 5992 Acked Write 1 Done Done Sent false
17573 5993 Acked Write 6 Done Done Sent false
17574 5994 Acked Read 3 Done Done Sent false
17575 5995 Acked Read 8 Done Done Sent false
17576 5996 Acked Flush 0 Done Done Sent false
17577 5997 Acked Read 10 Done Done Sent false
17578 5998 Acked Write 6 Done Done Sent false
17579 5999 Acked Write 1 Done Done Sent false
17580 6000 Acked Write 6 Done Done Sent false
17581 6001 Acked Read 5 Done Done Sent false
17582 6002 Acked Read 9 Done Done Sent false
17583 6003 Acked Write 8 Done Done Sent false
17584 6004 Acked Read 4 Done Done Sent false
17585 6005 Acked Read 9 Done Done Sent false
17586 6006 Acked Read 10 Done Done Sent false
17587 6007 Acked Read 5 Done Done Sent false
17588 6008 Acked Read 10 Done Done Sent false
17589 6009 Acked Write 3 Done Done Sent false
17590 6010 Acked Write 4 Done Done Sent false
17591 6011 Acked Write 5 Done Done Sent false
17592 6012 Acked Flush 0 Done Done Sent false
17593 6013 Acked Read 10 Done Done Sent false
17594 6014 Acked Flush 0 Done Done Sent false
17595 6015 Acked Write 2 Done Done Sent false
17596 6016 Acked Read 8 Done Done Sent false
17597 6017 Acked Read 9 Done Done Sent false
17598 6018 Acked Read 9 Done Done Sent false
17599 6019 Acked Write 8 Done Done Sent false
17600 6020 Acked Read 4 Done Done Sent false
17601 6021 Acked Read 2 Done Done Sent false
17602 6022 Acked Read 4 Done Done Sent false
17603 6023 Acked Write 5 Done Done Sent false
17604 6024 Acked Read 4 Done Done Sent false
17605 6025 Acked Read 2 Done Done Sent false
17606 6026 Acked Read 5 Done Done Sent false
17607 6027 Acked Flush 0 Done Done Sent false
17608 6028 Acked Read 1 Done Done Sent false
17609 6029 Acked Write 4 Done Done Sent false
17610 6030 Acked Read 3 Done Done Sent false
17611 6031 Acked Write 4 Done Done Sent false
17612 6032 Acked Write 8 Done Done Sent false
17613 6033 Acked Read 4 Done Done Sent false
17614 6034 Acked Read 7 Done Done Sent false
17615 6035 Acked Read 9 Done Done Sent false
17616 6036 Acked Write 8 Done Done Sent false
17617 6037 Acked Flush 0 Done Done Sent false
17618 6038 Acked Write 3 Done Done Sent false
17619 6039 Acked Read 1 Done Done Sent false
17620 6040 Acked Write 9 Done Done Sent false
17621 6041 Acked Write 1 Done Done Sent false
17622 6042 Acked Read 8 Done Done Sent false
17623 6043 Acked Write 5 Done Done Sent false
17624 6044 Acked Read 5 Done Done Sent false
17625 6045 Acked Write 3 Done Done Sent false
17626 6046 Acked Write 1 Done Done Sent false
17627 6047 Acked Read 6 Done Done Sent false
17628 6048 Acked Write 6 Done Done Sent false
17629 6049 Acked Write 8 Done Done Sent false
17630 6050 Acked Read 4 Done Done Sent false
17631 6051 Acked Read 10 Done Done Sent false
17632 6052 Acked Read 10 Done Done Sent false
17633 6053 Acked Flush 0 Done Done Sent false
17634 6054 Acked Write 3 Done Done Sent false
17635 6055 Acked Write 8 Done Done Sent false
17636 6056 Acked Write 7 Done Done Sent false
17637 6057 Acked Read 1 Done Done Sent false
17638 6058 Acked Flush 0 Done Done Sent false
17639 6059 Acked Read 5 Done Done Sent false
17640 6060 Acked Read 3 Done Done Sent false
17641 6061 Acked Write 2 Done Done Sent false
17642 6062 Acked Read 3 Done Done Sent false
17643 6063 Acked Read 9 Done Done Sent false
17644 6064 Acked Write 1 Done Done Sent false
17645 6065 Acked Read 9 Done Done Sent false
17646 6066 Acked Write 5 Done Done Sent false
17647 6067 Acked Read 9 Done Done Sent false
17648 6068 Acked Write 6 Done Done Sent false
17649 6069 Acked Write 4 Done Done Sent false
17650 6070 Acked Write 2 Done Done Sent false
17651 6071 Acked Write 8 Done Done Sent false
17652 6072 Acked Read 10 Done Done Sent false
17653 6073 Acked Write 10 Done Done Sent false
17654 6074 Acked Read 1 Done Done Sent false
17655 6075 Acked Write 3 Done Done Sent false
17656 6076 Acked Read 5 Done Done Sent false
17657 6077 Acked Read 3 Done Done Sent false
17658 6078 Acked Write 2 Done Done Sent false
17659 6079 Acked Write 9 Done Done Sent false
17660 6080 Acked Write 7 Done Done Sent false
17661 6081 Acked Write 4 Done Done Sent false
17662 6082 Acked Read 1 Done Done Sent false
17663 6083 Acked Read 2 Done Done Sent false
17664 6084 Acked Read 5 Done Done Sent false
17665 6085 Acked Read 8 Done Done Sent false
17666 6086 Acked Write 5 Done Done Sent false
17667 6087 Acked Read 9 Done Done Sent false
17668 6088 Acked Read 1 Done Done Sent false
17669 6089 Acked Read 1 Done Done Sent false
17670 6090 Acked Read 10 Done Done Sent false
17671 6091 Acked Read 5 Done Done Sent false
17672 6092 Acked Write 1 Done Done Sent false
17673 6093 Acked Write 10 Done Done Sent false
17674 6094 Acked Write 8 Done Done Sent false
17675 6095 Acked Write 2 Done Done Sent false
17676 6096 Acked Write 9 Done Done Sent false
17677 6097 Acked Read 3 Done Done Sent false
17678 6098 Acked Read 10 Done Done Sent false
17679 6099 Acked Write 4 Done Done Sent false
17680 6100 Acked Read 1 Done Done Sent false
17681 6101 Acked Flush 0 Done Done Sent false
17682 6102 Acked Write 6 Done Done Sent false
17683 6103 Acked Read 5 Done Done Sent false
17684 6104 Acked Flush 0 Done Done Sent false
17685 6105 Acked Write 8 Done Done Sent false
17686 6106 Acked Read 2 Done Done Sent false
17687 6107 Acked Write 8 Done Done Sent false
17688 6108 Acked Read 3 Done Done Sent false
17689 6109 Acked Write 9 Done Done Sent false
17690 6110 Acked Read 1 Done Done Sent false
17691 6111 Acked Read 8 Done Done Sent false
17692 6112 Acked Write 4 Done Done Sent false
17693 6113 Acked Write 9 Done Done Sent false
17694 6114 Acked Read 5 Done Done Sent false
17695 6115 Acked Write 5 Done Done Sent false
17696 6116 Acked Read 7 Done Done Sent false
17697 6117 Acked Write 3 Done Done Sent false
17698 6118 Acked Read 9 Done Done Sent false
17699 6119 Acked Write 10 Done Done Sent false
17700 6120 Acked Read 5 Done Done Sent false
17701 6121 Acked Read 9 Done Done Sent false
17702 6122 Acked Write 8 Done Done Sent false
17703 6123 Acked Write 8 Done Done Sent false
17704 6124 Acked Write 7 Done Done Sent false
17705 6125 Acked Read 7 Done Done Sent false
17706 6126 Acked Flush 0 Done Done Sent false
17707 6127 Acked Read 5 Done Done Sent false
17708 6128 Acked Write 5 Done Done Sent false
17709 6129 Acked Read 6 Done Done Sent false
17710 6130 Acked Flush 0 Done Done Sent false
17711 STATES DS:0 DS:1 DS:2 TOTAL
17712 Sent 0 0 302 302
17713 Done 302 302 0 604
17714 Skipped 0 0 0 0
17715 Error 0 0 0 0
17716 Last Flush: 6130 6130 5828
17717 Downstairs last five completed: 5828 5827 5826 5825 5824
17718 Upstairs last five completed: 6130 6129 6128 6127 6126
17719 CLIENT: Up:0 ds:302 act:2
17720 ----------------------------------------------------------------
17721 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17722 States: Active Active Offline
17723 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17724 5829 Acked Read 6 Done Done Sent false
17725 5830 Acked Flush 0 Done Done Sent false
17726 5831 Acked Read 2 Done Done Sent false
17727 5832 Acked Write 4 Done Done Sent false
17728 5833 Acked Write 8 Done Done Sent false
17729 5834 Acked Read 1 Done Done Sent false
17730 5835 Acked Read 10 Done Done Sent false
17731 5836 Acked Write 4 Done Done Sent false
17732 5837 Acked Read 2 Done Done Sent false
17733 5838 Acked Write 9 Done Done Sent false
17734 5839 Acked Read 3 Done Done Sent false
17735 5840 Acked Write 10 Done Done Sent false
17736 5841 Acked Read 4 Done Done Sent false
17737 5842 Acked Read 3 Done Done Sent false
17738 5843 Acked Read 10 Done Done Sent false
17739 5844 Acked Write 1 Done Done Sent false
17740 5845 Acked Write 10 Done Done Sent false
17741 5846 Acked Read 8 Done Done Sent false
17742 5847 Acked Read 4 Done Done Sent false
17743 5848 Acked Read 6 Done Done Sent false
17744 5849 Acked Read 1 Done Done Sent false
17745 5850 Acked Read 9 Done Done Sent false
17746 5851 Acked Read 2 Done Done Sent false
17747 5852 Acked Read 5 Done Done Sent false
17748 5853 Acked Write 6 Done Done Sent false
17749 5854 Acked Flush 0 Done Done Sent false
17750 5855 Acked Read 7 Done Done Sent false
17751 5856 Acked Write 7 Done Done Sent false
17752 5857 Acked Flush 0 Done Done Sent false
17753 5858 Acked Write 9 Done Done Sent false
17754 5859 Acked Read 10 Done Done Sent false
17755 5860 Acked Read 5 Done Done Sent false
17756 5861 Acked Read 3 Done Done Sent false
17757 5862 Acked Read 5 Done Done Sent false
17758 5863 Acked Write 9 Done Done Sent false
17759 5864 Acked Read 9 Done Done Sent false
17760 5865 Acked Read 9 Done Done Sent false
17761 5866 Acked Write 1 Done Done Sent false
17762 5867 Acked Write 5 Done Done Sent false
17763 5868 Acked Flush 0 Done Done Sent false
17764 5869 Acked Write 7 Done Done Sent false
17765 5870 Acked Write 8 Done Done Sent false
17766 5871 Acked Read 5 Done Done Sent false
17767 5872 Acked Write 9 Done Done Sent false
17768 5873 Acked Write 8 Done Done Sent false
17769 5874 Acked Write 10 Done Done Sent false
17770 5875 Acked Write 2 Done Done Sent false
17771 5876 Acked Read 7 Done Done Sent false
17772 5877 Acked Flush 0 Done Done Sent false
17773 5878 Acked Write 9 Done Done Sent false
17774 5879 Acked Write 8 Done Done Sent false
17775 5880 Acked Write 4 Done Done Sent false
17776 5881 Acked Write 4 Done Done Sent false
17777 5882 Acked Read 9 Done Done Sent false
17778 5883 Acked Read 7 Done Done Sent false
17779 5884 Acked Write 8 Done Done Sent false
17780 5885 Acked Read 8 Done Done Sent false
17781 5886 Acked Read 8 Done Done Sent false
17782 5887 Acked Read 10 Done Done Sent false
17783 5888 Acked Read 7 Done Done Sent false
17784 5889 Acked Read 3 Done Done Sent false
17785 5890 Acked Read 3 Done Done Sent false
17786 5891 Acked Read 10 Done Done Sent false
17787 5892 Acked Write 9 Done Done Sent false
17788 5893 Acked Read 8 Done Done Sent false
17789 5894 Acked Read 2 Done Done Sent false
17790 5895 Acked Write 8 Done Done Sent false
17791 5896 Acked Write 10 Done Done Sent false
17792 5897 Acked Flush 0 Done Done Sent false
17793 5898 Acked Read 8 Done Done Sent false
17794 5899 Acked Write 7 Done Done Sent false
17795 5900 Acked Write 2 Done Done Sent false
17796 5901 Acked Write 8 Done Done Sent false
17797 5902 Acked Write 8 Done Done Sent false
17798 5903 Acked Write 10 Done Done Sent false
17799 5904 Acked Read 10 Done Done Sent false
17800 5905 Acked Read 3 Done Done Sent false
17801 5906 Acked Read 4 Done Done Sent false
17802 5907 Acked Flush 0 Done Done Sent false
17803 5908 Acked Read 2 Done Done Sent false
17804 5909 Acked Read 8 Done Done Sent false
17805 5910 Acked Read 3 Done Done Sent false
17806 5911 Acked Read 7 Done Done Sent false
17807 5912 Acked Read 6 Done Done Sent false
17808 5913 Acked Write 6 Done Done Sent false
17809 5914 Acked Write 3 Done Done Sent false
17810 5915 Acked Read 1 Done Done Sent false
17811 5916 Acked Read 4 Done Done Sent false
17812 5917 Acked Read 7 Done Done Sent false
17813 5918 Acked Write 5 Done Done Sent false
17814 5919 Acked Read 6 Done Done Sent false
17815 5920 Acked Read 7 Done Done Sent false
17816 5921 Acked Write 4 Done Done Sent false
17817 5922 Acked Write 10 Done Done Sent false
17818 5923 Acked Read 3 Done Done Sent false
17819 5924 Acked Write 10 Done Done Sent false
17820 5925 Acked Read 5 Done Done Sent false
17821 5926 Acked Write 1 Done Done Sent false
17822 5927 Acked Write 3 Done Done Sent false
17823 5928 Acked Write 3 Done Done Sent false
17824 5929 Acked Read 10 Done Done Sent false
17825 5930 Acked Read 8 Done Done Sent false
17826 5931 Acked Read 3 Done Done Sent false
17827 5932 Acked Write 5 Done Done Sent false
17828 5933 Acked Read 9 Done Done Sent false
17829 5934 Acked Read 1 Done Done Sent false
17830 5935 Acked Write 6 Done Done Sent false
17831 5936 Acked Write 7 Done Done Sent false
17832 5937 Acked Write 9 Done Done Sent false
17833 5938 Acked Write 4 Done Done Sent false
17834 5939 Acked Read 10 Done Done Sent false
17835 5940 Acked Read 5 Done Done Sent false
17836 5941 Acked Flush 0 Done Done Sent false
17837 5942 Acked Write 8 Done Done Sent false
17838 5943 Acked Read 5 Done Done Sent false
17839 5944 Acked Read 8 Done Done Sent false
17840 5945 Acked Write 7 Done Done Sent false
17841 5946 Acked Read 3 Done Done Sent false
17842 5947 Acked Write 4 Done Done Sent false
17843 5948 Acked Read 2 Done Done Sent false
17844 5949 Acked Write 3 Done Done Sent false
17845 5950 Acked Flush 0 Done Done Sent false
17846 5951 Acked Flush 0 Done Done Sent false
17847 5952 Acked Read 9 Done Done Sent false
17848 5953 Acked Flush 0 Done Done Sent false
17849 5954 Acked Read 4 Done Done Sent false
17850 5955 Acked Flush 0 Done Done Sent false
17851 5956 Acked Read 10 Done Done Sent false
17852 5957 Acked Read 5 Done Done Sent false
17853 5958 Acked Write 10 Done Done Sent false
17854 5959 Acked Write 1 Done Done Sent false
17855 5960 Acked Write 10 Done Done Sent false
17856 5961 Acked Write 7 Done Done Sent false
17857 5962 Acked Read 10 Done Done Sent false
17858 5963 Acked Read 10 Done Done Sent false
17859 5964 Acked Write 6 Done Done Sent false
17860 5965 Acked Read 1 Done Done Sent false
17861 5966 Acked Read 5 Done Done Sent false
17862 5967 Acked Flush 0 Done Done Sent false
17863 5968 Acked Read 4 Done Done Sent false
17864 5969 Acked Read 9 Done Done Sent false
17865 5970 Acked Read 4 Done Done Sent false
17866 5971 Acked Read 6 Done Done Sent false
17867 5972 Acked Write 8 Done Done Sent false
17868 5973 Acked Write 3 Done Done Sent false
17869 5974 Acked Read 5 Done Done Sent false
17870 5975 Acked Read 4 Done Done Sent false
17871 5976 Acked Write 3 Done Done Sent false
17872 5977 Acked Write 9 Done Done Sent false
17873 5978 Acked Write 10 Done Done Sent false
17874 5979 Acked Write 7 Done Done Sent false
17875 5980 Acked Read 4 Done Done Sent false
17876 5981 Acked Read 2 Done Done Sent false
17877 5982 Acked Write 8 Done Done Sent false
17878 5983 Acked Read 3 Done Done Sent false
17879 5984 Acked Read 6 Done Done Sent false
17880 5985 Acked Read 7 Done Done Sent false
17881 5986 Acked Read 2 Done Done Sent false
17882 5987 Acked Flush 0 Done Done Sent false
17883 5988 Acked Read 3 Done Done Sent false
17884 5989 Acked Write 3 Done Done Sent false
17885 5990 Acked Read 7 Done Done Sent false
17886 5991 Acked Write 9 Done Done Sent false
17887 5992 Acked Write 1 Done Done Sent false
17888 5993 Acked Write 6 Done Done Sent false
17889 5994 Acked Read 3 Done Done Sent false
17890 5995 Acked Read 8 Done Done Sent false
17891 5996 Acked Flush 0 Done Done Sent false
17892 5997 Acked Read 10 Done Done Sent false
17893 5998 Acked Write 6 Done Done Sent false
17894 5999 Acked Write 1 Done Done Sent false
17895 6000 Acked Write 6 Done Done Sent false
17896 6001 Acked Read 5 Done Done Sent false
17897 6002 Acked Read 9 Done Done Sent false
17898 6003 Acked Write 8 Done Done Sent false
17899 6004 Acked Read 4 Done Done Sent false
17900 6005 Acked Read 9 Done Done Sent false
17901 6006 Acked Read 10 Done Done Sent false
17902 6007 Acked Read 5 Done Done Sent false
17903 6008 Acked Read 10 Done Done Sent false
17904 6009 Acked Write 3 Done Done Sent false
17905 6010 Acked Write 4 Done Done Sent false
17906 6011 Acked Write 5 Done Done Sent false
17907 6012 Acked Flush 0 Done Done Sent false
17908 6013 Acked Read 10 Done Done Sent false
17909 6014 Acked Flush 0 Done Done Sent false
17910 6015 Acked Write 2 Done Done Sent false
17911 6016 Acked Read 8 Done Done Sent false
17912 6017 Acked Read 9 Done Done Sent false
17913 6018 Acked Read 9 Done Done Sent false
17914 6019 Acked Write 8 Done Done Sent false
17915 6020 Acked Read 4 Done Done Sent false
17916 6021 Acked Read 2 Done Done Sent false
17917 6022 Acked Read 4 Done Done Sent false
17918 6023 Acked Write 5 Done Done Sent false
17919 6024 Acked Read 4 Done Done Sent false
17920 6025 Acked Read 2 Done Done Sent false
17921 6026 Acked Read 5 Done Done Sent false
17922 6027 Acked Flush 0 Done Done Sent false
17923 6028 Acked Read 1 Done Done Sent false
17924 6029 Acked Write 4 Done Done Sent false
17925 6030 Acked Read 3 Done Done Sent false
17926 6031 Acked Write 4 Done Done Sent false
17927 6032 Acked Write 8 Done Done Sent false
17928 6033 Acked Read 4 Done Done Sent false
17929 6034 Acked Read 7 Done Done Sent false
17930 6035 Acked Read 9 Done Done Sent false
17931 6036 Acked Write 8 Done Done Sent false
17932 6037 Acked Flush 0 Done Done Sent false
17933 6038 Acked Write 3 Done Done Sent false
17934 6039 Acked Read 1 Done Done Sent false
17935 6040 Acked Write 9 Done Done Sent false
17936 6041 Acked Write 1 Done Done Sent false
17937 6042 Acked Read 8 Done Done Sent false
17938 6043 Acked Write 5 Done Done Sent false
17939 6044 Acked Read 5 Done Done Sent false
17940 6045 Acked Write 3 Done Done Sent false
17941 6046 Acked Write 1 Done Done Sent false
17942 6047 Acked Read 6 Done Done Sent false
17943 6048 Acked Write 6 Done Done Sent false
17944 6049 Acked Write 8 Done Done Sent false
17945 6050 Acked Read 4 Done Done Sent false
17946 6051 Acked Read 10 Done Done Sent false
17947 6052 Acked Read 10 Done Done Sent false
17948 6053 Acked Flush 0 Done Done Sent false
17949 6054 Acked Write 3 Done Done Sent false
17950 6055 Acked Write 8 Done Done Sent false
17951 6056 Acked Write 7 Done Done Sent false
17952 6057 Acked Read 1 Done Done Sent false
17953 6058 Acked Flush 0 Done Done Sent false
17954 6059 Acked Read 5 Done Done Sent false
17955 6060 Acked Read 3 Done Done Sent false
17956 6061 Acked Write 2 Done Done Sent false
17957 6062 Acked Read 3 Done Done Sent false
17958 6063 Acked Read 9 Done Done Sent false
17959 6064 Acked Write 1 Done Done Sent false
17960 6065 Acked Read 9 Done Done Sent false
17961 6066 Acked Write 5 Done Done Sent false
17962 6067 Acked Read 9 Done Done Sent false
17963 6068 Acked Write 6 Done Done Sent false
17964 6069 Acked Write 4 Done Done Sent false
17965 6070 Acked Write 2 Done Done Sent false
17966 6071 Acked Write 8 Done Done Sent false
17967 6072 Acked Read 10 Done Done Sent false
17968 6073 Acked Write 10 Done Done Sent false
17969 6074 Acked Read 1 Done Done Sent false
17970 6075 Acked Write 3 Done Done Sent false
17971 6076 Acked Read 5 Done Done Sent false
17972 6077 Acked Read 3 Done Done Sent false
17973 6078 Acked Write 2 Done Done Sent false
17974 6079 Acked Write 9 Done Done Sent false
17975 6080 Acked Write 7 Done Done Sent false
17976 6081 Acked Write 4 Done Done Sent false
17977 6082 Acked Read 1 Done Done Sent false
17978 6083 Acked Read 2 Done Done Sent false
17979 6084 Acked Read 5 Done Done Sent false
17980 6085 Acked Read 8 Done Done Sent false
17981 6086 Acked Write 5 Done Done Sent false
17982 6087 Acked Read 9 Done Done Sent false
17983 6088 Acked Read 1 Done Done Sent false
17984 6089 Acked Read 1 Done Done Sent false
17985 6090 Acked Read 10 Done Done Sent false
17986 6091 Acked Read 5 Done Done Sent false
17987 6092 Acked Write 1 Done Done Sent false
17988 6093 Acked Write 10 Done Done Sent false
17989 6094 Acked Write 8 Done Done Sent false
17990 6095 Acked Write 2 Done Done Sent false
17991 6096 Acked Write 9 Done Done Sent false
17992 6097 Acked Read 3 Done Done Sent false
17993 6098 Acked Read 10 Done Done Sent false
17994 6099 Acked Write 4 Done Done Sent false
17995 6100 Acked Read 1 Done Done Sent false
17996 6101 Acked Flush 0 Done Done Sent false
17997 6102 Acked Write 6 Done Done Sent false
17998 6103 Acked Read 5 Done Done Sent false
17999 6104 Acked Flush 0 Done Done Sent false
18000 6105 Acked Write 8 Done Done Sent false
18001 6106 Acked Read 2 Done Done Sent false
18002 6107 Acked Write 8 Done Done Sent false
18003 6108 Acked Read 3 Done Done Sent false
18004 6109 Acked Write 9 Done Done Sent false
18005 6110 Acked Read 1 Done Done Sent false
18006 6111 Acked Read 8 Done Done Sent false
18007 6112 Acked Write 4 Done Done Sent false
18008 6113 Acked Write 9 Done Done Sent false
18009 6114 Acked Read 5 Done Done Sent false
18010 6115 Acked Write 5 Done Done Sent false
18011 6116 Acked Read 7 Done Done Sent false
18012 6117 Acked Write 3 Done Done Sent false
18013 6118 Acked Read 9 Done Done Sent false
18014 6119 Acked Write 10 Done Done Sent false
18015 6120 Acked Read 5 Done Done Sent false
18016 6121 Acked Read 9 Done Done Sent false
18017 6122 Acked Write 8 Done Done Sent false
18018 6123 Acked Write 8 Done Done Sent false
18019 6124 Acked Write 7 Done Done Sent false
18020 6125 Acked Read 7 Done Done Sent false
18021 6126 Acked Flush 0 Done Done Sent false
18022 6127 Acked Read 5 Done Done Sent false
18023 6128 Acked Write 5 Done Done Sent false
18024 6129 Acked Read 6 Done Done Sent false
18025 6130 Acked Flush 0 Done Done Sent false
18026 STATES DS:0 DS:1 DS:2 TOTAL
18027 Sent 0 0 302 302
18028 Done 302 302 0 604
18029 Skipped 0 0 0 0
18030 Error 0 0 0 0
18031 Last Flush: 6130 6130 5828
18032 Downstairs last five completed: 5828 5827 5826 5825 5824
18033 Upstairs last five completed: 6130 6129 6128 6127 6126
18034 CLIENT: Up:0 ds:302 act:2
180352024-12-04T13:34:30.055ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180362024-12-04T13:34:30.055ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53960) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180372024-12-04T13:34:30.056ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180382024-12-04T13:34:30.056ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180392024-12-04T13:34:30.056ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180402024-12-04T13:34:30.056ZINFOcrucible: send last flush ID to this DS: 5828 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180412024-12-04T13:34:30.057ZINFOcrucible: Replied this last flush ID: 5828 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180422024-12-04T13:34:30.057ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180432024-12-04T13:34:30.057ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180442024-12-04T13:34:30.057ZINFOcrucible: [2] client re-new 302 jobs since flush 5828 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180452024-12-04T13:34:30.057ZINFOcrucible: [2] Replayed 302 jobs since flush: 5828 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
18046 ----------------------------------------------------------------
18047 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18048 Downstairs last five completed: 6130 6129 6128 6127 6126
18049 Upstairs last five completed: 6130 6129 6128 6127 6126
18050 CLIENT: Up:0 ds:0 act:3
18051 Replay: All jobs finished
18052 Main task has work to do, go find it
18053 got dsc Stop(1)
18054 stop 1
18055 Dec 04 13:34:30.306 INFO[8820] Got stop action so:false kr:false
18056 request completed, latency_us: 350, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 40f613ef-b00c-400c-9441-a63b778624cd, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
180572024-12-04T13:34:30.307ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180582024-12-04T13:34:30.307ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180592024-12-04T13:34:30.307ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180602024-12-04T13:34:30.307ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
180612024-12-04T13:34:30.307ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
18062 Dec 04 13:34:30.307 INFO request completed, latency_us: 143, response_code: 200, uri: /state/cid/1, method: GET, req_id: 94d0240e-a2b7-44c1-8853-4ee8d71cab16, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
180632024-12-04T13:34:30.307ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
18064 [8820] Exited with: ExitStatus(unix_wait_status(9))
18065 [8820][1] reports Exit
18066 Dec 04 13:34:34.309 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/1, method: GET, req_id: 6a52e4bf-75c3-47c2-8fcd-ef9e76a3d986, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
18067 001/300 Write block 3895 len 40960 data: 3 3 3 4 4 4 4 4 5 4
18068 002/300 Flush
18069 003/300 Write block 879 len 28672 data: 2 2 2 3 4 4 4
18070 004/300 Read block 1612 len 28672
18071 005/300 Read block 3509 len 12288
18072 006/300 Write block 3243 len 24576 data: 4 3 2 2 2 2
18073 007/300 Read block 4149 len 40960
18074 008/300 Write block 199 len 8192 data: 3 3
18075 009/300 Flush
18076 010/300 Write block 4396 len 28672 data: 6 6 7 7 8 6 5
18077 011/300 Read block 1396 len 4096
18078 012/300 Read block 3551 len 16384
18079 013/300 Read block 3724 len 16384
18080 014/300 Write block 4645 len 24576 data: 2 3 4 4 4 4
18081 015/300 Read block 4874 len 4096
18082 016/300 Read block 2583 len 28672
18083 017/300 Read block 1286 len 40960
18084 018/300 Read block 2375 len 32768
18085 019/300 Read block 797 len 8192
18086 020/300 Flush
18087 021/300 Write block 2413 len 8192 data: 4 4
18088 022/300 Read block 1323 len 16384
18089 023/300 Read block 1318 len 16384
18090 024/300 Write block 4073 len 4096 data: 2
18091 025/300 Read block 472 len 12288
18092 026/300 Read block 1799 len 24576
18093 027/300 Flush
18094 028/300 Flush
18095 029/300 Read block 910 len 36864
18096 030/300 Read block 2471 len 16384
18097 031/300 Write block 1952 len 20480 data: 6 4 3 2 2
18098 032/300 Write block 1744 len 32768 data: 8 7 7 7 8 7 4 4
18099 033/300 Flush
18100 034/300 Read block 651 len 16384
18101 035/300 Read block 4081 len 16384
18102 036/300 Write block 3563 len 12288 data: 2 3 4
18103 037/300 Write block 3564 len 32768 data: 4 5 5 5 4 5 5 6
18104 038/300 Read block 2018 len 32768
18105 039/300 Write block 4549 len 4096 data: 3
18106 040/300 Write block 1275 len 40960 data: 3 3 3 4 2 2 3 3 3 4
18107 041/300 Write block 443 len 4096 data: 8
18108 042/300 Write block 3330 len 28672 data: 4 5 4 4 4 4 3
18109 043/300 Write block 2555 len 20480 data: 3 3 4 5 4
18110 044/300 Read block 4480 len 32768
18111 045/300 Read block 2877 len 20480
18112 046/300 Read block 2073 len 32768
18113 047/300 Read block 4214 len 8192
18114 048/300 Read block 4760 len 28672
18115 049/300 Read block 1226 len 20480
18116 050/300 Write block 2515 len 24576 data: 4 4 4 4 4 5
18117 051/300 Read block 248 len 16384
18118 052/300 Write block 2567 len 20480 data: 3 4 4 5 5
18119 053/300 Flush
18120 054/300 Read block 176 len 24576
18121 055/300 Write block 503 len 12288 data: 4 4 5
18122 056/300 Read block 525 len 16384
18123 057/300 Flush
18124 058/300 Read block 4959 len 40960
18125 059/300 Write block 3484 len 40960 data: 7 8 7 6 8 7 6 4 4 5
18126 060/300 Read block 3424 len 8192
18127 061/300 Read block 1527 len 12288
18128 062/300 Read block 2872 len 24576
18129 063/300 Read block 4097 len 32768
18130 064/300 Read block 1799 len 20480
18131 065/300 Read block 3349 len 24576
18132 066/300 Write block 1678 len 32768 data: 7 6 7 6 6 7 9 8
18133 067/300 Read block 1970 len 4096
18134 068/300 Write block 198 len 12288 data: 4 4 4
18135 069/300 Read block 3027 len 16384
18136 070/300 Read block 2660 len 24576
18137 071/300 Read block 3109 len 28672
18138 072/300 Read block 4620 len 28672
18139 073/300 Write block 3676 len 20480 data: 5 5 6 5 5
18140 074/300 Flush
18141 075/300 Read block 4849 len 24576
18142 076/300 Read block 3656 len 28672
18143 077/300 Read block 4519 len 16384
18144 078/300 Read block 1050 len 4096
18145 079/300 Read block 2151 len 8192
18146 080/300 Write block 656 len 40960 data: 3 3 2 2 2 2 4 4 4 4
18147 081/300 Read block 3768 len 12288
18148 082/300 Write block 3876 len 4096 data: 4
18149 083/300 Read block 2309 len 20480
18150 084/300 Read block 2718 len 24576
18151 085/300 Read block 4000 len 8192
18152 086/300 Write block 458 len 4096 data: 6
18153 087/300 Read block 420 len 4096
18154 088/300 Read block 3568 len 12288
18155 089/300 Read block 1766 len 4096
18156 090/300 Write block 4128 len 20480 data: 5 6 6 7 7
18157 091/300 Write block 559 len 12288 data: 2 3 3
18158 092/300 Write block 1015 len 28672 data: 4 5 4 4 4 5 4
18159 093/300 Write block 1653 len 4096 data: 5
18160 094/300 Read block 1329 len 8192
18161 095/300 Write block 534 len 16384 data: 3 3 3 4
18162 096/300 Write block 115 len 24576 data: 4 4 4 4 2 3
18163 097/300 Read block 801 len 40960
18164 098/300 Write block 4459 len 32768 data: 7 4 5 7 7 7 8 7
18165 099/300 Write block 2724 len 24576 data: 3 3 2 3 3 3
18166 100/300 Read block 2837 len 8192
18167 101/300 Read block 4714 len 20480
18168 102/300 Read block 3647 len 8192
18169 103/300 Read block 4776 len 16384
18170 104/300 Write block 4954 len 28672 data: 3 3 3 3 3 3 2
18171 105/300 Flush
18172 106/300 Write block 799 len 24576 data: 4 6 4 4 5 6
18173 107/300 Read block 3169 len 36864
18174 108/300 Write block 4941 len 32768 data: 2 2 2 2 3 5 5 4
18175 109/300 Write block 4684 len 36864 data: 3 4 3 3 3 3 4 3 3
18176 110/300 Write block 3007 len 32768 data: 6 6 5 5 5 5 5 6
18177 111/300 Write block 3686 len 16384 data: 4 6 7 8
18178 112/300 Read block 398 len 12288
18179 113/300 Read block 1085 len 8192
18180 114/300 Read block 1084 len 40960
18181 115/300 Read block 1268 len 28672
18182 116/300 Read block 4895 len 40960
18183 117/300 Read block 1510 len 20480
18184 118/300 Read block 827 len 16384
18185 119/300 Write block 316 len 36864 data: 4 4 4 3 3 3 4 4 4
18186 120/300 Read block 3775 len 4096
18187 121/300 Read block 3495 len 12288
18188 122/300 Write block 4769 len 40960 data: 4 4 4 5 4 4 3 3 3 3
18189 123/300 Read block 1918 len 20480
18190 124/300 Flush
18191 125/300 Read block 2116 len 4096
18192 126/300 Write block 1375 len 24576 data: 4 4 4 4 4 4
18193 127/300 Read block 2632 len 24576
18194 128/300 Read block 251 len 32768
18195 129/300 Write block 4119 len 16384 data: 5 6 7 7
18196 130/300 Flush
18197 131/300 Write block 4418 len 20480 data: 3 4 4 3 3
18198 132/300 Write block 798 len 28672 data: 3 5 7 5 5 6 7
18199 133/300 Flush
18200 134/300 Write block 312 len 16384 data: 3 3 2 3
18201 135/300 Write block 4041 len 32768 data: 3 2 3 3 3 3 3 4
18202 136/300 Write block 3401 len 24576 data: 10 11 11 10 9 8
18203 137/300 Write block 973 len 32768 data: 3 3 4 4 5 6 6 6
18204 138/300 Read block 4728 len 24576
18205 139/300 Write block 3541 len 32768 data: 5 4 5 5 5 6 6 5
18206 140/300 Write block 2557 len 4096 data: 5
18207 141/300 Flush
18208 142/300 Write block 4508 len 12288 data: 2 3 5
18209 143/300 Read block 4828 len 12288
18210 144/300 Read block 1407 len 40960
18211 145/300 Write block 429 len 8192 data: 3 3
18212 146/300 Read block 708 len 40960
18213 147/300 Write block 4193 len 24576 data: 4 4 3 4 6 5
18214 148/300 Write block 3635 len 4096 data: 5
18215 149/300 Write block 2636 len 40960 data: 6 6 6 6 6 4 4 4 4 5
18216 150/300 Write block 3234 len 32768 data: 6 6 7 6 5 3 3 5
18217 151/300 Write block 735 len 12288 data: 4 4 4
18218 152/300 Read block 438 len 28672
18219 153/300 Write block 824 len 24576 data: 6 3 4 4 4 4
18220 154/300 Write block 92 len 20480 data: 5 4 4 4 3
18221 155/300 Read block 2162 len 24576
18222 156/300 Read block 2316 len 8192
18223 157/300 Write block 3582 len 36864 data: 3 3 4 4 4 4 4 5 3
18224 158/300 Read block 2634 len 4096
18225 159/300 Flush
18226 160/300 Read block 253 len 8192
18227 161/300 Read block 4599 len 32768
18228 162/300 Write block 1768 len 16384 data: 8 7 6 4
18229 163/300 Write block 318 len 4096 data: 5
18230 164/300 Flush
18231 165/300 Write block 3269 len 28672 data: 3 3 3 3 3 3 2
18232 166/300 Read block 722 len 24576
18233 167/300 Read block 1467 len 16384
18234 168/300 Flush
18235 169/300 Read block 3160 len 12288
18236 170/300 Read block 3191 len 4096
18237 171/300 Read block 4222 len 20480
18238 172/300 Read block 2200 len 8192
18239 173/300 Write block 754 len 16384 data: 4 3 3 3
18240 174/300 Read block 910 len 8192
18241 175/300 Read block 2045 len 32768
18242 176/300 Read block 581 len 16384
18243 177/300 Write block 1341 len 20480 data: 5 2 4 5 6
18244 178/300 Write block 1304 len 16384 data: 3 4 3 2
18245 179/300 Read block 4527 len 36864
18246 180/300 Write block 3215 len 16384 data: 8 9 7 7
18247 181/300 Read block 243 len 36864
18248 182/300 Flush
18249 183/300 Read block 2444 len 28672
18250 184/300 Read block 1020 len 28672
18251 185/300 Flush
18252 186/300 Read block 4937 len 8192
18253 187/300 Write block 3206 len 24576 data: 3 3 3 5 5 5
18254 188/300 Write block 3263 len 4096 data: 2
18255 189/300 Write block 3106 len 16384 data: 4 4 5 6
18256 190/300 Write block 3474 len 36864 data: 5 7 8 8 8 8 9 7 6
18257 191/300 Write block 1423 len 36864 data: 6 7 7 6 5 4 3 3 4
18258 192/300 Write block 1477 len 8192 data: 6 8
18259 193/300 Write block 453 len 4096 data: 4
18260 194/300 Read block 472 len 8192
18261 195/300 Read block 1646 len 4096
18262 196/300 Write block 1338 len 32768 data: 5 5 5 6 3 5 6 7
18263 197/300 Write block 3710 len 32768 data: 2 2 2 2 4 6 8 9
18264 198/300 Write block 452 len 28672 data: 3 5 4 5 6 6 7
18265 199/300 Read block 4692 len 16384
18266 200/300 Read block 2711 len 24576
18267 201/300 Read block 48 len 24576
18268 202/300 Flush
18269 203/300 Write block 1746 len 24576 data: 8 8 9 8 5 5
18270 204/300 Write block 4900 len 4096 data: 4
18271 205/300 Write block 3158 len 4096 data: 5
18272 206/300 Read block 516 len 32768
18273 207/300 Read block 1562 len 36864
18274 208/300 Read block 687 len 24576
18275 209/300 Write block 3559 len 36864 data: 2 2 2 2 3 5 6 6 6
18276 210/300 Flush
18277 211/300 Read block 4903 len 24576
18278 212/300 Read block 2389 len 36864
18279 213/300 Write block 86 len 40960 data: 5 5 5 5 4 5 6 5 5 5
18280 214/300 Write block 1347 len 8192 data: 5 6
18281 215/300 Read block 2604 len 32768
18282 216/300 Write block 1360 len 4096 data: 5
18283 217/300 Write block 2555 len 32768 data: 4 4 6 6 5 4 4 4
18284 218/300 Read block 2300 len 40960
18285 219/300 Read block 2107 len 12288
18286 220/300 Write block 1661 len 24576 data: 4 4 5 3 3 2
18287 221/300 Read block 4181 len 8192
18288 222/300 Write block 2042 len 36864 data: 5 4 6 7 7 5 5 5 5
18289 223/300 Read block 186 len 8192
18290 224/300 Read block 2021 len 16384
18291 225/300 Write block 1326 len 12288 data: 5 5 3
18292 226/300 Read block 3810 len 24576
18293 227/300 Read block 4895 len 12288
18294 228/300 Flush
18295 229/300 Read block 3267 len 28672
18296 230/300 Read block 819 len 4096
18297 231/300 Write block 3138 len 24576 data: 5 4 6 6 6 5
18298 232/300 Write block 3151 len 20480 data: 6 6 8 7 6
18299 233/300 Read block 2252 len 4096
18300 234/300 Read block 1206 len 32768
18301 235/300 Read block 1852 len 4096
18302 236/300 Write block 4492 len 32768 data: 6 4 3 3 3 3 3 3
18303 237/300 Read block 4715 len 4096
18304 238/300 Read block 606 len 8192
18305 239/300 Read block 1875 len 4096
18306 240/300 Write block 1292 len 16384 data: 5 6 5 5
18307 241/300 Read block 4515 len 32768
18308 242/300 Flush
18309 243/300 Write block 2418 len 12288 data: 4 4 4
18310 244/300 Read block 885 len 24576
18311 245/300 Read block 245 len 40960
18312 246/300 Read block 503 len 24576
18313 247/300 Flush
18314 248/300 Write block 680 len 20480 data: 3 4 2 3 3
18315 249/300 Read block 192 len 40960
18316 250/300 Flush
18317 251/300 Write block 1241 len 32768 data: 4 5 5 6 6 5 5 4
18318 252/300 Flush
18319 253/300 Read block 3449 len 32768
18320 254/300 Write block 2245 len 4096 data: 6
18321 255/300 Read block 112 len 28672
18322 256/300 Flush
18323 257/300 Flush
18324 258/300 Flush
18325 259/300 Read block 1155 len 40960
18326 260/300 Write block 3034 len 24576 data: 3 3 3 3 3 3
18327 261/300 Read block 636 len 12288
18328 262/300 Read block 1144 len 32768
18329 263/300 Write block 252 len 8192 data: 8 7
18330 264/300 Write block 2189 len 28672 data: 4 5 6 6 6 5 5
18331 265/300 Write block 3104 len 8192 data: 4 5
18332 266/300 Read block 624 len 40960
18333 267/300 Write block 3680 len 36864 data: 6 6 6 5 4 4 5 7 8
18334 268/300 Read block 4814 len 28672
18335 269/300 Read block 1349 len 32768
18336 270/300 Flush
18337 271/300 Flush
18338 272/300 Write block 2973 len 24576 data: 4 2 2 2 2 3
18339 273/300 Write block 3514 len 32768 data: 5 5 5 6 5 5 3 4
18340 274/300 Read block 4502 len 28672
18341 275/300 Read block 1254 len 20480
18342 276/300 Flush
18343 277/300 Write block 1376 len 36864 data: 5 5 5 5 5 3 3 4 4
18344 278/300 Read block 3825 len 28672
18345 279/300 Write block 62 len 32768 data: 4 3 4 6 6 6 4 4
18346 280/300 Read block 2074 len 36864
18347 281/300 Flush
18348 282/300 Write block 2536 len 36864 data: 3 6 5 5 6 5 4 4 4
18349 283/300 Write block 2744 len 8192 data: 3 3
18350 284/300 Read block 3824 len 36864
18351 285/300 Read block 894 len 4096
18352 286/300 Write block 3402 len 40960 data: 12 12 11 10 9 7 5 5 3 3
18353 287/300 Read block 2820 len 36864
18354 288/300 Read block 4781 len 40960
18355 289/300 Write block 3294 len 16384 data: 5 6 8 8
18356 290/300 Read block 2267 len 24576
18357 291/300 Read block 2179 len 4096
18358 292/300 Read block 90 len 12288
18359 293/300 Write block 1741 len 16384 data: 6 8 8 9
18360 294/300 Read block 3742 len 12288
18361 295/300 Write block 3515 len 24576 data: 6 6 7 6 6 4
18362 296/300 Write block 931 len 12288 data: 2 3 3
18363 297/300 Read block 4295 len 20480
18364 298/300 Flush
18365 299/300 Write block 4129 len 40960 data: 7 7 8 8 4 4 4 4 4 3
18366 300/300 Write block 4715 len 16384 data: 3 3 4 5
18367 301/300 Write block 2872 len 4096 data: 6
18368 Main task has work to do, go find it
18369 got dsc Start(1)
18370 start 1
18371 Dec 04 13:34:34.540 INFO request completed, latency_us: 233, response_code: 204, uri: /start/cid/1, method: POST, req_id: 627b11bb-17f4-4ba1-90ea-24af0c0b3099, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
18372 Replay: started 1, returned:Ok(())
18373 ----------------------------------------------------------------
18374 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18375 States: Active Offline Active
18376 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18377 6131 Acked Write 10 Done Sent Done false
18378 6132 Acked Flush 0 Done Sent Done false
18379 6133 Acked Write 7 Done Sent Done false
18380 6134 Acked Read 7 Done Sent Done false
18381 6135 Acked Read 3 Done Sent Done false
18382 6136 Acked Write 6 Done Sent Done false
18383 6137 Acked Read 10 Done Sent Done false
18384 6138 Acked Write 2 Done Sent Done false
18385 6139 Acked Flush 0 Done Sent Done false
18386 6140 Acked Write 7 Done Sent Done false
18387 6141 Acked Read 1 Done Sent Done false
18388 6142 Acked Read 4 Done Sent Done false
18389 6143 Acked Read 4 Done Sent Done false
18390 6144 Acked Write 6 Done Sent Done false
18391 6145 Acked Read 1 Done Sent Done false
18392 6146 Acked Read 7 Done Sent Done false
18393 6147 Acked Read 10 Done Sent Done false
18394 6148 Acked Read 8 Done Sent Done false
18395 6149 Acked Read 2 Done Sent Done false
18396 6150 Acked Flush 0 Done Sent Done false
18397 6151 Acked Write 2 Done Sent Done false
18398 6152 Acked Read 4 Done Sent Done false
18399 6153 Acked Read 4 Done Sent Done false
18400 6154 Acked Write 1 Done Sent Done false
18401 6155 Acked Read 3 Done Sent Done false
18402 6156 Acked Read 6 Done Sent Done false
18403 6157 Acked Flush 0 Done Sent Done false
18404 6158 Acked Flush 0 Done Sent Done false
18405 6159 Acked Read 9 Done Sent Done false
18406 6160 Acked Read 4 Done Sent Done false
18407 6161 Acked Write 5 Done Sent Done false
18408 6162 Acked Write 8 Done Sent Done false
18409 6163 Acked Flush 0 Done Sent Done false
18410 6164 Acked Read 4 Done Sent Done false
18411 6165 Acked Read 4 Done Sent Done false
18412 6166 Acked Write 3 Done Sent Done false
18413 6167 Acked Write 8 Done Sent Done false
18414 6168 Acked Read 8 Done Sent Done false
18415 6169 Acked Write 1 Done Sent Done false
18416 6170 Acked Write 10 Done Sent Done false
18417 6171 Acked Write 1 Done Sent Done false
18418 6172 Acked Write 7 Done Sent Done false
18419 6173 Acked Write 5 Done Sent Done false
18420 6174 Acked Read 8 Done Sent Done false
18421 6175 Acked Read 5 Done Sent Done false
18422 6176 Acked Read 8 Done Sent Done false
18423 6177 Acked Read 2 Done Sent Done false
18424 6178 Acked Read 7 Done Sent Done false
18425 6179 Acked Read 5 Done Sent Done false
18426 6180 Acked Write 6 Done Sent Done false
18427 6181 Acked Read 4 Done Sent Done false
18428 6182 Acked Write 5 Done Sent Done false
18429 6183 Acked Flush 0 Done Sent Done false
18430 6184 Acked Read 6 Done Sent Done false
18431 6185 Acked Write 3 Done Sent Done false
18432 6186 Acked Read 4 Done Sent Done false
18433 6187 Acked Flush 0 Done Sent Done false
18434 6188 Acked Read 10 Done Sent Done false
18435 6189 Acked Write 10 Done Sent Done false
18436 6190 Acked Read 2 Done Sent Done false
18437 6191 Acked Read 3 Done Sent Done false
18438 6192 Acked Read 6 Done Sent Done false
18439 6193 Acked Read 8 Done Sent Done false
18440 6194 Acked Read 5 Done Sent Done false
18441 6195 Acked Read 6 Done Sent Done false
18442 6196 Acked Write 8 Done Sent Done false
18443 6197 Acked Read 1 Done Sent Done false
18444 6198 Acked Write 3 Done Sent Done false
18445 6199 Acked Read 4 Done Sent Done false
18446 6200 Acked Read 6 Done Sent Done false
18447 6201 Acked Read 7 Done Sent Done false
18448 6202 Acked Read 7 Done Sent Done false
18449 6203 Acked Write 5 Done Sent Done false
18450 6204 Acked Flush 0 Done Sent Done false
18451 6205 Acked Read 6 Done Sent Done false
18452 6206 Acked Read 7 Done Sent Done false
18453 6207 Acked Read 4 Done Sent Done false
18454 6208 Acked Read 1 Done Sent Done false
18455 6209 Acked Read 2 Done Sent Done false
18456 6210 Acked Write 10 Done Sent Done false
18457 6211 Acked Read 3 Done Sent Done false
18458 6212 Acked Write 1 Done Sent Done false
18459 6213 Acked Read 5 Done Sent Done false
18460 6214 Acked Read 6 Done Sent Done false
18461 6215 Acked Read 2 Done Sent Done false
18462 6216 Acked Write 1 Done Sent Done false
18463 6217 Acked Read 1 Done Sent Done false
18464 6218 Acked Read 3 Done Sent Done false
18465 6219 Acked Read 1 Done Sent Done false
18466 6220 Acked Write 5 Done Sent Done false
18467 6221 Acked Write 3 Done Sent Done false
18468 6222 Acked Write 7 Done Sent Done false
18469 6223 Acked Write 1 Done Sent Done false
18470 6224 Acked Read 2 Done Sent Done false
18471 6225 Acked Write 4 Done Sent Done false
18472 6226 Acked Write 6 Done Sent Done false
18473 6227 Acked Read 10 Done Sent Done false
18474 6228 Acked Write 8 Done Sent Done false
18475 6229 Acked Write 6 Done Sent Done false
18476 6230 Acked Read 2 Done Sent Done false
18477 6231 Acked Read 5 Done Sent Done false
18478 6232 Acked Read 2 Done Sent Done false
18479 6233 Acked Read 4 Done Sent Done false
18480 6234 Acked Write 7 Done Sent Done false
18481 6235 Acked Flush 0 Done Sent Done false
18482 6236 Acked Write 6 Done Sent Done false
18483 6237 Acked Read 9 Done Sent Done false
18484 6238 Acked Write 8 Done Sent Done false
18485 6239 Acked Write 9 Done Sent Done false
18486 6240 Acked Write 8 Done Sent Done false
18487 6241 Acked Write 4 Done Sent Done false
18488 6242 Acked Read 3 Done Sent Done false
18489 6243 Acked Read 2 Done Sent Done false
18490 6244 Acked Read 10 Done Sent Done false
18491 6245 Acked Read 7 Done Sent Done false
18492 6246 Acked Read 10 Done Sent Done false
18493 6247 Acked Read 5 Done Sent Done false
18494 6248 Acked Read 4 Done Sent Done false
18495 6249 Acked Write 9 Done Sent Done false
18496 6250 Acked Read 1 Done Sent Done false
18497 6251 Acked Read 3 Done Sent Done false
18498 6252 Acked Write 10 Done Sent Done false
18499 6253 Acked Read 5 Done Sent Done false
18500 6254 Acked Flush 0 Done Sent Done false
18501 6255 Acked Read 1 Done Sent Done false
18502 6256 Acked Write 6 Done Sent Done false
18503 6257 Acked Read 6 Done Sent Done false
18504 6258 Acked Read 8 Done Sent Done false
18505 6259 Acked Write 4 Done Sent Done false
18506 6260 Acked Flush 0 Done Sent Done false
18507 6261 Acked Write 5 Done Sent Done false
18508 6262 Acked Write 7 Done Sent Done false
18509 6263 Acked Flush 0 Done Sent Done false
18510 6264 Acked Write 4 Done Sent Done false
18511 6265 Acked Write 8 Done Sent Done false
18512 6266 Acked Write 6 Done Sent Done false
18513 6267 Acked Write 8 Done Sent Done false
18514 6268 Acked Read 6 Done Sent Done false
18515 6269 Acked Write 8 Done Sent Done false
18516 6270 Acked Write 1 Done Sent Done false
18517 6271 Acked Flush 0 Done Sent Done false
18518 6272 Acked Write 3 Done Sent Done false
18519 6273 Acked Read 3 Done Sent Done false
18520 6274 Acked Read 10 Done Sent Done false
18521 6275 Acked Write 2 Done Sent Done false
18522 6276 Acked Read 10 Done Sent Done false
18523 6277 Acked Write 6 Done Sent Done false
18524 6278 Acked Write 1 Done Sent Done false
18525 6279 Acked Write 10 Done Sent Done false
18526 6280 Acked Write 8 Done Sent Done false
18527 6281 Acked Write 3 Done Sent Done false
18528 6282 Acked Read 7 Done Sent Done false
18529 6283 Acked Write 6 Done Sent Done false
18530 6284 Acked Write 5 Done Sent Done false
18531 6285 Acked Read 6 Done Sent Done false
18532 6286 Acked Read 2 Done Sent Done false
18533 6287 Acked Write 9 Done Sent Done false
18534 6288 Acked Read 1 Done Sent Done false
18535 6289 Acked Flush 0 Done Sent Done false
18536 6290 Acked Read 2 Done Sent Done false
18537 6291 Acked Read 8 Done Sent Done false
18538 6292 Acked Write 4 Done Sent Done false
18539 6293 Acked Write 1 Done Sent Done false
18540 6294 Acked Flush 0 Done Sent Done false
18541 6295 Acked Write 7 Done Sent Done false
18542 6296 Acked Read 6 Done Sent Done false
18543 6297 Acked Read 4 Done Sent Done false
18544 6298 Acked Flush 0 Done Sent Done false
18545 6299 Acked Read 3 Done Sent Done false
18546 6300 Acked Read 1 Done Sent Done false
18547 6301 Acked Read 5 Done Sent Done false
18548 6302 Acked Read 2 Done Sent Done false
18549 6303 Acked Write 4 Done Sent Done false
18550 6304 Acked Read 2 Done Sent Done false
18551 6305 Acked Read 8 Done Sent Done false
18552 6306 Acked Read 4 Done Sent Done false
18553 6307 Acked Write 5 Done Sent Done false
18554 6308 Acked Write 4 Done Sent Done false
18555 6309 Acked Read 9 Done Sent Done false
18556 6310 Acked Write 4 Done Sent Done false
18557 6311 Acked Read 9 Done Sent Done false
18558 6312 Acked Flush 0 Done Sent Done false
18559 6313 Acked Read 7 Done Sent Done false
18560 6314 Acked Read 7 Done Sent Done false
18561 6315 Acked Flush 0 Done Sent Done false
18562 6316 Acked Read 2 Done Sent Done false
18563 6317 Acked Write 6 Done Sent Done false
18564 6318 Acked Write 1 Done Sent Done false
18565 6319 Acked Write 4 Done Sent Done false
18566 6320 Acked Write 9 Done Sent Done false
18567 6321 Acked Write 9 Done Sent Done false
18568 6322 Acked Write 2 Done Sent Done false
18569 6323 Acked Write 1 Done Sent Done false
18570 6324 Acked Read 2 Done Sent Done false
18571 6325 Acked Read 1 Done Sent Done false
18572 6326 Acked Write 8 Done Sent Done false
18573 6327 Acked Write 8 Done Sent Done false
18574 6328 Acked Write 7 Done Sent Done false
18575 6329 Acked Read 4 Done Sent Done false
18576 6330 Acked Read 6 Done Sent Done false
18577 6331 Acked Read 6 Done Sent Done false
18578 6332 Acked Flush 0 Done Sent Done false
18579 6333 Acked Write 6 Done Sent Done false
18580 6334 Acked Write 1 Done Sent Done false
18581 6335 Acked Write 1 Done Sent Done false
18582 6336 Acked Read 8 Done Sent Done false
18583 6337 Acked Read 9 Done Sent Done false
18584 6338 Acked Read 6 Done Sent Done false
18585 6339 Acked Write 9 Done Sent Done false
18586 6340 Acked Flush 0 Done Sent Done false
18587 6341 Acked Read 6 Done Sent Done false
18588 6342 Acked Read 9 Done Sent Done false
18589 6343 Acked Write 10 Done Sent Done false
18590 6344 Acked Write 2 Done Sent Done false
18591 6345 Acked Read 8 Done Sent Done false
18592 6346 Acked Write 1 Done Sent Done false
18593 6347 Acked Write 8 Done Sent Done false
18594 6348 Acked Read 10 Done Sent Done false
18595 6349 Acked Read 3 Done Sent Done false
18596 6350 Acked Write 6 Done Sent Done false
18597 6351 Acked Read 2 Done Sent Done false
18598 6352 Acked Write 9 Done Sent Done false
18599 6353 Acked Read 2 Done Sent Done false
18600 6354 Acked Read 4 Done Sent Done false
18601 6355 Acked Write 3 Done Sent Done false
18602 6356 Acked Read 6 Done Sent Done false
18603 6357 Acked Read 3 Done Sent Done false
18604 6358 Acked Flush 0 Done Sent Done false
18605 6359 Acked Read 7 Done Sent Done false
18606 6360 Acked Read 1 Done Sent Done false
18607 6361 Acked Write 6 Done Sent Done false
18608 6362 Acked Write 5 Done Sent Done false
18609 6363 Acked Read 1 Done Sent Done false
18610 6364 Acked Read 8 Done Sent Done false
18611 6365 Acked Read 1 Done Sent Done false
18612 6366 Acked Write 8 Done Sent Done false
18613 6367 Acked Read 1 Done Sent Done false
18614 6368 Acked Read 2 Done Sent Done false
18615 6369 Acked Read 1 Done Sent Done false
18616 6370 Acked Write 4 Done Sent Done false
18617 6371 Acked Read 8 Done Sent Done false
18618 6372 Acked Flush 0 Done Sent Done false
18619 6373 Acked Write 3 Done Sent Done false
18620 6374 Acked Read 6 Done Sent Done false
18621 6375 Acked Read 10 Done Sent Done false
18622 6376 Acked Read 6 Done Sent Done false
18623 6377 Acked Flush 0 Done Sent Done false
18624 6378 Acked Write 5 Done Sent Done false
18625 6379 Acked Read 10 Done Sent Done false
18626 6380 Acked Flush 0 Done Sent Done false
18627 6381 Acked Write 8 Done Sent Done false
18628 6382 Acked Flush 0 Done Sent Done false
18629 6383 Acked Read 8 Done Sent Done false
18630 6384 Acked Write 1 Done Sent Done false
18631 6385 Acked Read 7 Done Sent Done false
18632 6386 Acked Flush 0 Done Sent Done false
18633 6387 Acked Flush 0 Done Sent Done false
18634 6388 Acked Flush 0 Done Sent Done false
18635 6389 Acked Read 10 Done Sent Done false
18636 6390 Acked Write 6 Done Sent Done false
18637 6391 Acked Read 3 Done Sent Done false
18638 6392 Acked Read 8 Done Sent Done false
18639 6393 Acked Write 2 Done Sent Done false
18640 6394 Acked Write 7 Done Sent Done false
18641 6395 Acked Write 2 Done Sent Done false
18642 6396 Acked Read 10 Done Sent Done false
18643 6397 Acked Write 9 Done Sent Done false
18644 6398 Acked Read 7 Done Sent Done false
18645 6399 Acked Read 8 Done Sent Done false
18646 6400 Acked Flush 0 Done Sent Done false
18647 6401 Acked Flush 0 Done Sent Done false
18648 6402 Acked Write 6 Done Sent Done false
18649 6403 Acked Write 8 Done Sent Done false
18650 6404 Acked Read 7 Done Sent Done false
18651 6405 Acked Read 5 Done Sent Done false
18652 6406 Acked Flush 0 Done Sent Done false
18653 6407 Acked Write 9 Done Sent Done false
18654 6408 Acked Read 7 Done Sent Done false
18655 6409 Acked Write 8 Done Sent Done false
18656 6410 Acked Read 9 Done Sent Done false
18657 6411 Acked Flush 0 Done Sent Done false
18658 6412 Acked Write 9 Done Sent Done false
18659 6413 Acked Write 2 Done Sent Done false
18660 6414 Acked Read 9 Done Sent Done false
18661 6415 Acked Read 1 Done Sent Done false
18662 6416 Acked Write 10 Done Sent Done false
18663 6417 Acked Read 9 Done Sent Done false
18664 6418 Acked Read 10 Done Sent Done false
18665 6419 Acked Write 4 Done Sent Done false
18666 6420 Acked Read 6 Done Sent Done false
18667 6421 Acked Read 1 Done Sent Done false
18668 6422 Acked Read 3 Done Sent Done false
18669 6423 Acked Write 4 Done Sent Done false
18670 6424 Acked Read 3 Done Sent Done false
18671 6425 Acked Write 6 Done Sent Done false
18672 6426 Acked Write 3 Done Sent Done false
18673 6427 Acked Read 5 Done Sent Done false
18674 6428 Acked Flush 0 Done Sent Done false
18675 6429 Acked Write 10 Done Sent Done false
18676 6430 Acked Write 4 Done Sent Done false
18677 6431 Acked Write 1 Done Sent Done false
18678 STATES DS:0 DS:1 DS:2 TOTAL
18679 Sent 0 301 0 301
18680 Done 301 0 301 602
18681 Skipped 0 0 0 0
18682 Error 0 0 0 0
18683 Last Flush: 6428 6130 6428
18684 Downstairs last five completed: 6130 6129 6128 6127 6126
18685 Upstairs last five completed: 6431 6430 6429 6428 6427
18686 CLIENT: Up:0 ds:301 act:2
18687 ----------------------------------------------------------------
18688 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18689 States: Active Offline Active
18690 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18691 6131 Acked Write 10 Done Sent Done false
18692 6132 Acked Flush 0 Done Sent Done false
18693 6133 Acked Write 7 Done Sent Done false
18694 6134 Acked Read 7 Done Sent Done false
18695 6135 Acked Read 3 Done Sent Done false
18696 6136 Acked Write 6 Done Sent Done false
18697 6137 Acked Read 10 Done Sent Done false
18698 6138 Acked Write 2 Done Sent Done false
18699 6139 Acked Flush 0 Done Sent Done false
18700 6140 Acked Write 7 Done Sent Done false
18701 6141 Acked Read 1 Done Sent Done false
18702 6142 Acked Read 4 Done Sent Done false
18703 6143 Acked Read 4 Done Sent Done false
18704 6144 Acked Write 6 Done Sent Done false
18705 6145 Acked Read 1 Done Sent Done false
18706 6146 Acked Read 7 Done Sent Done false
18707 6147 Acked Read 10 Done Sent Done false
18708 6148 Acked Read 8 Done Sent Done false
18709 6149 Acked Read 2 Done Sent Done false
18710 6150 Acked Flush 0 Done Sent Done false
18711 6151 Acked Write 2 Done Sent Done false
18712 6152 Acked Read 4 Done Sent Done false
18713 6153 Acked Read 4 Done Sent Done false
18714 6154 Acked Write 1 Done Sent Done false
18715 6155 Acked Read 3 Done Sent Done false
18716 6156 Acked Read 6 Done Sent Done false
18717 6157 Acked Flush 0 Done Sent Done false
18718 6158 Acked Flush 0 Done Sent Done false
18719 6159 Acked Read 9 Done Sent Done false
18720 6160 Acked Read 4 Done Sent Done false
18721 6161 Acked Write 5 Done Sent Done false
18722 6162 Acked Write 8 Done Sent Done false
18723 6163 Acked Flush 0 Done Sent Done false
18724 6164 Acked Read 4 Done Sent Done false
18725 6165 Acked Read 4 Done Sent Done false
18726 6166 Acked Write 3 Done Sent Done false
18727 6167 Acked Write 8 Done Sent Done false
18728 6168 Acked Read 8 Done Sent Done false
18729 6169 Acked Write 1 Done Sent Done false
18730 6170 Acked Write 10 Done Sent Done false
18731 6171 Acked Write 1 Done Sent Done false
18732 6172 Acked Write 7 Done Sent Done false
18733 6173 Acked Write 5 Done Sent Done false
18734 6174 Acked Read 8 Done Sent Done false
18735 6175 Acked Read 5 Done Sent Done false
18736 6176 Acked Read 8 Done Sent Done false
18737 6177 Acked Read 2 Done Sent Done false
18738 6178 Acked Read 7 Done Sent Done false
18739 6179 Acked Read 5 Done Sent Done false
18740 6180 Acked Write 6 Done Sent Done false
18741 6181 Acked Read 4 Done Sent Done false
18742 6182 Acked Write 5 Done Sent Done false
18743 6183 Acked Flush 0 Done Sent Done false
18744 6184 Acked Read 6 Done Sent Done false
18745 6185 Acked Write 3 Done Sent Done false
18746 6186 Acked Read 4 Done Sent Done false
18747 6187 Acked Flush 0 Done Sent Done false
18748 6188 Acked Read 10 Done Sent Done false
18749 6189 Acked Write 10 Done Sent Done false
18750 6190 Acked Read 2 Done Sent Done false
18751 6191 Acked Read 3 Done Sent Done false
18752 6192 Acked Read 6 Done Sent Done false
18753 6193 Acked Read 8 Done Sent Done false
18754 6194 Acked Read 5 Done Sent Done false
18755 6195 Acked Read 6 Done Sent Done false
18756 6196 Acked Write 8 Done Sent Done false
18757 6197 Acked Read 1 Done Sent Done false
18758 6198 Acked Write 3 Done Sent Done false
18759 6199 Acked Read 4 Done Sent Done false
18760 6200 Acked Read 6 Done Sent Done false
18761 6201 Acked Read 7 Done Sent Done false
18762 6202 Acked Read 7 Done Sent Done false
18763 6203 Acked Write 5 Done Sent Done false
18764 6204 Acked Flush 0 Done Sent Done false
18765 6205 Acked Read 6 Done Sent Done false
18766 6206 Acked Read 7 Done Sent Done false
18767 6207 Acked Read 4 Done Sent Done false
18768 6208 Acked Read 1 Done Sent Done false
18769 6209 Acked Read 2 Done Sent Done false
18770 6210 Acked Write 10 Done Sent Done false
18771 6211 Acked Read 3 Done Sent Done false
18772 6212 Acked Write 1 Done Sent Done false
18773 6213 Acked Read 5 Done Sent Done false
18774 6214 Acked Read 6 Done Sent Done false
18775 6215 Acked Read 2 Done Sent Done false
18776 6216 Acked Write 1 Done Sent Done false
18777 6217 Acked Read 1 Done Sent Done false
18778 6218 Acked Read 3 Done Sent Done false
18779 6219 Acked Read 1 Done Sent Done false
18780 6220 Acked Write 5 Done Sent Done false
18781 6221 Acked Write 3 Done Sent Done false
18782 6222 Acked Write 7 Done Sent Done false
18783 6223 Acked Write 1 Done Sent Done false
18784 6224 Acked Read 2 Done Sent Done false
18785 6225 Acked Write 4 Done Sent Done false
18786 6226 Acked Write 6 Done Sent Done false
18787 6227 Acked Read 10 Done Sent Done false
18788 6228 Acked Write 8 Done Sent Done false
18789 6229 Acked Write 6 Done Sent Done false
18790 6230 Acked Read 2 Done Sent Done false
18791 6231 Acked Read 5 Done Sent Done false
18792 6232 Acked Read 2 Done Sent Done false
18793 6233 Acked Read 4 Done Sent Done false
18794 6234 Acked Write 7 Done Sent Done false
18795 6235 Acked Flush 0 Done Sent Done false
18796 6236 Acked Write 6 Done Sent Done false
18797 6237 Acked Read 9 Done Sent Done false
18798 6238 Acked Write 8 Done Sent Done false
18799 6239 Acked Write 9 Done Sent Done false
18800 6240 Acked Write 8 Done Sent Done false
18801 6241 Acked Write 4 Done Sent Done false
18802 6242 Acked Read 3 Done Sent Done false
18803 6243 Acked Read 2 Done Sent Done false
18804 6244 Acked Read 10 Done Sent Done false
18805 6245 Acked Read 7 Done Sent Done false
18806 6246 Acked Read 10 Done Sent Done false
18807 6247 Acked Read 5 Done Sent Done false
18808 6248 Acked Read 4 Done Sent Done false
18809 6249 Acked Write 9 Done Sent Done false
18810 6250 Acked Read 1 Done Sent Done false
18811 6251 Acked Read 3 Done Sent Done false
18812 6252 Acked Write 10 Done Sent Done false
18813 6253 Acked Read 5 Done Sent Done false
18814 6254 Acked Flush 0 Done Sent Done false
18815 6255 Acked Read 1 Done Sent Done false
18816 6256 Acked Write 6 Done Sent Done false
18817 6257 Acked Read 6 Done Sent Done false
18818 6258 Acked Read 8 Done Sent Done false
18819 6259 Acked Write 4 Done Sent Done false
18820 6260 Acked Flush 0 Done Sent Done false
18821 6261 Acked Write 5 Done Sent Done false
18822 6262 Acked Write 7 Done Sent Done false
18823 6263 Acked Flush 0 Done Sent Done false
18824 6264 Acked Write 4 Done Sent Done false
18825 6265 Acked Write 8 Done Sent Done false
18826 6266 Acked Write 6 Done Sent Done false
18827 6267 Acked Write 8 Done Sent Done false
18828 6268 Acked Read 6 Done Sent Done false
18829 6269 Acked Write 8 Done Sent Done false
18830 6270 Acked Write 1 Done Sent Done false
18831 6271 Acked Flush 0 Done Sent Done false
18832 6272 Acked Write 3 Done Sent Done false
18833 6273 Acked Read 3 Done Sent Done false
18834 6274 Acked Read 10 Done Sent Done false
18835 6275 Acked Write 2 Done Sent Done false
18836 6276 Acked Read 10 Done Sent Done false
18837 6277 Acked Write 6 Done Sent Done false
18838 6278 Acked Write 1 Done Sent Done false
18839 6279 Acked Write 10 Done Sent Done false
18840 6280 Acked Write 8 Done Sent Done false
18841 6281 Acked Write 3 Done Sent Done false
18842 6282 Acked Read 7 Done Sent Done false
18843 6283 Acked Write 6 Done Sent Done false
18844 6284 Acked Write 5 Done Sent Done false
18845 6285 Acked Read 6 Done Sent Done false
18846 6286 Acked Read 2 Done Sent Done false
18847 6287 Acked Write 9 Done Sent Done false
18848 6288 Acked Read 1 Done Sent Done false
18849 6289 Acked Flush 0 Done Sent Done false
18850 6290 Acked Read 2 Done Sent Done false
18851 6291 Acked Read 8 Done Sent Done false
18852 6292 Acked Write 4 Done Sent Done false
18853 6293 Acked Write 1 Done Sent Done false
18854 6294 Acked Flush 0 Done Sent Done false
18855 6295 Acked Write 7 Done Sent Done false
18856 6296 Acked Read 6 Done Sent Done false
18857 6297 Acked Read 4 Done Sent Done false
18858 6298 Acked Flush 0 Done Sent Done false
18859 6299 Acked Read 3 Done Sent Done false
18860 6300 Acked Read 1 Done Sent Done false
18861 6301 Acked Read 5 Done Sent Done false
18862 6302 Acked Read 2 Done Sent Done false
18863 6303 Acked Write 4 Done Sent Done false
18864 6304 Acked Read 2 Done Sent Done false
18865 6305 Acked Read 8 Done Sent Done false
18866 6306 Acked Read 4 Done Sent Done false
18867 6307 Acked Write 5 Done Sent Done false
18868 6308 Acked Write 4 Done Sent Done false
18869 6309 Acked Read 9 Done Sent Done false
18870 6310 Acked Write 4 Done Sent Done false
18871 6311 Acked Read 9 Done Sent Done false
18872 6312 Acked Flush 0 Done Sent Done false
18873 6313 Acked Read 7 Done Sent Done false
18874 6314 Acked Read 7 Done Sent Done false
18875 6315 Acked Flush 0 Done Sent Done false
18876 6316 Acked Read 2 Done Sent Done false
18877 6317 Acked Write 6 Done Sent Done false
18878 6318 Acked Write 1 Done Sent Done false
18879 6319 Acked Write 4 Done Sent Done false
18880 6320 Acked Write 9 Done Sent Done false
18881 6321 Acked Write 9 Done Sent Done false
18882 6322 Acked Write 2 Done Sent Done false
18883 6323 Acked Write 1 Done Sent Done false
18884 6324 Acked Read 2 Done Sent Done false
18885 6325 Acked Read 1 Done Sent Done false
18886 6326 Acked Write 8 Done Sent Done false
18887 6327 Acked Write 8 Done Sent Done false
18888 6328 Acked Write 7 Done Sent Done false
18889 6329 Acked Read 4 Done Sent Done false
18890 6330 Acked Read 6 Done Sent Done false
18891 6331 Acked Read 6 Done Sent Done false
18892 6332 Acked Flush 0 Done Sent Done false
18893 6333 Acked Write 6 Done Sent Done false
18894 6334 Acked Write 1 Done Sent Done false
18895 6335 Acked Write 1 Done Sent Done false
18896 6336 Acked Read 8 Done Sent Done false
18897 6337 Acked Read 9 Done Sent Done false
18898 6338 Acked Read 6 Done Sent Done false
18899 6339 Acked Write 9 Done Sent Done false
18900 6340 Acked Flush 0 Done Sent Done false
18901 6341 Acked Read 6 Done Sent Done false
18902 6342 Acked Read 9 Done Sent Done false
18903 6343 Acked Write 10 Done Sent Done false
18904 6344 Acked Write 2 Done Sent Done false
18905 6345 Acked Read 8 Done Sent Done false
18906 6346 Acked Write 1 Done Sent Done false
18907 6347 Acked Write 8 Done Sent Done false
18908 6348 Acked Read 10 Done Sent Done false
18909 6349 Acked Read 3 Done Sent Done false
18910 6350 Acked Write 6 Done Sent Done false
18911 6351 Acked Read 2 Done Sent Done false
18912 6352 Acked Write 9 Done Sent Done false
18913 6353 Acked Read 2 Done Sent Done false
18914 6354 Acked Read 4 Done Sent Done false
18915 6355 Acked Write 3 Done Sent Done false
18916 6356 Acked Read 6 Done Sent Done false
18917 6357 Acked Read 3 Done Sent Done false
18918 6358 Acked Flush 0 Done Sent Done false
18919 6359 Acked Read 7 Done Sent Done false
18920 6360 Acked Read 1 Done Sent Done false
18921 6361 Acked Write 6 Done Sent Done false
18922 6362 Acked Write 5 Done Sent Done false
18923 6363 Acked Read 1 Done Sent Done false
18924 6364 Acked Read 8 Done Sent Done false
18925 6365 Acked Read 1 Done Sent Done false
18926 6366 Acked Write 8 Done Sent Done false
18927 6367 Acked Read 1 Done Sent Done false
18928 6368 Acked Read 2 Done Sent Done false
18929 6369 Acked Read 1 Done Sent Done false
18930 6370 Acked Write 4 Done Sent Done false
18931 6371 Acked Read 8 Done Sent Done false
18932 6372 Acked Flush 0 Done Sent Done false
18933 6373 Acked Write 3 Done Sent Done false
18934 6374 Acked Read 6 Done Sent Done false
18935 6375 Acked Read 10 Done Sent Done false
18936 6376 Acked Read 6 Done Sent Done false
18937 6377 Acked Flush 0 Done Sent Done false
18938 6378 Acked Write 5 Done Sent Done false
18939 6379 Acked Read 10 Done Sent Done false
18940 6380 Acked Flush 0 Done Sent Done false
18941 6381 Acked Write 8 Done Sent Done false
18942 6382 Acked Flush 0 Done Sent Done false
18943 6383 Acked Read 8 Done Sent Done false
18944 6384 Acked Write 1 Done Sent Done false
18945 6385 Acked Read 7 Done Sent Done false
18946 6386 Acked Flush 0 Done Sent Done false
18947 6387 Acked Flush 0 Done Sent Done false
18948 6388 Acked Flush 0 Done Sent Done false
18949 6389 Acked Read 10 Done Sent Done false
18950 6390 Acked Write 6 Done Sent Done false
18951 6391 Acked Read 3 Done Sent Done false
18952 6392 Acked Read 8 Done Sent Done false
18953 6393 Acked Write 2 Done Sent Done false
18954 6394 Acked Write 7 Done Sent Done false
18955 6395 Acked Write 2 Done Sent Done false
18956 6396 Acked Read 10 Done Sent Done false
18957 6397 Acked Write 9 Done Sent Done false
18958 6398 Acked Read 7 Done Sent Done false
18959 6399 Acked Read 8 Done Sent Done false
18960 6400 Acked Flush 0 Done Sent Done false
18961 6401 Acked Flush 0 Done Sent Done false
18962 6402 Acked Write 6 Done Sent Done false
18963 6403 Acked Write 8 Done Sent Done false
18964 6404 Acked Read 7 Done Sent Done false
18965 6405 Acked Read 5 Done Sent Done false
18966 6406 Acked Flush 0 Done Sent Done false
18967 6407 Acked Write 9 Done Sent Done false
18968 6408 Acked Read 7 Done Sent Done false
18969 6409 Acked Write 8 Done Sent Done false
18970 6410 Acked Read 9 Done Sent Done false
18971 6411 Acked Flush 0 Done Sent Done false
18972 6412 Acked Write 9 Done Sent Done false
18973 6413 Acked Write 2 Done Sent Done false
18974 6414 Acked Read 9 Done Sent Done false
18975 6415 Acked Read 1 Done Sent Done false
18976 6416 Acked Write 10 Done Sent Done false
18977 6417 Acked Read 9 Done Sent Done false
18978 6418 Acked Read 10 Done Sent Done false
18979 6419 Acked Write 4 Done Sent Done false
18980 6420 Acked Read 6 Done Sent Done false
18981 6421 Acked Read 1 Done Sent Done false
18982 6422 Acked Read 3 Done Sent Done false
18983 6423 Acked Write 4 Done Sent Done false
18984 6424 Acked Read 3 Done Sent Done false
18985 6425 Acked Write 6 Done Sent Done false
18986 6426 Acked Write 3 Done Sent Done false
18987 6427 Acked Read 5 Done Sent Done false
18988 6428 Acked Flush 0 Done Sent Done false
18989 6429 Acked Write 10 Done Sent Done false
18990 6430 Acked Write 4 Done Sent Done false
18991 6431 Acked Write 1 Done Sent Done false
18992 6432 Acked Flush 0 Done Sent Done false
18993 STATES DS:0 DS:1 DS:2 TOTAL
18994 Sent 0 302 0 302
18995 Done 302 0 302 604
18996 Skipped 0 0 0 0
18997 Error 0 0 0 0
18998 Last Flush: 6432 6130 6432
18999 Downstairs last five completed: 6130 6129 6128 6127 6126
19000 Upstairs last five completed: 6432 6431 6430 6429 6428
19001 CLIENT: Up:0 ds:302 act:2
190022024-12-04T13:34:40.308ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
190032024-12-04T13:34:40.308ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
190042024-12-04T13:34:40.308ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
190052024-12-04T13:34:40.308ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
190062024-12-04T13:34:40.308ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
19007 [8820] got start action
19008 [8820] I am going to restart
19009 Starting downstairs at port 8820
19010 Make output file at "/tmp/dsc/downstairs-8820.txt"
19011 [8820][1] reports Starting
19012 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1221)
19013 [8820][1] reports Running
19014 ----------------------------------------------------------------
19015 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19016 States: Active Offline Active
19017 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19018 6131 Acked Write 10 Done Sent Done false
19019 6132 Acked Flush 0 Done Sent Done false
19020 6133 Acked Write 7 Done Sent Done false
19021 6134 Acked Read 7 Done Sent Done false
19022 6135 Acked Read 3 Done Sent Done false
19023 6136 Acked Write 6 Done Sent Done false
19024 6137 Acked Read 10 Done Sent Done false
19025 6138 Acked Write 2 Done Sent Done false
19026 6139 Acked Flush 0 Done Sent Done false
19027 6140 Acked Write 7 Done Sent Done false
19028 6141 Acked Read 1 Done Sent Done false
19029 6142 Acked Read 4 Done Sent Done false
19030 6143 Acked Read 4 Done Sent Done false
19031 6144 Acked Write 6 Done Sent Done false
19032 6145 Acked Read 1 Done Sent Done false
19033 6146 Acked Read 7 Done Sent Done false
19034 6147 Acked Read 10 Done Sent Done false
19035 6148 Acked Read 8 Done Sent Done false
19036 6149 Acked Read 2 Done Sent Done false
19037 6150 Acked Flush 0 Done Sent Done false
19038 6151 Acked Write 2 Done Sent Done false
19039 6152 Acked Read 4 Done Sent Done false
19040 6153 Acked Read 4 Done Sent Done false
19041 6154 Acked Write 1 Done Sent Done false
19042 6155 Acked Read 3 Done Sent Done false
19043 6156 Acked Read 6 Done Sent Done false
19044 6157 Acked Flush 0 Done Sent Done false
19045 6158 Acked Flush 0 Done Sent Done false
19046 6159 Acked Read 9 Done Sent Done false
19047 6160 Acked Read 4 Done Sent Done false
19048 6161 Acked Write 5 Done Sent Done false
19049 6162 Acked Write 8 Done Sent Done false
19050 6163 Acked Flush 0 Done Sent Done false
19051 6164 Acked Read 4 Done Sent Done false
19052 6165 Acked Read 4 Done Sent Done false
19053 6166 Acked Write 3 Done Sent Done false
19054 6167 Acked Write 8 Done Sent Done false
19055 6168 Acked Read 8 Done Sent Done false
19056 6169 Acked Write 1 Done Sent Done false
19057 6170 Acked Write 10 Done Sent Done false
19058 6171 Acked Write 1 Done Sent Done false
19059 6172 Acked Write 7 Done Sent Done false
19060 6173 Acked Write 5 Done Sent Done false
19061 6174 Acked Read 8 Done Sent Done false
19062 6175 Acked Read 5 Done Sent Done false
19063 6176 Acked Read 8 Done Sent Done false
19064 6177 Acked Read 2 Done Sent Done false
19065 6178 Acked Read 7 Done Sent Done false
19066 6179 Acked Read 5 Done Sent Done false
19067 6180 Acked Write 6 Done Sent Done false
19068 6181 Acked Read 4 Done Sent Done false
19069 6182 Acked Write 5 Done Sent Done false
19070 6183 Acked Flush 0 Done Sent Done false
19071 6184 Acked Read 6 Done Sent Done false
19072 6185 Acked Write 3 Done Sent Done false
19073 6186 Acked Read 4 Done Sent Done false
19074 6187 Acked Flush 0 Done Sent Done false
19075 6188 Acked Read 10 Done Sent Done false
19076 6189 Acked Write 10 Done Sent Done false
19077 6190 Acked Read 2 Done Sent Done false
19078 6191 Acked Read 3 Done Sent Done false
19079 6192 Acked Read 6 Done Sent Done false
19080 6193 Acked Read 8 Done Sent Done false
19081 6194 Acked Read 5 Done Sent Done false
19082 6195 Acked Read 6 Done Sent Done false
19083 6196 Acked Write 8 Done Sent Done false
19084 6197 Acked Read 1 Done Sent Done false
19085 6198 Acked Write 3 Done Sent Done false
19086 6199 Acked Read 4 Done Sent Done false
19087 6200 Acked Read 6 Done Sent Done false
19088 6201 Acked Read 7 Done Sent Done false
19089 6202 Acked Read 7 Done Sent Done false
19090 6203 Acked Write 5 Done Sent Done false
19091 6204 Acked Flush 0 Done Sent Done false
19092 6205 Acked Read 6 Done Sent Done false
19093 6206 Acked Read 7 Done Sent Done false
19094 6207 Acked Read 4 Done Sent Done false
19095 6208 Acked Read 1 Done Sent Done false
19096 6209 Acked Read 2 Done Sent Done false
19097 6210 Acked Write 10 Done Sent Done false
19098 6211 Acked Read 3 Done Sent Done false
19099 6212 Acked Write 1 Done Sent Done false
19100 6213 Acked Read 5 Done Sent Done false
19101 6214 Acked Read 6 Done Sent Done false
19102 6215 Acked Read 2 Done Sent Done false
19103 6216 Acked Write 1 Done Sent Done false
19104 6217 Acked Read 1 Done Sent Done false
19105 6218 Acked Read 3 Done Sent Done false
19106 6219 Acked Read 1 Done Sent Done false
19107 6220 Acked Write 5 Done Sent Done false
19108 6221 Acked Write 3 Done Sent Done false
19109 6222 Acked Write 7 Done Sent Done false
19110 6223 Acked Write 1 Done Sent Done false
19111 6224 Acked Read 2 Done Sent Done false
19112 6225 Acked Write 4 Done Sent Done false
19113 6226 Acked Write 6 Done Sent Done false
19114 6227 Acked Read 10 Done Sent Done false
19115 6228 Acked Write 8 Done Sent Done false
19116 6229 Acked Write 6 Done Sent Done false
19117 6230 Acked Read 2 Done Sent Done false
19118 6231 Acked Read 5 Done Sent Done false
19119 6232 Acked Read 2 Done Sent Done false
19120 6233 Acked Read 4 Done Sent Done false
19121 6234 Acked Write 7 Done Sent Done false
19122 6235 Acked Flush 0 Done Sent Done false
19123 6236 Acked Write 6 Done Sent Done false
19124 6237 Acked Read 9 Done Sent Done false
19125 6238 Acked Write 8 Done Sent Done false
19126 6239 Acked Write 9 Done Sent Done false
19127 6240 Acked Write 8 Done Sent Done false
19128 6241 Acked Write 4 Done Sent Done false
19129 6242 Acked Read 3 Done Sent Done false
19130 6243 Acked Read 2 Done Sent Done false
19131 6244 Acked Read 10 Done Sent Done false
19132 6245 Acked Read 7 Done Sent Done false
19133 6246 Acked Read 10 Done Sent Done false
19134 6247 Acked Read 5 Done Sent Done false
19135 6248 Acked Read 4 Done Sent Done false
19136 6249 Acked Write 9 Done Sent Done false
19137 6250 Acked Read 1 Done Sent Done false
19138 6251 Acked Read 3 Done Sent Done false
19139 6252 Acked Write 10 Done Sent Done false
19140 6253 Acked Read 5 Done Sent Done false
19141 6254 Acked Flush 0 Done Sent Done false
19142 6255 Acked Read 1 Done Sent Done false
19143 6256 Acked Write 6 Done Sent Done false
19144 6257 Acked Read 6 Done Sent Done false
19145 6258 Acked Read 8 Done Sent Done false
19146 6259 Acked Write 4 Done Sent Done false
19147 6260 Acked Flush 0 Done Sent Done false
19148 6261 Acked Write 5 Done Sent Done false
19149 6262 Acked Write 7 Done Sent Done false
19150 6263 Acked Flush 0 Done Sent Done false
19151 6264 Acked Write 4 Done Sent Done false
19152 6265 Acked Write 8 Done Sent Done false
19153 6266 Acked Write 6 Done Sent Done false
19154 6267 Acked Write 8 Done Sent Done false
19155 6268 Acked Read 6 Done Sent Done false
19156 6269 Acked Write 8 Done Sent Done false
19157 6270 Acked Write 1 Done Sent Done false
19158 6271 Acked Flush 0 Done Sent Done false
19159 6272 Acked Write 3 Done Sent Done false
19160 6273 Acked Read 3 Done Sent Done false
19161 6274 Acked Read 10 Done Sent Done false
19162 6275 Acked Write 2 Done Sent Done false
19163 6276 Acked Read 10 Done Sent Done false
19164 6277 Acked Write 6 Done Sent Done false
19165 6278 Acked Write 1 Done Sent Done false
19166 6279 Acked Write 10 Done Sent Done false
19167 6280 Acked Write 8 Done Sent Done false
19168 6281 Acked Write 3 Done Sent Done false
19169 6282 Acked Read 7 Done Sent Done false
19170 6283 Acked Write 6 Done Sent Done false
19171 6284 Acked Write 5 Done Sent Done false
19172 6285 Acked Read 6 Done Sent Done false
19173 6286 Acked Read 2 Done Sent Done false
19174 6287 Acked Write 9 Done Sent Done false
19175 6288 Acked Read 1 Done Sent Done false
19176 6289 Acked Flush 0 Done Sent Done false
19177 6290 Acked Read 2 Done Sent Done false
19178 6291 Acked Read 8 Done Sent Done false
19179 6292 Acked Write 4 Done Sent Done false
19180 6293 Acked Write 1 Done Sent Done false
19181 6294 Acked Flush 0 Done Sent Done false
19182 6295 Acked Write 7 Done Sent Done false
19183 6296 Acked Read 6 Done Sent Done false
19184 6297 Acked Read 4 Done Sent Done false
19185 6298 Acked Flush 0 Done Sent Done false
19186 6299 Acked Read 3 Done Sent Done false
19187 6300 Acked Read 1 Done Sent Done false
19188 6301 Acked Read 5 Done Sent Done false
19189 6302 Acked Read 2 Done Sent Done false
19190 6303 Acked Write 4 Done Sent Done false
19191 6304 Acked Read 2 Done Sent Done false
19192 6305 Acked Read 8 Done Sent Done false
19193 6306 Acked Read 4 Done Sent Done false
19194 6307 Acked Write 5 Done Sent Done false
19195 6308 Acked Write 4 Done Sent Done false
19196 6309 Acked Read 9 Done Sent Done false
19197 6310 Acked Write 4 Done Sent Done false
19198 6311 Acked Read 9 Done Sent Done false
19199 6312 Acked Flush 0 Done Sent Done false
19200 6313 Acked Read 7 Done Sent Done false
19201 6314 Acked Read 7 Done Sent Done false
19202 6315 Acked Flush 0 Done Sent Done false
19203 6316 Acked Read 2 Done Sent Done false
19204 6317 Acked Write 6 Done Sent Done false
19205 6318 Acked Write 1 Done Sent Done false
19206 6319 Acked Write 4 Done Sent Done false
19207 6320 Acked Write 9 Done Sent Done false
19208 6321 Acked Write 9 Done Sent Done false
19209 6322 Acked Write 2 Done Sent Done false
19210 6323 Acked Write 1 Done Sent Done false
19211 6324 Acked Read 2 Done Sent Done false
19212 6325 Acked Read 1 Done Sent Done false
19213 6326 Acked Write 8 Done Sent Done false
19214 6327 Acked Write 8 Done Sent Done false
19215 6328 Acked Write 7 Done Sent Done false
19216 6329 Acked Read 4 Done Sent Done false
19217 6330 Acked Read 6 Done Sent Done false
19218 6331 Acked Read 6 Done Sent Done false
19219 6332 Acked Flush 0 Done Sent Done false
19220 6333 Acked Write 6 Done Sent Done false
19221 6334 Acked Write 1 Done Sent Done false
19222 6335 Acked Write 1 Done Sent Done false
19223 6336 Acked Read 8 Done Sent Done false
19224 6337 Acked Read 9 Done Sent Done false
19225 6338 Acked Read 6 Done Sent Done false
19226 6339 Acked Write 9 Done Sent Done false
19227 6340 Acked Flush 0 Done Sent Done false
19228 6341 Acked Read 6 Done Sent Done false
19229 6342 Acked Read 9 Done Sent Done false
19230 6343 Acked Write 10 Done Sent Done false
19231 6344 Acked Write 2 Done Sent Done false
19232 6345 Acked Read 8 Done Sent Done false
19233 6346 Acked Write 1 Done Sent Done false
19234 6347 Acked Write 8 Done Sent Done false
19235 6348 Acked Read 10 Done Sent Done false
19236 6349 Acked Read 3 Done Sent Done false
19237 6350 Acked Write 6 Done Sent Done false
19238 6351 Acked Read 2 Done Sent Done false
19239 6352 Acked Write 9 Done Sent Done false
19240 6353 Acked Read 2 Done Sent Done false
19241 6354 Acked Read 4 Done Sent Done false
19242 6355 Acked Write 3 Done Sent Done false
19243 6356 Acked Read 6 Done Sent Done false
19244 6357 Acked Read 3 Done Sent Done false
19245 6358 Acked Flush 0 Done Sent Done false
19246 6359 Acked Read 7 Done Sent Done false
19247 6360 Acked Read 1 Done Sent Done false
19248 6361 Acked Write 6 Done Sent Done false
19249 6362 Acked Write 5 Done Sent Done false
19250 6363 Acked Read 1 Done Sent Done false
19251 6364 Acked Read 8 Done Sent Done false
19252 6365 Acked Read 1 Done Sent Done false
19253 6366 Acked Write 8 Done Sent Done false
19254 6367 Acked Read 1 Done Sent Done false
19255 6368 Acked Read 2 Done Sent Done false
19256 6369 Acked Read 1 Done Sent Done false
19257 6370 Acked Write 4 Done Sent Done false
19258 6371 Acked Read 8 Done Sent Done false
19259 6372 Acked Flush 0 Done Sent Done false
19260 6373 Acked Write 3 Done Sent Done false
19261 6374 Acked Read 6 Done Sent Done false
19262 6375 Acked Read 10 Done Sent Done false
19263 6376 Acked Read 6 Done Sent Done false
19264 6377 Acked Flush 0 Done Sent Done false
19265 6378 Acked Write 5 Done Sent Done false
19266 6379 Acked Read 10 Done Sent Done false
19267 6380 Acked Flush 0 Done Sent Done false
19268 6381 Acked Write 8 Done Sent Done false
19269 6382 Acked Flush 0 Done Sent Done false
19270 6383 Acked Read 8 Done Sent Done false
19271 6384 Acked Write 1 Done Sent Done false
19272 6385 Acked Read 7 Done Sent Done false
19273 6386 Acked Flush 0 Done Sent Done false
19274 6387 Acked Flush 0 Done Sent Done false
19275 6388 Acked Flush 0 Done Sent Done false
19276 6389 Acked Read 10 Done Sent Done false
19277 6390 Acked Write 6 Done Sent Done false
19278 6391 Acked Read 3 Done Sent Done false
19279 6392 Acked Read 8 Done Sent Done false
19280 6393 Acked Write 2 Done Sent Done false
19281 6394 Acked Write 7 Done Sent Done false
19282 6395 Acked Write 2 Done Sent Done false
19283 6396 Acked Read 10 Done Sent Done false
19284 6397 Acked Write 9 Done Sent Done false
19285 6398 Acked Read 7 Done Sent Done false
19286 6399 Acked Read 8 Done Sent Done false
19287 6400 Acked Flush 0 Done Sent Done false
19288 6401 Acked Flush 0 Done Sent Done false
19289 6402 Acked Write 6 Done Sent Done false
19290 6403 Acked Write 8 Done Sent Done false
19291 6404 Acked Read 7 Done Sent Done false
19292 6405 Acked Read 5 Done Sent Done false
19293 6406 Acked Flush 0 Done Sent Done false
19294 6407 Acked Write 9 Done Sent Done false
19295 6408 Acked Read 7 Done Sent Done false
19296 6409 Acked Write 8 Done Sent Done false
19297 6410 Acked Read 9 Done Sent Done false
19298 6411 Acked Flush 0 Done Sent Done false
19299 6412 Acked Write 9 Done Sent Done false
19300 6413 Acked Write 2 Done Sent Done false
19301 6414 Acked Read 9 Done Sent Done false
19302 6415 Acked Read 1 Done Sent Done false
19303 6416 Acked Write 10 Done Sent Done false
19304 6417 Acked Read 9 Done Sent Done false
19305 6418 Acked Read 10 Done Sent Done false
19306 6419 Acked Write 4 Done Sent Done false
19307 6420 Acked Read 6 Done Sent Done false
19308 6421 Acked Read 1 Done Sent Done false
19309 6422 Acked Read 3 Done Sent Done false
19310 6423 Acked Write 4 Done Sent Done false
19311 6424 Acked Read 3 Done Sent Done false
19312 6425 Acked Write 6 Done Sent Done false
19313 6426 Acked Write 3 Done Sent Done false
19314 6427 Acked Read 5 Done Sent Done false
19315 6428 Acked Flush 0 Done Sent Done false
19316 6429 Acked Write 10 Done Sent Done false
19317 6430 Acked Write 4 Done Sent Done false
19318 6431 Acked Write 1 Done Sent Done false
19319 6432 Acked Flush 0 Done Sent Done false
19320 STATES DS:0 DS:1 DS:2 TOTAL
19321 Sent 0 302 0 302
19322 Done 302 0 302 604
19323 Skipped 0 0 0 0
19324 Error 0 0 0 0
19325 Last Flush: 6432 6130 6432
19326 Downstairs last five completed: 6130 6129 6128 6127 6126
19327 Upstairs last five completed: 6432 6431 6430 6429 6428
19328 CLIENT: Up:0 ds:302 act:2
19329 ----------------------------------------------------------------
19330 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19331 States: Active Offline Active
19332 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19333 6131 Acked Write 10 Done Sent Done false
19334 6132 Acked Flush 0 Done Sent Done false
19335 6133 Acked Write 7 Done Sent Done false
19336 6134 Acked Read 7 Done Sent Done false
19337 6135 Acked Read 3 Done Sent Done false
19338 6136 Acked Write 6 Done Sent Done false
19339 6137 Acked Read 10 Done Sent Done false
19340 6138 Acked Write 2 Done Sent Done false
19341 6139 Acked Flush 0 Done Sent Done false
19342 6140 Acked Write 7 Done Sent Done false
19343 6141 Acked Read 1 Done Sent Done false
19344 6142 Acked Read 4 Done Sent Done false
19345 6143 Acked Read 4 Done Sent Done false
19346 6144 Acked Write 6 Done Sent Done false
19347 6145 Acked Read 1 Done Sent Done false
19348 6146 Acked Read 7 Done Sent Done false
19349 6147 Acked Read 10 Done Sent Done false
19350 6148 Acked Read 8 Done Sent Done false
19351 6149 Acked Read 2 Done Sent Done false
19352 6150 Acked Flush 0 Done Sent Done false
19353 6151 Acked Write 2 Done Sent Done false
19354 6152 Acked Read 4 Done Sent Done false
19355 6153 Acked Read 4 Done Sent Done false
19356 6154 Acked Write 1 Done Sent Done false
19357 6155 Acked Read 3 Done Sent Done false
19358 6156 Acked Read 6 Done Sent Done false
19359 6157 Acked Flush 0 Done Sent Done false
19360 6158 Acked Flush 0 Done Sent Done false
19361 6159 Acked Read 9 Done Sent Done false
19362 6160 Acked Read 4 Done Sent Done false
19363 6161 Acked Write 5 Done Sent Done false
19364 6162 Acked Write 8 Done Sent Done false
19365 6163 Acked Flush 0 Done Sent Done false
19366 6164 Acked Read 4 Done Sent Done false
19367 6165 Acked Read 4 Done Sent Done false
19368 6166 Acked Write 3 Done Sent Done false
19369 6167 Acked Write 8 Done Sent Done false
19370 6168 Acked Read 8 Done Sent Done false
19371 6169 Acked Write 1 Done Sent Done false
19372 6170 Acked Write 10 Done Sent Done false
19373 6171 Acked Write 1 Done Sent Done false
19374 6172 Acked Write 7 Done Sent Done false
19375 6173 Acked Write 5 Done Sent Done false
19376 6174 Acked Read 8 Done Sent Done false
19377 6175 Acked Read 5 Done Sent Done false
19378 6176 Acked Read 8 Done Sent Done false
19379 6177 Acked Read 2 Done Sent Done false
19380 6178 Acked Read 7 Done Sent Done false
19381 6179 Acked Read 5 Done Sent Done false
19382 6180 Acked Write 6 Done Sent Done false
19383 6181 Acked Read 4 Done Sent Done false
19384 6182 Acked Write 5 Done Sent Done false
19385 6183 Acked Flush 0 Done Sent Done false
19386 6184 Acked Read 6 Done Sent Done false
19387 6185 Acked Write 3 Done Sent Done false
19388 6186 Acked Read 4 Done Sent Done false
19389 6187 Acked Flush 0 Done Sent Done false
19390 6188 Acked Read 10 Done Sent Done false
19391 6189 Acked Write 10 Done Sent Done false
19392 6190 Acked Read 2 Done Sent Done false
19393 6191 Acked Read 3 Done Sent Done false
19394 6192 Acked Read 6 Done Sent Done false
19395 6193 Acked Read 8 Done Sent Done false
19396 6194 Acked Read 5 Done Sent Done false
19397 6195 Acked Read 6 Done Sent Done false
19398 6196 Acked Write 8 Done Sent Done false
19399 6197 Acked Read 1 Done Sent Done false
19400 6198 Acked Write 3 Done Sent Done false
19401 6199 Acked Read 4 Done Sent Done false
19402 6200 Acked Read 6 Done Sent Done false
19403 6201 Acked Read 7 Done Sent Done false
19404 6202 Acked Read 7 Done Sent Done false
19405 6203 Acked Write 5 Done Sent Done false
19406 6204 Acked Flush 0 Done Sent Done false
19407 6205 Acked Read 6 Done Sent Done false
19408 6206 Acked Read 7 Done Sent Done false
19409 6207 Acked Read 4 Done Sent Done false
19410 6208 Acked Read 1 Done Sent Done false
19411 6209 Acked Read 2 Done Sent Done false
19412 6210 Acked Write 10 Done Sent Done false
19413 6211 Acked Read 3 Done Sent Done false
19414 6212 Acked Write 1 Done Sent Done false
19415 6213 Acked Read 5 Done Sent Done false
19416 6214 Acked Read 6 Done Sent Done false
19417 6215 Acked Read 2 Done Sent Done false
19418 6216 Acked Write 1 Done Sent Done false
19419 6217 Acked Read 1 Done Sent Done false
19420 6218 Acked Read 3 Done Sent Done false
19421 6219 Acked Read 1 Done Sent Done false
19422 6220 Acked Write 5 Done Sent Done false
19423 6221 Acked Write 3 Done Sent Done false
19424 6222 Acked Write 7 Done Sent Done false
19425 6223 Acked Write 1 Done Sent Done false
19426 6224 Acked Read 2 Done Sent Done false
19427 6225 Acked Write 4 Done Sent Done false
19428 6226 Acked Write 6 Done Sent Done false
19429 6227 Acked Read 10 Done Sent Done false
19430 6228 Acked Write 8 Done Sent Done false
19431 6229 Acked Write 6 Done Sent Done false
19432 6230 Acked Read 2 Done Sent Done false
19433 6231 Acked Read 5 Done Sent Done false
19434 6232 Acked Read 2 Done Sent Done false
19435 6233 Acked Read 4 Done Sent Done false
19436 6234 Acked Write 7 Done Sent Done false
19437 6235 Acked Flush 0 Done Sent Done false
19438 6236 Acked Write 6 Done Sent Done false
19439 6237 Acked Read 9 Done Sent Done false
19440 6238 Acked Write 8 Done Sent Done false
19441 6239 Acked Write 9 Done Sent Done false
19442 6240 Acked Write 8 Done Sent Done false
19443 6241 Acked Write 4 Done Sent Done false
19444 6242 Acked Read 3 Done Sent Done false
19445 6243 Acked Read 2 Done Sent Done false
19446 6244 Acked Read 10 Done Sent Done false
19447 6245 Acked Read 7 Done Sent Done false
19448 6246 Acked Read 10 Done Sent Done false
19449 6247 Acked Read 5 Done Sent Done false
19450 6248 Acked Read 4 Done Sent Done false
19451 6249 Acked Write 9 Done Sent Done false
19452 6250 Acked Read 1 Done Sent Done false
19453 6251 Acked Read 3 Done Sent Done false
19454 6252 Acked Write 10 Done Sent Done false
19455 6253 Acked Read 5 Done Sent Done false
19456 6254 Acked Flush 0 Done Sent Done false
19457 6255 Acked Read 1 Done Sent Done false
19458 6256 Acked Write 6 Done Sent Done false
19459 6257 Acked Read 6 Done Sent Done false
19460 6258 Acked Read 8 Done Sent Done false
19461 6259 Acked Write 4 Done Sent Done false
19462 6260 Acked Flush 0 Done Sent Done false
19463 6261 Acked Write 5 Done Sent Done false
19464 6262 Acked Write 7 Done Sent Done false
19465 6263 Acked Flush 0 Done Sent Done false
19466 6264 Acked Write 4 Done Sent Done false
19467 6265 Acked Write 8 Done Sent Done false
19468 6266 Acked Write 6 Done Sent Done false
19469 6267 Acked Write 8 Done Sent Done false
19470 6268 Acked Read 6 Done Sent Done false
19471 6269 Acked Write 8 Done Sent Done false
19472 6270 Acked Write 1 Done Sent Done false
19473 6271 Acked Flush 0 Done Sent Done false
19474 6272 Acked Write 3 Done Sent Done false
19475 6273 Acked Read 3 Done Sent Done false
19476 6274 Acked Read 10 Done Sent Done false
19477 6275 Acked Write 2 Done Sent Done false
19478 6276 Acked Read 10 Done Sent Done false
19479 6277 Acked Write 6 Done Sent Done false
19480 6278 Acked Write 1 Done Sent Done false
19481 6279 Acked Write 10 Done Sent Done false
19482 6280 Acked Write 8 Done Sent Done false
19483 6281 Acked Write 3 Done Sent Done false
19484 6282 Acked Read 7 Done Sent Done false
19485 6283 Acked Write 6 Done Sent Done false
19486 6284 Acked Write 5 Done Sent Done false
19487 6285 Acked Read 6 Done Sent Done false
19488 6286 Acked Read 2 Done Sent Done false
19489 6287 Acked Write 9 Done Sent Done false
19490 6288 Acked Read 1 Done Sent Done false
19491 6289 Acked Flush 0 Done Sent Done false
19492 6290 Acked Read 2 Done Sent Done false
19493 6291 Acked Read 8 Done Sent Done false
19494 6292 Acked Write 4 Done Sent Done false
19495 6293 Acked Write 1 Done Sent Done false
19496 6294 Acked Flush 0 Done Sent Done false
19497 6295 Acked Write 7 Done Sent Done false
19498 6296 Acked Read 6 Done Sent Done false
19499 6297 Acked Read 4 Done Sent Done false
19500 6298 Acked Flush 0 Done Sent Done false
19501 6299 Acked Read 3 Done Sent Done false
19502 6300 Acked Read 1 Done Sent Done false
19503 6301 Acked Read 5 Done Sent Done false
19504 6302 Acked Read 2 Done Sent Done false
19505 6303 Acked Write 4 Done Sent Done false
19506 6304 Acked Read 2 Done Sent Done false
19507 6305 Acked Read 8 Done Sent Done false
19508 6306 Acked Read 4 Done Sent Done false
19509 6307 Acked Write 5 Done Sent Done false
19510 6308 Acked Write 4 Done Sent Done false
19511 6309 Acked Read 9 Done Sent Done false
19512 6310 Acked Write 4 Done Sent Done false
19513 6311 Acked Read 9 Done Sent Done false
19514 6312 Acked Flush 0 Done Sent Done false
19515 6313 Acked Read 7 Done Sent Done false
19516 6314 Acked Read 7 Done Sent Done false
19517 6315 Acked Flush 0 Done Sent Done false
19518 6316 Acked Read 2 Done Sent Done false
19519 6317 Acked Write 6 Done Sent Done false
19520 6318 Acked Write 1 Done Sent Done false
19521 6319 Acked Write 4 Done Sent Done false
19522 6320 Acked Write 9 Done Sent Done false
19523 6321 Acked Write 9 Done Sent Done false
19524 6322 Acked Write 2 Done Sent Done false
19525 6323 Acked Write 1 Done Sent Done false
19526 6324 Acked Read 2 Done Sent Done false
19527 6325 Acked Read 1 Done Sent Done false
19528 6326 Acked Write 8 Done Sent Done false
19529 6327 Acked Write 8 Done Sent Done false
19530 6328 Acked Write 7 Done Sent Done false
19531 6329 Acked Read 4 Done Sent Done false
19532 6330 Acked Read 6 Done Sent Done false
19533 6331 Acked Read 6 Done Sent Done false
19534 6332 Acked Flush 0 Done Sent Done false
19535 6333 Acked Write 6 Done Sent Done false
19536 6334 Acked Write 1 Done Sent Done false
19537 6335 Acked Write 1 Done Sent Done false
19538 6336 Acked Read 8 Done Sent Done false
19539 6337 Acked Read 9 Done Sent Done false
19540 6338 Acked Read 6 Done Sent Done false
19541 6339 Acked Write 9 Done Sent Done false
19542 6340 Acked Flush 0 Done Sent Done false
19543 6341 Acked Read 6 Done Sent Done false
19544 6342 Acked Read 9 Done Sent Done false
19545 6343 Acked Write 10 Done Sent Done false
19546 6344 Acked Write 2 Done Sent Done false
19547 6345 Acked Read 8 Done Sent Done false
19548 6346 Acked Write 1 Done Sent Done false
19549 6347 Acked Write 8 Done Sent Done false
19550 6348 Acked Read 10 Done Sent Done false
19551 6349 Acked Read 3 Done Sent Done false
19552 6350 Acked Write 6 Done Sent Done false
19553 6351 Acked Read 2 Done Sent Done false
19554 6352 Acked Write 9 Done Sent Done false
19555 6353 Acked Read 2 Done Sent Done false
19556 6354 Acked Read 4 Done Sent Done false
19557 6355 Acked Write 3 Done Sent Done false
19558 6356 Acked Read 6 Done Sent Done false
19559 6357 Acked Read 3 Done Sent Done false
19560 6358 Acked Flush 0 Done Sent Done false
19561 6359 Acked Read 7 Done Sent Done false
19562 6360 Acked Read 1 Done Sent Done false
19563 6361 Acked Write 6 Done Sent Done false
19564 6362 Acked Write 5 Done Sent Done false
19565 6363 Acked Read 1 Done Sent Done false
19566 6364 Acked Read 8 Done Sent Done false
19567 6365 Acked Read 1 Done Sent Done false
19568 6366 Acked Write 8 Done Sent Done false
19569 6367 Acked Read 1 Done Sent Done false
19570 6368 Acked Read 2 Done Sent Done false
19571 6369 Acked Read 1 Done Sent Done false
19572 6370 Acked Write 4 Done Sent Done false
19573 6371 Acked Read 8 Done Sent Done false
19574 6372 Acked Flush 0 Done Sent Done false
19575 6373 Acked Write 3 Done Sent Done false
19576 6374 Acked Read 6 Done Sent Done false
19577 6375 Acked Read 10 Done Sent Done false
19578 6376 Acked Read 6 Done Sent Done false
19579 6377 Acked Flush 0 Done Sent Done false
19580 6378 Acked Write 5 Done Sent Done false
19581 6379 Acked Read 10 Done Sent Done false
19582 6380 Acked Flush 0 Done Sent Done false
19583 6381 Acked Write 8 Done Sent Done false
19584 6382 Acked Flush 0 Done Sent Done false
19585 6383 Acked Read 8 Done Sent Done false
19586 6384 Acked Write 1 Done Sent Done false
19587 6385 Acked Read 7 Done Sent Done false
19588 6386 Acked Flush 0 Done Sent Done false
19589 6387 Acked Flush 0 Done Sent Done false
19590 6388 Acked Flush 0 Done Sent Done false
19591 6389 Acked Read 10 Done Sent Done false
19592 6390 Acked Write 6 Done Sent Done false
19593 6391 Acked Read 3 Done Sent Done false
19594 6392 Acked Read 8 Done Sent Done false
19595 6393 Acked Write 2 Done Sent Done false
19596 6394 Acked Write 7 Done Sent Done false
19597 6395 Acked Write 2 Done Sent Done false
19598 6396 Acked Read 10 Done Sent Done false
19599 6397 Acked Write 9 Done Sent Done false
19600 6398 Acked Read 7 Done Sent Done false
19601 6399 Acked Read 8 Done Sent Done false
19602 6400 Acked Flush 0 Done Sent Done false
19603 6401 Acked Flush 0 Done Sent Done false
19604 6402 Acked Write 6 Done Sent Done false
19605 6403 Acked Write 8 Done Sent Done false
19606 6404 Acked Read 7 Done Sent Done false
19607 6405 Acked Read 5 Done Sent Done false
19608 6406 Acked Flush 0 Done Sent Done false
19609 6407 Acked Write 9 Done Sent Done false
19610 6408 Acked Read 7 Done Sent Done false
19611 6409 Acked Write 8 Done Sent Done false
19612 6410 Acked Read 9 Done Sent Done false
19613 6411 Acked Flush 0 Done Sent Done false
19614 6412 Acked Write 9 Done Sent Done false
19615 6413 Acked Write 2 Done Sent Done false
19616 6414 Acked Read 9 Done Sent Done false
19617 6415 Acked Read 1 Done Sent Done false
19618 6416 Acked Write 10 Done Sent Done false
19619 6417 Acked Read 9 Done Sent Done false
19620 6418 Acked Read 10 Done Sent Done false
19621 6419 Acked Write 4 Done Sent Done false
19622 6420 Acked Read 6 Done Sent Done false
19623 6421 Acked Read 1 Done Sent Done false
19624 6422 Acked Read 3 Done Sent Done false
19625 6423 Acked Write 4 Done Sent Done false
19626 6424 Acked Read 3 Done Sent Done false
19627 6425 Acked Write 6 Done Sent Done false
19628 6426 Acked Write 3 Done Sent Done false
19629 6427 Acked Read 5 Done Sent Done false
19630 6428 Acked Flush 0 Done Sent Done false
19631 6429 Acked Write 10 Done Sent Done false
19632 6430 Acked Write 4 Done Sent Done false
19633 6431 Acked Write 1 Done Sent Done false
19634 6432 Acked Flush 0 Done Sent Done false
19635 STATES DS:0 DS:1 DS:2 TOTAL
19636 Sent 0 302 0 302
19637 Done 302 0 302 604
19638 Skipped 0 0 0 0
19639 Error 0 0 0 0
19640 Last Flush: 6432 6130 6432
19641 Downstairs last five completed: 6130 6129 6128 6127 6126
19642 Upstairs last five completed: 6432 6431 6430 6429 6428
19643 CLIENT: Up:0 ds:302 act:2
196442024-12-04T13:34:50.309ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196452024-12-04T13:34:50.309ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34345) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196462024-12-04T13:34:50.310ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196472024-12-04T13:34:50.310ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196482024-12-04T13:34:50.310ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196492024-12-04T13:34:50.310ZINFOcrucible: send last flush ID to this DS: 6130 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196502024-12-04T13:34:50.311ZINFOcrucible: Replied this last flush ID: 6130 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196512024-12-04T13:34:50.311ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196522024-12-04T13:34:50.311ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196532024-12-04T13:34:50.311ZINFOcrucible: [1] client re-new 302 jobs since flush 6130 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196542024-12-04T13:34:50.311ZINFOcrucible: [1] Replayed 302 jobs since flush: 6130 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
19655 ----------------------------------------------------------------
19656 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19657 Downstairs last five completed: 6432 6431 6430 6429 6428
19658 Upstairs last five completed: 6432 6431 6430 6429 6428
19659 CLIENT: Up:0 ds:0 act:3
19660 Replay: All jobs finished
19661 Main task has work to do, go find it
19662 got dsc Stop(1)
19663 stop 1
19664 [8820] Got stop action so:false kr:false
19665 Dec 04 13:34:50.550 INFO request completed, latency_us: 382, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 5f78b67e-daed-4090-9f2f-e3053382280a, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
196662024-12-04T13:34:50.550ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196672024-12-04T13:34:50.550ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196682024-12-04T13:34:50.550ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196692024-12-04T13:34:50.550ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196702024-12-04T13:34:50.550ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
196712024-12-04T13:34:50.550ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
19672 Dec 04 13:34:50.550 INFO request completed, latency_us: 143, response_code: 200, uri: /state/cid/1, method: GET, req_id: 07cd3c71-5649-4b88-92d9-5897c4032bb5, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
19673 [8820] Exited with: ExitStatus(unix_wait_status(9))
19674 [8820][1] reports Exit
19675 Dec 04 13:34:54.553 INFO request completed, latency_us: 191, response_code: 200, uri: /state/cid/1, method: GET, req_id: 505d7f5d-4003-4378-b800-8dd165b5f8b1, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
19676 001/300 Write block 3629 len 24576 data: 6 6 5 5 6 5
19677 002/300 Read block 4982 len 16384
19678 003/300 Read block 3255 len 4096
19679 004/300 Write block 253 len 4096 data: 8
19680 005/300 Write block 1050 len 28672 data: 5 6 6 7 6 7 7
19681 006/300 Write block 3863 len 4096 data: 5
19682 007/300 Read block 3292 len 8192
19683 008/300 Read block 2754 len 32768
19684 009/300 Write block 4674 len 32768 data: 3 3 3 4 4 4 3 2
19685 010/300 Write block 1056 len 28672 data: 8 8 8 7 6 5 5
19686 011/300 Write block 288 len 8192 data: 3 2
19687 012/300 Write block 2389 len 24576 data: 5 5 5 5 4 4
19688 013/300 Write block 16 len 4096 data: 10
19689 014/300 Write block 1457 len 8192 data: 8 8
19690 015/300 Read block 2185 len 16384
19691 016/300 Read block 4386 len 20480
19692 017/300 Write block 1555 len 28672 data: 3 3 2 3 3 3 4
19693 018/300 Read block 4794 len 8192
19694 019/300 Write block 3318 len 28672 data: 5 6 9 9 7 7 5
19695 020/300 Read block 4351 len 24576
19696 021/300 Read block 1308 len 20480
19697 022/300 Read block 3261 len 8192
19698 023/300 Write block 3199 len 24576 data: 2 3 3 3 3 3
19699 024/300 Write block 2570 len 36864 data: 6 6 3 3 3 2 2 3 3
19700 025/300 Read block 1372 len 32768
19701 026/300 Flush
19702 027/300 Write block 3103 len 32768 data: 5 5 6 5 5 6 7 5
19703 028/300 Write block 52 len 28672 data: 2 3 4 6 5 5 5
19704 029/300 Read block 4219 len 4096
19705 030/300 Read block 4269 len 8192
19706 031/300 Read block 1549 len 36864
19707 032/300 Read block 2928 len 8192
19708 033/300 Read block 3225 len 28672
19709 034/300 Write block 3293 len 32768 data: 4 6 7 9 9 8 7 7
19710 035/300 Read block 4477 len 4096
19711 036/300 Flush
19712 037/300 Write block 62 len 20480 data: 5 4 5 7 7
19713 038/300 Read block 2997 len 28672
19714 039/300 Read block 987 len 40960
19715 040/300 Read block 1443 len 32768
19716 041/300 Read block 3289 len 28672
19717 042/300 Read block 3521 len 24576
19718 043/300 Read block 3282 len 4096
19719 044/300 Read block 4570 len 16384
19720 045/300 Write block 2211 len 32768 data: 4 4 4 3 3 3 4 4
19721 046/300 Write block 2256 len 8192 data: 4 3
19722 047/300 Write block 4836 len 28672 data: 2 3 3 3 4 4 4
19723 048/300 Write block 4020 len 4096 data: 4
19724 049/300 Write block 1210 len 24576 data: 6 5 5 5 5 6
19725 050/300 Read block 1171 len 24576
19726 051/300 Read block 2375 len 12288
19727 052/300 Flush
19728 053/300 Flush
19729 054/300 Write block 1483 len 24576 data: 4 4 4 3 3 3
19730 055/300 Flush
19731 056/300 Flush
19732 057/300 Write block 4220 len 24576 data: 5 5 5 4 3 3
19733 058/300 Write block 3926 len 20480 data: 4 4 5 6 5
19734 059/300 Read block 1491 len 24576
19735 060/300 Read block 3591 len 8192
19736 061/300 Read block 3083 len 24576
19737 062/300 Read block 3388 len 12288
19738 063/300 Write block 2153 len 32768 data: 3 4 4 4 3 3 3 2
19739 064/300 Flush
19740 065/300 Read block 1786 len 12288
19741 066/300 Read block 3176 len 36864
19742 067/300 Flush
19743 068/300 Read block 1866 len 36864
19744 069/300 Read block 2055 len 32768
19745 070/300 Read block 2978 len 32768
19746 071/300 Read block 404 len 28672
19747 072/300 Read block 1886 len 28672
19748 073/300 Read block 2475 len 20480
19749 074/300 Read block 1234 len 8192
19750 075/300 Read block 3213 len 4096
19751 076/300 Write block 2068 len 32768 data: 2 2 2 3 4 4 3 3
19752 077/300 Read block 123 len 24576
19753 078/300 Write block 3788 len 28672 data: 5 5 4 3 3 4 4
19754 079/300 Read block 916 len 28672
19755 080/300 Write block 2738 len 20480 data: 5 4 4 4 3
19756 081/300 Write block 3217 len 40960 data: 8 8 4 4 4 4 3 5 5 6
19757 082/300 Read block 191 len 8192
19758 083/300 Write block 153 len 4096 data: 5
19759 084/300 Read block 3307 len 40960
19760 085/300 Read block 3835 len 36864
19761 086/300 Write block 4133 len 24576 data: 5 5 5 5 5 4
19762 087/300 Write block 4954 len 4096 data: 4
19763 088/300 Write block 969 len 16384 data: 3 3 3 3
19764 089/300 Read block 1064 len 28672
19765 090/300 Read block 28 len 36864
19766 091/300 Flush
19767 092/300 Read block 2910 len 40960
19768 093/300 Read block 2229 len 32768
19769 094/300 Write block 4847 len 28672 data: 4 3 3 3 5 3 3
19770 095/300 Write block 4983 len 12288 data: 4 4 5
19771 096/300 Read block 2618 len 12288
19772 097/300 Write block 497 len 36864 data: 2 3 3 3 4 3 5 5 6
19773 098/300 Write block 2146 len 24576 data: 5 4 4 4 3 3
19774 099/300 Read block 3681 len 8192
19775 100/300 Read block 3136 len 40960
19776 101/300 Read block 1462 len 36864
19777 102/300 Read block 4484 len 20480
19778 103/300 Write block 36 len 8192 data: 4 4
19779 104/300 Read block 150 len 24576
19780 105/300 Read block 4127 len 28672
19781 106/300 Write block 2516 len 24576 data: 5 5 5 5 6 6
19782 107/300 Read block 4777 len 20480
19783 108/300 Flush
19784 109/300 Read block 2423 len 28672
19785 110/300 Write block 4047 len 4096 data: 4
19786 111/300 Read block 1069 len 40960
19787 112/300 Write block 981 len 12288 data: 5 4 6
19788 113/300 Read block 173 len 24576
19789 114/300 Write block 268 len 12288 data: 5 5 4
19790 115/300 Read block 2656 len 16384
19791 116/300 Read block 3889 len 24576
19792 117/300 Write block 2353 len 12288 data: 8 6 5
19793 118/300 Read block 3774 len 20480
19794 119/300 Read block 3246 len 12288
19795 120/300 Read block 2358 len 32768
19796 121/300 Read block 246 len 32768
19797 122/300 Read block 2649 len 12288
19798 123/300 Write block 2085 len 36864 data: 5 4 3 4 5 4 4 3 2
19799 124/300 Read block 3087 len 28672
19800 125/300 Flush
19801 126/300 Write block 3519 len 24576 data: 7 5 5 3 2 2
19802 127/300 Read block 912 len 4096
19803 128/300 Read block 1996 len 20480
19804 129/300 Write block 2031 len 4096 data: 5
19805 130/300 Flush
19806 131/300 Read block 1166 len 12288
19807 132/300 Read block 3180 len 24576
19808 133/300 Read block 3742 len 16384
19809 134/300 Read block 1078 len 20480
19810 135/300 Write block 3942 len 28672 data: 7 8 6 7 6 5 6
19811 136/300 Flush
19812 137/300 Write block 701 len 40960 data: 4 3 3 3 4 4 4 5 4 3
19813 138/300 Read block 517 len 16384
19814 139/300 Read block 1915 len 12288
19815 140/300 Flush
19816 141/300 Read block 1200 len 20480
19817 142/300 Read block 3622 len 4096
19818 143/300 Read block 4810 len 36864
19819 144/300 Write block 825 len 28672 data: 4 5 5 5 5 4 5
19820 145/300 Read block 519 len 40960
19821 146/300 Read block 4237 len 20480
19822 147/300 Read block 4831 len 16384
19823 148/300 Read block 4185 len 20480
19824 149/300 Flush
19825 150/300 Flush
19826 151/300 Read block 989 len 40960
19827 152/300 Read block 4331 len 8192
19828 153/300 Write block 3237 len 28672 data: 7 6 4 4 6 5 5
19829 154/300 Write block 637 len 16384 data: 4 3 3 3
19830 155/300 Read block 1910 len 20480
19831 156/300 Read block 3980 len 32768
19832 157/300 Write block 2889 len 24576 data: 4 3 2 2 3 4
19833 158/300 Read block 3873 len 32768
19834 159/300 Write block 3498 len 40960 data: 4 4 4 6 6 6 4 3 2 3
19835 160/300 Write block 4653 len 16384 data: 3 3 3 3
19836 161/300 Read block 1108 len 4096
19837 162/300 Read block 351 len 20480
19838 163/300 Write block 1545 len 40960 data: 5 5 4 4 3 4 5 5 5 4
19839 164/300 Read block 681 len 20480
19840 165/300 Write block 2576 len 28672 data: 3 4 4 3 3 3 2
19841 166/300 Write block 4210 len 20480 data: 3 3 2 2 3
19842 167/300 Write block 1598 len 32768 data: 5 4 3 3 3 3 2 3
19843 168/300 Write block 1851 len 24576 data: 6 8 6 4 4 4
19844 169/300 Write block 3515 len 16384 data: 7 7 8 7
19845 170/300 Read block 3817 len 12288
19846 171/300 Read block 3890 len 32768
19847 172/300 Write block 3335 len 32768 data: 5 4 2 2 2 2 2 2
19848 173/300 Write block 1061 len 12288 data: 6 6 5
19849 174/300 Read block 2574 len 8192
19850 175/300 Read block 387 len 8192
19851 176/300 Write block 630 len 12288 data: 5 4 3
19852 177/300 Read block 278 len 4096
19853 178/300 Read block 3128 len 36864
19854 179/300 Write block 1188 len 28672 data: 4 5 4 3 4 4 4
19855 180/300 Write block 450 len 36864 data: 5 4 4 6 5 6 7 7 8
19856 181/300 Read block 1734 len 8192
19857 182/300 Read block 3061 len 12288
19858 183/300 Flush
19859 184/300 Read block 2381 len 32768
19860 185/300 Write block 4593 len 36864 data: 4 4 4 4 5 5 5 5 6
19861 186/300 Read block 986 len 8192
19862 187/300 Write block 724 len 36864 data: 3 3 4 5 5 6 6 7 6
19863 188/300 Read block 2718 len 32768
19864 189/300 Write block 3962 len 24576 data: 6 6 5 4 5 4
19865 190/300 Write block 1579 len 8192 data: 3 4
19866 191/300 Write block 2803 len 12288 data: 2 4 4
19867 192/300 Read block 3845 len 20480
19868 193/300 Read block 4006 len 32768
19869 194/300 Read block 90 len 4096
19870 195/300 Read block 2542 len 40960
19871 196/300 Write block 4421 len 32768 data: 4 4 3 4 4 4 4 3
19872 197/300 Write block 1693 len 16384 data: 8 5 6 6
19873 198/300 Write block 3543 len 4096 data: 6
19874 199/300 Read block 4622 len 4096
19875 200/300 Write block 4769 len 16384 data: 5 5 5 6
19876 201/300 Read block 2280 len 28672
19877 202/300 Flush
19878 203/300 Read block 557 len 8192
19879 204/300 Write block 200 len 12288 data: 5 3 3
19880 205/300 Read block 1823 len 28672
19881 206/300 Read block 848 len 20480
19882 207/300 Read block 1850 len 24576
19883 208/300 Read block 2918 len 16384
19884 209/300 Write block 3342 len 24576 data: 3 3 3 2 3 5
19885 210/300 Read block 3437 len 12288
19886 211/300 Write block 1691 len 24576 data: 6 8 9 6 7 7
19887 212/300 Write block 639 len 36864 data: 4 4 2 3 3 4 4 4 4
19888 213/300 Flush
19889 214/300 Write block 1308 len 16384 data: 2 2 2 2
19890 215/300 Read block 757 len 24576
19891 216/300 Write block 2508 len 12288 data: 3 2 2
19892 217/300 Read block 2230 len 40960
19893 218/300 Read block 4038 len 24576
19894 219/300 Read block 3076 len 16384
19895 220/300 Read block 3583 len 24576
19896 221/300 Read block 4756 len 28672
19897 222/300 Write block 151 len 8192 data: 2 4
19898 223/300 Write block 1763 len 16384 data: 9 8 7 7
19899 224/300 Read block 3267 len 20480
19900 225/300 Read block 647 len 8192
19901 226/300 Read block 1780 len 16384
19902 227/300 Read block 4663 len 40960
19903 228/300 Read block 3772 len 12288
19904 229/300 Flush
19905 230/300 Write block 4229 len 32768 data: 3 3 5 4 4 4 5 6
19906 231/300 Flush
19907 232/300 Write block 1076 len 12288 data: 5 8 7
19908 233/300 Flush
19909 234/300 Read block 364 len 12288
19910 235/300 Read block 1693 len 36864
19911 236/300 Write block 245 len 12288 data: 4 4 5
19912 237/300 Write block 1640 len 28672 data: 2 2 2 2 2 2 2
19913 238/300 Write block 2011 len 36864 data: 4 4 3 3 3 3 2 3 4
19914 239/300 Read block 2917 len 40960
19915 240/300 Read block 562 len 4096
19916 241/300 Read block 1421 len 36864
19917 242/300 Flush
19918 243/300 Read block 2395 len 20480
19919 244/300 Read block 3363 len 8192
19920 245/300 Write block 499 len 4096 data: 4
19921 246/300 Read block 4629 len 8192
19922 247/300 Read block 2441 len 8192
19923 248/300 Read block 1994 len 36864
19924 249/300 Read block 3912 len 24576
19925 250/300 Read block 3628 len 8192
19926 251/300 Write block 3363 len 40960 data: 6 6 6 6 5 5 6 5 4 3
19927 252/300 Flush
19928 253/300 Write block 4260 len 16384 data: 4 4 4 4
19929 254/300 Write block 4878 len 24576 data: 3 2 2 2 2 2
19930 255/300 Write block 2412 len 8192 data: 5 5
19931 256/300 Write block 4234 len 32768 data: 5 6 7 6 6 5 5 5
19932 257/300 Read block 3797 len 40960
19933 258/300 Write block 3850 len 8192 data: 3 2
19934 259/300 Write block 4284 len 40960 data: 3 3 3 3 3 3 2 2 2 3
19935 260/300 Write block 791 len 16384 data: 5 4 3 3
19936 261/300 Read block 522 len 8192
19937 262/300 Read block 2496 len 32768
19938 263/300 Write block 2183 len 4096 data: 5
19939 264/300 Write block 2404 len 24576 data: 5 4 4 3 4 3
19940 265/300 Write block 2867 len 4096 data: 6
19941 266/300 Write block 3353 len 12288 data: 2 2 2
19942 267/300 Write block 61 len 4096 data: 5
19943 268/300 Read block 2041 len 28672
19944 269/300 Write block 2844 len 40960 data: 6 5 6 6 6 6 4 5 7 5
19945 270/300 Read block 2738 len 8192
19946 271/300 Read block 3061 len 28672
19947 272/300 Write block 118 len 24576 data: 5 3 4 3 3 4
19948 273/300 Write block 4432 len 20480 data: 2 3 4 5 5
19949 274/300 Read block 3217 len 12288
19950 275/300 Write block 1015 len 20480 data: 5 6 5 5 5
19951 276/300 Write block 483 len 16384 data: 4 3 5 5
19952 277/300 Write block 3791 len 16384 data: 4 4 5 5
19953 278/300 Write block 1639 len 40960 data: 2 3 3 3 3 3 3 3 2 3
19954 279/300 Read block 2295 len 32768
19955 280/300 Read block 272 len 32768
19956 281/300 Write block 1524 len 32768 data: 4 7 8 7 6 7 7 6
19957 282/300 Read block 4434 len 12288
19958 283/300 Write block 1949 len 36864 data: 8 7 8 7 5 4 3 3 2
19959 284/300 Flush
19960 285/300 Read block 1542 len 40960
19961 286/300 Read block 3215 len 28672
19962 287/300 Read block 997 len 24576
19963 288/300 Read block 4872 len 20480
19964 289/300 Write block 3680 len 36864 data: 7 7 7 6 5 5 6 8 9
19965 290/300 Write block 1602 len 24576 data: 4 4 3 4 3 5
19966 291/300 Read block 3181 len 16384
19967 292/300 Read block 4369 len 12288
19968 293/300 Read block 351 len 12288
19969 294/300 Flush
19970 295/300 Write block 1985 len 40960 data: 4 4 3 4 4 4 5 4 4 4
19971 296/300 Write block 2773 len 20480 data: 7 7 7 5 5
19972 297/300 Read block 686 len 32768
19973 298/300 Read block 2204 len 16384
19974 299/300 Read block 832 len 20480
19975 300/300 Write block 4516 len 32768 data: 4 4 5 6 7 7 6 7
19976 301/300 Read block 3309 len 24576
19977 Main task has work to do, go find it
19978 got dsc Start(1)
19979 start 1
19980 Dec 04 13:34:54.791 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/1, method: POST, req_id: 8dc72670-8edf-4279-a5d5-d744f40ae526, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
19981 Replay: started 1, returned:Ok(())
19982 ----------------------------------------------------------------
19983 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19984 States: Active Offline Active
19985 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19986 6433 Acked Write 6 Done Sent Done false
19987 6434 Acked Read 4 Done Sent Done false
19988 6435 Acked Read 1 Done Sent Done false
19989 6436 Acked Write 1 Done Sent Done false
19990 6437 Acked Write 7 Done Sent Done false
19991 6438 Acked Write 1 Done Sent Done false
19992 6439 Acked Read 2 Done Sent Done false
19993 6440 Acked Read 8 Done Sent Done false
19994 6441 Acked Write 8 Done Sent Done false
19995 6442 Acked Write 7 Done Sent Done false
19996 6443 Acked Write 2 Done Sent Done false
19997 6444 Acked Write 6 Done Sent Done false
19998 6445 Acked Write 1 Done Sent Done false
19999 6446 Acked Write 2 Done Sent Done false
20000 6447 Acked Read 4 Done Sent Done false
20001 6448 Acked Read 5 Done Sent Done false
20002 6449 Acked Write 7 Done Sent Done false
20003 6450 Acked Read 2 Done Sent Done false
20004 6451 Acked Write 7 Done Sent Done false
20005 6452 Acked Read 6 Done Sent Done false
20006 6453 Acked Read 5 Done Sent Done false
20007 6454 Acked Read 2 Done Sent Done false
20008 6455 Acked Write 6 Done Sent Done false
20009 6456 Acked Write 9 Done Sent Done false
20010 6457 Acked Read 8 Done Sent Done false
20011 6458 Acked Flush 0 Done Sent Done false
20012 6459 Acked Write 8 Done Sent Done false
20013 6460 Acked Write 7 Done Sent Done false
20014 6461 Acked Read 1 Done Sent Done false
20015 6462 Acked Read 2 Done Sent Done false
20016 6463 Acked Read 9 Done Sent Done false
20017 6464 Acked Read 2 Done Sent Done false
20018 6465 Acked Read 7 Done Sent Done false
20019 6466 Acked Write 8 Done Sent Done false
20020 6467 Acked Read 1 Done Sent Done false
20021 6468 Acked Flush 0 Done Sent Done false
20022 6469 Acked Write 5 Done Sent Done false
20023 6470 Acked Read 7 Done Sent Done false
20024 6471 Acked Read 10 Done Sent Done false
20025 6472 Acked Read 8 Done Sent Done false
20026 6473 Acked Read 7 Done Sent Done false
20027 6474 Acked Read 6 Done Sent Done false
20028 6475 Acked Read 1 Done Sent Done false
20029 6476 Acked Read 4 Done Sent Done false
20030 6477 Acked Write 8 Done Sent Done false
20031 6478 Acked Write 2 Done Sent Done false
20032 6479 Acked Write 7 Done Sent Done false
20033 6480 Acked Write 1 Done Sent Done false
20034 6481 Acked Write 6 Done Sent Done false
20035 6482 Acked Read 6 Done Sent Done false
20036 6483 Acked Read 3 Done Sent Done false
20037 6484 Acked Flush 0 Done Sent Done false
20038 6485 Acked Flush 0 Done Sent Done false
20039 6486 Acked Write 6 Done Sent Done false
20040 6487 Acked Flush 0 Done Sent Done false
20041 6488 Acked Flush 0 Done Sent Done false
20042 6489 Acked Write 6 Done Sent Done false
20043 6490 Acked Write 5 Done Sent Done false
20044 6491 Acked Read 6 Done Sent Done false
20045 6492 Acked Read 2 Done Sent Done false
20046 6493 Acked Read 6 Done Sent Done false
20047 6494 Acked Read 3 Done Sent Done false
20048 6495 Acked Write 8 Done Sent Done false
20049 6496 Acked Flush 0 Done Sent Done false
20050 6497 Acked Read 3 Done Sent Done false
20051 6498 Acked Read 9 Done Sent Done false
20052 6499 Acked Flush 0 Done Sent Done false
20053 6500 Acked Read 9 Done Sent Done false
20054 6501 Acked Read 8 Done Sent Done false
20055 6502 Acked Read 8 Done Sent Done false
20056 6503 Acked Read 7 Done Sent Done false
20057 6504 Acked Read 7 Done Sent Done false
20058 6505 Acked Read 5 Done Sent Done false
20059 6506 Acked Read 2 Done Sent Done false
20060 6507 Acked Read 1 Done Sent Done false
20061 6508 Acked Write 8 Done Sent Done false
20062 6509 Acked Read 6 Done Sent Done false
20063 6510 Acked Write 7 Done Sent Done false
20064 6511 Acked Read 7 Done Sent Done false
20065 6512 Acked Write 5 Done Sent Done false
20066 6513 Acked Write 10 Done Sent Done false
20067 6514 Acked Read 2 Done Sent Done false
20068 6515 Acked Write 1 Done Sent Done false
20069 6516 Acked Read 10 Done Sent Done false
20070 6517 Acked Read 9 Done Sent Done false
20071 6518 Acked Write 6 Done Sent Done false
20072 6519 Acked Write 1 Done Sent Done false
20073 6520 Acked Write 4 Done Sent Done false
20074 6521 Acked Read 7 Done Sent Done false
20075 6522 Acked Read 9 Done Sent Done false
20076 6523 Acked Flush 0 Done Sent Done false
20077 6524 Acked Read 10 Done Sent Done false
20078 6525 Acked Read 8 Done Sent Done false
20079 6526 Acked Write 7 Done Sent Done false
20080 6527 Acked Write 3 Done Sent Done false
20081 6528 Acked Read 3 Done Sent Done false
20082 6529 Acked Write 9 Done Sent Done false
20083 6530 Acked Write 6 Done Sent Done false
20084 6531 Acked Read 2 Done Sent Done false
20085 6532 Acked Read 10 Done Sent Done false
20086 6533 Acked Read 9 Done Sent Done false
20087 6534 Acked Read 5 Done Sent Done false
20088 6535 Acked Write 2 Done Sent Done false
20089 6536 Acked Read 6 Done Sent Done false
20090 6537 Acked Read 7 Done Sent Done false
20091 6538 Acked Write 6 Done Sent Done false
20092 6539 Acked Read 5 Done Sent Done false
20093 6540 Acked Flush 0 Done Sent Done false
20094 6541 Acked Read 7 Done Sent Done false
20095 6542 Acked Write 1 Done Sent Done false
20096 6543 Acked Read 10 Done Sent Done false
20097 6544 Acked Write 3 Done Sent Done false
20098 6545 Acked Read 6 Done Sent Done false
20099 6546 Acked Write 3 Done Sent Done false
20100 6547 Acked Read 4 Done Sent Done false
20101 6548 Acked Read 6 Done Sent Done false
20102 6549 Acked Write 3 Done Sent Done false
20103 6550 Acked Read 5 Done Sent Done false
20104 6551 Acked Read 3 Done Sent Done false
20105 6552 Acked Read 8 Done Sent Done false
20106 6553 Acked Read 8 Done Sent Done false
20107 6554 Acked Read 3 Done Sent Done false
20108 6555 Acked Write 9 Done Sent Done false
20109 6556 Acked Read 7 Done Sent Done false
20110 6557 Acked Flush 0 Done Sent Done false
20111 6558 Acked Write 6 Done Sent Done false
20112 6559 Acked Read 1 Done Sent Done false
20113 6560 Acked Read 5 Done Sent Done false
20114 6561 Acked Write 1 Done Sent Done false
20115 6562 Acked Flush 0 Done Sent Done false
20116 6563 Acked Read 3 Done Sent Done false
20117 6564 Acked Read 6 Done Sent Done false
20118 6565 Acked Read 4 Done Sent Done false
20119 6566 Acked Read 5 Done Sent Done false
20120 6567 Acked Write 7 Done Sent Done false
20121 6568 Acked Flush 0 Done Sent Done false
20122 6569 Acked Write 10 Done Sent Done false
20123 6570 Acked Read 4 Done Sent Done false
20124 6571 Acked Read 3 Done Sent Done false
20125 6572 Acked Flush 0 Done Sent Done false
20126 6573 Acked Read 5 Done Sent Done false
20127 6574 Acked Read 1 Done Sent Done false
20128 6575 Acked Read 9 Done Sent Done false
20129 6576 Acked Write 7 Done Sent Done false
20130 6577 Acked Read 10 Done Sent Done false
20131 6578 Acked Read 5 Done Sent Done false
20132 6579 Acked Read 4 Done Sent Done false
20133 6580 Acked Read 5 Done Sent Done false
20134 6581 Acked Flush 0 Done Sent Done false
20135 6582 Acked Flush 0 Done Sent Done false
20136 6583 Acked Read 10 Done Sent Done false
20137 6584 Acked Read 2 Done Sent Done false
20138 6585 Acked Write 7 Done Sent Done false
20139 6586 Acked Write 4 Done Sent Done false
20140 6587 Acked Read 5 Done Sent Done false
20141 6588 Acked Read 8 Done Sent Done false
20142 6589 Acked Write 6 Done Sent Done false
20143 6590 Acked Read 8 Done Sent Done false
20144 6591 Acked Write 10 Done Sent Done false
20145 6592 Acked Write 4 Done Sent Done false
20146 6593 Acked Read 1 Done Sent Done false
20147 6594 Acked Read 5 Done Sent Done false
20148 6595 Acked Write 10 Done Sent Done false
20149 6596 Acked Read 5 Done Sent Done false
20150 6597 Acked Write 7 Done Sent Done false
20151 6598 Acked Write 5 Done Sent Done false
20152 6599 Acked Write 8 Done Sent Done false
20153 6600 Acked Write 6 Done Sent Done false
20154 6601 Acked Write 4 Done Sent Done false
20155 6602 Acked Read 3 Done Sent Done false
20156 6603 Acked Read 8 Done Sent Done false
20157 6604 Acked Write 8 Done Sent Done false
20158 6605 Acked Write 3 Done Sent Done false
20159 6606 Acked Read 2 Done Sent Done false
20160 6607 Acked Read 2 Done Sent Done false
20161 6608 Acked Write 3 Done Sent Done false
20162 6609 Acked Read 1 Done Sent Done false
20163 6610 Acked Read 9 Done Sent Done false
20164 6611 Acked Write 7 Done Sent Done false
20165 6612 Acked Write 9 Done Sent Done false
20166 6613 Acked Read 2 Done Sent Done false
20167 6614 Acked Read 3 Done Sent Done false
20168 6615 Acked Flush 0 Done Sent Done false
20169 6616 Acked Read 8 Done Sent Done false
20170 6617 Acked Write 9 Done Sent Done false
20171 6618 Acked Read 2 Done Sent Done false
20172 6619 Acked Write 9 Done Sent Done false
20173 6620 Acked Read 8 Done Sent Done false
20174 6621 Acked Write 6 Done Sent Done false
20175 6622 Acked Write 2 Done Sent Done false
20176 6623 Acked Write 3 Done Sent Done false
20177 6624 Acked Read 5 Done Sent Done false
20178 6625 Acked Read 8 Done Sent Done false
20179 6626 Acked Read 1 Done Sent Done false
20180 6627 Acked Read 10 Done Sent Done false
20181 6628 Acked Write 8 Done Sent Done false
20182 6629 Acked Write 4 Done Sent Done false
20183 6630 Acked Write 1 Done Sent Done false
20184 6631 Acked Read 1 Done Sent Done false
20185 6632 Acked Write 4 Done Sent Done false
20186 6633 Acked Read 7 Done Sent Done false
20187 6634 Acked Flush 0 Done Sent Done false
20188 6635 Acked Read 2 Done Sent Done false
20189 6636 Acked Write 3 Done Sent Done false
20190 6637 Acked Read 7 Done Sent Done false
20191 6638 Acked Read 5 Done Sent Done false
20192 6639 Acked Read 6 Done Sent Done false
20193 6640 Acked Read 4 Done Sent Done false
20194 6641 Acked Write 6 Done Sent Done false
20195 6642 Acked Read 3 Done Sent Done false
20196 6643 Acked Write 6 Done Sent Done false
20197 6644 Acked Write 9 Done Sent Done false
20198 6645 Acked Flush 0 Done Sent Done false
20199 6646 Acked Write 4 Done Sent Done false
20200 6647 Acked Read 6 Done Sent Done false
20201 6648 Acked Write 3 Done Sent Done false
20202 6649 Acked Read 10 Done Sent Done false
20203 6650 Acked Read 6 Done Sent Done false
20204 6651 Acked Read 4 Done Sent Done false
20205 6652 Acked Read 6 Done Sent Done false
20206 6653 Acked Read 7 Done Sent Done false
20207 6654 Acked Write 2 Done Sent Done false
20208 6655 Acked Write 4 Done Sent Done false
20209 6656 Acked Read 5 Done Sent Done false
20210 6657 Acked Read 2 Done Sent Done false
20211 6658 Acked Read 4 Done Sent Done false
20212 6659 Acked Read 10 Done Sent Done false
20213 6660 Acked Read 3 Done Sent Done false
20214 6661 Acked Flush 0 Done Sent Done false
20215 6662 Acked Write 8 Done Sent Done false
20216 6663 Acked Flush 0 Done Sent Done false
20217 6664 Acked Write 3 Done Sent Done false
20218 6665 Acked Flush 0 Done Sent Done false
20219 6666 Acked Read 3 Done Sent Done false
20220 6667 Acked Read 9 Done Sent Done false
20221 6668 Acked Write 3 Done Sent Done false
20222 6669 Acked Write 7 Done Sent Done false
20223 6670 Acked Write 9 Done Sent Done false
20224 6671 Acked Read 10 Done Sent Done false
20225 6672 Acked Read 1 Done Sent Done false
20226 6673 Acked Read 9 Done Sent Done false
20227 6674 Acked Flush 0 Done Sent Done false
20228 6675 Acked Read 5 Done Sent Done false
20229 6676 Acked Read 2 Done Sent Done false
20230 6677 Acked Write 1 Done Sent Done false
20231 6678 Acked Read 2 Done Sent Done false
20232 6679 Acked Read 2 Done Sent Done false
20233 6680 Acked Read 9 Done Sent Done false
20234 6681 Acked Read 6 Done Sent Done false
20235 6682 Acked Read 2 Done Sent Done false
20236 6683 Acked Write 10 Done Sent Done false
20237 6684 Acked Flush 0 Done Sent Done false
20238 6685 Acked Write 4 Done Sent Done false
20239 6686 Acked Write 6 Done Sent Done false
20240 6687 Acked Write 2 Done Sent Done false
20241 6688 Acked Write 8 Done Sent Done false
20242 6689 Acked Read 10 Done Sent Done false
20243 6690 Acked Write 2 Done Sent Done false
20244 6691 Acked Write 10 Done Sent Done false
20245 6692 Acked Write 4 Done Sent Done false
20246 6693 Acked Read 2 Done Sent Done false
20247 6694 Acked Read 8 Done Sent Done false
20248 6695 Acked Write 1 Done Sent Done false
20249 6696 Acked Write 6 Done Sent Done false
20250 6697 Acked Write 1 Done Sent Done false
20251 6698 Acked Write 3 Done Sent Done false
20252 6699 Acked Write 1 Done Sent Done false
20253 6700 Acked Read 7 Done Sent Done false
20254 6701 Acked Write 10 Done Sent Done false
20255 6702 Acked Read 2 Done Sent Done false
20256 6703 Acked Read 7 Done Sent Done false
20257 6704 Acked Write 6 Done Sent Done false
20258 6705 Acked Write 5 Done Sent Done false
20259 6706 Acked Read 3 Done Sent Done false
20260 6707 Acked Write 5 Done Sent Done false
20261 6708 Acked Write 4 Done Sent Done false
20262 6709 Acked Write 4 Done Sent Done false
20263 6710 Acked Write 10 Done Sent Done false
20264 6711 Acked Read 8 Done Sent Done false
20265 6712 Acked Read 8 Done Sent Done false
20266 6713 Acked Write 8 Done Sent Done false
20267 6714 Acked Read 3 Done Sent Done false
20268 6715 Acked Write 9 Done Sent Done false
20269 6716 Acked Flush 0 Done Sent Done false
20270 6717 Acked Read 10 Done Sent Done false
20271 6718 Acked Read 7 Done Sent Done false
20272 6719 Acked Read 6 Done Sent Done false
20273 6720 Acked Read 5 Done Sent Done false
20274 6721 Acked Write 9 Done Sent Done false
20275 6722 Acked Write 6 Done Sent Done false
20276 6723 Acked Read 4 Done Sent Done false
20277 6724 Acked Read 3 Done Sent Done false
20278 6725 Acked Read 3 Done Sent Done false
20279 6726 Acked Flush 0 Done Sent Done false
20280 6727 Acked Write 10 Done Sent Done false
20281 6728 Acked Write 5 Done Sent Done false
20282 6729 Acked Read 8 Done Sent Done false
20283 6730 Acked Read 4 Done Sent Done false
20284 6731 Acked Read 5 Done Sent Done false
20285 6732 Acked Write 8 Done Sent Done false
20286 6733 Acked Read 6 Done Sent Done false
20287 STATES DS:0 DS:1 DS:2 TOTAL
20288 Sent 0 301 0 301
20289 Done 301 0 301 602
20290 Skipped 0 0 0 0
20291 Error 0 0 0 0
20292 Last Flush: 6726 6432 6726
20293 Downstairs last five completed: 6432 6431 6430 6429 6428
20294 Upstairs last five completed: 6733 6732 6731 6730 6729
20295 CLIENT: Up:0 ds:301 act:2
20296 [8820] got start action
20297 [8820] I am going to restart
20298 Starting downstairs at port 8820
20299 Make output file at "/tmp/dsc/downstairs-8820.txt"
20300 [8820][1] reports Starting
20301 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1222)
20302 [8820][1] reports Running
20303 ----------------------------------------------------------------
20304 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20305 States: Active Offline Active
20306 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20307 6433 Acked Write 6 Done Sent Done false
20308 6434 Acked Read 4 Done Sent Done false
20309 6435 Acked Read 1 Done Sent Done false
20310 6436 Acked Write 1 Done Sent Done false
20311 6437 Acked Write 7 Done Sent Done false
20312 6438 Acked Write 1 Done Sent Done false
20313 6439 Acked Read 2 Done Sent Done false
20314 6440 Acked Read 8 Done Sent Done false
20315 6441 Acked Write 8 Done Sent Done false
20316 6442 Acked Write 7 Done Sent Done false
20317 6443 Acked Write 2 Done Sent Done false
20318 6444 Acked Write 6 Done Sent Done false
20319 6445 Acked Write 1 Done Sent Done false
20320 6446 Acked Write 2 Done Sent Done false
20321 6447 Acked Read 4 Done Sent Done false
20322 6448 Acked Read 5 Done Sent Done false
20323 6449 Acked Write 7 Done Sent Done false
20324 6450 Acked Read 2 Done Sent Done false
20325 6451 Acked Write 7 Done Sent Done false
20326 6452 Acked Read 6 Done Sent Done false
20327 6453 Acked Read 5 Done Sent Done false
20328 6454 Acked Read 2 Done Sent Done false
20329 6455 Acked Write 6 Done Sent Done false
20330 6456 Acked Write 9 Done Sent Done false
20331 6457 Acked Read 8 Done Sent Done false
20332 6458 Acked Flush 0 Done Sent Done false
20333 6459 Acked Write 8 Done Sent Done false
20334 6460 Acked Write 7 Done Sent Done false
20335 6461 Acked Read 1 Done Sent Done false
20336 6462 Acked Read 2 Done Sent Done false
20337 6463 Acked Read 9 Done Sent Done false
20338 6464 Acked Read 2 Done Sent Done false
20339 6465 Acked Read 7 Done Sent Done false
20340 6466 Acked Write 8 Done Sent Done false
20341 6467 Acked Read 1 Done Sent Done false
20342 6468 Acked Flush 0 Done Sent Done false
20343 6469 Acked Write 5 Done Sent Done false
20344 6470 Acked Read 7 Done Sent Done false
20345 6471 Acked Read 10 Done Sent Done false
20346 6472 Acked Read 8 Done Sent Done false
20347 6473 Acked Read 7 Done Sent Done false
20348 6474 Acked Read 6 Done Sent Done false
20349 6475 Acked Read 1 Done Sent Done false
20350 6476 Acked Read 4 Done Sent Done false
20351 6477 Acked Write 8 Done Sent Done false
20352 6478 Acked Write 2 Done Sent Done false
20353 6479 Acked Write 7 Done Sent Done false
20354 6480 Acked Write 1 Done Sent Done false
20355 6481 Acked Write 6 Done Sent Done false
20356 6482 Acked Read 6 Done Sent Done false
20357 6483 Acked Read 3 Done Sent Done false
20358 6484 Acked Flush 0 Done Sent Done false
20359 6485 Acked Flush 0 Done Sent Done false
20360 6486 Acked Write 6 Done Sent Done false
20361 6487 Acked Flush 0 Done Sent Done false
20362 6488 Acked Flush 0 Done Sent Done false
20363 6489 Acked Write 6 Done Sent Done false
20364 6490 Acked Write 5 Done Sent Done false
20365 6491 Acked Read 6 Done Sent Done false
20366 6492 Acked Read 2 Done Sent Done false
20367 6493 Acked Read 6 Done Sent Done false
20368 6494 Acked Read 3 Done Sent Done false
20369 6495 Acked Write 8 Done Sent Done false
20370 6496 Acked Flush 0 Done Sent Done false
20371 6497 Acked Read 3 Done Sent Done false
20372 6498 Acked Read 9 Done Sent Done false
20373 6499 Acked Flush 0 Done Sent Done false
20374 6500 Acked Read 9 Done Sent Done false
20375 6501 Acked Read 8 Done Sent Done false
20376 6502 Acked Read 8 Done Sent Done false
20377 6503 Acked Read 7 Done Sent Done false
20378 6504 Acked Read 7 Done Sent Done false
20379 6505 Acked Read 5 Done Sent Done false
20380 6506 Acked Read 2 Done Sent Done false
20381 6507 Acked Read 1 Done Sent Done false
20382 6508 Acked Write 8 Done Sent Done false
20383 6509 Acked Read 6 Done Sent Done false
20384 6510 Acked Write 7 Done Sent Done false
20385 6511 Acked Read 7 Done Sent Done false
20386 6512 Acked Write 5 Done Sent Done false
20387 6513 Acked Write 10 Done Sent Done false
20388 6514 Acked Read 2 Done Sent Done false
20389 6515 Acked Write 1 Done Sent Done false
20390 6516 Acked Read 10 Done Sent Done false
20391 6517 Acked Read 9 Done Sent Done false
20392 6518 Acked Write 6 Done Sent Done false
20393 6519 Acked Write 1 Done Sent Done false
20394 6520 Acked Write 4 Done Sent Done false
20395 6521 Acked Read 7 Done Sent Done false
20396 6522 Acked Read 9 Done Sent Done false
20397 6523 Acked Flush 0 Done Sent Done false
20398 6524 Acked Read 10 Done Sent Done false
20399 6525 Acked Read 8 Done Sent Done false
20400 6526 Acked Write 7 Done Sent Done false
20401 6527 Acked Write 3 Done Sent Done false
20402 6528 Acked Read 3 Done Sent Done false
20403 6529 Acked Write 9 Done Sent Done false
20404 6530 Acked Write 6 Done Sent Done false
20405 6531 Acked Read 2 Done Sent Done false
20406 6532 Acked Read 10 Done Sent Done false
20407 6533 Acked Read 9 Done Sent Done false
20408 6534 Acked Read 5 Done Sent Done false
20409 6535 Acked Write 2 Done Sent Done false
20410 6536 Acked Read 6 Done Sent Done false
20411 6537 Acked Read 7 Done Sent Done false
20412 6538 Acked Write 6 Done Sent Done false
20413 6539 Acked Read 5 Done Sent Done false
20414 6540 Acked Flush 0 Done Sent Done false
20415 6541 Acked Read 7 Done Sent Done false
20416 6542 Acked Write 1 Done Sent Done false
20417 6543 Acked Read 10 Done Sent Done false
20418 6544 Acked Write 3 Done Sent Done false
20419 6545 Acked Read 6 Done Sent Done false
20420 6546 Acked Write 3 Done Sent Done false
20421 6547 Acked Read 4 Done Sent Done false
20422 6548 Acked Read 6 Done Sent Done false
20423 6549 Acked Write 3 Done Sent Done false
20424 6550 Acked Read 5 Done Sent Done false
20425 6551 Acked Read 3 Done Sent Done false
20426 6552 Acked Read 8 Done Sent Done false
20427 6553 Acked Read 8 Done Sent Done false
20428 6554 Acked Read 3 Done Sent Done false
20429 6555 Acked Write 9 Done Sent Done false
20430 6556 Acked Read 7 Done Sent Done false
20431 6557 Acked Flush 0 Done Sent Done false
20432 6558 Acked Write 6 Done Sent Done false
20433 6559 Acked Read 1 Done Sent Done false
20434 6560 Acked Read 5 Done Sent Done false
20435 6561 Acked Write 1 Done Sent Done false
20436 6562 Acked Flush 0 Done Sent Done false
20437 6563 Acked Read 3 Done Sent Done false
20438 6564 Acked Read 6 Done Sent Done false
20439 6565 Acked Read 4 Done Sent Done false
20440 6566 Acked Read 5 Done Sent Done false
20441 6567 Acked Write 7 Done Sent Done false
20442 6568 Acked Flush 0 Done Sent Done false
20443 6569 Acked Write 10 Done Sent Done false
20444 6570 Acked Read 4 Done Sent Done false
20445 6571 Acked Read 3 Done Sent Done false
20446 6572 Acked Flush 0 Done Sent Done false
20447 6573 Acked Read 5 Done Sent Done false
20448 6574 Acked Read 1 Done Sent Done false
20449 6575 Acked Read 9 Done Sent Done false
20450 6576 Acked Write 7 Done Sent Done false
20451 6577 Acked Read 10 Done Sent Done false
20452 6578 Acked Read 5 Done Sent Done false
20453 6579 Acked Read 4 Done Sent Done false
20454 6580 Acked Read 5 Done Sent Done false
20455 6581 Acked Flush 0 Done Sent Done false
20456 6582 Acked Flush 0 Done Sent Done false
20457 6583 Acked Read 10 Done Sent Done false
20458 6584 Acked Read 2 Done Sent Done false
20459 6585 Acked Write 7 Done Sent Done false
20460 6586 Acked Write 4 Done Sent Done false
20461 6587 Acked Read 5 Done Sent Done false
20462 6588 Acked Read 8 Done Sent Done false
20463 6589 Acked Write 6 Done Sent Done false
20464 6590 Acked Read 8 Done Sent Done false
20465 6591 Acked Write 10 Done Sent Done false
20466 6592 Acked Write 4 Done Sent Done false
20467 6593 Acked Read 1 Done Sent Done false
20468 6594 Acked Read 5 Done Sent Done false
20469 6595 Acked Write 10 Done Sent Done false
20470 6596 Acked Read 5 Done Sent Done false
20471 6597 Acked Write 7 Done Sent Done false
20472 6598 Acked Write 5 Done Sent Done false
20473 6599 Acked Write 8 Done Sent Done false
20474 6600 Acked Write 6 Done Sent Done false
20475 6601 Acked Write 4 Done Sent Done false
20476 6602 Acked Read 3 Done Sent Done false
20477 6603 Acked Read 8 Done Sent Done false
20478 6604 Acked Write 8 Done Sent Done false
20479 6605 Acked Write 3 Done Sent Done false
20480 6606 Acked Read 2 Done Sent Done false
20481 6607 Acked Read 2 Done Sent Done false
20482 6608 Acked Write 3 Done Sent Done false
20483 6609 Acked Read 1 Done Sent Done false
20484 6610 Acked Read 9 Done Sent Done false
20485 6611 Acked Write 7 Done Sent Done false
20486 6612 Acked Write 9 Done Sent Done false
20487 6613 Acked Read 2 Done Sent Done false
20488 6614 Acked Read 3 Done Sent Done false
20489 6615 Acked Flush 0 Done Sent Done false
20490 6616 Acked Read 8 Done Sent Done false
20491 6617 Acked Write 9 Done Sent Done false
20492 6618 Acked Read 2 Done Sent Done false
20493 6619 Acked Write 9 Done Sent Done false
20494 6620 Acked Read 8 Done Sent Done false
20495 6621 Acked Write 6 Done Sent Done false
20496 6622 Acked Write 2 Done Sent Done false
20497 6623 Acked Write 3 Done Sent Done false
20498 6624 Acked Read 5 Done Sent Done false
20499 6625 Acked Read 8 Done Sent Done false
20500 6626 Acked Read 1 Done Sent Done false
20501 6627 Acked Read 10 Done Sent Done false
20502 6628 Acked Write 8 Done Sent Done false
20503 6629 Acked Write 4 Done Sent Done false
20504 6630 Acked Write 1 Done Sent Done false
20505 6631 Acked Read 1 Done Sent Done false
20506 6632 Acked Write 4 Done Sent Done false
20507 6633 Acked Read 7 Done Sent Done false
20508 6634 Acked Flush 0 Done Sent Done false
20509 6635 Acked Read 2 Done Sent Done false
20510 6636 Acked Write 3 Done Sent Done false
20511 6637 Acked Read 7 Done Sent Done false
20512 6638 Acked Read 5 Done Sent Done false
20513 6639 Acked Read 6 Done Sent Done false
20514 6640 Acked Read 4 Done Sent Done false
20515 6641 Acked Write 6 Done Sent Done false
20516 6642 Acked Read 3 Done Sent Done false
20517 6643 Acked Write 6 Done Sent Done false
20518 6644 Acked Write 9 Done Sent Done false
20519 6645 Acked Flush 0 Done Sent Done false
20520 6646 Acked Write 4 Done Sent Done false
20521 6647 Acked Read 6 Done Sent Done false
20522 6648 Acked Write 3 Done Sent Done false
20523 6649 Acked Read 10 Done Sent Done false
20524 6650 Acked Read 6 Done Sent Done false
20525 6651 Acked Read 4 Done Sent Done false
20526 6652 Acked Read 6 Done Sent Done false
20527 6653 Acked Read 7 Done Sent Done false
20528 6654 Acked Write 2 Done Sent Done false
20529 6655 Acked Write 4 Done Sent Done false
20530 6656 Acked Read 5 Done Sent Done false
20531 6657 Acked Read 2 Done Sent Done false
20532 6658 Acked Read 4 Done Sent Done false
20533 6659 Acked Read 10 Done Sent Done false
20534 6660 Acked Read 3 Done Sent Done false
20535 6661 Acked Flush 0 Done Sent Done false
20536 6662 Acked Write 8 Done Sent Done false
20537 6663 Acked Flush 0 Done Sent Done false
20538 6664 Acked Write 3 Done Sent Done false
20539 6665 Acked Flush 0 Done Sent Done false
20540 6666 Acked Read 3 Done Sent Done false
20541 6667 Acked Read 9 Done Sent Done false
20542 6668 Acked Write 3 Done Sent Done false
20543 6669 Acked Write 7 Done Sent Done false
20544 6670 Acked Write 9 Done Sent Done false
20545 6671 Acked Read 10 Done Sent Done false
20546 6672 Acked Read 1 Done Sent Done false
20547 6673 Acked Read 9 Done Sent Done false
20548 6674 Acked Flush 0 Done Sent Done false
20549 6675 Acked Read 5 Done Sent Done false
20550 6676 Acked Read 2 Done Sent Done false
20551 6677 Acked Write 1 Done Sent Done false
20552 6678 Acked Read 2 Done Sent Done false
20553 6679 Acked Read 2 Done Sent Done false
20554 6680 Acked Read 9 Done Sent Done false
20555 6681 Acked Read 6 Done Sent Done false
20556 6682 Acked Read 2 Done Sent Done false
20557 6683 Acked Write 10 Done Sent Done false
20558 6684 Acked Flush 0 Done Sent Done false
20559 6685 Acked Write 4 Done Sent Done false
20560 6686 Acked Write 6 Done Sent Done false
20561 6687 Acked Write 2 Done Sent Done false
20562 6688 Acked Write 8 Done Sent Done false
20563 6689 Acked Read 10 Done Sent Done false
20564 6690 Acked Write 2 Done Sent Done false
20565 6691 Acked Write 10 Done Sent Done false
20566 6692 Acked Write 4 Done Sent Done false
20567 6693 Acked Read 2 Done Sent Done false
20568 6694 Acked Read 8 Done Sent Done false
20569 6695 Acked Write 1 Done Sent Done false
20570 6696 Acked Write 6 Done Sent Done false
20571 6697 Acked Write 1 Done Sent Done false
20572 6698 Acked Write 3 Done Sent Done false
20573 6699 Acked Write 1 Done Sent Done false
20574 6700 Acked Read 7 Done Sent Done false
20575 6701 Acked Write 10 Done Sent Done false
20576 6702 Acked Read 2 Done Sent Done false
20577 6703 Acked Read 7 Done Sent Done false
20578 6704 Acked Write 6 Done Sent Done false
20579 6705 Acked Write 5 Done Sent Done false
20580 6706 Acked Read 3 Done Sent Done false
20581 6707 Acked Write 5 Done Sent Done false
20582 6708 Acked Write 4 Done Sent Done false
20583 6709 Acked Write 4 Done Sent Done false
20584 6710 Acked Write 10 Done Sent Done false
20585 6711 Acked Read 8 Done Sent Done false
20586 6712 Acked Read 8 Done Sent Done false
20587 6713 Acked Write 8 Done Sent Done false
20588 6714 Acked Read 3 Done Sent Done false
20589 6715 Acked Write 9 Done Sent Done false
20590 6716 Acked Flush 0 Done Sent Done false
20591 6717 Acked Read 10 Done Sent Done false
20592 6718 Acked Read 7 Done Sent Done false
20593 6719 Acked Read 6 Done Sent Done false
20594 6720 Acked Read 5 Done Sent Done false
20595 6721 Acked Write 9 Done Sent Done false
20596 6722 Acked Write 6 Done Sent Done false
20597 6723 Acked Read 4 Done Sent Done false
20598 6724 Acked Read 3 Done Sent Done false
20599 6725 Acked Read 3 Done Sent Done false
20600 6726 Acked Flush 0 Done Sent Done false
20601 6727 Acked Write 10 Done Sent Done false
20602 6728 Acked Write 5 Done Sent Done false
20603 6729 Acked Read 8 Done Sent Done false
20604 6730 Acked Read 4 Done Sent Done false
20605 6731 Acked Read 5 Done Sent Done false
20606 6732 Acked Write 8 Done Sent Done false
20607 6733 Acked Read 6 Done Sent Done false
20608 6734 Acked Flush 0 Done Sent Done false
20609 STATES DS:0 DS:1 DS:2 TOTAL
20610 Sent 0 302 0 302
20611 Done 302 0 302 604
20612 Skipped 0 0 0 0
20613 Error 0 0 0 0
20614 Last Flush: 6734 6432 6734
20615 Downstairs last five completed: 6432 6431 6430 6429 6428
20616 Upstairs last five completed: 6734 6733 6732 6731 6730
20617 CLIENT: Up:0 ds:302 act:2
206182024-12-04T13:35:00.552ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206192024-12-04T13:35:00.552ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44162) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206202024-12-04T13:35:00.553ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206212024-12-04T13:35:00.553ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206222024-12-04T13:35:00.553ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206232024-12-04T13:35:00.553ZINFOcrucible: send last flush ID to this DS: 6432 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206242024-12-04T13:35:00.554ZINFOcrucible: Replied this last flush ID: 6432 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206252024-12-04T13:35:00.554ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206262024-12-04T13:35:00.554ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206272024-12-04T13:35:00.554ZINFOcrucible: [1] client re-new 302 jobs since flush 6432 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
206282024-12-04T13:35:00.554ZINFOcrucible: [1] Replayed 302 jobs since flush: 6432 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
20629 ----------------------------------------------------------------
20630 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20631 Downstairs last five completed: 6734 6733 6732 6731 6730
20632 Upstairs last five completed: 6734 6733 6732 6731 6730
20633 CLIENT: Up:0 ds:0 act:3
20634 Replay: All jobs finished
20635 Main task has work to do, go find it
20636 got dsc Stop(0)
20637 stop 0
20638 Dec 04 13:35:02.795 INFO request completed, latency_us: [8810] Got stop action so:false kr:false
20639 334, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 934a8a08-edaf-4aba-b6d9-98a2b1c23786, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
206402024-12-04T13:35:02.796ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
20641 Dec 04 13:35:02.796{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:35:02.796753626Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
20642 INFO request completed, latency_us: 142, response_code: 200, uri: /state/cid/0{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:35:02.796807996Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
20643 , method: GET, req_id: d9204084-b125-44b9-860b-cb9cb6691440, remote_addr: {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:35:02.796829296Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
20644 127.0{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:35:02.796879977Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
20645 .0.1:37804{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:35:02.796913107Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
20646 , local_addr: 127.0.0.1:9998
20647 [8810] Exited with: ExitStatus(unix_wait_status(9))
20648 [8810][0] reports Exit
20649 Dec 04 13:35:06.799 INFO request completed, latency_us: 231, response_code: 200, uri: /state/cid/0, method: GET, req_id: 15ca9356-298b-4bbb-870c-9d6c87b51d0e, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
20650 001/300 Read block 55 len 16384
20651 002/300 Read block 2162 len 20480
20652 003/300 Read block 1646 len 16384
20653 004/300 Flush
20654 005/300 Read block 1368 len 24576
20655 006/300 Read block 3603 len 40960
20656 007/300 Read block 1982 len 36864
20657 008/300 Write block 2909 len 36864 data: 3 3 2 2 2 2 2 2 2
20658 009/300 Write block 1919 len 32768 data: 3 3 2 2 2 2 2 2
20659 010/300 Flush
20660 011/300 Read block 3474 len 12288
20661 012/300 Write block 4227 len 12288 data: 3 3 4
20662 013/300 Read block 2206 len 4096
20663 014/300 Read block 1315 len 32768
20664 015/300 Read block 4457 len 36864
20665 016/300 Read block 1730 len 28672
20666 017/300 Write block 811 len 4096 data: 8
20667 018/300 Write block 3609 len 32768 data: 4 5 4 4 4 5 5 4
20668 019/300 Flush
20669 020/300 Read block 4324 len 20480
20670 021/300 Write block 3367 len 16384 data: 6 6 7 6
20671 022/300 Read block 966 len 12288
20672 023/300 Read block 758 len 24576
20673 024/300 Write block 2509 len 24576 data: 3 3 2 3 3 3
20674 025/300 Read block 4676 len 40960
20675 026/300 Flush
20676 027/300 Write block 852 len 4096 data: 3
20677 028/300 Write block 4682 len 36864 data: 2 3 4 5 4 4 4 4 5
20678 029/300 Read block 4834 len 36864
20679 030/300 Flush
20680 031/300 Write block 1141 len 40960 data: 5 5 4 3 3 3 4 4 4 4
20681 032/300 Read block 1684 len 20480
20682 033/300 Read block 2340 len 12288
20683 034/300 Write block 3257 len 20480 data: 2 3 3 3 3
20684 035/300 Read block 4977 len 24576
20685 036/300 Write block 2779 len 16384 data: 3 3 4 5
20686 037/300 Write block 2409 len 24576 data: 4 5 5 6 6 5
20687 038/300 Read block 2475 len 32768
20688 039/300 Flush
20689 040/300 Write block 4575 len 24576 data: 3 2 4 4 3 3
20690 041/300 Write block 541 len 36864 data: 7 6 6 6 6 6 5 4 3
20691 042/300 Read block 56 len 28672
20692 043/300 Write block 3444 len 28672 data: 6 5 5 4 3 3 4
20693 044/300 Write block 1918 len 32768 data: 4 4 4 3 3 3 3 3
20694 045/300 Read block 527 len 4096
20695 046/300 Write block 2872 len 36864 data: 7 6 7 6 5 3 4 4 4
20696 047/300 Read block 109 len 24576
20697 048/300 Write block 3691 len 28672 data: 6 7 7 5 5 5 5
20698 049/300 Read block 2317 len 16384
20699 050/300 Write block 3230 len 16384 data: 6 5 5 5
20700 051/300 Read block 3786 len 40960
20701 052/300 Write block 2109 len 16384 data: 6 6 4 3
20702 053/300 Read block 1261 len 40960
20703 054/300 Write block 306 len 28672 data: 5 5 3 3 3 3 4
20704 055/300 Write block 508 len 16384 data: 4 6 6 6
20705 056/300 Write block 4676 len 20480 data: 4 5 5 5 4
20706 057/300 Write block 1902 len 4096 data: 4
20707 058/300 Read block 1117 len 40960
20708 059/300 Read block 2469 len 4096
20709 060/300 Write block 2228 len 36864 data: 3 3 3 3 3 3 5 5 5
20710 061/300 Flush
20711 062/300 Read block 4049 len 16384
20712 063/300 Write block 4675 len 8192 data: 4 5
20713 064/300 Write block 1357 len 24576 data: 3 3 4 6 5 5
20714 065/300 Read block 3250 len 36864
20715 066/300 Read block 3953 len 40960
20716 067/300 Write block 4735 len 8192 data: 5 5
20717 068/300 Flush
20718 069/300 Read block 1950 len 8192
20719 070/300 Read block 2094 len 4096
20720 071/300 Read block 3961 len 16384
20721 072/300 Read block 4303 len 8192
20722 073/300 Write block 2567 len 36864 data: 4 5 5 7 7 4 4 4 3
20723 074/300 Write block 3750 len 12288 data: 3 3 4
20724 075/300 Write block 2989 len 32768 data: 6 5 4 4 4 4 3 4
20725 076/300 Flush
20726 077/300 Write block 3140 len 8192 data: 7 7
20727 078/300 Read block 2233 len 40960
20728 079/300 Flush
20729 080/300 Write block 397 len 40960 data: 5 5 4 4 4 4 5 6 6 5
20730 081/300 Read block 1881 len 8192
20731 082/300 Flush
20732 083/300 Write block 1522 len 32768 data: 3 4 5 8 9 8 7 8
20733 084/300 Read block 1158 len 16384
20734 085/300 Write block 434 len 40960 data: 2 3 3 3 5 4 5 6 8 9
20735 086/300 Read block 2120 len 40960
20736 087/300 Read block 863 len 8192
20737 088/300 Flush
20738 089/300 Read block 2415 len 36864
20739 090/300 Write block 1590 len 4096 data: 2
20740 091/300 Read block 844 len 16384
20741 092/300 Write block 3811 len 8192 data: 4 4
20742 093/300 Write block 3690 len 40960 data: 6 7 8 8 6 6 6 6 5 5
20743 094/300 Read block 716 len 40960
20744 095/300 Write block 4557 len 28672 data: 3 3 2 3 3 2 2
20745 096/300 Read block 651 len 4096
20746 097/300 Read block 517 len 40960
20747 098/300 Write block 122 len 24576 data: 4 5 4 4 5 4
20748 099/300 Read block 2569 len 28672
20749 100/300 Flush
20750 101/300 Write block 3310 len 28672 data: 4 4 4 4 4 6 5
20751 102/300 Flush
20752 103/300 Flush
20753 104/300 Read block 786 len 36864
20754 105/300 Read block 4663 len 24576
20755 106/300 Write block 1362 len 16384 data: 6 4 4 3
20756 107/300 Write block 2934 len 36864 data: 3 5 5 4 3 2 2 3 4
20757 108/300 Write block 3474 len 32768 data: 6 8 9 9 9 9 10 8
20758 109/300 Read block 578 len 20480
20759 110/300 Flush
20760 111/300 Read block 4869 len 40960
20761 112/300 Write block 1793 len 36864 data: 3 3 2 2 2 2 2 2 2
20762 113/300 Read block 4889 len 36864
20763 114/300 Read block 2700 len 16384
20764 115/300 Flush
20765 116/300 Write block 2691 len 40960 data: 3 3 3 2 2 2 2 2 2 2
20766 117/300 Flush
20767 118/300 Read block 4718 len 40960
20768 119/300 Read block 2355 len 28672
20769 120/300 Write block 4363 len 28672 data: 3 3 3 3 2 2 2
20770 121/300 Flush
20771 122/300 Read block 4307 len 36864
20772 123/300 Write block 4291 len 40960 data: 3 3 4 3 3 4 4 4 5 5
20773 124/300 Write block 4064 len 20480 data: 4 4 4 3 3
20774 125/300 Read block 3707 len 16384
20775 126/300 Write block 323 len 12288 data: 5 5 3
20776 127/300 Read block 1030 len 40960
20777 128/300 Flush
20778 129/300 Read block 3544 len 16384
20779 130/300 Read block 1488 len 8192
20780 131/300 Write block 453 len 24576 data: 7 6 7 8 8 9
20781 132/300 Flush
20782 133/300 Write block 2629 len 8192 data: 4 3
20783 134/300 Read block 2261 len 4096
20784 135/300 Flush
20785 136/300 Write block 4075 len 20480 data: 3 4 4 3 3
20786 137/300 Write block 4793 len 36864 data: 3 3 4 4 4 6 5 5 6
20787 138/300 Flush
20788 139/300 Read block 267 len 40960
20789 140/300 Write block 2139 len 24576 data: 6 7 6 6 6 5
20790 141/300 Write block 311 len 24576 data: 4 5 4 3 4 5
20791 142/300 Read block 2273 len 8192
20792 143/300 Read block 1341 len 8192
20793 144/300 Write block 3969 len 20480 data: 4 4 4 4 4
20794 145/300 Read block 1552 len 4096
20795 146/300 Read block 4861 len 20480
20796 147/300 Write block 2797 len 20480 data: 4 5 4 3 3
20797 148/300 Read block 4135 len 40960
20798 149/300 Write block 387 len 24576 data: 4 4 5 6 6 6
20799 150/300 Read block 2955 len 4096
20800 151/300 Read block 4072 len 36864
20801 152/300 Write block 1212 len 20480 data: 6 6 6 7 6
20802 153/300 Read block 407 len 40960
20803 154/300 Write block 4114 len 28672 data: 5 4 4 4 5 6 7
20804 155/300 Read block 4819 len 4096
20805 156/300 Read block 2706 len 40960
20806 157/300 Write block 371 len 12288 data: 6 6 6
20807 158/300 Write block 1608 len 40960 data: 5 4 4 5 5 5 5 4 3 3
20808 159/300 Write block 4010 len 16384 data: 7 5 4 3
20809 160/300 Write block 4187 len 8192 data: 3 3
20810 161/300 Read block 3245 len 32768
20811 162/300 Read block 1461 len 32768
20812 163/300 Write block 3526 len 24576 data: 4 6 6 7 6 6
20813 164/300 Write block 1412 len 16384 data: 3 3 4 4
20814 165/300 Read block 3134 len 24576
20815 166/300 Read block 4409 len 8192
20816 167/300 Read block 1479 len 4096
20817 168/300 Read block 4704 len 20480
20818 169/300 Read block 2309 len 12288
20819 170/300 Read block 2291 len 4096
20820 171/300 Read block 384 len 36864
20821 172/300 Read block 2555 len 8192
20822 173/300 Write block 3837 len 24576 data: 2 2 2 2 2 2
20823 174/300 Read block 3762 len 36864
20824 175/300 Read block 4671 len 28672
20825 176/300 Read block 2700 len 16384
20826 177/300 Read block 1771 len 40960
20827 178/300 Write block 1911 len 12288 data: 6 6 6
20828 179/300 Write block 3586 len 36864 data: 5 5 5 6 4 3 2 3 5
20829 180/300 Read block 2281 len 36864
20830 181/300 Read block 4563 len 36864
20831 182/300 Read block 3737 len 28672
20832 183/300 Read block 671 len 24576
20833 184/300 Read block 656 len 8192
20834 185/300 Read block 2581 len 40960
20835 186/300 Write block 411 len 24576 data: 3 3 3 2 3 3
20836 187/300 Read block 2835 len 24576
20837 188/300 Read block 3887 len 32768
20838 189/300 Read block 3761 len 28672
20839 190/300 Write block 1288 len 4096 data: 4
20840 191/300 Read block 1797 len 28672
20841 192/300 Read block 3777 len 36864
20842 193/300 Flush
20843 194/300 Read block 4622 len 24576
20844 195/300 Read block 916 len 20480
20845 196/300 Read block 1278 len 4096
20846 197/300 Read block 4164 len 12288
20847 198/300 Read block 4923 len 12288
20848 199/300 Flush
20849 200/300 Read block 136 len 4096
20850 201/300 Read block 1796 len 36864
20851 202/300 Read block 4143 len 16384
20852 203/300 Read block 2748 len 40960
20853 204/300 Write block 2944 len 16384 data: 5 5 4 3
20854 205/300 Read block 3735 len 20480
20855 206/300 Write block 866 len 32768 data: 2 3 3 3 3 3 2 2
20856 207/300 Write block 1392 len 20480 data: 3 2 2 2 2
20857 208/300 Flush
20858 209/300 Write block 828 len 16384 data: 6 6 5 6
20859 210/300 Flush
20860 211/300 Read block 3628 len 24576
20861 212/300 Read block 2823 len 12288
20862 213/300 Read block 632 len 20480
20863 214/300 Flush
20864 215/300 Read block 1546 len 28672
20865 216/300 Write block 4748 len 12288 data: 3 3 4
20866 217/300 Write block 3962 len 28672 data: 7 7 6 5 6 5 4
20867 218/300 Write block 1375 len 12288 data: 5 6 6
20868 219/300 Read block 462 len 20480
20869 220/300 Write block 2016 len 16384 data: 4 3 4 5
20870 221/300 Read block 1787 len 32768
20871 222/300 Read block 2617 len 8192
20872 223/300 Read block 3799 len 4096
20873 224/300 Write block 2440 len 16384 data: 5 4 4 3
20874 225/300 Write block 847 len 32768 data: 2 2 3 2 3 4 3 3
20875 226/300 Read block 2580 len 16384
20876 227/300 Read block 4714 len 36864
20877 228/300 Flush
20878 229/300 Read block 1433 len 24576
20879 230/300 Read block 2963 len 28672
20880 231/300 Read block 1072 len 16384
20881 232/300 Read block 2523 len 24576
20882 233/300 Write block 1476 len 16384 data: 7 7 9 6
20883 234/300 Write block 1887 len 36864 data: 5 7 7 7 6 6 6 6 6
20884 235/300 Read block 611 len 8192
20885 236/300 Read block 600 len 32768
20886 237/300 Write block 174 len 20480 data: 5 5 4 3 3
20887 238/300 Write block 730 len 40960 data: 7 8 7 6 5 5 5 5 5 6
20888 239/300 Write block 4592 len 16384 data: 4 5 5 5
20889 240/300 Read block 2406 len 40960
20890 241/300 Write block 4838 len 28672 data: 4 4 5 5 5 4 4
20891 242/300 Read block 356 len 36864
20892 243/300 Write block 518 len 24576 data: 3 2 4 4 5 4
20893 244/300 Read block 3806 len 32768
20894 245/300 Read block 321 len 12288
20895 246/300 Read block 1668 len 36864
20896 247/300 Read block 1316 len 40960
20897 248/300 Read block 4151 len 16384
20898 249/300 Read block 2458 len 12288
20899 250/300 Write block 4942 len 40960 data: 3 3 3 4 6 6 5 4 4 4
20900 251/300 Write block 882 len 20480 data: 4 5 5 5 5
20901 252/300 Flush
20902 253/300 Write block 1320 len 12288 data: 4 4 5
20903 254/300 Write block 469 len 20480 data: 3 3 3 2 3
20904 255/300 Write block 2248 len 24576 data: 5 5 3 3 3 3
20905 256/300 Write block 4900 len 24576 data: 5 5 5 5 6 6
20906 257/300 Write block 2758 len 20480 data: 4 4 6 5 7
20907 258/300 Write block 1879 len 36864 data: 4 5 5 4 3 3 3 4 6
20908 259/300 Flush
20909 260/300 Read block 746 len 8192
20910 261/300 Write block 486 len 36864 data: 6 5 5 6 5 5 2 2 2
20911 262/300 Write block 681 len 16384 data: 5 3 4 4
20912 263/300 Write block 2304 len 32768 data: 4 3 3 3 2 2 2 2
20913 264/300 Write block 4973 len 40960 data: 4 4 4 4 5 5 6 5 5 4
20914 265/300 Read block 3946 len 36864
20915 266/300 Read block 4278 len 8192
20916 267/300 Read block 951 len 24576
20917 268/300 Read block 2681 len 4096
20918 269/300 Flush
20919 270/300 Read block 652 len 40960
20920 271/300 Write block 802 len 28672 data: 6 7 8 6 8 8 7
20921 272/300 Read block 4684 len 20480
20922 273/300 Write block 3356 len 8192 data: 4 4
20923 274/300 Write block 1512 len 32768 data: 9 8 8 8 7 5 5 4
20924 275/300 Read block 1494 len 36864
20925 276/300 Write block 2566 len 4096 data: 3
20926 277/300 Read block 557 len 36864
20927 278/300 Read block 1515 len 36864
20928 279/300 Read block 3081 len 16384
20929 280/300 Write block 2599 len 40960 data: 3 5 5 6 6 4 4 4 2 2
20930 281/300 Write block 447 len 4096 data: 5
20931 282/300 Read block 1523 len 16384
20932 283/300 Flush
20933 284/300 Write block 2023 len 16384 data: 5 4 5 6
20934 285/300 Read block 2197 len 4096
20935 286/300 Read block 2603 len 20480
20936 287/300 Write block 3763 len 36864 data: 6 6 7 7 5 4 4 4 4
20937 288/300 Flush
20938 289/300 Write block 1420 len 36864 data: 7 8 6 7 8 8 7 6 5
20939 290/300 Read block 4197 len 40960
20940 291/300 Write block 2190 len 8192 data: 6 7
20941 292/300 Read block 3966 len 36864
20942 293/300 Write block 4770 len 20480 data: 6 6 7 5 5
20943 294/300 Read block 2503 len 28672
20944 295/300 Read block 4193 len 8192
20945 296/300 Read block 3152 len 12288
20946 297/300 Read block 1316 len 24576
20947 298/300 Read block 2242 len 8192
20948 299/300 Read block 3262 len 36864
20949 300/300 Read block 4966 len 40960
20950 301/300 Read block 4198 len 8192
20951 Main task has work to do, go find it
20952 got dsc Start(0)
20953 start 0
20954 Dec 04 13:35:07.042 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/0, method: POST, req_id: 5a2cfe8b-61bb-42b5-8e97-31a947bcd765, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
20955 Replay: started 0, returned:Ok(())
20956 ----------------------------------------------------------------
20957 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20958 States: Offline Active Active
20959 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20960 6735 Acked Read 4 Sent Done Done false
20961 6736 Acked Read 5 Sent Done Done false
20962 6737 Acked Read 4 Sent Done Done false
20963 6738 Acked Flush 0 Sent Done Done false
20964 6739 Acked Read 6 Sent Done Done false
20965 6740 Acked Read 10 Sent Done Done false
20966 6741 Acked Read 9 Sent Done Done false
20967 6742 Acked Write 9 Sent Done Done false
20968 6743 Acked Write 8 Sent Done Done false
20969 6744 Acked Flush 0 Sent Done Done false
20970 6745 Acked Read 3 Sent Done Done false
20971 6746 Acked Write 3 Sent Done Done false
20972 6747 Acked Read 1 Sent Done Done false
20973 6748 Acked Read 8 Sent Done Done false
20974 6749 Acked Read 9 Sent Done Done false
20975 6750 Acked Read 7 Sent Done Done false
20976 6751 Acked Write 1 Sent Done Done false
20977 6752 Acked Write 8 Sent Done Done false
20978 6753 Acked Flush 0 Sent Done Done false
20979 6754 Acked Read 5 Sent Done Done false
20980 6755 Acked Write 4 Sent Done Done false
20981 6756 Acked Read 3 Sent Done Done false
20982 6757 Acked Read 6 Sent Done Done false
20983 6758 Acked Write 6 Sent Done Done false
20984 6759 Acked Read 10 Sent Done Done false
20985 6760 Acked Flush 0 Sent Done Done false
20986 6761 Acked Write 1 Sent Done Done false
20987 6762 Acked Write 9 Sent Done Done false
20988 6763 Acked Read 9 Sent Done Done false
20989 6764 Acked Flush 0 Sent Done Done false
20990 6765 Acked Write 10 Sent Done Done false
20991 6766 Acked Read 5 Sent Done Done false
20992 6767 Acked Read 3 Sent Done Done false
20993 6768 Acked Write 5 Sent Done Done false
20994 6769 Acked Read 6 Sent Done Done false
20995 6770 Acked Write 4 Sent Done Done false
20996 6771 Acked Write 6 Sent Done Done false
20997 6772 Acked Read 8 Sent Done Done false
20998 6773 Acked Flush 0 Sent Done Done false
20999 6774 Acked Write 6 Sent Done Done false
21000 6775 Acked Write 9 Sent Done Done false
21001 6776 Acked Read 7 Sent Done Done false
21002 6777 Acked Write 7 Sent Done Done false
21003 6778 Acked Write 8 Sent Done Done false
21004 6779 Acked Read 1 Sent Done Done false
21005 6780 Acked Write 9 Sent Done Done false
21006 6781 Acked Read 6 Sent Done Done false
21007 6782 Acked Write 7 Sent Done Done false
21008 6783 Acked Read 4 Sent Done Done false
21009 6784 Acked Write 4 Sent Done Done false
21010 6785 Acked Read 10 Sent Done Done false
21011 6786 Acked Write 4 Sent Done Done false
21012 6787 Acked Read 10 Sent Done Done false
21013 6788 Acked Write 7 Sent Done Done false
21014 6789 Acked Write 4 Sent Done Done false
21015 6790 Acked Write 5 Sent Done Done false
21016 6791 Acked Write 1 Sent Done Done false
21017 6792 Acked Read 10 Sent Done Done false
21018 6793 Acked Read 1 Sent Done Done false
21019 6794 Acked Write 9 Sent Done Done false
21020 6795 Acked Flush 0 Sent Done Done false
21021 6796 Acked Read 4 Sent Done Done false
21022 6797 Acked Write 2 Sent Done Done false
21023 6798 Acked Write 6 Sent Done Done false
21024 6799 Acked Read 9 Sent Done Done false
21025 6800 Acked Read 10 Sent Done Done false
21026 6801 Acked Write 2 Sent Done Done false
21027 6802 Acked Flush 0 Sent Done Done false
21028 6803 Acked Read 2 Sent Done Done false
21029 6804 Acked Read 1 Sent Done Done false
21030 6805 Acked Read 4 Sent Done Done false
21031 6806 Acked Read 2 Sent Done Done false
21032 6807 Acked Write 9 Sent Done Done false
21033 6808 Acked Write 3 Sent Done Done false
21034 6809 Acked Write 8 Sent Done Done false
21035 6810 Acked Flush 0 Sent Done Done false
21036 6811 Acked Write 2 Sent Done Done false
21037 6812 Acked Read 10 Sent Done Done false
21038 6813 Acked Flush 0 Sent Done Done false
21039 6814 Acked Write 10 Sent Done Done false
21040 6815 Acked Read 2 Sent Done Done false
21041 6816 Acked Flush 0 Sent Done Done false
21042 6817 Acked Write 8 Sent Done Done false
21043 6818 Acked Read 4 Sent Done Done false
21044 6819 Acked Write 10 Sent Done Done false
21045 6820 Acked Read 10 Sent Done Done false
21046 6821 Acked Read 2 Sent Done Done false
21047 6822 Acked Flush 0 Sent Done Done false
21048 6823 Acked Read 9 Sent Done Done false
21049 6824 Acked Write 1 Sent Done Done false
21050 6825 Acked Read 4 Sent Done Done false
21051 6826 Acked Write 2 Sent Done Done false
21052 6827 Acked Write 10 Sent Done Done false
21053 6828 Acked Read 10 Sent Done Done false
21054 6829 Acked Write 7 Sent Done Done false
21055 6830 Acked Read 1 Sent Done Done false
21056 6831 Acked Read 10 Sent Done Done false
21057 6832 Acked Write 6 Sent Done Done false
21058 6833 Acked Read 7 Sent Done Done false
21059 6834 Acked Flush 0 Sent Done Done false
21060 6835 Acked Write 7 Sent Done Done false
21061 6836 Acked Flush 0 Sent Done Done false
21062 6837 Acked Flush 0 Sent Done Done false
21063 6838 Acked Read 9 Sent Done Done false
21064 6839 Acked Read 6 Sent Done Done false
21065 6840 Acked Write 4 Sent Done Done false
21066 6841 Acked Write 9 Sent Done Done false
21067 6842 Acked Write 8 Sent Done Done false
21068 6843 Acked Read 5 Sent Done Done false
21069 6844 Acked Flush 0 Sent Done Done false
21070 6845 Acked Read 10 Sent Done Done false
21071 6846 Acked Write 9 Sent Done Done false
21072 6847 Acked Read 9 Sent Done Done false
21073 6848 Acked Read 4 Sent Done Done false
21074 6849 Acked Flush 0 Sent Done Done false
21075 6850 Acked Write 10 Sent Done Done false
21076 6851 Acked Flush 0 Sent Done Done false
21077 6852 Acked Read 10 Sent Done Done false
21078 6853 Acked Read 7 Sent Done Done false
21079 6854 Acked Write 7 Sent Done Done false
21080 6855 Acked Flush 0 Sent Done Done false
21081 6856 Acked Read 9 Sent Done Done false
21082 6857 Acked Write 10 Sent Done Done false
21083 6858 Acked Write 5 Sent Done Done false
21084 6859 Acked Read 4 Sent Done Done false
21085 6860 Acked Write 3 Sent Done Done false
21086 6861 Acked Read 10 Sent Done Done false
21087 6862 Acked Flush 0 Sent Done Done false
21088 6863 Acked Read 4 Sent Done Done false
21089 6864 Acked Read 2 Sent Done Done false
21090 6865 Acked Write 6 Sent Done Done false
21091 6866 Acked Flush 0 Sent Done Done false
21092 6867 Acked Write 2 Sent Done Done false
21093 6868 Acked Read 1 Sent Done Done false
21094 6869 Acked Flush 0 Sent Done Done false
21095 6870 Acked Write 5 Sent Done Done false
21096 6871 Acked Write 9 Sent Done Done false
21097 6872 Acked Flush 0 Sent Done Done false
21098 6873 Acked Read 10 Sent Done Done false
21099 6874 Acked Write 6 Sent Done Done false
21100 6875 Acked Write 6 Sent Done Done false
21101 6876 Acked Read 2 Sent Done Done false
21102 6877 Acked Read 2 Sent Done Done false
21103 6878 Acked Write 5 Sent Done Done false
21104 6879 Acked Read 1 Sent Done Done false
21105 6880 Acked Read 5 Sent Done Done false
21106 6881 Acked Write 5 Sent Done Done false
21107 6882 Acked Read 10 Sent Done Done false
21108 6883 Acked Write 6 Sent Done Done false
21109 6884 Acked Read 1 Sent Done Done false
21110 6885 Acked Read 9 Sent Done Done false
21111 6886 Acked Write 5 Sent Done Done false
21112 6887 Acked Read 10 Sent Done Done false
21113 6888 Acked Write 7 Sent Done Done false
21114 6889 Acked Read 1 Sent Done Done false
21115 6890 Acked Read 10 Sent Done Done false
21116 6891 Acked Write 3 Sent Done Done false
21117 6892 Acked Write 10 Sent Done Done false
21118 6893 Acked Write 4 Sent Done Done false
21119 6894 Acked Write 2 Sent Done Done false
21120 6895 Acked Read 8 Sent Done Done false
21121 6896 Acked Read 8 Sent Done Done false
21122 6897 Acked Write 6 Sent Done Done false
21123 6898 Acked Write 4 Sent Done Done false
21124 6899 Acked Read 6 Sent Done Done false
21125 6900 Acked Read 2 Sent Done Done false
21126 6901 Acked Read 1 Sent Done Done false
21127 6902 Acked Read 5 Sent Done Done false
21128 6903 Acked Read 3 Sent Done Done false
21129 6904 Acked Read 1 Sent Done Done false
21130 6905 Acked Read 9 Sent Done Done false
21131 6906 Acked Read 2 Sent Done Done false
21132 6907 Acked Write 6 Sent Done Done false
21133 6908 Acked Read 9 Sent Done Done false
21134 6909 Acked Read 7 Sent Done Done false
21135 6910 Acked Read 4 Sent Done Done false
21136 6911 Acked Read 10 Sent Done Done false
21137 6912 Acked Write 3 Sent Done Done false
21138 6913 Acked Write 9 Sent Done Done false
21139 6914 Acked Read 9 Sent Done Done false
21140 6915 Acked Read 9 Sent Done Done false
21141 6916 Acked Read 7 Sent Done Done false
21142 6917 Acked Read 6 Sent Done Done false
21143 6918 Acked Read 2 Sent Done Done false
21144 6919 Acked Read 10 Sent Done Done false
21145 6920 Acked Write 6 Sent Done Done false
21146 6921 Acked Read 6 Sent Done Done false
21147 6922 Acked Read 8 Sent Done Done false
21148 6923 Acked Read 7 Sent Done Done false
21149 6924 Acked Write 1 Sent Done Done false
21150 6925 Acked Read 7 Sent Done Done false
21151 6926 Acked Read 9 Sent Done Done false
21152 6927 Acked Flush 0 Sent Done Done false
21153 6928 Acked Read 6 Sent Done Done false
21154 6929 Acked Read 5 Sent Done Done false
21155 6930 Acked Read 1 Sent Done Done false
21156 6931 Acked Read 3 Sent Done Done false
21157 6932 Acked Read 3 Sent Done Done false
21158 6933 Acked Flush 0 Sent Done Done false
21159 6934 Acked Read 1 Sent Done Done false
21160 6935 Acked Read 9 Sent Done Done false
21161 6936 Acked Read 4 Sent Done Done false
21162 6937 Acked Read 10 Sent Done Done false
21163 6938 Acked Write 4 Sent Done Done false
21164 6939 Acked Read 5 Sent Done Done false
21165 6940 Acked Write 8 Sent Done Done false
21166 6941 Acked Write 5 Sent Done Done false
21167 6942 Acked Flush 0 Sent Done Done false
21168 6943 Acked Write 4 Sent Done Done false
21169 6944 Acked Flush 0 Sent Done Done false
21170 6945 Acked Read 6 Sent Done Done false
21171 6946 Acked Read 3 Sent Done Done false
21172 6947 Acked Read 5 Sent Done Done false
21173 6948 Acked Flush 0 Sent Done Done false
21174 6949 Acked Read 7 Sent Done Done false
21175 6950 Acked Write 3 Sent Done Done false
21176 6951 Acked Write 7 Sent Done Done false
21177 6952 Acked Write 3 Sent Done Done false
21178 6953 Acked Read 5 Sent Done Done false
21179 6954 Acked Write 4 Sent Done Done false
21180 6955 Acked Read 8 Sent Done Done false
21181 6956 Acked Read 2 Sent Done Done false
21182 6957 Acked Read 1 Sent Done Done false
21183 6958 Acked Write 4 Sent Done Done false
21184 6959 Acked Write 8 Sent Done Done false
21185 6960 Acked Read 4 Sent Done Done false
21186 6961 Acked Read 9 Sent Done Done false
21187 6962 Acked Flush 0 Sent Done Done false
21188 6963 Acked Read 6 Sent Done Done false
21189 6964 Acked Read 7 Sent Done Done false
21190 6965 Acked Read 4 Sent Done Done false
21191 6966 Acked Read 6 Sent Done Done false
21192 6967 Acked Write 4 Sent Done Done false
21193 6968 Acked Write 9 Sent Done Done false
21194 6969 Acked Read 2 Sent Done Done false
21195 6970 Acked Read 8 Sent Done Done false
21196 6971 Acked Write 5 Sent Done Done false
21197 6972 Acked Write 10 Sent Done Done false
21198 6973 Acked Write 4 Sent Done Done false
21199 6974 Acked Read 10 Sent Done Done false
21200 6975 Acked Write 7 Sent Done Done false
21201 6976 Acked Read 9 Sent Done Done false
21202 6977 Acked Write 6 Sent Done Done false
21203 6978 Acked Read 8 Sent Done Done false
21204 6979 Acked Read 3 Sent Done Done false
21205 6980 Acked Read 9 Sent Done Done false
21206 6981 Acked Read 10 Sent Done Done false
21207 6982 Acked Read 4 Sent Done Done false
21208 6983 Acked Read 3 Sent Done Done false
21209 6984 Acked Write 10 Sent Done Done false
21210 6985 Acked Write 5 Sent Done Done false
21211 6986 Acked Flush 0 Sent Done Done false
21212 6987 Acked Write 3 Sent Done Done false
21213 6988 Acked Write 5 Sent Done Done false
21214 6989 Acked Write 6 Sent Done Done false
21215 6990 Acked Write 6 Sent Done Done false
21216 6991 Acked Write 5 Sent Done Done false
21217 6992 Acked Write 9 Sent Done Done false
21218 6993 Acked Flush 0 Sent Done Done false
21219 6994 Acked Read 2 Sent Done Done false
21220 6995 Acked Write 9 Sent Done Done false
21221 6996 Acked Write 4 Sent Done Done false
21222 6997 Acked Write 8 Sent Done Done false
21223 6998 Acked Write 10 Sent Done Done false
21224 6999 Acked Read 9 Sent Done Done false
21225 7000 Acked Read 2 Sent Done Done false
21226 7001 Acked Read 6 Sent Done Done false
21227 7002 Acked Read 1 Sent Done Done false
21228 7003 Acked Flush 0 Sent Done Done false
21229 7004 Acked Read 10 Sent Done Done false
21230 7005 Acked Write 7 Sent Done Done false
21231 7006 Acked Read 5 Sent Done Done false
21232 7007 Acked Write 2 Sent Done Done false
21233 7008 Acked Write 8 Sent Done Done false
21234 7009 Acked Read 9 Sent Done Done false
21235 7010 Acked Write 1 Sent Done Done false
21236 7011 Acked Read 9 Sent Done Done false
21237 7012 Acked Read 9 Sent Done Done false
21238 7013 Acked Read 4 Sent Done Done false
21239 7014 Acked Write 10 Sent Done Done false
21240 7015 Acked Write 1 Sent Done Done false
21241 7016 Acked Read 4 Sent Done Done false
21242 7017 Acked Flush 0 Sent Done Done false
21243 7018 Acked Write 4 Sent Done Done false
21244 7019 Acked Read 1 Sent Done Done false
21245 7020 Acked Read 5 Sent Done Done false
21246 7021 Acked Write 9 Sent Done Done false
21247 7022 Acked Flush 0 Sent Done Done false
21248 7023 Acked Write 9 Sent Done Done false
21249 7024 Acked Read 10 Sent Done Done false
21250 7025 Acked Write 2 Sent Done Done false
21251 7026 Acked Read 9 Sent Done Done false
21252 7027 Acked Write 5 Sent Done Done false
21253 7028 Acked Read 7 Sent Done Done false
21254 7029 Acked Read 2 Sent Done Done false
21255 7030 Acked Read 3 Sent Done Done false
21256 7031 Acked Read 6 Sent Done Done false
21257 7032 Acked Read 2 Sent Done Done false
21258 7033 Acked Read 9 Sent Done Done false
21259 7034 Acked Read 10 Sent Done Done false
21260 7035 Acked Read 2 Sent Done Done false
21261 STATES DS:0 DS:1 DS:2 TOTAL
21262 Sent 301 0 0 301
21263 Done 0 301 301 602
21264 Skipped 0 0 0 0
21265 Error 0 0 0 0
21266 Last Flush: 6734 7022 7022
21267 Downstairs last five completed: 6734 6733 6732 6731 6730
21268 Upstairs last five completed: 7035 7034 7033 7032 7031
21269 CLIENT: Up:0 ds:301 act:2
21270 [8810] got start action
21271 [8810] I am going to restart
21272 Starting downstairs at port 8810
21273 Make output file at "/tmp/dsc/downstairs-8810.txt"
21274 [8810][0] reports Starting
21275 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1223)
21276 [8810][0] reports Running
21277 ----------------------------------------------------------------
21278 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21279 States: Offline Active Active
21280 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21281 6735 Acked Read 4 Sent Done Done false
21282 6736 Acked Read 5 Sent Done Done false
21283 6737 Acked Read 4 Sent Done Done false
21284 6738 Acked Flush 0 Sent Done Done false
21285 6739 Acked Read 6 Sent Done Done false
21286 6740 Acked Read 10 Sent Done Done false
21287 6741 Acked Read 9 Sent Done Done false
21288 6742 Acked Write 9 Sent Done Done false
21289 6743 Acked Write 8 Sent Done Done false
21290 6744 Acked Flush 0 Sent Done Done false
21291 6745 Acked Read 3 Sent Done Done false
21292 6746 Acked Write 3 Sent Done Done false
21293 6747 Acked Read 1 Sent Done Done false
21294 6748 Acked Read 8 Sent Done Done false
21295 6749 Acked Read 9 Sent Done Done false
21296 6750 Acked Read 7 Sent Done Done false
21297 6751 Acked Write 1 Sent Done Done false
21298 6752 Acked Write 8 Sent Done Done false
21299 6753 Acked Flush 0 Sent Done Done false
21300 6754 Acked Read 5 Sent Done Done false
21301 6755 Acked Write 4 Sent Done Done false
21302 6756 Acked Read 3 Sent Done Done false
21303 6757 Acked Read 6 Sent Done Done false
21304 6758 Acked Write 6 Sent Done Done false
21305 6759 Acked Read 10 Sent Done Done false
21306 6760 Acked Flush 0 Sent Done Done false
21307 6761 Acked Write 1 Sent Done Done false
21308 6762 Acked Write 9 Sent Done Done false
21309 6763 Acked Read 9 Sent Done Done false
21310 6764 Acked Flush 0 Sent Done Done false
21311 6765 Acked Write 10 Sent Done Done false
21312 6766 Acked Read 5 Sent Done Done false
21313 6767 Acked Read 3 Sent Done Done false
21314 6768 Acked Write 5 Sent Done Done false
21315 6769 Acked Read 6 Sent Done Done false
21316 6770 Acked Write 4 Sent Done Done false
21317 6771 Acked Write 6 Sent Done Done false
21318 6772 Acked Read 8 Sent Done Done false
21319 6773 Acked Flush 0 Sent Done Done false
21320 6774 Acked Write 6 Sent Done Done false
21321 6775 Acked Write 9 Sent Done Done false
21322 6776 Acked Read 7 Sent Done Done false
21323 6777 Acked Write 7 Sent Done Done false
21324 6778 Acked Write 8 Sent Done Done false
21325 6779 Acked Read 1 Sent Done Done false
21326 6780 Acked Write 9 Sent Done Done false
21327 6781 Acked Read 6 Sent Done Done false
21328 6782 Acked Write 7 Sent Done Done false
21329 6783 Acked Read 4 Sent Done Done false
21330 6784 Acked Write 4 Sent Done Done false
21331 6785 Acked Read 10 Sent Done Done false
21332 6786 Acked Write 4 Sent Done Done false
21333 6787 Acked Read 10 Sent Done Done false
21334 6788 Acked Write 7 Sent Done Done false
21335 6789 Acked Write 4 Sent Done Done false
21336 6790 Acked Write 5 Sent Done Done false
21337 6791 Acked Write 1 Sent Done Done false
21338 6792 Acked Read 10 Sent Done Done false
21339 6793 Acked Read 1 Sent Done Done false
21340 6794 Acked Write 9 Sent Done Done false
21341 6795 Acked Flush 0 Sent Done Done false
21342 6796 Acked Read 4 Sent Done Done false
21343 6797 Acked Write 2 Sent Done Done false
21344 6798 Acked Write 6 Sent Done Done false
21345 6799 Acked Read 9 Sent Done Done false
21346 6800 Acked Read 10 Sent Done Done false
21347 6801 Acked Write 2 Sent Done Done false
21348 6802 Acked Flush 0 Sent Done Done false
21349 6803 Acked Read 2 Sent Done Done false
21350 6804 Acked Read 1 Sent Done Done false
21351 6805 Acked Read 4 Sent Done Done false
21352 6806 Acked Read 2 Sent Done Done false
21353 6807 Acked Write 9 Sent Done Done false
21354 6808 Acked Write 3 Sent Done Done false
21355 6809 Acked Write 8 Sent Done Done false
21356 6810 Acked Flush 0 Sent Done Done false
21357 6811 Acked Write 2 Sent Done Done false
21358 6812 Acked Read 10 Sent Done Done false
21359 6813 Acked Flush 0 Sent Done Done false
21360 6814 Acked Write 10 Sent Done Done false
21361 6815 Acked Read 2 Sent Done Done false
21362 6816 Acked Flush 0 Sent Done Done false
21363 6817 Acked Write 8 Sent Done Done false
21364 6818 Acked Read 4 Sent Done Done false
21365 6819 Acked Write 10 Sent Done Done false
21366 6820 Acked Read 10 Sent Done Done false
21367 6821 Acked Read 2 Sent Done Done false
21368 6822 Acked Flush 0 Sent Done Done false
21369 6823 Acked Read 9 Sent Done Done false
21370 6824 Acked Write 1 Sent Done Done false
21371 6825 Acked Read 4 Sent Done Done false
21372 6826 Acked Write 2 Sent Done Done false
21373 6827 Acked Write 10 Sent Done Done false
21374 6828 Acked Read 10 Sent Done Done false
21375 6829 Acked Write 7 Sent Done Done false
21376 6830 Acked Read 1 Sent Done Done false
21377 6831 Acked Read 10 Sent Done Done false
21378 6832 Acked Write 6 Sent Done Done false
21379 6833 Acked Read 7 Sent Done Done false
21380 6834 Acked Flush 0 Sent Done Done false
21381 6835 Acked Write 7 Sent Done Done false
21382 6836 Acked Flush 0 Sent Done Done false
21383 6837 Acked Flush 0 Sent Done Done false
21384 6838 Acked Read 9 Sent Done Done false
21385 6839 Acked Read 6 Sent Done Done false
21386 6840 Acked Write 4 Sent Done Done false
21387 6841 Acked Write 9 Sent Done Done false
21388 6842 Acked Write 8 Sent Done Done false
21389 6843 Acked Read 5 Sent Done Done false
21390 6844 Acked Flush 0 Sent Done Done false
21391 6845 Acked Read 10 Sent Done Done false
21392 6846 Acked Write 9 Sent Done Done false
21393 6847 Acked Read 9 Sent Done Done false
21394 6848 Acked Read 4 Sent Done Done false
21395 6849 Acked Flush 0 Sent Done Done false
21396 6850 Acked Write 10 Sent Done Done false
21397 6851 Acked Flush 0 Sent Done Done false
21398 6852 Acked Read 10 Sent Done Done false
21399 6853 Acked Read 7 Sent Done Done false
21400 6854 Acked Write 7 Sent Done Done false
21401 6855 Acked Flush 0 Sent Done Done false
21402 6856 Acked Read 9 Sent Done Done false
21403 6857 Acked Write 10 Sent Done Done false
21404 6858 Acked Write 5 Sent Done Done false
21405 6859 Acked Read 4 Sent Done Done false
21406 6860 Acked Write 3 Sent Done Done false
21407 6861 Acked Read 10 Sent Done Done false
21408 6862 Acked Flush 0 Sent Done Done false
21409 6863 Acked Read 4 Sent Done Done false
21410 6864 Acked Read 2 Sent Done Done false
21411 6865 Acked Write 6 Sent Done Done false
21412 6866 Acked Flush 0 Sent Done Done false
21413 6867 Acked Write 2 Sent Done Done false
21414 6868 Acked Read 1 Sent Done Done false
21415 6869 Acked Flush 0 Sent Done Done false
21416 6870 Acked Write 5 Sent Done Done false
21417 6871 Acked Write 9 Sent Done Done false
21418 6872 Acked Flush 0 Sent Done Done false
21419 6873 Acked Read 10 Sent Done Done false
21420 6874 Acked Write 6 Sent Done Done false
21421 6875 Acked Write 6 Sent Done Done false
21422 6876 Acked Read 2 Sent Done Done false
21423 6877 Acked Read 2 Sent Done Done false
21424 6878 Acked Write 5 Sent Done Done false
21425 6879 Acked Read 1 Sent Done Done false
21426 6880 Acked Read 5 Sent Done Done false
21427 6881 Acked Write 5 Sent Done Done false
21428 6882 Acked Read 10 Sent Done Done false
21429 6883 Acked Write 6 Sent Done Done false
21430 6884 Acked Read 1 Sent Done Done false
21431 6885 Acked Read 9 Sent Done Done false
21432 6886 Acked Write 5 Sent Done Done false
21433 6887 Acked Read 10 Sent Done Done false
21434 6888 Acked Write 7 Sent Done Done false
21435 6889 Acked Read 1 Sent Done Done false
21436 6890 Acked Read 10 Sent Done Done false
21437 6891 Acked Write 3 Sent Done Done false
21438 6892 Acked Write 10 Sent Done Done false
21439 6893 Acked Write 4 Sent Done Done false
21440 6894 Acked Write 2 Sent Done Done false
21441 6895 Acked Read 8 Sent Done Done false
21442 6896 Acked Read 8 Sent Done Done false
21443 6897 Acked Write 6 Sent Done Done false
21444 6898 Acked Write 4 Sent Done Done false
21445 6899 Acked Read 6 Sent Done Done false
21446 6900 Acked Read 2 Sent Done Done false
21447 6901 Acked Read 1 Sent Done Done false
21448 6902 Acked Read 5 Sent Done Done false
21449 6903 Acked Read 3 Sent Done Done false
21450 6904 Acked Read 1 Sent Done Done false
21451 6905 Acked Read 9 Sent Done Done false
21452 6906 Acked Read 2 Sent Done Done false
21453 6907 Acked Write 6 Sent Done Done false
21454 6908 Acked Read 9 Sent Done Done false
21455 6909 Acked Read 7 Sent Done Done false
21456 6910 Acked Read 4 Sent Done Done false
21457 6911 Acked Read 10 Sent Done Done false
21458 6912 Acked Write 3 Sent Done Done false
21459 6913 Acked Write 9 Sent Done Done false
21460 6914 Acked Read 9 Sent Done Done false
21461 6915 Acked Read 9 Sent Done Done false
21462 6916 Acked Read 7 Sent Done Done false
21463 6917 Acked Read 6 Sent Done Done false
21464 6918 Acked Read 2 Sent Done Done false
21465 6919 Acked Read 10 Sent Done Done false
21466 6920 Acked Write 6 Sent Done Done false
21467 6921 Acked Read 6 Sent Done Done false
21468 6922 Acked Read 8 Sent Done Done false
21469 6923 Acked Read 7 Sent Done Done false
21470 6924 Acked Write 1 Sent Done Done false
21471 6925 Acked Read 7 Sent Done Done false
21472 6926 Acked Read 9 Sent Done Done false
21473 6927 Acked Flush 0 Sent Done Done false
21474 6928 Acked Read 6 Sent Done Done false
21475 6929 Acked Read 5 Sent Done Done false
21476 6930 Acked Read 1 Sent Done Done false
21477 6931 Acked Read 3 Sent Done Done false
21478 6932 Acked Read 3 Sent Done Done false
21479 6933 Acked Flush 0 Sent Done Done false
21480 6934 Acked Read 1 Sent Done Done false
21481 6935 Acked Read 9 Sent Done Done false
21482 6936 Acked Read 4 Sent Done Done false
21483 6937 Acked Read 10 Sent Done Done false
21484 6938 Acked Write 4 Sent Done Done false
21485 6939 Acked Read 5 Sent Done Done false
21486 6940 Acked Write 8 Sent Done Done false
21487 6941 Acked Write 5 Sent Done Done false
21488 6942 Acked Flush 0 Sent Done Done false
21489 6943 Acked Write 4 Sent Done Done false
21490 6944 Acked Flush 0 Sent Done Done false
21491 6945 Acked Read 6 Sent Done Done false
21492 6946 Acked Read 3 Sent Done Done false
21493 6947 Acked Read 5 Sent Done Done false
21494 6948 Acked Flush 0 Sent Done Done false
21495 6949 Acked Read 7 Sent Done Done false
21496 6950 Acked Write 3 Sent Done Done false
21497 6951 Acked Write 7 Sent Done Done false
21498 6952 Acked Write 3 Sent Done Done false
21499 6953 Acked Read 5 Sent Done Done false
21500 6954 Acked Write 4 Sent Done Done false
21501 6955 Acked Read 8 Sent Done Done false
21502 6956 Acked Read 2 Sent Done Done false
21503 6957 Acked Read 1 Sent Done Done false
21504 6958 Acked Write 4 Sent Done Done false
21505 6959 Acked Write 8 Sent Done Done false
21506 6960 Acked Read 4 Sent Done Done false
21507 6961 Acked Read 9 Sent Done Done false
21508 6962 Acked Flush 0 Sent Done Done false
21509 6963 Acked Read 6 Sent Done Done false
21510 6964 Acked Read 7 Sent Done Done false
21511 6965 Acked Read 4 Sent Done Done false
21512 6966 Acked Read 6 Sent Done Done false
21513 6967 Acked Write 4 Sent Done Done false
21514 6968 Acked Write 9 Sent Done Done false
21515 6969 Acked Read 2 Sent Done Done false
21516 6970 Acked Read 8 Sent Done Done false
21517 6971 Acked Write 5 Sent Done Done false
21518 6972 Acked Write 10 Sent Done Done false
21519 6973 Acked Write 4 Sent Done Done false
21520 6974 Acked Read 10 Sent Done Done false
21521 6975 Acked Write 7 Sent Done Done false
21522 6976 Acked Read 9 Sent Done Done false
21523 6977 Acked Write 6 Sent Done Done false
21524 6978 Acked Read 8 Sent Done Done false
21525 6979 Acked Read 3 Sent Done Done false
21526 6980 Acked Read 9 Sent Done Done false
21527 6981 Acked Read 10 Sent Done Done false
21528 6982 Acked Read 4 Sent Done Done false
21529 6983 Acked Read 3 Sent Done Done false
21530 6984 Acked Write 10 Sent Done Done false
21531 6985 Acked Write 5 Sent Done Done false
21532 6986 Acked Flush 0 Sent Done Done false
21533 6987 Acked Write 3 Sent Done Done false
21534 6988 Acked Write 5 Sent Done Done false
21535 6989 Acked Write 6 Sent Done Done false
21536 6990 Acked Write 6 Sent Done Done false
21537 6991 Acked Write 5 Sent Done Done false
21538 6992 Acked Write 9 Sent Done Done false
21539 6993 Acked Flush 0 Sent Done Done false
21540 6994 Acked Read 2 Sent Done Done false
21541 6995 Acked Write 9 Sent Done Done false
21542 6996 Acked Write 4 Sent Done Done false
21543 6997 Acked Write 8 Sent Done Done false
21544 6998 Acked Write 10 Sent Done Done false
21545 6999 Acked Read 9 Sent Done Done false
21546 7000 Acked Read 2 Sent Done Done false
21547 7001 Acked Read 6 Sent Done Done false
21548 7002 Acked Read 1 Sent Done Done false
21549 7003 Acked Flush 0 Sent Done Done false
21550 7004 Acked Read 10 Sent Done Done false
21551 7005 Acked Write 7 Sent Done Done false
21552 7006 Acked Read 5 Sent Done Done false
21553 7007 Acked Write 2 Sent Done Done false
21554 7008 Acked Write 8 Sent Done Done false
21555 7009 Acked Read 9 Sent Done Done false
21556 7010 Acked Write 1 Sent Done Done false
21557 7011 Acked Read 9 Sent Done Done false
21558 7012 Acked Read 9 Sent Done Done false
21559 7013 Acked Read 4 Sent Done Done false
21560 7014 Acked Write 10 Sent Done Done false
21561 7015 Acked Write 1 Sent Done Done false
21562 7016 Acked Read 4 Sent Done Done false
21563 7017 Acked Flush 0 Sent Done Done false
21564 7018 Acked Write 4 Sent Done Done false
21565 7019 Acked Read 1 Sent Done Done false
21566 7020 Acked Read 5 Sent Done Done false
21567 7021 Acked Write 9 Sent Done Done false
21568 7022 Acked Flush 0 Sent Done Done false
21569 7023 Acked Write 9 Sent Done Done false
21570 7024 Acked Read 10 Sent Done Done false
21571 7025 Acked Write 2 Sent Done Done false
21572 7026 Acked Read 9 Sent Done Done false
21573 7027 Acked Write 5 Sent Done Done false
21574 7028 Acked Read 7 Sent Done Done false
21575 7029 Acked Read 2 Sent Done Done false
21576 7030 Acked Read 3 Sent Done Done false
21577 7031 Acked Read 6 Sent Done Done false
21578 7032 Acked Read 2 Sent Done Done false
21579 7033 Acked Read 9 Sent Done Done false
21580 7034 Acked Read 10 Sent Done Done false
21581 7035 Acked Read 2 Sent Done Done false
21582 7036 Acked Flush 0 Sent Done Done false
21583 STATES DS:0 DS:1 DS:2 TOTAL
21584 Sent 302 0 0 302
21585 Done 0 302 302 604
21586 Skipped 0 0 0 0
21587 Error 0 0 0 0
21588 Last Flush: 6734 7036 7036
21589 Downstairs last five completed: 6734 6733 6732 6731 6730
21590 Upstairs last five completed: 7036 7035 7034 7033 7032
21591 CLIENT: Up:0 ds:302 act:2
215922024-12-04T13:35:12.797ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
215932024-12-04T13:35:12.797ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58481) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
215942024-12-04T13:35:12.798ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
215952024-12-04T13:35:12.799ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
215962024-12-04T13:35:12.799ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
215972024-12-04T13:35:12.799ZINFOcrucible: send last flush ID to this DS: 6734 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
215982024-12-04T13:35:12.799ZINFOcrucible: Replied this last flush ID: 6734 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
215992024-12-04T13:35:12.799ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
216002024-12-04T13:35:12.799ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
216012024-12-04T13:35:12.799ZINFOcrucible: [0] client re-new 302 jobs since flush 6734 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
216022024-12-04T13:35:12.799ZINFOcrucible: [0] Replayed 302 jobs since flush: 6734 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21603 ----------------------------------------------------------------
21604 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21605 Downstairs last five completed: 7036 7035 7034 7033 7032
21606 Upstairs last five completed: 7036 7035 7034 7033 7032
21607 CLIENT: Up:0 ds:0 act:3
21608 Replay: All jobs finished
21609 Main task has work to do, go find it
21610 got dsc Stop(0)
21611 stop 0
21612 Dec 04 13:35:15.047 INFO request completed, latency_us: 349, [8810] Got stop action so:false kr:false
21613 response_code: 204, uri: /stop/cid/0, method: POST, req_id: e167a8f6-1b11-4120-bd51-1cc17caac05c, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
21614 Dec 04 13:35:15.047 INFO{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:35:15.047942017Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
21615 request completed, latency_us: 131, response_code: 200, uri: /state/cid/0, method: GET, req_id: 4b19180f-f7f1-4e6e-aff3-753dd2595699, remote_addr{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:35:15.048010678Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"0","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
21616 : 127.0.0.1:37804, local_addr: 127.0.{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:35:15.048040808Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
21617 0.1:9998
216182024-12-04T13:35:15.048ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
216192024-12-04T13:35:15.048ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
216202024-12-04T13:35:15.048ZINFOcrucible: client task is exiting = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
21621 [8810] Exited with: ExitStatus(unix_wait_status(9))
21622 [8810][0] reports Exit
21623 Dec 04 13:35:19.050 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/0, method: GET, req_id: 4ad76e6d-3edc-4025-97eb-d2c99cb5b1b1, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
21624 001/300 Flush
21625 002/300 Write block 3023 len 28672 data: 4 3 3 4 3 3 3
21626 003/300 Write block 4828 len 40960 data: 5 5 5 5 4 3 3 3 3 4
21627 004/300 Read block 1308 len 16384
21628 005/300 Read block 2290 len 20480
21629 006/300 Write block 99 len 20480 data: 4 4 4 5 4
21630 007/300 Write block 4369 len 12288 data: 3 3 3
21631 008/300 Write block 4801 len 32768 data: 7 6 7 6 6 5 4 3
21632 009/300 Write block 4030 len 36864 data: 6 6 6 5 5 5 3 2 2
21633 010/300 Read block 4783 len 24576
21634 011/300 Read block 4274 len 32768
21635 012/300 Write block 1985 len 36864 data: 5 5 4 5 5 5 6 5 5
21636 013/300 Read block 347 len 4096
21637 014/300 Read block 1360 len 24576
21638 015/300 Write block 4974 len 28672 data: 5 5 5 6 6 7 6
21639 016/300 Write block 2291 len 20480 data: 7 8 8 8 6
21640 017/300 Write block 293 len 40960 data: 3 5 6 7 6 5 4 2 3 3
21641 018/300 Read block 4196 len 32768
21642 019/300 Write block 3884 len 40960 data: 6 5 4 3 3 3 2 2 2 2
21643 020/300 Read block 2922 len 40960
21644 021/300 Write block 994 len 20480 data: 6 4 4 4 3
21645 022/300 Write block 2461 len 20480 data: 5 5 6 6 6
21646 023/300 Read block 1964 len 24576
21647 024/300 Write block 3463 len 36864 data: 5 5 4 5 6 6 4 4 3
21648 025/300 Write block 279 len 8192 data: 6 4
21649 026/300 Read block 4340 len 8192
21650 027/300 Read block 1713 len 36864
21651 028/300 Read block 1540 len 36864
21652 029/300 Flush
21653 030/300 Read block 435 len 40960
21654 031/300 Read block 2601 len 28672
21655 032/300 Read block 1328 len 20480
21656 033/300 Read block 467 len 24576
21657 034/300 Read block 3166 len 16384
21658 035/300 Read block 3818 len 28672
21659 036/300 Flush
21660 037/300 Write block 2838 len 32768 data: 6 6 6 5 5 5 7 6
21661 038/300 Read block 1094 len 20480
21662 039/300 Write block 3499 len 32768 data: 5 5 7 7 7 5 4 3
21663 040/300 Write block 471 len 28672 data: 4 3 4 3 3 4 6
21664 041/300 Write block 4667 len 20480 data: 3 2 2 2 3
21665 042/300 Read block 4520 len 20480
21666 043/300 Flush
21667 044/300 Write block 3598 len 24576 data: 4 4 4 4 4 4
21668 045/300 Read block 1762 len 16384
21669 046/300 Read block 2570 len 24576
21670 047/300 Write block 1596 len 12288 data: 7 5 6
21671 048/300 Read block 3684 len 16384
21672 049/300 Read block 1373 len 12288
21673 050/300 Write block 2363 len 36864 data: 3 4 4 4 4 6 6 7 7
21674 051/300 Read block 103 len 12288
21675 052/300 Read block 3794 len 32768
21676 053/300 Read block 2570 len 32768
21677 054/300 Read block 1562 len 36864
21678 055/300 Flush
21679 056/300 Read block 1087 len 40960
21680 057/300 Flush
21681 058/300 Read block 2752 len 28672
21682 059/300 Write block 4426 len 4096 data: 5
21683 060/300 Read block 1184 len 8192
21684 061/300 Write block 3080 len 8192 data: 3 3
21685 062/300 Write block 2399 len 36864 data: 5 5 4 5 5 6 5 5 4
21686 063/300 Read block 668 len 36864
21687 064/300 Read block 241 len 24576
21688 065/300 Read block 3174 len 16384
21689 066/300 Flush
21690 067/300 Read block 2355 len 8192
21691 068/300 Write block 1383 len 20480 data: 5 5 4 3 3
21692 069/300 Write block 576 len 20480 data: 2 3 3 4 4
21693 070/300 Write block 2857 len 20480 data: 3 3 3 3 4
21694 071/300 Write block 3991 len 32768 data: 4 4 4 3 4 3 3 3
21695 072/300 Write block 4602 len 24576 data: 6 6 7 7 7 6
21696 073/300 Read block 853 len 12288
21697 074/300 Flush
21698 075/300 Write block 1546 len 40960 data: 6 5 5 4 5 6 6 6 5 4
21699 076/300 Write block 2826 len 40960 data: 6 6 5 6 5 5 5 3 2 3
21700 077/300 Write block 4674 len 32768 data: 4 5 6 6 6 6 5 3
21701 078/300 Read block 1703 len 36864
21702 079/300 Read block 2597 len 40960
21703 080/300 Read block 2370 len 24576
21704 081/300 Write block 1375 len 4096 data: 6
21705 082/300 Read block 1994 len 4096
21706 083/300 Read block 3808 len 36864
21707 084/300 Read block 1295 len 4096
21708 085/300 Flush
21709 086/300 Read block 2020 len 32768
21710 087/300 Read block 1588 len 36864
21711 088/300 Write block 304 len 24576 data: 4 5 6 6 4 4
21712 089/300 Write block 4318 len 4096 data: 3
21713 090/300 Read block 2279 len 8192
21714 091/300 Write block 1264 len 8192 data: 4 4
21715 092/300 Read block 1859 len 8192
21716 093/300 Read block 2419 len 40960
21717 094/300 Read block 3309 len 32768
21718 095/300 Read block 2747 len 16384
21719 096/300 Write block 1369 len 32768 data: 4 4 5 6 3 4 7 7
21720 097/300 Write block 1557 len 16384 data: 3 4 4 4
21721 098/300 Write block 499 len 28672 data: 5 4 5 4 6 6 7
21722 099/300 Read block 4295 len 20480
21723 100/300 Write block 1183 len 28672 data: 5 5 5 4 4 5 6
21724 101/300 Write block 1755 len 4096 data: 3
21725 102/300 Write block 2268 len 24576 data: 4 4 4 4 3 4
21726 103/300 Write block 564 len 28672 data: 6 6 4 5 6 5 5
21727 104/300 Read block 4536 len 36864
21728 105/300 Write block 1182 len 28672 data: 5 6 6 6 5 5 6
21729 106/300 Write block 4436 len 4096 data: 6
21730 107/300 Write block 245 len 40960 data: 5 5 6 6 6 7 7 9 9 7
21731 108/300 Write block 4923 len 12288 data: 5 4 3
21732 109/300 Write block 2081 len 16384 data: 5 4 4 4
21733 110/300 Flush
21734 111/300 Read block 1999 len 36864
21735 112/300 Write block 945 len 24576 data: 5 5 6 7 6 6
21736 113/300 Read block 3991 len 8192
21737 114/300 Read block 1679 len 24576
21738 115/300 Read block 4781 len 28672
21739 116/300 Read block 14 len 8192
21740 117/300 Write block 1484 len 40960 data: 5 5 4 4 4 3 3 3 3 4
21741 118/300 Write block 550 len 8192 data: 4 2
21742 119/300 Write block 63 len 28672 data: 5 6 8 8 7 5 5
21743 120/300 Write block 843 len 40960 data: 3 3 3 2 3 3 4 3 4 5
21744 121/300 Read block 4981 len 40960
21745 122/300 Write block 4819 len 40960 data: 3 2 2 2 2 2 2 4 4 6
21746 123/300 Read block 2697 len 12288
21747 124/300 Read block 3803 len 24576
21748 125/300 Read block 294 len 28672
21749 126/300 Read block 1868 len 16384
21750 127/300 Read block 4092 len 24576
21751 128/300 Write block 509 len 20480 data: 7 7 7 4 4
21752 129/300 Read block 856 len 32768
21753 130/300 Write block 3138 len 28672 data: 6 5 8 8 7 6 4
21754 131/300 Read block 744 len 40960
21755 132/300 Read block 2640 len 40960
21756 133/300 Write block 3147 len 32768 data: 4 5 6 5 7 7 9 8
21757 134/300 Flush
21758 135/300 Read block 367 len 28672
21759 136/300 Write block 2739 len 40960 data: 5 5 5 4 3 4 4 3 4 4
21760 137/300 Write block 1460 len 16384 data: 4 4 4 3
21761 138/300 Write block 1162 len 8192 data: 6 7
21762 139/300 Write block 1751 len 28672 data: 6 4 4 4 4 3 3
21763 140/300 Flush
21764 141/300 Read block 179 len 20480
21765 142/300 Read block 4362 len 16384
21766 143/300 Write block 1706 len 8192 data: 2 3
21767 144/300 Read block 2963 len 36864
21768 145/300 Write block 549 len 28672 data: 4 5 3 2 2 2 2
21769 146/300 Read block 1660 len 36864
21770 147/300 Write block 2265 len 20480 data: 4 4 4 5 5
21771 148/300 Read block 3073 len 28672
21772 149/300 Flush
21773 150/300 Flush
21774 151/300 Read block 3952 len 16384
21775 152/300 Read block 1139 len 20480
21776 153/300 Read block 2154 len 16384
21777 154/300 Read block 2020 len 24576
21778 155/300 Read block 27 len 24576
21779 156/300 Write block 4247 len 32768 data: 7 8 9 6 5 4 4 4
21780 157/300 Flush
21781 158/300 Write block 2196 len 32768 data: 6 4 3 3 2 4 4 4
21782 159/300 Read block 284 len 12288
21783 160/300 Write block 327 len 40960 data: 3 4 5 6 5 4 3 3 3 2
21784 161/300 Read block 3916 len 36864
21785 162/300 Read block 1520 len 4096
21786 163/300 Write block 4645 len 24576 data: 3 4 5 5 5 5
21787 164/300 Flush
21788 165/300 Flush
21789 166/300 Write block 3622 len 32768 data: 2 4 5 6 5 6 6 7
21790 167/300 Read block 4124 len 24576
21791 168/300 Write block 4240 len 8192 data: 6 6
21792 169/300 Read block 2779 len 40960
21793 170/300 Read block 4286 len 4096
21794 171/300 Flush
21795 172/300 Write block 2273 len 4096 data: 5
21796 173/300 Read block 454 len 16384
21797 174/300 Read block 119 len 28672
21798 175/300 Write block 4019 len 24576 data: 5 5 3 4 4 3
21799 176/300 Write block 2848 len 20480 data: 7 7 5 6 8
21800 177/300 Read block 2172 len 16384
21801 178/300 Read block 618 len 16384
21802 179/300 Write block 1542 len 24576 data: 4 5 5 6 7 6
21803 180/300 Write block 2256 len 20480 data: 5 4 3 3 3
21804 181/300 Write block 4357 len 28672 data: 5 3 3 3 3 3 4
21805 182/300 Write block 589 len 8192 data: 2 2
21806 183/300 Read block 4636 len 28672
21807 184/300 Read block 4740 len 36864
21808 185/300 Flush
21809 186/300 Read block 134 len 16384
21810 187/300 Read block 2450 len 20480
21811 188/300 Write block 3547 len 36864 data: 7 6 6 5 7 7 6 5 3
21812 189/300 Read block 3333 len 16384
21813 190/300 Write block 1520 len 36864 data: 3 3 4 5 6 9 10 9 8
21814 191/300 Write block 956 len 12288 data: 3 3 3
21815 192/300 Write block 967 len 32768 data: 4 4 4 4 4 4 4 4
21816 193/300 Read block 4397 len 12288
21817 194/300 Read block 3959 len 28672
21818 195/300 Read block 3591 len 36864
21819 196/300 Read block 1882 len 12288
21820 197/300 Read block 1583 len 24576
21821 198/300 Write block 278 len 24576 data: 5 7 5 3 3 3
21822 199/300 Read block 3568 len 4096
21823 200/300 Write block 4418 len 24576 data: 4 5 5 5 5 4
21824 201/300 Read block 2841 len 20480
21825 202/300 Read block 2443 len 20480
21826 203/300 Read block 1925 len 16384
21827 204/300 Read block 3833 len 36864
21828 205/300 Read block 1541 len 4096
21829 206/300 Write block 4588 len 4096 data: 5
21830 207/300 Read block 3662 len 20480
21831 208/300 Write block 2922 len 28672 data: 5 5 5 6 6 6 6
21832 209/300 Read block 2968 len 28672
21833 210/300 Read block 238 len 16384
21834 211/300 Write block 4431 len 8192 data: 2 3
21835 212/300 Write block 597 len 4096 data: 5
21836 213/300 Read block 2191 len 32768
21837 214/300 Write block 4305 len 8192 data: 3 4
21838 215/300 Write block 2750 len 12288 data: 5 4 4
21839 216/300 Read block 3612 len 4096
21840 217/300 Read block 2513 len 28672
21841 218/300 Read block 1080 len 36864
21842 219/300 Write block 1109 len 36864 data: 3 3 4 6 6 8 7 6 7
21843 220/300 Write block 2639 len 36864 data: 7 7 5 5 5 5 6 6 7
21844 221/300 Flush
21845 222/300 Write block 3628 len 8192 data: 7 8
21846 223/300 Write block 521 len 4096 data: 5
21847 224/300 Read block 4564 len 16384
21848 225/300 Write block 1758 len 28672 data: 3 2 2 7 10 10 9
21849 226/300 Read block 1101 len 4096
21850 227/300 Read block 1232 len 36864
21851 228/300 Write block 4422 len 8192 data: 6 5
21852 229/300 Flush
21853 230/300 Write block 4518 len 16384 data: 6 7 8 8
21854 231/300 Read block 3565 len 40960
21855 232/300 Write block 727 len 24576 data: 6 6 7 8 9 8
21856 233/300 Write block 4779 len 24576 data: 4 3 4 5 5 4
21857 234/300 Read block 1915 len 40960
21858 235/300 Read block 539 len 32768
21859 236/300 Write block 2305 len 16384 data: 4 4 4 3
21860 237/300 Read block 252 len 8192
21861 238/300 Read block 4034 len 24576
21862 239/300 Read block 3055 len 12288
21863 240/300 Read block 4113 len 24576
21864 241/300 Flush
21865 242/300 Read block 4265 len 24576
21866 243/300 Read block 3125 len 32768
21867 244/300 Write block 1135 len 36864 data: 4 5 5 6 5 5 6 6 5
21868 245/300 Write block 2020 len 20480 data: 4 5 5 6 5
21869 246/300 Read block 694 len 24576
21870 247/300 Read block 1569 len 36864
21871 248/300 Write block 339 len 32768 data: 3 3 2 3 4 5 4 4
21872 249/300 Read block 2640 len 12288
21873 250/300 Read block 579 len 32768
21874 251/300 Write block 4191 len 32768 data: 4 4 5 5 4 5 7 6
21875 252/300 Read block 3671 len 28672
21876 253/300 Read block 1241 len 24576
21877 254/300 Write block 3252 len 20480 data: 3 3 3 4 2
21878 255/300 Write block 520 len 28672 data: 5 6 6 5 4 4 4
21879 256/300 Write block 2634 len 32768 data: 6 5 7 7 7 8 8 6
21880 257/300 Write block 163 len 16384 data: 7 7 7 5
21881 258/300 Read block 1950 len 12288
21882 259/300 Write block 982 len 32768 data: 5 7 7 7 7 5 6 6
21883 260/300 Read block 1573 len 4096
21884 261/300 Read block 4320 len 20480
21885 262/300 Write block 3768 len 20480 data: 5 5 5 5 4
21886 263/300 Flush
21887 264/300 Write block 2641 len 36864 data: 7 6 6 6 7 7 8 6 4
21888 265/300 Read block 4449 len 24576
21889 266/300 Flush
21890 267/300 Read block 2296 len 4096
21891 268/300 Read block 3098 len 20480
21892 269/300 Read block 443 len 8192
21893 270/300 Write block 2839 len 4096 data: 7
21894 271/300 Read block 227 len 12288
21895 272/300 Write block 17 len 16384 data: 9 5 3 2
21896 273/300 Read block 278 len 4096
21897 274/300 Read block 2427 len 24576
21898 275/300 Read block 1926 len 4096
21899 276/300 Read block 4040 len 12288
21900 277/300 Write block 2811 len 4096 data: 2
21901 278/300 Read block 2267 len 20480
21902 279/300 Write block 1777 len 4096 data: 2
21903 280/300 Write block 2462 len 16384 data: 6 7 7 7
21904 281/300 Read block 1494 len 4096
21905 282/300 Read block 2410 len 28672
21906 283/300 Read block 3586 len 24576
21907 284/300 Read block 4649 len 4096
21908 285/300 Read block 1701 len 28672
21909 286/300 Write block 859 len 32768 data: 4 5 4 3 2 2 2 3
21910 287/300 Write block 4619 len 36864 data: 6 5 5 4 4 4 3 2 2
21911 288/300 Read block 2088 len 8192
21912 289/300 Write block 2983 len 16384 data: 6 7 7 6
21913 290/300 Write block 4459 len 24576 data: 8 5 6 8 8 8
21914 291/300 Write block 95 len 4096 data: 6
21915 292/300 Read block 3042 len 24576
21916 293/300 Write block 3538 len 16384 data: 4 4 4 6
21917 294/300 Read block 3512 len 24576
21918 295/300 Write block 4441 len 4096 data: 4
21919 296/300 Read block 2449 len 40960
21920 297/300 Read block 2060 len 16384
21921 298/300 Write block 1645 len 20480 data: 4 4 3 4 4
21922 299/300 Write block 2728 len 40960 data: 4 4 3 3 4 4 4 4 4 4
21923 300/300 Read block 796 len 28672
21924 301/300 Read block 3663 len 24576
21925 Main task has work to do, go find it
21926 got dsc Start(0)
21927 start 0
21928 Dec 04 13:35:19.287 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/0, method: POST, req_id: 489bec23-bf0c-46ef-a763-e9f8030dc1a5, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
21929 Replay: started 0, returned:Ok(())
21930 ----------------------------------------------------------------
21931 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21932 States: Offline Active Active
21933 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21934 7037 Acked Flush 0 Sent Done Done false
21935 7038 Acked Write 7 Sent Done Done false
21936 7039 Acked Write 10 Sent Done Done false
21937 7040 Acked Read 4 Sent Done Done false
21938 7041 Acked Read 5 Sent Done Done false
21939 7042 Acked Write 5 Sent Done Done false
21940 7043 Acked Write 3 Sent Done Done false
21941 7044 Acked Write 8 Sent Done Done false
21942 7045 Acked Write 9 Sent Done Done false
21943 7046 Acked Read 6 Sent Done Done false
21944 7047 Acked Read 8 Sent Done Done false
21945 7048 Acked Write 9 Sent Done Done false
21946 7049 Acked Read 1 Sent Done Done false
21947 7050 Acked Read 6 Sent Done Done false
21948 7051 Acked Write 7 Sent Done Done false
21949 7052 Acked Write 5 Sent Done Done false
21950 7053 Acked Write 10 Sent Done Done false
21951 7054 Acked Read 8 Sent Done Done false
21952 7055 Acked Write 10 Sent Done Done false
21953 7056 Acked Read 10 Sent Done Done false
21954 7057 Acked Write 5 Sent Done Done false
21955 7058 Acked Write 5 Sent Done Done false
21956 7059 Acked Read 6 Sent Done Done false
21957 7060 Acked Write 9 Sent Done Done false
21958 7061 Acked Write 2 Sent Done Done false
21959 7062 Acked Read 2 Sent Done Done false
21960 7063 Acked Read 9 Sent Done Done false
21961 7064 Acked Read 9 Sent Done Done false
21962 7065 Acked Flush 0 Sent Done Done false
21963 7066 Acked Read 10 Sent Done Done false
21964 7067 Acked Read 7 Sent Done Done false
21965 7068 Acked Read 5 Sent Done Done false
21966 7069 Acked Read 6 Sent Done Done false
21967 7070 Acked Read 4 Sent Done Done false
21968 7071 Acked Read 7 Sent Done Done false
21969 7072 Acked Flush 0 Sent Done Done false
21970 7073 Acked Write 8 Sent Done Done false
21971 7074 Acked Read 5 Sent Done Done false
21972 7075 Acked Write 8 Sent Done Done false
21973 7076 Acked Write 7 Sent Done Done false
21974 7077 Acked Write 5 Sent Done Done false
21975 7078 Acked Read 5 Sent Done Done false
21976 7079 Acked Flush 0 Sent Done Done false
21977 7080 Acked Write 6 Sent Done Done false
21978 7081 Acked Read 4 Sent Done Done false
21979 7082 Acked Read 6 Sent Done Done false
21980 7083 Acked Write 3 Sent Done Done false
21981 7084 Acked Read 4 Sent Done Done false
21982 7085 Acked Read 3 Sent Done Done false
21983 7086 Acked Write 9 Sent Done Done false
21984 7087 Acked Read 3 Sent Done Done false
21985 7088 Acked Read 8 Sent Done Done false
21986 7089 Acked Read 8 Sent Done Done false
21987 7090 Acked Read 9 Sent Done Done false
21988 7091 Acked Flush 0 Sent Done Done false
21989 7092 Acked Read 10 Sent Done Done false
21990 7093 Acked Flush 0 Sent Done Done false
21991 7094 Acked Read 7 Sent Done Done false
21992 7095 Acked Write 1 Sent Done Done false
21993 7096 Acked Read 2 Sent Done Done false
21994 7097 Acked Write 2 Sent Done Done false
21995 7098 Acked Write 9 Sent Done Done false
21996 7099 Acked Read 9 Sent Done Done false
21997 7100 Acked Read 6 Sent Done Done false
21998 7101 Acked Read 4 Sent Done Done false
21999 7102 Acked Flush 0 Sent Done Done false
22000 7103 Acked Read 2 Sent Done Done false
22001 7104 Acked Write 5 Sent Done Done false
22002 7105 Acked Write 5 Sent Done Done false
22003 7106 Acked Write 5 Sent Done Done false
22004 7107 Acked Write 8 Sent Done Done false
22005 7108 Acked Write 6 Sent Done Done false
22006 7109 Acked Read 3 Sent Done Done false
22007 7110 Acked Flush 0 Sent Done Done false
22008 7111 Acked Write 10 Sent Done Done false
22009 7112 Acked Write 10 Sent Done Done false
22010 7113 Acked Write 8 Sent Done Done false
22011 7114 Acked Read 9 Sent Done Done false
22012 7115 Acked Read 10 Sent Done Done false
22013 7116 Acked Read 6 Sent Done Done false
22014 7117 Acked Write 1 Sent Done Done false
22015 7118 Acked Read 1 Sent Done Done false
22016 7119 Acked Read 9 Sent Done Done false
22017 7120 Acked Read 1 Sent Done Done false
22018 7121 Acked Flush 0 Sent Done Done false
22019 7122 Acked Read 8 Sent Done Done false
22020 7123 Acked Read 9 Sent Done Done false
22021 7124 Acked Write 6 Sent Done Done false
22022 7125 Acked Write 1 Sent Done Done false
22023 7126 Acked Read 2 Sent Done Done false
22024 7127 Acked Write 2 Sent Done Done false
22025 7128 Acked Read 2 Sent Done Done false
22026 7129 Acked Read 10 Sent Done Done false
22027 7130 Acked Read 8 Sent Done Done false
22028 7131 Acked Read 4 Sent Done Done false
22029 7132 Acked Write 8 Sent Done Done false
22030 7133 Acked Write 4 Sent Done Done false
22031 7134 Acked Write 7 Sent Done Done false
22032 7135 Acked Read 5 Sent Done Done false
22033 7136 Acked Write 7 Sent Done Done false
22034 7137 Acked Write 1 Sent Done Done false
22035 7138 Acked Write 6 Sent Done Done false
22036 7139 Acked Write 7 Sent Done Done false
22037 7140 Acked Read 9 Sent Done Done false
22038 7141 Acked Write 7 Sent Done Done false
22039 7142 Acked Write 1 Sent Done Done false
22040 7143 Acked Write 10 Sent Done Done false
22041 7144 Acked Write 3 Sent Done Done false
22042 7145 Acked Write 4 Sent Done Done false
22043 7146 Acked Flush 0 Sent Done Done false
22044 7147 Acked Read 9 Sent Done Done false
22045 7148 Acked Write 6 Sent Done Done false
22046 7149 Acked Read 2 Sent Done Done false
22047 7150 Acked Read 6 Sent Done Done false
22048 7151 Acked Read 7 Sent Done Done false
22049 7152 Acked Read 2 Sent Done Done false
22050 7153 Acked Write 10 Sent Done Done false
22051 7154 Acked Write 2 Sent Done Done false
22052 7155 Acked Write 7 Sent Done Done false
22053 7156 Acked Write 10 Sent Done Done false
22054 7157 Acked Read 10 Sent Done Done false
22055 7158 Acked Write 10 Sent Done Done false
22056 7159 Acked Read 3 Sent Done Done false
22057 7160 Acked Read 6 Sent Done Done false
22058 7161 Acked Read 7 Sent Done Done false
22059 7162 Acked Read 4 Sent Done Done false
22060 7163 Acked Read 6 Sent Done Done false
22061 7164 Acked Write 5 Sent Done Done false
22062 7165 Acked Read 8 Sent Done Done false
22063 7166 Acked Write 7 Sent Done Done false
22064 7167 Acked Read 10 Sent Done Done false
22065 7168 Acked Read 10 Sent Done Done false
22066 7169 Acked Write 8 Sent Done Done false
22067 7170 Acked Flush 0 Sent Done Done false
22068 7171 Acked Read 7 Sent Done Done false
22069 7172 Acked Write 10 Sent Done Done false
22070 7173 Acked Write 4 Sent Done Done false
22071 7174 Acked Write 2 Sent Done Done false
22072 7175 Acked Write 7 Sent Done Done false
22073 7176 Acked Flush 0 Sent Done Done false
22074 7177 Acked Read 5 Sent Done Done false
22075 7178 Acked Read 4 Sent Done Done false
22076 7179 Acked Write 2 Sent Done Done false
22077 7180 Acked Read 9 Sent Done Done false
22078 7181 Acked Write 7 Sent Done Done false
22079 7182 Acked Read 9 Sent Done Done false
22080 7183 Acked Write 5 Sent Done Done false
22081 7184 Acked Read 7 Sent Done Done false
22082 7185 Acked Flush 0 Sent Done Done false
22083 7186 Acked Flush 0 Sent Done Done false
22084 7187 Acked Read 4 Sent Done Done false
22085 7188 Acked Read 5 Sent Done Done false
22086 7189 Acked Read 4 Sent Done Done false
22087 7190 Acked Read 6 Sent Done Done false
22088 7191 Acked Read 6 Sent Done Done false
22089 7192 Acked Write 8 Sent Done Done false
22090 7193 Acked Flush 0 Sent Done Done false
22091 7194 Acked Write 8 Sent Done Done false
22092 7195 Acked Read 3 Sent Done Done false
22093 7196 Acked Write 10 Sent Done Done false
22094 7197 Acked Read 9 Sent Done Done false
22095 7198 Acked Read 1 Sent Done Done false
22096 7199 Acked Write 6 Sent Done Done false
22097 7200 Acked Flush 0 Sent Done Done false
22098 7201 Acked Flush 0 Sent Done Done false
22099 7202 Acked Write 8 Sent Done Done false
22100 7203 Acked Read 6 Sent Done Done false
22101 7204 Acked Write 2 Sent Done Done false
22102 7205 Acked Read 10 Sent Done Done false
22103 7206 Acked Read 1 Sent Done Done false
22104 7207 Acked Flush 0 Sent Done Done false
22105 7208 Acked Write 1 Sent Done Done false
22106 7209 Acked Read 4 Sent Done Done false
22107 7210 Acked Read 7 Sent Done Done false
22108 7211 Acked Write 6 Sent Done Done false
22109 7212 Acked Write 5 Sent Done Done false
22110 7213 Acked Read 4 Sent Done Done false
22111 7214 Acked Read 4 Sent Done Done false
22112 7215 Acked Write 6 Sent Done Done false
22113 7216 Acked Write 5 Sent Done Done false
22114 7217 Acked Write 7 Sent Done Done false
22115 7218 Acked Write 2 Sent Done Done false
22116 7219 Acked Read 7 Sent Done Done false
22117 7220 Acked Read 9 Sent Done Done false
22118 7221 Acked Flush 0 Sent Done Done false
22119 7222 Acked Read 4 Sent Done Done false
22120 7223 Acked Read 5 Sent Done Done false
22121 7224 Acked Write 9 Sent Done Done false
22122 7225 Acked Read 4 Sent Done Done false
22123 7226 Acked Write 9 Sent Done Done false
22124 7227 Acked Write 3 Sent Done Done false
22125 7228 Acked Write 8 Sent Done Done false
22126 7229 Acked Read 3 Sent Done Done false
22127 7230 Acked Read 7 Sent Done Done false
22128 7231 Acked Read 9 Sent Done Done false
22129 7232 Acked Read 3 Sent Done Done false
22130 7233 Acked Read 6 Sent Done Done false
22131 7234 Acked Write 6 Sent Done Done false
22132 7235 Acked Read 1 Sent Done Done false
22133 7236 Acked Write 6 Sent Done Done false
22134 7237 Acked Read 5 Sent Done Done false
22135 7238 Acked Read 5 Sent Done Done false
22136 7239 Acked Read 4 Sent Done Done false
22137 7240 Acked Read 9 Sent Done Done false
22138 7241 Acked Read 1 Sent Done Done false
22139 7242 Acked Write 1 Sent Done Done false
22140 7243 Acked Read 5 Sent Done Done false
22141 7244 Acked Write 7 Sent Done Done false
22142 7245 Acked Read 7 Sent Done Done false
22143 7246 Acked Read 4 Sent Done Done false
22144 7247 Acked Write 2 Sent Done Done false
22145 7248 Acked Write 1 Sent Done Done false
22146 7249 Acked Read 8 Sent Done Done false
22147 7250 Acked Write 2 Sent Done Done false
22148 7251 Acked Write 3 Sent Done Done false
22149 7252 Acked Read 1 Sent Done Done false
22150 7253 Acked Read 7 Sent Done Done false
22151 7254 Acked Read 9 Sent Done Done false
22152 7255 Acked Write 9 Sent Done Done false
22153 7256 Acked Write 9 Sent Done Done false
22154 7257 Acked Flush 0 Sent Done Done false
22155 7258 Acked Write 2 Sent Done Done false
22156 7259 Acked Write 1 Sent Done Done false
22157 7260 Acked Read 4 Sent Done Done false
22158 7261 Acked Write 7 Sent Done Done false
22159 7262 Acked Read 1 Sent Done Done false
22160 7263 Acked Read 9 Sent Done Done false
22161 7264 Acked Write 2 Sent Done Done false
22162 7265 Acked Flush 0 Sent Done Done false
22163 7266 Acked Write 4 Sent Done Done false
22164 7267 Acked Read 10 Sent Done Done false
22165 7268 Acked Write 6 Sent Done Done false
22166 7269 Acked Write 6 Sent Done Done false
22167 7270 Acked Read 10 Sent Done Done false
22168 7271 Acked Read 8 Sent Done Done false
22169 7272 Acked Write 4 Sent Done Done false
22170 7273 Acked Read 2 Sent Done Done false
22171 7274 Acked Read 6 Sent Done Done false
22172 7275 Acked Read 3 Sent Done Done false
22173 7276 Acked Read 6 Sent Done Done false
22174 7277 Acked Flush 0 Sent Done Done false
22175 7278 Acked Read 6 Sent Done Done false
22176 7279 Acked Read 8 Sent Done Done false
22177 7280 Acked Write 9 Sent Done Done false
22178 7281 Acked Write 5 Sent Done Done false
22179 7282 Acked Read 6 Sent Done Done false
22180 7283 Acked Read 9 Sent Done Done false
22181 7284 Acked Write 8 Sent Done Done false
22182 7285 Acked Read 3 Sent Done Done false
22183 7286 Acked Read 8 Sent Done Done false
22184 7287 Acked Write 8 Sent Done Done false
22185 7288 Acked Read 7 Sent Done Done false
22186 7289 Acked Read 6 Sent Done Done false
22187 7290 Acked Write 5 Sent Done Done false
22188 7291 Acked Write 7 Sent Done Done false
22189 7292 Acked Write 8 Sent Done Done false
22190 7293 Acked Write 4 Sent Done Done false
22191 7294 Acked Read 3 Sent Done Done false
22192 7295 Acked Write 8 Sent Done Done false
22193 7296 Acked Read 1 Sent Done Done false
22194 7297 Acked Read 5 Sent Done Done false
22195 7298 Acked Write 5 Sent Done Done false
22196 7299 Acked Flush 0 Sent Done Done false
22197 7300 Acked Write 9 Sent Done Done false
22198 7301 Acked Read 6 Sent Done Done false
22199 7302 Acked Flush 0 Sent Done Done false
22200 7303 Acked Read 1 Sent Done Done false
22201 7304 Acked Read 5 Sent Done Done false
22202 7305 Acked Read 2 Sent Done Done false
22203 7306 Acked Write 1 Sent Done Done false
22204 7307 Acked Read 3 Sent Done Done false
22205 7308 Acked Write 4 Sent Done Done false
22206 7309 Acked Read 1 Sent Done Done false
22207 7310 Acked Read 6 Sent Done Done false
22208 7311 Acked Read 1 Sent Done Done false
22209 7312 Acked Read 3 Sent Done Done false
22210 7313 Acked Write 1 Sent Done Done false
22211 7314 Acked Read 5 Sent Done Done false
22212 7315 Acked Write 1 Sent Done Done false
22213 7316 Acked Write 4 Sent Done Done false
22214 7317 Acked Read 1 Sent Done Done false
22215 7318 Acked Read 7 Sent Done Done false
22216 7319 Acked Read 6 Sent Done Done false
22217 7320 Acked Read 1 Sent Done Done false
22218 7321 Acked Read 7 Sent Done Done false
22219 7322 Acked Write 8 Sent Done Done false
22220 7323 Acked Write 9 Sent Done Done false
22221 7324 Acked Read 2 Sent Done Done false
22222 7325 Acked Write 4 Sent Done Done false
22223 7326 Acked Write 6 Sent Done Done false
22224 7327 Acked Write 1 Sent Done Done false
22225 7328 Acked Read 6 Sent Done Done false
22226 7329 Acked Write 4 Sent Done Done false
22227 7330 Acked Read 6 Sent Done Done false
22228 7331 Acked Write 1 Sent Done Done false
22229 7332 Acked Read 10 Sent Done Done false
22230 7333 Acked Read 4 Sent Done Done false
22231 7334 Acked Write 5 Sent Done Done false
22232 7335 Acked Write 10 Sent Done Done false
22233 7336 Acked Read 7 Sent Done Done false
22234 7337 Acked Read 6 Sent Done Done false
22235 STATES DS:0 DS:1 DS:2 TOTAL
22236 Sent 301 0 0 301
22237 Done 0 301 301 602
22238 Skipped 0 0 0 0
22239 Error 0 0 0 0
22240 Last Flush: 7036 7302 7302
22241 Downstairs last five completed: 7036 7035 7034 7033 7032
22242 Upstairs last five completed: 7337 7336 7335 7334 7333
22243 CLIENT: Up:0 ds:301 act:2
22244 [8810] got start action
22245 [8810] I am going to restart
22246 Starting downstairs at port 8810
22247 Make output file at "/tmp/dsc/downstairs-8810.txt"
22248 [8810][0] reports Starting
22249 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1224)
22250 [8810][0] reports Running
22251 ----------------------------------------------------------------
22252 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22253 States: Offline Active Active
22254 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22255 7037 Acked Flush 0 Sent Done Done false
22256 7038 Acked Write 7 Sent Done Done false
22257 7039 Acked Write 10 Sent Done Done false
22258 7040 Acked Read 4 Sent Done Done false
22259 7041 Acked Read 5 Sent Done Done false
22260 7042 Acked Write 5 Sent Done Done false
22261 7043 Acked Write 3 Sent Done Done false
22262 7044 Acked Write 8 Sent Done Done false
22263 7045 Acked Write 9 Sent Done Done false
22264 7046 Acked Read 6 Sent Done Done false
22265 7047 Acked Read 8 Sent Done Done false
22266 7048 Acked Write 9 Sent Done Done false
22267 7049 Acked Read 1 Sent Done Done false
22268 7050 Acked Read 6 Sent Done Done false
22269 7051 Acked Write 7 Sent Done Done false
22270 7052 Acked Write 5 Sent Done Done false
22271 7053 Acked Write 10 Sent Done Done false
22272 7054 Acked Read 8 Sent Done Done false
22273 7055 Acked Write 10 Sent Done Done false
22274 7056 Acked Read 10 Sent Done Done false
22275 7057 Acked Write 5 Sent Done Done false
22276 7058 Acked Write 5 Sent Done Done false
22277 7059 Acked Read 6 Sent Done Done false
22278 7060 Acked Write 9 Sent Done Done false
22279 7061 Acked Write 2 Sent Done Done false
22280 7062 Acked Read 2 Sent Done Done false
22281 7063 Acked Read 9 Sent Done Done false
22282 7064 Acked Read 9 Sent Done Done false
22283 7065 Acked Flush 0 Sent Done Done false
22284 7066 Acked Read 10 Sent Done Done false
22285 7067 Acked Read 7 Sent Done Done false
22286 7068 Acked Read 5 Sent Done Done false
22287 7069 Acked Read 6 Sent Done Done false
22288 7070 Acked Read 4 Sent Done Done false
22289 7071 Acked Read 7 Sent Done Done false
22290 7072 Acked Flush 0 Sent Done Done false
22291 7073 Acked Write 8 Sent Done Done false
22292 7074 Acked Read 5 Sent Done Done false
22293 7075 Acked Write 8 Sent Done Done false
22294 7076 Acked Write 7 Sent Done Done false
22295 7077 Acked Write 5 Sent Done Done false
22296 7078 Acked Read 5 Sent Done Done false
22297 7079 Acked Flush 0 Sent Done Done false
22298 7080 Acked Write 6 Sent Done Done false
22299 7081 Acked Read 4 Sent Done Done false
22300 7082 Acked Read 6 Sent Done Done false
22301 7083 Acked Write 3 Sent Done Done false
22302 7084 Acked Read 4 Sent Done Done false
22303 7085 Acked Read 3 Sent Done Done false
22304 7086 Acked Write 9 Sent Done Done false
22305 7087 Acked Read 3 Sent Done Done false
22306 7088 Acked Read 8 Sent Done Done false
22307 7089 Acked Read 8 Sent Done Done false
22308 7090 Acked Read 9 Sent Done Done false
22309 7091 Acked Flush 0 Sent Done Done false
22310 7092 Acked Read 10 Sent Done Done false
22311 7093 Acked Flush 0 Sent Done Done false
22312 7094 Acked Read 7 Sent Done Done false
22313 7095 Acked Write 1 Sent Done Done false
22314 7096 Acked Read 2 Sent Done Done false
22315 7097 Acked Write 2 Sent Done Done false
22316 7098 Acked Write 9 Sent Done Done false
22317 7099 Acked Read 9 Sent Done Done false
22318 7100 Acked Read 6 Sent Done Done false
22319 7101 Acked Read 4 Sent Done Done false
22320 7102 Acked Flush 0 Sent Done Done false
22321 7103 Acked Read 2 Sent Done Done false
22322 7104 Acked Write 5 Sent Done Done false
22323 7105 Acked Write 5 Sent Done Done false
22324 7106 Acked Write 5 Sent Done Done false
22325 7107 Acked Write 8 Sent Done Done false
22326 7108 Acked Write 6 Sent Done Done false
22327 7109 Acked Read 3 Sent Done Done false
22328 7110 Acked Flush 0 Sent Done Done false
22329 7111 Acked Write 10 Sent Done Done false
22330 7112 Acked Write 10 Sent Done Done false
22331 7113 Acked Write 8 Sent Done Done false
22332 7114 Acked Read 9 Sent Done Done false
22333 7115 Acked Read 10 Sent Done Done false
22334 7116 Acked Read 6 Sent Done Done false
22335 7117 Acked Write 1 Sent Done Done false
22336 7118 Acked Read 1 Sent Done Done false
22337 7119 Acked Read 9 Sent Done Done false
22338 7120 Acked Read 1 Sent Done Done false
22339 7121 Acked Flush 0 Sent Done Done false
22340 7122 Acked Read 8 Sent Done Done false
22341 7123 Acked Read 9 Sent Done Done false
22342 7124 Acked Write 6 Sent Done Done false
22343 7125 Acked Write 1 Sent Done Done false
22344 7126 Acked Read 2 Sent Done Done false
22345 7127 Acked Write 2 Sent Done Done false
22346 7128 Acked Read 2 Sent Done Done false
22347 7129 Acked Read 10 Sent Done Done false
22348 7130 Acked Read 8 Sent Done Done false
22349 7131 Acked Read 4 Sent Done Done false
22350 7132 Acked Write 8 Sent Done Done false
22351 7133 Acked Write 4 Sent Done Done false
22352 7134 Acked Write 7 Sent Done Done false
22353 7135 Acked Read 5 Sent Done Done false
22354 7136 Acked Write 7 Sent Done Done false
22355 7137 Acked Write 1 Sent Done Done false
22356 7138 Acked Write 6 Sent Done Done false
22357 7139 Acked Write 7 Sent Done Done false
22358 7140 Acked Read 9 Sent Done Done false
22359 7141 Acked Write 7 Sent Done Done false
22360 7142 Acked Write 1 Sent Done Done false
22361 7143 Acked Write 10 Sent Done Done false
22362 7144 Acked Write 3 Sent Done Done false
22363 7145 Acked Write 4 Sent Done Done false
22364 7146 Acked Flush 0 Sent Done Done false
22365 7147 Acked Read 9 Sent Done Done false
22366 7148 Acked Write 6 Sent Done Done false
22367 7149 Acked Read 2 Sent Done Done false
22368 7150 Acked Read 6 Sent Done Done false
22369 7151 Acked Read 7 Sent Done Done false
22370 7152 Acked Read 2 Sent Done Done false
22371 7153 Acked Write 10 Sent Done Done false
22372 7154 Acked Write 2 Sent Done Done false
22373 7155 Acked Write 7 Sent Done Done false
22374 7156 Acked Write 10 Sent Done Done false
22375 7157 Acked Read 10 Sent Done Done false
22376 7158 Acked Write 10 Sent Done Done false
22377 7159 Acked Read 3 Sent Done Done false
22378 7160 Acked Read 6 Sent Done Done false
22379 7161 Acked Read 7 Sent Done Done false
22380 7162 Acked Read 4 Sent Done Done false
22381 7163 Acked Read 6 Sent Done Done false
22382 7164 Acked Write 5 Sent Done Done false
22383 7165 Acked Read 8 Sent Done Done false
22384 7166 Acked Write 7 Sent Done Done false
22385 7167 Acked Read 10 Sent Done Done false
22386 7168 Acked Read 10 Sent Done Done false
22387 7169 Acked Write 8 Sent Done Done false
22388 7170 Acked Flush 0 Sent Done Done false
22389 7171 Acked Read 7 Sent Done Done false
22390 7172 Acked Write 10 Sent Done Done false
22391 7173 Acked Write 4 Sent Done Done false
22392 7174 Acked Write 2 Sent Done Done false
22393 7175 Acked Write 7 Sent Done Done false
22394 7176 Acked Flush 0 Sent Done Done false
22395 7177 Acked Read 5 Sent Done Done false
22396 7178 Acked Read 4 Sent Done Done false
22397 7179 Acked Write 2 Sent Done Done false
22398 7180 Acked Read 9 Sent Done Done false
22399 7181 Acked Write 7 Sent Done Done false
22400 7182 Acked Read 9 Sent Done Done false
22401 7183 Acked Write 5 Sent Done Done false
22402 7184 Acked Read 7 Sent Done Done false
22403 7185 Acked Flush 0 Sent Done Done false
22404 7186 Acked Flush 0 Sent Done Done false
22405 7187 Acked Read 4 Sent Done Done false
22406 7188 Acked Read 5 Sent Done Done false
22407 7189 Acked Read 4 Sent Done Done false
22408 7190 Acked Read 6 Sent Done Done false
22409 7191 Acked Read 6 Sent Done Done false
22410 7192 Acked Write 8 Sent Done Done false
22411 7193 Acked Flush 0 Sent Done Done false
22412 7194 Acked Write 8 Sent Done Done false
22413 7195 Acked Read 3 Sent Done Done false
22414 7196 Acked Write 10 Sent Done Done false
22415 7197 Acked Read 9 Sent Done Done false
22416 7198 Acked Read 1 Sent Done Done false
22417 7199 Acked Write 6 Sent Done Done false
22418 7200 Acked Flush 0 Sent Done Done false
22419 7201 Acked Flush 0 Sent Done Done false
22420 7202 Acked Write 8 Sent Done Done false
22421 7203 Acked Read 6 Sent Done Done false
22422 7204 Acked Write 2 Sent Done Done false
22423 7205 Acked Read 10 Sent Done Done false
22424 7206 Acked Read 1 Sent Done Done false
22425 7207 Acked Flush 0 Sent Done Done false
22426 7208 Acked Write 1 Sent Done Done false
22427 7209 Acked Read 4 Sent Done Done false
22428 7210 Acked Read 7 Sent Done Done false
22429 7211 Acked Write 6 Sent Done Done false
22430 7212 Acked Write 5 Sent Done Done false
22431 7213 Acked Read 4 Sent Done Done false
22432 7214 Acked Read 4 Sent Done Done false
22433 7215 Acked Write 6 Sent Done Done false
22434 7216 Acked Write 5 Sent Done Done false
22435 7217 Acked Write 7 Sent Done Done false
22436 7218 Acked Write 2 Sent Done Done false
22437 7219 Acked Read 7 Sent Done Done false
22438 7220 Acked Read 9 Sent Done Done false
22439 7221 Acked Flush 0 Sent Done Done false
22440 7222 Acked Read 4 Sent Done Done false
22441 7223 Acked Read 5 Sent Done Done false
22442 7224 Acked Write 9 Sent Done Done false
22443 7225 Acked Read 4 Sent Done Done false
22444 7226 Acked Write 9 Sent Done Done false
22445 7227 Acked Write 3 Sent Done Done false
22446 7228 Acked Write 8 Sent Done Done false
22447 7229 Acked Read 3 Sent Done Done false
22448 7230 Acked Read 7 Sent Done Done false
22449 7231 Acked Read 9 Sent Done Done false
22450 7232 Acked Read 3 Sent Done Done false
22451 7233 Acked Read 6 Sent Done Done false
22452 7234 Acked Write 6 Sent Done Done false
22453 7235 Acked Read 1 Sent Done Done false
22454 7236 Acked Write 6 Sent Done Done false
22455 7237 Acked Read 5 Sent Done Done false
22456 7238 Acked Read 5 Sent Done Done false
22457 7239 Acked Read 4 Sent Done Done false
22458 7240 Acked Read 9 Sent Done Done false
22459 7241 Acked Read 1 Sent Done Done false
22460 7242 Acked Write 1 Sent Done Done false
22461 7243 Acked Read 5 Sent Done Done false
22462 7244 Acked Write 7 Sent Done Done false
22463 7245 Acked Read 7 Sent Done Done false
22464 7246 Acked Read 4 Sent Done Done false
22465 7247 Acked Write 2 Sent Done Done false
22466 7248 Acked Write 1 Sent Done Done false
22467 7249 Acked Read 8 Sent Done Done false
22468 7250 Acked Write 2 Sent Done Done false
22469 7251 Acked Write 3 Sent Done Done false
22470 7252 Acked Read 1 Sent Done Done false
22471 7253 Acked Read 7 Sent Done Done false
22472 7254 Acked Read 9 Sent Done Done false
22473 7255 Acked Write 9 Sent Done Done false
22474 7256 Acked Write 9 Sent Done Done false
22475 7257 Acked Flush 0 Sent Done Done false
22476 7258 Acked Write 2 Sent Done Done false
22477 7259 Acked Write 1 Sent Done Done false
22478 7260 Acked Read 4 Sent Done Done false
22479 7261 Acked Write 7 Sent Done Done false
22480 7262 Acked Read 1 Sent Done Done false
22481 7263 Acked Read 9 Sent Done Done false
22482 7264 Acked Write 2 Sent Done Done false
22483 7265 Acked Flush 0 Sent Done Done false
22484 7266 Acked Write 4 Sent Done Done false
22485 7267 Acked Read 10 Sent Done Done false
22486 7268 Acked Write 6 Sent Done Done false
22487 7269 Acked Write 6 Sent Done Done false
22488 7270 Acked Read 10 Sent Done Done false
22489 7271 Acked Read 8 Sent Done Done false
22490 7272 Acked Write 4 Sent Done Done false
22491 7273 Acked Read 2 Sent Done Done false
22492 7274 Acked Read 6 Sent Done Done false
22493 7275 Acked Read 3 Sent Done Done false
22494 7276 Acked Read 6 Sent Done Done false
22495 7277 Acked Flush 0 Sent Done Done false
22496 7278 Acked Read 6 Sent Done Done false
22497 7279 Acked Read 8 Sent Done Done false
22498 7280 Acked Write 9 Sent Done Done false
22499 7281 Acked Write 5 Sent Done Done false
22500 7282 Acked Read 6 Sent Done Done false
22501 7283 Acked Read 9 Sent Done Done false
22502 7284 Acked Write 8 Sent Done Done false
22503 7285 Acked Read 3 Sent Done Done false
22504 7286 Acked Read 8 Sent Done Done false
22505 7287 Acked Write 8 Sent Done Done false
22506 7288 Acked Read 7 Sent Done Done false
22507 7289 Acked Read 6 Sent Done Done false
22508 7290 Acked Write 5 Sent Done Done false
22509 7291 Acked Write 7 Sent Done Done false
22510 7292 Acked Write 8 Sent Done Done false
22511 7293 Acked Write 4 Sent Done Done false
22512 7294 Acked Read 3 Sent Done Done false
22513 7295 Acked Write 8 Sent Done Done false
22514 7296 Acked Read 1 Sent Done Done false
22515 7297 Acked Read 5 Sent Done Done false
22516 7298 Acked Write 5 Sent Done Done false
22517 7299 Acked Flush 0 Sent Done Done false
22518 7300 Acked Write 9 Sent Done Done false
22519 7301 Acked Read 6 Sent Done Done false
22520 7302 Acked Flush 0 Sent Done Done false
22521 7303 Acked Read 1 Sent Done Done false
22522 7304 Acked Read 5 Sent Done Done false
22523 7305 Acked Read 2 Sent Done Done false
22524 7306 Acked Write 1 Sent Done Done false
22525 7307 Acked Read 3 Sent Done Done false
22526 7308 Acked Write 4 Sent Done Done false
22527 7309 Acked Read 1 Sent Done Done false
22528 7310 Acked Read 6 Sent Done Done false
22529 7311 Acked Read 1 Sent Done Done false
22530 7312 Acked Read 3 Sent Done Done false
22531 7313 Acked Write 1 Sent Done Done false
22532 7314 Acked Read 5 Sent Done Done false
22533 7315 Acked Write 1 Sent Done Done false
22534 7316 Acked Write 4 Sent Done Done false
22535 7317 Acked Read 1 Sent Done Done false
22536 7318 Acked Read 7 Sent Done Done false
22537 7319 Acked Read 6 Sent Done Done false
22538 7320 Acked Read 1 Sent Done Done false
22539 7321 Acked Read 7 Sent Done Done false
22540 7322 Acked Write 8 Sent Done Done false
22541 7323 Acked Write 9 Sent Done Done false
22542 7324 Acked Read 2 Sent Done Done false
22543 7325 Acked Write 4 Sent Done Done false
22544 7326 Acked Write 6 Sent Done Done false
22545 7327 Acked Write 1 Sent Done Done false
22546 7328 Acked Read 6 Sent Done Done false
22547 7329 Acked Write 4 Sent Done Done false
22548 7330 Acked Read 6 Sent Done Done false
22549 7331 Acked Write 1 Sent Done Done false
22550 7332 Acked Read 10 Sent Done Done false
22551 7333 Acked Read 4 Sent Done Done false
22552 7334 Acked Write 5 Sent Done Done false
22553 7335 Acked Write 10 Sent Done Done false
22554 7336 Acked Read 7 Sent Done Done false
22555 7337 Acked Read 6 Sent Done Done false
22556 7338 Acked Flush 0 Sent Done Done false
22557 STATES DS:0 DS:1 DS:2 TOTAL
22558 Sent 302 0 0 302
22559 Done 0 302 302 604
22560 Skipped 0 0 0 0
22561 Error 0 0 0 0
22562 Last Flush: 7036 7338 7338
22563 Downstairs last five completed: 7036 7035 7034 7033 7032
22564 Upstairs last five completed: 7338 7337 7336 7335 7334
22565 CLIENT: Up:0 ds:302 act:2
225662024-12-04T13:35:25.049ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225672024-12-04T13:35:25.049ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62169) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225682024-12-04T13:35:25.050ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225692024-12-04T13:35:25.050ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225702024-12-04T13:35:25.050ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225712024-12-04T13:35:25.050ZINFOcrucible: send last flush ID to this DS: 7036 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225722024-12-04T13:35:25.050ZINFOcrucible: Replied this last flush ID: 7036 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225732024-12-04T13:35:25.050ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225742024-12-04T13:35:25.050ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225752024-12-04T13:35:25.050ZINFOcrucible: [0] client re-new 302 jobs since flush 7036 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225762024-12-04T13:35:25.051ZINFOcrucible: [0] Replayed 302 jobs since flush: 7036 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
22577 ----------------------------------------------------------------
22578 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22579 Downstairs last five completed: 7338 7337 7336 7335 7334
22580 Upstairs last five completed: 7338 7337 7336 7335 7334
22581 CLIENT: Up:0 ds:0 act:3
22582 Replay: All jobs finished
22583 Main task has work to do, go find it
22584 got dsc Stop(2)
22585 stop 2
22586 [8830] Got stop action so:false kr:false
22587 Dec 04 13:35:27.292 INFO request completed, latency_us: 309, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 3e7ee29d-ece2-498d-bc1c-34e969220f4e, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
225882024-12-04T13:35:27.292ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225892024-12-04T13:35:27.292ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225902024-12-04T13:35:27.292ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225912024-12-04T13:35:27.292ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225922024-12-04T13:35:27.292ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
225932024-12-04T13:35:27.292ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
22594 Dec 04 13:35:27.292 INFO request completed, latency_us: 156, response_code: 200, uri: /state/cid/2, method: GET, req_id: fd564732-e518-4c77-82c4-2db0dab61d3d, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
22595 [8830] Exited with: ExitStatus(unix_wait_status(9))
22596 [8830][2] reports Exit
22597 Dec 04 13:35:31.295 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/2, method: GET, req_id: 59931040-4fe5-4fc7-95da-2d6a2f57cd57, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
22598 001/300 Write block 3274 len 32768 data: 4 3 3 2 2 2 2 2
22599 002/300 Write block 2607 len 16384 data: 3 3 2 3
22600 003/300 Read block 2272 len 32768
22601 004/300 Read block 4823 len 36864
22602 005/300 Write block 1768 len 12288 data: 9 8 7
22603 006/300 Write block 1012 len 32768 data: 4 3 4 6 7 6 6 6
22604 007/300 Write block 4423 len 40960 data: 6 5 5 6 5 4 2 2 3 4
22605 008/300 Flush
22606 009/300 Read block 640 len 4096
22607 010/300 Read block 1913 len 12288
22608 011/300 Write block 1097 len 24576 data: 4 6 6 6 6 6
22609 012/300 Write block 3041 len 40960 data: 2 2 3 2 2 3 3 3 3 5
22610 013/300 Read block 3715 len 4096
22611 014/300 Read block 4316 len 36864
22612 015/300 Read block 2659 len 20480
22613 016/300 Write block 4362 len 4096 data: 4
22614 017/300 Write block 529 len 24576 data: 2 2 2 2 2 4
22615 018/300 Read block 4317 len 24576
22616 019/300 Read block 1589 len 36864
22617 020/300 Read block 4735 len 4096
22618 021/300 Write block 2430 len 16384 data: 5 5 5 5
22619 022/300 Read block 4107 len 40960
22620 023/300 Read block 1808 len 20480
22621 024/300 Read block 2387 len 8192
22622 025/300 Read block 612 len 36864
22623 026/300 Write block 4313 len 24576 data: 4 4 3 4 3 4
22624 027/300 Write block 4472 len 32768 data: 8 6 6 6 6 3 3 4
22625 028/300 Flush
22626 029/300 Write block 2418 len 4096 data: 5
22627 030/300 Read block 4436 len 16384
22628 031/300 Read block 1267 len 32768
22629 032/300 Write block 308 len 20480 data: 5 5 4 5 6
22630 033/300 Read block 2093 len 12288
22631 034/300 Read block 4968 len 36864
22632 035/300 Read block 2249 len 4096
22633 036/300 Write block 4360 len 12288 data: 4 4 5
22634 037/300 Write block 2942 len 40960 data: 5 5 6 6 5 4 3 3 4 4
22635 038/300 Read block 4149 len 28672
22636 039/300 Write block 158 len 40960 data: 5 5 7 8 8 8 8 8 6 7
22637 040/300 Write block 4620 len 12288 data: 6 6 5
22638 041/300 Read block 1396 len 4096
22639 042/300 Read block 2051 len 20480
22640 043/300 Write block 24 len 8192 data: 5 4
22641 044/300 Write block 885 len 16384 data: 6 6 5 5
22642 045/300 Read block 125 len 24576
22643 046/300 Read block 4488 len 4096
22644 047/300 Read block 2175 len 36864
22645 048/300 Write block 3783 len 8192 data: 4 2
22646 049/300 Write block 1191 len 8192 data: 4 5
22647 050/300 Read block 529 len 40960
22648 051/300 Write block 587 len 32768 data: 2 2 3 3 4 5 5 5
22649 052/300 Read block 1202 len 4096
22650 053/300 Write block 1086 len 8192 data: 2 2
22651 054/300 Read block 1794 len 12288
22652 055/300 Read block 4811 len 16384
22653 056/300 Read block 615 len 20480
22654 057/300 Write block 1898 len 24576 data: 6 6 4 3 5 4
22655 058/300 Write block 3354 len 32768 data: 3 3 5 5 3 4 5 5
22656 059/300 Read block 600 len 28672
22657 060/300 Read block 3793 len 4096
22658 061/300 Read block 213 len 4096
22659 062/300 Read block 4054 len 24576
22660 063/300 Write block 1410 len 36864 data: 3 3 4 4 5 5 4 4 4
22661 064/300 Write block 1453 len 28672 data: 5 5 7 7 9 9 7
22662 065/300 Read block 1844 len 4096
22663 066/300 Read block 2824 len 12288
22664 067/300 Write block 253 len 12288 data: 10 8 7
22665 068/300 Read block 3291 len 4096
22666 069/300 Flush
22667 070/300 Read block 1688 len 8192
22668 071/300 Read block 3032 len 16384
22669 072/300 Read block 561 len 36864
22670 073/300 Read block 4344 len 28672
22671 074/300 Write block 4030 len 8192 data: 7 7
22672 075/300 Read block 4762 len 32768
22673 076/300 Write block 549 len 20480 data: 5 6 4 3 3
22674 077/300 Flush
22675 078/300 Read block 3629 len 24576
22676 079/300 Read block 4445 len 20480
22677 080/300 Write block 45 len 16384 data: 4 3 4 3
22678 081/300 Flush
22679 082/300 Flush
22680 083/300 Write block 274 len 24576 data: 6 6 6 5 6 8
22681 084/300 Read block 644 len 36864
22682 085/300 Read block 4752 len 28672
22683 086/300 Write block 4698 len 40960 data: 5 4 4 3 4 5 6 4 6 5
22684 087/300 Read block 4592 len 8192
22685 088/300 Write block 1082 len 28672 data: 6 5 3 3 3 3 2
22686 089/300 Read block 3446 len 8192
22687 090/300 Write block 3193 len 12288 data: 4 5 5
22688 091/300 Read block 2421 len 4096
22689 092/300 Read block 2090 len 40960
22690 093/300 Write block 1928 len 8192 data: 3 4
22691 094/300 Write block 2454 len 36864 data: 3 3 4 5 4 5 5 6 7
22692 095/300 Read block 2073 len 16384
22693 096/300 Write block 4228 len 28672 data: 4 5 4 6 5 5 6
22694 097/300 Read block 1983 len 36864
22695 098/300 Read block 1687 len 8192
22696 099/300 Read block 1537 len 20480
22697 100/300 Write block 3727 len 36864 data: 2 2 3 3 3 4 3 3 4
22698 101/300 Read block 107 len 4096
22699 102/300 Write block 4358 len 20480 data: 4 4 5 5 6
22700 103/300 Write block 2744 len 8192 data: 5 5
22701 104/300 Read block 2829 len 20480
22702 105/300 Write block 2279 len 20480 data: 8 5 4 4 4
22703 106/300 Write block 2415 len 4096 data: 5
22704 107/300 Read block 3078 len 24576
22705 108/300 Read block 4716 len 36864
22706 109/300 Flush
22707 110/300 Read block 4109 len 4096
22708 111/300 Write block 2269 len 28672 data: 6 5 5 4 6 5 6
22709 112/300 Flush
22710 113/300 Flush
22711 114/300 Write block 878 len 4096 data: 2
22712 115/300 Read block 1681 len 24576
22713 116/300 Read block 436 len 36864
22714 117/300 Read block 2214 len 28672
22715 118/300 Write block 241 len 16384 data: 3 3 4 4
22716 119/300 Read block 1202 len 36864
22717 120/300 Flush
22718 121/300 Read block 399 len 16384
22719 122/300 Read block 1323 len 32768
22720 123/300 Write block 1516 len 12288 data: 8 6 6
22721 124/300 Read block 2712 len 4096
22722 125/300 Read block 153 len 36864
22723 126/300 Write block 1533 len 12288 data: 5 5 4
22724 127/300 Read block 876 len 40960
22725 128/300 Read block 3553 len 36864
22726 129/300 Write block 1808 len 20480 data: 5 5 5 4 3
22727 130/300 Write block 1541 len 16384 data: 4 5 6 6
22728 131/300 Read block 4973 len 28672
22729 132/300 Write block 3253 len 4096 data: 4
22730 133/300 Write block 4625 len 20480 data: 4 3 3 2 2
22731 134/300 Read block 4187 len 40960
22732 135/300 Write block 4530 len 32768 data: 3 2 3 3 3 3 4 4
22733 136/300 Flush
22734 137/300 Read block 2119 len 24576
22735 138/300 Write block 758 len 4096 data: 3
22736 139/300 Write block 542 len 16384 data: 7 7 7 7
22737 140/300 Write block 5 len 12288 data: 5 5 6
22738 141/300 Write block 1612 len 28672 data: 6 6 6 5 4 4 4
22739 142/300 Read block 2802 len 8192
22740 143/300 Read block 4780 len 4096
22741 144/300 Read block 4749 len 32768
22742 145/300 Flush
22743 146/300 Read block 3294 len 16384
22744 147/300 Read block 4659 len 12288
22745 148/300 Read block 3105 len 24576
22746 149/300 Read block 2449 len 8192
22747 150/300 Read block 3777 len 32768
22748 151/300 Write block 4536 len 16384 data: 5 5 4 4
22749 152/300 Write block 3886 len 20480 data: 5 4 4 4 3
22750 153/300 Write block 2863 len 8192 data: 5 4
22751 154/300 Read block 4938 len 20480
22752 155/300 Write block 169 len 12288 data: 8 8 7
22753 156/300 Read block 4618 len 32768
22754 157/300 Read block 442 len 16384
22755 158/300 Read block 4708 len 28672
22756 159/300 Read block 4273 len 24576
22757 160/300 Read block 3069 len 24576
22758 161/300 Read block 3602 len 12288
22759 162/300 Read block 3539 len 16384
22760 163/300 Read block 2649 len 4096
22761 164/300 Flush
22762 165/300 Flush
22763 166/300 Write block 4062 len 16384 data: 5 4 5 5
22764 167/300 Read block 2046 len 28672
22765 168/300 Write block 3514 len 40960 data: 6 8 8 9 8 8 6 6 4 3
22766 169/300 Read block 2015 len 12288
22767 170/300 Write block 4832 len 16384 data: 5 4 4 4
22768 171/300 Read block 3181 len 8192
22769 172/300 Flush
22770 173/300 Write block 4126 len 28672 data: 5 5 6 8 8 9 9
22771 174/300 Write block 3301 len 40960 data: 8 7 6 4 5 4 4 4 4 5
22772 175/300 Read block 3195 len 20480
22773 176/300 Read block 4348 len 28672
22774 177/300 Write block 1768 len 16384 data: 10 9 8 5
22775 178/300 Read block 2862 len 32768
22776 179/300 Write block 747 len 36864 data: 3 3 3 2 2 2 3 5 4
22777 180/300 Write block 3850 len 16384 data: 4 3 2 2
22778 181/300 Write block 4336 len 36864 data: 7 6 6 8 5 5 4 5 5
22779 182/300 Read block 2424 len 8192
22780 183/300 Write block 4706 len 20480 data: 7 6 5 4 4
22781 184/300 Write block 4234 len 36864 data: 7 7 8 7 7 6 7 7 6
22782 185/300 Write block 2252 len 16384 data: 4 4 4 3
22783 186/300 Read block 1224 len 12288
22784 187/300 Write block 4615 len 4096 data: 5
22785 188/300 Write block 643 len 28672 data: 4 5 5 5 5 4 4
22786 189/300 Write block 1698 len 16384 data: 5 4 4 4
22787 190/300 Read block 4587 len 16384
22788 191/300 Write block 154 len 12288 data: 5 5 5
22789 192/300 Write block 4363 len 16384 data: 5 4 4 4
22790 193/300 Write block 2478 len 40960 data: 7 7 7 7 6 8 8 7 6 6
22791 194/300 Flush
22792 195/300 Write block 1632 len 32768 data: 5 5 5 4 4 3 2 3
22793 196/300 Read block 2957 len 36864
22794 197/300 Write block 4734 len 20480 data: 5 6 6 3 3
22795 198/300 Write block 1281 len 28672 data: 4 4 4 5 4 4 4
22796 199/300 Write block 3870 len 20480 data: 5 4 4 5 4
22797 200/300 Write block 2358 len 32768 data: 2 3 3 2 2 4 5 5
22798 201/300 Read block 425 len 4096
22799 202/300 Write block 1317 len 20480 data: 3 4 4 5 5
22800 203/300 Write block 1558 len 12288 data: 5 5 5
22801 204/300 Read block 2638 len 36864
22802 205/300 Write block 2771 len 24576 data: 7 8 8 8 8 6
22803 206/300 Write block 3143 len 16384 data: 7 5 3 4
22804 207/300 Write block 4609 len 36864 data: 7 7 4 3 4 5 6 5 5
22805 208/300 Read block 1614 len 40960
22806 209/300 Write block 2118 len 28672 data: 2 3 4 5 5 5 5
22807 210/300 Flush
22808 211/300 Read block 982 len 32768
22809 212/300 Read block 3573 len 40960
22810 213/300 Flush
22811 214/300 Write block 3278 len 36864 data: 3 3 3 3 2 3 3 2 2
22812 215/300 Write block 1835 len 28672 data: 4 5 4 6 7 6 6
22813 216/300 Read block 2529 len 40960
22814 217/300 Read block 178 len 24576
22815 218/300 Write block 2850 len 20480 data: 6 7 9 6 5
22816 219/300 Read block 4797 len 16384
22817 220/300 Write block 2889 len 28672 data: 5 4 3 3 4 5 5
22818 221/300 Write block 2885 len 32768 data: 3 3 3 4 6 5 4 4
22819 222/300 Read block 1915 len 32768
22820 223/300 Read block 3178 len 36864
22821 224/300 Read block 1940 len 28672
22822 225/300 Write block 2384 len 36864 data: 4 5 5 4 4 6 6 6 6
22823 226/300 Read block 3154 len 20480
22824 227/300 Flush
22825 228/300 Read block 4406 len 24576
22826 229/300 Write block 1274 len 24576 data: 3 4 4 4 5 3
22827 230/300 Read block 3913 len 40960
22828 231/300 Read block 3608 len 16384
22829 232/300 Write block 3756 len 24576 data: 3 5 5 6 5 5
22830 233/300 Write block 3019 len 28672 data: 5 3 4 4 5 4 4
22831 234/300 Write block 1889 len 20480 data: 8 8 7 7 7
22832 235/300 Read block 355 len 28672
22833 236/300 Read block 3451 len 16384
22834 237/300 Write block 1980 len 36864 data: 4 4 4 4 5 6 6 5 6
22835 238/300 Read block 2779 len 40960
22836 239/300 Read block 2933 len 8192
22837 240/300 Write block 1970 len 12288 data: 3 3 3
22838 241/300 Write block 679 len 16384 data: 3 4 6 4
22839 242/300 Read block 3897 len 8192
22840 243/300 Read block 812 len 16384
22841 244/300 Read block 582 len 32768
22842 245/300 Write block 3373 len 24576 data: 4 4 4 4 6 7
22843 246/300 Read block 3880 len 8192
22844 247/300 Read block 1282 len 20480
22845 248/300 Write block 3830 len 28672 data: 5 5 4 4 2 2 2
22846 249/300 Read block 845 len 36864
22847 250/300 Read block 1725 len 36864
22848 251/300 Write block 3773 len 28672 data: 5 5 6 6 5 5 5
22849 252/300 Write block 751 len 40960 data: 3 3 4 6 5 4 4 4 3 3
22850 253/300 Read block 1653 len 4096
22851 254/300 Read block 2421 len 40960
22852 255/300 Write block 2440 len 8192 data: 6 5
22853 256/300 Flush
22854 257/300 Write block 1729 len 8192 data: 6 7
22855 258/300 Write block 2471 len 20480 data: 5 5 5 6 6
22856 259/300 Read block 892 len 32768
22857 260/300 Write block 869 len 12288 data: 4 4 4
22858 261/300 Write block 164 len 20480 data: 9 9 7 8 7
22859 262/300 Read block 3824 len 20480
22860 263/300 Flush
22861 264/300 Write block 3838 len 8192 data: 3 3
22862 265/300 Write block 2535 len 20480 data: 2 4 7 6 6
22863 266/300 Read block 2966 len 32768
22864 267/300 Read block 4182 len 16384
22865 268/300 Write block 1843 len 24576 data: 6 5 6 6 6 4
22866 269/300 Read block 434 len 4096
22867 270/300 Flush
22868 271/300 Write block 2193 len 12288 data: 7 6 6
22869 272/300 Read block 112 len 32768
22870 273/300 Write block 2032 len 8192 data: 3 3
22871 274/300 Read block 1348 len 32768
22872 275/300 Write block 4959 len 12288 data: 4 3 2
22873 276/300 Flush
22874 277/300 Read block 3723 len 20480
22875 278/300 Write block 611 len 4096 data: 5
22876 279/300 Read block 4444 len 16384
22877 280/300 Read block 430 len 32768
22878 281/300 Read block 2963 len 24576
22879 282/300 Read block 31 len 40960
22880 283/300 Write block 1746 len 12288 data: 9 9 10
22881 284/300 Read block 1550 len 4096
22882 285/300 Read block 3376 len 32768
22883 286/300 Write block 2961 len 4096 data: 3
22884 287/300 Write block 1747 len 32768 data: 10 11 9 6 7 5 5 5
22885 288/300 Read block 2194 len 12288
22886 289/300 Read block 1029 len 8192
22887 290/300 Read block 4463 len 12288
22888 291/300 Write block 1683 len 32768 data: 8 10 9 8 7 6 4 5
22889 292/300 Read block 125 len 36864
22890 293/300 Read block 1601 len 16384
22891 294/300 Read block 3832 len 36864
22892 295/300 Read block 1614 len 4096
22893 296/300 Read block 3274 len 8192
22894 297/300 Read block 486 len 16384
22895 298/300 Read block 2797 len 32768
22896 299/300 Write block 250 len 4096 data: 8
22897 300/300 Write block 2189 len 40960 data: 5 7 8 7 8 7 7 7 5 4
22898 301/300 Read block 45 len 8192
22899 Main task has work to do, go find it
22900 got dsc Start(2)
22901 start 2
22902 Dec 04 13:35:31.529 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/2, method: POST, req_id: 096e9fb7-0651-4481-9046-962341dab163, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
22903 Replay: started 2, returned:Ok(())
22904 ----------------------------------------------------------------
22905 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22906 States: Active Active Offline
22907 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22908 7339 Acked Write 8 Done Done Sent false
22909 7340 Acked Write 4 Done Done Sent false
22910 7341 Acked Read 8 Done Done Sent false
22911 7342 Acked Read 9 Done Done Sent false
22912 7343 Acked Write 3 Done Done Sent false
22913 7344 Acked Write 8 Done Done Sent false
22914 7345 Acked Write 10 Done Done Sent false
22915 7346 Acked Flush 0 Done Done Sent false
22916 7347 Acked Read 1 Done Done Sent false
22917 7348 Acked Read 3 Done Done Sent false
22918 7349 Acked Write 6 Done Done Sent false
22919 7350 Acked Write 10 Done Done Sent false
22920 7351 Acked Read 1 Done Done Sent false
22921 7352 Acked Read 9 Done Done Sent false
22922 7353 Acked Read 5 Done Done Sent false
22923 7354 Acked Write 1 Done Done Sent false
22924 7355 Acked Write 6 Done Done Sent false
22925 7356 Acked Read 6 Done Done Sent false
22926 7357 Acked Read 9 Done Done Sent false
22927 7358 Acked Read 1 Done Done Sent false
22928 7359 Acked Write 4 Done Done Sent false
22929 7360 Acked Read 10 Done Done Sent false
22930 7361 Acked Read 5 Done Done Sent false
22931 7362 Acked Read 2 Done Done Sent false
22932 7363 Acked Read 9 Done Done Sent false
22933 7364 Acked Write 6 Done Done Sent false
22934 7365 Acked Write 8 Done Done Sent false
22935 7366 Acked Flush 0 Done Done Sent false
22936 7367 Acked Write 1 Done Done Sent false
22937 7368 Acked Read 4 Done Done Sent false
22938 7369 Acked Read 8 Done Done Sent false
22939 7370 Acked Write 5 Done Done Sent false
22940 7371 Acked Read 3 Done Done Sent false
22941 7372 Acked Read 9 Done Done Sent false
22942 7373 Acked Read 1 Done Done Sent false
22943 7374 Acked Write 3 Done Done Sent false
22944 7375 Acked Write 10 Done Done Sent false
22945 7376 Acked Read 7 Done Done Sent false
22946 7377 Acked Write 10 Done Done Sent false
22947 7378 Acked Write 3 Done Done Sent false
22948 7379 Acked Read 1 Done Done Sent false
22949 7380 Acked Read 5 Done Done Sent false
22950 7381 Acked Write 2 Done Done Sent false
22951 7382 Acked Write 4 Done Done Sent false
22952 7383 Acked Read 6 Done Done Sent false
22953 7384 Acked Read 1 Done Done Sent false
22954 7385 Acked Read 9 Done Done Sent false
22955 7386 Acked Write 2 Done Done Sent false
22956 7387 Acked Write 2 Done Done Sent false
22957 7388 Acked Read 10 Done Done Sent false
22958 7389 Acked Write 8 Done Done Sent false
22959 7390 Acked Read 1 Done Done Sent false
22960 7391 Acked Write 2 Done Done Sent false
22961 7392 Acked Read 3 Done Done Sent false
22962 7393 Acked Read 4 Done Done Sent false
22963 7394 Acked Read 5 Done Done Sent false
22964 7395 Acked Write 6 Done Done Sent false
22965 7396 Acked Write 8 Done Done Sent false
22966 7397 Acked Read 7 Done Done Sent false
22967 7398 Acked Read 1 Done Done Sent false
22968 7399 Acked Read 1 Done Done Sent false
22969 7400 Acked Read 6 Done Done Sent false
22970 7401 Acked Write 9 Done Done Sent false
22971 7402 Acked Write 7 Done Done Sent false
22972 7403 Acked Read 1 Done Done Sent false
22973 7404 Acked Read 3 Done Done Sent false
22974 7405 Acked Write 3 Done Done Sent false
22975 7406 Acked Read 1 Done Done Sent false
22976 7407 Acked Flush 0 Done Done Sent false
22977 7408 Acked Read 2 Done Done Sent false
22978 7409 Acked Read 4 Done Done Sent false
22979 7410 Acked Read 9 Done Done Sent false
22980 7411 Acked Read 7 Done Done Sent false
22981 7412 Acked Write 2 Done Done Sent false
22982 7413 Acked Read 8 Done Done Sent false
22983 7414 Acked Write 5 Done Done Sent false
22984 7415 Acked Flush 0 Done Done Sent false
22985 7416 Acked Read 6 Done Done Sent false
22986 7417 Acked Read 5 Done Done Sent false
22987 7418 Acked Write 4 Done Done Sent false
22988 7419 Acked Flush 0 Done Done Sent false
22989 7420 Acked Flush 0 Done Done Sent false
22990 7421 Acked Write 6 Done Done Sent false
22991 7422 Acked Read 9 Done Done Sent false
22992 7423 Acked Read 7 Done Done Sent false
22993 7424 Acked Write 10 Done Done Sent false
22994 7425 Acked Read 2 Done Done Sent false
22995 7426 Acked Write 7 Done Done Sent false
22996 7427 Acked Read 2 Done Done Sent false
22997 7428 Acked Write 3 Done Done Sent false
22998 7429 Acked Read 1 Done Done Sent false
22999 7430 Acked Read 10 Done Done Sent false
23000 7431 Acked Write 2 Done Done Sent false
23001 7432 Acked Write 9 Done Done Sent false
23002 7433 Acked Read 4 Done Done Sent false
23003 7434 Acked Write 7 Done Done Sent false
23004 7435 Acked Read 9 Done Done Sent false
23005 7436 Acked Read 2 Done Done Sent false
23006 7437 Acked Read 5 Done Done Sent false
23007 7438 Acked Write 9 Done Done Sent false
23008 7439 Acked Read 1 Done Done Sent false
23009 7440 Acked Write 5 Done Done Sent false
23010 7441 Acked Write 2 Done Done Sent false
23011 7442 Acked Read 5 Done Done Sent false
23012 7443 Acked Write 5 Done Done Sent false
23013 7444 Acked Write 1 Done Done Sent false
23014 7445 Acked Read 6 Done Done Sent false
23015 7446 Acked Read 9 Done Done Sent false
23016 7447 Acked Flush 0 Done Done Sent false
23017 7448 Acked Read 1 Done Done Sent false
23018 7449 Acked Write 7 Done Done Sent false
23019 7450 Acked Flush 0 Done Done Sent false
23020 7451 Acked Flush 0 Done Done Sent false
23021 7452 Acked Write 1 Done Done Sent false
23022 7453 Acked Read 6 Done Done Sent false
23023 7454 Acked Read 9 Done Done Sent false
23024 7455 Acked Read 7 Done Done Sent false
23025 7456 Acked Write 4 Done Done Sent false
23026 7457 Acked Read 9 Done Done Sent false
23027 7458 Acked Flush 0 Done Done Sent false
23028 7459 Acked Read 4 Done Done Sent false
23029 7460 Acked Read 8 Done Done Sent false
23030 7461 Acked Write 3 Done Done Sent false
23031 7462 Acked Read 1 Done Done Sent false
23032 7463 Acked Read 9 Done Done Sent false
23033 7464 Acked Write 3 Done Done Sent false
23034 7465 Acked Read 10 Done Done Sent false
23035 7466 Acked Read 9 Done Done Sent false
23036 7467 Acked Write 5 Done Done Sent false
23037 7468 Acked Write 4 Done Done Sent false
23038 7469 Acked Read 7 Done Done Sent false
23039 7470 Acked Write 1 Done Done Sent false
23040 7471 Acked Write 5 Done Done Sent false
23041 7472 Acked Read 10 Done Done Sent false
23042 7473 Acked Write 8 Done Done Sent false
23043 7474 Acked Flush 0 Done Done Sent false
23044 7475 Acked Read 6 Done Done Sent false
23045 7476 Acked Write 1 Done Done Sent false
23046 7477 Acked Write 4 Done Done Sent false
23047 7478 Acked Write 3 Done Done Sent false
23048 7479 Acked Write 7 Done Done Sent false
23049 7480 Acked Read 2 Done Done Sent false
23050 7481 Acked Read 1 Done Done Sent false
23051 7482 Acked Read 8 Done Done Sent false
23052 7483 Acked Flush 0 Done Done Sent false
23053 7484 Acked Read 4 Done Done Sent false
23054 7485 Acked Read 3 Done Done Sent false
23055 7486 Acked Read 6 Done Done Sent false
23056 7487 Acked Read 2 Done Done Sent false
23057 7488 Acked Read 8 Done Done Sent false
23058 7489 Acked Write 4 Done Done Sent false
23059 7490 Acked Write 5 Done Done Sent false
23060 7491 Acked Write 2 Done Done Sent false
23061 7492 Acked Read 5 Done Done Sent false
23062 7493 Acked Write 3 Done Done Sent false
23063 7494 Acked Read 8 Done Done Sent false
23064 7495 Acked Read 4 Done Done Sent false
23065 7496 Acked Read 7 Done Done Sent false
23066 7497 Acked Read 6 Done Done Sent false
23067 7498 Acked Read 6 Done Done Sent false
23068 7499 Acked Read 3 Done Done Sent false
23069 7500 Acked Read 4 Done Done Sent false
23070 7501 Acked Read 1 Done Done Sent false
23071 7502 Acked Flush 0 Done Done Sent false
23072 7503 Acked Flush 0 Done Done Sent false
23073 7504 Acked Write 4 Done Done Sent false
23074 7505 Acked Read 7 Done Done Sent false
23075 7506 Acked Write 10 Done Done Sent false
23076 7507 Acked Read 3 Done Done Sent false
23077 7508 Acked Write 4 Done Done Sent false
23078 7509 Acked Read 2 Done Done Sent false
23079 7510 Acked Flush 0 Done Done Sent false
23080 7511 Acked Write 7 Done Done Sent false
23081 7512 Acked Write 10 Done Done Sent false
23082 7513 Acked Read 5 Done Done Sent false
23083 7514 Acked Read 7 Done Done Sent false
23084 7515 Acked Write 4 Done Done Sent false
23085 7516 Acked Read 8 Done Done Sent false
23086 7517 Acked Write 9 Done Done Sent false
23087 7518 Acked Write 4 Done Done Sent false
23088 7519 Acked Write 9 Done Done Sent false
23089 7520 Acked Read 2 Done Done Sent false
23090 7521 Acked Write 5 Done Done Sent false
23091 7522 Acked Write 9 Done Done Sent false
23092 7523 Acked Write 4 Done Done Sent false
23093 7524 Acked Read 3 Done Done Sent false
23094 7525 Acked Write 1 Done Done Sent false
23095 7526 Acked Write 7 Done Done Sent false
23096 7527 Acked Write 4 Done Done Sent false
23097 7528 Acked Read 4 Done Done Sent false
23098 7529 Acked Write 3 Done Done Sent false
23099 7530 Acked Write 4 Done Done Sent false
23100 7531 Acked Write 10 Done Done Sent false
23101 7532 Acked Flush 0 Done Done Sent false
23102 7533 Acked Write 8 Done Done Sent false
23103 7534 Acked Read 9 Done Done Sent false
23104 7535 Acked Write 5 Done Done Sent false
23105 7536 Acked Write 7 Done Done Sent false
23106 7537 Acked Write 5 Done Done Sent false
23107 7538 Acked Write 8 Done Done Sent false
23108 7539 Acked Read 1 Done Done Sent false
23109 7540 Acked Write 5 Done Done Sent false
23110 7541 Acked Write 3 Done Done Sent false
23111 7542 Acked Read 9 Done Done Sent false
23112 7543 Acked Write 6 Done Done Sent false
23113 7544 Acked Write 4 Done Done Sent false
23114 7545 Acked Write 9 Done Done Sent false
23115 7546 Acked Read 10 Done Done Sent false
23116 7547 Acked Write 7 Done Done Sent false
23117 7548 Acked Flush 0 Done Done Sent false
23118 7549 Acked Read 8 Done Done Sent false
23119 7550 Acked Read 10 Done Done Sent false
23120 7551 Acked Flush 0 Done Done Sent false
23121 7552 Acked Write 9 Done Done Sent false
23122 7553 Acked Write 7 Done Done Sent false
23123 7554 Acked Read 10 Done Done Sent false
23124 7555 Acked Read 6 Done Done Sent false
23125 7556 Acked Write 5 Done Done Sent false
23126 7557 Acked Read 4 Done Done Sent false
23127 7558 Acked Write 7 Done Done Sent false
23128 7559 Acked Write 8 Done Done Sent false
23129 7560 Acked Read 8 Done Done Sent false
23130 7561 Acked Read 9 Done Done Sent false
23131 7562 Acked Read 7 Done Done Sent false
23132 7563 Acked Write 9 Done Done Sent false
23133 7564 Acked Read 5 Done Done Sent false
23134 7565 Acked Flush 0 Done Done Sent false
23135 7566 Acked Read 6 Done Done Sent false
23136 7567 Acked Write 6 Done Done Sent false
23137 7568 Acked Read 10 Done Done Sent false
23138 7569 Acked Read 4 Done Done Sent false
23139 7570 Acked Write 6 Done Done Sent false
23140 7571 Acked Write 7 Done Done Sent false
23141 7572 Acked Write 5 Done Done Sent false
23142 7573 Acked Read 7 Done Done Sent false
23143 7574 Acked Read 4 Done Done Sent false
23144 7575 Acked Write 9 Done Done Sent false
23145 7576 Acked Read 10 Done Done Sent false
23146 7577 Acked Read 2 Done Done Sent false
23147 7578 Acked Write 3 Done Done Sent false
23148 7579 Acked Write 4 Done Done Sent false
23149 7580 Acked Read 2 Done Done Sent false
23150 7581 Acked Read 4 Done Done Sent false
23151 7582 Acked Read 8 Done Done Sent false
23152 7583 Acked Write 6 Done Done Sent false
23153 7584 Acked Read 2 Done Done Sent false
23154 7585 Acked Read 5 Done Done Sent false
23155 7586 Acked Write 7 Done Done Sent false
23156 7587 Acked Read 9 Done Done Sent false
23157 7588 Acked Read 9 Done Done Sent false
23158 7589 Acked Write 7 Done Done Sent false
23159 7590 Acked Write 10 Done Done Sent false
23160 7591 Acked Read 1 Done Done Sent false
23161 7592 Acked Read 10 Done Done Sent false
23162 7593 Acked Write 2 Done Done Sent false
23163 7594 Acked Flush 0 Done Done Sent false
23164 7595 Acked Write 2 Done Done Sent false
23165 7596 Acked Write 5 Done Done Sent false
23166 7597 Acked Read 8 Done Done Sent false
23167 7598 Acked Write 3 Done Done Sent false
23168 7599 Acked Write 5 Done Done Sent false
23169 7600 Acked Read 5 Done Done Sent false
23170 7601 Acked Flush 0 Done Done Sent false
23171 7602 Acked Write 2 Done Done Sent false
23172 7603 Acked Write 5 Done Done Sent false
23173 7604 Acked Read 8 Done Done Sent false
23174 7605 Acked Read 4 Done Done Sent false
23175 7606 Acked Write 6 Done Done Sent false
23176 7607 Acked Read 1 Done Done Sent false
23177 7608 Acked Flush 0 Done Done Sent false
23178 7609 Acked Write 3 Done Done Sent false
23179 7610 Acked Read 8 Done Done Sent false
23180 7611 Acked Write 2 Done Done Sent false
23181 7612 Acked Read 8 Done Done Sent false
23182 7613 Acked Write 3 Done Done Sent false
23183 7614 Acked Flush 0 Done Done Sent false
23184 7615 Acked Read 5 Done Done Sent false
23185 7616 Acked Write 1 Done Done Sent false
23186 7617 Acked Read 4 Done Done Sent false
23187 7618 Acked Read 8 Done Done Sent false
23188 7619 Acked Read 6 Done Done Sent false
23189 7620 Acked Read 10 Done Done Sent false
23190 7621 Acked Write 3 Done Done Sent false
23191 7622 Acked Read 1 Done Done Sent false
23192 7623 Acked Read 8 Done Done Sent false
23193 7624 Acked Write 1 Done Done Sent false
23194 7625 Acked Write 8 Done Done Sent false
23195 7626 Acked Read 3 Done Done Sent false
23196 7627 Acked Read 2 Done Done Sent false
23197 7628 Acked Read 3 Done Done Sent false
23198 7629 Acked Write 8 Done Done Sent false
23199 7630 Acked Read 9 Done Done Sent false
23200 7631 Acked Read 4 Done Done Sent false
23201 7632 Acked Read 9 Done Done Sent false
23202 7633 Acked Read 1 Done Done Sent false
23203 7634 Acked Read 2 Done Done Sent false
23204 7635 Acked Read 4 Done Done Sent false
23205 7636 Acked Read 8 Done Done Sent false
23206 7637 Acked Write 1 Done Done Sent false
23207 7638 Acked Write 10 Done Done Sent false
23208 7639 Acked Read 2 Done Done Sent false
23209 STATES DS:0 DS:1 DS:2 TOTAL
23210 Sent 0 0 301 301
23211 Done 301 301 0 602
23212 Skipped 0 0 0 0
23213 Error 0 0 0 0
23214 Last Flush: 7614 7614 7338
23215 Downstairs last five completed: 7338 7337 7336 7335 7334
23216 Upstairs last five completed: 7639 7638 7637 7636 7635
23217 CLIENT: Up:0 ds:301 act:2
23218 [8830] got start action
23219 [8830] I am going to restart
23220 Starting downstairs at port 8830
23221 Make output file at "/tmp/dsc/downstairs-8830.txt"
23222 [8830][2] reports Starting
23223 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1225)
23224 [8830][2] reports Running
23225 ----------------------------------------------------------------
23226 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23227 States: Active Active Offline
23228 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23229 7339 Acked Write 8 Done Done Sent false
23230 7340 Acked Write 4 Done Done Sent false
23231 7341 Acked Read 8 Done Done Sent false
23232 7342 Acked Read 9 Done Done Sent false
23233 7343 Acked Write 3 Done Done Sent false
23234 7344 Acked Write 8 Done Done Sent false
23235 7345 Acked Write 10 Done Done Sent false
23236 7346 Acked Flush 0 Done Done Sent false
23237 7347 Acked Read 1 Done Done Sent false
23238 7348 Acked Read 3 Done Done Sent false
23239 7349 Acked Write 6 Done Done Sent false
23240 7350 Acked Write 10 Done Done Sent false
23241 7351 Acked Read 1 Done Done Sent false
23242 7352 Acked Read 9 Done Done Sent false
23243 7353 Acked Read 5 Done Done Sent false
23244 7354 Acked Write 1 Done Done Sent false
23245 7355 Acked Write 6 Done Done Sent false
23246 7356 Acked Read 6 Done Done Sent false
23247 7357 Acked Read 9 Done Done Sent false
23248 7358 Acked Read 1 Done Done Sent false
23249 7359 Acked Write 4 Done Done Sent false
23250 7360 Acked Read 10 Done Done Sent false
23251 7361 Acked Read 5 Done Done Sent false
23252 7362 Acked Read 2 Done Done Sent false
23253 7363 Acked Read 9 Done Done Sent false
23254 7364 Acked Write 6 Done Done Sent false
23255 7365 Acked Write 8 Done Done Sent false
23256 7366 Acked Flush 0 Done Done Sent false
23257 7367 Acked Write 1 Done Done Sent false
23258 7368 Acked Read 4 Done Done Sent false
23259 7369 Acked Read 8 Done Done Sent false
23260 7370 Acked Write 5 Done Done Sent false
23261 7371 Acked Read 3 Done Done Sent false
23262 7372 Acked Read 9 Done Done Sent false
23263 7373 Acked Read 1 Done Done Sent false
23264 7374 Acked Write 3 Done Done Sent false
23265 7375 Acked Write 10 Done Done Sent false
23266 7376 Acked Read 7 Done Done Sent false
23267 7377 Acked Write 10 Done Done Sent false
23268 7378 Acked Write 3 Done Done Sent false
23269 7379 Acked Read 1 Done Done Sent false
23270 7380 Acked Read 5 Done Done Sent false
23271 7381 Acked Write 2 Done Done Sent false
23272 7382 Acked Write 4 Done Done Sent false
23273 7383 Acked Read 6 Done Done Sent false
23274 7384 Acked Read 1 Done Done Sent false
23275 7385 Acked Read 9 Done Done Sent false
23276 7386 Acked Write 2 Done Done Sent false
23277 7387 Acked Write 2 Done Done Sent false
23278 7388 Acked Read 10 Done Done Sent false
23279 7389 Acked Write 8 Done Done Sent false
23280 7390 Acked Read 1 Done Done Sent false
23281 7391 Acked Write 2 Done Done Sent false
23282 7392 Acked Read 3 Done Done Sent false
23283 7393 Acked Read 4 Done Done Sent false
23284 7394 Acked Read 5 Done Done Sent false
23285 7395 Acked Write 6 Done Done Sent false
23286 7396 Acked Write 8 Done Done Sent false
23287 7397 Acked Read 7 Done Done Sent false
23288 7398 Acked Read 1 Done Done Sent false
23289 7399 Acked Read 1 Done Done Sent false
23290 7400 Acked Read 6 Done Done Sent false
23291 7401 Acked Write 9 Done Done Sent false
23292 7402 Acked Write 7 Done Done Sent false
23293 7403 Acked Read 1 Done Done Sent false
23294 7404 Acked Read 3 Done Done Sent false
23295 7405 Acked Write 3 Done Done Sent false
23296 7406 Acked Read 1 Done Done Sent false
23297 7407 Acked Flush 0 Done Done Sent false
23298 7408 Acked Read 2 Done Done Sent false
23299 7409 Acked Read 4 Done Done Sent false
23300 7410 Acked Read 9 Done Done Sent false
23301 7411 Acked Read 7 Done Done Sent false
23302 7412 Acked Write 2 Done Done Sent false
23303 7413 Acked Read 8 Done Done Sent false
23304 7414 Acked Write 5 Done Done Sent false
23305 7415 Acked Flush 0 Done Done Sent false
23306 7416 Acked Read 6 Done Done Sent false
23307 7417 Acked Read 5 Done Done Sent false
23308 7418 Acked Write 4 Done Done Sent false
23309 7419 Acked Flush 0 Done Done Sent false
23310 7420 Acked Flush 0 Done Done Sent false
23311 7421 Acked Write 6 Done Done Sent false
23312 7422 Acked Read 9 Done Done Sent false
23313 7423 Acked Read 7 Done Done Sent false
23314 7424 Acked Write 10 Done Done Sent false
23315 7425 Acked Read 2 Done Done Sent false
23316 7426 Acked Write 7 Done Done Sent false
23317 7427 Acked Read 2 Done Done Sent false
23318 7428 Acked Write 3 Done Done Sent false
23319 7429 Acked Read 1 Done Done Sent false
23320 7430 Acked Read 10 Done Done Sent false
23321 7431 Acked Write 2 Done Done Sent false
23322 7432 Acked Write 9 Done Done Sent false
23323 7433 Acked Read 4 Done Done Sent false
23324 7434 Acked Write 7 Done Done Sent false
23325 7435 Acked Read 9 Done Done Sent false
23326 7436 Acked Read 2 Done Done Sent false
23327 7437 Acked Read 5 Done Done Sent false
23328 7438 Acked Write 9 Done Done Sent false
23329 7439 Acked Read 1 Done Done Sent false
23330 7440 Acked Write 5 Done Done Sent false
23331 7441 Acked Write 2 Done Done Sent false
23332 7442 Acked Read 5 Done Done Sent false
23333 7443 Acked Write 5 Done Done Sent false
23334 7444 Acked Write 1 Done Done Sent false
23335 7445 Acked Read 6 Done Done Sent false
23336 7446 Acked Read 9 Done Done Sent false
23337 7447 Acked Flush 0 Done Done Sent false
23338 7448 Acked Read 1 Done Done Sent false
23339 7449 Acked Write 7 Done Done Sent false
23340 7450 Acked Flush 0 Done Done Sent false
23341 7451 Acked Flush 0 Done Done Sent false
23342 7452 Acked Write 1 Done Done Sent false
23343 7453 Acked Read 6 Done Done Sent false
23344 7454 Acked Read 9 Done Done Sent false
23345 7455 Acked Read 7 Done Done Sent false
23346 7456 Acked Write 4 Done Done Sent false
23347 7457 Acked Read 9 Done Done Sent false
23348 7458 Acked Flush 0 Done Done Sent false
23349 7459 Acked Read 4 Done Done Sent false
23350 7460 Acked Read 8 Done Done Sent false
23351 7461 Acked Write 3 Done Done Sent false
23352 7462 Acked Read 1 Done Done Sent false
23353 7463 Acked Read 9 Done Done Sent false
23354 7464 Acked Write 3 Done Done Sent false
23355 7465 Acked Read 10 Done Done Sent false
23356 7466 Acked Read 9 Done Done Sent false
23357 7467 Acked Write 5 Done Done Sent false
23358 7468 Acked Write 4 Done Done Sent false
23359 7469 Acked Read 7 Done Done Sent false
23360 7470 Acked Write 1 Done Done Sent false
23361 7471 Acked Write 5 Done Done Sent false
23362 7472 Acked Read 10 Done Done Sent false
23363 7473 Acked Write 8 Done Done Sent false
23364 7474 Acked Flush 0 Done Done Sent false
23365 7475 Acked Read 6 Done Done Sent false
23366 7476 Acked Write 1 Done Done Sent false
23367 7477 Acked Write 4 Done Done Sent false
23368 7478 Acked Write 3 Done Done Sent false
23369 7479 Acked Write 7 Done Done Sent false
23370 7480 Acked Read 2 Done Done Sent false
23371 7481 Acked Read 1 Done Done Sent false
23372 7482 Acked Read 8 Done Done Sent false
23373 7483 Acked Flush 0 Done Done Sent false
23374 7484 Acked Read 4 Done Done Sent false
23375 7485 Acked Read 3 Done Done Sent false
23376 7486 Acked Read 6 Done Done Sent false
23377 7487 Acked Read 2 Done Done Sent false
23378 7488 Acked Read 8 Done Done Sent false
23379 7489 Acked Write 4 Done Done Sent false
23380 7490 Acked Write 5 Done Done Sent false
23381 7491 Acked Write 2 Done Done Sent false
23382 7492 Acked Read 5 Done Done Sent false
23383 7493 Acked Write 3 Done Done Sent false
23384 7494 Acked Read 8 Done Done Sent false
23385 7495 Acked Read 4 Done Done Sent false
23386 7496 Acked Read 7 Done Done Sent false
23387 7497 Acked Read 6 Done Done Sent false
23388 7498 Acked Read 6 Done Done Sent false
23389 7499 Acked Read 3 Done Done Sent false
23390 7500 Acked Read 4 Done Done Sent false
23391 7501 Acked Read 1 Done Done Sent false
23392 7502 Acked Flush 0 Done Done Sent false
23393 7503 Acked Flush 0 Done Done Sent false
23394 7504 Acked Write 4 Done Done Sent false
23395 7505 Acked Read 7 Done Done Sent false
23396 7506 Acked Write 10 Done Done Sent false
23397 7507 Acked Read 3 Done Done Sent false
23398 7508 Acked Write 4 Done Done Sent false
23399 7509 Acked Read 2 Done Done Sent false
23400 7510 Acked Flush 0 Done Done Sent false
23401 7511 Acked Write 7 Done Done Sent false
23402 7512 Acked Write 10 Done Done Sent false
23403 7513 Acked Read 5 Done Done Sent false
23404 7514 Acked Read 7 Done Done Sent false
23405 7515 Acked Write 4 Done Done Sent false
23406 7516 Acked Read 8 Done Done Sent false
23407 7517 Acked Write 9 Done Done Sent false
23408 7518 Acked Write 4 Done Done Sent false
23409 7519 Acked Write 9 Done Done Sent false
23410 7520 Acked Read 2 Done Done Sent false
23411 7521 Acked Write 5 Done Done Sent false
23412 7522 Acked Write 9 Done Done Sent false
23413 7523 Acked Write 4 Done Done Sent false
23414 7524 Acked Read 3 Done Done Sent false
23415 7525 Acked Write 1 Done Done Sent false
23416 7526 Acked Write 7 Done Done Sent false
23417 7527 Acked Write 4 Done Done Sent false
23418 7528 Acked Read 4 Done Done Sent false
23419 7529 Acked Write 3 Done Done Sent false
23420 7530 Acked Write 4 Done Done Sent false
23421 7531 Acked Write 10 Done Done Sent false
23422 7532 Acked Flush 0 Done Done Sent false
23423 7533 Acked Write 8 Done Done Sent false
23424 7534 Acked Read 9 Done Done Sent false
23425 7535 Acked Write 5 Done Done Sent false
23426 7536 Acked Write 7 Done Done Sent false
23427 7537 Acked Write 5 Done Done Sent false
23428 7538 Acked Write 8 Done Done Sent false
23429 7539 Acked Read 1 Done Done Sent false
23430 7540 Acked Write 5 Done Done Sent false
23431 7541 Acked Write 3 Done Done Sent false
23432 7542 Acked Read 9 Done Done Sent false
23433 7543 Acked Write 6 Done Done Sent false
23434 7544 Acked Write 4 Done Done Sent false
23435 7545 Acked Write 9 Done Done Sent false
23436 7546 Acked Read 10 Done Done Sent false
23437 7547 Acked Write 7 Done Done Sent false
23438 7548 Acked Flush 0 Done Done Sent false
23439 7549 Acked Read 8 Done Done Sent false
23440 7550 Acked Read 10 Done Done Sent false
23441 7551 Acked Flush 0 Done Done Sent false
23442 7552 Acked Write 9 Done Done Sent false
23443 7553 Acked Write 7 Done Done Sent false
23444 7554 Acked Read 10 Done Done Sent false
23445 7555 Acked Read 6 Done Done Sent false
23446 7556 Acked Write 5 Done Done Sent false
23447 7557 Acked Read 4 Done Done Sent false
23448 7558 Acked Write 7 Done Done Sent false
23449 7559 Acked Write 8 Done Done Sent false
23450 7560 Acked Read 8 Done Done Sent false
23451 7561 Acked Read 9 Done Done Sent false
23452 7562 Acked Read 7 Done Done Sent false
23453 7563 Acked Write 9 Done Done Sent false
23454 7564 Acked Read 5 Done Done Sent false
23455 7565 Acked Flush 0 Done Done Sent false
23456 7566 Acked Read 6 Done Done Sent false
23457 7567 Acked Write 6 Done Done Sent false
23458 7568 Acked Read 10 Done Done Sent false
23459 7569 Acked Read 4 Done Done Sent false
23460 7570 Acked Write 6 Done Done Sent false
23461 7571 Acked Write 7 Done Done Sent false
23462 7572 Acked Write 5 Done Done Sent false
23463 7573 Acked Read 7 Done Done Sent false
23464 7574 Acked Read 4 Done Done Sent false
23465 7575 Acked Write 9 Done Done Sent false
23466 7576 Acked Read 10 Done Done Sent false
23467 7577 Acked Read 2 Done Done Sent false
23468 7578 Acked Write 3 Done Done Sent false
23469 7579 Acked Write 4 Done Done Sent false
23470 7580 Acked Read 2 Done Done Sent false
23471 7581 Acked Read 4 Done Done Sent false
23472 7582 Acked Read 8 Done Done Sent false
23473 7583 Acked Write 6 Done Done Sent false
23474 7584 Acked Read 2 Done Done Sent false
23475 7585 Acked Read 5 Done Done Sent false
23476 7586 Acked Write 7 Done Done Sent false
23477 7587 Acked Read 9 Done Done Sent false
23478 7588 Acked Read 9 Done Done Sent false
23479 7589 Acked Write 7 Done Done Sent false
23480 7590 Acked Write 10 Done Done Sent false
23481 7591 Acked Read 1 Done Done Sent false
23482 7592 Acked Read 10 Done Done Sent false
23483 7593 Acked Write 2 Done Done Sent false
23484 7594 Acked Flush 0 Done Done Sent false
23485 7595 Acked Write 2 Done Done Sent false
23486 7596 Acked Write 5 Done Done Sent false
23487 7597 Acked Read 8 Done Done Sent false
23488 7598 Acked Write 3 Done Done Sent false
23489 7599 Acked Write 5 Done Done Sent false
23490 7600 Acked Read 5 Done Done Sent false
23491 7601 Acked Flush 0 Done Done Sent false
23492 7602 Acked Write 2 Done Done Sent false
23493 7603 Acked Write 5 Done Done Sent false
23494 7604 Acked Read 8 Done Done Sent false
23495 7605 Acked Read 4 Done Done Sent false
23496 7606 Acked Write 6 Done Done Sent false
23497 7607 Acked Read 1 Done Done Sent false
23498 7608 Acked Flush 0 Done Done Sent false
23499 7609 Acked Write 3 Done Done Sent false
23500 7610 Acked Read 8 Done Done Sent false
23501 7611 Acked Write 2 Done Done Sent false
23502 7612 Acked Read 8 Done Done Sent false
23503 7613 Acked Write 3 Done Done Sent false
23504 7614 Acked Flush 0 Done Done Sent false
23505 7615 Acked Read 5 Done Done Sent false
23506 7616 Acked Write 1 Done Done Sent false
23507 7617 Acked Read 4 Done Done Sent false
23508 7618 Acked Read 8 Done Done Sent false
23509 7619 Acked Read 6 Done Done Sent false
23510 7620 Acked Read 10 Done Done Sent false
23511 7621 Acked Write 3 Done Done Sent false
23512 7622 Acked Read 1 Done Done Sent false
23513 7623 Acked Read 8 Done Done Sent false
23514 7624 Acked Write 1 Done Done Sent false
23515 7625 Acked Write 8 Done Done Sent false
23516 7626 Acked Read 3 Done Done Sent false
23517 7627 Acked Read 2 Done Done Sent false
23518 7628 Acked Read 3 Done Done Sent false
23519 7629 Acked Write 8 Done Done Sent false
23520 7630 Acked Read 9 Done Done Sent false
23521 7631 Acked Read 4 Done Done Sent false
23522 7632 Acked Read 9 Done Done Sent false
23523 7633 Acked Read 1 Done Done Sent false
23524 7634 Acked Read 2 Done Done Sent false
23525 7635 Acked Read 4 Done Done Sent false
23526 7636 Acked Read 8 Done Done Sent false
23527 7637 Acked Write 1 Done Done Sent false
23528 7638 Acked Write 10 Done Done Sent false
23529 7639 Acked Read 2 Done Done Sent false
23530 7640 Acked Flush 0 Done Done Sent false
23531 STATES DS:0 DS:1 DS:2 TOTAL
23532 Sent 0 0 302 302
23533 Done 302 302 0 604
23534 Skipped 0 0 0 0
23535 Error 0 0 0 0
23536 Last Flush: 7640 7640 7338
23537 Downstairs last five completed: 7338 7337 7336 7335 7334
23538 Upstairs last five completed: 7640 7639 7638 7637 7636
23539 CLIENT: Up:0 ds:302 act:2
235402024-12-04T13:35:37.294ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235412024-12-04T13:35:37.294ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40850) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235422024-12-04T13:35:37.295ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235432024-12-04T13:35:37.296ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235442024-12-04T13:35:37.296ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235452024-12-04T13:35:37.296ZINFOcrucible: send last flush ID to this DS: 7338 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235462024-12-04T13:35:37.296ZINFOcrucible: Replied this last flush ID: 7338 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235472024-12-04T13:35:37.296ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235482024-12-04T13:35:37.296ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235492024-12-04T13:35:37.296ZINFOcrucible: [2] client re-new 302 jobs since flush 7338 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235502024-12-04T13:35:37.297ZINFOcrucible: [2] Replayed 302 jobs since flush: 7338 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
23551 ----------------------------------------------------------------
23552 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23553 Downstairs last five completed: 7640 7639 7638 7637 7636
23554 Upstairs last five completed: 7640 7639 7638 7637 7636
23555 CLIENT: Up:0 ds:0 act:3
23556 Replay: All jobs finished
23557 Main task has work to do, go find it
23558 got dsc Stop(1)
23559 stop 1
23560 [8820] Got stop action so:false kr:false
23561 Dec 04 13:35:39.534 INFO request completed, latency_us: 330, response_code: 204, uri: /stop/cid/1, method: POST, req_id: d3eec682-d795-41ba-a08a-cd6264ee9d1f, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
23562 Dec 04 13:35:39.535 INFO request completed, latency_us: 109, response_code: 200, uri: /state/cid/1, method: GET, req_id: 8827ea12-4b06-4b66-9c67-28be22914891, remote_addr: 127.0.0.1:37804, local_addr: 127.0{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:35:39.535433237Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
23563 .0.1:9998
235642024-12-04T13:35:39.535ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235652024-12-04T13:35:39.535ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235662024-12-04T13:35:39.535ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235672024-12-04T13:35:39.535ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
235682024-12-04T13:35:39.535ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
23569 [8820] Exited with: ExitStatus(unix_wait_status(9))
23570 [8820][1] reports Exit
23571 Dec 04 13:35:43.537 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/1, method: GET, req_id: eaf458d4-a3f9-4a6b-ac3d-2c4eba483722, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
23572 001/300 Read block 2548 len 24576
23573 002/300 Read block 2707 len 4096
23574 003/300 Write block 1454 len 32768 data: 6 8 8 10 10 8 5 5
23575 004/300 Read block 248 len 32768
23576 005/300 Read block 4353 len 36864
23577 006/300 Write block 4203 len 40960 data: 3 3 3 3 3 3 3 4 4 3
23578 007/300 Flush
23579 008/300 Write block 849 len 28672 data: 5 4 5 6 4 4 4
23580 009/300 Read block 1416 len 32768
23581 010/300 Read block 4035 len 32768
23582 011/300 Read block 1090 len 20480
23583 012/300 Flush
23584 013/300 Write block 4362 len 24576 data: 7 6 5 5 5 3
23585 014/300 Read block 1964 len 8192
23586 015/300 Flush
23587 016/300 Read block 3167 len 16384
23588 017/300 Write block 4023 len 4096 data: 5
23589 018/300 Write block 643 len 20480 data: 5 6 6 6 6
23590 019/300 Read block 1475 len 20480
23591 020/300 Write block 2564 len 32768 data: 4 3 4 5 6 6 8 8
23592 021/300 Read block 1242 len 8192
23593 022/300 Write block 2724 len 32768 data: 4 4 3 4 5 5 4 4
23594 023/300 Read block 1139 len 32768
23595 024/300 Flush
23596 025/300 Write block 741 len 28672 data: 4 3 3 3 4 4 4
23597 026/300 Write block 4319 len 32768 data: 2 2 2 2 2 2 3 5
23598 027/300 Read block 2309 len 24576
23599 028/300 Write block 2338 len 24576 data: 4 3 4 3 3 5
23600 029/300 Write block 2393 len 24576 data: 5 5 6 6 5 5
23601 030/300 Write block 2548 len 28672 data: 4 4 3 3 3 2 2
23602 031/300 Read block 2733 len 32768
23603 032/300 Write block 4928 len 28672 data: 3 3 4 4 3 3 3
23604 033/300 Write block 4080 len 20480 data: 4 4 4 4 3
23605 034/300 Read block 3574 len 20480
23606 035/300 Write block 173 len 28672 data: 6 6 6 5 4 4 3
23607 036/300 Read block 2560 len 32768
23608 037/300 Flush
23609 038/300 Read block 3753 len 12288
23610 039/300 Write block 4035 len 12288 data: 6 4 3
23611 040/300 Read block 2459 len 16384
23612 041/300 Read block 4074 len 32768
23613 042/300 Read block 4365 len 16384
23614 043/300 Write block 1924 len 24576 data: 4 4 3 3 4 5
23615 044/300 Read block 1964 len 16384
23616 045/300 Read block 1258 len 36864
23617 046/300 Read block 3067 len 12288
23618 047/300 Write block 1814 len 16384 data: 4 6 5 5
23619 048/300 Write block 4624 len 16384 data: 5 5 4 4
23620 049/300 Write block 3764 len 4096 data: 7
23621 050/300 Read block 2179 len 28672
23622 051/300 Write block 2903 len 8192 data: 4 3
23623 052/300 Read block 772 len 28672
23624 053/300 Write block 1281 len 24576 data: 5 5 5 6 5 5
23625 054/300 Read block 4903 len 4096
23626 055/300 Read block 4216 len 28672
23627 056/300 Write block 1738 len 12288 data: 3 3 3
23628 057/300 Read block 1266 len 16384
23629 058/300 Flush
23630 059/300 Read block 4916 len 28672
23631 060/300 Read block 1895 len 4096
23632 061/300 Read block 2640 len 40960
23633 062/300 Write block 4221 len 40960 data: 6 6 5 4 4 3 4 5 6 5
23634 063/300 Read block 757 len 36864
23635 064/300 Read block 1198 len 28672
23636 065/300 Flush
23637 066/300 Write block 3409 len 28672 data: 6 4 4 4 3 3 4
23638 067/300 Read block 4463 len 20480
23639 068/300 Read block 3962 len 16384
23640 069/300 Write block 4533 len 32768 data: 4 4 4 6 6 5 5 4
23641 070/300 Write block 4721 len 28672 data: 4 5 4 4 4 3 3
23642 071/300 Write block 3908 len 36864 data: 3 4 4 4 3 3 3 4 3
23643 072/300 Read block 502 len 40960
23644 073/300 Read block 382 len 20480
23645 074/300 Read block 1295 len 12288
23646 075/300 Write block 4071 len 20480 data: 3 3 3 2 4
23647 076/300 Flush
23648 077/300 Write block 4986 len 8192 data: 4 4
23649 078/300 Flush
23650 079/300 Write block 1970 len 20480 data: 4 4 4 3 4
23651 080/300 Read block 4606 len 12288
23652 081/300 Flush
23653 082/300 Write block 117 len 20480 data: 5 6 4 5 4
23654 083/300 Write block 660 len 40960 data: 3 3 5 5 5 5 4 4 3 5
23655 084/300 Flush
23656 085/300 Read block 413 len 32768
23657 086/300 Read block 518 len 4096
23658 087/300 Write block 4605 len 4096 data: 8
23659 088/300 Write block 4159 len 8192 data: 12 12
23660 089/300 Read block 3926 len 40960
23661 090/300 Write block 2332 len 8192 data: 2 3
23662 091/300 Write block 2822 len 4096 data: 5
23663 092/300 Read block 2387 len 12288
23664 093/300 Flush
23665 094/300 Read block 839 len 20480
23666 095/300 Write block 2890 len 16384 data: 6 5 5 5
23667 096/300 Flush
23668 097/300 Write block 1696 len 8192 data: 8 5
23669 098/300 Read block 922 len 24576
23670 099/300 Write block 3046 len 8192 data: 4 4
23671 100/300 Write block 752 len 28672 data: 4 5 7 6 5 5 5
23672 101/300 Read block 4964 len 8192
23673 102/300 Write block 1634 len 24576 data: 6 5 5 4 3 4
23674 103/300 Write block 1104 len 40960 data: 6 6 6 4 3 4 4 5 7 7
23675 104/300 Read block 997 len 16384
23676 105/300 Write block 2837 len 8192 data: 4 7
23677 106/300 Read block 1171 len 4096
23678 107/300 Write block 1366 len 24576 data: 4 4 4 5 5 6
23679 108/300 Write block 3235 len 24576 data: 7 8 8 7 5 5
23680 109/300 Write block 119 len 32768 data: 5 6 5 5 6 5 5 6
23681 110/300 Read block 2116 len 12288
23682 111/300 Read block 279 len 40960
23683 112/300 Read block 3394 len 32768
23684 113/300 Write block 3769 len 40960 data: 6 6 6 5 6 6 7 7 6 6
23685 114/300 Write block 2363 len 24576 data: 5 6 6 5 5 7
23686 115/300 Write block 1637 len 8192 data: 5 4
23687 116/300 Write block 152 len 32768 data: 5 6 6 6 6 5 6 6
23688 117/300 Read block 1418 len 40960
23689 118/300 Write block 2718 len 16384 data: 2 3 4 4
23690 119/300 Read block 1049 len 4096
23691 120/300 Flush
23692 121/300 Write block 3176 len 32768 data: 6 5 4 5 4 4 4 3
23693 122/300 Write block 2964 len 28672 data: 4 4 3 3 4 4 4
23694 123/300 Flush
23695 124/300 Read block 4215 len 28672
23696 125/300 Read block 3194 len 24576
23697 126/300 Read block 4591 len 8192
23698 127/300 Write block 4074 len 24576 data: 3 5 5 5 4 4
23699 128/300 Read block 263 len 40960
23700 129/300 Read block 464 len 12288
23701 130/300 Flush
23702 131/300 Write block 3969 len 36864 data: 5 5 5 5 5 6 6 3 4
23703 132/300 Read block 1486 len 20480
23704 133/300 Write block 4874 len 4096 data: 4
23705 134/300 Write block 1923 len 16384 data: 4 5 5 4
23706 135/300 Read block 2886 len 36864
23707 136/300 Write block 1488 len 36864 data: 5 4 4 4 4 5 4 4 4
23708 137/300 Write block 12 len 20480 data: 8 7 7 7 11
23709 138/300 Read block 3897 len 32768
23710 139/300 Read block 4847 len 24576
23711 140/300 Read block 693 len 16384
23712 141/300 Read block 4150 len 24576
23713 142/300 Read block 1310 len 28672
23714 143/300 Read block 2216 len 24576
23715 144/300 Write block 2824 len 20480 data: 4 5 7 7 6
23716 145/300 Read block 2760 len 12288
23717 146/300 Write block 2315 len 4096 data: 6
23718 147/300 Read block 3197 len 8192
23719 148/300 Read block 1129 len 40960
23720 149/300 Write block 4688 len 8192 data: 5 5
23721 150/300 Read block 674 len 36864
23722 151/300 Write block 615 len 16384 data: 3 3 3 4
23723 152/300 Read block 2635 len 28672
23724 153/300 Read block 3502 len 40960
23725 154/300 Read block 2408 len 32768
23726 155/300 Read block 2882 len 28672
23727 156/300 Write block 4438 len 24576 data: 4 5 5 5 6 5
23728 157/300 Read block 4192 len 4096
23729 158/300 Read block 3151 len 8192
23730 159/300 Read block 2211 len 16384
23731 160/300 Write block 2055 len 4096 data: 5
23732 161/300 Read block 4982 len 16384
23733 162/300 Write block 208 len 32768 data: 6 3 3 3 3 2 3 3
23734 163/300 Read block 4435 len 4096
23735 164/300 Write block 1847 len 28672 data: 7 5 6 6 7 9 7
23736 165/300 Read block 4120 len 36864
23737 166/300 Write block 3918 len 40960 data: 3 3 3 4 4 3 3 4 5 5
23738 167/300 Write block 3255 len 4096 data: 5
23739 168/300 Read block 1786 len 8192
23740 169/300 Read block 3969 len 16384
23741 170/300 Read block 1324 len 8192
23742 171/300 Write block 2789 len 16384 data: 6 6 7 6
23743 172/300 Read block 3562 len 24576
23744 173/300 Read block 1530 len 20480
23745 174/300 Read block 3610 len 36864
23746 175/300 Read block 353 len 4096
23747 176/300 Write block 4387 len 28672 data: 5 5 4 4 5 5 6
23748 177/300 Read block 1956 len 40960
23749 178/300 Read block 3612 len 20480
23750 179/300 Write block 668 len 20480 data: 4 6 6 4 4
23751 180/300 Write block 4370 len 40960 data: 4 4 3 2 3 3 4 4 5 4
23752 181/300 Read block 2556 len 8192
23753 182/300 Read block 3784 len 16384
23754 183/300 Write block 2247 len 20480 data: 9 6 6 4 4
23755 184/300 Read block 1390 len 32768
23756 185/300 Write block 550 len 4096 data: 7
23757 186/300 Read block 3141 len 16384
23758 187/300 Write block 533 len 32768 data: 3 5 4 4 5 4 6 6
23759 188/300 Write block 3402 len 4096 data: 13
23760 189/300 Read block 2770 len 24576
23761 190/300 Read block 4094 len 8192
23762 191/300 Read block 4317 len 16384
23763 192/300 Read block 224 len 24576
23764 193/300 Read block 2621 len 32768
23765 194/300 Read block 4753 len 20480
23766 195/300 Write block 3106 len 36864 data: 6 6 7 8 6 5 4 4 3
23767 196/300 Write block 4567 len 32768 data: 4 4 4 4 3 4 6 4
23768 197/300 Read block 1459 len 4096
23769 198/300 Read block 2520 len 8192
23770 199/300 Write block 2286 len 36864 data: 4 5 4 5 7 8 9 9 9
23771 200/300 Write block 3549 len 28672 data: 7 6 8 8 7 6 4
23772 201/300 Write block 3698 len 16384 data: 6 6 5 6
23773 202/300 Read block 3827 len 8192
23774 203/300 Write block 1080 len 28672 data: 6 6 7 6 4 4 4
23775 204/300 Read block 2232 len 12288
23776 205/300 Write block 4542 len 12288 data: 4 5 5
23777 206/300 Read block 3148 len 40960
23778 207/300 Read block 4379 len 12288
23779 208/300 Flush
23780 209/300 Read block 2384 len 12288
23781 210/300 Write block 28 len 40960 data: 4 4 3 4 4 4 4 4 5 5
23782 211/300 Read block 1645 len 40960
23783 212/300 Read block 4180 len 32768
23784 213/300 Write block 1425 len 4096 data: 9
23785 214/300 Read block 3043 len 28672
23786 215/300 Read block 40 len 20480
23787 216/300 Read block 4959 len 32768
23788 217/300 Write block 2530 len 36864 data: 2 2 2 2 2 3 5 8 7
23789 218/300 Read block 4195 len 20480
23790 219/300 Read block 2253 len 16384
23791 220/300 Write block 581 len 36864 data: 3 5 5 3 3 2 3 3 4
23792 221/300 Read block 4258 len 32768
23793 222/300 Read block 1363 len 12288
23794 223/300 Write block 3599 len 32768 data: 5 5 5 5 5 4 4 3
23795 224/300 Read block 1722 len 16384
23796 225/300 Read block 279 len 8192
23797 226/300 Read block 3220 len 20480
23798 227/300 Write block 346 len 24576 data: 5 4 4 4 4 3
23799 228/300 Read block 1472 len 40960
23800 229/300 Read block 2269 len 40960
23801 230/300 Write block 1033 len 40960 data: 4 4 3 4 3 3 3 3 3 3
23802 231/300 Read block 1145 len 20480
23803 232/300 Read block 1220 len 16384
23804 233/300 Read block 4232 len 36864
23805 234/300 Read block 3808 len 28672
23806 235/300 Read block 2861 len 32768
23807 236/300 Read block 63 len 32768
23808 237/300 Write block 2901 len 32768 data: 5 4 5 4 4 4 4 3
23809 238/300 Write block 1886 len 4096 data: 5
23810 239/300 Write block 442 len 20480 data: 9 10 6 7 7
23811 240/300 Read block 1469 len 8192
23812 241/300 Read block 807 len 20480
23813 242/300 Write block 2828 len 16384 data: 7 7 6 6
23814 243/300 Read block 4207 len 4096
23815 244/300 Read block 1600 len 28672
23816 245/300 Read block 1239 len 28672
23817 246/300 Write block 294 len 12288 data: 6 7 8
23818 247/300 Read block 2577 len 16384
23819 248/300 Write block 1450 len 4096 data: 3
23820 249/300 Read block 942 len 32768
23821 250/300 Read block 2376 len 8192
23822 251/300 Read block 2305 len 4096
23823 252/300 Read block 1018 len 20480
23824 253/300 Read block 4619 len 20480
23825 254/300 Read block 675 len 12288
23826 255/300 Read block 872 len 24576
23827 256/300 Write block 2962 len 24576 data: 3 3 5 5 4 4
23828 257/300 Read block 483 len 8192
23829 258/300 Write block 1619 len 28672 data: 4 3 3 4 3 3 3
23830 259/300 Write block 1634 len 4096 data: 7
23831 260/300 Write block 3794 len 16384 data: 6 4 3 2
23832 261/300 Read block 2800 len 12288
23833 262/300 Write block 2814 len 32768 data: 2 2 2 3 3 3 4 5
23834 263/300 Flush
23835 264/300 Write block 1610 len 12288 data: 5 6 7
23836 265/300 Write block 4073 len 24576 data: 4 4 6 6 6 5
23837 266/300 Read block 1516 len 28672
23838 267/300 Write block 2167 len 16384 data: 7 7 6 7
23839 268/300 Write block 1867 len 4096 data: 5
23840 269/300 Read block 662 len 24576
23841 270/300 Read block 1985 len 8192
23842 271/300 Write block 1514 len 12288 data: 9 9 9
23843 272/300 Write block 3901 len 16384 data: 5 5 6 5
23844 273/300 Read block 1908 len 40960
23845 274/300 Read block 1689 len 8192
23846 275/300 Write block 1426 len 24576 data: 8 7 6 4 4 5
23847 276/300 Write block 4342 len 12288 data: 5 6 6
23848 277/300 Read block 1203 len 32768
23849 278/300 Read block 3667 len 24576
23850 279/300 Read block 4130 len 12288
23851 280/300 Write block 313 len 12288 data: 5 4 5
23852 281/300 Write block 3854 len 16384 data: 2 2 3 3
23853 282/300 Write block 4817 len 28672 data: 3 3 4 3 3 3 3
23854 283/300 Read block 2735 len 12288
23855 284/300 Write block 2100 len 24576 data: 2 3 4 4 4 4
23856 285/300 Write block 122 len 28672 data: 6 7 6 6 7 5 4
23857 286/300 Read block 793 len 16384
23858 287/300 Read block 3568 len 12288
23859 288/300 Read block 474 len 20480
23860 289/300 Read block 3360 len 28672
23861 290/300 Write block 4941 len 12288 data: 3 4 4
23862 291/300 Flush
23863 292/300 Write block 4063 len 28672 data: 5 6 6 5 4 4 2
23864 293/300 Read block 1849 len 24576
23865 294/300 Read block 4403 len 12288
23866 295/300 Write block 1818 len 28672 data: 5 4 4 4 3 3 3
23867 296/300 Read block 2963 len 20480
23868 297/300 Read block 2166 len 24576
23869 298/300 Read block 2832 len 8192
23870 299/300 Write block 2144 len 12288 data: 6 4 6
23871 300/300 Read block 2263 len 4096
23872 301/300 Read block 325 len 32768
23873 Main task has work to do, go find it
23874 got dsc Start(1)
23875 start 1
23876 Dec 04 13:35:43.775 INFO request completed, latency_us: 196, response_code: 204, uri: /start/cid/1, method: POST, req_id: 0d371f5d-3751-42ca-891e-e41641a7102c, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
23877 Replay: started 1, returned:Ok(())
23878 ----------------------------------------------------------------
23879 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23880 States: Active Offline Active
23881 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23882 7641 Acked Read 6 Done Sent Done false
23883 7642 Acked Read 1 Done Sent Done false
23884 7643 Acked Write 8 Done Sent Done false
23885 7644 Acked Read 8 Done Sent Done false
23886 7645 Acked Read 9 Done Sent Done false
23887 7646 Acked Write 10 Done Sent Done false
23888 7647 Acked Flush 0 Done Sent Done false
23889 7648 Acked Write 7 Done Sent Done false
23890 7649 Acked Read 8 Done Sent Done false
23891 7650 Acked Read 8 Done Sent Done false
23892 7651 Acked Read 5 Done Sent Done false
23893 7652 Acked Flush 0 Done Sent Done false
23894 7653 Acked Write 6 Done Sent Done false
23895 7654 Acked Read 2 Done Sent Done false
23896 7655 Acked Flush 0 Done Sent Done false
23897 7656 Acked Read 4 Done Sent Done false
23898 7657 Acked Write 1 Done Sent Done false
23899 7658 Acked Write 5 Done Sent Done false
23900 7659 Acked Read 5 Done Sent Done false
23901 7660 Acked Write 8 Done Sent Done false
23902 7661 Acked Read 2 Done Sent Done false
23903 7662 Acked Write 8 Done Sent Done false
23904 7663 Acked Read 8 Done Sent Done false
23905 7664 Acked Flush 0 Done Sent Done false
23906 7665 Acked Write 7 Done Sent Done false
23907 7666 Acked Write 8 Done Sent Done false
23908 7667 Acked Read 6 Done Sent Done false
23909 7668 Acked Write 6 Done Sent Done false
23910 7669 Acked Write 6 Done Sent Done false
23911 7670 Acked Write 7 Done Sent Done false
23912 7671 Acked Read 8 Done Sent Done false
23913 7672 Acked Write 7 Done Sent Done false
23914 7673 Acked Write 5 Done Sent Done false
23915 7674 Acked Read 5 Done Sent Done false
23916 7675 Acked Write 7 Done Sent Done false
23917 7676 Acked Read 8 Done Sent Done false
23918 7677 Acked Flush 0 Done Sent Done false
23919 7678 Acked Read 3 Done Sent Done false
23920 7679 Acked Write 3 Done Sent Done false
23921 7680 Acked Read 4 Done Sent Done false
23922 7681 Acked Read 8 Done Sent Done false
23923 7682 Acked Read 4 Done Sent Done false
23924 7683 Acked Write 6 Done Sent Done false
23925 7684 Acked Read 4 Done Sent Done false
23926 7685 Acked Read 9 Done Sent Done false
23927 7686 Acked Read 3 Done Sent Done false
23928 7687 Acked Write 4 Done Sent Done false
23929 7688 Acked Write 4 Done Sent Done false
23930 7689 Acked Write 1 Done Sent Done false
23931 7690 Acked Read 7 Done Sent Done false
23932 7691 Acked Write 2 Done Sent Done false
23933 7692 Acked Read 7 Done Sent Done false
23934 7693 Acked Write 6 Done Sent Done false
23935 7694 Acked Read 1 Done Sent Done false
23936 7695 Acked Read 7 Done Sent Done false
23937 7696 Acked Write 3 Done Sent Done false
23938 7697 Acked Read 4 Done Sent Done false
23939 7698 Acked Flush 0 Done Sent Done false
23940 7699 Acked Read 7 Done Sent Done false
23941 7700 Acked Read 1 Done Sent Done false
23942 7701 Acked Read 10 Done Sent Done false
23943 7702 Acked Write 10 Done Sent Done false
23944 7703 Acked Read 9 Done Sent Done false
23945 7704 Acked Read 7 Done Sent Done false
23946 7705 Acked Flush 0 Done Sent Done false
23947 7706 Acked Write 7 Done Sent Done false
23948 7707 Acked Read 5 Done Sent Done false
23949 7708 Acked Read 4 Done Sent Done false
23950 7709 Acked Write 8 Done Sent Done false
23951 7710 Acked Write 7 Done Sent Done false
23952 7711 Acked Write 9 Done Sent Done false
23953 7712 Acked Read 10 Done Sent Done false
23954 7713 Acked Read 5 Done Sent Done false
23955 7714 Acked Read 3 Done Sent Done false
23956 7715 Acked Write 5 Done Sent Done false
23957 7716 Acked Flush 0 Done Sent Done false
23958 7717 Acked Write 2 Done Sent Done false
23959 7718 Acked Flush 0 Done Sent Done false
23960 7719 Acked Write 5 Done Sent Done false
23961 7720 Acked Read 3 Done Sent Done false
23962 7721 Acked Flush 0 Done Sent Done false
23963 7722 Acked Write 5 Done Sent Done false
23964 7723 Acked Write 10 Done Sent Done false
23965 7724 Acked Flush 0 Done Sent Done false
23966 7725 Acked Read 8 Done Sent Done false
23967 7726 Acked Read 1 Done Sent Done false
23968 7727 Acked Write 1 Done Sent Done false
23969 7728 Acked Write 2 Done Sent Done false
23970 7729 Acked Read 10 Done Sent Done false
23971 7730 Acked Write 2 Done Sent Done false
23972 7731 Acked Write 1 Done Sent Done false
23973 7732 Acked Read 3 Done Sent Done false
23974 7733 Acked Flush 0 Done Sent Done false
23975 7734 Acked Read 5 Done Sent Done false
23976 7735 Acked Write 4 Done Sent Done false
23977 7736 Acked Flush 0 Done Sent Done false
23978 7737 Acked Write 2 Done Sent Done false
23979 7738 Acked Read 6 Done Sent Done false
23980 7739 Acked Write 2 Done Sent Done false
23981 7740 Acked Write 7 Done Sent Done false
23982 7741 Acked Read 2 Done Sent Done false
23983 7742 Acked Write 6 Done Sent Done false
23984 7743 Acked Write 10 Done Sent Done false
23985 7744 Acked Read 4 Done Sent Done false
23986 7745 Acked Write 2 Done Sent Done false
23987 7746 Acked Read 1 Done Sent Done false
23988 7747 Acked Write 6 Done Sent Done false
23989 7748 Acked Write 6 Done Sent Done false
23990 7749 Acked Write 8 Done Sent Done false
23991 7750 Acked Read 3 Done Sent Done false
23992 7751 Acked Read 10 Done Sent Done false
23993 7752 Acked Read 8 Done Sent Done false
23994 7753 Acked Write 10 Done Sent Done false
23995 7754 Acked Write 6 Done Sent Done false
23996 7755 Acked Write 2 Done Sent Done false
23997 7756 Acked Write 8 Done Sent Done false
23998 7757 Acked Read 10 Done Sent Done false
23999 7758 Acked Write 4 Done Sent Done false
24000 7759 Acked Read 1 Done Sent Done false
24001 7760 Acked Flush 0 Done Sent Done false
24002 7761 Acked Write 8 Done Sent Done false
24003 7762 Acked Write 7 Done Sent Done false
24004 7763 Acked Flush 0 Done Sent Done false
24005 7764 Acked Read 7 Done Sent Done false
24006 7765 Acked Read 6 Done Sent Done false
24007 7766 Acked Read 2 Done Sent Done false
24008 7767 Acked Write 6 Done Sent Done false
24009 7768 Acked Read 10 Done Sent Done false
24010 7769 Acked Read 3 Done Sent Done false
24011 7770 Acked Flush 0 Done Sent Done false
24012 7771 Acked Write 9 Done Sent Done false
24013 7772 Acked Read 5 Done Sent Done false
24014 7773 Acked Write 1 Done Sent Done false
24015 7774 Acked Write 4 Done Sent Done false
24016 7775 Acked Read 9 Done Sent Done false
24017 7776 Acked Write 9 Done Sent Done false
24018 7777 Acked Write 5 Done Sent Done false
24019 7778 Acked Read 8 Done Sent Done false
24020 7779 Acked Read 6 Done Sent Done false
24021 7780 Acked Read 4 Done Sent Done false
24022 7781 Acked Read 6 Done Sent Done false
24023 7782 Acked Read 7 Done Sent Done false
24024 7783 Acked Read 6 Done Sent Done false
24025 7784 Acked Write 5 Done Sent Done false
24026 7785 Acked Read 3 Done Sent Done false
24027 7786 Acked Write 1 Done Sent Done false
24028 7787 Acked Read 2 Done Sent Done false
24029 7788 Acked Read 10 Done Sent Done false
24030 7789 Acked Write 2 Done Sent Done false
24031 7790 Acked Read 9 Done Sent Done false
24032 7791 Acked Write 4 Done Sent Done false
24033 7792 Acked Read 7 Done Sent Done false
24034 7793 Acked Read 10 Done Sent Done false
24035 7794 Acked Read 8 Done Sent Done false
24036 7795 Acked Read 7 Done Sent Done false
24037 7796 Acked Write 6 Done Sent Done false
24038 7797 Acked Read 1 Done Sent Done false
24039 7798 Acked Read 2 Done Sent Done false
24040 7799 Acked Read 4 Done Sent Done false
24041 7800 Acked Write 1 Done Sent Done false
24042 7801 Acked Read 4 Done Sent Done false
24043 7802 Acked Write 8 Done Sent Done false
24044 7803 Acked Read 1 Done Sent Done false
24045 7804 Acked Write 7 Done Sent Done false
24046 7805 Acked Read 9 Done Sent Done false
24047 7806 Acked Write 10 Done Sent Done false
24048 7807 Acked Write 1 Done Sent Done false
24049 7808 Acked Read 2 Done Sent Done false
24050 7809 Acked Read 4 Done Sent Done false
24051 7810 Acked Read 2 Done Sent Done false
24052 7811 Acked Write 4 Done Sent Done false
24053 7812 Acked Read 6 Done Sent Done false
24054 7813 Acked Read 5 Done Sent Done false
24055 7814 Acked Read 9 Done Sent Done false
24056 7815 Acked Read 1 Done Sent Done false
24057 7816 Acked Write 7 Done Sent Done false
24058 7817 Acked Read 10 Done Sent Done false
24059 7818 Acked Read 5 Done Sent Done false
24060 7819 Acked Write 5 Done Sent Done false
24061 7820 Acked Write 10 Done Sent Done false
24062 7821 Acked Read 2 Done Sent Done false
24063 7822 Acked Read 4 Done Sent Done false
24064 7823 Acked Write 5 Done Sent Done false
24065 7824 Acked Read 8 Done Sent Done false
24066 7825 Acked Write 1 Done Sent Done false
24067 7826 Acked Read 4 Done Sent Done false
24068 7827 Acked Write 8 Done Sent Done false
24069 7828 Acked Write 1 Done Sent Done false
24070 7829 Acked Read 6 Done Sent Done false
24071 7830 Acked Read 2 Done Sent Done false
24072 7831 Acked Read 4 Done Sent Done false
24073 7832 Acked Read 6 Done Sent Done false
24074 7833 Acked Read 8 Done Sent Done false
24075 7834 Acked Read 5 Done Sent Done false
24076 7835 Acked Write 9 Done Sent Done false
24077 7836 Acked Write 8 Done Sent Done false
24078 7837 Acked Read 1 Done Sent Done false
24079 7838 Acked Read 2 Done Sent Done false
24080 7839 Acked Write 9 Done Sent Done false
24081 7840 Acked Write 7 Done Sent Done false
24082 7841 Acked Write 4 Done Sent Done false
24083 7842 Acked Read 2 Done Sent Done false
24084 7843 Acked Write 7 Done Sent Done false
24085 7844 Acked Read 3 Done Sent Done false
24086 7845 Acked Write 3 Done Sent Done false
24087 7846 Acked Read 10 Done Sent Done false
24088 7847 Acked Read 3 Done Sent Done false
24089 7848 Acked Flush 0 Done Sent Done false
24090 7849 Acked Read 3 Done Sent Done false
24091 7850 Acked Write 10 Done Sent Done false
24092 7851 Acked Read 10 Done Sent Done false
24093 7852 Acked Read 8 Done Sent Done false
24094 7853 Acked Write 1 Done Sent Done false
24095 7854 Acked Read 7 Done Sent Done false
24096 7855 Acked Read 5 Done Sent Done false
24097 7856 Acked Read 8 Done Sent Done false
24098 7857 Acked Write 9 Done Sent Done false
24099 7858 Acked Read 5 Done Sent Done false
24100 7859 Acked Read 4 Done Sent Done false
24101 7860 Acked Write 9 Done Sent Done false
24102 7861 Acked Read 8 Done Sent Done false
24103 7862 Acked Read 3 Done Sent Done false
24104 7863 Acked Write 8 Done Sent Done false
24105 7864 Acked Read 4 Done Sent Done false
24106 7865 Acked Read 2 Done Sent Done false
24107 7866 Acked Read 5 Done Sent Done false
24108 7867 Acked Write 6 Done Sent Done false
24109 7868 Acked Read 10 Done Sent Done false
24110 7869 Acked Read 10 Done Sent Done false
24111 7870 Acked Write 10 Done Sent Done false
24112 7871 Acked Read 5 Done Sent Done false
24113 7872 Acked Read 4 Done Sent Done false
24114 7873 Acked Read 9 Done Sent Done false
24115 7874 Acked Read 7 Done Sent Done false
24116 7875 Acked Read 8 Done Sent Done false
24117 7876 Acked Read 8 Done Sent Done false
24118 7877 Acked Write 8 Done Sent Done false
24119 7878 Acked Write 1 Done Sent Done false
24120 7879 Acked Write 5 Done Sent Done false
24121 7880 Acked Read 2 Done Sent Done false
24122 7881 Acked Read 5 Done Sent Done false
24123 7882 Acked Write 4 Done Sent Done false
24124 7883 Acked Read 1 Done Sent Done false
24125 7884 Acked Read 7 Done Sent Done false
24126 7885 Acked Read 7 Done Sent Done false
24127 7886 Acked Write 3 Done Sent Done false
24128 7887 Acked Read 4 Done Sent Done false
24129 7888 Acked Write 1 Done Sent Done false
24130 7889 Acked Read 8 Done Sent Done false
24131 7890 Acked Read 2 Done Sent Done false
24132 7891 Acked Read 1 Done Sent Done false
24133 7892 Acked Read 5 Done Sent Done false
24134 7893 Acked Read 5 Done Sent Done false
24135 7894 Acked Read 3 Done Sent Done false
24136 7895 Acked Read 6 Done Sent Done false
24137 7896 Acked Write 6 Done Sent Done false
24138 7897 Acked Read 2 Done Sent Done false
24139 7898 Acked Write 7 Done Sent Done false
24140 7899 Acked Write 1 Done Sent Done false
24141 7900 Acked Write 4 Done Sent Done false
24142 7901 Acked Read 3 Done Sent Done false
24143 7902 Acked Write 8 Done Sent Done false
24144 7903 Acked Flush 0 Done Sent Done false
24145 7904 Acked Write 3 Done Sent Done false
24146 7905 Acked Write 6 Done Sent Done false
24147 7906 Acked Read 7 Done Sent Done false
24148 7907 Acked Write 4 Done Sent Done false
24149 7908 Acked Write 1 Done Sent Done false
24150 7909 Acked Read 6 Done Sent Done false
24151 7910 Acked Read 2 Done Sent Done false
24152 7911 Acked Write 3 Done Sent Done false
24153 7912 Acked Write 4 Done Sent Done false
24154 7913 Acked Read 10 Done Sent Done false
24155 7914 Acked Read 2 Done Sent Done false
24156 7915 Acked Write 6 Done Sent Done false
24157 7916 Acked Write 3 Done Sent Done false
24158 7917 Acked Read 8 Done Sent Done false
24159 7918 Acked Read 6 Done Sent Done false
24160 7919 Acked Read 3 Done Sent Done false
24161 7920 Acked Write 3 Done Sent Done false
24162 7921 Acked Write 4 Done Sent Done false
24163 7922 Acked Write 7 Done Sent Done false
24164 7923 Acked Read 3 Done Sent Done false
24165 7924 Acked Write 6 Done Sent Done false
24166 7925 Acked Write 7 Done Sent Done false
24167 7926 Acked Read 4 Done Sent Done false
24168 7927 Acked Read 3 Done Sent Done false
24169 7928 Acked Read 5 Done Sent Done false
24170 7929 Acked Read 7 Done Sent Done false
24171 7930 Acked Write 3 Done Sent Done false
24172 7931 Acked Flush 0 Done Sent Done false
24173 7932 Acked Write 7 Done Sent Done false
24174 7933 Acked Read 6 Done Sent Done false
24175 7934 Acked Read 3 Done Sent Done false
24176 7935 Acked Write 7 Done Sent Done false
24177 7936 Acked Read 5 Done Sent Done false
24178 7937 Acked Read 6 Done Sent Done false
24179 7938 Acked Read 2 Done Sent Done false
24180 7939 Acked Write 3 Done Sent Done false
24181 7940 Acked Read 1 Done Sent Done false
24182 7941 Acked Read 8 Done Sent Done false
24183 STATES DS:0 DS:1 DS:2 TOTAL
24184 Sent 0 301 0 301
24185 Done 301 0 301 602
24186 Skipped 0 0 0 0
24187 Error 0 0 0 0
24188 Last Flush: 7931 7640 7931
24189 Downstairs last five completed: 7640 7639 7638 7637 7636
24190 Upstairs last five completed: 7941 7940 7939 7938 7937
24191 CLIENT: Up:0 ds:301 act:2
24192 ----------------------------------------------------------------
24193 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24194 States: Active Offline Active
24195 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24196 7641 Acked Read 6 Done Sent Done false
24197 7642 Acked Read 1 Done Sent Done false
24198 7643 Acked Write 8 Done Sent Done false
24199 7644 Acked Read 8 Done Sent Done false
24200 7645 Acked Read 9 Done Sent Done false
24201 7646 Acked Write 10 Done Sent Done false
24202 7647 Acked Flush 0 Done Sent Done false
24203 7648 Acked Write 7 Done Sent Done false
24204 7649 Acked Read 8 Done Sent Done false
24205 7650 Acked Read 8 Done Sent Done false
24206 7651 Acked Read 5 Done Sent Done false
24207 7652 Acked Flush 0 Done Sent Done false
24208 7653 Acked Write 6 Done Sent Done false
24209 7654 Acked Read 2 Done Sent Done false
24210 7655 Acked Flush 0 Done Sent Done false
24211 7656 Acked Read 4 Done Sent Done false
24212 7657 Acked Write 1 Done Sent Done false
24213 7658 Acked Write 5 Done Sent Done false
24214 7659 Acked Read 5 Done Sent Done false
24215 7660 Acked Write 8 Done Sent Done false
24216 7661 Acked Read 2 Done Sent Done false
24217 7662 Acked Write 8 Done Sent Done false
24218 7663 Acked Read 8 Done Sent Done false
24219 7664 Acked Flush 0 Done Sent Done false
24220 7665 Acked Write 7 Done Sent Done false
24221 7666 Acked Write 8 Done Sent Done false
24222 7667 Acked Read 6 Done Sent Done false
24223 7668 Acked Write 6 Done Sent Done false
24224 7669 Acked Write 6 Done Sent Done false
24225 7670 Acked Write 7 Done Sent Done false
24226 7671 Acked Read 8 Done Sent Done false
24227 7672 Acked Write 7 Done Sent Done false
24228 7673 Acked Write 5 Done Sent Done false
24229 7674 Acked Read 5 Done Sent Done false
24230 7675 Acked Write 7 Done Sent Done false
24231 7676 Acked Read 8 Done Sent Done false
24232 7677 Acked Flush 0 Done Sent Done false
24233 7678 Acked Read 3 Done Sent Done false
24234 7679 Acked Write 3 Done Sent Done false
24235 7680 Acked Read 4 Done Sent Done false
24236 7681 Acked Read 8 Done Sent Done false
24237 7682 Acked Read 4 Done Sent Done false
24238 7683 Acked Write 6 Done Sent Done false
24239 7684 Acked Read 4 Done Sent Done false
24240 7685 Acked Read 9 Done Sent Done false
24241 7686 Acked Read 3 Done Sent Done false
24242 7687 Acked Write 4 Done Sent Done false
24243 7688 Acked Write 4 Done Sent Done false
24244 7689 Acked Write 1 Done Sent Done false
24245 7690 Acked Read 7 Done Sent Done false
24246 7691 Acked Write 2 Done Sent Done false
24247 7692 Acked Read 7 Done Sent Done false
24248 7693 Acked Write 6 Done Sent Done false
24249 7694 Acked Read 1 Done Sent Done false
24250 7695 Acked Read 7 Done Sent Done false
24251 7696 Acked Write 3 Done Sent Done false
24252 7697 Acked Read 4 Done Sent Done false
24253 7698 Acked Flush 0 Done Sent Done false
24254 7699 Acked Read 7 Done Sent Done false
24255 7700 Acked Read 1 Done Sent Done false
24256 7701 Acked Read 10 Done Sent Done false
24257 7702 Acked Write 10 Done Sent Done false
24258 7703 Acked Read 9 Done Sent Done false
24259 7704 Acked Read 7 Done Sent Done false
24260 7705 Acked Flush 0 Done Sent Done false
24261 7706 Acked Write 7 Done Sent Done false
24262 7707 Acked Read 5 Done Sent Done false
24263 7708 Acked Read 4 Done Sent Done false
24264 7709 Acked Write 8 Done Sent Done false
24265 7710 Acked Write 7 Done Sent Done false
24266 7711 Acked Write 9 Done Sent Done false
24267 7712 Acked Read 10 Done Sent Done false
24268 7713 Acked Read 5 Done Sent Done false
24269 7714 Acked Read 3 Done Sent Done false
24270 7715 Acked Write 5 Done Sent Done false
24271 7716 Acked Flush 0 Done Sent Done false
24272 7717 Acked Write 2 Done Sent Done false
24273 7718 Acked Flush 0 Done Sent Done false
24274 7719 Acked Write 5 Done Sent Done false
24275 7720 Acked Read 3 Done Sent Done false
24276 7721 Acked Flush 0 Done Sent Done false
24277 7722 Acked Write 5 Done Sent Done false
24278 7723 Acked Write 10 Done Sent Done false
24279 7724 Acked Flush 0 Done Sent Done false
24280 7725 Acked Read 8 Done Sent Done false
24281 7726 Acked Read 1 Done Sent Done false
24282 7727 Acked Write 1 Done Sent Done false
24283 7728 Acked Write 2 Done Sent Done false
24284 7729 Acked Read 10 Done Sent Done false
24285 7730 Acked Write 2 Done Sent Done false
24286 7731 Acked Write 1 Done Sent Done false
24287 7732 Acked Read 3 Done Sent Done false
24288 7733 Acked Flush 0 Done Sent Done false
24289 7734 Acked Read 5 Done Sent Done false
24290 7735 Acked Write 4 Done Sent Done false
24291 7736 Acked Flush 0 Done Sent Done false
24292 7737 Acked Write 2 Done Sent Done false
24293 7738 Acked Read 6 Done Sent Done false
24294 7739 Acked Write 2 Done Sent Done false
24295 7740 Acked Write 7 Done Sent Done false
24296 7741 Acked Read 2 Done Sent Done false
24297 7742 Acked Write 6 Done Sent Done false
24298 7743 Acked Write 10 Done Sent Done false
24299 7744 Acked Read 4 Done Sent Done false
24300 7745 Acked Write 2 Done Sent Done false
24301 7746 Acked Read 1 Done Sent Done false
24302 7747 Acked Write 6 Done Sent Done false
24303 7748 Acked Write 6 Done Sent Done false
24304 7749 Acked Write 8 Done Sent Done false
24305 7750 Acked Read 3 Done Sent Done false
24306 7751 Acked Read 10 Done Sent Done false
24307 7752 Acked Read 8 Done Sent Done false
24308 7753 Acked Write 10 Done Sent Done false
24309 7754 Acked Write 6 Done Sent Done false
24310 7755 Acked Write 2 Done Sent Done false
24311 7756 Acked Write 8 Done Sent Done false
24312 7757 Acked Read 10 Done Sent Done false
24313 7758 Acked Write 4 Done Sent Done false
24314 7759 Acked Read 1 Done Sent Done false
24315 7760 Acked Flush 0 Done Sent Done false
24316 7761 Acked Write 8 Done Sent Done false
24317 7762 Acked Write 7 Done Sent Done false
24318 7763 Acked Flush 0 Done Sent Done false
24319 7764 Acked Read 7 Done Sent Done false
24320 7765 Acked Read 6 Done Sent Done false
24321 7766 Acked Read 2 Done Sent Done false
24322 7767 Acked Write 6 Done Sent Done false
24323 7768 Acked Read 10 Done Sent Done false
24324 7769 Acked Read 3 Done Sent Done false
24325 7770 Acked Flush 0 Done Sent Done false
24326 7771 Acked Write 9 Done Sent Done false
24327 7772 Acked Read 5 Done Sent Done false
24328 7773 Acked Write 1 Done Sent Done false
24329 7774 Acked Write 4 Done Sent Done false
24330 7775 Acked Read 9 Done Sent Done false
24331 7776 Acked Write 9 Done Sent Done false
24332 7777 Acked Write 5 Done Sent Done false
24333 7778 Acked Read 8 Done Sent Done false
24334 7779 Acked Read 6 Done Sent Done false
24335 7780 Acked Read 4 Done Sent Done false
24336 7781 Acked Read 6 Done Sent Done false
24337 7782 Acked Read 7 Done Sent Done false
24338 7783 Acked Read 6 Done Sent Done false
24339 7784 Acked Write 5 Done Sent Done false
24340 7785 Acked Read 3 Done Sent Done false
24341 7786 Acked Write 1 Done Sent Done false
24342 7787 Acked Read 2 Done Sent Done false
24343 7788 Acked Read 10 Done Sent Done false
24344 7789 Acked Write 2 Done Sent Done false
24345 7790 Acked Read 9 Done Sent Done false
24346 7791 Acked Write 4 Done Sent Done false
24347 7792 Acked Read 7 Done Sent Done false
24348 7793 Acked Read 10 Done Sent Done false
24349 7794 Acked Read 8 Done Sent Done false
24350 7795 Acked Read 7 Done Sent Done false
24351 7796 Acked Write 6 Done Sent Done false
24352 7797 Acked Read 1 Done Sent Done false
24353 7798 Acked Read 2 Done Sent Done false
24354 7799 Acked Read 4 Done Sent Done false
24355 7800 Acked Write 1 Done Sent Done false
24356 7801 Acked Read 4 Done Sent Done false
24357 7802 Acked Write 8 Done Sent Done false
24358 7803 Acked Read 1 Done Sent Done false
24359 7804 Acked Write 7 Done Sent Done false
24360 7805 Acked Read 9 Done Sent Done false
24361 7806 Acked Write 10 Done Sent Done false
24362 7807 Acked Write 1 Done Sent Done false
24363 7808 Acked Read 2 Done Sent Done false
24364 7809 Acked Read 4 Done Sent Done false
24365 7810 Acked Read 2 Done Sent Done false
24366 7811 Acked Write 4 Done Sent Done false
24367 7812 Acked Read 6 Done Sent Done false
24368 7813 Acked Read 5 Done Sent Done false
24369 7814 Acked Read 9 Done Sent Done false
24370 7815 Acked Read 1 Done Sent Done false
24371 7816 Acked Write 7 Done Sent Done false
24372 7817 Acked Read 10 Done Sent Done false
24373 7818 Acked Read 5 Done Sent Done false
24374 7819 Acked Write 5 Done Sent Done false
24375 7820 Acked Write 10 Done Sent Done false
24376 7821 Acked Read 2 Done Sent Done false
24377 7822 Acked Read 4 Done Sent Done false
24378 7823 Acked Write 5 Done Sent Done false
24379 7824 Acked Read 8 Done Sent Done false
24380 7825 Acked Write 1 Done Sent Done false
24381 7826 Acked Read 4 Done Sent Done false
24382 7827 Acked Write 8 Done Sent Done false
24383 7828 Acked Write 1 Done Sent Done false
24384 7829 Acked Read 6 Done Sent Done false
24385 7830 Acked Read 2 Done Sent Done false
24386 7831 Acked Read 4 Done Sent Done false
24387 7832 Acked Read 6 Done Sent Done false
24388 7833 Acked Read 8 Done Sent Done false
24389 7834 Acked Read 5 Done Sent Done false
24390 7835 Acked Write 9 Done Sent Done false
24391 7836 Acked Write 8 Done Sent Done false
24392 7837 Acked Read 1 Done Sent Done false
24393 7838 Acked Read 2 Done Sent Done false
24394 7839 Acked Write 9 Done Sent Done false
24395 7840 Acked Write 7 Done Sent Done false
24396 7841 Acked Write 4 Done Sent Done false
24397 7842 Acked Read 2 Done Sent Done false
24398 7843 Acked Write 7 Done Sent Done false
24399 7844 Acked Read 3 Done Sent Done false
24400 7845 Acked Write 3 Done Sent Done false
24401 7846 Acked Read 10 Done Sent Done false
24402 7847 Acked Read 3 Done Sent Done false
24403 7848 Acked Flush 0 Done Sent Done false
24404 7849 Acked Read 3 Done Sent Done false
24405 7850 Acked Write 10 Done Sent Done false
24406 7851 Acked Read 10 Done Sent Done false
24407 7852 Acked Read 8 Done Sent Done false
24408 7853 Acked Write 1 Done Sent Done false
24409 7854 Acked Read 7 Done Sent Done false
24410 7855 Acked Read 5 Done Sent Done false
24411 7856 Acked Read 8 Done Sent Done false
24412 7857 Acked Write 9 Done Sent Done false
24413 7858 Acked Read 5 Done Sent Done false
24414 7859 Acked Read 4 Done Sent Done false
24415 7860 Acked Write 9 Done Sent Done false
24416 7861 Acked Read 8 Done Sent Done false
24417 7862 Acked Read 3 Done Sent Done false
24418 7863 Acked Write 8 Done Sent Done false
24419 7864 Acked Read 4 Done Sent Done false
24420 7865 Acked Read 2 Done Sent Done false
24421 7866 Acked Read 5 Done Sent Done false
24422 7867 Acked Write 6 Done Sent Done false
24423 7868 Acked Read 10 Done Sent Done false
24424 7869 Acked Read 10 Done Sent Done false
24425 7870 Acked Write 10 Done Sent Done false
24426 7871 Acked Read 5 Done Sent Done false
24427 7872 Acked Read 4 Done Sent Done false
24428 7873 Acked Read 9 Done Sent Done false
24429 7874 Acked Read 7 Done Sent Done false
24430 7875 Acked Read 8 Done Sent Done false
24431 7876 Acked Read 8 Done Sent Done false
24432 7877 Acked Write 8 Done Sent Done false
24433 7878 Acked Write 1 Done Sent Done false
24434 7879 Acked Write 5 Done Sent Done false
24435 7880 Acked Read 2 Done Sent Done false
24436 7881 Acked Read 5 Done Sent Done false
24437 7882 Acked Write 4 Done Sent Done false
24438 7883 Acked Read 1 Done Sent Done false
24439 7884 Acked Read 7 Done Sent Done false
24440 7885 Acked Read 7 Done Sent Done false
24441 7886 Acked Write 3 Done Sent Done false
24442 7887 Acked Read 4 Done Sent Done false
24443 7888 Acked Write 1 Done Sent Done false
24444 7889 Acked Read 8 Done Sent Done false
24445 7890 Acked Read 2 Done Sent Done false
24446 7891 Acked Read 1 Done Sent Done false
24447 7892 Acked Read 5 Done Sent Done false
24448 7893 Acked Read 5 Done Sent Done false
24449 7894 Acked Read 3 Done Sent Done false
24450 7895 Acked Read 6 Done Sent Done false
24451 7896 Acked Write 6 Done Sent Done false
24452 7897 Acked Read 2 Done Sent Done false
24453 7898 Acked Write 7 Done Sent Done false
24454 7899 Acked Write 1 Done Sent Done false
24455 7900 Acked Write 4 Done Sent Done false
24456 7901 Acked Read 3 Done Sent Done false
24457 7902 Acked Write 8 Done Sent Done false
24458 7903 Acked Flush 0 Done Sent Done false
24459 7904 Acked Write 3 Done Sent Done false
24460 7905 Acked Write 6 Done Sent Done false
24461 7906 Acked Read 7 Done Sent Done false
24462 7907 Acked Write 4 Done Sent Done false
24463 7908 Acked Write 1 Done Sent Done false
24464 7909 Acked Read 6 Done Sent Done false
24465 7910 Acked Read 2 Done Sent Done false
24466 7911 Acked Write 3 Done Sent Done false
24467 7912 Acked Write 4 Done Sent Done false
24468 7913 Acked Read 10 Done Sent Done false
24469 7914 Acked Read 2 Done Sent Done false
24470 7915 Acked Write 6 Done Sent Done false
24471 7916 Acked Write 3 Done Sent Done false
24472 7917 Acked Read 8 Done Sent Done false
24473 7918 Acked Read 6 Done Sent Done false
24474 7919 Acked Read 3 Done Sent Done false
24475 7920 Acked Write 3 Done Sent Done false
24476 7921 Acked Write 4 Done Sent Done false
24477 7922 Acked Write 7 Done Sent Done false
24478 7923 Acked Read 3 Done Sent Done false
24479 7924 Acked Write 6 Done Sent Done false
24480 7925 Acked Write 7 Done Sent Done false
24481 7926 Acked Read 4 Done Sent Done false
24482 7927 Acked Read 3 Done Sent Done false
24483 7928 Acked Read 5 Done Sent Done false
24484 7929 Acked Read 7 Done Sent Done false
24485 7930 Acked Write 3 Done Sent Done false
24486 7931 Acked Flush 0 Done Sent Done false
24487 7932 Acked Write 7 Done Sent Done false
24488 7933 Acked Read 6 Done Sent Done false
24489 7934 Acked Read 3 Done Sent Done false
24490 7935 Acked Write 7 Done Sent Done false
24491 7936 Acked Read 5 Done Sent Done false
24492 7937 Acked Read 6 Done Sent Done false
24493 7938 Acked Read 2 Done Sent Done false
24494 7939 Acked Write 3 Done Sent Done false
24495 7940 Acked Read 1 Done Sent Done false
24496 7941 Acked Read 8 Done Sent Done false
24497 7942 Acked Flush 0 Done Sent Done false
24498 STATES DS:0 DS:1 DS:2 TOTAL
24499 Sent 0 302 0 302
24500 Done 302 0 302 604
24501 Skipped 0 0 0 0
24502 Error 0 0 0 0
24503 Last Flush: 7942 7640 7942
24504 Downstairs last five completed: 7640 7639 7638 7637 7636
24505 Upstairs last five completed: 7942 7941 7940 7939 7938
24506 CLIENT: Up:0 ds:302 act:2
245072024-12-04T13:35:49.536ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
245082024-12-04T13:35:49.536ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
245092024-12-04T13:35:49.536ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
245102024-12-04T13:35:49.536ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
245112024-12-04T13:35:49.536ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
24512 [8820] got start action
24513 [8820] I am going to restart
24514 Starting downstairs at port 8820
24515 Make output file at "/tmp/dsc/downstairs-8820.txt"
24516 [8820][1] reports Starting
24517 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1226)
24518 [8820][1] reports Running
24519 ----------------------------------------------------------------
24520 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24521 States: Active Offline Active
24522 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24523 7641 Acked Read 6 Done Sent Done false
24524 7642 Acked Read 1 Done Sent Done false
24525 7643 Acked Write 8 Done Sent Done false
24526 7644 Acked Read 8 Done Sent Done false
24527 7645 Acked Read 9 Done Sent Done false
24528 7646 Acked Write 10 Done Sent Done false
24529 7647 Acked Flush 0 Done Sent Done false
24530 7648 Acked Write 7 Done Sent Done false
24531 7649 Acked Read 8 Done Sent Done false
24532 7650 Acked Read 8 Done Sent Done false
24533 7651 Acked Read 5 Done Sent Done false
24534 7652 Acked Flush 0 Done Sent Done false
24535 7653 Acked Write 6 Done Sent Done false
24536 7654 Acked Read 2 Done Sent Done false
24537 7655 Acked Flush 0 Done Sent Done false
24538 7656 Acked Read 4 Done Sent Done false
24539 7657 Acked Write 1 Done Sent Done false
24540 7658 Acked Write 5 Done Sent Done false
24541 7659 Acked Read 5 Done Sent Done false
24542 7660 Acked Write 8 Done Sent Done false
24543 7661 Acked Read 2 Done Sent Done false
24544 7662 Acked Write 8 Done Sent Done false
24545 7663 Acked Read 8 Done Sent Done false
24546 7664 Acked Flush 0 Done Sent Done false
24547 7665 Acked Write 7 Done Sent Done false
24548 7666 Acked Write 8 Done Sent Done false
24549 7667 Acked Read 6 Done Sent Done false
24550 7668 Acked Write 6 Done Sent Done false
24551 7669 Acked Write 6 Done Sent Done false
24552 7670 Acked Write 7 Done Sent Done false
24553 7671 Acked Read 8 Done Sent Done false
24554 7672 Acked Write 7 Done Sent Done false
24555 7673 Acked Write 5 Done Sent Done false
24556 7674 Acked Read 5 Done Sent Done false
24557 7675 Acked Write 7 Done Sent Done false
24558 7676 Acked Read 8 Done Sent Done false
24559 7677 Acked Flush 0 Done Sent Done false
24560 7678 Acked Read 3 Done Sent Done false
24561 7679 Acked Write 3 Done Sent Done false
24562 7680 Acked Read 4 Done Sent Done false
24563 7681 Acked Read 8 Done Sent Done false
24564 7682 Acked Read 4 Done Sent Done false
24565 7683 Acked Write 6 Done Sent Done false
24566 7684 Acked Read 4 Done Sent Done false
24567 7685 Acked Read 9 Done Sent Done false
24568 7686 Acked Read 3 Done Sent Done false
24569 7687 Acked Write 4 Done Sent Done false
24570 7688 Acked Write 4 Done Sent Done false
24571 7689 Acked Write 1 Done Sent Done false
24572 7690 Acked Read 7 Done Sent Done false
24573 7691 Acked Write 2 Done Sent Done false
24574 7692 Acked Read 7 Done Sent Done false
24575 7693 Acked Write 6 Done Sent Done false
24576 7694 Acked Read 1 Done Sent Done false
24577 7695 Acked Read 7 Done Sent Done false
24578 7696 Acked Write 3 Done Sent Done false
24579 7697 Acked Read 4 Done Sent Done false
24580 7698 Acked Flush 0 Done Sent Done false
24581 7699 Acked Read 7 Done Sent Done false
24582 7700 Acked Read 1 Done Sent Done false
24583 7701 Acked Read 10 Done Sent Done false
24584 7702 Acked Write 10 Done Sent Done false
24585 7703 Acked Read 9 Done Sent Done false
24586 7704 Acked Read 7 Done Sent Done false
24587 7705 Acked Flush 0 Done Sent Done false
24588 7706 Acked Write 7 Done Sent Done false
24589 7707 Acked Read 5 Done Sent Done false
24590 7708 Acked Read 4 Done Sent Done false
24591 7709 Acked Write 8 Done Sent Done false
24592 7710 Acked Write 7 Done Sent Done false
24593 7711 Acked Write 9 Done Sent Done false
24594 7712 Acked Read 10 Done Sent Done false
24595 7713 Acked Read 5 Done Sent Done false
24596 7714 Acked Read 3 Done Sent Done false
24597 7715 Acked Write 5 Done Sent Done false
24598 7716 Acked Flush 0 Done Sent Done false
24599 7717 Acked Write 2 Done Sent Done false
24600 7718 Acked Flush 0 Done Sent Done false
24601 7719 Acked Write 5 Done Sent Done false
24602 7720 Acked Read 3 Done Sent Done false
24603 7721 Acked Flush 0 Done Sent Done false
24604 7722 Acked Write 5 Done Sent Done false
24605 7723 Acked Write 10 Done Sent Done false
24606 7724 Acked Flush 0 Done Sent Done false
24607 7725 Acked Read 8 Done Sent Done false
24608 7726 Acked Read 1 Done Sent Done false
24609 7727 Acked Write 1 Done Sent Done false
24610 7728 Acked Write 2 Done Sent Done false
24611 7729 Acked Read 10 Done Sent Done false
24612 7730 Acked Write 2 Done Sent Done false
24613 7731 Acked Write 1 Done Sent Done false
24614 7732 Acked Read 3 Done Sent Done false
24615 7733 Acked Flush 0 Done Sent Done false
24616 7734 Acked Read 5 Done Sent Done false
24617 7735 Acked Write 4 Done Sent Done false
24618 7736 Acked Flush 0 Done Sent Done false
24619 7737 Acked Write 2 Done Sent Done false
24620 7738 Acked Read 6 Done Sent Done false
24621 7739 Acked Write 2 Done Sent Done false
24622 7740 Acked Write 7 Done Sent Done false
24623 7741 Acked Read 2 Done Sent Done false
24624 7742 Acked Write 6 Done Sent Done false
24625 7743 Acked Write 10 Done Sent Done false
24626 7744 Acked Read 4 Done Sent Done false
24627 7745 Acked Write 2 Done Sent Done false
24628 7746 Acked Read 1 Done Sent Done false
24629 7747 Acked Write 6 Done Sent Done false
24630 7748 Acked Write 6 Done Sent Done false
24631 7749 Acked Write 8 Done Sent Done false
24632 7750 Acked Read 3 Done Sent Done false
24633 7751 Acked Read 10 Done Sent Done false
24634 7752 Acked Read 8 Done Sent Done false
24635 7753 Acked Write 10 Done Sent Done false
24636 7754 Acked Write 6 Done Sent Done false
24637 7755 Acked Write 2 Done Sent Done false
24638 7756 Acked Write 8 Done Sent Done false
24639 7757 Acked Read 10 Done Sent Done false
24640 7758 Acked Write 4 Done Sent Done false
24641 7759 Acked Read 1 Done Sent Done false
24642 7760 Acked Flush 0 Done Sent Done false
24643 7761 Acked Write 8 Done Sent Done false
24644 7762 Acked Write 7 Done Sent Done false
24645 7763 Acked Flush 0 Done Sent Done false
24646 7764 Acked Read 7 Done Sent Done false
24647 7765 Acked Read 6 Done Sent Done false
24648 7766 Acked Read 2 Done Sent Done false
24649 7767 Acked Write 6 Done Sent Done false
24650 7768 Acked Read 10 Done Sent Done false
24651 7769 Acked Read 3 Done Sent Done false
24652 7770 Acked Flush 0 Done Sent Done false
24653 7771 Acked Write 9 Done Sent Done false
24654 7772 Acked Read 5 Done Sent Done false
24655 7773 Acked Write 1 Done Sent Done false
24656 7774 Acked Write 4 Done Sent Done false
24657 7775 Acked Read 9 Done Sent Done false
24658 7776 Acked Write 9 Done Sent Done false
24659 7777 Acked Write 5 Done Sent Done false
24660 7778 Acked Read 8 Done Sent Done false
24661 7779 Acked Read 6 Done Sent Done false
24662 7780 Acked Read 4 Done Sent Done false
24663 7781 Acked Read 6 Done Sent Done false
24664 7782 Acked Read 7 Done Sent Done false
24665 7783 Acked Read 6 Done Sent Done false
24666 7784 Acked Write 5 Done Sent Done false
24667 7785 Acked Read 3 Done Sent Done false
24668 7786 Acked Write 1 Done Sent Done false
24669 7787 Acked Read 2 Done Sent Done false
24670 7788 Acked Read 10 Done Sent Done false
24671 7789 Acked Write 2 Done Sent Done false
24672 7790 Acked Read 9 Done Sent Done false
24673 7791 Acked Write 4 Done Sent Done false
24674 7792 Acked Read 7 Done Sent Done false
24675 7793 Acked Read 10 Done Sent Done false
24676 7794 Acked Read 8 Done Sent Done false
24677 7795 Acked Read 7 Done Sent Done false
24678 7796 Acked Write 6 Done Sent Done false
24679 7797 Acked Read 1 Done Sent Done false
24680 7798 Acked Read 2 Done Sent Done false
24681 7799 Acked Read 4 Done Sent Done false
24682 7800 Acked Write 1 Done Sent Done false
24683 7801 Acked Read 4 Done Sent Done false
24684 7802 Acked Write 8 Done Sent Done false
24685 7803 Acked Read 1 Done Sent Done false
24686 7804 Acked Write 7 Done Sent Done false
24687 7805 Acked Read 9 Done Sent Done false
24688 7806 Acked Write 10 Done Sent Done false
24689 7807 Acked Write 1 Done Sent Done false
24690 7808 Acked Read 2 Done Sent Done false
24691 7809 Acked Read 4 Done Sent Done false
24692 7810 Acked Read 2 Done Sent Done false
24693 7811 Acked Write 4 Done Sent Done false
24694 7812 Acked Read 6 Done Sent Done false
24695 7813 Acked Read 5 Done Sent Done false
24696 7814 Acked Read 9 Done Sent Done false
24697 7815 Acked Read 1 Done Sent Done false
24698 7816 Acked Write 7 Done Sent Done false
24699 7817 Acked Read 10 Done Sent Done false
24700 7818 Acked Read 5 Done Sent Done false
24701 7819 Acked Write 5 Done Sent Done false
24702 7820 Acked Write 10 Done Sent Done false
24703 7821 Acked Read 2 Done Sent Done false
24704 7822 Acked Read 4 Done Sent Done false
24705 7823 Acked Write 5 Done Sent Done false
24706 7824 Acked Read 8 Done Sent Done false
24707 7825 Acked Write 1 Done Sent Done false
24708 7826 Acked Read 4 Done Sent Done false
24709 7827 Acked Write 8 Done Sent Done false
24710 7828 Acked Write 1 Done Sent Done false
24711 7829 Acked Read 6 Done Sent Done false
24712 7830 Acked Read 2 Done Sent Done false
24713 7831 Acked Read 4 Done Sent Done false
24714 7832 Acked Read 6 Done Sent Done false
24715 7833 Acked Read 8 Done Sent Done false
24716 7834 Acked Read 5 Done Sent Done false
24717 7835 Acked Write 9 Done Sent Done false
24718 7836 Acked Write 8 Done Sent Done false
24719 7837 Acked Read 1 Done Sent Done false
24720 7838 Acked Read 2 Done Sent Done false
24721 7839 Acked Write 9 Done Sent Done false
24722 7840 Acked Write 7 Done Sent Done false
24723 7841 Acked Write 4 Done Sent Done false
24724 7842 Acked Read 2 Done Sent Done false
24725 7843 Acked Write 7 Done Sent Done false
24726 7844 Acked Read 3 Done Sent Done false
24727 7845 Acked Write 3 Done Sent Done false
24728 7846 Acked Read 10 Done Sent Done false
24729 7847 Acked Read 3 Done Sent Done false
24730 7848 Acked Flush 0 Done Sent Done false
24731 7849 Acked Read 3 Done Sent Done false
24732 7850 Acked Write 10 Done Sent Done false
24733 7851 Acked Read 10 Done Sent Done false
24734 7852 Acked Read 8 Done Sent Done false
24735 7853 Acked Write 1 Done Sent Done false
24736 7854 Acked Read 7 Done Sent Done false
24737 7855 Acked Read 5 Done Sent Done false
24738 7856 Acked Read 8 Done Sent Done false
24739 7857 Acked Write 9 Done Sent Done false
24740 7858 Acked Read 5 Done Sent Done false
24741 7859 Acked Read 4 Done Sent Done false
24742 7860 Acked Write 9 Done Sent Done false
24743 7861 Acked Read 8 Done Sent Done false
24744 7862 Acked Read 3 Done Sent Done false
24745 7863 Acked Write 8 Done Sent Done false
24746 7864 Acked Read 4 Done Sent Done false
24747 7865 Acked Read 2 Done Sent Done false
24748 7866 Acked Read 5 Done Sent Done false
24749 7867 Acked Write 6 Done Sent Done false
24750 7868 Acked Read 10 Done Sent Done false
24751 7869 Acked Read 10 Done Sent Done false
24752 7870 Acked Write 10 Done Sent Done false
24753 7871 Acked Read 5 Done Sent Done false
24754 7872 Acked Read 4 Done Sent Done false
24755 7873 Acked Read 9 Done Sent Done false
24756 7874 Acked Read 7 Done Sent Done false
24757 7875 Acked Read 8 Done Sent Done false
24758 7876 Acked Read 8 Done Sent Done false
24759 7877 Acked Write 8 Done Sent Done false
24760 7878 Acked Write 1 Done Sent Done false
24761 7879 Acked Write 5 Done Sent Done false
24762 7880 Acked Read 2 Done Sent Done false
24763 7881 Acked Read 5 Done Sent Done false
24764 7882 Acked Write 4 Done Sent Done false
24765 7883 Acked Read 1 Done Sent Done false
24766 7884 Acked Read 7 Done Sent Done false
24767 7885 Acked Read 7 Done Sent Done false
24768 7886 Acked Write 3 Done Sent Done false
24769 7887 Acked Read 4 Done Sent Done false
24770 7888 Acked Write 1 Done Sent Done false
24771 7889 Acked Read 8 Done Sent Done false
24772 7890 Acked Read 2 Done Sent Done false
24773 7891 Acked Read 1 Done Sent Done false
24774 7892 Acked Read 5 Done Sent Done false
24775 7893 Acked Read 5 Done Sent Done false
24776 7894 Acked Read 3 Done Sent Done false
24777 7895 Acked Read 6 Done Sent Done false
24778 7896 Acked Write 6 Done Sent Done false
24779 7897 Acked Read 2 Done Sent Done false
24780 7898 Acked Write 7 Done Sent Done false
24781 7899 Acked Write 1 Done Sent Done false
24782 7900 Acked Write 4 Done Sent Done false
24783 7901 Acked Read 3 Done Sent Done false
24784 7902 Acked Write 8 Done Sent Done false
24785 7903 Acked Flush 0 Done Sent Done false
24786 7904 Acked Write 3 Done Sent Done false
24787 7905 Acked Write 6 Done Sent Done false
24788 7906 Acked Read 7 Done Sent Done false
24789 7907 Acked Write 4 Done Sent Done false
24790 7908 Acked Write 1 Done Sent Done false
24791 7909 Acked Read 6 Done Sent Done false
24792 7910 Acked Read 2 Done Sent Done false
24793 7911 Acked Write 3 Done Sent Done false
24794 7912 Acked Write 4 Done Sent Done false
24795 7913 Acked Read 10 Done Sent Done false
24796 7914 Acked Read 2 Done Sent Done false
24797 7915 Acked Write 6 Done Sent Done false
24798 7916 Acked Write 3 Done Sent Done false
24799 7917 Acked Read 8 Done Sent Done false
24800 7918 Acked Read 6 Done Sent Done false
24801 7919 Acked Read 3 Done Sent Done false
24802 7920 Acked Write 3 Done Sent Done false
24803 7921 Acked Write 4 Done Sent Done false
24804 7922 Acked Write 7 Done Sent Done false
24805 7923 Acked Read 3 Done Sent Done false
24806 7924 Acked Write 6 Done Sent Done false
24807 7925 Acked Write 7 Done Sent Done false
24808 7926 Acked Read 4 Done Sent Done false
24809 7927 Acked Read 3 Done Sent Done false
24810 7928 Acked Read 5 Done Sent Done false
24811 7929 Acked Read 7 Done Sent Done false
24812 7930 Acked Write 3 Done Sent Done false
24813 7931 Acked Flush 0 Done Sent Done false
24814 7932 Acked Write 7 Done Sent Done false
24815 7933 Acked Read 6 Done Sent Done false
24816 7934 Acked Read 3 Done Sent Done false
24817 7935 Acked Write 7 Done Sent Done false
24818 7936 Acked Read 5 Done Sent Done false
24819 7937 Acked Read 6 Done Sent Done false
24820 7938 Acked Read 2 Done Sent Done false
24821 7939 Acked Write 3 Done Sent Done false
24822 7940 Acked Read 1 Done Sent Done false
24823 7941 Acked Read 8 Done Sent Done false
24824 7942 Acked Flush 0 Done Sent Done false
24825 STATES DS:0 DS:1 DS:2 TOTAL
24826 Sent 0 302 0 302
24827 Done 302 0 302 604
24828 Skipped 0 0 0 0
24829 Error 0 0 0 0
24830 Last Flush: 7942 7640 7942
24831 Downstairs last five completed: 7640 7639 7638 7637 7636
24832 Upstairs last five completed: 7942 7941 7940 7939 7938
24833 CLIENT: Up:0 ds:302 act:2
24834 ----------------------------------------------------------------
24835 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24836 States: Active Offline Active
24837 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24838 7641 Acked Read 6 Done Sent Done false
24839 7642 Acked Read 1 Done Sent Done false
24840 7643 Acked Write 8 Done Sent Done false
24841 7644 Acked Read 8 Done Sent Done false
24842 7645 Acked Read 9 Done Sent Done false
24843 7646 Acked Write 10 Done Sent Done false
24844 7647 Acked Flush 0 Done Sent Done false
24845 7648 Acked Write 7 Done Sent Done false
24846 7649 Acked Read 8 Done Sent Done false
24847 7650 Acked Read 8 Done Sent Done false
24848 7651 Acked Read 5 Done Sent Done false
24849 7652 Acked Flush 0 Done Sent Done false
24850 7653 Acked Write 6 Done Sent Done false
24851 7654 Acked Read 2 Done Sent Done false
24852 7655 Acked Flush 0 Done Sent Done false
24853 7656 Acked Read 4 Done Sent Done false
24854 7657 Acked Write 1 Done Sent Done false
24855 7658 Acked Write 5 Done Sent Done false
24856 7659 Acked Read 5 Done Sent Done false
24857 7660 Acked Write 8 Done Sent Done false
24858 7661 Acked Read 2 Done Sent Done false
24859 7662 Acked Write 8 Done Sent Done false
24860 7663 Acked Read 8 Done Sent Done false
24861 7664 Acked Flush 0 Done Sent Done false
24862 7665 Acked Write 7 Done Sent Done false
24863 7666 Acked Write 8 Done Sent Done false
24864 7667 Acked Read 6 Done Sent Done false
24865 7668 Acked Write 6 Done Sent Done false
24866 7669 Acked Write 6 Done Sent Done false
24867 7670 Acked Write 7 Done Sent Done false
24868 7671 Acked Read 8 Done Sent Done false
24869 7672 Acked Write 7 Done Sent Done false
24870 7673 Acked Write 5 Done Sent Done false
24871 7674 Acked Read 5 Done Sent Done false
24872 7675 Acked Write 7 Done Sent Done false
24873 7676 Acked Read 8 Done Sent Done false
24874 7677 Acked Flush 0 Done Sent Done false
24875 7678 Acked Read 3 Done Sent Done false
24876 7679 Acked Write 3 Done Sent Done false
24877 7680 Acked Read 4 Done Sent Done false
24878 7681 Acked Read 8 Done Sent Done false
24879 7682 Acked Read 4 Done Sent Done false
24880 7683 Acked Write 6 Done Sent Done false
24881 7684 Acked Read 4 Done Sent Done false
24882 7685 Acked Read 9 Done Sent Done false
24883 7686 Acked Read 3 Done Sent Done false
24884 7687 Acked Write 4 Done Sent Done false
24885 7688 Acked Write 4 Done Sent Done false
24886 7689 Acked Write 1 Done Sent Done false
24887 7690 Acked Read 7 Done Sent Done false
24888 7691 Acked Write 2 Done Sent Done false
24889 7692 Acked Read 7 Done Sent Done false
24890 7693 Acked Write 6 Done Sent Done false
24891 7694 Acked Read 1 Done Sent Done false
24892 7695 Acked Read 7 Done Sent Done false
24893 7696 Acked Write 3 Done Sent Done false
24894 7697 Acked Read 4 Done Sent Done false
24895 7698 Acked Flush 0 Done Sent Done false
24896 7699 Acked Read 7 Done Sent Done false
24897 7700 Acked Read 1 Done Sent Done false
24898 7701 Acked Read 10 Done Sent Done false
24899 7702 Acked Write 10 Done Sent Done false
24900 7703 Acked Read 9 Done Sent Done false
24901 7704 Acked Read 7 Done Sent Done false
24902 7705 Acked Flush 0 Done Sent Done false
24903 7706 Acked Write 7 Done Sent Done false
24904 7707 Acked Read 5 Done Sent Done false
24905 7708 Acked Read 4 Done Sent Done false
24906 7709 Acked Write 8 Done Sent Done false
24907 7710 Acked Write 7 Done Sent Done false
24908 7711 Acked Write 9 Done Sent Done false
24909 7712 Acked Read 10 Done Sent Done false
24910 7713 Acked Read 5 Done Sent Done false
24911 7714 Acked Read 3 Done Sent Done false
24912 7715 Acked Write 5 Done Sent Done false
24913 7716 Acked Flush 0 Done Sent Done false
24914 7717 Acked Write 2 Done Sent Done false
24915 7718 Acked Flush 0 Done Sent Done false
24916 7719 Acked Write 5 Done Sent Done false
24917 7720 Acked Read 3 Done Sent Done false
24918 7721 Acked Flush 0 Done Sent Done false
24919 7722 Acked Write 5 Done Sent Done false
24920 7723 Acked Write 10 Done Sent Done false
24921 7724 Acked Flush 0 Done Sent Done false
24922 7725 Acked Read 8 Done Sent Done false
24923 7726 Acked Read 1 Done Sent Done false
24924 7727 Acked Write 1 Done Sent Done false
24925 7728 Acked Write 2 Done Sent Done false
24926 7729 Acked Read 10 Done Sent Done false
24927 7730 Acked Write 2 Done Sent Done false
24928 7731 Acked Write 1 Done Sent Done false
24929 7732 Acked Read 3 Done Sent Done false
24930 7733 Acked Flush 0 Done Sent Done false
24931 7734 Acked Read 5 Done Sent Done false
24932 7735 Acked Write 4 Done Sent Done false
24933 7736 Acked Flush 0 Done Sent Done false
24934 7737 Acked Write 2 Done Sent Done false
24935 7738 Acked Read 6 Done Sent Done false
24936 7739 Acked Write 2 Done Sent Done false
24937 7740 Acked Write 7 Done Sent Done false
24938 7741 Acked Read 2 Done Sent Done false
24939 7742 Acked Write 6 Done Sent Done false
24940 7743 Acked Write 10 Done Sent Done false
24941 7744 Acked Read 4 Done Sent Done false
24942 7745 Acked Write 2 Done Sent Done false
24943 7746 Acked Read 1 Done Sent Done false
24944 7747 Acked Write 6 Done Sent Done false
24945 7748 Acked Write 6 Done Sent Done false
24946 7749 Acked Write 8 Done Sent Done false
24947 7750 Acked Read 3 Done Sent Done false
24948 7751 Acked Read 10 Done Sent Done false
24949 7752 Acked Read 8 Done Sent Done false
24950 7753 Acked Write 10 Done Sent Done false
24951 7754 Acked Write 6 Done Sent Done false
24952 7755 Acked Write 2 Done Sent Done false
24953 7756 Acked Write 8 Done Sent Done false
24954 7757 Acked Read 10 Done Sent Done false
24955 7758 Acked Write 4 Done Sent Done false
24956 7759 Acked Read 1 Done Sent Done false
24957 7760 Acked Flush 0 Done Sent Done false
24958 7761 Acked Write 8 Done Sent Done false
24959 7762 Acked Write 7 Done Sent Done false
24960 7763 Acked Flush 0 Done Sent Done false
24961 7764 Acked Read 7 Done Sent Done false
24962 7765 Acked Read 6 Done Sent Done false
24963 7766 Acked Read 2 Done Sent Done false
24964 7767 Acked Write 6 Done Sent Done false
24965 7768 Acked Read 10 Done Sent Done false
24966 7769 Acked Read 3 Done Sent Done false
24967 7770 Acked Flush 0 Done Sent Done false
24968 7771 Acked Write 9 Done Sent Done false
24969 7772 Acked Read 5 Done Sent Done false
24970 7773 Acked Write 1 Done Sent Done false
24971 7774 Acked Write 4 Done Sent Done false
24972 7775 Acked Read 9 Done Sent Done false
24973 7776 Acked Write 9 Done Sent Done false
24974 7777 Acked Write 5 Done Sent Done false
24975 7778 Acked Read 8 Done Sent Done false
24976 7779 Acked Read 6 Done Sent Done false
24977 7780 Acked Read 4 Done Sent Done false
24978 7781 Acked Read 6 Done Sent Done false
24979 7782 Acked Read 7 Done Sent Done false
24980 7783 Acked Read 6 Done Sent Done false
24981 7784 Acked Write 5 Done Sent Done false
24982 7785 Acked Read 3 Done Sent Done false
24983 7786 Acked Write 1 Done Sent Done false
24984 7787 Acked Read 2 Done Sent Done false
24985 7788 Acked Read 10 Done Sent Done false
24986 7789 Acked Write 2 Done Sent Done false
24987 7790 Acked Read 9 Done Sent Done false
24988 7791 Acked Write 4 Done Sent Done false
24989 7792 Acked Read 7 Done Sent Done false
24990 7793 Acked Read 10 Done Sent Done false
24991 7794 Acked Read 8 Done Sent Done false
24992 7795 Acked Read 7 Done Sent Done false
24993 7796 Acked Write 6 Done Sent Done false
24994 7797 Acked Read 1 Done Sent Done false
24995 7798 Acked Read 2 Done Sent Done false
24996 7799 Acked Read 4 Done Sent Done false
24997 7800 Acked Write 1 Done Sent Done false
24998 7801 Acked Read 4 Done Sent Done false
24999 7802 Acked Write 8 Done Sent Done false
25000 7803 Acked Read 1 Done Sent Done false
25001 7804 Acked Write 7 Done Sent Done false
25002 7805 Acked Read 9 Done Sent Done false
25003 7806 Acked Write 10 Done Sent Done false
25004 7807 Acked Write 1 Done Sent Done false
25005 7808 Acked Read 2 Done Sent Done false
25006 7809 Acked Read 4 Done Sent Done false
25007 7810 Acked Read 2 Done Sent Done false
25008 7811 Acked Write 4 Done Sent Done false
25009 7812 Acked Read 6 Done Sent Done false
25010 7813 Acked Read 5 Done Sent Done false
25011 7814 Acked Read 9 Done Sent Done false
25012 7815 Acked Read 1 Done Sent Done false
25013 7816 Acked Write 7 Done Sent Done false
25014 7817 Acked Read 10 Done Sent Done false
25015 7818 Acked Read 5 Done Sent Done false
25016 7819 Acked Write 5 Done Sent Done false
25017 7820 Acked Write 10 Done Sent Done false
25018 7821 Acked Read 2 Done Sent Done false
25019 7822 Acked Read 4 Done Sent Done false
25020 7823 Acked Write 5 Done Sent Done false
25021 7824 Acked Read 8 Done Sent Done false
25022 7825 Acked Write 1 Done Sent Done false
25023 7826 Acked Read 4 Done Sent Done false
25024 7827 Acked Write 8 Done Sent Done false
25025 7828 Acked Write 1 Done Sent Done false
25026 7829 Acked Read 6 Done Sent Done false
25027 7830 Acked Read 2 Done Sent Done false
25028 7831 Acked Read 4 Done Sent Done false
25029 7832 Acked Read 6 Done Sent Done false
25030 7833 Acked Read 8 Done Sent Done false
25031 7834 Acked Read 5 Done Sent Done false
25032 7835 Acked Write 9 Done Sent Done false
25033 7836 Acked Write 8 Done Sent Done false
25034 7837 Acked Read 1 Done Sent Done false
25035 7838 Acked Read 2 Done Sent Done false
25036 7839 Acked Write 9 Done Sent Done false
25037 7840 Acked Write 7 Done Sent Done false
25038 7841 Acked Write 4 Done Sent Done false
25039 7842 Acked Read 2 Done Sent Done false
25040 7843 Acked Write 7 Done Sent Done false
25041 7844 Acked Read 3 Done Sent Done false
25042 7845 Acked Write 3 Done Sent Done false
25043 7846 Acked Read 10 Done Sent Done false
25044 7847 Acked Read 3 Done Sent Done false
25045 7848 Acked Flush 0 Done Sent Done false
25046 7849 Acked Read 3 Done Sent Done false
25047 7850 Acked Write 10 Done Sent Done false
25048 7851 Acked Read 10 Done Sent Done false
25049 7852 Acked Read 8 Done Sent Done false
25050 7853 Acked Write 1 Done Sent Done false
25051 7854 Acked Read 7 Done Sent Done false
25052 7855 Acked Read 5 Done Sent Done false
25053 7856 Acked Read 8 Done Sent Done false
25054 7857 Acked Write 9 Done Sent Done false
25055 7858 Acked Read 5 Done Sent Done false
25056 7859 Acked Read 4 Done Sent Done false
25057 7860 Acked Write 9 Done Sent Done false
25058 7861 Acked Read 8 Done Sent Done false
25059 7862 Acked Read 3 Done Sent Done false
25060 7863 Acked Write 8 Done Sent Done false
25061 7864 Acked Read 4 Done Sent Done false
25062 7865 Acked Read 2 Done Sent Done false
25063 7866 Acked Read 5 Done Sent Done false
25064 7867 Acked Write 6 Done Sent Done false
25065 7868 Acked Read 10 Done Sent Done false
25066 7869 Acked Read 10 Done Sent Done false
25067 7870 Acked Write 10 Done Sent Done false
25068 7871 Acked Read 5 Done Sent Done false
25069 7872 Acked Read 4 Done Sent Done false
25070 7873 Acked Read 9 Done Sent Done false
25071 7874 Acked Read 7 Done Sent Done false
25072 7875 Acked Read 8 Done Sent Done false
25073 7876 Acked Read 8 Done Sent Done false
25074 7877 Acked Write 8 Done Sent Done false
25075 7878 Acked Write 1 Done Sent Done false
25076 7879 Acked Write 5 Done Sent Done false
25077 7880 Acked Read 2 Done Sent Done false
25078 7881 Acked Read 5 Done Sent Done false
25079 7882 Acked Write 4 Done Sent Done false
25080 7883 Acked Read 1 Done Sent Done false
25081 7884 Acked Read 7 Done Sent Done false
25082 7885 Acked Read 7 Done Sent Done false
25083 7886 Acked Write 3 Done Sent Done false
25084 7887 Acked Read 4 Done Sent Done false
25085 7888 Acked Write 1 Done Sent Done false
25086 7889 Acked Read 8 Done Sent Done false
25087 7890 Acked Read 2 Done Sent Done false
25088 7891 Acked Read 1 Done Sent Done false
25089 7892 Acked Read 5 Done Sent Done false
25090 7893 Acked Read 5 Done Sent Done false
25091 7894 Acked Read 3 Done Sent Done false
25092 7895 Acked Read 6 Done Sent Done false
25093 7896 Acked Write 6 Done Sent Done false
25094 7897 Acked Read 2 Done Sent Done false
25095 7898 Acked Write 7 Done Sent Done false
25096 7899 Acked Write 1 Done Sent Done false
25097 7900 Acked Write 4 Done Sent Done false
25098 7901 Acked Read 3 Done Sent Done false
25099 7902 Acked Write 8 Done Sent Done false
25100 7903 Acked Flush 0 Done Sent Done false
25101 7904 Acked Write 3 Done Sent Done false
25102 7905 Acked Write 6 Done Sent Done false
25103 7906 Acked Read 7 Done Sent Done false
25104 7907 Acked Write 4 Done Sent Done false
25105 7908 Acked Write 1 Done Sent Done false
25106 7909 Acked Read 6 Done Sent Done false
25107 7910 Acked Read 2 Done Sent Done false
25108 7911 Acked Write 3 Done Sent Done false
25109 7912 Acked Write 4 Done Sent Done false
25110 7913 Acked Read 10 Done Sent Done false
25111 7914 Acked Read 2 Done Sent Done false
25112 7915 Acked Write 6 Done Sent Done false
25113 7916 Acked Write 3 Done Sent Done false
25114 7917 Acked Read 8 Done Sent Done false
25115 7918 Acked Read 6 Done Sent Done false
25116 7919 Acked Read 3 Done Sent Done false
25117 7920 Acked Write 3 Done Sent Done false
25118 7921 Acked Write 4 Done Sent Done false
25119 7922 Acked Write 7 Done Sent Done false
25120 7923 Acked Read 3 Done Sent Done false
25121 7924 Acked Write 6 Done Sent Done false
25122 7925 Acked Write 7 Done Sent Done false
25123 7926 Acked Read 4 Done Sent Done false
25124 7927 Acked Read 3 Done Sent Done false
25125 7928 Acked Read 5 Done Sent Done false
25126 7929 Acked Read 7 Done Sent Done false
25127 7930 Acked Write 3 Done Sent Done false
25128 7931 Acked Flush 0 Done Sent Done false
25129 7932 Acked Write 7 Done Sent Done false
25130 7933 Acked Read 6 Done Sent Done false
25131 7934 Acked Read 3 Done Sent Done false
25132 7935 Acked Write 7 Done Sent Done false
25133 7936 Acked Read 5 Done Sent Done false
25134 7937 Acked Read 6 Done Sent Done false
25135 7938 Acked Read 2 Done Sent Done false
25136 7939 Acked Write 3 Done Sent Done false
25137 7940 Acked Read 1 Done Sent Done false
25138 7941 Acked Read 8 Done Sent Done false
25139 7942 Acked Flush 0 Done Sent Done false
25140 STATES DS:0 DS:1 DS:2 TOTAL
25141 Sent 0 302 0 302
25142 Done 302 0 302 604
25143 Skipped 0 0 0 0
25144 Error 0 0 0 0
25145 Last Flush: 7942 7640 7942
25146 Downstairs last five completed: 7640 7639 7638 7637 7636
25147 Upstairs last five completed: 7942 7941 7940 7939 7938
25148 CLIENT: Up:0 ds:302 act:2
251492024-12-04T13:35:59.538ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251502024-12-04T13:35:59.538ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43478) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251512024-12-04T13:35:59.539ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251522024-12-04T13:35:59.540ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251532024-12-04T13:35:59.540ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251542024-12-04T13:35:59.540ZINFOcrucible: send last flush ID to this DS: 7640 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251552024-12-04T13:35:59.540ZINFOcrucible: Replied this last flush ID: 7640 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251562024-12-04T13:35:59.540ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251572024-12-04T13:35:59.540ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251582024-12-04T13:35:59.540ZINFOcrucible: [1] client re-new 302 jobs since flush 7640 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251592024-12-04T13:35:59.540ZINFOcrucible: [1] Replayed 302 jobs since flush: 7640 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
25160 ----------------------------------------------------------------
25161 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25162 Downstairs last five completed: 7942 7941 7940 7939 7938
25163 Upstairs last five completed: 7942 7941 7940 7939 7938
25164 CLIENT: Up:0 ds:0 act:3
25165 Replay: All jobs finished
25166 Main task has work to do, go find it
25167 got dsc Stop(2)
25168 stop 2
25169 Dec 04 13:35:59.783 INFO request completed[8830] Got stop action so:false kr:false
25170 , latency_us: 345, response_code: 204, uri: /stop/cid/2, method: POST, req_id: b40a76a7-2bdf-4858-b307-2ae9c68508d9, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
251712024-12-04T13:35:59.783ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251722024-12-04T13:35:59.783ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251732024-12-04T13:35:59.784ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251742024-12-04T13:35:59.784ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251752024-12-04T13:35:59.784ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
251762024-12-04T13:35:59.784ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
25177 Dec 04 13:35:59.784 INFO request completed, latency_us: 239, response_code: 200, uri: /state/cid/2, method: GET, req_id: aa38880a-4b62-40ff-a036-a7aadc713ab5, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
25178 [8830] Exited with: ExitStatus(unix_wait_status(9))
25179 [8830][2] reports Exit
25180 Dec 04 13:36:03.785 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/2, method: GET, req_id: acf85301-0101-4dd6-8285-06abd4d5fb30, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
25181 001/300 Write block 2203 len 32768 data: 5 4 3 3 4 4 3 2
25182 002/300 Flush
25183 003/300 Read block 2791 len 16384
25184 004/300 Read block 4948 len 40960
25185 005/300 Read block 1471 len 24576
25186 006/300 Write block 1339 len 16384 data: 6 6 7 4
25187 007/300 Read block 2686 len 36864
25188 008/300 Read block 4509 len 28672
25189 009/300 Write block 3028 len 40960 data: 4 4 2 2 3 2 4 4 4 4
25190 010/300 Write block 215 len 36864 data: 4 4 3 3 3 3 3 2 3
25191 011/300 Read block 1099 len 4096
25192 012/300 Flush
25193 013/300 Read block 4034 len 36864
25194 014/300 Write block 1761 len 8192 data: 8 11
25195 015/300 Write block 1169 len 20480 data: 5 3 3 2 3
25196 016/300 Read block 729 len 8192
25197 017/300 Write block 4664 len 36864 data: 5 4 3 4 3 3 3 4 3
25198 018/300 Read block 1666 len 28672
25199 019/300 Read block 1838 len 4096
25200 020/300 Flush
25201 021/300 Read block 593 len 32768
25202 022/300 Read block 2970 len 36864
25203 023/300 Read block 4246 len 20480
25204 024/300 Read block 1005 len 32768
25205 025/300 Read block 4849 len 36864
25206 026/300 Flush
25207 027/300 Read block 2361 len 4096
25208 028/300 Read block 468 len 40960
25209 029/300 Read block 4990 len 32768
25210 030/300 Write block 4098 len 28672 data: 3 3 2 2 2 2 3
25211 031/300 Read block 3213 len 8192
25212 032/300 Flush
25213 033/300 Write block 179 len 36864 data: 4 3 3 2 2 2 3 5 5
25214 034/300 Read block 365 len 40960
25215 035/300 Flush
25216 036/300 Read block 3600 len 32768
25217 037/300 Read block 4008 len 8192
25218 038/300 Read block 3557 len 32768
25219 039/300 Flush
25220 040/300 Read block 3324 len 28672
25221 041/300 Flush
25222 042/300 Read block 1838 len 40960
25223 043/300 Read block 3682 len 8192
25224 044/300 Write block 3953 len 4096 data: 3
25225 045/300 Read block 3985 len 8192
25226 046/300 Flush
25227 047/300 Read block 3202 len 8192
25228 048/300 Write block 2049 len 36864 data: 6 6 5 5 6 6 6 6 6
25229 049/300 Read block 1120 len 28672
25230 050/300 Read block 4493 len 28672
25231 051/300 Read block 3908 len 4096
25232 052/300 Write block 4558 len 24576 data: 4 3 4 4 3 3
25233 053/300 Read block 4825 len 36864
25234 054/300 Flush
25235 055/300 Write block 3256 len 40960 data: 3 3 4 4 4 4 2 3 3 3
25236 056/300 Read block 4551 len 16384
25237 057/300 Read block 4282 len 40960
25238 058/300 Write block 4481 len 36864 data: 4 5 7 8 8 9 9 8 6
25239 059/300 Read block 3314 len 24576
25240 060/300 Read block 3275 len 36864
25241 061/300 Read block 2095 len 28672
25242 062/300 Read block 1105 len 20480
25243 063/300 Write block 2667 len 20480 data: 7 6 6 7 7
25244 064/300 Read block 1344 len 32768
25245 065/300 Read block 979 len 12288
25246 066/300 Read block 1300 len 36864
25247 067/300 Write block 482 len 40960 data: 5 5 4 6 7 6 6 7 6 6
25248 068/300 Read block 2450 len 8192
25249 069/300 Read block 1974 len 40960
25250 070/300 Flush
25251 071/300 Write block 4303 len 8192 data: 3 3
25252 072/300 Read block 1176 len 8192
25253 073/300 Read block 4672 len 24576
25254 074/300 Write block 3148 len 36864 data: 6 7 6 8 8 10 9 7 6
25255 075/300 Read block 4731 len 28672
25256 076/300 Write block 2613 len 24576 data: 2 2 3 4 4 4
25257 077/300 Read block 2282 len 16384
25258 078/300 Write block 2538 len 16384 data: 8 7 7 6
25259 079/300 Write block 96 len 40960 data: 4 3 3 5 5 5 6 5 4 4
25260 080/300 Write block 775 len 12288 data: 5 4 4
25261 081/300 Write block 821 len 16384 data: 5 5 5 7
25262 082/300 Write block 3471 len 4096 data: 4
25263 083/300 Write block 1784 len 28672 data: 5 4 4 5 5 3 3
25264 084/300 Read block 4923 len 8192
25265 085/300 Read block 3076 len 32768
25266 086/300 Read block 2156 len 20480
25267 087/300 Read block 230 len 12288
25268 088/300 Read block 3504 len 32768
25269 089/300 Flush
25270 090/300 Write block 3002 len 4096 data: 9
25271 091/300 Read block 164 len 8192
25272 092/300 Read block 1900 len 4096
25273 093/300 Read block 3684 len 8192
25274 094/300 Write block 4051 len 32768 data: 2 2 3 3 3 3 6 6
25275 095/300 Read block 954 len 32768
25276 096/300 Write block 2843 len 12288 data: 6 8 7
25277 097/300 Write block 1568 len 8192 data: 5 5
25278 098/300 Read block 3980 len 8192
25279 099/300 Write block 4679 len 28672 data: 7 6 4 3 4 5 6
25280 100/300 Read block 4057 len 32768
25281 101/300 Write block 3239 len 8192 data: 6 6
25282 102/300 Write block 4975 len 20480 data: 6 6 7 7 8
25283 103/300 Write block 2040 len 8192 data: 3 3
25284 104/300 Read block 2606 len 24576
25285 105/300 Read block 2234 len 36864
25286 106/300 Read block 2712 len 32768
25287 107/300 Read block 2333 len 32768
25288 108/300 Read block 4442 len 24576
25289 109/300 Read block 1819 len 16384
25290 110/300 Read block 3907 len 36864
25291 111/300 Read block 373 len 40960
25292 112/300 Flush
25293 113/300 Write block 4837 len 28672 data: 5 5 5 6 6 6 5
25294 114/300 Read block 3647 len 8192
25295 115/300 Write block 2281 len 16384 data: 5 5 5 5
25296 116/300 Read block 840 len 40960
25297 117/300 Read block 1510 len 4096
25298 118/300 Read block 2893 len 20480
25299 119/300 Read block 4463 len 20480
25300 120/300 Read block 3041 len 36864
25301 121/300 Read block 3086 len 4096
25302 122/300 Read block 1570 len 16384
25303 123/300 Read block 226 len 12288
25304 124/300 Write block 1956 len 12288 data: 4 3 3
25305 125/300 Write block 3180 len 40960 data: 5 5 5 4 2 2 2 2 3 3
25306 126/300 Flush
25307 127/300 Flush
25308 128/300 Read block 457 len 16384
25309 129/300 Flush
25310 130/300 Read block 3682 len 24576
25311 131/300 Write block 3856 len 28672 data: 4 4 3 4 5 5 5
25312 132/300 Read block 789 len 40960
25313 133/300 Read block 3817 len 40960
25314 134/300 Read block 4145 len 24576
25315 135/300 Read block 4286 len 32768
25316 136/300 Flush
25317 137/300 Write block 684 len 36864 data: 5 4 4 4 3 3 4 5 4
25318 138/300 Read block 3819 len 40960
25319 139/300 Read block 3310 len 20480
25320 140/300 Write block 3048 len 32768 data: 4 4 6 7 6 6 6 8
25321 141/300 Read block 1873 len 16384
25322 142/300 Read block 2717 len 32768
25323 143/300 Flush
25324 144/300 Write block 1833 len 36864 data: 4 4 5 6 5 7 8 7 7
25325 145/300 Write block 594 len 12288 data: 6 5 5
25326 146/300 Read block 4847 len 20480
25327 147/300 Read block 3317 len 32768
25328 148/300 Read block 143 len 20480
25329 149/300 Write block 4416 len 24576 data: 3 3 5 6 6 6
25330 150/300 Read block 1064 len 20480
25331 151/300 Write block 3642 len 36864 data: 3 2 3 4 4 4 3 3 3
25332 152/300 Write block 1199 len 40960 data: 4 4 5 6 6 6 6 6 6 6
25333 153/300 Read block 2621 len 24576
25334 154/300 Write block 47 len 8192 data: 5 4
25335 155/300 Write block 218 len 36864 data: 4 4 4 4 3 4 3 3 2
25336 156/300 Write block 3145 len 24576 data: 4 5 5 7 8 7
25337 157/300 Write block 4587 len 20480 data: 5 6 5 5 5
25338 158/300 Flush
25339 159/300 Write block 864 len 24576 data: 3 3 4 4 4 5
25340 160/300 Write block 142 len 8192 data: 3 3
25341 161/300 Read block 3889 len 16384
25342 162/300 Read block 2058 len 36864
25343 163/300 Write block 672 len 8192 data: 5 5
25344 164/300 Write block 4270 len 24576 data: 7 7 6 4 6 6
25345 165/300 Read block 3656 len 24576
25346 166/300 Write block 3597 len 4096 data: 3
25347 167/300 Read block 4767 len 8192
25348 168/300 Read block 2175 len 12288
25349 169/300 Read block 947 len 36864
25350 170/300 Read block 1422 len 36864
25351 171/300 Read block 3806 len 28672
25352 172/300 Read block 1482 len 4096
25353 173/300 Read block 969 len 4096
25354 174/300 Write block 2958 len 20480 data: 3 3 3 4 4
25355 175/300 Read block 648 len 32768
25356 176/300 Write block 2677 len 36864 data: 3 4 4 4 4 4 4 4 4
25357 177/300 Read block 4748 len 16384
25358 178/300 Read block 4405 len 20480
25359 179/300 Write block 3223 len 16384 data: 4 6 6 7
25360 180/300 Read block 3692 len 36864
25361 181/300 Flush
25362 182/300 Read block 472 len 28672
25363 183/300 Read block 360 len 4096
25364 184/300 Read block 2010 len 28672
25365 185/300 Read block 4669 len 24576
25366 186/300 Read block 451 len 24576
25367 187/300 Read block 1497 len 8192
25368 188/300 Write block 2456 len 8192 data: 5 6
25369 189/300 Flush
25370 190/300 Read block 4156 len 20480
25371 191/300 Write block 4697 len 16384 data: 6 6 5 5
25372 192/300 Read block 1253 len 40960
25373 193/300 Read block 1304 len 36864
25374 194/300 Write block 4437 len 40960 data: 5 5 6 6 6 7 6 5 4 4
25375 195/300 Flush
25376 196/300 Read block 1081 len 8192
25377 197/300 Write block 4343 len 36864 data: 7 7 5 5 4 5 5 5 6
25378 198/300 Read block 4007 len 28672
25379 199/300 Write block 2707 len 16384 data: 6 6 6 7
25380 200/300 Flush
25381 201/300 Write block 3123 len 40960 data: 4 4 5 6 6 5 5 5 4 3
25382 202/300 Write block 1450 len 28672 data: 4 5 5 6 7 9 9
25383 203/300 Write block 2397 len 24576 data: 6 6 6 6 5 6
25384 204/300 Write block 915 len 16384 data: 3 3 3 4
25385 205/300 Read block 3415 len 12288
25386 206/300 Write block 2709 len 20480 data: 7 8 7 7 4
25387 207/300 Write block 3841 len 4096 data: 3
25388 208/300 Read block 3519 len 4096
25389 209/300 Read block 328 len 12288
25390 210/300 Read block 801 len 20480
25391 211/300 Read block 3794 len 4096
25392 212/300 Write block 28 len 16384 data: 5 5 4 5
25393 213/300 Read block 740 len 12288
25394 214/300 Read block 2215 len 4096
25395 215/300 Read block 3032 len 28672
25396 216/300 Write block 2313 len 32768 data: 6 5 7 4 4 4 5 7
25397 217/300 Flush
25398 218/300 Flush
25399 219/300 Read block 3371 len 24576
25400 220/300 Read block 3997 len 4096
25401 221/300 Write block 2990 len 12288 data: 6 5 5
25402 222/300 Read block 2221 len 16384
25403 223/300 Write block 3158 len 4096 data: 6
25404 224/300 Read block 4868 len 16384
25405 225/300 Read block 2589 len 16384
25406 226/300 Read block 2862 len 12288
25407 227/300 Read block 2882 len 36864
25408 228/300 Write block 952 len 24576 data: 5 4 3 3 4 4
25409 229/300 Flush
25410 230/300 Write block 2106 len 24576 data: 6 6 6 7 7 5
25411 231/300 Read block 175 len 40960
25412 232/300 Write block 1769 len 16384 data: 10 9 6 4
25413 233/300 Read block 2418 len 40960
25414 234/300 Flush
25415 235/300 Write block 2287 len 24576 data: 6 5 6 8 9 10
25416 236/300 Read block 1151 len 16384
25417 237/300 Read block 2710 len 12288
25418 238/300 Read block 3636 len 12288
25419 239/300 Read block 4032 len 36864
25420 240/300 Read block 3674 len 40960
25421 241/300 Flush
25422 242/300 Write block 3346 len 24576 data: 4 6 6 7 6 5
25423 243/300 Write block 1295 len 24576 data: 6 6 6 7 5 5
25424 244/300 Flush
25425 245/300 Read block 4654 len 20480
25426 246/300 Write block 2638 len 32768 data: 8 9 9 8 7 7 7 8
25427 247/300 Flush
25428 248/300 Write block 4072 len 8192 data: 4 5
25429 249/300 Read block 1364 len 16384
25430 250/300 Read block 1873 len 24576
25431 251/300 Read block 2637 len 24576
25432 252/300 Read block 3515 len 28672
25433 253/300 Read block 1620 len 24576
25434 254/300 Flush
25435 255/300 Read block 2096 len 16384
25436 256/300 Read block 2117 len 12288
25437 257/300 Write block 3280 len 36864 data: 4 4 3 4 4 3 3 2 2
25438 258/300 Read block 1631 len 12288
25439 259/300 Read block 3131 len 20480
25440 260/300 Read block 4539 len 32768
25441 261/300 Read block 3184 len 20480
25442 262/300 Read block 4267 len 36864
25443 263/300 Read block 1814 len 8192
25444 264/300 Read block 2504 len 12288
25445 265/300 Flush
25446 266/300 Write block 1343 len 8192 data: 6 7
25447 267/300 Write block 3870 len 40960 data: 6 5 5 6 5 4 5 6 6 6
25448 268/300 Flush
25449 269/300 Read block 1853 len 32768
25450 270/300 Write block 4941 len 16384 data: 4 5 5 4
25451 271/300 Read block 832 len 40960
25452 272/300 Read block 3930 len 16384
25453 273/300 Read block 1629 len 12288
25454 274/300 Write block 4012 len 16384 data: 5 4 3 4
25455 275/300 Read block 4597 len 32768
25456 276/300 Write block 1555 len 8192 data: 5 4
25457 277/300 Read block 2067 len 16384
25458 278/300 Write block 3969 len 12288 data: 6 6 6
25459 279/300 Write block 139 len 16384 data: 5 3 3 4
25460 280/300 Read block 1184 len 36864
25461 281/300 Flush
25462 282/300 Write block 1509 len 8192 data: 5 7
25463 283/300 Read block 4079 len 16384
25464 284/300 Read block 4494 len 4096
25465 285/300 Read block 2988 len 12288
25466 286/300 Read block 3006 len 28672
25467 287/300 Write block 4994 len 12288 data: 3 4 4
25468 288/300 Write block 644 len 20480 data: 7 7 7 7 5
25469 289/300 Write block 1064 len 28672 data: 5 7 8 8 9 9 11
25470 290/300 Read block 1152 len 32768
25471 291/300 Flush
25472 292/300 Write block 1393 len 12288 data: 3 3 3
25473 293/300 Flush
25474 294/300 Read block 4864 len 4096
25475 295/300 Write block 1468 len 16384 data: 4 4 7 9
25476 296/300 Write block 2445 len 32768 data: 4 4 3 4 3 2 2 3
25477 297/300 Read block 2410 len 8192
25478 298/300 Write block 3780 len 4096 data: 5
25479 299/300 Write block 2271 len 20480 data: 6 5 7 6 7
25480 300/300 Write block 2981 len 32768 data: 6 6 7 8 8 7 6 7
25481 301/300 Flush
25482 Main task has work to do, go find it
25483 got dsc Start(2)
25484 start 2
25485 Dec 04 13:36:04.032 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/2, method: POST, req_id: e4dac6f4-e169-4ae7-8132-c556aef5f4aa, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
25486 Replay: started 2, returned:Ok(())
25487 ----------------------------------------------------------------
25488 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25489 States: Active Active Offline
25490 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25491 7943 Acked Write 8 Done Done Sent false
25492 7944 Acked Flush 0 Done Done Sent false
25493 7945 Acked Read 4 Done Done Sent false
25494 7946 Acked Read 10 Done Done Sent false
25495 7947 Acked Read 6 Done Done Sent false
25496 7948 Acked Write 4 Done Done Sent false
25497 7949 Acked Read 9 Done Done Sent false
25498 7950 Acked Read 7 Done Done Sent false
25499 7951 Acked Write 10 Done Done Sent false
25500 7952 Acked Write 9 Done Done Sent false
25501 7953 Acked Read 1 Done Done Sent false
25502 7954 Acked Flush 0 Done Done Sent false
25503 7955 Acked Read 9 Done Done Sent false
25504 7956 Acked Write 2 Done Done Sent false
25505 7957 Acked Write 5 Done Done Sent false
25506 7958 Acked Read 2 Done Done Sent false
25507 7959 Acked Write 9 Done Done Sent false
25508 7960 Acked Read 7 Done Done Sent false
25509 7961 Acked Read 1 Done Done Sent false
25510 7962 Acked Flush 0 Done Done Sent false
25511 7963 Acked Read 8 Done Done Sent false
25512 7964 Acked Read 9 Done Done Sent false
25513 7965 Acked Read 5 Done Done Sent false
25514 7966 Acked Read 8 Done Done Sent false
25515 7967 Acked Read 9 Done Done Sent false
25516 7968 Acked Flush 0 Done Done Sent false
25517 7969 Acked Read 1 Done Done Sent false
25518 7970 Acked Read 10 Done Done Sent false
25519 7971 Acked Read 8 Done Done Sent false
25520 7972 Acked Write 7 Done Done Sent false
25521 7973 Acked Read 2 Done Done Sent false
25522 7974 Acked Flush 0 Done Done Sent false
25523 7975 Acked Write 9 Done Done Sent false
25524 7976 Acked Read 10 Done Done Sent false
25525 7977 Acked Flush 0 Done Done Sent false
25526 7978 Acked Read 8 Done Done Sent false
25527 7979 Acked Read 2 Done Done Sent false
25528 7980 Acked Read 8 Done Done Sent false
25529 7981 Acked Flush 0 Done Done Sent false
25530 7982 Acked Read 7 Done Done Sent false
25531 7983 Acked Flush 0 Done Done Sent false
25532 7984 Acked Read 10 Done Done Sent false
25533 7985 Acked Read 2 Done Done Sent false
25534 7986 Acked Write 1 Done Done Sent false
25535 7987 Acked Read 2 Done Done Sent false
25536 7988 Acked Flush 0 Done Done Sent false
25537 7989 Acked Read 2 Done Done Sent false
25538 7990 Acked Write 9 Done Done Sent false
25539 7991 Acked Read 7 Done Done Sent false
25540 7992 Acked Read 7 Done Done Sent false
25541 7993 Acked Read 1 Done Done Sent false
25542 7994 Acked Write 6 Done Done Sent false
25543 7995 Acked Read 9 Done Done Sent false
25544 7996 Acked Flush 0 Done Done Sent false
25545 7997 Acked Write 10 Done Done Sent false
25546 7998 Acked Read 4 Done Done Sent false
25547 7999 Acked Read 10 Done Done Sent false
25548 8000 Acked Write 9 Done Done Sent false
25549 8001 Acked Read 6 Done Done Sent false
25550 8002 Acked Read 9 Done Done Sent false
25551 8003 Acked Read 7 Done Done Sent false
25552 8004 Acked Read 5 Done Done Sent false
25553 8005 Acked Write 5 Done Done Sent false
25554 8006 Acked Read 8 Done Done Sent false
25555 8007 Acked Read 3 Done Done Sent false
25556 8008 Acked Read 9 Done Done Sent false
25557 8009 Acked Write 10 Done Done Sent false
25558 8010 Acked Read 2 Done Done Sent false
25559 8011 Acked Read 10 Done Done Sent false
25560 8012 Acked Flush 0 Done Done Sent false
25561 8013 Acked Write 2 Done Done Sent false
25562 8014 Acked Read 2 Done Done Sent false
25563 8015 Acked Read 6 Done Done Sent false
25564 8016 Acked Write 9 Done Done Sent false
25565 8017 Acked Read 7 Done Done Sent false
25566 8018 Acked Write 6 Done Done Sent false
25567 8019 Acked Read 4 Done Done Sent false
25568 8020 Acked Write 4 Done Done Sent false
25569 8021 Acked Write 10 Done Done Sent false
25570 8022 Acked Write 3 Done Done Sent false
25571 8023 Acked Write 4 Done Done Sent false
25572 8024 Acked Write 1 Done Done Sent false
25573 8025 Acked Write 7 Done Done Sent false
25574 8026 Acked Read 2 Done Done Sent false
25575 8027 Acked Read 8 Done Done Sent false
25576 8028 Acked Read 5 Done Done Sent false
25577 8029 Acked Read 3 Done Done Sent false
25578 8030 Acked Read 8 Done Done Sent false
25579 8031 Acked Flush 0 Done Done Sent false
25580 8032 Acked Write 1 Done Done Sent false
25581 8033 Acked Read 2 Done Done Sent false
25582 8034 Acked Read 1 Done Done Sent false
25583 8035 Acked Read 2 Done Done Sent false
25584 8036 Acked Write 8 Done Done Sent false
25585 8037 Acked Read 8 Done Done Sent false
25586 8038 Acked Write 3 Done Done Sent false
25587 8039 Acked Write 2 Done Done Sent false
25588 8040 Acked Read 2 Done Done Sent false
25589 8041 Acked Write 7 Done Done Sent false
25590 8042 Acked Read 8 Done Done Sent false
25591 8043 Acked Write 2 Done Done Sent false
25592 8044 Acked Write 5 Done Done Sent false
25593 8045 Acked Write 2 Done Done Sent false
25594 8046 Acked Read 6 Done Done Sent false
25595 8047 Acked Read 9 Done Done Sent false
25596 8048 Acked Read 8 Done Done Sent false
25597 8049 Acked Read 8 Done Done Sent false
25598 8050 Acked Read 6 Done Done Sent false
25599 8051 Acked Read 4 Done Done Sent false
25600 8052 Acked Read 9 Done Done Sent false
25601 8053 Acked Read 10 Done Done Sent false
25602 8054 Acked Flush 0 Done Done Sent false
25603 8055 Acked Write 7 Done Done Sent false
25604 8056 Acked Read 2 Done Done Sent false
25605 8057 Acked Write 4 Done Done Sent false
25606 8058 Acked Read 10 Done Done Sent false
25607 8059 Acked Read 1 Done Done Sent false
25608 8060 Acked Read 5 Done Done Sent false
25609 8061 Acked Read 5 Done Done Sent false
25610 8062 Acked Read 9 Done Done Sent false
25611 8063 Acked Read 1 Done Done Sent false
25612 8064 Acked Read 4 Done Done Sent false
25613 8065 Acked Read 3 Done Done Sent false
25614 8066 Acked Write 3 Done Done Sent false
25615 8067 Acked Write 10 Done Done Sent false
25616 8068 Acked Flush 0 Done Done Sent false
25617 8069 Acked Flush 0 Done Done Sent false
25618 8070 Acked Read 4 Done Done Sent false
25619 8071 Acked Flush 0 Done Done Sent false
25620 8072 Acked Read 6 Done Done Sent false
25621 8073 Acked Write 7 Done Done Sent false
25622 8074 Acked Read 10 Done Done Sent false
25623 8075 Acked Read 10 Done Done Sent false
25624 8076 Acked Read 6 Done Done Sent false
25625 8077 Acked Read 8 Done Done Sent false
25626 8078 Acked Flush 0 Done Done Sent false
25627 8079 Acked Write 9 Done Done Sent false
25628 8080 Acked Read 10 Done Done Sent false
25629 8081 Acked Read 5 Done Done Sent false
25630 8082 Acked Write 8 Done Done Sent false
25631 8083 Acked Read 4 Done Done Sent false
25632 8084 Acked Read 8 Done Done Sent false
25633 8085 Acked Flush 0 Done Done Sent false
25634 8086 Acked Write 9 Done Done Sent false
25635 8087 Acked Write 3 Done Done Sent false
25636 8088 Acked Read 5 Done Done Sent false
25637 8089 Acked Read 8 Done Done Sent false
25638 8090 Acked Read 5 Done Done Sent false
25639 8091 Acked Write 6 Done Done Sent false
25640 8092 Acked Read 5 Done Done Sent false
25641 8093 Acked Write 9 Done Done Sent false
25642 8094 Acked Write 10 Done Done Sent false
25643 8095 Acked Read 6 Done Done Sent false
25644 8096 Acked Write 2 Done Done Sent false
25645 8097 Acked Write 9 Done Done Sent false
25646 8098 Acked Write 6 Done Done Sent false
25647 8099 Acked Write 5 Done Done Sent false
25648 8100 Acked Flush 0 Done Done Sent false
25649 8101 Acked Write 6 Done Done Sent false
25650 8102 Acked Write 2 Done Done Sent false
25651 8103 Acked Read 4 Done Done Sent false
25652 8104 Acked Read 9 Done Done Sent false
25653 8105 Acked Write 2 Done Done Sent false
25654 8106 Acked Write 6 Done Done Sent false
25655 8107 Acked Read 6 Done Done Sent false
25656 8108 Acked Write 1 Done Done Sent false
25657 8109 Acked Read 2 Done Done Sent false
25658 8110 Acked Read 3 Done Done Sent false
25659 8111 Acked Read 9 Done Done Sent false
25660 8112 Acked Read 9 Done Done Sent false
25661 8113 Acked Read 7 Done Done Sent false
25662 8114 Acked Read 1 Done Done Sent false
25663 8115 Acked Read 1 Done Done Sent false
25664 8116 Acked Write 5 Done Done Sent false
25665 8117 Acked Read 8 Done Done Sent false
25666 8118 Acked Write 9 Done Done Sent false
25667 8119 Acked Read 4 Done Done Sent false
25668 8120 Acked Read 5 Done Done Sent false
25669 8121 Acked Write 4 Done Done Sent false
25670 8122 Acked Read 9 Done Done Sent false
25671 8123 Acked Flush 0 Done Done Sent false
25672 8124 Acked Read 7 Done Done Sent false
25673 8125 Acked Read 1 Done Done Sent false
25674 8126 Acked Read 7 Done Done Sent false
25675 8127 Acked Read 6 Done Done Sent false
25676 8128 Acked Read 6 Done Done Sent false
25677 8129 Acked Read 2 Done Done Sent false
25678 8130 Acked Write 2 Done Done Sent false
25679 8131 Acked Flush 0 Done Done Sent false
25680 8132 Acked Read 5 Done Done Sent false
25681 8133 Acked Write 4 Done Done Sent false
25682 8134 Acked Read 10 Done Done Sent false
25683 8135 Acked Read 9 Done Done Sent false
25684 8136 Acked Write 10 Done Done Sent false
25685 8137 Acked Flush 0 Done Done Sent false
25686 8138 Acked Read 2 Done Done Sent false
25687 8139 Acked Write 9 Done Done Sent false
25688 8140 Acked Read 7 Done Done Sent false
25689 8141 Acked Write 4 Done Done Sent false
25690 8142 Acked Flush 0 Done Done Sent false
25691 8143 Acked Write 10 Done Done Sent false
25692 8144 Acked Write 7 Done Done Sent false
25693 8145 Acked Write 6 Done Done Sent false
25694 8146 Acked Write 4 Done Done Sent false
25695 8147 Acked Read 3 Done Done Sent false
25696 8148 Acked Write 5 Done Done Sent false
25697 8149 Acked Write 1 Done Done Sent false
25698 8150 Acked Read 1 Done Done Sent false
25699 8151 Acked Read 3 Done Done Sent false
25700 8152 Acked Read 5 Done Done Sent false
25701 8153 Acked Read 1 Done Done Sent false
25702 8154 Acked Write 4 Done Done Sent false
25703 8155 Acked Read 3 Done Done Sent false
25704 8156 Acked Read 1 Done Done Sent false
25705 8157 Acked Read 7 Done Done Sent false
25706 8158 Acked Write 8 Done Done Sent false
25707 8159 Acked Flush 0 Done Done Sent false
25708 8160 Acked Flush 0 Done Done Sent false
25709 8161 Acked Read 6 Done Done Sent false
25710 8162 Acked Read 1 Done Done Sent false
25711 8163 Acked Write 3 Done Done Sent false
25712 8164 Acked Read 4 Done Done Sent false
25713 8165 Acked Write 1 Done Done Sent false
25714 8166 Acked Read 4 Done Done Sent false
25715 8167 Acked Read 4 Done Done Sent false
25716 8168 Acked Read 3 Done Done Sent false
25717 8169 Acked Read 9 Done Done Sent false
25718 8170 Acked Write 6 Done Done Sent false
25719 8171 Acked Flush 0 Done Done Sent false
25720 8172 Acked Write 6 Done Done Sent false
25721 8173 Acked Read 10 Done Done Sent false
25722 8174 Acked Write 4 Done Done Sent false
25723 8175 Acked Read 10 Done Done Sent false
25724 8176 Acked Flush 0 Done Done Sent false
25725 8177 Acked Write 6 Done Done Sent false
25726 8178 Acked Read 4 Done Done Sent false
25727 8179 Acked Read 3 Done Done Sent false
25728 8180 Acked Read 3 Done Done Sent false
25729 8181 Acked Read 9 Done Done Sent false
25730 8182 Acked Read 10 Done Done Sent false
25731 8183 Acked Flush 0 Done Done Sent false
25732 8184 Acked Write 6 Done Done Sent false
25733 8185 Acked Write 6 Done Done Sent false
25734 8186 Acked Flush 0 Done Done Sent false
25735 8187 Acked Read 5 Done Done Sent false
25736 8188 Acked Write 8 Done Done Sent false
25737 8189 Acked Flush 0 Done Done Sent false
25738 8190 Acked Write 2 Done Done Sent false
25739 8191 Acked Read 4 Done Done Sent false
25740 8192 Acked Read 6 Done Done Sent false
25741 8193 Acked Read 6 Done Done Sent false
25742 8194 Acked Read 7 Done Done Sent false
25743 8195 Acked Read 6 Done Done Sent false
25744 8196 Acked Flush 0 Done Done Sent false
25745 8197 Acked Read 4 Done Done Sent false
25746 8198 Acked Read 3 Done Done Sent false
25747 8199 Acked Write 9 Done Done Sent false
25748 8200 Acked Read 3 Done Done Sent false
25749 8201 Acked Read 5 Done Done Sent false
25750 8202 Acked Read 8 Done Done Sent false
25751 8203 Acked Read 5 Done Done Sent false
25752 8204 Acked Read 9 Done Done Sent false
25753 8205 Acked Read 2 Done Done Sent false
25754 8206 Acked Read 3 Done Done Sent false
25755 8207 Acked Flush 0 Done Done Sent false
25756 8208 Acked Write 2 Done Done Sent false
25757 8209 Acked Write 10 Done Done Sent false
25758 8210 Acked Flush 0 Done Done Sent false
25759 8211 Acked Read 8 Done Done Sent false
25760 8212 Acked Write 4 Done Done Sent false
25761 8213 Acked Read 10 Done Done Sent false
25762 8214 Acked Read 4 Done Done Sent false
25763 8215 Acked Read 3 Done Done Sent false
25764 8216 Acked Write 4 Done Done Sent false
25765 8217 Acked Read 8 Done Done Sent false
25766 8218 Acked Write 2 Done Done Sent false
25767 8219 Acked Read 4 Done Done Sent false
25768 8220 Acked Write 3 Done Done Sent false
25769 8221 Acked Write 4 Done Done Sent false
25770 8222 Acked Read 9 Done Done Sent false
25771 8223 Acked Flush 0 Done Done Sent false
25772 8224 Acked Write 2 Done Done Sent false
25773 8225 Acked Read 4 Done Done Sent false
25774 8226 Acked Read 1 Done Done Sent false
25775 8227 Acked Read 3 Done Done Sent false
25776 8228 Acked Read 7 Done Done Sent false
25777 8229 Acked Write 3 Done Done Sent false
25778 8230 Acked Write 5 Done Done Sent false
25779 8231 Acked Write 7 Done Done Sent false
25780 8232 Acked Read 8 Done Done Sent false
25781 8233 Acked Flush 0 Done Done Sent false
25782 8234 Acked Write 3 Done Done Sent false
25783 8235 Acked Flush 0 Done Done Sent false
25784 8236 Acked Read 1 Done Done Sent false
25785 8237 Acked Write 4 Done Done Sent false
25786 8238 Acked Write 8 Done Done Sent false
25787 8239 Acked Read 2 Done Done Sent false
25788 8240 Acked Write 1 Done Done Sent false
25789 8241 Acked Write 5 Done Done Sent false
25790 8242 Acked Write 8 Done Done Sent false
25791 8243 Acked Flush 0 Done Done Sent false
25792 STATES DS:0 DS:1 DS:2 TOTAL
25793 Sent 0 0 301 301
25794 Done 301 301 0 602
25795 Skipped 0 0 0 0
25796 Error 0 0 0 0
25797 Last Flush: 8243 8243 7942
25798 Downstairs last five completed: 7942 7941 7940 7939 7938
25799 Upstairs last five completed: 8243 8242 8241 8240 8239
25800 CLIENT: Up:0 ds:301 act:2
25801 [8830] got start action
25802 [8830] I am going to restart
25803 Starting downstairs at port 8830
25804 Make output file at "/tmp/dsc/downstairs-8830.txt"
25805 [8830][2] reports Starting
25806 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1227)
25807 [8830][2] reports Running
25808 ----------------------------------------------------------------
25809 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25810 States: Active Active Offline
25811 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25812 7943 Acked Write 8 Done Done Sent false
25813 7944 Acked Flush 0 Done Done Sent false
25814 7945 Acked Read 4 Done Done Sent false
25815 7946 Acked Read 10 Done Done Sent false
25816 7947 Acked Read 6 Done Done Sent false
25817 7948 Acked Write 4 Done Done Sent false
25818 7949 Acked Read 9 Done Done Sent false
25819 7950 Acked Read 7 Done Done Sent false
25820 7951 Acked Write 10 Done Done Sent false
25821 7952 Acked Write 9 Done Done Sent false
25822 7953 Acked Read 1 Done Done Sent false
25823 7954 Acked Flush 0 Done Done Sent false
25824 7955 Acked Read 9 Done Done Sent false
25825 7956 Acked Write 2 Done Done Sent false
25826 7957 Acked Write 5 Done Done Sent false
25827 7958 Acked Read 2 Done Done Sent false
25828 7959 Acked Write 9 Done Done Sent false
25829 7960 Acked Read 7 Done Done Sent false
25830 7961 Acked Read 1 Done Done Sent false
25831 7962 Acked Flush 0 Done Done Sent false
25832 7963 Acked Read 8 Done Done Sent false
25833 7964 Acked Read 9 Done Done Sent false
25834 7965 Acked Read 5 Done Done Sent false
25835 7966 Acked Read 8 Done Done Sent false
25836 7967 Acked Read 9 Done Done Sent false
25837 7968 Acked Flush 0 Done Done Sent false
25838 7969 Acked Read 1 Done Done Sent false
25839 7970 Acked Read 10 Done Done Sent false
25840 7971 Acked Read 8 Done Done Sent false
25841 7972 Acked Write 7 Done Done Sent false
25842 7973 Acked Read 2 Done Done Sent false
25843 7974 Acked Flush 0 Done Done Sent false
25844 7975 Acked Write 9 Done Done Sent false
25845 7976 Acked Read 10 Done Done Sent false
25846 7977 Acked Flush 0 Done Done Sent false
25847 7978 Acked Read 8 Done Done Sent false
25848 7979 Acked Read 2 Done Done Sent false
25849 7980 Acked Read 8 Done Done Sent false
25850 7981 Acked Flush 0 Done Done Sent false
25851 7982 Acked Read 7 Done Done Sent false
25852 7983 Acked Flush 0 Done Done Sent false
25853 7984 Acked Read 10 Done Done Sent false
25854 7985 Acked Read 2 Done Done Sent false
25855 7986 Acked Write 1 Done Done Sent false
25856 7987 Acked Read 2 Done Done Sent false
25857 7988 Acked Flush 0 Done Done Sent false
25858 7989 Acked Read 2 Done Done Sent false
25859 7990 Acked Write 9 Done Done Sent false
25860 7991 Acked Read 7 Done Done Sent false
25861 7992 Acked Read 7 Done Done Sent false
25862 7993 Acked Read 1 Done Done Sent false
25863 7994 Acked Write 6 Done Done Sent false
25864 7995 Acked Read 9 Done Done Sent false
25865 7996 Acked Flush 0 Done Done Sent false
25866 7997 Acked Write 10 Done Done Sent false
25867 7998 Acked Read 4 Done Done Sent false
25868 7999 Acked Read 10 Done Done Sent false
25869 8000 Acked Write 9 Done Done Sent false
25870 8001 Acked Read 6 Done Done Sent false
25871 8002 Acked Read 9 Done Done Sent false
25872 8003 Acked Read 7 Done Done Sent false
25873 8004 Acked Read 5 Done Done Sent false
25874 8005 Acked Write 5 Done Done Sent false
25875 8006 Acked Read 8 Done Done Sent false
25876 8007 Acked Read 3 Done Done Sent false
25877 8008 Acked Read 9 Done Done Sent false
25878 8009 Acked Write 10 Done Done Sent false
25879 8010 Acked Read 2 Done Done Sent false
25880 8011 Acked Read 10 Done Done Sent false
25881 8012 Acked Flush 0 Done Done Sent false
25882 8013 Acked Write 2 Done Done Sent false
25883 8014 Acked Read 2 Done Done Sent false
25884 8015 Acked Read 6 Done Done Sent false
25885 8016 Acked Write 9 Done Done Sent false
25886 8017 Acked Read 7 Done Done Sent false
25887 8018 Acked Write 6 Done Done Sent false
25888 8019 Acked Read 4 Done Done Sent false
25889 8020 Acked Write 4 Done Done Sent false
25890 8021 Acked Write 10 Done Done Sent false
25891 8022 Acked Write 3 Done Done Sent false
25892 8023 Acked Write 4 Done Done Sent false
25893 8024 Acked Write 1 Done Done Sent false
25894 8025 Acked Write 7 Done Done Sent false
25895 8026 Acked Read 2 Done Done Sent false
25896 8027 Acked Read 8 Done Done Sent false
25897 8028 Acked Read 5 Done Done Sent false
25898 8029 Acked Read 3 Done Done Sent false
25899 8030 Acked Read 8 Done Done Sent false
25900 8031 Acked Flush 0 Done Done Sent false
25901 8032 Acked Write 1 Done Done Sent false
25902 8033 Acked Read 2 Done Done Sent false
25903 8034 Acked Read 1 Done Done Sent false
25904 8035 Acked Read 2 Done Done Sent false
25905 8036 Acked Write 8 Done Done Sent false
25906 8037 Acked Read 8 Done Done Sent false
25907 8038 Acked Write 3 Done Done Sent false
25908 8039 Acked Write 2 Done Done Sent false
25909 8040 Acked Read 2 Done Done Sent false
25910 8041 Acked Write 7 Done Done Sent false
25911 8042 Acked Read 8 Done Done Sent false
25912 8043 Acked Write 2 Done Done Sent false
25913 8044 Acked Write 5 Done Done Sent false
25914 8045 Acked Write 2 Done Done Sent false
25915 8046 Acked Read 6 Done Done Sent false
25916 8047 Acked Read 9 Done Done Sent false
25917 8048 Acked Read 8 Done Done Sent false
25918 8049 Acked Read 8 Done Done Sent false
25919 8050 Acked Read 6 Done Done Sent false
25920 8051 Acked Read 4 Done Done Sent false
25921 8052 Acked Read 9 Done Done Sent false
25922 8053 Acked Read 10 Done Done Sent false
25923 8054 Acked Flush 0 Done Done Sent false
25924 8055 Acked Write 7 Done Done Sent false
25925 8056 Acked Read 2 Done Done Sent false
25926 8057 Acked Write 4 Done Done Sent false
25927 8058 Acked Read 10 Done Done Sent false
25928 8059 Acked Read 1 Done Done Sent false
25929 8060 Acked Read 5 Done Done Sent false
25930 8061 Acked Read 5 Done Done Sent false
25931 8062 Acked Read 9 Done Done Sent false
25932 8063 Acked Read 1 Done Done Sent false
25933 8064 Acked Read 4 Done Done Sent false
25934 8065 Acked Read 3 Done Done Sent false
25935 8066 Acked Write 3 Done Done Sent false
25936 8067 Acked Write 10 Done Done Sent false
25937 8068 Acked Flush 0 Done Done Sent false
25938 8069 Acked Flush 0 Done Done Sent false
25939 8070 Acked Read 4 Done Done Sent false
25940 8071 Acked Flush 0 Done Done Sent false
25941 8072 Acked Read 6 Done Done Sent false
25942 8073 Acked Write 7 Done Done Sent false
25943 8074 Acked Read 10 Done Done Sent false
25944 8075 Acked Read 10 Done Done Sent false
25945 8076 Acked Read 6 Done Done Sent false
25946 8077 Acked Read 8 Done Done Sent false
25947 8078 Acked Flush 0 Done Done Sent false
25948 8079 Acked Write 9 Done Done Sent false
25949 8080 Acked Read 10 Done Done Sent false
25950 8081 Acked Read 5 Done Done Sent false
25951 8082 Acked Write 8 Done Done Sent false
25952 8083 Acked Read 4 Done Done Sent false
25953 8084 Acked Read 8 Done Done Sent false
25954 8085 Acked Flush 0 Done Done Sent false
25955 8086 Acked Write 9 Done Done Sent false
25956 8087 Acked Write 3 Done Done Sent false
25957 8088 Acked Read 5 Done Done Sent false
25958 8089 Acked Read 8 Done Done Sent false
25959 8090 Acked Read 5 Done Done Sent false
25960 8091 Acked Write 6 Done Done Sent false
25961 8092 Acked Read 5 Done Done Sent false
25962 8093 Acked Write 9 Done Done Sent false
25963 8094 Acked Write 10 Done Done Sent false
25964 8095 Acked Read 6 Done Done Sent false
25965 8096 Acked Write 2 Done Done Sent false
25966 8097 Acked Write 9 Done Done Sent false
25967 8098 Acked Write 6 Done Done Sent false
25968 8099 Acked Write 5 Done Done Sent false
25969 8100 Acked Flush 0 Done Done Sent false
25970 8101 Acked Write 6 Done Done Sent false
25971 8102 Acked Write 2 Done Done Sent false
25972 8103 Acked Read 4 Done Done Sent false
25973 8104 Acked Read 9 Done Done Sent false
25974 8105 Acked Write 2 Done Done Sent false
25975 8106 Acked Write 6 Done Done Sent false
25976 8107 Acked Read 6 Done Done Sent false
25977 8108 Acked Write 1 Done Done Sent false
25978 8109 Acked Read 2 Done Done Sent false
25979 8110 Acked Read 3 Done Done Sent false
25980 8111 Acked Read 9 Done Done Sent false
25981 8112 Acked Read 9 Done Done Sent false
25982 8113 Acked Read 7 Done Done Sent false
25983 8114 Acked Read 1 Done Done Sent false
25984 8115 Acked Read 1 Done Done Sent false
25985 8116 Acked Write 5 Done Done Sent false
25986 8117 Acked Read 8 Done Done Sent false
25987 8118 Acked Write 9 Done Done Sent false
25988 8119 Acked Read 4 Done Done Sent false
25989 8120 Acked Read 5 Done Done Sent false
25990 8121 Acked Write 4 Done Done Sent false
25991 8122 Acked Read 9 Done Done Sent false
25992 8123 Acked Flush 0 Done Done Sent false
25993 8124 Acked Read 7 Done Done Sent false
25994 8125 Acked Read 1 Done Done Sent false
25995 8126 Acked Read 7 Done Done Sent false
25996 8127 Acked Read 6 Done Done Sent false
25997 8128 Acked Read 6 Done Done Sent false
25998 8129 Acked Read 2 Done Done Sent false
25999 8130 Acked Write 2 Done Done Sent false
26000 8131 Acked Flush 0 Done Done Sent false
26001 8132 Acked Read 5 Done Done Sent false
26002 8133 Acked Write 4 Done Done Sent false
26003 8134 Acked Read 10 Done Done Sent false
26004 8135 Acked Read 9 Done Done Sent false
26005 8136 Acked Write 10 Done Done Sent false
26006 8137 Acked Flush 0 Done Done Sent false
26007 8138 Acked Read 2 Done Done Sent false
26008 8139 Acked Write 9 Done Done Sent false
26009 8140 Acked Read 7 Done Done Sent false
26010 8141 Acked Write 4 Done Done Sent false
26011 8142 Acked Flush 0 Done Done Sent false
26012 8143 Acked Write 10 Done Done Sent false
26013 8144 Acked Write 7 Done Done Sent false
26014 8145 Acked Write 6 Done Done Sent false
26015 8146 Acked Write 4 Done Done Sent false
26016 8147 Acked Read 3 Done Done Sent false
26017 8148 Acked Write 5 Done Done Sent false
26018 8149 Acked Write 1 Done Done Sent false
26019 8150 Acked Read 1 Done Done Sent false
26020 8151 Acked Read 3 Done Done Sent false
26021 8152 Acked Read 5 Done Done Sent false
26022 8153 Acked Read 1 Done Done Sent false
26023 8154 Acked Write 4 Done Done Sent false
26024 8155 Acked Read 3 Done Done Sent false
26025 8156 Acked Read 1 Done Done Sent false
26026 8157 Acked Read 7 Done Done Sent false
26027 8158 Acked Write 8 Done Done Sent false
26028 8159 Acked Flush 0 Done Done Sent false
26029 8160 Acked Flush 0 Done Done Sent false
26030 8161 Acked Read 6 Done Done Sent false
26031 8162 Acked Read 1 Done Done Sent false
26032 8163 Acked Write 3 Done Done Sent false
26033 8164 Acked Read 4 Done Done Sent false
26034 8165 Acked Write 1 Done Done Sent false
26035 8166 Acked Read 4 Done Done Sent false
26036 8167 Acked Read 4 Done Done Sent false
26037 8168 Acked Read 3 Done Done Sent false
26038 8169 Acked Read 9 Done Done Sent false
26039 8170 Acked Write 6 Done Done Sent false
26040 8171 Acked Flush 0 Done Done Sent false
26041 8172 Acked Write 6 Done Done Sent false
26042 8173 Acked Read 10 Done Done Sent false
26043 8174 Acked Write 4 Done Done Sent false
26044 8175 Acked Read 10 Done Done Sent false
26045 8176 Acked Flush 0 Done Done Sent false
26046 8177 Acked Write 6 Done Done Sent false
26047 8178 Acked Read 4 Done Done Sent false
26048 8179 Acked Read 3 Done Done Sent false
26049 8180 Acked Read 3 Done Done Sent false
26050 8181 Acked Read 9 Done Done Sent false
26051 8182 Acked Read 10 Done Done Sent false
26052 8183 Acked Flush 0 Done Done Sent false
26053 8184 Acked Write 6 Done Done Sent false
26054 8185 Acked Write 6 Done Done Sent false
26055 8186 Acked Flush 0 Done Done Sent false
26056 8187 Acked Read 5 Done Done Sent false
26057 8188 Acked Write 8 Done Done Sent false
26058 8189 Acked Flush 0 Done Done Sent false
26059 8190 Acked Write 2 Done Done Sent false
26060 8191 Acked Read 4 Done Done Sent false
26061 8192 Acked Read 6 Done Done Sent false
26062 8193 Acked Read 6 Done Done Sent false
26063 8194 Acked Read 7 Done Done Sent false
26064 8195 Acked Read 6 Done Done Sent false
26065 8196 Acked Flush 0 Done Done Sent false
26066 8197 Acked Read 4 Done Done Sent false
26067 8198 Acked Read 3 Done Done Sent false
26068 8199 Acked Write 9 Done Done Sent false
26069 8200 Acked Read 3 Done Done Sent false
26070 8201 Acked Read 5 Done Done Sent false
26071 8202 Acked Read 8 Done Done Sent false
26072 8203 Acked Read 5 Done Done Sent false
26073 8204 Acked Read 9 Done Done Sent false
26074 8205 Acked Read 2 Done Done Sent false
26075 8206 Acked Read 3 Done Done Sent false
26076 8207 Acked Flush 0 Done Done Sent false
26077 8208 Acked Write 2 Done Done Sent false
26078 8209 Acked Write 10 Done Done Sent false
26079 8210 Acked Flush 0 Done Done Sent false
26080 8211 Acked Read 8 Done Done Sent false
26081 8212 Acked Write 4 Done Done Sent false
26082 8213 Acked Read 10 Done Done Sent false
26083 8214 Acked Read 4 Done Done Sent false
26084 8215 Acked Read 3 Done Done Sent false
26085 8216 Acked Write 4 Done Done Sent false
26086 8217 Acked Read 8 Done Done Sent false
26087 8218 Acked Write 2 Done Done Sent false
26088 8219 Acked Read 4 Done Done Sent false
26089 8220 Acked Write 3 Done Done Sent false
26090 8221 Acked Write 4 Done Done Sent false
26091 8222 Acked Read 9 Done Done Sent false
26092 8223 Acked Flush 0 Done Done Sent false
26093 8224 Acked Write 2 Done Done Sent false
26094 8225 Acked Read 4 Done Done Sent false
26095 8226 Acked Read 1 Done Done Sent false
26096 8227 Acked Read 3 Done Done Sent false
26097 8228 Acked Read 7 Done Done Sent false
26098 8229 Acked Write 3 Done Done Sent false
26099 8230 Acked Write 5 Done Done Sent false
26100 8231 Acked Write 7 Done Done Sent false
26101 8232 Acked Read 8 Done Done Sent false
26102 8233 Acked Flush 0 Done Done Sent false
26103 8234 Acked Write 3 Done Done Sent false
26104 8235 Acked Flush 0 Done Done Sent false
26105 8236 Acked Read 1 Done Done Sent false
26106 8237 Acked Write 4 Done Done Sent false
26107 8238 Acked Write 8 Done Done Sent false
26108 8239 Acked Read 2 Done Done Sent false
26109 8240 Acked Write 1 Done Done Sent false
26110 8241 Acked Write 5 Done Done Sent false
26111 8242 Acked Write 8 Done Done Sent false
26112 8243 Acked Flush 0 Done Done Sent false
26113 STATES DS:0 DS:1 DS:2 TOTAL
26114 Sent 0 0 301 301
26115 Done 301 301 0 602
26116 Skipped 0 0 0 0
26117 Error 0 0 0 0
26118 Last Flush: 8243 8243 7942
26119 Downstairs last five completed: 7942 7941 7940 7939 7938
26120 Upstairs last five completed: 8243 8242 8241 8240 8239
26121 CLIENT: Up:0 ds:301 act:2
261222024-12-04T13:36:09.784ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261232024-12-04T13:36:09.784ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38184) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261242024-12-04T13:36:09.785ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261252024-12-04T13:36:09.786ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261262024-12-04T13:36:09.786ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261272024-12-04T13:36:09.786ZINFOcrucible: send last flush ID to this DS: 7942 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261282024-12-04T13:36:09.786ZINFOcrucible: Replied this last flush ID: 7942 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261292024-12-04T13:36:09.786ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261302024-12-04T13:36:09.786ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261312024-12-04T13:36:09.786ZINFOcrucible: [2] client re-new 301 jobs since flush 7942 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261322024-12-04T13:36:09.786ZINFOcrucible: [2] Replayed 301 jobs since flush: 7942 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
26133 ----------------------------------------------------------------
26134 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26135 Downstairs last five completed: 8243 8242 8241 8240 8239
26136 Upstairs last five completed: 8243 8242 8241 8240 8239
26137 CLIENT: Up:0 ds:0 act:3
26138 Replay: All jobs finished
26139 Main task has work to do, go find it
26140 got dsc Stop(2)
26141 stop 2
26142 [8830] Got stop action so:false kr:false
26143 Dec 04 13:36:12.036 INFO request completed, latency_us: 305, response_code: 204, uri: /stop/cid/2, method: POST, req_id: c1daf592-24c0-4c7f-8523-f2126bae8fc4, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
261442024-12-04T13:36:12.037ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
26145 Dec 04 13:36:12.037 INFO request completed, latency_us: 133, response_code: 200, uri: /state/cid/2, method: GET, req_id: b3fe4e44-0040-4266-9f4b-c700148c1859, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:36:12.037717851Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"2","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
26146 remote_addr: 127.0.0.1:37804, local_addr: 127.0.0{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:36:12.037745262Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
26147 .1:9998
261482024-12-04T13:36:12.037ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261492024-12-04T13:36:12.037ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
261502024-12-04T13:36:12.037ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
26151 [8830] Exited with: ExitStatus(unix_wait_status(9))
26152 [8830][2] reports Exit
26153 Dec 04 13:36:16.039 INFO request completed, latency_us: 192, response_code: 200, uri: /state/cid/2, method: GET, req_id: f8c95bc1-5fbc-48b5-aac4-bce994a26064, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
26154 001/300 Read block 2907 len 12288
26155 002/300 Read block 3438 len 36864
26156 003/300 Write block 3845 len 20480 data: 3 4 4 4 4
26157 004/300 Write block 1820 len 16384 data: 5 5 4 4
26158 005/300 Write block 1785 len 36864 data: 5 5 6 6 4 4 3 3 4
26159 006/300 Flush
26160 007/300 Read block 1027 len 4096
26161 008/300 Read block 234 len 12288
26162 009/300 Read block 3778 len 24576
26163 010/300 Read block 4723 len 12288
26164 011/300 Flush
26165 012/300 Write block 4607 len 4096 data: 7
26166 013/300 Read block 1066 len 32768
26167 014/300 Read block 2379 len 40960
26168 015/300 Write block 3385 len 16384 data: 3 3 2 3
26169 016/300 Read block 4854 len 20480
26170 017/300 Read block 1020 len 28672
26171 018/300 Read block 2704 len 4096
26172 019/300 Write block 92 len 8192 data: 7 6
26173 020/300 Write block 4778 len 4096 data: 4
26174 021/300 Write block 1439 len 24576 data: 5 6 6 6 3 2
26175 022/300 Write block 3563 len 8192 data: 4 6
26176 023/300 Read block 1736 len 8192
26177 024/300 Write block 2211 len 4096 data: 5
26178 025/300 Write block 2039 len 28672 data: 3 4 4 6 5 7 8
26179 026/300 Write block 1215 len 24576 data: 8 7 5 5 2 3
26180 027/300 Flush
26181 028/300 Write block 3563 len 20480 data: 5 7 7 7 7
26182 029/300 Read block 3457 len 28672
26183 030/300 Write block 2137 len 12288 data: 8 7 7
26184 031/300 Flush
26185 032/300 Read block 257 len 4096
26186 033/300 Write block 4245 len 8192 data: 7 7
26187 034/300 Write block 3979 len 12288 data: 4 4 4
26188 035/300 Write block 2194 len 8192 data: 8 8
26189 036/300 Write block 1194 len 28672 data: 5 3 3 3 3 5 5
26190 037/300 Read block 3473 len 16384
26191 038/300 Read block 2696 len 32768
26192 039/300 Read block 3451 len 40960
26193 040/300 Read block 4745 len 28672
26194 041/300 Read block 481 len 12288
26195 042/300 Read block 4991 len 24576
26196 043/300 Write block 2322 len 12288 data: 7 6 6
26197 044/300 Write block 1255 len 12288 data: 5 5 6
26198 045/300 Read block 4315 len 28672
26199 046/300 Write block 3259 len 36864 data: 5 5 5 3 4 4 4 3 2
26200 047/300 Write block 176 len 40960 data: 6 5 5 5 4 4 3 3 3 4
26201 048/300 Write block 234 len 16384 data: 3 3 3 3
26202 049/300 Flush
26203 050/300 Read block 4541 len 24576
26204 051/300 Read block 816 len 32768
26205 052/300 Write block 3469 len 32768 data: 5 5 5 3 4 7 9 10
26206 053/300 Write block 4369 len 36864 data: 4 5 5 4 3 4 4 5 5
26207 054/300 Read block 4434 len 8192
26208 055/300 Write block 1960 len 36864 data: 4 4 4 4 4 2 2 2 2
26209 056/300 Write block 1411 len 36864 data: 4 5 5 6 6 5 5 5 5
26210 057/300 Write block 3288 len 32768 data: 3 2 2 2 3 5 7 8
26211 058/300 Read block 57 len 28672
26212 059/300 Read block 2233 len 20480
26213 060/300 Read block 2159 len 28672
26214 061/300 Flush
26215 062/300 Write block 3947 len 20480 data: 6 7 6 7 5
26216 063/300 Write block 2280 len 32768 data: 6 6 6 6 6 4 5 7
26217 064/300 Write block 125 len 36864 data: 7 8 6 5 4 5 5 5 4
26218 065/300 Write block 4624 len 8192 data: 6 6
26219 066/300 Read block 1371 len 36864
26220 067/300 Write block 933 len 4096 data: 4
26221 068/300 Read block 179 len 4096
26222 069/300 Read block 2458 len 40960
26223 070/300 Write block 4631 len 8192 data: 2 3
26224 071/300 Write block 398 len 40960 data: 6 5 5 5 5 6 7 7 6 4
26225 072/300 Write block 410 len 32768 data: 4 4 4 4 3 4 4 5
26226 073/300 Read block 4698 len 20480
26227 074/300 Write block 3697 len 20480 data: 7 7 7 6 7
26228 075/300 Write block 2580 len 8192 data: 4 4
26229 076/300 Flush
26230 077/300 Write block 605 len 4096 data: 5
26231 078/300 Read block 2781 len 20480
26232 079/300 Write block 4524 len 20480 data: 6 6 6 6 5
26233 080/300 Read block 1445 len 36864
26234 081/300 Read block 6 len 12288
26235 082/300 Read block 4597 len 40960
26236 083/300 Write block 1078 len 12288 data: 8 6 7
26237 084/300 Flush
26238 085/300 Flush
26239 086/300 Write block 1311 len 40960 data: 3 2 2 2 2 2 4 5 5 6
26240 087/300 Write block 2716 len 4096 data: 2
26241 088/300 Read block 3440 len 4096
26242 089/300 Read block 1383 len 28672
26243 090/300 Write block 2872 len 36864 data: 8 7 8 7 6 4 5 5 5
26244 091/300 Write block 3968 len 16384 data: 5 7 7 7
26245 092/300 Read block 913 len 4096
26246 093/300 Write block 1031 len 32768 data: 5 4 5 5 4 5 4 4
26247 094/300 Write block 802 len 28672 data: 7 8 9 7 9 9 8
26248 095/300 Flush
26249 096/300 Write block 2006 len 28672 data: 3 3 3 3 4 5 5
26250 097/300 Write block 568 len 36864 data: 7 6 6 4 4 4 3 2 3
26251 098/300 Read block 4651 len 40960
26252 099/300 Read block 4713 len 36864
26253 100/300 Read block 3891 len 36864
26254 101/300 Read block 1457 len 8192
26255 102/300 Write block 2488 len 40960 data: 5 5 3 2 2 2 2 2 2 2
26256 103/300 Read block 3204 len 24576
26257 104/300 Read block 2193 len 4096
26258 105/300 Read block 508 len 32768
26259 106/300 Read block 4602 len 12288
26260 107/300 Read block 3031 len 4096
26261 108/300 Write block 3461 len 4096 data: 4
26262 109/300 Read block 517 len 8192
26263 110/300 Read block 3705 len 28672
26264 111/300 Write block 4099 len 28672 data: 4 3 3 3 3 4 3
26265 112/300 Read block 1695 len 28672
26266 113/300 Write block 3096 len 32768 data: 4 3 3 3 4 4 4 6
26267 114/300 Flush
26268 115/300 Read block 2641 len 8192
26269 116/300 Read block 2843 len 32768
26270 117/300 Read block 4318 len 36864
26271 118/300 Read block 1832 len 24576
26272 119/300 Write block 2190 len 8192 data: 8 9
26273 120/300 Flush
26274 121/300 Read block 1728 len 24576
26275 122/300 Read block 69 len 36864
26276 123/300 Read block 1930 len 28672
26277 124/300 Write block 1083 len 4096 data: 7
26278 125/300 Read block 1100 len 36864
26279 126/300 Read block 4514 len 8192
26280 127/300 Write block 4502 len 20480 data: 3 3 3 3 3
26281 128/300 Read block 565 len 12288
26282 129/300 Write block 4419 len 24576 data: 7 7 7 7 7 6
26283 130/300 Read block 4430 len 20480
26284 131/300 Write block 2720 len 40960 data: 5 5 4 3 5 5 4 5 6 6
26285 132/300 Read block 4591 len 4096
26286 133/300 Flush
26287 134/300 Write block 2770 len 40960 data: 5 8 9 9 9 9 7 6 3 4
26288 135/300 Write block 1005 len 40960 data: 6 5 5 5 3 4 3 5 4 5
26289 136/300 Read block 2150 len 16384
26290 137/300 Write block 4149 len 28672 data: 4 3 3 3 6 6 8
26291 138/300 Read block 3971 len 24576
26292 139/300 Read block 2472 len 8192
26293 140/300 Write block 1300 len 4096 data: 6
26294 141/300 Read block 2384 len 12288
26295 142/300 Read block 1725 len 4096
26296 143/300 Read block 4383 len 20480
26297 144/300 Read block 219 len 40960
26298 145/300 Write block 938 len 40960 data: 4 4 6 5 5 6 5 6 6 7
26299 146/300 Read block 1294 len 32768
26300 147/300 Write block 2007 len 28672 data: 4 4 4 5 6 6 4
26301 148/300 Flush
26302 149/300 Write block 668 len 16384 data: 5 7 7 5
26303 150/300 Write block 1559 len 40960 data: 6 6 5 4 4 4 6 6 6 6
26304 151/300 Flush
26305 152/300 Read block 106 len 36864
26306 153/300 Write block 1387 len 40960 data: 4 3 4 4 4 4 4 4 4 3
26307 154/300 Write block 2041 len 40960 data: 5 7 6 8 9 8 6 6 7 7
26308 155/300 Write block 2848 len 28672 data: 8 8 7 8 10 7 6
26309 156/300 Read block 2745 len 32768
26310 157/300 Write block 469 len 40960 data: 4 4 5 4 5 4 4 5 7 5
26311 158/300 Write block 1633 len 12288 data: 6 8 6
26312 159/300 Write block 2037 len 4096 data: 3
26313 160/300 Read block 3700 len 12288
26314 161/300 Flush
26315 162/300 Flush
26316 163/300 Flush
26317 164/300 Flush
26318 165/300 Read block 2256 len 36864
26319 166/300 Read block 1138 len 4096
26320 167/300 Write block 27 len 20480 data: 5 6 6 5 6
26321 168/300 Flush
26322 169/300 Read block 12 len 20480
26323 170/300 Flush
26324 171/300 Read block 3093 len 24576
26325 172/300 Flush
26326 173/300 Read block 2756 len 8192
26327 174/300 Read block 3641 len 20480
26328 175/300 Write block 937 len 36864 data: 4 5 5 7 6 6 7 6 7
26329 176/300 Write block 2218 len 28672 data: 5 4 3 4 4 4 3
26330 177/300 Read block 701 len 28672
26331 178/300 Write block 2357 len 28672 data: 3 3 4 4 3 3 6
26332 179/300 Read block 2351 len 32768
26333 180/300 Read block 4255 len 36864
26334 181/300 Read block 3748 len 8192
26335 182/300 Read block 1403 len 40960
26336 183/300 Read block 1631 len 32768
26337 184/300 Read block 1447 len 28672
26338 185/300 Write block 2871 len 36864 data: 6 9 8 9 8 7 5 6 6
26339 186/300 Write block 2936 len 20480 data: 6 5 4 3 3
26340 187/300 Write block 4172 len 32768 data: 6 5 4 6 6 5 5 5
26341 188/300 Read block 4731 len 16384
26342 189/300 Read block 3351 len 20480
26343 190/300 Write block 4680 len 12288 data: 7 5 4
26344 191/300 Write block 3792 len 8192 data: 5 6
26345 192/300 Read block 253 len 8192
26346 193/300 Write block 964 len 16384 data: 4 3 3 5
26347 194/300 Write block 2529 len 36864 data: 4 3 3 3 3 3 4 6 9
26348 195/300 Write block 3247 len 12288 data: 3 3 2
26349 196/300 Flush
26350 197/300 Write block 3843 len 12288 data: 2 2 4
26351 198/300 Write block 4945 len 28672 data: 5 7 7 6 5 5 5
26352 199/300 Read block 3495 len 40960
26353 200/300 Read block 619 len 4096
26354 201/300 Read block 4768 len 24576
26355 202/300 Write block 3221 len 40960 data: 5 5 5 7 7 8 4 3 6 7
26356 203/300 Flush
26357 204/300 Read block 3718 len 12288
26358 205/300 Read block 1301 len 32768
26359 206/300 Write block 746 len 36864 data: 5 5 4 4 3 4 5 6 8
26360 207/300 Write block 2733 len 12288 data: 5 5 5
26361 208/300 Write block 1902 len 40960 data: 6 5 4 5 4 3 3 3 4 7
26362 209/300 Read block 3059 len 28672
26363 210/300 Write block 1083 len 4096 data: 8
26364 211/300 Read block 452 len 40960
26365 212/300 Write block 1649 len 12288 data: 5 4 5
26366 213/300 Flush
26367 214/300 Write block 1966 len 28672 data: 3 3 3 2 5 5 5
26368 215/300 Read block 2609 len 28672
26369 216/300 Flush
26370 217/300 Write block 128 len 32768 data: 6 5 6 6 6 5 2 2
26371 218/300 Write block 2414 len 12288 data: 6 6 4
26372 219/300 Write block 3154 len 36864 data: 10 8 7 5 7 5 6 5 5
26373 220/300 Write block 1717 len 20480 data: 4 3 3 3 3
26374 221/300 Read block 580 len 28672
26375 222/300 Write block 4302 len 28672 data: 3 4 4 4 5 4 4
26376 223/300 Write block 4628 len 20480 data: 3 3 2 3 4
26377 224/300 Write block 979 len 28672 data: 7 7 6 6 8 8 8
26378 225/300 Read block 968 len 24576
26379 226/300 Read block 2848 len 24576
26380 227/300 Read block 4187 len 12288
26381 228/300 Write block 801 len 24576 data: 6 8 9 10 8 10
26382 229/300 Write block 1114 len 28672 data: 9 8 7 8 7 4 4
26383 230/300 Read block 3736 len 36864
26384 231/300 Write block 3645 len 40960 data: 5 5 5 4 4 4 3 4 4 4
26385 232/300 Write block 3272 len 24576 data: 4 4 5 4 4 3
26386 233/300 Flush
26387 234/300 Write block 1133 len 40960 data: 2 2 5 6 6 7 6 6 7 7
26388 235/300 Write block 2797 len 12288 data: 5 6 5
26389 236/300 Write block 2571 len 24576 data: 9 5 5 5 4 4
26390 237/300 Read block 1785 len 16384
26391 238/300 Write block 2973 len 28672 data: 5 3 3 3 3 4 3
26392 239/300 Write block 2397 len 8192 data: 7 7
26393 240/300 Read block 4849 len 20480
26394 241/300 Write block 2640 len 12288 data: 10 9 8
26395 242/300 Read block 3467 len 36864
26396 243/300 Write block 4574 len 20480 data: 5 4 3 5 5
26397 244/300 Read block 3661 len 12288
26398 245/300 Read block 4964 len 4096
26399 246/300 Write block 4379 len 28672 data: 5 3 3 3 3 3 4
26400 247/300 Flush
26401 248/300 Write block 1891 len 36864 data: 8 8 8 7 7 7 5 7 7
26402 249/300 Write block 4983 len 20480 data: 5 5 6 5 5
26403 250/300 Flush
26404 251/300 Flush
26405 252/300 Write block 2520 len 8192 data: 7 7
26406 253/300 Read block 1758 len 20480
26407 254/300 Write block 2832 len 20480 data: 6 4 3 4 4
26408 255/300 Read block 275 len 40960
26409 256/300 Read block 2958 len 12288
26410 257/300 Read block 267 len 24576
26411 258/300 Write block 3456 len 8192 data: 3 3
26412 259/300 Read block 85 len 36864
26413 260/300 Read block 3273 len 40960
26414 261/300 Write block 791 len 4096 data: 6
26415 262/300 Read block 4225 len 12288
26416 263/300 Read block 1072 len 16384
26417 264/300 Read block 1385 len 8192
26418 265/300 Write block 93 len 12288 data: 7 6 7
26419 266/300 Read block 44 len 28672
26420 267/300 Write block 2379 len 24576 data: 6 6 5 5 5 5
26421 268/300 Write block 766 len 8192 data: 5 5
26422 269/300 Read block 952 len 28672
26423 270/300 Read block 331 len 40960
26424 271/300 Flush
26425 272/300 Read block 1875 len 24576
26426 273/300 Read block 301 len 20480
26427 274/300 Read block 3805 len 4096
26428 275/300 Read block 4900 len 8192
26429 276/300 Write block 1515 len 32768 data: 10 10 7 7 5 4 4 5
26430 277/300 Read block 4207 len 40960
26431 278/300 Read block 2973 len 4096
26432 279/300 Flush
26433 280/300 Write block 1651 len 8192 data: 6 5
26434 281/300 Read block 1544 len 16384
26435 282/300 Write block 2894 len 40960 data: 6 6 5 5 5 4 4 6 5 6
26436 283/300 Write block 15 len 36864 data: 8 12 10 6 4 3 3 3 4
26437 284/300 Read block 1157 len 16384
26438 285/300 Write block 796 len 24576 data: 3 3 4 6 8 7
26439 286/300 Write block 4789 len 20480 data: 4 4 3 4 4
26440 287/300 Read block 541 len 24576
26441 288/300 Read block 2503 len 16384
26442 289/300 Read block 4605 len 12288
26443 290/300 Write block 3198 len 4096 data: 3
26444 291/300 Write block 2773 len 24576 data: 10 10 10 8 7 4
26445 292/300 Read block 3871 len 16384
26446 293/300 Read block 438 len 40960
26447 294/300 Write block 1912 len 36864 data: 7 7 6 6 6 5 5 5 5
26448 295/300 Write block 4575 len 8192 data: 5 4
26449 296/300 Write block 698 len 32768 data: 3 3 4 5 4 4 4 5
26450 297/300 Read block 2685 len 36864
26451 298/300 Read block 1104 len 32768
26452 299/300 Read block 4576 len 40960
26453 300/300 Write block 2825 len 16384 data: 6 8 8 8
26454 301/300 Read block 2892 len 12288
26455 Main task has work to do, go find it
26456 got dsc Start(2)
26457 start 2
26458 Dec 04 13:36:16.268 INFO request completed, latency_us: 192, response_code: 204, uri: /start/cid/2, method: POST, req_id: 61e83257-5a23-4a52-aebe-1bc309c65bd3, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
26459 Replay: started 2, returned:Ok(())
26460 ----------------------------------------------------------------
26461 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26462 States: Active Active Offline
26463 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26464 8244 Acked Read 3 Done Done Sent false
26465 8245 Acked Read 9 Done Done Sent false
26466 8246 Acked Write 5 Done Done Sent false
26467 8247 Acked Write 4 Done Done Sent false
26468 8248 Acked Write 9 Done Done Sent false
26469 8249 Acked Flush 0 Done Done Sent false
26470 8250 Acked Read 1 Done Done Sent false
26471 8251 Acked Read 3 Done Done Sent false
26472 8252 Acked Read 6 Done Done Sent false
26473 8253 Acked Read 3 Done Done Sent false
26474 8254 Acked Flush 0 Done Done Sent false
26475 8255 Acked Write 1 Done Done Sent false
26476 8256 Acked Read 8 Done Done Sent false
26477 8257 Acked Read 10 Done Done Sent false
26478 8258 Acked Write 4 Done Done Sent false
26479 8259 Acked Read 5 Done Done Sent false
26480 8260 Acked Read 7 Done Done Sent false
26481 8261 Acked Read 1 Done Done Sent false
26482 8262 Acked Write 2 Done Done Sent false
26483 8263 Acked Write 1 Done Done Sent false
26484 8264 Acked Write 6 Done Done Sent false
26485 8265 Acked Write 2 Done Done Sent false
26486 8266 Acked Read 2 Done Done Sent false
26487 8267 Acked Write 1 Done Done Sent false
26488 8268 Acked Write 7 Done Done Sent false
26489 8269 Acked Write 6 Done Done Sent false
26490 8270 Acked Flush 0 Done Done Sent false
26491 8271 Acked Write 5 Done Done Sent false
26492 8272 Acked Read 7 Done Done Sent false
26493 8273 Acked Write 3 Done Done Sent false
26494 8274 Acked Flush 0 Done Done Sent false
26495 8275 Acked Read 1 Done Done Sent false
26496 8276 Acked Write 2 Done Done Sent false
26497 8277 Acked Write 3 Done Done Sent false
26498 8278 Acked Write 2 Done Done Sent false
26499 8279 Acked Write 7 Done Done Sent false
26500 8280 Acked Read 4 Done Done Sent false
26501 8281 Acked Read 8 Done Done Sent false
26502 8282 Acked Read 10 Done Done Sent false
26503 8283 Acked Read 7 Done Done Sent false
26504 8284 Acked Read 3 Done Done Sent false
26505 8285 Acked Read 6 Done Done Sent false
26506 8286 Acked Write 3 Done Done Sent false
26507 8287 Acked Write 3 Done Done Sent false
26508 8288 Acked Read 7 Done Done Sent false
26509 8289 Acked Write 9 Done Done Sent false
26510 8290 Acked Write 10 Done Done Sent false
26511 8291 Acked Write 4 Done Done Sent false
26512 8292 Acked Flush 0 Done Done Sent false
26513 8293 Acked Read 6 Done Done Sent false
26514 8294 Acked Read 8 Done Done Sent false
26515 8295 Acked Write 8 Done Done Sent false
26516 8296 Acked Write 9 Done Done Sent false
26517 8297 Acked Read 2 Done Done Sent false
26518 8298 Acked Write 9 Done Done Sent false
26519 8299 Acked Write 9 Done Done Sent false
26520 8300 Acked Write 8 Done Done Sent false
26521 8301 Acked Read 7 Done Done Sent false
26522 8302 Acked Read 5 Done Done Sent false
26523 8303 Acked Read 7 Done Done Sent false
26524 8304 Acked Flush 0 Done Done Sent false
26525 8305 Acked Write 5 Done Done Sent false
26526 8306 Acked Write 8 Done Done Sent false
26527 8307 Acked Write 9 Done Done Sent false
26528 8308 Acked Write 2 Done Done Sent false
26529 8309 Acked Read 9 Done Done Sent false
26530 8310 Acked Write 1 Done Done Sent false
26531 8311 Acked Read 1 Done Done Sent false
26532 8312 Acked Read 10 Done Done Sent false
26533 8313 Acked Write 2 Done Done Sent false
26534 8314 Acked Write 10 Done Done Sent false
26535 8315 Acked Write 8 Done Done Sent false
26536 8316 Acked Read 5 Done Done Sent false
26537 8317 Acked Write 5 Done Done Sent false
26538 8318 Acked Write 2 Done Done Sent false
26539 8319 Acked Flush 0 Done Done Sent false
26540 8320 Acked Write 1 Done Done Sent false
26541 8321 Acked Read 5 Done Done Sent false
26542 8322 Acked Write 5 Done Done Sent false
26543 8323 Acked Read 9 Done Done Sent false
26544 8324 Acked Read 3 Done Done Sent false
26545 8325 Acked Read 10 Done Done Sent false
26546 8326 Acked Write 3 Done Done Sent false
26547 8327 Acked Flush 0 Done Done Sent false
26548 8328 Acked Flush 0 Done Done Sent false
26549 8329 Acked Write 10 Done Done Sent false
26550 8330 Acked Write 1 Done Done Sent false
26551 8331 Acked Read 1 Done Done Sent false
26552 8332 Acked Read 7 Done Done Sent false
26553 8333 Acked Write 9 Done Done Sent false
26554 8334 Acked Write 4 Done Done Sent false
26555 8335 Acked Read 1 Done Done Sent false
26556 8336 Acked Write 8 Done Done Sent false
26557 8337 Acked Write 7 Done Done Sent false
26558 8338 Acked Flush 0 Done Done Sent false
26559 8339 Acked Write 7 Done Done Sent false
26560 8340 Acked Write 9 Done Done Sent false
26561 8341 Acked Read 10 Done Done Sent false
26562 8342 Acked Read 9 Done Done Sent false
26563 8343 Acked Read 9 Done Done Sent false
26564 8344 Acked Read 2 Done Done Sent false
26565 8345 Acked Write 10 Done Done Sent false
26566 8346 Acked Read 6 Done Done Sent false
26567 8347 Acked Read 1 Done Done Sent false
26568 8348 Acked Read 8 Done Done Sent false
26569 8349 Acked Read 3 Done Done Sent false
26570 8350 Acked Read 1 Done Done Sent false
26571 8351 Acked Write 1 Done Done Sent false
26572 8352 Acked Read 2 Done Done Sent false
26573 8353 Acked Read 7 Done Done Sent false
26574 8354 Acked Write 7 Done Done Sent false
26575 8355 Acked Read 7 Done Done Sent false
26576 8356 Acked Write 8 Done Done Sent false
26577 8357 Acked Flush 0 Done Done Sent false
26578 8358 Acked Read 2 Done Done Sent false
26579 8359 Acked Read 8 Done Done Sent false
26580 8360 Acked Read 9 Done Done Sent false
26581 8361 Acked Read 6 Done Done Sent false
26582 8362 Acked Write 2 Done Done Sent false
26583 8363 Acked Flush 0 Done Done Sent false
26584 8364 Acked Read 6 Done Done Sent false
26585 8365 Acked Read 9 Done Done Sent false
26586 8366 Acked Read 7 Done Done Sent false
26587 8367 Acked Write 1 Done Done Sent false
26588 8368 Acked Read 9 Done Done Sent false
26589 8369 Acked Read 2 Done Done Sent false
26590 8370 Acked Write 5 Done Done Sent false
26591 8371 Acked Read 3 Done Done Sent false
26592 8372 Acked Write 6 Done Done Sent false
26593 8373 Acked Read 5 Done Done Sent false
26594 8374 Acked Write 10 Done Done Sent false
26595 8375 Acked Read 1 Done Done Sent false
26596 8376 Acked Flush 0 Done Done Sent false
26597 8377 Acked Write 10 Done Done Sent false
26598 8378 Acked Write 10 Done Done Sent false
26599 8379 Acked Read 4 Done Done Sent false
26600 8380 Acked Write 7 Done Done Sent false
26601 8381 Acked Read 6 Done Done Sent false
26602 8382 Acked Read 2 Done Done Sent false
26603 8383 Acked Write 1 Done Done Sent false
26604 8384 Acked Read 3 Done Done Sent false
26605 8385 Acked Read 1 Done Done Sent false
26606 8386 Acked Read 5 Done Done Sent false
26607 8387 Acked Read 10 Done Done Sent false
26608 8388 Acked Write 10 Done Done Sent false
26609 8389 Acked Read 8 Done Done Sent false
26610 8390 Acked Write 7 Done Done Sent false
26611 8391 Acked Flush 0 Done Done Sent false
26612 8392 Acked Write 4 Done Done Sent false
26613 8393 Acked Write 10 Done Done Sent false
26614 8394 Acked Flush 0 Done Done Sent false
26615 8395 Acked Read 9 Done Done Sent false
26616 8396 Acked Write 10 Done Done Sent false
26617 8397 Acked Write 10 Done Done Sent false
26618 8398 Acked Write 7 Done Done Sent false
26619 8399 Acked Read 8 Done Done Sent false
26620 8400 Acked Write 10 Done Done Sent false
26621 8401 Acked Write 3 Done Done Sent false
26622 8402 Acked Write 1 Done Done Sent false
26623 8403 Acked Read 3 Done Done Sent false
26624 8404 Acked Flush 0 Done Done Sent false
26625 8405 Acked Flush 0 Done Done Sent false
26626 8406 Acked Flush 0 Done Done Sent false
26627 8407 Acked Flush 0 Done Done Sent false
26628 8408 Acked Read 9 Done Done Sent false
26629 8409 Acked Read 1 Done Done Sent false
26630 8410 Acked Write 5 Done Done Sent false
26631 8411 Acked Flush 0 Done Done Sent false
26632 8412 Acked Read 5 Done Done Sent false
26633 8413 Acked Flush 0 Done Done Sent false
26634 8414 Acked Read 6 Done Done Sent false
26635 8415 Acked Flush 0 Done Done Sent false
26636 8416 Acked Read 2 Done Done Sent false
26637 8417 Acked Read 5 Done Done Sent false
26638 8418 Acked Write 9 Done Done Sent false
26639 8419 Acked Write 7 Done Done Sent false
26640 8420 Acked Read 7 Done Done Sent false
26641 8421 Acked Write 7 Done Done Sent false
26642 8422 Acked Read 8 Done Done Sent false
26643 8423 Acked Read 9 Done Done Sent false
26644 8424 Acked Read 2 Done Done Sent false
26645 8425 Acked Read 10 Done Done Sent false
26646 8426 Acked Read 8 Done Done Sent false
26647 8427 Acked Read 7 Done Done Sent false
26648 8428 Acked Write 9 Done Done Sent false
26649 8429 Acked Write 5 Done Done Sent false
26650 8430 Acked Write 8 Done Done Sent false
26651 8431 Acked Read 4 Done Done Sent false
26652 8432 Acked Read 5 Done Done Sent false
26653 8433 Acked Write 3 Done Done Sent false
26654 8434 Acked Write 2 Done Done Sent false
26655 8435 Acked Read 2 Done Done Sent false
26656 8436 Acked Write 4 Done Done Sent false
26657 8437 Acked Write 9 Done Done Sent false
26658 8438 Acked Write 3 Done Done Sent false
26659 8439 Acked Flush 0 Done Done Sent false
26660 8440 Acked Write 3 Done Done Sent false
26661 8441 Acked Write 7 Done Done Sent false
26662 8442 Acked Read 10 Done Done Sent false
26663 8443 Acked Read 1 Done Done Sent false
26664 8444 Acked Read 6 Done Done Sent false
26665 8445 Acked Write 10 Done Done Sent false
26666 8446 Acked Flush 0 Done Done Sent false
26667 8447 Acked Read 3 Done Done Sent false
26668 8448 Acked Read 8 Done Done Sent false
26669 8449 Acked Write 9 Done Done Sent false
26670 8450 Acked Write 3 Done Done Sent false
26671 8451 Acked Write 10 Done Done Sent false
26672 8452 Acked Read 7 Done Done Sent false
26673 8453 Acked Write 1 Done Done Sent false
26674 8454 Acked Read 10 Done Done Sent false
26675 8455 Acked Write 3 Done Done Sent false
26676 8456 Acked Flush 0 Done Done Sent false
26677 8457 Acked Write 7 Done Done Sent false
26678 8458 Acked Read 7 Done Done Sent false
26679 8459 Acked Flush 0 Done Done Sent false
26680 8460 Acked Write 8 Done Done Sent false
26681 8461 Acked Write 3 Done Done Sent false
26682 8462 Acked Write 9 Done Done Sent false
26683 8463 Acked Write 5 Done Done Sent false
26684 8464 Acked Read 7 Done Done Sent false
26685 8465 Acked Write 7 Done Done Sent false
26686 8466 Acked Write 5 Done Done Sent false
26687 8467 Acked Write 7 Done Done Sent false
26688 8468 Acked Read 6 Done Done Sent false
26689 8469 Acked Read 6 Done Done Sent false
26690 8470 Acked Read 3 Done Done Sent false
26691 8471 Acked Write 6 Done Done Sent false
26692 8472 Acked Write 7 Done Done Sent false
26693 8473 Acked Read 9 Done Done Sent false
26694 8474 Acked Write 10 Done Done Sent false
26695 8475 Acked Write 6 Done Done Sent false
26696 8476 Acked Flush 0 Done Done Sent false
26697 8477 Acked Write 10 Done Done Sent false
26698 8478 Acked Write 3 Done Done Sent false
26699 8479 Acked Write 6 Done Done Sent false
26700 8480 Acked Read 4 Done Done Sent false
26701 8481 Acked Write 7 Done Done Sent false
26702 8482 Acked Write 2 Done Done Sent false
26703 8483 Acked Read 5 Done Done Sent false
26704 8484 Acked Write 3 Done Done Sent false
26705 8485 Acked Read 9 Done Done Sent false
26706 8486 Acked Write 5 Done Done Sent false
26707 8487 Acked Read 3 Done Done Sent false
26708 8488 Acked Read 1 Done Done Sent false
26709 8489 Acked Write 7 Done Done Sent false
26710 8490 Acked Flush 0 Done Done Sent false
26711 8491 Acked Write 9 Done Done Sent false
26712 8492 Acked Write 5 Done Done Sent false
26713 8493 Acked Flush 0 Done Done Sent false
26714 8494 Acked Flush 0 Done Done Sent false
26715 8495 Acked Write 2 Done Done Sent false
26716 8496 Acked Read 5 Done Done Sent false
26717 8497 Acked Write 5 Done Done Sent false
26718 8498 Acked Read 10 Done Done Sent false
26719 8499 Acked Read 3 Done Done Sent false
26720 8500 Acked Read 6 Done Done Sent false
26721 8501 Acked Write 2 Done Done Sent false
26722 8502 Acked Read 9 Done Done Sent false
26723 8503 Acked Read 10 Done Done Sent false
26724 8504 Acked Write 1 Done Done Sent false
26725 8505 Acked Read 3 Done Done Sent false
26726 8506 Acked Read 4 Done Done Sent false
26727 8507 Acked Read 2 Done Done Sent false
26728 8508 Acked Write 3 Done Done Sent false
26729 8509 Acked Read 7 Done Done Sent false
26730 8510 Acked Write 6 Done Done Sent false
26731 8511 Acked Write 2 Done Done Sent false
26732 8512 Acked Read 7 Done Done Sent false
26733 8513 Acked Read 10 Done Done Sent false
26734 8514 Acked Flush 0 Done Done Sent false
26735 8515 Acked Read 6 Done Done Sent false
26736 8516 Acked Read 5 Done Done Sent false
26737 8517 Acked Read 1 Done Done Sent false
26738 8518 Acked Read 2 Done Done Sent false
26739 8519 Acked Write 8 Done Done Sent false
26740 8520 Acked Read 10 Done Done Sent false
26741 8521 Acked Read 1 Done Done Sent false
26742 8522 Acked Flush 0 Done Done Sent false
26743 8523 Acked Write 2 Done Done Sent false
26744 8524 Acked Read 4 Done Done Sent false
26745 8525 Acked Write 10 Done Done Sent false
26746 8526 Acked Write 9 Done Done Sent false
26747 8527 Acked Read 4 Done Done Sent false
26748 8528 Acked Write 6 Done Done Sent false
26749 8529 Acked Write 5 Done Done Sent false
26750 8530 Acked Read 6 Done Done Sent false
26751 8531 Acked Read 4 Done Done Sent false
26752 8532 Acked Read 3 Done Done Sent false
26753 8533 Acked Write 1 Done Done Sent false
26754 8534 Acked Write 6 Done Done Sent false
26755 8535 Acked Read 4 Done Done Sent false
26756 8536 Acked Read 10 Done Done Sent false
26757 8537 Acked Write 9 Done Done Sent false
26758 8538 Acked Write 2 Done Done Sent false
26759 8539 Acked Write 8 Done Done Sent false
26760 8540 Acked Read 9 Done Done Sent false
26761 8541 Acked Read 8 Done Done Sent false
26762 8542 Acked Read 10 Done Done Sent false
26763 8543 Acked Write 4 Done Done Sent false
26764 8544 Acked Read 3 Done Done Sent false
26765 STATES DS:0 DS:1 DS:2 TOTAL
26766 Sent 0 0 301 301
26767 Done 301 301 0 602
26768 Skipped 0 0 0 0
26769 Error 0 0 0 0
26770 Last Flush: 8522 8522 8243
26771 Downstairs last five completed: 8243 8242 8241 8240 8239
26772 Upstairs last five completed: 8544 8543 8542 8541 8540
26773 CLIENT: Up:0 ds:301 act:2
26774 [8830] got start action
26775 [8830] I am going to restart
26776 Starting downstairs at port 8830
26777 Make output file at "/tmp/dsc/downstairs-8830.txt"
26778 [8830][2] reports Starting
26779 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1228)
26780 [8830][2] reports Running
26781 ----------------------------------------------------------------
26782 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26783 States: Active Active Offline
26784 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26785 8244 Acked Read 3 Done Done Sent false
26786 8245 Acked Read 9 Done Done Sent false
26787 8246 Acked Write 5 Done Done Sent false
26788 8247 Acked Write 4 Done Done Sent false
26789 8248 Acked Write 9 Done Done Sent false
26790 8249 Acked Flush 0 Done Done Sent false
26791 8250 Acked Read 1 Done Done Sent false
26792 8251 Acked Read 3 Done Done Sent false
26793 8252 Acked Read 6 Done Done Sent false
26794 8253 Acked Read 3 Done Done Sent false
26795 8254 Acked Flush 0 Done Done Sent false
26796 8255 Acked Write 1 Done Done Sent false
26797 8256 Acked Read 8 Done Done Sent false
26798 8257 Acked Read 10 Done Done Sent false
26799 8258 Acked Write 4 Done Done Sent false
26800 8259 Acked Read 5 Done Done Sent false
26801 8260 Acked Read 7 Done Done Sent false
26802 8261 Acked Read 1 Done Done Sent false
26803 8262 Acked Write 2 Done Done Sent false
26804 8263 Acked Write 1 Done Done Sent false
26805 8264 Acked Write 6 Done Done Sent false
26806 8265 Acked Write 2 Done Done Sent false
26807 8266 Acked Read 2 Done Done Sent false
26808 8267 Acked Write 1 Done Done Sent false
26809 8268 Acked Write 7 Done Done Sent false
26810 8269 Acked Write 6 Done Done Sent false
26811 8270 Acked Flush 0 Done Done Sent false
26812 8271 Acked Write 5 Done Done Sent false
26813 8272 Acked Read 7 Done Done Sent false
26814 8273 Acked Write 3 Done Done Sent false
26815 8274 Acked Flush 0 Done Done Sent false
26816 8275 Acked Read 1 Done Done Sent false
26817 8276 Acked Write 2 Done Done Sent false
26818 8277 Acked Write 3 Done Done Sent false
26819 8278 Acked Write 2 Done Done Sent false
26820 8279 Acked Write 7 Done Done Sent false
26821 8280 Acked Read 4 Done Done Sent false
26822 8281 Acked Read 8 Done Done Sent false
26823 8282 Acked Read 10 Done Done Sent false
26824 8283 Acked Read 7 Done Done Sent false
26825 8284 Acked Read 3 Done Done Sent false
26826 8285 Acked Read 6 Done Done Sent false
26827 8286 Acked Write 3 Done Done Sent false
26828 8287 Acked Write 3 Done Done Sent false
26829 8288 Acked Read 7 Done Done Sent false
26830 8289 Acked Write 9 Done Done Sent false
26831 8290 Acked Write 10 Done Done Sent false
26832 8291 Acked Write 4 Done Done Sent false
26833 8292 Acked Flush 0 Done Done Sent false
26834 8293 Acked Read 6 Done Done Sent false
26835 8294 Acked Read 8 Done Done Sent false
26836 8295 Acked Write 8 Done Done Sent false
26837 8296 Acked Write 9 Done Done Sent false
26838 8297 Acked Read 2 Done Done Sent false
26839 8298 Acked Write 9 Done Done Sent false
26840 8299 Acked Write 9 Done Done Sent false
26841 8300 Acked Write 8 Done Done Sent false
26842 8301 Acked Read 7 Done Done Sent false
26843 8302 Acked Read 5 Done Done Sent false
26844 8303 Acked Read 7 Done Done Sent false
26845 8304 Acked Flush 0 Done Done Sent false
26846 8305 Acked Write 5 Done Done Sent false
26847 8306 Acked Write 8 Done Done Sent false
26848 8307 Acked Write 9 Done Done Sent false
26849 8308 Acked Write 2 Done Done Sent false
26850 8309 Acked Read 9 Done Done Sent false
26851 8310 Acked Write 1 Done Done Sent false
26852 8311 Acked Read 1 Done Done Sent false
26853 8312 Acked Read 10 Done Done Sent false
26854 8313 Acked Write 2 Done Done Sent false
26855 8314 Acked Write 10 Done Done Sent false
26856 8315 Acked Write 8 Done Done Sent false
26857 8316 Acked Read 5 Done Done Sent false
26858 8317 Acked Write 5 Done Done Sent false
26859 8318 Acked Write 2 Done Done Sent false
26860 8319 Acked Flush 0 Done Done Sent false
26861 8320 Acked Write 1 Done Done Sent false
26862 8321 Acked Read 5 Done Done Sent false
26863 8322 Acked Write 5 Done Done Sent false
26864 8323 Acked Read 9 Done Done Sent false
26865 8324 Acked Read 3 Done Done Sent false
26866 8325 Acked Read 10 Done Done Sent false
26867 8326 Acked Write 3 Done Done Sent false
26868 8327 Acked Flush 0 Done Done Sent false
26869 8328 Acked Flush 0 Done Done Sent false
26870 8329 Acked Write 10 Done Done Sent false
26871 8330 Acked Write 1 Done Done Sent false
26872 8331 Acked Read 1 Done Done Sent false
26873 8332 Acked Read 7 Done Done Sent false
26874 8333 Acked Write 9 Done Done Sent false
26875 8334 Acked Write 4 Done Done Sent false
26876 8335 Acked Read 1 Done Done Sent false
26877 8336 Acked Write 8 Done Done Sent false
26878 8337 Acked Write 7 Done Done Sent false
26879 8338 Acked Flush 0 Done Done Sent false
26880 8339 Acked Write 7 Done Done Sent false
26881 8340 Acked Write 9 Done Done Sent false
26882 8341 Acked Read 10 Done Done Sent false
26883 8342 Acked Read 9 Done Done Sent false
26884 8343 Acked Read 9 Done Done Sent false
26885 8344 Acked Read 2 Done Done Sent false
26886 8345 Acked Write 10 Done Done Sent false
26887 8346 Acked Read 6 Done Done Sent false
26888 8347 Acked Read 1 Done Done Sent false
26889 8348 Acked Read 8 Done Done Sent false
26890 8349 Acked Read 3 Done Done Sent false
26891 8350 Acked Read 1 Done Done Sent false
26892 8351 Acked Write 1 Done Done Sent false
26893 8352 Acked Read 2 Done Done Sent false
26894 8353 Acked Read 7 Done Done Sent false
26895 8354 Acked Write 7 Done Done Sent false
26896 8355 Acked Read 7 Done Done Sent false
26897 8356 Acked Write 8 Done Done Sent false
26898 8357 Acked Flush 0 Done Done Sent false
26899 8358 Acked Read 2 Done Done Sent false
26900 8359 Acked Read 8 Done Done Sent false
26901 8360 Acked Read 9 Done Done Sent false
26902 8361 Acked Read 6 Done Done Sent false
26903 8362 Acked Write 2 Done Done Sent false
26904 8363 Acked Flush 0 Done Done Sent false
26905 8364 Acked Read 6 Done Done Sent false
26906 8365 Acked Read 9 Done Done Sent false
26907 8366 Acked Read 7 Done Done Sent false
26908 8367 Acked Write 1 Done Done Sent false
26909 8368 Acked Read 9 Done Done Sent false
26910 8369 Acked Read 2 Done Done Sent false
26911 8370 Acked Write 5 Done Done Sent false
26912 8371 Acked Read 3 Done Done Sent false
26913 8372 Acked Write 6 Done Done Sent false
26914 8373 Acked Read 5 Done Done Sent false
26915 8374 Acked Write 10 Done Done Sent false
26916 8375 Acked Read 1 Done Done Sent false
26917 8376 Acked Flush 0 Done Done Sent false
26918 8377 Acked Write 10 Done Done Sent false
26919 8378 Acked Write 10 Done Done Sent false
26920 8379 Acked Read 4 Done Done Sent false
26921 8380 Acked Write 7 Done Done Sent false
26922 8381 Acked Read 6 Done Done Sent false
26923 8382 Acked Read 2 Done Done Sent false
26924 8383 Acked Write 1 Done Done Sent false
26925 8384 Acked Read 3 Done Done Sent false
26926 8385 Acked Read 1 Done Done Sent false
26927 8386 Acked Read 5 Done Done Sent false
26928 8387 Acked Read 10 Done Done Sent false
26929 8388 Acked Write 10 Done Done Sent false
26930 8389 Acked Read 8 Done Done Sent false
26931 8390 Acked Write 7 Done Done Sent false
26932 8391 Acked Flush 0 Done Done Sent false
26933 8392 Acked Write 4 Done Done Sent false
26934 8393 Acked Write 10 Done Done Sent false
26935 8394 Acked Flush 0 Done Done Sent false
26936 8395 Acked Read 9 Done Done Sent false
26937 8396 Acked Write 10 Done Done Sent false
26938 8397 Acked Write 10 Done Done Sent false
26939 8398 Acked Write 7 Done Done Sent false
26940 8399 Acked Read 8 Done Done Sent false
26941 8400 Acked Write 10 Done Done Sent false
26942 8401 Acked Write 3 Done Done Sent false
26943 8402 Acked Write 1 Done Done Sent false
26944 8403 Acked Read 3 Done Done Sent false
26945 8404 Acked Flush 0 Done Done Sent false
26946 8405 Acked Flush 0 Done Done Sent false
26947 8406 Acked Flush 0 Done Done Sent false
26948 8407 Acked Flush 0 Done Done Sent false
26949 8408 Acked Read 9 Done Done Sent false
26950 8409 Acked Read 1 Done Done Sent false
26951 8410 Acked Write 5 Done Done Sent false
26952 8411 Acked Flush 0 Done Done Sent false
26953 8412 Acked Read 5 Done Done Sent false
26954 8413 Acked Flush 0 Done Done Sent false
26955 8414 Acked Read 6 Done Done Sent false
26956 8415 Acked Flush 0 Done Done Sent false
26957 8416 Acked Read 2 Done Done Sent false
26958 8417 Acked Read 5 Done Done Sent false
26959 8418 Acked Write 9 Done Done Sent false
26960 8419 Acked Write 7 Done Done Sent false
26961 8420 Acked Read 7 Done Done Sent false
26962 8421 Acked Write 7 Done Done Sent false
26963 8422 Acked Read 8 Done Done Sent false
26964 8423 Acked Read 9 Done Done Sent false
26965 8424 Acked Read 2 Done Done Sent false
26966 8425 Acked Read 10 Done Done Sent false
26967 8426 Acked Read 8 Done Done Sent false
26968 8427 Acked Read 7 Done Done Sent false
26969 8428 Acked Write 9 Done Done Sent false
26970 8429 Acked Write 5 Done Done Sent false
26971 8430 Acked Write 8 Done Done Sent false
26972 8431 Acked Read 4 Done Done Sent false
26973 8432 Acked Read 5 Done Done Sent false
26974 8433 Acked Write 3 Done Done Sent false
26975 8434 Acked Write 2 Done Done Sent false
26976 8435 Acked Read 2 Done Done Sent false
26977 8436 Acked Write 4 Done Done Sent false
26978 8437 Acked Write 9 Done Done Sent false
26979 8438 Acked Write 3 Done Done Sent false
26980 8439 Acked Flush 0 Done Done Sent false
26981 8440 Acked Write 3 Done Done Sent false
26982 8441 Acked Write 7 Done Done Sent false
26983 8442 Acked Read 10 Done Done Sent false
26984 8443 Acked Read 1 Done Done Sent false
26985 8444 Acked Read 6 Done Done Sent false
26986 8445 Acked Write 10 Done Done Sent false
26987 8446 Acked Flush 0 Done Done Sent false
26988 8447 Acked Read 3 Done Done Sent false
26989 8448 Acked Read 8 Done Done Sent false
26990 8449 Acked Write 9 Done Done Sent false
26991 8450 Acked Write 3 Done Done Sent false
26992 8451 Acked Write 10 Done Done Sent false
26993 8452 Acked Read 7 Done Done Sent false
26994 8453 Acked Write 1 Done Done Sent false
26995 8454 Acked Read 10 Done Done Sent false
26996 8455 Acked Write 3 Done Done Sent false
26997 8456 Acked Flush 0 Done Done Sent false
26998 8457 Acked Write 7 Done Done Sent false
26999 8458 Acked Read 7 Done Done Sent false
27000 8459 Acked Flush 0 Done Done Sent false
27001 8460 Acked Write 8 Done Done Sent false
27002 8461 Acked Write 3 Done Done Sent false
27003 8462 Acked Write 9 Done Done Sent false
27004 8463 Acked Write 5 Done Done Sent false
27005 8464 Acked Read 7 Done Done Sent false
27006 8465 Acked Write 7 Done Done Sent false
27007 8466 Acked Write 5 Done Done Sent false
27008 8467 Acked Write 7 Done Done Sent false
27009 8468 Acked Read 6 Done Done Sent false
27010 8469 Acked Read 6 Done Done Sent false
27011 8470 Acked Read 3 Done Done Sent false
27012 8471 Acked Write 6 Done Done Sent false
27013 8472 Acked Write 7 Done Done Sent false
27014 8473 Acked Read 9 Done Done Sent false
27015 8474 Acked Write 10 Done Done Sent false
27016 8475 Acked Write 6 Done Done Sent false
27017 8476 Acked Flush 0 Done Done Sent false
27018 8477 Acked Write 10 Done Done Sent false
27019 8478 Acked Write 3 Done Done Sent false
27020 8479 Acked Write 6 Done Done Sent false
27021 8480 Acked Read 4 Done Done Sent false
27022 8481 Acked Write 7 Done Done Sent false
27023 8482 Acked Write 2 Done Done Sent false
27024 8483 Acked Read 5 Done Done Sent false
27025 8484 Acked Write 3 Done Done Sent false
27026 8485 Acked Read 9 Done Done Sent false
27027 8486 Acked Write 5 Done Done Sent false
27028 8487 Acked Read 3 Done Done Sent false
27029 8488 Acked Read 1 Done Done Sent false
27030 8489 Acked Write 7 Done Done Sent false
27031 8490 Acked Flush 0 Done Done Sent false
27032 8491 Acked Write 9 Done Done Sent false
27033 8492 Acked Write 5 Done Done Sent false
27034 8493 Acked Flush 0 Done Done Sent false
27035 8494 Acked Flush 0 Done Done Sent false
27036 8495 Acked Write 2 Done Done Sent false
27037 8496 Acked Read 5 Done Done Sent false
27038 8497 Acked Write 5 Done Done Sent false
27039 8498 Acked Read 10 Done Done Sent false
27040 8499 Acked Read 3 Done Done Sent false
27041 8500 Acked Read 6 Done Done Sent false
27042 8501 Acked Write 2 Done Done Sent false
27043 8502 Acked Read 9 Done Done Sent false
27044 8503 Acked Read 10 Done Done Sent false
27045 8504 Acked Write 1 Done Done Sent false
27046 8505 Acked Read 3 Done Done Sent false
27047 8506 Acked Read 4 Done Done Sent false
27048 8507 Acked Read 2 Done Done Sent false
27049 8508 Acked Write 3 Done Done Sent false
27050 8509 Acked Read 7 Done Done Sent false
27051 8510 Acked Write 6 Done Done Sent false
27052 8511 Acked Write 2 Done Done Sent false
27053 8512 Acked Read 7 Done Done Sent false
27054 8513 Acked Read 10 Done Done Sent false
27055 8514 Acked Flush 0 Done Done Sent false
27056 8515 Acked Read 6 Done Done Sent false
27057 8516 Acked Read 5 Done Done Sent false
27058 8517 Acked Read 1 Done Done Sent false
27059 8518 Acked Read 2 Done Done Sent false
27060 8519 Acked Write 8 Done Done Sent false
27061 8520 Acked Read 10 Done Done Sent false
27062 8521 Acked Read 1 Done Done Sent false
27063 8522 Acked Flush 0 Done Done Sent false
27064 8523 Acked Write 2 Done Done Sent false
27065 8524 Acked Read 4 Done Done Sent false
27066 8525 Acked Write 10 Done Done Sent false
27067 8526 Acked Write 9 Done Done Sent false
27068 8527 Acked Read 4 Done Done Sent false
27069 8528 Acked Write 6 Done Done Sent false
27070 8529 Acked Write 5 Done Done Sent false
27071 8530 Acked Read 6 Done Done Sent false
27072 8531 Acked Read 4 Done Done Sent false
27073 8532 Acked Read 3 Done Done Sent false
27074 8533 Acked Write 1 Done Done Sent false
27075 8534 Acked Write 6 Done Done Sent false
27076 8535 Acked Read 4 Done Done Sent false
27077 8536 Acked Read 10 Done Done Sent false
27078 8537 Acked Write 9 Done Done Sent false
27079 8538 Acked Write 2 Done Done Sent false
27080 8539 Acked Write 8 Done Done Sent false
27081 8540 Acked Read 9 Done Done Sent false
27082 8541 Acked Read 8 Done Done Sent false
27083 8542 Acked Read 10 Done Done Sent false
27084 8543 Acked Write 4 Done Done Sent false
27085 8544 Acked Read 3 Done Done Sent false
27086 8545 Acked Flush 0 Done Done Sent false
27087 STATES DS:0 DS:1 DS:2 TOTAL
27088 Sent 0 0 302 302
27089 Done 302 302 0 604
27090 Skipped 0 0 0 0
27091 Error 0 0 0 0
27092 Last Flush: 8545 8545 8243
27093 Downstairs last five completed: 8243 8242 8241 8240 8239
27094 Upstairs last five completed: 8545 8544 8543 8542 8541
27095 CLIENT: Up:0 ds:302 act:2
270962024-12-04T13:36:22.038ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
270972024-12-04T13:36:22.039ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53373) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
270982024-12-04T13:36:22.040ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
270992024-12-04T13:36:22.040ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271002024-12-04T13:36:22.040ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271012024-12-04T13:36:22.040ZINFOcrucible: send last flush ID to this DS: 8243 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271022024-12-04T13:36:22.040ZINFOcrucible: Replied this last flush ID: 8243 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271032024-12-04T13:36:22.040ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271042024-12-04T13:36:22.040ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271052024-12-04T13:36:22.040ZINFOcrucible: [2] client re-new 302 jobs since flush 8243 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271062024-12-04T13:36:22.041ZINFOcrucible: [2] Replayed 302 jobs since flush: 8243 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
27107 ----------------------------------------------------------------
27108 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27109 Downstairs last five completed: 8545 8544 8543 8542 8541
27110 Upstairs last five completed: 8545 8544 8543 8542 8541
27111 CLIENT: Up:0 ds:0 act:3
27112 Replay: All jobs finished
27113 Main task has work to do, go find it
27114 got dsc Stop(2)
27115 stop 2
27116 [8830] Got stop action so:false kr:false
27117 Dec 04 13:36:24.273 INFO request completed, latency_us: 336, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 75d57035-30d7-44cd-9678-5062ae1fc244, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
271182024-12-04T13:36:24.273ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271192024-12-04T13:36:24.273ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271202024-12-04T13:36:24.273ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271212024-12-04T13:36:24.273ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
271222024-12-04T13:36:24.273ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
27123 Dec 04 13:36:24.273 INFO request completed, latency_us: 138, response_code: 200, uri: /state/cid/2, method: GET, req_id: 0bd862e2-d4a1-4b96-93f9-184ba22a0b9d, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
271242024-12-04T13:36:24.273ZINFOcrucible: client task is exiting = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
27125 [8830] Exited with: ExitStatus(unix_wait_status(9))
27126 [8830][2] reports Exit
27127 Dec 04 13:36:28.276 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9fb791f9-db12-4763-85cb-5afbf77f7956, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
27128 001/300 Flush
27129 002/300 Read block 4251 len 36864
27130 003/300 Write block 4158 len 12288 data: 9 13 13
27131 004/300 Read block 3330 len 32768
27132 005/300 Write block 2964 len 28672 data: 6 6 5 5 5 5 5
27133 006/300 Read block 693 len 36864
27134 007/300 Read block 917 len 8192
27135 008/300 Write block 3718 len 36864 data: 7 6 6 6 6 5 5 5 2
27136 009/300 Flush
27137 010/300 Read block 2444 len 20480
27138 011/300 Write block 515 len 4096 data: 5
27139 012/300 Write block 2311 len 8192 data: 3 3
27140 013/300 Write block 3028 len 16384 data: 5 5 3 3
27141 014/300 Read block 4122 len 32768
27142 015/300 Read block 89 len 32768
27143 016/300 Read block 2766 len 4096
27144 017/300 Write block 1619 len 20480 data: 5 4 4 5 4
27145 018/300 Write block 3308 len 40960 data: 5 5 6 5 5 5 5 7 6 5
27146 019/300 Flush
27147 020/300 Write block 1940 len 40960 data: 6 6 7 6 6 8 7 8 8 9
27148 021/300 Read block 500 len 36864
27149 022/300 Write block 4412 len 40960 data: 3 4 4 3 4 4 6 8 8 8
27150 023/300 Read block 1268 len 40960
27151 024/300 Write block 3349 len 12288 data: 8 7 6
27152 025/300 Read block 2637 len 32768
27153 026/300 Write block 1016 len 40960 data: 8 7 7 7 6 5 4 6 6 7
27154 027/300 Read block 894 len 32768
27155 028/300 Read block 2412 len 8192
27156 029/300 Read block 271 len 20480
27157 030/300 Write block 4191 len 20480 data: 5 5 6 6 5
27158 031/300 Read block 243 len 36864
27159 032/300 Read block 152 len 28672
27160 033/300 Read block 2833 len 12288
27161 034/300 Flush
27162 035/300 Write block 4248 len 40960 data: 9 10 7 6 5 5 5 4 4 3
27163 036/300 Read block 694 len 20480
27164 037/300 Read block 2156 len 20480
27165 038/300 Flush
27166 039/300 Write block 3056 len 16384 data: 8 7 6 5
27167 040/300 Write block 1670 len 36864 data: 2 3 4 4 5 5 5 6 8
27168 041/300 Read block 3859 len 12288
27169 042/300 Write block 1414 len 16384 data: 7 7 6 6
27170 043/300 Flush
27171 044/300 Read block 1973 len 12288
27172 045/300 Flush
27173 046/300 Write block 3495 len 36864 data: 6 6 5 5 6 6 8 8 8
27174 047/300 Write block 3386 len 36864 data: 4 3 4 4 5 5 5 5 5
27175 048/300 Read block 1074 len 16384
27176 049/300 Write block 2331 len 8192 data: 2 3
27177 050/300 Write block 2631 len 8192 data: 5 6
27178 051/300 Flush
27179 052/300 Write block 4372 len 24576 data: 5 4 5 5 6 6
27180 053/300 Write block 4042 len 4096 data: 3
27181 054/300 Read block 1430 len 20480
27182 055/300 Read block 1442 len 12288
27183 056/300 Read block 1780 len 32768
27184 057/300 Write block 1572 len 20480 data: 3 3 2 2 4
27185 058/300 Read block 2400 len 12288
27186 059/300 Read block 572 len 32768
27187 060/300 Write block 3488 len 8192 data: 9 8
27188 061/300 Read block 2413 len 12288
27189 062/300 Write block 1583 len 24576 data: 5 4 4 4 3 3
27190 063/300 Read block 1220 len 20480
27191 064/300 Write block 3975 len 8192 data: 7 4
27192 065/300 Read block 2641 len 4096
27193 066/300 Read block 2279 len 40960
27194 067/300 Read block 2567 len 32768
27195 068/300 Read block 537 len 40960
27196 069/300 Read block 1268 len 24576
27197 070/300 Read block 4294 len 24576
27198 071/300 Read block 1303 len 40960
27199 072/300 Write block 409 len 36864 data: 4 5 5 5 5 4 5 5 6
27200 073/300 Write block 4765 len 40960 data: 5 5 5 5 6 7 7 8 6 6
27201 074/300 Write block 1016 len 12288 data: 9 8 8
27202 075/300 Flush
27203 076/300 Read block 2860 len 24576
27204 077/300 Read block 1426 len 36864
27205 078/300 Read block 1418 len 16384
27206 079/300 Read block 918 len 16384
27207 080/300 Write block 1524 len 28672 data: 7 10 11 10 9 9 8
27208 081/300 Write block 2486 len 8192 data: 7 7
27209 082/300 Read block 4159 len 20480
27210 083/300 Flush
27211 084/300 Read block 4775 len 16384
27212 085/300 Read block 4972 len 32768
27213 086/300 Read block 3490 len 40960
27214 087/300 Read block 3591 len 36864
27215 088/300 Write block 2470 len 20480 data: 7 6 6 6 7
27216 089/300 Flush
27217 090/300 Write block 408 len 12288 data: 4 5 6
27218 091/300 Write block 4657 len 28672 data: 2 2 4 5 5 5 4
27219 092/300 Read block 1668 len 32768
27220 093/300 Write block 2066 len 36864 data: 2 2 3 3 3 4 5 5 4
27221 094/300 Write block 4745 len 8192 data: 3 4
27222 095/300 Flush
27223 096/300 Write block 201 len 24576 data: 4 4 3 4 3 5
27224 097/300 Read block 566 len 16384
27225 098/300 Read block 2993 len 24576
27226 099/300 Write block 3091 len 24576 data: 5 5 4 3 3 5
27227 100/300 Flush
27228 101/300 Flush
27229 102/300 Read block 1581 len 4096
27230 103/300 Read block 3529 len 28672
27231 104/300 Read block 4158 len 4096
27232 105/300 Write block 529 len 28672 data: 3 3 3 3 4 6 5
27233 106/300 Read block 2970 len 32768
27234 107/300 Flush
27235 108/300 Read block 3158 len 20480
27236 109/300 Read block 3834 len 12288
27237 110/300 Read block 3064 len 16384
27238 111/300 Write block 1460 len 36864 data: 6 6 5 4 4 4 4 3 5
27239 112/300 Read block 1365 len 4096
27240 113/300 Read block 212 len 32768
27241 114/300 Write block 363 len 12288 data: 4 5 5
27242 115/300 Read block 4220 len 28672
27243 116/300 Write block 2836 len 32768 data: 5 5 8 8 7 6 6 7
27244 117/300 Flush
27245 118/300 Flush
27246 119/300 Read block 1236 len 20480
27247 120/300 Read block 3303 len 36864
27248 121/300 Write block 1040 len 40960 data: 4 4 4 3 3 4 4 5 5 5
27249 122/300 Write block 1496 len 12288 data: 5 6 7
27250 123/300 Read block 4240 len 20480
27251 124/300 Write block 3537 len 16384 data: 4 5 5 5
27252 125/300 Read block 486 len 8192
27253 126/300 Read block 751 len 20480
27254 127/300 Write block 3086 len 20480 data: 2 3 4 5 5
27255 128/300 Read block 1145 len 16384
27256 129/300 Read block 4088 len 16384
27257 130/300 Read block 2351 len 24576
27258 131/300 Read block 583 len 8192
27259 132/300 Read block 3022 len 36864
27260 133/300 Write block 3763 len 24576 data: 7 8 8 8 6 6
27261 134/300 Write block 2868 len 24576 data: 7 8 7 7 10 9
27262 135/300 Write block 3921 len 32768 data: 5 5 4 4 5 6 6 6
27263 136/300 Read block 2556 len 8192
27264 137/300 Write block 591 len 12288 data: 5 6 6
27265 138/300 Read block 4088 len 36864
27266 139/300 Write block 1738 len 20480 data: 4 4 4 7 9
27267 140/300 Write block 2532 len 24576 data: 4 4 4 5 7 10
27268 141/300 Read block 4753 len 4096
27269 142/300 Write block 4590 len 20480 data: 6 6 5 6 6
27270 143/300 Read block 4564 len 40960
27271 144/300 Read block 157 len 12288
27272 145/300 Read block 4496 len 4096
27273 146/300 Read block 1436 len 12288
27274 147/300 Flush
27275 148/300 Read block 3873 len 36864
27276 149/300 Read block 1600 len 12288
27277 150/300 Write block 3709 len 36864 data: 3 3 3 3 3 5 7 9 10
27278 151/300 Write block 1247 len 8192 data: 6 5
27279 152/300 Read block 2276 len 4096
27280 153/300 Read block 2237 len 24576
27281 154/300 Read block 534 len 36864
27282 155/300 Read block 1382 len 12288
27283 156/300 Read block 2565 len 8192
27284 157/300 Flush
27285 158/300 Write block 318 len 40960 data: 6 4 4 4 5 6 6 4 2 4
27286 159/300 Read block 2116 len 36864
27287 160/300 Read block 1468 len 40960
27288 161/300 Read block 1363 len 4096
27289 162/300 Flush
27290 163/300 Read block 3222 len 32768
27291 164/300 Write block 3808 len 28672 data: 3 5 4 5 5 4 5
27292 165/300 Flush
27293 166/300 Read block 4104 len 28672
27294 167/300 Read block 670 len 8192
27295 168/300 Read block 596 len 8192
27296 169/300 Write block 4759 len 28672 data: 4 4 4 4 4 6 6
27297 170/300 Read block 3685 len 4096
27298 171/300 Read block 3826 len 40960
27299 172/300 Read block 801 len 4096
27300 173/300 Write block 4703 len 20480 data: 6 7 5 8 7
27301 174/300 Write block 2306 len 24576 data: 5 5 4 3 3 4
27302 175/300 Write block 660 len 12288 data: 4 4 6
27303 176/300 Read block 1563 len 24576
27304 177/300 Write block 2650 len 8192 data: 3 6
27305 178/300 Write block 2719 len 20480 data: 4 6 6 5 4
27306 179/300 Write block 3775 len 32768 data: 8 8 7 7 6 6 5 4
27307 180/300 Write block 314 len 24576 data: 5 6 6 5 7 5
27308 181/300 Write block 1181 len 8192 data: 4 6
27309 182/300 Write block 4461 len 4096 data: 7
27310 183/300 Read block 2974 len 16384
27311 184/300 Read block 1904 len 20480
27312 185/300 Write block 4296 len 24576 data: 5 5 5 6 6 5
27313 186/300 Read block 2440 len 36864
27314 187/300 Write block 1721 len 28672 data: 4 3 4 4 4 5 5
27315 188/300 Read block 572 len 28672
27316 189/300 Write block 4443 len 20480 data: 7 6 5 5 4
27317 190/300 Read block 1986 len 32768
27318 191/300 Write block 2056 len 8192 data: 7 7
27319 192/300 Read block 2982 len 16384
27320 193/300 Read block 2141 len 4096
27321 194/300 Read block 4302 len 32768
27322 195/300 Read block 4432 len 24576
27323 196/300 Read block 1947 len 40960
27324 197/300 Read block 2560 len 8192
27325 198/300 Write block 1561 len 32768 data: 6 5 5 5 7 7 7 7
27326 199/300 Read block 575 len 16384
27327 200/300 Write block 889 len 12288 data: 4 4 5
27328 201/300 Read block 431 len 24576
27329 202/300 Read block 3317 len 4096
27330 203/300 Read block 4268 len 20480
27331 204/300 Read block 1101 len 32768
27332 205/300 Write block 2072 len 12288 data: 6 6 5
27333 206/300 Write block 3212 len 4096 data: 7
27334 207/300 Write block 1145 len 16384 data: 4 4 5 5
27335 208/300 Write block 4034 len 28672 data: 6 7 5 4 3 2 3
27336 209/300 Write block 4172 len 24576 data: 7 6 5 7 7 6
27337 210/300 Write block 4119 len 20480 data: 7 8 8 8 6
27338 211/300 Write block 4258 len 12288 data: 2 4 5
27339 212/300 Read block 1394 len 24576
27340 213/300 Write block 2030 len 28672 data: 4 6 4 4 3 3 3
27341 214/300 Write block 2445 len 36864 data: 5 5 4 5 4 3 3 4 3
27342 215/300 Write block 4783 len 24576 data: 6 5 4 4 4 4
27343 216/300 Write block 3746 len 28672 data: 5 5 3 2 4 4 5
27344 217/300 Read block 325 len 16384
27345 218/300 Read block 189 len 8192
27346 219/300 Write block 3133 len 12288 data: 4 4 5
27347 220/300 Write block 4183 len 8192 data: 4 4
27348 221/300 Read block 3587 len 16384
27349 222/300 Write block 4694 len 32768 data: 5 5 6 7 7 6 6 4
27350 223/300 Write block 3764 len 24576 data: 9 9 9 7 7 7
27351 224/300 Write block 1013 len 4096 data: 5
27352 225/300 Write block 4501 len 4096 data: 3
27353 226/300 Read block 4436 len 28672
27354 227/300 Read block 4044 len 16384
27355 228/300 Read block 986 len 32768
27356 229/300 Read block 2567 len 24576
27357 230/300 Read block 1504 len 32768
27358 231/300 Write block 1983 len 36864 data: 5 6 7 7 6 7 6 6 7
27359 232/300 Write block 1561 len 36864 data: 7 6 6 6 8 8 8 8 6
27360 233/300 Write block 27 len 32768 data: 6 7 7 6 7 5 5 5
27361 234/300 Write block 3814 len 28672 data: 6 5 6 7 5 4 4
27362 235/300 Read block 3114 len 40960
27363 236/300 Read block 360 len 24576
27364 237/300 Read block 2672 len 24576
27365 238/300 Read block 2943 len 16384
27366 239/300 Read block 1547 len 4096
27367 240/300 Read block 672 len 4096
27368 241/300 Write block 2985 len 40960 data: 9 8 7 8 7 7 6 6 5 5
27369 242/300 Write block 3783 len 32768 data: 5 3 3 3 4 6 6 5
27370 243/300 Read block 3494 len 32768
27371 244/300 Read block 4648 len 4096
27372 245/300 Read block 243 len 40960
27373 246/300 Read block 2821 len 12288
27374 247/300 Read block 2972 len 20480
27375 248/300 Write block 1018 len 28672 data: 9 8 7 6 5 7 7
27376 249/300 Write block 1828 len 12288 data: 4 5 4
27377 250/300 Flush
27378 251/300 Read block 2179 len 28672
27379 252/300 Write block 3171 len 40960 data: 5 6 5 4 6 7 6 5 6 6
27380 253/300 Write block 3569 len 12288 data: 6 6 7
27381 254/300 Read block 4152 len 8192
27382 255/300 Read block 689 len 12288
27383 256/300 Write block 1740 len 32768 data: 5 8 10 9 10 8 10 11
27384 257/300 Write block 118 len 8192 data: 7 6
27385 258/300 Write block 965 len 16384 data: 4 4 6 5
27386 259/300 Flush
27387 260/300 Read block 4625 len 32768
27388 261/300 Write block 331 len 40960 data: 6 5 4 4 4 3 3 3 4 4
27389 262/300 Read block 595 len 8192
27390 263/300 Flush
27391 264/300 Read block 2160 len 12288
27392 265/300 Read block 4406 len 20480
27393 266/300 Write block 4046 len 40960 data: 4 5 5 3 2 3 3 4 4 4
27394 267/300 Write block 2578 len 32768 data: 5 4 5 5 3 2 2 3
27395 268/300 Read block 4198 len 40960
27396 269/300 Read block 2823 len 12288
27397 270/300 Read block 2609 len 8192
27398 271/300 Write block 2526 len 40960 data: 5 5 4 5 4 4 5 5 5 6
27399 272/300 Flush
27400 273/300 Read block 612 len 40960
27401 274/300 Read block 2965 len 32768
27402 275/300 Write block 3431 len 36864 data: 4 7 7 7 9 9 8 8 8
27403 276/300 Read block 2529 len 32768
27404 277/300 Write block 4379 len 40960 data: 6 4 4 4 4 4 5 4 6 6
27405 278/300 Read block 4359 len 16384
27406 279/300 Write block 970 len 24576 data: 5 5 5 5 5 5
27407 280/300 Read block 4470 len 12288
27408 281/300 Flush
27409 282/300 Read block 2070 len 12288
27410 283/300 Write block 4280 len 32768 data: 2 3 3 3 4 4 4 4
27411 284/300 Write block 2264 len 8192 data: 4 5
27412 285/300 Write block 1209 len 40960 data: 7 7 6 7 7 7 9 8 6 6
27413 286/300 Read block 2388 len 28672
27414 287/300 Write block 1538 len 20480 data: 4 4 5 5 6
27415 288/300 Read block 4370 len 20480
27416 289/300 Write block 3595 len 32768 data: 5 3 4 5 6 6 6 6
27417 290/300 Read block 118 len 32768
27418 291/300 Read block 1504 len 28672
27419 292/300 Write block 831 len 28672 data: 7 5 7 6 6 5 6
27420 293/300 Write block 1072 len 36864 data: 6 5 5 4 6 9 9 7 8
27421 294/300 Write block 1340 len 16384 data: 7 8 5 7
27422 295/300 Read block 4040 len 16384
27423 296/300 Read block 1631 len 16384
27424 297/300 Write block 4339 len 24576 data: 9 6 6 6 8 8
27425 298/300 Read block 486 len 40960
27426 299/300 Read block 4410 len 12288
27427 300/300 Write block 4268 len 8192 data: 5 7
27428 301/300 Read block 1485 len 16384
27429 Main task has work to do, go find it
27430 got dsc Start(2)
27431 start 2
27432 Dec 04 13:36:28.513 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/2, method: POST, req_id: a5a25416-5b81-4a9e-8095-cee239b1e8ac, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
27433 Replay: started 2, returned:Ok(())
27434 ----------------------------------------------------------------
27435 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27436 States: Active Active Offline
27437 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27438 8546 Acked Flush 0 Done Done Sent false
27439 8547 Acked Read 9 Done Done Sent false
27440 8548 Acked Write 3 Done Done Sent false
27441 8549 Acked Read 8 Done Done Sent false
27442 8550 Acked Write 7 Done Done Sent false
27443 8551 Acked Read 9 Done Done Sent false
27444 8552 Acked Read 2 Done Done Sent false
27445 8553 Acked Write 9 Done Done Sent false
27446 8554 Acked Flush 0 Done Done Sent false
27447 8555 Acked Read 5 Done Done Sent false
27448 8556 Acked Write 1 Done Done Sent false
27449 8557 Acked Write 2 Done Done Sent false
27450 8558 Acked Write 4 Done Done Sent false
27451 8559 Acked Read 8 Done Done Sent false
27452 8560 Acked Read 8 Done Done Sent false
27453 8561 Acked Read 1 Done Done Sent false
27454 8562 Acked Write 5 Done Done Sent false
27455 8563 Acked Write 10 Done Done Sent false
27456 8564 Acked Flush 0 Done Done Sent false
27457 8565 Acked Write 10 Done Done Sent false
27458 8566 Acked Read 9 Done Done Sent false
27459 8567 Acked Write 10 Done Done Sent false
27460 8568 Acked Read 10 Done Done Sent false
27461 8569 Acked Write 3 Done Done Sent false
27462 8570 Acked Read 8 Done Done Sent false
27463 8571 Acked Write 10 Done Done Sent false
27464 8572 Acked Read 8 Done Done Sent false
27465 8573 Acked Read 2 Done Done Sent false
27466 8574 Acked Read 5 Done Done Sent false
27467 8575 Acked Write 5 Done Done Sent false
27468 8576 Acked Read 9 Done Done Sent false
27469 8577 Acked Read 7 Done Done Sent false
27470 8578 Acked Read 3 Done Done Sent false
27471 8579 Acked Flush 0 Done Done Sent false
27472 8580 Acked Write 10 Done Done Sent false
27473 8581 Acked Read 5 Done Done Sent false
27474 8582 Acked Read 5 Done Done Sent false
27475 8583 Acked Flush 0 Done Done Sent false
27476 8584 Acked Write 4 Done Done Sent false
27477 8585 Acked Write 9 Done Done Sent false
27478 8586 Acked Read 3 Done Done Sent false
27479 8587 Acked Write 4 Done Done Sent false
27480 8588 Acked Flush 0 Done Done Sent false
27481 8589 Acked Read 3 Done Done Sent false
27482 8590 Acked Flush 0 Done Done Sent false
27483 8591 Acked Write 9 Done Done Sent false
27484 8592 Acked Write 9 Done Done Sent false
27485 8593 Acked Read 4 Done Done Sent false
27486 8594 Acked Write 2 Done Done Sent false
27487 8595 Acked Write 2 Done Done Sent false
27488 8596 Acked Flush 0 Done Done Sent false
27489 8597 Acked Write 6 Done Done Sent false
27490 8598 Acked Write 1 Done Done Sent false
27491 8599 Acked Read 5 Done Done Sent false
27492 8600 Acked Read 3 Done Done Sent false
27493 8601 Acked Read 8 Done Done Sent false
27494 8602 Acked Write 5 Done Done Sent false
27495 8603 Acked Read 3 Done Done Sent false
27496 8604 Acked Read 8 Done Done Sent false
27497 8605 Acked Write 2 Done Done Sent false
27498 8606 Acked Read 3 Done Done Sent false
27499 8607 Acked Write 6 Done Done Sent false
27500 8608 Acked Read 5 Done Done Sent false
27501 8609 Acked Write 2 Done Done Sent false
27502 8610 Acked Read 1 Done Done Sent false
27503 8611 Acked Read 10 Done Done Sent false
27504 8612 Acked Read 8 Done Done Sent false
27505 8613 Acked Read 10 Done Done Sent false
27506 8614 Acked Read 6 Done Done Sent false
27507 8615 Acked Read 6 Done Done Sent false
27508 8616 Acked Read 10 Done Done Sent false
27509 8617 Acked Write 9 Done Done Sent false
27510 8618 Acked Write 10 Done Done Sent false
27511 8619 Acked Write 3 Done Done Sent false
27512 8620 Acked Flush 0 Done Done Sent false
27513 8621 Acked Read 6 Done Done Sent false
27514 8622 Acked Read 9 Done Done Sent false
27515 8623 Acked Read 4 Done Done Sent false
27516 8624 Acked Read 4 Done Done Sent false
27517 8625 Acked Write 7 Done Done Sent false
27518 8626 Acked Write 2 Done Done Sent false
27519 8627 Acked Read 5 Done Done Sent false
27520 8628 Acked Flush 0 Done Done Sent false
27521 8629 Acked Read 4 Done Done Sent false
27522 8630 Acked Read 8 Done Done Sent false
27523 8631 Acked Read 10 Done Done Sent false
27524 8632 Acked Read 9 Done Done Sent false
27525 8633 Acked Write 5 Done Done Sent false
27526 8634 Acked Flush 0 Done Done Sent false
27527 8635 Acked Write 3 Done Done Sent false
27528 8636 Acked Write 7 Done Done Sent false
27529 8637 Acked Read 8 Done Done Sent false
27530 8638 Acked Write 9 Done Done Sent false
27531 8639 Acked Write 2 Done Done Sent false
27532 8640 Acked Flush 0 Done Done Sent false
27533 8641 Acked Write 6 Done Done Sent false
27534 8642 Acked Read 4 Done Done Sent false
27535 8643 Acked Read 6 Done Done Sent false
27536 8644 Acked Write 6 Done Done Sent false
27537 8645 Acked Flush 0 Done Done Sent false
27538 8646 Acked Flush 0 Done Done Sent false
27539 8647 Acked Read 1 Done Done Sent false
27540 8648 Acked Read 7 Done Done Sent false
27541 8649 Acked Read 1 Done Done Sent false
27542 8650 Acked Write 7 Done Done Sent false
27543 8651 Acked Read 8 Done Done Sent false
27544 8652 Acked Flush 0 Done Done Sent false
27545 8653 Acked Read 5 Done Done Sent false
27546 8654 Acked Read 3 Done Done Sent false
27547 8655 Acked Read 4 Done Done Sent false
27548 8656 Acked Write 9 Done Done Sent false
27549 8657 Acked Read 1 Done Done Sent false
27550 8658 Acked Read 8 Done Done Sent false
27551 8659 Acked Write 3 Done Done Sent false
27552 8660 Acked Read 7 Done Done Sent false
27553 8661 Acked Write 8 Done Done Sent false
27554 8662 Acked Flush 0 Done Done Sent false
27555 8663 Acked Flush 0 Done Done Sent false
27556 8664 Acked Read 5 Done Done Sent false
27557 8665 Acked Read 9 Done Done Sent false
27558 8666 Acked Write 10 Done Done Sent false
27559 8667 Acked Write 3 Done Done Sent false
27560 8668 Acked Read 5 Done Done Sent false
27561 8669 Acked Write 4 Done Done Sent false
27562 8670 Acked Read 2 Done Done Sent false
27563 8671 Acked Read 5 Done Done Sent false
27564 8672 Acked Write 5 Done Done Sent false
27565 8673 Acked Read 4 Done Done Sent false
27566 8674 Acked Read 4 Done Done Sent false
27567 8675 Acked Read 6 Done Done Sent false
27568 8676 Acked Read 2 Done Done Sent false
27569 8677 Acked Read 9 Done Done Sent false
27570 8678 Acked Write 6 Done Done Sent false
27571 8679 Acked Write 6 Done Done Sent false
27572 8680 Acked Write 8 Done Done Sent false
27573 8681 Acked Read 2 Done Done Sent false
27574 8682 Acked Write 3 Done Done Sent false
27575 8683 Acked Read 9 Done Done Sent false
27576 8684 Acked Write 5 Done Done Sent false
27577 8685 Acked Write 6 Done Done Sent false
27578 8686 Acked Read 1 Done Done Sent false
27579 8687 Acked Write 5 Done Done Sent false
27580 8688 Acked Read 10 Done Done Sent false
27581 8689 Acked Read 3 Done Done Sent false
27582 8690 Acked Read 1 Done Done Sent false
27583 8691 Acked Read 3 Done Done Sent false
27584 8692 Acked Flush 0 Done Done Sent false
27585 8693 Acked Read 9 Done Done Sent false
27586 8694 Acked Read 3 Done Done Sent false
27587 8695 Acked Write 9 Done Done Sent false
27588 8696 Acked Write 2 Done Done Sent false
27589 8697 Acked Read 1 Done Done Sent false
27590 8698 Acked Read 6 Done Done Sent false
27591 8699 Acked Read 9 Done Done Sent false
27592 8700 Acked Read 3 Done Done Sent false
27593 8701 Acked Read 2 Done Done Sent false
27594 8702 Acked Flush 0 Done Done Sent false
27595 8703 Acked Write 10 Done Done Sent false
27596 8704 Acked Read 9 Done Done Sent false
27597 8705 Acked Read 10 Done Done Sent false
27598 8706 Acked Read 1 Done Done Sent false
27599 8707 Acked Flush 0 Done Done Sent false
27600 8708 Acked Read 8 Done Done Sent false
27601 8709 Acked Write 7 Done Done Sent false
27602 8710 Acked Flush 0 Done Done Sent false
27603 8711 Acked Read 7 Done Done Sent false
27604 8712 Acked Read 2 Done Done Sent false
27605 8713 Acked Read 2 Done Done Sent false
27606 8714 Acked Write 7 Done Done Sent false
27607 8715 Acked Read 1 Done Done Sent false
27608 8716 Acked Read 10 Done Done Sent false
27609 8717 Acked Read 1 Done Done Sent false
27610 8718 Acked Write 5 Done Done Sent false
27611 8719 Acked Write 6 Done Done Sent false
27612 8720 Acked Write 3 Done Done Sent false
27613 8721 Acked Read 6 Done Done Sent false
27614 8722 Acked Write 2 Done Done Sent false
27615 8723 Acked Write 5 Done Done Sent false
27616 8724 Acked Write 8 Done Done Sent false
27617 8725 Acked Write 6 Done Done Sent false
27618 8726 Acked Write 2 Done Done Sent false
27619 8727 Acked Write 1 Done Done Sent false
27620 8728 Acked Read 4 Done Done Sent false
27621 8729 Acked Read 5 Done Done Sent false
27622 8730 Acked Write 6 Done Done Sent false
27623 8731 Acked Read 9 Done Done Sent false
27624 8732 Acked Write 7 Done Done Sent false
27625 8733 Acked Read 7 Done Done Sent false
27626 8734 Acked Write 5 Done Done Sent false
27627 8735 Acked Read 8 Done Done Sent false
27628 8736 Acked Write 2 Done Done Sent false
27629 8737 Acked Read 4 Done Done Sent false
27630 8738 Acked Read 1 Done Done Sent false
27631 8739 Acked Read 8 Done Done Sent false
27632 8740 Acked Read 6 Done Done Sent false
27633 8741 Acked Read 10 Done Done Sent false
27634 8742 Acked Read 2 Done Done Sent false
27635 8743 Acked Write 8 Done Done Sent false
27636 8744 Acked Read 4 Done Done Sent false
27637 8745 Acked Write 3 Done Done Sent false
27638 8746 Acked Read 6 Done Done Sent false
27639 8747 Acked Read 1 Done Done Sent false
27640 8748 Acked Read 5 Done Done Sent false
27641 8749 Acked Read 8 Done Done Sent false
27642 8750 Acked Write 3 Done Done Sent false
27643 8751 Acked Write 1 Done Done Sent false
27644 8752 Acked Write 4 Done Done Sent false
27645 8753 Acked Write 7 Done Done Sent false
27646 8754 Acked Write 6 Done Done Sent false
27647 8755 Acked Write 5 Done Done Sent false
27648 8756 Acked Write 3 Done Done Sent false
27649 8757 Acked Read 6 Done Done Sent false
27650 8758 Acked Write 7 Done Done Sent false
27651 8759 Acked Write 9 Done Done Sent false
27652 8760 Acked Write 6 Done Done Sent false
27653 8761 Acked Write 7 Done Done Sent false
27654 8762 Acked Read 4 Done Done Sent false
27655 8763 Acked Read 2 Done Done Sent false
27656 8764 Acked Write 3 Done Done Sent false
27657 8765 Acked Write 2 Done Done Sent false
27658 8766 Acked Read 4 Done Done Sent false
27659 8767 Acked Write 8 Done Done Sent false
27660 8768 Acked Write 6 Done Done Sent false
27661 8769 Acked Write 1 Done Done Sent false
27662 8770 Acked Write 1 Done Done Sent false
27663 8771 Acked Read 7 Done Done Sent false
27664 8772 Acked Read 4 Done Done Sent false
27665 8773 Acked Read 8 Done Done Sent false
27666 8774 Acked Read 6 Done Done Sent false
27667 8775 Acked Read 8 Done Done Sent false
27668 8776 Acked Write 9 Done Done Sent false
27669 8777 Acked Write 9 Done Done Sent false
27670 8778 Acked Write 8 Done Done Sent false
27671 8779 Acked Write 7 Done Done Sent false
27672 8780 Acked Read 10 Done Done Sent false
27673 8781 Acked Read 6 Done Done Sent false
27674 8782 Acked Read 6 Done Done Sent false
27675 8783 Acked Read 4 Done Done Sent false
27676 8784 Acked Read 1 Done Done Sent false
27677 8785 Acked Read 1 Done Done Sent false
27678 8786 Acked Write 10 Done Done Sent false
27679 8787 Acked Write 8 Done Done Sent false
27680 8788 Acked Read 8 Done Done Sent false
27681 8789 Acked Read 1 Done Done Sent false
27682 8790 Acked Read 10 Done Done Sent false
27683 8791 Acked Read 3 Done Done Sent false
27684 8792 Acked Read 5 Done Done Sent false
27685 8793 Acked Write 7 Done Done Sent false
27686 8794 Acked Write 3 Done Done Sent false
27687 8795 Acked Flush 0 Done Done Sent false
27688 8796 Acked Read 7 Done Done Sent false
27689 8797 Acked Write 10 Done Done Sent false
27690 8798 Acked Write 3 Done Done Sent false
27691 8799 Acked Read 2 Done Done Sent false
27692 8800 Acked Read 3 Done Done Sent false
27693 8801 Acked Write 8 Done Done Sent false
27694 8802 Acked Write 2 Done Done Sent false
27695 8803 Acked Write 4 Done Done Sent false
27696 8804 Acked Flush 0 Done Done Sent false
27697 8805 Acked Read 8 Done Done Sent false
27698 8806 Acked Write 10 Done Done Sent false
27699 8807 Acked Read 2 Done Done Sent false
27700 8808 Acked Flush 0 Done Done Sent false
27701 8809 Acked Read 3 Done Done Sent false
27702 8810 Acked Read 5 Done Done Sent false
27703 8811 Acked Write 10 Done Done Sent false
27704 8812 Acked Write 8 Done Done Sent false
27705 8813 Acked Read 10 Done Done Sent false
27706 8814 Acked Read 3 Done Done Sent false
27707 8815 Acked Read 2 Done Done Sent false
27708 8816 Acked Write 10 Done Done Sent false
27709 8817 Acked Flush 0 Done Done Sent false
27710 8818 Acked Read 10 Done Done Sent false
27711 8819 Acked Read 8 Done Done Sent false
27712 8820 Acked Write 9 Done Done Sent false
27713 8821 Acked Read 8 Done Done Sent false
27714 8822 Acked Write 10 Done Done Sent false
27715 8823 Acked Read 4 Done Done Sent false
27716 8824 Acked Write 6 Done Done Sent false
27717 8825 Acked Read 3 Done Done Sent false
27718 8826 Acked Flush 0 Done Done Sent false
27719 8827 Acked Read 3 Done Done Sent false
27720 8828 Acked Write 8 Done Done Sent false
27721 8829 Acked Write 2 Done Done Sent false
27722 8830 Acked Write 10 Done Done Sent false
27723 8831 Acked Read 7 Done Done Sent false
27724 8832 Acked Write 5 Done Done Sent false
27725 8833 Acked Read 5 Done Done Sent false
27726 8834 Acked Write 8 Done Done Sent false
27727 8835 Acked Read 8 Done Done Sent false
27728 8836 Acked Read 7 Done Done Sent false
27729 8837 Acked Write 7 Done Done Sent false
27730 8838 Acked Write 9 Done Done Sent false
27731 8839 Acked Write 4 Done Done Sent false
27732 8840 Acked Read 4 Done Done Sent false
27733 8841 Acked Read 4 Done Done Sent false
27734 8842 Acked Write 6 Done Done Sent false
27735 8843 Acked Read 10 Done Done Sent false
27736 8844 Acked Read 3 Done Done Sent false
27737 8845 Acked Write 2 Done Done Sent false
27738 8846 Acked Read 4 Done Done Sent false
27739 STATES DS:0 DS:1 DS:2 TOTAL
27740 Sent 0 0 301 301
27741 Done 301 301 0 602
27742 Skipped 0 0 0 0
27743 Error 0 0 0 0
27744 Last Flush: 8826 8826 8545
27745 Downstairs last five completed: 8545 8544 8543 8542 8541
27746 Upstairs last five completed: 8846 8845 8844 8843 8842
27747 CLIENT: Up:0 ds:301 act:2
27748 [8830] got start action
27749 [8830] I am going to restart
27750 Starting downstairs at port 8830
27751 Make output file at "/tmp/dsc/downstairs-8830.txt"
27752 [8830][2] reports Starting
27753 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1229)
27754 [8830][2] reports Running
27755 ----------------------------------------------------------------
27756 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27757 States: Active Active Offline
27758 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27759 8546 Acked Flush 0 Done Done Sent false
27760 8547 Acked Read 9 Done Done Sent false
27761 8548 Acked Write 3 Done Done Sent false
27762 8549 Acked Read 8 Done Done Sent false
27763 8550 Acked Write 7 Done Done Sent false
27764 8551 Acked Read 9 Done Done Sent false
27765 8552 Acked Read 2 Done Done Sent false
27766 8553 Acked Write 9 Done Done Sent false
27767 8554 Acked Flush 0 Done Done Sent false
27768 8555 Acked Read 5 Done Done Sent false
27769 8556 Acked Write 1 Done Done Sent false
27770 8557 Acked Write 2 Done Done Sent false
27771 8558 Acked Write 4 Done Done Sent false
27772 8559 Acked Read 8 Done Done Sent false
27773 8560 Acked Read 8 Done Done Sent false
27774 8561 Acked Read 1 Done Done Sent false
27775 8562 Acked Write 5 Done Done Sent false
27776 8563 Acked Write 10 Done Done Sent false
27777 8564 Acked Flush 0 Done Done Sent false
27778 8565 Acked Write 10 Done Done Sent false
27779 8566 Acked Read 9 Done Done Sent false
27780 8567 Acked Write 10 Done Done Sent false
27781 8568 Acked Read 10 Done Done Sent false
27782 8569 Acked Write 3 Done Done Sent false
27783 8570 Acked Read 8 Done Done Sent false
27784 8571 Acked Write 10 Done Done Sent false
27785 8572 Acked Read 8 Done Done Sent false
27786 8573 Acked Read 2 Done Done Sent false
27787 8574 Acked Read 5 Done Done Sent false
27788 8575 Acked Write 5 Done Done Sent false
27789 8576 Acked Read 9 Done Done Sent false
27790 8577 Acked Read 7 Done Done Sent false
27791 8578 Acked Read 3 Done Done Sent false
27792 8579 Acked Flush 0 Done Done Sent false
27793 8580 Acked Write 10 Done Done Sent false
27794 8581 Acked Read 5 Done Done Sent false
27795 8582 Acked Read 5 Done Done Sent false
27796 8583 Acked Flush 0 Done Done Sent false
27797 8584 Acked Write 4 Done Done Sent false
27798 8585 Acked Write 9 Done Done Sent false
27799 8586 Acked Read 3 Done Done Sent false
27800 8587 Acked Write 4 Done Done Sent false
27801 8588 Acked Flush 0 Done Done Sent false
27802 8589 Acked Read 3 Done Done Sent false
27803 8590 Acked Flush 0 Done Done Sent false
27804 8591 Acked Write 9 Done Done Sent false
27805 8592 Acked Write 9 Done Done Sent false
27806 8593 Acked Read 4 Done Done Sent false
27807 8594 Acked Write 2 Done Done Sent false
27808 8595 Acked Write 2 Done Done Sent false
27809 8596 Acked Flush 0 Done Done Sent false
27810 8597 Acked Write 6 Done Done Sent false
27811 8598 Acked Write 1 Done Done Sent false
27812 8599 Acked Read 5 Done Done Sent false
27813 8600 Acked Read 3 Done Done Sent false
27814 8601 Acked Read 8 Done Done Sent false
27815 8602 Acked Write 5 Done Done Sent false
27816 8603 Acked Read 3 Done Done Sent false
27817 8604 Acked Read 8 Done Done Sent false
27818 8605 Acked Write 2 Done Done Sent false
27819 8606 Acked Read 3 Done Done Sent false
27820 8607 Acked Write 6 Done Done Sent false
27821 8608 Acked Read 5 Done Done Sent false
27822 8609 Acked Write 2 Done Done Sent false
27823 8610 Acked Read 1 Done Done Sent false
27824 8611 Acked Read 10 Done Done Sent false
27825 8612 Acked Read 8 Done Done Sent false
27826 8613 Acked Read 10 Done Done Sent false
27827 8614 Acked Read 6 Done Done Sent false
27828 8615 Acked Read 6 Done Done Sent false
27829 8616 Acked Read 10 Done Done Sent false
27830 8617 Acked Write 9 Done Done Sent false
27831 8618 Acked Write 10 Done Done Sent false
27832 8619 Acked Write 3 Done Done Sent false
27833 8620 Acked Flush 0 Done Done Sent false
27834 8621 Acked Read 6 Done Done Sent false
27835 8622 Acked Read 9 Done Done Sent false
27836 8623 Acked Read 4 Done Done Sent false
27837 8624 Acked Read 4 Done Done Sent false
27838 8625 Acked Write 7 Done Done Sent false
27839 8626 Acked Write 2 Done Done Sent false
27840 8627 Acked Read 5 Done Done Sent false
27841 8628 Acked Flush 0 Done Done Sent false
27842 8629 Acked Read 4 Done Done Sent false
27843 8630 Acked Read 8 Done Done Sent false
27844 8631 Acked Read 10 Done Done Sent false
27845 8632 Acked Read 9 Done Done Sent false
27846 8633 Acked Write 5 Done Done Sent false
27847 8634 Acked Flush 0 Done Done Sent false
27848 8635 Acked Write 3 Done Done Sent false
27849 8636 Acked Write 7 Done Done Sent false
27850 8637 Acked Read 8 Done Done Sent false
27851 8638 Acked Write 9 Done Done Sent false
27852 8639 Acked Write 2 Done Done Sent false
27853 8640 Acked Flush 0 Done Done Sent false
27854 8641 Acked Write 6 Done Done Sent false
27855 8642 Acked Read 4 Done Done Sent false
27856 8643 Acked Read 6 Done Done Sent false
27857 8644 Acked Write 6 Done Done Sent false
27858 8645 Acked Flush 0 Done Done Sent false
27859 8646 Acked Flush 0 Done Done Sent false
27860 8647 Acked Read 1 Done Done Sent false
27861 8648 Acked Read 7 Done Done Sent false
27862 8649 Acked Read 1 Done Done Sent false
27863 8650 Acked Write 7 Done Done Sent false
27864 8651 Acked Read 8 Done Done Sent false
27865 8652 Acked Flush 0 Done Done Sent false
27866 8653 Acked Read 5 Done Done Sent false
27867 8654 Acked Read 3 Done Done Sent false
27868 8655 Acked Read 4 Done Done Sent false
27869 8656 Acked Write 9 Done Done Sent false
27870 8657 Acked Read 1 Done Done Sent false
27871 8658 Acked Read 8 Done Done Sent false
27872 8659 Acked Write 3 Done Done Sent false
27873 8660 Acked Read 7 Done Done Sent false
27874 8661 Acked Write 8 Done Done Sent false
27875 8662 Acked Flush 0 Done Done Sent false
27876 8663 Acked Flush 0 Done Done Sent false
27877 8664 Acked Read 5 Done Done Sent false
27878 8665 Acked Read 9 Done Done Sent false
27879 8666 Acked Write 10 Done Done Sent false
27880 8667 Acked Write 3 Done Done Sent false
27881 8668 Acked Read 5 Done Done Sent false
27882 8669 Acked Write 4 Done Done Sent false
27883 8670 Acked Read 2 Done Done Sent false
27884 8671 Acked Read 5 Done Done Sent false
27885 8672 Acked Write 5 Done Done Sent false
27886 8673 Acked Read 4 Done Done Sent false
27887 8674 Acked Read 4 Done Done Sent false
27888 8675 Acked Read 6 Done Done Sent false
27889 8676 Acked Read 2 Done Done Sent false
27890 8677 Acked Read 9 Done Done Sent false
27891 8678 Acked Write 6 Done Done Sent false
27892 8679 Acked Write 6 Done Done Sent false
27893 8680 Acked Write 8 Done Done Sent false
27894 8681 Acked Read 2 Done Done Sent false
27895 8682 Acked Write 3 Done Done Sent false
27896 8683 Acked Read 9 Done Done Sent false
27897 8684 Acked Write 5 Done Done Sent false
27898 8685 Acked Write 6 Done Done Sent false
27899 8686 Acked Read 1 Done Done Sent false
27900 8687 Acked Write 5 Done Done Sent false
27901 8688 Acked Read 10 Done Done Sent false
27902 8689 Acked Read 3 Done Done Sent false
27903 8690 Acked Read 1 Done Done Sent false
27904 8691 Acked Read 3 Done Done Sent false
27905 8692 Acked Flush 0 Done Done Sent false
27906 8693 Acked Read 9 Done Done Sent false
27907 8694 Acked Read 3 Done Done Sent false
27908 8695 Acked Write 9 Done Done Sent false
27909 8696 Acked Write 2 Done Done Sent false
27910 8697 Acked Read 1 Done Done Sent false
27911 8698 Acked Read 6 Done Done Sent false
27912 8699 Acked Read 9 Done Done Sent false
27913 8700 Acked Read 3 Done Done Sent false
27914 8701 Acked Read 2 Done Done Sent false
27915 8702 Acked Flush 0 Done Done Sent false
27916 8703 Acked Write 10 Done Done Sent false
27917 8704 Acked Read 9 Done Done Sent false
27918 8705 Acked Read 10 Done Done Sent false
27919 8706 Acked Read 1 Done Done Sent false
27920 8707 Acked Flush 0 Done Done Sent false
27921 8708 Acked Read 8 Done Done Sent false
27922 8709 Acked Write 7 Done Done Sent false
27923 8710 Acked Flush 0 Done Done Sent false
27924 8711 Acked Read 7 Done Done Sent false
27925 8712 Acked Read 2 Done Done Sent false
27926 8713 Acked Read 2 Done Done Sent false
27927 8714 Acked Write 7 Done Done Sent false
27928 8715 Acked Read 1 Done Done Sent false
27929 8716 Acked Read 10 Done Done Sent false
27930 8717 Acked Read 1 Done Done Sent false
27931 8718 Acked Write 5 Done Done Sent false
27932 8719 Acked Write 6 Done Done Sent false
27933 8720 Acked Write 3 Done Done Sent false
27934 8721 Acked Read 6 Done Done Sent false
27935 8722 Acked Write 2 Done Done Sent false
27936 8723 Acked Write 5 Done Done Sent false
27937 8724 Acked Write 8 Done Done Sent false
27938 8725 Acked Write 6 Done Done Sent false
27939 8726 Acked Write 2 Done Done Sent false
27940 8727 Acked Write 1 Done Done Sent false
27941 8728 Acked Read 4 Done Done Sent false
27942 8729 Acked Read 5 Done Done Sent false
27943 8730 Acked Write 6 Done Done Sent false
27944 8731 Acked Read 9 Done Done Sent false
27945 8732 Acked Write 7 Done Done Sent false
27946 8733 Acked Read 7 Done Done Sent false
27947 8734 Acked Write 5 Done Done Sent false
27948 8735 Acked Read 8 Done Done Sent false
27949 8736 Acked Write 2 Done Done Sent false
27950 8737 Acked Read 4 Done Done Sent false
27951 8738 Acked Read 1 Done Done Sent false
27952 8739 Acked Read 8 Done Done Sent false
27953 8740 Acked Read 6 Done Done Sent false
27954 8741 Acked Read 10 Done Done Sent false
27955 8742 Acked Read 2 Done Done Sent false
27956 8743 Acked Write 8 Done Done Sent false
27957 8744 Acked Read 4 Done Done Sent false
27958 8745 Acked Write 3 Done Done Sent false
27959 8746 Acked Read 6 Done Done Sent false
27960 8747 Acked Read 1 Done Done Sent false
27961 8748 Acked Read 5 Done Done Sent false
27962 8749 Acked Read 8 Done Done Sent false
27963 8750 Acked Write 3 Done Done Sent false
27964 8751 Acked Write 1 Done Done Sent false
27965 8752 Acked Write 4 Done Done Sent false
27966 8753 Acked Write 7 Done Done Sent false
27967 8754 Acked Write 6 Done Done Sent false
27968 8755 Acked Write 5 Done Done Sent false
27969 8756 Acked Write 3 Done Done Sent false
27970 8757 Acked Read 6 Done Done Sent false
27971 8758 Acked Write 7 Done Done Sent false
27972 8759 Acked Write 9 Done Done Sent false
27973 8760 Acked Write 6 Done Done Sent false
27974 8761 Acked Write 7 Done Done Sent false
27975 8762 Acked Read 4 Done Done Sent false
27976 8763 Acked Read 2 Done Done Sent false
27977 8764 Acked Write 3 Done Done Sent false
27978 8765 Acked Write 2 Done Done Sent false
27979 8766 Acked Read 4 Done Done Sent false
27980 8767 Acked Write 8 Done Done Sent false
27981 8768 Acked Write 6 Done Done Sent false
27982 8769 Acked Write 1 Done Done Sent false
27983 8770 Acked Write 1 Done Done Sent false
27984 8771 Acked Read 7 Done Done Sent false
27985 8772 Acked Read 4 Done Done Sent false
27986 8773 Acked Read 8 Done Done Sent false
27987 8774 Acked Read 6 Done Done Sent false
27988 8775 Acked Read 8 Done Done Sent false
27989 8776 Acked Write 9 Done Done Sent false
27990 8777 Acked Write 9 Done Done Sent false
27991 8778 Acked Write 8 Done Done Sent false
27992 8779 Acked Write 7 Done Done Sent false
27993 8780 Acked Read 10 Done Done Sent false
27994 8781 Acked Read 6 Done Done Sent false
27995 8782 Acked Read 6 Done Done Sent false
27996 8783 Acked Read 4 Done Done Sent false
27997 8784 Acked Read 1 Done Done Sent false
27998 8785 Acked Read 1 Done Done Sent false
27999 8786 Acked Write 10 Done Done Sent false
28000 8787 Acked Write 8 Done Done Sent false
28001 8788 Acked Read 8 Done Done Sent false
28002 8789 Acked Read 1 Done Done Sent false
28003 8790 Acked Read 10 Done Done Sent false
28004 8791 Acked Read 3 Done Done Sent false
28005 8792 Acked Read 5 Done Done Sent false
28006 8793 Acked Write 7 Done Done Sent false
28007 8794 Acked Write 3 Done Done Sent false
28008 8795 Acked Flush 0 Done Done Sent false
28009 8796 Acked Read 7 Done Done Sent false
28010 8797 Acked Write 10 Done Done Sent false
28011 8798 Acked Write 3 Done Done Sent false
28012 8799 Acked Read 2 Done Done Sent false
28013 8800 Acked Read 3 Done Done Sent false
28014 8801 Acked Write 8 Done Done Sent false
28015 8802 Acked Write 2 Done Done Sent false
28016 8803 Acked Write 4 Done Done Sent false
28017 8804 Acked Flush 0 Done Done Sent false
28018 8805 Acked Read 8 Done Done Sent false
28019 8806 Acked Write 10 Done Done Sent false
28020 8807 Acked Read 2 Done Done Sent false
28021 8808 Acked Flush 0 Done Done Sent false
28022 8809 Acked Read 3 Done Done Sent false
28023 8810 Acked Read 5 Done Done Sent false
28024 8811 Acked Write 10 Done Done Sent false
28025 8812 Acked Write 8 Done Done Sent false
28026 8813 Acked Read 10 Done Done Sent false
28027 8814 Acked Read 3 Done Done Sent false
28028 8815 Acked Read 2 Done Done Sent false
28029 8816 Acked Write 10 Done Done Sent false
28030 8817 Acked Flush 0 Done Done Sent false
28031 8818 Acked Read 10 Done Done Sent false
28032 8819 Acked Read 8 Done Done Sent false
28033 8820 Acked Write 9 Done Done Sent false
28034 8821 Acked Read 8 Done Done Sent false
28035 8822 Acked Write 10 Done Done Sent false
28036 8823 Acked Read 4 Done Done Sent false
28037 8824 Acked Write 6 Done Done Sent false
28038 8825 Acked Read 3 Done Done Sent false
28039 8826 Acked Flush 0 Done Done Sent false
28040 8827 Acked Read 3 Done Done Sent false
28041 8828 Acked Write 8 Done Done Sent false
28042 8829 Acked Write 2 Done Done Sent false
28043 8830 Acked Write 10 Done Done Sent false
28044 8831 Acked Read 7 Done Done Sent false
28045 8832 Acked Write 5 Done Done Sent false
28046 8833 Acked Read 5 Done Done Sent false
28047 8834 Acked Write 8 Done Done Sent false
28048 8835 Acked Read 8 Done Done Sent false
28049 8836 Acked Read 7 Done Done Sent false
28050 8837 Acked Write 7 Done Done Sent false
28051 8838 Acked Write 9 Done Done Sent false
28052 8839 Acked Write 4 Done Done Sent false
28053 8840 Acked Read 4 Done Done Sent false
28054 8841 Acked Read 4 Done Done Sent false
28055 8842 Acked Write 6 Done Done Sent false
28056 8843 Acked Read 10 Done Done Sent false
28057 8844 Acked Read 3 Done Done Sent false
28058 8845 Acked Write 2 Done Done Sent false
28059 8846 Acked Read 4 Done Done Sent false
28060 8847 Acked Flush 0 Done Done Sent false
28061 STATES DS:0 DS:1 DS:2 TOTAL
28062 Sent 0 0 302 302
28063 Done 302 302 0 604
28064 Skipped 0 0 0 0
28065 Error 0 0 0 0
28066 Last Flush: 8847 8847 8545
28067 Downstairs last five completed: 8545 8544 8543 8542 8541
28068 Upstairs last five completed: 8847 8846 8845 8844 8843
28069 CLIENT: Up:0 ds:302 act:2
280702024-12-04T13:36:34.275ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280712024-12-04T13:36:34.275ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48951) = io task client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280722024-12-04T13:36:34.276ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280732024-12-04T13:36:34.276ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280742024-12-04T13:36:34.276ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280752024-12-04T13:36:34.276ZINFOcrucible: send last flush ID to this DS: 8545 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280762024-12-04T13:36:34.277ZINFOcrucible: Replied this last flush ID: 8545 = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280772024-12-04T13:36:34.277ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280782024-12-04T13:36:34.277ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280792024-12-04T13:36:34.277ZINFOcrucible: [2] client re-new 302 jobs since flush 8545 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280802024-12-04T13:36:34.277ZINFOcrucible: [2] Replayed 302 jobs since flush: 8545 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
28081 ----------------------------------------------------------------
28082 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28083 Downstairs last five completed: 8847 8846 8845 8844 8843
28084 Upstairs last five completed: 8847 8846 8845 8844 8843
28085 CLIENT: Up:0 ds:0 act:3
28086 Replay: All jobs finished
28087 Main task has work to do, go find it
28088 got dsc Stop(1)
28089 stop 1
28090 Dec 04 13:36:36.517 INFO[8820] Got stop action so:false kr:false
28091 request completed, latency_us: 341, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 6b97f719-cf27-4fcf-a7f2-0023a9c011d5, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
28092 Dec 04 13:36:36.517 INFO request completed, latency_us: 149, response_code: 200, uri: /state/cid/1, method: GET, req_id: 51d26427-2037-4b5a-b21b-17d13c9d83fb, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
280932024-12-04T13:36:36.517ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280942024-12-04T13:36:36.517ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280952024-12-04T13:36:36.517ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280962024-12-04T13:36:36.518ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280972024-12-04T13:36:36.518ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
280982024-12-04T13:36:36.518ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
28099 [8820] Exited with: ExitStatus(unix_wait_status(9))
28100 [8820][1] reports Exit
28101 Dec 04 13:36:40.520 INFO request completed, latency_us: 231, response_code: 200, uri: /state/cid/1, method: GET, req_id: 21274638-0788-4ba1-98b8-1362624abb44, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
28102 001/300 Read block 4406 len 24576
28103 002/300 Read block 3461 len 8192
28104 003/300 Write block 4267 len 24576 data: 4 6 8 8 8 7
28105 004/300 Read block 21 len 36864
28106 005/300 Read block 389 len 8192
28107 006/300 Read block 1519 len 28672
28108 007/300 Read block 3458 len 40960
28109 008/300 Read block 1475 len 20480
28110 009/300 Write block 83 len 12288 data: 5 5 6
28111 010/300 Read block 3768 len 12288
28112 011/300 Read block 3328 len 40960
28113 012/300 Write block 2937 len 40960 data: 6 5 4 4 4 6 6 7 7 6
28114 013/300 Write block 3540 len 20480 data: 6 7 5 7 6
28115 014/300 Write block 4356 len 20480 data: 4 6 5 5 6
28116 015/300 Read block 3197 len 28672
28117 016/300 Write block 1970 len 20480 data: 6 6 6 4 5
28118 017/300 Read block 2347 len 4096
28119 018/300 Write block 2672 len 4096 data: 5
28120 019/300 Read block 2766 len 8192
28121 020/300 Read block 2107 len 24576
28122 021/300 Write block 669 len 8192 data: 8 8
28123 022/300 Write block 4838 len 40960 data: 6 6 7 7 7 6 5 4 4 5
28124 023/300 Write block 1618 len 40960 data: 5 6 5 5 6 5 4 4 3 3
28125 024/300 Read block 3915 len 4096
28126 025/300 Write block 1240 len 28672 data: 4 5 6 6 7 7 6
28127 026/300 Read block 3298 len 40960
28128 027/300 Read block 1970 len 20480
28129 028/300 Read block 3729 len 8192
28130 029/300 Write block 6 len 16384 data: 6 7 5 5
28131 030/300 Write block 3318 len 20480 data: 6 7 10 10 8
28132 031/300 Read block 630 len 16384
28133 032/300 Write block 3069 len 8192 data: 2 3
28134 033/300 Flush
28135 034/300 Write block 1958 len 20480 data: 4 3 5 5 5
28136 035/300 Write block 2949 len 16384 data: 4 5 5 5
28137 036/300 Read block 3062 len 40960
28138 037/300 Read block 1300 len 32768
28139 038/300 Read block 4242 len 28672
28140 039/300 Write block 2130 len 16384 data: 3 4 5 5
28141 040/300 Write block 461 len 20480 data: 3 2 2 2 3
28142 041/300 Read block 3264 len 36864
28143 042/300 Read block 2195 len 4096
28144 043/300 Read block 279 len 8192
28145 044/300 Write block 1070 len 4096 data: 12
28146 045/300 Read block 1375 len 32768
28147 046/300 Flush
28148 047/300 Write block 4060 len 40960 data: 6 6 6 6 7 7 6 5 5 3
28149 048/300 Write block 4657 len 28672 data: 3 3 5 6 6 6 5
28150 049/300 Flush
28151 050/300 Read block 4250 len 16384
28152 051/300 Write block 2651 len 32768 data: 7 3 4 5 5 5 5 5
28153 052/300 Read block 1537 len 28672
28154 053/300 Read block 1128 len 28672
28155 054/300 Read block 2865 len 40960
28156 055/300 Write block 2859 len 40960 data: 4 4 5 4 6 5 5 6 7 8
28157 056/300 Flush
28158 057/300 Write block 1542 len 4096 data: 7
28159 058/300 Read block 1799 len 4096
28160 059/300 Read block 681 len 16384
28161 060/300 Write block 1602 len 32768 data: 5 5 4 5 4 6 6 5
28162 061/300 Read block 614 len 12288
28163 062/300 Write block 1314 len 8192 data: 3 3
28164 063/300 Read block 3837 len 12288
28165 064/300 Read block 286 len 20480
28166 065/300 Write block 1319 len 8192 data: 6 7
28167 066/300 Write block 4143 len 40960 data: 2 2 2 3 3 3 5 4 4 4
28168 067/300 Read block 2376 len 32768
28169 068/300 Read block 3720 len 4096
28170 069/300 Write block 3243 len 24576 data: 6 4 3 3 4 4
28171 070/300 Flush
28172 071/300 Read block 897 len 28672
28173 072/300 Read block 4853 len 12288
28174 073/300 Flush
28175 074/300 Read block 2349 len 24576
28176 075/300 Write block 2252 len 24576 data: 5 5 5 4 6 5
28177 076/300 Write block 3141 len 28672 data: 9 8 8 6 5 6 6
28178 077/300 Read block 2632 len 36864
28179 078/300 Write block 2789 len 20480 data: 7 7 8 7 4
28180 079/300 Write block 4891 len 20480 data: 6 6 5 4 4
28181 080/300 Read block 4116 len 32768
28182 081/300 Read block 4318 len 32768
28183 082/300 Write block 2374 len 12288 data: 4 3 5
28184 083/300 Write block 4465 len 32768 data: 9 8 7 8 8 9 9 9
28185 084/300 Read block 4242 len 20480
28186 085/300 Write block 4478 len 40960 data: 4 5 4 5 6 8 9 9 10 10
28187 086/300 Read block 4368 len 36864
28188 087/300 Write block 921 len 20480 data: 5 4 5 5 5
28189 088/300 Write block 3166 len 24576 data: 5 4 3 3 3 6
28190 089/300 Read block 2748 len 8192
28191 090/300 Read block 427 len 4096
28192 091/300 Flush
28193 092/300 Write block 4589 len 24576 data: 6 7 7 6 7 7
28194 093/300 Write block 3023 len 20480 data: 6 5 5 5 4
28195 094/300 Read block 3579 len 16384
28196 095/300 Write block 1797 len 8192 data: 3 3
28197 096/300 Read block 4753 len 12288
28198 097/300 Read block 4380 len 36864
28199 098/300 Write block 71 len 24576 data: 3 5 5 5 5 3
28200 099/300 Write block 1837 len 4096 data: 6
28201 100/300 Write block 3093 len 20480 data: 5 4 4 6 4
28202 101/300 Write block 3713 len 36864 data: 4 6 8 10 11 8 7 7 7
28203 102/300 Read block 3820 len 28672
28204 103/300 Flush
28205 104/300 Read block 2079 len 24576
28206 105/300 Flush
28207 106/300 Write block 2182 len 24576 data: 5 6 6 6 7 7
28208 107/300 Read block 189 len 4096
28209 108/300 Flush
28210 109/300 Write block 3844 len 12288 data: 3 5 5
28211 110/300 Write block 3330 len 40960 data: 5 6 5 5 5 6 5 3 3 3
28212 111/300 Read block 2976 len 8192
28213 112/300 Flush
28214 113/300 Read block 155 len 16384
28215 114/300 Write block 3440 len 4096 data: 6
28216 115/300 Write block 430 len 24576 data: 4 3 2 2 3 4
28217 116/300 Read block 3587 len 40960
28218 117/300 Read block 2549 len 8192
28219 118/300 Read block 237 len 4096
28220 119/300 Read block 758 len 36864
28221 120/300 Read block 178 len 8192
28222 121/300 Write block 4649 len 20480 data: 6 6 4 3 4
28223 122/300 Write block 603 len 28672 data: 6 5 6 4 4 5 4
28224 123/300 Read block 3721 len 20480
28225 124/300 Read block 592 len 40960
28226 125/300 Read block 3010 len 24576
28227 126/300 Write block 4717 len 24576 data: 5 6 5 5 5 6
28228 127/300 Read block 74 len 16384
28229 128/300 Read block 330 len 28672
28230 129/300 Write block 3232 len 20480 data: 6 6 7 8 9
28231 130/300 Read block 1134 len 20480
28232 131/300 Read block 3378 len 32768
28233 132/300 Write block 1606 len 8192 data: 5 7
28234 133/300 Read block 699 len 16384
28235 134/300 Write block 2321 len 20480 data: 7 8 7 7 5
28236 135/300 Read block 3866 len 16384
28237 136/300 Write block 4260 len 20480 data: 6 5 5 5 4
28238 137/300 Write block 1119 len 32768 data: 5 5 5 4 4 4 3 3
28239 138/300 Read block 225 len 32768
28240 139/300 Flush
28241 140/300 Read block 4163 len 8192
28242 141/300 Read block 2999 len 28672
28243 142/300 Flush
28244 143/300 Write block 4802 len 24576 data: 7 8 7 7 6 5
28245 144/300 Write block 1761 len 36864 data: 9 12 11 10 8 8 7 11 11
28246 145/300 Read block 3651 len 4096
28247 146/300 Read block 2341 len 40960
28248 147/300 Read block 757 len 36864
28249 148/300 Flush
28250 149/300 Read block 4562 len 12288
28251 150/300 Read block 719 len 4096
28252 151/300 Read block 1950 len 16384
28253 152/300 Read block 145 len 20480
28254 153/300 Write block 3697 len 24576 data: 8 8 8 7 8 6
28255 154/300 Write block 2726 len 28672 data: 5 6 7 7 5 5 5
28256 155/300 Write block 4985 len 28672 data: 7 6 6 4 5 4 4
28257 156/300 Write block 4657 len 28672 data: 4 4 6 7 7 7 6
28258 157/300 Write block 416 len 20480 data: 6 7 5 5 5
28259 158/300 Read block 4527 len 28672
28260 159/300 Read block 3957 len 40960
28261 160/300 Write block 3852 len 12288 data: 3 3 3
28262 161/300 Write block 420 len 16384 data: 6 5 4 4
28263 162/300 Write block 4492 len 32768 data: 7 5 4 4 4 4 4 4
28264 163/300 Write block 4639 len 16384 data: 3 3 2 2
28265 164/300 Read block 1637 len 8192
28266 165/300 Write block 2918 len 8192 data: 3 3
28267 166/300 Flush
28268 167/300 Read block 445 len 20480
28269 168/300 Write block 4955 len 28672 data: 4 4 4 4 5 4 3
28270 169/300 Write block 1911 len 16384 data: 8 8 8 7
28271 170/300 Read block 2472 len 40960
28272 171/300 Read block 2132 len 16384
28273 172/300 Read block 4185 len 32768
28274 173/300 Read block 1447 len 28672
28275 174/300 Read block 3505 len 36864
28276 175/300 Flush
28277 176/300 Write block 4485 len 12288 data: 10 11 11
28278 177/300 Write block 1022 len 4096 data: 6
28279 178/300 Flush
28280 179/300 Flush
28281 180/300 Read block 3347 len 8192
28282 181/300 Read block 4980 len 28672
28283 182/300 Read block 2187 len 4096
28284 183/300 Write block 77 len 16384 data: 3 3 4 5
28285 184/300 Write block 3082 len 8192 data: 3 3
28286 185/300 Read block 1086 len 40960
28287 186/300 Write block 2431 len 32768 data: 6 6 6 4 2 4 5 6
28288 187/300 Flush
28289 188/300 Read block 4844 len 40960
28290 189/300 Read block 4890 len 16384
28291 190/300 Write block 2806 len 24576 data: 4 3 3 4 4 3
28292 191/300 Flush
28293 192/300 Read block 4423 len 24576
28294 193/300 Write block 4645 len 36864 data: 4 5 6 6 7 7 5 4 5
28295 194/300 Read block 1424 len 32768
28296 195/300 Write block 1757 len 12288 data: 4 4 3
28297 196/300 Write block 1525 len 16384 data: 11 12 11 10
28298 197/300 Write block 1395 len 16384 data: 5 4 4 5
28299 198/300 Write block 848 len 16384 data: 4 6 5 6
28300 199/300 Read block 4105 len 32768
28301 200/300 Write block 3696 len 40960 data: 7 9 9 9 8 9 7 6 5 4
28302 201/300 Write block 3944 len 28672 data: 7 8 7 7 8 7 8
28303 202/300 Read block 2458 len 16384
28304 203/300 Flush
28305 204/300 Read block 3861 len 36864
28306 205/300 Read block 1344 len 28672
28307 206/300 Read block 1904 len 4096
28308 207/300 Read block 2120 len 32768
28309 208/300 Write block 2899 len 20480 data: 5 5 7 6 7
28310 209/300 Read block 33 len 28672
28311 210/300 Write block 1844 len 8192 data: 6 7
28312 211/300 Write block 3773 len 8192 data: 7 7
28313 212/300 Write block 591 len 36864 data: 6 7 7 7 6 6 6 4 4
28314 213/300 Read block 1796 len 16384
28315 214/300 Flush
28316 215/300 Read block 2880 len 36864
28317 216/300 Write block 40 len 16384 data: 3 4 3 3
28318 217/300 Write block 826 len 20480 data: 6 6 7 7 6
28319 218/300 Write block 54 len 24576 data: 5 7 6 6 6 4
28320 219/300 Write block 3579 len 12288 data: 3 3 3
28321 220/300 Write block 2407 len 40960 data: 5 5 5 6 6 7 7 7 7 5
28322 221/300 Read block 1957 len 36864
28323 222/300 Write block 4733 len 36864 data: 4 6 7 7 4 4 5 5 4
28324 223/300 Read block 2452 len 24576
28325 224/300 Read block 1155 len 32768
28326 225/300 Read block 3200 len 40960
28327 226/300 Flush
28328 227/300 Flush
28329 228/300 Read block 3501 len 36864
28330 229/300 Read block 2418 len 40960
28331 230/300 Write block 1010 len 36864 data: 5 4 6 6 6 7 10 9 10
28332 231/300 Write block 3635 len 16384 data: 6 5 5 6
28333 232/300 Write block 3414 len 28672 data: 4 5 5 6 8 7 7
28334 233/300 Flush
28335 234/300 Write block 1357 len 40960 data: 4 4 5 7 6 7 5 5 4 5
28336 235/300 Read block 3066 len 20480
28337 236/300 Read block 2604 len 4096
28338 237/300 Write block 4335 len 32768 data: 7 8 7 7 10 7 7 7
28339 238/300 Write block 950 len 24576 data: 7 6 6 5 4 4
28340 239/300 Read block 2900 len 4096
28341 240/300 Read block 1857 len 36864
28342 241/300 Flush
28343 242/300 Read block 4145 len 36864
28344 243/300 Flush
28345 244/300 Flush
28346 245/300 Write block 3641 len 36864 data: 3 4 3 4 6 6 6 5 5
28347 246/300 Read block 2667 len 16384
28348 247/300 Read block 4294 len 4096
28349 248/300 Write block 931 len 32768 data: 3 4 5 3 4 4 5 6
28350 249/300 Read block 1739 len 4096
28351 250/300 Write block 3225 len 40960 data: 8 9 5 4 7 8 6 7 7 8
28352 251/300 Read block 2588 len 8192
28353 252/300 Read block 4742 len 40960
28354 253/300 Write block 308 len 40960 data: 6 6 5 6 7 6 6 7 7 6
28355 254/300 Write block 166 len 16384 data: 8 9 8 9
28356 255/300 Read block 2388 len 24576
28357 256/300 Write block 2810 len 40960 data: 5 4 2 2 3 3 3 4 4 4
28358 257/300 Flush
28359 258/300 Read block 3351 len 8192
28360 259/300 Write block 1717 len 12288 data: 5 4 4
28361 260/300 Write block 3241 len 32768 data: 7 6 7 5 4 4 5 5
28362 261/300 Read block 746 len 8192
28363 262/300 Read block 1563 len 32768
28364 263/300 Read block 3014 len 28672
28365 264/300 Read block 3466 len 36864
28366 265/300 Read block 1715 len 12288
28367 266/300 Read block 2564 len 32768
28368 267/300 Write block 2230 len 36864 data: 4 4 4 4 6 6 6 4 5
28369 268/300 Read block 3342 len 28672
28370 269/300 Read block 4604 len 32768
28371 270/300 Write block 1577 len 32768 data: 4 4 4 5 5 5 6 5
28372 271/300 Read block 3417 len 12288
28373 272/300 Read block 3269 len 28672
28374 273/300 Write block 4373 len 16384 data: 5 6 6 7
28375 274/300 Read block 3959 len 40960
28376 275/300 Write block 3642 len 8192 data: 5 4
28377 276/300 Write block 427 len 16384 data: 2 4 4 5
28378 277/300 Read block 1294 len 40960
28379 278/300 Write block 2788 len 24576 data: 6 8 8 9 8 5
28380 279/300 Read block 1399 len 16384
28381 280/300 Write block 4449 len 12288 data: 4 4 4
28382 281/300 Read block 1022 len 28672
28383 282/300 Read block 3719 len 16384
28384 283/300 Write block 339 len 32768 data: 5 5 3 4 5 6 5 6
28385 284/300 Read block 2471 len 16384
28386 285/300 Write block 2587 len 28672 data: 5 5 7 6 5 5 5
28387 286/300 Read block 1693 len 40960
28388 287/300 Write block 34 len 32768 data: 6 5 6 6 4 3 4 5
28389 288/300 Write block 3986 len 36864 data: 2 2 3 3 4 5 5 5 4
28390 289/300 Flush
28391 290/300 Read block 4324 len 4096
28392 291/300 Read block 1584 len 12288
28393 292/300 Read block 3796 len 20480
28394 293/300 Read block 561 len 32768
28395 294/300 Write block 4866 len 4096 data: 6
28396 295/300 Flush
28397 296/300 Write block 4685 len 40960 data: 7 5 5 6 6 6 4 4 5 6
28398 297/300 Write block 3896 len 40960 data: 4 4 5 5 5 6 6 7 6 4
28399 298/300 Flush
28400 299/300 Read block 3077 len 12288
28401 300/300 Write block 1116 len 4096 data: 8
28402 301/300 Read block 4235 len 4096
28403 Main task has work to do, go find it
28404 got dsc Start(1)
28405 start 1
28406 Dec 04 13:36:40.756 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/1, method: POST, req_id: cc8b5737-477f-47eb-9b79-f3fb9bb6ed69, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
28407 Replay: started 1, returned:Ok(())
28408 ----------------------------------------------------------------
28409 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28410 States: Active Offline Active
28411 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28412 8848 Acked Read 6 Done Sent Done false
28413 8849 Acked Read 2 Done Sent Done false
28414 8850 Acked Write 6 Done Sent Done false
28415 8851 Acked Read 9 Done Sent Done false
28416 8852 Acked Read 2 Done Sent Done false
28417 8853 Acked Read 7 Done Sent Done false
28418 8854 Acked Read 10 Done Sent Done false
28419 8855 Acked Read 5 Done Sent Done false
28420 8856 Acked Write 3 Done Sent Done false
28421 8857 Acked Read 3 Done Sent Done false
28422 8858 Acked Read 10 Done Sent Done false
28423 8859 Acked Write 10 Done Sent Done false
28424 8860 Acked Write 5 Done Sent Done false
28425 8861 Acked Write 5 Done Sent Done false
28426 8862 Acked Read 7 Done Sent Done false
28427 8863 Acked Write 5 Done Sent Done false
28428 8864 Acked Read 1 Done Sent Done false
28429 8865 Acked Write 1 Done Sent Done false
28430 8866 Acked Read 2 Done Sent Done false
28431 8867 Acked Read 6 Done Sent Done false
28432 8868 Acked Write 2 Done Sent Done false
28433 8869 Acked Write 10 Done Sent Done false
28434 8870 Acked Write 10 Done Sent Done false
28435 8871 Acked Read 1 Done Sent Done false
28436 8872 Acked Write 7 Done Sent Done false
28437 8873 Acked Read 10 Done Sent Done false
28438 8874 Acked Read 5 Done Sent Done false
28439 8875 Acked Read 2 Done Sent Done false
28440 8876 Acked Write 4 Done Sent Done false
28441 8877 Acked Write 5 Done Sent Done false
28442 8878 Acked Read 4 Done Sent Done false
28443 8879 Acked Write 2 Done Sent Done false
28444 8880 Acked Flush 0 Done Sent Done false
28445 8881 Acked Write 5 Done Sent Done false
28446 8882 Acked Write 4 Done Sent Done false
28447 8883 Acked Read 10 Done Sent Done false
28448 8884 Acked Read 8 Done Sent Done false
28449 8885 Acked Read 7 Done Sent Done false
28450 8886 Acked Write 4 Done Sent Done false
28451 8887 Acked Write 5 Done Sent Done false
28452 8888 Acked Read 9 Done Sent Done false
28453 8889 Acked Read 1 Done Sent Done false
28454 8890 Acked Read 2 Done Sent Done false
28455 8891 Acked Write 1 Done Sent Done false
28456 8892 Acked Read 8 Done Sent Done false
28457 8893 Acked Flush 0 Done Sent Done false
28458 8894 Acked Write 10 Done Sent Done false
28459 8895 Acked Write 7 Done Sent Done false
28460 8896 Acked Flush 0 Done Sent Done false
28461 8897 Acked Read 4 Done Sent Done false
28462 8898 Acked Write 8 Done Sent Done false
28463 8899 Acked Read 7 Done Sent Done false
28464 8900 Acked Read 7 Done Sent Done false
28465 8901 Acked Read 10 Done Sent Done false
28466 8902 Acked Write 10 Done Sent Done false
28467 8903 Acked Flush 0 Done Sent Done false
28468 8904 Acked Write 1 Done Sent Done false
28469 8905 Acked Read 1 Done Sent Done false
28470 8906 Acked Read 4 Done Sent Done false
28471 8907 Acked Write 8 Done Sent Done false
28472 8908 Acked Read 3 Done Sent Done false
28473 8909 Acked Write 2 Done Sent Done false
28474 8910 Acked Read 3 Done Sent Done false
28475 8911 Acked Read 5 Done Sent Done false
28476 8912 Acked Write 2 Done Sent Done false
28477 8913 Acked Write 10 Done Sent Done false
28478 8914 Acked Read 8 Done Sent Done false
28479 8915 Acked Read 1 Done Sent Done false
28480 8916 Acked Write 6 Done Sent Done false
28481 8917 Acked Flush 0 Done Sent Done false
28482 8918 Acked Read 7 Done Sent Done false
28483 8919 Acked Read 3 Done Sent Done false
28484 8920 Acked Flush 0 Done Sent Done false
28485 8921 Acked Read 6 Done Sent Done false
28486 8922 Acked Write 6 Done Sent Done false
28487 8923 Acked Write 7 Done Sent Done false
28488 8924 Acked Read 9 Done Sent Done false
28489 8925 Acked Write 5 Done Sent Done false
28490 8926 Acked Write 5 Done Sent Done false
28491 8927 Acked Read 8 Done Sent Done false
28492 8928 Acked Read 8 Done Sent Done false
28493 8929 Acked Write 3 Done Sent Done false
28494 8930 Acked Write 8 Done Sent Done false
28495 8931 Acked Read 5 Done Sent Done false
28496 8932 Acked Write 10 Done Sent Done false
28497 8933 Acked Read 9 Done Sent Done false
28498 8934 Acked Write 5 Done Sent Done false
28499 8935 Acked Write 6 Done Sent Done false
28500 8936 Acked Read 2 Done Sent Done false
28501 8937 Acked Read 1 Done Sent Done false
28502 8938 Acked Flush 0 Done Sent Done false
28503 8939 Acked Write 6 Done Sent Done false
28504 8940 Acked Write 5 Done Sent Done false
28505 8941 Acked Read 4 Done Sent Done false
28506 8942 Acked Write 2 Done Sent Done false
28507 8943 Acked Read 3 Done Sent Done false
28508 8944 Acked Read 9 Done Sent Done false
28509 8945 Acked Write 6 Done Sent Done false
28510 8946 Acked Write 1 Done Sent Done false
28511 8947 Acked Write 5 Done Sent Done false
28512 8948 Acked Write 9 Done Sent Done false
28513 8949 Acked Read 7 Done Sent Done false
28514 8950 Acked Flush 0 Done Sent Done false
28515 8951 Acked Read 6 Done Sent Done false
28516 8952 Acked Flush 0 Done Sent Done false
28517 8953 Acked Write 6 Done Sent Done false
28518 8954 Acked Read 1 Done Sent Done false
28519 8955 Acked Flush 0 Done Sent Done false
28520 8956 Acked Write 3 Done Sent Done false
28521 8957 Acked Write 10 Done Sent Done false
28522 8958 Acked Read 2 Done Sent Done false
28523 8959 Acked Flush 0 Done Sent Done false
28524 8960 Acked Read 4 Done Sent Done false
28525 8961 Acked Write 1 Done Sent Done false
28526 8962 Acked Write 6 Done Sent Done false
28527 8963 Acked Read 10 Done Sent Done false
28528 8964 Acked Read 2 Done Sent Done false
28529 8965 Acked Read 1 Done Sent Done false
28530 8966 Acked Read 9 Done Sent Done false
28531 8967 Acked Read 2 Done Sent Done false
28532 8968 Acked Write 5 Done Sent Done false
28533 8969 Acked Write 7 Done Sent Done false
28534 8970 Acked Read 5 Done Sent Done false
28535 8971 Acked Read 10 Done Sent Done false
28536 8972 Acked Read 6 Done Sent Done false
28537 8973 Acked Write 6 Done Sent Done false
28538 8974 Acked Read 4 Done Sent Done false
28539 8975 Acked Read 7 Done Sent Done false
28540 8976 Acked Write 5 Done Sent Done false
28541 8977 Acked Read 5 Done Sent Done false
28542 8978 Acked Read 8 Done Sent Done false
28543 8979 Acked Write 2 Done Sent Done false
28544 8980 Acked Read 4 Done Sent Done false
28545 8981 Acked Write 5 Done Sent Done false
28546 8982 Acked Read 4 Done Sent Done false
28547 8983 Acked Write 5 Done Sent Done false
28548 8984 Acked Write 8 Done Sent Done false
28549 8985 Acked Read 8 Done Sent Done false
28550 8986 Acked Flush 0 Done Sent Done false
28551 8987 Acked Read 2 Done Sent Done false
28552 8988 Acked Read 7 Done Sent Done false
28553 8989 Acked Flush 0 Done Sent Done false
28554 8990 Acked Write 6 Done Sent Done false
28555 8991 Acked Write 9 Done Sent Done false
28556 8992 Acked Read 1 Done Sent Done false
28557 8993 Acked Read 10 Done Sent Done false
28558 8994 Acked Read 9 Done Sent Done false
28559 8995 Acked Flush 0 Done Sent Done false
28560 8996 Acked Read 3 Done Sent Done false
28561 8997 Acked Read 1 Done Sent Done false
28562 8998 Acked Read 4 Done Sent Done false
28563 8999 Acked Read 5 Done Sent Done false
28564 9000 Acked Write 6 Done Sent Done false
28565 9001 Acked Write 7 Done Sent Done false
28566 9002 Acked Write 7 Done Sent Done false
28567 9003 Acked Write 7 Done Sent Done false
28568 9004 Acked Write 5 Done Sent Done false
28569 9005 Acked Read 7 Done Sent Done false
28570 9006 Acked Read 10 Done Sent Done false
28571 9007 Acked Write 3 Done Sent Done false
28572 9008 Acked Write 4 Done Sent Done false
28573 9009 Acked Write 8 Done Sent Done false
28574 9010 Acked Write 4 Done Sent Done false
28575 9011 Acked Read 2 Done Sent Done false
28576 9012 Acked Write 2 Done Sent Done false
28577 9013 Acked Flush 0 Done Sent Done false
28578 9014 Acked Read 5 Done Sent Done false
28579 9015 Acked Write 7 Done Sent Done false
28580 9016 Acked Write 4 Done Sent Done false
28581 9017 Acked Read 10 Done Sent Done false
28582 9018 Acked Read 4 Done Sent Done false
28583 9019 Acked Read 8 Done Sent Done false
28584 9020 Acked Read 7 Done Sent Done false
28585 9021 Acked Read 9 Done Sent Done false
28586 9022 Acked Flush 0 Done Sent Done false
28587 9023 Acked Write 3 Done Sent Done false
28588 9024 Acked Write 1 Done Sent Done false
28589 9025 Acked Flush 0 Done Sent Done false
28590 9026 Acked Flush 0 Done Sent Done false
28591 9027 Acked Read 2 Done Sent Done false
28592 9028 Acked Read 7 Done Sent Done false
28593 9029 Acked Read 1 Done Sent Done false
28594 9030 Acked Write 4 Done Sent Done false
28595 9031 Acked Write 2 Done Sent Done false
28596 9032 Acked Read 10 Done Sent Done false
28597 9033 Acked Write 8 Done Sent Done false
28598 9034 Acked Flush 0 Done Sent Done false
28599 9035 Acked Read 10 Done Sent Done false
28600 9036 Acked Read 4 Done Sent Done false
28601 9037 Acked Write 6 Done Sent Done false
28602 9038 Acked Flush 0 Done Sent Done false
28603 9039 Acked Read 6 Done Sent Done false
28604 9040 Acked Write 9 Done Sent Done false
28605 9041 Acked Read 8 Done Sent Done false
28606 9042 Acked Write 3 Done Sent Done false
28607 9043 Acked Write 4 Done Sent Done false
28608 9044 Acked Write 4 Done Sent Done false
28609 9045 Acked Write 4 Done Sent Done false
28610 9046 Acked Read 8 Done Sent Done false
28611 9047 Acked Write 10 Done Sent Done false
28612 9048 Acked Write 7 Done Sent Done false
28613 9049 Acked Read 4 Done Sent Done false
28614 9050 Acked Flush 0 Done Sent Done false
28615 9051 Acked Read 9 Done Sent Done false
28616 9052 Acked Read 7 Done Sent Done false
28617 9053 Acked Read 1 Done Sent Done false
28618 9054 Acked Read 8 Done Sent Done false
28619 9055 Acked Write 5 Done Sent Done false
28620 9056 Acked Read 7 Done Sent Done false
28621 9057 Acked Write 2 Done Sent Done false
28622 9058 Acked Write 2 Done Sent Done false
28623 9059 Acked Write 9 Done Sent Done false
28624 9060 Acked Read 4 Done Sent Done false
28625 9061 Acked Flush 0 Done Sent Done false
28626 9062 Acked Read 9 Done Sent Done false
28627 9063 Acked Write 4 Done Sent Done false
28628 9064 Acked Write 5 Done Sent Done false
28629 9065 Acked Write 6 Done Sent Done false
28630 9066 Acked Write 3 Done Sent Done false
28631 9067 Acked Write 10 Done Sent Done false
28632 9068 Acked Read 9 Done Sent Done false
28633 9069 Acked Write 9 Done Sent Done false
28634 9070 Acked Read 6 Done Sent Done false
28635 9071 Acked Read 8 Done Sent Done false
28636 9072 Acked Read 10 Done Sent Done false
28637 9073 Acked Flush 0 Done Sent Done false
28638 9074 Acked Flush 0 Done Sent Done false
28639 9075 Acked Read 9 Done Sent Done false
28640 9076 Acked Read 10 Done Sent Done false
28641 9077 Acked Write 9 Done Sent Done false
28642 9078 Acked Write 4 Done Sent Done false
28643 9079 Acked Write 7 Done Sent Done false
28644 9080 Acked Flush 0 Done Sent Done false
28645 9081 Acked Write 10 Done Sent Done false
28646 9082 Acked Read 5 Done Sent Done false
28647 9083 Acked Read 1 Done Sent Done false
28648 9084 Acked Write 8 Done Sent Done false
28649 9085 Acked Write 6 Done Sent Done false
28650 9086 Acked Read 1 Done Sent Done false
28651 9087 Acked Read 9 Done Sent Done false
28652 9088 Acked Flush 0 Done Sent Done false
28653 9089 Acked Read 9 Done Sent Done false
28654 9090 Acked Flush 0 Done Sent Done false
28655 9091 Acked Flush 0 Done Sent Done false
28656 9092 Acked Write 9 Done Sent Done false
28657 9093 Acked Read 4 Done Sent Done false
28658 9094 Acked Read 1 Done Sent Done false
28659 9095 Acked Write 8 Done Sent Done false
28660 9096 Acked Read 1 Done Sent Done false
28661 9097 Acked Write 10 Done Sent Done false
28662 9098 Acked Read 2 Done Sent Done false
28663 9099 Acked Read 10 Done Sent Done false
28664 9100 Acked Write 10 Done Sent Done false
28665 9101 Acked Write 4 Done Sent Done false
28666 9102 Acked Read 6 Done Sent Done false
28667 9103 Acked Write 10 Done Sent Done false
28668 9104 Acked Flush 0 Done Sent Done false
28669 9105 Acked Read 2 Done Sent Done false
28670 9106 Acked Write 3 Done Sent Done false
28671 9107 Acked Write 8 Done Sent Done false
28672 9108 Acked Read 2 Done Sent Done false
28673 9109 Acked Read 8 Done Sent Done false
28674 9110 Acked Read 7 Done Sent Done false
28675 9111 Acked Read 9 Done Sent Done false
28676 9112 Acked Read 3 Done Sent Done false
28677 9113 Acked Read 8 Done Sent Done false
28678 9114 Acked Write 9 Done Sent Done false
28679 9115 Acked Read 7 Done Sent Done false
28680 9116 Acked Read 8 Done Sent Done false
28681 9117 Acked Write 8 Done Sent Done false
28682 9118 Acked Read 3 Done Sent Done false
28683 9119 Acked Read 7 Done Sent Done false
28684 9120 Acked Write 4 Done Sent Done false
28685 9121 Acked Read 10 Done Sent Done false
28686 9122 Acked Write 2 Done Sent Done false
28687 9123 Acked Write 4 Done Sent Done false
28688 9124 Acked Read 10 Done Sent Done false
28689 9125 Acked Write 6 Done Sent Done false
28690 9126 Acked Read 4 Done Sent Done false
28691 9127 Acked Write 3 Done Sent Done false
28692 9128 Acked Read 7 Done Sent Done false
28693 9129 Acked Read 4 Done Sent Done false
28694 9130 Acked Write 8 Done Sent Done false
28695 9131 Acked Read 4 Done Sent Done false
28696 9132 Acked Write 7 Done Sent Done false
28697 9133 Acked Read 10 Done Sent Done false
28698 9134 Acked Write 8 Done Sent Done false
28699 9135 Acked Write 9 Done Sent Done false
28700 9136 Acked Flush 0 Done Sent Done false
28701 9137 Acked Read 1 Done Sent Done false
28702 9138 Acked Read 3 Done Sent Done false
28703 9139 Acked Read 5 Done Sent Done false
28704 9140 Acked Read 8 Done Sent Done false
28705 9141 Acked Write 1 Done Sent Done false
28706 9142 Acked Flush 0 Done Sent Done false
28707 9143 Acked Write 10 Done Sent Done false
28708 9144 Acked Write 10 Done Sent Done false
28709 9145 Acked Flush 0 Done Sent Done false
28710 9146 Acked Read 3 Done Sent Done false
28711 9147 Acked Write 1 Done Sent Done false
28712 9148 Acked Read 1 Done Sent Done false
28713 STATES DS:0 DS:1 DS:2 TOTAL
28714 Sent 0 301 0 301
28715 Done 301 0 301 602
28716 Skipped 0 0 0 0
28717 Error 0 0 0 0
28718 Last Flush: 9145 8847 9145
28719 Downstairs last five completed: 8847 8846 8845 8844 8843
28720 Upstairs last five completed: 9148 9147 9146 9145 9144
28721 CLIENT: Up:0 ds:301 act:2
28722 [8820] got start action
28723 [8820] I am going to restart
28724 Starting downstairs at port 8820
28725 Make output file at "/tmp/dsc/downstairs-8820.txt"
28726 [8820][1] reports Starting
28727 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1230)
28728 [8820][1] reports Running
28729 ----------------------------------------------------------------
28730 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28731 States: Active Offline Active
28732 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28733 8848 Acked Read 6 Done Sent Done false
28734 8849 Acked Read 2 Done Sent Done false
28735 8850 Acked Write 6 Done Sent Done false
28736 8851 Acked Read 9 Done Sent Done false
28737 8852 Acked Read 2 Done Sent Done false
28738 8853 Acked Read 7 Done Sent Done false
28739 8854 Acked Read 10 Done Sent Done false
28740 8855 Acked Read 5 Done Sent Done false
28741 8856 Acked Write 3 Done Sent Done false
28742 8857 Acked Read 3 Done Sent Done false
28743 8858 Acked Read 10 Done Sent Done false
28744 8859 Acked Write 10 Done Sent Done false
28745 8860 Acked Write 5 Done Sent Done false
28746 8861 Acked Write 5 Done Sent Done false
28747 8862 Acked Read 7 Done Sent Done false
28748 8863 Acked Write 5 Done Sent Done false
28749 8864 Acked Read 1 Done Sent Done false
28750 8865 Acked Write 1 Done Sent Done false
28751 8866 Acked Read 2 Done Sent Done false
28752 8867 Acked Read 6 Done Sent Done false
28753 8868 Acked Write 2 Done Sent Done false
28754 8869 Acked Write 10 Done Sent Done false
28755 8870 Acked Write 10 Done Sent Done false
28756 8871 Acked Read 1 Done Sent Done false
28757 8872 Acked Write 7 Done Sent Done false
28758 8873 Acked Read 10 Done Sent Done false
28759 8874 Acked Read 5 Done Sent Done false
28760 8875 Acked Read 2 Done Sent Done false
28761 8876 Acked Write 4 Done Sent Done false
28762 8877 Acked Write 5 Done Sent Done false
28763 8878 Acked Read 4 Done Sent Done false
28764 8879 Acked Write 2 Done Sent Done false
28765 8880 Acked Flush 0 Done Sent Done false
28766 8881 Acked Write 5 Done Sent Done false
28767 8882 Acked Write 4 Done Sent Done false
28768 8883 Acked Read 10 Done Sent Done false
28769 8884 Acked Read 8 Done Sent Done false
28770 8885 Acked Read 7 Done Sent Done false
28771 8886 Acked Write 4 Done Sent Done false
28772 8887 Acked Write 5 Done Sent Done false
28773 8888 Acked Read 9 Done Sent Done false
28774 8889 Acked Read 1 Done Sent Done false
28775 8890 Acked Read 2 Done Sent Done false
28776 8891 Acked Write 1 Done Sent Done false
28777 8892 Acked Read 8 Done Sent Done false
28778 8893 Acked Flush 0 Done Sent Done false
28779 8894 Acked Write 10 Done Sent Done false
28780 8895 Acked Write 7 Done Sent Done false
28781 8896 Acked Flush 0 Done Sent Done false
28782 8897 Acked Read 4 Done Sent Done false
28783 8898 Acked Write 8 Done Sent Done false
28784 8899 Acked Read 7 Done Sent Done false
28785 8900 Acked Read 7 Done Sent Done false
28786 8901 Acked Read 10 Done Sent Done false
28787 8902 Acked Write 10 Done Sent Done false
28788 8903 Acked Flush 0 Done Sent Done false
28789 8904 Acked Write 1 Done Sent Done false
28790 8905 Acked Read 1 Done Sent Done false
28791 8906 Acked Read 4 Done Sent Done false
28792 8907 Acked Write 8 Done Sent Done false
28793 8908 Acked Read 3 Done Sent Done false
28794 8909 Acked Write 2 Done Sent Done false
28795 8910 Acked Read 3 Done Sent Done false
28796 8911 Acked Read 5 Done Sent Done false
28797 8912 Acked Write 2 Done Sent Done false
28798 8913 Acked Write 10 Done Sent Done false
28799 8914 Acked Read 8 Done Sent Done false
28800 8915 Acked Read 1 Done Sent Done false
28801 8916 Acked Write 6 Done Sent Done false
28802 8917 Acked Flush 0 Done Sent Done false
28803 8918 Acked Read 7 Done Sent Done false
28804 8919 Acked Read 3 Done Sent Done false
28805 8920 Acked Flush 0 Done Sent Done false
28806 8921 Acked Read 6 Done Sent Done false
28807 8922 Acked Write 6 Done Sent Done false
28808 8923 Acked Write 7 Done Sent Done false
28809 8924 Acked Read 9 Done Sent Done false
28810 8925 Acked Write 5 Done Sent Done false
28811 8926 Acked Write 5 Done Sent Done false
28812 8927 Acked Read 8 Done Sent Done false
28813 8928 Acked Read 8 Done Sent Done false
28814 8929 Acked Write 3 Done Sent Done false
28815 8930 Acked Write 8 Done Sent Done false
28816 8931 Acked Read 5 Done Sent Done false
28817 8932 Acked Write 10 Done Sent Done false
28818 8933 Acked Read 9 Done Sent Done false
28819 8934 Acked Write 5 Done Sent Done false
28820 8935 Acked Write 6 Done Sent Done false
28821 8936 Acked Read 2 Done Sent Done false
28822 8937 Acked Read 1 Done Sent Done false
28823 8938 Acked Flush 0 Done Sent Done false
28824 8939 Acked Write 6 Done Sent Done false
28825 8940 Acked Write 5 Done Sent Done false
28826 8941 Acked Read 4 Done Sent Done false
28827 8942 Acked Write 2 Done Sent Done false
28828 8943 Acked Read 3 Done Sent Done false
28829 8944 Acked Read 9 Done Sent Done false
28830 8945 Acked Write 6 Done Sent Done false
28831 8946 Acked Write 1 Done Sent Done false
28832 8947 Acked Write 5 Done Sent Done false
28833 8948 Acked Write 9 Done Sent Done false
28834 8949 Acked Read 7 Done Sent Done false
28835 8950 Acked Flush 0 Done Sent Done false
28836 8951 Acked Read 6 Done Sent Done false
28837 8952 Acked Flush 0 Done Sent Done false
28838 8953 Acked Write 6 Done Sent Done false
28839 8954 Acked Read 1 Done Sent Done false
28840 8955 Acked Flush 0 Done Sent Done false
28841 8956 Acked Write 3 Done Sent Done false
28842 8957 Acked Write 10 Done Sent Done false
28843 8958 Acked Read 2 Done Sent Done false
28844 8959 Acked Flush 0 Done Sent Done false
28845 8960 Acked Read 4 Done Sent Done false
28846 8961 Acked Write 1 Done Sent Done false
28847 8962 Acked Write 6 Done Sent Done false
28848 8963 Acked Read 10 Done Sent Done false
28849 8964 Acked Read 2 Done Sent Done false
28850 8965 Acked Read 1 Done Sent Done false
28851 8966 Acked Read 9 Done Sent Done false
28852 8967 Acked Read 2 Done Sent Done false
28853 8968 Acked Write 5 Done Sent Done false
28854 8969 Acked Write 7 Done Sent Done false
28855 8970 Acked Read 5 Done Sent Done false
28856 8971 Acked Read 10 Done Sent Done false
28857 8972 Acked Read 6 Done Sent Done false
28858 8973 Acked Write 6 Done Sent Done false
28859 8974 Acked Read 4 Done Sent Done false
28860 8975 Acked Read 7 Done Sent Done false
28861 8976 Acked Write 5 Done Sent Done false
28862 8977 Acked Read 5 Done Sent Done false
28863 8978 Acked Read 8 Done Sent Done false
28864 8979 Acked Write 2 Done Sent Done false
28865 8980 Acked Read 4 Done Sent Done false
28866 8981 Acked Write 5 Done Sent Done false
28867 8982 Acked Read 4 Done Sent Done false
28868 8983 Acked Write 5 Done Sent Done false
28869 8984 Acked Write 8 Done Sent Done false
28870 8985 Acked Read 8 Done Sent Done false
28871 8986 Acked Flush 0 Done Sent Done false
28872 8987 Acked Read 2 Done Sent Done false
28873 8988 Acked Read 7 Done Sent Done false
28874 8989 Acked Flush 0 Done Sent Done false
28875 8990 Acked Write 6 Done Sent Done false
28876 8991 Acked Write 9 Done Sent Done false
28877 8992 Acked Read 1 Done Sent Done false
28878 8993 Acked Read 10 Done Sent Done false
28879 8994 Acked Read 9 Done Sent Done false
28880 8995 Acked Flush 0 Done Sent Done false
28881 8996 Acked Read 3 Done Sent Done false
28882 8997 Acked Read 1 Done Sent Done false
28883 8998 Acked Read 4 Done Sent Done false
28884 8999 Acked Read 5 Done Sent Done false
28885 9000 Acked Write 6 Done Sent Done false
28886 9001 Acked Write 7 Done Sent Done false
28887 9002 Acked Write 7 Done Sent Done false
28888 9003 Acked Write 7 Done Sent Done false
28889 9004 Acked Write 5 Done Sent Done false
28890 9005 Acked Read 7 Done Sent Done false
28891 9006 Acked Read 10 Done Sent Done false
28892 9007 Acked Write 3 Done Sent Done false
28893 9008 Acked Write 4 Done Sent Done false
28894 9009 Acked Write 8 Done Sent Done false
28895 9010 Acked Write 4 Done Sent Done false
28896 9011 Acked Read 2 Done Sent Done false
28897 9012 Acked Write 2 Done Sent Done false
28898 9013 Acked Flush 0 Done Sent Done false
28899 9014 Acked Read 5 Done Sent Done false
28900 9015 Acked Write 7 Done Sent Done false
28901 9016 Acked Write 4 Done Sent Done false
28902 9017 Acked Read 10 Done Sent Done false
28903 9018 Acked Read 4 Done Sent Done false
28904 9019 Acked Read 8 Done Sent Done false
28905 9020 Acked Read 7 Done Sent Done false
28906 9021 Acked Read 9 Done Sent Done false
28907 9022 Acked Flush 0 Done Sent Done false
28908 9023 Acked Write 3 Done Sent Done false
28909 9024 Acked Write 1 Done Sent Done false
28910 9025 Acked Flush 0 Done Sent Done false
28911 9026 Acked Flush 0 Done Sent Done false
28912 9027 Acked Read 2 Done Sent Done false
28913 9028 Acked Read 7 Done Sent Done false
28914 9029 Acked Read 1 Done Sent Done false
28915 9030 Acked Write 4 Done Sent Done false
28916 9031 Acked Write 2 Done Sent Done false
28917 9032 Acked Read 10 Done Sent Done false
28918 9033 Acked Write 8 Done Sent Done false
28919 9034 Acked Flush 0 Done Sent Done false
28920 9035 Acked Read 10 Done Sent Done false
28921 9036 Acked Read 4 Done Sent Done false
28922 9037 Acked Write 6 Done Sent Done false
28923 9038 Acked Flush 0 Done Sent Done false
28924 9039 Acked Read 6 Done Sent Done false
28925 9040 Acked Write 9 Done Sent Done false
28926 9041 Acked Read 8 Done Sent Done false
28927 9042 Acked Write 3 Done Sent Done false
28928 9043 Acked Write 4 Done Sent Done false
28929 9044 Acked Write 4 Done Sent Done false
28930 9045 Acked Write 4 Done Sent Done false
28931 9046 Acked Read 8 Done Sent Done false
28932 9047 Acked Write 10 Done Sent Done false
28933 9048 Acked Write 7 Done Sent Done false
28934 9049 Acked Read 4 Done Sent Done false
28935 9050 Acked Flush 0 Done Sent Done false
28936 9051 Acked Read 9 Done Sent Done false
28937 9052 Acked Read 7 Done Sent Done false
28938 9053 Acked Read 1 Done Sent Done false
28939 9054 Acked Read 8 Done Sent Done false
28940 9055 Acked Write 5 Done Sent Done false
28941 9056 Acked Read 7 Done Sent Done false
28942 9057 Acked Write 2 Done Sent Done false
28943 9058 Acked Write 2 Done Sent Done false
28944 9059 Acked Write 9 Done Sent Done false
28945 9060 Acked Read 4 Done Sent Done false
28946 9061 Acked Flush 0 Done Sent Done false
28947 9062 Acked Read 9 Done Sent Done false
28948 9063 Acked Write 4 Done Sent Done false
28949 9064 Acked Write 5 Done Sent Done false
28950 9065 Acked Write 6 Done Sent Done false
28951 9066 Acked Write 3 Done Sent Done false
28952 9067 Acked Write 10 Done Sent Done false
28953 9068 Acked Read 9 Done Sent Done false
28954 9069 Acked Write 9 Done Sent Done false
28955 9070 Acked Read 6 Done Sent Done false
28956 9071 Acked Read 8 Done Sent Done false
28957 9072 Acked Read 10 Done Sent Done false
28958 9073 Acked Flush 0 Done Sent Done false
28959 9074 Acked Flush 0 Done Sent Done false
28960 9075 Acked Read 9 Done Sent Done false
28961 9076 Acked Read 10 Done Sent Done false
28962 9077 Acked Write 9 Done Sent Done false
28963 9078 Acked Write 4 Done Sent Done false
28964 9079 Acked Write 7 Done Sent Done false
28965 9080 Acked Flush 0 Done Sent Done false
28966 9081 Acked Write 10 Done Sent Done false
28967 9082 Acked Read 5 Done Sent Done false
28968 9083 Acked Read 1 Done Sent Done false
28969 9084 Acked Write 8 Done Sent Done false
28970 9085 Acked Write 6 Done Sent Done false
28971 9086 Acked Read 1 Done Sent Done false
28972 9087 Acked Read 9 Done Sent Done false
28973 9088 Acked Flush 0 Done Sent Done false
28974 9089 Acked Read 9 Done Sent Done false
28975 9090 Acked Flush 0 Done Sent Done false
28976 9091 Acked Flush 0 Done Sent Done false
28977 9092 Acked Write 9 Done Sent Done false
28978 9093 Acked Read 4 Done Sent Done false
28979 9094 Acked Read 1 Done Sent Done false
28980 9095 Acked Write 8 Done Sent Done false
28981 9096 Acked Read 1 Done Sent Done false
28982 9097 Acked Write 10 Done Sent Done false
28983 9098 Acked Read 2 Done Sent Done false
28984 9099 Acked Read 10 Done Sent Done false
28985 9100 Acked Write 10 Done Sent Done false
28986 9101 Acked Write 4 Done Sent Done false
28987 9102 Acked Read 6 Done Sent Done false
28988 9103 Acked Write 10 Done Sent Done false
28989 9104 Acked Flush 0 Done Sent Done false
28990 9105 Acked Read 2 Done Sent Done false
28991 9106 Acked Write 3 Done Sent Done false
28992 9107 Acked Write 8 Done Sent Done false
28993 9108 Acked Read 2 Done Sent Done false
28994 9109 Acked Read 8 Done Sent Done false
28995 9110 Acked Read 7 Done Sent Done false
28996 9111 Acked Read 9 Done Sent Done false
28997 9112 Acked Read 3 Done Sent Done false
28998 9113 Acked Read 8 Done Sent Done false
28999 9114 Acked Write 9 Done Sent Done false
29000 9115 Acked Read 7 Done Sent Done false
29001 9116 Acked Read 8 Done Sent Done false
29002 9117 Acked Write 8 Done Sent Done false
29003 9118 Acked Read 3 Done Sent Done false
29004 9119 Acked Read 7 Done Sent Done false
29005 9120 Acked Write 4 Done Sent Done false
29006 9121 Acked Read 10 Done Sent Done false
29007 9122 Acked Write 2 Done Sent Done false
29008 9123 Acked Write 4 Done Sent Done false
29009 9124 Acked Read 10 Done Sent Done false
29010 9125 Acked Write 6 Done Sent Done false
29011 9126 Acked Read 4 Done Sent Done false
29012 9127 Acked Write 3 Done Sent Done false
29013 9128 Acked Read 7 Done Sent Done false
29014 9129 Acked Read 4 Done Sent Done false
29015 9130 Acked Write 8 Done Sent Done false
29016 9131 Acked Read 4 Done Sent Done false
29017 9132 Acked Write 7 Done Sent Done false
29018 9133 Acked Read 10 Done Sent Done false
29019 9134 Acked Write 8 Done Sent Done false
29020 9135 Acked Write 9 Done Sent Done false
29021 9136 Acked Flush 0 Done Sent Done false
29022 9137 Acked Read 1 Done Sent Done false
29023 9138 Acked Read 3 Done Sent Done false
29024 9139 Acked Read 5 Done Sent Done false
29025 9140 Acked Read 8 Done Sent Done false
29026 9141 Acked Write 1 Done Sent Done false
29027 9142 Acked Flush 0 Done Sent Done false
29028 9143 Acked Write 10 Done Sent Done false
29029 9144 Acked Write 10 Done Sent Done false
29030 9145 Acked Flush 0 Done Sent Done false
29031 9146 Acked Read 3 Done Sent Done false
29032 9147 Acked Write 1 Done Sent Done false
29033 9148 Acked Read 1 Done Sent Done false
29034 9149 Acked Flush 0 Done Sent Done false
29035 STATES DS:0 DS:1 DS:2 TOTAL
29036 Sent 0 302 0 302
29037 Done 302 0 302 604
29038 Skipped 0 0 0 0
29039 Error 0 0 0 0
29040 Last Flush: 9149 8847 9149
29041 Downstairs last five completed: 8847 8846 8845 8844 8843
29042 Upstairs last five completed: 9149 9148 9147 9146 9145
29043 CLIENT: Up:0 ds:302 act:2
290442024-12-04T13:36:46.519ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290452024-12-04T13:36:46.519ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42932) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290462024-12-04T13:36:46.520ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290472024-12-04T13:36:46.521ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290482024-12-04T13:36:46.521ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290492024-12-04T13:36:46.521ZINFOcrucible: send last flush ID to this DS: 8847 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290502024-12-04T13:36:46.521ZINFOcrucible: Replied this last flush ID: 8847 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290512024-12-04T13:36:46.521ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290522024-12-04T13:36:46.521ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290532024-12-04T13:36:46.521ZINFOcrucible: [1] client re-new 302 jobs since flush 8847 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290542024-12-04T13:36:46.521ZINFOcrucible: [1] Replayed 302 jobs since flush: 8847 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
29055 ----------------------------------------------------------------
29056 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29057 Downstairs last five completed: 9149 9148 9147 9146 9145
29058 Upstairs last five completed: 9149 9148 9147 9146 9145
29059 CLIENT: Up:0 ds:0 act:3
29060 Replay: All jobs finished
29061 Main task has work to do, go find it
29062 got dsc Stop(0)
29063 stop 0
29064 Dec 04 13:36:48.761 INFO request completed, latency_us: 347, response_code: 204, uri:[8810] Got stop action so:false kr:false
29065 /stop/cid/0, method: POST, req_id: 8e3ea9f1-cdc9-4da7-b796-e5d4d6415694, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
290662024-12-04T13:36:48.762ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290672024-12-04T13:36:48.762ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290682024-12-04T13:36:48.762ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290692024-12-04T13:36:48.762ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290702024-12-04T13:36:48.762ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
290712024-12-04T13:36:48.762ZINFOcrucible: client task is exiting = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
29072 Dec 04 13:36:48.762 INFO request completed, latency_us: 140, response_code: 200, uri: /state/cid/0, method: GET, req_id: 33fd7882-75f3-4adf-9662-92d7c2855dc6, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
29073 [8810] Exited with: ExitStatus(unix_wait_status(9))
29074 [8810][0] reports Exit
29075 Dec 04 13:36:52.765 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/0, method: GET, req_id: f3977f9c-fba1-4080-8576-2a6761b3a673, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
29076 001/300 Read block 2025 len 32768
29077 002/300 Read block 4516 len 4096
29078 003/300 Read block 4494 len 40960
29079 004/300 Write block 1067 len 24576 data: 9 10 10 13 9 7
29080 005/300 Read block 1597 len 32768
29081 006/300 Read block 3693 len 36864
29082 007/300 Write block 3686 len 24576 data: 7 9 10 9 7 8
29083 008/300 Read block 3479 len 24576
29084 009/300 Write block 2291 len 28672 data: 10 11 10 10 7 5 4
29085 010/300 Read block 1957 len 36864
29086 011/300 Read block 4873 len 12288
29087 012/300 Write block 4247 len 16384 data: 8 10 11 8
29088 013/300 Read block 4487 len 8192
29089 014/300 Read block 2683 len 32768
29090 015/300 Read block 4774 len 12288
29091 016/300 Read block 1657 len 8192
29092 017/300 Write block 3859 len 20480 data: 5 6 6 6 6
29093 018/300 Flush
29094 019/300 Write block 2895 len 36864 data: 7 6 6 6 6 6 8 7 8
29095 020/300 Write block 1191 len 20480 data: 5 6 5 6 4
29096 021/300 Write block 3126 len 24576 data: 7 7 6 6 6 5
29097 022/300 Flush
29098 023/300 Read block 2900 len 40960
29099 024/300 Read block 1889 len 24576
29100 025/300 Write block 2111 len 24576 data: 6 4 2 2 2 2
29101 026/300 Read block 4720 len 4096
29102 027/300 Write block 2817 len 8192 data: 5 5
29103 028/300 Write block 4782 len 28672 data: 6 7 6 5 5 5 5
29104 029/300 Read block 2807 len 4096
29105 030/300 Write block 2602 len 20480 data: 7 7 5 5 5
29106 031/300 Write block 486 len 8192 data: 8 7
29107 032/300 Read block 179 len 12288
29108 033/300 Write block 1384 len 36864 data: 6 5 4 5 4 5 5 5 5
29109 034/300 Write block 487 len 28672 data: 8 7 8 7 7 3 3
29110 035/300 Write block 2351 len 40960 data: 7 9 9 7 6 4 4 4 5 5
29111 036/300 Read block 2066 len 40960
29112 037/300 Flush
29113 038/300 Read block 1528 len 4096
29114 039/300 Write block 412 len 36864 data: 6 6 5 6 7 8 6 6 7
29115 040/300 Read block 2348 len 28672
29116 041/300 Read block 1599 len 4096
29117 042/300 Read block 827 len 36864
29118 043/300 Read block 742 len 12288
29119 044/300 Write block 2239 len 4096 data: 5
29120 045/300 Read block 1726 len 40960
29121 046/300 Write block 918 len 8192 data: 5 4
29122 047/300 Write block 2597 len 40960 data: 5 4 4 6 6 8 8 6 6 6
29123 048/300 Read block 1651 len 40960
29124 049/300 Flush
29125 050/300 Write block 3797 len 20480 data: 3 2 2 3 3
29126 051/300 Write block 3304 len 20480 data: 5 6 5 5 6
29127 052/300 Read block 242 len 20480
29128 053/300 Write block 3476 len 32768 data: 11 10 10 10 11 9 7 6
29129 054/300 Write block 4254 len 28672 data: 6 5 5 4 3 5 7
29130 055/300 Write block 1286 len 12288 data: 6 5 5
29131 056/300 Flush
29132 057/300 Read block 1468 len 16384
29133 058/300 Read block 1250 len 20480
29134 059/300 Read block 142 len 20480
29135 060/300 Read block 1246 len 16384
29136 061/300 Flush
29137 062/300 Flush
29138 063/300 Flush
29139 064/300 Write block 1642 len 16384 data: 4 4 4 5
29140 065/300 Read block 439 len 28672
29141 066/300 Read block 422 len 4096
29142 067/300 Write block 4646 len 4096 data: 6
29143 068/300 Read block 4349 len 20480
29144 069/300 Read block 4117 len 8192
29145 070/300 Write block 2236 len 28672 data: 7 5 6 6 5 4 5
29146 071/300 Read block 2775 len 28672
29147 072/300 Write block 1193 len 16384 data: 6 7 5 4
29148 073/300 Read block 91 len 36864
29149 074/300 Write block 2306 len 20480 data: 6 6 5 4 4
29150 075/300 Write block 1849 len 8192 data: 7 7
29151 076/300 Read block 1126 len 24576
29152 077/300 Write block 2126 len 36864 data: 6 6 5 3 4 5 6 6 8
29153 078/300 Read block 1285 len 24576
29154 079/300 Read block 4333 len 28672
29155 080/300 Read block 2638 len 4096
29156 081/300 Write block 1315 len 36864 data: 4 3 5 6 7 8 6 6 5
29157 082/300 Read block 3376 len 12288
29158 083/300 Read block 1349 len 24576
29159 084/300 Read block 2742 len 16384
29160 085/300 Write block 4720 len 20480 data: 6 6 7 5 5
29161 086/300 Read block 786 len 20480
29162 087/300 Read block 991 len 12288
29163 088/300 Write block 1546 len 12288 data: 8 7 6
29164 089/300 Write block 4326 len 24576 data: 6 6 5 7 8 10
29165 090/300 Read block 1698 len 12288
29166 091/300 Write block 2920 len 28672 data: 3 5 6 6 6 7 7
29167 092/300 Read block 3533 len 16384
29168 093/300 Read block 1080 len 36864
29169 094/300 Read block 3589 len 36864
29170 095/300 Read block 2491 len 12288
29171 096/300 Read block 3959 len 20480
29172 097/300 Read block 4209 len 4096
29173 098/300 Write block 4077 len 24576 data: 7 6 5 5 5 5
29174 099/300 Read block 1420 len 36864
29175 100/300 Read block 289 len 32768
29176 101/300 Write block 1483 len 32768 data: 5 6 6 5 5 6 5 5
29177 102/300 Read block 3839 len 20480
29178 103/300 Read block 855 len 4096
29179 104/300 Read block 2214 len 20480
29180 105/300 Write block 443 len 4096 data: 11
29181 106/300 Write block 2854 len 36864 data: 7 5 2 4 4 5 5 6 5
29182 107/300 Read block 4694 len 32768
29183 108/300 Read block 2603 len 24576
29184 109/300 Read block 1624 len 28672
29185 110/300 Read block 1619 len 4096
29186 111/300 Write block 2194 len 12288 data: 9 9 8
29187 112/300 Read block 1323 len 40960
29188 113/300 Write block 371 len 12288 data: 7 7 7
29189 114/300 Write block 3479 len 8192 data: 11 12
29190 115/300 Read block 2737 len 36864
29191 116/300 Read block 3046 len 12288
29192 117/300 Write block 2025 len 24576 data: 6 7 6 6 5 5
29193 118/300 Write block 3753 len 12288 data: 4 3 3
29194 119/300 Write block 2132 len 32768 data: 7 7 9 6 8 9 8 8
29195 120/300 Read block 4860 len 16384
29196 121/300 Write block 2663 len 12288 data: 2 4 5
29197 122/300 Write block 1940 len 20480 data: 7 7 8 7 7
29198 123/300 Flush
29199 124/300 Write block 3974 len 16384 data: 7 8 5 5
29200 125/300 Read block 777 len 32768
29201 126/300 Flush
29202 127/300 Read block 4848 len 12288
29203 128/300 Write block 229 len 8192 data: 4 5
29204 129/300 Read block 2711 len 24576
29205 130/300 Read block 4349 len 12288
29206 131/300 Write block 1681 len 28672 data: 7 7 9 11 10 9 8
29207 132/300 Write block 2660 len 32768 data: 4 4 3 3 5 6 6 8
29208 133/300 Write block 1670 len 28672 data: 3 4 5 5 6 6 6
29209 134/300 Write block 4764 len 28672 data: 7 7 6 6 6 7 8
29210 135/300 Flush
29211 136/300 Read block 361 len 24576
29212 137/300 Read block 4117 len 12288
29213 138/300 Read block 144 len 32768
29214 139/300 Flush
29215 140/300 Write block 3611 len 12288 data: 5 5 5
29216 141/300 Write block 206 len 16384 data: 6 5 7 4
29217 142/300 Flush
29218 143/300 Flush
29219 144/300 Read block 4922 len 24576
29220 145/300 Read block 2891 len 4096
29221 146/300 Write block 253 len 20480 data: 11 9 8 5 4
29222 147/300 Flush
29223 148/300 Write block 748 len 40960 data: 5 5 4 5 6 7 9 7 6 6
29224 149/300 Read block 4820 len 32768
29225 150/300 Read block 2600 len 40960
29226 151/300 Write block 391 len 16384 data: 7 7 8 7
29227 152/300 Read block 4866 len 8192
29228 153/300 Write block 4072 len 24576 data: 5 6 5 7 7 8
29229 154/300 Read block 688 len 24576
29230 155/300 Read block 1154 len 20480
29231 156/300 Flush
29232 157/300 Write block 375 len 36864 data: 6 5 4 3 2 2 3 3 3
29233 158/300 Read block 3861 len 24576
29234 159/300 Read block 2957 len 16384
29235 160/300 Read block 801 len 20480
29236 161/300 Read block 3472 len 28672
29237 162/300 Read block 2463 len 24576
29238 163/300 Read block 4118 len 36864
29239 164/300 Flush
29240 165/300 Write block 4748 len 36864 data: 4 4 5 4 5 5 6 7 7
29241 166/300 Read block 2984 len 20480
29242 167/300 Read block 4081 len 36864
29243 168/300 Read block 402 len 40960
29244 169/300 Write block 702 len 36864 data: 5 5 5 6 5 5 6 5 4
29245 170/300 Write block 3436 len 40960 data: 10 9 9 9 7 5 5 6 7 6
29246 171/300 Read block 4329 len 8192
29247 172/300 Read block 4635 len 32768
29248 173/300 Write block 1403 len 16384 data: 6 6 6 6
29249 174/300 Read block 2930 len 36864
29250 175/300 Write block 1128 len 20480 data: 3 2 3 3 2
29251 176/300 Write block 898 len 32768 data: 5 5 5 4 3 3 3 2
29252 177/300 Read block 2700 len 16384
29253 178/300 Write block 1067 len 32768 data: 10 11 11 14 10 8 6 6
29254 179/300 Write block 100 len 24576 data: 6 6 7 6 5 5
29255 180/300 Read block 1577 len 28672
29256 181/300 Write block 2450 len 8192 data: 4 4
29257 182/300 Write block 4479 len 8192 data: 6 5
29258 183/300 Read block 1711 len 12288
29259 184/300 Read block 4108 len 32768
29260 185/300 Read block 464 len 20480
29261 186/300 Read block 1556 len 32768
29262 187/300 Write block 2810 len 16384 data: 6 5 3 3
29263 188/300 Write block 939 len 4096 data: 6
29264 189/300 Write block 904 len 40960 data: 4 3 2 2 2 2 2 2 3 4
29265 190/300 Read block 1561 len 24576
29266 191/300 Read block 1935 len 32768
29267 192/300 Read block 4746 len 16384
29268 193/300 Flush
29269 194/300 Read block 4810 len 8192
29270 195/300 Read block 3698 len 24576
29271 196/300 Write block 4790 len 16384 data: 5 4 5 5
29272 197/300 Write block 511 len 32768 data: 8 5 5 6 6 4 4 4
29273 198/300 Write block 3597 len 24576 data: 5 6 7 7 7 7
29274 199/300 Write block 1841 len 40960 data: 8 6 7 7 8 7 8 6 8 8
29275 200/300 Write block 2346 len 20480 data: 7 8 10 8 8
29276 201/300 Read block 2922 len 40960
29277 202/300 Read block 3120 len 8192
29278 203/300 Read block 2913 len 8192
29279 204/300 Write block 746 len 4096 data: 6
29280 205/300 Read block 3401 len 16384
29281 206/300 Read block 1147 len 16384
29282 207/300 Write block 301 len 28672 data: 4 4 5 5 6 7 7
29283 208/300 Write block 3207 len 12288 data: 4 4 6
29284 209/300 Read block 3494 len 32768
29285 210/300 Read block 3560 len 40960
29286 211/300 Read block 4535 len 8192
29287 212/300 Write block 4799 len 36864 data: 6 6 8 8 9 8 8 7 6
29288 213/300 Write block 978 len 20480 data: 7 8 8 7 7
29289 214/300 Write block 1843 len 16384 data: 8 8 9 8
29290 215/300 Write block 3518 len 28672 data: 9 9 7 7 5 4 3
29291 216/300 Write block 4738 len 20480 data: 5 6 6 5 4
29292 217/300 Write block 4098 len 24576 data: 4 5 4 4 4 4
29293 218/300 Read block 887 len 8192
29294 219/300 Write block 3466 len 40960 data: 6 7 7 6 6 6 4 5 8 10
29295 220/300 Write block 4435 len 4096 data: 6
29296 221/300 Read block 430 len 8192
29297 222/300 Read block 3271 len 32768
29298 223/300 Read block 3684 len 36864
29299 224/300 Read block 3691 len 32768
29300 225/300 Read block 215 len 32768
29301 226/300 Write block 1078 len 4096 data: 10
29302 227/300 Flush
29303 228/300 Read block 1876 len 16384
29304 229/300 Read block 641 len 24576
29305 230/300 Write block 321 len 16384 data: 5 6 7 7
29306 231/300 Read block 1231 len 24576
29307 232/300 Write block 2664 len 40960 data: 6 7 7 9 7 7 8 8 6 5
29308 233/300 Write block 950 len 20480 data: 8 7 7 6 5
29309 234/300 Read block 2706 len 36864
29310 235/300 Read block 1914 len 4096
29311 236/300 Write block 1572 len 16384 data: 4 4 3 3
29312 237/300 Flush
29313 238/300 Write block 1908 len 20480 data: 4 4 5 9 9
29314 239/300 Read block 3796 len 16384
29315 240/300 Read block 994 len 32768
29316 241/300 Write block 2401 len 20480 data: 6 7 6 7 6
29317 242/300 Read block 1631 len 4096
29318 243/300 Read block 561 len 8192
29319 244/300 Write block 3923 len 16384 data: 5 5 6 7
29320 245/300 Read block 4980 len 4096
29321 246/300 Write block 3089 len 24576 data: 6 6 6 6 6 5
29322 247/300 Read block 436 len 32768
29323 248/300 Write block 269 len 40960 data: 6 5 3 4 5 7 7 7 6 7
29324 249/300 Write block 549 len 24576 data: 6 8 5 4 4 3
29325 250/300 Flush
29326 251/300 Read block 686 len 16384
29327 252/300 Read block 4391 len 32768
29328 253/300 Write block 4840 len 40960 data: 8 8 8 7 6 5 5 6 4 4
29329 254/300 Read block 2055 len 16384
29330 255/300 Write block 3075 len 16384 data: 3 3 3 3
29331 256/300 Read block 4520 len 12288
29332 257/300 Write block 4152 len 16384 data: 5 7 7 9
29333 258/300 Read block 2262 len 16384
29334 259/300 Read block 2310 len 16384
29335 260/300 Write block 3233 len 20480 data: 8 9 9 10 9
29336 261/300 Read block 2298 len 4096
29337 262/300 Write block 2422 len 4096 data: 4
29338 263/300 Read block 3529 len 28672
29339 264/300 Read block 387 len 40960
29340 265/300 Read block 733 len 8192
29341 266/300 Write block 4678 len 12288 data: 7 8 8
29342 267/300 Write block 1492 len 28672 data: 5 6 5 5 6 7 8
29343 268/300 Read block 1661 len 36864
29344 269/300 Write block 958 len 40960 data: 4 4 4 4 5 5 5 5 5 7
29345 270/300 Read block 2256 len 12288
29346 271/300 Write block 1312 len 20480 data: 3 3 4 5 4
29347 272/300 Read block 4160 len 20480
29348 273/300 Write block 3755 len 16384 data: 4 4 6 6
29349 274/300 Write block 4189 len 36864 data: 2 2 6 6 7 7 6 6 8
29350 275/300 Read block 4169 len 16384
29351 276/300 Read block 2120 len 16384
29352 277/300 Write block 1790 len 8192 data: 5 4
29353 278/300 Read block 4982 len 8192
29354 279/300 Read block 541 len 16384
29355 280/300 Read block 3024 len 12288
29356 281/300 Flush
29357 282/300 Write block 2212 len 32768 data: 5 5 4 4 4 5 6 5
29358 283/300 Write block 524 len 32768 data: 5 5 5 3 3 4 4 4
29359 284/300 Read block 222 len 16384
29360 285/300 Write block 1474 len 20480 data: 8 8 8 8 10
29361 286/300 Read block 2928 len 16384
29362 287/300 Read block 2940 len 40960
29363 288/300 Write block 3251 len 24576 data: 3 4 5 4 6 4
29364 289/300 Read block 1489 len 16384
29365 290/300 Write block 861 len 32768 data: 5 4 3 4 4 5 5 5
29366 291/300 Read block 3428 len 32768
29367 292/300 Write block 133 len 12288 data: 6 3 3
29368 293/300 Write block 1491 len 16384 data: 5 6 7 6
29369 294/300 Flush
29370 295/300 Read block 3162 len 16384
29371 296/300 Write block 1472 len 20480 data: 8 7 9 9 9
29372 297/300 Read block 3150 len 24576
29373 298/300 Read block 1871 len 32768
29374 299/300 Write block 3540 len 8192 data: 7 8
29375 300/300 Read block 4611 len 12288
29376 301/300 Read block 2277 len 12288
29377 Main task has work to do, go find it
29378 got dsc Start(0)
29379 start 0
29380 Dec 04 13:36:53.001 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: 50bcb2ad-b8cd-4c43-8a1d-7d93868ae553, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
29381 Replay: started 0, returned:Ok(())
29382 ----------------------------------------------------------------
29383 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29384 States: Offline Active Active
29385 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29386 9150 Acked Read 8 Sent Done Done false
29387 9151 Acked Read 1 Sent Done Done false
29388 9152 Acked Read 10 Sent Done Done false
29389 9153 Acked Write 6 Sent Done Done false
29390 9154 Acked Read 8 Sent Done Done false
29391 9155 Acked Read 9 Sent Done Done false
29392 9156 Acked Write 6 Sent Done Done false
29393 9157 Acked Read 6 Sent Done Done false
29394 9158 Acked Write 7 Sent Done Done false
29395 9159 Acked Read 9 Sent Done Done false
29396 9160 Acked Read 3 Sent Done Done false
29397 9161 Acked Write 4 Sent Done Done false
29398 9162 Acked Read 2 Sent Done Done false
29399 9163 Acked Read 8 Sent Done Done false
29400 9164 Acked Read 3 Sent Done Done false
29401 9165 Acked Read 2 Sent Done Done false
29402 9166 Acked Write 5 Sent Done Done false
29403 9167 Acked Flush 0 Sent Done Done false
29404 9168 Acked Write 9 Sent Done Done false
29405 9169 Acked Write 5 Sent Done Done false
29406 9170 Acked Write 6 Sent Done Done false
29407 9171 Acked Flush 0 Sent Done Done false
29408 9172 Acked Read 10 Sent Done Done false
29409 9173 Acked Read 6 Sent Done Done false
29410 9174 Acked Write 6 Sent Done Done false
29411 9175 Acked Read 1 Sent Done Done false
29412 9176 Acked Write 2 Sent Done Done false
29413 9177 Acked Write 7 Sent Done Done false
29414 9178 Acked Read 1 Sent Done Done false
29415 9179 Acked Write 5 Sent Done Done false
29416 9180 Acked Write 2 Sent Done Done false
29417 9181 Acked Read 3 Sent Done Done false
29418 9182 Acked Write 9 Sent Done Done false
29419 9183 Acked Write 7 Sent Done Done false
29420 9184 Acked Write 10 Sent Done Done false
29421 9185 Acked Read 10 Sent Done Done false
29422 9186 Acked Flush 0 Sent Done Done false
29423 9187 Acked Read 1 Sent Done Done false
29424 9188 Acked Write 9 Sent Done Done false
29425 9189 Acked Read 7 Sent Done Done false
29426 9190 Acked Read 1 Sent Done Done false
29427 9191 Acked Read 9 Sent Done Done false
29428 9192 Acked Read 3 Sent Done Done false
29429 9193 Acked Write 1 Sent Done Done false
29430 9194 Acked Read 10 Sent Done Done false
29431 9195 Acked Write 2 Sent Done Done false
29432 9196 Acked Write 10 Sent Done Done false
29433 9197 Acked Read 10 Sent Done Done false
29434 9198 Acked Flush 0 Sent Done Done false
29435 9199 Acked Write 5 Sent Done Done false
29436 9200 Acked Write 5 Sent Done Done false
29437 9201 Acked Read 5 Sent Done Done false
29438 9202 Acked Write 8 Sent Done Done false
29439 9203 Acked Write 7 Sent Done Done false
29440 9204 Acked Write 3 Sent Done Done false
29441 9205 Acked Flush 0 Sent Done Done false
29442 9206 Acked Read 4 Sent Done Done false
29443 9207 Acked Read 5 Sent Done Done false
29444 9208 Acked Read 5 Sent Done Done false
29445 9209 Acked Read 4 Sent Done Done false
29446 9210 Acked Flush 0 Sent Done Done false
29447 9211 Acked Flush 0 Sent Done Done false
29448 9212 Acked Flush 0 Sent Done Done false
29449 9213 Acked Write 4 Sent Done Done false
29450 9214 Acked Read 7 Sent Done Done false
29451 9215 Acked Read 1 Sent Done Done false
29452 9216 Acked Write 1 Sent Done Done false
29453 9217 Acked Read 5 Sent Done Done false
29454 9218 Acked Read 2 Sent Done Done false
29455 9219 Acked Write 7 Sent Done Done false
29456 9220 Acked Read 7 Sent Done Done false
29457 9221 Acked Write 4 Sent Done Done false
29458 9222 Acked Read 9 Sent Done Done false
29459 9223 Acked Write 5 Sent Done Done false
29460 9224 Acked Write 2 Sent Done Done false
29461 9225 Acked Read 6 Sent Done Done false
29462 9226 Acked Write 9 Sent Done Done false
29463 9227 Acked Read 6 Sent Done Done false
29464 9228 Acked Read 7 Sent Done Done false
29465 9229 Acked Read 1 Sent Done Done false
29466 9230 Acked Write 9 Sent Done Done false
29467 9231 Acked Read 3 Sent Done Done false
29468 9232 Acked Read 6 Sent Done Done false
29469 9233 Acked Read 4 Sent Done Done false
29470 9234 Acked Write 5 Sent Done Done false
29471 9235 Acked Read 5 Sent Done Done false
29472 9236 Acked Read 3 Sent Done Done false
29473 9237 Acked Write 3 Sent Done Done false
29474 9238 Acked Write 6 Sent Done Done false
29475 9239 Acked Read 3 Sent Done Done false
29476 9240 Acked Write 7 Sent Done Done false
29477 9241 Acked Read 4 Sent Done Done false
29478 9242 Acked Read 9 Sent Done Done false
29479 9243 Acked Read 9 Sent Done Done false
29480 9244 Acked Read 3 Sent Done Done false
29481 9245 Acked Read 5 Sent Done Done false
29482 9246 Acked Read 1 Sent Done Done false
29483 9247 Acked Write 6 Sent Done Done false
29484 9248 Acked Read 9 Sent Done Done false
29485 9249 Acked Read 8 Sent Done Done false
29486 9250 Acked Write 8 Sent Done Done false
29487 9251 Acked Read 5 Sent Done Done false
29488 9252 Acked Read 1 Sent Done Done false
29489 9253 Acked Read 5 Sent Done Done false
29490 9254 Acked Write 1 Sent Done Done false
29491 9255 Acked Write 9 Sent Done Done false
29492 9256 Acked Read 8 Sent Done Done false
29493 9257 Acked Read 6 Sent Done Done false
29494 9258 Acked Read 7 Sent Done Done false
29495 9259 Acked Read 1 Sent Done Done false
29496 9260 Acked Write 3 Sent Done Done false
29497 9261 Acked Read 10 Sent Done Done false
29498 9262 Acked Write 3 Sent Done Done false
29499 9263 Acked Write 2 Sent Done Done false
29500 9264 Acked Read 9 Sent Done Done false
29501 9265 Acked Read 3 Sent Done Done false
29502 9266 Acked Write 6 Sent Done Done false
29503 9267 Acked Write 3 Sent Done Done false
29504 9268 Acked Write 8 Sent Done Done false
29505 9269 Acked Read 4 Sent Done Done false
29506 9270 Acked Write 3 Sent Done Done false
29507 9271 Acked Write 5 Sent Done Done false
29508 9272 Acked Flush 0 Sent Done Done false
29509 9273 Acked Write 4 Sent Done Done false
29510 9274 Acked Read 8 Sent Done Done false
29511 9275 Acked Flush 0 Sent Done Done false
29512 9276 Acked Read 3 Sent Done Done false
29513 9277 Acked Write 2 Sent Done Done false
29514 9278 Acked Read 6 Sent Done Done false
29515 9279 Acked Read 3 Sent Done Done false
29516 9280 Acked Write 7 Sent Done Done false
29517 9281 Acked Write 8 Sent Done Done false
29518 9282 Acked Write 7 Sent Done Done false
29519 9283 Acked Write 7 Sent Done Done false
29520 9284 Acked Flush 0 Sent Done Done false
29521 9285 Acked Read 6 Sent Done Done false
29522 9286 Acked Read 3 Sent Done Done false
29523 9287 Acked Read 8 Sent Done Done false
29524 9288 Acked Flush 0 Sent Done Done false
29525 9289 Acked Write 3 Sent Done Done false
29526 9290 Acked Write 4 Sent Done Done false
29527 9291 Acked Flush 0 Sent Done Done false
29528 9292 Acked Flush 0 Sent Done Done false
29529 9293 Acked Read 6 Sent Done Done false
29530 9294 Acked Read 1 Sent Done Done false
29531 9295 Acked Write 5 Sent Done Done false
29532 9296 Acked Flush 0 Sent Done Done false
29533 9297 Acked Write 10 Sent Done Done false
29534 9298 Acked Read 8 Sent Done Done false
29535 9299 Acked Read 10 Sent Done Done false
29536 9300 Acked Write 4 Sent Done Done false
29537 9301 Acked Read 2 Sent Done Done false
29538 9302 Acked Write 6 Sent Done Done false
29539 9303 Acked Read 6 Sent Done Done false
29540 9304 Acked Read 5 Sent Done Done false
29541 9305 Acked Flush 0 Sent Done Done false
29542 9306 Acked Write 9 Sent Done Done false
29543 9307 Acked Read 6 Sent Done Done false
29544 9308 Acked Read 4 Sent Done Done false
29545 9309 Acked Read 5 Sent Done Done false
29546 9310 Acked Read 7 Sent Done Done false
29547 9311 Acked Read 6 Sent Done Done false
29548 9312 Acked Read 9 Sent Done Done false
29549 9313 Acked Flush 0 Sent Done Done false
29550 9314 Acked Write 9 Sent Done Done false
29551 9315 Acked Read 5 Sent Done Done false
29552 9316 Acked Read 9 Sent Done Done false
29553 9317 Acked Read 10 Sent Done Done false
29554 9318 Acked Write 9 Sent Done Done false
29555 9319 Acked Write 10 Sent Done Done false
29556 9320 Acked Read 2 Sent Done Done false
29557 9321 Acked Read 8 Sent Done Done false
29558 9322 Acked Write 4 Sent Done Done false
29559 9323 Acked Read 9 Sent Done Done false
29560 9324 Acked Write 5 Sent Done Done false
29561 9325 Acked Write 8 Sent Done Done false
29562 9326 Acked Read 4 Sent Done Done false
29563 9327 Acked Write 8 Sent Done Done false
29564 9328 Acked Write 6 Sent Done Done false
29565 9329 Acked Read 7 Sent Done Done false
29566 9330 Acked Write 2 Sent Done Done false
29567 9331 Acked Write 2 Sent Done Done false
29568 9332 Acked Read 3 Sent Done Done false
29569 9333 Acked Read 8 Sent Done Done false
29570 9334 Acked Read 5 Sent Done Done false
29571 9335 Acked Read 8 Sent Done Done false
29572 9336 Acked Write 4 Sent Done Done false
29573 9337 Acked Write 1 Sent Done Done false
29574 9338 Acked Write 10 Sent Done Done false
29575 9339 Acked Read 6 Sent Done Done false
29576 9340 Acked Read 8 Sent Done Done false
29577 9341 Acked Read 4 Sent Done Done false
29578 9342 Acked Flush 0 Sent Done Done false
29579 9343 Acked Read 2 Sent Done Done false
29580 9344 Acked Read 6 Sent Done Done false
29581 9345 Acked Write 4 Sent Done Done false
29582 9346 Acked Write 8 Sent Done Done false
29583 9347 Acked Write 6 Sent Done Done false
29584 9348 Acked Write 10 Sent Done Done false
29585 9349 Acked Write 5 Sent Done Done false
29586 9350 Acked Read 10 Sent Done Done false
29587 9351 Acked Read 2 Sent Done Done false
29588 9352 Acked Read 2 Sent Done Done false
29589 9353 Acked Write 1 Sent Done Done false
29590 9354 Acked Read 4 Sent Done Done false
29591 9355 Acked Read 4 Sent Done Done false
29592 9356 Acked Write 7 Sent Done Done false
29593 9357 Acked Write 3 Sent Done Done false
29594 9358 Acked Read 8 Sent Done Done false
29595 9359 Acked Read 10 Sent Done Done false
29596 9360 Acked Read 2 Sent Done Done false
29597 9361 Acked Write 9 Sent Done Done false
29598 9362 Acked Write 5 Sent Done Done false
29599 9363 Acked Write 4 Sent Done Done false
29600 9364 Acked Write 7 Sent Done Done false
29601 9365 Acked Write 5 Sent Done Done false
29602 9366 Acked Write 6 Sent Done Done false
29603 9367 Acked Read 2 Sent Done Done false
29604 9368 Acked Write 10 Sent Done Done false
29605 9369 Acked Write 1 Sent Done Done false
29606 9370 Acked Read 2 Sent Done Done false
29607 9371 Acked Read 8 Sent Done Done false
29608 9372 Acked Read 9 Sent Done Done false
29609 9373 Acked Read 8 Sent Done Done false
29610 9374 Acked Read 8 Sent Done Done false
29611 9375 Acked Write 1 Sent Done Done false
29612 9376 Acked Flush 0 Sent Done Done false
29613 9377 Acked Read 4 Sent Done Done false
29614 9378 Acked Read 6 Sent Done Done false
29615 9379 Acked Write 4 Sent Done Done false
29616 9380 Acked Read 6 Sent Done Done false
29617 9381 Acked Write 10 Sent Done Done false
29618 9382 Acked Write 5 Sent Done Done false
29619 9383 Acked Read 9 Sent Done Done false
29620 9384 Acked Read 1 Sent Done Done false
29621 9385 Acked Write 4 Sent Done Done false
29622 9386 Acked Flush 0 Sent Done Done false
29623 9387 Acked Write 5 Sent Done Done false
29624 9388 Acked Read 4 Sent Done Done false
29625 9389 Acked Read 8 Sent Done Done false
29626 9390 Acked Write 5 Sent Done Done false
29627 9391 Acked Read 1 Sent Done Done false
29628 9392 Acked Read 2 Sent Done Done false
29629 9393 Acked Write 4 Sent Done Done false
29630 9394 Acked Read 1 Sent Done Done false
29631 9395 Acked Write 6 Sent Done Done false
29632 9396 Acked Read 8 Sent Done Done false
29633 9397 Acked Write 10 Sent Done Done false
29634 9398 Acked Write 6 Sent Done Done false
29635 9399 Acked Flush 0 Sent Done Done false
29636 9400 Acked Read 4 Sent Done Done false
29637 9401 Acked Read 8 Sent Done Done false
29638 9402 Acked Write 10 Sent Done Done false
29639 9403 Acked Read 4 Sent Done Done false
29640 9404 Acked Write 4 Sent Done Done false
29641 9405 Acked Read 3 Sent Done Done false
29642 9406 Acked Write 4 Sent Done Done false
29643 9407 Acked Read 4 Sent Done Done false
29644 9408 Acked Read 4 Sent Done Done false
29645 9409 Acked Write 5 Sent Done Done false
29646 9410 Acked Read 1 Sent Done Done false
29647 9411 Acked Write 1 Sent Done Done false
29648 9412 Acked Read 7 Sent Done Done false
29649 9413 Acked Read 10 Sent Done Done false
29650 9414 Acked Read 2 Sent Done Done false
29651 9415 Acked Write 3 Sent Done Done false
29652 9416 Acked Write 7 Sent Done Done false
29653 9417 Acked Read 9 Sent Done Done false
29654 9418 Acked Write 10 Sent Done Done false
29655 9419 Acked Read 3 Sent Done Done false
29656 9420 Acked Write 5 Sent Done Done false
29657 9421 Acked Read 5 Sent Done Done false
29658 9422 Acked Write 4 Sent Done Done false
29659 9423 Acked Write 9 Sent Done Done false
29660 9424 Acked Read 4 Sent Done Done false
29661 9425 Acked Read 4 Sent Done Done false
29662 9426 Acked Write 2 Sent Done Done false
29663 9427 Acked Read 2 Sent Done Done false
29664 9428 Acked Read 4 Sent Done Done false
29665 9429 Acked Read 3 Sent Done Done false
29666 9430 Acked Flush 0 Sent Done Done false
29667 9431 Acked Write 8 Sent Done Done false
29668 9432 Acked Write 8 Sent Done Done false
29669 9433 Acked Read 4 Sent Done Done false
29670 9434 Acked Write 5 Sent Done Done false
29671 9435 Acked Read 4 Sent Done Done false
29672 9436 Acked Read 10 Sent Done Done false
29673 9437 Acked Write 6 Sent Done Done false
29674 9438 Acked Read 4 Sent Done Done false
29675 9439 Acked Write 8 Sent Done Done false
29676 9440 Acked Read 8 Sent Done Done false
29677 9441 Acked Write 3 Sent Done Done false
29678 9442 Acked Write 4 Sent Done Done false
29679 9443 Acked Flush 0 Sent Done Done false
29680 9444 Acked Read 4 Sent Done Done false
29681 9445 Acked Write 5 Sent Done Done false
29682 9446 Acked Read 6 Sent Done Done false
29683 9447 Acked Read 8 Sent Done Done false
29684 9448 Acked Write 2 Sent Done Done false
29685 9449 Acked Read 3 Sent Done Done false
29686 9450 Acked Read 3 Sent Done Done false
29687 STATES DS:0 DS:1 DS:2 TOTAL
29688 Sent 301 0 0 301
29689 Done 0 301 301 602
29690 Skipped 0 0 0 0
29691 Error 0 0 0 0
29692 Last Flush: 9149 9443 9443
29693 Downstairs last five completed: 9149 9148 9147 9146 9145
29694 Upstairs last five completed: 9450 9449 9448 9447 9446
29695 CLIENT: Up:0 ds:301 act:2
29696 [8810] got start action
29697 [8810] I am going to restart
29698 Starting downstairs at port 8810
29699 Make output file at "/tmp/dsc/downstairs-8810.txt"
29700 [8810][0] reports Starting
29701 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1231)
29702 [8810][0] reports Running
29703 ----------------------------------------------------------------
29704 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29705 States: Offline Active Active
29706 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29707 9150 Acked Read 8 Sent Done Done false
29708 9151 Acked Read 1 Sent Done Done false
29709 9152 Acked Read 10 Sent Done Done false
29710 9153 Acked Write 6 Sent Done Done false
29711 9154 Acked Read 8 Sent Done Done false
29712 9155 Acked Read 9 Sent Done Done false
29713 9156 Acked Write 6 Sent Done Done false
29714 9157 Acked Read 6 Sent Done Done false
29715 9158 Acked Write 7 Sent Done Done false
29716 9159 Acked Read 9 Sent Done Done false
29717 9160 Acked Read 3 Sent Done Done false
29718 9161 Acked Write 4 Sent Done Done false
29719 9162 Acked Read 2 Sent Done Done false
29720 9163 Acked Read 8 Sent Done Done false
29721 9164 Acked Read 3 Sent Done Done false
29722 9165 Acked Read 2 Sent Done Done false
29723 9166 Acked Write 5 Sent Done Done false
29724 9167 Acked Flush 0 Sent Done Done false
29725 9168 Acked Write 9 Sent Done Done false
29726 9169 Acked Write 5 Sent Done Done false
29727 9170 Acked Write 6 Sent Done Done false
29728 9171 Acked Flush 0 Sent Done Done false
29729 9172 Acked Read 10 Sent Done Done false
29730 9173 Acked Read 6 Sent Done Done false
29731 9174 Acked Write 6 Sent Done Done false
29732 9175 Acked Read 1 Sent Done Done false
29733 9176 Acked Write 2 Sent Done Done false
29734 9177 Acked Write 7 Sent Done Done false
29735 9178 Acked Read 1 Sent Done Done false
29736 9179 Acked Write 5 Sent Done Done false
29737 9180 Acked Write 2 Sent Done Done false
29738 9181 Acked Read 3 Sent Done Done false
29739 9182 Acked Write 9 Sent Done Done false
29740 9183 Acked Write 7 Sent Done Done false
29741 9184 Acked Write 10 Sent Done Done false
29742 9185 Acked Read 10 Sent Done Done false
29743 9186 Acked Flush 0 Sent Done Done false
29744 9187 Acked Read 1 Sent Done Done false
29745 9188 Acked Write 9 Sent Done Done false
29746 9189 Acked Read 7 Sent Done Done false
29747 9190 Acked Read 1 Sent Done Done false
29748 9191 Acked Read 9 Sent Done Done false
29749 9192 Acked Read 3 Sent Done Done false
29750 9193 Acked Write 1 Sent Done Done false
29751 9194 Acked Read 10 Sent Done Done false
29752 9195 Acked Write 2 Sent Done Done false
29753 9196 Acked Write 10 Sent Done Done false
29754 9197 Acked Read 10 Sent Done Done false
29755 9198 Acked Flush 0 Sent Done Done false
29756 9199 Acked Write 5 Sent Done Done false
29757 9200 Acked Write 5 Sent Done Done false
29758 9201 Acked Read 5 Sent Done Done false
29759 9202 Acked Write 8 Sent Done Done false
29760 9203 Acked Write 7 Sent Done Done false
29761 9204 Acked Write 3 Sent Done Done false
29762 9205 Acked Flush 0 Sent Done Done false
29763 9206 Acked Read 4 Sent Done Done false
29764 9207 Acked Read 5 Sent Done Done false
29765 9208 Acked Read 5 Sent Done Done false
29766 9209 Acked Read 4 Sent Done Done false
29767 9210 Acked Flush 0 Sent Done Done false
29768 9211 Acked Flush 0 Sent Done Done false
29769 9212 Acked Flush 0 Sent Done Done false
29770 9213 Acked Write 4 Sent Done Done false
29771 9214 Acked Read 7 Sent Done Done false
29772 9215 Acked Read 1 Sent Done Done false
29773 9216 Acked Write 1 Sent Done Done false
29774 9217 Acked Read 5 Sent Done Done false
29775 9218 Acked Read 2 Sent Done Done false
29776 9219 Acked Write 7 Sent Done Done false
29777 9220 Acked Read 7 Sent Done Done false
29778 9221 Acked Write 4 Sent Done Done false
29779 9222 Acked Read 9 Sent Done Done false
29780 9223 Acked Write 5 Sent Done Done false
29781 9224 Acked Write 2 Sent Done Done false
29782 9225 Acked Read 6 Sent Done Done false
29783 9226 Acked Write 9 Sent Done Done false
29784 9227 Acked Read 6 Sent Done Done false
29785 9228 Acked Read 7 Sent Done Done false
29786 9229 Acked Read 1 Sent Done Done false
29787 9230 Acked Write 9 Sent Done Done false
29788 9231 Acked Read 3 Sent Done Done false
29789 9232 Acked Read 6 Sent Done Done false
29790 9233 Acked Read 4 Sent Done Done false
29791 9234 Acked Write 5 Sent Done Done false
29792 9235 Acked Read 5 Sent Done Done false
29793 9236 Acked Read 3 Sent Done Done false
29794 9237 Acked Write 3 Sent Done Done false
29795 9238 Acked Write 6 Sent Done Done false
29796 9239 Acked Read 3 Sent Done Done false
29797 9240 Acked Write 7 Sent Done Done false
29798 9241 Acked Read 4 Sent Done Done false
29799 9242 Acked Read 9 Sent Done Done false
29800 9243 Acked Read 9 Sent Done Done false
29801 9244 Acked Read 3 Sent Done Done false
29802 9245 Acked Read 5 Sent Done Done false
29803 9246 Acked Read 1 Sent Done Done false
29804 9247 Acked Write 6 Sent Done Done false
29805 9248 Acked Read 9 Sent Done Done false
29806 9249 Acked Read 8 Sent Done Done false
29807 9250 Acked Write 8 Sent Done Done false
29808 9251 Acked Read 5 Sent Done Done false
29809 9252 Acked Read 1 Sent Done Done false
29810 9253 Acked Read 5 Sent Done Done false
29811 9254 Acked Write 1 Sent Done Done false
29812 9255 Acked Write 9 Sent Done Done false
29813 9256 Acked Read 8 Sent Done Done false
29814 9257 Acked Read 6 Sent Done Done false
29815 9258 Acked Read 7 Sent Done Done false
29816 9259 Acked Read 1 Sent Done Done false
29817 9260 Acked Write 3 Sent Done Done false
29818 9261 Acked Read 10 Sent Done Done false
29819 9262 Acked Write 3 Sent Done Done false
29820 9263 Acked Write 2 Sent Done Done false
29821 9264 Acked Read 9 Sent Done Done false
29822 9265 Acked Read 3 Sent Done Done false
29823 9266 Acked Write 6 Sent Done Done false
29824 9267 Acked Write 3 Sent Done Done false
29825 9268 Acked Write 8 Sent Done Done false
29826 9269 Acked Read 4 Sent Done Done false
29827 9270 Acked Write 3 Sent Done Done false
29828 9271 Acked Write 5 Sent Done Done false
29829 9272 Acked Flush 0 Sent Done Done false
29830 9273 Acked Write 4 Sent Done Done false
29831 9274 Acked Read 8 Sent Done Done false
29832 9275 Acked Flush 0 Sent Done Done false
29833 9276 Acked Read 3 Sent Done Done false
29834 9277 Acked Write 2 Sent Done Done false
29835 9278 Acked Read 6 Sent Done Done false
29836 9279 Acked Read 3 Sent Done Done false
29837 9280 Acked Write 7 Sent Done Done false
29838 9281 Acked Write 8 Sent Done Done false
29839 9282 Acked Write 7 Sent Done Done false
29840 9283 Acked Write 7 Sent Done Done false
29841 9284 Acked Flush 0 Sent Done Done false
29842 9285 Acked Read 6 Sent Done Done false
29843 9286 Acked Read 3 Sent Done Done false
29844 9287 Acked Read 8 Sent Done Done false
29845 9288 Acked Flush 0 Sent Done Done false
29846 9289 Acked Write 3 Sent Done Done false
29847 9290 Acked Write 4 Sent Done Done false
29848 9291 Acked Flush 0 Sent Done Done false
29849 9292 Acked Flush 0 Sent Done Done false
29850 9293 Acked Read 6 Sent Done Done false
29851 9294 Acked Read 1 Sent Done Done false
29852 9295 Acked Write 5 Sent Done Done false
29853 9296 Acked Flush 0 Sent Done Done false
29854 9297 Acked Write 10 Sent Done Done false
29855 9298 Acked Read 8 Sent Done Done false
29856 9299 Acked Read 10 Sent Done Done false
29857 9300 Acked Write 4 Sent Done Done false
29858 9301 Acked Read 2 Sent Done Done false
29859 9302 Acked Write 6 Sent Done Done false
29860 9303 Acked Read 6 Sent Done Done false
29861 9304 Acked Read 5 Sent Done Done false
29862 9305 Acked Flush 0 Sent Done Done false
29863 9306 Acked Write 9 Sent Done Done false
29864 9307 Acked Read 6 Sent Done Done false
29865 9308 Acked Read 4 Sent Done Done false
29866 9309 Acked Read 5 Sent Done Done false
29867 9310 Acked Read 7 Sent Done Done false
29868 9311 Acked Read 6 Sent Done Done false
29869 9312 Acked Read 9 Sent Done Done false
29870 9313 Acked Flush 0 Sent Done Done false
29871 9314 Acked Write 9 Sent Done Done false
29872 9315 Acked Read 5 Sent Done Done false
29873 9316 Acked Read 9 Sent Done Done false
29874 9317 Acked Read 10 Sent Done Done false
29875 9318 Acked Write 9 Sent Done Done false
29876 9319 Acked Write 10 Sent Done Done false
29877 9320 Acked Read 2 Sent Done Done false
29878 9321 Acked Read 8 Sent Done Done false
29879 9322 Acked Write 4 Sent Done Done false
29880 9323 Acked Read 9 Sent Done Done false
29881 9324 Acked Write 5 Sent Done Done false
29882 9325 Acked Write 8 Sent Done Done false
29883 9326 Acked Read 4 Sent Done Done false
29884 9327 Acked Write 8 Sent Done Done false
29885 9328 Acked Write 6 Sent Done Done false
29886 9329 Acked Read 7 Sent Done Done false
29887 9330 Acked Write 2 Sent Done Done false
29888 9331 Acked Write 2 Sent Done Done false
29889 9332 Acked Read 3 Sent Done Done false
29890 9333 Acked Read 8 Sent Done Done false
29891 9334 Acked Read 5 Sent Done Done false
29892 9335 Acked Read 8 Sent Done Done false
29893 9336 Acked Write 4 Sent Done Done false
29894 9337 Acked Write 1 Sent Done Done false
29895 9338 Acked Write 10 Sent Done Done false
29896 9339 Acked Read 6 Sent Done Done false
29897 9340 Acked Read 8 Sent Done Done false
29898 9341 Acked Read 4 Sent Done Done false
29899 9342 Acked Flush 0 Sent Done Done false
29900 9343 Acked Read 2 Sent Done Done false
29901 9344 Acked Read 6 Sent Done Done false
29902 9345 Acked Write 4 Sent Done Done false
29903 9346 Acked Write 8 Sent Done Done false
29904 9347 Acked Write 6 Sent Done Done false
29905 9348 Acked Write 10 Sent Done Done false
29906 9349 Acked Write 5 Sent Done Done false
29907 9350 Acked Read 10 Sent Done Done false
29908 9351 Acked Read 2 Sent Done Done false
29909 9352 Acked Read 2 Sent Done Done false
29910 9353 Acked Write 1 Sent Done Done false
29911 9354 Acked Read 4 Sent Done Done false
29912 9355 Acked Read 4 Sent Done Done false
29913 9356 Acked Write 7 Sent Done Done false
29914 9357 Acked Write 3 Sent Done Done false
29915 9358 Acked Read 8 Sent Done Done false
29916 9359 Acked Read 10 Sent Done Done false
29917 9360 Acked Read 2 Sent Done Done false
29918 9361 Acked Write 9 Sent Done Done false
29919 9362 Acked Write 5 Sent Done Done false
29920 9363 Acked Write 4 Sent Done Done false
29921 9364 Acked Write 7 Sent Done Done false
29922 9365 Acked Write 5 Sent Done Done false
29923 9366 Acked Write 6 Sent Done Done false
29924 9367 Acked Read 2 Sent Done Done false
29925 9368 Acked Write 10 Sent Done Done false
29926 9369 Acked Write 1 Sent Done Done false
29927 9370 Acked Read 2 Sent Done Done false
29928 9371 Acked Read 8 Sent Done Done false
29929 9372 Acked Read 9 Sent Done Done false
29930 9373 Acked Read 8 Sent Done Done false
29931 9374 Acked Read 8 Sent Done Done false
29932 9375 Acked Write 1 Sent Done Done false
29933 9376 Acked Flush 0 Sent Done Done false
29934 9377 Acked Read 4 Sent Done Done false
29935 9378 Acked Read 6 Sent Done Done false
29936 9379 Acked Write 4 Sent Done Done false
29937 9380 Acked Read 6 Sent Done Done false
29938 9381 Acked Write 10 Sent Done Done false
29939 9382 Acked Write 5 Sent Done Done false
29940 9383 Acked Read 9 Sent Done Done false
29941 9384 Acked Read 1 Sent Done Done false
29942 9385 Acked Write 4 Sent Done Done false
29943 9386 Acked Flush 0 Sent Done Done false
29944 9387 Acked Write 5 Sent Done Done false
29945 9388 Acked Read 4 Sent Done Done false
29946 9389 Acked Read 8 Sent Done Done false
29947 9390 Acked Write 5 Sent Done Done false
29948 9391 Acked Read 1 Sent Done Done false
29949 9392 Acked Read 2 Sent Done Done false
29950 9393 Acked Write 4 Sent Done Done false
29951 9394 Acked Read 1 Sent Done Done false
29952 9395 Acked Write 6 Sent Done Done false
29953 9396 Acked Read 8 Sent Done Done false
29954 9397 Acked Write 10 Sent Done Done false
29955 9398 Acked Write 6 Sent Done Done false
29956 9399 Acked Flush 0 Sent Done Done false
29957 9400 Acked Read 4 Sent Done Done false
29958 9401 Acked Read 8 Sent Done Done false
29959 9402 Acked Write 10 Sent Done Done false
29960 9403 Acked Read 4 Sent Done Done false
29961 9404 Acked Write 4 Sent Done Done false
29962 9405 Acked Read 3 Sent Done Done false
29963 9406 Acked Write 4 Sent Done Done false
29964 9407 Acked Read 4 Sent Done Done false
29965 9408 Acked Read 4 Sent Done Done false
29966 9409 Acked Write 5 Sent Done Done false
29967 9410 Acked Read 1 Sent Done Done false
29968 9411 Acked Write 1 Sent Done Done false
29969 9412 Acked Read 7 Sent Done Done false
29970 9413 Acked Read 10 Sent Done Done false
29971 9414 Acked Read 2 Sent Done Done false
29972 9415 Acked Write 3 Sent Done Done false
29973 9416 Acked Write 7 Sent Done Done false
29974 9417 Acked Read 9 Sent Done Done false
29975 9418 Acked Write 10 Sent Done Done false
29976 9419 Acked Read 3 Sent Done Done false
29977 9420 Acked Write 5 Sent Done Done false
29978 9421 Acked Read 5 Sent Done Done false
29979 9422 Acked Write 4 Sent Done Done false
29980 9423 Acked Write 9 Sent Done Done false
29981 9424 Acked Read 4 Sent Done Done false
29982 9425 Acked Read 4 Sent Done Done false
29983 9426 Acked Write 2 Sent Done Done false
29984 9427 Acked Read 2 Sent Done Done false
29985 9428 Acked Read 4 Sent Done Done false
29986 9429 Acked Read 3 Sent Done Done false
29987 9430 Acked Flush 0 Sent Done Done false
29988 9431 Acked Write 8 Sent Done Done false
29989 9432 Acked Write 8 Sent Done Done false
29990 9433 Acked Read 4 Sent Done Done false
29991 9434 Acked Write 5 Sent Done Done false
29992 9435 Acked Read 4 Sent Done Done false
29993 9436 Acked Read 10 Sent Done Done false
29994 9437 Acked Write 6 Sent Done Done false
29995 9438 Acked Read 4 Sent Done Done false
29996 9439 Acked Write 8 Sent Done Done false
29997 9440 Acked Read 8 Sent Done Done false
29998 9441 Acked Write 3 Sent Done Done false
29999 9442 Acked Write 4 Sent Done Done false
30000 9443 Acked Flush 0 Sent Done Done false
30001 9444 Acked Read 4 Sent Done Done false
30002 9445 Acked Write 5 Sent Done Done false
30003 9446 Acked Read 6 Sent Done Done false
30004 9447 Acked Read 8 Sent Done Done false
30005 9448 Acked Write 2 Sent Done Done false
30006 9449 Acked Read 3 Sent Done Done false
30007 9450 Acked Read 3 Sent Done Done false
30008 9451 Acked Flush 0 Sent Done Done false
30009 STATES DS:0 DS:1 DS:2 TOTAL
30010 Sent 302 0 0 302
30011 Done 0 302 302 604
30012 Skipped 0 0 0 0
30013 Error 0 0 0 0
30014 Last Flush: 9149 9451 9451
30015 Downstairs last five completed: 9149 9148 9147 9146 9145
30016 Upstairs last five completed: 9451 9450 9449 9448 9447
30017 CLIENT: Up:0 ds:302 act:2
300182024-12-04T13:36:58.763ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300192024-12-04T13:36:58.763ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50275) = io task client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300202024-12-04T13:36:58.763ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300212024-12-04T13:36:58.764ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300222024-12-04T13:36:58.764ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300232024-12-04T13:36:58.764ZINFOcrucible: send last flush ID to this DS: 9149 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300242024-12-04T13:36:58.764ZINFOcrucible: Replied this last flush ID: 9149 = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300252024-12-04T13:36:58.764ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300262024-12-04T13:36:58.764ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300272024-12-04T13:36:58.764ZINFOcrucible: [0] client re-new 302 jobs since flush 9149 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300282024-12-04T13:36:58.765ZINFOcrucible: [0] Replayed 302 jobs since flush: 9149 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
30029 ----------------------------------------------------------------
30030 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30031 Downstairs last five completed: 9451 9450 9449 9448 9447
30032 Upstairs last five completed: 9451 9450 9449 9448 9447
30033 CLIENT: Up:0 ds:0 act:3
30034 Replay: All jobs finished
30035 Main task has work to do, go find it
30036 got dsc Stop(1)
30037 stop 1
30038 Dec 04 13:37:01.006 INFO request completed, latency_us: [8820] Got stop action so:false kr:false
30039 337, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 9da89704-13f9-460e-ac9c-031a0b4b5a94, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
300402024-12-04T13:37:01.007ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300412024-12-04T13:37:01.007ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
30042 Dec 04 13:37:01.007 INFO request completed, latency_us: {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:37:01.007400109Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
30043 116, response_code: 200, uri: /state/cid/1, method: GET, req_id: 239dabef-22db-4c10-92a7-be32350deb49, remote_addr: 127.0{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:37:01.007426529Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
30044 .0.1:37804, local_addr: 127.0.0.1:9998
300452024-12-04T13:37:01.007ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
300462024-12-04T13:37:01.007ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
30047 [8820] Exited with: ExitStatus(unix_wait_status(9))
30048 [8820][1] reports Exit
30049 Dec 04 13:37:05.009 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/1, method: GET, req_id: a21e4a95-fb57-4247-ade8-7a53df180a5b, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
30050 001/300 Write block 4878 len 12288 data: 4 3 3
30051 002/300 Flush
30052 003/300 Write block 905 len 12288 data: 4 3 3
30053 004/300 Write block 1593 len 24576 data: 5 6 7 8 6 7
30054 005/300 Read block 818 len 12288
30055 006/300 Write block 3106 len 16384 data: 7 7 8 9
30056 007/300 Write block 3972 len 28672 data: 6 6 8 9 6 6 3
30057 008/300 Read block 865 len 16384
30058 009/300 Write block 1412 len 8192 data: 6 6
30059 010/300 Flush
30060 011/300 Read block 3662 len 32768
30061 012/300 Write block 4460 len 24576 data: 6 8 9 9 9 10
30062 013/300 Read block 4748 len 8192
30063 014/300 Read block 705 len 32768
30064 015/300 Read block 1419 len 40960
30065 016/300 Read block 2175 len 16384
30066 017/300 Write block 1261 len 16384 data: 3 4 4 5
30067 018/300 Flush
30068 019/300 Read block 4437 len 24576
30069 020/300 Read block 1558 len 4096
30070 021/300 Write block 669 len 20480 data: 9 9 6 6 6
30071 022/300 Read block 2365 len 24576
30072 023/300 Write block 127 len 4096 data: 7
30073 024/300 Read block 4597 len 16384
30074 025/300 Read block 1482 len 16384
30075 026/300 Read block 4023 len 16384
30076 027/300 Read block 1128 len 8192
30077 028/300 Write block 3841 len 24576 data: 4 3 3 4 6 6
30078 029/300 Read block 2769 len 16384
30079 030/300 Write block 1536 len 36864 data: 4 5 5 5 6 6 8 7 7
30080 031/300 Write block 2173 len 8192 data: 8 8
30081 032/300 Read block 3312 len 24576
30082 033/300 Read block 1706 len 40960
30083 034/300 Write block 4407 len 28672 data: 6 5 5 4 3 4 5
30084 035/300 Read block 539 len 8192
30085 036/300 Write block 775 len 28672 data: 6 5 5 3 3 3 2
30086 037/300 Read block 2696 len 36864
30087 038/300 Read block 1648 len 32768
30088 039/300 Read block 1506 len 8192
30089 040/300 Flush
30090 041/300 Read block 1616 len 16384
30091 042/300 Read block 2222 len 28672
30092 043/300 Flush
30093 044/300 Flush
30094 045/300 Write block 3233 len 4096 data: 9
30095 046/300 Write block 3110 len 20480 data: 7 6 5 5 4
30096 047/300 Read block 727 len 40960
30097 048/300 Write block 3484 len 40960 data: 8 9 8 7 10 9 7 5 5 6
30098 049/300 Write block 613 len 12288 data: 2 2 4
30099 050/300 Write block 4714 len 36864 data: 3 4 4 6 7 6 7 7 8
30100 051/300 Read block 2082 len 28672
30101 052/300 Flush
30102 053/300 Write block 2790 len 24576 data: 9 10 9 6 5 5
30103 054/300 Read block 825 len 16384
30104 055/300 Write block 2029 len 20480 data: 6 6 7 5 5
30105 056/300 Write block 4281 len 24576 data: 4 4 4 5 5 5
30106 057/300 Read block 4271 len 28672
30107 058/300 Read block 366 len 24576
30108 059/300 Flush
30109 060/300 Read block 402 len 24576
30110 061/300 Write block 3732 len 36864 data: 5 4 4 5 4 4 4 6 6
30111 062/300 Write block 577 len 16384 data: 4 4 5 5
30112 063/300 Read block 1458 len 40960
30113 064/300 Write block 4129 len 32768 data: 9 9 10 10 6 6 6 6
30114 065/300 Read block 3276 len 24576
30115 066/300 Flush
30116 067/300 Write block 3680 len 16384 data: 8 8 8 7
30117 068/300 Write block 29 len 28672 data: 8 7 8 6 6 7 6
30118 069/300 Read block 242 len 16384
30119 070/300 Read block 851 len 20480
30120 071/300 Write block 4800 len 36864 data: 7 9 9 10 9 9 8 7 4
30121 072/300 Write block 4905 len 28672 data: 7 6 6 7 8 6 4
30122 073/300 Read block 3484 len 32768
30123 074/300 Write block 657 len 40960 data: 4 3 3 5 5 7 6 6 6 5
30124 075/300 Write block 4600 len 32768 data: 6 7 7 7 8 9 8 8
30125 076/300 Flush
30126 077/300 Read block 2546 len 32768
30127 078/300 Write block 1630 len 8192 data: 3 4
30128 079/300 Flush
30129 080/300 Flush
30130 081/300 Read block 3855 len 24576
30131 082/300 Read block 2047 len 28672
30132 083/300 Write block 4380 len 40960 data: 5 5 5 5 5 6 5 7 7 5
30133 084/300 Read block 2446 len 4096
30134 085/300 Write block 3808 len 28672 data: 4 6 5 6 6 5 7
30135 086/300 Read block 1839 len 12288
30136 087/300 Read block 2458 len 32768
30137 088/300 Read block 3106 len 8192
30138 089/300 Read block 820 len 12288
30139 090/300 Read block 4709 len 32768
30140 091/300 Read block 3431 len 40960
30141 092/300 Read block 4168 len 4096
30142 093/300 Write block 4138 len 28672 data: 5 2 2 2 2 3 3
30143 094/300 Flush
30144 095/300 Flush
30145 096/300 Flush
30146 097/300 Read block 1283 len 28672
30147 098/300 Read block 407 len 4096
30148 099/300 Write block 1113 len 36864 data: 8 10 9 9 9 8 6 6 6
30149 100/300 Read block 99 len 16384
30150 101/300 Read block 4074 len 24576
30151 102/300 Write block 3995 len 24576 data: 5 4 4 4 3 3
30152 103/300 Write block 2893 len 32768 data: 6 7 8 7 7 7 7 7
30153 104/300 Write block 301 len 32768 data: 5 5 6 6 7 8 8 7
30154 105/300 Write block 547 len 24576 data: 6 5 7 9 6 5
30155 106/300 Read block 4832 len 8192
30156 107/300 Flush
30157 108/300 Write block 1972 len 16384 data: 7 5 6 4
30158 109/300 Read block 3866 len 16384
30159 110/300 Write block 2471 len 12288 data: 7 7 7
30160 111/300 Write block 1868 len 12288 data: 4 4 4
30161 112/300 Write block 1123 len 36864 data: 5 5 4 4 3 4 3 4 4
30162 113/300 Write block 1372 len 4096 data: 7
30163 114/300 Write block 791 len 16384 data: 7 5 4 4
30164 115/300 Read block 1666 len 12288
30165 116/300 Flush
30166 117/300 Read block 439 len 16384
30167 118/300 Write block 2234 len 12288 data: 7 7 8
30168 119/300 Flush
30169 120/300 Write block 4554 len 8192 data: 3 3
30170 121/300 Read block 557 len 8192
30171 122/300 Read block 4394 len 20480
30172 123/300 Write block 1560 len 32768 data: 7 8 7 7 7 9 9 9
30173 124/300 Write block 4007 len 8192 data: 5 6
30174 125/300 Read block 4967 len 40960
30175 126/300 Read block 4750 len 12288
30176 127/300 Read block 1856 len 12288
30177 128/300 Write block 3498 len 24576 data: 6 7 7 9 9 9
30178 129/300 Read block 1360 len 4096
30179 130/300 Read block 2483 len 12288
30180 131/300 Write block 1064 len 8192 data: 6 8
30181 132/300 Write block 1650 len 40960 data: 5 7 6 6 5 6 6 6 7 6
30182 133/300 Write block 2759 len 12288 data: 5 7 6
30183 134/300 Write block 3228 len 16384 data: 5 8 9 7
30184 135/300 Write block 4599 len 28672 data: 6 7 8 8 8 9 10
30185 136/300 Read block 1658 len 20480
30186 137/300 Read block 4293 len 40960
30187 138/300 Read block 775 len 8192
30188 139/300 Read block 3677 len 12288
30189 140/300 Read block 4163 len 4096
30190 141/300 Write block 1202 len 20480 data: 7 7 7 7 7
30191 142/300 Read block 1595 len 36864
30192 143/300 Read block 3877 len 8192
30193 144/300 Read block 2961 len 28672
30194 145/300 Write block 980 len 16384 data: 9 8 8 9
30195 146/300 Read block 2059 len 20480
30196 147/300 Write block 2061 len 32768 data: 4 3 3 3 2 3 3 4
30197 148/300 Write block 534 len 16384 data: 7 6 5 6
30198 149/300 Read block 1461 len 4096
30199 150/300 Flush
30200 151/300 Read block 2088 len 8192
30201 152/300 Write block 2180 len 36864 data: 7 6 6 7 7 7 8 8 7
30202 153/300 Write block 3368 len 24576 data: 7 8 7 5 4 5
30203 154/300 Read block 654 len 36864
30204 155/300 Write block 3365 len 16384 data: 7 7 7 8
30205 156/300 Read block 1109 len 12288
30206 157/300 Write block 4670 len 8192 data: 4 5
30207 158/300 Flush
30208 159/300 Write block 1472 len 4096 data: 9
30209 160/300 Read block 4117 len 36864
30210 161/300 Read block 918 len 4096
30211 162/300 Write block 4210 len 12288 data: 5 5 4
30212 163/300 Read block 3323 len 32768
30213 164/300 Read block 2991 len 36864
30214 165/300 Write block 1037 len 36864 data: 5 5 4 5 5 5 4 4 5
30215 166/300 Read block 123 len 24576
30216 167/300 Read block 76 len 4096
30217 168/300 Read block 4397 len 12288
30218 169/300 Read block 1348 len 36864
30219 170/300 Write block 2139 len 36864 data: 9 8 7 7 7 7 5 7 5
30220 171/300 Read block 3411 len 28672
30221 172/300 Read block 1214 len 32768
30222 173/300 Read block 3748 len 32768
30223 174/300 Write block 1168 len 8192 data: 4 6
30224 175/300 Write block 4376 len 16384 data: 8 7 6 7
30225 176/300 Flush
30226 177/300 Write block 2993 len 36864 data: 6 6 4 5 4 4 4 5 8
30227 178/300 Write block 1003 len 12288 data: 4 5 7
30228 179/300 Read block 4158 len 8192
30229 180/300 Write block 2499 len 16384 data: 3 3 4 3
30230 181/300 Write block 522 len 4096 data: 7
30231 182/300 Flush
30232 183/300 Write block 3806 len 32768 data: 4 3 5 7 6 7 7 6
30233 184/300 Read block 4527 len 4096
30234 185/300 Write block 305 len 8192 data: 8 9
30235 186/300 Read block 1560 len 28672
30236 187/300 Read block 827 len 16384
30237 188/300 Write block 2873 len 4096 data: 10
30238 189/300 Read block 1438 len 16384
30239 190/300 Write block 2387 len 36864 data: 5 5 7 7 7 7 6 6 7
30240 191/300 Flush
30241 192/300 Write block 1901 len 40960 data: 4 7 6 5 6 5 4 5 5 6
30242 193/300 Read block 191 len 36864
30243 194/300 Read block 4770 len 4096
30244 195/300 Write block 4900 len 40960 data: 6 6 6 6 7 8 7 7 8 9
30245 196/300 Write block 435 len 20480 data: 5 4 4 6 5
30246 197/300 Write block 4489 len 16384 data: 7 7 7 8
30247 198/300 Write block 4657 len 24576 data: 5 5 7 8 8 8
30248 199/300 Write block 4376 len 8192 data: 9 8
30249 200/300 Read block 2283 len 36864
30250 201/300 Write block 2709 len 32768 data: 8 9 8 8 5 3 3 3
30251 202/300 Read block 2911 len 12288
30252 203/300 Write block 2274 len 40960 data: 7 8 8 9 9 9 7 7 7 7
30253 204/300 Write block 3156 len 28672 data: 8 6 8 6 7 6 6
30254 205/300 Read block 4138 len 32768
30255 206/300 Write block 484 len 28672 data: 5 7 9 9 8 9 8
30256 207/300 Read block 3186 len 16384
30257 208/300 Write block 894 len 16384 data: 4 3 4 5
30258 209/300 Write block 1126 len 8192 data: 5 4
30259 210/300 Read block 939 len 28672
30260 211/300 Read block 779 len 28672
30261 212/300 Read block 4353 len 8192
30262 213/300 Read block 339 len 20480
30263 214/300 Write block 146 len 40960 data: 3 3 3 3 2 3 6 7 7 7
30264 215/300 Read block 2583 len 36864
30265 216/300 Write block 4798 len 20480 data: 7 7 8 10 10
30266 217/300 Read block 3940 len 24576
30267 218/300 Write block 4720 len 16384 data: 8 8 9 6
30268 219/300 Write block 3119 len 16384 data: 4 5 5 4
30269 220/300 Write block 3022 len 24576 data: 5 7 6 6 6 5
30270 221/300 Write block 2604 len 16384 data: 7 7 7 4
30271 222/300 Write block 1135 len 40960 data: 6 7 7 8 7 7 8 8 6 4
30272 223/300 Write block 903 len 4096 data: 4
30273 224/300 Write block 161 len 8192 data: 9 9
30274 225/300 Write block 1202 len 16384 data: 8 8 8 8
30275 226/300 Read block 4958 len 16384
30276 227/300 Flush
30277 228/300 Read block 785 len 8192
30278 229/300 Read block 4545 len 8192
30279 230/300 Read block 339 len 8192
30280 231/300 Read block 652 len 24576
30281 232/300 Write block 3574 len 36864 data: 3 3 3 3 3 4 4 4 4
30282 233/300 Write block 2420 len 12288 data: 5 4 5
30283 234/300 Read block 4837 len 24576
30284 235/300 Read block 3191 len 8192
30285 236/300 Read block 2511 len 24576
30286 237/300 Read block 1325 len 32768
30287 238/300 Read block 1917 len 40960
30288 239/300 Write block 3655 len 20480 data: 4 4 5 5 4
30289 240/300 Write block 4536 len 24576 data: 7 7 6 6 5 4
30290 241/300 Write block 1748 len 8192 data: 12 10
30291 242/300 Read block 862 len 24576
30292 243/300 Write block 60 len 24576 data: 5 6 6 6 7 9
30293 244/300 Write block 3536 len 32768 data: 3 5 6 6 8 9 6 8
30294 245/300 Write block 3200 len 20480 data: 4 4 4 4 4
30295 246/300 Read block 2562 len 28672
30296 247/300 Write block 519 len 32768 data: 3 6 7 8 6 6 6 6
30297 248/300 Read block 50 len 8192
30298 249/300 Flush
30299 250/300 Read block 4481 len 20480
30300 251/300 Read block 3038 len 40960
30301 252/300 Write block 2735 len 12288 data: 6 5 5
30302 253/300 Write block 2779 len 32768 data: 5 4 5 6 5 5 4 6
30303 254/300 Write block 3753 len 32768 data: 5 4 5 5 7 7 7 6
30304 255/300 Read block 2572 len 36864
30305 256/300 Write block 2855 len 24576 data: 6 3 5 5 6 6
30306 257/300 Flush
30307 258/300 Write block 1108 len 24576 data: 4 5 5 6 8 9
30308 259/300 Read block 2521 len 36864
30309 260/300 Read block 1305 len 16384
30310 261/300 Read block 868 len 4096
30311 262/300 Read block 1006 len 20480
30312 263/300 Flush
30313 264/300 Write block 383 len 20480 data: 4 4 3 4 5
30314 265/300 Write block 4351 len 40960 data: 7 6 5 6 5 5 7 6 6 7
30315 266/300 Write block 3668 len 36864 data: 5 6 6 7 8 8 6 6 6
30316 267/300 Write block 933 len 4096 data: 6
30317 268/300 Write block 2155 len 32768 data: 5 5 4 4 4 3 3 3
30318 269/300 Flush
30319 270/300 Flush
30320 271/300 Read block 4497 len 32768
30321 272/300 Flush
30322 273/300 Write block 4627 len 40960 data: 5 4 4 3 4 5 5 4 4 4
30323 274/300 Read block 1868 len 20480
30324 275/300 Read block 4921 len 16384
30325 276/300 Write block 593 len 40960 data: 8 8 7 7 7 5 5 6 7 6
30326 277/300 Write block 1221 len 24576 data: 3 3 3 3 4 4
30327 278/300 Write block 1099 len 36864 data: 7 7 7 7 6 7 7 7 5
30328 279/300 Read block 825 len 8192
30329 280/300 Read block 1265 len 32768
30330 281/300 Write block 1042 len 28672 data: 6 5 5 6 5 6 6
30331 282/300 Read block 2733 len 32768
30332 283/300 Write block 2619 len 4096 data: 4
30333 284/300 Read block 4274 len 12288
30334 285/300 Read block 3081 len 40960
30335 286/300 Write block 1113 len 16384 data: 10 11 10 10
30336 287/300 Read block 4743 len 4096
30337 288/300 Read block 3519 len 20480
30338 289/300 Write block 2291 len 36864 data: 11 12 11 11 8 6 5 5 4
30339 290/300 Read block 1986 len 8192
30340 291/300 Flush
30341 292/300 Read block 2552 len 24576
30342 293/300 Read block 4524 len 24576
30343 294/300 Write block 1171 len 20480 data: 4 3 4 3 3
30344 295/300 Read block 4172 len 40960
30345 296/300 Read block 1903 len 24576
30346 297/300 Read block 1514 len 8192
30347 298/300 Write block 1657 len 36864 data: 7 8 7 6 5 5 6 4 4
30348 299/300 Read block 2293 len 32768
30349 300/300 Write block 197 len 24576 data: 4 5 5 6 5 5
30350 301/300 Read block 1776 len 4096
30351 Main task has work to do, go find it
30352 got dsc Start(1)
30353 start 1
30354 Dec 04 13:37:05.234 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/1, method: POST, req_id: c045b06a-d43c-4a25-8811-ab4d2c20439b, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
30355 Replay: started 1, returned:Ok(())
30356 ----------------------------------------------------------------
30357 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30358 States: Active Offline Active
30359 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30360 9452 Acked Write 3 Done Sent Done false
30361 9453 Acked Flush 0 Done Sent Done false
30362 9454 Acked Write 3 Done Sent Done false
30363 9455 Acked Write 6 Done Sent Done false
30364 9456 Acked Read 3 Done Sent Done false
30365 9457 Acked Write 4 Done Sent Done false
30366 9458 Acked Write 7 Done Sent Done false
30367 9459 Acked Read 4 Done Sent Done false
30368 9460 Acked Write 2 Done Sent Done false
30369 9461 Acked Flush 0 Done Sent Done false
30370 9462 Acked Read 8 Done Sent Done false
30371 9463 Acked Write 6 Done Sent Done false
30372 9464 Acked Read 2 Done Sent Done false
30373 9465 Acked Read 8 Done Sent Done false
30374 9466 Acked Read 10 Done Sent Done false
30375 9467 Acked Read 4 Done Sent Done false
30376 9468 Acked Write 4 Done Sent Done false
30377 9469 Acked Flush 0 Done Sent Done false
30378 9470 Acked Read 6 Done Sent Done false
30379 9471 Acked Read 1 Done Sent Done false
30380 9472 Acked Write 5 Done Sent Done false
30381 9473 Acked Read 6 Done Sent Done false
30382 9474 Acked Write 1 Done Sent Done false
30383 9475 Acked Read 4 Done Sent Done false
30384 9476 Acked Read 4 Done Sent Done false
30385 9477 Acked Read 4 Done Sent Done false
30386 9478 Acked Read 2 Done Sent Done false
30387 9479 Acked Write 6 Done Sent Done false
30388 9480 Acked Read 4 Done Sent Done false
30389 9481 Acked Write 9 Done Sent Done false
30390 9482 Acked Write 2 Done Sent Done false
30391 9483 Acked Read 6 Done Sent Done false
30392 9484 Acked Read 10 Done Sent Done false
30393 9485 Acked Write 7 Done Sent Done false
30394 9486 Acked Read 2 Done Sent Done false
30395 9487 Acked Write 7 Done Sent Done false
30396 9488 Acked Read 9 Done Sent Done false
30397 9489 Acked Read 8 Done Sent Done false
30398 9490 Acked Read 2 Done Sent Done false
30399 9491 Acked Flush 0 Done Sent Done false
30400 9492 Acked Read 4 Done Sent Done false
30401 9493 Acked Read 7 Done Sent Done false
30402 9494 Acked Flush 0 Done Sent Done false
30403 9495 Acked Flush 0 Done Sent Done false
30404 9496 Acked Write 1 Done Sent Done false
30405 9497 Acked Write 5 Done Sent Done false
30406 9498 Acked Read 10 Done Sent Done false
30407 9499 Acked Write 10 Done Sent Done false
30408 9500 Acked Write 3 Done Sent Done false
30409 9501 Acked Write 9 Done Sent Done false
30410 9502 Acked Read 7 Done Sent Done false
30411 9503 Acked Flush 0 Done Sent Done false
30412 9504 Acked Write 6 Done Sent Done false
30413 9505 Acked Read 4 Done Sent Done false
30414 9506 Acked Write 5 Done Sent Done false
30415 9507 Acked Write 6 Done Sent Done false
30416 9508 Acked Read 7 Done Sent Done false
30417 9509 Acked Read 6 Done Sent Done false
30418 9510 Acked Flush 0 Done Sent Done false
30419 9511 Acked Read 6 Done Sent Done false
30420 9512 Acked Write 9 Done Sent Done false
30421 9513 Acked Write 4 Done Sent Done false
30422 9514 Acked Read 10 Done Sent Done false
30423 9515 Acked Write 8 Done Sent Done false
30424 9516 Acked Read 6 Done Sent Done false
30425 9517 Acked Flush 0 Done Sent Done false
30426 9518 Acked Write 4 Done Sent Done false
30427 9519 Acked Write 7 Done Sent Done false
30428 9520 Acked Read 4 Done Sent Done false
30429 9521 Acked Read 5 Done Sent Done false
30430 9522 Acked Write 9 Done Sent Done false
30431 9523 Acked Write 7 Done Sent Done false
30432 9524 Acked Read 8 Done Sent Done false
30433 9525 Acked Write 10 Done Sent Done false
30434 9526 Acked Write 8 Done Sent Done false
30435 9527 Acked Flush 0 Done Sent Done false
30436 9528 Acked Read 8 Done Sent Done false
30437 9529 Acked Write 2 Done Sent Done false
30438 9530 Acked Flush 0 Done Sent Done false
30439 9531 Acked Flush 0 Done Sent Done false
30440 9532 Acked Read 6 Done Sent Done false
30441 9533 Acked Read 7 Done Sent Done false
30442 9534 Acked Write 10 Done Sent Done false
30443 9535 Acked Read 1 Done Sent Done false
30444 9536 Acked Write 7 Done Sent Done false
30445 9537 Acked Read 3 Done Sent Done false
30446 9538 Acked Read 8 Done Sent Done false
30447 9539 Acked Read 2 Done Sent Done false
30448 9540 Acked Read 3 Done Sent Done false
30449 9541 Acked Read 8 Done Sent Done false
30450 9542 Acked Read 10 Done Sent Done false
30451 9543 Acked Read 1 Done Sent Done false
30452 9544 Acked Write 7 Done Sent Done false
30453 9545 Acked Flush 0 Done Sent Done false
30454 9546 Acked Flush 0 Done Sent Done false
30455 9547 Acked Flush 0 Done Sent Done false
30456 9548 Acked Read 7 Done Sent Done false
30457 9549 Acked Read 1 Done Sent Done false
30458 9550 Acked Write 9 Done Sent Done false
30459 9551 Acked Read 4 Done Sent Done false
30460 9552 Acked Read 6 Done Sent Done false
30461 9553 Acked Write 6 Done Sent Done false
30462 9554 Acked Write 8 Done Sent Done false
30463 9555 Acked Write 8 Done Sent Done false
30464 9556 Acked Write 6 Done Sent Done false
30465 9557 Acked Read 2 Done Sent Done false
30466 9558 Acked Flush 0 Done Sent Done false
30467 9559 Acked Write 4 Done Sent Done false
30468 9560 Acked Read 4 Done Sent Done false
30469 9561 Acked Write 3 Done Sent Done false
30470 9562 Acked Write 3 Done Sent Done false
30471 9563 Acked Write 9 Done Sent Done false
30472 9564 Acked Write 1 Done Sent Done false
30473 9565 Acked Write 4 Done Sent Done false
30474 9566 Acked Read 3 Done Sent Done false
30475 9567 Acked Flush 0 Done Sent Done false
30476 9568 Acked Read 4 Done Sent Done false
30477 9569 Acked Write 3 Done Sent Done false
30478 9570 Acked Flush 0 Done Sent Done false
30479 9571 Acked Write 2 Done Sent Done false
30480 9572 Acked Read 2 Done Sent Done false
30481 9573 Acked Read 5 Done Sent Done false
30482 9574 Acked Write 8 Done Sent Done false
30483 9575 Acked Write 2 Done Sent Done false
30484 9576 Acked Read 10 Done Sent Done false
30485 9577 Acked Read 3 Done Sent Done false
30486 9578 Acked Read 3 Done Sent Done false
30487 9579 Acked Write 6 Done Sent Done false
30488 9580 Acked Read 1 Done Sent Done false
30489 9581 Acked Read 3 Done Sent Done false
30490 9582 Acked Write 2 Done Sent Done false
30491 9583 Acked Write 10 Done Sent Done false
30492 9584 Acked Write 3 Done Sent Done false
30493 9585 Acked Write 4 Done Sent Done false
30494 9586 Acked Write 7 Done Sent Done false
30495 9587 Acked Read 5 Done Sent Done false
30496 9588 Acked Read 10 Done Sent Done false
30497 9589 Acked Read 2 Done Sent Done false
30498 9590 Acked Read 3 Done Sent Done false
30499 9591 Acked Read 1 Done Sent Done false
30500 9592 Acked Write 5 Done Sent Done false
30501 9593 Acked Read 9 Done Sent Done false
30502 9594 Acked Read 2 Done Sent Done false
30503 9595 Acked Read 7 Done Sent Done false
30504 9596 Acked Write 4 Done Sent Done false
30505 9597 Acked Read 5 Done Sent Done false
30506 9598 Acked Write 8 Done Sent Done false
30507 9599 Acked Write 4 Done Sent Done false
30508 9600 Acked Read 1 Done Sent Done false
30509 9601 Acked Flush 0 Done Sent Done false
30510 9602 Acked Read 2 Done Sent Done false
30511 9603 Acked Write 9 Done Sent Done false
30512 9604 Acked Write 6 Done Sent Done false
30513 9605 Acked Read 9 Done Sent Done false
30514 9606 Acked Write 4 Done Sent Done false
30515 9607 Acked Read 3 Done Sent Done false
30516 9608 Acked Write 2 Done Sent Done false
30517 9609 Acked Flush 0 Done Sent Done false
30518 9610 Acked Write 1 Done Sent Done false
30519 9611 Acked Read 9 Done Sent Done false
30520 9612 Acked Read 1 Done Sent Done false
30521 9613 Acked Write 3 Done Sent Done false
30522 9614 Acked Read 8 Done Sent Done false
30523 9615 Acked Read 9 Done Sent Done false
30524 9616 Acked Write 9 Done Sent Done false
30525 9617 Acked Read 6 Done Sent Done false
30526 9618 Acked Read 1 Done Sent Done false
30527 9619 Acked Read 3 Done Sent Done false
30528 9620 Acked Read 9 Done Sent Done false
30529 9621 Acked Write 9 Done Sent Done false
30530 9622 Acked Read 7 Done Sent Done false
30531 9623 Acked Read 8 Done Sent Done false
30532 9624 Acked Read 8 Done Sent Done false
30533 9625 Acked Write 2 Done Sent Done false
30534 9626 Acked Write 4 Done Sent Done false
30535 9627 Acked Flush 0 Done Sent Done false
30536 9628 Acked Write 9 Done Sent Done false
30537 9629 Acked Write 3 Done Sent Done false
30538 9630 Acked Read 2 Done Sent Done false
30539 9631 Acked Write 4 Done Sent Done false
30540 9632 Acked Write 1 Done Sent Done false
30541 9633 Acked Flush 0 Done Sent Done false
30542 9634 Acked Write 8 Done Sent Done false
30543 9635 Acked Read 1 Done Sent Done false
30544 9636 Acked Write 2 Done Sent Done false
30545 9637 Acked Read 7 Done Sent Done false
30546 9638 Acked Read 4 Done Sent Done false
30547 9639 Acked Write 1 Done Sent Done false
30548 9640 Acked Read 4 Done Sent Done false
30549 9641 Acked Write 9 Done Sent Done false
30550 9642 Acked Flush 0 Done Sent Done false
30551 9643 Acked Write 10 Done Sent Done false
30552 9644 Acked Read 9 Done Sent Done false
30553 9645 Acked Read 1 Done Sent Done false
30554 9646 Acked Write 10 Done Sent Done false
30555 9647 Acked Write 5 Done Sent Done false
30556 9648 Acked Write 4 Done Sent Done false
30557 9649 Acked Write 6 Done Sent Done false
30558 9650 Acked Write 2 Done Sent Done false
30559 9651 Acked Read 9 Done Sent Done false
30560 9652 Acked Write 8 Done Sent Done false
30561 9653 Acked Read 3 Done Sent Done false
30562 9654 Acked Write 10 Done Sent Done false
30563 9655 Acked Write 7 Done Sent Done false
30564 9656 Acked Read 8 Done Sent Done false
30565 9657 Acked Write 7 Done Sent Done false
30566 9658 Acked Read 4 Done Sent Done false
30567 9659 Acked Write 4 Done Sent Done false
30568 9660 Acked Write 2 Done Sent Done false
30569 9661 Acked Read 7 Done Sent Done false
30570 9662 Acked Read 7 Done Sent Done false
30571 9663 Acked Read 2 Done Sent Done false
30572 9664 Acked Read 5 Done Sent Done false
30573 9665 Acked Write 10 Done Sent Done false
30574 9666 Acked Read 9 Done Sent Done false
30575 9667 Acked Write 5 Done Sent Done false
30576 9668 Acked Read 6 Done Sent Done false
30577 9669 Acked Write 4 Done Sent Done false
30578 9670 Acked Write 4 Done Sent Done false
30579 9671 Acked Write 6 Done Sent Done false
30580 9672 Acked Write 4 Done Sent Done false
30581 9673 Acked Write 10 Done Sent Done false
30582 9674 Acked Write 1 Done Sent Done false
30583 9675 Acked Write 2 Done Sent Done false
30584 9676 Acked Write 4 Done Sent Done false
30585 9677 Acked Read 4 Done Sent Done false
30586 9678 Acked Flush 0 Done Sent Done false
30587 9679 Acked Read 2 Done Sent Done false
30588 9680 Acked Read 2 Done Sent Done false
30589 9681 Acked Read 2 Done Sent Done false
30590 9682 Acked Read 6 Done Sent Done false
30591 9683 Acked Write 9 Done Sent Done false
30592 9684 Acked Write 3 Done Sent Done false
30593 9685 Acked Read 6 Done Sent Done false
30594 9686 Acked Read 2 Done Sent Done false
30595 9687 Acked Read 6 Done Sent Done false
30596 9688 Acked Read 8 Done Sent Done false
30597 9689 Acked Read 10 Done Sent Done false
30598 9690 Acked Write 5 Done Sent Done false
30599 9691 Acked Write 6 Done Sent Done false
30600 9692 Acked Write 2 Done Sent Done false
30601 9693 Acked Read 6 Done Sent Done false
30602 9694 Acked Write 6 Done Sent Done false
30603 9695 Acked Write 8 Done Sent Done false
30604 9696 Acked Write 5 Done Sent Done false
30605 9697 Acked Read 7 Done Sent Done false
30606 9698 Acked Write 8 Done Sent Done false
30607 9699 Acked Read 2 Done Sent Done false
30608 9700 Acked Flush 0 Done Sent Done false
30609 9701 Acked Read 5 Done Sent Done false
30610 9702 Acked Read 10 Done Sent Done false
30611 9703 Acked Write 3 Done Sent Done false
30612 9704 Acked Write 8 Done Sent Done false
30613 9705 Acked Write 8 Done Sent Done false
30614 9706 Acked Read 9 Done Sent Done false
30615 9707 Acked Write 6 Done Sent Done false
30616 9708 Acked Flush 0 Done Sent Done false
30617 9709 Acked Write 6 Done Sent Done false
30618 9710 Acked Read 9 Done Sent Done false
30619 9711 Acked Read 4 Done Sent Done false
30620 9712 Acked Read 1 Done Sent Done false
30621 9713 Acked Read 5 Done Sent Done false
30622 9714 Acked Flush 0 Done Sent Done false
30623 9715 Acked Write 5 Done Sent Done false
30624 9716 Acked Write 10 Done Sent Done false
30625 9717 Acked Write 9 Done Sent Done false
30626 9718 Acked Write 1 Done Sent Done false
30627 9719 Acked Write 8 Done Sent Done false
30628 9720 Acked Flush 0 Done Sent Done false
30629 9721 Acked Flush 0 Done Sent Done false
30630 9722 Acked Read 8 Done Sent Done false
30631 9723 Acked Flush 0 Done Sent Done false
30632 9724 Acked Write 10 Done Sent Done false
30633 9725 Acked Read 5 Done Sent Done false
30634 9726 Acked Read 4 Done Sent Done false
30635 9727 Acked Write 10 Done Sent Done false
30636 9728 Acked Write 6 Done Sent Done false
30637 9729 Acked Write 9 Done Sent Done false
30638 9730 Acked Read 2 Done Sent Done false
30639 9731 Acked Read 8 Done Sent Done false
30640 9732 Acked Write 7 Done Sent Done false
30641 9733 Acked Read 8 Done Sent Done false
30642 9734 Acked Write 1 Done Sent Done false
30643 9735 Acked Read 3 Done Sent Done false
30644 9736 Acked Read 10 Done Sent Done false
30645 9737 Acked Write 4 Done Sent Done false
30646 9738 Acked Read 1 Done Sent Done false
30647 9739 Acked Read 5 Done Sent Done false
30648 9740 Acked Write 9 Done Sent Done false
30649 9741 Acked Read 2 Done Sent Done false
30650 9742 Acked Flush 0 Done Sent Done false
30651 9743 Acked Read 6 Done Sent Done false
30652 9744 Acked Read 6 Done Sent Done false
30653 9745 Acked Write 5 Done Sent Done false
30654 9746 Acked Read 10 Done Sent Done false
30655 9747 Acked Read 6 Done Sent Done false
30656 9748 Acked Read 2 Done Sent Done false
30657 9749 Acked Write 9 Done Sent Done false
30658 9750 Acked Read 8 Done Sent Done false
30659 9751 Acked Write 6 Done Sent Done false
30660 9752 Acked Read 1 Done Sent Done false
30661 STATES DS:0 DS:1 DS:2 TOTAL
30662 Sent 0 301 0 301
30663 Done 301 0 301 602
30664 Skipped 0 0 0 0
30665 Error 0 0 0 0
30666 Last Flush: 9742 9451 9742
30667 Downstairs last five completed: 9451 9450 9449 9448 9447
30668 Upstairs last five completed: 9752 9751 9750 9749 9748
30669 CLIENT: Up:0 ds:301 act:2
30670 [8820] got start action
30671 [8820] I am going to restart
30672 Starting downstairs at port 8820
30673 Make output file at "/tmp/dsc/downstairs-8820.txt"
30674 [8820][1] reports Starting
30675 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1232)
30676 [8820][1] reports Running
30677 ----------------------------------------------------------------
30678 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30679 States: Active Offline Active
30680 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30681 9452 Acked Write 3 Done Sent Done false
30682 9453 Acked Flush 0 Done Sent Done false
30683 9454 Acked Write 3 Done Sent Done false
30684 9455 Acked Write 6 Done Sent Done false
30685 9456 Acked Read 3 Done Sent Done false
30686 9457 Acked Write 4 Done Sent Done false
30687 9458 Acked Write 7 Done Sent Done false
30688 9459 Acked Read 4 Done Sent Done false
30689 9460 Acked Write 2 Done Sent Done false
30690 9461 Acked Flush 0 Done Sent Done false
30691 9462 Acked Read 8 Done Sent Done false
30692 9463 Acked Write 6 Done Sent Done false
30693 9464 Acked Read 2 Done Sent Done false
30694 9465 Acked Read 8 Done Sent Done false
30695 9466 Acked Read 10 Done Sent Done false
30696 9467 Acked Read 4 Done Sent Done false
30697 9468 Acked Write 4 Done Sent Done false
30698 9469 Acked Flush 0 Done Sent Done false
30699 9470 Acked Read 6 Done Sent Done false
30700 9471 Acked Read 1 Done Sent Done false
30701 9472 Acked Write 5 Done Sent Done false
30702 9473 Acked Read 6 Done Sent Done false
30703 9474 Acked Write 1 Done Sent Done false
30704 9475 Acked Read 4 Done Sent Done false
30705 9476 Acked Read 4 Done Sent Done false
30706 9477 Acked Read 4 Done Sent Done false
30707 9478 Acked Read 2 Done Sent Done false
30708 9479 Acked Write 6 Done Sent Done false
30709 9480 Acked Read 4 Done Sent Done false
30710 9481 Acked Write 9 Done Sent Done false
30711 9482 Acked Write 2 Done Sent Done false
30712 9483 Acked Read 6 Done Sent Done false
30713 9484 Acked Read 10 Done Sent Done false
30714 9485 Acked Write 7 Done Sent Done false
30715 9486 Acked Read 2 Done Sent Done false
30716 9487 Acked Write 7 Done Sent Done false
30717 9488 Acked Read 9 Done Sent Done false
30718 9489 Acked Read 8 Done Sent Done false
30719 9490 Acked Read 2 Done Sent Done false
30720 9491 Acked Flush 0 Done Sent Done false
30721 9492 Acked Read 4 Done Sent Done false
30722 9493 Acked Read 7 Done Sent Done false
30723 9494 Acked Flush 0 Done Sent Done false
30724 9495 Acked Flush 0 Done Sent Done false
30725 9496 Acked Write 1 Done Sent Done false
30726 9497 Acked Write 5 Done Sent Done false
30727 9498 Acked Read 10 Done Sent Done false
30728 9499 Acked Write 10 Done Sent Done false
30729 9500 Acked Write 3 Done Sent Done false
30730 9501 Acked Write 9 Done Sent Done false
30731 9502 Acked Read 7 Done Sent Done false
30732 9503 Acked Flush 0 Done Sent Done false
30733 9504 Acked Write 6 Done Sent Done false
30734 9505 Acked Read 4 Done Sent Done false
30735 9506 Acked Write 5 Done Sent Done false
30736 9507 Acked Write 6 Done Sent Done false
30737 9508 Acked Read 7 Done Sent Done false
30738 9509 Acked Read 6 Done Sent Done false
30739 9510 Acked Flush 0 Done Sent Done false
30740 9511 Acked Read 6 Done Sent Done false
30741 9512 Acked Write 9 Done Sent Done false
30742 9513 Acked Write 4 Done Sent Done false
30743 9514 Acked Read 10 Done Sent Done false
30744 9515 Acked Write 8 Done Sent Done false
30745 9516 Acked Read 6 Done Sent Done false
30746 9517 Acked Flush 0 Done Sent Done false
30747 9518 Acked Write 4 Done Sent Done false
30748 9519 Acked Write 7 Done Sent Done false
30749 9520 Acked Read 4 Done Sent Done false
30750 9521 Acked Read 5 Done Sent Done false
30751 9522 Acked Write 9 Done Sent Done false
30752 9523 Acked Write 7 Done Sent Done false
30753 9524 Acked Read 8 Done Sent Done false
30754 9525 Acked Write 10 Done Sent Done false
30755 9526 Acked Write 8 Done Sent Done false
30756 9527 Acked Flush 0 Done Sent Done false
30757 9528 Acked Read 8 Done Sent Done false
30758 9529 Acked Write 2 Done Sent Done false
30759 9530 Acked Flush 0 Done Sent Done false
30760 9531 Acked Flush 0 Done Sent Done false
30761 9532 Acked Read 6 Done Sent Done false
30762 9533 Acked Read 7 Done Sent Done false
30763 9534 Acked Write 10 Done Sent Done false
30764 9535 Acked Read 1 Done Sent Done false
30765 9536 Acked Write 7 Done Sent Done false
30766 9537 Acked Read 3 Done Sent Done false
30767 9538 Acked Read 8 Done Sent Done false
30768 9539 Acked Read 2 Done Sent Done false
30769 9540 Acked Read 3 Done Sent Done false
30770 9541 Acked Read 8 Done Sent Done false
30771 9542 Acked Read 10 Done Sent Done false
30772 9543 Acked Read 1 Done Sent Done false
30773 9544 Acked Write 7 Done Sent Done false
30774 9545 Acked Flush 0 Done Sent Done false
30775 9546 Acked Flush 0 Done Sent Done false
30776 9547 Acked Flush 0 Done Sent Done false
30777 9548 Acked Read 7 Done Sent Done false
30778 9549 Acked Read 1 Done Sent Done false
30779 9550 Acked Write 9 Done Sent Done false
30780 9551 Acked Read 4 Done Sent Done false
30781 9552 Acked Read 6 Done Sent Done false
30782 9553 Acked Write 6 Done Sent Done false
30783 9554 Acked Write 8 Done Sent Done false
30784 9555 Acked Write 8 Done Sent Done false
30785 9556 Acked Write 6 Done Sent Done false
30786 9557 Acked Read 2 Done Sent Done false
30787 9558 Acked Flush 0 Done Sent Done false
30788 9559 Acked Write 4 Done Sent Done false
30789 9560 Acked Read 4 Done Sent Done false
30790 9561 Acked Write 3 Done Sent Done false
30791 9562 Acked Write 3 Done Sent Done false
30792 9563 Acked Write 9 Done Sent Done false
30793 9564 Acked Write 1 Done Sent Done false
30794 9565 Acked Write 4 Done Sent Done false
30795 9566 Acked Read 3 Done Sent Done false
30796 9567 Acked Flush 0 Done Sent Done false
30797 9568 Acked Read 4 Done Sent Done false
30798 9569 Acked Write 3 Done Sent Done false
30799 9570 Acked Flush 0 Done Sent Done false
30800 9571 Acked Write 2 Done Sent Done false
30801 9572 Acked Read 2 Done Sent Done false
30802 9573 Acked Read 5 Done Sent Done false
30803 9574 Acked Write 8 Done Sent Done false
30804 9575 Acked Write 2 Done Sent Done false
30805 9576 Acked Read 10 Done Sent Done false
30806 9577 Acked Read 3 Done Sent Done false
30807 9578 Acked Read 3 Done Sent Done false
30808 9579 Acked Write 6 Done Sent Done false
30809 9580 Acked Read 1 Done Sent Done false
30810 9581 Acked Read 3 Done Sent Done false
30811 9582 Acked Write 2 Done Sent Done false
30812 9583 Acked Write 10 Done Sent Done false
30813 9584 Acked Write 3 Done Sent Done false
30814 9585 Acked Write 4 Done Sent Done false
30815 9586 Acked Write 7 Done Sent Done false
30816 9587 Acked Read 5 Done Sent Done false
30817 9588 Acked Read 10 Done Sent Done false
30818 9589 Acked Read 2 Done Sent Done false
30819 9590 Acked Read 3 Done Sent Done false
30820 9591 Acked Read 1 Done Sent Done false
30821 9592 Acked Write 5 Done Sent Done false
30822 9593 Acked Read 9 Done Sent Done false
30823 9594 Acked Read 2 Done Sent Done false
30824 9595 Acked Read 7 Done Sent Done false
30825 9596 Acked Write 4 Done Sent Done false
30826 9597 Acked Read 5 Done Sent Done false
30827 9598 Acked Write 8 Done Sent Done false
30828 9599 Acked Write 4 Done Sent Done false
30829 9600 Acked Read 1 Done Sent Done false
30830 9601 Acked Flush 0 Done Sent Done false
30831 9602 Acked Read 2 Done Sent Done false
30832 9603 Acked Write 9 Done Sent Done false
30833 9604 Acked Write 6 Done Sent Done false
30834 9605 Acked Read 9 Done Sent Done false
30835 9606 Acked Write 4 Done Sent Done false
30836 9607 Acked Read 3 Done Sent Done false
30837 9608 Acked Write 2 Done Sent Done false
30838 9609 Acked Flush 0 Done Sent Done false
30839 9610 Acked Write 1 Done Sent Done false
30840 9611 Acked Read 9 Done Sent Done false
30841 9612 Acked Read 1 Done Sent Done false
30842 9613 Acked Write 3 Done Sent Done false
30843 9614 Acked Read 8 Done Sent Done false
30844 9615 Acked Read 9 Done Sent Done false
30845 9616 Acked Write 9 Done Sent Done false
30846 9617 Acked Read 6 Done Sent Done false
30847 9618 Acked Read 1 Done Sent Done false
30848 9619 Acked Read 3 Done Sent Done false
30849 9620 Acked Read 9 Done Sent Done false
30850 9621 Acked Write 9 Done Sent Done false
30851 9622 Acked Read 7 Done Sent Done false
30852 9623 Acked Read 8 Done Sent Done false
30853 9624 Acked Read 8 Done Sent Done false
30854 9625 Acked Write 2 Done Sent Done false
30855 9626 Acked Write 4 Done Sent Done false
30856 9627 Acked Flush 0 Done Sent Done false
30857 9628 Acked Write 9 Done Sent Done false
30858 9629 Acked Write 3 Done Sent Done false
30859 9630 Acked Read 2 Done Sent Done false
30860 9631 Acked Write 4 Done Sent Done false
30861 9632 Acked Write 1 Done Sent Done false
30862 9633 Acked Flush 0 Done Sent Done false
30863 9634 Acked Write 8 Done Sent Done false
30864 9635 Acked Read 1 Done Sent Done false
30865 9636 Acked Write 2 Done Sent Done false
30866 9637 Acked Read 7 Done Sent Done false
30867 9638 Acked Read 4 Done Sent Done false
30868 9639 Acked Write 1 Done Sent Done false
30869 9640 Acked Read 4 Done Sent Done false
30870 9641 Acked Write 9 Done Sent Done false
30871 9642 Acked Flush 0 Done Sent Done false
30872 9643 Acked Write 10 Done Sent Done false
30873 9644 Acked Read 9 Done Sent Done false
30874 9645 Acked Read 1 Done Sent Done false
30875 9646 Acked Write 10 Done Sent Done false
30876 9647 Acked Write 5 Done Sent Done false
30877 9648 Acked Write 4 Done Sent Done false
30878 9649 Acked Write 6 Done Sent Done false
30879 9650 Acked Write 2 Done Sent Done false
30880 9651 Acked Read 9 Done Sent Done false
30881 9652 Acked Write 8 Done Sent Done false
30882 9653 Acked Read 3 Done Sent Done false
30883 9654 Acked Write 10 Done Sent Done false
30884 9655 Acked Write 7 Done Sent Done false
30885 9656 Acked Read 8 Done Sent Done false
30886 9657 Acked Write 7 Done Sent Done false
30887 9658 Acked Read 4 Done Sent Done false
30888 9659 Acked Write 4 Done Sent Done false
30889 9660 Acked Write 2 Done Sent Done false
30890 9661 Acked Read 7 Done Sent Done false
30891 9662 Acked Read 7 Done Sent Done false
30892 9663 Acked Read 2 Done Sent Done false
30893 9664 Acked Read 5 Done Sent Done false
30894 9665 Acked Write 10 Done Sent Done false
30895 9666 Acked Read 9 Done Sent Done false
30896 9667 Acked Write 5 Done Sent Done false
30897 9668 Acked Read 6 Done Sent Done false
30898 9669 Acked Write 4 Done Sent Done false
30899 9670 Acked Write 4 Done Sent Done false
30900 9671 Acked Write 6 Done Sent Done false
30901 9672 Acked Write 4 Done Sent Done false
30902 9673 Acked Write 10 Done Sent Done false
30903 9674 Acked Write 1 Done Sent Done false
30904 9675 Acked Write 2 Done Sent Done false
30905 9676 Acked Write 4 Done Sent Done false
30906 9677 Acked Read 4 Done Sent Done false
30907 9678 Acked Flush 0 Done Sent Done false
30908 9679 Acked Read 2 Done Sent Done false
30909 9680 Acked Read 2 Done Sent Done false
30910 9681 Acked Read 2 Done Sent Done false
30911 9682 Acked Read 6 Done Sent Done false
30912 9683 Acked Write 9 Done Sent Done false
30913 9684 Acked Write 3 Done Sent Done false
30914 9685 Acked Read 6 Done Sent Done false
30915 9686 Acked Read 2 Done Sent Done false
30916 9687 Acked Read 6 Done Sent Done false
30917 9688 Acked Read 8 Done Sent Done false
30918 9689 Acked Read 10 Done Sent Done false
30919 9690 Acked Write 5 Done Sent Done false
30920 9691 Acked Write 6 Done Sent Done false
30921 9692 Acked Write 2 Done Sent Done false
30922 9693 Acked Read 6 Done Sent Done false
30923 9694 Acked Write 6 Done Sent Done false
30924 9695 Acked Write 8 Done Sent Done false
30925 9696 Acked Write 5 Done Sent Done false
30926 9697 Acked Read 7 Done Sent Done false
30927 9698 Acked Write 8 Done Sent Done false
30928 9699 Acked Read 2 Done Sent Done false
30929 9700 Acked Flush 0 Done Sent Done false
30930 9701 Acked Read 5 Done Sent Done false
30931 9702 Acked Read 10 Done Sent Done false
30932 9703 Acked Write 3 Done Sent Done false
30933 9704 Acked Write 8 Done Sent Done false
30934 9705 Acked Write 8 Done Sent Done false
30935 9706 Acked Read 9 Done Sent Done false
30936 9707 Acked Write 6 Done Sent Done false
30937 9708 Acked Flush 0 Done Sent Done false
30938 9709 Acked Write 6 Done Sent Done false
30939 9710 Acked Read 9 Done Sent Done false
30940 9711 Acked Read 4 Done Sent Done false
30941 9712 Acked Read 1 Done Sent Done false
30942 9713 Acked Read 5 Done Sent Done false
30943 9714 Acked Flush 0 Done Sent Done false
30944 9715 Acked Write 5 Done Sent Done false
30945 9716 Acked Write 10 Done Sent Done false
30946 9717 Acked Write 9 Done Sent Done false
30947 9718 Acked Write 1 Done Sent Done false
30948 9719 Acked Write 8 Done Sent Done false
30949 9720 Acked Flush 0 Done Sent Done false
30950 9721 Acked Flush 0 Done Sent Done false
30951 9722 Acked Read 8 Done Sent Done false
30952 9723 Acked Flush 0 Done Sent Done false
30953 9724 Acked Write 10 Done Sent Done false
30954 9725 Acked Read 5 Done Sent Done false
30955 9726 Acked Read 4 Done Sent Done false
30956 9727 Acked Write 10 Done Sent Done false
30957 9728 Acked Write 6 Done Sent Done false
30958 9729 Acked Write 9 Done Sent Done false
30959 9730 Acked Read 2 Done Sent Done false
30960 9731 Acked Read 8 Done Sent Done false
30961 9732 Acked Write 7 Done Sent Done false
30962 9733 Acked Read 8 Done Sent Done false
30963 9734 Acked Write 1 Done Sent Done false
30964 9735 Acked Read 3 Done Sent Done false
30965 9736 Acked Read 10 Done Sent Done false
30966 9737 Acked Write 4 Done Sent Done false
30967 9738 Acked Read 1 Done Sent Done false
30968 9739 Acked Read 5 Done Sent Done false
30969 9740 Acked Write 9 Done Sent Done false
30970 9741 Acked Read 2 Done Sent Done false
30971 9742 Acked Flush 0 Done Sent Done false
30972 9743 Acked Read 6 Done Sent Done false
30973 9744 Acked Read 6 Done Sent Done false
30974 9745 Acked Write 5 Done Sent Done false
30975 9746 Acked Read 10 Done Sent Done false
30976 9747 Acked Read 6 Done Sent Done false
30977 9748 Acked Read 2 Done Sent Done false
30978 9749 Acked Write 9 Done Sent Done false
30979 9750 Acked Read 8 Done Sent Done false
30980 9751 Acked Write 6 Done Sent Done false
30981 9752 Acked Read 1 Done Sent Done false
30982 9753 Acked Flush 0 Done Sent Done false
30983 STATES DS:0 DS:1 DS:2 TOTAL
30984 Sent 0 302 0 302
30985 Done 302 0 302 604
30986 Skipped 0 0 0 0
30987 Error 0 0 0 0
30988 Last Flush: 9753 9451 9753
30989 Downstairs last five completed: 9451 9450 9449 9448 9447
30990 Upstairs last five completed: 9753 9752 9751 9750 9749
30991 CLIENT: Up:0 ds:302 act:2
309922024-12-04T13:37:11.007ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
309932024-12-04T13:37:11.007ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48764) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
309942024-12-04T13:37:11.008ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
309952024-12-04T13:37:11.009ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
309962024-12-04T13:37:11.009ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
309972024-12-04T13:37:11.009ZINFOcrucible: send last flush ID to this DS: 9451 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
309982024-12-04T13:37:11.009ZINFOcrucible: Replied this last flush ID: 9451 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
309992024-12-04T13:37:11.009ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
310002024-12-04T13:37:11.009ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
310012024-12-04T13:37:11.009ZINFOcrucible: [1] client re-new 302 jobs since flush 9451 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
310022024-12-04T13:37:11.010ZINFOcrucible: [1] Replayed 302 jobs since flush: 9451 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31003 ----------------------------------------------------------------
31004 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31005 Downstairs last five completed: 9753 9752 9751 9750 9749
31006 Upstairs last five completed: 9753 9752 9751 9750 9749
31007 CLIENT: Up:0 ds:0 act:3
31008 Replay: All jobs finished
31009 Main task has work to do, go find it
31010 got dsc Stop(1)
31011 stop 1
31012 Dec 04 13:37:13.238 INFO[8820] Got stop action so:false kr:false
31013 request completed, latency_us: 350, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 31ce66ec-b474-4405-9207-26ee7c87bb2b, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
310142024-12-04T13:37:13.239ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31015 Dec 04 13:37:13.239 INFO request completed, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:37:13.239554976Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"":"io task","client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
31016 latency_us: 111, response_code: 200, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:37:13.239577186Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
31017 uri: /state/cid/1, method: GET, req_id: f7c2eff8-82c4-4359-9049-4ce0d2c2726d, remote_addr: 127.0.0.1:37804, local_addr: 127.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:37:13.239604146Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
31018 0.0.1:9998
310192024-12-04T13:37:13.239ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
310202024-12-04T13:37:13.239ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31021 [8820] Exited with: ExitStatus(unix_wait_status(9))
31022 [8820][1] reports Exit
31023 Dec 04 13:37:17.241 INFO request completed, latency_us: 193, response_code: 200, uri: /state/cid/1, method: GET, req_id: 335347df-aa62-4b55-b66b-b193a2764a73, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
31024 001/300 Read block 4708 len 24576
31025 002/300 Read block 2124 len 4096
31026 003/300 Read block 1601 len 20480
31027 004/300 Write block 2894 len 40960 data: 8 9 8 8 8 8 8 9 8 9
31028 005/300 Flush
31029 006/300 Write block 52 len 32768 data: 3 4 6 8 7 7 7 5
31030 007/300 Write block 3090 len 4096 data: 7
31031 008/300 Read block 801 len 28672
31032 009/300 Flush
31033 010/300 Flush
31034 011/300 Read block 1432 len 36864
31035 012/300 Write block 2073 len 36864 data: 7 6 4 3 3 4 4 4 6
31036 013/300 Read block 2244 len 24576
31037 014/300 Write block 1058 len 36864 data: 9 8 7 7 7 6 7 9 9
31038 015/300 Write block 3846 len 4096 data: 7
31039 016/300 Read block 4913 len 28672
31040 017/300 Read block 1966 len 24576
31041 018/300 Write block 3547 len 24576 data: 8 7 8 7 9 9
31042 019/300 Read block 4878 len 4096
31043 020/300 Write block 4976 len 36864 data: 7 8 8 9 7 6 5 6 6
31044 021/300 Write block 1008 len 8192 data: 6 4
31045 022/300 Read block 255 len 28672
31046 023/300 Read block 2111 len 32768
31047 024/300 Read block 4400 len 32768
31048 025/300 Read block 4601 len 32768
31049 026/300 Read block 797 len 8192
31050 027/300 Write block 1781 len 40960 data: 4 4 5 6 6 6 7 7 5 6
31051 028/300 Read block 2682 len 40960
31052 029/300 Write block 184 len 32768 data: 4 5 6 6 5 6 6 7
31053 030/300 Read block 2486 len 32768
31054 031/300 Write block 101 len 40960 data: 7 8 7 6 6 4 3 3 2 2
31055 032/300 Write block 3043 len 36864 data: 4 3 3 5 5 5 5 7 8
31056 033/300 Read block 326 len 24576
31057 034/300 Flush
31058 035/300 Read block 4385 len 40960
31059 036/300 Read block 2310 len 16384
31060 037/300 Write block 3562 len 16384 data: 3 6 8 8
31061 038/300 Flush
31062 039/300 Write block 67 len 28672 data: 8 6 6 3 4 6 6
31063 040/300 Read block 4606 len 16384
31064 041/300 Write block 3437 len 24576 data: 10 10 10 8 6 6
31065 042/300 Write block 532 len 4096 data: 4
31066 043/300 Read block 761 len 28672
31067 044/300 Read block 2816 len 40960
31068 045/300 Read block 3701 len 40960
31069 046/300 Read block 1174 len 36864
31070 047/300 Read block 4630 len 16384
31071 048/300 Read block 2413 len 16384
31072 049/300 Read block 2329 len 28672
31073 050/300 Write block 68 len 20480 data: 7 7 4 5 7
31074 051/300 Read block 2116 len 8192
31075 052/300 Write block 3295 len 32768 data: 9 10 10 9 8 8 9 8
31076 053/300 Write block 4284 len 36864 data: 6 6 6 5 4 4 3 4 4
31077 054/300 Write block 3549 len 40960 data: 9 8 10 10 8 7 5 3 3 2
31078 055/300 Write block 880 len 4096 data: 3
31079 056/300 Write block 3967 len 8192 data: 6 6
31080 057/300 Read block 2897 len 8192
31081 058/300 Flush
31082 059/300 Read block 2609 len 8192
31083 060/300 Write block 292 len 24576 data: 3 4 7 8 9 7
31084 061/300 Flush
31085 062/300 Flush
31086 063/300 Read block 664 len 40960
31087 064/300 Write block 4578 len 40960 data: 6 4 4 3 4 4 5 5 5 6
31088 065/300 Write block 4712 len 16384 data: 3 4 4 5
31089 066/300 Write block 4898 len 12288 data: 4 4 7
31090 067/300 Write block 4626 len 20480 data: 5 6 5 5 4
31091 068/300 Write block 3293 len 20480 data: 6 8 10 11 11
31092 069/300 Read block 1943 len 32768
31093 070/300 Write block 3858 len 36864 data: 4 6 7 7 7 7 5 5 5
31094 071/300 Write block 4730 len 28672 data: 4 4 4 5 7 8 8
31095 072/300 Read block 1008 len 32768
31096 073/300 Read block 4778 len 4096
31097 074/300 Write block 1373 len 8192 data: 4 5
31098 075/300 Write block 4298 len 8192 data: 6 7
31099 076/300 Read block 4514 len 16384
31100 077/300 Read block 2911 len 28672
31101 078/300 Read block 1042 len 12288
31102 079/300 Write block 92 len 12288 data: 8 8 7
31103 080/300 Write block 331 len 32768 data: 7 6 5 5 5 4 4 4
31104 081/300 Write block 1826 len 36864 data: 3 4 5 6 5 5 5 5 5
31105 082/300 Read block 2601 len 40960
31106 083/300 Read block 1532 len 24576
31107 084/300 Read block 1189 len 24576
31108 085/300 Write block 3438 len 36864 data: 11 11 9 7 7 7 8 7 6
31109 086/300 Read block 1717 len 36864
31110 087/300 Read block 4130 len 28672
31111 088/300 Read block 812 len 36864
31112 089/300 Write block 3834 len 20480 data: 3 3 3 3 4
31113 090/300 Read block 4635 len 36864
31114 091/300 Read block 2668 len 24576
31115 092/300 Read block 4115 len 36864
31116 093/300 Write block 2243 len 20480 data: 5 5 7 6 10
31117 094/300 Write block 2959 len 36864 data: 4 4 5 5 4 7 7 6 6
31118 095/300 Read block 3070 len 4096
31119 096/300 Read block 2717 len 40960
31120 097/300 Read block 427 len 20480
31121 098/300 Read block 4168 len 12288
31122 099/300 Read block 4471 len 8192
31123 100/300 Write block 3191 len 4096 data: 3
31124 101/300 Write block 1724 len 24576 data: 5 5 6 6 7 7
31125 102/300 Write block 1741 len 4096 data: 9
31126 103/300 Read block 2532 len 36864
31127 104/300 Write block 3979 len 4096 data: 5
31128 105/300 Read block 1365 len 20480
31129 106/300 Write block 4909 len 8192 data: 10 7
31130 107/300 Write block 947 len 16384 data: 8 8 7 9
31131 108/300 Read block 1787 len 20480
31132 109/300 Read block 1930 len 20480
31133 110/300 Write block 214 len 28672 data: 4 5 5 4 5 5 5
31134 111/300 Read block 771 len 12288
31135 112/300 Write block 1845 len 32768 data: 10 9 9 7 9 9 8 10
31136 113/300 Read block 830 len 16384
31137 114/300 Flush
31138 115/300 Write block 2897 len 36864 data: 9 9 9 9 10 9 10 5 5
31139 116/300 Read block 2872 len 36864
31140 117/300 Read block 12 len 20480
31141 118/300 Read block 4449 len 32768
31142 119/300 Read block 2745 len 24576
31143 120/300 Read block 2974 len 32768
31144 121/300 Read block 3295 len 32768
31145 122/300 Read block 4090 len 8192
31146 123/300 Read block 1621 len 16384
31147 124/300 Write block 400 len 12288 data: 6 6 6
31148 125/300 Read block 2670 len 24576
31149 126/300 Read block 787 len 16384
31150 127/300 Read block 4221 len 8192
31151 128/300 Read block 195 len 20480
31152 129/300 Write block 1142 len 8192 data: 9 7
31153 130/300 Read block 2919 len 32768
31154 131/300 Write block 3730 len 28672 data: 4 4 6 5 5 6 5
31155 132/300 Write block 1863 len 36864 data: 5 3 3 4 6 5 5 5 4
31156 133/300 Write block 2536 len 40960 data: 8 11 9 8 8 7 5 5 5 5
31157 134/300 Read block 509 len 12288
31158 135/300 Read block 1537 len 16384
31159 136/300 Write block 4593 len 4096 data: 8
31160 137/300 Read block 1052 len 8192
31161 138/300 Write block 4143 len 32768 data: 4 4 3 4 4 4 6 5
31162 139/300 Write block 1989 len 16384 data: 7 7 8 6
31163 140/300 Write block 4023 len 16384 data: 6 4 3 5
31164 141/300 Read block 2947 len 32768
31165 142/300 Read block 2051 len 32768
31166 143/300 Write block 4693 len 16384 data: 6 7 6 7
31167 144/300 Write block 1525 len 40960 data: 12 13 12 11 10 9 7 7 6 6
31168 145/300 Read block 1001 len 40960
31169 146/300 Read block 1418 len 12288
31170 147/300 Read block 1062 len 36864
31171 148/300 Read block 1226 len 20480
31172 149/300 Write block 218 len 28672 data: 6 6 6 5 4 5 4
31173 150/300 Write block 30 len 12288 data: 8 9 7
31174 151/300 Read block 1323 len 4096
31175 152/300 Read block 3541 len 36864
31176 153/300 Read block 1757 len 40960
31177 154/300 Write block 327 len 4096 data: 5
31178 155/300 Write block 4450 len 24576 data: 5 5 3 2 3 3
31179 156/300 Read block 4188 len 16384
31180 157/300 Read block 735 len 20480
31181 158/300 Read block 311 len 32768
31182 159/300 Read block 2617 len 28672
31183 160/300 Read block 1479 len 40960
31184 161/300 Read block 3000 len 28672
31185 162/300 Write block 618 len 28672 data: 5 5 5 6 5 5 4
31186 163/300 Flush
31187 164/300 Read block 2470 len 32768
31188 165/300 Read block 154 len 40960
31189 166/300 Read block 1408 len 32768
31190 167/300 Write block 1507 len 28672 data: 5 5 6 8 9 10 9
31191 168/300 Write block 3007 len 16384 data: 7 7 6 6
31192 169/300 Write block 311 len 32768 data: 7 8 7 7 8 8 7 8
31193 170/300 Write block 639 len 36864 data: 5 5 3 4 6 8 8 8 8
31194 171/300 Read block 625 len 20480
31195 172/300 Write block 2378 len 12288 data: 6 7 7
31196 173/300 Read block 4113 len 16384
31197 174/300 Read block 535 len 36864
31198 175/300 Read block 4804 len 4096
31199 176/300 Write block 2405 len 36864 data: 7 6 6 6 6 7 7 8 8
31200 177/300 Flush
31201 178/300 Read block 3582 len 12288
31202 179/300 Read block 1566 len 20480
31203 180/300 Write block 4306 len 4096 data: 6
31204 181/300 Write block 4150 len 40960 data: 6 5 6 8 8 10 8 8 10 14
31205 182/300 Write block 2707 len 16384 data: 7 7 9 10
31206 183/300 Read block 903 len 16384
31207 184/300 Read block 3741 len 40960
31208 185/300 Write block 2633 len 32768 data: 6 7 6 8 8 9 10 11
31209 186/300 Flush
31210 187/300 Flush
31211 188/300 Write block 313 len 28672 data: 8 8 9 9 8 9 6
31212 189/300 Write block 4895 len 24576 data: 5 5 3 5 5 8
31213 190/300 Read block 4177 len 28672
31214 191/300 Write block 1970 len 20480 data: 7 7 8 6 7
31215 192/300 Write block 3907 len 24576 data: 3 4 5 5 5 4
31216 193/300 Write block 4678 len 24576 data: 8 9 9 6 5 5
31217 194/300 Write block 1993 len 4096 data: 6
31218 195/300 Read block 3061 len 20480
31219 196/300 Read block 1287 len 24576
31220 197/300 Read block 2785 len 4096
31221 198/300 Write block 3804 len 36864 data: 4 5 5 4 6 8 7 8 8
31222 199/300 Flush
31223 200/300 Flush
31224 201/300 Read block 3934 len 36864
31225 202/300 Flush
31226 203/300 Write block 3890 len 36864 data: 4 3 3 3 2 4 5 5 6
31227 204/300 Write block 2076 len 12288 data: 4 4 5
31228 205/300 Write block 1268 len 20480 data: 4 5 4 5 5
31229 206/300 Write block 2723 len 20480 data: 5 6 6 6 7
31230 207/300 Write block 4228 len 4096 data: 6
31231 208/300 Read block 1349 len 28672
31232 209/300 Flush
31233 210/300 Flush
31234 211/300 Read block 4007 len 20480
31235 212/300 Read block 623 len 32768
31236 213/300 Read block 1962 len 8192
31237 214/300 Write block 2360 len 32768 data: 6 4 4 7 7 7 6 6
31238 215/300 Write block 2495 len 8192 data: 3 3
31239 216/300 Flush
31240 217/300 Write block 1104 len 12288 data: 8 8 8
31241 218/300 Write block 4418 len 40960 data: 7 9 9 9 8 8 7 6 7 6
31242 219/300 Read block 2310 len 32768
31243 220/300 Read block 4015 len 28672
31244 221/300 Read block 1514 len 40960
31245 222/300 Read block 4115 len 32768
31246 223/300 Write block 1608 len 12288 data: 7 6 6
31247 224/300 Read block 3446 len 32768
31248 225/300 Write block 4465 len 28672 data: 11 9 8 9 9 10 10
31249 226/300 Read block 4852 len 40960
31250 227/300 Flush
31251 228/300 Read block 3636 len 4096
31252 229/300 Write block 3020 len 20480 data: 4 5 6 8 7
31253 230/300 Read block 423 len 16384
31254 231/300 Read block 4066 len 28672
31255 232/300 Flush
31256 233/300 Read block 3279 len 28672
31257 234/300 Read block 3149 len 4096
31258 235/300 Write block 3793 len 20480 data: 7 7 5 4 4
31259 236/300 Read block 4385 len 24576
31260 237/300 Read block 2611 len 36864
31261 238/300 Read block 103 len 24576
31262 239/300 Write block 4714 len 4096 data: 5
31263 240/300 Read block 1774 len 8192
31264 241/300 Flush
31265 242/300 Flush
31266 243/300 Read block 1047 len 32768
31267 244/300 Read block 4412 len 20480
31268 245/300 Read block 4752 len 16384
31269 246/300 Read block 985 len 32768
31270 247/300 Read block 3598 len 24576
31271 248/300 Read block 588 len 8192
31272 249/300 Write block 145 len 40960 data: 4 4 4 4 4 3 4 7 8 8
31273 250/300 Read block 3203 len 24576
31274 251/300 Read block 1653 len 8192
31275 252/300 Write block 2574 len 32768 data: 6 5 5 5 6 5 6 6
31276 253/300 Write block 860 len 8192 data: 6 6
31277 254/300 Write block 166 len 32768 data: 9 10 9 10 9 8 6 7
31278 255/300 Write block 3765 len 20480 data: 10 10 8 8 8
31279 256/300 Flush
31280 257/300 Write block 4293 len 24576 data: 5 4 4 6 6 7
31281 258/300 Read block 525 len 16384
31282 259/300 Read block 1361 len 40960
31283 260/300 Read block 4385 len 12288
31284 261/300 Read block 4035 len 28672
31285 262/300 Read block 2247 len 36864
31286 263/300 Write block 1011 len 12288 data: 5 7 7
31287 264/300 Write block 3990 len 40960 data: 5 6 6 6 5 6 5 5 5 4
31288 265/300 Read block 846 len 36864
31289 266/300 Write block 87 len 36864 data: 6 6 6 5 6 9 9 8 8
31290 267/300 Write block 3335 len 16384 data: 7 6 4 4
31291 268/300 Read block 4204 len 36864
31292 269/300 Read block 3564 len 36864
31293 270/300 Read block 4920 len 36864
31294 271/300 Read block 1628 len 36864
31295 272/300 Read block 558 len 36864
31296 273/300 Read block 4638 len 20480
31297 274/300 Read block 1431 len 40960
31298 275/300 Read block 3516 len 16384
31299 276/300 Read block 4156 len 16384
31300 277/300 Write block 3110 len 12288 data: 8 7 6
31301 278/300 Write block 4132 len 36864 data: 11 7 7 7 7 6 6 3 3
31302 279/300 Read block 2319 len 4096
31303 280/300 Write block 696 len 4096 data: 3
31304 281/300 Write block 1029 len 12288 data: 3 3 6
31305 282/300 Write block 4783 len 20480 data: 8 7 6 6 6
31306 283/300 Write block 1163 len 4096 data: 8
31307 284/300 Read block 4784 len 36864
31308 285/300 Write block 1816 len 4096 data: 6
31309 286/300 Read block 977 len 12288
31310 287/300 Flush
31311 288/300 Read block 740 len 20480
31312 289/300 Write block 696 len 16384 data: 4 3 4 4
31313 290/300 Write block 1250 len 16384 data: 6 5 4 4
31314 291/300 Write block 1111 len 28672 data: 7 9 11 12 11 11 10
31315 292/300 Write block 2345 len 4096 data: 6
31316 293/300 Read block 1847 len 40960
31317 294/300 Read block 645 len 4096
31318 295/300 Read block 4851 len 20480
31319 296/300 Write block 3775 len 16384 data: 9 9 8 8
31320 297/300 Write block 909 len 28672 data: 3 3 3 4 5 4 4
31321 298/300 Read block 471 len 4096
31322 299/300 Write block 2894 len 40960 data: 9 10 9 10 10 10 10 11 10 11
31323 300/300 Flush
31324 301/300 Write block 4905 len 28672 data: 9 8 8 9 11 8 5
31325 Main task has work to do, go find it
31326 got dsc Start(1)
31327 start 1
31328 Dec 04 13:37:17.488 INFO request completed, latency_us: 201, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7a85651b-aaee-4483-b7fc-cc3cfb992144, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
31329 Replay: started 1, returned:Ok(())
31330 ----------------------------------------------------------------
31331 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31332 States: Active Offline Active
31333 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31334 9754 Acked Read 6 Done Sent Done false
31335 9755 Acked Read 1 Done Sent Done false
31336 9756 Acked Read 5 Done Sent Done false
31337 9757 Acked Write 10 Done Sent Done false
31338 9758 Acked Flush 0 Done Sent Done false
31339 9759 Acked Write 8 Done Sent Done false
31340 9760 Acked Write 1 Done Sent Done false
31341 9761 Acked Read 7 Done Sent Done false
31342 9762 Acked Flush 0 Done Sent Done false
31343 9763 Acked Flush 0 Done Sent Done false
31344 9764 Acked Read 9 Done Sent Done false
31345 9765 Acked Write 9 Done Sent Done false
31346 9766 Acked Read 6 Done Sent Done false
31347 9767 Acked Write 9 Done Sent Done false
31348 9768 Acked Write 1 Done Sent Done false
31349 9769 Acked Read 7 Done Sent Done false
31350 9770 Acked Read 6 Done Sent Done false
31351 9771 Acked Write 6 Done Sent Done false
31352 9772 Acked Read 1 Done Sent Done false
31353 9773 Acked Write 9 Done Sent Done false
31354 9774 Acked Write 2 Done Sent Done false
31355 9775 Acked Read 7 Done Sent Done false
31356 9776 Acked Read 8 Done Sent Done false
31357 9777 Acked Read 8 Done Sent Done false
31358 9778 Acked Read 8 Done Sent Done false
31359 9779 Acked Read 2 Done Sent Done false
31360 9780 Acked Write 10 Done Sent Done false
31361 9781 Acked Read 10 Done Sent Done false
31362 9782 Acked Write 8 Done Sent Done false
31363 9783 Acked Read 8 Done Sent Done false
31364 9784 Acked Write 10 Done Sent Done false
31365 9785 Acked Write 9 Done Sent Done false
31366 9786 Acked Read 6 Done Sent Done false
31367 9787 Acked Flush 0 Done Sent Done false
31368 9788 Acked Read 10 Done Sent Done false
31369 9789 Acked Read 4 Done Sent Done false
31370 9790 Acked Write 4 Done Sent Done false
31371 9791 Acked Flush 0 Done Sent Done false
31372 9792 Acked Write 7 Done Sent Done false
31373 9793 Acked Read 4 Done Sent Done false
31374 9794 Acked Write 6 Done Sent Done false
31375 9795 Acked Write 1 Done Sent Done false
31376 9796 Acked Read 7 Done Sent Done false
31377 9797 Acked Read 10 Done Sent Done false
31378 9798 Acked Read 10 Done Sent Done false
31379 9799 Acked Read 9 Done Sent Done false
31380 9800 Acked Read 4 Done Sent Done false
31381 9801 Acked Read 4 Done Sent Done false
31382 9802 Acked Read 7 Done Sent Done false
31383 9803 Acked Write 5 Done Sent Done false
31384 9804 Acked Read 2 Done Sent Done false
31385 9805 Acked Write 8 Done Sent Done false
31386 9806 Acked Write 9 Done Sent Done false
31387 9807 Acked Write 10 Done Sent Done false
31388 9808 Acked Write 1 Done Sent Done false
31389 9809 Acked Write 2 Done Sent Done false
31390 9810 Acked Read 2 Done Sent Done false
31391 9811 Acked Flush 0 Done Sent Done false
31392 9812 Acked Read 2 Done Sent Done false
31393 9813 Acked Write 6 Done Sent Done false
31394 9814 Acked Flush 0 Done Sent Done false
31395 9815 Acked Flush 0 Done Sent Done false
31396 9816 Acked Read 10 Done Sent Done false
31397 9817 Acked Write 10 Done Sent Done false
31398 9818 Acked Write 4 Done Sent Done false
31399 9819 Acked Write 3 Done Sent Done false
31400 9820 Acked Write 5 Done Sent Done false
31401 9821 Acked Write 5 Done Sent Done false
31402 9822 Acked Read 8 Done Sent Done false
31403 9823 Acked Write 9 Done Sent Done false
31404 9824 Acked Write 7 Done Sent Done false
31405 9825 Acked Read 8 Done Sent Done false
31406 9826 Acked Read 1 Done Sent Done false
31407 9827 Acked Write 2 Done Sent Done false
31408 9828 Acked Write 2 Done Sent Done false
31409 9829 Acked Read 4 Done Sent Done false
31410 9830 Acked Read 7 Done Sent Done false
31411 9831 Acked Read 3 Done Sent Done false
31412 9832 Acked Write 3 Done Sent Done false
31413 9833 Acked Write 8 Done Sent Done false
31414 9834 Acked Write 9 Done Sent Done false
31415 9835 Acked Read 10 Done Sent Done false
31416 9836 Acked Read 6 Done Sent Done false
31417 9837 Acked Read 6 Done Sent Done false
31418 9838 Acked Write 9 Done Sent Done false
31419 9839 Acked Read 9 Done Sent Done false
31420 9840 Acked Read 7 Done Sent Done false
31421 9841 Acked Read 9 Done Sent Done false
31422 9842 Acked Write 5 Done Sent Done false
31423 9843 Acked Read 9 Done Sent Done false
31424 9844 Acked Read 6 Done Sent Done false
31425 9845 Acked Read 9 Done Sent Done false
31426 9846 Acked Write 5 Done Sent Done false
31427 9847 Acked Write 9 Done Sent Done false
31428 9848 Acked Read 1 Done Sent Done false
31429 9849 Acked Read 10 Done Sent Done false
31430 9850 Acked Read 5 Done Sent Done false
31431 9851 Acked Read 3 Done Sent Done false
31432 9852 Acked Read 2 Done Sent Done false
31433 9853 Acked Write 1 Done Sent Done false
31434 9854 Acked Write 6 Done Sent Done false
31435 9855 Acked Write 1 Done Sent Done false
31436 9856 Acked Read 9 Done Sent Done false
31437 9857 Acked Write 1 Done Sent Done false
31438 9858 Acked Read 5 Done Sent Done false
31439 9859 Acked Write 2 Done Sent Done false
31440 9860 Acked Write 4 Done Sent Done false
31441 9861 Acked Read 5 Done Sent Done false
31442 9862 Acked Read 5 Done Sent Done false
31443 9863 Acked Write 7 Done Sent Done false
31444 9864 Acked Read 3 Done Sent Done false
31445 9865 Acked Write 8 Done Sent Done false
31446 9866 Acked Read 4 Done Sent Done false
31447 9867 Acked Flush 0 Done Sent Done false
31448 9868 Acked Write 9 Done Sent Done false
31449 9869 Acked Read 9 Done Sent Done false
31450 9870 Acked Read 5 Done Sent Done false
31451 9871 Acked Read 8 Done Sent Done false
31452 9872 Acked Read 6 Done Sent Done false
31453 9873 Acked Read 8 Done Sent Done false
31454 9874 Acked Read 8 Done Sent Done false
31455 9875 Acked Read 2 Done Sent Done false
31456 9876 Acked Read 4 Done Sent Done false
31457 9877 Acked Write 3 Done Sent Done false
31458 9878 Acked Read 6 Done Sent Done false
31459 9879 Acked Read 4 Done Sent Done false
31460 9880 Acked Read 2 Done Sent Done false
31461 9881 Acked Read 5 Done Sent Done false
31462 9882 Acked Write 2 Done Sent Done false
31463 9883 Acked Read 8 Done Sent Done false
31464 9884 Acked Write 7 Done Sent Done false
31465 9885 Acked Write 9 Done Sent Done false
31466 9886 Acked Write 10 Done Sent Done false
31467 9887 Acked Read 3 Done Sent Done false
31468 9888 Acked Read 4 Done Sent Done false
31469 9889 Acked Write 1 Done Sent Done false
31470 9890 Acked Read 2 Done Sent Done false
31471 9891 Acked Write 8 Done Sent Done false
31472 9892 Acked Write 4 Done Sent Done false
31473 9893 Acked Write 4 Done Sent Done false
31474 9894 Acked Read 8 Done Sent Done false
31475 9895 Acked Read 8 Done Sent Done false
31476 9896 Acked Write 4 Done Sent Done false
31477 9897 Acked Write 10 Done Sent Done false
31478 9898 Acked Read 10 Done Sent Done false
31479 9899 Acked Read 3 Done Sent Done false
31480 9900 Acked Read 9 Done Sent Done false
31481 9901 Acked Read 5 Done Sent Done false
31482 9902 Acked Write 7 Done Sent Done false
31483 9903 Acked Write 3 Done Sent Done false
31484 9904 Acked Read 1 Done Sent Done false
31485 9905 Acked Read 9 Done Sent Done false
31486 9906 Acked Read 10 Done Sent Done false
31487 9907 Acked Write 1 Done Sent Done false
31488 9908 Acked Write 6 Done Sent Done false
31489 9909 Acked Read 4 Done Sent Done false
31490 9910 Acked Read 5 Done Sent Done false
31491 9911 Acked Read 8 Done Sent Done false
31492 9912 Acked Read 7 Done Sent Done false
31493 9913 Acked Read 10 Done Sent Done false
31494 9914 Acked Read 7 Done Sent Done false
31495 9915 Acked Write 7 Done Sent Done false
31496 9916 Acked Flush 0 Done Sent Done false
31497 9917 Acked Read 8 Done Sent Done false
31498 9918 Acked Read 10 Done Sent Done false
31499 9919 Acked Read 8 Done Sent Done false
31500 9920 Acked Write 7 Done Sent Done false
31501 9921 Acked Write 4 Done Sent Done false
31502 9922 Acked Write 8 Done Sent Done false
31503 9923 Acked Write 9 Done Sent Done false
31504 9924 Acked Read 5 Done Sent Done false
31505 9925 Acked Write 3 Done Sent Done false
31506 9926 Acked Read 4 Done Sent Done false
31507 9927 Acked Read 9 Done Sent Done false
31508 9928 Acked Read 1 Done Sent Done false
31509 9929 Acked Write 9 Done Sent Done false
31510 9930 Acked Flush 0 Done Sent Done false
31511 9931 Acked Read 3 Done Sent Done false
31512 9932 Acked Read 5 Done Sent Done false
31513 9933 Acked Write 1 Done Sent Done false
31514 9934 Acked Write 10 Done Sent Done false
31515 9935 Acked Write 4 Done Sent Done false
31516 9936 Acked Read 4 Done Sent Done false
31517 9937 Acked Read 10 Done Sent Done false
31518 9938 Acked Write 8 Done Sent Done false
31519 9939 Acked Flush 0 Done Sent Done false
31520 9940 Acked Flush 0 Done Sent Done false
31521 9941 Acked Write 7 Done Sent Done false
31522 9942 Acked Write 6 Done Sent Done false
31523 9943 Acked Read 7 Done Sent Done false
31524 9944 Acked Write 5 Done Sent Done false
31525 9945 Acked Write 6 Done Sent Done false
31526 9946 Acked Write 6 Done Sent Done false
31527 9947 Acked Write 1 Done Sent Done false
31528 9948 Acked Read 5 Done Sent Done false
31529 9949 Acked Read 6 Done Sent Done false
31530 9950 Acked Read 1 Done Sent Done false
31531 9951 Acked Write 9 Done Sent Done false
31532 9952 Acked Flush 0 Done Sent Done false
31533 9953 Acked Flush 0 Done Sent Done false
31534 9954 Acked Read 9 Done Sent Done false
31535 9955 Acked Flush 0 Done Sent Done false
31536 9956 Acked Write 9 Done Sent Done false
31537 9957 Acked Write 3 Done Sent Done false
31538 9958 Acked Write 5 Done Sent Done false
31539 9959 Acked Write 5 Done Sent Done false
31540 9960 Acked Write 1 Done Sent Done false
31541 9961 Acked Read 7 Done Sent Done false
31542 9962 Acked Flush 0 Done Sent Done false
31543 9963 Acked Flush 0 Done Sent Done false
31544 9964 Acked Read 5 Done Sent Done false
31545 9965 Acked Read 8 Done Sent Done false
31546 9966 Acked Read 2 Done Sent Done false
31547 9967 Acked Write 8 Done Sent Done false
31548 9968 Acked Write 2 Done Sent Done false
31549 9969 Acked Flush 0 Done Sent Done false
31550 9970 Acked Write 3 Done Sent Done false
31551 9971 Acked Write 10 Done Sent Done false
31552 9972 Acked Read 8 Done Sent Done false
31553 9973 Acked Read 7 Done Sent Done false
31554 9974 Acked Read 10 Done Sent Done false
31555 9975 Acked Read 8 Done Sent Done false
31556 9976 Acked Write 3 Done Sent Done false
31557 9977 Acked Read 8 Done Sent Done false
31558 9978 Acked Write 7 Done Sent Done false
31559 9979 Acked Read 10 Done Sent Done false
31560 9980 Acked Flush 0 Done Sent Done false
31561 9981 Acked Read 1 Done Sent Done false
31562 9982 Acked Write 5 Done Sent Done false
31563 9983 Acked Read 4 Done Sent Done false
31564 9984 Acked Read 7 Done Sent Done false
31565 9985 Acked Flush 0 Done Sent Done false
31566 9986 Acked Read 7 Done Sent Done false
31567 9987 Acked Read 1 Done Sent Done false
31568 9988 Acked Write 5 Done Sent Done false
31569 9989 Acked Read 6 Done Sent Done false
31570 9990 Acked Read 9 Done Sent Done false
31571 9991 Acked Read 6 Done Sent Done false
31572 9992 Acked Write 1 Done Sent Done false
31573 9993 Acked Read 2 Done Sent Done false
31574 9994 Acked Flush 0 Done Sent Done false
31575 9995 Acked Flush 0 Done Sent Done false
31576 9996 Acked Read 8 Done Sent Done false
31577 9997 Acked Read 5 Done Sent Done false
31578 9998 Acked Read 4 Done Sent Done false
31579 9999 Acked Read 8 Done Sent Done false
31580 10000 Acked Read 6 Done Sent Done false
31581 10001 Acked Read 2 Done Sent Done false
31582 10002 Acked Write 10 Done Sent Done false
31583 10003 Acked Read 6 Done Sent Done false
31584 10004 Acked Read 2 Done Sent Done false
31585 10005 Acked Write 8 Done Sent Done false
31586 10006 Acked Write 2 Done Sent Done false
31587 10007 Acked Write 8 Done Sent Done false
31588 10008 Acked Write 5 Done Sent Done false
31589 10009 Acked Flush 0 Done Sent Done false
31590 10010 Acked Write 6 Done Sent Done false
31591 10011 Acked Read 4 Done Sent Done false
31592 10012 Acked Read 10 Done Sent Done false
31593 10013 Acked Read 3 Done Sent Done false
31594 10014 Acked Read 7 Done Sent Done false
31595 10015 Acked Read 9 Done Sent Done false
31596 10016 Acked Write 3 Done Sent Done false
31597 10017 Acked Write 10 Done Sent Done false
31598 10018 Acked Read 9 Done Sent Done false
31599 10019 Acked Write 9 Done Sent Done false
31600 10020 Acked Write 4 Done Sent Done false
31601 10021 Acked Read 9 Done Sent Done false
31602 10022 Acked Read 9 Done Sent Done false
31603 10023 Acked Read 9 Done Sent Done false
31604 10024 Acked Read 9 Done Sent Done false
31605 10025 Acked Read 9 Done Sent Done false
31606 10026 Acked Read 5 Done Sent Done false
31607 10027 Acked Read 10 Done Sent Done false
31608 10028 Acked Read 4 Done Sent Done false
31609 10029 Acked Read 4 Done Sent Done false
31610 10030 Acked Write 3 Done Sent Done false
31611 10031 Acked Write 9 Done Sent Done false
31612 10032 Acked Read 1 Done Sent Done false
31613 10033 Acked Write 1 Done Sent Done false
31614 10034 Acked Write 3 Done Sent Done false
31615 10035 Acked Write 5 Done Sent Done false
31616 10036 Acked Write 1 Done Sent Done false
31617 10037 Acked Read 9 Done Sent Done false
31618 10038 Acked Write 1 Done Sent Done false
31619 10039 Acked Read 3 Done Sent Done false
31620 10040 Acked Flush 0 Done Sent Done false
31621 10041 Acked Read 5 Done Sent Done false
31622 10042 Acked Write 4 Done Sent Done false
31623 10043 Acked Write 4 Done Sent Done false
31624 10044 Acked Write 7 Done Sent Done false
31625 10045 Acked Write 1 Done Sent Done false
31626 10046 Acked Read 10 Done Sent Done false
31627 10047 Acked Read 1 Done Sent Done false
31628 10048 Acked Read 5 Done Sent Done false
31629 10049 Acked Write 4 Done Sent Done false
31630 10050 Acked Write 7 Done Sent Done false
31631 10051 Acked Read 1 Done Sent Done false
31632 10052 Acked Write 10 Done Sent Done false
31633 10053 Acked Flush 0 Done Sent Done false
31634 10054 Acked Write 7 Done Sent Done false
31635 STATES DS:0 DS:1 DS:2 TOTAL
31636 Sent 0 301 0 301
31637 Done 301 0 301 602
31638 Skipped 0 0 0 0
31639 Error 0 0 0 0
31640 Last Flush: 10053 9753 10053
31641 Downstairs last five completed: 9753 9752 9751 9750 9749
31642 Upstairs last five completed: 10054 10053 10052 10051 10050
31643 CLIENT: Up:0 ds:301 act:2
31644 [8820] got start action
31645 [8820] I am going to restart
31646 Starting downstairs at port 8820
31647 Make output file at "/tmp/dsc/downstairs-8820.txt"
31648 [8820][1] reports Starting
31649 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1233)
31650 [8820][1] reports Running
31651 ----------------------------------------------------------------
31652 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31653 States: Active Offline Active
31654 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31655 9754 Acked Read 6 Done Sent Done false
31656 9755 Acked Read 1 Done Sent Done false
31657 9756 Acked Read 5 Done Sent Done false
31658 9757 Acked Write 10 Done Sent Done false
31659 9758 Acked Flush 0 Done Sent Done false
31660 9759 Acked Write 8 Done Sent Done false
31661 9760 Acked Write 1 Done Sent Done false
31662 9761 Acked Read 7 Done Sent Done false
31663 9762 Acked Flush 0 Done Sent Done false
31664 9763 Acked Flush 0 Done Sent Done false
31665 9764 Acked Read 9 Done Sent Done false
31666 9765 Acked Write 9 Done Sent Done false
31667 9766 Acked Read 6 Done Sent Done false
31668 9767 Acked Write 9 Done Sent Done false
31669 9768 Acked Write 1 Done Sent Done false
31670 9769 Acked Read 7 Done Sent Done false
31671 9770 Acked Read 6 Done Sent Done false
31672 9771 Acked Write 6 Done Sent Done false
31673 9772 Acked Read 1 Done Sent Done false
31674 9773 Acked Write 9 Done Sent Done false
31675 9774 Acked Write 2 Done Sent Done false
31676 9775 Acked Read 7 Done Sent Done false
31677 9776 Acked Read 8 Done Sent Done false
31678 9777 Acked Read 8 Done Sent Done false
31679 9778 Acked Read 8 Done Sent Done false
31680 9779 Acked Read 2 Done Sent Done false
31681 9780 Acked Write 10 Done Sent Done false
31682 9781 Acked Read 10 Done Sent Done false
31683 9782 Acked Write 8 Done Sent Done false
31684 9783 Acked Read 8 Done Sent Done false
31685 9784 Acked Write 10 Done Sent Done false
31686 9785 Acked Write 9 Done Sent Done false
31687 9786 Acked Read 6 Done Sent Done false
31688 9787 Acked Flush 0 Done Sent Done false
31689 9788 Acked Read 10 Done Sent Done false
31690 9789 Acked Read 4 Done Sent Done false
31691 9790 Acked Write 4 Done Sent Done false
31692 9791 Acked Flush 0 Done Sent Done false
31693 9792 Acked Write 7 Done Sent Done false
31694 9793 Acked Read 4 Done Sent Done false
31695 9794 Acked Write 6 Done Sent Done false
31696 9795 Acked Write 1 Done Sent Done false
31697 9796 Acked Read 7 Done Sent Done false
31698 9797 Acked Read 10 Done Sent Done false
31699 9798 Acked Read 10 Done Sent Done false
31700 9799 Acked Read 9 Done Sent Done false
31701 9800 Acked Read 4 Done Sent Done false
31702 9801 Acked Read 4 Done Sent Done false
31703 9802 Acked Read 7 Done Sent Done false
31704 9803 Acked Write 5 Done Sent Done false
31705 9804 Acked Read 2 Done Sent Done false
31706 9805 Acked Write 8 Done Sent Done false
31707 9806 Acked Write 9 Done Sent Done false
31708 9807 Acked Write 10 Done Sent Done false
31709 9808 Acked Write 1 Done Sent Done false
31710 9809 Acked Write 2 Done Sent Done false
31711 9810 Acked Read 2 Done Sent Done false
31712 9811 Acked Flush 0 Done Sent Done false
31713 9812 Acked Read 2 Done Sent Done false
31714 9813 Acked Write 6 Done Sent Done false
31715 9814 Acked Flush 0 Done Sent Done false
31716 9815 Acked Flush 0 Done Sent Done false
31717 9816 Acked Read 10 Done Sent Done false
31718 9817 Acked Write 10 Done Sent Done false
31719 9818 Acked Write 4 Done Sent Done false
31720 9819 Acked Write 3 Done Sent Done false
31721 9820 Acked Write 5 Done Sent Done false
31722 9821 Acked Write 5 Done Sent Done false
31723 9822 Acked Read 8 Done Sent Done false
31724 9823 Acked Write 9 Done Sent Done false
31725 9824 Acked Write 7 Done Sent Done false
31726 9825 Acked Read 8 Done Sent Done false
31727 9826 Acked Read 1 Done Sent Done false
31728 9827 Acked Write 2 Done Sent Done false
31729 9828 Acked Write 2 Done Sent Done false
31730 9829 Acked Read 4 Done Sent Done false
31731 9830 Acked Read 7 Done Sent Done false
31732 9831 Acked Read 3 Done Sent Done false
31733 9832 Acked Write 3 Done Sent Done false
31734 9833 Acked Write 8 Done Sent Done false
31735 9834 Acked Write 9 Done Sent Done false
31736 9835 Acked Read 10 Done Sent Done false
31737 9836 Acked Read 6 Done Sent Done false
31738 9837 Acked Read 6 Done Sent Done false
31739 9838 Acked Write 9 Done Sent Done false
31740 9839 Acked Read 9 Done Sent Done false
31741 9840 Acked Read 7 Done Sent Done false
31742 9841 Acked Read 9 Done Sent Done false
31743 9842 Acked Write 5 Done Sent Done false
31744 9843 Acked Read 9 Done Sent Done false
31745 9844 Acked Read 6 Done Sent Done false
31746 9845 Acked Read 9 Done Sent Done false
31747 9846 Acked Write 5 Done Sent Done false
31748 9847 Acked Write 9 Done Sent Done false
31749 9848 Acked Read 1 Done Sent Done false
31750 9849 Acked Read 10 Done Sent Done false
31751 9850 Acked Read 5 Done Sent Done false
31752 9851 Acked Read 3 Done Sent Done false
31753 9852 Acked Read 2 Done Sent Done false
31754 9853 Acked Write 1 Done Sent Done false
31755 9854 Acked Write 6 Done Sent Done false
31756 9855 Acked Write 1 Done Sent Done false
31757 9856 Acked Read 9 Done Sent Done false
31758 9857 Acked Write 1 Done Sent Done false
31759 9858 Acked Read 5 Done Sent Done false
31760 9859 Acked Write 2 Done Sent Done false
31761 9860 Acked Write 4 Done Sent Done false
31762 9861 Acked Read 5 Done Sent Done false
31763 9862 Acked Read 5 Done Sent Done false
31764 9863 Acked Write 7 Done Sent Done false
31765 9864 Acked Read 3 Done Sent Done false
31766 9865 Acked Write 8 Done Sent Done false
31767 9866 Acked Read 4 Done Sent Done false
31768 9867 Acked Flush 0 Done Sent Done false
31769 9868 Acked Write 9 Done Sent Done false
31770 9869 Acked Read 9 Done Sent Done false
31771 9870 Acked Read 5 Done Sent Done false
31772 9871 Acked Read 8 Done Sent Done false
31773 9872 Acked Read 6 Done Sent Done false
31774 9873 Acked Read 8 Done Sent Done false
31775 9874 Acked Read 8 Done Sent Done false
31776 9875 Acked Read 2 Done Sent Done false
31777 9876 Acked Read 4 Done Sent Done false
31778 9877 Acked Write 3 Done Sent Done false
31779 9878 Acked Read 6 Done Sent Done false
31780 9879 Acked Read 4 Done Sent Done false
31781 9880 Acked Read 2 Done Sent Done false
31782 9881 Acked Read 5 Done Sent Done false
31783 9882 Acked Write 2 Done Sent Done false
31784 9883 Acked Read 8 Done Sent Done false
31785 9884 Acked Write 7 Done Sent Done false
31786 9885 Acked Write 9 Done Sent Done false
31787 9886 Acked Write 10 Done Sent Done false
31788 9887 Acked Read 3 Done Sent Done false
31789 9888 Acked Read 4 Done Sent Done false
31790 9889 Acked Write 1 Done Sent Done false
31791 9890 Acked Read 2 Done Sent Done false
31792 9891 Acked Write 8 Done Sent Done false
31793 9892 Acked Write 4 Done Sent Done false
31794 9893 Acked Write 4 Done Sent Done false
31795 9894 Acked Read 8 Done Sent Done false
31796 9895 Acked Read 8 Done Sent Done false
31797 9896 Acked Write 4 Done Sent Done false
31798 9897 Acked Write 10 Done Sent Done false
31799 9898 Acked Read 10 Done Sent Done false
31800 9899 Acked Read 3 Done Sent Done false
31801 9900 Acked Read 9 Done Sent Done false
31802 9901 Acked Read 5 Done Sent Done false
31803 9902 Acked Write 7 Done Sent Done false
31804 9903 Acked Write 3 Done Sent Done false
31805 9904 Acked Read 1 Done Sent Done false
31806 9905 Acked Read 9 Done Sent Done false
31807 9906 Acked Read 10 Done Sent Done false
31808 9907 Acked Write 1 Done Sent Done false
31809 9908 Acked Write 6 Done Sent Done false
31810 9909 Acked Read 4 Done Sent Done false
31811 9910 Acked Read 5 Done Sent Done false
31812 9911 Acked Read 8 Done Sent Done false
31813 9912 Acked Read 7 Done Sent Done false
31814 9913 Acked Read 10 Done Sent Done false
31815 9914 Acked Read 7 Done Sent Done false
31816 9915 Acked Write 7 Done Sent Done false
31817 9916 Acked Flush 0 Done Sent Done false
31818 9917 Acked Read 8 Done Sent Done false
31819 9918 Acked Read 10 Done Sent Done false
31820 9919 Acked Read 8 Done Sent Done false
31821 9920 Acked Write 7 Done Sent Done false
31822 9921 Acked Write 4 Done Sent Done false
31823 9922 Acked Write 8 Done Sent Done false
31824 9923 Acked Write 9 Done Sent Done false
31825 9924 Acked Read 5 Done Sent Done false
31826 9925 Acked Write 3 Done Sent Done false
31827 9926 Acked Read 4 Done Sent Done false
31828 9927 Acked Read 9 Done Sent Done false
31829 9928 Acked Read 1 Done Sent Done false
31830 9929 Acked Write 9 Done Sent Done false
31831 9930 Acked Flush 0 Done Sent Done false
31832 9931 Acked Read 3 Done Sent Done false
31833 9932 Acked Read 5 Done Sent Done false
31834 9933 Acked Write 1 Done Sent Done false
31835 9934 Acked Write 10 Done Sent Done false
31836 9935 Acked Write 4 Done Sent Done false
31837 9936 Acked Read 4 Done Sent Done false
31838 9937 Acked Read 10 Done Sent Done false
31839 9938 Acked Write 8 Done Sent Done false
31840 9939 Acked Flush 0 Done Sent Done false
31841 9940 Acked Flush 0 Done Sent Done false
31842 9941 Acked Write 7 Done Sent Done false
31843 9942 Acked Write 6 Done Sent Done false
31844 9943 Acked Read 7 Done Sent Done false
31845 9944 Acked Write 5 Done Sent Done false
31846 9945 Acked Write 6 Done Sent Done false
31847 9946 Acked Write 6 Done Sent Done false
31848 9947 Acked Write 1 Done Sent Done false
31849 9948 Acked Read 5 Done Sent Done false
31850 9949 Acked Read 6 Done Sent Done false
31851 9950 Acked Read 1 Done Sent Done false
31852 9951 Acked Write 9 Done Sent Done false
31853 9952 Acked Flush 0 Done Sent Done false
31854 9953 Acked Flush 0 Done Sent Done false
31855 9954 Acked Read 9 Done Sent Done false
31856 9955 Acked Flush 0 Done Sent Done false
31857 9956 Acked Write 9 Done Sent Done false
31858 9957 Acked Write 3 Done Sent Done false
31859 9958 Acked Write 5 Done Sent Done false
31860 9959 Acked Write 5 Done Sent Done false
31861 9960 Acked Write 1 Done Sent Done false
31862 9961 Acked Read 7 Done Sent Done false
31863 9962 Acked Flush 0 Done Sent Done false
31864 9963 Acked Flush 0 Done Sent Done false
31865 9964 Acked Read 5 Done Sent Done false
31866 9965 Acked Read 8 Done Sent Done false
31867 9966 Acked Read 2 Done Sent Done false
31868 9967 Acked Write 8 Done Sent Done false
31869 9968 Acked Write 2 Done Sent Done false
31870 9969 Acked Flush 0 Done Sent Done false
31871 9970 Acked Write 3 Done Sent Done false
31872 9971 Acked Write 10 Done Sent Done false
31873 9972 Acked Read 8 Done Sent Done false
31874 9973 Acked Read 7 Done Sent Done false
31875 9974 Acked Read 10 Done Sent Done false
31876 9975 Acked Read 8 Done Sent Done false
31877 9976 Acked Write 3 Done Sent Done false
31878 9977 Acked Read 8 Done Sent Done false
31879 9978 Acked Write 7 Done Sent Done false
31880 9979 Acked Read 10 Done Sent Done false
31881 9980 Acked Flush 0 Done Sent Done false
31882 9981 Acked Read 1 Done Sent Done false
31883 9982 Acked Write 5 Done Sent Done false
31884 9983 Acked Read 4 Done Sent Done false
31885 9984 Acked Read 7 Done Sent Done false
31886 9985 Acked Flush 0 Done Sent Done false
31887 9986 Acked Read 7 Done Sent Done false
31888 9987 Acked Read 1 Done Sent Done false
31889 9988 Acked Write 5 Done Sent Done false
31890 9989 Acked Read 6 Done Sent Done false
31891 9990 Acked Read 9 Done Sent Done false
31892 9991 Acked Read 6 Done Sent Done false
31893 9992 Acked Write 1 Done Sent Done false
31894 9993 Acked Read 2 Done Sent Done false
31895 9994 Acked Flush 0 Done Sent Done false
31896 9995 Acked Flush 0 Done Sent Done false
31897 9996 Acked Read 8 Done Sent Done false
31898 9997 Acked Read 5 Done Sent Done false
31899 9998 Acked Read 4 Done Sent Done false
31900 9999 Acked Read 8 Done Sent Done false
31901 10000 Acked Read 6 Done Sent Done false
31902 10001 Acked Read 2 Done Sent Done false
31903 10002 Acked Write 10 Done Sent Done false
31904 10003 Acked Read 6 Done Sent Done false
31905 10004 Acked Read 2 Done Sent Done false
31906 10005 Acked Write 8 Done Sent Done false
31907 10006 Acked Write 2 Done Sent Done false
31908 10007 Acked Write 8 Done Sent Done false
31909 10008 Acked Write 5 Done Sent Done false
31910 10009 Acked Flush 0 Done Sent Done false
31911 10010 Acked Write 6 Done Sent Done false
31912 10011 Acked Read 4 Done Sent Done false
31913 10012 Acked Read 10 Done Sent Done false
31914 10013 Acked Read 3 Done Sent Done false
31915 10014 Acked Read 7 Done Sent Done false
31916 10015 Acked Read 9 Done Sent Done false
31917 10016 Acked Write 3 Done Sent Done false
31918 10017 Acked Write 10 Done Sent Done false
31919 10018 Acked Read 9 Done Sent Done false
31920 10019 Acked Write 9 Done Sent Done false
31921 10020 Acked Write 4 Done Sent Done false
31922 10021 Acked Read 9 Done Sent Done false
31923 10022 Acked Read 9 Done Sent Done false
31924 10023 Acked Read 9 Done Sent Done false
31925 10024 Acked Read 9 Done Sent Done false
31926 10025 Acked Read 9 Done Sent Done false
31927 10026 Acked Read 5 Done Sent Done false
31928 10027 Acked Read 10 Done Sent Done false
31929 10028 Acked Read 4 Done Sent Done false
31930 10029 Acked Read 4 Done Sent Done false
31931 10030 Acked Write 3 Done Sent Done false
31932 10031 Acked Write 9 Done Sent Done false
31933 10032 Acked Read 1 Done Sent Done false
31934 10033 Acked Write 1 Done Sent Done false
31935 10034 Acked Write 3 Done Sent Done false
31936 10035 Acked Write 5 Done Sent Done false
31937 10036 Acked Write 1 Done Sent Done false
31938 10037 Acked Read 9 Done Sent Done false
31939 10038 Acked Write 1 Done Sent Done false
31940 10039 Acked Read 3 Done Sent Done false
31941 10040 Acked Flush 0 Done Sent Done false
31942 10041 Acked Read 5 Done Sent Done false
31943 10042 Acked Write 4 Done Sent Done false
31944 10043 Acked Write 4 Done Sent Done false
31945 10044 Acked Write 7 Done Sent Done false
31946 10045 Acked Write 1 Done Sent Done false
31947 10046 Acked Read 10 Done Sent Done false
31948 10047 Acked Read 1 Done Sent Done false
31949 10048 Acked Read 5 Done Sent Done false
31950 10049 Acked Write 4 Done Sent Done false
31951 10050 Acked Write 7 Done Sent Done false
31952 10051 Acked Read 1 Done Sent Done false
31953 10052 Acked Write 10 Done Sent Done false
31954 10053 Acked Flush 0 Done Sent Done false
31955 10054 Acked Write 7 Done Sent Done false
31956 10055 Acked Flush 0 Done Sent Done false
31957 STATES DS:0 DS:1 DS:2 TOTAL
31958 Sent 0 302 0 302
31959 Done 302 0 302 604
31960 Skipped 0 0 0 0
31961 Error 0 0 0 0
31962 Last Flush: 10055 9753 10055
31963 Downstairs last five completed: 9753 9752 9751 9750 9749
31964 Upstairs last five completed: 10055 10054 10053 10052 10051
31965 CLIENT: Up:0 ds:302 act:2
319662024-12-04T13:37:23.241ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319672024-12-04T13:37:23.241ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40181) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319682024-12-04T13:37:23.242ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319692024-12-04T13:37:23.242ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319702024-12-04T13:37:23.242ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319712024-12-04T13:37:23.242ZINFOcrucible: send last flush ID to this DS: 9753 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319722024-12-04T13:37:23.243ZINFOcrucible: Replied this last flush ID: 9753 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319732024-12-04T13:37:23.243ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319742024-12-04T13:37:23.243ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319752024-12-04T13:37:23.243ZINFOcrucible: [1] client re-new 302 jobs since flush 9753 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319762024-12-04T13:37:23.243ZINFOcrucible: [1] Replayed 302 jobs since flush: 9753 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31977 ----------------------------------------------------------------
31978 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31979 Downstairs last five completed: 10055 10054 10053 10052 10051
31980 Upstairs last five completed: 10055 10054 10053 10052 10051
31981 CLIENT: Up:0 ds:0 act:3
31982 Replay: All jobs finished
31983 Main task has work to do, go find it
31984 got dsc Stop(1)
31985 stop 1
31986 Dec 04 13:37:25.493 INFO request completed, latency_us[8820] Got stop action so:false kr:false
31987 : 356, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 640eff53-58e6-4828-acf0-4d1fc92d9cd9, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
319882024-12-04T13:37:25.494ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
319892024-12-04T13:37:25.494ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31990 Dec 04 13:37:25.494 INFO request completed, latency_us: 127, response_code: 200, uri{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2024-12-04T13:37:25.494616526Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
31991 : /state/cid/1, method: GET, req_id: e7799b04-d11c-4b93-aca7-375ca028b523, remote_addr: 127.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:37:25.494647296Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
31992 0.0.1:37804, local_addr: 127.0.0.{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:37:25.494694556Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1203,"client":"1","":"downstairs","session_id":"01834428-904b-4e36-8fe6-2e6a94afb6ff"}
31993 1:9998
319942024-12-04T13:37:25.494ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
31995 [8820] Exited with: ExitStatus(unix_wait_status(9))
31996 [8820][1] reports Exit
31997 Dec 04 13:37:29.496 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/1, method: GET, req_id: a3a91b89-2b7d-4998-99d9-e364fb8ef0e1, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
31998 001/300 Write block 3767 len 8192 data: 9 9
31999 002/300 Read block 181 len 16384
32000 003/300 Read block 4199 len 32768
32001 004/300 Read block 1137 len 36864
32002 005/300 Read block 3435 len 20480
32003 006/300 Read block 4092 len 40960
32004 007/300 Read block 960 len 28672
32005 008/300 Read block 2809 len 20480
32006 009/300 Read block 1448 len 40960
32007 010/300 Read block 798 len 36864
32008 011/300 Read block 179 len 8192
32009 012/300 Read block 1524 len 4096
32010 013/300 Write block 510 len 24576 data: 8 9 6 6 7 7
32011 014/300 Read block 1766 len 32768
32012 015/300 Read block 2170 len 20480
32013 016/300 Read block 547 len 24576
32014 017/300 Read block 3269 len 16384
32015 018/300 Write block 345 len 4096 data: 6
32016 019/300 Read block 1451 len 4096
32017 020/300 Write block 403 len 40960 data: 7 8 8 7 5 5 6 7 6 7
32018 021/300 Flush
32019 022/300 Read block 2097 len 28672
32020 023/300 Read block 4586 len 40960
32021 024/300 Write block 4142 len 12288 data: 3 5 5
32022 025/300 Write block 3945 len 24576 data: 9 8 8 9 8 9
32023 026/300 Read block 946 len 24576
32024 027/300 Flush
32025 028/300 Read block 4374 len 8192
32026 029/300 Read block 1137 len 20480
32027 030/300 Read block 3271 len 16384
32028 031/300 Write block 3972 len 4096 data: 7
32029 032/300 Flush
32030 033/300 Write block 1206 len 4096 data: 8
32031 034/300 Write block 644 len 12288 data: 9 9 9
32032 035/300 Flush
32033 036/300 Read block 4619 len 24576
32034 037/300 Write block 3613 len 16384 data: 6 6 6 5
32035 038/300 Read block 1928 len 36864
32036 039/300 Write block 4013 len 32768 data: 5 4 5 5 6 6 6 6
32037 040/300 Read block 2174 len 32768
32038 041/300 Read block 879 len 40960
32039 042/300 Write block 454 len 36864 data: 7 8 9 9 10 5 3 4 3
32040 043/300 Read block 1617 len 20480
32041 044/300 Read block 4153 len 12288
32042 045/300 Read block 3400 len 36864
32043 046/300 Write block 4828 len 20480 data: 7 6 6 6 6
32044 047/300 Flush
32045 048/300 Write block 1825 len 24576 data: 3 4 5 6 7 6
32046 049/300 Read block 2157 len 16384
32047 050/300 Read block 2243 len 20480
32048 051/300 Flush
32049 052/300 Write block 3182 len 8192 data: 6 5
32050 053/300 Write block 3897 len 4096 data: 6
32051 054/300 Write block 3361 len 32768 data: 6 6 7 7 8 8 8 9
32052 055/300 Flush
32053 056/300 Write block 4301 len 16384 data: 6 4 5 5
32054 057/300 Write block 3144 len 40960 data: 7 6 7 7 8 9 8 9 9 11
32055 058/300 Read block 4078 len 12288
32056 059/300 Read block 117 len 8192
32057 060/300 Read block 3259 len 36864
32058 061/300 Write block 3806 len 16384 data: 6 5 7 9
32059 062/300 Flush
32060 063/300 Flush
32061 064/300 Read block 3776 len 12288
32062 065/300 Read block 1372 len 24576
32063 066/300 Read block 1649 len 8192
32064 067/300 Write block 2476 len 4096 data: 4
32065 068/300 Write block 3901 len 16384 data: 7 7 8 7
32066 069/300 Read block 12 len 16384
32067 070/300 Read block 914 len 8192
32068 071/300 Read block 1626 len 20480
32069 072/300 Read block 3765 len 36864
32070 073/300 Write block 1231 len 16384 data: 3 5 5 5
32071 074/300 Read block 3011 len 12288
32072 075/300 Write block 140 len 28672 data: 4 4 5 4 4 5 5
32073 076/300 Write block 228 len 40960 data: 4 5 6 5 3 2 4 4 4 4
32074 077/300 Read block 2678 len 28672
32075 078/300 Write block 968 len 36864 data: 6 5 6 6 6 6 6 6 5
32076 079/300 Read block 1868 len 32768
32077 080/300 Write block 3030 len 32768 data: 4 4 4 3 5 5 5 5
32078 081/300 Write block 3336 len 36864 data: 7 5 5 4 3 3 4 4 4
32079 082/300 Write block 2841 len 32768 data: 7 7 8 9 8 7 7 9
32080 083/300 Flush
32081 084/300 Write block 3100 len 24576 data: 5 5 5 7 6 7
32082 085/300 Write block 661 len 40960 data: 6 8 7 7 7 6 5 6 10 10
32083 086/300 Flush
32084 087/300 Read block 3854 len 36864
32085 088/300 Write block 3274 len 16384 data: 6 5 5 4
32086 089/300 Write block 2985 len 8192 data: 10 9
32087 090/300 Read block 265 len 16384
32088 091/300 Read block 3497 len 8192
32089 092/300 Read block 2156 len 20480
32090 093/300 Read block 1669 len 36864
32091 094/300 Flush
32092 095/300 Flush
32093 096/300 Read block 501 len 20480
32094 097/300 Write block 3118 len 32768 data: 4 5 6 6 5 5 5 6
32095 098/300 Flush
32096 099/300 Flush
32097 100/300 Read block 520 len 16384
32098 101/300 Write block 4872 len 24576 data: 6 4 5 3 3 3
32099 102/300 Write block 3091 len 40960 data: 7 7 7 6 5 7 5 4 4 6
32100 103/300 Read block 3557 len 24576
32101 104/300 Write block 1515 len 4096 data: 11
32102 105/300 Write block 3496 len 4096 data: 7
32103 106/300 Write block 879 len 40960 data: 3 4 3 5 6 6 7 7 6 6
32104 107/300 Read block 1475 len 12288
32105 108/300 Write block 4687 len 40960 data: 6 7 7 7 5 5 7 8 7 8
32106 109/300 Read block 4240 len 36864
32107 110/300 Read block 4845 len 28672
32108 111/300 Write block 4199 len 16384 data: 4 6 4 5
32109 112/300 Read block 2104 len 4096
32110 113/300 Write block 3135 len 32768 data: 6 4 4 7 6 9 10 9
32111 114/300 Write block 3110 len 20480 data: 9 8 7 6 5
32112 115/300 Read block 3992 len 20480
32113 116/300 Read block 1143 len 16384
32114 117/300 Read block 3897 len 40960
32115 118/300 Read block 1327 len 16384
32116 119/300 Read block 4836 len 8192
32117 120/300 Write block 4486 len 24576 data: 12 12 9 8 8 8
32118 121/300 Read block 2025 len 40960
32119 122/300 Read block 2600 len 12288
32120 123/300 Flush
32121 124/300 Flush
32122 125/300 Read block 1740 len 4096
32123 126/300 Read block 2740 len 20480
32124 127/300 Read block 665 len 8192
32125 128/300 Write block 526 len 8192 data: 7 4
32126 129/300 Write block 4633 len 8192 data: 6 5
32127 130/300 Read block 1912 len 40960
32128 131/300 Read block 22 len 20480
32129 132/300 Flush
32130 133/300 Read block 4563 len 32768
32131 134/300 Read block 953 len 28672
32132 135/300 Write block 2547 len 12288 data: 4 5 5
32133 136/300 Read block 2907 len 36864
32134 137/300 Write block 3312 len 4096 data: 6
32135 138/300 Read block 3612 len 40960
32136 139/300 Flush
32137 140/300 Write block 2878 len 36864 data: 7 7 6 3 4 4 4 4 4
32138 141/300 Read block 495 len 24576
32139 142/300 Read block 2861 len 32768
32140 143/300 Flush
32141 144/300 Read block 3470 len 12288
32142 145/300 Read block 4817 len 4096
32143 146/300 Read block 2084 len 20480
32144 147/300 Read block 2301 len 40960
32145 148/300 Read block 1309 len 4096
32146 149/300 Read block 1665 len 24576
32147 150/300 Write block 2919 len 16384 data: 4 4 6 7
32148 151/300 Write block 1549 len 36864 data: 5 6 7 7 7 6 6 5 4
32149 152/300 Write block 4121 len 28672 data: 9 9 7 6 5 6 6
32150 153/300 Write block 1603 len 8192 data: 6 5
32151 154/300 Read block 352 len 12288
32152 155/300 Write block 4144 len 12288 data: 6 4 5
32153 156/300 Write block 2783 len 32768 data: 6 6 5 7 5 7 9 10
32154 157/300 Read block 1525 len 36864
32155 158/300 Write block 635 len 32768 data: 5 4 5 4 6 6 4 5
32156 159/300 Read block 1539 len 20480
32157 160/300 Read block 2877 len 4096
32158 161/300 Write block 4453 len 4096 data: 3
32159 162/300 Write block 4193 len 36864 data: 8 8 7 7 9 7 5 7 5
32160 163/300 Read block 2624 len 8192
32161 164/300 Read block 1209 len 24576
32162 165/300 Write block 4545 len 28672 data: 3 4 4 4 4 3 3
32163 166/300 Read block 2073 len 4096
32164 167/300 Write block 3335 len 28672 data: 8 8 6 6 5 4 4
32165 168/300 Read block 1256 len 28672
32166 169/300 Read block 1640 len 24576
32167 170/300 Write block 4608 len 8192 data: 6 8
32168 171/300 Flush
32169 172/300 Read block 2803 len 8192
32170 173/300 Read block 399 len 28672
32171 174/300 Read block 1496 len 4096
32172 175/300 Read block 2414 len 4096
32173 176/300 Read block 3180 len 40960
32174 177/300 Write block 2502 len 8192 data: 4 3
32175 178/300 Read block 2624 len 16384
32176 179/300 Write block 3646 len 20480 data: 7 7 6 6 5
32177 180/300 Read block 166 len 8192
32178 181/300 Flush
32179 182/300 Write block 1574 len 40960 data: 4 4 5 5 5 5 6 6 6 7
32180 183/300 Write block 3210 len 4096 data: 6
32181 184/300 Read block 2490 len 16384
32182 185/300 Read block 274 len 12288
32183 186/300 Read block 3943 len 28672
32184 187/300 Read block 2419 len 32768
32185 188/300 Write block 149 len 20480 data: 5 4 5 8 9
32186 189/300 Write block 2492 len 12288 data: 3 3 3
32187 190/300 Flush
32188 191/300 Write block 3368 len 40960 data: 10 9 8 6 5 6 5 5 5 7
32189 192/300 Read block 3756 len 8192
32190 193/300 Write block 3976 len 8192 data: 7 7
32191 194/300 Write block 3272 len 32768 data: 5 5 7 6 6 5 4 4
32192 195/300 Read block 1701 len 4096
32193 196/300 Read block 169 len 12288
32194 197/300 Write block 2720 len 20480 data: 7 7 6 6 7
32195 198/300 Read block 4534 len 16384
32196 199/300 Write block 3059 len 4096 data: 6
32197 200/300 Read block 2006 len 20480
32198 201/300 Read block 1493 len 36864
32199 202/300 Read block 1082 len 20480
32200 203/300 Write block 3769 len 36864 data: 9 7 7 6 8 8 10 10 9
32201 204/300 Write block 3052 len 28672 data: 7 7 7 9 9 8 7
32202 205/300 Write block 816 len 36864 data: 3 5 4 3 4 6 6 6 8
32203 206/300 Write block 1833 len 40960 data: 6 6 6 7 7 8 9 8 9 7
32204 207/300 Read block 1457 len 4096
32205 208/300 Read block 1678 len 20480
32206 209/300 Write block 800 len 12288 data: 9 8 9
32207 210/300 Read block 3859 len 32768
32208 211/300 Read block 2088 len 16384
32209 212/300 Write block 824 len 32768 data: 9 5 7 7 8 8 7 8
32210 213/300 Read block 3513 len 16384
32211 214/300 Write block 3888 len 36864 data: 5 5 5 4 4 4 3 5 6
32212 215/300 Read block 2353 len 28672
32213 216/300 Write block 3173 len 8192 data: 6 5
32214 217/300 Flush
32215 218/300 Write block 1080 len 12288 data: 9 7 8
32216 219/300 Flush
32217 220/300 Read block 563 len 28672
32218 221/300 Read block 3971 len 20480
32219 222/300 Read block 4520 len 32768
32220 223/300 Read block 3559 len 8192
32221 224/300 Read block 3524 len 4096
32222 225/300 Read block 1225 len 12288
32223 226/300 Read block 888 len 36864
32224 227/300 Flush
32225 228/300 Write block 4495 len 36864 data: 5 5 5 5 5 3 4 4 4
32226 229/300 Read block 3324 len 12288
32227 230/300 Write block 342 len 16384 data: 5 6 7 7
32228 231/300 Flush
32229 232/300 Read block 3106 len 36864
32230 233/300 Read block 4040 len 24576
32231 234/300 Write block 157 len 32768 data: 6 7 7 8 10 10 9 10
32232 235/300 Write block 3923 len 16384 data: 6 6 7 8
32233 236/300 Flush
32234 237/300 Write block 3204 len 24576 data: 5 3 4 5 5 7
32235 238/300 Write block 4238 len 28672 data: 8 7 8 8 7 5 7
32236 239/300 Write block 634 len 8192 data: 4 6
32237 240/300 Read block 4872 len 36864
32238 241/300 Write block 712 len 36864 data: 4 4 3 3 2 2 2 3 3
32239 242/300 Read block 4977 len 36864
32240 243/300 Read block 349 len 36864
32241 244/300 Read block 296 len 8192
32242 245/300 Write block 3867 len 32768 data: 5 5 5 7 6 6 7 6
32243 246/300 Read block 2474 len 36864
32244 247/300 Read block 1999 len 32768
32245 248/300 Write block 1768 len 20480 data: 12 12 10 7 5
32246 249/300 Write block 1956 len 40960 data: 5 4 5 4 6 6 6 5 5 3
32247 250/300 Read block 372 len 40960
32248 251/300 Read block 2055 len 36864
32249 252/300 Read block 415 len 8192
32250 253/300 Flush
32251 254/300 Read block 2638 len 28672
32252 255/300 Write block 1989 len 12288 data: 8 8 9
32253 256/300 Flush
32254 257/300 Read block 2136 len 28672
32255 258/300 Write block 688 len 28672 data: 4 4 5 6 5 5 4
32256 259/300 Write block 3238 len 16384 data: 8 7 7 8
32257 260/300 Flush
32258 261/300 Read block 2722 len 36864
32259 262/300 Read block 4508 len 32768
32260 263/300 Write block 4190 len 12288 data: 3 7 7
32261 264/300 Read block 28 len 32768
32262 265/300 Read block 1508 len 16384
32263 266/300 Write block 185 len 24576 data: 6 7 7 6 7 7
32264 267/300 Read block 858 len 40960
32265 268/300 Write block 3763 len 40960 data: 8 10 11 11 10 10 10 8 8 7
32266 269/300 Write block 4782 len 32768 data: 7 9 8 7 7 7 6 5
32267 270/300 Read block 316 len 28672
32268 271/300 Read block 4557 len 4096
32269 272/300 Write block 3757 len 20480 data: 8 8 8 7 6
32270 273/300 Write block 1011 len 40960 data: 6 8 8 7 8 11 10 11 9 8
32271 274/300 Read block 3686 len 8192
32272 275/300 Write block 3795 len 8192 data: 6 5
32273 276/300 Read block 1560 len 16384
32274 277/300 Write block 2752 len 16384 data: 5 4 4 3
32275 278/300 Read block 3164 len 20480
32276 279/300 Write block 4835 len 28672 data: 5 4 6 7 7 9 9
32277 280/300 Flush
32278 281/300 Read block 3269 len 32768
32279 282/300 Read block 4312 len 36864
32280 283/300 Write block 3422 len 32768 data: 5 4 5 4 4 3 3 3
32281 284/300 Read block 3773 len 36864
32282 285/300 Write block 4474 len 4096 data: 7
32283 286/300 Write block 3787 len 32768 data: 5 7 7 6 5 6 8 8
32284 287/300 Write block 1136 len 32768 data: 8 8 9 8 8 9 10 8
32285 288/300 Write block 4240 len 40960 data: 9 9 8 6 8 8 8 9 11 12
32286 289/300 Read block 2454 len 8192
32287 290/300 Write block 3006 len 16384 data: 8 8 8 7
32288 291/300 Read block 392 len 4096
32289 292/300 Flush
32290 293/300 Read block 4828 len 32768
32291 294/300 Read block 2720 len 16384
32292 295/300 Flush
32293 296/300 Write block 3160 len 24576 data: 8 7 7 5 5 5
32294 297/300 Read block 170 len 40960
32295 298/300 Read block 2843 len 32768
32296 299/300 Write block 4832 len 16384 data: 7 5 5 6
32297 300/300 Write block 4455 len 4096 data: 4
32298 301/300 Write block 4206 len 16384 data: 4 4 4 4
32299 Main task has work to do, go find it
32300 got dsc Start(1)
32301 start 1
32302 Dec 04 13:37:29.736 INFO request completed, latency_us: 290, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3e6d0d03-12f1-4fdf-8ca4-01b398aa395f, remote_addr: 127.0.0.1:37804, local_addr: 127.0.0.1:9998
32303 Replay: started 1, returned:Ok(())
32304 ----------------------------------------------------------------
32305 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32306 States: Active Offline Active
32307 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32308 10056 Acked Write 2 Done Sent Done false
32309 10057 Acked Read 4 Done Sent Done false
32310 10058 Acked Read 8 Done Sent Done false
32311 10059 Acked Read 9 Done Sent Done false
32312 10060 Acked Read 5 Done Sent Done false
32313 10061 Acked Read 10 Done Sent Done false
32314 10062 Acked Read 7 Done Sent Done false
32315 10063 Acked Read 5 Done Sent Done false
32316 10064 Acked Read 10 Done Sent Done false
32317 10065 Acked Read 9 Done Sent Done false
32318 10066 Acked Read 2 Done Sent Done false
32319 10067 Acked Read 1 Done Sent Done false
32320 10068 Acked Write 6 Done Sent Done false
32321 10069 Acked Read 8 Done Sent Done false
32322 10070 Acked Read 5 Done Sent Done false
32323 10071 Acked Read 6 Done Sent Done false
32324 10072 Acked Read 4 Done Sent Done false
32325 10073 Acked Write 1 Done Sent Done false
32326 10074 Acked Read 1 Done Sent Done false
32327 10075 Acked Write 10 Done Sent Done false
32328 10076 Acked Flush 0 Done Sent Done false
32329 10077 Acked Read 7 Done Sent Done false
32330 10078 Acked Read 10 Done Sent Done false
32331 10079 Acked Write 3 Done Sent Done false
32332 10080 Acked Write 6 Done Sent Done false
32333 10081 Acked Read 6 Done Sent Done false
32334 10082 Acked Flush 0 Done Sent Done false
32335 10083 Acked Read 2 Done Sent Done false
32336 10084 Acked Read 5 Done Sent Done false
32337 10085 Acked Read 4 Done Sent Done false
32338 10086 Acked Write 1 Done Sent Done false
32339 10087 Acked Flush 0 Done Sent Done false
32340 10088 Acked Write 1 Done Sent Done false
32341 10089 Acked Write 3 Done Sent Done false
32342 10090 Acked Flush 0 Done Sent Done false
32343 10091 Acked Read 6 Done Sent Done false
32344 10092 Acked Write 4 Done Sent Done false
32345 10093 Acked Read 9 Done Sent Done false
32346 10094 Acked Write 8 Done Sent Done false
32347 10095 Acked Read 8 Done Sent Done false
32348 10096 Acked Read 10 Done Sent Done false
32349 10097 Acked Write 9 Done Sent Done false
32350 10098 Acked Read 5 Done Sent Done false
32351 10099 Acked Read 3 Done Sent Done false
32352 10100 Acked Read 9 Done Sent Done false
32353 10101 Acked Write 5 Done Sent Done false
32354 10102 Acked Flush 0 Done Sent Done false
32355 10103 Acked Write 6 Done Sent Done false
32356 10104 Acked Read 4 Done Sent Done false
32357 10105 Acked Read 5 Done Sent Done false
32358 10106 Acked Flush 0 Done Sent Done false
32359 10107 Acked Write 2 Done Sent Done false
32360 10108 Acked Write 1 Done Sent Done false
32361 10109 Acked Write 8 Done Sent Done false
32362 10110 Acked Flush 0 Done Sent Done false
32363 10111 Acked Write 4 Done Sent Done false
32364 10112 Acked Write 10 Done Sent Done false
32365 10113 Acked Read 3 Done Sent Done false
32366 10114 Acked Read 2 Done Sent Done false
32367 10115 Acked Read 9 Done Sent Done false
32368 10116 Acked Write 4 Done Sent Done false
32369 10117 Acked Flush 0 Done Sent Done false
32370 10118 Acked Flush 0 Done Sent Done false
32371 10119 Acked Read 3 Done Sent Done false
32372 10120 Acked Read 6 Done Sent Done false
32373 10121 Acked Read 2 Done Sent Done false
32374 10122 Acked Write 1 Done Sent Done false
32375 10123 Acked Write 4 Done Sent Done false
32376 10124 Acked Read 4 Done Sent Done false
32377 10125 Acked Read 2 Done Sent Done false
32378 10126 Acked Read 5 Done Sent Done false
32379 10127 Acked Read 9 Done Sent Done false
32380 10128 Acked Write 4 Done Sent Done false
32381 10129 Acked Read 3 Done Sent Done false
32382 10130 Acked Write 7 Done Sent Done false
32383 10131 Acked Write 10 Done Sent Done false
32384 10132 Acked Read 7 Done Sent Done false
32385 10133 Acked Write 9 Done Sent Done false
32386 10134 Acked Read 8 Done Sent Done false
32387 10135 Acked Write 8 Done Sent Done false
32388 10136 Acked Write 9 Done Sent Done false
32389 10137 Acked Write 8 Done Sent Done false
32390 10138 Acked Flush 0 Done Sent Done false
32391 10139 Acked Write 6 Done Sent Done false
32392 10140 Acked Write 10 Done Sent Done false
32393 10141 Acked Flush 0 Done Sent Done false
32394 10142 Acked Read 9 Done Sent Done false
32395 10143 Acked Write 4 Done Sent Done false
32396 10144 Acked Write 2 Done Sent Done false
32397 10145 Acked Read 4 Done Sent Done false
32398 10146 Acked Read 2 Done Sent Done false
32399 10147 Acked Read 5 Done Sent Done false
32400 10148 Acked Read 9 Done Sent Done false
32401 10149 Acked Flush 0 Done Sent Done false
32402 10150 Acked Flush 0 Done Sent Done false
32403 10151 Acked Read 5 Done Sent Done false
32404 10152 Acked Write 8 Done Sent Done false
32405 10153 Acked Flush 0 Done Sent Done false
32406 10154 Acked Flush 0 Done Sent Done false
32407 10155 Acked Read 4 Done Sent Done false
32408 10156 Acked Write 6 Done Sent Done false
32409 10157 Acked Write 10 Done Sent Done false
32410 10158 Acked Read 6 Done Sent Done false
32411 10159 Acked Write 1 Done Sent Done false
32412 10160 Acked Write 1 Done Sent Done false
32413 10161 Acked Write 10 Done Sent Done false
32414 10162 Acked Read 3 Done Sent Done false
32415 10163 Acked Write 10 Done Sent Done false
32416 10164 Acked Read 9 Done Sent Done false
32417 10165 Acked Read 7 Done Sent Done false
32418 10166 Acked Write 4 Done Sent Done false
32419 10167 Acked Read 1 Done Sent Done false
32420 10168 Acked Write 8 Done Sent Done false
32421 10169 Acked Write 5 Done Sent Done false
32422 10170 Acked Read 5 Done Sent Done false
32423 10171 Acked Read 4 Done Sent Done false
32424 10172 Acked Read 10 Done Sent Done false
32425 10173 Acked Read 4 Done Sent Done false
32426 10174 Acked Read 2 Done Sent Done false
32427 10175 Acked Write 6 Done Sent Done false
32428 10176 Acked Read 10 Done Sent Done false
32429 10177 Acked Read 3 Done Sent Done false
32430 10178 Acked Flush 0 Done Sent Done false
32431 10179 Acked Flush 0 Done Sent Done false
32432 10180 Acked Read 1 Done Sent Done false
32433 10181 Acked Read 5 Done Sent Done false
32434 10182 Acked Read 2 Done Sent Done false
32435 10183 Acked Write 2 Done Sent Done false
32436 10184 Acked Write 2 Done Sent Done false
32437 10185 Acked Read 10 Done Sent Done false
32438 10186 Acked Read 5 Done Sent Done false
32439 10187 Acked Flush 0 Done Sent Done false
32440 10188 Acked Read 8 Done Sent Done false
32441 10189 Acked Read 7 Done Sent Done false
32442 10190 Acked Write 3 Done Sent Done false
32443 10191 Acked Read 9 Done Sent Done false
32444 10192 Acked Write 1 Done Sent Done false
32445 10193 Acked Read 10 Done Sent Done false
32446 10194 Acked Flush 0 Done Sent Done false
32447 10195 Acked Write 9 Done Sent Done false
32448 10196 Acked Read 6 Done Sent Done false
32449 10197 Acked Read 8 Done Sent Done false
32450 10198 Acked Flush 0 Done Sent Done false
32451 10199 Acked Read 3 Done Sent Done false
32452 10200 Acked Read 1 Done Sent Done false
32453 10201 Acked Read 5 Done Sent Done false
32454 10202 Acked Read 10 Done Sent Done false
32455 10203 Acked Read 1 Done Sent Done false
32456 10204 Acked Read 6 Done Sent Done false
32457 10205 Acked Write 4 Done Sent Done false
32458 10206 Acked Write 9 Done Sent Done false
32459 10207 Acked Write 7 Done Sent Done false
32460 10208 Acked Write 2 Done Sent Done false
32461 10209 Acked Read 3 Done Sent Done false
32462 10210 Acked Write 3 Done Sent Done false
32463 10211 Acked Write 8 Done Sent Done false
32464 10212 Acked Read 9 Done Sent Done false
32465 10213 Acked Write 8 Done Sent Done false
32466 10214 Acked Read 5 Done Sent Done false
32467 10215 Acked Read 1 Done Sent Done false
32468 10216 Acked Write 1 Done Sent Done false
32469 10217 Acked Write 9 Done Sent Done false
32470 10218 Acked Read 2 Done Sent Done false
32471 10219 Acked Read 6 Done Sent Done false
32472 10220 Acked Write 7 Done Sent Done false
32473 10221 Acked Read 1 Done Sent Done false
32474 10222 Acked Write 7 Done Sent Done false
32475 10223 Acked Read 7 Done Sent Done false
32476 10224 Acked Read 6 Done Sent Done false
32477 10225 Acked Write 2 Done Sent Done false
32478 10226 Acked Flush 0 Done Sent Done false
32479 10227 Acked Read 2 Done Sent Done false
32480 10228 Acked Read 7 Done Sent Done false
32481 10229 Acked Read 1 Done Sent Done false
32482 10230 Acked Read 1 Done Sent Done false
32483 10231 Acked Read 10 Done Sent Done false
32484 10232 Acked Write 2 Done Sent Done false
32485 10233 Acked Read 4 Done Sent Done false
32486 10234 Acked Write 5 Done Sent Done false
32487 10235 Acked Read 2 Done Sent Done false
32488 10236 Acked Flush 0 Done Sent Done false
32489 10237 Acked Write 10 Done Sent Done false
32490 10238 Acked Write 1 Done Sent Done false
32491 10239 Acked Read 4 Done Sent Done false
32492 10240 Acked Read 3 Done Sent Done false
32493 10241 Acked Read 7 Done Sent Done false
32494 10242 Acked Read 8 Done Sent Done false
32495 10243 Acked Write 5 Done Sent Done false
32496 10244 Acked Write 3 Done Sent Done false
32497 10245 Acked Flush 0 Done Sent Done false
32498 10246 Acked Write 10 Done Sent Done false
32499 10247 Acked Read 2 Done Sent Done false
32500 10248 Acked Write 2 Done Sent Done false
32501 10249 Acked Write 8 Done Sent Done false
32502 10250 Acked Read 1 Done Sent Done false
32503 10251 Acked Read 3 Done Sent Done false
32504 10252 Acked Write 5 Done Sent Done false
32505 10253 Acked Read 4 Done Sent Done false
32506 10254 Acked Write 1 Done Sent Done false
32507 10255 Acked Read 5 Done Sent Done false
32508 10256 Acked Read 9 Done Sent Done false
32509 10257 Acked Read 5 Done Sent Done false
32510 10258 Acked Write 9 Done Sent Done false
32511 10259 Acked Write 7 Done Sent Done false
32512 10260 Acked Write 9 Done Sent Done false
32513 10261 Acked Write 10 Done Sent Done false
32514 10262 Acked Read 1 Done Sent Done false
32515 10263 Acked Read 5 Done Sent Done false
32516 10264 Acked Write 3 Done Sent Done false
32517 10265 Acked Read 8 Done Sent Done false
32518 10266 Acked Read 4 Done Sent Done false
32519 10267 Acked Write 8 Done Sent Done false
32520 10268 Acked Read 4 Done Sent Done false
32521 10269 Acked Write 9 Done Sent Done false
32522 10270 Acked Read 7 Done Sent Done false
32523 10271 Acked Write 2 Done Sent Done false
32524 10272 Acked Flush 0 Done Sent Done false
32525 10273 Acked Write 3 Done Sent Done false
32526 10274 Acked Flush 0 Done Sent Done false
32527 10275 Acked Read 7 Done Sent Done false
32528 10276 Acked Read 5 Done Sent Done false
32529 10277 Acked Read 8 Done Sent Done false
32530 10278 Acked Read 2 Done Sent Done false
32531 10279 Acked Read 1 Done Sent Done false
32532 10280 Acked Read 3 Done Sent Done false
32533 10281 Acked Read 9 Done Sent Done false
32534 10282 Acked Flush 0 Done Sent Done false
32535 10283 Acked Write 9 Done Sent Done false
32536 10284 Acked Read 3 Done Sent Done false
32537 10285 Acked Write 4 Done Sent Done false
32538 10286 Acked Flush 0 Done Sent Done false
32539 10287 Acked Read 9 Done Sent Done false
32540 10288 Acked Read 6 Done Sent Done false
32541 10289 Acked Write 8 Done Sent Done false
32542 10290 Acked Write 4 Done Sent Done false
32543 10291 Acked Flush 0 Done Sent Done false
32544 10292 Acked Write 6 Done Sent Done false
32545 10293 Acked Write 7 Done Sent Done false
32546 10294 Acked Write 2 Done Sent Done false
32547 10295 Acked Read 9 Done Sent Done false
32548 10296 Acked Write 9 Done Sent Done false
32549 10297 Acked Read 9 Done Sent Done false
32550 10298 Acked Read 9 Done Sent Done false
32551 10299 Acked Read 2 Done Sent Done false
32552 10300 Acked Write 8 Done Sent Done false
32553 10301 Acked Read 9 Done Sent Done false
32554 10302 Acked Read 8 Done Sent Done false
32555 10303 Acked Write 5 Done Sent Done false
32556 10304 Acked Write 10 Done Sent Done false
32557 10305 Acked Read 10 Done Sent Done false
32558 10306 Acked Read 9 Done Sent Done false
32559 10307 Acked Read 2 Done Sent Done false
32560 10308 Acked Flush 0 Done Sent Done false
32561 10309 Acked Read 7 Done Sent Done false
32562 10310 Acked Write 3 Done Sent Done false
32563 10311 Acked Flush 0 Done Sent Done false
32564 10312 Acked Read 7 Done Sent Done false
32565 10313 Acked Write 7 Done Sent Done false
32566 10314 Acked Write 4 Done Sent Done false
32567 10315 Acked Flush 0 Done Sent Done false
32568 10316 Acked Read 9 Done Sent Done false
32569 10317 Acked Read 8 Done Sent Done false
32570 10318 Acked Write 3 Done Sent Done false
32571 10319 Acked Read 8 Done Sent Done false
32572 10320 Acked Read 4 Done Sent Done false
32573 10321 Acked Write 6 Done Sent Done false
32574 10322 Acked Read 10 Done Sent Done false
32575 10323 Acked Write 10 Done Sent Done false
32576 10324 Acked Write 8 Done Sent Done false
32577 10325 Acked Read 7 Done Sent Done false
32578 10326 Acked Read 1 Done Sent Done false
32579 10327 Acked Write 5 Done Sent Done false
32580 10328 Acked Write 10 Done Sent Done false
32581 10329 Acked Read 2 Done Sent Done false
32582 10330 Acked Write 2 Done Sent Done false
32583 10331 Acked Read 4 Done Sent Done false
32584 10332 Acked Write 4 Done Sent Done false
32585 10333 Acked Read 5 Done Sent Done false
32586 10334 Acked Write 7 Done Sent Done false
32587 10335 Acked Flush 0 Done Sent Done false
32588 10336 Acked Read 8 Done Sent Done false
32589 10337 Acked Read 9 Done Sent Done false
32590 10338 Acked Write 8 Done Sent Done false
32591 10339 Acked Read 9 Done Sent Done false
32592 10340 Acked Write 1 Done Sent Done false
32593 10341 Acked Write 8 Done Sent Done false
32594 10342 Acked Write 8 Done Sent Done false
32595 10343 Acked Write 10 Done Sent Done false
32596 10344 Acked Read 2 Done Sent Done false
32597 10345 Acked Write 4 Done Sent Done false
32598 10346 Acked Read 1 Done Sent Done false
32599 10347 Acked Flush 0 Done Sent Done false
32600 10348 Acked Read 8 Done Sent Done false
32601 10349 Acked Read 4 Done Sent Done false
32602 10350 Acked Flush 0 Done Sent Done false
32603 10351 Acked Write 6 Done Sent Done false
32604 10352 Acked Read 10 Done Sent Done false
32605 10353 Acked Read 8 Done Sent Done false
32606 10354 Acked Write 4 Done Sent Done false
32607 10355 Acked Write 1 Done Sent Done false
32608 10356 Acked Write 4 Done Sent Done false
32609 STATES DS:0 DS:1 DS:2 TOTAL
32610 Sent 0 301 0 301
32611 Done 301 0 301 602
32612 Skipped 0 0 0 0
32613 Error 0 0 0 0
32614 Last Flush: 10350 10055 10350
32615 Downstairs last five completed: 10055 10054 10053 10052 10051
32616 Upstairs last five completed: 10356 10355 10354 10353 10352
32617 CLIENT: Up:0 ds:301 act:2
32618 ----------------------------------------------------------------
32619 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32620 States: Active Offline Active
32621 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32622 10056 Acked Write 2 Done Sent Done false
32623 10057 Acked Read 4 Done Sent Done false
32624 10058 Acked Read 8 Done Sent Done false
32625 10059 Acked Read 9 Done Sent Done false
32626 10060 Acked Read 5 Done Sent Done false
32627 10061 Acked Read 10 Done Sent Done false
32628 10062 Acked Read 7 Done Sent Done false
32629 10063 Acked Read 5 Done Sent Done false
32630 10064 Acked Read 10 Done Sent Done false
32631 10065 Acked Read 9 Done Sent Done false
32632 10066 Acked Read 2 Done Sent Done false
32633 10067 Acked Read 1 Done Sent Done false
32634 10068 Acked Write 6 Done Sent Done false
32635 10069 Acked Read 8 Done Sent Done false
32636 10070 Acked Read 5 Done Sent Done false
32637 10071 Acked Read 6 Done Sent Done false
32638 10072 Acked Read 4 Done Sent Done false
32639 10073 Acked Write 1 Done Sent Done false
32640 10074 Acked Read 1 Done Sent Done false
32641 10075 Acked Write 10 Done Sent Done false
32642 10076 Acked Flush 0 Done Sent Done false
32643 10077 Acked Read 7 Done Sent Done false
32644 10078 Acked Read 10 Done Sent Done false
32645 10079 Acked Write 3 Done Sent Done false
32646 10080 Acked Write 6 Done Sent Done false
32647 10081 Acked Read 6 Done Sent Done false
32648 10082 Acked Flush 0 Done Sent Done false
32649 10083 Acked Read 2 Done Sent Done false
32650 10084 Acked Read 5 Done Sent Done false
32651 10085 Acked Read 4 Done Sent Done false
32652 10086 Acked Write 1 Done Sent Done false
32653 10087 Acked Flush 0 Done Sent Done false
32654 10088 Acked Write 1 Done Sent Done false
32655 10089 Acked Write 3 Done Sent Done false
32656 10090 Acked Flush 0 Done Sent Done false
32657 10091 Acked Read 6 Done Sent Done false
32658 10092 Acked Write 4 Done Sent Done false
32659 10093 Acked Read 9 Done Sent Done false
32660 10094 Acked Write 8 Done Sent Done false
32661 10095 Acked Read 8 Done Sent Done false
32662 10096 Acked Read 10 Done Sent Done false
32663 10097 Acked Write 9 Done Sent Done false
32664 10098 Acked Read 5 Done Sent Done false
32665 10099 Acked Read 3 Done Sent Done false
32666 10100 Acked Read 9 Done Sent Done false
32667 10101 Acked Write 5 Done Sent Done false
32668 10102 Acked Flush 0 Done Sent Done false
32669 10103 Acked Write 6 Done Sent Done false
32670 10104 Acked Read 4 Done Sent Done false
32671 10105 Acked Read 5 Done Sent Done false
32672 10106 Acked Flush 0 Done Sent Done false
32673 10107 Acked Write 2 Done Sent Done false
32674 10108 Acked Write 1 Done Sent Done false
32675 10109 Acked Write 8 Done Sent Done false
32676 10110 Acked Flush 0 Done Sent Done false
32677 10111 Acked Write 4 Done Sent Done false
32678 10112 Acked Write 10 Done Sent Done false
32679 10113 Acked Read 3 Done Sent Done false
32680 10114 Acked Read 2 Done Sent Done false
32681 10115 Acked Read 9 Done Sent Done false
32682 10116 Acked Write 4 Done Sent Done false
32683 10117 Acked Flush 0 Done Sent Done false
32684 10118 Acked Flush 0 Done Sent Done false
32685 10119 Acked Read 3 Done Sent Done false
32686 10120 Acked Read 6 Done Sent Done false
32687 10121 Acked Read 2 Done Sent Done false
32688 10122 Acked Write 1 Done Sent Done false
32689 10123 Acked Write 4 Done Sent Done false
32690 10124 Acked Read 4 Done Sent Done false
32691 10125 Acked Read 2 Done Sent Done false
32692 10126 Acked Read 5 Done Sent Done false
32693 10127 Acked Read 9 Done Sent Done false
32694 10128 Acked Write 4 Done Sent Done false
32695 10129 Acked Read 3 Done Sent Done false
32696 10130 Acked Write 7 Done Sent Done false
32697 10131 Acked Write 10 Done Sent Done false
32698 10132 Acked Read 7 Done Sent Done false
32699 10133 Acked Write 9 Done Sent Done false
32700 10134 Acked Read 8 Done Sent Done false
32701 10135 Acked Write 8 Done Sent Done false
32702 10136 Acked Write 9 Done Sent Done false
32703 10137 Acked Write 8 Done Sent Done false
32704 10138 Acked Flush 0 Done Sent Done false
32705 10139 Acked Write 6 Done Sent Done false
32706 10140 Acked Write 10 Done Sent Done false
32707 10141 Acked Flush 0 Done Sent Done false
32708 10142 Acked Read 9 Done Sent Done false
32709 10143 Acked Write 4 Done Sent Done false
32710 10144 Acked Write 2 Done Sent Done false
32711 10145 Acked Read 4 Done Sent Done false
32712 10146 Acked Read 2 Done Sent Done false
32713 10147 Acked Read 5 Done Sent Done false
32714 10148 Acked Read 9 Done Sent Done false
32715 10149 Acked Flush 0 Done Sent Done false
32716 10150 Acked Flush 0 Done Sent Done false
32717 10151 Acked Read 5 Done Sent Done false
32718 10152 Acked Write 8 Done Sent Done false
32719 10153 Acked Flush 0 Done Sent Done false
32720 10154 Acked Flush 0 Done Sent Done false
32721 10155 Acked Read 4 Done Sent Done false
32722 10156 Acked Write 6 Done Sent Done false
32723 10157 Acked Write 10 Done Sent Done false
32724 10158 Acked Read 6 Done Sent Done false
32725 10159 Acked Write 1 Done Sent Done false
32726 10160 Acked Write 1 Done Sent Done false
32727 10161 Acked Write 10 Done Sent Done false
32728 10162 Acked Read 3 Done Sent Done false
32729 10163 Acked Write 10 Done Sent Done false
32730 10164 Acked Read 9 Done Sent Done false
32731 10165 Acked Read 7 Done Sent Done false
32732 10166 Acked Write 4 Done Sent Done false
32733 10167 Acked Read 1 Done Sent Done false
32734 10168 Acked Write 8 Done Sent Done false
32735 10169 Acked Write 5 Done Sent Done false
32736 10170 Acked Read 5 Done Sent Done false
32737 10171 Acked Read 4 Done Sent Done false
32738 10172 Acked Read 10 Done Sent Done false
32739 10173 Acked Read 4 Done Sent Done false
32740 10174 Acked Read 2 Done Sent Done false
32741 10175 Acked Write 6 Done Sent Done false
32742 10176 Acked Read 10 Done Sent Done false
32743 10177 Acked Read 3 Done Sent Done false
32744 10178 Acked Flush 0 Done Sent Done false
32745 10179 Acked Flush 0 Done Sent Done false
32746 10180 Acked Read 1 Done Sent Done false
32747 10181 Acked Read 5 Done Sent Done false
32748 10182 Acked Read 2 Done Sent Done false
32749 10183 Acked Write 2 Done Sent Done false
32750 10184 Acked Write 2 Done Sent Done false
32751 10185 Acked Read 10 Done Sent Done false
32752 10186 Acked Read 5 Done Sent Done false
32753 10187 Acked Flush 0 Done Sent Done false
32754 10188 Acked Read 8 Done Sent Done false
32755 10189 Acked Read 7 Done Sent Done false
32756 10190 Acked Write 3 Done Sent Done false
32757 10191 Acked Read 9 Done Sent Done false
32758 10192 Acked Write 1 Done Sent Done false
32759 10193 Acked Read 10 Done Sent Done false
32760 10194 Acked Flush 0 Done Sent Done false
32761 10195 Acked Write 9 Done Sent Done false
32762 10196 Acked Read 6 Done Sent Done false
32763 10197 Acked Read 8 Done Sent Done false
32764 10198 Acked Flush 0 Done Sent Done false
32765 10199 Acked Read 3 Done Sent Done false
32766 10200 Acked Read 1 Done Sent Done false
32767 10201 Acked Read 5 Done Sent Done false
32768 10202 Acked Read 10 Done Sent Done false
32769 10203 Acked Read 1 Done Sent Done false
32770 10204 Acked Read 6 Done Sent Done false
32771 10205 Acked Write 4 Done Sent Done false
32772 10206 Acked Write 9 Done Sent Done false
32773 10207 Acked Write 7 Done Sent Done false
32774 10208 Acked Write 2 Done Sent Done false
32775 10209 Acked Read 3 Done Sent Done false
32776 10210 Acked Write 3 Done Sent Done false
32777 10211 Acked Write 8 Done Sent Done false
32778 10212 Acked Read 9 Done Sent Done false
32779 10213 Acked Write 8 Done Sent Done false
32780 10214 Acked Read 5 Done Sent Done false
32781 10215 Acked Read 1 Done Sent Done false
32782 10216 Acked Write 1 Done Sent Done false
32783 10217 Acked Write 9 Done Sent Done false
32784 10218 Acked Read 2 Done Sent Done false
32785 10219 Acked Read 6 Done Sent Done false
32786 10220 Acked Write 7 Done Sent Done false
32787 10221 Acked Read 1 Done Sent Done false
32788 10222 Acked Write 7 Done Sent Done false
32789 10223 Acked Read 7 Done Sent Done false
32790 10224 Acked Read 6 Done Sent Done false
32791 10225 Acked Write 2 Done Sent Done false
32792 10226 Acked Flush 0 Done Sent Done false
32793 10227 Acked Read 2 Done Sent Done false
32794 10228 Acked Read 7 Done Sent Done false
32795 10229 Acked Read 1 Done Sent Done false
32796 10230 Acked Read 1 Done Sent Done false
32797 10231 Acked Read 10 Done Sent Done false
32798 10232 Acked Write 2 Done Sent Done false
32799 10233 Acked Read 4 Done Sent Done false
32800 10234 Acked Write 5 Done Sent Done false
32801 10235 Acked Read 2 Done Sent Done false
32802 10236 Acked Flush 0 Done Sent Done false
32803 10237 Acked Write 10 Done Sent Done false
32804 10238 Acked Write 1 Done Sent Done false
32805 10239 Acked Read 4 Done Sent Done false
32806 10240 Acked Read 3 Done Sent Done false
32807 10241 Acked Read 7 Done Sent Done false
32808 10242 Acked Read 8 Done Sent Done false
32809 10243 Acked Write 5 Done Sent Done false
32810 10244 Acked Write 3 Done Sent Done false
32811 10245 Acked Flush 0 Done Sent Done false
32812 10246 Acked Write 10 Done Sent Done false
32813 10247 Acked Read 2 Done Sent Done false
32814 10248 Acked Write 2 Done Sent Done false
32815 10249 Acked Write 8 Done Sent Done false
32816 10250 Acked Read 1 Done Sent Done false
32817 10251 Acked Read 3 Done Sent Done false
32818 10252 Acked Write 5 Done Sent Done false
32819 10253 Acked Read 4 Done Sent Done false
32820 10254 Acked Write 1 Done Sent Done false
32821 10255 Acked Read 5 Done Sent Done false
32822 10256 Acked Read 9 Done Sent Done false
32823 10257 Acked Read 5 Done Sent Done false
32824 10258 Acked Write 9 Done Sent Done false
32825 10259 Acked Write 7 Done Sent Done false
32826 10260 Acked Write 9 Done Sent Done false
32827 10261 Acked Write 10 Done Sent Done false
32828 10262 Acked Read 1 Done Sent Done false
32829 10263 Acked Read 5 Done Sent Done false
32830 10264 Acked Write 3 Done Sent Done false
32831 10265 Acked Read 8 Done Sent Done false
32832 10266 Acked Read 4 Done Sent Done false
32833 10267 Acked Write 8 Done Sent Done false
32834 10268 Acked Read 4 Done Sent Done false
32835 10269 Acked Write 9 Done Sent Done false
32836 10270 Acked Read 7 Done Sent Done false
32837 10271 Acked Write 2 Done Sent Done false
32838 10272 Acked Flush 0 Done Sent Done false
32839 10273 Acked Write 3 Done Sent Done false
32840 10274 Acked Flush 0 Done Sent Done false
32841 10275 Acked Read 7 Done Sent Done false
32842 10276 Acked Read 5 Done Sent Done false
32843 10277 Acked Read 8 Done Sent Done false
32844 10278 Acked Read 2 Done Sent Done false
32845 10279 Acked Read 1 Done Sent Done false
32846 10280 Acked Read 3 Done Sent Done false
32847 10281 Acked Read 9 Done Sent Done false
32848 10282 Acked Flush 0 Done Sent Done false
32849 10283 Acked Write 9 Done Sent Done false
32850 10284 Acked Read 3 Done Sent Done false
32851 10285 Acked Write 4 Done Sent Done false
32852 10286 Acked Flush 0 Done Sent Done false
32853 10287 Acked Read 9 Done Sent Done false
32854 10288 Acked Read 6 Done Sent Done false
32855 10289 Acked Write 8 Done Sent Done false
32856 10290 Acked Write 4 Done Sent Done false
32857 10291 Acked Flush 0 Done Sent Done false
32858 10292 Acked Write 6 Done Sent Done false
32859 10293 Acked Write 7 Done Sent Done false
32860 10294 Acked Write 2 Done Sent Done false
32861 10295 Acked Read 9 Done Sent Done false
32862 10296 Acked Write 9 Done Sent Done false
32863 10297 Acked Read 9 Done Sent Done false
32864 10298 Acked Read 9 Done Sent Done false
32865 10299 Acked Read 2 Done Sent Done false
32866 10300 Acked Write 8 Done Sent Done false
32867 10301 Acked Read 9 Done Sent Done false
32868 10302 Acked Read 8 Done Sent Done false
32869 10303 Acked Write 5 Done Sent Done false
32870 10304 Acked Write 10 Done Sent Done false
32871 10305 Acked Read 10 Done Sent Done false
32872 10306 Acked Read 9 Done Sent Done false
32873 10307 Acked Read 2 Done Sent Done false
32874 10308 Acked Flush 0 Done Sent Done false
32875 10309 Acked Read 7 Done Sent Done false
32876 10310 Acked Write 3 Done Sent Done false
32877 10311 Acked Flush 0 Done Sent Done false
32878 10312 Acked Read 7 Done Sent Done false
32879 10313 Acked Write 7 Done Sent Done false
32880 10314 Acked Write 4 Done Sent Done false
32881 10315 Acked Flush 0 Done Sent Done false
32882 10316 Acked Read 9 Done Sent Done false
32883 10317 Acked Read 8 Done Sent Done false
32884 10318 Acked Write 3 Done Sent Done false
32885 10319 Acked Read 8 Done Sent Done false
32886 10320 Acked Read 4 Done Sent Done false
32887 10321 Acked Write 6 Done Sent Done false
32888 10322 Acked Read 10 Done Sent Done false
32889 10323 Acked Write 10 Done Sent Done false
32890 10324 Acked Write 8 Done Sent Done false
32891 10325 Acked Read 7 Done Sent Done false
32892 10326 Acked Read 1 Done Sent Done false
32893 10327 Acked Write 5 Done Sent Done false
32894 10328 Acked Write 10 Done Sent Done false
32895 10329 Acked Read 2 Done Sent Done false
32896 10330 Acked Write 2 Done Sent Done false
32897 10331 Acked Read 4 Done Sent Done false
32898 10332 Acked Write 4 Done Sent Done false
32899 10333 Acked Read 5 Done Sent Done false
32900 10334 Acked Write 7 Done Sent Done false
32901 10335 Acked Flush 0 Done Sent Done false
32902 10336 Acked Read 8 Done Sent Done false
32903 10337 Acked Read 9 Done Sent Done false
32904 10338 Acked Write 8 Done Sent Done false
32905 10339 Acked Read 9 Done Sent Done false
32906 10340 Acked Write 1 Done Sent Done false
32907 10341 Acked Write 8 Done Sent Done false
32908 10342 Acked Write 8 Done Sent Done false
32909 10343 Acked Write 10 Done Sent Done false
32910 10344 Acked Read 2 Done Sent Done false
32911 10345 Acked Write 4 Done Sent Done false
32912 10346 Acked Read 1 Done Sent Done false
32913 10347 Acked Flush 0 Done Sent Done false
32914 10348 Acked Read 8 Done Sent Done false
32915 10349 Acked Read 4 Done Sent Done false
32916 10350 Acked Flush 0 Done Sent Done false
32917 10351 Acked Write 6 Done Sent Done false
32918 10352 Acked Read 10 Done Sent Done false
32919 10353 Acked Read 8 Done Sent Done false
32920 10354 Acked Write 4 Done Sent Done false
32921 10355 Acked Write 1 Done Sent Done false
32922 10356 Acked Write 4 Done Sent Done false
32923 10357 Acked Flush 0 Done Sent Done false
32924 STATES DS:0 DS:1 DS:2 TOTAL
32925 Sent 0 302 0 302
32926 Done 302 0 302 604
32927 Skipped 0 0 0 0
32928 Error 0 0 0 0
32929 Last Flush: 10357 10055 10357
32930 Downstairs last five completed: 10055 10054 10053 10052 10051
32931 Upstairs last five completed: 10357 10356 10355 10354 10353
32932 CLIENT: Up:0 ds:302 act:2
329332024-12-04T13:37:35.495ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
329342024-12-04T13:37:35.495ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
329352024-12-04T13:37:35.495ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
329362024-12-04T13:37:35.495ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
329372024-12-04T13:37:35.495ZINFOcrucible: client task is exiting = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
32938 [8820] got start action
32939 [8820] I am going to restart
32940 Starting downstairs at port 8820
32941 Make output file at "/tmp/dsc/downstairs-8820.txt"
32942 [8820][1] reports Starting
32943 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1234)
32944 [8820][1] reports Running
32945 ----------------------------------------------------------------
32946 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32947 States: Active Offline Active
32948 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32949 10056 Acked Write 2 Done Sent Done false
32950 10057 Acked Read 4 Done Sent Done false
32951 10058 Acked Read 8 Done Sent Done false
32952 10059 Acked Read 9 Done Sent Done false
32953 10060 Acked Read 5 Done Sent Done false
32954 10061 Acked Read 10 Done Sent Done false
32955 10062 Acked Read 7 Done Sent Done false
32956 10063 Acked Read 5 Done Sent Done false
32957 10064 Acked Read 10 Done Sent Done false
32958 10065 Acked Read 9 Done Sent Done false
32959 10066 Acked Read 2 Done Sent Done false
32960 10067 Acked Read 1 Done Sent Done false
32961 10068 Acked Write 6 Done Sent Done false
32962 10069 Acked Read 8 Done Sent Done false
32963 10070 Acked Read 5 Done Sent Done false
32964 10071 Acked Read 6 Done Sent Done false
32965 10072 Acked Read 4 Done Sent Done false
32966 10073 Acked Write 1 Done Sent Done false
32967 10074 Acked Read 1 Done Sent Done false
32968 10075 Acked Write 10 Done Sent Done false
32969 10076 Acked Flush 0 Done Sent Done false
32970 10077 Acked Read 7 Done Sent Done false
32971 10078 Acked Read 10 Done Sent Done false
32972 10079 Acked Write 3 Done Sent Done false
32973 10080 Acked Write 6 Done Sent Done false
32974 10081 Acked Read 6 Done Sent Done false
32975 10082 Acked Flush 0 Done Sent Done false
32976 10083 Acked Read 2 Done Sent Done false
32977 10084 Acked Read 5 Done Sent Done false
32978 10085 Acked Read 4 Done Sent Done false
32979 10086 Acked Write 1 Done Sent Done false
32980 10087 Acked Flush 0 Done Sent Done false
32981 10088 Acked Write 1 Done Sent Done false
32982 10089 Acked Write 3 Done Sent Done false
32983 10090 Acked Flush 0 Done Sent Done false
32984 10091 Acked Read 6 Done Sent Done false
32985 10092 Acked Write 4 Done Sent Done false
32986 10093 Acked Read 9 Done Sent Done false
32987 10094 Acked Write 8 Done Sent Done false
32988 10095 Acked Read 8 Done Sent Done false
32989 10096 Acked Read 10 Done Sent Done false
32990 10097 Acked Write 9 Done Sent Done false
32991 10098 Acked Read 5 Done Sent Done false
32992 10099 Acked Read 3 Done Sent Done false
32993 10100 Acked Read 9 Done Sent Done false
32994 10101 Acked Write 5 Done Sent Done false
32995 10102 Acked Flush 0 Done Sent Done false
32996 10103 Acked Write 6 Done Sent Done false
32997 10104 Acked Read 4 Done Sent Done false
32998 10105 Acked Read 5 Done Sent Done false
32999 10106 Acked Flush 0 Done Sent Done false
33000 10107 Acked Write 2 Done Sent Done false
33001 10108 Acked Write 1 Done Sent Done false
33002 10109 Acked Write 8 Done Sent Done false
33003 10110 Acked Flush 0 Done Sent Done false
33004 10111 Acked Write 4 Done Sent Done false
33005 10112 Acked Write 10 Done Sent Done false
33006 10113 Acked Read 3 Done Sent Done false
33007 10114 Acked Read 2 Done Sent Done false
33008 10115 Acked Read 9 Done Sent Done false
33009 10116 Acked Write 4 Done Sent Done false
33010 10117 Acked Flush 0 Done Sent Done false
33011 10118 Acked Flush 0 Done Sent Done false
33012 10119 Acked Read 3 Done Sent Done false
33013 10120 Acked Read 6 Done Sent Done false
33014 10121 Acked Read 2 Done Sent Done false
33015 10122 Acked Write 1 Done Sent Done false
33016 10123 Acked Write 4 Done Sent Done false
33017 10124 Acked Read 4 Done Sent Done false
33018 10125 Acked Read 2 Done Sent Done false
33019 10126 Acked Read 5 Done Sent Done false
33020 10127 Acked Read 9 Done Sent Done false
33021 10128 Acked Write 4 Done Sent Done false
33022 10129 Acked Read 3 Done Sent Done false
33023 10130 Acked Write 7 Done Sent Done false
33024 10131 Acked Write 10 Done Sent Done false
33025 10132 Acked Read 7 Done Sent Done false
33026 10133 Acked Write 9 Done Sent Done false
33027 10134 Acked Read 8 Done Sent Done false
33028 10135 Acked Write 8 Done Sent Done false
33029 10136 Acked Write 9 Done Sent Done false
33030 10137 Acked Write 8 Done Sent Done false
33031 10138 Acked Flush 0 Done Sent Done false
33032 10139 Acked Write 6 Done Sent Done false
33033 10140 Acked Write 10 Done Sent Done false
33034 10141 Acked Flush 0 Done Sent Done false
33035 10142 Acked Read 9 Done Sent Done false
33036 10143 Acked Write 4 Done Sent Done false
33037 10144 Acked Write 2 Done Sent Done false
33038 10145 Acked Read 4 Done Sent Done false
33039 10146 Acked Read 2 Done Sent Done false
33040 10147 Acked Read 5 Done Sent Done false
33041 10148 Acked Read 9 Done Sent Done false
33042 10149 Acked Flush 0 Done Sent Done false
33043 10150 Acked Flush 0 Done Sent Done false
33044 10151 Acked Read 5 Done Sent Done false
33045 10152 Acked Write 8 Done Sent Done false
33046 10153 Acked Flush 0 Done Sent Done false
33047 10154 Acked Flush 0 Done Sent Done false
33048 10155 Acked Read 4 Done Sent Done false
33049 10156 Acked Write 6 Done Sent Done false
33050 10157 Acked Write 10 Done Sent Done false
33051 10158 Acked Read 6 Done Sent Done false
33052 10159 Acked Write 1 Done Sent Done false
33053 10160 Acked Write 1 Done Sent Done false
33054 10161 Acked Write 10 Done Sent Done false
33055 10162 Acked Read 3 Done Sent Done false
33056 10163 Acked Write 10 Done Sent Done false
33057 10164 Acked Read 9 Done Sent Done false
33058 10165 Acked Read 7 Done Sent Done false
33059 10166 Acked Write 4 Done Sent Done false
33060 10167 Acked Read 1 Done Sent Done false
33061 10168 Acked Write 8 Done Sent Done false
33062 10169 Acked Write 5 Done Sent Done false
33063 10170 Acked Read 5 Done Sent Done false
33064 10171 Acked Read 4 Done Sent Done false
33065 10172 Acked Read 10 Done Sent Done false
33066 10173 Acked Read 4 Done Sent Done false
33067 10174 Acked Read 2 Done Sent Done false
33068 10175 Acked Write 6 Done Sent Done false
33069 10176 Acked Read 10 Done Sent Done false
33070 10177 Acked Read 3 Done Sent Done false
33071 10178 Acked Flush 0 Done Sent Done false
33072 10179 Acked Flush 0 Done Sent Done false
33073 10180 Acked Read 1 Done Sent Done false
33074 10181 Acked Read 5 Done Sent Done false
33075 10182 Acked Read 2 Done Sent Done false
33076 10183 Acked Write 2 Done Sent Done false
33077 10184 Acked Write 2 Done Sent Done false
33078 10185 Acked Read 10 Done Sent Done false
33079 10186 Acked Read 5 Done Sent Done false
33080 10187 Acked Flush 0 Done Sent Done false
33081 10188 Acked Read 8 Done Sent Done false
33082 10189 Acked Read 7 Done Sent Done false
33083 10190 Acked Write 3 Done Sent Done false
33084 10191 Acked Read 9 Done Sent Done false
33085 10192 Acked Write 1 Done Sent Done false
33086 10193 Acked Read 10 Done Sent Done false
33087 10194 Acked Flush 0 Done Sent Done false
33088 10195 Acked Write 9 Done Sent Done false
33089 10196 Acked Read 6 Done Sent Done false
33090 10197 Acked Read 8 Done Sent Done false
33091 10198 Acked Flush 0 Done Sent Done false
33092 10199 Acked Read 3 Done Sent Done false
33093 10200 Acked Read 1 Done Sent Done false
33094 10201 Acked Read 5 Done Sent Done false
33095 10202 Acked Read 10 Done Sent Done false
33096 10203 Acked Read 1 Done Sent Done false
33097 10204 Acked Read 6 Done Sent Done false
33098 10205 Acked Write 4 Done Sent Done false
33099 10206 Acked Write 9 Done Sent Done false
33100 10207 Acked Write 7 Done Sent Done false
33101 10208 Acked Write 2 Done Sent Done false
33102 10209 Acked Read 3 Done Sent Done false
33103 10210 Acked Write 3 Done Sent Done false
33104 10211 Acked Write 8 Done Sent Done false
33105 10212 Acked Read 9 Done Sent Done false
33106 10213 Acked Write 8 Done Sent Done false
33107 10214 Acked Read 5 Done Sent Done false
33108 10215 Acked Read 1 Done Sent Done false
33109 10216 Acked Write 1 Done Sent Done false
33110 10217 Acked Write 9 Done Sent Done false
33111 10218 Acked Read 2 Done Sent Done false
33112 10219 Acked Read 6 Done Sent Done false
33113 10220 Acked Write 7 Done Sent Done false
33114 10221 Acked Read 1 Done Sent Done false
33115 10222 Acked Write 7 Done Sent Done false
33116 10223 Acked Read 7 Done Sent Done false
33117 10224 Acked Read 6 Done Sent Done false
33118 10225 Acked Write 2 Done Sent Done false
33119 10226 Acked Flush 0 Done Sent Done false
33120 10227 Acked Read 2 Done Sent Done false
33121 10228 Acked Read 7 Done Sent Done false
33122 10229 Acked Read 1 Done Sent Done false
33123 10230 Acked Read 1 Done Sent Done false
33124 10231 Acked Read 10 Done Sent Done false
33125 10232 Acked Write 2 Done Sent Done false
33126 10233 Acked Read 4 Done Sent Done false
33127 10234 Acked Write 5 Done Sent Done false
33128 10235 Acked Read 2 Done Sent Done false
33129 10236 Acked Flush 0 Done Sent Done false
33130 10237 Acked Write 10 Done Sent Done false
33131 10238 Acked Write 1 Done Sent Done false
33132 10239 Acked Read 4 Done Sent Done false
33133 10240 Acked Read 3 Done Sent Done false
33134 10241 Acked Read 7 Done Sent Done false
33135 10242 Acked Read 8 Done Sent Done false
33136 10243 Acked Write 5 Done Sent Done false
33137 10244 Acked Write 3 Done Sent Done false
33138 10245 Acked Flush 0 Done Sent Done false
33139 10246 Acked Write 10 Done Sent Done false
33140 10247 Acked Read 2 Done Sent Done false
33141 10248 Acked Write 2 Done Sent Done false
33142 10249 Acked Write 8 Done Sent Done false
33143 10250 Acked Read 1 Done Sent Done false
33144 10251 Acked Read 3 Done Sent Done false
33145 10252 Acked Write 5 Done Sent Done false
33146 10253 Acked Read 4 Done Sent Done false
33147 10254 Acked Write 1 Done Sent Done false
33148 10255 Acked Read 5 Done Sent Done false
33149 10256 Acked Read 9 Done Sent Done false
33150 10257 Acked Read 5 Done Sent Done false
33151 10258 Acked Write 9 Done Sent Done false
33152 10259 Acked Write 7 Done Sent Done false
33153 10260 Acked Write 9 Done Sent Done false
33154 10261 Acked Write 10 Done Sent Done false
33155 10262 Acked Read 1 Done Sent Done false
33156 10263 Acked Read 5 Done Sent Done false
33157 10264 Acked Write 3 Done Sent Done false
33158 10265 Acked Read 8 Done Sent Done false
33159 10266 Acked Read 4 Done Sent Done false
33160 10267 Acked Write 8 Done Sent Done false
33161 10268 Acked Read 4 Done Sent Done false
33162 10269 Acked Write 9 Done Sent Done false
33163 10270 Acked Read 7 Done Sent Done false
33164 10271 Acked Write 2 Done Sent Done false
33165 10272 Acked Flush 0 Done Sent Done false
33166 10273 Acked Write 3 Done Sent Done false
33167 10274 Acked Flush 0 Done Sent Done false
33168 10275 Acked Read 7 Done Sent Done false
33169 10276 Acked Read 5 Done Sent Done false
33170 10277 Acked Read 8 Done Sent Done false
33171 10278 Acked Read 2 Done Sent Done false
33172 10279 Acked Read 1 Done Sent Done false
33173 10280 Acked Read 3 Done Sent Done false
33174 10281 Acked Read 9 Done Sent Done false
33175 10282 Acked Flush 0 Done Sent Done false
33176 10283 Acked Write 9 Done Sent Done false
33177 10284 Acked Read 3 Done Sent Done false
33178 10285 Acked Write 4 Done Sent Done false
33179 10286 Acked Flush 0 Done Sent Done false
33180 10287 Acked Read 9 Done Sent Done false
33181 10288 Acked Read 6 Done Sent Done false
33182 10289 Acked Write 8 Done Sent Done false
33183 10290 Acked Write 4 Done Sent Done false
33184 10291 Acked Flush 0 Done Sent Done false
33185 10292 Acked Write 6 Done Sent Done false
33186 10293 Acked Write 7 Done Sent Done false
33187 10294 Acked Write 2 Done Sent Done false
33188 10295 Acked Read 9 Done Sent Done false
33189 10296 Acked Write 9 Done Sent Done false
33190 10297 Acked Read 9 Done Sent Done false
33191 10298 Acked Read 9 Done Sent Done false
33192 10299 Acked Read 2 Done Sent Done false
33193 10300 Acked Write 8 Done Sent Done false
33194 10301 Acked Read 9 Done Sent Done false
33195 10302 Acked Read 8 Done Sent Done false
33196 10303 Acked Write 5 Done Sent Done false
33197 10304 Acked Write 10 Done Sent Done false
33198 10305 Acked Read 10 Done Sent Done false
33199 10306 Acked Read 9 Done Sent Done false
33200 10307 Acked Read 2 Done Sent Done false
33201 10308 Acked Flush 0 Done Sent Done false
33202 10309 Acked Read 7 Done Sent Done false
33203 10310 Acked Write 3 Done Sent Done false
33204 10311 Acked Flush 0 Done Sent Done false
33205 10312 Acked Read 7 Done Sent Done false
33206 10313 Acked Write 7 Done Sent Done false
33207 10314 Acked Write 4 Done Sent Done false
33208 10315 Acked Flush 0 Done Sent Done false
33209 10316 Acked Read 9 Done Sent Done false
33210 10317 Acked Read 8 Done Sent Done false
33211 10318 Acked Write 3 Done Sent Done false
33212 10319 Acked Read 8 Done Sent Done false
33213 10320 Acked Read 4 Done Sent Done false
33214 10321 Acked Write 6 Done Sent Done false
33215 10322 Acked Read 10 Done Sent Done false
33216 10323 Acked Write 10 Done Sent Done false
33217 10324 Acked Write 8 Done Sent Done false
33218 10325 Acked Read 7 Done Sent Done false
33219 10326 Acked Read 1 Done Sent Done false
33220 10327 Acked Write 5 Done Sent Done false
33221 10328 Acked Write 10 Done Sent Done false
33222 10329 Acked Read 2 Done Sent Done false
33223 10330 Acked Write 2 Done Sent Done false
33224 10331 Acked Read 4 Done Sent Done false
33225 10332 Acked Write 4 Done Sent Done false
33226 10333 Acked Read 5 Done Sent Done false
33227 10334 Acked Write 7 Done Sent Done false
33228 10335 Acked Flush 0 Done Sent Done false
33229 10336 Acked Read 8 Done Sent Done false
33230 10337 Acked Read 9 Done Sent Done false
33231 10338 Acked Write 8 Done Sent Done false
33232 10339 Acked Read 9 Done Sent Done false
33233 10340 Acked Write 1 Done Sent Done false
33234 10341 Acked Write 8 Done Sent Done false
33235 10342 Acked Write 8 Done Sent Done false
33236 10343 Acked Write 10 Done Sent Done false
33237 10344 Acked Read 2 Done Sent Done false
33238 10345 Acked Write 4 Done Sent Done false
33239 10346 Acked Read 1 Done Sent Done false
33240 10347 Acked Flush 0 Done Sent Done false
33241 10348 Acked Read 8 Done Sent Done false
33242 10349 Acked Read 4 Done Sent Done false
33243 10350 Acked Flush 0 Done Sent Done false
33244 10351 Acked Write 6 Done Sent Done false
33245 10352 Acked Read 10 Done Sent Done false
33246 10353 Acked Read 8 Done Sent Done false
33247 10354 Acked Write 4 Done Sent Done false
33248 10355 Acked Write 1 Done Sent Done false
33249 10356 Acked Write 4 Done Sent Done false
33250 10357 Acked Flush 0 Done Sent Done false
33251 STATES DS:0 DS:1 DS:2 TOTAL
33252 Sent 0 302 0 302
33253 Done 302 0 302 604
33254 Skipped 0 0 0 0
33255 Error 0 0 0 0
33256 Last Flush: 10357 10055 10357
33257 Downstairs last five completed: 10055 10054 10053 10052 10051
33258 Upstairs last five completed: 10357 10356 10355 10354 10353
33259 CLIENT: Up:0 ds:302 act:2
33260 ----------------------------------------------------------------
33261 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33262 States: Active Offline Active
33263 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33264 10056 Acked Write 2 Done Sent Done false
33265 10057 Acked Read 4 Done Sent Done false
33266 10058 Acked Read 8 Done Sent Done false
33267 10059 Acked Read 9 Done Sent Done false
33268 10060 Acked Read 5 Done Sent Done false
33269 10061 Acked Read 10 Done Sent Done false
33270 10062 Acked Read 7 Done Sent Done false
33271 10063 Acked Read 5 Done Sent Done false
33272 10064 Acked Read 10 Done Sent Done false
33273 10065 Acked Read 9 Done Sent Done false
33274 10066 Acked Read 2 Done Sent Done false
33275 10067 Acked Read 1 Done Sent Done false
33276 10068 Acked Write 6 Done Sent Done false
33277 10069 Acked Read 8 Done Sent Done false
33278 10070 Acked Read 5 Done Sent Done false
33279 10071 Acked Read 6 Done Sent Done false
33280 10072 Acked Read 4 Done Sent Done false
33281 10073 Acked Write 1 Done Sent Done false
33282 10074 Acked Read 1 Done Sent Done false
33283 10075 Acked Write 10 Done Sent Done false
33284 10076 Acked Flush 0 Done Sent Done false
33285 10077 Acked Read 7 Done Sent Done false
33286 10078 Acked Read 10 Done Sent Done false
33287 10079 Acked Write 3 Done Sent Done false
33288 10080 Acked Write 6 Done Sent Done false
33289 10081 Acked Read 6 Done Sent Done false
33290 10082 Acked Flush 0 Done Sent Done false
33291 10083 Acked Read 2 Done Sent Done false
33292 10084 Acked Read 5 Done Sent Done false
33293 10085 Acked Read 4 Done Sent Done false
33294 10086 Acked Write 1 Done Sent Done false
33295 10087 Acked Flush 0 Done Sent Done false
33296 10088 Acked Write 1 Done Sent Done false
33297 10089 Acked Write 3 Done Sent Done false
33298 10090 Acked Flush 0 Done Sent Done false
33299 10091 Acked Read 6 Done Sent Done false
33300 10092 Acked Write 4 Done Sent Done false
33301 10093 Acked Read 9 Done Sent Done false
33302 10094 Acked Write 8 Done Sent Done false
33303 10095 Acked Read 8 Done Sent Done false
33304 10096 Acked Read 10 Done Sent Done false
33305 10097 Acked Write 9 Done Sent Done false
33306 10098 Acked Read 5 Done Sent Done false
33307 10099 Acked Read 3 Done Sent Done false
33308 10100 Acked Read 9 Done Sent Done false
33309 10101 Acked Write 5 Done Sent Done false
33310 10102 Acked Flush 0 Done Sent Done false
33311 10103 Acked Write 6 Done Sent Done false
33312 10104 Acked Read 4 Done Sent Done false
33313 10105 Acked Read 5 Done Sent Done false
33314 10106 Acked Flush 0 Done Sent Done false
33315 10107 Acked Write 2 Done Sent Done false
33316 10108 Acked Write 1 Done Sent Done false
33317 10109 Acked Write 8 Done Sent Done false
33318 10110 Acked Flush 0 Done Sent Done false
33319 10111 Acked Write 4 Done Sent Done false
33320 10112 Acked Write 10 Done Sent Done false
33321 10113 Acked Read 3 Done Sent Done false
33322 10114 Acked Read 2 Done Sent Done false
33323 10115 Acked Read 9 Done Sent Done false
33324 10116 Acked Write 4 Done Sent Done false
33325 10117 Acked Flush 0 Done Sent Done false
33326 10118 Acked Flush 0 Done Sent Done false
33327 10119 Acked Read 3 Done Sent Done false
33328 10120 Acked Read 6 Done Sent Done false
33329 10121 Acked Read 2 Done Sent Done false
33330 10122 Acked Write 1 Done Sent Done false
33331 10123 Acked Write 4 Done Sent Done false
33332 10124 Acked Read 4 Done Sent Done false
33333 10125 Acked Read 2 Done Sent Done false
33334 10126 Acked Read 5 Done Sent Done false
33335 10127 Acked Read 9 Done Sent Done false
33336 10128 Acked Write 4 Done Sent Done false
33337 10129 Acked Read 3 Done Sent Done false
33338 10130 Acked Write 7 Done Sent Done false
33339 10131 Acked Write 10 Done Sent Done false
33340 10132 Acked Read 7 Done Sent Done false
33341 10133 Acked Write 9 Done Sent Done false
33342 10134 Acked Read 8 Done Sent Done false
33343 10135 Acked Write 8 Done Sent Done false
33344 10136 Acked Write 9 Done Sent Done false
33345 10137 Acked Write 8 Done Sent Done false
33346 10138 Acked Flush 0 Done Sent Done false
33347 10139 Acked Write 6 Done Sent Done false
33348 10140 Acked Write 10 Done Sent Done false
33349 10141 Acked Flush 0 Done Sent Done false
33350 10142 Acked Read 9 Done Sent Done false
33351 10143 Acked Write 4 Done Sent Done false
33352 10144 Acked Write 2 Done Sent Done false
33353 10145 Acked Read 4 Done Sent Done false
33354 10146 Acked Read 2 Done Sent Done false
33355 10147 Acked Read 5 Done Sent Done false
33356 10148 Acked Read 9 Done Sent Done false
33357 10149 Acked Flush 0 Done Sent Done false
33358 10150 Acked Flush 0 Done Sent Done false
33359 10151 Acked Read 5 Done Sent Done false
33360 10152 Acked Write 8 Done Sent Done false
33361 10153 Acked Flush 0 Done Sent Done false
33362 10154 Acked Flush 0 Done Sent Done false
33363 10155 Acked Read 4 Done Sent Done false
33364 10156 Acked Write 6 Done Sent Done false
33365 10157 Acked Write 10 Done Sent Done false
33366 10158 Acked Read 6 Done Sent Done false
33367 10159 Acked Write 1 Done Sent Done false
33368 10160 Acked Write 1 Done Sent Done false
33369 10161 Acked Write 10 Done Sent Done false
33370 10162 Acked Read 3 Done Sent Done false
33371 10163 Acked Write 10 Done Sent Done false
33372 10164 Acked Read 9 Done Sent Done false
33373 10165 Acked Read 7 Done Sent Done false
33374 10166 Acked Write 4 Done Sent Done false
33375 10167 Acked Read 1 Done Sent Done false
33376 10168 Acked Write 8 Done Sent Done false
33377 10169 Acked Write 5 Done Sent Done false
33378 10170 Acked Read 5 Done Sent Done false
33379 10171 Acked Read 4 Done Sent Done false
33380 10172 Acked Read 10 Done Sent Done false
33381 10173 Acked Read 4 Done Sent Done false
33382 10174 Acked Read 2 Done Sent Done false
33383 10175 Acked Write 6 Done Sent Done false
33384 10176 Acked Read 10 Done Sent Done false
33385 10177 Acked Read 3 Done Sent Done false
33386 10178 Acked Flush 0 Done Sent Done false
33387 10179 Acked Flush 0 Done Sent Done false
33388 10180 Acked Read 1 Done Sent Done false
33389 10181 Acked Read 5 Done Sent Done false
33390 10182 Acked Read 2 Done Sent Done false
33391 10183 Acked Write 2 Done Sent Done false
33392 10184 Acked Write 2 Done Sent Done false
33393 10185 Acked Read 10 Done Sent Done false
33394 10186 Acked Read 5 Done Sent Done false
33395 10187 Acked Flush 0 Done Sent Done false
33396 10188 Acked Read 8 Done Sent Done false
33397 10189 Acked Read 7 Done Sent Done false
33398 10190 Acked Write 3 Done Sent Done false
33399 10191 Acked Read 9 Done Sent Done false
33400 10192 Acked Write 1 Done Sent Done false
33401 10193 Acked Read 10 Done Sent Done false
33402 10194 Acked Flush 0 Done Sent Done false
33403 10195 Acked Write 9 Done Sent Done false
33404 10196 Acked Read 6 Done Sent Done false
33405 10197 Acked Read 8 Done Sent Done false
33406 10198 Acked Flush 0 Done Sent Done false
33407 10199 Acked Read 3 Done Sent Done false
33408 10200 Acked Read 1 Done Sent Done false
33409 10201 Acked Read 5 Done Sent Done false
33410 10202 Acked Read 10 Done Sent Done false
33411 10203 Acked Read 1 Done Sent Done false
33412 10204 Acked Read 6 Done Sent Done false
33413 10205 Acked Write 4 Done Sent Done false
33414 10206 Acked Write 9 Done Sent Done false
33415 10207 Acked Write 7 Done Sent Done false
33416 10208 Acked Write 2 Done Sent Done false
33417 10209 Acked Read 3 Done Sent Done false
33418 10210 Acked Write 3 Done Sent Done false
33419 10211 Acked Write 8 Done Sent Done false
33420 10212 Acked Read 9 Done Sent Done false
33421 10213 Acked Write 8 Done Sent Done false
33422 10214 Acked Read 5 Done Sent Done false
33423 10215 Acked Read 1 Done Sent Done false
33424 10216 Acked Write 1 Done Sent Done false
33425 10217 Acked Write 9 Done Sent Done false
33426 10218 Acked Read 2 Done Sent Done false
33427 10219 Acked Read 6 Done Sent Done false
33428 10220 Acked Write 7 Done Sent Done false
33429 10221 Acked Read 1 Done Sent Done false
33430 10222 Acked Write 7 Done Sent Done false
33431 10223 Acked Read 7 Done Sent Done false
33432 10224 Acked Read 6 Done Sent Done false
33433 10225 Acked Write 2 Done Sent Done false
33434 10226 Acked Flush 0 Done Sent Done false
33435 10227 Acked Read 2 Done Sent Done false
33436 10228 Acked Read 7 Done Sent Done false
33437 10229 Acked Read 1 Done Sent Done false
33438 10230 Acked Read 1 Done Sent Done false
33439 10231 Acked Read 10 Done Sent Done false
33440 10232 Acked Write 2 Done Sent Done false
33441 10233 Acked Read 4 Done Sent Done false
33442 10234 Acked Write 5 Done Sent Done false
33443 10235 Acked Read 2 Done Sent Done false
33444 10236 Acked Flush 0 Done Sent Done false
33445 10237 Acked Write 10 Done Sent Done false
33446 10238 Acked Write 1 Done Sent Done false
33447 10239 Acked Read 4 Done Sent Done false
33448 10240 Acked Read 3 Done Sent Done false
33449 10241 Acked Read 7 Done Sent Done false
33450 10242 Acked Read 8 Done Sent Done false
33451 10243 Acked Write 5 Done Sent Done false
33452 10244 Acked Write 3 Done Sent Done false
33453 10245 Acked Flush 0 Done Sent Done false
33454 10246 Acked Write 10 Done Sent Done false
33455 10247 Acked Read 2 Done Sent Done false
33456 10248 Acked Write 2 Done Sent Done false
33457 10249 Acked Write 8 Done Sent Done false
33458 10250 Acked Read 1 Done Sent Done false
33459 10251 Acked Read 3 Done Sent Done false
33460 10252 Acked Write 5 Done Sent Done false
33461 10253 Acked Read 4 Done Sent Done false
33462 10254 Acked Write 1 Done Sent Done false
33463 10255 Acked Read 5 Done Sent Done false
33464 10256 Acked Read 9 Done Sent Done false
33465 10257 Acked Read 5 Done Sent Done false
33466 10258 Acked Write 9 Done Sent Done false
33467 10259 Acked Write 7 Done Sent Done false
33468 10260 Acked Write 9 Done Sent Done false
33469 10261 Acked Write 10 Done Sent Done false
33470 10262 Acked Read 1 Done Sent Done false
33471 10263 Acked Read 5 Done Sent Done false
33472 10264 Acked Write 3 Done Sent Done false
33473 10265 Acked Read 8 Done Sent Done false
33474 10266 Acked Read 4 Done Sent Done false
33475 10267 Acked Write 8 Done Sent Done false
33476 10268 Acked Read 4 Done Sent Done false
33477 10269 Acked Write 9 Done Sent Done false
33478 10270 Acked Read 7 Done Sent Done false
33479 10271 Acked Write 2 Done Sent Done false
33480 10272 Acked Flush 0 Done Sent Done false
33481 10273 Acked Write 3 Done Sent Done false
33482 10274 Acked Flush 0 Done Sent Done false
33483 10275 Acked Read 7 Done Sent Done false
33484 10276 Acked Read 5 Done Sent Done false
33485 10277 Acked Read 8 Done Sent Done false
33486 10278 Acked Read 2 Done Sent Done false
33487 10279 Acked Read 1 Done Sent Done false
33488 10280 Acked Read 3 Done Sent Done false
33489 10281 Acked Read 9 Done Sent Done false
33490 10282 Acked Flush 0 Done Sent Done false
33491 10283 Acked Write 9 Done Sent Done false
33492 10284 Acked Read 3 Done Sent Done false
33493 10285 Acked Write 4 Done Sent Done false
33494 10286 Acked Flush 0 Done Sent Done false
33495 10287 Acked Read 9 Done Sent Done false
33496 10288 Acked Read 6 Done Sent Done false
33497 10289 Acked Write 8 Done Sent Done false
33498 10290 Acked Write 4 Done Sent Done false
33499 10291 Acked Flush 0 Done Sent Done false
33500 10292 Acked Write 6 Done Sent Done false
33501 10293 Acked Write 7 Done Sent Done false
33502 10294 Acked Write 2 Done Sent Done false
33503 10295 Acked Read 9 Done Sent Done false
33504 10296 Acked Write 9 Done Sent Done false
33505 10297 Acked Read 9 Done Sent Done false
33506 10298 Acked Read 9 Done Sent Done false
33507 10299 Acked Read 2 Done Sent Done false
33508 10300 Acked Write 8 Done Sent Done false
33509 10301 Acked Read 9 Done Sent Done false
33510 10302 Acked Read 8 Done Sent Done false
33511 10303 Acked Write 5 Done Sent Done false
33512 10304 Acked Write 10 Done Sent Done false
33513 10305 Acked Read 10 Done Sent Done false
33514 10306 Acked Read 9 Done Sent Done false
33515 10307 Acked Read 2 Done Sent Done false
33516 10308 Acked Flush 0 Done Sent Done false
33517 10309 Acked Read 7 Done Sent Done false
33518 10310 Acked Write 3 Done Sent Done false
33519 10311 Acked Flush 0 Done Sent Done false
33520 10312 Acked Read 7 Done Sent Done false
33521 10313 Acked Write 7 Done Sent Done false
33522 10314 Acked Write 4 Done Sent Done false
33523 10315 Acked Flush 0 Done Sent Done false
33524 10316 Acked Read 9 Done Sent Done false
33525 10317 Acked Read 8 Done Sent Done false
33526 10318 Acked Write 3 Done Sent Done false
33527 10319 Acked Read 8 Done Sent Done false
33528 10320 Acked Read 4 Done Sent Done false
33529 10321 Acked Write 6 Done Sent Done false
33530 10322 Acked Read 10 Done Sent Done false
33531 10323 Acked Write 10 Done Sent Done false
33532 10324 Acked Write 8 Done Sent Done false
33533 10325 Acked Read 7 Done Sent Done false
33534 10326 Acked Read 1 Done Sent Done false
33535 10327 Acked Write 5 Done Sent Done false
33536 10328 Acked Write 10 Done Sent Done false
33537 10329 Acked Read 2 Done Sent Done false
33538 10330 Acked Write 2 Done Sent Done false
33539 10331 Acked Read 4 Done Sent Done false
33540 10332 Acked Write 4 Done Sent Done false
33541 10333 Acked Read 5 Done Sent Done false
33542 10334 Acked Write 7 Done Sent Done false
33543 10335 Acked Flush 0 Done Sent Done false
33544 10336 Acked Read 8 Done Sent Done false
33545 10337 Acked Read 9 Done Sent Done false
33546 10338 Acked Write 8 Done Sent Done false
33547 10339 Acked Read 9 Done Sent Done false
33548 10340 Acked Write 1 Done Sent Done false
33549 10341 Acked Write 8 Done Sent Done false
33550 10342 Acked Write 8 Done Sent Done false
33551 10343 Acked Write 10 Done Sent Done false
33552 10344 Acked Read 2 Done Sent Done false
33553 10345 Acked Write 4 Done Sent Done false
33554 10346 Acked Read 1 Done Sent Done false
33555 10347 Acked Flush 0 Done Sent Done false
33556 10348 Acked Read 8 Done Sent Done false
33557 10349 Acked Read 4 Done Sent Done false
33558 10350 Acked Flush 0 Done Sent Done false
33559 10351 Acked Write 6 Done Sent Done false
33560 10352 Acked Read 10 Done Sent Done false
33561 10353 Acked Read 8 Done Sent Done false
33562 10354 Acked Write 4 Done Sent Done false
33563 10355 Acked Write 1 Done Sent Done false
33564 10356 Acked Write 4 Done Sent Done false
33565 10357 Acked Flush 0 Done Sent Done false
33566 STATES DS:0 DS:1 DS:2 TOTAL
33567 Sent 0 302 0 302
33568 Done 302 0 302 604
33569 Skipped 0 0 0 0
33570 Error 0 0 0 0
33571 Last Flush: 10357 10055 10357
33572 Downstairs last five completed: 10055 10054 10053 10052 10051
33573 Upstairs last five completed: 10357 10356 10355 10354 10353
33574 CLIENT: Up:0 ds:302 act:2
335752024-12-04T13:37:45.496ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335762024-12-04T13:37:45.496ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36981) = io task client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335772024-12-04T13:37:45.497ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335782024-12-04T13:37:45.497ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335792024-12-04T13:37:45.497ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335802024-12-04T13:37:45.497ZINFOcrucible: send last flush ID to this DS: 10055 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335812024-12-04T13:37:45.498ZINFOcrucible: Replied this last flush ID: 10055 = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335822024-12-04T13:37:45.498ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335832024-12-04T13:37:45.498ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335842024-12-04T13:37:45.498ZINFOcrucible: [1] client re-new 302 jobs since flush 10055 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
335852024-12-04T13:37:45.498ZINFOcrucible: [1] Replayed 302 jobs since flush: 10055 = downstairs session_id = 01834428-904b-4e36-8fe6-2e6a94afb6ff
33586 ----------------------------------------------------------------
33587 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33588 Downstairs last five completed: 10357 10356 10355 10354 10353
33589 Upstairs last five completed: 10357 10356 10355 10354 10353
33590 CLIENT: Up:0 ds:0 act:3
33591 Replay: All jobs finished
33592 Test replay has completed
33593 Wrote out file "/tmp/test_replay_verify.log"
33594 CLIENT: Tests done. All submitted work has been ACK'd
33595 ----------------------------------------------------------------
33596 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33597 Downstairs last five completed: 10357 10356 10355 10354 10353
33598 Upstairs last five completed: 10357 10356 10355 10354 10353
33599 CLIENT: Up:0 ds:0 act:3
33600 CLIENT: All jobs finished, all DS active.
33601 Replays:30 time: 6:59
33602 Do final verify
33603 Dec 04 13:37:45.821 INFO accepted connection, remote_addr: 127.0.0.1:51174, local_addr: 127.0.0.1:9998
33604 Dec 04 13:37:45.821 INFO request completed, latency_us: 192, response_code: 200, uri: /regioninfo, method: GET, req_id: e82e64fb-0cad-4687-af18-6a0b131e8d86, remote_addr: 127.0.0.1:51174, local_addr: 127.0.0.1:9998
336052024-12-04T13:37:45.822ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
33606 Dec 04 13:37:45.822 INFO request completed, latency_us: 86, response_code: 200, uri: /regioncount, method: GET, req_id: 8d8f4fb0-6838-47b2-b658-dfed206aa8da, remote_addr: 127.0.0.1:51174, local_addr: 127.0.0.1:9998
336072024-12-04T13:37:45.822ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
33608 Dec 04 13:37:45.823 INFO request completed, latency_us: 108, response_code: 200, uri: /port/cid/0, method: GET, req_id: 1ce7bd3a-a17a-400c-9a67-ce080c8d71e0, remote_addr: 127.0.0.1:51174, local_addr: 127.0.0.1:9998
33609 Dec 04 13:37:45.823 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/1, method: GET, req_id: ce447981-d388-4de0-a675-89012ecba1cc, remote_addr: 127.0.0.1:51174, local_addr: 127.0.0.1:9998
33610 Dec 04 13:37:45.824 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/2, method: GET, req_id: 45fbc5b5-0acf-4e48-9eec-b004574fa101, remote_addr: 127.0.0.1:51174, local_addr: 127.0.0.1:9998
336112024-12-04T13:37:45.824ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
336122024-12-04T13:37:45.824ZINFOcrucible: Upstairs starts
336132024-12-04T13:37:45.824ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "0e1f7f60aff9f26b0d737c97120445edbded1ad3", git_commit_timestamp: "2024-12-04T13:14:36.000000000Z", git_branch: "renovate/tokio-util-0.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
336142024-12-04T13:37:45.824ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
336152024-12-04T13:37:45.824ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
336162024-12-04T13:37:45.825ZINFOcrucible: Crucible 4729a4f7-6355-4e9e-8802-5f214be11210 has session id: 4dd988e3-133c-4d74-9dd8-7ada3db6a131 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336172024-12-04T13:37:45.825ZINFOcrucible: Upstairs opts: Upstairs UUID: 4729a4f7-6355-4e9e-8802-5f214be11210, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336182024-12-04T13:37:45.825ZINFOcrucible: Crucible stats registered with UUID: 4729a4f7-6355-4e9e-8802-5f214be11210 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336192024-12-04T13:37:45.858ZINFOcrucible: Downstairs in volume = 3 task = crutest
336202024-12-04T13:37:45.858ZINFOcrucible: The guest has requested activation with gen:3
336212024-12-04T13:37:45.858ZINFOcrucible: 4729a4f7-6355-4e9e-8802-5f214be11210 active request set session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336222024-12-04T13:37:45.858ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336232024-12-04T13:37:45.858ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336242024-12-04T13:37:45.858ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336252024-12-04T13:37:45.858ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336262024-12-04T13:37:45.858ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336272024-12-04T13:37:45.858ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336282024-12-04T13:37:45.858ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336292024-12-04T13:37:45.858ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336302024-12-04T13:37:45.858ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336312024-12-04T13:37:45.858ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41586) = io task client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336322024-12-04T13:37:45.858ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54355) = io task client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336332024-12-04T13:37:45.859ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40754) = io task client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336342024-12-04T13:37:45.859ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336352024-12-04T13:37:45.859ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336362024-12-04T13:37:45.859ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336372024-12-04T13:37:45.859ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336382024-12-04T13:37:45.859ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336392024-12-04T13:37:45.859ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336402024-12-04T13:37:45.860ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336412024-12-04T13:37:45.860ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336422024-12-04T13:37:45.860ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336432024-12-04T13:37:45.861ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336442024-12-04T13:37:45.861ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336452024-12-04T13:37:45.861ZINFOcrucible: Waiting for more clients to be ready session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336462024-12-04T13:37:45.861ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336472024-12-04T13:37:45.861ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336482024-12-04T13:37:45.861ZINFOcrucible: Waiting for more clients to be ready session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336492024-12-04T13:37:45.861ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336502024-12-04T13:37:45.861ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336512024-12-04T13:37:45.861ZINFOcrucible: [0]R flush_numbers[0..12]: [881, 915, 893, 910, 888, 901, 914, 912, 907, 893, 915, 916] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336522024-12-04T13:37:45.861ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336532024-12-04T13:37:45.861ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336542024-12-04T13:37:45.861ZINFOcrucible: [1]R flush_numbers[0..12]: [881, 915, 893, 910, 888, 901, 914, 912, 907, 893, 915, 916] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336552024-12-04T13:37:45.861ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336562024-12-04T13:37:45.861ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336572024-12-04T13:37:45.861ZINFOcrucible: [2]R flush_numbers[0..12]: [881, 915, 893, 910, 888, 901, 914, 912, 907, 893, 915, 916] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336582024-12-04T13:37:45.861ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336592024-12-04T13:37:45.861ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336602024-12-04T13:37:45.861ZINFOcrucible: Max found gen is 3 = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
33661 {"msg":"Generation requested: 3 >= found:3","v":0,"name":"crucible","level":30,"time":"2024-12-04T13:37:45.861375839Z","hostname":"w-01JE8XRQYRPEW5CK8FQV8GJNJ9","pid":1239,"":"downstairs","":"downstairs","session_idWait for a query_work_queue command to finish before sending IO
33662 ":"4dd988e3-133c-4d74-9dd8-7ada3db6a131"}
336632024-12-04T13:37:45.861ZINFOcrucible: Next flush: 919 = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336642024-12-04T13:37:45.861ZINFOcrucible: All extents match = downstairs session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336652024-12-04T13:37:45.861ZINFOcrucible: No downstairs reconciliation required session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336662024-12-04T13:37:45.861ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336672024-12-04T13:37:45.861ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336682024-12-04T13:37:45.861ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336692024-12-04T13:37:45.861ZINFOcrucible: All required reconciliation work is completed session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336702024-12-04T13:37:45.861ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336712024-12-04T13:37:45.861ZINFOcrucible: 4729a4f7-6355-4e9e-8802-5f214be11210 is now active with session: 4dd988e3-133c-4d74-9dd8-7ada3db6a131 session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336722024-12-04T13:37:45.861ZINFOcrucible: Set Active after no reconciliation session_id = 4dd988e3-133c-4d74-9dd8-7ada3db6a131
336732024-12-04T13:37:45.861ZINFOcrucible: The guest has finished waiting for activation with:3
33674 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
33675 Loading write count information from file "/tmp/test_replay_verify.log"
33676 Read and Verify all blocks (0..5000 range:false)
33677 Wrote out file "/tmp/test_replay_verify.log"
33678 Verify test completed
33679 Wrote out file "/tmp/test_replay_verify.log"
33680 CLIENT: Tests done. All submitted work has been ACK'd
33681 ----------------------------------------------------------------
33682 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
33683 States: Active Active Active
33684 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33685 1000 Acked Read 256 Done Done Done false
33686 1001 Acked Read 256 Done Done Done false
33687 1002 Acked Read 256 Done Done Done false
33688 1003 Acked Read 256 Done Done Done false
33689 1004 Acked Read 256 Done Done Done false
33690 1005 Acked Read 256 Done Done Done false
33691 1006 Acked Read 256 Done Done Done false
33692 1007 Acked Read 256 Done Done Done false
33693 1008 Acked Read 256 Done Done Done false
33694 1009 Acked Read 256 Done Done Done false
33695 1010 Acked Read 256 Done Done Done false
33696 1011 Acked Read 256 Done Done Done false
33697 1012 Acked Read 256 Done Done Done false
33698 1013 Acked Read 256 Done Done Done false
33699 1014 Acked Read 256 Done Done Done false
33700 1015 Acked Read 256 Done Done Done false
33701 1016 Acked Read 256 Done Done Done false
33702 1017 Acked Read 256 Done Done Done false
33703 1018 Acked Read 256 Done Done Done false
33704 1019 Acked Read 136 Done Done Done false
33705 STATES DS:0 DS:1 DS:2 TOTAL
33706 Sent 0 0 0 0
33707 Done 20 20 20 60
33708 Skipped 0 0 0 0
33709 Error 0 0 0 0
33710 Last Flush: 0 0 0
33711 Downstairs last five completed:
33712 Upstairs last five completed: 1019 1018 1017 1016 1015
33713 CLIENT: Up:0 ds:20 act:3
33714 ----------------------------------------------------------------
33715 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
33716 Downstairs last five completed: 1020 1019 1018 1017 1016
33717 Upstairs last five completed: 1020 1019 1018 1017 1016
33718 CLIENT: Up:0 ds:0 act:3
33719 CLIENT: All crucible jobs finished, exiting program
337202024-12-04T13:37:50.266ZWARNcrucible: Guest handle has been dropped
33721 Dec 04 13:37:50.329 INFO accepted connection, remote_addr: 127.0.0.1:43970, local_addr: 127.0.0.1:9998
33722 Main task has work to do, go find it
33723 got dsc Shutdown
33724 Shutdown
33725 Shut it down
33726 Dec 04 13:37:50.330 INFO request completed, latency_us: 152, response_code[8810] Disable keep_running
33727 :[8810] Got stop action so:false kr:false
33728 204, uri:[8820] Disable keep_running
33729 [8820] Got stop action so:false kr:false
33730 /shutdown[8830] Disable keep_running
33731 , [8830] Got stop action so:false kr:false
33732 method: POST, req_id: 9ca41eb5-3c09-40c1-aad5-138a92d0f16d, remote_addr: 127.0.0.1:43970, local_addr: 127.0.0.1:9998
33733 [8820] Exited with: ExitStatus(unix_wait_status(9))
33734 [8820][1] reports Exit
33735 [8810] Exited with: ExitStatus(unix_wait_status(9))
33736 [8810][0] reports Exit
33737 [8830] Exited with: ExitStatus(unix_wait_status(9))
33738 [8830][2] reports Exit
33739 December 4, 2024 at 01:37:55 PM UTC Test ends with 0