Max level shown:
1 starting January 11, 2025 at 06:05:20 AM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.25954694
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.022724718
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.022210604
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8820
17 Starting downstairs at port 8810
18 Make output file at "/tmp/dsc/downstairs-8820.txt"
19 Starting downstairs at port 8830
20 Make output file at "/tmp/dsc/downstairs-8810.txt"
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8820][1] initial start wait reports Starting
23 [8810][0] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1194)
26 [8820][1] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1195)
28 [8830][2] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1196)
30 [8810][0] initial start wait reports Running
31 All downstairs are running
32 Jan 11 06:05:20.563 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 11 06:05:25.660 INFO accepted connection, remote_addr: 127.0.0.1:37830, local_addr: 127.0.0.1:9998
36 Jan 11 06:05:25.662 INFO request completed, latency_us: 470, response_code: 200, uri: /regioninfo, method: GET, req_id: 570131ad-169b-4be8-87f5-f28d2ca4593f, remote_addr: 127.0.0.1:37830, local_addr: 127.0.0.1:9998
37 Jan 11 06:05:25.665 INFO request completed, latency_us: 110, response_code: 200, uri: /regioncount, method: GET, req_id: 805b0659-4097-4d0b-8eed-4da32d74ec64, remote_addr: 127.0.0.1:37830, local_addr: 127.0.0.1:9998
382025-01-11T06:05:25.664ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-11T06:05:25.665ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 11 06:05:25.666 INFO request completed, latency_us: 147, response_code: 200, uri: /port/cid/0, method: GET, req_id: ea124fad-fc8e-4f47-a4c6-0b8adb952d0a, remote_addr: 127.0.0.1:37830, local_addr: 127.0.0.1:9998
41 Jan 11 06:05:25.666 INFO request completed, latency_us: 90, response_code: 200, uri: /port/cid/1, method: GET, req_id: 3d9d6518-4394-4b1d-9a6f-dd105f5e43c2, remote_addr: 127.0.0.1:37830, local_addr: 127.0.0.1:9998
42 Jan 11 06:05:25.667 INFO request completed, latency_us: 90, response_code: 200, uri: /port/cid/2, method: GET, req_id: 19373135-e77a-4f2e-92ea-68011d0c68c3, remote_addr: 127.0.0.1:37830, local_addr: 127.0.0.1:9998
432025-01-11T06:05:25.667ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-11T06:05:25.669ZINFOcrucible: Upstairs starts
452025-01-11T06:05:25.669ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "afbd0ec341edbe8e076286ae0030d2abadb28437", git_commit_timestamp: "2025-01-11T05:49:56.000000000Z", git_branch: "renovate/opentelemetry-0.x", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-11T06:05:25.669ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-11T06:05:25.669ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-11T06:05:25.670ZINFOcrucible: Crucible a639371d-0191-4dc0-883e-cbac9e49f2f2 has session id: 3cee1d12-f70c-4666-81eb-a8898bec9056 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
492025-01-11T06:05:25.670ZINFOcrucible: Upstairs opts: Upstairs UUID: a639371d-0191-4dc0-883e-cbac9e49f2f2, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
502025-01-11T06:05:25.670ZINFOcrucible: Crucible stats registered with UUID: a639371d-0191-4dc0-883e-cbac9e49f2f2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
512025-01-11T06:05:25.705ZINFOcrucible: Downstairs in volume = 3 task = crutest
522025-01-11T06:05:25.705ZINFOcrucible: The guest has requested activation with gen:1
532025-01-11T06:05:25.705ZINFOcrucible: a639371d-0191-4dc0-883e-cbac9e49f2f2 active request set session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
542025-01-11T06:05:25.705ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
552025-01-11T06:05:25.706ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
562025-01-11T06:05:25.706ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
572025-01-11T06:05:25.706ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
582025-01-11T06:05:25.706ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
592025-01-11T06:05:25.706ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
602025-01-11T06:05:25.706ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
612025-01-11T06:05:25.706ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
622025-01-11T06:05:25.706ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
632025-01-11T06:05:25.706ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48070) = io task client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
642025-01-11T06:05:25.706ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39312) = io task client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
652025-01-11T06:05:25.706ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40111) = io task client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
662025-01-11T06:05:25.708ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
672025-01-11T06:05:25.708ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
682025-01-11T06:05:25.708ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
692025-01-11T06:05:25.708ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
702025-01-11T06:05:25.708ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
712025-01-11T06:05:25.708ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
722025-01-11T06:05:25.709ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
732025-01-11T06:05:25.709ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
742025-01-11T06:05:25.709ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
752025-01-11T06:05:25.710ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
762025-01-11T06:05:25.710ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
772025-01-11T06:05:25.710ZINFOcrucible: Waiting for more clients to be ready session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
782025-01-11T06:05:25.710ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
792025-01-11T06:05:25.710ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
802025-01-11T06:05:25.710ZINFOcrucible: Waiting for more clients to be ready session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
812025-01-11T06:05:25.710ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
822025-01-11T06:05:25.710ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
832025-01-11T06:05:25.710ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
842025-01-11T06:05:25.710ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
852025-01-11T06:05:25.710ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
862025-01-11T06:05:25.710ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
872025-01-11T06:05:25.710ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
882025-01-11T06:05:25.710ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
892025-01-11T06:05:25.710ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
902025-01-11T06:05:25.710ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
912025-01-11T06:05:25.710ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
922025-01-11T06:05:25.710ZINFOcrucible: Max found gen is 1 = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
932025-01-11T06:05:25.710ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
942025-01-11T06:05:25.710ZINFOcrucible: Next flush: 1 = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
952025-01-11T06:05:25.711ZINFOcrucible: All extents match = downstairs session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
962025-01-11T06:05:25.711ZINFOcrucible: No downstairs reconciliation required session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
972025-01-11T06:05:25.711ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
982025-01-11T06:05:25.711ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
992025-01-11T06:05:25.711ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1002025-01-11T06:05:25.711ZINFOcrucible: All required reconciliation work is completed session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1012025-01-11T06:05:25.711ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1022025-01-11T06:05:25.711ZINFOcrucible: a639371d-0191-4dc0-883e-cbac9e49f2f2 is now active with session: 3cee1d12-f70c-4666-81eb-a8898bec9056 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1032025-01-11T06:05:25.711ZINFOcrucible: Set Active after no reconciliation session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1042025-01-11T06:05:25.711ZINFOcrucible: The guest has finished waiting for activation with:1
105 Activate successful
106 Wait for a query_work_queue command to finish before sending IO
107 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1021 Acked Read 256 Done Done Done false
117 1022 Acked Read 256 Done Done Done false
118 1023 Acked Read 256 Done Done Done false
119 1024 Acked Read 256 Done Done Done false
120 1025 Acked Read 256 Done Done Done false
121 1026 Acked Read 256 Done Done Done false
122 1027 Acked Read 256 Done Done Done false
123 1028 Acked Read 256 Done Done Done false
124 1029 Acked Read 256 Done Done Done false
125 1030 Acked Read 256 Done Done Done false
126 1031 Acked Read 256 Done Done Done false
127 1032 Acked Read 256 Done Done Done false
128 1033 Acked Read 256 Done Done Done false
129 1034 Acked Read 256 Done Done Done false
130 1035 Acked Read 256 Done Done Done false
131 1036 Acked Read 256 Done Done Done false
132 1037 Acked Read 256 Done Done Done false
133 1038 Acked Read 256 Done Done Done false
134 1039 Acked Read 256 Done Done Done false
135 1040 Acked Read 136 Done Done Done false
136 STATES DS:0 DS:1 DS:2 TOTAL
137 Sent 0 0 0 0
138 Done 20 20 20 60
139 Skipped 0 0 0 0
140 Error 0 0 0 0
141 Last Flush: 1020 1020 1020
142 Downstairs last five completed: 1020 1019 1018 1017 1016
143 Upstairs last five completed: 1040 1039 1038 1037 1036
144 CLIENT: Up:0 ds:20 act:3
145 ----------------------------------------------------------------
146 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
147 Downstairs last five completed: 1041 1040 1039 1038 1037
148 Upstairs last five completed: 1041 1040 1039 1038 1037
149 CLIENT: Up:0 ds:0 act:3
150 CLIENT: All crucible jobs finished, exiting program
1512025-01-11T06:05:30.308ZWARNcrucible: Guest handle has been dropped
1522025-01-11T06:05:30.309ZWARNcrucible: client_stop_rx closed unexpectedly: RecvError(()) = io task client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1532025-01-11T06:05:30.309ZWARNcrucible: client task is sending Done(QueueClosed) = io task client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1542025-01-11T06:05:30.309ZWARNcrucible: client task could not reply to main task; shutting down? = io task client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
1552025-01-11T06:05:30.309ZINFOcrucible: client task is exiting = io task client = 0 session_id = 3cee1d12-f70c-4666-81eb-a8898bec9056
156 Replay loop starts now January 11, 2025 at 06:05:30 AM UTC
157 Jan 11 06:05:30.380 INFO accepted connection, remote_addr: 127.0.0.1:58694, local_addr: 127.0.0.1:9998
158 Jan 11 06:05:30.381 INFO request completed, latency_us: 147, response_code: 200, uri: /regioninfo, method: GET, req_id: 5ff96843-f486-4e2b-8004-28bc5501aa33, remote_addr: 127.0.0.1:58694, local_addr: 127.0.0.1:9998
1592025-01-11T06:05:30.381ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
160 Jan 11 06:05:30.382 INFO request completed, latency_us: 102, response_code: 200, uri: /regioncount, method: GET, req_id: bf015d3f-7b9d-43b3-9a1c-65af1b9becdb, remote_addr: 127.0.0.1:58694, local_addr: 127.0.0.1:9998
1612025-01-11T06:05:30.382ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
162 Jan 11 06:05:30.382 INFO request completed, latency_us: 112, response_code: 200, uri: /port/cid/0, method: GET, req_id: 79662e47-d622-47de-be8c-bc91effba876, remote_addr: 127.0.0.1:58694, local_addr: 127.0.0.1:9998
163 Jan 11 06:05:30.383 INFO request completed, latency_us: 91, response_code: 200, uri: /port/cid/1, method: GET, req_id: 4e4a9278-c39f-4ce2-9d04-22065bbc0ea7, remote_addr: 127.0.0.1:58694, local_addr: 127.0.0.1:9998
164 Jan 11 06:05:30.383 INFO request completed, latency_us: 92, response_code: 200, uri: /port/cid/2, method: GET, req_id: 38b955c8-4238-42ec-a908-2156d6e975e8, remote_addr: 127.0.0.1:58694, local_addr: 127.0.0.1:9998
1652025-01-11T06:05:30.384ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1662025-01-11T06:05:30.384ZINFOcrucible: Upstairs starts
1672025-01-11T06:05:30.384ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "afbd0ec341edbe8e076286ae0030d2abadb28437", git_commit_timestamp: "2025-01-11T05:49:56.000000000Z", git_branch: "renovate/opentelemetry-0.x", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1682025-01-11T06:05:30.384ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1692025-01-11T06:05:30.384ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1702025-01-11T06:05:30.384ZINFOcrucible: Crucible b37855bc-4bce-4c84-a2d3-ed2cb2fc6655 has session id: e3b437a4-33d5-43e8-8b38-4c1c18ad0389 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1712025-01-11T06:05:30.384ZINFOcrucible: Upstairs opts: Upstairs UUID: b37855bc-4bce-4c84-a2d3-ed2cb2fc6655, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1722025-01-11T06:05:30.384ZINFOcrucible: Crucible stats registered with UUID: b37855bc-4bce-4c84-a2d3-ed2cb2fc6655 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1732025-01-11T06:05:30.419ZINFOcrucible: Downstairs in volume = 3 task = crutest
1742025-01-11T06:05:30.419ZINFOcrucible: The guest has requested activation with gen:2
1752025-01-11T06:05:30.419ZINFOcrucible: b37855bc-4bce-4c84-a2d3-ed2cb2fc6655 active request set session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1762025-01-11T06:05:30.419ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1772025-01-11T06:05:30.419ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1782025-01-11T06:05:30.419ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1792025-01-11T06:05:30.419ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1802025-01-11T06:05:30.419ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1812025-01-11T06:05:30.419ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1822025-01-11T06:05:30.419ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1832025-01-11T06:05:30.419ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1842025-01-11T06:05:30.419ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1852025-01-11T06:05:30.419ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59781) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1862025-01-11T06:05:30.420ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63914) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1872025-01-11T06:05:30.420ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59292) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1882025-01-11T06:05:30.420ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1892025-01-11T06:05:30.420ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1902025-01-11T06:05:30.420ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1912025-01-11T06:05:30.420ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1922025-01-11T06:05:30.420ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1932025-01-11T06:05:30.420ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1942025-01-11T06:05:30.421ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1952025-01-11T06:05:30.421ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1962025-01-11T06:05:30.421ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1972025-01-11T06:05:30.422ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1982025-01-11T06:05:30.422ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1992025-01-11T06:05:30.422ZINFOcrucible: Waiting for more clients to be ready session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2002025-01-11T06:05:30.422ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2012025-01-11T06:05:30.422ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2022025-01-11T06:05:30.422ZINFOcrucible: Waiting for more clients to be ready session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2032025-01-11T06:05:30.422ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2042025-01-11T06:05:30.422ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2052025-01-11T06:05:30.422ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2062025-01-11T06:05:30.422ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2072025-01-11T06:05:30.422ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2082025-01-11T06:05:30.422ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2092025-01-11T06:05:30.422ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2102025-01-11T06:05:30.422ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2112025-01-11T06:05:30.422ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2122025-01-11T06:05:30.422ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2132025-01-11T06:05:30.422ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2142025-01-11T06:05:30.422ZINFOcrucible: Max found gen is 2 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2152025-01-11T06:05:30.422ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2162025-01-11T06:05:30.422ZINFOcrucible: Next flush: 2 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2172025-01-11T06:05:30.422ZINFOcrucible: All extents match = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
218 {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30Activate successful
219 ,Wait for a query_work_queue command to finish before sending IO
220 "time":"2025-01-11T06:05:30.422415267Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
2212025-01-11T06:05:30.422ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2222025-01-11T06:05:30.422ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2232025-01-11T06:05:30.422ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2242025-01-11T06:05:30.422ZINFOcrucible: All required reconciliation work is completed session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2252025-01-11T06:05:30.422ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2262025-01-11T06:05:30.422ZINFOcrucible: b37855bc-4bce-4c84-a2d3-ed2cb2fc6655 is now active with session: e3b437a4-33d5-43e8-8b38-4c1c18ad0389 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2272025-01-11T06:05:30.422ZINFOcrucible: Set Active after no reconciliation session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2282025-01-11T06:05:30.422ZINFOcrucible: The guest has finished waiting for activation with:2
229 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
230 Loading write count information from file "/tmp/test_replay_verify.log"
231 Jan 11 06:05:30.456 INFO accepted connection, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
232 Main task has work to do, go find it
233 got dsc Stop(1)
234 Jan 11 06:05:30.457stop 1
235 INFO request completed, latency_us: 174, response_code: 204, uri: /stop/cid/1, method: POST, req_id: cac001db-7fc4-4e94-9fb5-f8a6a47ddef8, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
236 [8820] Got stop action so:false kr:false
2372025-01-11T06:05:30.457ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2382025-01-11T06:05:30.457ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2392025-01-11T06:05:30.457ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2402025-01-11T06:05:30.457ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2412025-01-11T06:05:30.457ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2422025-01-11T06:05:30.457ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
243 Jan 11 06:05:30.458 INFO request completed, latency_us: 310, response_code: 200, uri: /state/cid/1, method: GET, req_id: bf7d1317-d185-4ee8-aa9d-90db5f58ba9d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
244 [8820] Exited with: ExitStatus(unix_wait_status(9))
245 [8820][1] reports Exit
246 Jan 11 06:05:34.460 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/1, method: GET, req_id: fd3a3bac-e887-45d5-9f52-ae0d93b4c479, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
247 001/300 Write block 323 len 40960 data: 2 2 2 2 2 2 2 2 2 2
248 002/300 Write block 859 len 28672 data: 2 2 2 2 2 2 2
249 003/300 Write block 947 len 40960 data: 2 2 2 2 2 2 2 2 2 2
250 004/300 Write block 4243 len 8192 data: 2 2
251 005/300 Write block 3446 len 28672 data: 2 2 2 2 2 2 2
252 006/300 Write block 336 len 20480 data: 2 2 2 2 2
253 007/300 Read block 1478 len 32768
254 008/300 Write block 2727 len 40960 data: 2 2 2 2 2 2 2 2 2 2
255 009/300 Read block 3081 len 12288
256 010/300 Read block 1110 len 36864
257 011/300 Read block 1950 len 28672
258 012/300 Flush
259 013/300 Write block 2796 len 36864 data: 2 2 2 2 2 2 2 2 2
260 014/300 Read block 1644 len 24576
261 015/300 Write block 4764 len 4096 data: 2
262 016/300 Read block 466 len 12288
263 017/300 Write block 4138 len 28672 data: 2 2 2 2 2 2 2
264 018/300 Write block 3211 len 8192 data: 2 2
265 019/300 Read block 985 len 12288
266 020/300 Flush
267 021/300 Read block 4172 len 8192
268 022/300 Read block 2010 len 40960
269 023/300 Read block 2075 len 4096
270 024/300 Write block 141 len 32768 data: 2 2 2 2 2 2 2 2
271 025/300 Read block 2718 len 28672
272 026/300 Write block 1690 len 12288 data: 2 2 2
273 027/300 Read block 3677 len 36864
274 028/300 Write block 4295 len 32768 data: 2 2 2 2 2 2 2 2
275 029/300 Write block 2 len 4096 data: 2
276 030/300 Write block 1323 len 8192 data: 2 2
277 031/300 Read block 3032 len 16384
278 032/300 Flush
279 033/300 Read block 2171 len 24576
280 034/300 Read block 4059 len 32768
281 035/300 Flush
282 036/300 Flush
283 037/300 Write block 549 len 8192 data: 2 2
284 038/300 Write block 3732 len 24576 data: 2 2 2 2 2 2
285 039/300 Read block 4501 len 4096
286 040/300 Read block 46 len 32768
287 041/300 Write block 3776 len 28672 data: 2 2 2 2 2 2 2
288 042/300 Write block 4242 len 24576 data: 2 3 3 2 2 2
289 043/300 Write block 1235 len 32768 data: 2 2 2 2 2 2 2 2
290 044/300 Read block 1389 len 8192
291 045/300 Flush
292 046/300 Write block 4861 len 4096 data: 2
293 047/300 Write block 2231 len 12288 data: 2 2 2
294 048/300 Write block 4455 len 16384 data: 2 2 2 2
295 049/300 Read block 316 len 16384
296 050/300 Write block 2381 len 12288 data: 2 2 2
297 051/300 Read block 2748 len 12288
298 052/300 Write block 4050 len 12288 data: 2 2 2
299 053/300 Read block 3031 len 4096
300 054/300 Read block 1652 len 36864
301 055/300 Write block 1993 len 20480 data: 2 2 2 2 2
302 056/300 Flush
303 057/300 Write block 2949 len 28672 data: 2 2 2 2 2 2 2
304 058/300 Read block 2230 len 20480
305 059/300 Read block 258 len 28672
306 060/300 Read block 1101 len 16384
307 061/300 Read block 864 len 28672
308 062/300 Write block 1715 len 16384 data: 2 2 2 2
309 063/300 Read block 936 len 20480
310 064/300 Write block 3596 len 32768 data: 2 2 2 2 2 2 2 2
311 065/300 Write block 4773 len 36864 data: 2 2 2 2 2 2 2 2 2
312 066/300 Write block 4403 len 16384 data: 2 2 2 2
313 067/300 Write block 2097 len 28672 data: 2 2 2 2 2 2 2
314 068/300 Write block 4483 len 28672 data: 2 2 2 2 2 2 2
315 069/300 Write block 911 len 40960 data: 2 2 2 2 2 2 2 2 2 2
316 070/300 Read block 3226 len 4096
317 071/300 Read block 70 len 28672
318 072/300 Write block 1742 len 20480 data: 2 2 2 2 2
319 073/300 Flush
320 074/300 Write block 785 len 36864 data: 2 2 2 2 2 2 2 2 2
321 075/300 Write block 2847 len 28672 data: 2 2 2 2 2 2 2
322 076/300 Read block 267 len 32768
323 077/300 Write block 4003 len 8192 data: 2 2
324 078/300 Read block 1245 len 40960
325 079/300 Read block 4101 len 36864
326 080/300 Read block 4978 len 8192
327 081/300 Read block 3618 len 36864
328 082/300 Write block 483 len 40960 data: 2 2 2 2 2 2 2 2 2 2
329 083/300 Flush
330 084/300 Flush
331 085/300 Read block 4572 len 36864
332 086/300 Write block 4503 len 8192 data: 2 2
333 087/300 Flush
334 088/300 Write block 4874 len 32768 data: 2 2 2 2 2 2 2 2
335 089/300 Read block 4140 len 32768
336 090/300 Write block 3505 len 28672 data: 2 2 2 2 2 2 2
337 091/300 Write block 3672 len 20480 data: 2 2 2 2 2
338 092/300 Write block 3870 len 32768 data: 2 2 2 2 2 2 2 2
339 093/300 Write block 3727 len 12288 data: 2 2 2
340 094/300 Write block 4695 len 20480 data: 2 2 2 2 2
341 095/300 Read block 738 len 12288
342 096/300 Read block 201 len 20480
343 097/300 Read block 2228 len 40960
344 098/300 Write block 3098 len 24576 data: 2 2 2 2 2 2
345 099/300 Read block 940 len 40960
346 100/300 Read block 3340 len 16384
347 101/300 Write block 4253 len 32768 data: 2 2 2 2 2 2 2 2
348 102/300 Read block 1482 len 16384
349 103/300 Write block 3957 len 12288 data: 2 2 2
350 104/300 Write block 3591 len 4096 data: 2
351 105/300 Write block 1129 len 8192 data: 2 2
352 106/300 Flush
353 107/300 Write block 2346 len 32768 data: 2 2 2 2 2 2 2 2
354 108/300 Write block 3994 len 20480 data: 2 2 2 2 2
355 109/300 Read block 3582 len 4096
356 110/300 Read block 2459 len 36864
357 111/300 Write block 4928 len 32768 data: 2 2 2 2 2 2 2 2
358 112/300 Read block 2644 len 32768
359 113/300 Read block 3654 len 40960
360 114/300 Write block 537 len 12288 data: 2 2 2
361 115/300 Read block 686 len 28672
362 116/300 Write block 3475 len 32768 data: 2 2 2 2 2 2 2 2
363 117/300 Write block 2020 len 4096 data: 2
364 118/300 Write block 1720 len 28672 data: 2 2 2 2 2 2 2
365 119/300 Write block 4506 len 40960 data: 2 2 2 2 2 2 2 2 2 2
366 120/300 Flush
367 121/300 Read block 4803 len 28672
368 122/300 Write block 2279 len 40960 data: 2 2 2 2 2 2 2 2 2 2
369 123/300 Flush
370 124/300 Write block 1058 len 24576 data: 2 2 2 2 2 2
371 125/300 Write block 1724 len 28672 data: 3 3 3 2 2 2 2
372 126/300 Read block 2908 len 8192
373 127/300 Write block 1640 len 8192 data: 2 2
374 128/300 Read block 1207 len 12288
375 129/300 Read block 2463 len 20480
376 130/300 Flush
377 131/300 Read block 1552 len 16384
378 132/300 Flush
379 133/300 Write block 2218 len 40960 data: 2 2 2 2 2 2 2 2 2 2
380 134/300 Write block 708 len 32768 data: 2 2 2 2 2 2 2 2
381 135/300 Write block 2290 len 24576 data: 2 2 2 2 2 2
382 136/300 Read block 1638 len 40960
383 137/300 Write block 2392 len 16384 data: 2 2 2 2
384 138/300 Write block 4122 len 20480 data: 2 2 2 2 2
385 139/300 Read block 228 len 12288
386 140/300 Write block 4922 len 12288 data: 2 2 2
387 141/300 Write block 1190 len 16384 data: 2 2 2 2
388 142/300 Write block 1240 len 8192 data: 3 3
389 143/300 Read block 90 len 8192
390 144/300 Flush
391 145/300 Flush
392 146/300 Read block 810 len 36864
393 147/300 Read block 4732 len 40960
394 148/300 Write block 1722 len 16384 data: 3 3 4 4
395 149/300 Flush
396 150/300 Read block 3610 len 12288
397 151/300 Write block 1497 len 28672 data: 2 2 2 2 2 2 2
398 152/300 Read block 877 len 20480
399 153/300 Read block 1314 len 12288
400 154/300 Read block 3274 len 16384
401 155/300 Write block 1015 len 4096 data: 2
402 156/300 Read block 858 len 28672
403 157/300 Read block 3220 len 16384
404 158/300 Flush
405 159/300 Write block 1118 len 20480 data: 2 2 2 2 2
406 160/300 Write block 2623 len 24576 data: 2 2 2 2 2 2
407 161/300 Read block 4411 len 28672
408 162/300 Read block 4163 len 40960
409 163/300 Read block 2955 len 40960
410 164/300 Read block 4325 len 36864
411 165/300 Write block 402 len 20480 data: 2 2 2 2 2
412 166/300 Write block 878 len 20480 data: 2 2 2 2 2
413 167/300 Write block 2175 len 40960 data: 2 2 2 2 2 2 2 2 2 2
414 168/300 Write block 4463 len 36864 data: 2 2 2 2 2 2 2 2 2
415 169/300 Read block 4632 len 28672
416 170/300 Write block 822 len 24576 data: 2 2 2 2 2 2
417 171/300 Read block 2196 len 36864
418 172/300 Read block 1433 len 12288
419 173/300 Write block 800 len 28672 data: 2 2 2 2 2 2 2
420 174/300 Flush
421 175/300 Write block 4671 len 36864 data: 2 2 2 2 2 2 2 2 2
422 176/300 Write block 1635 len 16384 data: 2 2 2 2
423 177/300 Write block 4343 len 16384 data: 2 2 2 2
424 178/300 Write block 1945 len 24576 data: 2 2 2 2 2 2
425 179/300 Read block 3804 len 12288
426 180/300 Read block 61 len 20480
427 181/300 Write block 4490 len 12288 data: 2 2 2
428 182/300 Read block 2830 len 40960
429 183/300 Read block 4759 len 16384
430 184/300 Read block 2614 len 40960
431 185/300 Read block 550 len 36864
432 186/300 Flush
433 187/300 Read block 4083 len 32768
434 188/300 Write block 4686 len 36864 data: 2 2 2 2 2 2 2 2 2
435 189/300 Flush
436 190/300 Read block 1492 len 16384
437 191/300 Read block 2409 len 28672
438 192/300 Read block 221 len 8192
439 193/300 Read block 4009 len 4096
440 194/300 Read block 2503 len 28672
441 195/300 Read block 52 len 32768
442 196/300 Read block 1383 len 32768
443 197/300 Read block 338 len 36864
444 198/300 Read block 1873 len 40960
445 199/300 Read block 3772 len 16384
446 200/300 Write block 2351 len 32768 data: 3 3 3 2 2 2 2 2
447 201/300 Write block 4453 len 12288 data: 2 2 3
448 202/300 Read block 1902 len 4096
449 203/300 Read block 2402 len 36864
450 204/300 Write block 2333 len 16384 data: 2 2 2 2
451 205/300 Write block 1857 len 40960 data: 2 2 2 2 2 2 2 2 2 2
452 206/300 Write block 2907 len 12288 data: 2 2 2
453 207/300 Write block 4585 len 12288 data: 2 2 2
454 208/300 Read block 2155 len 20480
455 209/300 Write block 2535 len 8192 data: 2 2
456 210/300 Read block 4517 len 32768
457 211/300 Read block 471 len 16384
458 212/300 Flush
459 213/300 Read block 1017 len 20480
460 214/300 Flush
461 215/300 Write block 2596 len 24576 data: 2 2 2 2 2 2
462 216/300 Read block 3288 len 8192
463 217/300 Write block 1754 len 28672 data: 2 2 2 2 2 2 2
464 218/300 Write block 1326 len 36864 data: 2 2 2 2 2 2 2 2 2
465 219/300 Write block 4678 len 32768 data: 3 3 2 2 2 2 2 2
466 220/300 Write block 4311 len 8192 data: 2 2
467 221/300 Read block 1989 len 36864
468 222/300 Flush
469 223/300 Read block 2802 len 8192
470 224/300 Write block 1510 len 20480 data: 2 2 2 2 2
471 225/300 Flush
472 226/300 Flush
473 227/300 Write block 375 len 32768 data: 2 2 2 2 2 2 2 2
474 228/300 Write block 530 len 16384 data: 2 2 2 2
475 229/300 Read block 593 len 36864
476 230/300 Write block 37 len 32768 data: 2 2 2 2 2 2 2 2
477 231/300 Read block 4665 len 12288
478 232/300 Read block 2206 len 40960
479 233/300 Read block 1406 len 24576
480 234/300 Write block 1395 len 4096 data: 2
481 235/300 Read block 4276 len 32768
482 236/300 Read block 4148 len 36864
483 237/300 Write block 3475 len 24576 data: 3 3 3 3 3 3
484 238/300 Read block 4446 len 32768
485 239/300 Read block 4838 len 8192
486 240/300 Read block 345 len 4096
487 241/300 Read block 4787 len 20480
488 242/300 Read block 1664 len 16384
489 243/300 Write block 4219 len 20480 data: 2 2 2 2 2
490 244/300 Write block 1952 len 32768 data: 2 2 2 2 2 2 2 2
491 245/300 Read block 1518 len 36864
492 246/300 Write block 290 len 32768 data: 2 2 2 2 2 2 2 2
493 247/300 Read block 752 len 12288
494 248/300 Read block 2196 len 40960
495 249/300 Flush
496 250/300 Flush
497 251/300 Read block 3414 len 12288
498 252/300 Write block 4126 len 20480 data: 3 2 2 2 2
499 253/300 Read block 4870 len 32768
500 254/300 Write block 2420 len 20480 data: 2 2 2 2 2
501 255/300 Write block 2942 len 36864 data: 2 2 2 2 2 2 2 3 3
502 256/300 Read block 1217 len 20480
503 257/300 Write block 4083 len 28672 data: 2 2 2 2 2 2 2
504 258/300 Read block 2741 len 24576
505 259/300 Read block 4593 len 8192
506 260/300 Read block 3689 len 40960
507 261/300 Read block 1817 len 24576
508 262/300 Write block 773 len 32768 data: 2 2 2 2 2 2 2 2
509 263/300 Write block 3533 len 8192 data: 2 2
510 264/300 Write block 3594 len 20480 data: 2 2 3 3 3
511 265/300 Write block 466 len 36864 data: 2 2 2 2 2 2 2 2 2
512 266/300 Write block 4463 len 32768 data: 3 3 3 3 3 3 3 3
513 267/300 Read block 2654 len 24576
514 268/300 Write block 3939 len 24576 data: 2 2 2 2 2 2
515 269/300 Read block 3425 len 40960
516 270/300 Read block 935 len 32768
517 271/300 Read block 2637 len 40960
518 272/300 Read block 4358 len 28672
519 273/300 Read block 852 len 16384
520 274/300 Read block 1928 len 40960
521 275/300 Write block 3505 len 16384 data: 3 3 3 3
522 276/300 Read block 1094 len 24576
523 277/300 Read block 675 len 12288
524 278/300 Flush
525 279/300 Read block 627 len 20480
526 280/300 Read block 4597 len 40960
527 281/300 Read block 1404 len 16384
528 282/300 Read block 2614 len 24576
529 283/300 Write block 708 len 36864 data: 3 3 3 3 3 3 3 3 2
530 284/300 Read block 4355 len 4096
531 285/300 Read block 3744 len 16384
532 286/300 Read block 1279 len 36864
533 287/300 Flush
534 288/300 Write block 2073 len 28672 data: 2 2 2 2 2 2 2
535 289/300 Read block 3578 len 36864
536 290/300 Read block 4961 len 4096
537 291/300 Read block 4225 len 40960
538 292/300 Flush
539 293/300 Read block 3659 len 12288
540 294/300 Write block 800 len 4096 data: 3
541 295/300 Write block 3331 len 12288 data: 2 2 2
542 296/300 Read block 1281 len 36864
543 297/300 Write block 877 len 32768 data: 2 3 3 3 3 3 2 2
544 298/300 Write block 4607 len 16384 data: 2 2 2 2
545 299/300 Write block 1621 len 32768 data: 2 2 2 2 2 2 2 2
546 300/300 Write block 4169 len 28672 data: 2 2 2 2 2 2 2
547 301/300 Read block 569 len 8192
548 Main task has work to do, go find it
549 got dsc Start(1)
550 start 1
551 Jan 11 06:05:34.704 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/1, method: POST, req_id: c6868c49-4259-4052-9e3d-cf13a0d15d82, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
552 Replay: started 1, returned:Ok(())
553 ----------------------------------------------------------------
554 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
555 States: Active Offline Active
556 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
557 1000 Acked Write 10 Done Sent Done false
558 1001 Acked Write 7 Done Sent Done false
559 1002 Acked Write 10 Done Sent Done false
560 1003 Acked Write 2 Done Sent Done false
561 1004 Acked Write 7 Done Sent Done false
562 1005 Acked Write 5 Done Sent Done false
563 1006 Acked Read 8 Done Sent Done false
564 1007 Acked Write 10 Done Sent Done false
565 1008 Acked Read 3 Done Sent Done false
566 1009 Acked Read 9 Done Sent Done false
567 1010 Acked Read 7 Done Sent Done false
568 1011 Acked Flush 0 Done Sent Done false
569 1012 Acked Write 9 Done Sent Done false
570 1013 Acked Read 6 Done Sent Done false
571 1014 Acked Write 1 Done Sent Done false
572 1015 Acked Read 3 Done Sent Done false
573 1016 Acked Write 7 Done Sent Done false
574 1017 Acked Write 2 Done Sent Done false
575 1018 Acked Read 3 Done Sent Done false
576 1019 Acked Flush 0 Done Sent Done false
577 1020 Acked Read 2 Done Sent Done false
578 1021 Acked Read 10 Done Sent Done false
579 1022 Acked Read 1 Done Sent Done false
580 1023 Acked Write 8 Done Sent Done false
581 1024 Acked Read 7 Done Sent Done false
582 1025 Acked Write 3 Done Sent Done false
583 1026 Acked Read 9 Done Sent Done false
584 1027 Acked Write 8 Done Sent Done false
585 1028 Acked Write 1 Done Sent Done false
586 1029 Acked Write 2 Done Sent Done false
587 1030 Acked Read 4 Done Sent Done false
588 1031 Acked Flush 0 Done Sent Done false
589 1032 Acked Read 6 Done Sent Done false
590 1033 Acked Read 8 Done Sent Done false
591 1034 Acked Flush 0 Done Sent Done false
592 1035 Acked Flush 0 Done Sent Done false
593 1036 Acked Write 2 Done Sent Done false
594 1037 Acked Write 6 Done Sent Done false
595 1038 Acked Read 1 Done Sent Done false
596 1039 Acked Read 8 Done Sent Done false
597 1040 Acked Write 7 Done Sent Done false
598 1041 Acked Write 6 Done Sent Done false
599 1042 Acked Write 8 Done Sent Done false
600 1043 Acked Read 2 Done Sent Done false
601 1044 Acked Flush 0 Done Sent Done false
602 1045 Acked Write 1 Done Sent Done false
603 1046 Acked Write 3 Done Sent Done false
604 1047 Acked Write 4 Done Sent Done false
605 1048 Acked Read 4 Done Sent Done false
606 1049 Acked Write 3 Done Sent Done false
607 1050 Acked Read 3 Done Sent Done false
608 1051 Acked Write 3 Done Sent Done false
609 1052 Acked Read 1 Done Sent Done false
610 1053 Acked Read 9 Done Sent Done false
611 1054 Acked Write 5 Done Sent Done false
612 1055 Acked Flush 0 Done Sent Done false
613 1056 Acked Write 7 Done Sent Done false
614 1057 Acked Read 5 Done Sent Done false
615 1058 Acked Read 7 Done Sent Done false
616 1059 Acked Read 4 Done Sent Done false
617 1060 Acked Read 7 Done Sent Done false
618 1061 Acked Write 4 Done Sent Done false
619 1062 Acked Read 5 Done Sent Done false
620 1063 Acked Write 8 Done Sent Done false
621 1064 Acked Write 9 Done Sent Done false
622 1065 Acked Write 4 Done Sent Done false
623 1066 Acked Write 7 Done Sent Done false
624 1067 Acked Write 7 Done Sent Done false
625 1068 Acked Write 10 Done Sent Done false
626 1069 Acked Read 1 Done Sent Done false
627 1070 Acked Read 7 Done Sent Done false
628 1071 Acked Write 5 Done Sent Done false
629 1072 Acked Flush 0 Done Sent Done false
630 1073 Acked Write 9 Done Sent Done false
631 1074 Acked Write 7 Done Sent Done false
632 1075 Acked Read 8 Done Sent Done false
633 1076 Acked Write 2 Done Sent Done false
634 1077 Acked Read 10 Done Sent Done false
635 1078 Acked Read 9 Done Sent Done false
636 1079 Acked Read 2 Done Sent Done false
637 1080 Acked Read 9 Done Sent Done false
638 1081 Acked Write 10 Done Sent Done false
639 1082 Acked Flush 0 Done Sent Done false
640 1083 Acked Flush 0 Done Sent Done false
641 1084 Acked Read 9 Done Sent Done false
642 1085 Acked Write 2 Done Sent Done false
643 1086 Acked Flush 0 Done Sent Done false
644 1087 Acked Write 8 Done Sent Done false
645 1088 Acked Read 8 Done Sent Done false
646 1089 Acked Write 7 Done Sent Done false
647 1090 Acked Write 5 Done Sent Done false
648 1091 Acked Write 8 Done Sent Done false
649 1092 Acked Write 3 Done Sent Done false
650 1093 Acked Write 5 Done Sent Done false
651 1094 Acked Read 3 Done Sent Done false
652 1095 Acked Read 5 Done Sent Done false
653 1096 Acked Read 10 Done Sent Done false
654 1097 Acked Write 6 Done Sent Done false
655 1098 Acked Read 10 Done Sent Done false
656 1099 Acked Read 4 Done Sent Done false
657 1100 Acked Write 8 Done Sent Done false
658 1101 Acked Read 4 Done Sent Done false
659 1102 Acked Write 3 Done Sent Done false
660 1103 Acked Write 1 Done Sent Done false
661 1104 Acked Write 2 Done Sent Done false
662 1105 Acked Flush 0 Done Sent Done false
663 1106 Acked Write 8 Done Sent Done false
664 1107 Acked Write 5 Done Sent Done false
665 1108 Acked Read 1 Done Sent Done false
666 1109 Acked Read 9 Done Sent Done false
667 1110 Acked Write 8 Done Sent Done false
668 1111 Acked Read 8 Done Sent Done false
669 1112 Acked Read 10 Done Sent Done false
670 1113 Acked Write 3 Done Sent Done false
671 1114 Acked Read 7 Done Sent Done false
672 1115 Acked Write 8 Done Sent Done false
673 1116 Acked Write 1 Done Sent Done false
674 1117 Acked Write 7 Done Sent Done false
675 1118 Acked Write 10 Done Sent Done false
676 1119 Acked Flush 0 Done Sent Done false
677 1120 Acked Read 7 Done Sent Done false
678 1121 Acked Write 10 Done Sent Done false
679 1122 Acked Flush 0 Done Sent Done false
680 1123 Acked Write 6 Done Sent Done false
681 1124 Acked Write 7 Done Sent Done false
682 1125 Acked Read 2 Done Sent Done false
683 1126 Acked Write 2 Done Sent Done false
684 1127 Acked Read 3 Done Sent Done false
685 1128 Acked Read 5 Done Sent Done false
686 1129 Acked Flush 0 Done Sent Done false
687 1130 Acked Read 4 Done Sent Done false
688 1131 Acked Flush 0 Done Sent Done false
689 1132 Acked Write 10 Done Sent Done false
690 1133 Acked Write 8 Done Sent Done false
691 1134 Acked Write 6 Done Sent Done false
692 1135 Acked Read 10 Done Sent Done false
693 1136 Acked Write 4 Done Sent Done false
694 1137 Acked Write 5 Done Sent Done false
695 1138 Acked Read 3 Done Sent Done false
696 1139 Acked Write 3 Done Sent Done false
697 1140 Acked Write 4 Done Sent Done false
698 1141 Acked Write 2 Done Sent Done false
699 1142 Acked Read 2 Done Sent Done false
700 1143 Acked Flush 0 Done Sent Done false
701 1144 Acked Flush 0 Done Sent Done false
702 1145 Acked Read 9 Done Sent Done false
703 1146 Acked Read 10 Done Sent Done false
704 1147 Acked Write 4 Done Sent Done false
705 1148 Acked Flush 0 Done Sent Done false
706 1149 Acked Read 3 Done Sent Done false
707 1150 Acked Write 7 Done Sent Done false
708 1151 Acked Read 5 Done Sent Done false
709 1152 Acked Read 3 Done Sent Done false
710 1153 Acked Read 4 Done Sent Done false
711 1154 Acked Write 1 Done Sent Done false
712 1155 Acked Read 7 Done Sent Done false
713 1156 Acked Read 4 Done Sent Done false
714 1157 Acked Flush 0 Done Sent Done false
715 1158 Acked Write 5 Done Sent Done false
716 1159 Acked Write 6 Done Sent Done false
717 1160 Acked Read 7 Done Sent Done false
718 1161 Acked Read 10 Done Sent Done false
719 1162 Acked Read 10 Done Sent Done false
720 1163 Acked Read 9 Done Sent Done false
721 1164 Acked Write 5 Done Sent Done false
722 1165 Acked Write 5 Done Sent Done false
723 1166 Acked Write 10 Done Sent Done false
724 1167 Acked Write 9 Done Sent Done false
725 1168 Acked Read 7 Done Sent Done false
726 1169 Acked Write 6 Done Sent Done false
727 1170 Acked Read 9 Done Sent Done false
728 1171 Acked Read 3 Done Sent Done false
729 1172 Acked Write 7 Done Sent Done false
730 1173 Acked Flush 0 Done Sent Done false
731 1174 Acked Write 9 Done Sent Done false
732 1175 Acked Write 4 Done Sent Done false
733 1176 Acked Write 4 Done Sent Done false
734 1177 Acked Write 6 Done Sent Done false
735 1178 Acked Read 3 Done Sent Done false
736 1179 Acked Read 5 Done Sent Done false
737 1180 Acked Write 3 Done Sent Done false
738 1181 Acked Read 10 Done Sent Done false
739 1182 Acked Read 4 Done Sent Done false
740 1183 Acked Read 10 Done Sent Done false
741 1184 Acked Read 9 Done Sent Done false
742 1185 Acked Flush 0 Done Sent Done false
743 1186 Acked Read 8 Done Sent Done false
744 1187 Acked Write 9 Done Sent Done false
745 1188 Acked Flush 0 Done Sent Done false
746 1189 Acked Read 4 Done Sent Done false
747 1190 Acked Read 7 Done Sent Done false
748 1191 Acked Read 2 Done Sent Done false
749 1192 Acked Read 1 Done Sent Done false
750 1193 Acked Read 7 Done Sent Done false
751 1194 Acked Read 8 Done Sent Done false
752 1195 Acked Read 8 Done Sent Done false
753 1196 Acked Read 9 Done Sent Done false
754 1197 Acked Read 10 Done Sent Done false
755 1198 Acked Read 4 Done Sent Done false
756 1199 Acked Write 8 Done Sent Done false
757 1200 Acked Write 3 Done Sent Done false
758 1201 Acked Read 1 Done Sent Done false
759 1202 Acked Read 9 Done Sent Done false
760 1203 Acked Write 4 Done Sent Done false
761 1204 Acked Write 10 Done Sent Done false
762 1205 Acked Write 3 Done Sent Done false
763 1206 Acked Write 3 Done Sent Done false
764 1207 Acked Read 5 Done Sent Done false
765 1208 Acked Write 2 Done Sent Done false
766 1209 Acked Read 8 Done Sent Done false
767 1210 Acked Read 4 Done Sent Done false
768 1211 Acked Flush 0 Done Sent Done false
769 1212 Acked Read 5 Done Sent Done false
770 1213 Acked Flush 0 Done Sent Done false
771 1214 Acked Write 6 Done Sent Done false
772 1215 Acked Read 2 Done Sent Done false
773 1216 Acked Write 7 Done Sent Done false
774 1217 Acked Write 9 Done Sent Done false
775 1218 Acked Write 8 Done Sent Done false
776 1219 Acked Write 2 Done Sent Done false
777 1220 Acked Read 9 Done Sent Done false
778 1221 Acked Flush 0 Done Sent Done false
779 1222 Acked Read 2 Done Sent Done false
780 1223 Acked Write 5 Done Sent Done false
781 1224 Acked Flush 0 Done Sent Done false
782 1225 Acked Flush 0 Done Sent Done false
783 1226 Acked Write 8 Done Sent Done false
784 1227 Acked Write 4 Done Sent Done false
785 1228 Acked Read 9 Done Sent Done false
786 1229 Acked Write 8 Done Sent Done false
787 1230 Acked Read 3 Done Sent Done false
788 1231 Acked Read 10 Done Sent Done false
789 1232 Acked Read 6 Done Sent Done false
790 1233 Acked Write 1 Done Sent Done false
791 1234 Acked Read 8 Done Sent Done false
792 1235 Acked Read 9 Done Sent Done false
793 1236 Acked Write 6 Done Sent Done false
794 1237 Acked Read 8 Done Sent Done false
795 1238 Acked Read 2 Done Sent Done false
796 1239 Acked Read 1 Done Sent Done false
797 1240 Acked Read 5 Done Sent Done false
798 1241 Acked Read 4 Done Sent Done false
799 1242 Acked Write 5 Done Sent Done false
800 1243 Acked Write 8 Done Sent Done false
801 1244 Acked Read 9 Done Sent Done false
802 1245 Acked Write 8 Done Sent Done false
803 1246 Acked Read 3 Done Sent Done false
804 1247 Acked Read 10 Done Sent Done false
805 1248 Acked Flush 0 Done Sent Done false
806 1249 Acked Flush 0 Done Sent Done false
807 1250 Acked Read 3 Done Sent Done false
808 1251 Acked Write 5 Done Sent Done false
809 1252 Acked Read 8 Done Sent Done false
810 1253 Acked Write 5 Done Sent Done false
811 1254 Acked Write 9 Done Sent Done false
812 1255 Acked Read 5 Done Sent Done false
813 1256 Acked Write 7 Done Sent Done false
814 1257 Acked Read 6 Done Sent Done false
815 1258 Acked Read 2 Done Sent Done false
816 1259 Acked Read 10 Done Sent Done false
817 1260 Acked Read 6 Done Sent Done false
818 1261 Acked Write 8 Done Sent Done false
819 1262 Acked Write 2 Done Sent Done false
820 1263 Acked Write 5 Done Sent Done false
821 1264 Acked Write 9 Done Sent Done false
822 1265 Acked Write 8 Done Sent Done false
823 1266 Acked Read 6 Done Sent Done false
824 1267 Acked Write 6 Done Sent Done false
825 1268 Acked Read 10 Done Sent Done false
826 1269 Acked Read 8 Done Sent Done false
827 1270 Acked Read 10 Done Sent Done false
828 1271 Acked Read 7 Done Sent Done false
829 1272 Acked Read 4 Done Sent Done false
830 1273 Acked Read 10 Done Sent Done false
831 1274 Acked Write 4 Done Sent Done false
832 1275 Acked Read 6 Done Sent Done false
833 1276 Acked Read 3 Done Sent Done false
834 1277 Acked Flush 0 Done Sent Done false
835 1278 Acked Read 5 Done Sent Done false
836 1279 Acked Read 10 Done Sent Done false
837 1280 Acked Read 4 Done Sent Done false
838 1281 Acked Read 6 Done Sent Done false
839 1282 Acked Write 9 Done Sent Done false
840 1283 Acked Read 1 Done Sent Done false
841 1284 Acked Read 4 Done Sent Done false
842 1285 Acked Read 9 Done Sent Done false
843 1286 Acked Flush 0 Done Sent Done false
844 1287 Acked Write 7 Done Sent Done false
845 1288 Acked Read 9 Done Sent Done false
846 1289 Acked Read 1 Done Sent Done false
847 1290 Acked Read 10 Done Sent Done false
848 1291 Acked Flush 0 Done Sent Done false
849 1292 Acked Read 3 Done Sent Done false
850 1293 Acked Write 1 Done Sent Done false
851 1294 Acked Write 3 Done Sent Done false
852 1295 Acked Read 9 Done Sent Done false
853 1296 Acked Write 8 Done Sent Done false
854 1297 Acked Write 4 Done Sent Done false
855 1298 Acked Write 8 Done Sent Done false
856 1299 Acked Write 7 Done Sent Done false
857 1300 Acked Read 2 Done Sent Done false
858 STATES DS:0 DS:1 DS:2 TOTAL
859 Sent 0 301 0 301
860 Done 301 0 301 602
861 Skipped 0 0 0 0
862 Error 0 0 0 0
863 Last Flush: 1291 0 1291
864 Downstairs last five completed:
865 Upstairs last five completed: 1300 1299 1298 1297 1296
866 CLIENT: Up:0 ds:301 act:2
867 [8820] got start action
868 [8820] I am going to restart
869 Starting downstairs at port 8820
870 Make output file at "/tmp/dsc/downstairs-8820.txt"
871 [8820][1] reports Starting
872 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1205)
873 [8820][1] reports Running
874 ----------------------------------------------------------------
875 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
876 States: Active Offline Active
877 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
878 1000 Acked Write 10 Done Sent Done false
879 1001 Acked Write 7 Done Sent Done false
880 1002 Acked Write 10 Done Sent Done false
881 1003 Acked Write 2 Done Sent Done false
882 1004 Acked Write 7 Done Sent Done false
883 1005 Acked Write 5 Done Sent Done false
884 1006 Acked Read 8 Done Sent Done false
885 1007 Acked Write 10 Done Sent Done false
886 1008 Acked Read 3 Done Sent Done false
887 1009 Acked Read 9 Done Sent Done false
888 1010 Acked Read 7 Done Sent Done false
889 1011 Acked Flush 0 Done Sent Done false
890 1012 Acked Write 9 Done Sent Done false
891 1013 Acked Read 6 Done Sent Done false
892 1014 Acked Write 1 Done Sent Done false
893 1015 Acked Read 3 Done Sent Done false
894 1016 Acked Write 7 Done Sent Done false
895 1017 Acked Write 2 Done Sent Done false
896 1018 Acked Read 3 Done Sent Done false
897 1019 Acked Flush 0 Done Sent Done false
898 1020 Acked Read 2 Done Sent Done false
899 1021 Acked Read 10 Done Sent Done false
900 1022 Acked Read 1 Done Sent Done false
901 1023 Acked Write 8 Done Sent Done false
902 1024 Acked Read 7 Done Sent Done false
903 1025 Acked Write 3 Done Sent Done false
904 1026 Acked Read 9 Done Sent Done false
905 1027 Acked Write 8 Done Sent Done false
906 1028 Acked Write 1 Done Sent Done false
907 1029 Acked Write 2 Done Sent Done false
908 1030 Acked Read 4 Done Sent Done false
909 1031 Acked Flush 0 Done Sent Done false
910 1032 Acked Read 6 Done Sent Done false
911 1033 Acked Read 8 Done Sent Done false
912 1034 Acked Flush 0 Done Sent Done false
913 1035 Acked Flush 0 Done Sent Done false
914 1036 Acked Write 2 Done Sent Done false
915 1037 Acked Write 6 Done Sent Done false
916 1038 Acked Read 1 Done Sent Done false
917 1039 Acked Read 8 Done Sent Done false
918 1040 Acked Write 7 Done Sent Done false
919 1041 Acked Write 6 Done Sent Done false
920 1042 Acked Write 8 Done Sent Done false
921 1043 Acked Read 2 Done Sent Done false
922 1044 Acked Flush 0 Done Sent Done false
923 1045 Acked Write 1 Done Sent Done false
924 1046 Acked Write 3 Done Sent Done false
925 1047 Acked Write 4 Done Sent Done false
926 1048 Acked Read 4 Done Sent Done false
927 1049 Acked Write 3 Done Sent Done false
928 1050 Acked Read 3 Done Sent Done false
929 1051 Acked Write 3 Done Sent Done false
930 1052 Acked Read 1 Done Sent Done false
931 1053 Acked Read 9 Done Sent Done false
932 1054 Acked Write 5 Done Sent Done false
933 1055 Acked Flush 0 Done Sent Done false
934 1056 Acked Write 7 Done Sent Done false
935 1057 Acked Read 5 Done Sent Done false
936 1058 Acked Read 7 Done Sent Done false
937 1059 Acked Read 4 Done Sent Done false
938 1060 Acked Read 7 Done Sent Done false
939 1061 Acked Write 4 Done Sent Done false
940 1062 Acked Read 5 Done Sent Done false
941 1063 Acked Write 8 Done Sent Done false
942 1064 Acked Write 9 Done Sent Done false
943 1065 Acked Write 4 Done Sent Done false
944 1066 Acked Write 7 Done Sent Done false
945 1067 Acked Write 7 Done Sent Done false
946 1068 Acked Write 10 Done Sent Done false
947 1069 Acked Read 1 Done Sent Done false
948 1070 Acked Read 7 Done Sent Done false
949 1071 Acked Write 5 Done Sent Done false
950 1072 Acked Flush 0 Done Sent Done false
951 1073 Acked Write 9 Done Sent Done false
952 1074 Acked Write 7 Done Sent Done false
953 1075 Acked Read 8 Done Sent Done false
954 1076 Acked Write 2 Done Sent Done false
955 1077 Acked Read 10 Done Sent Done false
956 1078 Acked Read 9 Done Sent Done false
957 1079 Acked Read 2 Done Sent Done false
958 1080 Acked Read 9 Done Sent Done false
959 1081 Acked Write 10 Done Sent Done false
960 1082 Acked Flush 0 Done Sent Done false
961 1083 Acked Flush 0 Done Sent Done false
962 1084 Acked Read 9 Done Sent Done false
963 1085 Acked Write 2 Done Sent Done false
964 1086 Acked Flush 0 Done Sent Done false
965 1087 Acked Write 8 Done Sent Done false
966 1088 Acked Read 8 Done Sent Done false
967 1089 Acked Write 7 Done Sent Done false
968 1090 Acked Write 5 Done Sent Done false
969 1091 Acked Write 8 Done Sent Done false
970 1092 Acked Write 3 Done Sent Done false
971 1093 Acked Write 5 Done Sent Done false
972 1094 Acked Read 3 Done Sent Done false
973 1095 Acked Read 5 Done Sent Done false
974 1096 Acked Read 10 Done Sent Done false
975 1097 Acked Write 6 Done Sent Done false
976 1098 Acked Read 10 Done Sent Done false
977 1099 Acked Read 4 Done Sent Done false
978 1100 Acked Write 8 Done Sent Done false
979 1101 Acked Read 4 Done Sent Done false
980 1102 Acked Write 3 Done Sent Done false
981 1103 Acked Write 1 Done Sent Done false
982 1104 Acked Write 2 Done Sent Done false
983 1105 Acked Flush 0 Done Sent Done false
984 1106 Acked Write 8 Done Sent Done false
985 1107 Acked Write 5 Done Sent Done false
986 1108 Acked Read 1 Done Sent Done false
987 1109 Acked Read 9 Done Sent Done false
988 1110 Acked Write 8 Done Sent Done false
989 1111 Acked Read 8 Done Sent Done false
990 1112 Acked Read 10 Done Sent Done false
991 1113 Acked Write 3 Done Sent Done false
992 1114 Acked Read 7 Done Sent Done false
993 1115 Acked Write 8 Done Sent Done false
994 1116 Acked Write 1 Done Sent Done false
995 1117 Acked Write 7 Done Sent Done false
996 1118 Acked Write 10 Done Sent Done false
997 1119 Acked Flush 0 Done Sent Done false
998 1120 Acked Read 7 Done Sent Done false
999 1121 Acked Write 10 Done Sent Done false
1000 1122 Acked Flush 0 Done Sent Done false
1001 1123 Acked Write 6 Done Sent Done false
1002 1124 Acked Write 7 Done Sent Done false
1003 1125 Acked Read 2 Done Sent Done false
1004 1126 Acked Write 2 Done Sent Done false
1005 1127 Acked Read 3 Done Sent Done false
1006 1128 Acked Read 5 Done Sent Done false
1007 1129 Acked Flush 0 Done Sent Done false
1008 1130 Acked Read 4 Done Sent Done false
1009 1131 Acked Flush 0 Done Sent Done false
1010 1132 Acked Write 10 Done Sent Done false
1011 1133 Acked Write 8 Done Sent Done false
1012 1134 Acked Write 6 Done Sent Done false
1013 1135 Acked Read 10 Done Sent Done false
1014 1136 Acked Write 4 Done Sent Done false
1015 1137 Acked Write 5 Done Sent Done false
1016 1138 Acked Read 3 Done Sent Done false
1017 1139 Acked Write 3 Done Sent Done false
1018 1140 Acked Write 4 Done Sent Done false
1019 1141 Acked Write 2 Done Sent Done false
1020 1142 Acked Read 2 Done Sent Done false
1021 1143 Acked Flush 0 Done Sent Done false
1022 1144 Acked Flush 0 Done Sent Done false
1023 1145 Acked Read 9 Done Sent Done false
1024 1146 Acked Read 10 Done Sent Done false
1025 1147 Acked Write 4 Done Sent Done false
1026 1148 Acked Flush 0 Done Sent Done false
1027 1149 Acked Read 3 Done Sent Done false
1028 1150 Acked Write 7 Done Sent Done false
1029 1151 Acked Read 5 Done Sent Done false
1030 1152 Acked Read 3 Done Sent Done false
1031 1153 Acked Read 4 Done Sent Done false
1032 1154 Acked Write 1 Done Sent Done false
1033 1155 Acked Read 7 Done Sent Done false
1034 1156 Acked Read 4 Done Sent Done false
1035 1157 Acked Flush 0 Done Sent Done false
1036 1158 Acked Write 5 Done Sent Done false
1037 1159 Acked Write 6 Done Sent Done false
1038 1160 Acked Read 7 Done Sent Done false
1039 1161 Acked Read 10 Done Sent Done false
1040 1162 Acked Read 10 Done Sent Done false
1041 1163 Acked Read 9 Done Sent Done false
1042 1164 Acked Write 5 Done Sent Done false
1043 1165 Acked Write 5 Done Sent Done false
1044 1166 Acked Write 10 Done Sent Done false
1045 1167 Acked Write 9 Done Sent Done false
1046 1168 Acked Read 7 Done Sent Done false
1047 1169 Acked Write 6 Done Sent Done false
1048 1170 Acked Read 9 Done Sent Done false
1049 1171 Acked Read 3 Done Sent Done false
1050 1172 Acked Write 7 Done Sent Done false
1051 1173 Acked Flush 0 Done Sent Done false
1052 1174 Acked Write 9 Done Sent Done false
1053 1175 Acked Write 4 Done Sent Done false
1054 1176 Acked Write 4 Done Sent Done false
1055 1177 Acked Write 6 Done Sent Done false
1056 1178 Acked Read 3 Done Sent Done false
1057 1179 Acked Read 5 Done Sent Done false
1058 1180 Acked Write 3 Done Sent Done false
1059 1181 Acked Read 10 Done Sent Done false
1060 1182 Acked Read 4 Done Sent Done false
1061 1183 Acked Read 10 Done Sent Done false
1062 1184 Acked Read 9 Done Sent Done false
1063 1185 Acked Flush 0 Done Sent Done false
1064 1186 Acked Read 8 Done Sent Done false
1065 1187 Acked Write 9 Done Sent Done false
1066 1188 Acked Flush 0 Done Sent Done false
1067 1189 Acked Read 4 Done Sent Done false
1068 1190 Acked Read 7 Done Sent Done false
1069 1191 Acked Read 2 Done Sent Done false
1070 1192 Acked Read 1 Done Sent Done false
1071 1193 Acked Read 7 Done Sent Done false
1072 1194 Acked Read 8 Done Sent Done false
1073 1195 Acked Read 8 Done Sent Done false
1074 1196 Acked Read 9 Done Sent Done false
1075 1197 Acked Read 10 Done Sent Done false
1076 1198 Acked Read 4 Done Sent Done false
1077 1199 Acked Write 8 Done Sent Done false
1078 1200 Acked Write 3 Done Sent Done false
1079 1201 Acked Read 1 Done Sent Done false
1080 1202 Acked Read 9 Done Sent Done false
1081 1203 Acked Write 4 Done Sent Done false
1082 1204 Acked Write 10 Done Sent Done false
1083 1205 Acked Write 3 Done Sent Done false
1084 1206 Acked Write 3 Done Sent Done false
1085 1207 Acked Read 5 Done Sent Done false
1086 1208 Acked Write 2 Done Sent Done false
1087 1209 Acked Read 8 Done Sent Done false
1088 1210 Acked Read 4 Done Sent Done false
1089 1211 Acked Flush 0 Done Sent Done false
1090 1212 Acked Read 5 Done Sent Done false
1091 1213 Acked Flush 0 Done Sent Done false
1092 1214 Acked Write 6 Done Sent Done false
1093 1215 Acked Read 2 Done Sent Done false
1094 1216 Acked Write 7 Done Sent Done false
1095 1217 Acked Write 9 Done Sent Done false
1096 1218 Acked Write 8 Done Sent Done false
1097 1219 Acked Write 2 Done Sent Done false
1098 1220 Acked Read 9 Done Sent Done false
1099 1221 Acked Flush 0 Done Sent Done false
1100 1222 Acked Read 2 Done Sent Done false
1101 1223 Acked Write 5 Done Sent Done false
1102 1224 Acked Flush 0 Done Sent Done false
1103 1225 Acked Flush 0 Done Sent Done false
1104 1226 Acked Write 8 Done Sent Done false
1105 1227 Acked Write 4 Done Sent Done false
1106 1228 Acked Read 9 Done Sent Done false
1107 1229 Acked Write 8 Done Sent Done false
1108 1230 Acked Read 3 Done Sent Done false
1109 1231 Acked Read 10 Done Sent Done false
1110 1232 Acked Read 6 Done Sent Done false
1111 1233 Acked Write 1 Done Sent Done false
1112 1234 Acked Read 8 Done Sent Done false
1113 1235 Acked Read 9 Done Sent Done false
1114 1236 Acked Write 6 Done Sent Done false
1115 1237 Acked Read 8 Done Sent Done false
1116 1238 Acked Read 2 Done Sent Done false
1117 1239 Acked Read 1 Done Sent Done false
1118 1240 Acked Read 5 Done Sent Done false
1119 1241 Acked Read 4 Done Sent Done false
1120 1242 Acked Write 5 Done Sent Done false
1121 1243 Acked Write 8 Done Sent Done false
1122 1244 Acked Read 9 Done Sent Done false
1123 1245 Acked Write 8 Done Sent Done false
1124 1246 Acked Read 3 Done Sent Done false
1125 1247 Acked Read 10 Done Sent Done false
1126 1248 Acked Flush 0 Done Sent Done false
1127 1249 Acked Flush 0 Done Sent Done false
1128 1250 Acked Read 3 Done Sent Done false
1129 1251 Acked Write 5 Done Sent Done false
1130 1252 Acked Read 8 Done Sent Done false
1131 1253 Acked Write 5 Done Sent Done false
1132 1254 Acked Write 9 Done Sent Done false
1133 1255 Acked Read 5 Done Sent Done false
1134 1256 Acked Write 7 Done Sent Done false
1135 1257 Acked Read 6 Done Sent Done false
1136 1258 Acked Read 2 Done Sent Done false
1137 1259 Acked Read 10 Done Sent Done false
1138 1260 Acked Read 6 Done Sent Done false
1139 1261 Acked Write 8 Done Sent Done false
1140 1262 Acked Write 2 Done Sent Done false
1141 1263 Acked Write 5 Done Sent Done false
1142 1264 Acked Write 9 Done Sent Done false
1143 1265 Acked Write 8 Done Sent Done false
1144 1266 Acked Read 6 Done Sent Done false
1145 1267 Acked Write 6 Done Sent Done false
1146 1268 Acked Read 10 Done Sent Done false
1147 1269 Acked Read 8 Done Sent Done false
1148 1270 Acked Read 10 Done Sent Done false
1149 1271 Acked Read 7 Done Sent Done false
1150 1272 Acked Read 4 Done Sent Done false
1151 1273 Acked Read 10 Done Sent Done false
1152 1274 Acked Write 4 Done Sent Done false
1153 1275 Acked Read 6 Done Sent Done false
1154 1276 Acked Read 3 Done Sent Done false
1155 1277 Acked Flush 0 Done Sent Done false
1156 1278 Acked Read 5 Done Sent Done false
1157 1279 Acked Read 10 Done Sent Done false
1158 1280 Acked Read 4 Done Sent Done false
1159 1281 Acked Read 6 Done Sent Done false
1160 1282 Acked Write 9 Done Sent Done false
1161 1283 Acked Read 1 Done Sent Done false
1162 1284 Acked Read 4 Done Sent Done false
1163 1285 Acked Read 9 Done Sent Done false
1164 1286 Acked Flush 0 Done Sent Done false
1165 1287 Acked Write 7 Done Sent Done false
1166 1288 Acked Read 9 Done Sent Done false
1167 1289 Acked Read 1 Done Sent Done false
1168 1290 Acked Read 10 Done Sent Done false
1169 1291 Acked Flush 0 Done Sent Done false
1170 1292 Acked Read 3 Done Sent Done false
1171 1293 Acked Write 1 Done Sent Done false
1172 1294 Acked Write 3 Done Sent Done false
1173 1295 Acked Read 9 Done Sent Done false
1174 1296 Acked Write 8 Done Sent Done false
1175 1297 Acked Write 4 Done Sent Done false
1176 1298 Acked Write 8 Done Sent Done false
1177 1299 Acked Write 7 Done Sent Done false
1178 1300 Acked Read 2 Done Sent Done false
1179 1301 Acked Flush 0 Done Sent Done false
1180 STATES DS:0 DS:1 DS:2 TOTAL
1181 Sent 0 302 0 302
1182 Done 302 0 302 604
1183 Skipped 0 0 0 0
1184 Error 0 0 0 0
1185 Last Flush: 1301 0 1301
1186 Downstairs last five completed:
1187 Upstairs last five completed: 1301 1300 1299 1298 1297
1188 CLIENT: Up:0 ds:302 act:2
11892025-01-11T06:05:40.458ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11902025-01-11T06:05:40.459ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49428) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11912025-01-11T06:05:40.460ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11922025-01-11T06:05:40.460ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11932025-01-11T06:05:40.460ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11942025-01-11T06:05:40.460ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11952025-01-11T06:05:40.460ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11962025-01-11T06:05:40.460ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11972025-01-11T06:05:40.460ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11982025-01-11T06:05:40.461ZINFOcrucible: [1] client re-new 302 jobs since flush 0 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11992025-01-11T06:05:40.461ZINFOcrucible: [1] Replayed 302 jobs since flush: 0 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1200 ----------------------------------------------------------------
1201 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1202 Downstairs last five completed: 1301 1300 1299 1298 1297
1203 Upstairs last five completed: 1301 1300 1299 1298 1297
1204 CLIENT: Up:0 ds:0 act:3
1205 Replay: All jobs finished
1206 Main task has work to do, go find it
1207 got dsc Stop(0)
1208 stop 0
1209 Jan 11 06:05:42.710 INFO request completed, latency_us: 349, response_code: 204, uri: /stop/cid/0, method: POST[8810] Got stop action so:false kr:false
1210 , req_id: 96b76d8a-f8f1-44bb-8341-35ba206e540d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
1211 Jan 11 06:05:42.711 INFO request completed, latency_us: 116, response_code: 200, uri: /state/cid/0, method: GET, req_id: 6a47bbb6-5b9d-4e77-a91d-1fb66c748184, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
12122025-01-11T06:05:42.711ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
12132025-01-11T06:05:42.711ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
12142025-01-11T06:05:42.711ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
12152025-01-11T06:05:42.711ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
12162025-01-11T06:05:42.711ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
12172025-01-11T06:05:42.711ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
1218 [8810] Exited with: ExitStatus(unix_wait_status(9))
1219 [8810][0] reports Exit
1220 Jan 11 06:05:46.714 INFO request completed, latency_us: 326, response_code: 200, uri: /state/cid/0, method: GET, req_id: 16f40cee-a749-4ecc-84d9-80fd240483a9, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
1221 001/300 Write block 1233 len 20480 data: 2 2 3 3 3
1222 002/300 Read block 2016 len 28672
1223 003/300 Read block 918 len 8192
1224 004/300 Write block 3238 len 4096 data: 2
1225 005/300 Flush
1226 006/300 Write block 325 len 28672 data: 3 3 3 3 3 3 3
1227 007/300 Read block 4198 len 8192
1228 008/300 Read block 1738 len 12288
1229 009/300 Write block 3455 len 16384 data: 2 2 2 2
1230 010/300 Write block 3488 len 36864 data: 2 2 2 2 2 2 2 2 2
1231 011/300 Write block 3270 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1232 012/300 Read block 3660 len 28672
1233 013/300 Read block 1316 len 8192
1234 014/300 Read block 1024 len 4096
1235 015/300 Read block 542 len 12288
1236 016/300 Read block 3430 len 36864
1237 017/300 Flush
1238 018/300 Write block 425 len 32768 data: 2 2 2 2 2 2 2 2
1239 019/300 Read block 2267 len 32768
1240 020/300 Write block 909 len 8192 data: 2 2
1241 021/300 Read block 3751 len 28672
1242 022/300 Read block 3067 len 28672
1243 023/300 Read block 1035 len 28672
1244 024/300 Read block 2695 len 32768
1245 025/300 Write block 4666 len 32768 data: 2 2 2 2 2 3 3 3
1246 026/300 Read block 1142 len 16384
1247 027/300 Write block 669 len 36864 data: 2 2 2 2 2 2 2 2 2
1248 028/300 Read block 4612 len 40960
1249 029/300 Write block 3186 len 20480 data: 2 2 2 2 2
1250 030/300 Flush
1251 031/300 Write block 2313 len 12288 data: 2 2 2
1252 032/300 Write block 4359 len 16384 data: 2 2 2 2
1253 033/300 Read block 3185 len 28672
1254 034/300 Read block 2273 len 12288
1255 035/300 Flush
1256 036/300 Read block 180 len 24576
1257 037/300 Read block 2474 len 28672
1258 038/300 Read block 4193 len 16384
1259 039/300 Read block 1075 len 24576
1260 040/300 Read block 4102 len 40960
1261 041/300 Flush
1262 042/300 Write block 37 len 16384 data: 3 3 3 3
1263 043/300 Read block 4267 len 24576
1264 044/300 Write block 480 len 28672 data: 2 2 2 3 3 3 3
1265 045/300 Write block 4301 len 16384 data: 3 3 2 2
1266 046/300 Read block 2026 len 12288
1267 047/300 Read block 3919 len 24576
1268 048/300 Read block 112 len 32768
1269 049/300 Write block 1669 len 20480 data: 2 2 2 2 2
1270 050/300 Write block 2704 len 28672 data: 2 2 2 2 2 2 2
1271 051/300 Write block 2299 len 28672 data: 2 2 2 2 2 2 2
1272 052/300 Write block 3425 len 12288 data: 2 2 2
1273 053/300 Read block 1649 len 12288
1274 054/300 Read block 2391 len 40960
1275 055/300 Write block 3444 len 32768 data: 2 2 3 3 3 3 3 3
1276 056/300 Read block 3067 len 4096
1277 057/300 Read block 3614 len 16384
1278 058/300 Write block 3227 len 8192 data: 2 2
1279 059/300 Read block 3980 len 12288
1280 060/300 Write block 304 len 24576 data: 2 2 2 2 2 2
1281 061/300 Read block 1497 len 4096
1282 062/300 Read block 680 len 12288
1283 063/300 Write block 2527 len 12288 data: 2 2 2
1284 064/300 Read block 1518 len 16384
1285 065/300 Flush
1286 066/300 Write block 1226 len 36864 data: 2 2 2 2 2 2 2 3 3
1287 067/300 Write block 921 len 4096 data: 2
1288 068/300 Read block 434 len 12288
1289 069/300 Read block 1185 len 8192
1290 070/300 Write block 1061 len 20480 data: 3 3 3 2 2
1291 071/300 Write block 3576 len 4096 data: 2
1292 072/300 Write block 946 len 4096 data: 2
1293 073/300 Flush
1294 074/300 Write block 3399 len 24576 data: 2 2 2 2 2 2
1295 075/300 Flush
1296 076/300 Read block 2173 len 12288
1297 077/300 Flush
1298 078/300 Flush
1299 079/300 Write block 466 len 16384 data: 3 3 3 3
1300 080/300 Flush
1301 081/300 Write block 4351 len 12288 data: 2 2 2
1302 082/300 Read block 167 len 20480
1303 083/300 Read block 1652 len 12288
1304 084/300 Read block 3296 len 4096
1305 085/300 Write block 3076 len 8192 data: 2 2
1306 086/300 Read block 1517 len 12288
1307 087/300 Read block 1252 len 36864
1308 088/300 Write block 830 len 36864 data: 2 2 2 2 2 2 2 2 2
1309 089/300 Read block 141 len 8192
1310 090/300 Read block 1929 len 32768
1311 091/300 Read block 3167 len 16384
1312 092/300 Read block 3380 len 36864
1313 093/300 Write block 743 len 16384 data: 2 2 2 2
1314 094/300 Write block 4811 len 12288 data: 2 2 2
1315 095/300 Read block 1792 len 40960
1316 096/300 Read block 3291 len 16384
1317 097/300 Read block 838 len 28672
1318 098/300 Write block 1879 len 28672 data: 2 2 2 2 2 2 2
1319 099/300 Read block 2896 len 16384
1320 100/300 Write block 1175 len 20480 data: 2 2 2 2 2
1321 101/300 Write block 4948 len 28672 data: 2 2 2 2 2 2 2
1322 102/300 Write block 2512 len 8192 data: 2 2
1323 103/300 Read block 1104 len 16384
1324 104/300 Write block 1278 len 16384 data: 2 2 2 2
1325 105/300 Write block 902 len 40960 data: 2 2 2 2 2 2 2 3 3 3
1326 106/300 Write block 3246 len 36864 data: 2 2 2 2 2 2 2 2 2
1327 107/300 Flush
1328 108/300 Read block 2406 len 36864
1329 109/300 Write block 586 len 4096 data: 2
1330 110/300 Write block 1566 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1331 111/300 Read block 4521 len 16384
1332 112/300 Write block 3680 len 16384 data: 2 2 2 2
1333 113/300 Flush
1334 114/300 Write block 2522 len 16384 data: 2 2 2 2
1335 115/300 Read block 166 len 16384
1336 116/300 Read block 4723 len 36864
1337 117/300 Write block 3337 len 8192 data: 2 2
1338 118/300 Read block 97 len 12288
1339 119/300 Read block 3944 len 20480
1340 120/300 Write block 4836 len 4096 data: 2
1341 121/300 Write block 3940 len 28672 data: 3 3 3 3 3 2 2
1342 122/300 Read block 4493 len 24576
1343 123/300 Read block 130 len 16384
1344 124/300 Read block 1183 len 20480
1345 125/300 Read block 297 len 12288
1346 126/300 Write block 965 len 12288 data: 2 2 2
1347 127/300 Read block 831 len 36864
1348 128/300 Write block 1825 len 24576 data: 2 2 2 2 2 2
1349 129/300 Read block 3037 len 16384
1350 130/300 Flush
1351 131/300 Write block 4590 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1352 132/300 Write block 1418 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1353 133/300 Write block 1846 len 20480 data: 2 2 2 2 2
1354 134/300 Read block 4307 len 36864
1355 135/300 Read block 2602 len 20480
1356 136/300 Read block 3419 len 20480
1357 137/300 Write block 4972 len 12288 data: 2 2 2
1358 138/300 Write block 577 len 4096 data: 2
1359 139/300 Read block 2299 len 20480
1360 140/300 Flush
1361 141/300 Read block 4775 len 8192
1362 142/300 Read block 2048 len 24576
1363 143/300 Write block 166 len 16384 data: 2 2 2 2
1364 144/300 Read block 1608 len 12288
1365 145/300 Write block 236 len 24576 data: 2 2 2 2 2 2
1366 146/300 Read block 3766 len 4096
1367 147/300 Read block 4172 len 20480
1368 148/300 Write block 3746 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1369 149/300 Read block 806 len 28672
1370 150/300 Write block 2682 len 8192 data: 2 2
1371 151/300 Read block 1507 len 32768
1372 152/300 Read block 2185 len 32768
1373 153/300 Write block 316 len 4096 data: 2
1374 154/300 Flush
1375 155/300 Write block 2509 len 32768 data: 2 2 2 3 3 2 2 2
1376 156/300 Read block 4931 len 40960
1377 157/300 Read block 519 len 20480
1378 158/300 Read block 3679 len 20480
1379 159/300 Read block 606 len 28672
1380 160/300 Read block 2951 len 16384
1381 161/300 Write block 4709 len 4096 data: 2
1382 162/300 Read block 321 len 40960
1383 163/300 Flush
1384 164/300 Write block 2027 len 24576 data: 2 2 2 2 2 2
1385 165/300 Read block 459 len 40960
1386 166/300 Write block 1993 len 8192 data: 3 3
1387 167/300 Write block 3655 len 12288 data: 2 2 2
1388 168/300 Write block 3513 len 16384 data: 2 2 2 2
1389 169/300 Flush
1390 170/300 Read block 4872 len 24576
1391 171/300 Write block 3119 len 20480 data: 2 2 2 2 2
1392 172/300 Flush
1393 173/300 Write block 956 len 4096 data: 3
1394 174/300 Read block 3527 len 4096
1395 175/300 Write block 2818 len 28672 data: 2 2 2 2 2 2 2
1396 176/300 Flush
1397 177/300 Read block 3316 len 40960
1398 178/300 Read block 2656 len 20480
1399 179/300 Read block 344 len 40960
1400 180/300 Flush
1401 181/300 Read block 1903 len 8192
1402 182/300 Read block 1940 len 20480
1403 183/300 Write block 1700 len 24576 data: 2 2 2 2 2 2
1404 184/300 Read block 2303 len 36864
1405 185/300 Write block 385 len 28672 data: 2 2 2 2 2 2 2
1406 186/300 Read block 3333 len 8192
1407 187/300 Read block 748 len 36864
1408 188/300 Read block 325 len 40960
1409 189/300 Flush
1410 190/300 Write block 4565 len 8192 data: 2 2
1411 191/300 Flush
1412 192/300 Write block 2891 len 24576 data: 2 2 2 2 2 2
1413 193/300 Write block 2787 len 36864 data: 2 2 2 2 2 2 2 2 2
1414 194/300 Write block 1471 len 4096 data: 2
1415 195/300 Write block 2173 len 12288 data: 2 2 3
1416 196/300 Write block 4531 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1417 197/300 Read block 3909 len 8192
1418 198/300 Read block 1337 len 32768
1419 199/300 Flush
1420 200/300 Read block 748 len 16384
1421 201/300 Read block 1431 len 36864
1422 202/300 Read block 1332 len 16384
1423 203/300 Read block 552 len 36864
1424 204/300 Read block 625 len 20480
1425 205/300 Write block 1290 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1426 206/300 Write block 3547 len 8192 data: 2 2
1427 207/300 Write block 565 len 16384 data: 2 2 2 2
1428 208/300 Write block 3522 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1429 209/300 Write block 2358 len 16384 data: 3 2 2 2
1430 210/300 Read block 3311 len 24576
1431 211/300 Read block 676 len 32768
1432 212/300 Read block 4011 len 32768
1433 213/300 Read block 3783 len 8192
1434 214/300 Write block 1780 len 36864 data: 2 2 2 2 2 2 2 2 2
1435 215/300 Read block 2833 len 40960
1436 216/300 Write block 23 len 8192 data: 2 2
1437 217/300 Read block 4018 len 40960
1438 218/300 Read block 569 len 24576
1439 219/300 Read block 4687 len 36864
1440 220/300 Read block 1683 len 16384
1441 221/300 Read block 14 len 40960
1442 222/300 Read block 4105 len 36864
1443 223/300 Write block 2395 len 40960 data: 3 2 2 2 2 2 2 2 2 2
1444 224/300 Write block 3560 len 32768 data: 2 2 2 2 2 2 2 2
1445 225/300 Read block 3219 len 40960
1446 226/300 Read block 3071 len 28672
1447 227/300 Write block 1727 len 28672 data: 3 3 3 3 2 2 2
1448 228/300 Flush
1449 229/300 Write block 980 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1450 230/300 Write block 2474 len 28672 data: 2 2 2 2 2 2 2
1451 231/300 Read block 1174 len 40960
1452 232/300 Read block 2189 len 16384
1453 233/300 Flush
1454 234/300 Flush
1455 235/300 Read block 1329 len 8192
1456 236/300 Read block 4745 len 40960
1457 237/300 Read block 4873 len 24576
1458 238/300 Read block 3634 len 16384
1459 239/300 Write block 3810 len 32768 data: 2 2 2 2 2 2 2 2
1460 240/300 Read block 1481 len 24576
1461 241/300 Read block 1685 len 36864
1462 242/300 Read block 1239 len 36864
1463 243/300 Read block 3239 len 8192
1464 244/300 Read block 2767 len 4096
1465 245/300 Read block 1223 len 28672
1466 246/300 Flush
1467 247/300 Write block 3277 len 32768 data: 3 3 3 2 2 2 2 2
1468 248/300 Flush
1469 249/300 Write block 3450 len 12288 data: 4 4 3
1470 250/300 Read block 4450 len 16384
1471 251/300 Flush
1472 252/300 Read block 4799 len 28672
1473 253/300 Write block 578 len 24576 data: 2 2 2 2 2 2
1474 254/300 Read block 1990 len 8192
1475 255/300 Write block 4264 len 4096 data: 2
1476 256/300 Write block 1106 len 12288 data: 2 2 2
1477 257/300 Write block 2426 len 28672 data: 2 2 2 2 2 2 2
1478 258/300 Read block 3859 len 24576
1479 259/300 Flush
1480 260/300 Write block 3027 len 32768 data: 2 2 2 2 2 2 2 2
1481 261/300 Read block 3405 len 16384
1482 262/300 Write block 4487 len 20480 data: 3 3 3 3 3
1483 263/300 Read block 1541 len 36864
1484 264/300 Flush
1485 265/300 Write block 4238 len 28672 data: 2 2 2 2 3 4 4
1486 266/300 Read block 4999 len 4096
1487 267/300 Read block 4362 len 24576
1488 268/300 Read block 270 len 24576
1489 269/300 Read block 3841 len 20480
1490 270/300 Read block 339 len 24576
1491 271/300 Read block 3165 len 20480
1492 272/300 Write block 1827 len 28672 data: 3 3 3 3 2 2 2
1493 273/300 Read block 3193 len 32768
1494 274/300 Flush
1495 275/300 Read block 2240 len 28672
1496 276/300 Write block 4437 len 4096 data: 2
1497 277/300 Write block 3999 len 20480 data: 2 2 2 2 3
1498 278/300 Write block 3100 len 32768 data: 3 3 3 3 2 2 2 2
1499 279/300 Read block 1058 len 28672
1500 280/300 Read block 3313 len 12288
1501 281/300 Write block 4297 len 28672 data: 3 3 3 3 4 4 3
1502 282/300 Write block 3004 len 24576 data: 2 2 2 2 2 2
1503 283/300 Read block 4666 len 40960
1504 284/300 Read block 4761 len 32768
1505 285/300 Write block 4107 len 4096 data: 2
1506 286/300 Write block 4696 len 36864 data: 3 3 3 3 2 2 2 2 2
1507 287/300 Write block 2642 len 32768 data: 2 2 2 2 2 2 2 2
1508 288/300 Write block 4826 len 12288 data: 2 2 2
1509 289/300 Read block 1244 len 32768
1510 290/300 Read block 2739 len 16384
1511 291/300 Write block 2271 len 12288 data: 2 2 2
1512 292/300 Read block 3887 len 8192
1513 293/300 Write block 2156 len 4096 data: 2
1514 294/300 Read block 183 len 28672
1515 295/300 Write block 4141 len 24576 data: 3 3 3 3 2 2
1516 296/300 Read block 752 len 20480
1517 297/300 Read block 3398 len 12288
1518 298/300 Read block 246 len 12288
1519 299/300 Flush
1520 300/300 Read block 1303 len 24576
1521 301/300 Write block 4904 len 24576 data: 2 2 2 2 2 2
1522 Main task has work to do, go find it
1523 got dsc Start(0)
1524 start 0
1525 Jan 11 06:05:46.959 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/0, method: POST, req_id: 8cc8b6c8-9910-4815-ab25-505ddb0828c1, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
1526 Replay: started 0, returned:Ok(())
1527 ----------------------------------------------------------------
1528 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1529 States: Offline Active Active
1530 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1531 1302 Acked Write 5 Sent Done Done false
1532 1303 Acked Read 7 Sent Done Done false
1533 1304 Acked Read 2 Sent Done Done false
1534 1305 Acked Write 1 Sent Done Done false
1535 1306 Acked Flush 0 Sent Done Done false
1536 1307 Acked Write 7 Sent Done Done false
1537 1308 Acked Read 2 Sent Done Done false
1538 1309 Acked Read 3 Sent Done Done false
1539 1310 Acked Write 4 Sent Done Done false
1540 1311 Acked Write 9 Sent Done Done false
1541 1312 Acked Write 10 Sent Done Done false
1542 1313 Acked Read 7 Sent Done Done false
1543 1314 Acked Read 2 Sent Done Done false
1544 1315 Acked Read 1 Sent Done Done false
1545 1316 Acked Read 3 Sent Done Done false
1546 1317 Acked Read 9 Sent Done Done false
1547 1318 Acked Flush 0 Sent Done Done false
1548 1319 Acked Write 8 Sent Done Done false
1549 1320 Acked Read 8 Sent Done Done false
1550 1321 Acked Write 2 Sent Done Done false
1551 1322 Acked Read 7 Sent Done Done false
1552 1323 Acked Read 7 Sent Done Done false
1553 1324 Acked Read 7 Sent Done Done false
1554 1325 Acked Read 8 Sent Done Done false
1555 1326 Acked Write 8 Sent Done Done false
1556 1327 Acked Read 4 Sent Done Done false
1557 1328 Acked Write 9 Sent Done Done false
1558 1329 Acked Read 10 Sent Done Done false
1559 1330 Acked Write 5 Sent Done Done false
1560 1331 Acked Flush 0 Sent Done Done false
1561 1332 Acked Write 3 Sent Done Done false
1562 1333 Acked Write 4 Sent Done Done false
1563 1334 Acked Read 7 Sent Done Done false
1564 1335 Acked Read 3 Sent Done Done false
1565 1336 Acked Flush 0 Sent Done Done false
1566 1337 Acked Read 6 Sent Done Done false
1567 1338 Acked Read 7 Sent Done Done false
1568 1339 Acked Read 4 Sent Done Done false
1569 1340 Acked Read 6 Sent Done Done false
1570 1341 Acked Read 10 Sent Done Done false
1571 1342 Acked Flush 0 Sent Done Done false
1572 1343 Acked Write 4 Sent Done Done false
1573 1344 Acked Read 6 Sent Done Done false
1574 1345 Acked Write 7 Sent Done Done false
1575 1346 Acked Write 4 Sent Done Done false
1576 1347 Acked Read 3 Sent Done Done false
1577 1348 Acked Read 6 Sent Done Done false
1578 1349 Acked Read 8 Sent Done Done false
1579 1350 Acked Write 5 Sent Done Done false
1580 1351 Acked Write 7 Sent Done Done false
1581 1352 Acked Write 7 Sent Done Done false
1582 1353 Acked Write 3 Sent Done Done false
1583 1354 Acked Read 3 Sent Done Done false
1584 1355 Acked Read 10 Sent Done Done false
1585 1356 Acked Write 8 Sent Done Done false
1586 1357 Acked Read 1 Sent Done Done false
1587 1358 Acked Read 4 Sent Done Done false
1588 1359 Acked Write 2 Sent Done Done false
1589 1360 Acked Read 3 Sent Done Done false
1590 1361 Acked Write 6 Sent Done Done false
1591 1362 Acked Read 1 Sent Done Done false
1592 1363 Acked Read 3 Sent Done Done false
1593 1364 Acked Write 3 Sent Done Done false
1594 1365 Acked Read 4 Sent Done Done false
1595 1366 Acked Flush 0 Sent Done Done false
1596 1367 Acked Write 9 Sent Done Done false
1597 1368 Acked Write 1 Sent Done Done false
1598 1369 Acked Read 3 Sent Done Done false
1599 1370 Acked Read 2 Sent Done Done false
1600 1371 Acked Write 5 Sent Done Done false
1601 1372 Acked Write 1 Sent Done Done false
1602 1373 Acked Write 1 Sent Done Done false
1603 1374 Acked Flush 0 Sent Done Done false
1604 1375 Acked Write 6 Sent Done Done false
1605 1376 Acked Flush 0 Sent Done Done false
1606 1377 Acked Read 3 Sent Done Done false
1607 1378 Acked Flush 0 Sent Done Done false
1608 1379 Acked Flush 0 Sent Done Done false
1609 1380 Acked Write 4 Sent Done Done false
1610 1381 Acked Flush 0 Sent Done Done false
1611 1382 Acked Write 3 Sent Done Done false
1612 1383 Acked Read 5 Sent Done Done false
1613 1384 Acked Read 3 Sent Done Done false
1614 1385 Acked Read 1 Sent Done Done false
1615 1386 Acked Write 2 Sent Done Done false
1616 1387 Acked Read 3 Sent Done Done false
1617 1388 Acked Read 9 Sent Done Done false
1618 1389 Acked Write 9 Sent Done Done false
1619 1390 Acked Read 2 Sent Done Done false
1620 1391 Acked Read 8 Sent Done Done false
1621 1392 Acked Read 4 Sent Done Done false
1622 1393 Acked Read 9 Sent Done Done false
1623 1394 Acked Write 4 Sent Done Done false
1624 1395 Acked Write 3 Sent Done Done false
1625 1396 Acked Read 10 Sent Done Done false
1626 1397 Acked Read 4 Sent Done Done false
1627 1398 Acked Read 7 Sent Done Done false
1628 1399 Acked Write 7 Sent Done Done false
1629 1400 Acked Read 4 Sent Done Done false
1630 1401 Acked Write 5 Sent Done Done false
1631 1402 Acked Write 7 Sent Done Done false
1632 1403 Acked Write 2 Sent Done Done false
1633 1404 Acked Read 4 Sent Done Done false
1634 1405 Acked Write 4 Sent Done Done false
1635 1406 Acked Write 10 Sent Done Done false
1636 1407 Acked Write 9 Sent Done Done false
1637 1408 Acked Flush 0 Sent Done Done false
1638 1409 Acked Read 9 Sent Done Done false
1639 1410 Acked Write 1 Sent Done Done false
1640 1411 Acked Write 10 Sent Done Done false
1641 1412 Acked Read 4 Sent Done Done false
1642 1413 Acked Write 4 Sent Done Done false
1643 1414 Acked Flush 0 Sent Done Done false
1644 1415 Acked Write 4 Sent Done Done false
1645 1416 Acked Read 4 Sent Done Done false
1646 1417 Acked Read 9 Sent Done Done false
1647 1418 Acked Write 2 Sent Done Done false
1648 1419 Acked Read 3 Sent Done Done false
1649 1420 Acked Read 5 Sent Done Done false
1650 1421 Acked Write 1 Sent Done Done false
1651 1422 Acked Write 7 Sent Done Done false
1652 1423 Acked Read 6 Sent Done Done false
1653 1424 Acked Read 4 Sent Done Done false
1654 1425 Acked Read 5 Sent Done Done false
1655 1426 Acked Read 3 Sent Done Done false
1656 1427 Acked Write 3 Sent Done Done false
1657 1428 Acked Read 9 Sent Done Done false
1658 1429 Acked Write 6 Sent Done Done false
1659 1430 Acked Read 4 Sent Done Done false
1660 1431 Acked Flush 0 Sent Done Done false
1661 1432 Acked Write 10 Sent Done Done false
1662 1433 Acked Write 10 Sent Done Done false
1663 1434 Acked Write 5 Sent Done Done false
1664 1435 Acked Read 9 Sent Done Done false
1665 1436 Acked Read 5 Sent Done Done false
1666 1437 Acked Read 5 Sent Done Done false
1667 1438 Acked Write 3 Sent Done Done false
1668 1439 Acked Write 1 Sent Done Done false
1669 1440 Acked Read 5 Sent Done Done false
1670 1441 Acked Flush 0 Sent Done Done false
1671 1442 Acked Read 2 Sent Done Done false
1672 1443 Acked Read 6 Sent Done Done false
1673 1444 Acked Write 4 Sent Done Done false
1674 1445 Acked Read 3 Sent Done Done false
1675 1446 Acked Write 6 Sent Done Done false
1676 1447 Acked Read 1 Sent Done Done false
1677 1448 Acked Read 5 Sent Done Done false
1678 1449 Acked Write 10 Sent Done Done false
1679 1450 Acked Read 7 Sent Done Done false
1680 1451 Acked Write 2 Sent Done Done false
1681 1452 Acked Read 8 Sent Done Done false
1682 1453 Acked Read 8 Sent Done Done false
1683 1454 Acked Write 1 Sent Done Done false
1684 1455 Acked Flush 0 Sent Done Done false
1685 1456 Acked Write 8 Sent Done Done false
1686 1457 Acked Read 10 Sent Done Done false
1687 1458 Acked Read 5 Sent Done Done false
1688 1459 Acked Read 5 Sent Done Done false
1689 1460 Acked Read 7 Sent Done Done false
1690 1461 Acked Read 4 Sent Done Done false
1691 1462 Acked Write 1 Sent Done Done false
1692 1463 Acked Read 10 Sent Done Done false
1693 1464 Acked Flush 0 Sent Done Done false
1694 1465 Acked Write 6 Sent Done Done false
1695 1466 Acked Read 10 Sent Done Done false
1696 1467 Acked Write 2 Sent Done Done false
1697 1468 Acked Write 3 Sent Done Done false
1698 1469 Acked Write 4 Sent Done Done false
1699 1470 Acked Flush 0 Sent Done Done false
1700 1471 Acked Read 6 Sent Done Done false
1701 1472 Acked Write 5 Sent Done Done false
1702 1473 Acked Flush 0 Sent Done Done false
1703 1474 Acked Write 1 Sent Done Done false
1704 1475 Acked Read 1 Sent Done Done false
1705 1476 Acked Write 7 Sent Done Done false
1706 1477 Acked Flush 0 Sent Done Done false
1707 1478 Acked Read 10 Sent Done Done false
1708 1479 Acked Read 5 Sent Done Done false
1709 1480 Acked Read 10 Sent Done Done false
1710 1481 Acked Flush 0 Sent Done Done false
1711 1482 Acked Read 2 Sent Done Done false
1712 1483 Acked Read 5 Sent Done Done false
1713 1484 Acked Write 6 Sent Done Done false
1714 1485 Acked Read 9 Sent Done Done false
1715 1486 Acked Write 7 Sent Done Done false
1716 1487 Acked Read 2 Sent Done Done false
1717 1488 Acked Read 9 Sent Done Done false
1718 1489 Acked Read 10 Sent Done Done false
1719 1490 Acked Flush 0 Sent Done Done false
1720 1491 Acked Write 2 Sent Done Done false
1721 1492 Acked Flush 0 Sent Done Done false
1722 1493 Acked Write 6 Sent Done Done false
1723 1494 Acked Write 9 Sent Done Done false
1724 1495 Acked Write 1 Sent Done Done false
1725 1496 Acked Write 3 Sent Done Done false
1726 1497 Acked Write 10 Sent Done Done false
1727 1498 Acked Read 2 Sent Done Done false
1728 1499 Acked Read 8 Sent Done Done false
1729 1500 Acked Flush 0 Sent Done Done false
1730 1501 Acked Read 4 Sent Done Done false
1731 1502 Acked Read 9 Sent Done Done false
1732 1503 Acked Read 4 Sent Done Done false
1733 1504 Acked Read 9 Sent Done Done false
1734 1505 Acked Read 5 Sent Done Done false
1735 1506 Acked Write 10 Sent Done Done false
1736 1507 Acked Write 2 Sent Done Done false
1737 1508 Acked Write 4 Sent Done Done false
1738 1509 Acked Write 10 Sent Done Done false
1739 1510 Acked Write 4 Sent Done Done false
1740 1511 Acked Read 6 Sent Done Done false
1741 1512 Acked Read 8 Sent Done Done false
1742 1513 Acked Read 8 Sent Done Done false
1743 1514 Acked Read 2 Sent Done Done false
1744 1515 Acked Write 9 Sent Done Done false
1745 1516 Acked Read 10 Sent Done Done false
1746 1517 Acked Write 2 Sent Done Done false
1747 1518 Acked Read 10 Sent Done Done false
1748 1519 Acked Read 6 Sent Done Done false
1749 1520 Acked Read 9 Sent Done Done false
1750 1521 Acked Read 4 Sent Done Done false
1751 1522 Acked Read 10 Sent Done Done false
1752 1523 Acked Read 9 Sent Done Done false
1753 1524 Acked Write 10 Sent Done Done false
1754 1525 Acked Write 8 Sent Done Done false
1755 1526 Acked Read 10 Sent Done Done false
1756 1527 Acked Read 7 Sent Done Done false
1757 1528 Acked Write 7 Sent Done Done false
1758 1529 Acked Flush 0 Sent Done Done false
1759 1530 Acked Write 10 Sent Done Done false
1760 1531 Acked Write 7 Sent Done Done false
1761 1532 Acked Read 10 Sent Done Done false
1762 1533 Acked Read 4 Sent Done Done false
1763 1534 Acked Flush 0 Sent Done Done false
1764 1535 Acked Flush 0 Sent Done Done false
1765 1536 Acked Read 2 Sent Done Done false
1766 1537 Acked Read 10 Sent Done Done false
1767 1538 Acked Read 6 Sent Done Done false
1768 1539 Acked Read 4 Sent Done Done false
1769 1540 Acked Write 8 Sent Done Done false
1770 1541 Acked Read 6 Sent Done Done false
1771 1542 Acked Read 9 Sent Done Done false
1772 1543 Acked Read 9 Sent Done Done false
1773 1544 Acked Read 2 Sent Done Done false
1774 1545 Acked Read 1 Sent Done Done false
1775 1546 Acked Read 7 Sent Done Done false
1776 1547 Acked Flush 0 Sent Done Done false
1777 1548 Acked Write 8 Sent Done Done false
1778 1549 Acked Flush 0 Sent Done Done false
1779 1550 Acked Write 3 Sent Done Done false
1780 1551 Acked Read 4 Sent Done Done false
1781 1552 Acked Flush 0 Sent Done Done false
1782 1553 Acked Read 7 Sent Done Done false
1783 1554 Acked Write 6 Sent Done Done false
1784 1555 Acked Read 2 Sent Done Done false
1785 1556 Acked Write 1 Sent Done Done false
1786 1557 Acked Write 3 Sent Done Done false
1787 1558 Acked Write 7 Sent Done Done false
1788 1559 Acked Read 6 Sent Done Done false
1789 1560 Acked Flush 0 Sent Done Done false
1790 1561 Acked Write 8 Sent Done Done false
1791 1562 Acked Read 4 Sent Done Done false
1792 1563 Acked Write 5 Sent Done Done false
1793 1564 Acked Read 9 Sent Done Done false
1794 1565 Acked Flush 0 Sent Done Done false
1795 1566 Acked Write 7 Sent Done Done false
1796 1567 Acked Read 1 Sent Done Done false
1797 1568 Acked Read 6 Sent Done Done false
1798 1569 Acked Read 6 Sent Done Done false
1799 1570 Acked Read 5 Sent Done Done false
1800 1571 Acked Read 6 Sent Done Done false
1801 1572 Acked Read 5 Sent Done Done false
1802 1573 Acked Write 7 Sent Done Done false
1803 1574 Acked Read 8 Sent Done Done false
1804 1575 Acked Flush 0 Sent Done Done false
1805 1576 Acked Read 7 Sent Done Done false
1806 1577 Acked Write 1 Sent Done Done false
1807 1578 Acked Write 5 Sent Done Done false
1808 1579 Acked Write 8 Sent Done Done false
1809 1580 Acked Read 7 Sent Done Done false
1810 1581 Acked Read 3 Sent Done Done false
1811 1582 Acked Write 7 Sent Done Done false
1812 1583 Acked Write 6 Sent Done Done false
1813 1584 Acked Read 10 Sent Done Done false
1814 1585 Acked Read 8 Sent Done Done false
1815 1586 Acked Write 1 Sent Done Done false
1816 1587 Acked Write 9 Sent Done Done false
1817 1588 Acked Write 8 Sent Done Done false
1818 1589 Acked Write 3 Sent Done Done false
1819 1590 Acked Read 8 Sent Done Done false
1820 1591 Acked Read 4 Sent Done Done false
1821 1592 Acked Write 3 Sent Done Done false
1822 1593 Acked Read 2 Sent Done Done false
1823 1594 Acked Write 1 Sent Done Done false
1824 1595 Acked Read 7 Sent Done Done false
1825 1596 Acked Write 6 Sent Done Done false
1826 1597 Acked Read 5 Sent Done Done false
1827 1598 Acked Read 3 Sent Done Done false
1828 1599 Acked Read 3 Sent Done Done false
1829 1600 Acked Flush 0 Sent Done Done false
1830 1601 Acked Read 6 Sent Done Done false
1831 1602 Acked Write 6 Sent Done Done false
1832 STATES DS:0 DS:1 DS:2 TOTAL
1833 Sent 301 0 0 301
1834 Done 0 301 301 602
1835 Skipped 0 0 0 0
1836 Error 0 0 0 0
1837 Last Flush: 1301 1600 1600
1838 Downstairs last five completed: 1301 1300 1299 1298 1297
1839 Upstairs last five completed: 1602 1601 1600 1599 1598
1840 CLIENT: Up:0 ds:301 act:2
1841 ----------------------------------------------------------------
1842 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1843 States: Offline Active Active
1844 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1845 1302 Acked Write 5 Sent Done Done false
1846 1303 Acked Read 7 Sent Done Done false
1847 1304 Acked Read 2 Sent Done Done false
1848 1305 Acked Write 1 Sent Done Done false
1849 1306 Acked Flush 0 Sent Done Done false
1850 1307 Acked Write 7 Sent Done Done false
1851 1308 Acked Read 2 Sent Done Done false
1852 1309 Acked Read 3 Sent Done Done false
1853 1310 Acked Write 4 Sent Done Done false
1854 1311 Acked Write 9 Sent Done Done false
1855 1312 Acked Write 10 Sent Done Done false
1856 1313 Acked Read 7 Sent Done Done false
1857 1314 Acked Read 2 Sent Done Done false
1858 1315 Acked Read 1 Sent Done Done false
1859 1316 Acked Read 3 Sent Done Done false
1860 1317 Acked Read 9 Sent Done Done false
1861 1318 Acked Flush 0 Sent Done Done false
1862 1319 Acked Write 8 Sent Done Done false
1863 1320 Acked Read 8 Sent Done Done false
1864 1321 Acked Write 2 Sent Done Done false
1865 1322 Acked Read 7 Sent Done Done false
1866 1323 Acked Read 7 Sent Done Done false
1867 1324 Acked Read 7 Sent Done Done false
1868 1325 Acked Read 8 Sent Done Done false
1869 1326 Acked Write 8 Sent Done Done false
1870 1327 Acked Read 4 Sent Done Done false
1871 1328 Acked Write 9 Sent Done Done false
1872 1329 Acked Read 10 Sent Done Done false
1873 1330 Acked Write 5 Sent Done Done false
1874 1331 Acked Flush 0 Sent Done Done false
1875 1332 Acked Write 3 Sent Done Done false
1876 1333 Acked Write 4 Sent Done Done false
1877 1334 Acked Read 7 Sent Done Done false
1878 1335 Acked Read 3 Sent Done Done false
1879 1336 Acked Flush 0 Sent Done Done false
1880 1337 Acked Read 6 Sent Done Done false
1881 1338 Acked Read 7 Sent Done Done false
1882 1339 Acked Read 4 Sent Done Done false
1883 1340 Acked Read 6 Sent Done Done false
1884 1341 Acked Read 10 Sent Done Done false
1885 1342 Acked Flush 0 Sent Done Done false
1886 1343 Acked Write 4 Sent Done Done false
1887 1344 Acked Read 6 Sent Done Done false
1888 1345 Acked Write 7 Sent Done Done false
1889 1346 Acked Write 4 Sent Done Done false
1890 1347 Acked Read 3 Sent Done Done false
1891 1348 Acked Read 6 Sent Done Done false
1892 1349 Acked Read 8 Sent Done Done false
1893 1350 Acked Write 5 Sent Done Done false
1894 1351 Acked Write 7 Sent Done Done false
1895 1352 Acked Write 7 Sent Done Done false
1896 1353 Acked Write 3 Sent Done Done false
1897 1354 Acked Read 3 Sent Done Done false
1898 1355 Acked Read 10 Sent Done Done false
1899 1356 Acked Write 8 Sent Done Done false
1900 1357 Acked Read 1 Sent Done Done false
1901 1358 Acked Read 4 Sent Done Done false
1902 1359 Acked Write 2 Sent Done Done false
1903 1360 Acked Read 3 Sent Done Done false
1904 1361 Acked Write 6 Sent Done Done false
1905 1362 Acked Read 1 Sent Done Done false
1906 1363 Acked Read 3 Sent Done Done false
1907 1364 Acked Write 3 Sent Done Done false
1908 1365 Acked Read 4 Sent Done Done false
1909 1366 Acked Flush 0 Sent Done Done false
1910 1367 Acked Write 9 Sent Done Done false
1911 1368 Acked Write 1 Sent Done Done false
1912 1369 Acked Read 3 Sent Done Done false
1913 1370 Acked Read 2 Sent Done Done false
1914 1371 Acked Write 5 Sent Done Done false
1915 1372 Acked Write 1 Sent Done Done false
1916 1373 Acked Write 1 Sent Done Done false
1917 1374 Acked Flush 0 Sent Done Done false
1918 1375 Acked Write 6 Sent Done Done false
1919 1376 Acked Flush 0 Sent Done Done false
1920 1377 Acked Read 3 Sent Done Done false
1921 1378 Acked Flush 0 Sent Done Done false
1922 1379 Acked Flush 0 Sent Done Done false
1923 1380 Acked Write 4 Sent Done Done false
1924 1381 Acked Flush 0 Sent Done Done false
1925 1382 Acked Write 3 Sent Done Done false
1926 1383 Acked Read 5 Sent Done Done false
1927 1384 Acked Read 3 Sent Done Done false
1928 1385 Acked Read 1 Sent Done Done false
1929 1386 Acked Write 2 Sent Done Done false
1930 1387 Acked Read 3 Sent Done Done false
1931 1388 Acked Read 9 Sent Done Done false
1932 1389 Acked Write 9 Sent Done Done false
1933 1390 Acked Read 2 Sent Done Done false
1934 1391 Acked Read 8 Sent Done Done false
1935 1392 Acked Read 4 Sent Done Done false
1936 1393 Acked Read 9 Sent Done Done false
1937 1394 Acked Write 4 Sent Done Done false
1938 1395 Acked Write 3 Sent Done Done false
1939 1396 Acked Read 10 Sent Done Done false
1940 1397 Acked Read 4 Sent Done Done false
1941 1398 Acked Read 7 Sent Done Done false
1942 1399 Acked Write 7 Sent Done Done false
1943 1400 Acked Read 4 Sent Done Done false
1944 1401 Acked Write 5 Sent Done Done false
1945 1402 Acked Write 7 Sent Done Done false
1946 1403 Acked Write 2 Sent Done Done false
1947 1404 Acked Read 4 Sent Done Done false
1948 1405 Acked Write 4 Sent Done Done false
1949 1406 Acked Write 10 Sent Done Done false
1950 1407 Acked Write 9 Sent Done Done false
1951 1408 Acked Flush 0 Sent Done Done false
1952 1409 Acked Read 9 Sent Done Done false
1953 1410 Acked Write 1 Sent Done Done false
1954 1411 Acked Write 10 Sent Done Done false
1955 1412 Acked Read 4 Sent Done Done false
1956 1413 Acked Write 4 Sent Done Done false
1957 1414 Acked Flush 0 Sent Done Done false
1958 1415 Acked Write 4 Sent Done Done false
1959 1416 Acked Read 4 Sent Done Done false
1960 1417 Acked Read 9 Sent Done Done false
1961 1418 Acked Write 2 Sent Done Done false
1962 1419 Acked Read 3 Sent Done Done false
1963 1420 Acked Read 5 Sent Done Done false
1964 1421 Acked Write 1 Sent Done Done false
1965 1422 Acked Write 7 Sent Done Done false
1966 1423 Acked Read 6 Sent Done Done false
1967 1424 Acked Read 4 Sent Done Done false
1968 1425 Acked Read 5 Sent Done Done false
1969 1426 Acked Read 3 Sent Done Done false
1970 1427 Acked Write 3 Sent Done Done false
1971 1428 Acked Read 9 Sent Done Done false
1972 1429 Acked Write 6 Sent Done Done false
1973 1430 Acked Read 4 Sent Done Done false
1974 1431 Acked Flush 0 Sent Done Done false
1975 1432 Acked Write 10 Sent Done Done false
1976 1433 Acked Write 10 Sent Done Done false
1977 1434 Acked Write 5 Sent Done Done false
1978 1435 Acked Read 9 Sent Done Done false
1979 1436 Acked Read 5 Sent Done Done false
1980 1437 Acked Read 5 Sent Done Done false
1981 1438 Acked Write 3 Sent Done Done false
1982 1439 Acked Write 1 Sent Done Done false
1983 1440 Acked Read 5 Sent Done Done false
1984 1441 Acked Flush 0 Sent Done Done false
1985 1442 Acked Read 2 Sent Done Done false
1986 1443 Acked Read 6 Sent Done Done false
1987 1444 Acked Write 4 Sent Done Done false
1988 1445 Acked Read 3 Sent Done Done false
1989 1446 Acked Write 6 Sent Done Done false
1990 1447 Acked Read 1 Sent Done Done false
1991 1448 Acked Read 5 Sent Done Done false
1992 1449 Acked Write 10 Sent Done Done false
1993 1450 Acked Read 7 Sent Done Done false
1994 1451 Acked Write 2 Sent Done Done false
1995 1452 Acked Read 8 Sent Done Done false
1996 1453 Acked Read 8 Sent Done Done false
1997 1454 Acked Write 1 Sent Done Done false
1998 1455 Acked Flush 0 Sent Done Done false
1999 1456 Acked Write 8 Sent Done Done false
2000 1457 Acked Read 10 Sent Done Done false
2001 1458 Acked Read 5 Sent Done Done false
2002 1459 Acked Read 5 Sent Done Done false
2003 1460 Acked Read 7 Sent Done Done false
2004 1461 Acked Read 4 Sent Done Done false
2005 1462 Acked Write 1 Sent Done Done false
2006 1463 Acked Read 10 Sent Done Done false
2007 1464 Acked Flush 0 Sent Done Done false
2008 1465 Acked Write 6 Sent Done Done false
2009 1466 Acked Read 10 Sent Done Done false
2010 1467 Acked Write 2 Sent Done Done false
2011 1468 Acked Write 3 Sent Done Done false
2012 1469 Acked Write 4 Sent Done Done false
2013 1470 Acked Flush 0 Sent Done Done false
2014 1471 Acked Read 6 Sent Done Done false
2015 1472 Acked Write 5 Sent Done Done false
2016 1473 Acked Flush 0 Sent Done Done false
2017 1474 Acked Write 1 Sent Done Done false
2018 1475 Acked Read 1 Sent Done Done false
2019 1476 Acked Write 7 Sent Done Done false
2020 1477 Acked Flush 0 Sent Done Done false
2021 1478 Acked Read 10 Sent Done Done false
2022 1479 Acked Read 5 Sent Done Done false
2023 1480 Acked Read 10 Sent Done Done false
2024 1481 Acked Flush 0 Sent Done Done false
2025 1482 Acked Read 2 Sent Done Done false
2026 1483 Acked Read 5 Sent Done Done false
2027 1484 Acked Write 6 Sent Done Done false
2028 1485 Acked Read 9 Sent Done Done false
2029 1486 Acked Write 7 Sent Done Done false
2030 1487 Acked Read 2 Sent Done Done false
2031 1488 Acked Read 9 Sent Done Done false
2032 1489 Acked Read 10 Sent Done Done false
2033 1490 Acked Flush 0 Sent Done Done false
2034 1491 Acked Write 2 Sent Done Done false
2035 1492 Acked Flush 0 Sent Done Done false
2036 1493 Acked Write 6 Sent Done Done false
2037 1494 Acked Write 9 Sent Done Done false
2038 1495 Acked Write 1 Sent Done Done false
2039 1496 Acked Write 3 Sent Done Done false
2040 1497 Acked Write 10 Sent Done Done false
2041 1498 Acked Read 2 Sent Done Done false
2042 1499 Acked Read 8 Sent Done Done false
2043 1500 Acked Flush 0 Sent Done Done false
2044 1501 Acked Read 4 Sent Done Done false
2045 1502 Acked Read 9 Sent Done Done false
2046 1503 Acked Read 4 Sent Done Done false
2047 1504 Acked Read 9 Sent Done Done false
2048 1505 Acked Read 5 Sent Done Done false
2049 1506 Acked Write 10 Sent Done Done false
2050 1507 Acked Write 2 Sent Done Done false
2051 1508 Acked Write 4 Sent Done Done false
2052 1509 Acked Write 10 Sent Done Done false
2053 1510 Acked Write 4 Sent Done Done false
2054 1511 Acked Read 6 Sent Done Done false
2055 1512 Acked Read 8 Sent Done Done false
2056 1513 Acked Read 8 Sent Done Done false
2057 1514 Acked Read 2 Sent Done Done false
2058 1515 Acked Write 9 Sent Done Done false
2059 1516 Acked Read 10 Sent Done Done false
2060 1517 Acked Write 2 Sent Done Done false
2061 1518 Acked Read 10 Sent Done Done false
2062 1519 Acked Read 6 Sent Done Done false
2063 1520 Acked Read 9 Sent Done Done false
2064 1521 Acked Read 4 Sent Done Done false
2065 1522 Acked Read 10 Sent Done Done false
2066 1523 Acked Read 9 Sent Done Done false
2067 1524 Acked Write 10 Sent Done Done false
2068 1525 Acked Write 8 Sent Done Done false
2069 1526 Acked Read 10 Sent Done Done false
2070 1527 Acked Read 7 Sent Done Done false
2071 1528 Acked Write 7 Sent Done Done false
2072 1529 Acked Flush 0 Sent Done Done false
2073 1530 Acked Write 10 Sent Done Done false
2074 1531 Acked Write 7 Sent Done Done false
2075 1532 Acked Read 10 Sent Done Done false
2076 1533 Acked Read 4 Sent Done Done false
2077 1534 Acked Flush 0 Sent Done Done false
2078 1535 Acked Flush 0 Sent Done Done false
2079 1536 Acked Read 2 Sent Done Done false
2080 1537 Acked Read 10 Sent Done Done false
2081 1538 Acked Read 6 Sent Done Done false
2082 1539 Acked Read 4 Sent Done Done false
2083 1540 Acked Write 8 Sent Done Done false
2084 1541 Acked Read 6 Sent Done Done false
2085 1542 Acked Read 9 Sent Done Done false
2086 1543 Acked Read 9 Sent Done Done false
2087 1544 Acked Read 2 Sent Done Done false
2088 1545 Acked Read 1 Sent Done Done false
2089 1546 Acked Read 7 Sent Done Done false
2090 1547 Acked Flush 0 Sent Done Done false
2091 1548 Acked Write 8 Sent Done Done false
2092 1549 Acked Flush 0 Sent Done Done false
2093 1550 Acked Write 3 Sent Done Done false
2094 1551 Acked Read 4 Sent Done Done false
2095 1552 Acked Flush 0 Sent Done Done false
2096 1553 Acked Read 7 Sent Done Done false
2097 1554 Acked Write 6 Sent Done Done false
2098 1555 Acked Read 2 Sent Done Done false
2099 1556 Acked Write 1 Sent Done Done false
2100 1557 Acked Write 3 Sent Done Done false
2101 1558 Acked Write 7 Sent Done Done false
2102 1559 Acked Read 6 Sent Done Done false
2103 1560 Acked Flush 0 Sent Done Done false
2104 1561 Acked Write 8 Sent Done Done false
2105 1562 Acked Read 4 Sent Done Done false
2106 1563 Acked Write 5 Sent Done Done false
2107 1564 Acked Read 9 Sent Done Done false
2108 1565 Acked Flush 0 Sent Done Done false
2109 1566 Acked Write 7 Sent Done Done false
2110 1567 Acked Read 1 Sent Done Done false
2111 1568 Acked Read 6 Sent Done Done false
2112 1569 Acked Read 6 Sent Done Done false
2113 1570 Acked Read 5 Sent Done Done false
2114 1571 Acked Read 6 Sent Done Done false
2115 1572 Acked Read 5 Sent Done Done false
2116 1573 Acked Write 7 Sent Done Done false
2117 1574 Acked Read 8 Sent Done Done false
2118 1575 Acked Flush 0 Sent Done Done false
2119 1576 Acked Read 7 Sent Done Done false
2120 1577 Acked Write 1 Sent Done Done false
2121 1578 Acked Write 5 Sent Done Done false
2122 1579 Acked Write 8 Sent Done Done false
2123 1580 Acked Read 7 Sent Done Done false
2124 1581 Acked Read 3 Sent Done Done false
2125 1582 Acked Write 7 Sent Done Done false
2126 1583 Acked Write 6 Sent Done Done false
2127 1584 Acked Read 10 Sent Done Done false
2128 1585 Acked Read 8 Sent Done Done false
2129 1586 Acked Write 1 Sent Done Done false
2130 1587 Acked Write 9 Sent Done Done false
2131 1588 Acked Write 8 Sent Done Done false
2132 1589 Acked Write 3 Sent Done Done false
2133 1590 Acked Read 8 Sent Done Done false
2134 1591 Acked Read 4 Sent Done Done false
2135 1592 Acked Write 3 Sent Done Done false
2136 1593 Acked Read 2 Sent Done Done false
2137 1594 Acked Write 1 Sent Done Done false
2138 1595 Acked Read 7 Sent Done Done false
2139 1596 Acked Write 6 Sent Done Done false
2140 1597 Acked Read 5 Sent Done Done false
2141 1598 Acked Read 3 Sent Done Done false
2142 1599 Acked Read 3 Sent Done Done false
2143 1600 Acked Flush 0 Sent Done Done false
2144 1601 Acked Read 6 Sent Done Done false
2145 1602 Acked Write 6 Sent Done Done false
2146 1603 Acked Flush 0 Sent Done Done false
2147 STATES DS:0 DS:1 DS:2 TOTAL
2148 Sent 302 0 0 302
2149 Done 0 302 302 604
2150 Skipped 0 0 0 0
2151 Error 0 0 0 0
2152 Last Flush: 1301 1603 1603
2153 Downstairs last five completed: 1301 1300 1299 1298 1297
2154 Upstairs last five completed: 1603 1602 1601 1600 1599
2155 CLIENT: Up:0 ds:302 act:2
21562025-01-11T06:05:52.713ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
21572025-01-11T06:05:52.713ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
21582025-01-11T06:05:52.713ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
21592025-01-11T06:05:52.713ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
21602025-01-11T06:05:52.713ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2161 [8810] got start action
2162 [8810] I am going to restart
2163 Starting downstairs at port 8810
2164 Make output file at "/tmp/dsc/downstairs-8810.txt"
2165 [8810][0] reports Starting
2166 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1206)
2167 [8810][0] reports Running
2168 ----------------------------------------------------------------
2169 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2170 States: Offline Active Active
2171 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2172 1302 Acked Write 5 Sent Done Done false
2173 1303 Acked Read 7 Sent Done Done false
2174 1304 Acked Read 2 Sent Done Done false
2175 1305 Acked Write 1 Sent Done Done false
2176 1306 Acked Flush 0 Sent Done Done false
2177 1307 Acked Write 7 Sent Done Done false
2178 1308 Acked Read 2 Sent Done Done false
2179 1309 Acked Read 3 Sent Done Done false
2180 1310 Acked Write 4 Sent Done Done false
2181 1311 Acked Write 9 Sent Done Done false
2182 1312 Acked Write 10 Sent Done Done false
2183 1313 Acked Read 7 Sent Done Done false
2184 1314 Acked Read 2 Sent Done Done false
2185 1315 Acked Read 1 Sent Done Done false
2186 1316 Acked Read 3 Sent Done Done false
2187 1317 Acked Read 9 Sent Done Done false
2188 1318 Acked Flush 0 Sent Done Done false
2189 1319 Acked Write 8 Sent Done Done false
2190 1320 Acked Read 8 Sent Done Done false
2191 1321 Acked Write 2 Sent Done Done false
2192 1322 Acked Read 7 Sent Done Done false
2193 1323 Acked Read 7 Sent Done Done false
2194 1324 Acked Read 7 Sent Done Done false
2195 1325 Acked Read 8 Sent Done Done false
2196 1326 Acked Write 8 Sent Done Done false
2197 1327 Acked Read 4 Sent Done Done false
2198 1328 Acked Write 9 Sent Done Done false
2199 1329 Acked Read 10 Sent Done Done false
2200 1330 Acked Write 5 Sent Done Done false
2201 1331 Acked Flush 0 Sent Done Done false
2202 1332 Acked Write 3 Sent Done Done false
2203 1333 Acked Write 4 Sent Done Done false
2204 1334 Acked Read 7 Sent Done Done false
2205 1335 Acked Read 3 Sent Done Done false
2206 1336 Acked Flush 0 Sent Done Done false
2207 1337 Acked Read 6 Sent Done Done false
2208 1338 Acked Read 7 Sent Done Done false
2209 1339 Acked Read 4 Sent Done Done false
2210 1340 Acked Read 6 Sent Done Done false
2211 1341 Acked Read 10 Sent Done Done false
2212 1342 Acked Flush 0 Sent Done Done false
2213 1343 Acked Write 4 Sent Done Done false
2214 1344 Acked Read 6 Sent Done Done false
2215 1345 Acked Write 7 Sent Done Done false
2216 1346 Acked Write 4 Sent Done Done false
2217 1347 Acked Read 3 Sent Done Done false
2218 1348 Acked Read 6 Sent Done Done false
2219 1349 Acked Read 8 Sent Done Done false
2220 1350 Acked Write 5 Sent Done Done false
2221 1351 Acked Write 7 Sent Done Done false
2222 1352 Acked Write 7 Sent Done Done false
2223 1353 Acked Write 3 Sent Done Done false
2224 1354 Acked Read 3 Sent Done Done false
2225 1355 Acked Read 10 Sent Done Done false
2226 1356 Acked Write 8 Sent Done Done false
2227 1357 Acked Read 1 Sent Done Done false
2228 1358 Acked Read 4 Sent Done Done false
2229 1359 Acked Write 2 Sent Done Done false
2230 1360 Acked Read 3 Sent Done Done false
2231 1361 Acked Write 6 Sent Done Done false
2232 1362 Acked Read 1 Sent Done Done false
2233 1363 Acked Read 3 Sent Done Done false
2234 1364 Acked Write 3 Sent Done Done false
2235 1365 Acked Read 4 Sent Done Done false
2236 1366 Acked Flush 0 Sent Done Done false
2237 1367 Acked Write 9 Sent Done Done false
2238 1368 Acked Write 1 Sent Done Done false
2239 1369 Acked Read 3 Sent Done Done false
2240 1370 Acked Read 2 Sent Done Done false
2241 1371 Acked Write 5 Sent Done Done false
2242 1372 Acked Write 1 Sent Done Done false
2243 1373 Acked Write 1 Sent Done Done false
2244 1374 Acked Flush 0 Sent Done Done false
2245 1375 Acked Write 6 Sent Done Done false
2246 1376 Acked Flush 0 Sent Done Done false
2247 1377 Acked Read 3 Sent Done Done false
2248 1378 Acked Flush 0 Sent Done Done false
2249 1379 Acked Flush 0 Sent Done Done false
2250 1380 Acked Write 4 Sent Done Done false
2251 1381 Acked Flush 0 Sent Done Done false
2252 1382 Acked Write 3 Sent Done Done false
2253 1383 Acked Read 5 Sent Done Done false
2254 1384 Acked Read 3 Sent Done Done false
2255 1385 Acked Read 1 Sent Done Done false
2256 1386 Acked Write 2 Sent Done Done false
2257 1387 Acked Read 3 Sent Done Done false
2258 1388 Acked Read 9 Sent Done Done false
2259 1389 Acked Write 9 Sent Done Done false
2260 1390 Acked Read 2 Sent Done Done false
2261 1391 Acked Read 8 Sent Done Done false
2262 1392 Acked Read 4 Sent Done Done false
2263 1393 Acked Read 9 Sent Done Done false
2264 1394 Acked Write 4 Sent Done Done false
2265 1395 Acked Write 3 Sent Done Done false
2266 1396 Acked Read 10 Sent Done Done false
2267 1397 Acked Read 4 Sent Done Done false
2268 1398 Acked Read 7 Sent Done Done false
2269 1399 Acked Write 7 Sent Done Done false
2270 1400 Acked Read 4 Sent Done Done false
2271 1401 Acked Write 5 Sent Done Done false
2272 1402 Acked Write 7 Sent Done Done false
2273 1403 Acked Write 2 Sent Done Done false
2274 1404 Acked Read 4 Sent Done Done false
2275 1405 Acked Write 4 Sent Done Done false
2276 1406 Acked Write 10 Sent Done Done false
2277 1407 Acked Write 9 Sent Done Done false
2278 1408 Acked Flush 0 Sent Done Done false
2279 1409 Acked Read 9 Sent Done Done false
2280 1410 Acked Write 1 Sent Done Done false
2281 1411 Acked Write 10 Sent Done Done false
2282 1412 Acked Read 4 Sent Done Done false
2283 1413 Acked Write 4 Sent Done Done false
2284 1414 Acked Flush 0 Sent Done Done false
2285 1415 Acked Write 4 Sent Done Done false
2286 1416 Acked Read 4 Sent Done Done false
2287 1417 Acked Read 9 Sent Done Done false
2288 1418 Acked Write 2 Sent Done Done false
2289 1419 Acked Read 3 Sent Done Done false
2290 1420 Acked Read 5 Sent Done Done false
2291 1421 Acked Write 1 Sent Done Done false
2292 1422 Acked Write 7 Sent Done Done false
2293 1423 Acked Read 6 Sent Done Done false
2294 1424 Acked Read 4 Sent Done Done false
2295 1425 Acked Read 5 Sent Done Done false
2296 1426 Acked Read 3 Sent Done Done false
2297 1427 Acked Write 3 Sent Done Done false
2298 1428 Acked Read 9 Sent Done Done false
2299 1429 Acked Write 6 Sent Done Done false
2300 1430 Acked Read 4 Sent Done Done false
2301 1431 Acked Flush 0 Sent Done Done false
2302 1432 Acked Write 10 Sent Done Done false
2303 1433 Acked Write 10 Sent Done Done false
2304 1434 Acked Write 5 Sent Done Done false
2305 1435 Acked Read 9 Sent Done Done false
2306 1436 Acked Read 5 Sent Done Done false
2307 1437 Acked Read 5 Sent Done Done false
2308 1438 Acked Write 3 Sent Done Done false
2309 1439 Acked Write 1 Sent Done Done false
2310 1440 Acked Read 5 Sent Done Done false
2311 1441 Acked Flush 0 Sent Done Done false
2312 1442 Acked Read 2 Sent Done Done false
2313 1443 Acked Read 6 Sent Done Done false
2314 1444 Acked Write 4 Sent Done Done false
2315 1445 Acked Read 3 Sent Done Done false
2316 1446 Acked Write 6 Sent Done Done false
2317 1447 Acked Read 1 Sent Done Done false
2318 1448 Acked Read 5 Sent Done Done false
2319 1449 Acked Write 10 Sent Done Done false
2320 1450 Acked Read 7 Sent Done Done false
2321 1451 Acked Write 2 Sent Done Done false
2322 1452 Acked Read 8 Sent Done Done false
2323 1453 Acked Read 8 Sent Done Done false
2324 1454 Acked Write 1 Sent Done Done false
2325 1455 Acked Flush 0 Sent Done Done false
2326 1456 Acked Write 8 Sent Done Done false
2327 1457 Acked Read 10 Sent Done Done false
2328 1458 Acked Read 5 Sent Done Done false
2329 1459 Acked Read 5 Sent Done Done false
2330 1460 Acked Read 7 Sent Done Done false
2331 1461 Acked Read 4 Sent Done Done false
2332 1462 Acked Write 1 Sent Done Done false
2333 1463 Acked Read 10 Sent Done Done false
2334 1464 Acked Flush 0 Sent Done Done false
2335 1465 Acked Write 6 Sent Done Done false
2336 1466 Acked Read 10 Sent Done Done false
2337 1467 Acked Write 2 Sent Done Done false
2338 1468 Acked Write 3 Sent Done Done false
2339 1469 Acked Write 4 Sent Done Done false
2340 1470 Acked Flush 0 Sent Done Done false
2341 1471 Acked Read 6 Sent Done Done false
2342 1472 Acked Write 5 Sent Done Done false
2343 1473 Acked Flush 0 Sent Done Done false
2344 1474 Acked Write 1 Sent Done Done false
2345 1475 Acked Read 1 Sent Done Done false
2346 1476 Acked Write 7 Sent Done Done false
2347 1477 Acked Flush 0 Sent Done Done false
2348 1478 Acked Read 10 Sent Done Done false
2349 1479 Acked Read 5 Sent Done Done false
2350 1480 Acked Read 10 Sent Done Done false
2351 1481 Acked Flush 0 Sent Done Done false
2352 1482 Acked Read 2 Sent Done Done false
2353 1483 Acked Read 5 Sent Done Done false
2354 1484 Acked Write 6 Sent Done Done false
2355 1485 Acked Read 9 Sent Done Done false
2356 1486 Acked Write 7 Sent Done Done false
2357 1487 Acked Read 2 Sent Done Done false
2358 1488 Acked Read 9 Sent Done Done false
2359 1489 Acked Read 10 Sent Done Done false
2360 1490 Acked Flush 0 Sent Done Done false
2361 1491 Acked Write 2 Sent Done Done false
2362 1492 Acked Flush 0 Sent Done Done false
2363 1493 Acked Write 6 Sent Done Done false
2364 1494 Acked Write 9 Sent Done Done false
2365 1495 Acked Write 1 Sent Done Done false
2366 1496 Acked Write 3 Sent Done Done false
2367 1497 Acked Write 10 Sent Done Done false
2368 1498 Acked Read 2 Sent Done Done false
2369 1499 Acked Read 8 Sent Done Done false
2370 1500 Acked Flush 0 Sent Done Done false
2371 1501 Acked Read 4 Sent Done Done false
2372 1502 Acked Read 9 Sent Done Done false
2373 1503 Acked Read 4 Sent Done Done false
2374 1504 Acked Read 9 Sent Done Done false
2375 1505 Acked Read 5 Sent Done Done false
2376 1506 Acked Write 10 Sent Done Done false
2377 1507 Acked Write 2 Sent Done Done false
2378 1508 Acked Write 4 Sent Done Done false
2379 1509 Acked Write 10 Sent Done Done false
2380 1510 Acked Write 4 Sent Done Done false
2381 1511 Acked Read 6 Sent Done Done false
2382 1512 Acked Read 8 Sent Done Done false
2383 1513 Acked Read 8 Sent Done Done false
2384 1514 Acked Read 2 Sent Done Done false
2385 1515 Acked Write 9 Sent Done Done false
2386 1516 Acked Read 10 Sent Done Done false
2387 1517 Acked Write 2 Sent Done Done false
2388 1518 Acked Read 10 Sent Done Done false
2389 1519 Acked Read 6 Sent Done Done false
2390 1520 Acked Read 9 Sent Done Done false
2391 1521 Acked Read 4 Sent Done Done false
2392 1522 Acked Read 10 Sent Done Done false
2393 1523 Acked Read 9 Sent Done Done false
2394 1524 Acked Write 10 Sent Done Done false
2395 1525 Acked Write 8 Sent Done Done false
2396 1526 Acked Read 10 Sent Done Done false
2397 1527 Acked Read 7 Sent Done Done false
2398 1528 Acked Write 7 Sent Done Done false
2399 1529 Acked Flush 0 Sent Done Done false
2400 1530 Acked Write 10 Sent Done Done false
2401 1531 Acked Write 7 Sent Done Done false
2402 1532 Acked Read 10 Sent Done Done false
2403 1533 Acked Read 4 Sent Done Done false
2404 1534 Acked Flush 0 Sent Done Done false
2405 1535 Acked Flush 0 Sent Done Done false
2406 1536 Acked Read 2 Sent Done Done false
2407 1537 Acked Read 10 Sent Done Done false
2408 1538 Acked Read 6 Sent Done Done false
2409 1539 Acked Read 4 Sent Done Done false
2410 1540 Acked Write 8 Sent Done Done false
2411 1541 Acked Read 6 Sent Done Done false
2412 1542 Acked Read 9 Sent Done Done false
2413 1543 Acked Read 9 Sent Done Done false
2414 1544 Acked Read 2 Sent Done Done false
2415 1545 Acked Read 1 Sent Done Done false
2416 1546 Acked Read 7 Sent Done Done false
2417 1547 Acked Flush 0 Sent Done Done false
2418 1548 Acked Write 8 Sent Done Done false
2419 1549 Acked Flush 0 Sent Done Done false
2420 1550 Acked Write 3 Sent Done Done false
2421 1551 Acked Read 4 Sent Done Done false
2422 1552 Acked Flush 0 Sent Done Done false
2423 1553 Acked Read 7 Sent Done Done false
2424 1554 Acked Write 6 Sent Done Done false
2425 1555 Acked Read 2 Sent Done Done false
2426 1556 Acked Write 1 Sent Done Done false
2427 1557 Acked Write 3 Sent Done Done false
2428 1558 Acked Write 7 Sent Done Done false
2429 1559 Acked Read 6 Sent Done Done false
2430 1560 Acked Flush 0 Sent Done Done false
2431 1561 Acked Write 8 Sent Done Done false
2432 1562 Acked Read 4 Sent Done Done false
2433 1563 Acked Write 5 Sent Done Done false
2434 1564 Acked Read 9 Sent Done Done false
2435 1565 Acked Flush 0 Sent Done Done false
2436 1566 Acked Write 7 Sent Done Done false
2437 1567 Acked Read 1 Sent Done Done false
2438 1568 Acked Read 6 Sent Done Done false
2439 1569 Acked Read 6 Sent Done Done false
2440 1570 Acked Read 5 Sent Done Done false
2441 1571 Acked Read 6 Sent Done Done false
2442 1572 Acked Read 5 Sent Done Done false
2443 1573 Acked Write 7 Sent Done Done false
2444 1574 Acked Read 8 Sent Done Done false
2445 1575 Acked Flush 0 Sent Done Done false
2446 1576 Acked Read 7 Sent Done Done false
2447 1577 Acked Write 1 Sent Done Done false
2448 1578 Acked Write 5 Sent Done Done false
2449 1579 Acked Write 8 Sent Done Done false
2450 1580 Acked Read 7 Sent Done Done false
2451 1581 Acked Read 3 Sent Done Done false
2452 1582 Acked Write 7 Sent Done Done false
2453 1583 Acked Write 6 Sent Done Done false
2454 1584 Acked Read 10 Sent Done Done false
2455 1585 Acked Read 8 Sent Done Done false
2456 1586 Acked Write 1 Sent Done Done false
2457 1587 Acked Write 9 Sent Done Done false
2458 1588 Acked Write 8 Sent Done Done false
2459 1589 Acked Write 3 Sent Done Done false
2460 1590 Acked Read 8 Sent Done Done false
2461 1591 Acked Read 4 Sent Done Done false
2462 1592 Acked Write 3 Sent Done Done false
2463 1593 Acked Read 2 Sent Done Done false
2464 1594 Acked Write 1 Sent Done Done false
2465 1595 Acked Read 7 Sent Done Done false
2466 1596 Acked Write 6 Sent Done Done false
2467 1597 Acked Read 5 Sent Done Done false
2468 1598 Acked Read 3 Sent Done Done false
2469 1599 Acked Read 3 Sent Done Done false
2470 1600 Acked Flush 0 Sent Done Done false
2471 1601 Acked Read 6 Sent Done Done false
2472 1602 Acked Write 6 Sent Done Done false
2473 1603 Acked Flush 0 Sent Done Done false
2474 STATES DS:0 DS:1 DS:2 TOTAL
2475 Sent 302 0 0 302
2476 Done 0 302 302 604
2477 Skipped 0 0 0 0
2478 Error 0 0 0 0
2479 Last Flush: 1301 1603 1603
2480 Downstairs last five completed: 1301 1300 1299 1298 1297
2481 Upstairs last five completed: 1603 1602 1601 1600 1599
2482 CLIENT: Up:0 ds:302 act:2
2483 ----------------------------------------------------------------
2484 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2485 States: Offline Active Active
2486 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2487 1302 Acked Write 5 Sent Done Done false
2488 1303 Acked Read 7 Sent Done Done false
2489 1304 Acked Read 2 Sent Done Done false
2490 1305 Acked Write 1 Sent Done Done false
2491 1306 Acked Flush 0 Sent Done Done false
2492 1307 Acked Write 7 Sent Done Done false
2493 1308 Acked Read 2 Sent Done Done false
2494 1309 Acked Read 3 Sent Done Done false
2495 1310 Acked Write 4 Sent Done Done false
2496 1311 Acked Write 9 Sent Done Done false
2497 1312 Acked Write 10 Sent Done Done false
2498 1313 Acked Read 7 Sent Done Done false
2499 1314 Acked Read 2 Sent Done Done false
2500 1315 Acked Read 1 Sent Done Done false
2501 1316 Acked Read 3 Sent Done Done false
2502 1317 Acked Read 9 Sent Done Done false
2503 1318 Acked Flush 0 Sent Done Done false
2504 1319 Acked Write 8 Sent Done Done false
2505 1320 Acked Read 8 Sent Done Done false
2506 1321 Acked Write 2 Sent Done Done false
2507 1322 Acked Read 7 Sent Done Done false
2508 1323 Acked Read 7 Sent Done Done false
2509 1324 Acked Read 7 Sent Done Done false
2510 1325 Acked Read 8 Sent Done Done false
2511 1326 Acked Write 8 Sent Done Done false
2512 1327 Acked Read 4 Sent Done Done false
2513 1328 Acked Write 9 Sent Done Done false
2514 1329 Acked Read 10 Sent Done Done false
2515 1330 Acked Write 5 Sent Done Done false
2516 1331 Acked Flush 0 Sent Done Done false
2517 1332 Acked Write 3 Sent Done Done false
2518 1333 Acked Write 4 Sent Done Done false
2519 1334 Acked Read 7 Sent Done Done false
2520 1335 Acked Read 3 Sent Done Done false
2521 1336 Acked Flush 0 Sent Done Done false
2522 1337 Acked Read 6 Sent Done Done false
2523 1338 Acked Read 7 Sent Done Done false
2524 1339 Acked Read 4 Sent Done Done false
2525 1340 Acked Read 6 Sent Done Done false
2526 1341 Acked Read 10 Sent Done Done false
2527 1342 Acked Flush 0 Sent Done Done false
2528 1343 Acked Write 4 Sent Done Done false
2529 1344 Acked Read 6 Sent Done Done false
2530 1345 Acked Write 7 Sent Done Done false
2531 1346 Acked Write 4 Sent Done Done false
2532 1347 Acked Read 3 Sent Done Done false
2533 1348 Acked Read 6 Sent Done Done false
2534 1349 Acked Read 8 Sent Done Done false
2535 1350 Acked Write 5 Sent Done Done false
2536 1351 Acked Write 7 Sent Done Done false
2537 1352 Acked Write 7 Sent Done Done false
2538 1353 Acked Write 3 Sent Done Done false
2539 1354 Acked Read 3 Sent Done Done false
2540 1355 Acked Read 10 Sent Done Done false
2541 1356 Acked Write 8 Sent Done Done false
2542 1357 Acked Read 1 Sent Done Done false
2543 1358 Acked Read 4 Sent Done Done false
2544 1359 Acked Write 2 Sent Done Done false
2545 1360 Acked Read 3 Sent Done Done false
2546 1361 Acked Write 6 Sent Done Done false
2547 1362 Acked Read 1 Sent Done Done false
2548 1363 Acked Read 3 Sent Done Done false
2549 1364 Acked Write 3 Sent Done Done false
2550 1365 Acked Read 4 Sent Done Done false
2551 1366 Acked Flush 0 Sent Done Done false
2552 1367 Acked Write 9 Sent Done Done false
2553 1368 Acked Write 1 Sent Done Done false
2554 1369 Acked Read 3 Sent Done Done false
2555 1370 Acked Read 2 Sent Done Done false
2556 1371 Acked Write 5 Sent Done Done false
2557 1372 Acked Write 1 Sent Done Done false
2558 1373 Acked Write 1 Sent Done Done false
2559 1374 Acked Flush 0 Sent Done Done false
2560 1375 Acked Write 6 Sent Done Done false
2561 1376 Acked Flush 0 Sent Done Done false
2562 1377 Acked Read 3 Sent Done Done false
2563 1378 Acked Flush 0 Sent Done Done false
2564 1379 Acked Flush 0 Sent Done Done false
2565 1380 Acked Write 4 Sent Done Done false
2566 1381 Acked Flush 0 Sent Done Done false
2567 1382 Acked Write 3 Sent Done Done false
2568 1383 Acked Read 5 Sent Done Done false
2569 1384 Acked Read 3 Sent Done Done false
2570 1385 Acked Read 1 Sent Done Done false
2571 1386 Acked Write 2 Sent Done Done false
2572 1387 Acked Read 3 Sent Done Done false
2573 1388 Acked Read 9 Sent Done Done false
2574 1389 Acked Write 9 Sent Done Done false
2575 1390 Acked Read 2 Sent Done Done false
2576 1391 Acked Read 8 Sent Done Done false
2577 1392 Acked Read 4 Sent Done Done false
2578 1393 Acked Read 9 Sent Done Done false
2579 1394 Acked Write 4 Sent Done Done false
2580 1395 Acked Write 3 Sent Done Done false
2581 1396 Acked Read 10 Sent Done Done false
2582 1397 Acked Read 4 Sent Done Done false
2583 1398 Acked Read 7 Sent Done Done false
2584 1399 Acked Write 7 Sent Done Done false
2585 1400 Acked Read 4 Sent Done Done false
2586 1401 Acked Write 5 Sent Done Done false
2587 1402 Acked Write 7 Sent Done Done false
2588 1403 Acked Write 2 Sent Done Done false
2589 1404 Acked Read 4 Sent Done Done false
2590 1405 Acked Write 4 Sent Done Done false
2591 1406 Acked Write 10 Sent Done Done false
2592 1407 Acked Write 9 Sent Done Done false
2593 1408 Acked Flush 0 Sent Done Done false
2594 1409 Acked Read 9 Sent Done Done false
2595 1410 Acked Write 1 Sent Done Done false
2596 1411 Acked Write 10 Sent Done Done false
2597 1412 Acked Read 4 Sent Done Done false
2598 1413 Acked Write 4 Sent Done Done false
2599 1414 Acked Flush 0 Sent Done Done false
2600 1415 Acked Write 4 Sent Done Done false
2601 1416 Acked Read 4 Sent Done Done false
2602 1417 Acked Read 9 Sent Done Done false
2603 1418 Acked Write 2 Sent Done Done false
2604 1419 Acked Read 3 Sent Done Done false
2605 1420 Acked Read 5 Sent Done Done false
2606 1421 Acked Write 1 Sent Done Done false
2607 1422 Acked Write 7 Sent Done Done false
2608 1423 Acked Read 6 Sent Done Done false
2609 1424 Acked Read 4 Sent Done Done false
2610 1425 Acked Read 5 Sent Done Done false
2611 1426 Acked Read 3 Sent Done Done false
2612 1427 Acked Write 3 Sent Done Done false
2613 1428 Acked Read 9 Sent Done Done false
2614 1429 Acked Write 6 Sent Done Done false
2615 1430 Acked Read 4 Sent Done Done false
2616 1431 Acked Flush 0 Sent Done Done false
2617 1432 Acked Write 10 Sent Done Done false
2618 1433 Acked Write 10 Sent Done Done false
2619 1434 Acked Write 5 Sent Done Done false
2620 1435 Acked Read 9 Sent Done Done false
2621 1436 Acked Read 5 Sent Done Done false
2622 1437 Acked Read 5 Sent Done Done false
2623 1438 Acked Write 3 Sent Done Done false
2624 1439 Acked Write 1 Sent Done Done false
2625 1440 Acked Read 5 Sent Done Done false
2626 1441 Acked Flush 0 Sent Done Done false
2627 1442 Acked Read 2 Sent Done Done false
2628 1443 Acked Read 6 Sent Done Done false
2629 1444 Acked Write 4 Sent Done Done false
2630 1445 Acked Read 3 Sent Done Done false
2631 1446 Acked Write 6 Sent Done Done false
2632 1447 Acked Read 1 Sent Done Done false
2633 1448 Acked Read 5 Sent Done Done false
2634 1449 Acked Write 10 Sent Done Done false
2635 1450 Acked Read 7 Sent Done Done false
2636 1451 Acked Write 2 Sent Done Done false
2637 1452 Acked Read 8 Sent Done Done false
2638 1453 Acked Read 8 Sent Done Done false
2639 1454 Acked Write 1 Sent Done Done false
2640 1455 Acked Flush 0 Sent Done Done false
2641 1456 Acked Write 8 Sent Done Done false
2642 1457 Acked Read 10 Sent Done Done false
2643 1458 Acked Read 5 Sent Done Done false
2644 1459 Acked Read 5 Sent Done Done false
2645 1460 Acked Read 7 Sent Done Done false
2646 1461 Acked Read 4 Sent Done Done false
2647 1462 Acked Write 1 Sent Done Done false
2648 1463 Acked Read 10 Sent Done Done false
2649 1464 Acked Flush 0 Sent Done Done false
2650 1465 Acked Write 6 Sent Done Done false
2651 1466 Acked Read 10 Sent Done Done false
2652 1467 Acked Write 2 Sent Done Done false
2653 1468 Acked Write 3 Sent Done Done false
2654 1469 Acked Write 4 Sent Done Done false
2655 1470 Acked Flush 0 Sent Done Done false
2656 1471 Acked Read 6 Sent Done Done false
2657 1472 Acked Write 5 Sent Done Done false
2658 1473 Acked Flush 0 Sent Done Done false
2659 1474 Acked Write 1 Sent Done Done false
2660 1475 Acked Read 1 Sent Done Done false
2661 1476 Acked Write 7 Sent Done Done false
2662 1477 Acked Flush 0 Sent Done Done false
2663 1478 Acked Read 10 Sent Done Done false
2664 1479 Acked Read 5 Sent Done Done false
2665 1480 Acked Read 10 Sent Done Done false
2666 1481 Acked Flush 0 Sent Done Done false
2667 1482 Acked Read 2 Sent Done Done false
2668 1483 Acked Read 5 Sent Done Done false
2669 1484 Acked Write 6 Sent Done Done false
2670 1485 Acked Read 9 Sent Done Done false
2671 1486 Acked Write 7 Sent Done Done false
2672 1487 Acked Read 2 Sent Done Done false
2673 1488 Acked Read 9 Sent Done Done false
2674 1489 Acked Read 10 Sent Done Done false
2675 1490 Acked Flush 0 Sent Done Done false
2676 1491 Acked Write 2 Sent Done Done false
2677 1492 Acked Flush 0 Sent Done Done false
2678 1493 Acked Write 6 Sent Done Done false
2679 1494 Acked Write 9 Sent Done Done false
2680 1495 Acked Write 1 Sent Done Done false
2681 1496 Acked Write 3 Sent Done Done false
2682 1497 Acked Write 10 Sent Done Done false
2683 1498 Acked Read 2 Sent Done Done false
2684 1499 Acked Read 8 Sent Done Done false
2685 1500 Acked Flush 0 Sent Done Done false
2686 1501 Acked Read 4 Sent Done Done false
2687 1502 Acked Read 9 Sent Done Done false
2688 1503 Acked Read 4 Sent Done Done false
2689 1504 Acked Read 9 Sent Done Done false
2690 1505 Acked Read 5 Sent Done Done false
2691 1506 Acked Write 10 Sent Done Done false
2692 1507 Acked Write 2 Sent Done Done false
2693 1508 Acked Write 4 Sent Done Done false
2694 1509 Acked Write 10 Sent Done Done false
2695 1510 Acked Write 4 Sent Done Done false
2696 1511 Acked Read 6 Sent Done Done false
2697 1512 Acked Read 8 Sent Done Done false
2698 1513 Acked Read 8 Sent Done Done false
2699 1514 Acked Read 2 Sent Done Done false
2700 1515 Acked Write 9 Sent Done Done false
2701 1516 Acked Read 10 Sent Done Done false
2702 1517 Acked Write 2 Sent Done Done false
2703 1518 Acked Read 10 Sent Done Done false
2704 1519 Acked Read 6 Sent Done Done false
2705 1520 Acked Read 9 Sent Done Done false
2706 1521 Acked Read 4 Sent Done Done false
2707 1522 Acked Read 10 Sent Done Done false
2708 1523 Acked Read 9 Sent Done Done false
2709 1524 Acked Write 10 Sent Done Done false
2710 1525 Acked Write 8 Sent Done Done false
2711 1526 Acked Read 10 Sent Done Done false
2712 1527 Acked Read 7 Sent Done Done false
2713 1528 Acked Write 7 Sent Done Done false
2714 1529 Acked Flush 0 Sent Done Done false
2715 1530 Acked Write 10 Sent Done Done false
2716 1531 Acked Write 7 Sent Done Done false
2717 1532 Acked Read 10 Sent Done Done false
2718 1533 Acked Read 4 Sent Done Done false
2719 1534 Acked Flush 0 Sent Done Done false
2720 1535 Acked Flush 0 Sent Done Done false
2721 1536 Acked Read 2 Sent Done Done false
2722 1537 Acked Read 10 Sent Done Done false
2723 1538 Acked Read 6 Sent Done Done false
2724 1539 Acked Read 4 Sent Done Done false
2725 1540 Acked Write 8 Sent Done Done false
2726 1541 Acked Read 6 Sent Done Done false
2727 1542 Acked Read 9 Sent Done Done false
2728 1543 Acked Read 9 Sent Done Done false
2729 1544 Acked Read 2 Sent Done Done false
2730 1545 Acked Read 1 Sent Done Done false
2731 1546 Acked Read 7 Sent Done Done false
2732 1547 Acked Flush 0 Sent Done Done false
2733 1548 Acked Write 8 Sent Done Done false
2734 1549 Acked Flush 0 Sent Done Done false
2735 1550 Acked Write 3 Sent Done Done false
2736 1551 Acked Read 4 Sent Done Done false
2737 1552 Acked Flush 0 Sent Done Done false
2738 1553 Acked Read 7 Sent Done Done false
2739 1554 Acked Write 6 Sent Done Done false
2740 1555 Acked Read 2 Sent Done Done false
2741 1556 Acked Write 1 Sent Done Done false
2742 1557 Acked Write 3 Sent Done Done false
2743 1558 Acked Write 7 Sent Done Done false
2744 1559 Acked Read 6 Sent Done Done false
2745 1560 Acked Flush 0 Sent Done Done false
2746 1561 Acked Write 8 Sent Done Done false
2747 1562 Acked Read 4 Sent Done Done false
2748 1563 Acked Write 5 Sent Done Done false
2749 1564 Acked Read 9 Sent Done Done false
2750 1565 Acked Flush 0 Sent Done Done false
2751 1566 Acked Write 7 Sent Done Done false
2752 1567 Acked Read 1 Sent Done Done false
2753 1568 Acked Read 6 Sent Done Done false
2754 1569 Acked Read 6 Sent Done Done false
2755 1570 Acked Read 5 Sent Done Done false
2756 1571 Acked Read 6 Sent Done Done false
2757 1572 Acked Read 5 Sent Done Done false
2758 1573 Acked Write 7 Sent Done Done false
2759 1574 Acked Read 8 Sent Done Done false
2760 1575 Acked Flush 0 Sent Done Done false
2761 1576 Acked Read 7 Sent Done Done false
2762 1577 Acked Write 1 Sent Done Done false
2763 1578 Acked Write 5 Sent Done Done false
2764 1579 Acked Write 8 Sent Done Done false
2765 1580 Acked Read 7 Sent Done Done false
2766 1581 Acked Read 3 Sent Done Done false
2767 1582 Acked Write 7 Sent Done Done false
2768 1583 Acked Write 6 Sent Done Done false
2769 1584 Acked Read 10 Sent Done Done false
2770 1585 Acked Read 8 Sent Done Done false
2771 1586 Acked Write 1 Sent Done Done false
2772 1587 Acked Write 9 Sent Done Done false
2773 1588 Acked Write 8 Sent Done Done false
2774 1589 Acked Write 3 Sent Done Done false
2775 1590 Acked Read 8 Sent Done Done false
2776 1591 Acked Read 4 Sent Done Done false
2777 1592 Acked Write 3 Sent Done Done false
2778 1593 Acked Read 2 Sent Done Done false
2779 1594 Acked Write 1 Sent Done Done false
2780 1595 Acked Read 7 Sent Done Done false
2781 1596 Acked Write 6 Sent Done Done false
2782 1597 Acked Read 5 Sent Done Done false
2783 1598 Acked Read 3 Sent Done Done false
2784 1599 Acked Read 3 Sent Done Done false
2785 1600 Acked Flush 0 Sent Done Done false
2786 1601 Acked Read 6 Sent Done Done false
2787 1602 Acked Write 6 Sent Done Done false
2788 1603 Acked Flush 0 Sent Done Done false
2789 STATES DS:0 DS:1 DS:2 TOTAL
2790 Sent 302 0 0 302
2791 Done 0 302 302 604
2792 Skipped 0 0 0 0
2793 Error 0 0 0 0
2794 Last Flush: 1301 1603 1603
2795 Downstairs last five completed: 1301 1300 1299 1298 1297
2796 Upstairs last five completed: 1603 1602 1601 1600 1599
2797 CLIENT: Up:0 ds:302 act:2
27982025-01-11T06:06:02.715ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
27992025-01-11T06:06:02.715ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:32826) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28002025-01-11T06:06:02.716ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28012025-01-11T06:06:02.717ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28022025-01-11T06:06:02.717ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28032025-01-11T06:06:02.717ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28042025-01-11T06:06:02.717ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28052025-01-11T06:06:02.717ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28062025-01-11T06:06:02.717ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28072025-01-11T06:06:02.717ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28082025-01-11T06:06:02.717ZINFOcrucible: [0] Replayed 302 jobs since flush: 1301 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2809 ----------------------------------------------------------------
2810 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2811 Downstairs last five completed: 1603 1602 1601 1600 1599
2812 Upstairs last five completed: 1603 1602 1601 1600 1599
2813 CLIENT: Up:0 ds:0 act:3
2814 Replay: All jobs finished
2815 Main task has work to do, go find it
2816 got dsc Stop(2)
2817 stop 2
2818 Jan 11 06:06:02.969 INFO request completed, latency_us[8830] Got stop action so:false kr:false
2819 : 346, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 22d0ee02-b9e9-48f5-befc-4be5dc7f6a95, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
2820 Jan 11 06:06:02.970 INFO request completed, latency_us: 114, response_code: 200, uri: /state/cid/2, method: GET, req_id: 59d46079-efe2-474a-bde7-f7a0b35bd85a, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
28212025-01-11T06:06:02.970ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28222025-01-11T06:06:02.970ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28232025-01-11T06:06:02.970ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28242025-01-11T06:06:02.970ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28252025-01-11T06:06:02.970ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28262025-01-11T06:06:02.970ZINFOcrucible: client task is exiting = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
2827 [8830] Exited with: ExitStatus(unix_wait_status(9))
2828 [8830][2] reports Exit
2829 Jan 11 06:06:06.973 INFO request completed, latency_us: 273, response_code: 200, uri: /state/cid/2, method: GET, req_id: ea727946-effb-4e41-ac66-d17338bbd54d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
2830 001/300 Read block 1761 len 36864
2831 002/300 Flush
2832 003/300 Write block 4559 len 20480 data: 2 2 2 2 2
2833 004/300 Write block 1711 len 12288 data: 2 2 2
2834 005/300 Read block 3168 len 40960
2835 006/300 Read block 3524 len 40960
2836 007/300 Read block 1312 len 40960
2837 008/300 Read block 2813 len 4096
2838 009/300 Read block 4804 len 32768
2839 010/300 Write block 1883 len 12288 data: 3 3 3
2840 011/300 Write block 2053 len 8192 data: 2 2
2841 012/300 Write block 4061 len 16384 data: 2 2 2 2
2842 013/300 Write block 4460 len 32768 data: 2 2 2 4 4 4 4 4
2843 014/300 Flush
2844 015/300 Write block 4779 len 40960 data: 3 3 3 2 2 2 2 2 2 2
2845 016/300 Read block 187 len 20480
2846 017/300 Write block 3023 len 8192 data: 2 2
2847 018/300 Flush
2848 019/300 Write block 1025 len 24576 data: 2 2 2 2 2 2
2849 020/300 Read block 2374 len 12288
2850 021/300 Write block 4044 len 24576 data: 2 2 2 2 2 2
2851 022/300 Read block 2327 len 40960
2852 023/300 Read block 619 len 40960
2853 024/300 Read block 4748 len 24576
2854 025/300 Read block 3026 len 40960
2855 026/300 Write block 1850 len 32768 data: 3 2 2 2 2 2 2 3
2856 027/300 Read block 2777 len 28672
2857 028/300 Write block 1254 len 32768 data: 2 2 2 2 2 2 2 2
2858 029/300 Read block 1344 len 4096
2859 030/300 Write block 3023 len 32768 data: 3 3 2 2 3 3 3 3
2860 031/300 Flush
2861 032/300 Write block 1870 len 32768 data: 2 2 2 2 2 2 2 2
2862 033/300 Read block 3485 len 24576
2863 034/300 Read block 4152 len 32768
2864 035/300 Read block 3367 len 28672
2865 036/300 Write block 4422 len 8192 data: 2 2
2866 037/300 Write block 986 len 8192 data: 3 3
2867 038/300 Write block 3366 len 16384 data: 2 2 2 2
2868 039/300 Read block 3283 len 28672
2869 040/300 Write block 693 len 4096 data: 2
2870 041/300 Read block 4309 len 28672
2871 042/300 Read block 2699 len 4096
2872 043/300 Write block 2053 len 36864 data: 3 3 2 2 2 2 2 2 2
2873 044/300 Read block 9 len 28672
2874 045/300 Write block 88 len 32768 data: 2 2 2 2 2 2 2 2
2875 046/300 Read block 4658 len 20480
2876 047/300 Read block 3975 len 20480
2877 048/300 Read block 1270 len 8192
2878 049/300 Flush
2879 050/300 Read block 1530 len 4096
2880 051/300 Write block 1953 len 8192 data: 3 3
2881 052/300 Read block 3376 len 16384
2882 053/300 Read block 2488 len 20480
2883 054/300 Write block 3731 len 40960 data: 2 3 3 3 3 3 3 2 2 2
2884 055/300 Flush
2885 056/300 Write block 3658 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2886 057/300 Write block 3583 len 32768 data: 2 2 2 2 2 2 2 2
2887 058/300 Flush
2888 059/300 Write block 2752 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2889 060/300 Read block 4194 len 12288
2890 061/300 Read block 652 len 32768
2891 062/300 Write block 2949 len 12288 data: 4 4 3
2892 063/300 Read block 277 len 24576
2893 064/300 Write block 3480 len 4096 data: 4
2894 065/300 Flush
2895 066/300 Write block 514 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2896 067/300 Read block 3129 len 24576
2897 068/300 Write block 2640 len 20480 data: 2 2 3 3 3
2898 069/300 Write block 1222 len 4096 data: 2
2899 070/300 Read block 2409 len 24576
2900 071/300 Write block 3689 len 20480 data: 2 2 2 2 2
2901 072/300 Read block 143 len 8192
2902 073/300 Write block 4316 len 24576 data: 2 2 2 2 2 2
2903 074/300 Write block 4557 len 32768 data: 2 2 3 3 3 3 3 2
2904 075/300 Read block 2857 len 20480
2905 076/300 Read block 924 len 36864
2906 077/300 Read block 1352 len 36864
2907 078/300 Read block 4070 len 16384
2908 079/300 Flush
2909 080/300 Read block 1449 len 40960
2910 081/300 Read block 3354 len 12288
2911 082/300 Write block 3063 len 12288 data: 2 2 2
2912 083/300 Write block 3814 len 4096 data: 3
2913 084/300 Write block 4364 len 12288 data: 2 2 2
2914 085/300 Read block 1236 len 32768
2915 086/300 Flush
2916 087/300 Write block 4602 len 8192 data: 2 2
2917 088/300 Write block 3651 len 20480 data: 2 2 2 2 3
2918 089/300 Read block 1040 len 24576
2919 090/300 Write block 564 len 8192 data: 2 3
2920 091/300 Read block 3672 len 16384
2921 092/300 Read block 1631 len 12288
2922 093/300 Read block 4821 len 12288
2923 094/300 Write block 4161 len 16384 data: 2 2 2 2
2924 095/300 Write block 2899 len 40960 data: 2 2 2 2 2 2 2 2 3 3
2925 096/300 Write block 2627 len 24576 data: 3 3 2 2 2 2
2926 097/300 Write block 1486 len 28672 data: 2 2 2 2 2 2 2
2927 098/300 Write block 3161 len 12288 data: 2 2 2
2928 099/300 Flush
2929 100/300 Write block 2813 len 16384 data: 2 2 2 2
2930 101/300 Read block 1478 len 4096
2931 102/300 Write block 4041 len 12288 data: 2 2 2
2932 103/300 Flush
2933 104/300 Read block 3073 len 4096
2934 105/300 Read block 3936 len 12288
2935 106/300 Write block 4264 len 28672 data: 3 2 2 2 2 2 2
2936 107/300 Write block 304 len 36864 data: 3 3 3 3 3 3 2 2 2
2937 108/300 Write block 2066 len 12288 data: 2 2 2
2938 109/300 Read block 2795 len 20480
2939 110/300 Read block 1333 len 8192
2940 111/300 Read block 2309 len 4096
2941 112/300 Write block 1307 len 32768 data: 2 2 2 2 2 2 2 2
2942 113/300 Write block 488 len 4096 data: 3
2943 114/300 Write block 4727 len 4096 data: 2
2944 115/300 Read block 3100 len 12288
2945 116/300 Read block 2889 len 8192
2946 117/300 Write block 2393 len 20480 data: 3 3 4 3 3
2947 118/300 Read block 2272 len 4096
2948 119/300 Read block 3597 len 20480
2949 120/300 Read block 433 len 12288
2950 121/300 Write block 3592 len 16384 data: 2 2 3 3
2951 122/300 Write block 3756 len 20480 data: 2 2 2 2 2
2952 123/300 Write block 2862 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2953 124/300 Write block 3089 len 40960 data: 2 2 2 2 2 2 2 2 2 3
2954 125/300 Write block 3558 len 20480 data: 2 2 3 3 3
2955 126/300 Write block 1145 len 20480 data: 2 2 2 2 2
2956 127/300 Read block 3696 len 32768
2957 128/300 Flush
2958 129/300 Read block 3025 len 4096
2959 130/300 Read block 3652 len 20480
2960 131/300 Read block 1674 len 8192
2961 132/300 Read block 1638 len 28672
2962 133/300 Write block 3979 len 8192 data: 2 2
2963 134/300 Write block 1133 len 16384 data: 2 2 2 2
2964 135/300 Read block 2452 len 32768
2965 136/300 Read block 1272 len 20480
2966 137/300 Write block 1202 len 24576 data: 2 2 2 2 2 2
2967 138/300 Write block 1248 len 20480 data: 2 2 2 2 2
2968 139/300 Write block 2127 len 28672 data: 2 2 2 2 2 2 2
2969 140/300 Write block 1642 len 8192 data: 2 2
2970 141/300 Read block 968 len 32768
2971 142/300 Read block 3840 len 36864
2972 143/300 Read block 779 len 16384
2973 144/300 Read block 1501 len 8192
2974 145/300 Flush
2975 146/300 Read block 4252 len 20480
2976 147/300 Read block 3099 len 12288
2977 148/300 Read block 3144 len 12288
2978 149/300 Read block 227 len 16384
2979 150/300 Read block 1984 len 4096
2980 151/300 Write block 2927 len 24576 data: 2 2 2 2 2 2
2981 152/300 Write block 282 len 40960 data: 2 2 2 2 2 2 2 2 3 3
2982 153/300 Write block 4983 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2983 154/300 Write block 1646 len 12288 data: 2 2 2
2984 155/300 Write block 3458 len 24576 data: 3 2 2 2 2 2
2985 156/300 Read block 491 len 4096
2986 157/300 Write block 2648 len 8192 data: 3 3
2987 158/300 Write block 3030 len 16384 data: 4 3 3 3
2988 159/300 Read block 3499 len 16384
2989 160/300 Read block 1315 len 28672
2990 161/300 Flush
2991 162/300 Write block 3207 len 24576 data: 2 2 2 2 3 3
2992 163/300 Read block 2624 len 16384
2993 164/300 Write block 183 len 28672 data: 2 2 2 2 2 2 2
2994 165/300 Write block 4950 len 20480 data: 3 3 3 3 3
2995 166/300 Flush
2996 167/300 Write block 242 len 24576 data: 2 2 2 2 2 2
2997 168/300 Read block 2992 len 36864
2998 169/300 Read block 3512 len 32768
2999 170/300 Flush
3000 171/300 Write block 2785 len 12288 data: 2 2 3
3001 172/300 Read block 1543 len 28672
3002 173/300 Flush
3003 174/300 Read block 1897 len 32768
3004 175/300 Read block 1726 len 8192
3005 176/300 Write block 2636 len 28672 data: 2 2 2 2 3 3 4
3006 177/300 Flush
3007 178/300 Flush
3008 179/300 Read block 3444 len 40960
3009 180/300 Read block 1059 len 24576
3010 181/300 Flush
3011 182/300 Read block 977 len 24576
3012 183/300 Write block 2135 len 16384 data: 2 2 2 2
3013 184/300 Read block 4254 len 8192
3014 185/300 Write block 4557 len 40960 data: 3 3 4 4 4 4 4 3 3 3
3015 186/300 Read block 4307 len 12288
3016 187/300 Read block 3559 len 12288
3017 188/300 Flush
3018 189/300 Read block 266 len 16384
3019 190/300 Read block 534 len 20480
3020 191/300 Read block 1050 len 4096
3021 192/300 Write block 2916 len 16384 data: 2 2 2 2
3022 193/300 Read block 3069 len 32768
3023 194/300 Write block 744 len 32768 data: 3 3 3 2 2 2 2 2
3024 195/300 Read block 1201 len 24576
3025 196/300 Read block 4031 len 40960
3026 197/300 Read block 4050 len 32768
3027 198/300 Read block 647 len 40960
3028 199/300 Read block 1121 len 32768
3029 200/300 Write block 366 len 4096 data: 2
3030 201/300 Flush
3031 202/300 Read block 984 len 4096
3032 203/300 Write block 3186 len 8192 data: 3 3
3033 204/300 Write block 1553 len 20480 data: 2 2 2 2 2
3034 205/300 Read block 237 len 32768
3035 206/300 Read block 4386 len 24576
3036 207/300 Read block 2015 len 40960
3037 208/300 Flush
3038 209/300 Write block 1812 len 8192 data: 2 2
3039 210/300 Write block 1183 len 36864 data: 2 2 2 2 2 2 2 3 3
3040 211/300 Read block 290 len 32768
3041 212/300 Write block 3121 len 8192 data: 3 3
3042 213/300 Read block 3142 len 16384
3043 214/300 Read block 1096 len 36864
3044 215/300 Flush
3045 216/300 Read block 1762 len 20480
3046 217/300 Read block 1878 len 16384
3047 218/300 Write block 3692 len 20480 data: 3 3 2 2 2
3048 219/300 Write block 722 len 36864 data: 2 2 2 2 2 2 2 2 2
3049 220/300 Write block 106 len 32768 data: 2 2 2 2 2 2 2 2
3050 221/300 Read block 1812 len 40960
3051 222/300 Write block 1438 len 32768 data: 2 2 2 2 2 2 2 2
3052 223/300 Read block 4178 len 36864
3053 224/300 Flush
3054 225/300 Write block 2923 len 40960 data: 2 2 2 2 3 3 3 3 3 3
3055 226/300 Flush
3056 227/300 Read block 739 len 12288
3057 228/300 Write block 3883 len 28672 data: 2 2 2 2 2 2 2
3058 229/300 Read block 3880 len 28672
3059 230/300 Write block 938 len 20480 data: 2 2 2 2 2
3060 231/300 Read block 401 len 36864
3061 232/300 Write block 2772 len 36864 data: 2 2 2 2 2 2 2 2 2
3062 233/300 Flush
3063 234/300 Flush
3064 235/300 Write block 4768 len 28672 data: 2 2 2 2 2 3 3
3065 236/300 Write block 3027 len 20480 data: 4 4 4 5 4
3066 237/300 Read block 1998 len 8192
3067 238/300 Read block 4593 len 20480
3068 239/300 Read block 4067 len 40960
3069 240/300 Read block 4344 len 20480
3070 241/300 Read block 2553 len 20480
3071 242/300 Read block 3872 len 28672
3072 243/300 Read block 95 len 16384
3073 244/300 Read block 1164 len 4096
3074 245/300 Read block 759 len 4096
3075 246/300 Flush
3076 247/300 Write block 4760 len 32768 data: 2 2 2 2 3 2 2 2
3077 248/300 Read block 763 len 12288
3078 249/300 Write block 517 len 28672 data: 3 3 3 3 3 3 3
3079 250/300 Read block 3158 len 32768
3080 251/300 Write block 2255 len 24576 data: 2 2 2 2 2 2
3081 252/300 Read block 241 len 24576
3082 253/300 Write block 80 len 12288 data: 2 2 2
3083 254/300 Read block 1912 len 8192
3084 255/300 Flush
3085 256/300 Read block 3100 len 40960
3086 257/300 Read block 4855 len 20480
3087 258/300 Read block 3402 len 16384
3088 259/300 Write block 3473 len 32768 data: 2 2 4 4 4 4 4 5
3089 260/300 Write block 2094 len 24576 data: 2 2 2 3 3 3
3090 261/300 Write block 1976 len 8192 data: 2 2
3091 262/300 Read block 1788 len 16384
3092 263/300 Read block 1767 len 16384
3093 264/300 Read block 1761 len 8192
3094 265/300 Flush
3095 266/300 Read block 3518 len 36864
3096 267/300 Write block 2082 len 28672 data: 2 2 2 2 2 2 2
3097 268/300 Write block 1004 len 36864 data: 2 2 2 2 2 2 2 2 2
3098 269/300 Write block 3501 len 32768 data: 2 2 2 2 4 4 4 4
3099 270/300 Read block 484 len 40960
3100 271/300 Flush
3101 272/300 Flush
3102 273/300 Write block 3301 len 4096 data: 2
3103 274/300 Write block 3448 len 8192 data: 4 4
3104 275/300 Write block 1896 len 20480 data: 2 2 2 2 2
3105 276/300 Write block 4849 len 16384 data: 2 2 2 2
3106 277/300 Flush
3107 278/300 Write block 4691 len 40960 data: 3 3 3 3 3 4 4 4 4 3
3108 279/300 Write block 1358 len 32768 data: 2 2 2 2 2 2 2 2
3109 280/300 Write block 2092 len 32768 data: 2 2 3 3 3 4 4 4
3110 281/300 Write block 1187 len 8192 data: 3 3
3111 282/300 Write block 3050 len 4096 data: 2
3112 283/300 Flush
3113 284/300 Write block 4295 len 40960 data: 3 3 4 4 4 4 5 5 4 3
3114 285/300 Read block 3547 len 24576
3115 286/300 Write block 1482 len 8192 data: 2 2
3116 287/300 Write block 1264 len 4096 data: 2
3117 288/300 Read block 2406 len 28672
3118 289/300 Flush
3119 290/300 Read block 15 len 12288
3120 291/300 Write block 1604 len 8192 data: 2 2
3121 292/300 Read block 742 len 28672
3122 293/300 Flush
3123 294/300 Read block 2826 len 32768
3124 295/300 Read block 3337 len 4096
3125 296/300 Flush
3126 297/300 Read block 3171 len 36864
3127 298/300 Read block 3790 len 32768
3128 299/300 Flush
3129 300/300 Read block 1846 len 32768
3130 301/300 Read block 659 len 40960
3131 Main task has work to do, go find it
3132 got dsc Start(2)
3133 start 2
3134 Jan 11 06:06:07.208 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/2, method: POST, req_id: 4286b50b-cd60-41ba-a515-2925564bb90d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
3135 Replay: started 2, returned:Ok(())
3136 ----------------------------------------------------------------
3137 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3138 States: Active Active Offline
3139 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3140 1604 Acked Read 9 Done Done Sent false
3141 1605 Acked Flush 0 Done Done Sent false
3142 1606 Acked Write 5 Done Done Sent false
3143 1607 Acked Write 3 Done Done Sent false
3144 1608 Acked Read 10 Done Done Sent false
3145 1609 Acked Read 10 Done Done Sent false
3146 1610 Acked Read 10 Done Done Sent false
3147 1611 Acked Read 1 Done Done Sent false
3148 1612 Acked Read 8 Done Done Sent false
3149 1613 Acked Write 3 Done Done Sent false
3150 1614 Acked Write 2 Done Done Sent false
3151 1615 Acked Write 4 Done Done Sent false
3152 1616 Acked Write 8 Done Done Sent false
3153 1617 Acked Flush 0 Done Done Sent false
3154 1618 Acked Write 10 Done Done Sent false
3155 1619 Acked Read 5 Done Done Sent false
3156 1620 Acked Write 2 Done Done Sent false
3157 1621 Acked Flush 0 Done Done Sent false
3158 1622 Acked Write 6 Done Done Sent false
3159 1623 Acked Read 3 Done Done Sent false
3160 1624 Acked Write 6 Done Done Sent false
3161 1625 Acked Read 10 Done Done Sent false
3162 1626 Acked Read 10 Done Done Sent false
3163 1627 Acked Read 6 Done Done Sent false
3164 1628 Acked Read 10 Done Done Sent false
3165 1629 Acked Write 8 Done Done Sent false
3166 1630 Acked Read 7 Done Done Sent false
3167 1631 Acked Write 8 Done Done Sent false
3168 1632 Acked Read 1 Done Done Sent false
3169 1633 Acked Write 8 Done Done Sent false
3170 1634 Acked Flush 0 Done Done Sent false
3171 1635 Acked Write 8 Done Done Sent false
3172 1636 Acked Read 6 Done Done Sent false
3173 1637 Acked Read 8 Done Done Sent false
3174 1638 Acked Read 7 Done Done Sent false
3175 1639 Acked Write 2 Done Done Sent false
3176 1640 Acked Write 2 Done Done Sent false
3177 1641 Acked Write 4 Done Done Sent false
3178 1642 Acked Read 7 Done Done Sent false
3179 1643 Acked Write 1 Done Done Sent false
3180 1644 Acked Read 7 Done Done Sent false
3181 1645 Acked Read 1 Done Done Sent false
3182 1646 Acked Write 9 Done Done Sent false
3183 1647 Acked Read 7 Done Done Sent false
3184 1648 Acked Write 8 Done Done Sent false
3185 1649 Acked Read 5 Done Done Sent false
3186 1650 Acked Read 5 Done Done Sent false
3187 1651 Acked Read 2 Done Done Sent false
3188 1652 Acked Flush 0 Done Done Sent false
3189 1653 Acked Read 1 Done Done Sent false
3190 1654 Acked Write 2 Done Done Sent false
3191 1655 Acked Read 4 Done Done Sent false
3192 1656 Acked Read 5 Done Done Sent false
3193 1657 Acked Write 10 Done Done Sent false
3194 1658 Acked Flush 0 Done Done Sent false
3195 1659 Acked Write 10 Done Done Sent false
3196 1660 Acked Write 8 Done Done Sent false
3197 1661 Acked Flush 0 Done Done Sent false
3198 1662 Acked Write 10 Done Done Sent false
3199 1663 Acked Read 3 Done Done Sent false
3200 1664 Acked Read 8 Done Done Sent false
3201 1665 Acked Write 3 Done Done Sent false
3202 1666 Acked Read 6 Done Done Sent false
3203 1667 Acked Write 1 Done Done Sent false
3204 1668 Acked Flush 0 Done Done Sent false
3205 1669 Acked Write 10 Done Done Sent false
3206 1670 Acked Read 6 Done Done Sent false
3207 1671 Acked Write 5 Done Done Sent false
3208 1672 Acked Write 1 Done Done Sent false
3209 1673 Acked Read 6 Done Done Sent false
3210 1674 Acked Write 5 Done Done Sent false
3211 1675 Acked Read 2 Done Done Sent false
3212 1676 Acked Write 6 Done Done Sent false
3213 1677 Acked Write 8 Done Done Sent false
3214 1678 Acked Read 5 Done Done Sent false
3215 1679 Acked Read 9 Done Done Sent false
3216 1680 Acked Read 9 Done Done Sent false
3217 1681 Acked Read 4 Done Done Sent false
3218 1682 Acked Flush 0 Done Done Sent false
3219 1683 Acked Read 10 Done Done Sent false
3220 1684 Acked Read 3 Done Done Sent false
3221 1685 Acked Write 3 Done Done Sent false
3222 1686 Acked Write 1 Done Done Sent false
3223 1687 Acked Write 3 Done Done Sent false
3224 1688 Acked Read 8 Done Done Sent false
3225 1689 Acked Flush 0 Done Done Sent false
3226 1690 Acked Write 2 Done Done Sent false
3227 1691 Acked Write 5 Done Done Sent false
3228 1692 Acked Read 6 Done Done Sent false
3229 1693 Acked Write 2 Done Done Sent false
3230 1694 Acked Read 4 Done Done Sent false
3231 1695 Acked Read 3 Done Done Sent false
3232 1696 Acked Read 3 Done Done Sent false
3233 1697 Acked Write 4 Done Done Sent false
3234 1698 Acked Write 10 Done Done Sent false
3235 1699 Acked Write 6 Done Done Sent false
3236 1700 Acked Write 7 Done Done Sent false
3237 1701 Acked Write 3 Done Done Sent false
3238 1702 Acked Flush 0 Done Done Sent false
3239 1703 Acked Write 4 Done Done Sent false
3240 1704 Acked Read 1 Done Done Sent false
3241 1705 Acked Write 3 Done Done Sent false
3242 1706 Acked Flush 0 Done Done Sent false
3243 1707 Acked Read 1 Done Done Sent false
3244 1708 Acked Read 3 Done Done Sent false
3245 1709 Acked Write 7 Done Done Sent false
3246 1710 Acked Write 9 Done Done Sent false
3247 1711 Acked Write 3 Done Done Sent false
3248 1712 Acked Read 5 Done Done Sent false
3249 1713 Acked Read 2 Done Done Sent false
3250 1714 Acked Read 1 Done Done Sent false
3251 1715 Acked Write 8 Done Done Sent false
3252 1716 Acked Write 1 Done Done Sent false
3253 1717 Acked Write 1 Done Done Sent false
3254 1718 Acked Read 3 Done Done Sent false
3255 1719 Acked Read 2 Done Done Sent false
3256 1720 Acked Write 5 Done Done Sent false
3257 1721 Acked Read 1 Done Done Sent false
3258 1722 Acked Read 5 Done Done Sent false
3259 1723 Acked Read 3 Done Done Sent false
3260 1724 Acked Write 4 Done Done Sent false
3261 1725 Acked Write 5 Done Done Sent false
3262 1726 Acked Write 10 Done Done Sent false
3263 1727 Acked Write 10 Done Done Sent false
3264 1728 Acked Write 5 Done Done Sent false
3265 1729 Acked Write 5 Done Done Sent false
3266 1730 Acked Read 8 Done Done Sent false
3267 1731 Acked Flush 0 Done Done Sent false
3268 1732 Acked Read 1 Done Done Sent false
3269 1733 Acked Read 5 Done Done Sent false
3270 1734 Acked Read 2 Done Done Sent false
3271 1735 Acked Read 7 Done Done Sent false
3272 1736 Acked Write 2 Done Done Sent false
3273 1737 Acked Write 4 Done Done Sent false
3274 1738 Acked Read 8 Done Done Sent false
3275 1739 Acked Read 5 Done Done Sent false
3276 1740 Acked Write 6 Done Done Sent false
3277 1741 Acked Write 5 Done Done Sent false
3278 1742 Acked Write 7 Done Done Sent false
3279 1743 Acked Write 2 Done Done Sent false
3280 1744 Acked Read 8 Done Done Sent false
3281 1745 Acked Read 9 Done Done Sent false
3282 1746 Acked Read 4 Done Done Sent false
3283 1747 Acked Read 2 Done Done Sent false
3284 1748 Acked Flush 0 Done Done Sent false
3285 1749 Acked Read 5 Done Done Sent false
3286 1750 Acked Read 3 Done Done Sent false
3287 1751 Acked Read 3 Done Done Sent false
3288 1752 Acked Read 4 Done Done Sent false
3289 1753 Acked Read 1 Done Done Sent false
3290 1754 Acked Write 6 Done Done Sent false
3291 1755 Acked Write 10 Done Done Sent false
3292 1756 Acked Write 10 Done Done Sent false
3293 1757 Acked Write 3 Done Done Sent false
3294 1758 Acked Write 6 Done Done Sent false
3295 1759 Acked Read 1 Done Done Sent false
3296 1760 Acked Write 2 Done Done Sent false
3297 1761 Acked Write 4 Done Done Sent false
3298 1762 Acked Read 4 Done Done Sent false
3299 1763 Acked Read 7 Done Done Sent false
3300 1764 Acked Flush 0 Done Done Sent false
3301 1765 Acked Write 6 Done Done Sent false
3302 1766 Acked Read 4 Done Done Sent false
3303 1767 Acked Write 7 Done Done Sent false
3304 1768 Acked Write 5 Done Done Sent false
3305 1769 Acked Flush 0 Done Done Sent false
3306 1770 Acked Write 6 Done Done Sent false
3307 1771 Acked Read 9 Done Done Sent false
3308 1772 Acked Read 8 Done Done Sent false
3309 1773 Acked Flush 0 Done Done Sent false
3310 1774 Acked Write 3 Done Done Sent false
3311 1775 Acked Read 7 Done Done Sent false
3312 1776 Acked Flush 0 Done Done Sent false
3313 1777 Acked Read 8 Done Done Sent false
3314 1778 Acked Read 2 Done Done Sent false
3315 1779 Acked Write 7 Done Done Sent false
3316 1780 Acked Flush 0 Done Done Sent false
3317 1781 Acked Flush 0 Done Done Sent false
3318 1782 Acked Read 10 Done Done Sent false
3319 1783 Acked Read 6 Done Done Sent false
3320 1784 Acked Flush 0 Done Done Sent false
3321 1785 Acked Read 6 Done Done Sent false
3322 1786 Acked Write 4 Done Done Sent false
3323 1787 Acked Read 2 Done Done Sent false
3324 1788 Acked Write 10 Done Done Sent false
3325 1789 Acked Read 3 Done Done Sent false
3326 1790 Acked Read 3 Done Done Sent false
3327 1791 Acked Flush 0 Done Done Sent false
3328 1792 Acked Read 4 Done Done Sent false
3329 1793 Acked Read 5 Done Done Sent false
3330 1794 Acked Read 1 Done Done Sent false
3331 1795 Acked Write 4 Done Done Sent false
3332 1796 Acked Read 8 Done Done Sent false
3333 1797 Acked Write 8 Done Done Sent false
3334 1798 Acked Read 6 Done Done Sent false
3335 1799 Acked Read 10 Done Done Sent false
3336 1800 Acked Read 8 Done Done Sent false
3337 1801 Acked Read 10 Done Done Sent false
3338 1802 Acked Read 8 Done Done Sent false
3339 1803 Acked Write 1 Done Done Sent false
3340 1804 Acked Flush 0 Done Done Sent false
3341 1805 Acked Read 1 Done Done Sent false
3342 1806 Acked Write 2 Done Done Sent false
3343 1807 Acked Write 5 Done Done Sent false
3344 1808 Acked Read 8 Done Done Sent false
3345 1809 Acked Read 6 Done Done Sent false
3346 1810 Acked Read 10 Done Done Sent false
3347 1811 Acked Flush 0 Done Done Sent false
3348 1812 Acked Write 2 Done Done Sent false
3349 1813 Acked Write 9 Done Done Sent false
3350 1814 Acked Read 8 Done Done Sent false
3351 1815 Acked Write 2 Done Done Sent false
3352 1816 Acked Read 4 Done Done Sent false
3353 1817 Acked Read 9 Done Done Sent false
3354 1818 Acked Flush 0 Done Done Sent false
3355 1819 Acked Read 5 Done Done Sent false
3356 1820 Acked Read 4 Done Done Sent false
3357 1821 Acked Write 5 Done Done Sent false
3358 1822 Acked Write 9 Done Done Sent false
3359 1823 Acked Write 8 Done Done Sent false
3360 1824 Acked Read 10 Done Done Sent false
3361 1825 Acked Write 8 Done Done Sent false
3362 1826 Acked Read 9 Done Done Sent false
3363 1827 Acked Flush 0 Done Done Sent false
3364 1828 Acked Write 10 Done Done Sent false
3365 1829 Acked Flush 0 Done Done Sent false
3366 1830 Acked Read 3 Done Done Sent false
3367 1831 Acked Write 7 Done Done Sent false
3368 1832 Acked Read 7 Done Done Sent false
3369 1833 Acked Write 5 Done Done Sent false
3370 1834 Acked Read 9 Done Done Sent false
3371 1835 Acked Write 9 Done Done Sent false
3372 1836 Acked Flush 0 Done Done Sent false
3373 1837 Acked Flush 0 Done Done Sent false
3374 1838 Acked Write 7 Done Done Sent false
3375 1839 Acked Write 5 Done Done Sent false
3376 1840 Acked Read 2 Done Done Sent false
3377 1841 Acked Read 5 Done Done Sent false
3378 1842 Acked Read 10 Done Done Sent false
3379 1843 Acked Read 5 Done Done Sent false
3380 1844 Acked Read 5 Done Done Sent false
3381 1845 Acked Read 7 Done Done Sent false
3382 1846 Acked Read 4 Done Done Sent false
3383 1847 Acked Read 1 Done Done Sent false
3384 1848 Acked Read 1 Done Done Sent false
3385 1849 Acked Flush 0 Done Done Sent false
3386 1850 Acked Write 8 Done Done Sent false
3387 1851 Acked Read 3 Done Done Sent false
3388 1852 Acked Write 7 Done Done Sent false
3389 1853 Acked Read 8 Done Done Sent false
3390 1854 Acked Write 6 Done Done Sent false
3391 1855 Acked Read 6 Done Done Sent false
3392 1856 Acked Write 3 Done Done Sent false
3393 1857 Acked Read 2 Done Done Sent false
3394 1858 Acked Flush 0 Done Done Sent false
3395 1859 Acked Read 10 Done Done Sent false
3396 1860 Acked Read 5 Done Done Sent false
3397 1861 Acked Read 4 Done Done Sent false
3398 1862 Acked Write 8 Done Done Sent false
3399 1863 Acked Write 6 Done Done Sent false
3400 1864 Acked Write 2 Done Done Sent false
3401 1865 Acked Read 4 Done Done Sent false
3402 1866 Acked Read 4 Done Done Sent false
3403 1867 Acked Read 2 Done Done Sent false
3404 1868 Acked Flush 0 Done Done Sent false
3405 1869 Acked Read 9 Done Done Sent false
3406 1870 Acked Write 7 Done Done Sent false
3407 1871 Acked Write 9 Done Done Sent false
3408 1872 Acked Write 8 Done Done Sent false
3409 1873 Acked Read 10 Done Done Sent false
3410 1874 Acked Flush 0 Done Done Sent false
3411 1875 Acked Flush 0 Done Done Sent false
3412 1876 Acked Write 1 Done Done Sent false
3413 1877 Acked Write 2 Done Done Sent false
3414 1878 Acked Write 5 Done Done Sent false
3415 1879 Acked Write 4 Done Done Sent false
3416 1880 Acked Flush 0 Done Done Sent false
3417 1881 Acked Write 10 Done Done Sent false
3418 1882 Acked Write 8 Done Done Sent false
3419 1883 Acked Write 8 Done Done Sent false
3420 1884 Acked Write 2 Done Done Sent false
3421 1885 Acked Write 1 Done Done Sent false
3422 1886 Acked Flush 0 Done Done Sent false
3423 1887 Acked Write 10 Done Done Sent false
3424 1888 Acked Read 6 Done Done Sent false
3425 1889 Acked Write 2 Done Done Sent false
3426 1890 Acked Write 1 Done Done Sent false
3427 1891 Acked Read 7 Done Done Sent false
3428 1892 Acked Flush 0 Done Done Sent false
3429 1893 Acked Read 3 Done Done Sent false
3430 1894 Acked Write 2 Done Done Sent false
3431 1895 Acked Read 7 Done Done Sent false
3432 1896 Acked Flush 0 Done Done Sent false
3433 1897 Acked Read 8 Done Done Sent false
3434 1898 Acked Read 1 Done Done Sent false
3435 1899 Acked Flush 0 Done Done Sent false
3436 1900 Acked Read 9 Done Done Sent false
3437 1901 Acked Read 8 Done Done Sent false
3438 1902 Acked Flush 0 Done Done Sent false
3439 1903 Acked Read 8 Done Done Sent false
3440 1904 Acked Read 10 Done Done Sent false
3441 STATES DS:0 DS:1 DS:2 TOTAL
3442 Sent 0 0 301 301
3443 Done 301 301 0 602
3444 Skipped 0 0 0 0
3445 Error 0 0 0 0
3446 Last Flush: 1902 1902 1603
3447 Downstairs last five completed: 1603 1602 1601 1600 1599
3448 Upstairs last five completed: 1904 1903 1902 1901 1900
3449 CLIENT: Up:0 ds:301 act:2
3450 [8830] got start action
3451 [8830] I am going to restart
3452 Starting downstairs at port 8830
3453 Make output file at "/tmp/dsc/downstairs-8830.txt"
3454 [8830][2] reports Starting
3455 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1207)
3456 [8830][2] reports Running
3457 ----------------------------------------------------------------
3458 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3459 States: Active Active Offline
3460 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3461 1604 Acked Read 9 Done Done Sent false
3462 1605 Acked Flush 0 Done Done Sent false
3463 1606 Acked Write 5 Done Done Sent false
3464 1607 Acked Write 3 Done Done Sent false
3465 1608 Acked Read 10 Done Done Sent false
3466 1609 Acked Read 10 Done Done Sent false
3467 1610 Acked Read 10 Done Done Sent false
3468 1611 Acked Read 1 Done Done Sent false
3469 1612 Acked Read 8 Done Done Sent false
3470 1613 Acked Write 3 Done Done Sent false
3471 1614 Acked Write 2 Done Done Sent false
3472 1615 Acked Write 4 Done Done Sent false
3473 1616 Acked Write 8 Done Done Sent false
3474 1617 Acked Flush 0 Done Done Sent false
3475 1618 Acked Write 10 Done Done Sent false
3476 1619 Acked Read 5 Done Done Sent false
3477 1620 Acked Write 2 Done Done Sent false
3478 1621 Acked Flush 0 Done Done Sent false
3479 1622 Acked Write 6 Done Done Sent false
3480 1623 Acked Read 3 Done Done Sent false
3481 1624 Acked Write 6 Done Done Sent false
3482 1625 Acked Read 10 Done Done Sent false
3483 1626 Acked Read 10 Done Done Sent false
3484 1627 Acked Read 6 Done Done Sent false
3485 1628 Acked Read 10 Done Done Sent false
3486 1629 Acked Write 8 Done Done Sent false
3487 1630 Acked Read 7 Done Done Sent false
3488 1631 Acked Write 8 Done Done Sent false
3489 1632 Acked Read 1 Done Done Sent false
3490 1633 Acked Write 8 Done Done Sent false
3491 1634 Acked Flush 0 Done Done Sent false
3492 1635 Acked Write 8 Done Done Sent false
3493 1636 Acked Read 6 Done Done Sent false
3494 1637 Acked Read 8 Done Done Sent false
3495 1638 Acked Read 7 Done Done Sent false
3496 1639 Acked Write 2 Done Done Sent false
3497 1640 Acked Write 2 Done Done Sent false
3498 1641 Acked Write 4 Done Done Sent false
3499 1642 Acked Read 7 Done Done Sent false
3500 1643 Acked Write 1 Done Done Sent false
3501 1644 Acked Read 7 Done Done Sent false
3502 1645 Acked Read 1 Done Done Sent false
3503 1646 Acked Write 9 Done Done Sent false
3504 1647 Acked Read 7 Done Done Sent false
3505 1648 Acked Write 8 Done Done Sent false
3506 1649 Acked Read 5 Done Done Sent false
3507 1650 Acked Read 5 Done Done Sent false
3508 1651 Acked Read 2 Done Done Sent false
3509 1652 Acked Flush 0 Done Done Sent false
3510 1653 Acked Read 1 Done Done Sent false
3511 1654 Acked Write 2 Done Done Sent false
3512 1655 Acked Read 4 Done Done Sent false
3513 1656 Acked Read 5 Done Done Sent false
3514 1657 Acked Write 10 Done Done Sent false
3515 1658 Acked Flush 0 Done Done Sent false
3516 1659 Acked Write 10 Done Done Sent false
3517 1660 Acked Write 8 Done Done Sent false
3518 1661 Acked Flush 0 Done Done Sent false
3519 1662 Acked Write 10 Done Done Sent false
3520 1663 Acked Read 3 Done Done Sent false
3521 1664 Acked Read 8 Done Done Sent false
3522 1665 Acked Write 3 Done Done Sent false
3523 1666 Acked Read 6 Done Done Sent false
3524 1667 Acked Write 1 Done Done Sent false
3525 1668 Acked Flush 0 Done Done Sent false
3526 1669 Acked Write 10 Done Done Sent false
3527 1670 Acked Read 6 Done Done Sent false
3528 1671 Acked Write 5 Done Done Sent false
3529 1672 Acked Write 1 Done Done Sent false
3530 1673 Acked Read 6 Done Done Sent false
3531 1674 Acked Write 5 Done Done Sent false
3532 1675 Acked Read 2 Done Done Sent false
3533 1676 Acked Write 6 Done Done Sent false
3534 1677 Acked Write 8 Done Done Sent false
3535 1678 Acked Read 5 Done Done Sent false
3536 1679 Acked Read 9 Done Done Sent false
3537 1680 Acked Read 9 Done Done Sent false
3538 1681 Acked Read 4 Done Done Sent false
3539 1682 Acked Flush 0 Done Done Sent false
3540 1683 Acked Read 10 Done Done Sent false
3541 1684 Acked Read 3 Done Done Sent false
3542 1685 Acked Write 3 Done Done Sent false
3543 1686 Acked Write 1 Done Done Sent false
3544 1687 Acked Write 3 Done Done Sent false
3545 1688 Acked Read 8 Done Done Sent false
3546 1689 Acked Flush 0 Done Done Sent false
3547 1690 Acked Write 2 Done Done Sent false
3548 1691 Acked Write 5 Done Done Sent false
3549 1692 Acked Read 6 Done Done Sent false
3550 1693 Acked Write 2 Done Done Sent false
3551 1694 Acked Read 4 Done Done Sent false
3552 1695 Acked Read 3 Done Done Sent false
3553 1696 Acked Read 3 Done Done Sent false
3554 1697 Acked Write 4 Done Done Sent false
3555 1698 Acked Write 10 Done Done Sent false
3556 1699 Acked Write 6 Done Done Sent false
3557 1700 Acked Write 7 Done Done Sent false
3558 1701 Acked Write 3 Done Done Sent false
3559 1702 Acked Flush 0 Done Done Sent false
3560 1703 Acked Write 4 Done Done Sent false
3561 1704 Acked Read 1 Done Done Sent false
3562 1705 Acked Write 3 Done Done Sent false
3563 1706 Acked Flush 0 Done Done Sent false
3564 1707 Acked Read 1 Done Done Sent false
3565 1708 Acked Read 3 Done Done Sent false
3566 1709 Acked Write 7 Done Done Sent false
3567 1710 Acked Write 9 Done Done Sent false
3568 1711 Acked Write 3 Done Done Sent false
3569 1712 Acked Read 5 Done Done Sent false
3570 1713 Acked Read 2 Done Done Sent false
3571 1714 Acked Read 1 Done Done Sent false
3572 1715 Acked Write 8 Done Done Sent false
3573 1716 Acked Write 1 Done Done Sent false
3574 1717 Acked Write 1 Done Done Sent false
3575 1718 Acked Read 3 Done Done Sent false
3576 1719 Acked Read 2 Done Done Sent false
3577 1720 Acked Write 5 Done Done Sent false
3578 1721 Acked Read 1 Done Done Sent false
3579 1722 Acked Read 5 Done Done Sent false
3580 1723 Acked Read 3 Done Done Sent false
3581 1724 Acked Write 4 Done Done Sent false
3582 1725 Acked Write 5 Done Done Sent false
3583 1726 Acked Write 10 Done Done Sent false
3584 1727 Acked Write 10 Done Done Sent false
3585 1728 Acked Write 5 Done Done Sent false
3586 1729 Acked Write 5 Done Done Sent false
3587 1730 Acked Read 8 Done Done Sent false
3588 1731 Acked Flush 0 Done Done Sent false
3589 1732 Acked Read 1 Done Done Sent false
3590 1733 Acked Read 5 Done Done Sent false
3591 1734 Acked Read 2 Done Done Sent false
3592 1735 Acked Read 7 Done Done Sent false
3593 1736 Acked Write 2 Done Done Sent false
3594 1737 Acked Write 4 Done Done Sent false
3595 1738 Acked Read 8 Done Done Sent false
3596 1739 Acked Read 5 Done Done Sent false
3597 1740 Acked Write 6 Done Done Sent false
3598 1741 Acked Write 5 Done Done Sent false
3599 1742 Acked Write 7 Done Done Sent false
3600 1743 Acked Write 2 Done Done Sent false
3601 1744 Acked Read 8 Done Done Sent false
3602 1745 Acked Read 9 Done Done Sent false
3603 1746 Acked Read 4 Done Done Sent false
3604 1747 Acked Read 2 Done Done Sent false
3605 1748 Acked Flush 0 Done Done Sent false
3606 1749 Acked Read 5 Done Done Sent false
3607 1750 Acked Read 3 Done Done Sent false
3608 1751 Acked Read 3 Done Done Sent false
3609 1752 Acked Read 4 Done Done Sent false
3610 1753 Acked Read 1 Done Done Sent false
3611 1754 Acked Write 6 Done Done Sent false
3612 1755 Acked Write 10 Done Done Sent false
3613 1756 Acked Write 10 Done Done Sent false
3614 1757 Acked Write 3 Done Done Sent false
3615 1758 Acked Write 6 Done Done Sent false
3616 1759 Acked Read 1 Done Done Sent false
3617 1760 Acked Write 2 Done Done Sent false
3618 1761 Acked Write 4 Done Done Sent false
3619 1762 Acked Read 4 Done Done Sent false
3620 1763 Acked Read 7 Done Done Sent false
3621 1764 Acked Flush 0 Done Done Sent false
3622 1765 Acked Write 6 Done Done Sent false
3623 1766 Acked Read 4 Done Done Sent false
3624 1767 Acked Write 7 Done Done Sent false
3625 1768 Acked Write 5 Done Done Sent false
3626 1769 Acked Flush 0 Done Done Sent false
3627 1770 Acked Write 6 Done Done Sent false
3628 1771 Acked Read 9 Done Done Sent false
3629 1772 Acked Read 8 Done Done Sent false
3630 1773 Acked Flush 0 Done Done Sent false
3631 1774 Acked Write 3 Done Done Sent false
3632 1775 Acked Read 7 Done Done Sent false
3633 1776 Acked Flush 0 Done Done Sent false
3634 1777 Acked Read 8 Done Done Sent false
3635 1778 Acked Read 2 Done Done Sent false
3636 1779 Acked Write 7 Done Done Sent false
3637 1780 Acked Flush 0 Done Done Sent false
3638 1781 Acked Flush 0 Done Done Sent false
3639 1782 Acked Read 10 Done Done Sent false
3640 1783 Acked Read 6 Done Done Sent false
3641 1784 Acked Flush 0 Done Done Sent false
3642 1785 Acked Read 6 Done Done Sent false
3643 1786 Acked Write 4 Done Done Sent false
3644 1787 Acked Read 2 Done Done Sent false
3645 1788 Acked Write 10 Done Done Sent false
3646 1789 Acked Read 3 Done Done Sent false
3647 1790 Acked Read 3 Done Done Sent false
3648 1791 Acked Flush 0 Done Done Sent false
3649 1792 Acked Read 4 Done Done Sent false
3650 1793 Acked Read 5 Done Done Sent false
3651 1794 Acked Read 1 Done Done Sent false
3652 1795 Acked Write 4 Done Done Sent false
3653 1796 Acked Read 8 Done Done Sent false
3654 1797 Acked Write 8 Done Done Sent false
3655 1798 Acked Read 6 Done Done Sent false
3656 1799 Acked Read 10 Done Done Sent false
3657 1800 Acked Read 8 Done Done Sent false
3658 1801 Acked Read 10 Done Done Sent false
3659 1802 Acked Read 8 Done Done Sent false
3660 1803 Acked Write 1 Done Done Sent false
3661 1804 Acked Flush 0 Done Done Sent false
3662 1805 Acked Read 1 Done Done Sent false
3663 1806 Acked Write 2 Done Done Sent false
3664 1807 Acked Write 5 Done Done Sent false
3665 1808 Acked Read 8 Done Done Sent false
3666 1809 Acked Read 6 Done Done Sent false
3667 1810 Acked Read 10 Done Done Sent false
3668 1811 Acked Flush 0 Done Done Sent false
3669 1812 Acked Write 2 Done Done Sent false
3670 1813 Acked Write 9 Done Done Sent false
3671 1814 Acked Read 8 Done Done Sent false
3672 1815 Acked Write 2 Done Done Sent false
3673 1816 Acked Read 4 Done Done Sent false
3674 1817 Acked Read 9 Done Done Sent false
3675 1818 Acked Flush 0 Done Done Sent false
3676 1819 Acked Read 5 Done Done Sent false
3677 1820 Acked Read 4 Done Done Sent false
3678 1821 Acked Write 5 Done Done Sent false
3679 1822 Acked Write 9 Done Done Sent false
3680 1823 Acked Write 8 Done Done Sent false
3681 1824 Acked Read 10 Done Done Sent false
3682 1825 Acked Write 8 Done Done Sent false
3683 1826 Acked Read 9 Done Done Sent false
3684 1827 Acked Flush 0 Done Done Sent false
3685 1828 Acked Write 10 Done Done Sent false
3686 1829 Acked Flush 0 Done Done Sent false
3687 1830 Acked Read 3 Done Done Sent false
3688 1831 Acked Write 7 Done Done Sent false
3689 1832 Acked Read 7 Done Done Sent false
3690 1833 Acked Write 5 Done Done Sent false
3691 1834 Acked Read 9 Done Done Sent false
3692 1835 Acked Write 9 Done Done Sent false
3693 1836 Acked Flush 0 Done Done Sent false
3694 1837 Acked Flush 0 Done Done Sent false
3695 1838 Acked Write 7 Done Done Sent false
3696 1839 Acked Write 5 Done Done Sent false
3697 1840 Acked Read 2 Done Done Sent false
3698 1841 Acked Read 5 Done Done Sent false
3699 1842 Acked Read 10 Done Done Sent false
3700 1843 Acked Read 5 Done Done Sent false
3701 1844 Acked Read 5 Done Done Sent false
3702 1845 Acked Read 7 Done Done Sent false
3703 1846 Acked Read 4 Done Done Sent false
3704 1847 Acked Read 1 Done Done Sent false
3705 1848 Acked Read 1 Done Done Sent false
3706 1849 Acked Flush 0 Done Done Sent false
3707 1850 Acked Write 8 Done Done Sent false
3708 1851 Acked Read 3 Done Done Sent false
3709 1852 Acked Write 7 Done Done Sent false
3710 1853 Acked Read 8 Done Done Sent false
3711 1854 Acked Write 6 Done Done Sent false
3712 1855 Acked Read 6 Done Done Sent false
3713 1856 Acked Write 3 Done Done Sent false
3714 1857 Acked Read 2 Done Done Sent false
3715 1858 Acked Flush 0 Done Done Sent false
3716 1859 Acked Read 10 Done Done Sent false
3717 1860 Acked Read 5 Done Done Sent false
3718 1861 Acked Read 4 Done Done Sent false
3719 1862 Acked Write 8 Done Done Sent false
3720 1863 Acked Write 6 Done Done Sent false
3721 1864 Acked Write 2 Done Done Sent false
3722 1865 Acked Read 4 Done Done Sent false
3723 1866 Acked Read 4 Done Done Sent false
3724 1867 Acked Read 2 Done Done Sent false
3725 1868 Acked Flush 0 Done Done Sent false
3726 1869 Acked Read 9 Done Done Sent false
3727 1870 Acked Write 7 Done Done Sent false
3728 1871 Acked Write 9 Done Done Sent false
3729 1872 Acked Write 8 Done Done Sent false
3730 1873 Acked Read 10 Done Done Sent false
3731 1874 Acked Flush 0 Done Done Sent false
3732 1875 Acked Flush 0 Done Done Sent false
3733 1876 Acked Write 1 Done Done Sent false
3734 1877 Acked Write 2 Done Done Sent false
3735 1878 Acked Write 5 Done Done Sent false
3736 1879 Acked Write 4 Done Done Sent false
3737 1880 Acked Flush 0 Done Done Sent false
3738 1881 Acked Write 10 Done Done Sent false
3739 1882 Acked Write 8 Done Done Sent false
3740 1883 Acked Write 8 Done Done Sent false
3741 1884 Acked Write 2 Done Done Sent false
3742 1885 Acked Write 1 Done Done Sent false
3743 1886 Acked Flush 0 Done Done Sent false
3744 1887 Acked Write 10 Done Done Sent false
3745 1888 Acked Read 6 Done Done Sent false
3746 1889 Acked Write 2 Done Done Sent false
3747 1890 Acked Write 1 Done Done Sent false
3748 1891 Acked Read 7 Done Done Sent false
3749 1892 Acked Flush 0 Done Done Sent false
3750 1893 Acked Read 3 Done Done Sent false
3751 1894 Acked Write 2 Done Done Sent false
3752 1895 Acked Read 7 Done Done Sent false
3753 1896 Acked Flush 0 Done Done Sent false
3754 1897 Acked Read 8 Done Done Sent false
3755 1898 Acked Read 1 Done Done Sent false
3756 1899 Acked Flush 0 Done Done Sent false
3757 1900 Acked Read 9 Done Done Sent false
3758 1901 Acked Read 8 Done Done Sent false
3759 1902 Acked Flush 0 Done Done Sent false
3760 1903 Acked Read 8 Done Done Sent false
3761 1904 Acked Read 10 Done Done Sent false
3762 1905 Acked Flush 0 Done Done Sent false
3763 STATES DS:0 DS:1 DS:2 TOTAL
3764 Sent 0 0 302 302
3765 Done 302 302 0 604
3766 Skipped 0 0 0 0
3767 Error 0 0 0 0
3768 Last Flush: 1905 1905 1603
3769 Downstairs last five completed: 1603 1602 1601 1600 1599
3770 Upstairs last five completed: 1905 1904 1903 1902 1901
3771 CLIENT: Up:0 ds:302 act:2
37722025-01-11T06:06:12.971ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37732025-01-11T06:06:12.971ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39300) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37742025-01-11T06:06:12.972ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37752025-01-11T06:06:12.973ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37762025-01-11T06:06:12.973ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37772025-01-11T06:06:12.973ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37782025-01-11T06:06:12.973ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37792025-01-11T06:06:12.973ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37802025-01-11T06:06:12.973ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37812025-01-11T06:06:12.973ZINFOcrucible: [2] client re-new 302 jobs since flush 1603 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37822025-01-11T06:06:12.973ZINFOcrucible: [2] Replayed 302 jobs since flush: 1603 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
3783 ----------------------------------------------------------------
3784 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3785 Downstairs last five completed: 1905 1904 1903 1902 1901
3786 Upstairs last five completed: 1905 1904 1903 1902 1901
3787 CLIENT: Up:0 ds:0 act:3
3788 Replay: All jobs finished
3789 Main task has work to do, go find it
3790 got dsc Stop(1)
3791 stop 1
3792 Jan 11 06:06:15.213 INFO request completed, latency_us: 322, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 7dcf8c25-fb63-4d86-a885-89a13fc99381, remote_addr:[8820] Got stop action so:false kr:false
3793 127.0.0.1:59925, local_addr: 127.0.0.1:9998
37942025-01-11T06:06:15.214ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37952025-01-11T06:06:15.214ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
37962025-01-11T06:06:15.214ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
3797 Jan 11 06:06:15.214 INFO request completed{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:06:15.214680473Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
3798 , latency_us: 110, response_code: 200, uri: /state/cid/1, method:{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:06:15.214693771Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
3799 GET, req_id: 2ea1b236-3365-4807-bdbb-f29c2b0cc2c7, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
38002025-01-11T06:06:15.214ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
3801 [8820] Exited with: ExitStatus(unix_wait_status(9))
3802 [8820][1] reports Exit
3803 Jan 11 06:06:19.217 INFO request completed, latency_us: 322, response_code: 200, uri: /state/cid/1, method: GET, req_id: ec3df2f2-479c-4074-bedc-8be50f0cb43f, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
3804 001/300 Write block 3842 len 4096 data: 2
3805 002/300 Read block 2737 len 24576
3806 003/300 Write block 4 len 20480 data: 2 2 2 2 2
3807 004/300 Write block 1271 len 20480 data: 2 2 2 2 2
3808 005/300 Flush
3809 006/300 Flush
3810 007/300 Write block 4631 len 20480 data: 2 2 2 2 2
3811 008/300 Read block 4307 len 36864
3812 009/300 Read block 4958 len 32768
3813 010/300 Write block 1531 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3814 011/300 Write block 879 len 40960 data: 4 4 4 4 3 3 2 2 2 2
3815 012/300 Write block 4861 len 4096 data: 3
3816 013/300 Write block 94 len 36864 data: 3 3 2 2 2 2 2 2 2
3817 014/300 Read block 2653 len 8192
3818 015/300 Write block 883 len 36864 data: 4 4 3 3 3 3 2 2 2
3819 016/300 Read block 1730 len 36864
3820 017/300 Read block 3079 len 12288
3821 018/300 Read block 2847 len 32768
3822 019/300 Flush
3823 020/300 Read block 1201 len 16384
3824 021/300 Read block 2264 len 36864
3825 022/300 Read block 2508 len 8192
3826 023/300 Read block 860 len 8192
3827 024/300 Read block 646 len 40960
3828 025/300 Write block 3630 len 16384 data: 2 2 2 2
3829 026/300 Read block 3079 len 36864
3830 027/300 Read block 3520 len 8192
3831 028/300 Write block 3871 len 28672 data: 3 3 3 3 3 3 3
3832 029/300 Read block 2222 len 40960
3833 030/300 Write block 3037 len 4096 data: 2
3834 031/300 Write block 2817 len 32768 data: 2 3 3 3 3 3 3 3
3835 032/300 Write block 1736 len 36864 data: 2 2 2 2 2 2 3 3 3
3836 033/300 Read block 4496 len 16384
3837 034/300 Write block 4128 len 4096 data: 3
3838 035/300 Read block 737 len 28672
3839 036/300 Read block 622 len 4096
3840 037/300 Write block 807 len 32768 data: 2 2 2 2 2 2 2 2
3841 038/300 Write block 2197 len 16384 data: 2 2 2 2
3842 039/300 Write block 4465 len 20480 data: 5 5 5 4 4
3843 040/300 Write block 464 len 4096 data: 2
3844 041/300 Read block 3808 len 12288
3845 042/300 Write block 4618 len 20480 data: 2 2 2 2 2
3846 043/300 Read block 3645 len 24576
3847 044/300 Flush
3848 045/300 Write block 2921 len 16384 data: 2 2 3 3
3849 046/300 Flush
3850 047/300 Read block 4297 len 4096
3851 048/300 Read block 3082 len 12288
3852 049/300 Write block 4211 len 16384 data: 2 2 2 2
3853 050/300 Flush
3854 051/300 Write block 655 len 20480 data: 2 2 2 2 2
3855 052/300 Write block 49 len 24576 data: 2 2 2 2 2 2
3856 053/300 Read block 2352 len 16384
3857 054/300 Read block 1734 len 32768
3858 055/300 Read block 82 len 8192
3859 056/300 Read block 2117 len 20480
3860 057/300 Write block 2245 len 20480 data: 2 2 2 2 2
3861 058/300 Write block 2975 len 4096 data: 2
3862 059/300 Read block 3575 len 32768
3863 060/300 Write block 2933 len 28672 data: 2 2 2 2 2 2 2
3864 061/300 Write block 4308 len 24576 data: 2 2 2 3 3 2
3865 062/300 Read block 2206 len 12288
3866 063/300 Flush
3867 064/300 Flush
3868 065/300 Read block 3702 len 12288
3869 066/300 Write block 3696 len 24576 data: 3 2 2 2 2 2
3870 067/300 Read block 3259 len 36864
3871 068/300 Write block 1112 len 40960 data: 2 2 2 2 2 2 3 3 3 3
3872 069/300 Write block 3909 len 16384 data: 2 2 2 2
3873 070/300 Read block 3028 len 40960
3874 071/300 Write block 4881 len 4096 data: 3
3875 072/300 Read block 3239 len 16384
3876 073/300 Read block 1865 len 28672
3877 074/300 Read block 62 len 32768
3878 075/300 Read block 1733 len 36864
3879 076/300 Read block 537 len 28672
3880 077/300 Write block 4949 len 20480 data: 3 4 4 4 4
3881 078/300 Flush
3882 079/300 Flush
3883 080/300 Write block 1468 len 32768 data: 2 2 2 3 2 2 2 2
3884 081/300 Read block 2516 len 8192
3885 082/300 Flush
3886 083/300 Write block 1578 len 20480 data: 2 2 2 2 2
3887 084/300 Flush
3888 085/300 Write block 2642 len 20480 data: 5 4 4 3 3
3889 086/300 Read block 1397 len 16384
3890 087/300 Write block 4583 len 28672 data: 2 2 3 3 3 2 2
3891 088/300 Write block 396 len 40960 data: 2 2 2 2 2 2 3 3 3 3
3892 089/300 Write block 3014 len 4096 data: 2
3893 090/300 Read block 4775 len 16384
3894 091/300 Write block 2977 len 16384 data: 2 2 2 2
3895 092/300 Read block 2770 len 24576
3896 093/300 Read block 1064 len 16384
3897 094/300 Write block 4474 len 36864 data: 2 2 2 2 2 2 2 2 2
3898 095/300 Read block 2878 len 20480
3899 096/300 Write block 2183 len 28672 data: 3 3 2 2 2 2 2
3900 097/300 Write block 322 len 20480 data: 2 3 3 4 4
3901 098/300 Read block 4102 len 32768
3902 099/300 Read block 4956 len 24576
3903 100/300 Read block 2609 len 16384
3904 101/300 Write block 2284 len 36864 data: 3 3 3 3 3 2 3 3 3
3905 102/300 Write block 3818 len 36864 data: 2 2 2 2 2 2 2 2 2
3906 103/300 Write block 4988 len 36864 data: 3 3 3 3 3 2 2 2 2
3907 104/300 Read block 4485 len 16384
3908 105/300 Read block 4490 len 28672
3909 106/300 Write block 4651 len 28672 data: 2 2 2 2 2 2 2
3910 107/300 Write block 773 len 4096 data: 3
3911 108/300 Read block 1024 len 28672
3912 109/300 Read block 74 len 28672
3913 110/300 Write block 2042 len 36864 data: 2 2 2 2 2 2 2 2 2
3914 111/300 Read block 3954 len 28672
3915 112/300 Write block 3340 len 8192 data: 2 2
3916 113/300 Read block 1831 len 24576
3917 114/300 Read block 2519 len 12288
3918 115/300 Write block 282 len 20480 data: 3 3 3 3 3
3919 116/300 Read block 3554 len 24576
3920 117/300 Write block 3405 len 36864 data: 2 2 2 2 2 2 2 2 2
3921 118/300 Write block 355 len 36864 data: 2 2 2 2 2 2 2 2 2
3922 119/300 Write block 441 len 12288 data: 2 2 2
3923 120/300 Read block 259 len 28672
3924 121/300 Write block 4817 len 4096 data: 2
3925 122/300 Read block 1071 len 12288
3926 123/300 Write block 2509 len 20480 data: 3 3 3 4 4
3927 124/300 Write block 4425 len 28672 data: 2 2 2 2 2 2 2
3928 125/300 Write block 1914 len 4096 data: 2
3929 126/300 Read block 2996 len 24576
3930 127/300 Read block 2998 len 40960
3931 128/300 Read block 4124 len 20480
3932 129/300 Write block 2201 len 32768 data: 2 2 2 2 2 2 2 2
3933 130/300 Write block 3154 len 24576 data: 2 2 2 2 2 2
3934 131/300 Flush
3935 132/300 Write block 4746 len 36864 data: 2 2 2 2 2 2 2 2 2
3936 133/300 Read block 673 len 16384
3937 134/300 Read block 2883 len 4096
3938 135/300 Read block 4966 len 8192
3939 136/300 Read block 645 len 20480
3940 137/300 Read block 605 len 16384
3941 138/300 Read block 2825 len 40960
3942 139/300 Write block 3471 len 16384 data: 2 2 3 3
3943 140/300 Write block 2701 len 16384 data: 2 2 2 3
3944 141/300 Read block 2879 len 40960
3945 142/300 Write block 650 len 16384 data: 2 2 2 2
3946 143/300 Write block 4144 len 32768 data: 4 3 3 2 2 2 2 2
3947 144/300 Read block 2087 len 32768
3948 145/300 Read block 2906 len 16384
3949 146/300 Flush
3950 147/300 Flush
3951 148/300 Write block 1842 len 40960 data: 2 2 2 2 3 3 3 3 4 3
3952 149/300 Write block 2562 len 36864 data: 2 2 2 2 2 2 2 2 2
3953 150/300 Write block 2552 len 32768 data: 2 2 2 2 2 2 2 2
3954 151/300 Read block 297 len 8192
3955 152/300 Read block 1772 len 32768
3956 153/300 Read block 3273 len 40960
3957 154/300 Write block 3591 len 32768 data: 3 3 3 4 4 4 4 4
3958 155/300 Write block 4986 len 8192 data: 3 3
3959 156/300 Read block 4244 len 28672
3960 157/300 Read block 3262 len 20480
3961 158/300 Write block 1871 len 12288 data: 3 3 3
3962 159/300 Read block 4056 len 4096
3963 160/300 Read block 758 len 28672
3964 161/300 Write block 1413 len 16384 data: 2 2 2 2
3965 162/300 Read block 4832 len 16384
3966 163/300 Flush
3967 164/300 Read block 4508 len 28672
3968 165/300 Write block 2857 len 40960 data: 2 2 2 2 2 3 3 3 3 3
3969 166/300 Read block 81 len 36864
3970 167/300 Read block 3291 len 12288
3971 168/300 Read block 1081 len 40960
3972 169/300 Read block 325 len 8192
3973 170/300 Read block 3253 len 12288
3974 171/300 Read block 3884 len 36864
3975 172/300 Read block 2342 len 40960
3976 173/300 Write block 200 len 28672 data: 2 2 2 2 2 2 2
3977 174/300 Write block 4662 len 40960 data: 2 2 2 2 3 3 3 3 3 4
3978 175/300 Write block 2197 len 12288 data: 3 3 3
3979 176/300 Write block 1462 len 20480 data: 2 2 2 2 2
3980 177/300 Write block 4975 len 28672 data: 2 2 2 2 2 2 2
3981 178/300 Write block 2430 len 8192 data: 3 3
3982 179/300 Read block 3009 len 24576
3983 180/300 Read block 4512 len 40960
3984 181/300 Write block 1900 len 36864 data: 3 2 2 2 2 2 2 2 2
3985 182/300 Write block 3418 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3986 183/300 Read block 2360 len 32768
3987 184/300 Read block 4088 len 24576
3988 185/300 Write block 1357 len 28672 data: 2 3 3 3 3 3 3
3989 186/300 Write block 2574 len 32768 data: 2 2 2 2 2 2 2 2
3990 187/300 Read block 2145 len 24576
3991 188/300 Read block 1799 len 12288
3992 189/300 Read block 3232 len 40960
3993 190/300 Read block 3757 len 4096
3994 191/300 Write block 4146 len 20480 data: 4 3 3 3 3
3995 192/300 Read block 4774 len 16384
3996 193/300 Read block 3953 len 32768
3997 194/300 Flush
3998 195/300 Read block 3815 len 24576
3999 196/300 Write block 979 len 28672 data: 2 3 3 3 3 3 3
4000 197/300 Read block 230 len 12288
4001 198/300 Read block 405 len 8192
4002 199/300 Read block 2857 len 4096
4003 200/300 Read block 3062 len 4096
4004 201/300 Read block 4318 len 12288
4005 202/300 Write block 291 len 24576 data: 4 3 3 3 3 3
4006 203/300 Read block 4945 len 28672
4007 204/300 Write block 4865 len 24576 data: 2 2 2 2 2 2
4008 205/300 Write block 3499 len 36864 data: 2 2 3 3 3 3 5 5 5
4009 206/300 Write block 1808 len 40960 data: 2 2 2 2 3 3 2 2 2 2
4010 207/300 Write block 4767 len 20480 data: 3 3 3 3 3
4011 208/300 Read block 3791 len 24576
4012 209/300 Read block 3654 len 40960
4013 210/300 Read block 968 len 32768
4014 211/300 Read block 2848 len 4096
4015 212/300 Read block 3922 len 28672
4016 213/300 Write block 2794 len 32768 data: 3 3 3 3 3 3 3 3
4017 214/300 Write block 1521 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4018 215/300 Read block 2039 len 36864
4019 216/300 Write block 4719 len 24576 data: 2 2 2 2 2 2
4020 217/300 Write block 978 len 16384 data: 2 3 4 4
4021 218/300 Read block 3219 len 8192
4022 219/300 Write block 4789 len 32768 data: 2 2 2 2 2 2 2 2
4023 220/300 Write block 2863 len 20480 data: 4 4 4 4 3
4024 221/300 Read block 4261 len 12288
4025 222/300 Write block 3661 len 36864 data: 3 3 3 3 3 3 3 2 2
4026 223/300 Write block 788 len 36864 data: 3 3 3 3 3 3 2 2 2
4027 224/300 Write block 2097 len 36864 data: 5 5 5 3 3 3 3 2 2
4028 225/300 Write block 1620 len 8192 data: 2 3
4029 226/300 Write block 4699 len 4096 data: 5
4030 227/300 Flush
4031 228/300 Flush
4032 229/300 Write block 3960 len 28672 data: 2 2 2 2 2 2 2
4033 230/300 Read block 4679 len 20480
4034 231/300 Read block 2064 len 16384
4035 232/300 Write block 1218 len 32768 data: 2 2 2 2 3 2 2 2
4036 233/300 Write block 542 len 20480 data: 2 2 2 2 2
4037 234/300 Write block 4049 len 16384 data: 3 3 3 3
4038 235/300 Read block 3409 len 32768
4039 236/300 Write block 4562 len 24576 data: 5 5 4 4 4 2
4040 237/300 Read block 1412 len 4096
4041 238/300 Write block 1413 len 16384 data: 3 3 3 3
4042 239/300 Write block 3209 len 8192 data: 3 3
4043 240/300 Write block 1025 len 20480 data: 3 3 3 3 3
4044 241/300 Write block 4065 len 32768 data: 2 2 2 2 2 2 2 2
4045 242/300 Write block 2216 len 28672 data: 2 2 3 3 3 3 3
4046 243/300 Read block 2511 len 8192
4047 244/300 Flush
4048 245/300 Read block 2068 len 24576
4049 246/300 Read block 4791 len 4096
4050 247/300 Read block 4344 len 4096
4051 248/300 Write block 2862 len 12288 data: 4 5 5
4052 249/300 Flush
4053 250/300 Read block 4697 len 32768
4054 251/300 Write block 187 len 16384 data: 3 3 3 2
4055 252/300 Write block 2865 len 36864 data: 5 5 4 3 3 3 3 2 2
4056 253/300 Read block 2549 len 16384
4057 254/300 Read block 4364 len 8192
4058 255/300 Read block 541 len 4096
4059 256/300 Write block 790 len 24576 data: 4 4 4 4 3 3
4060 257/300 Write block 516 len 12288 data: 3 4 4
4061 258/300 Write block 4502 len 36864 data: 2 3 3 2 3 3 3 3 3
4062 259/300 Read block 2093 len 16384
4063 260/300 Read block 2135 len 24576
4064 261/300 Read block 18 len 32768
4065 262/300 Read block 4419 len 24576
4066 263/300 Read block 132 len 28672
4067 264/300 Write block 133 len 4096 data: 2
4068 265/300 Read block 2224 len 8192
4069 266/300 Read block 4707 len 40960
4070 267/300 Read block 3627 len 32768
4071 268/300 Read block 2831 len 40960
4072 269/300 Flush
4073 270/300 Write block 320 len 24576 data: 2 2 3 4 4 5
4074 271/300 Read block 774 len 4096
4075 272/300 Read block 2134 len 12288
4076 273/300 Flush
4077 274/300 Flush
4078 275/300 Write block 1001 len 40960 data: 2 2 2 3 3 3 3 3 3 3
4079 276/300 Write block 1442 len 36864 data: 3 3 3 3 2 2 2 2 2
4080 277/300 Read block 2252 len 28672
4081 278/300 Write block 4647 len 24576 data: 2 2 2 2 3 3
4082 279/300 Read block 3080 len 12288
4083 280/300 Read block 469 len 12288
4084 281/300 Write block 4440 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4085 282/300 Read block 383 len 20480
4086 283/300 Read block 4012 len 8192
4087 284/300 Write block 1497 len 36864 data: 3 3 3 3 3 3 3 2 2
4088 285/300 Write block 578 len 36864 data: 3 3 3 3 3 3 2 2 3
4089 286/300 Flush
4090 287/300 Write block 4774 len 8192 data: 4 3
4091 288/300 Write block 2247 len 36864 data: 3 3 3 2 2 2 2 2 3
4092 289/300 Flush
4093 290/300 Write block 3865 len 16384 data: 2 2 2 2
4094 291/300 Write block 542 len 4096 data: 3
4095 292/300 Read block 4177 len 4096
4096 293/300 Read block 1897 len 32768
4097 294/300 Write block 3567 len 16384 data: 3 2 2 2
4098 295/300 Write block 2098 len 20480 data: 6 6 4 4 4
4099 296/300 Write block 2591 len 24576 data: 2 2 2 2 2 3
4100 297/300 Flush
4101 298/300 Flush
4102 299/300 Read block 4142 len 4096
4103 300/300 Read block 2720 len 32768
4104 301/300 Write block 3100 len 20480 data: 4 4 4 4 3
4105 Main task has work to do, go find it
4106 got dsc Start(1)
4107 start 1
4108 Jan 11 06:06:19.451 INFO request completed, latency_us: 263, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7cd0a57b-9cab-45e3-b76f-8974ec16f18c, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
4109 Replay: started 1, returned:Ok(())
4110 ----------------------------------------------------------------
4111 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4112 States: Active Offline Active
4113 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4114 1906 Acked Write 1 Done Sent Done false
4115 1907 Acked Read 6 Done Sent Done false
4116 1908 Acked Write 5 Done Sent Done false
4117 1909 Acked Write 5 Done Sent Done false
4118 1910 Acked Flush 0 Done Sent Done false
4119 1911 Acked Flush 0 Done Sent Done false
4120 1912 Acked Write 5 Done Sent Done false
4121 1913 Acked Read 9 Done Sent Done false
4122 1914 Acked Read 8 Done Sent Done false
4123 1915 Acked Write 10 Done Sent Done false
4124 1916 Acked Write 10 Done Sent Done false
4125 1917 Acked Write 1 Done Sent Done false
4126 1918 Acked Write 9 Done Sent Done false
4127 1919 Acked Read 2 Done Sent Done false
4128 1920 Acked Write 9 Done Sent Done false
4129 1921 Acked Read 9 Done Sent Done false
4130 1922 Acked Read 3 Done Sent Done false
4131 1923 Acked Read 8 Done Sent Done false
4132 1924 Acked Flush 0 Done Sent Done false
4133 1925 Acked Read 4 Done Sent Done false
4134 1926 Acked Read 9 Done Sent Done false
4135 1927 Acked Read 2 Done Sent Done false
4136 1928 Acked Read 2 Done Sent Done false
4137 1929 Acked Read 10 Done Sent Done false
4138 1930 Acked Write 4 Done Sent Done false
4139 1931 Acked Read 9 Done Sent Done false
4140 1932 Acked Read 2 Done Sent Done false
4141 1933 Acked Write 7 Done Sent Done false
4142 1934 Acked Read 10 Done Sent Done false
4143 1935 Acked Write 1 Done Sent Done false
4144 1936 Acked Write 8 Done Sent Done false
4145 1937 Acked Write 9 Done Sent Done false
4146 1938 Acked Read 4 Done Sent Done false
4147 1939 Acked Write 1 Done Sent Done false
4148 1940 Acked Read 7 Done Sent Done false
4149 1941 Acked Read 1 Done Sent Done false
4150 1942 Acked Write 8 Done Sent Done false
4151 1943 Acked Write 4 Done Sent Done false
4152 1944 Acked Write 5 Done Sent Done false
4153 1945 Acked Write 1 Done Sent Done false
4154 1946 Acked Read 3 Done Sent Done false
4155 1947 Acked Write 5 Done Sent Done false
4156 1948 Acked Read 6 Done Sent Done false
4157 1949 Acked Flush 0 Done Sent Done false
4158 1950 Acked Write 4 Done Sent Done false
4159 1951 Acked Flush 0 Done Sent Done false
4160 1952 Acked Read 1 Done Sent Done false
4161 1953 Acked Read 3 Done Sent Done false
4162 1954 Acked Write 4 Done Sent Done false
4163 1955 Acked Flush 0 Done Sent Done false
4164 1956 Acked Write 5 Done Sent Done false
4165 1957 Acked Write 6 Done Sent Done false
4166 1958 Acked Read 4 Done Sent Done false
4167 1959 Acked Read 8 Done Sent Done false
4168 1960 Acked Read 2 Done Sent Done false
4169 1961 Acked Read 5 Done Sent Done false
4170 1962 Acked Write 5 Done Sent Done false
4171 1963 Acked Write 1 Done Sent Done false
4172 1964 Acked Read 8 Done Sent Done false
4173 1965 Acked Write 7 Done Sent Done false
4174 1966 Acked Write 6 Done Sent Done false
4175 1967 Acked Read 3 Done Sent Done false
4176 1968 Acked Flush 0 Done Sent Done false
4177 1969 Acked Flush 0 Done Sent Done false
4178 1970 Acked Read 3 Done Sent Done false
4179 1971 Acked Write 6 Done Sent Done false
4180 1972 Acked Read 9 Done Sent Done false
4181 1973 Acked Write 10 Done Sent Done false
4182 1974 Acked Write 4 Done Sent Done false
4183 1975 Acked Read 10 Done Sent Done false
4184 1976 Acked Write 1 Done Sent Done false
4185 1977 Acked Read 4 Done Sent Done false
4186 1978 Acked Read 7 Done Sent Done false
4187 1979 Acked Read 8 Done Sent Done false
4188 1980 Acked Read 9 Done Sent Done false
4189 1981 Acked Read 7 Done Sent Done false
4190 1982 Acked Write 5 Done Sent Done false
4191 1983 Acked Flush 0 Done Sent Done false
4192 1984 Acked Flush 0 Done Sent Done false
4193 1985 Acked Write 8 Done Sent Done false
4194 1986 Acked Read 2 Done Sent Done false
4195 1987 Acked Flush 0 Done Sent Done false
4196 1988 Acked Write 5 Done Sent Done false
4197 1989 Acked Flush 0 Done Sent Done false
4198 1990 Acked Write 5 Done Sent Done false
4199 1991 Acked Read 4 Done Sent Done false
4200 1992 Acked Write 7 Done Sent Done false
4201 1993 Acked Write 10 Done Sent Done false
4202 1994 Acked Write 1 Done Sent Done false
4203 1995 Acked Read 4 Done Sent Done false
4204 1996 Acked Write 4 Done Sent Done false
4205 1997 Acked Read 6 Done Sent Done false
4206 1998 Acked Read 4 Done Sent Done false
4207 1999 Acked Write 9 Done Sent Done false
4208 2000 Acked Read 5 Done Sent Done false
4209 2001 Acked Write 7 Done Sent Done false
4210 2002 Acked Write 5 Done Sent Done false
4211 2003 Acked Read 8 Done Sent Done false
4212 2004 Acked Read 6 Done Sent Done false
4213 2005 Acked Read 4 Done Sent Done false
4214 2006 Acked Write 9 Done Sent Done false
4215 2007 Acked Write 9 Done Sent Done false
4216 2008 Acked Write 9 Done Sent Done false
4217 2009 Acked Read 4 Done Sent Done false
4218 2010 Acked Read 7 Done Sent Done false
4219 2011 Acked Write 7 Done Sent Done false
4220 2012 Acked Write 1 Done Sent Done false
4221 2013 Acked Read 7 Done Sent Done false
4222 2014 Acked Read 7 Done Sent Done false
4223 2015 Acked Write 9 Done Sent Done false
4224 2016 Acked Read 7 Done Sent Done false
4225 2017 Acked Write 2 Done Sent Done false
4226 2018 Acked Read 6 Done Sent Done false
4227 2019 Acked Read 3 Done Sent Done false
4228 2020 Acked Write 5 Done Sent Done false
4229 2021 Acked Read 6 Done Sent Done false
4230 2022 Acked Write 9 Done Sent Done false
4231 2023 Acked Write 9 Done Sent Done false
4232 2024 Acked Write 3 Done Sent Done false
4233 2025 Acked Read 7 Done Sent Done false
4234 2026 Acked Write 1 Done Sent Done false
4235 2027 Acked Read 3 Done Sent Done false
4236 2028 Acked Write 5 Done Sent Done false
4237 2029 Acked Write 7 Done Sent Done false
4238 2030 Acked Write 1 Done Sent Done false
4239 2031 Acked Read 6 Done Sent Done false
4240 2032 Acked Read 10 Done Sent Done false
4241 2033 Acked Read 5 Done Sent Done false
4242 2034 Acked Write 8 Done Sent Done false
4243 2035 Acked Write 6 Done Sent Done false
4244 2036 Acked Flush 0 Done Sent Done false
4245 2037 Acked Write 9 Done Sent Done false
4246 2038 Acked Read 4 Done Sent Done false
4247 2039 Acked Read 1 Done Sent Done false
4248 2040 Acked Read 2 Done Sent Done false
4249 2041 Acked Read 5 Done Sent Done false
4250 2042 Acked Read 4 Done Sent Done false
4251 2043 Acked Read 10 Done Sent Done false
4252 2044 Acked Write 4 Done Sent Done false
4253 2045 Acked Write 4 Done Sent Done false
4254 2046 Acked Read 10 Done Sent Done false
4255 2047 Acked Write 4 Done Sent Done false
4256 2048 Acked Write 8 Done Sent Done false
4257 2049 Acked Read 8 Done Sent Done false
4258 2050 Acked Read 4 Done Sent Done false
4259 2051 Acked Flush 0 Done Sent Done false
4260 2052 Acked Flush 0 Done Sent Done false
4261 2053 Acked Write 10 Done Sent Done false
4262 2054 Acked Write 9 Done Sent Done false
4263 2055 Acked Write 8 Done Sent Done false
4264 2056 Acked Read 2 Done Sent Done false
4265 2057 Acked Read 8 Done Sent Done false
4266 2058 Acked Read 10 Done Sent Done false
4267 2059 Acked Write 8 Done Sent Done false
4268 2060 Acked Write 2 Done Sent Done false
4269 2061 Acked Read 7 Done Sent Done false
4270 2062 Acked Read 5 Done Sent Done false
4271 2063 Acked Write 3 Done Sent Done false
4272 2064 Acked Read 1 Done Sent Done false
4273 2065 Acked Read 7 Done Sent Done false
4274 2066 Acked Write 4 Done Sent Done false
4275 2067 Acked Read 4 Done Sent Done false
4276 2068 Acked Flush 0 Done Sent Done false
4277 2069 Acked Read 7 Done Sent Done false
4278 2070 Acked Write 10 Done Sent Done false
4279 2071 Acked Read 9 Done Sent Done false
4280 2072 Acked Read 3 Done Sent Done false
4281 2073 Acked Read 10 Done Sent Done false
4282 2074 Acked Read 2 Done Sent Done false
4283 2075 Acked Read 3 Done Sent Done false
4284 2076 Acked Read 9 Done Sent Done false
4285 2077 Acked Read 10 Done Sent Done false
4286 2078 Acked Write 7 Done Sent Done false
4287 2079 Acked Write 10 Done Sent Done false
4288 2080 Acked Write 3 Done Sent Done false
4289 2081 Acked Write 5 Done Sent Done false
4290 2082 Acked Write 7 Done Sent Done false
4291 2083 Acked Write 2 Done Sent Done false
4292 2084 Acked Read 6 Done Sent Done false
4293 2085 Acked Read 10 Done Sent Done false
4294 2086 Acked Write 9 Done Sent Done false
4295 2087 Acked Write 10 Done Sent Done false
4296 2088 Acked Read 8 Done Sent Done false
4297 2089 Acked Read 6 Done Sent Done false
4298 2090 Acked Write 7 Done Sent Done false
4299 2091 Acked Write 8 Done Sent Done false
4300 2092 Acked Read 6 Done Sent Done false
4301 2093 Acked Read 3 Done Sent Done false
4302 2094 Acked Read 10 Done Sent Done false
4303 2095 Acked Read 1 Done Sent Done false
4304 2096 Acked Write 5 Done Sent Done false
4305 2097 Acked Read 4 Done Sent Done false
4306 2098 Acked Read 8 Done Sent Done false
4307 2099 Acked Flush 0 Done Sent Done false
4308 2100 Acked Read 6 Done Sent Done false
4309 2101 Acked Write 7 Done Sent Done false
4310 2102 Acked Read 3 Done Sent Done false
4311 2103 Acked Read 2 Done Sent Done false
4312 2104 Acked Read 1 Done Sent Done false
4313 2105 Acked Read 1 Done Sent Done false
4314 2106 Acked Read 3 Done Sent Done false
4315 2107 Acked Write 6 Done Sent Done false
4316 2108 Acked Read 7 Done Sent Done false
4317 2109 Acked Write 6 Done Sent Done false
4318 2110 Acked Write 9 Done Sent Done false
4319 2111 Acked Write 10 Done Sent Done false
4320 2112 Acked Write 5 Done Sent Done false
4321 2113 Acked Read 6 Done Sent Done false
4322 2114 Acked Read 10 Done Sent Done false
4323 2115 Acked Read 8 Done Sent Done false
4324 2116 Acked Read 1 Done Sent Done false
4325 2117 Acked Read 7 Done Sent Done false
4326 2118 Acked Write 8 Done Sent Done false
4327 2119 Acked Write 10 Done Sent Done false
4328 2120 Acked Read 9 Done Sent Done false
4329 2121 Acked Write 6 Done Sent Done false
4330 2122 Acked Write 4 Done Sent Done false
4331 2123 Acked Read 2 Done Sent Done false
4332 2124 Acked Write 8 Done Sent Done false
4333 2125 Acked Write 5 Done Sent Done false
4334 2126 Acked Read 3 Done Sent Done false
4335 2127 Acked Write 9 Done Sent Done false
4336 2128 Acked Write 9 Done Sent Done false
4337 2129 Acked Write 9 Done Sent Done false
4338 2130 Acked Write 2 Done Sent Done false
4339 2131 Acked Write 1 Done Sent Done false
4340 2132 Acked Flush 0 Done Sent Done false
4341 2133 Acked Flush 0 Done Sent Done false
4342 2134 Acked Write 7 Done Sent Done false
4343 2135 Acked Read 5 Done Sent Done false
4344 2136 Acked Read 4 Done Sent Done false
4345 2137 Acked Write 8 Done Sent Done false
4346 2138 Acked Write 5 Done Sent Done false
4347 2139 Acked Write 4 Done Sent Done false
4348 2140 Acked Read 8 Done Sent Done false
4349 2141 Acked Write 6 Done Sent Done false
4350 2142 Acked Read 1 Done Sent Done false
4351 2143 Acked Write 4 Done Sent Done false
4352 2144 Acked Write 2 Done Sent Done false
4353 2145 Acked Write 5 Done Sent Done false
4354 2146 Acked Write 8 Done Sent Done false
4355 2147 Acked Write 7 Done Sent Done false
4356 2148 Acked Read 2 Done Sent Done false
4357 2149 Acked Flush 0 Done Sent Done false
4358 2150 Acked Read 6 Done Sent Done false
4359 2151 Acked Read 1 Done Sent Done false
4360 2152 Acked Read 1 Done Sent Done false
4361 2153 Acked Write 3 Done Sent Done false
4362 2154 Acked Flush 0 Done Sent Done false
4363 2155 Acked Read 8 Done Sent Done false
4364 2156 Acked Write 4 Done Sent Done false
4365 2157 Acked Write 9 Done Sent Done false
4366 2158 Acked Read 4 Done Sent Done false
4367 2159 Acked Read 2 Done Sent Done false
4368 2160 Acked Read 1 Done Sent Done false
4369 2161 Acked Write 6 Done Sent Done false
4370 2162 Acked Write 3 Done Sent Done false
4371 2163 Acked Write 9 Done Sent Done false
4372 2164 Acked Read 4 Done Sent Done false
4373 2165 Acked Read 6 Done Sent Done false
4374 2166 Acked Read 8 Done Sent Done false
4375 2167 Acked Read 6 Done Sent Done false
4376 2168 Acked Read 7 Done Sent Done false
4377 2169 Acked Write 1 Done Sent Done false
4378 2170 Acked Read 2 Done Sent Done false
4379 2171 Acked Read 10 Done Sent Done false
4380 2172 Acked Read 8 Done Sent Done false
4381 2173 Acked Read 10 Done Sent Done false
4382 2174 Acked Flush 0 Done Sent Done false
4383 2175 Acked Write 6 Done Sent Done false
4384 2176 Acked Read 1 Done Sent Done false
4385 2177 Acked Read 3 Done Sent Done false
4386 2178 Acked Flush 0 Done Sent Done false
4387 2179 Acked Flush 0 Done Sent Done false
4388 2180 Acked Write 10 Done Sent Done false
4389 2181 Acked Write 9 Done Sent Done false
4390 2182 Acked Read 7 Done Sent Done false
4391 2183 Acked Write 6 Done Sent Done false
4392 2184 Acked Read 3 Done Sent Done false
4393 2185 Acked Read 3 Done Sent Done false
4394 2186 Acked Write 10 Done Sent Done false
4395 2187 Acked Read 5 Done Sent Done false
4396 2188 Acked Read 2 Done Sent Done false
4397 2189 Acked Write 9 Done Sent Done false
4398 2190 Acked Write 9 Done Sent Done false
4399 2191 Acked Flush 0 Done Sent Done false
4400 2192 Acked Write 2 Done Sent Done false
4401 2193 Acked Write 9 Done Sent Done false
4402 2194 Acked Flush 0 Done Sent Done false
4403 2195 Acked Write 4 Done Sent Done false
4404 2196 Acked Write 1 Done Sent Done false
4405 2197 Acked Read 1 Done Sent Done false
4406 2198 Acked Read 8 Done Sent Done false
4407 2199 Acked Write 4 Done Sent Done false
4408 2200 Acked Write 5 Done Sent Done false
4409 2201 Acked Write 6 Done Sent Done false
4410 2202 Acked Flush 0 Done Sent Done false
4411 2203 Acked Flush 0 Done Sent Done false
4412 2204 Acked Read 1 Done Sent Done false
4413 2205 Acked Read 8 Done Sent Done false
4414 2206 Acked Write 5 Done Sent Done false
4415 STATES DS:0 DS:1 DS:2 TOTAL
4416 Sent 0 301 0 301
4417 Done 301 0 301 602
4418 Skipped 0 0 0 0
4419 Error 0 0 0 0
4420 Last Flush: 2203 1905 2203
4421 Downstairs last five completed: 1905 1904 1903 1902 1901
4422 Upstairs last five completed: 2206 2205 2204 2203 2202
4423 CLIENT: Up:0 ds:301 act:2
4424 [8820] got start action
4425 [8820] I am going to restart
4426 Starting downstairs at port 8820
4427 Make output file at "/tmp/dsc/downstairs-8820.txt"
4428 [8820][1] reports Starting
4429 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1208)
4430 [8820][1] reports Running
4431 ----------------------------------------------------------------
4432 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4433 States: Active Offline Active
4434 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4435 1906 Acked Write 1 Done Sent Done false
4436 1907 Acked Read 6 Done Sent Done false
4437 1908 Acked Write 5 Done Sent Done false
4438 1909 Acked Write 5 Done Sent Done false
4439 1910 Acked Flush 0 Done Sent Done false
4440 1911 Acked Flush 0 Done Sent Done false
4441 1912 Acked Write 5 Done Sent Done false
4442 1913 Acked Read 9 Done Sent Done false
4443 1914 Acked Read 8 Done Sent Done false
4444 1915 Acked Write 10 Done Sent Done false
4445 1916 Acked Write 10 Done Sent Done false
4446 1917 Acked Write 1 Done Sent Done false
4447 1918 Acked Write 9 Done Sent Done false
4448 1919 Acked Read 2 Done Sent Done false
4449 1920 Acked Write 9 Done Sent Done false
4450 1921 Acked Read 9 Done Sent Done false
4451 1922 Acked Read 3 Done Sent Done false
4452 1923 Acked Read 8 Done Sent Done false
4453 1924 Acked Flush 0 Done Sent Done false
4454 1925 Acked Read 4 Done Sent Done false
4455 1926 Acked Read 9 Done Sent Done false
4456 1927 Acked Read 2 Done Sent Done false
4457 1928 Acked Read 2 Done Sent Done false
4458 1929 Acked Read 10 Done Sent Done false
4459 1930 Acked Write 4 Done Sent Done false
4460 1931 Acked Read 9 Done Sent Done false
4461 1932 Acked Read 2 Done Sent Done false
4462 1933 Acked Write 7 Done Sent Done false
4463 1934 Acked Read 10 Done Sent Done false
4464 1935 Acked Write 1 Done Sent Done false
4465 1936 Acked Write 8 Done Sent Done false
4466 1937 Acked Write 9 Done Sent Done false
4467 1938 Acked Read 4 Done Sent Done false
4468 1939 Acked Write 1 Done Sent Done false
4469 1940 Acked Read 7 Done Sent Done false
4470 1941 Acked Read 1 Done Sent Done false
4471 1942 Acked Write 8 Done Sent Done false
4472 1943 Acked Write 4 Done Sent Done false
4473 1944 Acked Write 5 Done Sent Done false
4474 1945 Acked Write 1 Done Sent Done false
4475 1946 Acked Read 3 Done Sent Done false
4476 1947 Acked Write 5 Done Sent Done false
4477 1948 Acked Read 6 Done Sent Done false
4478 1949 Acked Flush 0 Done Sent Done false
4479 1950 Acked Write 4 Done Sent Done false
4480 1951 Acked Flush 0 Done Sent Done false
4481 1952 Acked Read 1 Done Sent Done false
4482 1953 Acked Read 3 Done Sent Done false
4483 1954 Acked Write 4 Done Sent Done false
4484 1955 Acked Flush 0 Done Sent Done false
4485 1956 Acked Write 5 Done Sent Done false
4486 1957 Acked Write 6 Done Sent Done false
4487 1958 Acked Read 4 Done Sent Done false
4488 1959 Acked Read 8 Done Sent Done false
4489 1960 Acked Read 2 Done Sent Done false
4490 1961 Acked Read 5 Done Sent Done false
4491 1962 Acked Write 5 Done Sent Done false
4492 1963 Acked Write 1 Done Sent Done false
4493 1964 Acked Read 8 Done Sent Done false
4494 1965 Acked Write 7 Done Sent Done false
4495 1966 Acked Write 6 Done Sent Done false
4496 1967 Acked Read 3 Done Sent Done false
4497 1968 Acked Flush 0 Done Sent Done false
4498 1969 Acked Flush 0 Done Sent Done false
4499 1970 Acked Read 3 Done Sent Done false
4500 1971 Acked Write 6 Done Sent Done false
4501 1972 Acked Read 9 Done Sent Done false
4502 1973 Acked Write 10 Done Sent Done false
4503 1974 Acked Write 4 Done Sent Done false
4504 1975 Acked Read 10 Done Sent Done false
4505 1976 Acked Write 1 Done Sent Done false
4506 1977 Acked Read 4 Done Sent Done false
4507 1978 Acked Read 7 Done Sent Done false
4508 1979 Acked Read 8 Done Sent Done false
4509 1980 Acked Read 9 Done Sent Done false
4510 1981 Acked Read 7 Done Sent Done false
4511 1982 Acked Write 5 Done Sent Done false
4512 1983 Acked Flush 0 Done Sent Done false
4513 1984 Acked Flush 0 Done Sent Done false
4514 1985 Acked Write 8 Done Sent Done false
4515 1986 Acked Read 2 Done Sent Done false
4516 1987 Acked Flush 0 Done Sent Done false
4517 1988 Acked Write 5 Done Sent Done false
4518 1989 Acked Flush 0 Done Sent Done false
4519 1990 Acked Write 5 Done Sent Done false
4520 1991 Acked Read 4 Done Sent Done false
4521 1992 Acked Write 7 Done Sent Done false
4522 1993 Acked Write 10 Done Sent Done false
4523 1994 Acked Write 1 Done Sent Done false
4524 1995 Acked Read 4 Done Sent Done false
4525 1996 Acked Write 4 Done Sent Done false
4526 1997 Acked Read 6 Done Sent Done false
4527 1998 Acked Read 4 Done Sent Done false
4528 1999 Acked Write 9 Done Sent Done false
4529 2000 Acked Read 5 Done Sent Done false
4530 2001 Acked Write 7 Done Sent Done false
4531 2002 Acked Write 5 Done Sent Done false
4532 2003 Acked Read 8 Done Sent Done false
4533 2004 Acked Read 6 Done Sent Done false
4534 2005 Acked Read 4 Done Sent Done false
4535 2006 Acked Write 9 Done Sent Done false
4536 2007 Acked Write 9 Done Sent Done false
4537 2008 Acked Write 9 Done Sent Done false
4538 2009 Acked Read 4 Done Sent Done false
4539 2010 Acked Read 7 Done Sent Done false
4540 2011 Acked Write 7 Done Sent Done false
4541 2012 Acked Write 1 Done Sent Done false
4542 2013 Acked Read 7 Done Sent Done false
4543 2014 Acked Read 7 Done Sent Done false
4544 2015 Acked Write 9 Done Sent Done false
4545 2016 Acked Read 7 Done Sent Done false
4546 2017 Acked Write 2 Done Sent Done false
4547 2018 Acked Read 6 Done Sent Done false
4548 2019 Acked Read 3 Done Sent Done false
4549 2020 Acked Write 5 Done Sent Done false
4550 2021 Acked Read 6 Done Sent Done false
4551 2022 Acked Write 9 Done Sent Done false
4552 2023 Acked Write 9 Done Sent Done false
4553 2024 Acked Write 3 Done Sent Done false
4554 2025 Acked Read 7 Done Sent Done false
4555 2026 Acked Write 1 Done Sent Done false
4556 2027 Acked Read 3 Done Sent Done false
4557 2028 Acked Write 5 Done Sent Done false
4558 2029 Acked Write 7 Done Sent Done false
4559 2030 Acked Write 1 Done Sent Done false
4560 2031 Acked Read 6 Done Sent Done false
4561 2032 Acked Read 10 Done Sent Done false
4562 2033 Acked Read 5 Done Sent Done false
4563 2034 Acked Write 8 Done Sent Done false
4564 2035 Acked Write 6 Done Sent Done false
4565 2036 Acked Flush 0 Done Sent Done false
4566 2037 Acked Write 9 Done Sent Done false
4567 2038 Acked Read 4 Done Sent Done false
4568 2039 Acked Read 1 Done Sent Done false
4569 2040 Acked Read 2 Done Sent Done false
4570 2041 Acked Read 5 Done Sent Done false
4571 2042 Acked Read 4 Done Sent Done false
4572 2043 Acked Read 10 Done Sent Done false
4573 2044 Acked Write 4 Done Sent Done false
4574 2045 Acked Write 4 Done Sent Done false
4575 2046 Acked Read 10 Done Sent Done false
4576 2047 Acked Write 4 Done Sent Done false
4577 2048 Acked Write 8 Done Sent Done false
4578 2049 Acked Read 8 Done Sent Done false
4579 2050 Acked Read 4 Done Sent Done false
4580 2051 Acked Flush 0 Done Sent Done false
4581 2052 Acked Flush 0 Done Sent Done false
4582 2053 Acked Write 10 Done Sent Done false
4583 2054 Acked Write 9 Done Sent Done false
4584 2055 Acked Write 8 Done Sent Done false
4585 2056 Acked Read 2 Done Sent Done false
4586 2057 Acked Read 8 Done Sent Done false
4587 2058 Acked Read 10 Done Sent Done false
4588 2059 Acked Write 8 Done Sent Done false
4589 2060 Acked Write 2 Done Sent Done false
4590 2061 Acked Read 7 Done Sent Done false
4591 2062 Acked Read 5 Done Sent Done false
4592 2063 Acked Write 3 Done Sent Done false
4593 2064 Acked Read 1 Done Sent Done false
4594 2065 Acked Read 7 Done Sent Done false
4595 2066 Acked Write 4 Done Sent Done false
4596 2067 Acked Read 4 Done Sent Done false
4597 2068 Acked Flush 0 Done Sent Done false
4598 2069 Acked Read 7 Done Sent Done false
4599 2070 Acked Write 10 Done Sent Done false
4600 2071 Acked Read 9 Done Sent Done false
4601 2072 Acked Read 3 Done Sent Done false
4602 2073 Acked Read 10 Done Sent Done false
4603 2074 Acked Read 2 Done Sent Done false
4604 2075 Acked Read 3 Done Sent Done false
4605 2076 Acked Read 9 Done Sent Done false
4606 2077 Acked Read 10 Done Sent Done false
4607 2078 Acked Write 7 Done Sent Done false
4608 2079 Acked Write 10 Done Sent Done false
4609 2080 Acked Write 3 Done Sent Done false
4610 2081 Acked Write 5 Done Sent Done false
4611 2082 Acked Write 7 Done Sent Done false
4612 2083 Acked Write 2 Done Sent Done false
4613 2084 Acked Read 6 Done Sent Done false
4614 2085 Acked Read 10 Done Sent Done false
4615 2086 Acked Write 9 Done Sent Done false
4616 2087 Acked Write 10 Done Sent Done false
4617 2088 Acked Read 8 Done Sent Done false
4618 2089 Acked Read 6 Done Sent Done false
4619 2090 Acked Write 7 Done Sent Done false
4620 2091 Acked Write 8 Done Sent Done false
4621 2092 Acked Read 6 Done Sent Done false
4622 2093 Acked Read 3 Done Sent Done false
4623 2094 Acked Read 10 Done Sent Done false
4624 2095 Acked Read 1 Done Sent Done false
4625 2096 Acked Write 5 Done Sent Done false
4626 2097 Acked Read 4 Done Sent Done false
4627 2098 Acked Read 8 Done Sent Done false
4628 2099 Acked Flush 0 Done Sent Done false
4629 2100 Acked Read 6 Done Sent Done false
4630 2101 Acked Write 7 Done Sent Done false
4631 2102 Acked Read 3 Done Sent Done false
4632 2103 Acked Read 2 Done Sent Done false
4633 2104 Acked Read 1 Done Sent Done false
4634 2105 Acked Read 1 Done Sent Done false
4635 2106 Acked Read 3 Done Sent Done false
4636 2107 Acked Write 6 Done Sent Done false
4637 2108 Acked Read 7 Done Sent Done false
4638 2109 Acked Write 6 Done Sent Done false
4639 2110 Acked Write 9 Done Sent Done false
4640 2111 Acked Write 10 Done Sent Done false
4641 2112 Acked Write 5 Done Sent Done false
4642 2113 Acked Read 6 Done Sent Done false
4643 2114 Acked Read 10 Done Sent Done false
4644 2115 Acked Read 8 Done Sent Done false
4645 2116 Acked Read 1 Done Sent Done false
4646 2117 Acked Read 7 Done Sent Done false
4647 2118 Acked Write 8 Done Sent Done false
4648 2119 Acked Write 10 Done Sent Done false
4649 2120 Acked Read 9 Done Sent Done false
4650 2121 Acked Write 6 Done Sent Done false
4651 2122 Acked Write 4 Done Sent Done false
4652 2123 Acked Read 2 Done Sent Done false
4653 2124 Acked Write 8 Done Sent Done false
4654 2125 Acked Write 5 Done Sent Done false
4655 2126 Acked Read 3 Done Sent Done false
4656 2127 Acked Write 9 Done Sent Done false
4657 2128 Acked Write 9 Done Sent Done false
4658 2129 Acked Write 9 Done Sent Done false
4659 2130 Acked Write 2 Done Sent Done false
4660 2131 Acked Write 1 Done Sent Done false
4661 2132 Acked Flush 0 Done Sent Done false
4662 2133 Acked Flush 0 Done Sent Done false
4663 2134 Acked Write 7 Done Sent Done false
4664 2135 Acked Read 5 Done Sent Done false
4665 2136 Acked Read 4 Done Sent Done false
4666 2137 Acked Write 8 Done Sent Done false
4667 2138 Acked Write 5 Done Sent Done false
4668 2139 Acked Write 4 Done Sent Done false
4669 2140 Acked Read 8 Done Sent Done false
4670 2141 Acked Write 6 Done Sent Done false
4671 2142 Acked Read 1 Done Sent Done false
4672 2143 Acked Write 4 Done Sent Done false
4673 2144 Acked Write 2 Done Sent Done false
4674 2145 Acked Write 5 Done Sent Done false
4675 2146 Acked Write 8 Done Sent Done false
4676 2147 Acked Write 7 Done Sent Done false
4677 2148 Acked Read 2 Done Sent Done false
4678 2149 Acked Flush 0 Done Sent Done false
4679 2150 Acked Read 6 Done Sent Done false
4680 2151 Acked Read 1 Done Sent Done false
4681 2152 Acked Read 1 Done Sent Done false
4682 2153 Acked Write 3 Done Sent Done false
4683 2154 Acked Flush 0 Done Sent Done false
4684 2155 Acked Read 8 Done Sent Done false
4685 2156 Acked Write 4 Done Sent Done false
4686 2157 Acked Write 9 Done Sent Done false
4687 2158 Acked Read 4 Done Sent Done false
4688 2159 Acked Read 2 Done Sent Done false
4689 2160 Acked Read 1 Done Sent Done false
4690 2161 Acked Write 6 Done Sent Done false
4691 2162 Acked Write 3 Done Sent Done false
4692 2163 Acked Write 9 Done Sent Done false
4693 2164 Acked Read 4 Done Sent Done false
4694 2165 Acked Read 6 Done Sent Done false
4695 2166 Acked Read 8 Done Sent Done false
4696 2167 Acked Read 6 Done Sent Done false
4697 2168 Acked Read 7 Done Sent Done false
4698 2169 Acked Write 1 Done Sent Done false
4699 2170 Acked Read 2 Done Sent Done false
4700 2171 Acked Read 10 Done Sent Done false
4701 2172 Acked Read 8 Done Sent Done false
4702 2173 Acked Read 10 Done Sent Done false
4703 2174 Acked Flush 0 Done Sent Done false
4704 2175 Acked Write 6 Done Sent Done false
4705 2176 Acked Read 1 Done Sent Done false
4706 2177 Acked Read 3 Done Sent Done false
4707 2178 Acked Flush 0 Done Sent Done false
4708 2179 Acked Flush 0 Done Sent Done false
4709 2180 Acked Write 10 Done Sent Done false
4710 2181 Acked Write 9 Done Sent Done false
4711 2182 Acked Read 7 Done Sent Done false
4712 2183 Acked Write 6 Done Sent Done false
4713 2184 Acked Read 3 Done Sent Done false
4714 2185 Acked Read 3 Done Sent Done false
4715 2186 Acked Write 10 Done Sent Done false
4716 2187 Acked Read 5 Done Sent Done false
4717 2188 Acked Read 2 Done Sent Done false
4718 2189 Acked Write 9 Done Sent Done false
4719 2190 Acked Write 9 Done Sent Done false
4720 2191 Acked Flush 0 Done Sent Done false
4721 2192 Acked Write 2 Done Sent Done false
4722 2193 Acked Write 9 Done Sent Done false
4723 2194 Acked Flush 0 Done Sent Done false
4724 2195 Acked Write 4 Done Sent Done false
4725 2196 Acked Write 1 Done Sent Done false
4726 2197 Acked Read 1 Done Sent Done false
4727 2198 Acked Read 8 Done Sent Done false
4728 2199 Acked Write 4 Done Sent Done false
4729 2200 Acked Write 5 Done Sent Done false
4730 2201 Acked Write 6 Done Sent Done false
4731 2202 Acked Flush 0 Done Sent Done false
4732 2203 Acked Flush 0 Done Sent Done false
4733 2204 Acked Read 1 Done Sent Done false
4734 2205 Acked Read 8 Done Sent Done false
4735 2206 Acked Write 5 Done Sent Done false
4736 2207 Acked Flush 0 Done Sent Done false
4737 STATES DS:0 DS:1 DS:2 TOTAL
4738 Sent 0 302 0 302
4739 Done 302 0 302 604
4740 Skipped 0 0 0 0
4741 Error 0 0 0 0
4742 Last Flush: 2207 1905 2207
4743 Downstairs last five completed: 1905 1904 1903 1902 1901
4744 Upstairs last five completed: 2207 2206 2205 2204 2203
4745 CLIENT: Up:0 ds:302 act:2
47462025-01-11T06:06:25.216ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47472025-01-11T06:06:25.216ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41015) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47482025-01-11T06:06:25.217ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47492025-01-11T06:06:25.218ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47502025-01-11T06:06:25.218ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47512025-01-11T06:06:25.218ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47522025-01-11T06:06:25.218ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47532025-01-11T06:06:25.218ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47542025-01-11T06:06:25.218ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47552025-01-11T06:06:25.218ZINFOcrucible: [1] client re-new 302 jobs since flush 1905 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47562025-01-11T06:06:25.219ZINFOcrucible: [1] Replayed 302 jobs since flush: 1905 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
4757 ----------------------------------------------------------------
4758 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4759 Downstairs last five completed: 2207 2206 2205 2204 2203
4760 Upstairs last five completed: 2207 2206 2205 2204 2203
4761 CLIENT: Up:0 ds:0 act:3
4762 Replay: All jobs finished
4763 Main task has work to do, go find it
4764 got dsc Stop(0)
4765 stop 0
4766 Jan 11 06:06:27.457 INFO request completed, latency_us: 374[8810] Got stop action so:false kr:false
4767 , response_code: 204, uri: /stop/cid/0, method: POST, req_id: ffdbbf90-613f-45c5-9a9c-2fb59a204ab7, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
47682025-01-11T06:06:27.457ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47692025-01-11T06:06:27.457ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47702025-01-11T06:06:27.457ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47712025-01-11T06:06:27.457ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
47722025-01-11T06:06:27.457ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
4773 Jan 11 06:06:27.458 INFO request completed, latency_us: 183, response_code: 200, uri: /state/cid/0, method: GET, req_id: 919a2581-e275-4d07-8ec7-762e34b54d98, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
47742025-01-11T06:06:27.458ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
4775 [8810] Exited with: ExitStatus(unix_wait_status(9))
4776 [8810][0] reports Exit
4777 Jan 11 06:06:31.460 INFO request completed, latency_us: 415, response_code: 200, uri: /state/cid/0, method: GET, req_id: 92a301d3-c1aa-4f8f-84ab-9007b9475797, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
4778 001/300 Write block 4990 len 12288 data: 4 4 4
4779 002/300 Read block 548 len 40960
4780 003/300 Flush
4781 004/300 Write block 2118 len 8192 data: 2 2
4782 005/300 Flush
4783 006/300 Read block 1179 len 12288
4784 007/300 Read block 535 len 28672
4785 008/300 Write block 1023 len 32768 data: 2 2 4 4 4 4 4 3
4786 009/300 Write block 3037 len 20480 data: 3 2 2 2 2
4787 010/300 Read block 2179 len 8192
4788 011/300 Read block 4638 len 12288
4789 012/300 Read block 1556 len 12288
4790 013/300 Read block 3397 len 16384
4791 014/300 Write block 321 len 32768 data: 3 4 5 5 6 5 4 4
4792 015/300 Read block 4618 len 12288
4793 016/300 Read block 27 len 12288
4794 017/300 Read block 1923 len 12288
4795 018/300 Write block 4025 len 20480 data: 2 2 2 2 2
4796 019/300 Read block 3167 len 28672
4797 020/300 Read block 186 len 12288
4798 021/300 Write block 3973 len 28672 data: 2 2 2 2 2 2 3
4799 022/300 Write block 1846 len 32768 data: 4 4 4 4 5 4 3 3
4800 023/300 Write block 270 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4801 024/300 Flush
4802 025/300 Write block 2320 len 24576 data: 2 2 2 2 2 2
4803 026/300 Flush
4804 027/300 Read block 1099 len 40960
4805 028/300 Write block 1990 len 16384 data: 2 2 2 4
4806 029/300 Write block 1935 len 36864 data: 2 2 2 2 2 2 2 2 2
4807 030/300 Write block 3409 len 40960 data: 3 3 3 3 3 2 2 2 2 3
4808 031/300 Flush
4809 032/300 Write block 1200 len 24576 data: 2 2 3 3 3 3
4810 033/300 Read block 3263 len 12288
4811 034/300 Write block 1480 len 12288 data: 2 2 3
4812 035/300 Read block 4993 len 20480
4813 036/300 Read block 1328 len 12288
4814 037/300 Write block 1278 len 40960 data: 3 3 3 3 2 2 2 2 2 2
4815 038/300 Flush
4816 039/300 Write block 1242 len 8192 data: 3 2
4817 040/300 Read block 2281 len 28672
4818 041/300 Write block 4766 len 32768 data: 3 4 4 4 4 4 3 4
4819 042/300 Write block 374 len 40960 data: 2 3 3 3 3 3 3 3 3 2
4820 043/300 Read block 4478 len 12288
4821 044/300 Write block 334 len 4096 data: 2
4822 045/300 Write block 4037 len 40960 data: 2 2 2 2 3 3 3 3 3 3
4823 046/300 Write block 2869 len 20480 data: 4 4 4 3 3
4824 047/300 Read block 3902 len 12288
4825 048/300 Read block 3302 len 40960
4826 049/300 Read block 818 len 4096
4827 050/300 Write block 3666 len 28672 data: 4 4 3 3 2 2 3
4828 051/300 Write block 3954 len 4096 data: 2
4829 052/300 Read block 3411 len 8192
4830 053/300 Write block 4074 len 28672 data: 2 2 2 2 2 2 2
4831 054/300 Read block 4648 len 8192
4832 055/300 Write block 49 len 28672 data: 3 3 3 3 3 3 2
4833 056/300 Write block 971 len 4096 data: 2
4834 057/300 Read block 1444 len 16384
4835 058/300 Read block 2092 len 36864
4836 059/300 Read block 397 len 28672
4837 060/300 Read block 2848 len 24576
4838 061/300 Read block 814 len 20480
4839 062/300 Read block 617 len 16384
4840 063/300 Read block 3390 len 40960
4841 064/300 Read block 2156 len 32768
4842 065/300 Write block 648 len 40960 data: 2 2 3 3 3 3 2 3 3 3
4843 066/300 Write block 605 len 8192 data: 2 2
4844 067/300 Read block 2981 len 40960
4845 068/300 Write block 707 len 16384 data: 2 4 4 4
4846 069/300 Write block 2011 len 36864 data: 2 2 2 2 2 2 2 2 2
4847 070/300 Read block 287 len 28672
4848 071/300 Write block 250 len 32768 data: 2 2 2 2 2 2 2 2
4849 072/300 Read block 4864 len 12288
4850 073/300 Flush
4851 074/300 Read block 2229 len 40960
4852 075/300 Read block 2279 len 32768
4853 076/300 Write block 4320 len 12288 data: 3 3 2
4854 077/300 Read block 2343 len 40960
4855 078/300 Read block 2778 len 36864
4856 079/300 Write block 4262 len 32768 data: 2 2 4 3 3 3 3 3
4857 080/300 Read block 3959 len 8192
4858 081/300 Write block 547 len 40960 data: 2 2 3 3 2 2 2 2 2 2
4859 082/300 Flush
4860 083/300 Flush
4861 084/300 Read block 3162 len 12288
4862 085/300 Read block 4853 len 8192
4863 086/300 Read block 4217 len 16384
4864 087/300 Read block 1114 len 28672
4865 088/300 Read block 2069 len 4096
4866 089/300 Read block 33 len 4096
4867 090/300 Flush
4868 091/300 Read block 926 len 24576
4869 092/300 Flush
4870 093/300 Read block 4001 len 16384
4871 094/300 Flush
4872 095/300 Write block 22 len 36864 data: 2 3 3 2 2 2 2 2 2
4873 096/300 Write block 4468 len 24576 data: 5 5 4 3 2 2
4874 097/300 Write block 3457 len 32768 data: 3 4 3 3 3 3 3 2
4875 098/300 Write block 993 len 20480 data: 2 2 2 2 2
4876 099/300 Write block 2150 len 32768 data: 2 2 2 2 2 2 3 2
4877 100/300 Read block 721 len 20480
4878 101/300 Flush
4879 102/300 Read block 4803 len 4096
4880 103/300 Write block 4406 len 28672 data: 3 2 2 2 2 2 2
4881 104/300 Write block 2213 len 40960 data: 2 2 2 3 3 4 4 4 4 4
4882 105/300 Read block 854 len 24576
4883 106/300 Read block 3830 len 24576
4884 107/300 Write block 1146 len 36864 data: 3 3 3 3 2 2 2 2 2
4885 108/300 Write block 959 len 12288 data: 2 2 2
4886 109/300 Read block 983 len 8192
4887 110/300 Write block 94 len 36864 data: 4 4 3 3 3 3 3 3 3
4888 111/300 Read block 4442 len 16384
4889 112/300 Write block 1023 len 4096 data: 3
4890 113/300 Read block 2263 len 4096
4891 114/300 Write block 385 len 36864 data: 3 3 3 3 3 3 3 2 2
4892 115/300 Read block 1800 len 8192
4893 116/300 Read block 3454 len 32768
4894 117/300 Read block 4332 len 28672
4895 118/300 Read block 598 len 36864
4896 119/300 Read block 4021 len 36864
4897 120/300 Write block 2301 len 8192 data: 3 3
4898 121/300 Write block 2931 len 32768 data: 4 4 3 3 3 3 3 3
4899 122/300 Read block 647 len 28672
4900 123/300 Write block 548 len 8192 data: 3 4
4901 124/300 Read block 2968 len 36864
4902 125/300 Write block 1246 len 4096 data: 2
4903 126/300 Write block 1690 len 16384 data: 3 3 3 2
4904 127/300 Write block 4971 len 24576 data: 2 3 3 3 3 3
4905 128/300 Write block 4067 len 40960 data: 3 3 3 3 3 3 2 3 3 3
4906 129/300 Write block 80 len 20480 data: 3 3 3 2 2
4907 130/300 Read block 4019 len 20480
4908 131/300 Read block 2072 len 32768
4909 132/300 Read block 4336 len 16384
4910 133/300 Read block 4355 len 36864
4911 134/300 Read block 310 len 40960
4912 135/300 Read block 3620 len 4096
4913 136/300 Read block 1496 len 12288
4914 137/300 Write block 586 len 28672 data: 4 2 2 2 2 2 2
4915 138/300 Write block 618 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4916 139/300 Read block 982 len 32768
4917 140/300 Read block 4841 len 8192
4918 141/300 Write block 3844 len 36864 data: 2 2 2 2 2 2 2 2 2
4919 142/300 Read block 4464 len 40960
4920 143/300 Read block 526 len 20480
4921 144/300 Write block 2312 len 4096 data: 2
4922 145/300 Flush
4923 146/300 Read block 4910 len 12288
4924 147/300 Write block 1097 len 8192 data: 2 2
4925 148/300 Write block 1700 len 12288 data: 3 3 3
4926 149/300 Read block 4273 len 28672
4927 150/300 Read block 2784 len 32768
4928 151/300 Write block 743 len 16384 data: 3 4 4 4
4929 152/300 Read block 4298 len 16384
4930 153/300 Write block 4990 len 40960 data: 5 5 5 3 3 3 3 2 2 2
4931 154/300 Write block 1329 len 12288 data: 3 3 3
4932 155/300 Read block 1721 len 32768
4933 156/300 Flush
4934 157/300 Read block 4780 len 40960
4935 158/300 Read block 1485 len 24576
4936 159/300 Write block 4435 len 12288 data: 2 2 3
4937 160/300 Write block 238 len 20480 data: 3 3 3 3 3
4938 161/300 Write block 2331 len 20480 data: 2 2 3 3 3
4939 162/300 Read block 624 len 16384
4940 163/300 Write block 1623 len 24576 data: 3 3 3 3 3 3
4941 164/300 Read block 2085 len 40960
4942 165/300 Flush
4943 166/300 Read block 51 len 12288
4944 167/300 Write block 4296 len 32768 data: 4 5 5 5 5 6 6 5
4945 168/300 Read block 2134 len 36864
4946 169/300 Read block 2013 len 28672
4947 170/300 Read block 861 len 4096
4948 171/300 Write block 130 len 24576 data: 2 2 2 3 2 2
4949 172/300 Flush
4950 173/300 Read block 1227 len 16384
4951 174/300 Write block 3473 len 40960 data: 4 4 5 5 5 5 5 6 3 3
4952 175/300 Write block 3906 len 32768 data: 2 2 2 3 3 3 3 2
4953 176/300 Write block 1037 len 8192 data: 2 2
4954 177/300 Write block 3027 len 8192 data: 5 5
4955 178/300 Write block 3555 len 4096 data: 2
4956 179/300 Read block 2153 len 24576
4957 180/300 Read block 35 len 28672
4958 181/300 Read block 3087 len 28672
4959 182/300 Read block 633 len 32768
4960 183/300 Write block 4871 len 16384 data: 2 2 2 3
4961 184/300 Write block 2039 len 24576 data: 2 2 2 3 3 3
4962 185/300 Read block 3069 len 4096
4963 186/300 Read block 2997 len 28672
4964 187/300 Write block 4197 len 20480 data: 2 2 2 2 2
4965 188/300 Write block 3552 len 8192 data: 2 2
4966 189/300 Write block 4459 len 40960 data: 2 3 3 3 5 5 6 6 6 6
4967 190/300 Read block 208 len 12288
4968 191/300 Read block 2304 len 32768
4969 192/300 Read block 4198 len 20480
4970 193/300 Read block 2439 len 24576
4971 194/300 Write block 2701 len 32768 data: 3 3 3 4 3 3 3 3
4972 195/300 Read block 1910 len 36864
4973 196/300 Read block 991 len 16384
4974 197/300 Read block 821 len 4096
4975 198/300 Read block 2285 len 16384
4976 199/300 Read block 3742 len 40960
4977 200/300 Read block 1029 len 36864
4978 201/300 Read block 3708 len 12288
4979 202/300 Read block 2279 len 20480
4980 203/300 Write block 242 len 8192 data: 4 3
4981 204/300 Write block 1025 len 16384 data: 5 5 5 5
4982 205/300 Read block 2495 len 40960
4983 206/300 Flush
4984 207/300 Write block 970 len 36864 data: 2 3 2 2 2 2 2 2 3
4985 208/300 Write block 106 len 24576 data: 3 3 3 3 3 3
4986 209/300 Write block 1309 len 24576 data: 3 3 3 3 3 3
4987 210/300 Read block 1548 len 24576
4988 211/300 Flush
4989 212/300 Read block 1836 len 32768
4990 213/300 Write block 971 len 32768 data: 4 3 3 3 3 3 3 4
4991 214/300 Read block 2974 len 36864
4992 215/300 Write block 3948 len 24576 data: 2 2 2 2 2 2
4993 216/300 Read block 1900 len 16384
4994 217/300 Read block 2874 len 20480
4995 218/300 Read block 1870 len 12288
4996 219/300 Read block 2533 len 20480
4997 220/300 Read block 1131 len 20480
4998 221/300 Read block 1322 len 4096
4999 222/300 Flush
5000 223/300 Read block 3396 len 16384
5001 224/300 Read block 3865 len 40960
5002 225/300 Read block 4293 len 24576
5003 226/300 Read block 4067 len 16384
5004 227/300 Read block 4156 len 20480
5005 228/300 Read block 4898 len 12288
5006 229/300 Read block 125 len 24576
5007 230/300 Flush
5008 231/300 Read block 4505 len 12288
5009 232/300 Write block 1842 len 20480 data: 3 3 3 3 5
5010 233/300 Write block 948 len 32768 data: 3 3 3 3 3 3 3 3
5011 234/300 Read block 2851 len 4096
5012 235/300 Read block 1680 len 40960
5013 236/300 Write block 2851 len 4096 data: 3
5014 237/300 Read block 3731 len 24576
5015 238/300 Read block 4144 len 28672
5016 239/300 Write block 3978 len 24576 data: 3 4 3 2 2 2
5017 240/300 Flush
5018 241/300 Read block 459 len 32768
5019 242/300 Write block 1979 len 28672 data: 2 2 2 2 2 2 2
5020 243/300 Write block 1227 len 32768 data: 3 3 3 3 3 3 4 4
5021 244/300 Flush
5022 245/300 Read block 3210 len 28672
5023 246/300 Flush
5024 247/300 Write block 3178 len 24576 data: 2 2 2 2 2 2
5025 248/300 Write block 1736 len 4096 data: 3
5026 249/300 Write block 2184 len 32768 data: 4 3 3 3 3 3 2 2
5027 250/300 Write block 1873 len 20480 data: 4 3 3 3 3
5028 251/300 Write block 208 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5029 252/300 Read block 1835 len 40960
5030 253/300 Write block 1859 len 12288 data: 3 3 3
5031 254/300 Read block 4627 len 24576
5032 255/300 Read block 290 len 4096
5033 256/300 Write block 1531 len 8192 data: 3 3
5034 257/300 Read block 2913 len 40960
5035 258/300 Read block 359 len 8192
5036 259/300 Write block 4708 len 16384 data: 2 3 2 2
5037 260/300 Write block 2439 len 16384 data: 2 2 2 2
5038 261/300 Read block 2996 len 36864
5039 262/300 Read block 1995 len 16384
5040 263/300 Write block 2873 len 32768 data: 4 2 2 2 2 2 2 2
5041 264/300 Read block 1961 len 40960
5042 265/300 Read block 3716 len 20480
5043 266/300 Write block 373 len 24576 data: 2 3 4 4 4 4
5044 267/300 Read block 4578 len 4096
5045 268/300 Read block 3926 len 40960
5046 269/300 Flush
5047 270/300 Read block 332 len 24576
5048 271/300 Read block 4640 len 16384
5049 272/300 Write block 4224 len 12288 data: 2 2 2
5050 273/300 Flush
5051 274/300 Write block 3798 len 16384 data: 2 2 2 2
5052 275/300 Write block 1685 len 20480 data: 2 2 2 2 2
5053 276/300 Read block 1747 len 40960
5054 277/300 Write block 1654 len 16384 data: 2 2 2 2
5055 278/300 Read block 1541 len 20480
5056 279/300 Read block 1570 len 20480
5057 280/300 Read block 100 len 4096
5058 281/300 Write block 3132 len 28672 data: 2 2 2 2 2 2 2
5059 282/300 Write block 4078 len 16384 data: 3 3 3 2
5060 283/300 Write block 1857 len 32768 data: 4 3 4 4 4 3 3 3
5061 284/300 Write block 2000 len 28672 data: 2 2 2 2 2 2 2
5062 285/300 Read block 4516 len 16384
5063 286/300 Write block 2269 len 4096 data: 2
5064 287/300 Write block 3301 len 4096 data: 3
5065 288/300 Read block 51 len 8192
5066 289/300 Read block 239 len 20480
5067 290/300 Write block 4205 len 36864 data: 2 2 2 2 2 2 3 3 3
5068 291/300 Read block 4799 len 12288
5069 292/300 Write block 404 len 36864 data: 4 4 3 2 2 2 2 2 2
5070 293/300 Write block 366 len 8192 data: 3 2
5071 294/300 Read block 4415 len 12288
5072 295/300 Read block 221 len 16384
5073 296/300 Write block 3576 len 36864 data: 3 2 2 2 2 2 2 3 3
5074 297/300 Read block 3705 len 36864
5075 298/300 Read block 3057 len 4096
5076 299/300 Write block 686 len 32768 data: 2 2 2 2 2 2 2 3
5077 300/300 Write block 4450 len 28672 data: 2 2 2 3 3 4 3
5078 301/300 Write block 4011 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5079 Main task has work to do, go find it
5080 got dsc Start(0)
5081 start 0
5082 Jan 11 06:06:31.704 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/0, method: POST, req_id: eddf66fc-2523-4a16-ba94-e39030bdb52d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
5083 Replay: started 0, returned:Ok(())
5084 ----------------------------------------------------------------
5085 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5086 States: Offline Active Active
5087 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5088 2208 Acked Write 3 Sent Done Done false
5089 2209 Acked Read 10 Sent Done Done false
5090 2210 Acked Flush 0 Sent Done Done false
5091 2211 Acked Write 2 Sent Done Done false
5092 2212 Acked Flush 0 Sent Done Done false
5093 2213 Acked Read 3 Sent Done Done false
5094 2214 Acked Read 7 Sent Done Done false
5095 2215 Acked Write 8 Sent Done Done false
5096 2216 Acked Write 5 Sent Done Done false
5097 2217 Acked Read 2 Sent Done Done false
5098 2218 Acked Read 3 Sent Done Done false
5099 2219 Acked Read 3 Sent Done Done false
5100 2220 Acked Read 4 Sent Done Done false
5101 2221 Acked Write 8 Sent Done Done false
5102 2222 Acked Read 3 Sent Done Done false
5103 2223 Acked Read 3 Sent Done Done false
5104 2224 Acked Read 3 Sent Done Done false
5105 2225 Acked Write 5 Sent Done Done false
5106 2226 Acked Read 7 Sent Done Done false
5107 2227 Acked Read 3 Sent Done Done false
5108 2228 Acked Write 7 Sent Done Done false
5109 2229 Acked Write 8 Sent Done Done false
5110 2230 Acked Write 10 Sent Done Done false
5111 2231 Acked Flush 0 Sent Done Done false
5112 2232 Acked Write 6 Sent Done Done false
5113 2233 Acked Flush 0 Sent Done Done false
5114 2234 Acked Read 10 Sent Done Done false
5115 2235 Acked Write 4 Sent Done Done false
5116 2236 Acked Write 9 Sent Done Done false
5117 2237 Acked Write 10 Sent Done Done false
5118 2238 Acked Flush 0 Sent Done Done false
5119 2239 Acked Write 6 Sent Done Done false
5120 2240 Acked Read 3 Sent Done Done false
5121 2241 Acked Write 3 Sent Done Done false
5122 2242 Acked Read 5 Sent Done Done false
5123 2243 Acked Read 3 Sent Done Done false
5124 2244 Acked Write 10 Sent Done Done false
5125 2245 Acked Flush 0 Sent Done Done false
5126 2246 Acked Write 2 Sent Done Done false
5127 2247 Acked Read 7 Sent Done Done false
5128 2248 Acked Write 8 Sent Done Done false
5129 2249 Acked Write 10 Sent Done Done false
5130 2250 Acked Read 3 Sent Done Done false
5131 2251 Acked Write 1 Sent Done Done false
5132 2252 Acked Write 10 Sent Done Done false
5133 2253 Acked Write 5 Sent Done Done false
5134 2254 Acked Read 3 Sent Done Done false
5135 2255 Acked Read 10 Sent Done Done false
5136 2256 Acked Read 1 Sent Done Done false
5137 2257 Acked Write 7 Sent Done Done false
5138 2258 Acked Write 1 Sent Done Done false
5139 2259 Acked Read 2 Sent Done Done false
5140 2260 Acked Write 7 Sent Done Done false
5141 2261 Acked Read 2 Sent Done Done false
5142 2262 Acked Write 7 Sent Done Done false
5143 2263 Acked Write 1 Sent Done Done false
5144 2264 Acked Read 4 Sent Done Done false
5145 2265 Acked Read 9 Sent Done Done false
5146 2266 Acked Read 7 Sent Done Done false
5147 2267 Acked Read 6 Sent Done Done false
5148 2268 Acked Read 5 Sent Done Done false
5149 2269 Acked Read 4 Sent Done Done false
5150 2270 Acked Read 10 Sent Done Done false
5151 2271 Acked Read 8 Sent Done Done false
5152 2272 Acked Write 10 Sent Done Done false
5153 2273 Acked Write 2 Sent Done Done false
5154 2274 Acked Read 10 Sent Done Done false
5155 2275 Acked Write 4 Sent Done Done false
5156 2276 Acked Write 9 Sent Done Done false
5157 2277 Acked Read 7 Sent Done Done false
5158 2278 Acked Write 8 Sent Done Done false
5159 2279 Acked Read 3 Sent Done Done false
5160 2280 Acked Flush 0 Sent Done Done false
5161 2281 Acked Read 10 Sent Done Done false
5162 2282 Acked Read 8 Sent Done Done false
5163 2283 Acked Write 3 Sent Done Done false
5164 2284 Acked Read 10 Sent Done Done false
5165 2285 Acked Read 9 Sent Done Done false
5166 2286 Acked Write 8 Sent Done Done false
5167 2287 Acked Read 2 Sent Done Done false
5168 2288 Acked Write 10 Sent Done Done false
5169 2289 Acked Flush 0 Sent Done Done false
5170 2290 Acked Flush 0 Sent Done Done false
5171 2291 Acked Read 3 Sent Done Done false
5172 2292 Acked Read 2 Sent Done Done false
5173 2293 Acked Read 4 Sent Done Done false
5174 2294 Acked Read 7 Sent Done Done false
5175 2295 Acked Read 1 Sent Done Done false
5176 2296 Acked Read 1 Sent Done Done false
5177 2297 Acked Flush 0 Sent Done Done false
5178 2298 Acked Read 6 Sent Done Done false
5179 2299 Acked Flush 0 Sent Done Done false
5180 2300 Acked Read 4 Sent Done Done false
5181 2301 Acked Flush 0 Sent Done Done false
5182 2302 Acked Write 9 Sent Done Done false
5183 2303 Acked Write 6 Sent Done Done false
5184 2304 Acked Write 8 Sent Done Done false
5185 2305 Acked Write 5 Sent Done Done false
5186 2306 Acked Write 8 Sent Done Done false
5187 2307 Acked Read 5 Sent Done Done false
5188 2308 Acked Flush 0 Sent Done Done false
5189 2309 Acked Read 1 Sent Done Done false
5190 2310 Acked Write 7 Sent Done Done false
5191 2311 Acked Write 10 Sent Done Done false
5192 2312 Acked Read 6 Sent Done Done false
5193 2313 Acked Read 6 Sent Done Done false
5194 2314 Acked Write 9 Sent Done Done false
5195 2315 Acked Write 3 Sent Done Done false
5196 2316 Acked Read 2 Sent Done Done false
5197 2317 Acked Write 9 Sent Done Done false
5198 2318 Acked Read 4 Sent Done Done false
5199 2319 Acked Write 1 Sent Done Done false
5200 2320 Acked Read 1 Sent Done Done false
5201 2321 Acked Write 9 Sent Done Done false
5202 2322 Acked Read 2 Sent Done Done false
5203 2323 Acked Read 8 Sent Done Done false
5204 2324 Acked Read 7 Sent Done Done false
5205 2325 Acked Read 9 Sent Done Done false
5206 2326 Acked Read 9 Sent Done Done false
5207 2327 Acked Write 2 Sent Done Done false
5208 2328 Acked Write 8 Sent Done Done false
5209 2329 Acked Read 7 Sent Done Done false
5210 2330 Acked Write 2 Sent Done Done false
5211 2331 Acked Read 9 Sent Done Done false
5212 2332 Acked Write 1 Sent Done Done false
5213 2333 Acked Write 4 Sent Done Done false
5214 2334 Acked Write 6 Sent Done Done false
5215 2335 Acked Write 10 Sent Done Done false
5216 2336 Acked Write 5 Sent Done Done false
5217 2337 Acked Read 5 Sent Done Done false
5218 2338 Acked Read 8 Sent Done Done false
5219 2339 Acked Read 4 Sent Done Done false
5220 2340 Acked Read 9 Sent Done Done false
5221 2341 Acked Read 10 Sent Done Done false
5222 2342 Acked Read 1 Sent Done Done false
5223 2343 Acked Read 3 Sent Done Done false
5224 2344 Acked Write 7 Sent Done Done false
5225 2345 Acked Write 10 Sent Done Done false
5226 2346 Acked Read 8 Sent Done Done false
5227 2347 Acked Read 2 Sent Done Done false
5228 2348 Acked Write 9 Sent Done Done false
5229 2349 Acked Read 10 Sent Done Done false
5230 2350 Acked Read 5 Sent Done Done false
5231 2351 Acked Write 1 Sent Done Done false
5232 2352 Acked Flush 0 Sent Done Done false
5233 2353 Acked Read 3 Sent Done Done false
5234 2354 Acked Write 2 Sent Done Done false
5235 2355 Acked Write 3 Sent Done Done false
5236 2356 Acked Read 7 Sent Done Done false
5237 2357 Acked Read 8 Sent Done Done false
5238 2358 Acked Write 4 Sent Done Done false
5239 2359 Acked Read 4 Sent Done Done false
5240 2360 Acked Write 10 Sent Done Done false
5241 2361 Acked Write 3 Sent Done Done false
5242 2362 Acked Read 8 Sent Done Done false
5243 2363 Acked Flush 0 Sent Done Done false
5244 2364 Acked Read 10 Sent Done Done false
5245 2365 Acked Read 6 Sent Done Done false
5246 2366 Acked Write 3 Sent Done Done false
5247 2367 Acked Write 5 Sent Done Done false
5248 2368 Acked Write 5 Sent Done Done false
5249 2369 Acked Read 4 Sent Done Done false
5250 2370 Acked Write 6 Sent Done Done false
5251 2371 Acked Read 10 Sent Done Done false
5252 2372 Acked Flush 0 Sent Done Done false
5253 2373 Acked Read 3 Sent Done Done false
5254 2374 Acked Write 8 Sent Done Done false
5255 2375 Acked Read 9 Sent Done Done false
5256 2376 Acked Read 7 Sent Done Done false
5257 2377 Acked Read 1 Sent Done Done false
5258 2378 Acked Write 6 Sent Done Done false
5259 2379 Acked Flush 0 Sent Done Done false
5260 2380 Acked Read 4 Sent Done Done false
5261 2381 Acked Write 10 Sent Done Done false
5262 2382 Acked Write 8 Sent Done Done false
5263 2383 Acked Write 2 Sent Done Done false
5264 2384 Acked Write 2 Sent Done Done false
5265 2385 Acked Write 1 Sent Done Done false
5266 2386 Acked Read 6 Sent Done Done false
5267 2387 Acked Read 7 Sent Done Done false
5268 2388 Acked Read 7 Sent Done Done false
5269 2389 Acked Read 8 Sent Done Done false
5270 2390 Acked Write 4 Sent Done Done false
5271 2391 Acked Write 6 Sent Done Done false
5272 2392 Acked Read 1 Sent Done Done false
5273 2393 Acked Read 7 Sent Done Done false
5274 2394 Acked Write 5 Sent Done Done false
5275 2395 Acked Write 2 Sent Done Done false
5276 2396 Acked Write 10 Sent Done Done false
5277 2397 Acked Read 3 Sent Done Done false
5278 2398 Acked Read 8 Sent Done Done false
5279 2399 Acked Read 5 Sent Done Done false
5280 2400 Acked Read 6 Sent Done Done false
5281 2401 Acked Write 8 Sent Done Done false
5282 2402 Acked Read 9 Sent Done Done false
5283 2403 Acked Read 4 Sent Done Done false
5284 2404 Acked Read 1 Sent Done Done false
5285 2405 Acked Read 4 Sent Done Done false
5286 2406 Acked Read 10 Sent Done Done false
5287 2407 Acked Read 9 Sent Done Done false
5288 2408 Acked Read 3 Sent Done Done false
5289 2409 Acked Read 5 Sent Done Done false
5290 2410 Acked Write 2 Sent Done Done false
5291 2411 Acked Write 4 Sent Done Done false
5292 2412 Acked Read 10 Sent Done Done false
5293 2413 Acked Flush 0 Sent Done Done false
5294 2414 Acked Write 9 Sent Done Done false
5295 2415 Acked Write 6 Sent Done Done false
5296 2416 Acked Write 6 Sent Done Done false
5297 2417 Acked Read 6 Sent Done Done false
5298 2418 Acked Flush 0 Sent Done Done false
5299 2419 Acked Read 8 Sent Done Done false
5300 2420 Acked Write 8 Sent Done Done false
5301 2421 Acked Read 9 Sent Done Done false
5302 2422 Acked Write 6 Sent Done Done false
5303 2423 Acked Read 4 Sent Done Done false
5304 2424 Acked Read 5 Sent Done Done false
5305 2425 Acked Read 3 Sent Done Done false
5306 2426 Acked Read 5 Sent Done Done false
5307 2427 Acked Read 5 Sent Done Done false
5308 2428 Acked Read 1 Sent Done Done false
5309 2429 Acked Flush 0 Sent Done Done false
5310 2430 Acked Read 4 Sent Done Done false
5311 2431 Acked Read 10 Sent Done Done false
5312 2432 Acked Read 6 Sent Done Done false
5313 2433 Acked Read 4 Sent Done Done false
5314 2434 Acked Read 5 Sent Done Done false
5315 2435 Acked Read 3 Sent Done Done false
5316 2436 Acked Read 6 Sent Done Done false
5317 2437 Acked Flush 0 Sent Done Done false
5318 2438 Acked Read 3 Sent Done Done false
5319 2439 Acked Write 5 Sent Done Done false
5320 2440 Acked Write 8 Sent Done Done false
5321 2441 Acked Read 1 Sent Done Done false
5322 2442 Acked Read 10 Sent Done Done false
5323 2443 Acked Write 1 Sent Done Done false
5324 2444 Acked Read 6 Sent Done Done false
5325 2445 Acked Read 7 Sent Done Done false
5326 2446 Acked Write 6 Sent Done Done false
5327 2447 Acked Flush 0 Sent Done Done false
5328 2448 Acked Read 8 Sent Done Done false
5329 2449 Acked Write 7 Sent Done Done false
5330 2450 Acked Write 8 Sent Done Done false
5331 2451 Acked Flush 0 Sent Done Done false
5332 2452 Acked Read 7 Sent Done Done false
5333 2453 Acked Flush 0 Sent Done Done false
5334 2454 Acked Write 6 Sent Done Done false
5335 2455 Acked Write 1 Sent Done Done false
5336 2456 Acked Write 8 Sent Done Done false
5337 2457 Acked Write 5 Sent Done Done false
5338 2458 Acked Write 10 Sent Done Done false
5339 2459 Acked Read 10 Sent Done Done false
5340 2460 Acked Write 3 Sent Done Done false
5341 2461 Acked Read 6 Sent Done Done false
5342 2462 Acked Read 1 Sent Done Done false
5343 2463 Acked Write 2 Sent Done Done false
5344 2464 Acked Read 10 Sent Done Done false
5345 2465 Acked Read 2 Sent Done Done false
5346 2466 Acked Write 4 Sent Done Done false
5347 2467 Acked Write 4 Sent Done Done false
5348 2468 Acked Read 9 Sent Done Done false
5349 2469 Acked Read 4 Sent Done Done false
5350 2470 Acked Write 8 Sent Done Done false
5351 2471 Acked Read 10 Sent Done Done false
5352 2472 Acked Read 5 Sent Done Done false
5353 2473 Acked Write 6 Sent Done Done false
5354 2474 Acked Read 1 Sent Done Done false
5355 2475 Acked Read 10 Sent Done Done false
5356 2476 Acked Flush 0 Sent Done Done false
5357 2477 Acked Read 6 Sent Done Done false
5358 2478 Acked Read 4 Sent Done Done false
5359 2479 Acked Write 3 Sent Done Done false
5360 2480 Acked Flush 0 Sent Done Done false
5361 2481 Acked Write 4 Sent Done Done false
5362 2482 Acked Write 5 Sent Done Done false
5363 2483 Acked Read 10 Sent Done Done false
5364 2484 Acked Write 4 Sent Done Done false
5365 2485 Acked Read 5 Sent Done Done false
5366 2486 Acked Read 5 Sent Done Done false
5367 2487 Acked Read 1 Sent Done Done false
5368 2488 Acked Write 7 Sent Done Done false
5369 2489 Acked Write 4 Sent Done Done false
5370 2490 Acked Write 8 Sent Done Done false
5371 2491 Acked Write 7 Sent Done Done false
5372 2492 Acked Read 4 Sent Done Done false
5373 2493 Acked Write 1 Sent Done Done false
5374 2494 Acked Write 1 Sent Done Done false
5375 2495 Acked Read 2 Sent Done Done false
5376 2496 Acked Read 5 Sent Done Done false
5377 2497 Acked Write 9 Sent Done Done false
5378 2498 Acked Read 3 Sent Done Done false
5379 2499 Acked Write 9 Sent Done Done false
5380 2500 Acked Write 2 Sent Done Done false
5381 2501 Acked Read 3 Sent Done Done false
5382 2502 Acked Read 4 Sent Done Done false
5383 2503 Acked Write 9 Sent Done Done false
5384 2504 Acked Read 9 Sent Done Done false
5385 2505 Acked Read 1 Sent Done Done false
5386 2506 Acked Write 8 Sent Done Done false
5387 2507 Acked Write 7 Sent Done Done false
5388 2508 Acked Write 10 Sent Done Done false
5389 STATES DS:0 DS:1 DS:2 TOTAL
5390 Sent 301 0 0 301
5391 Done 0 301 301 602
5392 Skipped 0 0 0 0
5393 Error 0 0 0 0
5394 Last Flush: 2207 2480 2480
5395 Downstairs last five completed: 2207 2206 2205 2204 2203
5396 Upstairs last five completed: 2508 2507 2506 2505 2504
5397 CLIENT: Up:0 ds:301 act:2
5398 [8810] got start action
5399 [8810] I am going to restart
5400 Starting downstairs at port 8810
5401 Make output file at "/tmp/dsc/downstairs-8810.txt"
5402 [8810][0] reports Starting
5403 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1209)
5404 [8810][0] reports Running
5405 ----------------------------------------------------------------
5406 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5407 States: Offline Active Active
5408 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5409 2208 Acked Write 3 Sent Done Done false
5410 2209 Acked Read 10 Sent Done Done false
5411 2210 Acked Flush 0 Sent Done Done false
5412 2211 Acked Write 2 Sent Done Done false
5413 2212 Acked Flush 0 Sent Done Done false
5414 2213 Acked Read 3 Sent Done Done false
5415 2214 Acked Read 7 Sent Done Done false
5416 2215 Acked Write 8 Sent Done Done false
5417 2216 Acked Write 5 Sent Done Done false
5418 2217 Acked Read 2 Sent Done Done false
5419 2218 Acked Read 3 Sent Done Done false
5420 2219 Acked Read 3 Sent Done Done false
5421 2220 Acked Read 4 Sent Done Done false
5422 2221 Acked Write 8 Sent Done Done false
5423 2222 Acked Read 3 Sent Done Done false
5424 2223 Acked Read 3 Sent Done Done false
5425 2224 Acked Read 3 Sent Done Done false
5426 2225 Acked Write 5 Sent Done Done false
5427 2226 Acked Read 7 Sent Done Done false
5428 2227 Acked Read 3 Sent Done Done false
5429 2228 Acked Write 7 Sent Done Done false
5430 2229 Acked Write 8 Sent Done Done false
5431 2230 Acked Write 10 Sent Done Done false
5432 2231 Acked Flush 0 Sent Done Done false
5433 2232 Acked Write 6 Sent Done Done false
5434 2233 Acked Flush 0 Sent Done Done false
5435 2234 Acked Read 10 Sent Done Done false
5436 2235 Acked Write 4 Sent Done Done false
5437 2236 Acked Write 9 Sent Done Done false
5438 2237 Acked Write 10 Sent Done Done false
5439 2238 Acked Flush 0 Sent Done Done false
5440 2239 Acked Write 6 Sent Done Done false
5441 2240 Acked Read 3 Sent Done Done false
5442 2241 Acked Write 3 Sent Done Done false
5443 2242 Acked Read 5 Sent Done Done false
5444 2243 Acked Read 3 Sent Done Done false
5445 2244 Acked Write 10 Sent Done Done false
5446 2245 Acked Flush 0 Sent Done Done false
5447 2246 Acked Write 2 Sent Done Done false
5448 2247 Acked Read 7 Sent Done Done false
5449 2248 Acked Write 8 Sent Done Done false
5450 2249 Acked Write 10 Sent Done Done false
5451 2250 Acked Read 3 Sent Done Done false
5452 2251 Acked Write 1 Sent Done Done false
5453 2252 Acked Write 10 Sent Done Done false
5454 2253 Acked Write 5 Sent Done Done false
5455 2254 Acked Read 3 Sent Done Done false
5456 2255 Acked Read 10 Sent Done Done false
5457 2256 Acked Read 1 Sent Done Done false
5458 2257 Acked Write 7 Sent Done Done false
5459 2258 Acked Write 1 Sent Done Done false
5460 2259 Acked Read 2 Sent Done Done false
5461 2260 Acked Write 7 Sent Done Done false
5462 2261 Acked Read 2 Sent Done Done false
5463 2262 Acked Write 7 Sent Done Done false
5464 2263 Acked Write 1 Sent Done Done false
5465 2264 Acked Read 4 Sent Done Done false
5466 2265 Acked Read 9 Sent Done Done false
5467 2266 Acked Read 7 Sent Done Done false
5468 2267 Acked Read 6 Sent Done Done false
5469 2268 Acked Read 5 Sent Done Done false
5470 2269 Acked Read 4 Sent Done Done false
5471 2270 Acked Read 10 Sent Done Done false
5472 2271 Acked Read 8 Sent Done Done false
5473 2272 Acked Write 10 Sent Done Done false
5474 2273 Acked Write 2 Sent Done Done false
5475 2274 Acked Read 10 Sent Done Done false
5476 2275 Acked Write 4 Sent Done Done false
5477 2276 Acked Write 9 Sent Done Done false
5478 2277 Acked Read 7 Sent Done Done false
5479 2278 Acked Write 8 Sent Done Done false
5480 2279 Acked Read 3 Sent Done Done false
5481 2280 Acked Flush 0 Sent Done Done false
5482 2281 Acked Read 10 Sent Done Done false
5483 2282 Acked Read 8 Sent Done Done false
5484 2283 Acked Write 3 Sent Done Done false
5485 2284 Acked Read 10 Sent Done Done false
5486 2285 Acked Read 9 Sent Done Done false
5487 2286 Acked Write 8 Sent Done Done false
5488 2287 Acked Read 2 Sent Done Done false
5489 2288 Acked Write 10 Sent Done Done false
5490 2289 Acked Flush 0 Sent Done Done false
5491 2290 Acked Flush 0 Sent Done Done false
5492 2291 Acked Read 3 Sent Done Done false
5493 2292 Acked Read 2 Sent Done Done false
5494 2293 Acked Read 4 Sent Done Done false
5495 2294 Acked Read 7 Sent Done Done false
5496 2295 Acked Read 1 Sent Done Done false
5497 2296 Acked Read 1 Sent Done Done false
5498 2297 Acked Flush 0 Sent Done Done false
5499 2298 Acked Read 6 Sent Done Done false
5500 2299 Acked Flush 0 Sent Done Done false
5501 2300 Acked Read 4 Sent Done Done false
5502 2301 Acked Flush 0 Sent Done Done false
5503 2302 Acked Write 9 Sent Done Done false
5504 2303 Acked Write 6 Sent Done Done false
5505 2304 Acked Write 8 Sent Done Done false
5506 2305 Acked Write 5 Sent Done Done false
5507 2306 Acked Write 8 Sent Done Done false
5508 2307 Acked Read 5 Sent Done Done false
5509 2308 Acked Flush 0 Sent Done Done false
5510 2309 Acked Read 1 Sent Done Done false
5511 2310 Acked Write 7 Sent Done Done false
5512 2311 Acked Write 10 Sent Done Done false
5513 2312 Acked Read 6 Sent Done Done false
5514 2313 Acked Read 6 Sent Done Done false
5515 2314 Acked Write 9 Sent Done Done false
5516 2315 Acked Write 3 Sent Done Done false
5517 2316 Acked Read 2 Sent Done Done false
5518 2317 Acked Write 9 Sent Done Done false
5519 2318 Acked Read 4 Sent Done Done false
5520 2319 Acked Write 1 Sent Done Done false
5521 2320 Acked Read 1 Sent Done Done false
5522 2321 Acked Write 9 Sent Done Done false
5523 2322 Acked Read 2 Sent Done Done false
5524 2323 Acked Read 8 Sent Done Done false
5525 2324 Acked Read 7 Sent Done Done false
5526 2325 Acked Read 9 Sent Done Done false
5527 2326 Acked Read 9 Sent Done Done false
5528 2327 Acked Write 2 Sent Done Done false
5529 2328 Acked Write 8 Sent Done Done false
5530 2329 Acked Read 7 Sent Done Done false
5531 2330 Acked Write 2 Sent Done Done false
5532 2331 Acked Read 9 Sent Done Done false
5533 2332 Acked Write 1 Sent Done Done false
5534 2333 Acked Write 4 Sent Done Done false
5535 2334 Acked Write 6 Sent Done Done false
5536 2335 Acked Write 10 Sent Done Done false
5537 2336 Acked Write 5 Sent Done Done false
5538 2337 Acked Read 5 Sent Done Done false
5539 2338 Acked Read 8 Sent Done Done false
5540 2339 Acked Read 4 Sent Done Done false
5541 2340 Acked Read 9 Sent Done Done false
5542 2341 Acked Read 10 Sent Done Done false
5543 2342 Acked Read 1 Sent Done Done false
5544 2343 Acked Read 3 Sent Done Done false
5545 2344 Acked Write 7 Sent Done Done false
5546 2345 Acked Write 10 Sent Done Done false
5547 2346 Acked Read 8 Sent Done Done false
5548 2347 Acked Read 2 Sent Done Done false
5549 2348 Acked Write 9 Sent Done Done false
5550 2349 Acked Read 10 Sent Done Done false
5551 2350 Acked Read 5 Sent Done Done false
5552 2351 Acked Write 1 Sent Done Done false
5553 2352 Acked Flush 0 Sent Done Done false
5554 2353 Acked Read 3 Sent Done Done false
5555 2354 Acked Write 2 Sent Done Done false
5556 2355 Acked Write 3 Sent Done Done false
5557 2356 Acked Read 7 Sent Done Done false
5558 2357 Acked Read 8 Sent Done Done false
5559 2358 Acked Write 4 Sent Done Done false
5560 2359 Acked Read 4 Sent Done Done false
5561 2360 Acked Write 10 Sent Done Done false
5562 2361 Acked Write 3 Sent Done Done false
5563 2362 Acked Read 8 Sent Done Done false
5564 2363 Acked Flush 0 Sent Done Done false
5565 2364 Acked Read 10 Sent Done Done false
5566 2365 Acked Read 6 Sent Done Done false
5567 2366 Acked Write 3 Sent Done Done false
5568 2367 Acked Write 5 Sent Done Done false
5569 2368 Acked Write 5 Sent Done Done false
5570 2369 Acked Read 4 Sent Done Done false
5571 2370 Acked Write 6 Sent Done Done false
5572 2371 Acked Read 10 Sent Done Done false
5573 2372 Acked Flush 0 Sent Done Done false
5574 2373 Acked Read 3 Sent Done Done false
5575 2374 Acked Write 8 Sent Done Done false
5576 2375 Acked Read 9 Sent Done Done false
5577 2376 Acked Read 7 Sent Done Done false
5578 2377 Acked Read 1 Sent Done Done false
5579 2378 Acked Write 6 Sent Done Done false
5580 2379 Acked Flush 0 Sent Done Done false
5581 2380 Acked Read 4 Sent Done Done false
5582 2381 Acked Write 10 Sent Done Done false
5583 2382 Acked Write 8 Sent Done Done false
5584 2383 Acked Write 2 Sent Done Done false
5585 2384 Acked Write 2 Sent Done Done false
5586 2385 Acked Write 1 Sent Done Done false
5587 2386 Acked Read 6 Sent Done Done false
5588 2387 Acked Read 7 Sent Done Done false
5589 2388 Acked Read 7 Sent Done Done false
5590 2389 Acked Read 8 Sent Done Done false
5591 2390 Acked Write 4 Sent Done Done false
5592 2391 Acked Write 6 Sent Done Done false
5593 2392 Acked Read 1 Sent Done Done false
5594 2393 Acked Read 7 Sent Done Done false
5595 2394 Acked Write 5 Sent Done Done false
5596 2395 Acked Write 2 Sent Done Done false
5597 2396 Acked Write 10 Sent Done Done false
5598 2397 Acked Read 3 Sent Done Done false
5599 2398 Acked Read 8 Sent Done Done false
5600 2399 Acked Read 5 Sent Done Done false
5601 2400 Acked Read 6 Sent Done Done false
5602 2401 Acked Write 8 Sent Done Done false
5603 2402 Acked Read 9 Sent Done Done false
5604 2403 Acked Read 4 Sent Done Done false
5605 2404 Acked Read 1 Sent Done Done false
5606 2405 Acked Read 4 Sent Done Done false
5607 2406 Acked Read 10 Sent Done Done false
5608 2407 Acked Read 9 Sent Done Done false
5609 2408 Acked Read 3 Sent Done Done false
5610 2409 Acked Read 5 Sent Done Done false
5611 2410 Acked Write 2 Sent Done Done false
5612 2411 Acked Write 4 Sent Done Done false
5613 2412 Acked Read 10 Sent Done Done false
5614 2413 Acked Flush 0 Sent Done Done false
5615 2414 Acked Write 9 Sent Done Done false
5616 2415 Acked Write 6 Sent Done Done false
5617 2416 Acked Write 6 Sent Done Done false
5618 2417 Acked Read 6 Sent Done Done false
5619 2418 Acked Flush 0 Sent Done Done false
5620 2419 Acked Read 8 Sent Done Done false
5621 2420 Acked Write 8 Sent Done Done false
5622 2421 Acked Read 9 Sent Done Done false
5623 2422 Acked Write 6 Sent Done Done false
5624 2423 Acked Read 4 Sent Done Done false
5625 2424 Acked Read 5 Sent Done Done false
5626 2425 Acked Read 3 Sent Done Done false
5627 2426 Acked Read 5 Sent Done Done false
5628 2427 Acked Read 5 Sent Done Done false
5629 2428 Acked Read 1 Sent Done Done false
5630 2429 Acked Flush 0 Sent Done Done false
5631 2430 Acked Read 4 Sent Done Done false
5632 2431 Acked Read 10 Sent Done Done false
5633 2432 Acked Read 6 Sent Done Done false
5634 2433 Acked Read 4 Sent Done Done false
5635 2434 Acked Read 5 Sent Done Done false
5636 2435 Acked Read 3 Sent Done Done false
5637 2436 Acked Read 6 Sent Done Done false
5638 2437 Acked Flush 0 Sent Done Done false
5639 2438 Acked Read 3 Sent Done Done false
5640 2439 Acked Write 5 Sent Done Done false
5641 2440 Acked Write 8 Sent Done Done false
5642 2441 Acked Read 1 Sent Done Done false
5643 2442 Acked Read 10 Sent Done Done false
5644 2443 Acked Write 1 Sent Done Done false
5645 2444 Acked Read 6 Sent Done Done false
5646 2445 Acked Read 7 Sent Done Done false
5647 2446 Acked Write 6 Sent Done Done false
5648 2447 Acked Flush 0 Sent Done Done false
5649 2448 Acked Read 8 Sent Done Done false
5650 2449 Acked Write 7 Sent Done Done false
5651 2450 Acked Write 8 Sent Done Done false
5652 2451 Acked Flush 0 Sent Done Done false
5653 2452 Acked Read 7 Sent Done Done false
5654 2453 Acked Flush 0 Sent Done Done false
5655 2454 Acked Write 6 Sent Done Done false
5656 2455 Acked Write 1 Sent Done Done false
5657 2456 Acked Write 8 Sent Done Done false
5658 2457 Acked Write 5 Sent Done Done false
5659 2458 Acked Write 10 Sent Done Done false
5660 2459 Acked Read 10 Sent Done Done false
5661 2460 Acked Write 3 Sent Done Done false
5662 2461 Acked Read 6 Sent Done Done false
5663 2462 Acked Read 1 Sent Done Done false
5664 2463 Acked Write 2 Sent Done Done false
5665 2464 Acked Read 10 Sent Done Done false
5666 2465 Acked Read 2 Sent Done Done false
5667 2466 Acked Write 4 Sent Done Done false
5668 2467 Acked Write 4 Sent Done Done false
5669 2468 Acked Read 9 Sent Done Done false
5670 2469 Acked Read 4 Sent Done Done false
5671 2470 Acked Write 8 Sent Done Done false
5672 2471 Acked Read 10 Sent Done Done false
5673 2472 Acked Read 5 Sent Done Done false
5674 2473 Acked Write 6 Sent Done Done false
5675 2474 Acked Read 1 Sent Done Done false
5676 2475 Acked Read 10 Sent Done Done false
5677 2476 Acked Flush 0 Sent Done Done false
5678 2477 Acked Read 6 Sent Done Done false
5679 2478 Acked Read 4 Sent Done Done false
5680 2479 Acked Write 3 Sent Done Done false
5681 2480 Acked Flush 0 Sent Done Done false
5682 2481 Acked Write 4 Sent Done Done false
5683 2482 Acked Write 5 Sent Done Done false
5684 2483 Acked Read 10 Sent Done Done false
5685 2484 Acked Write 4 Sent Done Done false
5686 2485 Acked Read 5 Sent Done Done false
5687 2486 Acked Read 5 Sent Done Done false
5688 2487 Acked Read 1 Sent Done Done false
5689 2488 Acked Write 7 Sent Done Done false
5690 2489 Acked Write 4 Sent Done Done false
5691 2490 Acked Write 8 Sent Done Done false
5692 2491 Acked Write 7 Sent Done Done false
5693 2492 Acked Read 4 Sent Done Done false
5694 2493 Acked Write 1 Sent Done Done false
5695 2494 Acked Write 1 Sent Done Done false
5696 2495 Acked Read 2 Sent Done Done false
5697 2496 Acked Read 5 Sent Done Done false
5698 2497 Acked Write 9 Sent Done Done false
5699 2498 Acked Read 3 Sent Done Done false
5700 2499 Acked Write 9 Sent Done Done false
5701 2500 Acked Write 2 Sent Done Done false
5702 2501 Acked Read 3 Sent Done Done false
5703 2502 Acked Read 4 Sent Done Done false
5704 2503 Acked Write 9 Sent Done Done false
5705 2504 Acked Read 9 Sent Done Done false
5706 2505 Acked Read 1 Sent Done Done false
5707 2506 Acked Write 8 Sent Done Done false
5708 2507 Acked Write 7 Sent Done Done false
5709 2508 Acked Write 10 Sent Done Done false
5710 2509 Acked Flush 0 Sent Done Done false
5711 STATES DS:0 DS:1 DS:2 TOTAL
5712 Sent 302 0 0 302
5713 Done 0 302 302 604
5714 Skipped 0 0 0 0
5715 Error 0 0 0 0
5716 Last Flush: 2207 2509 2509
5717 Downstairs last five completed: 2207 2206 2205 2204 2203
5718 Upstairs last five completed: 2509 2508 2507 2506 2505
5719 CLIENT: Up:0 ds:302 act:2
57202025-01-11T06:06:37.458ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57212025-01-11T06:06:37.459ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48473) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57222025-01-11T06:06:37.460ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57232025-01-11T06:06:37.460ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57242025-01-11T06:06:37.460ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57252025-01-11T06:06:37.460ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57262025-01-11T06:06:37.460ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57272025-01-11T06:06:37.460ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57282025-01-11T06:06:37.460ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57292025-01-11T06:06:37.460ZINFOcrucible: [0] client re-new 302 jobs since flush 2207 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57302025-01-11T06:06:37.461ZINFOcrucible: [0] Replayed 302 jobs since flush: 2207 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
5731 ----------------------------------------------------------------
5732 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5733 Downstairs last five completed: 2509 2508 2507 2506 2505
5734 Upstairs last five completed: 2509 2508 2507 2506 2505
5735 CLIENT: Up:0 ds:0 act:3
5736 Replay: All jobs finished
5737 Main task has work to do, go find it
5738 got dsc Stop(1)
5739 stop 1
5740 Jan 11 06:06:39.710 INFO [8820] Got stop action so:false kr:false
5741 request completed, latency_us: 362, response_code: 204, uri: /stop/cid/1, method: POST, req_id: d4e591bd-bd98-4ac3-8313-3106b25c8445, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
57422025-01-11T06:06:39.711ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57432025-01-11T06:06:39.711ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57442025-01-11T06:06:39.711ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57452025-01-11T06:06:39.711ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
57462025-01-11T06:06:39.711ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
5747 Jan 11 06:06:39.711 INFO request completed, latency_us: 144, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3597b34b-d474-4627-9359-0c8a2b8e5c3c, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
57482025-01-11T06:06:39.711ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
5749 [8820] Exited with: ExitStatus(unix_wait_status(9))
5750 [8820][1] reports Exit
5751 Jan 11 06:06:43.714 INFO request completed, latency_us: 225, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4b8ef38f-c44f-467d-8c46-d96e2e560310, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
5752 001/300 Read block 4906 len 40960
5753 002/300 Read block 2704 len 16384
5754 003/300 Read block 4204 len 32768
5755 004/300 Write block 92 len 32768 data: 3 3 5 5 4 4 4 4
5756 005/300 Write block 1327 len 40960 data: 3 3 4 4 4 3 3 3 2 2
5757 006/300 Flush
5758 007/300 Write block 3766 len 8192 data: 2 2
5759 008/300 Read block 397 len 4096
5760 009/300 Flush
5761 010/300 Write block 3038 len 20480 data: 3 3 3 3 2
5762 011/300 Write block 3145 len 28672 data: 2 2 2 2 2 2 2
5763 012/300 Flush
5764 013/300 Read block 4 len 12288
5765 014/300 Flush
5766 015/300 Flush
5767 016/300 Read block 2198 len 28672
5768 017/300 Flush
5769 018/300 Write block 4940 len 24576 data: 2 2 2 2 2 2
5770 019/300 Read block 4866 len 20480
5771 020/300 Write block 1826 len 36864 data: 3 4 4 4 4 3 3 3 2
5772 021/300 Read block 2245 len 20480
5773 022/300 Write block 4505 len 36864 data: 3 4 4 4 4 4 3 3 3
5774 023/300 Write block 2055 len 32768 data: 3 3 3 3 3 3 3 2
5775 024/300 Write block 3199 len 24576 data: 2 2 2 2 2 2
5776 025/300 Read block 792 len 12288
5777 026/300 Write block 4451 len 12288 data: 3 3 4
5778 027/300 Write block 4795 len 12288 data: 3 3 2
5779 028/300 Read block 5 len 36864
5780 029/300 Read block 1263 len 20480
5781 030/300 Read block 1060 len 28672
5782 031/300 Write block 969 len 32768 data: 2 3 5 4 4 4 4 4
5783 032/300 Read block 3258 len 28672
5784 033/300 Write block 750 len 20480 data: 3 3 2 2 2
5785 034/300 Write block 4516 len 28672 data: 2 2 2 2 2 2 2
5786 035/300 Write block 278 len 32768 data: 3 3 2 2 4 4 4 4
5787 036/300 Read block 492 len 12288
5788 037/300 Write block 2331 len 12288 data: 3 3 4
5789 038/300 Read block 2513 len 12288
5790 039/300 Write block 2430 len 16384 data: 4 4 3 2
5791 040/300 Read block 1246 len 28672
5792 041/300 Read block 2966 len 28672
5793 042/300 Read block 1504 len 36864
5794 043/300 Read block 545 len 16384
5795 044/300 Write block 497 len 32768 data: 2 2 2 2 2 2 2 2
5796 045/300 Read block 1970 len 16384
5797 046/300 Read block 4428 len 36864
5798 047/300 Read block 1975 len 16384
5799 048/300 Write block 396 len 32768 data: 3 3 3 3 3 3 4 4
5800 049/300 Write block 191 len 12288 data: 2 2 2
5801 050/300 Write block 1862 len 8192 data: 4 4
5802 051/300 Read block 4616 len 40960
5803 052/300 Flush
5804 053/300 Write block 4427 len 12288 data: 3 3 3
5805 054/300 Read block 4340 len 4096
5806 055/300 Flush
5807 056/300 Read block 2756 len 16384
5808 057/300 Flush
5809 058/300 Write block 1703 len 36864 data: 3 3 3 2 2 2 2 2 3
5810 059/300 Write block 4904 len 32768 data: 3 3 3 3 3 3 2 2
5811 060/300 Read block 1732 len 4096
5812 061/300 Write block 134 len 32768 data: 3 3 2 2 2 2 2 3
5813 062/300 Read block 1229 len 20480
5814 063/300 Read block 306 len 16384
5815 064/300 Read block 2808 len 28672
5816 065/300 Write block 539 len 32768 data: 3 2 2 4 3 3 3 3
5817 066/300 Read block 2768 len 12288
5818 067/300 Write block 281 len 12288 data: 3 5 5
5819 068/300 Read block 379 len 16384
5820 069/300 Write block 135 len 40960 data: 4 3 3 3 3 3 4 3 3 3
5821 070/300 Flush
5822 071/300 Write block 1405 len 24576 data: 2 2 2 2 2 2
5823 072/300 Write block 3330 len 16384 data: 2 3 3 3
5824 073/300 Write block 117 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5825 074/300 Write block 1899 len 4096 data: 3
5826 075/300 Write block 3786 len 20480 data: 2 2 2 2 2
5827 076/300 Write block 4731 len 12288 data: 2 2 2
5828 077/300 Write block 4596 len 16384 data: 3 3 3 3
5829 078/300 Write block 4967 len 36864 data: 2 2 2 2 3 4 4 4 4
5830 079/300 Write block 4750 len 4096 data: 3
5831 080/300 Write block 2226 len 32768 data: 3 3 2 2 2 3 3 3
5832 081/300 Write block 976 len 8192 data: 5 4
5833 082/300 Read block 2101 len 20480
5834 083/300 Read block 252 len 24576
5835 084/300 Read block 381 len 40960
5836 085/300 Read block 4895 len 28672
5837 086/300 Write block 3793 len 36864 data: 2 2 2 2 2 3 3 3 3
5838 087/300 Read block 1228 len 28672
5839 088/300 Read block 2908 len 20480
5840 089/300 Write block 4635 len 32768 data: 3 2 2 2 2 2 2 2
5841 090/300 Write block 4901 len 16384 data: 2 2 2 4
5842 091/300 Write block 2924 len 12288 data: 4 3 3
5843 092/300 Read block 1343 len 32768
5844 093/300 Read block 1342 len 12288
5845 094/300 Read block 4812 len 8192
5846 095/300 Write block 4698 len 40960 data: 5 6 4 3 3 3 3 2 2 2
5847 096/300 Write block 1939 len 16384 data: 3 3 3 3
5848 097/300 Read block 2555 len 28672
5849 098/300 Read block 302 len 16384
5850 099/300 Read block 2276 len 40960
5851 100/300 Read block 4830 len 28672
5852 101/300 Write block 2281 len 36864 data: 3 3 3 4 4 4 4 4 3
5853 102/300 Read block 1772 len 36864
5854 103/300 Flush
5855 104/300 Write block 3255 len 32768 data: 2 2 2 2 2 2 2 2
5856 105/300 Read block 1261 len 36864
5857 106/300 Read block 2906 len 8192
5858 107/300 Write block 2946 len 32768 data: 3 3 3 5 5 4 3 3
5859 108/300 Read block 1472 len 8192
5860 109/300 Write block 2958 len 28672 data: 2 2 2 2 2 2 2
5861 110/300 Read block 4357 len 12288
5862 111/300 Read block 365 len 16384
5863 112/300 Flush
5864 113/300 Write block 1321 len 24576 data: 2 2 3 3 2 3
5865 114/300 Read block 4460 len 36864
5866 115/300 Write block 3398 len 32768 data: 2 3 3 3 3 3 3 3
5867 116/300 Read block 3931 len 40960
5868 117/300 Read block 4039 len 8192
5869 118/300 Read block 3523 len 32768
5870 119/300 Read block 4697 len 24576
5871 120/300 Read block 1433 len 16384
5872 121/300 Flush
5873 122/300 Read block 1522 len 8192
5874 123/300 Write block 4505 len 36864 data: 4 5 5 5 5 5 4 4 4
5875 124/300 Flush
5876 125/300 Write block 1846 len 12288 data: 6 5 5
5877 126/300 Write block 2732 len 16384 data: 3 3 3 3
5878 127/300 Read block 214 len 28672
5879 128/300 Read block 3823 len 8192
5880 129/300 Read block 3176 len 36864
5881 130/300 Write block 2762 len 8192 data: 2 2
5882 131/300 Read block 2775 len 4096
5883 132/300 Flush
5884 133/300 Read block 938 len 8192
5885 134/300 Flush
5886 135/300 Write block 3585 len 8192 data: 3 3
5887 136/300 Write block 4822 len 12288 data: 2 2 2
5888 137/300 Write block 1289 len 12288 data: 2 3 3
5889 138/300 Read block 2579 len 24576
5890 139/300 Write block 861 len 32768 data: 3 3 3 3 3 2 2 2
5891 140/300 Write block 722 len 12288 data: 3 3 3
5892 141/300 Read block 340 len 16384
5893 142/300 Write block 2574 len 20480 data: 3 3 3 3 3
5894 143/300 Write block 661 len 12288 data: 2 2 2
5895 144/300 Flush
5896 145/300 Write block 2940 len 32768 data: 2 2 3 3 3 3 4 4
5897 146/300 Write block 3479 len 32768 data: 6 7 4 4 2 2 2 2
5898 147/300 Write block 3098 len 40960 data: 4 3 5 5 5 5 4 3 3 3
5899 148/300 Write block 1691 len 16384 data: 4 4 3 2
5900 149/300 Read block 3212 len 28672
5901 150/300 Write block 299 len 12288 data: 2 2 2
5902 151/300 Read block 864 len 40960
5903 152/300 Read block 3127 len 36864
5904 153/300 Read block 2234 len 28672
5905 154/300 Read block 578 len 4096
5906 155/300 Read block 1852 len 32768
5907 156/300 Read block 1159 len 4096
5908 157/300 Read block 1895 len 24576
5909 158/300 Read block 3525 len 24576
5910 159/300 Read block 3207 len 16384
5911 160/300 Read block 2138 len 32768
5912 161/300 Write block 4266 len 28672 data: 4 4 4 4 3 2 2
5913 162/300 Flush
5914 163/300 Write block 2096 len 16384 data: 4 6 7 7
5915 164/300 Read block 640 len 8192
5916 165/300 Read block 1336 len 28672
5917 166/300 Read block 3853 len 12288
5918 167/300 Write block 1884 len 28672 data: 4 4 2 2 2 2 2
5919 168/300 Write block 2677 len 28672 data: 2 2 2 2 2 3 3
5920 169/300 Read block 4859 len 36864
5921 170/300 Write block 4397 len 24576 data: 2 2 2 2 2 2
5922 171/300 Read block 589 len 4096
5923 172/300 Write block 1561 len 32768 data: 2 2 2 2 2 3 3 3
5924 173/300 Read block 1737 len 16384
5925 174/300 Read block 924 len 40960
5926 175/300 Write block 53 len 24576 data: 4 4 3 2 2 2
5927 176/300 Read block 2013 len 36864
5928 177/300 Read block 1859 len 16384
5929 178/300 Read block 2539 len 24576
5930 179/300 Read block 221 len 16384
5931 180/300 Write block 4406 len 36864 data: 4 3 3 3 3 3 3 2 2
5932 181/300 Read block 2818 len 36864
5933 182/300 Read block 2962 len 24576
5934 183/300 Write block 1943 len 8192 data: 3 2
5935 184/300 Write block 3561 len 4096 data: 4
5936 185/300 Read block 511 len 28672
5937 186/300 Flush
5938 187/300 Write block 3509 len 32768 data: 3 3 3 2 3 3 3 3
5939 188/300 Write block 3132 len 8192 data: 3 3
5940 189/300 Read block 2417 len 4096
5941 190/300 Write block 4973 len 32768 data: 5 5 5 4 3 3 3 3
5942 191/300 Read block 4775 len 40960
5943 192/300 Write block 4568 len 8192 data: 2 2
5944 193/300 Write block 1010 len 16384 data: 4 3 3 2
5945 194/300 Read block 1484 len 8192
5946 195/300 Read block 2131 len 16384
5947 196/300 Read block 4456 len 28672
5948 197/300 Write block 1203 len 36864 data: 4 4 4 3 3 2 2 2 2
5949 198/300 Write block 632 len 8192 data: 2 2
5950 199/300 Read block 2859 len 24576
5951 200/300 Write block 846 len 16384 data: 2 2 2 2
5952 201/300 Read block 1326 len 12288
5953 202/300 Read block 1071 len 16384
5954 203/300 Read block 1969 len 40960
5955 204/300 Write block 2268 len 28672 data: 2 3 2 3 3 3 2
5956 205/300 Write block 1121 len 12288 data: 4 3 2
5957 206/300 Read block 1926 len 36864
5958 207/300 Flush
5959 208/300 Write block 1502 len 32768 data: 4 4 3 3 2 2 2 2
5960 209/300 Write block 4741 len 4096 data: 2
5961 210/300 Write block 10 len 28672 data: 2 2 2 2 2 2 2
5962 211/300 Write block 2934 len 36864 data: 4 4 4 4 4 3 3 3 4
5963 212/300 Write block 2195 len 28672 data: 2 2 4 4 4 3 3
5964 213/300 Flush
5965 214/300 Write block 2562 len 36864 data: 3 3 3 3 3 3 3 3 3
5966 215/300 Write block 1710 len 24576 data: 3 4 3 3 2 3
5967 216/300 Read block 4030 len 36864
5968 217/300 Write block 34 len 8192 data: 2 2
5969 218/300 Read block 1286 len 12288
5970 219/300 Read block 136 len 8192
5971 220/300 Read block 4401 len 8192
5972 221/300 Flush
5973 222/300 Read block 1039 len 32768
5974 223/300 Read block 789 len 36864
5975 224/300 Read block 3984 len 4096
5976 225/300 Read block 4460 len 8192
5977 226/300 Write block 3840 len 4096 data: 2
5978 227/300 Write block 3716 len 16384 data: 2 2 2 2
5979 228/300 Write block 1867 len 8192 data: 2 2
5980 229/300 Write block 2430 len 24576 data: 5 5 4 3 2 2
5981 230/300 Write block 2535 len 40960 data: 3 3 2 2 2 2 2 2 2 2
5982 231/300 Write block 323 len 40960 data: 6 6 7 6 5 5 4 4 4 3
5983 232/300 Write block 1667 len 24576 data: 2 2 3 3 3 3
5984 233/300 Read block 4479 len 40960
5985 234/300 Write block 4738 len 32768 data: 2 2 2 3 2 2 2 2
5986 235/300 Read block 2899 len 28672
5987 236/300 Read block 700 len 16384
5988 237/300 Write block 2519 len 8192 data: 2 2
5989 238/300 Write block 334 len 12288 data: 3 2 3
5990 239/300 Write block 3315 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5991 240/300 Write block 1378 len 24576 data: 2 2 2 2 2 2
5992 241/300 Read block 1905 len 32768
5993 242/300 Write block 2896 len 40960 data: 3 2 2 3 3 3 3 3 3 3
5994 243/300 Write block 3824 len 40960 data: 3 3 3 2 2 2 2 2 2 2
5995 244/300 Write block 4943 len 40960 data: 3 3 3 2 2 3 4 5 5 5
5996 245/300 Read block 647 len 32768
5997 246/300 Write block 1007 len 8192 data: 4 4
5998 247/300 Write block 3585 len 32768 data: 4 4 3 3 3 3 4 4
5999 248/300 Read block 4233 len 4096
6000 249/300 Write block 176 len 8192 data: 2 2
6001 250/300 Write block 811 len 24576 data: 3 3 3 3 2 2
6002 251/300 Read block 1936 len 32768
6003 252/300 Read block 1507 len 32768
6004 253/300 Read block 3201 len 8192
6005 254/300 Write block 491 len 28672 data: 3 3 2 2 2 2 3
6006 255/300 Read block 84 len 4096
6007 256/300 Write block 1497 len 4096 data: 4
6008 257/300 Write block 4633 len 24576 data: 3 3 4 3 3 3
6009 258/300 Read block 3988 len 32768
6010 259/300 Read block 3978 len 16384
6011 260/300 Read block 838 len 4096
6012 261/300 Write block 464 len 40960 data: 3 2 4 4 4 4 3 3 3 3
6013 262/300 Read block 421 len 36864
6014 263/300 Read block 127 len 20480
6015 264/300 Write block 704 len 24576 data: 2 2 2 3 5 5
6016 265/300 Read block 903 len 36864
6017 266/300 Read block 2081 len 32768
6018 267/300 Read block 615 len 4096
6019 268/300 Flush
6020 269/300 Write block 1569 len 16384 data: 3 3 3 3
6021 270/300 Write block 3366 len 20480 data: 3 3 3 3 2
6022 271/300 Write block 3089 len 28672 data: 3 3 3 3 3 3 3
6023 272/300 Write block 81 len 20480 data: 4 4 3 3 2
6024 273/300 Read block 4269 len 20480
6025 274/300 Read block 2567 len 12288
6026 275/300 Write block 405 len 36864 data: 5 4 3 3 3 3 3 3 2
6027 276/300 Read block 2538 len 12288
6028 277/300 Read block 1985 len 40960
6029 278/300 Write block 3902 len 20480 data: 2 2 2 2 3
6030 279/300 Write block 3061 len 8192 data: 2 2
6031 280/300 Read block 112 len 40960
6032 281/300 Write block 3381 len 36864 data: 2 2 2 2 2 2 2 2 2
6033 282/300 Read block 4117 len 20480
6034 283/300 Read block 2510 len 24576
6035 284/300 Read block 1234 len 40960
6036 285/300 Write block 2230 len 40960 data: 3 4 4 4 2 2 2 2 2 2
6037 286/300 Write block 3113 len 20480 data: 2 2 2 2 2
6038 287/300 Write block 4403 len 20480 data: 3 3 3 5 4
6039 288/300 Read block 2694 len 40960
6040 289/300 Read block 3106 len 20480
6041 290/300 Read block 3819 len 36864
6042 291/300 Write block 2215 len 32768 data: 3 4 4 5 5 5 5 5
6043 292/300 Write block 1518 len 40960 data: 2 2 2 3 3 3 3 3 3 3
6044 293/300 Read block 2643 len 20480
6045 294/300 Read block 4192 len 4096
6046 295/300 Write block 241 len 32768 data: 4 5 4 3 3 3 3 2
6047 296/300 Flush
6048 297/300 Write block 2274 len 8192 data: 3 2
6049 298/300 Flush
6050 299/300 Read block 2418 len 16384
6051 300/300 Write block 4072 len 28672 data: 4 3 4 4 4 3 4
6052 301/300 Write block 2760 len 4096 data: 3
6053 Main task has work to do, go find it
6054 got dsc Start(1)
6055 start 1
6056 Jan 11 06:06:43.953 INFO request completed, latency_us: 241, response_code: 204, uri: /start/cid/1, method: POST, req_id: 87adcd71-97ff-47f4-b2de-ef2f0db664f3, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
6057 Replay: started 1, returned:Ok(())
6058 ----------------------------------------------------------------
6059 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6060 States: Active Offline Active
6061 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6062 2510 Acked Read 10 Done Sent Done false
6063 2511 Acked Read 4 Done Sent Done false
6064 2512 Acked Read 8 Done Sent Done false
6065 2513 Acked Write 8 Done Sent Done false
6066 2514 Acked Write 10 Done Sent Done false
6067 2515 Acked Flush 0 Done Sent Done false
6068 2516 Acked Write 2 Done Sent Done false
6069 2517 Acked Read 1 Done Sent Done false
6070 2518 Acked Flush 0 Done Sent Done false
6071 2519 Acked Write 5 Done Sent Done false
6072 2520 Acked Write 7 Done Sent Done false
6073 2521 Acked Flush 0 Done Sent Done false
6074 2522 Acked Read 3 Done Sent Done false
6075 2523 Acked Flush 0 Done Sent Done false
6076 2524 Acked Flush 0 Done Sent Done false
6077 2525 Acked Read 7 Done Sent Done false
6078 2526 Acked Flush 0 Done Sent Done false
6079 2527 Acked Write 6 Done Sent Done false
6080 2528 Acked Read 5 Done Sent Done false
6081 2529 Acked Write 9 Done Sent Done false
6082 2530 Acked Read 5 Done Sent Done false
6083 2531 Acked Write 9 Done Sent Done false
6084 2532 Acked Write 8 Done Sent Done false
6085 2533 Acked Write 6 Done Sent Done false
6086 2534 Acked Read 3 Done Sent Done false
6087 2535 Acked Write 3 Done Sent Done false
6088 2536 Acked Write 3 Done Sent Done false
6089 2537 Acked Read 9 Done Sent Done false
6090 2538 Acked Read 5 Done Sent Done false
6091 2539 Acked Read 7 Done Sent Done false
6092 2540 Acked Write 8 Done Sent Done false
6093 2541 Acked Read 7 Done Sent Done false
6094 2542 Acked Write 5 Done Sent Done false
6095 2543 Acked Write 7 Done Sent Done false
6096 2544 Acked Write 8 Done Sent Done false
6097 2545 Acked Read 3 Done Sent Done false
6098 2546 Acked Write 3 Done Sent Done false
6099 2547 Acked Read 3 Done Sent Done false
6100 2548 Acked Write 4 Done Sent Done false
6101 2549 Acked Read 7 Done Sent Done false
6102 2550 Acked Read 7 Done Sent Done false
6103 2551 Acked Read 9 Done Sent Done false
6104 2552 Acked Read 4 Done Sent Done false
6105 2553 Acked Write 8 Done Sent Done false
6106 2554 Acked Read 4 Done Sent Done false
6107 2555 Acked Read 9 Done Sent Done false
6108 2556 Acked Read 4 Done Sent Done false
6109 2557 Acked Write 8 Done Sent Done false
6110 2558 Acked Write 3 Done Sent Done false
6111 2559 Acked Write 2 Done Sent Done false
6112 2560 Acked Read 10 Done Sent Done false
6113 2561 Acked Flush 0 Done Sent Done false
6114 2562 Acked Write 3 Done Sent Done false
6115 2563 Acked Read 1 Done Sent Done false
6116 2564 Acked Flush 0 Done Sent Done false
6117 2565 Acked Read 4 Done Sent Done false
6118 2566 Acked Flush 0 Done Sent Done false
6119 2567 Acked Write 9 Done Sent Done false
6120 2568 Acked Write 8 Done Sent Done false
6121 2569 Acked Read 1 Done Sent Done false
6122 2570 Acked Write 8 Done Sent Done false
6123 2571 Acked Read 5 Done Sent Done false
6124 2572 Acked Read 4 Done Sent Done false
6125 2573 Acked Read 7 Done Sent Done false
6126 2574 Acked Write 8 Done Sent Done false
6127 2575 Acked Read 3 Done Sent Done false
6128 2576 Acked Write 3 Done Sent Done false
6129 2577 Acked Read 4 Done Sent Done false
6130 2578 Acked Write 10 Done Sent Done false
6131 2579 Acked Flush 0 Done Sent Done false
6132 2580 Acked Write 6 Done Sent Done false
6133 2581 Acked Write 4 Done Sent Done false
6134 2582 Acked Write 10 Done Sent Done false
6135 2583 Acked Write 1 Done Sent Done false
6136 2584 Acked Write 5 Done Sent Done false
6137 2585 Acked Write 3 Done Sent Done false
6138 2586 Acked Write 4 Done Sent Done false
6139 2587 Acked Write 9 Done Sent Done false
6140 2588 Acked Write 1 Done Sent Done false
6141 2589 Acked Write 8 Done Sent Done false
6142 2590 Acked Write 2 Done Sent Done false
6143 2591 Acked Read 5 Done Sent Done false
6144 2592 Acked Read 6 Done Sent Done false
6145 2593 Acked Read 10 Done Sent Done false
6146 2594 Acked Read 7 Done Sent Done false
6147 2595 Acked Write 9 Done Sent Done false
6148 2596 Acked Read 7 Done Sent Done false
6149 2597 Acked Read 5 Done Sent Done false
6150 2598 Acked Write 8 Done Sent Done false
6151 2599 Acked Write 4 Done Sent Done false
6152 2600 Acked Write 3 Done Sent Done false
6153 2601 Acked Read 8 Done Sent Done false
6154 2602 Acked Read 3 Done Sent Done false
6155 2603 Acked Read 2 Done Sent Done false
6156 2604 Acked Write 10 Done Sent Done false
6157 2605 Acked Write 4 Done Sent Done false
6158 2606 Acked Read 7 Done Sent Done false
6159 2607 Acked Read 4 Done Sent Done false
6160 2608 Acked Read 10 Done Sent Done false
6161 2609 Acked Read 7 Done Sent Done false
6162 2610 Acked Write 9 Done Sent Done false
6163 2611 Acked Read 9 Done Sent Done false
6164 2612 Acked Flush 0 Done Sent Done false
6165 2613 Acked Write 8 Done Sent Done false
6166 2614 Acked Read 9 Done Sent Done false
6167 2615 Acked Read 2 Done Sent Done false
6168 2616 Acked Write 8 Done Sent Done false
6169 2617 Acked Read 2 Done Sent Done false
6170 2618 Acked Write 7 Done Sent Done false
6171 2619 Acked Read 3 Done Sent Done false
6172 2620 Acked Read 4 Done Sent Done false
6173 2621 Acked Flush 0 Done Sent Done false
6174 2622 Acked Write 6 Done Sent Done false
6175 2623 Acked Read 9 Done Sent Done false
6176 2624 Acked Write 8 Done Sent Done false
6177 2625 Acked Read 10 Done Sent Done false
6178 2626 Acked Read 2 Done Sent Done false
6179 2627 Acked Read 8 Done Sent Done false
6180 2628 Acked Read 6 Done Sent Done false
6181 2629 Acked Read 4 Done Sent Done false
6182 2630 Acked Flush 0 Done Sent Done false
6183 2631 Acked Read 2 Done Sent Done false
6184 2632 Acked Write 9 Done Sent Done false
6185 2633 Acked Flush 0 Done Sent Done false
6186 2634 Acked Write 3 Done Sent Done false
6187 2635 Acked Write 4 Done Sent Done false
6188 2636 Acked Read 7 Done Sent Done false
6189 2637 Acked Read 2 Done Sent Done false
6190 2638 Acked Read 9 Done Sent Done false
6191 2639 Acked Write 2 Done Sent Done false
6192 2640 Acked Read 1 Done Sent Done false
6193 2641 Acked Flush 0 Done Sent Done false
6194 2642 Acked Read 2 Done Sent Done false
6195 2643 Acked Flush 0 Done Sent Done false
6196 2644 Acked Write 2 Done Sent Done false
6197 2645 Acked Write 3 Done Sent Done false
6198 2646 Acked Write 3 Done Sent Done false
6199 2647 Acked Read 6 Done Sent Done false
6200 2648 Acked Write 8 Done Sent Done false
6201 2649 Acked Write 3 Done Sent Done false
6202 2650 Acked Read 4 Done Sent Done false
6203 2651 Acked Write 5 Done Sent Done false
6204 2652 Acked Write 3 Done Sent Done false
6205 2653 Acked Flush 0 Done Sent Done false
6206 2654 Acked Write 8 Done Sent Done false
6207 2655 Acked Write 8 Done Sent Done false
6208 2656 Acked Write 10 Done Sent Done false
6209 2657 Acked Write 4 Done Sent Done false
6210 2658 Acked Read 7 Done Sent Done false
6211 2659 Acked Write 3 Done Sent Done false
6212 2660 Acked Read 10 Done Sent Done false
6213 2661 Acked Read 9 Done Sent Done false
6214 2662 Acked Read 7 Done Sent Done false
6215 2663 Acked Read 1 Done Sent Done false
6216 2664 Acked Read 8 Done Sent Done false
6217 2665 Acked Read 1 Done Sent Done false
6218 2666 Acked Read 6 Done Sent Done false
6219 2667 Acked Read 6 Done Sent Done false
6220 2668 Acked Read 4 Done Sent Done false
6221 2669 Acked Read 8 Done Sent Done false
6222 2670 Acked Write 7 Done Sent Done false
6223 2671 Acked Flush 0 Done Sent Done false
6224 2672 Acked Write 4 Done Sent Done false
6225 2673 Acked Read 2 Done Sent Done false
6226 2674 Acked Read 7 Done Sent Done false
6227 2675 Acked Read 3 Done Sent Done false
6228 2676 Acked Write 7 Done Sent Done false
6229 2677 Acked Write 7 Done Sent Done false
6230 2678 Acked Read 9 Done Sent Done false
6231 2679 Acked Write 6 Done Sent Done false
6232 2680 Acked Read 1 Done Sent Done false
6233 2681 Acked Write 8 Done Sent Done false
6234 2682 Acked Read 4 Done Sent Done false
6235 2683 Acked Read 10 Done Sent Done false
6236 2684 Acked Write 6 Done Sent Done false
6237 2685 Acked Read 9 Done Sent Done false
6238 2686 Acked Read 4 Done Sent Done false
6239 2687 Acked Read 6 Done Sent Done false
6240 2688 Acked Read 4 Done Sent Done false
6241 2689 Acked Write 9 Done Sent Done false
6242 2690 Acked Read 9 Done Sent Done false
6243 2691 Acked Read 6 Done Sent Done false
6244 2692 Acked Write 2 Done Sent Done false
6245 2693 Acked Write 1 Done Sent Done false
6246 2694 Acked Read 7 Done Sent Done false
6247 2695 Acked Flush 0 Done Sent Done false
6248 2696 Acked Write 8 Done Sent Done false
6249 2697 Acked Write 2 Done Sent Done false
6250 2698 Acked Read 1 Done Sent Done false
6251 2699 Acked Write 8 Done Sent Done false
6252 2700 Acked Read 10 Done Sent Done false
6253 2701 Acked Write 2 Done Sent Done false
6254 2702 Acked Write 4 Done Sent Done false
6255 2703 Acked Read 2 Done Sent Done false
6256 2704 Acked Read 4 Done Sent Done false
6257 2705 Acked Read 7 Done Sent Done false
6258 2706 Acked Write 9 Done Sent Done false
6259 2707 Acked Write 2 Done Sent Done false
6260 2708 Acked Read 6 Done Sent Done false
6261 2709 Acked Write 4 Done Sent Done false
6262 2710 Acked Read 3 Done Sent Done false
6263 2711 Acked Read 4 Done Sent Done false
6264 2712 Acked Read 10 Done Sent Done false
6265 2713 Acked Write 7 Done Sent Done false
6266 2714 Acked Write 3 Done Sent Done false
6267 2715 Acked Read 9 Done Sent Done false
6268 2716 Acked Flush 0 Done Sent Done false
6269 2717 Acked Write 8 Done Sent Done false
6270 2718 Acked Write 1 Done Sent Done false
6271 2719 Acked Write 7 Done Sent Done false
6272 2720 Acked Write 9 Done Sent Done false
6273 2721 Acked Write 7 Done Sent Done false
6274 2722 Acked Flush 0 Done Sent Done false
6275 2723 Acked Write 9 Done Sent Done false
6276 2724 Acked Write 6 Done Sent Done false
6277 2725 Acked Read 9 Done Sent Done false
6278 2726 Acked Write 2 Done Sent Done false
6279 2727 Acked Read 3 Done Sent Done false
6280 2728 Acked Read 2 Done Sent Done false
6281 2729 Acked Read 2 Done Sent Done false
6282 2730 Acked Flush 0 Done Sent Done false
6283 2731 Acked Read 8 Done Sent Done false
6284 2732 Acked Read 9 Done Sent Done false
6285 2733 Acked Read 1 Done Sent Done false
6286 2734 Acked Read 2 Done Sent Done false
6287 2735 Acked Write 1 Done Sent Done false
6288 2736 Acked Write 4 Done Sent Done false
6289 2737 Acked Write 2 Done Sent Done false
6290 2738 Acked Write 6 Done Sent Done false
6291 2739 Acked Write 10 Done Sent Done false
6292 2740 Acked Write 10 Done Sent Done false
6293 2741 Acked Write 6 Done Sent Done false
6294 2742 Acked Read 10 Done Sent Done false
6295 2743 Acked Write 8 Done Sent Done false
6296 2744 Acked Read 7 Done Sent Done false
6297 2745 Acked Read 4 Done Sent Done false
6298 2746 Acked Write 2 Done Sent Done false
6299 2747 Acked Write 3 Done Sent Done false
6300 2748 Acked Write 10 Done Sent Done false
6301 2749 Acked Write 6 Done Sent Done false
6302 2750 Acked Read 8 Done Sent Done false
6303 2751 Acked Write 10 Done Sent Done false
6304 2752 Acked Write 10 Done Sent Done false
6305 2753 Acked Write 10 Done Sent Done false
6306 2754 Acked Read 8 Done Sent Done false
6307 2755 Acked Write 2 Done Sent Done false
6308 2756 Acked Write 8 Done Sent Done false
6309 2757 Acked Read 1 Done Sent Done false
6310 2758 Acked Write 2 Done Sent Done false
6311 2759 Acked Write 6 Done Sent Done false
6312 2760 Acked Read 8 Done Sent Done false
6313 2761 Acked Read 8 Done Sent Done false
6314 2762 Acked Read 2 Done Sent Done false
6315 2763 Acked Write 7 Done Sent Done false
6316 2764 Acked Read 1 Done Sent Done false
6317 2765 Acked Write 1 Done Sent Done false
6318 2766 Acked Write 6 Done Sent Done false
6319 2767 Acked Read 8 Done Sent Done false
6320 2768 Acked Read 4 Done Sent Done false
6321 2769 Acked Read 1 Done Sent Done false
6322 2770 Acked Write 10 Done Sent Done false
6323 2771 Acked Read 9 Done Sent Done false
6324 2772 Acked Read 5 Done Sent Done false
6325 2773 Acked Write 6 Done Sent Done false
6326 2774 Acked Read 9 Done Sent Done false
6327 2775 Acked Read 8 Done Sent Done false
6328 2776 Acked Read 1 Done Sent Done false
6329 2777 Acked Flush 0 Done Sent Done false
6330 2778 Acked Write 4 Done Sent Done false
6331 2779 Acked Write 5 Done Sent Done false
6332 2780 Acked Write 7 Done Sent Done false
6333 2781 Acked Write 5 Done Sent Done false
6334 2782 Acked Read 5 Done Sent Done false
6335 2783 Acked Read 3 Done Sent Done false
6336 2784 Acked Write 9 Done Sent Done false
6337 2785 Acked Read 3 Done Sent Done false
6338 2786 Acked Read 10 Done Sent Done false
6339 2787 Acked Write 5 Done Sent Done false
6340 2788 Acked Write 2 Done Sent Done false
6341 2789 Acked Read 10 Done Sent Done false
6342 2790 Acked Write 9 Done Sent Done false
6343 2791 Acked Read 5 Done Sent Done false
6344 2792 Acked Read 6 Done Sent Done false
6345 2793 Acked Read 10 Done Sent Done false
6346 2794 Acked Write 10 Done Sent Done false
6347 2795 Acked Write 5 Done Sent Done false
6348 2796 Acked Write 5 Done Sent Done false
6349 2797 Acked Read 10 Done Sent Done false
6350 2798 Acked Read 5 Done Sent Done false
6351 2799 Acked Read 9 Done Sent Done false
6352 2800 Acked Write 8 Done Sent Done false
6353 2801 Acked Write 10 Done Sent Done false
6354 2802 Acked Read 5 Done Sent Done false
6355 2803 Acked Read 1 Done Sent Done false
6356 2804 Acked Write 8 Done Sent Done false
6357 2805 Acked Flush 0 Done Sent Done false
6358 2806 Acked Write 2 Done Sent Done false
6359 2807 Acked Flush 0 Done Sent Done false
6360 2808 Acked Read 4 Done Sent Done false
6361 2809 Acked Write 7 Done Sent Done false
6362 2810 Acked Write 1 Done Sent Done false
6363 STATES DS:0 DS:1 DS:2 TOTAL
6364 Sent 0 301 0 301
6365 Done 301 0 301 602
6366 Skipped 0 0 0 0
6367 Error 0 0 0 0
6368 Last Flush: 2807 2509 2807
6369 Downstairs last five completed: 2509 2508 2507 2506 2505
6370 Upstairs last five completed: 2810 2809 2808 2807 2806
6371 CLIENT: Up:0 ds:301 act:2
6372 ----------------------------------------------------------------
6373 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6374 States: Active Offline Active
6375 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6376 2510 Acked Read 10 Done Sent Done false
6377 2511 Acked Read 4 Done Sent Done false
6378 2512 Acked Read 8 Done Sent Done false
6379 2513 Acked Write 8 Done Sent Done false
6380 2514 Acked Write 10 Done Sent Done false
6381 2515 Acked Flush 0 Done Sent Done false
6382 2516 Acked Write 2 Done Sent Done false
6383 2517 Acked Read 1 Done Sent Done false
6384 2518 Acked Flush 0 Done Sent Done false
6385 2519 Acked Write 5 Done Sent Done false
6386 2520 Acked Write 7 Done Sent Done false
6387 2521 Acked Flush 0 Done Sent Done false
6388 2522 Acked Read 3 Done Sent Done false
6389 2523 Acked Flush 0 Done Sent Done false
6390 2524 Acked Flush 0 Done Sent Done false
6391 2525 Acked Read 7 Done Sent Done false
6392 2526 Acked Flush 0 Done Sent Done false
6393 2527 Acked Write 6 Done Sent Done false
6394 2528 Acked Read 5 Done Sent Done false
6395 2529 Acked Write 9 Done Sent Done false
6396 2530 Acked Read 5 Done Sent Done false
6397 2531 Acked Write 9 Done Sent Done false
6398 2532 Acked Write 8 Done Sent Done false
6399 2533 Acked Write 6 Done Sent Done false
6400 2534 Acked Read 3 Done Sent Done false
6401 2535 Acked Write 3 Done Sent Done false
6402 2536 Acked Write 3 Done Sent Done false
6403 2537 Acked Read 9 Done Sent Done false
6404 2538 Acked Read 5 Done Sent Done false
6405 2539 Acked Read 7 Done Sent Done false
6406 2540 Acked Write 8 Done Sent Done false
6407 2541 Acked Read 7 Done Sent Done false
6408 2542 Acked Write 5 Done Sent Done false
6409 2543 Acked Write 7 Done Sent Done false
6410 2544 Acked Write 8 Done Sent Done false
6411 2545 Acked Read 3 Done Sent Done false
6412 2546 Acked Write 3 Done Sent Done false
6413 2547 Acked Read 3 Done Sent Done false
6414 2548 Acked Write 4 Done Sent Done false
6415 2549 Acked Read 7 Done Sent Done false
6416 2550 Acked Read 7 Done Sent Done false
6417 2551 Acked Read 9 Done Sent Done false
6418 2552 Acked Read 4 Done Sent Done false
6419 2553 Acked Write 8 Done Sent Done false
6420 2554 Acked Read 4 Done Sent Done false
6421 2555 Acked Read 9 Done Sent Done false
6422 2556 Acked Read 4 Done Sent Done false
6423 2557 Acked Write 8 Done Sent Done false
6424 2558 Acked Write 3 Done Sent Done false
6425 2559 Acked Write 2 Done Sent Done false
6426 2560 Acked Read 10 Done Sent Done false
6427 2561 Acked Flush 0 Done Sent Done false
6428 2562 Acked Write 3 Done Sent Done false
6429 2563 Acked Read 1 Done Sent Done false
6430 2564 Acked Flush 0 Done Sent Done false
6431 2565 Acked Read 4 Done Sent Done false
6432 2566 Acked Flush 0 Done Sent Done false
6433 2567 Acked Write 9 Done Sent Done false
6434 2568 Acked Write 8 Done Sent Done false
6435 2569 Acked Read 1 Done Sent Done false
6436 2570 Acked Write 8 Done Sent Done false
6437 2571 Acked Read 5 Done Sent Done false
6438 2572 Acked Read 4 Done Sent Done false
6439 2573 Acked Read 7 Done Sent Done false
6440 2574 Acked Write 8 Done Sent Done false
6441 2575 Acked Read 3 Done Sent Done false
6442 2576 Acked Write 3 Done Sent Done false
6443 2577 Acked Read 4 Done Sent Done false
6444 2578 Acked Write 10 Done Sent Done false
6445 2579 Acked Flush 0 Done Sent Done false
6446 2580 Acked Write 6 Done Sent Done false
6447 2581 Acked Write 4 Done Sent Done false
6448 2582 Acked Write 10 Done Sent Done false
6449 2583 Acked Write 1 Done Sent Done false
6450 2584 Acked Write 5 Done Sent Done false
6451 2585 Acked Write 3 Done Sent Done false
6452 2586 Acked Write 4 Done Sent Done false
6453 2587 Acked Write 9 Done Sent Done false
6454 2588 Acked Write 1 Done Sent Done false
6455 2589 Acked Write 8 Done Sent Done false
6456 2590 Acked Write 2 Done Sent Done false
6457 2591 Acked Read 5 Done Sent Done false
6458 2592 Acked Read 6 Done Sent Done false
6459 2593 Acked Read 10 Done Sent Done false
6460 2594 Acked Read 7 Done Sent Done false
6461 2595 Acked Write 9 Done Sent Done false
6462 2596 Acked Read 7 Done Sent Done false
6463 2597 Acked Read 5 Done Sent Done false
6464 2598 Acked Write 8 Done Sent Done false
6465 2599 Acked Write 4 Done Sent Done false
6466 2600 Acked Write 3 Done Sent Done false
6467 2601 Acked Read 8 Done Sent Done false
6468 2602 Acked Read 3 Done Sent Done false
6469 2603 Acked Read 2 Done Sent Done false
6470 2604 Acked Write 10 Done Sent Done false
6471 2605 Acked Write 4 Done Sent Done false
6472 2606 Acked Read 7 Done Sent Done false
6473 2607 Acked Read 4 Done Sent Done false
6474 2608 Acked Read 10 Done Sent Done false
6475 2609 Acked Read 7 Done Sent Done false
6476 2610 Acked Write 9 Done Sent Done false
6477 2611 Acked Read 9 Done Sent Done false
6478 2612 Acked Flush 0 Done Sent Done false
6479 2613 Acked Write 8 Done Sent Done false
6480 2614 Acked Read 9 Done Sent Done false
6481 2615 Acked Read 2 Done Sent Done false
6482 2616 Acked Write 8 Done Sent Done false
6483 2617 Acked Read 2 Done Sent Done false
6484 2618 Acked Write 7 Done Sent Done false
6485 2619 Acked Read 3 Done Sent Done false
6486 2620 Acked Read 4 Done Sent Done false
6487 2621 Acked Flush 0 Done Sent Done false
6488 2622 Acked Write 6 Done Sent Done false
6489 2623 Acked Read 9 Done Sent Done false
6490 2624 Acked Write 8 Done Sent Done false
6491 2625 Acked Read 10 Done Sent Done false
6492 2626 Acked Read 2 Done Sent Done false
6493 2627 Acked Read 8 Done Sent Done false
6494 2628 Acked Read 6 Done Sent Done false
6495 2629 Acked Read 4 Done Sent Done false
6496 2630 Acked Flush 0 Done Sent Done false
6497 2631 Acked Read 2 Done Sent Done false
6498 2632 Acked Write 9 Done Sent Done false
6499 2633 Acked Flush 0 Done Sent Done false
6500 2634 Acked Write 3 Done Sent Done false
6501 2635 Acked Write 4 Done Sent Done false
6502 2636 Acked Read 7 Done Sent Done false
6503 2637 Acked Read 2 Done Sent Done false
6504 2638 Acked Read 9 Done Sent Done false
6505 2639 Acked Write 2 Done Sent Done false
6506 2640 Acked Read 1 Done Sent Done false
6507 2641 Acked Flush 0 Done Sent Done false
6508 2642 Acked Read 2 Done Sent Done false
6509 2643 Acked Flush 0 Done Sent Done false
6510 2644 Acked Write 2 Done Sent Done false
6511 2645 Acked Write 3 Done Sent Done false
6512 2646 Acked Write 3 Done Sent Done false
6513 2647 Acked Read 6 Done Sent Done false
6514 2648 Acked Write 8 Done Sent Done false
6515 2649 Acked Write 3 Done Sent Done false
6516 2650 Acked Read 4 Done Sent Done false
6517 2651 Acked Write 5 Done Sent Done false
6518 2652 Acked Write 3 Done Sent Done false
6519 2653 Acked Flush 0 Done Sent Done false
6520 2654 Acked Write 8 Done Sent Done false
6521 2655 Acked Write 8 Done Sent Done false
6522 2656 Acked Write 10 Done Sent Done false
6523 2657 Acked Write 4 Done Sent Done false
6524 2658 Acked Read 7 Done Sent Done false
6525 2659 Acked Write 3 Done Sent Done false
6526 2660 Acked Read 10 Done Sent Done false
6527 2661 Acked Read 9 Done Sent Done false
6528 2662 Acked Read 7 Done Sent Done false
6529 2663 Acked Read 1 Done Sent Done false
6530 2664 Acked Read 8 Done Sent Done false
6531 2665 Acked Read 1 Done Sent Done false
6532 2666 Acked Read 6 Done Sent Done false
6533 2667 Acked Read 6 Done Sent Done false
6534 2668 Acked Read 4 Done Sent Done false
6535 2669 Acked Read 8 Done Sent Done false
6536 2670 Acked Write 7 Done Sent Done false
6537 2671 Acked Flush 0 Done Sent Done false
6538 2672 Acked Write 4 Done Sent Done false
6539 2673 Acked Read 2 Done Sent Done false
6540 2674 Acked Read 7 Done Sent Done false
6541 2675 Acked Read 3 Done Sent Done false
6542 2676 Acked Write 7 Done Sent Done false
6543 2677 Acked Write 7 Done Sent Done false
6544 2678 Acked Read 9 Done Sent Done false
6545 2679 Acked Write 6 Done Sent Done false
6546 2680 Acked Read 1 Done Sent Done false
6547 2681 Acked Write 8 Done Sent Done false
6548 2682 Acked Read 4 Done Sent Done false
6549 2683 Acked Read 10 Done Sent Done false
6550 2684 Acked Write 6 Done Sent Done false
6551 2685 Acked Read 9 Done Sent Done false
6552 2686 Acked Read 4 Done Sent Done false
6553 2687 Acked Read 6 Done Sent Done false
6554 2688 Acked Read 4 Done Sent Done false
6555 2689 Acked Write 9 Done Sent Done false
6556 2690 Acked Read 9 Done Sent Done false
6557 2691 Acked Read 6 Done Sent Done false
6558 2692 Acked Write 2 Done Sent Done false
6559 2693 Acked Write 1 Done Sent Done false
6560 2694 Acked Read 7 Done Sent Done false
6561 2695 Acked Flush 0 Done Sent Done false
6562 2696 Acked Write 8 Done Sent Done false
6563 2697 Acked Write 2 Done Sent Done false
6564 2698 Acked Read 1 Done Sent Done false
6565 2699 Acked Write 8 Done Sent Done false
6566 2700 Acked Read 10 Done Sent Done false
6567 2701 Acked Write 2 Done Sent Done false
6568 2702 Acked Write 4 Done Sent Done false
6569 2703 Acked Read 2 Done Sent Done false
6570 2704 Acked Read 4 Done Sent Done false
6571 2705 Acked Read 7 Done Sent Done false
6572 2706 Acked Write 9 Done Sent Done false
6573 2707 Acked Write 2 Done Sent Done false
6574 2708 Acked Read 6 Done Sent Done false
6575 2709 Acked Write 4 Done Sent Done false
6576 2710 Acked Read 3 Done Sent Done false
6577 2711 Acked Read 4 Done Sent Done false
6578 2712 Acked Read 10 Done Sent Done false
6579 2713 Acked Write 7 Done Sent Done false
6580 2714 Acked Write 3 Done Sent Done false
6581 2715 Acked Read 9 Done Sent Done false
6582 2716 Acked Flush 0 Done Sent Done false
6583 2717 Acked Write 8 Done Sent Done false
6584 2718 Acked Write 1 Done Sent Done false
6585 2719 Acked Write 7 Done Sent Done false
6586 2720 Acked Write 9 Done Sent Done false
6587 2721 Acked Write 7 Done Sent Done false
6588 2722 Acked Flush 0 Done Sent Done false
6589 2723 Acked Write 9 Done Sent Done false
6590 2724 Acked Write 6 Done Sent Done false
6591 2725 Acked Read 9 Done Sent Done false
6592 2726 Acked Write 2 Done Sent Done false
6593 2727 Acked Read 3 Done Sent Done false
6594 2728 Acked Read 2 Done Sent Done false
6595 2729 Acked Read 2 Done Sent Done false
6596 2730 Acked Flush 0 Done Sent Done false
6597 2731 Acked Read 8 Done Sent Done false
6598 2732 Acked Read 9 Done Sent Done false
6599 2733 Acked Read 1 Done Sent Done false
6600 2734 Acked Read 2 Done Sent Done false
6601 2735 Acked Write 1 Done Sent Done false
6602 2736 Acked Write 4 Done Sent Done false
6603 2737 Acked Write 2 Done Sent Done false
6604 2738 Acked Write 6 Done Sent Done false
6605 2739 Acked Write 10 Done Sent Done false
6606 2740 Acked Write 10 Done Sent Done false
6607 2741 Acked Write 6 Done Sent Done false
6608 2742 Acked Read 10 Done Sent Done false
6609 2743 Acked Write 8 Done Sent Done false
6610 2744 Acked Read 7 Done Sent Done false
6611 2745 Acked Read 4 Done Sent Done false
6612 2746 Acked Write 2 Done Sent Done false
6613 2747 Acked Write 3 Done Sent Done false
6614 2748 Acked Write 10 Done Sent Done false
6615 2749 Acked Write 6 Done Sent Done false
6616 2750 Acked Read 8 Done Sent Done false
6617 2751 Acked Write 10 Done Sent Done false
6618 2752 Acked Write 10 Done Sent Done false
6619 2753 Acked Write 10 Done Sent Done false
6620 2754 Acked Read 8 Done Sent Done false
6621 2755 Acked Write 2 Done Sent Done false
6622 2756 Acked Write 8 Done Sent Done false
6623 2757 Acked Read 1 Done Sent Done false
6624 2758 Acked Write 2 Done Sent Done false
6625 2759 Acked Write 6 Done Sent Done false
6626 2760 Acked Read 8 Done Sent Done false
6627 2761 Acked Read 8 Done Sent Done false
6628 2762 Acked Read 2 Done Sent Done false
6629 2763 Acked Write 7 Done Sent Done false
6630 2764 Acked Read 1 Done Sent Done false
6631 2765 Acked Write 1 Done Sent Done false
6632 2766 Acked Write 6 Done Sent Done false
6633 2767 Acked Read 8 Done Sent Done false
6634 2768 Acked Read 4 Done Sent Done false
6635 2769 Acked Read 1 Done Sent Done false
6636 2770 Acked Write 10 Done Sent Done false
6637 2771 Acked Read 9 Done Sent Done false
6638 2772 Acked Read 5 Done Sent Done false
6639 2773 Acked Write 6 Done Sent Done false
6640 2774 Acked Read 9 Done Sent Done false
6641 2775 Acked Read 8 Done Sent Done false
6642 2776 Acked Read 1 Done Sent Done false
6643 2777 Acked Flush 0 Done Sent Done false
6644 2778 Acked Write 4 Done Sent Done false
6645 2779 Acked Write 5 Done Sent Done false
6646 2780 Acked Write 7 Done Sent Done false
6647 2781 Acked Write 5 Done Sent Done false
6648 2782 Acked Read 5 Done Sent Done false
6649 2783 Acked Read 3 Done Sent Done false
6650 2784 Acked Write 9 Done Sent Done false
6651 2785 Acked Read 3 Done Sent Done false
6652 2786 Acked Read 10 Done Sent Done false
6653 2787 Acked Write 5 Done Sent Done false
6654 2788 Acked Write 2 Done Sent Done false
6655 2789 Acked Read 10 Done Sent Done false
6656 2790 Acked Write 9 Done Sent Done false
6657 2791 Acked Read 5 Done Sent Done false
6658 2792 Acked Read 6 Done Sent Done false
6659 2793 Acked Read 10 Done Sent Done false
6660 2794 Acked Write 10 Done Sent Done false
6661 2795 Acked Write 5 Done Sent Done false
6662 2796 Acked Write 5 Done Sent Done false
6663 2797 Acked Read 10 Done Sent Done false
6664 2798 Acked Read 5 Done Sent Done false
6665 2799 Acked Read 9 Done Sent Done false
6666 2800 Acked Write 8 Done Sent Done false
6667 2801 Acked Write 10 Done Sent Done false
6668 2802 Acked Read 5 Done Sent Done false
6669 2803 Acked Read 1 Done Sent Done false
6670 2804 Acked Write 8 Done Sent Done false
6671 2805 Acked Flush 0 Done Sent Done false
6672 2806 Acked Write 2 Done Sent Done false
6673 2807 Acked Flush 0 Done Sent Done false
6674 2808 Acked Read 4 Done Sent Done false
6675 2809 Acked Write 7 Done Sent Done false
6676 2810 Acked Write 1 Done Sent Done false
6677 2811 Acked Flush 0 Done Sent Done false
6678 STATES DS:0 DS:1 DS:2 TOTAL
6679 Sent 0 302 0 302
6680 Done 302 0 302 604
6681 Skipped 0 0 0 0
6682 Error 0 0 0 0
6683 Last Flush: 2811 2509 2811
6684 Downstairs last five completed: 2509 2508 2507 2506 2505
6685 Upstairs last five completed: 2811 2810 2809 2808 2807
6686 CLIENT: Up:0 ds:302 act:2
66872025-01-11T06:06:49.712ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
66882025-01-11T06:06:49.712ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
66892025-01-11T06:06:49.712ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
66902025-01-11T06:06:49.712ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
66912025-01-11T06:06:49.712ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
6692 ----------------------------------------------------------------
6693 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6694 States: Active Offline Active
6695 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6696 2510 Acked Read 10 Done Sent Done false
6697 2511 Acked Read 4 Done Sent Done false
6698 2512 Acked Read 8 Done Sent Done false
6699 2513 Acked Write 8 Done Sent Done false
6700 2514 Acked Write 10 Done Sent Done false
6701 2515 Acked Flush 0 Done Sent Done false
6702 2516 Acked Write 2 Done Sent Done false
6703 2517 Acked Read 1 Done Sent Done false
6704 2518 Acked Flush 0 Done Sent Done false
6705 2519 Acked Write 5 Done Sent Done false
6706 2520 Acked Write 7 Done Sent Done false
6707 2521 Acked Flush 0 Done Sent Done false
6708 2522 Acked Read 3 Done Sent Done false
6709 2523 Acked Flush 0 Done Sent Done false
6710 2524 Acked Flush 0 Done Sent Done false
6711 2525 Acked Read 7 Done Sent Done false
6712 2526 Acked Flush 0 Done Sent Done false
6713 2527 Acked Write 6 Done Sent Done false
6714 2528 Acked Read 5 Done Sent Done false
6715 2529 Acked Write 9 Done Sent Done false
6716 2530 Acked Read 5 Done Sent Done false
6717 2531 Acked Write 9 Done Sent Done false
6718 2532 Acked Write 8 Done Sent Done false
6719 2533 Acked Write 6 Done Sent Done false
6720 2534 Acked Read 3 Done Sent Done false
6721 2535 Acked Write 3 Done Sent Done false
6722 2536 Acked Write 3 Done Sent Done false
6723 2537 Acked Read 9 Done Sent Done false
6724 2538 Acked Read 5 Done Sent Done false
6725 2539 Acked Read 7 Done Sent Done false
6726 2540 Acked Write 8 Done Sent Done false
6727 2541 Acked Read 7 Done Sent Done false
6728 2542 Acked Write 5 Done Sent Done false
6729 2543 Acked Write 7 Done Sent Done false
6730 2544 Acked Write 8 Done Sent Done false
6731 2545 Acked Read 3 Done Sent Done false
6732 2546 Acked Write 3 Done Sent Done false
6733 2547 Acked Read 3 Done Sent Done false
6734 2548 Acked Write 4 Done Sent Done false
6735 2549 Acked Read 7 Done Sent Done false
6736 2550 Acked Read 7 Done Sent Done false
6737 2551 Acked Read 9 Done Sent Done false
6738 2552 Acked Read 4 Done Sent Done false
6739 2553 Acked Write 8 Done Sent Done false
6740 2554 Acked Read 4 Done Sent Done false
6741 2555 Acked Read 9 Done Sent Done false
6742 2556 Acked Read 4 Done Sent Done false
6743 2557 Acked Write 8 Done Sent Done false
6744 2558 Acked Write 3 Done Sent Done false
6745 2559 Acked Write 2 Done Sent Done false
6746 2560 Acked Read 10 Done Sent Done false
6747 2561 Acked Flush 0 Done Sent Done false
6748 2562 Acked Write 3 Done Sent Done false
6749 2563 Acked Read 1 Done Sent Done false
6750 2564 Acked Flush 0 Done Sent Done false
6751 2565 Acked Read 4 Done Sent Done false
6752 2566 Acked Flush 0 Done Sent Done false
6753 2567 Acked Write 9 Done Sent Done false
6754 2568 Acked Write 8 Done Sent Done false
6755 2569 Acked Read 1 Done Sent Done false
6756 2570 Acked Write 8 Done Sent Done false
6757 2571 Acked Read 5 Done Sent Done false
6758 2572 Acked Read 4 Done Sent Done false
6759 2573 Acked Read 7 Done Sent Done false
6760 2574 Acked Write 8 Done Sent Done false
6761 2575 Acked Read 3 Done Sent Done false
6762 2576 Acked Write 3 Done Sent Done false
6763 2577 Acked Read 4 Done Sent Done false
6764 2578 Acked Write 10 Done Sent Done false
6765 2579 Acked Flush 0 Done Sent Done false
6766 2580 Acked Write 6 Done Sent Done false
6767 2581 Acked Write 4 Done Sent Done false
6768 2582 Acked Write 10 Done Sent Done false
6769 2583 Acked Write 1 Done Sent Done false
6770 2584 Acked Write 5 Done Sent Done false
6771 2585 Acked Write 3 Done Sent Done false
6772 2586 Acked Write 4 Done Sent Done false
6773 2587 Acked Write 9 Done Sent Done false
6774 2588 Acked Write 1 Done Sent Done false
6775 2589 Acked Write 8 Done Sent Done false
6776 2590 Acked Write 2 Done Sent Done false
6777 2591 Acked Read 5 Done Sent Done false
6778 2592 Acked Read 6 Done Sent Done false
6779 2593 Acked Read 10 Done Sent Done false
6780 2594 Acked Read 7 Done Sent Done false
6781 2595 Acked Write 9 Done Sent Done false
6782 2596 Acked Read 7 Done Sent Done false
6783 2597 Acked Read 5 Done Sent Done false
6784 2598 Acked Write 8 Done Sent Done false
6785 2599 Acked Write 4 Done Sent Done false
6786 2600 Acked Write 3 Done Sent Done false
6787 2601 Acked Read 8 Done Sent Done false
6788 2602 Acked Read 3 Done Sent Done false
6789 2603 Acked Read 2 Done Sent Done false
6790 2604 Acked Write 10 Done Sent Done false
6791 2605 Acked Write 4 Done Sent Done false
6792 2606 Acked Read 7 Done Sent Done false
6793 2607 Acked Read 4 Done Sent Done false
6794 2608 Acked Read 10 Done Sent Done false
6795 2609 Acked Read 7 Done Sent Done false
6796 2610 Acked Write 9 Done Sent Done false
6797 2611 Acked Read 9 Done Sent Done false
6798 2612 Acked Flush 0 Done Sent Done false
6799 2613 Acked Write 8 Done Sent Done false
6800 2614 Acked Read 9 Done Sent Done false
6801 2615 Acked Read 2 Done Sent Done false
6802 2616 Acked Write 8 Done Sent Done false
6803 2617 Acked Read 2 Done Sent Done false
6804 2618 Acked Write 7 Done Sent Done false
6805 2619 Acked Read 3 Done Sent Done false
6806 2620 Acked Read 4 Done Sent Done false
6807 2621 Acked Flush 0 Done Sent Done false
6808 2622 Acked Write 6 Done Sent Done false
6809 2623 Acked Read 9 Done Sent Done false
6810 2624 Acked Write 8 Done Sent Done false
6811 2625 Acked Read 10 Done Sent Done false
6812 2626 Acked Read 2 Done Sent Done false
6813 2627 Acked Read 8 Done Sent Done false
6814 2628 Acked Read 6 Done Sent Done false
6815 2629 Acked Read 4 Done Sent Done false
6816 2630 Acked Flush 0 Done Sent Done false
6817 2631 Acked Read 2 Done Sent Done false
6818 2632 Acked Write 9 Done Sent Done false
6819 2633 Acked Flush 0 Done Sent Done false
6820 2634 Acked Write 3 Done Sent Done false
6821 2635 Acked Write 4 Done Sent Done false
6822 2636 Acked Read 7 Done Sent Done false
6823 2637 Acked Read 2 Done Sent Done false
6824 2638 Acked Read 9 Done Sent Done false
6825 2639 Acked Write 2 Done Sent Done false
6826 2640 Acked Read 1 Done Sent Done false
6827 2641 Acked Flush 0 Done Sent Done false
6828 2642 Acked Read 2 Done Sent Done false
6829 2643 Acked Flush 0 Done Sent Done false
6830 2644 Acked Write 2 Done Sent Done false
6831 2645 Acked Write 3 Done Sent Done false
6832 2646 Acked Write 3 Done Sent Done false
6833 2647 Acked Read 6 Done Sent Done false
6834 2648 Acked Write 8 Done Sent Done false
6835 2649 Acked Write 3 Done Sent Done false
6836 2650 Acked Read 4 Done Sent Done false
6837 2651 Acked Write 5 Done Sent Done false
6838 2652 Acked Write 3 Done Sent Done false
6839 2653 Acked Flush 0 Done Sent Done false
6840 2654 Acked Write 8 Done Sent Done false
6841 2655 Acked Write 8 Done Sent Done false
6842 2656 Acked Write 10 Done Sent Done false
6843 2657 Acked Write 4 Done Sent Done false
6844 2658 Acked Read 7 Done Sent Done false
6845 2659 Acked Write 3 Done Sent Done false
6846 2660 Acked Read 10 Done Sent Done false
6847 2661 Acked Read 9 Done Sent Done false
6848 2662 Acked Read 7 Done Sent Done false
6849 2663 Acked Read 1 Done Sent Done false
6850 2664 Acked Read 8 Done Sent Done false
6851 2665 Acked Read 1 Done Sent Done false
6852 2666 Acked Read 6 Done Sent Done false
6853 2667 Acked Read 6 Done Sent Done false
6854 2668 Acked Read 4 Done Sent Done false
6855 2669 Acked Read 8 Done Sent Done false
6856 2670 Acked Write 7 Done Sent Done false
6857 2671 Acked Flush 0 Done Sent Done false
6858 2672 Acked Write 4 Done Sent Done false
6859 2673 Acked Read 2 Done Sent Done false
6860 2674 Acked Read 7 Done Sent Done false
6861 2675 Acked Read 3 Done Sent Done false
6862 2676 Acked Write 7 Done Sent Done false
6863 2677 Acked Write 7 Done Sent Done false
6864 2678 Acked Read 9 Done Sent Done false
6865 2679 Acked Write 6 Done Sent Done false
6866 2680 Acked Read 1 Done Sent Done false
6867 2681 Acked Write 8 Done Sent Done false
6868 2682 Acked Read 4 Done Sent Done false
6869 2683 Acked Read 10 Done Sent Done false
6870 2684 Acked Write 6 Done Sent Done false
6871 2685 Acked Read 9 Done Sent Done false
6872 2686 Acked Read 4 Done Sent Done false
6873 2687 Acked Read 6 Done Sent Done false
6874 2688 Acked Read 4 Done Sent Done false
6875 2689 Acked Write 9 Done Sent Done false
6876 2690 Acked Read 9 Done Sent Done false
6877 2691 Acked Read 6 Done Sent Done false
6878 2692 Acked Write 2 Done Sent Done false
6879 2693 Acked Write 1 Done Sent Done false
6880 2694 Acked Read 7 Done Sent Done false
6881 2695 Acked Flush 0 Done Sent Done false
6882 2696 Acked Write 8 Done Sent Done false
6883 2697 Acked Write 2 Done Sent Done false
6884 2698 Acked Read 1 Done Sent Done false
6885 2699 Acked Write 8 Done Sent Done false
6886 2700 Acked Read 10 Done Sent Done false
6887 2701 Acked Write 2 Done Sent Done false
6888 2702 Acked Write 4 Done Sent Done false
6889 2703 Acked Read 2 Done Sent Done false
6890 2704 Acked Read 4 Done Sent Done false
6891 2705 Acked Read 7 Done Sent Done false
6892 2706 Acked Write 9 Done Sent Done false
6893 2707 Acked Write 2 Done Sent Done false
6894 2708 Acked Read 6 Done Sent Done false
6895 2709 Acked Write 4 Done Sent Done false
6896 2710 Acked Read 3 Done Sent Done false
6897 2711 Acked Read 4 Done Sent Done false
6898 2712 Acked Read 10 Done Sent Done false
6899 2713 Acked Write 7 Done Sent Done false
6900 2714 Acked Write 3 Done Sent Done false
6901 2715 Acked Read 9 Done Sent Done false
6902 2716 Acked Flush 0 Done Sent Done false
6903 2717 Acked Write 8 Done Sent Done false
6904 2718 Acked Write 1 Done Sent Done false
6905 2719 Acked Write 7 Done Sent Done false
6906 2720 Acked Write 9 Done Sent Done false
6907 2721 Acked Write 7 Done Sent Done false
6908 2722 Acked Flush 0 Done Sent Done false
6909 2723 Acked Write 9 Done Sent Done false
6910 2724 Acked Write 6 Done Sent Done false
6911 2725 Acked Read 9 Done Sent Done false
6912 2726 Acked Write 2 Done Sent Done false
6913 2727 Acked Read 3 Done Sent Done false
6914 2728 Acked Read 2 Done Sent Done false
6915 2729 Acked Read 2 Done Sent Done false
6916 2730 Acked Flush 0 Done Sent Done false
6917 2731 Acked Read 8 Done Sent Done false
6918 2732 Acked Read 9 Done Sent Done false
6919 2733 Acked Read 1 Done Sent Done false
6920 2734 Acked Read 2 Done Sent Done false
6921 2735 Acked Write 1 Done Sent Done false
6922 2736 Acked Write 4 Done Sent Done false
6923 2737 Acked Write 2 Done Sent Done false
6924 2738 Acked Write 6 Done Sent Done false
6925 2739 Acked Write 10 Done Sent Done false
6926 2740 Acked Write 10 Done Sent Done false
6927 2741 Acked Write 6 Done Sent Done false
6928 2742 Acked Read 10 Done Sent Done false
6929 2743 Acked Write 8 Done Sent Done false
6930 2744 Acked Read 7 Done Sent Done false
6931 2745 Acked Read 4 Done Sent Done false
6932 2746 Acked Write 2 Done Sent Done false
6933 2747 Acked Write 3 Done Sent Done false
6934 2748 Acked Write 10 Done Sent Done false
6935 2749 Acked Write 6 Done Sent Done false
6936 2750 Acked Read 8 Done Sent Done false
6937 2751 Acked Write 10 Done Sent Done false
6938 2752 Acked Write 10 Done Sent Done false
6939 2753 Acked Write 10 Done Sent Done false
6940 2754 Acked Read 8 Done Sent Done false
6941 2755 Acked Write 2 Done Sent Done false
6942 2756 Acked Write 8 Done Sent Done false
6943 2757 Acked Read 1 Done Sent Done false
6944 2758 Acked Write 2 Done Sent Done false
6945 2759 Acked Write 6 Done Sent Done false
6946 2760 Acked Read 8 Done Sent Done false
6947 2761 Acked Read 8 Done Sent Done false
6948 2762 Acked Read 2 Done Sent Done false
6949 2763 Acked Write 7 Done Sent Done false
6950 2764 Acked Read 1 Done Sent Done false
6951 2765 Acked Write 1 Done Sent Done false
6952 2766 Acked Write 6 Done Sent Done false
6953 2767 Acked Read 8 Done Sent Done false
6954 2768 Acked Read 4 Done Sent Done false
6955 2769 Acked Read 1 Done Sent Done false
6956 2770 Acked Write 10 Done Sent Done false
6957 2771 Acked Read 9 Done Sent Done false
6958 2772 Acked Read 5 Done Sent Done false
6959 2773 Acked Write 6 Done Sent Done false
6960 2774 Acked Read 9 Done Sent Done false
6961 2775 Acked Read 8 Done Sent Done false
6962 2776 Acked Read 1 Done Sent Done false
6963 2777 Acked Flush 0 Done Sent Done false
6964 2778 Acked Write 4 Done Sent Done false
6965 2779 Acked Write 5 Done Sent Done false
6966 2780 Acked Write 7 Done Sent Done false
6967 2781 Acked Write 5 Done Sent Done false
6968 2782 Acked Read 5 Done Sent Done false
6969 2783 Acked Read 3 Done Sent Done false
6970 2784 Acked Write 9 Done Sent Done false
6971 2785 Acked Read 3 Done Sent Done false
6972 2786 Acked Read 10 Done Sent Done false
6973 2787 Acked Write 5 Done Sent Done false
6974 2788 Acked Write 2 Done Sent Done false
6975 2789 Acked Read 10 Done Sent Done false
6976 2790 Acked Write 9 Done Sent Done false
6977 2791 Acked Read 5 Done Sent Done false
6978 2792 Acked Read 6 Done Sent Done false
6979 2793 Acked Read 10 Done Sent Done false
6980 2794 Acked Write 10 Done Sent Done false
6981 2795 Acked Write 5 Done Sent Done false
6982 2796 Acked Write 5 Done Sent Done false
6983 2797 Acked Read 10 Done Sent Done false
6984 2798 Acked Read 5 Done Sent Done false
6985 2799 Acked Read 9 Done Sent Done false
6986 2800 Acked Write 8 Done Sent Done false
6987 2801 Acked Write 10 Done Sent Done false
6988 2802 Acked Read 5 Done Sent Done false
6989 2803 Acked Read 1 Done Sent Done false
6990 2804 Acked Write 8 Done Sent Done false
6991 2805 Acked Flush 0 Done Sent Done false
6992 2806 Acked Write 2 Done Sent Done false
6993 2807 Acked Flush 0 Done Sent Done false
6994 2808 Acked Read 4 Done Sent Done false
6995 2809 Acked Write 7 Done Sent Done false
6996 2810 Acked Write 1 Done Sent Done false
6997 2811 Acked Flush 0 Done Sent Done false
6998 STATES DS:0 DS:1 DS:2 TOTAL
6999 Sent 0 302 0 302
7000 Done 302 0 302 604
7001 Skipped 0 0 0 0
7002 Error 0 0 0 0
7003 Last Flush: 2811 2509 2811
7004 Downstairs last five completed: 2509 2508 2507 2506 2505
7005 Upstairs last five completed: 2811 2810 2809 2808 2807
7006 CLIENT: Up:0 ds:302 act:2
7007 [8820] got start action
7008 [8820] I am going to restart
7009 Starting downstairs at port 8820
7010 Make output file at "/tmp/dsc/downstairs-8820.txt"
7011 [8820][1] reports Starting
7012 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1210)
7013 [8820][1] reports Running
7014 ----------------------------------------------------------------
7015 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7016 States: Active Offline Active
7017 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7018 2510 Acked Read 10 Done Sent Done false
7019 2511 Acked Read 4 Done Sent Done false
7020 2512 Acked Read 8 Done Sent Done false
7021 2513 Acked Write 8 Done Sent Done false
7022 2514 Acked Write 10 Done Sent Done false
7023 2515 Acked Flush 0 Done Sent Done false
7024 2516 Acked Write 2 Done Sent Done false
7025 2517 Acked Read 1 Done Sent Done false
7026 2518 Acked Flush 0 Done Sent Done false
7027 2519 Acked Write 5 Done Sent Done false
7028 2520 Acked Write 7 Done Sent Done false
7029 2521 Acked Flush 0 Done Sent Done false
7030 2522 Acked Read 3 Done Sent Done false
7031 2523 Acked Flush 0 Done Sent Done false
7032 2524 Acked Flush 0 Done Sent Done false
7033 2525 Acked Read 7 Done Sent Done false
7034 2526 Acked Flush 0 Done Sent Done false
7035 2527 Acked Write 6 Done Sent Done false
7036 2528 Acked Read 5 Done Sent Done false
7037 2529 Acked Write 9 Done Sent Done false
7038 2530 Acked Read 5 Done Sent Done false
7039 2531 Acked Write 9 Done Sent Done false
7040 2532 Acked Write 8 Done Sent Done false
7041 2533 Acked Write 6 Done Sent Done false
7042 2534 Acked Read 3 Done Sent Done false
7043 2535 Acked Write 3 Done Sent Done false
7044 2536 Acked Write 3 Done Sent Done false
7045 2537 Acked Read 9 Done Sent Done false
7046 2538 Acked Read 5 Done Sent Done false
7047 2539 Acked Read 7 Done Sent Done false
7048 2540 Acked Write 8 Done Sent Done false
7049 2541 Acked Read 7 Done Sent Done false
7050 2542 Acked Write 5 Done Sent Done false
7051 2543 Acked Write 7 Done Sent Done false
7052 2544 Acked Write 8 Done Sent Done false
7053 2545 Acked Read 3 Done Sent Done false
7054 2546 Acked Write 3 Done Sent Done false
7055 2547 Acked Read 3 Done Sent Done false
7056 2548 Acked Write 4 Done Sent Done false
7057 2549 Acked Read 7 Done Sent Done false
7058 2550 Acked Read 7 Done Sent Done false
7059 2551 Acked Read 9 Done Sent Done false
7060 2552 Acked Read 4 Done Sent Done false
7061 2553 Acked Write 8 Done Sent Done false
7062 2554 Acked Read 4 Done Sent Done false
7063 2555 Acked Read 9 Done Sent Done false
7064 2556 Acked Read 4 Done Sent Done false
7065 2557 Acked Write 8 Done Sent Done false
7066 2558 Acked Write 3 Done Sent Done false
7067 2559 Acked Write 2 Done Sent Done false
7068 2560 Acked Read 10 Done Sent Done false
7069 2561 Acked Flush 0 Done Sent Done false
7070 2562 Acked Write 3 Done Sent Done false
7071 2563 Acked Read 1 Done Sent Done false
7072 2564 Acked Flush 0 Done Sent Done false
7073 2565 Acked Read 4 Done Sent Done false
7074 2566 Acked Flush 0 Done Sent Done false
7075 2567 Acked Write 9 Done Sent Done false
7076 2568 Acked Write 8 Done Sent Done false
7077 2569 Acked Read 1 Done Sent Done false
7078 2570 Acked Write 8 Done Sent Done false
7079 2571 Acked Read 5 Done Sent Done false
7080 2572 Acked Read 4 Done Sent Done false
7081 2573 Acked Read 7 Done Sent Done false
7082 2574 Acked Write 8 Done Sent Done false
7083 2575 Acked Read 3 Done Sent Done false
7084 2576 Acked Write 3 Done Sent Done false
7085 2577 Acked Read 4 Done Sent Done false
7086 2578 Acked Write 10 Done Sent Done false
7087 2579 Acked Flush 0 Done Sent Done false
7088 2580 Acked Write 6 Done Sent Done false
7089 2581 Acked Write 4 Done Sent Done false
7090 2582 Acked Write 10 Done Sent Done false
7091 2583 Acked Write 1 Done Sent Done false
7092 2584 Acked Write 5 Done Sent Done false
7093 2585 Acked Write 3 Done Sent Done false
7094 2586 Acked Write 4 Done Sent Done false
7095 2587 Acked Write 9 Done Sent Done false
7096 2588 Acked Write 1 Done Sent Done false
7097 2589 Acked Write 8 Done Sent Done false
7098 2590 Acked Write 2 Done Sent Done false
7099 2591 Acked Read 5 Done Sent Done false
7100 2592 Acked Read 6 Done Sent Done false
7101 2593 Acked Read 10 Done Sent Done false
7102 2594 Acked Read 7 Done Sent Done false
7103 2595 Acked Write 9 Done Sent Done false
7104 2596 Acked Read 7 Done Sent Done false
7105 2597 Acked Read 5 Done Sent Done false
7106 2598 Acked Write 8 Done Sent Done false
7107 2599 Acked Write 4 Done Sent Done false
7108 2600 Acked Write 3 Done Sent Done false
7109 2601 Acked Read 8 Done Sent Done false
7110 2602 Acked Read 3 Done Sent Done false
7111 2603 Acked Read 2 Done Sent Done false
7112 2604 Acked Write 10 Done Sent Done false
7113 2605 Acked Write 4 Done Sent Done false
7114 2606 Acked Read 7 Done Sent Done false
7115 2607 Acked Read 4 Done Sent Done false
7116 2608 Acked Read 10 Done Sent Done false
7117 2609 Acked Read 7 Done Sent Done false
7118 2610 Acked Write 9 Done Sent Done false
7119 2611 Acked Read 9 Done Sent Done false
7120 2612 Acked Flush 0 Done Sent Done false
7121 2613 Acked Write 8 Done Sent Done false
7122 2614 Acked Read 9 Done Sent Done false
7123 2615 Acked Read 2 Done Sent Done false
7124 2616 Acked Write 8 Done Sent Done false
7125 2617 Acked Read 2 Done Sent Done false
7126 2618 Acked Write 7 Done Sent Done false
7127 2619 Acked Read 3 Done Sent Done false
7128 2620 Acked Read 4 Done Sent Done false
7129 2621 Acked Flush 0 Done Sent Done false
7130 2622 Acked Write 6 Done Sent Done false
7131 2623 Acked Read 9 Done Sent Done false
7132 2624 Acked Write 8 Done Sent Done false
7133 2625 Acked Read 10 Done Sent Done false
7134 2626 Acked Read 2 Done Sent Done false
7135 2627 Acked Read 8 Done Sent Done false
7136 2628 Acked Read 6 Done Sent Done false
7137 2629 Acked Read 4 Done Sent Done false
7138 2630 Acked Flush 0 Done Sent Done false
7139 2631 Acked Read 2 Done Sent Done false
7140 2632 Acked Write 9 Done Sent Done false
7141 2633 Acked Flush 0 Done Sent Done false
7142 2634 Acked Write 3 Done Sent Done false
7143 2635 Acked Write 4 Done Sent Done false
7144 2636 Acked Read 7 Done Sent Done false
7145 2637 Acked Read 2 Done Sent Done false
7146 2638 Acked Read 9 Done Sent Done false
7147 2639 Acked Write 2 Done Sent Done false
7148 2640 Acked Read 1 Done Sent Done false
7149 2641 Acked Flush 0 Done Sent Done false
7150 2642 Acked Read 2 Done Sent Done false
7151 2643 Acked Flush 0 Done Sent Done false
7152 2644 Acked Write 2 Done Sent Done false
7153 2645 Acked Write 3 Done Sent Done false
7154 2646 Acked Write 3 Done Sent Done false
7155 2647 Acked Read 6 Done Sent Done false
7156 2648 Acked Write 8 Done Sent Done false
7157 2649 Acked Write 3 Done Sent Done false
7158 2650 Acked Read 4 Done Sent Done false
7159 2651 Acked Write 5 Done Sent Done false
7160 2652 Acked Write 3 Done Sent Done false
7161 2653 Acked Flush 0 Done Sent Done false
7162 2654 Acked Write 8 Done Sent Done false
7163 2655 Acked Write 8 Done Sent Done false
7164 2656 Acked Write 10 Done Sent Done false
7165 2657 Acked Write 4 Done Sent Done false
7166 2658 Acked Read 7 Done Sent Done false
7167 2659 Acked Write 3 Done Sent Done false
7168 2660 Acked Read 10 Done Sent Done false
7169 2661 Acked Read 9 Done Sent Done false
7170 2662 Acked Read 7 Done Sent Done false
7171 2663 Acked Read 1 Done Sent Done false
7172 2664 Acked Read 8 Done Sent Done false
7173 2665 Acked Read 1 Done Sent Done false
7174 2666 Acked Read 6 Done Sent Done false
7175 2667 Acked Read 6 Done Sent Done false
7176 2668 Acked Read 4 Done Sent Done false
7177 2669 Acked Read 8 Done Sent Done false
7178 2670 Acked Write 7 Done Sent Done false
7179 2671 Acked Flush 0 Done Sent Done false
7180 2672 Acked Write 4 Done Sent Done false
7181 2673 Acked Read 2 Done Sent Done false
7182 2674 Acked Read 7 Done Sent Done false
7183 2675 Acked Read 3 Done Sent Done false
7184 2676 Acked Write 7 Done Sent Done false
7185 2677 Acked Write 7 Done Sent Done false
7186 2678 Acked Read 9 Done Sent Done false
7187 2679 Acked Write 6 Done Sent Done false
7188 2680 Acked Read 1 Done Sent Done false
7189 2681 Acked Write 8 Done Sent Done false
7190 2682 Acked Read 4 Done Sent Done false
7191 2683 Acked Read 10 Done Sent Done false
7192 2684 Acked Write 6 Done Sent Done false
7193 2685 Acked Read 9 Done Sent Done false
7194 2686 Acked Read 4 Done Sent Done false
7195 2687 Acked Read 6 Done Sent Done false
7196 2688 Acked Read 4 Done Sent Done false
7197 2689 Acked Write 9 Done Sent Done false
7198 2690 Acked Read 9 Done Sent Done false
7199 2691 Acked Read 6 Done Sent Done false
7200 2692 Acked Write 2 Done Sent Done false
7201 2693 Acked Write 1 Done Sent Done false
7202 2694 Acked Read 7 Done Sent Done false
7203 2695 Acked Flush 0 Done Sent Done false
7204 2696 Acked Write 8 Done Sent Done false
7205 2697 Acked Write 2 Done Sent Done false
7206 2698 Acked Read 1 Done Sent Done false
7207 2699 Acked Write 8 Done Sent Done false
7208 2700 Acked Read 10 Done Sent Done false
7209 2701 Acked Write 2 Done Sent Done false
7210 2702 Acked Write 4 Done Sent Done false
7211 2703 Acked Read 2 Done Sent Done false
7212 2704 Acked Read 4 Done Sent Done false
7213 2705 Acked Read 7 Done Sent Done false
7214 2706 Acked Write 9 Done Sent Done false
7215 2707 Acked Write 2 Done Sent Done false
7216 2708 Acked Read 6 Done Sent Done false
7217 2709 Acked Write 4 Done Sent Done false
7218 2710 Acked Read 3 Done Sent Done false
7219 2711 Acked Read 4 Done Sent Done false
7220 2712 Acked Read 10 Done Sent Done false
7221 2713 Acked Write 7 Done Sent Done false
7222 2714 Acked Write 3 Done Sent Done false
7223 2715 Acked Read 9 Done Sent Done false
7224 2716 Acked Flush 0 Done Sent Done false
7225 2717 Acked Write 8 Done Sent Done false
7226 2718 Acked Write 1 Done Sent Done false
7227 2719 Acked Write 7 Done Sent Done false
7228 2720 Acked Write 9 Done Sent Done false
7229 2721 Acked Write 7 Done Sent Done false
7230 2722 Acked Flush 0 Done Sent Done false
7231 2723 Acked Write 9 Done Sent Done false
7232 2724 Acked Write 6 Done Sent Done false
7233 2725 Acked Read 9 Done Sent Done false
7234 2726 Acked Write 2 Done Sent Done false
7235 2727 Acked Read 3 Done Sent Done false
7236 2728 Acked Read 2 Done Sent Done false
7237 2729 Acked Read 2 Done Sent Done false
7238 2730 Acked Flush 0 Done Sent Done false
7239 2731 Acked Read 8 Done Sent Done false
7240 2732 Acked Read 9 Done Sent Done false
7241 2733 Acked Read 1 Done Sent Done false
7242 2734 Acked Read 2 Done Sent Done false
7243 2735 Acked Write 1 Done Sent Done false
7244 2736 Acked Write 4 Done Sent Done false
7245 2737 Acked Write 2 Done Sent Done false
7246 2738 Acked Write 6 Done Sent Done false
7247 2739 Acked Write 10 Done Sent Done false
7248 2740 Acked Write 10 Done Sent Done false
7249 2741 Acked Write 6 Done Sent Done false
7250 2742 Acked Read 10 Done Sent Done false
7251 2743 Acked Write 8 Done Sent Done false
7252 2744 Acked Read 7 Done Sent Done false
7253 2745 Acked Read 4 Done Sent Done false
7254 2746 Acked Write 2 Done Sent Done false
7255 2747 Acked Write 3 Done Sent Done false
7256 2748 Acked Write 10 Done Sent Done false
7257 2749 Acked Write 6 Done Sent Done false
7258 2750 Acked Read 8 Done Sent Done false
7259 2751 Acked Write 10 Done Sent Done false
7260 2752 Acked Write 10 Done Sent Done false
7261 2753 Acked Write 10 Done Sent Done false
7262 2754 Acked Read 8 Done Sent Done false
7263 2755 Acked Write 2 Done Sent Done false
7264 2756 Acked Write 8 Done Sent Done false
7265 2757 Acked Read 1 Done Sent Done false
7266 2758 Acked Write 2 Done Sent Done false
7267 2759 Acked Write 6 Done Sent Done false
7268 2760 Acked Read 8 Done Sent Done false
7269 2761 Acked Read 8 Done Sent Done false
7270 2762 Acked Read 2 Done Sent Done false
7271 2763 Acked Write 7 Done Sent Done false
7272 2764 Acked Read 1 Done Sent Done false
7273 2765 Acked Write 1 Done Sent Done false
7274 2766 Acked Write 6 Done Sent Done false
7275 2767 Acked Read 8 Done Sent Done false
7276 2768 Acked Read 4 Done Sent Done false
7277 2769 Acked Read 1 Done Sent Done false
7278 2770 Acked Write 10 Done Sent Done false
7279 2771 Acked Read 9 Done Sent Done false
7280 2772 Acked Read 5 Done Sent Done false
7281 2773 Acked Write 6 Done Sent Done false
7282 2774 Acked Read 9 Done Sent Done false
7283 2775 Acked Read 8 Done Sent Done false
7284 2776 Acked Read 1 Done Sent Done false
7285 2777 Acked Flush 0 Done Sent Done false
7286 2778 Acked Write 4 Done Sent Done false
7287 2779 Acked Write 5 Done Sent Done false
7288 2780 Acked Write 7 Done Sent Done false
7289 2781 Acked Write 5 Done Sent Done false
7290 2782 Acked Read 5 Done Sent Done false
7291 2783 Acked Read 3 Done Sent Done false
7292 2784 Acked Write 9 Done Sent Done false
7293 2785 Acked Read 3 Done Sent Done false
7294 2786 Acked Read 10 Done Sent Done false
7295 2787 Acked Write 5 Done Sent Done false
7296 2788 Acked Write 2 Done Sent Done false
7297 2789 Acked Read 10 Done Sent Done false
7298 2790 Acked Write 9 Done Sent Done false
7299 2791 Acked Read 5 Done Sent Done false
7300 2792 Acked Read 6 Done Sent Done false
7301 2793 Acked Read 10 Done Sent Done false
7302 2794 Acked Write 10 Done Sent Done false
7303 2795 Acked Write 5 Done Sent Done false
7304 2796 Acked Write 5 Done Sent Done false
7305 2797 Acked Read 10 Done Sent Done false
7306 2798 Acked Read 5 Done Sent Done false
7307 2799 Acked Read 9 Done Sent Done false
7308 2800 Acked Write 8 Done Sent Done false
7309 2801 Acked Write 10 Done Sent Done false
7310 2802 Acked Read 5 Done Sent Done false
7311 2803 Acked Read 1 Done Sent Done false
7312 2804 Acked Write 8 Done Sent Done false
7313 2805 Acked Flush 0 Done Sent Done false
7314 2806 Acked Write 2 Done Sent Done false
7315 2807 Acked Flush 0 Done Sent Done false
7316 2808 Acked Read 4 Done Sent Done false
7317 2809 Acked Write 7 Done Sent Done false
7318 2810 Acked Write 1 Done Sent Done false
7319 2811 Acked Flush 0 Done Sent Done false
7320 STATES DS:0 DS:1 DS:2 TOTAL
7321 Sent 0 302 0 302
7322 Done 302 0 302 604
7323 Skipped 0 0 0 0
7324 Error 0 0 0 0
7325 Last Flush: 2811 2509 2811
7326 Downstairs last five completed: 2509 2508 2507 2506 2505
7327 Upstairs last five completed: 2811 2810 2809 2808 2807
7328 CLIENT: Up:0 ds:302 act:2
73292025-01-11T06:06:59.714ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73302025-01-11T06:06:59.714ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47939) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73312025-01-11T06:06:59.715ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73322025-01-11T06:06:59.716ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73332025-01-11T06:06:59.716ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73342025-01-11T06:06:59.716ZINFOcrucible: send last flush ID to this DS: 2509 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73352025-01-11T06:06:59.716ZINFOcrucible: Replied this last flush ID: 2509 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73362025-01-11T06:06:59.716ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73372025-01-11T06:06:59.716ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73382025-01-11T06:06:59.716ZINFOcrucible: [1] client re-new 302 jobs since flush 2509 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73392025-01-11T06:06:59.717ZINFOcrucible: [1] Replayed 302 jobs since flush: 2509 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
7340 ----------------------------------------------------------------
7341 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7342 Downstairs last five completed: 2811 2810 2809 2808 2807
7343 Upstairs last five completed: 2811 2810 2809 2808 2807
7344 CLIENT: Up:0 ds:0 act:3
7345 Replay: All jobs finished
7346 Main task has work to do, go find it
7347 got dsc Stop(2)
7348 stop 2
7349 [8830] Got stop action so:false kr:false
7350 Jan 11 06:06:59.963 INFO request completed, latency_us: 381, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 8e5979f8-2975-4ae5-b8c9-8deb2c1aea6a, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
73512025-01-11T06:06:59.964ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73522025-01-11T06:06:59.964ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73532025-01-11T06:06:59.964ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
73542025-01-11T06:06:59.964ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
7355 Jan 11 06:06:59.964 INFO request completed, latency_us: {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:06:59.96455426Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
7356 126, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7f81c26c-4cd4-4fd4-901b-f9bf78d9f4d1, remote_addr: 127.0.0.1:59925, local_addr: 127.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:06:59.964596125Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
7357 0.0.1:9998
7358 [8830] Exited with: ExitStatus(unix_wait_status(9))
7359 [8830][2] reports Exit
7360 Jan 11 06:07:03.967 INFO request completed, latency_us: 314, response_code: 200, uri: /state/cid/2, method: GET, req_id: 02770750-7a4c-407c-916c-98940abf3c0c, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
7361 001/300 Read block 2584 len 40960
7362 002/300 Flush
7363 003/300 Write block 3524 len 32768 data: 3 3 3 3 3 3 3 3
7364 004/300 Read block 1924 len 24576
7365 005/300 Read block 829 len 8192
7366 006/300 Flush
7367 007/300 Write block 2763 len 12288 data: 3 2 2
7368 008/300 Read block 3708 len 32768
7369 009/300 Write block 3227 len 28672 data: 3 3 2 2 2 2 2
7370 010/300 Read block 603 len 4096
7371 011/300 Write block 3430 len 32768 data: 2 2 2 2 2 2 2 2
7372 012/300 Read block 3660 len 16384
7373 013/300 Read block 4052 len 16384
7374 014/300 Flush
7375 015/300 Read block 129 len 32768
7376 016/300 Read block 4003 len 16384
7377 017/300 Read block 4601 len 40960
7378 018/300 Flush
7379 019/300 Write block 273 len 36864 data: 3 3 3 3 3 4 4 3 4
7380 020/300 Read block 4330 len 24576
7381 021/300 Write block 1979 len 4096 data: 3
7382 022/300 Write block 3612 len 16384 data: 2 2 2 2
7383 023/300 Write block 740 len 24576 data: 2 2 2 4 5 5
7384 024/300 Write block 3349 len 8192 data: 2 2
7385 025/300 Read block 2178 len 8192
7386 026/300 Write block 3293 len 4096 data: 2
7387 027/300 Read block 25 len 8192
7388 028/300 Read block 4728 len 8192
7389 029/300 Read block 2894 len 24576
7390 030/300 Read block 58 len 32768
7391 031/300 Write block 3345 len 36864 data: 2 2 2 2 3 3 2 2 2
7392 032/300 Write block 4616 len 36864 data: 2 2 3 3 3 3 3 2 2
7393 033/300 Write block 3366 len 24576 data: 4 4 4 4 3 2
7394 034/300 Write block 3424 len 8192 data: 3 4
7395 035/300 Read block 1877 len 20480
7396 036/300 Write block 654 len 28672 data: 3 4 4 4 3 3 2
7397 037/300 Read block 1566 len 32768
7398 038/300 Write block 329 len 16384 data: 5 5 5 4
7399 039/300 Flush
7400 040/300 Write block 2258 len 8192 data: 3 3
7401 041/300 Read block 939 len 36864
7402 042/300 Read block 1266 len 40960
7403 043/300 Read block 2272 len 32768
7404 044/300 Read block 4578 len 12288
7405 045/300 Flush
7406 046/300 Write block 2520 len 36864 data: 3 2 3 3 3 3 2 3 3
7407 047/300 Read block 2452 len 36864
7408 048/300 Read block 20 len 16384
7409 049/300 Read block 569 len 4096
7410 050/300 Write block 4029 len 40960 data: 3 2 2 2 2 2 2 2 3 3
7411 051/300 Read block 2667 len 20480
7412 052/300 Write block 528 len 4096 data: 2
7413 053/300 Write block 40 len 28672 data: 4 3 3 3 3 2 2
7414 054/300 Write block 2463 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7415 055/300 Write block 3277 len 28672 data: 4 4 4 3 3 3 3
7416 056/300 Write block 4447 len 36864 data: 3 3 3 3 4 4 5 4 5
7417 057/300 Read block 3750 len 4096
7418 058/300 Read block 524 len 12288
7419 059/300 Write block 3597 len 24576 data: 5 5 3 3 3 3
7420 060/300 Read block 4525 len 28672
7421 061/300 Read block 2267 len 40960
7422 062/300 Read block 4814 len 28672
7423 063/300 Write block 2291 len 4096 data: 4
7424 064/300 Read block 2760 len 12288
7425 065/300 Read block 1411 len 24576
7426 066/300 Read block 4352 len 20480
7427 067/300 Read block 4432 len 12288
7428 068/300 Write block 2853 len 40960 data: 3 2 2 2 3 3 3 3 3 5
7429 069/300 Read block 3008 len 24576
7430 070/300 Read block 3903 len 28672
7431 071/300 Read block 1712 len 20480
7432 072/300 Read block 2109 len 24576
7433 073/300 Read block 4182 len 12288
7434 074/300 Read block 1769 len 32768
7435 075/300 Write block 737 len 4096 data: 2
7436 076/300 Write block 767 len 28672 data: 2 2 2 2 2 2 4
7437 077/300 Write block 1593 len 36864 data: 2 2 2 2 2 2 2 2 2
7438 078/300 Write block 1183 len 4096 data: 3
7439 079/300 Read block 345 len 12288
7440 080/300 Read block 1418 len 40960
7441 081/300 Write block 4461 len 32768 data: 4 4 6 6 7 7 7 7
7442 082/300 Flush
7443 083/300 Write block 2543 len 28672 data: 3 3 2 2 2 2 2
7444 084/300 Read block 3993 len 4096
7445 085/300 Write block 2928 len 32768 data: 4 4 4 5 5 4 5 5
7446 086/300 Write block 1583 len 4096 data: 2
7447 087/300 Read block 1531 len 32768
7448 088/300 Read block 3233 len 4096
7449 089/300 Write block 2464 len 20480 data: 3 3 3 3 3
7450 090/300 Read block 4565 len 16384
7451 091/300 Write block 488 len 16384 data: 4 3 3 4
7452 092/300 Write block 3512 len 28672 data: 3 4 4 4 4 2 2
7453 093/300 Flush
7454 094/300 Read block 4928 len 20480
7455 095/300 Write block 3864 len 16384 data: 2 3 3 3
7456 096/300 Write block 3237 len 40960 data: 2 3 2 2 2 2 2 2 2 3
7457 097/300 Read block 2140 len 12288
7458 098/300 Flush
7459 099/300 Write block 1189 len 24576 data: 3 4 4 3 3 2
7460 100/300 Read block 2745 len 12288
7461 101/300 Read block 3684 len 8192
7462 102/300 Read block 1760 len 32768
7463 103/300 Read block 854 len 4096
7464 104/300 Read block 3549 len 28672
7465 105/300 Read block 1296 len 20480
7466 106/300 Write block 575 len 28672 data: 2 2 3 4 4 4 4
7467 107/300 Write block 41 len 28672 data: 4 4 4 4 3 3 2
7468 108/300 Write block 54 len 40960 data: 5 4 3 3 3 2 2 2 2 2
7469 109/300 Read block 4012 len 8192
7470 110/300 Flush
7471 111/300 Read block 3229 len 8192
7472 112/300 Write block 4683 len 32768 data: 3 3 3 3 3 3 3 3
7473 113/300 Write block 396 len 8192 data: 4 4
7474 114/300 Write block 2747 len 24576 data: 2 2 2 2 2 3
7475 115/300 Flush
7476 116/300 Write block 490 len 40960 data: 4 5 4 3 3 3 3 4 3 3
7477 117/300 Flush
7478 118/300 Read block 4873 len 32768
7479 119/300 Write block 2613 len 24576 data: 2 2 2 2 2 2
7480 120/300 Read block 4106 len 8192
7481 121/300 Flush
7482 122/300 Write block 2333 len 40960 data: 5 4 4 3 2 2 2 2 2 2
7483 123/300 Write block 2583 len 4096 data: 2
7484 124/300 Read block 3770 len 28672
7485 125/300 Read block 3277 len 12288
7486 126/300 Write block 1137 len 20480 data: 2 2 2 2 2
7487 127/300 Read block 1566 len 40960
7488 128/300 Read block 2402 len 36864
7489 129/300 Write block 2967 len 36864 data: 2 2 2 2 2 2 2 2 3
7490 130/300 Write block 1764 len 8192 data: 2 2
7491 131/300 Write block 3572 len 12288 data: 2 2 2
7492 132/300 Write block 2736 len 28672 data: 3 2 2 2 2 2 2
7493 133/300 Flush
7494 134/300 Read block 1211 len 28672
7495 135/300 Read block 1918 len 12288
7496 136/300 Write block 39 len 36864 data: 4 5 5 5 5 5 4 4 3
7497 137/300 Read block 1038 len 40960
7498 138/300 Write block 4955 len 12288 data: 2 2 2
7499 139/300 Write block 1491 len 28672 data: 3 3 2 2 2 2 5
7500 140/300 Write block 441 len 24576 data: 3 3 3 2 2 2
7501 141/300 Read block 4247 len 36864
7502 142/300 Read block 4610 len 20480
7503 143/300 Write block 4298 len 36864 data: 6 6 6 7 7 6 4 2 2
7504 144/300 Write block 4367 len 12288 data: 2 2 2
7505 145/300 Read block 1716 len 32768
7506 146/300 Write block 1410 len 8192 data: 3 2
7507 147/300 Write block 4631 len 28672 data: 3 3 4 4 5 4 4
7508 148/300 Read block 2973 len 40960
7509 149/300 Write block 3177 len 20480 data: 2 3 3 3 3
7510 150/300 Read block 1568 len 36864
7511 151/300 Read block 3861 len 24576
7512 152/300 Write block 2682 len 20480 data: 4 4 2 2 2
7513 153/300 Write block 552 len 4096 data: 3
7514 154/300 Read block 4834 len 12288
7515 155/300 Write block 4243 len 8192 data: 5 5
7516 156/300 Write block 1827 len 28672 data: 5 5 5 5 4 4 4
7517 157/300 Read block 4175 len 28672
7518 158/300 Read block 118 len 28672
7519 159/300 Read block 2670 len 8192
7520 160/300 Write block 2958 len 12288 data: 3 3 3
7521 161/300 Read block 1047 len 28672
7522 162/300 Write block 3929 len 16384 data: 2 2 2 2
7523 163/300 Flush
7524 164/300 Write block 4827 len 40960 data: 3 3 2 2 2 2 2 2 2 3
7525 165/300 Read block 3220 len 40960
7526 166/300 Read block 3701 len 12288
7527 167/300 Write block 1093 len 12288 data: 2 2 2
7528 168/300 Write block 2252 len 28672 data: 3 3 3 4 3 3 4
7529 169/300 Read block 395 len 36864
7530 170/300 Read block 3432 len 8192
7531 171/300 Read block 4384 len 12288
7532 172/300 Flush
7533 173/300 Flush
7534 174/300 Write block 950 len 20480 data: 4 4 4 4 4
7535 175/300 Write block 2849 len 28672 data: 3 3 4 3 4 3 3
7536 176/300 Read block 2339 len 24576
7537 177/300 Read block 3578 len 20480
7538 178/300 Write block 954 len 12288 data: 5 4 4
7539 179/300 Read block 1899 len 40960
7540 180/300 Read block 806 len 32768
7541 181/300 Read block 4223 len 4096
7542 182/300 Write block 2355 len 12288 data: 3 3 3
7543 183/300 Read block 4683 len 36864
7544 184/300 Read block 1242 len 32768
7545 185/300 Write block 2051 len 36864 data: 2 2 4 4 4 4 4 4 4
7546 186/300 Write block 4208 len 20480 data: 3 3 3 4 4
7547 187/300 Write block 2711 len 28672 data: 2 2 2 2 2 2 2
7548 188/300 Flush
7549 189/300 Write block 3542 len 4096 data: 2
7550 190/300 Flush
7551 191/300 Write block 720 len 40960 data: 2 2 4 4 4 3 3 3 3 3
7552 192/300 Write block 10 len 28672 data: 3 3 3 3 3 3 3
7553 193/300 Read block 3370 len 32768
7554 194/300 Write block 498 len 28672 data: 4 4 3 3 3 3 3
7555 195/300 Write block 4409 len 28672 data: 4 4 4 4 3 3 2
7556 196/300 Write block 1676 len 40960 data: 2 2 2 2 2 2 2 2 2 3
7557 197/300 Flush
7558 198/300 Flush
7559 199/300 Write block 1677 len 4096 data: 3
7560 200/300 Write block 2914 len 4096 data: 2
7561 201/300 Write block 2546 len 20480 data: 3 3 3 3 2
7562 202/300 Write block 2782 len 24576 data: 2 2 2 3 3 4
7563 203/300 Write block 4175 len 8192 data: 3 2
7564 204/300 Read block 855 len 28672
7565 205/300 Read block 3540 len 24576
7566 206/300 Read block 3613 len 40960
7567 207/300 Read block 3457 len 8192
7568 208/300 Read block 3824 len 12288
7569 209/300 Read block 1618 len 24576
7570 210/300 Read block 2416 len 12288
7571 211/300 Write block 4528 len 8192 data: 2 2
7572 212/300 Write block 3651 len 4096 data: 3
7573 213/300 Read block 2389 len 8192
7574 214/300 Read block 3944 len 36864
7575 215/300 Flush
7576 216/300 Read block 2737 len 28672
7577 217/300 Read block 2006 len 16384
7578 218/300 Write block 2804 len 28672 data: 3 2 2 2 2 2 2
7579 219/300 Read block 3895 len 32768
7580 220/300 Read block 2132 len 36864
7581 221/300 Read block 3806 len 36864
7582 222/300 Write block 4931 len 4096 data: 3
7583 223/300 Write block 3137 len 12288 data: 3 3 2
7584 224/300 Read block 1580 len 12288
7585 225/300 Flush
7586 226/300 Read block 2481 len 16384
7587 227/300 Read block 1852 len 4096
7588 228/300 Read block 1075 len 36864
7589 229/300 Read block 915 len 16384
7590 230/300 Flush
7591 231/300 Write block 58 len 20480 data: 4 3 3 3 3
7592 232/300 Write block 3113 len 40960 data: 3 3 3 3 3 2 3 3 4 4
7593 233/300 Read block 2391 len 4096
7594 234/300 Read block 3127 len 4096
7595 235/300 Write block 2062 len 40960 data: 3 2 2 2 3 3 3 2 2 2
7596 236/300 Read block 4219 len 20480
7597 237/300 Write block 2943 len 20480 data: 4 4 4 5 5
7598 238/300 Read block 1172 len 12288
7599 239/300 Read block 3271 len 40960
7600 240/300 Read block 306 len 12288
7601 241/300 Write block 659 len 8192 data: 4 3
7602 242/300 Read block 909 len 4096
7603 243/300 Read block 4763 len 12288
7604 244/300 Write block 2815 len 40960 data: 3 3 3 4 4 4 4 4 4 4
7605 245/300 Read block 3356 len 24576
7606 246/300 Write block 4258 len 8192 data: 3 3
7607 247/300 Write block 2223 len 4096 data: 3
7608 248/300 Read block 4991 len 16384
7609 249/300 Write block 4288 len 28672 data: 2 2 2 2 2 2 2
7610 250/300 Read block 1453 len 16384
7611 251/300 Write block 3046 len 28672 data: 2 2 2 2 3 2 2
7612 252/300 Flush
7613 253/300 Write block 909 len 20480 data: 4 4 4 3 3
7614 254/300 Read block 1727 len 24576
7615 255/300 Write block 4002 len 12288 data: 3 4 3
7616 256/300 Write block 2540 len 16384 data: 3 3 3 4
7617 257/300 Read block 1659 len 40960
7618 258/300 Flush
7619 259/300 Write block 3757 len 36864 data: 3 3 3 3 2 2 2 2 2
7620 260/300 Write block 3575 len 32768 data: 2 4 3 3 3 3 3 3
7621 261/300 Read block 3330 len 40960
7622 262/300 Write block 4577 len 24576 data: 2 2 2 2 2 2
7623 263/300 Write block 294 len 12288 data: 4 4 4
7624 264/300 Read block 1084 len 32768
7625 265/300 Write block 762 len 20480 data: 2 2 2 2 2
7626 266/300 Write block 723 len 8192 data: 5 5
7627 267/300 Read block 3990 len 32768
7628 268/300 Write block 3779 len 20480 data: 3 3 3 3 2
7629 269/300 Read block 1136 len 28672
7630 270/300 Read block 1694 len 32768
7631 271/300 Write block 2192 len 4096 data: 2
7632 272/300 Write block 2448 len 4096 data: 2
7633 273/300 Write block 3816 len 28672 data: 3 3 3 3 3 3 3
7634 274/300 Read block 2811 len 12288
7635 275/300 Write block 1780 len 32768 data: 3 3 3 3 3 3 3 3
7636 276/300 Read block 1865 len 4096
7637 277/300 Read block 4034 len 32768
7638 278/300 Read block 616 len 40960
7639 279/300 Read block 4433 len 28672
7640 280/300 Read block 828 len 16384
7641 281/300 Write block 1423 len 36864 data: 3 3 3 3 3 2 2 2 2
7642 282/300 Write block 3621 len 20480 data: 2 2 2 2 2
7643 283/300 Read block 3602 len 20480
7644 284/300 Read block 4487 len 8192
7645 285/300 Flush
7646 286/300 Read block 3481 len 12288
7647 287/300 Read block 1010 len 4096
7648 288/300 Flush
7649 289/300 Read block 2281 len 20480
7650 290/300 Write block 1402 len 16384 data: 2 2 2 3
7651 291/300 Read block 1222 len 16384
7652 292/300 Write block 4155 len 4096 data: 2
7653 293/300 Read block 3521 len 12288
7654 294/300 Write block 2103 len 16384 data: 4 3 3 2
7655 295/300 Read block 807 len 12288
7656 296/300 Read block 3591 len 28672
7657 297/300 Read block 48 len 40960
7658 298/300 Read block 3279 len 4096
7659 299/300 Flush
7660 300/300 Write block 3267 len 40960 data: 2 2 2 3 3 3 3 3 3 3
7661 301/300 Read block 3876 len 12288
7662 Main task has work to do, go find it
7663 got dsc Start(2)
7664 start 2
7665 Jan 11 06:07:04.205 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/2, method: POST, req_id: 0d1e860c-17ba-4f48-8f02-75da5b9f3527, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
7666 Replay: started 2, returned:Ok(())
7667 ----------------------------------------------------------------
7668 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7669 States: Active Active Offline
7670 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7671 2812 Acked Read 10 Done Done Sent false
7672 2813 Acked Flush 0 Done Done Sent false
7673 2814 Acked Write 8 Done Done Sent false
7674 2815 Acked Read 6 Done Done Sent false
7675 2816 Acked Read 2 Done Done Sent false
7676 2817 Acked Flush 0 Done Done Sent false
7677 2818 Acked Write 3 Done Done Sent false
7678 2819 Acked Read 8 Done Done Sent false
7679 2820 Acked Write 7 Done Done Sent false
7680 2821 Acked Read 1 Done Done Sent false
7681 2822 Acked Write 8 Done Done Sent false
7682 2823 Acked Read 4 Done Done Sent false
7683 2824 Acked Read 4 Done Done Sent false
7684 2825 Acked Flush 0 Done Done Sent false
7685 2826 Acked Read 8 Done Done Sent false
7686 2827 Acked Read 4 Done Done Sent false
7687 2828 Acked Read 10 Done Done Sent false
7688 2829 Acked Flush 0 Done Done Sent false
7689 2830 Acked Write 9 Done Done Sent false
7690 2831 Acked Read 6 Done Done Sent false
7691 2832 Acked Write 1 Done Done Sent false
7692 2833 Acked Write 4 Done Done Sent false
7693 2834 Acked Write 6 Done Done Sent false
7694 2835 Acked Write 2 Done Done Sent false
7695 2836 Acked Read 2 Done Done Sent false
7696 2837 Acked Write 1 Done Done Sent false
7697 2838 Acked Read 2 Done Done Sent false
7698 2839 Acked Read 2 Done Done Sent false
7699 2840 Acked Read 6 Done Done Sent false
7700 2841 Acked Read 8 Done Done Sent false
7701 2842 Acked Write 9 Done Done Sent false
7702 2843 Acked Write 9 Done Done Sent false
7703 2844 Acked Write 6 Done Done Sent false
7704 2845 Acked Write 2 Done Done Sent false
7705 2846 Acked Read 5 Done Done Sent false
7706 2847 Acked Write 7 Done Done Sent false
7707 2848 Acked Read 8 Done Done Sent false
7708 2849 Acked Write 4 Done Done Sent false
7709 2850 Acked Flush 0 Done Done Sent false
7710 2851 Acked Write 2 Done Done Sent false
7711 2852 Acked Read 9 Done Done Sent false
7712 2853 Acked Read 10 Done Done Sent false
7713 2854 Acked Read 8 Done Done Sent false
7714 2855 Acked Read 3 Done Done Sent false
7715 2856 Acked Flush 0 Done Done Sent false
7716 2857 Acked Write 9 Done Done Sent false
7717 2858 Acked Read 9 Done Done Sent false
7718 2859 Acked Read 4 Done Done Sent false
7719 2860 Acked Read 1 Done Done Sent false
7720 2861 Acked Write 10 Done Done Sent false
7721 2862 Acked Read 5 Done Done Sent false
7722 2863 Acked Write 1 Done Done Sent false
7723 2864 Acked Write 7 Done Done Sent false
7724 2865 Acked Write 10 Done Done Sent false
7725 2866 Acked Write 7 Done Done Sent false
7726 2867 Acked Write 9 Done Done Sent false
7727 2868 Acked Read 1 Done Done Sent false
7728 2869 Acked Read 3 Done Done Sent false
7729 2870 Acked Write 6 Done Done Sent false
7730 2871 Acked Read 7 Done Done Sent false
7731 2872 Acked Read 10 Done Done Sent false
7732 2873 Acked Read 7 Done Done Sent false
7733 2874 Acked Write 1 Done Done Sent false
7734 2875 Acked Read 3 Done Done Sent false
7735 2876 Acked Read 6 Done Done Sent false
7736 2877 Acked Read 5 Done Done Sent false
7737 2878 Acked Read 3 Done Done Sent false
7738 2879 Acked Write 10 Done Done Sent false
7739 2880 Acked Read 6 Done Done Sent false
7740 2881 Acked Read 7 Done Done Sent false
7741 2882 Acked Read 5 Done Done Sent false
7742 2883 Acked Read 6 Done Done Sent false
7743 2884 Acked Read 3 Done Done Sent false
7744 2885 Acked Read 8 Done Done Sent false
7745 2886 Acked Write 1 Done Done Sent false
7746 2887 Acked Write 7 Done Done Sent false
7747 2888 Acked Write 9 Done Done Sent false
7748 2889 Acked Write 1 Done Done Sent false
7749 2890 Acked Read 3 Done Done Sent false
7750 2891 Acked Read 10 Done Done Sent false
7751 2892 Acked Write 8 Done Done Sent false
7752 2893 Acked Flush 0 Done Done Sent false
7753 2894 Acked Write 7 Done Done Sent false
7754 2895 Acked Read 1 Done Done Sent false
7755 2896 Acked Write 8 Done Done Sent false
7756 2897 Acked Write 1 Done Done Sent false
7757 2898 Acked Read 8 Done Done Sent false
7758 2899 Acked Read 1 Done Done Sent false
7759 2900 Acked Write 5 Done Done Sent false
7760 2901 Acked Read 4 Done Done Sent false
7761 2902 Acked Write 4 Done Done Sent false
7762 2903 Acked Write 7 Done Done Sent false
7763 2904 Acked Flush 0 Done Done Sent false
7764 2905 Acked Read 5 Done Done Sent false
7765 2906 Acked Write 4 Done Done Sent false
7766 2907 Acked Write 10 Done Done Sent false
7767 2908 Acked Read 3 Done Done Sent false
7768 2909 Acked Flush 0 Done Done Sent false
7769 2910 Acked Write 6 Done Done Sent false
7770 2911 Acked Read 3 Done Done Sent false
7771 2912 Acked Read 2 Done Done Sent false
7772 2913 Acked Read 8 Done Done Sent false
7773 2914 Acked Read 1 Done Done Sent false
7774 2915 Acked Read 7 Done Done Sent false
7775 2916 Acked Read 5 Done Done Sent false
7776 2917 Acked Write 7 Done Done Sent false
7777 2918 Acked Write 7 Done Done Sent false
7778 2919 Acked Write 10 Done Done Sent false
7779 2920 Acked Read 2 Done Done Sent false
7780 2921 Acked Flush 0 Done Done Sent false
7781 2922 Acked Read 2 Done Done Sent false
7782 2923 Acked Write 8 Done Done Sent false
7783 2924 Acked Write 2 Done Done Sent false
7784 2925 Acked Write 6 Done Done Sent false
7785 2926 Acked Flush 0 Done Done Sent false
7786 2927 Acked Write 10 Done Done Sent false
7787 2928 Acked Flush 0 Done Done Sent false
7788 2929 Acked Read 8 Done Done Sent false
7789 2930 Acked Write 6 Done Done Sent false
7790 2931 Acked Read 2 Done Done Sent false
7791 2932 Acked Flush 0 Done Done Sent false
7792 2933 Acked Write 10 Done Done Sent false
7793 2934 Acked Write 1 Done Done Sent false
7794 2935 Acked Read 7 Done Done Sent false
7795 2936 Acked Read 3 Done Done Sent false
7796 2937 Acked Write 5 Done Done Sent false
7797 2938 Acked Read 10 Done Done Sent false
7798 2939 Acked Read 9 Done Done Sent false
7799 2940 Acked Write 9 Done Done Sent false
7800 2941 Acked Write 2 Done Done Sent false
7801 2942 Acked Write 3 Done Done Sent false
7802 2943 Acked Write 7 Done Done Sent false
7803 2944 Acked Flush 0 Done Done Sent false
7804 2945 Acked Read 7 Done Done Sent false
7805 2946 Acked Read 3 Done Done Sent false
7806 2947 Acked Write 9 Done Done Sent false
7807 2948 Acked Read 10 Done Done Sent false
7808 2949 Acked Write 3 Done Done Sent false
7809 2950 Acked Write 7 Done Done Sent false
7810 2951 Acked Write 6 Done Done Sent false
7811 2952 Acked Read 9 Done Done Sent false
7812 2953 Acked Read 5 Done Done Sent false
7813 2954 Acked Write 9 Done Done Sent false
7814 2955 Acked Write 3 Done Done Sent false
7815 2956 Acked Read 8 Done Done Sent false
7816 2957 Acked Write 2 Done Done Sent false
7817 2958 Acked Write 7 Done Done Sent false
7818 2959 Acked Read 10 Done Done Sent false
7819 2960 Acked Write 5 Done Done Sent false
7820 2961 Acked Read 9 Done Done Sent false
7821 2962 Acked Read 6 Done Done Sent false
7822 2963 Acked Write 5 Done Done Sent false
7823 2964 Acked Write 1 Done Done Sent false
7824 2965 Acked Read 3 Done Done Sent false
7825 2966 Acked Write 2 Done Done Sent false
7826 2967 Acked Write 7 Done Done Sent false
7827 2968 Acked Read 7 Done Done Sent false
7828 2969 Acked Read 7 Done Done Sent false
7829 2970 Acked Read 2 Done Done Sent false
7830 2971 Acked Write 3 Done Done Sent false
7831 2972 Acked Read 7 Done Done Sent false
7832 2973 Acked Write 4 Done Done Sent false
7833 2974 Acked Flush 0 Done Done Sent false
7834 2975 Acked Write 10 Done Done Sent false
7835 2976 Acked Read 10 Done Done Sent false
7836 2977 Acked Read 3 Done Done Sent false
7837 2978 Acked Write 3 Done Done Sent false
7838 2979 Acked Write 7 Done Done Sent false
7839 2980 Acked Read 9 Done Done Sent false
7840 2981 Acked Read 2 Done Done Sent false
7841 2982 Acked Read 3 Done Done Sent false
7842 2983 Acked Flush 0 Done Done Sent false
7843 2984 Acked Flush 0 Done Done Sent false
7844 2985 Acked Write 5 Done Done Sent false
7845 2986 Acked Write 7 Done Done Sent false
7846 2987 Acked Read 6 Done Done Sent false
7847 2988 Acked Read 5 Done Done Sent false
7848 2989 Acked Write 3 Done Done Sent false
7849 2990 Acked Read 10 Done Done Sent false
7850 2991 Acked Read 8 Done Done Sent false
7851 2992 Acked Read 1 Done Done Sent false
7852 2993 Acked Write 3 Done Done Sent false
7853 2994 Acked Read 9 Done Done Sent false
7854 2995 Acked Read 8 Done Done Sent false
7855 2996 Acked Write 9 Done Done Sent false
7856 2997 Acked Write 5 Done Done Sent false
7857 2998 Acked Write 7 Done Done Sent false
7858 2999 Acked Flush 0 Done Done Sent false
7859 3000 Acked Write 1 Done Done Sent false
7860 3001 Acked Flush 0 Done Done Sent false
7861 3002 Acked Write 10 Done Done Sent false
7862 3003 Acked Write 7 Done Done Sent false
7863 3004 Acked Read 8 Done Done Sent false
7864 3005 Acked Write 7 Done Done Sent false
7865 3006 Acked Write 7 Done Done Sent false
7866 3007 Acked Write 10 Done Done Sent false
7867 3008 Acked Flush 0 Done Done Sent false
7868 3009 Acked Flush 0 Done Done Sent false
7869 3010 Acked Write 1 Done Done Sent false
7870 3011 Acked Write 1 Done Done Sent false
7871 3012 Acked Write 5 Done Done Sent false
7872 3013 Acked Write 6 Done Done Sent false
7873 3014 Acked Write 2 Done Done Sent false
7874 3015 Acked Read 7 Done Done Sent false
7875 3016 Acked Read 6 Done Done Sent false
7876 3017 Acked Read 10 Done Done Sent false
7877 3018 Acked Read 2 Done Done Sent false
7878 3019 Acked Read 3 Done Done Sent false
7879 3020 Acked Read 6 Done Done Sent false
7880 3021 Acked Read 3 Done Done Sent false
7881 3022 Acked Write 2 Done Done Sent false
7882 3023 Acked Write 1 Done Done Sent false
7883 3024 Acked Read 2 Done Done Sent false
7884 3025 Acked Read 9 Done Done Sent false
7885 3026 Acked Flush 0 Done Done Sent false
7886 3027 Acked Read 7 Done Done Sent false
7887 3028 Acked Read 4 Done Done Sent false
7888 3029 Acked Write 7 Done Done Sent false
7889 3030 Acked Read 8 Done Done Sent false
7890 3031 Acked Read 9 Done Done Sent false
7891 3032 Acked Read 9 Done Done Sent false
7892 3033 Acked Write 1 Done Done Sent false
7893 3034 Acked Write 3 Done Done Sent false
7894 3035 Acked Read 3 Done Done Sent false
7895 3036 Acked Flush 0 Done Done Sent false
7896 3037 Acked Read 4 Done Done Sent false
7897 3038 Acked Read 1 Done Done Sent false
7898 3039 Acked Read 9 Done Done Sent false
7899 3040 Acked Read 4 Done Done Sent false
7900 3041 Acked Flush 0 Done Done Sent false
7901 3042 Acked Write 5 Done Done Sent false
7902 3043 Acked Write 10 Done Done Sent false
7903 3044 Acked Read 1 Done Done Sent false
7904 3045 Acked Read 1 Done Done Sent false
7905 3046 Acked Write 10 Done Done Sent false
7906 3047 Acked Read 5 Done Done Sent false
7907 3048 Acked Write 5 Done Done Sent false
7908 3049 Acked Read 3 Done Done Sent false
7909 3050 Acked Read 10 Done Done Sent false
7910 3051 Acked Read 3 Done Done Sent false
7911 3052 Acked Write 2 Done Done Sent false
7912 3053 Acked Read 1 Done Done Sent false
7913 3054 Acked Read 3 Done Done Sent false
7914 3055 Acked Write 10 Done Done Sent false
7915 3056 Acked Read 6 Done Done Sent false
7916 3057 Acked Write 2 Done Done Sent false
7917 3058 Acked Write 1 Done Done Sent false
7918 3059 Acked Read 4 Done Done Sent false
7919 3060 Acked Write 7 Done Done Sent false
7920 3061 Acked Read 4 Done Done Sent false
7921 3062 Acked Write 7 Done Done Sent false
7922 3063 Acked Flush 0 Done Done Sent false
7923 3064 Acked Write 5 Done Done Sent false
7924 3065 Acked Read 6 Done Done Sent false
7925 3066 Acked Write 3 Done Done Sent false
7926 3067 Acked Write 4 Done Done Sent false
7927 3068 Acked Read 10 Done Done Sent false
7928 3069 Acked Flush 0 Done Done Sent false
7929 3070 Acked Write 9 Done Done Sent false
7930 3071 Acked Write 8 Done Done Sent false
7931 3072 Acked Read 10 Done Done Sent false
7932 3073 Acked Write 6 Done Done Sent false
7933 3074 Acked Write 3 Done Done Sent false
7934 3075 Acked Read 8 Done Done Sent false
7935 3076 Acked Write 5 Done Done Sent false
7936 3077 Acked Write 2 Done Done Sent false
7937 3078 Acked Read 8 Done Done Sent false
7938 3079 Acked Write 5 Done Done Sent false
7939 3080 Acked Read 7 Done Done Sent false
7940 3081 Acked Read 8 Done Done Sent false
7941 3082 Acked Write 1 Done Done Sent false
7942 3083 Acked Write 1 Done Done Sent false
7943 3084 Acked Write 7 Done Done Sent false
7944 3085 Acked Read 3 Done Done Sent false
7945 3086 Acked Write 8 Done Done Sent false
7946 3087 Acked Read 1 Done Done Sent false
7947 3088 Acked Read 8 Done Done Sent false
7948 3089 Acked Read 10 Done Done Sent false
7949 3090 Acked Read 7 Done Done Sent false
7950 3091 Acked Read 4 Done Done Sent false
7951 3092 Acked Write 9 Done Done Sent false
7952 3093 Acked Write 5 Done Done Sent false
7953 3094 Acked Read 5 Done Done Sent false
7954 3095 Acked Read 2 Done Done Sent false
7955 3096 Acked Flush 0 Done Done Sent false
7956 3097 Acked Read 3 Done Done Sent false
7957 3098 Acked Read 1 Done Done Sent false
7958 3099 Acked Flush 0 Done Done Sent false
7959 3100 Acked Read 5 Done Done Sent false
7960 3101 Acked Write 4 Done Done Sent false
7961 3102 Acked Read 4 Done Done Sent false
7962 3103 Acked Write 1 Done Done Sent false
7963 3104 Acked Read 3 Done Done Sent false
7964 3105 Acked Write 4 Done Done Sent false
7965 3106 Acked Read 3 Done Done Sent false
7966 3107 Acked Read 7 Done Done Sent false
7967 3108 Acked Read 10 Done Done Sent false
7968 3109 Acked Read 1 Done Done Sent false
7969 3110 Acked Flush 0 Done Done Sent false
7970 3111 Acked Write 10 Done Done Sent false
7971 3112 Acked Read 3 Done Done Sent false
7972 STATES DS:0 DS:1 DS:2 TOTAL
7973 Sent 0 0 301 301
7974 Done 301 301 0 602
7975 Skipped 0 0 0 0
7976 Error 0 0 0 0
7977 Last Flush: 3110 3110 2811
7978 Downstairs last five completed: 2811 2810 2809 2808 2807
7979 Upstairs last five completed: 3112 3111 3110 3109 3108
7980 CLIENT: Up:0 ds:301 act:2
7981 [8830] got start action
7982 [8830] I am going to restart
7983 Starting downstairs at port 8830
7984 Make output file at "/tmp/dsc/downstairs-8830.txt"
7985 [8830][2] reports Starting
7986 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1211)
7987 [8830][2] reports Running
7988 ----------------------------------------------------------------
7989 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7990 States: Active Active Offline
7991 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7992 2812 Acked Read 10 Done Done Sent false
7993 2813 Acked Flush 0 Done Done Sent false
7994 2814 Acked Write 8 Done Done Sent false
7995 2815 Acked Read 6 Done Done Sent false
7996 2816 Acked Read 2 Done Done Sent false
7997 2817 Acked Flush 0 Done Done Sent false
7998 2818 Acked Write 3 Done Done Sent false
7999 2819 Acked Read 8 Done Done Sent false
8000 2820 Acked Write 7 Done Done Sent false
8001 2821 Acked Read 1 Done Done Sent false
8002 2822 Acked Write 8 Done Done Sent false
8003 2823 Acked Read 4 Done Done Sent false
8004 2824 Acked Read 4 Done Done Sent false
8005 2825 Acked Flush 0 Done Done Sent false
8006 2826 Acked Read 8 Done Done Sent false
8007 2827 Acked Read 4 Done Done Sent false
8008 2828 Acked Read 10 Done Done Sent false
8009 2829 Acked Flush 0 Done Done Sent false
8010 2830 Acked Write 9 Done Done Sent false
8011 2831 Acked Read 6 Done Done Sent false
8012 2832 Acked Write 1 Done Done Sent false
8013 2833 Acked Write 4 Done Done Sent false
8014 2834 Acked Write 6 Done Done Sent false
8015 2835 Acked Write 2 Done Done Sent false
8016 2836 Acked Read 2 Done Done Sent false
8017 2837 Acked Write 1 Done Done Sent false
8018 2838 Acked Read 2 Done Done Sent false
8019 2839 Acked Read 2 Done Done Sent false
8020 2840 Acked Read 6 Done Done Sent false
8021 2841 Acked Read 8 Done Done Sent false
8022 2842 Acked Write 9 Done Done Sent false
8023 2843 Acked Write 9 Done Done Sent false
8024 2844 Acked Write 6 Done Done Sent false
8025 2845 Acked Write 2 Done Done Sent false
8026 2846 Acked Read 5 Done Done Sent false
8027 2847 Acked Write 7 Done Done Sent false
8028 2848 Acked Read 8 Done Done Sent false
8029 2849 Acked Write 4 Done Done Sent false
8030 2850 Acked Flush 0 Done Done Sent false
8031 2851 Acked Write 2 Done Done Sent false
8032 2852 Acked Read 9 Done Done Sent false
8033 2853 Acked Read 10 Done Done Sent false
8034 2854 Acked Read 8 Done Done Sent false
8035 2855 Acked Read 3 Done Done Sent false
8036 2856 Acked Flush 0 Done Done Sent false
8037 2857 Acked Write 9 Done Done Sent false
8038 2858 Acked Read 9 Done Done Sent false
8039 2859 Acked Read 4 Done Done Sent false
8040 2860 Acked Read 1 Done Done Sent false
8041 2861 Acked Write 10 Done Done Sent false
8042 2862 Acked Read 5 Done Done Sent false
8043 2863 Acked Write 1 Done Done Sent false
8044 2864 Acked Write 7 Done Done Sent false
8045 2865 Acked Write 10 Done Done Sent false
8046 2866 Acked Write 7 Done Done Sent false
8047 2867 Acked Write 9 Done Done Sent false
8048 2868 Acked Read 1 Done Done Sent false
8049 2869 Acked Read 3 Done Done Sent false
8050 2870 Acked Write 6 Done Done Sent false
8051 2871 Acked Read 7 Done Done Sent false
8052 2872 Acked Read 10 Done Done Sent false
8053 2873 Acked Read 7 Done Done Sent false
8054 2874 Acked Write 1 Done Done Sent false
8055 2875 Acked Read 3 Done Done Sent false
8056 2876 Acked Read 6 Done Done Sent false
8057 2877 Acked Read 5 Done Done Sent false
8058 2878 Acked Read 3 Done Done Sent false
8059 2879 Acked Write 10 Done Done Sent false
8060 2880 Acked Read 6 Done Done Sent false
8061 2881 Acked Read 7 Done Done Sent false
8062 2882 Acked Read 5 Done Done Sent false
8063 2883 Acked Read 6 Done Done Sent false
8064 2884 Acked Read 3 Done Done Sent false
8065 2885 Acked Read 8 Done Done Sent false
8066 2886 Acked Write 1 Done Done Sent false
8067 2887 Acked Write 7 Done Done Sent false
8068 2888 Acked Write 9 Done Done Sent false
8069 2889 Acked Write 1 Done Done Sent false
8070 2890 Acked Read 3 Done Done Sent false
8071 2891 Acked Read 10 Done Done Sent false
8072 2892 Acked Write 8 Done Done Sent false
8073 2893 Acked Flush 0 Done Done Sent false
8074 2894 Acked Write 7 Done Done Sent false
8075 2895 Acked Read 1 Done Done Sent false
8076 2896 Acked Write 8 Done Done Sent false
8077 2897 Acked Write 1 Done Done Sent false
8078 2898 Acked Read 8 Done Done Sent false
8079 2899 Acked Read 1 Done Done Sent false
8080 2900 Acked Write 5 Done Done Sent false
8081 2901 Acked Read 4 Done Done Sent false
8082 2902 Acked Write 4 Done Done Sent false
8083 2903 Acked Write 7 Done Done Sent false
8084 2904 Acked Flush 0 Done Done Sent false
8085 2905 Acked Read 5 Done Done Sent false
8086 2906 Acked Write 4 Done Done Sent false
8087 2907 Acked Write 10 Done Done Sent false
8088 2908 Acked Read 3 Done Done Sent false
8089 2909 Acked Flush 0 Done Done Sent false
8090 2910 Acked Write 6 Done Done Sent false
8091 2911 Acked Read 3 Done Done Sent false
8092 2912 Acked Read 2 Done Done Sent false
8093 2913 Acked Read 8 Done Done Sent false
8094 2914 Acked Read 1 Done Done Sent false
8095 2915 Acked Read 7 Done Done Sent false
8096 2916 Acked Read 5 Done Done Sent false
8097 2917 Acked Write 7 Done Done Sent false
8098 2918 Acked Write 7 Done Done Sent false
8099 2919 Acked Write 10 Done Done Sent false
8100 2920 Acked Read 2 Done Done Sent false
8101 2921 Acked Flush 0 Done Done Sent false
8102 2922 Acked Read 2 Done Done Sent false
8103 2923 Acked Write 8 Done Done Sent false
8104 2924 Acked Write 2 Done Done Sent false
8105 2925 Acked Write 6 Done Done Sent false
8106 2926 Acked Flush 0 Done Done Sent false
8107 2927 Acked Write 10 Done Done Sent false
8108 2928 Acked Flush 0 Done Done Sent false
8109 2929 Acked Read 8 Done Done Sent false
8110 2930 Acked Write 6 Done Done Sent false
8111 2931 Acked Read 2 Done Done Sent false
8112 2932 Acked Flush 0 Done Done Sent false
8113 2933 Acked Write 10 Done Done Sent false
8114 2934 Acked Write 1 Done Done Sent false
8115 2935 Acked Read 7 Done Done Sent false
8116 2936 Acked Read 3 Done Done Sent false
8117 2937 Acked Write 5 Done Done Sent false
8118 2938 Acked Read 10 Done Done Sent false
8119 2939 Acked Read 9 Done Done Sent false
8120 2940 Acked Write 9 Done Done Sent false
8121 2941 Acked Write 2 Done Done Sent false
8122 2942 Acked Write 3 Done Done Sent false
8123 2943 Acked Write 7 Done Done Sent false
8124 2944 Acked Flush 0 Done Done Sent false
8125 2945 Acked Read 7 Done Done Sent false
8126 2946 Acked Read 3 Done Done Sent false
8127 2947 Acked Write 9 Done Done Sent false
8128 2948 Acked Read 10 Done Done Sent false
8129 2949 Acked Write 3 Done Done Sent false
8130 2950 Acked Write 7 Done Done Sent false
8131 2951 Acked Write 6 Done Done Sent false
8132 2952 Acked Read 9 Done Done Sent false
8133 2953 Acked Read 5 Done Done Sent false
8134 2954 Acked Write 9 Done Done Sent false
8135 2955 Acked Write 3 Done Done Sent false
8136 2956 Acked Read 8 Done Done Sent false
8137 2957 Acked Write 2 Done Done Sent false
8138 2958 Acked Write 7 Done Done Sent false
8139 2959 Acked Read 10 Done Done Sent false
8140 2960 Acked Write 5 Done Done Sent false
8141 2961 Acked Read 9 Done Done Sent false
8142 2962 Acked Read 6 Done Done Sent false
8143 2963 Acked Write 5 Done Done Sent false
8144 2964 Acked Write 1 Done Done Sent false
8145 2965 Acked Read 3 Done Done Sent false
8146 2966 Acked Write 2 Done Done Sent false
8147 2967 Acked Write 7 Done Done Sent false
8148 2968 Acked Read 7 Done Done Sent false
8149 2969 Acked Read 7 Done Done Sent false
8150 2970 Acked Read 2 Done Done Sent false
8151 2971 Acked Write 3 Done Done Sent false
8152 2972 Acked Read 7 Done Done Sent false
8153 2973 Acked Write 4 Done Done Sent false
8154 2974 Acked Flush 0 Done Done Sent false
8155 2975 Acked Write 10 Done Done Sent false
8156 2976 Acked Read 10 Done Done Sent false
8157 2977 Acked Read 3 Done Done Sent false
8158 2978 Acked Write 3 Done Done Sent false
8159 2979 Acked Write 7 Done Done Sent false
8160 2980 Acked Read 9 Done Done Sent false
8161 2981 Acked Read 2 Done Done Sent false
8162 2982 Acked Read 3 Done Done Sent false
8163 2983 Acked Flush 0 Done Done Sent false
8164 2984 Acked Flush 0 Done Done Sent false
8165 2985 Acked Write 5 Done Done Sent false
8166 2986 Acked Write 7 Done Done Sent false
8167 2987 Acked Read 6 Done Done Sent false
8168 2988 Acked Read 5 Done Done Sent false
8169 2989 Acked Write 3 Done Done Sent false
8170 2990 Acked Read 10 Done Done Sent false
8171 2991 Acked Read 8 Done Done Sent false
8172 2992 Acked Read 1 Done Done Sent false
8173 2993 Acked Write 3 Done Done Sent false
8174 2994 Acked Read 9 Done Done Sent false
8175 2995 Acked Read 8 Done Done Sent false
8176 2996 Acked Write 9 Done Done Sent false
8177 2997 Acked Write 5 Done Done Sent false
8178 2998 Acked Write 7 Done Done Sent false
8179 2999 Acked Flush 0 Done Done Sent false
8180 3000 Acked Write 1 Done Done Sent false
8181 3001 Acked Flush 0 Done Done Sent false
8182 3002 Acked Write 10 Done Done Sent false
8183 3003 Acked Write 7 Done Done Sent false
8184 3004 Acked Read 8 Done Done Sent false
8185 3005 Acked Write 7 Done Done Sent false
8186 3006 Acked Write 7 Done Done Sent false
8187 3007 Acked Write 10 Done Done Sent false
8188 3008 Acked Flush 0 Done Done Sent false
8189 3009 Acked Flush 0 Done Done Sent false
8190 3010 Acked Write 1 Done Done Sent false
8191 3011 Acked Write 1 Done Done Sent false
8192 3012 Acked Write 5 Done Done Sent false
8193 3013 Acked Write 6 Done Done Sent false
8194 3014 Acked Write 2 Done Done Sent false
8195 3015 Acked Read 7 Done Done Sent false
8196 3016 Acked Read 6 Done Done Sent false
8197 3017 Acked Read 10 Done Done Sent false
8198 3018 Acked Read 2 Done Done Sent false
8199 3019 Acked Read 3 Done Done Sent false
8200 3020 Acked Read 6 Done Done Sent false
8201 3021 Acked Read 3 Done Done Sent false
8202 3022 Acked Write 2 Done Done Sent false
8203 3023 Acked Write 1 Done Done Sent false
8204 3024 Acked Read 2 Done Done Sent false
8205 3025 Acked Read 9 Done Done Sent false
8206 3026 Acked Flush 0 Done Done Sent false
8207 3027 Acked Read 7 Done Done Sent false
8208 3028 Acked Read 4 Done Done Sent false
8209 3029 Acked Write 7 Done Done Sent false
8210 3030 Acked Read 8 Done Done Sent false
8211 3031 Acked Read 9 Done Done Sent false
8212 3032 Acked Read 9 Done Done Sent false
8213 3033 Acked Write 1 Done Done Sent false
8214 3034 Acked Write 3 Done Done Sent false
8215 3035 Acked Read 3 Done Done Sent false
8216 3036 Acked Flush 0 Done Done Sent false
8217 3037 Acked Read 4 Done Done Sent false
8218 3038 Acked Read 1 Done Done Sent false
8219 3039 Acked Read 9 Done Done Sent false
8220 3040 Acked Read 4 Done Done Sent false
8221 3041 Acked Flush 0 Done Done Sent false
8222 3042 Acked Write 5 Done Done Sent false
8223 3043 Acked Write 10 Done Done Sent false
8224 3044 Acked Read 1 Done Done Sent false
8225 3045 Acked Read 1 Done Done Sent false
8226 3046 Acked Write 10 Done Done Sent false
8227 3047 Acked Read 5 Done Done Sent false
8228 3048 Acked Write 5 Done Done Sent false
8229 3049 Acked Read 3 Done Done Sent false
8230 3050 Acked Read 10 Done Done Sent false
8231 3051 Acked Read 3 Done Done Sent false
8232 3052 Acked Write 2 Done Done Sent false
8233 3053 Acked Read 1 Done Done Sent false
8234 3054 Acked Read 3 Done Done Sent false
8235 3055 Acked Write 10 Done Done Sent false
8236 3056 Acked Read 6 Done Done Sent false
8237 3057 Acked Write 2 Done Done Sent false
8238 3058 Acked Write 1 Done Done Sent false
8239 3059 Acked Read 4 Done Done Sent false
8240 3060 Acked Write 7 Done Done Sent false
8241 3061 Acked Read 4 Done Done Sent false
8242 3062 Acked Write 7 Done Done Sent false
8243 3063 Acked Flush 0 Done Done Sent false
8244 3064 Acked Write 5 Done Done Sent false
8245 3065 Acked Read 6 Done Done Sent false
8246 3066 Acked Write 3 Done Done Sent false
8247 3067 Acked Write 4 Done Done Sent false
8248 3068 Acked Read 10 Done Done Sent false
8249 3069 Acked Flush 0 Done Done Sent false
8250 3070 Acked Write 9 Done Done Sent false
8251 3071 Acked Write 8 Done Done Sent false
8252 3072 Acked Read 10 Done Done Sent false
8253 3073 Acked Write 6 Done Done Sent false
8254 3074 Acked Write 3 Done Done Sent false
8255 3075 Acked Read 8 Done Done Sent false
8256 3076 Acked Write 5 Done Done Sent false
8257 3077 Acked Write 2 Done Done Sent false
8258 3078 Acked Read 8 Done Done Sent false
8259 3079 Acked Write 5 Done Done Sent false
8260 3080 Acked Read 7 Done Done Sent false
8261 3081 Acked Read 8 Done Done Sent false
8262 3082 Acked Write 1 Done Done Sent false
8263 3083 Acked Write 1 Done Done Sent false
8264 3084 Acked Write 7 Done Done Sent false
8265 3085 Acked Read 3 Done Done Sent false
8266 3086 Acked Write 8 Done Done Sent false
8267 3087 Acked Read 1 Done Done Sent false
8268 3088 Acked Read 8 Done Done Sent false
8269 3089 Acked Read 10 Done Done Sent false
8270 3090 Acked Read 7 Done Done Sent false
8271 3091 Acked Read 4 Done Done Sent false
8272 3092 Acked Write 9 Done Done Sent false
8273 3093 Acked Write 5 Done Done Sent false
8274 3094 Acked Read 5 Done Done Sent false
8275 3095 Acked Read 2 Done Done Sent false
8276 3096 Acked Flush 0 Done Done Sent false
8277 3097 Acked Read 3 Done Done Sent false
8278 3098 Acked Read 1 Done Done Sent false
8279 3099 Acked Flush 0 Done Done Sent false
8280 3100 Acked Read 5 Done Done Sent false
8281 3101 Acked Write 4 Done Done Sent false
8282 3102 Acked Read 4 Done Done Sent false
8283 3103 Acked Write 1 Done Done Sent false
8284 3104 Acked Read 3 Done Done Sent false
8285 3105 Acked Write 4 Done Done Sent false
8286 3106 Acked Read 3 Done Done Sent false
8287 3107 Acked Read 7 Done Done Sent false
8288 3108 Acked Read 10 Done Done Sent false
8289 3109 Acked Read 1 Done Done Sent false
8290 3110 Acked Flush 0 Done Done Sent false
8291 3111 Acked Write 10 Done Done Sent false
8292 3112 Acked Read 3 Done Done Sent false
8293 3113 Acked Flush 0 Done Done Sent false
8294 STATES DS:0 DS:1 DS:2 TOTAL
8295 Sent 0 0 302 302
8296 Done 302 302 0 604
8297 Skipped 0 0 0 0
8298 Error 0 0 0 0
8299 Last Flush: 3113 3113 2811
8300 Downstairs last five completed: 2811 2810 2809 2808 2807
8301 Upstairs last five completed: 3113 3112 3111 3110 3109
8302 CLIENT: Up:0 ds:302 act:2
83032025-01-11T06:07:09.965ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83042025-01-11T06:07:09.966ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42781) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83052025-01-11T06:07:09.967ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83062025-01-11T06:07:09.967ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83072025-01-11T06:07:09.967ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83082025-01-11T06:07:09.967ZINFOcrucible: send last flush ID to this DS: 2811 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83092025-01-11T06:07:09.968ZINFOcrucible: Replied this last flush ID: 2811 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83102025-01-11T06:07:09.968ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83112025-01-11T06:07:09.968ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83122025-01-11T06:07:09.968ZINFOcrucible: [2] client re-new 302 jobs since flush 2811 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83132025-01-11T06:07:09.968ZINFOcrucible: [2] Replayed 302 jobs since flush: 2811 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
8314 ----------------------------------------------------------------
8315 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8316 Downstairs last five completed: 3113 3112 3111 3110 3109
8317 Upstairs last five completed: 3113 3112 3111 3110 3109
8318 CLIENT: Up:0 ds:0 act:3
8319 Replay: All jobs finished
8320 Main task has work to do, go find it
8321 got dsc Stop(0)
8322 stop 0
8323 Jan 11 06:07:12.211 [8810] Got stop action so:false kr:false
8324 INFO request completed, latency_us: 357, response_code: 204, uri: /stop/cid/0, method: POST, req_id: b6e80cb2-b244-45f2-ba4b-763322a3b450, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
8325 Jan 11 06:07:12.211 INFO request completed, latency_us: 108, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7b3047b0-9a28-4967-810f-2ee08fe6f13c, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
83262025-01-11T06:07:12.211ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83272025-01-11T06:07:12.212ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83282025-01-11T06:07:12.212ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83292025-01-11T06:07:12.212ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83302025-01-11T06:07:12.212ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
83312025-01-11T06:07:12.212ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
8332 [8810] Exited with: ExitStatus(unix_wait_status(9))
8333 [8810][0] reports Exit
8334 Jan 11 06:07:16.214 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/0, method: GET, req_id: cc14485e-d50c-4c09-9db0-d4a44966b017, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
8335 001/300 Flush
8336 002/300 Flush
8337 003/300 Flush
8338 004/300 Write block 2380 len 40960 data: 2 3 3 3 2 2 2 2 2 2
8339 005/300 Write block 2998 len 20480 data: 2 2 2 2 2
8340 006/300 Read block 1618 len 36864
8341 007/300 Read block 4215 len 12288
8342 008/300 Write block 2859 len 24576 data: 4 4 4 6 6 6
8343 009/300 Read block 3168 len 28672
8344 010/300 Read block 3358 len 8192
8345 011/300 Write block 1487 len 20480 data: 3 3 3 3 4
8346 012/300 Write block 3706 len 12288 data: 2 2 2
8347 013/300 Read block 3362 len 28672
8348 014/300 Write block 2835 len 8192 data: 2 2
8349 015/300 Write block 2808 len 8192 data: 3 3
8350 016/300 Write block 1417 len 12288 data: 2 3 3
8351 017/300 Read block 4261 len 40960
8352 018/300 Read block 1184 len 24576
8353 019/300 Read block 3600 len 36864
8354 020/300 Read block 3732 len 4096
8355 021/300 Read block 1367 len 4096
8356 022/300 Read block 953 len 36864
8357 023/300 Read block 1262 len 40960
8358 024/300 Read block 124 len 24576
8359 025/300 Read block 4955 len 20480
8360 026/300 Write block 3254 len 40960 data: 3 3 3 3 3 3 3 3 3 2
8361 027/300 Write block 3364 len 32768 data: 2 2 5 5 5 5 4 3
8362 028/300 Write block 4401 len 12288 data: 3 3 4
8363 029/300 Read block 4297 len 24576
8364 030/300 Write block 3899 len 24576 data: 2 2 2 3 3 3
8365 031/300 Flush
8366 032/300 Read block 1930 len 12288
8367 033/300 Read block 1166 len 40960
8368 034/300 Write block 3669 len 36864 data: 4 3 3 4 3 3 3 3 2
8369 035/300 Read block 2774 len 12288
8370 036/300 Read block 1511 len 16384
8371 037/300 Read block 4437 len 40960
8372 038/300 Write block 4627 len 8192 data: 2 2
8373 039/300 Write block 2204 len 28672 data: 3 3 3 3 3 2 2
8374 040/300 Write block 2388 len 32768 data: 3 3 2 2 3 4 4 5
8375 041/300 Flush
8376 042/300 Read block 3241 len 32768
8377 043/300 Write block 11 len 20480 data: 4 4 4 4 4
8378 044/300 Flush
8379 045/300 Write block 1430 len 8192 data: 3 3
8380 046/300 Read block 309 len 28672
8381 047/300 Read block 551 len 12288
8382 048/300 Write block 906 len 32768 data: 3 3 3 5 5 5 4 4
8383 049/300 Read block 1809 len 12288
8384 050/300 Write block 1121 len 16384 data: 5 4 3 2
8385 051/300 Write block 1150 len 12288 data: 3 3 3
8386 052/300 Write block 2543 len 4096 data: 5
8387 053/300 Write block 2 len 28672 data: 3 2 3 3 3 3 3
8388 054/300 Write block 4667 len 36864 data: 4 4 4 4 5 4 4 3 3
8389 055/300 Write block 664 len 24576 data: 2 2 2 2 2 3
8390 056/300 Read block 393 len 32768
8391 057/300 Flush
8392 058/300 Read block 4534 len 28672
8393 059/300 Read block 1843 len 8192
8394 060/300 Read block 1754 len 40960
8395 061/300 Read block 3799 len 20480
8396 062/300 Read block 1998 len 8192
8397 063/300 Write block 483 len 20480 data: 4 4 4 4 3
8398 064/300 Flush
8399 065/300 Write block 392 len 16384 data: 3 3 2 2
8400 066/300 Write block 1302 len 40960 data: 2 2 2 2 2 3 3 4 4 4
8401 067/300 Write block 3066 len 36864 data: 2 2 2 2 2 2 2 2 2
8402 068/300 Read block 868 len 4096
8403 069/300 Read block 1868 len 32768
8404 070/300 Write block 548 len 40960 data: 4 5 4 3 4 3 3 3 3 2
8405 071/300 Write block 1729 len 4096 data: 4
8406 072/300 Write block 2661 len 16384 data: 2 2 2 2
8407 073/300 Read block 93 len 24576
8408 074/300 Write block 1523 len 20480 data: 4 4 4 4 4
8409 075/300 Write block 62 len 16384 data: 4 3 2 2
8410 076/300 Read block 3401 len 20480
8411 077/300 Read block 3233 len 16384
8412 078/300 Write block 1 len 24576 data: 2 4 3 4 4 4
8413 079/300 Read block 2478 len 12288
8414 080/300 Read block 2996 len 4096
8415 081/300 Write block 1891 len 20480 data: 2 2 2 2 2
8416 082/300 Read block 4708 len 16384
8417 083/300 Read block 1372 len 12288
8418 084/300 Write block 3597 len 32768 data: 6 6 4 4 4 4 3 2
8419 085/300 Write block 4160 len 40960 data: 2 3 3 3 3 2 2 2 2 3
8420 086/300 Read block 4311 len 16384
8421 087/300 Write block 4983 len 40960 data: 3 3 3 4 4 4 4 6 6 6
8422 088/300 Read block 2238 len 20480
8423 089/300 Read block 4842 len 40960
8424 090/300 Write block 2334 len 4096 data: 5
8425 091/300 Read block 987 len 20480
8426 092/300 Read block 1996 len 36864
8427 093/300 Read block 344 len 36864
8428 094/300 Read block 1341 len 24576
8429 095/300 Write block 1896 len 4096 data: 3
8430 096/300 Write block 3703 len 36864 data: 2 2 2 3 3 3 2 2 2
8431 097/300 Read block 2969 len 12288
8432 098/300 Read block 3729 len 40960
8433 099/300 Read block 3732 len 28672
8434 100/300 Write block 1441 len 24576 data: 3 4 4 4 4 3
8435 101/300 Read block 4664 len 16384
8436 102/300 Read block 3299 len 20480
8437 103/300 Write block 69 len 20480 data: 2 2 2 2 2
8438 104/300 Write block 1612 len 12288 data: 2 2 2
8439 105/300 Read block 1358 len 32768
8440 106/300 Read block 4856 len 20480
8441 107/300 Flush
8442 108/300 Write block 4576 len 16384 data: 2 3 3 3
8443 109/300 Read block 800 len 4096
8444 110/300 Flush
8445 111/300 Flush
8446 112/300 Read block 825 len 24576
8447 113/300 Read block 4173 len 20480
8448 114/300 Write block 1361 len 36864 data: 4 4 4 3 3 2 2 2 2
8449 115/300 Write block 4774 len 16384 data: 5 4 3 3
8450 116/300 Read block 2313 len 20480
8451 117/300 Write block 1961 len 20480 data: 2 2 2 2 2
8452 118/300 Read block 4520 len 4096
8453 119/300 Read block 4772 len 24576
8454 120/300 Read block 2255 len 16384
8455 121/300 Read block 1726 len 24576
8456 122/300 Read block 227 len 40960
8457 123/300 Write block 3500 len 12288 data: 3 4 4
8458 124/300 Write block 1926 len 32768 data: 2 2 2 2 2 2 2 2
8459 125/300 Read block 2469 len 36864
8460 126/300 Write block 3285 len 4096 data: 2
8461 127/300 Read block 2250 len 28672
8462 128/300 Read block 438 len 24576
8463 129/300 Write block 1927 len 24576 data: 3 3 3 3 3 3
8464 130/300 Write block 4061 len 28672 data: 3 3 3 3 3 3 4
8465 131/300 Flush
8466 132/300 Flush
8467 133/300 Read block 2595 len 36864
8468 134/300 Write block 4257 len 4096 data: 3
8469 135/300 Read block 4229 len 24576
8470 136/300 Read block 666 len 12288
8471 137/300 Flush
8472 138/300 Read block 4212 len 16384
8473 139/300 Write block 414 len 16384 data: 2 2 2 2
8474 140/300 Write block 3206 len 24576 data: 2 3 3 4 4 4
8475 141/300 Write block 1559 len 40960 data: 2 2 3 3 3 3 3 4 4 4
8476 142/300 Write block 6 len 24576 data: 5 4 4 2 4 5
8477 143/300 Read block 3601 len 28672
8478 144/300 Read block 3890 len 24576
8479 145/300 Read block 4392 len 12288
8480 146/300 Read block 1205 len 16384
8481 147/300 Write block 4834 len 32768 data: 3 3 4 2 2 2 2 2
8482 148/300 Read block 3184 len 8192
8483 149/300 Write block 2645 len 4096 data: 4
8484 150/300 Write block 595 len 20480 data: 2 2 2 2 2
8485 151/300 Read block 3119 len 8192
8486 152/300 Read block 839 len 40960
8487 153/300 Write block 2688 len 28672 data: 2 2 2 2 2 2 2
8488 154/300 Read block 2967 len 8192
8489 155/300 Read block 2368 len 20480
8490 156/300 Write block 414 len 28672 data: 3 3 3 3 2 2 2
8491 157/300 Write block 3173 len 12288 data: 2 2 2
8492 158/300 Read block 1646 len 4096
8493 159/300 Write block 2376 len 24576 data: 2 2 2 2 3 4
8494 160/300 Write block 4375 len 4096 data: 2
8495 161/300 Read block 3014 len 4096
8496 162/300 Read block 3722 len 36864
8497 163/300 Read block 1631 len 24576
8498 164/300 Read block 1802 len 16384
8499 165/300 Write block 2547 len 32768 data: 4 4 4 3 2 3 3 3
8500 166/300 Read block 2888 len 24576
8501 167/300 Write block 3085 len 32768 data: 2 2 2 2 4 4 4 4
8502 168/300 Write block 2940 len 8192 data: 4 4
8503 169/300 Write block 1356 len 12288 data: 2 3 4
8504 170/300 Write block 3880 len 36864 data: 2 2 2 3 3 3 3 3 3
8505 171/300 Write block 362 len 32768 data: 3 3 2 2 4 3 2 2
8506 172/300 Write block 4710 len 36864 data: 3 3 2 2 2 2 2 2 2
8507 173/300 Read block 2090 len 24576
8508 174/300 Read block 4257 len 16384
8509 175/300 Read block 907 len 32768
8510 176/300 Write block 885 len 20480 data: 4 4 4 4 3
8511 177/300 Read block 3831 len 16384
8512 178/300 Read block 1295 len 20480
8513 179/300 Flush
8514 180/300 Read block 639 len 36864
8515 181/300 Flush
8516 182/300 Flush
8517 183/300 Read block 810 len 8192
8518 184/300 Read block 4366 len 24576
8519 185/300 Read block 3286 len 16384
8520 186/300 Write block 685 len 40960 data: 2 3 3 3 3 3 3 3 4 2
8521 187/300 Write block 4263 len 4096 data: 3
8522 188/300 Flush
8523 189/300 Write block 4203 len 24576 data: 2 2 3 3 3 4
8524 190/300 Read block 2872 len 36864
8525 191/300 Write block 1829 len 36864 data: 6 6 5 5 5 3 2 2 2
8526 192/300 Read block 1371 len 20480
8527 193/300 Read block 3586 len 36864
8528 194/300 Read block 993 len 40960
8529 195/300 Write block 2677 len 40960 data: 3 3 3 3 3 5 5 3 3 3
8530 196/300 Read block 2407 len 24576
8531 197/300 Write block 537 len 16384 data: 3 3 4 3
8532 198/300 Read block 2621 len 12288
8533 199/300 Write block 3004 len 36864 data: 3 3 3 3 3 3 2 2 2
8534 200/300 Read block 2377 len 12288
8535 201/300 Read block 72 len 24576
8536 202/300 Write block 4026 len 24576 data: 3 3 3 4 3 3
8537 203/300 Read block 2936 len 12288
8538 204/300 Write block 4319 len 36864 data: 3 4 4 3 2 2 2 2 2
8539 205/300 Write block 1089 len 28672 data: 2 2 2 2 3 3 3
8540 206/300 Write block 1328 len 20480 data: 4 5 5 5 4
8541 207/300 Write block 848 len 32768 data: 3 3 2 2 2 2 2 2
8542 208/300 Read block 1863 len 16384
8543 209/300 Read block 1579 len 4096
8544 210/300 Read block 2233 len 20480
8545 211/300 Read block 837 len 4096
8546 212/300 Read block 1884 len 12288
8547 213/300 Read block 1301 len 24576
8548 214/300 Read block 4375 len 8192
8549 215/300 Write block 2572 len 20480 data: 2 2 4 4 4
8550 216/300 Write block 1001 len 40960 data: 3 3 3 4 4 4 5 5 4 5
8551 217/300 Read block 2769 len 28672
8552 218/300 Read block 101 len 28672
8553 219/300 Write block 342 len 24576 data: 2 2 2 2 2 2
8554 220/300 Read block 1109 len 16384
8555 221/300 Read block 2528 len 32768
8556 222/300 Read block 666 len 8192
8557 223/300 Read block 3246 len 24576
8558 224/300 Write block 3588 len 24576 data: 4 4 4 5 5 4
8559 225/300 Read block 3997 len 4096
8560 226/300 Read block 4225 len 4096
8561 227/300 Write block 3763 len 12288 data: 3 3 3
8562 228/300 Flush
8563 229/300 Read block 255 len 12288
8564 230/300 Write block 3305 len 12288 data: 2 2 2
8565 231/300 Write block 1363 len 24576 data: 5 4 4 3 3 3
8566 232/300 Read block 4614 len 36864
8567 233/300 Write block 3727 len 40960 data: 3 3 3 2 3 4 4 4 4 4
8568 234/300 Write block 2827 len 24576 data: 2 2 2 2 2 2
8569 235/300 Read block 2505 len 24576
8570 236/300 Read block 3040 len 4096
8571 237/300 Write block 3134 len 16384 data: 3 3 3 4
8572 238/300 Write block 279 len 16384 data: 5 4 5 6
8573 239/300 Read block 3877 len 4096
8574 240/300 Read block 3971 len 36864
8575 241/300 Read block 4988 len 20480
8576 242/300 Write block 172 len 12288 data: 2 2 2
8577 243/300 Read block 4903 len 24576
8578 244/300 Write block 1326 len 16384 data: 4 4 5 6
8579 245/300 Read block 301 len 36864
8580 246/300 Read block 2140 len 12288
8581 247/300 Flush
8582 248/300 Read block 4198 len 8192
8583 249/300 Read block 588 len 28672
8584 250/300 Read block 1590 len 16384
8585 251/300 Flush
8586 252/300 Read block 259 len 12288
8587 253/300 Read block 2189 len 32768
8588 254/300 Flush
8589 255/300 Write block 2990 len 4096 data: 2
8590 256/300 Write block 2128 len 40960 data: 3 3 3 3 3 3 2 3 3 3
8591 257/300 Flush
8592 258/300 Flush
8593 259/300 Write block 3751 len 24576 data: 3 3 3 3 3 3
8594 260/300 Read block 3553 len 36864
8595 261/300 Read block 4083 len 20480
8596 262/300 Flush
8597 263/300 Write block 4170 len 12288 data: 3 3 3
8598 264/300 Read block 674 len 36864
8599 265/300 Read block 2521 len 36864
8600 266/300 Write block 2 len 40960 data: 5 4 5 5 6 5 5 3 5 6
8601 267/300 Write block 286 len 32768 data: 4 3 3 3 4 5 4 4
8602 268/300 Read block 2435 len 36864
8603 269/300 Read block 272 len 4096
8604 270/300 Read block 63 len 32768
8605 271/300 Write block 2985 len 32768 data: 2 2 2 2 2 3 2 2
8606 272/300 Write block 3711 len 12288 data: 3 2 2
8607 273/300 Read block 3119 len 20480
8608 274/300 Write block 3780 len 24576 data: 4 4 4 3 2 2
8609 275/300 Read block 552 len 36864
8610 276/300 Read block 4549 len 16384
8611 277/300 Read block 88 len 28672
8612 278/300 Write block 4135 len 28672 data: 2 2 2 3 3 3 4
8613 279/300 Read block 433 len 40960
8614 280/300 Read block 3924 len 24576
8615 281/300 Write block 4158 len 28672 data: 2 2 3 4 4 4 4
8616 282/300 Read block 4155 len 16384
8617 283/300 Flush
8618 284/300 Read block 174 len 8192
8619 285/300 Flush
8620 286/300 Flush
8621 287/300 Write block 2182 len 16384 data: 3 4 5 4
8622 288/300 Flush
8623 289/300 Read block 3461 len 28672
8624 290/300 Write block 2903 len 20480 data: 4 4 4 3 4
8625 291/300 Read block 2627 len 16384
8626 292/300 Read block 4208 len 8192
8627 293/300 Read block 585 len 12288
8628 294/300 Read block 1276 len 20480
8629 295/300 Write block 1372 len 36864 data: 2 2 2 2 2 2 3 3 3
8630 296/300 Flush
8631 297/300 Write block 4130 len 32768 data: 3 2 2 2 2 3 3 3
8632 298/300 Read block 3213 len 16384
8633 299/300 Read block 2202 len 8192
8634 300/300 Write block 3003 len 20480 data: 2 4 4 4 4
8635 301/300 Flush
8636 Main task has work to do, go find it
8637 got dsc Start(0)
8638 start 0
8639 Jan 11 06:07:16.460 INFO request completed, latency_us: 218, response_code: 204, uri: /start/cid/0, method: POST, req_id: c25b1748-efde-48b3-bd6b-94adcf58e444, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
8640 Replay: started 0, returned:Ok(())
8641 ----------------------------------------------------------------
8642 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8643 States: Offline Active Active
8644 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8645 3114 Acked Flush 0 Sent Done Done false
8646 3115 Acked Flush 0 Sent Done Done false
8647 3116 Acked Flush 0 Sent Done Done false
8648 3117 Acked Write 10 Sent Done Done false
8649 3118 Acked Write 5 Sent Done Done false
8650 3119 Acked Read 9 Sent Done Done false
8651 3120 Acked Read 3 Sent Done Done false
8652 3121 Acked Write 6 Sent Done Done false
8653 3122 Acked Read 7 Sent Done Done false
8654 3123 Acked Read 2 Sent Done Done false
8655 3124 Acked Write 5 Sent Done Done false
8656 3125 Acked Write 3 Sent Done Done false
8657 3126 Acked Read 7 Sent Done Done false
8658 3127 Acked Write 2 Sent Done Done false
8659 3128 Acked Write 2 Sent Done Done false
8660 3129 Acked Write 3 Sent Done Done false
8661 3130 Acked Read 10 Sent Done Done false
8662 3131 Acked Read 6 Sent Done Done false
8663 3132 Acked Read 9 Sent Done Done false
8664 3133 Acked Read 1 Sent Done Done false
8665 3134 Acked Read 1 Sent Done Done false
8666 3135 Acked Read 9 Sent Done Done false
8667 3136 Acked Read 10 Sent Done Done false
8668 3137 Acked Read 6 Sent Done Done false
8669 3138 Acked Read 5 Sent Done Done false
8670 3139 Acked Write 10 Sent Done Done false
8671 3140 Acked Write 8 Sent Done Done false
8672 3141 Acked Write 3 Sent Done Done false
8673 3142 Acked Read 6 Sent Done Done false
8674 3143 Acked Write 6 Sent Done Done false
8675 3144 Acked Flush 0 Sent Done Done false
8676 3145 Acked Read 3 Sent Done Done false
8677 3146 Acked Read 10 Sent Done Done false
8678 3147 Acked Write 9 Sent Done Done false
8679 3148 Acked Read 3 Sent Done Done false
8680 3149 Acked Read 4 Sent Done Done false
8681 3150 Acked Read 10 Sent Done Done false
8682 3151 Acked Write 2 Sent Done Done false
8683 3152 Acked Write 7 Sent Done Done false
8684 3153 Acked Write 8 Sent Done Done false
8685 3154 Acked Flush 0 Sent Done Done false
8686 3155 Acked Read 8 Sent Done Done false
8687 3156 Acked Write 5 Sent Done Done false
8688 3157 Acked Flush 0 Sent Done Done false
8689 3158 Acked Write 2 Sent Done Done false
8690 3159 Acked Read 7 Sent Done Done false
8691 3160 Acked Read 3 Sent Done Done false
8692 3161 Acked Write 8 Sent Done Done false
8693 3162 Acked Read 3 Sent Done Done false
8694 3163 Acked Write 4 Sent Done Done false
8695 3164 Acked Write 3 Sent Done Done false
8696 3165 Acked Write 1 Sent Done Done false
8697 3166 Acked Write 7 Sent Done Done false
8698 3167 Acked Write 9 Sent Done Done false
8699 3168 Acked Write 6 Sent Done Done false
8700 3169 Acked Read 8 Sent Done Done false
8701 3170 Acked Flush 0 Sent Done Done false
8702 3171 Acked Read 7 Sent Done Done false
8703 3172 Acked Read 2 Sent Done Done false
8704 3173 Acked Read 10 Sent Done Done false
8705 3174 Acked Read 5 Sent Done Done false
8706 3175 Acked Read 2 Sent Done Done false
8707 3176 Acked Write 5 Sent Done Done false
8708 3177 Acked Flush 0 Sent Done Done false
8709 3178 Acked Write 4 Sent Done Done false
8710 3179 Acked Write 10 Sent Done Done false
8711 3180 Acked Write 9 Sent Done Done false
8712 3181 Acked Read 1 Sent Done Done false
8713 3182 Acked Read 8 Sent Done Done false
8714 3183 Acked Write 10 Sent Done Done false
8715 3184 Acked Write 1 Sent Done Done false
8716 3185 Acked Write 4 Sent Done Done false
8717 3186 Acked Read 6 Sent Done Done false
8718 3187 Acked Write 5 Sent Done Done false
8719 3188 Acked Write 4 Sent Done Done false
8720 3189 Acked Read 5 Sent Done Done false
8721 3190 Acked Read 4 Sent Done Done false
8722 3191 Acked Write 6 Sent Done Done false
8723 3192 Acked Read 3 Sent Done Done false
8724 3193 Acked Read 1 Sent Done Done false
8725 3194 Acked Write 5 Sent Done Done false
8726 3195 Acked Read 4 Sent Done Done false
8727 3196 Acked Read 3 Sent Done Done false
8728 3197 Acked Write 8 Sent Done Done false
8729 3198 Acked Write 10 Sent Done Done false
8730 3199 Acked Read 4 Sent Done Done false
8731 3200 Acked Write 10 Sent Done Done false
8732 3201 Acked Read 5 Sent Done Done false
8733 3202 Acked Read 10 Sent Done Done false
8734 3203 Acked Write 1 Sent Done Done false
8735 3204 Acked Read 5 Sent Done Done false
8736 3205 Acked Read 9 Sent Done Done false
8737 3206 Acked Read 9 Sent Done Done false
8738 3207 Acked Read 6 Sent Done Done false
8739 3208 Acked Write 1 Sent Done Done false
8740 3209 Acked Write 9 Sent Done Done false
8741 3210 Acked Read 3 Sent Done Done false
8742 3211 Acked Read 10 Sent Done Done false
8743 3212 Acked Read 7 Sent Done Done false
8744 3213 Acked Write 6 Sent Done Done false
8745 3214 Acked Read 4 Sent Done Done false
8746 3215 Acked Read 5 Sent Done Done false
8747 3216 Acked Write 5 Sent Done Done false
8748 3217 Acked Write 3 Sent Done Done false
8749 3218 Acked Read 8 Sent Done Done false
8750 3219 Acked Read 5 Sent Done Done false
8751 3220 Acked Flush 0 Sent Done Done false
8752 3221 Acked Write 4 Sent Done Done false
8753 3222 Acked Read 1 Sent Done Done false
8754 3223 Acked Flush 0 Sent Done Done false
8755 3224 Acked Flush 0 Sent Done Done false
8756 3225 Acked Read 6 Sent Done Done false
8757 3226 Acked Read 5 Sent Done Done false
8758 3227 Acked Write 9 Sent Done Done false
8759 3228 Acked Write 4 Sent Done Done false
8760 3229 Acked Read 5 Sent Done Done false
8761 3230 Acked Write 5 Sent Done Done false
8762 3231 Acked Read 1 Sent Done Done false
8763 3232 Acked Read 6 Sent Done Done false
8764 3233 Acked Read 4 Sent Done Done false
8765 3234 Acked Read 6 Sent Done Done false
8766 3235 Acked Read 10 Sent Done Done false
8767 3236 Acked Write 3 Sent Done Done false
8768 3237 Acked Write 8 Sent Done Done false
8769 3238 Acked Read 9 Sent Done Done false
8770 3239 Acked Write 1 Sent Done Done false
8771 3240 Acked Read 7 Sent Done Done false
8772 3241 Acked Read 6 Sent Done Done false
8773 3242 Acked Write 6 Sent Done Done false
8774 3243 Acked Write 7 Sent Done Done false
8775 3244 Acked Flush 0 Sent Done Done false
8776 3245 Acked Flush 0 Sent Done Done false
8777 3246 Acked Read 9 Sent Done Done false
8778 3247 Acked Write 1 Sent Done Done false
8779 3248 Acked Read 6 Sent Done Done false
8780 3249 Acked Read 3 Sent Done Done false
8781 3250 Acked Flush 0 Sent Done Done false
8782 3251 Acked Read 4 Sent Done Done false
8783 3252 Acked Write 4 Sent Done Done false
8784 3253 Acked Write 6 Sent Done Done false
8785 3254 Acked Write 10 Sent Done Done false
8786 3255 Acked Write 6 Sent Done Done false
8787 3256 Acked Read 7 Sent Done Done false
8788 3257 Acked Read 6 Sent Done Done false
8789 3258 Acked Read 3 Sent Done Done false
8790 3259 Acked Read 4 Sent Done Done false
8791 3260 Acked Write 8 Sent Done Done false
8792 3261 Acked Read 2 Sent Done Done false
8793 3262 Acked Write 1 Sent Done Done false
8794 3263 Acked Write 5 Sent Done Done false
8795 3264 Acked Read 2 Sent Done Done false
8796 3265 Acked Read 10 Sent Done Done false
8797 3266 Acked Write 7 Sent Done Done false
8798 3267 Acked Read 2 Sent Done Done false
8799 3268 Acked Read 5 Sent Done Done false
8800 3269 Acked Write 7 Sent Done Done false
8801 3270 Acked Write 3 Sent Done Done false
8802 3271 Acked Read 1 Sent Done Done false
8803 3272 Acked Write 6 Sent Done Done false
8804 3273 Acked Write 1 Sent Done Done false
8805 3274 Acked Read 1 Sent Done Done false
8806 3275 Acked Read 9 Sent Done Done false
8807 3276 Acked Read 6 Sent Done Done false
8808 3277 Acked Read 4 Sent Done Done false
8809 3278 Acked Write 8 Sent Done Done false
8810 3279 Acked Read 6 Sent Done Done false
8811 3280 Acked Write 8 Sent Done Done false
8812 3281 Acked Write 2 Sent Done Done false
8813 3282 Acked Write 3 Sent Done Done false
8814 3283 Acked Write 9 Sent Done Done false
8815 3284 Acked Write 8 Sent Done Done false
8816 3285 Acked Write 9 Sent Done Done false
8817 3286 Acked Read 6 Sent Done Done false
8818 3287 Acked Read 4 Sent Done Done false
8819 3288 Acked Read 8 Sent Done Done false
8820 3289 Acked Write 5 Sent Done Done false
8821 3290 Acked Read 4 Sent Done Done false
8822 3291 Acked Read 5 Sent Done Done false
8823 3292 Acked Flush 0 Sent Done Done false
8824 3293 Acked Read 9 Sent Done Done false
8825 3294 Acked Flush 0 Sent Done Done false
8826 3295 Acked Flush 0 Sent Done Done false
8827 3296 Acked Read 2 Sent Done Done false
8828 3297 Acked Read 6 Sent Done Done false
8829 3298 Acked Read 4 Sent Done Done false
8830 3299 Acked Write 10 Sent Done Done false
8831 3300 Acked Write 1 Sent Done Done false
8832 3301 Acked Flush 0 Sent Done Done false
8833 3302 Acked Write 6 Sent Done Done false
8834 3303 Acked Read 9 Sent Done Done false
8835 3304 Acked Write 9 Sent Done Done false
8836 3305 Acked Read 5 Sent Done Done false
8837 3306 Acked Read 9 Sent Done Done false
8838 3307 Acked Read 10 Sent Done Done false
8839 3308 Acked Write 10 Sent Done Done false
8840 3309 Acked Read 6 Sent Done Done false
8841 3310 Acked Write 4 Sent Done Done false
8842 3311 Acked Read 3 Sent Done Done false
8843 3312 Acked Write 9 Sent Done Done false
8844 3313 Acked Read 3 Sent Done Done false
8845 3314 Acked Read 6 Sent Done Done false
8846 3315 Acked Write 6 Sent Done Done false
8847 3316 Acked Read 3 Sent Done Done false
8848 3317 Acked Write 9 Sent Done Done false
8849 3318 Acked Write 7 Sent Done Done false
8850 3319 Acked Write 5 Sent Done Done false
8851 3320 Acked Write 8 Sent Done Done false
8852 3321 Acked Read 4 Sent Done Done false
8853 3322 Acked Read 1 Sent Done Done false
8854 3323 Acked Read 5 Sent Done Done false
8855 3324 Acked Read 1 Sent Done Done false
8856 3325 Acked Read 3 Sent Done Done false
8857 3326 Acked Read 6 Sent Done Done false
8858 3327 Acked Read 2 Sent Done Done false
8859 3328 Acked Write 5 Sent Done Done false
8860 3329 Acked Write 10 Sent Done Done false
8861 3330 Acked Read 7 Sent Done Done false
8862 3331 Acked Read 7 Sent Done Done false
8863 3332 Acked Write 6 Sent Done Done false
8864 3333 Acked Read 4 Sent Done Done false
8865 3334 Acked Read 8 Sent Done Done false
8866 3335 Acked Read 2 Sent Done Done false
8867 3336 Acked Read 6 Sent Done Done false
8868 3337 Acked Write 6 Sent Done Done false
8869 3338 Acked Read 1 Sent Done Done false
8870 3339 Acked Read 1 Sent Done Done false
8871 3340 Acked Write 3 Sent Done Done false
8872 3341 Acked Flush 0 Sent Done Done false
8873 3342 Acked Read 3 Sent Done Done false
8874 3343 Acked Write 3 Sent Done Done false
8875 3344 Acked Write 6 Sent Done Done false
8876 3345 Acked Read 9 Sent Done Done false
8877 3346 Acked Write 10 Sent Done Done false
8878 3347 Acked Write 6 Sent Done Done false
8879 3348 Acked Read 6 Sent Done Done false
8880 3349 Acked Read 1 Sent Done Done false
8881 3350 Acked Write 4 Sent Done Done false
8882 3351 Acked Write 4 Sent Done Done false
8883 3352 Acked Read 1 Sent Done Done false
8884 3353 Acked Read 9 Sent Done Done false
8885 3354 Acked Read 5 Sent Done Done false
8886 3355 Acked Write 3 Sent Done Done false
8887 3356 Acked Read 6 Sent Done Done false
8888 3357 Acked Write 4 Sent Done Done false
8889 3358 Acked Read 9 Sent Done Done false
8890 3359 Acked Read 3 Sent Done Done false
8891 3360 Acked Flush 0 Sent Done Done false
8892 3361 Acked Read 2 Sent Done Done false
8893 3362 Acked Read 7 Sent Done Done false
8894 3363 Acked Read 4 Sent Done Done false
8895 3364 Acked Flush 0 Sent Done Done false
8896 3365 Acked Read 3 Sent Done Done false
8897 3366 Acked Read 8 Sent Done Done false
8898 3367 Acked Flush 0 Sent Done Done false
8899 3368 Acked Write 1 Sent Done Done false
8900 3369 Acked Write 10 Sent Done Done false
8901 3370 Acked Flush 0 Sent Done Done false
8902 3371 Acked Flush 0 Sent Done Done false
8903 3372 Acked Write 6 Sent Done Done false
8904 3373 Acked Read 9 Sent Done Done false
8905 3374 Acked Read 5 Sent Done Done false
8906 3375 Acked Flush 0 Sent Done Done false
8907 3376 Acked Write 3 Sent Done Done false
8908 3377 Acked Read 9 Sent Done Done false
8909 3378 Acked Read 9 Sent Done Done false
8910 3379 Acked Write 10 Sent Done Done false
8911 3380 Acked Write 8 Sent Done Done false
8912 3381 Acked Read 9 Sent Done Done false
8913 3382 Acked Read 1 Sent Done Done false
8914 3383 Acked Read 8 Sent Done Done false
8915 3384 Acked Write 8 Sent Done Done false
8916 3385 Acked Write 3 Sent Done Done false
8917 3386 Acked Read 5 Sent Done Done false
8918 3387 Acked Write 6 Sent Done Done false
8919 3388 Acked Read 9 Sent Done Done false
8920 3389 Acked Read 4 Sent Done Done false
8921 3390 Acked Read 7 Sent Done Done false
8922 3391 Acked Write 7 Sent Done Done false
8923 3392 Acked Read 10 Sent Done Done false
8924 3393 Acked Read 6 Sent Done Done false
8925 3394 Acked Write 7 Sent Done Done false
8926 3395 Acked Read 4 Sent Done Done false
8927 3396 Acked Flush 0 Sent Done Done false
8928 3397 Acked Read 2 Sent Done Done false
8929 3398 Acked Flush 0 Sent Done Done false
8930 3399 Acked Flush 0 Sent Done Done false
8931 3400 Acked Write 4 Sent Done Done false
8932 3401 Acked Flush 0 Sent Done Done false
8933 3402 Acked Read 7 Sent Done Done false
8934 3403 Acked Write 5 Sent Done Done false
8935 3404 Acked Read 4 Sent Done Done false
8936 3405 Acked Read 2 Sent Done Done false
8937 3406 Acked Read 3 Sent Done Done false
8938 3407 Acked Read 5 Sent Done Done false
8939 3408 Acked Write 9 Sent Done Done false
8940 3409 Acked Flush 0 Sent Done Done false
8941 3410 Acked Write 8 Sent Done Done false
8942 3411 Acked Read 4 Sent Done Done false
8943 3412 Acked Read 2 Sent Done Done false
8944 3413 Acked Write 5 Sent Done Done false
8945 3414 Acked Flush 0 Sent Done Done false
8946 STATES DS:0 DS:1 DS:2 TOTAL
8947 Sent 301 0 0 301
8948 Done 0 301 301 602
8949 Skipped 0 0 0 0
8950 Error 0 0 0 0
8951 Last Flush: 3113 3414 3414
8952 Downstairs last five completed: 3113 3112 3111 3110 3109
8953 Upstairs last five completed: 3414 3413 3412 3411 3410
8954 CLIENT: Up:0 ds:301 act:2
8955 ----------------------------------------------------------------
8956 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8957 States: Offline Active Active
8958 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8959 3114 Acked Flush 0 Sent Done Done false
8960 3115 Acked Flush 0 Sent Done Done false
8961 3116 Acked Flush 0 Sent Done Done false
8962 3117 Acked Write 10 Sent Done Done false
8963 3118 Acked Write 5 Sent Done Done false
8964 3119 Acked Read 9 Sent Done Done false
8965 3120 Acked Read 3 Sent Done Done false
8966 3121 Acked Write 6 Sent Done Done false
8967 3122 Acked Read 7 Sent Done Done false
8968 3123 Acked Read 2 Sent Done Done false
8969 3124 Acked Write 5 Sent Done Done false
8970 3125 Acked Write 3 Sent Done Done false
8971 3126 Acked Read 7 Sent Done Done false
8972 3127 Acked Write 2 Sent Done Done false
8973 3128 Acked Write 2 Sent Done Done false
8974 3129 Acked Write 3 Sent Done Done false
8975 3130 Acked Read 10 Sent Done Done false
8976 3131 Acked Read 6 Sent Done Done false
8977 3132 Acked Read 9 Sent Done Done false
8978 3133 Acked Read 1 Sent Done Done false
8979 3134 Acked Read 1 Sent Done Done false
8980 3135 Acked Read 9 Sent Done Done false
8981 3136 Acked Read 10 Sent Done Done false
8982 3137 Acked Read 6 Sent Done Done false
8983 3138 Acked Read 5 Sent Done Done false
8984 3139 Acked Write 10 Sent Done Done false
8985 3140 Acked Write 8 Sent Done Done false
8986 3141 Acked Write 3 Sent Done Done false
8987 3142 Acked Read 6 Sent Done Done false
8988 3143 Acked Write 6 Sent Done Done false
8989 3144 Acked Flush 0 Sent Done Done false
8990 3145 Acked Read 3 Sent Done Done false
8991 3146 Acked Read 10 Sent Done Done false
8992 3147 Acked Write 9 Sent Done Done false
8993 3148 Acked Read 3 Sent Done Done false
8994 3149 Acked Read 4 Sent Done Done false
8995 3150 Acked Read 10 Sent Done Done false
8996 3151 Acked Write 2 Sent Done Done false
8997 3152 Acked Write 7 Sent Done Done false
8998 3153 Acked Write 8 Sent Done Done false
8999 3154 Acked Flush 0 Sent Done Done false
9000 3155 Acked Read 8 Sent Done Done false
9001 3156 Acked Write 5 Sent Done Done false
9002 3157 Acked Flush 0 Sent Done Done false
9003 3158 Acked Write 2 Sent Done Done false
9004 3159 Acked Read 7 Sent Done Done false
9005 3160 Acked Read 3 Sent Done Done false
9006 3161 Acked Write 8 Sent Done Done false
9007 3162 Acked Read 3 Sent Done Done false
9008 3163 Acked Write 4 Sent Done Done false
9009 3164 Acked Write 3 Sent Done Done false
9010 3165 Acked Write 1 Sent Done Done false
9011 3166 Acked Write 7 Sent Done Done false
9012 3167 Acked Write 9 Sent Done Done false
9013 3168 Acked Write 6 Sent Done Done false
9014 3169 Acked Read 8 Sent Done Done false
9015 3170 Acked Flush 0 Sent Done Done false
9016 3171 Acked Read 7 Sent Done Done false
9017 3172 Acked Read 2 Sent Done Done false
9018 3173 Acked Read 10 Sent Done Done false
9019 3174 Acked Read 5 Sent Done Done false
9020 3175 Acked Read 2 Sent Done Done false
9021 3176 Acked Write 5 Sent Done Done false
9022 3177 Acked Flush 0 Sent Done Done false
9023 3178 Acked Write 4 Sent Done Done false
9024 3179 Acked Write 10 Sent Done Done false
9025 3180 Acked Write 9 Sent Done Done false
9026 3181 Acked Read 1 Sent Done Done false
9027 3182 Acked Read 8 Sent Done Done false
9028 3183 Acked Write 10 Sent Done Done false
9029 3184 Acked Write 1 Sent Done Done false
9030 3185 Acked Write 4 Sent Done Done false
9031 3186 Acked Read 6 Sent Done Done false
9032 3187 Acked Write 5 Sent Done Done false
9033 3188 Acked Write 4 Sent Done Done false
9034 3189 Acked Read 5 Sent Done Done false
9035 3190 Acked Read 4 Sent Done Done false
9036 3191 Acked Write 6 Sent Done Done false
9037 3192 Acked Read 3 Sent Done Done false
9038 3193 Acked Read 1 Sent Done Done false
9039 3194 Acked Write 5 Sent Done Done false
9040 3195 Acked Read 4 Sent Done Done false
9041 3196 Acked Read 3 Sent Done Done false
9042 3197 Acked Write 8 Sent Done Done false
9043 3198 Acked Write 10 Sent Done Done false
9044 3199 Acked Read 4 Sent Done Done false
9045 3200 Acked Write 10 Sent Done Done false
9046 3201 Acked Read 5 Sent Done Done false
9047 3202 Acked Read 10 Sent Done Done false
9048 3203 Acked Write 1 Sent Done Done false
9049 3204 Acked Read 5 Sent Done Done false
9050 3205 Acked Read 9 Sent Done Done false
9051 3206 Acked Read 9 Sent Done Done false
9052 3207 Acked Read 6 Sent Done Done false
9053 3208 Acked Write 1 Sent Done Done false
9054 3209 Acked Write 9 Sent Done Done false
9055 3210 Acked Read 3 Sent Done Done false
9056 3211 Acked Read 10 Sent Done Done false
9057 3212 Acked Read 7 Sent Done Done false
9058 3213 Acked Write 6 Sent Done Done false
9059 3214 Acked Read 4 Sent Done Done false
9060 3215 Acked Read 5 Sent Done Done false
9061 3216 Acked Write 5 Sent Done Done false
9062 3217 Acked Write 3 Sent Done Done false
9063 3218 Acked Read 8 Sent Done Done false
9064 3219 Acked Read 5 Sent Done Done false
9065 3220 Acked Flush 0 Sent Done Done false
9066 3221 Acked Write 4 Sent Done Done false
9067 3222 Acked Read 1 Sent Done Done false
9068 3223 Acked Flush 0 Sent Done Done false
9069 3224 Acked Flush 0 Sent Done Done false
9070 3225 Acked Read 6 Sent Done Done false
9071 3226 Acked Read 5 Sent Done Done false
9072 3227 Acked Write 9 Sent Done Done false
9073 3228 Acked Write 4 Sent Done Done false
9074 3229 Acked Read 5 Sent Done Done false
9075 3230 Acked Write 5 Sent Done Done false
9076 3231 Acked Read 1 Sent Done Done false
9077 3232 Acked Read 6 Sent Done Done false
9078 3233 Acked Read 4 Sent Done Done false
9079 3234 Acked Read 6 Sent Done Done false
9080 3235 Acked Read 10 Sent Done Done false
9081 3236 Acked Write 3 Sent Done Done false
9082 3237 Acked Write 8 Sent Done Done false
9083 3238 Acked Read 9 Sent Done Done false
9084 3239 Acked Write 1 Sent Done Done false
9085 3240 Acked Read 7 Sent Done Done false
9086 3241 Acked Read 6 Sent Done Done false
9087 3242 Acked Write 6 Sent Done Done false
9088 3243 Acked Write 7 Sent Done Done false
9089 3244 Acked Flush 0 Sent Done Done false
9090 3245 Acked Flush 0 Sent Done Done false
9091 3246 Acked Read 9 Sent Done Done false
9092 3247 Acked Write 1 Sent Done Done false
9093 3248 Acked Read 6 Sent Done Done false
9094 3249 Acked Read 3 Sent Done Done false
9095 3250 Acked Flush 0 Sent Done Done false
9096 3251 Acked Read 4 Sent Done Done false
9097 3252 Acked Write 4 Sent Done Done false
9098 3253 Acked Write 6 Sent Done Done false
9099 3254 Acked Write 10 Sent Done Done false
9100 3255 Acked Write 6 Sent Done Done false
9101 3256 Acked Read 7 Sent Done Done false
9102 3257 Acked Read 6 Sent Done Done false
9103 3258 Acked Read 3 Sent Done Done false
9104 3259 Acked Read 4 Sent Done Done false
9105 3260 Acked Write 8 Sent Done Done false
9106 3261 Acked Read 2 Sent Done Done false
9107 3262 Acked Write 1 Sent Done Done false
9108 3263 Acked Write 5 Sent Done Done false
9109 3264 Acked Read 2 Sent Done Done false
9110 3265 Acked Read 10 Sent Done Done false
9111 3266 Acked Write 7 Sent Done Done false
9112 3267 Acked Read 2 Sent Done Done false
9113 3268 Acked Read 5 Sent Done Done false
9114 3269 Acked Write 7 Sent Done Done false
9115 3270 Acked Write 3 Sent Done Done false
9116 3271 Acked Read 1 Sent Done Done false
9117 3272 Acked Write 6 Sent Done Done false
9118 3273 Acked Write 1 Sent Done Done false
9119 3274 Acked Read 1 Sent Done Done false
9120 3275 Acked Read 9 Sent Done Done false
9121 3276 Acked Read 6 Sent Done Done false
9122 3277 Acked Read 4 Sent Done Done false
9123 3278 Acked Write 8 Sent Done Done false
9124 3279 Acked Read 6 Sent Done Done false
9125 3280 Acked Write 8 Sent Done Done false
9126 3281 Acked Write 2 Sent Done Done false
9127 3282 Acked Write 3 Sent Done Done false
9128 3283 Acked Write 9 Sent Done Done false
9129 3284 Acked Write 8 Sent Done Done false
9130 3285 Acked Write 9 Sent Done Done false
9131 3286 Acked Read 6 Sent Done Done false
9132 3287 Acked Read 4 Sent Done Done false
9133 3288 Acked Read 8 Sent Done Done false
9134 3289 Acked Write 5 Sent Done Done false
9135 3290 Acked Read 4 Sent Done Done false
9136 3291 Acked Read 5 Sent Done Done false
9137 3292 Acked Flush 0 Sent Done Done false
9138 3293 Acked Read 9 Sent Done Done false
9139 3294 Acked Flush 0 Sent Done Done false
9140 3295 Acked Flush 0 Sent Done Done false
9141 3296 Acked Read 2 Sent Done Done false
9142 3297 Acked Read 6 Sent Done Done false
9143 3298 Acked Read 4 Sent Done Done false
9144 3299 Acked Write 10 Sent Done Done false
9145 3300 Acked Write 1 Sent Done Done false
9146 3301 Acked Flush 0 Sent Done Done false
9147 3302 Acked Write 6 Sent Done Done false
9148 3303 Acked Read 9 Sent Done Done false
9149 3304 Acked Write 9 Sent Done Done false
9150 3305 Acked Read 5 Sent Done Done false
9151 3306 Acked Read 9 Sent Done Done false
9152 3307 Acked Read 10 Sent Done Done false
9153 3308 Acked Write 10 Sent Done Done false
9154 3309 Acked Read 6 Sent Done Done false
9155 3310 Acked Write 4 Sent Done Done false
9156 3311 Acked Read 3 Sent Done Done false
9157 3312 Acked Write 9 Sent Done Done false
9158 3313 Acked Read 3 Sent Done Done false
9159 3314 Acked Read 6 Sent Done Done false
9160 3315 Acked Write 6 Sent Done Done false
9161 3316 Acked Read 3 Sent Done Done false
9162 3317 Acked Write 9 Sent Done Done false
9163 3318 Acked Write 7 Sent Done Done false
9164 3319 Acked Write 5 Sent Done Done false
9165 3320 Acked Write 8 Sent Done Done false
9166 3321 Acked Read 4 Sent Done Done false
9167 3322 Acked Read 1 Sent Done Done false
9168 3323 Acked Read 5 Sent Done Done false
9169 3324 Acked Read 1 Sent Done Done false
9170 3325 Acked Read 3 Sent Done Done false
9171 3326 Acked Read 6 Sent Done Done false
9172 3327 Acked Read 2 Sent Done Done false
9173 3328 Acked Write 5 Sent Done Done false
9174 3329 Acked Write 10 Sent Done Done false
9175 3330 Acked Read 7 Sent Done Done false
9176 3331 Acked Read 7 Sent Done Done false
9177 3332 Acked Write 6 Sent Done Done false
9178 3333 Acked Read 4 Sent Done Done false
9179 3334 Acked Read 8 Sent Done Done false
9180 3335 Acked Read 2 Sent Done Done false
9181 3336 Acked Read 6 Sent Done Done false
9182 3337 Acked Write 6 Sent Done Done false
9183 3338 Acked Read 1 Sent Done Done false
9184 3339 Acked Read 1 Sent Done Done false
9185 3340 Acked Write 3 Sent Done Done false
9186 3341 Acked Flush 0 Sent Done Done false
9187 3342 Acked Read 3 Sent Done Done false
9188 3343 Acked Write 3 Sent Done Done false
9189 3344 Acked Write 6 Sent Done Done false
9190 3345 Acked Read 9 Sent Done Done false
9191 3346 Acked Write 10 Sent Done Done false
9192 3347 Acked Write 6 Sent Done Done false
9193 3348 Acked Read 6 Sent Done Done false
9194 3349 Acked Read 1 Sent Done Done false
9195 3350 Acked Write 4 Sent Done Done false
9196 3351 Acked Write 4 Sent Done Done false
9197 3352 Acked Read 1 Sent Done Done false
9198 3353 Acked Read 9 Sent Done Done false
9199 3354 Acked Read 5 Sent Done Done false
9200 3355 Acked Write 3 Sent Done Done false
9201 3356 Acked Read 6 Sent Done Done false
9202 3357 Acked Write 4 Sent Done Done false
9203 3358 Acked Read 9 Sent Done Done false
9204 3359 Acked Read 3 Sent Done Done false
9205 3360 Acked Flush 0 Sent Done Done false
9206 3361 Acked Read 2 Sent Done Done false
9207 3362 Acked Read 7 Sent Done Done false
9208 3363 Acked Read 4 Sent Done Done false
9209 3364 Acked Flush 0 Sent Done Done false
9210 3365 Acked Read 3 Sent Done Done false
9211 3366 Acked Read 8 Sent Done Done false
9212 3367 Acked Flush 0 Sent Done Done false
9213 3368 Acked Write 1 Sent Done Done false
9214 3369 Acked Write 10 Sent Done Done false
9215 3370 Acked Flush 0 Sent Done Done false
9216 3371 Acked Flush 0 Sent Done Done false
9217 3372 Acked Write 6 Sent Done Done false
9218 3373 Acked Read 9 Sent Done Done false
9219 3374 Acked Read 5 Sent Done Done false
9220 3375 Acked Flush 0 Sent Done Done false
9221 3376 Acked Write 3 Sent Done Done false
9222 3377 Acked Read 9 Sent Done Done false
9223 3378 Acked Read 9 Sent Done Done false
9224 3379 Acked Write 10 Sent Done Done false
9225 3380 Acked Write 8 Sent Done Done false
9226 3381 Acked Read 9 Sent Done Done false
9227 3382 Acked Read 1 Sent Done Done false
9228 3383 Acked Read 8 Sent Done Done false
9229 3384 Acked Write 8 Sent Done Done false
9230 3385 Acked Write 3 Sent Done Done false
9231 3386 Acked Read 5 Sent Done Done false
9232 3387 Acked Write 6 Sent Done Done false
9233 3388 Acked Read 9 Sent Done Done false
9234 3389 Acked Read 4 Sent Done Done false
9235 3390 Acked Read 7 Sent Done Done false
9236 3391 Acked Write 7 Sent Done Done false
9237 3392 Acked Read 10 Sent Done Done false
9238 3393 Acked Read 6 Sent Done Done false
9239 3394 Acked Write 7 Sent Done Done false
9240 3395 Acked Read 4 Sent Done Done false
9241 3396 Acked Flush 0 Sent Done Done false
9242 3397 Acked Read 2 Sent Done Done false
9243 3398 Acked Flush 0 Sent Done Done false
9244 3399 Acked Flush 0 Sent Done Done false
9245 3400 Acked Write 4 Sent Done Done false
9246 3401 Acked Flush 0 Sent Done Done false
9247 3402 Acked Read 7 Sent Done Done false
9248 3403 Acked Write 5 Sent Done Done false
9249 3404 Acked Read 4 Sent Done Done false
9250 3405 Acked Read 2 Sent Done Done false
9251 3406 Acked Read 3 Sent Done Done false
9252 3407 Acked Read 5 Sent Done Done false
9253 3408 Acked Write 9 Sent Done Done false
9254 3409 Acked Flush 0 Sent Done Done false
9255 3410 Acked Write 8 Sent Done Done false
9256 3411 Acked Read 4 Sent Done Done false
9257 3412 Acked Read 2 Sent Done Done false
9258 3413 Acked Write 5 Sent Done Done false
9259 3414 Acked Flush 0 Sent Done Done false
9260 STATES DS:0 DS:1 DS:2 TOTAL
9261 Sent 301 0 0 301
9262 Done 0 301 301 602
9263 Skipped 0 0 0 0
9264 Error 0 0 0 0
9265 Last Flush: 3113 3414 3414
9266 Downstairs last five completed: 3113 3112 3111 3110 3109
9267 Upstairs last five completed: 3414 3413 3412 3411 3410
9268 CLIENT: Up:0 ds:301 act:2
92692025-01-11T06:07:22.213ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
92702025-01-11T06:07:22.213ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
92712025-01-11T06:07:22.213ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
92722025-01-11T06:07:22.213ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
92732025-01-11T06:07:22.213ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
9274 ----------------------------------------------------------------
9275 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9276 States: Offline Active Active
9277 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9278 3114 Acked Flush 0 Sent Done Done false
9279 3115 Acked Flush 0 Sent Done Done false
9280 3116 Acked Flush 0 Sent Done Done false
9281 3117 Acked Write 10 Sent Done Done false
9282 3118 Acked Write 5 Sent Done Done false
9283 3119 Acked Read 9 Sent Done Done false
9284 3120 Acked Read 3 Sent Done Done false
9285 3121 Acked Write 6 Sent Done Done false
9286 3122 Acked Read 7 Sent Done Done false
9287 3123 Acked Read 2 Sent Done Done false
9288 3124 Acked Write 5 Sent Done Done false
9289 3125 Acked Write 3 Sent Done Done false
9290 3126 Acked Read 7 Sent Done Done false
9291 3127 Acked Write 2 Sent Done Done false
9292 3128 Acked Write 2 Sent Done Done false
9293 3129 Acked Write 3 Sent Done Done false
9294 3130 Acked Read 10 Sent Done Done false
9295 3131 Acked Read 6 Sent Done Done false
9296 3132 Acked Read 9 Sent Done Done false
9297 3133 Acked Read 1 Sent Done Done false
9298 3134 Acked Read 1 Sent Done Done false
9299 3135 Acked Read 9 Sent Done Done false
9300 3136 Acked Read 10 Sent Done Done false
9301 3137 Acked Read 6 Sent Done Done false
9302 3138 Acked Read 5 Sent Done Done false
9303 3139 Acked Write 10 Sent Done Done false
9304 3140 Acked Write 8 Sent Done Done false
9305 3141 Acked Write 3 Sent Done Done false
9306 3142 Acked Read 6 Sent Done Done false
9307 3143 Acked Write 6 Sent Done Done false
9308 3144 Acked Flush 0 Sent Done Done false
9309 3145 Acked Read 3 Sent Done Done false
9310 3146 Acked Read 10 Sent Done Done false
9311 3147 Acked Write 9 Sent Done Done false
9312 3148 Acked Read 3 Sent Done Done false
9313 3149 Acked Read 4 Sent Done Done false
9314 3150 Acked Read 10 Sent Done Done false
9315 3151 Acked Write 2 Sent Done Done false
9316 3152 Acked Write 7 Sent Done Done false
9317 3153 Acked Write 8 Sent Done Done false
9318 3154 Acked Flush 0 Sent Done Done false
9319 3155 Acked Read 8 Sent Done Done false
9320 3156 Acked Write 5 Sent Done Done false
9321 3157 Acked Flush 0 Sent Done Done false
9322 3158 Acked Write 2 Sent Done Done false
9323 3159 Acked Read 7 Sent Done Done false
9324 3160 Acked Read 3 Sent Done Done false
9325 3161 Acked Write 8 Sent Done Done false
9326 3162 Acked Read 3 Sent Done Done false
9327 3163 Acked Write 4 Sent Done Done false
9328 3164 Acked Write 3 Sent Done Done false
9329 3165 Acked Write 1 Sent Done Done false
9330 3166 Acked Write 7 Sent Done Done false
9331 3167 Acked Write 9 Sent Done Done false
9332 3168 Acked Write 6 Sent Done Done false
9333 3169 Acked Read 8 Sent Done Done false
9334 3170 Acked Flush 0 Sent Done Done false
9335 3171 Acked Read 7 Sent Done Done false
9336 3172 Acked Read 2 Sent Done Done false
9337 3173 Acked Read 10 Sent Done Done false
9338 3174 Acked Read 5 Sent Done Done false
9339 3175 Acked Read 2 Sent Done Done false
9340 3176 Acked Write 5 Sent Done Done false
9341 3177 Acked Flush 0 Sent Done Done false
9342 3178 Acked Write 4 Sent Done Done false
9343 3179 Acked Write 10 Sent Done Done false
9344 3180 Acked Write 9 Sent Done Done false
9345 3181 Acked Read 1 Sent Done Done false
9346 3182 Acked Read 8 Sent Done Done false
9347 3183 Acked Write 10 Sent Done Done false
9348 3184 Acked Write 1 Sent Done Done false
9349 3185 Acked Write 4 Sent Done Done false
9350 3186 Acked Read 6 Sent Done Done false
9351 3187 Acked Write 5 Sent Done Done false
9352 3188 Acked Write 4 Sent Done Done false
9353 3189 Acked Read 5 Sent Done Done false
9354 3190 Acked Read 4 Sent Done Done false
9355 3191 Acked Write 6 Sent Done Done false
9356 3192 Acked Read 3 Sent Done Done false
9357 3193 Acked Read 1 Sent Done Done false
9358 3194 Acked Write 5 Sent Done Done false
9359 3195 Acked Read 4 Sent Done Done false
9360 3196 Acked Read 3 Sent Done Done false
9361 3197 Acked Write 8 Sent Done Done false
9362 3198 Acked Write 10 Sent Done Done false
9363 3199 Acked Read 4 Sent Done Done false
9364 3200 Acked Write 10 Sent Done Done false
9365 3201 Acked Read 5 Sent Done Done false
9366 3202 Acked Read 10 Sent Done Done false
9367 3203 Acked Write 1 Sent Done Done false
9368 3204 Acked Read 5 Sent Done Done false
9369 3205 Acked Read 9 Sent Done Done false
9370 3206 Acked Read 9 Sent Done Done false
9371 3207 Acked Read 6 Sent Done Done false
9372 3208 Acked Write 1 Sent Done Done false
9373 3209 Acked Write 9 Sent Done Done false
9374 3210 Acked Read 3 Sent Done Done false
9375 3211 Acked Read 10 Sent Done Done false
9376 3212 Acked Read 7 Sent Done Done false
9377 3213 Acked Write 6 Sent Done Done false
9378 3214 Acked Read 4 Sent Done Done false
9379 3215 Acked Read 5 Sent Done Done false
9380 3216 Acked Write 5 Sent Done Done false
9381 3217 Acked Write 3 Sent Done Done false
9382 3218 Acked Read 8 Sent Done Done false
9383 3219 Acked Read 5 Sent Done Done false
9384 3220 Acked Flush 0 Sent Done Done false
9385 3221 Acked Write 4 Sent Done Done false
9386 3222 Acked Read 1 Sent Done Done false
9387 3223 Acked Flush 0 Sent Done Done false
9388 3224 Acked Flush 0 Sent Done Done false
9389 3225 Acked Read 6 Sent Done Done false
9390 3226 Acked Read 5 Sent Done Done false
9391 3227 Acked Write 9 Sent Done Done false
9392 3228 Acked Write 4 Sent Done Done false
9393 3229 Acked Read 5 Sent Done Done false
9394 3230 Acked Write 5 Sent Done Done false
9395 3231 Acked Read 1 Sent Done Done false
9396 3232 Acked Read 6 Sent Done Done false
9397 3233 Acked Read 4 Sent Done Done false
9398 3234 Acked Read 6 Sent Done Done false
9399 3235 Acked Read 10 Sent Done Done false
9400 3236 Acked Write 3 Sent Done Done false
9401 3237 Acked Write 8 Sent Done Done false
9402 3238 Acked Read 9 Sent Done Done false
9403 3239 Acked Write 1 Sent Done Done false
9404 3240 Acked Read 7 Sent Done Done false
9405 3241 Acked Read 6 Sent Done Done false
9406 3242 Acked Write 6 Sent Done Done false
9407 3243 Acked Write 7 Sent Done Done false
9408 3244 Acked Flush 0 Sent Done Done false
9409 3245 Acked Flush 0 Sent Done Done false
9410 3246 Acked Read 9 Sent Done Done false
9411 3247 Acked Write 1 Sent Done Done false
9412 3248 Acked Read 6 Sent Done Done false
9413 3249 Acked Read 3 Sent Done Done false
9414 3250 Acked Flush 0 Sent Done Done false
9415 3251 Acked Read 4 Sent Done Done false
9416 3252 Acked Write 4 Sent Done Done false
9417 3253 Acked Write 6 Sent Done Done false
9418 3254 Acked Write 10 Sent Done Done false
9419 3255 Acked Write 6 Sent Done Done false
9420 3256 Acked Read 7 Sent Done Done false
9421 3257 Acked Read 6 Sent Done Done false
9422 3258 Acked Read 3 Sent Done Done false
9423 3259 Acked Read 4 Sent Done Done false
9424 3260 Acked Write 8 Sent Done Done false
9425 3261 Acked Read 2 Sent Done Done false
9426 3262 Acked Write 1 Sent Done Done false
9427 3263 Acked Write 5 Sent Done Done false
9428 3264 Acked Read 2 Sent Done Done false
9429 3265 Acked Read 10 Sent Done Done false
9430 3266 Acked Write 7 Sent Done Done false
9431 3267 Acked Read 2 Sent Done Done false
9432 3268 Acked Read 5 Sent Done Done false
9433 3269 Acked Write 7 Sent Done Done false
9434 3270 Acked Write 3 Sent Done Done false
9435 3271 Acked Read 1 Sent Done Done false
9436 3272 Acked Write 6 Sent Done Done false
9437 3273 Acked Write 1 Sent Done Done false
9438 3274 Acked Read 1 Sent Done Done false
9439 3275 Acked Read 9 Sent Done Done false
9440 3276 Acked Read 6 Sent Done Done false
9441 3277 Acked Read 4 Sent Done Done false
9442 3278 Acked Write 8 Sent Done Done false
9443 3279 Acked Read 6 Sent Done Done false
9444 3280 Acked Write 8 Sent Done Done false
9445 3281 Acked Write 2 Sent Done Done false
9446 3282 Acked Write 3 Sent Done Done false
9447 3283 Acked Write 9 Sent Done Done false
9448 3284 Acked Write 8 Sent Done Done false
9449 3285 Acked Write 9 Sent Done Done false
9450 3286 Acked Read 6 Sent Done Done false
9451 3287 Acked Read 4 Sent Done Done false
9452 3288 Acked Read 8 Sent Done Done false
9453 3289 Acked Write 5 Sent Done Done false
9454 3290 Acked Read 4 Sent Done Done false
9455 3291 Acked Read 5 Sent Done Done false
9456 3292 Acked Flush 0 Sent Done Done false
9457 3293 Acked Read 9 Sent Done Done false
9458 3294 Acked Flush 0 Sent Done Done false
9459 3295 Acked Flush 0 Sent Done Done false
9460 3296 Acked Read 2 Sent Done Done false
9461 3297 Acked Read 6 Sent Done Done false
9462 3298 Acked Read 4 Sent Done Done false
9463 3299 Acked Write 10 Sent Done Done false
9464 3300 Acked Write 1 Sent Done Done false
9465 3301 Acked Flush 0 Sent Done Done false
9466 3302 Acked Write 6 Sent Done Done false
9467 3303 Acked Read 9 Sent Done Done false
9468 3304 Acked Write 9 Sent Done Done false
9469 3305 Acked Read 5 Sent Done Done false
9470 3306 Acked Read 9 Sent Done Done false
9471 3307 Acked Read 10 Sent Done Done false
9472 3308 Acked Write 10 Sent Done Done false
9473 3309 Acked Read 6 Sent Done Done false
9474 3310 Acked Write 4 Sent Done Done false
9475 3311 Acked Read 3 Sent Done Done false
9476 3312 Acked Write 9 Sent Done Done false
9477 3313 Acked Read 3 Sent Done Done false
9478 3314 Acked Read 6 Sent Done Done false
9479 3315 Acked Write 6 Sent Done Done false
9480 3316 Acked Read 3 Sent Done Done false
9481 3317 Acked Write 9 Sent Done Done false
9482 3318 Acked Write 7 Sent Done Done false
9483 3319 Acked Write 5 Sent Done Done false
9484 3320 Acked Write 8 Sent Done Done false
9485 3321 Acked Read 4 Sent Done Done false
9486 3322 Acked Read 1 Sent Done Done false
9487 3323 Acked Read 5 Sent Done Done false
9488 3324 Acked Read 1 Sent Done Done false
9489 3325 Acked Read 3 Sent Done Done false
9490 3326 Acked Read 6 Sent Done Done false
9491 3327 Acked Read 2 Sent Done Done false
9492 3328 Acked Write 5 Sent Done Done false
9493 3329 Acked Write 10 Sent Done Done false
9494 3330 Acked Read 7 Sent Done Done false
9495 3331 Acked Read 7 Sent Done Done false
9496 3332 Acked Write 6 Sent Done Done false
9497 3333 Acked Read 4 Sent Done Done false
9498 3334 Acked Read 8 Sent Done Done false
9499 3335 Acked Read 2 Sent Done Done false
9500 3336 Acked Read 6 Sent Done Done false
9501 3337 Acked Write 6 Sent Done Done false
9502 3338 Acked Read 1 Sent Done Done false
9503 3339 Acked Read 1 Sent Done Done false
9504 3340 Acked Write 3 Sent Done Done false
9505 3341 Acked Flush 0 Sent Done Done false
9506 3342 Acked Read 3 Sent Done Done false
9507 3343 Acked Write 3 Sent Done Done false
9508 3344 Acked Write 6 Sent Done Done false
9509 3345 Acked Read 9 Sent Done Done false
9510 3346 Acked Write 10 Sent Done Done false
9511 3347 Acked Write 6 Sent Done Done false
9512 3348 Acked Read 6 Sent Done Done false
9513 3349 Acked Read 1 Sent Done Done false
9514 3350 Acked Write 4 Sent Done Done false
9515 3351 Acked Write 4 Sent Done Done false
9516 3352 Acked Read 1 Sent Done Done false
9517 3353 Acked Read 9 Sent Done Done false
9518 3354 Acked Read 5 Sent Done Done false
9519 3355 Acked Write 3 Sent Done Done false
9520 3356 Acked Read 6 Sent Done Done false
9521 3357 Acked Write 4 Sent Done Done false
9522 3358 Acked Read 9 Sent Done Done false
9523 3359 Acked Read 3 Sent Done Done false
9524 3360 Acked Flush 0 Sent Done Done false
9525 3361 Acked Read 2 Sent Done Done false
9526 3362 Acked Read 7 Sent Done Done false
9527 3363 Acked Read 4 Sent Done Done false
9528 3364 Acked Flush 0 Sent Done Done false
9529 3365 Acked Read 3 Sent Done Done false
9530 3366 Acked Read 8 Sent Done Done false
9531 3367 Acked Flush 0 Sent Done Done false
9532 3368 Acked Write 1 Sent Done Done false
9533 3369 Acked Write 10 Sent Done Done false
9534 3370 Acked Flush 0 Sent Done Done false
9535 3371 Acked Flush 0 Sent Done Done false
9536 3372 Acked Write 6 Sent Done Done false
9537 3373 Acked Read 9 Sent Done Done false
9538 3374 Acked Read 5 Sent Done Done false
9539 3375 Acked Flush 0 Sent Done Done false
9540 3376 Acked Write 3 Sent Done Done false
9541 3377 Acked Read 9 Sent Done Done false
9542 3378 Acked Read 9 Sent Done Done false
9543 3379 Acked Write 10 Sent Done Done false
9544 3380 Acked Write 8 Sent Done Done false
9545 3381 Acked Read 9 Sent Done Done false
9546 3382 Acked Read 1 Sent Done Done false
9547 3383 Acked Read 8 Sent Done Done false
9548 3384 Acked Write 8 Sent Done Done false
9549 3385 Acked Write 3 Sent Done Done false
9550 3386 Acked Read 5 Sent Done Done false
9551 3387 Acked Write 6 Sent Done Done false
9552 3388 Acked Read 9 Sent Done Done false
9553 3389 Acked Read 4 Sent Done Done false
9554 3390 Acked Read 7 Sent Done Done false
9555 3391 Acked Write 7 Sent Done Done false
9556 3392 Acked Read 10 Sent Done Done false
9557 3393 Acked Read 6 Sent Done Done false
9558 3394 Acked Write 7 Sent Done Done false
9559 3395 Acked Read 4 Sent Done Done false
9560 3396 Acked Flush 0 Sent Done Done false
9561 3397 Acked Read 2 Sent Done Done false
9562 3398 Acked Flush 0 Sent Done Done false
9563 3399 Acked Flush 0 Sent Done Done false
9564 3400 Acked Write 4 Sent Done Done false
9565 3401 Acked Flush 0 Sent Done Done false
9566 3402 Acked Read 7 Sent Done Done false
9567 3403 Acked Write 5 Sent Done Done false
9568 3404 Acked Read 4 Sent Done Done false
9569 3405 Acked Read 2 Sent Done Done false
9570 3406 Acked Read 3 Sent Done Done false
9571 3407 Acked Read 5 Sent Done Done false
9572 3408 Acked Write 9 Sent Done Done false
9573 3409 Acked Flush 0 Sent Done Done false
9574 3410 Acked Write 8 Sent Done Done false
9575 3411 Acked Read 4 Sent Done Done false
9576 3412 Acked Read 2 Sent Done Done false
9577 3413 Acked Write 5 Sent Done Done false
9578 3414 Acked Flush 0 Sent Done Done false
9579 STATES DS:0 DS:1 DS:2 TOTAL
9580 Sent 301 0 0 301
9581 Done 0 301 301 602
9582 Skipped 0 0 0 0
9583 Error 0 0 0 0
9584 Last Flush: 3113 3414 3414
9585 Downstairs last five completed: 3113 3112 3111 3110 3109
9586 Upstairs last five completed: 3414 3413 3412 3411 3410
9587 CLIENT: Up:0 ds:301 act:2
9588 [8810] got start action
9589 [8810] I am going to restart
9590 Starting downstairs at port 8810
9591 Make output file at "/tmp/dsc/downstairs-8810.txt"
9592 [8810][0] reports Starting
9593 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1212)
9594 [8810][0] reports Running
9595 ----------------------------------------------------------------
9596 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9597 States: Offline Active Active
9598 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9599 3114 Acked Flush 0 Sent Done Done false
9600 3115 Acked Flush 0 Sent Done Done false
9601 3116 Acked Flush 0 Sent Done Done false
9602 3117 Acked Write 10 Sent Done Done false
9603 3118 Acked Write 5 Sent Done Done false
9604 3119 Acked Read 9 Sent Done Done false
9605 3120 Acked Read 3 Sent Done Done false
9606 3121 Acked Write 6 Sent Done Done false
9607 3122 Acked Read 7 Sent Done Done false
9608 3123 Acked Read 2 Sent Done Done false
9609 3124 Acked Write 5 Sent Done Done false
9610 3125 Acked Write 3 Sent Done Done false
9611 3126 Acked Read 7 Sent Done Done false
9612 3127 Acked Write 2 Sent Done Done false
9613 3128 Acked Write 2 Sent Done Done false
9614 3129 Acked Write 3 Sent Done Done false
9615 3130 Acked Read 10 Sent Done Done false
9616 3131 Acked Read 6 Sent Done Done false
9617 3132 Acked Read 9 Sent Done Done false
9618 3133 Acked Read 1 Sent Done Done false
9619 3134 Acked Read 1 Sent Done Done false
9620 3135 Acked Read 9 Sent Done Done false
9621 3136 Acked Read 10 Sent Done Done false
9622 3137 Acked Read 6 Sent Done Done false
9623 3138 Acked Read 5 Sent Done Done false
9624 3139 Acked Write 10 Sent Done Done false
9625 3140 Acked Write 8 Sent Done Done false
9626 3141 Acked Write 3 Sent Done Done false
9627 3142 Acked Read 6 Sent Done Done false
9628 3143 Acked Write 6 Sent Done Done false
9629 3144 Acked Flush 0 Sent Done Done false
9630 3145 Acked Read 3 Sent Done Done false
9631 3146 Acked Read 10 Sent Done Done false
9632 3147 Acked Write 9 Sent Done Done false
9633 3148 Acked Read 3 Sent Done Done false
9634 3149 Acked Read 4 Sent Done Done false
9635 3150 Acked Read 10 Sent Done Done false
9636 3151 Acked Write 2 Sent Done Done false
9637 3152 Acked Write 7 Sent Done Done false
9638 3153 Acked Write 8 Sent Done Done false
9639 3154 Acked Flush 0 Sent Done Done false
9640 3155 Acked Read 8 Sent Done Done false
9641 3156 Acked Write 5 Sent Done Done false
9642 3157 Acked Flush 0 Sent Done Done false
9643 3158 Acked Write 2 Sent Done Done false
9644 3159 Acked Read 7 Sent Done Done false
9645 3160 Acked Read 3 Sent Done Done false
9646 3161 Acked Write 8 Sent Done Done false
9647 3162 Acked Read 3 Sent Done Done false
9648 3163 Acked Write 4 Sent Done Done false
9649 3164 Acked Write 3 Sent Done Done false
9650 3165 Acked Write 1 Sent Done Done false
9651 3166 Acked Write 7 Sent Done Done false
9652 3167 Acked Write 9 Sent Done Done false
9653 3168 Acked Write 6 Sent Done Done false
9654 3169 Acked Read 8 Sent Done Done false
9655 3170 Acked Flush 0 Sent Done Done false
9656 3171 Acked Read 7 Sent Done Done false
9657 3172 Acked Read 2 Sent Done Done false
9658 3173 Acked Read 10 Sent Done Done false
9659 3174 Acked Read 5 Sent Done Done false
9660 3175 Acked Read 2 Sent Done Done false
9661 3176 Acked Write 5 Sent Done Done false
9662 3177 Acked Flush 0 Sent Done Done false
9663 3178 Acked Write 4 Sent Done Done false
9664 3179 Acked Write 10 Sent Done Done false
9665 3180 Acked Write 9 Sent Done Done false
9666 3181 Acked Read 1 Sent Done Done false
9667 3182 Acked Read 8 Sent Done Done false
9668 3183 Acked Write 10 Sent Done Done false
9669 3184 Acked Write 1 Sent Done Done false
9670 3185 Acked Write 4 Sent Done Done false
9671 3186 Acked Read 6 Sent Done Done false
9672 3187 Acked Write 5 Sent Done Done false
9673 3188 Acked Write 4 Sent Done Done false
9674 3189 Acked Read 5 Sent Done Done false
9675 3190 Acked Read 4 Sent Done Done false
9676 3191 Acked Write 6 Sent Done Done false
9677 3192 Acked Read 3 Sent Done Done false
9678 3193 Acked Read 1 Sent Done Done false
9679 3194 Acked Write 5 Sent Done Done false
9680 3195 Acked Read 4 Sent Done Done false
9681 3196 Acked Read 3 Sent Done Done false
9682 3197 Acked Write 8 Sent Done Done false
9683 3198 Acked Write 10 Sent Done Done false
9684 3199 Acked Read 4 Sent Done Done false
9685 3200 Acked Write 10 Sent Done Done false
9686 3201 Acked Read 5 Sent Done Done false
9687 3202 Acked Read 10 Sent Done Done false
9688 3203 Acked Write 1 Sent Done Done false
9689 3204 Acked Read 5 Sent Done Done false
9690 3205 Acked Read 9 Sent Done Done false
9691 3206 Acked Read 9 Sent Done Done false
9692 3207 Acked Read 6 Sent Done Done false
9693 3208 Acked Write 1 Sent Done Done false
9694 3209 Acked Write 9 Sent Done Done false
9695 3210 Acked Read 3 Sent Done Done false
9696 3211 Acked Read 10 Sent Done Done false
9697 3212 Acked Read 7 Sent Done Done false
9698 3213 Acked Write 6 Sent Done Done false
9699 3214 Acked Read 4 Sent Done Done false
9700 3215 Acked Read 5 Sent Done Done false
9701 3216 Acked Write 5 Sent Done Done false
9702 3217 Acked Write 3 Sent Done Done false
9703 3218 Acked Read 8 Sent Done Done false
9704 3219 Acked Read 5 Sent Done Done false
9705 3220 Acked Flush 0 Sent Done Done false
9706 3221 Acked Write 4 Sent Done Done false
9707 3222 Acked Read 1 Sent Done Done false
9708 3223 Acked Flush 0 Sent Done Done false
9709 3224 Acked Flush 0 Sent Done Done false
9710 3225 Acked Read 6 Sent Done Done false
9711 3226 Acked Read 5 Sent Done Done false
9712 3227 Acked Write 9 Sent Done Done false
9713 3228 Acked Write 4 Sent Done Done false
9714 3229 Acked Read 5 Sent Done Done false
9715 3230 Acked Write 5 Sent Done Done false
9716 3231 Acked Read 1 Sent Done Done false
9717 3232 Acked Read 6 Sent Done Done false
9718 3233 Acked Read 4 Sent Done Done false
9719 3234 Acked Read 6 Sent Done Done false
9720 3235 Acked Read 10 Sent Done Done false
9721 3236 Acked Write 3 Sent Done Done false
9722 3237 Acked Write 8 Sent Done Done false
9723 3238 Acked Read 9 Sent Done Done false
9724 3239 Acked Write 1 Sent Done Done false
9725 3240 Acked Read 7 Sent Done Done false
9726 3241 Acked Read 6 Sent Done Done false
9727 3242 Acked Write 6 Sent Done Done false
9728 3243 Acked Write 7 Sent Done Done false
9729 3244 Acked Flush 0 Sent Done Done false
9730 3245 Acked Flush 0 Sent Done Done false
9731 3246 Acked Read 9 Sent Done Done false
9732 3247 Acked Write 1 Sent Done Done false
9733 3248 Acked Read 6 Sent Done Done false
9734 3249 Acked Read 3 Sent Done Done false
9735 3250 Acked Flush 0 Sent Done Done false
9736 3251 Acked Read 4 Sent Done Done false
9737 3252 Acked Write 4 Sent Done Done false
9738 3253 Acked Write 6 Sent Done Done false
9739 3254 Acked Write 10 Sent Done Done false
9740 3255 Acked Write 6 Sent Done Done false
9741 3256 Acked Read 7 Sent Done Done false
9742 3257 Acked Read 6 Sent Done Done false
9743 3258 Acked Read 3 Sent Done Done false
9744 3259 Acked Read 4 Sent Done Done false
9745 3260 Acked Write 8 Sent Done Done false
9746 3261 Acked Read 2 Sent Done Done false
9747 3262 Acked Write 1 Sent Done Done false
9748 3263 Acked Write 5 Sent Done Done false
9749 3264 Acked Read 2 Sent Done Done false
9750 3265 Acked Read 10 Sent Done Done false
9751 3266 Acked Write 7 Sent Done Done false
9752 3267 Acked Read 2 Sent Done Done false
9753 3268 Acked Read 5 Sent Done Done false
9754 3269 Acked Write 7 Sent Done Done false
9755 3270 Acked Write 3 Sent Done Done false
9756 3271 Acked Read 1 Sent Done Done false
9757 3272 Acked Write 6 Sent Done Done false
9758 3273 Acked Write 1 Sent Done Done false
9759 3274 Acked Read 1 Sent Done Done false
9760 3275 Acked Read 9 Sent Done Done false
9761 3276 Acked Read 6 Sent Done Done false
9762 3277 Acked Read 4 Sent Done Done false
9763 3278 Acked Write 8 Sent Done Done false
9764 3279 Acked Read 6 Sent Done Done false
9765 3280 Acked Write 8 Sent Done Done false
9766 3281 Acked Write 2 Sent Done Done false
9767 3282 Acked Write 3 Sent Done Done false
9768 3283 Acked Write 9 Sent Done Done false
9769 3284 Acked Write 8 Sent Done Done false
9770 3285 Acked Write 9 Sent Done Done false
9771 3286 Acked Read 6 Sent Done Done false
9772 3287 Acked Read 4 Sent Done Done false
9773 3288 Acked Read 8 Sent Done Done false
9774 3289 Acked Write 5 Sent Done Done false
9775 3290 Acked Read 4 Sent Done Done false
9776 3291 Acked Read 5 Sent Done Done false
9777 3292 Acked Flush 0 Sent Done Done false
9778 3293 Acked Read 9 Sent Done Done false
9779 3294 Acked Flush 0 Sent Done Done false
9780 3295 Acked Flush 0 Sent Done Done false
9781 3296 Acked Read 2 Sent Done Done false
9782 3297 Acked Read 6 Sent Done Done false
9783 3298 Acked Read 4 Sent Done Done false
9784 3299 Acked Write 10 Sent Done Done false
9785 3300 Acked Write 1 Sent Done Done false
9786 3301 Acked Flush 0 Sent Done Done false
9787 3302 Acked Write 6 Sent Done Done false
9788 3303 Acked Read 9 Sent Done Done false
9789 3304 Acked Write 9 Sent Done Done false
9790 3305 Acked Read 5 Sent Done Done false
9791 3306 Acked Read 9 Sent Done Done false
9792 3307 Acked Read 10 Sent Done Done false
9793 3308 Acked Write 10 Sent Done Done false
9794 3309 Acked Read 6 Sent Done Done false
9795 3310 Acked Write 4 Sent Done Done false
9796 3311 Acked Read 3 Sent Done Done false
9797 3312 Acked Write 9 Sent Done Done false
9798 3313 Acked Read 3 Sent Done Done false
9799 3314 Acked Read 6 Sent Done Done false
9800 3315 Acked Write 6 Sent Done Done false
9801 3316 Acked Read 3 Sent Done Done false
9802 3317 Acked Write 9 Sent Done Done false
9803 3318 Acked Write 7 Sent Done Done false
9804 3319 Acked Write 5 Sent Done Done false
9805 3320 Acked Write 8 Sent Done Done false
9806 3321 Acked Read 4 Sent Done Done false
9807 3322 Acked Read 1 Sent Done Done false
9808 3323 Acked Read 5 Sent Done Done false
9809 3324 Acked Read 1 Sent Done Done false
9810 3325 Acked Read 3 Sent Done Done false
9811 3326 Acked Read 6 Sent Done Done false
9812 3327 Acked Read 2 Sent Done Done false
9813 3328 Acked Write 5 Sent Done Done false
9814 3329 Acked Write 10 Sent Done Done false
9815 3330 Acked Read 7 Sent Done Done false
9816 3331 Acked Read 7 Sent Done Done false
9817 3332 Acked Write 6 Sent Done Done false
9818 3333 Acked Read 4 Sent Done Done false
9819 3334 Acked Read 8 Sent Done Done false
9820 3335 Acked Read 2 Sent Done Done false
9821 3336 Acked Read 6 Sent Done Done false
9822 3337 Acked Write 6 Sent Done Done false
9823 3338 Acked Read 1 Sent Done Done false
9824 3339 Acked Read 1 Sent Done Done false
9825 3340 Acked Write 3 Sent Done Done false
9826 3341 Acked Flush 0 Sent Done Done false
9827 3342 Acked Read 3 Sent Done Done false
9828 3343 Acked Write 3 Sent Done Done false
9829 3344 Acked Write 6 Sent Done Done false
9830 3345 Acked Read 9 Sent Done Done false
9831 3346 Acked Write 10 Sent Done Done false
9832 3347 Acked Write 6 Sent Done Done false
9833 3348 Acked Read 6 Sent Done Done false
9834 3349 Acked Read 1 Sent Done Done false
9835 3350 Acked Write 4 Sent Done Done false
9836 3351 Acked Write 4 Sent Done Done false
9837 3352 Acked Read 1 Sent Done Done false
9838 3353 Acked Read 9 Sent Done Done false
9839 3354 Acked Read 5 Sent Done Done false
9840 3355 Acked Write 3 Sent Done Done false
9841 3356 Acked Read 6 Sent Done Done false
9842 3357 Acked Write 4 Sent Done Done false
9843 3358 Acked Read 9 Sent Done Done false
9844 3359 Acked Read 3 Sent Done Done false
9845 3360 Acked Flush 0 Sent Done Done false
9846 3361 Acked Read 2 Sent Done Done false
9847 3362 Acked Read 7 Sent Done Done false
9848 3363 Acked Read 4 Sent Done Done false
9849 3364 Acked Flush 0 Sent Done Done false
9850 3365 Acked Read 3 Sent Done Done false
9851 3366 Acked Read 8 Sent Done Done false
9852 3367 Acked Flush 0 Sent Done Done false
9853 3368 Acked Write 1 Sent Done Done false
9854 3369 Acked Write 10 Sent Done Done false
9855 3370 Acked Flush 0 Sent Done Done false
9856 3371 Acked Flush 0 Sent Done Done false
9857 3372 Acked Write 6 Sent Done Done false
9858 3373 Acked Read 9 Sent Done Done false
9859 3374 Acked Read 5 Sent Done Done false
9860 3375 Acked Flush 0 Sent Done Done false
9861 3376 Acked Write 3 Sent Done Done false
9862 3377 Acked Read 9 Sent Done Done false
9863 3378 Acked Read 9 Sent Done Done false
9864 3379 Acked Write 10 Sent Done Done false
9865 3380 Acked Write 8 Sent Done Done false
9866 3381 Acked Read 9 Sent Done Done false
9867 3382 Acked Read 1 Sent Done Done false
9868 3383 Acked Read 8 Sent Done Done false
9869 3384 Acked Write 8 Sent Done Done false
9870 3385 Acked Write 3 Sent Done Done false
9871 3386 Acked Read 5 Sent Done Done false
9872 3387 Acked Write 6 Sent Done Done false
9873 3388 Acked Read 9 Sent Done Done false
9874 3389 Acked Read 4 Sent Done Done false
9875 3390 Acked Read 7 Sent Done Done false
9876 3391 Acked Write 7 Sent Done Done false
9877 3392 Acked Read 10 Sent Done Done false
9878 3393 Acked Read 6 Sent Done Done false
9879 3394 Acked Write 7 Sent Done Done false
9880 3395 Acked Read 4 Sent Done Done false
9881 3396 Acked Flush 0 Sent Done Done false
9882 3397 Acked Read 2 Sent Done Done false
9883 3398 Acked Flush 0 Sent Done Done false
9884 3399 Acked Flush 0 Sent Done Done false
9885 3400 Acked Write 4 Sent Done Done false
9886 3401 Acked Flush 0 Sent Done Done false
9887 3402 Acked Read 7 Sent Done Done false
9888 3403 Acked Write 5 Sent Done Done false
9889 3404 Acked Read 4 Sent Done Done false
9890 3405 Acked Read 2 Sent Done Done false
9891 3406 Acked Read 3 Sent Done Done false
9892 3407 Acked Read 5 Sent Done Done false
9893 3408 Acked Write 9 Sent Done Done false
9894 3409 Acked Flush 0 Sent Done Done false
9895 3410 Acked Write 8 Sent Done Done false
9896 3411 Acked Read 4 Sent Done Done false
9897 3412 Acked Read 2 Sent Done Done false
9898 3413 Acked Write 5 Sent Done Done false
9899 3414 Acked Flush 0 Sent Done Done false
9900 STATES DS:0 DS:1 DS:2 TOTAL
9901 Sent 301 0 0 301
9902 Done 0 301 301 602
9903 Skipped 0 0 0 0
9904 Error 0 0 0 0
9905 Last Flush: 3113 3414 3414
9906 Downstairs last five completed: 3113 3112 3111 3110 3109
9907 Upstairs last five completed: 3414 3413 3412 3411 3410
9908 CLIENT: Up:0 ds:301 act:2
99092025-01-11T06:07:32.214ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99102025-01-11T06:07:32.214ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35675) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99112025-01-11T06:07:32.215ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99122025-01-11T06:07:32.216ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99132025-01-11T06:07:32.216ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99142025-01-11T06:07:32.216ZINFOcrucible: send last flush ID to this DS: 3113 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99152025-01-11T06:07:32.216ZINFOcrucible: Replied this last flush ID: 3113 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99162025-01-11T06:07:32.216ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99172025-01-11T06:07:32.216ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99182025-01-11T06:07:32.216ZINFOcrucible: [0] client re-new 301 jobs since flush 3113 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99192025-01-11T06:07:32.216ZINFOcrucible: [0] Replayed 301 jobs since flush: 3113 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
9920 ----------------------------------------------------------------
9921 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9922 Downstairs last five completed: 3414 3413 3412 3411 3410
9923 Upstairs last five completed: 3414 3413 3412 3411 3410
9924 CLIENT: Up:0 ds:0 act:3
9925 Replay: All jobs finished
9926 Main task has work to do, go find it
9927 got dsc Stop(2)
9928 stop 2
9929 [8830] Got stop action so:false kr:false
9930 Jan 11 06:07:32.469 INFO request completed, latency_us: 336, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 1af901d6-e6b2-4c4a-b11b-dfc68890bb92, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
99312025-01-11T06:07:32.470ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
9932 Jan 11 06:07:32.470 INFO request completed, latency_us: 123, response_code: 200, uri: /state/cid/2, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:07:32.470653222Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
9933 method: GET, req_id: 19070f19-7714-4298-bbe0-0267b0d09418, remote_addr: 127{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:07:32.470684548Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
9934 .0.0.1:59925, local_addr: 127.0.0.1:9998{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:07:32.470759458Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
9935 
99362025-01-11T06:07:32.470ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
99372025-01-11T06:07:32.470ZINFOcrucible: client task is exiting = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
9938 [8830] Exited with: ExitStatus(unix_wait_status(9))
9939 [8830][2] reports Exit
9940 Jan 11 06:07:36.473 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/2, method: GET, req_id: b408f813-22a2-41b5-8706-d950954a38e4, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
9941 001/300 Read block 361 len 24576
9942 002/300 Write block 4609 len 20480 data: 3 3 2 2 2
9943 003/300 Read block 1410 len 28672
9944 004/300 Read block 4269 len 16384
9945 005/300 Flush
9946 006/300 Read block 3818 len 8192
9947 007/300 Write block 211 len 4096 data: 3
9948 008/300 Read block 3436 len 40960
9949 009/300 Write block 4584 len 20480 data: 3 4 4 4 3
9950 010/300 Read block 4525 len 32768
9951 011/300 Read block 4056 len 12288
9952 012/300 Read block 605 len 16384
9953 013/300 Write block 3696 len 40960 data: 4 3 3 3 3 3 2 3 3 3
9954 014/300 Read block 4687 len 20480
9955 015/300 Read block 4031 len 24576
9956 016/300 Read block 1242 len 16384
9957 017/300 Write block 1148 len 16384 data: 4 4 4 4
9958 018/300 Flush
9959 019/300 Write block 1783 len 12288 data: 4 4 4
9960 020/300 Flush
9961 021/300 Flush
9962 022/300 Write block 1104 len 8192 data: 2 2
9963 023/300 Read block 4027 len 36864
9964 024/300 Flush
9965 025/300 Read block 4327 len 36864
9966 026/300 Read block 2613 len 32768
9967 027/300 Read block 3337 len 8192
9968 028/300 Read block 2975 len 8192
9969 029/300 Read block 3004 len 20480
9970 030/300 Read block 3217 len 20480
9971 031/300 Read block 3198 len 28672
9972 032/300 Write block 2647 len 12288 data: 3 4 4
9973 033/300 Write block 4052 len 20480 data: 4 2 2 2 2
9974 034/300 Write block 4658 len 8192 data: 2 2
9975 035/300 Flush
9976 036/300 Read block 4411 len 24576
9977 037/300 Read block 2518 len 40960
9978 038/300 Write block 1130 len 24576 data: 3 2 2 3 3 3
9979 039/300 Flush
9980 040/300 Read block 3165 len 28672
9981 041/300 Read block 4390 len 12288
9982 042/300 Write block 2017 len 16384 data: 3 3 3 3
9983 043/300 Read block 2786 len 32768
9984 044/300 Read block 3141 len 8192
9985 045/300 Flush
9986 046/300 Read block 1676 len 32768
9987 047/300 Read block 4810 len 20480
9988 048/300 Write block 1062 len 12288 data: 4 4 3
9989 049/300 Write block 2050 len 32768 data: 3 3 3 5 5 5 5 5
9990 050/300 Read block 1361 len 16384
9991 051/300 Read block 90 len 20480
9992 052/300 Read block 4690 len 20480
9993 053/300 Read block 3138 len 32768
9994 054/300 Flush
9995 055/300 Flush
9996 056/300 Write block 129 len 12288 data: 2 3 3
9997 057/300 Read block 3998 len 20480
9998 058/300 Read block 1474 len 8192
9999 059/300 Write block 875 len 24576 data: 2 2 3 4 5 5
10000 060/300 Flush
10001 061/300 Write block 1357 len 8192 data: 4 5
10002 062/300 Read block 4176 len 36864
10003 063/300 Write block 2555 len 36864 data: 3 3 3 3 3 2 2 4 4
10004 064/300 Read block 4677 len 4096
10005 065/300 Flush
10006 066/300 Write block 1730 len 40960 data: 4 3 3 3 2 2 4 3 3 3
10007 067/300 Flush
10008 068/300 Read block 4290 len 32768
10009 069/300 Read block 4353 len 4096
10010 070/300 Read block 3220 len 8192
10011 071/300 Read block 4236 len 36864
10012 072/300 Write block 4262 len 20480 data: 3 4 5 4 5
10013 073/300 Read block 1924 len 36864
10014 074/300 Read block 4672 len 4096
10015 075/300 Read block 1838 len 12288
10016 076/300 Read block 1877 len 20480
10017 077/300 Read block 3958 len 36864
10018 078/300 Write block 2439 len 24576 data: 3 3 3 3 2 2
10019 079/300 Write block 906 len 12288 data: 4 4 4
10020 080/300 Flush
10021 081/300 Read block 1156 len 20480
10022 082/300 Read block 941 len 36864
10023 083/300 Write block 3883 len 20480 data: 4 4 4 4 4
10024 084/300 Write block 3790 len 20480 data: 3 2 2 3 3
10025 085/300 Read block 789 len 4096
10026 086/300 Read block 1568 len 16384
10027 087/300 Read block 545 len 16384
10028 088/300 Flush
10029 089/300 Write block 3204 len 40960 data: 3 2 3 4 4 5 5 5 4 2
10030 090/300 Read block 2614 len 12288
10031 091/300 Read block 1312 len 36864
10032 092/300 Read block 258 len 32768
10033 093/300 Read block 4623 len 36864
10034 094/300 Read block 4031 len 28672
10035 095/300 Read block 4756 len 12288
10036 096/300 Write block 870 len 20480 data: 2 2 2 2 2
10037 097/300 Read block 451 len 16384
10038 098/300 Write block 3972 len 8192 data: 2 3
10039 099/300 Read block 2688 len 20480
10040 100/300 Write block 996 len 28672 data: 3 3 2 2 2 4 4
10041 101/300 Write block 3084 len 36864 data: 2 3 3 3 3 5 5 5 5
10042 102/300 Flush
10043 103/300 Read block 3795 len 36864
10044 104/300 Write block 4829 len 32768 data: 3 3 3 3 3 4 4 5
10045 105/300 Write block 3216 len 28672 data: 2 2 2 2 2 2 2
10046 106/300 Read block 2606 len 28672
10047 107/300 Read block 3855 len 24576
10048 108/300 Read block 963 len 20480
10049 109/300 Flush
10050 110/300 Read block 3711 len 8192
10051 111/300 Read block 276 len 32768
10052 112/300 Write block 227 len 40960 data: 2 2 2 2 2 2 2 2 2 3
10053 113/300 Flush
10054 114/300 Read block 4161 len 32768
10055 115/300 Write block 2735 len 36864 data: 4 4 3 3 3 3 3 3 2
10056 116/300 Write block 2015 len 40960 data: 3 3 4 4 4 4 2 2 2 2
10057 117/300 Write block 2041 len 40960 data: 3 4 4 4 3 3 3 3 3 4
10058 118/300 Read block 2160 len 4096
10059 119/300 Read block 857 len 12288
10060 120/300 Read block 2377 len 32768
10061 121/300 Read block 2267 len 36864
10062 122/300 Write block 2801 len 24576 data: 4 3 3 4 3 3
10063 123/300 Read block 2870 len 16384
10064 124/300 Write block 3153 len 4096 data: 2
10065 125/300 Read block 3807 len 32768
10066 126/300 Write block 1211 len 40960 data: 3 2 2 2 2 2 2 3 3 3
10067 127/300 Write block 2833 len 8192 data: 2 2
10068 128/300 Write block 2674 len 16384 data: 2 2 2 4
10069 129/300 Write block 971 len 24576 data: 6 5 5 5 5 6
10070 130/300 Read block 2211 len 40960
10071 131/300 Read block 2156 len 16384
10072 132/300 Read block 2159 len 4096
10073 133/300 Flush
10074 134/300 Write block 23 len 4096 data: 4
10075 135/300 Write block 4122 len 4096 data: 3
10076 136/300 Read block 59 len 12288
10077 137/300 Read block 4543 len 24576
10078 138/300 Flush
10079 139/300 Write block 754 len 24576 data: 3 2 2 2 2 2
10080 140/300 Read block 4266 len 8192
10081 141/300 Read block 3812 len 28672
10082 142/300 Read block 293 len 36864
10083 143/300 Read block 722 len 24576
10084 144/300 Read block 4171 len 20480
10085 145/300 Read block 4536 len 16384
10086 146/300 Flush
10087 147/300 Read block 2733 len 8192
10088 148/300 Write block 524 len 40960 data: 2 2 2 2 3 2 3 3 3 3
10089 149/300 Write block 4892 len 16384 data: 2 2 2 2
10090 150/300 Read block 3081 len 32768
10091 151/300 Read block 973 len 20480
10092 152/300 Read block 3609 len 8192
10093 153/300 Read block 1289 len 32768
10094 154/300 Read block 4423 len 16384
10095 155/300 Write block 3387 len 8192 data: 3 3
10096 156/300 Flush
10097 157/300 Flush
10098 158/300 Write block 1940 len 12288 data: 4 4 4
10099 159/300 Read block 923 len 24576
10100 160/300 Write block 2895 len 16384 data: 3 4 3 3
10101 161/300 Write block 2928 len 20480 data: 5 5 5 6 6
10102 162/300 Read block 1469 len 8192
10103 163/300 Write block 3628 len 40960 data: 2 2 3 3 3 3 2 2 2 2
10104 164/300 Flush
10105 165/300 Read block 4919 len 32768
10106 166/300 Read block 2219 len 16384
10107 167/300 Write block 4134 len 20480 data: 3 4 4 4 4
10108 168/300 Write block 2430 len 24576 data: 6 6 5 4 3 3
10109 169/300 Read block 3065 len 8192
10110 170/300 Read block 1999 len 12288
10111 171/300 Write block 4893 len 4096 data: 3
10112 172/300 Read block 3187 len 8192
10113 173/300 Read block 4582 len 36864
10114 174/300 Read block 3276 len 20480
10115 175/300 Read block 1021 len 8192
10116 176/300 Write block 2441 len 40960 data: 4 4 3 3 2 2 2 3 2 2
10117 177/300 Write block 3272 len 36864 data: 4 4 4 4 4 5 5 5 4
10118 178/300 Read block 2724 len 20480
10119 179/300 Write block 1365 len 24576 data: 5 4 4 4 3 2
10120 180/300 Read block 4355 len 24576
10121 181/300 Read block 4015 len 24576
10122 182/300 Write block 2618 len 12288 data: 3 2 2
10123 183/300 Write block 3605 len 8192 data: 2 2
10124 184/300 Read block 4572 len 32768
10125 185/300 Read block 276 len 24576
10126 186/300 Write block 1471 len 40960 data: 4 3 3 3 3 2 2 2 2 3
10127 187/300 Write block 4358 len 12288 data: 2 3 3
10128 188/300 Read block 1707 len 12288
10129 189/300 Read block 204 len 40960
10130 190/300 Read block 3251 len 36864
10131 191/300 Write block 3836 len 12288 data: 2 2 2
10132 192/300 Read block 1688 len 16384
10133 193/300 Flush
10134 194/300 Read block 3847 len 20480
10135 195/300 Write block 62 len 28672 data: 5 4 3 3 2 2 2
10136 196/300 Flush
10137 197/300 Read block 4131 len 28672
10138 198/300 Write block 1261 len 36864 data: 3 2 2 3 2 2 2 2 2
10139 199/300 Read block 2512 len 4096
10140 200/300 Flush
10141 201/300 Read block 1963 len 20480
10142 202/300 Flush
10143 203/300 Write block 1927 len 28672 data: 4 4 4 4 4 4 3
10144 204/300 Read block 4821 len 12288
10145 205/300 Write block 4610 len 8192 data: 4 3
10146 206/300 Flush
10147 207/300 Read block 4410 len 8192
10148 208/300 Read block 2920 len 12288
10149 209/300 Flush
10150 210/300 Write block 3295 len 8192 data: 2 2
10151 211/300 Write block 3989 len 28672 data: 2 2 2 2 2 3 3
10152 212/300 Read block 361 len 12288
10153 213/300 Write block 1366 len 32768 data: 5 5 5 4 3 2 3 3
10154 214/300 Flush
10155 215/300 Read block 4877 len 4096
10156 216/300 Write block 2229 len 8192 data: 3 4
10157 217/300 Read block 2800 len 8192
10158 218/300 Write block 1748 len 28672 data: 2 2 2 2 2 2 3
10159 219/300 Read block 3 len 36864
10160 220/300 Read block 3684 len 8192
10161 221/300 Write block 464 len 28672 data: 4 3 5 5 5 5 4
10162 222/300 Read block 2522 len 4096
10163 223/300 Read block 2043 len 20480
10164 224/300 Write block 2780 len 28672 data: 3 2 3 3 3 4 4
10165 225/300 Flush
10166 226/300 Read block 598 len 24576
10167 227/300 Flush
10168 228/300 Read block 853 len 32768
10169 229/300 Read block 3328 len 28672
10170 230/300 Write block 596 len 8192 data: 3 3
10171 231/300 Write block 758 len 24576 data: 3 3 2 2 3 3
10172 232/300 Write block 128 len 32768 data: 2 3 4 4 3 4 4 5
10173 233/300 Write block 4127 len 24576 data: 3 4 3 4 3 3
10174 234/300 Write block 1958 len 28672 data: 3 3 2 3 3 3 3
10175 235/300 Write block 3147 len 40960 data: 3 3 3 3 3 2 3 3 3 3
10176 236/300 Read block 962 len 8192
10177 237/300 Read block 2541 len 36864
10178 238/300 Write block 3759 len 8192 data: 4 4
10179 239/300 Read block 3437 len 8192
10180 240/300 Write block 3342 len 4096 data: 2
10181 241/300 Write block 3444 len 24576 data: 3 3 4 4 5 5
10182 242/300 Read block 3527 len 16384
10183 243/300 Write block 4607 len 12288 data: 3 3 4
10184 244/300 Read block 3578 len 28672
10185 245/300 Read block 4757 len 16384
10186 246/300 Write block 568 len 12288 data: 3 2 2
10187 247/300 Write block 2077 len 32768 data: 3 3 3 2 2 3 3 3
10188 248/300 Write block 4088 len 20480 data: 3 3 2 2 2
10189 249/300 Write block 840 len 4096 data: 2
10190 250/300 Read block 232 len 28672
10191 251/300 Read block 2159 len 24576
10192 252/300 Read block 4663 len 20480
10193 253/300 Write block 533 len 4096 data: 4
10194 254/300 Write block 2479 len 24576 data: 3 3 2 2 2 2
10195 255/300 Write block 642 len 16384 data: 2 2 2 2
10196 256/300 Read block 4508 len 24576
10197 257/300 Write block 2489 len 16384 data: 2 2 2 2
10198 258/300 Read block 806 len 4096
10199 259/300 Write block 2568 len 20480 data: 4 4 4 2 3
10200 260/300 Read block 2712 len 28672
10201 261/300 Read block 1902 len 8192
10202 262/300 Read block 4078 len 12288
10203 263/300 Flush
10204 264/300 Read block 4380 len 12288
10205 265/300 Read block 4789 len 40960
10206 266/300 Flush
10207 267/300 Write block 4215 len 40960 data: 2 2 2 2 3 3 3 3 3 3
10208 268/300 Flush
10209 269/300 Write block 2771 len 28672 data: 2 3 3 3 3 3 3
10210 270/300 Write block 227 len 36864 data: 3 3 3 3 3 3 3 3 3
10211 271/300 Read block 2675 len 36864
10212 272/300 Write block 1252 len 32768 data: 3 2 3 3 3 3 3 3
10213 273/300 Read block 2640 len 24576
10214 274/300 Read block 1458 len 12288
10215 275/300 Write block 63 len 36864 data: 5 4 4 3 3 3 3 3 3
10216 276/300 Flush
10217 277/300 Write block 2187 len 20480 data: 4 4 4 3 3
10218 278/300 Flush
10219 279/300 Read block 4665 len 32768
10220 280/300 Read block 1197 len 16384
10221 281/300 Read block 4837 len 12288
10222 282/300 Read block 2509 len 36864
10223 283/300 Flush
10224 284/300 Write block 3836 len 4096 data: 3
10225 285/300 Read block 4305 len 16384
10226 286/300 Write block 1096 len 36864 data: 2 3 3 2 2 2 2 2 3
10227 287/300 Read block 1749 len 36864
10228 288/300 Read block 3551 len 20480
10229 289/300 Write block 30 len 32768 data: 3 2 2 2 3 3 2 4
10230 290/300 Write block 4397 len 40960 data: 3 3 3 3 4 4 5 4 4 6
10231 291/300 Write block 2281 len 4096 data: 4
10232 292/300 Write block 1021 len 24576 data: 2 2 4 3 6 6
10233 293/300 Write block 1498 len 4096 data: 4
10234 294/300 Write block 2818 len 32768 data: 5 5 5 5 5 5 5 2
10235 295/300 Write block 1220 len 4096 data: 4
10236 296/300 Write block 2372 len 32768 data: 2 2 2 2 3 3 3 3
10237 297/300 Read block 1093 len 4096
10238 298/300 Write block 3269 len 32768 data: 3 4 4 5 5 5 5 5
10239 299/300 Read block 1293 len 32768
10240 300/300 Write block 3769 len 28672 data: 2 2 2 2 2 2 2
10241 301/300 Flush
10242 Main task has work to do, go find it
10243 got dsc Start(2)
10244 start 2
10245 Jan 11 06:07:36.710 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/2, method: POST, req_id: bfc25ac8-d622-487f-9ed4-5993f2454109, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
10246 Replay: started 2, returned:Ok(())
10247 ----------------------------------------------------------------
10248 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10249 States: Active Active Offline
10250 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10251 3415 Acked Read 6 Done Done Sent false
10252 3416 Acked Write 5 Done Done Sent false
10253 3417 Acked Read 7 Done Done Sent false
10254 3418 Acked Read 4 Done Done Sent false
10255 3419 Acked Flush 0 Done Done Sent false
10256 3420 Acked Read 2 Done Done Sent false
10257 3421 Acked Write 1 Done Done Sent false
10258 3422 Acked Read 10 Done Done Sent false
10259 3423 Acked Write 5 Done Done Sent false
10260 3424 Acked Read 8 Done Done Sent false
10261 3425 Acked Read 3 Done Done Sent false
10262 3426 Acked Read 4 Done Done Sent false
10263 3427 Acked Write 10 Done Done Sent false
10264 3428 Acked Read 5 Done Done Sent false
10265 3429 Acked Read 6 Done Done Sent false
10266 3430 Acked Read 4 Done Done Sent false
10267 3431 Acked Write 4 Done Done Sent false
10268 3432 Acked Flush 0 Done Done Sent false
10269 3433 Acked Write 3 Done Done Sent false
10270 3434 Acked Flush 0 Done Done Sent false
10271 3435 Acked Flush 0 Done Done Sent false
10272 3436 Acked Write 2 Done Done Sent false
10273 3437 Acked Read 9 Done Done Sent false
10274 3438 Acked Flush 0 Done Done Sent false
10275 3439 Acked Read 9 Done Done Sent false
10276 3440 Acked Read 8 Done Done Sent false
10277 3441 Acked Read 2 Done Done Sent false
10278 3442 Acked Read 2 Done Done Sent false
10279 3443 Acked Read 5 Done Done Sent false
10280 3444 Acked Read 5 Done Done Sent false
10281 3445 Acked Read 7 Done Done Sent false
10282 3446 Acked Write 3 Done Done Sent false
10283 3447 Acked Write 5 Done Done Sent false
10284 3448 Acked Write 2 Done Done Sent false
10285 3449 Acked Flush 0 Done Done Sent false
10286 3450 Acked Read 6 Done Done Sent false
10287 3451 Acked Read 10 Done Done Sent false
10288 3452 Acked Write 6 Done Done Sent false
10289 3453 Acked Flush 0 Done Done Sent false
10290 3454 Acked Read 7 Done Done Sent false
10291 3455 Acked Read 3 Done Done Sent false
10292 3456 Acked Write 4 Done Done Sent false
10293 3457 Acked Read 8 Done Done Sent false
10294 3458 Acked Read 2 Done Done Sent false
10295 3459 Acked Flush 0 Done Done Sent false
10296 3460 Acked Read 8 Done Done Sent false
10297 3461 Acked Read 5 Done Done Sent false
10298 3462 Acked Write 3 Done Done Sent false
10299 3463 Acked Write 8 Done Done Sent false
10300 3464 Acked Read 4 Done Done Sent false
10301 3465 Acked Read 5 Done Done Sent false
10302 3466 Acked Read 5 Done Done Sent false
10303 3467 Acked Read 8 Done Done Sent false
10304 3468 Acked Flush 0 Done Done Sent false
10305 3469 Acked Flush 0 Done Done Sent false
10306 3470 Acked Write 3 Done Done Sent false
10307 3471 Acked Read 5 Done Done Sent false
10308 3472 Acked Read 2 Done Done Sent false
10309 3473 Acked Write 6 Done Done Sent false
10310 3474 Acked Flush 0 Done Done Sent false
10311 3475 Acked Write 2 Done Done Sent false
10312 3476 Acked Read 9 Done Done Sent false
10313 3477 Acked Write 9 Done Done Sent false
10314 3478 Acked Read 1 Done Done Sent false
10315 3479 Acked Flush 0 Done Done Sent false
10316 3480 Acked Write 10 Done Done Sent false
10317 3481 Acked Flush 0 Done Done Sent false
10318 3482 Acked Read 8 Done Done Sent false
10319 3483 Acked Read 1 Done Done Sent false
10320 3484 Acked Read 2 Done Done Sent false
10321 3485 Acked Read 9 Done Done Sent false
10322 3486 Acked Write 5 Done Done Sent false
10323 3487 Acked Read 9 Done Done Sent false
10324 3488 Acked Read 1 Done Done Sent false
10325 3489 Acked Read 3 Done Done Sent false
10326 3490 Acked Read 5 Done Done Sent false
10327 3491 Acked Read 9 Done Done Sent false
10328 3492 Acked Write 6 Done Done Sent false
10329 3493 Acked Write 3 Done Done Sent false
10330 3494 Acked Flush 0 Done Done Sent false
10331 3495 Acked Read 5 Done Done Sent false
10332 3496 Acked Read 9 Done Done Sent false
10333 3497 Acked Write 5 Done Done Sent false
10334 3498 Acked Write 5 Done Done Sent false
10335 3499 Acked Read 1 Done Done Sent false
10336 3500 Acked Read 4 Done Done Sent false
10337 3501 Acked Read 4 Done Done Sent false
10338 3502 Acked Flush 0 Done Done Sent false
10339 3503 Acked Write 10 Done Done Sent false
10340 3504 Acked Read 3 Done Done Sent false
10341 3505 Acked Read 9 Done Done Sent false
10342 3506 Acked Read 8 Done Done Sent false
10343 3507 Acked Read 9 Done Done Sent false
10344 3508 Acked Read 7 Done Done Sent false
10345 3509 Acked Read 3 Done Done Sent false
10346 3510 Acked Write 5 Done Done Sent false
10347 3511 Acked Read 4 Done Done Sent false
10348 3512 Acked Write 2 Done Done Sent false
10349 3513 Acked Read 5 Done Done Sent false
10350 3514 Acked Write 7 Done Done Sent false
10351 3515 Acked Write 9 Done Done Sent false
10352 3516 Acked Flush 0 Done Done Sent false
10353 3517 Acked Read 9 Done Done Sent false
10354 3518 Acked Write 8 Done Done Sent false
10355 3519 Acked Write 7 Done Done Sent false
10356 3520 Acked Read 7 Done Done Sent false
10357 3521 Acked Read 6 Done Done Sent false
10358 3522 Acked Read 5 Done Done Sent false
10359 3523 Acked Flush 0 Done Done Sent false
10360 3524 Acked Read 2 Done Done Sent false
10361 3525 Acked Read 8 Done Done Sent false
10362 3526 Acked Write 10 Done Done Sent false
10363 3527 Acked Flush 0 Done Done Sent false
10364 3528 Acked Read 8 Done Done Sent false
10365 3529 Acked Write 9 Done Done Sent false
10366 3530 Acked Write 10 Done Done Sent false
10367 3531 Acked Write 10 Done Done Sent false
10368 3532 Acked Read 1 Done Done Sent false
10369 3533 Acked Read 3 Done Done Sent false
10370 3534 Acked Read 8 Done Done Sent false
10371 3535 Acked Read 9 Done Done Sent false
10372 3536 Acked Write 6 Done Done Sent false
10373 3537 Acked Read 4 Done Done Sent false
10374 3538 Acked Write 1 Done Done Sent false
10375 3539 Acked Read 8 Done Done Sent false
10376 3540 Acked Write 10 Done Done Sent false
10377 3541 Acked Write 2 Done Done Sent false
10378 3542 Acked Write 4 Done Done Sent false
10379 3543 Acked Write 6 Done Done Sent false
10380 3544 Acked Read 10 Done Done Sent false
10381 3545 Acked Read 4 Done Done Sent false
10382 3546 Acked Read 1 Done Done Sent false
10383 3547 Acked Flush 0 Done Done Sent false
10384 3548 Acked Write 1 Done Done Sent false
10385 3549 Acked Write 1 Done Done Sent false
10386 3550 Acked Read 3 Done Done Sent false
10387 3551 Acked Read 6 Done Done Sent false
10388 3552 Acked Flush 0 Done Done Sent false
10389 3553 Acked Write 6 Done Done Sent false
10390 3554 Acked Read 2 Done Done Sent false
10391 3555 Acked Read 7 Done Done Sent false
10392 3556 Acked Read 9 Done Done Sent false
10393 3557 Acked Read 6 Done Done Sent false
10394 3558 Acked Read 5 Done Done Sent false
10395 3559 Acked Read 4 Done Done Sent false
10396 3560 Acked Flush 0 Done Done Sent false
10397 3561 Acked Read 2 Done Done Sent false
10398 3562 Acked Write 10 Done Done Sent false
10399 3563 Acked Write 4 Done Done Sent false
10400 3564 Acked Read 8 Done Done Sent false
10401 3565 Acked Read 5 Done Done Sent false
10402 3566 Acked Read 2 Done Done Sent false
10403 3567 Acked Read 8 Done Done Sent false
10404 3568 Acked Read 4 Done Done Sent false
10405 3569 Acked Write 2 Done Done Sent false
10406 3570 Acked Flush 0 Done Done Sent false
10407 3571 Acked Flush 0 Done Done Sent false
10408 3572 Acked Write 3 Done Done Sent false
10409 3573 Acked Read 6 Done Done Sent false
10410 3574 Acked Write 4 Done Done Sent false
10411 3575 Acked Write 5 Done Done Sent false
10412 3576 Acked Read 2 Done Done Sent false
10413 3577 Acked Write 10 Done Done Sent false
10414 3578 Acked Flush 0 Done Done Sent false
10415 3579 Acked Read 8 Done Done Sent false
10416 3580 Acked Read 4 Done Done Sent false
10417 3581 Acked Write 5 Done Done Sent false
10418 3582 Acked Write 6 Done Done Sent false
10419 3583 Acked Read 2 Done Done Sent false
10420 3584 Acked Read 3 Done Done Sent false
10421 3585 Acked Write 1 Done Done Sent false
10422 3586 Acked Read 2 Done Done Sent false
10423 3587 Acked Read 9 Done Done Sent false
10424 3588 Acked Read 5 Done Done Sent false
10425 3589 Acked Read 2 Done Done Sent false
10426 3590 Acked Write 10 Done Done Sent false
10427 3591 Acked Write 9 Done Done Sent false
10428 3592 Acked Read 5 Done Done Sent false
10429 3593 Acked Write 6 Done Done Sent false
10430 3594 Acked Read 6 Done Done Sent false
10431 3595 Acked Read 6 Done Done Sent false
10432 3596 Acked Write 3 Done Done Sent false
10433 3597 Acked Write 2 Done Done Sent false
10434 3598 Acked Read 8 Done Done Sent false
10435 3599 Acked Read 6 Done Done Sent false
10436 3600 Acked Write 10 Done Done Sent false
10437 3601 Acked Write 3 Done Done Sent false
10438 3602 Acked Read 3 Done Done Sent false
10439 3603 Acked Read 10 Done Done Sent false
10440 3604 Acked Read 9 Done Done Sent false
10441 3605 Acked Write 3 Done Done Sent false
10442 3606 Acked Read 4 Done Done Sent false
10443 3607 Acked Flush 0 Done Done Sent false
10444 3608 Acked Read 5 Done Done Sent false
10445 3609 Acked Write 7 Done Done Sent false
10446 3610 Acked Flush 0 Done Done Sent false
10447 3611 Acked Read 7 Done Done Sent false
10448 3612 Acked Write 9 Done Done Sent false
10449 3613 Acked Read 1 Done Done Sent false
10450 3614 Acked Flush 0 Done Done Sent false
10451 3615 Acked Read 5 Done Done Sent false
10452 3616 Acked Flush 0 Done Done Sent false
10453 3617 Acked Write 7 Done Done Sent false
10454 3618 Acked Read 3 Done Done Sent false
10455 3619 Acked Write 2 Done Done Sent false
10456 3620 Acked Flush 0 Done Done Sent false
10457 3621 Acked Read 2 Done Done Sent false
10458 3622 Acked Read 3 Done Done Sent false
10459 3623 Acked Flush 0 Done Done Sent false
10460 3624 Acked Write 2 Done Done Sent false
10461 3625 Acked Write 7 Done Done Sent false
10462 3626 Acked Read 3 Done Done Sent false
10463 3627 Acked Write 8 Done Done Sent false
10464 3628 Acked Flush 0 Done Done Sent false
10465 3629 Acked Read 1 Done Done Sent false
10466 3630 Acked Write 2 Done Done Sent false
10467 3631 Acked Read 2 Done Done Sent false
10468 3632 Acked Write 7 Done Done Sent false
10469 3633 Acked Read 9 Done Done Sent false
10470 3634 Acked Read 2 Done Done Sent false
10471 3635 Acked Write 7 Done Done Sent false
10472 3636 Acked Read 1 Done Done Sent false
10473 3637 Acked Read 5 Done Done Sent false
10474 3638 Acked Write 7 Done Done Sent false
10475 3639 Acked Flush 0 Done Done Sent false
10476 3640 Acked Read 6 Done Done Sent false
10477 3641 Acked Flush 0 Done Done Sent false
10478 3642 Acked Read 8 Done Done Sent false
10479 3643 Acked Read 7 Done Done Sent false
10480 3644 Acked Write 2 Done Done Sent false
10481 3645 Acked Write 6 Done Done Sent false
10482 3646 Acked Write 8 Done Done Sent false
10483 3647 Acked Write 6 Done Done Sent false
10484 3648 Acked Write 7 Done Done Sent false
10485 3649 Acked Write 10 Done Done Sent false
10486 3650 Acked Read 2 Done Done Sent false
10487 3651 Acked Read 9 Done Done Sent false
10488 3652 Acked Write 2 Done Done Sent false
10489 3653 Acked Read 2 Done Done Sent false
10490 3654 Acked Write 1 Done Done Sent false
10491 3655 Acked Write 6 Done Done Sent false
10492 3656 Acked Read 4 Done Done Sent false
10493 3657 Acked Write 3 Done Done Sent false
10494 3658 Acked Read 7 Done Done Sent false
10495 3659 Acked Read 4 Done Done Sent false
10496 3660 Acked Write 3 Done Done Sent false
10497 3661 Acked Write 8 Done Done Sent false
10498 3662 Acked Write 5 Done Done Sent false
10499 3663 Acked Write 1 Done Done Sent false
10500 3664 Acked Read 7 Done Done Sent false
10501 3665 Acked Read 6 Done Done Sent false
10502 3666 Acked Read 5 Done Done Sent false
10503 3667 Acked Write 1 Done Done Sent false
10504 3668 Acked Write 6 Done Done Sent false
10505 3669 Acked Write 4 Done Done Sent false
10506 3670 Acked Read 6 Done Done Sent false
10507 3671 Acked Write 4 Done Done Sent false
10508 3672 Acked Read 1 Done Done Sent false
10509 3673 Acked Write 5 Done Done Sent false
10510 3674 Acked Read 7 Done Done Sent false
10511 3675 Acked Read 2 Done Done Sent false
10512 3676 Acked Read 3 Done Done Sent false
10513 3677 Acked Flush 0 Done Done Sent false
10514 3678 Acked Read 3 Done Done Sent false
10515 3679 Acked Read 10 Done Done Sent false
10516 3680 Acked Flush 0 Done Done Sent false
10517 3681 Acked Write 10 Done Done Sent false
10518 3682 Acked Flush 0 Done Done Sent false
10519 3683 Acked Write 7 Done Done Sent false
10520 3684 Acked Write 9 Done Done Sent false
10521 3685 Acked Read 9 Done Done Sent false
10522 3686 Acked Write 8 Done Done Sent false
10523 3687 Acked Read 6 Done Done Sent false
10524 3688 Acked Read 3 Done Done Sent false
10525 3689 Acked Write 9 Done Done Sent false
10526 3690 Acked Flush 0 Done Done Sent false
10527 3691 Acked Write 5 Done Done Sent false
10528 3692 Acked Flush 0 Done Done Sent false
10529 3693 Acked Read 8 Done Done Sent false
10530 3694 Acked Read 4 Done Done Sent false
10531 3695 Acked Read 3 Done Done Sent false
10532 3696 Acked Read 9 Done Done Sent false
10533 3697 Acked Flush 0 Done Done Sent false
10534 3698 Acked Write 1 Done Done Sent false
10535 3699 Acked Read 4 Done Done Sent false
10536 3700 Acked Write 9 Done Done Sent false
10537 3701 Acked Read 9 Done Done Sent false
10538 3702 Acked Read 5 Done Done Sent false
10539 3703 Acked Write 8 Done Done Sent false
10540 3704 Acked Write 10 Done Done Sent false
10541 3705 Acked Write 1 Done Done Sent false
10542 3706 Acked Write 6 Done Done Sent false
10543 3707 Acked Write 1 Done Done Sent false
10544 3708 Acked Write 8 Done Done Sent false
10545 3709 Acked Write 1 Done Done Sent false
10546 3710 Acked Write 8 Done Done Sent false
10547 3711 Acked Read 1 Done Done Sent false
10548 3712 Acked Write 8 Done Done Sent false
10549 3713 Acked Read 8 Done Done Sent false
10550 3714 Acked Write 7 Done Done Sent false
10551 3715 Acked Flush 0 Done Done Sent false
10552 STATES DS:0 DS:1 DS:2 TOTAL
10553 Sent 0 0 301 301
10554 Done 301 301 0 602
10555 Skipped 0 0 0 0
10556 Error 0 0 0 0
10557 Last Flush: 3715 3715 3414
10558 Downstairs last five completed: 3414 3413 3412 3411 3410
10559 Upstairs last five completed: 3715 3714 3713 3712 3711
10560 CLIENT: Up:0 ds:301 act:2
10561 [8830] got start action
10562 [8830] I am going to restart
10563 Starting downstairs at port 8830
10564 Make output file at "/tmp/dsc/downstairs-8830.txt"
10565 [8830][2] reports Starting
10566 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1213)
10567 [8830][2] reports Running
10568 ----------------------------------------------------------------
10569 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10570 States: Active Active Offline
10571 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10572 3415 Acked Read 6 Done Done Sent false
10573 3416 Acked Write 5 Done Done Sent false
10574 3417 Acked Read 7 Done Done Sent false
10575 3418 Acked Read 4 Done Done Sent false
10576 3419 Acked Flush 0 Done Done Sent false
10577 3420 Acked Read 2 Done Done Sent false
10578 3421 Acked Write 1 Done Done Sent false
10579 3422 Acked Read 10 Done Done Sent false
10580 3423 Acked Write 5 Done Done Sent false
10581 3424 Acked Read 8 Done Done Sent false
10582 3425 Acked Read 3 Done Done Sent false
10583 3426 Acked Read 4 Done Done Sent false
10584 3427 Acked Write 10 Done Done Sent false
10585 3428 Acked Read 5 Done Done Sent false
10586 3429 Acked Read 6 Done Done Sent false
10587 3430 Acked Read 4 Done Done Sent false
10588 3431 Acked Write 4 Done Done Sent false
10589 3432 Acked Flush 0 Done Done Sent false
10590 3433 Acked Write 3 Done Done Sent false
10591 3434 Acked Flush 0 Done Done Sent false
10592 3435 Acked Flush 0 Done Done Sent false
10593 3436 Acked Write 2 Done Done Sent false
10594 3437 Acked Read 9 Done Done Sent false
10595 3438 Acked Flush 0 Done Done Sent false
10596 3439 Acked Read 9 Done Done Sent false
10597 3440 Acked Read 8 Done Done Sent false
10598 3441 Acked Read 2 Done Done Sent false
10599 3442 Acked Read 2 Done Done Sent false
10600 3443 Acked Read 5 Done Done Sent false
10601 3444 Acked Read 5 Done Done Sent false
10602 3445 Acked Read 7 Done Done Sent false
10603 3446 Acked Write 3 Done Done Sent false
10604 3447 Acked Write 5 Done Done Sent false
10605 3448 Acked Write 2 Done Done Sent false
10606 3449 Acked Flush 0 Done Done Sent false
10607 3450 Acked Read 6 Done Done Sent false
10608 3451 Acked Read 10 Done Done Sent false
10609 3452 Acked Write 6 Done Done Sent false
10610 3453 Acked Flush 0 Done Done Sent false
10611 3454 Acked Read 7 Done Done Sent false
10612 3455 Acked Read 3 Done Done Sent false
10613 3456 Acked Write 4 Done Done Sent false
10614 3457 Acked Read 8 Done Done Sent false
10615 3458 Acked Read 2 Done Done Sent false
10616 3459 Acked Flush 0 Done Done Sent false
10617 3460 Acked Read 8 Done Done Sent false
10618 3461 Acked Read 5 Done Done Sent false
10619 3462 Acked Write 3 Done Done Sent false
10620 3463 Acked Write 8 Done Done Sent false
10621 3464 Acked Read 4 Done Done Sent false
10622 3465 Acked Read 5 Done Done Sent false
10623 3466 Acked Read 5 Done Done Sent false
10624 3467 Acked Read 8 Done Done Sent false
10625 3468 Acked Flush 0 Done Done Sent false
10626 3469 Acked Flush 0 Done Done Sent false
10627 3470 Acked Write 3 Done Done Sent false
10628 3471 Acked Read 5 Done Done Sent false
10629 3472 Acked Read 2 Done Done Sent false
10630 3473 Acked Write 6 Done Done Sent false
10631 3474 Acked Flush 0 Done Done Sent false
10632 3475 Acked Write 2 Done Done Sent false
10633 3476 Acked Read 9 Done Done Sent false
10634 3477 Acked Write 9 Done Done Sent false
10635 3478 Acked Read 1 Done Done Sent false
10636 3479 Acked Flush 0 Done Done Sent false
10637 3480 Acked Write 10 Done Done Sent false
10638 3481 Acked Flush 0 Done Done Sent false
10639 3482 Acked Read 8 Done Done Sent false
10640 3483 Acked Read 1 Done Done Sent false
10641 3484 Acked Read 2 Done Done Sent false
10642 3485 Acked Read 9 Done Done Sent false
10643 3486 Acked Write 5 Done Done Sent false
10644 3487 Acked Read 9 Done Done Sent false
10645 3488 Acked Read 1 Done Done Sent false
10646 3489 Acked Read 3 Done Done Sent false
10647 3490 Acked Read 5 Done Done Sent false
10648 3491 Acked Read 9 Done Done Sent false
10649 3492 Acked Write 6 Done Done Sent false
10650 3493 Acked Write 3 Done Done Sent false
10651 3494 Acked Flush 0 Done Done Sent false
10652 3495 Acked Read 5 Done Done Sent false
10653 3496 Acked Read 9 Done Done Sent false
10654 3497 Acked Write 5 Done Done Sent false
10655 3498 Acked Write 5 Done Done Sent false
10656 3499 Acked Read 1 Done Done Sent false
10657 3500 Acked Read 4 Done Done Sent false
10658 3501 Acked Read 4 Done Done Sent false
10659 3502 Acked Flush 0 Done Done Sent false
10660 3503 Acked Write 10 Done Done Sent false
10661 3504 Acked Read 3 Done Done Sent false
10662 3505 Acked Read 9 Done Done Sent false
10663 3506 Acked Read 8 Done Done Sent false
10664 3507 Acked Read 9 Done Done Sent false
10665 3508 Acked Read 7 Done Done Sent false
10666 3509 Acked Read 3 Done Done Sent false
10667 3510 Acked Write 5 Done Done Sent false
10668 3511 Acked Read 4 Done Done Sent false
10669 3512 Acked Write 2 Done Done Sent false
10670 3513 Acked Read 5 Done Done Sent false
10671 3514 Acked Write 7 Done Done Sent false
10672 3515 Acked Write 9 Done Done Sent false
10673 3516 Acked Flush 0 Done Done Sent false
10674 3517 Acked Read 9 Done Done Sent false
10675 3518 Acked Write 8 Done Done Sent false
10676 3519 Acked Write 7 Done Done Sent false
10677 3520 Acked Read 7 Done Done Sent false
10678 3521 Acked Read 6 Done Done Sent false
10679 3522 Acked Read 5 Done Done Sent false
10680 3523 Acked Flush 0 Done Done Sent false
10681 3524 Acked Read 2 Done Done Sent false
10682 3525 Acked Read 8 Done Done Sent false
10683 3526 Acked Write 10 Done Done Sent false
10684 3527 Acked Flush 0 Done Done Sent false
10685 3528 Acked Read 8 Done Done Sent false
10686 3529 Acked Write 9 Done Done Sent false
10687 3530 Acked Write 10 Done Done Sent false
10688 3531 Acked Write 10 Done Done Sent false
10689 3532 Acked Read 1 Done Done Sent false
10690 3533 Acked Read 3 Done Done Sent false
10691 3534 Acked Read 8 Done Done Sent false
10692 3535 Acked Read 9 Done Done Sent false
10693 3536 Acked Write 6 Done Done Sent false
10694 3537 Acked Read 4 Done Done Sent false
10695 3538 Acked Write 1 Done Done Sent false
10696 3539 Acked Read 8 Done Done Sent false
10697 3540 Acked Write 10 Done Done Sent false
10698 3541 Acked Write 2 Done Done Sent false
10699 3542 Acked Write 4 Done Done Sent false
10700 3543 Acked Write 6 Done Done Sent false
10701 3544 Acked Read 10 Done Done Sent false
10702 3545 Acked Read 4 Done Done Sent false
10703 3546 Acked Read 1 Done Done Sent false
10704 3547 Acked Flush 0 Done Done Sent false
10705 3548 Acked Write 1 Done Done Sent false
10706 3549 Acked Write 1 Done Done Sent false
10707 3550 Acked Read 3 Done Done Sent false
10708 3551 Acked Read 6 Done Done Sent false
10709 3552 Acked Flush 0 Done Done Sent false
10710 3553 Acked Write 6 Done Done Sent false
10711 3554 Acked Read 2 Done Done Sent false
10712 3555 Acked Read 7 Done Done Sent false
10713 3556 Acked Read 9 Done Done Sent false
10714 3557 Acked Read 6 Done Done Sent false
10715 3558 Acked Read 5 Done Done Sent false
10716 3559 Acked Read 4 Done Done Sent false
10717 3560 Acked Flush 0 Done Done Sent false
10718 3561 Acked Read 2 Done Done Sent false
10719 3562 Acked Write 10 Done Done Sent false
10720 3563 Acked Write 4 Done Done Sent false
10721 3564 Acked Read 8 Done Done Sent false
10722 3565 Acked Read 5 Done Done Sent false
10723 3566 Acked Read 2 Done Done Sent false
10724 3567 Acked Read 8 Done Done Sent false
10725 3568 Acked Read 4 Done Done Sent false
10726 3569 Acked Write 2 Done Done Sent false
10727 3570 Acked Flush 0 Done Done Sent false
10728 3571 Acked Flush 0 Done Done Sent false
10729 3572 Acked Write 3 Done Done Sent false
10730 3573 Acked Read 6 Done Done Sent false
10731 3574 Acked Write 4 Done Done Sent false
10732 3575 Acked Write 5 Done Done Sent false
10733 3576 Acked Read 2 Done Done Sent false
10734 3577 Acked Write 10 Done Done Sent false
10735 3578 Acked Flush 0 Done Done Sent false
10736 3579 Acked Read 8 Done Done Sent false
10737 3580 Acked Read 4 Done Done Sent false
10738 3581 Acked Write 5 Done Done Sent false
10739 3582 Acked Write 6 Done Done Sent false
10740 3583 Acked Read 2 Done Done Sent false
10741 3584 Acked Read 3 Done Done Sent false
10742 3585 Acked Write 1 Done Done Sent false
10743 3586 Acked Read 2 Done Done Sent false
10744 3587 Acked Read 9 Done Done Sent false
10745 3588 Acked Read 5 Done Done Sent false
10746 3589 Acked Read 2 Done Done Sent false
10747 3590 Acked Write 10 Done Done Sent false
10748 3591 Acked Write 9 Done Done Sent false
10749 3592 Acked Read 5 Done Done Sent false
10750 3593 Acked Write 6 Done Done Sent false
10751 3594 Acked Read 6 Done Done Sent false
10752 3595 Acked Read 6 Done Done Sent false
10753 3596 Acked Write 3 Done Done Sent false
10754 3597 Acked Write 2 Done Done Sent false
10755 3598 Acked Read 8 Done Done Sent false
10756 3599 Acked Read 6 Done Done Sent false
10757 3600 Acked Write 10 Done Done Sent false
10758 3601 Acked Write 3 Done Done Sent false
10759 3602 Acked Read 3 Done Done Sent false
10760 3603 Acked Read 10 Done Done Sent false
10761 3604 Acked Read 9 Done Done Sent false
10762 3605 Acked Write 3 Done Done Sent false
10763 3606 Acked Read 4 Done Done Sent false
10764 3607 Acked Flush 0 Done Done Sent false
10765 3608 Acked Read 5 Done Done Sent false
10766 3609 Acked Write 7 Done Done Sent false
10767 3610 Acked Flush 0 Done Done Sent false
10768 3611 Acked Read 7 Done Done Sent false
10769 3612 Acked Write 9 Done Done Sent false
10770 3613 Acked Read 1 Done Done Sent false
10771 3614 Acked Flush 0 Done Done Sent false
10772 3615 Acked Read 5 Done Done Sent false
10773 3616 Acked Flush 0 Done Done Sent false
10774 3617 Acked Write 7 Done Done Sent false
10775 3618 Acked Read 3 Done Done Sent false
10776 3619 Acked Write 2 Done Done Sent false
10777 3620 Acked Flush 0 Done Done Sent false
10778 3621 Acked Read 2 Done Done Sent false
10779 3622 Acked Read 3 Done Done Sent false
10780 3623 Acked Flush 0 Done Done Sent false
10781 3624 Acked Write 2 Done Done Sent false
10782 3625 Acked Write 7 Done Done Sent false
10783 3626 Acked Read 3 Done Done Sent false
10784 3627 Acked Write 8 Done Done Sent false
10785 3628 Acked Flush 0 Done Done Sent false
10786 3629 Acked Read 1 Done Done Sent false
10787 3630 Acked Write 2 Done Done Sent false
10788 3631 Acked Read 2 Done Done Sent false
10789 3632 Acked Write 7 Done Done Sent false
10790 3633 Acked Read 9 Done Done Sent false
10791 3634 Acked Read 2 Done Done Sent false
10792 3635 Acked Write 7 Done Done Sent false
10793 3636 Acked Read 1 Done Done Sent false
10794 3637 Acked Read 5 Done Done Sent false
10795 3638 Acked Write 7 Done Done Sent false
10796 3639 Acked Flush 0 Done Done Sent false
10797 3640 Acked Read 6 Done Done Sent false
10798 3641 Acked Flush 0 Done Done Sent false
10799 3642 Acked Read 8 Done Done Sent false
10800 3643 Acked Read 7 Done Done Sent false
10801 3644 Acked Write 2 Done Done Sent false
10802 3645 Acked Write 6 Done Done Sent false
10803 3646 Acked Write 8 Done Done Sent false
10804 3647 Acked Write 6 Done Done Sent false
10805 3648 Acked Write 7 Done Done Sent false
10806 3649 Acked Write 10 Done Done Sent false
10807 3650 Acked Read 2 Done Done Sent false
10808 3651 Acked Read 9 Done Done Sent false
10809 3652 Acked Write 2 Done Done Sent false
10810 3653 Acked Read 2 Done Done Sent false
10811 3654 Acked Write 1 Done Done Sent false
10812 3655 Acked Write 6 Done Done Sent false
10813 3656 Acked Read 4 Done Done Sent false
10814 3657 Acked Write 3 Done Done Sent false
10815 3658 Acked Read 7 Done Done Sent false
10816 3659 Acked Read 4 Done Done Sent false
10817 3660 Acked Write 3 Done Done Sent false
10818 3661 Acked Write 8 Done Done Sent false
10819 3662 Acked Write 5 Done Done Sent false
10820 3663 Acked Write 1 Done Done Sent false
10821 3664 Acked Read 7 Done Done Sent false
10822 3665 Acked Read 6 Done Done Sent false
10823 3666 Acked Read 5 Done Done Sent false
10824 3667 Acked Write 1 Done Done Sent false
10825 3668 Acked Write 6 Done Done Sent false
10826 3669 Acked Write 4 Done Done Sent false
10827 3670 Acked Read 6 Done Done Sent false
10828 3671 Acked Write 4 Done Done Sent false
10829 3672 Acked Read 1 Done Done Sent false
10830 3673 Acked Write 5 Done Done Sent false
10831 3674 Acked Read 7 Done Done Sent false
10832 3675 Acked Read 2 Done Done Sent false
10833 3676 Acked Read 3 Done Done Sent false
10834 3677 Acked Flush 0 Done Done Sent false
10835 3678 Acked Read 3 Done Done Sent false
10836 3679 Acked Read 10 Done Done Sent false
10837 3680 Acked Flush 0 Done Done Sent false
10838 3681 Acked Write 10 Done Done Sent false
10839 3682 Acked Flush 0 Done Done Sent false
10840 3683 Acked Write 7 Done Done Sent false
10841 3684 Acked Write 9 Done Done Sent false
10842 3685 Acked Read 9 Done Done Sent false
10843 3686 Acked Write 8 Done Done Sent false
10844 3687 Acked Read 6 Done Done Sent false
10845 3688 Acked Read 3 Done Done Sent false
10846 3689 Acked Write 9 Done Done Sent false
10847 3690 Acked Flush 0 Done Done Sent false
10848 3691 Acked Write 5 Done Done Sent false
10849 3692 Acked Flush 0 Done Done Sent false
10850 3693 Acked Read 8 Done Done Sent false
10851 3694 Acked Read 4 Done Done Sent false
10852 3695 Acked Read 3 Done Done Sent false
10853 3696 Acked Read 9 Done Done Sent false
10854 3697 Acked Flush 0 Done Done Sent false
10855 3698 Acked Write 1 Done Done Sent false
10856 3699 Acked Read 4 Done Done Sent false
10857 3700 Acked Write 9 Done Done Sent false
10858 3701 Acked Read 9 Done Done Sent false
10859 3702 Acked Read 5 Done Done Sent false
10860 3703 Acked Write 8 Done Done Sent false
10861 3704 Acked Write 10 Done Done Sent false
10862 3705 Acked Write 1 Done Done Sent false
10863 3706 Acked Write 6 Done Done Sent false
10864 3707 Acked Write 1 Done Done Sent false
10865 3708 Acked Write 8 Done Done Sent false
10866 3709 Acked Write 1 Done Done Sent false
10867 3710 Acked Write 8 Done Done Sent false
10868 3711 Acked Read 1 Done Done Sent false
10869 3712 Acked Write 8 Done Done Sent false
10870 3713 Acked Read 8 Done Done Sent false
10871 3714 Acked Write 7 Done Done Sent false
10872 3715 Acked Flush 0 Done Done Sent false
10873 STATES DS:0 DS:1 DS:2 TOTAL
10874 Sent 0 0 301 301
10875 Done 301 301 0 602
10876 Skipped 0 0 0 0
10877 Error 0 0 0 0
10878 Last Flush: 3715 3715 3414
10879 Downstairs last five completed: 3414 3413 3412 3411 3410
10880 Upstairs last five completed: 3715 3714 3713 3712 3711
10881 CLIENT: Up:0 ds:301 act:2
108822025-01-11T06:07:42.471ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108832025-01-11T06:07:42.471ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61641) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108842025-01-11T06:07:42.473ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108852025-01-11T06:07:42.473ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108862025-01-11T06:07:42.473ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108872025-01-11T06:07:42.473ZINFOcrucible: send last flush ID to this DS: 3414 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108882025-01-11T06:07:42.474ZINFOcrucible: Replied this last flush ID: 3414 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108892025-01-11T06:07:42.474ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108902025-01-11T06:07:42.474ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108912025-01-11T06:07:42.474ZINFOcrucible: [2] client re-new 301 jobs since flush 3414 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
108922025-01-11T06:07:42.474ZINFOcrucible: [2] Replayed 301 jobs since flush: 3414 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
10893 ----------------------------------------------------------------
10894 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10895 Downstairs last five completed: 3715 3714 3713 3712 3711
10896 Upstairs last five completed: 3715 3714 3713 3712 3711
10897 CLIENT: Up:0 ds:0 act:3
10898 Replay: All jobs finished
10899 Main task has work to do, go find it
10900 got dsc Stop(1)
10901 stop 1
10902 [8820] Got stop action so:false kr:false
10903 Jan 11 06:07:44.715 INFO request completed, latency_us: 360, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 635d93cb-93d5-4703-ba5f-e02db6672dd2, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
10904 Jan 11 06:07:44.716 INFO request completed, latency_us: 157, response_code: 200, uri{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:07:44.716283545Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
10905 : /state/cid/1, method: GET, req_id: dec822ff-d566-45c6-9801-9b5216e6745a, remote_addr: 127.0.0.1:59925{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:07:44.716364954Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
10906 , {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:07:44.716401739Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
10907 local_addr: 127.0.0.1:9998
109082025-01-11T06:07:44.716ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
109092025-01-11T06:07:44.716ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
109102025-01-11T06:07:44.716ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
10911 [8820] Exited with: ExitStatus(unix_wait_status(9))
10912 [8820][1] reports Exit
10913 Jan 11 06:07:48.719 INFO request completed, latency_us: 225, response_code: 200, uri: /state/cid/1, method: GET, req_id: 55b0ca38-5fb8-4107-844b-b9fbec8d901e, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
10914 001/300 Read block 1596 len 4096
10915 002/300 Read block 1101 len 24576
10916 003/300 Write block 386 len 40960 data: 4 4 4 4 4 4 4 4 3 3
10917 004/300 Write block 2230 len 12288 data: 5 5 5
10918 005/300 Read block 1234 len 32768
10919 006/300 Read block 3223 len 8192
10920 007/300 Write block 3387 len 12288 data: 4 4 3
10921 008/300 Write block 3712 len 24576 data: 3 3 2 2 3 3
10922 009/300 Write block 3569 len 16384 data: 3 3 2 3
10923 010/300 Read block 1959 len 40960
10924 011/300 Read block 4298 len 20480
10925 012/300 Read block 1130 len 4096
10926 013/300 Write block 4546 len 8192 data: 2 2
10927 014/300 Read block 3410 len 16384
10928 015/300 Read block 1353 len 28672
10929 016/300 Read block 2061 len 20480
10930 017/300 Flush
10931 018/300 Write block 2241 len 24576 data: 2 2 2 2 3 3
10932 019/300 Read block 846 len 16384
10933 020/300 Write block 3188 len 32768 data: 3 3 3 2 2 2 2 2
10934 021/300 Read block 2127 len 24576
10935 022/300 Read block 848 len 12288
10936 023/300 Write block 1573 len 12288 data: 3 3 3
10937 024/300 Read block 292 len 16384
10938 025/300 Read block 2231 len 8192
10939 026/300 Write block 39 len 12288 data: 5 6 6
10940 027/300 Write block 3147 len 32768 data: 4 4 4 4 4 3 4 4
10941 028/300 Write block 853 len 8192 data: 3 3
10942 029/300 Flush
10943 030/300 Read block 3641 len 40960
10944 031/300 Read block 4395 len 16384
10945 032/300 Write block 1069 len 16384 data: 2 2 2 2
10946 033/300 Read block 3898 len 4096
10947 034/300 Flush
10948 035/300 Read block 2662 len 36864
10949 036/300 Write block 842 len 28672 data: 2 2 2 2 3 3 4
10950 037/300 Flush
10951 038/300 Write block 2076 len 40960 data: 3 4 4 4 3 3 4 4 4 3
10952 039/300 Write block 1734 len 8192 data: 3 3
10953 040/300 Flush
10954 041/300 Read block 239 len 20480
10955 042/300 Read block 306 len 8192
10956 043/300 Read block 1845 len 32768
10957 044/300 Read block 4059 len 24576
10958 045/300 Write block 3089 len 20480 data: 6 6 6 6 4
10959 046/300 Write block 3429 len 24576 data: 2 3 3 3 3 3
10960 047/300 Read block 1872 len 36864
10961 048/300 Write block 1477 len 20480 data: 3 3 3 4 3
10962 049/300 Read block 2947 len 36864
10963 050/300 Read block 902 len 24576
10964 051/300 Read block 661 len 40960
10965 052/300 Write block 2151 len 20480 data: 3 3 3 3 3
10966 053/300 Flush
10967 054/300 Write block 1615 len 32768 data: 2 2 2 2 2 3 4 3
10968 055/300 Write block 2941 len 8192 data: 5 5
10969 056/300 Read block 1002 len 12288
10970 057/300 Write block 4138 len 36864 data: 5 4 4 5 4 4 5 4 5
10971 058/300 Read block 3432 len 32768
10972 059/300 Flush
10973 060/300 Write block 4746 len 12288 data: 3 3 3
10974 061/300 Write block 1001 len 32768 data: 5 5 4 5 5 5 6 6
10975 062/300 Read block 3983 len 24576
10976 063/300 Write block 129 len 12288 data: 4 5 5
10977 064/300 Write block 2978 len 20480 data: 3 3 3 2 2
10978 065/300 Read block 4331 len 28672
10979 066/300 Write block 1298 len 28672 data: 3 3 2 2 3 3 3
10980 067/300 Read block 3363 len 8192
10981 068/300 Read block 969 len 12288
10982 069/300 Write block 3682 len 24576 data: 3 3 2 2 2 2
10983 070/300 Write block 4357 len 40960 data: 2 3 4 4 3 3 2 3 3 3
10984 071/300 Write block 432 len 28672 data: 3 2 2 2 2 2 2
10985 072/300 Write block 631 len 12288 data: 2 3 3
10986 073/300 Flush
10987 074/300 Read block 3858 len 40960
10988 075/300 Read block 3355 len 20480
10989 076/300 Read block 3068 len 16384
10990 077/300 Write block 1991 len 40960 data: 3 3 5 4 3 3 3 2 2 3
10991 078/300 Write block 1095 len 28672 data: 4 3 4 4 3 3 3
10992 079/300 Read block 4107 len 40960
10993 080/300 Read block 3086 len 40960
10994 081/300 Read block 3907 len 16384
10995 082/300 Flush
10996 083/300 Read block 2608 len 36864
10997 084/300 Write block 2093 len 4096 data: 3
10998 085/300 Read block 3453 len 36864
10999 086/300 Read block 990 len 12288
11000 087/300 Flush
11001 088/300 Read block 1292 len 24576
11002 089/300 Read block 4575 len 8192
11003 090/300 Read block 438 len 24576
11004 091/300 Write block 2695 len 28672 data: 2 2 2 2 2 2 4
11005 092/300 Flush
11006 093/300 Flush
11007 094/300 Read block 107 len 12288
11008 095/300 Read block 2465 len 20480
11009 096/300 Read block 2699 len 4096
11010 097/300 Write block 614 len 20480 data: 2 2 2 2 3
11011 098/300 Read block 4418 len 24576
11012 099/300 Read block 2883 len 12288
11013 100/300 Write block 887 len 12288 data: 5 5 4
11014 101/300 Write block 474 len 36864 data: 3 2 2 2 2 2 3 3 3
11015 102/300 Flush
11016 103/300 Read block 3452 len 28672
11017 104/300 Read block 1791 len 4096
11018 105/300 Read block 2747 len 4096
11019 106/300 Write block 1716 len 4096 data: 3
11020 107/300 Write block 1774 len 32768 data: 2 2 2 2 2 2 4 4
11021 108/300 Read block 1530 len 20480
11022 109/300 Read block 4857 len 8192
11023 110/300 Write block 2560 len 12288 data: 3 3 5
11024 111/300 Write block 4649 len 20480 data: 3 3 4 4 3
11025 112/300 Read block 4472 len 12288
11026 113/300 Read block 308 len 24576
11027 114/300 Write block 641 len 4096 data: 2
11028 115/300 Write block 2547 len 12288 data: 5 5 5
11029 116/300 Write block 1511 len 8192 data: 3 3
11030 117/300 Flush
11031 118/300 Read block 4193 len 16384
11032 119/300 Read block 3720 len 4096
11033 120/300 Write block 4649 len 12288 data: 4 4 5
11034 121/300 Read block 4158 len 28672
11035 122/300 Read block 3813 len 28672
11036 123/300 Read block 2638 len 20480
11037 124/300 Write block 839 len 40960 data: 2 3 2 3 3 3 3 4 4 5
11038 125/300 Read block 4477 len 32768
11039 126/300 Read block 140 len 36864
11040 127/300 Write block 3782 len 4096 data: 5
11041 128/300 Read block 3222 len 40960
11042 129/300 Read block 2366 len 16384
11043 130/300 Flush
11044 131/300 Read block 3816 len 36864
11045 132/300 Write block 3303 len 40960 data: 2 2 3 3 3 2 2 2 2 2
11046 133/300 Write block 3578 len 36864 data: 4 4 4 4 4 4 4 5 5
11047 134/300 Write block 3973 len 16384 data: 4 3 3 3
11048 135/300 Read block 3119 len 4096
11049 136/300 Read block 1103 len 28672
11050 137/300 Flush
11051 138/300 Flush
11052 139/300 Read block 3300 len 4096
11053 140/300 Flush
11054 141/300 Read block 358 len 4096
11055 142/300 Write block 3748 len 8192 data: 3 3
11056 143/300 Read block 4163 len 36864
11057 144/300 Read block 1299 len 8192
11058 145/300 Read block 3998 len 28672
11059 146/300 Write block 217 len 4096 data: 3
11060 147/300 Read block 4560 len 16384
11061 148/300 Read block 3762 len 36864
11062 149/300 Read block 786 len 24576
11063 150/300 Read block 2659 len 16384
11064 151/300 Read block 1743 len 28672
11065 152/300 Write block 2081 len 12288 data: 4 5 5
11066 153/300 Read block 93 len 16384
11067 154/300 Flush
11068 155/300 Flush
11069 156/300 Write block 443 len 28672 data: 4 3 3 3 2 2 2
11070 157/300 Write block 3706 len 8192 data: 4 4
11071 158/300 Write block 3621 len 12288 data: 3 3 3
11072 159/300 Read block 4871 len 8192
11073 160/300 Read block 2139 len 20480
11074 161/300 Read block 2981 len 8192
11075 162/300 Read block 4493 len 24576
11076 163/300 Read block 4971 len 32768
11077 164/300 Write block 2809 len 28672 data: 4 3 2 2 3 3 4
11078 165/300 Read block 2620 len 8192
11079 166/300 Read block 391 len 40960
11080 167/300 Read block 4249 len 12288
11081 168/300 Write block 4825 len 40960 data: 2 3 4 4 4 4 4 4 4 5
11082 169/300 Write block 835 len 16384 data: 3 3 3 3
11083 170/300 Read block 2556 len 20480
11084 171/300 Read block 4489 len 12288
11085 172/300 Read block 3655 len 32768
11086 173/300 Write block 3297 len 24576 data: 2 2 2 2 4 2
11087 174/300 Read block 4511 len 8192
11088 175/300 Read block 3766 len 28672
11089 176/300 Read block 3995 len 20480
11090 177/300 Read block 1577 len 4096
11091 178/300 Write block 722 len 28672 data: 5 6 6 4 4 4 4
11092 179/300 Write block 1864 len 8192 data: 4 3
11093 180/300 Write block 225 len 12288 data: 2 2 4
11094 181/300 Write block 3464 len 16384 data: 3 2 2 2
11095 182/300 Write block 3748 len 20480 data: 4 4 3 4 4
11096 183/300 Write block 648 len 12288 data: 3 3 4
11097 184/300 Write block 2107 len 8192 data: 2 2
11098 185/300 Read block 1026 len 32768
11099 186/300 Read block 2755 len 32768
11100 187/300 Read block 4699 len 4096
11101 188/300 Read block 2871 len 28672
11102 189/300 Write block 3805 len 16384 data: 2 2 2 2
11103 190/300 Write block 4318 len 32768 data: 3 4 5 5 4 3 3 3
11104 191/300 Read block 2263 len 8192
11105 192/300 Write block 2303 len 24576 data: 3 3 3 2 2 2
11106 193/300 Write block 2282 len 40960 data: 4 4 5 5 5 5 5 4 4 5
11107 194/300 Read block 4675 len 28672
11108 195/300 Write block 4866 len 8192 data: 3 3
11109 196/300 Read block 4873 len 32768
11110 197/300 Write block 4468 len 20480 data: 8 6 5 4 3
11111 198/300 Write block 1719 len 28672 data: 2 3 3 4 4 5 5
11112 199/300 Read block 1908 len 28672
11113 200/300 Read block 498 len 24576
11114 201/300 Read block 3483 len 20480
11115 202/300 Read block 1954 len 4096
11116 203/300 Read block 404 len 24576
11117 204/300 Flush
11118 205/300 Write block 532 len 8192 data: 4 5
11119 206/300 Write block 3993 len 36864 data: 3 4 4 3 3 3 3 3 3
11120 207/300 Read block 105 len 24576
11121 208/300 Write block 393 len 36864 data: 5 4 4 5 5 4 4 4 4
11122 209/300 Read block 4581 len 4096
11123 210/300 Read block 3202 len 32768
11124 211/300 Write block 2597 len 36864 data: 3 3 3 3 3 2 2 2 2
11125 212/300 Write block 3583 len 12288 data: 5 5 6
11126 213/300 Write block 3013 len 4096 data: 2
11127 214/300 Write block 4580 len 24576 data: 3 3 3 3 4 5
11128 215/300 Read block 961 len 12288
11129 216/300 Write block 2884 len 28672 data: 2 2 2 2 2 2 2
11130 217/300 Write block 958 len 12288 data: 2 3 3
11131 218/300 Write block 2140 len 20480 data: 2 2 2 2 2
11132 219/300 Read block 4459 len 12288
11133 220/300 Read block 2156 len 8192
11134 221/300 Write block 1797 len 12288 data: 2 2 2
11135 222/300 Read block 1944 len 24576
11136 223/300 Read block 4894 len 4096
11137 224/300 Read block 239 len 40960
11138 225/300 Read block 4025 len 20480
11139 226/300 Write block 4369 len 40960 data: 3 2 2 2 2 2 3 2 2 2
11140 227/300 Read block 1573 len 12288
11141 228/300 Write block 1142 len 20480 data: 2 2 2 3 4
11142 229/300 Read block 1102 len 40960
11143 230/300 Read block 1131 len 12288
11144 231/300 Read block 2281 len 24576
11145 232/300 Read block 1819 len 28672
11146 233/300 Read block 3239 len 36864
11147 234/300 Read block 2128 len 36864
11148 235/300 Write block 2804 len 8192 data: 5 4
11149 236/300 Read block 3156 len 8192
11150 237/300 Write block 4127 len 32768 data: 4 5 4 5 4 4 3 4
11151 238/300 Read block 1347 len 24576
11152 239/300 Read block 4522 len 36864
11153 240/300 Write block 2859 len 40960 data: 5 5 5 7 7 7 6 6 5 4
11154 241/300 Read block 2635 len 32768
11155 242/300 Write block 2260 len 24576 data: 3 2 2 2 2 2
11156 243/300 Read block 286 len 16384
11157 244/300 Read block 4405 len 4096
11158 245/300 Read block 2526 len 32768
11159 246/300 Read block 3957 len 40960
11160 247/300 Read block 4258 len 8192
11161 248/300 Read block 1395 len 4096
11162 249/300 Flush
11163 250/300 Read block 901 len 40960
11164 251/300 Write block 2533 len 16384 data: 2 2 4 4
11165 252/300 Read block 4484 len 24576
11166 253/300 Read block 2030 len 20480
11167 254/300 Write block 2055 len 8192 data: 6 6
11168 255/300 Write block 3014 len 28672 data: 3 2 2 2 2 2 2
11169 256/300 Read block 294 len 32768
11170 257/300 Read block 1019 len 32768
11171 258/300 Write block 1565 len 40960 data: 4 5 5 5 4 4 4 4 4 4
11172 259/300 Write block 3524 len 32768 data: 4 4 4 4 4 4 4 4
11173 260/300 Read block 154 len 40960
11174 261/300 Write block 367 len 20480 data: 4 3 3 2 2
11175 262/300 Read block 1316 len 40960
11176 263/300 Read block 2193 len 28672
11177 264/300 Write block 4043 len 24576 data: 4 4 4 4 3 3
11178 265/300 Read block 1364 len 20480
11179 266/300 Read block 410 len 8192
11180 267/300 Read block 2764 len 16384
11181 268/300 Read block 1882 len 16384
11182 269/300 Read block 226 len 24576
11183 270/300 Read block 381 len 36864
11184 271/300 Read block 566 len 24576
11185 272/300 Read block 2793 len 28672
11186 273/300 Flush
11187 274/300 Write block 2087 len 24576 data: 3 3 2 2 2 3
11188 275/300 Read block 1590 len 4096
11189 276/300 Read block 755 len 32768
11190 277/300 Write block 1311 len 20480 data: 5 4 4 4 2
11191 278/300 Read block 2807 len 16384
11192 279/300 Read block 3665 len 32768
11193 280/300 Write block 2868 len 16384 data: 5 5 5 5
11194 281/300 Write block 3035 len 36864 data: 2 2 4 4 4 4 4 3 2
11195 282/300 Read block 4318 len 4096
11196 283/300 Read block 4213 len 36864
11197 284/300 Write block 4841 len 20480 data: 3 2 2 2 2
11198 285/300 Flush
11199 286/300 Read block 608 len 40960
11200 287/300 Write block 2211 len 24576 data: 2 2 3 3 4 5
11201 288/300 Write block 518 len 12288 data: 5 4 4
11202 289/300 Read block 1020 len 40960
11203 290/300 Write block 3672 len 40960 data: 5 4 4 4 4 3 2 2 3 3
11204 291/300 Write block 3303 len 4096 data: 3
11205 292/300 Write block 2079 len 28672 data: 5 4 5 6 6 5 4
11206 293/300 Read block 2563 len 8192
11207 294/300 Read block 3240 len 12288
11208 295/300 Read block 3938 len 16384
11209 296/300 Read block 3783 len 32768
11210 297/300 Write block 3036 len 28672 data: 3 5 5 5 5 5 4
11211 298/300 Flush
11212 299/300 Write block 1136 len 28672 data: 3 3 3 3 3 3 3
11213 300/300 Read block 3730 len 16384
11214 301/300 Read block 2541 len 32768
11215 Main task has work to do, go find it
11216 got dsc Start(1)
11217 start 1
11218 Jan 11 06:07:48.967 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/1, method: POST, req_id: 72d79fa4-a76d-46a7-ae79-e86c3b28734f, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
11219 Replay: started 1, returned:Ok(())
11220 ----------------------------------------------------------------
11221 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11222 States: Active Offline Active
11223 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11224 3716 Acked Read 1 Done Sent Done false
11225 3717 Acked Read 6 Done Sent Done false
11226 3718 Acked Write 10 Done Sent Done false
11227 3719 Acked Write 3 Done Sent Done false
11228 3720 Acked Read 8 Done Sent Done false
11229 3721 Acked Read 2 Done Sent Done false
11230 3722 Acked Write 3 Done Sent Done false
11231 3723 Acked Write 6 Done Sent Done false
11232 3724 Acked Write 4 Done Sent Done false
11233 3725 Acked Read 10 Done Sent Done false
11234 3726 Acked Read 5 Done Sent Done false
11235 3727 Acked Read 1 Done Sent Done false
11236 3728 Acked Write 2 Done Sent Done false
11237 3729 Acked Read 4 Done Sent Done false
11238 3730 Acked Read 7 Done Sent Done false
11239 3731 Acked Read 5 Done Sent Done false
11240 3732 Acked Flush 0 Done Sent Done false
11241 3733 Acked Write 6 Done Sent Done false
11242 3734 Acked Read 4 Done Sent Done false
11243 3735 Acked Write 8 Done Sent Done false
11244 3736 Acked Read 6 Done Sent Done false
11245 3737 Acked Read 3 Done Sent Done false
11246 3738 Acked Write 3 Done Sent Done false
11247 3739 Acked Read 4 Done Sent Done false
11248 3740 Acked Read 2 Done Sent Done false
11249 3741 Acked Write 3 Done Sent Done false
11250 3742 Acked Write 8 Done Sent Done false
11251 3743 Acked Write 2 Done Sent Done false
11252 3744 Acked Flush 0 Done Sent Done false
11253 3745 Acked Read 10 Done Sent Done false
11254 3746 Acked Read 4 Done Sent Done false
11255 3747 Acked Write 4 Done Sent Done false
11256 3748 Acked Read 1 Done Sent Done false
11257 3749 Acked Flush 0 Done Sent Done false
11258 3750 Acked Read 9 Done Sent Done false
11259 3751 Acked Write 7 Done Sent Done false
11260 3752 Acked Flush 0 Done Sent Done false
11261 3753 Acked Write 10 Done Sent Done false
11262 3754 Acked Write 2 Done Sent Done false
11263 3755 Acked Flush 0 Done Sent Done false
11264 3756 Acked Read 5 Done Sent Done false
11265 3757 Acked Read 2 Done Sent Done false
11266 3758 Acked Read 8 Done Sent Done false
11267 3759 Acked Read 6 Done Sent Done false
11268 3760 Acked Write 5 Done Sent Done false
11269 3761 Acked Write 6 Done Sent Done false
11270 3762 Acked Read 9 Done Sent Done false
11271 3763 Acked Write 5 Done Sent Done false
11272 3764 Acked Read 9 Done Sent Done false
11273 3765 Acked Read 6 Done Sent Done false
11274 3766 Acked Read 10 Done Sent Done false
11275 3767 Acked Write 5 Done Sent Done false
11276 3768 Acked Flush 0 Done Sent Done false
11277 3769 Acked Write 8 Done Sent Done false
11278 3770 Acked Write 2 Done Sent Done false
11279 3771 Acked Read 3 Done Sent Done false
11280 3772 Acked Write 9 Done Sent Done false
11281 3773 Acked Read 8 Done Sent Done false
11282 3774 Acked Flush 0 Done Sent Done false
11283 3775 Acked Write 3 Done Sent Done false
11284 3776 Acked Write 8 Done Sent Done false
11285 3777 Acked Read 6 Done Sent Done false
11286 3778 Acked Write 3 Done Sent Done false
11287 3779 Acked Write 5 Done Sent Done false
11288 3780 Acked Read 7 Done Sent Done false
11289 3781 Acked Write 7 Done Sent Done false
11290 3782 Acked Read 2 Done Sent Done false
11291 3783 Acked Read 3 Done Sent Done false
11292 3784 Acked Write 6 Done Sent Done false
11293 3785 Acked Write 10 Done Sent Done false
11294 3786 Acked Write 7 Done Sent Done false
11295 3787 Acked Write 3 Done Sent Done false
11296 3788 Acked Flush 0 Done Sent Done false
11297 3789 Acked Read 10 Done Sent Done false
11298 3790 Acked Read 5 Done Sent Done false
11299 3791 Acked Read 4 Done Sent Done false
11300 3792 Acked Write 10 Done Sent Done false
11301 3793 Acked Write 7 Done Sent Done false
11302 3794 Acked Read 10 Done Sent Done false
11303 3795 Acked Read 10 Done Sent Done false
11304 3796 Acked Read 4 Done Sent Done false
11305 3797 Acked Flush 0 Done Sent Done false
11306 3798 Acked Read 9 Done Sent Done false
11307 3799 Acked Write 1 Done Sent Done false
11308 3800 Acked Read 9 Done Sent Done false
11309 3801 Acked Read 3 Done Sent Done false
11310 3802 Acked Flush 0 Done Sent Done false
11311 3803 Acked Read 6 Done Sent Done false
11312 3804 Acked Read 2 Done Sent Done false
11313 3805 Acked Read 6 Done Sent Done false
11314 3806 Acked Write 7 Done Sent Done false
11315 3807 Acked Flush 0 Done Sent Done false
11316 3808 Acked Flush 0 Done Sent Done false
11317 3809 Acked Read 3 Done Sent Done false
11318 3810 Acked Read 5 Done Sent Done false
11319 3811 Acked Read 1 Done Sent Done false
11320 3812 Acked Write 5 Done Sent Done false
11321 3813 Acked Read 6 Done Sent Done false
11322 3814 Acked Read 3 Done Sent Done false
11323 3815 Acked Write 3 Done Sent Done false
11324 3816 Acked Write 9 Done Sent Done false
11325 3817 Acked Flush 0 Done Sent Done false
11326 3818 Acked Read 7 Done Sent Done false
11327 3819 Acked Read 1 Done Sent Done false
11328 3820 Acked Read 1 Done Sent Done false
11329 3821 Acked Write 1 Done Sent Done false
11330 3822 Acked Write 8 Done Sent Done false
11331 3823 Acked Read 5 Done Sent Done false
11332 3824 Acked Read 2 Done Sent Done false
11333 3825 Acked Write 3 Done Sent Done false
11334 3826 Acked Write 5 Done Sent Done false
11335 3827 Acked Read 3 Done Sent Done false
11336 3828 Acked Read 6 Done Sent Done false
11337 3829 Acked Write 1 Done Sent Done false
11338 3830 Acked Write 3 Done Sent Done false
11339 3831 Acked Write 2 Done Sent Done false
11340 3832 Acked Flush 0 Done Sent Done false
11341 3833 Acked Read 4 Done Sent Done false
11342 3834 Acked Read 1 Done Sent Done false
11343 3835 Acked Write 3 Done Sent Done false
11344 3836 Acked Read 7 Done Sent Done false
11345 3837 Acked Read 7 Done Sent Done false
11346 3838 Acked Read 5 Done Sent Done false
11347 3839 Acked Write 10 Done Sent Done false
11348 3840 Acked Read 8 Done Sent Done false
11349 3841 Acked Read 9 Done Sent Done false
11350 3842 Acked Write 1 Done Sent Done false
11351 3843 Acked Read 10 Done Sent Done false
11352 3844 Acked Read 4 Done Sent Done false
11353 3845 Acked Flush 0 Done Sent Done false
11354 3846 Acked Read 9 Done Sent Done false
11355 3847 Acked Write 10 Done Sent Done false
11356 3848 Acked Write 9 Done Sent Done false
11357 3849 Acked Write 4 Done Sent Done false
11358 3850 Acked Read 1 Done Sent Done false
11359 3851 Acked Read 7 Done Sent Done false
11360 3852 Acked Flush 0 Done Sent Done false
11361 3853 Acked Flush 0 Done Sent Done false
11362 3854 Acked Read 1 Done Sent Done false
11363 3855 Acked Flush 0 Done Sent Done false
11364 3856 Acked Read 1 Done Sent Done false
11365 3857 Acked Write 2 Done Sent Done false
11366 3858 Acked Read 9 Done Sent Done false
11367 3859 Acked Read 2 Done Sent Done false
11368 3860 Acked Read 7 Done Sent Done false
11369 3861 Acked Write 1 Done Sent Done false
11370 3862 Acked Read 4 Done Sent Done false
11371 3863 Acked Read 9 Done Sent Done false
11372 3864 Acked Read 6 Done Sent Done false
11373 3865 Acked Read 4 Done Sent Done false
11374 3866 Acked Read 7 Done Sent Done false
11375 3867 Acked Write 3 Done Sent Done false
11376 3868 Acked Read 4 Done Sent Done false
11377 3869 Acked Flush 0 Done Sent Done false
11378 3870 Acked Flush 0 Done Sent Done false
11379 3871 Acked Write 7 Done Sent Done false
11380 3872 Acked Write 2 Done Sent Done false
11381 3873 Acked Write 3 Done Sent Done false
11382 3874 Acked Read 2 Done Sent Done false
11383 3875 Acked Read 5 Done Sent Done false
11384 3876 Acked Read 2 Done Sent Done false
11385 3877 Acked Read 6 Done Sent Done false
11386 3878 Acked Read 8 Done Sent Done false
11387 3879 Acked Write 7 Done Sent Done false
11388 3880 Acked Read 2 Done Sent Done false
11389 3881 Acked Read 10 Done Sent Done false
11390 3882 Acked Read 3 Done Sent Done false
11391 3883 Acked Write 10 Done Sent Done false
11392 3884 Acked Write 4 Done Sent Done false
11393 3885 Acked Read 5 Done Sent Done false
11394 3886 Acked Read 3 Done Sent Done false
11395 3887 Acked Read 8 Done Sent Done false
11396 3888 Acked Write 6 Done Sent Done false
11397 3889 Acked Read 2 Done Sent Done false
11398 3890 Acked Read 7 Done Sent Done false
11399 3891 Acked Read 5 Done Sent Done false
11400 3892 Acked Read 1 Done Sent Done false
11401 3893 Acked Write 7 Done Sent Done false
11402 3894 Acked Write 2 Done Sent Done false
11403 3895 Acked Write 3 Done Sent Done false
11404 3896 Acked Write 4 Done Sent Done false
11405 3897 Acked Write 5 Done Sent Done false
11406 3898 Acked Write 3 Done Sent Done false
11407 3899 Acked Write 2 Done Sent Done false
11408 3900 Acked Read 8 Done Sent Done false
11409 3901 Acked Read 8 Done Sent Done false
11410 3902 Acked Read 1 Done Sent Done false
11411 3903 Acked Read 7 Done Sent Done false
11412 3904 Acked Write 4 Done Sent Done false
11413 3905 Acked Write 8 Done Sent Done false
11414 3906 Acked Read 2 Done Sent Done false
11415 3907 Acked Write 6 Done Sent Done false
11416 3908 Acked Write 10 Done Sent Done false
11417 3909 Acked Read 7 Done Sent Done false
11418 3910 Acked Write 2 Done Sent Done false
11419 3911 Acked Read 8 Done Sent Done false
11420 3912 Acked Write 5 Done Sent Done false
11421 3913 Acked Write 7 Done Sent Done false
11422 3914 Acked Read 7 Done Sent Done false
11423 3915 Acked Read 6 Done Sent Done false
11424 3916 Acked Read 5 Done Sent Done false
11425 3917 Acked Read 1 Done Sent Done false
11426 3918 Acked Read 6 Done Sent Done false
11427 3919 Acked Flush 0 Done Sent Done false
11428 3920 Acked Write 2 Done Sent Done false
11429 3921 Acked Write 9 Done Sent Done false
11430 3922 Acked Read 6 Done Sent Done false
11431 3923 Acked Write 9 Done Sent Done false
11432 3924 Acked Read 1 Done Sent Done false
11433 3925 Acked Read 8 Done Sent Done false
11434 3926 Acked Write 9 Done Sent Done false
11435 3927 Acked Write 3 Done Sent Done false
11436 3928 Acked Write 1 Done Sent Done false
11437 3929 Acked Write 6 Done Sent Done false
11438 3930 Acked Read 3 Done Sent Done false
11439 3931 Acked Write 7 Done Sent Done false
11440 3932 Acked Write 3 Done Sent Done false
11441 3933 Acked Write 5 Done Sent Done false
11442 3934 Acked Read 3 Done Sent Done false
11443 3935 Acked Read 2 Done Sent Done false
11444 3936 Acked Write 3 Done Sent Done false
11445 3937 Acked Read 6 Done Sent Done false
11446 3938 Acked Read 1 Done Sent Done false
11447 3939 Acked Read 10 Done Sent Done false
11448 3940 Acked Read 5 Done Sent Done false
11449 3941 Acked Write 10 Done Sent Done false
11450 3942 Acked Read 3 Done Sent Done false
11451 3943 Acked Write 5 Done Sent Done false
11452 3944 Acked Read 10 Done Sent Done false
11453 3945 Acked Read 3 Done Sent Done false
11454 3946 Acked Read 6 Done Sent Done false
11455 3947 Acked Read 7 Done Sent Done false
11456 3948 Acked Read 9 Done Sent Done false
11457 3949 Acked Read 9 Done Sent Done false
11458 3950 Acked Write 2 Done Sent Done false
11459 3951 Acked Read 2 Done Sent Done false
11460 3952 Acked Write 8 Done Sent Done false
11461 3953 Acked Read 6 Done Sent Done false
11462 3954 Acked Read 9 Done Sent Done false
11463 3955 Acked Write 10 Done Sent Done false
11464 3956 Acked Read 8 Done Sent Done false
11465 3957 Acked Write 6 Done Sent Done false
11466 3958 Acked Read 4 Done Sent Done false
11467 3959 Acked Read 1 Done Sent Done false
11468 3960 Acked Read 8 Done Sent Done false
11469 3961 Acked Read 10 Done Sent Done false
11470 3962 Acked Read 2 Done Sent Done false
11471 3963 Acked Read 1 Done Sent Done false
11472 3964 Acked Flush 0 Done Sent Done false
11473 3965 Acked Read 10 Done Sent Done false
11474 3966 Acked Write 4 Done Sent Done false
11475 3967 Acked Read 6 Done Sent Done false
11476 3968 Acked Read 5 Done Sent Done false
11477 3969 Acked Write 2 Done Sent Done false
11478 3970 Acked Write 7 Done Sent Done false
11479 3971 Acked Read 8 Done Sent Done false
11480 3972 Acked Read 8 Done Sent Done false
11481 3973 Acked Write 10 Done Sent Done false
11482 3974 Acked Write 8 Done Sent Done false
11483 3975 Acked Read 10 Done Sent Done false
11484 3976 Acked Write 5 Done Sent Done false
11485 3977 Acked Read 10 Done Sent Done false
11486 3978 Acked Read 7 Done Sent Done false
11487 3979 Acked Write 6 Done Sent Done false
11488 3980 Acked Read 5 Done Sent Done false
11489 3981 Acked Read 2 Done Sent Done false
11490 3982 Acked Read 4 Done Sent Done false
11491 3983 Acked Read 4 Done Sent Done false
11492 3984 Acked Read 6 Done Sent Done false
11493 3985 Acked Read 9 Done Sent Done false
11494 3986 Acked Read 6 Done Sent Done false
11495 3987 Acked Read 7 Done Sent Done false
11496 3988 Acked Flush 0 Done Sent Done false
11497 3989 Acked Write 6 Done Sent Done false
11498 3990 Acked Read 1 Done Sent Done false
11499 3991 Acked Read 8 Done Sent Done false
11500 3992 Acked Write 5 Done Sent Done false
11501 3993 Acked Read 4 Done Sent Done false
11502 3994 Acked Read 8 Done Sent Done false
11503 3995 Acked Write 4 Done Sent Done false
11504 3996 Acked Write 9 Done Sent Done false
11505 3997 Acked Read 1 Done Sent Done false
11506 3998 Acked Read 9 Done Sent Done false
11507 3999 Acked Write 5 Done Sent Done false
11508 4000 Acked Flush 0 Done Sent Done false
11509 4001 Acked Read 10 Done Sent Done false
11510 4002 Acked Write 6 Done Sent Done false
11511 4003 Acked Write 3 Done Sent Done false
11512 4004 Acked Read 10 Done Sent Done false
11513 4005 Acked Write 10 Done Sent Done false
11514 4006 Acked Write 1 Done Sent Done false
11515 4007 Acked Write 7 Done Sent Done false
11516 4008 Acked Read 2 Done Sent Done false
11517 4009 Acked Read 3 Done Sent Done false
11518 4010 Acked Read 4 Done Sent Done false
11519 4011 Acked Read 8 Done Sent Done false
11520 4012 Acked Write 7 Done Sent Done false
11521 4013 Acked Flush 0 Done Sent Done false
11522 4014 Acked Write 7 Done Sent Done false
11523 4015 Acked Read 4 Done Sent Done false
11524 4016 Acked Read 8 Done Sent Done false
11525 STATES DS:0 DS:1 DS:2 TOTAL
11526 Sent 0 301 0 301
11527 Done 301 0 301 602
11528 Skipped 0 0 0 0
11529 Error 0 0 0 0
11530 Last Flush: 4013 3715 4013
11531 Downstairs last five completed: 3715 3714 3713 3712 3711
11532 Upstairs last five completed: 4016 4015 4014 4013 4012
11533 CLIENT: Up:0 ds:301 act:2
11534 [8820] got start action
11535 [8820] I am going to restart
11536 Starting downstairs at port 8820
11537 Make output file at "/tmp/dsc/downstairs-8820.txt"
11538 [8820][1] reports Starting
11539 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1214)
11540 [8820][1] reports Running
11541 ----------------------------------------------------------------
11542 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11543 States: Active Offline Active
11544 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11545 3716 Acked Read 1 Done Sent Done false
11546 3717 Acked Read 6 Done Sent Done false
11547 3718 Acked Write 10 Done Sent Done false
11548 3719 Acked Write 3 Done Sent Done false
11549 3720 Acked Read 8 Done Sent Done false
11550 3721 Acked Read 2 Done Sent Done false
11551 3722 Acked Write 3 Done Sent Done false
11552 3723 Acked Write 6 Done Sent Done false
11553 3724 Acked Write 4 Done Sent Done false
11554 3725 Acked Read 10 Done Sent Done false
11555 3726 Acked Read 5 Done Sent Done false
11556 3727 Acked Read 1 Done Sent Done false
11557 3728 Acked Write 2 Done Sent Done false
11558 3729 Acked Read 4 Done Sent Done false
11559 3730 Acked Read 7 Done Sent Done false
11560 3731 Acked Read 5 Done Sent Done false
11561 3732 Acked Flush 0 Done Sent Done false
11562 3733 Acked Write 6 Done Sent Done false
11563 3734 Acked Read 4 Done Sent Done false
11564 3735 Acked Write 8 Done Sent Done false
11565 3736 Acked Read 6 Done Sent Done false
11566 3737 Acked Read 3 Done Sent Done false
11567 3738 Acked Write 3 Done Sent Done false
11568 3739 Acked Read 4 Done Sent Done false
11569 3740 Acked Read 2 Done Sent Done false
11570 3741 Acked Write 3 Done Sent Done false
11571 3742 Acked Write 8 Done Sent Done false
11572 3743 Acked Write 2 Done Sent Done false
11573 3744 Acked Flush 0 Done Sent Done false
11574 3745 Acked Read 10 Done Sent Done false
11575 3746 Acked Read 4 Done Sent Done false
11576 3747 Acked Write 4 Done Sent Done false
11577 3748 Acked Read 1 Done Sent Done false
11578 3749 Acked Flush 0 Done Sent Done false
11579 3750 Acked Read 9 Done Sent Done false
11580 3751 Acked Write 7 Done Sent Done false
11581 3752 Acked Flush 0 Done Sent Done false
11582 3753 Acked Write 10 Done Sent Done false
11583 3754 Acked Write 2 Done Sent Done false
11584 3755 Acked Flush 0 Done Sent Done false
11585 3756 Acked Read 5 Done Sent Done false
11586 3757 Acked Read 2 Done Sent Done false
11587 3758 Acked Read 8 Done Sent Done false
11588 3759 Acked Read 6 Done Sent Done false
11589 3760 Acked Write 5 Done Sent Done false
11590 3761 Acked Write 6 Done Sent Done false
11591 3762 Acked Read 9 Done Sent Done false
11592 3763 Acked Write 5 Done Sent Done false
11593 3764 Acked Read 9 Done Sent Done false
11594 3765 Acked Read 6 Done Sent Done false
11595 3766 Acked Read 10 Done Sent Done false
11596 3767 Acked Write 5 Done Sent Done false
11597 3768 Acked Flush 0 Done Sent Done false
11598 3769 Acked Write 8 Done Sent Done false
11599 3770 Acked Write 2 Done Sent Done false
11600 3771 Acked Read 3 Done Sent Done false
11601 3772 Acked Write 9 Done Sent Done false
11602 3773 Acked Read 8 Done Sent Done false
11603 3774 Acked Flush 0 Done Sent Done false
11604 3775 Acked Write 3 Done Sent Done false
11605 3776 Acked Write 8 Done Sent Done false
11606 3777 Acked Read 6 Done Sent Done false
11607 3778 Acked Write 3 Done Sent Done false
11608 3779 Acked Write 5 Done Sent Done false
11609 3780 Acked Read 7 Done Sent Done false
11610 3781 Acked Write 7 Done Sent Done false
11611 3782 Acked Read 2 Done Sent Done false
11612 3783 Acked Read 3 Done Sent Done false
11613 3784 Acked Write 6 Done Sent Done false
11614 3785 Acked Write 10 Done Sent Done false
11615 3786 Acked Write 7 Done Sent Done false
11616 3787 Acked Write 3 Done Sent Done false
11617 3788 Acked Flush 0 Done Sent Done false
11618 3789 Acked Read 10 Done Sent Done false
11619 3790 Acked Read 5 Done Sent Done false
11620 3791 Acked Read 4 Done Sent Done false
11621 3792 Acked Write 10 Done Sent Done false
11622 3793 Acked Write 7 Done Sent Done false
11623 3794 Acked Read 10 Done Sent Done false
11624 3795 Acked Read 10 Done Sent Done false
11625 3796 Acked Read 4 Done Sent Done false
11626 3797 Acked Flush 0 Done Sent Done false
11627 3798 Acked Read 9 Done Sent Done false
11628 3799 Acked Write 1 Done Sent Done false
11629 3800 Acked Read 9 Done Sent Done false
11630 3801 Acked Read 3 Done Sent Done false
11631 3802 Acked Flush 0 Done Sent Done false
11632 3803 Acked Read 6 Done Sent Done false
11633 3804 Acked Read 2 Done Sent Done false
11634 3805 Acked Read 6 Done Sent Done false
11635 3806 Acked Write 7 Done Sent Done false
11636 3807 Acked Flush 0 Done Sent Done false
11637 3808 Acked Flush 0 Done Sent Done false
11638 3809 Acked Read 3 Done Sent Done false
11639 3810 Acked Read 5 Done Sent Done false
11640 3811 Acked Read 1 Done Sent Done false
11641 3812 Acked Write 5 Done Sent Done false
11642 3813 Acked Read 6 Done Sent Done false
11643 3814 Acked Read 3 Done Sent Done false
11644 3815 Acked Write 3 Done Sent Done false
11645 3816 Acked Write 9 Done Sent Done false
11646 3817 Acked Flush 0 Done Sent Done false
11647 3818 Acked Read 7 Done Sent Done false
11648 3819 Acked Read 1 Done Sent Done false
11649 3820 Acked Read 1 Done Sent Done false
11650 3821 Acked Write 1 Done Sent Done false
11651 3822 Acked Write 8 Done Sent Done false
11652 3823 Acked Read 5 Done Sent Done false
11653 3824 Acked Read 2 Done Sent Done false
11654 3825 Acked Write 3 Done Sent Done false
11655 3826 Acked Write 5 Done Sent Done false
11656 3827 Acked Read 3 Done Sent Done false
11657 3828 Acked Read 6 Done Sent Done false
11658 3829 Acked Write 1 Done Sent Done false
11659 3830 Acked Write 3 Done Sent Done false
11660 3831 Acked Write 2 Done Sent Done false
11661 3832 Acked Flush 0 Done Sent Done false
11662 3833 Acked Read 4 Done Sent Done false
11663 3834 Acked Read 1 Done Sent Done false
11664 3835 Acked Write 3 Done Sent Done false
11665 3836 Acked Read 7 Done Sent Done false
11666 3837 Acked Read 7 Done Sent Done false
11667 3838 Acked Read 5 Done Sent Done false
11668 3839 Acked Write 10 Done Sent Done false
11669 3840 Acked Read 8 Done Sent Done false
11670 3841 Acked Read 9 Done Sent Done false
11671 3842 Acked Write 1 Done Sent Done false
11672 3843 Acked Read 10 Done Sent Done false
11673 3844 Acked Read 4 Done Sent Done false
11674 3845 Acked Flush 0 Done Sent Done false
11675 3846 Acked Read 9 Done Sent Done false
11676 3847 Acked Write 10 Done Sent Done false
11677 3848 Acked Write 9 Done Sent Done false
11678 3849 Acked Write 4 Done Sent Done false
11679 3850 Acked Read 1 Done Sent Done false
11680 3851 Acked Read 7 Done Sent Done false
11681 3852 Acked Flush 0 Done Sent Done false
11682 3853 Acked Flush 0 Done Sent Done false
11683 3854 Acked Read 1 Done Sent Done false
11684 3855 Acked Flush 0 Done Sent Done false
11685 3856 Acked Read 1 Done Sent Done false
11686 3857 Acked Write 2 Done Sent Done false
11687 3858 Acked Read 9 Done Sent Done false
11688 3859 Acked Read 2 Done Sent Done false
11689 3860 Acked Read 7 Done Sent Done false
11690 3861 Acked Write 1 Done Sent Done false
11691 3862 Acked Read 4 Done Sent Done false
11692 3863 Acked Read 9 Done Sent Done false
11693 3864 Acked Read 6 Done Sent Done false
11694 3865 Acked Read 4 Done Sent Done false
11695 3866 Acked Read 7 Done Sent Done false
11696 3867 Acked Write 3 Done Sent Done false
11697 3868 Acked Read 4 Done Sent Done false
11698 3869 Acked Flush 0 Done Sent Done false
11699 3870 Acked Flush 0 Done Sent Done false
11700 3871 Acked Write 7 Done Sent Done false
11701 3872 Acked Write 2 Done Sent Done false
11702 3873 Acked Write 3 Done Sent Done false
11703 3874 Acked Read 2 Done Sent Done false
11704 3875 Acked Read 5 Done Sent Done false
11705 3876 Acked Read 2 Done Sent Done false
11706 3877 Acked Read 6 Done Sent Done false
11707 3878 Acked Read 8 Done Sent Done false
11708 3879 Acked Write 7 Done Sent Done false
11709 3880 Acked Read 2 Done Sent Done false
11710 3881 Acked Read 10 Done Sent Done false
11711 3882 Acked Read 3 Done Sent Done false
11712 3883 Acked Write 10 Done Sent Done false
11713 3884 Acked Write 4 Done Sent Done false
11714 3885 Acked Read 5 Done Sent Done false
11715 3886 Acked Read 3 Done Sent Done false
11716 3887 Acked Read 8 Done Sent Done false
11717 3888 Acked Write 6 Done Sent Done false
11718 3889 Acked Read 2 Done Sent Done false
11719 3890 Acked Read 7 Done Sent Done false
11720 3891 Acked Read 5 Done Sent Done false
11721 3892 Acked Read 1 Done Sent Done false
11722 3893 Acked Write 7 Done Sent Done false
11723 3894 Acked Write 2 Done Sent Done false
11724 3895 Acked Write 3 Done Sent Done false
11725 3896 Acked Write 4 Done Sent Done false
11726 3897 Acked Write 5 Done Sent Done false
11727 3898 Acked Write 3 Done Sent Done false
11728 3899 Acked Write 2 Done Sent Done false
11729 3900 Acked Read 8 Done Sent Done false
11730 3901 Acked Read 8 Done Sent Done false
11731 3902 Acked Read 1 Done Sent Done false
11732 3903 Acked Read 7 Done Sent Done false
11733 3904 Acked Write 4 Done Sent Done false
11734 3905 Acked Write 8 Done Sent Done false
11735 3906 Acked Read 2 Done Sent Done false
11736 3907 Acked Write 6 Done Sent Done false
11737 3908 Acked Write 10 Done Sent Done false
11738 3909 Acked Read 7 Done Sent Done false
11739 3910 Acked Write 2 Done Sent Done false
11740 3911 Acked Read 8 Done Sent Done false
11741 3912 Acked Write 5 Done Sent Done false
11742 3913 Acked Write 7 Done Sent Done false
11743 3914 Acked Read 7 Done Sent Done false
11744 3915 Acked Read 6 Done Sent Done false
11745 3916 Acked Read 5 Done Sent Done false
11746 3917 Acked Read 1 Done Sent Done false
11747 3918 Acked Read 6 Done Sent Done false
11748 3919 Acked Flush 0 Done Sent Done false
11749 3920 Acked Write 2 Done Sent Done false
11750 3921 Acked Write 9 Done Sent Done false
11751 3922 Acked Read 6 Done Sent Done false
11752 3923 Acked Write 9 Done Sent Done false
11753 3924 Acked Read 1 Done Sent Done false
11754 3925 Acked Read 8 Done Sent Done false
11755 3926 Acked Write 9 Done Sent Done false
11756 3927 Acked Write 3 Done Sent Done false
11757 3928 Acked Write 1 Done Sent Done false
11758 3929 Acked Write 6 Done Sent Done false
11759 3930 Acked Read 3 Done Sent Done false
11760 3931 Acked Write 7 Done Sent Done false
11761 3932 Acked Write 3 Done Sent Done false
11762 3933 Acked Write 5 Done Sent Done false
11763 3934 Acked Read 3 Done Sent Done false
11764 3935 Acked Read 2 Done Sent Done false
11765 3936 Acked Write 3 Done Sent Done false
11766 3937 Acked Read 6 Done Sent Done false
11767 3938 Acked Read 1 Done Sent Done false
11768 3939 Acked Read 10 Done Sent Done false
11769 3940 Acked Read 5 Done Sent Done false
11770 3941 Acked Write 10 Done Sent Done false
11771 3942 Acked Read 3 Done Sent Done false
11772 3943 Acked Write 5 Done Sent Done false
11773 3944 Acked Read 10 Done Sent Done false
11774 3945 Acked Read 3 Done Sent Done false
11775 3946 Acked Read 6 Done Sent Done false
11776 3947 Acked Read 7 Done Sent Done false
11777 3948 Acked Read 9 Done Sent Done false
11778 3949 Acked Read 9 Done Sent Done false
11779 3950 Acked Write 2 Done Sent Done false
11780 3951 Acked Read 2 Done Sent Done false
11781 3952 Acked Write 8 Done Sent Done false
11782 3953 Acked Read 6 Done Sent Done false
11783 3954 Acked Read 9 Done Sent Done false
11784 3955 Acked Write 10 Done Sent Done false
11785 3956 Acked Read 8 Done Sent Done false
11786 3957 Acked Write 6 Done Sent Done false
11787 3958 Acked Read 4 Done Sent Done false
11788 3959 Acked Read 1 Done Sent Done false
11789 3960 Acked Read 8 Done Sent Done false
11790 3961 Acked Read 10 Done Sent Done false
11791 3962 Acked Read 2 Done Sent Done false
11792 3963 Acked Read 1 Done Sent Done false
11793 3964 Acked Flush 0 Done Sent Done false
11794 3965 Acked Read 10 Done Sent Done false
11795 3966 Acked Write 4 Done Sent Done false
11796 3967 Acked Read 6 Done Sent Done false
11797 3968 Acked Read 5 Done Sent Done false
11798 3969 Acked Write 2 Done Sent Done false
11799 3970 Acked Write 7 Done Sent Done false
11800 3971 Acked Read 8 Done Sent Done false
11801 3972 Acked Read 8 Done Sent Done false
11802 3973 Acked Write 10 Done Sent Done false
11803 3974 Acked Write 8 Done Sent Done false
11804 3975 Acked Read 10 Done Sent Done false
11805 3976 Acked Write 5 Done Sent Done false
11806 3977 Acked Read 10 Done Sent Done false
11807 3978 Acked Read 7 Done Sent Done false
11808 3979 Acked Write 6 Done Sent Done false
11809 3980 Acked Read 5 Done Sent Done false
11810 3981 Acked Read 2 Done Sent Done false
11811 3982 Acked Read 4 Done Sent Done false
11812 3983 Acked Read 4 Done Sent Done false
11813 3984 Acked Read 6 Done Sent Done false
11814 3985 Acked Read 9 Done Sent Done false
11815 3986 Acked Read 6 Done Sent Done false
11816 3987 Acked Read 7 Done Sent Done false
11817 3988 Acked Flush 0 Done Sent Done false
11818 3989 Acked Write 6 Done Sent Done false
11819 3990 Acked Read 1 Done Sent Done false
11820 3991 Acked Read 8 Done Sent Done false
11821 3992 Acked Write 5 Done Sent Done false
11822 3993 Acked Read 4 Done Sent Done false
11823 3994 Acked Read 8 Done Sent Done false
11824 3995 Acked Write 4 Done Sent Done false
11825 3996 Acked Write 9 Done Sent Done false
11826 3997 Acked Read 1 Done Sent Done false
11827 3998 Acked Read 9 Done Sent Done false
11828 3999 Acked Write 5 Done Sent Done false
11829 4000 Acked Flush 0 Done Sent Done false
11830 4001 Acked Read 10 Done Sent Done false
11831 4002 Acked Write 6 Done Sent Done false
11832 4003 Acked Write 3 Done Sent Done false
11833 4004 Acked Read 10 Done Sent Done false
11834 4005 Acked Write 10 Done Sent Done false
11835 4006 Acked Write 1 Done Sent Done false
11836 4007 Acked Write 7 Done Sent Done false
11837 4008 Acked Read 2 Done Sent Done false
11838 4009 Acked Read 3 Done Sent Done false
11839 4010 Acked Read 4 Done Sent Done false
11840 4011 Acked Read 8 Done Sent Done false
11841 4012 Acked Write 7 Done Sent Done false
11842 4013 Acked Flush 0 Done Sent Done false
11843 4014 Acked Write 7 Done Sent Done false
11844 4015 Acked Read 4 Done Sent Done false
11845 4016 Acked Read 8 Done Sent Done false
11846 4017 Acked Flush 0 Done Sent Done false
11847 STATES DS:0 DS:1 DS:2 TOTAL
11848 Sent 0 302 0 302
11849 Done 302 0 302 604
11850 Skipped 0 0 0 0
11851 Error 0 0 0 0
11852 Last Flush: 4017 3715 4017
11853 Downstairs last five completed: 3715 3714 3713 3712 3711
11854 Upstairs last five completed: 4017 4016 4015 4014 4013
11855 CLIENT: Up:0 ds:302 act:2
118562025-01-11T06:07:54.716ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118572025-01-11T06:07:54.717ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34515) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118582025-01-11T06:07:54.718ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118592025-01-11T06:07:54.718ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118602025-01-11T06:07:54.718ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118612025-01-11T06:07:54.718ZINFOcrucible: send last flush ID to this DS: 3715 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118622025-01-11T06:07:54.719ZINFOcrucible: Replied this last flush ID: 3715 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118632025-01-11T06:07:54.719ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118642025-01-11T06:07:54.719ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118652025-01-11T06:07:54.719ZINFOcrucible: [1] client re-new 302 jobs since flush 3715 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118662025-01-11T06:07:54.719ZINFOcrucible: [1] Replayed 302 jobs since flush: 3715 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11867 ----------------------------------------------------------------
11868 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11869 Downstairs last five completed: 4017 4016 4015 4014 4013
11870 Upstairs last five completed: 4017 4016 4015 4014 4013
11871 CLIENT: Up:0 ds:0 act:3
11872 Replay: All jobs finished
11873 Main task has work to do, go find it
11874 got dsc Stop(1)
11875 stop 1
11876 [8820] Got stop action so:false kr:false
11877 Jan 11 06:07:56.971 INFO request completed, latency_us: 323, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 2813d932-2813-4db8-9938-d02f9c2cc0c4, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
118782025-01-11T06:07:56.972ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118792025-01-11T06:07:56.972ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118802025-01-11T06:07:56.972ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11881 Jan 11 06:07:56.972 INFO request completed, latency_us: 168, response_code: 200, uri: /state/cid/1, method: GET, req_id: 14259136-1a96-4a63-8bca-062bc725403d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
118822025-01-11T06:07:56.972ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118832025-01-11T06:07:56.972ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
118842025-01-11T06:07:56.972ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
11885 [8820] Exited with: ExitStatus(unix_wait_status(9))
11886 [8820][1] reports Exit
11887 Jan 11 06:08:00.974 INFO request completed, latency_us: 241, response_code: 200, uri: /state/cid/1, method: GET, req_id: 95902876-6797-4299-826b-eabe8f2cefdc, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
11888 001/300 Write block 983 len 16384 data: 4 4 4 4
11889 002/300 Write block 1182 len 12288 data: 2 4 3
11890 003/300 Flush
11891 004/300 Write block 3185 len 16384 data: 2 4 4 4
11892 005/300 Read block 2901 len 20480
11893 006/300 Write block 2990 len 4096 data: 4
11894 007/300 Read block 3932 len 24576
11895 008/300 Read block 2403 len 36864
11896 009/300 Flush
11897 010/300 Read block 720 len 8192
11898 011/300 Flush
11899 012/300 Write block 3545 len 16384 data: 2 2 3 3
11900 013/300 Read block 2605 len 4096
11901 014/300 Write block 4296 len 20480 data: 5 6 7 7 7
11902 015/300 Write block 2945 len 24576 data: 5 6 6 4 6 6
11903 016/300 Read block 887 len 28672
11904 017/300 Write block 2228 len 36864 data: 3 4 6 6 6 5 3 3 3
11905 018/300 Write block 2038 len 12288 data: 2 3 3
11906 019/300 Write block 816 len 28672 data: 3 2 2 2 2 2 3
11907 020/300 Read block 3297 len 20480
11908 021/300 Read block 1833 len 28672
11909 022/300 Read block 1111 len 28672
11910 023/300 Read block 1221 len 12288
11911 024/300 Read block 4255 len 40960
11912 025/300 Write block 78 len 32768 data: 2 2 4 5 5 4 4 3
11913 026/300 Read block 3639 len 40960
11914 027/300 Read block 2817 len 24576
11915 028/300 Write block 3606 len 40960 data: 3 2 2 2 2 2 3 3 3 3
11916 029/300 Write block 2299 len 12288 data: 3 3 4
11917 030/300 Write block 2681 len 8192 data: 4 6
11918 031/300 Write block 1773 len 20480 data: 2 3 3 3 3
11919 032/300 Write block 694 len 16384 data: 3 2 2 2
11920 033/300 Read block 1573 len 24576
11921 034/300 Write block 2228 len 20480 data: 4 5 7 7 7
11922 035/300 Read block 4747 len 12288
11923 036/300 Read block 4492 len 36864
11924 037/300 Write block 3826 len 8192 data: 4 3
11925 038/300 Read block 781 len 4096
11926 039/300 Read block 1127 len 24576
11927 040/300 Read block 4360 len 28672
11928 041/300 Read block 4226 len 24576
11929 042/300 Flush
11930 043/300 Write block 1352 len 20480 data: 2 2 2 2 3
11931 044/300 Read block 2599 len 16384
11932 045/300 Read block 2746 len 16384
11933 046/300 Write block 3037 len 36864 data: 6 6 6 6 6 5 3 2 2
11934 047/300 Read block 4167 len 8192
11935 048/300 Read block 1335 len 36864
11936 049/300 Write block 1404 len 28672 data: 3 4 3 3 3 3 4
11937 050/300 Write block 486 len 12288 data: 5 4 5
11938 051/300 Flush
11939 052/300 Flush
11940 053/300 Read block 4230 len 12288
11941 054/300 Write block 384 len 36864 data: 2 4 5 5 5 5 5 5 5
11942 055/300 Read block 227 len 20480
11943 056/300 Read block 3237 len 16384
11944 057/300 Write block 4524 len 36864 data: 2 2 2 2 3 3 2 3 3
11945 058/300 Read block 1894 len 40960
11946 059/300 Read block 2483 len 8192
11947 060/300 Write block 4816 len 28672 data: 2 3 2 2 2 2 3
11948 061/300 Read block 3634 len 8192
11949 062/300 Write block 3481 len 28672 data: 5 5 3 3 3 3 2
11950 063/300 Write block 4825 len 32768 data: 3 4 5 5 5 5 5 5
11951 064/300 Read block 1322 len 4096
11952 065/300 Read block 2485 len 16384
11953 066/300 Read block 2033 len 24576
11954 067/300 Write block 749 len 40960 data: 3 4 4 3 3 4 3 3 3 4
11955 068/300 Write block 4954 len 24576 data: 4 3 3 3 2 2
11956 069/300 Write block 848 len 32768 data: 6 4 3 3 3 4 4 3
11957 070/300 Read block 1034 len 16384
11958 071/300 Write block 2250 len 32768 data: 3 3 4 4 4 5 4 4
11959 072/300 Write block 3757 len 20480 data: 4 4 5 5 3
11960 073/300 Write block 3714 len 20480 data: 3 3 4 4 3
11961 074/300 Write block 1899 len 40960 data: 4 4 3 3 3 3 3 3 3 3
11962 075/300 Read block 2759 len 36864
11963 076/300 Read block 1005 len 28672
11964 077/300 Write block 605 len 20480 data: 3 3 2 2 2
11965 078/300 Read block 2369 len 40960
11966 079/300 Write block 1977 len 12288 data: 3 2 4
11967 080/300 Flush
11968 081/300 Read block 1693 len 12288
11969 082/300 Read block 1138 len 32768
11970 083/300 Read block 2907 len 12288
11971 084/300 Write block 830 len 28672 data: 3 3 3 3 3 4 4
11972 085/300 Write block 1296 len 20480 data: 3 3 4 4 3
11973 086/300 Read block 4788 len 16384
11974 087/300 Write block 508 len 16384 data: 2 2 2 2
11975 088/300 Read block 423 len 28672
11976 089/300 Write block 4033 len 4096 data: 3
11977 090/300 Write block 1305 len 4096 data: 3
11978 091/300 Write block 4200 len 28672 data: 3 3 2 3 3 4 4
11979 092/300 Write block 1795 len 12288 data: 2 2 3
11980 093/300 Read block 1332 len 40960
11981 094/300 Read block 4419 len 24576
11982 095/300 Write block 3730 len 20480 data: 3 4 5 5 5
11983 096/300 Write block 1178 len 28672 data: 3 3 2 2 3 5 4
11984 097/300 Read block 2852 len 4096
11985 098/300 Read block 3186 len 8192
11986 099/300 Write block 3697 len 20480 data: 4 4 4 4 4
11987 100/300 Write block 1197 len 8192 data: 2 2
11988 101/300 Write block 1872 len 40960 data: 4 5 4 4 4 4 2 3 3 3
11989 102/300 Read block 737 len 24576
11990 103/300 Write block 3819 len 20480 data: 4 4 4 4 3
11991 104/300 Read block 1939 len 32768
11992 105/300 Read block 1005 len 36864
11993 106/300 Read block 4950 len 32768
11994 107/300 Flush
11995 108/300 Write block 2344 len 20480 data: 2 2 3 3 3
11996 109/300 Flush
11997 110/300 Read block 794 len 16384
11998 111/300 Read block 3625 len 16384
11999 112/300 Write block 2587 len 32768 data: 2 2 2 2 3 3 3 3
12000 113/300 Write block 2268 len 24576 data: 3 4 3 4 4 4
12001 114/300 Write block 2041 len 24576 data: 4 5 5 5 4 4
12002 115/300 Read block 4067 len 20480
12003 116/300 Flush
12004 117/300 Flush
12005 118/300 Read block 1542 len 24576
12006 119/300 Read block 2117 len 8192
12007 120/300 Read block 921 len 32768
12008 121/300 Flush
12009 122/300 Write block 2532 len 32768 data: 2 3 3 5 5 3 3 3
12010 123/300 Read block 1322 len 16384
12011 124/300 Write block 4271 len 24576 data: 3 3 2 2 2 2
12012 125/300 Read block 3617 len 40960
12013 126/300 Write block 2304 len 28672 data: 4 4 3 3 3 2 2
12014 127/300 Write block 1264 len 28672 data: 4 3 3 3 3 3 2
12015 128/300 Read block 1066 len 16384
12016 129/300 Read block 2693 len 8192
12017 130/300 Read block 3200 len 28672
12018 131/300 Write block 4312 len 32768 data: 4 3 2 2 3 3 4 5
12019 132/300 Read block 2670 len 36864
12020 133/300 Write block 3948 len 20480 data: 3 3 3 3 3
12021 134/300 Flush
12022 135/300 Read block 4546 len 32768
12023 136/300 Read block 4790 len 12288
12024 137/300 Write block 4987 len 20480 data: 5 5 5 7 7
12025 138/300 Read block 623 len 32768
12026 139/300 Write block 1114 len 28672 data: 3 3 3 3 4 4 4
12027 140/300 Write block 213 len 16384 data: 3 3 3 3
12028 141/300 Write block 1931 len 32768 data: 5 5 4 2 3 3 3 3
12029 142/300 Read block 2135 len 8192
12030 143/300 Flush
12031 144/300 Read block 167 len 20480
12032 145/300 Read block 1303 len 8192
12033 146/300 Write block 4628 len 40960 data: 3 2 2 4 4 5 5 6 5 5
12034 147/300 Read block 4351 len 20480
12035 148/300 Write block 187 len 36864 data: 4 4 4 3 3 3 3 2 2
12036 149/300 Write block 4698 len 8192 data: 6 7
12037 150/300 Flush
12038 151/300 Flush
12039 152/300 Read block 4140 len 4096
12040 153/300 Write block 4448 len 20480 data: 4 4 4 5 5
12041 154/300 Read block 2312 len 32768
12042 155/300 Write block 631 len 8192 data: 3 4
12043 156/300 Write block 3450 len 36864 data: 5 5 4 2 2 3 3 4 5
12044 157/300 Write block 4454 len 8192 data: 5 6
12045 158/300 Read block 4660 len 36864
12046 159/300 Read block 2917 len 12288
12047 160/300 Write block 500 len 28672 data: 4 4 4 4 4 2 2
12048 161/300 Write block 398 len 24576 data: 5 5 5 5 5 5
12049 162/300 Write block 4071 len 16384 data: 4 5 4 5
12050 163/300 Write block 4697 len 16384 data: 5 7 8 5
12051 164/300 Read block 2400 len 16384
12052 165/300 Read block 3939 len 36864
12053 166/300 Read block 3406 len 20480
12054 167/300 Read block 3511 len 8192
12055 168/300 Flush
12056 169/300 Flush
12057 170/300 Write block 351 len 40960 data: 2 2 2 2 3 3 3 3 3 3
12058 171/300 Write block 2455 len 40960 data: 2 2 2 2 2 2 2 2 3 4
12059 172/300 Read block 4389 len 20480
12060 173/300 Read block 3003 len 36864
12061 174/300 Flush
12062 175/300 Write block 4146 len 8192 data: 6 4
12063 176/300 Read block 3648 len 16384
12064 177/300 Read block 1195 len 12288
12065 178/300 Read block 4120 len 12288
12066 179/300 Read block 4779 len 4096
12067 180/300 Write block 3786 len 8192 data: 3 3
12068 181/300 Read block 469 len 8192
12069 182/300 Read block 3565 len 20480
12070 183/300 Write block 2157 len 40960 data: 3 2 2 2 2 2 2 2 2 2
12071 184/300 Write block 4957 len 12288 data: 4 3 3
12072 185/300 Read block 459 len 40960
12073 186/300 Flush
12074 187/300 Read block 1353 len 28672
12075 188/300 Write block 4689 len 4096 data: 4
12076 189/300 Read block 4535 len 24576
12077 190/300 Flush
12078 191/300 Read block 400 len 32768
12079 192/300 Read block 2556 len 16384
12080 193/300 Read block 1132 len 4096
12081 194/300 Read block 2356 len 28672
12082 195/300 Read block 3810 len 12288
12083 196/300 Write block 1708 len 4096 data: 3
12084 197/300 Read block 4603 len 12288
12085 198/300 Flush
12086 199/300 Read block 1491 len 4096
12087 200/300 Read block 3550 len 32768
12088 201/300 Read block 1517 len 20480
12089 202/300 Write block 1428 len 8192 data: 3 3
12090 203/300 Write block 279 len 16384 data: 6 5 6 7
12091 204/300 Read block 4890 len 4096
12092 205/300 Read block 52 len 40960
12093 206/300 Read block 1892 len 16384
12094 207/300 Read block 1244 len 28672
12095 208/300 Read block 3182 len 4096
12096 209/300 Write block 1987 len 20480 data: 2 2 2 3 4
12097 210/300 Read block 4595 len 16384
12098 211/300 Read block 4139 len 4096
12099 212/300 Read block 2297 len 16384
12100 213/300 Write block 2343 len 16384 data: 2 3 3 4
12101 214/300 Read block 2980 len 40960
12102 215/300 Read block 3461 len 28672
12103 216/300 Read block 119 len 20480
12104 217/300 Write block 4432 len 20480 data: 2 2 2 3 3
12105 218/300 Write block 2765 len 16384 data: 3 2 2 2
12106 219/300 Read block 3745 len 32768
12107 220/300 Read block 4705 len 28672
12108 221/300 Flush
12109 222/300 Read block 4900 len 8192
12110 223/300 Read block 1034 len 24576
12111 224/300 Read block 1442 len 4096
12112 225/300 Write block 2577 len 4096 data: 4
12113 226/300 Write block 3867 len 40960 data: 4 3 2 3 4 4 4 4 4 4
12114 227/300 Write block 2093 len 4096 data: 4
12115 228/300 Read block 2727 len 40960
12116 229/300 Read block 4422 len 16384
12117 230/300 Read block 4199 len 28672
12118 231/300 Write block 838 len 8192 data: 4 3
12119 232/300 Write block 2940 len 4096 data: 5
12120 233/300 Write block 4236 len 40960 data: 2 2 3 3 3 3 4 6 6 3
12121 234/300 Read block 165 len 20480
12122 235/300 Read block 1911 len 32768
12123 236/300 Flush
12124 237/300 Flush
12125 238/300 Read block 1610 len 8192
12126 239/300 Flush
12127 240/300 Read block 2665 len 32768
12128 241/300 Read block 2227 len 16384
12129 242/300 Read block 1060 len 32768
12130 243/300 Read block 3434 len 40960
12131 244/300 Write block 1290 len 32768 data: 4 4 3 3 3 3 4 4
12132 245/300 Read block 4685 len 28672
12133 246/300 Read block 4865 len 40960
12134 247/300 Write block 4237 len 16384 data: 3 4 4 4
12135 248/300 Read block 2055 len 20480
12136 249/300 Write block 4508 len 8192 data: 6 6
12137 250/300 Read block 3077 len 8192
12138 251/300 Read block 739 len 20480
12139 252/300 Write block 3349 len 12288 data: 4 4 3
12140 253/300 Write block 2030 len 4096 data: 3
12141 254/300 Read block 281 len 32768
12142 255/300 Flush
12143 256/300 Write block 671 len 16384 data: 3 3 3 3
12144 257/300 Write block 2968 len 36864 data: 3 3 3 3 3 3 3 4 2
12145 258/300 Read block 2235 len 12288
12146 259/300 Write block 3179 len 32768 data: 4 4 4 3 3 2 3 5
12147 260/300 Write block 4247 len 16384 data: 3 2 2 2
12148 261/300 Read block 1631 len 20480
12149 262/300 Read block 2573 len 36864
12150 263/300 Flush
12151 264/300 Write block 922 len 12288 data: 2 2 2
12152 265/300 Write block 2076 len 36864 data: 4 5 5 6 5 6 7 7 6
12153 266/300 Write block 1009 len 8192 data: 5 6
12154 267/300 Read block 3150 len 28672
12155 268/300 Write block 576 len 32768 data: 3 4 5 5 5 5 4 4
12156 269/300 Write block 192 len 40960 data: 4 4 3 3 2 2 2 2 3 3
12157 270/300 Read block 3431 len 36864
12158 271/300 Write block 4304 len 4096 data: 5
12159 272/300 Read block 956 len 28672
12160 273/300 Read block 2180 len 24576
12161 274/300 Read block 4715 len 12288
12162 275/300 Write block 160 len 36864 data: 2 2 2 2 2 2 3 3 3
12163 276/300 Write block 4528 len 8192 data: 4 4
12164 277/300 Write block 3928 len 36864 data: 2 3 3 3 3 2 2 2 2
12165 278/300 Flush
12166 279/300 Flush
12167 280/300 Read block 4082 len 24576
12168 281/300 Read block 542 len 24576
12169 282/300 Write block 4902 len 32768 data: 3 3 5 4 4 4 4 4
12170 283/300 Flush
12171 284/300 Read block 261 len 12288
12172 285/300 Read block 3512 len 12288
12173 286/300 Flush
12174 287/300 Write block 3772 len 32768 data: 3 3 3 3 3 3 3 4
12175 288/300 Write block 4717 len 8192 data: 3 3
12176 289/300 Read block 4900 len 12288
12177 290/300 Read block 4357 len 32768
12178 291/300 Read block 1060 len 4096
12179 292/300 Write block 3191 len 40960 data: 3 3 3 3 3 2 2 2 3 3
12180 293/300 Write block 1109 len 16384 data: 2 2 2 3
12181 294/300 Write block 4790 len 40960 data: 3 3 3 3 3 4 4 3 2 2
12182 295/300 Read block 1622 len 20480
12183 296/300 Read block 2766 len 40960
12184 297/300 Read block 1139 len 16384
12185 298/300 Read block 1767 len 8192
12186 299/300 Read block 4308 len 12288
12187 300/300 Write block 3268 len 20480 data: 3 4 5 5 6
12188 301/300 Write block 2122 len 36864 data: 2 2 2 2 2 3 4 4 4
12189 Main task has work to do, go find it
12190 got dsc Start(1)
12191 start 1
12192 Jan 11 06:08:01.215 INFO request completed, latency_us: 292, response_code: 204, uri: /start/cid/1, method: POST, req_id: aea0cb68-a714-4244-8da4-9cf29915e2a4, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
12193 Replay: started 1, returned:Ok(())
12194 ----------------------------------------------------------------
12195 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12196 States: Active Offline Active
12197 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12198 4018 Acked Write 4 Done Sent Done false
12199 4019 Acked Write 3 Done Sent Done false
12200 4020 Acked Flush 0 Done Sent Done false
12201 4021 Acked Write 4 Done Sent Done false
12202 4022 Acked Read 5 Done Sent Done false
12203 4023 Acked Write 1 Done Sent Done false
12204 4024 Acked Read 6 Done Sent Done false
12205 4025 Acked Read 9 Done Sent Done false
12206 4026 Acked Flush 0 Done Sent Done false
12207 4027 Acked Read 2 Done Sent Done false
12208 4028 Acked Flush 0 Done Sent Done false
12209 4029 Acked Write 4 Done Sent Done false
12210 4030 Acked Read 1 Done Sent Done false
12211 4031 Acked Write 5 Done Sent Done false
12212 4032 Acked Write 6 Done Sent Done false
12213 4033 Acked Read 7 Done Sent Done false
12214 4034 Acked Write 9 Done Sent Done false
12215 4035 Acked Write 3 Done Sent Done false
12216 4036 Acked Write 7 Done Sent Done false
12217 4037 Acked Read 5 Done Sent Done false
12218 4038 Acked Read 7 Done Sent Done false
12219 4039 Acked Read 7 Done Sent Done false
12220 4040 Acked Read 3 Done Sent Done false
12221 4041 Acked Read 10 Done Sent Done false
12222 4042 Acked Write 8 Done Sent Done false
12223 4043 Acked Read 10 Done Sent Done false
12224 4044 Acked Read 6 Done Sent Done false
12225 4045 Acked Write 10 Done Sent Done false
12226 4046 Acked Write 3 Done Sent Done false
12227 4047 Acked Write 2 Done Sent Done false
12228 4048 Acked Write 5 Done Sent Done false
12229 4049 Acked Write 4 Done Sent Done false
12230 4050 Acked Read 6 Done Sent Done false
12231 4051 Acked Write 5 Done Sent Done false
12232 4052 Acked Read 3 Done Sent Done false
12233 4053 Acked Read 9 Done Sent Done false
12234 4054 Acked Write 2 Done Sent Done false
12235 4055 Acked Read 1 Done Sent Done false
12236 4056 Acked Read 6 Done Sent Done false
12237 4057 Acked Read 7 Done Sent Done false
12238 4058 Acked Read 6 Done Sent Done false
12239 4059 Acked Flush 0 Done Sent Done false
12240 4060 Acked Write 5 Done Sent Done false
12241 4061 Acked Read 4 Done Sent Done false
12242 4062 Acked Read 4 Done Sent Done false
12243 4063 Acked Write 9 Done Sent Done false
12244 4064 Acked Read 2 Done Sent Done false
12245 4065 Acked Read 9 Done Sent Done false
12246 4066 Acked Write 7 Done Sent Done false
12247 4067 Acked Write 3 Done Sent Done false
12248 4068 Acked Flush 0 Done Sent Done false
12249 4069 Acked Flush 0 Done Sent Done false
12250 4070 Acked Read 3 Done Sent Done false
12251 4071 Acked Write 9 Done Sent Done false
12252 4072 Acked Read 5 Done Sent Done false
12253 4073 Acked Read 4 Done Sent Done false
12254 4074 Acked Write 9 Done Sent Done false
12255 4075 Acked Read 10 Done Sent Done false
12256 4076 Acked Read 2 Done Sent Done false
12257 4077 Acked Write 7 Done Sent Done false
12258 4078 Acked Read 2 Done Sent Done false
12259 4079 Acked Write 7 Done Sent Done false
12260 4080 Acked Write 8 Done Sent Done false
12261 4081 Acked Read 1 Done Sent Done false
12262 4082 Acked Read 4 Done Sent Done false
12263 4083 Acked Read 6 Done Sent Done false
12264 4084 Acked Write 10 Done Sent Done false
12265 4085 Acked Write 6 Done Sent Done false
12266 4086 Acked Write 8 Done Sent Done false
12267 4087 Acked Read 4 Done Sent Done false
12268 4088 Acked Write 8 Done Sent Done false
12269 4089 Acked Write 5 Done Sent Done false
12270 4090 Acked Write 5 Done Sent Done false
12271 4091 Acked Write 10 Done Sent Done false
12272 4092 Acked Read 9 Done Sent Done false
12273 4093 Acked Read 7 Done Sent Done false
12274 4094 Acked Write 5 Done Sent Done false
12275 4095 Acked Read 10 Done Sent Done false
12276 4096 Acked Write 3 Done Sent Done false
12277 4097 Acked Flush 0 Done Sent Done false
12278 4098 Acked Read 3 Done Sent Done false
12279 4099 Acked Read 8 Done Sent Done false
12280 4100 Acked Read 3 Done Sent Done false
12281 4101 Acked Write 7 Done Sent Done false
12282 4102 Acked Write 5 Done Sent Done false
12283 4103 Acked Read 4 Done Sent Done false
12284 4104 Acked Write 4 Done Sent Done false
12285 4105 Acked Read 7 Done Sent Done false
12286 4106 Acked Write 1 Done Sent Done false
12287 4107 Acked Write 1 Done Sent Done false
12288 4108 Acked Write 7 Done Sent Done false
12289 4109 Acked Write 3 Done Sent Done false
12290 4110 Acked Read 10 Done Sent Done false
12291 4111 Acked Read 6 Done Sent Done false
12292 4112 Acked Write 5 Done Sent Done false
12293 4113 Acked Write 7 Done Sent Done false
12294 4114 Acked Read 1 Done Sent Done false
12295 4115 Acked Read 2 Done Sent Done false
12296 4116 Acked Write 5 Done Sent Done false
12297 4117 Acked Write 2 Done Sent Done false
12298 4118 Acked Write 10 Done Sent Done false
12299 4119 Acked Read 6 Done Sent Done false
12300 4120 Acked Write 5 Done Sent Done false
12301 4121 Acked Read 8 Done Sent Done false
12302 4122 Acked Read 9 Done Sent Done false
12303 4123 Acked Read 8 Done Sent Done false
12304 4124 Acked Flush 0 Done Sent Done false
12305 4125 Acked Write 5 Done Sent Done false
12306 4126 Acked Flush 0 Done Sent Done false
12307 4127 Acked Read 4 Done Sent Done false
12308 4128 Acked Read 4 Done Sent Done false
12309 4129 Acked Write 8 Done Sent Done false
12310 4130 Acked Write 6 Done Sent Done false
12311 4131 Acked Write 6 Done Sent Done false
12312 4132 Acked Read 5 Done Sent Done false
12313 4133 Acked Flush 0 Done Sent Done false
12314 4134 Acked Flush 0 Done Sent Done false
12315 4135 Acked Read 6 Done Sent Done false
12316 4136 Acked Read 2 Done Sent Done false
12317 4137 Acked Read 8 Done Sent Done false
12318 4138 Acked Flush 0 Done Sent Done false
12319 4139 Acked Write 8 Done Sent Done false
12320 4140 Acked Read 4 Done Sent Done false
12321 4141 Acked Write 6 Done Sent Done false
12322 4142 Acked Read 10 Done Sent Done false
12323 4143 Acked Write 7 Done Sent Done false
12324 4144 Acked Write 7 Done Sent Done false
12325 4145 Acked Read 4 Done Sent Done false
12326 4146 Acked Read 2 Done Sent Done false
12327 4147 Acked Read 7 Done Sent Done false
12328 4148 Acked Write 8 Done Sent Done false
12329 4149 Acked Read 9 Done Sent Done false
12330 4150 Acked Write 5 Done Sent Done false
12331 4151 Acked Flush 0 Done Sent Done false
12332 4152 Acked Read 8 Done Sent Done false
12333 4153 Acked Read 3 Done Sent Done false
12334 4154 Acked Write 5 Done Sent Done false
12335 4155 Acked Read 8 Done Sent Done false
12336 4156 Acked Write 7 Done Sent Done false
12337 4157 Acked Write 4 Done Sent Done false
12338 4158 Acked Write 8 Done Sent Done false
12339 4159 Acked Read 2 Done Sent Done false
12340 4160 Acked Flush 0 Done Sent Done false
12341 4161 Acked Read 5 Done Sent Done false
12342 4162 Acked Read 2 Done Sent Done false
12343 4163 Acked Write 10 Done Sent Done false
12344 4164 Acked Read 5 Done Sent Done false
12345 4165 Acked Write 9 Done Sent Done false
12346 4166 Acked Write 2 Done Sent Done false
12347 4167 Acked Flush 0 Done Sent Done false
12348 4168 Acked Flush 0 Done Sent Done false
12349 4169 Acked Read 1 Done Sent Done false
12350 4170 Acked Write 5 Done Sent Done false
12351 4171 Acked Read 8 Done Sent Done false
12352 4172 Acked Write 2 Done Sent Done false
12353 4173 Acked Write 9 Done Sent Done false
12354 4174 Acked Write 2 Done Sent Done false
12355 4175 Acked Read 9 Done Sent Done false
12356 4176 Acked Read 3 Done Sent Done false
12357 4177 Acked Write 7 Done Sent Done false
12358 4178 Acked Write 6 Done Sent Done false
12359 4179 Acked Write 4 Done Sent Done false
12360 4180 Acked Write 4 Done Sent Done false
12361 4181 Acked Read 4 Done Sent Done false
12362 4182 Acked Read 9 Done Sent Done false
12363 4183 Acked Read 5 Done Sent Done false
12364 4184 Acked Read 2 Done Sent Done false
12365 4185 Acked Flush 0 Done Sent Done false
12366 4186 Acked Flush 0 Done Sent Done false
12367 4187 Acked Write 10 Done Sent Done false
12368 4188 Acked Write 10 Done Sent Done false
12369 4189 Acked Read 5 Done Sent Done false
12370 4190 Acked Read 9 Done Sent Done false
12371 4191 Acked Flush 0 Done Sent Done false
12372 4192 Acked Write 2 Done Sent Done false
12373 4193 Acked Read 4 Done Sent Done false
12374 4194 Acked Read 3 Done Sent Done false
12375 4195 Acked Read 3 Done Sent Done false
12376 4196 Acked Read 1 Done Sent Done false
12377 4197 Acked Write 2 Done Sent Done false
12378 4198 Acked Read 2 Done Sent Done false
12379 4199 Acked Read 5 Done Sent Done false
12380 4200 Acked Write 10 Done Sent Done false
12381 4201 Acked Write 3 Done Sent Done false
12382 4202 Acked Read 10 Done Sent Done false
12383 4203 Acked Flush 0 Done Sent Done false
12384 4204 Acked Read 7 Done Sent Done false
12385 4205 Acked Write 1 Done Sent Done false
12386 4206 Acked Read 6 Done Sent Done false
12387 4207 Acked Flush 0 Done Sent Done false
12388 4208 Acked Read 8 Done Sent Done false
12389 4209 Acked Read 4 Done Sent Done false
12390 4210 Acked Read 1 Done Sent Done false
12391 4211 Acked Read 7 Done Sent Done false
12392 4212 Acked Read 3 Done Sent Done false
12393 4213 Acked Write 1 Done Sent Done false
12394 4214 Acked Read 3 Done Sent Done false
12395 4215 Acked Flush 0 Done Sent Done false
12396 4216 Acked Read 1 Done Sent Done false
12397 4217 Acked Read 8 Done Sent Done false
12398 4218 Acked Read 5 Done Sent Done false
12399 4219 Acked Write 2 Done Sent Done false
12400 4220 Acked Write 4 Done Sent Done false
12401 4221 Acked Read 1 Done Sent Done false
12402 4222 Acked Read 10 Done Sent Done false
12403 4223 Acked Read 4 Done Sent Done false
12404 4224 Acked Read 7 Done Sent Done false
12405 4225 Acked Read 1 Done Sent Done false
12406 4226 Acked Write 5 Done Sent Done false
12407 4227 Acked Read 4 Done Sent Done false
12408 4228 Acked Read 1 Done Sent Done false
12409 4229 Acked Read 4 Done Sent Done false
12410 4230 Acked Write 4 Done Sent Done false
12411 4231 Acked Read 10 Done Sent Done false
12412 4232 Acked Read 7 Done Sent Done false
12413 4233 Acked Read 5 Done Sent Done false
12414 4234 Acked Write 5 Done Sent Done false
12415 4235 Acked Write 4 Done Sent Done false
12416 4236 Acked Read 8 Done Sent Done false
12417 4237 Acked Read 7 Done Sent Done false
12418 4238 Acked Flush 0 Done Sent Done false
12419 4239 Acked Read 2 Done Sent Done false
12420 4240 Acked Read 6 Done Sent Done false
12421 4241 Acked Read 1 Done Sent Done false
12422 4242 Acked Write 1 Done Sent Done false
12423 4243 Acked Write 10 Done Sent Done false
12424 4244 Acked Write 1 Done Sent Done false
12425 4245 Acked Read 10 Done Sent Done false
12426 4246 Acked Read 4 Done Sent Done false
12427 4247 Acked Read 7 Done Sent Done false
12428 4248 Acked Write 2 Done Sent Done false
12429 4249 Acked Write 1 Done Sent Done false
12430 4250 Acked Write 10 Done Sent Done false
12431 4251 Acked Read 5 Done Sent Done false
12432 4252 Acked Read 8 Done Sent Done false
12433 4253 Acked Flush 0 Done Sent Done false
12434 4254 Acked Flush 0 Done Sent Done false
12435 4255 Acked Read 2 Done Sent Done false
12436 4256 Acked Flush 0 Done Sent Done false
12437 4257 Acked Read 8 Done Sent Done false
12438 4258 Acked Read 4 Done Sent Done false
12439 4259 Acked Read 8 Done Sent Done false
12440 4260 Acked Read 10 Done Sent Done false
12441 4261 Acked Write 8 Done Sent Done false
12442 4262 Acked Read 7 Done Sent Done false
12443 4263 Acked Read 10 Done Sent Done false
12444 4264 Acked Write 4 Done Sent Done false
12445 4265 Acked Read 5 Done Sent Done false
12446 4266 Acked Write 2 Done Sent Done false
12447 4267 Acked Read 2 Done Sent Done false
12448 4268 Acked Read 5 Done Sent Done false
12449 4269 Acked Write 3 Done Sent Done false
12450 4270 Acked Write 1 Done Sent Done false
12451 4271 Acked Read 8 Done Sent Done false
12452 4272 Acked Flush 0 Done Sent Done false
12453 4273 Acked Write 4 Done Sent Done false
12454 4274 Acked Write 9 Done Sent Done false
12455 4275 Acked Read 3 Done Sent Done false
12456 4276 Acked Write 8 Done Sent Done false
12457 4277 Acked Write 4 Done Sent Done false
12458 4278 Acked Read 5 Done Sent Done false
12459 4279 Acked Read 9 Done Sent Done false
12460 4280 Acked Flush 0 Done Sent Done false
12461 4281 Acked Write 3 Done Sent Done false
12462 4282 Acked Write 9 Done Sent Done false
12463 4283 Acked Write 2 Done Sent Done false
12464 4284 Acked Read 7 Done Sent Done false
12465 4285 Acked Write 8 Done Sent Done false
12466 4286 Acked Write 10 Done Sent Done false
12467 4287 Acked Read 9 Done Sent Done false
12468 4288 Acked Write 1 Done Sent Done false
12469 4289 Acked Read 7 Done Sent Done false
12470 4290 Acked Read 6 Done Sent Done false
12471 4291 Acked Read 3 Done Sent Done false
12472 4292 Acked Write 9 Done Sent Done false
12473 4293 Acked Write 2 Done Sent Done false
12474 4294 Acked Write 9 Done Sent Done false
12475 4295 Acked Flush 0 Done Sent Done false
12476 4296 Acked Flush 0 Done Sent Done false
12477 4297 Acked Read 6 Done Sent Done false
12478 4298 Acked Read 6 Done Sent Done false
12479 4299 Acked Write 8 Done Sent Done false
12480 4300 Acked Flush 0 Done Sent Done false
12481 4301 Acked Read 3 Done Sent Done false
12482 4302 Acked Read 3 Done Sent Done false
12483 4303 Acked Flush 0 Done Sent Done false
12484 4304 Acked Write 8 Done Sent Done false
12485 4305 Acked Write 2 Done Sent Done false
12486 4306 Acked Read 3 Done Sent Done false
12487 4307 Acked Read 8 Done Sent Done false
12488 4308 Acked Read 1 Done Sent Done false
12489 4309 Acked Write 10 Done Sent Done false
12490 4310 Acked Write 4 Done Sent Done false
12491 4311 Acked Write 10 Done Sent Done false
12492 4312 Acked Read 5 Done Sent Done false
12493 4313 Acked Read 10 Done Sent Done false
12494 4314 Acked Read 4 Done Sent Done false
12495 4315 Acked Read 2 Done Sent Done false
12496 4316 Acked Read 3 Done Sent Done false
12497 4317 Acked Write 5 Done Sent Done false
12498 4318 Acked Write 9 Done Sent Done false
12499 STATES DS:0 DS:1 DS:2 TOTAL
12500 Sent 0 301 0 301
12501 Done 301 0 301 602
12502 Skipped 0 0 0 0
12503 Error 0 0 0 0
12504 Last Flush: 4303 4017 4303
12505 Downstairs last five completed: 4017 4016 4015 4014 4013
12506 Upstairs last five completed: 4318 4317 4316 4315 4314
12507 CLIENT: Up:0 ds:301 act:2
12508 [8820] got start action
12509 [8820] I am going to restart
12510 Starting downstairs at port 8820
12511 Make output file at "/tmp/dsc/downstairs-8820.txt"
12512 [8820][1] reports Starting
12513 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1215)
12514 [8820][1] reports Running
12515 ----------------------------------------------------------------
12516 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12517 States: Active Offline Active
12518 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12519 4018 Acked Write 4 Done Sent Done false
12520 4019 Acked Write 3 Done Sent Done false
12521 4020 Acked Flush 0 Done Sent Done false
12522 4021 Acked Write 4 Done Sent Done false
12523 4022 Acked Read 5 Done Sent Done false
12524 4023 Acked Write 1 Done Sent Done false
12525 4024 Acked Read 6 Done Sent Done false
12526 4025 Acked Read 9 Done Sent Done false
12527 4026 Acked Flush 0 Done Sent Done false
12528 4027 Acked Read 2 Done Sent Done false
12529 4028 Acked Flush 0 Done Sent Done false
12530 4029 Acked Write 4 Done Sent Done false
12531 4030 Acked Read 1 Done Sent Done false
12532 4031 Acked Write 5 Done Sent Done false
12533 4032 Acked Write 6 Done Sent Done false
12534 4033 Acked Read 7 Done Sent Done false
12535 4034 Acked Write 9 Done Sent Done false
12536 4035 Acked Write 3 Done Sent Done false
12537 4036 Acked Write 7 Done Sent Done false
12538 4037 Acked Read 5 Done Sent Done false
12539 4038 Acked Read 7 Done Sent Done false
12540 4039 Acked Read 7 Done Sent Done false
12541 4040 Acked Read 3 Done Sent Done false
12542 4041 Acked Read 10 Done Sent Done false
12543 4042 Acked Write 8 Done Sent Done false
12544 4043 Acked Read 10 Done Sent Done false
12545 4044 Acked Read 6 Done Sent Done false
12546 4045 Acked Write 10 Done Sent Done false
12547 4046 Acked Write 3 Done Sent Done false
12548 4047 Acked Write 2 Done Sent Done false
12549 4048 Acked Write 5 Done Sent Done false
12550 4049 Acked Write 4 Done Sent Done false
12551 4050 Acked Read 6 Done Sent Done false
12552 4051 Acked Write 5 Done Sent Done false
12553 4052 Acked Read 3 Done Sent Done false
12554 4053 Acked Read 9 Done Sent Done false
12555 4054 Acked Write 2 Done Sent Done false
12556 4055 Acked Read 1 Done Sent Done false
12557 4056 Acked Read 6 Done Sent Done false
12558 4057 Acked Read 7 Done Sent Done false
12559 4058 Acked Read 6 Done Sent Done false
12560 4059 Acked Flush 0 Done Sent Done false
12561 4060 Acked Write 5 Done Sent Done false
12562 4061 Acked Read 4 Done Sent Done false
12563 4062 Acked Read 4 Done Sent Done false
12564 4063 Acked Write 9 Done Sent Done false
12565 4064 Acked Read 2 Done Sent Done false
12566 4065 Acked Read 9 Done Sent Done false
12567 4066 Acked Write 7 Done Sent Done false
12568 4067 Acked Write 3 Done Sent Done false
12569 4068 Acked Flush 0 Done Sent Done false
12570 4069 Acked Flush 0 Done Sent Done false
12571 4070 Acked Read 3 Done Sent Done false
12572 4071 Acked Write 9 Done Sent Done false
12573 4072 Acked Read 5 Done Sent Done false
12574 4073 Acked Read 4 Done Sent Done false
12575 4074 Acked Write 9 Done Sent Done false
12576 4075 Acked Read 10 Done Sent Done false
12577 4076 Acked Read 2 Done Sent Done false
12578 4077 Acked Write 7 Done Sent Done false
12579 4078 Acked Read 2 Done Sent Done false
12580 4079 Acked Write 7 Done Sent Done false
12581 4080 Acked Write 8 Done Sent Done false
12582 4081 Acked Read 1 Done Sent Done false
12583 4082 Acked Read 4 Done Sent Done false
12584 4083 Acked Read 6 Done Sent Done false
12585 4084 Acked Write 10 Done Sent Done false
12586 4085 Acked Write 6 Done Sent Done false
12587 4086 Acked Write 8 Done Sent Done false
12588 4087 Acked Read 4 Done Sent Done false
12589 4088 Acked Write 8 Done Sent Done false
12590 4089 Acked Write 5 Done Sent Done false
12591 4090 Acked Write 5 Done Sent Done false
12592 4091 Acked Write 10 Done Sent Done false
12593 4092 Acked Read 9 Done Sent Done false
12594 4093 Acked Read 7 Done Sent Done false
12595 4094 Acked Write 5 Done Sent Done false
12596 4095 Acked Read 10 Done Sent Done false
12597 4096 Acked Write 3 Done Sent Done false
12598 4097 Acked Flush 0 Done Sent Done false
12599 4098 Acked Read 3 Done Sent Done false
12600 4099 Acked Read 8 Done Sent Done false
12601 4100 Acked Read 3 Done Sent Done false
12602 4101 Acked Write 7 Done Sent Done false
12603 4102 Acked Write 5 Done Sent Done false
12604 4103 Acked Read 4 Done Sent Done false
12605 4104 Acked Write 4 Done Sent Done false
12606 4105 Acked Read 7 Done Sent Done false
12607 4106 Acked Write 1 Done Sent Done false
12608 4107 Acked Write 1 Done Sent Done false
12609 4108 Acked Write 7 Done Sent Done false
12610 4109 Acked Write 3 Done Sent Done false
12611 4110 Acked Read 10 Done Sent Done false
12612 4111 Acked Read 6 Done Sent Done false
12613 4112 Acked Write 5 Done Sent Done false
12614 4113 Acked Write 7 Done Sent Done false
12615 4114 Acked Read 1 Done Sent Done false
12616 4115 Acked Read 2 Done Sent Done false
12617 4116 Acked Write 5 Done Sent Done false
12618 4117 Acked Write 2 Done Sent Done false
12619 4118 Acked Write 10 Done Sent Done false
12620 4119 Acked Read 6 Done Sent Done false
12621 4120 Acked Write 5 Done Sent Done false
12622 4121 Acked Read 8 Done Sent Done false
12623 4122 Acked Read 9 Done Sent Done false
12624 4123 Acked Read 8 Done Sent Done false
12625 4124 Acked Flush 0 Done Sent Done false
12626 4125 Acked Write 5 Done Sent Done false
12627 4126 Acked Flush 0 Done Sent Done false
12628 4127 Acked Read 4 Done Sent Done false
12629 4128 Acked Read 4 Done Sent Done false
12630 4129 Acked Write 8 Done Sent Done false
12631 4130 Acked Write 6 Done Sent Done false
12632 4131 Acked Write 6 Done Sent Done false
12633 4132 Acked Read 5 Done Sent Done false
12634 4133 Acked Flush 0 Done Sent Done false
12635 4134 Acked Flush 0 Done Sent Done false
12636 4135 Acked Read 6 Done Sent Done false
12637 4136 Acked Read 2 Done Sent Done false
12638 4137 Acked Read 8 Done Sent Done false
12639 4138 Acked Flush 0 Done Sent Done false
12640 4139 Acked Write 8 Done Sent Done false
12641 4140 Acked Read 4 Done Sent Done false
12642 4141 Acked Write 6 Done Sent Done false
12643 4142 Acked Read 10 Done Sent Done false
12644 4143 Acked Write 7 Done Sent Done false
12645 4144 Acked Write 7 Done Sent Done false
12646 4145 Acked Read 4 Done Sent Done false
12647 4146 Acked Read 2 Done Sent Done false
12648 4147 Acked Read 7 Done Sent Done false
12649 4148 Acked Write 8 Done Sent Done false
12650 4149 Acked Read 9 Done Sent Done false
12651 4150 Acked Write 5 Done Sent Done false
12652 4151 Acked Flush 0 Done Sent Done false
12653 4152 Acked Read 8 Done Sent Done false
12654 4153 Acked Read 3 Done Sent Done false
12655 4154 Acked Write 5 Done Sent Done false
12656 4155 Acked Read 8 Done Sent Done false
12657 4156 Acked Write 7 Done Sent Done false
12658 4157 Acked Write 4 Done Sent Done false
12659 4158 Acked Write 8 Done Sent Done false
12660 4159 Acked Read 2 Done Sent Done false
12661 4160 Acked Flush 0 Done Sent Done false
12662 4161 Acked Read 5 Done Sent Done false
12663 4162 Acked Read 2 Done Sent Done false
12664 4163 Acked Write 10 Done Sent Done false
12665 4164 Acked Read 5 Done Sent Done false
12666 4165 Acked Write 9 Done Sent Done false
12667 4166 Acked Write 2 Done Sent Done false
12668 4167 Acked Flush 0 Done Sent Done false
12669 4168 Acked Flush 0 Done Sent Done false
12670 4169 Acked Read 1 Done Sent Done false
12671 4170 Acked Write 5 Done Sent Done false
12672 4171 Acked Read 8 Done Sent Done false
12673 4172 Acked Write 2 Done Sent Done false
12674 4173 Acked Write 9 Done Sent Done false
12675 4174 Acked Write 2 Done Sent Done false
12676 4175 Acked Read 9 Done Sent Done false
12677 4176 Acked Read 3 Done Sent Done false
12678 4177 Acked Write 7 Done Sent Done false
12679 4178 Acked Write 6 Done Sent Done false
12680 4179 Acked Write 4 Done Sent Done false
12681 4180 Acked Write 4 Done Sent Done false
12682 4181 Acked Read 4 Done Sent Done false
12683 4182 Acked Read 9 Done Sent Done false
12684 4183 Acked Read 5 Done Sent Done false
12685 4184 Acked Read 2 Done Sent Done false
12686 4185 Acked Flush 0 Done Sent Done false
12687 4186 Acked Flush 0 Done Sent Done false
12688 4187 Acked Write 10 Done Sent Done false
12689 4188 Acked Write 10 Done Sent Done false
12690 4189 Acked Read 5 Done Sent Done false
12691 4190 Acked Read 9 Done Sent Done false
12692 4191 Acked Flush 0 Done Sent Done false
12693 4192 Acked Write 2 Done Sent Done false
12694 4193 Acked Read 4 Done Sent Done false
12695 4194 Acked Read 3 Done Sent Done false
12696 4195 Acked Read 3 Done Sent Done false
12697 4196 Acked Read 1 Done Sent Done false
12698 4197 Acked Write 2 Done Sent Done false
12699 4198 Acked Read 2 Done Sent Done false
12700 4199 Acked Read 5 Done Sent Done false
12701 4200 Acked Write 10 Done Sent Done false
12702 4201 Acked Write 3 Done Sent Done false
12703 4202 Acked Read 10 Done Sent Done false
12704 4203 Acked Flush 0 Done Sent Done false
12705 4204 Acked Read 7 Done Sent Done false
12706 4205 Acked Write 1 Done Sent Done false
12707 4206 Acked Read 6 Done Sent Done false
12708 4207 Acked Flush 0 Done Sent Done false
12709 4208 Acked Read 8 Done Sent Done false
12710 4209 Acked Read 4 Done Sent Done false
12711 4210 Acked Read 1 Done Sent Done false
12712 4211 Acked Read 7 Done Sent Done false
12713 4212 Acked Read 3 Done Sent Done false
12714 4213 Acked Write 1 Done Sent Done false
12715 4214 Acked Read 3 Done Sent Done false
12716 4215 Acked Flush 0 Done Sent Done false
12717 4216 Acked Read 1 Done Sent Done false
12718 4217 Acked Read 8 Done Sent Done false
12719 4218 Acked Read 5 Done Sent Done false
12720 4219 Acked Write 2 Done Sent Done false
12721 4220 Acked Write 4 Done Sent Done false
12722 4221 Acked Read 1 Done Sent Done false
12723 4222 Acked Read 10 Done Sent Done false
12724 4223 Acked Read 4 Done Sent Done false
12725 4224 Acked Read 7 Done Sent Done false
12726 4225 Acked Read 1 Done Sent Done false
12727 4226 Acked Write 5 Done Sent Done false
12728 4227 Acked Read 4 Done Sent Done false
12729 4228 Acked Read 1 Done Sent Done false
12730 4229 Acked Read 4 Done Sent Done false
12731 4230 Acked Write 4 Done Sent Done false
12732 4231 Acked Read 10 Done Sent Done false
12733 4232 Acked Read 7 Done Sent Done false
12734 4233 Acked Read 5 Done Sent Done false
12735 4234 Acked Write 5 Done Sent Done false
12736 4235 Acked Write 4 Done Sent Done false
12737 4236 Acked Read 8 Done Sent Done false
12738 4237 Acked Read 7 Done Sent Done false
12739 4238 Acked Flush 0 Done Sent Done false
12740 4239 Acked Read 2 Done Sent Done false
12741 4240 Acked Read 6 Done Sent Done false
12742 4241 Acked Read 1 Done Sent Done false
12743 4242 Acked Write 1 Done Sent Done false
12744 4243 Acked Write 10 Done Sent Done false
12745 4244 Acked Write 1 Done Sent Done false
12746 4245 Acked Read 10 Done Sent Done false
12747 4246 Acked Read 4 Done Sent Done false
12748 4247 Acked Read 7 Done Sent Done false
12749 4248 Acked Write 2 Done Sent Done false
12750 4249 Acked Write 1 Done Sent Done false
12751 4250 Acked Write 10 Done Sent Done false
12752 4251 Acked Read 5 Done Sent Done false
12753 4252 Acked Read 8 Done Sent Done false
12754 4253 Acked Flush 0 Done Sent Done false
12755 4254 Acked Flush 0 Done Sent Done false
12756 4255 Acked Read 2 Done Sent Done false
12757 4256 Acked Flush 0 Done Sent Done false
12758 4257 Acked Read 8 Done Sent Done false
12759 4258 Acked Read 4 Done Sent Done false
12760 4259 Acked Read 8 Done Sent Done false
12761 4260 Acked Read 10 Done Sent Done false
12762 4261 Acked Write 8 Done Sent Done false
12763 4262 Acked Read 7 Done Sent Done false
12764 4263 Acked Read 10 Done Sent Done false
12765 4264 Acked Write 4 Done Sent Done false
12766 4265 Acked Read 5 Done Sent Done false
12767 4266 Acked Write 2 Done Sent Done false
12768 4267 Acked Read 2 Done Sent Done false
12769 4268 Acked Read 5 Done Sent Done false
12770 4269 Acked Write 3 Done Sent Done false
12771 4270 Acked Write 1 Done Sent Done false
12772 4271 Acked Read 8 Done Sent Done false
12773 4272 Acked Flush 0 Done Sent Done false
12774 4273 Acked Write 4 Done Sent Done false
12775 4274 Acked Write 9 Done Sent Done false
12776 4275 Acked Read 3 Done Sent Done false
12777 4276 Acked Write 8 Done Sent Done false
12778 4277 Acked Write 4 Done Sent Done false
12779 4278 Acked Read 5 Done Sent Done false
12780 4279 Acked Read 9 Done Sent Done false
12781 4280 Acked Flush 0 Done Sent Done false
12782 4281 Acked Write 3 Done Sent Done false
12783 4282 Acked Write 9 Done Sent Done false
12784 4283 Acked Write 2 Done Sent Done false
12785 4284 Acked Read 7 Done Sent Done false
12786 4285 Acked Write 8 Done Sent Done false
12787 4286 Acked Write 10 Done Sent Done false
12788 4287 Acked Read 9 Done Sent Done false
12789 4288 Acked Write 1 Done Sent Done false
12790 4289 Acked Read 7 Done Sent Done false
12791 4290 Acked Read 6 Done Sent Done false
12792 4291 Acked Read 3 Done Sent Done false
12793 4292 Acked Write 9 Done Sent Done false
12794 4293 Acked Write 2 Done Sent Done false
12795 4294 Acked Write 9 Done Sent Done false
12796 4295 Acked Flush 0 Done Sent Done false
12797 4296 Acked Flush 0 Done Sent Done false
12798 4297 Acked Read 6 Done Sent Done false
12799 4298 Acked Read 6 Done Sent Done false
12800 4299 Acked Write 8 Done Sent Done false
12801 4300 Acked Flush 0 Done Sent Done false
12802 4301 Acked Read 3 Done Sent Done false
12803 4302 Acked Read 3 Done Sent Done false
12804 4303 Acked Flush 0 Done Sent Done false
12805 4304 Acked Write 8 Done Sent Done false
12806 4305 Acked Write 2 Done Sent Done false
12807 4306 Acked Read 3 Done Sent Done false
12808 4307 Acked Read 8 Done Sent Done false
12809 4308 Acked Read 1 Done Sent Done false
12810 4309 Acked Write 10 Done Sent Done false
12811 4310 Acked Write 4 Done Sent Done false
12812 4311 Acked Write 10 Done Sent Done false
12813 4312 Acked Read 5 Done Sent Done false
12814 4313 Acked Read 10 Done Sent Done false
12815 4314 Acked Read 4 Done Sent Done false
12816 4315 Acked Read 2 Done Sent Done false
12817 4316 Acked Read 3 Done Sent Done false
12818 4317 Acked Write 5 Done Sent Done false
12819 4318 Acked Write 9 Done Sent Done false
12820 4319 Acked Flush 0 Done Sent Done false
12821 STATES DS:0 DS:1 DS:2 TOTAL
12822 Sent 0 302 0 302
12823 Done 302 0 302 604
12824 Skipped 0 0 0 0
12825 Error 0 0 0 0
12826 Last Flush: 4319 4017 4319
12827 Downstairs last five completed: 4017 4016 4015 4014 4013
12828 Upstairs last five completed: 4319 4318 4317 4316 4315
12829 CLIENT: Up:0 ds:302 act:2
128302025-01-11T06:08:06.974ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128312025-01-11T06:08:06.974ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35449) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128322025-01-11T06:08:06.975ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128332025-01-11T06:08:06.976ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128342025-01-11T06:08:06.976ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128352025-01-11T06:08:06.976ZINFOcrucible: send last flush ID to this DS: 4017 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128362025-01-11T06:08:06.976ZINFOcrucible: Replied this last flush ID: 4017 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128372025-01-11T06:08:06.976ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128382025-01-11T06:08:06.976ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128392025-01-11T06:08:06.976ZINFOcrucible: [1] client re-new 302 jobs since flush 4017 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128402025-01-11T06:08:06.976ZINFOcrucible: [1] Replayed 302 jobs since flush: 4017 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
12841 ----------------------------------------------------------------
12842 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12843 Downstairs last five completed: 4319 4318 4317 4316 4315
12844 Upstairs last five completed: 4319 4318 4317 4316 4315
12845 CLIENT: Up:0 ds:0 act:3
12846 Replay: All jobs finished
12847 Main task has work to do, go find it
12848 got dsc Stop(0)
12849 stop 0
12850 [8810] Got stop action so:false kr:false
12851 Jan 11 06:08:09.220 INFO request completed, latency_us: 382, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 053c46cd-7739-44a3-ab1a-c879d9e04723, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
12852 Jan 11 06:08:09.221 INFO request completed, latency_us: 113, response_code: 200, uri: /state/cid/0, method: GET, req_id: 27f8347b-1ddb-463b-bb74-33701eaa1a6a, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
128532025-01-11T06:08:09.221ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128542025-01-11T06:08:09.221ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128552025-01-11T06:08:09.221ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128562025-01-11T06:08:09.221ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128572025-01-11T06:08:09.221ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
128582025-01-11T06:08:09.221ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
12859 [8810] Exited with: ExitStatus(unix_wait_status(9))
12860 [8810][0] reports Exit
12861 Jan 11 06:08:13.224 INFO request completed, latency_us: 361, response_code: 200, uri: /state/cid/0, method: GET, req_id: e1ca291a-8a6c-4dd0-b5a6-0a4f2bdd7ad5, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
12862 001/300 Flush
12863 002/300 Flush
12864 003/300 Read block 4922 len 20480
12865 004/300 Write block 3498 len 28672 data: 2 3 4 5 5 4 4
12866 005/300 Read block 3729 len 4096
12867 006/300 Write block 2280 len 12288 data: 3 5 5
12868 007/300 Flush
12869 008/300 Read block 1440 len 32768
12870 009/300 Read block 2623 len 16384
12871 010/300 Write block 3102 len 4096 data: 6
12872 011/300 Flush
12873 012/300 Write block 683 len 40960 data: 2 2 3 4 4 4 4 4 4 4
12874 013/300 Read block 1085 len 20480
12875 014/300 Flush
12876 015/300 Write block 2455 len 12288 data: 3 3 3
12877 016/300 Write block 4074 len 40960 data: 6 5 5 4 5 4 4 3 2 3
12878 017/300 Read block 1599 len 32768
12879 018/300 Flush
12880 019/300 Write block 1501 len 32768 data: 4 5 5 4 4 3 3 3
12881 020/300 Write block 1544 len 32768 data: 2 2 2 2 2 2 2 2
12882 021/300 Read block 654 len 32768
12883 022/300 Read block 1828 len 4096
12884 023/300 Read block 273 len 16384
12885 024/300 Flush
12886 025/300 Flush
12887 026/300 Read block 2013 len 28672
12888 027/300 Read block 522 len 36864
12889 028/300 Read block 2724 len 24576
12890 029/300 Write block 551 len 40960 data: 4 5 4 4 4 4 3 2 2 2
12891 030/300 Flush
12892 031/300 Read block 3023 len 32768
12893 032/300 Write block 2631 len 36864 data: 3 3 2 2 2 3 3 3 3
12894 033/300 Read block 3766 len 12288
12895 034/300 Write block 4408 len 40960 data: 4 5 5 5 5 4 4 3 2 2
12896 035/300 Write block 1819 len 8192 data: 2 2
12897 036/300 Write block 1772 len 12288 data: 2 3 4
12898 037/300 Read block 2096 len 16384
12899 038/300 Write block 1059 len 40960 data: 3 3 4 5 5 4 3 2 2 2
12900 039/300 Write block 114 len 28672 data: 2 2 2 3 3 3 3
12901 040/300 Write block 1696 len 32768 data: 2 2 2 2 4 4 4 4
12902 041/300 Read block 400 len 16384
12903 042/300 Write block 4422 len 28672 data: 3 3 2 3 3 4 4
12904 043/300 Read block 811 len 28672
12905 044/300 Read block 152 len 28672
12906 045/300 Write block 2274 len 4096 data: 4
12907 046/300 Read block 1193 len 16384
12908 047/300 Read block 2968 len 36864
12909 048/300 Write block 3589 len 40960 data: 5 5 6 6 5 5 5 5 7 7
12910 049/300 Flush
12911 050/300 Write block 4655 len 16384 data: 3 3 3 3
12912 051/300 Read block 289 len 36864
12913 052/300 Write block 354 len 20480 data: 3 4 4 4 4
12914 053/300 Write block 2312 len 32768 data: 3 3 3 3 2 2 2 2
12915 054/300 Write block 4739 len 36864 data: 3 3 4 3 3 3 3 4 4
12916 055/300 Read block 1856 len 4096
12917 056/300 Read block 3714 len 36864
12918 057/300 Write block 2192 len 8192 data: 3 2
12919 058/300 Write block 2623 len 32768 data: 3 3 3 3 4 4 3 3
12920 059/300 Write block 641 len 16384 data: 3 3 3 3
12921 060/300 Read block 4533 len 32768
12922 061/300 Read block 4399 len 24576
12923 062/300 Flush
12924 063/300 Write block 3946 len 28672 data: 3 2 4 4 4 4 4
12925 064/300 Read block 4296 len 24576
12926 065/300 Write block 2516 len 12288 data: 3 2 2
12927 066/300 Read block 3411 len 4096
12928 067/300 Write block 3512 len 40960 data: 4 5 5 5 5 3 3 2 2 2
12929 068/300 Write block 3910 len 28672 data: 4 4 4 3 2 2 2
12930 069/300 Write block 2624 len 24576 data: 4 4 4 5 5 4
12931 070/300 Flush
12932 071/300 Write block 101 len 24576 data: 4 4 2 2 2 4
12933 072/300 Read block 2313 len 40960
12934 073/300 Read block 903 len 32768
12935 074/300 Write block 256 len 32768 data: 3 3 2 2 2 2 2 2
12936 075/300 Read block 4308 len 8192
12937 076/300 Read block 721 len 36864
12938 077/300 Read block 253 len 8192
12939 078/300 Read block 887 len 12288
12940 079/300 Read block 2355 len 36864
12941 080/300 Read block 1488 len 4096
12942 081/300 Write block 3473 len 20480 data: 5 5 6 6 6
12943 082/300 Flush
12944 083/300 Write block 478 len 16384 data: 3 3 4 4
12945 084/300 Flush
12946 085/300 Flush
12947 086/300 Read block 2209 len 28672
12948 087/300 Read block 3658 len 32768
12949 088/300 Write block 498 len 28672 data: 5 5 5 5 5 5 5
12950 089/300 Read block 4739 len 40960
12951 090/300 Read block 3435 len 8192
12952 091/300 Write block 912 len 24576 data: 5 5 3 3 3 3
12953 092/300 Read block 3406 len 36864
12954 093/300 Flush
12955 094/300 Write block 2927 len 20480 data: 4 6 6 6 7
12956 095/300 Read block 406 len 24576
12957 096/300 Read block 2516 len 28672
12958 097/300 Write block 3000 len 40960 data: 3 3 3 3 5 5 5 5 4 4
12959 098/300 Write block 4673 len 8192 data: 5 4
12960 099/300 Read block 1858 len 32768
12961 100/300 Read block 2611 len 8192
12962 101/300 Write block 1610 len 4096 data: 2
12963 102/300 Read block 4183 len 36864
12964 103/300 Write block 945 len 20480 data: 2 3 3 4 4
12965 104/300 Read block 1451 len 16384
12966 105/300 Flush
12967 106/300 Read block 2415 len 12288
12968 107/300 Flush
12969 108/300 Write block 1932 len 28672 data: 6 5 3 4 4 4 4
12970 109/300 Write block 3369 len 16384 data: 6 5 4 2
12971 110/300 Flush
12972 111/300 Write block 241 len 24576 data: 5 6 5 4 4 4
12973 112/300 Read block 3373 len 36864
12974 113/300 Read block 4833 len 24576
12975 114/300 Flush
12976 115/300 Read block 1448 len 28672
12977 116/300 Flush
12978 117/300 Write block 4566 len 16384 data: 5 3 3 3
12979 118/300 Read block 3296 len 20480
12980 119/300 Write block 3830 len 20480 data: 3 3 3 3 2
12981 120/300 Read block 1830 len 24576
12982 121/300 Flush
12983 122/300 Write block 4819 len 28672 data: 3 3 3 4 3 3 4
12984 123/300 Read block 1655 len 28672
12985 124/300 Write block 3723 len 32768 data: 2 2 2 2 4 4 4 4
12986 125/300 Flush
12987 126/300 Write block 2827 len 36864 data: 3 3 3 3 3 3 3 3 3
12988 127/300 Write block 4674 len 4096 data: 5
12989 128/300 Write block 2450 len 16384 data: 3 2 2 2
12990 129/300 Read block 4104 len 36864
12991 130/300 Read block 1780 len 36864
12992 131/300 Read block 3681 len 36864
12993 132/300 Write block 4604 len 20480 data: 2 2 2 4 4
12994 133/300 Read block 3725 len 24576
12995 134/300 Write block 56 len 36864 data: 4 4 5 4 4 4 6 6 5
12996 135/300 Write block 2648 len 4096 data: 5
12997 136/300 Write block 1477 len 40960 data: 4 4 4 5 4 4 3 2 2 3
12998 137/300 Read block 3818 len 12288
12999 138/300 Write block 333 len 36864 data: 2 4 3 4 3 3 3 3 2
13000 139/300 Write block 4573 len 8192 data: 2 2
13001 140/300 Read block 4002 len 32768
13002 141/300 Read block 4787 len 40960
13003 142/300 Read block 4162 len 20480
13004 143/300 Write block 2967 len 32768 data: 3 4 4 4 4 4 4 4
13005 144/300 Read block 2501 len 28672
13006 145/300 Read block 3012 len 12288
13007 146/300 Read block 827 len 8192
13008 147/300 Read block 73 len 24576
13009 148/300 Write block 1348 len 36864 data: 2 2 2 2 3 3 3 3 4
13010 149/300 Write block 2232 len 28672 data: 8 6 4 4 4 3 3
13011 150/300 Read block 4269 len 20480
13012 151/300 Read block 4191 len 4096
13013 152/300 Read block 814 len 40960
13014 153/300 Write block 2813 len 32768 data: 4 4 5 4 4 6 6 6
13015 154/300 Write block 3923 len 12288 data: 2 2 2
13016 155/300 Read block 2269 len 32768
13017 156/300 Flush
13018 157/300 Flush
13019 158/300 Read block 4459 len 36864
13020 159/300 Read block 655 len 40960
13021 160/300 Write block 593 len 28672 data: 2 2 3 4 4 3 3
13022 161/300 Read block 3048 len 36864
13023 162/300 Write block 3177 len 8192 data: 3 4
13024 163/300 Write block 3505 len 16384 data: 6 6 6 5
13025 164/300 Write block 3941 len 4096 data: 4
13026 165/300 Write block 3661 len 12288 data: 4 4 4
13027 166/300 Write block 4263 len 12288 data: 5 6 5
13028 167/300 Write block 2132 len 12288 data: 4 4 3
13029 168/300 Read block 1231 len 24576
13030 169/300 Write block 4333 len 8192 data: 2 2
13031 170/300 Write block 4154 len 16384 data: 2 3 2 2
13032 171/300 Write block 4939 len 40960 data: 2 3 3 3 4 4 4 3 3 4
13033 172/300 Read block 4817 len 20480
13034 173/300 Read block 335 len 28672
13035 174/300 Write block 1214 len 16384 data: 3 3 3 3
13036 175/300 Read block 1490 len 4096
13037 176/300 Write block 1839 len 24576 data: 2 2 2 4 4 4
13038 177/300 Write block 3061 len 36864 data: 3 3 3 3 3 3 3 3 3
13039 178/300 Read block 4737 len 16384
13040 179/300 Read block 2119 len 24576
13041 180/300 Read block 3228 len 40960
13042 181/300 Read block 2320 len 32768
13043 182/300 Write block 1582 len 16384 data: 3 3 2 2
13044 183/300 Read block 3473 len 28672
13045 184/300 Write block 3157 len 28672 data: 3 3 3 2 3 3 3
13046 185/300 Write block 3210 len 8192 data: 6 6
13047 186/300 Read block 1820 len 4096
13048 187/300 Flush
13049 188/300 Read block 4163 len 28672
13050 189/300 Read block 373 len 36864
13051 190/300 Flush
13052 191/300 Read block 4431 len 8192
13053 192/300 Read block 2704 len 16384
13054 193/300 Write block 3033 len 20480 data: 4 3 3 4 7
13055 194/300 Read block 4957 len 28672
13056 195/300 Read block 2888 len 32768
13057 196/300 Write block 861 len 12288 data: 4 4 4
13058 197/300 Read block 2541 len 16384
13059 198/300 Read block 2967 len 8192
13060 199/300 Read block 1222 len 28672
13061 200/300 Read block 87 len 16384
13062 201/300 Read block 2933 len 16384
13063 202/300 Read block 1783 len 16384
13064 203/300 Write block 1698 len 12288 data: 3 3 5
13065 204/300 Read block 4251 len 32768
13066 205/300 Read block 1979 len 8192
13067 206/300 Read block 1593 len 8192
13068 207/300 Read block 3903 len 24576
13069 208/300 Read block 2113 len 28672
13070 209/300 Write block 840 len 24576 data: 4 3 4 4 4 4
13071 210/300 Read block 9 len 12288
13072 211/300 Read block 2943 len 8192
13073 212/300 Flush
13074 213/300 Write block 3588 len 28672 data: 5 6 6 7 7 6 6
13075 214/300 Flush
13076 215/300 Write block 2352 len 4096 data: 4
13077 216/300 Write block 4922 len 24576 data: 3 3 3 2 2 2
13078 217/300 Read block 934 len 36864
13079 218/300 Flush
13080 219/300 Read block 2099 len 12288
13081 220/300 Read block 2652 len 32768
13082 221/300 Read block 1811 len 28672
13083 222/300 Read block 3491 len 40960
13084 223/300 Write block 3140 len 4096 data: 2
13085 224/300 Write block 4515 len 12288 data: 3 3 3
13086 225/300 Flush
13087 226/300 Read block 4725 len 36864
13088 227/300 Write block 3258 len 4096 data: 4
13089 228/300 Read block 4552 len 32768
13090 229/300 Read block 2881 len 32768
13091 230/300 Write block 41 len 36864 data: 7 6 6 6 5 5 4 2 4
13092 231/300 Read block 4115 len 20480
13093 232/300 Write block 2581 len 8192 data: 3 2
13094 233/300 Write block 3646 len 16384 data: 2 2 2 2
13095 234/300 Read block 3509 len 40960
13096 235/300 Read block 3795 len 16384
13097 236/300 Write block 2543 len 36864 data: 6 4 3 4 6 6 6 4 3
13098 237/300 Write block 4133 len 20480 data: 4 5 5 5 5
13099 238/300 Read block 2700 len 24576
13100 239/300 Write block 2828 len 40960 data: 4 4 4 4 4 4 4 4 3 2
13101 240/300 Flush
13102 241/300 Read block 1586 len 40960
13103 242/300 Read block 3304 len 28672
13104 243/300 Flush
13105 244/300 Write block 2491 len 8192 data: 3 3
13106 245/300 Read block 4542 len 8192
13107 246/300 Write block 688 len 12288 data: 5 5 5
13108 247/300 Read block 3118 len 24576
13109 248/300 Read block 4526 len 12288
13110 249/300 Write block 1974 len 28672 data: 2 2 3 4 3 5 3
13111 250/300 Write block 3371 len 20480 data: 5 3 2 2 2
13112 251/300 Read block 1070 len 36864
13113 252/300 Write block 3381 len 12288 data: 3 3 3
13114 253/300 Write block 1078 len 8192 data: 2 2
13115 254/300 Write block 4841 len 4096 data: 4
13116 255/300 Read block 3856 len 36864
13117 256/300 Read block 4147 len 40960
13118 257/300 Flush
13119 258/300 Read block 1556 len 36864
13120 259/300 Write block 2083 len 16384 data: 8 7 5 3
13121 260/300 Read block 3073 len 40960
13122 261/300 Write block 173 len 8192 data: 3 3
13123 262/300 Read block 409 len 12288
13124 263/300 Read block 4746 len 40960
13125 264/300 Write block 4304 len 4096 data: 6
13126 265/300 Read block 1079 len 12288
13127 266/300 Write block 79 len 4096 data: 3
13128 267/300 Flush
13129 268/300 Read block 1325 len 20480
13130 269/300 Write block 1875 len 24576 data: 5 5 5 3 4 4
13131 270/300 Write block 1528 len 24576 data: 3 3 3 4 4 3
13132 271/300 Write block 4749 len 24576 data: 3 4 3 3 3 3
13133 272/300 Write block 2824 len 12288 data: 6 3 2
13134 273/300 Write block 841 len 24576 data: 4 5 5 5 5 5
13135 274/300 Write block 1989 len 24576 data: 3 4 5 4 6 5
13136 275/300 Write block 1113 len 12288 data: 3 4 4
13137 276/300 Write block 3772 len 8192 data: 4 4
13138 277/300 Write block 2771 len 20480 data: 3 4 4 4 4
13139 278/300 Flush
13140 279/300 Read block 4270 len 16384
13141 280/300 Read block 4553 len 4096
13142 281/300 Read block 711 len 24576
13143 282/300 Read block 2013 len 16384
13144 283/300 Write block 4580 len 12288 data: 4 4 4
13145 284/300 Read block 3570 len 40960
13146 285/300 Read block 914 len 12288
13147 286/300 Write block 1769 len 16384 data: 2 2 2 3
13148 287/300 Write block 726 len 32768 data: 5 5 5 4 3 2 2 2
13149 288/300 Write block 1422 len 36864 data: 3 4 4 4 4 4 4 4 4
13150 289/300 Write block 3469 len 8192 data: 2 2
13151 290/300 Write block 2951 len 20480 data: 5 4 4 3 3
13152 291/300 Read block 4163 len 40960
13153 292/300 Flush
13154 293/300 Write block 2418 len 24576 data: 2 2 3 3 3 3
13155 294/300 Read block 4272 len 16384
13156 295/300 Read block 3256 len 16384
13157 296/300 Read block 4996 len 12288
13158 297/300 Read block 1959 len 20480
13159 298/300 Read block 4320 len 12288
13160 299/300 Read block 4888 len 4096
13161 300/300 Write block 2188 len 8192 data: 5 5
13162 301/300 Read block 1811 len 16384
13163 Main task has work to do, go find it
13164 got dsc Start(0)
13165 start 0
13166 Jan 11 06:08:13.467 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/0, method: POST, req_id: 9384ddf4-e9d0-4c0a-8873-0158bac84861, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
13167 Replay: started 0, returned:Ok(())
13168 ----------------------------------------------------------------
13169 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13170 States: Offline Active Active
13171 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13172 4320 Acked Flush 0 Sent Done Done false
13173 4321 Acked Flush 0 Sent Done Done false
13174 4322 Acked Read 5 Sent Done Done false
13175 4323 Acked Write 7 Sent Done Done false
13176 4324 Acked Read 1 Sent Done Done false
13177 4325 Acked Write 3 Sent Done Done false
13178 4326 Acked Flush 0 Sent Done Done false
13179 4327 Acked Read 8 Sent Done Done false
13180 4328 Acked Read 4 Sent Done Done false
13181 4329 Acked Write 1 Sent Done Done false
13182 4330 Acked Flush 0 Sent Done Done false
13183 4331 Acked Write 10 Sent Done Done false
13184 4332 Acked Read 5 Sent Done Done false
13185 4333 Acked Flush 0 Sent Done Done false
13186 4334 Acked Write 3 Sent Done Done false
13187 4335 Acked Write 10 Sent Done Done false
13188 4336 Acked Read 8 Sent Done Done false
13189 4337 Acked Flush 0 Sent Done Done false
13190 4338 Acked Write 8 Sent Done Done false
13191 4339 Acked Write 8 Sent Done Done false
13192 4340 Acked Read 8 Sent Done Done false
13193 4341 Acked Read 1 Sent Done Done false
13194 4342 Acked Read 4 Sent Done Done false
13195 4343 Acked Flush 0 Sent Done Done false
13196 4344 Acked Flush 0 Sent Done Done false
13197 4345 Acked Read 7 Sent Done Done false
13198 4346 Acked Read 9 Sent Done Done false
13199 4347 Acked Read 6 Sent Done Done false
13200 4348 Acked Write 10 Sent Done Done false
13201 4349 Acked Flush 0 Sent Done Done false
13202 4350 Acked Read 8 Sent Done Done false
13203 4351 Acked Write 9 Sent Done Done false
13204 4352 Acked Read 3 Sent Done Done false
13205 4353 Acked Write 10 Sent Done Done false
13206 4354 Acked Write 2 Sent Done Done false
13207 4355 Acked Write 3 Sent Done Done false
13208 4356 Acked Read 4 Sent Done Done false
13209 4357 Acked Write 10 Sent Done Done false
13210 4358 Acked Write 7 Sent Done Done false
13211 4359 Acked Write 8 Sent Done Done false
13212 4360 Acked Read 4 Sent Done Done false
13213 4361 Acked Write 7 Sent Done Done false
13214 4362 Acked Read 7 Sent Done Done false
13215 4363 Acked Read 7 Sent Done Done false
13216 4364 Acked Write 1 Sent Done Done false
13217 4365 Acked Read 4 Sent Done Done false
13218 4366 Acked Read 9 Sent Done Done false
13219 4367 Acked Write 10 Sent Done Done false
13220 4368 Acked Flush 0 Sent Done Done false
13221 4369 Acked Write 4 Sent Done Done false
13222 4370 Acked Read 9 Sent Done Done false
13223 4371 Acked Write 5 Sent Done Done false
13224 4372 Acked Write 8 Sent Done Done false
13225 4373 Acked Write 9 Sent Done Done false
13226 4374 Acked Read 1 Sent Done Done false
13227 4375 Acked Read 9 Sent Done Done false
13228 4376 Acked Write 2 Sent Done Done false
13229 4377 Acked Write 8 Sent Done Done false
13230 4378 Acked Write 4 Sent Done Done false
13231 4379 Acked Read 8 Sent Done Done false
13232 4380 Acked Read 6 Sent Done Done false
13233 4381 Acked Flush 0 Sent Done Done false
13234 4382 Acked Write 7 Sent Done Done false
13235 4383 Acked Read 6 Sent Done Done false
13236 4384 Acked Write 3 Sent Done Done false
13237 4385 Acked Read 1 Sent Done Done false
13238 4386 Acked Write 10 Sent Done Done false
13239 4387 Acked Write 7 Sent Done Done false
13240 4388 Acked Write 6 Sent Done Done false
13241 4389 Acked Flush 0 Sent Done Done false
13242 4390 Acked Write 6 Sent Done Done false
13243 4391 Acked Read 10 Sent Done Done false
13244 4392 Acked Read 8 Sent Done Done false
13245 4393 Acked Write 8 Sent Done Done false
13246 4394 Acked Read 2 Sent Done Done false
13247 4395 Acked Read 9 Sent Done Done false
13248 4396 Acked Read 2 Sent Done Done false
13249 4397 Acked Read 3 Sent Done Done false
13250 4398 Acked Read 9 Sent Done Done false
13251 4399 Acked Read 1 Sent Done Done false
13252 4400 Acked Write 5 Sent Done Done false
13253 4401 Acked Flush 0 Sent Done Done false
13254 4402 Acked Write 4 Sent Done Done false
13255 4403 Acked Flush 0 Sent Done Done false
13256 4404 Acked Flush 0 Sent Done Done false
13257 4405 Acked Read 7 Sent Done Done false
13258 4406 Acked Read 8 Sent Done Done false
13259 4407 Acked Write 7 Sent Done Done false
13260 4408 Acked Read 10 Sent Done Done false
13261 4409 Acked Read 2 Sent Done Done false
13262 4410 Acked Write 6 Sent Done Done false
13263 4411 Acked Read 9 Sent Done Done false
13264 4412 Acked Flush 0 Sent Done Done false
13265 4413 Acked Write 5 Sent Done Done false
13266 4414 Acked Read 6 Sent Done Done false
13267 4415 Acked Read 7 Sent Done Done false
13268 4416 Acked Write 10 Sent Done Done false
13269 4417 Acked Write 2 Sent Done Done false
13270 4418 Acked Read 8 Sent Done Done false
13271 4419 Acked Read 2 Sent Done Done false
13272 4420 Acked Write 1 Sent Done Done false
13273 4421 Acked Read 9 Sent Done Done false
13274 4422 Acked Write 5 Sent Done Done false
13275 4423 Acked Read 4 Sent Done Done false
13276 4424 Acked Flush 0 Sent Done Done false
13277 4425 Acked Read 3 Sent Done Done false
13278 4426 Acked Flush 0 Sent Done Done false
13279 4427 Acked Write 7 Sent Done Done false
13280 4428 Acked Write 4 Sent Done Done false
13281 4429 Acked Flush 0 Sent Done Done false
13282 4430 Acked Write 6 Sent Done Done false
13283 4431 Acked Read 9 Sent Done Done false
13284 4432 Acked Read 6 Sent Done Done false
13285 4433 Acked Flush 0 Sent Done Done false
13286 4434 Acked Read 7 Sent Done Done false
13287 4435 Acked Flush 0 Sent Done Done false
13288 4436 Acked Write 4 Sent Done Done false
13289 4437 Acked Read 5 Sent Done Done false
13290 4438 Acked Write 5 Sent Done Done false
13291 4439 Acked Read 6 Sent Done Done false
13292 4440 Acked Flush 0 Sent Done Done false
13293 4441 Acked Write 7 Sent Done Done false
13294 4442 Acked Read 7 Sent Done Done false
13295 4443 Acked Write 8 Sent Done Done false
13296 4444 Acked Flush 0 Sent Done Done false
13297 4445 Acked Write 9 Sent Done Done false
13298 4446 Acked Write 1 Sent Done Done false
13299 4447 Acked Write 4 Sent Done Done false
13300 4448 Acked Read 9 Sent Done Done false
13301 4449 Acked Read 9 Sent Done Done false
13302 4450 Acked Read 9 Sent Done Done false
13303 4451 Acked Write 5 Sent Done Done false
13304 4452 Acked Read 6 Sent Done Done false
13305 4453 Acked Write 9 Sent Done Done false
13306 4454 Acked Write 1 Sent Done Done false
13307 4455 Acked Write 10 Sent Done Done false
13308 4456 Acked Read 3 Sent Done Done false
13309 4457 Acked Write 9 Sent Done Done false
13310 4458 Acked Write 2 Sent Done Done false
13311 4459 Acked Read 8 Sent Done Done false
13312 4460 Acked Read 10 Sent Done Done false
13313 4461 Acked Read 5 Sent Done Done false
13314 4462 Acked Write 8 Sent Done Done false
13315 4463 Acked Read 7 Sent Done Done false
13316 4464 Acked Read 3 Sent Done Done false
13317 4465 Acked Read 2 Sent Done Done false
13318 4466 Acked Read 6 Sent Done Done false
13319 4467 Acked Write 9 Sent Done Done false
13320 4468 Acked Write 7 Sent Done Done false
13321 4469 Acked Read 5 Sent Done Done false
13322 4470 Acked Read 1 Sent Done Done false
13323 4471 Acked Read 10 Sent Done Done false
13324 4472 Acked Write 8 Sent Done Done false
13325 4473 Acked Write 3 Sent Done Done false
13326 4474 Acked Read 8 Sent Done Done false
13327 4475 Acked Flush 0 Sent Done Done false
13328 4476 Acked Flush 0 Sent Done Done false
13329 4477 Acked Read 9 Sent Done Done false
13330 4478 Acked Read 10 Sent Done Done false
13331 4479 Acked Write 7 Sent Done Done false
13332 4480 Acked Read 9 Sent Done Done false
13333 4481 Acked Write 2 Sent Done Done false
13334 4482 Acked Write 4 Sent Done Done false
13335 4483 Acked Write 1 Sent Done Done false
13336 4484 Acked Write 3 Sent Done Done false
13337 4485 Acked Write 3 Sent Done Done false
13338 4486 Acked Write 3 Sent Done Done false
13339 4487 Acked Read 6 Sent Done Done false
13340 4488 Acked Write 2 Sent Done Done false
13341 4489 Acked Write 4 Sent Done Done false
13342 4490 Acked Write 10 Sent Done Done false
13343 4491 Acked Read 5 Sent Done Done false
13344 4492 Acked Read 7 Sent Done Done false
13345 4493 Acked Write 4 Sent Done Done false
13346 4494 Acked Read 1 Sent Done Done false
13347 4495 Acked Write 6 Sent Done Done false
13348 4496 Acked Write 9 Sent Done Done false
13349 4497 Acked Read 4 Sent Done Done false
13350 4498 Acked Read 6 Sent Done Done false
13351 4499 Acked Read 10 Sent Done Done false
13352 4500 Acked Read 8 Sent Done Done false
13353 4501 Acked Write 4 Sent Done Done false
13354 4502 Acked Read 7 Sent Done Done false
13355 4503 Acked Write 7 Sent Done Done false
13356 4504 Acked Write 2 Sent Done Done false
13357 4505 Acked Read 1 Sent Done Done false
13358 4506 Acked Flush 0 Sent Done Done false
13359 4507 Acked Read 7 Sent Done Done false
13360 4508 Acked Read 9 Sent Done Done false
13361 4509 Acked Flush 0 Sent Done Done false
13362 4510 Acked Read 2 Sent Done Done false
13363 4511 Acked Read 4 Sent Done Done false
13364 4512 Acked Write 5 Sent Done Done false
13365 4513 Acked Read 7 Sent Done Done false
13366 4514 Acked Read 8 Sent Done Done false
13367 4515 Acked Write 3 Sent Done Done false
13368 4516 Acked Read 4 Sent Done Done false
13369 4517 Acked Read 2 Sent Done Done false
13370 4518 Acked Read 7 Sent Done Done false
13371 4519 Acked Read 4 Sent Done Done false
13372 4520 Acked Read 4 Sent Done Done false
13373 4521 Acked Read 4 Sent Done Done false
13374 4522 Acked Write 3 Sent Done Done false
13375 4523 Acked Read 8 Sent Done Done false
13376 4524 Acked Read 2 Sent Done Done false
13377 4525 Acked Read 2 Sent Done Done false
13378 4526 Acked Read 6 Sent Done Done false
13379 4527 Acked Read 7 Sent Done Done false
13380 4528 Acked Write 6 Sent Done Done false
13381 4529 Acked Read 3 Sent Done Done false
13382 4530 Acked Read 2 Sent Done Done false
13383 4531 Acked Flush 0 Sent Done Done false
13384 4532 Acked Write 7 Sent Done Done false
13385 4533 Acked Flush 0 Sent Done Done false
13386 4534 Acked Write 1 Sent Done Done false
13387 4535 Acked Write 6 Sent Done Done false
13388 4536 Acked Read 9 Sent Done Done false
13389 4537 Acked Flush 0 Sent Done Done false
13390 4538 Acked Read 3 Sent Done Done false
13391 4539 Acked Read 8 Sent Done Done false
13392 4540 Acked Read 7 Sent Done Done false
13393 4541 Acked Read 10 Sent Done Done false
13394 4542 Acked Write 1 Sent Done Done false
13395 4543 Acked Write 3 Sent Done Done false
13396 4544 Acked Flush 0 Sent Done Done false
13397 4545 Acked Read 9 Sent Done Done false
13398 4546 Acked Write 1 Sent Done Done false
13399 4547 Acked Read 8 Sent Done Done false
13400 4548 Acked Read 8 Sent Done Done false
13401 4549 Acked Write 9 Sent Done Done false
13402 4550 Acked Read 5 Sent Done Done false
13403 4551 Acked Write 2 Sent Done Done false
13404 4552 Acked Write 4 Sent Done Done false
13405 4553 Acked Read 10 Sent Done Done false
13406 4554 Acked Read 4 Sent Done Done false
13407 4555 Acked Write 9 Sent Done Done false
13408 4556 Acked Write 5 Sent Done Done false
13409 4557 Acked Read 6 Sent Done Done false
13410 4558 Acked Write 10 Sent Done Done false
13411 4559 Acked Flush 0 Sent Done Done false
13412 4560 Acked Read 10 Sent Done Done false
13413 4561 Acked Read 7 Sent Done Done false
13414 4562 Acked Flush 0 Sent Done Done false
13415 4563 Acked Write 2 Sent Done Done false
13416 4564 Acked Read 2 Sent Done Done false
13417 4565 Acked Write 3 Sent Done Done false
13418 4566 Acked Read 6 Sent Done Done false
13419 4567 Acked Read 3 Sent Done Done false
13420 4568 Acked Write 7 Sent Done Done false
13421 4569 Acked Write 5 Sent Done Done false
13422 4570 Acked Read 9 Sent Done Done false
13423 4571 Acked Write 3 Sent Done Done false
13424 4572 Acked Write 2 Sent Done Done false
13425 4573 Acked Write 1 Sent Done Done false
13426 4574 Acked Read 9 Sent Done Done false
13427 4575 Acked Read 10 Sent Done Done false
13428 4576 Acked Flush 0 Sent Done Done false
13429 4577 Acked Read 9 Sent Done Done false
13430 4578 Acked Write 4 Sent Done Done false
13431 4579 Acked Read 10 Sent Done Done false
13432 4580 Acked Write 2 Sent Done Done false
13433 4581 Acked Read 3 Sent Done Done false
13434 4582 Acked Read 10 Sent Done Done false
13435 4583 Acked Write 1 Sent Done Done false
13436 4584 Acked Read 3 Sent Done Done false
13437 4585 Acked Write 1 Sent Done Done false
13438 4586 Acked Flush 0 Sent Done Done false
13439 4587 Acked Read 5 Sent Done Done false
13440 4588 Acked Write 6 Sent Done Done false
13441 4589 Acked Write 6 Sent Done Done false
13442 4590 Acked Write 6 Sent Done Done false
13443 4591 Acked Write 3 Sent Done Done false
13444 4592 Acked Write 6 Sent Done Done false
13445 4593 Acked Write 6 Sent Done Done false
13446 4594 Acked Write 3 Sent Done Done false
13447 4595 Acked Write 2 Sent Done Done false
13448 4596 Acked Write 5 Sent Done Done false
13449 4597 Acked Flush 0 Sent Done Done false
13450 4598 Acked Read 4 Sent Done Done false
13451 4599 Acked Read 1 Sent Done Done false
13452 4600 Acked Read 6 Sent Done Done false
13453 4601 Acked Read 4 Sent Done Done false
13454 4602 Acked Write 3 Sent Done Done false
13455 4603 Acked Read 10 Sent Done Done false
13456 4604 Acked Read 3 Sent Done Done false
13457 4605 Acked Write 4 Sent Done Done false
13458 4606 Acked Write 8 Sent Done Done false
13459 4607 Acked Write 9 Sent Done Done false
13460 4608 Acked Write 2 Sent Done Done false
13461 4609 Acked Write 5 Sent Done Done false
13462 4610 Acked Read 10 Sent Done Done false
13463 4611 Acked Flush 0 Sent Done Done false
13464 4612 Acked Write 6 Sent Done Done false
13465 4613 Acked Read 4 Sent Done Done false
13466 4614 Acked Read 4 Sent Done Done false
13467 4615 Acked Read 3 Sent Done Done false
13468 4616 Acked Read 5 Sent Done Done false
13469 4617 Acked Read 3 Sent Done Done false
13470 4618 Acked Read 1 Sent Done Done false
13471 4619 Acked Write 2 Sent Done Done false
13472 4620 Acked Read 4 Sent Done Done false
13473 STATES DS:0 DS:1 DS:2 TOTAL
13474 Sent 301 0 0 301
13475 Done 0 301 301 602
13476 Skipped 0 0 0 0
13477 Error 0 0 0 0
13478 Last Flush: 4319 4611 4611
13479 Downstairs last five completed: 4319 4318 4317 4316 4315
13480 Upstairs last five completed: 4620 4619 4618 4617 4616
13481 CLIENT: Up:0 ds:301 act:2
13482 [8810] got start action
13483 [8810] I am going to restart
13484 Starting downstairs at port 8810
13485 Make output file at "/tmp/dsc/downstairs-8810.txt"
13486 [8810][0] reports Starting
13487 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1216)
13488 [8810][0] reports Running
13489 ----------------------------------------------------------------
13490 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13491 States: Offline Active Active
13492 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13493 4320 Acked Flush 0 Sent Done Done false
13494 4321 Acked Flush 0 Sent Done Done false
13495 4322 Acked Read 5 Sent Done Done false
13496 4323 Acked Write 7 Sent Done Done false
13497 4324 Acked Read 1 Sent Done Done false
13498 4325 Acked Write 3 Sent Done Done false
13499 4326 Acked Flush 0 Sent Done Done false
13500 4327 Acked Read 8 Sent Done Done false
13501 4328 Acked Read 4 Sent Done Done false
13502 4329 Acked Write 1 Sent Done Done false
13503 4330 Acked Flush 0 Sent Done Done false
13504 4331 Acked Write 10 Sent Done Done false
13505 4332 Acked Read 5 Sent Done Done false
13506 4333 Acked Flush 0 Sent Done Done false
13507 4334 Acked Write 3 Sent Done Done false
13508 4335 Acked Write 10 Sent Done Done false
13509 4336 Acked Read 8 Sent Done Done false
13510 4337 Acked Flush 0 Sent Done Done false
13511 4338 Acked Write 8 Sent Done Done false
13512 4339 Acked Write 8 Sent Done Done false
13513 4340 Acked Read 8 Sent Done Done false
13514 4341 Acked Read 1 Sent Done Done false
13515 4342 Acked Read 4 Sent Done Done false
13516 4343 Acked Flush 0 Sent Done Done false
13517 4344 Acked Flush 0 Sent Done Done false
13518 4345 Acked Read 7 Sent Done Done false
13519 4346 Acked Read 9 Sent Done Done false
13520 4347 Acked Read 6 Sent Done Done false
13521 4348 Acked Write 10 Sent Done Done false
13522 4349 Acked Flush 0 Sent Done Done false
13523 4350 Acked Read 8 Sent Done Done false
13524 4351 Acked Write 9 Sent Done Done false
13525 4352 Acked Read 3 Sent Done Done false
13526 4353 Acked Write 10 Sent Done Done false
13527 4354 Acked Write 2 Sent Done Done false
13528 4355 Acked Write 3 Sent Done Done false
13529 4356 Acked Read 4 Sent Done Done false
13530 4357 Acked Write 10 Sent Done Done false
13531 4358 Acked Write 7 Sent Done Done false
13532 4359 Acked Write 8 Sent Done Done false
13533 4360 Acked Read 4 Sent Done Done false
13534 4361 Acked Write 7 Sent Done Done false
13535 4362 Acked Read 7 Sent Done Done false
13536 4363 Acked Read 7 Sent Done Done false
13537 4364 Acked Write 1 Sent Done Done false
13538 4365 Acked Read 4 Sent Done Done false
13539 4366 Acked Read 9 Sent Done Done false
13540 4367 Acked Write 10 Sent Done Done false
13541 4368 Acked Flush 0 Sent Done Done false
13542 4369 Acked Write 4 Sent Done Done false
13543 4370 Acked Read 9 Sent Done Done false
13544 4371 Acked Write 5 Sent Done Done false
13545 4372 Acked Write 8 Sent Done Done false
13546 4373 Acked Write 9 Sent Done Done false
13547 4374 Acked Read 1 Sent Done Done false
13548 4375 Acked Read 9 Sent Done Done false
13549 4376 Acked Write 2 Sent Done Done false
13550 4377 Acked Write 8 Sent Done Done false
13551 4378 Acked Write 4 Sent Done Done false
13552 4379 Acked Read 8 Sent Done Done false
13553 4380 Acked Read 6 Sent Done Done false
13554 4381 Acked Flush 0 Sent Done Done false
13555 4382 Acked Write 7 Sent Done Done false
13556 4383 Acked Read 6 Sent Done Done false
13557 4384 Acked Write 3 Sent Done Done false
13558 4385 Acked Read 1 Sent Done Done false
13559 4386 Acked Write 10 Sent Done Done false
13560 4387 Acked Write 7 Sent Done Done false
13561 4388 Acked Write 6 Sent Done Done false
13562 4389 Acked Flush 0 Sent Done Done false
13563 4390 Acked Write 6 Sent Done Done false
13564 4391 Acked Read 10 Sent Done Done false
13565 4392 Acked Read 8 Sent Done Done false
13566 4393 Acked Write 8 Sent Done Done false
13567 4394 Acked Read 2 Sent Done Done false
13568 4395 Acked Read 9 Sent Done Done false
13569 4396 Acked Read 2 Sent Done Done false
13570 4397 Acked Read 3 Sent Done Done false
13571 4398 Acked Read 9 Sent Done Done false
13572 4399 Acked Read 1 Sent Done Done false
13573 4400 Acked Write 5 Sent Done Done false
13574 4401 Acked Flush 0 Sent Done Done false
13575 4402 Acked Write 4 Sent Done Done false
13576 4403 Acked Flush 0 Sent Done Done false
13577 4404 Acked Flush 0 Sent Done Done false
13578 4405 Acked Read 7 Sent Done Done false
13579 4406 Acked Read 8 Sent Done Done false
13580 4407 Acked Write 7 Sent Done Done false
13581 4408 Acked Read 10 Sent Done Done false
13582 4409 Acked Read 2 Sent Done Done false
13583 4410 Acked Write 6 Sent Done Done false
13584 4411 Acked Read 9 Sent Done Done false
13585 4412 Acked Flush 0 Sent Done Done false
13586 4413 Acked Write 5 Sent Done Done false
13587 4414 Acked Read 6 Sent Done Done false
13588 4415 Acked Read 7 Sent Done Done false
13589 4416 Acked Write 10 Sent Done Done false
13590 4417 Acked Write 2 Sent Done Done false
13591 4418 Acked Read 8 Sent Done Done false
13592 4419 Acked Read 2 Sent Done Done false
13593 4420 Acked Write 1 Sent Done Done false
13594 4421 Acked Read 9 Sent Done Done false
13595 4422 Acked Write 5 Sent Done Done false
13596 4423 Acked Read 4 Sent Done Done false
13597 4424 Acked Flush 0 Sent Done Done false
13598 4425 Acked Read 3 Sent Done Done false
13599 4426 Acked Flush 0 Sent Done Done false
13600 4427 Acked Write 7 Sent Done Done false
13601 4428 Acked Write 4 Sent Done Done false
13602 4429 Acked Flush 0 Sent Done Done false
13603 4430 Acked Write 6 Sent Done Done false
13604 4431 Acked Read 9 Sent Done Done false
13605 4432 Acked Read 6 Sent Done Done false
13606 4433 Acked Flush 0 Sent Done Done false
13607 4434 Acked Read 7 Sent Done Done false
13608 4435 Acked Flush 0 Sent Done Done false
13609 4436 Acked Write 4 Sent Done Done false
13610 4437 Acked Read 5 Sent Done Done false
13611 4438 Acked Write 5 Sent Done Done false
13612 4439 Acked Read 6 Sent Done Done false
13613 4440 Acked Flush 0 Sent Done Done false
13614 4441 Acked Write 7 Sent Done Done false
13615 4442 Acked Read 7 Sent Done Done false
13616 4443 Acked Write 8 Sent Done Done false
13617 4444 Acked Flush 0 Sent Done Done false
13618 4445 Acked Write 9 Sent Done Done false
13619 4446 Acked Write 1 Sent Done Done false
13620 4447 Acked Write 4 Sent Done Done false
13621 4448 Acked Read 9 Sent Done Done false
13622 4449 Acked Read 9 Sent Done Done false
13623 4450 Acked Read 9 Sent Done Done false
13624 4451 Acked Write 5 Sent Done Done false
13625 4452 Acked Read 6 Sent Done Done false
13626 4453 Acked Write 9 Sent Done Done false
13627 4454 Acked Write 1 Sent Done Done false
13628 4455 Acked Write 10 Sent Done Done false
13629 4456 Acked Read 3 Sent Done Done false
13630 4457 Acked Write 9 Sent Done Done false
13631 4458 Acked Write 2 Sent Done Done false
13632 4459 Acked Read 8 Sent Done Done false
13633 4460 Acked Read 10 Sent Done Done false
13634 4461 Acked Read 5 Sent Done Done false
13635 4462 Acked Write 8 Sent Done Done false
13636 4463 Acked Read 7 Sent Done Done false
13637 4464 Acked Read 3 Sent Done Done false
13638 4465 Acked Read 2 Sent Done Done false
13639 4466 Acked Read 6 Sent Done Done false
13640 4467 Acked Write 9 Sent Done Done false
13641 4468 Acked Write 7 Sent Done Done false
13642 4469 Acked Read 5 Sent Done Done false
13643 4470 Acked Read 1 Sent Done Done false
13644 4471 Acked Read 10 Sent Done Done false
13645 4472 Acked Write 8 Sent Done Done false
13646 4473 Acked Write 3 Sent Done Done false
13647 4474 Acked Read 8 Sent Done Done false
13648 4475 Acked Flush 0 Sent Done Done false
13649 4476 Acked Flush 0 Sent Done Done false
13650 4477 Acked Read 9 Sent Done Done false
13651 4478 Acked Read 10 Sent Done Done false
13652 4479 Acked Write 7 Sent Done Done false
13653 4480 Acked Read 9 Sent Done Done false
13654 4481 Acked Write 2 Sent Done Done false
13655 4482 Acked Write 4 Sent Done Done false
13656 4483 Acked Write 1 Sent Done Done false
13657 4484 Acked Write 3 Sent Done Done false
13658 4485 Acked Write 3 Sent Done Done false
13659 4486 Acked Write 3 Sent Done Done false
13660 4487 Acked Read 6 Sent Done Done false
13661 4488 Acked Write 2 Sent Done Done false
13662 4489 Acked Write 4 Sent Done Done false
13663 4490 Acked Write 10 Sent Done Done false
13664 4491 Acked Read 5 Sent Done Done false
13665 4492 Acked Read 7 Sent Done Done false
13666 4493 Acked Write 4 Sent Done Done false
13667 4494 Acked Read 1 Sent Done Done false
13668 4495 Acked Write 6 Sent Done Done false
13669 4496 Acked Write 9 Sent Done Done false
13670 4497 Acked Read 4 Sent Done Done false
13671 4498 Acked Read 6 Sent Done Done false
13672 4499 Acked Read 10 Sent Done Done false
13673 4500 Acked Read 8 Sent Done Done false
13674 4501 Acked Write 4 Sent Done Done false
13675 4502 Acked Read 7 Sent Done Done false
13676 4503 Acked Write 7 Sent Done Done false
13677 4504 Acked Write 2 Sent Done Done false
13678 4505 Acked Read 1 Sent Done Done false
13679 4506 Acked Flush 0 Sent Done Done false
13680 4507 Acked Read 7 Sent Done Done false
13681 4508 Acked Read 9 Sent Done Done false
13682 4509 Acked Flush 0 Sent Done Done false
13683 4510 Acked Read 2 Sent Done Done false
13684 4511 Acked Read 4 Sent Done Done false
13685 4512 Acked Write 5 Sent Done Done false
13686 4513 Acked Read 7 Sent Done Done false
13687 4514 Acked Read 8 Sent Done Done false
13688 4515 Acked Write 3 Sent Done Done false
13689 4516 Acked Read 4 Sent Done Done false
13690 4517 Acked Read 2 Sent Done Done false
13691 4518 Acked Read 7 Sent Done Done false
13692 4519 Acked Read 4 Sent Done Done false
13693 4520 Acked Read 4 Sent Done Done false
13694 4521 Acked Read 4 Sent Done Done false
13695 4522 Acked Write 3 Sent Done Done false
13696 4523 Acked Read 8 Sent Done Done false
13697 4524 Acked Read 2 Sent Done Done false
13698 4525 Acked Read 2 Sent Done Done false
13699 4526 Acked Read 6 Sent Done Done false
13700 4527 Acked Read 7 Sent Done Done false
13701 4528 Acked Write 6 Sent Done Done false
13702 4529 Acked Read 3 Sent Done Done false
13703 4530 Acked Read 2 Sent Done Done false
13704 4531 Acked Flush 0 Sent Done Done false
13705 4532 Acked Write 7 Sent Done Done false
13706 4533 Acked Flush 0 Sent Done Done false
13707 4534 Acked Write 1 Sent Done Done false
13708 4535 Acked Write 6 Sent Done Done false
13709 4536 Acked Read 9 Sent Done Done false
13710 4537 Acked Flush 0 Sent Done Done false
13711 4538 Acked Read 3 Sent Done Done false
13712 4539 Acked Read 8 Sent Done Done false
13713 4540 Acked Read 7 Sent Done Done false
13714 4541 Acked Read 10 Sent Done Done false
13715 4542 Acked Write 1 Sent Done Done false
13716 4543 Acked Write 3 Sent Done Done false
13717 4544 Acked Flush 0 Sent Done Done false
13718 4545 Acked Read 9 Sent Done Done false
13719 4546 Acked Write 1 Sent Done Done false
13720 4547 Acked Read 8 Sent Done Done false
13721 4548 Acked Read 8 Sent Done Done false
13722 4549 Acked Write 9 Sent Done Done false
13723 4550 Acked Read 5 Sent Done Done false
13724 4551 Acked Write 2 Sent Done Done false
13725 4552 Acked Write 4 Sent Done Done false
13726 4553 Acked Read 10 Sent Done Done false
13727 4554 Acked Read 4 Sent Done Done false
13728 4555 Acked Write 9 Sent Done Done false
13729 4556 Acked Write 5 Sent Done Done false
13730 4557 Acked Read 6 Sent Done Done false
13731 4558 Acked Write 10 Sent Done Done false
13732 4559 Acked Flush 0 Sent Done Done false
13733 4560 Acked Read 10 Sent Done Done false
13734 4561 Acked Read 7 Sent Done Done false
13735 4562 Acked Flush 0 Sent Done Done false
13736 4563 Acked Write 2 Sent Done Done false
13737 4564 Acked Read 2 Sent Done Done false
13738 4565 Acked Write 3 Sent Done Done false
13739 4566 Acked Read 6 Sent Done Done false
13740 4567 Acked Read 3 Sent Done Done false
13741 4568 Acked Write 7 Sent Done Done false
13742 4569 Acked Write 5 Sent Done Done false
13743 4570 Acked Read 9 Sent Done Done false
13744 4571 Acked Write 3 Sent Done Done false
13745 4572 Acked Write 2 Sent Done Done false
13746 4573 Acked Write 1 Sent Done Done false
13747 4574 Acked Read 9 Sent Done Done false
13748 4575 Acked Read 10 Sent Done Done false
13749 4576 Acked Flush 0 Sent Done Done false
13750 4577 Acked Read 9 Sent Done Done false
13751 4578 Acked Write 4 Sent Done Done false
13752 4579 Acked Read 10 Sent Done Done false
13753 4580 Acked Write 2 Sent Done Done false
13754 4581 Acked Read 3 Sent Done Done false
13755 4582 Acked Read 10 Sent Done Done false
13756 4583 Acked Write 1 Sent Done Done false
13757 4584 Acked Read 3 Sent Done Done false
13758 4585 Acked Write 1 Sent Done Done false
13759 4586 Acked Flush 0 Sent Done Done false
13760 4587 Acked Read 5 Sent Done Done false
13761 4588 Acked Write 6 Sent Done Done false
13762 4589 Acked Write 6 Sent Done Done false
13763 4590 Acked Write 6 Sent Done Done false
13764 4591 Acked Write 3 Sent Done Done false
13765 4592 Acked Write 6 Sent Done Done false
13766 4593 Acked Write 6 Sent Done Done false
13767 4594 Acked Write 3 Sent Done Done false
13768 4595 Acked Write 2 Sent Done Done false
13769 4596 Acked Write 5 Sent Done Done false
13770 4597 Acked Flush 0 Sent Done Done false
13771 4598 Acked Read 4 Sent Done Done false
13772 4599 Acked Read 1 Sent Done Done false
13773 4600 Acked Read 6 Sent Done Done false
13774 4601 Acked Read 4 Sent Done Done false
13775 4602 Acked Write 3 Sent Done Done false
13776 4603 Acked Read 10 Sent Done Done false
13777 4604 Acked Read 3 Sent Done Done false
13778 4605 Acked Write 4 Sent Done Done false
13779 4606 Acked Write 8 Sent Done Done false
13780 4607 Acked Write 9 Sent Done Done false
13781 4608 Acked Write 2 Sent Done Done false
13782 4609 Acked Write 5 Sent Done Done false
13783 4610 Acked Read 10 Sent Done Done false
13784 4611 Acked Flush 0 Sent Done Done false
13785 4612 Acked Write 6 Sent Done Done false
13786 4613 Acked Read 4 Sent Done Done false
13787 4614 Acked Read 4 Sent Done Done false
13788 4615 Acked Read 3 Sent Done Done false
13789 4616 Acked Read 5 Sent Done Done false
13790 4617 Acked Read 3 Sent Done Done false
13791 4618 Acked Read 1 Sent Done Done false
13792 4619 Acked Write 2 Sent Done Done false
13793 4620 Acked Read 4 Sent Done Done false
13794 4621 Acked Flush 0 Sent Done Done false
13795 STATES DS:0 DS:1 DS:2 TOTAL
13796 Sent 302 0 0 302
13797 Done 0 302 302 604
13798 Skipped 0 0 0 0
13799 Error 0 0 0 0
13800 Last Flush: 4319 4621 4621
13801 Downstairs last five completed: 4319 4318 4317 4316 4315
13802 Upstairs last five completed: 4621 4620 4619 4618 4617
13803 CLIENT: Up:0 ds:302 act:2
138042025-01-11T06:08:19.223ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138052025-01-11T06:08:19.223ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36235) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138062025-01-11T06:08:19.225ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138072025-01-11T06:08:19.225ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138082025-01-11T06:08:19.225ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138092025-01-11T06:08:19.225ZINFOcrucible: send last flush ID to this DS: 4319 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138102025-01-11T06:08:19.226ZINFOcrucible: Replied this last flush ID: 4319 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138112025-01-11T06:08:19.226ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138122025-01-11T06:08:19.226ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138132025-01-11T06:08:19.226ZINFOcrucible: [0] client re-new 302 jobs since flush 4319 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138142025-01-11T06:08:19.226ZINFOcrucible: [0] Replayed 302 jobs since flush: 4319 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
13815 ----------------------------------------------------------------
13816 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13817 Downstairs last five completed: 4621 4620 4619 4618 4617
13818 Upstairs last five completed: 4621 4620 4619 4618 4617
13819 CLIENT: Up:0 ds:0 act:3
13820 Replay: All jobs finished
13821 Main task has work to do, go find it
13822 got dsc Stop(2)
13823 stop 2
13824 [8830] Got stop action so:false kr:false
13825 Jan 11 06:08:21.472 INFO request completed, latency_us: 422, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 4aaa384a-fe9c-4fff-99ee-5c86c4226b30, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
138262025-01-11T06:08:21.473ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138272025-01-11T06:08:21.473ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
138282025-01-11T06:08:21.473ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
13829 Jan 11 06:08:21.473 INFO request completed{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:08:21.473573312Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
13830 , latency_us: 128, response_code: 200, {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:08:21.47358759Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
13831 uri: /state/cid/2, method: GET, req_id: 1d380966-6fc4-4294-af89-b2a6957157ef, remote_addr: 127.0.0.1:59925, local_addr: 127{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:08:21.47366741Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
13832 .0.0.1:9998
13833 [8830] Exited with: ExitStatus(unix_wait_status(9))
13834 [8830][2] reports Exit
13835 Jan 11 06:08:25.475 INFO request completed, latency_us: 301, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6839171b-b56d-4ed2-800a-1d18f83b1cc7, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
13836 001/300 Read block 3590 len 24576
13837 002/300 Write block 2324 len 40960 data: 3 3 2 2 2 2 2 4 4 6
13838 003/300 Read block 559 len 16384
13839 004/300 Read block 4908 len 16384
13840 005/300 Read block 1748 len 24576
13841 006/300 Read block 618 len 12288
13842 007/300 Read block 4421 len 36864
13843 008/300 Write block 527 len 16384 data: 3 4 3 4
13844 009/300 Read block 664 len 16384
13845 010/300 Read block 3849 len 24576
13846 011/300 Flush
13847 012/300 Flush
13848 013/300 Write block 1680 len 32768 data: 3 3 3 3 3 4 3 3
13849 014/300 Read block 2504 len 32768
13850 015/300 Read block 2370 len 36864
13851 016/300 Read block 3147 len 24576
13852 017/300 Write block 1239 len 36864 data: 3 4 4 4 3 2 2 3 2
13853 018/300 Write block 4463 len 32768 data: 7 7 8 8 8 9 7 6
13854 019/300 Read block 4388 len 20480
13855 020/300 Read block 4239 len 28672
13856 021/300 Read block 449 len 8192
13857 022/300 Read block 1944 len 32768
13858 023/300 Write block 1206 len 36864 data: 4 4 3 3 3 4 3 3 4
13859 024/300 Flush
13860 025/300 Write block 180 len 32768 data: 2 2 2 3 3 3 3 5
13861 026/300 Write block 1534 len 24576 data: 3 3 3 3 3 3
13862 027/300 Read block 2008 len 12288
13863 028/300 Flush
13864 029/300 Read block 1186 len 8192
13865 030/300 Write block 1327 len 4096 data: 5
13866 031/300 Write block 4683 len 28672 data: 4 4 4 4 4 4 5
13867 032/300 Read block 1252 len 12288
13868 033/300 Read block 4183 len 24576
13869 034/300 Read block 4686 len 40960
13870 035/300 Read block 4465 len 16384
13871 036/300 Read block 2254 len 24576
13872 037/300 Flush
13873 038/300 Write block 2253 len 28672 data: 5 5 6 5 5 5 4
13874 039/300 Read block 3228 len 20480
13875 040/300 Read block 2595 len 20480
13876 041/300 Write block 4978 len 28672 data: 4 4 4 3 2 4 4
13877 042/300 Flush
13878 043/300 Flush
13879 044/300 Read block 2042 len 4096
13880 045/300 Write block 2174 len 12288 data: 3 4 3
13881 046/300 Write block 4448 len 36864 data: 5 5 5 6 6 6 6 7 4
13882 047/300 Read block 4260 len 4096
13883 048/300 Read block 1015 len 8192
13884 049/300 Write block 3994 len 32768 data: 5 5 4 4 4 4 4 4
13885 050/300 Flush
13886 051/300 Write block 3408 len 4096 data: 3
13887 052/300 Write block 3316 len 40960 data: 3 3 3 3 3 3 3 3 3 2
13888 053/300 Write block 1851 len 8192 data: 5 4
13889 054/300 Read block 361 len 16384
13890 055/300 Write block 4517 len 28672 data: 4 3 3 3 3 3 2
13891 056/300 Write block 3693 len 4096 data: 4
13892 057/300 Write block 3325 len 32768 data: 3 2 2 2 2 3 4 4
13893 058/300 Write block 1022 len 32768 data: 3 5 4 7 7 6 6 5
13894 059/300 Write block 3744 len 36864 data: 2 2 3 3 5 5 4 5 5
13895 060/300 Read block 3358 len 20480
13896 061/300 Write block 1068 len 16384 data: 3 3 3 3
13897 062/300 Flush
13898 063/300 Read block 4367 len 4096
13899 064/300 Write block 4934 len 40960 data: 3 3 2 2 2 3 4 4 4 5
13900 065/300 Write block 3095 len 28672 data: 4 3 3 5 4 6 6
13901 066/300 Read block 3300 len 20480
13902 067/300 Read block 2934 len 40960
13903 068/300 Read block 789 len 8192
13904 069/300 Flush
13905 070/300 Write block 2376 len 36864 data: 4 4 4 4 4 5 4 4 3
13906 071/300 Read block 2122 len 28672
13907 072/300 Write block 709 len 12288 data: 6 5 4
13908 073/300 Write block 3780 len 28672 data: 5 5 6 4 3 3 4
13909 074/300 Read block 3989 len 24576
13910 075/300 Write block 2611 len 4096 data: 2
13911 076/300 Read block 4419 len 32768
13912 077/300 Write block 693 len 24576 data: 5 4 3 3 3 2
13913 078/300 Write block 3541 len 20480 data: 2 3 2 2 3
13914 079/300 Read block 2126 len 8192
13915 080/300 Read block 2117 len 8192
13916 081/300 Write block 1399 len 24576 data: 2 2 2 3 3 4
13917 082/300 Read block 663 len 28672
13918 083/300 Read block 4843 len 8192
13919 084/300 Read block 2347 len 8192
13920 085/300 Write block 2178 len 16384 data: 3 3 3 3
13921 086/300 Flush
13922 087/300 Read block 2340 len 40960
13923 088/300 Read block 2616 len 24576
13924 089/300 Read block 3523 len 32768
13925 090/300 Write block 4892 len 20480 data: 3 4 3 3 2
13926 091/300 Write block 3484 len 12288 data: 4 4 4
13927 092/300 Write block 1220 len 36864 data: 5 3 4 3 3 3 3 4 4
13928 093/300 Write block 4358 len 12288 data: 4 5 5
13929 094/300 Write block 2345 len 12288 data: 4 5 4
13930 095/300 Read block 2699 len 16384
13931 096/300 Read block 3050 len 12288
13932 097/300 Read block 2150 len 28672
13933 098/300 Flush
13934 099/300 Flush
13935 100/300 Write block 3671 len 4096 data: 4
13936 101/300 Write block 1404 len 12288 data: 5 5 4
13937 102/300 Flush
13938 103/300 Write block 4858 len 20480 data: 2 2 2 4 2
13939 104/300 Flush
13940 105/300 Write block 375 len 4096 data: 5
13941 106/300 Flush
13942 107/300 Write block 3191 len 20480 data: 4 4 4 4 4
13943 108/300 Read block 3962 len 20480
13944 109/300 Write block 4822 len 20480 data: 5 4 4 5 5
13945 110/300 Write block 2945 len 24576 data: 6 7 7 5 7 7
13946 111/300 Write block 1617 len 20480 data: 3 3 3 4 5
13947 112/300 Read block 1605 len 20480
13948 113/300 Flush
13949 114/300 Write block 3918 len 40960 data: 2 2 2 2 2 3 3 3 2 2
13950 115/300 Read block 637 len 36864
13951 116/300 Read block 381 len 4096
13952 117/300 Write block 4188 len 20480 data: 2 2 2 2 2
13953 118/300 Read block 3501 len 24576
13954 119/300 Read block 3070 len 20480
13955 120/300 Read block 727 len 20480
13956 121/300 Read block 2884 len 36864
13957 122/300 Read block 3692 len 32768
13958 123/300 Read block 1608 len 24576
13959 124/300 Write block 2072 len 4096 data: 2
13960 125/300 Read block 4062 len 16384
13961 126/300 Write block 3613 len 20480 data: 4 4 4 2 2
13962 127/300 Read block 2474 len 8192
13963 128/300 Read block 3061 len 32768
13964 129/300 Read block 438 len 40960
13965 130/300 Read block 4336 len 8192
13966 131/300 Flush
13967 132/300 Read block 635 len 40960
13968 133/300 Write block 3404 len 4096 data: 4
13969 134/300 Read block 2393 len 16384
13970 135/300 Read block 2552 len 16384
13971 136/300 Read block 626 len 40960
13972 137/300 Write block 1524 len 20480 data: 5 5 5 5 4
13973 138/300 Write block 719 len 28672 data: 2 3 3 6 7 7 5
13974 139/300 Flush
13975 140/300 Write block 1955 len 32768 data: 3 3 3 4 4 3 4 4
13976 141/300 Read block 1537 len 20480
13977 142/300 Write block 4246 len 16384 data: 3 4 3 3
13978 143/300 Write block 3560 len 24576 data: 4 5 4 3 3 3
13979 144/300 Read block 3049 len 24576
13980 145/300 Read block 3113 len 24576
13981 146/300 Read block 2225 len 8192
13982 147/300 Write block 4621 len 16384 data: 4 4 3 3
13983 148/300 Flush
13984 149/300 Read block 385 len 40960
13985 150/300 Flush
13986 151/300 Write block 1754 len 20480 data: 4 3 3 3 3
13987 152/300 Read block 2087 len 12288
13988 153/300 Read block 4775 len 4096
13989 154/300 Flush
13990 155/300 Write block 2785 len 12288 data: 5 5 5
13991 156/300 Flush
13992 157/300 Write block 1959 len 12288 data: 5 4 5
13993 158/300 Read block 2957 len 24576
13994 159/300 Flush
13995 160/300 Read block 3766 len 36864
13996 161/300 Write block 4018 len 16384 data: 3 3 3 2
13997 162/300 Write block 1872 len 24576 data: 5 6 5 6 6 6
13998 163/300 Write block 2599 len 20480 data: 4 4 4 3 3
13999 164/300 Read block 4315 len 40960
14000 165/300 Write block 4963 len 16384 data: 2 2 2 2
14001 166/300 Flush
14002 167/300 Read block 2957 len 36864
14003 168/300 Write block 4005 len 16384 data: 2 2 2 2
14004 169/300 Write block 4098 len 40960 data: 2 2 2 2 2 2 2 2 2 3
14005 170/300 Read block 3261 len 24576
14006 171/300 Flush
14007 172/300 Write block 4586 len 16384 data: 5 5 4 3
14008 173/300 Read block 3105 len 40960
14009 174/300 Flush
14010 175/300 Write block 1829 len 8192 data: 7 7
14011 176/300 Read block 695 len 24576
14012 177/300 Read block 1975 len 12288
14013 178/300 Write block 1393 len 8192 data: 2 2
14014 179/300 Read block 3055 len 36864
14015 180/300 Read block 807 len 12288
14016 181/300 Write block 3884 len 24576 data: 5 5 5 5 4 3
14017 182/300 Write block 1396 len 16384 data: 2 2 2 3
14018 183/300 Write block 2069 len 32768 data: 3 3 3 3 3 3 3 5
14019 184/300 Write block 4219 len 8192 data: 4 4
14020 185/300 Write block 3343 len 12288 data: 2 2 3
14021 186/300 Write block 1231 len 8192 data: 4 4
14022 187/300 Read block 1059 len 16384
14023 188/300 Write block 2615 len 8192 data: 3 3
14024 189/300 Write block 1494 len 32768 data: 3 3 3 6 5 4 4 5
14025 190/300 Read block 2857 len 8192
14026 191/300 Read block 239 len 36864
14027 192/300 Read block 4458 len 8192
14028 193/300 Write block 285 len 12288 data: 5 5 4
14029 194/300 Read block 310 len 16384
14030 195/300 Write block 2620 len 36864 data: 3 2 2 4 5 5 5 6 6
14031 196/300 Flush
14032 197/300 Write block 3097 len 12288 data: 4 6 5
14033 198/300 Write block 3407 len 32768 data: 3 4 4 4 4 4 4 3
14034 199/300 Write block 2741 len 36864 data: 4 4 3 2 2 2 3 3 3
14035 200/300 Read block 2533 len 40960
14036 201/300 Read block 4318 len 16384
14037 202/300 Write block 2604 len 16384 data: 3 3 2 2
14038 203/300 Read block 2674 len 32768
14039 204/300 Write block 4800 len 16384 data: 2 2 2 2
14040 205/300 Read block 2749 len 32768
14041 206/300 Read block 3592 len 32768
14042 207/300 Read block 3052 len 36864
14043 208/300 Read block 2895 len 36864
14044 209/300 Read block 3576 len 32768
14045 210/300 Write block 662 len 24576 data: 3 3 3 3 3 3
14046 211/300 Read block 2233 len 36864
14047 212/300 Flush
14048 213/300 Read block 2957 len 24576
14049 214/300 Write block 188 len 40960 data: 5 5 4 4 5 5 4 4 3 3
14050 215/300 Read block 1376 len 40960
14051 216/300 Write block 3704 len 32768 data: 4 4 5 5 4 3 3 4
14052 217/300 Flush
14053 218/300 Read block 2933 len 20480
14054 219/300 Read block 264 len 20480
14055 220/300 Write block 4951 len 24576 data: 6 6 5 5 4 4
14056 221/300 Flush
14057 222/300 Write block 4475 len 32768 data: 3 3 3 3 3 3 3 3
14058 223/300 Write block 3680 len 8192 data: 4 4
14059 224/300 Write block 355 len 32768 data: 5 5 5 5 4 4 3 4
14060 225/300 Read block 762 len 36864
14061 226/300 Read block 338 len 12288
14062 227/300 Read block 325 len 36864
14063 228/300 Write block 4506 len 24576 data: 6 6 7 7 6 5
14064 229/300 Read block 2647 len 20480
14065 230/300 Read block 3115 len 32768
14066 231/300 Read block 4060 len 36864
14067 232/300 Write block 1704 len 36864 data: 4 4 3 3 4 3 4 5 4
14068 233/300 Flush
14069 234/300 Read block 4074 len 4096
14070 235/300 Write block 32 len 36864 data: 3 3 4 4 3 5 4 6 7
14071 236/300 Write block 4928 len 4096 data: 3
14072 237/300 Read block 3218 len 40960
14073 238/300 Read block 1891 len 32768
14074 239/300 Read block 994 len 28672
14075 240/300 Read block 1813 len 36864
14076 241/300 Read block 3127 len 36864
14077 242/300 Read block 923 len 40960
14078 243/300 Flush
14079 244/300 Write block 2132 len 12288 data: 5 5 4
14080 245/300 Flush
14081 246/300 Read block 2087 len 12288
14082 247/300 Write block 3532 len 40960 data: 2 3 3 2 2 2 2 2 2 3
14083 248/300 Read block 1608 len 20480
14084 249/300 Flush
14085 250/300 Read block 3045 len 4096
14086 251/300 Write block 251 len 8192 data: 3 3
14087 252/300 Read block 1490 len 12288
14088 253/300 Write block 1203 len 40960 data: 5 5 5 5 5 4 4 4 5 4
14089 254/300 Read block 2034 len 32768
14090 255/300 Write block 2967 len 20480 data: 4 5 5 5 5
14091 256/300 Flush
14092 257/300 Write block 4809 len 4096 data: 2
14093 258/300 Read block 72 len 4096
14094 259/300 Read block 2628 len 16384
14095 260/300 Write block 2959 len 32768 data: 4 4 3 3 3 3 2 2
14096 261/300 Read block 4688 len 16384
14097 262/300 Write block 247 len 36864 data: 4 3 2 3 4 4 3 3 3
14098 263/300 Write block 2396 len 24576 data: 4 4 3 3 3 3
14099 264/300 Write block 2734 len 40960 data: 4 5 5 4 4 4 4 5 5 4
14100 265/300 Read block 2546 len 12288
14101 266/300 Flush
14102 267/300 Write block 692 len 8192 data: 5 6
14103 268/300 Read block 2247 len 40960
14104 269/300 Read block 3134 len 32768
14105 270/300 Read block 2631 len 36864
14106 271/300 Read block 479 len 24576
14107 272/300 Read block 3959 len 36864
14108 273/300 Read block 4177 len 36864
14109 274/300 Write block 717 len 4096 data: 2
14110 275/300 Write block 4743 len 12288 data: 4 4 4
14111 276/300 Write block 4802 len 32768 data: 3 3 2 2 2 2 2 3
14112 277/300 Read block 2466 len 24576
14113 278/300 Read block 780 len 32768
14114 279/300 Read block 1630 len 20480
14115 280/300 Write block 298 len 40960 data: 2 3 3 3 2 2 4 4 4 4
14116 281/300 Read block 2677 len 12288
14117 282/300 Read block 1879 len 40960
14118 283/300 Flush
14119 284/300 Flush
14120 285/300 Read block 3851 len 36864
14121 286/300 Write block 2005 len 28672 data: 3 3 2 2 2 2 3
14122 287/300 Flush
14123 288/300 Read block 1519 len 28672
14124 289/300 Read block 2741 len 20480
14125 290/300 Read block 2879 len 20480
14126 291/300 Write block 2299 len 16384 data: 4 4 5 4
14127 292/300 Write block 2203 len 36864 data: 3 4 4 4 4 4 3 3 3
14128 293/300 Read block 3674 len 28672
14129 294/300 Write block 4265 len 20480 data: 6 6 5 5 5
14130 295/300 Read block 628 len 24576
14131 296/300 Read block 3929 len 8192
14132 297/300 Write block 4271 len 12288 data: 4 4 3
14133 298/300 Write block 3249 len 20480 data: 3 3 3 3 3
14134 299/300 Read block 1315 len 28672
14135 300/300 Read block 1737 len 28672
14136 301/300 Read block 4535 len 16384
14137 Main task has work to do, go find it
14138 got dsc Start(2)
14139 start 2
14140 Jan 11 06:08:25.722 INFO request completed, latency_us: 203, response_code: 204, uri: /start/cid/2, method: POST, req_id: c3bef822-312a-4f80-8cbd-49e389b12491, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
14141 Replay: started 2, returned:Ok(())
14142 ----------------------------------------------------------------
14143 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14144 States: Active Active Offline
14145 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14146 4622 Acked Read 6 Done Done Sent false
14147 4623 Acked Write 10 Done Done Sent false
14148 4624 Acked Read 4 Done Done Sent false
14149 4625 Acked Read 4 Done Done Sent false
14150 4626 Acked Read 6 Done Done Sent false
14151 4627 Acked Read 3 Done Done Sent false
14152 4628 Acked Read 9 Done Done Sent false
14153 4629 Acked Write 4 Done Done Sent false
14154 4630 Acked Read 4 Done Done Sent false
14155 4631 Acked Read 6 Done Done Sent false
14156 4632 Acked Flush 0 Done Done Sent false
14157 4633 Acked Flush 0 Done Done Sent false
14158 4634 Acked Write 8 Done Done Sent false
14159 4635 Acked Read 8 Done Done Sent false
14160 4636 Acked Read 9 Done Done Sent false
14161 4637 Acked Read 6 Done Done Sent false
14162 4638 Acked Write 9 Done Done Sent false
14163 4639 Acked Write 8 Done Done Sent false
14164 4640 Acked Read 5 Done Done Sent false
14165 4641 Acked Read 7 Done Done Sent false
14166 4642 Acked Read 2 Done Done Sent false
14167 4643 Acked Read 8 Done Done Sent false
14168 4644 Acked Write 9 Done Done Sent false
14169 4645 Acked Flush 0 Done Done Sent false
14170 4646 Acked Write 8 Done Done Sent false
14171 4647 Acked Write 6 Done Done Sent false
14172 4648 Acked Read 3 Done Done Sent false
14173 4649 Acked Flush 0 Done Done Sent false
14174 4650 Acked Read 2 Done Done Sent false
14175 4651 Acked Write 1 Done Done Sent false
14176 4652 Acked Write 7 Done Done Sent false
14177 4653 Acked Read 3 Done Done Sent false
14178 4654 Acked Read 6 Done Done Sent false
14179 4655 Acked Read 10 Done Done Sent false
14180 4656 Acked Read 4 Done Done Sent false
14181 4657 Acked Read 6 Done Done Sent false
14182 4658 Acked Flush 0 Done Done Sent false
14183 4659 Acked Write 7 Done Done Sent false
14184 4660 Acked Read 5 Done Done Sent false
14185 4661 Acked Read 5 Done Done Sent false
14186 4662 Acked Write 7 Done Done Sent false
14187 4663 Acked Flush 0 Done Done Sent false
14188 4664 Acked Flush 0 Done Done Sent false
14189 4665 Acked Read 1 Done Done Sent false
14190 4666 Acked Write 3 Done Done Sent false
14191 4667 Acked Write 9 Done Done Sent false
14192 4668 Acked Read 1 Done Done Sent false
14193 4669 Acked Read 2 Done Done Sent false
14194 4670 Acked Write 8 Done Done Sent false
14195 4671 Acked Flush 0 Done Done Sent false
14196 4672 Acked Write 1 Done Done Sent false
14197 4673 Acked Write 10 Done Done Sent false
14198 4674 Acked Write 2 Done Done Sent false
14199 4675 Acked Read 4 Done Done Sent false
14200 4676 Acked Write 7 Done Done Sent false
14201 4677 Acked Write 1 Done Done Sent false
14202 4678 Acked Write 8 Done Done Sent false
14203 4679 Acked Write 8 Done Done Sent false
14204 4680 Acked Write 9 Done Done Sent false
14205 4681 Acked Read 5 Done Done Sent false
14206 4682 Acked Write 4 Done Done Sent false
14207 4683 Acked Flush 0 Done Done Sent false
14208 4684 Acked Read 1 Done Done Sent false
14209 4685 Acked Write 10 Done Done Sent false
14210 4686 Acked Write 7 Done Done Sent false
14211 4687 Acked Read 5 Done Done Sent false
14212 4688 Acked Read 10 Done Done Sent false
14213 4689 Acked Read 2 Done Done Sent false
14214 4690 Acked Flush 0 Done Done Sent false
14215 4691 Acked Write 9 Done Done Sent false
14216 4692 Acked Read 7 Done Done Sent false
14217 4693 Acked Write 3 Done Done Sent false
14218 4694 Acked Write 7 Done Done Sent false
14219 4695 Acked Read 6 Done Done Sent false
14220 4696 Acked Write 1 Done Done Sent false
14221 4697 Acked Read 8 Done Done Sent false
14222 4698 Acked Write 6 Done Done Sent false
14223 4699 Acked Write 5 Done Done Sent false
14224 4700 Acked Read 2 Done Done Sent false
14225 4701 Acked Read 2 Done Done Sent false
14226 4702 Acked Write 6 Done Done Sent false
14227 4703 Acked Read 7 Done Done Sent false
14228 4704 Acked Read 2 Done Done Sent false
14229 4705 Acked Read 2 Done Done Sent false
14230 4706 Acked Write 4 Done Done Sent false
14231 4707 Acked Flush 0 Done Done Sent false
14232 4708 Acked Read 10 Done Done Sent false
14233 4709 Acked Read 6 Done Done Sent false
14234 4710 Acked Read 8 Done Done Sent false
14235 4711 Acked Write 5 Done Done Sent false
14236 4712 Acked Write 3 Done Done Sent false
14237 4713 Acked Write 9 Done Done Sent false
14238 4714 Acked Write 3 Done Done Sent false
14239 4715 Acked Write 3 Done Done Sent false
14240 4716 Acked Read 4 Done Done Sent false
14241 4717 Acked Read 3 Done Done Sent false
14242 4718 Acked Read 7 Done Done Sent false
14243 4719 Acked Flush 0 Done Done Sent false
14244 4720 Acked Flush 0 Done Done Sent false
14245 4721 Acked Write 1 Done Done Sent false
14246 4722 Acked Write 3 Done Done Sent false
14247 4723 Acked Flush 0 Done Done Sent false
14248 4724 Acked Write 5 Done Done Sent false
14249 4725 Acked Flush 0 Done Done Sent false
14250 4726 Acked Write 1 Done Done Sent false
14251 4727 Acked Flush 0 Done Done Sent false
14252 4728 Acked Write 5 Done Done Sent false
14253 4729 Acked Read 5 Done Done Sent false
14254 4730 Acked Write 5 Done Done Sent false
14255 4731 Acked Write 6 Done Done Sent false
14256 4732 Acked Write 5 Done Done Sent false
14257 4733 Acked Read 5 Done Done Sent false
14258 4734 Acked Flush 0 Done Done Sent false
14259 4735 Acked Write 10 Done Done Sent false
14260 4736 Acked Read 9 Done Done Sent false
14261 4737 Acked Read 1 Done Done Sent false
14262 4738 Acked Write 5 Done Done Sent false
14263 4739 Acked Read 6 Done Done Sent false
14264 4740 Acked Read 5 Done Done Sent false
14265 4741 Acked Read 5 Done Done Sent false
14266 4742 Acked Read 9 Done Done Sent false
14267 4743 Acked Read 8 Done Done Sent false
14268 4744 Acked Read 6 Done Done Sent false
14269 4745 Acked Write 1 Done Done Sent false
14270 4746 Acked Read 4 Done Done Sent false
14271 4747 Acked Write 5 Done Done Sent false
14272 4748 Acked Read 2 Done Done Sent false
14273 4749 Acked Read 8 Done Done Sent false
14274 4750 Acked Read 10 Done Done Sent false
14275 4751 Acked Read 2 Done Done Sent false
14276 4752 Acked Flush 0 Done Done Sent false
14277 4753 Acked Read 10 Done Done Sent false
14278 4754 Acked Write 1 Done Done Sent false
14279 4755 Acked Read 4 Done Done Sent false
14280 4756 Acked Read 4 Done Done Sent false
14281 4757 Acked Read 10 Done Done Sent false
14282 4758 Acked Write 5 Done Done Sent false
14283 4759 Acked Write 7 Done Done Sent false
14284 4760 Acked Flush 0 Done Done Sent false
14285 4761 Acked Write 8 Done Done Sent false
14286 4762 Acked Read 5 Done Done Sent false
14287 4763 Acked Write 4 Done Done Sent false
14288 4764 Acked Write 6 Done Done Sent false
14289 4765 Acked Read 6 Done Done Sent false
14290 4766 Acked Read 6 Done Done Sent false
14291 4767 Acked Read 2 Done Done Sent false
14292 4768 Acked Write 4 Done Done Sent false
14293 4769 Acked Flush 0 Done Done Sent false
14294 4770 Acked Read 10 Done Done Sent false
14295 4771 Acked Flush 0 Done Done Sent false
14296 4772 Acked Write 5 Done Done Sent false
14297 4773 Acked Read 3 Done Done Sent false
14298 4774 Acked Read 1 Done Done Sent false
14299 4775 Acked Flush 0 Done Done Sent false
14300 4776 Acked Write 3 Done Done Sent false
14301 4777 Acked Flush 0 Done Done Sent false
14302 4778 Acked Write 3 Done Done Sent false
14303 4779 Acked Read 6 Done Done Sent false
14304 4780 Acked Flush 0 Done Done Sent false
14305 4781 Acked Read 9 Done Done Sent false
14306 4782 Acked Write 4 Done Done Sent false
14307 4783 Acked Write 6 Done Done Sent false
14308 4784 Acked Write 5 Done Done Sent false
14309 4785 Acked Read 10 Done Done Sent false
14310 4786 Acked Write 4 Done Done Sent false
14311 4787 Acked Flush 0 Done Done Sent false
14312 4788 Acked Read 9 Done Done Sent false
14313 4789 Acked Write 4 Done Done Sent false
14314 4790 Acked Write 10 Done Done Sent false
14315 4791 Acked Read 6 Done Done Sent false
14316 4792 Acked Flush 0 Done Done Sent false
14317 4793 Acked Write 4 Done Done Sent false
14318 4794 Acked Read 10 Done Done Sent false
14319 4795 Acked Flush 0 Done Done Sent false
14320 4796 Acked Write 2 Done Done Sent false
14321 4797 Acked Read 6 Done Done Sent false
14322 4798 Acked Read 3 Done Done Sent false
14323 4799 Acked Write 2 Done Done Sent false
14324 4800 Acked Read 9 Done Done Sent false
14325 4801 Acked Read 3 Done Done Sent false
14326 4802 Acked Write 6 Done Done Sent false
14327 4803 Acked Write 4 Done Done Sent false
14328 4804 Acked Write 8 Done Done Sent false
14329 4805 Acked Write 2 Done Done Sent false
14330 4806 Acked Write 3 Done Done Sent false
14331 4807 Acked Write 2 Done Done Sent false
14332 4808 Acked Read 4 Done Done Sent false
14333 4809 Acked Write 2 Done Done Sent false
14334 4810 Acked Write 8 Done Done Sent false
14335 4811 Acked Read 2 Done Done Sent false
14336 4812 Acked Read 9 Done Done Sent false
14337 4813 Acked Read 2 Done Done Sent false
14338 4814 Acked Write 3 Done Done Sent false
14339 4815 Acked Read 4 Done Done Sent false
14340 4816 Acked Write 9 Done Done Sent false
14341 4817 Acked Flush 0 Done Done Sent false
14342 4818 Acked Write 3 Done Done Sent false
14343 4819 Acked Write 8 Done Done Sent false
14344 4820 Acked Write 9 Done Done Sent false
14345 4821 Acked Read 10 Done Done Sent false
14346 4822 Acked Read 4 Done Done Sent false
14347 4823 Acked Write 4 Done Done Sent false
14348 4824 Acked Read 8 Done Done Sent false
14349 4825 Acked Write 4 Done Done Sent false
14350 4826 Acked Read 8 Done Done Sent false
14351 4827 Acked Read 8 Done Done Sent false
14352 4828 Acked Read 9 Done Done Sent false
14353 4829 Acked Read 9 Done Done Sent false
14354 4830 Acked Read 8 Done Done Sent false
14355 4831 Acked Write 6 Done Done Sent false
14356 4832 Acked Read 9 Done Done Sent false
14357 4833 Acked Flush 0 Done Done Sent false
14358 4834 Acked Read 6 Done Done Sent false
14359 4835 Acked Write 10 Done Done Sent false
14360 4836 Acked Read 10 Done Done Sent false
14361 4837 Acked Write 8 Done Done Sent false
14362 4838 Acked Flush 0 Done Done Sent false
14363 4839 Acked Read 5 Done Done Sent false
14364 4840 Acked Read 5 Done Done Sent false
14365 4841 Acked Write 6 Done Done Sent false
14366 4842 Acked Flush 0 Done Done Sent false
14367 4843 Acked Write 8 Done Done Sent false
14368 4844 Acked Write 2 Done Done Sent false
14369 4845 Acked Write 8 Done Done Sent false
14370 4846 Acked Read 9 Done Done Sent false
14371 4847 Acked Read 3 Done Done Sent false
14372 4848 Acked Read 9 Done Done Sent false
14373 4849 Acked Write 6 Done Done Sent false
14374 4850 Acked Read 5 Done Done Sent false
14375 4851 Acked Read 8 Done Done Sent false
14376 4852 Acked Read 9 Done Done Sent false
14377 4853 Acked Write 9 Done Done Sent false
14378 4854 Acked Flush 0 Done Done Sent false
14379 4855 Acked Read 1 Done Done Sent false
14380 4856 Acked Write 9 Done Done Sent false
14381 4857 Acked Write 1 Done Done Sent false
14382 4858 Acked Read 10 Done Done Sent false
14383 4859 Acked Read 8 Done Done Sent false
14384 4860 Acked Read 7 Done Done Sent false
14385 4861 Acked Read 9 Done Done Sent false
14386 4862 Acked Read 9 Done Done Sent false
14387 4863 Acked Read 10 Done Done Sent false
14388 4864 Acked Flush 0 Done Done Sent false
14389 4865 Acked Write 3 Done Done Sent false
14390 4866 Acked Flush 0 Done Done Sent false
14391 4867 Acked Read 3 Done Done Sent false
14392 4868 Acked Write 10 Done Done Sent false
14393 4869 Acked Read 5 Done Done Sent false
14394 4870 Acked Flush 0 Done Done Sent false
14395 4871 Acked Read 1 Done Done Sent false
14396 4872 Acked Write 2 Done Done Sent false
14397 4873 Acked Read 3 Done Done Sent false
14398 4874 Acked Write 10 Done Done Sent false
14399 4875 Acked Read 8 Done Done Sent false
14400 4876 Acked Write 5 Done Done Sent false
14401 4877 Acked Flush 0 Done Done Sent false
14402 4878 Acked Write 1 Done Done Sent false
14403 4879 Acked Read 1 Done Done Sent false
14404 4880 Acked Read 4 Done Done Sent false
14405 4881 Acked Write 8 Done Done Sent false
14406 4882 Acked Read 4 Done Done Sent false
14407 4883 Acked Write 9 Done Done Sent false
14408 4884 Acked Write 6 Done Done Sent false
14409 4885 Acked Write 10 Done Done Sent false
14410 4886 Acked Read 3 Done Done Sent false
14411 4887 Acked Flush 0 Done Done Sent false
14412 4888 Acked Write 2 Done Done Sent false
14413 4889 Acked Read 10 Done Done Sent false
14414 4890 Acked Read 8 Done Done Sent false
14415 4891 Acked Read 9 Done Done Sent false
14416 4892 Acked Read 6 Done Done Sent false
14417 4893 Acked Read 9 Done Done Sent false
14418 4894 Acked Read 9 Done Done Sent false
14419 4895 Acked Write 1 Done Done Sent false
14420 4896 Acked Write 3 Done Done Sent false
14421 4897 Acked Write 8 Done Done Sent false
14422 4898 Acked Read 6 Done Done Sent false
14423 4899 Acked Read 8 Done Done Sent false
14424 4900 Acked Read 5 Done Done Sent false
14425 4901 Acked Write 10 Done Done Sent false
14426 4902 Acked Read 3 Done Done Sent false
14427 4903 Acked Read 10 Done Done Sent false
14428 4904 Acked Flush 0 Done Done Sent false
14429 4905 Acked Flush 0 Done Done Sent false
14430 4906 Acked Read 9 Done Done Sent false
14431 4907 Acked Write 7 Done Done Sent false
14432 4908 Acked Flush 0 Done Done Sent false
14433 4909 Acked Read 7 Done Done Sent false
14434 4910 Acked Read 5 Done Done Sent false
14435 4911 Acked Read 5 Done Done Sent false
14436 4912 Acked Write 4 Done Done Sent false
14437 4913 Acked Write 9 Done Done Sent false
14438 4914 Acked Read 7 Done Done Sent false
14439 4915 Acked Write 5 Done Done Sent false
14440 4916 Acked Read 6 Done Done Sent false
14441 4917 Acked Read 2 Done Done Sent false
14442 4918 Acked Write 3 Done Done Sent false
14443 4919 Acked Write 5 Done Done Sent false
14444 4920 Acked Read 7 Done Done Sent false
14445 4921 Acked Read 7 Done Done Sent false
14446 4922 Acked Read 4 Done Done Sent false
14447 STATES DS:0 DS:1 DS:2 TOTAL
14448 Sent 0 0 301 301
14449 Done 301 301 0 602
14450 Skipped 0 0 0 0
14451 Error 0 0 0 0
14452 Last Flush: 4908 4908 4621
14453 Downstairs last five completed: 4621 4620 4619 4618 4617
14454 Upstairs last five completed: 4922 4921 4920 4919 4918
14455 CLIENT: Up:0 ds:301 act:2
14456 [8830] got start action
14457 [8830] I am going to restart
14458 Starting downstairs at port 8830
14459 Make output file at "/tmp/dsc/downstairs-8830.txt"
14460 [8830][2] reports Starting
14461 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1217)
14462 [8830][2] reports Running
14463 ----------------------------------------------------------------
14464 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14465 States: Active Active Offline
14466 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14467 4622 Acked Read 6 Done Done Sent false
14468 4623 Acked Write 10 Done Done Sent false
14469 4624 Acked Read 4 Done Done Sent false
14470 4625 Acked Read 4 Done Done Sent false
14471 4626 Acked Read 6 Done Done Sent false
14472 4627 Acked Read 3 Done Done Sent false
14473 4628 Acked Read 9 Done Done Sent false
14474 4629 Acked Write 4 Done Done Sent false
14475 4630 Acked Read 4 Done Done Sent false
14476 4631 Acked Read 6 Done Done Sent false
14477 4632 Acked Flush 0 Done Done Sent false
14478 4633 Acked Flush 0 Done Done Sent false
14479 4634 Acked Write 8 Done Done Sent false
14480 4635 Acked Read 8 Done Done Sent false
14481 4636 Acked Read 9 Done Done Sent false
14482 4637 Acked Read 6 Done Done Sent false
14483 4638 Acked Write 9 Done Done Sent false
14484 4639 Acked Write 8 Done Done Sent false
14485 4640 Acked Read 5 Done Done Sent false
14486 4641 Acked Read 7 Done Done Sent false
14487 4642 Acked Read 2 Done Done Sent false
14488 4643 Acked Read 8 Done Done Sent false
14489 4644 Acked Write 9 Done Done Sent false
14490 4645 Acked Flush 0 Done Done Sent false
14491 4646 Acked Write 8 Done Done Sent false
14492 4647 Acked Write 6 Done Done Sent false
14493 4648 Acked Read 3 Done Done Sent false
14494 4649 Acked Flush 0 Done Done Sent false
14495 4650 Acked Read 2 Done Done Sent false
14496 4651 Acked Write 1 Done Done Sent false
14497 4652 Acked Write 7 Done Done Sent false
14498 4653 Acked Read 3 Done Done Sent false
14499 4654 Acked Read 6 Done Done Sent false
14500 4655 Acked Read 10 Done Done Sent false
14501 4656 Acked Read 4 Done Done Sent false
14502 4657 Acked Read 6 Done Done Sent false
14503 4658 Acked Flush 0 Done Done Sent false
14504 4659 Acked Write 7 Done Done Sent false
14505 4660 Acked Read 5 Done Done Sent false
14506 4661 Acked Read 5 Done Done Sent false
14507 4662 Acked Write 7 Done Done Sent false
14508 4663 Acked Flush 0 Done Done Sent false
14509 4664 Acked Flush 0 Done Done Sent false
14510 4665 Acked Read 1 Done Done Sent false
14511 4666 Acked Write 3 Done Done Sent false
14512 4667 Acked Write 9 Done Done Sent false
14513 4668 Acked Read 1 Done Done Sent false
14514 4669 Acked Read 2 Done Done Sent false
14515 4670 Acked Write 8 Done Done Sent false
14516 4671 Acked Flush 0 Done Done Sent false
14517 4672 Acked Write 1 Done Done Sent false
14518 4673 Acked Write 10 Done Done Sent false
14519 4674 Acked Write 2 Done Done Sent false
14520 4675 Acked Read 4 Done Done Sent false
14521 4676 Acked Write 7 Done Done Sent false
14522 4677 Acked Write 1 Done Done Sent false
14523 4678 Acked Write 8 Done Done Sent false
14524 4679 Acked Write 8 Done Done Sent false
14525 4680 Acked Write 9 Done Done Sent false
14526 4681 Acked Read 5 Done Done Sent false
14527 4682 Acked Write 4 Done Done Sent false
14528 4683 Acked Flush 0 Done Done Sent false
14529 4684 Acked Read 1 Done Done Sent false
14530 4685 Acked Write 10 Done Done Sent false
14531 4686 Acked Write 7 Done Done Sent false
14532 4687 Acked Read 5 Done Done Sent false
14533 4688 Acked Read 10 Done Done Sent false
14534 4689 Acked Read 2 Done Done Sent false
14535 4690 Acked Flush 0 Done Done Sent false
14536 4691 Acked Write 9 Done Done Sent false
14537 4692 Acked Read 7 Done Done Sent false
14538 4693 Acked Write 3 Done Done Sent false
14539 4694 Acked Write 7 Done Done Sent false
14540 4695 Acked Read 6 Done Done Sent false
14541 4696 Acked Write 1 Done Done Sent false
14542 4697 Acked Read 8 Done Done Sent false
14543 4698 Acked Write 6 Done Done Sent false
14544 4699 Acked Write 5 Done Done Sent false
14545 4700 Acked Read 2 Done Done Sent false
14546 4701 Acked Read 2 Done Done Sent false
14547 4702 Acked Write 6 Done Done Sent false
14548 4703 Acked Read 7 Done Done Sent false
14549 4704 Acked Read 2 Done Done Sent false
14550 4705 Acked Read 2 Done Done Sent false
14551 4706 Acked Write 4 Done Done Sent false
14552 4707 Acked Flush 0 Done Done Sent false
14553 4708 Acked Read 10 Done Done Sent false
14554 4709 Acked Read 6 Done Done Sent false
14555 4710 Acked Read 8 Done Done Sent false
14556 4711 Acked Write 5 Done Done Sent false
14557 4712 Acked Write 3 Done Done Sent false
14558 4713 Acked Write 9 Done Done Sent false
14559 4714 Acked Write 3 Done Done Sent false
14560 4715 Acked Write 3 Done Done Sent false
14561 4716 Acked Read 4 Done Done Sent false
14562 4717 Acked Read 3 Done Done Sent false
14563 4718 Acked Read 7 Done Done Sent false
14564 4719 Acked Flush 0 Done Done Sent false
14565 4720 Acked Flush 0 Done Done Sent false
14566 4721 Acked Write 1 Done Done Sent false
14567 4722 Acked Write 3 Done Done Sent false
14568 4723 Acked Flush 0 Done Done Sent false
14569 4724 Acked Write 5 Done Done Sent false
14570 4725 Acked Flush 0 Done Done Sent false
14571 4726 Acked Write 1 Done Done Sent false
14572 4727 Acked Flush 0 Done Done Sent false
14573 4728 Acked Write 5 Done Done Sent false
14574 4729 Acked Read 5 Done Done Sent false
14575 4730 Acked Write 5 Done Done Sent false
14576 4731 Acked Write 6 Done Done Sent false
14577 4732 Acked Write 5 Done Done Sent false
14578 4733 Acked Read 5 Done Done Sent false
14579 4734 Acked Flush 0 Done Done Sent false
14580 4735 Acked Write 10 Done Done Sent false
14581 4736 Acked Read 9 Done Done Sent false
14582 4737 Acked Read 1 Done Done Sent false
14583 4738 Acked Write 5 Done Done Sent false
14584 4739 Acked Read 6 Done Done Sent false
14585 4740 Acked Read 5 Done Done Sent false
14586 4741 Acked Read 5 Done Done Sent false
14587 4742 Acked Read 9 Done Done Sent false
14588 4743 Acked Read 8 Done Done Sent false
14589 4744 Acked Read 6 Done Done Sent false
14590 4745 Acked Write 1 Done Done Sent false
14591 4746 Acked Read 4 Done Done Sent false
14592 4747 Acked Write 5 Done Done Sent false
14593 4748 Acked Read 2 Done Done Sent false
14594 4749 Acked Read 8 Done Done Sent false
14595 4750 Acked Read 10 Done Done Sent false
14596 4751 Acked Read 2 Done Done Sent false
14597 4752 Acked Flush 0 Done Done Sent false
14598 4753 Acked Read 10 Done Done Sent false
14599 4754 Acked Write 1 Done Done Sent false
14600 4755 Acked Read 4 Done Done Sent false
14601 4756 Acked Read 4 Done Done Sent false
14602 4757 Acked Read 10 Done Done Sent false
14603 4758 Acked Write 5 Done Done Sent false
14604 4759 Acked Write 7 Done Done Sent false
14605 4760 Acked Flush 0 Done Done Sent false
14606 4761 Acked Write 8 Done Done Sent false
14607 4762 Acked Read 5 Done Done Sent false
14608 4763 Acked Write 4 Done Done Sent false
14609 4764 Acked Write 6 Done Done Sent false
14610 4765 Acked Read 6 Done Done Sent false
14611 4766 Acked Read 6 Done Done Sent false
14612 4767 Acked Read 2 Done Done Sent false
14613 4768 Acked Write 4 Done Done Sent false
14614 4769 Acked Flush 0 Done Done Sent false
14615 4770 Acked Read 10 Done Done Sent false
14616 4771 Acked Flush 0 Done Done Sent false
14617 4772 Acked Write 5 Done Done Sent false
14618 4773 Acked Read 3 Done Done Sent false
14619 4774 Acked Read 1 Done Done Sent false
14620 4775 Acked Flush 0 Done Done Sent false
14621 4776 Acked Write 3 Done Done Sent false
14622 4777 Acked Flush 0 Done Done Sent false
14623 4778 Acked Write 3 Done Done Sent false
14624 4779 Acked Read 6 Done Done Sent false
14625 4780 Acked Flush 0 Done Done Sent false
14626 4781 Acked Read 9 Done Done Sent false
14627 4782 Acked Write 4 Done Done Sent false
14628 4783 Acked Write 6 Done Done Sent false
14629 4784 Acked Write 5 Done Done Sent false
14630 4785 Acked Read 10 Done Done Sent false
14631 4786 Acked Write 4 Done Done Sent false
14632 4787 Acked Flush 0 Done Done Sent false
14633 4788 Acked Read 9 Done Done Sent false
14634 4789 Acked Write 4 Done Done Sent false
14635 4790 Acked Write 10 Done Done Sent false
14636 4791 Acked Read 6 Done Done Sent false
14637 4792 Acked Flush 0 Done Done Sent false
14638 4793 Acked Write 4 Done Done Sent false
14639 4794 Acked Read 10 Done Done Sent false
14640 4795 Acked Flush 0 Done Done Sent false
14641 4796 Acked Write 2 Done Done Sent false
14642 4797 Acked Read 6 Done Done Sent false
14643 4798 Acked Read 3 Done Done Sent false
14644 4799 Acked Write 2 Done Done Sent false
14645 4800 Acked Read 9 Done Done Sent false
14646 4801 Acked Read 3 Done Done Sent false
14647 4802 Acked Write 6 Done Done Sent false
14648 4803 Acked Write 4 Done Done Sent false
14649 4804 Acked Write 8 Done Done Sent false
14650 4805 Acked Write 2 Done Done Sent false
14651 4806 Acked Write 3 Done Done Sent false
14652 4807 Acked Write 2 Done Done Sent false
14653 4808 Acked Read 4 Done Done Sent false
14654 4809 Acked Write 2 Done Done Sent false
14655 4810 Acked Write 8 Done Done Sent false
14656 4811 Acked Read 2 Done Done Sent false
14657 4812 Acked Read 9 Done Done Sent false
14658 4813 Acked Read 2 Done Done Sent false
14659 4814 Acked Write 3 Done Done Sent false
14660 4815 Acked Read 4 Done Done Sent false
14661 4816 Acked Write 9 Done Done Sent false
14662 4817 Acked Flush 0 Done Done Sent false
14663 4818 Acked Write 3 Done Done Sent false
14664 4819 Acked Write 8 Done Done Sent false
14665 4820 Acked Write 9 Done Done Sent false
14666 4821 Acked Read 10 Done Done Sent false
14667 4822 Acked Read 4 Done Done Sent false
14668 4823 Acked Write 4 Done Done Sent false
14669 4824 Acked Read 8 Done Done Sent false
14670 4825 Acked Write 4 Done Done Sent false
14671 4826 Acked Read 8 Done Done Sent false
14672 4827 Acked Read 8 Done Done Sent false
14673 4828 Acked Read 9 Done Done Sent false
14674 4829 Acked Read 9 Done Done Sent false
14675 4830 Acked Read 8 Done Done Sent false
14676 4831 Acked Write 6 Done Done Sent false
14677 4832 Acked Read 9 Done Done Sent false
14678 4833 Acked Flush 0 Done Done Sent false
14679 4834 Acked Read 6 Done Done Sent false
14680 4835 Acked Write 10 Done Done Sent false
14681 4836 Acked Read 10 Done Done Sent false
14682 4837 Acked Write 8 Done Done Sent false
14683 4838 Acked Flush 0 Done Done Sent false
14684 4839 Acked Read 5 Done Done Sent false
14685 4840 Acked Read 5 Done Done Sent false
14686 4841 Acked Write 6 Done Done Sent false
14687 4842 Acked Flush 0 Done Done Sent false
14688 4843 Acked Write 8 Done Done Sent false
14689 4844 Acked Write 2 Done Done Sent false
14690 4845 Acked Write 8 Done Done Sent false
14691 4846 Acked Read 9 Done Done Sent false
14692 4847 Acked Read 3 Done Done Sent false
14693 4848 Acked Read 9 Done Done Sent false
14694 4849 Acked Write 6 Done Done Sent false
14695 4850 Acked Read 5 Done Done Sent false
14696 4851 Acked Read 8 Done Done Sent false
14697 4852 Acked Read 9 Done Done Sent false
14698 4853 Acked Write 9 Done Done Sent false
14699 4854 Acked Flush 0 Done Done Sent false
14700 4855 Acked Read 1 Done Done Sent false
14701 4856 Acked Write 9 Done Done Sent false
14702 4857 Acked Write 1 Done Done Sent false
14703 4858 Acked Read 10 Done Done Sent false
14704 4859 Acked Read 8 Done Done Sent false
14705 4860 Acked Read 7 Done Done Sent false
14706 4861 Acked Read 9 Done Done Sent false
14707 4862 Acked Read 9 Done Done Sent false
14708 4863 Acked Read 10 Done Done Sent false
14709 4864 Acked Flush 0 Done Done Sent false
14710 4865 Acked Write 3 Done Done Sent false
14711 4866 Acked Flush 0 Done Done Sent false
14712 4867 Acked Read 3 Done Done Sent false
14713 4868 Acked Write 10 Done Done Sent false
14714 4869 Acked Read 5 Done Done Sent false
14715 4870 Acked Flush 0 Done Done Sent false
14716 4871 Acked Read 1 Done Done Sent false
14717 4872 Acked Write 2 Done Done Sent false
14718 4873 Acked Read 3 Done Done Sent false
14719 4874 Acked Write 10 Done Done Sent false
14720 4875 Acked Read 8 Done Done Sent false
14721 4876 Acked Write 5 Done Done Sent false
14722 4877 Acked Flush 0 Done Done Sent false
14723 4878 Acked Write 1 Done Done Sent false
14724 4879 Acked Read 1 Done Done Sent false
14725 4880 Acked Read 4 Done Done Sent false
14726 4881 Acked Write 8 Done Done Sent false
14727 4882 Acked Read 4 Done Done Sent false
14728 4883 Acked Write 9 Done Done Sent false
14729 4884 Acked Write 6 Done Done Sent false
14730 4885 Acked Write 10 Done Done Sent false
14731 4886 Acked Read 3 Done Done Sent false
14732 4887 Acked Flush 0 Done Done Sent false
14733 4888 Acked Write 2 Done Done Sent false
14734 4889 Acked Read 10 Done Done Sent false
14735 4890 Acked Read 8 Done Done Sent false
14736 4891 Acked Read 9 Done Done Sent false
14737 4892 Acked Read 6 Done Done Sent false
14738 4893 Acked Read 9 Done Done Sent false
14739 4894 Acked Read 9 Done Done Sent false
14740 4895 Acked Write 1 Done Done Sent false
14741 4896 Acked Write 3 Done Done Sent false
14742 4897 Acked Write 8 Done Done Sent false
14743 4898 Acked Read 6 Done Done Sent false
14744 4899 Acked Read 8 Done Done Sent false
14745 4900 Acked Read 5 Done Done Sent false
14746 4901 Acked Write 10 Done Done Sent false
14747 4902 Acked Read 3 Done Done Sent false
14748 4903 Acked Read 10 Done Done Sent false
14749 4904 Acked Flush 0 Done Done Sent false
14750 4905 Acked Flush 0 Done Done Sent false
14751 4906 Acked Read 9 Done Done Sent false
14752 4907 Acked Write 7 Done Done Sent false
14753 4908 Acked Flush 0 Done Done Sent false
14754 4909 Acked Read 7 Done Done Sent false
14755 4910 Acked Read 5 Done Done Sent false
14756 4911 Acked Read 5 Done Done Sent false
14757 4912 Acked Write 4 Done Done Sent false
14758 4913 Acked Write 9 Done Done Sent false
14759 4914 Acked Read 7 Done Done Sent false
14760 4915 Acked Write 5 Done Done Sent false
14761 4916 Acked Read 6 Done Done Sent false
14762 4917 Acked Read 2 Done Done Sent false
14763 4918 Acked Write 3 Done Done Sent false
14764 4919 Acked Write 5 Done Done Sent false
14765 4920 Acked Read 7 Done Done Sent false
14766 4921 Acked Read 7 Done Done Sent false
14767 4922 Acked Read 4 Done Done Sent false
14768 4923 Acked Flush 0 Done Done Sent false
14769 STATES DS:0 DS:1 DS:2 TOTAL
14770 Sent 0 0 302 302
14771 Done 302 302 0 604
14772 Skipped 0 0 0 0
14773 Error 0 0 0 0
14774 Last Flush: 4923 4923 4621
14775 Downstairs last five completed: 4621 4620 4619 4618 4617
14776 Upstairs last five completed: 4923 4922 4921 4920 4919
14777 CLIENT: Up:0 ds:302 act:2
147782025-01-11T06:08:31.474ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147792025-01-11T06:08:31.475ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63423) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147802025-01-11T06:08:31.476ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147812025-01-11T06:08:31.477ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147822025-01-11T06:08:31.477ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147832025-01-11T06:08:31.477ZINFOcrucible: send last flush ID to this DS: 4621 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147842025-01-11T06:08:31.477ZINFOcrucible: Replied this last flush ID: 4621 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147852025-01-11T06:08:31.477ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147862025-01-11T06:08:31.477ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147872025-01-11T06:08:31.477ZINFOcrucible: [2] client re-new 302 jobs since flush 4621 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
147882025-01-11T06:08:31.477ZINFOcrucible: [2] Replayed 302 jobs since flush: 4621 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
14789 ----------------------------------------------------------------
14790 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14791 Downstairs last five completed: 4923 4922 4921 4920 4919
14792 Upstairs last five completed: 4923 4922 4921 4920 4919
14793 CLIENT: Up:0 ds:0 act:3
14794 Replay: All jobs finished
14795 Main task has work to do, go find it
14796 got dsc Stop(1)
14797 stop 1
14798 Jan 11 06:08:33.727 INFO [8820] Got stop action so:false kr:false
14799 request completed, latency_us: 329, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 60622e43-9088-4b09-a3af-8143757ade51, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
148002025-01-11T06:08:33.728ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
148012025-01-11T06:08:33.728ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
148022025-01-11T06:08:33.728ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
148032025-01-11T06:08:33.728ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
148042025-01-11T06:08:33.728ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
14805 Jan 11 06:08:33.728 INFO request completed, latency_us: 137, response_code: 200, uri: /state/cid/1, method: GET, req_id: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:08:33.728431205Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
14806 084e0535-2895-4979-add3-729b2949e496, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
14807 [8820] Exited with: ExitStatus(unix_wait_status(9))
14808 [8820][1] reports Exit
14809 Jan 11 06:08:37.730 INFO request completed, latency_us: 268, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0e0ca3c1-d4de-496e-9984-7fe3889d525b, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
14810 001/300 Write block 970 len 12288 data: 4 7 6
14811 002/300 Flush
14812 003/300 Read block 1330 len 28672
14813 004/300 Write block 4256 len 32768 data: 3 4 4 4 3 2 4 6
14814 005/300 Read block 506 len 24576
14815 006/300 Write block 2266 len 32768 data: 2 2 4 5 4 5 5 5
14816 007/300 Read block 236 len 20480
14817 008/300 Write block 1133 len 24576 data: 4 4 4 4 4 4
14818 009/300 Write block 3190 len 16384 data: 4 5 5 5
14819 010/300 Write block 4697 len 20480 data: 6 8 9 6 4
14820 011/300 Read block 2492 len 8192
14821 012/300 Flush
14822 013/300 Read block 1178 len 8192
14823 014/300 Read block 4541 len 32768
14824 015/300 Read block 2200 len 28672
14825 016/300 Read block 2292 len 36864
14826 017/300 Flush
14827 018/300 Read block 2527 len 8192
14828 019/300 Write block 3953 len 36864 data: 3 3 2 2 3 3 3 3 3
14829 020/300 Write block 1123 len 24576 data: 4 3 2 2 2 2
14830 021/300 Read block 2371 len 36864
14831 022/300 Write block 4070 len 20480 data: 4 5 6 5 7
14832 023/300 Read block 1957 len 12288
14833 024/300 Read block 2242 len 12288
14834 025/300 Flush
14835 026/300 Write block 4538 len 16384 data: 3 3 3 2
14836 027/300 Read block 3480 len 36864
14837 028/300 Read block 2224 len 8192
14838 029/300 Flush
14839 030/300 Read block 3501 len 32768
14840 031/300 Flush
14841 032/300 Flush
14842 033/300 Flush
14843 034/300 Read block 4066 len 8192
14844 035/300 Write block 3822 len 12288 data: 5 4 4
14845 036/300 Read block 1181 len 36864
14846 037/300 Flush
14847 038/300 Write block 3110 len 20480 data: 2 2 2 4 4
14848 039/300 Write block 4797 len 36864 data: 4 3 3 3 3 4 4 3 3
14849 040/300 Write block 1487 len 36864 data: 4 4 4 4 5 4 3 4 4
14850 041/300 Read block 2889 len 20480
14851 042/300 Read block 3634 len 20480
14852 043/300 Write block 3007 len 12288 data: 6 5 5
14853 044/300 Write block 1092 len 36864 data: 3 4 4 5 4 5 5 4 4
14854 045/300 Write block 2518 len 32768 data: 3 3 4 3 4 4 4 4
14855 046/300 Write block 4304 len 24576 data: 7 3 3 2 3 3
14856 047/300 Read block 4424 len 24576
14857 048/300 Read block 3426 len 4096
14858 049/300 Read block 2497 len 20480
14859 050/300 Write block 867 len 20480 data: 3 3 2 3 3
14860 051/300 Read block 4964 len 4096
14861 052/300 Write block 2898 len 36864 data: 4 4 4 4 4 5 5 5 4
14862 053/300 Read block 2422 len 32768
14863 054/300 Read block 632 len 36864
14864 055/300 Read block 2907 len 32768
14865 056/300 Read block 385 len 16384
14866 057/300 Read block 4441 len 16384
14867 058/300 Write block 1800 len 40960 data: 2 2 2 2 2 2 2 2 3 3
14868 059/300 Write block 3468 len 16384 data: 2 3 3 3
14869 060/300 Read block 4884 len 24576
14870 061/300 Write block 3581 len 4096 data: 5
14871 062/300 Write block 937 len 32768 data: 2 3 3 3 3 3 2 2
14872 063/300 Write block 984 len 28672 data: 5 5 5 4 3 3 2
14873 064/300 Write block 3283 len 12288 data: 4 3 3
14874 065/300 Read block 628 len 28672
14875 066/300 Read block 770 len 36864
14876 067/300 Read block 1229 len 40960
14877 068/300 Write block 43 len 8192 data: 7 7
14878 069/300 Read block 4316 len 8192
14879 070/300 Read block 2038 len 12288
14880 071/300 Read block 3332 len 40960
14881 072/300 Flush
14882 073/300 Write block 2197 len 20480 data: 5 5 5 4 4
14883 074/300 Read block 3349 len 12288
14884 075/300 Write block 724 len 40960 data: 8 6 6 6 6 5 4 3 3 3
14885 076/300 Read block 732 len 4096
14886 077/300 Read block 2289 len 28672
14887 078/300 Write block 2132 len 4096 data: 6
14888 079/300 Write block 3203 len 4096 data: 3
14889 080/300 Read block 1016 len 20480
14890 081/300 Read block 758 len 24576
14891 082/300 Read block 2427 len 16384
14892 083/300 Write block 3399 len 36864 data: 4 4 4 4 4 5 4 3 4
14893 084/300 Read block 444 len 4096
14894 085/300 Read block 98 len 24576
14895 086/300 Read block 2234 len 36864
14896 087/300 Read block 909 len 12288
14897 088/300 Read block 196 len 24576
14898 089/300 Write block 761 len 32768 data: 3 4 4 3 3 3 3 3
14899 090/300 Read block 4743 len 16384
14900 091/300 Write block 204 len 12288 data: 3 3 3
14901 092/300 Read block 60 len 12288
14902 093/300 Write block 1611 len 36864 data: 2 3 3 3 3 3 4 4 4
14903 094/300 Read block 38 len 8192
14904 095/300 Flush
14905 096/300 Read block 2553 len 32768
14906 097/300 Write block 2847 len 4096 data: 3
14907 098/300 Write block 1325 len 24576 data: 3 5 6 6 7 6
14908 099/300 Write block 4575 len 28672 data: 2 3 4 4 4 5 5
14909 100/300 Read block 2437 len 4096
14910 101/300 Write block 3521 len 24576 data: 3 3 3 5 5 5
14911 102/300 Read block 2071 len 24576
14912 103/300 Write block 840 len 8192 data: 5 5
14913 104/300 Read block 195 len 8192
14914 105/300 Read block 2721 len 28672
14915 106/300 Read block 3055 len 36864
14916 107/300 Write block 615 len 20480 data: 3 3 3 4 3
14917 108/300 Read block 3558 len 24576
14918 109/300 Write block 2393 len 36864 data: 5 5 6 5 5 4 4 4 4
14919 110/300 Read block 2195 len 20480
14920 111/300 Read block 862 len 12288
14921 112/300 Write block 1943 len 24576 data: 4 3 3 3 3 3
14922 113/300 Read block 1307 len 28672
14923 114/300 Read block 3274 len 20480
14924 115/300 Write block 1487 len 16384 data: 5 5 5 5
14925 116/300 Flush
14926 117/300 Read block 4612 len 32768
14927 118/300 Read block 1208 len 24576
14928 119/300 Read block 4966 len 8192
14929 120/300 Write block 791 len 28672 data: 5 5 5 4 4 3 2
14930 121/300 Read block 4284 len 20480
14931 122/300 Write block 3469 len 40960 data: 4 4 4 3 6 6 7 7 7 6
14932 123/300 Read block 30 len 32768
14933 124/300 Flush
14934 125/300 Read block 3570 len 20480
14935 126/300 Write block 1748 len 16384 data: 3 3 3 3
14936 127/300 Write block 4074 len 12288 data: 8 6 6
14937 128/300 Read block 4704 len 36864
14938 129/300 Write block 4234 len 16384 data: 2 2 3 4
14939 130/300 Read block 4816 len 20480
14940 131/300 Read block 3773 len 36864
14941 132/300 Read block 3458 len 24576
14942 133/300 Read block 4275 len 28672
14943 134/300 Read block 4688 len 8192
14944 135/300 Read block 4606 len 20480
14945 136/300 Read block 2776 len 8192
14946 137/300 Read block 794 len 20480
14947 138/300 Write block 1373 len 8192 data: 4 3
14948 139/300 Write block 3827 len 40960 data: 4 3 3 4 4 4 4 3 2 4
14949 140/300 Read block 4471 len 36864
14950 141/300 Write block 2421 len 16384 data: 4 4 4 3
14951 142/300 Flush
14952 143/300 Read block 1569 len 4096
14953 144/300 Read block 4811 len 16384
14954 145/300 Write block 2370 len 36864 data: 2 2 3 3 3 3 5 5 5
14955 146/300 Read block 1201 len 8192
14956 147/300 Write block 3702 len 4096 data: 3
14957 148/300 Write block 1556 len 40960 data: 3 3 2 3 3 4 4 4 4 5
14958 149/300 Write block 2686 len 12288 data: 4 2 3
14959 150/300 Write block 1767 len 16384 data: 2 2 3 3
14960 151/300 Write block 824 len 28672 data: 3 3 3 3 2 2 4
14961 152/300 Read block 3905 len 8192
14962 153/300 Read block 2244 len 12288
14963 154/300 Read block 4518 len 40960
14964 155/300 Read block 4013 len 8192
14965 156/300 Read block 1771 len 12288
14966 157/300 Read block 403 len 32768
14967 158/300 Flush
14968 159/300 Write block 4210 len 8192 data: 4 5
14969 160/300 Read block 1517 len 24576
14970 161/300 Flush
14971 162/300 Write block 4281 len 40960 data: 2 2 2 2 2 2 2 3 3 3
14972 163/300 Read block 4842 len 20480
14973 164/300 Write block 3868 len 4096 data: 4
14974 165/300 Read block 3335 len 36864
14975 166/300 Write block 2344 len 32768 data: 4 5 6 5 4 3 3 4
14976 167/300 Write block 4422 len 40960 data: 4 4 3 4 4 5 5 4 3 3
14977 168/300 Read block 2688 len 28672
14978 169/300 Write block 2501 len 12288 data: 2 2 2
14979 170/300 Read block 4643 len 36864
14980 171/300 Read block 3093 len 8192
14981 172/300 Read block 1567 len 20480
14982 173/300 Flush
14983 174/300 Write block 1535 len 12288 data: 4 4 4
14984 175/300 Read block 1303 len 20480
14985 176/300 Read block 1117 len 16384
14986 177/300 Write block 3139 len 40960 data: 3 3 2 2 2 2 3 3 5 5
14987 178/300 Flush
14988 179/300 Flush
14989 180/300 Flush
14990 181/300 Flush
14991 182/300 Flush
14992 183/300 Read block 4180 len 24576
14993 184/300 Write block 4662 len 4096 data: 3
14994 185/300 Read block 3257 len 24576
14995 186/300 Read block 1309 len 32768
14996 187/300 Read block 671 len 40960
14997 188/300 Read block 1164 len 24576
14998 189/300 Read block 3838 len 16384
14999 190/300 Read block 3063 len 28672
15000 191/300 Write block 277 len 36864 data: 4 5 7 6 7 8 6 5 6
15001 192/300 Read block 4581 len 16384
15002 193/300 Write block 1056 len 4096 data: 2
15003 194/300 Write block 1743 len 20480 data: 4 4 3 3 2
15004 195/300 Read block 3836 len 20480
15005 196/300 Read block 2330 len 32768
15006 197/300 Write block 2446 len 36864 data: 3 3 4 3 4 3 3 3 2
15007 198/300 Write block 2681 len 32768 data: 5 7 6 4 4 5 3 4
15008 199/300 Read block 1363 len 40960
15009 200/300 Flush
15010 201/300 Read block 1167 len 32768
15011 202/300 Read block 3903 len 4096
15012 203/300 Write block 3240 len 24576 data: 3 3 3 3 3 3
15013 204/300 Read block 538 len 16384
15014 205/300 Write block 4465 len 16384 data: 9 9 9 10
15015 206/300 Write block 2444 len 36864 data: 4 3 4 4 5 4 5 4 4
15016 207/300 Read block 1164 len 24576
15017 208/300 Write block 4914 len 40960 data: 2 2 2 2 2 2 2 2 4 4
15018 209/300 Read block 2065 len 36864
15019 210/300 Write block 4072 len 20480 data: 7 6 9 7 7
15020 211/300 Write block 2339 len 32768 data: 3 3 3 3 3 5 6 7
15021 212/300 Read block 4919 len 8192
15022 213/300 Flush
15023 214/300 Write block 2301 len 16384 data: 6 5 4 5
15024 215/300 Flush
15025 216/300 Read block 1668 len 8192
15026 217/300 Read block 1300 len 40960
15027 218/300 Read block 1875 len 12288
15028 219/300 Write block 247 len 20480 data: 5 4 3 4 5
15029 220/300 Write block 2079 len 24576 data: 7 6 7 8 9 8
15030 221/300 Write block 3020 len 16384 data: 3 2 2 4
15031 222/300 Flush
15032 223/300 Write block 4890 len 16384 data: 2 2 4 5
15033 224/300 Flush
15034 225/300 Read block 4677 len 12288
15035 226/300 Read block 1772 len 32768
15036 227/300 Write block 2999 len 40960 data: 3 4 4 4 4 6 6 6 7 6
15037 228/300 Write block 3236 len 36864 data: 2 3 4 3 4 4 4 4 4
15038 229/300 Read block 3136 len 32768
15039 230/300 Write block 2146 len 28672 data: 2 2 2 2 3 4 4
15040 231/300 Read block 1683 len 40960
15041 232/300 Read block 2849 len 12288
15042 233/300 Read block 2342 len 20480
15043 234/300 Write block 853 len 28672 data: 5 5 4 2 2 2 3
15044 235/300 Read block 2141 len 28672
15045 236/300 Read block 3110 len 24576
15046 237/300 Write block 3378 len 28672 data: 2 2 2 4 4 4 3
15047 238/300 Read block 2674 len 24576
15048 239/300 Flush
15049 240/300 Read block 1797 len 8192
15050 241/300 Read block 2297 len 36864
15051 242/300 Write block 747 len 4096 data: 3
15052 243/300 Read block 1638 len 32768
15053 244/300 Flush
15054 245/300 Flush
15055 246/300 Write block 1171 len 40960 data: 2 2 2 2 3 3 3 4 4 3
15056 247/300 Write block 2659 len 20480 data: 2 2 3 3 3
15057 248/300 Flush
15058 249/300 Read block 1074 len 32768
15059 250/300 Write block 4086 len 32768 data: 3 3 4 4 3 3 3 2
15060 251/300 Write block 1784 len 16384 data: 5 5 4 4
15061 252/300 Write block 1569 len 36864 data: 5 5 5 5 5 5 4 2 2
15062 253/300 Read block 1135 len 16384
15063 254/300 Read block 2121 len 8192
15064 255/300 Read block 4837 len 16384
15065 256/300 Read block 4940 len 16384
15066 257/300 Read block 3425 len 32768
15067 258/300 Read block 2188 len 32768
15068 259/300 Write block 2102 len 32768 data: 5 5 4 4 3 3 3 2
15069 260/300 Read block 3944 len 32768
15070 261/300 Write block 3151 len 28672 data: 5 4 5 5 4 4 4
15071 262/300 Read block 4616 len 28672
15072 263/300 Write block 1443 len 36864 data: 5 5 5 4 3 3 3 3 2
15073 264/300 Write block 4655 len 24576 data: 4 4 4 4 3 2
15074 265/300 Write block 934 len 16384 data: 2 2 2 3
15075 266/300 Write block 3903 len 28672 data: 4 4 3 4 3 3 4
15076 267/300 Read block 3958 len 36864
15077 268/300 Write block 318 len 32768 data: 2 2 3 4 5 7 7 8
15078 269/300 Read block 319 len 28672
15079 270/300 Read block 1486 len 24576
15080 271/300 Read block 777 len 24576
15081 272/300 Read block 272 len 36864
15082 273/300 Read block 1606 len 12288
15083 274/300 Write block 2772 len 4096 data: 5
15084 275/300 Read block 71 len 16384
15085 276/300 Read block 189 len 32768
15086 277/300 Write block 2431 len 36864 data: 7 6 5 4 4 2 2 2 4
15087 278/300 Read block 39 len 32768
15088 279/300 Write block 4948 len 12288 data: 5 5 6
15089 280/300 Write block 2005 len 40960 data: 4 4 3 3 3 3 4 3 3 3
15090 281/300 Flush
15091 282/300 Write block 4148 len 24576 data: 4 4 4 3 2 2
15092 283/300 Write block 4876 len 28672 data: 3 3 3 3 3 4 2
15093 284/300 Write block 1977 len 12288 data: 5 4 6
15094 285/300 Write block 2547 len 32768 data: 7 7 7 5 4 4 4 4
15095 286/300 Read block 3119 len 36864
15096 287/300 Write block 2436 len 20480 data: 3 3 3 5 4
15097 288/300 Write block 4031 len 20480 data: 4 3 4 3 3
15098 289/300 Read block 2603 len 24576
15099 290/300 Read block 1615 len 36864
15100 291/300 Write block 1225 len 8192 data: 4 4
15101 292/300 Read block 4615 len 32768
15102 293/300 Flush
15103 294/300 Read block 282 len 8192
15104 295/300 Read block 3455 len 4096
15105 296/300 Read block 2101 len 32768
15106 297/300 Read block 2320 len 20480
15107 298/300 Read block 505 len 24576
15108 299/300 Write block 3889 len 20480 data: 4 2 2 2 2
15109 300/300 Flush
15110 301/300 Write block 18 len 12288 data: 2 2 2
15111 Main task has work to do, go find it
15112 got dsc Start(1)
15113 start 1
15114 Jan 11 06:08:37.980 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/1, method: POST, req_id: 9b8d3a22-1738-4d0f-a678-57f58bee7966, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
15115 Replay: started 1, returned:Ok(())
15116 ----------------------------------------------------------------
15117 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15118 States: Active Offline Active
15119 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15120 4924 Acked Write 3 Done Sent Done false
15121 4925 Acked Flush 0 Done Sent Done false
15122 4926 Acked Read 7 Done Sent Done false
15123 4927 Acked Write 8 Done Sent Done false
15124 4928 Acked Read 6 Done Sent Done false
15125 4929 Acked Write 8 Done Sent Done false
15126 4930 Acked Read 5 Done Sent Done false
15127 4931 Acked Write 6 Done Sent Done false
15128 4932 Acked Write 4 Done Sent Done false
15129 4933 Acked Write 5 Done Sent Done false
15130 4934 Acked Read 2 Done Sent Done false
15131 4935 Acked Flush 0 Done Sent Done false
15132 4936 Acked Read 2 Done Sent Done false
15133 4937 Acked Read 8 Done Sent Done false
15134 4938 Acked Read 7 Done Sent Done false
15135 4939 Acked Read 9 Done Sent Done false
15136 4940 Acked Flush 0 Done Sent Done false
15137 4941 Acked Read 2 Done Sent Done false
15138 4942 Acked Write 9 Done Sent Done false
15139 4943 Acked Write 6 Done Sent Done false
15140 4944 Acked Read 9 Done Sent Done false
15141 4945 Acked Write 5 Done Sent Done false
15142 4946 Acked Read 3 Done Sent Done false
15143 4947 Acked Read 3 Done Sent Done false
15144 4948 Acked Flush 0 Done Sent Done false
15145 4949 Acked Write 4 Done Sent Done false
15146 4950 Acked Read 9 Done Sent Done false
15147 4951 Acked Read 2 Done Sent Done false
15148 4952 Acked Flush 0 Done Sent Done false
15149 4953 Acked Read 8 Done Sent Done false
15150 4954 Acked Flush 0 Done Sent Done false
15151 4955 Acked Flush 0 Done Sent Done false
15152 4956 Acked Flush 0 Done Sent Done false
15153 4957 Acked Read 2 Done Sent Done false
15154 4958 Acked Write 3 Done Sent Done false
15155 4959 Acked Read 9 Done Sent Done false
15156 4960 Acked Flush 0 Done Sent Done false
15157 4961 Acked Write 5 Done Sent Done false
15158 4962 Acked Write 9 Done Sent Done false
15159 4963 Acked Write 9 Done Sent Done false
15160 4964 Acked Read 5 Done Sent Done false
15161 4965 Acked Read 5 Done Sent Done false
15162 4966 Acked Write 3 Done Sent Done false
15163 4967 Acked Write 9 Done Sent Done false
15164 4968 Acked Write 8 Done Sent Done false
15165 4969 Acked Write 6 Done Sent Done false
15166 4970 Acked Read 6 Done Sent Done false
15167 4971 Acked Read 1 Done Sent Done false
15168 4972 Acked Read 5 Done Sent Done false
15169 4973 Acked Write 5 Done Sent Done false
15170 4974 Acked Read 1 Done Sent Done false
15171 4975 Acked Write 9 Done Sent Done false
15172 4976 Acked Read 8 Done Sent Done false
15173 4977 Acked Read 9 Done Sent Done false
15174 4978 Acked Read 8 Done Sent Done false
15175 4979 Acked Read 4 Done Sent Done false
15176 4980 Acked Read 4 Done Sent Done false
15177 4981 Acked Write 10 Done Sent Done false
15178 4982 Acked Write 4 Done Sent Done false
15179 4983 Acked Read 6 Done Sent Done false
15180 4984 Acked Write 1 Done Sent Done false
15181 4985 Acked Write 8 Done Sent Done false
15182 4986 Acked Write 7 Done Sent Done false
15183 4987 Acked Write 3 Done Sent Done false
15184 4988 Acked Read 7 Done Sent Done false
15185 4989 Acked Read 9 Done Sent Done false
15186 4990 Acked Read 10 Done Sent Done false
15187 4991 Acked Write 2 Done Sent Done false
15188 4992 Acked Read 2 Done Sent Done false
15189 4993 Acked Read 3 Done Sent Done false
15190 4994 Acked Read 10 Done Sent Done false
15191 4995 Acked Flush 0 Done Sent Done false
15192 4996 Acked Write 5 Done Sent Done false
15193 4997 Acked Read 3 Done Sent Done false
15194 4998 Acked Write 10 Done Sent Done false
15195 4999 Acked Read 1 Done Sent Done false
15196 5000 Acked Read 7 Done Sent Done false
15197 5001 Acked Write 1 Done Sent Done false
15198 5002 Acked Write 1 Done Sent Done false
15199 5003 Acked Read 5 Done Sent Done false
15200 5004 Acked Read 6 Done Sent Done false
15201 5005 Acked Read 4 Done Sent Done false
15202 5006 Acked Write 9 Done Sent Done false
15203 5007 Acked Read 1 Done Sent Done false
15204 5008 Acked Read 6 Done Sent Done false
15205 5009 Acked Read 9 Done Sent Done false
15206 5010 Acked Read 3 Done Sent Done false
15207 5011 Acked Read 6 Done Sent Done false
15208 5012 Acked Write 8 Done Sent Done false
15209 5013 Acked Read 4 Done Sent Done false
15210 5014 Acked Write 3 Done Sent Done false
15211 5015 Acked Read 3 Done Sent Done false
15212 5016 Acked Write 9 Done Sent Done false
15213 5017 Acked Read 2 Done Sent Done false
15214 5018 Acked Flush 0 Done Sent Done false
15215 5019 Acked Read 8 Done Sent Done false
15216 5020 Acked Write 1 Done Sent Done false
15217 5021 Acked Write 6 Done Sent Done false
15218 5022 Acked Write 7 Done Sent Done false
15219 5023 Acked Read 1 Done Sent Done false
15220 5024 Acked Write 6 Done Sent Done false
15221 5025 Acked Read 6 Done Sent Done false
15222 5026 Acked Write 2 Done Sent Done false
15223 5027 Acked Read 2 Done Sent Done false
15224 5028 Acked Read 7 Done Sent Done false
15225 5029 Acked Read 9 Done Sent Done false
15226 5030 Acked Write 5 Done Sent Done false
15227 5031 Acked Read 6 Done Sent Done false
15228 5032 Acked Write 9 Done Sent Done false
15229 5033 Acked Read 5 Done Sent Done false
15230 5034 Acked Read 3 Done Sent Done false
15231 5035 Acked Write 6 Done Sent Done false
15232 5036 Acked Read 7 Done Sent Done false
15233 5037 Acked Read 5 Done Sent Done false
15234 5038 Acked Write 4 Done Sent Done false
15235 5039 Acked Flush 0 Done Sent Done false
15236 5040 Acked Read 8 Done Sent Done false
15237 5041 Acked Read 6 Done Sent Done false
15238 5042 Acked Read 2 Done Sent Done false
15239 5043 Acked Write 7 Done Sent Done false
15240 5044 Acked Read 5 Done Sent Done false
15241 5045 Acked Write 10 Done Sent Done false
15242 5046 Acked Read 8 Done Sent Done false
15243 5047 Acked Flush 0 Done Sent Done false
15244 5048 Acked Read 5 Done Sent Done false
15245 5049 Acked Write 4 Done Sent Done false
15246 5050 Acked Write 3 Done Sent Done false
15247 5051 Acked Read 9 Done Sent Done false
15248 5052 Acked Write 4 Done Sent Done false
15249 5053 Acked Read 5 Done Sent Done false
15250 5054 Acked Read 9 Done Sent Done false
15251 5055 Acked Read 6 Done Sent Done false
15252 5056 Acked Read 7 Done Sent Done false
15253 5057 Acked Read 2 Done Sent Done false
15254 5058 Acked Read 5 Done Sent Done false
15255 5059 Acked Read 2 Done Sent Done false
15256 5060 Acked Read 5 Done Sent Done false
15257 5061 Acked Write 2 Done Sent Done false
15258 5062 Acked Write 10 Done Sent Done false
15259 5063 Acked Read 9 Done Sent Done false
15260 5064 Acked Write 4 Done Sent Done false
15261 5065 Acked Flush 0 Done Sent Done false
15262 5066 Acked Read 1 Done Sent Done false
15263 5067 Acked Read 4 Done Sent Done false
15264 5068 Acked Write 9 Done Sent Done false
15265 5069 Acked Read 2 Done Sent Done false
15266 5070 Acked Write 1 Done Sent Done false
15267 5071 Acked Write 10 Done Sent Done false
15268 5072 Acked Write 3 Done Sent Done false
15269 5073 Acked Write 4 Done Sent Done false
15270 5074 Acked Write 7 Done Sent Done false
15271 5075 Acked Read 2 Done Sent Done false
15272 5076 Acked Read 3 Done Sent Done false
15273 5077 Acked Read 10 Done Sent Done false
15274 5078 Acked Read 2 Done Sent Done false
15275 5079 Acked Read 3 Done Sent Done false
15276 5080 Acked Read 8 Done Sent Done false
15277 5081 Acked Flush 0 Done Sent Done false
15278 5082 Acked Write 2 Done Sent Done false
15279 5083 Acked Read 6 Done Sent Done false
15280 5084 Acked Flush 0 Done Sent Done false
15281 5085 Acked Write 10 Done Sent Done false
15282 5086 Acked Read 5 Done Sent Done false
15283 5087 Acked Write 1 Done Sent Done false
15284 5088 Acked Read 9 Done Sent Done false
15285 5089 Acked Write 8 Done Sent Done false
15286 5090 Acked Write 10 Done Sent Done false
15287 5091 Acked Read 7 Done Sent Done false
15288 5092 Acked Write 3 Done Sent Done false
15289 5093 Acked Read 9 Done Sent Done false
15290 5094 Acked Read 2 Done Sent Done false
15291 5095 Acked Read 5 Done Sent Done false
15292 5096 Acked Flush 0 Done Sent Done false
15293 5097 Acked Write 3 Done Sent Done false
15294 5098 Acked Read 5 Done Sent Done false
15295 5099 Acked Read 4 Done Sent Done false
15296 5100 Acked Write 10 Done Sent Done false
15297 5101 Acked Flush 0 Done Sent Done false
15298 5102 Acked Flush 0 Done Sent Done false
15299 5103 Acked Flush 0 Done Sent Done false
15300 5104 Acked Flush 0 Done Sent Done false
15301 5105 Acked Flush 0 Done Sent Done false
15302 5106 Acked Read 6 Done Sent Done false
15303 5107 Acked Write 1 Done Sent Done false
15304 5108 Acked Read 6 Done Sent Done false
15305 5109 Acked Read 8 Done Sent Done false
15306 5110 Acked Read 10 Done Sent Done false
15307 5111 Acked Read 6 Done Sent Done false
15308 5112 Acked Read 4 Done Sent Done false
15309 5113 Acked Read 7 Done Sent Done false
15310 5114 Acked Write 9 Done Sent Done false
15311 5115 Acked Read 4 Done Sent Done false
15312 5116 Acked Write 1 Done Sent Done false
15313 5117 Acked Write 5 Done Sent Done false
15314 5118 Acked Read 5 Done Sent Done false
15315 5119 Acked Read 8 Done Sent Done false
15316 5120 Acked Write 9 Done Sent Done false
15317 5121 Acked Write 8 Done Sent Done false
15318 5122 Acked Read 10 Done Sent Done false
15319 5123 Acked Flush 0 Done Sent Done false
15320 5124 Acked Read 8 Done Sent Done false
15321 5125 Acked Read 1 Done Sent Done false
15322 5126 Acked Write 6 Done Sent Done false
15323 5127 Acked Read 4 Done Sent Done false
15324 5128 Acked Write 4 Done Sent Done false
15325 5129 Acked Write 9 Done Sent Done false
15326 5130 Acked Read 6 Done Sent Done false
15327 5131 Acked Write 10 Done Sent Done false
15328 5132 Acked Read 9 Done Sent Done false
15329 5133 Acked Write 5 Done Sent Done false
15330 5134 Acked Write 8 Done Sent Done false
15331 5135 Acked Read 2 Done Sent Done false
15332 5136 Acked Flush 0 Done Sent Done false
15333 5137 Acked Write 4 Done Sent Done false
15334 5138 Acked Flush 0 Done Sent Done false
15335 5139 Acked Read 2 Done Sent Done false
15336 5140 Acked Read 10 Done Sent Done false
15337 5141 Acked Read 3 Done Sent Done false
15338 5142 Acked Write 5 Done Sent Done false
15339 5143 Acked Write 6 Done Sent Done false
15340 5144 Acked Write 4 Done Sent Done false
15341 5145 Acked Flush 0 Done Sent Done false
15342 5146 Acked Write 4 Done Sent Done false
15343 5147 Acked Flush 0 Done Sent Done false
15344 5148 Acked Read 3 Done Sent Done false
15345 5149 Acked Read 8 Done Sent Done false
15346 5150 Acked Write 10 Done Sent Done false
15347 5151 Acked Write 9 Done Sent Done false
15348 5152 Acked Read 8 Done Sent Done false
15349 5153 Acked Write 7 Done Sent Done false
15350 5154 Acked Read 10 Done Sent Done false
15351 5155 Acked Read 3 Done Sent Done false
15352 5156 Acked Read 5 Done Sent Done false
15353 5157 Acked Write 7 Done Sent Done false
15354 5158 Acked Read 7 Done Sent Done false
15355 5159 Acked Read 6 Done Sent Done false
15356 5160 Acked Write 7 Done Sent Done false
15357 5161 Acked Read 6 Done Sent Done false
15358 5162 Acked Flush 0 Done Sent Done false
15359 5163 Acked Read 2 Done Sent Done false
15360 5164 Acked Read 9 Done Sent Done false
15361 5165 Acked Write 1 Done Sent Done false
15362 5166 Acked Read 8 Done Sent Done false
15363 5167 Acked Flush 0 Done Sent Done false
15364 5168 Acked Flush 0 Done Sent Done false
15365 5169 Acked Write 10 Done Sent Done false
15366 5170 Acked Write 5 Done Sent Done false
15367 5171 Acked Flush 0 Done Sent Done false
15368 5172 Acked Read 8 Done Sent Done false
15369 5173 Acked Write 8 Done Sent Done false
15370 5174 Acked Write 4 Done Sent Done false
15371 5175 Acked Write 9 Done Sent Done false
15372 5176 Acked Read 4 Done Sent Done false
15373 5177 Acked Read 2 Done Sent Done false
15374 5178 Acked Read 4 Done Sent Done false
15375 5179 Acked Read 4 Done Sent Done false
15376 5180 Acked Read 8 Done Sent Done false
15377 5181 Acked Read 8 Done Sent Done false
15378 5182 Acked Write 8 Done Sent Done false
15379 5183 Acked Read 8 Done Sent Done false
15380 5184 Acked Write 7 Done Sent Done false
15381 5185 Acked Read 7 Done Sent Done false
15382 5186 Acked Write 9 Done Sent Done false
15383 5187 Acked Write 6 Done Sent Done false
15384 5188 Acked Write 4 Done Sent Done false
15385 5189 Acked Write 7 Done Sent Done false
15386 5190 Acked Read 9 Done Sent Done false
15387 5191 Acked Write 8 Done Sent Done false
15388 5192 Acked Read 7 Done Sent Done false
15389 5193 Acked Read 6 Done Sent Done false
15390 5194 Acked Read 6 Done Sent Done false
15391 5195 Acked Read 9 Done Sent Done false
15392 5196 Acked Read 3 Done Sent Done false
15393 5197 Acked Write 1 Done Sent Done false
15394 5198 Acked Read 4 Done Sent Done false
15395 5199 Acked Read 8 Done Sent Done false
15396 5200 Acked Write 9 Done Sent Done false
15397 5201 Acked Read 8 Done Sent Done false
15398 5202 Acked Write 3 Done Sent Done false
15399 5203 Acked Write 10 Done Sent Done false
15400 5204 Acked Flush 0 Done Sent Done false
15401 5205 Acked Write 6 Done Sent Done false
15402 5206 Acked Write 7 Done Sent Done false
15403 5207 Acked Write 3 Done Sent Done false
15404 5208 Acked Write 8 Done Sent Done false
15405 5209 Acked Read 9 Done Sent Done false
15406 5210 Acked Write 5 Done Sent Done false
15407 5211 Acked Write 5 Done Sent Done false
15408 5212 Acked Read 6 Done Sent Done false
15409 5213 Acked Read 9 Done Sent Done false
15410 5214 Acked Write 2 Done Sent Done false
15411 5215 Acked Read 8 Done Sent Done false
15412 5216 Acked Flush 0 Done Sent Done false
15413 5217 Acked Read 2 Done Sent Done false
15414 5218 Acked Read 1 Done Sent Done false
15415 5219 Acked Read 8 Done Sent Done false
15416 5220 Acked Read 5 Done Sent Done false
15417 5221 Acked Read 6 Done Sent Done false
15418 5222 Acked Write 5 Done Sent Done false
15419 5223 Acked Flush 0 Done Sent Done false
15420 5224 Acked Write 3 Done Sent Done false
15421 STATES DS:0 DS:1 DS:2 TOTAL
15422 Sent 0 301 0 301
15423 Done 301 0 301 602
15424 Skipped 0 0 0 0
15425 Error 0 0 0 0
15426 Last Flush: 5223 4923 5223
15427 Downstairs last five completed: 4923 4922 4921 4920 4919
15428 Upstairs last five completed: 5224 5223 5222 5221 5220
15429 CLIENT: Up:0 ds:301 act:2
15430 [8820] got start action
15431 [8820] I am going to restart
15432 Starting downstairs at port 8820
15433 Make output file at "/tmp/dsc/downstairs-8820.txt"
15434 [8820][1] reports Starting
15435 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
15436 [8820][1] reports Running
15437 ----------------------------------------------------------------
15438 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15439 States: Active Offline Active
15440 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15441 4924 Acked Write 3 Done Sent Done false
15442 4925 Acked Flush 0 Done Sent Done false
15443 4926 Acked Read 7 Done Sent Done false
15444 4927 Acked Write 8 Done Sent Done false
15445 4928 Acked Read 6 Done Sent Done false
15446 4929 Acked Write 8 Done Sent Done false
15447 4930 Acked Read 5 Done Sent Done false
15448 4931 Acked Write 6 Done Sent Done false
15449 4932 Acked Write 4 Done Sent Done false
15450 4933 Acked Write 5 Done Sent Done false
15451 4934 Acked Read 2 Done Sent Done false
15452 4935 Acked Flush 0 Done Sent Done false
15453 4936 Acked Read 2 Done Sent Done false
15454 4937 Acked Read 8 Done Sent Done false
15455 4938 Acked Read 7 Done Sent Done false
15456 4939 Acked Read 9 Done Sent Done false
15457 4940 Acked Flush 0 Done Sent Done false
15458 4941 Acked Read 2 Done Sent Done false
15459 4942 Acked Write 9 Done Sent Done false
15460 4943 Acked Write 6 Done Sent Done false
15461 4944 Acked Read 9 Done Sent Done false
15462 4945 Acked Write 5 Done Sent Done false
15463 4946 Acked Read 3 Done Sent Done false
15464 4947 Acked Read 3 Done Sent Done false
15465 4948 Acked Flush 0 Done Sent Done false
15466 4949 Acked Write 4 Done Sent Done false
15467 4950 Acked Read 9 Done Sent Done false
15468 4951 Acked Read 2 Done Sent Done false
15469 4952 Acked Flush 0 Done Sent Done false
15470 4953 Acked Read 8 Done Sent Done false
15471 4954 Acked Flush 0 Done Sent Done false
15472 4955 Acked Flush 0 Done Sent Done false
15473 4956 Acked Flush 0 Done Sent Done false
15474 4957 Acked Read 2 Done Sent Done false
15475 4958 Acked Write 3 Done Sent Done false
15476 4959 Acked Read 9 Done Sent Done false
15477 4960 Acked Flush 0 Done Sent Done false
15478 4961 Acked Write 5 Done Sent Done false
15479 4962 Acked Write 9 Done Sent Done false
15480 4963 Acked Write 9 Done Sent Done false
15481 4964 Acked Read 5 Done Sent Done false
15482 4965 Acked Read 5 Done Sent Done false
15483 4966 Acked Write 3 Done Sent Done false
15484 4967 Acked Write 9 Done Sent Done false
15485 4968 Acked Write 8 Done Sent Done false
15486 4969 Acked Write 6 Done Sent Done false
15487 4970 Acked Read 6 Done Sent Done false
15488 4971 Acked Read 1 Done Sent Done false
15489 4972 Acked Read 5 Done Sent Done false
15490 4973 Acked Write 5 Done Sent Done false
15491 4974 Acked Read 1 Done Sent Done false
15492 4975 Acked Write 9 Done Sent Done false
15493 4976 Acked Read 8 Done Sent Done false
15494 4977 Acked Read 9 Done Sent Done false
15495 4978 Acked Read 8 Done Sent Done false
15496 4979 Acked Read 4 Done Sent Done false
15497 4980 Acked Read 4 Done Sent Done false
15498 4981 Acked Write 10 Done Sent Done false
15499 4982 Acked Write 4 Done Sent Done false
15500 4983 Acked Read 6 Done Sent Done false
15501 4984 Acked Write 1 Done Sent Done false
15502 4985 Acked Write 8 Done Sent Done false
15503 4986 Acked Write 7 Done Sent Done false
15504 4987 Acked Write 3 Done Sent Done false
15505 4988 Acked Read 7 Done Sent Done false
15506 4989 Acked Read 9 Done Sent Done false
15507 4990 Acked Read 10 Done Sent Done false
15508 4991 Acked Write 2 Done Sent Done false
15509 4992 Acked Read 2 Done Sent Done false
15510 4993 Acked Read 3 Done Sent Done false
15511 4994 Acked Read 10 Done Sent Done false
15512 4995 Acked Flush 0 Done Sent Done false
15513 4996 Acked Write 5 Done Sent Done false
15514 4997 Acked Read 3 Done Sent Done false
15515 4998 Acked Write 10 Done Sent Done false
15516 4999 Acked Read 1 Done Sent Done false
15517 5000 Acked Read 7 Done Sent Done false
15518 5001 Acked Write 1 Done Sent Done false
15519 5002 Acked Write 1 Done Sent Done false
15520 5003 Acked Read 5 Done Sent Done false
15521 5004 Acked Read 6 Done Sent Done false
15522 5005 Acked Read 4 Done Sent Done false
15523 5006 Acked Write 9 Done Sent Done false
15524 5007 Acked Read 1 Done Sent Done false
15525 5008 Acked Read 6 Done Sent Done false
15526 5009 Acked Read 9 Done Sent Done false
15527 5010 Acked Read 3 Done Sent Done false
15528 5011 Acked Read 6 Done Sent Done false
15529 5012 Acked Write 8 Done Sent Done false
15530 5013 Acked Read 4 Done Sent Done false
15531 5014 Acked Write 3 Done Sent Done false
15532 5015 Acked Read 3 Done Sent Done false
15533 5016 Acked Write 9 Done Sent Done false
15534 5017 Acked Read 2 Done Sent Done false
15535 5018 Acked Flush 0 Done Sent Done false
15536 5019 Acked Read 8 Done Sent Done false
15537 5020 Acked Write 1 Done Sent Done false
15538 5021 Acked Write 6 Done Sent Done false
15539 5022 Acked Write 7 Done Sent Done false
15540 5023 Acked Read 1 Done Sent Done false
15541 5024 Acked Write 6 Done Sent Done false
15542 5025 Acked Read 6 Done Sent Done false
15543 5026 Acked Write 2 Done Sent Done false
15544 5027 Acked Read 2 Done Sent Done false
15545 5028 Acked Read 7 Done Sent Done false
15546 5029 Acked Read 9 Done Sent Done false
15547 5030 Acked Write 5 Done Sent Done false
15548 5031 Acked Read 6 Done Sent Done false
15549 5032 Acked Write 9 Done Sent Done false
15550 5033 Acked Read 5 Done Sent Done false
15551 5034 Acked Read 3 Done Sent Done false
15552 5035 Acked Write 6 Done Sent Done false
15553 5036 Acked Read 7 Done Sent Done false
15554 5037 Acked Read 5 Done Sent Done false
15555 5038 Acked Write 4 Done Sent Done false
15556 5039 Acked Flush 0 Done Sent Done false
15557 5040 Acked Read 8 Done Sent Done false
15558 5041 Acked Read 6 Done Sent Done false
15559 5042 Acked Read 2 Done Sent Done false
15560 5043 Acked Write 7 Done Sent Done false
15561 5044 Acked Read 5 Done Sent Done false
15562 5045 Acked Write 10 Done Sent Done false
15563 5046 Acked Read 8 Done Sent Done false
15564 5047 Acked Flush 0 Done Sent Done false
15565 5048 Acked Read 5 Done Sent Done false
15566 5049 Acked Write 4 Done Sent Done false
15567 5050 Acked Write 3 Done Sent Done false
15568 5051 Acked Read 9 Done Sent Done false
15569 5052 Acked Write 4 Done Sent Done false
15570 5053 Acked Read 5 Done Sent Done false
15571 5054 Acked Read 9 Done Sent Done false
15572 5055 Acked Read 6 Done Sent Done false
15573 5056 Acked Read 7 Done Sent Done false
15574 5057 Acked Read 2 Done Sent Done false
15575 5058 Acked Read 5 Done Sent Done false
15576 5059 Acked Read 2 Done Sent Done false
15577 5060 Acked Read 5 Done Sent Done false
15578 5061 Acked Write 2 Done Sent Done false
15579 5062 Acked Write 10 Done Sent Done false
15580 5063 Acked Read 9 Done Sent Done false
15581 5064 Acked Write 4 Done Sent Done false
15582 5065 Acked Flush 0 Done Sent Done false
15583 5066 Acked Read 1 Done Sent Done false
15584 5067 Acked Read 4 Done Sent Done false
15585 5068 Acked Write 9 Done Sent Done false
15586 5069 Acked Read 2 Done Sent Done false
15587 5070 Acked Write 1 Done Sent Done false
15588 5071 Acked Write 10 Done Sent Done false
15589 5072 Acked Write 3 Done Sent Done false
15590 5073 Acked Write 4 Done Sent Done false
15591 5074 Acked Write 7 Done Sent Done false
15592 5075 Acked Read 2 Done Sent Done false
15593 5076 Acked Read 3 Done Sent Done false
15594 5077 Acked Read 10 Done Sent Done false
15595 5078 Acked Read 2 Done Sent Done false
15596 5079 Acked Read 3 Done Sent Done false
15597 5080 Acked Read 8 Done Sent Done false
15598 5081 Acked Flush 0 Done Sent Done false
15599 5082 Acked Write 2 Done Sent Done false
15600 5083 Acked Read 6 Done Sent Done false
15601 5084 Acked Flush 0 Done Sent Done false
15602 5085 Acked Write 10 Done Sent Done false
15603 5086 Acked Read 5 Done Sent Done false
15604 5087 Acked Write 1 Done Sent Done false
15605 5088 Acked Read 9 Done Sent Done false
15606 5089 Acked Write 8 Done Sent Done false
15607 5090 Acked Write 10 Done Sent Done false
15608 5091 Acked Read 7 Done Sent Done false
15609 5092 Acked Write 3 Done Sent Done false
15610 5093 Acked Read 9 Done Sent Done false
15611 5094 Acked Read 2 Done Sent Done false
15612 5095 Acked Read 5 Done Sent Done false
15613 5096 Acked Flush 0 Done Sent Done false
15614 5097 Acked Write 3 Done Sent Done false
15615 5098 Acked Read 5 Done Sent Done false
15616 5099 Acked Read 4 Done Sent Done false
15617 5100 Acked Write 10 Done Sent Done false
15618 5101 Acked Flush 0 Done Sent Done false
15619 5102 Acked Flush 0 Done Sent Done false
15620 5103 Acked Flush 0 Done Sent Done false
15621 5104 Acked Flush 0 Done Sent Done false
15622 5105 Acked Flush 0 Done Sent Done false
15623 5106 Acked Read 6 Done Sent Done false
15624 5107 Acked Write 1 Done Sent Done false
15625 5108 Acked Read 6 Done Sent Done false
15626 5109 Acked Read 8 Done Sent Done false
15627 5110 Acked Read 10 Done Sent Done false
15628 5111 Acked Read 6 Done Sent Done false
15629 5112 Acked Read 4 Done Sent Done false
15630 5113 Acked Read 7 Done Sent Done false
15631 5114 Acked Write 9 Done Sent Done false
15632 5115 Acked Read 4 Done Sent Done false
15633 5116 Acked Write 1 Done Sent Done false
15634 5117 Acked Write 5 Done Sent Done false
15635 5118 Acked Read 5 Done Sent Done false
15636 5119 Acked Read 8 Done Sent Done false
15637 5120 Acked Write 9 Done Sent Done false
15638 5121 Acked Write 8 Done Sent Done false
15639 5122 Acked Read 10 Done Sent Done false
15640 5123 Acked Flush 0 Done Sent Done false
15641 5124 Acked Read 8 Done Sent Done false
15642 5125 Acked Read 1 Done Sent Done false
15643 5126 Acked Write 6 Done Sent Done false
15644 5127 Acked Read 4 Done Sent Done false
15645 5128 Acked Write 4 Done Sent Done false
15646 5129 Acked Write 9 Done Sent Done false
15647 5130 Acked Read 6 Done Sent Done false
15648 5131 Acked Write 10 Done Sent Done false
15649 5132 Acked Read 9 Done Sent Done false
15650 5133 Acked Write 5 Done Sent Done false
15651 5134 Acked Write 8 Done Sent Done false
15652 5135 Acked Read 2 Done Sent Done false
15653 5136 Acked Flush 0 Done Sent Done false
15654 5137 Acked Write 4 Done Sent Done false
15655 5138 Acked Flush 0 Done Sent Done false
15656 5139 Acked Read 2 Done Sent Done false
15657 5140 Acked Read 10 Done Sent Done false
15658 5141 Acked Read 3 Done Sent Done false
15659 5142 Acked Write 5 Done Sent Done false
15660 5143 Acked Write 6 Done Sent Done false
15661 5144 Acked Write 4 Done Sent Done false
15662 5145 Acked Flush 0 Done Sent Done false
15663 5146 Acked Write 4 Done Sent Done false
15664 5147 Acked Flush 0 Done Sent Done false
15665 5148 Acked Read 3 Done Sent Done false
15666 5149 Acked Read 8 Done Sent Done false
15667 5150 Acked Write 10 Done Sent Done false
15668 5151 Acked Write 9 Done Sent Done false
15669 5152 Acked Read 8 Done Sent Done false
15670 5153 Acked Write 7 Done Sent Done false
15671 5154 Acked Read 10 Done Sent Done false
15672 5155 Acked Read 3 Done Sent Done false
15673 5156 Acked Read 5 Done Sent Done false
15674 5157 Acked Write 7 Done Sent Done false
15675 5158 Acked Read 7 Done Sent Done false
15676 5159 Acked Read 6 Done Sent Done false
15677 5160 Acked Write 7 Done Sent Done false
15678 5161 Acked Read 6 Done Sent Done false
15679 5162 Acked Flush 0 Done Sent Done false
15680 5163 Acked Read 2 Done Sent Done false
15681 5164 Acked Read 9 Done Sent Done false
15682 5165 Acked Write 1 Done Sent Done false
15683 5166 Acked Read 8 Done Sent Done false
15684 5167 Acked Flush 0 Done Sent Done false
15685 5168 Acked Flush 0 Done Sent Done false
15686 5169 Acked Write 10 Done Sent Done false
15687 5170 Acked Write 5 Done Sent Done false
15688 5171 Acked Flush 0 Done Sent Done false
15689 5172 Acked Read 8 Done Sent Done false
15690 5173 Acked Write 8 Done Sent Done false
15691 5174 Acked Write 4 Done Sent Done false
15692 5175 Acked Write 9 Done Sent Done false
15693 5176 Acked Read 4 Done Sent Done false
15694 5177 Acked Read 2 Done Sent Done false
15695 5178 Acked Read 4 Done Sent Done false
15696 5179 Acked Read 4 Done Sent Done false
15697 5180 Acked Read 8 Done Sent Done false
15698 5181 Acked Read 8 Done Sent Done false
15699 5182 Acked Write 8 Done Sent Done false
15700 5183 Acked Read 8 Done Sent Done false
15701 5184 Acked Write 7 Done Sent Done false
15702 5185 Acked Read 7 Done Sent Done false
15703 5186 Acked Write 9 Done Sent Done false
15704 5187 Acked Write 6 Done Sent Done false
15705 5188 Acked Write 4 Done Sent Done false
15706 5189 Acked Write 7 Done Sent Done false
15707 5190 Acked Read 9 Done Sent Done false
15708 5191 Acked Write 8 Done Sent Done false
15709 5192 Acked Read 7 Done Sent Done false
15710 5193 Acked Read 6 Done Sent Done false
15711 5194 Acked Read 6 Done Sent Done false
15712 5195 Acked Read 9 Done Sent Done false
15713 5196 Acked Read 3 Done Sent Done false
15714 5197 Acked Write 1 Done Sent Done false
15715 5198 Acked Read 4 Done Sent Done false
15716 5199 Acked Read 8 Done Sent Done false
15717 5200 Acked Write 9 Done Sent Done false
15718 5201 Acked Read 8 Done Sent Done false
15719 5202 Acked Write 3 Done Sent Done false
15720 5203 Acked Write 10 Done Sent Done false
15721 5204 Acked Flush 0 Done Sent Done false
15722 5205 Acked Write 6 Done Sent Done false
15723 5206 Acked Write 7 Done Sent Done false
15724 5207 Acked Write 3 Done Sent Done false
15725 5208 Acked Write 8 Done Sent Done false
15726 5209 Acked Read 9 Done Sent Done false
15727 5210 Acked Write 5 Done Sent Done false
15728 5211 Acked Write 5 Done Sent Done false
15729 5212 Acked Read 6 Done Sent Done false
15730 5213 Acked Read 9 Done Sent Done false
15731 5214 Acked Write 2 Done Sent Done false
15732 5215 Acked Read 8 Done Sent Done false
15733 5216 Acked Flush 0 Done Sent Done false
15734 5217 Acked Read 2 Done Sent Done false
15735 5218 Acked Read 1 Done Sent Done false
15736 5219 Acked Read 8 Done Sent Done false
15737 5220 Acked Read 5 Done Sent Done false
15738 5221 Acked Read 6 Done Sent Done false
15739 5222 Acked Write 5 Done Sent Done false
15740 5223 Acked Flush 0 Done Sent Done false
15741 5224 Acked Write 3 Done Sent Done false
15742 5225 Acked Flush 0 Done Sent Done false
15743 STATES DS:0 DS:1 DS:2 TOTAL
15744 Sent 0 302 0 302
15745 Done 302 0 302 604
15746 Skipped 0 0 0 0
15747 Error 0 0 0 0
15748 Last Flush: 5225 4923 5225
15749 Downstairs last five completed: 4923 4922 4921 4920 4919
15750 Upstairs last five completed: 5225 5224 5223 5222 5221
15751 CLIENT: Up:0 ds:302 act:2
157522025-01-11T06:08:43.729ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157532025-01-11T06:08:43.729ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46270) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157542025-01-11T06:08:43.730ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157552025-01-11T06:08:43.730ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157562025-01-11T06:08:43.730ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157572025-01-11T06:08:43.730ZINFOcrucible: send last flush ID to this DS: 4923 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157582025-01-11T06:08:43.731ZINFOcrucible: Replied this last flush ID: 4923 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157592025-01-11T06:08:43.731ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157602025-01-11T06:08:43.731ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157612025-01-11T06:08:43.731ZINFOcrucible: [1] client re-new 302 jobs since flush 4923 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157622025-01-11T06:08:43.731ZINFOcrucible: [1] Replayed 302 jobs since flush: 4923 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
15763 ----------------------------------------------------------------
15764 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15765 Downstairs last five completed: 5225 5224 5223 5222 5221
15766 Upstairs last five completed: 5225 5224 5223 5222 5221
15767 CLIENT: Up:0 ds:0 act:3
15768 Replay: All jobs finished
15769 Main task has work to do, go find it
15770 got dsc Stop(1)
15771 stop 1
15772 Jan 11 06:08:45.985[8820] Got stop action so:false kr:false
15773 INFO request completed, latency_us: 371, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 7e4ef7f6-4ee5-4b79-8866-c4c8c6ed0726, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
157742025-01-11T06:08:45.986ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157752025-01-11T06:08:45.986ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
15776 Jan 11 06:08:45.986 INFO request completed, latency_us: 117, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:08:45.986769197Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
15777 response_code: 200, uri: /state/cid/1, method: GET, req_id: 6907628d-1c78-44a8-9187-d0a61cf90df5, remote_addr: 127.0.0.1:59925, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:08:45.986798523Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
15778 local_addr: 127.0.0.1:9998
157792025-01-11T06:08:45.986ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
157802025-01-11T06:08:45.986ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
15781 [8820] Exited with: ExitStatus(unix_wait_status(9))
15782 [8820][1] reports Exit
15783 Jan 11 06:08:49.989 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/1, method: GET, req_id: 97d9aa4f-bdc7-48d6-bede-b254a122922e, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
15784 001/300 Write block 151 len 32768 data: 2 2 2 2 2 2 2 2
15785 002/300 Read block 816 len 28672
15786 003/300 Write block 4266 len 40960 data: 7 6 6 6 4 5 5 4 3 3
15787 004/300 Write block 829 len 16384 data: 3 5 4 4
15788 005/300 Read block 985 len 36864
15789 006/300 Read block 3914 len 12288
15790 007/300 Write block 386 len 36864 data: 6 6 6 6 6 6 6 6 5
15791 008/300 Read block 3181 len 4096
15792 009/300 Flush
15793 010/300 Write block 428 len 40960 data: 3 3 3 3 4 3 3 3 3 3
15794 011/300 Write block 496 len 36864 data: 4 5 6 6 6 6 6 6 6
15795 012/300 Read block 1727 len 36864
15796 013/300 Read block 4794 len 20480
15797 014/300 Write block 4961 len 32768 data: 2 2 3 3 3 3 3 3
15798 015/300 Write block 1989 len 36864 data: 4 5 6 5 7 6 4 4 4
15799 016/300 Write block 3244 len 24576 data: 5 4 4 3 3 4
15800 017/300 Write block 2347 len 4096 data: 6
15801 018/300 Read block 3407 len 8192
15802 019/300 Write block 687 len 8192 data: 5 6
15803 020/300 Read block 2778 len 40960
15804 021/300 Read block 4340 len 12288
15805 022/300 Read block 4935 len 28672
15806 023/300 Read block 4373 len 28672
15807 024/300 Write block 95 len 24576 data: 6 5 5 5 5 4
15808 025/300 Write block 1893 len 36864 data: 3 3 3 4 3 3 5 5 4
15809 026/300 Write block 3360 len 28672 data: 2 2 2 2 3 3 6
15810 027/300 Write block 310 len 12288 data: 3 3 3
15811 028/300 Read block 2028 len 36864
15812 029/300 Read block 1861 len 4096
15813 030/300 Read block 3683 len 40960
15814 031/300 Write block 4008 len 36864 data: 3 2 2 3 3 3 3 3 3
15815 032/300 Write block 3491 len 12288 data: 3 3 3
15816 033/300 Read block 2905 len 28672
15817 034/300 Write block 189 len 16384 data: 6 5 5 6
15818 035/300 Read block 3938 len 40960
15819 036/300 Read block 3576 len 36864
15820 037/300 Flush
15821 038/300 Read block 4000 len 8192
15822 039/300 Write block 1968 len 8192 data: 2 2
15823 040/300 Write block 2059 len 28672 data: 5 4 4 4 3 3 3
15824 041/300 Flush
15825 042/300 Read block 4014 len 40960
15826 043/300 Write block 1757 len 36864 data: 4 4 3 3 2 2 2 3 3
15827 044/300 Read block 1676 len 28672
15828 045/300 Write block 1607 len 40960 data: 2 2 2 3 3 4 4 4 4 4
15829 046/300 Flush
15830 047/300 Write block 1829 len 12288 data: 8 8 6
15831 048/300 Write block 233 len 8192 data: 4 4
15832 049/300 Read block 3857 len 32768
15833 050/300 Read block 2529 len 28672
15834 051/300 Read block 968 len 36864
15835 052/300 Read block 4319 len 20480
15836 053/300 Write block 3697 len 24576 data: 5 5 5 5 5 4
15837 054/300 Read block 3712 len 28672
15838 055/300 Flush
15839 056/300 Write block 4483 len 12288 data: 3 3 3
15840 057/300 Write block 2886 len 20480 data: 3 3 3 3 3
15841 058/300 Read block 4031 len 4096
15842 059/300 Read block 689 len 4096
15843 060/300 Read block 3422 len 40960
15844 061/300 Read block 2398 len 16384
15845 062/300 Read block 202 len 32768
15846 063/300 Read block 3427 len 4096
15847 064/300 Write block 3399 len 8192 data: 5 5
15848 065/300 Read block 3293 len 28672
15849 066/300 Read block 3025 len 28672
15850 067/300 Flush
15851 068/300 Read block 2198 len 4096
15852 069/300 Read block 917 len 12288
15853 070/300 Flush
15854 071/300 Write block 667 len 20480 data: 4 3 4 3 4
15855 072/300 Read block 2918 len 28672
15856 073/300 Write block 2880 len 12288 data: 3 2 2
15857 074/300 Write block 1542 len 16384 data: 2 2 3 3
15858 075/300 Flush
15859 076/300 Write block 4957 len 12288 data: 5 4 4
15860 077/300 Read block 4736 len 32768
15861 078/300 Read block 3914 len 24576
15862 079/300 Read block 4968 len 24576
15863 080/300 Write block 754 len 28672 data: 5 4 4 4 5 4 3
15864 081/300 Read block 1777 len 36864
15865 082/300 Write block 4534 len 24576 data: 3 3 3 3 4 4
15866 083/300 Read block 4687 len 40960
15867 084/300 Write block 4127 len 24576 data: 5 6 5 6 5 5
15868 085/300 Read block 3737 len 12288
15869 086/300 Write block 2449 len 12288 data: 5 6 5
15870 087/300 Read block 4856 len 16384
15871 088/300 Flush
15872 089/300 Read block 803 len 20480
15873 090/300 Read block 4644 len 16384
15874 091/300 Write block 4844 len 16384 data: 3 3 2 2
15875 092/300 Write block 933 len 24576 data: 2 3 3 3 4 4
15876 093/300 Read block 1352 len 24576
15877 094/300 Flush
15878 095/300 Read block 579 len 28672
15879 096/300 Flush
15880 097/300 Read block 2380 len 32768
15881 098/300 Read block 3639 len 36864
15882 099/300 Write block 495 len 24576 data: 4 5 6 7 7 7
15883 100/300 Write block 4772 len 28672 data: 4 5 6 5 4 4 3
15884 101/300 Read block 2046 len 16384
15885 102/300 Read block 3352 len 12288
15886 103/300 Write block 2067 len 16384 data: 4 4 4 4
15887 104/300 Write block 1124 len 20480 data: 4 3 3 3 3
15888 105/300 Read block 1115 len 28672
15889 106/300 Read block 227 len 40960
15890 107/300 Read block 2832 len 36864
15891 108/300 Write block 3432 len 16384 data: 4 4 4 3
15892 109/300 Read block 936 len 8192
15893 110/300 Write block 4667 len 40960 data: 5 5 5 5 6 5 6 6 4 3
15894 111/300 Flush
15895 112/300 Flush
15896 113/300 Read block 2135 len 32768
15897 114/300 Write block 2706 len 24576 data: 4 4 4 3 3 3
15898 115/300 Read block 4408 len 16384
15899 116/300 Write block 336 len 16384 data: 5 4 4 4
15900 117/300 Read block 4922 len 12288
15901 118/300 Read block 621 len 12288
15902 119/300 Read block 3233 len 24576
15903 120/300 Read block 4999 len 4096
15904 121/300 Flush
15905 122/300 Read block 943 len 16384
15906 123/300 Read block 3498 len 4096
15907 124/300 Read block 2999 len 16384
15908 125/300 Flush
15909 126/300 Flush
15910 127/300 Read block 2331 len 28672
15911 128/300 Read block 1424 len 8192
15912 129/300 Read block 172 len 20480
15913 130/300 Read block 1862 len 40960
15914 131/300 Read block 1961 len 40960
15915 132/300 Write block 4040 len 36864 data: 3 4 4 5 5 5 5 4 4
15916 133/300 Read block 1794 len 12288
15917 134/300 Read block 3522 len 36864
15918 135/300 Read block 2773 len 32768
15919 136/300 Read block 646 len 36864
15920 137/300 Read block 996 len 12288
15921 138/300 Read block 4892 len 16384
15922 139/300 Read block 4627 len 16384
15923 140/300 Read block 1119 len 28672
15924 141/300 Write block 1901 len 36864 data: 5 4 4 4 4 4 4 4 2
15925 142/300 Write block 1470 len 12288 data: 3 5 4
15926 143/300 Write block 2580 len 32768 data: 3 4 3 3 2 2 2 3
15927 144/300 Flush
15928 145/300 Write block 3817 len 16384 data: 4 4 5 5
15929 146/300 Read block 3888 len 4096
15930 147/300 Flush
15931 148/300 Flush
15932 149/300 Write block 4183 len 28672 data: 2 2 2 2 2 3 3
15933 150/300 Write block 2932 len 4096 data: 7
15934 151/300 Write block 3151 len 24576 data: 6 5 6 6 5 5
15935 152/300 Read block 3615 len 40960
15936 153/300 Write block 2933 len 24576 data: 5 6 6 5 5 5
15937 154/300 Flush
15938 155/300 Write block 1230 len 20480 data: 4 5 5 5 5
15939 156/300 Read block 2950 len 16384
15940 157/300 Read block 1909 len 16384
15941 158/300 Write block 1987 len 20480 data: 3 3 5 6 7
15942 159/300 Write block 786 len 36864 data: 3 3 4 4 5 6 6 6 5
15943 160/300 Read block 4120 len 36864
15944 161/300 Write block 4662 len 20480 data: 4 3 3 3 4
15945 162/300 Write block 1540 len 16384 data: 3 2 3 3
15946 163/300 Read block 1708 len 24576
15947 164/300 Flush
15948 165/300 Read block 1145 len 20480
15949 166/300 Write block 1701 len 28672 data: 5 5 5 5 5 4 4
15950 167/300 Read block 3679 len 32768
15951 168/300 Write block 2716 len 36864 data: 3 3 2 2 2 2 2 2 2
15952 169/300 Write block 3082 len 36864 data: 2 2 3 4 4 4 4 7 7
15953 170/300 Flush
15954 171/300 Read block 4031 len 12288
15955 172/300 Read block 4752 len 16384
15956 173/300 Read block 4903 len 12288
15957 174/300 Write block 1682 len 24576 data: 4 4 4 5 4 4
15958 175/300 Read block 4332 len 4096
15959 176/300 Flush
15960 177/300 Write block 3631 len 12288 data: 4 4 4
15961 178/300 Write block 4733 len 16384 data: 3 2 2 2
15962 179/300 Write block 3446 len 8192 data: 5 5
15963 180/300 Write block 1221 len 40960 data: 4 5 4 4 5 5 5 5 4 5
15964 181/300 Flush
15965 182/300 Flush
15966 183/300 Read block 3715 len 20480
15967 184/300 Write block 278 len 36864 data: 6 8 7 8 9 7 6 7 6
15968 185/300 Read block 2591 len 24576
15969 186/300 Write block 1177 len 28672 data: 4 5 5 4 3 4 6
15970 187/300 Write block 2648 len 4096 data: 6
15971 188/300 Write block 447 len 24576 data: 3 3 3 2 2 2
15972 189/300 Read block 4696 len 20480
15973 190/300 Read block 4621 len 40960
15974 191/300 Flush
15975 192/300 Write block 2469 len 32768 data: 3 3 3 3 2 3 3 3
15976 193/300 Write block 2981 len 28672 data: 3 3 2 2 3 3 3
15977 194/300 Read block 1004 len 4096
15978 195/300 Read block 4670 len 40960
15979 196/300 Write block 647 len 28672 data: 2 4 4 5 4 4 4
15980 197/300 Read block 633 len 40960
15981 198/300 Read block 3821 len 40960
15982 199/300 Write block 4170 len 4096 data: 4
15983 200/300 Write block 4253 len 16384 data: 3 3 3 4
15984 201/300 Read block 2188 len 28672
15985 202/300 Read block 49 len 32768
15986 203/300 Read block 3876 len 32768
15987 204/300 Read block 2466 len 40960
15988 205/300 Write block 1326 len 16384 data: 6 7 7 8
15989 206/300 Write block 3711 len 24576 data: 5 4 4 4 4 5
15990 207/300 Read block 2897 len 8192
15991 208/300 Read block 2672 len 40960
15992 209/300 Read block 4102 len 4096
15993 210/300 Read block 961 len 24576
15994 211/300 Write block 2935 len 12288 data: 7 6 6
15995 212/300 Read block 185 len 24576
15996 213/300 Read block 4770 len 16384
15997 214/300 Write block 2157 len 8192 data: 4 3
15998 215/300 Read block 935 len 24576
15999 216/300 Read block 3879 len 16384
16000 217/300 Flush
16001 218/300 Write block 2030 len 8192 data: 4 3
16002 219/300 Write block 4507 len 40960 data: 7 8 8 7 6 5 5 3 4 4
16003 220/300 Flush
16004 221/300 Read block 4738 len 28672
16005 222/300 Read block 3136 len 16384
16006 223/300 Read block 1274 len 32768
16007 224/300 Write block 2522 len 28672 data: 5 5 5 5 3 4 4
16008 225/300 Flush
16009 226/300 Write block 3476 len 8192 data: 8 8
16010 227/300 Write block 1700 len 16384 data: 6 6 6 6
16011 228/300 Write block 109 len 32768 data: 4 4 4 3 3 3 3 3
16012 229/300 Write block 4665 len 36864 data: 4 5 6 6 6 6 7 6 7
16013 230/300 Write block 2524 len 40960 data: 6 6 4 5 5 3 2 2 3 4
16014 231/300 Flush
16015 232/300 Read block 636 len 32768
16016 233/300 Write block 3088 len 20480 data: 5 8 8 7 7
16017 234/300 Write block 4870 len 12288 data: 3 3 3
16018 235/300 Read block 3282 len 12288
16019 236/300 Read block 3944 len 4096
16020 237/300 Read block 4596 len 32768
16021 238/300 Read block 3905 len 32768
16022 239/300 Write block 2688 len 32768 data: 5 3 3 3 3 3 3 3
16023 240/300 Read block 2342 len 8192
16024 241/300 Write block 3202 len 20480 data: 3 4 4 3 4
16025 242/300 Flush
16026 243/300 Read block 2246 len 4096
16027 244/300 Write block 248 len 12288 data: 5 4 5
16028 245/300 Read block 2307 len 16384
16029 246/300 Write block 3530 len 36864 data: 5 5 3 4 4 3 3 3 3
16030 247/300 Read block 1126 len 8192
16031 248/300 Write block 3432 len 12288 data: 5 5 5
16032 249/300 Read block 1491 len 16384
16033 250/300 Read block 4690 len 36864
16034 251/300 Read block 2116 len 36864
16035 252/300 Write block 804 len 4096 data: 3
16036 253/300 Read block 3080 len 32768
16037 254/300 Read block 2439 len 36864
16038 255/300 Read block 3643 len 4096
16039 256/300 Write block 2745 len 20480 data: 3 3 4 4 4
16040 257/300 Read block 4765 len 36864
16041 258/300 Read block 4842 len 24576
16042 259/300 Read block 2369 len 8192
16043 260/300 Write block 992 len 40960 data: 2 3 3 3 4 4 3 3 3 6
16044 261/300 Write block 1430 len 16384 data: 5 4 2 2
16045 262/300 Read block 3386 len 40960
16046 263/300 Flush
16047 264/300 Write block 30 len 40960 data: 4 3 4 4 5 5 4 6 5 7
16048 265/300 Flush
16049 266/300 Write block 719 len 36864 data: 3 4 4 7 8 9 7 7 7
16050 267/300 Read block 962 len 8192
16051 268/300 Read block 3883 len 40960
16052 269/300 Read block 4048 len 16384
16053 270/300 Read block 4007 len 28672
16054 271/300 Read block 4063 len 16384
16055 272/300 Write block 490 len 24576 data: 5 6 5 4 4 5
16056 273/300 Write block 3513 len 8192 data: 6 6
16057 274/300 Write block 562 len 8192 data: 2 2
16058 275/300 Write block 1414 len 16384 data: 4 4 4 3
16059 276/300 Write block 1452 len 36864 data: 2 2 2 2 2 2 2 2 2
16060 277/300 Write block 4453 len 36864 data: 7 7 8 5 3 3 3 4 5
16061 278/300 Read block 168 len 28672
16062 279/300 Write block 3087 len 8192 data: 5 6
16063 280/300 Write block 4567 len 32768 data: 4 4 4 2 2 2 3 3
16064 281/300 Write block 354 len 32768 data: 4 6 6 6 6 5 5 4
16065 282/300 Flush
16066 283/300 Write block 2113 len 4096 data: 2
16067 284/300 Read block 1107 len 8192
16068 285/300 Read block 4880 len 16384
16069 286/300 Read block 271 len 32768
16070 287/300 Read block 2390 len 24576
16071 288/300 Write block 2083 len 40960 data: 10 9 6 4 4 4 3 3 3 4
16072 289/300 Read block 3767 len 12288
16073 290/300 Write block 223 len 8192 data: 2 2
16074 291/300 Read block 4993 len 8192
16075 292/300 Write block 4558 len 32768 data: 4 5 5 5 6 6 5 5
16076 293/300 Read block 2964 len 4096
16077 294/300 Write block 1937 len 4096 data: 5
16078 295/300 Read block 4846 len 20480
16079 296/300 Write block 1067 len 40960 data: 3 4 4 4 4 3 2 2 2 2
16080 297/300 Write block 2755 len 40960 data: 3 3 3 3 3 4 3 3 4 3
16081 298/300 Read block 1932 len 16384
16082 299/300 Read block 2347 len 20480
16083 300/300 Read block 4043 len 24576
16084 301/300 Write block 4589 len 28672 data: 4 3 3 3 3 3 3
16085 Main task has work to do, go find it
16086 got dsc Start(1)
16087 start 1
16088 Jan 11 06:08:50.240 INFO request completed, latency_us: 274, response_code: 204, uri: /start/cid/1, method: POST, req_id: 9556f496-afe7-43db-bef3-36b47b5cc685, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
16089 Replay: started 1, returned:Ok(())
16090 ----------------------------------------------------------------
16091 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16092 States: Active Offline Active
16093 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16094 5226 Acked Write 8 Done Sent Done false
16095 5227 Acked Read 7 Done Sent Done false
16096 5228 Acked Write 10 Done Sent Done false
16097 5229 Acked Write 4 Done Sent Done false
16098 5230 Acked Read 9 Done Sent Done false
16099 5231 Acked Read 3 Done Sent Done false
16100 5232 Acked Write 9 Done Sent Done false
16101 5233 Acked Read 1 Done Sent Done false
16102 5234 Acked Flush 0 Done Sent Done false
16103 5235 Acked Write 10 Done Sent Done false
16104 5236 Acked Write 9 Done Sent Done false
16105 5237 Acked Read 9 Done Sent Done false
16106 5238 Acked Read 5 Done Sent Done false
16107 5239 Acked Write 8 Done Sent Done false
16108 5240 Acked Write 9 Done Sent Done false
16109 5241 Acked Write 6 Done Sent Done false
16110 5242 Acked Write 1 Done Sent Done false
16111 5243 Acked Read 2 Done Sent Done false
16112 5244 Acked Write 2 Done Sent Done false
16113 5245 Acked Read 10 Done Sent Done false
16114 5246 Acked Read 3 Done Sent Done false
16115 5247 Acked Read 7 Done Sent Done false
16116 5248 Acked Read 7 Done Sent Done false
16117 5249 Acked Write 6 Done Sent Done false
16118 5250 Acked Write 9 Done Sent Done false
16119 5251 Acked Write 7 Done Sent Done false
16120 5252 Acked Write 3 Done Sent Done false
16121 5253 Acked Read 9 Done Sent Done false
16122 5254 Acked Read 1 Done Sent Done false
16123 5255 Acked Read 10 Done Sent Done false
16124 5256 Acked Write 9 Done Sent Done false
16125 5257 Acked Write 3 Done Sent Done false
16126 5258 Acked Read 7 Done Sent Done false
16127 5259 Acked Write 4 Done Sent Done false
16128 5260 Acked Read 10 Done Sent Done false
16129 5261 Acked Read 9 Done Sent Done false
16130 5262 Acked Flush 0 Done Sent Done false
16131 5263 Acked Read 2 Done Sent Done false
16132 5264 Acked Write 2 Done Sent Done false
16133 5265 Acked Write 7 Done Sent Done false
16134 5266 Acked Flush 0 Done Sent Done false
16135 5267 Acked Read 10 Done Sent Done false
16136 5268 Acked Write 9 Done Sent Done false
16137 5269 Acked Read 7 Done Sent Done false
16138 5270 Acked Write 10 Done Sent Done false
16139 5271 Acked Flush 0 Done Sent Done false
16140 5272 Acked Write 3 Done Sent Done false
16141 5273 Acked Write 2 Done Sent Done false
16142 5274 Acked Read 8 Done Sent Done false
16143 5275 Acked Read 7 Done Sent Done false
16144 5276 Acked Read 9 Done Sent Done false
16145 5277 Acked Read 5 Done Sent Done false
16146 5278 Acked Write 6 Done Sent Done false
16147 5279 Acked Read 7 Done Sent Done false
16148 5280 Acked Flush 0 Done Sent Done false
16149 5281 Acked Write 3 Done Sent Done false
16150 5282 Acked Write 5 Done Sent Done false
16151 5283 Acked Read 1 Done Sent Done false
16152 5284 Acked Read 1 Done Sent Done false
16153 5285 Acked Read 10 Done Sent Done false
16154 5286 Acked Read 4 Done Sent Done false
16155 5287 Acked Read 8 Done Sent Done false
16156 5288 Acked Read 1 Done Sent Done false
16157 5289 Acked Write 2 Done Sent Done false
16158 5290 Acked Read 7 Done Sent Done false
16159 5291 Acked Read 7 Done Sent Done false
16160 5292 Acked Flush 0 Done Sent Done false
16161 5293 Acked Read 1 Done Sent Done false
16162 5294 Acked Read 3 Done Sent Done false
16163 5295 Acked Flush 0 Done Sent Done false
16164 5296 Acked Write 5 Done Sent Done false
16165 5297 Acked Read 7 Done Sent Done false
16166 5298 Acked Write 3 Done Sent Done false
16167 5299 Acked Write 4 Done Sent Done false
16168 5300 Acked Flush 0 Done Sent Done false
16169 5301 Acked Write 3 Done Sent Done false
16170 5302 Acked Read 8 Done Sent Done false
16171 5303 Acked Read 6 Done Sent Done false
16172 5304 Acked Read 6 Done Sent Done false
16173 5305 Acked Write 7 Done Sent Done false
16174 5306 Acked Read 9 Done Sent Done false
16175 5307 Acked Write 6 Done Sent Done false
16176 5308 Acked Read 10 Done Sent Done false
16177 5309 Acked Write 6 Done Sent Done false
16178 5310 Acked Read 3 Done Sent Done false
16179 5311 Acked Write 3 Done Sent Done false
16180 5312 Acked Read 4 Done Sent Done false
16181 5313 Acked Flush 0 Done Sent Done false
16182 5314 Acked Read 5 Done Sent Done false
16183 5315 Acked Read 4 Done Sent Done false
16184 5316 Acked Write 4 Done Sent Done false
16185 5317 Acked Write 6 Done Sent Done false
16186 5318 Acked Read 6 Done Sent Done false
16187 5319 Acked Flush 0 Done Sent Done false
16188 5320 Acked Read 7 Done Sent Done false
16189 5321 Acked Flush 0 Done Sent Done false
16190 5322 Acked Read 8 Done Sent Done false
16191 5323 Acked Read 9 Done Sent Done false
16192 5324 Acked Write 6 Done Sent Done false
16193 5325 Acked Write 7 Done Sent Done false
16194 5326 Acked Read 4 Done Sent Done false
16195 5327 Acked Read 3 Done Sent Done false
16196 5328 Acked Write 4 Done Sent Done false
16197 5329 Acked Write 5 Done Sent Done false
16198 5330 Acked Read 7 Done Sent Done false
16199 5331 Acked Read 10 Done Sent Done false
16200 5332 Acked Read 9 Done Sent Done false
16201 5333 Acked Write 4 Done Sent Done false
16202 5334 Acked Read 2 Done Sent Done false
16203 5335 Acked Write 10 Done Sent Done false
16204 5336 Acked Flush 0 Done Sent Done false
16205 5337 Acked Flush 0 Done Sent Done false
16206 5338 Acked Read 8 Done Sent Done false
16207 5339 Acked Write 6 Done Sent Done false
16208 5340 Acked Read 4 Done Sent Done false
16209 5341 Acked Write 4 Done Sent Done false
16210 5342 Acked Read 3 Done Sent Done false
16211 5343 Acked Read 3 Done Sent Done false
16212 5344 Acked Read 6 Done Sent Done false
16213 5345 Acked Read 1 Done Sent Done false
16214 5346 Acked Flush 0 Done Sent Done false
16215 5347 Acked Read 4 Done Sent Done false
16216 5348 Acked Read 1 Done Sent Done false
16217 5349 Acked Read 4 Done Sent Done false
16218 5350 Acked Flush 0 Done Sent Done false
16219 5351 Acked Flush 0 Done Sent Done false
16220 5352 Acked Read 7 Done Sent Done false
16221 5353 Acked Read 2 Done Sent Done false
16222 5354 Acked Read 5 Done Sent Done false
16223 5355 Acked Read 10 Done Sent Done false
16224 5356 Acked Read 10 Done Sent Done false
16225 5357 Acked Write 9 Done Sent Done false
16226 5358 Acked Read 3 Done Sent Done false
16227 5359 Acked Read 9 Done Sent Done false
16228 5360 Acked Read 8 Done Sent Done false
16229 5361 Acked Read 9 Done Sent Done false
16230 5362 Acked Read 3 Done Sent Done false
16231 5363 Acked Read 4 Done Sent Done false
16232 5364 Acked Read 4 Done Sent Done false
16233 5365 Acked Read 7 Done Sent Done false
16234 5366 Acked Write 9 Done Sent Done false
16235 5367 Acked Write 3 Done Sent Done false
16236 5368 Acked Write 8 Done Sent Done false
16237 5369 Acked Flush 0 Done Sent Done false
16238 5370 Acked Write 4 Done Sent Done false
16239 5371 Acked Read 1 Done Sent Done false
16240 5372 Acked Flush 0 Done Sent Done false
16241 5373 Acked Flush 0 Done Sent Done false
16242 5374 Acked Write 7 Done Sent Done false
16243 5375 Acked Write 1 Done Sent Done false
16244 5376 Acked Write 6 Done Sent Done false
16245 5377 Acked Read 10 Done Sent Done false
16246 5378 Acked Write 6 Done Sent Done false
16247 5379 Acked Flush 0 Done Sent Done false
16248 5380 Acked Write 5 Done Sent Done false
16249 5381 Acked Read 4 Done Sent Done false
16250 5382 Acked Read 4 Done Sent Done false
16251 5383 Acked Write 5 Done Sent Done false
16252 5384 Acked Write 9 Done Sent Done false
16253 5385 Acked Read 9 Done Sent Done false
16254 5386 Acked Write 5 Done Sent Done false
16255 5387 Acked Write 4 Done Sent Done false
16256 5388 Acked Read 6 Done Sent Done false
16257 5389 Acked Flush 0 Done Sent Done false
16258 5390 Acked Read 5 Done Sent Done false
16259 5391 Acked Write 7 Done Sent Done false
16260 5392 Acked Read 8 Done Sent Done false
16261 5393 Acked Write 9 Done Sent Done false
16262 5394 Acked Write 9 Done Sent Done false
16263 5395 Acked Flush 0 Done Sent Done false
16264 5396 Acked Read 3 Done Sent Done false
16265 5397 Acked Read 4 Done Sent Done false
16266 5398 Acked Read 3 Done Sent Done false
16267 5399 Acked Write 6 Done Sent Done false
16268 5400 Acked Read 1 Done Sent Done false
16269 5401 Acked Flush 0 Done Sent Done false
16270 5402 Acked Write 3 Done Sent Done false
16271 5403 Acked Write 4 Done Sent Done false
16272 5404 Acked Write 2 Done Sent Done false
16273 5405 Acked Write 10 Done Sent Done false
16274 5406 Acked Flush 0 Done Sent Done false
16275 5407 Acked Flush 0 Done Sent Done false
16276 5408 Acked Read 5 Done Sent Done false
16277 5409 Acked Write 9 Done Sent Done false
16278 5410 Acked Read 6 Done Sent Done false
16279 5411 Acked Write 7 Done Sent Done false
16280 5412 Acked Write 1 Done Sent Done false
16281 5413 Acked Write 6 Done Sent Done false
16282 5414 Acked Read 5 Done Sent Done false
16283 5415 Acked Read 10 Done Sent Done false
16284 5416 Acked Flush 0 Done Sent Done false
16285 5417 Acked Write 8 Done Sent Done false
16286 5418 Acked Write 7 Done Sent Done false
16287 5419 Acked Read 1 Done Sent Done false
16288 5420 Acked Read 10 Done Sent Done false
16289 5421 Acked Write 7 Done Sent Done false
16290 5422 Acked Read 10 Done Sent Done false
16291 5423 Acked Read 10 Done Sent Done false
16292 5424 Acked Write 1 Done Sent Done false
16293 5425 Acked Write 4 Done Sent Done false
16294 5426 Acked Read 7 Done Sent Done false
16295 5427 Acked Read 8 Done Sent Done false
16296 5428 Acked Read 8 Done Sent Done false
16297 5429 Acked Read 10 Done Sent Done false
16298 5430 Acked Write 4 Done Sent Done false
16299 5431 Acked Write 6 Done Sent Done false
16300 5432 Acked Read 2 Done Sent Done false
16301 5433 Acked Read 10 Done Sent Done false
16302 5434 Acked Read 1 Done Sent Done false
16303 5435 Acked Read 6 Done Sent Done false
16304 5436 Acked Write 3 Done Sent Done false
16305 5437 Acked Read 6 Done Sent Done false
16306 5438 Acked Read 4 Done Sent Done false
16307 5439 Acked Write 2 Done Sent Done false
16308 5440 Acked Read 6 Done Sent Done false
16309 5441 Acked Read 4 Done Sent Done false
16310 5442 Acked Flush 0 Done Sent Done false
16311 5443 Acked Write 2 Done Sent Done false
16312 5444 Acked Write 10 Done Sent Done false
16313 5445 Acked Flush 0 Done Sent Done false
16314 5446 Acked Read 7 Done Sent Done false
16315 5447 Acked Read 4 Done Sent Done false
16316 5448 Acked Read 8 Done Sent Done false
16317 5449 Acked Write 7 Done Sent Done false
16318 5450 Acked Flush 0 Done Sent Done false
16319 5451 Acked Write 2 Done Sent Done false
16320 5452 Acked Write 4 Done Sent Done false
16321 5453 Acked Write 8 Done Sent Done false
16322 5454 Acked Write 9 Done Sent Done false
16323 5455 Acked Write 10 Done Sent Done false
16324 5456 Acked Flush 0 Done Sent Done false
16325 5457 Acked Read 8 Done Sent Done false
16326 5458 Acked Write 5 Done Sent Done false
16327 5459 Acked Write 3 Done Sent Done false
16328 5460 Acked Read 3 Done Sent Done false
16329 5461 Acked Read 1 Done Sent Done false
16330 5462 Acked Read 8 Done Sent Done false
16331 5463 Acked Read 8 Done Sent Done false
16332 5464 Acked Write 8 Done Sent Done false
16333 5465 Acked Read 2 Done Sent Done false
16334 5466 Acked Write 5 Done Sent Done false
16335 5467 Acked Flush 0 Done Sent Done false
16336 5468 Acked Read 1 Done Sent Done false
16337 5469 Acked Write 3 Done Sent Done false
16338 5470 Acked Read 4 Done Sent Done false
16339 5471 Acked Write 9 Done Sent Done false
16340 5472 Acked Read 2 Done Sent Done false
16341 5473 Acked Write 3 Done Sent Done false
16342 5474 Acked Read 4 Done Sent Done false
16343 5475 Acked Read 9 Done Sent Done false
16344 5476 Acked Read 9 Done Sent Done false
16345 5477 Acked Write 1 Done Sent Done false
16346 5478 Acked Read 8 Done Sent Done false
16347 5479 Acked Read 9 Done Sent Done false
16348 5480 Acked Read 1 Done Sent Done false
16349 5481 Acked Write 5 Done Sent Done false
16350 5482 Acked Read 9 Done Sent Done false
16351 5483 Acked Read 6 Done Sent Done false
16352 5484 Acked Read 2 Done Sent Done false
16353 5485 Acked Write 10 Done Sent Done false
16354 5486 Acked Write 4 Done Sent Done false
16355 5487 Acked Read 10 Done Sent Done false
16356 5488 Acked Flush 0 Done Sent Done false
16357 5489 Acked Write 10 Done Sent Done false
16358 5490 Acked Flush 0 Done Sent Done false
16359 5491 Acked Write 9 Done Sent Done false
16360 5492 Acked Read 2 Done Sent Done false
16361 5493 Acked Read 10 Done Sent Done false
16362 5494 Acked Read 4 Done Sent Done false
16363 5495 Acked Read 7 Done Sent Done false
16364 5496 Acked Read 4 Done Sent Done false
16365 5497 Acked Write 6 Done Sent Done false
16366 5498 Acked Write 2 Done Sent Done false
16367 5499 Acked Write 2 Done Sent Done false
16368 5500 Acked Write 4 Done Sent Done false
16369 5501 Acked Write 9 Done Sent Done false
16370 5502 Acked Write 9 Done Sent Done false
16371 5503 Acked Read 7 Done Sent Done false
16372 5504 Acked Write 2 Done Sent Done false
16373 5505 Acked Write 8 Done Sent Done false
16374 5506 Acked Write 8 Done Sent Done false
16375 5507 Acked Flush 0 Done Sent Done false
16376 5508 Acked Write 1 Done Sent Done false
16377 5509 Acked Read 2 Done Sent Done false
16378 5510 Acked Read 4 Done Sent Done false
16379 5511 Acked Read 8 Done Sent Done false
16380 5512 Acked Read 6 Done Sent Done false
16381 5513 Acked Write 10 Done Sent Done false
16382 5514 Acked Read 3 Done Sent Done false
16383 5515 Acked Write 2 Done Sent Done false
16384 5516 Acked Read 2 Done Sent Done false
16385 5517 Acked Write 8 Done Sent Done false
16386 5518 Acked Read 1 Done Sent Done false
16387 5519 Acked Write 1 Done Sent Done false
16388 5520 Acked Read 5 Done Sent Done false
16389 5521 Acked Write 10 Done Sent Done false
16390 5522 Acked Write 10 Done Sent Done false
16391 5523 Acked Read 4 Done Sent Done false
16392 5524 Acked Read 5 Done Sent Done false
16393 5525 Acked Read 6 Done Sent Done false
16394 5526 Acked Write 7 Done Sent Done false
16395 STATES DS:0 DS:1 DS:2 TOTAL
16396 Sent 0 301 0 301
16397 Done 301 0 301 602
16398 Skipped 0 0 0 0
16399 Error 0 0 0 0
16400 Last Flush: 5507 5225 5507
16401 Downstairs last five completed: 5225 5224 5223 5222 5221
16402 Upstairs last five completed: 5526 5525 5524 5523 5522
16403 CLIENT: Up:0 ds:301 act:2
16404 [8820] got start action
16405 [8820] I am going to restart
16406 Starting downstairs at port 8820
16407 Make output file at "/tmp/dsc/downstairs-8820.txt"
16408 [8820][1] reports Starting
16409 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1219)
16410 [8820][1] reports Running
16411 ----------------------------------------------------------------
16412 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16413 States: Active Offline Active
16414 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16415 5226 Acked Write 8 Done Sent Done false
16416 5227 Acked Read 7 Done Sent Done false
16417 5228 Acked Write 10 Done Sent Done false
16418 5229 Acked Write 4 Done Sent Done false
16419 5230 Acked Read 9 Done Sent Done false
16420 5231 Acked Read 3 Done Sent Done false
16421 5232 Acked Write 9 Done Sent Done false
16422 5233 Acked Read 1 Done Sent Done false
16423 5234 Acked Flush 0 Done Sent Done false
16424 5235 Acked Write 10 Done Sent Done false
16425 5236 Acked Write 9 Done Sent Done false
16426 5237 Acked Read 9 Done Sent Done false
16427 5238 Acked Read 5 Done Sent Done false
16428 5239 Acked Write 8 Done Sent Done false
16429 5240 Acked Write 9 Done Sent Done false
16430 5241 Acked Write 6 Done Sent Done false
16431 5242 Acked Write 1 Done Sent Done false
16432 5243 Acked Read 2 Done Sent Done false
16433 5244 Acked Write 2 Done Sent Done false
16434 5245 Acked Read 10 Done Sent Done false
16435 5246 Acked Read 3 Done Sent Done false
16436 5247 Acked Read 7 Done Sent Done false
16437 5248 Acked Read 7 Done Sent Done false
16438 5249 Acked Write 6 Done Sent Done false
16439 5250 Acked Write 9 Done Sent Done false
16440 5251 Acked Write 7 Done Sent Done false
16441 5252 Acked Write 3 Done Sent Done false
16442 5253 Acked Read 9 Done Sent Done false
16443 5254 Acked Read 1 Done Sent Done false
16444 5255 Acked Read 10 Done Sent Done false
16445 5256 Acked Write 9 Done Sent Done false
16446 5257 Acked Write 3 Done Sent Done false
16447 5258 Acked Read 7 Done Sent Done false
16448 5259 Acked Write 4 Done Sent Done false
16449 5260 Acked Read 10 Done Sent Done false
16450 5261 Acked Read 9 Done Sent Done false
16451 5262 Acked Flush 0 Done Sent Done false
16452 5263 Acked Read 2 Done Sent Done false
16453 5264 Acked Write 2 Done Sent Done false
16454 5265 Acked Write 7 Done Sent Done false
16455 5266 Acked Flush 0 Done Sent Done false
16456 5267 Acked Read 10 Done Sent Done false
16457 5268 Acked Write 9 Done Sent Done false
16458 5269 Acked Read 7 Done Sent Done false
16459 5270 Acked Write 10 Done Sent Done false
16460 5271 Acked Flush 0 Done Sent Done false
16461 5272 Acked Write 3 Done Sent Done false
16462 5273 Acked Write 2 Done Sent Done false
16463 5274 Acked Read 8 Done Sent Done false
16464 5275 Acked Read 7 Done Sent Done false
16465 5276 Acked Read 9 Done Sent Done false
16466 5277 Acked Read 5 Done Sent Done false
16467 5278 Acked Write 6 Done Sent Done false
16468 5279 Acked Read 7 Done Sent Done false
16469 5280 Acked Flush 0 Done Sent Done false
16470 5281 Acked Write 3 Done Sent Done false
16471 5282 Acked Write 5 Done Sent Done false
16472 5283 Acked Read 1 Done Sent Done false
16473 5284 Acked Read 1 Done Sent Done false
16474 5285 Acked Read 10 Done Sent Done false
16475 5286 Acked Read 4 Done Sent Done false
16476 5287 Acked Read 8 Done Sent Done false
16477 5288 Acked Read 1 Done Sent Done false
16478 5289 Acked Write 2 Done Sent Done false
16479 5290 Acked Read 7 Done Sent Done false
16480 5291 Acked Read 7 Done Sent Done false
16481 5292 Acked Flush 0 Done Sent Done false
16482 5293 Acked Read 1 Done Sent Done false
16483 5294 Acked Read 3 Done Sent Done false
16484 5295 Acked Flush 0 Done Sent Done false
16485 5296 Acked Write 5 Done Sent Done false
16486 5297 Acked Read 7 Done Sent Done false
16487 5298 Acked Write 3 Done Sent Done false
16488 5299 Acked Write 4 Done Sent Done false
16489 5300 Acked Flush 0 Done Sent Done false
16490 5301 Acked Write 3 Done Sent Done false
16491 5302 Acked Read 8 Done Sent Done false
16492 5303 Acked Read 6 Done Sent Done false
16493 5304 Acked Read 6 Done Sent Done false
16494 5305 Acked Write 7 Done Sent Done false
16495 5306 Acked Read 9 Done Sent Done false
16496 5307 Acked Write 6 Done Sent Done false
16497 5308 Acked Read 10 Done Sent Done false
16498 5309 Acked Write 6 Done Sent Done false
16499 5310 Acked Read 3 Done Sent Done false
16500 5311 Acked Write 3 Done Sent Done false
16501 5312 Acked Read 4 Done Sent Done false
16502 5313 Acked Flush 0 Done Sent Done false
16503 5314 Acked Read 5 Done Sent Done false
16504 5315 Acked Read 4 Done Sent Done false
16505 5316 Acked Write 4 Done Sent Done false
16506 5317 Acked Write 6 Done Sent Done false
16507 5318 Acked Read 6 Done Sent Done false
16508 5319 Acked Flush 0 Done Sent Done false
16509 5320 Acked Read 7 Done Sent Done false
16510 5321 Acked Flush 0 Done Sent Done false
16511 5322 Acked Read 8 Done Sent Done false
16512 5323 Acked Read 9 Done Sent Done false
16513 5324 Acked Write 6 Done Sent Done false
16514 5325 Acked Write 7 Done Sent Done false
16515 5326 Acked Read 4 Done Sent Done false
16516 5327 Acked Read 3 Done Sent Done false
16517 5328 Acked Write 4 Done Sent Done false
16518 5329 Acked Write 5 Done Sent Done false
16519 5330 Acked Read 7 Done Sent Done false
16520 5331 Acked Read 10 Done Sent Done false
16521 5332 Acked Read 9 Done Sent Done false
16522 5333 Acked Write 4 Done Sent Done false
16523 5334 Acked Read 2 Done Sent Done false
16524 5335 Acked Write 10 Done Sent Done false
16525 5336 Acked Flush 0 Done Sent Done false
16526 5337 Acked Flush 0 Done Sent Done false
16527 5338 Acked Read 8 Done Sent Done false
16528 5339 Acked Write 6 Done Sent Done false
16529 5340 Acked Read 4 Done Sent Done false
16530 5341 Acked Write 4 Done Sent Done false
16531 5342 Acked Read 3 Done Sent Done false
16532 5343 Acked Read 3 Done Sent Done false
16533 5344 Acked Read 6 Done Sent Done false
16534 5345 Acked Read 1 Done Sent Done false
16535 5346 Acked Flush 0 Done Sent Done false
16536 5347 Acked Read 4 Done Sent Done false
16537 5348 Acked Read 1 Done Sent Done false
16538 5349 Acked Read 4 Done Sent Done false
16539 5350 Acked Flush 0 Done Sent Done false
16540 5351 Acked Flush 0 Done Sent Done false
16541 5352 Acked Read 7 Done Sent Done false
16542 5353 Acked Read 2 Done Sent Done false
16543 5354 Acked Read 5 Done Sent Done false
16544 5355 Acked Read 10 Done Sent Done false
16545 5356 Acked Read 10 Done Sent Done false
16546 5357 Acked Write 9 Done Sent Done false
16547 5358 Acked Read 3 Done Sent Done false
16548 5359 Acked Read 9 Done Sent Done false
16549 5360 Acked Read 8 Done Sent Done false
16550 5361 Acked Read 9 Done Sent Done false
16551 5362 Acked Read 3 Done Sent Done false
16552 5363 Acked Read 4 Done Sent Done false
16553 5364 Acked Read 4 Done Sent Done false
16554 5365 Acked Read 7 Done Sent Done false
16555 5366 Acked Write 9 Done Sent Done false
16556 5367 Acked Write 3 Done Sent Done false
16557 5368 Acked Write 8 Done Sent Done false
16558 5369 Acked Flush 0 Done Sent Done false
16559 5370 Acked Write 4 Done Sent Done false
16560 5371 Acked Read 1 Done Sent Done false
16561 5372 Acked Flush 0 Done Sent Done false
16562 5373 Acked Flush 0 Done Sent Done false
16563 5374 Acked Write 7 Done Sent Done false
16564 5375 Acked Write 1 Done Sent Done false
16565 5376 Acked Write 6 Done Sent Done false
16566 5377 Acked Read 10 Done Sent Done false
16567 5378 Acked Write 6 Done Sent Done false
16568 5379 Acked Flush 0 Done Sent Done false
16569 5380 Acked Write 5 Done Sent Done false
16570 5381 Acked Read 4 Done Sent Done false
16571 5382 Acked Read 4 Done Sent Done false
16572 5383 Acked Write 5 Done Sent Done false
16573 5384 Acked Write 9 Done Sent Done false
16574 5385 Acked Read 9 Done Sent Done false
16575 5386 Acked Write 5 Done Sent Done false
16576 5387 Acked Write 4 Done Sent Done false
16577 5388 Acked Read 6 Done Sent Done false
16578 5389 Acked Flush 0 Done Sent Done false
16579 5390 Acked Read 5 Done Sent Done false
16580 5391 Acked Write 7 Done Sent Done false
16581 5392 Acked Read 8 Done Sent Done false
16582 5393 Acked Write 9 Done Sent Done false
16583 5394 Acked Write 9 Done Sent Done false
16584 5395 Acked Flush 0 Done Sent Done false
16585 5396 Acked Read 3 Done Sent Done false
16586 5397 Acked Read 4 Done Sent Done false
16587 5398 Acked Read 3 Done Sent Done false
16588 5399 Acked Write 6 Done Sent Done false
16589 5400 Acked Read 1 Done Sent Done false
16590 5401 Acked Flush 0 Done Sent Done false
16591 5402 Acked Write 3 Done Sent Done false
16592 5403 Acked Write 4 Done Sent Done false
16593 5404 Acked Write 2 Done Sent Done false
16594 5405 Acked Write 10 Done Sent Done false
16595 5406 Acked Flush 0 Done Sent Done false
16596 5407 Acked Flush 0 Done Sent Done false
16597 5408 Acked Read 5 Done Sent Done false
16598 5409 Acked Write 9 Done Sent Done false
16599 5410 Acked Read 6 Done Sent Done false
16600 5411 Acked Write 7 Done Sent Done false
16601 5412 Acked Write 1 Done Sent Done false
16602 5413 Acked Write 6 Done Sent Done false
16603 5414 Acked Read 5 Done Sent Done false
16604 5415 Acked Read 10 Done Sent Done false
16605 5416 Acked Flush 0 Done Sent Done false
16606 5417 Acked Write 8 Done Sent Done false
16607 5418 Acked Write 7 Done Sent Done false
16608 5419 Acked Read 1 Done Sent Done false
16609 5420 Acked Read 10 Done Sent Done false
16610 5421 Acked Write 7 Done Sent Done false
16611 5422 Acked Read 10 Done Sent Done false
16612 5423 Acked Read 10 Done Sent Done false
16613 5424 Acked Write 1 Done Sent Done false
16614 5425 Acked Write 4 Done Sent Done false
16615 5426 Acked Read 7 Done Sent Done false
16616 5427 Acked Read 8 Done Sent Done false
16617 5428 Acked Read 8 Done Sent Done false
16618 5429 Acked Read 10 Done Sent Done false
16619 5430 Acked Write 4 Done Sent Done false
16620 5431 Acked Write 6 Done Sent Done false
16621 5432 Acked Read 2 Done Sent Done false
16622 5433 Acked Read 10 Done Sent Done false
16623 5434 Acked Read 1 Done Sent Done false
16624 5435 Acked Read 6 Done Sent Done false
16625 5436 Acked Write 3 Done Sent Done false
16626 5437 Acked Read 6 Done Sent Done false
16627 5438 Acked Read 4 Done Sent Done false
16628 5439 Acked Write 2 Done Sent Done false
16629 5440 Acked Read 6 Done Sent Done false
16630 5441 Acked Read 4 Done Sent Done false
16631 5442 Acked Flush 0 Done Sent Done false
16632 5443 Acked Write 2 Done Sent Done false
16633 5444 Acked Write 10 Done Sent Done false
16634 5445 Acked Flush 0 Done Sent Done false
16635 5446 Acked Read 7 Done Sent Done false
16636 5447 Acked Read 4 Done Sent Done false
16637 5448 Acked Read 8 Done Sent Done false
16638 5449 Acked Write 7 Done Sent Done false
16639 5450 Acked Flush 0 Done Sent Done false
16640 5451 Acked Write 2 Done Sent Done false
16641 5452 Acked Write 4 Done Sent Done false
16642 5453 Acked Write 8 Done Sent Done false
16643 5454 Acked Write 9 Done Sent Done false
16644 5455 Acked Write 10 Done Sent Done false
16645 5456 Acked Flush 0 Done Sent Done false
16646 5457 Acked Read 8 Done Sent Done false
16647 5458 Acked Write 5 Done Sent Done false
16648 5459 Acked Write 3 Done Sent Done false
16649 5460 Acked Read 3 Done Sent Done false
16650 5461 Acked Read 1 Done Sent Done false
16651 5462 Acked Read 8 Done Sent Done false
16652 5463 Acked Read 8 Done Sent Done false
16653 5464 Acked Write 8 Done Sent Done false
16654 5465 Acked Read 2 Done Sent Done false
16655 5466 Acked Write 5 Done Sent Done false
16656 5467 Acked Flush 0 Done Sent Done false
16657 5468 Acked Read 1 Done Sent Done false
16658 5469 Acked Write 3 Done Sent Done false
16659 5470 Acked Read 4 Done Sent Done false
16660 5471 Acked Write 9 Done Sent Done false
16661 5472 Acked Read 2 Done Sent Done false
16662 5473 Acked Write 3 Done Sent Done false
16663 5474 Acked Read 4 Done Sent Done false
16664 5475 Acked Read 9 Done Sent Done false
16665 5476 Acked Read 9 Done Sent Done false
16666 5477 Acked Write 1 Done Sent Done false
16667 5478 Acked Read 8 Done Sent Done false
16668 5479 Acked Read 9 Done Sent Done false
16669 5480 Acked Read 1 Done Sent Done false
16670 5481 Acked Write 5 Done Sent Done false
16671 5482 Acked Read 9 Done Sent Done false
16672 5483 Acked Read 6 Done Sent Done false
16673 5484 Acked Read 2 Done Sent Done false
16674 5485 Acked Write 10 Done Sent Done false
16675 5486 Acked Write 4 Done Sent Done false
16676 5487 Acked Read 10 Done Sent Done false
16677 5488 Acked Flush 0 Done Sent Done false
16678 5489 Acked Write 10 Done Sent Done false
16679 5490 Acked Flush 0 Done Sent Done false
16680 5491 Acked Write 9 Done Sent Done false
16681 5492 Acked Read 2 Done Sent Done false
16682 5493 Acked Read 10 Done Sent Done false
16683 5494 Acked Read 4 Done Sent Done false
16684 5495 Acked Read 7 Done Sent Done false
16685 5496 Acked Read 4 Done Sent Done false
16686 5497 Acked Write 6 Done Sent Done false
16687 5498 Acked Write 2 Done Sent Done false
16688 5499 Acked Write 2 Done Sent Done false
16689 5500 Acked Write 4 Done Sent Done false
16690 5501 Acked Write 9 Done Sent Done false
16691 5502 Acked Write 9 Done Sent Done false
16692 5503 Acked Read 7 Done Sent Done false
16693 5504 Acked Write 2 Done Sent Done false
16694 5505 Acked Write 8 Done Sent Done false
16695 5506 Acked Write 8 Done Sent Done false
16696 5507 Acked Flush 0 Done Sent Done false
16697 5508 Acked Write 1 Done Sent Done false
16698 5509 Acked Read 2 Done Sent Done false
16699 5510 Acked Read 4 Done Sent Done false
16700 5511 Acked Read 8 Done Sent Done false
16701 5512 Acked Read 6 Done Sent Done false
16702 5513 Acked Write 10 Done Sent Done false
16703 5514 Acked Read 3 Done Sent Done false
16704 5515 Acked Write 2 Done Sent Done false
16705 5516 Acked Read 2 Done Sent Done false
16706 5517 Acked Write 8 Done Sent Done false
16707 5518 Acked Read 1 Done Sent Done false
16708 5519 Acked Write 1 Done Sent Done false
16709 5520 Acked Read 5 Done Sent Done false
16710 5521 Acked Write 10 Done Sent Done false
16711 5522 Acked Write 10 Done Sent Done false
16712 5523 Acked Read 4 Done Sent Done false
16713 5524 Acked Read 5 Done Sent Done false
16714 5525 Acked Read 6 Done Sent Done false
16715 5526 Acked Write 7 Done Sent Done false
16716 5527 Acked Flush 0 Done Sent Done false
16717 STATES DS:0 DS:1 DS:2 TOTAL
16718 Sent 0 302 0 302
16719 Done 302 0 302 604
16720 Skipped 0 0 0 0
16721 Error 0 0 0 0
16722 Last Flush: 5527 5225 5527
16723 Downstairs last five completed: 5225 5224 5223 5222 5221
16724 Upstairs last five completed: 5527 5526 5525 5524 5523
16725 CLIENT: Up:0 ds:302 act:2
167262025-01-11T06:08:55.988ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167272025-01-11T06:08:55.988ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61407) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167282025-01-11T06:08:55.989ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167292025-01-11T06:08:55.990ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167302025-01-11T06:08:55.990ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167312025-01-11T06:08:55.990ZINFOcrucible: send last flush ID to this DS: 5225 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167322025-01-11T06:08:55.990ZINFOcrucible: Replied this last flush ID: 5225 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167332025-01-11T06:08:55.990ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167342025-01-11T06:08:55.990ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167352025-01-11T06:08:55.990ZINFOcrucible: [1] client re-new 302 jobs since flush 5225 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167362025-01-11T06:08:55.990ZINFOcrucible: [1] Replayed 302 jobs since flush: 5225 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
16737 ----------------------------------------------------------------
16738 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16739 Downstairs last five completed: 5527 5526 5525 5524 5523
16740 Upstairs last five completed: 5527 5526 5525 5524 5523
16741 CLIENT: Up:0 ds:0 act:3
16742 Replay: All jobs finished
16743 Main task has work to do, go find it
16744 got dsc Stop(2)
16745 stop 2
16746 [8830] Got stop action so:false kr:false
16747 Jan 11 06:08:58.246 INFO request completed, latency_us: 351, response_code: 204, uri: /stop/cid/2, method: POST, req_id: d2285687-b212-4252-82f6-2bfb2b013366, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
167482025-01-11T06:08:58.246ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
16749 Jan 11 06:08:58.246 INFO request completed, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:08:58.24690308Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
16750 latency_us: 123, response_code: 200, uri: /state/cid/2, method: GET, req_id: 07a9d808-12de-41dd-9606-57a6ed3d43fb, remote_addr: 127.0.0.1:59925{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:08:58.246961872Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
16751 , local_addr: 127.0.0.1:9998
167522025-01-11T06:08:58.246ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167532025-01-11T06:08:58.247ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
167542025-01-11T06:08:58.247ZINFOcrucible: client task is exiting = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
16755 [8830] Exited with: ExitStatus(unix_wait_status(9))
16756 [8830][2] reports Exit
16757 Jan 11 06:09:02.249 INFO request completed, latency_us: 219, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9f288c07-d038-4727-b43b-083f4b4dec21, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
16758 001/300 Write block 2525 len 32768 data: 7 5 6 6 4 3 3 4
16759 002/300 Read block 2196 len 40960
16760 003/300 Flush
16761 004/300 Write block 1243 len 12288 data: 4 3 3
16762 005/300 Write block 4569 len 32768 data: 5 3 3 3 4 4 3 4
16763 006/300 Write block 1298 len 24576 data: 5 5 4 3 4 4
16764 007/300 Read block 3918 len 12288
16765 008/300 Write block 3812 len 12288 data: 3 3 4
16766 009/300 Write block 2512 len 36864 data: 5 5 3 3 4 3 4 4 5
16767 010/300 Read block 335 len 28672
16768 011/300 Flush
16769 012/300 Read block 1324 len 28672
16770 013/300 Write block 1244 len 24576 data: 4 4 4 3 3 3
16771 014/300 Write block 2002 len 4096 data: 3
16772 015/300 Read block 3751 len 40960
16773 016/300 Write block 2691 len 12288 data: 4 4 4
16774 017/300 Write block 1481 len 24576 data: 5 5 4 3 3 4
16775 018/300 Read block 1799 len 32768
16776 019/300 Read block 4675 len 24576
16777 020/300 Flush
16778 021/300 Write block 1278 len 20480 data: 4 4 4 4 3
16779 022/300 Write block 3282 len 32768 data: 4 5 4 4 2 2 2 2
16780 023/300 Read block 2523 len 12288
16781 024/300 Read block 2785 len 24576
16782 025/300 Flush
16783 026/300 Read block 1632 len 20480
16784 027/300 Write block 22 len 8192 data: 3 5
16785 028/300 Read block 2664 len 20480
16786 029/300 Flush
16787 030/300 Read block 2800 len 12288
16788 031/300 Read block 1724 len 12288
16789 032/300 Read block 2582 len 36864
16790 033/300 Read block 4086 len 20480
16791 034/300 Write block 4193 len 40960 data: 2 2 2 2 3 3 3 4 4 3
16792 035/300 Read block 1452 len 32768
16793 036/300 Write block 3606 len 28672 data: 4 3 3 3 3 3 4
16794 037/300 Write block 3902 len 24576 data: 4 5 5 4 5 4
16795 038/300 Flush
16796 039/300 Read block 572 len 8192
16797 040/300 Write block 3014 len 36864 data: 4 3 3 3 3 3 4 3 3
16798 041/300 Write block 3704 len 28672 data: 5 5 6 6 5 4 4
16799 042/300 Read block 525 len 28672
16800 043/300 Write block 3322 len 40960 data: 4 4 4 4 3 3 3 3 4 5
16801 044/300 Flush
16802 045/300 Read block 1793 len 4096
16803 046/300 Write block 1079 len 8192 data: 3 2
16804 047/300 Write block 2642 len 32768 data: 6 5 5 5 4 4 7 5
16805 048/300 Read block 515 len 20480
16806 049/300 Write block 4629 len 32768 data: 3 3 5 5 6 6 7 6
16807 050/300 Read block 984 len 4096
16808 051/300 Flush
16809 052/300 Flush
16810 053/300 Read block 188 len 24576
16811 054/300 Write block 1766 len 8192 data: 2 3
16812 055/300 Flush
16813 056/300 Write block 1885 len 20480 data: 5 3 3 3 3
16814 057/300 Read block 1106 len 24576
16815 058/300 Write block 692 len 32768 data: 6 7 5 4 4 4 3 2
16816 059/300 Write block 704 len 40960 data: 3 3 3 4 6 7 6 5 4 4
16817 060/300 Write block 4033 len 8192 data: 5 4
16818 061/300 Read block 3348 len 24576
16819 062/300 Flush
16820 063/300 Read block 2491 len 8192
16821 064/300 Write block 1916 len 16384 data: 2 2 2 2
16822 065/300 Read block 4667 len 32768
16823 066/300 Read block 247 len 12288
16824 067/300 Read block 3165 len 16384
16825 068/300 Read block 2513 len 28672
16826 069/300 Read block 1525 len 16384
16827 070/300 Read block 287 len 8192
16828 071/300 Read block 4178 len 24576
16829 072/300 Write block 4310 len 40960 data: 3 4 5 4 3 3 4 4 5 6
16830 073/300 Write block 4747 len 24576 data: 5 4 4 5 4 4
16831 074/300 Write block 4285 len 40960 data: 3 3 3 4 4 4 3 3 3 3
16832 075/300 Write block 3261 len 24576 data: 4 4 3 2 2 2
16833 076/300 Read block 1920 len 36864
16834 077/300 Read block 3070 len 16384
16835 078/300 Read block 4259 len 16384
16836 079/300 Write block 468 len 36864 data: 6 6 5 4 4 4 4 3 3
16837 080/300 Write block 4890 len 32768 data: 3 3 5 6 4 4 3 2
16838 081/300 Read block 2954 len 32768
16839 082/300 Read block 1926 len 36864
16840 083/300 Write block 2947 len 36864 data: 8 6 8 8 6 5 5 4 4
16841 084/300 Write block 1679 len 36864 data: 3 4 4 5 5 5 6 5 5
16842 085/300 Read block 3728 len 40960
16843 086/300 Read block 4968 len 12288
16844 087/300 Read block 1043 len 28672
16845 088/300 Read block 4796 len 12288
16846 089/300 Read block 2453 len 8192
16847 090/300 Flush
16848 091/300 Read block 3172 len 12288
16849 092/300 Read block 4123 len 8192
16850 093/300 Read block 246 len 36864
16851 094/300 Write block 804 len 28672 data: 4 3 3 3 3 3 3
16852 095/300 Read block 3998 len 40960
16853 096/300 Read block 3146 len 28672
16854 097/300 Write block 1205 len 4096 data: 6
16855 098/300 Write block 1655 len 12288 data: 3 3 3
16856 099/300 Read block 605 len 16384
16857 100/300 Read block 2024 len 32768
16858 101/300 Read block 2845 len 4096
16859 102/300 Read block 1612 len 8192
16860 103/300 Write block 4031 len 36864 data: 5 4 6 5 4 3 4 4 3
16861 104/300 Write block 1362 len 28672 data: 5 6 5 6 6 6 6
16862 105/300 Read block 4040 len 36864
16863 106/300 Write block 2589 len 16384 data: 3 3 4 4
16864 107/300 Read block 4743 len 28672
16865 108/300 Write block 961 len 20480 data: 3 2 2 2 3
16866 109/300 Read block 1119 len 32768
16867 110/300 Read block 705 len 20480
16868 111/300 Read block 4880 len 36864
16869 112/300 Write block 1685 len 4096 data: 7
16870 113/300 Write block 4126 len 28672 data: 4 6 7 6 7 6 6
16871 114/300 Write block 2862 len 28672 data: 8 8 8 7 7 6 6
16872 115/300 Write block 3072 len 40960 data: 3 3 3 2 3 3 2 2 2 2
16873 116/300 Write block 4208 len 28672 data: 5 4 5 6 5 4 3
16874 117/300 Read block 2662 len 28672
16875 118/300 Read block 930 len 4096
16876 119/300 Read block 4683 len 36864
16877 120/300 Read block 4653 len 32768
16878 121/300 Read block 1324 len 4096
16879 122/300 Read block 1021 len 4096
16880 123/300 Read block 283 len 40960
16881 124/300 Read block 2095 len 12288
16882 125/300 Read block 3073 len 12288
16883 126/300 Write block 4529 len 20480 data: 5 3 4 4 3
16884 127/300 Read block 1643 len 28672
16885 128/300 Read block 1843 len 16384
16886 129/300 Write block 483 len 8192 data: 5 5
16887 130/300 Write block 4110 len 40960 data: 2 2 2 2 2 2 2 2 2 2
16888 131/300 Flush
16889 132/300 Read block 2714 len 12288
16890 133/300 Write block 2182 len 4096 data: 4
16891 134/300 Read block 3000 len 4096
16892 135/300 Write block 1690 len 36864 data: 4 5 5 4 3 2 3 3 4
16893 136/300 Read block 2393 len 24576
16894 137/300 Write block 1766 len 4096 data: 3
16895 138/300 Flush
16896 139/300 Flush
16897 140/300 Read block 3742 len 16384
16898 141/300 Read block 3132 len 4096
16899 142/300 Read block 4078 len 24576
16900 143/300 Read block 2345 len 28672
16901 144/300 Read block 95 len 36864
16902 145/300 Read block 1753 len 36864
16903 146/300 Read block 1224 len 28672
16904 147/300 Read block 2394 len 28672
16905 148/300 Read block 4806 len 4096
16906 149/300 Read block 80 len 4096
16907 150/300 Write block 3030 len 24576 data: 6 5 4 5 4 4
16908 151/300 Write block 1355 len 12288 data: 4 5 5
16909 152/300 Read block 2130 len 40960
16910 153/300 Read block 3213 len 8192
16911 154/300 Read block 1778 len 28672
16912 155/300 Write block 1055 len 36864 data: 2 3 2 3 4 4 5 6 6
16913 156/300 Read block 0 len 20480
16914 157/300 Flush
16915 158/300 Write block 2237 len 32768 data: 4 4 3 2 3 3 3 3
16916 159/300 Write block 4107 len 28672 data: 4 2 2 3 3 3 3
16917 160/300 Write block 279 len 36864 data: 9 8 9 10 8 7 8 7 5
16918 161/300 Read block 3004 len 28672
16919 162/300 Read block 2857 len 16384
16920 163/300 Read block 246 len 12288
16921 164/300 Write block 4824 len 20480 data: 5 6 6 6 6
16922 165/300 Write block 825 len 12288 data: 4 4 4
16923 166/300 Read block 38 len 16384
16924 167/300 Write block 645 len 40960 data: 3 2 3 5 5 6 5 5 5 4
16925 168/300 Read block 2615 len 16384
16926 169/300 Read block 4861 len 36864
16927 170/300 Read block 3882 len 28672
16928 171/300 Read block 803 len 32768
16929 172/300 Write block 1693 len 16384 data: 5 4 3 4
16930 173/300 Read block 2749 len 20480
16931 174/300 Write block 3575 len 12288 data: 3 5 4
16932 175/300 Read block 953 len 36864
16933 176/300 Read block 473 len 24576
16934 177/300 Read block 3482 len 16384
16935 178/300 Read block 3375 len 4096
16936 179/300 Write block 4258 len 32768 data: 5 5 4 3 5 7 7 7
16937 180/300 Write block 1809 len 24576 data: 4 3 3 4 4 3
16938 181/300 Flush
16939 182/300 Read block 1452 len 20480
16940 183/300 Flush
16941 184/300 Read block 352 len 36864
16942 185/300 Flush
16943 186/300 Read block 2021 len 32768
16944 187/300 Read block 4958 len 8192
16945 188/300 Read block 3990 len 40960
16946 189/300 Write block 1434 len 28672 data: 2 2 2 2 3 3 3
16947 190/300 Read block 3340 len 40960
16948 191/300 Write block 1072 len 40960 data: 4 3 3 3 3 2 3 4 3 2
16949 192/300 Read block 4677 len 12288
16950 193/300 Write block 2297 len 24576 data: 2 2 5 5 7 6
16951 194/300 Read block 1052 len 12288
16952 195/300 Read block 4809 len 8192
16953 196/300 Read block 4046 len 12288
16954 197/300 Read block 3111 len 36864
16955 198/300 Read block 1408 len 4096
16956 199/300 Read block 3168 len 4096
16957 200/300 Read block 1903 len 40960
16958 201/300 Read block 750 len 16384
16959 202/300 Write block 236 len 12288 data: 4 3 4
16960 203/300 Read block 751 len 28672
16961 204/300 Write block 3082 len 12288 data: 3 3 4
16962 205/300 Write block 2970 len 20480 data: 6 6 5 5 5
16963 206/300 Read block 4164 len 4096
16964 207/300 Read block 3278 len 40960
16965 208/300 Read block 74 len 28672
16966 209/300 Write block 3403 len 12288 data: 5 6 5
16967 210/300 Write block 2774 len 24576 data: 5 5 4 4 3 3
16968 211/300 Read block 968 len 12288
16969 212/300 Read block 4273 len 12288
16970 213/300 Write block 2717 len 32768 data: 4 3 3 3 3 3 3 3
16971 214/300 Write block 4792 len 28672 data: 4 4 4 5 5 5 4
16972 215/300 Read block 1898 len 32768
16973 216/300 Write block 1279 len 24576 data: 5 5 5 4 3 3
16974 217/300 Read block 869 len 28672
16975 218/300 Write block 3376 len 28672 data: 2 2 3 3 3 5 5
16976 219/300 Write block 626 len 32768 data: 3 3 2 2 2 4 5 4
16977 220/300 Read block 1007 len 28672
16978 221/300 Read block 4622 len 4096
16979 222/300 Write block 3408 len 8192 data: 5 5
16980 223/300 Write block 1887 len 4096 data: 4
16981 224/300 Write block 53 len 12288 data: 5 6 5
16982 225/300 Read block 1149 len 8192
16983 226/300 Write block 1211 len 40960 data: 6 5 4 5 4 4 4 4 4 6
16984 227/300 Write block 1621 len 40960 data: 6 4 4 4 4 4 4 4 2 2
16985 228/300 Flush
16986 229/300 Flush
16987 230/300 Read block 379 len 24576
16988 231/300 Read block 255 len 8192
16989 232/300 Write block 2271 len 4096 data: 6
16990 233/300 Flush
16991 234/300 Write block 953 len 8192 data: 5 6
16992 235/300 Read block 4274 len 20480
16993 236/300 Flush
16994 237/300 Read block 1690 len 12288
16995 238/300 Read block 264 len 40960
16996 239/300 Read block 1977 len 32768
16997 240/300 Read block 3913 len 24576
16998 241/300 Write block 4277 len 12288 data: 2 2 2
16999 242/300 Write block 4351 len 24576 data: 3 3 3 2 2 2
17000 243/300 Write block 1080 len 8192 data: 4 3
17001 244/300 Write block 751 len 16384 data: 5 4 4 6
17002 245/300 Read block 1425 len 40960
17003 246/300 Flush
17004 247/300 Read block 2457 len 40960
17005 248/300 Read block 3142 len 12288
17006 249/300 Write block 1033 len 4096 data: 2
17007 250/300 Write block 3158 len 32768 data: 4 4 3 4 4 4 2 2
17008 251/300 Read block 3709 len 20480
17009 252/300 Write block 3778 len 20480 data: 4 5 6 6 7
17010 253/300 Read block 4684 len 24576
17011 254/300 Write block 3588 len 40960 data: 6 7 7 8 8 7 7 6 6 8
17012 255/300 Write block 1881 len 28672 data: 4 3 4 5 6 4 5
17013 256/300 Read block 2876 len 20480
17014 257/300 Write block 2110 len 36864 data: 2 2 2 3 2 2 2 2 3
17015 258/300 Write block 2859 len 20480 data: 6 6 6 9 9
17016 259/300 Read block 2522 len 40960
17017 260/300 Read block 897 len 36864
17018 261/300 Read block 1456 len 32768
17019 262/300 Write block 2096 len 28672 data: 5 7 8 8 5 5 6
17020 263/300 Read block 2680 len 12288
17021 264/300 Write block 3686 len 16384 data: 3 3 2 3
17022 265/300 Read block 4817 len 8192
17023 266/300 Read block 4873 len 40960
17024 267/300 Read block 423 len 4096
17025 268/300 Read block 1142 len 20480
17026 269/300 Flush
17027 270/300 Write block 3707 len 16384 data: 7 6 5 5
17028 271/300 Read block 74 len 12288
17029 272/300 Flush
17030 273/300 Read block 1895 len 8192
17031 274/300 Read block 1652 len 28672
17032 275/300 Read block 595 len 32768
17033 276/300 Read block 1753 len 28672
17034 277/300 Read block 3638 len 12288
17035 278/300 Flush
17036 279/300 Write block 2363 len 36864 data: 2 2 2 2 2 2 2 3 3
17037 280/300 Read block 2079 len 28672
17038 281/300 Read block 944 len 40960
17039 282/300 Read block 2898 len 24576
17040 283/300 Read block 1285 len 36864
17041 284/300 Read block 2860 len 28672
17042 285/300 Write block 4511 len 20480 data: 7 6 6 4 5
17043 286/300 Read block 3167 len 36864
17044 287/300 Read block 3670 len 40960
17045 288/300 Write block 4270 len 16384 data: 5 6 6 5
17046 289/300 Write block 68 len 12288 data: 4 4 4
17047 290/300 Read block 2407 len 12288
17048 291/300 Flush
17049 292/300 Read block 1694 len 36864
17050 293/300 Read block 3942 len 32768
17051 294/300 Write block 1449 len 8192 data: 4 4
17052 295/300 Read block 4022 len 40960
17053 296/300 Read block 759 len 24576
17054 297/300 Write block 634 len 4096 data: 2
17055 298/300 Read block 2587 len 16384
17056 299/300 Read block 3412 len 16384
17057 300/300 Read block 2104 len 36864
17058 301/300 Write block 2244 len 20480 data: 4 4 4 4 4
17059 Main task has work to do, go find it
17060 got dsc Start(2)
17061 start 2
17062 Jan 11 06:09:02.503 INFO request completed, latency_us: 275, response_code: 204, uri: /start/cid/2, method: POST, req_id: ecbebd36-0275-481f-90ba-927b50947d30, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
17063 Replay: started 2, returned:Ok(())
17064 ----------------------------------------------------------------
17065 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17066 States: Active Active Offline
17067 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17068 5528 Acked Write 8 Done Done Sent false
17069 5529 Acked Read 10 Done Done Sent false
17070 5530 Acked Flush 0 Done Done Sent false
17071 5531 Acked Write 3 Done Done Sent false
17072 5532 Acked Write 8 Done Done Sent false
17073 5533 Acked Write 6 Done Done Sent false
17074 5534 Acked Read 3 Done Done Sent false
17075 5535 Acked Write 3 Done Done Sent false
17076 5536 Acked Write 9 Done Done Sent false
17077 5537 Acked Read 7 Done Done Sent false
17078 5538 Acked Flush 0 Done Done Sent false
17079 5539 Acked Read 7 Done Done Sent false
17080 5540 Acked Write 6 Done Done Sent false
17081 5541 Acked Write 1 Done Done Sent false
17082 5542 Acked Read 10 Done Done Sent false
17083 5543 Acked Write 3 Done Done Sent false
17084 5544 Acked Write 6 Done Done Sent false
17085 5545 Acked Read 8 Done Done Sent false
17086 5546 Acked Read 6 Done Done Sent false
17087 5547 Acked Flush 0 Done Done Sent false
17088 5548 Acked Write 5 Done Done Sent false
17089 5549 Acked Write 8 Done Done Sent false
17090 5550 Acked Read 3 Done Done Sent false
17091 5551 Acked Read 6 Done Done Sent false
17092 5552 Acked Flush 0 Done Done Sent false
17093 5553 Acked Read 5 Done Done Sent false
17094 5554 Acked Write 2 Done Done Sent false
17095 5555 Acked Read 5 Done Done Sent false
17096 5556 Acked Flush 0 Done Done Sent false
17097 5557 Acked Read 3 Done Done Sent false
17098 5558 Acked Read 3 Done Done Sent false
17099 5559 Acked Read 9 Done Done Sent false
17100 5560 Acked Read 5 Done Done Sent false
17101 5561 Acked Write 10 Done Done Sent false
17102 5562 Acked Read 8 Done Done Sent false
17103 5563 Acked Write 7 Done Done Sent false
17104 5564 Acked Write 6 Done Done Sent false
17105 5565 Acked Flush 0 Done Done Sent false
17106 5566 Acked Read 2 Done Done Sent false
17107 5567 Acked Write 9 Done Done Sent false
17108 5568 Acked Write 7 Done Done Sent false
17109 5569 Acked Read 7 Done Done Sent false
17110 5570 Acked Write 10 Done Done Sent false
17111 5571 Acked Flush 0 Done Done Sent false
17112 5572 Acked Read 1 Done Done Sent false
17113 5573 Acked Write 2 Done Done Sent false
17114 5574 Acked Write 8 Done Done Sent false
17115 5575 Acked Read 5 Done Done Sent false
17116 5576 Acked Write 8 Done Done Sent false
17117 5577 Acked Read 1 Done Done Sent false
17118 5578 Acked Flush 0 Done Done Sent false
17119 5579 Acked Flush 0 Done Done Sent false
17120 5580 Acked Read 6 Done Done Sent false
17121 5581 Acked Write 2 Done Done Sent false
17122 5582 Acked Flush 0 Done Done Sent false
17123 5583 Acked Write 5 Done Done Sent false
17124 5584 Acked Read 6 Done Done Sent false
17125 5585 Acked Write 8 Done Done Sent false
17126 5586 Acked Write 10 Done Done Sent false
17127 5587 Acked Write 2 Done Done Sent false
17128 5588 Acked Read 6 Done Done Sent false
17129 5589 Acked Flush 0 Done Done Sent false
17130 5590 Acked Read 2 Done Done Sent false
17131 5591 Acked Write 4 Done Done Sent false
17132 5592 Acked Read 8 Done Done Sent false
17133 5593 Acked Read 3 Done Done Sent false
17134 5594 Acked Read 4 Done Done Sent false
17135 5595 Acked Read 7 Done Done Sent false
17136 5596 Acked Read 4 Done Done Sent false
17137 5597 Acked Read 2 Done Done Sent false
17138 5598 Acked Read 6 Done Done Sent false
17139 5599 Acked Write 10 Done Done Sent false
17140 5600 Acked Write 6 Done Done Sent false
17141 5601 Acked Write 10 Done Done Sent false
17142 5602 Acked Write 6 Done Done Sent false
17143 5603 Acked Read 9 Done Done Sent false
17144 5604 Acked Read 4 Done Done Sent false
17145 5605 Acked Read 4 Done Done Sent false
17146 5606 Acked Write 9 Done Done Sent false
17147 5607 Acked Write 8 Done Done Sent false
17148 5608 Acked Read 8 Done Done Sent false
17149 5609 Acked Read 9 Done Done Sent false
17150 5610 Acked Write 9 Done Done Sent false
17151 5611 Acked Write 9 Done Done Sent false
17152 5612 Acked Read 10 Done Done Sent false
17153 5613 Acked Read 3 Done Done Sent false
17154 5614 Acked Read 7 Done Done Sent false
17155 5615 Acked Read 3 Done Done Sent false
17156 5616 Acked Read 2 Done Done Sent false
17157 5617 Acked Flush 0 Done Done Sent false
17158 5618 Acked Read 3 Done Done Sent false
17159 5619 Acked Read 2 Done Done Sent false
17160 5620 Acked Read 9 Done Done Sent false
17161 5621 Acked Write 7 Done Done Sent false
17162 5622 Acked Read 10 Done Done Sent false
17163 5623 Acked Read 7 Done Done Sent false
17164 5624 Acked Write 1 Done Done Sent false
17165 5625 Acked Write 3 Done Done Sent false
17166 5626 Acked Read 4 Done Done Sent false
17167 5627 Acked Read 8 Done Done Sent false
17168 5628 Acked Read 1 Done Done Sent false
17169 5629 Acked Read 2 Done Done Sent false
17170 5630 Acked Write 9 Done Done Sent false
17171 5631 Acked Write 7 Done Done Sent false
17172 5632 Acked Read 9 Done Done Sent false
17173 5633 Acked Write 4 Done Done Sent false
17174 5634 Acked Read 7 Done Done Sent false
17175 5635 Acked Write 5 Done Done Sent false
17176 5636 Acked Read 8 Done Done Sent false
17177 5637 Acked Read 5 Done Done Sent false
17178 5638 Acked Read 9 Done Done Sent false
17179 5639 Acked Write 1 Done Done Sent false
17180 5640 Acked Write 7 Done Done Sent false
17181 5641 Acked Write 7 Done Done Sent false
17182 5642 Acked Write 10 Done Done Sent false
17183 5643 Acked Write 7 Done Done Sent false
17184 5644 Acked Read 7 Done Done Sent false
17185 5645 Acked Read 1 Done Done Sent false
17186 5646 Acked Read 9 Done Done Sent false
17187 5647 Acked Read 8 Done Done Sent false
17188 5648 Acked Read 1 Done Done Sent false
17189 5649 Acked Read 1 Done Done Sent false
17190 5650 Acked Read 10 Done Done Sent false
17191 5651 Acked Read 3 Done Done Sent false
17192 5652 Acked Read 3 Done Done Sent false
17193 5653 Acked Write 5 Done Done Sent false
17194 5654 Acked Read 7 Done Done Sent false
17195 5655 Acked Read 4 Done Done Sent false
17196 5656 Acked Write 2 Done Done Sent false
17197 5657 Acked Write 10 Done Done Sent false
17198 5658 Acked Flush 0 Done Done Sent false
17199 5659 Acked Read 3 Done Done Sent false
17200 5660 Acked Write 1 Done Done Sent false
17201 5661 Acked Read 1 Done Done Sent false
17202 5662 Acked Write 9 Done Done Sent false
17203 5663 Acked Read 6 Done Done Sent false
17204 5664 Acked Write 1 Done Done Sent false
17205 5665 Acked Flush 0 Done Done Sent false
17206 5666 Acked Flush 0 Done Done Sent false
17207 5667 Acked Read 4 Done Done Sent false
17208 5668 Acked Read 1 Done Done Sent false
17209 5669 Acked Read 6 Done Done Sent false
17210 5670 Acked Read 7 Done Done Sent false
17211 5671 Acked Read 9 Done Done Sent false
17212 5672 Acked Read 9 Done Done Sent false
17213 5673 Acked Read 7 Done Done Sent false
17214 5674 Acked Read 7 Done Done Sent false
17215 5675 Acked Read 1 Done Done Sent false
17216 5676 Acked Read 1 Done Done Sent false
17217 5677 Acked Write 6 Done Done Sent false
17218 5678 Acked Write 3 Done Done Sent false
17219 5679 Acked Read 10 Done Done Sent false
17220 5680 Acked Read 2 Done Done Sent false
17221 5681 Acked Read 7 Done Done Sent false
17222 5682 Acked Write 9 Done Done Sent false
17223 5683 Acked Read 5 Done Done Sent false
17224 5684 Acked Flush 0 Done Done Sent false
17225 5685 Acked Write 8 Done Done Sent false
17226 5686 Acked Write 7 Done Done Sent false
17227 5687 Acked Write 9 Done Done Sent false
17228 5688 Acked Read 7 Done Done Sent false
17229 5689 Acked Read 4 Done Done Sent false
17230 5690 Acked Read 3 Done Done Sent false
17231 5691 Acked Write 5 Done Done Sent false
17232 5692 Acked Write 3 Done Done Sent false
17233 5693 Acked Read 4 Done Done Sent false
17234 5694 Acked Write 10 Done Done Sent false
17235 5695 Acked Read 4 Done Done Sent false
17236 5696 Acked Read 9 Done Done Sent false
17237 5697 Acked Read 7 Done Done Sent false
17238 5698 Acked Read 8 Done Done Sent false
17239 5699 Acked Write 4 Done Done Sent false
17240 5700 Acked Read 5 Done Done Sent false
17241 5701 Acked Write 3 Done Done Sent false
17242 5702 Acked Read 9 Done Done Sent false
17243 5703 Acked Read 6 Done Done Sent false
17244 5704 Acked Read 4 Done Done Sent false
17245 5705 Acked Read 1 Done Done Sent false
17246 5706 Acked Write 8 Done Done Sent false
17247 5707 Acked Write 6 Done Done Sent false
17248 5708 Acked Flush 0 Done Done Sent false
17249 5709 Acked Read 5 Done Done Sent false
17250 5710 Acked Flush 0 Done Done Sent false
17251 5711 Acked Read 9 Done Done Sent false
17252 5712 Acked Flush 0 Done Done Sent false
17253 5713 Acked Read 8 Done Done Sent false
17254 5714 Acked Read 2 Done Done Sent false
17255 5715 Acked Read 10 Done Done Sent false
17256 5716 Acked Write 7 Done Done Sent false
17257 5717 Acked Read 10 Done Done Sent false
17258 5718 Acked Write 10 Done Done Sent false
17259 5719 Acked Read 3 Done Done Sent false
17260 5720 Acked Write 6 Done Done Sent false
17261 5721 Acked Read 3 Done Done Sent false
17262 5722 Acked Read 2 Done Done Sent false
17263 5723 Acked Read 3 Done Done Sent false
17264 5724 Acked Read 9 Done Done Sent false
17265 5725 Acked Read 1 Done Done Sent false
17266 5726 Acked Read 1 Done Done Sent false
17267 5727 Acked Read 10 Done Done Sent false
17268 5728 Acked Read 4 Done Done Sent false
17269 5729 Acked Write 3 Done Done Sent false
17270 5730 Acked Read 7 Done Done Sent false
17271 5731 Acked Write 3 Done Done Sent false
17272 5732 Acked Write 5 Done Done Sent false
17273 5733 Acked Read 1 Done Done Sent false
17274 5734 Acked Read 10 Done Done Sent false
17275 5735 Acked Read 7 Done Done Sent false
17276 5736 Acked Write 3 Done Done Sent false
17277 5737 Acked Write 6 Done Done Sent false
17278 5738 Acked Read 3 Done Done Sent false
17279 5739 Acked Read 3 Done Done Sent false
17280 5740 Acked Write 8 Done Done Sent false
17281 5741 Acked Write 7 Done Done Sent false
17282 5742 Acked Read 8 Done Done Sent false
17283 5743 Acked Write 6 Done Done Sent false
17284 5744 Acked Read 7 Done Done Sent false
17285 5745 Acked Write 7 Done Done Sent false
17286 5746 Acked Write 8 Done Done Sent false
17287 5747 Acked Read 7 Done Done Sent false
17288 5748 Acked Read 1 Done Done Sent false
17289 5749 Acked Write 2 Done Done Sent false
17290 5750 Acked Write 1 Done Done Sent false
17291 5751 Acked Write 3 Done Done Sent false
17292 5752 Acked Read 2 Done Done Sent false
17293 5753 Acked Write 10 Done Done Sent false
17294 5754 Acked Write 10 Done Done Sent false
17295 5755 Acked Flush 0 Done Done Sent false
17296 5756 Acked Flush 0 Done Done Sent false
17297 5757 Acked Read 6 Done Done Sent false
17298 5758 Acked Read 2 Done Done Sent false
17299 5759 Acked Write 1 Done Done Sent false
17300 5760 Acked Flush 0 Done Done Sent false
17301 5761 Acked Write 2 Done Done Sent false
17302 5762 Acked Read 5 Done Done Sent false
17303 5763 Acked Flush 0 Done Done Sent false
17304 5764 Acked Read 3 Done Done Sent false
17305 5765 Acked Read 10 Done Done Sent false
17306 5766 Acked Read 8 Done Done Sent false
17307 5767 Acked Read 6 Done Done Sent false
17308 5768 Acked Write 3 Done Done Sent false
17309 5769 Acked Write 6 Done Done Sent false
17310 5770 Acked Write 2 Done Done Sent false
17311 5771 Acked Write 4 Done Done Sent false
17312 5772 Acked Read 10 Done Done Sent false
17313 5773 Acked Flush 0 Done Done Sent false
17314 5774 Acked Read 10 Done Done Sent false
17315 5775 Acked Read 3 Done Done Sent false
17316 5776 Acked Write 1 Done Done Sent false
17317 5777 Acked Write 8 Done Done Sent false
17318 5778 Acked Read 5 Done Done Sent false
17319 5779 Acked Write 5 Done Done Sent false
17320 5780 Acked Read 6 Done Done Sent false
17321 5781 Acked Write 10 Done Done Sent false
17322 5782 Acked Write 7 Done Done Sent false
17323 5783 Acked Read 5 Done Done Sent false
17324 5784 Acked Write 9 Done Done Sent false
17325 5785 Acked Write 5 Done Done Sent false
17326 5786 Acked Read 10 Done Done Sent false
17327 5787 Acked Read 9 Done Done Sent false
17328 5788 Acked Read 8 Done Done Sent false
17329 5789 Acked Write 7 Done Done Sent false
17330 5790 Acked Read 3 Done Done Sent false
17331 5791 Acked Write 4 Done Done Sent false
17332 5792 Acked Read 2 Done Done Sent false
17333 5793 Acked Read 10 Done Done Sent false
17334 5794 Acked Read 1 Done Done Sent false
17335 5795 Acked Read 5 Done Done Sent false
17336 5796 Acked Flush 0 Done Done Sent false
17337 5797 Acked Write 4 Done Done Sent false
17338 5798 Acked Read 3 Done Done Sent false
17339 5799 Acked Flush 0 Done Done Sent false
17340 5800 Acked Read 2 Done Done Sent false
17341 5801 Acked Read 7 Done Done Sent false
17342 5802 Acked Read 8 Done Done Sent false
17343 5803 Acked Read 7 Done Done Sent false
17344 5804 Acked Read 3 Done Done Sent false
17345 5805 Acked Flush 0 Done Done Sent false
17346 5806 Acked Write 9 Done Done Sent false
17347 5807 Acked Read 7 Done Done Sent false
17348 5808 Acked Read 10 Done Done Sent false
17349 5809 Acked Read 6 Done Done Sent false
17350 5810 Acked Read 9 Done Done Sent false
17351 5811 Acked Read 7 Done Done Sent false
17352 5812 Acked Write 5 Done Done Sent false
17353 5813 Acked Read 9 Done Done Sent false
17354 5814 Acked Read 10 Done Done Sent false
17355 5815 Acked Write 4 Done Done Sent false
17356 5816 Acked Write 3 Done Done Sent false
17357 5817 Acked Read 3 Done Done Sent false
17358 5818 Acked Flush 0 Done Done Sent false
17359 5819 Acked Read 9 Done Done Sent false
17360 5820 Acked Read 8 Done Done Sent false
17361 5821 Acked Write 2 Done Done Sent false
17362 5822 Acked Read 10 Done Done Sent false
17363 5823 Acked Read 6 Done Done Sent false
17364 5824 Acked Write 1 Done Done Sent false
17365 5825 Acked Read 4 Done Done Sent false
17366 5826 Acked Read 4 Done Done Sent false
17367 5827 Acked Read 9 Done Done Sent false
17368 5828 Acked Write 5 Done Done Sent false
17369 STATES DS:0 DS:1 DS:2 TOTAL
17370 Sent 0 0 301 301
17371 Done 301 301 0 602
17372 Skipped 0 0 0 0
17373 Error 0 0 0 0
17374 Last Flush: 5818 5818 5527
17375 Downstairs last five completed: 5527 5526 5525 5524 5523
17376 Upstairs last five completed: 5828 5827 5826 5825 5824
17377 CLIENT: Up:0 ds:301 act:2
17378 [8830] got start action
17379 [8830] I am going to restart
17380 Starting downstairs at port 8830
17381 Make output file at "/tmp/dsc/downstairs-8830.txt"
17382 [8830][2] reports Starting
17383 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1220)
17384 [8830][2] reports Running
17385 ----------------------------------------------------------------
17386 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17387 States: Active Active Offline
17388 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17389 5528 Acked Write 8 Done Done Sent false
17390 5529 Acked Read 10 Done Done Sent false
17391 5530 Acked Flush 0 Done Done Sent false
17392 5531 Acked Write 3 Done Done Sent false
17393 5532 Acked Write 8 Done Done Sent false
17394 5533 Acked Write 6 Done Done Sent false
17395 5534 Acked Read 3 Done Done Sent false
17396 5535 Acked Write 3 Done Done Sent false
17397 5536 Acked Write 9 Done Done Sent false
17398 5537 Acked Read 7 Done Done Sent false
17399 5538 Acked Flush 0 Done Done Sent false
17400 5539 Acked Read 7 Done Done Sent false
17401 5540 Acked Write 6 Done Done Sent false
17402 5541 Acked Write 1 Done Done Sent false
17403 5542 Acked Read 10 Done Done Sent false
17404 5543 Acked Write 3 Done Done Sent false
17405 5544 Acked Write 6 Done Done Sent false
17406 5545 Acked Read 8 Done Done Sent false
17407 5546 Acked Read 6 Done Done Sent false
17408 5547 Acked Flush 0 Done Done Sent false
17409 5548 Acked Write 5 Done Done Sent false
17410 5549 Acked Write 8 Done Done Sent false
17411 5550 Acked Read 3 Done Done Sent false
17412 5551 Acked Read 6 Done Done Sent false
17413 5552 Acked Flush 0 Done Done Sent false
17414 5553 Acked Read 5 Done Done Sent false
17415 5554 Acked Write 2 Done Done Sent false
17416 5555 Acked Read 5 Done Done Sent false
17417 5556 Acked Flush 0 Done Done Sent false
17418 5557 Acked Read 3 Done Done Sent false
17419 5558 Acked Read 3 Done Done Sent false
17420 5559 Acked Read 9 Done Done Sent false
17421 5560 Acked Read 5 Done Done Sent false
17422 5561 Acked Write 10 Done Done Sent false
17423 5562 Acked Read 8 Done Done Sent false
17424 5563 Acked Write 7 Done Done Sent false
17425 5564 Acked Write 6 Done Done Sent false
17426 5565 Acked Flush 0 Done Done Sent false
17427 5566 Acked Read 2 Done Done Sent false
17428 5567 Acked Write 9 Done Done Sent false
17429 5568 Acked Write 7 Done Done Sent false
17430 5569 Acked Read 7 Done Done Sent false
17431 5570 Acked Write 10 Done Done Sent false
17432 5571 Acked Flush 0 Done Done Sent false
17433 5572 Acked Read 1 Done Done Sent false
17434 5573 Acked Write 2 Done Done Sent false
17435 5574 Acked Write 8 Done Done Sent false
17436 5575 Acked Read 5 Done Done Sent false
17437 5576 Acked Write 8 Done Done Sent false
17438 5577 Acked Read 1 Done Done Sent false
17439 5578 Acked Flush 0 Done Done Sent false
17440 5579 Acked Flush 0 Done Done Sent false
17441 5580 Acked Read 6 Done Done Sent false
17442 5581 Acked Write 2 Done Done Sent false
17443 5582 Acked Flush 0 Done Done Sent false
17444 5583 Acked Write 5 Done Done Sent false
17445 5584 Acked Read 6 Done Done Sent false
17446 5585 Acked Write 8 Done Done Sent false
17447 5586 Acked Write 10 Done Done Sent false
17448 5587 Acked Write 2 Done Done Sent false
17449 5588 Acked Read 6 Done Done Sent false
17450 5589 Acked Flush 0 Done Done Sent false
17451 5590 Acked Read 2 Done Done Sent false
17452 5591 Acked Write 4 Done Done Sent false
17453 5592 Acked Read 8 Done Done Sent false
17454 5593 Acked Read 3 Done Done Sent false
17455 5594 Acked Read 4 Done Done Sent false
17456 5595 Acked Read 7 Done Done Sent false
17457 5596 Acked Read 4 Done Done Sent false
17458 5597 Acked Read 2 Done Done Sent false
17459 5598 Acked Read 6 Done Done Sent false
17460 5599 Acked Write 10 Done Done Sent false
17461 5600 Acked Write 6 Done Done Sent false
17462 5601 Acked Write 10 Done Done Sent false
17463 5602 Acked Write 6 Done Done Sent false
17464 5603 Acked Read 9 Done Done Sent false
17465 5604 Acked Read 4 Done Done Sent false
17466 5605 Acked Read 4 Done Done Sent false
17467 5606 Acked Write 9 Done Done Sent false
17468 5607 Acked Write 8 Done Done Sent false
17469 5608 Acked Read 8 Done Done Sent false
17470 5609 Acked Read 9 Done Done Sent false
17471 5610 Acked Write 9 Done Done Sent false
17472 5611 Acked Write 9 Done Done Sent false
17473 5612 Acked Read 10 Done Done Sent false
17474 5613 Acked Read 3 Done Done Sent false
17475 5614 Acked Read 7 Done Done Sent false
17476 5615 Acked Read 3 Done Done Sent false
17477 5616 Acked Read 2 Done Done Sent false
17478 5617 Acked Flush 0 Done Done Sent false
17479 5618 Acked Read 3 Done Done Sent false
17480 5619 Acked Read 2 Done Done Sent false
17481 5620 Acked Read 9 Done Done Sent false
17482 5621 Acked Write 7 Done Done Sent false
17483 5622 Acked Read 10 Done Done Sent false
17484 5623 Acked Read 7 Done Done Sent false
17485 5624 Acked Write 1 Done Done Sent false
17486 5625 Acked Write 3 Done Done Sent false
17487 5626 Acked Read 4 Done Done Sent false
17488 5627 Acked Read 8 Done Done Sent false
17489 5628 Acked Read 1 Done Done Sent false
17490 5629 Acked Read 2 Done Done Sent false
17491 5630 Acked Write 9 Done Done Sent false
17492 5631 Acked Write 7 Done Done Sent false
17493 5632 Acked Read 9 Done Done Sent false
17494 5633 Acked Write 4 Done Done Sent false
17495 5634 Acked Read 7 Done Done Sent false
17496 5635 Acked Write 5 Done Done Sent false
17497 5636 Acked Read 8 Done Done Sent false
17498 5637 Acked Read 5 Done Done Sent false
17499 5638 Acked Read 9 Done Done Sent false
17500 5639 Acked Write 1 Done Done Sent false
17501 5640 Acked Write 7 Done Done Sent false
17502 5641 Acked Write 7 Done Done Sent false
17503 5642 Acked Write 10 Done Done Sent false
17504 5643 Acked Write 7 Done Done Sent false
17505 5644 Acked Read 7 Done Done Sent false
17506 5645 Acked Read 1 Done Done Sent false
17507 5646 Acked Read 9 Done Done Sent false
17508 5647 Acked Read 8 Done Done Sent false
17509 5648 Acked Read 1 Done Done Sent false
17510 5649 Acked Read 1 Done Done Sent false
17511 5650 Acked Read 10 Done Done Sent false
17512 5651 Acked Read 3 Done Done Sent false
17513 5652 Acked Read 3 Done Done Sent false
17514 5653 Acked Write 5 Done Done Sent false
17515 5654 Acked Read 7 Done Done Sent false
17516 5655 Acked Read 4 Done Done Sent false
17517 5656 Acked Write 2 Done Done Sent false
17518 5657 Acked Write 10 Done Done Sent false
17519 5658 Acked Flush 0 Done Done Sent false
17520 5659 Acked Read 3 Done Done Sent false
17521 5660 Acked Write 1 Done Done Sent false
17522 5661 Acked Read 1 Done Done Sent false
17523 5662 Acked Write 9 Done Done Sent false
17524 5663 Acked Read 6 Done Done Sent false
17525 5664 Acked Write 1 Done Done Sent false
17526 5665 Acked Flush 0 Done Done Sent false
17527 5666 Acked Flush 0 Done Done Sent false
17528 5667 Acked Read 4 Done Done Sent false
17529 5668 Acked Read 1 Done Done Sent false
17530 5669 Acked Read 6 Done Done Sent false
17531 5670 Acked Read 7 Done Done Sent false
17532 5671 Acked Read 9 Done Done Sent false
17533 5672 Acked Read 9 Done Done Sent false
17534 5673 Acked Read 7 Done Done Sent false
17535 5674 Acked Read 7 Done Done Sent false
17536 5675 Acked Read 1 Done Done Sent false
17537 5676 Acked Read 1 Done Done Sent false
17538 5677 Acked Write 6 Done Done Sent false
17539 5678 Acked Write 3 Done Done Sent false
17540 5679 Acked Read 10 Done Done Sent false
17541 5680 Acked Read 2 Done Done Sent false
17542 5681 Acked Read 7 Done Done Sent false
17543 5682 Acked Write 9 Done Done Sent false
17544 5683 Acked Read 5 Done Done Sent false
17545 5684 Acked Flush 0 Done Done Sent false
17546 5685 Acked Write 8 Done Done Sent false
17547 5686 Acked Write 7 Done Done Sent false
17548 5687 Acked Write 9 Done Done Sent false
17549 5688 Acked Read 7 Done Done Sent false
17550 5689 Acked Read 4 Done Done Sent false
17551 5690 Acked Read 3 Done Done Sent false
17552 5691 Acked Write 5 Done Done Sent false
17553 5692 Acked Write 3 Done Done Sent false
17554 5693 Acked Read 4 Done Done Sent false
17555 5694 Acked Write 10 Done Done Sent false
17556 5695 Acked Read 4 Done Done Sent false
17557 5696 Acked Read 9 Done Done Sent false
17558 5697 Acked Read 7 Done Done Sent false
17559 5698 Acked Read 8 Done Done Sent false
17560 5699 Acked Write 4 Done Done Sent false
17561 5700 Acked Read 5 Done Done Sent false
17562 5701 Acked Write 3 Done Done Sent false
17563 5702 Acked Read 9 Done Done Sent false
17564 5703 Acked Read 6 Done Done Sent false
17565 5704 Acked Read 4 Done Done Sent false
17566 5705 Acked Read 1 Done Done Sent false
17567 5706 Acked Write 8 Done Done Sent false
17568 5707 Acked Write 6 Done Done Sent false
17569 5708 Acked Flush 0 Done Done Sent false
17570 5709 Acked Read 5 Done Done Sent false
17571 5710 Acked Flush 0 Done Done Sent false
17572 5711 Acked Read 9 Done Done Sent false
17573 5712 Acked Flush 0 Done Done Sent false
17574 5713 Acked Read 8 Done Done Sent false
17575 5714 Acked Read 2 Done Done Sent false
17576 5715 Acked Read 10 Done Done Sent false
17577 5716 Acked Write 7 Done Done Sent false
17578 5717 Acked Read 10 Done Done Sent false
17579 5718 Acked Write 10 Done Done Sent false
17580 5719 Acked Read 3 Done Done Sent false
17581 5720 Acked Write 6 Done Done Sent false
17582 5721 Acked Read 3 Done Done Sent false
17583 5722 Acked Read 2 Done Done Sent false
17584 5723 Acked Read 3 Done Done Sent false
17585 5724 Acked Read 9 Done Done Sent false
17586 5725 Acked Read 1 Done Done Sent false
17587 5726 Acked Read 1 Done Done Sent false
17588 5727 Acked Read 10 Done Done Sent false
17589 5728 Acked Read 4 Done Done Sent false
17590 5729 Acked Write 3 Done Done Sent false
17591 5730 Acked Read 7 Done Done Sent false
17592 5731 Acked Write 3 Done Done Sent false
17593 5732 Acked Write 5 Done Done Sent false
17594 5733 Acked Read 1 Done Done Sent false
17595 5734 Acked Read 10 Done Done Sent false
17596 5735 Acked Read 7 Done Done Sent false
17597 5736 Acked Write 3 Done Done Sent false
17598 5737 Acked Write 6 Done Done Sent false
17599 5738 Acked Read 3 Done Done Sent false
17600 5739 Acked Read 3 Done Done Sent false
17601 5740 Acked Write 8 Done Done Sent false
17602 5741 Acked Write 7 Done Done Sent false
17603 5742 Acked Read 8 Done Done Sent false
17604 5743 Acked Write 6 Done Done Sent false
17605 5744 Acked Read 7 Done Done Sent false
17606 5745 Acked Write 7 Done Done Sent false
17607 5746 Acked Write 8 Done Done Sent false
17608 5747 Acked Read 7 Done Done Sent false
17609 5748 Acked Read 1 Done Done Sent false
17610 5749 Acked Write 2 Done Done Sent false
17611 5750 Acked Write 1 Done Done Sent false
17612 5751 Acked Write 3 Done Done Sent false
17613 5752 Acked Read 2 Done Done Sent false
17614 5753 Acked Write 10 Done Done Sent false
17615 5754 Acked Write 10 Done Done Sent false
17616 5755 Acked Flush 0 Done Done Sent false
17617 5756 Acked Flush 0 Done Done Sent false
17618 5757 Acked Read 6 Done Done Sent false
17619 5758 Acked Read 2 Done Done Sent false
17620 5759 Acked Write 1 Done Done Sent false
17621 5760 Acked Flush 0 Done Done Sent false
17622 5761 Acked Write 2 Done Done Sent false
17623 5762 Acked Read 5 Done Done Sent false
17624 5763 Acked Flush 0 Done Done Sent false
17625 5764 Acked Read 3 Done Done Sent false
17626 5765 Acked Read 10 Done Done Sent false
17627 5766 Acked Read 8 Done Done Sent false
17628 5767 Acked Read 6 Done Done Sent false
17629 5768 Acked Write 3 Done Done Sent false
17630 5769 Acked Write 6 Done Done Sent false
17631 5770 Acked Write 2 Done Done Sent false
17632 5771 Acked Write 4 Done Done Sent false
17633 5772 Acked Read 10 Done Done Sent false
17634 5773 Acked Flush 0 Done Done Sent false
17635 5774 Acked Read 10 Done Done Sent false
17636 5775 Acked Read 3 Done Done Sent false
17637 5776 Acked Write 1 Done Done Sent false
17638 5777 Acked Write 8 Done Done Sent false
17639 5778 Acked Read 5 Done Done Sent false
17640 5779 Acked Write 5 Done Done Sent false
17641 5780 Acked Read 6 Done Done Sent false
17642 5781 Acked Write 10 Done Done Sent false
17643 5782 Acked Write 7 Done Done Sent false
17644 5783 Acked Read 5 Done Done Sent false
17645 5784 Acked Write 9 Done Done Sent false
17646 5785 Acked Write 5 Done Done Sent false
17647 5786 Acked Read 10 Done Done Sent false
17648 5787 Acked Read 9 Done Done Sent false
17649 5788 Acked Read 8 Done Done Sent false
17650 5789 Acked Write 7 Done Done Sent false
17651 5790 Acked Read 3 Done Done Sent false
17652 5791 Acked Write 4 Done Done Sent false
17653 5792 Acked Read 2 Done Done Sent false
17654 5793 Acked Read 10 Done Done Sent false
17655 5794 Acked Read 1 Done Done Sent false
17656 5795 Acked Read 5 Done Done Sent false
17657 5796 Acked Flush 0 Done Done Sent false
17658 5797 Acked Write 4 Done Done Sent false
17659 5798 Acked Read 3 Done Done Sent false
17660 5799 Acked Flush 0 Done Done Sent false
17661 5800 Acked Read 2 Done Done Sent false
17662 5801 Acked Read 7 Done Done Sent false
17663 5802 Acked Read 8 Done Done Sent false
17664 5803 Acked Read 7 Done Done Sent false
17665 5804 Acked Read 3 Done Done Sent false
17666 5805 Acked Flush 0 Done Done Sent false
17667 5806 Acked Write 9 Done Done Sent false
17668 5807 Acked Read 7 Done Done Sent false
17669 5808 Acked Read 10 Done Done Sent false
17670 5809 Acked Read 6 Done Done Sent false
17671 5810 Acked Read 9 Done Done Sent false
17672 5811 Acked Read 7 Done Done Sent false
17673 5812 Acked Write 5 Done Done Sent false
17674 5813 Acked Read 9 Done Done Sent false
17675 5814 Acked Read 10 Done Done Sent false
17676 5815 Acked Write 4 Done Done Sent false
17677 5816 Acked Write 3 Done Done Sent false
17678 5817 Acked Read 3 Done Done Sent false
17679 5818 Acked Flush 0 Done Done Sent false
17680 5819 Acked Read 9 Done Done Sent false
17681 5820 Acked Read 8 Done Done Sent false
17682 5821 Acked Write 2 Done Done Sent false
17683 5822 Acked Read 10 Done Done Sent false
17684 5823 Acked Read 6 Done Done Sent false
17685 5824 Acked Write 1 Done Done Sent false
17686 5825 Acked Read 4 Done Done Sent false
17687 5826 Acked Read 4 Done Done Sent false
17688 5827 Acked Read 9 Done Done Sent false
17689 5828 Acked Write 5 Done Done Sent false
17690 5829 Acked Flush 0 Done Done Sent false
17691 STATES DS:0 DS:1 DS:2 TOTAL
17692 Sent 0 0 302 302
17693 Done 302 302 0 604
17694 Skipped 0 0 0 0
17695 Error 0 0 0 0
17696 Last Flush: 5829 5829 5527
17697 Downstairs last five completed: 5527 5526 5525 5524 5523
17698 Upstairs last five completed: 5829 5828 5827 5826 5825
17699 CLIENT: Up:0 ds:302 act:2
177002025-01-11T06:09:08.248ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177012025-01-11T06:09:08.248ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55524) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177022025-01-11T06:09:08.249ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177032025-01-11T06:09:08.250ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177042025-01-11T06:09:08.250ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177052025-01-11T06:09:08.250ZINFOcrucible: send last flush ID to this DS: 5527 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177062025-01-11T06:09:08.250ZINFOcrucible: Replied this last flush ID: 5527 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177072025-01-11T06:09:08.250ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177082025-01-11T06:09:08.250ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177092025-01-11T06:09:08.250ZINFOcrucible: [2] client re-new 302 jobs since flush 5527 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177102025-01-11T06:09:08.250ZINFOcrucible: [2] Replayed 302 jobs since flush: 5527 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
17711 ----------------------------------------------------------------
17712 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17713 Downstairs last five completed: 5829 5828 5827 5826 5825
17714 Upstairs last five completed: 5829 5828 5827 5826 5825
17715 CLIENT: Up:0 ds:0 act:3
17716 Replay: All jobs finished
17717 Main task has work to do, go find it
17718 got dsc Stop(0)
17719 stop 0
17720 Jan 11 06:09:10.508[8810] Got stop action so:false kr:false
17721 INFO request completed, latency_us: 372, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 4df6ac8b-7d00-4dc8-aa33-f050acb9e08e, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
17722 Jan 11 06:09:10.509 INFO request completed, latency_us: 130, response_code: 200, uri: /state/cid/0, method: GET, req_id: c733e681-22ec-4f6d-adea-039664fb0b88, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
177232025-01-11T06:09:10.509ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177242025-01-11T06:09:10.509ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177252025-01-11T06:09:10.509ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177262025-01-11T06:09:10.509ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177272025-01-11T06:09:10.509ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
177282025-01-11T06:09:10.509ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
17729 [8810] Exited with: ExitStatus(unix_wait_status(9))
17730 [8810][0] reports Exit
17731 Jan 11 06:09:14.510 INFO request completed, latency_us: 260, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7ff82f72-7395-4af8-8893-d30a4b8e2942, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
17732 001/300 Write block 3967 len 8192 data: 2 2
17733 002/300 Write block 4039 len 32768 data: 4 4 5 5 6 6 6 6
17734 003/300 Read block 637 len 36864
17735 004/300 Write block 3879 len 24576 data: 2 3 3 3 5 6
17736 005/300 Read block 2722 len 32768
17737 006/300 Write block 3199 len 24576 data: 4 4 3 4 5 5
17738 007/300 Write block 689 len 20480 data: 6 6 5 7 8
17739 008/300 Read block 1082 len 40960
17740 009/300 Read block 2968 len 36864
17741 010/300 Read block 3416 len 24576
17742 011/300 Write block 765 len 36864 data: 4 4 4 4 3 3 3 3 5
17743 012/300 Write block 4734 len 20480 data: 3 3 3 2 3
17744 013/300 Read block 171 len 16384
17745 014/300 Flush
17746 015/300 Read block 3592 len 16384
17747 016/300 Write block 166 len 12288 data: 4 4 4
17748 017/300 Write block 2716 len 40960 data: 4 5 4 4 4 4 4 4 4 2
17749 018/300 Read block 3215 len 28672
17750 019/300 Flush
17751 020/300 Write block 1166 len 20480 data: 2 2 2 2 2
17752 021/300 Read block 1359 len 24576
17753 022/300 Write block 2140 len 28672 data: 3 3 3 3 3 2 3
17754 023/300 Read block 4513 len 20480
17755 024/300 Write block 2798 len 12288 data: 4 4 4
17756 025/300 Read block 557 len 8192
17757 026/300 Read block 2762 len 12288
17758 027/300 Write block 717 len 16384 data: 3 2 4 5
17759 028/300 Read block 3269 len 12288
17760 029/300 Read block 4542 len 28672
17761 030/300 Read block 375 len 24576
17762 031/300 Read block 3049 len 20480
17763 032/300 Flush
17764 033/300 Read block 1066 len 20480
17765 034/300 Read block 577 len 32768
17766 035/300 Read block 4737 len 40960
17767 036/300 Read block 4259 len 40960
17768 037/300 Write block 393 len 40960 data: 7 6 5 6 6 6 6 6 6 6
17769 038/300 Write block 4515 len 36864 data: 6 5 5 4 4 4 4 4 3
17770 039/300 Write block 2195 len 36864 data: 3 3 6 6 6 5 5 3 4
17771 040/300 Read block 786 len 32768
17772 041/300 Read block 1734 len 40960
17773 042/300 Read block 1113 len 40960
17774 043/300 Write block 4326 len 16384 data: 3 3 2 2
17775 044/300 Write block 3558 len 32768 data: 3 3 5 6 5 4 4 4
17776 045/300 Read block 2216 len 12288
17777 046/300 Write block 1092 len 20480 data: 4 5 5 6 5
17778 047/300 Read block 2386 len 36864
17779 048/300 Read block 2967 len 12288
17780 049/300 Write block 137 len 40960 data: 4 4 4 4 5 4 4 4 3 3
17781 050/300 Read block 3555 len 24576
17782 051/300 Flush
17783 052/300 Read block 431 len 16384
17784 053/300 Read block 2174 len 8192
17785 054/300 Read block 950 len 40960
17786 055/300 Write block 3567 len 16384 data: 4 3 4 4
17787 056/300 Write block 686 len 40960 data: 5 6 7 7 7 6 8 9 6 5
17788 057/300 Read block 960 len 40960
17789 058/300 Read block 77 len 8192
17790 059/300 Write block 679 len 12288 data: 2 2 2
17791 060/300 Read block 3149 len 24576
17792 061/300 Read block 2215 len 16384
17793 062/300 Write block 1479 len 12288 data: 5 6 6
17794 063/300 Read block 2320 len 16384
17795 064/300 Read block 712 len 8192
17796 065/300 Flush
17797 066/300 Write block 1683 len 28672 data: 6 6 8 6 6 3 3
17798 067/300 Write block 2029 len 16384 data: 3 5 4 3
17799 068/300 Write block 1732 len 4096 data: 4
17800 069/300 Flush
17801 070/300 Read block 1407 len 4096
17802 071/300 Read block 952 len 24576
17803 072/300 Write block 57 len 36864 data: 5 6 5 5 5 7 7 6 5
17804 073/300 Write block 2007 len 32768 data: 4 4 4 4 5 4 4 4
17805 074/300 Write block 2855 len 36864 data: 4 3 4 4 7 7 7 10 10
17806 075/300 Read block 1612 len 16384
17807 076/300 Read block 421 len 8192
17808 077/300 Write block 4833 len 4096 data: 5
17809 078/300 Write block 2924 len 8192 data: 5 4
17810 079/300 Write block 4717 len 36864 data: 4 4 3 3 3 3 3 3 2
17811 080/300 Read block 1246 len 24576
17812 081/300 Read block 2261 len 8192
17813 082/300 Write block 4113 len 16384 data: 4 3 3 3
17814 083/300 Flush
17815 084/300 Write block 3569 len 36864 data: 5 5 3 4 3 3 4 6 5
17816 085/300 Read block 0 len 32768
17817 086/300 Read block 3279 len 40960
17818 087/300 Write block 3426 len 28672 data: 4 4 2 3 4 4 6
17819 088/300 Write block 2152 len 4096 data: 5
17820 089/300 Read block 309 len 20480
17821 090/300 Read block 683 len 24576
17822 091/300 Read block 799 len 32768
17823 092/300 Read block 1343 len 32768
17824 093/300 Read block 1018 len 40960
17825 094/300 Read block 1244 len 4096
17826 095/300 Read block 3245 len 20480
17827 096/300 Write block 3789 len 4096 data: 3
17828 097/300 Read block 253 len 24576
17829 098/300 Read block 1561 len 32768
17830 099/300 Flush
17831 100/300 Flush
17832 101/300 Write block 1879 len 36864 data: 5 5 5 4 5 6 7 5 6
17833 102/300 Write block 200 len 4096 data: 4
17834 103/300 Read block 3311 len 24576
17835 104/300 Read block 3145 len 20480
17836 105/300 Write block 4996 len 8192 data: 4 3
17837 106/300 Read block 2156 len 4096
17838 107/300 Write block 3269 len 24576 data: 5 6 6 7 6 6
17839 108/300 Write block 2953 len 12288 data: 6 5 5
17840 109/300 Read block 2080 len 20480
17841 110/300 Read block 1257 len 20480
17842 111/300 Read block 468 len 32768
17843 112/300 Write block 4181 len 16384 data: 2 2 3 3
17844 113/300 Read block 2519 len 8192
17845 114/300 Write block 1610 len 4096 data: 4
17846 115/300 Read block 207 len 40960
17847 116/300 Read block 3250 len 20480
17848 117/300 Write block 3221 len 20480 data: 3 3 2 2 2
17849 118/300 Write block 3502 len 24576 data: 6 5 5 7 7 7
17850 119/300 Read block 3774 len 32768
17851 120/300 Read block 2520 len 12288
17852 121/300 Read block 614 len 8192
17853 122/300 Read block 3228 len 32768
17854 123/300 Read block 1284 len 36864
17855 124/300 Read block 3820 len 32768
17856 125/300 Flush
17857 126/300 Flush
17858 127/300 Read block 4432 len 8192
17859 128/300 Write block 1404 len 32768 data: 6 6 5 4 4 4 5 3
17860 129/300 Read block 4684 len 40960
17861 130/300 Read block 624 len 32768
17862 131/300 Write block 2257 len 24576 data: 6 6 5 4 3 3
17863 132/300 Read block 484 len 4096
17864 133/300 Write block 2934 len 28672 data: 7 8 7 7 6 4 6
17865 134/300 Read block 2578 len 12288
17866 135/300 Write block 2831 len 36864 data: 5 5 5 5 5 4 3 2 2
17867 136/300 Write block 2012 len 12288 data: 5 5 5
17868 137/300 Read block 4579 len 36864
17869 138/300 Write block 2524 len 32768 data: 7 8 6 7 7 5 4 4
17870 139/300 Read block 1917 len 40960
17871 140/300 Read block 2905 len 36864
17872 141/300 Read block 967 len 36864
17873 142/300 Write block 1742 len 36864 data: 4 5 5 4 4 3 4 4 4
17874 143/300 Read block 2567 len 12288
17875 144/300 Write block 689 len 36864 data: 8 8 7 9 10 7 6 5 5
17876 145/300 Flush
17877 146/300 Read block 1680 len 24576
17878 147/300 Read block 1846 len 32768
17879 148/300 Read block 3752 len 20480
17880 149/300 Read block 479 len 16384
17881 150/300 Flush
17882 151/300 Read block 229 len 20480
17883 152/300 Write block 3364 len 8192 data: 4 4
17884 153/300 Write block 3213 len 8192 data: 3 2
17885 154/300 Write block 3731 len 12288 data: 5 6 6
17886 155/300 Read block 4278 len 20480
17887 156/300 Read block 1119 len 16384
17888 157/300 Write block 2192 len 40960 data: 4 3 2 4 4 7 7 7 6 6
17889 158/300 Read block 272 len 40960
17890 159/300 Read block 3809 len 28672
17891 160/300 Write block 2478 len 12288 data: 3 4 4
17892 161/300 Read block 3839 len 12288
17893 162/300 Read block 3322 len 12288
17894 163/300 Read block 300 len 24576
17895 164/300 Read block 1995 len 12288
17896 165/300 Write block 934 len 36864 data: 4 4 4 5 5 4 4 4 4
17897 166/300 Write block 1121 len 8192 data: 6 5
17898 167/300 Write block 94 len 28672 data: 6 7 6 6 6 6 5
17899 168/300 Read block 3644 len 40960
17900 169/300 Read block 2151 len 16384
17901 170/300 Write block 19 len 4096 data: 3
17902 171/300 Read block 3577 len 28672
17903 172/300 Write block 4722 len 8192 data: 4 4
17904 173/300 Read block 4740 len 40960
17905 174/300 Write block 2039 len 4096 data: 4
17906 175/300 Flush
17907 176/300 Write block 1630 len 32768 data: 3 2 2 2 2 3 3 3
17908 177/300 Read block 1455 len 28672
17909 178/300 Read block 3020 len 12288
17910 179/300 Write block 1093 len 32768 data: 6 6 7 6 6 6 5 5
17911 180/300 Write block 3112 len 12288 data: 3 5 5
17912 181/300 Read block 3157 len 24576
17913 182/300 Write block 3612 len 28672 data: 5 5 5 5 3 3 2
17914 183/300 Write block 490 len 24576 data: 6 7 6 5 5 6
17915 184/300 Read block 3313 len 28672
17916 185/300 Read block 3028 len 32768
17917 186/300 Read block 1681 len 32768
17918 187/300 Write block 949 len 36864 data: 5 5 5 5 6 7 5 5 2
17919 188/300 Write block 4044 len 28672 data: 7 7 7 5 5 4 4
17920 189/300 Write block 668 len 28672 data: 4 5 4 5 4 4 4
17921 190/300 Read block 2341 len 16384
17922 191/300 Read block 4526 len 32768
17923 192/300 Read block 4399 len 8192
17924 193/300 Flush
17925 194/300 Write block 2918 len 4096 data: 3
17926 195/300 Read block 1371 len 8192
17927 196/300 Flush
17928 197/300 Read block 2317 len 12288
17929 198/300 Read block 222 len 24576
17930 199/300 Flush
17931 200/300 Read block 4545 len 12288
17932 201/300 Read block 1590 len 36864
17933 202/300 Write block 2219 len 24576 data: 6 6 6 6 4 3
17934 203/300 Read block 1719 len 28672
17935 204/300 Write block 4160 len 16384 data: 4 5 5 5
17936 205/300 Read block 4038 len 4096
17937 206/300 Flush
17938 207/300 Read block 4767 len 32768
17939 208/300 Read block 4679 len 28672
17940 209/300 Read block 4179 len 20480
17941 210/300 Read block 903 len 24576
17942 211/300 Read block 2326 len 28672
17943 212/300 Flush
17944 213/300 Read block 1585 len 24576
17945 214/300 Read block 1036 len 36864
17946 215/300 Read block 648 len 8192
17947 216/300 Write block 497 len 20480 data: 7 8 8 8 7
17948 217/300 Read block 1952 len 12288
17949 218/300 Read block 3468 len 4096
17950 219/300 Read block 4656 len 4096
17951 220/300 Read block 2462 len 36864
17952 221/300 Flush
17953 222/300 Read block 3061 len 32768
17954 223/300 Write block 3642 len 40960 data: 2 2 2 2 3 3 3 3 2 4
17955 224/300 Read block 2727 len 12288
17956 225/300 Read block 2501 len 32768
17957 226/300 Read block 1943 len 4096
17958 227/300 Flush
17959 228/300 Read block 4817 len 32768
17960 229/300 Write block 4802 len 36864 data: 5 5 4 4 3 3 3 4 2
17961 230/300 Read block 3250 len 28672
17962 231/300 Write block 179 len 4096 data: 2
17963 232/300 Write block 2906 len 36864 data: 5 5 4 3 2 2 2 2 3
17964 233/300 Write block 2770 len 20480 data: 2 4 6 5 6
17965 234/300 Read block 2926 len 28672
17966 235/300 Read block 2345 len 40960
17967 236/300 Read block 1924 len 28672
17968 237/300 Read block 3079 len 4096
17969 238/300 Read block 1023 len 12288
17970 239/300 Write block 1943 len 36864 data: 5 4 4 4 4 4 3 3 2
17971 240/300 Read block 3418 len 40960
17972 241/300 Write block 3429 len 36864 data: 4 5 5 7 6 6 4 3 3
17973 242/300 Write block 2755 len 20480 data: 4 4 4 4 4
17974 243/300 Write block 1337 len 28672 data: 2 2 2 2 2 2 2
17975 244/300 Read block 769 len 24576
17976 245/300 Write block 2111 len 4096 data: 3
17977 246/300 Write block 2897 len 8192 data: 4 5
17978 247/300 Read block 4436 len 24576
17979 248/300 Read block 87 len 16384
17980 249/300 Write block 53 len 4096 data: 6
17981 250/300 Write block 116 len 28672 data: 4 4 4 4 4 3 3
17982 251/300 Write block 4069 len 40960 data: 4 5 6 8 7 10 8 8 5 6
17983 252/300 Write block 2274 len 40960 data: 5 3 2 2 2 3 4 6 6 5
17984 253/300 Read block 3110 len 16384
17985 254/300 Flush
17986 255/300 Read block 2775 len 40960
17987 256/300 Read block 713 len 40960
17988 257/300 Read block 1862 len 16384
17989 258/300 Read block 3253 len 28672
17990 259/300 Read block 2914 len 16384
17991 260/300 Read block 3626 len 20480
17992 261/300 Write block 3852 len 4096 data: 3
17993 262/300 Read block 2871 len 36864
17994 263/300 Read block 3250 len 24576
17995 264/300 Write block 3245 len 4096 data: 5
17996 265/300 Flush
17997 266/300 Write block 4526 len 12288 data: 3 3 5
17998 267/300 Read block 1529 len 32768
17999 268/300 Read block 1924 len 24576
18000 269/300 Flush
18001 270/300 Read block 1620 len 12288
18002 271/300 Write block 3851 len 32768 data: 3 4 2 2 2 2 2 2
18003 272/300 Write block 3852 len 16384 data: 5 3 3 3
18004 273/300 Read block 2113 len 20480
18005 274/300 Write block 4207 len 32768 data: 4 6 5 6 7 6 5 4
18006 275/300 Write block 4081 len 12288 data: 4 3 4
18007 276/300 Read block 297 len 20480
18008 277/300 Write block 2947 len 28672 data: 9 7 9 9 7 6 7
18009 278/300 Write block 2380 len 40960 data: 5 6 5 5 4 3 3 3 4 4
18010 279/300 Read block 4240 len 32768
18011 280/300 Write block 4013 len 32768 data: 4 4 4 4 3 4 4 4
18012 281/300 Read block 1479 len 20480
18013 282/300 Flush
18014 283/300 Read block 3839 len 40960
18015 284/300 Write block 4087 len 36864 data: 4 5 5 4 4 4 3 2 2
18016 285/300 Read block 1344 len 12288
18017 286/300 Read block 150 len 20480
18018 287/300 Write block 2568 len 28672 data: 5 5 5 3 4 3 5
18019 288/300 Flush
18020 289/300 Read block 4590 len 20480
18021 290/300 Write block 4600 len 36864 data: 2 2 3 3 3 3 3 5 5
18022 291/300 Write block 4285 len 8192 data: 4 4
18023 292/300 Write block 1014 len 16384 data: 2 3 2 2
18024 293/300 Read block 4581 len 20480
18025 294/300 Read block 4633 len 32768
18026 295/300 Read block 4720 len 4096
18027 296/300 Write block 1392 len 40960 data: 2 3 3 3 3 3 3 4 3 3
18028 297/300 Write block 2939 len 12288 data: 5 7 6
18029 298/300 Write block 935 len 32768 data: 5 5 6 6 5 5 5 5
18030 299/300 Read block 4530 len 20480
18031 300/300 Write block 2866 len 20480 data: 8 7 7 6 6
18032 301/300 Read block 1541 len 8192
18033 Main task has work to do, go find it
18034 got dsc Start(0)
18035 start 0
18036 Jan 11 06:09:14.764 INFO request completed, latency_us: 231, response_code: 204, uri: /start/cid/0, method: POST, req_id: 780eeae2-8101-43d5-9946-9fb1171a9d31, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
18037 Replay: started 0, returned:Ok(())
18038 ----------------------------------------------------------------
18039 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18040 States: Offline Active Active
18041 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18042 5830 Acked Write 2 Sent Done Done false
18043 5831 Acked Write 8 Sent Done Done false
18044 5832 Acked Read 9 Sent Done Done false
18045 5833 Acked Write 6 Sent Done Done false
18046 5834 Acked Read 8 Sent Done Done false
18047 5835 Acked Write 6 Sent Done Done false
18048 5836 Acked Write 5 Sent Done Done false
18049 5837 Acked Read 10 Sent Done Done false
18050 5838 Acked Read 9 Sent Done Done false
18051 5839 Acked Read 6 Sent Done Done false
18052 5840 Acked Write 9 Sent Done Done false
18053 5841 Acked Write 5 Sent Done Done false
18054 5842 Acked Read 4 Sent Done Done false
18055 5843 Acked Flush 0 Sent Done Done false
18056 5844 Acked Read 4 Sent Done Done false
18057 5845 Acked Write 3 Sent Done Done false
18058 5846 Acked Write 10 Sent Done Done false
18059 5847 Acked Read 7 Sent Done Done false
18060 5848 Acked Flush 0 Sent Done Done false
18061 5849 Acked Write 5 Sent Done Done false
18062 5850 Acked Read 6 Sent Done Done false
18063 5851 Acked Write 7 Sent Done Done false
18064 5852 Acked Read 5 Sent Done Done false
18065 5853 Acked Write 3 Sent Done Done false
18066 5854 Acked Read 2 Sent Done Done false
18067 5855 Acked Read 3 Sent Done Done false
18068 5856 Acked Write 4 Sent Done Done false
18069 5857 Acked Read 3 Sent Done Done false
18070 5858 Acked Read 7 Sent Done Done false
18071 5859 Acked Read 6 Sent Done Done false
18072 5860 Acked Read 5 Sent Done Done false
18073 5861 Acked Flush 0 Sent Done Done false
18074 5862 Acked Read 5 Sent Done Done false
18075 5863 Acked Read 8 Sent Done Done false
18076 5864 Acked Read 10 Sent Done Done false
18077 5865 Acked Read 10 Sent Done Done false
18078 5866 Acked Write 10 Sent Done Done false
18079 5867 Acked Write 9 Sent Done Done false
18080 5868 Acked Write 9 Sent Done Done false
18081 5869 Acked Read 8 Sent Done Done false
18082 5870 Acked Read 10 Sent Done Done false
18083 5871 Acked Read 10 Sent Done Done false
18084 5872 Acked Write 4 Sent Done Done false
18085 5873 Acked Write 8 Sent Done Done false
18086 5874 Acked Read 3 Sent Done Done false
18087 5875 Acked Write 5 Sent Done Done false
18088 5876 Acked Read 9 Sent Done Done false
18089 5877 Acked Read 3 Sent Done Done false
18090 5878 Acked Write 10 Sent Done Done false
18091 5879 Acked Read 6 Sent Done Done false
18092 5880 Acked Flush 0 Sent Done Done false
18093 5881 Acked Read 4 Sent Done Done false
18094 5882 Acked Read 2 Sent Done Done false
18095 5883 Acked Read 10 Sent Done Done false
18096 5884 Acked Write 4 Sent Done Done false
18097 5885 Acked Write 10 Sent Done Done false
18098 5886 Acked Read 10 Sent Done Done false
18099 5887 Acked Read 2 Sent Done Done false
18100 5888 Acked Write 3 Sent Done Done false
18101 5889 Acked Read 6 Sent Done Done false
18102 5890 Acked Read 4 Sent Done Done false
18103 5891 Acked Write 3 Sent Done Done false
18104 5892 Acked Read 4 Sent Done Done false
18105 5893 Acked Read 2 Sent Done Done false
18106 5894 Acked Flush 0 Sent Done Done false
18107 5895 Acked Write 7 Sent Done Done false
18108 5896 Acked Write 4 Sent Done Done false
18109 5897 Acked Write 1 Sent Done Done false
18110 5898 Acked Flush 0 Sent Done Done false
18111 5899 Acked Read 1 Sent Done Done false
18112 5900 Acked Read 6 Sent Done Done false
18113 5901 Acked Write 9 Sent Done Done false
18114 5902 Acked Write 8 Sent Done Done false
18115 5903 Acked Write 9 Sent Done Done false
18116 5904 Acked Read 4 Sent Done Done false
18117 5905 Acked Read 2 Sent Done Done false
18118 5906 Acked Write 1 Sent Done Done false
18119 5907 Acked Write 2 Sent Done Done false
18120 5908 Acked Write 9 Sent Done Done false
18121 5909 Acked Read 6 Sent Done Done false
18122 5910 Acked Read 2 Sent Done Done false
18123 5911 Acked Write 4 Sent Done Done false
18124 5912 Acked Flush 0 Sent Done Done false
18125 5913 Acked Write 9 Sent Done Done false
18126 5914 Acked Read 8 Sent Done Done false
18127 5915 Acked Read 10 Sent Done Done false
18128 5916 Acked Write 7 Sent Done Done false
18129 5917 Acked Write 1 Sent Done Done false
18130 5918 Acked Read 5 Sent Done Done false
18131 5919 Acked Read 6 Sent Done Done false
18132 5920 Acked Read 8 Sent Done Done false
18133 5921 Acked Read 8 Sent Done Done false
18134 5922 Acked Read 10 Sent Done Done false
18135 5923 Acked Read 1 Sent Done Done false
18136 5924 Acked Read 5 Sent Done Done false
18137 5925 Acked Write 1 Sent Done Done false
18138 5926 Acked Read 6 Sent Done Done false
18139 5927 Acked Read 8 Sent Done Done false
18140 5928 Acked Flush 0 Sent Done Done false
18141 5929 Acked Flush 0 Sent Done Done false
18142 5930 Acked Write 9 Sent Done Done false
18143 5931 Acked Write 1 Sent Done Done false
18144 5932 Acked Read 6 Sent Done Done false
18145 5933 Acked Read 5 Sent Done Done false
18146 5934 Acked Write 2 Sent Done Done false
18147 5935 Acked Read 1 Sent Done Done false
18148 5936 Acked Write 6 Sent Done Done false
18149 5937 Acked Write 3 Sent Done Done false
18150 5938 Acked Read 5 Sent Done Done false
18151 5939 Acked Read 5 Sent Done Done false
18152 5940 Acked Read 8 Sent Done Done false
18153 5941 Acked Write 4 Sent Done Done false
18154 5942 Acked Read 2 Sent Done Done false
18155 5943 Acked Write 1 Sent Done Done false
18156 5944 Acked Read 10 Sent Done Done false
18157 5945 Acked Read 5 Sent Done Done false
18158 5946 Acked Write 5 Sent Done Done false
18159 5947 Acked Write 6 Sent Done Done false
18160 5948 Acked Read 8 Sent Done Done false
18161 5949 Acked Read 3 Sent Done Done false
18162 5950 Acked Read 2 Sent Done Done false
18163 5951 Acked Read 8 Sent Done Done false
18164 5952 Acked Read 9 Sent Done Done false
18165 5953 Acked Read 8 Sent Done Done false
18166 5954 Acked Flush 0 Sent Done Done false
18167 5955 Acked Flush 0 Sent Done Done false
18168 5956 Acked Read 2 Sent Done Done false
18169 5957 Acked Write 8 Sent Done Done false
18170 5958 Acked Read 10 Sent Done Done false
18171 5959 Acked Read 8 Sent Done Done false
18172 5960 Acked Write 6 Sent Done Done false
18173 5961 Acked Read 1 Sent Done Done false
18174 5962 Acked Write 7 Sent Done Done false
18175 5963 Acked Read 3 Sent Done Done false
18176 5964 Acked Write 9 Sent Done Done false
18177 5965 Acked Write 3 Sent Done Done false
18178 5966 Acked Read 9 Sent Done Done false
18179 5967 Acked Write 8 Sent Done Done false
18180 5968 Acked Read 10 Sent Done Done false
18181 5969 Acked Read 9 Sent Done Done false
18182 5970 Acked Read 9 Sent Done Done false
18183 5971 Acked Write 9 Sent Done Done false
18184 5972 Acked Read 3 Sent Done Done false
18185 5973 Acked Write 9 Sent Done Done false
18186 5974 Acked Flush 0 Sent Done Done false
18187 5975 Acked Read 6 Sent Done Done false
18188 5976 Acked Read 8 Sent Done Done false
18189 5977 Acked Read 5 Sent Done Done false
18190 5978 Acked Read 4 Sent Done Done false
18191 5979 Acked Flush 0 Sent Done Done false
18192 5980 Acked Read 5 Sent Done Done false
18193 5981 Acked Write 2 Sent Done Done false
18194 5982 Acked Write 2 Sent Done Done false
18195 5983 Acked Write 3 Sent Done Done false
18196 5984 Acked Read 5 Sent Done Done false
18197 5985 Acked Read 4 Sent Done Done false
18198 5986 Acked Write 10 Sent Done Done false
18199 5987 Acked Read 10 Sent Done Done false
18200 5988 Acked Read 7 Sent Done Done false
18201 5989 Acked Write 3 Sent Done Done false
18202 5990 Acked Read 3 Sent Done Done false
18203 5991 Acked Read 3 Sent Done Done false
18204 5992 Acked Read 6 Sent Done Done false
18205 5993 Acked Read 3 Sent Done Done false
18206 5994 Acked Write 9 Sent Done Done false
18207 5995 Acked Write 2 Sent Done Done false
18208 5996 Acked Write 7 Sent Done Done false
18209 5997 Acked Read 10 Sent Done Done false
18210 5998 Acked Read 4 Sent Done Done false
18211 5999 Acked Write 1 Sent Done Done false
18212 6000 Acked Read 7 Sent Done Done false
18213 6001 Acked Write 2 Sent Done Done false
18214 6002 Acked Read 10 Sent Done Done false
18215 6003 Acked Write 1 Sent Done Done false
18216 6004 Acked Flush 0 Sent Done Done false
18217 6005 Acked Write 8 Sent Done Done false
18218 6006 Acked Read 7 Sent Done Done false
18219 6007 Acked Read 3 Sent Done Done false
18220 6008 Acked Write 8 Sent Done Done false
18221 6009 Acked Write 3 Sent Done Done false
18222 6010 Acked Read 6 Sent Done Done false
18223 6011 Acked Write 7 Sent Done Done false
18224 6012 Acked Write 6 Sent Done Done false
18225 6013 Acked Read 7 Sent Done Done false
18226 6014 Acked Read 8 Sent Done Done false
18227 6015 Acked Read 8 Sent Done Done false
18228 6016 Acked Write 9 Sent Done Done false
18229 6017 Acked Write 7 Sent Done Done false
18230 6018 Acked Write 7 Sent Done Done false
18231 6019 Acked Read 4 Sent Done Done false
18232 6020 Acked Read 8 Sent Done Done false
18233 6021 Acked Read 2 Sent Done Done false
18234 6022 Acked Flush 0 Sent Done Done false
18235 6023 Acked Write 1 Sent Done Done false
18236 6024 Acked Read 2 Sent Done Done false
18237 6025 Acked Flush 0 Sent Done Done false
18238 6026 Acked Read 3 Sent Done Done false
18239 6027 Acked Read 6 Sent Done Done false
18240 6028 Acked Flush 0 Sent Done Done false
18241 6029 Acked Read 3 Sent Done Done false
18242 6030 Acked Read 9 Sent Done Done false
18243 6031 Acked Write 6 Sent Done Done false
18244 6032 Acked Read 7 Sent Done Done false
18245 6033 Acked Write 4 Sent Done Done false
18246 6034 Acked Read 1 Sent Done Done false
18247 6035 Acked Flush 0 Sent Done Done false
18248 6036 Acked Read 8 Sent Done Done false
18249 6037 Acked Read 7 Sent Done Done false
18250 6038 Acked Read 5 Sent Done Done false
18251 6039 Acked Read 6 Sent Done Done false
18252 6040 Acked Read 7 Sent Done Done false
18253 6041 Acked Flush 0 Sent Done Done false
18254 6042 Acked Read 6 Sent Done Done false
18255 6043 Acked Read 9 Sent Done Done false
18256 6044 Acked Read 2 Sent Done Done false
18257 6045 Acked Write 5 Sent Done Done false
18258 6046 Acked Read 3 Sent Done Done false
18259 6047 Acked Read 1 Sent Done Done false
18260 6048 Acked Read 1 Sent Done Done false
18261 6049 Acked Read 9 Sent Done Done false
18262 6050 Acked Flush 0 Sent Done Done false
18263 6051 Acked Read 8 Sent Done Done false
18264 6052 Acked Write 10 Sent Done Done false
18265 6053 Acked Read 3 Sent Done Done false
18266 6054 Acked Read 8 Sent Done Done false
18267 6055 Acked Read 1 Sent Done Done false
18268 6056 Acked Flush 0 Sent Done Done false
18269 6057 Acked Read 8 Sent Done Done false
18270 6058 Acked Write 9 Sent Done Done false
18271 6059 Acked Read 7 Sent Done Done false
18272 6060 Acked Write 1 Sent Done Done false
18273 6061 Acked Write 9 Sent Done Done false
18274 6062 Acked Write 5 Sent Done Done false
18275 6063 Acked Read 7 Sent Done Done false
18276 6064 Acked Read 10 Sent Done Done false
18277 6065 Acked Read 7 Sent Done Done false
18278 6066 Acked Read 1 Sent Done Done false
18279 6067 Acked Read 3 Sent Done Done false
18280 6068 Acked Write 9 Sent Done Done false
18281 6069 Acked Read 10 Sent Done Done false
18282 6070 Acked Write 9 Sent Done Done false
18283 6071 Acked Write 5 Sent Done Done false
18284 6072 Acked Write 7 Sent Done Done false
18285 6073 Acked Read 6 Sent Done Done false
18286 6074 Acked Write 1 Sent Done Done false
18287 6075 Acked Write 2 Sent Done Done false
18288 6076 Acked Read 6 Sent Done Done false
18289 6077 Acked Read 4 Sent Done Done false
18290 6078 Acked Write 1 Sent Done Done false
18291 6079 Acked Write 7 Sent Done Done false
18292 6080 Acked Write 10 Sent Done Done false
18293 6081 Acked Write 10 Sent Done Done false
18294 6082 Acked Read 4 Sent Done Done false
18295 6083 Acked Flush 0 Sent Done Done false
18296 6084 Acked Read 10 Sent Done Done false
18297 6085 Acked Read 10 Sent Done Done false
18298 6086 Acked Read 4 Sent Done Done false
18299 6087 Acked Read 7 Sent Done Done false
18300 6088 Acked Read 4 Sent Done Done false
18301 6089 Acked Read 5 Sent Done Done false
18302 6090 Acked Write 1 Sent Done Done false
18303 6091 Acked Read 9 Sent Done Done false
18304 6092 Acked Read 6 Sent Done Done false
18305 6093 Acked Write 1 Sent Done Done false
18306 6094 Acked Flush 0 Sent Done Done false
18307 6095 Acked Write 3 Sent Done Done false
18308 6096 Acked Read 8 Sent Done Done false
18309 6097 Acked Read 6 Sent Done Done false
18310 6098 Acked Flush 0 Sent Done Done false
18311 6099 Acked Read 3 Sent Done Done false
18312 6100 Acked Write 8 Sent Done Done false
18313 6101 Acked Write 4 Sent Done Done false
18314 6102 Acked Read 5 Sent Done Done false
18315 6103 Acked Write 8 Sent Done Done false
18316 6104 Acked Write 3 Sent Done Done false
18317 6105 Acked Read 5 Sent Done Done false
18318 6106 Acked Write 7 Sent Done Done false
18319 6107 Acked Write 10 Sent Done Done false
18320 6108 Acked Read 8 Sent Done Done false
18321 6109 Acked Write 8 Sent Done Done false
18322 6110 Acked Read 5 Sent Done Done false
18323 6111 Acked Flush 0 Sent Done Done false
18324 6112 Acked Read 10 Sent Done Done false
18325 6113 Acked Write 9 Sent Done Done false
18326 6114 Acked Read 3 Sent Done Done false
18327 6115 Acked Read 5 Sent Done Done false
18328 6116 Acked Write 7 Sent Done Done false
18329 6117 Acked Flush 0 Sent Done Done false
18330 6118 Acked Read 5 Sent Done Done false
18331 6119 Acked Write 9 Sent Done Done false
18332 6120 Acked Write 2 Sent Done Done false
18333 6121 Acked Write 4 Sent Done Done false
18334 6122 Acked Read 5 Sent Done Done false
18335 6123 Acked Read 8 Sent Done Done false
18336 6124 Acked Read 1 Sent Done Done false
18337 6125 Acked Write 10 Sent Done Done false
18338 6126 Acked Write 3 Sent Done Done false
18339 6127 Acked Write 8 Sent Done Done false
18340 6128 Acked Read 5 Sent Done Done false
18341 6129 Acked Write 5 Sent Done Done false
18342 6130 Acked Read 2 Sent Done Done false
18343 STATES DS:0 DS:1 DS:2 TOTAL
18344 Sent 301 0 0 301
18345 Done 0 301 301 602
18346 Skipped 0 0 0 0
18347 Error 0 0 0 0
18348 Last Flush: 5829 6117 6117
18349 Downstairs last five completed: 5829 5828 5827 5826 5825
18350 Upstairs last five completed: 6130 6129 6128 6127 6126
18351 CLIENT: Up:0 ds:301 act:2
18352 [8810] got start action
18353 [8810] I am going to restart
18354 Starting downstairs at port 8810
18355 Make output file at "/tmp/dsc/downstairs-8810.txt"
18356 [8810][0] reports Starting
18357 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1221)
18358 [8810][0] reports Running
18359 ----------------------------------------------------------------
18360 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18361 States: Offline Active Active
18362 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18363 5830 Acked Write 2 Sent Done Done false
18364 5831 Acked Write 8 Sent Done Done false
18365 5832 Acked Read 9 Sent Done Done false
18366 5833 Acked Write 6 Sent Done Done false
18367 5834 Acked Read 8 Sent Done Done false
18368 5835 Acked Write 6 Sent Done Done false
18369 5836 Acked Write 5 Sent Done Done false
18370 5837 Acked Read 10 Sent Done Done false
18371 5838 Acked Read 9 Sent Done Done false
18372 5839 Acked Read 6 Sent Done Done false
18373 5840 Acked Write 9 Sent Done Done false
18374 5841 Acked Write 5 Sent Done Done false
18375 5842 Acked Read 4 Sent Done Done false
18376 5843 Acked Flush 0 Sent Done Done false
18377 5844 Acked Read 4 Sent Done Done false
18378 5845 Acked Write 3 Sent Done Done false
18379 5846 Acked Write 10 Sent Done Done false
18380 5847 Acked Read 7 Sent Done Done false
18381 5848 Acked Flush 0 Sent Done Done false
18382 5849 Acked Write 5 Sent Done Done false
18383 5850 Acked Read 6 Sent Done Done false
18384 5851 Acked Write 7 Sent Done Done false
18385 5852 Acked Read 5 Sent Done Done false
18386 5853 Acked Write 3 Sent Done Done false
18387 5854 Acked Read 2 Sent Done Done false
18388 5855 Acked Read 3 Sent Done Done false
18389 5856 Acked Write 4 Sent Done Done false
18390 5857 Acked Read 3 Sent Done Done false
18391 5858 Acked Read 7 Sent Done Done false
18392 5859 Acked Read 6 Sent Done Done false
18393 5860 Acked Read 5 Sent Done Done false
18394 5861 Acked Flush 0 Sent Done Done false
18395 5862 Acked Read 5 Sent Done Done false
18396 5863 Acked Read 8 Sent Done Done false
18397 5864 Acked Read 10 Sent Done Done false
18398 5865 Acked Read 10 Sent Done Done false
18399 5866 Acked Write 10 Sent Done Done false
18400 5867 Acked Write 9 Sent Done Done false
18401 5868 Acked Write 9 Sent Done Done false
18402 5869 Acked Read 8 Sent Done Done false
18403 5870 Acked Read 10 Sent Done Done false
18404 5871 Acked Read 10 Sent Done Done false
18405 5872 Acked Write 4 Sent Done Done false
18406 5873 Acked Write 8 Sent Done Done false
18407 5874 Acked Read 3 Sent Done Done false
18408 5875 Acked Write 5 Sent Done Done false
18409 5876 Acked Read 9 Sent Done Done false
18410 5877 Acked Read 3 Sent Done Done false
18411 5878 Acked Write 10 Sent Done Done false
18412 5879 Acked Read 6 Sent Done Done false
18413 5880 Acked Flush 0 Sent Done Done false
18414 5881 Acked Read 4 Sent Done Done false
18415 5882 Acked Read 2 Sent Done Done false
18416 5883 Acked Read 10 Sent Done Done false
18417 5884 Acked Write 4 Sent Done Done false
18418 5885 Acked Write 10 Sent Done Done false
18419 5886 Acked Read 10 Sent Done Done false
18420 5887 Acked Read 2 Sent Done Done false
18421 5888 Acked Write 3 Sent Done Done false
18422 5889 Acked Read 6 Sent Done Done false
18423 5890 Acked Read 4 Sent Done Done false
18424 5891 Acked Write 3 Sent Done Done false
18425 5892 Acked Read 4 Sent Done Done false
18426 5893 Acked Read 2 Sent Done Done false
18427 5894 Acked Flush 0 Sent Done Done false
18428 5895 Acked Write 7 Sent Done Done false
18429 5896 Acked Write 4 Sent Done Done false
18430 5897 Acked Write 1 Sent Done Done false
18431 5898 Acked Flush 0 Sent Done Done false
18432 5899 Acked Read 1 Sent Done Done false
18433 5900 Acked Read 6 Sent Done Done false
18434 5901 Acked Write 9 Sent Done Done false
18435 5902 Acked Write 8 Sent Done Done false
18436 5903 Acked Write 9 Sent Done Done false
18437 5904 Acked Read 4 Sent Done Done false
18438 5905 Acked Read 2 Sent Done Done false
18439 5906 Acked Write 1 Sent Done Done false
18440 5907 Acked Write 2 Sent Done Done false
18441 5908 Acked Write 9 Sent Done Done false
18442 5909 Acked Read 6 Sent Done Done false
18443 5910 Acked Read 2 Sent Done Done false
18444 5911 Acked Write 4 Sent Done Done false
18445 5912 Acked Flush 0 Sent Done Done false
18446 5913 Acked Write 9 Sent Done Done false
18447 5914 Acked Read 8 Sent Done Done false
18448 5915 Acked Read 10 Sent Done Done false
18449 5916 Acked Write 7 Sent Done Done false
18450 5917 Acked Write 1 Sent Done Done false
18451 5918 Acked Read 5 Sent Done Done false
18452 5919 Acked Read 6 Sent Done Done false
18453 5920 Acked Read 8 Sent Done Done false
18454 5921 Acked Read 8 Sent Done Done false
18455 5922 Acked Read 10 Sent Done Done false
18456 5923 Acked Read 1 Sent Done Done false
18457 5924 Acked Read 5 Sent Done Done false
18458 5925 Acked Write 1 Sent Done Done false
18459 5926 Acked Read 6 Sent Done Done false
18460 5927 Acked Read 8 Sent Done Done false
18461 5928 Acked Flush 0 Sent Done Done false
18462 5929 Acked Flush 0 Sent Done Done false
18463 5930 Acked Write 9 Sent Done Done false
18464 5931 Acked Write 1 Sent Done Done false
18465 5932 Acked Read 6 Sent Done Done false
18466 5933 Acked Read 5 Sent Done Done false
18467 5934 Acked Write 2 Sent Done Done false
18468 5935 Acked Read 1 Sent Done Done false
18469 5936 Acked Write 6 Sent Done Done false
18470 5937 Acked Write 3 Sent Done Done false
18471 5938 Acked Read 5 Sent Done Done false
18472 5939 Acked Read 5 Sent Done Done false
18473 5940 Acked Read 8 Sent Done Done false
18474 5941 Acked Write 4 Sent Done Done false
18475 5942 Acked Read 2 Sent Done Done false
18476 5943 Acked Write 1 Sent Done Done false
18477 5944 Acked Read 10 Sent Done Done false
18478 5945 Acked Read 5 Sent Done Done false
18479 5946 Acked Write 5 Sent Done Done false
18480 5947 Acked Write 6 Sent Done Done false
18481 5948 Acked Read 8 Sent Done Done false
18482 5949 Acked Read 3 Sent Done Done false
18483 5950 Acked Read 2 Sent Done Done false
18484 5951 Acked Read 8 Sent Done Done false
18485 5952 Acked Read 9 Sent Done Done false
18486 5953 Acked Read 8 Sent Done Done false
18487 5954 Acked Flush 0 Sent Done Done false
18488 5955 Acked Flush 0 Sent Done Done false
18489 5956 Acked Read 2 Sent Done Done false
18490 5957 Acked Write 8 Sent Done Done false
18491 5958 Acked Read 10 Sent Done Done false
18492 5959 Acked Read 8 Sent Done Done false
18493 5960 Acked Write 6 Sent Done Done false
18494 5961 Acked Read 1 Sent Done Done false
18495 5962 Acked Write 7 Sent Done Done false
18496 5963 Acked Read 3 Sent Done Done false
18497 5964 Acked Write 9 Sent Done Done false
18498 5965 Acked Write 3 Sent Done Done false
18499 5966 Acked Read 9 Sent Done Done false
18500 5967 Acked Write 8 Sent Done Done false
18501 5968 Acked Read 10 Sent Done Done false
18502 5969 Acked Read 9 Sent Done Done false
18503 5970 Acked Read 9 Sent Done Done false
18504 5971 Acked Write 9 Sent Done Done false
18505 5972 Acked Read 3 Sent Done Done false
18506 5973 Acked Write 9 Sent Done Done false
18507 5974 Acked Flush 0 Sent Done Done false
18508 5975 Acked Read 6 Sent Done Done false
18509 5976 Acked Read 8 Sent Done Done false
18510 5977 Acked Read 5 Sent Done Done false
18511 5978 Acked Read 4 Sent Done Done false
18512 5979 Acked Flush 0 Sent Done Done false
18513 5980 Acked Read 5 Sent Done Done false
18514 5981 Acked Write 2 Sent Done Done false
18515 5982 Acked Write 2 Sent Done Done false
18516 5983 Acked Write 3 Sent Done Done false
18517 5984 Acked Read 5 Sent Done Done false
18518 5985 Acked Read 4 Sent Done Done false
18519 5986 Acked Write 10 Sent Done Done false
18520 5987 Acked Read 10 Sent Done Done false
18521 5988 Acked Read 7 Sent Done Done false
18522 5989 Acked Write 3 Sent Done Done false
18523 5990 Acked Read 3 Sent Done Done false
18524 5991 Acked Read 3 Sent Done Done false
18525 5992 Acked Read 6 Sent Done Done false
18526 5993 Acked Read 3 Sent Done Done false
18527 5994 Acked Write 9 Sent Done Done false
18528 5995 Acked Write 2 Sent Done Done false
18529 5996 Acked Write 7 Sent Done Done false
18530 5997 Acked Read 10 Sent Done Done false
18531 5998 Acked Read 4 Sent Done Done false
18532 5999 Acked Write 1 Sent Done Done false
18533 6000 Acked Read 7 Sent Done Done false
18534 6001 Acked Write 2 Sent Done Done false
18535 6002 Acked Read 10 Sent Done Done false
18536 6003 Acked Write 1 Sent Done Done false
18537 6004 Acked Flush 0 Sent Done Done false
18538 6005 Acked Write 8 Sent Done Done false
18539 6006 Acked Read 7 Sent Done Done false
18540 6007 Acked Read 3 Sent Done Done false
18541 6008 Acked Write 8 Sent Done Done false
18542 6009 Acked Write 3 Sent Done Done false
18543 6010 Acked Read 6 Sent Done Done false
18544 6011 Acked Write 7 Sent Done Done false
18545 6012 Acked Write 6 Sent Done Done false
18546 6013 Acked Read 7 Sent Done Done false
18547 6014 Acked Read 8 Sent Done Done false
18548 6015 Acked Read 8 Sent Done Done false
18549 6016 Acked Write 9 Sent Done Done false
18550 6017 Acked Write 7 Sent Done Done false
18551 6018 Acked Write 7 Sent Done Done false
18552 6019 Acked Read 4 Sent Done Done false
18553 6020 Acked Read 8 Sent Done Done false
18554 6021 Acked Read 2 Sent Done Done false
18555 6022 Acked Flush 0 Sent Done Done false
18556 6023 Acked Write 1 Sent Done Done false
18557 6024 Acked Read 2 Sent Done Done false
18558 6025 Acked Flush 0 Sent Done Done false
18559 6026 Acked Read 3 Sent Done Done false
18560 6027 Acked Read 6 Sent Done Done false
18561 6028 Acked Flush 0 Sent Done Done false
18562 6029 Acked Read 3 Sent Done Done false
18563 6030 Acked Read 9 Sent Done Done false
18564 6031 Acked Write 6 Sent Done Done false
18565 6032 Acked Read 7 Sent Done Done false
18566 6033 Acked Write 4 Sent Done Done false
18567 6034 Acked Read 1 Sent Done Done false
18568 6035 Acked Flush 0 Sent Done Done false
18569 6036 Acked Read 8 Sent Done Done false
18570 6037 Acked Read 7 Sent Done Done false
18571 6038 Acked Read 5 Sent Done Done false
18572 6039 Acked Read 6 Sent Done Done false
18573 6040 Acked Read 7 Sent Done Done false
18574 6041 Acked Flush 0 Sent Done Done false
18575 6042 Acked Read 6 Sent Done Done false
18576 6043 Acked Read 9 Sent Done Done false
18577 6044 Acked Read 2 Sent Done Done false
18578 6045 Acked Write 5 Sent Done Done false
18579 6046 Acked Read 3 Sent Done Done false
18580 6047 Acked Read 1 Sent Done Done false
18581 6048 Acked Read 1 Sent Done Done false
18582 6049 Acked Read 9 Sent Done Done false
18583 6050 Acked Flush 0 Sent Done Done false
18584 6051 Acked Read 8 Sent Done Done false
18585 6052 Acked Write 10 Sent Done Done false
18586 6053 Acked Read 3 Sent Done Done false
18587 6054 Acked Read 8 Sent Done Done false
18588 6055 Acked Read 1 Sent Done Done false
18589 6056 Acked Flush 0 Sent Done Done false
18590 6057 Acked Read 8 Sent Done Done false
18591 6058 Acked Write 9 Sent Done Done false
18592 6059 Acked Read 7 Sent Done Done false
18593 6060 Acked Write 1 Sent Done Done false
18594 6061 Acked Write 9 Sent Done Done false
18595 6062 Acked Write 5 Sent Done Done false
18596 6063 Acked Read 7 Sent Done Done false
18597 6064 Acked Read 10 Sent Done Done false
18598 6065 Acked Read 7 Sent Done Done false
18599 6066 Acked Read 1 Sent Done Done false
18600 6067 Acked Read 3 Sent Done Done false
18601 6068 Acked Write 9 Sent Done Done false
18602 6069 Acked Read 10 Sent Done Done false
18603 6070 Acked Write 9 Sent Done Done false
18604 6071 Acked Write 5 Sent Done Done false
18605 6072 Acked Write 7 Sent Done Done false
18606 6073 Acked Read 6 Sent Done Done false
18607 6074 Acked Write 1 Sent Done Done false
18608 6075 Acked Write 2 Sent Done Done false
18609 6076 Acked Read 6 Sent Done Done false
18610 6077 Acked Read 4 Sent Done Done false
18611 6078 Acked Write 1 Sent Done Done false
18612 6079 Acked Write 7 Sent Done Done false
18613 6080 Acked Write 10 Sent Done Done false
18614 6081 Acked Write 10 Sent Done Done false
18615 6082 Acked Read 4 Sent Done Done false
18616 6083 Acked Flush 0 Sent Done Done false
18617 6084 Acked Read 10 Sent Done Done false
18618 6085 Acked Read 10 Sent Done Done false
18619 6086 Acked Read 4 Sent Done Done false
18620 6087 Acked Read 7 Sent Done Done false
18621 6088 Acked Read 4 Sent Done Done false
18622 6089 Acked Read 5 Sent Done Done false
18623 6090 Acked Write 1 Sent Done Done false
18624 6091 Acked Read 9 Sent Done Done false
18625 6092 Acked Read 6 Sent Done Done false
18626 6093 Acked Write 1 Sent Done Done false
18627 6094 Acked Flush 0 Sent Done Done false
18628 6095 Acked Write 3 Sent Done Done false
18629 6096 Acked Read 8 Sent Done Done false
18630 6097 Acked Read 6 Sent Done Done false
18631 6098 Acked Flush 0 Sent Done Done false
18632 6099 Acked Read 3 Sent Done Done false
18633 6100 Acked Write 8 Sent Done Done false
18634 6101 Acked Write 4 Sent Done Done false
18635 6102 Acked Read 5 Sent Done Done false
18636 6103 Acked Write 8 Sent Done Done false
18637 6104 Acked Write 3 Sent Done Done false
18638 6105 Acked Read 5 Sent Done Done false
18639 6106 Acked Write 7 Sent Done Done false
18640 6107 Acked Write 10 Sent Done Done false
18641 6108 Acked Read 8 Sent Done Done false
18642 6109 Acked Write 8 Sent Done Done false
18643 6110 Acked Read 5 Sent Done Done false
18644 6111 Acked Flush 0 Sent Done Done false
18645 6112 Acked Read 10 Sent Done Done false
18646 6113 Acked Write 9 Sent Done Done false
18647 6114 Acked Read 3 Sent Done Done false
18648 6115 Acked Read 5 Sent Done Done false
18649 6116 Acked Write 7 Sent Done Done false
18650 6117 Acked Flush 0 Sent Done Done false
18651 6118 Acked Read 5 Sent Done Done false
18652 6119 Acked Write 9 Sent Done Done false
18653 6120 Acked Write 2 Sent Done Done false
18654 6121 Acked Write 4 Sent Done Done false
18655 6122 Acked Read 5 Sent Done Done false
18656 6123 Acked Read 8 Sent Done Done false
18657 6124 Acked Read 1 Sent Done Done false
18658 6125 Acked Write 10 Sent Done Done false
18659 6126 Acked Write 3 Sent Done Done false
18660 6127 Acked Write 8 Sent Done Done false
18661 6128 Acked Read 5 Sent Done Done false
18662 6129 Acked Write 5 Sent Done Done false
18663 6130 Acked Read 2 Sent Done Done false
18664 6131 Acked Flush 0 Sent Done Done false
18665 STATES DS:0 DS:1 DS:2 TOTAL
18666 Sent 302 0 0 302
18667 Done 0 302 302 604
18668 Skipped 0 0 0 0
18669 Error 0 0 0 0
18670 Last Flush: 5829 6131 6131
18671 Downstairs last five completed: 5829 5828 5827 5826 5825
18672 Upstairs last five completed: 6131 6130 6129 6128 6127
18673 CLIENT: Up:0 ds:302 act:2
186742025-01-11T06:09:20.510ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186752025-01-11T06:09:20.510ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59117) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186762025-01-11T06:09:20.511ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186772025-01-11T06:09:20.512ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186782025-01-11T06:09:20.512ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186792025-01-11T06:09:20.512ZINFOcrucible: send last flush ID to this DS: 5829 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186802025-01-11T06:09:20.512ZINFOcrucible: Replied this last flush ID: 5829 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186812025-01-11T06:09:20.512ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186822025-01-11T06:09:20.512ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186832025-01-11T06:09:20.512ZINFOcrucible: [0] client re-new 302 jobs since flush 5829 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
186842025-01-11T06:09:20.512ZINFOcrucible: [0] Replayed 302 jobs since flush: 5829 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
18685 ----------------------------------------------------------------
18686 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18687 Downstairs last five completed: 6131 6130 6129 6128 6127
18688 Upstairs last five completed: 6131 6130 6129 6128 6127
18689 CLIENT: Up:0 ds:0 act:3
18690 Replay: All jobs finished
18691 Main task has work to do, go find it
18692 got dsc Stop(1)
18693 stop 1
18694 Jan 11 06:09:22.769 INFO request completed, latency_us:[8820] Got stop action so:false kr:false
18695 382, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 649e9d84-abf2-4ad5-a76c-a50c19b52758, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
18696 Jan 11 06:09:22.770 INFO request completed, latency_us: 116, response_code: 200{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:09:22.770499574Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
18697 , uri: /state/cid/1, method: GET, req_id: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:09:22.770573094Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
18698 51b5a26a-9b64-471e-aff5-7643335f46a6, remote_addr:{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:09:22.77060721Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
18699 127.0.0.1:59925, local_addr: 127.0.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:09:22.770637636Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
18700 0.1:9998
187012025-01-11T06:09:22.770ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
187022025-01-11T06:09:22.770ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
18703 [8820] Exited with: ExitStatus(unix_wait_status(9))
18704 [8820][1] reports Exit
18705 Jan 11 06:09:26.773 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/1, method: GET, req_id: 95af666f-1313-43dc-aba8-adfbd8b44001, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
18706 001/300 Read block 1330 len 28672
18707 002/300 Write block 3481 len 40960 data: 6 6 4 5 5 5 3 3 3 3
18708 003/300 Read block 3310 len 36864
18709 004/300 Read block 2707 len 32768
18710 005/300 Read block 19 len 4096
18711 006/300 Flush
18712 007/300 Read block 30 len 20480
18713 008/300 Read block 20 len 20480
18714 009/300 Write block 3520 len 4096 data: 3
18715 010/300 Write block 639 len 24576 data: 2 2 4 4 4 4
18716 011/300 Flush
18717 012/300 Write block 4124 len 20480 data: 3 3 5 7 8
18718 013/300 Write block 4985 len 24576 data: 4 5 6 6 6 8
18719 014/300 Read block 2432 len 4096
18720 015/300 Read block 1811 len 20480
18721 016/300 Read block 4877 len 40960
18722 017/300 Read block 316 len 4096
18723 018/300 Read block 1847 len 36864
18724 019/300 Flush
18725 020/300 Write block 1533 len 8192 data: 4 4
18726 021/300 Write block 3032 len 24576 data: 5 6 5 5 5 8
18727 022/300 Write block 52 len 24576 data: 4 7 7 6 5 6
18728 023/300 Write block 3178 len 4096 data: 5
18729 024/300 Write block 2265 len 8192 data: 3 3
18730 025/300 Flush
18731 026/300 Read block 3478 len 32768
18732 027/300 Write block 4895 len 36864 data: 5 4 3 2 2 2 3 4 4
18733 028/300 Flush
18734 029/300 Write block 4172 len 8192 data: 4 3
18735 030/300 Flush
18736 031/300 Write block 202 len 40960 data: 3 3 4 4 4 2 3 3 3 4
18737 032/300 Read block 1817 len 24576
18738 033/300 Flush
18739 034/300 Write block 4405 len 24576 data: 5 7 5 5 6 6
18740 035/300 Read block 3276 len 36864
18741 036/300 Read block 764 len 8192
18742 037/300 Read block 4812 len 24576
18743 038/300 Flush
18744 039/300 Read block 2910 len 12288
18745 040/300 Read block 528 len 4096
18746 041/300 Write block 39 len 24576 data: 8 8 8 7 8 8
18747 042/300 Write block 1669 len 40960 data: 4 4 4 4 3 2 2 3 4 3
18748 043/300 Read block 1049 len 32768
18749 044/300 Read block 1933 len 20480
18750 045/300 Read block 2921 len 20480
18751 046/300 Write block 2774 len 20480 data: 7 6 5 5 4
18752 047/300 Write block 4980 len 36864 data: 5 4 3 5 5 5 6 7 7
18753 048/300 Flush
18754 049/300 Read block 3515 len 28672
18755 050/300 Write block 4965 len 36864 data: 4 4 4 4 3 3 4 5 6
18756 051/300 Read block 2755 len 12288
18757 052/300 Write block 528 len 12288 data: 5 4 5
18758 053/300 Write block 2630 len 20480 data: 4 4 4 3 3
18759 054/300 Write block 422 len 8192 data: 2 2
18760 055/300 Write block 4447 len 40960 data: 4 6 6 6 7 7 8 8 9 6
18761 056/300 Write block 172 len 12288 data: 3 4 4
18762 057/300 Write block 3431 len 8192 data: 6 8
18763 058/300 Write block 3948 len 16384 data: 5 5 5 5
18764 059/300 Write block 2422 len 32768 data: 5 5 4 2 3 3 3 3
18765 060/300 Write block 1558 len 12288 data: 3 4 4
18766 061/300 Read block 3817 len 20480
18767 062/300 Read block 854 len 8192
18768 063/300 Read block 757 len 28672
18769 064/300 Read block 636 len 12288
18770 065/300 Read block 1800 len 8192
18771 066/300 Read block 3311 len 16384
18772 067/300 Read block 606 len 4096
18773 068/300 Read block 166 len 40960
18774 069/300 Write block 3673 len 20480 data: 5 5 5 5 4
18775 070/300 Flush
18776 071/300 Write block 2572 len 12288 data: 5 4 6
18777 072/300 Write block 942 len 12288 data: 6 3 3
18778 073/300 Write block 591 len 24576 data: 3 3 3 3 4 5
18779 074/300 Read block 4436 len 20480
18780 075/300 Flush
18781 076/300 Read block 1035 len 16384
18782 077/300 Write block 3342 len 8192 data: 3 3
18783 078/300 Read block 661 len 28672
18784 079/300 Write block 1184 len 24576 data: 5 3 3 4 4 4
18785 080/300 Read block 426 len 12288
18786 081/300 Read block 4636 len 36864
18787 082/300 Read block 4577 len 8192
18788 083/300 Write block 1951 len 36864 data: 3 3 4 4 4 4 4 5 6
18789 084/300 Read block 1444 len 16384
18790 085/300 Write block 3720 len 36864 data: 2 2 2 3 3 3 3 5 5
18791 086/300 Write block 619 len 36864 data: 4 3 3 3 3 3 3 4 4
18792 087/300 Read block 1884 len 20480
18793 088/300 Write block 2863 len 32768 data: 11 9 8 9 8 8 7 7
18794 089/300 Read block 4409 len 28672
18795 090/300 Flush
18796 091/300 Write block 2045 len 36864 data: 5 5 4 4 4 5 4 4 6
18797 092/300 Read block 1953 len 8192
18798 093/300 Write block 4004 len 8192 data: 4 3
18799 094/300 Flush
18800 095/300 Read block 2548 len 36864
18801 096/300 Read block 778 len 24576
18802 097/300 Write block 2088 len 40960 data: 5 4 4 4 5 5 4 4 6 8
18803 098/300 Write block 396 len 12288 data: 7 7 7
18804 099/300 Write block 1398 len 32768 data: 4 5 4 4 4 4 7 7
18805 100/300 Write block 3526 len 8192 data: 6 5
18806 101/300 Flush
18807 102/300 Read block 2066 len 36864
18808 103/300 Write block 1793 len 40960 data: 2 2 3 3 4 3 3 3 3 3
18809 104/300 Write block 1073 len 16384 data: 4 4 4 4
18810 105/300 Read block 1700 len 32768
18811 106/300 Write block 4395 len 36864 data: 2 2 4 4 4 4 5 5 6
18812 107/300 Write block 2407 len 4096 data: 2
18813 108/300 Read block 2571 len 28672
18814 109/300 Read block 2645 len 40960
18815 110/300 Write block 4211 len 4096 data: 8
18816 111/300 Write block 530 len 36864 data: 6 4 5 6 2 2 2 4 4
18817 112/300 Read block 3837 len 16384
18818 113/300 Read block 48 len 8192
18819 114/300 Write block 2949 len 4096 data: 10
18820 115/300 Write block 836 len 24576 data: 5 4 5 4 6 6
18821 116/300 Read block 2307 len 36864
18822 117/300 Read block 3328 len 36864
18823 118/300 Read block 2372 len 28672
18824 119/300 Write block 4661 len 8192 data: 2 5
18825 120/300 Read block 941 len 24576
18826 121/300 Write block 4194 len 24576 data: 3 3 3 4 4 4
18827 122/300 Read block 2058 len 4096
18828 123/300 Read block 3154 len 40960
18829 124/300 Write block 2277 len 16384 data: 3 3 4 5
18830 125/300 Write block 2085 len 40960 data: 7 5 5 6 5 5 5 6 6 5
18831 126/300 Flush
18832 127/300 Read block 3560 len 40960
18833 128/300 Flush
18834 129/300 Write block 880 len 24576 data: 6 5 5 5 5 5
18835 130/300 Flush
18836 131/300 Write block 2575 len 28672 data: 5 5 5 4 3 4 5
18837 132/300 Read block 3438 len 20480
18838 133/300 Read block 2197 len 32768
18839 134/300 Read block 3521 len 24576
18840 135/300 Flush
18841 136/300 Read block 2397 len 32768
18842 137/300 Read block 1184 len 28672
18843 138/300 Write block 1749 len 20480 data: 5 5 4 3 3
18844 139/300 Write block 1317 len 24576 data: 2 2 2 2 3 3
18845 140/300 Write block 1687 len 36864 data: 7 4 4 5 6 6 6 5 4
18846 141/300 Read block 629 len 36864
18847 142/300 Write block 3008 len 32768 data: 7 6 3 3 3 3 5 4
18848 143/300 Read block 4949 len 20480
18849 144/300 Write block 3975 len 28672 data: 4 4 3 4 5 4 3
18850 145/300 Read block 3133 len 40960
18851 146/300 Read block 2290 len 40960
18852 147/300 Write block 145 len 20480 data: 4 4 3 3 2
18853 148/300 Write block 76 len 24576 data: 2 2 3 4 5 6
18854 149/300 Read block 3696 len 36864
18855 150/300 Write block 3290 len 12288 data: 2 2 2
18856 151/300 Write block 4796 len 32768 data: 6 6 5 4 4 4 6 6
18857 152/300 Flush
18858 153/300 Read block 3184 len 28672
18859 154/300 Write block 824 len 4096 data: 4
18860 155/300 Write block 3580 len 16384 data: 5 6 5 6
18861 156/300 Read block 3930 len 28672
18862 157/300 Read block 3427 len 8192
18863 158/300 Write block 468 len 12288 data: 7 7 6
18864 159/300 Read block 3753 len 36864
18865 160/300 Write block 4116 len 28672 data: 4 3 3 3 2 2 4
18866 161/300 Read block 800 len 12288
18867 162/300 Write block 3820 len 4096 data: 6
18868 163/300 Write block 3532 len 8192 data: 4 5
18869 164/300 Read block 4747 len 16384
18870 165/300 Write block 2123 len 32768 data: 3 3 3 3 4 5 5 5
18871 166/300 Write block 416 len 12288 data: 4 4 3
18872 167/300 Write block 1767 len 28672 data: 4 3 4 4 3 4 4
18873 168/300 Read block 2799 len 4096
18874 169/300 Read block 1515 len 20480
18875 170/300 Write block 1573 len 28672 data: 6 6 5 3 3 3 3
18876 171/300 Read block 4306 len 20480
18877 172/300 Read block 1682 len 4096
18878 173/300 Read block 692 len 36864
18879 174/300 Read block 3436 len 40960
18880 175/300 Write block 782 len 28672 data: 2 2 2 3 4 4 5
18881 176/300 Write block 4266 len 4096 data: 8
18882 177/300 Read block 212 len 32768
18883 178/300 Read block 2867 len 24576
18884 179/300 Write block 3783 len 32768 data: 5 4 4 5 4 3 4 4
18885 180/300 Write block 2363 len 24576 data: 3 3 3 3 3 3
18886 181/300 Read block 27 len 36864
18887 182/300 Write block 3185 len 4096 data: 4
18888 183/300 Read block 100 len 40960
18889 184/300 Read block 2253 len 32768
18890 185/300 Write block 1292 len 12288 data: 4 4 4
18891 186/300 Read block 1976 len 28672
18892 187/300 Write block 2393 len 20480 data: 6 6 7 6 6
18893 188/300 Write block 3495 len 40960 data: 3 3 2 3 4 5 6 7 6 6
18894 189/300 Read block 376 len 16384
18895 190/300 Read block 4624 len 20480
18896 191/300 Read block 4416 len 24576
18897 192/300 Read block 521 len 40960
18898 193/300 Write block 1682 len 16384 data: 6 7 7 9
18899 194/300 Read block 965 len 40960
18900 195/300 Write block 1663 len 36864 data: 2 2 2 2 3 3 5 5 5
18901 196/300 Read block 2067 len 20480
18902 197/300 Write block 2238 len 8192 data: 5 4
18903 198/300 Write block 740 len 32768 data: 3 3 3 5 6 6 5 4
18904 199/300 Read block 1363 len 32768
18905 200/300 Flush
18906 201/300 Read block 512 len 12288
18907 202/300 Write block 3115 len 8192 data: 4 4
18908 203/300 Read block 3763 len 8192
18909 204/300 Flush
18910 205/300 Write block 3677 len 4096 data: 5
18911 206/300 Write block 2866 len 36864 data: 10 9 9 8 8 6 4 5 3
18912 207/300 Read block 2210 len 16384
18913 208/300 Read block 3255 len 8192
18914 209/300 Read block 1478 len 36864
18915 210/300 Read block 819 len 32768
18916 211/300 Read block 2570 len 8192
18917 212/300 Write block 4045 len 8192 data: 8 8
18918 213/300 Read block 2118 len 4096
18919 214/300 Write block 4051 len 20480 data: 4 5 3 3 3
18920 215/300 Read block 3296 len 16384
18921 216/300 Read block 3668 len 16384
18922 217/300 Read block 2657 len 16384
18923 218/300 Read block 3849 len 28672
18924 219/300 Read block 4729 len 28672
18925 220/300 Flush
18926 221/300 Flush
18927 222/300 Read block 4486 len 40960
18928 223/300 Read block 919 len 24576
18929 224/300 Read block 3459 len 40960
18930 225/300 Read block 1199 len 8192
18931 226/300 Read block 4225 len 40960
18932 227/300 Write block 1976 len 36864 data: 4 6 5 7 4 3 3 3 3
18933 228/300 Write block 216 len 28672 data: 4 4 2 2 2 2 2
18934 229/300 Read block 759 len 40960
18935 230/300 Read block 21 len 12288
18936 231/300 Read block 3876 len 40960
18937 232/300 Read block 1483 len 12288
18938 233/300 Read block 389 len 32768
18939 234/300 Read block 1609 len 20480
18940 235/300 Read block 3003 len 8192
18941 236/300 Write block 3218 len 8192 data: 3 3
18942 237/300 Write block 3602 len 32768 data: 5 4 3 3 5 4 4 4
18943 238/300 Write block 544 len 36864 data: 4 4 4 3 5 6 5 5 6
18944 239/300 Read block 2937 len 16384
18945 240/300 Read block 771 len 40960
18946 241/300 Read block 132 len 8192
18947 242/300 Write block 4032 len 28672 data: 5 7 6 5 4 5 5
18948 243/300 Read block 378 len 40960
18949 244/300 Write block 1679 len 28672 data: 4 5 5 7 8 8 10
18950 245/300 Read block 2200 len 16384
18951 246/300 Read block 2708 len 36864
18952 247/300 Write block 3334 len 16384 data: 2 2 2 3
18953 248/300 Write block 4888 len 28672 data: 2 2 4 4 6 7 5
18954 249/300 Read block 3567 len 36864
18955 250/300 Read block 4947 len 4096
18956 251/300 Read block 4230 len 36864
18957 252/300 Read block 2174 len 32768
18958 253/300 Read block 2971 len 8192
18959 254/300 Read block 1937 len 28672
18960 255/300 Read block 289 len 16384
18961 256/300 Write block 2520 len 12288 data: 6 4 6
18962 257/300 Read block 1646 len 24576
18963 258/300 Read block 4539 len 4096
18964 259/300 Write block 101 len 32768 data: 5 5 3 3 3 5 4 4
18965 260/300 Read block 3858 len 16384
18966 261/300 Write block 3264 len 24576 data: 3 3 3 3 4 6
18967 262/300 Read block 4063 len 20480
18968 263/300 Read block 3436 len 4096
18969 264/300 Write block 4538 len 36864 data: 5 5 4 3 2 2 2 2 3
18970 265/300 Write block 2747 len 8192 data: 5 5
18971 266/300 Read block 1754 len 24576
18972 267/300 Read block 2754 len 12288
18973 268/300 Read block 3132 len 36864
18974 269/300 Write block 3389 len 4096 data: 4
18975 270/300 Read block 4378 len 12288
18976 271/300 Read block 887 len 12288
18977 272/300 Read block 3085 len 32768
18978 273/300 Flush
18979 274/300 Write block 2402 len 12288 data: 3 3 3
18980 275/300 Write block 4565 len 4096 data: 6
18981 276/300 Write block 4621 len 4096 data: 5
18982 277/300 Read block 2212 len 28672
18983 278/300 Read block 3496 len 40960
18984 279/300 Read block 3156 len 16384
18985 280/300 Read block 390 len 28672
18986 281/300 Read block 1736 len 24576
18987 282/300 Write block 190 len 32768 data: 6 6 7 6 5 5 4 4
18988 283/300 Write block 1749 len 40960 data: 6 6 5 4 4 5 4 4 5 5
18989 284/300 Flush
18990 285/300 Read block 345 len 36864
18991 286/300 Write block 75 len 8192 data: 2 3
18992 287/300 Read block 4446 len 32768
18993 288/300 Flush
18994 289/300 Write block 3697 len 16384 data: 6 6 6 6
18995 290/300 Read block 4521 len 40960
18996 291/300 Read block 16 len 4096
18997 292/300 Read block 1218 len 20480
18998 293/300 Read block 2873 len 12288
18999 294/300 Flush
19000 295/300 Read block 3062 len 28672
19001 296/300 Read block 4966 len 12288
19002 297/300 Read block 1114 len 4096
19003 298/300 Write block 3362 len 28672 data: 3 3 5 5 7 6 6
19004 299/300 Read block 270 len 36864
19005 300/300 Read block 3975 len 40960
19006 301/300 Read block 4653 len 4096
19007 Main task has work to do, go find it
19008 got dsc Start(1)
19009 start 1
19010 Jan 11 06:09:27.024 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/1, method: POST, req_id: 49c5def1-c943-43bb-bf67-74ca431c3e52, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
19011 Replay: started 1, returned:Ok(())
19012 ----------------------------------------------------------------
19013 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19014 States: Active Offline Active
19015 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19016 6132 Acked Read 7 Done Sent Done false
19017 6133 Acked Write 10 Done Sent Done false
19018 6134 Acked Read 9 Done Sent Done false
19019 6135 Acked Read 8 Done Sent Done false
19020 6136 Acked Read 1 Done Sent Done false
19021 6137 Acked Flush 0 Done Sent Done false
19022 6138 Acked Read 5 Done Sent Done false
19023 6139 Acked Read 5 Done Sent Done false
19024 6140 Acked Write 1 Done Sent Done false
19025 6141 Acked Write 6 Done Sent Done false
19026 6142 Acked Flush 0 Done Sent Done false
19027 6143 Acked Write 5 Done Sent Done false
19028 6144 Acked Write 6 Done Sent Done false
19029 6145 Acked Read 1 Done Sent Done false
19030 6146 Acked Read 5 Done Sent Done false
19031 6147 Acked Read 10 Done Sent Done false
19032 6148 Acked Read 1 Done Sent Done false
19033 6149 Acked Read 9 Done Sent Done false
19034 6150 Acked Flush 0 Done Sent Done false
19035 6151 Acked Write 2 Done Sent Done false
19036 6152 Acked Write 6 Done Sent Done false
19037 6153 Acked Write 6 Done Sent Done false
19038 6154 Acked Write 1 Done Sent Done false
19039 6155 Acked Write 2 Done Sent Done false
19040 6156 Acked Flush 0 Done Sent Done false
19041 6157 Acked Read 8 Done Sent Done false
19042 6158 Acked Write 9 Done Sent Done false
19043 6159 Acked Flush 0 Done Sent Done false
19044 6160 Acked Write 2 Done Sent Done false
19045 6161 Acked Flush 0 Done Sent Done false
19046 6162 Acked Write 10 Done Sent Done false
19047 6163 Acked Read 6 Done Sent Done false
19048 6164 Acked Flush 0 Done Sent Done false
19049 6165 Acked Write 6 Done Sent Done false
19050 6166 Acked Read 9 Done Sent Done false
19051 6167 Acked Read 2 Done Sent Done false
19052 6168 Acked Read 6 Done Sent Done false
19053 6169 Acked Flush 0 Done Sent Done false
19054 6170 Acked Read 3 Done Sent Done false
19055 6171 Acked Read 1 Done Sent Done false
19056 6172 Acked Write 6 Done Sent Done false
19057 6173 Acked Write 10 Done Sent Done false
19058 6174 Acked Read 8 Done Sent Done false
19059 6175 Acked Read 5 Done Sent Done false
19060 6176 Acked Read 5 Done Sent Done false
19061 6177 Acked Write 5 Done Sent Done false
19062 6178 Acked Write 9 Done Sent Done false
19063 6179 Acked Flush 0 Done Sent Done false
19064 6180 Acked Read 7 Done Sent Done false
19065 6181 Acked Write 9 Done Sent Done false
19066 6182 Acked Read 3 Done Sent Done false
19067 6183 Acked Write 3 Done Sent Done false
19068 6184 Acked Write 5 Done Sent Done false
19069 6185 Acked Write 2 Done Sent Done false
19070 6186 Acked Write 10 Done Sent Done false
19071 6187 Acked Write 3 Done Sent Done false
19072 6188 Acked Write 2 Done Sent Done false
19073 6189 Acked Write 4 Done Sent Done false
19074 6190 Acked Write 8 Done Sent Done false
19075 6191 Acked Write 3 Done Sent Done false
19076 6192 Acked Read 5 Done Sent Done false
19077 6193 Acked Read 2 Done Sent Done false
19078 6194 Acked Read 7 Done Sent Done false
19079 6195 Acked Read 3 Done Sent Done false
19080 6196 Acked Read 2 Done Sent Done false
19081 6197 Acked Read 4 Done Sent Done false
19082 6198 Acked Read 1 Done Sent Done false
19083 6199 Acked Read 10 Done Sent Done false
19084 6200 Acked Write 5 Done Sent Done false
19085 6201 Acked Flush 0 Done Sent Done false
19086 6202 Acked Write 3 Done Sent Done false
19087 6203 Acked Write 3 Done Sent Done false
19088 6204 Acked Write 6 Done Sent Done false
19089 6205 Acked Read 5 Done Sent Done false
19090 6206 Acked Flush 0 Done Sent Done false
19091 6207 Acked Read 4 Done Sent Done false
19092 6208 Acked Write 2 Done Sent Done false
19093 6209 Acked Read 7 Done Sent Done false
19094 6210 Acked Write 6 Done Sent Done false
19095 6211 Acked Read 3 Done Sent Done false
19096 6212 Acked Read 9 Done Sent Done false
19097 6213 Acked Read 2 Done Sent Done false
19098 6214 Acked Write 9 Done Sent Done false
19099 6215 Acked Read 4 Done Sent Done false
19100 6216 Acked Write 9 Done Sent Done false
19101 6217 Acked Write 9 Done Sent Done false
19102 6218 Acked Read 5 Done Sent Done false
19103 6219 Acked Write 8 Done Sent Done false
19104 6220 Acked Read 7 Done Sent Done false
19105 6221 Acked Flush 0 Done Sent Done false
19106 6222 Acked Write 9 Done Sent Done false
19107 6223 Acked Read 2 Done Sent Done false
19108 6224 Acked Write 2 Done Sent Done false
19109 6225 Acked Flush 0 Done Sent Done false
19110 6226 Acked Read 9 Done Sent Done false
19111 6227 Acked Read 6 Done Sent Done false
19112 6228 Acked Write 10 Done Sent Done false
19113 6229 Acked Write 3 Done Sent Done false
19114 6230 Acked Write 8 Done Sent Done false
19115 6231 Acked Write 2 Done Sent Done false
19116 6232 Acked Flush 0 Done Sent Done false
19117 6233 Acked Read 9 Done Sent Done false
19118 6234 Acked Write 10 Done Sent Done false
19119 6235 Acked Write 4 Done Sent Done false
19120 6236 Acked Read 8 Done Sent Done false
19121 6237 Acked Write 9 Done Sent Done false
19122 6238 Acked Write 1 Done Sent Done false
19123 6239 Acked Read 7 Done Sent Done false
19124 6240 Acked Read 10 Done Sent Done false
19125 6241 Acked Write 1 Done Sent Done false
19126 6242 Acked Write 9 Done Sent Done false
19127 6243 Acked Read 4 Done Sent Done false
19128 6244 Acked Read 2 Done Sent Done false
19129 6245 Acked Write 1 Done Sent Done false
19130 6246 Acked Write 6 Done Sent Done false
19131 6247 Acked Read 9 Done Sent Done false
19132 6248 Acked Read 9 Done Sent Done false
19133 6249 Acked Read 7 Done Sent Done false
19134 6250 Acked Write 2 Done Sent Done false
19135 6251 Acked Read 6 Done Sent Done false
19136 6252 Acked Write 6 Done Sent Done false
19137 6253 Acked Read 1 Done Sent Done false
19138 6254 Acked Read 10 Done Sent Done false
19139 6255 Acked Write 4 Done Sent Done false
19140 6256 Acked Write 10 Done Sent Done false
19141 6257 Acked Flush 0 Done Sent Done false
19142 6258 Acked Read 10 Done Sent Done false
19143 6259 Acked Flush 0 Done Sent Done false
19144 6260 Acked Write 6 Done Sent Done false
19145 6261 Acked Flush 0 Done Sent Done false
19146 6262 Acked Write 7 Done Sent Done false
19147 6263 Acked Read 5 Done Sent Done false
19148 6264 Acked Read 8 Done Sent Done false
19149 6265 Acked Read 6 Done Sent Done false
19150 6266 Acked Flush 0 Done Sent Done false
19151 6267 Acked Read 8 Done Sent Done false
19152 6268 Acked Read 7 Done Sent Done false
19153 6269 Acked Write 5 Done Sent Done false
19154 6270 Acked Write 6 Done Sent Done false
19155 6271 Acked Write 9 Done Sent Done false
19156 6272 Acked Read 9 Done Sent Done false
19157 6273 Acked Write 8 Done Sent Done false
19158 6274 Acked Read 5 Done Sent Done false
19159 6275 Acked Write 7 Done Sent Done false
19160 6276 Acked Read 10 Done Sent Done false
19161 6277 Acked Read 10 Done Sent Done false
19162 6278 Acked Write 5 Done Sent Done false
19163 6279 Acked Write 6 Done Sent Done false
19164 6280 Acked Read 9 Done Sent Done false
19165 6281 Acked Write 3 Done Sent Done false
19166 6282 Acked Write 8 Done Sent Done false
19167 6283 Acked Flush 0 Done Sent Done false
19168 6284 Acked Read 7 Done Sent Done false
19169 6285 Acked Write 1 Done Sent Done false
19170 6286 Acked Write 4 Done Sent Done false
19171 6287 Acked Read 7 Done Sent Done false
19172 6288 Acked Read 2 Done Sent Done false
19173 6289 Acked Write 3 Done Sent Done false
19174 6290 Acked Read 9 Done Sent Done false
19175 6291 Acked Write 7 Done Sent Done false
19176 6292 Acked Read 3 Done Sent Done false
19177 6293 Acked Write 1 Done Sent Done false
19178 6294 Acked Write 2 Done Sent Done false
19179 6295 Acked Read 4 Done Sent Done false
19180 6296 Acked Write 8 Done Sent Done false
19181 6297 Acked Write 3 Done Sent Done false
19182 6298 Acked Write 7 Done Sent Done false
19183 6299 Acked Read 1 Done Sent Done false
19184 6300 Acked Read 5 Done Sent Done false
19185 6301 Acked Write 7 Done Sent Done false
19186 6302 Acked Read 5 Done Sent Done false
19187 6303 Acked Read 1 Done Sent Done false
19188 6304 Acked Read 9 Done Sent Done false
19189 6305 Acked Read 10 Done Sent Done false
19190 6306 Acked Write 7 Done Sent Done false
19191 6307 Acked Write 1 Done Sent Done false
19192 6308 Acked Read 8 Done Sent Done false
19193 6309 Acked Read 6 Done Sent Done false
19194 6310 Acked Write 8 Done Sent Done false
19195 6311 Acked Write 6 Done Sent Done false
19196 6312 Acked Read 9 Done Sent Done false
19197 6313 Acked Write 1 Done Sent Done false
19198 6314 Acked Read 10 Done Sent Done false
19199 6315 Acked Read 8 Done Sent Done false
19200 6316 Acked Write 3 Done Sent Done false
19201 6317 Acked Read 7 Done Sent Done false
19202 6318 Acked Write 5 Done Sent Done false
19203 6319 Acked Write 10 Done Sent Done false
19204 6320 Acked Read 4 Done Sent Done false
19205 6321 Acked Read 5 Done Sent Done false
19206 6322 Acked Read 6 Done Sent Done false
19207 6323 Acked Read 10 Done Sent Done false
19208 6324 Acked Write 4 Done Sent Done false
19209 6325 Acked Read 10 Done Sent Done false
19210 6326 Acked Write 9 Done Sent Done false
19211 6327 Acked Read 5 Done Sent Done false
19212 6328 Acked Write 2 Done Sent Done false
19213 6329 Acked Write 8 Done Sent Done false
19214 6330 Acked Read 8 Done Sent Done false
19215 6331 Acked Flush 0 Done Sent Done false
19216 6332 Acked Read 3 Done Sent Done false
19217 6333 Acked Write 2 Done Sent Done false
19218 6334 Acked Read 2 Done Sent Done false
19219 6335 Acked Flush 0 Done Sent Done false
19220 6336 Acked Write 1 Done Sent Done false
19221 6337 Acked Write 9 Done Sent Done false
19222 6338 Acked Read 4 Done Sent Done false
19223 6339 Acked Read 2 Done Sent Done false
19224 6340 Acked Read 9 Done Sent Done false
19225 6341 Acked Read 8 Done Sent Done false
19226 6342 Acked Read 2 Done Sent Done false
19227 6343 Acked Write 2 Done Sent Done false
19228 6344 Acked Read 1 Done Sent Done false
19229 6345 Acked Write 5 Done Sent Done false
19230 6346 Acked Read 4 Done Sent Done false
19231 6347 Acked Read 4 Done Sent Done false
19232 6348 Acked Read 4 Done Sent Done false
19233 6349 Acked Read 7 Done Sent Done false
19234 6350 Acked Read 7 Done Sent Done false
19235 6351 Acked Flush 0 Done Sent Done false
19236 6352 Acked Flush 0 Done Sent Done false
19237 6353 Acked Read 10 Done Sent Done false
19238 6354 Acked Read 6 Done Sent Done false
19239 6355 Acked Read 10 Done Sent Done false
19240 6356 Acked Read 2 Done Sent Done false
19241 6357 Acked Read 10 Done Sent Done false
19242 6358 Acked Write 9 Done Sent Done false
19243 6359 Acked Write 7 Done Sent Done false
19244 6360 Acked Read 10 Done Sent Done false
19245 6361 Acked Read 3 Done Sent Done false
19246 6362 Acked Read 10 Done Sent Done false
19247 6363 Acked Read 3 Done Sent Done false
19248 6364 Acked Read 8 Done Sent Done false
19249 6365 Acked Read 5 Done Sent Done false
19250 6366 Acked Read 2 Done Sent Done false
19251 6367 Acked Write 2 Done Sent Done false
19252 6368 Acked Write 8 Done Sent Done false
19253 6369 Acked Write 9 Done Sent Done false
19254 6370 Acked Read 4 Done Sent Done false
19255 6371 Acked Read 10 Done Sent Done false
19256 6372 Acked Read 2 Done Sent Done false
19257 6373 Acked Write 7 Done Sent Done false
19258 6374 Acked Read 10 Done Sent Done false
19259 6375 Acked Write 7 Done Sent Done false
19260 6376 Acked Read 4 Done Sent Done false
19261 6377 Acked Read 9 Done Sent Done false
19262 6378 Acked Write 4 Done Sent Done false
19263 6379 Acked Write 7 Done Sent Done false
19264 6380 Acked Read 9 Done Sent Done false
19265 6381 Acked Read 1 Done Sent Done false
19266 6382 Acked Read 9 Done Sent Done false
19267 6383 Acked Read 8 Done Sent Done false
19268 6384 Acked Read 2 Done Sent Done false
19269 6385 Acked Read 7 Done Sent Done false
19270 6386 Acked Read 4 Done Sent Done false
19271 6387 Acked Write 3 Done Sent Done false
19272 6388 Acked Read 6 Done Sent Done false
19273 6389 Acked Read 1 Done Sent Done false
19274 6390 Acked Write 8 Done Sent Done false
19275 6391 Acked Read 4 Done Sent Done false
19276 6392 Acked Write 6 Done Sent Done false
19277 6393 Acked Read 5 Done Sent Done false
19278 6394 Acked Read 1 Done Sent Done false
19279 6395 Acked Write 9 Done Sent Done false
19280 6396 Acked Write 2 Done Sent Done false
19281 6397 Acked Read 6 Done Sent Done false
19282 6398 Acked Read 3 Done Sent Done false
19283 6399 Acked Read 9 Done Sent Done false
19284 6400 Acked Write 1 Done Sent Done false
19285 6401 Acked Read 3 Done Sent Done false
19286 6402 Acked Read 3 Done Sent Done false
19287 6403 Acked Read 8 Done Sent Done false
19288 6404 Acked Flush 0 Done Sent Done false
19289 6405 Acked Write 3 Done Sent Done false
19290 6406 Acked Write 1 Done Sent Done false
19291 6407 Acked Write 1 Done Sent Done false
19292 6408 Acked Read 7 Done Sent Done false
19293 6409 Acked Read 10 Done Sent Done false
19294 6410 Acked Read 4 Done Sent Done false
19295 6411 Acked Read 7 Done Sent Done false
19296 6412 Acked Read 6 Done Sent Done false
19297 6413 Acked Write 8 Done Sent Done false
19298 6414 Acked Write 10 Done Sent Done false
19299 6415 Acked Flush 0 Done Sent Done false
19300 6416 Acked Read 9 Done Sent Done false
19301 6417 Acked Write 2 Done Sent Done false
19302 6418 Acked Read 8 Done Sent Done false
19303 6419 Acked Flush 0 Done Sent Done false
19304 6420 Acked Write 4 Done Sent Done false
19305 6421 Acked Read 10 Done Sent Done false
19306 6422 Acked Read 1 Done Sent Done false
19307 6423 Acked Read 5 Done Sent Done false
19308 6424 Acked Read 3 Done Sent Done false
19309 6425 Acked Flush 0 Done Sent Done false
19310 6426 Acked Read 7 Done Sent Done false
19311 6427 Acked Read 3 Done Sent Done false
19312 6428 Acked Read 1 Done Sent Done false
19313 6429 Acked Write 7 Done Sent Done false
19314 6430 Acked Read 9 Done Sent Done false
19315 6431 Acked Read 10 Done Sent Done false
19316 6432 Acked Read 1 Done Sent Done false
19317 STATES DS:0 DS:1 DS:2 TOTAL
19318 Sent 0 301 0 301
19319 Done 301 0 301 602
19320 Skipped 0 0 0 0
19321 Error 0 0 0 0
19322 Last Flush: 6425 6131 6425
19323 Downstairs last five completed: 6131 6130 6129 6128 6127
19324 Upstairs last five completed: 6432 6431 6430 6429 6428
19325 CLIENT: Up:0 ds:301 act:2
19326 [8820] got start action
19327 [8820] I am going to restart
19328 Starting downstairs at port 8820
19329 Make output file at "/tmp/dsc/downstairs-8820.txt"
19330 [8820][1] reports Starting
19331 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1222)
19332 [8820][1] reports Running
19333 ----------------------------------------------------------------
19334 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19335 States: Active Offline Active
19336 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19337 6132 Acked Read 7 Done Sent Done false
19338 6133 Acked Write 10 Done Sent Done false
19339 6134 Acked Read 9 Done Sent Done false
19340 6135 Acked Read 8 Done Sent Done false
19341 6136 Acked Read 1 Done Sent Done false
19342 6137 Acked Flush 0 Done Sent Done false
19343 6138 Acked Read 5 Done Sent Done false
19344 6139 Acked Read 5 Done Sent Done false
19345 6140 Acked Write 1 Done Sent Done false
19346 6141 Acked Write 6 Done Sent Done false
19347 6142 Acked Flush 0 Done Sent Done false
19348 6143 Acked Write 5 Done Sent Done false
19349 6144 Acked Write 6 Done Sent Done false
19350 6145 Acked Read 1 Done Sent Done false
19351 6146 Acked Read 5 Done Sent Done false
19352 6147 Acked Read 10 Done Sent Done false
19353 6148 Acked Read 1 Done Sent Done false
19354 6149 Acked Read 9 Done Sent Done false
19355 6150 Acked Flush 0 Done Sent Done false
19356 6151 Acked Write 2 Done Sent Done false
19357 6152 Acked Write 6 Done Sent Done false
19358 6153 Acked Write 6 Done Sent Done false
19359 6154 Acked Write 1 Done Sent Done false
19360 6155 Acked Write 2 Done Sent Done false
19361 6156 Acked Flush 0 Done Sent Done false
19362 6157 Acked Read 8 Done Sent Done false
19363 6158 Acked Write 9 Done Sent Done false
19364 6159 Acked Flush 0 Done Sent Done false
19365 6160 Acked Write 2 Done Sent Done false
19366 6161 Acked Flush 0 Done Sent Done false
19367 6162 Acked Write 10 Done Sent Done false
19368 6163 Acked Read 6 Done Sent Done false
19369 6164 Acked Flush 0 Done Sent Done false
19370 6165 Acked Write 6 Done Sent Done false
19371 6166 Acked Read 9 Done Sent Done false
19372 6167 Acked Read 2 Done Sent Done false
19373 6168 Acked Read 6 Done Sent Done false
19374 6169 Acked Flush 0 Done Sent Done false
19375 6170 Acked Read 3 Done Sent Done false
19376 6171 Acked Read 1 Done Sent Done false
19377 6172 Acked Write 6 Done Sent Done false
19378 6173 Acked Write 10 Done Sent Done false
19379 6174 Acked Read 8 Done Sent Done false
19380 6175 Acked Read 5 Done Sent Done false
19381 6176 Acked Read 5 Done Sent Done false
19382 6177 Acked Write 5 Done Sent Done false
19383 6178 Acked Write 9 Done Sent Done false
19384 6179 Acked Flush 0 Done Sent Done false
19385 6180 Acked Read 7 Done Sent Done false
19386 6181 Acked Write 9 Done Sent Done false
19387 6182 Acked Read 3 Done Sent Done false
19388 6183 Acked Write 3 Done Sent Done false
19389 6184 Acked Write 5 Done Sent Done false
19390 6185 Acked Write 2 Done Sent Done false
19391 6186 Acked Write 10 Done Sent Done false
19392 6187 Acked Write 3 Done Sent Done false
19393 6188 Acked Write 2 Done Sent Done false
19394 6189 Acked Write 4 Done Sent Done false
19395 6190 Acked Write 8 Done Sent Done false
19396 6191 Acked Write 3 Done Sent Done false
19397 6192 Acked Read 5 Done Sent Done false
19398 6193 Acked Read 2 Done Sent Done false
19399 6194 Acked Read 7 Done Sent Done false
19400 6195 Acked Read 3 Done Sent Done false
19401 6196 Acked Read 2 Done Sent Done false
19402 6197 Acked Read 4 Done Sent Done false
19403 6198 Acked Read 1 Done Sent Done false
19404 6199 Acked Read 10 Done Sent Done false
19405 6200 Acked Write 5 Done Sent Done false
19406 6201 Acked Flush 0 Done Sent Done false
19407 6202 Acked Write 3 Done Sent Done false
19408 6203 Acked Write 3 Done Sent Done false
19409 6204 Acked Write 6 Done Sent Done false
19410 6205 Acked Read 5 Done Sent Done false
19411 6206 Acked Flush 0 Done Sent Done false
19412 6207 Acked Read 4 Done Sent Done false
19413 6208 Acked Write 2 Done Sent Done false
19414 6209 Acked Read 7 Done Sent Done false
19415 6210 Acked Write 6 Done Sent Done false
19416 6211 Acked Read 3 Done Sent Done false
19417 6212 Acked Read 9 Done Sent Done false
19418 6213 Acked Read 2 Done Sent Done false
19419 6214 Acked Write 9 Done Sent Done false
19420 6215 Acked Read 4 Done Sent Done false
19421 6216 Acked Write 9 Done Sent Done false
19422 6217 Acked Write 9 Done Sent Done false
19423 6218 Acked Read 5 Done Sent Done false
19424 6219 Acked Write 8 Done Sent Done false
19425 6220 Acked Read 7 Done Sent Done false
19426 6221 Acked Flush 0 Done Sent Done false
19427 6222 Acked Write 9 Done Sent Done false
19428 6223 Acked Read 2 Done Sent Done false
19429 6224 Acked Write 2 Done Sent Done false
19430 6225 Acked Flush 0 Done Sent Done false
19431 6226 Acked Read 9 Done Sent Done false
19432 6227 Acked Read 6 Done Sent Done false
19433 6228 Acked Write 10 Done Sent Done false
19434 6229 Acked Write 3 Done Sent Done false
19435 6230 Acked Write 8 Done Sent Done false
19436 6231 Acked Write 2 Done Sent Done false
19437 6232 Acked Flush 0 Done Sent Done false
19438 6233 Acked Read 9 Done Sent Done false
19439 6234 Acked Write 10 Done Sent Done false
19440 6235 Acked Write 4 Done Sent Done false
19441 6236 Acked Read 8 Done Sent Done false
19442 6237 Acked Write 9 Done Sent Done false
19443 6238 Acked Write 1 Done Sent Done false
19444 6239 Acked Read 7 Done Sent Done false
19445 6240 Acked Read 10 Done Sent Done false
19446 6241 Acked Write 1 Done Sent Done false
19447 6242 Acked Write 9 Done Sent Done false
19448 6243 Acked Read 4 Done Sent Done false
19449 6244 Acked Read 2 Done Sent Done false
19450 6245 Acked Write 1 Done Sent Done false
19451 6246 Acked Write 6 Done Sent Done false
19452 6247 Acked Read 9 Done Sent Done false
19453 6248 Acked Read 9 Done Sent Done false
19454 6249 Acked Read 7 Done Sent Done false
19455 6250 Acked Write 2 Done Sent Done false
19456 6251 Acked Read 6 Done Sent Done false
19457 6252 Acked Write 6 Done Sent Done false
19458 6253 Acked Read 1 Done Sent Done false
19459 6254 Acked Read 10 Done Sent Done false
19460 6255 Acked Write 4 Done Sent Done false
19461 6256 Acked Write 10 Done Sent Done false
19462 6257 Acked Flush 0 Done Sent Done false
19463 6258 Acked Read 10 Done Sent Done false
19464 6259 Acked Flush 0 Done Sent Done false
19465 6260 Acked Write 6 Done Sent Done false
19466 6261 Acked Flush 0 Done Sent Done false
19467 6262 Acked Write 7 Done Sent Done false
19468 6263 Acked Read 5 Done Sent Done false
19469 6264 Acked Read 8 Done Sent Done false
19470 6265 Acked Read 6 Done Sent Done false
19471 6266 Acked Flush 0 Done Sent Done false
19472 6267 Acked Read 8 Done Sent Done false
19473 6268 Acked Read 7 Done Sent Done false
19474 6269 Acked Write 5 Done Sent Done false
19475 6270 Acked Write 6 Done Sent Done false
19476 6271 Acked Write 9 Done Sent Done false
19477 6272 Acked Read 9 Done Sent Done false
19478 6273 Acked Write 8 Done Sent Done false
19479 6274 Acked Read 5 Done Sent Done false
19480 6275 Acked Write 7 Done Sent Done false
19481 6276 Acked Read 10 Done Sent Done false
19482 6277 Acked Read 10 Done Sent Done false
19483 6278 Acked Write 5 Done Sent Done false
19484 6279 Acked Write 6 Done Sent Done false
19485 6280 Acked Read 9 Done Sent Done false
19486 6281 Acked Write 3 Done Sent Done false
19487 6282 Acked Write 8 Done Sent Done false
19488 6283 Acked Flush 0 Done Sent Done false
19489 6284 Acked Read 7 Done Sent Done false
19490 6285 Acked Write 1 Done Sent Done false
19491 6286 Acked Write 4 Done Sent Done false
19492 6287 Acked Read 7 Done Sent Done false
19493 6288 Acked Read 2 Done Sent Done false
19494 6289 Acked Write 3 Done Sent Done false
19495 6290 Acked Read 9 Done Sent Done false
19496 6291 Acked Write 7 Done Sent Done false
19497 6292 Acked Read 3 Done Sent Done false
19498 6293 Acked Write 1 Done Sent Done false
19499 6294 Acked Write 2 Done Sent Done false
19500 6295 Acked Read 4 Done Sent Done false
19501 6296 Acked Write 8 Done Sent Done false
19502 6297 Acked Write 3 Done Sent Done false
19503 6298 Acked Write 7 Done Sent Done false
19504 6299 Acked Read 1 Done Sent Done false
19505 6300 Acked Read 5 Done Sent Done false
19506 6301 Acked Write 7 Done Sent Done false
19507 6302 Acked Read 5 Done Sent Done false
19508 6303 Acked Read 1 Done Sent Done false
19509 6304 Acked Read 9 Done Sent Done false
19510 6305 Acked Read 10 Done Sent Done false
19511 6306 Acked Write 7 Done Sent Done false
19512 6307 Acked Write 1 Done Sent Done false
19513 6308 Acked Read 8 Done Sent Done false
19514 6309 Acked Read 6 Done Sent Done false
19515 6310 Acked Write 8 Done Sent Done false
19516 6311 Acked Write 6 Done Sent Done false
19517 6312 Acked Read 9 Done Sent Done false
19518 6313 Acked Write 1 Done Sent Done false
19519 6314 Acked Read 10 Done Sent Done false
19520 6315 Acked Read 8 Done Sent Done false
19521 6316 Acked Write 3 Done Sent Done false
19522 6317 Acked Read 7 Done Sent Done false
19523 6318 Acked Write 5 Done Sent Done false
19524 6319 Acked Write 10 Done Sent Done false
19525 6320 Acked Read 4 Done Sent Done false
19526 6321 Acked Read 5 Done Sent Done false
19527 6322 Acked Read 6 Done Sent Done false
19528 6323 Acked Read 10 Done Sent Done false
19529 6324 Acked Write 4 Done Sent Done false
19530 6325 Acked Read 10 Done Sent Done false
19531 6326 Acked Write 9 Done Sent Done false
19532 6327 Acked Read 5 Done Sent Done false
19533 6328 Acked Write 2 Done Sent Done false
19534 6329 Acked Write 8 Done Sent Done false
19535 6330 Acked Read 8 Done Sent Done false
19536 6331 Acked Flush 0 Done Sent Done false
19537 6332 Acked Read 3 Done Sent Done false
19538 6333 Acked Write 2 Done Sent Done false
19539 6334 Acked Read 2 Done Sent Done false
19540 6335 Acked Flush 0 Done Sent Done false
19541 6336 Acked Write 1 Done Sent Done false
19542 6337 Acked Write 9 Done Sent Done false
19543 6338 Acked Read 4 Done Sent Done false
19544 6339 Acked Read 2 Done Sent Done false
19545 6340 Acked Read 9 Done Sent Done false
19546 6341 Acked Read 8 Done Sent Done false
19547 6342 Acked Read 2 Done Sent Done false
19548 6343 Acked Write 2 Done Sent Done false
19549 6344 Acked Read 1 Done Sent Done false
19550 6345 Acked Write 5 Done Sent Done false
19551 6346 Acked Read 4 Done Sent Done false
19552 6347 Acked Read 4 Done Sent Done false
19553 6348 Acked Read 4 Done Sent Done false
19554 6349 Acked Read 7 Done Sent Done false
19555 6350 Acked Read 7 Done Sent Done false
19556 6351 Acked Flush 0 Done Sent Done false
19557 6352 Acked Flush 0 Done Sent Done false
19558 6353 Acked Read 10 Done Sent Done false
19559 6354 Acked Read 6 Done Sent Done false
19560 6355 Acked Read 10 Done Sent Done false
19561 6356 Acked Read 2 Done Sent Done false
19562 6357 Acked Read 10 Done Sent Done false
19563 6358 Acked Write 9 Done Sent Done false
19564 6359 Acked Write 7 Done Sent Done false
19565 6360 Acked Read 10 Done Sent Done false
19566 6361 Acked Read 3 Done Sent Done false
19567 6362 Acked Read 10 Done Sent Done false
19568 6363 Acked Read 3 Done Sent Done false
19569 6364 Acked Read 8 Done Sent Done false
19570 6365 Acked Read 5 Done Sent Done false
19571 6366 Acked Read 2 Done Sent Done false
19572 6367 Acked Write 2 Done Sent Done false
19573 6368 Acked Write 8 Done Sent Done false
19574 6369 Acked Write 9 Done Sent Done false
19575 6370 Acked Read 4 Done Sent Done false
19576 6371 Acked Read 10 Done Sent Done false
19577 6372 Acked Read 2 Done Sent Done false
19578 6373 Acked Write 7 Done Sent Done false
19579 6374 Acked Read 10 Done Sent Done false
19580 6375 Acked Write 7 Done Sent Done false
19581 6376 Acked Read 4 Done Sent Done false
19582 6377 Acked Read 9 Done Sent Done false
19583 6378 Acked Write 4 Done Sent Done false
19584 6379 Acked Write 7 Done Sent Done false
19585 6380 Acked Read 9 Done Sent Done false
19586 6381 Acked Read 1 Done Sent Done false
19587 6382 Acked Read 9 Done Sent Done false
19588 6383 Acked Read 8 Done Sent Done false
19589 6384 Acked Read 2 Done Sent Done false
19590 6385 Acked Read 7 Done Sent Done false
19591 6386 Acked Read 4 Done Sent Done false
19592 6387 Acked Write 3 Done Sent Done false
19593 6388 Acked Read 6 Done Sent Done false
19594 6389 Acked Read 1 Done Sent Done false
19595 6390 Acked Write 8 Done Sent Done false
19596 6391 Acked Read 4 Done Sent Done false
19597 6392 Acked Write 6 Done Sent Done false
19598 6393 Acked Read 5 Done Sent Done false
19599 6394 Acked Read 1 Done Sent Done false
19600 6395 Acked Write 9 Done Sent Done false
19601 6396 Acked Write 2 Done Sent Done false
19602 6397 Acked Read 6 Done Sent Done false
19603 6398 Acked Read 3 Done Sent Done false
19604 6399 Acked Read 9 Done Sent Done false
19605 6400 Acked Write 1 Done Sent Done false
19606 6401 Acked Read 3 Done Sent Done false
19607 6402 Acked Read 3 Done Sent Done false
19608 6403 Acked Read 8 Done Sent Done false
19609 6404 Acked Flush 0 Done Sent Done false
19610 6405 Acked Write 3 Done Sent Done false
19611 6406 Acked Write 1 Done Sent Done false
19612 6407 Acked Write 1 Done Sent Done false
19613 6408 Acked Read 7 Done Sent Done false
19614 6409 Acked Read 10 Done Sent Done false
19615 6410 Acked Read 4 Done Sent Done false
19616 6411 Acked Read 7 Done Sent Done false
19617 6412 Acked Read 6 Done Sent Done false
19618 6413 Acked Write 8 Done Sent Done false
19619 6414 Acked Write 10 Done Sent Done false
19620 6415 Acked Flush 0 Done Sent Done false
19621 6416 Acked Read 9 Done Sent Done false
19622 6417 Acked Write 2 Done Sent Done false
19623 6418 Acked Read 8 Done Sent Done false
19624 6419 Acked Flush 0 Done Sent Done false
19625 6420 Acked Write 4 Done Sent Done false
19626 6421 Acked Read 10 Done Sent Done false
19627 6422 Acked Read 1 Done Sent Done false
19628 6423 Acked Read 5 Done Sent Done false
19629 6424 Acked Read 3 Done Sent Done false
19630 6425 Acked Flush 0 Done Sent Done false
19631 6426 Acked Read 7 Done Sent Done false
19632 6427 Acked Read 3 Done Sent Done false
19633 6428 Acked Read 1 Done Sent Done false
19634 6429 Acked Write 7 Done Sent Done false
19635 6430 Acked Read 9 Done Sent Done false
19636 6431 Acked Read 10 Done Sent Done false
19637 6432 Acked Read 1 Done Sent Done false
19638 6433 Acked Flush 0 Done Sent Done false
19639 STATES DS:0 DS:1 DS:2 TOTAL
19640 Sent 0 302 0 302
19641 Done 302 0 302 604
19642 Skipped 0 0 0 0
19643 Error 0 0 0 0
19644 Last Flush: 6433 6131 6433
19645 Downstairs last five completed: 6131 6130 6129 6128 6127
19646 Upstairs last five completed: 6433 6432 6431 6430 6429
19647 CLIENT: Up:0 ds:302 act:2
196482025-01-11T06:09:32.772ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196492025-01-11T06:09:32.772ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46684) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196502025-01-11T06:09:32.773ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196512025-01-11T06:09:32.774ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196522025-01-11T06:09:32.774ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196532025-01-11T06:09:32.774ZINFOcrucible: send last flush ID to this DS: 6131 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196542025-01-11T06:09:32.774ZINFOcrucible: Replied this last flush ID: 6131 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196552025-01-11T06:09:32.774ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196562025-01-11T06:09:32.774ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196572025-01-11T06:09:32.774ZINFOcrucible: [1] client re-new 302 jobs since flush 6131 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196582025-01-11T06:09:32.775ZINFOcrucible: [1] Replayed 302 jobs since flush: 6131 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
19659 ----------------------------------------------------------------
19660 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19661 Downstairs last five completed: 6433 6432 6431 6430 6429
19662 Upstairs last five completed: 6433 6432 6431 6430 6429
19663 CLIENT: Up:0 ds:0 act:3
19664 Replay: All jobs finished
19665 Main task has work to do, go find it
19666 got dsc Stop(2)
19667 stop 2
19668 Jan 11 06:09:35.029 INFO request completed[8830] Got stop action so:false kr:false
19669 , latency_us: 366, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 70454415-68c6-42af-bd13-cee7a24429de, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
196702025-01-11T06:09:35.030ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196712025-01-11T06:09:35.030ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196722025-01-11T06:09:35.030ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196732025-01-11T06:09:35.030ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
196742025-01-11T06:09:35.030ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
19675 Jan 11 06:09:35.030 INFO request completed, latency_us: 179, response_code{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:09:35.030458959Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
19676 : 200, uri: /state/cid/2, method: GET, req_id: 1fef375d-2204-4964-bdbd-1f64eee26783, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
19677 [8830] Exited with: ExitStatus(unix_wait_status(9))
19678 [8830][2] reports Exit
19679 Jan 11 06:09:39.033 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/2, method: GET, req_id: b512446e-cb22-4a4f-bcce-318b31e86fc2, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
19680 001/300 Read block 188 len 32768
19681 002/300 Read block 966 len 36864
19682 003/300 Read block 3047 len 24576
19683 004/300 Read block 3940 len 28672
19684 005/300 Write block 1532 len 8192 data: 5 5
19685 006/300 Write block 940 len 20480 data: 6 6 7 4 4
19686 007/300 Read block 3690 len 20480
19687 008/300 Flush
19688 009/300 Write block 697 len 4096 data: 6
19689 010/300 Write block 1548 len 32768 data: 3 3 3 3 2 3 3 3
19690 011/300 Read block 2599 len 20480
19691 012/300 Read block 1110 len 16384
19692 013/300 Read block 2615 len 12288
19693 014/300 Write block 3146 len 40960 data: 4 6 6 5 5 7 6 7 7 6
19694 015/300 Read block 1806 len 12288
19695 016/300 Read block 2169 len 28672
19696 017/300 Write block 3624 len 20480 data: 3 3 2 2 3
19697 018/300 Flush
19698 019/300 Write block 1630 len 40960 data: 4 3 3 3 3 4 4 4 3 2
19699 020/300 Read block 1832 len 20480
19700 021/300 Read block 2202 len 20480
19701 022/300 Read block 4893 len 8192
19702 023/300 Read block 95 len 36864
19703 024/300 Read block 1106 len 16384
19704 025/300 Read block 1369 len 16384
19705 026/300 Flush
19706 027/300 Read block 745 len 12288
19707 028/300 Read block 1417 len 40960
19708 029/300 Read block 654 len 8192
19709 030/300 Read block 2782 len 24576
19710 031/300 Read block 3614 len 20480
19711 032/300 Write block 904 len 28672 data: 3 3 5 5 5 6 6
19712 033/300 Flush
19713 034/300 Read block 449 len 36864
19714 035/300 Write block 2580 len 4096 data: 5
19715 036/300 Read block 3250 len 32768
19716 037/300 Write block 3299 len 12288 data: 3 3 5
19717 038/300 Read block 1898 len 12288
19718 039/300 Write block 1149 len 32768 data: 5 5 5 4 3 3 2 2
19719 040/300 Read block 2275 len 36864
19720 041/300 Write block 1353 len 20480 data: 4 4 5 6 6
19721 042/300 Write block 3136 len 12288 data: 4 5 4
19722 043/300 Read block 1536 len 36864
19723 044/300 Read block 4267 len 8192
19724 045/300 Read block 2786 len 8192
19725 046/300 Write block 1144 len 8192 data: 3 4
19726 047/300 Write block 3679 len 20480 data: 3 5 5 4 4
19727 048/300 Read block 4525 len 20480
19728 049/300 Read block 367 len 28672
19729 050/300 Write block 2166 len 12288 data: 3 2 2
19730 051/300 Read block 1523 len 12288
19731 052/300 Read block 967 len 40960
19732 053/300 Read block 4429 len 36864
19733 054/300 Write block 4607 len 12288 data: 6 6 5
19734 055/300 Flush
19735 056/300 Read block 979 len 4096
19736 057/300 Write block 4924 len 16384 data: 4 3 3 3
19737 058/300 Read block 3282 len 32768
19738 059/300 Read block 1827 len 12288
19739 060/300 Read block 1120 len 20480
19740 061/300 Write block 2223 len 20480 data: 5 4 3 4 4
19741 062/300 Read block 3861 len 28672
19742 063/300 Write block 3895 len 8192 data: 2 2
19743 064/300 Read block 315 len 40960
19744 065/300 Write block 2472 len 36864 data: 4 3 4 4 4 3 4 5 5
19745 066/300 Read block 4715 len 12288
19746 067/300 Write block 1446 len 24576 data: 5 4 4 5 5 3
19747 068/300 Read block 565 len 24576
19748 069/300 Read block 4786 len 16384
19749 070/300 Read block 1378 len 24576
19750 071/300 Write block 4715 len 8192 data: 3 3
19751 072/300 Write block 1749 len 28672 data: 7 7 6 5 5 6 5
19752 073/300 Read block 3409 len 20480
19753 074/300 Write block 734 len 32768 data: 2 2 2 3 2 2 4 4
19754 075/300 Write block 3337 len 24576 data: 4 3 2 3 3 4
19755 076/300 Read block 4805 len 32768
19756 077/300 Write block 3127 len 4096 data: 2
19757 078/300 Write block 41 len 8192 data: 9 8
19758 079/300 Write block 4251 len 8192 data: 2 2
19759 080/300 Read block 3728 len 36864
19760 081/300 Read block 405 len 16384
19761 082/300 Read block 4080 len 16384
19762 083/300 Write block 4774 len 20480 data: 7 6 5 5 4
19763 084/300 Write block 1331 len 16384 data: 6 5 4 4
19764 085/300 Read block 3318 len 16384
19765 086/300 Write block 211 len 28672 data: 5 3 4 4 4 5 5
19766 087/300 Read block 4715 len 16384
19767 088/300 Write block 2809 len 40960 data: 5 4 3 3 5 5 6 5 5 7
19768 089/300 Write block 2456 len 36864 data: 4 4 3 3 3 3 3 4 5
19769 090/300 Write block 4595 len 40960 data: 4 4 4 4 4 3 3 4 4 4
19770 091/300 Write block 1770 len 28672 data: 5 4 5 5 5 4 4
19771 092/300 Write block 4983 len 16384 data: 6 6 6 7
19772 093/300 Read block 4573 len 4096
19773 094/300 Flush
19774 095/300 Read block 3317 len 24576
19775 096/300 Flush
19776 097/300 Write block 1792 len 12288 data: 2 3 3
19777 098/300 Read block 587 len 36864
19778 099/300 Read block 2027 len 20480
19779 100/300 Read block 2254 len 28672
19780 101/300 Flush
19781 102/300 Write block 316 len 4096 data: 3
19782 103/300 Read block 4005 len 28672
19783 104/300 Read block 1917 len 8192
19784 105/300 Read block 582 len 16384
19785 106/300 Flush
19786 107/300 Flush
19787 108/300 Write block 4796 len 24576 data: 7 7 6 5 5 5
19788 109/300 Read block 505 len 28672
19789 110/300 Write block 2396 len 16384 data: 7 7 5 5
19790 111/300 Write block 750 len 36864 data: 5 6 5 5 7 5 5 5 6
19791 112/300 Write block 3276 len 20480 data: 6 6 6 6 5
19792 113/300 Flush
19793 114/300 Write block 4352 len 40960 data: 4 4 3 3 3 3 5 6 6 4
19794 115/300 Write block 780 len 4096 data: 3
19795 116/300 Write block 4686 len 36864 data: 5 5 5 6 4 4 4 4 4
19796 117/300 Read block 3221 len 4096
19797 118/300 Write block 2479 len 12288 data: 6 6 3
19798 119/300 Read block 336 len 40960
19799 120/300 Write block 4258 len 12288 data: 6 6 5
19800 121/300 Write block 2643 len 32768 data: 6 6 6 5 5 8 6 2
19801 122/300 Read block 4893 len 40960
19802 123/300 Read block 3138 len 12288
19803 124/300 Read block 66 len 4096
19804 125/300 Write block 4782 len 36864 data: 3 3 3 3 3 3 3 3 4
19805 126/300 Read block 4040 len 16384
19806 127/300 Flush
19807 128/300 Flush
19808 129/300 Read block 1634 len 36864
19809 130/300 Read block 3979 len 40960
19810 131/300 Read block 2763 len 24576
19811 132/300 Write block 4614 len 28672 data: 2 2 3 3 4 4 4
19812 133/300 Flush
19813 134/300 Write block 1755 len 36864 data: 6 5 6 6 4 4 3 3 3
19814 135/300 Write block 2695 len 24576 data: 4 3 3 3 3 3
19815 136/300 Write block 2394 len 32768 data: 7 8 8 8 6 6 5 5
19816 137/300 Read block 4390 len 12288
19817 138/300 Flush
19818 139/300 Write block 2875 len 12288 data: 3 3 3
19819 140/300 Write block 371 len 24576 data: 3 2 3 4 6 5
19820 141/300 Write block 4747 len 8192 data: 6 5
19821 142/300 Write block 1358 len 4096 data: 6
19822 143/300 Flush
19823 144/300 Write block 2191 len 28672 data: 4 5 4 3 5 5 8
19824 145/300 Read block 1800 len 36864
19825 146/300 Flush
19826 147/300 Write block 1739 len 4096 data: 4
19827 148/300 Read block 2708 len 8192
19828 149/300 Read block 2908 len 12288
19829 150/300 Read block 4051 len 28672
19830 151/300 Read block 418 len 12288
19831 152/300 Write block 288 len 4096 data: 4
19832 153/300 Write block 4085 len 32768 data: 3 4 5 6 6 5 5 5
19833 154/300 Read block 4213 len 4096
19834 155/300 Flush
19835 156/300 Read block 1555 len 16384
19836 157/300 Write block 729 len 36864 data: 6 5 4 4 4 3 3 3 4
19837 158/300 Flush
19838 159/300 Flush
19839 160/300 Write block 1490 len 32768 data: 6 6 5 4 5 5 4 7
19840 161/300 Read block 1617 len 4096
19841 162/300 Read block 1328 len 36864
19842 163/300 Write block 1097 len 40960 data: 7 7 6 6 4 3 3 4 3 3
19843 164/300 Read block 2131 len 20480
19844 165/300 Write block 2950 len 4096 data: 10
19845 166/300 Read block 2541 len 32768
19846 167/300 Read block 1188 len 28672
19847 168/300 Write block 2778 len 36864 data: 5 4 4 3 4 4 4 6 6
19848 169/300 Read block 1264 len 36864
19849 170/300 Read block 2068 len 20480
19850 171/300 Write block 598 len 32768 data: 4 4 2 2 2 2 2 4
19851 172/300 Write block 4236 len 32768 data: 4 5 5 5 5 4 5 7
19852 173/300 Write block 1467 len 4096 data: 2
19853 174/300 Write block 1618 len 28672 data: 5 5 5 7 5 5 5
19854 175/300 Read block 2772 len 28672
19855 176/300 Read block 4355 len 24576
19856 177/300 Write block 3975 len 36864 data: 5 5 4 5 6 5 4 3 3
19857 178/300 Read block 4422 len 32768
19858 179/300 Read block 891 len 28672
19859 180/300 Read block 3934 len 20480
19860 181/300 Write block 58 len 20480 data: 7 6 6 6 8
19861 182/300 Write block 1196 len 8192 data: 2 3
19862 183/300 Read block 1224 len 12288
19863 184/300 Flush
19864 185/300 Read block 1106 len 4096
19865 186/300 Read block 3003 len 16384
19866 187/300 Read block 4981 len 16384
19867 188/300 Read block 1851 len 12288
19868 189/300 Read block 1668 len 4096
19869 190/300 Read block 3555 len 4096
19870 191/300 Read block 1974 len 20480
19871 192/300 Write block 2876 len 36864 data: 4 4 3 3 4 3 3 2 3
19872 193/300 Write block 2841 len 12288 data: 2 2 2
19873 194/300 Write block 2099 len 20480 data: 9 6 6 7 6
19874 195/300 Read block 4499 len 16384
19875 196/300 Read block 4713 len 12288
19876 197/300 Read block 3124 len 8192
19877 198/300 Flush
19878 199/300 Write block 3286 len 12288 data: 3 3 3
19879 200/300 Read block 4004 len 4096
19880 201/300 Write block 85 len 12288 data: 4 2 2
19881 202/300 Write block 1845 len 16384 data: 4 7 6 6
19882 203/300 Read block 1958 len 8192
19883 204/300 Flush
19884 205/300 Write block 3141 len 4096 data: 3
19885 206/300 Write block 1843 len 36864 data: 5 5 5 8 7 7 5 6 6
19886 207/300 Write block 2208 len 40960 data: 5 4 4 4 3 4 4 5 6 5
19887 208/300 Read block 1186 len 4096
19888 209/300 Write block 4606 len 36864 data: 4 7 7 6 5 4 3 3 3
19889 210/300 Flush
19890 211/300 Flush
19891 212/300 Read block 595 len 28672
19892 213/300 Flush
19893 214/300 Write block 4645 len 20480 data: 2 2 3 3 5
19894 215/300 Write block 3755 len 8192 data: 4 4
19895 216/300 Flush
19896 217/300 Read block 2857 len 8192
19897 218/300 Read block 3601 len 12288
19898 219/300 Flush
19899 220/300 Write block 4268 len 28672 data: 7 7 6 7 7 6 4
19900 221/300 Read block 1581 len 20480
19901 222/300 Write block 1315 len 28672 data: 3 2 3 3 3 3 4
19902 223/300 Read block 4950 len 16384
19903 224/300 Write block 30 len 16384 data: 5 4 5 5
19904 225/300 Read block 4655 len 12288
19905 226/300 Flush
19906 227/300 Write block 3074 len 28672 data: 4 3 4 4 3 3 3
19907 228/300 Write block 1540 len 4096 data: 4
19908 229/300 Write block 4006 len 40960 data: 3 3 4 3 3 4 4 5 5 5
19909 230/300 Write block 3715 len 40960 data: 5 6 5 4 3 3 3 3 4 4
19910 231/300 Write block 1773 len 36864 data: 6 6 5 5 4 3 3 5 5
19911 232/300 Read block 2070 len 32768
19912 233/300 Read block 372 len 8192
19913 234/300 Read block 2077 len 8192
19914 235/300 Flush
19915 236/300 Write block 662 len 32768 data: 4 4 4 4 4 5 5 6
19916 237/300 Flush
19917 238/300 Flush
19918 239/300 Read block 3696 len 4096
19919 240/300 Write block 1750 len 8192 data: 8 7
19920 241/300 Write block 1208 len 8192 data: 5 5
19921 242/300 Read block 4 len 20480
19922 243/300 Write block 1718 len 32768 data: 3 3 4 4 5 5 6 6
19923 244/300 Read block 1681 len 4096
19924 245/300 Write block 3933 len 16384 data: 3 3 3 3
19925 246/300 Flush
19926 247/300 Write block 4580 len 4096 data: 6
19927 248/300 Read block 2361 len 28672
19928 249/300 Write block 89 len 40960 data: 3 3 3 4 4 7 8 7 7 7
19929 250/300 Read block 2445 len 16384
19930 251/300 Write block 1556 len 36864 data: 4 4 4 5 5 5 5 5 5
19931 252/300 Write block 2209 len 4096 data: 5
19932 253/300 Read block 3666 len 4096
19933 254/300 Read block 4343 len 36864
19934 255/300 Read block 3136 len 32768
19935 256/300 Write block 1176 len 20480 data: 4 5 6 6 5
19936 257/300 Read block 2395 len 36864
19937 258/300 Read block 2314 len 20480
19938 259/300 Read block 4706 len 16384
19939 260/300 Read block 2596 len 8192
19940 261/300 Read block 4926 len 40960
19941 262/300 Read block 3482 len 16384
19942 263/300 Read block 618 len 36864
19943 264/300 Read block 811 len 40960
19944 265/300 Write block 4330 len 20480 data: 2 2 2 3 3
19945 266/300 Read block 4297 len 16384
19946 267/300 Read block 2839 len 16384
19947 268/300 Flush
19948 269/300 Read block 444 len 16384
19949 270/300 Write block 2629 len 8192 data: 5 5
19950 271/300 Write block 3280 len 36864 data: 6 4 5 6 5 5 4 4 4
19951 272/300 Read block 2846 len 40960
19952 273/300 Write block 4465 len 32768 data: 10 10 10 11 8 7 5 4
19953 274/300 Flush
19954 275/300 Read block 3957 len 20480
19955 276/300 Read block 2831 len 8192
19956 277/300 Write block 3375 len 24576 data: 3 3 3 4 4 4
19957 278/300 Write block 2731 len 8192 data: 3 4
19958 279/300 Read block 4142 len 20480
19959 280/300 Read block 1571 len 32768
19960 281/300 Read block 2837 len 24576
19961 282/300 Write block 1108 len 36864 data: 3 3 3 3 4 4 5 5 4
19962 283/300 Flush
19963 284/300 Read block 2381 len 40960
19964 285/300 Write block 3366 len 36864 data: 8 7 7 7 6 6 4 3 3
19965 286/300 Write block 3942 len 8192 data: 4 4
19966 287/300 Flush
19967 288/300 Write block 1251 len 28672 data: 3 4 3 4 4 4 4
19968 289/300 Read block 1890 len 28672
19969 290/300 Read block 4267 len 4096
19970 291/300 Write block 839 len 28672 data: 5 7 7 6 6 6 6
19971 292/300 Read block 746 len 24576
19972 293/300 Read block 2247 len 20480
19973 294/300 Write block 1964 len 36864 data: 4 3 2 2 3 3 2 2 2
19974 295/300 Read block 4089 len 4096
19975 296/300 Read block 3888 len 4096
19976 297/300 Read block 1982 len 36864
19977 298/300 Read block 3936 len 36864
19978 299/300 Read block 3667 len 8192
19979 300/300 Read block 2965 len 28672
19980 301/300 Write block 2600 len 28672 data: 5 5 4 4 4 4 3
19981 Main task has work to do, go find it
19982 got dsc Start(2)
19983 start 2
19984 Jan 11 06:09:39.271 INFO request completed, latency_us: 236, response_code: 204, uri: /start/cid/2, method: POST, req_id: aa082a27-3866-48aa-9df2-574941bc528b, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
19985 Replay: started 2, returned:Ok(())
19986 ----------------------------------------------------------------
19987 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19988 States: Active Active Offline
19989 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19990 6434 Acked Read 8 Done Done Sent false
19991 6435 Acked Read 9 Done Done Sent false
19992 6436 Acked Read 6 Done Done Sent false
19993 6437 Acked Read 7 Done Done Sent false
19994 6438 Acked Write 2 Done Done Sent false
19995 6439 Acked Write 5 Done Done Sent false
19996 6440 Acked Read 5 Done Done Sent false
19997 6441 Acked Flush 0 Done Done Sent false
19998 6442 Acked Write 1 Done Done Sent false
19999 6443 Acked Write 8 Done Done Sent false
20000 6444 Acked Read 5 Done Done Sent false
20001 6445 Acked Read 4 Done Done Sent false
20002 6446 Acked Read 3 Done Done Sent false
20003 6447 Acked Write 10 Done Done Sent false
20004 6448 Acked Read 3 Done Done Sent false
20005 6449 Acked Read 7 Done Done Sent false
20006 6450 Acked Write 5 Done Done Sent false
20007 6451 Acked Flush 0 Done Done Sent false
20008 6452 Acked Write 10 Done Done Sent false
20009 6453 Acked Read 5 Done Done Sent false
20010 6454 Acked Read 5 Done Done Sent false
20011 6455 Acked Read 2 Done Done Sent false
20012 6456 Acked Read 9 Done Done Sent false
20013 6457 Acked Read 4 Done Done Sent false
20014 6458 Acked Read 4 Done Done Sent false
20015 6459 Acked Flush 0 Done Done Sent false
20016 6460 Acked Read 3 Done Done Sent false
20017 6461 Acked Read 10 Done Done Sent false
20018 6462 Acked Read 2 Done Done Sent false
20019 6463 Acked Read 6 Done Done Sent false
20020 6464 Acked Read 5 Done Done Sent false
20021 6465 Acked Write 7 Done Done Sent false
20022 6466 Acked Flush 0 Done Done Sent false
20023 6467 Acked Read 9 Done Done Sent false
20024 6468 Acked Write 1 Done Done Sent false
20025 6469 Acked Read 8 Done Done Sent false
20026 6470 Acked Write 3 Done Done Sent false
20027 6471 Acked Read 3 Done Done Sent false
20028 6472 Acked Write 8 Done Done Sent false
20029 6473 Acked Read 9 Done Done Sent false
20030 6474 Acked Write 5 Done Done Sent false
20031 6475 Acked Write 3 Done Done Sent false
20032 6476 Acked Read 9 Done Done Sent false
20033 6477 Acked Read 2 Done Done Sent false
20034 6478 Acked Read 2 Done Done Sent false
20035 6479 Acked Write 2 Done Done Sent false
20036 6480 Acked Write 5 Done Done Sent false
20037 6481 Acked Read 5 Done Done Sent false
20038 6482 Acked Read 7 Done Done Sent false
20039 6483 Acked Write 3 Done Done Sent false
20040 6484 Acked Read 3 Done Done Sent false
20041 6485 Acked Read 10 Done Done Sent false
20042 6486 Acked Read 9 Done Done Sent false
20043 6487 Acked Write 3 Done Done Sent false
20044 6488 Acked Flush 0 Done Done Sent false
20045 6489 Acked Read 1 Done Done Sent false
20046 6490 Acked Write 4 Done Done Sent false
20047 6491 Acked Read 8 Done Done Sent false
20048 6492 Acked Read 3 Done Done Sent false
20049 6493 Acked Read 5 Done Done Sent false
20050 6494 Acked Write 5 Done Done Sent false
20051 6495 Acked Read 7 Done Done Sent false
20052 6496 Acked Write 2 Done Done Sent false
20053 6497 Acked Read 10 Done Done Sent false
20054 6498 Acked Write 9 Done Done Sent false
20055 6499 Acked Read 3 Done Done Sent false
20056 6500 Acked Write 6 Done Done Sent false
20057 6501 Acked Read 6 Done Done Sent false
20058 6502 Acked Read 4 Done Done Sent false
20059 6503 Acked Read 6 Done Done Sent false
20060 6504 Acked Write 2 Done Done Sent false
20061 6505 Acked Write 7 Done Done Sent false
20062 6506 Acked Read 5 Done Done Sent false
20063 6507 Acked Write 8 Done Done Sent false
20064 6508 Acked Write 6 Done Done Sent false
20065 6509 Acked Read 8 Done Done Sent false
20066 6510 Acked Write 1 Done Done Sent false
20067 6511 Acked Write 2 Done Done Sent false
20068 6512 Acked Write 2 Done Done Sent false
20069 6513 Acked Read 9 Done Done Sent false
20070 6514 Acked Read 4 Done Done Sent false
20071 6515 Acked Read 4 Done Done Sent false
20072 6516 Acked Write 5 Done Done Sent false
20073 6517 Acked Write 4 Done Done Sent false
20074 6518 Acked Read 4 Done Done Sent false
20075 6519 Acked Write 7 Done Done Sent false
20076 6520 Acked Read 4 Done Done Sent false
20077 6521 Acked Write 10 Done Done Sent false
20078 6522 Acked Write 9 Done Done Sent false
20079 6523 Acked Write 10 Done Done Sent false
20080 6524 Acked Write 7 Done Done Sent false
20081 6525 Acked Write 4 Done Done Sent false
20082 6526 Acked Read 1 Done Done Sent false
20083 6527 Acked Flush 0 Done Done Sent false
20084 6528 Acked Read 6 Done Done Sent false
20085 6529 Acked Flush 0 Done Done Sent false
20086 6530 Acked Write 3 Done Done Sent false
20087 6531 Acked Read 9 Done Done Sent false
20088 6532 Acked Read 5 Done Done Sent false
20089 6533 Acked Read 7 Done Done Sent false
20090 6534 Acked Flush 0 Done Done Sent false
20091 6535 Acked Write 1 Done Done Sent false
20092 6536 Acked Read 7 Done Done Sent false
20093 6537 Acked Read 2 Done Done Sent false
20094 6538 Acked Read 4 Done Done Sent false
20095 6539 Acked Flush 0 Done Done Sent false
20096 6540 Acked Flush 0 Done Done Sent false
20097 6541 Acked Write 6 Done Done Sent false
20098 6542 Acked Read 7 Done Done Sent false
20099 6543 Acked Write 4 Done Done Sent false
20100 6544 Acked Write 9 Done Done Sent false
20101 6545 Acked Write 5 Done Done Sent false
20102 6546 Acked Flush 0 Done Done Sent false
20103 6547 Acked Write 10 Done Done Sent false
20104 6548 Acked Write 1 Done Done Sent false
20105 6549 Acked Write 9 Done Done Sent false
20106 6550 Acked Read 1 Done Done Sent false
20107 6551 Acked Write 3 Done Done Sent false
20108 6552 Acked Read 10 Done Done Sent false
20109 6553 Acked Write 3 Done Done Sent false
20110 6554 Acked Write 8 Done Done Sent false
20111 6555 Acked Read 10 Done Done Sent false
20112 6556 Acked Read 3 Done Done Sent false
20113 6557 Acked Read 1 Done Done Sent false
20114 6558 Acked Write 9 Done Done Sent false
20115 6559 Acked Read 4 Done Done Sent false
20116 6560 Acked Flush 0 Done Done Sent false
20117 6561 Acked Flush 0 Done Done Sent false
20118 6562 Acked Read 9 Done Done Sent false
20119 6563 Acked Read 10 Done Done Sent false
20120 6564 Acked Read 6 Done Done Sent false
20121 6565 Acked Write 7 Done Done Sent false
20122 6566 Acked Flush 0 Done Done Sent false
20123 6567 Acked Write 9 Done Done Sent false
20124 6568 Acked Write 6 Done Done Sent false
20125 6569 Acked Write 8 Done Done Sent false
20126 6570 Acked Read 3 Done Done Sent false
20127 6571 Acked Flush 0 Done Done Sent false
20128 6572 Acked Write 3 Done Done Sent false
20129 6573 Acked Write 6 Done Done Sent false
20130 6574 Acked Write 2 Done Done Sent false
20131 6575 Acked Write 1 Done Done Sent false
20132 6576 Acked Flush 0 Done Done Sent false
20133 6577 Acked Write 7 Done Done Sent false
20134 6578 Acked Read 9 Done Done Sent false
20135 6579 Acked Flush 0 Done Done Sent false
20136 6580 Acked Write 1 Done Done Sent false
20137 6581 Acked Read 2 Done Done Sent false
20138 6582 Acked Read 3 Done Done Sent false
20139 6583 Acked Read 7 Done Done Sent false
20140 6584 Acked Read 3 Done Done Sent false
20141 6585 Acked Write 1 Done Done Sent false
20142 6586 Acked Write 8 Done Done Sent false
20143 6587 Acked Read 1 Done Done Sent false
20144 6588 Acked Flush 0 Done Done Sent false
20145 6589 Acked Read 4 Done Done Sent false
20146 6590 Acked Write 9 Done Done Sent false
20147 6591 Acked Flush 0 Done Done Sent false
20148 6592 Acked Flush 0 Done Done Sent false
20149 6593 Acked Write 8 Done Done Sent false
20150 6594 Acked Read 1 Done Done Sent false
20151 6595 Acked Read 9 Done Done Sent false
20152 6596 Acked Write 10 Done Done Sent false
20153 6597 Acked Read 5 Done Done Sent false
20154 6598 Acked Write 1 Done Done Sent false
20155 6599 Acked Read 8 Done Done Sent false
20156 6600 Acked Read 7 Done Done Sent false
20157 6601 Acked Write 9 Done Done Sent false
20158 6602 Acked Read 9 Done Done Sent false
20159 6603 Acked Read 5 Done Done Sent false
20160 6604 Acked Write 8 Done Done Sent false
20161 6605 Acked Write 8 Done Done Sent false
20162 6606 Acked Write 1 Done Done Sent false
20163 6607 Acked Write 7 Done Done Sent false
20164 6608 Acked Read 7 Done Done Sent false
20165 6609 Acked Read 6 Done Done Sent false
20166 6610 Acked Write 9 Done Done Sent false
20167 6611 Acked Read 8 Done Done Sent false
20168 6612 Acked Read 7 Done Done Sent false
20169 6613 Acked Read 5 Done Done Sent false
20170 6614 Acked Write 5 Done Done Sent false
20171 6615 Acked Write 2 Done Done Sent false
20172 6616 Acked Read 3 Done Done Sent false
20173 6617 Acked Flush 0 Done Done Sent false
20174 6618 Acked Read 1 Done Done Sent false
20175 6619 Acked Read 4 Done Done Sent false
20176 6620 Acked Read 4 Done Done Sent false
20177 6621 Acked Read 3 Done Done Sent false
20178 6622 Acked Read 1 Done Done Sent false
20179 6623 Acked Read 1 Done Done Sent false
20180 6624 Acked Read 5 Done Done Sent false
20181 6625 Acked Write 9 Done Done Sent false
20182 6626 Acked Write 3 Done Done Sent false
20183 6627 Acked Write 5 Done Done Sent false
20184 6628 Acked Read 4 Done Done Sent false
20185 6629 Acked Read 3 Done Done Sent false
20186 6630 Acked Read 2 Done Done Sent false
20187 6631 Acked Flush 0 Done Done Sent false
20188 6632 Acked Write 3 Done Done Sent false
20189 6633 Acked Read 1 Done Done Sent false
20190 6634 Acked Write 3 Done Done Sent false
20191 6635 Acked Write 4 Done Done Sent false
20192 6636 Acked Read 2 Done Done Sent false
20193 6637 Acked Flush 0 Done Done Sent false
20194 6638 Acked Write 1 Done Done Sent false
20195 6639 Acked Write 9 Done Done Sent false
20196 6640 Acked Write 10 Done Done Sent false
20197 6641 Acked Read 1 Done Done Sent false
20198 6642 Acked Write 9 Done Done Sent false
20199 6643 Acked Flush 0 Done Done Sent false
20200 6644 Acked Flush 0 Done Done Sent false
20201 6645 Acked Read 7 Done Done Sent false
20202 6646 Acked Flush 0 Done Done Sent false
20203 6647 Acked Write 5 Done Done Sent false
20204 6648 Acked Write 2 Done Done Sent false
20205 6649 Acked Flush 0 Done Done Sent false
20206 6650 Acked Read 2 Done Done Sent false
20207 6651 Acked Read 3 Done Done Sent false
20208 6652 Acked Flush 0 Done Done Sent false
20209 6653 Acked Write 7 Done Done Sent false
20210 6654 Acked Read 5 Done Done Sent false
20211 6655 Acked Write 7 Done Done Sent false
20212 6656 Acked Read 4 Done Done Sent false
20213 6657 Acked Write 4 Done Done Sent false
20214 6658 Acked Read 3 Done Done Sent false
20215 6659 Acked Flush 0 Done Done Sent false
20216 6660 Acked Write 7 Done Done Sent false
20217 6661 Acked Write 1 Done Done Sent false
20218 6662 Acked Write 10 Done Done Sent false
20219 6663 Acked Write 10 Done Done Sent false
20220 6664 Acked Write 9 Done Done Sent false
20221 6665 Acked Read 8 Done Done Sent false
20222 6666 Acked Read 2 Done Done Sent false
20223 6667 Acked Read 2 Done Done Sent false
20224 6668 Acked Flush 0 Done Done Sent false
20225 6669 Acked Write 8 Done Done Sent false
20226 6670 Acked Flush 0 Done Done Sent false
20227 6671 Acked Flush 0 Done Done Sent false
20228 6672 Acked Read 1 Done Done Sent false
20229 6673 Acked Write 2 Done Done Sent false
20230 6674 Acked Write 2 Done Done Sent false
20231 6675 Acked Read 5 Done Done Sent false
20232 6676 Acked Write 8 Done Done Sent false
20233 6677 Acked Read 1 Done Done Sent false
20234 6678 Acked Write 4 Done Done Sent false
20235 6679 Acked Flush 0 Done Done Sent false
20236 6680 Acked Write 1 Done Done Sent false
20237 6681 Acked Read 7 Done Done Sent false
20238 6682 Acked Write 10 Done Done Sent false
20239 6683 Acked Read 4 Done Done Sent false
20240 6684 Acked Write 9 Done Done Sent false
20241 6685 Acked Write 1 Done Done Sent false
20242 6686 Acked Read 1 Done Done Sent false
20243 6687 Acked Read 9 Done Done Sent false
20244 6688 Acked Read 8 Done Done Sent false
20245 6689 Acked Write 5 Done Done Sent false
20246 6690 Acked Read 9 Done Done Sent false
20247 6691 Acked Read 5 Done Done Sent false
20248 6692 Acked Read 4 Done Done Sent false
20249 6693 Acked Read 2 Done Done Sent false
20250 6694 Acked Read 10 Done Done Sent false
20251 6695 Acked Read 4 Done Done Sent false
20252 6696 Acked Read 9 Done Done Sent false
20253 6697 Acked Read 10 Done Done Sent false
20254 6698 Acked Write 5 Done Done Sent false
20255 6699 Acked Read 4 Done Done Sent false
20256 6700 Acked Read 4 Done Done Sent false
20257 6701 Acked Flush 0 Done Done Sent false
20258 6702 Acked Read 4 Done Done Sent false
20259 6703 Acked Write 2 Done Done Sent false
20260 6704 Acked Write 9 Done Done Sent false
20261 6705 Acked Read 10 Done Done Sent false
20262 6706 Acked Write 8 Done Done Sent false
20263 6707 Acked Flush 0 Done Done Sent false
20264 6708 Acked Read 5 Done Done Sent false
20265 6709 Acked Read 2 Done Done Sent false
20266 6710 Acked Write 6 Done Done Sent false
20267 6711 Acked Write 2 Done Done Sent false
20268 6712 Acked Read 5 Done Done Sent false
20269 6713 Acked Read 8 Done Done Sent false
20270 6714 Acked Read 6 Done Done Sent false
20271 6715 Acked Write 9 Done Done Sent false
20272 6716 Acked Flush 0 Done Done Sent false
20273 6717 Acked Read 10 Done Done Sent false
20274 6718 Acked Write 9 Done Done Sent false
20275 6719 Acked Write 2 Done Done Sent false
20276 6720 Acked Flush 0 Done Done Sent false
20277 6721 Acked Write 7 Done Done Sent false
20278 6722 Acked Read 7 Done Done Sent false
20279 6723 Acked Read 1 Done Done Sent false
20280 6724 Acked Write 7 Done Done Sent false
20281 6725 Acked Read 6 Done Done Sent false
20282 6726 Acked Read 5 Done Done Sent false
20283 6727 Acked Write 9 Done Done Sent false
20284 6728 Acked Read 1 Done Done Sent false
20285 6729 Acked Read 1 Done Done Sent false
20286 6730 Acked Read 9 Done Done Sent false
20287 6731 Acked Read 9 Done Done Sent false
20288 6732 Acked Read 2 Done Done Sent false
20289 6733 Acked Read 7 Done Done Sent false
20290 6734 Acked Write 7 Done Done Sent false
20291 STATES DS:0 DS:1 DS:2 TOTAL
20292 Sent 0 0 301 301
20293 Done 301 301 0 602
20294 Skipped 0 0 0 0
20295 Error 0 0 0 0
20296 Last Flush: 6720 6720 6433
20297 Downstairs last five completed: 6433 6432 6431 6430 6429
20298 Upstairs last five completed: 6734 6733 6732 6731 6730
20299 CLIENT: Up:0 ds:301 act:2
20300 [8830] got start action
20301 [8830] I am going to restart
20302 Starting downstairs at port 8830
20303 Make output file at "/tmp/dsc/downstairs-8830.txt"
20304 [8830][2] reports Starting
20305 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
20306 [8830][2] reports Running
20307 ----------------------------------------------------------------
20308 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20309 States: Active Active Offline
20310 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20311 6434 Acked Read 8 Done Done Sent false
20312 6435 Acked Read 9 Done Done Sent false
20313 6436 Acked Read 6 Done Done Sent false
20314 6437 Acked Read 7 Done Done Sent false
20315 6438 Acked Write 2 Done Done Sent false
20316 6439 Acked Write 5 Done Done Sent false
20317 6440 Acked Read 5 Done Done Sent false
20318 6441 Acked Flush 0 Done Done Sent false
20319 6442 Acked Write 1 Done Done Sent false
20320 6443 Acked Write 8 Done Done Sent false
20321 6444 Acked Read 5 Done Done Sent false
20322 6445 Acked Read 4 Done Done Sent false
20323 6446 Acked Read 3 Done Done Sent false
20324 6447 Acked Write 10 Done Done Sent false
20325 6448 Acked Read 3 Done Done Sent false
20326 6449 Acked Read 7 Done Done Sent false
20327 6450 Acked Write 5 Done Done Sent false
20328 6451 Acked Flush 0 Done Done Sent false
20329 6452 Acked Write 10 Done Done Sent false
20330 6453 Acked Read 5 Done Done Sent false
20331 6454 Acked Read 5 Done Done Sent false
20332 6455 Acked Read 2 Done Done Sent false
20333 6456 Acked Read 9 Done Done Sent false
20334 6457 Acked Read 4 Done Done Sent false
20335 6458 Acked Read 4 Done Done Sent false
20336 6459 Acked Flush 0 Done Done Sent false
20337 6460 Acked Read 3 Done Done Sent false
20338 6461 Acked Read 10 Done Done Sent false
20339 6462 Acked Read 2 Done Done Sent false
20340 6463 Acked Read 6 Done Done Sent false
20341 6464 Acked Read 5 Done Done Sent false
20342 6465 Acked Write 7 Done Done Sent false
20343 6466 Acked Flush 0 Done Done Sent false
20344 6467 Acked Read 9 Done Done Sent false
20345 6468 Acked Write 1 Done Done Sent false
20346 6469 Acked Read 8 Done Done Sent false
20347 6470 Acked Write 3 Done Done Sent false
20348 6471 Acked Read 3 Done Done Sent false
20349 6472 Acked Write 8 Done Done Sent false
20350 6473 Acked Read 9 Done Done Sent false
20351 6474 Acked Write 5 Done Done Sent false
20352 6475 Acked Write 3 Done Done Sent false
20353 6476 Acked Read 9 Done Done Sent false
20354 6477 Acked Read 2 Done Done Sent false
20355 6478 Acked Read 2 Done Done Sent false
20356 6479 Acked Write 2 Done Done Sent false
20357 6480 Acked Write 5 Done Done Sent false
20358 6481 Acked Read 5 Done Done Sent false
20359 6482 Acked Read 7 Done Done Sent false
20360 6483 Acked Write 3 Done Done Sent false
20361 6484 Acked Read 3 Done Done Sent false
20362 6485 Acked Read 10 Done Done Sent false
20363 6486 Acked Read 9 Done Done Sent false
20364 6487 Acked Write 3 Done Done Sent false
20365 6488 Acked Flush 0 Done Done Sent false
20366 6489 Acked Read 1 Done Done Sent false
20367 6490 Acked Write 4 Done Done Sent false
20368 6491 Acked Read 8 Done Done Sent false
20369 6492 Acked Read 3 Done Done Sent false
20370 6493 Acked Read 5 Done Done Sent false
20371 6494 Acked Write 5 Done Done Sent false
20372 6495 Acked Read 7 Done Done Sent false
20373 6496 Acked Write 2 Done Done Sent false
20374 6497 Acked Read 10 Done Done Sent false
20375 6498 Acked Write 9 Done Done Sent false
20376 6499 Acked Read 3 Done Done Sent false
20377 6500 Acked Write 6 Done Done Sent false
20378 6501 Acked Read 6 Done Done Sent false
20379 6502 Acked Read 4 Done Done Sent false
20380 6503 Acked Read 6 Done Done Sent false
20381 6504 Acked Write 2 Done Done Sent false
20382 6505 Acked Write 7 Done Done Sent false
20383 6506 Acked Read 5 Done Done Sent false
20384 6507 Acked Write 8 Done Done Sent false
20385 6508 Acked Write 6 Done Done Sent false
20386 6509 Acked Read 8 Done Done Sent false
20387 6510 Acked Write 1 Done Done Sent false
20388 6511 Acked Write 2 Done Done Sent false
20389 6512 Acked Write 2 Done Done Sent false
20390 6513 Acked Read 9 Done Done Sent false
20391 6514 Acked Read 4 Done Done Sent false
20392 6515 Acked Read 4 Done Done Sent false
20393 6516 Acked Write 5 Done Done Sent false
20394 6517 Acked Write 4 Done Done Sent false
20395 6518 Acked Read 4 Done Done Sent false
20396 6519 Acked Write 7 Done Done Sent false
20397 6520 Acked Read 4 Done Done Sent false
20398 6521 Acked Write 10 Done Done Sent false
20399 6522 Acked Write 9 Done Done Sent false
20400 6523 Acked Write 10 Done Done Sent false
20401 6524 Acked Write 7 Done Done Sent false
20402 6525 Acked Write 4 Done Done Sent false
20403 6526 Acked Read 1 Done Done Sent false
20404 6527 Acked Flush 0 Done Done Sent false
20405 6528 Acked Read 6 Done Done Sent false
20406 6529 Acked Flush 0 Done Done Sent false
20407 6530 Acked Write 3 Done Done Sent false
20408 6531 Acked Read 9 Done Done Sent false
20409 6532 Acked Read 5 Done Done Sent false
20410 6533 Acked Read 7 Done Done Sent false
20411 6534 Acked Flush 0 Done Done Sent false
20412 6535 Acked Write 1 Done Done Sent false
20413 6536 Acked Read 7 Done Done Sent false
20414 6537 Acked Read 2 Done Done Sent false
20415 6538 Acked Read 4 Done Done Sent false
20416 6539 Acked Flush 0 Done Done Sent false
20417 6540 Acked Flush 0 Done Done Sent false
20418 6541 Acked Write 6 Done Done Sent false
20419 6542 Acked Read 7 Done Done Sent false
20420 6543 Acked Write 4 Done Done Sent false
20421 6544 Acked Write 9 Done Done Sent false
20422 6545 Acked Write 5 Done Done Sent false
20423 6546 Acked Flush 0 Done Done Sent false
20424 6547 Acked Write 10 Done Done Sent false
20425 6548 Acked Write 1 Done Done Sent false
20426 6549 Acked Write 9 Done Done Sent false
20427 6550 Acked Read 1 Done Done Sent false
20428 6551 Acked Write 3 Done Done Sent false
20429 6552 Acked Read 10 Done Done Sent false
20430 6553 Acked Write 3 Done Done Sent false
20431 6554 Acked Write 8 Done Done Sent false
20432 6555 Acked Read 10 Done Done Sent false
20433 6556 Acked Read 3 Done Done Sent false
20434 6557 Acked Read 1 Done Done Sent false
20435 6558 Acked Write 9 Done Done Sent false
20436 6559 Acked Read 4 Done Done Sent false
20437 6560 Acked Flush 0 Done Done Sent false
20438 6561 Acked Flush 0 Done Done Sent false
20439 6562 Acked Read 9 Done Done Sent false
20440 6563 Acked Read 10 Done Done Sent false
20441 6564 Acked Read 6 Done Done Sent false
20442 6565 Acked Write 7 Done Done Sent false
20443 6566 Acked Flush 0 Done Done Sent false
20444 6567 Acked Write 9 Done Done Sent false
20445 6568 Acked Write 6 Done Done Sent false
20446 6569 Acked Write 8 Done Done Sent false
20447 6570 Acked Read 3 Done Done Sent false
20448 6571 Acked Flush 0 Done Done Sent false
20449 6572 Acked Write 3 Done Done Sent false
20450 6573 Acked Write 6 Done Done Sent false
20451 6574 Acked Write 2 Done Done Sent false
20452 6575 Acked Write 1 Done Done Sent false
20453 6576 Acked Flush 0 Done Done Sent false
20454 6577 Acked Write 7 Done Done Sent false
20455 6578 Acked Read 9 Done Done Sent false
20456 6579 Acked Flush 0 Done Done Sent false
20457 6580 Acked Write 1 Done Done Sent false
20458 6581 Acked Read 2 Done Done Sent false
20459 6582 Acked Read 3 Done Done Sent false
20460 6583 Acked Read 7 Done Done Sent false
20461 6584 Acked Read 3 Done Done Sent false
20462 6585 Acked Write 1 Done Done Sent false
20463 6586 Acked Write 8 Done Done Sent false
20464 6587 Acked Read 1 Done Done Sent false
20465 6588 Acked Flush 0 Done Done Sent false
20466 6589 Acked Read 4 Done Done Sent false
20467 6590 Acked Write 9 Done Done Sent false
20468 6591 Acked Flush 0 Done Done Sent false
20469 6592 Acked Flush 0 Done Done Sent false
20470 6593 Acked Write 8 Done Done Sent false
20471 6594 Acked Read 1 Done Done Sent false
20472 6595 Acked Read 9 Done Done Sent false
20473 6596 Acked Write 10 Done Done Sent false
20474 6597 Acked Read 5 Done Done Sent false
20475 6598 Acked Write 1 Done Done Sent false
20476 6599 Acked Read 8 Done Done Sent false
20477 6600 Acked Read 7 Done Done Sent false
20478 6601 Acked Write 9 Done Done Sent false
20479 6602 Acked Read 9 Done Done Sent false
20480 6603 Acked Read 5 Done Done Sent false
20481 6604 Acked Write 8 Done Done Sent false
20482 6605 Acked Write 8 Done Done Sent false
20483 6606 Acked Write 1 Done Done Sent false
20484 6607 Acked Write 7 Done Done Sent false
20485 6608 Acked Read 7 Done Done Sent false
20486 6609 Acked Read 6 Done Done Sent false
20487 6610 Acked Write 9 Done Done Sent false
20488 6611 Acked Read 8 Done Done Sent false
20489 6612 Acked Read 7 Done Done Sent false
20490 6613 Acked Read 5 Done Done Sent false
20491 6614 Acked Write 5 Done Done Sent false
20492 6615 Acked Write 2 Done Done Sent false
20493 6616 Acked Read 3 Done Done Sent false
20494 6617 Acked Flush 0 Done Done Sent false
20495 6618 Acked Read 1 Done Done Sent false
20496 6619 Acked Read 4 Done Done Sent false
20497 6620 Acked Read 4 Done Done Sent false
20498 6621 Acked Read 3 Done Done Sent false
20499 6622 Acked Read 1 Done Done Sent false
20500 6623 Acked Read 1 Done Done Sent false
20501 6624 Acked Read 5 Done Done Sent false
20502 6625 Acked Write 9 Done Done Sent false
20503 6626 Acked Write 3 Done Done Sent false
20504 6627 Acked Write 5 Done Done Sent false
20505 6628 Acked Read 4 Done Done Sent false
20506 6629 Acked Read 3 Done Done Sent false
20507 6630 Acked Read 2 Done Done Sent false
20508 6631 Acked Flush 0 Done Done Sent false
20509 6632 Acked Write 3 Done Done Sent false
20510 6633 Acked Read 1 Done Done Sent false
20511 6634 Acked Write 3 Done Done Sent false
20512 6635 Acked Write 4 Done Done Sent false
20513 6636 Acked Read 2 Done Done Sent false
20514 6637 Acked Flush 0 Done Done Sent false
20515 6638 Acked Write 1 Done Done Sent false
20516 6639 Acked Write 9 Done Done Sent false
20517 6640 Acked Write 10 Done Done Sent false
20518 6641 Acked Read 1 Done Done Sent false
20519 6642 Acked Write 9 Done Done Sent false
20520 6643 Acked Flush 0 Done Done Sent false
20521 6644 Acked Flush 0 Done Done Sent false
20522 6645 Acked Read 7 Done Done Sent false
20523 6646 Acked Flush 0 Done Done Sent false
20524 6647 Acked Write 5 Done Done Sent false
20525 6648 Acked Write 2 Done Done Sent false
20526 6649 Acked Flush 0 Done Done Sent false
20527 6650 Acked Read 2 Done Done Sent false
20528 6651 Acked Read 3 Done Done Sent false
20529 6652 Acked Flush 0 Done Done Sent false
20530 6653 Acked Write 7 Done Done Sent false
20531 6654 Acked Read 5 Done Done Sent false
20532 6655 Acked Write 7 Done Done Sent false
20533 6656 Acked Read 4 Done Done Sent false
20534 6657 Acked Write 4 Done Done Sent false
20535 6658 Acked Read 3 Done Done Sent false
20536 6659 Acked Flush 0 Done Done Sent false
20537 6660 Acked Write 7 Done Done Sent false
20538 6661 Acked Write 1 Done Done Sent false
20539 6662 Acked Write 10 Done Done Sent false
20540 6663 Acked Write 10 Done Done Sent false
20541 6664 Acked Write 9 Done Done Sent false
20542 6665 Acked Read 8 Done Done Sent false
20543 6666 Acked Read 2 Done Done Sent false
20544 6667 Acked Read 2 Done Done Sent false
20545 6668 Acked Flush 0 Done Done Sent false
20546 6669 Acked Write 8 Done Done Sent false
20547 6670 Acked Flush 0 Done Done Sent false
20548 6671 Acked Flush 0 Done Done Sent false
20549 6672 Acked Read 1 Done Done Sent false
20550 6673 Acked Write 2 Done Done Sent false
20551 6674 Acked Write 2 Done Done Sent false
20552 6675 Acked Read 5 Done Done Sent false
20553 6676 Acked Write 8 Done Done Sent false
20554 6677 Acked Read 1 Done Done Sent false
20555 6678 Acked Write 4 Done Done Sent false
20556 6679 Acked Flush 0 Done Done Sent false
20557 6680 Acked Write 1 Done Done Sent false
20558 6681 Acked Read 7 Done Done Sent false
20559 6682 Acked Write 10 Done Done Sent false
20560 6683 Acked Read 4 Done Done Sent false
20561 6684 Acked Write 9 Done Done Sent false
20562 6685 Acked Write 1 Done Done Sent false
20563 6686 Acked Read 1 Done Done Sent false
20564 6687 Acked Read 9 Done Done Sent false
20565 6688 Acked Read 8 Done Done Sent false
20566 6689 Acked Write 5 Done Done Sent false
20567 6690 Acked Read 9 Done Done Sent false
20568 6691 Acked Read 5 Done Done Sent false
20569 6692 Acked Read 4 Done Done Sent false
20570 6693 Acked Read 2 Done Done Sent false
20571 6694 Acked Read 10 Done Done Sent false
20572 6695 Acked Read 4 Done Done Sent false
20573 6696 Acked Read 9 Done Done Sent false
20574 6697 Acked Read 10 Done Done Sent false
20575 6698 Acked Write 5 Done Done Sent false
20576 6699 Acked Read 4 Done Done Sent false
20577 6700 Acked Read 4 Done Done Sent false
20578 6701 Acked Flush 0 Done Done Sent false
20579 6702 Acked Read 4 Done Done Sent false
20580 6703 Acked Write 2 Done Done Sent false
20581 6704 Acked Write 9 Done Done Sent false
20582 6705 Acked Read 10 Done Done Sent false
20583 6706 Acked Write 8 Done Done Sent false
20584 6707 Acked Flush 0 Done Done Sent false
20585 6708 Acked Read 5 Done Done Sent false
20586 6709 Acked Read 2 Done Done Sent false
20587 6710 Acked Write 6 Done Done Sent false
20588 6711 Acked Write 2 Done Done Sent false
20589 6712 Acked Read 5 Done Done Sent false
20590 6713 Acked Read 8 Done Done Sent false
20591 6714 Acked Read 6 Done Done Sent false
20592 6715 Acked Write 9 Done Done Sent false
20593 6716 Acked Flush 0 Done Done Sent false
20594 6717 Acked Read 10 Done Done Sent false
20595 6718 Acked Write 9 Done Done Sent false
20596 6719 Acked Write 2 Done Done Sent false
20597 6720 Acked Flush 0 Done Done Sent false
20598 6721 Acked Write 7 Done Done Sent false
20599 6722 Acked Read 7 Done Done Sent false
20600 6723 Acked Read 1 Done Done Sent false
20601 6724 Acked Write 7 Done Done Sent false
20602 6725 Acked Read 6 Done Done Sent false
20603 6726 Acked Read 5 Done Done Sent false
20604 6727 Acked Write 9 Done Done Sent false
20605 6728 Acked Read 1 Done Done Sent false
20606 6729 Acked Read 1 Done Done Sent false
20607 6730 Acked Read 9 Done Done Sent false
20608 6731 Acked Read 9 Done Done Sent false
20609 6732 Acked Read 2 Done Done Sent false
20610 6733 Acked Read 7 Done Done Sent false
20611 6734 Acked Write 7 Done Done Sent false
20612 6735 Acked Flush 0 Done Done Sent false
20613 STATES DS:0 DS:1 DS:2 TOTAL
20614 Sent 0 0 302 302
20615 Done 302 302 0 604
20616 Skipped 0 0 0 0
20617 Error 0 0 0 0
20618 Last Flush: 6735 6735 6433
20619 Downstairs last five completed: 6433 6432 6431 6430 6429
20620 Upstairs last five completed: 6735 6734 6733 6732 6731
20621 CLIENT: Up:0 ds:302 act:2
206222025-01-11T06:09:45.031ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206232025-01-11T06:09:45.031ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62616) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206242025-01-11T06:09:45.032ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206252025-01-11T06:09:45.033ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206262025-01-11T06:09:45.033ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206272025-01-11T06:09:45.033ZINFOcrucible: send last flush ID to this DS: 6433 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206282025-01-11T06:09:45.033ZINFOcrucible: Replied this last flush ID: 6433 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206292025-01-11T06:09:45.033ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206302025-01-11T06:09:45.033ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206312025-01-11T06:09:45.033ZINFOcrucible: [2] client re-new 302 jobs since flush 6433 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206322025-01-11T06:09:45.033ZINFOcrucible: [2] Replayed 302 jobs since flush: 6433 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
20633 ----------------------------------------------------------------
20634 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20635 Downstairs last five completed: 6735 6734 6733 6732 6731
20636 Upstairs last five completed: 6735 6734 6733 6732 6731
20637 CLIENT: Up:0 ds:0 act:3
20638 Replay: All jobs finished
20639 Main task has work to do, go find it
20640 got dsc Stop(0)
20641 stop 0
20642 Jan 11 06:09:47.276 INFO request completed, latency_us: 390, response_code[8810] Got stop action so:false kr:false
20643 : 204, uri: /stop/cid/0, method: POST, req_id: b9cb78a5-eb15-4f11-8505-3f33575996a6, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
20644 Jan 11 06:09:47.277 INFO request completed, latency_us: 114, response_code: 200, uri: /state/cid/0, method: GET, req_id: 72084211-0a4f-473b-8f6f-7fa6304705bb, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
206452025-01-11T06:09:47.277ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206462025-01-11T06:09:47.277ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206472025-01-11T06:09:47.277ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206482025-01-11T06:09:47.277ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206492025-01-11T06:09:47.277ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
206502025-01-11T06:09:47.277ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
20651 [8810] Exited with: ExitStatus(unix_wait_status(9))
20652 [8810][0] reports Exit
20653 Jan 11 06:09:51.280 INFO request completed, latency_us: 273, response_code: 200, uri: /state/cid/0, method: GET, req_id: b9bad338-96c2-4f7a-8f82-d88c7815b58f, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
20654 001/300 Read block 955 len 28672
20655 002/300 Read block 3743 len 32768
20656 003/300 Write block 2362 len 28672 data: 2 4 4 4 4 4 4
20657 004/300 Read block 675 len 12288
20658 005/300 Read block 1833 len 12288
20659 006/300 Write block 2455 len 20480 data: 4 5 5 4 4
20660 007/300 Read block 4732 len 40960
20661 008/300 Read block 1306 len 20480
20662 009/300 Read block 4468 len 4096
20663 010/300 Write block 2482 len 24576 data: 3 3 3 2 2 2
20664 011/300 Read block 2456 len 36864
20665 012/300 Write block 994 len 16384 data: 4 4 5 5
20666 013/300 Write block 2146 len 24576 data: 4 3 3 3 4 5
20667 014/300 Read block 1512 len 8192
20668 015/300 Read block 3919 len 8192
20669 016/300 Write block 1510 len 8192 data: 3 4
20670 017/300 Read block 1468 len 4096
20671 018/300 Write block 4062 len 8192 data: 4 4
20672 019/300 Read block 1880 len 24576
20673 020/300 Read block 1986 len 16384
20674 021/300 Write block 2621 len 40960 data: 3 3 5 6 6 6 7 7 6 6
20675 022/300 Flush
20676 023/300 Flush
20677 024/300 Read block 4898 len 8192
20678 025/300 Flush
20679 026/300 Write block 3428 len 4096 data: 3
20680 027/300 Write block 915 len 20480 data: 4 4 4 3 3
20681 028/300 Write block 2718 len 12288 data: 5 5 5
20682 029/300 Flush
20683 030/300 Read block 2656 len 40960
20684 031/300 Read block 3604 len 28672
20685 032/300 Read block 1823 len 32768
20686 033/300 Write block 2445 len 4096 data: 4
20687 034/300 Write block 3728 len 40960 data: 6 5 5 6 7 7 6 5 5 4
20688 035/300 Write block 4691 len 20480 data: 5 5 5 5 4
20689 036/300 Write block 4473 len 36864 data: 3 3 4 4 4 4 4 4 4
20690 037/300 Read block 4634 len 8192
20691 038/300 Read block 2042 len 40960
20692 039/300 Read block 1334 len 28672
20693 040/300 Write block 1728 len 4096 data: 4
20694 041/300 Read block 3047 len 8192
20695 042/300 Write block 3123 len 24576 data: 3 2 2 2 3 2
20696 043/300 Read block 1768 len 32768
20697 044/300 Write block 3057 len 24576 data: 2 2 2 2 4 4
20698 045/300 Read block 4481 len 20480
20699 046/300 Read block 912 len 20480
20700 047/300 Write block 41 len 40960 data: 10 9 9 9 6 6 5 3 5 4
20701 048/300 Flush
20702 049/300 Read block 1371 len 20480
20703 050/300 Read block 3460 len 8192
20704 051/300 Read block 949 len 32768
20705 052/300 Read block 3552 len 24576
20706 053/300 Read block 1801 len 36864
20707 054/300 Write block 4616 len 36864 data: 4 4 5 5 5 6 5 4 4
20708 055/300 Read block 1675 len 24576
20709 056/300 Write block 1308 len 28672 data: 4 5 5 6 5 5 5
20710 057/300 Read block 622 len 16384
20711 058/300 Read block 152 len 36864
20712 059/300 Write block 3227 len 36864 data: 4 4 3 3 3 3 3 2 2
20713 060/300 Read block 2626 len 36864
20714 061/300 Write block 1133 len 8192 data: 5 5
20715 062/300 Read block 151 len 20480
20716 063/300 Read block 1159 len 4096
20717 064/300 Write block 3603 len 12288 data: 5 4 4
20718 065/300 Write block 2000 len 36864 data: 4 3 4 3 3 5 5 5 5
20719 066/300 Read block 2352 len 28672
20720 067/300 Flush
20721 068/300 Write block 3712 len 40960 data: 5 5 5 6 7 6 5 4 4 4
20722 069/300 Read block 4947 len 36864
20723 070/300 Read block 1535 len 12288
20724 071/300 Write block 827 len 36864 data: 5 3 4 6 5 5 4 4 5
20725 072/300 Read block 502 len 16384
20726 073/300 Write block 2126 len 12288 data: 4 5 6
20727 074/300 Write block 4085 len 24576 data: 4 5 6 7 7 6
20728 075/300 Read block 4063 len 36864
20729 076/300 Read block 546 len 8192
20730 077/300 Read block 2111 len 40960
20731 078/300 Read block 3406 len 32768
20732 079/300 Read block 3197 len 12288
20733 080/300 Read block 2122 len 4096
20734 081/300 Read block 160 len 24576
20735 082/300 Read block 4317 len 8192
20736 083/300 Write block 2721 len 16384 data: 5 5 5 5
20737 084/300 Flush
20738 085/300 Write block 1328 len 20480 data: 8 9 7 7 6
20739 086/300 Read block 3775 len 4096
20740 087/300 Write block 2769 len 8192 data: 2 3
20741 088/300 Write block 2132 len 16384 data: 7 6 5 4
20742 089/300 Write block 4134 len 32768 data: 6 6 6 6 6 5 5 6
20743 090/300 Read block 428 len 4096
20744 091/300 Write block 3031 len 16384 data: 6 6 7 6
20745 092/300 Read block 3801 len 4096
20746 093/300 Read block 3444 len 16384
20747 094/300 Read block 1620 len 28672
20748 095/300 Write block 4611 len 40960 data: 5 4 4 4 3 5 5 6 6 6
20749 096/300 Write block 948 len 40960 data: 5 6 6 6 6 7 8 6 6 3
20750 097/300 Read block 4992 len 32768
20751 098/300 Write block 3967 len 8192 data: 3 3
20752 099/300 Write block 4481 len 24576 data: 5 4 4 4 4 3
20753 100/300 Read block 4675 len 20480
20754 101/300 Write block 463 len 12288 data: 2 5 4
20755 102/300 Read block 3869 len 40960
20756 103/300 Read block 3816 len 28672
20757 104/300 Write block 2271 len 16384 data: 7 6 6 6
20758 105/300 Read block 43 len 36864
20759 106/300 Flush
20760 107/300 Read block 545 len 16384
20761 108/300 Write block 220 len 4096 data: 3
20762 109/300 Read block 93 len 16384
20763 110/300 Write block 4544 len 32768 data: 3 3 4 3 2 2 2 2
20764 111/300 Write block 909 len 40960 data: 7 7 6 6 6 4 5 5 5 4
20765 112/300 Write block 2029 len 40960 data: 4 6 5 4 2 2 2 2 2 3
20766 113/300 Read block 2700 len 12288
20767 114/300 Read block 1711 len 8192
20768 115/300 Write block 2796 len 40960 data: 4 4 5 5 5 5 4 4 6 5
20769 116/300 Read block 4899 len 28672
20770 117/300 Read block 2444 len 8192
20771 118/300 Read block 4233 len 36864
20772 119/300 Read block 1495 len 12288
20773 120/300 Read block 4398 len 32768
20774 121/300 Read block 2123 len 40960
20775 122/300 Read block 1068 len 36864
20776 123/300 Read block 1181 len 8192
20777 124/300 Write block 2373 len 24576 data: 4 4 4 6 6 6
20778 125/300 Read block 3669 len 24576
20779 126/300 Read block 336 len 4096
20780 127/300 Read block 3179 len 24576
20781 128/300 Read block 3765 len 20480
20782 129/300 Read block 2009 len 8192
20783 130/300 Read block 2896 len 4096
20784 131/300 Read block 3068 len 12288
20785 132/300 Write block 1461 len 8192 data: 2 3
20786 133/300 Read block 3418 len 40960
20787 134/300 Read block 4634 len 20480
20788 135/300 Write block 4074 len 32768 data: 11 9 9 6 7 5 5 5
20789 136/300 Read block 1789 len 40960
20790 137/300 Write block 4592 len 28672 data: 4 4 4 5 5 5 5
20791 138/300 Write block 2327 len 32768 data: 3 3 3 3 5 5 7 6
20792 139/300 Write block 375 len 40960 data: 7 6 5 5 4 4 4 4 3 3
20793 140/300 Read block 1502 len 4096
20794 141/300 Read block 2616 len 24576
20795 142/300 Flush
20796 143/300 Flush
20797 144/300 Flush
20798 145/300 Write block 496 len 12288 data: 6 8 9
20799 146/300 Write block 300 len 8192 data: 4 4
20800 147/300 Read block 4217 len 24576
20801 148/300 Write block 814 len 12288 data: 4 3 4
20802 149/300 Read block 4010 len 40960
20803 150/300 Flush
20804 151/300 Read block 82 len 4096
20805 152/300 Read block 3669 len 12288
20806 153/300 Read block 612 len 4096
20807 154/300 Flush
20808 155/300 Write block 4121 len 12288 data: 3 5 3
20809 156/300 Read block 2760 len 40960
20810 157/300 Write block 2912 len 40960 data: 3 3 4 2 3 3 4 3 2 3
20811 158/300 Read block 2408 len 36864
20812 159/300 Write block 2438 len 28672 data: 4 6 5 5 5 4 5
20813 160/300 Write block 3904 len 8192 data: 6 5
20814 161/300 Read block 4493 len 32768
20815 162/300 Read block 805 len 24576
20816 163/300 Write block 4000 len 12288 data: 5 5 4
20817 164/300 Read block 2842 len 40960
20818 165/300 Write block 2051 len 4096 data: 5
20819 166/300 Read block 1296 len 20480
20820 167/300 Read block 4834 len 16384
20821 168/300 Read block 2628 len 40960
20822 169/300 Read block 2260 len 40960
20823 170/300 Write block 3590 len 24576 data: 8 9 9 8 8 7
20824 171/300 Write block 1092 len 20480 data: 5 7 7 8 7
20825 172/300 Write block 71 len 16384 data: 4 3 3 2
20826 173/300 Read block 1488 len 28672
20827 174/300 Flush
20828 175/300 Read block 3412 len 24576
20829 176/300 Read block 2767 len 16384
20830 177/300 Write block 1937 len 20480 data: 6 5 4 5 5
20831 178/300 Read block 1182 len 12288
20832 179/300 Write block 4522 len 28672 data: 5 4 3 3 4 4 6
20833 180/300 Read block 3201 len 36864
20834 181/300 Flush
20835 182/300 Write block 568 len 36864 data: 4 3 3 2 2 2 2 3 4
20836 183/300 Write block 3104 len 24576 data: 5 4 4 4 2 2
20837 184/300 Read block 3847 len 12288
20838 185/300 Write block 3093 len 40960 data: 5 4 5 4 5 7 6 7 7 7
20839 186/300 Flush
20840 187/300 Read block 2710 len 8192
20841 188/300 Read block 1907 len 4096
20842 189/300 Read block 2823 len 24576
20843 190/300 Flush
20844 191/300 Read block 1412 len 4096
20845 192/300 Read block 1668 len 8192
20846 193/300 Read block 3086 len 24576
20847 194/300 Read block 731 len 40960
20848 195/300 Read block 713 len 8192
20849 196/300 Flush
20850 197/300 Read block 4467 len 24576
20851 198/300 Write block 317 len 8192 data: 2 3
20852 199/300 Read block 3263 len 24576
20853 200/300 Write block 1682 len 36864 data: 8 9 9 11 7 8 5 5 6
20854 201/300 Read block 1493 len 8192
20855 202/300 Read block 2796 len 8192
20856 203/300 Read block 3795 len 36864
20857 204/300 Write block 886 len 20480 data: 5 6 6 5 3
20858 205/300 Flush
20859 206/300 Flush
20860 207/300 Read block 1770 len 32768
20861 208/300 Write block 2563 len 20480 data: 5 4 4 4 4
20862 209/300 Flush
20863 210/300 Write block 2209 len 20480 data: 6 5 5 4 5
20864 211/300 Read block 2224 len 20480
20865 212/300 Write block 3013 len 16384 data: 4 6 5 4
20866 213/300 Read block 4596 len 16384
20867 214/300 Read block 2003 len 4096
20868 215/300 Read block 901 len 40960
20869 216/300 Write block 1586 len 20480 data: 2 2 2 2 2
20870 217/300 Flush
20871 218/300 Read block 1276 len 4096
20872 219/300 Flush
20873 220/300 Flush
20874 221/300 Write block 2289 len 32768 data: 5 5 6 4 3 3 3 2
20875 222/300 Write block 2335 len 36864 data: 5 4 3 3 4 4 4 4 4
20876 223/300 Write block 769 len 16384 data: 4 4 4 4
20877 224/300 Read block 2755 len 4096
20878 225/300 Read block 971 len 40960
20879 226/300 Write block 235 len 28672 data: 4 5 4 5 4 4 6
20880 227/300 Read block 1552 len 36864
20881 228/300 Read block 531 len 32768
20882 229/300 Write block 4296 len 8192 data: 6 7
20883 230/300 Read block 3823 len 36864
20884 231/300 Write block 3777 len 16384 data: 4 5 6 7
20885 232/300 Flush
20886 233/300 Write block 2296 len 8192 data: 3 3
20887 234/300 Flush
20888 235/300 Read block 2344 len 16384
20889 236/300 Read block 3138 len 4096
20890 237/300 Read block 3687 len 20480
20891 238/300 Flush
20892 239/300 Read block 3007 len 40960
20893 240/300 Read block 360 len 12288
20894 241/300 Write block 3038 len 36864 data: 7 7 7 7 6 4 3 3 3
20895 242/300 Write block 2501 len 8192 data: 3 3
20896 243/300 Flush
20897 244/300 Read block 4608 len 20480
20898 245/300 Read block 3046 len 4096
20899 246/300 Write block 4468 len 24576 data: 12 9 8 6 5 4
20900 247/300 Write block 2645 len 20480 data: 7 6 6 9 7
20901 248/300 Write block 1287 len 12288 data: 3 2 3
20902 249/300 Read block 4007 len 28672
20903 250/300 Read block 2957 len 4096
20904 251/300 Flush
20905 252/300 Read block 2777 len 20480
20906 253/300 Flush
20907 254/300 Write block 764 len 28672 data: 4 5 5 5 5 5 5
20908 255/300 Read block 1867 len 24576
20909 256/300 Write block 2907 len 8192 data: 6 5
20910 257/300 Write block 325 len 20480 data: 9 7 6 6 6
20911 258/300 Write block 3099 len 40960 data: 7 8 8 8 6 6 5 5 5 3
20912 259/300 Write block 4147 len 16384 data: 5 5 5 5
20913 260/300 Write block 1374 len 4096 data: 4
20914 261/300 Write block 223 len 32768 data: 3 3 3 3 5 4 4 4
20915 262/300 Read block 1638 len 24576
20916 263/300 Flush
20917 264/300 Read block 2872 len 12288
20918 265/300 Flush
20919 266/300 Write block 4270 len 32768 data: 7 8 8 7 5 4 3 3
20920 267/300 Write block 449 len 4096 data: 4
20921 268/300 Read block 4141 len 20480
20922 269/300 Read block 4269 len 20480
20923 270/300 Read block 1138 len 40960
20924 271/300 Read block 2343 len 40960
20925 272/300 Flush
20926 273/300 Write block 1839 len 12288 data: 3 3 3
20927 274/300 Write block 3729 len 24576 data: 6 6 7 8 8 7
20928 275/300 Read block 999 len 8192
20929 276/300 Read block 2348 len 20480
20930 277/300 Read block 4101 len 24576
20931 278/300 Write block 180 len 16384 data: 3 3 3 4
20932 279/300 Read block 2948 len 4096
20933 280/300 Write block 818 len 8192 data: 3 3
20934 281/300 Write block 1428 len 12288 data: 5 5 6
20935 282/300 Flush
20936 283/300 Read block 637 len 40960
20937 284/300 Write block 965 len 12288 data: 4 3 3
20938 285/300 Read block 1979 len 16384
20939 286/300 Write block 3065 len 16384 data: 4 4 4 4
20940 287/300 Read block 3489 len 12288
20941 288/300 Read block 3099 len 4096
20942 289/300 Read block 2060 len 4096
20943 290/300 Read block 4509 len 20480
20944 291/300 Read block 4070 len 32768
20945 292/300 Read block 220 len 12288
20946 293/300 Read block 1003 len 28672
20947 294/300 Read block 2820 len 24576
20948 295/300 Write block 4889 len 32768 data: 3 5 5 7 8 6 6 5
20949 296/300 Read block 4185 len 40960
20950 297/300 Read block 2570 len 32768
20951 298/300 Read block 3000 len 40960
20952 299/300 Write block 1032 len 16384 data: 2 3 2 2
20953 300/300 Read block 623 len 12288
20954 301/300 Read block 2773 len 20480
20955 Main task has work to do, go find it
20956 got dsc Start(0)
20957 start 0
20958 Jan 11 06:09:51.524 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/0, method: POST, req_id: ba1ca9e0-3a48-4de3-9f50-95131c31bb00, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
20959 Replay: started 0, returned:Ok(())
20960 ----------------------------------------------------------------
20961 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20962 States: Offline Active Active
20963 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20964 6736 Acked Read 7 Sent Done Done false
20965 6737 Acked Read 8 Sent Done Done false
20966 6738 Acked Write 7 Sent Done Done false
20967 6739 Acked Read 3 Sent Done Done false
20968 6740 Acked Read 3 Sent Done Done false
20969 6741 Acked Write 5 Sent Done Done false
20970 6742 Acked Read 10 Sent Done Done false
20971 6743 Acked Read 5 Sent Done Done false
20972 6744 Acked Read 1 Sent Done Done false
20973 6745 Acked Write 6 Sent Done Done false
20974 6746 Acked Read 9 Sent Done Done false
20975 6747 Acked Write 4 Sent Done Done false
20976 6748 Acked Write 6 Sent Done Done false
20977 6749 Acked Read 2 Sent Done Done false
20978 6750 Acked Read 2 Sent Done Done false
20979 6751 Acked Write 2 Sent Done Done false
20980 6752 Acked Read 1 Sent Done Done false
20981 6753 Acked Write 2 Sent Done Done false
20982 6754 Acked Read 6 Sent Done Done false
20983 6755 Acked Read 4 Sent Done Done false
20984 6756 Acked Write 10 Sent Done Done false
20985 6757 Acked Flush 0 Sent Done Done false
20986 6758 Acked Flush 0 Sent Done Done false
20987 6759 Acked Read 2 Sent Done Done false
20988 6760 Acked Flush 0 Sent Done Done false
20989 6761 Acked Write 1 Sent Done Done false
20990 6762 Acked Write 5 Sent Done Done false
20991 6763 Acked Write 3 Sent Done Done false
20992 6764 Acked Flush 0 Sent Done Done false
20993 6765 Acked Read 10 Sent Done Done false
20994 6766 Acked Read 7 Sent Done Done false
20995 6767 Acked Read 8 Sent Done Done false
20996 6768 Acked Write 1 Sent Done Done false
20997 6769 Acked Write 10 Sent Done Done false
20998 6770 Acked Write 5 Sent Done Done false
20999 6771 Acked Write 9 Sent Done Done false
21000 6772 Acked Read 2 Sent Done Done false
21001 6773 Acked Read 10 Sent Done Done false
21002 6774 Acked Read 7 Sent Done Done false
21003 6775 Acked Write 1 Sent Done Done false
21004 6776 Acked Read 2 Sent Done Done false
21005 6777 Acked Write 6 Sent Done Done false
21006 6778 Acked Read 8 Sent Done Done false
21007 6779 Acked Write 6 Sent Done Done false
21008 6780 Acked Read 5 Sent Done Done false
21009 6781 Acked Read 5 Sent Done Done false
21010 6782 Acked Write 10 Sent Done Done false
21011 6783 Acked Flush 0 Sent Done Done false
21012 6784 Acked Read 5 Sent Done Done false
21013 6785 Acked Read 2 Sent Done Done false
21014 6786 Acked Read 8 Sent Done Done false
21015 6787 Acked Read 6 Sent Done Done false
21016 6788 Acked Read 9 Sent Done Done false
21017 6789 Acked Write 9 Sent Done Done false
21018 6790 Acked Read 6 Sent Done Done false
21019 6791 Acked Write 7 Sent Done Done false
21020 6792 Acked Read 4 Sent Done Done false
21021 6793 Acked Read 9 Sent Done Done false
21022 6794 Acked Write 9 Sent Done Done false
21023 6795 Acked Read 9 Sent Done Done false
21024 6796 Acked Write 2 Sent Done Done false
21025 6797 Acked Read 5 Sent Done Done false
21026 6798 Acked Read 1 Sent Done Done false
21027 6799 Acked Write 3 Sent Done Done false
21028 6800 Acked Write 9 Sent Done Done false
21029 6801 Acked Read 7 Sent Done Done false
21030 6802 Acked Flush 0 Sent Done Done false
21031 6803 Acked Write 10 Sent Done Done false
21032 6804 Acked Read 9 Sent Done Done false
21033 6805 Acked Read 3 Sent Done Done false
21034 6806 Acked Write 9 Sent Done Done false
21035 6807 Acked Read 4 Sent Done Done false
21036 6808 Acked Write 3 Sent Done Done false
21037 6809 Acked Write 6 Sent Done Done false
21038 6810 Acked Read 9 Sent Done Done false
21039 6811 Acked Read 2 Sent Done Done false
21040 6812 Acked Read 10 Sent Done Done false
21041 6813 Acked Read 8 Sent Done Done false
21042 6814 Acked Read 3 Sent Done Done false
21043 6815 Acked Read 1 Sent Done Done false
21044 6816 Acked Read 6 Sent Done Done false
21045 6817 Acked Read 2 Sent Done Done false
21046 6818 Acked Write 4 Sent Done Done false
21047 6819 Acked Flush 0 Sent Done Done false
21048 6820 Acked Write 5 Sent Done Done false
21049 6821 Acked Read 1 Sent Done Done false
21050 6822 Acked Write 2 Sent Done Done false
21051 6823 Acked Write 4 Sent Done Done false
21052 6824 Acked Write 8 Sent Done Done false
21053 6825 Acked Read 1 Sent Done Done false
21054 6826 Acked Write 4 Sent Done Done false
21055 6827 Acked Read 1 Sent Done Done false
21056 6828 Acked Read 4 Sent Done Done false
21057 6829 Acked Read 7 Sent Done Done false
21058 6830 Acked Write 10 Sent Done Done false
21059 6831 Acked Write 10 Sent Done Done false
21060 6832 Acked Read 8 Sent Done Done false
21061 6833 Acked Write 2 Sent Done Done false
21062 6834 Acked Write 6 Sent Done Done false
21063 6835 Acked Read 5 Sent Done Done false
21064 6836 Acked Write 3 Sent Done Done false
21065 6837 Acked Read 10 Sent Done Done false
21066 6838 Acked Read 7 Sent Done Done false
21067 6839 Acked Write 4 Sent Done Done false
21068 6840 Acked Read 9 Sent Done Done false
21069 6841 Acked Flush 0 Sent Done Done false
21070 6842 Acked Read 4 Sent Done Done false
21071 6843 Acked Write 1 Sent Done Done false
21072 6844 Acked Read 4 Sent Done Done false
21073 6845 Acked Write 8 Sent Done Done false
21074 6846 Acked Write 10 Sent Done Done false
21075 6847 Acked Write 10 Sent Done Done false
21076 6848 Acked Read 3 Sent Done Done false
21077 6849 Acked Read 2 Sent Done Done false
21078 6850 Acked Write 10 Sent Done Done false
21079 6851 Acked Read 7 Sent Done Done false
21080 6852 Acked Read 2 Sent Done Done false
21081 6853 Acked Read 9 Sent Done Done false
21082 6854 Acked Read 3 Sent Done Done false
21083 6855 Acked Read 8 Sent Done Done false
21084 6856 Acked Read 10 Sent Done Done false
21085 6857 Acked Read 9 Sent Done Done false
21086 6858 Acked Read 2 Sent Done Done false
21087 6859 Acked Write 6 Sent Done Done false
21088 6860 Acked Read 6 Sent Done Done false
21089 6861 Acked Read 1 Sent Done Done false
21090 6862 Acked Read 6 Sent Done Done false
21091 6863 Acked Read 5 Sent Done Done false
21092 6864 Acked Read 2 Sent Done Done false
21093 6865 Acked Read 1 Sent Done Done false
21094 6866 Acked Read 3 Sent Done Done false
21095 6867 Acked Write 2 Sent Done Done false
21096 6868 Acked Read 10 Sent Done Done false
21097 6869 Acked Read 5 Sent Done Done false
21098 6870 Acked Write 8 Sent Done Done false
21099 6871 Acked Read 10 Sent Done Done false
21100 6872 Acked Write 7 Sent Done Done false
21101 6873 Acked Write 8 Sent Done Done false
21102 6874 Acked Write 10 Sent Done Done false
21103 6875 Acked Read 1 Sent Done Done false
21104 6876 Acked Read 6 Sent Done Done false
21105 6877 Acked Flush 0 Sent Done Done false
21106 6878 Acked Flush 0 Sent Done Done false
21107 6879 Acked Flush 0 Sent Done Done false
21108 6880 Acked Write 3 Sent Done Done false
21109 6881 Acked Write 2 Sent Done Done false
21110 6882 Acked Read 6 Sent Done Done false
21111 6883 Acked Write 3 Sent Done Done false
21112 6884 Acked Read 10 Sent Done Done false
21113 6885 Acked Flush 0 Sent Done Done false
21114 6886 Acked Read 1 Sent Done Done false
21115 6887 Acked Read 3 Sent Done Done false
21116 6888 Acked Read 1 Sent Done Done false
21117 6889 Acked Flush 0 Sent Done Done false
21118 6890 Acked Write 3 Sent Done Done false
21119 6891 Acked Read 10 Sent Done Done false
21120 6892 Acked Write 10 Sent Done Done false
21121 6893 Acked Read 9 Sent Done Done false
21122 6894 Acked Write 7 Sent Done Done false
21123 6895 Acked Write 2 Sent Done Done false
21124 6896 Acked Read 8 Sent Done Done false
21125 6897 Acked Read 6 Sent Done Done false
21126 6898 Acked Write 3 Sent Done Done false
21127 6899 Acked Read 10 Sent Done Done false
21128 6900 Acked Write 1 Sent Done Done false
21129 6901 Acked Read 5 Sent Done Done false
21130 6902 Acked Read 4 Sent Done Done false
21131 6903 Acked Read 10 Sent Done Done false
21132 6904 Acked Read 10 Sent Done Done false
21133 6905 Acked Write 6 Sent Done Done false
21134 6906 Acked Write 5 Sent Done Done false
21135 6907 Acked Write 4 Sent Done Done false
21136 6908 Acked Read 7 Sent Done Done false
21137 6909 Acked Flush 0 Sent Done Done false
21138 6910 Acked Read 6 Sent Done Done false
21139 6911 Acked Read 4 Sent Done Done false
21140 6912 Acked Write 5 Sent Done Done false
21141 6913 Acked Read 3 Sent Done Done false
21142 6914 Acked Write 7 Sent Done Done false
21143 6915 Acked Read 9 Sent Done Done false
21144 6916 Acked Flush 0 Sent Done Done false
21145 6917 Acked Write 9 Sent Done Done false
21146 6918 Acked Write 6 Sent Done Done false
21147 6919 Acked Read 3 Sent Done Done false
21148 6920 Acked Write 10 Sent Done Done false
21149 6921 Acked Flush 0 Sent Done Done false
21150 6922 Acked Read 2 Sent Done Done false
21151 6923 Acked Read 1 Sent Done Done false
21152 6924 Acked Read 6 Sent Done Done false
21153 6925 Acked Flush 0 Sent Done Done false
21154 6926 Acked Read 1 Sent Done Done false
21155 6927 Acked Read 2 Sent Done Done false
21156 6928 Acked Read 6 Sent Done Done false
21157 6929 Acked Read 10 Sent Done Done false
21158 6930 Acked Read 2 Sent Done Done false
21159 6931 Acked Flush 0 Sent Done Done false
21160 6932 Acked Read 6 Sent Done Done false
21161 6933 Acked Write 2 Sent Done Done false
21162 6934 Acked Read 6 Sent Done Done false
21163 6935 Acked Write 9 Sent Done Done false
21164 6936 Acked Read 2 Sent Done Done false
21165 6937 Acked Read 2 Sent Done Done false
21166 6938 Acked Read 9 Sent Done Done false
21167 6939 Acked Write 5 Sent Done Done false
21168 6940 Acked Flush 0 Sent Done Done false
21169 6941 Acked Flush 0 Sent Done Done false
21170 6942 Acked Read 8 Sent Done Done false
21171 6943 Acked Write 5 Sent Done Done false
21172 6944 Acked Flush 0 Sent Done Done false
21173 6945 Acked Write 5 Sent Done Done false
21174 6946 Acked Read 5 Sent Done Done false
21175 6947 Acked Write 4 Sent Done Done false
21176 6948 Acked Read 4 Sent Done Done false
21177 6949 Acked Read 1 Sent Done Done false
21178 6950 Acked Read 10 Sent Done Done false
21179 6951 Acked Write 5 Sent Done Done false
21180 6952 Acked Flush 0 Sent Done Done false
21181 6953 Acked Read 1 Sent Done Done false
21182 6954 Acked Flush 0 Sent Done Done false
21183 6955 Acked Flush 0 Sent Done Done false
21184 6956 Acked Write 8 Sent Done Done false
21185 6957 Acked Write 9 Sent Done Done false
21186 6958 Acked Write 4 Sent Done Done false
21187 6959 Acked Read 1 Sent Done Done false
21188 6960 Acked Read 10 Sent Done Done false
21189 6961 Acked Write 7 Sent Done Done false
21190 6962 Acked Read 9 Sent Done Done false
21191 6963 Acked Read 8 Sent Done Done false
21192 6964 Acked Write 2 Sent Done Done false
21193 6965 Acked Read 9 Sent Done Done false
21194 6966 Acked Write 4 Sent Done Done false
21195 6967 Acked Flush 0 Sent Done Done false
21196 6968 Acked Write 2 Sent Done Done false
21197 6969 Acked Flush 0 Sent Done Done false
21198 6970 Acked Read 4 Sent Done Done false
21199 6971 Acked Read 1 Sent Done Done false
21200 6972 Acked Read 5 Sent Done Done false
21201 6973 Acked Flush 0 Sent Done Done false
21202 6974 Acked Read 10 Sent Done Done false
21203 6975 Acked Read 3 Sent Done Done false
21204 6976 Acked Write 9 Sent Done Done false
21205 6977 Acked Write 2 Sent Done Done false
21206 6978 Acked Flush 0 Sent Done Done false
21207 6979 Acked Read 5 Sent Done Done false
21208 6980 Acked Read 1 Sent Done Done false
21209 6981 Acked Write 6 Sent Done Done false
21210 6982 Acked Write 5 Sent Done Done false
21211 6983 Acked Write 3 Sent Done Done false
21212 6984 Acked Read 7 Sent Done Done false
21213 6985 Acked Read 1 Sent Done Done false
21214 6986 Acked Flush 0 Sent Done Done false
21215 6987 Acked Read 5 Sent Done Done false
21216 6988 Acked Flush 0 Sent Done Done false
21217 6989 Acked Write 7 Sent Done Done false
21218 6990 Acked Read 6 Sent Done Done false
21219 6991 Acked Write 2 Sent Done Done false
21220 6992 Acked Write 5 Sent Done Done false
21221 6993 Acked Write 10 Sent Done Done false
21222 6994 Acked Write 4 Sent Done Done false
21223 6995 Acked Write 1 Sent Done Done false
21224 6996 Acked Write 8 Sent Done Done false
21225 6997 Acked Read 6 Sent Done Done false
21226 6998 Acked Flush 0 Sent Done Done false
21227 6999 Acked Read 3 Sent Done Done false
21228 7000 Acked Flush 0 Sent Done Done false
21229 7001 Acked Write 8 Sent Done Done false
21230 7002 Acked Write 1 Sent Done Done false
21231 7003 Acked Read 5 Sent Done Done false
21232 7004 Acked Read 5 Sent Done Done false
21233 7005 Acked Read 10 Sent Done Done false
21234 7006 Acked Read 10 Sent Done Done false
21235 7007 Acked Flush 0 Sent Done Done false
21236 7008 Acked Write 3 Sent Done Done false
21237 7009 Acked Write 6 Sent Done Done false
21238 7010 Acked Read 2 Sent Done Done false
21239 7011 Acked Read 5 Sent Done Done false
21240 7012 Acked Read 6 Sent Done Done false
21241 7013 Acked Write 4 Sent Done Done false
21242 7014 Acked Read 1 Sent Done Done false
21243 7015 Acked Write 2 Sent Done Done false
21244 7016 Acked Write 3 Sent Done Done false
21245 7017 Acked Flush 0 Sent Done Done false
21246 7018 Acked Read 10 Sent Done Done false
21247 7019 Acked Write 3 Sent Done Done false
21248 7020 Acked Read 4 Sent Done Done false
21249 7021 Acked Write 4 Sent Done Done false
21250 7022 Acked Read 3 Sent Done Done false
21251 7023 Acked Read 1 Sent Done Done false
21252 7024 Acked Read 1 Sent Done Done false
21253 7025 Acked Read 5 Sent Done Done false
21254 7026 Acked Read 8 Sent Done Done false
21255 7027 Acked Read 3 Sent Done Done false
21256 7028 Acked Read 7 Sent Done Done false
21257 7029 Acked Read 6 Sent Done Done false
21258 7030 Acked Write 8 Sent Done Done false
21259 7031 Acked Read 10 Sent Done Done false
21260 7032 Acked Read 8 Sent Done Done false
21261 7033 Acked Read 10 Sent Done Done false
21262 7034 Acked Write 4 Sent Done Done false
21263 7035 Acked Read 3 Sent Done Done false
21264 7036 Acked Read 5 Sent Done Done false
21265 STATES DS:0 DS:1 DS:2 TOTAL
21266 Sent 301 0 0 301
21267 Done 0 301 301 602
21268 Skipped 0 0 0 0
21269 Error 0 0 0 0
21270 Last Flush: 6735 7017 7017
21271 Downstairs last five completed: 6735 6734 6733 6732 6731
21272 Upstairs last five completed: 7036 7035 7034 7033 7032
21273 CLIENT: Up:0 ds:301 act:2
21274 [8810] got start action
21275 [8810] I am going to restart
21276 Starting downstairs at port 8810
21277 Make output file at "/tmp/dsc/downstairs-8810.txt"
21278 [8810][0] reports Starting
21279 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1224)
21280 [8810][0] reports Running
21281 ----------------------------------------------------------------
21282 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21283 States: Offline Active Active
21284 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21285 6736 Acked Read 7 Sent Done Done false
21286 6737 Acked Read 8 Sent Done Done false
21287 6738 Acked Write 7 Sent Done Done false
21288 6739 Acked Read 3 Sent Done Done false
21289 6740 Acked Read 3 Sent Done Done false
21290 6741 Acked Write 5 Sent Done Done false
21291 6742 Acked Read 10 Sent Done Done false
21292 6743 Acked Read 5 Sent Done Done false
21293 6744 Acked Read 1 Sent Done Done false
21294 6745 Acked Write 6 Sent Done Done false
21295 6746 Acked Read 9 Sent Done Done false
21296 6747 Acked Write 4 Sent Done Done false
21297 6748 Acked Write 6 Sent Done Done false
21298 6749 Acked Read 2 Sent Done Done false
21299 6750 Acked Read 2 Sent Done Done false
21300 6751 Acked Write 2 Sent Done Done false
21301 6752 Acked Read 1 Sent Done Done false
21302 6753 Acked Write 2 Sent Done Done false
21303 6754 Acked Read 6 Sent Done Done false
21304 6755 Acked Read 4 Sent Done Done false
21305 6756 Acked Write 10 Sent Done Done false
21306 6757 Acked Flush 0 Sent Done Done false
21307 6758 Acked Flush 0 Sent Done Done false
21308 6759 Acked Read 2 Sent Done Done false
21309 6760 Acked Flush 0 Sent Done Done false
21310 6761 Acked Write 1 Sent Done Done false
21311 6762 Acked Write 5 Sent Done Done false
21312 6763 Acked Write 3 Sent Done Done false
21313 6764 Acked Flush 0 Sent Done Done false
21314 6765 Acked Read 10 Sent Done Done false
21315 6766 Acked Read 7 Sent Done Done false
21316 6767 Acked Read 8 Sent Done Done false
21317 6768 Acked Write 1 Sent Done Done false
21318 6769 Acked Write 10 Sent Done Done false
21319 6770 Acked Write 5 Sent Done Done false
21320 6771 Acked Write 9 Sent Done Done false
21321 6772 Acked Read 2 Sent Done Done false
21322 6773 Acked Read 10 Sent Done Done false
21323 6774 Acked Read 7 Sent Done Done false
21324 6775 Acked Write 1 Sent Done Done false
21325 6776 Acked Read 2 Sent Done Done false
21326 6777 Acked Write 6 Sent Done Done false
21327 6778 Acked Read 8 Sent Done Done false
21328 6779 Acked Write 6 Sent Done Done false
21329 6780 Acked Read 5 Sent Done Done false
21330 6781 Acked Read 5 Sent Done Done false
21331 6782 Acked Write 10 Sent Done Done false
21332 6783 Acked Flush 0 Sent Done Done false
21333 6784 Acked Read 5 Sent Done Done false
21334 6785 Acked Read 2 Sent Done Done false
21335 6786 Acked Read 8 Sent Done Done false
21336 6787 Acked Read 6 Sent Done Done false
21337 6788 Acked Read 9 Sent Done Done false
21338 6789 Acked Write 9 Sent Done Done false
21339 6790 Acked Read 6 Sent Done Done false
21340 6791 Acked Write 7 Sent Done Done false
21341 6792 Acked Read 4 Sent Done Done false
21342 6793 Acked Read 9 Sent Done Done false
21343 6794 Acked Write 9 Sent Done Done false
21344 6795 Acked Read 9 Sent Done Done false
21345 6796 Acked Write 2 Sent Done Done false
21346 6797 Acked Read 5 Sent Done Done false
21347 6798 Acked Read 1 Sent Done Done false
21348 6799 Acked Write 3 Sent Done Done false
21349 6800 Acked Write 9 Sent Done Done false
21350 6801 Acked Read 7 Sent Done Done false
21351 6802 Acked Flush 0 Sent Done Done false
21352 6803 Acked Write 10 Sent Done Done false
21353 6804 Acked Read 9 Sent Done Done false
21354 6805 Acked Read 3 Sent Done Done false
21355 6806 Acked Write 9 Sent Done Done false
21356 6807 Acked Read 4 Sent Done Done false
21357 6808 Acked Write 3 Sent Done Done false
21358 6809 Acked Write 6 Sent Done Done false
21359 6810 Acked Read 9 Sent Done Done false
21360 6811 Acked Read 2 Sent Done Done false
21361 6812 Acked Read 10 Sent Done Done false
21362 6813 Acked Read 8 Sent Done Done false
21363 6814 Acked Read 3 Sent Done Done false
21364 6815 Acked Read 1 Sent Done Done false
21365 6816 Acked Read 6 Sent Done Done false
21366 6817 Acked Read 2 Sent Done Done false
21367 6818 Acked Write 4 Sent Done Done false
21368 6819 Acked Flush 0 Sent Done Done false
21369 6820 Acked Write 5 Sent Done Done false
21370 6821 Acked Read 1 Sent Done Done false
21371 6822 Acked Write 2 Sent Done Done false
21372 6823 Acked Write 4 Sent Done Done false
21373 6824 Acked Write 8 Sent Done Done false
21374 6825 Acked Read 1 Sent Done Done false
21375 6826 Acked Write 4 Sent Done Done false
21376 6827 Acked Read 1 Sent Done Done false
21377 6828 Acked Read 4 Sent Done Done false
21378 6829 Acked Read 7 Sent Done Done false
21379 6830 Acked Write 10 Sent Done Done false
21380 6831 Acked Write 10 Sent Done Done false
21381 6832 Acked Read 8 Sent Done Done false
21382 6833 Acked Write 2 Sent Done Done false
21383 6834 Acked Write 6 Sent Done Done false
21384 6835 Acked Read 5 Sent Done Done false
21385 6836 Acked Write 3 Sent Done Done false
21386 6837 Acked Read 10 Sent Done Done false
21387 6838 Acked Read 7 Sent Done Done false
21388 6839 Acked Write 4 Sent Done Done false
21389 6840 Acked Read 9 Sent Done Done false
21390 6841 Acked Flush 0 Sent Done Done false
21391 6842 Acked Read 4 Sent Done Done false
21392 6843 Acked Write 1 Sent Done Done false
21393 6844 Acked Read 4 Sent Done Done false
21394 6845 Acked Write 8 Sent Done Done false
21395 6846 Acked Write 10 Sent Done Done false
21396 6847 Acked Write 10 Sent Done Done false
21397 6848 Acked Read 3 Sent Done Done false
21398 6849 Acked Read 2 Sent Done Done false
21399 6850 Acked Write 10 Sent Done Done false
21400 6851 Acked Read 7 Sent Done Done false
21401 6852 Acked Read 2 Sent Done Done false
21402 6853 Acked Read 9 Sent Done Done false
21403 6854 Acked Read 3 Sent Done Done false
21404 6855 Acked Read 8 Sent Done Done false
21405 6856 Acked Read 10 Sent Done Done false
21406 6857 Acked Read 9 Sent Done Done false
21407 6858 Acked Read 2 Sent Done Done false
21408 6859 Acked Write 6 Sent Done Done false
21409 6860 Acked Read 6 Sent Done Done false
21410 6861 Acked Read 1 Sent Done Done false
21411 6862 Acked Read 6 Sent Done Done false
21412 6863 Acked Read 5 Sent Done Done false
21413 6864 Acked Read 2 Sent Done Done false
21414 6865 Acked Read 1 Sent Done Done false
21415 6866 Acked Read 3 Sent Done Done false
21416 6867 Acked Write 2 Sent Done Done false
21417 6868 Acked Read 10 Sent Done Done false
21418 6869 Acked Read 5 Sent Done Done false
21419 6870 Acked Write 8 Sent Done Done false
21420 6871 Acked Read 10 Sent Done Done false
21421 6872 Acked Write 7 Sent Done Done false
21422 6873 Acked Write 8 Sent Done Done false
21423 6874 Acked Write 10 Sent Done Done false
21424 6875 Acked Read 1 Sent Done Done false
21425 6876 Acked Read 6 Sent Done Done false
21426 6877 Acked Flush 0 Sent Done Done false
21427 6878 Acked Flush 0 Sent Done Done false
21428 6879 Acked Flush 0 Sent Done Done false
21429 6880 Acked Write 3 Sent Done Done false
21430 6881 Acked Write 2 Sent Done Done false
21431 6882 Acked Read 6 Sent Done Done false
21432 6883 Acked Write 3 Sent Done Done false
21433 6884 Acked Read 10 Sent Done Done false
21434 6885 Acked Flush 0 Sent Done Done false
21435 6886 Acked Read 1 Sent Done Done false
21436 6887 Acked Read 3 Sent Done Done false
21437 6888 Acked Read 1 Sent Done Done false
21438 6889 Acked Flush 0 Sent Done Done false
21439 6890 Acked Write 3 Sent Done Done false
21440 6891 Acked Read 10 Sent Done Done false
21441 6892 Acked Write 10 Sent Done Done false
21442 6893 Acked Read 9 Sent Done Done false
21443 6894 Acked Write 7 Sent Done Done false
21444 6895 Acked Write 2 Sent Done Done false
21445 6896 Acked Read 8 Sent Done Done false
21446 6897 Acked Read 6 Sent Done Done false
21447 6898 Acked Write 3 Sent Done Done false
21448 6899 Acked Read 10 Sent Done Done false
21449 6900 Acked Write 1 Sent Done Done false
21450 6901 Acked Read 5 Sent Done Done false
21451 6902 Acked Read 4 Sent Done Done false
21452 6903 Acked Read 10 Sent Done Done false
21453 6904 Acked Read 10 Sent Done Done false
21454 6905 Acked Write 6 Sent Done Done false
21455 6906 Acked Write 5 Sent Done Done false
21456 6907 Acked Write 4 Sent Done Done false
21457 6908 Acked Read 7 Sent Done Done false
21458 6909 Acked Flush 0 Sent Done Done false
21459 6910 Acked Read 6 Sent Done Done false
21460 6911 Acked Read 4 Sent Done Done false
21461 6912 Acked Write 5 Sent Done Done false
21462 6913 Acked Read 3 Sent Done Done false
21463 6914 Acked Write 7 Sent Done Done false
21464 6915 Acked Read 9 Sent Done Done false
21465 6916 Acked Flush 0 Sent Done Done false
21466 6917 Acked Write 9 Sent Done Done false
21467 6918 Acked Write 6 Sent Done Done false
21468 6919 Acked Read 3 Sent Done Done false
21469 6920 Acked Write 10 Sent Done Done false
21470 6921 Acked Flush 0 Sent Done Done false
21471 6922 Acked Read 2 Sent Done Done false
21472 6923 Acked Read 1 Sent Done Done false
21473 6924 Acked Read 6 Sent Done Done false
21474 6925 Acked Flush 0 Sent Done Done false
21475 6926 Acked Read 1 Sent Done Done false
21476 6927 Acked Read 2 Sent Done Done false
21477 6928 Acked Read 6 Sent Done Done false
21478 6929 Acked Read 10 Sent Done Done false
21479 6930 Acked Read 2 Sent Done Done false
21480 6931 Acked Flush 0 Sent Done Done false
21481 6932 Acked Read 6 Sent Done Done false
21482 6933 Acked Write 2 Sent Done Done false
21483 6934 Acked Read 6 Sent Done Done false
21484 6935 Acked Write 9 Sent Done Done false
21485 6936 Acked Read 2 Sent Done Done false
21486 6937 Acked Read 2 Sent Done Done false
21487 6938 Acked Read 9 Sent Done Done false
21488 6939 Acked Write 5 Sent Done Done false
21489 6940 Acked Flush 0 Sent Done Done false
21490 6941 Acked Flush 0 Sent Done Done false
21491 6942 Acked Read 8 Sent Done Done false
21492 6943 Acked Write 5 Sent Done Done false
21493 6944 Acked Flush 0 Sent Done Done false
21494 6945 Acked Write 5 Sent Done Done false
21495 6946 Acked Read 5 Sent Done Done false
21496 6947 Acked Write 4 Sent Done Done false
21497 6948 Acked Read 4 Sent Done Done false
21498 6949 Acked Read 1 Sent Done Done false
21499 6950 Acked Read 10 Sent Done Done false
21500 6951 Acked Write 5 Sent Done Done false
21501 6952 Acked Flush 0 Sent Done Done false
21502 6953 Acked Read 1 Sent Done Done false
21503 6954 Acked Flush 0 Sent Done Done false
21504 6955 Acked Flush 0 Sent Done Done false
21505 6956 Acked Write 8 Sent Done Done false
21506 6957 Acked Write 9 Sent Done Done false
21507 6958 Acked Write 4 Sent Done Done false
21508 6959 Acked Read 1 Sent Done Done false
21509 6960 Acked Read 10 Sent Done Done false
21510 6961 Acked Write 7 Sent Done Done false
21511 6962 Acked Read 9 Sent Done Done false
21512 6963 Acked Read 8 Sent Done Done false
21513 6964 Acked Write 2 Sent Done Done false
21514 6965 Acked Read 9 Sent Done Done false
21515 6966 Acked Write 4 Sent Done Done false
21516 6967 Acked Flush 0 Sent Done Done false
21517 6968 Acked Write 2 Sent Done Done false
21518 6969 Acked Flush 0 Sent Done Done false
21519 6970 Acked Read 4 Sent Done Done false
21520 6971 Acked Read 1 Sent Done Done false
21521 6972 Acked Read 5 Sent Done Done false
21522 6973 Acked Flush 0 Sent Done Done false
21523 6974 Acked Read 10 Sent Done Done false
21524 6975 Acked Read 3 Sent Done Done false
21525 6976 Acked Write 9 Sent Done Done false
21526 6977 Acked Write 2 Sent Done Done false
21527 6978 Acked Flush 0 Sent Done Done false
21528 6979 Acked Read 5 Sent Done Done false
21529 6980 Acked Read 1 Sent Done Done false
21530 6981 Acked Write 6 Sent Done Done false
21531 6982 Acked Write 5 Sent Done Done false
21532 6983 Acked Write 3 Sent Done Done false
21533 6984 Acked Read 7 Sent Done Done false
21534 6985 Acked Read 1 Sent Done Done false
21535 6986 Acked Flush 0 Sent Done Done false
21536 6987 Acked Read 5 Sent Done Done false
21537 6988 Acked Flush 0 Sent Done Done false
21538 6989 Acked Write 7 Sent Done Done false
21539 6990 Acked Read 6 Sent Done Done false
21540 6991 Acked Write 2 Sent Done Done false
21541 6992 Acked Write 5 Sent Done Done false
21542 6993 Acked Write 10 Sent Done Done false
21543 6994 Acked Write 4 Sent Done Done false
21544 6995 Acked Write 1 Sent Done Done false
21545 6996 Acked Write 8 Sent Done Done false
21546 6997 Acked Read 6 Sent Done Done false
21547 6998 Acked Flush 0 Sent Done Done false
21548 6999 Acked Read 3 Sent Done Done false
21549 7000 Acked Flush 0 Sent Done Done false
21550 7001 Acked Write 8 Sent Done Done false
21551 7002 Acked Write 1 Sent Done Done false
21552 7003 Acked Read 5 Sent Done Done false
21553 7004 Acked Read 5 Sent Done Done false
21554 7005 Acked Read 10 Sent Done Done false
21555 7006 Acked Read 10 Sent Done Done false
21556 7007 Acked Flush 0 Sent Done Done false
21557 7008 Acked Write 3 Sent Done Done false
21558 7009 Acked Write 6 Sent Done Done false
21559 7010 Acked Read 2 Sent Done Done false
21560 7011 Acked Read 5 Sent Done Done false
21561 7012 Acked Read 6 Sent Done Done false
21562 7013 Acked Write 4 Sent Done Done false
21563 7014 Acked Read 1 Sent Done Done false
21564 7015 Acked Write 2 Sent Done Done false
21565 7016 Acked Write 3 Sent Done Done false
21566 7017 Acked Flush 0 Sent Done Done false
21567 7018 Acked Read 10 Sent Done Done false
21568 7019 Acked Write 3 Sent Done Done false
21569 7020 Acked Read 4 Sent Done Done false
21570 7021 Acked Write 4 Sent Done Done false
21571 7022 Acked Read 3 Sent Done Done false
21572 7023 Acked Read 1 Sent Done Done false
21573 7024 Acked Read 1 Sent Done Done false
21574 7025 Acked Read 5 Sent Done Done false
21575 7026 Acked Read 8 Sent Done Done false
21576 7027 Acked Read 3 Sent Done Done false
21577 7028 Acked Read 7 Sent Done Done false
21578 7029 Acked Read 6 Sent Done Done false
21579 7030 Acked Write 8 Sent Done Done false
21580 7031 Acked Read 10 Sent Done Done false
21581 7032 Acked Read 8 Sent Done Done false
21582 7033 Acked Read 10 Sent Done Done false
21583 7034 Acked Write 4 Sent Done Done false
21584 7035 Acked Read 3 Sent Done Done false
21585 7036 Acked Read 5 Sent Done Done false
21586 7037 Acked Flush 0 Sent Done Done false
21587 STATES DS:0 DS:1 DS:2 TOTAL
21588 Sent 302 0 0 302
21589 Done 0 302 302 604
21590 Skipped 0 0 0 0
21591 Error 0 0 0 0
21592 Last Flush: 6735 7037 7037
21593 Downstairs last five completed: 6735 6734 6733 6732 6731
21594 Upstairs last five completed: 7037 7036 7035 7034 7033
21595 CLIENT: Up:0 ds:302 act:2
215962025-01-11T06:09:57.279ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
215972025-01-11T06:09:57.279ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35300) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
215982025-01-11T06:09:57.280ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
215992025-01-11T06:09:57.281ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
216002025-01-11T06:09:57.281ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
216012025-01-11T06:09:57.281ZINFOcrucible: send last flush ID to this DS: 6735 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
216022025-01-11T06:09:57.281ZINFOcrucible: Replied this last flush ID: 6735 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
216032025-01-11T06:09:57.281ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
216042025-01-11T06:09:57.281ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
216052025-01-11T06:09:57.281ZINFOcrucible: [0] client re-new 302 jobs since flush 6735 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
216062025-01-11T06:09:57.281ZINFOcrucible: [0] Replayed 302 jobs since flush: 6735 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
21607 ----------------------------------------------------------------
21608 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21609 Downstairs last five completed: 7037 7036 7035 7034 7033
21610 Upstairs last five completed: 7037 7036 7035 7034 7033
21611 CLIENT: Up:0 ds:0 act:3
21612 Replay: All jobs finished
21613 Main task has work to do, go find it
21614 got dsc Stop(0)
21615 stop 0
21616 Jan 11 06:09:59.529 INFO [8810] Got stop action so:false kr:false
21617 request completed, latency_us: 329, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 16798511-7b0d-46f4-90ff-3a6941590a38, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
216182025-01-11T06:09:59.530ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
21619 Jan 11 06:09:59.530 INFO request completed{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:09:59.530171054Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"0","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
21620 , latency_us: 125, response_code: 200, uri{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:09:59.530209229Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
21621 : /state/cid/0, method: GET, req_id: 099a8f82-80f2-4b0b-9b1c-c2dec7a45d1e, remote_addr{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:09:59.530251234Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"0","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
21622 : 127.0.0.1:59925, local_addr: 127.{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:09:59.530313036Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"0","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
21623 0.0.1:9998
216242025-01-11T06:09:59.530ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
21625 [8810] Exited with: ExitStatus(unix_wait_status(9))
21626 [8810][0] reports Exit
21627 Jan 11 06:10:03.531 INFO request completed, latency_us: 212, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7b10a31a-2c89-4172-8924-210e85345a7e, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
21628 001/300 Read block 3183 len 28672
21629 002/300 Read block 905 len 24576
21630 003/300 Read block 1710 len 8192
21631 004/300 Read block 2735 len 20480
21632 005/300 Read block 4511 len 16384
21633 006/300 Read block 2423 len 20480
21634 007/300 Read block 337 len 12288
21635 008/300 Read block 382 len 8192
21636 009/300 Read block 4386 len 36864
21637 010/300 Write block 540 len 32768 data: 4 3 5 4 5 5 5 4
21638 011/300 Read block 4393 len 20480
21639 012/300 Read block 4879 len 24576
21640 013/300 Read block 2312 len 8192
21641 014/300 Read block 595 len 36864
21642 015/300 Flush
21643 016/300 Read block 2711 len 24576
21644 017/300 Write block 3795 len 12288 data: 3 3 3
21645 018/300 Read block 720 len 20480
21646 019/300 Read block 4373 len 12288
21647 020/300 Write block 2784 len 20480 data: 5 7 7 6 3
21648 021/300 Read block 703 len 16384
21649 022/300 Write block 4862 len 40960 data: 3 2 2 3 4 4 3 3 4 4
21650 023/300 Flush
21651 024/300 Read block 1739 len 40960
21652 025/300 Read block 2752 len 4096
21653 026/300 Write block 123 len 36864 data: 3 3 3 3 2 3 5 6 6
21654 027/300 Flush
21655 028/300 Read block 2578 len 36864
21656 029/300 Read block 2249 len 4096
21657 030/300 Write block 2081 len 12288 data: 8 9 11
21658 031/300 Read block 4491 len 28672
21659 032/300 Flush
21660 033/300 Write block 534 len 28672 data: 3 3 3 5 5 5 5
21661 034/300 Read block 694 len 28672
21662 035/300 Write block 334 len 32768 data: 5 4 6 5 5 5 4 3
21663 036/300 Write block 2592 len 20480 data: 5 4 4 3 4
21664 037/300 Read block 3644 len 40960
21665 038/300 Read block 2281 len 8192
21666 039/300 Read block 1237 len 16384
21667 040/300 Read block 1492 len 12288
21668 041/300 Read block 3164 len 4096
21669 042/300 Write block 4680 len 28672 data: 3 3 3 5 5 5 6
21670 043/300 Flush
21671 044/300 Read block 4416 len 4096
21672 045/300 Write block 4355 len 16384 data: 4 4 4 6
21673 046/300 Read block 4930 len 32768
21674 047/300 Read block 3 len 28672
21675 048/300 Flush
21676 049/300 Read block 1952 len 20480
21677 050/300 Read block 4792 len 8192
21678 051/300 Read block 4004 len 28672
21679 052/300 Read block 4811 len 36864
21680 053/300 Write block 4442 len 20480 data: 3 3 3 3 3
21681 054/300 Read block 34 len 40960
21682 055/300 Write block 510 len 40960 data: 3 3 2 2 3 3 4 5 6 5
21683 056/300 Read block 1328 len 32768
21684 057/300 Read block 2409 len 4096
21685 058/300 Read block 126 len 12288
21686 059/300 Write block 3331 len 36864 data: 6 5 4 3 3 3 5 4 3
21687 060/300 Read block 114 len 12288
21688 061/300 Read block 1646 len 28672
21689 062/300 Write block 2232 len 24576 data: 9 7 5 5 5 5
21690 063/300 Write block 3051 len 16384 data: 3 3 2 2
21691 064/300 Write block 3514 len 40960 data: 7 6 6 4 4 3 4 4 4 4
21692 065/300 Write block 4355 len 40960 data: 5 5 5 7 7 7 5 4 3 4
21693 066/300 Write block 4355 len 40960 data: 6 6 6 8 8 8 6 5 4 5
21694 067/300 Flush
21695 068/300 Read block 2819 len 32768
21696 069/300 Write block 4436 len 4096 data: 4
21697 070/300 Read block 1320 len 8192
21698 071/300 Write block 656 len 12288 data: 5 5 4
21699 072/300 Read block 137 len 28672
21700 073/300 Write block 609 len 12288 data: 3 2 2
21701 074/300 Write block 1144 len 8192 data: 4 5
21702 075/300 Write block 1999 len 28672 data: 3 5 4 5 4 4 6
21703 076/300 Write block 4417 len 40960 data: 3 2 2 2 2 5 5 4 5 5
21704 077/300 Write block 2456 len 12288 data: 6 6 5
21705 078/300 Write block 4250 len 12288 data: 3 3 3
21706 079/300 Read block 2563 len 24576
21707 080/300 Write block 2400 len 12288 data: 6 6 4
21708 081/300 Write block 4917 len 8192 data: 3 3
21709 082/300 Write block 3562 len 40960 data: 6 5 5 5 3 5 4 6 6 4
21710 083/300 Write block 1783 len 40960 data: 5 6 6 5 5 3 2 2 2 3
21711 084/300 Read block 41 len 20480
21712 085/300 Flush
21713 086/300 Read block 3623 len 40960
21714 087/300 Read block 2920 len 12288
21715 088/300 Flush
21716 089/300 Read block 683 len 20480
21717 090/300 Write block 4781 len 24576 data: 4 4 4 4 4 4
21718 091/300 Read block 220 len 4096
21719 092/300 Flush
21720 093/300 Read block 3452 len 32768
21721 094/300 Read block 4962 len 24576
21722 095/300 Read block 2428 len 12288
21723 096/300 Read block 284 len 16384
21724 097/300 Write block 1257 len 16384 data: 5 4 4 3
21725 098/300 Read block 250 len 16384
21726 099/300 Write block 1561 len 4096 data: 6
21727 100/300 Read block 709 len 24576
21728 101/300 Read block 2513 len 12288
21729 102/300 Flush
21730 103/300 Flush
21731 104/300 Write block 93 len 12288 data: 5 8 9
21732 105/300 Read block 1183 len 4096
21733 106/300 Flush
21734 107/300 Read block 3699 len 36864
21735 108/300 Write block 2711 len 12288 data: 4 3 3
21736 109/300 Read block 3557 len 4096
21737 110/300 Write block 4174 len 24576 data: 3 4 3 2 2 2
21738 111/300 Read block 1435 len 28672
21739 112/300 Write block 2164 len 24576 data: 3 3 4 3 3 2
21740 113/300 Read block 4405 len 24576
21741 114/300 Read block 3255 len 24576
21742 115/300 Read block 1913 len 16384
21743 116/300 Read block 1499 len 28672
21744 117/300 Write block 3882 len 8192 data: 4 6
21745 118/300 Read block 570 len 4096
21746 119/300 Read block 809 len 4096
21747 120/300 Write block 3510 len 20480 data: 4 4 5 7 8
21748 121/300 Read block 3368 len 4096
21749 122/300 Read block 2800 len 8192
21750 123/300 Read block 2057 len 20480
21751 124/300 Read block 263 len 36864
21752 125/300 Write block 1521 len 40960 data: 4 4 5 6 6 6 6 5 4 4
21753 126/300 Write block 2178 len 28672 data: 4 4 4 4 5 5 6
21754 127/300 Write block 735 len 24576 data: 4 4 5 3 3 5
21755 128/300 Read block 3363 len 12288
21756 129/300 Write block 3225 len 32768 data: 3 2 5 5 4 4 4 4
21757 130/300 Read block 4074 len 12288
21758 131/300 Write block 4045 len 40960 data: 9 9 6 6 5 5 5 6 4 4
21759 132/300 Write block 3672 len 40960 data: 6 6 6 6 6 6 3 4 6 6
21760 133/300 Flush
21761 134/300 Read block 3633 len 20480
21762 135/300 Flush
21763 136/300 Write block 2366 len 20480 data: 5 5 5 3 4
21764 137/300 Write block 697 len 16384 data: 7 4 3 2
21765 138/300 Read block 1440 len 20480
21766 139/300 Read block 1866 len 20480
21767 140/300 Read block 4958 len 36864
21768 141/300 Flush
21769 142/300 Flush
21770 143/300 Write block 3624 len 12288 data: 4 4 3
21771 144/300 Write block 4796 len 24576 data: 8 8 7 6 6 6
21772 145/300 Read block 1791 len 12288
21773 146/300 Read block 4009 len 8192
21774 147/300 Read block 3318 len 40960
21775 148/300 Read block 4185 len 40960
21776 149/300 Write block 3693 len 12288 data: 5 3 3
21777 150/300 Write block 700 len 24576 data: 3 2 2 2 4 4
21778 151/300 Read block 2455 len 36864
21779 152/300 Read block 2660 len 28672
21780 153/300 Read block 3886 len 16384
21781 154/300 Write block 517 len 16384 data: 6 7 6 5
21782 155/300 Flush
21783 156/300 Read block 3169 len 36864
21784 157/300 Read block 2869 len 24576
21785 158/300 Read block 3002 len 4096
21786 159/300 Read block 2868 len 12288
21787 160/300 Flush
21788 161/300 Read block 2170 len 16384
21789 162/300 Read block 258 len 16384
21790 163/300 Read block 1308 len 24576
21791 164/300 Flush
21792 165/300 Read block 3335 len 28672
21793 166/300 Read block 4662 len 4096
21794 167/300 Read block 323 len 36864
21795 168/300 Read block 1575 len 8192
21796 169/300 Write block 1571 len 40960 data: 6 6 7 7 6 4 4 4 4 3
21797 170/300 Write block 417 len 16384 data: 5 4 3 3
21798 171/300 Read block 340 len 40960
21799 172/300 Read block 3527 len 16384
21800 173/300 Write block 672 len 16384 data: 5 5 5 3
21801 174/300 Write block 2031 len 4096 data: 6
21802 175/300 Flush
21803 176/300 Write block 824 len 32768 data: 5 5 5 6 4 5 7 6
21804 177/300 Write block 449 len 24576 data: 5 3 3 3 2 2
21805 178/300 Write block 2931 len 24576 data: 8 8 6 8 9 8
21806 179/300 Flush
21807 180/300 Read block 1397 len 28672
21808 181/300 Write block 331 len 40960 data: 6 5 3 6 5 7 6 6 6 5
21809 182/300 Flush
21810 183/300 Read block 3930 len 28672
21811 184/300 Read block 4374 len 4096
21812 185/300 Write block 1668 len 32768 data: 4 6 6 6 5 4 3 3
21813 186/300 Read block 4849 len 24576
21814 187/300 Read block 4896 len 4096
21815 188/300 Read block 3694 len 36864
21816 189/300 Read block 221 len 12288
21817 190/300 Read block 208 len 12288
21818 191/300 Read block 3695 len 12288
21819 192/300 Write block 192 len 40960 data: 8 7 6 6 5 5 3 3 5 4
21820 193/300 Write block 3282 len 28672 data: 6 7 6 6 5 5 5
21821 194/300 Write block 224 len 28672 data: 4 4 4 6 5 5 5
21822 195/300 Write block 2220 len 20480 data: 7 7 7 6 5
21823 196/300 Read block 538 len 28672
21824 197/300 Write block 4262 len 4096 data: 6
21825 198/300 Read block 892 len 4096
21826 199/300 Write block 3182 len 28672 data: 4 4 3 5 6 5 5
21827 200/300 Flush
21828 201/300 Read block 2501 len 8192
21829 202/300 Read block 1189 len 36864
21830 203/300 Read block 3709 len 16384
21831 204/300 Write block 1823 len 24576 data: 2 2 3 4 6 6
21832 205/300 Read block 1123 len 28672
21833 206/300 Write block 2526 len 24576 data: 7 8 8 6 5 5
21834 207/300 Read block 2573 len 8192
21835 208/300 Write block 2375 len 12288 data: 5 7 7
21836 209/300 Write block 1427 len 20480 data: 5 6 6 7 5
21837 210/300 Read block 362 len 40960
21838 211/300 Flush
21839 212/300 Read block 4335 len 20480
21840 213/300 Read block 1569 len 12288
21841 214/300 Write block 3153 len 40960 data: 8 8 7 6 5 5 5 4 5 5
21842 215/300 Write block 2905 len 40960 data: 6 6 7 6 4 3 3 4 4 5
21843 216/300 Read block 361 len 32768
21844 217/300 Write block 1679 len 16384 data: 5 6 6 9
21845 218/300 Read block 4723 len 8192
21846 219/300 Read block 3771 len 28672
21847 220/300 Flush
21848 221/300 Read block 4026 len 4096
21849 222/300 Read block 3649 len 24576
21850 223/300 Write block 2710 len 16384 data: 4 5 4 4
21851 224/300 Read block 4395 len 8192
21852 225/300 Write block 1640 len 8192 data: 3 3
21853 226/300 Read block 1468 len 20480
21854 227/300 Read block 2609 len 20480
21855 228/300 Read block 2108 len 28672
21856 229/300 Write block 1112 len 40960 data: 5 5 6 6 5 4 5 5 5 7
21857 230/300 Read block 1037 len 4096
21858 231/300 Write block 195 len 28672 data: 7 6 6 4 4 6 5
21859 232/300 Read block 3072 len 40960
21860 233/300 Write block 558 len 28672 data: 3 3 3 2 3 3 3
21861 234/300 Write block 928 len 20480 data: 2 2 2 2 2
21862 235/300 Write block 4764 len 16384 data: 4 3 4 5
21863 236/300 Read block 921 len 28672
21864 237/300 Read block 3989 len 20480
21865 238/300 Read block 3964 len 32768
21866 239/300 Flush
21867 240/300 Read block 2143 len 8192
21868 241/300 Read block 4223 len 20480
21869 242/300 Write block 276 len 4096 data: 4
21870 243/300 Read block 2578 len 24576
21871 244/300 Write block 1729 len 24576 data: 5 5 4 5 4 4
21872 245/300 Flush
21873 246/300 Read block 2239 len 4096
21874 247/300 Write block 363 len 4096 data: 4
21875 248/300 Write block 1492 len 4096 data: 6
21876 249/300 Read block 1877 len 36864
21877 250/300 Write block 1299 len 24576 data: 6 5 4 5 5 4
21878 251/300 Read block 3686 len 40960
21879 252/300 Read block 2532 len 40960
21880 253/300 Read block 2670 len 36864
21881 254/300 Flush
21882 255/300 Read block 1660 len 16384
21883 256/300 Read block 1864 len 36864
21884 257/300 Read block 4321 len 32768
21885 258/300 Write block 1365 len 12288 data: 7 7 7
21886 259/300 Flush
21887 260/300 Read block 867 len 16384
21888 261/300 Write block 1698 len 32768 data: 5 4 7 7 7 7 6 6
21889 262/300 Read block 550 len 36864
21890 263/300 Write block 3813 len 40960 data: 4 5 3 4 5 5 6 7 5 6
21891 264/300 Flush
21892 265/300 Read block 403 len 20480
21893 266/300 Write block 4792 len 12288 data: 5 5 5
21894 267/300 Write block 1524 len 40960 data: 7 7 7 7 6 5 5 5 6 6
21895 268/300 Write block 3844 len 8192 data: 3 3
21896 269/300 Read block 4804 len 12288
21897 270/300 Write block 65 len 4096 data: 6
21898 271/300 Write block 3616 len 20480 data: 4 4 3 2 2
21899 272/300 Write block 4154 len 40960 data: 3 4 3 3 3 3 5 6 6 6
21900 273/300 Write block 1714 len 40960 data: 3 4 4 3 4 4 5 5 6 6
21901 274/300 Write block 3000 len 32768 data: 5 5 5 5 7 7 7 8
21902 275/300 Read block 2770 len 4096
21903 276/300 Write block 4066 len 24576 data: 4 5 4 5 6 7
21904 277/300 Write block 588 len 36864 data: 3 3 3 4 4 4 4 5 6
21905 278/300 Write block 3792 len 20480 data: 3 4 4 4 4
21906 279/300 Flush
21907 280/300 Write block 1251 len 20480 data: 4 5 4 5 5
21908 281/300 Flush
21909 282/300 Read block 4808 len 12288
21910 283/300 Read block 3633 len 32768
21911 284/300 Write block 2294 len 40960 data: 4 4 4 4 3 6 6 8 7 5
21912 285/300 Read block 2707 len 8192
21913 286/300 Read block 4541 len 20480
21914 287/300 Write block 1172 len 4096 data: 3
21915 288/300 Read block 1508 len 16384
21916 289/300 Read block 2718 len 16384
21917 290/300 Read block 2393 len 16384
21918 291/300 Read block 681 len 8192
21919 292/300 Write block 1493 len 28672 data: 5 6 6 5 8 6 5
21920 293/300 Write block 3827 len 16384 data: 5 4 4 5
21921 294/300 Read block 2589 len 40960
21922 295/300 Read block 3928 len 36864
21923 296/300 Read block 1621 len 8192
21924 297/300 Read block 4503 len 20480
21925 298/300 Write block 750 len 16384 data: 6 7 6 6
21926 299/300 Read block 4829 len 8192
21927 300/300 Write block 3299 len 32768 data: 4 4 6 3 4 3 4 4
21928 301/300 Write block 1615 len 36864 data: 5 5 5 6 6 6 8 6 6
21929 Main task has work to do, go find it
21930 got dsc Start(0)
21931 start 0
21932 Jan 11 06:10:03.775 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/0, method: POST, req_id: cb3cdd3b-dd65-4f38-93ac-3fb52baebee0, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
21933 Replay: started 0, returned:Ok(())
21934 ----------------------------------------------------------------
21935 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21936 States: Offline Active Active
21937 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21938 7038 Acked Read 7 Sent Done Done false
21939 7039 Acked Read 6 Sent Done Done false
21940 7040 Acked Read 2 Sent Done Done false
21941 7041 Acked Read 5 Sent Done Done false
21942 7042 Acked Read 4 Sent Done Done false
21943 7043 Acked Read 5 Sent Done Done false
21944 7044 Acked Read 3 Sent Done Done false
21945 7045 Acked Read 2 Sent Done Done false
21946 7046 Acked Read 9 Sent Done Done false
21947 7047 Acked Write 8 Sent Done Done false
21948 7048 Acked Read 5 Sent Done Done false
21949 7049 Acked Read 6 Sent Done Done false
21950 7050 Acked Read 2 Sent Done Done false
21951 7051 Acked Read 9 Sent Done Done false
21952 7052 Acked Flush 0 Sent Done Done false
21953 7053 Acked Read 6 Sent Done Done false
21954 7054 Acked Write 3 Sent Done Done false
21955 7055 Acked Read 5 Sent Done Done false
21956 7056 Acked Read 3 Sent Done Done false
21957 7057 Acked Write 5 Sent Done Done false
21958 7058 Acked Read 4 Sent Done Done false
21959 7059 Acked Write 10 Sent Done Done false
21960 7060 Acked Flush 0 Sent Done Done false
21961 7061 Acked Read 10 Sent Done Done false
21962 7062 Acked Read 1 Sent Done Done false
21963 7063 Acked Write 9 Sent Done Done false
21964 7064 Acked Flush 0 Sent Done Done false
21965 7065 Acked Read 9 Sent Done Done false
21966 7066 Acked Read 1 Sent Done Done false
21967 7067 Acked Write 3 Sent Done Done false
21968 7068 Acked Read 7 Sent Done Done false
21969 7069 Acked Flush 0 Sent Done Done false
21970 7070 Acked Write 7 Sent Done Done false
21971 7071 Acked Read 7 Sent Done Done false
21972 7072 Acked Write 8 Sent Done Done false
21973 7073 Acked Write 5 Sent Done Done false
21974 7074 Acked Read 10 Sent Done Done false
21975 7075 Acked Read 2 Sent Done Done false
21976 7076 Acked Read 4 Sent Done Done false
21977 7077 Acked Read 3 Sent Done Done false
21978 7078 Acked Read 1 Sent Done Done false
21979 7079 Acked Write 7 Sent Done Done false
21980 7080 Acked Flush 0 Sent Done Done false
21981 7081 Acked Read 1 Sent Done Done false
21982 7082 Acked Write 4 Sent Done Done false
21983 7083 Acked Read 8 Sent Done Done false
21984 7084 Acked Read 7 Sent Done Done false
21985 7085 Acked Flush 0 Sent Done Done false
21986 7086 Acked Read 5 Sent Done Done false
21987 7087 Acked Read 2 Sent Done Done false
21988 7088 Acked Read 7 Sent Done Done false
21989 7089 Acked Read 9 Sent Done Done false
21990 7090 Acked Write 5 Sent Done Done false
21991 7091 Acked Read 10 Sent Done Done false
21992 7092 Acked Write 10 Sent Done Done false
21993 7093 Acked Read 8 Sent Done Done false
21994 7094 Acked Read 1 Sent Done Done false
21995 7095 Acked Read 3 Sent Done Done false
21996 7096 Acked Write 9 Sent Done Done false
21997 7097 Acked Read 3 Sent Done Done false
21998 7098 Acked Read 7 Sent Done Done false
21999 7099 Acked Write 6 Sent Done Done false
22000 7100 Acked Write 4 Sent Done Done false
22001 7101 Acked Write 10 Sent Done Done false
22002 7102 Acked Write 10 Sent Done Done false
22003 7103 Acked Write 10 Sent Done Done false
22004 7104 Acked Flush 0 Sent Done Done false
22005 7105 Acked Read 8 Sent Done Done false
22006 7106 Acked Write 1 Sent Done Done false
22007 7107 Acked Read 2 Sent Done Done false
22008 7108 Acked Write 3 Sent Done Done false
22009 7109 Acked Read 7 Sent Done Done false
22010 7110 Acked Write 3 Sent Done Done false
22011 7111 Acked Write 2 Sent Done Done false
22012 7112 Acked Write 7 Sent Done Done false
22013 7113 Acked Write 10 Sent Done Done false
22014 7114 Acked Write 3 Sent Done Done false
22015 7115 Acked Write 3 Sent Done Done false
22016 7116 Acked Read 6 Sent Done Done false
22017 7117 Acked Write 3 Sent Done Done false
22018 7118 Acked Write 2 Sent Done Done false
22019 7119 Acked Write 10 Sent Done Done false
22020 7120 Acked Write 10 Sent Done Done false
22021 7121 Acked Read 5 Sent Done Done false
22022 7122 Acked Flush 0 Sent Done Done false
22023 7123 Acked Read 10 Sent Done Done false
22024 7124 Acked Read 3 Sent Done Done false
22025 7125 Acked Flush 0 Sent Done Done false
22026 7126 Acked Read 5 Sent Done Done false
22027 7127 Acked Write 6 Sent Done Done false
22028 7128 Acked Read 1 Sent Done Done false
22029 7129 Acked Flush 0 Sent Done Done false
22030 7130 Acked Read 8 Sent Done Done false
22031 7131 Acked Read 6 Sent Done Done false
22032 7132 Acked Read 3 Sent Done Done false
22033 7133 Acked Read 4 Sent Done Done false
22034 7134 Acked Write 4 Sent Done Done false
22035 7135 Acked Read 4 Sent Done Done false
22036 7136 Acked Write 1 Sent Done Done false
22037 7137 Acked Read 6 Sent Done Done false
22038 7138 Acked Read 3 Sent Done Done false
22039 7139 Acked Flush 0 Sent Done Done false
22040 7140 Acked Flush 0 Sent Done Done false
22041 7141 Acked Write 3 Sent Done Done false
22042 7142 Acked Read 1 Sent Done Done false
22043 7143 Acked Flush 0 Sent Done Done false
22044 7144 Acked Read 9 Sent Done Done false
22045 7145 Acked Write 3 Sent Done Done false
22046 7146 Acked Read 1 Sent Done Done false
22047 7147 Acked Write 6 Sent Done Done false
22048 7148 Acked Read 7 Sent Done Done false
22049 7149 Acked Write 6 Sent Done Done false
22050 7150 Acked Read 6 Sent Done Done false
22051 7151 Acked Read 6 Sent Done Done false
22052 7152 Acked Read 4 Sent Done Done false
22053 7153 Acked Read 7 Sent Done Done false
22054 7154 Acked Write 2 Sent Done Done false
22055 7155 Acked Read 1 Sent Done Done false
22056 7156 Acked Read 1 Sent Done Done false
22057 7157 Acked Write 5 Sent Done Done false
22058 7158 Acked Read 1 Sent Done Done false
22059 7159 Acked Read 2 Sent Done Done false
22060 7160 Acked Read 5 Sent Done Done false
22061 7161 Acked Read 9 Sent Done Done false
22062 7162 Acked Write 10 Sent Done Done false
22063 7163 Acked Write 7 Sent Done Done false
22064 7164 Acked Write 6 Sent Done Done false
22065 7165 Acked Read 3 Sent Done Done false
22066 7166 Acked Write 8 Sent Done Done false
22067 7167 Acked Read 3 Sent Done Done false
22068 7168 Acked Write 10 Sent Done Done false
22069 7169 Acked Write 10 Sent Done Done false
22070 7170 Acked Flush 0 Sent Done Done false
22071 7171 Acked Read 5 Sent Done Done false
22072 7172 Acked Flush 0 Sent Done Done false
22073 7173 Acked Write 5 Sent Done Done false
22074 7174 Acked Write 4 Sent Done Done false
22075 7175 Acked Read 5 Sent Done Done false
22076 7176 Acked Read 5 Sent Done Done false
22077 7177 Acked Read 9 Sent Done Done false
22078 7178 Acked Flush 0 Sent Done Done false
22079 7179 Acked Flush 0 Sent Done Done false
22080 7180 Acked Write 3 Sent Done Done false
22081 7181 Acked Write 6 Sent Done Done false
22082 7182 Acked Read 3 Sent Done Done false
22083 7183 Acked Read 2 Sent Done Done false
22084 7184 Acked Read 10 Sent Done Done false
22085 7185 Acked Read 10 Sent Done Done false
22086 7186 Acked Write 3 Sent Done Done false
22087 7187 Acked Write 6 Sent Done Done false
22088 7188 Acked Read 9 Sent Done Done false
22089 7189 Acked Read 7 Sent Done Done false
22090 7190 Acked Read 4 Sent Done Done false
22091 7191 Acked Write 4 Sent Done Done false
22092 7192 Acked Flush 0 Sent Done Done false
22093 7193 Acked Read 9 Sent Done Done false
22094 7194 Acked Read 6 Sent Done Done false
22095 7195 Acked Read 1 Sent Done Done false
22096 7196 Acked Read 3 Sent Done Done false
22097 7197 Acked Flush 0 Sent Done Done false
22098 7198 Acked Read 4 Sent Done Done false
22099 7199 Acked Read 4 Sent Done Done false
22100 7200 Acked Read 6 Sent Done Done false
22101 7201 Acked Flush 0 Sent Done Done false
22102 7202 Acked Read 7 Sent Done Done false
22103 7203 Acked Read 1 Sent Done Done false
22104 7204 Acked Read 9 Sent Done Done false
22105 7205 Acked Read 2 Sent Done Done false
22106 7206 Acked Write 10 Sent Done Done false
22107 7207 Acked Write 4 Sent Done Done false
22108 7208 Acked Read 10 Sent Done Done false
22109 7209 Acked Read 4 Sent Done Done false
22110 7210 Acked Write 4 Sent Done Done false
22111 7211 Acked Write 1 Sent Done Done false
22112 7212 Acked Flush 0 Sent Done Done false
22113 7213 Acked Write 8 Sent Done Done false
22114 7214 Acked Write 6 Sent Done Done false
22115 7215 Acked Write 6 Sent Done Done false
22116 7216 Acked Flush 0 Sent Done Done false
22117 7217 Acked Read 7 Sent Done Done false
22118 7218 Acked Write 10 Sent Done Done false
22119 7219 Acked Flush 0 Sent Done Done false
22120 7220 Acked Read 7 Sent Done Done false
22121 7221 Acked Read 1 Sent Done Done false
22122 7222 Acked Write 8 Sent Done Done false
22123 7223 Acked Read 6 Sent Done Done false
22124 7224 Acked Read 1 Sent Done Done false
22125 7225 Acked Read 9 Sent Done Done false
22126 7226 Acked Read 3 Sent Done Done false
22127 7227 Acked Read 3 Sent Done Done false
22128 7228 Acked Read 3 Sent Done Done false
22129 7229 Acked Write 10 Sent Done Done false
22130 7230 Acked Write 7 Sent Done Done false
22131 7231 Acked Write 7 Sent Done Done false
22132 7232 Acked Write 5 Sent Done Done false
22133 7233 Acked Read 7 Sent Done Done false
22134 7234 Acked Write 1 Sent Done Done false
22135 7235 Acked Read 1 Sent Done Done false
22136 7236 Acked Write 7 Sent Done Done false
22137 7237 Acked Flush 0 Sent Done Done false
22138 7238 Acked Read 2 Sent Done Done false
22139 7239 Acked Read 9 Sent Done Done false
22140 7240 Acked Read 4 Sent Done Done false
22141 7241 Acked Write 6 Sent Done Done false
22142 7242 Acked Read 7 Sent Done Done false
22143 7243 Acked Write 6 Sent Done Done false
22144 7244 Acked Read 2 Sent Done Done false
22145 7245 Acked Write 3 Sent Done Done false
22146 7246 Acked Write 5 Sent Done Done false
22147 7247 Acked Read 10 Sent Done Done false
22148 7248 Acked Flush 0 Sent Done Done false
22149 7249 Acked Read 5 Sent Done Done false
22150 7250 Acked Read 3 Sent Done Done false
22151 7251 Acked Write 10 Sent Done Done false
22152 7252 Acked Write 10 Sent Done Done false
22153 7253 Acked Read 8 Sent Done Done false
22154 7254 Acked Write 4 Sent Done Done false
22155 7255 Acked Read 2 Sent Done Done false
22156 7256 Acked Read 7 Sent Done Done false
22157 7257 Acked Flush 0 Sent Done Done false
22158 7258 Acked Read 1 Sent Done Done false
22159 7259 Acked Read 6 Sent Done Done false
22160 7260 Acked Write 4 Sent Done Done false
22161 7261 Acked Read 2 Sent Done Done false
22162 7262 Acked Write 2 Sent Done Done false
22163 7263 Acked Read 5 Sent Done Done false
22164 7264 Acked Read 5 Sent Done Done false
22165 7265 Acked Read 7 Sent Done Done false
22166 7266 Acked Write 10 Sent Done Done false
22167 7267 Acked Read 1 Sent Done Done false
22168 7268 Acked Write 7 Sent Done Done false
22169 7269 Acked Read 10 Sent Done Done false
22170 7270 Acked Write 7 Sent Done Done false
22171 7271 Acked Write 5 Sent Done Done false
22172 7272 Acked Write 4 Sent Done Done false
22173 7273 Acked Read 7 Sent Done Done false
22174 7274 Acked Read 5 Sent Done Done false
22175 7275 Acked Read 8 Sent Done Done false
22176 7276 Acked Flush 0 Sent Done Done false
22177 7277 Acked Read 2 Sent Done Done false
22178 7278 Acked Read 5 Sent Done Done false
22179 7279 Acked Write 1 Sent Done Done false
22180 7280 Acked Read 6 Sent Done Done false
22181 7281 Acked Write 6 Sent Done Done false
22182 7282 Acked Flush 0 Sent Done Done false
22183 7283 Acked Read 1 Sent Done Done false
22184 7284 Acked Write 1 Sent Done Done false
22185 7285 Acked Write 1 Sent Done Done false
22186 7286 Acked Read 9 Sent Done Done false
22187 7287 Acked Write 6 Sent Done Done false
22188 7288 Acked Read 10 Sent Done Done false
22189 7289 Acked Read 10 Sent Done Done false
22190 7290 Acked Read 9 Sent Done Done false
22191 7291 Acked Flush 0 Sent Done Done false
22192 7292 Acked Read 4 Sent Done Done false
22193 7293 Acked Read 9 Sent Done Done false
22194 7294 Acked Read 8 Sent Done Done false
22195 7295 Acked Write 3 Sent Done Done false
22196 7296 Acked Flush 0 Sent Done Done false
22197 7297 Acked Read 4 Sent Done Done false
22198 7298 Acked Write 8 Sent Done Done false
22199 7299 Acked Read 9 Sent Done Done false
22200 7300 Acked Write 10 Sent Done Done false
22201 7301 Acked Flush 0 Sent Done Done false
22202 7302 Acked Read 5 Sent Done Done false
22203 7303 Acked Write 3 Sent Done Done false
22204 7304 Acked Write 10 Sent Done Done false
22205 7305 Acked Write 2 Sent Done Done false
22206 7306 Acked Read 3 Sent Done Done false
22207 7307 Acked Write 1 Sent Done Done false
22208 7308 Acked Write 5 Sent Done Done false
22209 7309 Acked Write 10 Sent Done Done false
22210 7310 Acked Write 10 Sent Done Done false
22211 7311 Acked Write 8 Sent Done Done false
22212 7312 Acked Read 1 Sent Done Done false
22213 7313 Acked Write 6 Sent Done Done false
22214 7314 Acked Write 9 Sent Done Done false
22215 7315 Acked Write 5 Sent Done Done false
22216 7316 Acked Flush 0 Sent Done Done false
22217 7317 Acked Write 5 Sent Done Done false
22218 7318 Acked Flush 0 Sent Done Done false
22219 7319 Acked Read 3 Sent Done Done false
22220 7320 Acked Read 8 Sent Done Done false
22221 7321 Acked Write 10 Sent Done Done false
22222 7322 Acked Read 2 Sent Done Done false
22223 7323 Acked Read 5 Sent Done Done false
22224 7324 Acked Write 1 Sent Done Done false
22225 7325 Acked Read 4 Sent Done Done false
22226 7326 Acked Read 4 Sent Done Done false
22227 7327 Acked Read 4 Sent Done Done false
22228 7328 Acked Read 2 Sent Done Done false
22229 7329 Acked Write 7 Sent Done Done false
22230 7330 Acked Write 4 Sent Done Done false
22231 7331 Acked Read 10 Sent Done Done false
22232 7332 Acked Read 9 Sent Done Done false
22233 7333 Acked Read 2 Sent Done Done false
22234 7334 Acked Read 5 Sent Done Done false
22235 7335 Acked Write 4 Sent Done Done false
22236 7336 Acked Read 2 Sent Done Done false
22237 7337 Acked Write 8 Sent Done Done false
22238 7338 Acked Write 9 Sent Done Sent false
22239 STATES DS:0 DS:1 DS:2 TOTAL
22240 Sent 301 0 1 302
22241 Done 0 301 300 601
22242 Skipped 0 0 0 0
22243 Error 0 0 0 0
22244 Last Flush: 7037 7318 7318
22245 Downstairs last five completed: 7037 7036 7035 7034 7033
22246 Upstairs last five completed: 7338 7337 7336 7335 7334
22247 CLIENT: Up:0 ds:301 act:2
22248 [8810] got start action
22249 [8810] I am going to restart
22250 Starting downstairs at port 8810
22251 Make output file at "/tmp/dsc/downstairs-8810.txt"
22252 [8810][0] reports Starting
22253 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1225)
22254 [8810][0] reports Running
22255 ----------------------------------------------------------------
22256 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22257 States: Offline Active Active
22258 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22259 7038 Acked Read 7 Sent Done Done false
22260 7039 Acked Read 6 Sent Done Done false
22261 7040 Acked Read 2 Sent Done Done false
22262 7041 Acked Read 5 Sent Done Done false
22263 7042 Acked Read 4 Sent Done Done false
22264 7043 Acked Read 5 Sent Done Done false
22265 7044 Acked Read 3 Sent Done Done false
22266 7045 Acked Read 2 Sent Done Done false
22267 7046 Acked Read 9 Sent Done Done false
22268 7047 Acked Write 8 Sent Done Done false
22269 7048 Acked Read 5 Sent Done Done false
22270 7049 Acked Read 6 Sent Done Done false
22271 7050 Acked Read 2 Sent Done Done false
22272 7051 Acked Read 9 Sent Done Done false
22273 7052 Acked Flush 0 Sent Done Done false
22274 7053 Acked Read 6 Sent Done Done false
22275 7054 Acked Write 3 Sent Done Done false
22276 7055 Acked Read 5 Sent Done Done false
22277 7056 Acked Read 3 Sent Done Done false
22278 7057 Acked Write 5 Sent Done Done false
22279 7058 Acked Read 4 Sent Done Done false
22280 7059 Acked Write 10 Sent Done Done false
22281 7060 Acked Flush 0 Sent Done Done false
22282 7061 Acked Read 10 Sent Done Done false
22283 7062 Acked Read 1 Sent Done Done false
22284 7063 Acked Write 9 Sent Done Done false
22285 7064 Acked Flush 0 Sent Done Done false
22286 7065 Acked Read 9 Sent Done Done false
22287 7066 Acked Read 1 Sent Done Done false
22288 7067 Acked Write 3 Sent Done Done false
22289 7068 Acked Read 7 Sent Done Done false
22290 7069 Acked Flush 0 Sent Done Done false
22291 7070 Acked Write 7 Sent Done Done false
22292 7071 Acked Read 7 Sent Done Done false
22293 7072 Acked Write 8 Sent Done Done false
22294 7073 Acked Write 5 Sent Done Done false
22295 7074 Acked Read 10 Sent Done Done false
22296 7075 Acked Read 2 Sent Done Done false
22297 7076 Acked Read 4 Sent Done Done false
22298 7077 Acked Read 3 Sent Done Done false
22299 7078 Acked Read 1 Sent Done Done false
22300 7079 Acked Write 7 Sent Done Done false
22301 7080 Acked Flush 0 Sent Done Done false
22302 7081 Acked Read 1 Sent Done Done false
22303 7082 Acked Write 4 Sent Done Done false
22304 7083 Acked Read 8 Sent Done Done false
22305 7084 Acked Read 7 Sent Done Done false
22306 7085 Acked Flush 0 Sent Done Done false
22307 7086 Acked Read 5 Sent Done Done false
22308 7087 Acked Read 2 Sent Done Done false
22309 7088 Acked Read 7 Sent Done Done false
22310 7089 Acked Read 9 Sent Done Done false
22311 7090 Acked Write 5 Sent Done Done false
22312 7091 Acked Read 10 Sent Done Done false
22313 7092 Acked Write 10 Sent Done Done false
22314 7093 Acked Read 8 Sent Done Done false
22315 7094 Acked Read 1 Sent Done Done false
22316 7095 Acked Read 3 Sent Done Done false
22317 7096 Acked Write 9 Sent Done Done false
22318 7097 Acked Read 3 Sent Done Done false
22319 7098 Acked Read 7 Sent Done Done false
22320 7099 Acked Write 6 Sent Done Done false
22321 7100 Acked Write 4 Sent Done Done false
22322 7101 Acked Write 10 Sent Done Done false
22323 7102 Acked Write 10 Sent Done Done false
22324 7103 Acked Write 10 Sent Done Done false
22325 7104 Acked Flush 0 Sent Done Done false
22326 7105 Acked Read 8 Sent Done Done false
22327 7106 Acked Write 1 Sent Done Done false
22328 7107 Acked Read 2 Sent Done Done false
22329 7108 Acked Write 3 Sent Done Done false
22330 7109 Acked Read 7 Sent Done Done false
22331 7110 Acked Write 3 Sent Done Done false
22332 7111 Acked Write 2 Sent Done Done false
22333 7112 Acked Write 7 Sent Done Done false
22334 7113 Acked Write 10 Sent Done Done false
22335 7114 Acked Write 3 Sent Done Done false
22336 7115 Acked Write 3 Sent Done Done false
22337 7116 Acked Read 6 Sent Done Done false
22338 7117 Acked Write 3 Sent Done Done false
22339 7118 Acked Write 2 Sent Done Done false
22340 7119 Acked Write 10 Sent Done Done false
22341 7120 Acked Write 10 Sent Done Done false
22342 7121 Acked Read 5 Sent Done Done false
22343 7122 Acked Flush 0 Sent Done Done false
22344 7123 Acked Read 10 Sent Done Done false
22345 7124 Acked Read 3 Sent Done Done false
22346 7125 Acked Flush 0 Sent Done Done false
22347 7126 Acked Read 5 Sent Done Done false
22348 7127 Acked Write 6 Sent Done Done false
22349 7128 Acked Read 1 Sent Done Done false
22350 7129 Acked Flush 0 Sent Done Done false
22351 7130 Acked Read 8 Sent Done Done false
22352 7131 Acked Read 6 Sent Done Done false
22353 7132 Acked Read 3 Sent Done Done false
22354 7133 Acked Read 4 Sent Done Done false
22355 7134 Acked Write 4 Sent Done Done false
22356 7135 Acked Read 4 Sent Done Done false
22357 7136 Acked Write 1 Sent Done Done false
22358 7137 Acked Read 6 Sent Done Done false
22359 7138 Acked Read 3 Sent Done Done false
22360 7139 Acked Flush 0 Sent Done Done false
22361 7140 Acked Flush 0 Sent Done Done false
22362 7141 Acked Write 3 Sent Done Done false
22363 7142 Acked Read 1 Sent Done Done false
22364 7143 Acked Flush 0 Sent Done Done false
22365 7144 Acked Read 9 Sent Done Done false
22366 7145 Acked Write 3 Sent Done Done false
22367 7146 Acked Read 1 Sent Done Done false
22368 7147 Acked Write 6 Sent Done Done false
22369 7148 Acked Read 7 Sent Done Done false
22370 7149 Acked Write 6 Sent Done Done false
22371 7150 Acked Read 6 Sent Done Done false
22372 7151 Acked Read 6 Sent Done Done false
22373 7152 Acked Read 4 Sent Done Done false
22374 7153 Acked Read 7 Sent Done Done false
22375 7154 Acked Write 2 Sent Done Done false
22376 7155 Acked Read 1 Sent Done Done false
22377 7156 Acked Read 1 Sent Done Done false
22378 7157 Acked Write 5 Sent Done Done false
22379 7158 Acked Read 1 Sent Done Done false
22380 7159 Acked Read 2 Sent Done Done false
22381 7160 Acked Read 5 Sent Done Done false
22382 7161 Acked Read 9 Sent Done Done false
22383 7162 Acked Write 10 Sent Done Done false
22384 7163 Acked Write 7 Sent Done Done false
22385 7164 Acked Write 6 Sent Done Done false
22386 7165 Acked Read 3 Sent Done Done false
22387 7166 Acked Write 8 Sent Done Done false
22388 7167 Acked Read 3 Sent Done Done false
22389 7168 Acked Write 10 Sent Done Done false
22390 7169 Acked Write 10 Sent Done Done false
22391 7170 Acked Flush 0 Sent Done Done false
22392 7171 Acked Read 5 Sent Done Done false
22393 7172 Acked Flush 0 Sent Done Done false
22394 7173 Acked Write 5 Sent Done Done false
22395 7174 Acked Write 4 Sent Done Done false
22396 7175 Acked Read 5 Sent Done Done false
22397 7176 Acked Read 5 Sent Done Done false
22398 7177 Acked Read 9 Sent Done Done false
22399 7178 Acked Flush 0 Sent Done Done false
22400 7179 Acked Flush 0 Sent Done Done false
22401 7180 Acked Write 3 Sent Done Done false
22402 7181 Acked Write 6 Sent Done Done false
22403 7182 Acked Read 3 Sent Done Done false
22404 7183 Acked Read 2 Sent Done Done false
22405 7184 Acked Read 10 Sent Done Done false
22406 7185 Acked Read 10 Sent Done Done false
22407 7186 Acked Write 3 Sent Done Done false
22408 7187 Acked Write 6 Sent Done Done false
22409 7188 Acked Read 9 Sent Done Done false
22410 7189 Acked Read 7 Sent Done Done false
22411 7190 Acked Read 4 Sent Done Done false
22412 7191 Acked Write 4 Sent Done Done false
22413 7192 Acked Flush 0 Sent Done Done false
22414 7193 Acked Read 9 Sent Done Done false
22415 7194 Acked Read 6 Sent Done Done false
22416 7195 Acked Read 1 Sent Done Done false
22417 7196 Acked Read 3 Sent Done Done false
22418 7197 Acked Flush 0 Sent Done Done false
22419 7198 Acked Read 4 Sent Done Done false
22420 7199 Acked Read 4 Sent Done Done false
22421 7200 Acked Read 6 Sent Done Done false
22422 7201 Acked Flush 0 Sent Done Done false
22423 7202 Acked Read 7 Sent Done Done false
22424 7203 Acked Read 1 Sent Done Done false
22425 7204 Acked Read 9 Sent Done Done false
22426 7205 Acked Read 2 Sent Done Done false
22427 7206 Acked Write 10 Sent Done Done false
22428 7207 Acked Write 4 Sent Done Done false
22429 7208 Acked Read 10 Sent Done Done false
22430 7209 Acked Read 4 Sent Done Done false
22431 7210 Acked Write 4 Sent Done Done false
22432 7211 Acked Write 1 Sent Done Done false
22433 7212 Acked Flush 0 Sent Done Done false
22434 7213 Acked Write 8 Sent Done Done false
22435 7214 Acked Write 6 Sent Done Done false
22436 7215 Acked Write 6 Sent Done Done false
22437 7216 Acked Flush 0 Sent Done Done false
22438 7217 Acked Read 7 Sent Done Done false
22439 7218 Acked Write 10 Sent Done Done false
22440 7219 Acked Flush 0 Sent Done Done false
22441 7220 Acked Read 7 Sent Done Done false
22442 7221 Acked Read 1 Sent Done Done false
22443 7222 Acked Write 8 Sent Done Done false
22444 7223 Acked Read 6 Sent Done Done false
22445 7224 Acked Read 1 Sent Done Done false
22446 7225 Acked Read 9 Sent Done Done false
22447 7226 Acked Read 3 Sent Done Done false
22448 7227 Acked Read 3 Sent Done Done false
22449 7228 Acked Read 3 Sent Done Done false
22450 7229 Acked Write 10 Sent Done Done false
22451 7230 Acked Write 7 Sent Done Done false
22452 7231 Acked Write 7 Sent Done Done false
22453 7232 Acked Write 5 Sent Done Done false
22454 7233 Acked Read 7 Sent Done Done false
22455 7234 Acked Write 1 Sent Done Done false
22456 7235 Acked Read 1 Sent Done Done false
22457 7236 Acked Write 7 Sent Done Done false
22458 7237 Acked Flush 0 Sent Done Done false
22459 7238 Acked Read 2 Sent Done Done false
22460 7239 Acked Read 9 Sent Done Done false
22461 7240 Acked Read 4 Sent Done Done false
22462 7241 Acked Write 6 Sent Done Done false
22463 7242 Acked Read 7 Sent Done Done false
22464 7243 Acked Write 6 Sent Done Done false
22465 7244 Acked Read 2 Sent Done Done false
22466 7245 Acked Write 3 Sent Done Done false
22467 7246 Acked Write 5 Sent Done Done false
22468 7247 Acked Read 10 Sent Done Done false
22469 7248 Acked Flush 0 Sent Done Done false
22470 7249 Acked Read 5 Sent Done Done false
22471 7250 Acked Read 3 Sent Done Done false
22472 7251 Acked Write 10 Sent Done Done false
22473 7252 Acked Write 10 Sent Done Done false
22474 7253 Acked Read 8 Sent Done Done false
22475 7254 Acked Write 4 Sent Done Done false
22476 7255 Acked Read 2 Sent Done Done false
22477 7256 Acked Read 7 Sent Done Done false
22478 7257 Acked Flush 0 Sent Done Done false
22479 7258 Acked Read 1 Sent Done Done false
22480 7259 Acked Read 6 Sent Done Done false
22481 7260 Acked Write 4 Sent Done Done false
22482 7261 Acked Read 2 Sent Done Done false
22483 7262 Acked Write 2 Sent Done Done false
22484 7263 Acked Read 5 Sent Done Done false
22485 7264 Acked Read 5 Sent Done Done false
22486 7265 Acked Read 7 Sent Done Done false
22487 7266 Acked Write 10 Sent Done Done false
22488 7267 Acked Read 1 Sent Done Done false
22489 7268 Acked Write 7 Sent Done Done false
22490 7269 Acked Read 10 Sent Done Done false
22491 7270 Acked Write 7 Sent Done Done false
22492 7271 Acked Write 5 Sent Done Done false
22493 7272 Acked Write 4 Sent Done Done false
22494 7273 Acked Read 7 Sent Done Done false
22495 7274 Acked Read 5 Sent Done Done false
22496 7275 Acked Read 8 Sent Done Done false
22497 7276 Acked Flush 0 Sent Done Done false
22498 7277 Acked Read 2 Sent Done Done false
22499 7278 Acked Read 5 Sent Done Done false
22500 7279 Acked Write 1 Sent Done Done false
22501 7280 Acked Read 6 Sent Done Done false
22502 7281 Acked Write 6 Sent Done Done false
22503 7282 Acked Flush 0 Sent Done Done false
22504 7283 Acked Read 1 Sent Done Done false
22505 7284 Acked Write 1 Sent Done Done false
22506 7285 Acked Write 1 Sent Done Done false
22507 7286 Acked Read 9 Sent Done Done false
22508 7287 Acked Write 6 Sent Done Done false
22509 7288 Acked Read 10 Sent Done Done false
22510 7289 Acked Read 10 Sent Done Done false
22511 7290 Acked Read 9 Sent Done Done false
22512 7291 Acked Flush 0 Sent Done Done false
22513 7292 Acked Read 4 Sent Done Done false
22514 7293 Acked Read 9 Sent Done Done false
22515 7294 Acked Read 8 Sent Done Done false
22516 7295 Acked Write 3 Sent Done Done false
22517 7296 Acked Flush 0 Sent Done Done false
22518 7297 Acked Read 4 Sent Done Done false
22519 7298 Acked Write 8 Sent Done Done false
22520 7299 Acked Read 9 Sent Done Done false
22521 7300 Acked Write 10 Sent Done Done false
22522 7301 Acked Flush 0 Sent Done Done false
22523 7302 Acked Read 5 Sent Done Done false
22524 7303 Acked Write 3 Sent Done Done false
22525 7304 Acked Write 10 Sent Done Done false
22526 7305 Acked Write 2 Sent Done Done false
22527 7306 Acked Read 3 Sent Done Done false
22528 7307 Acked Write 1 Sent Done Done false
22529 7308 Acked Write 5 Sent Done Done false
22530 7309 Acked Write 10 Sent Done Done false
22531 7310 Acked Write 10 Sent Done Done false
22532 7311 Acked Write 8 Sent Done Done false
22533 7312 Acked Read 1 Sent Done Done false
22534 7313 Acked Write 6 Sent Done Done false
22535 7314 Acked Write 9 Sent Done Done false
22536 7315 Acked Write 5 Sent Done Done false
22537 7316 Acked Flush 0 Sent Done Done false
22538 7317 Acked Write 5 Sent Done Done false
22539 7318 Acked Flush 0 Sent Done Done false
22540 7319 Acked Read 3 Sent Done Done false
22541 7320 Acked Read 8 Sent Done Done false
22542 7321 Acked Write 10 Sent Done Done false
22543 7322 Acked Read 2 Sent Done Done false
22544 7323 Acked Read 5 Sent Done Done false
22545 7324 Acked Write 1 Sent Done Done false
22546 7325 Acked Read 4 Sent Done Done false
22547 7326 Acked Read 4 Sent Done Done false
22548 7327 Acked Read 4 Sent Done Done false
22549 7328 Acked Read 2 Sent Done Done false
22550 7329 Acked Write 7 Sent Done Done false
22551 7330 Acked Write 4 Sent Done Done false
22552 7331 Acked Read 10 Sent Done Done false
22553 7332 Acked Read 9 Sent Done Done false
22554 7333 Acked Read 2 Sent Done Done false
22555 7334 Acked Read 5 Sent Done Done false
22556 7335 Acked Write 4 Sent Done Done false
22557 7336 Acked Read 2 Sent Done Done false
22558 7337 Acked Write 8 Sent Done Done false
22559 7338 Acked Write 9 Sent Done Done false
22560 7339 Acked Flush 0 Sent Done Done false
22561 STATES DS:0 DS:1 DS:2 TOTAL
22562 Sent 302 0 0 302
22563 Done 0 302 302 604
22564 Skipped 0 0 0 0
22565 Error 0 0 0 0
22566 Last Flush: 7037 7339 7339
22567 Downstairs last five completed: 7037 7036 7035 7034 7033
22568 Upstairs last five completed: 7339 7338 7337 7336 7335
22569 CLIENT: Up:0 ds:302 act:2
225702025-01-11T06:10:09.530ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225712025-01-11T06:10:09.530ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35440) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225722025-01-11T06:10:09.532ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225732025-01-11T06:10:09.532ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225742025-01-11T06:10:09.532ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225752025-01-11T06:10:09.532ZINFOcrucible: send last flush ID to this DS: 7037 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225762025-01-11T06:10:09.532ZINFOcrucible: Replied this last flush ID: 7037 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225772025-01-11T06:10:09.532ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225782025-01-11T06:10:09.532ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225792025-01-11T06:10:09.532ZINFOcrucible: [0] client re-new 302 jobs since flush 7037 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225802025-01-11T06:10:09.533ZINFOcrucible: [0] Replayed 302 jobs since flush: 7037 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
22581 ----------------------------------------------------------------
22582 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22583 Downstairs last five completed: 7339 7338 7337 7336 7335
22584 Upstairs last five completed: 7339 7338 7337 7336 7335
22585 CLIENT: Up:0 ds:0 act:3
22586 Replay: All jobs finished
22587 Main task has work to do, go find it
22588 got dsc Stop(1)
22589 stop 1
22590 [8820] Got stop action so:false kr:false
22591 Jan 11 06:10:11.780 INFO request completed, latency_us: 324, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 9ed76779-9d00-49e6-9208-326bb29af252, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
225922025-01-11T06:10:11.781ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225932025-01-11T06:10:11.781ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
22594 Jan 11 06:10:11.781 INFO request completed{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:10:11.781536676Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
22595 , latency_us: 156, response_code: 200, uri: /state/cid/1, method: GET, req_id: 9b3b1604-6e4e-488f-91b5-a89e8c39fd82, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
225962025-01-11T06:10:11.781ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225972025-01-11T06:10:11.781ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
225982025-01-11T06:10:11.781ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
22599 [8820] Exited with: ExitStatus(unix_wait_status(9))
22600 [8820][1] reports Exit
22601 Jan 11 06:10:15.784 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/1, method: GET, req_id: 05b634b4-bfc6-4fa2-9c30-3b86ab084c35, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
22602 001/300 Flush
22603 002/300 Write block 4696 len 32768 data: 5 7 9 10 7 5 4 4
22604 003/300 Read block 4442 len 8192
22605 004/300 Read block 2001 len 20480
22606 005/300 Write block 1641 len 36864 data: 4 3 3 2 2 3 3 3 2
22607 006/300 Write block 4574 len 32768 data: 5 4 5 5 5 5 7 6
22608 007/300 Write block 4781 len 8192 data: 5 5
22609 008/300 Write block 1662 len 8192 data: 2 3
22610 009/300 Write block 2325 len 12288 data: 4 3 4
22611 010/300 Write block 2295 len 12288 data: 5 5 5
22612 011/300 Read block 945 len 12288
22613 012/300 Flush
22614 013/300 Read block 4380 len 28672
22615 014/300 Read block 2910 len 4096
22616 015/300 Read block 954 len 4096
22617 016/300 Write block 56 len 24576 data: 6 7 8 7 7 7
22618 017/300 Flush
22619 018/300 Write block 1875 len 8192 data: 7 7
22620 019/300 Read block 4307 len 4096
22621 020/300 Read block 2720 len 32768
22622 021/300 Read block 841 len 12288
22623 022/300 Flush
22624 023/300 Read block 4582 len 4096
22625 024/300 Write block 1615 len 4096 data: 6
22626 025/300 Write block 3093 len 12288 data: 6 5 6
22627 026/300 Write block 187 len 16384 data: 6 6 7 7
22628 027/300 Write block 1617 len 24576 data: 6 7 7 7 9 7
22629 028/300 Write block 48 len 4096 data: 4
22630 029/300 Write block 3312 len 24576 data: 3 2 2 3 4 4
22631 030/300 Flush
22632 031/300 Read block 913 len 20480
22633 032/300 Read block 886 len 28672
22634 033/300 Write block 504 len 24576 data: 7 3 3 2 3 3
22635 034/300 Write block 3427 len 16384 data: 5 4 5 6
22636 035/300 Read block 244 len 36864
22637 036/300 Read block 1631 len 28672
22638 037/300 Write block 1119 len 8192 data: 6 6
22639 038/300 Read block 556 len 16384
22640 039/300 Write block 3468 len 32768 data: 3 5 5 5 4 7 7 8
22641 040/300 Write block 4058 len 4096 data: 2
22642 041/300 Read block 1485 len 12288
22643 042/300 Flush
22644 043/300 Write block 1012 len 28672 data: 4 3 3 4 3 3 2
22645 044/300 Read block 234 len 16384
22646 045/300 Read block 1662 len 8192
22647 046/300 Read block 2720 len 28672
22648 047/300 Write block 2299 len 4096 data: 7
22649 048/300 Flush
22650 049/300 Read block 2095 len 8192
22651 050/300 Write block 1940 len 40960 data: 6 6 5 6 5 5 5 5 5 4
22652 051/300 Read block 2088 len 16384
22653 052/300 Flush
22654 053/300 Write block 883 len 4096 data: 6
22655 054/300 Read block 972 len 24576
22656 055/300 Flush
22657 056/300 Read block 4660 len 36864
22658 057/300 Write block 3555 len 36864 data: 3 2 2 4 4 6 7 7 6
22659 058/300 Read block 4046 len 16384
22660 059/300 Read block 3548 len 8192
22661 060/300 Read block 1657 len 32768
22662 061/300 Read block 3246 len 16384
22663 062/300 Read block 4747 len 24576
22664 063/300 Write block 1545 len 28672 data: 4 3 3 4 4 4 4
22665 064/300 Write block 312 len 16384 data: 4 2 2 2
22666 065/300 Write block 2237 len 40960 data: 6 6 5 3 4 4 4 5 5 5
22667 066/300 Read block 1889 len 24576
22668 067/300 Read block 2780 len 40960
22669 068/300 Read block 1550 len 36864
22670 069/300 Flush
22671 070/300 Read block 646 len 16384
22672 071/300 Read block 774 len 12288
22673 072/300 Write block 3555 len 8192 data: 4 3
22674 073/300 Read block 3955 len 20480
22675 074/300 Write block 4656 len 16384 data: 5 5 5 4
22676 075/300 Read block 4118 len 40960
22677 076/300 Flush
22678 077/300 Read block 432 len 32768
22679 078/300 Write block 3462 len 36864 data: 4 4 4 3 3 3 4 6 6
22680 079/300 Write block 3065 len 28672 data: 5 5 5 5 4 3 3
22681 080/300 Write block 2873 len 40960 data: 6 4 4 5 5 4 4 5 4 4
22682 081/300 Flush
22683 082/300 Read block 0 len 28672
22684 083/300 Read block 2417 len 8192
22685 084/300 Write block 214 len 4096 data: 5
22686 085/300 Write block 3230 len 20480 data: 5 5 5 4 3
22687 086/300 Write block 2604 len 20480 data: 5 5 4 3 2
22688 087/300 Write block 1211 len 4096 data: 7
22689 088/300 Write block 4048 len 4096 data: 7
22690 089/300 Read block 3780 len 36864
22691 090/300 Write block 3823 len 8192 data: 5 5
22692 091/300 Write block 2776 len 12288 data: 6 6 6
22693 092/300 Read block 1783 len 32768
22694 093/300 Write block 3047 len 28672 data: 3 3 3 4 4 4 3
22695 094/300 Write block 103 len 32768 data: 4 4 4 6 5 5 5 5
22696 095/300 Flush
22697 096/300 Write block 143 len 4096 data: 5
22698 097/300 Read block 2612 len 20480
22699 098/300 Read block 2651 len 4096
22700 099/300 Flush
22701 100/300 Write block 3514 len 40960 data: 9 7 7 5 5 4 5 5 5 5
22702 101/300 Write block 3218 len 28672 data: 4 4 3 4 4 3 3
22703 102/300 Flush
22704 103/300 Read block 646 len 40960
22705 104/300 Write block 3290 len 24576 data: 3 3 3 3 2 3
22706 105/300 Read block 436 len 8192
22707 106/300 Read block 1589 len 16384
22708 107/300 Write block 2767 len 32768 data: 3 3 3 4 5 7 6 8
22709 108/300 Read block 582 len 12288
22710 109/300 Read block 214 len 36864
22711 110/300 Write block 1890 len 12288 data: 3 3 3
22712 111/300 Read block 129 len 12288
22713 112/300 Write block 4841 len 20480 data: 5 3 3 4 4
22714 113/300 Read block 2696 len 40960
22715 114/300 Write block 3042 len 8192 data: 7 5
22716 115/300 Read block 663 len 32768
22717 116/300 Write block 3466 len 36864 data: 4 4 5 7 7 6 5 8 8
22718 117/300 Read block 4954 len 32768
22719 118/300 Write block 201 len 4096 data: 6
22720 119/300 Read block 568 len 12288
22721 120/300 Read block 3746 len 8192
22722 121/300 Read block 1764 len 28672
22723 122/300 Read block 4951 len 4096
22724 123/300 Read block 3910 len 40960
22725 124/300 Read block 2532 len 32768
22726 125/300 Flush
22727 126/300 Write block 1805 len 8192 data: 3 3
22728 127/300 Write block 1093 len 36864 data: 8 8 9 8 8 8 7 7 5
22729 128/300 Write block 2163 len 32768 data: 3 4 4 5 4 4 3 2
22730 129/300 Write block 1609 len 20480 data: 3 5 4 5 5
22731 130/300 Read block 3147 len 36864
22732 131/300 Read block 1673 len 20480
22733 132/300 Write block 3008 len 36864 data: 8 7 4 4 4 5 7 6 5
22734 133/300 Write block 4397 len 40960 data: 5 5 5 5 6 6 7 5 6 8
22735 134/300 Write block 3341 len 12288 data: 4 5 4
22736 135/300 Write block 3494 len 40960 data: 3 4 4 3 4 5 6 7 8 7
22737 136/300 Read block 2060 len 12288
22738 137/300 Flush
22739 138/300 Write block 4074 len 16384 data: 12 10 10 7
22740 139/300 Read block 4623 len 36864
22741 140/300 Read block 2053 len 24576
22742 141/300 Flush
22743 142/300 Write block 2266 len 12288 data: 4 3 5
22744 143/300 Flush
22745 144/300 Read block 2311 len 36864
22746 145/300 Flush
22747 146/300 Write block 4364 len 32768 data: 6 4 4 3 3 4 3 3
22748 147/300 Read block 1395 len 20480
22749 148/300 Write block 1569 len 16384 data: 6 6 7 7
22750 149/300 Read block 4259 len 32768
22751 150/300 Write block 86 len 24576 data: 3 3 3 4 4 4
22752 151/300 Read block 4867 len 24576
22753 152/300 Write block 1620 len 12288 data: 8 10 8
22754 153/300 Read block 1243 len 20480
22755 154/300 Read block 478 len 36864
22756 155/300 Write block 1271 len 12288 data: 3 3 3
22757 156/300 Read block 509 len 36864
22758 157/300 Read block 2528 len 12288
22759 158/300 Write block 4455 len 12288 data: 10 7 4
22760 159/300 Read block 3642 len 20480
22761 160/300 Read block 3638 len 4096
22762 161/300 Write block 4378 len 4096 data: 3
22763 162/300 Read block 342 len 24576
22764 163/300 Read block 2747 len 40960
22765 164/300 Write block 1142 len 40960 data: 4 3 5 6 5 4 5 6 6 6
22766 165/300 Write block 405 len 36864 data: 6 5 4 4 4 4 4 4 3
22767 166/300 Read block 705 len 28672
22768 167/300 Read block 4492 len 32768
22769 168/300 Read block 711 len 20480
22770 169/300 Write block 1844 len 36864 data: 6 6 9 8 8 6 7 7 5
22771 170/300 Write block 4582 len 32768 data: 5 4 5 6 6 6 5 5
22772 171/300 Read block 4268 len 40960
22773 172/300 Read block 2473 len 24576
22774 173/300 Read block 2510 len 24576
22775 174/300 Read block 233 len 4096
22776 175/300 Read block 1953 len 24576
22777 176/300 Write block 2459 len 24576 data: 5 4 4 4 5 6
22778 177/300 Read block 2886 len 24576
22779 178/300 Read block 2292 len 36864
22780 179/300 Write block 3847 len 8192 data: 3 3
22781 180/300 Flush
22782 181/300 Read block 4416 len 32768
22783 182/300 Write block 4714 len 24576 data: 3 4 4 5 5 4
22784 183/300 Read block 4169 len 24576
22785 184/300 Write block 4740 len 32768 data: 4 5 4 5 5 5 5 7
22786 185/300 Read block 1281 len 36864
22787 186/300 Read block 4895 len 28672
22788 187/300 Write block 4550 len 32768 data: 3 3 2 2 2 2 2 4
22789 188/300 Read block 1343 len 24576
22790 189/300 Read block 907 len 40960
22791 190/300 Read block 76 len 4096
22792 191/300 Read block 4694 len 8192
22793 192/300 Flush
22794 193/300 Read block 717 len 16384
22795 194/300 Write block 4471 len 36864 data: 7 6 5 4 5 5 5 5 5
22796 195/300 Write block 1532 len 32768 data: 7 7 5 5 5 5 4 4
22797 196/300 Read block 1367 len 12288
22798 197/300 Write block 3403 len 32768 data: 6 7 6 4 5 6 6 5
22799 198/300 Read block 694 len 40960
22800 199/300 Write block 4355 len 20480 data: 7 7 7 9 9
22801 200/300 Write block 594 len 24576 data: 5 6 7 5 5 5
22802 201/300 Read block 1172 len 8192
22803 202/300 Write block 3277 len 4096 data: 7
22804 203/300 Write block 3343 len 12288 data: 5 3 4
22805 204/300 Read block 4445 len 16384
22806 205/300 Read block 4948 len 12288
22807 206/300 Read block 4466 len 32768
22808 207/300 Flush
22809 208/300 Write block 1754 len 12288 data: 7 7 6
22810 209/300 Flush
22811 210/300 Flush
22812 211/300 Read block 1280 len 20480
22813 212/300 Read block 1421 len 28672
22814 213/300 Read block 2927 len 24576
22815 214/300 Read block 3814 len 40960
22816 215/300 Read block 875 len 40960
22817 216/300 Write block 850 len 12288 data: 4 4 4
22818 217/300 Flush
22819 218/300 Read block 4875 len 8192
22820 219/300 Read block 4315 len 36864
22821 220/300 Read block 2641 len 24576
22822 221/300 Write block 3019 len 4096 data: 4
22823 222/300 Read block 3591 len 20480
22824 223/300 Read block 4366 len 4096
22825 224/300 Read block 2252 len 28672
22826 225/300 Write block 2870 len 16384 data: 9 7 5 7
22827 226/300 Read block 2678 len 32768
22828 227/300 Flush
22829 228/300 Write block 4989 len 24576 data: 7 9 8 7 4 4
22830 229/300 Write block 370 len 20480 data: 3 4 3 4 5
22831 230/300 Read block 3206 len 36864
22832 231/300 Read block 1543 len 36864
22833 232/300 Write block 2606 len 24576 data: 5 4 3 2 2 3
22834 233/300 Read block 334 len 4096
22835 234/300 Flush
22836 235/300 Read block 3105 len 12288
22837 236/300 Read block 3306 len 8192
22838 237/300 Write block 132 len 40960 data: 4 5 5 6 4 5 5 5 5 6
22839 238/300 Write block 1838 len 4096 data: 2
22840 239/300 Write block 1867 len 28672 data: 3 3 2 3 4 6 7
22841 240/300 Flush
22842 241/300 Read block 3326 len 20480
22843 242/300 Write block 4769 len 36864 data: 5 5 5 5 6 8 7 6 6
22844 243/300 Write block 1933 len 36864 data: 6 4 5 5 7 6 5 7 7
22845 244/300 Flush
22846 245/300 Write block 4153 len 36864 data: 3 4 5 4 4 4 4 6 7
22847 246/300 Read block 2137 len 40960
22848 247/300 Write block 3815 len 12288 data: 4 5 6
22849 248/300 Read block 1417 len 24576
22850 249/300 Read block 826 len 8192
22851 250/300 Write block 2013 len 8192 data: 6 6
22852 251/300 Write block 2750 len 32768 data: 3 3 4 3 3 5 5 5
22853 252/300 Write block 3671 len 4096 data: 5
22854 253/300 Read block 4798 len 20480
22855 254/300 Read block 1804 len 40960
22856 255/300 Write block 564 len 20480 data: 4 4 3 3 5
22857 256/300 Write block 1514 len 16384 data: 3 2 2 2
22858 257/300 Write block 4160 len 4096 data: 7
22859 258/300 Write block 870 len 20480 data: 4 4 3 3 3
22860 259/300 Write block 1400 len 20480 data: 5 5 5 5 8
22861 260/300 Write block 1212 len 24576 data: 6 5 6 5 5 5
22862 261/300 Read block 4611 len 20480
22863 262/300 Read block 3321 len 4096
22864 263/300 Read block 4096 len 28672
22865 264/300 Write block 4187 len 24576 data: 3 4 4 3 3 3
22866 265/300 Flush
22867 266/300 Write block 4448 len 36864 data: 7 7 7 8 8 9 9 11 8
22868 267/300 Flush
22869 268/300 Read block 1130 len 16384
22870 269/300 Read block 128 len 20480
22871 270/300 Write block 1548 len 16384 data: 5 5 5 5
22872 271/300 Read block 4484 len 24576
22873 272/300 Write block 3862 len 16384 data: 2 2 3 4
22874 273/300 Flush
22875 274/300 Read block 811 len 12288
22876 275/300 Read block 3330 len 24576
22877 276/300 Flush
22878 277/300 Read block 1097 len 20480
22879 278/300 Write block 1682 len 32768 data: 10 10 10 12 8 9 6 6
22880 279/300 Write block 1286 len 32768 data: 3 4 3 4 5 5 5 5
22881 280/300 Write block 794 len 36864 data: 6 5 4 3 2 2 4 3 3
22882 281/300 Write block 463 len 24576 data: 3 6 5 6 6 8
22883 282/300 Flush
22884 283/300 Read block 445 len 12288
22885 284/300 Read block 155 len 12288
22886 285/300 Flush
22887 286/300 Read block 2820 len 36864
22888 287/300 Read block 3364 len 28672
22889 288/300 Write block 4308 len 40960 data: 4 4 4 5 6 5 4 4 5 5
22890 289/300 Read block 485 len 12288
22891 290/300 Flush
22892 291/300 Write block 4980 len 12288 data: 6 5 4
22893 292/300 Write block 3767 len 24576 data: 3 2 3 3 3 5
22894 293/300 Write block 3050 len 28672 data: 5 5 5 4 3 2 2
22895 294/300 Read block 2417 len 8192
22896 295/300 Write block 4298 len 16384 data: 8 8 8 8
22897 296/300 Flush
22898 297/300 Flush
22899 298/300 Write block 4043 len 12288 data: 7 8 10
22900 299/300 Write block 2766 len 4096 data: 3
22901 300/300 Read block 4213 len 40960
22902 301/300 Write block 875 len 28672 data: 3 3 4 5 6 7 6
22903 Main task has work to do, go find it
22904 got dsc Start(1)
22905 start 1
22906 Jan 11 06:10:16.016 INFO request completed, latency_us: 371, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6a8612aa-8e33-4888-b5a2-a3885331845a, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
22907 Replay: started 1, returned:Ok(())
22908 ----------------------------------------------------------------
22909 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22910 States: Active Offline Active
22911 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22912 7340 Acked Flush 0 Done Sent Done false
22913 7341 Acked Write 8 Done Sent Done false
22914 7342 Acked Read 2 Done Sent Done false
22915 7343 Acked Read 5 Done Sent Done false
22916 7344 Acked Write 9 Done Sent Done false
22917 7345 Acked Write 8 Done Sent Done false
22918 7346 Acked Write 2 Done Sent Done false
22919 7347 Acked Write 2 Done Sent Done false
22920 7348 Acked Write 3 Done Sent Done false
22921 7349 Acked Write 3 Done Sent Done false
22922 7350 Acked Read 3 Done Sent Done false
22923 7351 Acked Flush 0 Done Sent Done false
22924 7352 Acked Read 7 Done Sent Done false
22925 7353 Acked Read 1 Done Sent Done false
22926 7354 Acked Read 1 Done Sent Done false
22927 7355 Acked Write 6 Done Sent Done false
22928 7356 Acked Flush 0 Done Sent Done false
22929 7357 Acked Write 2 Done Sent Done false
22930 7358 Acked Read 1 Done Sent Done false
22931 7359 Acked Read 8 Done Sent Done false
22932 7360 Acked Read 3 Done Sent Done false
22933 7361 Acked Flush 0 Done Sent Done false
22934 7362 Acked Read 1 Done Sent Done false
22935 7363 Acked Write 1 Done Sent Done false
22936 7364 Acked Write 3 Done Sent Done false
22937 7365 Acked Write 4 Done Sent Done false
22938 7366 Acked Write 6 Done Sent Done false
22939 7367 Acked Write 1 Done Sent Done false
22940 7368 Acked Write 6 Done Sent Done false
22941 7369 Acked Flush 0 Done Sent Done false
22942 7370 Acked Read 5 Done Sent Done false
22943 7371 Acked Read 7 Done Sent Done false
22944 7372 Acked Write 6 Done Sent Done false
22945 7373 Acked Write 4 Done Sent Done false
22946 7374 Acked Read 9 Done Sent Done false
22947 7375 Acked Read 7 Done Sent Done false
22948 7376 Acked Write 2 Done Sent Done false
22949 7377 Acked Read 4 Done Sent Done false
22950 7378 Acked Write 8 Done Sent Done false
22951 7379 Acked Write 1 Done Sent Done false
22952 7380 Acked Read 3 Done Sent Done false
22953 7381 Acked Flush 0 Done Sent Done false
22954 7382 Acked Write 7 Done Sent Done false
22955 7383 Acked Read 4 Done Sent Done false
22956 7384 Acked Read 2 Done Sent Done false
22957 7385 Acked Read 7 Done Sent Done false
22958 7386 Acked Write 1 Done Sent Done false
22959 7387 Acked Flush 0 Done Sent Done false
22960 7388 Acked Read 2 Done Sent Done false
22961 7389 Acked Write 10 Done Sent Done false
22962 7390 Acked Read 4 Done Sent Done false
22963 7391 Acked Flush 0 Done Sent Done false
22964 7392 Acked Write 1 Done Sent Done false
22965 7393 Acked Read 6 Done Sent Done false
22966 7394 Acked Flush 0 Done Sent Done false
22967 7395 Acked Read 9 Done Sent Done false
22968 7396 Acked Write 9 Done Sent Done false
22969 7397 Acked Read 4 Done Sent Done false
22970 7398 Acked Read 2 Done Sent Done false
22971 7399 Acked Read 8 Done Sent Done false
22972 7400 Acked Read 4 Done Sent Done false
22973 7401 Acked Read 6 Done Sent Done false
22974 7402 Acked Write 7 Done Sent Done false
22975 7403 Acked Write 4 Done Sent Done false
22976 7404 Acked Write 10 Done Sent Done false
22977 7405 Acked Read 6 Done Sent Done false
22978 7406 Acked Read 10 Done Sent Done false
22979 7407 Acked Read 9 Done Sent Done false
22980 7408 Acked Flush 0 Done Sent Done false
22981 7409 Acked Read 4 Done Sent Done false
22982 7410 Acked Read 3 Done Sent Done false
22983 7411 Acked Write 2 Done Sent Done false
22984 7412 Acked Read 5 Done Sent Done false
22985 7413 Acked Write 4 Done Sent Done false
22986 7414 Acked Read 10 Done Sent Done false
22987 7415 Acked Flush 0 Done Sent Done false
22988 7416 Acked Read 8 Done Sent Done false
22989 7417 Acked Write 9 Done Sent Done false
22990 7418 Acked Write 7 Done Sent Done false
22991 7419 Acked Write 10 Done Sent Done false
22992 7420 Acked Flush 0 Done Sent Done false
22993 7421 Acked Read 7 Done Sent Done false
22994 7422 Acked Read 2 Done Sent Done false
22995 7423 Acked Write 1 Done Sent Done false
22996 7424 Acked Write 5 Done Sent Done false
22997 7425 Acked Write 5 Done Sent Done false
22998 7426 Acked Write 1 Done Sent Done false
22999 7427 Acked Write 1 Done Sent Done false
23000 7428 Acked Read 9 Done Sent Done false
23001 7429 Acked Write 2 Done Sent Done false
23002 7430 Acked Write 3 Done Sent Done false
23003 7431 Acked Read 8 Done Sent Done false
23004 7432 Acked Write 7 Done Sent Done false
23005 7433 Acked Write 8 Done Sent Done false
23006 7434 Acked Flush 0 Done Sent Done false
23007 7435 Acked Write 1 Done Sent Done false
23008 7436 Acked Read 5 Done Sent Done false
23009 7437 Acked Read 1 Done Sent Done false
23010 7438 Acked Flush 0 Done Sent Done false
23011 7439 Acked Write 10 Done Sent Done false
23012 7440 Acked Write 7 Done Sent Done false
23013 7441 Acked Flush 0 Done Sent Done false
23014 7442 Acked Read 10 Done Sent Done false
23015 7443 Acked Write 6 Done Sent Done false
23016 7444 Acked Read 2 Done Sent Done false
23017 7445 Acked Read 4 Done Sent Done false
23018 7446 Acked Write 8 Done Sent Done false
23019 7447 Acked Read 3 Done Sent Done false
23020 7448 Acked Read 9 Done Sent Done false
23021 7449 Acked Write 3 Done Sent Done false
23022 7450 Acked Read 3 Done Sent Done false
23023 7451 Acked Write 5 Done Sent Done false
23024 7452 Acked Read 10 Done Sent Done false
23025 7453 Acked Write 2 Done Sent Done false
23026 7454 Acked Read 8 Done Sent Done false
23027 7455 Acked Write 9 Done Sent Done false
23028 7456 Acked Read 8 Done Sent Done false
23029 7457 Acked Write 1 Done Sent Done false
23030 7458 Acked Read 3 Done Sent Done false
23031 7459 Acked Read 2 Done Sent Done false
23032 7460 Acked Read 7 Done Sent Done false
23033 7461 Acked Read 1 Done Sent Done false
23034 7462 Acked Read 10 Done Sent Done false
23035 7463 Acked Read 8 Done Sent Done false
23036 7464 Acked Flush 0 Done Sent Done false
23037 7465 Acked Write 2 Done Sent Done false
23038 7466 Acked Write 9 Done Sent Done false
23039 7467 Acked Write 8 Done Sent Done false
23040 7468 Acked Write 5 Done Sent Done false
23041 7469 Acked Read 9 Done Sent Done false
23042 7470 Acked Read 5 Done Sent Done false
23043 7471 Acked Write 9 Done Sent Done false
23044 7472 Acked Write 10 Done Sent Done false
23045 7473 Acked Write 3 Done Sent Done false
23046 7474 Acked Write 10 Done Sent Done false
23047 7475 Acked Read 3 Done Sent Done false
23048 7476 Acked Flush 0 Done Sent Done false
23049 7477 Acked Write 4 Done Sent Done false
23050 7478 Acked Read 9 Done Sent Done false
23051 7479 Acked Read 6 Done Sent Done false
23052 7480 Acked Flush 0 Done Sent Done false
23053 7481 Acked Write 3 Done Sent Done false
23054 7482 Acked Flush 0 Done Sent Done false
23055 7483 Acked Read 9 Done Sent Done false
23056 7484 Acked Flush 0 Done Sent Done false
23057 7485 Acked Write 8 Done Sent Done false
23058 7486 Acked Read 5 Done Sent Done false
23059 7487 Acked Write 4 Done Sent Done false
23060 7488 Acked Read 8 Done Sent Done false
23061 7489 Acked Write 6 Done Sent Done false
23062 7490 Acked Read 6 Done Sent Done false
23063 7491 Acked Write 3 Done Sent Done false
23064 7492 Acked Read 5 Done Sent Done false
23065 7493 Acked Read 9 Done Sent Done false
23066 7494 Acked Write 3 Done Sent Done false
23067 7495 Acked Read 9 Done Sent Done false
23068 7496 Acked Read 3 Done Sent Done false
23069 7497 Acked Write 3 Done Sent Done false
23070 7498 Acked Read 5 Done Sent Done false
23071 7499 Acked Read 1 Done Sent Done false
23072 7500 Acked Write 1 Done Sent Done false
23073 7501 Acked Read 6 Done Sent Done false
23074 7502 Acked Read 10 Done Sent Done false
23075 7503 Acked Write 10 Done Sent Done false
23076 7504 Acked Write 9 Done Sent Done false
23077 7505 Acked Read 7 Done Sent Done false
23078 7506 Acked Read 8 Done Sent Done false
23079 7507 Acked Read 5 Done Sent Done false
23080 7508 Acked Write 9 Done Sent Done false
23081 7509 Acked Write 8 Done Sent Done false
23082 7510 Acked Read 10 Done Sent Done false
23083 7511 Acked Read 6 Done Sent Done false
23084 7512 Acked Read 6 Done Sent Done false
23085 7513 Acked Read 1 Done Sent Done false
23086 7514 Acked Read 6 Done Sent Done false
23087 7515 Acked Write 6 Done Sent Done false
23088 7516 Acked Read 6 Done Sent Done false
23089 7517 Acked Read 9 Done Sent Done false
23090 7518 Acked Write 2 Done Sent Done false
23091 7519 Acked Flush 0 Done Sent Done false
23092 7520 Acked Read 8 Done Sent Done false
23093 7521 Acked Write 6 Done Sent Done false
23094 7522 Acked Read 6 Done Sent Done false
23095 7523 Acked Write 8 Done Sent Done false
23096 7524 Acked Read 9 Done Sent Done false
23097 7525 Acked Read 7 Done Sent Done false
23098 7526 Acked Write 8 Done Sent Done false
23099 7527 Acked Read 6 Done Sent Done false
23100 7528 Acked Read 10 Done Sent Done false
23101 7529 Acked Read 1 Done Sent Done false
23102 7530 Acked Read 2 Done Sent Done false
23103 7531 Acked Flush 0 Done Sent Done false
23104 7532 Acked Read 4 Done Sent Done false
23105 7533 Acked Write 9 Done Sent Done false
23106 7534 Acked Write 8 Done Sent Done false
23107 7535 Acked Read 3 Done Sent Done false
23108 7536 Acked Write 8 Done Sent Done false
23109 7537 Acked Read 10 Done Sent Done false
23110 7538 Acked Write 5 Done Sent Done false
23111 7539 Acked Write 6 Done Sent Done false
23112 7540 Acked Read 2 Done Sent Done false
23113 7541 Acked Write 1 Done Sent Done false
23114 7542 Acked Write 3 Done Sent Done false
23115 7543 Acked Read 4 Done Sent Done false
23116 7544 Acked Read 3 Done Sent Done false
23117 7545 Acked Read 8 Done Sent Done false
23118 7546 Acked Flush 0 Done Sent Done false
23119 7547 Acked Write 3 Done Sent Done false
23120 7548 Acked Flush 0 Done Sent Done false
23121 7549 Acked Flush 0 Done Sent Done false
23122 7550 Acked Read 5 Done Sent Done false
23123 7551 Acked Read 7 Done Sent Done false
23124 7552 Acked Read 6 Done Sent Done false
23125 7553 Acked Read 10 Done Sent Done false
23126 7554 Acked Read 10 Done Sent Done false
23127 7555 Acked Write 3 Done Sent Done false
23128 7556 Acked Flush 0 Done Sent Done false
23129 7557 Acked Read 2 Done Sent Done false
23130 7558 Acked Read 9 Done Sent Done false
23131 7559 Acked Read 6 Done Sent Done false
23132 7560 Acked Write 1 Done Sent Done false
23133 7561 Acked Read 5 Done Sent Done false
23134 7562 Acked Read 1 Done Sent Done false
23135 7563 Acked Read 7 Done Sent Done false
23136 7564 Acked Write 4 Done Sent Done false
23137 7565 Acked Read 8 Done Sent Done false
23138 7566 Acked Flush 0 Done Sent Done false
23139 7567 Acked Write 6 Done Sent Done false
23140 7568 Acked Write 5 Done Sent Done false
23141 7569 Acked Read 9 Done Sent Done false
23142 7570 Acked Read 9 Done Sent Done false
23143 7571 Acked Write 6 Done Sent Done false
23144 7572 Acked Read 1 Done Sent Done false
23145 7573 Acked Flush 0 Done Sent Done false
23146 7574 Acked Read 3 Done Sent Done false
23147 7575 Acked Read 2 Done Sent Done false
23148 7576 Acked Write 10 Done Sent Done false
23149 7577 Acked Write 1 Done Sent Done false
23150 7578 Acked Write 7 Done Sent Done false
23151 7579 Acked Flush 0 Done Sent Done false
23152 7580 Acked Read 5 Done Sent Done false
23153 7581 Acked Write 9 Done Sent Done false
23154 7582 Acked Write 9 Done Sent Done false
23155 7583 Acked Flush 0 Done Sent Done false
23156 7584 Acked Write 9 Done Sent Done false
23157 7585 Acked Read 10 Done Sent Done false
23158 7586 Acked Write 3 Done Sent Done false
23159 7587 Acked Read 6 Done Sent Done false
23160 7588 Acked Read 2 Done Sent Done false
23161 7589 Acked Write 2 Done Sent Done false
23162 7590 Acked Write 8 Done Sent Done false
23163 7591 Acked Write 1 Done Sent Done false
23164 7592 Acked Read 5 Done Sent Done false
23165 7593 Acked Read 10 Done Sent Done false
23166 7594 Acked Write 5 Done Sent Done false
23167 7595 Acked Write 4 Done Sent Done false
23168 7596 Acked Write 1 Done Sent Done false
23169 7597 Acked Write 5 Done Sent Done false
23170 7598 Acked Write 5 Done Sent Done false
23171 7599 Acked Write 6 Done Sent Done false
23172 7600 Acked Read 5 Done Sent Done false
23173 7601 Acked Read 1 Done Sent Done false
23174 7602 Acked Read 7 Done Sent Done false
23175 7603 Acked Write 6 Done Sent Done false
23176 7604 Acked Flush 0 Done Sent Done false
23177 7605 Acked Write 9 Done Sent Done false
23178 7606 Acked Flush 0 Done Sent Done false
23179 7607 Acked Read 4 Done Sent Done false
23180 7608 Acked Read 5 Done Sent Done false
23181 7609 Acked Write 4 Done Sent Done false
23182 7610 Acked Read 6 Done Sent Done false
23183 7611 Acked Write 4 Done Sent Done false
23184 7612 Acked Flush 0 Done Sent Done false
23185 7613 Acked Read 3 Done Sent Done false
23186 7614 Acked Read 6 Done Sent Done false
23187 7615 Acked Flush 0 Done Sent Done false
23188 7616 Acked Read 5 Done Sent Done false
23189 7617 Acked Write 8 Done Sent Done false
23190 7618 Acked Write 8 Done Sent Done false
23191 7619 Acked Write 9 Done Sent Done false
23192 7620 Acked Write 6 Done Sent Done false
23193 7621 Acked Flush 0 Done Sent Done false
23194 7622 Acked Read 3 Done Sent Done false
23195 7623 Acked Read 3 Done Sent Done false
23196 7624 Acked Flush 0 Done Sent Done false
23197 7625 Acked Read 9 Done Sent Done false
23198 7626 Acked Read 7 Done Sent Done false
23199 7627 Acked Write 10 Done Sent Done false
23200 7628 Acked Read 3 Done Sent Done false
23201 7629 Acked Flush 0 Done Sent Done false
23202 7630 Acked Write 3 Done Sent Done false
23203 7631 Acked Write 6 Done Sent Done false
23204 7632 Acked Write 7 Done Sent Done false
23205 7633 Acked Read 2 Done Sent Done false
23206 7634 Acked Write 4 Done Sent Done false
23207 7635 Acked Flush 0 Done Sent Done false
23208 7636 Acked Flush 0 Done Sent Done false
23209 7637 Acked Write 3 Done Sent Done false
23210 7638 Acked Write 1 Done Sent Done false
23211 7639 Acked Read 10 Done Sent Done false
23212 7640 Acked Write 7 Done Sent Done false
23213 STATES DS:0 DS:1 DS:2 TOTAL
23214 Sent 0 301 0 301
23215 Done 301 0 301 602
23216 Skipped 0 0 0 0
23217 Error 0 0 0 0
23218 Last Flush: 7636 7339 7636
23219 Downstairs last five completed: 7339 7338 7337 7336 7335
23220 Upstairs last five completed: 7640 7639 7638 7637 7636
23221 CLIENT: Up:0 ds:301 act:2
23222 [8820] got start action
23223 [8820] I am going to restart
23224 Starting downstairs at port 8820
23225 Make output file at "/tmp/dsc/downstairs-8820.txt"
23226 [8820][1] reports Starting
23227 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1226)
23228 [8820][1] reports Running
23229 ----------------------------------------------------------------
23230 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23231 States: Active Offline Active
23232 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23233 7340 Acked Flush 0 Done Sent Done false
23234 7341 Acked Write 8 Done Sent Done false
23235 7342 Acked Read 2 Done Sent Done false
23236 7343 Acked Read 5 Done Sent Done false
23237 7344 Acked Write 9 Done Sent Done false
23238 7345 Acked Write 8 Done Sent Done false
23239 7346 Acked Write 2 Done Sent Done false
23240 7347 Acked Write 2 Done Sent Done false
23241 7348 Acked Write 3 Done Sent Done false
23242 7349 Acked Write 3 Done Sent Done false
23243 7350 Acked Read 3 Done Sent Done false
23244 7351 Acked Flush 0 Done Sent Done false
23245 7352 Acked Read 7 Done Sent Done false
23246 7353 Acked Read 1 Done Sent Done false
23247 7354 Acked Read 1 Done Sent Done false
23248 7355 Acked Write 6 Done Sent Done false
23249 7356 Acked Flush 0 Done Sent Done false
23250 7357 Acked Write 2 Done Sent Done false
23251 7358 Acked Read 1 Done Sent Done false
23252 7359 Acked Read 8 Done Sent Done false
23253 7360 Acked Read 3 Done Sent Done false
23254 7361 Acked Flush 0 Done Sent Done false
23255 7362 Acked Read 1 Done Sent Done false
23256 7363 Acked Write 1 Done Sent Done false
23257 7364 Acked Write 3 Done Sent Done false
23258 7365 Acked Write 4 Done Sent Done false
23259 7366 Acked Write 6 Done Sent Done false
23260 7367 Acked Write 1 Done Sent Done false
23261 7368 Acked Write 6 Done Sent Done false
23262 7369 Acked Flush 0 Done Sent Done false
23263 7370 Acked Read 5 Done Sent Done false
23264 7371 Acked Read 7 Done Sent Done false
23265 7372 Acked Write 6 Done Sent Done false
23266 7373 Acked Write 4 Done Sent Done false
23267 7374 Acked Read 9 Done Sent Done false
23268 7375 Acked Read 7 Done Sent Done false
23269 7376 Acked Write 2 Done Sent Done false
23270 7377 Acked Read 4 Done Sent Done false
23271 7378 Acked Write 8 Done Sent Done false
23272 7379 Acked Write 1 Done Sent Done false
23273 7380 Acked Read 3 Done Sent Done false
23274 7381 Acked Flush 0 Done Sent Done false
23275 7382 Acked Write 7 Done Sent Done false
23276 7383 Acked Read 4 Done Sent Done false
23277 7384 Acked Read 2 Done Sent Done false
23278 7385 Acked Read 7 Done Sent Done false
23279 7386 Acked Write 1 Done Sent Done false
23280 7387 Acked Flush 0 Done Sent Done false
23281 7388 Acked Read 2 Done Sent Done false
23282 7389 Acked Write 10 Done Sent Done false
23283 7390 Acked Read 4 Done Sent Done false
23284 7391 Acked Flush 0 Done Sent Done false
23285 7392 Acked Write 1 Done Sent Done false
23286 7393 Acked Read 6 Done Sent Done false
23287 7394 Acked Flush 0 Done Sent Done false
23288 7395 Acked Read 9 Done Sent Done false
23289 7396 Acked Write 9 Done Sent Done false
23290 7397 Acked Read 4 Done Sent Done false
23291 7398 Acked Read 2 Done Sent Done false
23292 7399 Acked Read 8 Done Sent Done false
23293 7400 Acked Read 4 Done Sent Done false
23294 7401 Acked Read 6 Done Sent Done false
23295 7402 Acked Write 7 Done Sent Done false
23296 7403 Acked Write 4 Done Sent Done false
23297 7404 Acked Write 10 Done Sent Done false
23298 7405 Acked Read 6 Done Sent Done false
23299 7406 Acked Read 10 Done Sent Done false
23300 7407 Acked Read 9 Done Sent Done false
23301 7408 Acked Flush 0 Done Sent Done false
23302 7409 Acked Read 4 Done Sent Done false
23303 7410 Acked Read 3 Done Sent Done false
23304 7411 Acked Write 2 Done Sent Done false
23305 7412 Acked Read 5 Done Sent Done false
23306 7413 Acked Write 4 Done Sent Done false
23307 7414 Acked Read 10 Done Sent Done false
23308 7415 Acked Flush 0 Done Sent Done false
23309 7416 Acked Read 8 Done Sent Done false
23310 7417 Acked Write 9 Done Sent Done false
23311 7418 Acked Write 7 Done Sent Done false
23312 7419 Acked Write 10 Done Sent Done false
23313 7420 Acked Flush 0 Done Sent Done false
23314 7421 Acked Read 7 Done Sent Done false
23315 7422 Acked Read 2 Done Sent Done false
23316 7423 Acked Write 1 Done Sent Done false
23317 7424 Acked Write 5 Done Sent Done false
23318 7425 Acked Write 5 Done Sent Done false
23319 7426 Acked Write 1 Done Sent Done false
23320 7427 Acked Write 1 Done Sent Done false
23321 7428 Acked Read 9 Done Sent Done false
23322 7429 Acked Write 2 Done Sent Done false
23323 7430 Acked Write 3 Done Sent Done false
23324 7431 Acked Read 8 Done Sent Done false
23325 7432 Acked Write 7 Done Sent Done false
23326 7433 Acked Write 8 Done Sent Done false
23327 7434 Acked Flush 0 Done Sent Done false
23328 7435 Acked Write 1 Done Sent Done false
23329 7436 Acked Read 5 Done Sent Done false
23330 7437 Acked Read 1 Done Sent Done false
23331 7438 Acked Flush 0 Done Sent Done false
23332 7439 Acked Write 10 Done Sent Done false
23333 7440 Acked Write 7 Done Sent Done false
23334 7441 Acked Flush 0 Done Sent Done false
23335 7442 Acked Read 10 Done Sent Done false
23336 7443 Acked Write 6 Done Sent Done false
23337 7444 Acked Read 2 Done Sent Done false
23338 7445 Acked Read 4 Done Sent Done false
23339 7446 Acked Write 8 Done Sent Done false
23340 7447 Acked Read 3 Done Sent Done false
23341 7448 Acked Read 9 Done Sent Done false
23342 7449 Acked Write 3 Done Sent Done false
23343 7450 Acked Read 3 Done Sent Done false
23344 7451 Acked Write 5 Done Sent Done false
23345 7452 Acked Read 10 Done Sent Done false
23346 7453 Acked Write 2 Done Sent Done false
23347 7454 Acked Read 8 Done Sent Done false
23348 7455 Acked Write 9 Done Sent Done false
23349 7456 Acked Read 8 Done Sent Done false
23350 7457 Acked Write 1 Done Sent Done false
23351 7458 Acked Read 3 Done Sent Done false
23352 7459 Acked Read 2 Done Sent Done false
23353 7460 Acked Read 7 Done Sent Done false
23354 7461 Acked Read 1 Done Sent Done false
23355 7462 Acked Read 10 Done Sent Done false
23356 7463 Acked Read 8 Done Sent Done false
23357 7464 Acked Flush 0 Done Sent Done false
23358 7465 Acked Write 2 Done Sent Done false
23359 7466 Acked Write 9 Done Sent Done false
23360 7467 Acked Write 8 Done Sent Done false
23361 7468 Acked Write 5 Done Sent Done false
23362 7469 Acked Read 9 Done Sent Done false
23363 7470 Acked Read 5 Done Sent Done false
23364 7471 Acked Write 9 Done Sent Done false
23365 7472 Acked Write 10 Done Sent Done false
23366 7473 Acked Write 3 Done Sent Done false
23367 7474 Acked Write 10 Done Sent Done false
23368 7475 Acked Read 3 Done Sent Done false
23369 7476 Acked Flush 0 Done Sent Done false
23370 7477 Acked Write 4 Done Sent Done false
23371 7478 Acked Read 9 Done Sent Done false
23372 7479 Acked Read 6 Done Sent Done false
23373 7480 Acked Flush 0 Done Sent Done false
23374 7481 Acked Write 3 Done Sent Done false
23375 7482 Acked Flush 0 Done Sent Done false
23376 7483 Acked Read 9 Done Sent Done false
23377 7484 Acked Flush 0 Done Sent Done false
23378 7485 Acked Write 8 Done Sent Done false
23379 7486 Acked Read 5 Done Sent Done false
23380 7487 Acked Write 4 Done Sent Done false
23381 7488 Acked Read 8 Done Sent Done false
23382 7489 Acked Write 6 Done Sent Done false
23383 7490 Acked Read 6 Done Sent Done false
23384 7491 Acked Write 3 Done Sent Done false
23385 7492 Acked Read 5 Done Sent Done false
23386 7493 Acked Read 9 Done Sent Done false
23387 7494 Acked Write 3 Done Sent Done false
23388 7495 Acked Read 9 Done Sent Done false
23389 7496 Acked Read 3 Done Sent Done false
23390 7497 Acked Write 3 Done Sent Done false
23391 7498 Acked Read 5 Done Sent Done false
23392 7499 Acked Read 1 Done Sent Done false
23393 7500 Acked Write 1 Done Sent Done false
23394 7501 Acked Read 6 Done Sent Done false
23395 7502 Acked Read 10 Done Sent Done false
23396 7503 Acked Write 10 Done Sent Done false
23397 7504 Acked Write 9 Done Sent Done false
23398 7505 Acked Read 7 Done Sent Done false
23399 7506 Acked Read 8 Done Sent Done false
23400 7507 Acked Read 5 Done Sent Done false
23401 7508 Acked Write 9 Done Sent Done false
23402 7509 Acked Write 8 Done Sent Done false
23403 7510 Acked Read 10 Done Sent Done false
23404 7511 Acked Read 6 Done Sent Done false
23405 7512 Acked Read 6 Done Sent Done false
23406 7513 Acked Read 1 Done Sent Done false
23407 7514 Acked Read 6 Done Sent Done false
23408 7515 Acked Write 6 Done Sent Done false
23409 7516 Acked Read 6 Done Sent Done false
23410 7517 Acked Read 9 Done Sent Done false
23411 7518 Acked Write 2 Done Sent Done false
23412 7519 Acked Flush 0 Done Sent Done false
23413 7520 Acked Read 8 Done Sent Done false
23414 7521 Acked Write 6 Done Sent Done false
23415 7522 Acked Read 6 Done Sent Done false
23416 7523 Acked Write 8 Done Sent Done false
23417 7524 Acked Read 9 Done Sent Done false
23418 7525 Acked Read 7 Done Sent Done false
23419 7526 Acked Write 8 Done Sent Done false
23420 7527 Acked Read 6 Done Sent Done false
23421 7528 Acked Read 10 Done Sent Done false
23422 7529 Acked Read 1 Done Sent Done false
23423 7530 Acked Read 2 Done Sent Done false
23424 7531 Acked Flush 0 Done Sent Done false
23425 7532 Acked Read 4 Done Sent Done false
23426 7533 Acked Write 9 Done Sent Done false
23427 7534 Acked Write 8 Done Sent Done false
23428 7535 Acked Read 3 Done Sent Done false
23429 7536 Acked Write 8 Done Sent Done false
23430 7537 Acked Read 10 Done Sent Done false
23431 7538 Acked Write 5 Done Sent Done false
23432 7539 Acked Write 6 Done Sent Done false
23433 7540 Acked Read 2 Done Sent Done false
23434 7541 Acked Write 1 Done Sent Done false
23435 7542 Acked Write 3 Done Sent Done false
23436 7543 Acked Read 4 Done Sent Done false
23437 7544 Acked Read 3 Done Sent Done false
23438 7545 Acked Read 8 Done Sent Done false
23439 7546 Acked Flush 0 Done Sent Done false
23440 7547 Acked Write 3 Done Sent Done false
23441 7548 Acked Flush 0 Done Sent Done false
23442 7549 Acked Flush 0 Done Sent Done false
23443 7550 Acked Read 5 Done Sent Done false
23444 7551 Acked Read 7 Done Sent Done false
23445 7552 Acked Read 6 Done Sent Done false
23446 7553 Acked Read 10 Done Sent Done false
23447 7554 Acked Read 10 Done Sent Done false
23448 7555 Acked Write 3 Done Sent Done false
23449 7556 Acked Flush 0 Done Sent Done false
23450 7557 Acked Read 2 Done Sent Done false
23451 7558 Acked Read 9 Done Sent Done false
23452 7559 Acked Read 6 Done Sent Done false
23453 7560 Acked Write 1 Done Sent Done false
23454 7561 Acked Read 5 Done Sent Done false
23455 7562 Acked Read 1 Done Sent Done false
23456 7563 Acked Read 7 Done Sent Done false
23457 7564 Acked Write 4 Done Sent Done false
23458 7565 Acked Read 8 Done Sent Done false
23459 7566 Acked Flush 0 Done Sent Done false
23460 7567 Acked Write 6 Done Sent Done false
23461 7568 Acked Write 5 Done Sent Done false
23462 7569 Acked Read 9 Done Sent Done false
23463 7570 Acked Read 9 Done Sent Done false
23464 7571 Acked Write 6 Done Sent Done false
23465 7572 Acked Read 1 Done Sent Done false
23466 7573 Acked Flush 0 Done Sent Done false
23467 7574 Acked Read 3 Done Sent Done false
23468 7575 Acked Read 2 Done Sent Done false
23469 7576 Acked Write 10 Done Sent Done false
23470 7577 Acked Write 1 Done Sent Done false
23471 7578 Acked Write 7 Done Sent Done false
23472 7579 Acked Flush 0 Done Sent Done false
23473 7580 Acked Read 5 Done Sent Done false
23474 7581 Acked Write 9 Done Sent Done false
23475 7582 Acked Write 9 Done Sent Done false
23476 7583 Acked Flush 0 Done Sent Done false
23477 7584 Acked Write 9 Done Sent Done false
23478 7585 Acked Read 10 Done Sent Done false
23479 7586 Acked Write 3 Done Sent Done false
23480 7587 Acked Read 6 Done Sent Done false
23481 7588 Acked Read 2 Done Sent Done false
23482 7589 Acked Write 2 Done Sent Done false
23483 7590 Acked Write 8 Done Sent Done false
23484 7591 Acked Write 1 Done Sent Done false
23485 7592 Acked Read 5 Done Sent Done false
23486 7593 Acked Read 10 Done Sent Done false
23487 7594 Acked Write 5 Done Sent Done false
23488 7595 Acked Write 4 Done Sent Done false
23489 7596 Acked Write 1 Done Sent Done false
23490 7597 Acked Write 5 Done Sent Done false
23491 7598 Acked Write 5 Done Sent Done false
23492 7599 Acked Write 6 Done Sent Done false
23493 7600 Acked Read 5 Done Sent Done false
23494 7601 Acked Read 1 Done Sent Done false
23495 7602 Acked Read 7 Done Sent Done false
23496 7603 Acked Write 6 Done Sent Done false
23497 7604 Acked Flush 0 Done Sent Done false
23498 7605 Acked Write 9 Done Sent Done false
23499 7606 Acked Flush 0 Done Sent Done false
23500 7607 Acked Read 4 Done Sent Done false
23501 7608 Acked Read 5 Done Sent Done false
23502 7609 Acked Write 4 Done Sent Done false
23503 7610 Acked Read 6 Done Sent Done false
23504 7611 Acked Write 4 Done Sent Done false
23505 7612 Acked Flush 0 Done Sent Done false
23506 7613 Acked Read 3 Done Sent Done false
23507 7614 Acked Read 6 Done Sent Done false
23508 7615 Acked Flush 0 Done Sent Done false
23509 7616 Acked Read 5 Done Sent Done false
23510 7617 Acked Write 8 Done Sent Done false
23511 7618 Acked Write 8 Done Sent Done false
23512 7619 Acked Write 9 Done Sent Done false
23513 7620 Acked Write 6 Done Sent Done false
23514 7621 Acked Flush 0 Done Sent Done false
23515 7622 Acked Read 3 Done Sent Done false
23516 7623 Acked Read 3 Done Sent Done false
23517 7624 Acked Flush 0 Done Sent Done false
23518 7625 Acked Read 9 Done Sent Done false
23519 7626 Acked Read 7 Done Sent Done false
23520 7627 Acked Write 10 Done Sent Done false
23521 7628 Acked Read 3 Done Sent Done false
23522 7629 Acked Flush 0 Done Sent Done false
23523 7630 Acked Write 3 Done Sent Done false
23524 7631 Acked Write 6 Done Sent Done false
23525 7632 Acked Write 7 Done Sent Done false
23526 7633 Acked Read 2 Done Sent Done false
23527 7634 Acked Write 4 Done Sent Done false
23528 7635 Acked Flush 0 Done Sent Done false
23529 7636 Acked Flush 0 Done Sent Done false
23530 7637 Acked Write 3 Done Sent Done false
23531 7638 Acked Write 1 Done Sent Done false
23532 7639 Acked Read 10 Done Sent Done false
23533 7640 Acked Write 7 Done Sent Done false
23534 7641 Acked Flush 0 Done Sent Done false
23535 STATES DS:0 DS:1 DS:2 TOTAL
23536 Sent 0 302 0 302
23537 Done 302 0 302 604
23538 Skipped 0 0 0 0
23539 Error 0 0 0 0
23540 Last Flush: 7641 7339 7641
23541 Downstairs last five completed: 7339 7338 7337 7336 7335
23542 Upstairs last five completed: 7641 7640 7639 7638 7637
23543 CLIENT: Up:0 ds:302 act:2
235442025-01-11T06:10:21.782ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235452025-01-11T06:10:21.783ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47414) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235462025-01-11T06:10:21.784ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235472025-01-11T06:10:21.785ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235482025-01-11T06:10:21.785ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235492025-01-11T06:10:21.785ZINFOcrucible: send last flush ID to this DS: 7339 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235502025-01-11T06:10:21.785ZINFOcrucible: Replied this last flush ID: 7339 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235512025-01-11T06:10:21.785ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235522025-01-11T06:10:21.785ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235532025-01-11T06:10:21.785ZINFOcrucible: [1] client re-new 302 jobs since flush 7339 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235542025-01-11T06:10:21.785ZINFOcrucible: [1] Replayed 302 jobs since flush: 7339 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
23555 ----------------------------------------------------------------
23556 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23557 Downstairs last five completed: 7641 7640 7639 7638 7637
23558 Upstairs last five completed: 7641 7640 7639 7638 7637
23559 CLIENT: Up:0 ds:0 act:3
23560 Replay: All jobs finished
23561 Main task has work to do, go find it
23562 got dsc Stop(2)
23563 stop 2
23564 [8830] Got stop action so:false kr:false
23565 Jan 11 06:10:24.021 INFO request completed, latency_us: 326, response_code: 204, uri: /stop/cid/2, method: POST, req_id: bd22cb05-1b10-4b39-8308-c663ff0d6c75, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
23566 Jan 11 06:10:24.022 INFO request completed, latency_us: 161, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4920d41c-0ae3-4c13-bf7b-419796392ce9, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
235672025-01-11T06:10:24.022ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235682025-01-11T06:10:24.022ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235692025-01-11T06:10:24.022ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235702025-01-11T06:10:24.022ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235712025-01-11T06:10:24.022ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
235722025-01-11T06:10:24.022ZINFOcrucible: client task is exiting = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
23573 [8830] Exited with: ExitStatus(unix_wait_status(9))
23574 [8830][2] reports Exit
23575 Jan 11 06:10:28.024 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/2, method: GET, req_id: 57207905-228c-454d-ae5c-aae556e0ebc9, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
23576 001/300 Read block 4440 len 24576
23577 002/300 Write block 1143 len 40960 data: 4 6 7 6 5 6 7 7 7 5
23578 003/300 Write block 3377 len 40960 data: 4 5 5 5 6 6 5 4 3 3
23579 004/300 Read block 2231 len 36864
23580 005/300 Read block 4960 len 20480
23581 006/300 Read block 1683 len 8192
23582 007/300 Read block 2662 len 28672
23583 008/300 Write block 4943 len 28672 data: 6 5 5 4 4 6 6
23584 009/300 Flush
23585 010/300 Flush
23586 011/300 Read block 3016 len 8192
23587 012/300 Write block 4682 len 24576 data: 4 6 6 6 7 6
23588 013/300 Read block 130 len 28672
23589 014/300 Read block 3321 len 4096
23590 015/300 Read block 3915 len 8192
23591 016/300 Read block 1899 len 16384
23592 017/300 Read block 4576 len 40960
23593 018/300 Write block 1827 len 4096 data: 7
23594 019/300 Read block 3954 len 40960
23595 020/300 Write block 1532 len 8192 data: 8 8
23596 021/300 Flush
23597 022/300 Read block 608 len 24576
23598 023/300 Read block 3277 len 24576
23599 024/300 Write block 2807 len 24576 data: 3 4 6 5 4 4
23600 025/300 Write block 3217 len 8192 data: 3 5
23601 026/300 Flush
23602 027/300 Read block 368 len 20480
23603 028/300 Write block 4287 len 32768 data: 4 5 5 5 4 4 4 4
23604 029/300 Write block 809 len 28672 data: 4 4 4 4 4 5 4
23605 030/300 Write block 2319 len 24576 data: 3 3 3 3 3 4
23606 031/300 Read block 3665 len 12288
23607 032/300 Read block 4568 len 4096
23608 033/300 Write block 1242 len 8192 data: 5 5
23609 034/300 Write block 4786 len 40960 data: 5 4 4 4 5 4 6 6 6 6
23610 035/300 Write block 4872 len 28672 data: 4 3 4 3 4 4 4
23611 036/300 Write block 4270 len 24576 data: 8 9 9 8 6 5
23612 037/300 Write block 4875 len 28672 data: 4 5 5 5 4 4 5
23613 038/300 Read block 2011 len 32768
23614 039/300 Read block 3687 len 40960
23615 040/300 Write block 3107 len 4096 data: 6
23616 041/300 Read block 2641 len 32768
23617 042/300 Read block 21 len 40960
23618 043/300 Read block 1249 len 40960
23619 044/300 Write block 4895 len 12288 data: 7 6 4
23620 045/300 Read block 1961 len 32768
23621 046/300 Write block 255 len 40960 data: 4 4 4 3 3 3 3 3 3 2
23622 047/300 Read block 982 len 8192
23623 048/300 Read block 2247 len 20480
23624 049/300 Write block 3752 len 20480 data: 6 4 4 5 5
23625 050/300 Read block 1893 len 20480
23626 051/300 Read block 2884 len 16384
23627 052/300 Write block 4225 len 16384 data: 3 3 2 2
23628 053/300 Read block 231 len 28672
23629 054/300 Write block 1233 len 12288 data: 6 6 4
23630 055/300 Flush
23631 056/300 Read block 525 len 4096
23632 057/300 Write block 4279 len 36864 data: 3 2 3 3 3 3 5 5 5
23633 058/300 Write block 2478 len 40960 data: 5 7 7 4 4 4 4 3 3 3
23634 059/300 Read block 2378 len 28672
23635 060/300 Write block 4219 len 40960 data: 5 5 4 4 4 4 4 4 3 3
23636 061/300 Read block 462 len 32768
23637 062/300 Read block 4497 len 8192
23638 063/300 Read block 1499 len 20480
23639 064/300 Write block 939 len 32768 data: 6 7 7 8 5 5 3 4
23640 065/300 Write block 4966 len 12288 data: 5 5 5
23641 066/300 Write block 4431 len 32768 data: 4 3 3 3 4 5 4 2
23642 067/300 Write block 1791 len 40960 data: 3 4 4 4 4 4 5 4 4 4
23643 068/300 Write block 979 len 16384 data: 4 5 5 4
23644 069/300 Read block 831 len 28672
23645 070/300 Read block 1223 len 32768
23646 071/300 Flush
23647 072/300 Read block 1556 len 8192
23648 073/300 Read block 681 len 20480
23649 074/300 Flush
23650 075/300 Flush
23651 076/300 Read block 3847 len 4096
23652 077/300 Flush
23653 078/300 Read block 3999 len 8192
23654 079/300 Read block 2047 len 32768
23655 080/300 Write block 2717 len 40960 data: 6 6 6 6 6 6 6 6 3 2
23656 081/300 Write block 351 len 4096 data: 3
23657 082/300 Read block 3362 len 8192
23658 083/300 Read block 123 len 24576
23659 084/300 Read block 4534 len 16384
23660 085/300 Read block 4617 len 40960
23661 086/300 Write block 243 len 36864 data: 6 5 5 5 6 6 5 6 6
23662 087/300 Flush
23663 088/300 Write block 2514 len 40960 data: 4 4 5 4 5 5 7 5 7 6
23664 089/300 Write block 2792 len 32768 data: 3 3 4 4 5 5 6 6
23665 090/300 Read block 410 len 4096
23666 091/300 Read block 48 len 20480
23667 092/300 Read block 2540 len 4096
23668 093/300 Flush
23669 094/300 Write block 4179 len 32768 data: 3 2 3 3 4 4 3 3
23670 095/300 Write block 517 len 28672 data: 7 8 7 6 4 4 4
23671 096/300 Read block 2997 len 4096
23672 097/300 Read block 1103 len 36864
23673 098/300 Write block 3864 len 20480 data: 4 5 4 5 5
23674 099/300 Read block 2287 len 28672
23675 100/300 Read block 2953 len 20480
23676 101/300 Read block 1587 len 12288
23677 102/300 Write block 2217 len 16384 data: 6 6 7 8
23678 103/300 Read block 2457 len 20480
23679 104/300 Read block 3986 len 36864
23680 105/300 Read block 3182 len 28672
23681 106/300 Write block 3987 len 8192 data: 2 2
23682 107/300 Read block 4118 len 4096
23683 108/300 Write block 2714 len 12288 data: 3 3 5
23684 109/300 Write block 4068 len 16384 data: 5 6 7 8
23685 110/300 Read block 2619 len 28672
23686 111/300 Write block 806 len 4096 data: 4
23687 112/300 Write block 339 len 8192 data: 7 6
23688 113/300 Read block 2640 len 28672
23689 114/300 Read block 2687 len 40960
23690 115/300 Read block 1558 len 24576
23691 116/300 Read block 2203 len 16384
23692 117/300 Flush
23693 118/300 Flush
23694 119/300 Read block 2713 len 12288
23695 120/300 Write block 309 len 32768 data: 4 4 4 5 3 3 3 4
23696 121/300 Read block 4686 len 4096
23697 122/300 Flush
23698 123/300 Write block 4633 len 28672 data: 7 7 8 7 6 4 3
23699 124/300 Read block 4038 len 20480
23700 125/300 Write block 3416 len 20480 data: 3 3 4 3 3
23701 126/300 Write block 195 len 12288 data: 8 7 7
23702 127/300 Read block 4590 len 32768
23703 128/300 Flush
23704 129/300 Write block 4732 len 36864 data: 3 4 4 4 4 3 4 4 5
23705 130/300 Write block 4202 len 32768 data: 4 4 4 5 5 5 7 6
23706 131/300 Read block 4745 len 16384
23707 132/300 Write block 3376 len 4096 data: 4
23708 133/300 Read block 4243 len 24576
23709 134/300 Read block 1071 len 28672
23710 135/300 Flush
23711 136/300 Flush
23712 137/300 Flush
23713 138/300 Read block 1338 len 28672
23714 139/300 Read block 2407 len 40960
23715 140/300 Write block 408 len 24576 data: 5 5 5 5 5 4
23716 141/300 Read block 3971 len 16384
23717 142/300 Write block 4764 len 40960 data: 5 4 5 6 5 6 6 6 6 7
23718 143/300 Read block 4165 len 16384
23719 144/300 Read block 1218 len 12288
23720 145/300 Read block 274 len 32768
23721 146/300 Write block 582 len 32768 data: 5 5 3 3 5 3 4 4
23722 147/300 Write block 3061 len 16384 data: 5 5 4 4
23723 148/300 Read block 87 len 40960
23724 149/300 Read block 1804 len 24576
23725 150/300 Write block 3433 len 36864 data: 7 7 5 4 4 2 2 2 2
23726 151/300 Write block 4748 len 4096 data: 6
23727 152/300 Flush
23728 153/300 Read block 974 len 36864
23729 154/300 Read block 1340 len 4096
23730 155/300 Read block 1008 len 16384
23731 156/300 Read block 1730 len 4096
23732 157/300 Write block 3317 len 16384 data: 5 4 4 4
23733 158/300 Write block 441 len 8192 data: 4 4
23734 159/300 Flush
23735 160/300 Write block 1017 len 24576 data: 4 3 2 2 3 4
23736 161/300 Read block 3627 len 4096
23737 162/300 Write block 4984 len 24576 data: 7 7 8 8 8 8
23738 163/300 Write block 2766 len 28672 data: 4 4 4 4 5 6 8
23739 164/300 Write block 2924 len 36864 data: 6 5 4 5 7 7 7 9 9
23740 165/300 Read block 3498 len 4096
23741 166/300 Read block 1953 len 24576
23742 167/300 Read block 496 len 28672
23743 168/300 Read block 4660 len 28672
23744 169/300 Write block 2092 len 32768 data: 7 7 6 5 7 9 9 10
23745 170/300 Flush
23746 171/300 Read block 3402 len 28672
23747 172/300 Write block 546 len 8192 data: 6 5
23748 173/300 Flush
23749 174/300 Read block 4480 len 32768
23750 175/300 Read block 3528 len 12288
23751 176/300 Write block 4436 len 16384 data: 6 5 3 2
23752 177/300 Read block 4622 len 12288
23753 178/300 Read block 2596 len 20480
23754 179/300 Write block 1331 len 36864 data: 8 7 5 5 3 3 3 3 3
23755 180/300 Write block 2782 len 28672 data: 5 5 6 8 8 7 4
23756 181/300 Write block 2907 len 32768 data: 8 7 5 4 4 5 5 6
23757 182/300 Read block 2570 len 28672
23758 183/300 Write block 1755 len 12288 data: 8 7 7
23759 184/300 Write block 226 len 40960 data: 5 7 6 6 6 4 4 5 5 5
23760 185/300 Flush
23761 186/300 Write block 921 len 32768 data: 3 3 3 3 2 2 2 3
23762 187/300 Write block 1668 len 12288 data: 5 7 7
23763 188/300 Read block 2497 len 24576
23764 189/300 Flush
23765 190/300 Read block 3005 len 12288
23766 191/300 Read block 3489 len 8192
23767 192/300 Write block 3900 len 8192 data: 3 3
23768 193/300 Read block 4631 len 40960
23769 194/300 Flush
23770 195/300 Read block 72 len 36864
23771 196/300 Read block 2352 len 8192
23772 197/300 Write block 90 len 16384 data: 5 5 5 6
23773 198/300 Write block 960 len 32768 data: 4 4 3 3 3 5 4 4
23774 199/300 Read block 1114 len 32768
23775 200/300 Read block 1396 len 32768
23776 201/300 Write block 94 len 32768 data: 9 10 8 8 8 7 6 6
23777 202/300 Write block 2719 len 36864 data: 7 7 7 7 7 7 4 3 3
23778 203/300 Write block 4088 len 28672 data: 8 8 7 6 6 4 3
23779 204/300 Read block 4581 len 20480
23780 205/300 Write block 522 len 20480 data: 5 5 3 3 3
23781 206/300 Flush
23782 207/300 Write block 2101 len 8192 data: 7 8
23783 208/300 Read block 3286 len 32768
23784 209/300 Write block 2750 len 32768 data: 4 4 5 4 4 6 6 6
23785 210/300 Flush
23786 211/300 Write block 2677 len 16384 data: 5 4 4 4
23787 212/300 Write block 2751 len 36864 data: 5 6 5 5 7 7 7 5 5
23788 213/300 Read block 2134 len 24576
23789 214/300 Read block 4602 len 32768
23790 215/300 Flush
23791 216/300 Write block 2707 len 32768 data: 5 5 4 5 6 5 5 4
23792 217/300 Write block 3354 len 24576 data: 2 2 2 2 2 2
23793 218/300 Flush
23794 219/300 Write block 894 len 4096 data: 2
23795 220/300 Write block 1847 len 8192 data: 9 9
23796 221/300 Write block 4137 len 4096 data: 7
23797 222/300 Flush
23798 223/300 Write block 1593 len 20480 data: 3 3 3 3 3
23799 224/300 Read block 31 len 16384
23800 225/300 Read block 4795 len 40960
23801 226/300 Write block 104 len 40960 data: 5 5 7 6 6 6 6 5 4 4
23802 227/300 Read block 827 len 24576
23803 228/300 Write block 3062 len 8192 data: 6 5
23804 229/300 Read block 4638 len 40960
23805 230/300 Read block 2800 len 24576
23806 231/300 Flush
23807 232/300 Read block 651 len 28672
23808 233/300 Flush
23809 234/300 Write block 1427 len 8192 data: 6 7
23810 235/300 Read block 2787 len 36864
23811 236/300 Flush
23812 237/300 Read block 3166 len 28672
23813 238/300 Read block 945 len 4096
23814 239/300 Write block 1669 len 36864 data: 8 8 7 6 5 4 4 4 5
23815 240/300 Write block 2300 len 16384 data: 7 9 8 6
23816 241/300 Write block 1008 len 4096 data: 7
23817 242/300 Read block 147 len 12288
23818 243/300 Write block 686 len 28672 data: 6 7 8 9 9 8 10
23819 244/300 Read block 1258 len 36864
23820 245/300 Write block 104 len 16384 data: 6 6 8 7
23821 246/300 Read block 1046 len 24576
23822 247/300 Read block 2969 len 28672
23823 248/300 Write block 2518 len 20480 data: 6 6 8 6 8
23824 249/300 Flush
23825 250/300 Read block 2479 len 20480
23826 251/300 Write block 593 len 28672 data: 5 6 7 8 6 6 6
23827 252/300 Write block 2366 len 40960 data: 6 6 6 4 5 4 4 5 5 6
23828 253/300 Flush
23829 254/300 Write block 1515 len 24576 data: 3 3 3 3 3 3
23830 255/300 Read block 2658 len 28672
23831 256/300 Write block 4250 len 40960 data: 4 4 4 4 4 4 5 5 7 7
23832 257/300 Write block 1603 len 16384 data: 2 3 3 2
23833 258/300 Read block 900 len 16384
23834 259/300 Write block 528 len 28672 data: 6 5 7 5 6 7 4
23835 260/300 Read block 1632 len 12288
23836 261/300 Read block 3091 len 12288
23837 262/300 Read block 1247 len 8192
23838 263/300 Read block 346 len 28672
23839 264/300 Write block 1393 len 12288 data: 4 4 4
23840 265/300 Read block 927 len 16384
23841 266/300 Write block 858 len 24576 data: 3 4 3 5 5 5
23842 267/300 Write block 4086 len 8192 data: 6 7
23843 268/300 Write block 2551 len 16384 data: 5 5 5 5
23844 269/300 Read block 284 len 28672
23845 270/300 Write block 4675 len 28672 data: 5 4 3 4 4 4 4
23846 271/300 Write block 2873 len 8192 data: 8 5
23847 272/300 Read block 19 len 32768
23848 273/300 Read block 2829 len 8192
23849 274/300 Write block 2248 len 8192 data: 5 4
23850 275/300 Flush
23851 276/300 Read block 4549 len 8192
23852 277/300 Flush
23853 278/300 Write block 2510 len 20480 data: 4 4 6 6 5
23854 279/300 Flush
23855 280/300 Write block 793 len 36864 data: 7 7 6 5 4 3 3 5 4
23856 281/300 Read block 1105 len 32768
23857 282/300 Flush
23858 283/300 Read block 4185 len 40960
23859 284/300 Read block 2540 len 28672
23860 285/300 Write block 649 len 36864 data: 6 7 6 6 6 5 5 6 6
23861 286/300 Read block 1896 len 20480
23862 287/300 Write block 2556 len 36864 data: 4 4 4 4 4 4 6 6 5
23863 288/300 Flush
23864 289/300 Write block 112 len 40960 data: 5 5 4 4 5 5 5 5 5 4
23865 290/300 Write block 4030 len 32768 data: 4 6 6 8 7 6 5 6
23866 291/300 Read block 4658 len 32768
23867 292/300 Write block 3152 len 12288 data: 7 9 9
23868 293/300 Write block 1852 len 28672 data: 6 4 3 3 3 5 4
23869 294/300 Write block 964 len 12288 data: 4 6 5
23870 295/300 Read block 2424 len 16384
23871 296/300 Write block 10 len 36864 data: 6 7 5 5 5 5 4 2 3
23872 297/300 Read block 3632 len 40960
23873 298/300 Read block 261 len 20480
23874 299/300 Read block 1075 len 12288
23875 300/300 Read block 1405 len 24576
23876 301/300 Read block 266 len 32768
23877 Main task has work to do, go find it
23878 got dsc Start(2)
23879 start 2
23880 Jan 11 06:10:28.266 INFO request completed, latency_us: 244, response_code: 204, uri: /start/cid/2, method: POST, req_id: c89139dd-ab83-4039-b81c-ef10d7e41299, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
23881 Replay: started 2, returned:Ok(())
23882 ----------------------------------------------------------------
23883 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23884 States: Active Active Offline
23885 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23886 7642 Acked Read 6 Done Done Sent false
23887 7643 Acked Write 10 Done Done Sent false
23888 7644 Acked Write 10 Done Done Sent false
23889 7645 Acked Read 9 Done Done Sent false
23890 7646 Acked Read 5 Done Done Sent false
23891 7647 Acked Read 2 Done Done Sent false
23892 7648 Acked Read 7 Done Done Sent false
23893 7649 Acked Write 7 Done Done Sent false
23894 7650 Acked Flush 0 Done Done Sent false
23895 7651 Acked Flush 0 Done Done Sent false
23896 7652 Acked Read 2 Done Done Sent false
23897 7653 Acked Write 6 Done Done Sent false
23898 7654 Acked Read 7 Done Done Sent false
23899 7655 Acked Read 1 Done Done Sent false
23900 7656 Acked Read 2 Done Done Sent false
23901 7657 Acked Read 4 Done Done Sent false
23902 7658 Acked Read 10 Done Done Sent false
23903 7659 Acked Write 1 Done Done Sent false
23904 7660 Acked Read 10 Done Done Sent false
23905 7661 Acked Write 2 Done Done Sent false
23906 7662 Acked Flush 0 Done Done Sent false
23907 7663 Acked Read 6 Done Done Sent false
23908 7664 Acked Read 6 Done Done Sent false
23909 7665 Acked Write 6 Done Done Sent false
23910 7666 Acked Write 2 Done Done Sent false
23911 7667 Acked Flush 0 Done Done Sent false
23912 7668 Acked Read 5 Done Done Sent false
23913 7669 Acked Write 8 Done Done Sent false
23914 7670 Acked Write 7 Done Done Sent false
23915 7671 Acked Write 6 Done Done Sent false
23916 7672 Acked Read 3 Done Done Sent false
23917 7673 Acked Read 1 Done Done Sent false
23918 7674 Acked Write 2 Done Done Sent false
23919 7675 Acked Write 10 Done Done Sent false
23920 7676 Acked Write 7 Done Done Sent false
23921 7677 Acked Write 6 Done Done Sent false
23922 7678 Acked Write 7 Done Done Sent false
23923 7679 Acked Read 8 Done Done Sent false
23924 7680 Acked Read 10 Done Done Sent false
23925 7681 Acked Write 1 Done Done Sent false
23926 7682 Acked Read 8 Done Done Sent false
23927 7683 Acked Read 10 Done Done Sent false
23928 7684 Acked Read 10 Done Done Sent false
23929 7685 Acked Write 3 Done Done Sent false
23930 7686 Acked Read 8 Done Done Sent false
23931 7687 Acked Write 10 Done Done Sent false
23932 7688 Acked Read 2 Done Done Sent false
23933 7689 Acked Read 5 Done Done Sent false
23934 7690 Acked Write 5 Done Done Sent false
23935 7691 Acked Read 5 Done Done Sent false
23936 7692 Acked Read 4 Done Done Sent false
23937 7693 Acked Write 4 Done Done Sent false
23938 7694 Acked Read 7 Done Done Sent false
23939 7695 Acked Write 3 Done Done Sent false
23940 7696 Acked Flush 0 Done Done Sent false
23941 7697 Acked Read 1 Done Done Sent false
23942 7698 Acked Write 9 Done Done Sent false
23943 7699 Acked Write 10 Done Done Sent false
23944 7700 Acked Read 7 Done Done Sent false
23945 7701 Acked Write 10 Done Done Sent false
23946 7702 Acked Read 8 Done Done Sent false
23947 7703 Acked Read 2 Done Done Sent false
23948 7704 Acked Read 5 Done Done Sent false
23949 7705 Acked Write 8 Done Done Sent false
23950 7706 Acked Write 3 Done Done Sent false
23951 7707 Acked Write 8 Done Done Sent false
23952 7708 Acked Write 10 Done Done Sent false
23953 7709 Acked Write 4 Done Done Sent false
23954 7710 Acked Read 7 Done Done Sent false
23955 7711 Acked Read 8 Done Done Sent false
23956 7712 Acked Flush 0 Done Done Sent false
23957 7713 Acked Read 2 Done Done Sent false
23958 7714 Acked Read 5 Done Done Sent false
23959 7715 Acked Flush 0 Done Done Sent false
23960 7716 Acked Flush 0 Done Done Sent false
23961 7717 Acked Read 1 Done Done Sent false
23962 7718 Acked Flush 0 Done Done Sent false
23963 7719 Acked Read 2 Done Done Sent false
23964 7720 Acked Read 8 Done Done Sent false
23965 7721 Acked Write 10 Done Done Sent false
23966 7722 Acked Write 1 Done Done Sent false
23967 7723 Acked Read 2 Done Done Sent false
23968 7724 Acked Read 6 Done Done Sent false
23969 7725 Acked Read 4 Done Done Sent false
23970 7726 Acked Read 10 Done Done Sent false
23971 7727 Acked Write 9 Done Done Sent false
23972 7728 Acked Flush 0 Done Done Sent false
23973 7729 Acked Write 10 Done Done Sent false
23974 7730 Acked Write 8 Done Done Sent false
23975 7731 Acked Read 1 Done Done Sent false
23976 7732 Acked Read 5 Done Done Sent false
23977 7733 Acked Read 1 Done Done Sent false
23978 7734 Acked Flush 0 Done Done Sent false
23979 7735 Acked Write 8 Done Done Sent false
23980 7736 Acked Write 7 Done Done Sent false
23981 7737 Acked Read 1 Done Done Sent false
23982 7738 Acked Read 9 Done Done Sent false
23983 7739 Acked Write 5 Done Done Sent false
23984 7740 Acked Read 7 Done Done Sent false
23985 7741 Acked Read 5 Done Done Sent false
23986 7742 Acked Read 3 Done Done Sent false
23987 7743 Acked Write 4 Done Done Sent false
23988 7744 Acked Read 5 Done Done Sent false
23989 7745 Acked Read 9 Done Done Sent false
23990 7746 Acked Read 7 Done Done Sent false
23991 7747 Acked Write 2 Done Done Sent false
23992 7748 Acked Read 1 Done Done Sent false
23993 7749 Acked Write 3 Done Done Sent false
23994 7750 Acked Write 4 Done Done Sent false
23995 7751 Acked Read 7 Done Done Sent false
23996 7752 Acked Write 1 Done Done Sent false
23997 7753 Acked Write 2 Done Done Sent false
23998 7754 Acked Read 7 Done Done Sent false
23999 7755 Acked Read 10 Done Done Sent false
24000 7756 Acked Read 6 Done Done Sent false
24001 7757 Acked Read 4 Done Done Sent false
24002 7758 Acked Flush 0 Done Done Sent false
24003 7759 Acked Flush 0 Done Done Sent false
24004 7760 Acked Read 3 Done Done Sent false
24005 7761 Acked Write 8 Done Done Sent false
24006 7762 Acked Read 1 Done Done Sent false
24007 7763 Acked Flush 0 Done Done Sent false
24008 7764 Acked Write 7 Done Done Sent false
24009 7765 Acked Read 5 Done Done Sent false
24010 7766 Acked Write 5 Done Done Sent false
24011 7767 Acked Write 3 Done Done Sent false
24012 7768 Acked Read 8 Done Done Sent false
24013 7769 Acked Flush 0 Done Done Sent false
24014 7770 Acked Write 9 Done Done Sent false
24015 7771 Acked Write 8 Done Done Sent false
24016 7772 Acked Read 4 Done Done Sent false
24017 7773 Acked Write 1 Done Done Sent false
24018 7774 Acked Read 6 Done Done Sent false
24019 7775 Acked Read 7 Done Done Sent false
24020 7776 Acked Flush 0 Done Done Sent false
24021 7777 Acked Flush 0 Done Done Sent false
24022 7778 Acked Flush 0 Done Done Sent false
24023 7779 Acked Read 7 Done Done Sent false
24024 7780 Acked Read 10 Done Done Sent false
24025 7781 Acked Write 6 Done Done Sent false
24026 7782 Acked Read 4 Done Done Sent false
24027 7783 Acked Write 10 Done Done Sent false
24028 7784 Acked Read 4 Done Done Sent false
24029 7785 Acked Read 3 Done Done Sent false
24030 7786 Acked Read 8 Done Done Sent false
24031 7787 Acked Write 8 Done Done Sent false
24032 7788 Acked Write 4 Done Done Sent false
24033 7789 Acked Read 10 Done Done Sent false
24034 7790 Acked Read 6 Done Done Sent false
24035 7791 Acked Write 9 Done Done Sent false
24036 7792 Acked Write 1 Done Done Sent false
24037 7793 Acked Flush 0 Done Done Sent false
24038 7794 Acked Read 9 Done Done Sent false
24039 7795 Acked Read 1 Done Done Sent false
24040 7796 Acked Read 4 Done Done Sent false
24041 7797 Acked Read 1 Done Done Sent false
24042 7798 Acked Write 4 Done Done Sent false
24043 7799 Acked Write 2 Done Done Sent false
24044 7800 Acked Flush 0 Done Done Sent false
24045 7801 Acked Write 6 Done Done Sent false
24046 7802 Acked Read 1 Done Done Sent false
24047 7803 Acked Write 6 Done Done Sent false
24048 7804 Acked Write 7 Done Done Sent false
24049 7805 Acked Write 9 Done Done Sent false
24050 7806 Acked Read 1 Done Done Sent false
24051 7807 Acked Read 6 Done Done Sent false
24052 7808 Acked Read 7 Done Done Sent false
24053 7809 Acked Read 7 Done Done Sent false
24054 7810 Acked Write 8 Done Done Sent false
24055 7811 Acked Flush 0 Done Done Sent false
24056 7812 Acked Read 7 Done Done Sent false
24057 7813 Acked Write 2 Done Done Sent false
24058 7814 Acked Flush 0 Done Done Sent false
24059 7815 Acked Read 8 Done Done Sent false
24060 7816 Acked Read 3 Done Done Sent false
24061 7817 Acked Write 4 Done Done Sent false
24062 7818 Acked Read 3 Done Done Sent false
24063 7819 Acked Read 5 Done Done Sent false
24064 7820 Acked Write 9 Done Done Sent false
24065 7821 Acked Write 7 Done Done Sent false
24066 7822 Acked Write 8 Done Done Sent false
24067 7823 Acked Read 7 Done Done Sent false
24068 7824 Acked Write 3 Done Done Sent false
24069 7825 Acked Write 10 Done Done Sent false
24070 7826 Acked Flush 0 Done Done Sent false
24071 7827 Acked Write 8 Done Done Sent false
24072 7828 Acked Write 3 Done Done Sent false
24073 7829 Acked Read 6 Done Done Sent false
24074 7830 Acked Flush 0 Done Done Sent false
24075 7831 Acked Read 3 Done Done Sent false
24076 7832 Acked Read 2 Done Done Sent false
24077 7833 Acked Write 2 Done Done Sent false
24078 7834 Acked Read 10 Done Done Sent false
24079 7835 Acked Flush 0 Done Done Sent false
24080 7836 Acked Read 9 Done Done Sent false
24081 7837 Acked Read 2 Done Done Sent false
24082 7838 Acked Write 4 Done Done Sent false
24083 7839 Acked Write 8 Done Done Sent false
24084 7840 Acked Read 8 Done Done Sent false
24085 7841 Acked Read 8 Done Done Sent false
24086 7842 Acked Write 8 Done Done Sent false
24087 7843 Acked Write 9 Done Done Sent false
24088 7844 Acked Write 7 Done Done Sent false
24089 7845 Acked Read 5 Done Done Sent false
24090 7846 Acked Write 5 Done Done Sent false
24091 7847 Acked Flush 0 Done Done Sent false
24092 7848 Acked Write 2 Done Done Sent false
24093 7849 Acked Read 8 Done Done Sent false
24094 7850 Acked Write 8 Done Done Sent false
24095 7851 Acked Flush 0 Done Done Sent false
24096 7852 Acked Write 4 Done Done Sent false
24097 7853 Acked Write 9 Done Done Sent false
24098 7854 Acked Read 6 Done Done Sent false
24099 7855 Acked Read 8 Done Done Sent false
24100 7856 Acked Flush 0 Done Done Sent false
24101 7857 Acked Write 8 Done Done Sent false
24102 7858 Acked Write 6 Done Done Sent false
24103 7859 Acked Flush 0 Done Done Sent false
24104 7860 Acked Write 1 Done Done Sent false
24105 7861 Acked Write 2 Done Done Sent false
24106 7862 Acked Write 1 Done Done Sent false
24107 7863 Acked Flush 0 Done Done Sent false
24108 7864 Acked Write 5 Done Done Sent false
24109 7865 Acked Read 4 Done Done Sent false
24110 7866 Acked Read 10 Done Done Sent false
24111 7867 Acked Write 10 Done Done Sent false
24112 7868 Acked Read 6 Done Done Sent false
24113 7869 Acked Write 2 Done Done Sent false
24114 7870 Acked Read 10 Done Done Sent false
24115 7871 Acked Read 6 Done Done Sent false
24116 7872 Acked Flush 0 Done Done Sent false
24117 7873 Acked Read 7 Done Done Sent false
24118 7874 Acked Flush 0 Done Done Sent false
24119 7875 Acked Write 2 Done Done Sent false
24120 7876 Acked Read 9 Done Done Sent false
24121 7877 Acked Flush 0 Done Done Sent false
24122 7878 Acked Read 7 Done Done Sent false
24123 7879 Acked Read 1 Done Done Sent false
24124 7880 Acked Write 9 Done Done Sent false
24125 7881 Acked Write 4 Done Done Sent false
24126 7882 Acked Write 1 Done Done Sent false
24127 7883 Acked Read 3 Done Done Sent false
24128 7884 Acked Write 7 Done Done Sent false
24129 7885 Acked Read 9 Done Done Sent false
24130 7886 Acked Write 4 Done Done Sent false
24131 7887 Acked Read 6 Done Done Sent false
24132 7888 Acked Read 7 Done Done Sent false
24133 7889 Acked Write 5 Done Done Sent false
24134 7890 Acked Flush 0 Done Done Sent false
24135 7891 Acked Read 5 Done Done Sent false
24136 7892 Acked Write 7 Done Done Sent false
24137 7893 Acked Write 10 Done Done Sent false
24138 7894 Acked Flush 0 Done Done Sent false
24139 7895 Acked Write 6 Done Done Sent false
24140 7896 Acked Read 7 Done Done Sent false
24141 7897 Acked Write 10 Done Done Sent false
24142 7898 Acked Write 4 Done Done Sent false
24143 7899 Acked Read 4 Done Done Sent false
24144 7900 Acked Write 7 Done Done Sent false
24145 7901 Acked Read 3 Done Done Sent false
24146 7902 Acked Read 3 Done Done Sent false
24147 7903 Acked Read 2 Done Done Sent false
24148 7904 Acked Read 7 Done Done Sent false
24149 7905 Acked Write 3 Done Done Sent false
24150 7906 Acked Read 4 Done Done Sent false
24151 7907 Acked Write 6 Done Done Sent false
24152 7908 Acked Write 2 Done Done Sent false
24153 7909 Acked Write 4 Done Done Sent false
24154 7910 Acked Read 7 Done Done Sent false
24155 7911 Acked Write 7 Done Done Sent false
24156 7912 Acked Write 2 Done Done Sent false
24157 7913 Acked Read 8 Done Done Sent false
24158 7914 Acked Read 2 Done Done Sent false
24159 7915 Acked Write 2 Done Done Sent false
24160 7916 Acked Flush 0 Done Done Sent false
24161 7917 Acked Read 2 Done Done Sent false
24162 7918 Acked Flush 0 Done Done Sent false
24163 7919 Acked Write 5 Done Done Sent false
24164 7920 Acked Flush 0 Done Done Sent false
24165 7921 Acked Write 9 Done Done Sent false
24166 7922 Acked Read 8 Done Done Sent false
24167 7923 Acked Flush 0 Done Done Sent false
24168 7924 Acked Read 10 Done Done Sent false
24169 7925 Acked Read 7 Done Done Sent false
24170 7926 Acked Write 9 Done Done Sent false
24171 7927 Acked Read 5 Done Done Sent false
24172 7928 Acked Write 9 Done Done Sent false
24173 7929 Acked Flush 0 Done Done Sent false
24174 7930 Acked Write 10 Done Done Sent false
24175 7931 Acked Write 8 Done Done Sent false
24176 7932 Acked Read 8 Done Done Sent false
24177 7933 Acked Write 3 Done Done Sent false
24178 7934 Acked Write 7 Done Done Sent false
24179 7935 Acked Write 3 Done Done Sent false
24180 7936 Acked Read 4 Done Done Sent false
24181 7937 Acked Write 9 Done Done Sent false
24182 7938 Acked Read 10 Done Done Sent false
24183 7939 Acked Read 5 Done Done Sent false
24184 7940 Acked Read 3 Done Done Sent false
24185 7941 Acked Read 6 Done Done Sent false
24186 7942 Acked Read 8 Done Done Sent false
24187 STATES DS:0 DS:1 DS:2 TOTAL
24188 Sent 0 0 301 301
24189 Done 301 301 0 602
24190 Skipped 0 0 0 0
24191 Error 0 0 0 0
24192 Last Flush: 7929 7929 7641
24193 Downstairs last five completed: 7641 7640 7639 7638 7637
24194 Upstairs last five completed: 7942 7941 7940 7939 7938
24195 CLIENT: Up:0 ds:301 act:2
24196 [8830] got start action
24197 [8830] I am going to restart
24198 Starting downstairs at port 8830
24199 Make output file at "/tmp/dsc/downstairs-8830.txt"
24200 [8830][2] reports Starting
24201 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1227)
24202 [8830][2] reports Running
24203 ----------------------------------------------------------------
24204 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24205 States: Active Active Offline
24206 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24207 7642 Acked Read 6 Done Done Sent false
24208 7643 Acked Write 10 Done Done Sent false
24209 7644 Acked Write 10 Done Done Sent false
24210 7645 Acked Read 9 Done Done Sent false
24211 7646 Acked Read 5 Done Done Sent false
24212 7647 Acked Read 2 Done Done Sent false
24213 7648 Acked Read 7 Done Done Sent false
24214 7649 Acked Write 7 Done Done Sent false
24215 7650 Acked Flush 0 Done Done Sent false
24216 7651 Acked Flush 0 Done Done Sent false
24217 7652 Acked Read 2 Done Done Sent false
24218 7653 Acked Write 6 Done Done Sent false
24219 7654 Acked Read 7 Done Done Sent false
24220 7655 Acked Read 1 Done Done Sent false
24221 7656 Acked Read 2 Done Done Sent false
24222 7657 Acked Read 4 Done Done Sent false
24223 7658 Acked Read 10 Done Done Sent false
24224 7659 Acked Write 1 Done Done Sent false
24225 7660 Acked Read 10 Done Done Sent false
24226 7661 Acked Write 2 Done Done Sent false
24227 7662 Acked Flush 0 Done Done Sent false
24228 7663 Acked Read 6 Done Done Sent false
24229 7664 Acked Read 6 Done Done Sent false
24230 7665 Acked Write 6 Done Done Sent false
24231 7666 Acked Write 2 Done Done Sent false
24232 7667 Acked Flush 0 Done Done Sent false
24233 7668 Acked Read 5 Done Done Sent false
24234 7669 Acked Write 8 Done Done Sent false
24235 7670 Acked Write 7 Done Done Sent false
24236 7671 Acked Write 6 Done Done Sent false
24237 7672 Acked Read 3 Done Done Sent false
24238 7673 Acked Read 1 Done Done Sent false
24239 7674 Acked Write 2 Done Done Sent false
24240 7675 Acked Write 10 Done Done Sent false
24241 7676 Acked Write 7 Done Done Sent false
24242 7677 Acked Write 6 Done Done Sent false
24243 7678 Acked Write 7 Done Done Sent false
24244 7679 Acked Read 8 Done Done Sent false
24245 7680 Acked Read 10 Done Done Sent false
24246 7681 Acked Write 1 Done Done Sent false
24247 7682 Acked Read 8 Done Done Sent false
24248 7683 Acked Read 10 Done Done Sent false
24249 7684 Acked Read 10 Done Done Sent false
24250 7685 Acked Write 3 Done Done Sent false
24251 7686 Acked Read 8 Done Done Sent false
24252 7687 Acked Write 10 Done Done Sent false
24253 7688 Acked Read 2 Done Done Sent false
24254 7689 Acked Read 5 Done Done Sent false
24255 7690 Acked Write 5 Done Done Sent false
24256 7691 Acked Read 5 Done Done Sent false
24257 7692 Acked Read 4 Done Done Sent false
24258 7693 Acked Write 4 Done Done Sent false
24259 7694 Acked Read 7 Done Done Sent false
24260 7695 Acked Write 3 Done Done Sent false
24261 7696 Acked Flush 0 Done Done Sent false
24262 7697 Acked Read 1 Done Done Sent false
24263 7698 Acked Write 9 Done Done Sent false
24264 7699 Acked Write 10 Done Done Sent false
24265 7700 Acked Read 7 Done Done Sent false
24266 7701 Acked Write 10 Done Done Sent false
24267 7702 Acked Read 8 Done Done Sent false
24268 7703 Acked Read 2 Done Done Sent false
24269 7704 Acked Read 5 Done Done Sent false
24270 7705 Acked Write 8 Done Done Sent false
24271 7706 Acked Write 3 Done Done Sent false
24272 7707 Acked Write 8 Done Done Sent false
24273 7708 Acked Write 10 Done Done Sent false
24274 7709 Acked Write 4 Done Done Sent false
24275 7710 Acked Read 7 Done Done Sent false
24276 7711 Acked Read 8 Done Done Sent false
24277 7712 Acked Flush 0 Done Done Sent false
24278 7713 Acked Read 2 Done Done Sent false
24279 7714 Acked Read 5 Done Done Sent false
24280 7715 Acked Flush 0 Done Done Sent false
24281 7716 Acked Flush 0 Done Done Sent false
24282 7717 Acked Read 1 Done Done Sent false
24283 7718 Acked Flush 0 Done Done Sent false
24284 7719 Acked Read 2 Done Done Sent false
24285 7720 Acked Read 8 Done Done Sent false
24286 7721 Acked Write 10 Done Done Sent false
24287 7722 Acked Write 1 Done Done Sent false
24288 7723 Acked Read 2 Done Done Sent false
24289 7724 Acked Read 6 Done Done Sent false
24290 7725 Acked Read 4 Done Done Sent false
24291 7726 Acked Read 10 Done Done Sent false
24292 7727 Acked Write 9 Done Done Sent false
24293 7728 Acked Flush 0 Done Done Sent false
24294 7729 Acked Write 10 Done Done Sent false
24295 7730 Acked Write 8 Done Done Sent false
24296 7731 Acked Read 1 Done Done Sent false
24297 7732 Acked Read 5 Done Done Sent false
24298 7733 Acked Read 1 Done Done Sent false
24299 7734 Acked Flush 0 Done Done Sent false
24300 7735 Acked Write 8 Done Done Sent false
24301 7736 Acked Write 7 Done Done Sent false
24302 7737 Acked Read 1 Done Done Sent false
24303 7738 Acked Read 9 Done Done Sent false
24304 7739 Acked Write 5 Done Done Sent false
24305 7740 Acked Read 7 Done Done Sent false
24306 7741 Acked Read 5 Done Done Sent false
24307 7742 Acked Read 3 Done Done Sent false
24308 7743 Acked Write 4 Done Done Sent false
24309 7744 Acked Read 5 Done Done Sent false
24310 7745 Acked Read 9 Done Done Sent false
24311 7746 Acked Read 7 Done Done Sent false
24312 7747 Acked Write 2 Done Done Sent false
24313 7748 Acked Read 1 Done Done Sent false
24314 7749 Acked Write 3 Done Done Sent false
24315 7750 Acked Write 4 Done Done Sent false
24316 7751 Acked Read 7 Done Done Sent false
24317 7752 Acked Write 1 Done Done Sent false
24318 7753 Acked Write 2 Done Done Sent false
24319 7754 Acked Read 7 Done Done Sent false
24320 7755 Acked Read 10 Done Done Sent false
24321 7756 Acked Read 6 Done Done Sent false
24322 7757 Acked Read 4 Done Done Sent false
24323 7758 Acked Flush 0 Done Done Sent false
24324 7759 Acked Flush 0 Done Done Sent false
24325 7760 Acked Read 3 Done Done Sent false
24326 7761 Acked Write 8 Done Done Sent false
24327 7762 Acked Read 1 Done Done Sent false
24328 7763 Acked Flush 0 Done Done Sent false
24329 7764 Acked Write 7 Done Done Sent false
24330 7765 Acked Read 5 Done Done Sent false
24331 7766 Acked Write 5 Done Done Sent false
24332 7767 Acked Write 3 Done Done Sent false
24333 7768 Acked Read 8 Done Done Sent false
24334 7769 Acked Flush 0 Done Done Sent false
24335 7770 Acked Write 9 Done Done Sent false
24336 7771 Acked Write 8 Done Done Sent false
24337 7772 Acked Read 4 Done Done Sent false
24338 7773 Acked Write 1 Done Done Sent false
24339 7774 Acked Read 6 Done Done Sent false
24340 7775 Acked Read 7 Done Done Sent false
24341 7776 Acked Flush 0 Done Done Sent false
24342 7777 Acked Flush 0 Done Done Sent false
24343 7778 Acked Flush 0 Done Done Sent false
24344 7779 Acked Read 7 Done Done Sent false
24345 7780 Acked Read 10 Done Done Sent false
24346 7781 Acked Write 6 Done Done Sent false
24347 7782 Acked Read 4 Done Done Sent false
24348 7783 Acked Write 10 Done Done Sent false
24349 7784 Acked Read 4 Done Done Sent false
24350 7785 Acked Read 3 Done Done Sent false
24351 7786 Acked Read 8 Done Done Sent false
24352 7787 Acked Write 8 Done Done Sent false
24353 7788 Acked Write 4 Done Done Sent false
24354 7789 Acked Read 10 Done Done Sent false
24355 7790 Acked Read 6 Done Done Sent false
24356 7791 Acked Write 9 Done Done Sent false
24357 7792 Acked Write 1 Done Done Sent false
24358 7793 Acked Flush 0 Done Done Sent false
24359 7794 Acked Read 9 Done Done Sent false
24360 7795 Acked Read 1 Done Done Sent false
24361 7796 Acked Read 4 Done Done Sent false
24362 7797 Acked Read 1 Done Done Sent false
24363 7798 Acked Write 4 Done Done Sent false
24364 7799 Acked Write 2 Done Done Sent false
24365 7800 Acked Flush 0 Done Done Sent false
24366 7801 Acked Write 6 Done Done Sent false
24367 7802 Acked Read 1 Done Done Sent false
24368 7803 Acked Write 6 Done Done Sent false
24369 7804 Acked Write 7 Done Done Sent false
24370 7805 Acked Write 9 Done Done Sent false
24371 7806 Acked Read 1 Done Done Sent false
24372 7807 Acked Read 6 Done Done Sent false
24373 7808 Acked Read 7 Done Done Sent false
24374 7809 Acked Read 7 Done Done Sent false
24375 7810 Acked Write 8 Done Done Sent false
24376 7811 Acked Flush 0 Done Done Sent false
24377 7812 Acked Read 7 Done Done Sent false
24378 7813 Acked Write 2 Done Done Sent false
24379 7814 Acked Flush 0 Done Done Sent false
24380 7815 Acked Read 8 Done Done Sent false
24381 7816 Acked Read 3 Done Done Sent false
24382 7817 Acked Write 4 Done Done Sent false
24383 7818 Acked Read 3 Done Done Sent false
24384 7819 Acked Read 5 Done Done Sent false
24385 7820 Acked Write 9 Done Done Sent false
24386 7821 Acked Write 7 Done Done Sent false
24387 7822 Acked Write 8 Done Done Sent false
24388 7823 Acked Read 7 Done Done Sent false
24389 7824 Acked Write 3 Done Done Sent false
24390 7825 Acked Write 10 Done Done Sent false
24391 7826 Acked Flush 0 Done Done Sent false
24392 7827 Acked Write 8 Done Done Sent false
24393 7828 Acked Write 3 Done Done Sent false
24394 7829 Acked Read 6 Done Done Sent false
24395 7830 Acked Flush 0 Done Done Sent false
24396 7831 Acked Read 3 Done Done Sent false
24397 7832 Acked Read 2 Done Done Sent false
24398 7833 Acked Write 2 Done Done Sent false
24399 7834 Acked Read 10 Done Done Sent false
24400 7835 Acked Flush 0 Done Done Sent false
24401 7836 Acked Read 9 Done Done Sent false
24402 7837 Acked Read 2 Done Done Sent false
24403 7838 Acked Write 4 Done Done Sent false
24404 7839 Acked Write 8 Done Done Sent false
24405 7840 Acked Read 8 Done Done Sent false
24406 7841 Acked Read 8 Done Done Sent false
24407 7842 Acked Write 8 Done Done Sent false
24408 7843 Acked Write 9 Done Done Sent false
24409 7844 Acked Write 7 Done Done Sent false
24410 7845 Acked Read 5 Done Done Sent false
24411 7846 Acked Write 5 Done Done Sent false
24412 7847 Acked Flush 0 Done Done Sent false
24413 7848 Acked Write 2 Done Done Sent false
24414 7849 Acked Read 8 Done Done Sent false
24415 7850 Acked Write 8 Done Done Sent false
24416 7851 Acked Flush 0 Done Done Sent false
24417 7852 Acked Write 4 Done Done Sent false
24418 7853 Acked Write 9 Done Done Sent false
24419 7854 Acked Read 6 Done Done Sent false
24420 7855 Acked Read 8 Done Done Sent false
24421 7856 Acked Flush 0 Done Done Sent false
24422 7857 Acked Write 8 Done Done Sent false
24423 7858 Acked Write 6 Done Done Sent false
24424 7859 Acked Flush 0 Done Done Sent false
24425 7860 Acked Write 1 Done Done Sent false
24426 7861 Acked Write 2 Done Done Sent false
24427 7862 Acked Write 1 Done Done Sent false
24428 7863 Acked Flush 0 Done Done Sent false
24429 7864 Acked Write 5 Done Done Sent false
24430 7865 Acked Read 4 Done Done Sent false
24431 7866 Acked Read 10 Done Done Sent false
24432 7867 Acked Write 10 Done Done Sent false
24433 7868 Acked Read 6 Done Done Sent false
24434 7869 Acked Write 2 Done Done Sent false
24435 7870 Acked Read 10 Done Done Sent false
24436 7871 Acked Read 6 Done Done Sent false
24437 7872 Acked Flush 0 Done Done Sent false
24438 7873 Acked Read 7 Done Done Sent false
24439 7874 Acked Flush 0 Done Done Sent false
24440 7875 Acked Write 2 Done Done Sent false
24441 7876 Acked Read 9 Done Done Sent false
24442 7877 Acked Flush 0 Done Done Sent false
24443 7878 Acked Read 7 Done Done Sent false
24444 7879 Acked Read 1 Done Done Sent false
24445 7880 Acked Write 9 Done Done Sent false
24446 7881 Acked Write 4 Done Done Sent false
24447 7882 Acked Write 1 Done Done Sent false
24448 7883 Acked Read 3 Done Done Sent false
24449 7884 Acked Write 7 Done Done Sent false
24450 7885 Acked Read 9 Done Done Sent false
24451 7886 Acked Write 4 Done Done Sent false
24452 7887 Acked Read 6 Done Done Sent false
24453 7888 Acked Read 7 Done Done Sent false
24454 7889 Acked Write 5 Done Done Sent false
24455 7890 Acked Flush 0 Done Done Sent false
24456 7891 Acked Read 5 Done Done Sent false
24457 7892 Acked Write 7 Done Done Sent false
24458 7893 Acked Write 10 Done Done Sent false
24459 7894 Acked Flush 0 Done Done Sent false
24460 7895 Acked Write 6 Done Done Sent false
24461 7896 Acked Read 7 Done Done Sent false
24462 7897 Acked Write 10 Done Done Sent false
24463 7898 Acked Write 4 Done Done Sent false
24464 7899 Acked Read 4 Done Done Sent false
24465 7900 Acked Write 7 Done Done Sent false
24466 7901 Acked Read 3 Done Done Sent false
24467 7902 Acked Read 3 Done Done Sent false
24468 7903 Acked Read 2 Done Done Sent false
24469 7904 Acked Read 7 Done Done Sent false
24470 7905 Acked Write 3 Done Done Sent false
24471 7906 Acked Read 4 Done Done Sent false
24472 7907 Acked Write 6 Done Done Sent false
24473 7908 Acked Write 2 Done Done Sent false
24474 7909 Acked Write 4 Done Done Sent false
24475 7910 Acked Read 7 Done Done Sent false
24476 7911 Acked Write 7 Done Done Sent false
24477 7912 Acked Write 2 Done Done Sent false
24478 7913 Acked Read 8 Done Done Sent false
24479 7914 Acked Read 2 Done Done Sent false
24480 7915 Acked Write 2 Done Done Sent false
24481 7916 Acked Flush 0 Done Done Sent false
24482 7917 Acked Read 2 Done Done Sent false
24483 7918 Acked Flush 0 Done Done Sent false
24484 7919 Acked Write 5 Done Done Sent false
24485 7920 Acked Flush 0 Done Done Sent false
24486 7921 Acked Write 9 Done Done Sent false
24487 7922 Acked Read 8 Done Done Sent false
24488 7923 Acked Flush 0 Done Done Sent false
24489 7924 Acked Read 10 Done Done Sent false
24490 7925 Acked Read 7 Done Done Sent false
24491 7926 Acked Write 9 Done Done Sent false
24492 7927 Acked Read 5 Done Done Sent false
24493 7928 Acked Write 9 Done Done Sent false
24494 7929 Acked Flush 0 Done Done Sent false
24495 7930 Acked Write 10 Done Done Sent false
24496 7931 Acked Write 8 Done Done Sent false
24497 7932 Acked Read 8 Done Done Sent false
24498 7933 Acked Write 3 Done Done Sent false
24499 7934 Acked Write 7 Done Done Sent false
24500 7935 Acked Write 3 Done Done Sent false
24501 7936 Acked Read 4 Done Done Sent false
24502 7937 Acked Write 9 Done Done Sent false
24503 7938 Acked Read 10 Done Done Sent false
24504 7939 Acked Read 5 Done Done Sent false
24505 7940 Acked Read 3 Done Done Sent false
24506 7941 Acked Read 6 Done Done Sent false
24507 7942 Acked Read 8 Done Done Sent false
24508 7943 Acked Flush 0 Done Done Sent false
24509 STATES DS:0 DS:1 DS:2 TOTAL
24510 Sent 0 0 302 302
24511 Done 302 302 0 604
24512 Skipped 0 0 0 0
24513 Error 0 0 0 0
24514 Last Flush: 7943 7943 7641
24515 Downstairs last five completed: 7641 7640 7639 7638 7637
24516 Upstairs last five completed: 7943 7942 7941 7940 7939
24517 CLIENT: Up:0 ds:302 act:2
245182025-01-11T06:10:34.023ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245192025-01-11T06:10:34.023ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55001) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245202025-01-11T06:10:34.024ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245212025-01-11T06:10:34.025ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245222025-01-11T06:10:34.025ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245232025-01-11T06:10:34.025ZINFOcrucible: send last flush ID to this DS: 7641 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245242025-01-11T06:10:34.025ZINFOcrucible: Replied this last flush ID: 7641 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245252025-01-11T06:10:34.025ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245262025-01-11T06:10:34.025ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245272025-01-11T06:10:34.025ZINFOcrucible: [2] client re-new 302 jobs since flush 7641 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245282025-01-11T06:10:34.025ZINFOcrucible: [2] Replayed 302 jobs since flush: 7641 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
24529 ----------------------------------------------------------------
24530 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24531 Downstairs last five completed: 7943 7942 7941 7940 7939
24532 Upstairs last five completed: 7943 7942 7941 7940 7939
24533 CLIENT: Up:0 ds:0 act:3
24534 Replay: All jobs finished
24535 Main task has work to do, go find it
24536 got dsc Stop(1)
24537 stop 1
24538 Jan 11 06:10:36.273 INFO request completed, latency_us: 384, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 72f5e2da-9f9d-4911-88e7-c47c7ee5efb1, remote_addr: 127.0.0.1:59925, local_addr:[8820] Got stop action so:false kr:false
24539 127.0.0.1:9998
245402025-01-11T06:10:36.273ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245412025-01-11T06:10:36.274ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245422025-01-11T06:10:36.274ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245432025-01-11T06:10:36.274ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
245442025-01-11T06:10:36.274ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
24545 Jan 11 06:10:36.274 INFO request completed, latency_us: 189, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1a153450-a560-4d87-90b2-6d503b2e83c7, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:10:36.274222917Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
24546 
24547 [8820] Exited with: ExitStatus(unix_wait_status(9))
24548 [8820][1] reports Exit
24549 Jan 11 06:10:40.276 INFO request completed, latency_us: 225, response_code: 200, uri: /state/cid/1, method: GET, req_id: 37d610df-5621-4fe7-aee6-c521ad4f1db5, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
24550 001/300 Read block 179 len 36864
24551 002/300 Flush
24552 003/300 Write block 3488 len 8192 data: 4 4
24553 004/300 Write block 1366 len 32768 data: 8 8 7 5 4 3 4 5
24554 005/300 Flush
24555 006/300 Flush
24556 007/300 Read block 3383 len 8192
24557 008/300 Flush
24558 009/300 Write block 3571 len 12288 data: 5 5 4
24559 010/300 Flush
24560 011/300 Read block 3511 len 12288
24561 012/300 Read block 2726 len 16384
24562 013/300 Write block 4412 len 8192 data: 6 5
24563 014/300 Read block 3252 len 32768
24564 015/300 Write block 2027 len 20480 data: 3 3 5 7 7
24565 016/300 Write block 891 len 16384 data: 3 2 2 3
24566 017/300 Read block 3752 len 32768
24567 018/300 Read block 3972 len 32768
24568 019/300 Write block 3217 len 32768 data: 4 6 5 4 5 5 4 4
24569 020/300 Write block 440 len 28672 data: 2 5 5 5 4 4 4
24570 021/300 Read block 3915 len 36864
24571 022/300 Read block 111 len 28672
24572 023/300 Write block 1626 len 8192 data: 5 5
24573 024/300 Read block 3853 len 16384
24574 025/300 Write block 1781 len 16384 data: 6 4 6 7
24575 026/300 Write block 836 len 20480 data: 6 5 6 6 8
24576 027/300 Read block 1679 len 16384
24577 028/300 Read block 3094 len 16384
24578 029/300 Read block 849 len 32768
24579 030/300 Read block 2572 len 24576
24580 031/300 Read block 388 len 32768
24581 032/300 Read block 4821 len 8192
24582 033/300 Write block 2980 len 8192 data: 4 4
24583 034/300 Read block 4556 len 20480
24584 035/300 Flush
24585 036/300 Write block 3571 len 40960 data: 6 6 5 4 5 7 6 5 5 6
24586 037/300 Read block 3387 len 28672
24587 038/300 Write block 1178 len 8192 data: 7 7
24588 039/300 Flush
24589 040/300 Flush
24590 041/300 Flush
24591 042/300 Read block 3669 len 32768
24592 043/300 Read block 1429 len 40960
24593 044/300 Write block 1952 len 8192 data: 4 5
24594 045/300 Read block 216 len 32768
24595 046/300 Write block 1643 len 24576 data: 4 3 3 4 4 4
24596 047/300 Read block 4149 len 20480
24597 048/300 Read block 3640 len 12288
24598 049/300 Write block 151 len 32768 data: 3 3 3 3 3 3 3 3
24599 050/300 Write block 1390 len 4096 data: 2
24600 051/300 Flush
24601 052/300 Write block 2993 len 4096 data: 2
24602 053/300 Read block 3863 len 20480
24603 054/300 Read block 4958 len 20480
24604 055/300 Read block 1663 len 4096
24605 056/300 Write block 2278 len 36864 data: 4 5 6 7 7 6 6 6 6
24606 057/300 Read block 3121 len 16384
24607 058/300 Read block 3072 len 28672
24608 059/300 Flush
24609 060/300 Write block 2973 len 8192 data: 6 6
24610 061/300 Read block 4904 len 28672
24611 062/300 Write block 689 len 20480 data: 10 10 9 11 11
24612 063/300 Write block 1176 len 20480 data: 5 6 8 8 6
24613 064/300 Flush
24614 065/300 Read block 2299 len 28672
24615 066/300 Read block 3604 len 24576
24616 067/300 Write block 3057 len 28672 data: 3 3 3 3 6 7 6
24617 068/300 Read block 2103 len 20480
24618 069/300 Write block 877 len 4096 data: 5
24619 070/300 Write block 3102 len 8192 data: 9 7
24620 071/300 Read block 3085 len 12288
24621 072/300 Read block 4264 len 28672
24622 073/300 Write block 3007 len 20480 data: 9 9 8 5 5
24623 074/300 Read block 4090 len 8192
24624 075/300 Read block 348 len 16384
24625 076/300 Read block 4428 len 4096
24626 077/300 Write block 4082 len 40960 data: 4 5 3 5 7 8 9 9 8 7
24627 078/300 Read block 1384 len 32768
24628 079/300 Write block 4193 len 12288 data: 3 4 4
24629 080/300 Read block 3931 len 8192
24630 081/300 Read block 2704 len 40960
24631 082/300 Flush
24632 083/300 Read block 771 len 40960
24633 084/300 Read block 1518 len 16384
24634 085/300 Write block 4332 len 20480 data: 3 4 4 2 2
24635 086/300 Write block 3791 len 8192 data: 3 4
24636 087/300 Write block 877 len 28672 data: 6 6 7 8 7 6 7
24637 088/300 Read block 3511 len 24576
24638 089/300 Write block 4942 len 24576 data: 5 7 6 6 5 5
24639 090/300 Write block 1521 len 4096 data: 5
24640 091/300 Flush
24641 092/300 Read block 3476 len 40960
24642 093/300 Flush
24643 094/300 Read block 3506 len 8192
24644 095/300 Read block 404 len 24576
24645 096/300 Write block 4265 len 20480 data: 8 9 7 8 8
24646 097/300 Write block 3826 len 28672 data: 5 6 5 5 6 5 5
24647 098/300 Read block 4738 len 12288
24648 099/300 Write block 1063 len 12288 data: 7 5 4
24649 100/300 Read block 2598 len 24576
24650 101/300 Read block 4124 len 20480
24651 102/300 Write block 1987 len 36864 data: 4 4 6 7 8 6 8 7 5
24652 103/300 Write block 4905 len 28672 data: 5 5 5 5 5 3 3
24653 104/300 Flush
24654 105/300 Write block 1008 len 20480 data: 8 6 7 4 5
24655 106/300 Read block 743 len 36864
24656 107/300 Read block 2916 len 24576
24657 108/300 Read block 3828 len 12288
24658 109/300 Read block 1031 len 28672
24659 110/300 Read block 3010 len 28672
24660 111/300 Read block 3500 len 20480
24661 112/300 Read block 3401 len 24576
24662 113/300 Read block 3815 len 8192
24663 114/300 Write block 2899 len 28672 data: 5 5 5 5 6 6 7
24664 115/300 Read block 3155 len 12288
24665 116/300 Write block 3650 len 40960 data: 3 5 3 3 3 4 3 3 3 3
24666 117/300 Flush
24667 118/300 Write block 340 len 4096 data: 7
24668 119/300 Read block 2837 len 24576
24669 120/300 Write block 3180 len 32768 data: 5 5 5 5 4 6 7 6
24670 121/300 Read block 450 len 32768
24671 122/300 Write block 2681 len 24576 data: 6 8 7 5 5 6
24672 123/300 Write block 789 len 20480 data: 5 6 7 7 8
24673 124/300 Flush
24674 125/300 Write block 301 len 20480 data: 5 3 3 5 5
24675 126/300 Write block 2315 len 20480 data: 4 3 3 3 4
24676 127/300 Write block 4522 len 32768 data: 6 5 4 4 5 5 7 6
24677 128/300 Flush
24678 129/300 Write block 2672 len 32768 data: 2 2 3 3 3 6 5 5
24679 130/300 Write block 1736 len 12288 data: 5 4 4
24680 131/300 Read block 783 len 36864
24681 132/300 Read block 198 len 8192
24682 133/300 Write block 225 len 16384 data: 5 6 8 7
24683 134/300 Read block 4956 len 40960
24684 135/300 Write block 120 len 8192 data: 6 5
24685 136/300 Read block 3068 len 16384
24686 137/300 Read block 893 len 12288
24687 138/300 Flush
24688 139/300 Write block 4916 len 20480 data: 3 4 4 3 3
24689 140/300 Flush
24690 141/300 Read block 2350 len 16384
24691 142/300 Read block 3449 len 24576
24692 143/300 Read block 3134 len 20480
24693 144/300 Flush
24694 145/300 Write block 315 len 40960 data: 4 5 3 4 3 4 5 6 8 8
24695 146/300 Read block 188 len 4096
24696 147/300 Read block 2918 len 4096
24697 148/300 Write block 86 len 4096 data: 4
24698 149/300 Read block 4929 len 16384
24699 150/300 Read block 4914 len 40960
24700 151/300 Read block 4530 len 4096
24701 152/300 Write block 2340 len 40960 data: 5 5 5 5 6 7 8 7 5 4
24702 153/300 Write block 4321 len 28672 data: 6 5 4 4 4 4 4
24703 154/300 Read block 135 len 32768
24704 155/300 Read block 2061 len 4096
24705 156/300 Read block 3431 len 32768
24706 157/300 Write block 1228 len 40960 data: 6 5 6 6 6 7 7 5 4 4
24707 158/300 Write block 2117 len 12288 data: 3 4 3
24708 159/300 Read block 3673 len 36864
24709 160/300 Write block 690 len 20480 data: 11 10 12 12 8
24710 161/300 Write block 510 len 24576 data: 4 4 3 3 4 4
24711 162/300 Write block 3622 len 24576 data: 4 4 5 5 4 3
24712 163/300 Flush
24713 164/300 Read block 3005 len 8192
24714 165/300 Read block 1169 len 36864
24715 166/300 Write block 898 len 8192 data: 2 2
24716 167/300 Flush
24717 168/300 Write block 2292 len 24576 data: 5 4 5 6 6 6
24718 169/300 Read block 67 len 12288
24719 170/300 Write block 2383 len 36864 data: 6 5 4 4 4 5 5 3 3
24720 171/300 Write block 2734 len 32768 data: 5 6 6 5 5 5 5 6
24721 172/300 Write block 2543 len 4096 data: 7
24722 173/300 Read block 1718 len 28672
24723 174/300 Read block 2499 len 28672
24724 175/300 Read block 2961 len 12288
24725 176/300 Write block 437 len 32768 data: 4 3 2 3 6 6 6 5
24726 177/300 Read block 1149 len 28672
24727 178/300 Read block 848 len 24576
24728 179/300 Write block 2351 len 8192 data: 5 5
24729 180/300 Flush
24730 181/300 Read block 3192 len 16384
24731 182/300 Read block 270 len 28672
24732 183/300 Write block 2505 len 24576 data: 2 2 2 2 4 5
24733 184/300 Flush
24734 185/300 Write block 79 len 8192 data: 5 6
24735 186/300 Write block 3018 len 16384 data: 4 5 5 4
24736 187/300 Write block 1578 len 36864 data: 5 5 4 3 4 4 3 3 3
24737 188/300 Read block 2055 len 16384
24738 189/300 Write block 3612 len 28672 data: 6 6 6 6 5 5 4
24739 190/300 Read block 3464 len 16384
24740 191/300 Write block 4872 len 28672 data: 5 4 5 5 6 6 6
24741 192/300 Flush
24742 193/300 Read block 1984 len 24576
24743 194/300 Read block 650 len 40960
24744 195/300 Read block 46 len 32768
24745 196/300 Read block 3624 len 36864
24746 197/300 Flush
24747 198/300 Write block 4880 len 12288 data: 5 6 3
24748 199/300 Write block 3348 len 16384 data: 3 5 5 4
24749 200/300 Write block 3350 len 24576 data: 6 5 3 3 3 3
24750 201/300 Read block 4894 len 4096
24751 202/300 Write block 3562 len 24576 data: 8 7 6 6 4 6
24752 203/300 Flush
24753 204/300 Read block 1654 len 16384
24754 205/300 Write block 1885 len 8192 data: 8 6
24755 206/300 Write block 4911 len 12288 data: 4 2 2
24756 207/300 Read block 589 len 4096
24757 208/300 Read block 2365 len 36864
24758 209/300 Read block 2987 len 20480
24759 210/300 Read block 2785 len 32768
24760 211/300 Write block 3526 len 36864 data: 7 6 5 5 6 6 5 6 5
24761 212/300 Read block 3426 len 12288
24762 213/300 Write block 4412 len 4096 data: 7
24763 214/300 Read block 2463 len 12288
24764 215/300 Read block 1496 len 28672
24765 216/300 Read block 3139 len 28672
24766 217/300 Read block 2063 len 8192
24767 218/300 Read block 1829 len 12288
24768 219/300 Read block 2805 len 36864
24769 220/300 Read block 1249 len 20480
24770 221/300 Read block 3874 len 4096
24771 222/300 Write block 546 len 16384 data: 7 6 6 7
24772 223/300 Write block 3593 len 16384 data: 9 9 8 7
24773 224/300 Read block 3720 len 16384
24774 225/300 Write block 2935 len 36864 data: 10 9 8 7 6 8 7 6 5
24775 226/300 Write block 624 len 36864 data: 4 4 5 5 3 3 3 5 6
24776 227/300 Read block 4611 len 4096
24777 228/300 Read block 1807 len 12288
24778 229/300 Write block 1593 len 40960 data: 4 4 4 4 4 3 3 3 3 2
24779 230/300 Write block 56 len 32768 data: 7 8 9 8 8 8 9 8
24780 231/300 Read block 3735 len 32768
24781 232/300 Read block 2561 len 24576
24782 233/300 Write block 3884 len 40960 data: 7 6 6 6 5 5 3 3 3 3
24783 234/300 Read block 2793 len 16384
24784 235/300 Read block 4193 len 40960
24785 236/300 Write block 4465 len 36864 data: 11 11 11 13 10 9 8 7 6
24786 237/300 Flush
24787 238/300 Read block 4988 len 40960
24788 239/300 Write block 1088 len 4096 data: 2
24789 240/300 Write block 2921 len 4096 data: 4
24790 241/300 Write block 1128 len 40960 data: 4 3 4 3 3 6 6 5 5 5
24791 242/300 Read block 2422 len 40960
24792 243/300 Write block 1520 len 28672 data: 4 6 5 6 8 8 8
24793 244/300 Write block 3831 len 4096 data: 6
24794 245/300 Write block 3467 len 36864 data: 5 6 8 8 7 6 9 9 9
24795 246/300 Read block 4023 len 32768
24796 247/300 Write block 317 len 20480 data: 4 5 4 5 6
24797 248/300 Write block 1719 len 12288 data: 5 6 6
24798 249/300 Flush
24799 250/300 Read block 348 len 4096
24800 251/300 Write block 4797 len 24576 data: 9 8 7 7 7 7
24801 252/300 Read block 1345 len 36864
24802 253/300 Read block 395 len 12288
24803 254/300 Write block 4308 len 12288 data: 5 5 5
24804 255/300 Write block 4624 len 16384 data: 5 2 2 3
24805 256/300 Write block 423 len 20480 data: 3 2 3 3 3
24806 257/300 Write block 3624 len 36864 data: 6 6 5 4 4 3 4 5 5
24807 258/300 Read block 1211 len 40960
24808 259/300 Read block 1966 len 16384
24809 260/300 Read block 714 len 20480
24810 261/300 Write block 1770 len 16384 data: 6 5 6 7
24811 262/300 Write block 2446 len 24576 data: 5 5 6 6 7 6
24812 263/300 Write block 973 len 4096 data: 6
24813 264/300 Read block 1915 len 24576
24814 265/300 Read block 288 len 8192
24815 266/300 Flush
24816 267/300 Read block 4679 len 40960
24817 268/300 Write block 3304 len 36864 data: 4 5 5 4 3 3 3 3 4
24818 269/300 Write block 1758 len 24576 data: 7 5 5 4 4 4
24819 270/300 Read block 97 len 16384
24820 271/300 Read block 1041 len 36864
24821 272/300 Write block 2193 len 32768 data: 5 4 6 6 9 8 8 7
24822 273/300 Read block 3458 len 40960
24823 274/300 Write block 639 len 36864 data: 3 3 5 5 5 5 4 3 4
24824 275/300 Read block 1816 len 24576
24825 276/300 Write block 768 len 16384 data: 6 6 6 5
24826 277/300 Write block 3030 len 28672 data: 7 7 7 8 7 6 6
24827 278/300 Write block 3970 len 28672 data: 2 2 3 5 4 6 6
24828 279/300 Write block 4228 len 36864 data: 4 2 2 2 2 2 3 3 5
24829 280/300 Read block 272 len 4096
24830 281/300 Flush
24831 282/300 Write block 4871 len 40960 data: 5 6 5 6 6 7 7 7 5 6
24832 283/300 Read block 1827 len 24576
24833 284/300 Read block 2649 len 4096
24834 285/300 Write block 3861 len 36864 data: 2 3 3 5 6 5 6 6 3
24835 286/300 Read block 1037 len 32768
24836 287/300 Read block 1547 len 40960
24837 288/300 Write block 1147 len 16384 data: 6 7 8 8
24838 289/300 Write block 4690 len 4096 data: 5
24839 290/300 Read block 3885 len 40960
24840 291/300 Write block 4131 len 24576 data: 7 7 5 7 7 7
24841 292/300 Write block 3476 len 4096 data: 9
24842 293/300 Read block 142 len 40960
24843 294/300 Read block 643 len 8192
24844 295/300 Flush
24845 296/300 Read block 4732 len 36864
24846 297/300 Read block 3999 len 36864
24847 298/300 Read block 2798 len 8192
24848 299/300 Read block 2502 len 32768
24849 300/300 Flush
24850 301/300 Write block 2696 len 32768 data: 4 4 4 4 4 5 4 4
24851 Main task has work to do, go find it
24852 got dsc Start(1)
24853 start 1
24854 Jan 11 06:10:40.516 INFO request completed, latency_us: 279, response_code: 204, uri: /start/cid/1, method: POST, req_id: 9357f055-62c3-43c8-806e-2e8d9b94eb83, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
24855 Replay: started 1, returned:Ok(())
24856 ----------------------------------------------------------------
24857 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24858 States: Active Offline Active
24859 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24860 7944 Acked Read 9 Done Sent Done false
24861 7945 Acked Flush 0 Done Sent Done false
24862 7946 Acked Write 2 Done Sent Done false
24863 7947 Acked Write 8 Done Sent Done false
24864 7948 Acked Flush 0 Done Sent Done false
24865 7949 Acked Flush 0 Done Sent Done false
24866 7950 Acked Read 2 Done Sent Done false
24867 7951 Acked Flush 0 Done Sent Done false
24868 7952 Acked Write 3 Done Sent Done false
24869 7953 Acked Flush 0 Done Sent Done false
24870 7954 Acked Read 3 Done Sent Done false
24871 7955 Acked Read 4 Done Sent Done false
24872 7956 Acked Write 2 Done Sent Done false
24873 7957 Acked Read 8 Done Sent Done false
24874 7958 Acked Write 5 Done Sent Done false
24875 7959 Acked Write 4 Done Sent Done false
24876 7960 Acked Read 8 Done Sent Done false
24877 7961 Acked Read 8 Done Sent Done false
24878 7962 Acked Write 8 Done Sent Done false
24879 7963 Acked Write 7 Done Sent Done false
24880 7964 Acked Read 9 Done Sent Done false
24881 7965 Acked Read 7 Done Sent Done false
24882 7966 Acked Write 2 Done Sent Done false
24883 7967 Acked Read 4 Done Sent Done false
24884 7968 Acked Write 4 Done Sent Done false
24885 7969 Acked Write 5 Done Sent Done false
24886 7970 Acked Read 4 Done Sent Done false
24887 7971 Acked Read 4 Done Sent Done false
24888 7972 Acked Read 8 Done Sent Done false
24889 7973 Acked Read 6 Done Sent Done false
24890 7974 Acked Read 8 Done Sent Done false
24891 7975 Acked Read 2 Done Sent Done false
24892 7976 Acked Write 2 Done Sent Done false
24893 7977 Acked Read 5 Done Sent Done false
24894 7978 Acked Flush 0 Done Sent Done false
24895 7979 Acked Write 10 Done Sent Done false
24896 7980 Acked Read 7 Done Sent Done false
24897 7981 Acked Write 2 Done Sent Done false
24898 7982 Acked Flush 0 Done Sent Done false
24899 7983 Acked Flush 0 Done Sent Done false
24900 7984 Acked Flush 0 Done Sent Done false
24901 7985 Acked Read 8 Done Sent Done false
24902 7986 Acked Read 10 Done Sent Done false
24903 7987 Acked Write 2 Done Sent Done false
24904 7988 Acked Read 8 Done Sent Done false
24905 7989 Acked Write 6 Done Sent Done false
24906 7990 Acked Read 5 Done Sent Done false
24907 7991 Acked Read 3 Done Sent Done false
24908 7992 Acked Write 8 Done Sent Done false
24909 7993 Acked Write 1 Done Sent Done false
24910 7994 Acked Flush 0 Done Sent Done false
24911 7995 Acked Write 1 Done Sent Done false
24912 7996 Acked Read 5 Done Sent Done false
24913 7997 Acked Read 5 Done Sent Done false
24914 7998 Acked Read 1 Done Sent Done false
24915 7999 Acked Write 9 Done Sent Done false
24916 8000 Acked Read 4 Done Sent Done false
24917 8001 Acked Read 7 Done Sent Done false
24918 8002 Acked Flush 0 Done Sent Done false
24919 8003 Acked Write 2 Done Sent Done false
24920 8004 Acked Read 7 Done Sent Done false
24921 8005 Acked Write 5 Done Sent Done false
24922 8006 Acked Write 5 Done Sent Done false
24923 8007 Acked Flush 0 Done Sent Done false
24924 8008 Acked Read 7 Done Sent Done false
24925 8009 Acked Read 6 Done Sent Done false
24926 8010 Acked Write 7 Done Sent Done false
24927 8011 Acked Read 5 Done Sent Done false
24928 8012 Acked Write 1 Done Sent Done false
24929 8013 Acked Write 2 Done Sent Done false
24930 8014 Acked Read 3 Done Sent Done false
24931 8015 Acked Read 7 Done Sent Done false
24932 8016 Acked Write 5 Done Sent Done false
24933 8017 Acked Read 2 Done Sent Done false
24934 8018 Acked Read 4 Done Sent Done false
24935 8019 Acked Read 1 Done Sent Done false
24936 8020 Acked Write 10 Done Sent Done false
24937 8021 Acked Read 8 Done Sent Done false
24938 8022 Acked Write 3 Done Sent Done false
24939 8023 Acked Read 2 Done Sent Done false
24940 8024 Acked Read 10 Done Sent Done false
24941 8025 Acked Flush 0 Done Sent Done false
24942 8026 Acked Read 10 Done Sent Done false
24943 8027 Acked Read 4 Done Sent Done false
24944 8028 Acked Write 5 Done Sent Done false
24945 8029 Acked Write 2 Done Sent Done false
24946 8030 Acked Write 7 Done Sent Done false
24947 8031 Acked Read 6 Done Sent Done false
24948 8032 Acked Write 6 Done Sent Done false
24949 8033 Acked Write 1 Done Sent Done false
24950 8034 Acked Flush 0 Done Sent Done false
24951 8035 Acked Read 10 Done Sent Done false
24952 8036 Acked Flush 0 Done Sent Done false
24953 8037 Acked Read 2 Done Sent Done false
24954 8038 Acked Read 6 Done Sent Done false
24955 8039 Acked Write 5 Done Sent Done false
24956 8040 Acked Write 7 Done Sent Done false
24957 8041 Acked Read 3 Done Sent Done false
24958 8042 Acked Write 3 Done Sent Done false
24959 8043 Acked Read 6 Done Sent Done false
24960 8044 Acked Read 5 Done Sent Done false
24961 8045 Acked Write 9 Done Sent Done false
24962 8046 Acked Write 7 Done Sent Done false
24963 8047 Acked Flush 0 Done Sent Done false
24964 8048 Acked Write 5 Done Sent Done false
24965 8049 Acked Read 9 Done Sent Done false
24966 8050 Acked Read 6 Done Sent Done false
24967 8051 Acked Read 3 Done Sent Done false
24968 8052 Acked Read 7 Done Sent Done false
24969 8053 Acked Read 7 Done Sent Done false
24970 8054 Acked Read 5 Done Sent Done false
24971 8055 Acked Read 6 Done Sent Done false
24972 8056 Acked Read 2 Done Sent Done false
24973 8057 Acked Write 7 Done Sent Done false
24974 8058 Acked Read 3 Done Sent Done false
24975 8059 Acked Write 10 Done Sent Done false
24976 8060 Acked Flush 0 Done Sent Done false
24977 8061 Acked Write 1 Done Sent Done false
24978 8062 Acked Read 6 Done Sent Done false
24979 8063 Acked Write 8 Done Sent Done false
24980 8064 Acked Read 8 Done Sent Done false
24981 8065 Acked Write 6 Done Sent Done false
24982 8066 Acked Write 5 Done Sent Done false
24983 8067 Acked Flush 0 Done Sent Done false
24984 8068 Acked Write 5 Done Sent Done false
24985 8069 Acked Write 5 Done Sent Done false
24986 8070 Acked Write 8 Done Sent Done false
24987 8071 Acked Flush 0 Done Sent Done false
24988 8072 Acked Write 8 Done Sent Done false
24989 8073 Acked Write 3 Done Sent Done false
24990 8074 Acked Read 9 Done Sent Done false
24991 8075 Acked Read 2 Done Sent Done false
24992 8076 Acked Write 4 Done Sent Done false
24993 8077 Acked Read 10 Done Sent Done false
24994 8078 Acked Write 2 Done Sent Done false
24995 8079 Acked Read 4 Done Sent Done false
24996 8080 Acked Read 3 Done Sent Done false
24997 8081 Acked Flush 0 Done Sent Done false
24998 8082 Acked Write 5 Done Sent Done false
24999 8083 Acked Flush 0 Done Sent Done false
25000 8084 Acked Read 4 Done Sent Done false
25001 8085 Acked Read 6 Done Sent Done false
25002 8086 Acked Read 5 Done Sent Done false
25003 8087 Acked Flush 0 Done Sent Done false
25004 8088 Acked Write 10 Done Sent Done false
25005 8089 Acked Read 1 Done Sent Done false
25006 8090 Acked Read 1 Done Sent Done false
25007 8091 Acked Write 1 Done Sent Done false
25008 8092 Acked Read 4 Done Sent Done false
25009 8093 Acked Read 10 Done Sent Done false
25010 8094 Acked Read 1 Done Sent Done false
25011 8095 Acked Write 10 Done Sent Done false
25012 8096 Acked Write 7 Done Sent Done false
25013 8097 Acked Read 8 Done Sent Done false
25014 8098 Acked Read 1 Done Sent Done false
25015 8099 Acked Read 8 Done Sent Done false
25016 8100 Acked Write 10 Done Sent Done false
25017 8101 Acked Write 3 Done Sent Done false
25018 8102 Acked Read 9 Done Sent Done false
25019 8103 Acked Write 5 Done Sent Done false
25020 8104 Acked Write 6 Done Sent Done false
25021 8105 Acked Write 6 Done Sent Done false
25022 8106 Acked Flush 0 Done Sent Done false
25023 8107 Acked Read 2 Done Sent Done false
25024 8108 Acked Read 9 Done Sent Done false
25025 8109 Acked Write 2 Done Sent Done false
25026 8110 Acked Flush 0 Done Sent Done false
25027 8111 Acked Write 6 Done Sent Done false
25028 8112 Acked Read 3 Done Sent Done false
25029 8113 Acked Write 9 Done Sent Done false
25030 8114 Acked Write 8 Done Sent Done false
25031 8115 Acked Write 1 Done Sent Done false
25032 8116 Acked Read 7 Done Sent Done false
25033 8117 Acked Read 7 Done Sent Done false
25034 8118 Acked Read 3 Done Sent Done false
25035 8119 Acked Write 8 Done Sent Done false
25036 8120 Acked Read 7 Done Sent Done false
25037 8121 Acked Read 6 Done Sent Done false
25038 8122 Acked Write 2 Done Sent Done false
25039 8123 Acked Flush 0 Done Sent Done false
25040 8124 Acked Read 4 Done Sent Done false
25041 8125 Acked Read 7 Done Sent Done false
25042 8126 Acked Write 6 Done Sent Done false
25043 8127 Acked Flush 0 Done Sent Done false
25044 8128 Acked Write 2 Done Sent Done false
25045 8129 Acked Write 4 Done Sent Done false
25046 8130 Acked Write 9 Done Sent Done false
25047 8131 Acked Read 4 Done Sent Done false
25048 8132 Acked Write 7 Done Sent Done false
25049 8133 Acked Read 4 Done Sent Done false
25050 8134 Acked Write 7 Done Sent Done false
25051 8135 Acked Flush 0 Done Sent Done false
25052 8136 Acked Read 6 Done Sent Done false
25053 8137 Acked Read 10 Done Sent Done false
25054 8138 Acked Read 8 Done Sent Done false
25055 8139 Acked Read 9 Done Sent Done false
25056 8140 Acked Flush 0 Done Sent Done false
25057 8141 Acked Write 3 Done Sent Done false
25058 8142 Acked Write 4 Done Sent Done false
25059 8143 Acked Write 6 Done Sent Done false
25060 8144 Acked Read 1 Done Sent Done false
25061 8145 Acked Write 6 Done Sent Done false
25062 8146 Acked Flush 0 Done Sent Done false
25063 8147 Acked Read 4 Done Sent Done false
25064 8148 Acked Write 2 Done Sent Done false
25065 8149 Acked Write 3 Done Sent Done false
25066 8150 Acked Read 1 Done Sent Done false
25067 8151 Acked Read 9 Done Sent Done false
25068 8152 Acked Read 5 Done Sent Done false
25069 8153 Acked Read 8 Done Sent Done false
25070 8154 Acked Write 9 Done Sent Done false
25071 8155 Acked Read 3 Done Sent Done false
25072 8156 Acked Write 1 Done Sent Done false
25073 8157 Acked Read 3 Done Sent Done false
25074 8158 Acked Read 7 Done Sent Done false
25075 8159 Acked Read 7 Done Sent Done false
25076 8160 Acked Read 2 Done Sent Done false
25077 8161 Acked Read 3 Done Sent Done false
25078 8162 Acked Read 9 Done Sent Done false
25079 8163 Acked Read 5 Done Sent Done false
25080 8164 Acked Read 1 Done Sent Done false
25081 8165 Acked Write 4 Done Sent Done false
25082 8166 Acked Write 4 Done Sent Done false
25083 8167 Acked Read 4 Done Sent Done false
25084 8168 Acked Write 9 Done Sent Done false
25085 8169 Acked Write 9 Done Sent Done false
25086 8170 Acked Read 1 Done Sent Done false
25087 8171 Acked Read 3 Done Sent Done false
25088 8172 Acked Write 10 Done Sent Done false
25089 8173 Acked Write 8 Done Sent Done false
25090 8174 Acked Read 8 Done Sent Done false
25091 8175 Acked Read 6 Done Sent Done false
25092 8176 Acked Write 10 Done Sent Done false
25093 8177 Acked Read 4 Done Sent Done false
25094 8178 Acked Read 10 Done Sent Done false
25095 8179 Acked Write 9 Done Sent Done false
25096 8180 Acked Flush 0 Done Sent Done false
25097 8181 Acked Read 10 Done Sent Done false
25098 8182 Acked Write 1 Done Sent Done false
25099 8183 Acked Write 1 Done Sent Done false
25100 8184 Acked Write 10 Done Sent Done false
25101 8185 Acked Read 10 Done Sent Done false
25102 8186 Acked Write 7 Done Sent Done false
25103 8187 Acked Write 1 Done Sent Done false
25104 8188 Acked Write 9 Done Sent Done false
25105 8189 Acked Read 8 Done Sent Done false
25106 8190 Acked Write 5 Done Sent Done false
25107 8191 Acked Write 3 Done Sent Done false
25108 8192 Acked Flush 0 Done Sent Done false
25109 8193 Acked Read 1 Done Sent Done false
25110 8194 Acked Write 6 Done Sent Done false
25111 8195 Acked Read 9 Done Sent Done false
25112 8196 Acked Read 3 Done Sent Done false
25113 8197 Acked Write 3 Done Sent Done false
25114 8198 Acked Write 4 Done Sent Done false
25115 8199 Acked Write 5 Done Sent Done false
25116 8200 Acked Write 9 Done Sent Done false
25117 8201 Acked Read 10 Done Sent Done false
25118 8202 Acked Read 4 Done Sent Done false
25119 8203 Acked Read 5 Done Sent Done false
25120 8204 Acked Write 4 Done Sent Done false
25121 8205 Acked Write 6 Done Sent Done false
25122 8206 Acked Write 1 Done Sent Done false
25123 8207 Acked Read 6 Done Sent Done false
25124 8208 Acked Read 2 Done Sent Done false
25125 8209 Acked Flush 0 Done Sent Done false
25126 8210 Acked Read 10 Done Sent Done false
25127 8211 Acked Write 9 Done Sent Done false
25128 8212 Acked Write 6 Done Sent Done false
25129 8213 Acked Read 4 Done Sent Done false
25130 8214 Acked Read 9 Done Sent Done false
25131 8215 Acked Write 8 Done Sent Done false
25132 8216 Acked Read 10 Done Sent Done false
25133 8217 Acked Write 9 Done Sent Done false
25134 8218 Acked Read 6 Done Sent Done false
25135 8219 Acked Write 4 Done Sent Done false
25136 8220 Acked Write 7 Done Sent Done false
25137 8221 Acked Write 7 Done Sent Done false
25138 8222 Acked Write 9 Done Sent Done false
25139 8223 Acked Read 1 Done Sent Done false
25140 8224 Acked Flush 0 Done Sent Done false
25141 8225 Acked Write 10 Done Sent Done false
25142 8226 Acked Read 6 Done Sent Done false
25143 8227 Acked Read 1 Done Sent Done false
25144 8228 Acked Write 9 Done Sent Done false
25145 8229 Acked Read 8 Done Sent Done false
25146 8230 Acked Read 10 Done Sent Done false
25147 8231 Acked Write 4 Done Sent Done false
25148 8232 Acked Write 1 Done Sent Done false
25149 8233 Acked Read 10 Done Sent Done false
25150 8234 Acked Write 6 Done Sent Done false
25151 8235 Acked Write 1 Done Sent Done false
25152 8236 Acked Read 10 Done Sent Done false
25153 8237 Acked Read 2 Done Sent Done false
25154 8238 Acked Flush 0 Done Sent Done false
25155 8239 Acked Read 9 Done Sent Done false
25156 8240 Acked Read 9 Done Sent Done false
25157 8241 Acked Read 2 Done Sent Done false
25158 8242 Acked Read 8 Done Sent Done false
25159 8243 Acked Flush 0 Done Sent Done false
25160 8244 Acked Write 8 Done Sent Done false
25161 STATES DS:0 DS:1 DS:2 TOTAL
25162 Sent 0 301 0 301
25163 Done 301 0 301 602
25164 Skipped 0 0 0 0
25165 Error 0 0 0 0
25166 Last Flush: 8243 7943 8243
25167 Downstairs last five completed: 7943 7942 7941 7940 7939
25168 Upstairs last five completed: 8244 8243 8242 8241 8240
25169 CLIENT: Up:0 ds:301 act:2
25170 [8820] got start action
25171 [8820] I am going to restart
25172 Starting downstairs at port 8820
25173 Make output file at "/tmp/dsc/downstairs-8820.txt"
25174 [8820][1] reports Starting
25175 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1228)
25176 [8820][1] reports Running
25177 ----------------------------------------------------------------
25178 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25179 States: Active Offline Active
25180 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25181 7944 Acked Read 9 Done Sent Done false
25182 7945 Acked Flush 0 Done Sent Done false
25183 7946 Acked Write 2 Done Sent Done false
25184 7947 Acked Write 8 Done Sent Done false
25185 7948 Acked Flush 0 Done Sent Done false
25186 7949 Acked Flush 0 Done Sent Done false
25187 7950 Acked Read 2 Done Sent Done false
25188 7951 Acked Flush 0 Done Sent Done false
25189 7952 Acked Write 3 Done Sent Done false
25190 7953 Acked Flush 0 Done Sent Done false
25191 7954 Acked Read 3 Done Sent Done false
25192 7955 Acked Read 4 Done Sent Done false
25193 7956 Acked Write 2 Done Sent Done false
25194 7957 Acked Read 8 Done Sent Done false
25195 7958 Acked Write 5 Done Sent Done false
25196 7959 Acked Write 4 Done Sent Done false
25197 7960 Acked Read 8 Done Sent Done false
25198 7961 Acked Read 8 Done Sent Done false
25199 7962 Acked Write 8 Done Sent Done false
25200 7963 Acked Write 7 Done Sent Done false
25201 7964 Acked Read 9 Done Sent Done false
25202 7965 Acked Read 7 Done Sent Done false
25203 7966 Acked Write 2 Done Sent Done false
25204 7967 Acked Read 4 Done Sent Done false
25205 7968 Acked Write 4 Done Sent Done false
25206 7969 Acked Write 5 Done Sent Done false
25207 7970 Acked Read 4 Done Sent Done false
25208 7971 Acked Read 4 Done Sent Done false
25209 7972 Acked Read 8 Done Sent Done false
25210 7973 Acked Read 6 Done Sent Done false
25211 7974 Acked Read 8 Done Sent Done false
25212 7975 Acked Read 2 Done Sent Done false
25213 7976 Acked Write 2 Done Sent Done false
25214 7977 Acked Read 5 Done Sent Done false
25215 7978 Acked Flush 0 Done Sent Done false
25216 7979 Acked Write 10 Done Sent Done false
25217 7980 Acked Read 7 Done Sent Done false
25218 7981 Acked Write 2 Done Sent Done false
25219 7982 Acked Flush 0 Done Sent Done false
25220 7983 Acked Flush 0 Done Sent Done false
25221 7984 Acked Flush 0 Done Sent Done false
25222 7985 Acked Read 8 Done Sent Done false
25223 7986 Acked Read 10 Done Sent Done false
25224 7987 Acked Write 2 Done Sent Done false
25225 7988 Acked Read 8 Done Sent Done false
25226 7989 Acked Write 6 Done Sent Done false
25227 7990 Acked Read 5 Done Sent Done false
25228 7991 Acked Read 3 Done Sent Done false
25229 7992 Acked Write 8 Done Sent Done false
25230 7993 Acked Write 1 Done Sent Done false
25231 7994 Acked Flush 0 Done Sent Done false
25232 7995 Acked Write 1 Done Sent Done false
25233 7996 Acked Read 5 Done Sent Done false
25234 7997 Acked Read 5 Done Sent Done false
25235 7998 Acked Read 1 Done Sent Done false
25236 7999 Acked Write 9 Done Sent Done false
25237 8000 Acked Read 4 Done Sent Done false
25238 8001 Acked Read 7 Done Sent Done false
25239 8002 Acked Flush 0 Done Sent Done false
25240 8003 Acked Write 2 Done Sent Done false
25241 8004 Acked Read 7 Done Sent Done false
25242 8005 Acked Write 5 Done Sent Done false
25243 8006 Acked Write 5 Done Sent Done false
25244 8007 Acked Flush 0 Done Sent Done false
25245 8008 Acked Read 7 Done Sent Done false
25246 8009 Acked Read 6 Done Sent Done false
25247 8010 Acked Write 7 Done Sent Done false
25248 8011 Acked Read 5 Done Sent Done false
25249 8012 Acked Write 1 Done Sent Done false
25250 8013 Acked Write 2 Done Sent Done false
25251 8014 Acked Read 3 Done Sent Done false
25252 8015 Acked Read 7 Done Sent Done false
25253 8016 Acked Write 5 Done Sent Done false
25254 8017 Acked Read 2 Done Sent Done false
25255 8018 Acked Read 4 Done Sent Done false
25256 8019 Acked Read 1 Done Sent Done false
25257 8020 Acked Write 10 Done Sent Done false
25258 8021 Acked Read 8 Done Sent Done false
25259 8022 Acked Write 3 Done Sent Done false
25260 8023 Acked Read 2 Done Sent Done false
25261 8024 Acked Read 10 Done Sent Done false
25262 8025 Acked Flush 0 Done Sent Done false
25263 8026 Acked Read 10 Done Sent Done false
25264 8027 Acked Read 4 Done Sent Done false
25265 8028 Acked Write 5 Done Sent Done false
25266 8029 Acked Write 2 Done Sent Done false
25267 8030 Acked Write 7 Done Sent Done false
25268 8031 Acked Read 6 Done Sent Done false
25269 8032 Acked Write 6 Done Sent Done false
25270 8033 Acked Write 1 Done Sent Done false
25271 8034 Acked Flush 0 Done Sent Done false
25272 8035 Acked Read 10 Done Sent Done false
25273 8036 Acked Flush 0 Done Sent Done false
25274 8037 Acked Read 2 Done Sent Done false
25275 8038 Acked Read 6 Done Sent Done false
25276 8039 Acked Write 5 Done Sent Done false
25277 8040 Acked Write 7 Done Sent Done false
25278 8041 Acked Read 3 Done Sent Done false
25279 8042 Acked Write 3 Done Sent Done false
25280 8043 Acked Read 6 Done Sent Done false
25281 8044 Acked Read 5 Done Sent Done false
25282 8045 Acked Write 9 Done Sent Done false
25283 8046 Acked Write 7 Done Sent Done false
25284 8047 Acked Flush 0 Done Sent Done false
25285 8048 Acked Write 5 Done Sent Done false
25286 8049 Acked Read 9 Done Sent Done false
25287 8050 Acked Read 6 Done Sent Done false
25288 8051 Acked Read 3 Done Sent Done false
25289 8052 Acked Read 7 Done Sent Done false
25290 8053 Acked Read 7 Done Sent Done false
25291 8054 Acked Read 5 Done Sent Done false
25292 8055 Acked Read 6 Done Sent Done false
25293 8056 Acked Read 2 Done Sent Done false
25294 8057 Acked Write 7 Done Sent Done false
25295 8058 Acked Read 3 Done Sent Done false
25296 8059 Acked Write 10 Done Sent Done false
25297 8060 Acked Flush 0 Done Sent Done false
25298 8061 Acked Write 1 Done Sent Done false
25299 8062 Acked Read 6 Done Sent Done false
25300 8063 Acked Write 8 Done Sent Done false
25301 8064 Acked Read 8 Done Sent Done false
25302 8065 Acked Write 6 Done Sent Done false
25303 8066 Acked Write 5 Done Sent Done false
25304 8067 Acked Flush 0 Done Sent Done false
25305 8068 Acked Write 5 Done Sent Done false
25306 8069 Acked Write 5 Done Sent Done false
25307 8070 Acked Write 8 Done Sent Done false
25308 8071 Acked Flush 0 Done Sent Done false
25309 8072 Acked Write 8 Done Sent Done false
25310 8073 Acked Write 3 Done Sent Done false
25311 8074 Acked Read 9 Done Sent Done false
25312 8075 Acked Read 2 Done Sent Done false
25313 8076 Acked Write 4 Done Sent Done false
25314 8077 Acked Read 10 Done Sent Done false
25315 8078 Acked Write 2 Done Sent Done false
25316 8079 Acked Read 4 Done Sent Done false
25317 8080 Acked Read 3 Done Sent Done false
25318 8081 Acked Flush 0 Done Sent Done false
25319 8082 Acked Write 5 Done Sent Done false
25320 8083 Acked Flush 0 Done Sent Done false
25321 8084 Acked Read 4 Done Sent Done false
25322 8085 Acked Read 6 Done Sent Done false
25323 8086 Acked Read 5 Done Sent Done false
25324 8087 Acked Flush 0 Done Sent Done false
25325 8088 Acked Write 10 Done Sent Done false
25326 8089 Acked Read 1 Done Sent Done false
25327 8090 Acked Read 1 Done Sent Done false
25328 8091 Acked Write 1 Done Sent Done false
25329 8092 Acked Read 4 Done Sent Done false
25330 8093 Acked Read 10 Done Sent Done false
25331 8094 Acked Read 1 Done Sent Done false
25332 8095 Acked Write 10 Done Sent Done false
25333 8096 Acked Write 7 Done Sent Done false
25334 8097 Acked Read 8 Done Sent Done false
25335 8098 Acked Read 1 Done Sent Done false
25336 8099 Acked Read 8 Done Sent Done false
25337 8100 Acked Write 10 Done Sent Done false
25338 8101 Acked Write 3 Done Sent Done false
25339 8102 Acked Read 9 Done Sent Done false
25340 8103 Acked Write 5 Done Sent Done false
25341 8104 Acked Write 6 Done Sent Done false
25342 8105 Acked Write 6 Done Sent Done false
25343 8106 Acked Flush 0 Done Sent Done false
25344 8107 Acked Read 2 Done Sent Done false
25345 8108 Acked Read 9 Done Sent Done false
25346 8109 Acked Write 2 Done Sent Done false
25347 8110 Acked Flush 0 Done Sent Done false
25348 8111 Acked Write 6 Done Sent Done false
25349 8112 Acked Read 3 Done Sent Done false
25350 8113 Acked Write 9 Done Sent Done false
25351 8114 Acked Write 8 Done Sent Done false
25352 8115 Acked Write 1 Done Sent Done false
25353 8116 Acked Read 7 Done Sent Done false
25354 8117 Acked Read 7 Done Sent Done false
25355 8118 Acked Read 3 Done Sent Done false
25356 8119 Acked Write 8 Done Sent Done false
25357 8120 Acked Read 7 Done Sent Done false
25358 8121 Acked Read 6 Done Sent Done false
25359 8122 Acked Write 2 Done Sent Done false
25360 8123 Acked Flush 0 Done Sent Done false
25361 8124 Acked Read 4 Done Sent Done false
25362 8125 Acked Read 7 Done Sent Done false
25363 8126 Acked Write 6 Done Sent Done false
25364 8127 Acked Flush 0 Done Sent Done false
25365 8128 Acked Write 2 Done Sent Done false
25366 8129 Acked Write 4 Done Sent Done false
25367 8130 Acked Write 9 Done Sent Done false
25368 8131 Acked Read 4 Done Sent Done false
25369 8132 Acked Write 7 Done Sent Done false
25370 8133 Acked Read 4 Done Sent Done false
25371 8134 Acked Write 7 Done Sent Done false
25372 8135 Acked Flush 0 Done Sent Done false
25373 8136 Acked Read 6 Done Sent Done false
25374 8137 Acked Read 10 Done Sent Done false
25375 8138 Acked Read 8 Done Sent Done false
25376 8139 Acked Read 9 Done Sent Done false
25377 8140 Acked Flush 0 Done Sent Done false
25378 8141 Acked Write 3 Done Sent Done false
25379 8142 Acked Write 4 Done Sent Done false
25380 8143 Acked Write 6 Done Sent Done false
25381 8144 Acked Read 1 Done Sent Done false
25382 8145 Acked Write 6 Done Sent Done false
25383 8146 Acked Flush 0 Done Sent Done false
25384 8147 Acked Read 4 Done Sent Done false
25385 8148 Acked Write 2 Done Sent Done false
25386 8149 Acked Write 3 Done Sent Done false
25387 8150 Acked Read 1 Done Sent Done false
25388 8151 Acked Read 9 Done Sent Done false
25389 8152 Acked Read 5 Done Sent Done false
25390 8153 Acked Read 8 Done Sent Done false
25391 8154 Acked Write 9 Done Sent Done false
25392 8155 Acked Read 3 Done Sent Done false
25393 8156 Acked Write 1 Done Sent Done false
25394 8157 Acked Read 3 Done Sent Done false
25395 8158 Acked Read 7 Done Sent Done false
25396 8159 Acked Read 7 Done Sent Done false
25397 8160 Acked Read 2 Done Sent Done false
25398 8161 Acked Read 3 Done Sent Done false
25399 8162 Acked Read 9 Done Sent Done false
25400 8163 Acked Read 5 Done Sent Done false
25401 8164 Acked Read 1 Done Sent Done false
25402 8165 Acked Write 4 Done Sent Done false
25403 8166 Acked Write 4 Done Sent Done false
25404 8167 Acked Read 4 Done Sent Done false
25405 8168 Acked Write 9 Done Sent Done false
25406 8169 Acked Write 9 Done Sent Done false
25407 8170 Acked Read 1 Done Sent Done false
25408 8171 Acked Read 3 Done Sent Done false
25409 8172 Acked Write 10 Done Sent Done false
25410 8173 Acked Write 8 Done Sent Done false
25411 8174 Acked Read 8 Done Sent Done false
25412 8175 Acked Read 6 Done Sent Done false
25413 8176 Acked Write 10 Done Sent Done false
25414 8177 Acked Read 4 Done Sent Done false
25415 8178 Acked Read 10 Done Sent Done false
25416 8179 Acked Write 9 Done Sent Done false
25417 8180 Acked Flush 0 Done Sent Done false
25418 8181 Acked Read 10 Done Sent Done false
25419 8182 Acked Write 1 Done Sent Done false
25420 8183 Acked Write 1 Done Sent Done false
25421 8184 Acked Write 10 Done Sent Done false
25422 8185 Acked Read 10 Done Sent Done false
25423 8186 Acked Write 7 Done Sent Done false
25424 8187 Acked Write 1 Done Sent Done false
25425 8188 Acked Write 9 Done Sent Done false
25426 8189 Acked Read 8 Done Sent Done false
25427 8190 Acked Write 5 Done Sent Done false
25428 8191 Acked Write 3 Done Sent Done false
25429 8192 Acked Flush 0 Done Sent Done false
25430 8193 Acked Read 1 Done Sent Done false
25431 8194 Acked Write 6 Done Sent Done false
25432 8195 Acked Read 9 Done Sent Done false
25433 8196 Acked Read 3 Done Sent Done false
25434 8197 Acked Write 3 Done Sent Done false
25435 8198 Acked Write 4 Done Sent Done false
25436 8199 Acked Write 5 Done Sent Done false
25437 8200 Acked Write 9 Done Sent Done false
25438 8201 Acked Read 10 Done Sent Done false
25439 8202 Acked Read 4 Done Sent Done false
25440 8203 Acked Read 5 Done Sent Done false
25441 8204 Acked Write 4 Done Sent Done false
25442 8205 Acked Write 6 Done Sent Done false
25443 8206 Acked Write 1 Done Sent Done false
25444 8207 Acked Read 6 Done Sent Done false
25445 8208 Acked Read 2 Done Sent Done false
25446 8209 Acked Flush 0 Done Sent Done false
25447 8210 Acked Read 10 Done Sent Done false
25448 8211 Acked Write 9 Done Sent Done false
25449 8212 Acked Write 6 Done Sent Done false
25450 8213 Acked Read 4 Done Sent Done false
25451 8214 Acked Read 9 Done Sent Done false
25452 8215 Acked Write 8 Done Sent Done false
25453 8216 Acked Read 10 Done Sent Done false
25454 8217 Acked Write 9 Done Sent Done false
25455 8218 Acked Read 6 Done Sent Done false
25456 8219 Acked Write 4 Done Sent Done false
25457 8220 Acked Write 7 Done Sent Done false
25458 8221 Acked Write 7 Done Sent Done false
25459 8222 Acked Write 9 Done Sent Done false
25460 8223 Acked Read 1 Done Sent Done false
25461 8224 Acked Flush 0 Done Sent Done false
25462 8225 Acked Write 10 Done Sent Done false
25463 8226 Acked Read 6 Done Sent Done false
25464 8227 Acked Read 1 Done Sent Done false
25465 8228 Acked Write 9 Done Sent Done false
25466 8229 Acked Read 8 Done Sent Done false
25467 8230 Acked Read 10 Done Sent Done false
25468 8231 Acked Write 4 Done Sent Done false
25469 8232 Acked Write 1 Done Sent Done false
25470 8233 Acked Read 10 Done Sent Done false
25471 8234 Acked Write 6 Done Sent Done false
25472 8235 Acked Write 1 Done Sent Done false
25473 8236 Acked Read 10 Done Sent Done false
25474 8237 Acked Read 2 Done Sent Done false
25475 8238 Acked Flush 0 Done Sent Done false
25476 8239 Acked Read 9 Done Sent Done false
25477 8240 Acked Read 9 Done Sent Done false
25478 8241 Acked Read 2 Done Sent Done false
25479 8242 Acked Read 8 Done Sent Done false
25480 8243 Acked Flush 0 Done Sent Done false
25481 8244 Acked Write 8 Done Sent Done false
25482 8245 Acked Flush 0 Done Sent Done false
25483 STATES DS:0 DS:1 DS:2 TOTAL
25484 Sent 0 302 0 302
25485 Done 302 0 302 604
25486 Skipped 0 0 0 0
25487 Error 0 0 0 0
25488 Last Flush: 8245 7943 8245
25489 Downstairs last five completed: 7943 7942 7941 7940 7939
25490 Upstairs last five completed: 8245 8244 8243 8242 8241
25491 CLIENT: Up:0 ds:302 act:2
254922025-01-11T06:10:46.275ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
254932025-01-11T06:10:46.275ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48246) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
254942025-01-11T06:10:46.276ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
254952025-01-11T06:10:46.277ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
254962025-01-11T06:10:46.277ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
254972025-01-11T06:10:46.277ZINFOcrucible: send last flush ID to this DS: 7943 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
254982025-01-11T06:10:46.277ZINFOcrucible: Replied this last flush ID: 7943 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
254992025-01-11T06:10:46.277ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
255002025-01-11T06:10:46.277ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
255012025-01-11T06:10:46.277ZINFOcrucible: [1] client re-new 302 jobs since flush 7943 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
255022025-01-11T06:10:46.277ZINFOcrucible: [1] Replayed 302 jobs since flush: 7943 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
25503 ----------------------------------------------------------------
25504 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25505 Downstairs last five completed: 8245 8244 8243 8242 8241
25506 Upstairs last five completed: 8245 8244 8243 8242 8241
25507 CLIENT: Up:0 ds:0 act:3
25508 Replay: All jobs finished
25509 Main task has work to do, go find it
25510 got dsc Stop(2)
25511 stop 2
25512 Jan 11 06:10:48.522[8830] Got stop action so:false kr:false
25513 INFO request completed, latency_us: 365, response_code: 204, uri: /stop/cid/2, method: POST, req_id: d09f8bea-5b2f-4510-a9fc-f2b7603cdb5f, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
255142025-01-11T06:10:48.523ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
25515 Jan 11 06:10:48.523 INFO request completed, latency_us: 125, response_code: 200, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:10:48.523685895Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
25516 uri: /state/cid/2, {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:10:48.523706522Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
25517 method: GET, req_id: fc6fe5ee-c8a5-457c-b866-bd3a4d9237d9{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:10:48.523741288Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
25518 , remote_addr: 127.0.0.1:59925, local_addr: {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:10:48.523758435Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
25519 127.0.0.1{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:10:48.523787052Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"2","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
25520 :9998
25521 [8830] Exited with: ExitStatus(unix_wait_status(9))
25522 [8830][2] reports Exit
25523 Jan 11 06:10:52.525 INFO request completed, latency_us: 248, response_code: 200, uri: /state/cid/2, method: GET, req_id: 0585efaa-2d4a-4058-b9fb-4125ab119eae, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
25524 001/300 Read block 3132 len 24576
25525 002/300 Read block 651 len 28672
25526 003/300 Write block 700 len 8192 data: 4 3
25527 004/300 Write block 3946 len 16384 data: 4 3 6 6
25528 005/300 Flush
25529 006/300 Read block 1488 len 36864
25530 007/300 Write block 4233 len 12288 data: 3 4 4
25531 008/300 Read block 4181 len 40960
25532 009/300 Write block 481 len 28672 data: 5 4 6 6 5 6 5
25533 010/300 Flush
25534 011/300 Read block 3530 len 40960
25535 012/300 Read block 1657 len 36864
25536 013/300 Read block 2350 len 36864
25537 014/300 Read block 801 len 36864
25538 015/300 Read block 4561 len 12288
25539 016/300 Write block 1081 len 28672 data: 4 2 2 2 2 2 2
25540 017/300 Write block 3215 len 36864 data: 2 3 5 7 6 5 6 6 5
25541 018/300 Read block 3752 len 32768
25542 019/300 Write block 4654 len 32768 data: 3 5 6 6 6 5 3 3
25543 020/300 Read block 794 len 24576
25544 021/300 Read block 3594 len 12288
25545 022/300 Write block 3220 len 36864 data: 6 7 7 6 5 4 3 6 6
25546 023/300 Write block 3320 len 4096 data: 5
25547 024/300 Write block 4885 len 28672 data: 2 2 2 3 4 6 6
25548 025/300 Write block 4947 len 24576 data: 6 7 7 7 7 7
25549 026/300 Read block 486 len 4096
25550 027/300 Read block 162 len 16384
25551 028/300 Write block 577 len 16384 data: 5 6 6 6
25552 029/300 Read block 499 len 32768
25553 030/300 Read block 978 len 20480
25554 031/300 Read block 4729 len 20480
25555 032/300 Write block 4626 len 28672 data: 3 4 4 4 4 6 6
25556 033/300 Write block 4086 len 8192 data: 8 9
25557 034/300 Flush
25558 035/300 Read block 1801 len 40960
25559 036/300 Write block 3477 len 32768 data: 9 7 7 8 7 7 5 6
25560 037/300 Write block 2739 len 28672 data: 6 6 7 6 5 3 4
25561 038/300 Read block 1843 len 28672
25562 039/300 Write block 2700 len 28672 data: 5 6 5 5 5 4 5
25563 040/300 Read block 2091 len 40960
25564 041/300 Read block 2968 len 28672
25565 042/300 Write block 776 len 8192 data: 3 3
25566 043/300 Read block 2755 len 12288
25567 044/300 Read block 1740 len 8192
25568 045/300 Read block 1464 len 40960
25569 046/300 Write block 3800 len 20480 data: 4 4 2 2 2
25570 047/300 Flush
25571 048/300 Read block 687 len 8192
25572 049/300 Flush
25573 050/300 Write block 1893 len 8192 data: 4 4
25574 051/300 Read block 1094 len 16384
25575 052/300 Read block 394 len 8192
25576 053/300 Write block 4222 len 16384 data: 5 5 5 5
25577 054/300 Read block 4880 len 20480
25578 055/300 Write block 3783 len 4096 data: 6
25579 056/300 Read block 3768 len 4096
25580 057/300 Write block 4381 len 24576 data: 2 2 2 2 2 2
25581 058/300 Write block 2367 len 4096 data: 7
25582 059/300 Flush
25583 060/300 Read block 2301 len 36864
25584 061/300 Read block 4440 len 8192
25585 062/300 Write block 4705 len 4096 data: 3
25586 063/300 Read block 2122 len 32768
25587 064/300 Read block 1281 len 12288
25588 065/300 Write block 2002 len 36864 data: 6 5 5 7 6 6 6 5 5
25589 066/300 Read block 1585 len 40960
25590 067/300 Write block 4752 len 24576 data: 5 4 4 2 2 2
25591 068/300 Write block 1943 len 8192 data: 7 6
25592 069/300 Write block 4569 len 40960 data: 6 4 4 4 5 6 5 6 6 6
25593 070/300 Flush
25594 071/300 Read block 3655 len 16384
25595 072/300 Read block 2438 len 4096
25596 073/300 Write block 2934 len 12288 data: 9 11 10
25597 074/300 Write block 886 len 4096 data: 6
25598 075/300 Write block 1886 len 24576 data: 7 7 4 4 4 4
25599 076/300 Read block 1317 len 8192
25600 077/300 Read block 3579 len 20480
25601 078/300 Flush
25602 079/300 Write block 2687 len 32768 data: 4 6 4 4 5 5 5 4
25603 080/300 Write block 2992 len 16384 data: 3 3 2 2
25604 081/300 Write block 1775 len 24576 data: 6 6 5 4 4 6
25605 082/300 Read block 2215 len 32768
25606 083/300 Read block 3224 len 12288
25607 084/300 Write block 1037 len 20480 data: 3 3 2 2 2
25608 085/300 Read block 1672 len 12288
25609 086/300 Write block 877 len 16384 data: 7 7 8 9
25610 087/300 Read block 3942 len 8192
25611 088/300 Write block 960 len 24576 data: 5 5 4 4 5 7
25612 089/300 Write block 1045 len 28672 data: 2 2 2 2 2 2 2
25613 090/300 Write block 355 len 8192 data: 7 7
25614 091/300 Write block 4502 len 24576 data: 3 4 4 5 7 8
25615 092/300 Write block 1641 len 28672 data: 5 4 5 4 4 5 5
25616 093/300 Read block 2812 len 36864
25617 094/300 Read block 4739 len 16384
25618 095/300 Read block 4298 len 16384
25619 096/300 Read block 1956 len 20480
25620 097/300 Write block 4501 len 20480 data: 2 4 5 5 6
25621 098/300 Read block 3081 len 36864
25622 099/300 Read block 1648 len 4096
25623 100/300 Write block 1909 len 12288 data: 3 2 2
25624 101/300 Read block 3126 len 28672
25625 102/300 Write block 645 len 8192 data: 5 4
25626 103/300 Read block 2954 len 36864
25627 104/300 Write block 2181 len 4096 data: 5
25628 105/300 Read block 4741 len 12288
25629 106/300 Read block 1058 len 24576
25630 107/300 Read block 1311 len 24576
25631 108/300 Read block 1104 len 40960
25632 109/300 Write block 4711 len 32768 data: 4 3 3 4 5 5 6 6
25633 110/300 Write block 1128 len 32768 data: 5 4 5 4 4 7 7 6
25634 111/300 Write block 2625 len 24576 data: 7 7 8 8 7 7
25635 112/300 Read block 3801 len 28672
25636 113/300 Read block 2810 len 8192
25637 114/300 Write block 987 len 8192 data: 5 4
25638 115/300 Read block 2083 len 8192
25639 116/300 Flush
25640 117/300 Read block 1102 len 32768
25641 118/300 Read block 2774 len 20480
25642 119/300 Write block 4641 len 40960 data: 3 3 2 2 3 3 4 4 6 5
25643 120/300 Write block 2488 len 36864 data: 2 3 3 4 4 2 2 2 2
25644 121/300 Read block 1103 len 36864
25645 122/300 Flush
25646 123/300 Read block 1422 len 20480
25647 124/300 Read block 4859 len 28672
25648 125/300 Flush
25649 126/300 Write block 4432 len 36864 data: 4 4 4 5 7 6 4 3 3
25650 127/300 Write block 2689 len 12288 data: 5 5 6
25651 128/300 Read block 1273 len 36864
25652 129/300 Read block 3988 len 20480
25653 130/300 Read block 2456 len 36864
25654 131/300 Read block 2719 len 36864
25655 132/300 Flush
25656 133/300 Read block 1030 len 12288
25657 134/300 Read block 2177 len 36864
25658 135/300 Read block 4494 len 36864
25659 136/300 Read block 2125 len 4096
25660 137/300 Flush
25661 138/300 Read block 2429 len 32768
25662 139/300 Write block 2766 len 8192 data: 5 5
25663 140/300 Write block 1962 len 4096 data: 5
25664 141/300 Write block 854 len 16384 data: 6 5 3 3
25665 142/300 Write block 588 len 16384 data: 5 5 4 5
25666 143/300 Read block 3645 len 28672
25667 144/300 Write block 188 len 28672 data: 7 8 8 7 9 8 7
25668 145/300 Read block 2086 len 8192
25669 146/300 Write block 39 len 16384 data: 9 9 11 10
25670 147/300 Write block 1390 len 40960 data: 3 2 3 5 5 5 4 4 5 6
25671 148/300 Read block 218 len 4096
25672 149/300 Write block 3546 len 36864 data: 3 4 4 2 2 2 3 3 2
25673 150/300 Read block 4209 len 12288
25674 151/300 Read block 36 len 12288
25675 152/300 Read block 2757 len 24576
25676 153/300 Write block 2248 len 4096 data: 6
25677 154/300 Read block 651 len 20480
25678 155/300 Read block 1225 len 20480
25679 156/300 Read block 4598 len 24576
25680 157/300 Flush
25681 158/300 Write block 2697 len 12288 data: 5 5 5
25682 159/300 Read block 3731 len 16384
25683 160/300 Write block 4682 len 40960 data: 5 7 7 7 8 7 6 7 6 6
25684 161/300 Read block 1728 len 12288
25685 162/300 Read block 4151 len 28672
25686 163/300 Read block 3174 len 32768
25687 164/300 Read block 3134 len 32768
25688 165/300 Write block 650 len 20480 data: 8 7 7 7 6
25689 166/300 Write block 3048 len 32768 data: 4 4 6 6 6 5 4 3
25690 167/300 Write block 4419 len 4096 data: 3
25691 168/300 Read block 136 len 24576
25692 169/300 Write block 2927 len 12288 data: 6 8 8
25693 170/300 Write block 3472 len 40960 data: 7 10 10 10 10 10 8 8 9 8
25694 171/300 Write block 632 len 28672 data: 7 5 3 2 2 2 2
25695 172/300 Read block 3514 len 32768
25696 173/300 Write block 1450 len 8192 data: 6 4
25697 174/300 Read block 2975 len 24576
25698 175/300 Write block 1150 len 4096 data: 9
25699 176/300 Read block 4354 len 28672
25700 177/300 Read block 735 len 8192
25701 178/300 Read block 1777 len 16384
25702 179/300 Read block 781 len 16384
25703 180/300 Read block 3170 len 20480
25704 181/300 Read block 1546 len 24576
25705 182/300 Write block 767 len 36864 data: 6 7 7 7 6 5 6 3 3
25706 183/300 Write block 1126 len 8192 data: 4 4
25707 184/300 Read block 1620 len 16384
25708 185/300 Read block 3565 len 36864
25709 186/300 Read block 2654 len 16384
25710 187/300 Flush
25711 188/300 Read block 1151 len 12288
25712 189/300 Read block 3425 len 36864
25713 190/300 Flush
25714 191/300 Write block 1493 len 36864 data: 6 7 7 6 9 7 6 5 6
25715 192/300 Read block 2736 len 36864
25716 193/300 Read block 600 len 40960
25717 194/300 Write block 4839 len 28672 data: 3 3 6 4 4 5 5
25718 195/300 Read block 1620 len 40960
25719 196/300 Flush
25720 197/300 Flush
25721 198/300 Write block 2458 len 4096 data: 6
25722 199/300 Write block 1894 len 20480 data: 5 4 5 4 4
25723 200/300 Read block 671 len 36864
25724 201/300 Read block 998 len 16384
25725 202/300 Write block 1993 len 8192 data: 9 8
25726 203/300 Read block 2854 len 40960
25727 204/300 Write block 3083 len 20480 data: 4 5 5 5 6
25728 205/300 Write block 4943 len 16384 data: 8 7 7 6
25729 206/300 Read block 1215 len 20480
25730 207/300 Read block 1467 len 20480
25731 208/300 Write block 4074 len 32768 data: 13 11 11 8 8 6 6 6
25732 209/300 Write block 3756 len 36864 data: 6 5 5 6 6 4 3 4 4
25733 210/300 Read block 2323 len 8192
25734 211/300 Flush
25735 212/300 Flush
25736 213/300 Flush
25737 214/300 Write block 1700 len 20480 data: 8 8 8 8 7
25738 215/300 Read block 2512 len 24576
25739 216/300 Read block 608 len 24576
25740 217/300 Write block 1698 len 40960 data: 6 5 9 9 9 9 8 7 5 5
25741 218/300 Write block 781 len 20480 data: 2 3 3 3 4
25742 219/300 Read block 2391 len 32768
25743 220/300 Write block 1761 len 12288 data: 5 5 5
25744 221/300 Write block 4127 len 20480 data: 8 9 7 8 8
25745 222/300 Write block 2513 len 16384 data: 7 6 5 6
25746 223/300 Read block 1714 len 24576
25747 224/300 Write block 2014 len 28672 data: 7 4 4 5 5 5 5
25748 225/300 Write block 1046 len 36864 data: 3 3 3 3 3 3 2 2 2
25749 226/300 Write block 2975 len 36864 data: 5 3 3 4 4 5 5 4 3
25750 227/300 Read block 4554 len 40960
25751 228/300 Read block 2229 len 40960
25752 229/300 Read block 160 len 8192
25753 230/300 Read block 615 len 24576
25754 231/300 Read block 2718 len 32768
25755 232/300 Read block 681 len 24576
25756 233/300 Read block 1339 len 20480
25757 234/300 Read block 2207 len 24576
25758 235/300 Read block 2192 len 36864
25759 236/300 Write block 2534 len 24576 data: 4 6 6 4 4 4
25760 237/300 Read block 703 len 28672
25761 238/300 Write block 3544 len 4096 data: 3
25762 239/300 Write block 2102 len 4096 data: 9
25763 240/300 Read block 1042 len 16384
25764 241/300 Write block 2772 len 40960 data: 9 7 9 7 7 7 7 5 5 4
25765 242/300 Read block 4442 len 28672
25766 243/300 Read block 379 len 20480
25767 244/300 Read block 3782 len 24576
25768 245/300 Read block 1250 len 28672
25769 246/300 Read block 394 len 40960
25770 247/300 Read block 786 len 32768
25771 248/300 Read block 414 len 36864
25772 249/300 Write block 4497 len 8192 data: 2 2
25773 250/300 Read block 4851 len 36864
25774 251/300 Read block 829 len 32768
25775 252/300 Read block 4032 len 12288
25776 253/300 Read block 2233 len 24576
25777 254/300 Write block 4333 len 32768 data: 5 5 3 3 2 2 2 2
25778 255/300 Write block 3105 len 20480 data: 6 6 7 4 3
25779 256/300 Write block 3116 len 12288 data: 5 4 3
25780 257/300 Write block 1579 len 40960 data: 6 5 4 5 5 4 4 4 3 3
25781 258/300 Read block 2274 len 12288
25782 259/300 Write block 2736 len 12288 data: 7 6 6
25783 260/300 Read block 4621 len 4096
25784 261/300 Read block 1422 len 28672
25785 262/300 Write block 4446 len 32768 data: 4 5 8 8 8 9 9 10
25786 263/300 Write block 2115 len 40960 data: 3 3 4 5 4 2 2 3 4 4
25787 264/300 Flush
25788 265/300 Read block 2717 len 12288
25789 266/300 Write block 3377 len 40960 data: 5 6 6 6 7 7 6 5 4 4
25790 267/300 Write block 3481 len 40960 data: 9 8 6 7 6 6 4 5 5 4
25791 268/300 Write block 3492 len 12288 data: 4 4 4
25792 269/300 Write block 3253 len 20480 data: 4 4 4 4 4
25793 270/300 Read block 112 len 40960
25794 271/300 Read block 4468 len 32768
25795 272/300 Write block 4829 len 20480 data: 6 6 6 6 6
25796 273/300 Flush
25797 274/300 Read block 3380 len 40960
25798 275/300 Read block 2621 len 40960
25799 276/300 Write block 3184 len 20480 data: 5 7 8 7 6
25800 277/300 Read block 4256 len 32768
25801 278/300 Write block 324 len 4096 data: 9
25802 279/300 Read block 2846 len 20480
25803 280/300 Write block 1637 len 24576 data: 5 4 3 4 6 5
25804 281/300 Write block 2317 len 4096 data: 4
25805 282/300 Write block 4788 len 32768 data: 5 5 6 5 7 7 7 7
25806 283/300 Flush
25807 284/300 Read block 1766 len 24576
25808 285/300 Write block 243 len 24576 data: 7 6 6 6 7 7
25809 286/300 Read block 1514 len 24576
25810 287/300 Write block 2397 len 36864 data: 9 7 7 7 7 5 4 4 2
25811 288/300 Flush
25812 289/300 Read block 1957 len 8192
25813 290/300 Flush
25814 291/300 Read block 2808 len 8192
25815 292/300 Read block 4964 len 12288
25816 293/300 Write block 393 len 12288 data: 8 7 6
25817 294/300 Read block 3166 len 20480
25818 295/300 Flush
25819 296/300 Read block 1476 len 40960
25820 297/300 Read block 1385 len 40960
25821 298/300 Write block 1592 len 28672 data: 2 5 5 5 5 5 4
25822 299/300 Read block 594 len 36864
25823 300/300 Write block 3548 len 12288 data: 5 3 3
25824 301/300 Read block 2090 len 24576
25825 Main task has work to do, go find it
25826 got dsc Start(2)
25827 start 2
25828 Jan 11 06:10:52.776 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/2, method: POST, req_id: a46d6ad2-0b71-454b-9b31-f1b1cdae96cd, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
25829 Replay: started 2, returned:Ok(())
25830 ----------------------------------------------------------------
25831 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25832 States: Active Active Offline
25833 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25834 8246 Acked Read 6 Done Done Sent false
25835 8247 Acked Read 7 Done Done Sent false
25836 8248 Acked Write 2 Done Done Sent false
25837 8249 Acked Write 4 Done Done Sent false
25838 8250 Acked Flush 0 Done Done Sent false
25839 8251 Acked Read 9 Done Done Sent false
25840 8252 Acked Write 3 Done Done Sent false
25841 8253 Acked Read 10 Done Done Sent false
25842 8254 Acked Write 7 Done Done Sent false
25843 8255 Acked Flush 0 Done Done Sent false
25844 8256 Acked Read 10 Done Done Sent false
25845 8257 Acked Read 9 Done Done Sent false
25846 8258 Acked Read 9 Done Done Sent false
25847 8259 Acked Read 9 Done Done Sent false
25848 8260 Acked Read 3 Done Done Sent false
25849 8261 Acked Write 7 Done Done Sent false
25850 8262 Acked Write 9 Done Done Sent false
25851 8263 Acked Read 8 Done Done Sent false
25852 8264 Acked Write 8 Done Done Sent false
25853 8265 Acked Read 6 Done Done Sent false
25854 8266 Acked Read 3 Done Done Sent false
25855 8267 Acked Write 9 Done Done Sent false
25856 8268 Acked Write 1 Done Done Sent false
25857 8269 Acked Write 7 Done Done Sent false
25858 8270 Acked Write 6 Done Done Sent false
25859 8271 Acked Read 1 Done Done Sent false
25860 8272 Acked Read 4 Done Done Sent false
25861 8273 Acked Write 4 Done Done Sent false
25862 8274 Acked Read 8 Done Done Sent false
25863 8275 Acked Read 5 Done Done Sent false
25864 8276 Acked Read 5 Done Done Sent false
25865 8277 Acked Write 7 Done Done Sent false
25866 8278 Acked Write 2 Done Done Sent false
25867 8279 Acked Flush 0 Done Done Sent false
25868 8280 Acked Read 10 Done Done Sent false
25869 8281 Acked Write 8 Done Done Sent false
25870 8282 Acked Write 7 Done Done Sent false
25871 8283 Acked Read 7 Done Done Sent false
25872 8284 Acked Write 7 Done Done Sent false
25873 8285 Acked Read 10 Done Done Sent false
25874 8286 Acked Read 7 Done Done Sent false
25875 8287 Acked Write 2 Done Done Sent false
25876 8288 Acked Read 3 Done Done Sent false
25877 8289 Acked Read 2 Done Done Sent false
25878 8290 Acked Read 10 Done Done Sent false
25879 8291 Acked Write 5 Done Done Sent false
25880 8292 Acked Flush 0 Done Done Sent false
25881 8293 Acked Read 2 Done Done Sent false
25882 8294 Acked Flush 0 Done Done Sent false
25883 8295 Acked Write 2 Done Done Sent false
25884 8296 Acked Read 4 Done Done Sent false
25885 8297 Acked Read 2 Done Done Sent false
25886 8298 Acked Write 4 Done Done Sent false
25887 8299 Acked Read 5 Done Done Sent false
25888 8300 Acked Write 1 Done Done Sent false
25889 8301 Acked Read 1 Done Done Sent false
25890 8302 Acked Write 6 Done Done Sent false
25891 8303 Acked Write 1 Done Done Sent false
25892 8304 Acked Flush 0 Done Done Sent false
25893 8305 Acked Read 9 Done Done Sent false
25894 8306 Acked Read 2 Done Done Sent false
25895 8307 Acked Write 1 Done Done Sent false
25896 8308 Acked Read 8 Done Done Sent false
25897 8309 Acked Read 3 Done Done Sent false
25898 8310 Acked Write 9 Done Done Sent false
25899 8311 Acked Read 10 Done Done Sent false
25900 8312 Acked Write 6 Done Done Sent false
25901 8313 Acked Write 2 Done Done Sent false
25902 8314 Acked Write 10 Done Done Sent false
25903 8315 Acked Flush 0 Done Done Sent false
25904 8316 Acked Read 4 Done Done Sent false
25905 8317 Acked Read 1 Done Done Sent false
25906 8318 Acked Write 3 Done Done Sent false
25907 8319 Acked Write 1 Done Done Sent false
25908 8320 Acked Write 6 Done Done Sent false
25909 8321 Acked Read 2 Done Done Sent false
25910 8322 Acked Read 5 Done Done Sent false
25911 8323 Acked Flush 0 Done Done Sent false
25912 8324 Acked Write 8 Done Done Sent false
25913 8325 Acked Write 4 Done Done Sent false
25914 8326 Acked Write 6 Done Done Sent false
25915 8327 Acked Read 8 Done Done Sent false
25916 8328 Acked Read 3 Done Done Sent false
25917 8329 Acked Write 5 Done Done Sent false
25918 8330 Acked Read 3 Done Done Sent false
25919 8331 Acked Write 4 Done Done Sent false
25920 8332 Acked Read 2 Done Done Sent false
25921 8333 Acked Write 6 Done Done Sent false
25922 8334 Acked Write 7 Done Done Sent false
25923 8335 Acked Write 2 Done Done Sent false
25924 8336 Acked Write 6 Done Done Sent false
25925 8337 Acked Write 7 Done Done Sent false
25926 8338 Acked Read 9 Done Done Sent false
25927 8339 Acked Read 4 Done Done Sent false
25928 8340 Acked Read 4 Done Done Sent false
25929 8341 Acked Read 5 Done Done Sent false
25930 8342 Acked Write 5 Done Done Sent false
25931 8343 Acked Read 9 Done Done Sent false
25932 8344 Acked Read 1 Done Done Sent false
25933 8345 Acked Write 3 Done Done Sent false
25934 8346 Acked Read 7 Done Done Sent false
25935 8347 Acked Write 2 Done Done Sent false
25936 8348 Acked Read 9 Done Done Sent false
25937 8349 Acked Write 1 Done Done Sent false
25938 8350 Acked Read 3 Done Done Sent false
25939 8351 Acked Read 6 Done Done Sent false
25940 8352 Acked Read 6 Done Done Sent false
25941 8353 Acked Read 10 Done Done Sent false
25942 8354 Acked Write 8 Done Done Sent false
25943 8355 Acked Write 8 Done Done Sent false
25944 8356 Acked Write 6 Done Done Sent false
25945 8357 Acked Read 7 Done Done Sent false
25946 8358 Acked Read 2 Done Done Sent false
25947 8359 Acked Write 2 Done Done Sent false
25948 8360 Acked Read 2 Done Done Sent false
25949 8361 Acked Flush 0 Done Done Sent false
25950 8362 Acked Read 8 Done Done Sent false
25951 8363 Acked Read 5 Done Done Sent false
25952 8364 Acked Write 10 Done Done Sent false
25953 8365 Acked Write 9 Done Done Sent false
25954 8366 Acked Read 9 Done Done Sent false
25955 8367 Acked Flush 0 Done Done Sent false
25956 8368 Acked Read 5 Done Done Sent false
25957 8369 Acked Read 7 Done Done Sent false
25958 8370 Acked Flush 0 Done Done Sent false
25959 8371 Acked Write 9 Done Done Sent false
25960 8372 Acked Write 3 Done Done Sent false
25961 8373 Acked Read 9 Done Done Sent false
25962 8374 Acked Read 5 Done Done Sent false
25963 8375 Acked Read 9 Done Done Sent false
25964 8376 Acked Read 9 Done Done Sent false
25965 8377 Acked Flush 0 Done Done Sent false
25966 8378 Acked Read 3 Done Done Sent false
25967 8379 Acked Read 9 Done Done Sent false
25968 8380 Acked Read 9 Done Done Sent false
25969 8381 Acked Read 1 Done Done Sent false
25970 8382 Acked Flush 0 Done Done Sent false
25971 8383 Acked Read 8 Done Done Sent false
25972 8384 Acked Write 2 Done Done Sent false
25973 8385 Acked Write 1 Done Done Sent false
25974 8386 Acked Write 4 Done Done Sent false
25975 8387 Acked Write 4 Done Done Sent false
25976 8388 Acked Read 7 Done Done Sent false
25977 8389 Acked Write 7 Done Done Sent false
25978 8390 Acked Read 2 Done Done Sent false
25979 8391 Acked Write 4 Done Done Sent false
25980 8392 Acked Write 10 Done Done Sent false
25981 8393 Acked Read 1 Done Done Sent false
25982 8394 Acked Write 9 Done Done Sent false
25983 8395 Acked Read 3 Done Done Sent false
25984 8396 Acked Read 3 Done Done Sent false
25985 8397 Acked Read 6 Done Done Sent false
25986 8398 Acked Write 1 Done Done Sent false
25987 8399 Acked Read 5 Done Done Sent false
25988 8400 Acked Read 5 Done Done Sent false
25989 8401 Acked Read 6 Done Done Sent false
25990 8402 Acked Flush 0 Done Done Sent false
25991 8403 Acked Write 3 Done Done Sent false
25992 8404 Acked Read 4 Done Done Sent false
25993 8405 Acked Write 10 Done Done Sent false
25994 8406 Acked Read 3 Done Done Sent false
25995 8407 Acked Read 7 Done Done Sent false
25996 8408 Acked Read 8 Done Done Sent false
25997 8409 Acked Read 8 Done Done Sent false
25998 8410 Acked Write 5 Done Done Sent false
25999 8411 Acked Write 8 Done Done Sent false
26000 8412 Acked Write 1 Done Done Sent false
26001 8413 Acked Read 6 Done Done Sent false
26002 8414 Acked Write 3 Done Done Sent false
26003 8415 Acked Write 10 Done Done Sent false
26004 8416 Acked Write 7 Done Done Sent false
26005 8417 Acked Read 8 Done Done Sent false
26006 8418 Acked Write 2 Done Done Sent false
26007 8419 Acked Read 6 Done Done Sent false
26008 8420 Acked Write 1 Done Done Sent false
26009 8421 Acked Read 7 Done Done Sent false
26010 8422 Acked Read 2 Done Done Sent false
26011 8423 Acked Read 4 Done Done Sent false
26012 8424 Acked Read 4 Done Done Sent false
26013 8425 Acked Read 5 Done Done Sent false
26014 8426 Acked Read 6 Done Done Sent false
26015 8427 Acked Write 9 Done Done Sent false
26016 8428 Acked Write 2 Done Done Sent false
26017 8429 Acked Read 4 Done Done Sent false
26018 8430 Acked Read 9 Done Done Sent false
26019 8431 Acked Read 4 Done Done Sent false
26020 8432 Acked Flush 0 Done Done Sent false
26021 8433 Acked Read 3 Done Done Sent false
26022 8434 Acked Read 9 Done Done Sent false
26023 8435 Acked Flush 0 Done Done Sent false
26024 8436 Acked Write 9 Done Done Sent false
26025 8437 Acked Read 9 Done Done Sent false
26026 8438 Acked Read 10 Done Done Sent false
26027 8439 Acked Write 7 Done Done Sent false
26028 8440 Acked Read 10 Done Done Sent false
26029 8441 Acked Flush 0 Done Done Sent false
26030 8442 Acked Flush 0 Done Done Sent false
26031 8443 Acked Write 1 Done Done Sent false
26032 8444 Acked Write 5 Done Done Sent false
26033 8445 Acked Read 9 Done Done Sent false
26034 8446 Acked Read 4 Done Done Sent false
26035 8447 Acked Write 2 Done Done Sent false
26036 8448 Acked Read 10 Done Done Sent false
26037 8449 Acked Write 5 Done Done Sent false
26038 8450 Acked Write 4 Done Done Sent false
26039 8451 Acked Read 5 Done Done Sent false
26040 8452 Acked Read 5 Done Done Sent false
26041 8453 Acked Write 8 Done Done Sent false
26042 8454 Acked Write 9 Done Done Sent false
26043 8455 Acked Read 2 Done Done Sent false
26044 8456 Acked Flush 0 Done Done Sent false
26045 8457 Acked Flush 0 Done Done Sent false
26046 8458 Acked Flush 0 Done Done Sent false
26047 8459 Acked Write 5 Done Done Sent false
26048 8460 Acked Read 6 Done Done Sent false
26049 8461 Acked Read 6 Done Done Sent false
26050 8462 Acked Write 10 Done Done Sent false
26051 8463 Acked Write 5 Done Done Sent false
26052 8464 Acked Read 8 Done Done Sent false
26053 8465 Acked Write 3 Done Done Sent false
26054 8466 Acked Write 5 Done Done Sent false
26055 8467 Acked Write 4 Done Done Sent false
26056 8468 Acked Read 6 Done Done Sent false
26057 8469 Acked Write 7 Done Done Sent false
26058 8470 Acked Write 9 Done Done Sent false
26059 8471 Acked Write 9 Done Done Sent false
26060 8472 Acked Read 10 Done Done Sent false
26061 8473 Acked Read 10 Done Done Sent false
26062 8474 Acked Read 2 Done Done Sent false
26063 8475 Acked Read 6 Done Done Sent false
26064 8476 Acked Read 8 Done Done Sent false
26065 8477 Acked Read 6 Done Done Sent false
26066 8478 Acked Read 5 Done Done Sent false
26067 8479 Acked Read 6 Done Done Sent false
26068 8480 Acked Read 9 Done Done Sent false
26069 8481 Acked Write 6 Done Done Sent false
26070 8482 Acked Read 7 Done Done Sent false
26071 8483 Acked Write 1 Done Done Sent false
26072 8484 Acked Write 1 Done Done Sent false
26073 8485 Acked Read 4 Done Done Sent false
26074 8486 Acked Write 10 Done Done Sent false
26075 8487 Acked Read 7 Done Done Sent false
26076 8488 Acked Read 5 Done Done Sent false
26077 8489 Acked Read 6 Done Done Sent false
26078 8490 Acked Read 7 Done Done Sent false
26079 8491 Acked Read 10 Done Done Sent false
26080 8492 Acked Read 8 Done Done Sent false
26081 8493 Acked Read 9 Done Done Sent false
26082 8494 Acked Write 2 Done Done Sent false
26083 8495 Acked Read 9 Done Done Sent false
26084 8496 Acked Read 8 Done Done Sent false
26085 8497 Acked Read 3 Done Done Sent false
26086 8498 Acked Read 6 Done Done Sent false
26087 8499 Acked Write 8 Done Done Sent false
26088 8500 Acked Write 5 Done Done Sent false
26089 8501 Acked Write 3 Done Done Sent false
26090 8502 Acked Write 10 Done Done Sent false
26091 8503 Acked Read 3 Done Done Sent false
26092 8504 Acked Write 3 Done Done Sent false
26093 8505 Acked Read 1 Done Done Sent false
26094 8506 Acked Read 7 Done Done Sent false
26095 8507 Acked Write 8 Done Done Sent false
26096 8508 Acked Write 10 Done Done Sent false
26097 8509 Acked Flush 0 Done Done Sent false
26098 8510 Acked Read 3 Done Done Sent false
26099 8511 Acked Write 10 Done Done Sent false
26100 8512 Acked Write 10 Done Done Sent false
26101 8513 Acked Write 3 Done Done Sent false
26102 8514 Acked Write 5 Done Done Sent false
26103 8515 Acked Read 10 Done Done Sent false
26104 8516 Acked Read 8 Done Done Sent false
26105 8517 Acked Write 5 Done Done Sent false
26106 8518 Acked Flush 0 Done Done Sent false
26107 8519 Acked Read 10 Done Done Sent false
26108 8520 Acked Read 10 Done Done Sent false
26109 8521 Acked Write 5 Done Done Sent false
26110 8522 Acked Read 8 Done Done Sent false
26111 8523 Acked Write 1 Done Done Sent false
26112 8524 Acked Read 5 Done Done Sent false
26113 8525 Acked Write 6 Done Done Sent false
26114 8526 Acked Write 1 Done Done Sent false
26115 8527 Acked Write 8 Done Done Sent false
26116 8528 Acked Flush 0 Done Done Sent false
26117 8529 Acked Read 6 Done Done Sent false
26118 8530 Acked Write 6 Done Done Sent false
26119 8531 Acked Read 6 Done Done Sent false
26120 8532 Acked Write 9 Done Done Sent false
26121 8533 Acked Flush 0 Done Done Sent false
26122 8534 Acked Read 2 Done Done Sent false
26123 8535 Acked Flush 0 Done Done Sent false
26124 8536 Acked Read 2 Done Done Sent false
26125 8537 Acked Read 3 Done Done Sent false
26126 8538 Acked Write 3 Done Done Sent false
26127 8539 Acked Read 5 Done Done Sent false
26128 8540 Acked Flush 0 Done Done Sent false
26129 8541 Acked Read 10 Done Done Sent false
26130 8542 Acked Read 10 Done Done Sent false
26131 8543 Acked Write 7 Done Done Sent false
26132 8544 Acked Read 9 Done Done Sent false
26133 8545 Acked Write 3 Done Done Sent false
26134 8546 Acked Read 6 Done Done Sent false
26135 STATES DS:0 DS:1 DS:2 TOTAL
26136 Sent 0 0 301 301
26137 Done 301 301 0 602
26138 Skipped 0 0 0 0
26139 Error 0 0 0 0
26140 Last Flush: 8540 8540 8245
26141 Downstairs last five completed: 8245 8244 8243 8242 8241
26142 Upstairs last five completed: 8546 8545 8544 8543 8542
26143 CLIENT: Up:0 ds:301 act:2
26144 [8830] got start action
26145 [8830] I am going to restart
26146 Starting downstairs at port 8830
26147 Make output file at "/tmp/dsc/downstairs-8830.txt"
26148 [8830][2] reports Starting
26149 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1229)
26150 [8830][2] reports Running
26151 ----------------------------------------------------------------
26152 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26153 States: Active Active Offline
26154 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26155 8246 Acked Read 6 Done Done Sent false
26156 8247 Acked Read 7 Done Done Sent false
26157 8248 Acked Write 2 Done Done Sent false
26158 8249 Acked Write 4 Done Done Sent false
26159 8250 Acked Flush 0 Done Done Sent false
26160 8251 Acked Read 9 Done Done Sent false
26161 8252 Acked Write 3 Done Done Sent false
26162 8253 Acked Read 10 Done Done Sent false
26163 8254 Acked Write 7 Done Done Sent false
26164 8255 Acked Flush 0 Done Done Sent false
26165 8256 Acked Read 10 Done Done Sent false
26166 8257 Acked Read 9 Done Done Sent false
26167 8258 Acked Read 9 Done Done Sent false
26168 8259 Acked Read 9 Done Done Sent false
26169 8260 Acked Read 3 Done Done Sent false
26170 8261 Acked Write 7 Done Done Sent false
26171 8262 Acked Write 9 Done Done Sent false
26172 8263 Acked Read 8 Done Done Sent false
26173 8264 Acked Write 8 Done Done Sent false
26174 8265 Acked Read 6 Done Done Sent false
26175 8266 Acked Read 3 Done Done Sent false
26176 8267 Acked Write 9 Done Done Sent false
26177 8268 Acked Write 1 Done Done Sent false
26178 8269 Acked Write 7 Done Done Sent false
26179 8270 Acked Write 6 Done Done Sent false
26180 8271 Acked Read 1 Done Done Sent false
26181 8272 Acked Read 4 Done Done Sent false
26182 8273 Acked Write 4 Done Done Sent false
26183 8274 Acked Read 8 Done Done Sent false
26184 8275 Acked Read 5 Done Done Sent false
26185 8276 Acked Read 5 Done Done Sent false
26186 8277 Acked Write 7 Done Done Sent false
26187 8278 Acked Write 2 Done Done Sent false
26188 8279 Acked Flush 0 Done Done Sent false
26189 8280 Acked Read 10 Done Done Sent false
26190 8281 Acked Write 8 Done Done Sent false
26191 8282 Acked Write 7 Done Done Sent false
26192 8283 Acked Read 7 Done Done Sent false
26193 8284 Acked Write 7 Done Done Sent false
26194 8285 Acked Read 10 Done Done Sent false
26195 8286 Acked Read 7 Done Done Sent false
26196 8287 Acked Write 2 Done Done Sent false
26197 8288 Acked Read 3 Done Done Sent false
26198 8289 Acked Read 2 Done Done Sent false
26199 8290 Acked Read 10 Done Done Sent false
26200 8291 Acked Write 5 Done Done Sent false
26201 8292 Acked Flush 0 Done Done Sent false
26202 8293 Acked Read 2 Done Done Sent false
26203 8294 Acked Flush 0 Done Done Sent false
26204 8295 Acked Write 2 Done Done Sent false
26205 8296 Acked Read 4 Done Done Sent false
26206 8297 Acked Read 2 Done Done Sent false
26207 8298 Acked Write 4 Done Done Sent false
26208 8299 Acked Read 5 Done Done Sent false
26209 8300 Acked Write 1 Done Done Sent false
26210 8301 Acked Read 1 Done Done Sent false
26211 8302 Acked Write 6 Done Done Sent false
26212 8303 Acked Write 1 Done Done Sent false
26213 8304 Acked Flush 0 Done Done Sent false
26214 8305 Acked Read 9 Done Done Sent false
26215 8306 Acked Read 2 Done Done Sent false
26216 8307 Acked Write 1 Done Done Sent false
26217 8308 Acked Read 8 Done Done Sent false
26218 8309 Acked Read 3 Done Done Sent false
26219 8310 Acked Write 9 Done Done Sent false
26220 8311 Acked Read 10 Done Done Sent false
26221 8312 Acked Write 6 Done Done Sent false
26222 8313 Acked Write 2 Done Done Sent false
26223 8314 Acked Write 10 Done Done Sent false
26224 8315 Acked Flush 0 Done Done Sent false
26225 8316 Acked Read 4 Done Done Sent false
26226 8317 Acked Read 1 Done Done Sent false
26227 8318 Acked Write 3 Done Done Sent false
26228 8319 Acked Write 1 Done Done Sent false
26229 8320 Acked Write 6 Done Done Sent false
26230 8321 Acked Read 2 Done Done Sent false
26231 8322 Acked Read 5 Done Done Sent false
26232 8323 Acked Flush 0 Done Done Sent false
26233 8324 Acked Write 8 Done Done Sent false
26234 8325 Acked Write 4 Done Done Sent false
26235 8326 Acked Write 6 Done Done Sent false
26236 8327 Acked Read 8 Done Done Sent false
26237 8328 Acked Read 3 Done Done Sent false
26238 8329 Acked Write 5 Done Done Sent false
26239 8330 Acked Read 3 Done Done Sent false
26240 8331 Acked Write 4 Done Done Sent false
26241 8332 Acked Read 2 Done Done Sent false
26242 8333 Acked Write 6 Done Done Sent false
26243 8334 Acked Write 7 Done Done Sent false
26244 8335 Acked Write 2 Done Done Sent false
26245 8336 Acked Write 6 Done Done Sent false
26246 8337 Acked Write 7 Done Done Sent false
26247 8338 Acked Read 9 Done Done Sent false
26248 8339 Acked Read 4 Done Done Sent false
26249 8340 Acked Read 4 Done Done Sent false
26250 8341 Acked Read 5 Done Done Sent false
26251 8342 Acked Write 5 Done Done Sent false
26252 8343 Acked Read 9 Done Done Sent false
26253 8344 Acked Read 1 Done Done Sent false
26254 8345 Acked Write 3 Done Done Sent false
26255 8346 Acked Read 7 Done Done Sent false
26256 8347 Acked Write 2 Done Done Sent false
26257 8348 Acked Read 9 Done Done Sent false
26258 8349 Acked Write 1 Done Done Sent false
26259 8350 Acked Read 3 Done Done Sent false
26260 8351 Acked Read 6 Done Done Sent false
26261 8352 Acked Read 6 Done Done Sent false
26262 8353 Acked Read 10 Done Done Sent false
26263 8354 Acked Write 8 Done Done Sent false
26264 8355 Acked Write 8 Done Done Sent false
26265 8356 Acked Write 6 Done Done Sent false
26266 8357 Acked Read 7 Done Done Sent false
26267 8358 Acked Read 2 Done Done Sent false
26268 8359 Acked Write 2 Done Done Sent false
26269 8360 Acked Read 2 Done Done Sent false
26270 8361 Acked Flush 0 Done Done Sent false
26271 8362 Acked Read 8 Done Done Sent false
26272 8363 Acked Read 5 Done Done Sent false
26273 8364 Acked Write 10 Done Done Sent false
26274 8365 Acked Write 9 Done Done Sent false
26275 8366 Acked Read 9 Done Done Sent false
26276 8367 Acked Flush 0 Done Done Sent false
26277 8368 Acked Read 5 Done Done Sent false
26278 8369 Acked Read 7 Done Done Sent false
26279 8370 Acked Flush 0 Done Done Sent false
26280 8371 Acked Write 9 Done Done Sent false
26281 8372 Acked Write 3 Done Done Sent false
26282 8373 Acked Read 9 Done Done Sent false
26283 8374 Acked Read 5 Done Done Sent false
26284 8375 Acked Read 9 Done Done Sent false
26285 8376 Acked Read 9 Done Done Sent false
26286 8377 Acked Flush 0 Done Done Sent false
26287 8378 Acked Read 3 Done Done Sent false
26288 8379 Acked Read 9 Done Done Sent false
26289 8380 Acked Read 9 Done Done Sent false
26290 8381 Acked Read 1 Done Done Sent false
26291 8382 Acked Flush 0 Done Done Sent false
26292 8383 Acked Read 8 Done Done Sent false
26293 8384 Acked Write 2 Done Done Sent false
26294 8385 Acked Write 1 Done Done Sent false
26295 8386 Acked Write 4 Done Done Sent false
26296 8387 Acked Write 4 Done Done Sent false
26297 8388 Acked Read 7 Done Done Sent false
26298 8389 Acked Write 7 Done Done Sent false
26299 8390 Acked Read 2 Done Done Sent false
26300 8391 Acked Write 4 Done Done Sent false
26301 8392 Acked Write 10 Done Done Sent false
26302 8393 Acked Read 1 Done Done Sent false
26303 8394 Acked Write 9 Done Done Sent false
26304 8395 Acked Read 3 Done Done Sent false
26305 8396 Acked Read 3 Done Done Sent false
26306 8397 Acked Read 6 Done Done Sent false
26307 8398 Acked Write 1 Done Done Sent false
26308 8399 Acked Read 5 Done Done Sent false
26309 8400 Acked Read 5 Done Done Sent false
26310 8401 Acked Read 6 Done Done Sent false
26311 8402 Acked Flush 0 Done Done Sent false
26312 8403 Acked Write 3 Done Done Sent false
26313 8404 Acked Read 4 Done Done Sent false
26314 8405 Acked Write 10 Done Done Sent false
26315 8406 Acked Read 3 Done Done Sent false
26316 8407 Acked Read 7 Done Done Sent false
26317 8408 Acked Read 8 Done Done Sent false
26318 8409 Acked Read 8 Done Done Sent false
26319 8410 Acked Write 5 Done Done Sent false
26320 8411 Acked Write 8 Done Done Sent false
26321 8412 Acked Write 1 Done Done Sent false
26322 8413 Acked Read 6 Done Done Sent false
26323 8414 Acked Write 3 Done Done Sent false
26324 8415 Acked Write 10 Done Done Sent false
26325 8416 Acked Write 7 Done Done Sent false
26326 8417 Acked Read 8 Done Done Sent false
26327 8418 Acked Write 2 Done Done Sent false
26328 8419 Acked Read 6 Done Done Sent false
26329 8420 Acked Write 1 Done Done Sent false
26330 8421 Acked Read 7 Done Done Sent false
26331 8422 Acked Read 2 Done Done Sent false
26332 8423 Acked Read 4 Done Done Sent false
26333 8424 Acked Read 4 Done Done Sent false
26334 8425 Acked Read 5 Done Done Sent false
26335 8426 Acked Read 6 Done Done Sent false
26336 8427 Acked Write 9 Done Done Sent false
26337 8428 Acked Write 2 Done Done Sent false
26338 8429 Acked Read 4 Done Done Sent false
26339 8430 Acked Read 9 Done Done Sent false
26340 8431 Acked Read 4 Done Done Sent false
26341 8432 Acked Flush 0 Done Done Sent false
26342 8433 Acked Read 3 Done Done Sent false
26343 8434 Acked Read 9 Done Done Sent false
26344 8435 Acked Flush 0 Done Done Sent false
26345 8436 Acked Write 9 Done Done Sent false
26346 8437 Acked Read 9 Done Done Sent false
26347 8438 Acked Read 10 Done Done Sent false
26348 8439 Acked Write 7 Done Done Sent false
26349 8440 Acked Read 10 Done Done Sent false
26350 8441 Acked Flush 0 Done Done Sent false
26351 8442 Acked Flush 0 Done Done Sent false
26352 8443 Acked Write 1 Done Done Sent false
26353 8444 Acked Write 5 Done Done Sent false
26354 8445 Acked Read 9 Done Done Sent false
26355 8446 Acked Read 4 Done Done Sent false
26356 8447 Acked Write 2 Done Done Sent false
26357 8448 Acked Read 10 Done Done Sent false
26358 8449 Acked Write 5 Done Done Sent false
26359 8450 Acked Write 4 Done Done Sent false
26360 8451 Acked Read 5 Done Done Sent false
26361 8452 Acked Read 5 Done Done Sent false
26362 8453 Acked Write 8 Done Done Sent false
26363 8454 Acked Write 9 Done Done Sent false
26364 8455 Acked Read 2 Done Done Sent false
26365 8456 Acked Flush 0 Done Done Sent false
26366 8457 Acked Flush 0 Done Done Sent false
26367 8458 Acked Flush 0 Done Done Sent false
26368 8459 Acked Write 5 Done Done Sent false
26369 8460 Acked Read 6 Done Done Sent false
26370 8461 Acked Read 6 Done Done Sent false
26371 8462 Acked Write 10 Done Done Sent false
26372 8463 Acked Write 5 Done Done Sent false
26373 8464 Acked Read 8 Done Done Sent false
26374 8465 Acked Write 3 Done Done Sent false
26375 8466 Acked Write 5 Done Done Sent false
26376 8467 Acked Write 4 Done Done Sent false
26377 8468 Acked Read 6 Done Done Sent false
26378 8469 Acked Write 7 Done Done Sent false
26379 8470 Acked Write 9 Done Done Sent false
26380 8471 Acked Write 9 Done Done Sent false
26381 8472 Acked Read 10 Done Done Sent false
26382 8473 Acked Read 10 Done Done Sent false
26383 8474 Acked Read 2 Done Done Sent false
26384 8475 Acked Read 6 Done Done Sent false
26385 8476 Acked Read 8 Done Done Sent false
26386 8477 Acked Read 6 Done Done Sent false
26387 8478 Acked Read 5 Done Done Sent false
26388 8479 Acked Read 6 Done Done Sent false
26389 8480 Acked Read 9 Done Done Sent false
26390 8481 Acked Write 6 Done Done Sent false
26391 8482 Acked Read 7 Done Done Sent false
26392 8483 Acked Write 1 Done Done Sent false
26393 8484 Acked Write 1 Done Done Sent false
26394 8485 Acked Read 4 Done Done Sent false
26395 8486 Acked Write 10 Done Done Sent false
26396 8487 Acked Read 7 Done Done Sent false
26397 8488 Acked Read 5 Done Done Sent false
26398 8489 Acked Read 6 Done Done Sent false
26399 8490 Acked Read 7 Done Done Sent false
26400 8491 Acked Read 10 Done Done Sent false
26401 8492 Acked Read 8 Done Done Sent false
26402 8493 Acked Read 9 Done Done Sent false
26403 8494 Acked Write 2 Done Done Sent false
26404 8495 Acked Read 9 Done Done Sent false
26405 8496 Acked Read 8 Done Done Sent false
26406 8497 Acked Read 3 Done Done Sent false
26407 8498 Acked Read 6 Done Done Sent false
26408 8499 Acked Write 8 Done Done Sent false
26409 8500 Acked Write 5 Done Done Sent false
26410 8501 Acked Write 3 Done Done Sent false
26411 8502 Acked Write 10 Done Done Sent false
26412 8503 Acked Read 3 Done Done Sent false
26413 8504 Acked Write 3 Done Done Sent false
26414 8505 Acked Read 1 Done Done Sent false
26415 8506 Acked Read 7 Done Done Sent false
26416 8507 Acked Write 8 Done Done Sent false
26417 8508 Acked Write 10 Done Done Sent false
26418 8509 Acked Flush 0 Done Done Sent false
26419 8510 Acked Read 3 Done Done Sent false
26420 8511 Acked Write 10 Done Done Sent false
26421 8512 Acked Write 10 Done Done Sent false
26422 8513 Acked Write 3 Done Done Sent false
26423 8514 Acked Write 5 Done Done Sent false
26424 8515 Acked Read 10 Done Done Sent false
26425 8516 Acked Read 8 Done Done Sent false
26426 8517 Acked Write 5 Done Done Sent false
26427 8518 Acked Flush 0 Done Done Sent false
26428 8519 Acked Read 10 Done Done Sent false
26429 8520 Acked Read 10 Done Done Sent false
26430 8521 Acked Write 5 Done Done Sent false
26431 8522 Acked Read 8 Done Done Sent false
26432 8523 Acked Write 1 Done Done Sent false
26433 8524 Acked Read 5 Done Done Sent false
26434 8525 Acked Write 6 Done Done Sent false
26435 8526 Acked Write 1 Done Done Sent false
26436 8527 Acked Write 8 Done Done Sent false
26437 8528 Acked Flush 0 Done Done Sent false
26438 8529 Acked Read 6 Done Done Sent false
26439 8530 Acked Write 6 Done Done Sent false
26440 8531 Acked Read 6 Done Done Sent false
26441 8532 Acked Write 9 Done Done Sent false
26442 8533 Acked Flush 0 Done Done Sent false
26443 8534 Acked Read 2 Done Done Sent false
26444 8535 Acked Flush 0 Done Done Sent false
26445 8536 Acked Read 2 Done Done Sent false
26446 8537 Acked Read 3 Done Done Sent false
26447 8538 Acked Write 3 Done Done Sent false
26448 8539 Acked Read 5 Done Done Sent false
26449 8540 Acked Flush 0 Done Done Sent false
26450 8541 Acked Read 10 Done Done Sent false
26451 8542 Acked Read 10 Done Done Sent false
26452 8543 Acked Write 7 Done Done Sent false
26453 8544 Acked Read 9 Done Done Sent false
26454 8545 Acked Write 3 Done Done Sent false
26455 8546 Acked Read 6 Done Done Sent false
26456 8547 Acked Flush 0 Done Done Sent false
26457 STATES DS:0 DS:1 DS:2 TOTAL
26458 Sent 0 0 302 302
26459 Done 302 302 0 604
26460 Skipped 0 0 0 0
26461 Error 0 0 0 0
26462 Last Flush: 8547 8547 8245
26463 Downstairs last five completed: 8245 8244 8243 8242 8241
26464 Upstairs last five completed: 8547 8546 8545 8544 8543
26465 CLIENT: Up:0 ds:302 act:2
264662025-01-11T06:10:58.525ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264672025-01-11T06:10:58.525ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40167) = io task client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264682025-01-11T06:10:58.526ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264692025-01-11T06:10:58.527ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264702025-01-11T06:10:58.527ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264712025-01-11T06:10:58.527ZINFOcrucible: send last flush ID to this DS: 8245 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264722025-01-11T06:10:58.527ZINFOcrucible: Replied this last flush ID: 8245 = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264732025-01-11T06:10:58.527ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264742025-01-11T06:10:58.527ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264752025-01-11T06:10:58.527ZINFOcrucible: [2] client re-new 302 jobs since flush 8245 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264762025-01-11T06:10:58.527ZINFOcrucible: [2] Replayed 302 jobs since flush: 8245 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
26477 ----------------------------------------------------------------
26478 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26479 Downstairs last five completed: 8547 8546 8545 8544 8543
26480 Upstairs last five completed: 8547 8546 8545 8544 8543
26481 CLIENT: Up:0 ds:0 act:3
26482 Replay: All jobs finished
26483 Main task has work to do, go find it
26484 got dsc Stop(1)
26485 stop 1
26486 Jan 11 06:11:00.781 [8820] Got stop action so:false kr:false
26487 INFO request completed, latency_us: 364, response_code: 204, uri: /stop/cid/1, method: POST, req_id: f60f148b-e46e-4657-9b46-9f260a18b658, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
26488 Jan 11 06:11:00.782 INFO request completed, latency_us: 126{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:11:00.782196294Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
26489 , response_code: 200, uri: /state/cid/1, method: GET, req_id: bbb75b43-9c6d-4b20-8d80-dc1343b4e360, remote_addr: 127.0.0.1{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:11:00.782271734Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
26490 :{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:11:00.782318468Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
26491 59925, local_addr: 127.0.0.1:9998
264922025-01-11T06:11:00.782ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264932025-01-11T06:11:00.782ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
264942025-01-11T06:11:00.782ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
26495 [8820] Exited with: ExitStatus(unix_wait_status(9))
26496 [8820][1] reports Exit
26497 Jan 11 06:11:04.784 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/1, method: GET, req_id: d106d02c-cd7b-4b3d-a9b2-660b1b49a72a, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
26498 001/300 Read block 148 len 40960
26499 002/300 Read block 379 len 16384
26500 003/300 Read block 2846 len 28672
26501 004/300 Read block 895 len 16384
26502 005/300 Read block 2257 len 28672
26503 006/300 Read block 4053 len 20480
26504 007/300 Write block 2056 len 32768 data: 7 6 5 6 5 5 5 4
26505 008/300 Read block 3731 len 4096
26506 009/300 Write block 2900 len 12288 data: 6 6 6
26507 010/300 Write block 1959 len 28672 data: 7 5 6 6 4 5 4
26508 011/300 Read block 2789 len 28672
26509 012/300 Write block 4156 len 12288 data: 5 5 5
26510 013/300 Read block 2192 len 8192
26511 014/300 Read block 2808 len 12288
26512 015/300 Flush
26513 016/300 Write block 988 len 32768 data: 5 4 3 2 3 4 5 5
26514 017/300 Flush
26515 018/300 Read block 1490 len 16384
26516 019/300 Write block 1337 len 12288 data: 4 4 4
26517 020/300 Read block 2298 len 4096
26518 021/300 Write block 1053 len 28672 data: 3 3 3 4 3 4 5
26519 022/300 Read block 224 len 16384
26520 023/300 Write block 3910 len 8192 data: 5 5
26521 024/300 Write block 4958 len 8192 data: 5 5
26522 025/300 Read block 329 len 4096
26523 026/300 Write block 794 len 40960 data: 8 7 6 5 4 4 6 5 4 3
26524 027/300 Read block 2796 len 12288
26525 028/300 Write block 2242 len 24576 data: 5 5 6 6 6 5
26526 029/300 Read block 2064 len 28672
26527 030/300 Read block 2148 len 40960
26528 031/300 Write block 2782 len 32768 data: 6 6 7 9 9 8 5 3
26529 032/300 Flush
26530 033/300 Flush
26531 034/300 Read block 4266 len 20480
26532 035/300 Write block 2517 len 32768 data: 5 7 7 9 7 9 7 8
26533 036/300 Write block 2652 len 16384 data: 2 2 2 2
26534 037/300 Read block 1320 len 16384
26535 038/300 Write block 3988 len 8192 data: 3 3
26536 039/300 Read block 1096 len 4096
26537 040/300 Read block 100 len 36864
26538 041/300 Write block 4812 len 28672 data: 3 3 2 2 3 4 3
26539 042/300 Flush
26540 043/300 Write block 4317 len 28672 data: 6 6 7 6 7 6 5
26541 044/300 Read block 2738 len 12288
26542 045/300 Write block 4944 len 24576 data: 8 8 7 7 8 8
26543 046/300 Write block 623 len 4096 data: 4
26544 047/300 Read block 2532 len 12288
26545 048/300 Read block 2602 len 40960
26546 049/300 Write block 1467 len 24576 data: 3 3 3 4 6 5
26547 050/300 Read block 2209 len 24576
26548 051/300 Read block 3570 len 20480
26549 052/300 Write block 3784 len 4096 data: 5
26550 053/300 Write block 4667 len 24576 data: 7 7 7 7 8 7
26551 054/300 Flush
26552 055/300 Read block 1425 len 12288
26553 056/300 Read block 3763 len 16384
26554 057/300 Read block 4577 len 32768
26555 058/300 Read block 3724 len 40960
26556 059/300 Write block 2711 len 36864 data: 7 6 6 5 4 6 7 7 8
26557 060/300 Read block 3263 len 28672
26558 061/300 Write block 1003 len 4096 data: 5
26559 062/300 Write block 305 len 32768 data: 6 5 5 4 5 5 5 6
26560 063/300 Read block 2855 len 20480
26561 064/300 Read block 4215 len 16384
26562 065/300 Read block 302 len 16384
26563 066/300 Write block 1252 len 32768 data: 6 5 6 6 5 6 5 5
26564 067/300 Write block 2119 len 8192 data: 5 3
26565 068/300 Write block 4854 len 32768 data: 2 2 2 2 3 3 3 5
26566 069/300 Read block 4318 len 4096
26567 070/300 Read block 1185 len 16384
26568 071/300 Read block 3122 len 16384
26569 072/300 Read block 719 len 8192
26570 073/300 Read block 920 len 12288
26571 074/300 Read block 4571 len 4096
26572 075/300 Write block 1048 len 16384 data: 4 4 4 4
26573 076/300 Read block 1474 len 32768
26574 077/300 Read block 2290 len 16384
26575 078/300 Write block 622 len 32768 data: 4 5 5 5 6 6 4 4
26576 079/300 Write block 1363 len 24576 data: 7 6 8 9 9 8
26577 080/300 Write block 3981 len 16384 data: 5 4 4 2
26578 081/300 Flush
26579 082/300 Read block 3859 len 32768
26580 083/300 Flush
26581 084/300 Write block 386 len 36864 data: 7 7 7 7 7 7 7 9 8
26582 085/300 Read block 3290 len 16384
26583 086/300 Write block 3155 len 16384 data: 8 7 6 6
26584 087/300 Read block 1501 len 12288
26585 088/300 Write block 809 len 8192 data: 5 5
26586 089/300 Read block 2700 len 8192
26587 090/300 Read block 193 len 16384
26588 091/300 Write block 3586 len 16384 data: 6 4 7 8
26589 092/300 Read block 2012 len 36864
26590 093/300 Read block 489 len 4096
26591 094/300 Write block 2531 len 24576 data: 6 5 5 5 7 7
26592 095/300 Read block 3749 len 36864
26593 096/300 Write block 809 len 24576 data: 6 6 5 5 5 6
26594 097/300 Flush
26595 098/300 Read block 3945 len 40960
26596 099/300 Read block 3094 len 16384
26597 100/300 Read block 1063 len 16384
26598 101/300 Read block 2902 len 28672
26599 102/300 Read block 4384 len 8192
26600 103/300 Write block 2498 len 4096 data: 2
26601 104/300 Read block 1882 len 28672
26602 105/300 Read block 2672 len 32768
26603 106/300 Read block 1681 len 36864
26604 107/300 Write block 3364 len 32768 data: 6 6 9 8 8 8 7 7
26605 108/300 Write block 4070 len 20480 data: 8 9 9 8 14
26606 109/300 Write block 3568 len 36864 data: 5 7 7 7 7 6 5 6 8
26607 110/300 Write block 2782 len 12288 data: 7 7 8
26608 111/300 Read block 3164 len 4096
26609 112/300 Read block 4792 len 8192
26610 113/300 Read block 4508 len 16384
26611 114/300 Write block 433 len 40960 data: 4 4 4 4 5 4 3 4 7 7
26612 115/300 Flush
26613 116/300 Write block 3631 len 24576 data: 6 6 5 3 3 3
26614 117/300 Read block 3759 len 20480
26615 118/300 Write block 1090 len 28672 data: 3 3 6 9 9 10 9
26616 119/300 Read block 3349 len 20480
26617 120/300 Write block 4114 len 40960 data: 4 4 5 4 4 4 3 4 6 4
26618 121/300 Write block 4162 len 12288 data: 7 7 5
26619 122/300 Read block 908 len 24576
26620 123/300 Read block 2041 len 28672
26621 124/300 Write block 184 len 36864 data: 4 4 4 7 8 9 9 8 10
26622 125/300 Flush
26623 126/300 Read block 3225 len 24576
26624 127/300 Flush
26625 128/300 Read block 2472 len 40960
26626 129/300 Write block 602 len 36864 data: 3 3 3 5 4 3 3 4 3
26627 130/300 Read block 3140 len 36864
26628 131/300 Read block 966 len 8192
26629 132/300 Read block 2408 len 16384
26630 133/300 Read block 3188 len 12288
26631 134/300 Read block 1758 len 8192
26632 135/300 Read block 4398 len 20480
26633 136/300 Write block 1997 len 12288 data: 5 3 4
26634 137/300 Read block 463 len 12288
26635 138/300 Read block 2443 len 8192
26636 139/300 Flush
26637 140/300 Flush
26638 141/300 Write block 85 len 20480 data: 5 5 4 4 5
26639 142/300 Read block 2576 len 20480
26640 143/300 Write block 2123 len 28672 data: 5 5 4 5 6 7 6
26641 144/300 Write block 2943 len 4096 data: 6
26642 145/300 Read block 4936 len 36864
26643 146/300 Read block 3770 len 36864
26644 147/300 Write block 3532 len 36864 data: 6 7 6 4 4 4 4 3 3
26645 148/300 Read block 1500 len 28672
26646 149/300 Read block 4010 len 36864
26647 150/300 Read block 1816 len 16384
26648 151/300 Read block 3329 len 16384
26649 152/300 Write block 2155 len 8192 data: 4 4
26650 153/300 Read block 562 len 36864
26651 154/300 Read block 1678 len 40960
26652 155/300 Write block 2186 len 28672 data: 4 5 6 6 4 5 6
26653 156/300 Flush
26654 157/300 Read block 4089 len 24576
26655 158/300 Read block 1182 len 28672
26656 159/300 Read block 3100 len 24576
26657 160/300 Write block 1431 len 24576 data: 6 3 3 3 3 3
26658 161/300 Read block 1240 len 20480
26659 162/300 Read block 2080 len 40960
26660 163/300 Read block 2210 len 24576
26661 164/300 Read block 4545 len 4096
26662 165/300 Write block 1183 len 24576 data: 7 6 4 4 5 5
26663 166/300 Write block 3429 len 12288 data: 6 7 7
26664 167/300 Read block 4168 len 20480
26665 168/300 Write block 4701 len 4096 data: 6
26666 169/300 Flush
26667 170/300 Read block 1418 len 4096
26668 171/300 Flush
26669 172/300 Read block 2982 len 12288
26670 173/300 Flush
26671 174/300 Write block 2620 len 28672 data: 4 4 4 6 7 8 8
26672 175/300 Write block 4612 len 8192 data: 5 5
26673 176/300 Read block 531 len 16384
26674 177/300 Flush
26675 178/300 Write block 2525 len 12288 data: 9 8 9
26676 179/300 Write block 298 len 8192 data: 3 4
26677 180/300 Read block 1678 len 16384
26678 181/300 Read block 4455 len 36864
26679 182/300 Read block 544 len 28672
26680 183/300 Read block 4421 len 12288
26681 184/300 Read block 2097 len 28672
26682 185/300 Write block 1234 len 8192 data: 8 6
26683 186/300 Write block 1662 len 16384 data: 3 4 3 3
26684 187/300 Write block 4560 len 40960 data: 6 6 7 7 6 7 6 5 5 7
26685 188/300 Write block 2144 len 36864 data: 4 3 5 4 4 4 5 6 6
26686 189/300 Read block 4562 len 32768
26687 190/300 Flush
26688 191/300 Write block 3802 len 8192 data: 3 3
26689 192/300 Read block 320 len 40960
26690 193/300 Write block 4707 len 4096 data: 3
26691 194/300 Read block 1178 len 40960
26692 195/300 Read block 3165 len 16384
26693 196/300 Write block 160 len 8192 data: 3 3
26694 197/300 Write block 1911 len 12288 data: 3 2 2
26695 198/300 Write block 2439 len 36864 data: 7 6 6 6 5 6 5 6 6
26696 199/300 Read block 3713 len 40960
26697 200/300 Flush
26698 201/300 Read block 596 len 40960
26699 202/300 Read block 2417 len 4096
26700 203/300 Write block 71 len 28672 data: 5 4 4 3 3 4 3
26701 204/300 Read block 56 len 8192
26702 205/300 Write block 2586 len 16384 data: 3 4 3 4
26703 206/300 Write block 831 len 32768 data: 7 6 5 5 6 7 6 7
26704 207/300 Read block 3487 len 32768
26705 208/300 Write block 2115 len 8192 data: 4 4
26706 209/300 Read block 2699 len 40960
26707 210/300 Read block 3535 len 28672
26708 211/300 Read block 590 len 32768
26709 212/300 Flush
26710 213/300 Write block 2666 len 20480 data: 2 2 2 2 2
26711 214/300 Read block 3227 len 28672
26712 215/300 Write block 2402 len 20480 data: 6 5 5 3 2
26713 216/300 Write block 3006 len 4096 data: 8
26714 217/300 Write block 961 len 12288 data: 6 5 5
26715 218/300 Write block 1836 len 24576 data: 3 3 3 4 4 4
26716 219/300 Write block 3184 len 4096 data: 6
26717 220/300 Read block 2406 len 40960
26718 221/300 Read block 1401 len 16384
26719 222/300 Read block 2214 len 28672
26720 223/300 Read block 978 len 28672
26721 224/300 Write block 3780 len 20480 data: 8 7 8 7 6
26722 225/300 Write block 1454 len 36864 data: 3 3 3 3 3 3 3 3 4
26723 226/300 Read block 2244 len 32768
26724 227/300 Write block 1179 len 40960 data: 9 7 4 5 8 7 5 5 6 6
26725 228/300 Read block 4578 len 24576
26726 229/300 Read block 1077 len 40960
26727 230/300 Flush
26728 231/300 Flush
26729 232/300 Write block 4676 len 20480 data: 5 4 5 5 5
26730 233/300 Write block 1218 len 12288 data: 5 5 7
26731 234/300 Write block 1878 len 4096 data: 4
26732 235/300 Read block 3675 len 20480
26733 236/300 Read block 1772 len 8192
26734 237/300 Write block 1541 len 36864 data: 3 4 4 4 5 4 4 6 6
26735 238/300 Write block 4391 len 4096 data: 2
26736 239/300 Flush
26737 240/300 Read block 562 len 36864
26738 241/300 Read block 877 len 24576
26739 242/300 Read block 3234 len 36864
26740 243/300 Read block 1241 len 32768
26741 244/300 Read block 4791 len 32768
26742 245/300 Read block 1322 len 32768
26743 246/300 Write block 3480 len 24576 data: 10 10 9 7 8 7
26744 247/300 Flush
26745 248/300 Read block 1159 len 32768
26746 249/300 Read block 848 len 24576
26747 250/300 Read block 589 len 24576
26748 251/300 Read block 793 len 32768
26749 252/300 Flush
26750 253/300 Read block 1331 len 40960
26751 254/300 Read block 1771 len 40960
26752 255/300 Read block 114 len 32768
26753 256/300 Write block 2470 len 16384 data: 4 4 5 4
26754 257/300 Write block 4267 len 20480 data: 8 9 9 9 10
26755 258/300 Read block 2205 len 28672
26756 259/300 Read block 2042 len 16384
26757 260/300 Write block 302 len 28672 data: 4 4 6 7 6 6 5
26758 261/300 Read block 3440 len 20480
26759 262/300 Write block 2305 len 32768 data: 5 4 4 4 3 3 2 4
26760 263/300 Read block 1546 len 32768
26761 264/300 Write block 634 len 40960 data: 4 3 3 3 3 4 4 6 6 6
26762 265/300 Write block 1162 len 28672 data: 2 2 2 2 3 3 3
26763 266/300 Read block 1682 len 24576
26764 267/300 Flush
26765 268/300 Read block 4484 len 28672
26766 269/300 Write block 478 len 8192 data: 4 4
26767 270/300 Read block 3134 len 4096
26768 271/300 Read block 3880 len 8192
26769 272/300 Write block 30 len 12288 data: 6 5 6
26770 273/300 Flush
26771 274/300 Read block 375 len 28672
26772 275/300 Write block 1217 len 8192 data: 6 6
26773 276/300 Write block 229 len 16384 data: 7 7 5 5
26774 277/300 Write block 610 len 32768 data: 4 3 2 2 3 4 4 4
26775 278/300 Write block 3147 len 12288 data: 7 7 6
26776 279/300 Write block 4161 len 20480 data: 8 8 8 6 3
26777 280/300 Write block 3490 len 20480 data: 5 4 5 5 5
26778 281/300 Read block 1780 len 8192
26779 282/300 Write block 1843 len 28672 data: 6 7 7 10 10 10 7
26780 283/300 Read block 2035 len 8192
26781 284/300 Write block 1139 len 32768 data: 4 4 4 5 5 7 8 7
26782 285/300 Flush
26783 286/300 Write block 372 len 16384 data: 4 5 6 8
26784 287/300 Read block 351 len 20480
26785 288/300 Read block 473 len 24576
26786 289/300 Read block 26 len 8192
26787 290/300 Write block 3272 len 28672 data: 8 7 7 6 7 8 7
26788 291/300 Write block 1722 len 24576 data: 7 7 7 7 4 4
26789 292/300 Read block 801 len 20480
26790 293/300 Read block 573 len 4096
26791 294/300 Read block 2722 len 40960
26792 295/300 Write block 2185 len 40960 data: 5 5 6 7 7 5 6 7 6 5
26793 296/300 Read block 1316 len 4096
26794 297/300 Read block 4384 len 20480
26795 298/300 Read block 521 len 4096
26796 299/300 Write block 2772 len 8192 data: 10 8
26797 300/300 Flush
26798 301/300 Read block 2374 len 36864
26799 Main task has work to do, go find it
26800 got dsc Start(1)
26801 start 1
26802 Jan 11 06:11:05.030 INFO request completed, latency_us: 226, response_code: 204, uri: /start/cid/1, method: POST, req_id: 49cc6721-526f-40c7-b370-95725004f281, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
26803 Replay: started 1, returned:Ok(())
26804 ----------------------------------------------------------------
26805 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26806 States: Active Offline Active
26807 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26808 8548 Acked Read 10 Done Sent Done false
26809 8549 Acked Read 4 Done Sent Done false
26810 8550 Acked Read 7 Done Sent Done false
26811 8551 Acked Read 4 Done Sent Done false
26812 8552 Acked Read 7 Done Sent Done false
26813 8553 Acked Read 5 Done Sent Done false
26814 8554 Acked Write 8 Done Sent Done false
26815 8555 Acked Read 1 Done Sent Done false
26816 8556 Acked Write 3 Done Sent Done false
26817 8557 Acked Write 7 Done Sent Done false
26818 8558 Acked Read 7 Done Sent Done false
26819 8559 Acked Write 3 Done Sent Done false
26820 8560 Acked Read 2 Done Sent Done false
26821 8561 Acked Read 3 Done Sent Done false
26822 8562 Acked Flush 0 Done Sent Done false
26823 8563 Acked Write 8 Done Sent Done false
26824 8564 Acked Flush 0 Done Sent Done false
26825 8565 Acked Read 4 Done Sent Done false
26826 8566 Acked Write 3 Done Sent Done false
26827 8567 Acked Read 1 Done Sent Done false
26828 8568 Acked Write 7 Done Sent Done false
26829 8569 Acked Read 4 Done Sent Done false
26830 8570 Acked Write 2 Done Sent Done false
26831 8571 Acked Write 2 Done Sent Done false
26832 8572 Acked Read 1 Done Sent Done false
26833 8573 Acked Write 10 Done Sent Done false
26834 8574 Acked Read 3 Done Sent Done false
26835 8575 Acked Write 6 Done Sent Done false
26836 8576 Acked Read 7 Done Sent Done false
26837 8577 Acked Read 10 Done Sent Done false
26838 8578 Acked Write 8 Done Sent Done false
26839 8579 Acked Flush 0 Done Sent Done false
26840 8580 Acked Flush 0 Done Sent Done false
26841 8581 Acked Read 5 Done Sent Done false
26842 8582 Acked Write 8 Done Sent Done false
26843 8583 Acked Write 4 Done Sent Done false
26844 8584 Acked Read 4 Done Sent Done false
26845 8585 Acked Write 2 Done Sent Done false
26846 8586 Acked Read 1 Done Sent Done false
26847 8587 Acked Read 9 Done Sent Done false
26848 8588 Acked Write 7 Done Sent Done false
26849 8589 Acked Flush 0 Done Sent Done false
26850 8590 Acked Write 7 Done Sent Done false
26851 8591 Acked Read 3 Done Sent Done false
26852 8592 Acked Write 6 Done Sent Done false
26853 8593 Acked Write 1 Done Sent Done false
26854 8594 Acked Read 3 Done Sent Done false
26855 8595 Acked Read 10 Done Sent Done false
26856 8596 Acked Write 6 Done Sent Done false
26857 8597 Acked Read 6 Done Sent Done false
26858 8598 Acked Read 5 Done Sent Done false
26859 8599 Acked Write 1 Done Sent Done false
26860 8600 Acked Write 6 Done Sent Done false
26861 8601 Acked Flush 0 Done Sent Done false
26862 8602 Acked Read 3 Done Sent Done false
26863 8603 Acked Read 4 Done Sent Done false
26864 8604 Acked Read 8 Done Sent Done false
26865 8605 Acked Read 10 Done Sent Done false
26866 8606 Acked Write 9 Done Sent Done false
26867 8607 Acked Read 7 Done Sent Done false
26868 8608 Acked Write 1 Done Sent Done false
26869 8609 Acked Write 8 Done Sent Done false
26870 8610 Acked Read 5 Done Sent Done false
26871 8611 Acked Read 4 Done Sent Done false
26872 8612 Acked Read 4 Done Sent Done false
26873 8613 Acked Write 8 Done Sent Done false
26874 8614 Acked Write 2 Done Sent Done false
26875 8615 Acked Write 8 Done Sent Done false
26876 8616 Acked Read 1 Done Sent Done false
26877 8617 Acked Read 4 Done Sent Done false
26878 8618 Acked Read 4 Done Sent Done false
26879 8619 Acked Read 2 Done Sent Done false
26880 8620 Acked Read 3 Done Sent Done false
26881 8621 Acked Read 1 Done Sent Done false
26882 8622 Acked Write 4 Done Sent Done false
26883 8623 Acked Read 8 Done Sent Done false
26884 8624 Acked Read 4 Done Sent Done false
26885 8625 Acked Write 8 Done Sent Done false
26886 8626 Acked Write 6 Done Sent Done false
26887 8627 Acked Write 4 Done Sent Done false
26888 8628 Acked Flush 0 Done Sent Done false
26889 8629 Acked Read 8 Done Sent Done false
26890 8630 Acked Flush 0 Done Sent Done false
26891 8631 Acked Write 9 Done Sent Done false
26892 8632 Acked Read 4 Done Sent Done false
26893 8633 Acked Write 4 Done Sent Done false
26894 8634 Acked Read 3 Done Sent Done false
26895 8635 Acked Write 2 Done Sent Done false
26896 8636 Acked Read 2 Done Sent Done false
26897 8637 Acked Read 4 Done Sent Done false
26898 8638 Acked Write 4 Done Sent Done false
26899 8639 Acked Read 9 Done Sent Done false
26900 8640 Acked Read 1 Done Sent Done false
26901 8641 Acked Write 6 Done Sent Done false
26902 8642 Acked Read 9 Done Sent Done false
26903 8643 Acked Write 6 Done Sent Done false
26904 8644 Acked Flush 0 Done Sent Done false
26905 8645 Acked Read 10 Done Sent Done false
26906 8646 Acked Read 4 Done Sent Done false
26907 8647 Acked Read 4 Done Sent Done false
26908 8648 Acked Read 7 Done Sent Done false
26909 8649 Acked Read 2 Done Sent Done false
26910 8650 Acked Write 1 Done Sent Done false
26911 8651 Acked Read 7 Done Sent Done false
26912 8652 Acked Read 8 Done Sent Done false
26913 8653 Acked Read 9 Done Sent Done false
26914 8654 Acked Write 8 Done Sent Done false
26915 8655 Acked Write 5 Done Sent Done false
26916 8656 Acked Write 9 Done Sent Done false
26917 8657 Acked Write 3 Done Sent Done false
26918 8658 Acked Read 1 Done Sent Done false
26919 8659 Acked Read 2 Done Sent Done false
26920 8660 Acked Read 4 Done Sent Done false
26921 8661 Acked Write 10 Done Sent Done false
26922 8662 Acked Flush 0 Done Sent Done false
26923 8663 Acked Write 6 Done Sent Done false
26924 8664 Acked Read 5 Done Sent Done false
26925 8665 Acked Write 7 Done Sent Done false
26926 8666 Acked Read 5 Done Sent Done false
26927 8667 Acked Write 10 Done Sent Done false
26928 8668 Acked Write 3 Done Sent Done false
26929 8669 Acked Read 6 Done Sent Done false
26930 8670 Acked Read 7 Done Sent Done false
26931 8671 Acked Write 9 Done Sent Done false
26932 8672 Acked Flush 0 Done Sent Done false
26933 8673 Acked Read 6 Done Sent Done false
26934 8674 Acked Flush 0 Done Sent Done false
26935 8675 Acked Read 10 Done Sent Done false
26936 8676 Acked Write 9 Done Sent Done false
26937 8677 Acked Read 9 Done Sent Done false
26938 8678 Acked Read 2 Done Sent Done false
26939 8679 Acked Read 4 Done Sent Done false
26940 8680 Acked Read 3 Done Sent Done false
26941 8681 Acked Read 2 Done Sent Done false
26942 8682 Acked Read 5 Done Sent Done false
26943 8683 Acked Write 3 Done Sent Done false
26944 8684 Acked Read 3 Done Sent Done false
26945 8685 Acked Read 2 Done Sent Done false
26946 8686 Acked Flush 0 Done Sent Done false
26947 8687 Acked Flush 0 Done Sent Done false
26948 8688 Acked Write 5 Done Sent Done false
26949 8689 Acked Read 5 Done Sent Done false
26950 8690 Acked Write 7 Done Sent Done false
26951 8691 Acked Write 1 Done Sent Done false
26952 8692 Acked Read 9 Done Sent Done false
26953 8693 Acked Read 9 Done Sent Done false
26954 8694 Acked Write 9 Done Sent Done false
26955 8695 Acked Read 7 Done Sent Done false
26956 8696 Acked Read 9 Done Sent Done false
26957 8697 Acked Read 4 Done Sent Done false
26958 8698 Acked Read 4 Done Sent Done false
26959 8699 Acked Write 2 Done Sent Done false
26960 8700 Acked Read 9 Done Sent Done false
26961 8701 Acked Read 10 Done Sent Done false
26962 8702 Acked Write 7 Done Sent Done false
26963 8703 Acked Flush 0 Done Sent Done false
26964 8704 Acked Read 6 Done Sent Done false
26965 8705 Acked Read 7 Done Sent Done false
26966 8706 Acked Read 6 Done Sent Done false
26967 8707 Acked Write 6 Done Sent Done false
26968 8708 Acked Read 5 Done Sent Done false
26969 8709 Acked Read 10 Done Sent Done false
26970 8710 Acked Read 6 Done Sent Done false
26971 8711 Acked Read 1 Done Sent Done false
26972 8712 Acked Write 6 Done Sent Done false
26973 8713 Acked Write 3 Done Sent Done false
26974 8714 Acked Read 5 Done Sent Done false
26975 8715 Acked Write 1 Done Sent Done false
26976 8716 Acked Flush 0 Done Sent Done false
26977 8717 Acked Read 1 Done Sent Done false
26978 8718 Acked Flush 0 Done Sent Done false
26979 8719 Acked Read 3 Done Sent Done false
26980 8720 Acked Flush 0 Done Sent Done false
26981 8721 Acked Write 7 Done Sent Done false
26982 8722 Acked Write 2 Done Sent Done false
26983 8723 Acked Read 4 Done Sent Done false
26984 8724 Acked Flush 0 Done Sent Done false
26985 8725 Acked Write 3 Done Sent Done false
26986 8726 Acked Write 2 Done Sent Done false
26987 8727 Acked Read 4 Done Sent Done false
26988 8728 Acked Read 9 Done Sent Done false
26989 8729 Acked Read 7 Done Sent Done false
26990 8730 Acked Read 3 Done Sent Done false
26991 8731 Acked Read 7 Done Sent Done false
26992 8732 Acked Write 2 Done Sent Done false
26993 8733 Acked Write 4 Done Sent Done false
26994 8734 Acked Write 10 Done Sent Done false
26995 8735 Acked Write 9 Done Sent Done false
26996 8736 Acked Read 8 Done Sent Done false
26997 8737 Acked Flush 0 Done Sent Done false
26998 8738 Acked Write 2 Done Sent Done false
26999 8739 Acked Read 10 Done Sent Done false
27000 8740 Acked Write 1 Done Sent Done false
27001 8741 Acked Read 10 Done Sent Done false
27002 8742 Acked Read 4 Done Sent Done false
27003 8743 Acked Write 2 Done Sent Done false
27004 8744 Acked Write 3 Done Sent Done false
27005 8745 Acked Write 9 Done Sent Done false
27006 8746 Acked Read 10 Done Sent Done false
27007 8747 Acked Flush 0 Done Sent Done false
27008 8748 Acked Read 10 Done Sent Done false
27009 8749 Acked Read 1 Done Sent Done false
27010 8750 Acked Write 7 Done Sent Done false
27011 8751 Acked Read 2 Done Sent Done false
27012 8752 Acked Write 4 Done Sent Done false
27013 8753 Acked Write 8 Done Sent Done false
27014 8754 Acked Read 8 Done Sent Done false
27015 8755 Acked Write 2 Done Sent Done false
27016 8756 Acked Read 10 Done Sent Done false
27017 8757 Acked Read 7 Done Sent Done false
27018 8758 Acked Read 8 Done Sent Done false
27019 8759 Acked Flush 0 Done Sent Done false
27020 8760 Acked Write 5 Done Sent Done false
27021 8761 Acked Read 7 Done Sent Done false
27022 8762 Acked Write 5 Done Sent Done false
27023 8763 Acked Write 1 Done Sent Done false
27024 8764 Acked Write 3 Done Sent Done false
27025 8765 Acked Write 6 Done Sent Done false
27026 8766 Acked Write 1 Done Sent Done false
27027 8767 Acked Read 10 Done Sent Done false
27028 8768 Acked Read 4 Done Sent Done false
27029 8769 Acked Read 7 Done Sent Done false
27030 8770 Acked Read 7 Done Sent Done false
27031 8771 Acked Write 5 Done Sent Done false
27032 8772 Acked Write 9 Done Sent Done false
27033 8773 Acked Read 8 Done Sent Done false
27034 8774 Acked Write 10 Done Sent Done false
27035 8775 Acked Read 6 Done Sent Done false
27036 8776 Acked Read 10 Done Sent Done false
27037 8777 Acked Flush 0 Done Sent Done false
27038 8778 Acked Flush 0 Done Sent Done false
27039 8779 Acked Write 5 Done Sent Done false
27040 8780 Acked Write 3 Done Sent Done false
27041 8781 Acked Write 1 Done Sent Done false
27042 8782 Acked Read 5 Done Sent Done false
27043 8783 Acked Read 2 Done Sent Done false
27044 8784 Acked Write 9 Done Sent Done false
27045 8785 Acked Write 1 Done Sent Done false
27046 8786 Acked Flush 0 Done Sent Done false
27047 8787 Acked Read 9 Done Sent Done false
27048 8788 Acked Read 6 Done Sent Done false
27049 8789 Acked Read 9 Done Sent Done false
27050 8790 Acked Read 8 Done Sent Done false
27051 8791 Acked Read 8 Done Sent Done false
27052 8792 Acked Read 8 Done Sent Done false
27053 8793 Acked Write 6 Done Sent Done false
27054 8794 Acked Flush 0 Done Sent Done false
27055 8795 Acked Read 8 Done Sent Done false
27056 8796 Acked Read 6 Done Sent Done false
27057 8797 Acked Read 6 Done Sent Done false
27058 8798 Acked Read 8 Done Sent Done false
27059 8799 Acked Flush 0 Done Sent Done false
27060 8800 Acked Read 10 Done Sent Done false
27061 8801 Acked Read 10 Done Sent Done false
27062 8802 Acked Read 8 Done Sent Done false
27063 8803 Acked Write 4 Done Sent Done false
27064 8804 Acked Write 5 Done Sent Done false
27065 8805 Acked Read 7 Done Sent Done false
27066 8806 Acked Read 4 Done Sent Done false
27067 8807 Acked Write 7 Done Sent Done false
27068 8808 Acked Read 5 Done Sent Done false
27069 8809 Acked Write 8 Done Sent Done false
27070 8810 Acked Read 8 Done Sent Done false
27071 8811 Acked Write 10 Done Sent Done false
27072 8812 Acked Write 7 Done Sent Done false
27073 8813 Acked Read 6 Done Sent Done false
27074 8814 Acked Flush 0 Done Sent Done false
27075 8815 Acked Read 7 Done Sent Done false
27076 8816 Acked Write 2 Done Sent Done false
27077 8817 Acked Read 1 Done Sent Done false
27078 8818 Acked Read 2 Done Sent Done false
27079 8819 Acked Write 3 Done Sent Done false
27080 8820 Acked Flush 0 Done Sent Done false
27081 8821 Acked Read 7 Done Sent Done false
27082 8822 Acked Write 2 Done Sent Done false
27083 8823 Acked Write 4 Done Sent Done false
27084 8824 Acked Write 8 Done Sent Done false
27085 8825 Acked Write 3 Done Sent Done false
27086 8826 Acked Write 5 Done Sent Done false
27087 8827 Acked Write 5 Done Sent Done false
27088 8828 Acked Read 2 Done Sent Done false
27089 8829 Acked Write 7 Done Sent Done false
27090 8830 Acked Read 2 Done Sent Done false
27091 8831 Acked Write 8 Done Sent Done false
27092 8832 Acked Flush 0 Done Sent Done false
27093 8833 Acked Write 4 Done Sent Done false
27094 8834 Acked Read 5 Done Sent Done false
27095 8835 Acked Read 6 Done Sent Done false
27096 8836 Acked Read 2 Done Sent Done false
27097 8837 Acked Write 7 Done Sent Done false
27098 8838 Acked Write 6 Done Sent Done false
27099 8839 Acked Read 5 Done Sent Done false
27100 8840 Acked Read 1 Done Sent Done false
27101 8841 Acked Read 10 Done Sent Done false
27102 8842 Acked Write 10 Done Sent Done false
27103 8843 Acked Read 1 Done Sent Done false
27104 8844 Acked Read 5 Done Sent Done false
27105 8845 Acked Read 1 Done Sent Done false
27106 8846 Acked Write 2 Done Sent Done false
27107 8847 Acked Flush 0 Done Sent Done false
27108 8848 Acked Read 9 Done Sent Done false
27109 STATES DS:0 DS:1 DS:2 TOTAL
27110 Sent 0 301 0 301
27111 Done 301 0 301 602
27112 Skipped 0 0 0 0
27113 Error 0 0 0 0
27114 Last Flush: 8847 8547 8847
27115 Downstairs last five completed: 8547 8546 8545 8544 8543
27116 Upstairs last five completed: 8848 8847 8846 8845 8844
27117 CLIENT: Up:0 ds:301 act:2
27118 [8820] got start action
27119 [8820] I am going to restart
27120 Starting downstairs at port 8820
27121 Make output file at "/tmp/dsc/downstairs-8820.txt"
27122 [8820][1] reports Starting
27123 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1230)
27124 [8820][1] reports Running
27125 ----------------------------------------------------------------
27126 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27127 States: Active Offline Active
27128 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27129 8548 Acked Read 10 Done Sent Done false
27130 8549 Acked Read 4 Done Sent Done false
27131 8550 Acked Read 7 Done Sent Done false
27132 8551 Acked Read 4 Done Sent Done false
27133 8552 Acked Read 7 Done Sent Done false
27134 8553 Acked Read 5 Done Sent Done false
27135 8554 Acked Write 8 Done Sent Done false
27136 8555 Acked Read 1 Done Sent Done false
27137 8556 Acked Write 3 Done Sent Done false
27138 8557 Acked Write 7 Done Sent Done false
27139 8558 Acked Read 7 Done Sent Done false
27140 8559 Acked Write 3 Done Sent Done false
27141 8560 Acked Read 2 Done Sent Done false
27142 8561 Acked Read 3 Done Sent Done false
27143 8562 Acked Flush 0 Done Sent Done false
27144 8563 Acked Write 8 Done Sent Done false
27145 8564 Acked Flush 0 Done Sent Done false
27146 8565 Acked Read 4 Done Sent Done false
27147 8566 Acked Write 3 Done Sent Done false
27148 8567 Acked Read 1 Done Sent Done false
27149 8568 Acked Write 7 Done Sent Done false
27150 8569 Acked Read 4 Done Sent Done false
27151 8570 Acked Write 2 Done Sent Done false
27152 8571 Acked Write 2 Done Sent Done false
27153 8572 Acked Read 1 Done Sent Done false
27154 8573 Acked Write 10 Done Sent Done false
27155 8574 Acked Read 3 Done Sent Done false
27156 8575 Acked Write 6 Done Sent Done false
27157 8576 Acked Read 7 Done Sent Done false
27158 8577 Acked Read 10 Done Sent Done false
27159 8578 Acked Write 8 Done Sent Done false
27160 8579 Acked Flush 0 Done Sent Done false
27161 8580 Acked Flush 0 Done Sent Done false
27162 8581 Acked Read 5 Done Sent Done false
27163 8582 Acked Write 8 Done Sent Done false
27164 8583 Acked Write 4 Done Sent Done false
27165 8584 Acked Read 4 Done Sent Done false
27166 8585 Acked Write 2 Done Sent Done false
27167 8586 Acked Read 1 Done Sent Done false
27168 8587 Acked Read 9 Done Sent Done false
27169 8588 Acked Write 7 Done Sent Done false
27170 8589 Acked Flush 0 Done Sent Done false
27171 8590 Acked Write 7 Done Sent Done false
27172 8591 Acked Read 3 Done Sent Done false
27173 8592 Acked Write 6 Done Sent Done false
27174 8593 Acked Write 1 Done Sent Done false
27175 8594 Acked Read 3 Done Sent Done false
27176 8595 Acked Read 10 Done Sent Done false
27177 8596 Acked Write 6 Done Sent Done false
27178 8597 Acked Read 6 Done Sent Done false
27179 8598 Acked Read 5 Done Sent Done false
27180 8599 Acked Write 1 Done Sent Done false
27181 8600 Acked Write 6 Done Sent Done false
27182 8601 Acked Flush 0 Done Sent Done false
27183 8602 Acked Read 3 Done Sent Done false
27184 8603 Acked Read 4 Done Sent Done false
27185 8604 Acked Read 8 Done Sent Done false
27186 8605 Acked Read 10 Done Sent Done false
27187 8606 Acked Write 9 Done Sent Done false
27188 8607 Acked Read 7 Done Sent Done false
27189 8608 Acked Write 1 Done Sent Done false
27190 8609 Acked Write 8 Done Sent Done false
27191 8610 Acked Read 5 Done Sent Done false
27192 8611 Acked Read 4 Done Sent Done false
27193 8612 Acked Read 4 Done Sent Done false
27194 8613 Acked Write 8 Done Sent Done false
27195 8614 Acked Write 2 Done Sent Done false
27196 8615 Acked Write 8 Done Sent Done false
27197 8616 Acked Read 1 Done Sent Done false
27198 8617 Acked Read 4 Done Sent Done false
27199 8618 Acked Read 4 Done Sent Done false
27200 8619 Acked Read 2 Done Sent Done false
27201 8620 Acked Read 3 Done Sent Done false
27202 8621 Acked Read 1 Done Sent Done false
27203 8622 Acked Write 4 Done Sent Done false
27204 8623 Acked Read 8 Done Sent Done false
27205 8624 Acked Read 4 Done Sent Done false
27206 8625 Acked Write 8 Done Sent Done false
27207 8626 Acked Write 6 Done Sent Done false
27208 8627 Acked Write 4 Done Sent Done false
27209 8628 Acked Flush 0 Done Sent Done false
27210 8629 Acked Read 8 Done Sent Done false
27211 8630 Acked Flush 0 Done Sent Done false
27212 8631 Acked Write 9 Done Sent Done false
27213 8632 Acked Read 4 Done Sent Done false
27214 8633 Acked Write 4 Done Sent Done false
27215 8634 Acked Read 3 Done Sent Done false
27216 8635 Acked Write 2 Done Sent Done false
27217 8636 Acked Read 2 Done Sent Done false
27218 8637 Acked Read 4 Done Sent Done false
27219 8638 Acked Write 4 Done Sent Done false
27220 8639 Acked Read 9 Done Sent Done false
27221 8640 Acked Read 1 Done Sent Done false
27222 8641 Acked Write 6 Done Sent Done false
27223 8642 Acked Read 9 Done Sent Done false
27224 8643 Acked Write 6 Done Sent Done false
27225 8644 Acked Flush 0 Done Sent Done false
27226 8645 Acked Read 10 Done Sent Done false
27227 8646 Acked Read 4 Done Sent Done false
27228 8647 Acked Read 4 Done Sent Done false
27229 8648 Acked Read 7 Done Sent Done false
27230 8649 Acked Read 2 Done Sent Done false
27231 8650 Acked Write 1 Done Sent Done false
27232 8651 Acked Read 7 Done Sent Done false
27233 8652 Acked Read 8 Done Sent Done false
27234 8653 Acked Read 9 Done Sent Done false
27235 8654 Acked Write 8 Done Sent Done false
27236 8655 Acked Write 5 Done Sent Done false
27237 8656 Acked Write 9 Done Sent Done false
27238 8657 Acked Write 3 Done Sent Done false
27239 8658 Acked Read 1 Done Sent Done false
27240 8659 Acked Read 2 Done Sent Done false
27241 8660 Acked Read 4 Done Sent Done false
27242 8661 Acked Write 10 Done Sent Done false
27243 8662 Acked Flush 0 Done Sent Done false
27244 8663 Acked Write 6 Done Sent Done false
27245 8664 Acked Read 5 Done Sent Done false
27246 8665 Acked Write 7 Done Sent Done false
27247 8666 Acked Read 5 Done Sent Done false
27248 8667 Acked Write 10 Done Sent Done false
27249 8668 Acked Write 3 Done Sent Done false
27250 8669 Acked Read 6 Done Sent Done false
27251 8670 Acked Read 7 Done Sent Done false
27252 8671 Acked Write 9 Done Sent Done false
27253 8672 Acked Flush 0 Done Sent Done false
27254 8673 Acked Read 6 Done Sent Done false
27255 8674 Acked Flush 0 Done Sent Done false
27256 8675 Acked Read 10 Done Sent Done false
27257 8676 Acked Write 9 Done Sent Done false
27258 8677 Acked Read 9 Done Sent Done false
27259 8678 Acked Read 2 Done Sent Done false
27260 8679 Acked Read 4 Done Sent Done false
27261 8680 Acked Read 3 Done Sent Done false
27262 8681 Acked Read 2 Done Sent Done false
27263 8682 Acked Read 5 Done Sent Done false
27264 8683 Acked Write 3 Done Sent Done false
27265 8684 Acked Read 3 Done Sent Done false
27266 8685 Acked Read 2 Done Sent Done false
27267 8686 Acked Flush 0 Done Sent Done false
27268 8687 Acked Flush 0 Done Sent Done false
27269 8688 Acked Write 5 Done Sent Done false
27270 8689 Acked Read 5 Done Sent Done false
27271 8690 Acked Write 7 Done Sent Done false
27272 8691 Acked Write 1 Done Sent Done false
27273 8692 Acked Read 9 Done Sent Done false
27274 8693 Acked Read 9 Done Sent Done false
27275 8694 Acked Write 9 Done Sent Done false
27276 8695 Acked Read 7 Done Sent Done false
27277 8696 Acked Read 9 Done Sent Done false
27278 8697 Acked Read 4 Done Sent Done false
27279 8698 Acked Read 4 Done Sent Done false
27280 8699 Acked Write 2 Done Sent Done false
27281 8700 Acked Read 9 Done Sent Done false
27282 8701 Acked Read 10 Done Sent Done false
27283 8702 Acked Write 7 Done Sent Done false
27284 8703 Acked Flush 0 Done Sent Done false
27285 8704 Acked Read 6 Done Sent Done false
27286 8705 Acked Read 7 Done Sent Done false
27287 8706 Acked Read 6 Done Sent Done false
27288 8707 Acked Write 6 Done Sent Done false
27289 8708 Acked Read 5 Done Sent Done false
27290 8709 Acked Read 10 Done Sent Done false
27291 8710 Acked Read 6 Done Sent Done false
27292 8711 Acked Read 1 Done Sent Done false
27293 8712 Acked Write 6 Done Sent Done false
27294 8713 Acked Write 3 Done Sent Done false
27295 8714 Acked Read 5 Done Sent Done false
27296 8715 Acked Write 1 Done Sent Done false
27297 8716 Acked Flush 0 Done Sent Done false
27298 8717 Acked Read 1 Done Sent Done false
27299 8718 Acked Flush 0 Done Sent Done false
27300 8719 Acked Read 3 Done Sent Done false
27301 8720 Acked Flush 0 Done Sent Done false
27302 8721 Acked Write 7 Done Sent Done false
27303 8722 Acked Write 2 Done Sent Done false
27304 8723 Acked Read 4 Done Sent Done false
27305 8724 Acked Flush 0 Done Sent Done false
27306 8725 Acked Write 3 Done Sent Done false
27307 8726 Acked Write 2 Done Sent Done false
27308 8727 Acked Read 4 Done Sent Done false
27309 8728 Acked Read 9 Done Sent Done false
27310 8729 Acked Read 7 Done Sent Done false
27311 8730 Acked Read 3 Done Sent Done false
27312 8731 Acked Read 7 Done Sent Done false
27313 8732 Acked Write 2 Done Sent Done false
27314 8733 Acked Write 4 Done Sent Done false
27315 8734 Acked Write 10 Done Sent Done false
27316 8735 Acked Write 9 Done Sent Done false
27317 8736 Acked Read 8 Done Sent Done false
27318 8737 Acked Flush 0 Done Sent Done false
27319 8738 Acked Write 2 Done Sent Done false
27320 8739 Acked Read 10 Done Sent Done false
27321 8740 Acked Write 1 Done Sent Done false
27322 8741 Acked Read 10 Done Sent Done false
27323 8742 Acked Read 4 Done Sent Done false
27324 8743 Acked Write 2 Done Sent Done false
27325 8744 Acked Write 3 Done Sent Done false
27326 8745 Acked Write 9 Done Sent Done false
27327 8746 Acked Read 10 Done Sent Done false
27328 8747 Acked Flush 0 Done Sent Done false
27329 8748 Acked Read 10 Done Sent Done false
27330 8749 Acked Read 1 Done Sent Done false
27331 8750 Acked Write 7 Done Sent Done false
27332 8751 Acked Read 2 Done Sent Done false
27333 8752 Acked Write 4 Done Sent Done false
27334 8753 Acked Write 8 Done Sent Done false
27335 8754 Acked Read 8 Done Sent Done false
27336 8755 Acked Write 2 Done Sent Done false
27337 8756 Acked Read 10 Done Sent Done false
27338 8757 Acked Read 7 Done Sent Done false
27339 8758 Acked Read 8 Done Sent Done false
27340 8759 Acked Flush 0 Done Sent Done false
27341 8760 Acked Write 5 Done Sent Done false
27342 8761 Acked Read 7 Done Sent Done false
27343 8762 Acked Write 5 Done Sent Done false
27344 8763 Acked Write 1 Done Sent Done false
27345 8764 Acked Write 3 Done Sent Done false
27346 8765 Acked Write 6 Done Sent Done false
27347 8766 Acked Write 1 Done Sent Done false
27348 8767 Acked Read 10 Done Sent Done false
27349 8768 Acked Read 4 Done Sent Done false
27350 8769 Acked Read 7 Done Sent Done false
27351 8770 Acked Read 7 Done Sent Done false
27352 8771 Acked Write 5 Done Sent Done false
27353 8772 Acked Write 9 Done Sent Done false
27354 8773 Acked Read 8 Done Sent Done false
27355 8774 Acked Write 10 Done Sent Done false
27356 8775 Acked Read 6 Done Sent Done false
27357 8776 Acked Read 10 Done Sent Done false
27358 8777 Acked Flush 0 Done Sent Done false
27359 8778 Acked Flush 0 Done Sent Done false
27360 8779 Acked Write 5 Done Sent Done false
27361 8780 Acked Write 3 Done Sent Done false
27362 8781 Acked Write 1 Done Sent Done false
27363 8782 Acked Read 5 Done Sent Done false
27364 8783 Acked Read 2 Done Sent Done false
27365 8784 Acked Write 9 Done Sent Done false
27366 8785 Acked Write 1 Done Sent Done false
27367 8786 Acked Flush 0 Done Sent Done false
27368 8787 Acked Read 9 Done Sent Done false
27369 8788 Acked Read 6 Done Sent Done false
27370 8789 Acked Read 9 Done Sent Done false
27371 8790 Acked Read 8 Done Sent Done false
27372 8791 Acked Read 8 Done Sent Done false
27373 8792 Acked Read 8 Done Sent Done false
27374 8793 Acked Write 6 Done Sent Done false
27375 8794 Acked Flush 0 Done Sent Done false
27376 8795 Acked Read 8 Done Sent Done false
27377 8796 Acked Read 6 Done Sent Done false
27378 8797 Acked Read 6 Done Sent Done false
27379 8798 Acked Read 8 Done Sent Done false
27380 8799 Acked Flush 0 Done Sent Done false
27381 8800 Acked Read 10 Done Sent Done false
27382 8801 Acked Read 10 Done Sent Done false
27383 8802 Acked Read 8 Done Sent Done false
27384 8803 Acked Write 4 Done Sent Done false
27385 8804 Acked Write 5 Done Sent Done false
27386 8805 Acked Read 7 Done Sent Done false
27387 8806 Acked Read 4 Done Sent Done false
27388 8807 Acked Write 7 Done Sent Done false
27389 8808 Acked Read 5 Done Sent Done false
27390 8809 Acked Write 8 Done Sent Done false
27391 8810 Acked Read 8 Done Sent Done false
27392 8811 Acked Write 10 Done Sent Done false
27393 8812 Acked Write 7 Done Sent Done false
27394 8813 Acked Read 6 Done Sent Done false
27395 8814 Acked Flush 0 Done Sent Done false
27396 8815 Acked Read 7 Done Sent Done false
27397 8816 Acked Write 2 Done Sent Done false
27398 8817 Acked Read 1 Done Sent Done false
27399 8818 Acked Read 2 Done Sent Done false
27400 8819 Acked Write 3 Done Sent Done false
27401 8820 Acked Flush 0 Done Sent Done false
27402 8821 Acked Read 7 Done Sent Done false
27403 8822 Acked Write 2 Done Sent Done false
27404 8823 Acked Write 4 Done Sent Done false
27405 8824 Acked Write 8 Done Sent Done false
27406 8825 Acked Write 3 Done Sent Done false
27407 8826 Acked Write 5 Done Sent Done false
27408 8827 Acked Write 5 Done Sent Done false
27409 8828 Acked Read 2 Done Sent Done false
27410 8829 Acked Write 7 Done Sent Done false
27411 8830 Acked Read 2 Done Sent Done false
27412 8831 Acked Write 8 Done Sent Done false
27413 8832 Acked Flush 0 Done Sent Done false
27414 8833 Acked Write 4 Done Sent Done false
27415 8834 Acked Read 5 Done Sent Done false
27416 8835 Acked Read 6 Done Sent Done false
27417 8836 Acked Read 2 Done Sent Done false
27418 8837 Acked Write 7 Done Sent Done false
27419 8838 Acked Write 6 Done Sent Done false
27420 8839 Acked Read 5 Done Sent Done false
27421 8840 Acked Read 1 Done Sent Done false
27422 8841 Acked Read 10 Done Sent Done false
27423 8842 Acked Write 10 Done Sent Done false
27424 8843 Acked Read 1 Done Sent Done false
27425 8844 Acked Read 5 Done Sent Done false
27426 8845 Acked Read 1 Done Sent Done false
27427 8846 Acked Write 2 Done Sent Done false
27428 8847 Acked Flush 0 Done Sent Done false
27429 8848 Acked Read 9 Done Sent Done false
27430 8849 Acked Flush 0 Done Sent Done false
27431 STATES DS:0 DS:1 DS:2 TOTAL
27432 Sent 0 302 0 302
27433 Done 302 0 302 604
27434 Skipped 0 0 0 0
27435 Error 0 0 0 0
27436 Last Flush: 8849 8547 8849
27437 Downstairs last five completed: 8547 8546 8545 8544 8543
27438 Upstairs last five completed: 8849 8848 8847 8846 8845
27439 CLIENT: Up:0 ds:302 act:2
274402025-01-11T06:11:10.783ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274412025-01-11T06:11:10.783ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58779) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274422025-01-11T06:11:10.784ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274432025-01-11T06:11:10.785ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274442025-01-11T06:11:10.785ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274452025-01-11T06:11:10.785ZINFOcrucible: send last flush ID to this DS: 8547 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274462025-01-11T06:11:10.785ZINFOcrucible: Replied this last flush ID: 8547 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274472025-01-11T06:11:10.785ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274482025-01-11T06:11:10.785ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274492025-01-11T06:11:10.785ZINFOcrucible: [1] client re-new 302 jobs since flush 8547 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274502025-01-11T06:11:10.785ZINFOcrucible: [1] Replayed 302 jobs since flush: 8547 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
27451 ----------------------------------------------------------------
27452 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27453 Downstairs last five completed: 8849 8848 8847 8846 8845
27454 Upstairs last five completed: 8849 8848 8847 8846 8845
27455 CLIENT: Up:0 ds:0 act:3
27456 Replay: All jobs finished
27457 Main task has work to do, go find it
27458 got dsc Stop(1)
27459 stop 1
27460 Jan 11 06:11:13.035[8820] Got stop action so:false kr:false
27461 INFO request completed, latency_us: 376, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 2da0eaac-225d-4494-b9ce-e14bfc318daf, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
27462 Jan 11 06:11:13.036 INFO request completed, latency_us: 147, response_code: 200, uri: /state/cid/1, method: GET, req_id: 823a02ae-1145-4cea-bed8-2fea182a7a78, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
274632025-01-11T06:11:13.036ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274642025-01-11T06:11:13.036ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274652025-01-11T06:11:13.036ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274662025-01-11T06:11:13.036ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274672025-01-11T06:11:13.036ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
274682025-01-11T06:11:13.036ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
27469 [8820] Exited with: ExitStatus(unix_wait_status(9))
27470 [8820][1] reports Exit
27471 Jan 11 06:11:17.037 INFO request completed, latency_us: 232, response_code: 200, uri: /state/cid/1, method: GET, req_id: e677421a-207b-40b6-87bc-80a07bc440ae, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
27472 001/300 Write block 4283 len 20480 data: 4 4 6 6 6
27473 002/300 Read block 4878 len 20480
27474 003/300 Read block 728 len 32768
27475 004/300 Write block 3128 len 16384 data: 3 2 2 2
27476 005/300 Write block 4939 len 40960 data: 4 5 5 6 9 9 9 8 8 9
27477 006/300 Write block 220 len 36864 data: 4 3 3 4 5 6 7 9 8
27478 007/300 Read block 1666 len 24576
27479 008/300 Read block 1912 len 12288
27480 009/300 Read block 4269 len 8192
27481 010/300 Read block 1648 len 4096
27482 011/300 Write block 97 len 20480 data: 9 9 8 7 7
27483 012/300 Read block 3631 len 32768
27484 013/300 Write block 35 len 8192 data: 6 5
27485 014/300 Read block 3968 len 28672
27486 015/300 Flush
27487 016/300 Read block 2327 len 12288
27488 017/300 Read block 2824 len 40960
27489 018/300 Read block 624 len 4096
27490 019/300 Read block 809 len 40960
27491 020/300 Read block 1951 len 32768
27492 021/300 Flush
27493 022/300 Read block 1967 len 24576
27494 023/300 Read block 2890 len 36864
27495 024/300 Write block 1828 len 12288 data: 7 9 9
27496 025/300 Write block 3819 len 32768 data: 7 8 6 7 6 6 4 6
27497 026/300 Read block 2264 len 16384
27498 027/300 Read block 2310 len 16384
27499 028/300 Write block 1230 len 36864 data: 7 7 7 8 9 7 5 5 3
27500 029/300 Write block 2569 len 36864 data: 6 6 4 6 5 7 6 6 6
27501 030/300 Read block 3521 len 28672
27502 031/300 Write block 2139 len 32768 data: 2 4 4 4 4 5 4 6
27503 032/300 Read block 3902 len 20480
27504 033/300 Write block 2667 len 4096 data: 3
27505 034/300 Write block 4964 len 20480 data: 4 5 6 6 6
27506 035/300 Write block 1362 len 24576 data: 6 8 7 9 10 10
27507 036/300 Write block 4761 len 12288 data: 3 3 3
27508 037/300 Write block 2203 len 28672 data: 5 5 5 5 5 6 7
27509 038/300 Flush
27510 039/300 Read block 4113 len 40960
27511 040/300 Read block 2999 len 40960
27512 041/300 Write block 3937 len 28672 data: 2 2 3 4 5 5 5
27513 042/300 Read block 4738 len 36864
27514 043/300 Read block 3309 len 36864
27515 044/300 Read block 3448 len 32768
27516 045/300 Read block 2329 len 16384
27517 046/300 Read block 738 len 28672
27518 047/300 Write block 4012 len 32768 data: 5 6 6 6 5 4 5 5
27519 048/300 Read block 1664 len 24576
27520 049/300 Read block 1363 len 8192
27521 050/300 Read block 3492 len 20480
27522 051/300 Write block 3218 len 20480 data: 8 7 7 8 8
27523 052/300 Write block 1575 len 4096 data: 7
27524 053/300 Write block 4532 len 40960 data: 5 4 4 4 4 4 6 6 5 4
27525 054/300 Write block 4243 len 24576 data: 8 7 4 4 5 4
27526 055/300 Read block 1641 len 28672
27527 056/300 Read block 1269 len 40960
27528 057/300 Write block 3822 len 12288 data: 8 7 7
27529 058/300 Write block 2744 len 28672 data: 4 5 4 6 6 5 5
27530 059/300 Write block 4918 len 4096 data: 5
27531 060/300 Read block 126 len 36864
27532 061/300 Read block 3500 len 20480
27533 062/300 Flush
27534 063/300 Write block 1849 len 28672 data: 8 8 8 7 5 4 4
27535 064/300 Read block 464 len 32768
27536 065/300 Write block 4087 len 32768 data: 10 10 10 9 8 7 5 4
27537 066/300 Write block 4878 len 36864 data: 8 6 7 7 4 2 2 3 3
27538 067/300 Flush
27539 068/300 Read block 1525 len 28672
27540 069/300 Read block 2455 len 12288
27541 070/300 Read block 1923 len 12288
27542 071/300 Read block 2564 len 16384
27543 072/300 Read block 3257 len 40960
27544 073/300 Write block 4828 len 32768 data: 7 7 7 7 7 7 6 5
27545 074/300 Write block 2299 len 32768 data: 8 8 10 9 7 6 6 5
27546 075/300 Read block 4820 len 20480
27547 076/300 Read block 4639 len 12288
27548 077/300 Write block 3785 len 20480 data: 5 6 5 4 5
27549 078/300 Write block 21 len 28672 data: 2 4 6 4 3 3 3
27550 079/300 Write block 1300 len 28672 data: 6 5 6 6 5 4 3
27551 080/300 Write block 79 len 36864 data: 6 7 7 6 5 5 6 6 5
27552 081/300 Read block 2531 len 28672
27553 082/300 Write block 2103 len 28672 data: 7 5 5 4 4 4 3
27554 083/300 Read block 2884 len 28672
27555 084/300 Write block 2050 len 4096 data: 6
27556 085/300 Read block 2481 len 24576
27557 086/300 Read block 3733 len 28672
27558 087/300 Write block 1511 len 24576 data: 5 4 3 4 4 4
27559 088/300 Write block 4287 len 32768 data: 7 6 6 6 5 5 5 5
27560 089/300 Read block 4806 len 12288
27561 090/300 Write block 4688 len 24576 data: 7 8 7 7 6 6
27562 091/300 Flush
27563 092/300 Write block 2329 len 28672 data: 4 4 6 6 8 7 6
27564 093/300 Read block 129 len 28672
27565 094/300 Flush
27566 095/300 Read block 1484 len 12288
27567 096/300 Read block 1799 len 32768
27568 097/300 Write block 587 len 20480 data: 4 6 6 5 6
27569 098/300 Read block 2069 len 40960
27570 099/300 Write block 1303 len 40960 data: 7 6 5 4 4 5 6 6 7 6
27571 100/300 Read block 179 len 40960
27572 101/300 Read block 1251 len 20480
27573 102/300 Write block 4075 len 4096 data: 12
27574 103/300 Read block 213 len 16384
27575 104/300 Read block 526 len 20480
27576 105/300 Write block 804 len 40960 data: 5 4 5 4 4 7 7 6 6 6
27577 106/300 Write block 309 len 40960 data: 6 6 6 7 4 4 5 6 5 6
27578 107/300 Write block 1990 len 20480 data: 8 9 7 10 9
27579 108/300 Read block 3284 len 36864
27580 109/300 Write block 190 len 12288 data: 10 9 11
27581 110/300 Write block 535 len 32768 data: 4 4 6 6 6 6 4 6
27582 111/300 Write block 2395 len 24576 data: 9 9 10 8 8 8
27583 112/300 Write block 4529 len 16384 data: 7 4 5 6
27584 113/300 Read block 3113 len 24576
27585 114/300 Write block 3506 len 12288 data: 8 8 6
27586 115/300 Read block 518 len 36864
27587 116/300 Flush
27588 117/300 Read block 2003 len 8192
27589 118/300 Write block 3044 len 24576 data: 4 4 4 4 5 5
27590 119/300 Read block 2091 len 40960
27591 120/300 Write block 636 len 28672 data: 4 4 4 5 5 7 7
27592 121/300 Read block 1191 len 12288
27593 122/300 Write block 155 len 20480 data: 4 4 4 4 2
27594 123/300 Read block 2694 len 24576
27595 124/300 Write block 1693 len 36864 data: 7 6 5 5 4 7 6 10 10
27596 125/300 Write block 400 len 36864 data: 7 7 7 6 5 7 6 5 6
27597 126/300 Write block 4852 len 36864 data: 3 2 3 3 3 3 4 4 4
27598 127/300 Write block 4313 len 24576 data: 6 5 5 6 7 7
27599 128/300 Write block 1790 len 24576 data: 3 4 5 5 5 5
27600 129/300 Write block 4230 len 4096 data: 3
27601 130/300 Read block 3828 len 8192
27602 131/300 Write block 1471 len 28672 data: 7 6 4 4 4 3 5
27603 132/300 Read block 3928 len 12288
27604 133/300 Write block 3272 len 8192 data: 9 8
27605 134/300 Write block 2530 len 24576 data: 6 7 6 6 6 8
27606 135/300 Read block 2172 len 28672
27607 136/300 Write block 3088 len 4096 data: 7
27608 137/300 Read block 4398 len 4096
27609 138/300 Read block 1508 len 8192
27610 139/300 Read block 2897 len 28672
27611 140/300 Write block 2416 len 24576 data: 2 2 3 3 4 5
27612 141/300 Read block 4768 len 28672
27613 142/300 Write block 189 len 36864 data: 10 11 10 12 9 8 9 8 8
27614 143/300 Flush
27615 144/300 Read block 2110 len 20480
27616 145/300 Read block 528 len 20480
27617 146/300 Read block 751 len 32768
27618 147/300 Write block 3076 len 24576 data: 5 5 4 4 4 3
27619 148/300 Read block 4341 len 16384
27620 149/300 Read block 2549 len 36864
27621 150/300 Write block 1590 len 16384 data: 3 2 3 6
27622 151/300 Write block 1198 len 24576 data: 3 2 3 3 4 6
27623 152/300 Flush
27624 153/300 Read block 4435 len 24576
27625 154/300 Read block 2575 len 36864
27626 155/300 Flush
27627 156/300 Write block 2803 len 40960 data: 5 7 6 4 4 5 7 6 5 5
27628 157/300 Read block 1118 len 20480
27629 158/300 Read block 1589 len 4096
27630 159/300 Write block 2478 len 28672 data: 6 8 8 5 5 5 5
27631 160/300 Write block 2103 len 20480 data: 8 6 6 5 5
27632 161/300 Write block 3323 len 4096 data: 5
27633 162/300 Write block 1534 len 32768 data: 6 6 6 6 5 5 5 4
27634 163/300 Read block 433 len 20480
27635 164/300 Write block 4650 len 32768 data: 6 6 5 4 4 6 7 7
27636 165/300 Read block 3853 len 20480
27637 166/300 Read block 928 len 20480
27638 167/300 Write block 42 len 24576 data: 11 10 10 7 7 6
27639 168/300 Write block 1983 len 12288 data: 4 4 3
27640 169/300 Read block 3710 len 32768
27641 170/300 Read block 4627 len 8192
27642 171/300 Read block 875 len 28672
27643 172/300 Write block 4795 len 4096 data: 8
27644 173/300 Write block 4843 len 28672 data: 5 6 6 3 3 2 3
27645 174/300 Read block 4481 len 32768
27646 175/300 Read block 2840 len 20480
27647 176/300 Read block 4853 len 36864
27648 177/300 Read block 2037 len 4096
27649 178/300 Write block 209 len 4096 data: 4
27650 179/300 Flush
27651 180/300 Read block 3865 len 40960
27652 181/300 Read block 2314 len 12288
27653 182/300 Read block 340 len 40960
27654 183/300 Flush
27655 184/300 Read block 4082 len 36864
27656 185/300 Read block 1568 len 8192
27657 186/300 Read block 1996 len 4096
27658 187/300 Read block 2542 len 24576
27659 188/300 Read block 195 len 24576
27660 189/300 Write block 1473 len 20480 data: 5 5 5 4 6
27661 190/300 Write block 2591 len 28672 data: 5 6 5 5 4 5 4
27662 191/300 Flush
27663 192/300 Write block 4260 len 16384 data: 6 4 7 8
27664 193/300 Read block 2761 len 32768
27665 194/300 Flush
27666 195/300 Write block 246 len 28672 data: 7 8 8 6 7 7 5
27667 196/300 Write block 3370 len 4096 data: 8
27668 197/300 Read block 1569 len 4096
27669 198/300 Read block 4790 len 36864
27670 199/300 Read block 2318 len 24576
27671 200/300 Write block 3023 len 12288 data: 5 4 3
27672 201/300 Write block 4121 len 12288 data: 5 7 5
27673 202/300 Write block 1309 len 36864 data: 7 7 8 7 6 6 4 3 4
27674 203/300 Read block 3613 len 8192
27675 204/300 Write block 2959 len 12288 data: 5 5 4
27676 205/300 Read block 3580 len 24576
27677 206/300 Write block 4045 len 20480 data: 11 10 7 8 6
27678 207/300 Write block 4212 len 8192 data: 7 6
27679 208/300 Write block 3852 len 28672 data: 6 4 4 4 3 3 3
27680 209/300 Write block 777 len 40960 data: 4 3 3 4 3 4 4 4 5 5
27681 210/300 Write block 914 len 40960 data: 5 6 6 6 5 4 3 4 4 4
27682 211/300 Flush
27683 212/300 Write block 4915 len 32768 data: 3 4 5 6 4 4 3 5
27684 213/300 Read block 4301 len 40960
27685 214/300 Write block 4006 len 40960 data: 4 4 5 4 4 5 6 7 7 7
27686 215/300 Read block 1516 len 12288
27687 216/300 Flush
27688 217/300 Flush
27689 218/300 Write block 4595 len 36864 data: 6 6 6 6 5 4 4 5 5
27690 219/300 Read block 883 len 36864
27691 220/300 Write block 945 len 32768 data: 4 5 4 6 7 7 7 7
27692 221/300 Read block 3202 len 4096
27693 222/300 Read block 1048 len 16384
27694 223/300 Read block 4778 len 12288
27695 224/300 Read block 4890 len 8192
27696 225/300 Write block 132 len 36864 data: 5 6 6 7 5 6 6 6 6
27697 226/300 Read block 100 len 24576
27698 227/300 Read block 3480 len 32768
27699 228/300 Read block 2594 len 40960
27700 229/300 Flush
27701 230/300 Flush
27702 231/300 Read block 4120 len 20480
27703 232/300 Write block 4203 len 36864 data: 5 5 6 6 6 8 7 7 9
27704 233/300 Write block 3125 len 36864 data: 3 3 4 4 3 3 3 4 4
27705 234/300 Write block 4982 len 20480 data: 5 7 8 8 9
27706 235/300 Read block 3262 len 12288
27707 236/300 Read block 398 len 24576
27708 237/300 Read block 1646 len 16384
27709 238/300 Read block 4668 len 40960
27710 239/300 Write block 1795 len 20480 data: 6 5 6 5 5
27711 240/300 Flush
27712 241/300 Write block 4711 len 28672 data: 5 4 4 5 6 6 7
27713 242/300 Flush
27714 243/300 Read block 1259 len 36864
27715 244/300 Read block 1596 len 4096
27716 245/300 Write block 210 len 12288 data: 4 6 4
27717 246/300 Read block 3759 len 16384
27718 247/300 Read block 1020 len 28672
27719 248/300 Read block 4378 len 20480
27720 249/300 Read block 353 len 24576
27721 250/300 Write block 1711 len 40960 data: 6 5 4 4 5 5 4 5 6 7
27722 251/300 Write block 2615 len 12288 data: 4 4 3
27723 252/300 Read block 1875 len 36864
27724 253/300 Read block 947 len 36864
27725 254/300 Read block 3339 len 16384
27726 255/300 Read block 3965 len 36864
27727 256/300 Read block 4970 len 24576
27728 257/300 Read block 1395 len 16384
27729 258/300 Read block 987 len 28672
27730 259/300 Read block 699 len 40960
27731 260/300 Read block 673 len 16384
27732 261/300 Read block 3096 len 28672
27733 262/300 Read block 2748 len 24576
27734 263/300 Write block 4373 len 8192 data: 3 3
27735 264/300 Read block 1549 len 4096
27736 265/300 Write block 4628 len 32768 data: 5 5 5 7 7 8 8 9
27737 266/300 Read block 4514 len 36864
27738 267/300 Write block 3187 len 24576 data: 8 7 4 5 6 6
27739 268/300 Read block 2546 len 12288
27740 269/300 Write block 3524 len 24576 data: 6 6 8 7 6 6
27741 270/300 Flush
27742 271/300 Read block 1040 len 12288
27743 272/300 Read block 4993 len 20480
27744 273/300 Write block 1577 len 28672 data: 5 6 7 6 5 6 6
27745 274/300 Write block 21 len 28672 data: 3 5 7 5 4 4 4
27746 275/300 Read block 4071 len 16384
27747 276/300 Flush
27748 277/300 Write block 1247 len 12288 data: 4 4 4
27749 278/300 Read block 1487 len 8192
27750 279/300 Read block 2234 len 16384
27751 280/300 Read block 3195 len 16384
27752 281/300 Read block 4890 len 40960
27753 282/300 Write block 4531 len 12288 data: 6 7 5
27754 283/300 Read block 4459 len 20480
27755 284/300 Read block 1755 len 20480
27756 285/300 Flush
27757 286/300 Write block 235 len 40960 data: 6 6 5 6 5 5 7 7 8 7
27758 287/300 Flush
27759 288/300 Read block 275 len 32768
27760 289/300 Write block 3905 len 8192 data: 6 6
27761 290/300 Read block 4614 len 12288
27762 291/300 Write block 4127 len 16384 data: 9 10 8 9
27763 292/300 Write block 1966 len 28672 data: 3 3 4 4 3 3 3
27764 293/300 Write block 1042 len 36864 data: 2 2 2 3 4 4 5 5 5
27765 294/300 Read block 3987 len 4096
27766 295/300 Write block 363 len 4096 data: 5
27767 296/300 Read block 3184 len 32768
27768 297/300 Write block 1601 len 36864 data: 4 3 3 4 4 3 3 3 4
27769 298/300 Write block 2852 len 28672 data: 4 5 4 5 4 5 5
27770 299/300 Read block 4354 len 8192
27771 300/300 Read block 4553 len 20480
27772 301/300 Write block 595 len 24576 data: 8 9 7 7 7 3
27773 Main task has work to do, go find it
27774 got dsc Start(1)
27775 start 1
27776 Jan 11 06:11:17.294 INFO request completed, latency_us: 286, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6157b386-74d3-49e1-a5c6-afeaf0f15d9d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
27777 Replay: started 1, returned:Ok(())
27778 ----------------------------------------------------------------
27779 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27780 States: Active Offline Active
27781 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27782 8850 Acked Write 5 Done Sent Done false
27783 8851 Acked Read 5 Done Sent Done false
27784 8852 Acked Read 8 Done Sent Done false
27785 8853 Acked Write 4 Done Sent Done false
27786 8854 Acked Write 10 Done Sent Done false
27787 8855 Acked Write 9 Done Sent Done false
27788 8856 Acked Read 6 Done Sent Done false
27789 8857 Acked Read 3 Done Sent Done false
27790 8858 Acked Read 2 Done Sent Done false
27791 8859 Acked Read 1 Done Sent Done false
27792 8860 Acked Write 5 Done Sent Done false
27793 8861 Acked Read 8 Done Sent Done false
27794 8862 Acked Write 2 Done Sent Done false
27795 8863 Acked Read 7 Done Sent Done false
27796 8864 Acked Flush 0 Done Sent Done false
27797 8865 Acked Read 3 Done Sent Done false
27798 8866 Acked Read 10 Done Sent Done false
27799 8867 Acked Read 1 Done Sent Done false
27800 8868 Acked Read 10 Done Sent Done false
27801 8869 Acked Read 8 Done Sent Done false
27802 8870 Acked Flush 0 Done Sent Done false
27803 8871 Acked Read 6 Done Sent Done false
27804 8872 Acked Read 9 Done Sent Done false
27805 8873 Acked Write 3 Done Sent Done false
27806 8874 Acked Write 8 Done Sent Done false
27807 8875 Acked Read 4 Done Sent Done false
27808 8876 Acked Read 4 Done Sent Done false
27809 8877 Acked Write 9 Done Sent Done false
27810 8878 Acked Write 9 Done Sent Done false
27811 8879 Acked Read 7 Done Sent Done false
27812 8880 Acked Write 8 Done Sent Done false
27813 8881 Acked Read 5 Done Sent Done false
27814 8882 Acked Write 1 Done Sent Done false
27815 8883 Acked Write 5 Done Sent Done false
27816 8884 Acked Write 6 Done Sent Done false
27817 8885 Acked Write 3 Done Sent Done false
27818 8886 Acked Write 7 Done Sent Done false
27819 8887 Acked Flush 0 Done Sent Done false
27820 8888 Acked Read 10 Done Sent Done false
27821 8889 Acked Read 10 Done Sent Done false
27822 8890 Acked Write 7 Done Sent Done false
27823 8891 Acked Read 9 Done Sent Done false
27824 8892 Acked Read 9 Done Sent Done false
27825 8893 Acked Read 8 Done Sent Done false
27826 8894 Acked Read 4 Done Sent Done false
27827 8895 Acked Read 7 Done Sent Done false
27828 8896 Acked Write 8 Done Sent Done false
27829 8897 Acked Read 6 Done Sent Done false
27830 8898 Acked Read 2 Done Sent Done false
27831 8899 Acked Read 5 Done Sent Done false
27832 8900 Acked Write 5 Done Sent Done false
27833 8901 Acked Write 1 Done Sent Done false
27834 8902 Acked Write 10 Done Sent Done false
27835 8903 Acked Write 6 Done Sent Done false
27836 8904 Acked Read 7 Done Sent Done false
27837 8905 Acked Read 10 Done Sent Done false
27838 8906 Acked Write 3 Done Sent Done false
27839 8907 Acked Write 7 Done Sent Done false
27840 8908 Acked Write 1 Done Sent Done false
27841 8909 Acked Read 9 Done Sent Done false
27842 8910 Acked Read 5 Done Sent Done false
27843 8911 Acked Flush 0 Done Sent Done false
27844 8912 Acked Write 7 Done Sent Done false
27845 8913 Acked Read 8 Done Sent Done false
27846 8914 Acked Write 8 Done Sent Done false
27847 8915 Acked Write 9 Done Sent Done false
27848 8916 Acked Flush 0 Done Sent Done false
27849 8917 Acked Read 7 Done Sent Done false
27850 8918 Acked Read 3 Done Sent Done false
27851 8919 Acked Read 3 Done Sent Done false
27852 8920 Acked Read 4 Done Sent Done false
27853 8921 Acked Read 10 Done Sent Done false
27854 8922 Acked Write 8 Done Sent Done false
27855 8923 Acked Write 8 Done Sent Done false
27856 8924 Acked Read 5 Done Sent Done false
27857 8925 Acked Read 3 Done Sent Done false
27858 8926 Acked Write 5 Done Sent Done false
27859 8927 Acked Write 7 Done Sent Done false
27860 8928 Acked Write 7 Done Sent Done false
27861 8929 Acked Write 9 Done Sent Done false
27862 8930 Acked Read 7 Done Sent Done false
27863 8931 Acked Write 7 Done Sent Done false
27864 8932 Acked Read 7 Done Sent Done false
27865 8933 Acked Write 1 Done Sent Done false
27866 8934 Acked Read 6 Done Sent Done false
27867 8935 Acked Read 7 Done Sent Done false
27868 8936 Acked Write 6 Done Sent Done false
27869 8937 Acked Write 8 Done Sent Done false
27870 8938 Acked Read 3 Done Sent Done false
27871 8939 Acked Write 6 Done Sent Done false
27872 8940 Acked Flush 0 Done Sent Done false
27873 8941 Acked Write 7 Done Sent Done false
27874 8942 Acked Read 7 Done Sent Done false
27875 8943 Acked Flush 0 Done Sent Done false
27876 8944 Acked Read 3 Done Sent Done false
27877 8945 Acked Read 8 Done Sent Done false
27878 8946 Acked Write 5 Done Sent Done false
27879 8947 Acked Read 10 Done Sent Done false
27880 8948 Acked Write 10 Done Sent Done false
27881 8949 Acked Read 10 Done Sent Done false
27882 8950 Acked Read 5 Done Sent Done false
27883 8951 Acked Write 1 Done Sent Done false
27884 8952 Acked Read 4 Done Sent Done false
27885 8953 Acked Read 5 Done Sent Done false
27886 8954 Acked Write 10 Done Sent Done false
27887 8955 Acked Write 10 Done Sent Done false
27888 8956 Acked Write 5 Done Sent Done false
27889 8957 Acked Read 9 Done Sent Done false
27890 8958 Acked Write 3 Done Sent Done false
27891 8959 Acked Write 8 Done Sent Done false
27892 8960 Acked Write 6 Done Sent Done false
27893 8961 Acked Write 4 Done Sent Done false
27894 8962 Acked Read 6 Done Sent Done false
27895 8963 Acked Write 3 Done Sent Done false
27896 8964 Acked Read 9 Done Sent Done false
27897 8965 Acked Flush 0 Done Sent Done false
27898 8966 Acked Read 2 Done Sent Done false
27899 8967 Acked Write 6 Done Sent Done false
27900 8968 Acked Read 10 Done Sent Done false
27901 8969 Acked Write 7 Done Sent Done false
27902 8970 Acked Read 3 Done Sent Done false
27903 8971 Acked Write 5 Done Sent Done false
27904 8972 Acked Read 6 Done Sent Done false
27905 8973 Acked Write 9 Done Sent Done false
27906 8974 Acked Write 9 Done Sent Done false
27907 8975 Acked Write 9 Done Sent Done false
27908 8976 Acked Write 6 Done Sent Done false
27909 8977 Acked Write 6 Done Sent Done false
27910 8978 Acked Write 1 Done Sent Done false
27911 8979 Acked Read 2 Done Sent Done false
27912 8980 Acked Write 7 Done Sent Done false
27913 8981 Acked Read 3 Done Sent Done false
27914 8982 Acked Write 2 Done Sent Done false
27915 8983 Acked Write 6 Done Sent Done false
27916 8984 Acked Read 7 Done Sent Done false
27917 8985 Acked Write 1 Done Sent Done false
27918 8986 Acked Read 1 Done Sent Done false
27919 8987 Acked Read 2 Done Sent Done false
27920 8988 Acked Read 7 Done Sent Done false
27921 8989 Acked Write 6 Done Sent Done false
27922 8990 Acked Read 7 Done Sent Done false
27923 8991 Acked Write 9 Done Sent Done false
27924 8992 Acked Flush 0 Done Sent Done false
27925 8993 Acked Read 5 Done Sent Done false
27926 8994 Acked Read 5 Done Sent Done false
27927 8995 Acked Read 8 Done Sent Done false
27928 8996 Acked Write 6 Done Sent Done false
27929 8997 Acked Read 4 Done Sent Done false
27930 8998 Acked Read 9 Done Sent Done false
27931 8999 Acked Write 4 Done Sent Done false
27932 9000 Acked Write 6 Done Sent Done false
27933 9001 Acked Flush 0 Done Sent Done false
27934 9002 Acked Read 6 Done Sent Done false
27935 9003 Acked Read 9 Done Sent Done false
27936 9004 Acked Flush 0 Done Sent Done false
27937 9005 Acked Write 10 Done Sent Done false
27938 9006 Acked Read 5 Done Sent Done false
27939 9007 Acked Read 1 Done Sent Done false
27940 9008 Acked Write 7 Done Sent Done false
27941 9009 Acked Write 5 Done Sent Done false
27942 9010 Acked Write 1 Done Sent Done false
27943 9011 Acked Write 8 Done Sent Done false
27944 9012 Acked Read 5 Done Sent Done false
27945 9013 Acked Write 8 Done Sent Done false
27946 9014 Acked Read 5 Done Sent Done false
27947 9015 Acked Read 5 Done Sent Done false
27948 9016 Acked Write 6 Done Sent Done false
27949 9017 Acked Write 3 Done Sent Done false
27950 9018 Acked Read 8 Done Sent Done false
27951 9019 Acked Read 2 Done Sent Done false
27952 9020 Acked Read 7 Done Sent Done false
27953 9021 Acked Write 1 Done Sent Done false
27954 9022 Acked Write 7 Done Sent Done false
27955 9023 Acked Read 8 Done Sent Done false
27956 9024 Acked Read 5 Done Sent Done false
27957 9025 Acked Read 9 Done Sent Done false
27958 9026 Acked Read 1 Done Sent Done false
27959 9027 Acked Write 1 Done Sent Done false
27960 9028 Acked Flush 0 Done Sent Done false
27961 9029 Acked Read 10 Done Sent Done false
27962 9030 Acked Read 3 Done Sent Done false
27963 9031 Acked Read 10 Done Sent Done false
27964 9032 Acked Flush 0 Done Sent Done false
27965 9033 Acked Read 9 Done Sent Done false
27966 9034 Acked Read 2 Done Sent Done false
27967 9035 Acked Read 1 Done Sent Done false
27968 9036 Acked Read 6 Done Sent Done false
27969 9037 Acked Read 6 Done Sent Done false
27970 9038 Acked Write 5 Done Sent Done false
27971 9039 Acked Write 7 Done Sent Done false
27972 9040 Acked Flush 0 Done Sent Done false
27973 9041 Acked Write 4 Done Sent Done false
27974 9042 Acked Read 8 Done Sent Done false
27975 9043 Acked Flush 0 Done Sent Done false
27976 9044 Acked Write 7 Done Sent Done false
27977 9045 Acked Write 1 Done Sent Done false
27978 9046 Acked Read 1 Done Sent Done false
27979 9047 Acked Read 9 Done Sent Done false
27980 9048 Acked Read 6 Done Sent Done false
27981 9049 Acked Write 3 Done Sent Done false
27982 9050 Acked Write 3 Done Sent Done false
27983 9051 Acked Write 9 Done Sent Done false
27984 9052 Acked Read 2 Done Sent Done false
27985 9053 Acked Write 3 Done Sent Done false
27986 9054 Acked Read 6 Done Sent Done false
27987 9055 Acked Write 5 Done Sent Done false
27988 9056 Acked Write 2 Done Sent Done false
27989 9057 Acked Write 7 Done Sent Done false
27990 9058 Acked Write 10 Done Sent Done false
27991 9059 Acked Write 10 Done Sent Done false
27992 9060 Acked Flush 0 Done Sent Done false
27993 9061 Acked Write 8 Done Sent Done false
27994 9062 Acked Read 10 Done Sent Done false
27995 9063 Acked Write 10 Done Sent Done false
27996 9064 Acked Read 3 Done Sent Done false
27997 9065 Acked Flush 0 Done Sent Done false
27998 9066 Acked Flush 0 Done Sent Done false
27999 9067 Acked Write 9 Done Sent Done false
28000 9068 Acked Read 9 Done Sent Done false
28001 9069 Acked Write 8 Done Sent Done false
28002 9070 Acked Read 1 Done Sent Done false
28003 9071 Acked Read 4 Done Sent Done false
28004 9072 Acked Read 3 Done Sent Done false
28005 9073 Acked Read 2 Done Sent Done false
28006 9074 Acked Write 9 Done Sent Done false
28007 9075 Acked Read 6 Done Sent Done false
28008 9076 Acked Read 8 Done Sent Done false
28009 9077 Acked Read 10 Done Sent Done false
28010 9078 Acked Flush 0 Done Sent Done false
28011 9079 Acked Flush 0 Done Sent Done false
28012 9080 Acked Read 5 Done Sent Done false
28013 9081 Acked Write 9 Done Sent Done false
28014 9082 Acked Write 9 Done Sent Done false
28015 9083 Acked Write 5 Done Sent Done false
28016 9084 Acked Read 3 Done Sent Done false
28017 9085 Acked Read 6 Done Sent Done false
28018 9086 Acked Read 4 Done Sent Done false
28019 9087 Acked Read 10 Done Sent Done false
28020 9088 Acked Write 5 Done Sent Done false
28021 9089 Acked Flush 0 Done Sent Done false
28022 9090 Acked Write 7 Done Sent Done false
28023 9091 Acked Flush 0 Done Sent Done false
28024 9092 Acked Read 9 Done Sent Done false
28025 9093 Acked Read 1 Done Sent Done false
28026 9094 Acked Write 3 Done Sent Done false
28027 9095 Acked Read 4 Done Sent Done false
28028 9096 Acked Read 7 Done Sent Done false
28029 9097 Acked Read 5 Done Sent Done false
28030 9098 Acked Read 6 Done Sent Done false
28031 9099 Acked Write 10 Done Sent Done false
28032 9100 Acked Write 3 Done Sent Done false
28033 9101 Acked Read 9 Done Sent Done false
28034 9102 Acked Read 9 Done Sent Done false
28035 9103 Acked Read 4 Done Sent Done false
28036 9104 Acked Read 9 Done Sent Done false
28037 9105 Acked Read 6 Done Sent Done false
28038 9106 Acked Read 4 Done Sent Done false
28039 9107 Acked Read 7 Done Sent Done false
28040 9108 Acked Read 10 Done Sent Done false
28041 9109 Acked Read 4 Done Sent Done false
28042 9110 Acked Read 7 Done Sent Done false
28043 9111 Acked Read 6 Done Sent Done false
28044 9112 Acked Write 2 Done Sent Done false
28045 9113 Acked Read 1 Done Sent Done false
28046 9114 Acked Write 8 Done Sent Done false
28047 9115 Acked Read 9 Done Sent Done false
28048 9116 Acked Write 6 Done Sent Done false
28049 9117 Acked Read 3 Done Sent Done false
28050 9118 Acked Write 6 Done Sent Done false
28051 9119 Acked Flush 0 Done Sent Done false
28052 9120 Acked Read 3 Done Sent Done false
28053 9121 Acked Read 5 Done Sent Done false
28054 9122 Acked Write 7 Done Sent Done false
28055 9123 Acked Write 7 Done Sent Done false
28056 9124 Acked Read 4 Done Sent Done false
28057 9125 Acked Flush 0 Done Sent Done false
28058 9126 Acked Write 3 Done Sent Done false
28059 9127 Acked Read 2 Done Sent Done false
28060 9128 Acked Read 4 Done Sent Done false
28061 9129 Acked Read 4 Done Sent Done false
28062 9130 Acked Read 10 Done Sent Done false
28063 9131 Acked Write 3 Done Sent Done false
28064 9132 Acked Read 5 Done Sent Done false
28065 9133 Acked Read 5 Done Sent Done false
28066 9134 Acked Flush 0 Done Sent Done false
28067 9135 Acked Write 10 Done Sent Done false
28068 9136 Acked Flush 0 Done Sent Done false
28069 9137 Acked Read 8 Done Sent Done false
28070 9138 Acked Write 2 Done Sent Done false
28071 9139 Acked Read 3 Done Sent Done false
28072 9140 Acked Write 4 Done Sent Done false
28073 9141 Acked Write 7 Done Sent Done false
28074 9142 Acked Write 9 Done Sent Done false
28075 9143 Acked Read 1 Done Sent Done false
28076 9144 Acked Write 1 Done Sent Done false
28077 9145 Acked Read 8 Done Sent Done false
28078 9146 Acked Write 9 Done Sent Done false
28079 9147 Acked Write 7 Done Sent Done false
28080 9148 Acked Read 2 Done Sent Done false
28081 9149 Acked Read 5 Done Sent Done false
28082 9150 Acked Write 6 Done Sent Done false
28083 STATES DS:0 DS:1 DS:2 TOTAL
28084 Sent 0 301 0 301
28085 Done 301 0 301 602
28086 Skipped 0 0 0 0
28087 Error 0 0 0 0
28088 Last Flush: 9136 8849 9136
28089 Downstairs last five completed: 8849 8848 8847 8846 8845
28090 Upstairs last five completed: 9150 9149 9148 9147 9146
28091 CLIENT: Up:0 ds:301 act:2
28092 ----------------------------------------------------------------
28093 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28094 States: Active Offline Active
28095 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28096 8850 Acked Write 5 Done Sent Done false
28097 8851 Acked Read 5 Done Sent Done false
28098 8852 Acked Read 8 Done Sent Done false
28099 8853 Acked Write 4 Done Sent Done false
28100 8854 Acked Write 10 Done Sent Done false
28101 8855 Acked Write 9 Done Sent Done false
28102 8856 Acked Read 6 Done Sent Done false
28103 8857 Acked Read 3 Done Sent Done false
28104 8858 Acked Read 2 Done Sent Done false
28105 8859 Acked Read 1 Done Sent Done false
28106 8860 Acked Write 5 Done Sent Done false
28107 8861 Acked Read 8 Done Sent Done false
28108 8862 Acked Write 2 Done Sent Done false
28109 8863 Acked Read 7 Done Sent Done false
28110 8864 Acked Flush 0 Done Sent Done false
28111 8865 Acked Read 3 Done Sent Done false
28112 8866 Acked Read 10 Done Sent Done false
28113 8867 Acked Read 1 Done Sent Done false
28114 8868 Acked Read 10 Done Sent Done false
28115 8869 Acked Read 8 Done Sent Done false
28116 8870 Acked Flush 0 Done Sent Done false
28117 8871 Acked Read 6 Done Sent Done false
28118 8872 Acked Read 9 Done Sent Done false
28119 8873 Acked Write 3 Done Sent Done false
28120 8874 Acked Write 8 Done Sent Done false
28121 8875 Acked Read 4 Done Sent Done false
28122 8876 Acked Read 4 Done Sent Done false
28123 8877 Acked Write 9 Done Sent Done false
28124 8878 Acked Write 9 Done Sent Done false
28125 8879 Acked Read 7 Done Sent Done false
28126 8880 Acked Write 8 Done Sent Done false
28127 8881 Acked Read 5 Done Sent Done false
28128 8882 Acked Write 1 Done Sent Done false
28129 8883 Acked Write 5 Done Sent Done false
28130 8884 Acked Write 6 Done Sent Done false
28131 8885 Acked Write 3 Done Sent Done false
28132 8886 Acked Write 7 Done Sent Done false
28133 8887 Acked Flush 0 Done Sent Done false
28134 8888 Acked Read 10 Done Sent Done false
28135 8889 Acked Read 10 Done Sent Done false
28136 8890 Acked Write 7 Done Sent Done false
28137 8891 Acked Read 9 Done Sent Done false
28138 8892 Acked Read 9 Done Sent Done false
28139 8893 Acked Read 8 Done Sent Done false
28140 8894 Acked Read 4 Done Sent Done false
28141 8895 Acked Read 7 Done Sent Done false
28142 8896 Acked Write 8 Done Sent Done false
28143 8897 Acked Read 6 Done Sent Done false
28144 8898 Acked Read 2 Done Sent Done false
28145 8899 Acked Read 5 Done Sent Done false
28146 8900 Acked Write 5 Done Sent Done false
28147 8901 Acked Write 1 Done Sent Done false
28148 8902 Acked Write 10 Done Sent Done false
28149 8903 Acked Write 6 Done Sent Done false
28150 8904 Acked Read 7 Done Sent Done false
28151 8905 Acked Read 10 Done Sent Done false
28152 8906 Acked Write 3 Done Sent Done false
28153 8907 Acked Write 7 Done Sent Done false
28154 8908 Acked Write 1 Done Sent Done false
28155 8909 Acked Read 9 Done Sent Done false
28156 8910 Acked Read 5 Done Sent Done false
28157 8911 Acked Flush 0 Done Sent Done false
28158 8912 Acked Write 7 Done Sent Done false
28159 8913 Acked Read 8 Done Sent Done false
28160 8914 Acked Write 8 Done Sent Done false
28161 8915 Acked Write 9 Done Sent Done false
28162 8916 Acked Flush 0 Done Sent Done false
28163 8917 Acked Read 7 Done Sent Done false
28164 8918 Acked Read 3 Done Sent Done false
28165 8919 Acked Read 3 Done Sent Done false
28166 8920 Acked Read 4 Done Sent Done false
28167 8921 Acked Read 10 Done Sent Done false
28168 8922 Acked Write 8 Done Sent Done false
28169 8923 Acked Write 8 Done Sent Done false
28170 8924 Acked Read 5 Done Sent Done false
28171 8925 Acked Read 3 Done Sent Done false
28172 8926 Acked Write 5 Done Sent Done false
28173 8927 Acked Write 7 Done Sent Done false
28174 8928 Acked Write 7 Done Sent Done false
28175 8929 Acked Write 9 Done Sent Done false
28176 8930 Acked Read 7 Done Sent Done false
28177 8931 Acked Write 7 Done Sent Done false
28178 8932 Acked Read 7 Done Sent Done false
28179 8933 Acked Write 1 Done Sent Done false
28180 8934 Acked Read 6 Done Sent Done false
28181 8935 Acked Read 7 Done Sent Done false
28182 8936 Acked Write 6 Done Sent Done false
28183 8937 Acked Write 8 Done Sent Done false
28184 8938 Acked Read 3 Done Sent Done false
28185 8939 Acked Write 6 Done Sent Done false
28186 8940 Acked Flush 0 Done Sent Done false
28187 8941 Acked Write 7 Done Sent Done false
28188 8942 Acked Read 7 Done Sent Done false
28189 8943 Acked Flush 0 Done Sent Done false
28190 8944 Acked Read 3 Done Sent Done false
28191 8945 Acked Read 8 Done Sent Done false
28192 8946 Acked Write 5 Done Sent Done false
28193 8947 Acked Read 10 Done Sent Done false
28194 8948 Acked Write 10 Done Sent Done false
28195 8949 Acked Read 10 Done Sent Done false
28196 8950 Acked Read 5 Done Sent Done false
28197 8951 Acked Write 1 Done Sent Done false
28198 8952 Acked Read 4 Done Sent Done false
28199 8953 Acked Read 5 Done Sent Done false
28200 8954 Acked Write 10 Done Sent Done false
28201 8955 Acked Write 10 Done Sent Done false
28202 8956 Acked Write 5 Done Sent Done false
28203 8957 Acked Read 9 Done Sent Done false
28204 8958 Acked Write 3 Done Sent Done false
28205 8959 Acked Write 8 Done Sent Done false
28206 8960 Acked Write 6 Done Sent Done false
28207 8961 Acked Write 4 Done Sent Done false
28208 8962 Acked Read 6 Done Sent Done false
28209 8963 Acked Write 3 Done Sent Done false
28210 8964 Acked Read 9 Done Sent Done false
28211 8965 Acked Flush 0 Done Sent Done false
28212 8966 Acked Read 2 Done Sent Done false
28213 8967 Acked Write 6 Done Sent Done false
28214 8968 Acked Read 10 Done Sent Done false
28215 8969 Acked Write 7 Done Sent Done false
28216 8970 Acked Read 3 Done Sent Done false
28217 8971 Acked Write 5 Done Sent Done false
28218 8972 Acked Read 6 Done Sent Done false
28219 8973 Acked Write 9 Done Sent Done false
28220 8974 Acked Write 9 Done Sent Done false
28221 8975 Acked Write 9 Done Sent Done false
28222 8976 Acked Write 6 Done Sent Done false
28223 8977 Acked Write 6 Done Sent Done false
28224 8978 Acked Write 1 Done Sent Done false
28225 8979 Acked Read 2 Done Sent Done false
28226 8980 Acked Write 7 Done Sent Done false
28227 8981 Acked Read 3 Done Sent Done false
28228 8982 Acked Write 2 Done Sent Done false
28229 8983 Acked Write 6 Done Sent Done false
28230 8984 Acked Read 7 Done Sent Done false
28231 8985 Acked Write 1 Done Sent Done false
28232 8986 Acked Read 1 Done Sent Done false
28233 8987 Acked Read 2 Done Sent Done false
28234 8988 Acked Read 7 Done Sent Done false
28235 8989 Acked Write 6 Done Sent Done false
28236 8990 Acked Read 7 Done Sent Done false
28237 8991 Acked Write 9 Done Sent Done false
28238 8992 Acked Flush 0 Done Sent Done false
28239 8993 Acked Read 5 Done Sent Done false
28240 8994 Acked Read 5 Done Sent Done false
28241 8995 Acked Read 8 Done Sent Done false
28242 8996 Acked Write 6 Done Sent Done false
28243 8997 Acked Read 4 Done Sent Done false
28244 8998 Acked Read 9 Done Sent Done false
28245 8999 Acked Write 4 Done Sent Done false
28246 9000 Acked Write 6 Done Sent Done false
28247 9001 Acked Flush 0 Done Sent Done false
28248 9002 Acked Read 6 Done Sent Done false
28249 9003 Acked Read 9 Done Sent Done false
28250 9004 Acked Flush 0 Done Sent Done false
28251 9005 Acked Write 10 Done Sent Done false
28252 9006 Acked Read 5 Done Sent Done false
28253 9007 Acked Read 1 Done Sent Done false
28254 9008 Acked Write 7 Done Sent Done false
28255 9009 Acked Write 5 Done Sent Done false
28256 9010 Acked Write 1 Done Sent Done false
28257 9011 Acked Write 8 Done Sent Done false
28258 9012 Acked Read 5 Done Sent Done false
28259 9013 Acked Write 8 Done Sent Done false
28260 9014 Acked Read 5 Done Sent Done false
28261 9015 Acked Read 5 Done Sent Done false
28262 9016 Acked Write 6 Done Sent Done false
28263 9017 Acked Write 3 Done Sent Done false
28264 9018 Acked Read 8 Done Sent Done false
28265 9019 Acked Read 2 Done Sent Done false
28266 9020 Acked Read 7 Done Sent Done false
28267 9021 Acked Write 1 Done Sent Done false
28268 9022 Acked Write 7 Done Sent Done false
28269 9023 Acked Read 8 Done Sent Done false
28270 9024 Acked Read 5 Done Sent Done false
28271 9025 Acked Read 9 Done Sent Done false
28272 9026 Acked Read 1 Done Sent Done false
28273 9027 Acked Write 1 Done Sent Done false
28274 9028 Acked Flush 0 Done Sent Done false
28275 9029 Acked Read 10 Done Sent Done false
28276 9030 Acked Read 3 Done Sent Done false
28277 9031 Acked Read 10 Done Sent Done false
28278 9032 Acked Flush 0 Done Sent Done false
28279 9033 Acked Read 9 Done Sent Done false
28280 9034 Acked Read 2 Done Sent Done false
28281 9035 Acked Read 1 Done Sent Done false
28282 9036 Acked Read 6 Done Sent Done false
28283 9037 Acked Read 6 Done Sent Done false
28284 9038 Acked Write 5 Done Sent Done false
28285 9039 Acked Write 7 Done Sent Done false
28286 9040 Acked Flush 0 Done Sent Done false
28287 9041 Acked Write 4 Done Sent Done false
28288 9042 Acked Read 8 Done Sent Done false
28289 9043 Acked Flush 0 Done Sent Done false
28290 9044 Acked Write 7 Done Sent Done false
28291 9045 Acked Write 1 Done Sent Done false
28292 9046 Acked Read 1 Done Sent Done false
28293 9047 Acked Read 9 Done Sent Done false
28294 9048 Acked Read 6 Done Sent Done false
28295 9049 Acked Write 3 Done Sent Done false
28296 9050 Acked Write 3 Done Sent Done false
28297 9051 Acked Write 9 Done Sent Done false
28298 9052 Acked Read 2 Done Sent Done false
28299 9053 Acked Write 3 Done Sent Done false
28300 9054 Acked Read 6 Done Sent Done false
28301 9055 Acked Write 5 Done Sent Done false
28302 9056 Acked Write 2 Done Sent Done false
28303 9057 Acked Write 7 Done Sent Done false
28304 9058 Acked Write 10 Done Sent Done false
28305 9059 Acked Write 10 Done Sent Done false
28306 9060 Acked Flush 0 Done Sent Done false
28307 9061 Acked Write 8 Done Sent Done false
28308 9062 Acked Read 10 Done Sent Done false
28309 9063 Acked Write 10 Done Sent Done false
28310 9064 Acked Read 3 Done Sent Done false
28311 9065 Acked Flush 0 Done Sent Done false
28312 9066 Acked Flush 0 Done Sent Done false
28313 9067 Acked Write 9 Done Sent Done false
28314 9068 Acked Read 9 Done Sent Done false
28315 9069 Acked Write 8 Done Sent Done false
28316 9070 Acked Read 1 Done Sent Done false
28317 9071 Acked Read 4 Done Sent Done false
28318 9072 Acked Read 3 Done Sent Done false
28319 9073 Acked Read 2 Done Sent Done false
28320 9074 Acked Write 9 Done Sent Done false
28321 9075 Acked Read 6 Done Sent Done false
28322 9076 Acked Read 8 Done Sent Done false
28323 9077 Acked Read 10 Done Sent Done false
28324 9078 Acked Flush 0 Done Sent Done false
28325 9079 Acked Flush 0 Done Sent Done false
28326 9080 Acked Read 5 Done Sent Done false
28327 9081 Acked Write 9 Done Sent Done false
28328 9082 Acked Write 9 Done Sent Done false
28329 9083 Acked Write 5 Done Sent Done false
28330 9084 Acked Read 3 Done Sent Done false
28331 9085 Acked Read 6 Done Sent Done false
28332 9086 Acked Read 4 Done Sent Done false
28333 9087 Acked Read 10 Done Sent Done false
28334 9088 Acked Write 5 Done Sent Done false
28335 9089 Acked Flush 0 Done Sent Done false
28336 9090 Acked Write 7 Done Sent Done false
28337 9091 Acked Flush 0 Done Sent Done false
28338 9092 Acked Read 9 Done Sent Done false
28339 9093 Acked Read 1 Done Sent Done false
28340 9094 Acked Write 3 Done Sent Done false
28341 9095 Acked Read 4 Done Sent Done false
28342 9096 Acked Read 7 Done Sent Done false
28343 9097 Acked Read 5 Done Sent Done false
28344 9098 Acked Read 6 Done Sent Done false
28345 9099 Acked Write 10 Done Sent Done false
28346 9100 Acked Write 3 Done Sent Done false
28347 9101 Acked Read 9 Done Sent Done false
28348 9102 Acked Read 9 Done Sent Done false
28349 9103 Acked Read 4 Done Sent Done false
28350 9104 Acked Read 9 Done Sent Done false
28351 9105 Acked Read 6 Done Sent Done false
28352 9106 Acked Read 4 Done Sent Done false
28353 9107 Acked Read 7 Done Sent Done false
28354 9108 Acked Read 10 Done Sent Done false
28355 9109 Acked Read 4 Done Sent Done false
28356 9110 Acked Read 7 Done Sent Done false
28357 9111 Acked Read 6 Done Sent Done false
28358 9112 Acked Write 2 Done Sent Done false
28359 9113 Acked Read 1 Done Sent Done false
28360 9114 Acked Write 8 Done Sent Done false
28361 9115 Acked Read 9 Done Sent Done false
28362 9116 Acked Write 6 Done Sent Done false
28363 9117 Acked Read 3 Done Sent Done false
28364 9118 Acked Write 6 Done Sent Done false
28365 9119 Acked Flush 0 Done Sent Done false
28366 9120 Acked Read 3 Done Sent Done false
28367 9121 Acked Read 5 Done Sent Done false
28368 9122 Acked Write 7 Done Sent Done false
28369 9123 Acked Write 7 Done Sent Done false
28370 9124 Acked Read 4 Done Sent Done false
28371 9125 Acked Flush 0 Done Sent Done false
28372 9126 Acked Write 3 Done Sent Done false
28373 9127 Acked Read 2 Done Sent Done false
28374 9128 Acked Read 4 Done Sent Done false
28375 9129 Acked Read 4 Done Sent Done false
28376 9130 Acked Read 10 Done Sent Done false
28377 9131 Acked Write 3 Done Sent Done false
28378 9132 Acked Read 5 Done Sent Done false
28379 9133 Acked Read 5 Done Sent Done false
28380 9134 Acked Flush 0 Done Sent Done false
28381 9135 Acked Write 10 Done Sent Done false
28382 9136 Acked Flush 0 Done Sent Done false
28383 9137 Acked Read 8 Done Sent Done false
28384 9138 Acked Write 2 Done Sent Done false
28385 9139 Acked Read 3 Done Sent Done false
28386 9140 Acked Write 4 Done Sent Done false
28387 9141 Acked Write 7 Done Sent Done false
28388 9142 Acked Write 9 Done Sent Done false
28389 9143 Acked Read 1 Done Sent Done false
28390 9144 Acked Write 1 Done Sent Done false
28391 9145 Acked Read 8 Done Sent Done false
28392 9146 Acked Write 9 Done Sent Done false
28393 9147 Acked Write 7 Done Sent Done false
28394 9148 Acked Read 2 Done Sent Done false
28395 9149 Acked Read 5 Done Sent Done false
28396 9150 Acked Write 6 Done Sent Done false
28397 9151 Acked Flush 0 Done Sent Done false
28398 STATES DS:0 DS:1 DS:2 TOTAL
28399 Sent 0 302 0 302
28400 Done 302 0 302 604
28401 Skipped 0 0 0 0
28402 Error 0 0 0 0
28403 Last Flush: 9151 8849 9151
28404 Downstairs last five completed: 8849 8848 8847 8846 8845
28405 Upstairs last five completed: 9151 9150 9149 9148 9147
28406 CLIENT: Up:0 ds:302 act:2
284072025-01-11T06:11:23.036ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
284082025-01-11T06:11:23.036ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
284092025-01-11T06:11:23.036ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
284102025-01-11T06:11:23.036ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
284112025-01-11T06:11:23.037ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
28412 ----------------------------------------------------------------
28413 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28414 States: Active Offline Active
28415 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28416 8850 Acked Write 5 Done Sent Done false
28417 8851 Acked Read 5 Done Sent Done false
28418 8852 Acked Read 8 Done Sent Done false
28419 8853 Acked Write 4 Done Sent Done false
28420 8854 Acked Write 10 Done Sent Done false
28421 8855 Acked Write 9 Done Sent Done false
28422 8856 Acked Read 6 Done Sent Done false
28423 8857 Acked Read 3 Done Sent Done false
28424 8858 Acked Read 2 Done Sent Done false
28425 8859 Acked Read 1 Done Sent Done false
28426 8860 Acked Write 5 Done Sent Done false
28427 8861 Acked Read 8 Done Sent Done false
28428 8862 Acked Write 2 Done Sent Done false
28429 8863 Acked Read 7 Done Sent Done false
28430 8864 Acked Flush 0 Done Sent Done false
28431 8865 Acked Read 3 Done Sent Done false
28432 8866 Acked Read 10 Done Sent Done false
28433 8867 Acked Read 1 Done Sent Done false
28434 8868 Acked Read 10 Done Sent Done false
28435 8869 Acked Read 8 Done Sent Done false
28436 8870 Acked Flush 0 Done Sent Done false
28437 8871 Acked Read 6 Done Sent Done false
28438 8872 Acked Read 9 Done Sent Done false
28439 8873 Acked Write 3 Done Sent Done false
28440 8874 Acked Write 8 Done Sent Done false
28441 8875 Acked Read 4 Done Sent Done false
28442 8876 Acked Read 4 Done Sent Done false
28443 8877 Acked Write 9 Done Sent Done false
28444 8878 Acked Write 9 Done Sent Done false
28445 8879 Acked Read 7 Done Sent Done false
28446 8880 Acked Write 8 Done Sent Done false
28447 8881 Acked Read 5 Done Sent Done false
28448 8882 Acked Write 1 Done Sent Done false
28449 8883 Acked Write 5 Done Sent Done false
28450 8884 Acked Write 6 Done Sent Done false
28451 8885 Acked Write 3 Done Sent Done false
28452 8886 Acked Write 7 Done Sent Done false
28453 8887 Acked Flush 0 Done Sent Done false
28454 8888 Acked Read 10 Done Sent Done false
28455 8889 Acked Read 10 Done Sent Done false
28456 8890 Acked Write 7 Done Sent Done false
28457 8891 Acked Read 9 Done Sent Done false
28458 8892 Acked Read 9 Done Sent Done false
28459 8893 Acked Read 8 Done Sent Done false
28460 8894 Acked Read 4 Done Sent Done false
28461 8895 Acked Read 7 Done Sent Done false
28462 8896 Acked Write 8 Done Sent Done false
28463 8897 Acked Read 6 Done Sent Done false
28464 8898 Acked Read 2 Done Sent Done false
28465 8899 Acked Read 5 Done Sent Done false
28466 8900 Acked Write 5 Done Sent Done false
28467 8901 Acked Write 1 Done Sent Done false
28468 8902 Acked Write 10 Done Sent Done false
28469 8903 Acked Write 6 Done Sent Done false
28470 8904 Acked Read 7 Done Sent Done false
28471 8905 Acked Read 10 Done Sent Done false
28472 8906 Acked Write 3 Done Sent Done false
28473 8907 Acked Write 7 Done Sent Done false
28474 8908 Acked Write 1 Done Sent Done false
28475 8909 Acked Read 9 Done Sent Done false
28476 8910 Acked Read 5 Done Sent Done false
28477 8911 Acked Flush 0 Done Sent Done false
28478 8912 Acked Write 7 Done Sent Done false
28479 8913 Acked Read 8 Done Sent Done false
28480 8914 Acked Write 8 Done Sent Done false
28481 8915 Acked Write 9 Done Sent Done false
28482 8916 Acked Flush 0 Done Sent Done false
28483 8917 Acked Read 7 Done Sent Done false
28484 8918 Acked Read 3 Done Sent Done false
28485 8919 Acked Read 3 Done Sent Done false
28486 8920 Acked Read 4 Done Sent Done false
28487 8921 Acked Read 10 Done Sent Done false
28488 8922 Acked Write 8 Done Sent Done false
28489 8923 Acked Write 8 Done Sent Done false
28490 8924 Acked Read 5 Done Sent Done false
28491 8925 Acked Read 3 Done Sent Done false
28492 8926 Acked Write 5 Done Sent Done false
28493 8927 Acked Write 7 Done Sent Done false
28494 8928 Acked Write 7 Done Sent Done false
28495 8929 Acked Write 9 Done Sent Done false
28496 8930 Acked Read 7 Done Sent Done false
28497 8931 Acked Write 7 Done Sent Done false
28498 8932 Acked Read 7 Done Sent Done false
28499 8933 Acked Write 1 Done Sent Done false
28500 8934 Acked Read 6 Done Sent Done false
28501 8935 Acked Read 7 Done Sent Done false
28502 8936 Acked Write 6 Done Sent Done false
28503 8937 Acked Write 8 Done Sent Done false
28504 8938 Acked Read 3 Done Sent Done false
28505 8939 Acked Write 6 Done Sent Done false
28506 8940 Acked Flush 0 Done Sent Done false
28507 8941 Acked Write 7 Done Sent Done false
28508 8942 Acked Read 7 Done Sent Done false
28509 8943 Acked Flush 0 Done Sent Done false
28510 8944 Acked Read 3 Done Sent Done false
28511 8945 Acked Read 8 Done Sent Done false
28512 8946 Acked Write 5 Done Sent Done false
28513 8947 Acked Read 10 Done Sent Done false
28514 8948 Acked Write 10 Done Sent Done false
28515 8949 Acked Read 10 Done Sent Done false
28516 8950 Acked Read 5 Done Sent Done false
28517 8951 Acked Write 1 Done Sent Done false
28518 8952 Acked Read 4 Done Sent Done false
28519 8953 Acked Read 5 Done Sent Done false
28520 8954 Acked Write 10 Done Sent Done false
28521 8955 Acked Write 10 Done Sent Done false
28522 8956 Acked Write 5 Done Sent Done false
28523 8957 Acked Read 9 Done Sent Done false
28524 8958 Acked Write 3 Done Sent Done false
28525 8959 Acked Write 8 Done Sent Done false
28526 8960 Acked Write 6 Done Sent Done false
28527 8961 Acked Write 4 Done Sent Done false
28528 8962 Acked Read 6 Done Sent Done false
28529 8963 Acked Write 3 Done Sent Done false
28530 8964 Acked Read 9 Done Sent Done false
28531 8965 Acked Flush 0 Done Sent Done false
28532 8966 Acked Read 2 Done Sent Done false
28533 8967 Acked Write 6 Done Sent Done false
28534 8968 Acked Read 10 Done Sent Done false
28535 8969 Acked Write 7 Done Sent Done false
28536 8970 Acked Read 3 Done Sent Done false
28537 8971 Acked Write 5 Done Sent Done false
28538 8972 Acked Read 6 Done Sent Done false
28539 8973 Acked Write 9 Done Sent Done false
28540 8974 Acked Write 9 Done Sent Done false
28541 8975 Acked Write 9 Done Sent Done false
28542 8976 Acked Write 6 Done Sent Done false
28543 8977 Acked Write 6 Done Sent Done false
28544 8978 Acked Write 1 Done Sent Done false
28545 8979 Acked Read 2 Done Sent Done false
28546 8980 Acked Write 7 Done Sent Done false
28547 8981 Acked Read 3 Done Sent Done false
28548 8982 Acked Write 2 Done Sent Done false
28549 8983 Acked Write 6 Done Sent Done false
28550 8984 Acked Read 7 Done Sent Done false
28551 8985 Acked Write 1 Done Sent Done false
28552 8986 Acked Read 1 Done Sent Done false
28553 8987 Acked Read 2 Done Sent Done false
28554 8988 Acked Read 7 Done Sent Done false
28555 8989 Acked Write 6 Done Sent Done false
28556 8990 Acked Read 7 Done Sent Done false
28557 8991 Acked Write 9 Done Sent Done false
28558 8992 Acked Flush 0 Done Sent Done false
28559 8993 Acked Read 5 Done Sent Done false
28560 8994 Acked Read 5 Done Sent Done false
28561 8995 Acked Read 8 Done Sent Done false
28562 8996 Acked Write 6 Done Sent Done false
28563 8997 Acked Read 4 Done Sent Done false
28564 8998 Acked Read 9 Done Sent Done false
28565 8999 Acked Write 4 Done Sent Done false
28566 9000 Acked Write 6 Done Sent Done false
28567 9001 Acked Flush 0 Done Sent Done false
28568 9002 Acked Read 6 Done Sent Done false
28569 9003 Acked Read 9 Done Sent Done false
28570 9004 Acked Flush 0 Done Sent Done false
28571 9005 Acked Write 10 Done Sent Done false
28572 9006 Acked Read 5 Done Sent Done false
28573 9007 Acked Read 1 Done Sent Done false
28574 9008 Acked Write 7 Done Sent Done false
28575 9009 Acked Write 5 Done Sent Done false
28576 9010 Acked Write 1 Done Sent Done false
28577 9011 Acked Write 8 Done Sent Done false
28578 9012 Acked Read 5 Done Sent Done false
28579 9013 Acked Write 8 Done Sent Done false
28580 9014 Acked Read 5 Done Sent Done false
28581 9015 Acked Read 5 Done Sent Done false
28582 9016 Acked Write 6 Done Sent Done false
28583 9017 Acked Write 3 Done Sent Done false
28584 9018 Acked Read 8 Done Sent Done false
28585 9019 Acked Read 2 Done Sent Done false
28586 9020 Acked Read 7 Done Sent Done false
28587 9021 Acked Write 1 Done Sent Done false
28588 9022 Acked Write 7 Done Sent Done false
28589 9023 Acked Read 8 Done Sent Done false
28590 9024 Acked Read 5 Done Sent Done false
28591 9025 Acked Read 9 Done Sent Done false
28592 9026 Acked Read 1 Done Sent Done false
28593 9027 Acked Write 1 Done Sent Done false
28594 9028 Acked Flush 0 Done Sent Done false
28595 9029 Acked Read 10 Done Sent Done false
28596 9030 Acked Read 3 Done Sent Done false
28597 9031 Acked Read 10 Done Sent Done false
28598 9032 Acked Flush 0 Done Sent Done false
28599 9033 Acked Read 9 Done Sent Done false
28600 9034 Acked Read 2 Done Sent Done false
28601 9035 Acked Read 1 Done Sent Done false
28602 9036 Acked Read 6 Done Sent Done false
28603 9037 Acked Read 6 Done Sent Done false
28604 9038 Acked Write 5 Done Sent Done false
28605 9039 Acked Write 7 Done Sent Done false
28606 9040 Acked Flush 0 Done Sent Done false
28607 9041 Acked Write 4 Done Sent Done false
28608 9042 Acked Read 8 Done Sent Done false
28609 9043 Acked Flush 0 Done Sent Done false
28610 9044 Acked Write 7 Done Sent Done false
28611 9045 Acked Write 1 Done Sent Done false
28612 9046 Acked Read 1 Done Sent Done false
28613 9047 Acked Read 9 Done Sent Done false
28614 9048 Acked Read 6 Done Sent Done false
28615 9049 Acked Write 3 Done Sent Done false
28616 9050 Acked Write 3 Done Sent Done false
28617 9051 Acked Write 9 Done Sent Done false
28618 9052 Acked Read 2 Done Sent Done false
28619 9053 Acked Write 3 Done Sent Done false
28620 9054 Acked Read 6 Done Sent Done false
28621 9055 Acked Write 5 Done Sent Done false
28622 9056 Acked Write 2 Done Sent Done false
28623 9057 Acked Write 7 Done Sent Done false
28624 9058 Acked Write 10 Done Sent Done false
28625 9059 Acked Write 10 Done Sent Done false
28626 9060 Acked Flush 0 Done Sent Done false
28627 9061 Acked Write 8 Done Sent Done false
28628 9062 Acked Read 10 Done Sent Done false
28629 9063 Acked Write 10 Done Sent Done false
28630 9064 Acked Read 3 Done Sent Done false
28631 9065 Acked Flush 0 Done Sent Done false
28632 9066 Acked Flush 0 Done Sent Done false
28633 9067 Acked Write 9 Done Sent Done false
28634 9068 Acked Read 9 Done Sent Done false
28635 9069 Acked Write 8 Done Sent Done false
28636 9070 Acked Read 1 Done Sent Done false
28637 9071 Acked Read 4 Done Sent Done false
28638 9072 Acked Read 3 Done Sent Done false
28639 9073 Acked Read 2 Done Sent Done false
28640 9074 Acked Write 9 Done Sent Done false
28641 9075 Acked Read 6 Done Sent Done false
28642 9076 Acked Read 8 Done Sent Done false
28643 9077 Acked Read 10 Done Sent Done false
28644 9078 Acked Flush 0 Done Sent Done false
28645 9079 Acked Flush 0 Done Sent Done false
28646 9080 Acked Read 5 Done Sent Done false
28647 9081 Acked Write 9 Done Sent Done false
28648 9082 Acked Write 9 Done Sent Done false
28649 9083 Acked Write 5 Done Sent Done false
28650 9084 Acked Read 3 Done Sent Done false
28651 9085 Acked Read 6 Done Sent Done false
28652 9086 Acked Read 4 Done Sent Done false
28653 9087 Acked Read 10 Done Sent Done false
28654 9088 Acked Write 5 Done Sent Done false
28655 9089 Acked Flush 0 Done Sent Done false
28656 9090 Acked Write 7 Done Sent Done false
28657 9091 Acked Flush 0 Done Sent Done false
28658 9092 Acked Read 9 Done Sent Done false
28659 9093 Acked Read 1 Done Sent Done false
28660 9094 Acked Write 3 Done Sent Done false
28661 9095 Acked Read 4 Done Sent Done false
28662 9096 Acked Read 7 Done Sent Done false
28663 9097 Acked Read 5 Done Sent Done false
28664 9098 Acked Read 6 Done Sent Done false
28665 9099 Acked Write 10 Done Sent Done false
28666 9100 Acked Write 3 Done Sent Done false
28667 9101 Acked Read 9 Done Sent Done false
28668 9102 Acked Read 9 Done Sent Done false
28669 9103 Acked Read 4 Done Sent Done false
28670 9104 Acked Read 9 Done Sent Done false
28671 9105 Acked Read 6 Done Sent Done false
28672 9106 Acked Read 4 Done Sent Done false
28673 9107 Acked Read 7 Done Sent Done false
28674 9108 Acked Read 10 Done Sent Done false
28675 9109 Acked Read 4 Done Sent Done false
28676 9110 Acked Read 7 Done Sent Done false
28677 9111 Acked Read 6 Done Sent Done false
28678 9112 Acked Write 2 Done Sent Done false
28679 9113 Acked Read 1 Done Sent Done false
28680 9114 Acked Write 8 Done Sent Done false
28681 9115 Acked Read 9 Done Sent Done false
28682 9116 Acked Write 6 Done Sent Done false
28683 9117 Acked Read 3 Done Sent Done false
28684 9118 Acked Write 6 Done Sent Done false
28685 9119 Acked Flush 0 Done Sent Done false
28686 9120 Acked Read 3 Done Sent Done false
28687 9121 Acked Read 5 Done Sent Done false
28688 9122 Acked Write 7 Done Sent Done false
28689 9123 Acked Write 7 Done Sent Done false
28690 9124 Acked Read 4 Done Sent Done false
28691 9125 Acked Flush 0 Done Sent Done false
28692 9126 Acked Write 3 Done Sent Done false
28693 9127 Acked Read 2 Done Sent Done false
28694 9128 Acked Read 4 Done Sent Done false
28695 9129 Acked Read 4 Done Sent Done false
28696 9130 Acked Read 10 Done Sent Done false
28697 9131 Acked Write 3 Done Sent Done false
28698 9132 Acked Read 5 Done Sent Done false
28699 9133 Acked Read 5 Done Sent Done false
28700 9134 Acked Flush 0 Done Sent Done false
28701 9135 Acked Write 10 Done Sent Done false
28702 9136 Acked Flush 0 Done Sent Done false
28703 9137 Acked Read 8 Done Sent Done false
28704 9138 Acked Write 2 Done Sent Done false
28705 9139 Acked Read 3 Done Sent Done false
28706 9140 Acked Write 4 Done Sent Done false
28707 9141 Acked Write 7 Done Sent Done false
28708 9142 Acked Write 9 Done Sent Done false
28709 9143 Acked Read 1 Done Sent Done false
28710 9144 Acked Write 1 Done Sent Done false
28711 9145 Acked Read 8 Done Sent Done false
28712 9146 Acked Write 9 Done Sent Done false
28713 9147 Acked Write 7 Done Sent Done false
28714 9148 Acked Read 2 Done Sent Done false
28715 9149 Acked Read 5 Done Sent Done false
28716 9150 Acked Write 6 Done Sent Done false
28717 9151 Acked Flush 0 Done Sent Done false
28718 STATES DS:0 DS:1 DS:2 TOTAL
28719 Sent 0 302 0 302
28720 Done 302 0 302 604
28721 Skipped 0 0 0 0
28722 Error 0 0 0 0
28723 Last Flush: 9151 8849 9151
28724 Downstairs last five completed: 8849 8848 8847 8846 8845
28725 Upstairs last five completed: 9151 9150 9149 9148 9147
28726 CLIENT: Up:0 ds:302 act:2
28727 [8820] got start action
28728 [8820] I am going to restart
28729 Starting downstairs at port 8820
28730 Make output file at "/tmp/dsc/downstairs-8820.txt"
28731 [8820][1] reports Starting
28732 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1231)
28733 [8820][1] reports Running
28734 ----------------------------------------------------------------
28735 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28736 States: Active Offline Active
28737 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28738 8850 Acked Write 5 Done Sent Done false
28739 8851 Acked Read 5 Done Sent Done false
28740 8852 Acked Read 8 Done Sent Done false
28741 8853 Acked Write 4 Done Sent Done false
28742 8854 Acked Write 10 Done Sent Done false
28743 8855 Acked Write 9 Done Sent Done false
28744 8856 Acked Read 6 Done Sent Done false
28745 8857 Acked Read 3 Done Sent Done false
28746 8858 Acked Read 2 Done Sent Done false
28747 8859 Acked Read 1 Done Sent Done false
28748 8860 Acked Write 5 Done Sent Done false
28749 8861 Acked Read 8 Done Sent Done false
28750 8862 Acked Write 2 Done Sent Done false
28751 8863 Acked Read 7 Done Sent Done false
28752 8864 Acked Flush 0 Done Sent Done false
28753 8865 Acked Read 3 Done Sent Done false
28754 8866 Acked Read 10 Done Sent Done false
28755 8867 Acked Read 1 Done Sent Done false
28756 8868 Acked Read 10 Done Sent Done false
28757 8869 Acked Read 8 Done Sent Done false
28758 8870 Acked Flush 0 Done Sent Done false
28759 8871 Acked Read 6 Done Sent Done false
28760 8872 Acked Read 9 Done Sent Done false
28761 8873 Acked Write 3 Done Sent Done false
28762 8874 Acked Write 8 Done Sent Done false
28763 8875 Acked Read 4 Done Sent Done false
28764 8876 Acked Read 4 Done Sent Done false
28765 8877 Acked Write 9 Done Sent Done false
28766 8878 Acked Write 9 Done Sent Done false
28767 8879 Acked Read 7 Done Sent Done false
28768 8880 Acked Write 8 Done Sent Done false
28769 8881 Acked Read 5 Done Sent Done false
28770 8882 Acked Write 1 Done Sent Done false
28771 8883 Acked Write 5 Done Sent Done false
28772 8884 Acked Write 6 Done Sent Done false
28773 8885 Acked Write 3 Done Sent Done false
28774 8886 Acked Write 7 Done Sent Done false
28775 8887 Acked Flush 0 Done Sent Done false
28776 8888 Acked Read 10 Done Sent Done false
28777 8889 Acked Read 10 Done Sent Done false
28778 8890 Acked Write 7 Done Sent Done false
28779 8891 Acked Read 9 Done Sent Done false
28780 8892 Acked Read 9 Done Sent Done false
28781 8893 Acked Read 8 Done Sent Done false
28782 8894 Acked Read 4 Done Sent Done false
28783 8895 Acked Read 7 Done Sent Done false
28784 8896 Acked Write 8 Done Sent Done false
28785 8897 Acked Read 6 Done Sent Done false
28786 8898 Acked Read 2 Done Sent Done false
28787 8899 Acked Read 5 Done Sent Done false
28788 8900 Acked Write 5 Done Sent Done false
28789 8901 Acked Write 1 Done Sent Done false
28790 8902 Acked Write 10 Done Sent Done false
28791 8903 Acked Write 6 Done Sent Done false
28792 8904 Acked Read 7 Done Sent Done false
28793 8905 Acked Read 10 Done Sent Done false
28794 8906 Acked Write 3 Done Sent Done false
28795 8907 Acked Write 7 Done Sent Done false
28796 8908 Acked Write 1 Done Sent Done false
28797 8909 Acked Read 9 Done Sent Done false
28798 8910 Acked Read 5 Done Sent Done false
28799 8911 Acked Flush 0 Done Sent Done false
28800 8912 Acked Write 7 Done Sent Done false
28801 8913 Acked Read 8 Done Sent Done false
28802 8914 Acked Write 8 Done Sent Done false
28803 8915 Acked Write 9 Done Sent Done false
28804 8916 Acked Flush 0 Done Sent Done false
28805 8917 Acked Read 7 Done Sent Done false
28806 8918 Acked Read 3 Done Sent Done false
28807 8919 Acked Read 3 Done Sent Done false
28808 8920 Acked Read 4 Done Sent Done false
28809 8921 Acked Read 10 Done Sent Done false
28810 8922 Acked Write 8 Done Sent Done false
28811 8923 Acked Write 8 Done Sent Done false
28812 8924 Acked Read 5 Done Sent Done false
28813 8925 Acked Read 3 Done Sent Done false
28814 8926 Acked Write 5 Done Sent Done false
28815 8927 Acked Write 7 Done Sent Done false
28816 8928 Acked Write 7 Done Sent Done false
28817 8929 Acked Write 9 Done Sent Done false
28818 8930 Acked Read 7 Done Sent Done false
28819 8931 Acked Write 7 Done Sent Done false
28820 8932 Acked Read 7 Done Sent Done false
28821 8933 Acked Write 1 Done Sent Done false
28822 8934 Acked Read 6 Done Sent Done false
28823 8935 Acked Read 7 Done Sent Done false
28824 8936 Acked Write 6 Done Sent Done false
28825 8937 Acked Write 8 Done Sent Done false
28826 8938 Acked Read 3 Done Sent Done false
28827 8939 Acked Write 6 Done Sent Done false
28828 8940 Acked Flush 0 Done Sent Done false
28829 8941 Acked Write 7 Done Sent Done false
28830 8942 Acked Read 7 Done Sent Done false
28831 8943 Acked Flush 0 Done Sent Done false
28832 8944 Acked Read 3 Done Sent Done false
28833 8945 Acked Read 8 Done Sent Done false
28834 8946 Acked Write 5 Done Sent Done false
28835 8947 Acked Read 10 Done Sent Done false
28836 8948 Acked Write 10 Done Sent Done false
28837 8949 Acked Read 10 Done Sent Done false
28838 8950 Acked Read 5 Done Sent Done false
28839 8951 Acked Write 1 Done Sent Done false
28840 8952 Acked Read 4 Done Sent Done false
28841 8953 Acked Read 5 Done Sent Done false
28842 8954 Acked Write 10 Done Sent Done false
28843 8955 Acked Write 10 Done Sent Done false
28844 8956 Acked Write 5 Done Sent Done false
28845 8957 Acked Read 9 Done Sent Done false
28846 8958 Acked Write 3 Done Sent Done false
28847 8959 Acked Write 8 Done Sent Done false
28848 8960 Acked Write 6 Done Sent Done false
28849 8961 Acked Write 4 Done Sent Done false
28850 8962 Acked Read 6 Done Sent Done false
28851 8963 Acked Write 3 Done Sent Done false
28852 8964 Acked Read 9 Done Sent Done false
28853 8965 Acked Flush 0 Done Sent Done false
28854 8966 Acked Read 2 Done Sent Done false
28855 8967 Acked Write 6 Done Sent Done false
28856 8968 Acked Read 10 Done Sent Done false
28857 8969 Acked Write 7 Done Sent Done false
28858 8970 Acked Read 3 Done Sent Done false
28859 8971 Acked Write 5 Done Sent Done false
28860 8972 Acked Read 6 Done Sent Done false
28861 8973 Acked Write 9 Done Sent Done false
28862 8974 Acked Write 9 Done Sent Done false
28863 8975 Acked Write 9 Done Sent Done false
28864 8976 Acked Write 6 Done Sent Done false
28865 8977 Acked Write 6 Done Sent Done false
28866 8978 Acked Write 1 Done Sent Done false
28867 8979 Acked Read 2 Done Sent Done false
28868 8980 Acked Write 7 Done Sent Done false
28869 8981 Acked Read 3 Done Sent Done false
28870 8982 Acked Write 2 Done Sent Done false
28871 8983 Acked Write 6 Done Sent Done false
28872 8984 Acked Read 7 Done Sent Done false
28873 8985 Acked Write 1 Done Sent Done false
28874 8986 Acked Read 1 Done Sent Done false
28875 8987 Acked Read 2 Done Sent Done false
28876 8988 Acked Read 7 Done Sent Done false
28877 8989 Acked Write 6 Done Sent Done false
28878 8990 Acked Read 7 Done Sent Done false
28879 8991 Acked Write 9 Done Sent Done false
28880 8992 Acked Flush 0 Done Sent Done false
28881 8993 Acked Read 5 Done Sent Done false
28882 8994 Acked Read 5 Done Sent Done false
28883 8995 Acked Read 8 Done Sent Done false
28884 8996 Acked Write 6 Done Sent Done false
28885 8997 Acked Read 4 Done Sent Done false
28886 8998 Acked Read 9 Done Sent Done false
28887 8999 Acked Write 4 Done Sent Done false
28888 9000 Acked Write 6 Done Sent Done false
28889 9001 Acked Flush 0 Done Sent Done false
28890 9002 Acked Read 6 Done Sent Done false
28891 9003 Acked Read 9 Done Sent Done false
28892 9004 Acked Flush 0 Done Sent Done false
28893 9005 Acked Write 10 Done Sent Done false
28894 9006 Acked Read 5 Done Sent Done false
28895 9007 Acked Read 1 Done Sent Done false
28896 9008 Acked Write 7 Done Sent Done false
28897 9009 Acked Write 5 Done Sent Done false
28898 9010 Acked Write 1 Done Sent Done false
28899 9011 Acked Write 8 Done Sent Done false
28900 9012 Acked Read 5 Done Sent Done false
28901 9013 Acked Write 8 Done Sent Done false
28902 9014 Acked Read 5 Done Sent Done false
28903 9015 Acked Read 5 Done Sent Done false
28904 9016 Acked Write 6 Done Sent Done false
28905 9017 Acked Write 3 Done Sent Done false
28906 9018 Acked Read 8 Done Sent Done false
28907 9019 Acked Read 2 Done Sent Done false
28908 9020 Acked Read 7 Done Sent Done false
28909 9021 Acked Write 1 Done Sent Done false
28910 9022 Acked Write 7 Done Sent Done false
28911 9023 Acked Read 8 Done Sent Done false
28912 9024 Acked Read 5 Done Sent Done false
28913 9025 Acked Read 9 Done Sent Done false
28914 9026 Acked Read 1 Done Sent Done false
28915 9027 Acked Write 1 Done Sent Done false
28916 9028 Acked Flush 0 Done Sent Done false
28917 9029 Acked Read 10 Done Sent Done false
28918 9030 Acked Read 3 Done Sent Done false
28919 9031 Acked Read 10 Done Sent Done false
28920 9032 Acked Flush 0 Done Sent Done false
28921 9033 Acked Read 9 Done Sent Done false
28922 9034 Acked Read 2 Done Sent Done false
28923 9035 Acked Read 1 Done Sent Done false
28924 9036 Acked Read 6 Done Sent Done false
28925 9037 Acked Read 6 Done Sent Done false
28926 9038 Acked Write 5 Done Sent Done false
28927 9039 Acked Write 7 Done Sent Done false
28928 9040 Acked Flush 0 Done Sent Done false
28929 9041 Acked Write 4 Done Sent Done false
28930 9042 Acked Read 8 Done Sent Done false
28931 9043 Acked Flush 0 Done Sent Done false
28932 9044 Acked Write 7 Done Sent Done false
28933 9045 Acked Write 1 Done Sent Done false
28934 9046 Acked Read 1 Done Sent Done false
28935 9047 Acked Read 9 Done Sent Done false
28936 9048 Acked Read 6 Done Sent Done false
28937 9049 Acked Write 3 Done Sent Done false
28938 9050 Acked Write 3 Done Sent Done false
28939 9051 Acked Write 9 Done Sent Done false
28940 9052 Acked Read 2 Done Sent Done false
28941 9053 Acked Write 3 Done Sent Done false
28942 9054 Acked Read 6 Done Sent Done false
28943 9055 Acked Write 5 Done Sent Done false
28944 9056 Acked Write 2 Done Sent Done false
28945 9057 Acked Write 7 Done Sent Done false
28946 9058 Acked Write 10 Done Sent Done false
28947 9059 Acked Write 10 Done Sent Done false
28948 9060 Acked Flush 0 Done Sent Done false
28949 9061 Acked Write 8 Done Sent Done false
28950 9062 Acked Read 10 Done Sent Done false
28951 9063 Acked Write 10 Done Sent Done false
28952 9064 Acked Read 3 Done Sent Done false
28953 9065 Acked Flush 0 Done Sent Done false
28954 9066 Acked Flush 0 Done Sent Done false
28955 9067 Acked Write 9 Done Sent Done false
28956 9068 Acked Read 9 Done Sent Done false
28957 9069 Acked Write 8 Done Sent Done false
28958 9070 Acked Read 1 Done Sent Done false
28959 9071 Acked Read 4 Done Sent Done false
28960 9072 Acked Read 3 Done Sent Done false
28961 9073 Acked Read 2 Done Sent Done false
28962 9074 Acked Write 9 Done Sent Done false
28963 9075 Acked Read 6 Done Sent Done false
28964 9076 Acked Read 8 Done Sent Done false
28965 9077 Acked Read 10 Done Sent Done false
28966 9078 Acked Flush 0 Done Sent Done false
28967 9079 Acked Flush 0 Done Sent Done false
28968 9080 Acked Read 5 Done Sent Done false
28969 9081 Acked Write 9 Done Sent Done false
28970 9082 Acked Write 9 Done Sent Done false
28971 9083 Acked Write 5 Done Sent Done false
28972 9084 Acked Read 3 Done Sent Done false
28973 9085 Acked Read 6 Done Sent Done false
28974 9086 Acked Read 4 Done Sent Done false
28975 9087 Acked Read 10 Done Sent Done false
28976 9088 Acked Write 5 Done Sent Done false
28977 9089 Acked Flush 0 Done Sent Done false
28978 9090 Acked Write 7 Done Sent Done false
28979 9091 Acked Flush 0 Done Sent Done false
28980 9092 Acked Read 9 Done Sent Done false
28981 9093 Acked Read 1 Done Sent Done false
28982 9094 Acked Write 3 Done Sent Done false
28983 9095 Acked Read 4 Done Sent Done false
28984 9096 Acked Read 7 Done Sent Done false
28985 9097 Acked Read 5 Done Sent Done false
28986 9098 Acked Read 6 Done Sent Done false
28987 9099 Acked Write 10 Done Sent Done false
28988 9100 Acked Write 3 Done Sent Done false
28989 9101 Acked Read 9 Done Sent Done false
28990 9102 Acked Read 9 Done Sent Done false
28991 9103 Acked Read 4 Done Sent Done false
28992 9104 Acked Read 9 Done Sent Done false
28993 9105 Acked Read 6 Done Sent Done false
28994 9106 Acked Read 4 Done Sent Done false
28995 9107 Acked Read 7 Done Sent Done false
28996 9108 Acked Read 10 Done Sent Done false
28997 9109 Acked Read 4 Done Sent Done false
28998 9110 Acked Read 7 Done Sent Done false
28999 9111 Acked Read 6 Done Sent Done false
29000 9112 Acked Write 2 Done Sent Done false
29001 9113 Acked Read 1 Done Sent Done false
29002 9114 Acked Write 8 Done Sent Done false
29003 9115 Acked Read 9 Done Sent Done false
29004 9116 Acked Write 6 Done Sent Done false
29005 9117 Acked Read 3 Done Sent Done false
29006 9118 Acked Write 6 Done Sent Done false
29007 9119 Acked Flush 0 Done Sent Done false
29008 9120 Acked Read 3 Done Sent Done false
29009 9121 Acked Read 5 Done Sent Done false
29010 9122 Acked Write 7 Done Sent Done false
29011 9123 Acked Write 7 Done Sent Done false
29012 9124 Acked Read 4 Done Sent Done false
29013 9125 Acked Flush 0 Done Sent Done false
29014 9126 Acked Write 3 Done Sent Done false
29015 9127 Acked Read 2 Done Sent Done false
29016 9128 Acked Read 4 Done Sent Done false
29017 9129 Acked Read 4 Done Sent Done false
29018 9130 Acked Read 10 Done Sent Done false
29019 9131 Acked Write 3 Done Sent Done false
29020 9132 Acked Read 5 Done Sent Done false
29021 9133 Acked Read 5 Done Sent Done false
29022 9134 Acked Flush 0 Done Sent Done false
29023 9135 Acked Write 10 Done Sent Done false
29024 9136 Acked Flush 0 Done Sent Done false
29025 9137 Acked Read 8 Done Sent Done false
29026 9138 Acked Write 2 Done Sent Done false
29027 9139 Acked Read 3 Done Sent Done false
29028 9140 Acked Write 4 Done Sent Done false
29029 9141 Acked Write 7 Done Sent Done false
29030 9142 Acked Write 9 Done Sent Done false
29031 9143 Acked Read 1 Done Sent Done false
29032 9144 Acked Write 1 Done Sent Done false
29033 9145 Acked Read 8 Done Sent Done false
29034 9146 Acked Write 9 Done Sent Done false
29035 9147 Acked Write 7 Done Sent Done false
29036 9148 Acked Read 2 Done Sent Done false
29037 9149 Acked Read 5 Done Sent Done false
29038 9150 Acked Write 6 Done Sent Done false
29039 9151 Acked Flush 0 Done Sent Done false
29040 STATES DS:0 DS:1 DS:2 TOTAL
29041 Sent 0 302 0 302
29042 Done 302 0 302 604
29043 Skipped 0 0 0 0
29044 Error 0 0 0 0
29045 Last Flush: 9151 8849 9151
29046 Downstairs last five completed: 8849 8848 8847 8846 8845
29047 Upstairs last five completed: 9151 9150 9149 9148 9147
29048 CLIENT: Up:0 ds:302 act:2
290492025-01-11T06:11:33.038ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290502025-01-11T06:11:33.038ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37316) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290512025-01-11T06:11:33.039ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290522025-01-11T06:11:33.039ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290532025-01-11T06:11:33.039ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290542025-01-11T06:11:33.039ZINFOcrucible: send last flush ID to this DS: 8849 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290552025-01-11T06:11:33.040ZINFOcrucible: Replied this last flush ID: 8849 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290562025-01-11T06:11:33.040ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290572025-01-11T06:11:33.040ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290582025-01-11T06:11:33.040ZINFOcrucible: [1] client re-new 302 jobs since flush 8849 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290592025-01-11T06:11:33.040ZINFOcrucible: [1] Replayed 302 jobs since flush: 8849 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
29060 ----------------------------------------------------------------
29061 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29062 Downstairs last five completed: 9151 9150 9149 9148 9147
29063 Upstairs last five completed: 9151 9150 9149 9148 9147
29064 CLIENT: Up:0 ds:0 act:3
29065 Replay: All jobs finished
29066 Main task has work to do, go find it
29067 got dsc Stop(1)
29068 stop 1
29069 [8820] Got stop action so:false kr:false
29070 Jan 11 06:11:33.306 INFO request completed, latency_us: 401, response_code: 204, uri: /stop/cid/1, method: POST, req_id: da12e891-3302-418e-a17d-958d5889d3d3, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
29071 Jan 11 06:11:33.307 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3ca10d02-c489-49c8-b4a7-a08c5ef40cfd, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
290722025-01-11T06:11:33.307ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290732025-01-11T06:11:33.307ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290742025-01-11T06:11:33.307ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290752025-01-11T06:11:33.307ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290762025-01-11T06:11:33.307ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
290772025-01-11T06:11:33.307ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
29078 [8820] Exited with: ExitStatus(unix_wait_status(9))
29079 [8820][1] reports Exit
29080 Jan 11 06:11:37.309 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/1, method: GET, req_id: b7d93876-d164-40c4-8616-836ab5d975b7, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
29081 001/300 Write block 1156 len 40960 data: 3 2 2 2 2 2 3 3 3 3
29082 002/300 Read block 3179 len 8192
29083 003/300 Read block 1619 len 12288
29084 004/300 Write block 318 len 36864 data: 7 5 6 7 7 9 10 10 8
29085 005/300 Read block 3389 len 28672
29086 006/300 Flush
29087 007/300 Read block 1587 len 4096
29088 008/300 Write block 1640 len 24576 data: 5 7 6 6 5 5
29089 009/300 Write block 2875 len 24576 data: 5 6 6 5 5 6
29090 010/300 Read block 2659 len 12288
29091 011/300 Read block 3355 len 8192
29092 012/300 Write block 4374 len 4096 data: 4
29093 013/300 Write block 4796 len 12288 data: 9 10 9
29094 014/300 Read block 4593 len 36864
29095 015/300 Flush
29096 016/300 Read block 2497 len 8192
29097 017/300 Read block 1100 len 24576
29098 018/300 Flush
29099 019/300 Read block 1604 len 12288
29100 020/300 Write block 394 len 28672 data: 9 7 8 8 8 7 8
29101 021/300 Read block 3372 len 28672
29102 022/300 Flush
29103 023/300 Write block 3270 len 20480 data: 7 7 10 9 8
29104 024/300 Flush
29105 025/300 Read block 3290 len 12288
29106 026/300 Read block 329 len 36864
29107 027/300 Read block 3656 len 4096
29108 028/300 Read block 465 len 8192
29109 029/300 Flush
29110 030/300 Write block 1884 len 36864 data: 7 9 8 8 5 5 5 5 4
29111 031/300 Read block 106 len 24576
29112 032/300 Write block 2308 len 4096 data: 5
29113 033/300 Write block 1764 len 4096 data: 4
29114 034/300 Read block 2576 len 4096
29115 035/300 Write block 3542 len 32768 data: 4 3 4 4 4 5 6 4
29116 036/300 Read block 595 len 36864
29117 037/300 Flush
29118 038/300 Read block 359 len 28672
29119 039/300 Write block 4074 len 16384 data: 15 13 12 9
29120 040/300 Read block 4292 len 4096
29121 041/300 Read block 2355 len 8192
29122 042/300 Write block 4303 len 16384 data: 7 8 4 4
29123 043/300 Write block 3486 len 4096 data: 7
29124 044/300 Write block 3820 len 36864 data: 9 7 9 8 8 5 7 7 6
29125 045/300 Write block 2445 len 40960 data: 6 7 7 7 7 8 7 5 4 3
29126 046/300 Read block 638 len 36864
29127 047/300 Write block 2656 len 28672 data: 2 2 2 3 3 4 4
29128 048/300 Read block 2882 len 24576
29129 049/300 Flush
29130 050/300 Read block 621 len 40960
29131 051/300 Write block 4916 len 24576 data: 5 6 7 5 5 4
29132 052/300 Read block 813 len 20480
29133 053/300 Write block 2046 len 4096 data: 6
29134 054/300 Read block 4879 len 16384
29135 055/300 Write block 1606 len 20480 data: 4 4 4 5 6
29136 056/300 Write block 4894 len 4096 data: 7
29137 057/300 Write block 2793 len 12288 data: 4 5 5
29138 058/300 Read block 2414 len 32768
29139 059/300 Write block 3898 len 4096 data: 2
29140 060/300 Write block 3461 len 20480 data: 4 5 5 5 4
29141 061/300 Write block 697 len 8192 data: 8 5
29142 062/300 Write block 4486 len 40960 data: 4 4 4 4 4 4 3 2 2 2
29143 063/300 Read block 1789 len 36864
29144 064/300 Write block 3759 len 16384 data: 7 7 5 4
29145 065/300 Read block 3857 len 16384
29146 066/300 Read block 876 len 12288
29147 067/300 Flush
29148 068/300 Read block 1858 len 40960
29149 069/300 Write block 4805 len 8192 data: 5 4
29150 070/300 Flush
29151 071/300 Read block 1018 len 20480
29152 072/300 Write block 1984 len 16384 data: 5 4 2 5
29153 073/300 Write block 3164 len 20480 data: 3 3 2 2 2
29154 074/300 Write block 149 len 28672 data: 3 2 4 4 4 4 5
29155 075/300 Flush
29156 076/300 Read block 2156 len 16384
29157 077/300 Read block 3241 len 40960
29158 078/300 Write block 340 len 12288 data: 8 4 3
29159 079/300 Flush
29160 080/300 Read block 1273 len 40960
29161 081/300 Read block 597 len 4096
29162 082/300 Read block 915 len 28672
29163 083/300 Write block 1290 len 32768 data: 6 6 6 6 5 4 5 5
29164 084/300 Read block 3413 len 8192
29165 085/300 Write block 1137 len 16384 data: 6 5 5 5
29166 086/300 Read block 4859 len 8192
29167 087/300 Write block 29 len 12288 data: 3 7 6
29168 088/300 Write block 3006 len 4096 data: 9
29169 089/300 Read block 3991 len 20480
29170 090/300 Read block 3774 len 16384
29171 091/300 Write block 2081 len 24576 data: 9 10 12 10 8 6
29172 092/300 Write block 1331 len 28672 data: 9 8 6 6 4 4 5
29173 093/300 Read block 528 len 4096
29174 094/300 Write block 273 len 8192 data: 4 4
29175 095/300 Read block 3022 len 36864
29176 096/300 Read block 2136 len 32768
29177 097/300 Write block 1474 len 36864 data: 6 6 5 7 5 6 7 7 6
29178 098/300 Read block 13 len 32768
29179 099/300 Read block 990 len 16384
29180 100/300 Read block 1152 len 4096
29181 101/300 Write block 4125 len 32768 data: 4 6 10 11 9 10 9 8
29182 102/300 Read block 5 len 40960
29183 103/300 Read block 1264 len 20480
29184 104/300 Read block 3911 len 20480
29185 105/300 Flush
29186 106/300 Read block 2001 len 32768
29187 107/300 Flush
29188 108/300 Read block 674 len 32768
29189 109/300 Write block 1215 len 12288 data: 6 6 7
29190 110/300 Read block 935 len 32768
29191 111/300 Read block 1048 len 20480
29192 112/300 Flush
29193 113/300 Read block 2970 len 16384
29194 114/300 Read block 785 len 20480
29195 115/300 Read block 478 len 32768
29196 116/300 Flush
29197 117/300 Write block 3492 len 12288 data: 6 6 6
29198 118/300 Write block 1232 len 32768 data: 8 9 10 8 6 6 4 4
29199 119/300 Read block 2231 len 12288
29200 120/300 Write block 345 len 40960 data: 3 3 3 2 2 2 4 3 3 5
29201 121/300 Write block 3087 len 24576 data: 7 8 9 9 8 8
29202 122/300 Write block 3180 len 4096 data: 6
29203 123/300 Write block 4256 len 32768 data: 6 6 8 8 7 5 8 9
29204 124/300 Flush
29205 125/300 Write block 3811 len 32768 data: 3 4 5 6 5 6 7 6
29206 126/300 Write block 124 len 20480 data: 4 4 4 3 4
29207 127/300 Flush
29208 128/300 Write block 4544 len 12288 data: 4 4 5
29209 129/300 Read block 4704 len 8192
29210 130/300 Read block 269 len 8192
29211 131/300 Read block 71 len 8192
29212 132/300 Write block 2902 len 8192 data: 7 7
29213 133/300 Read block 1178 len 32768
29214 134/300 Read block 2477 len 24576
29215 135/300 Write block 3156 len 40960 data: 8 7 7 6 5 6 6 5 4 4
29216 136/300 Write block 4838 len 24576 data: 3 4 4 7 5 6
29217 137/300 Flush
29218 138/300 Write block 3184 len 24576 data: 7 8 9 9 8 5
29219 139/300 Flush
29220 140/300 Read block 2510 len 32768
29221 141/300 Write block 3745 len 8192 data: 3 4
29222 142/300 Write block 3126 len 12288 data: 4 5 5
29223 143/300 Read block 2907 len 16384
29224 144/300 Read block 2593 len 40960
29225 145/300 Flush
29226 146/300 Read block 4606 len 16384
29227 147/300 Read block 3111 len 8192
29228 148/300 Read block 554 len 40960
29229 149/300 Read block 4046 len 24576
29230 150/300 Write block 215 len 24576 data: 5 6 6 3 3 5
29231 151/300 Read block 2735 len 12288
29232 152/300 Write block 79 len 16384 data: 7 8 8 7
29233 153/300 Write block 4210 len 24576 data: 8 10 8 7 5 3
29234 154/300 Write block 2891 len 24576 data: 3 3 3 3 4 5
29235 155/300 Flush
29236 156/300 Flush
29237 157/300 Flush
29238 158/300 Read block 4831 len 36864
29239 159/300 Write block 3132 len 20480 data: 5 5 4 4 5
29240 160/300 Read block 2841 len 4096
29241 161/300 Read block 1039 len 12288
29242 162/300 Write block 561 len 8192 data: 3 4
29243 163/300 Read block 892 len 12288
29244 164/300 Write block 3120 len 16384 data: 4 5 5 4
29245 165/300 Read block 343 len 4096
29246 166/300 Write block 2937 len 16384 data: 9 8 7 9
29247 167/300 Read block 53 len 24576
29248 168/300 Write block 3165 len 32768 data: 5 3 3 3 2 2 2 2
29249 169/300 Write block 4503 len 24576 data: 6 6 7 8 9 9
29250 170/300 Read block 1636 len 32768
29251 171/300 Read block 3168 len 28672
29252 172/300 Read block 724 len 12288
29253 173/300 Write block 2785 len 28672 data: 10 10 9 6 4 3 3
29254 174/300 Read block 2419 len 8192
29255 175/300 Read block 2290 len 32768
29256 176/300 Read block 146 len 8192
29257 177/300 Write block 3065 len 36864 data: 6 6 6 6 5 4 4 4 4
29258 178/300 Flush
29259 179/300 Write block 2588 len 32768 data: 4 5 4 6 7 6 6 5
29260 180/300 Read block 772 len 36864
29261 181/300 Flush
29262 182/300 Read block 307 len 40960
29263 183/300 Write block 1729 len 28672 data: 6 6 5 6 5 5 4
29264 184/300 Read block 3705 len 32768
29265 185/300 Write block 1392 len 12288 data: 4 6 6
29266 186/300 Flush
29267 187/300 Read block 1018 len 12288
29268 188/300 Read block 4469 len 40960
29269 189/300 Read block 2443 len 12288
29270 190/300 Write block 2483 len 36864 data: 6 6 4 4 4 3 4 4 5
29271 191/300 Read block 3095 len 40960
29272 192/300 Write block 4781 len 32768 data: 6 6 5 5 5 6 5 6
29273 193/300 Flush
29274 194/300 Read block 3280 len 4096
29275 195/300 Read block 1506 len 20480
29276 196/300 Read block 2538 len 4096
29277 197/300 Write block 3118 len 8192 data: 4 4
29278 198/300 Read block 2557 len 28672
29279 199/300 Read block 4568 len 24576
29280 200/300 Read block 1313 len 16384
29281 201/300 Flush
29282 202/300 Write block 2259 len 24576 data: 6 5 4 4 3 3
29283 203/300 Read block 3171 len 4096
29284 204/300 Read block 1517 len 8192
29285 205/300 Read block 4648 len 24576
29286 206/300 Read block 3919 len 8192
29287 207/300 Read block 374 len 28672
29288 208/300 Write block 3908 len 28672 data: 4 5 6 6 5 4 3
29289 209/300 Write block 1015 len 32768 data: 5 4 5 4 3 3 4 5
29290 210/300 Read block 4957 len 8192
29291 211/300 Read block 4098 len 4096
29292 212/300 Write block 2632 len 32768 data: 5 4 4 3 4 4 4 4
29293 213/300 Read block 1729 len 28672
29294 214/300 Read block 790 len 16384
29295 215/300 Write block 2770 len 8192 data: 6 7
29296 216/300 Write block 2251 len 36864 data: 4 5 6 6 7 6 7 7 7
29297 217/300 Write block 1883 len 8192 data: 6 8
29298 218/300 Read block 737 len 20480
29299 219/300 Read block 1409 len 40960
29300 220/300 Write block 2014 len 40960 data: 8 5 5 6 6 6 6 3 3 3
29301 221/300 Flush
29302 222/300 Write block 1553 len 8192 data: 4 4
29303 223/300 Read block 4395 len 36864
29304 224/300 Flush
29305 225/300 Read block 2384 len 4096
29306 226/300 Write block 1832 len 40960 data: 6 6 4 3 4 4 4 5 5 5
29307 227/300 Flush
29308 228/300 Read block 3138 len 16384
29309 229/300 Read block 4205 len 40960
29310 230/300 Read block 2847 len 40960
29311 231/300 Read block 1517 len 8192
29312 232/300 Write block 2775 len 4096 data: 8
29313 233/300 Read block 3880 len 24576
29314 234/300 Write block 2436 len 32768 data: 4 4 5 8 7 7 7 6
29315 235/300 Read block 1791 len 24576
29316 236/300 Write block 1225 len 40960 data: 6 6 6 7 6 8 8 9 10 11
29317 237/300 Write block 1586 len 24576 data: 5 4 4 3 4 3
29318 238/300 Read block 4752 len 28672
29319 239/300 Write block 3197 len 4096 data: 3
29320 240/300 Write block 3591 len 12288 data: 10 10 10
29321 241/300 Read block 1557 len 16384
29322 242/300 Write block 1964 len 36864 data: 6 5 4 4 5 5 4 4 4
29323 243/300 Flush
29324 244/300 Write block 3562 len 4096 data: 9
29325 245/300 Write block 2567 len 28672 data: 5 6 7 7 5 7 6
29326 246/300 Read block 4352 len 8192
29327 247/300 Write block 564 len 20480 data: 5 5 4 4 6
29328 248/300 Read block 2317 len 24576
29329 249/300 Read block 2741 len 8192
29330 250/300 Write block 3860 len 16384 data: 2 3 4 4
29331 251/300 Read block 130 len 12288
29332 252/300 Write block 2664 len 40960 data: 3 2 3 4 3 3 3 2 3 3
29333 253/300 Write block 2736 len 24576 data: 8 7 7 7 7 8
29334 254/300 Read block 338 len 28672
29335 255/300 Read block 3974 len 24576
29336 256/300 Write block 4043 len 24576 data: 8 9 12 11 8 9
29337 257/300 Flush
29338 258/300 Read block 1739 len 36864
29339 259/300 Write block 2966 len 28672 data: 3 5 6 6 7 7 6
29340 260/300 Read block 4408 len 24576
29341 261/300 Write block 112 len 24576 data: 6 6 5 5 6 6
29342 262/300 Write block 3752 len 8192 data: 7 5
29343 263/300 Flush
29344 264/300 Read block 676 len 32768
29345 265/300 Write block 4755 len 12288 data: 3 3 3
29346 266/300 Write block 839 len 28672 data: 7 9 8 7 7 7 7
29347 267/300 Read block 942 len 8192
29348 268/300 Write block 2983 len 4096 data: 4
29349 269/300 Write block 115 len 40960 data: 6 7 7 6 6 7 6 4 4 5
29350 270/300 Write block 4285 len 20480 data: 7 7 8 7 7
29351 271/300 Read block 3634 len 12288
29352 272/300 Flush
29353 273/300 Read block 1729 len 28672
29354 274/300 Flush
29355 275/300 Write block 4374 len 40960 data: 5 4 3 3 4 2 2 3 3 3
29356 276/300 Read block 926 len 16384
29357 277/300 Write block 51 len 28672 data: 4 5 8 8 7 8 9
29358 278/300 Read block 2435 len 8192
29359 279/300 Write block 4880 len 28672 data: 8 8 5 3 3 4 4
29360 280/300 Write block 4908 len 8192 data: 6 6
29361 281/300 Read block 2767 len 32768
29362 282/300 Write block 2540 len 4096 data: 4
29363 283/300 Read block 396 len 20480
29364 284/300 Read block 1060 len 24576
29365 285/300 Write block 1302 len 32768 data: 7 8 7 6 5 5 6 8
29366 286/300 Read block 2689 len 32768
29367 287/300 Write block 1251 len 12288 data: 5 7 6
29368 288/300 Read block 1764 len 16384
29369 289/300 Write block 3400 len 20480 data: 6 5 5 7 8
29370 290/300 Read block 1733 len 36864
29371 291/300 Flush
29372 292/300 Read block 1383 len 12288
29373 293/300 Write block 130 len 36864 data: 7 7 6 7 7 8 6 7 7
29374 294/300 Read block 160 len 12288
29375 295/300 Read block 271 len 20480
29376 296/300 Read block 562 len 16384
29377 297/300 Flush
29378 298/300 Read block 2254 len 32768
29379 299/300 Read block 3972 len 12288
29380 300/300 Read block 488 len 28672
29381 301/300 Read block 4311 len 36864
29382 Main task has work to do, go find it
29383 got dsc Start(1)
29384 start 1
29385 Jan 11 06:11:37.546 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3b69686d-2b9a-4dd9-ba0c-bf378e8386d6, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
29386 Replay: started 1, returned:Ok(())
29387 ----------------------------------------------------------------
29388 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29389 States: Active Offline Active
29390 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29391 9152 Acked Write 10 Done Sent Done false
29392 9153 Acked Read 2 Done Sent Done false
29393 9154 Acked Read 3 Done Sent Done false
29394 9155 Acked Write 9 Done Sent Done false
29395 9156 Acked Read 7 Done Sent Done false
29396 9157 Acked Flush 0 Done Sent Done false
29397 9158 Acked Read 1 Done Sent Done false
29398 9159 Acked Write 6 Done Sent Done false
29399 9160 Acked Write 6 Done Sent Done false
29400 9161 Acked Read 3 Done Sent Done false
29401 9162 Acked Read 2 Done Sent Done false
29402 9163 Acked Write 1 Done Sent Done false
29403 9164 Acked Write 3 Done Sent Done false
29404 9165 Acked Read 9 Done Sent Done false
29405 9166 Acked Flush 0 Done Sent Done false
29406 9167 Acked Read 2 Done Sent Done false
29407 9168 Acked Read 6 Done Sent Done false
29408 9169 Acked Flush 0 Done Sent Done false
29409 9170 Acked Read 3 Done Sent Done false
29410 9171 Acked Write 7 Done Sent Done false
29411 9172 Acked Read 7 Done Sent Done false
29412 9173 Acked Flush 0 Done Sent Done false
29413 9174 Acked Write 5 Done Sent Done false
29414 9175 Acked Flush 0 Done Sent Done false
29415 9176 Acked Read 3 Done Sent Done false
29416 9177 Acked Read 9 Done Sent Done false
29417 9178 Acked Read 1 Done Sent Done false
29418 9179 Acked Read 2 Done Sent Done false
29419 9180 Acked Flush 0 Done Sent Done false
29420 9181 Acked Write 9 Done Sent Done false
29421 9182 Acked Read 6 Done Sent Done false
29422 9183 Acked Write 1 Done Sent Done false
29423 9184 Acked Write 1 Done Sent Done false
29424 9185 Acked Read 1 Done Sent Done false
29425 9186 Acked Write 8 Done Sent Done false
29426 9187 Acked Read 9 Done Sent Done false
29427 9188 Acked Flush 0 Done Sent Done false
29428 9189 Acked Read 7 Done Sent Done false
29429 9190 Acked Write 4 Done Sent Done false
29430 9191 Acked Read 1 Done Sent Done false
29431 9192 Acked Read 2 Done Sent Done false
29432 9193 Acked Write 4 Done Sent Done false
29433 9194 Acked Write 1 Done Sent Done false
29434 9195 Acked Write 9 Done Sent Done false
29435 9196 Acked Write 10 Done Sent Done false
29436 9197 Acked Read 9 Done Sent Done false
29437 9198 Acked Write 7 Done Sent Done false
29438 9199 Acked Read 6 Done Sent Done false
29439 9200 Acked Flush 0 Done Sent Done false
29440 9201 Acked Read 10 Done Sent Done false
29441 9202 Acked Write 6 Done Sent Done false
29442 9203 Acked Read 5 Done Sent Done false
29443 9204 Acked Write 1 Done Sent Done false
29444 9205 Acked Read 4 Done Sent Done false
29445 9206 Acked Write 5 Done Sent Done false
29446 9207 Acked Write 1 Done Sent Done false
29447 9208 Acked Write 3 Done Sent Done false
29448 9209 Acked Read 8 Done Sent Done false
29449 9210 Acked Write 1 Done Sent Done false
29450 9211 Acked Write 5 Done Sent Done false
29451 9212 Acked Write 2 Done Sent Done false
29452 9213 Acked Write 10 Done Sent Done false
29453 9214 Acked Read 9 Done Sent Done false
29454 9215 Acked Write 4 Done Sent Done false
29455 9216 Acked Read 4 Done Sent Done false
29456 9217 Acked Read 3 Done Sent Done false
29457 9218 Acked Flush 0 Done Sent Done false
29458 9219 Acked Read 10 Done Sent Done false
29459 9220 Acked Write 2 Done Sent Done false
29460 9221 Acked Flush 0 Done Sent Done false
29461 9222 Acked Read 5 Done Sent Done false
29462 9223 Acked Write 4 Done Sent Done false
29463 9224 Acked Write 5 Done Sent Done false
29464 9225 Acked Write 7 Done Sent Done false
29465 9226 Acked Flush 0 Done Sent Done false
29466 9227 Acked Read 4 Done Sent Done false
29467 9228 Acked Read 10 Done Sent Done false
29468 9229 Acked Write 3 Done Sent Done false
29469 9230 Acked Flush 0 Done Sent Done false
29470 9231 Acked Read 10 Done Sent Done false
29471 9232 Acked Read 1 Done Sent Done false
29472 9233 Acked Read 7 Done Sent Done false
29473 9234 Acked Write 8 Done Sent Done false
29474 9235 Acked Read 2 Done Sent Done false
29475 9236 Acked Write 4 Done Sent Done false
29476 9237 Acked Read 2 Done Sent Done false
29477 9238 Acked Write 3 Done Sent Done false
29478 9239 Acked Write 1 Done Sent Done false
29479 9240 Acked Read 5 Done Sent Done false
29480 9241 Acked Read 4 Done Sent Done false
29481 9242 Acked Write 6 Done Sent Done false
29482 9243 Acked Write 7 Done Sent Done false
29483 9244 Acked Read 1 Done Sent Done false
29484 9245 Acked Write 2 Done Sent Done false
29485 9246 Acked Read 9 Done Sent Done false
29486 9247 Acked Read 8 Done Sent Done false
29487 9248 Acked Write 9 Done Sent Done false
29488 9249 Acked Read 8 Done Sent Done false
29489 9250 Acked Read 4 Done Sent Done false
29490 9251 Acked Read 1 Done Sent Done false
29491 9252 Acked Write 8 Done Sent Done false
29492 9253 Acked Read 10 Done Sent Done false
29493 9254 Acked Read 5 Done Sent Done false
29494 9255 Acked Read 5 Done Sent Done false
29495 9256 Acked Flush 0 Done Sent Done false
29496 9257 Acked Read 8 Done Sent Done false
29497 9258 Acked Flush 0 Done Sent Done false
29498 9259 Acked Read 8 Done Sent Done false
29499 9260 Acked Write 3 Done Sent Done false
29500 9261 Acked Read 8 Done Sent Done false
29501 9262 Acked Read 5 Done Sent Done false
29502 9263 Acked Flush 0 Done Sent Done false
29503 9264 Acked Read 4 Done Sent Done false
29504 9265 Acked Read 5 Done Sent Done false
29505 9266 Acked Read 8 Done Sent Done false
29506 9267 Acked Flush 0 Done Sent Done false
29507 9268 Acked Write 3 Done Sent Done false
29508 9269 Acked Write 8 Done Sent Done false
29509 9270 Acked Read 3 Done Sent Done false
29510 9271 Acked Write 10 Done Sent Done false
29511 9272 Acked Write 6 Done Sent Done false
29512 9273 Acked Write 1 Done Sent Done false
29513 9274 Acked Write 8 Done Sent Done false
29514 9275 Acked Flush 0 Done Sent Done false
29515 9276 Acked Write 8 Done Sent Done false
29516 9277 Acked Write 5 Done Sent Done false
29517 9278 Acked Flush 0 Done Sent Done false
29518 9279 Acked Write 3 Done Sent Done false
29519 9280 Acked Read 2 Done Sent Done false
29520 9281 Acked Read 2 Done Sent Done false
29521 9282 Acked Read 2 Done Sent Done false
29522 9283 Acked Write 2 Done Sent Done false
29523 9284 Acked Read 8 Done Sent Done false
29524 9285 Acked Read 6 Done Sent Done false
29525 9286 Acked Write 10 Done Sent Done false
29526 9287 Acked Write 6 Done Sent Done false
29527 9288 Acked Flush 0 Done Sent Done false
29528 9289 Acked Write 6 Done Sent Done false
29529 9290 Acked Flush 0 Done Sent Done false
29530 9291 Acked Read 8 Done Sent Done false
29531 9292 Acked Write 2 Done Sent Done false
29532 9293 Acked Write 3 Done Sent Done false
29533 9294 Acked Read 4 Done Sent Done false
29534 9295 Acked Read 10 Done Sent Done false
29535 9296 Acked Flush 0 Done Sent Done false
29536 9297 Acked Read 4 Done Sent Done false
29537 9298 Acked Read 2 Done Sent Done false
29538 9299 Acked Read 10 Done Sent Done false
29539 9300 Acked Read 6 Done Sent Done false
29540 9301 Acked Write 6 Done Sent Done false
29541 9302 Acked Read 3 Done Sent Done false
29542 9303 Acked Write 4 Done Sent Done false
29543 9304 Acked Write 6 Done Sent Done false
29544 9305 Acked Write 6 Done Sent Done false
29545 9306 Acked Flush 0 Done Sent Done false
29546 9307 Acked Flush 0 Done Sent Done false
29547 9308 Acked Flush 0 Done Sent Done false
29548 9309 Acked Read 9 Done Sent Done false
29549 9310 Acked Write 5 Done Sent Done false
29550 9311 Acked Read 1 Done Sent Done false
29551 9312 Acked Read 3 Done Sent Done false
29552 9313 Acked Write 2 Done Sent Done false
29553 9314 Acked Read 3 Done Sent Done false
29554 9315 Acked Write 4 Done Sent Done false
29555 9316 Acked Read 1 Done Sent Done false
29556 9317 Acked Write 4 Done Sent Done false
29557 9318 Acked Read 6 Done Sent Done false
29558 9319 Acked Write 8 Done Sent Done false
29559 9320 Acked Write 6 Done Sent Done false
29560 9321 Acked Read 8 Done Sent Done false
29561 9322 Acked Read 7 Done Sent Done false
29562 9323 Acked Read 3 Done Sent Done false
29563 9324 Acked Write 7 Done Sent Done false
29564 9325 Acked Read 2 Done Sent Done false
29565 9326 Acked Read 8 Done Sent Done false
29566 9327 Acked Read 2 Done Sent Done false
29567 9328 Acked Write 9 Done Sent Done false
29568 9329 Acked Flush 0 Done Sent Done false
29569 9330 Acked Write 8 Done Sent Done false
29570 9331 Acked Read 9 Done Sent Done false
29571 9332 Acked Flush 0 Done Sent Done false
29572 9333 Acked Read 10 Done Sent Done false
29573 9334 Acked Write 7 Done Sent Done false
29574 9335 Acked Read 8 Done Sent Done false
29575 9336 Acked Write 3 Done Sent Done false
29576 9337 Acked Flush 0 Done Sent Done false
29577 9338 Acked Read 3 Done Sent Done false
29578 9339 Acked Read 10 Done Sent Done false
29579 9340 Acked Read 3 Done Sent Done false
29580 9341 Acked Write 9 Done Sent Done false
29581 9342 Acked Read 10 Done Sent Done false
29582 9343 Acked Write 8 Done Sent Done false
29583 9344 Acked Flush 0 Done Sent Done false
29584 9345 Acked Read 1 Done Sent Done false
29585 9346 Acked Read 5 Done Sent Done false
29586 9347 Acked Read 1 Done Sent Done false
29587 9348 Acked Write 2 Done Sent Done false
29588 9349 Acked Read 7 Done Sent Done false
29589 9350 Acked Read 6 Done Sent Done false
29590 9351 Acked Read 4 Done Sent Done false
29591 9352 Acked Flush 0 Done Sent Done false
29592 9353 Acked Write 6 Done Sent Done false
29593 9354 Acked Read 1 Done Sent Done false
29594 9355 Acked Read 2 Done Sent Done false
29595 9356 Acked Read 6 Done Sent Done false
29596 9357 Acked Read 2 Done Sent Done false
29597 9358 Acked Read 7 Done Sent Done false
29598 9359 Acked Write 7 Done Sent Done false
29599 9360 Acked Write 8 Done Sent Done false
29600 9361 Acked Read 2 Done Sent Done false
29601 9362 Acked Read 1 Done Sent Done false
29602 9363 Acked Write 8 Done Sent Done false
29603 9364 Acked Read 7 Done Sent Done false
29604 9365 Acked Read 4 Done Sent Done false
29605 9366 Acked Write 2 Done Sent Done false
29606 9367 Acked Write 9 Done Sent Done false
29607 9368 Acked Write 2 Done Sent Done false
29608 9369 Acked Read 5 Done Sent Done false
29609 9370 Acked Read 10 Done Sent Done false
29610 9371 Acked Write 10 Done Sent Done false
29611 9372 Acked Flush 0 Done Sent Done false
29612 9373 Acked Write 2 Done Sent Done false
29613 9374 Acked Read 9 Done Sent Done false
29614 9375 Acked Flush 0 Done Sent Done false
29615 9376 Acked Read 1 Done Sent Done false
29616 9377 Acked Write 10 Done Sent Done false
29617 9378 Acked Flush 0 Done Sent Done false
29618 9379 Acked Read 4 Done Sent Done false
29619 9380 Acked Read 10 Done Sent Done false
29620 9381 Acked Read 10 Done Sent Done false
29621 9382 Acked Read 2 Done Sent Done false
29622 9383 Acked Write 1 Done Sent Done false
29623 9384 Acked Read 6 Done Sent Done false
29624 9385 Acked Write 8 Done Sent Done false
29625 9386 Acked Read 6 Done Sent Done false
29626 9387 Acked Write 10 Done Sent Done false
29627 9388 Acked Write 6 Done Sent Done false
29628 9389 Acked Read 7 Done Sent Done false
29629 9390 Acked Write 1 Done Sent Done false
29630 9391 Acked Write 3 Done Sent Done false
29631 9392 Acked Read 4 Done Sent Done false
29632 9393 Acked Write 9 Done Sent Done false
29633 9394 Acked Flush 0 Done Sent Done false
29634 9395 Acked Write 1 Done Sent Done false
29635 9396 Acked Write 7 Done Sent Done false
29636 9397 Acked Read 2 Done Sent Done false
29637 9398 Acked Write 5 Done Sent Done false
29638 9399 Acked Read 6 Done Sent Done false
29639 9400 Acked Read 2 Done Sent Done false
29640 9401 Acked Write 4 Done Sent Done false
29641 9402 Acked Read 3 Done Sent Done false
29642 9403 Acked Write 10 Done Sent Done false
29643 9404 Acked Write 6 Done Sent Done false
29644 9405 Acked Read 7 Done Sent Done false
29645 9406 Acked Read 6 Done Sent Done false
29646 9407 Acked Write 6 Done Sent Done false
29647 9408 Acked Flush 0 Done Sent Done false
29648 9409 Acked Read 9 Done Sent Done false
29649 9410 Acked Write 7 Done Sent Done false
29650 9411 Acked Read 6 Done Sent Done false
29651 9412 Acked Write 6 Done Sent Done false
29652 9413 Acked Write 2 Done Sent Done false
29653 9414 Acked Flush 0 Done Sent Done false
29654 9415 Acked Read 8 Done Sent Done false
29655 9416 Acked Write 3 Done Sent Done false
29656 9417 Acked Write 7 Done Sent Done false
29657 9418 Acked Read 2 Done Sent Done false
29658 9419 Acked Write 1 Done Sent Done false
29659 9420 Acked Write 10 Done Sent Done false
29660 9421 Acked Write 5 Done Sent Done false
29661 9422 Acked Read 3 Done Sent Done false
29662 9423 Acked Flush 0 Done Sent Done false
29663 9424 Acked Read 7 Done Sent Done false
29664 9425 Acked Flush 0 Done Sent Done false
29665 9426 Acked Write 10 Done Sent Done false
29666 9427 Acked Read 4 Done Sent Done false
29667 9428 Acked Write 7 Done Sent Done false
29668 9429 Acked Read 2 Done Sent Done false
29669 9430 Acked Write 7 Done Sent Done false
29670 9431 Acked Write 2 Done Sent Done false
29671 9432 Acked Read 8 Done Sent Done false
29672 9433 Acked Write 1 Done Sent Done false
29673 9434 Acked Read 5 Done Sent Done false
29674 9435 Acked Read 6 Done Sent Done false
29675 9436 Acked Write 8 Done Sent Done false
29676 9437 Acked Read 8 Done Sent Done false
29677 9438 Acked Write 3 Done Sent Done false
29678 9439 Acked Read 4 Done Sent Done false
29679 9440 Acked Write 5 Done Sent Done false
29680 9441 Acked Read 9 Done Sent Done false
29681 9442 Acked Flush 0 Done Sent Done false
29682 9443 Acked Read 3 Done Sent Done false
29683 9444 Acked Write 9 Done Sent Done false
29684 9445 Acked Read 3 Done Sent Done false
29685 9446 Acked Read 5 Done Sent Done false
29686 9447 Acked Read 4 Done Sent Done false
29687 9448 Acked Flush 0 Done Sent Done false
29688 9449 Acked Read 8 Done Sent Done false
29689 9450 Acked Read 3 Done Sent Done false
29690 9451 Acked Read 7 Done Sent Done false
29691 9452 Acked Read 9 Done Sent Done false
29692 STATES DS:0 DS:1 DS:2 TOTAL
29693 Sent 0 301 0 301
29694 Done 301 0 301 602
29695 Skipped 0 0 0 0
29696 Error 0 0 0 0
29697 Last Flush: 9448 9151 9448
29698 Downstairs last five completed: 9151 9150 9149 9148 9147
29699 Upstairs last five completed: 9452 9451 9450 9449 9448
29700 CLIENT: Up:0 ds:301 act:2
29701 [8820] got start action
29702 [8820] I am going to restart
29703 Starting downstairs at port 8820
29704 Make output file at "/tmp/dsc/downstairs-8820.txt"
29705 [8820][1] reports Starting
29706 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1232)
29707 [8820][1] reports Running
29708 ----------------------------------------------------------------
29709 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29710 States: Active Offline Active
29711 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29712 9152 Acked Write 10 Done Sent Done false
29713 9153 Acked Read 2 Done Sent Done false
29714 9154 Acked Read 3 Done Sent Done false
29715 9155 Acked Write 9 Done Sent Done false
29716 9156 Acked Read 7 Done Sent Done false
29717 9157 Acked Flush 0 Done Sent Done false
29718 9158 Acked Read 1 Done Sent Done false
29719 9159 Acked Write 6 Done Sent Done false
29720 9160 Acked Write 6 Done Sent Done false
29721 9161 Acked Read 3 Done Sent Done false
29722 9162 Acked Read 2 Done Sent Done false
29723 9163 Acked Write 1 Done Sent Done false
29724 9164 Acked Write 3 Done Sent Done false
29725 9165 Acked Read 9 Done Sent Done false
29726 9166 Acked Flush 0 Done Sent Done false
29727 9167 Acked Read 2 Done Sent Done false
29728 9168 Acked Read 6 Done Sent Done false
29729 9169 Acked Flush 0 Done Sent Done false
29730 9170 Acked Read 3 Done Sent Done false
29731 9171 Acked Write 7 Done Sent Done false
29732 9172 Acked Read 7 Done Sent Done false
29733 9173 Acked Flush 0 Done Sent Done false
29734 9174 Acked Write 5 Done Sent Done false
29735 9175 Acked Flush 0 Done Sent Done false
29736 9176 Acked Read 3 Done Sent Done false
29737 9177 Acked Read 9 Done Sent Done false
29738 9178 Acked Read 1 Done Sent Done false
29739 9179 Acked Read 2 Done Sent Done false
29740 9180 Acked Flush 0 Done Sent Done false
29741 9181 Acked Write 9 Done Sent Done false
29742 9182 Acked Read 6 Done Sent Done false
29743 9183 Acked Write 1 Done Sent Done false
29744 9184 Acked Write 1 Done Sent Done false
29745 9185 Acked Read 1 Done Sent Done false
29746 9186 Acked Write 8 Done Sent Done false
29747 9187 Acked Read 9 Done Sent Done false
29748 9188 Acked Flush 0 Done Sent Done false
29749 9189 Acked Read 7 Done Sent Done false
29750 9190 Acked Write 4 Done Sent Done false
29751 9191 Acked Read 1 Done Sent Done false
29752 9192 Acked Read 2 Done Sent Done false
29753 9193 Acked Write 4 Done Sent Done false
29754 9194 Acked Write 1 Done Sent Done false
29755 9195 Acked Write 9 Done Sent Done false
29756 9196 Acked Write 10 Done Sent Done false
29757 9197 Acked Read 9 Done Sent Done false
29758 9198 Acked Write 7 Done Sent Done false
29759 9199 Acked Read 6 Done Sent Done false
29760 9200 Acked Flush 0 Done Sent Done false
29761 9201 Acked Read 10 Done Sent Done false
29762 9202 Acked Write 6 Done Sent Done false
29763 9203 Acked Read 5 Done Sent Done false
29764 9204 Acked Write 1 Done Sent Done false
29765 9205 Acked Read 4 Done Sent Done false
29766 9206 Acked Write 5 Done Sent Done false
29767 9207 Acked Write 1 Done Sent Done false
29768 9208 Acked Write 3 Done Sent Done false
29769 9209 Acked Read 8 Done Sent Done false
29770 9210 Acked Write 1 Done Sent Done false
29771 9211 Acked Write 5 Done Sent Done false
29772 9212 Acked Write 2 Done Sent Done false
29773 9213 Acked Write 10 Done Sent Done false
29774 9214 Acked Read 9 Done Sent Done false
29775 9215 Acked Write 4 Done Sent Done false
29776 9216 Acked Read 4 Done Sent Done false
29777 9217 Acked Read 3 Done Sent Done false
29778 9218 Acked Flush 0 Done Sent Done false
29779 9219 Acked Read 10 Done Sent Done false
29780 9220 Acked Write 2 Done Sent Done false
29781 9221 Acked Flush 0 Done Sent Done false
29782 9222 Acked Read 5 Done Sent Done false
29783 9223 Acked Write 4 Done Sent Done false
29784 9224 Acked Write 5 Done Sent Done false
29785 9225 Acked Write 7 Done Sent Done false
29786 9226 Acked Flush 0 Done Sent Done false
29787 9227 Acked Read 4 Done Sent Done false
29788 9228 Acked Read 10 Done Sent Done false
29789 9229 Acked Write 3 Done Sent Done false
29790 9230 Acked Flush 0 Done Sent Done false
29791 9231 Acked Read 10 Done Sent Done false
29792 9232 Acked Read 1 Done Sent Done false
29793 9233 Acked Read 7 Done Sent Done false
29794 9234 Acked Write 8 Done Sent Done false
29795 9235 Acked Read 2 Done Sent Done false
29796 9236 Acked Write 4 Done Sent Done false
29797 9237 Acked Read 2 Done Sent Done false
29798 9238 Acked Write 3 Done Sent Done false
29799 9239 Acked Write 1 Done Sent Done false
29800 9240 Acked Read 5 Done Sent Done false
29801 9241 Acked Read 4 Done Sent Done false
29802 9242 Acked Write 6 Done Sent Done false
29803 9243 Acked Write 7 Done Sent Done false
29804 9244 Acked Read 1 Done Sent Done false
29805 9245 Acked Write 2 Done Sent Done false
29806 9246 Acked Read 9 Done Sent Done false
29807 9247 Acked Read 8 Done Sent Done false
29808 9248 Acked Write 9 Done Sent Done false
29809 9249 Acked Read 8 Done Sent Done false
29810 9250 Acked Read 4 Done Sent Done false
29811 9251 Acked Read 1 Done Sent Done false
29812 9252 Acked Write 8 Done Sent Done false
29813 9253 Acked Read 10 Done Sent Done false
29814 9254 Acked Read 5 Done Sent Done false
29815 9255 Acked Read 5 Done Sent Done false
29816 9256 Acked Flush 0 Done Sent Done false
29817 9257 Acked Read 8 Done Sent Done false
29818 9258 Acked Flush 0 Done Sent Done false
29819 9259 Acked Read 8 Done Sent Done false
29820 9260 Acked Write 3 Done Sent Done false
29821 9261 Acked Read 8 Done Sent Done false
29822 9262 Acked Read 5 Done Sent Done false
29823 9263 Acked Flush 0 Done Sent Done false
29824 9264 Acked Read 4 Done Sent Done false
29825 9265 Acked Read 5 Done Sent Done false
29826 9266 Acked Read 8 Done Sent Done false
29827 9267 Acked Flush 0 Done Sent Done false
29828 9268 Acked Write 3 Done Sent Done false
29829 9269 Acked Write 8 Done Sent Done false
29830 9270 Acked Read 3 Done Sent Done false
29831 9271 Acked Write 10 Done Sent Done false
29832 9272 Acked Write 6 Done Sent Done false
29833 9273 Acked Write 1 Done Sent Done false
29834 9274 Acked Write 8 Done Sent Done false
29835 9275 Acked Flush 0 Done Sent Done false
29836 9276 Acked Write 8 Done Sent Done false
29837 9277 Acked Write 5 Done Sent Done false
29838 9278 Acked Flush 0 Done Sent Done false
29839 9279 Acked Write 3 Done Sent Done false
29840 9280 Acked Read 2 Done Sent Done false
29841 9281 Acked Read 2 Done Sent Done false
29842 9282 Acked Read 2 Done Sent Done false
29843 9283 Acked Write 2 Done Sent Done false
29844 9284 Acked Read 8 Done Sent Done false
29845 9285 Acked Read 6 Done Sent Done false
29846 9286 Acked Write 10 Done Sent Done false
29847 9287 Acked Write 6 Done Sent Done false
29848 9288 Acked Flush 0 Done Sent Done false
29849 9289 Acked Write 6 Done Sent Done false
29850 9290 Acked Flush 0 Done Sent Done false
29851 9291 Acked Read 8 Done Sent Done false
29852 9292 Acked Write 2 Done Sent Done false
29853 9293 Acked Write 3 Done Sent Done false
29854 9294 Acked Read 4 Done Sent Done false
29855 9295 Acked Read 10 Done Sent Done false
29856 9296 Acked Flush 0 Done Sent Done false
29857 9297 Acked Read 4 Done Sent Done false
29858 9298 Acked Read 2 Done Sent Done false
29859 9299 Acked Read 10 Done Sent Done false
29860 9300 Acked Read 6 Done Sent Done false
29861 9301 Acked Write 6 Done Sent Done false
29862 9302 Acked Read 3 Done Sent Done false
29863 9303 Acked Write 4 Done Sent Done false
29864 9304 Acked Write 6 Done Sent Done false
29865 9305 Acked Write 6 Done Sent Done false
29866 9306 Acked Flush 0 Done Sent Done false
29867 9307 Acked Flush 0 Done Sent Done false
29868 9308 Acked Flush 0 Done Sent Done false
29869 9309 Acked Read 9 Done Sent Done false
29870 9310 Acked Write 5 Done Sent Done false
29871 9311 Acked Read 1 Done Sent Done false
29872 9312 Acked Read 3 Done Sent Done false
29873 9313 Acked Write 2 Done Sent Done false
29874 9314 Acked Read 3 Done Sent Done false
29875 9315 Acked Write 4 Done Sent Done false
29876 9316 Acked Read 1 Done Sent Done false
29877 9317 Acked Write 4 Done Sent Done false
29878 9318 Acked Read 6 Done Sent Done false
29879 9319 Acked Write 8 Done Sent Done false
29880 9320 Acked Write 6 Done Sent Done false
29881 9321 Acked Read 8 Done Sent Done false
29882 9322 Acked Read 7 Done Sent Done false
29883 9323 Acked Read 3 Done Sent Done false
29884 9324 Acked Write 7 Done Sent Done false
29885 9325 Acked Read 2 Done Sent Done false
29886 9326 Acked Read 8 Done Sent Done false
29887 9327 Acked Read 2 Done Sent Done false
29888 9328 Acked Write 9 Done Sent Done false
29889 9329 Acked Flush 0 Done Sent Done false
29890 9330 Acked Write 8 Done Sent Done false
29891 9331 Acked Read 9 Done Sent Done false
29892 9332 Acked Flush 0 Done Sent Done false
29893 9333 Acked Read 10 Done Sent Done false
29894 9334 Acked Write 7 Done Sent Done false
29895 9335 Acked Read 8 Done Sent Done false
29896 9336 Acked Write 3 Done Sent Done false
29897 9337 Acked Flush 0 Done Sent Done false
29898 9338 Acked Read 3 Done Sent Done false
29899 9339 Acked Read 10 Done Sent Done false
29900 9340 Acked Read 3 Done Sent Done false
29901 9341 Acked Write 9 Done Sent Done false
29902 9342 Acked Read 10 Done Sent Done false
29903 9343 Acked Write 8 Done Sent Done false
29904 9344 Acked Flush 0 Done Sent Done false
29905 9345 Acked Read 1 Done Sent Done false
29906 9346 Acked Read 5 Done Sent Done false
29907 9347 Acked Read 1 Done Sent Done false
29908 9348 Acked Write 2 Done Sent Done false
29909 9349 Acked Read 7 Done Sent Done false
29910 9350 Acked Read 6 Done Sent Done false
29911 9351 Acked Read 4 Done Sent Done false
29912 9352 Acked Flush 0 Done Sent Done false
29913 9353 Acked Write 6 Done Sent Done false
29914 9354 Acked Read 1 Done Sent Done false
29915 9355 Acked Read 2 Done Sent Done false
29916 9356 Acked Read 6 Done Sent Done false
29917 9357 Acked Read 2 Done Sent Done false
29918 9358 Acked Read 7 Done Sent Done false
29919 9359 Acked Write 7 Done Sent Done false
29920 9360 Acked Write 8 Done Sent Done false
29921 9361 Acked Read 2 Done Sent Done false
29922 9362 Acked Read 1 Done Sent Done false
29923 9363 Acked Write 8 Done Sent Done false
29924 9364 Acked Read 7 Done Sent Done false
29925 9365 Acked Read 4 Done Sent Done false
29926 9366 Acked Write 2 Done Sent Done false
29927 9367 Acked Write 9 Done Sent Done false
29928 9368 Acked Write 2 Done Sent Done false
29929 9369 Acked Read 5 Done Sent Done false
29930 9370 Acked Read 10 Done Sent Done false
29931 9371 Acked Write 10 Done Sent Done false
29932 9372 Acked Flush 0 Done Sent Done false
29933 9373 Acked Write 2 Done Sent Done false
29934 9374 Acked Read 9 Done Sent Done false
29935 9375 Acked Flush 0 Done Sent Done false
29936 9376 Acked Read 1 Done Sent Done false
29937 9377 Acked Write 10 Done Sent Done false
29938 9378 Acked Flush 0 Done Sent Done false
29939 9379 Acked Read 4 Done Sent Done false
29940 9380 Acked Read 10 Done Sent Done false
29941 9381 Acked Read 10 Done Sent Done false
29942 9382 Acked Read 2 Done Sent Done false
29943 9383 Acked Write 1 Done Sent Done false
29944 9384 Acked Read 6 Done Sent Done false
29945 9385 Acked Write 8 Done Sent Done false
29946 9386 Acked Read 6 Done Sent Done false
29947 9387 Acked Write 10 Done Sent Done false
29948 9388 Acked Write 6 Done Sent Done false
29949 9389 Acked Read 7 Done Sent Done false
29950 9390 Acked Write 1 Done Sent Done false
29951 9391 Acked Write 3 Done Sent Done false
29952 9392 Acked Read 4 Done Sent Done false
29953 9393 Acked Write 9 Done Sent Done false
29954 9394 Acked Flush 0 Done Sent Done false
29955 9395 Acked Write 1 Done Sent Done false
29956 9396 Acked Write 7 Done Sent Done false
29957 9397 Acked Read 2 Done Sent Done false
29958 9398 Acked Write 5 Done Sent Done false
29959 9399 Acked Read 6 Done Sent Done false
29960 9400 Acked Read 2 Done Sent Done false
29961 9401 Acked Write 4 Done Sent Done false
29962 9402 Acked Read 3 Done Sent Done false
29963 9403 Acked Write 10 Done Sent Done false
29964 9404 Acked Write 6 Done Sent Done false
29965 9405 Acked Read 7 Done Sent Done false
29966 9406 Acked Read 6 Done Sent Done false
29967 9407 Acked Write 6 Done Sent Done false
29968 9408 Acked Flush 0 Done Sent Done false
29969 9409 Acked Read 9 Done Sent Done false
29970 9410 Acked Write 7 Done Sent Done false
29971 9411 Acked Read 6 Done Sent Done false
29972 9412 Acked Write 6 Done Sent Done false
29973 9413 Acked Write 2 Done Sent Done false
29974 9414 Acked Flush 0 Done Sent Done false
29975 9415 Acked Read 8 Done Sent Done false
29976 9416 Acked Write 3 Done Sent Done false
29977 9417 Acked Write 7 Done Sent Done false
29978 9418 Acked Read 2 Done Sent Done false
29979 9419 Acked Write 1 Done Sent Done false
29980 9420 Acked Write 10 Done Sent Done false
29981 9421 Acked Write 5 Done Sent Done false
29982 9422 Acked Read 3 Done Sent Done false
29983 9423 Acked Flush 0 Done Sent Done false
29984 9424 Acked Read 7 Done Sent Done false
29985 9425 Acked Flush 0 Done Sent Done false
29986 9426 Acked Write 10 Done Sent Done false
29987 9427 Acked Read 4 Done Sent Done false
29988 9428 Acked Write 7 Done Sent Done false
29989 9429 Acked Read 2 Done Sent Done false
29990 9430 Acked Write 7 Done Sent Done false
29991 9431 Acked Write 2 Done Sent Done false
29992 9432 Acked Read 8 Done Sent Done false
29993 9433 Acked Write 1 Done Sent Done false
29994 9434 Acked Read 5 Done Sent Done false
29995 9435 Acked Read 6 Done Sent Done false
29996 9436 Acked Write 8 Done Sent Done false
29997 9437 Acked Read 8 Done Sent Done false
29998 9438 Acked Write 3 Done Sent Done false
29999 9439 Acked Read 4 Done Sent Done false
30000 9440 Acked Write 5 Done Sent Done false
30001 9441 Acked Read 9 Done Sent Done false
30002 9442 Acked Flush 0 Done Sent Done false
30003 9443 Acked Read 3 Done Sent Done false
30004 9444 Acked Write 9 Done Sent Done false
30005 9445 Acked Read 3 Done Sent Done false
30006 9446 Acked Read 5 Done Sent Done false
30007 9447 Acked Read 4 Done Sent Done false
30008 9448 Acked Flush 0 Done Sent Done false
30009 9449 Acked Read 8 Done Sent Done false
30010 9450 Acked Read 3 Done Sent Done false
30011 9451 Acked Read 7 Done Sent Done false
30012 9452 Acked Read 9 Done Sent Done false
30013 9453 Acked Flush 0 Done Sent Done false
30014 STATES DS:0 DS:1 DS:2 TOTAL
30015 Sent 0 302 0 302
30016 Done 302 0 302 604
30017 Skipped 0 0 0 0
30018 Error 0 0 0 0
30019 Last Flush: 9453 9151 9453
30020 Downstairs last five completed: 9151 9150 9149 9148 9147
30021 Upstairs last five completed: 9453 9452 9451 9450 9449
30022 CLIENT: Up:0 ds:302 act:2
300232025-01-11T06:11:43.308ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300242025-01-11T06:11:43.308ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50986) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300252025-01-11T06:11:43.310ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300262025-01-11T06:11:43.310ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300272025-01-11T06:11:43.310ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300282025-01-11T06:11:43.310ZINFOcrucible: send last flush ID to this DS: 9151 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300292025-01-11T06:11:43.310ZINFOcrucible: Replied this last flush ID: 9151 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300302025-01-11T06:11:43.310ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300312025-01-11T06:11:43.310ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300322025-01-11T06:11:43.310ZINFOcrucible: [1] client re-new 302 jobs since flush 9151 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300332025-01-11T06:11:43.311ZINFOcrucible: [1] Replayed 302 jobs since flush: 9151 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
30034 ----------------------------------------------------------------
30035 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30036 Downstairs last five completed: 9453 9452 9451 9450 9449
30037 Upstairs last five completed: 9453 9452 9451 9450 9449
30038 CLIENT: Up:0 ds:0 act:3
30039 Replay: All jobs finished
30040 Main task has work to do, go find it
30041 got dsc Stop(0)
30042 stop 0
30043 [8810] Got stop action so:false kr:false
30044 Jan 11 06:11:45.552 INFO request completed, latency_us: 385, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 5c752ef0-0bf1-4533-9300-f53d8191fec0, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
300452025-01-11T06:11:45.552ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
30046 Jan 11 06:11:45.552{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:11:45.55290576Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"0","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
30047 INFO request completed, latency_us: 167, response_code: 200, uri: /state/cid/0, method: GET{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:11:45.552965832Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
30048 , req_id: b96f2af3-1aa6-454a-885f-e767f18e892b, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
300492025-01-11T06:11:45.552ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300502025-01-11T06:11:45.553ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
300512025-01-11T06:11:45.553ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
30052 [8810] Exited with: ExitStatus(unix_wait_status(9))
30053 [8810][0] reports Exit
30054 Jan 11 06:11:49.555 INFO request completed, latency_us: 265, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5c1b1868-7397-4ad9-afea-1c2a014106fa, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
30055 001/300 Write block 504 len 8192 data: 8 4
30056 002/300 Write block 1933 len 40960 data: 7 5 6 6 8 7 6 8 8 6
30057 003/300 Write block 2463 len 12288 data: 6 7 4
30058 004/300 Write block 3336 len 16384 data: 4 6 5 4
30059 005/300 Read block 1295 len 40960
30060 006/300 Flush
30061 007/300 Flush
30062 008/300 Write block 1983 len 16384 data: 5 6 5 3
30063 009/300 Read block 4496 len 20480
30064 010/300 Read block 3170 len 24576
30065 011/300 Write block 3245 len 8192 data: 6 5
30066 012/300 Write block 4856 len 36864 data: 4 4 5 5 5 6 4 3 3
30067 013/300 Read block 619 len 32768
30068 014/300 Read block 3813 len 24576
30069 015/300 Write block 4533 len 12288 data: 6 5 5
30070 016/300 Write block 1893 len 8192 data: 5 6
30071 017/300 Read block 3133 len 36864
30072 018/300 Read block 3616 len 24576
30073 019/300 Read block 417 len 24576
30074 020/300 Write block 4063 len 32768 data: 5 4 4 5 6 6 7 9
30075 021/300 Write block 2817 len 20480 data: 6 8 7 7 6
30076 022/300 Read block 3634 len 12288
30077 023/300 Write block 4095 len 16384 data: 3 2 2 3
30078 024/300 Read block 2883 len 4096
30079 025/300 Read block 1563 len 8192
30080 026/300 Read block 1755 len 4096
30081 027/300 Flush
30082 028/300 Read block 4795 len 28672
30083 029/300 Flush
30084 030/300 Write block 774 len 12288 data: 4 4 4
30085 031/300 Read block 2741 len 24576
30086 032/300 Read block 14 len 20480
30087 033/300 Read block 3133 len 32768
30088 034/300 Read block 4862 len 28672
30089 035/300 Write block 4716 len 28672 data: 7 8 7 5 4 4 5
30090 036/300 Read block 4381 len 40960
30091 037/300 Read block 4853 len 8192
30092 038/300 Read block 2456 len 16384
30093 039/300 Read block 3871 len 24576
30094 040/300 Read block 2995 len 12288
30095 041/300 Read block 622 len 12288
30096 042/300 Read block 1435 len 20480
30097 043/300 Read block 1460 len 8192
30098 044/300 Write block 3808 len 4096 data: 3
30099 045/300 Read block 4797 len 36864
30100 046/300 Read block 1305 len 32768
30101 047/300 Write block 1248 len 8192 data: 5 5
30102 048/300 Flush
30103 049/300 Flush
30104 050/300 Flush
30105 051/300 Write block 960 len 28672 data: 6 7 6 6 6 8 6
30106 052/300 Write block 1826 len 28672 data: 5 8 8 10 10 7 7
30107 053/300 Read block 4782 len 12288
30108 054/300 Read block 556 len 8192
30109 055/300 Write block 2630 len 32768 data: 8 5 6 5 5 4 5 5
30110 056/300 Read block 1835 len 28672
30111 057/300 Read block 2709 len 32768
30112 058/300 Write block 2830 len 36864 data: 5 6 6 6 6 6 5 4 3
30113 059/300 Read block 3115 len 32768
30114 060/300 Write block 2894 len 32768 data: 4 5 6 5 6 6 7 7
30115 061/300 Read block 2687 len 12288
30116 062/300 Read block 2665 len 24576
30117 063/300 Flush
30118 064/300 Read block 789 len 24576
30119 065/300 Read block 2210 len 8192
30120 066/300 Read block 1183 len 20480
30121 067/300 Write block 3743 len 24576 data: 2 3 4 5 4 6
30122 068/300 Flush
30123 069/300 Write block 3028 len 36864 data: 6 5 8 8 8 9 8 7 7
30124 070/300 Read block 4096 len 4096
30125 071/300 Write block 1000 len 20480 data: 4 7 6 6 6
30126 072/300 Write block 3586 len 12288 data: 7 5 8
30127 073/300 Read block 4942 len 12288
30128 074/300 Write block 2068 len 8192 data: 5 5
30129 075/300 Read block 4042 len 16384
30130 076/300 Read block 4801 len 4096
30131 077/300 Read block 2805 len 24576
30132 078/300 Write block 4281 len 24576 data: 4 4 5 5 8 8
30133 079/300 Flush
30134 080/300 Write block 65 len 28672 data: 7 4 4 5 5 5 6
30135 081/300 Read block 1257 len 12288
30136 082/300 Write block 2841 len 36864 data: 3 3 3 2 2 2 4 3 4
30137 083/300 Read block 3894 len 4096
30138 084/300 Read block 3831 len 8192
30139 085/300 Write block 252 len 20480 data: 6 4 4 5 5
30140 086/300 Read block 866 len 40960
30141 087/300 Write block 3855 len 8192 data: 5 4
30142 088/300 Write block 321 len 36864 data: 8 8 10 11 11 9 7 7 7
30143 089/300 Read block 2092 len 36864
30144 090/300 Read block 2844 len 28672
30145 091/300 Read block 4338 len 40960
30146 092/300 Read block 1510 len 4096
30147 093/300 Read block 4773 len 32768
30148 094/300 Write block 3675 len 16384 data: 7 7 7 4
30149 095/300 Read block 2519 len 8192
30150 096/300 Read block 1875 len 28672
30151 097/300 Write block 4325 len 28672 data: 5 5 5 3 3 3 3
30152 098/300 Read block 3373 len 24576
30153 099/300 Write block 2288 len 36864 data: 6 6 6 7 6 5 6 7 7
30154 100/300 Read block 883 len 36864
30155 101/300 Read block 4453 len 40960
30156 102/300 Flush
30157 103/300 Flush
30158 104/300 Read block 2757 len 12288
30159 105/300 Read block 4529 len 24576
30160 106/300 Read block 3283 len 8192
30161 107/300 Flush
30162 108/300 Read block 3292 len 36864
30163 109/300 Read block 3518 len 16384
30164 110/300 Read block 553 len 40960
30165 111/300 Write block 3809 len 12288 data: 2 3 4
30166 112/300 Write block 92 len 28672 data: 6 7 10 11 9 10 10
30167 113/300 Write block 2079 len 16384 data: 8 7 10 11
30168 114/300 Read block 1973 len 28672
30169 115/300 Write block 3501 len 32768 data: 8 9 8 7 8 9 9 7
30170 116/300 Read block 2927 len 32768
30171 117/300 Write block 2157 len 24576 data: 5 4 3 3 3 3
30172 118/300 Write block 1279 len 32768 data: 6 6 6 5 4 4 3 4
30173 119/300 Write block 1899 len 20480 data: 6 6 6 5 5
30174 120/300 Read block 1345 len 36864
30175 121/300 Write block 4692 len 4096 data: 7
30176 122/300 Read block 3159 len 16384
30177 123/300 Read block 773 len 12288
30178 124/300 Write block 369 len 8192 data: 4 4
30179 125/300 Write block 2473 len 32768 data: 5 5 5 5 4 7 9 9
30180 126/300 Read block 1857 len 16384
30181 127/300 Read block 1785 len 4096
30182 128/300 Read block 605 len 36864
30183 129/300 Read block 1586 len 8192
30184 130/300 Write block 2456 len 36864 data: 7 7 7 6 5 5 5 7 8
30185 131/300 Write block 2121 len 24576 data: 3 4 6 6 5 6
30186 132/300 Read block 4127 len 4096
30187 133/300 Read block 2293 len 32768
30188 134/300 Write block 865 len 8192 data: 4 3
30189 135/300 Read block 1412 len 8192
30190 136/300 Read block 1512 len 40960
30191 137/300 Read block 4555 len 20480
30192 138/300 Read block 1383 len 28672
30193 139/300 Read block 567 len 8192
30194 140/300 Read block 752 len 16384
30195 141/300 Write block 2216 len 36864 data: 7 7 7 8 9 8 8 7 6
30196 142/300 Flush
30197 143/300 Write block 838 len 12288 data: 8 8 10
30198 144/300 Write block 3509 len 32768 data: 4 5 5 6 8 10 8 8
30199 145/300 Read block 565 len 36864
30200 146/300 Flush
30201 147/300 Write block 1220 len 24576 data: 8 5 6 5 5 7
30202 148/300 Read block 4786 len 40960
30203 149/300 Read block 4014 len 20480
30204 150/300 Read block 3588 len 8192
30205 151/300 Read block 1824 len 24576
30206 152/300 Read block 4438 len 20480
30207 153/300 Read block 1441 len 32768
30208 154/300 Write block 4454 len 20480 data: 10 12 9 5 4
30209 155/300 Write block 2044 len 28672 data: 6 6 7 5 5 5 7
30210 156/300 Write block 225 len 8192 data: 7 8
30211 157/300 Read block 1183 len 32768
30212 158/300 Flush
30213 159/300 Write block 2749 len 4096 data: 6
30214 160/300 Read block 4333 len 28672
30215 161/300 Flush
30216 162/300 Read block 416 len 8192
30217 163/300 Read block 165 len 16384
30218 164/300 Write block 4893 len 16384 data: 9 8 8 7
30219 165/300 Write block 2682 len 40960 data: 9 8 6 6 7 5 7 6 6 7
30220 166/300 Write block 1618 len 8192 data: 8 8
30221 167/300 Read block 728 len 20480
30222 168/300 Read block 1142 len 24576
30223 169/300 Read block 1441 len 24576
30224 170/300 Read block 3362 len 32768
30225 171/300 Write block 3578 len 40960 data: 6 6 7 7 6 7 6 7 8 6
30226 172/300 Write block 4874 len 16384 data: 7 7 8 8
30227 173/300 Read block 292 len 36864
30228 174/300 Write block 204 len 32768 data: 5 5 5 3 4 5 5 7
30229 175/300 Read block 3486 len 40960
30230 176/300 Write block 2076 len 8192 data: 6 6
30231 177/300 Read block 2102 len 32768
30232 178/300 Write block 4965 len 28672 data: 6 7 7 7 4 4 5
30233 179/300 Write block 3530 len 16384 data: 7 7 7 8
30234 180/300 Flush
30235 181/300 Flush
30236 182/300 Read block 447 len 8192
30237 183/300 Write block 2274 len 8192 data: 7 4
30238 184/300 Read block 2063 len 12288
30239 185/300 Read block 1059 len 20480
30240 186/300 Read block 852 len 40960
30241 187/300 Write block 3677 len 4096 data: 8
30242 188/300 Read block 1934 len 36864
30243 189/300 Flush
30244 190/300 Read block 2059 len 12288
30245 191/300 Read block 658 len 20480
30246 192/300 Write block 3131 len 8192 data: 4 6
30247 193/300 Read block 4655 len 4096
30248 194/300 Read block 3334 len 28672
30249 195/300 Read block 779 len 36864
30250 196/300 Read block 209 len 24576
30251 197/300 Read block 1760 len 20480
30252 198/300 Write block 3818 len 4096 data: 7
30253 199/300 Write block 4804 len 20480 data: 5 6 5 4 4
30254 200/300 Read block 4271 len 32768
30255 201/300 Write block 4228 len 32768 data: 5 3 4 3 3 4 5 5
30256 202/300 Write block 3283 len 12288 data: 8 7 7
30257 203/300 Read block 1155 len 12288
30258 204/300 Read block 3048 len 24576
30259 205/300 Write block 2300 len 24576 data: 9 11 10 8 7 7
30260 206/300 Read block 3996 len 4096
30261 207/300 Write block 2267 len 28672 data: 4 6 6 5 8 7 7
30262 208/300 Write block 3325 len 12288 data: 5 4 4
30263 209/300 Read block 231 len 4096
30264 210/300 Read block 822 len 12288
30265 211/300 Write block 4606 len 8192 data: 5 8
30266 212/300 Flush
30267 213/300 Write block 920 len 16384 data: 4 5 5 5
30268 214/300 Flush
30269 215/300 Read block 1652 len 16384
30270 216/300 Read block 2206 len 24576
30271 217/300 Write block 2861 len 16384 data: 8 11 12 10
30272 218/300 Read block 3116 len 12288
30273 219/300 Flush
30274 220/300 Read block 554 len 4096
30275 221/300 Read block 4715 len 36864
30276 222/300 Write block 4944 len 4096 data: 10
30277 223/300 Read block 3768 len 20480
30278 224/300 Flush
30279 225/300 Read block 2276 len 16384
30280 226/300 Flush
30281 227/300 Write block 2798 len 12288 data: 7 7 6
30282 228/300 Write block 2143 len 20480 data: 5 6 5 7 5
30283 229/300 Read block 3216 len 28672
30284 230/300 Write block 118 len 36864 data: 7 7 8 7 5 5 6 5 5
30285 231/300 Flush
30286 232/300 Read block 3728 len 24576
30287 233/300 Write block 3569 len 40960 data: 8 8 8 8 7 6 7 9 7 7
30288 234/300 Flush
30289 235/300 Read block 2784 len 36864
30290 236/300 Read block 3172 len 32768
30291 237/300 Write block 3876 len 8192 data: 5 4
30292 238/300 Write block 3355 len 24576 data: 4 3 3 3 3 3
30293 239/300 Write block 677 len 28672 data: 3 2 3 3 3 2 3
30294 240/300 Flush
30295 241/300 Write block 3934 len 8192 data: 4 4
30296 242/300 Write block 4359 len 8192 data: 10 9
30297 243/300 Write block 3393 len 24576 data: 2 2 2 2 2 3
30298 244/300 Write block 4575 len 16384 data: 6 7 7 7
30299 245/300 Write block 1755 len 24576 data: 9 8 8 8 6 6
30300 246/300 Flush
30301 247/300 Write block 3430 len 20480 data: 8 8 9 8 8
30302 248/300 Read block 1480 len 40960
30303 249/300 Read block 481 len 8192
30304 250/300 Read block 2731 len 12288
30305 251/300 Write block 3179 len 4096 data: 5
30306 252/300 Read block 3978 len 24576
30307 253/300 Read block 4983 len 40960
30308 254/300 Write block 1397 len 28672 data: 5 6 7 6 6 6 6
30309 255/300 Read block 1635 len 8192
30310 256/300 Flush
30311 257/300 Read block 2944 len 40960
30312 258/300 Write block 1714 len 36864 data: 5 6 6 5 6 7 8 7 8
30313 259/300 Read block 95 len 4096
30314 260/300 Read block 3987 len 40960
30315 261/300 Write block 2857 len 4096 data: 6
30316 262/300 Write block 1209 len 4096 data: 6
30317 263/300 Read block 4060 len 12288
30318 264/300 Write block 2286 len 36864 data: 7 6 7 7 7 8 7 6 7
30319 265/300 Write block 2197 len 20480 data: 10 9 9 8 7
30320 266/300 Read block 3136 len 36864
30321 267/300 Read block 188 len 40960
30322 268/300 Read block 1501 len 28672
30323 269/300 Read block 2058 len 36864
30324 270/300 Read block 4931 len 8192
30325 271/300 Write block 2219 len 28672 data: 9 10 9 9 8 7 4
30326 272/300 Read block 2797 len 16384
30327 273/300 Read block 4429 len 32768
30328 274/300 Read block 2833 len 8192
30329 275/300 Read block 2617 len 12288
30330 276/300 Write block 1581 len 40960 data: 6 7 7 5 5 6 5 5 4 5
30331 277/300 Write block 3419 len 12288 data: 4 4 3
30332 278/300 Write block 2275 len 8192 data: 5 3
30333 279/300 Write block 1251 len 40960 data: 6 8 7 7 7 6 7 6 6 4
30334 280/300 Read block 1020 len 40960
30335 281/300 Write block 2321 len 16384 data: 4 4 4 5
30336 282/300 Write block 1624 len 20480 data: 6 5 6 6 5
30337 283/300 Read block 290 len 12288
30338 284/300 Read block 3474 len 20480
30339 285/300 Read block 1672 len 32768
30340 286/300 Write block 684 len 32768 data: 3 4 7 8 9 11 12 11
30341 287/300 Read block 4645 len 24576
30342 288/300 Read block 17 len 24576
30343 289/300 Read block 3681 len 16384
30344 290/300 Write block 3783 len 8192 data: 8 7
30345 291/300 Flush
30346 292/300 Read block 3841 len 16384
30347 293/300 Write block 3377 len 24576 data: 6 7 7 7 8 8
30348 294/300 Read block 338 len 36864
30349 295/300 Read block 442 len 36864
30350 296/300 Read block 4977 len 28672
30351 297/300 Read block 3519 len 24576
30352 298/300 Write block 2319 len 16384 data: 5 4 5 5
30353 299/300 Write block 1789 len 16384 data: 3 4 5 6
30354 300/300 Write block 3267 len 28672 data: 4 5 7 8 8 11 10
30355 301/300 Write block 3625 len 36864 data: 7 6 5 5 4 5 7 7 6
30356 Main task has work to do, go find it
30357 got dsc Start(0)
30358 start 0
30359 Jan 11 06:11:49.799 INFO request completed, latency_us: 241, response_code: 204, uri: /start/cid/0, method: POST, req_id: 188973f8-1a44-4a81-bd5b-5d1838583a15, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
30360 Replay: started 0, returned:Ok(())
30361 ----------------------------------------------------------------
30362 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30363 States: Offline Active Active
30364 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30365 9454 Acked Write 2 Sent Done Done false
30366 9455 Acked Write 10 Sent Done Done false
30367 9456 Acked Write 3 Sent Done Done false
30368 9457 Acked Write 4 Sent Done Done false
30369 9458 Acked Read 10 Sent Done Done false
30370 9459 Acked Flush 0 Sent Done Done false
30371 9460 Acked Flush 0 Sent Done Done false
30372 9461 Acked Write 4 Sent Done Done false
30373 9462 Acked Read 5 Sent Done Done false
30374 9463 Acked Read 6 Sent Done Done false
30375 9464 Acked Write 2 Sent Done Done false
30376 9465 Acked Write 9 Sent Done Done false
30377 9466 Acked Read 8 Sent Done Done false
30378 9467 Acked Read 6 Sent Done Done false
30379 9468 Acked Write 3 Sent Done Done false
30380 9469 Acked Write 2 Sent Done Done false
30381 9470 Acked Read 9 Sent Done Done false
30382 9471 Acked Read 6 Sent Done Done false
30383 9472 Acked Read 6 Sent Done Done false
30384 9473 Acked Write 8 Sent Done Done false
30385 9474 Acked Write 5 Sent Done Done false
30386 9475 Acked Read 3 Sent Done Done false
30387 9476 Acked Write 4 Sent Done Done false
30388 9477 Acked Read 1 Sent Done Done false
30389 9478 Acked Read 2 Sent Done Done false
30390 9479 Acked Read 1 Sent Done Done false
30391 9480 Acked Flush 0 Sent Done Done false
30392 9481 Acked Read 7 Sent Done Done false
30393 9482 Acked Flush 0 Sent Done Done false
30394 9483 Acked Write 3 Sent Done Done false
30395 9484 Acked Read 6 Sent Done Done false
30396 9485 Acked Read 5 Sent Done Done false
30397 9486 Acked Read 8 Sent Done Done false
30398 9487 Acked Read 7 Sent Done Done false
30399 9488 Acked Write 7 Sent Done Done false
30400 9489 Acked Read 10 Sent Done Done false
30401 9490 Acked Read 2 Sent Done Done false
30402 9491 Acked Read 4 Sent Done Done false
30403 9492 Acked Read 6 Sent Done Done false
30404 9493 Acked Read 3 Sent Done Done false
30405 9494 Acked Read 3 Sent Done Done false
30406 9495 Acked Read 5 Sent Done Done false
30407 9496 Acked Read 2 Sent Done Done false
30408 9497 Acked Write 1 Sent Done Done false
30409 9498 Acked Read 9 Sent Done Done false
30410 9499 Acked Read 8 Sent Done Done false
30411 9500 Acked Write 2 Sent Done Done false
30412 9501 Acked Flush 0 Sent Done Done false
30413 9502 Acked Flush 0 Sent Done Done false
30414 9503 Acked Flush 0 Sent Done Done false
30415 9504 Acked Write 7 Sent Done Done false
30416 9505 Acked Write 7 Sent Done Done false
30417 9506 Acked Read 3 Sent Done Done false
30418 9507 Acked Read 2 Sent Done Done false
30419 9508 Acked Write 8 Sent Done Done false
30420 9509 Acked Read 7 Sent Done Done false
30421 9510 Acked Read 8 Sent Done Done false
30422 9511 Acked Write 9 Sent Done Done false
30423 9512 Acked Read 8 Sent Done Done false
30424 9513 Acked Write 8 Sent Done Done false
30425 9514 Acked Read 3 Sent Done Done false
30426 9515 Acked Read 6 Sent Done Done false
30427 9516 Acked Flush 0 Sent Done Done false
30428 9517 Acked Read 6 Sent Done Done false
30429 9518 Acked Read 2 Sent Done Done false
30430 9519 Acked Read 5 Sent Done Done false
30431 9520 Acked Write 6 Sent Done Done false
30432 9521 Acked Flush 0 Sent Done Done false
30433 9522 Acked Write 9 Sent Done Done false
30434 9523 Acked Read 1 Sent Done Done false
30435 9524 Acked Write 5 Sent Done Done false
30436 9525 Acked Write 3 Sent Done Done false
30437 9526 Acked Read 3 Sent Done Done false
30438 9527 Acked Write 2 Sent Done Done false
30439 9528 Acked Read 4 Sent Done Done false
30440 9529 Acked Read 1 Sent Done Done false
30441 9530 Acked Read 6 Sent Done Done false
30442 9531 Acked Write 6 Sent Done Done false
30443 9532 Acked Flush 0 Sent Done Done false
30444 9533 Acked Write 7 Sent Done Done false
30445 9534 Acked Read 3 Sent Done Done false
30446 9535 Acked Write 9 Sent Done Done false
30447 9536 Acked Read 1 Sent Done Done false
30448 9537 Acked Read 2 Sent Done Done false
30449 9538 Acked Write 5 Sent Done Done false
30450 9539 Acked Read 10 Sent Done Done false
30451 9540 Acked Write 2 Sent Done Done false
30452 9541 Acked Write 9 Sent Done Done false
30453 9542 Acked Read 9 Sent Done Done false
30454 9543 Acked Read 7 Sent Done Done false
30455 9544 Acked Read 10 Sent Done Done false
30456 9545 Acked Read 1 Sent Done Done false
30457 9546 Acked Read 8 Sent Done Done false
30458 9547 Acked Write 4 Sent Done Done false
30459 9548 Acked Read 2 Sent Done Done false
30460 9549 Acked Read 7 Sent Done Done false
30461 9550 Acked Write 7 Sent Done Done false
30462 9551 Acked Read 6 Sent Done Done false
30463 9552 Acked Write 9 Sent Done Done false
30464 9553 Acked Read 9 Sent Done Done false
30465 9554 Acked Read 10 Sent Done Done false
30466 9555 Acked Flush 0 Sent Done Done false
30467 9556 Acked Flush 0 Sent Done Done false
30468 9557 Acked Read 3 Sent Done Done false
30469 9558 Acked Read 6 Sent Done Done false
30470 9559 Acked Read 2 Sent Done Done false
30471 9560 Acked Flush 0 Sent Done Done false
30472 9561 Acked Read 9 Sent Done Done false
30473 9562 Acked Read 4 Sent Done Done false
30474 9563 Acked Read 10 Sent Done Done false
30475 9564 Acked Write 3 Sent Done Done false
30476 9565 Acked Write 7 Sent Done Done false
30477 9566 Acked Write 4 Sent Done Done false
30478 9567 Acked Read 7 Sent Done Done false
30479 9568 Acked Write 8 Sent Done Done false
30480 9569 Acked Read 8 Sent Done Done false
30481 9570 Acked Write 6 Sent Done Done false
30482 9571 Acked Write 8 Sent Done Done false
30483 9572 Acked Write 5 Sent Done Done false
30484 9573 Acked Read 9 Sent Done Done false
30485 9574 Acked Write 1 Sent Done Done false
30486 9575 Acked Read 4 Sent Done Done false
30487 9576 Acked Read 3 Sent Done Done false
30488 9577 Acked Write 2 Sent Done Done false
30489 9578 Acked Write 8 Sent Done Done false
30490 9579 Acked Read 4 Sent Done Done false
30491 9580 Acked Read 1 Sent Done Done false
30492 9581 Acked Read 9 Sent Done Done false
30493 9582 Acked Read 2 Sent Done Done false
30494 9583 Acked Write 9 Sent Done Done false
30495 9584 Acked Write 6 Sent Done Done false
30496 9585 Acked Read 1 Sent Done Done false
30497 9586 Acked Read 8 Sent Done Done false
30498 9587 Acked Write 2 Sent Done Done false
30499 9588 Acked Read 2 Sent Done Done false
30500 9589 Acked Read 10 Sent Done Done false
30501 9590 Acked Read 5 Sent Done Done false
30502 9591 Acked Read 7 Sent Done Done false
30503 9592 Acked Read 2 Sent Done Done false
30504 9593 Acked Read 4 Sent Done Done false
30505 9594 Acked Write 9 Sent Done Done false
30506 9595 Acked Flush 0 Sent Done Done false
30507 9596 Acked Write 3 Sent Done Done false
30508 9597 Acked Write 8 Sent Done Done false
30509 9598 Acked Read 9 Sent Done Done false
30510 9599 Acked Flush 0 Sent Done Done false
30511 9600 Acked Write 6 Sent Done Done false
30512 9601 Acked Read 10 Sent Done Done false
30513 9602 Acked Read 5 Sent Done Done false
30514 9603 Acked Read 2 Sent Done Done false
30515 9604 Acked Read 6 Sent Done Done false
30516 9605 Acked Read 5 Sent Done Done false
30517 9606 Acked Read 8 Sent Done Done false
30518 9607 Acked Write 5 Sent Done Done false
30519 9608 Acked Write 7 Sent Done Done false
30520 9609 Acked Write 2 Sent Done Done false
30521 9610 Acked Read 8 Sent Done Done false
30522 9611 Acked Flush 0 Sent Done Done false
30523 9612 Acked Write 1 Sent Done Done false
30524 9613 Acked Read 7 Sent Done Done false
30525 9614 Acked Flush 0 Sent Done Done false
30526 9615 Acked Read 2 Sent Done Done false
30527 9616 Acked Read 4 Sent Done Done false
30528 9617 Acked Write 4 Sent Done Done false
30529 9618 Acked Write 10 Sent Done Done false
30530 9619 Acked Write 2 Sent Done Done false
30531 9620 Acked Read 5 Sent Done Done false
30532 9621 Acked Read 6 Sent Done Done false
30533 9622 Acked Read 6 Sent Done Done false
30534 9623 Acked Read 8 Sent Done Done false
30535 9624 Acked Write 10 Sent Done Done false
30536 9625 Acked Write 4 Sent Done Done false
30537 9626 Acked Read 9 Sent Done Done false
30538 9627 Acked Write 8 Sent Done Done false
30539 9628 Acked Read 10 Sent Done Done false
30540 9629 Acked Write 2 Sent Done Done false
30541 9630 Acked Read 8 Sent Done Done false
30542 9631 Acked Write 7 Sent Done Done false
30543 9632 Acked Write 4 Sent Done Done false
30544 9633 Acked Flush 0 Sent Done Done false
30545 9634 Acked Flush 0 Sent Done Done false
30546 9635 Acked Read 2 Sent Done Done false
30547 9636 Acked Write 2 Sent Done Done false
30548 9637 Acked Read 3 Sent Done Done false
30549 9638 Acked Read 5 Sent Done Done false
30550 9639 Acked Read 10 Sent Done Done false
30551 9640 Acked Write 1 Sent Done Done false
30552 9641 Acked Read 9 Sent Done Done false
30553 9642 Acked Flush 0 Sent Done Done false
30554 9643 Acked Read 3 Sent Done Done false
30555 9644 Acked Read 5 Sent Done Done false
30556 9645 Acked Write 2 Sent Done Done false
30557 9646 Acked Read 1 Sent Done Done false
30558 9647 Acked Read 7 Sent Done Done false
30559 9648 Acked Read 9 Sent Done Done false
30560 9649 Acked Read 6 Sent Done Done false
30561 9650 Acked Read 5 Sent Done Done false
30562 9651 Acked Write 1 Sent Done Done false
30563 9652 Acked Write 5 Sent Done Done false
30564 9653 Acked Read 8 Sent Done Done false
30565 9654 Acked Write 8 Sent Done Done false
30566 9655 Acked Write 3 Sent Done Done false
30567 9656 Acked Read 3 Sent Done Done false
30568 9657 Acked Read 6 Sent Done Done false
30569 9658 Acked Write 6 Sent Done Done false
30570 9659 Acked Read 1 Sent Done Done false
30571 9660 Acked Write 7 Sent Done Done false
30572 9661 Acked Write 3 Sent Done Done false
30573 9662 Acked Read 1 Sent Done Done false
30574 9663 Acked Read 3 Sent Done Done false
30575 9664 Acked Write 2 Sent Done Done false
30576 9665 Acked Flush 0 Sent Done Done false
30577 9666 Acked Write 4 Sent Done Done false
30578 9667 Acked Flush 0 Sent Done Done false
30579 9668 Acked Read 4 Sent Done Done false
30580 9669 Acked Read 6 Sent Done Done false
30581 9670 Acked Write 4 Sent Done Done false
30582 9671 Acked Read 3 Sent Done Done false
30583 9672 Acked Flush 0 Sent Done Done false
30584 9673 Acked Read 1 Sent Done Done false
30585 9674 Acked Read 9 Sent Done Done false
30586 9675 Acked Write 1 Sent Done Done false
30587 9676 Acked Read 5 Sent Done Done false
30588 9677 Acked Flush 0 Sent Done Done false
30589 9678 Acked Read 4 Sent Done Done false
30590 9679 Acked Flush 0 Sent Done Done false
30591 9680 Acked Write 3 Sent Done Done false
30592 9681 Acked Write 5 Sent Done Done false
30593 9682 Acked Read 7 Sent Done Done false
30594 9683 Acked Write 9 Sent Done Done false
30595 9684 Acked Flush 0 Sent Done Done false
30596 9685 Acked Read 6 Sent Done Done false
30597 9686 Acked Write 10 Sent Done Done false
30598 9687 Acked Flush 0 Sent Done Done false
30599 9688 Acked Read 9 Sent Done Done false
30600 9689 Acked Read 8 Sent Done Done false
30601 9690 Acked Write 2 Sent Done Done false
30602 9691 Acked Write 6 Sent Done Done false
30603 9692 Acked Write 7 Sent Done Done false
30604 9693 Acked Flush 0 Sent Done Done false
30605 9694 Acked Write 2 Sent Done Done false
30606 9695 Acked Write 2 Sent Done Done false
30607 9696 Acked Write 6 Sent Done Done false
30608 9697 Acked Write 4 Sent Done Done false
30609 9698 Acked Write 6 Sent Done Done false
30610 9699 Acked Flush 0 Sent Done Done false
30611 9700 Acked Write 5 Sent Done Done false
30612 9701 Acked Read 10 Sent Done Done false
30613 9702 Acked Read 2 Sent Done Done false
30614 9703 Acked Read 3 Sent Done Done false
30615 9704 Acked Write 1 Sent Done Done false
30616 9705 Acked Read 6 Sent Done Done false
30617 9706 Acked Read 10 Sent Done Done false
30618 9707 Acked Write 7 Sent Done Done false
30619 9708 Acked Read 2 Sent Done Done false
30620 9709 Acked Flush 0 Sent Done Done false
30621 9710 Acked Read 10 Sent Done Done false
30622 9711 Acked Write 9 Sent Done Done false
30623 9712 Acked Read 1 Sent Done Done false
30624 9713 Acked Read 10 Sent Done Done false
30625 9714 Acked Write 1 Sent Done Done false
30626 9715 Acked Write 1 Sent Done Done false
30627 9716 Acked Read 3 Sent Done Done false
30628 9717 Acked Write 9 Sent Done Done false
30629 9718 Acked Write 5 Sent Done Done false
30630 9719 Acked Read 9 Sent Done Done false
30631 9720 Acked Read 10 Sent Done Done false
30632 9721 Acked Read 7 Sent Done Done false
30633 9722 Acked Read 9 Sent Done Done false
30634 9723 Acked Read 2 Sent Done Done false
30635 9724 Acked Write 7 Sent Done Done false
30636 9725 Acked Read 4 Sent Done Done false
30637 9726 Acked Read 8 Sent Done Done false
30638 9727 Acked Read 2 Sent Done Done false
30639 9728 Acked Read 3 Sent Done Done false
30640 9729 Acked Write 10 Sent Done Done false
30641 9730 Acked Write 3 Sent Done Done false
30642 9731 Acked Write 2 Sent Done Done false
30643 9732 Acked Write 10 Sent Done Done false
30644 9733 Acked Read 10 Sent Done Done false
30645 9734 Acked Write 4 Sent Done Done false
30646 9735 Acked Write 5 Sent Done Done false
30647 9736 Acked Read 3 Sent Done Done false
30648 9737 Acked Read 5 Sent Done Done false
30649 9738 Acked Read 8 Sent Done Done false
30650 9739 Acked Write 8 Sent Done Done false
30651 9740 Acked Read 6 Sent Done Done false
30652 9741 Acked Read 6 Sent Done Done false
30653 9742 Acked Read 4 Sent Done Done false
30654 9743 Acked Write 2 Sent Done Done false
30655 9744 Acked Flush 0 Sent Done Done false
30656 9745 Acked Read 4 Sent Done Done false
30657 9746 Acked Write 6 Sent Done Done false
30658 9747 Acked Read 9 Sent Done Done false
30659 9748 Acked Read 9 Sent Done Done false
30660 9749 Acked Read 7 Sent Done Done false
30661 9750 Acked Read 6 Sent Done Done false
30662 9751 Acked Write 4 Sent Done Done false
30663 9752 Acked Write 4 Sent Done Done false
30664 9753 Acked Write 7 Sent Done Done false
30665 9754 Acked Write 9 Sent Done Done false
30666 STATES DS:0 DS:1 DS:2 TOTAL
30667 Sent 301 0 0 301
30668 Done 0 301 301 602
30669 Skipped 0 0 0 0
30670 Error 0 0 0 0
30671 Last Flush: 9453 9744 9744
30672 Downstairs last five completed: 9453 9452 9451 9450 9449
30673 Upstairs last five completed: 9754 9753 9752 9751 9750
30674 CLIENT: Up:0 ds:301 act:2
30675 [8810] got start action
30676 [8810] I am going to restart
30677 Starting downstairs at port 8810
30678 Make output file at "/tmp/dsc/downstairs-8810.txt"
30679 [8810][0] reports Starting
30680 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1233)
30681 [8810][0] reports Running
30682 ----------------------------------------------------------------
30683 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30684 States: Offline Active Active
30685 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30686 9454 Acked Write 2 Sent Done Done false
30687 9455 Acked Write 10 Sent Done Done false
30688 9456 Acked Write 3 Sent Done Done false
30689 9457 Acked Write 4 Sent Done Done false
30690 9458 Acked Read 10 Sent Done Done false
30691 9459 Acked Flush 0 Sent Done Done false
30692 9460 Acked Flush 0 Sent Done Done false
30693 9461 Acked Write 4 Sent Done Done false
30694 9462 Acked Read 5 Sent Done Done false
30695 9463 Acked Read 6 Sent Done Done false
30696 9464 Acked Write 2 Sent Done Done false
30697 9465 Acked Write 9 Sent Done Done false
30698 9466 Acked Read 8 Sent Done Done false
30699 9467 Acked Read 6 Sent Done Done false
30700 9468 Acked Write 3 Sent Done Done false
30701 9469 Acked Write 2 Sent Done Done false
30702 9470 Acked Read 9 Sent Done Done false
30703 9471 Acked Read 6 Sent Done Done false
30704 9472 Acked Read 6 Sent Done Done false
30705 9473 Acked Write 8 Sent Done Done false
30706 9474 Acked Write 5 Sent Done Done false
30707 9475 Acked Read 3 Sent Done Done false
30708 9476 Acked Write 4 Sent Done Done false
30709 9477 Acked Read 1 Sent Done Done false
30710 9478 Acked Read 2 Sent Done Done false
30711 9479 Acked Read 1 Sent Done Done false
30712 9480 Acked Flush 0 Sent Done Done false
30713 9481 Acked Read 7 Sent Done Done false
30714 9482 Acked Flush 0 Sent Done Done false
30715 9483 Acked Write 3 Sent Done Done false
30716 9484 Acked Read 6 Sent Done Done false
30717 9485 Acked Read 5 Sent Done Done false
30718 9486 Acked Read 8 Sent Done Done false
30719 9487 Acked Read 7 Sent Done Done false
30720 9488 Acked Write 7 Sent Done Done false
30721 9489 Acked Read 10 Sent Done Done false
30722 9490 Acked Read 2 Sent Done Done false
30723 9491 Acked Read 4 Sent Done Done false
30724 9492 Acked Read 6 Sent Done Done false
30725 9493 Acked Read 3 Sent Done Done false
30726 9494 Acked Read 3 Sent Done Done false
30727 9495 Acked Read 5 Sent Done Done false
30728 9496 Acked Read 2 Sent Done Done false
30729 9497 Acked Write 1 Sent Done Done false
30730 9498 Acked Read 9 Sent Done Done false
30731 9499 Acked Read 8 Sent Done Done false
30732 9500 Acked Write 2 Sent Done Done false
30733 9501 Acked Flush 0 Sent Done Done false
30734 9502 Acked Flush 0 Sent Done Done false
30735 9503 Acked Flush 0 Sent Done Done false
30736 9504 Acked Write 7 Sent Done Done false
30737 9505 Acked Write 7 Sent Done Done false
30738 9506 Acked Read 3 Sent Done Done false
30739 9507 Acked Read 2 Sent Done Done false
30740 9508 Acked Write 8 Sent Done Done false
30741 9509 Acked Read 7 Sent Done Done false
30742 9510 Acked Read 8 Sent Done Done false
30743 9511 Acked Write 9 Sent Done Done false
30744 9512 Acked Read 8 Sent Done Done false
30745 9513 Acked Write 8 Sent Done Done false
30746 9514 Acked Read 3 Sent Done Done false
30747 9515 Acked Read 6 Sent Done Done false
30748 9516 Acked Flush 0 Sent Done Done false
30749 9517 Acked Read 6 Sent Done Done false
30750 9518 Acked Read 2 Sent Done Done false
30751 9519 Acked Read 5 Sent Done Done false
30752 9520 Acked Write 6 Sent Done Done false
30753 9521 Acked Flush 0 Sent Done Done false
30754 9522 Acked Write 9 Sent Done Done false
30755 9523 Acked Read 1 Sent Done Done false
30756 9524 Acked Write 5 Sent Done Done false
30757 9525 Acked Write 3 Sent Done Done false
30758 9526 Acked Read 3 Sent Done Done false
30759 9527 Acked Write 2 Sent Done Done false
30760 9528 Acked Read 4 Sent Done Done false
30761 9529 Acked Read 1 Sent Done Done false
30762 9530 Acked Read 6 Sent Done Done false
30763 9531 Acked Write 6 Sent Done Done false
30764 9532 Acked Flush 0 Sent Done Done false
30765 9533 Acked Write 7 Sent Done Done false
30766 9534 Acked Read 3 Sent Done Done false
30767 9535 Acked Write 9 Sent Done Done false
30768 9536 Acked Read 1 Sent Done Done false
30769 9537 Acked Read 2 Sent Done Done false
30770 9538 Acked Write 5 Sent Done Done false
30771 9539 Acked Read 10 Sent Done Done false
30772 9540 Acked Write 2 Sent Done Done false
30773 9541 Acked Write 9 Sent Done Done false
30774 9542 Acked Read 9 Sent Done Done false
30775 9543 Acked Read 7 Sent Done Done false
30776 9544 Acked Read 10 Sent Done Done false
30777 9545 Acked Read 1 Sent Done Done false
30778 9546 Acked Read 8 Sent Done Done false
30779 9547 Acked Write 4 Sent Done Done false
30780 9548 Acked Read 2 Sent Done Done false
30781 9549 Acked Read 7 Sent Done Done false
30782 9550 Acked Write 7 Sent Done Done false
30783 9551 Acked Read 6 Sent Done Done false
30784 9552 Acked Write 9 Sent Done Done false
30785 9553 Acked Read 9 Sent Done Done false
30786 9554 Acked Read 10 Sent Done Done false
30787 9555 Acked Flush 0 Sent Done Done false
30788 9556 Acked Flush 0 Sent Done Done false
30789 9557 Acked Read 3 Sent Done Done false
30790 9558 Acked Read 6 Sent Done Done false
30791 9559 Acked Read 2 Sent Done Done false
30792 9560 Acked Flush 0 Sent Done Done false
30793 9561 Acked Read 9 Sent Done Done false
30794 9562 Acked Read 4 Sent Done Done false
30795 9563 Acked Read 10 Sent Done Done false
30796 9564 Acked Write 3 Sent Done Done false
30797 9565 Acked Write 7 Sent Done Done false
30798 9566 Acked Write 4 Sent Done Done false
30799 9567 Acked Read 7 Sent Done Done false
30800 9568 Acked Write 8 Sent Done Done false
30801 9569 Acked Read 8 Sent Done Done false
30802 9570 Acked Write 6 Sent Done Done false
30803 9571 Acked Write 8 Sent Done Done false
30804 9572 Acked Write 5 Sent Done Done false
30805 9573 Acked Read 9 Sent Done Done false
30806 9574 Acked Write 1 Sent Done Done false
30807 9575 Acked Read 4 Sent Done Done false
30808 9576 Acked Read 3 Sent Done Done false
30809 9577 Acked Write 2 Sent Done Done false
30810 9578 Acked Write 8 Sent Done Done false
30811 9579 Acked Read 4 Sent Done Done false
30812 9580 Acked Read 1 Sent Done Done false
30813 9581 Acked Read 9 Sent Done Done false
30814 9582 Acked Read 2 Sent Done Done false
30815 9583 Acked Write 9 Sent Done Done false
30816 9584 Acked Write 6 Sent Done Done false
30817 9585 Acked Read 1 Sent Done Done false
30818 9586 Acked Read 8 Sent Done Done false
30819 9587 Acked Write 2 Sent Done Done false
30820 9588 Acked Read 2 Sent Done Done false
30821 9589 Acked Read 10 Sent Done Done false
30822 9590 Acked Read 5 Sent Done Done false
30823 9591 Acked Read 7 Sent Done Done false
30824 9592 Acked Read 2 Sent Done Done false
30825 9593 Acked Read 4 Sent Done Done false
30826 9594 Acked Write 9 Sent Done Done false
30827 9595 Acked Flush 0 Sent Done Done false
30828 9596 Acked Write 3 Sent Done Done false
30829 9597 Acked Write 8 Sent Done Done false
30830 9598 Acked Read 9 Sent Done Done false
30831 9599 Acked Flush 0 Sent Done Done false
30832 9600 Acked Write 6 Sent Done Done false
30833 9601 Acked Read 10 Sent Done Done false
30834 9602 Acked Read 5 Sent Done Done false
30835 9603 Acked Read 2 Sent Done Done false
30836 9604 Acked Read 6 Sent Done Done false
30837 9605 Acked Read 5 Sent Done Done false
30838 9606 Acked Read 8 Sent Done Done false
30839 9607 Acked Write 5 Sent Done Done false
30840 9608 Acked Write 7 Sent Done Done false
30841 9609 Acked Write 2 Sent Done Done false
30842 9610 Acked Read 8 Sent Done Done false
30843 9611 Acked Flush 0 Sent Done Done false
30844 9612 Acked Write 1 Sent Done Done false
30845 9613 Acked Read 7 Sent Done Done false
30846 9614 Acked Flush 0 Sent Done Done false
30847 9615 Acked Read 2 Sent Done Done false
30848 9616 Acked Read 4 Sent Done Done false
30849 9617 Acked Write 4 Sent Done Done false
30850 9618 Acked Write 10 Sent Done Done false
30851 9619 Acked Write 2 Sent Done Done false
30852 9620 Acked Read 5 Sent Done Done false
30853 9621 Acked Read 6 Sent Done Done false
30854 9622 Acked Read 6 Sent Done Done false
30855 9623 Acked Read 8 Sent Done Done false
30856 9624 Acked Write 10 Sent Done Done false
30857 9625 Acked Write 4 Sent Done Done false
30858 9626 Acked Read 9 Sent Done Done false
30859 9627 Acked Write 8 Sent Done Done false
30860 9628 Acked Read 10 Sent Done Done false
30861 9629 Acked Write 2 Sent Done Done false
30862 9630 Acked Read 8 Sent Done Done false
30863 9631 Acked Write 7 Sent Done Done false
30864 9632 Acked Write 4 Sent Done Done false
30865 9633 Acked Flush 0 Sent Done Done false
30866 9634 Acked Flush 0 Sent Done Done false
30867 9635 Acked Read 2 Sent Done Done false
30868 9636 Acked Write 2 Sent Done Done false
30869 9637 Acked Read 3 Sent Done Done false
30870 9638 Acked Read 5 Sent Done Done false
30871 9639 Acked Read 10 Sent Done Done false
30872 9640 Acked Write 1 Sent Done Done false
30873 9641 Acked Read 9 Sent Done Done false
30874 9642 Acked Flush 0 Sent Done Done false
30875 9643 Acked Read 3 Sent Done Done false
30876 9644 Acked Read 5 Sent Done Done false
30877 9645 Acked Write 2 Sent Done Done false
30878 9646 Acked Read 1 Sent Done Done false
30879 9647 Acked Read 7 Sent Done Done false
30880 9648 Acked Read 9 Sent Done Done false
30881 9649 Acked Read 6 Sent Done Done false
30882 9650 Acked Read 5 Sent Done Done false
30883 9651 Acked Write 1 Sent Done Done false
30884 9652 Acked Write 5 Sent Done Done false
30885 9653 Acked Read 8 Sent Done Done false
30886 9654 Acked Write 8 Sent Done Done false
30887 9655 Acked Write 3 Sent Done Done false
30888 9656 Acked Read 3 Sent Done Done false
30889 9657 Acked Read 6 Sent Done Done false
30890 9658 Acked Write 6 Sent Done Done false
30891 9659 Acked Read 1 Sent Done Done false
30892 9660 Acked Write 7 Sent Done Done false
30893 9661 Acked Write 3 Sent Done Done false
30894 9662 Acked Read 1 Sent Done Done false
30895 9663 Acked Read 3 Sent Done Done false
30896 9664 Acked Write 2 Sent Done Done false
30897 9665 Acked Flush 0 Sent Done Done false
30898 9666 Acked Write 4 Sent Done Done false
30899 9667 Acked Flush 0 Sent Done Done false
30900 9668 Acked Read 4 Sent Done Done false
30901 9669 Acked Read 6 Sent Done Done false
30902 9670 Acked Write 4 Sent Done Done false
30903 9671 Acked Read 3 Sent Done Done false
30904 9672 Acked Flush 0 Sent Done Done false
30905 9673 Acked Read 1 Sent Done Done false
30906 9674 Acked Read 9 Sent Done Done false
30907 9675 Acked Write 1 Sent Done Done false
30908 9676 Acked Read 5 Sent Done Done false
30909 9677 Acked Flush 0 Sent Done Done false
30910 9678 Acked Read 4 Sent Done Done false
30911 9679 Acked Flush 0 Sent Done Done false
30912 9680 Acked Write 3 Sent Done Done false
30913 9681 Acked Write 5 Sent Done Done false
30914 9682 Acked Read 7 Sent Done Done false
30915 9683 Acked Write 9 Sent Done Done false
30916 9684 Acked Flush 0 Sent Done Done false
30917 9685 Acked Read 6 Sent Done Done false
30918 9686 Acked Write 10 Sent Done Done false
30919 9687 Acked Flush 0 Sent Done Done false
30920 9688 Acked Read 9 Sent Done Done false
30921 9689 Acked Read 8 Sent Done Done false
30922 9690 Acked Write 2 Sent Done Done false
30923 9691 Acked Write 6 Sent Done Done false
30924 9692 Acked Write 7 Sent Done Done false
30925 9693 Acked Flush 0 Sent Done Done false
30926 9694 Acked Write 2 Sent Done Done false
30927 9695 Acked Write 2 Sent Done Done false
30928 9696 Acked Write 6 Sent Done Done false
30929 9697 Acked Write 4 Sent Done Done false
30930 9698 Acked Write 6 Sent Done Done false
30931 9699 Acked Flush 0 Sent Done Done false
30932 9700 Acked Write 5 Sent Done Done false
30933 9701 Acked Read 10 Sent Done Done false
30934 9702 Acked Read 2 Sent Done Done false
30935 9703 Acked Read 3 Sent Done Done false
30936 9704 Acked Write 1 Sent Done Done false
30937 9705 Acked Read 6 Sent Done Done false
30938 9706 Acked Read 10 Sent Done Done false
30939 9707 Acked Write 7 Sent Done Done false
30940 9708 Acked Read 2 Sent Done Done false
30941 9709 Acked Flush 0 Sent Done Done false
30942 9710 Acked Read 10 Sent Done Done false
30943 9711 Acked Write 9 Sent Done Done false
30944 9712 Acked Read 1 Sent Done Done false
30945 9713 Acked Read 10 Sent Done Done false
30946 9714 Acked Write 1 Sent Done Done false
30947 9715 Acked Write 1 Sent Done Done false
30948 9716 Acked Read 3 Sent Done Done false
30949 9717 Acked Write 9 Sent Done Done false
30950 9718 Acked Write 5 Sent Done Done false
30951 9719 Acked Read 9 Sent Done Done false
30952 9720 Acked Read 10 Sent Done Done false
30953 9721 Acked Read 7 Sent Done Done false
30954 9722 Acked Read 9 Sent Done Done false
30955 9723 Acked Read 2 Sent Done Done false
30956 9724 Acked Write 7 Sent Done Done false
30957 9725 Acked Read 4 Sent Done Done false
30958 9726 Acked Read 8 Sent Done Done false
30959 9727 Acked Read 2 Sent Done Done false
30960 9728 Acked Read 3 Sent Done Done false
30961 9729 Acked Write 10 Sent Done Done false
30962 9730 Acked Write 3 Sent Done Done false
30963 9731 Acked Write 2 Sent Done Done false
30964 9732 Acked Write 10 Sent Done Done false
30965 9733 Acked Read 10 Sent Done Done false
30966 9734 Acked Write 4 Sent Done Done false
30967 9735 Acked Write 5 Sent Done Done false
30968 9736 Acked Read 3 Sent Done Done false
30969 9737 Acked Read 5 Sent Done Done false
30970 9738 Acked Read 8 Sent Done Done false
30971 9739 Acked Write 8 Sent Done Done false
30972 9740 Acked Read 6 Sent Done Done false
30973 9741 Acked Read 6 Sent Done Done false
30974 9742 Acked Read 4 Sent Done Done false
30975 9743 Acked Write 2 Sent Done Done false
30976 9744 Acked Flush 0 Sent Done Done false
30977 9745 Acked Read 4 Sent Done Done false
30978 9746 Acked Write 6 Sent Done Done false
30979 9747 Acked Read 9 Sent Done Done false
30980 9748 Acked Read 9 Sent Done Done false
30981 9749 Acked Read 7 Sent Done Done false
30982 9750 Acked Read 6 Sent Done Done false
30983 9751 Acked Write 4 Sent Done Done false
30984 9752 Acked Write 4 Sent Done Done false
30985 9753 Acked Write 7 Sent Done Done false
30986 9754 Acked Write 9 Sent Done Done false
30987 9755 Acked Flush 0 Sent Done Done false
30988 STATES DS:0 DS:1 DS:2 TOTAL
30989 Sent 302 0 0 302
30990 Done 0 302 302 604
30991 Skipped 0 0 0 0
30992 Error 0 0 0 0
30993 Last Flush: 9453 9755 9755
30994 Downstairs last five completed: 9453 9452 9451 9450 9449
30995 Upstairs last five completed: 9755 9754 9753 9752 9751
30996 CLIENT: Up:0 ds:302 act:2
309972025-01-11T06:11:55.554ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
309982025-01-11T06:11:55.554ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36706) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
309992025-01-11T06:11:55.555ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310002025-01-11T06:11:55.556ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310012025-01-11T06:11:55.556ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310022025-01-11T06:11:55.556ZINFOcrucible: send last flush ID to this DS: 9453 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310032025-01-11T06:11:55.556ZINFOcrucible: Replied this last flush ID: 9453 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310042025-01-11T06:11:55.556ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310052025-01-11T06:11:55.556ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310062025-01-11T06:11:55.556ZINFOcrucible: [0] client re-new 302 jobs since flush 9453 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310072025-01-11T06:11:55.556ZINFOcrucible: [0] Replayed 302 jobs since flush: 9453 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
31008 ----------------------------------------------------------------
31009 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31010 Downstairs last five completed: 9755 9754 9753 9752 9751
31011 Upstairs last five completed: 9755 9754 9753 9752 9751
31012 CLIENT: Up:0 ds:0 act:3
31013 Replay: All jobs finished
31014 Main task has work to do, go find it
31015 got dsc Stop(1)
31016 stop 1
31017 [8820] Got stop action so:false kr:false
31018 Jan 11 06:11:57.804 INFO request completed, latency_us: 343, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 451dbfa1-9bdf-415e-b63d-98c89bdd7a4d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
310192025-01-11T06:11:57.805ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
31020 Jan 11 06:11:57.805 INFO request completed, latency_us: 151, response_code: 200, uri: /state/cid/1, method: GET, req_id: 49da6617-6e0b-44f0-914c-308b157b7389, remote_addr: 127.0.0.1:59925, local_addr: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-11T06:11:57.805797776Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1204,"":"io task","client":"1","":"downstairs","session_id":"e3b437a4-33d5-43e8-8b38-4c1c18ad0389"}
31021 127.0.0.1:9998
310222025-01-11T06:11:57.805ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310232025-01-11T06:11:57.805ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310242025-01-11T06:11:57.805ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
310252025-01-11T06:11:57.805ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
31026 [8820] Exited with: ExitStatus(unix_wait_status(9))
31027 [8820][1] reports Exit
31028 Jan 11 06:12:01.807 INFO request completed, latency_us: 223, response_code: 200, uri: /state/cid/1, method: GET, req_id: 49209c33-2b74-42f2-a011-5967faa0490d, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
31029 001/300 Read block 815 len 8192
31030 002/300 Write block 1218 len 40960 data: 7 6 9 6 7 6 6 8 7 7
31031 003/300 Read block 593 len 40960
31032 004/300 Write block 1857 len 28672 data: 6 5 5 5 5 5 5
31033 005/300 Read block 4333 len 4096
31034 006/300 Read block 4578 len 12288
31035 007/300 Read block 3618 len 36864
31036 008/300 Read block 2911 len 16384
31037 009/300 Write block 4104 len 36864 data: 3 3 3 5 3 3 4 4 4
31038 010/300 Read block 577 len 40960
31039 011/300 Write block 3099 len 24576 data: 8 9 9 10 8 7
31040 012/300 Read block 4149 len 24576
31041 013/300 Read block 4818 len 32768
31042 014/300 Read block 2878 len 20480
31043 015/300 Flush
31044 016/300 Read block 2842 len 28672
31045 017/300 Write block 1092 len 20480 data: 7 10 10 11 10
31046 018/300 Write block 3528 len 32768 data: 7 7 8 8 8 9 7 5
31047 019/300 Flush
31048 020/300 Read block 2585 len 20480
31049 021/300 Write block 3815 len 32768 data: 6 7 8 8 8 10 8 10
31050 022/300 Read block 3910 len 36864
31051 023/300 Write block 2649 len 16384 data: 8 3 2 3
31052 024/300 Read block 875 len 32768
31053 025/300 Write block 3227 len 28672 data: 7 7 5 6 6 6 5
31054 026/300 Write block 1441 len 28672 data: 4 5 6 6 6 6 5
31055 027/300 Write block 2593 len 12288 data: 7 7 6
31056 028/300 Read block 3562 len 8192
31057 029/300 Read block 3253 len 12288
31058 030/300 Write block 1999 len 28672 data: 5 6 5 7 6 6 8
31059 031/300 Write block 1395 len 36864 data: 6 5 6 7 8 7 7 7 7
31060 032/300 Read block 2770 len 32768
31061 033/300 Read block 3914 len 12288
31062 034/300 Write block 4871 len 40960 data: 6 7 6 8 8 9 9 9 7 9
31063 035/300 Write block 3050 len 40960 data: 7 7 7 6 5 4 3 4 4 4
31064 036/300 Read block 4667 len 36864
31065 037/300 Write block 2868 len 20480 data: 10 9 10 8 6
31066 038/300 Write block 839 len 40960 data: 9 11 9 8 8 8 8 6 5 7
31067 039/300 Write block 349 len 4096 data: 3
31068 040/300 Flush
31069 041/300 Read block 2500 len 16384
31070 042/300 Read block 2468 len 32768
31071 043/300 Write block 3668 len 36864 data: 4 5 4 6 7 7 7 8 8
31072 044/300 Write block 3551 len 28672 data: 3 4 4 3 5 4 3
31073 045/300 Write block 2620 len 36864 data: 5 5 5 7 8 9 9 9 9
31074 046/300 Read block 3487 len 28672
31075 047/300 Read block 1224 len 20480
31076 048/300 Flush
31077 049/300 Write block 834 len 28672 data: 6 7 8 7 9 10 12
31078 050/300 Read block 2007 len 4096
31079 051/300 Read block 2053 len 40960
31080 052/300 Read block 1143 len 8192
31081 053/300 Read block 4090 len 36864
31082 054/300 Write block 1940 len 32768 data: 9 9 7 8 7 6 6 6
31083 055/300 Read block 1178 len 28672
31084 056/300 Read block 1735 len 16384
31085 057/300 Read block 3080 len 28672
31086 058/300 Write block 281 len 8192 data: 10 11
31087 059/300 Read block 1218 len 4096
31088 060/300 Write block 3790 len 24576 data: 5 4 5 5 5 5
31089 061/300 Write block 4006 len 36864 data: 5 5 6 5 5 6 7 8 8
31090 062/300 Write block 4914 len 4096 data: 3
31091 063/300 Write block 2832 len 12288 data: 7 7 7
31092 064/300 Read block 53 len 16384
31093 065/300 Write block 1419 len 40960 data: 4 3 3 4 5 5 5 5 7 8
31094 066/300 Flush
31095 067/300 Write block 2538 len 20480 data: 5 5 5 4 4
31096 068/300 Write block 4498 len 16384 data: 3 2 2 3
31097 069/300 Read block 252 len 16384
31098 070/300 Read block 1455 len 16384
31099 071/300 Write block 2839 len 12288 data: 3 2 4
31100 072/300 Read block 1884 len 8192
31101 073/300 Read block 4354 len 40960
31102 074/300 Read block 3510 len 32768
31103 075/300 Write block 4095 len 8192 data: 4 3
31104 076/300 Write block 2935 len 36864 data: 12 11 10 9 8 10 8 7 7
31105 077/300 Write block 1767 len 24576 data: 5 4 5 7 6 7
31106 078/300 Write block 2392 len 4096 data: 4
31107 079/300 Read block 2532 len 24576
31108 080/300 Read block 3884 len 16384
31109 081/300 Read block 1414 len 12288
31110 082/300 Read block 3904 len 36864
31111 083/300 Write block 1067 len 16384 data: 4 5 5 5
31112 084/300 Read block 780 len 24576
31113 085/300 Write block 2789 len 32768 data: 5 4 4 4 5 6 6 6
31114 086/300 Write block 1624 len 36864 data: 7 6 7 7 6 3 5 4 4
31115 087/300 Write block 3759 len 20480 data: 8 8 6 5 5
31116 088/300 Read block 4690 len 8192
31117 089/300 Write block 4686 len 16384 data: 9 8 8 9
31118 090/300 Write block 906 len 8192 data: 6 6
31119 091/300 Write block 3384 len 12288 data: 6 5 5
31120 092/300 Flush
31121 093/300 Read block 2439 len 8192
31122 094/300 Read block 2842 len 16384
31123 095/300 Write block 4150 len 8192 data: 6 4
31124 096/300 Read block 1814 len 20480
31125 097/300 Read block 2777 len 8192
31126 098/300 Write block 2089 len 32768 data: 6 6 6 8 8 7 6 8
31127 099/300 Write block 3146 len 20480 data: 5 8 8 7 6
31128 100/300 Write block 2571 len 20480 data: 6 8 7 8 7
31129 101/300 Read block 2431 len 36864
31130 102/300 Read block 2858 len 36864
31131 103/300 Read block 4291 len 36864
31132 104/300 Read block 3532 len 40960
31133 105/300 Read block 1638 len 8192
31134 106/300 Read block 2504 len 8192
31135 107/300 Read block 2764 len 36864
31136 108/300 Read block 3922 len 28672
31137 109/300 Read block 2977 len 20480
31138 110/300 Read block 4534 len 40960
31139 111/300 Write block 4737 len 24576 data: 4 5 5 6 6 5
31140 112/300 Read block 3644 len 32768
31141 113/300 Write block 1443 len 20480 data: 7 7 7 7 6
31142 114/300 Flush
31143 115/300 Write block 3109 len 20480 data: 4 3 3 4 6
31144 116/300 Write block 116 len 36864 data: 8 8 8 8 9 8 6 6 7
31145 117/300 Read block 511 len 8192
31146 118/300 Write block 2299 len 20480 data: 9 10 12 11 9
31147 119/300 Write block 108 len 12288 data: 7 7 7
31148 120/300 Read block 3553 len 32768
31149 121/300 Read block 4125 len 28672
31150 122/300 Write block 1075 len 8192 data: 5 5
31151 123/300 Read block 4236 len 16384
31152 124/300 Write block 4534 len 16384 data: 6 6 5 5
31153 125/300 Read block 248 len 8192
31154 126/300 Read block 3974 len 40960
31155 127/300 Read block 3033 len 28672
31156 128/300 Read block 1253 len 4096
31157 129/300 Write block 2854 len 40960 data: 5 6 5 7 6 8 8 9 12 13
31158 130/300 Flush
31159 131/300 Read block 2280 len 16384
31160 132/300 Read block 2624 len 24576
31161 133/300 Read block 2169 len 20480
31162 134/300 Read block 642 len 36864
31163 135/300 Flush
31164 136/300 Read block 4865 len 4096
31165 137/300 Read block 274 len 8192
31166 138/300 Write block 4221 len 24576 data: 5 6 6 6 6 5
31167 139/300 Read block 4323 len 8192
31168 140/300 Write block 2182 len 12288 data: 6 6 7
31169 141/300 Read block 1955 len 4096
31170 142/300 Read block 1324 len 40960
31171 143/300 Read block 1491 len 40960
31172 144/300 Write block 4343 len 24576 data: 3 3 3 3 2 2
31173 145/300 Write block 3301 len 24576 data: 7 4 5 5 6 6
31174 146/300 Read block 751 len 16384
31175 147/300 Write block 2023 len 20480 data: 4 3 2 2 4
31176 148/300 Flush
31177 149/300 Read block 92 len 28672
31178 150/300 Flush
31179 151/300 Read block 3180 len 36864
31180 152/300 Read block 1178 len 40960
31181 153/300 Read block 3305 len 12288
31182 154/300 Read block 3559 len 8192
31183 155/300 Write block 3762 len 12288 data: 6 6 5
31184 156/300 Write block 3702 len 20480 data: 5 4 6 6 7
31185 157/300 Read block 1392 len 20480
31186 158/300 Read block 2945 len 24576
31187 159/300 Read block 1181 len 4096
31188 160/300 Read block 892 len 28672
31189 161/300 Flush
31190 162/300 Read block 3366 len 36864
31191 163/300 Read block 2746 len 4096
31192 164/300 Read block 3038 len 32768
31193 165/300 Flush
31194 166/300 Write block 4283 len 16384 data: 6 6 9 9
31195 167/300 Read block 433 len 36864
31196 168/300 Read block 3409 len 16384
31197 169/300 Read block 107 len 12288
31198 170/300 Read block 2548 len 40960
31199 171/300 Read block 4626 len 12288
31200 172/300 Write block 214 len 36864 data: 6 6 7 7 4 4 6 4 4
31201 173/300 Read block 3311 len 36864
31202 174/300 Flush
31203 175/300 Read block 1871 len 8192
31204 176/300 Write block 2475 len 8192 data: 6 6
31205 177/300 Flush
31206 178/300 Write block 4588 len 8192 data: 6 6
31207 179/300 Write block 139 len 32768 data: 7 7 7 5 6 5 5 5
31208 180/300 Write block 3456 len 4096 data: 4
31209 181/300 Read block 880 len 28672
31210 182/300 Write block 2280 len 4096 data: 7
31211 183/300 Read block 4311 len 32768
31212 184/300 Flush
31213 185/300 Read block 1512 len 8192
31214 186/300 Flush
31215 187/300 Read block 2386 len 36864
31216 188/300 Read block 1858 len 36864
31217 189/300 Write block 4954 len 24576 data: 6 5 5 6 6 6
31218 190/300 Read block 2792 len 4096
31219 191/300 Write block 783 len 20480 data: 5 5 6 6 5
31220 192/300 Read block 519 len 4096
31221 193/300 Read block 3973 len 12288
31222 194/300 Write block 4573 len 28672 data: 6 7 7 8 8 8 6
31223 195/300 Read block 2454 len 28672
31224 196/300 Read block 2624 len 20480
31225 197/300 Write block 994 len 40960 data: 6 6 6 6 4 4 5 8 7 7
31226 198/300 Read block 1426 len 36864
31227 199/300 Read block 672 len 8192
31228 200/300 Write block 4752 len 24576 data: 6 5 5 4 4 4
31229 201/300 Read block 138 len 16384
31230 202/300 Write block 4706 len 40960 data: 3 4 3 4 4 6 5 5 6 7
31231 203/300 Flush
31232 204/300 Write block 1220 len 4096 data: 10
31233 205/300 Read block 1860 len 20480
31234 206/300 Write block 2236 len 24576 data: 6 7 7 6 4 5
31235 207/300 Flush
31236 208/300 Read block 3008 len 16384
31237 209/300 Write block 4357 len 28672 data: 8 10 11 10 7 6 5
31238 210/300 Write block 4443 len 16384 data: 4 4 4 5
31239 211/300 Read block 1347 len 40960
31240 212/300 Read block 3400 len 20480
31241 213/300 Read block 1444 len 32768
31242 214/300 Write block 1700 len 24576 data: 11 11 10 10 9 8
31243 215/300 Read block 2714 len 12288
31244 216/300 Read block 476 len 12288
31245 217/300 Read block 3617 len 36864
31246 218/300 Write block 3040 len 40960 data: 8 8 8 6 5 5 5 5 6 6
31247 219/300 Write block 3464 len 28672 data: 6 5 5 6 7 9 9
31248 220/300 Read block 515 len 28672
31249 221/300 Write block 1826 len 28672 data: 6 9 9 11 11 8 8
31250 222/300 Write block 3794 len 28672 data: 6 6 5 4 4 4 5
31251 223/300 Read block 3868 len 28672
31252 224/300 Read block 3053 len 12288
31253 225/300 Write block 3261 len 4096 data: 5
31254 226/300 Write block 879 len 24576 data: 9 10 8 7 8 6
31255 227/300 Read block 2936 len 40960
31256 228/300 Read block 3955 len 16384
31257 229/300 Write block 167 len 28672 data: 5 5 3 2 2 4 5
31258 230/300 Read block 1140 len 4096
31259 231/300 Read block 1704 len 8192
31260 232/300 Read block 1275 len 4096
31261 233/300 Flush
31262 234/300 Write block 4956 len 32768 data: 6 7 7 7 2 3 3 4
31263 235/300 Read block 3531 len 20480
31264 236/300 Read block 2664 len 32768
31265 237/300 Read block 1715 len 36864
31266 238/300 Write block 1105 len 8192 data: 4 4
31267 239/300 Flush
31268 240/300 Write block 609 len 20480 data: 5 5 4 3 3
31269 241/300 Read block 134 len 36864
31270 242/300 Write block 2297 len 36864 data: 7 4 10 11 13 12 10 8 8
31271 243/300 Read block 2153 len 4096
31272 244/300 Read block 4505 len 8192
31273 245/300 Flush
31274 246/300 Flush
31275 247/300 Read block 1935 len 12288
31276 248/300 Write block 4825 len 28672 data: 7 7 7 8 8 8 8
31277 249/300 Write block 3161 len 24576 data: 7 7 6 5 6 4
31278 250/300 Write block 895 len 24576 data: 2 2 2 3 3 2
31279 251/300 Write block 2457 len 40960 data: 8 8 7 6 6 6 8 9 5 4
31280 252/300 Flush
31281 253/300 Write block 2868 len 4096 data: 11
31282 254/300 Read block 110 len 16384
31283 255/300 Read block 742 len 32768
31284 256/300 Read block 3113 len 16384
31285 257/300 Flush
31286 258/300 Write block 1668 len 32768 data: 6 9 9 8 7 6 5 5
31287 259/300 Read block 2031 len 20480
31288 260/300 Read block 2452 len 12288
31289 261/300 Write block 2585 len 40960 data: 3 4 5 5 6 5 7 8 8 8
31290 262/300 Read block 166 len 16384
31291 263/300 Write block 1598 len 24576 data: 5 4 4 5 4 4
31292 264/300 Write block 4967 len 16384 data: 8 8 5 5
31293 265/300 Write block 4743 len 36864 data: 6 6 6 6 8 7 5 6 5
31294 266/300 Read block 706 len 32768
31295 267/300 Flush
31296 268/300 Write block 2011 len 32768 data: 6 6 7 9 6 6 7 7
31297 269/300 Write block 2067 len 28672 data: 5 6 6 5 4 4 4
31298 270/300 Flush
31299 271/300 Write block 1793 len 20480 data: 6 6 7 6 7
31300 272/300 Read block 2498 len 28672
31301 273/300 Read block 4683 len 16384
31302 274/300 Write block 3980 len 28672 data: 6 6 5 5 3 2 2
31303 275/300 Read block 3928 len 24576
31304 276/300 Read block 1791 len 28672
31305 277/300 Read block 361 len 20480
31306 278/300 Write block 3760 len 12288 data: 9 7 7
31307 279/300 Flush
31308 280/300 Write block 2346 len 4096 data: 9
31309 281/300 Read block 3053 len 16384
31310 282/300 Write block 4191 len 16384 data: 4 4 4 5
31311 283/300 Read block 3831 len 20480
31312 284/300 Read block 4723 len 8192
31313 285/300 Read block 1405 len 24576
31314 286/300 Flush
31315 287/300 Write block 10 len 8192 data: 7 8
31316 288/300 Read block 3448 len 16384
31317 289/300 Write block 109 len 32768 data: 8 8 6 7 7 6 7 9
31318 290/300 Read block 1825 len 24576
31319 291/300 Flush
31320 292/300 Read block 2124 len 36864
31321 293/300 Write block 1476 len 24576 data: 6 8 6 7 8 8
31322 294/300 Write block 1497 len 24576 data: 10 8 7 6 7 6
31323 295/300 Read block 4348 len 24576
31324 296/300 Read block 2241 len 12288
31325 297/300 Read block 3068 len 32768
31326 298/300 Read block 880 len 16384
31327 299/300 Read block 3323 len 12288
31328 300/300 Write block 564 len 20480 data: 6 6 5 5 7
31329 301/300 Read block 497 len 32768
31330 Main task has work to do, go find it
31331 got dsc Start(1)
31332 start 1
31333 Jan 11 06:12:02.060 INFO request completed, latency_us: 236, response_code: 204, uri: /start/cid/1, method: POST, req_id: 6b2dc1a1-cf30-440e-9ac9-da3e7f8c1e10, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
31334 Replay: started 1, returned:Ok(())
31335 ----------------------------------------------------------------
31336 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31337 States: Active Offline Active
31338 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31339 9756 Acked Read 2 Done Sent Done false
31340 9757 Acked Write 10 Done Sent Done false
31341 9758 Acked Read 10 Done Sent Done false
31342 9759 Acked Write 7 Done Sent Done false
31343 9760 Acked Read 1 Done Sent Done false
31344 9761 Acked Read 3 Done Sent Done false
31345 9762 Acked Read 9 Done Sent Done false
31346 9763 Acked Read 4 Done Sent Done false
31347 9764 Acked Write 9 Done Sent Done false
31348 9765 Acked Read 10 Done Sent Done false
31349 9766 Acked Write 6 Done Sent Done false
31350 9767 Acked Read 6 Done Sent Done false
31351 9768 Acked Read 8 Done Sent Done false
31352 9769 Acked Read 5 Done Sent Done false
31353 9770 Acked Flush 0 Done Sent Done false
31354 9771 Acked Read 7 Done Sent Done false
31355 9772 Acked Write 5 Done Sent Done false
31356 9773 Acked Write 8 Done Sent Done false
31357 9774 Acked Flush 0 Done Sent Done false
31358 9775 Acked Read 5 Done Sent Done false
31359 9776 Acked Write 8 Done Sent Done false
31360 9777 Acked Read 9 Done Sent Done false
31361 9778 Acked Write 4 Done Sent Done false
31362 9779 Acked Read 8 Done Sent Done false
31363 9780 Acked Write 7 Done Sent Done false
31364 9781 Acked Write 7 Done Sent Done false
31365 9782 Acked Write 3 Done Sent Done false
31366 9783 Acked Read 2 Done Sent Done false
31367 9784 Acked Read 3 Done Sent Done false
31368 9785 Acked Write 7 Done Sent Done false
31369 9786 Acked Write 9 Done Sent Done false
31370 9787 Acked Read 8 Done Sent Done false
31371 9788 Acked Read 3 Done Sent Done false
31372 9789 Acked Write 10 Done Sent Done false
31373 9790 Acked Write 10 Done Sent Done false
31374 9791 Acked Read 9 Done Sent Done false
31375 9792 Acked Write 5 Done Sent Done false
31376 9793 Acked Write 10 Done Sent Done false
31377 9794 Acked Write 1 Done Sent Done false
31378 9795 Acked Flush 0 Done Sent Done false
31379 9796 Acked Read 4 Done Sent Done false
31380 9797 Acked Read 8 Done Sent Done false
31381 9798 Acked Write 9 Done Sent Done false
31382 9799 Acked Write 7 Done Sent Done false
31383 9800 Acked Write 9 Done Sent Done false
31384 9801 Acked Read 7 Done Sent Done false
31385 9802 Acked Read 5 Done Sent Done false
31386 9803 Acked Flush 0 Done Sent Done false
31387 9804 Acked Write 7 Done Sent Done false
31388 9805 Acked Read 1 Done Sent Done false
31389 9806 Acked Read 10 Done Sent Done false
31390 9807 Acked Read 2 Done Sent Done false
31391 9808 Acked Read 9 Done Sent Done false
31392 9809 Acked Write 8 Done Sent Done false
31393 9810 Acked Read 7 Done Sent Done false
31394 9811 Acked Read 4 Done Sent Done false
31395 9812 Acked Read 7 Done Sent Done false
31396 9813 Acked Write 2 Done Sent Done false
31397 9814 Acked Read 1 Done Sent Done false
31398 9815 Acked Write 6 Done Sent Done false
31399 9816 Acked Write 9 Done Sent Done false
31400 9817 Acked Write 1 Done Sent Done false
31401 9818 Acked Write 3 Done Sent Done false
31402 9819 Acked Read 4 Done Sent Done false
31403 9820 Acked Write 10 Done Sent Done false
31404 9821 Acked Flush 0 Done Sent Done false
31405 9822 Acked Write 5 Done Sent Done false
31406 9823 Acked Write 4 Done Sent Done false
31407 9824 Acked Read 4 Done Sent Done false
31408 9825 Acked Read 4 Done Sent Done false
31409 9826 Acked Write 3 Done Sent Done false
31410 9827 Acked Read 2 Done Sent Done false
31411 9828 Acked Read 10 Done Sent Done false
31412 9829 Acked Read 8 Done Sent Done false
31413 9830 Acked Write 2 Done Sent Done false
31414 9831 Acked Write 9 Done Sent Done false
31415 9832 Acked Write 6 Done Sent Done false
31416 9833 Acked Write 1 Done Sent Done false
31417 9834 Acked Read 6 Done Sent Done false
31418 9835 Acked Read 4 Done Sent Done false
31419 9836 Acked Read 3 Done Sent Done false
31420 9837 Acked Read 9 Done Sent Done false
31421 9838 Acked Write 4 Done Sent Done false
31422 9839 Acked Read 6 Done Sent Done false
31423 9840 Acked Write 8 Done Sent Done false
31424 9841 Acked Write 9 Done Sent Done false
31425 9842 Acked Write 5 Done Sent Done false
31426 9843 Acked Read 2 Done Sent Done false
31427 9844 Acked Write 4 Done Sent Done false
31428 9845 Acked Write 2 Done Sent Done false
31429 9846 Acked Write 3 Done Sent Done false
31430 9847 Acked Flush 0 Done Sent Done false
31431 9848 Acked Read 2 Done Sent Done false
31432 9849 Acked Read 4 Done Sent Done false
31433 9850 Acked Write 2 Done Sent Done false
31434 9851 Acked Read 5 Done Sent Done false
31435 9852 Acked Read 2 Done Sent Done false
31436 9853 Acked Write 8 Done Sent Done false
31437 9854 Acked Write 5 Done Sent Done false
31438 9855 Acked Write 5 Done Sent Done false
31439 9856 Acked Read 9 Done Sent Done false
31440 9857 Acked Read 9 Done Sent Done false
31441 9858 Acked Read 9 Done Sent Done false
31442 9859 Acked Read 10 Done Sent Done false
31443 9860 Acked Read 2 Done Sent Done false
31444 9861 Acked Read 2 Done Sent Done false
31445 9862 Acked Read 9 Done Sent Done false
31446 9863 Acked Read 7 Done Sent Done false
31447 9864 Acked Read 5 Done Sent Done false
31448 9865 Acked Read 10 Done Sent Done false
31449 9866 Acked Write 6 Done Sent Done false
31450 9867 Acked Read 8 Done Sent Done false
31451 9868 Acked Write 5 Done Sent Done false
31452 9869 Acked Flush 0 Done Sent Done false
31453 9870 Acked Write 5 Done Sent Done false
31454 9871 Acked Write 9 Done Sent Done false
31455 9872 Acked Read 2 Done Sent Done false
31456 9873 Acked Write 5 Done Sent Done false
31457 9874 Acked Write 3 Done Sent Done false
31458 9875 Acked Read 8 Done Sent Done false
31459 9876 Acked Read 7 Done Sent Done false
31460 9877 Acked Write 2 Done Sent Done false
31461 9878 Acked Read 4 Done Sent Done false
31462 9879 Acked Write 4 Done Sent Done false
31463 9880 Acked Read 2 Done Sent Done false
31464 9881 Acked Read 10 Done Sent Done false
31465 9882 Acked Read 7 Done Sent Done false
31466 9883 Acked Read 1 Done Sent Done false
31467 9884 Acked Write 10 Done Sent Done false
31468 9885 Acked Flush 0 Done Sent Done false
31469 9886 Acked Read 4 Done Sent Done false
31470 9887 Acked Read 6 Done Sent Done false
31471 9888 Acked Read 5 Done Sent Done false
31472 9889 Acked Read 9 Done Sent Done false
31473 9890 Acked Flush 0 Done Sent Done false
31474 9891 Acked Read 1 Done Sent Done false
31475 9892 Acked Read 2 Done Sent Done false
31476 9893 Acked Write 6 Done Sent Done false
31477 9894 Acked Read 2 Done Sent Done false
31478 9895 Acked Write 3 Done Sent Done false
31479 9896 Acked Read 1 Done Sent Done false
31480 9897 Acked Read 10 Done Sent Done false
31481 9898 Acked Read 10 Done Sent Done false
31482 9899 Acked Write 6 Done Sent Done false
31483 9900 Acked Write 6 Done Sent Done false
31484 9901 Acked Read 4 Done Sent Done false
31485 9902 Acked Write 5 Done Sent Done false
31486 9903 Acked Flush 0 Done Sent Done false
31487 9904 Acked Read 7 Done Sent Done false
31488 9905 Acked Flush 0 Done Sent Done false
31489 9906 Acked Read 9 Done Sent Done false
31490 9907 Acked Read 10 Done Sent Done false
31491 9908 Acked Read 3 Done Sent Done false
31492 9909 Acked Read 2 Done Sent Done false
31493 9910 Acked Write 3 Done Sent Done false
31494 9911 Acked Write 5 Done Sent Done false
31495 9912 Acked Read 5 Done Sent Done false
31496 9913 Acked Read 6 Done Sent Done false
31497 9914 Acked Read 1 Done Sent Done false
31498 9915 Acked Read 7 Done Sent Done false
31499 9916 Acked Flush 0 Done Sent Done false
31500 9917 Acked Read 9 Done Sent Done false
31501 9918 Acked Read 1 Done Sent Done false
31502 9919 Acked Read 8 Done Sent Done false
31503 9920 Acked Flush 0 Done Sent Done false
31504 9921 Acked Write 4 Done Sent Done false
31505 9922 Acked Read 9 Done Sent Done false
31506 9923 Acked Read 4 Done Sent Done false
31507 9924 Acked Read 3 Done Sent Done false
31508 9925 Acked Read 10 Done Sent Done false
31509 9926 Acked Read 3 Done Sent Done false
31510 9927 Acked Write 9 Done Sent Done false
31511 9928 Acked Read 9 Done Sent Done false
31512 9929 Acked Flush 0 Done Sent Done false
31513 9930 Acked Read 2 Done Sent Done false
31514 9931 Acked Write 2 Done Sent Done false
31515 9932 Acked Flush 0 Done Sent Done false
31516 9933 Acked Write 2 Done Sent Done false
31517 9934 Acked Write 8 Done Sent Done false
31518 9935 Acked Write 1 Done Sent Done false
31519 9936 Acked Read 7 Done Sent Done false
31520 9937 Acked Write 1 Done Sent Done false
31521 9938 Acked Read 8 Done Sent Done false
31522 9939 Acked Flush 0 Done Sent Done false
31523 9940 Acked Read 2 Done Sent Done false
31524 9941 Acked Flush 0 Done Sent Done false
31525 9942 Acked Read 9 Done Sent Done false
31526 9943 Acked Read 9 Done Sent Done false
31527 9944 Acked Write 6 Done Sent Done false
31528 9945 Acked Read 1 Done Sent Done false
31529 9946 Acked Write 5 Done Sent Done false
31530 9947 Acked Read 1 Done Sent Done false
31531 9948 Acked Read 3 Done Sent Done false
31532 9949 Acked Write 7 Done Sent Done false
31533 9950 Acked Read 7 Done Sent Done false
31534 9951 Acked Read 5 Done Sent Done false
31535 9952 Acked Write 10 Done Sent Done false
31536 9953 Acked Read 9 Done Sent Done false
31537 9954 Acked Read 2 Done Sent Done false
31538 9955 Acked Write 6 Done Sent Done false
31539 9956 Acked Read 4 Done Sent Done false
31540 9957 Acked Write 10 Done Sent Done false
31541 9958 Acked Flush 0 Done Sent Done false
31542 9959 Acked Write 1 Done Sent Done false
31543 9960 Acked Read 5 Done Sent Done false
31544 9961 Acked Write 6 Done Sent Done false
31545 9962 Acked Flush 0 Done Sent Done false
31546 9963 Acked Read 4 Done Sent Done false
31547 9964 Acked Write 7 Done Sent Done false
31548 9965 Acked Write 4 Done Sent Done false
31549 9966 Acked Read 10 Done Sent Done false
31550 9967 Acked Read 5 Done Sent Done false
31551 9968 Acked Read 8 Done Sent Done false
31552 9969 Acked Write 6 Done Sent Done false
31553 9970 Acked Read 3 Done Sent Done false
31554 9971 Acked Read 3 Done Sent Done false
31555 9972 Acked Read 9 Done Sent Done false
31556 9973 Acked Write 10 Done Sent Done false
31557 9974 Acked Write 7 Done Sent Done false
31558 9975 Acked Read 7 Done Sent Done false
31559 9976 Acked Write 7 Done Sent Done false
31560 9977 Acked Write 7 Done Sent Done false
31561 9978 Acked Read 7 Done Sent Done false
31562 9979 Acked Read 3 Done Sent Done false
31563 9980 Acked Write 1 Done Sent Done false
31564 9981 Acked Write 6 Done Sent Done false
31565 9982 Acked Read 10 Done Sent Done false
31566 9983 Acked Read 4 Done Sent Done false
31567 9984 Acked Write 7 Done Sent Done false
31568 9985 Acked Read 1 Done Sent Done false
31569 9986 Acked Read 2 Done Sent Done false
31570 9987 Acked Read 1 Done Sent Done false
31571 9988 Acked Flush 0 Done Sent Done false
31572 9989 Acked Write 8 Done Sent Done false
31573 9990 Acked Read 5 Done Sent Done false
31574 9991 Acked Read 8 Done Sent Done false
31575 9992 Acked Read 9 Done Sent Done false
31576 9993 Acked Write 2 Done Sent Done false
31577 9994 Acked Flush 0 Done Sent Done false
31578 9995 Acked Write 5 Done Sent Done false
31579 9996 Acked Read 9 Done Sent Done false
31580 9997 Acked Write 9 Done Sent Done false
31581 9998 Acked Read 1 Done Sent Done false
31582 9999 Acked Read 2 Done Sent Done false
31583 10000 Acked Flush 0 Done Sent Done false
31584 10001 Acked Flush 0 Done Sent Done false
31585 10002 Acked Read 3 Done Sent Done false
31586 10003 Acked Write 7 Done Sent Done false
31587 10004 Acked Write 6 Done Sent Done false
31588 10005 Acked Write 6 Done Sent Done false
31589 10006 Acked Write 10 Done Sent Done false
31590 10007 Acked Flush 0 Done Sent Done false
31591 10008 Acked Write 1 Done Sent Done false
31592 10009 Acked Read 4 Done Sent Done false
31593 10010 Acked Read 8 Done Sent Done false
31594 10011 Acked Read 4 Done Sent Done false
31595 10012 Acked Flush 0 Done Sent Done false
31596 10013 Acked Write 8 Done Sent Done false
31597 10014 Acked Read 5 Done Sent Done false
31598 10015 Acked Read 3 Done Sent Done false
31599 10016 Acked Write 10 Done Sent Done false
31600 10017 Acked Read 4 Done Sent Done false
31601 10018 Acked Write 6 Done Sent Done false
31602 10019 Acked Write 4 Done Sent Done false
31603 10020 Acked Write 9 Done Sent Done false
31604 10021 Acked Read 8 Done Sent Done false
31605 10022 Acked Flush 0 Done Sent Done false
31606 10023 Acked Write 8 Done Sent Done false
31607 10024 Acked Write 7 Done Sent Done false
31608 10025 Acked Flush 0 Done Sent Done false
31609 10026 Acked Write 5 Done Sent Done false
31610 10027 Acked Read 7 Done Sent Done false
31611 10028 Acked Read 4 Done Sent Done false
31612 10029 Acked Write 7 Done Sent Done false
31613 10030 Acked Read 6 Done Sent Done false
31614 10031 Acked Read 7 Done Sent Done false
31615 10032 Acked Read 5 Done Sent Done false
31616 10033 Acked Write 3 Done Sent Done false
31617 10034 Acked Flush 0 Done Sent Done false
31618 10035 Acked Write 1 Done Sent Done false
31619 10036 Acked Read 4 Done Sent Done false
31620 10037 Acked Write 4 Done Sent Done false
31621 10038 Acked Read 5 Done Sent Done false
31622 10039 Acked Read 2 Done Sent Done false
31623 10040 Acked Read 6 Done Sent Done false
31624 10041 Acked Flush 0 Done Sent Done false
31625 10042 Acked Write 2 Done Sent Done false
31626 10043 Acked Read 4 Done Sent Done false
31627 10044 Acked Write 8 Done Sent Done false
31628 10045 Acked Read 6 Done Sent Done false
31629 10046 Acked Flush 0 Done Sent Done false
31630 10047 Acked Read 9 Done Sent Done false
31631 10048 Acked Write 6 Done Sent Done false
31632 10049 Acked Write 6 Done Sent Done false
31633 10050 Acked Read 6 Done Sent Done false
31634 10051 Acked Read 3 Done Sent Done false
31635 10052 Acked Read 8 Done Sent Done false
31636 10053 Acked Read 4 Done Sent Done false
31637 10054 Acked Read 3 Done Sent Done false
31638 10055 Acked Write 5 Done Sent Done false
31639 10056 Acked Read 8 Done Sent Done false
31640 STATES DS:0 DS:1 DS:2 TOTAL
31641 Sent 0 301 0 301
31642 Done 301 0 301 602
31643 Skipped 0 0 0 0
31644 Error 0 0 0 0
31645 Last Flush: 10046 9755 10046
31646 Downstairs last five completed: 9755 9754 9753 9752 9751
31647 Upstairs last five completed: 10056 10055 10054 10053 10052
31648 CLIENT: Up:0 ds:301 act:2
31649 ----------------------------------------------------------------
31650 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31651 States: Active Offline Active
31652 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31653 9756 Acked Read 2 Done Sent Done false
31654 9757 Acked Write 10 Done Sent Done false
31655 9758 Acked Read 10 Done Sent Done false
31656 9759 Acked Write 7 Done Sent Done false
31657 9760 Acked Read 1 Done Sent Done false
31658 9761 Acked Read 3 Done Sent Done false
31659 9762 Acked Read 9 Done Sent Done false
31660 9763 Acked Read 4 Done Sent Done false
31661 9764 Acked Write 9 Done Sent Done false
31662 9765 Acked Read 10 Done Sent Done false
31663 9766 Acked Write 6 Done Sent Done false
31664 9767 Acked Read 6 Done Sent Done false
31665 9768 Acked Read 8 Done Sent Done false
31666 9769 Acked Read 5 Done Sent Done false
31667 9770 Acked Flush 0 Done Sent Done false
31668 9771 Acked Read 7 Done Sent Done false
31669 9772 Acked Write 5 Done Sent Done false
31670 9773 Acked Write 8 Done Sent Done false
31671 9774 Acked Flush 0 Done Sent Done false
31672 9775 Acked Read 5 Done Sent Done false
31673 9776 Acked Write 8 Done Sent Done false
31674 9777 Acked Read 9 Done Sent Done false
31675 9778 Acked Write 4 Done Sent Done false
31676 9779 Acked Read 8 Done Sent Done false
31677 9780 Acked Write 7 Done Sent Done false
31678 9781 Acked Write 7 Done Sent Done false
31679 9782 Acked Write 3 Done Sent Done false
31680 9783 Acked Read 2 Done Sent Done false
31681 9784 Acked Read 3 Done Sent Done false
31682 9785 Acked Write 7 Done Sent Done false
31683 9786 Acked Write 9 Done Sent Done false
31684 9787 Acked Read 8 Done Sent Done false
31685 9788 Acked Read 3 Done Sent Done false
31686 9789 Acked Write 10 Done Sent Done false
31687 9790 Acked Write 10 Done Sent Done false
31688 9791 Acked Read 9 Done Sent Done false
31689 9792 Acked Write 5 Done Sent Done false
31690 9793 Acked Write 10 Done Sent Done false
31691 9794 Acked Write 1 Done Sent Done false
31692 9795 Acked Flush 0 Done Sent Done false
31693 9796 Acked Read 4 Done Sent Done false
31694 9797 Acked Read 8 Done Sent Done false
31695 9798 Acked Write 9 Done Sent Done false
31696 9799 Acked Write 7 Done Sent Done false
31697 9800 Acked Write 9 Done Sent Done false
31698 9801 Acked Read 7 Done Sent Done false
31699 9802 Acked Read 5 Done Sent Done false
31700 9803 Acked Flush 0 Done Sent Done false
31701 9804 Acked Write 7 Done Sent Done false
31702 9805 Acked Read 1 Done Sent Done false
31703 9806 Acked Read 10 Done Sent Done false
31704 9807 Acked Read 2 Done Sent Done false
31705 9808 Acked Read 9 Done Sent Done false
31706 9809 Acked Write 8 Done Sent Done false
31707 9810 Acked Read 7 Done Sent Done false
31708 9811 Acked Read 4 Done Sent Done false
31709 9812 Acked Read 7 Done Sent Done false
31710 9813 Acked Write 2 Done Sent Done false
31711 9814 Acked Read 1 Done Sent Done false
31712 9815 Acked Write 6 Done Sent Done false
31713 9816 Acked Write 9 Done Sent Done false
31714 9817 Acked Write 1 Done Sent Done false
31715 9818 Acked Write 3 Done Sent Done false
31716 9819 Acked Read 4 Done Sent Done false
31717 9820 Acked Write 10 Done Sent Done false
31718 9821 Acked Flush 0 Done Sent Done false
31719 9822 Acked Write 5 Done Sent Done false
31720 9823 Acked Write 4 Done Sent Done false
31721 9824 Acked Read 4 Done Sent Done false
31722 9825 Acked Read 4 Done Sent Done false
31723 9826 Acked Write 3 Done Sent Done false
31724 9827 Acked Read 2 Done Sent Done false
31725 9828 Acked Read 10 Done Sent Done false
31726 9829 Acked Read 8 Done Sent Done false
31727 9830 Acked Write 2 Done Sent Done false
31728 9831 Acked Write 9 Done Sent Done false
31729 9832 Acked Write 6 Done Sent Done false
31730 9833 Acked Write 1 Done Sent Done false
31731 9834 Acked Read 6 Done Sent Done false
31732 9835 Acked Read 4 Done Sent Done false
31733 9836 Acked Read 3 Done Sent Done false
31734 9837 Acked Read 9 Done Sent Done false
31735 9838 Acked Write 4 Done Sent Done false
31736 9839 Acked Read 6 Done Sent Done false
31737 9840 Acked Write 8 Done Sent Done false
31738 9841 Acked Write 9 Done Sent Done false
31739 9842 Acked Write 5 Done Sent Done false
31740 9843 Acked Read 2 Done Sent Done false
31741 9844 Acked Write 4 Done Sent Done false
31742 9845 Acked Write 2 Done Sent Done false
31743 9846 Acked Write 3 Done Sent Done false
31744 9847 Acked Flush 0 Done Sent Done false
31745 9848 Acked Read 2 Done Sent Done false
31746 9849 Acked Read 4 Done Sent Done false
31747 9850 Acked Write 2 Done Sent Done false
31748 9851 Acked Read 5 Done Sent Done false
31749 9852 Acked Read 2 Done Sent Done false
31750 9853 Acked Write 8 Done Sent Done false
31751 9854 Acked Write 5 Done Sent Done false
31752 9855 Acked Write 5 Done Sent Done false
31753 9856 Acked Read 9 Done Sent Done false
31754 9857 Acked Read 9 Done Sent Done false
31755 9858 Acked Read 9 Done Sent Done false
31756 9859 Acked Read 10 Done Sent Done false
31757 9860 Acked Read 2 Done Sent Done false
31758 9861 Acked Read 2 Done Sent Done false
31759 9862 Acked Read 9 Done Sent Done false
31760 9863 Acked Read 7 Done Sent Done false
31761 9864 Acked Read 5 Done Sent Done false
31762 9865 Acked Read 10 Done Sent Done false
31763 9866 Acked Write 6 Done Sent Done false
31764 9867 Acked Read 8 Done Sent Done false
31765 9868 Acked Write 5 Done Sent Done false
31766 9869 Acked Flush 0 Done Sent Done false
31767 9870 Acked Write 5 Done Sent Done false
31768 9871 Acked Write 9 Done Sent Done false
31769 9872 Acked Read 2 Done Sent Done false
31770 9873 Acked Write 5 Done Sent Done false
31771 9874 Acked Write 3 Done Sent Done false
31772 9875 Acked Read 8 Done Sent Done false
31773 9876 Acked Read 7 Done Sent Done false
31774 9877 Acked Write 2 Done Sent Done false
31775 9878 Acked Read 4 Done Sent Done false
31776 9879 Acked Write 4 Done Sent Done false
31777 9880 Acked Read 2 Done Sent Done false
31778 9881 Acked Read 10 Done Sent Done false
31779 9882 Acked Read 7 Done Sent Done false
31780 9883 Acked Read 1 Done Sent Done false
31781 9884 Acked Write 10 Done Sent Done false
31782 9885 Acked Flush 0 Done Sent Done false
31783 9886 Acked Read 4 Done Sent Done false
31784 9887 Acked Read 6 Done Sent Done false
31785 9888 Acked Read 5 Done Sent Done false
31786 9889 Acked Read 9 Done Sent Done false
31787 9890 Acked Flush 0 Done Sent Done false
31788 9891 Acked Read 1 Done Sent Done false
31789 9892 Acked Read 2 Done Sent Done false
31790 9893 Acked Write 6 Done Sent Done false
31791 9894 Acked Read 2 Done Sent Done false
31792 9895 Acked Write 3 Done Sent Done false
31793 9896 Acked Read 1 Done Sent Done false
31794 9897 Acked Read 10 Done Sent Done false
31795 9898 Acked Read 10 Done Sent Done false
31796 9899 Acked Write 6 Done Sent Done false
31797 9900 Acked Write 6 Done Sent Done false
31798 9901 Acked Read 4 Done Sent Done false
31799 9902 Acked Write 5 Done Sent Done false
31800 9903 Acked Flush 0 Done Sent Done false
31801 9904 Acked Read 7 Done Sent Done false
31802 9905 Acked Flush 0 Done Sent Done false
31803 9906 Acked Read 9 Done Sent Done false
31804 9907 Acked Read 10 Done Sent Done false
31805 9908 Acked Read 3 Done Sent Done false
31806 9909 Acked Read 2 Done Sent Done false
31807 9910 Acked Write 3 Done Sent Done false
31808 9911 Acked Write 5 Done Sent Done false
31809 9912 Acked Read 5 Done Sent Done false
31810 9913 Acked Read 6 Done Sent Done false
31811 9914 Acked Read 1 Done Sent Done false
31812 9915 Acked Read 7 Done Sent Done false
31813 9916 Acked Flush 0 Done Sent Done false
31814 9917 Acked Read 9 Done Sent Done false
31815 9918 Acked Read 1 Done Sent Done false
31816 9919 Acked Read 8 Done Sent Done false
31817 9920 Acked Flush 0 Done Sent Done false
31818 9921 Acked Write 4 Done Sent Done false
31819 9922 Acked Read 9 Done Sent Done false
31820 9923 Acked Read 4 Done Sent Done false
31821 9924 Acked Read 3 Done Sent Done false
31822 9925 Acked Read 10 Done Sent Done false
31823 9926 Acked Read 3 Done Sent Done false
31824 9927 Acked Write 9 Done Sent Done false
31825 9928 Acked Read 9 Done Sent Done false
31826 9929 Acked Flush 0 Done Sent Done false
31827 9930 Acked Read 2 Done Sent Done false
31828 9931 Acked Write 2 Done Sent Done false
31829 9932 Acked Flush 0 Done Sent Done false
31830 9933 Acked Write 2 Done Sent Done false
31831 9934 Acked Write 8 Done Sent Done false
31832 9935 Acked Write 1 Done Sent Done false
31833 9936 Acked Read 7 Done Sent Done false
31834 9937 Acked Write 1 Done Sent Done false
31835 9938 Acked Read 8 Done Sent Done false
31836 9939 Acked Flush 0 Done Sent Done false
31837 9940 Acked Read 2 Done Sent Done false
31838 9941 Acked Flush 0 Done Sent Done false
31839 9942 Acked Read 9 Done Sent Done false
31840 9943 Acked Read 9 Done Sent Done false
31841 9944 Acked Write 6 Done Sent Done false
31842 9945 Acked Read 1 Done Sent Done false
31843 9946 Acked Write 5 Done Sent Done false
31844 9947 Acked Read 1 Done Sent Done false
31845 9948 Acked Read 3 Done Sent Done false
31846 9949 Acked Write 7 Done Sent Done false
31847 9950 Acked Read 7 Done Sent Done false
31848 9951 Acked Read 5 Done Sent Done false
31849 9952 Acked Write 10 Done Sent Done false
31850 9953 Acked Read 9 Done Sent Done false
31851 9954 Acked Read 2 Done Sent Done false
31852 9955 Acked Write 6 Done Sent Done false
31853 9956 Acked Read 4 Done Sent Done false
31854 9957 Acked Write 10 Done Sent Done false
31855 9958 Acked Flush 0 Done Sent Done false
31856 9959 Acked Write 1 Done Sent Done false
31857 9960 Acked Read 5 Done Sent Done false
31858 9961 Acked Write 6 Done Sent Done false
31859 9962 Acked Flush 0 Done Sent Done false
31860 9963 Acked Read 4 Done Sent Done false
31861 9964 Acked Write 7 Done Sent Done false
31862 9965 Acked Write 4 Done Sent Done false
31863 9966 Acked Read 10 Done Sent Done false
31864 9967 Acked Read 5 Done Sent Done false
31865 9968 Acked Read 8 Done Sent Done false
31866 9969 Acked Write 6 Done Sent Done false
31867 9970 Acked Read 3 Done Sent Done false
31868 9971 Acked Read 3 Done Sent Done false
31869 9972 Acked Read 9 Done Sent Done false
31870 9973 Acked Write 10 Done Sent Done false
31871 9974 Acked Write 7 Done Sent Done false
31872 9975 Acked Read 7 Done Sent Done false
31873 9976 Acked Write 7 Done Sent Done false
31874 9977 Acked Write 7 Done Sent Done false
31875 9978 Acked Read 7 Done Sent Done false
31876 9979 Acked Read 3 Done Sent Done false
31877 9980 Acked Write 1 Done Sent Done false
31878 9981 Acked Write 6 Done Sent Done false
31879 9982 Acked Read 10 Done Sent Done false
31880 9983 Acked Read 4 Done Sent Done false
31881 9984 Acked Write 7 Done Sent Done false
31882 9985 Acked Read 1 Done Sent Done false
31883 9986 Acked Read 2 Done Sent Done false
31884 9987 Acked Read 1 Done Sent Done false
31885 9988 Acked Flush 0 Done Sent Done false
31886 9989 Acked Write 8 Done Sent Done false
31887 9990 Acked Read 5 Done Sent Done false
31888 9991 Acked Read 8 Done Sent Done false
31889 9992 Acked Read 9 Done Sent Done false
31890 9993 Acked Write 2 Done Sent Done false
31891 9994 Acked Flush 0 Done Sent Done false
31892 9995 Acked Write 5 Done Sent Done false
31893 9996 Acked Read 9 Done Sent Done false
31894 9997 Acked Write 9 Done Sent Done false
31895 9998 Acked Read 1 Done Sent Done false
31896 9999 Acked Read 2 Done Sent Done false
31897 10000 Acked Flush 0 Done Sent Done false
31898 10001 Acked Flush 0 Done Sent Done false
31899 10002 Acked Read 3 Done Sent Done false
31900 10003 Acked Write 7 Done Sent Done false
31901 10004 Acked Write 6 Done Sent Done false
31902 10005 Acked Write 6 Done Sent Done false
31903 10006 Acked Write 10 Done Sent Done false
31904 10007 Acked Flush 0 Done Sent Done false
31905 10008 Acked Write 1 Done Sent Done false
31906 10009 Acked Read 4 Done Sent Done false
31907 10010 Acked Read 8 Done Sent Done false
31908 10011 Acked Read 4 Done Sent Done false
31909 10012 Acked Flush 0 Done Sent Done false
31910 10013 Acked Write 8 Done Sent Done false
31911 10014 Acked Read 5 Done Sent Done false
31912 10015 Acked Read 3 Done Sent Done false
31913 10016 Acked Write 10 Done Sent Done false
31914 10017 Acked Read 4 Done Sent Done false
31915 10018 Acked Write 6 Done Sent Done false
31916 10019 Acked Write 4 Done Sent Done false
31917 10020 Acked Write 9 Done Sent Done false
31918 10021 Acked Read 8 Done Sent Done false
31919 10022 Acked Flush 0 Done Sent Done false
31920 10023 Acked Write 8 Done Sent Done false
31921 10024 Acked Write 7 Done Sent Done false
31922 10025 Acked Flush 0 Done Sent Done false
31923 10026 Acked Write 5 Done Sent Done false
31924 10027 Acked Read 7 Done Sent Done false
31925 10028 Acked Read 4 Done Sent Done false
31926 10029 Acked Write 7 Done Sent Done false
31927 10030 Acked Read 6 Done Sent Done false
31928 10031 Acked Read 7 Done Sent Done false
31929 10032 Acked Read 5 Done Sent Done false
31930 10033 Acked Write 3 Done Sent Done false
31931 10034 Acked Flush 0 Done Sent Done false
31932 10035 Acked Write 1 Done Sent Done false
31933 10036 Acked Read 4 Done Sent Done false
31934 10037 Acked Write 4 Done Sent Done false
31935 10038 Acked Read 5 Done Sent Done false
31936 10039 Acked Read 2 Done Sent Done false
31937 10040 Acked Read 6 Done Sent Done false
31938 10041 Acked Flush 0 Done Sent Done false
31939 10042 Acked Write 2 Done Sent Done false
31940 10043 Acked Read 4 Done Sent Done false
31941 10044 Acked Write 8 Done Sent Done false
31942 10045 Acked Read 6 Done Sent Done false
31943 10046 Acked Flush 0 Done Sent Done false
31944 10047 Acked Read 9 Done Sent Done false
31945 10048 Acked Write 6 Done Sent Done false
31946 10049 Acked Write 6 Done Sent Done false
31947 10050 Acked Read 6 Done Sent Done false
31948 10051 Acked Read 3 Done Sent Done false
31949 10052 Acked Read 8 Done Sent Done false
31950 10053 Acked Read 4 Done Sent Done false
31951 10054 Acked Read 3 Done Sent Done false
31952 10055 Acked Write 5 Done Sent Done false
31953 10056 Acked Read 8 Done Sent Done false
31954 10057 Acked Flush 0 Done Sent Done false
31955 STATES DS:0 DS:1 DS:2 TOTAL
31956 Sent 0 302 0 302
31957 Done 302 0 302 604
31958 Skipped 0 0 0 0
31959 Error 0 0 0 0
31960 Last Flush: 10057 9755 10057
31961 Downstairs last five completed: 9755 9754 9753 9752 9751
31962 Upstairs last five completed: 10057 10056 10055 10054 10053
31963 CLIENT: Up:0 ds:302 act:2
319642025-01-11T06:12:07.807ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
319652025-01-11T06:12:07.807ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
319662025-01-11T06:12:07.807ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
319672025-01-11T06:12:07.807ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
319682025-01-11T06:12:07.808ZINFOcrucible: client task is exiting = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
31969 [8820] got start action
31970 [8820] I am going to restart
31971 Starting downstairs at port 8820
31972 Make output file at "/tmp/dsc/downstairs-8820.txt"
31973 [8820][1] reports Starting
31974 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1234)
31975 [8820][1] reports Running
31976 ----------------------------------------------------------------
31977 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31978 States: Active Offline Active
31979 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31980 9756 Acked Read 2 Done Sent Done false
31981 9757 Acked Write 10 Done Sent Done false
31982 9758 Acked Read 10 Done Sent Done false
31983 9759 Acked Write 7 Done Sent Done false
31984 9760 Acked Read 1 Done Sent Done false
31985 9761 Acked Read 3 Done Sent Done false
31986 9762 Acked Read 9 Done Sent Done false
31987 9763 Acked Read 4 Done Sent Done false
31988 9764 Acked Write 9 Done Sent Done false
31989 9765 Acked Read 10 Done Sent Done false
31990 9766 Acked Write 6 Done Sent Done false
31991 9767 Acked Read 6 Done Sent Done false
31992 9768 Acked Read 8 Done Sent Done false
31993 9769 Acked Read 5 Done Sent Done false
31994 9770 Acked Flush 0 Done Sent Done false
31995 9771 Acked Read 7 Done Sent Done false
31996 9772 Acked Write 5 Done Sent Done false
31997 9773 Acked Write 8 Done Sent Done false
31998 9774 Acked Flush 0 Done Sent Done false
31999 9775 Acked Read 5 Done Sent Done false
32000 9776 Acked Write 8 Done Sent Done false
32001 9777 Acked Read 9 Done Sent Done false
32002 9778 Acked Write 4 Done Sent Done false
32003 9779 Acked Read 8 Done Sent Done false
32004 9780 Acked Write 7 Done Sent Done false
32005 9781 Acked Write 7 Done Sent Done false
32006 9782 Acked Write 3 Done Sent Done false
32007 9783 Acked Read 2 Done Sent Done false
32008 9784 Acked Read 3 Done Sent Done false
32009 9785 Acked Write 7 Done Sent Done false
32010 9786 Acked Write 9 Done Sent Done false
32011 9787 Acked Read 8 Done Sent Done false
32012 9788 Acked Read 3 Done Sent Done false
32013 9789 Acked Write 10 Done Sent Done false
32014 9790 Acked Write 10 Done Sent Done false
32015 9791 Acked Read 9 Done Sent Done false
32016 9792 Acked Write 5 Done Sent Done false
32017 9793 Acked Write 10 Done Sent Done false
32018 9794 Acked Write 1 Done Sent Done false
32019 9795 Acked Flush 0 Done Sent Done false
32020 9796 Acked Read 4 Done Sent Done false
32021 9797 Acked Read 8 Done Sent Done false
32022 9798 Acked Write 9 Done Sent Done false
32023 9799 Acked Write 7 Done Sent Done false
32024 9800 Acked Write 9 Done Sent Done false
32025 9801 Acked Read 7 Done Sent Done false
32026 9802 Acked Read 5 Done Sent Done false
32027 9803 Acked Flush 0 Done Sent Done false
32028 9804 Acked Write 7 Done Sent Done false
32029 9805 Acked Read 1 Done Sent Done false
32030 9806 Acked Read 10 Done Sent Done false
32031 9807 Acked Read 2 Done Sent Done false
32032 9808 Acked Read 9 Done Sent Done false
32033 9809 Acked Write 8 Done Sent Done false
32034 9810 Acked Read 7 Done Sent Done false
32035 9811 Acked Read 4 Done Sent Done false
32036 9812 Acked Read 7 Done Sent Done false
32037 9813 Acked Write 2 Done Sent Done false
32038 9814 Acked Read 1 Done Sent Done false
32039 9815 Acked Write 6 Done Sent Done false
32040 9816 Acked Write 9 Done Sent Done false
32041 9817 Acked Write 1 Done Sent Done false
32042 9818 Acked Write 3 Done Sent Done false
32043 9819 Acked Read 4 Done Sent Done false
32044 9820 Acked Write 10 Done Sent Done false
32045 9821 Acked Flush 0 Done Sent Done false
32046 9822 Acked Write 5 Done Sent Done false
32047 9823 Acked Write 4 Done Sent Done false
32048 9824 Acked Read 4 Done Sent Done false
32049 9825 Acked Read 4 Done Sent Done false
32050 9826 Acked Write 3 Done Sent Done false
32051 9827 Acked Read 2 Done Sent Done false
32052 9828 Acked Read 10 Done Sent Done false
32053 9829 Acked Read 8 Done Sent Done false
32054 9830 Acked Write 2 Done Sent Done false
32055 9831 Acked Write 9 Done Sent Done false
32056 9832 Acked Write 6 Done Sent Done false
32057 9833 Acked Write 1 Done Sent Done false
32058 9834 Acked Read 6 Done Sent Done false
32059 9835 Acked Read 4 Done Sent Done false
32060 9836 Acked Read 3 Done Sent Done false
32061 9837 Acked Read 9 Done Sent Done false
32062 9838 Acked Write 4 Done Sent Done false
32063 9839 Acked Read 6 Done Sent Done false
32064 9840 Acked Write 8 Done Sent Done false
32065 9841 Acked Write 9 Done Sent Done false
32066 9842 Acked Write 5 Done Sent Done false
32067 9843 Acked Read 2 Done Sent Done false
32068 9844 Acked Write 4 Done Sent Done false
32069 9845 Acked Write 2 Done Sent Done false
32070 9846 Acked Write 3 Done Sent Done false
32071 9847 Acked Flush 0 Done Sent Done false
32072 9848 Acked Read 2 Done Sent Done false
32073 9849 Acked Read 4 Done Sent Done false
32074 9850 Acked Write 2 Done Sent Done false
32075 9851 Acked Read 5 Done Sent Done false
32076 9852 Acked Read 2 Done Sent Done false
32077 9853 Acked Write 8 Done Sent Done false
32078 9854 Acked Write 5 Done Sent Done false
32079 9855 Acked Write 5 Done Sent Done false
32080 9856 Acked Read 9 Done Sent Done false
32081 9857 Acked Read 9 Done Sent Done false
32082 9858 Acked Read 9 Done Sent Done false
32083 9859 Acked Read 10 Done Sent Done false
32084 9860 Acked Read 2 Done Sent Done false
32085 9861 Acked Read 2 Done Sent Done false
32086 9862 Acked Read 9 Done Sent Done false
32087 9863 Acked Read 7 Done Sent Done false
32088 9864 Acked Read 5 Done Sent Done false
32089 9865 Acked Read 10 Done Sent Done false
32090 9866 Acked Write 6 Done Sent Done false
32091 9867 Acked Read 8 Done Sent Done false
32092 9868 Acked Write 5 Done Sent Done false
32093 9869 Acked Flush 0 Done Sent Done false
32094 9870 Acked Write 5 Done Sent Done false
32095 9871 Acked Write 9 Done Sent Done false
32096 9872 Acked Read 2 Done Sent Done false
32097 9873 Acked Write 5 Done Sent Done false
32098 9874 Acked Write 3 Done Sent Done false
32099 9875 Acked Read 8 Done Sent Done false
32100 9876 Acked Read 7 Done Sent Done false
32101 9877 Acked Write 2 Done Sent Done false
32102 9878 Acked Read 4 Done Sent Done false
32103 9879 Acked Write 4 Done Sent Done false
32104 9880 Acked Read 2 Done Sent Done false
32105 9881 Acked Read 10 Done Sent Done false
32106 9882 Acked Read 7 Done Sent Done false
32107 9883 Acked Read 1 Done Sent Done false
32108 9884 Acked Write 10 Done Sent Done false
32109 9885 Acked Flush 0 Done Sent Done false
32110 9886 Acked Read 4 Done Sent Done false
32111 9887 Acked Read 6 Done Sent Done false
32112 9888 Acked Read 5 Done Sent Done false
32113 9889 Acked Read 9 Done Sent Done false
32114 9890 Acked Flush 0 Done Sent Done false
32115 9891 Acked Read 1 Done Sent Done false
32116 9892 Acked Read 2 Done Sent Done false
32117 9893 Acked Write 6 Done Sent Done false
32118 9894 Acked Read 2 Done Sent Done false
32119 9895 Acked Write 3 Done Sent Done false
32120 9896 Acked Read 1 Done Sent Done false
32121 9897 Acked Read 10 Done Sent Done false
32122 9898 Acked Read 10 Done Sent Done false
32123 9899 Acked Write 6 Done Sent Done false
32124 9900 Acked Write 6 Done Sent Done false
32125 9901 Acked Read 4 Done Sent Done false
32126 9902 Acked Write 5 Done Sent Done false
32127 9903 Acked Flush 0 Done Sent Done false
32128 9904 Acked Read 7 Done Sent Done false
32129 9905 Acked Flush 0 Done Sent Done false
32130 9906 Acked Read 9 Done Sent Done false
32131 9907 Acked Read 10 Done Sent Done false
32132 9908 Acked Read 3 Done Sent Done false
32133 9909 Acked Read 2 Done Sent Done false
32134 9910 Acked Write 3 Done Sent Done false
32135 9911 Acked Write 5 Done Sent Done false
32136 9912 Acked Read 5 Done Sent Done false
32137 9913 Acked Read 6 Done Sent Done false
32138 9914 Acked Read 1 Done Sent Done false
32139 9915 Acked Read 7 Done Sent Done false
32140 9916 Acked Flush 0 Done Sent Done false
32141 9917 Acked Read 9 Done Sent Done false
32142 9918 Acked Read 1 Done Sent Done false
32143 9919 Acked Read 8 Done Sent Done false
32144 9920 Acked Flush 0 Done Sent Done false
32145 9921 Acked Write 4 Done Sent Done false
32146 9922 Acked Read 9 Done Sent Done false
32147 9923 Acked Read 4 Done Sent Done false
32148 9924 Acked Read 3 Done Sent Done false
32149 9925 Acked Read 10 Done Sent Done false
32150 9926 Acked Read 3 Done Sent Done false
32151 9927 Acked Write 9 Done Sent Done false
32152 9928 Acked Read 9 Done Sent Done false
32153 9929 Acked Flush 0 Done Sent Done false
32154 9930 Acked Read 2 Done Sent Done false
32155 9931 Acked Write 2 Done Sent Done false
32156 9932 Acked Flush 0 Done Sent Done false
32157 9933 Acked Write 2 Done Sent Done false
32158 9934 Acked Write 8 Done Sent Done false
32159 9935 Acked Write 1 Done Sent Done false
32160 9936 Acked Read 7 Done Sent Done false
32161 9937 Acked Write 1 Done Sent Done false
32162 9938 Acked Read 8 Done Sent Done false
32163 9939 Acked Flush 0 Done Sent Done false
32164 9940 Acked Read 2 Done Sent Done false
32165 9941 Acked Flush 0 Done Sent Done false
32166 9942 Acked Read 9 Done Sent Done false
32167 9943 Acked Read 9 Done Sent Done false
32168 9944 Acked Write 6 Done Sent Done false
32169 9945 Acked Read 1 Done Sent Done false
32170 9946 Acked Write 5 Done Sent Done false
32171 9947 Acked Read 1 Done Sent Done false
32172 9948 Acked Read 3 Done Sent Done false
32173 9949 Acked Write 7 Done Sent Done false
32174 9950 Acked Read 7 Done Sent Done false
32175 9951 Acked Read 5 Done Sent Done false
32176 9952 Acked Write 10 Done Sent Done false
32177 9953 Acked Read 9 Done Sent Done false
32178 9954 Acked Read 2 Done Sent Done false
32179 9955 Acked Write 6 Done Sent Done false
32180 9956 Acked Read 4 Done Sent Done false
32181 9957 Acked Write 10 Done Sent Done false
32182 9958 Acked Flush 0 Done Sent Done false
32183 9959 Acked Write 1 Done Sent Done false
32184 9960 Acked Read 5 Done Sent Done false
32185 9961 Acked Write 6 Done Sent Done false
32186 9962 Acked Flush 0 Done Sent Done false
32187 9963 Acked Read 4 Done Sent Done false
32188 9964 Acked Write 7 Done Sent Done false
32189 9965 Acked Write 4 Done Sent Done false
32190 9966 Acked Read 10 Done Sent Done false
32191 9967 Acked Read 5 Done Sent Done false
32192 9968 Acked Read 8 Done Sent Done false
32193 9969 Acked Write 6 Done Sent Done false
32194 9970 Acked Read 3 Done Sent Done false
32195 9971 Acked Read 3 Done Sent Done false
32196 9972 Acked Read 9 Done Sent Done false
32197 9973 Acked Write 10 Done Sent Done false
32198 9974 Acked Write 7 Done Sent Done false
32199 9975 Acked Read 7 Done Sent Done false
32200 9976 Acked Write 7 Done Sent Done false
32201 9977 Acked Write 7 Done Sent Done false
32202 9978 Acked Read 7 Done Sent Done false
32203 9979 Acked Read 3 Done Sent Done false
32204 9980 Acked Write 1 Done Sent Done false
32205 9981 Acked Write 6 Done Sent Done false
32206 9982 Acked Read 10 Done Sent Done false
32207 9983 Acked Read 4 Done Sent Done false
32208 9984 Acked Write 7 Done Sent Done false
32209 9985 Acked Read 1 Done Sent Done false
32210 9986 Acked Read 2 Done Sent Done false
32211 9987 Acked Read 1 Done Sent Done false
32212 9988 Acked Flush 0 Done Sent Done false
32213 9989 Acked Write 8 Done Sent Done false
32214 9990 Acked Read 5 Done Sent Done false
32215 9991 Acked Read 8 Done Sent Done false
32216 9992 Acked Read 9 Done Sent Done false
32217 9993 Acked Write 2 Done Sent Done false
32218 9994 Acked Flush 0 Done Sent Done false
32219 9995 Acked Write 5 Done Sent Done false
32220 9996 Acked Read 9 Done Sent Done false
32221 9997 Acked Write 9 Done Sent Done false
32222 9998 Acked Read 1 Done Sent Done false
32223 9999 Acked Read 2 Done Sent Done false
32224 10000 Acked Flush 0 Done Sent Done false
32225 10001 Acked Flush 0 Done Sent Done false
32226 10002 Acked Read 3 Done Sent Done false
32227 10003 Acked Write 7 Done Sent Done false
32228 10004 Acked Write 6 Done Sent Done false
32229 10005 Acked Write 6 Done Sent Done false
32230 10006 Acked Write 10 Done Sent Done false
32231 10007 Acked Flush 0 Done Sent Done false
32232 10008 Acked Write 1 Done Sent Done false
32233 10009 Acked Read 4 Done Sent Done false
32234 10010 Acked Read 8 Done Sent Done false
32235 10011 Acked Read 4 Done Sent Done false
32236 10012 Acked Flush 0 Done Sent Done false
32237 10013 Acked Write 8 Done Sent Done false
32238 10014 Acked Read 5 Done Sent Done false
32239 10015 Acked Read 3 Done Sent Done false
32240 10016 Acked Write 10 Done Sent Done false
32241 10017 Acked Read 4 Done Sent Done false
32242 10018 Acked Write 6 Done Sent Done false
32243 10019 Acked Write 4 Done Sent Done false
32244 10020 Acked Write 9 Done Sent Done false
32245 10021 Acked Read 8 Done Sent Done false
32246 10022 Acked Flush 0 Done Sent Done false
32247 10023 Acked Write 8 Done Sent Done false
32248 10024 Acked Write 7 Done Sent Done false
32249 10025 Acked Flush 0 Done Sent Done false
32250 10026 Acked Write 5 Done Sent Done false
32251 10027 Acked Read 7 Done Sent Done false
32252 10028 Acked Read 4 Done Sent Done false
32253 10029 Acked Write 7 Done Sent Done false
32254 10030 Acked Read 6 Done Sent Done false
32255 10031 Acked Read 7 Done Sent Done false
32256 10032 Acked Read 5 Done Sent Done false
32257 10033 Acked Write 3 Done Sent Done false
32258 10034 Acked Flush 0 Done Sent Done false
32259 10035 Acked Write 1 Done Sent Done false
32260 10036 Acked Read 4 Done Sent Done false
32261 10037 Acked Write 4 Done Sent Done false
32262 10038 Acked Read 5 Done Sent Done false
32263 10039 Acked Read 2 Done Sent Done false
32264 10040 Acked Read 6 Done Sent Done false
32265 10041 Acked Flush 0 Done Sent Done false
32266 10042 Acked Write 2 Done Sent Done false
32267 10043 Acked Read 4 Done Sent Done false
32268 10044 Acked Write 8 Done Sent Done false
32269 10045 Acked Read 6 Done Sent Done false
32270 10046 Acked Flush 0 Done Sent Done false
32271 10047 Acked Read 9 Done Sent Done false
32272 10048 Acked Write 6 Done Sent Done false
32273 10049 Acked Write 6 Done Sent Done false
32274 10050 Acked Read 6 Done Sent Done false
32275 10051 Acked Read 3 Done Sent Done false
32276 10052 Acked Read 8 Done Sent Done false
32277 10053 Acked Read 4 Done Sent Done false
32278 10054 Acked Read 3 Done Sent Done false
32279 10055 Acked Write 5 Done Sent Done false
32280 10056 Acked Read 8 Done Sent Done false
32281 10057 Acked Flush 0 Done Sent Done false
32282 STATES DS:0 DS:1 DS:2 TOTAL
32283 Sent 0 302 0 302
32284 Done 302 0 302 604
32285 Skipped 0 0 0 0
32286 Error 0 0 0 0
32287 Last Flush: 10057 9755 10057
32288 Downstairs last five completed: 9755 9754 9753 9752 9751
32289 Upstairs last five completed: 10057 10056 10055 10054 10053
32290 CLIENT: Up:0 ds:302 act:2
32291 ----------------------------------------------------------------
32292 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32293 States: Active Offline Active
32294 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32295 9756 Acked Read 2 Done Sent Done false
32296 9757 Acked Write 10 Done Sent Done false
32297 9758 Acked Read 10 Done Sent Done false
32298 9759 Acked Write 7 Done Sent Done false
32299 9760 Acked Read 1 Done Sent Done false
32300 9761 Acked Read 3 Done Sent Done false
32301 9762 Acked Read 9 Done Sent Done false
32302 9763 Acked Read 4 Done Sent Done false
32303 9764 Acked Write 9 Done Sent Done false
32304 9765 Acked Read 10 Done Sent Done false
32305 9766 Acked Write 6 Done Sent Done false
32306 9767 Acked Read 6 Done Sent Done false
32307 9768 Acked Read 8 Done Sent Done false
32308 9769 Acked Read 5 Done Sent Done false
32309 9770 Acked Flush 0 Done Sent Done false
32310 9771 Acked Read 7 Done Sent Done false
32311 9772 Acked Write 5 Done Sent Done false
32312 9773 Acked Write 8 Done Sent Done false
32313 9774 Acked Flush 0 Done Sent Done false
32314 9775 Acked Read 5 Done Sent Done false
32315 9776 Acked Write 8 Done Sent Done false
32316 9777 Acked Read 9 Done Sent Done false
32317 9778 Acked Write 4 Done Sent Done false
32318 9779 Acked Read 8 Done Sent Done false
32319 9780 Acked Write 7 Done Sent Done false
32320 9781 Acked Write 7 Done Sent Done false
32321 9782 Acked Write 3 Done Sent Done false
32322 9783 Acked Read 2 Done Sent Done false
32323 9784 Acked Read 3 Done Sent Done false
32324 9785 Acked Write 7 Done Sent Done false
32325 9786 Acked Write 9 Done Sent Done false
32326 9787 Acked Read 8 Done Sent Done false
32327 9788 Acked Read 3 Done Sent Done false
32328 9789 Acked Write 10 Done Sent Done false
32329 9790 Acked Write 10 Done Sent Done false
32330 9791 Acked Read 9 Done Sent Done false
32331 9792 Acked Write 5 Done Sent Done false
32332 9793 Acked Write 10 Done Sent Done false
32333 9794 Acked Write 1 Done Sent Done false
32334 9795 Acked Flush 0 Done Sent Done false
32335 9796 Acked Read 4 Done Sent Done false
32336 9797 Acked Read 8 Done Sent Done false
32337 9798 Acked Write 9 Done Sent Done false
32338 9799 Acked Write 7 Done Sent Done false
32339 9800 Acked Write 9 Done Sent Done false
32340 9801 Acked Read 7 Done Sent Done false
32341 9802 Acked Read 5 Done Sent Done false
32342 9803 Acked Flush 0 Done Sent Done false
32343 9804 Acked Write 7 Done Sent Done false
32344 9805 Acked Read 1 Done Sent Done false
32345 9806 Acked Read 10 Done Sent Done false
32346 9807 Acked Read 2 Done Sent Done false
32347 9808 Acked Read 9 Done Sent Done false
32348 9809 Acked Write 8 Done Sent Done false
32349 9810 Acked Read 7 Done Sent Done false
32350 9811 Acked Read 4 Done Sent Done false
32351 9812 Acked Read 7 Done Sent Done false
32352 9813 Acked Write 2 Done Sent Done false
32353 9814 Acked Read 1 Done Sent Done false
32354 9815 Acked Write 6 Done Sent Done false
32355 9816 Acked Write 9 Done Sent Done false
32356 9817 Acked Write 1 Done Sent Done false
32357 9818 Acked Write 3 Done Sent Done false
32358 9819 Acked Read 4 Done Sent Done false
32359 9820 Acked Write 10 Done Sent Done false
32360 9821 Acked Flush 0 Done Sent Done false
32361 9822 Acked Write 5 Done Sent Done false
32362 9823 Acked Write 4 Done Sent Done false
32363 9824 Acked Read 4 Done Sent Done false
32364 9825 Acked Read 4 Done Sent Done false
32365 9826 Acked Write 3 Done Sent Done false
32366 9827 Acked Read 2 Done Sent Done false
32367 9828 Acked Read 10 Done Sent Done false
32368 9829 Acked Read 8 Done Sent Done false
32369 9830 Acked Write 2 Done Sent Done false
32370 9831 Acked Write 9 Done Sent Done false
32371 9832 Acked Write 6 Done Sent Done false
32372 9833 Acked Write 1 Done Sent Done false
32373 9834 Acked Read 6 Done Sent Done false
32374 9835 Acked Read 4 Done Sent Done false
32375 9836 Acked Read 3 Done Sent Done false
32376 9837 Acked Read 9 Done Sent Done false
32377 9838 Acked Write 4 Done Sent Done false
32378 9839 Acked Read 6 Done Sent Done false
32379 9840 Acked Write 8 Done Sent Done false
32380 9841 Acked Write 9 Done Sent Done false
32381 9842 Acked Write 5 Done Sent Done false
32382 9843 Acked Read 2 Done Sent Done false
32383 9844 Acked Write 4 Done Sent Done false
32384 9845 Acked Write 2 Done Sent Done false
32385 9846 Acked Write 3 Done Sent Done false
32386 9847 Acked Flush 0 Done Sent Done false
32387 9848 Acked Read 2 Done Sent Done false
32388 9849 Acked Read 4 Done Sent Done false
32389 9850 Acked Write 2 Done Sent Done false
32390 9851 Acked Read 5 Done Sent Done false
32391 9852 Acked Read 2 Done Sent Done false
32392 9853 Acked Write 8 Done Sent Done false
32393 9854 Acked Write 5 Done Sent Done false
32394 9855 Acked Write 5 Done Sent Done false
32395 9856 Acked Read 9 Done Sent Done false
32396 9857 Acked Read 9 Done Sent Done false
32397 9858 Acked Read 9 Done Sent Done false
32398 9859 Acked Read 10 Done Sent Done false
32399 9860 Acked Read 2 Done Sent Done false
32400 9861 Acked Read 2 Done Sent Done false
32401 9862 Acked Read 9 Done Sent Done false
32402 9863 Acked Read 7 Done Sent Done false
32403 9864 Acked Read 5 Done Sent Done false
32404 9865 Acked Read 10 Done Sent Done false
32405 9866 Acked Write 6 Done Sent Done false
32406 9867 Acked Read 8 Done Sent Done false
32407 9868 Acked Write 5 Done Sent Done false
32408 9869 Acked Flush 0 Done Sent Done false
32409 9870 Acked Write 5 Done Sent Done false
32410 9871 Acked Write 9 Done Sent Done false
32411 9872 Acked Read 2 Done Sent Done false
32412 9873 Acked Write 5 Done Sent Done false
32413 9874 Acked Write 3 Done Sent Done false
32414 9875 Acked Read 8 Done Sent Done false
32415 9876 Acked Read 7 Done Sent Done false
32416 9877 Acked Write 2 Done Sent Done false
32417 9878 Acked Read 4 Done Sent Done false
32418 9879 Acked Write 4 Done Sent Done false
32419 9880 Acked Read 2 Done Sent Done false
32420 9881 Acked Read 10 Done Sent Done false
32421 9882 Acked Read 7 Done Sent Done false
32422 9883 Acked Read 1 Done Sent Done false
32423 9884 Acked Write 10 Done Sent Done false
32424 9885 Acked Flush 0 Done Sent Done false
32425 9886 Acked Read 4 Done Sent Done false
32426 9887 Acked Read 6 Done Sent Done false
32427 9888 Acked Read 5 Done Sent Done false
32428 9889 Acked Read 9 Done Sent Done false
32429 9890 Acked Flush 0 Done Sent Done false
32430 9891 Acked Read 1 Done Sent Done false
32431 9892 Acked Read 2 Done Sent Done false
32432 9893 Acked Write 6 Done Sent Done false
32433 9894 Acked Read 2 Done Sent Done false
32434 9895 Acked Write 3 Done Sent Done false
32435 9896 Acked Read 1 Done Sent Done false
32436 9897 Acked Read 10 Done Sent Done false
32437 9898 Acked Read 10 Done Sent Done false
32438 9899 Acked Write 6 Done Sent Done false
32439 9900 Acked Write 6 Done Sent Done false
32440 9901 Acked Read 4 Done Sent Done false
32441 9902 Acked Write 5 Done Sent Done false
32442 9903 Acked Flush 0 Done Sent Done false
32443 9904 Acked Read 7 Done Sent Done false
32444 9905 Acked Flush 0 Done Sent Done false
32445 9906 Acked Read 9 Done Sent Done false
32446 9907 Acked Read 10 Done Sent Done false
32447 9908 Acked Read 3 Done Sent Done false
32448 9909 Acked Read 2 Done Sent Done false
32449 9910 Acked Write 3 Done Sent Done false
32450 9911 Acked Write 5 Done Sent Done false
32451 9912 Acked Read 5 Done Sent Done false
32452 9913 Acked Read 6 Done Sent Done false
32453 9914 Acked Read 1 Done Sent Done false
32454 9915 Acked Read 7 Done Sent Done false
32455 9916 Acked Flush 0 Done Sent Done false
32456 9917 Acked Read 9 Done Sent Done false
32457 9918 Acked Read 1 Done Sent Done false
32458 9919 Acked Read 8 Done Sent Done false
32459 9920 Acked Flush 0 Done Sent Done false
32460 9921 Acked Write 4 Done Sent Done false
32461 9922 Acked Read 9 Done Sent Done false
32462 9923 Acked Read 4 Done Sent Done false
32463 9924 Acked Read 3 Done Sent Done false
32464 9925 Acked Read 10 Done Sent Done false
32465 9926 Acked Read 3 Done Sent Done false
32466 9927 Acked Write 9 Done Sent Done false
32467 9928 Acked Read 9 Done Sent Done false
32468 9929 Acked Flush 0 Done Sent Done false
32469 9930 Acked Read 2 Done Sent Done false
32470 9931 Acked Write 2 Done Sent Done false
32471 9932 Acked Flush 0 Done Sent Done false
32472 9933 Acked Write 2 Done Sent Done false
32473 9934 Acked Write 8 Done Sent Done false
32474 9935 Acked Write 1 Done Sent Done false
32475 9936 Acked Read 7 Done Sent Done false
32476 9937 Acked Write 1 Done Sent Done false
32477 9938 Acked Read 8 Done Sent Done false
32478 9939 Acked Flush 0 Done Sent Done false
32479 9940 Acked Read 2 Done Sent Done false
32480 9941 Acked Flush 0 Done Sent Done false
32481 9942 Acked Read 9 Done Sent Done false
32482 9943 Acked Read 9 Done Sent Done false
32483 9944 Acked Write 6 Done Sent Done false
32484 9945 Acked Read 1 Done Sent Done false
32485 9946 Acked Write 5 Done Sent Done false
32486 9947 Acked Read 1 Done Sent Done false
32487 9948 Acked Read 3 Done Sent Done false
32488 9949 Acked Write 7 Done Sent Done false
32489 9950 Acked Read 7 Done Sent Done false
32490 9951 Acked Read 5 Done Sent Done false
32491 9952 Acked Write 10 Done Sent Done false
32492 9953 Acked Read 9 Done Sent Done false
32493 9954 Acked Read 2 Done Sent Done false
32494 9955 Acked Write 6 Done Sent Done false
32495 9956 Acked Read 4 Done Sent Done false
32496 9957 Acked Write 10 Done Sent Done false
32497 9958 Acked Flush 0 Done Sent Done false
32498 9959 Acked Write 1 Done Sent Done false
32499 9960 Acked Read 5 Done Sent Done false
32500 9961 Acked Write 6 Done Sent Done false
32501 9962 Acked Flush 0 Done Sent Done false
32502 9963 Acked Read 4 Done Sent Done false
32503 9964 Acked Write 7 Done Sent Done false
32504 9965 Acked Write 4 Done Sent Done false
32505 9966 Acked Read 10 Done Sent Done false
32506 9967 Acked Read 5 Done Sent Done false
32507 9968 Acked Read 8 Done Sent Done false
32508 9969 Acked Write 6 Done Sent Done false
32509 9970 Acked Read 3 Done Sent Done false
32510 9971 Acked Read 3 Done Sent Done false
32511 9972 Acked Read 9 Done Sent Done false
32512 9973 Acked Write 10 Done Sent Done false
32513 9974 Acked Write 7 Done Sent Done false
32514 9975 Acked Read 7 Done Sent Done false
32515 9976 Acked Write 7 Done Sent Done false
32516 9977 Acked Write 7 Done Sent Done false
32517 9978 Acked Read 7 Done Sent Done false
32518 9979 Acked Read 3 Done Sent Done false
32519 9980 Acked Write 1 Done Sent Done false
32520 9981 Acked Write 6 Done Sent Done false
32521 9982 Acked Read 10 Done Sent Done false
32522 9983 Acked Read 4 Done Sent Done false
32523 9984 Acked Write 7 Done Sent Done false
32524 9985 Acked Read 1 Done Sent Done false
32525 9986 Acked Read 2 Done Sent Done false
32526 9987 Acked Read 1 Done Sent Done false
32527 9988 Acked Flush 0 Done Sent Done false
32528 9989 Acked Write 8 Done Sent Done false
32529 9990 Acked Read 5 Done Sent Done false
32530 9991 Acked Read 8 Done Sent Done false
32531 9992 Acked Read 9 Done Sent Done false
32532 9993 Acked Write 2 Done Sent Done false
32533 9994 Acked Flush 0 Done Sent Done false
32534 9995 Acked Write 5 Done Sent Done false
32535 9996 Acked Read 9 Done Sent Done false
32536 9997 Acked Write 9 Done Sent Done false
32537 9998 Acked Read 1 Done Sent Done false
32538 9999 Acked Read 2 Done Sent Done false
32539 10000 Acked Flush 0 Done Sent Done false
32540 10001 Acked Flush 0 Done Sent Done false
32541 10002 Acked Read 3 Done Sent Done false
32542 10003 Acked Write 7 Done Sent Done false
32543 10004 Acked Write 6 Done Sent Done false
32544 10005 Acked Write 6 Done Sent Done false
32545 10006 Acked Write 10 Done Sent Done false
32546 10007 Acked Flush 0 Done Sent Done false
32547 10008 Acked Write 1 Done Sent Done false
32548 10009 Acked Read 4 Done Sent Done false
32549 10010 Acked Read 8 Done Sent Done false
32550 10011 Acked Read 4 Done Sent Done false
32551 10012 Acked Flush 0 Done Sent Done false
32552 10013 Acked Write 8 Done Sent Done false
32553 10014 Acked Read 5 Done Sent Done false
32554 10015 Acked Read 3 Done Sent Done false
32555 10016 Acked Write 10 Done Sent Done false
32556 10017 Acked Read 4 Done Sent Done false
32557 10018 Acked Write 6 Done Sent Done false
32558 10019 Acked Write 4 Done Sent Done false
32559 10020 Acked Write 9 Done Sent Done false
32560 10021 Acked Read 8 Done Sent Done false
32561 10022 Acked Flush 0 Done Sent Done false
32562 10023 Acked Write 8 Done Sent Done false
32563 10024 Acked Write 7 Done Sent Done false
32564 10025 Acked Flush 0 Done Sent Done false
32565 10026 Acked Write 5 Done Sent Done false
32566 10027 Acked Read 7 Done Sent Done false
32567 10028 Acked Read 4 Done Sent Done false
32568 10029 Acked Write 7 Done Sent Done false
32569 10030 Acked Read 6 Done Sent Done false
32570 10031 Acked Read 7 Done Sent Done false
32571 10032 Acked Read 5 Done Sent Done false
32572 10033 Acked Write 3 Done Sent Done false
32573 10034 Acked Flush 0 Done Sent Done false
32574 10035 Acked Write 1 Done Sent Done false
32575 10036 Acked Read 4 Done Sent Done false
32576 10037 Acked Write 4 Done Sent Done false
32577 10038 Acked Read 5 Done Sent Done false
32578 10039 Acked Read 2 Done Sent Done false
32579 10040 Acked Read 6 Done Sent Done false
32580 10041 Acked Flush 0 Done Sent Done false
32581 10042 Acked Write 2 Done Sent Done false
32582 10043 Acked Read 4 Done Sent Done false
32583 10044 Acked Write 8 Done Sent Done false
32584 10045 Acked Read 6 Done Sent Done false
32585 10046 Acked Flush 0 Done Sent Done false
32586 10047 Acked Read 9 Done Sent Done false
32587 10048 Acked Write 6 Done Sent Done false
32588 10049 Acked Write 6 Done Sent Done false
32589 10050 Acked Read 6 Done Sent Done false
32590 10051 Acked Read 3 Done Sent Done false
32591 10052 Acked Read 8 Done Sent Done false
32592 10053 Acked Read 4 Done Sent Done false
32593 10054 Acked Read 3 Done Sent Done false
32594 10055 Acked Write 5 Done Sent Done false
32595 10056 Acked Read 8 Done Sent Done false
32596 10057 Acked Flush 0 Done Sent Done false
32597 STATES DS:0 DS:1 DS:2 TOTAL
32598 Sent 0 302 0 302
32599 Done 302 0 302 604
32600 Skipped 0 0 0 0
32601 Error 0 0 0 0
32602 Last Flush: 10057 9755 10057
32603 Downstairs last five completed: 9755 9754 9753 9752 9751
32604 Upstairs last five completed: 10057 10056 10055 10054 10053
32605 CLIENT: Up:0 ds:302 act:2
326062025-01-11T06:12:17.808ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326072025-01-11T06:12:17.808ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38161) = io task client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326082025-01-11T06:12:17.809ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326092025-01-11T06:12:17.810ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326102025-01-11T06:12:17.810ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326112025-01-11T06:12:17.810ZINFOcrucible: send last flush ID to this DS: 9755 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326122025-01-11T06:12:17.810ZINFOcrucible: Replied this last flush ID: 9755 = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326132025-01-11T06:12:17.810ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326142025-01-11T06:12:17.810ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326152025-01-11T06:12:17.810ZINFOcrucible: [1] client re-new 302 jobs since flush 9755 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326162025-01-11T06:12:17.811ZINFOcrucible: [1] Replayed 302 jobs since flush: 9755 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
32617 ----------------------------------------------------------------
32618 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32619 Downstairs last five completed: 10057 10056 10055 10054 10053
32620 Upstairs last five completed: 10057 10056 10055 10054 10053
32621 CLIENT: Up:0 ds:0 act:3
32622 Replay: All jobs finished
32623 Main task has work to do, go find it
32624 got dsc Stop(0)
32625 stop 0
32626 [8810] Got stop action so:false kr:false
32627 Jan 11 06:12:18.069 INFO request completed, latency_us: 391, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 39f2bb59-1996-4185-b1d7-0f3c80cfe08f, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
326282025-01-11T06:12:18.070ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326292025-01-11T06:12:18.070ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326302025-01-11T06:12:18.070ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326312025-01-11T06:12:18.070ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326322025-01-11T06:12:18.070ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
326332025-01-11T06:12:18.070ZINFOcrucible: client task is exiting = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
32634 Jan 11 06:12:18.070 INFO request completed, latency_us: 178, response_code: 200, uri: /state/cid/0, method: GET, req_id: 43d6851b-a663-4a01-8785-c088403ba310, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
32635 [8810] Exited with: ExitStatus(unix_wait_status(9))
32636 [8810][0] reports Exit
32637 Jan 11 06:12:22.073 INFO request completed, latency_us: 229, response_code: 200, uri: /state/cid/0, method: GET, req_id: 51f13936-c986-4f94-8c1c-4185f0158a09, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
32638 001/300 Write block 4061 len 28672 data: 4 5 6 5 5 6 7
32639 002/300 Read block 4246 len 8192
32640 003/300 Read block 2596 len 16384
32641 004/300 Write block 1824 len 36864 data: 3 4 7 10 10 12 12 9 9
32642 005/300 Read block 2975 len 16384
32643 006/300 Read block 258 len 20480
32644 007/300 Write block 2504 len 40960 data: 2 3 3 3 3 5 6 5 7 8
32645 008/300 Write block 1489 len 4096 data: 6
32646 009/300 Read block 3718 len 20480
32647 010/300 Write block 4191 len 36864 data: 5 5 5 6 5 4 5 5 5
32648 011/300 Write block 2788 len 20480 data: 7 6 5 5 5
32649 012/300 Read block 4970 len 32768
32650 013/300 Write block 763 len 36864 data: 5 5 6 6 7 8 8 8 7
32651 014/300 Read block 4856 len 36864
32652 015/300 Write block 1508 len 24576 data: 4 3 4 6 5 4
32653 016/300 Read block 1067 len 24576
32654 017/300 Read block 1864 len 20480
32655 018/300 Read block 2625 len 40960
32656 019/300 Read block 530 len 36864
32657 020/300 Flush
32658 021/300 Read block 1620 len 8192
32659 022/300 Write block 4139 len 4096 data: 6
32660 023/300 Read block 3000 len 28672
32661 024/300 Write block 3275 len 28672 data: 7 8 9 8 7 7 5
32662 025/300 Read block 982 len 12288
32663 026/300 Flush
32664 027/300 Read block 2157 len 8192
32665 028/300 Read block 103 len 20480
32666 029/300 Write block 823 len 36864 data: 3 6 6 6 7 5 6 8 8
32667 030/300 Read block 3810 len 16384
32668 031/300 Read block 2633 len 36864
32669 032/300 Read block 4109 len 24576
32670 033/300 Read block 2206 len 24576
32671 034/300 Read block 2437 len 32768
32672 035/300 Read block 4459 len 40960
32673 036/300 Read block 2133 len 40960
32674 037/300 Read block 4019 len 16384
32675 038/300 Write block 778 len 32768 data: 4 4 5 4 5 6 6 7
32676 039/300 Read block 1119 len 40960
32677 040/300 Read block 1507 len 12288
32678 041/300 Write block 2890 len 36864 data: 4 4 4 4 5 6 7 6 7
32679 042/300 Write block 1057 len 16384 data: 4 5 6 5
32680 043/300 Flush
32681 044/300 Write block 1473 len 16384 data: 6 7 7 7
32682 045/300 Write block 303 len 4096 data: 5
32683 046/300 Read block 3170 len 16384
32684 047/300 Write block 2322 len 36864 data: 6 5 6 5 4 5 4 5 5
32685 048/300 Write block 148 len 12288 data: 4 4 3
32686 049/300 Write block 4031 len 32768 data: 7 7 9 8 7 6 7 6
32687 050/300 Write block 1895 len 16384 data: 5 6 5 5
32688 051/300 Read block 1241 len 12288
32689 052/300 Write block 1712 len 36864 data: 6 5 6 7 7 6 7 8 9
32690 053/300 Read block 3602 len 4096
32691 054/300 Write block 4343 len 40960 data: 4 4 4 4 3 3 2 2 4 5
32692 055/300 Flush
32693 056/300 Write block 3733 len 24576 data: 9 8 6 6 5 3
32694 057/300 Write block 2140 len 36864 data: 5 5 5 6 7 6 8 6 5
32695 058/300 Write block 1447 len 12288 data: 7 5 6
32696 059/300 Write block 455 len 28672 data: 2 2 2 2 2 2 2
32697 060/300 Write block 643 len 24576 data: 7 6 6 5 5 6
32698 061/300 Write block 1395 len 4096 data: 7
32699 062/300 Write block 54 len 20480 data: 9 8 9 10 10
32700 063/300 Flush
32701 064/300 Write block 1648 len 32768 data: 5 3 2 2 2 2 3 4
32702 065/300 Write block 2636 len 36864 data: 6 6 5 5 4 4 7 7 7
32703 066/300 Read block 3650 len 4096
32704 067/300 Read block 3195 len 36864
32705 068/300 Write block 4416 len 16384 data: 3 4 3 4
32706 069/300 Write block 1266 len 32768 data: 4 4 4 4 3 4 4 4
32707 070/300 Write block 3239 len 24576 data: 4 5 5 5 5 6
32708 071/300 Read block 1138 len 20480
32709 072/300 Read block 1036 len 12288
32710 073/300 Write block 2761 len 20480 data: 4 4 5 4 4
32711 074/300 Write block 1493 len 28672 data: 7 8 8 7 11 9 8
32712 075/300 Read block 483 len 12288
32713 076/300 Write block 2993 len 16384 data: 4 3 3 2
32714 077/300 Read block 1179 len 36864
32715 078/300 Read block 1278 len 28672
32716 079/300 Write block 4418 len 16384 data: 4 5 3 3
32717 080/300 Write block 1646 len 32768 data: 6 6 6 4 3 3 3 3
32718 081/300 Read block 4765 len 12288
32719 082/300 Read block 664 len 24576
32720 083/300 Flush
32721 084/300 Write block 4447 len 28672 data: 6 9 9 9 10 10 11
32722 085/300 Read block 4459 len 24576
32723 086/300 Read block 2495 len 16384
32724 087/300 Write block 2980 len 36864 data: 6 6 5 5 3 4 4 4 3
32725 088/300 Write block 1131 len 8192 data: 5 5
32726 089/300 Read block 1938 len 16384
32727 090/300 Read block 1072 len 32768
32728 091/300 Read block 1392 len 8192
32729 092/300 Write block 3755 len 12288 data: 6 7 6
32730 093/300 Flush
32731 094/300 Write block 318 len 40960 data: 8 6 7 9 9 11 12 12 10 8
32732 095/300 Write block 1715 len 8192 data: 8 8
32733 096/300 Read block 2159 len 40960
32734 097/300 Read block 385 len 4096
32735 098/300 Flush
32736 099/300 Read block 1092 len 40960
32737 100/300 Read block 771 len 12288
32738 101/300 Read block 1435 len 12288
32739 102/300 Read block 3518 len 28672
32740 103/300 Read block 3254 len 32768
32741 104/300 Read block 4388 len 40960
32742 105/300 Read block 1867 len 4096
32743 106/300 Write block 3311 len 32768 data: 4 5 3 3 4 5 6 5
32744 107/300 Read block 1078 len 36864
32745 108/300 Write block 788 len 24576 data: 6 6 7 8 8 9
32746 109/300 Write block 4485 len 8192 data: 5 5
32747 110/300 Write block 576 len 12288 data: 5 6 7
32748 111/300 Write block 3005 len 40960 data: 8 10 10 10 9 6 6 5 6 8
32749 112/300 Write block 1775 len 8192 data: 7 7
32750 113/300 Read block 683 len 20480
32751 114/300 Read block 3555 len 8192
32752 115/300 Write block 3647 len 36864 data: 4 4 4 4 6 4 4 4 5
32753 116/300 Write block 3551 len 4096 data: 4
32754 117/300 Read block 1317 len 16384
32755 118/300 Read block 742 len 4096
32756 119/300 Read block 4006 len 16384
32757 120/300 Read block 1172 len 32768
32758 121/300 Read block 579 len 32768
32759 122/300 Flush
32760 123/300 Write block 3212 len 40960 data: 5 4 3 3 4 6 9 8 8 9
32761 124/300 Read block 4015 len 20480
32762 125/300 Write block 4865 len 36864 data: 4 5 5 4 4 5 7 8 7
32763 126/300 Write block 967 len 40960 data: 5 2 3 5 8 7 7 6 6 7
32764 127/300 Write block 581 len 8192 data: 6 6
32765 128/300 Write block 3929 len 32768 data: 4 4 4 4 4 5 5 4
32766 129/300 Write block 3150 len 36864 data: 7 8 8 10 10 9 9 8 8
32767 130/300 Flush
32768 131/300 Write block 1712 len 20480 data: 7 6 7 9 9
32769 132/300 Write block 683 len 20480 data: 4 4 5 8 9
32770 133/300 Flush
32771 134/300 Write block 1885 len 24576 data: 10 9 9 6 6 6
32772 135/300 Read block 2729 len 24576
32773 136/300 Write block 1083 len 32768 data: 3 3 3 3 3 3 3 4
32774 137/300 Read block 3180 len 28672
32775 138/300 Flush
32776 139/300 Write block 3258 len 32768 data: 5 4 4 6 5 4 4 4
32777 140/300 Read block 2594 len 28672
32778 141/300 Flush
32779 142/300 Read block 742 len 36864
32780 143/300 Write block 4883 len 12288 data: 4 4 5
32781 144/300 Write block 3983 len 4096 data: 6
32782 145/300 Write block 593 len 28672 data: 6 7 9 10 8 8 8
32783 146/300 Write block 2135 len 36864 data: 5 4 4 3 3 6 6 6 7
32784 147/300 Read block 1969 len 28672
32785 148/300 Read block 4341 len 4096
32786 149/300 Write block 40 len 16384 data: 10 12 12 11
32787 150/300 Write block 1521 len 36864 data: 7 6 7 9 9 9 8 7 6
32788 151/300 Write block 4126 len 20480 data: 7 11 12 10 11
32789 152/300 Write block 722 len 40960 data: 8 9 10 8 8 8 7 7 6 5
32790 153/300 Write block 710 len 8192 data: 7 6
32791 154/300 Read block 2177 len 4096
32792 155/300 Write block 2445 len 12288 data: 7 8 8
32793 156/300 Write block 1870 len 16384 data: 4 5 7 8
32794 157/300 Flush
32795 158/300 Read block 1982 len 40960
32796 159/300 Read block 3574 len 24576
32797 160/300 Write block 304 len 32768 data: 7 8 7 7 6 7 7 7
32798 161/300 Write block 262 len 4096 data: 4
32799 162/300 Write block 3603 len 40960 data: 6 5 5 6 5 5 5 4 4 7
32800 163/300 Read block 1154 len 8192
32801 164/300 Read block 4538 len 28672
32802 165/300 Read block 2694 len 4096
32803 166/300 Write block 3162 len 8192 data: 8 7
32804 167/300 Read block 4449 len 8192
32805 168/300 Flush
32806 169/300 Flush
32807 170/300 Write block 3520 len 4096 data: 6
32808 171/300 Read block 4792 len 40960
32809 172/300 Read block 4454 len 16384
32810 173/300 Read block 4552 len 12288
32811 174/300 Write block 4914 len 20480 data: 4 4 6 7 8
32812 175/300 Flush
32813 176/300 Read block 1243 len 36864
32814 177/300 Read block 2698 len 4096
32815 178/300 Read block 1708 len 4096
32816 179/300 Read block 2101 len 24576
32817 180/300 Read block 3271 len 8192
32818 181/300 Flush
32819 182/300 Read block 372 len 8192
32820 183/300 Write block 3654 len 24576 data: 5 6 4 4 4 4
32821 184/300 Write block 4375 len 28672 data: 5 4 4 5 3 3 4
32822 185/300 Write block 3797 len 32768 data: 5 5 5 6 5 4 4 3
32823 186/300 Write block 297 len 16384 data: 3 4 5 5
32824 187/300 Read block 2930 len 20480
32825 188/300 Read block 1752 len 20480
32826 189/300 Read block 717 len 16384
32827 190/300 Write block 3994 len 20480 data: 6 6 5 5 5
32828 191/300 Read block 1587 len 12288
32829 192/300 Flush
32830 193/300 Read block 766 len 40960
32831 194/300 Write block 2062 len 16384 data: 6 5 4 4
32832 195/300 Flush
32833 196/300 Read block 2845 len 32768
32834 197/300 Write block 3762 len 16384 data: 8 7 6 4
32835 198/300 Read block 47 len 4096
32836 199/300 Read block 2981 len 4096
32837 200/300 Flush
32838 201/300 Write block 1626 len 36864 data: 8 8 7 4 6 5 5 4 4
32839 202/300 Write block 2036 len 4096 data: 3
32840 203/300 Read block 357 len 16384
32841 204/300 Write block 3944 len 12288 data: 4 3 5
32842 205/300 Write block 2673 len 36864 data: 4 4 4 4 7 6 6 5 7
32843 206/300 Write block 1658 len 32768 data: 2 2 2 2 4 5 4 4
32844 207/300 Read block 34 len 40960
32845 208/300 Write block 3563 len 16384 data: 8 7 7 5
32846 209/300 Read block 1378 len 20480
32847 210/300 Read block 4570 len 36864
32848 211/300 Read block 852 len 28672
32849 212/300 Write block 3109 len 12288 data: 5 4 4
32850 213/300 Write block 4726 len 36864 data: 2 3 2 2 2 3 4 5 5
32851 214/300 Read block 844 len 32768
32852 215/300 Read block 458 len 40960
32853 216/300 Read block 1681 len 8192
32854 217/300 Write block 394 len 12288 data: 10 8 9
32855 218/300 Read block 670 len 20480
32856 219/300 Read block 3264 len 12288
32857 220/300 Write block 4869 len 16384 data: 5 6 8 9
32858 221/300 Write block 1706 len 28672 data: 6 6 5 4 5 7 8
32859 222/300 Flush
32860 223/300 Read block 3077 len 20480
32861 224/300 Write block 1310 len 36864 data: 8 9 8 7 7 5 4 5 4
32862 225/300 Write block 723 len 36864 data: 10 11 9 9 9 8 8 7 6
32863 226/300 Flush
32864 227/300 Write block 746 len 16384 data: 6 5 3 4
32865 228/300 Flush
32866 229/300 Read block 4265 len 32768
32867 230/300 Write block 1153 len 40960 data: 4 4 3 4 3 3 3 3 3 4
32868 231/300 Read block 4322 len 24576
32869 232/300 Read block 3428 len 4096
32870 233/300 Read block 4217 len 8192
32871 234/300 Flush
32872 235/300 Write block 2486 len 16384 data: 5 5 4 5
32873 236/300 Flush
32874 237/300 Write block 470 len 28672 data: 7 5 5 5 5 4 4
32875 238/300 Write block 4646 len 16384 data: 4 5 5 7
32876 239/300 Read block 222 len 28672
32877 240/300 Write block 1575 len 40960 data: 8 5 6 7 8 7 7 8 8 6
32878 241/300 Write block 4320 len 28672 data: 7 8 7 6 5 6 6
32879 242/300 Read block 2547 len 12288
32880 243/300 Read block 1924 len 40960
32881 244/300 Write block 4384 len 8192 data: 3 3
32882 245/300 Read block 3011 len 36864
32883 246/300 Write block 2744 len 20480 data: 5 6 5 7 7
32884 247/300 Flush
32885 248/300 Write block 981 len 24576 data: 6 5 5 6 6 6
32886 249/300 Write block 840 len 32768 data: 13 10 9 9 9 9 7 6
32887 250/300 Flush
32888 251/300 Read block 1371 len 12288
32889 252/300 Read block 270 len 28672
32890 253/300 Write block 3764 len 20480 data: 7 5 3 4 3
32891 254/300 Read block 249 len 40960
32892 255/300 Write block 4593 len 32768 data: 5 5 7 7 7 7 6 5
32893 256/300 Flush
32894 257/300 Write block 1938 len 36864 data: 8 7 10 10 8 9 8 7 7
32895 258/300 Flush
32896 259/300 Write block 3431 len 20480 data: 9 10 9 9 6
32897 260/300 Flush
32898 261/300 Read block 4511 len 32768
32899 262/300 Write block 892 len 8192 data: 3 3
32900 263/300 Write block 984 len 20480 data: 7 7 7 6 6
32901 264/300 Write block 2630 len 12288 data: 9 6 7
32902 265/300 Read block 2622 len 36864
32903 266/300 Write block 304 len 20480 data: 8 9 8 8 7
32904 267/300 Write block 2920 len 24576 data: 3 5 3 4 7 6
32905 268/300 Write block 492 len 32768 data: 7 6 6 7 7 9 10 9
32906 269/300 Read block 3620 len 40960
32907 270/300 Write block 812 len 16384 data: 7 7 7 5
32908 271/300 Write block 2708 len 28672 data: 6 5 6 8 7 7 6
32909 272/300 Write block 2677 len 20480 data: 8 7 7 6 8
32910 273/300 Read block 458 len 36864
32911 274/300 Read block 1459 len 40960
32912 275/300 Read block 3979 len 12288
32913 276/300 Read block 855 len 36864
32914 277/300 Read block 342 len 28672
32915 278/300 Read block 4991 len 32768
32916 279/300 Read block 3088 len 20480
32917 280/300 Write block 4050 len 12288 data: 6 6 7
32918 281/300 Read block 1056 len 32768
32919 282/300 Write block 1796 len 24576 data: 7 8 6 6 5 4
32920 283/300 Write block 4429 len 20480 data: 5 4 5 5 5
32921 284/300 Read block 1541 len 4096
32922 285/300 Write block 563 len 20480 data: 4 7 7 6 6
32923 286/300 Write block 1137 len 8192 data: 7 6
32924 287/300 Flush
32925 288/300 Write block 2611 len 40960 data: 4 2 3 3 5 5 4 4 3 6
32926 289/300 Flush
32927 290/300 Write block 396 len 28672 data: 10 9 9 8 9 8 8
32928 291/300 Write block 1027 len 8192 data: 7 7
32929 292/300 Write block 3237 len 32768 data: 4 5 5 6 6 6 6 7
32930 293/300 Read block 2070 len 24576
32931 294/300 Write block 139 len 28672 data: 8 8 8 6 7 6 6
32932 295/300 Read block 3601 len 4096
32933 296/300 Write block 3721 len 32768 data: 5 4 5 5 4 4 6 7
32934 297/300 Write block 452 len 36864 data: 4 3 3 3 3 3 3 3 3
32935 298/300 Read block 1727 len 8192
32936 299/300 Write block 2944 len 40960 data: 5 7 8 10 8 11 11 8 7 8
32937 300/300 Read block 4230 len 36864
32938 301/300 Read block 4173 len 24576
32939 Main task has work to do, go find it
32940 got dsc Start(0)
32941 start 0
32942 Jan 11 06:12:22.308 INFO request completed, latency_us: 230, response_code: 204, uri: /start/cid/0, method: POST, req_id: b60d8604-a038-4ec6-a43d-e601406d2345, remote_addr: 127.0.0.1:59925, local_addr: 127.0.0.1:9998
32943 Replay: started 0, returned:Ok(())
32944 ----------------------------------------------------------------
32945 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32946 States: Offline Active Active
32947 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32948 10058 Acked Write 7 Sent Done Done false
32949 10059 Acked Read 2 Sent Done Done false
32950 10060 Acked Read 4 Sent Done Done false
32951 10061 Acked Write 9 Sent Done Done false
32952 10062 Acked Read 4 Sent Done Done false
32953 10063 Acked Read 5 Sent Done Done false
32954 10064 Acked Write 10 Sent Done Done false
32955 10065 Acked Write 1 Sent Done Done false
32956 10066 Acked Read 5 Sent Done Done false
32957 10067 Acked Write 9 Sent Done Done false
32958 10068 Acked Write 5 Sent Done Done false
32959 10069 Acked Read 8 Sent Done Done false
32960 10070 Acked Write 9 Sent Done Done false
32961 10071 Acked Read 9 Sent Done Done false
32962 10072 Acked Write 6 Sent Done Done false
32963 10073 Acked Read 6 Sent Done Done false
32964 10074 Acked Read 5 Sent Done Done false
32965 10075 Acked Read 10 Sent Done Done false
32966 10076 Acked Read 9 Sent Done Done false
32967 10077 Acked Flush 0 Sent Done Done false
32968 10078 Acked Read 2 Sent Done Done false
32969 10079 Acked Write 1 Sent Done Done false
32970 10080 Acked Read 7 Sent Done Done false
32971 10081 Acked Write 7 Sent Done Done false
32972 10082 Acked Read 3 Sent Done Done false
32973 10083 Acked Flush 0 Sent Done Done false
32974 10084 Acked Read 2 Sent Done Done false
32975 10085 Acked Read 5 Sent Done Done false
32976 10086 Acked Write 9 Sent Done Done false
32977 10087 Acked Read 4 Sent Done Done false
32978 10088 Acked Read 9 Sent Done Done false
32979 10089 Acked Read 6 Sent Done Done false
32980 10090 Acked Read 6 Sent Done Done false
32981 10091 Acked Read 8 Sent Done Done false
32982 10092 Acked Read 10 Sent Done Done false
32983 10093 Acked Read 10 Sent Done Done false
32984 10094 Acked Read 4 Sent Done Done false
32985 10095 Acked Write 8 Sent Done Done false
32986 10096 Acked Read 10 Sent Done Done false
32987 10097 Acked Read 3 Sent Done Done false
32988 10098 Acked Write 9 Sent Done Done false
32989 10099 Acked Write 4 Sent Done Done false
32990 10100 Acked Flush 0 Sent Done Done false
32991 10101 Acked Write 4 Sent Done Done false
32992 10102 Acked Write 1 Sent Done Done false
32993 10103 Acked Read 4 Sent Done Done false
32994 10104 Acked Write 9 Sent Done Done false
32995 10105 Acked Write 3 Sent Done Done false
32996 10106 Acked Write 8 Sent Done Done false
32997 10107 Acked Write 4 Sent Done Done false
32998 10108 Acked Read 3 Sent Done Done false
32999 10109 Acked Write 9 Sent Done Done false
33000 10110 Acked Read 1 Sent Done Done false
33001 10111 Acked Write 10 Sent Done Done false
33002 10112 Acked Flush 0 Sent Done Done false
33003 10113 Acked Write 6 Sent Done Done false
33004 10114 Acked Write 9 Sent Done Done false
33005 10115 Acked Write 3 Sent Done Done false
33006 10116 Acked Write 7 Sent Done Done false
33007 10117 Acked Write 6 Sent Done Done false
33008 10118 Acked Write 1 Sent Done Done false
33009 10119 Acked Write 5 Sent Done Done false
33010 10120 Acked Flush 0 Sent Done Done false
33011 10121 Acked Write 8 Sent Done Done false
33012 10122 Acked Write 9 Sent Done Done false
33013 10123 Acked Read 1 Sent Done Done false
33014 10124 Acked Read 9 Sent Done Done false
33015 10125 Acked Write 4 Sent Done Done false
33016 10126 Acked Write 8 Sent Done Done false
33017 10127 Acked Write 6 Sent Done Done false
33018 10128 Acked Read 5 Sent Done Done false
33019 10129 Acked Read 3 Sent Done Done false
33020 10130 Acked Write 5 Sent Done Done false
33021 10131 Acked Write 7 Sent Done Done false
33022 10132 Acked Read 3 Sent Done Done false
33023 10133 Acked Write 4 Sent Done Done false
33024 10134 Acked Read 9 Sent Done Done false
33025 10135 Acked Read 7 Sent Done Done false
33026 10136 Acked Write 4 Sent Done Done false
33027 10137 Acked Write 8 Sent Done Done false
33028 10138 Acked Read 3 Sent Done Done false
33029 10139 Acked Read 6 Sent Done Done false
33030 10140 Acked Flush 0 Sent Done Done false
33031 10141 Acked Write 7 Sent Done Done false
33032 10142 Acked Read 6 Sent Done Done false
33033 10143 Acked Read 4 Sent Done Done false
33034 10144 Acked Write 9 Sent Done Done false
33035 10145 Acked Write 2 Sent Done Done false
33036 10146 Acked Read 4 Sent Done Done false
33037 10147 Acked Read 8 Sent Done Done false
33038 10148 Acked Read 2 Sent Done Done false
33039 10149 Acked Write 3 Sent Done Done false
33040 10150 Acked Flush 0 Sent Done Done false
33041 10151 Acked Write 10 Sent Done Done false
33042 10152 Acked Write 2 Sent Done Done false
33043 10153 Acked Read 10 Sent Done Done false
33044 10154 Acked Read 1 Sent Done Done false
33045 10155 Acked Flush 0 Sent Done Done false
33046 10156 Acked Read 10 Sent Done Done false
33047 10157 Acked Read 3 Sent Done Done false
33048 10158 Acked Read 3 Sent Done Done false
33049 10159 Acked Read 7 Sent Done Done false
33050 10160 Acked Read 8 Sent Done Done false
33051 10161 Acked Read 10 Sent Done Done false
33052 10162 Acked Read 1 Sent Done Done false
33053 10163 Acked Write 8 Sent Done Done false
33054 10164 Acked Read 9 Sent Done Done false
33055 10165 Acked Write 6 Sent Done Done false
33056 10166 Acked Write 2 Sent Done Done false
33057 10167 Acked Write 3 Sent Done Done false
33058 10168 Acked Write 10 Sent Done Done false
33059 10169 Acked Write 2 Sent Done Done false
33060 10170 Acked Read 5 Sent Done Done false
33061 10171 Acked Read 2 Sent Done Done false
33062 10172 Acked Write 9 Sent Done Done false
33063 10173 Acked Write 1 Sent Done Done false
33064 10174 Acked Read 4 Sent Done Done false
33065 10175 Acked Read 1 Sent Done Done false
33066 10176 Acked Read 4 Sent Done Done false
33067 10177 Acked Read 8 Sent Done Done false
33068 10178 Acked Read 8 Sent Done Done false
33069 10179 Acked Flush 0 Sent Done Done false
33070 10180 Acked Write 10 Sent Done Done false
33071 10181 Acked Read 5 Sent Done Done false
33072 10182 Acked Write 9 Sent Done Done false
33073 10183 Acked Write 10 Sent Done Done false
33074 10184 Acked Write 2 Sent Done Done false
33075 10185 Acked Write 8 Sent Done Done false
33076 10186 Acked Write 9 Sent Done Done false
33077 10187 Acked Flush 0 Sent Done Done false
33078 10188 Acked Write 5 Sent Done Done false
33079 10189 Acked Write 5 Sent Done Done false
33080 10190 Acked Flush 0 Sent Done Done false
33081 10191 Acked Write 6 Sent Done Done false
33082 10192 Acked Read 6 Sent Done Done false
33083 10193 Acked Write 8 Sent Done Done false
33084 10194 Acked Read 7 Sent Done Done false
33085 10195 Acked Flush 0 Sent Done Done false
33086 10196 Acked Write 8 Sent Done Done false
33087 10197 Acked Read 7 Sent Done Done false
33088 10198 Acked Flush 0 Sent Done Done false
33089 10199 Acked Read 9 Sent Done Done false
33090 10200 Acked Write 3 Sent Done Done false
33091 10201 Acked Write 1 Sent Done Done false
33092 10202 Acked Write 7 Sent Done Done false
33093 10203 Acked Write 9 Sent Done Done false
33094 10204 Acked Read 7 Sent Done Done false
33095 10205 Acked Read 1 Sent Done Done false
33096 10206 Acked Write 4 Sent Done Done false
33097 10207 Acked Write 9 Sent Done Done false
33098 10208 Acked Write 5 Sent Done Done false
33099 10209 Acked Write 10 Sent Done Done false
33100 10210 Acked Write 2 Sent Done Done false
33101 10211 Acked Read 1 Sent Done Done false
33102 10212 Acked Write 3 Sent Done Done false
33103 10213 Acked Write 4 Sent Done Done false
33104 10214 Acked Flush 0 Sent Done Done false
33105 10215 Acked Read 10 Sent Done Done false
33106 10216 Acked Read 6 Sent Done Done false
33107 10217 Acked Write 8 Sent Done Done false
33108 10218 Acked Write 1 Sent Done Done false
33109 10219 Acked Write 10 Sent Done Done false
33110 10220 Acked Read 2 Sent Done Done false
33111 10221 Acked Read 7 Sent Done Done false
33112 10222 Acked Read 1 Sent Done Done false
33113 10223 Acked Write 2 Sent Done Done false
33114 10224 Acked Read 2 Sent Done Done false
33115 10225 Acked Flush 0 Sent Done Done false
33116 10226 Acked Flush 0 Sent Done Done false
33117 10227 Acked Write 1 Sent Done Done false
33118 10228 Acked Read 10 Sent Done Done false
33119 10229 Acked Read 4 Sent Done Done false
33120 10230 Acked Read 3 Sent Done Done false
33121 10231 Acked Write 5 Sent Done Done false
33122 10232 Acked Flush 0 Sent Done Done false
33123 10233 Acked Read 9 Sent Done Done false
33124 10234 Acked Read 1 Sent Done Done false
33125 10235 Acked Read 1 Sent Done Done false
33126 10236 Acked Read 6 Sent Done Done false
33127 10237 Acked Read 2 Sent Done Done false
33128 10238 Acked Flush 0 Sent Done Done false
33129 10239 Acked Read 2 Sent Done Done false
33130 10240 Acked Write 6 Sent Done Done false
33131 10241 Acked Write 7 Sent Done Done false
33132 10242 Acked Write 8 Sent Done Done false
33133 10243 Acked Write 4 Sent Done Done false
33134 10244 Acked Read 5 Sent Done Done false
33135 10245 Acked Read 5 Sent Done Done false
33136 10246 Acked Read 4 Sent Done Done false
33137 10247 Acked Write 5 Sent Done Done false
33138 10248 Acked Read 3 Sent Done Done false
33139 10249 Acked Flush 0 Sent Done Done false
33140 10250 Acked Read 10 Sent Done Done false
33141 10251 Acked Write 4 Sent Done Done false
33142 10252 Acked Flush 0 Sent Done Done false
33143 10253 Acked Read 8 Sent Done Done false
33144 10254 Acked Write 4 Sent Done Done false
33145 10255 Acked Read 1 Sent Done Done false
33146 10256 Acked Read 1 Sent Done Done false
33147 10257 Acked Flush 0 Sent Done Done false
33148 10258 Acked Write 9 Sent Done Done false
33149 10259 Acked Write 1 Sent Done Done false
33150 10260 Acked Read 4 Sent Done Done false
33151 10261 Acked Write 3 Sent Done Done false
33152 10262 Acked Write 9 Sent Done Done false
33153 10263 Acked Write 8 Sent Done Done false
33154 10264 Acked Read 10 Sent Done Done false
33155 10265 Acked Write 4 Sent Done Done false
33156 10266 Acked Read 5 Sent Done Done false
33157 10267 Acked Read 9 Sent Done Done false
33158 10268 Acked Read 7 Sent Done Done false
33159 10269 Acked Write 3 Sent Done Done false
33160 10270 Acked Write 9 Sent Done Done false
33161 10271 Acked Read 8 Sent Done Done false
33162 10272 Acked Read 10 Sent Done Done false
33163 10273 Acked Read 2 Sent Done Done false
33164 10274 Acked Write 3 Sent Done Done false
33165 10275 Acked Read 5 Sent Done Done false
33166 10276 Acked Read 3 Sent Done Done false
33167 10277 Acked Write 4 Sent Done Done false
33168 10278 Acked Write 7 Sent Done Done false
33169 10279 Acked Flush 0 Sent Done Done false
33170 10280 Acked Read 5 Sent Done Done false
33171 10281 Acked Write 9 Sent Done Done false
33172 10282 Acked Write 9 Sent Done Done false
33173 10283 Acked Flush 0 Sent Done Done false
33174 10284 Acked Write 4 Sent Done Done false
33175 10285 Acked Flush 0 Sent Done Done false
33176 10286 Acked Read 8 Sent Done Done false
33177 10287 Acked Write 10 Sent Done Done false
33178 10288 Acked Read 6 Sent Done Done false
33179 10289 Acked Read 1 Sent Done Done false
33180 10290 Acked Read 2 Sent Done Done false
33181 10291 Acked Flush 0 Sent Done Done false
33182 10292 Acked Write 4 Sent Done Done false
33183 10293 Acked Flush 0 Sent Done Done false
33184 10294 Acked Write 7 Sent Done Done false
33185 10295 Acked Write 4 Sent Done Done false
33186 10296 Acked Read 7 Sent Done Done false
33187 10297 Acked Write 10 Sent Done Done false
33188 10298 Acked Write 7 Sent Done Done false
33189 10299 Acked Read 3 Sent Done Done false
33190 10300 Acked Read 10 Sent Done Done false
33191 10301 Acked Write 2 Sent Done Done false
33192 10302 Acked Read 9 Sent Done Done false
33193 10303 Acked Write 5 Sent Done Done false
33194 10304 Acked Flush 0 Sent Done Done false
33195 10305 Acked Write 6 Sent Done Done false
33196 10306 Acked Write 8 Sent Done Done false
33197 10307 Acked Flush 0 Sent Done Done false
33198 10308 Acked Read 3 Sent Done Done false
33199 10309 Acked Read 7 Sent Done Done false
33200 10310 Acked Write 5 Sent Done Done false
33201 10311 Acked Read 10 Sent Done Done false
33202 10312 Acked Write 8 Sent Done Done false
33203 10313 Acked Flush 0 Sent Done Done false
33204 10314 Acked Write 9 Sent Done Done false
33205 10315 Acked Flush 0 Sent Done Done false
33206 10316 Acked Write 5 Sent Done Done false
33207 10317 Acked Flush 0 Sent Done Done false
33208 10318 Acked Read 8 Sent Done Done false
33209 10319 Acked Write 2 Sent Done Done false
33210 10320 Acked Write 5 Sent Done Done false
33211 10321 Acked Write 3 Sent Done Done false
33212 10322 Acked Read 9 Sent Done Done false
33213 10323 Acked Write 5 Sent Done Done false
33214 10324 Acked Write 6 Sent Done Done false
33215 10325 Acked Write 8 Sent Done Done false
33216 10326 Acked Read 10 Sent Done Done false
33217 10327 Acked Write 4 Sent Done Done false
33218 10328 Acked Write 7 Sent Done Done false
33219 10329 Acked Write 5 Sent Done Done false
33220 10330 Acked Read 9 Sent Done Done false
33221 10331 Acked Read 10 Sent Done Done false
33222 10332 Acked Read 3 Sent Done Done false
33223 10333 Acked Read 9 Sent Done Done false
33224 10334 Acked Read 7 Sent Done Done false
33225 10335 Acked Read 8 Sent Done Done false
33226 10336 Acked Read 5 Sent Done Done false
33227 10337 Acked Write 3 Sent Done Done false
33228 10338 Acked Read 8 Sent Done Done false
33229 10339 Acked Write 6 Sent Done Done false
33230 10340 Acked Write 5 Sent Done Done false
33231 10341 Acked Read 1 Sent Done Done false
33232 10342 Acked Write 5 Sent Done Done false
33233 10343 Acked Write 2 Sent Done Done false
33234 10344 Acked Flush 0 Sent Done Done false
33235 10345 Acked Write 10 Sent Done Done false
33236 10346 Acked Flush 0 Sent Done Done false
33237 10347 Acked Write 7 Sent Done Done false
33238 10348 Acked Write 2 Sent Done Done false
33239 10349 Acked Write 8 Sent Done Done false
33240 10350 Acked Read 6 Sent Done Done false
33241 10351 Acked Write 7 Sent Done Done false
33242 10352 Acked Read 1 Sent Done Done false
33243 10353 Acked Write 8 Sent Done Done false
33244 10354 Acked Write 9 Sent Done Done false
33245 10355 Acked Read 2 Sent Done Done false
33246 10356 Acked Write 10 Sent Done Done false
33247 10357 Acked Read 9 Sent Done Done false
33248 10358 Acked Read 6 Sent Done Done false
33249 STATES DS:0 DS:1 DS:2 TOTAL
33250 Sent 301 0 0 301
33251 Done 0 301 301 602
33252 Skipped 0 0 0 0
33253 Error 0 0 0 0
33254 Last Flush: 10057 10346 10346
33255 Downstairs last five completed: 10057 10056 10055 10054 10053
33256 Upstairs last five completed: 10358 10357 10356 10355 10354
33257 CLIENT: Up:0 ds:301 act:2
33258 [8810] got start action
33259 [8810] I am going to restart
33260 Starting downstairs at port 8810
33261 Make output file at "/tmp/dsc/downstairs-8810.txt"
33262 [8810][0] reports Starting
33263 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1235)
33264 [8810][0] reports Running
33265 ----------------------------------------------------------------
33266 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33267 States: Offline Active Active
33268 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33269 10058 Acked Write 7 Sent Done Done false
33270 10059 Acked Read 2 Sent Done Done false
33271 10060 Acked Read 4 Sent Done Done false
33272 10061 Acked Write 9 Sent Done Done false
33273 10062 Acked Read 4 Sent Done Done false
33274 10063 Acked Read 5 Sent Done Done false
33275 10064 Acked Write 10 Sent Done Done false
33276 10065 Acked Write 1 Sent Done Done false
33277 10066 Acked Read 5 Sent Done Done false
33278 10067 Acked Write 9 Sent Done Done false
33279 10068 Acked Write 5 Sent Done Done false
33280 10069 Acked Read 8 Sent Done Done false
33281 10070 Acked Write 9 Sent Done Done false
33282 10071 Acked Read 9 Sent Done Done false
33283 10072 Acked Write 6 Sent Done Done false
33284 10073 Acked Read 6 Sent Done Done false
33285 10074 Acked Read 5 Sent Done Done false
33286 10075 Acked Read 10 Sent Done Done false
33287 10076 Acked Read 9 Sent Done Done false
33288 10077 Acked Flush 0 Sent Done Done false
33289 10078 Acked Read 2 Sent Done Done false
33290 10079 Acked Write 1 Sent Done Done false
33291 10080 Acked Read 7 Sent Done Done false
33292 10081 Acked Write 7 Sent Done Done false
33293 10082 Acked Read 3 Sent Done Done false
33294 10083 Acked Flush 0 Sent Done Done false
33295 10084 Acked Read 2 Sent Done Done false
33296 10085 Acked Read 5 Sent Done Done false
33297 10086 Acked Write 9 Sent Done Done false
33298 10087 Acked Read 4 Sent Done Done false
33299 10088 Acked Read 9 Sent Done Done false
33300 10089 Acked Read 6 Sent Done Done false
33301 10090 Acked Read 6 Sent Done Done false
33302 10091 Acked Read 8 Sent Done Done false
33303 10092 Acked Read 10 Sent Done Done false
33304 10093 Acked Read 10 Sent Done Done false
33305 10094 Acked Read 4 Sent Done Done false
33306 10095 Acked Write 8 Sent Done Done false
33307 10096 Acked Read 10 Sent Done Done false
33308 10097 Acked Read 3 Sent Done Done false
33309 10098 Acked Write 9 Sent Done Done false
33310 10099 Acked Write 4 Sent Done Done false
33311 10100 Acked Flush 0 Sent Done Done false
33312 10101 Acked Write 4 Sent Done Done false
33313 10102 Acked Write 1 Sent Done Done false
33314 10103 Acked Read 4 Sent Done Done false
33315 10104 Acked Write 9 Sent Done Done false
33316 10105 Acked Write 3 Sent Done Done false
33317 10106 Acked Write 8 Sent Done Done false
33318 10107 Acked Write 4 Sent Done Done false
33319 10108 Acked Read 3 Sent Done Done false
33320 10109 Acked Write 9 Sent Done Done false
33321 10110 Acked Read 1 Sent Done Done false
33322 10111 Acked Write 10 Sent Done Done false
33323 10112 Acked Flush 0 Sent Done Done false
33324 10113 Acked Write 6 Sent Done Done false
33325 10114 Acked Write 9 Sent Done Done false
33326 10115 Acked Write 3 Sent Done Done false
33327 10116 Acked Write 7 Sent Done Done false
33328 10117 Acked Write 6 Sent Done Done false
33329 10118 Acked Write 1 Sent Done Done false
33330 10119 Acked Write 5 Sent Done Done false
33331 10120 Acked Flush 0 Sent Done Done false
33332 10121 Acked Write 8 Sent Done Done false
33333 10122 Acked Write 9 Sent Done Done false
33334 10123 Acked Read 1 Sent Done Done false
33335 10124 Acked Read 9 Sent Done Done false
33336 10125 Acked Write 4 Sent Done Done false
33337 10126 Acked Write 8 Sent Done Done false
33338 10127 Acked Write 6 Sent Done Done false
33339 10128 Acked Read 5 Sent Done Done false
33340 10129 Acked Read 3 Sent Done Done false
33341 10130 Acked Write 5 Sent Done Done false
33342 10131 Acked Write 7 Sent Done Done false
33343 10132 Acked Read 3 Sent Done Done false
33344 10133 Acked Write 4 Sent Done Done false
33345 10134 Acked Read 9 Sent Done Done false
33346 10135 Acked Read 7 Sent Done Done false
33347 10136 Acked Write 4 Sent Done Done false
33348 10137 Acked Write 8 Sent Done Done false
33349 10138 Acked Read 3 Sent Done Done false
33350 10139 Acked Read 6 Sent Done Done false
33351 10140 Acked Flush 0 Sent Done Done false
33352 10141 Acked Write 7 Sent Done Done false
33353 10142 Acked Read 6 Sent Done Done false
33354 10143 Acked Read 4 Sent Done Done false
33355 10144 Acked Write 9 Sent Done Done false
33356 10145 Acked Write 2 Sent Done Done false
33357 10146 Acked Read 4 Sent Done Done false
33358 10147 Acked Read 8 Sent Done Done false
33359 10148 Acked Read 2 Sent Done Done false
33360 10149 Acked Write 3 Sent Done Done false
33361 10150 Acked Flush 0 Sent Done Done false
33362 10151 Acked Write 10 Sent Done Done false
33363 10152 Acked Write 2 Sent Done Done false
33364 10153 Acked Read 10 Sent Done Done false
33365 10154 Acked Read 1 Sent Done Done false
33366 10155 Acked Flush 0 Sent Done Done false
33367 10156 Acked Read 10 Sent Done Done false
33368 10157 Acked Read 3 Sent Done Done false
33369 10158 Acked Read 3 Sent Done Done false
33370 10159 Acked Read 7 Sent Done Done false
33371 10160 Acked Read 8 Sent Done Done false
33372 10161 Acked Read 10 Sent Done Done false
33373 10162 Acked Read 1 Sent Done Done false
33374 10163 Acked Write 8 Sent Done Done false
33375 10164 Acked Read 9 Sent Done Done false
33376 10165 Acked Write 6 Sent Done Done false
33377 10166 Acked Write 2 Sent Done Done false
33378 10167 Acked Write 3 Sent Done Done false
33379 10168 Acked Write 10 Sent Done Done false
33380 10169 Acked Write 2 Sent Done Done false
33381 10170 Acked Read 5 Sent Done Done false
33382 10171 Acked Read 2 Sent Done Done false
33383 10172 Acked Write 9 Sent Done Done false
33384 10173 Acked Write 1 Sent Done Done false
33385 10174 Acked Read 4 Sent Done Done false
33386 10175 Acked Read 1 Sent Done Done false
33387 10176 Acked Read 4 Sent Done Done false
33388 10177 Acked Read 8 Sent Done Done false
33389 10178 Acked Read 8 Sent Done Done false
33390 10179 Acked Flush 0 Sent Done Done false
33391 10180 Acked Write 10 Sent Done Done false
33392 10181 Acked Read 5 Sent Done Done false
33393 10182 Acked Write 9 Sent Done Done false
33394 10183 Acked Write 10 Sent Done Done false
33395 10184 Acked Write 2 Sent Done Done false
33396 10185 Acked Write 8 Sent Done Done false
33397 10186 Acked Write 9 Sent Done Done false
33398 10187 Acked Flush 0 Sent Done Done false
33399 10188 Acked Write 5 Sent Done Done false
33400 10189 Acked Write 5 Sent Done Done false
33401 10190 Acked Flush 0 Sent Done Done false
33402 10191 Acked Write 6 Sent Done Done false
33403 10192 Acked Read 6 Sent Done Done false
33404 10193 Acked Write 8 Sent Done Done false
33405 10194 Acked Read 7 Sent Done Done false
33406 10195 Acked Flush 0 Sent Done Done false
33407 10196 Acked Write 8 Sent Done Done false
33408 10197 Acked Read 7 Sent Done Done false
33409 10198 Acked Flush 0 Sent Done Done false
33410 10199 Acked Read 9 Sent Done Done false
33411 10200 Acked Write 3 Sent Done Done false
33412 10201 Acked Write 1 Sent Done Done false
33413 10202 Acked Write 7 Sent Done Done false
33414 10203 Acked Write 9 Sent Done Done false
33415 10204 Acked Read 7 Sent Done Done false
33416 10205 Acked Read 1 Sent Done Done false
33417 10206 Acked Write 4 Sent Done Done false
33418 10207 Acked Write 9 Sent Done Done false
33419 10208 Acked Write 5 Sent Done Done false
33420 10209 Acked Write 10 Sent Done Done false
33421 10210 Acked Write 2 Sent Done Done false
33422 10211 Acked Read 1 Sent Done Done false
33423 10212 Acked Write 3 Sent Done Done false
33424 10213 Acked Write 4 Sent Done Done false
33425 10214 Acked Flush 0 Sent Done Done false
33426 10215 Acked Read 10 Sent Done Done false
33427 10216 Acked Read 6 Sent Done Done false
33428 10217 Acked Write 8 Sent Done Done false
33429 10218 Acked Write 1 Sent Done Done false
33430 10219 Acked Write 10 Sent Done Done false
33431 10220 Acked Read 2 Sent Done Done false
33432 10221 Acked Read 7 Sent Done Done false
33433 10222 Acked Read 1 Sent Done Done false
33434 10223 Acked Write 2 Sent Done Done false
33435 10224 Acked Read 2 Sent Done Done false
33436 10225 Acked Flush 0 Sent Done Done false
33437 10226 Acked Flush 0 Sent Done Done false
33438 10227 Acked Write 1 Sent Done Done false
33439 10228 Acked Read 10 Sent Done Done false
33440 10229 Acked Read 4 Sent Done Done false
33441 10230 Acked Read 3 Sent Done Done false
33442 10231 Acked Write 5 Sent Done Done false
33443 10232 Acked Flush 0 Sent Done Done false
33444 10233 Acked Read 9 Sent Done Done false
33445 10234 Acked Read 1 Sent Done Done false
33446 10235 Acked Read 1 Sent Done Done false
33447 10236 Acked Read 6 Sent Done Done false
33448 10237 Acked Read 2 Sent Done Done false
33449 10238 Acked Flush 0 Sent Done Done false
33450 10239 Acked Read 2 Sent Done Done false
33451 10240 Acked Write 6 Sent Done Done false
33452 10241 Acked Write 7 Sent Done Done false
33453 10242 Acked Write 8 Sent Done Done false
33454 10243 Acked Write 4 Sent Done Done false
33455 10244 Acked Read 5 Sent Done Done false
33456 10245 Acked Read 5 Sent Done Done false
33457 10246 Acked Read 4 Sent Done Done false
33458 10247 Acked Write 5 Sent Done Done false
33459 10248 Acked Read 3 Sent Done Done false
33460 10249 Acked Flush 0 Sent Done Done false
33461 10250 Acked Read 10 Sent Done Done false
33462 10251 Acked Write 4 Sent Done Done false
33463 10252 Acked Flush 0 Sent Done Done false
33464 10253 Acked Read 8 Sent Done Done false
33465 10254 Acked Write 4 Sent Done Done false
33466 10255 Acked Read 1 Sent Done Done false
33467 10256 Acked Read 1 Sent Done Done false
33468 10257 Acked Flush 0 Sent Done Done false
33469 10258 Acked Write 9 Sent Done Done false
33470 10259 Acked Write 1 Sent Done Done false
33471 10260 Acked Read 4 Sent Done Done false
33472 10261 Acked Write 3 Sent Done Done false
33473 10262 Acked Write 9 Sent Done Done false
33474 10263 Acked Write 8 Sent Done Done false
33475 10264 Acked Read 10 Sent Done Done false
33476 10265 Acked Write 4 Sent Done Done false
33477 10266 Acked Read 5 Sent Done Done false
33478 10267 Acked Read 9 Sent Done Done false
33479 10268 Acked Read 7 Sent Done Done false
33480 10269 Acked Write 3 Sent Done Done false
33481 10270 Acked Write 9 Sent Done Done false
33482 10271 Acked Read 8 Sent Done Done false
33483 10272 Acked Read 10 Sent Done Done false
33484 10273 Acked Read 2 Sent Done Done false
33485 10274 Acked Write 3 Sent Done Done false
33486 10275 Acked Read 5 Sent Done Done false
33487 10276 Acked Read 3 Sent Done Done false
33488 10277 Acked Write 4 Sent Done Done false
33489 10278 Acked Write 7 Sent Done Done false
33490 10279 Acked Flush 0 Sent Done Done false
33491 10280 Acked Read 5 Sent Done Done false
33492 10281 Acked Write 9 Sent Done Done false
33493 10282 Acked Write 9 Sent Done Done false
33494 10283 Acked Flush 0 Sent Done Done false
33495 10284 Acked Write 4 Sent Done Done false
33496 10285 Acked Flush 0 Sent Done Done false
33497 10286 Acked Read 8 Sent Done Done false
33498 10287 Acked Write 10 Sent Done Done false
33499 10288 Acked Read 6 Sent Done Done false
33500 10289 Acked Read 1 Sent Done Done false
33501 10290 Acked Read 2 Sent Done Done false
33502 10291 Acked Flush 0 Sent Done Done false
33503 10292 Acked Write 4 Sent Done Done false
33504 10293 Acked Flush 0 Sent Done Done false
33505 10294 Acked Write 7 Sent Done Done false
33506 10295 Acked Write 4 Sent Done Done false
33507 10296 Acked Read 7 Sent Done Done false
33508 10297 Acked Write 10 Sent Done Done false
33509 10298 Acked Write 7 Sent Done Done false
33510 10299 Acked Read 3 Sent Done Done false
33511 10300 Acked Read 10 Sent Done Done false
33512 10301 Acked Write 2 Sent Done Done false
33513 10302 Acked Read 9 Sent Done Done false
33514 10303 Acked Write 5 Sent Done Done false
33515 10304 Acked Flush 0 Sent Done Done false
33516 10305 Acked Write 6 Sent Done Done false
33517 10306 Acked Write 8 Sent Done Done false
33518 10307 Acked Flush 0 Sent Done Done false
33519 10308 Acked Read 3 Sent Done Done false
33520 10309 Acked Read 7 Sent Done Done false
33521 10310 Acked Write 5 Sent Done Done false
33522 10311 Acked Read 10 Sent Done Done false
33523 10312 Acked Write 8 Sent Done Done false
33524 10313 Acked Flush 0 Sent Done Done false
33525 10314 Acked Write 9 Sent Done Done false
33526 10315 Acked Flush 0 Sent Done Done false
33527 10316 Acked Write 5 Sent Done Done false
33528 10317 Acked Flush 0 Sent Done Done false
33529 10318 Acked Read 8 Sent Done Done false
33530 10319 Acked Write 2 Sent Done Done false
33531 10320 Acked Write 5 Sent Done Done false
33532 10321 Acked Write 3 Sent Done Done false
33533 10322 Acked Read 9 Sent Done Done false
33534 10323 Acked Write 5 Sent Done Done false
33535 10324 Acked Write 6 Sent Done Done false
33536 10325 Acked Write 8 Sent Done Done false
33537 10326 Acked Read 10 Sent Done Done false
33538 10327 Acked Write 4 Sent Done Done false
33539 10328 Acked Write 7 Sent Done Done false
33540 10329 Acked Write 5 Sent Done Done false
33541 10330 Acked Read 9 Sent Done Done false
33542 10331 Acked Read 10 Sent Done Done false
33543 10332 Acked Read 3 Sent Done Done false
33544 10333 Acked Read 9 Sent Done Done false
33545 10334 Acked Read 7 Sent Done Done false
33546 10335 Acked Read 8 Sent Done Done false
33547 10336 Acked Read 5 Sent Done Done false
33548 10337 Acked Write 3 Sent Done Done false
33549 10338 Acked Read 8 Sent Done Done false
33550 10339 Acked Write 6 Sent Done Done false
33551 10340 Acked Write 5 Sent Done Done false
33552 10341 Acked Read 1 Sent Done Done false
33553 10342 Acked Write 5 Sent Done Done false
33554 10343 Acked Write 2 Sent Done Done false
33555 10344 Acked Flush 0 Sent Done Done false
33556 10345 Acked Write 10 Sent Done Done false
33557 10346 Acked Flush 0 Sent Done Done false
33558 10347 Acked Write 7 Sent Done Done false
33559 10348 Acked Write 2 Sent Done Done false
33560 10349 Acked Write 8 Sent Done Done false
33561 10350 Acked Read 6 Sent Done Done false
33562 10351 Acked Write 7 Sent Done Done false
33563 10352 Acked Read 1 Sent Done Done false
33564 10353 Acked Write 8 Sent Done Done false
33565 10354 Acked Write 9 Sent Done Done false
33566 10355 Acked Read 2 Sent Done Done false
33567 10356 Acked Write 10 Sent Done Done false
33568 10357 Acked Read 9 Sent Done Done false
33569 10358 Acked Read 6 Sent Done Done false
33570 10359 Acked Flush 0 Sent Done Done false
33571 STATES DS:0 DS:1 DS:2 TOTAL
33572 Sent 302 0 0 302
33573 Done 0 302 302 604
33574 Skipped 0 0 0 0
33575 Error 0 0 0 0
33576 Last Flush: 10057 10359 10359
33577 Downstairs last five completed: 10057 10056 10055 10054 10053
33578 Upstairs last five completed: 10359 10358 10357 10356 10355
33579 CLIENT: Up:0 ds:302 act:2
335802025-01-11T06:12:28.072ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335812025-01-11T06:12:28.072ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33719) = io task client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335822025-01-11T06:12:28.073ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335832025-01-11T06:12:28.073ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335842025-01-11T06:12:28.073ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335852025-01-11T06:12:28.073ZINFOcrucible: send last flush ID to this DS: 10057 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335862025-01-11T06:12:28.074ZINFOcrucible: Replied this last flush ID: 10057 = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335872025-01-11T06:12:28.074ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335882025-01-11T06:12:28.074ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335892025-01-11T06:12:28.074ZINFOcrucible: [0] client re-new 302 jobs since flush 10057 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
335902025-01-11T06:12:28.074ZINFOcrucible: [0] Replayed 302 jobs since flush: 10057 = downstairs session_id = e3b437a4-33d5-43e8-8b38-4c1c18ad0389
33591 ----------------------------------------------------------------
33592 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33593 Downstairs last five completed: 10359 10358 10357 10356 10355
33594 Upstairs last five completed: 10359 10358 10357 10356 10355
33595 CLIENT: Up:0 ds:0 act:3
33596 Replay: All jobs finished
33597 Test replay has completed
33598 Wrote out file "/tmp/test_replay_verify.log"
33599 CLIENT: Tests done. All submitted work has been ACK'd
33600 ----------------------------------------------------------------
33601 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33602 Downstairs last five completed: 10359 10358 10357 10356 10355
33603 Upstairs last five completed: 10359 10358 10357 10356 10355
33604 CLIENT: Up:0 ds:0 act:3
33605 CLIENT: All jobs finished, all DS active.
336062025-01-11T06:12:30.317ZWARNcrucible: Guest handle has been dropped
33607 Replays:30 time: 7:00
33608 Do final verify
33609 Jan 11 06:12:30.393 INFO accepted connection, remote_addr: 127.0.0.1:61619, local_addr: 127.0.0.1:9998
33610 Jan 11 06:12:30.394 INFO request completed, latency_us: 180, response_code: 200, uri: /regioninfo, method: GET, req_id: 8331cb6a-b3af-4e58-a342-b6d403e995d2, remote_addr: 127.0.0.1:61619, local_addr: 127.0.0.1:9998
336112025-01-11T06:12:30.395ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
33612 Jan 11 06:12:30.395 INFO request completed, latency_us: 122, response_code: 200, uri: /regioncount, method: GET, req_id: d097f944-30ee-439e-8044-a35498b7b402, remote_addr: 127.0.0.1:61619, local_addr: 127.0.0.1:9998
336132025-01-11T06:12:30.395ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
33614 Jan 11 06:12:30.395 INFO request completed, latency_us: 106, response_code: 200, uri: /port/cid/0, method: GET, req_id: fd786973-4997-4c85-aaa2-b119c9a31bbd, remote_addr: 127.0.0.1:61619, local_addr: 127.0.0.1:9998
33615 Jan 11 06:12:30.396 INFO request completed, latency_us: 90, response_code: 200, uri: /port/cid/1, method: GET, req_id: bb1f9184-9d7a-44a0-9281-c1c30d546c01, remote_addr: 127.0.0.1:61619, local_addr: 127.0.0.1:9998
33616 Jan 11 06:12:30.396 INFO request completed, latency_us: 117, response_code: 200, uri: /port/cid/2, method: GET, req_id: af7a9a20-e9ed-44a4-8058-897b26fd1d2b, remote_addr: 127.0.0.1:61619, local_addr: 127.0.0.1:9998
336172025-01-11T06:12:30.397ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
336182025-01-11T06:12:30.397ZINFOcrucible: Upstairs starts
336192025-01-11T06:12:30.397ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "afbd0ec341edbe8e076286ae0030d2abadb28437", git_commit_timestamp: "2025-01-11T05:49:56.000000000Z", git_branch: "renovate/opentelemetry-0.x", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
336202025-01-11T06:12:30.397ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
336212025-01-11T06:12:30.397ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
336222025-01-11T06:12:30.397ZINFOcrucible: Crucible 9df28b79-f073-4e2a-8b7a-469c4f28c830 has session id: 531f9e55-801e-4d20-8c7f-d33c950c48a6 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336232025-01-11T06:12:30.398ZINFOcrucible: Upstairs opts: Upstairs UUID: 9df28b79-f073-4e2a-8b7a-469c4f28c830, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336242025-01-11T06:12:30.398ZINFOcrucible: Crucible stats registered with UUID: 9df28b79-f073-4e2a-8b7a-469c4f28c830 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336252025-01-11T06:12:30.431ZINFOcrucible: Downstairs in volume = 3 task = crutest
336262025-01-11T06:12:30.431ZINFOcrucible: The guest has requested activation with gen:3
336272025-01-11T06:12:30.431ZINFOcrucible: 9df28b79-f073-4e2a-8b7a-469c4f28c830 active request set session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336282025-01-11T06:12:30.431ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336292025-01-11T06:12:30.431ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336302025-01-11T06:12:30.431ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336312025-01-11T06:12:30.431ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336322025-01-11T06:12:30.431ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336332025-01-11T06:12:30.431ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336342025-01-11T06:12:30.431ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336352025-01-11T06:12:30.431ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336362025-01-11T06:12:30.431ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336372025-01-11T06:12:30.431ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52079) = io task client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336382025-01-11T06:12:30.431ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40505) = io task client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336392025-01-11T06:12:30.431ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46307) = io task client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336402025-01-11T06:12:30.432ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336412025-01-11T06:12:30.432ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336422025-01-11T06:12:30.432ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336432025-01-11T06:12:30.432ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336442025-01-11T06:12:30.432ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336452025-01-11T06:12:30.432ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336462025-01-11T06:12:30.433ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336472025-01-11T06:12:30.433ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336482025-01-11T06:12:30.433ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336492025-01-11T06:12:30.434ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336502025-01-11T06:12:30.434ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336512025-01-11T06:12:30.434ZINFOcrucible: Waiting for more clients to be ready session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336522025-01-11T06:12:30.434ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336532025-01-11T06:12:30.434ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336542025-01-11T06:12:30.434ZINFOcrucible: Waiting for more clients to be ready session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336552025-01-11T06:12:30.434ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336562025-01-11T06:12:30.434ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336572025-01-11T06:12:30.434ZINFOcrucible: [0]R flush_numbers[0..12]: [1014, 1034, 1019, 1034, 1034, 1032, 1011, 1024, 1032, 1032, 1034, 1032] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336582025-01-11T06:12:30.434ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336592025-01-11T06:12:30.434ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336602025-01-11T06:12:30.434ZINFOcrucible: [1]R flush_numbers[0..12]: [1014, 1034, 1019, 1034, 1034, 1032, 1011, 1024, 1032, 1032, 1034, 1032] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336612025-01-11T06:12:30.434ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336622025-01-11T06:12:30.434ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336632025-01-11T06:12:30.434ZINFOcrucible: [2]R flush_numbers[0..12]: [1014, 1034, 1019, 1034, 1034, 1032, 1011, 1024, 1032, 1032, 1034, 1032] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336642025-01-11T06:12:30.434ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336652025-01-11T06:12:30.434ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336662025-01-11T06:12:30.434ZINFOcrucible: Max found gen is 3 = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336672025-01-11T06:12:30.434ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
33668 {"Wait for a query_work_queue command to finish before sending IO
33669 msg":"Next flush: 1035","v":0,"name":"crucible","level":30,"time":"2025-01-11T06:12:30.434506609Z","hostname":"w-01JH9ZCNW59DXD5PZEYJ4H9CWG","pid":1240,"":"downstairs","":"downstairs","session_id":"531f9e55-801e-4d20-8c7f-d33c950c48a6"}
336702025-01-11T06:12:30.434ZINFOcrucible: All extents match = downstairs session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336712025-01-11T06:12:30.434ZINFOcrucible: No downstairs reconciliation required session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336722025-01-11T06:12:30.434ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336732025-01-11T06:12:30.434ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336742025-01-11T06:12:30.434ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336752025-01-11T06:12:30.434ZINFOcrucible: All required reconciliation work is completed session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336762025-01-11T06:12:30.434ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336772025-01-11T06:12:30.434ZINFOcrucible: 9df28b79-f073-4e2a-8b7a-469c4f28c830 is now active with session: 531f9e55-801e-4d20-8c7f-d33c950c48a6 session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336782025-01-11T06:12:30.434ZINFOcrucible: Set Active after no reconciliation session_id = 531f9e55-801e-4d20-8c7f-d33c950c48a6
336792025-01-11T06:12:30.434ZINFOcrucible: The guest has finished waiting for activation with:3
33680 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
33681 Loading write count information from file "/tmp/test_replay_verify.log"
33682 Read and Verify all blocks (0..5000 range:false)
33683 Wrote out file "/tmp/test_replay_verify.log"
33684 Verify test completed
33685 Wrote out file "/tmp/test_replay_verify.log"
33686 CLIENT: Tests done. All submitted work has been ACK'd
33687 ----------------------------------------------------------------
33688 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
33689 States: Active Active Active
33690 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33691 1000 Acked Read 256 Done Done Done false
33692 1001 Acked Read 256 Done Done Done false
33693 1002 Acked Read 256 Done Done Done false
33694 1003 Acked Read 256 Done Done Done false
33695 1004 Acked Read 256 Done Done Done false
33696 1005 Acked Read 256 Done Done Done false
33697 1006 Acked Read 256 Done Done Done false
33698 1007 Acked Read 256 Done Done Done false
33699 1008 Acked Read 256 Done Done Done false
33700 1009 Acked Read 256 Done Done Done false
33701 1010 Acked Read 256 Done Done Done false
33702 1011 Acked Read 256 Done Done Done false
33703 1012 Acked Read 256 Done Done Done false
33704 1013 Acked Read 256 Done Done Done false
33705 1014 Acked Read 256 Done Done Done false
33706 1015 Acked Read 256 Done Done Done false
33707 1016 Acked Read 256 Done Done Done false
33708 1017 Acked Read 256 Done Done Done false
33709 1018 Acked Read 136 Done Done Done false
33710 1019 Acked Read 256 Done Done Done false
33711 STATES DS:0 DS:1 DS:2 TOTAL
33712 Sent 0 0 0 0
33713 Done 20 20 20 60
33714 Skipped 0 0 0 0
33715 Error 0 0 0 0
33716 Last Flush: 0 0 0
33717 Downstairs last five completed:
33718 Upstairs last five completed: 1019 1018 1017 1016 1015
33719 CLIENT: Up:0 ds:20 act:3
33720 ----------------------------------------------------------------
33721 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
33722 Downstairs last five completed: 1020 1019 1018 1017 1016
33723 Upstairs last five completed: 1020 1019 1018 1017 1016
33724 CLIENT: Up:0 ds:0 act:3
33725 CLIENT: All crucible jobs finished, exiting program
337262025-01-11T06:12:34.847ZWARNcrucible: Guest handle has been dropped
33727 Jan 11 06:12:34.914 INFO accepted connection, remote_addr: 127.0.0.1:54229, local_addr: 127.0.0.1:9998
33728 Main task has work to do, go find it
33729 got dsc Shutdown
33730 Shutdown
33731 Jan 11 06:12:34.915Shut it down
33732 INFO request completed, latency_us: 156[8810] Disable keep_running
33733 , response_code: 204[8810] Got stop action so:false kr:false
33734 , uri: /shutdown, method: POST, req_id: 89e853c6-d4d3-441b-8be8-b855b6ab4bf5, remote_addr: 127.0.0.1:54229, local_addr: 127.0.0.1:9998
33735 [8830] Disable keep_running
33736 [8830] Got stop action so:false kr:false
33737 [8820] Disable keep_running
33738 [8820] Got stop action so:false kr:false
33739 [8820] Exited with: ExitStatus(unix_wait_status(9))
33740 [8820][1] reports Exit
33741 [8810] Exited with: ExitStatus(unix_wait_status(9))
33742 [8810][0] reports Exit
33743 [8830] Exited with: ExitStatus(unix_wait_status(9))
33744 [8830][2] reports Exit
33745 January 11, 2025 at 06:12:39 AM UTC Test ends with 0